request routing-engine login node <0|1>    !! Branch SRX devices (Pre-11.4R1.6)
rlogin -Jk -T <node0|node1>    !! High-end and Branch SRX devices (11.4R1.6+ for Branch models) from the shell

Documentation

[ # ]

  1. Log into Sidewinder Admin
  2. High Availability -> Select the current standby (backup) firewall
  3. Set the Mode to Primary
  4. Login CLI and reboot the firewall we are setting to primary (master)
  5. When the firewall comes back up, reboot the previous master (do not set firewall as standby in sidewinder admin)

[ # ]

request security ike debug-enable local remote level
show log /var/log/kmd
request security ike debug-disable

Notes:

  • This enables logging to the KMD log without the need to commit

  • SUMMARY: This is another option for typical ike/ipsec traceoptions to selectively troubleshoot VPN issues
  • PROBLEM OR GOAL: Enabling ike/ipsec traceoptions on the system can be very CPU intensive and can contribute to performance issues. Troubleshooting can be difficult with traceoptions as multiple VPNs may appear in the traceoptions output

Documentation

[ # ]

set system services web-management limits debug-level 9
commit
run show log httpd.log | match powered

Documentation

[ # ]

show configuration groups junos-defaults applications
show groups junos-defaults

[ # ]

show chassis routing-engine

[ # ]

show system processes summary
show system processes extensive

Notes

  • Summary will provide a brief overview with the top 3 processes
  • Extensive includes all processes

[ # ]

show system snapshot media internal

[ # ]

Security platforms running JunosOS handle incoming packets as follows:

  1. The software applies stateless policing filters and CoS classification to the packet at the ingress.
  2. If the packet does not drop, the software performs a session lookup to determine whether the packet belongs to an existing session. The Junos OS matches on six elements of traffic information for this determination—source IP address, destination IP address, source port number, destination port number, protocol number, and a session token.
  3. If the packet does not match an existing session, a new session is created. This process is referred to as the first-packet path.

The software takes the following steps during first-packet-path processing:

  1. Based on the protocol used and its session layer (TCP or UDP), the software starts a session timer. For TCP sessions, the default timeout is 30 minutes. For UDP sessions, the default timeout is 1 minute. These values are the defaults, and can be modified
  2. The software applies firewall SCREEN options.
  3. If destination NAT is used, the software performs address allocation.
  4. Next, the software performs the route lookup. If a route exists for the destination prefix, the software takes the next step. Otherwise, it drops the packet.
  5. The software determines the packet’s incoming zone by the interface through which it arrives. The software also determines the packet’s outgoing zone by the forwarding lookup.
  6. Based on incoming and outgoing zones, the corresponding security policy is determined and a security policy lookup takes place. The software checks the packet against defined policies to determine how to treat the packet.
  7. If source NAT is used, the software performs address allocation.
  8. The software sets up the ALG service vector.
  9. The software creates and installs the session. Furthermore, the software caches the decisions made for the first packet into a flow table, which subsequent packets of that flow use.
  10. The packet now enters the fast-path processing.

Subsequent packets of a flow are all subject to fast-path processing. The software takes the following steps during fast-path processing:

  1. The software applies firewall SCREEN options.
  2. The software performs TCP checks.
  3. The software applies NAT.
  4. The software applies an ALG.
  5. The software applies packet forwarding features, which include the following:
    • Stateless packet filters
    • Traffic shaping by packet
    • Packet encapsulation and transmission

[ # ]

Please note that this describes the process to upgrade an HA pair at JunOS code pre-11. Newer versions of the JunOS code allow for upgrading without corrupting the policy of the peer devices.

!! Note: interface names are the physical and not logical names
!! The following assumes node0 is master and node1 is backup
01.) download package to /var/tmp on both devices
02.) Disable node1\'s interfaces by running the following on node0. Commit will replicate to node1
  set interfaces ge-8/0/0 disable        [-- Should be node1's interfaces, NOT node0's
  set interfaces ge-8/0/1 disable
  set interfaces ge-8/0/2 disable
  set interfaces ge-8/0/3 disable
  set interfaces ge-8/0/4 disable
  set interfaces ge-8/0/5 disable
  set interfaces ge-8/0/6 disable
  set interfaces ge-8/0/7 disable
  set interfaces ge-8/0/8 disable
03.) Disable requiring three way handshake for session on node 0 (primary)
  set security flow tcp-session no-syn-check
  set security flow tcp-session no-sequence-check
04.) Save on node 0 (primary)
  commit
05.) Disconnect the fiber link (fab# interfaces) and the control interface cables
06.) Commit on both devices
07.) Upgrade node 1 (Backup)
  request system software add /var/tmp/junos-srx1k3k-10.4R3.4-domestic.tgz no-validate no-copy
  request system reboot
08.) Perform the following on node 1 (currently backup and now newly upgraded) to verify
  show version
  show chassis cluster status
  show chassis fpc pic-status
09.) After running "show chassis fpc pic-status," wait for the slots to come online, not Present before going to step 10
10.) Node 0 then Node 1, perform ALL the following commands
  delete interfaces ge-8/0/0 disable
  delete interfaces ge-8/0/1 disable
  delete interfaces ge-8/0/2 disable
  delete interfaces ge-8/0/3 disable
  delete interfaces ge-8/0/4 disable
  delete interfaces ge-8/0/5 disable
  delete interfaces ge-8/0/6 disable
  delete interfaces ge-8/0/7 disable
  delete interfaces ge-8/0/8 disable
  set interfaces ge-0/0/0 disable
  set interfaces ge-0/0/1 disable
  set interfaces ge-0/0/2 disable
  set interfaces ge-0/0/3 disable
  set interfaces ge-0/0/4 disable
  set interfaces ge-0/0/5 disable
  set interfaces ge-0/0/6 disable
  set interfaces ge-0/0/7 disable
  set interfaces ge-0/0/8 disable
11.) Save on both devices at same time  !! IMPORTANT TO BE DONE AT THE SAME TIME !!
  commit
12.) Verify that node1 has correctly taken over as master (if input increasing on monitor command, it has taken over)
  show security flow session summary
  run monitor interface traffic
13.) On node 0:
  request system software add /var/tmp/junos-srx1k3k-10.4R3.4-domestic.tgz no-validate no-copy
  request system reboot
14.) On node 0, after upgrade:
  show version
  show chassis cluster status
  show chassis fpc pic-status
15.) Wait for all interfaces to come "online" after "show chassis fpc pic-status" command
16.) Node 1 then Node 0 (this will failover so node0 is now master again)
  delete interfaces ge-0/0/0 disable
  delete interfaces ge-0/0/1 disable
  delete interfaces ge-0/0/2 disable
  delete interfaces ge-0/0/3 disable
  delete interfaces ge-0/0/4 disable
  delete interfaces ge-0/0/5 disable
  delete interfaces ge-0/0/6 disable
  delete interfaces ge-0/0/7 disable
  delete interfaces ge-0/0/8 disable
  set interfaces ge-8/0/0 disable
  set interfaces ge-8/0/1 disable
  set interfaces ge-8/0/2 disable
  set interfaces ge-8/0/3 disable
  set interfaces ge-8/0/4 disable
  set interfaces ge-8/0/5 disable
  set interfaces ge-8/0/6 disable
  set interfaces ge-8/0/7 disable
  set interfaces ge-8/0/8 disable
17.) Save on both devices at same time
  committ
18.) Reconnect control plane cable
19.) Veryify node0 is primary
  run show chassis cluster status
20.) Reboot Node1 and connect fab# interface cables between nodes while device is rebooting
21.) Verify node0 is still passing traffic
  run monitor interface traffic
22.) Wait for all interfaces to come "online"
  show chassis fpc pic-status
23.) Verify group 2 failover shows priority
24.) Re-enable interfaces on node1 and check for proper tcp sequence checks (run on node0, commit will replicate to node1)
  delete interfaces ge-8/0/0 disable
  delete interfaces ge-8/0/1 disable
  delete interfaces ge-8/0/2 disable
  delete interfaces ge-8/0/3 disable
  delete interfaces ge-8/0/4 disable
  delete interfaces ge-8/0/5 disable
  delete interfaces ge-8/0/6 disable
  delete interfaces ge-8/0/7 disable
  delete interfaces ge-8/0/8 disable

  delete security flow tcp-session no-syn-check
  delete security flow tcp-session no-sequence-check
25.) commit
26.) Verify failover group (group 0 and 1 should show primary or secondary and priorities)
  run show chassis cluster status
    26.a) If group 2 is not showing with priorities and status on node1 is "disabled", another reboot may be necessary. This is related to the fab# interfaces
    26.b) When node1 comes back online, verify fab interfaces are showing up and give a minute or 2 for "show chassis cluster status" to show priorities and status
    26.c) May take time due to sessions being synchronized
27.) Run on node0 to download and install IDP updates if needed. Status is for verifying progress of download or install
  run request security idp security-package download full-update
  run request security idp security-package download status

  run request security idp security-package install
  run request security idp security-package install status
28.) Verify versions match on both nodes and verify they are up to date
  run show security idp security-package-versionrun show
  run request security idp security-package download check-server
    28.a) Failover may be required to download IDP if no internet access on node0 (Per Juniper) or versions do not match

Notes:

  • If the firewalls can communicate during upgrade, the policies may become corrupted

[ # ]