ASA Clustering Deployment and Troubleshooting Lab LTRSEC-2740 Goran Saradzic and Per Hagen Agenda Overview (30min) ASA Clustering Lab (3.5hrs) • Lab POD Access Tasks divide into Router and Switch-based mechanisms • ASA clustering options • Equal Cost Multipath (ECMP) 1. 2. 3. • ASA Designs in Lab • Exercise workflow – – – – Review, deploy, verify Bring down one ASA Measure convergence Bring ASA back online LT RSEC-2740 Stand-alone ASAs via OSPF L3 / Individual Mode Cluster via OSPF L3 / Individual Mode Cluster via IP SLA • Ether-Channel (ECLB) © 2015 Cisco and/or its affiliates. All rights reserved. 4. 5. L2 /Spanned Mode Cluster in Routed (OSPF to Master ) L2 / Spanned Mode Cluster in Transparent Cisco Public 3 Cisco Public 4 . All rights reserved.Achieving the Best Uptime for Your Applications Ensuring service and application availability Tolerance to failure – continuing your critical client connections Solution resiliency – know your convergence times Elastic scale and capacity – easily address your future growth Efficient management – low complexity and overhead Support for redundant locations – ability to extend to multiple sites Workload mobility with security – migrate live apps across locations Traffic normalization for NGFW and NGIPS services LT RSEC-2740 © 2015 Cisco and/or its affiliates. Realizing True Values of ASA Firewall Clustering Scale to 16 Nodes Simple Mgmt State Sharing CCL One Master One Config LT RSEC-2740 © 2015 Cisco and/or its affiliates. Cisco Public 5 High Availability . All rights reserved. All rights reserved. firewall and context modes Must configure L2 spanned mode cluster to use Transparent firewall L3 Individual mode requires Routed firewall Multiple context mode works in both types of clustering Load Balancing Firew all Modes and Features Transparent Routed Multiple Contexts Individual Interface L3 Method ECMP/ PBR N/A* ✔ ✔ Spanned Interface L2 Method Ether-Channel LB ✔ ✔ ✔ LT RSEC-2740 © 2015 Cisco and/or its affiliates. Cisco Public 6 .Deployment Options Overview of ASA cluster types. cisco. All rights reserved. to access PODs. you will log in first to add your profile information. 7 LT RSEC-2740 © 2015 Cisco and/or its affiliates.Prep Lab Portal https://labops-out. Cisco Public 7 .com/labops/ilt Using Class Name. and then log back in. All rights reserved.Prep Pick a Pod LT RSEC-2740 © 2015 Cisco and/or its affiliates. Cisco Public 8 . Prep Access your POD 9 LT RSEC-2740 © 2015 Cisco and/or its affiliates. Cisco Public 9 . All rights reserved. 30 If needed. Host. and CSR sessions are auto-opened in SuperPutty on the JumpBox RDP (see next slide) . All rights reserved.Prep Lab Portal Diagram Open RDP Session Only ASA. 10 LT RSEC-2740 Click to RDP login: Administrator password: stgscvt © 2015 Cisco and/or its affiliates. you can increase the RDP resolution size appropriate to your display. Cisco Public 10 . CSRs. Cisco Public 11 . and test hosts are open via SuperPutty shortcut. add profile. All rights reserved. log back in • JumpBox RDP session (click from portal diagram) – RDP Login: administrator/stgscvt – Full screen makes it easier • ASAs.Prep Lab Access Credentials • Access Lab Portal with your email and lab-ID. using credentials: – ASA console: enable password is cisco – CSR SSH: auto-login: admin/cisco – Linux host SSH: auto-login: user/cisco LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved.1. re-login to all by double-clicking on ASA-CSR-ENDHOSTS link within Layouts Cisco Public 12 Prep . If any session times out.Login to All Devices via SuperPutty Shortcut Once inside the Jumpbox RDP ASA1 Enable Passwd: cisco ASA2 Enable Passwd: cisco CSR1 Login: admin/cisco Inside Jumpbox.0/24 CSR2 Login: admin/cisco Inside-host Login: user/cisco Outside-host Login: user/cisco LT RSEC-2740 © 2015 Cisco and/or its affiliates.16. doubleclick on SuperPutty and you will connect to all devices through an out-of-band management network 172. 140. All rights reserved.2.44) ping 10.140.10.2.2.10.30 ssh
[email protected]) ping 172.30 Inside-host (IP 10.16.10.2./client.16.iperf ASA1 show route show conn Outside-host (IP 172.140.Auto-arranged & Auto-login terminals in SuperPutty In the Jumpbox Double Click on a Shortcut Prep Reconnect via Layouts Double-click on ASA-CSR-ENDHOSTS Inside-host (IP 10.30) .10.16.16.44) .140.44 ASA2 show route show conn LT RSEC-2740 © 2015 Cisco and/or its affiliates.iperf CSR2 show ip route terminal monitor (to view log msgs) CSR1 show ip route terminal monitor (to view log msgs) Outside-host (IP 172. Cisco Public 13 ./server.44 ssh user@172. Cisco Public 14 .2.16. All rights reserved. Reset Your Switch… Refresh the POD switch: Open browser on jumpbox PC to Home Page preset to: http://172.40/ Click on the link that says Reset to (initial state) After 1min. Confirm successful reset as shown here On this home page are links to bring down/up ASA ports LT RSEC-2740 © 2015 Cisco and/or its affiliates.Prep Open IE or Firefox Home Page inside RDP Before You Start. Stand-alone ASAs as two equal OSPF paths for CSRs 2. Cisco Public 15 . Move to L3 cluster with CSR OSPF ECMP 3. L2 cluster in Transparent mode where CSRs peer directly LT RSEC-2740 © 2015 Cisco and/or its affiliates. by removing OSPF on ASA L3 cluster One IP Path over Ether-Channel Port Bundle 4. Move to L2 cluster in Routed mode with OSPF on cluster Master 5. Switch to IP SLA.Prep Tasks 1-5 Two IP Paths 1. All rights reserved. Prep Task Workflow Example Inside Outside ASA1 CSR1 CSR2 Preview – section shows an overview of items followed by detailed slides ASA2 Deploy CLI to change into new design Tests – section gives order of setup tasks needed to complete the testing Review ASA and CSR configurations Open ping/ssh/UDP connections Verify new topology with show outputs Find which ASA owns connection Proceed to test the new design Down a path that owns test connections Check for connection state recovery Record measured convergence LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 . 1.2 Details 1 Down ASA2 2 Open Conns 3 Up ASA2 IP 1.1.2.1.1. LT RSEC-2740 Steps © 2015 Cisco and/or its affiliates.1.3 ASA2 iPerf UDP connections are stateless and will continue to work as both ASAs will create an entry in the connection table.3 IP 1. All rights reserved.2 ASA1 Inside host Outside host CSR1 CSR2 IP 1.2.Asymmetric Traffic Flow without state sharing Test Conns Success UDP PASS ping FAIL ssh FAIL Inspected or Stateful Connections traversing ASAs IP 1. Task 1 Cisco Public 17 Ping and SSH will fail now as forward and return path of traffic must come to the same ASA .1. OSPF.ASA Clustering Modes Individual Interface Mode Spanned Etherchannel Mode • Cluster members form etherchannel • Cluster members share IP. allow NSF • Each ASA has unique IP address • Adjacent routers use routing (PBR. Task 4 and 5 Cisco Public 18 . All rights reserved. ECMP) Layer 3 Adjacent Etherchannel Cluster Control Link Cluster Control Link Etherchannel Layer 3 Adjacent Task 2 and 3 LT RSEC-2740 © 2015 Cisco and/or its affiliates. Gig1 [110/12] via 1.1. 00:10:58.140.CSR1#sh ip route Task 2 & 3 (snip) O 172. Gig1 Layer3 ASA Cluster Design Router (IP routes) Load-balancing Master CCL via switch Routers Load-balance to ASAs PBR or ECMP via OSPF.0 [110/12] via 1. .1. Gig1 [110/12] via 1.2. Gig1 CSR2#sh ip route (snip) O 10.1.2.10. 00:11:08.2.0/24 [110/12] via 1.3.16.3.3 releases enabled OSPF FastHellos.1. 00:07:41.2.2. Cisco Public 19 ASA 9.1. All rights reserved. IP SLA Inside IP-A1 Tw o IP paths IP-B1 ASA1 Cluster Control Link (CCL) used for: Updating state info between ASAs Rebalancing of asymmetric traffic Outside Tw o paths CSR1 CSR2 IP-A2 IP-B2 ASA2 LT RSEC-2740 Protocol Success UDP PASS ping PASS ssh PASS Slave ASA Indiv idual Interface Mode Contexts run in Routed (IP hop) © 2015 Cisco and/or its affiliates.1. allowing faster convergence on ASA failures. 00:18:25. 140.3) Group Port-channel Protocol Ports ------+-------------+-----------+---------1 Po1(SU) LACP Gi1/0/9(P) 2 Po2(SU) LACP Gi1/0/14(P) .0/24 master/a/asa1(config)# exec clu sh port-c summ Group Port-channel Protocol Span-cluste Ports ------+------------+--------+-----------+----1 Po1(U) LACP No Gi0/2(P) slave/a/asa2(config)# sh port-channel summary 1 Po1(U) LACP No Gi0/2(P) 172.8 (.Task 2 & 3 Layer 3 ASA Cluster – Routed Firewall Individual Interface Mode (ECMP) Inside VLAN 7 master/a/asa1(config)# sh run int Po1 interface Port-channel1 lacp max-bundle 8 slave/a/asa2(config)# sh run int Po1 interface Port-channel1 lacp max-bundle 8 Po1.1 (.0/24 Po1 1.1.3) ASA2 Slave LT RSEC-2740 © 2015 Cisco and/or its affiliates.2) Master ASA1 Outside VLAN 8 Po1.8 .2.2) CCL 10.0/24 Inside Host CSR1 1.10. CCL Po1.2.16.1. Cisco Public 20 Po1.1 (.1.7 (.7 . All rights reserved.0/24 Outside Host CSR2 Po2 Lab-3750-x-switch#sh etherchannel summary Each ASA unit peers independently to neighbor routers and maintains its ow n instance of the routing table. Workf low: (1) Open test connections (2) Determine the connection owner (3) Proceed to f ail the owner ASA (4) Measure conv ergence (5) Recov er down ASA Testing Resiliency – Task 2 & 3 Individual Interface Mode (Equal Cost Multi Path) ASA1 UP G0/2 Down or ASA2 UP G0/2 ASA1 G0/3 ASA1 Down UP Down or Down Test 3: Disable ASA node via cluster CLI or dow n CCL port Po1 Inside Host Outside Host CSR1 CSR2 Po2 Test 2: Simulate ASA crash w ith ‘crashinfo force page-fault’ CCL ASA2 © 2015 Cisco and/or its affiliates. All rights reserved. UP G0/3 CCL Test 1: Dow n ASA data port on the sw itch for unit that ow ns TCP/UDP conns LT RSEC-2740 ASA2 Cisco Public 21 . flags UIOB 10.44:55505 inside 10.2.44:5001 inside TCP outside 172.2. idle 0:00:00.140.140.2.10.10.140.10.30:38842. Y flag means stub or backup conn Cisco Public Active UDP connection 22 Active TCP connection .30:38842. pick ASA with UDP conn as owner. idle 0:01:01. and proceed to test. bytes 0.10.44:55505 inside UDP outside 172.30:22. idle 0:01:01. bytes 0. 212 most used TCP outside 172.2. flags y asa2:***************************************************************** 7 in use. flags 10. bytes 883470.16.30:22. idle 0:00:00. changeto context admin cluster exec show conn asa1(LOCAL):********************************************************** 7 in use. flags –Y master/a/admin(config)# LT RSEC-2740 © 2015 Cisco and/or its affiliates.Locating Owner ASA ASA1 !master/a/admin(config)# If UDP and TCP conns are on different ASAs.16. 2 most used UDP outside 172.44:5001 inside 10.16. 17 most used Cluster stub connections: 1 in use. All rights reserved. 18 most used Cluster stub connections: 1 in use.16. bytes 3910.140. and record in your convergence table © 2015 Cisco and/or its affiliates.Measuring Convergence Count (–nan%) UDP packets that were lost. Lost Pkts/Secs ping 9 (322-330) UDP iPerf 9 (326-334) ssh N/A ASA detects that owner unit went down Count the missed PINGs LT RSEC-2740 Protocol Cisco Public 23 . All rights reserved. Cisco Public 24 The latest ASA releases enabled Non-Stop Forwarding.1. * In Transparent.0.10. .1 to network 0. Gig1 Switch(s) load-balance traf f ic to ASAs using Ether-Channel C3750-X switch is used in this lab Only the Master ASA unit peers to neighboring routers and sync the routing table to all Slave ASA units.0.Layer 2 ASA Cluster Design Switch (Ether-channel) Load-balancing Task 4 & 5 CSR2# sh ip route (snip) Gateway of last resort is 172. LT RSEC-2740 ASA Spanned Cluster Mode ASA Context can run as Routed (IP hop) or Transparent (Bridging VLANs) firewall. routers connect directly © 2015 Cisco and/or its affiliates.0 [110/12] via 1. convergence on ASA failures.140.16. 00:21:20.2.0 O 10. CCL Switch Inside IP-A1 ASA1 Outside IP-B1 CSR1 CSR2 ASA2 One IP path over Ether-Channel Interface.1. All rights reserved.2. 1 ASA1 master/a/asa1(config)# sh port-channel summary Group Port-channel Protocol Span-cluster Ports -----+------------+--------+------------+-----2 Po2(U) LACP Yes Gi0/0(P) Gi0/1(P) 10. All rights reserved.16.8 .200 .----.2.0/24 Outside VLAN 8 Po4.----.1.140.----.2.200 Po4 CCL ASA2 Slave LT RSEC-2740 © 2015 Cisco and/or its affiliates.1.-+-.0/24 Inside Host Outside Host CSR1 .----.----+ ----.0/24 Po4 1.---1 Po4(SU) LACP G i1/0/ 7(P) Gi1 /0/8( P) Gi1/0/ 12(P ) Gi1 /0/13 (P) 172.7 .Task 4 Layer 2 ASA Cluster – Routed Firewall Spanned Interface (Ether-channel) Master Inside VLAN 7 Po4.1 CCL Lab-375 0-x# sh et herch annel summ ary Group Port -chan nel Proto col Po rts ------+ ---. Cisco Public 25 CSR2 .1.10.0/24 1. Layer 2 ASA Cluster– Transparent Firewall Task 5 Spanned Interface (Ether-channel) Master Inside VLAN 7 Po4.7 BVI1 ASA1 Po4.8 BVI1 Outside VLAN 8 CCL 10.10.140.0/24 172.16.2.0/24 Po4 Inside Host Outside Host CSR1 1.1.1.200/16 1.1.2.200/16 Po4 CSR1#sh ip route ospf Gateway of last resort is 1.1.2.200 to network 0.0.0.0 O*E2 O 0.0.0.0/0 [110/1] via 1.1.2.200, 00:00:15, Gig1 172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks 172.16.2.0/24 [110/2] via 1.1.2.200, 00:00:15, Gig1 CCL ASA2 Slave LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 26 CSR2 master/a/asa1(config)# sh run interface bvi1 interface BVI1 ip address 1.1.1.1 255.255.0.0 master/a/asa1/admin# sh mac -address-table interface mac address type Age(min) bridge-group --------------------------------- -----------------------outside 0050.56bf.dbc2 dynamic 1 1 inside 0050.56bf.34b8 dynamic 5 1 Workf low: (1) Open test connections (2) Determine the connection owner (3) Proceed to f ail the owner ASA (4) Measure conv ergence (5) Recov er down ASA Testing Resiliency – Task 4 & 5 Spanned Interface Mode (Ether-channel) ASA1 UP G0/0 Down or ASA2 UP G0/0 ASA1 ASA1 Down UP G0/1 or ASA2 UP Down G0/1 Down CCL Test 1A: Dow n 1st ASA port on the sw itch for unit that ow ns TCP/UDP conns Test 1B: Dow n 2nd ASA port on Sw itch (w orst-case scenario) Po4 Inside Host Outside Host CSR1 CSR2 Po4 G0/3 ASA1 UP Test 2: Simulate ASA crash w ith ‘crashinfo force page-fault’ CCL ASA2 LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 Down or ASA2 UP Test 3: Disable ASA node via cluster CLI or dow n CCL port G0/3 Down Task 1 Task 1: Stand-alone ASAs Interna l Preview IP 1.1.1.2 Tw o paths IP 1.1.2.2 ASA1 External Tw o paths CSR1 CSR2 IP 1.1.1.3 Familiarize yourself with POD topology and configurations IP 1.1.2.3 ASA2 Tests CSR1 and CSR2 load-balancing via OSPF Down ASA2 Two paths provided by ASA1 and ASA2, stand-alone firewalls NOT in failover or cluster Attempt connections between hosts Verify OSPF routes on CSR1 to outside Bring up downed ASA2 Verify OSPF routes on CSR2 to inside Check if connections are still active Attempt connections with two ASA active LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 28 All rights reserved.30 Inside host 1.2) Outside VLAN 8 Po1.140.0/24 Master . Cisco Public 29 External .2.7 (.200 .3) Po2.0/24 172.1.8 (.7 (.200 .0/24 Inside VLAN 7 Po1.2) 10.16.1.8 (.1.3) ASA2 LT RSEC-2740 gig2 gig1 .10.1 © 2015 Cisco and/or its affiliates.44 Outside host Internal CSR1 CSR2 Po2.200 .2.Task 1 Stand-alone ASAs Diagram ASA1 1.0/24 VLAN 15 VLAN 4 gig1 gig2 . 00:01:02. GigabitEthernet1 172. 3d14h. GigabitEthernet1 [110/1] via 1.1.1.0.0.1.1.0.0 Gateway of last resort is 172.3.0. GigabitEthernet1 [110/12] via 1. 00:00:28. GigabitEthernet1 172.2.2. 00:00:28.0. 00:10:56.0/24 [110/11] via 1.2. 00:00:28.2.0/8 is variably subnetted. 2 masks O 0.1.1.1.2.0/0 [110/1] via 172.0/24 is subnetted. GigabitEthernet1 CSR2# CSR1# LT RSEC-2740 © 2015 Cisco and/or its affiliates. GigabitEthernet1 10.16.1.0/24 [110/11] via 1. GigabitEthernet1 O 1.1.2.16.1. GigabitEthernet2 . 00:10:23. 1 subnets O 172.3.0.16.16.0.3.2.3 to network 0.2.0.Verify CSR1 and CSR2 routes to two next-hop ASAs CSR2 CSR1 !CSR1 OSPF routes !CSR2 OSPF routes !CSR1# !CSR2# sh ip route ospf sh ip route ospf (snip) (snip) Gateway of last resort is 1.1.1.0/8 is variably subnetted. 00:10:23. 00:01:02. Cisco Public 10.0 [110/12] via 1.1.1. GigabitEthernet2 1.2.1.1. GigabitEthernet1 [110/11] via 1.1.1. GigabitEthernet1 [110/12] via 1.140. GigabitEthernet1 O [110/13] via 1.2.2. 3 subnets.0 O*E2 O*E2 0.0.0/16 is variably subnetted.1.1. 2 masks 172.16.0.1.0. 00:10:23.1/32 [110/2] via 172. 2 masks 1. 2 masks O Task 1 30 172. 00:10:23.0.3.0. All rights reserved. 3 subnets.1.0.16. 3d14h.0.16.3. GigabitEthernet1 [110/11] via 1.1/32 [110/13] via 1.2. 3 subnets.3.0/0 [110/1] via 1. GigabitEthernet1 O 1. 00:00:28.3.16.3.10.0/24 [110/12] via 1.2.1 to network 0. 2 subnets.0/16 is variably subnetted.0.1.2.1. 00:01:02.1.1.1.2. 16.0 Gateway of last resort is 1.255.0.255. outside C 172.255. 1:35:11.1.2.0 255.255.200. 1:35:58. inside O 10.1. mgmt C 172. outside O 172.255.0. 1:35:58. outside O 172.0 255.0 255.255.255.2.0 [110/11] via 1.200.0 is directly connected.16.0.255.1.3. All rights reserved. outside asa2/admin# asa1/admin# LT RSEC-2740 © 2015 Cisco and/or its affiliates.0.200.140.0 0.10.1.255.2. 1:35:11.0 255.10.255. 1:35:11.0 255.0.16.200.0.255.1.1.0 255.1.200.255.0.2.0 [110/1] via 1.2.2.2.0 255.0 255.0 is directly connected.200.1.0 is directly connected. 1:35:58.255.200.140. inside C 1. outside O*E2 0. inside O O*E2 0.255.1.0. 1:35:58.0 [110/11] via 1.255.16.0 C 1.0.255.1.0 is directly connected.ASA1 and ASA2 routes to CSRs Verify Task 1 ASA2 ASA1 !changeto context admin to show OSPF routes !asa2# !asa1# changeto context admin changeto context admin !asa2/admin# !asa1/admin# sh route sh route Gateway of last resort is 1.1.1.255. mgmt O 172.0 255.2.1.0 is directly connected.1.0. Cisco Public 31 .0 [110/11] via 1.2.16.255 [110/12] via 1.2. inside C 1.1.200 to network 0.255.255. outside O 172.255 [110/12] via 1.255.3. outside C 1.2.1.200.1 255.200 to network 0. 1:35:11.0.255.255.1.0 0.16.1.255.255.1. outside 10.1 255.0 255.0 is directly connected.0 [110/1] via 1.0 [110/11] via 1.2.0.1. pointing to link: http://172.40/ Disable ASA2 G0/3 port Disable ASA2 G0/2 port LT RSEC-2740 Task 1 © 2015 Cisco and/or its affiliates. use browser home page on jumpbox PC. All rights reserved.16.Remove 2 nd path b/t CSRs Remove ASA2 Path Open IE/Firefox inside RDP Shut down ASA2 data ports on Switch To shutdown ASA2 ports on the switch. Cisco Public 32 .2. 0/24 is subnetted. 00:00:28.1.3.0/24 [110/11] via 1.16. GigabitEthernet1 O 172.2.2. 3d14h.0/0 [110/1] via 172.0.2.0/8 is variably subnetted. GigabitEthernet1 10.0.0.2. 00:01:02.2. 2 masks O Task 1 1.10. 1.0/24 [110/11] via 1.2.16.16.1.1.0. 1 subnets O CSR1# LT RSEC-2740 0. 00:10:23.0. GigabitEthernet2 33 172. GigabitEthernet1 1. GigabitEthernet1 O 172.1.1/32 [110/13] via 1.0.2.16.1.0 Gateway of last resort is 172. All rights reserved.0.16.2 to network 0.0. 3d14h.1.0. 00:00:28. GigabitEthernet1 172.0/16 is variably subnetted.0/16 is variably subnetted.1. 3 subnets. 2 masks O 172. GigabitEthernet2 .2.1.0.0.2.16.0.1.1.1. 3 subnets. GigabitEthernet1 CSR2# Cisco Public 10.1.0/0 [110/1] via 1.1/32 [110/2] via 172. 2 masks O © 2015 Cisco and/or its affiliates.16.0/8 is variably subnetted.1.0 [110/12] via 1.2. 3 subnets. 2 masks 1.0.1. 00:00:28.0/24 [110/12] via 1. 00:01:02.3.0.1.0.2.1.1 to network 0. 2 subnets.16.2.0 O*E2 O*E2 0.140.2.0.1.CSR1 and CSR2 routes to one ASA Verify One path between CSRs CSR2 CSR1 !CSR1 OSPF routes !CSR2 OSPF routes !CSR1# !CSR2# sh ip route ospf sh ip route ospf (snip) (snip) Gateway of last resort is 1. 44 -b 0. All rights reserved. vi or pico #This will allow you to run UDP traffic throughout duration of the lab pico client.iperf iperf -u –t 260 -i 1 -c 172.2. to allow iPerf to send for 4 hours #You can use your favorite UNIX editor installed.0941m user@lubuntu:~$ #Change –t flag to 20000.Modify iPerf Run Time Allow iPerf to run UDP throughout duration of your lab Task 1 InsideHost #user@lubuntu:~$ cat client.16.iperf iperf -u –t 20000 -i 1 -c 172.iperf #Change to: -t 20000 #user@lubuntu:~$ cat client.2. Cisco Public 34 Change iPerf –t flag to from 260 to 20000 iperf –help (snip) -t.0941m user@lubuntu:~$ LT RSEC-2740 © 2015 Cisco and/or its affiliates.44 -b 0. --time n time in seconds to transmit for (default 10 secs) .16. 16.10.140.44) ssh
[email protected] 1 Setup Test Connections iPerf UDP packets sending from Inside to Outside Host Inside-host (IP 10. Cisco Public 35 . All rights reserved.30) .16.30 (passwd: cisco) Ping Inside to Outside and SSH Outside to Inside LT RSEC-2740 © 2015 Cisco and/or its affiliates.2.2.10.iperf Inside-host (IP 10./client.140.30) ping 172.140.iperf Outside-host (IP 172.44 Outside-host (IP 172.16./server.44) .2. 0 sec Lost/Total 0/ 8 (0%) -----------------------------------------------------------[ ### When server is not receiving packets.5 KBytes 94.0.44 (172.44: icmp_req=2 ttl=62 time=1./server. ping to outside -lnx #user@inside-lnx:~$ ping 172.00 bits/sec 0. Server listens and receives client UDP traffic #user@outside-lnx:~$ Verify if you can ping.16.0 sec 12.2.16.Setup Test Conns Cont… Ping from inside to outside linux Task 1 Start iPerf UDP flow OutsideHos t InsideHost #On top left terminal.5 KBytes 94.16.0 sec 0.083 ms 0/ 20 (0% ) 112 KByte (default) 3] local 172.067 ms 0/ 0 (-nan%) [ 3] 22.00 Bytes 0.44. UDP port 5001 [ 11.5 sec 28.9 KBytes 106 Kbits/sec [ 3] 1.2. 112 KByte (default) #on bottom left terminal.10.44 PING 172.0. .1 Kbits/sec 0.075 ms Sending 1470 byte datagrams [ 3] 1.44 port 5001 connected with 10.10.63 ms Receiving 1470 byte datagrams UDP buffer size: Verify you can receive UDP.0 sec 11.0-22.0-23. Cisco Public 36 .2.0-24.44: icmp_req=1 ttl=62 time=1.iperf ------------------------------------------------------------ 64 bytes from 172.140.2./client.iperf Transfer Bandwidth Jitter ------------------------------------------------------------ [ ID] Interval Datagrams Client connecting to 172.0 sec 0. output will show (-nan%) 3] local 10.5 KBytes 94.1. All rights reserved.16.067 ms 0/ 0 (-nan%) Bandwidth © 2015 Cisco and/or its affiliates.2.00 Bytes 0.1 Kbits/sec 0.0.44) 56(84) bytes of data.16.00 bits/sec 0.0.16. start a 4min iperf UDP connection to outside -lnx ------------------------------------------------------------ #user@inside-lnx:~$ [ .2.067 ms 0/ 0 (-nan%) [ 3] 23.1 Kbits/ sec LT RSEC-2740 ### You can count the number of seconds server could not receive packets [ 3] 21.140.1 Kbits/sec 0.0 sec 11.30 port 56904 3] 0.44 port 5001 [ ID] Interval Transfer [ 3] 0.087 ms 0/ 8 (0%) UDP buffer size: [ 3] 0.2.2. #On top right terminal.30 port 46611 connected with 172.7 KBytes 94.2.16.61 ms Server listening on UDP port 5001 64 bytes from 172.1.0.16.0 sec 0.2.00 bits/sec 0.2.00 Bytes 0. 44 user@inside-lnx:~$ LT RSEC-2740 © 2015 Cisco and/or its affiliates. open ssh connection outside to inside #If this session locks up.44 user@outside-lnx:~$ user@inside-lnx:~$ ssh -l user 10.30's password: Verify you can ssh b/t hosts user@lubuntu:~$ #You can kill it by typing ‘~.140.2.30 user@10. All rights reserved. it should drop out within 5min w/ error user@outside-lnx:~$ user@lubuntu:~$ Write failed: Broken pipe ssh -l user 10.16.140.16.2.10.10.’ w/ no single quotes (snip) #Then re-open it Last login: Tue Nov 26 14:44:35 2013 from 172.Ssh from outside to inside linux Setup Test Conn Cont… Task 1 OutsideHost OutsideHost #On bottom right terminal.30
[email protected]. Cisco Public 37 .30's password: (snip) Last login: Tue Nov 26 14:44:35 2013 from 172.140. use browser on jumpbox PC and open link: http://172. Cisco Public 38 . All rights reserved.40/ Enable ASA2 G0/2 Task 1 Enable ASA2 G0/3 Tw o paths ASA1 CSR1 iPerf UDP connections are stateless and will continue to work as both ASAs will create an entry in the connection table.2.16. Tw o paths CSR2 ASA2 Ping and SSH will fail now as forward and return path of traffic must come to the same ASA LT RSEC-2740 © 2015 Cisco and/or its affiliates.Enable 2nd path b/t CSRs Re-enable ASA2 Open IE link inside RDP This will add asymmetry of traffic through ASAs To shutdown ASA1 or ASA2 ports on the switch. 2.1 to network 0. 2 masks 172. 3 subnets.1.0/24 [110/11] via 1.0/0 [110/1] via 172.0.1.CSR1 and CSR2 routes to two ASAs Verify CSR2 CSR1 !CSR1 OSPF routes !CSR2 OSPF routes !CSR1# CSR2# sh ip route ospf sh ip route ospf (snip) (snip) Gateway of last resort is 1.2. 00:10:23. GigabitEthernet1 [110/11] via 1. 00:10:56. GigabitEthernet1 [110/1] via 1. GigabitEthernet1 172.2.1/32 [110/2] via 172.2.16. 00:00:28. 00:10:23.2. GigabitEthernet2 .1.1.1. GigabitEthernet1 O 1.0.0.3. 00:10:23.1.0/0 [110/1] via 1.0/16 is variably subnetted. GigabitEthernet2 1. GigabitEthernet1 CSR2# CSR1# LT RSEC-2740 © 2015 Cisco and/or its affiliates.3 to network 0.0.16.0 [110/12] via 1.3.0/24 [110/12] via 1.1.0/24 [110/11] via 1. 00:01:02. 2 masks O Task 1 39 172.3.2.0. 00:00:28.2.16.0/24 is subnetted.1.0.16.1.3.2.1.2.0/16 is variably subnetted.1.1.1.0.0. 3 subnets.3.1.16. 1 subnets O 172.0.1.140. Cisco Public 10.2.0.3. 2 subnets.1.2. 00:01:02. GigabitEthernet1 [110/11] via 1.1.2. GigabitEthernet1 172. All rights reserved. GigabitEthernet1 [110/12] via 1.16. 3d14h.1. GigabitEthernet1 O 1.0. 00:00:28.0 O*E2 O*E2 0. 2 masks 1.0.0.2.3. 00:00:28.1/32 [110/13] via 1.10. GigabitEthernet1 [110/12] via 1.16.3.1. GigabitEthernet1 O [110/13] via 1.0.1.1.1.0.1.2. 00:10:23.1. GigabitEthernet1 10.0/8 is variably subnetted. 3 subnets.16. 3d14h.2.0.0 Gateway of last resort is 172.0/8 is variably subnetted.1.1.1. 00:01:02. 2 masks O 0. 10.44) ssh session still working? Inside-host (IP 10.Task 1 Verify Test Connections When traffic goes through two ASA not in a cluster… Inside-host (IP 10. Cisco Public 40 .10.140.16.44) UDP traffic still being received? Task 1 Pass / Fail ping UDP iPerf ssh Outside-host (IP 172.16.2.30) Here we just send packets… Protocol Outside-host (IP 172.140.30) ping still working? …what traffic is not able to pass these stateful devices? LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved.2. cfg to ASA1 and watch it become a master Tests Enter configuration on ASA2 slave via CLI and Open connections through cluster watch it detect and sync config from master Down ASA that owns the connection using one of four failure scenarios CSR1/CSR2 are still load-balancing via OSPF Two paths provided by ASA1 and ASA2.2.2.1.1. Cisco Public 41 . now maintain state as L3/Individual cluster Check if any connections become responsive Verify OSPF routes on CSR1 to outside Measure Convergence of connections Verify OSPF routes on CSR2 to inside Bring ports back Up and enable down ASA LT RSEC-2740 © 2015 Cisco and/or its affiliates.3 CSR2 Clear both ASA1 and ASA2 configurations ASA2 Copy task2-system.1.1.2 External ASA1 IP 1.2 Interna l IP 1.3 Form individual interface mode or L3 cluster CSR1 IP 1. All rights reserved.Task 2 CCL Task 2: L3 Cluster in OSPF Preview IP 1.1.1. 16.1.7 (.1.8 .16.2.1.200 CCL VLAN 25 Outside host Internal CSR1 Po2.10 1.2.2.10 1.1.30 Inside host 2.7 .2.2-172.200 .1.0/24 1. Cisco Public 42 External .2-1.1.1 (.200 .8 (.2-1.44 © 2015 Cisco and/or its affiliates.2) Master 10. All rights reserved.1. Outside VLAN 8 G0/3 .2) Po1.1 172.Task 2 Individual Cluster Diagram ASA1 1.0/24 VLAN 15 VLAN 4 gig1 gig2 .0/24 Each ASA node has a unique IP on inside and outside VLANs.1.16.1.2.3) mgmt_pool Inside_pool Outside_pool 172.1.1 .0/24 gig2 gig1 .10 ASA2 LT RSEC-2740 .2 Slave Po2.0/24 Master Inside VLAN 7 Po1.1.1 (.10.140.2.3) CSR2 G0/3 . "admin -context admin" !If prompted.255.cfg clacp system-mac auto system-priority 1 copy /noconfirm milan/task2 -admin. after enable enabling cluster mode. issuing 0 free TLS licenses for UC-IME clusterBeginning interface-mode individual force configuration replication from Master .cfg running-config cluster-interface GigabitEthernet0/3 ip 2. Cisco Public 43 . Detected !Clear configuration on ASA1 no cluster INFO:interface-mode UC-IME is enabled. !Detected Cluster Master... !Now wait 1 min for ASA1 to become Master through election process !Cluster unit asa1 transitioned from DISABLED to MASTER !Save configuration on Master !Cluster unit asa2 from DISABLED to SLAVE Cluster unit transitioned asa2 transitioned from DISABLED to SLAVE write memory all !Save configuration on Slave write memory all LT RSEC-2740 © 2015 Cisco and/or its affiliates. (8) priority 20 *** Output from config line 68.cfg task2-admin." *** Output !Force the change to individual mode cluster interface-mode individual force copy /noconfirm milan/task2 -system. ASA2 is slave in Individual mode cluster Enable the Cluster ASA1 Task 2 ASA2 !Must enable and change to system context !In system !Feedback context from clear ASA2 cfg. "arp timeout 14400" clusterINFO: groupAdmin fw context is required to get the interfaces local-unit asa2 from config line 65. clear config all !Bring up interface for CCL WARNING: Removing all contexts in the system interface GigabitEthernet0/3 Removing context 'admin' (7). and become a Slave unit End configuration replication from Master. remove these commands console-replicate 1952 bytes copied in 5.255. sync config.2 255. Done. "no arp permit-nonconnect.2... and apply ASA2 cfg changeto system asa2/a#(config)#ena changeto system config terminal config terminal ClusterDisabled/a(cfg-cluster)# clear config allCluster Master. you MUST confirm Y for YES.cfg ClusterDisabled/a/asa1(config)# enableCryptochecksum (changed): 0e8178ab 18e3d553 aabeee98 f2192418 ! ASA2 will detect the Master. Done sh cluster interface -mode no cluster interface -mode no shut INFO: Admin context is required to get the interfaces !Define*** cluster group Output from config line 64.0 Creating context 'admin'. All rights reserved..ASA1 is master.220 secs (390 bytes/sec) health-check 3 WARNING:holdtime Skip fetching the URL disk0:/a..2. 200.0 area 0 Ser ial N o.1.2.255.0 255. Cisco Public 44 0.0 0.0 [110/11] via 1.255. outside (snip) !master /a/a sa1(c onfig )# LT RSEC-2740 © 2015 Cisco and/or its affiliates.1.200 to network 0.1.0.255 [110/12] via 1.0. outside O 172.255.0 area 0 Ver sion : 9 .3(2) Gateway of last resort is 1.4 8a4 log-adj-changes (snip) ! Other m embe rs in the clust er: !master/a/admin(config)# Uni t "a sa2" in st ate S LAVE sh route ospf ID : 1 (snip) Ver sion : 9 . All rights reserved.9 b07 O 172.0.2.2.200.255.0.0 Ser ial N o.0 [110/1] via 1.255.200. inside CCL MAC : c 464.3(2) network 1.255.Review and Verify ASA nodes in cluster and OSPF routes ASA1 Task 2 ASA1 !ASA1 i s Ma ster and A SA2 i s Sla ve !Verify OSPF relationships to CSRs from admin context !master /a/a sa1(c onfig )# changeto context admin ! sh clus ter inf Cluster fw: On !master/a/admin(config)# sh run router Int erfa ce mo de: i ndivi dual !Verify configuration Output Thi s is "asa 1" in stat e MAS TER router ospf 1 ID : 0 network 1.1. 00:23:06.16.255.2.0 255.255.0.2.1 255. 00:23:06.0 255. 1 timers throttle spf 100 200 1000 CCL MAC : 5 057. 00:23:06.1.140.0 255.2. 2 O 10.1. outside .0. 00:23:06.: F CH161 07JG9 O*E2 CCL IP : 2 .1.a 8e1.3.2.16.2.: F CH161 07JEN timers pacing lsa-group 10 CCL IP : 2 .255.1.2.200.10.1.255.1 339.2.0 [110/11] via 1. GigabitEthernet1 172. GigabitEthernet1 10. Cisco Public 10.1.3.0/0 [110/1] via 172. 00:10:23.2.2. GigabitEthernet1 [110/11] via 1. 00:10:23. GigabitEthernet1 O 1.1.3.0. 00:00:28.16.1.0 Gateway of last resort is 172.1.0/24 [110/11] via 1.3.2. 00:01:02.1.1.1.2.1.2.16.1. 3d14h.0.0. 00:10:23.1.10.0. 00:00:28. GigabitEthernet1 O [110/13] via 1. 00:00:28.1.2.2.0 [110/12] via 1.1/32 [110/13] via 1. 00:10:23.3.3 to network 0.2.2.2. 3 subnets.1.1/32 [110/2] via 172.1 to network 0.0.3.0/16 is variably subnetted.2. 2 masks 1. 1 subnets O 172.0. GigabitEthernet2 . GigabitEthernet1 172.16.1.2.3. GigabitEthernet1 [110/12] via 1.0/24 [110/11] via 1.1.0/16 is variably subnetted.1. 3 subnets.0 O*E2 O*E2 0.1.0/8 is variably subnetted. GigabitEthernet1 [110/12] via 1.0. 00:01:02.0.1. 2 subnets. GigabitEthernet1 [110/11] via 1.16.0/0 [110/1] via 1. 2 masks O Task 2 45 172. GigabitEthernet1 [110/1] via 1.1.1.0.0.2.1. All rights reserved.3.1.1.0. 00:10:56.16.16. 3 subnets.0.0/8 is variably subnetted. 2 masks O 0. 00:00:28. 3d14h.3.0.0. GigabitEthernet1 CSR2# CSR1# LT RSEC-2740 © 2015 Cisco and/or its affiliates.1.16.CSR1 and CSR2 routes to two ASAs Verify CSR2 CSR1 !CSR1 OSPF routes !CSR2 OSPF routes !CSR1# !CSR2# sh ip route ospf sh ip route ospf (snip) (snip) Gateway of last resort is 1.1. GigabitEthernet1 O 1.2.0.2.1.1.16. GigabitEthernet2 1.1.140. 2 masks 172.0/24 is subnetted. 00:01:02.0/24 [110/12] via 1.0. 140.16.10.Task 2 Setup Test Connections iPerf UDP packets sending from Inside to Outside Host Inside-host (IP 10.140.44) ssh
[email protected]) .2. Cisco Public 46 .10.2.iperf Outside-host (IP 172.30 (passwd: cisco) Ping Inside to Outside and SSH Outside to Inside LT RSEC-2740 © 2015 Cisco and/or its affiliates.10.16.30) ping 172.iperf Inside-host (IP 10./server. All rights reserved./client.16.44 Outside-host (IP 172.140.30) .2. 0.9 KBytes 106 Kbits/sec [ 3] 1.0.0 sec 12.2.1 Kbits/sec 0.iperf ping 172./client.iperf [ ID] Interval Datagrams ------------------------------------------------------------ [ 3] 0.0-23.63 ms UDP buffer size: 112 KByte (default) -----------------------------------------------------------#In second terminal.083 ms 0/ 20 (0% ) UDP buffer size: 8 (0%) ### Again.16.30 port 56904 Bandwidth © 2015 Cisco and/or its affiliates.0.5 KBytes 94.0.00 bits/sec 0.2.2.10.0 sec 11. start iperf UDP connection to OutsideHost [ user@inside-lnx:~$ Transfer Bandwidth Jitter .16.1 Kbits/ sec LT RSEC-2740 Lost/Total 112 KByte (default) -----------------------------------------------------------[ 3] local 172.00 Bytes 0.30 port 46611 connected with 172.0 sec 0.0-24. All rights reserved. when server is not receiving packets.0-22.067 ms 0/ 0 (-nan%) [ 3] 23.2.5 KBytes 94.067 ms 0/ 0 (-nan%) 3] local 10.00 bits/sec 0.1.5 sec 28.087 ms 0/ 8 (0%) Sending 1470 byte datagrams [ 3] 0.44 (172.0 sec 11.5 KBytes 94.61 ms Receiving 1470 byte datagrams 64 bytes from 172.2.140.44 ------------------------------------------------------------ PING 172.00 Bytes 0.10.16.44 port 5001 [ ID] Interval Transfer [ 3] 0.0 sec 11.16.44: icmp_req=2 ttl=62 time=1.16.2.1. UDP port 5001 [ 3] 1.2. Cisco Public 47 .2. output will show (-nan%) ### You can count the number of seconds server could not receive packets [ 3] 21./server.2.2.067 ms 0/ 0 (-nan%) [ 3] 22.1 Kbits/sec 0.0 sec 0.16.00 Bytes 0.7 KBytes 94.0.1 Kbits/sec 0.44.140. Server listening on UDP port 5001 64 bytes from 172.44: icmp_req=1 ttl=62 time=1.075 ms 0/ Client connecting to 172.16.16.2.0 sec 0.Setup Test Conns Output InsideHost Ping from inside to outside linux Task 2 Start iPerf UDP flow OutsideHost #In first terminal.44 port 5001 connected with 10. watch the ping to OutsideHost #user@outside-lnx:~$ #user@inside-lnx:~$ .44) 56(84) bytes of data.00 bits/sec 0. Cisco Public 48 Task 2 .2. All rights reserved.44 user@inside-lnx:~$ # This will serve to measure how long it takes for TCP connection to recover # Enter a single character on this session during convergence to notice when session recovers # If you enter more output on LT RSEC-2740 this session.140.30
[email protected]'s password : (cisco is the password) (snip) Last login: Tue Nov 26 14:44:35 2013 from 172.10.140. TCP backoff mechanism will © 2015 Cisco and/or its affiliates.Ssh from outside to inside linux Setup Test Conns OutsideHost !user@outside-lnx:~$ ssh -l user 10.10.16. 10.2. Y flag means stub or backup conn Cisco Public Active UDP connection 49 Active TCP connection . bytes 3910. flags y asa2:***************************************************************** 7 in use. idle 0:01:01.10.44:5001 inside TCP outside 172. and proceed to test.140.44:55505 inside 10.16.44:55505 inside UDP outside 172. 17 most used Cluster stub connections: 1 in use.30:22. idle 0:00:00. idle 0:01:01.Locate Owner ASA Locate conn owner ASA Task 2 You will next down ASA that owns most connections ASA1 !master/a/admin(config)# If UDP and TCP conns are on different ASAs. bytes 883470.140.30:22. flags UIOB 10.2. flags –Y master/a/admin(config)# LT RSEC-2740 © 2015 Cisco and/or its affiliates.2.140. 212 most used TCP outside 172.16.30:38842.30:38842. changeto context admin cluster exec sh conn asa1(LOCAL):********************************************************** 7 in use. bytes 0.10. 18 most used Cluster stub connections: 1 in use.16. All rights reserved.44:5001 inside 10. idle 0:00:00.140.16. pick ASA with UDP conn as owner. bytes 0.2.10. 2 most used UDP outside 172. flags 10. Task 2 Testing Resiliency Summary Individual Mode (ECMP) – Proceed to next slide for detailed instructions ASA1 UP G0/2 Down or ASA2 UP G0/2 ASA1 G0/3 ASA1 Down UP Down or Down Test 3: Disable ASA node via cluster CLI or dow n CCL port Po1 Inside Host Outside Host CSR1 CSR2 Po2 Test 2: Simulate ASA crash w ith ‘crashinfo force page-fault’ CCL ASA2 (1) Determine the connection owner (2) Shut down the port on owner ASA © 2015 Cisco and/or its affiliates. UP G0/3 CCL Test 1: Dow n 1st ASA port on the sw itch for unit that ow ns TCP/UDP conns LT RSEC-2740 ASA2 Cisco Public 50 . All rights reserved. 10. and 3… Inside-host (IP 10.140.140.44) ssh session still working? ssh -l user 10.2.iperf Outside-host (IP 172.44 …after locating ASA unit that owns your connections.2.140./server.16./client.30) Still sending packets… . Cisco Public 51 .2. 2.iperf Outside-host (IP 172.16.Task 2 Verify Test Connections are up Measure connection convergence of each test: 1A.44) UDP packets arriving? .30 Inside-host (IP 10.16.30) ping still working? Ping 172. 1B.10.10. All rights reserved. LT RSEC-2740 © 2015 Cisco and/or its affiliates. pointing to link: http://172.16.40/ ping UDP iPerf ssh Disable ASA G0/2 port LT RSEC-2740 Task 2 © 2015 Cisco and/or its affiliates. All rights reserved.2. use browser home page on jumpbox PC.Remove the data port on owner ASA Test 1 Protocol Task 1 Lost Pkts/Secs Observe and record if any packets were lost and if there was any impact on SSH session Open IE/Firefox inside RDP To shutdown ASA2 ports on the switch. Cisco Public 52 . Cisco Public 53 . and record in your convergence table Compare PING req counts to find lost pkt count LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved.Measure Count how many UDP packets you lost Task 2 Count how many ping packets were lost Count (–nan%) UDP packets that were lost. and become a Slave unit !Cluster unit asa2 transitioned from DISABLED to SLAVE Down ASA may retry to join after 5min on its own. Cisco Public 54 . but will only transition to SLAVE after G0/2 is enabled LT RSEC-2740 © 2015 Cisco and/or its affiliates.Recover Down ASA Up or ‘no shut’ G0/2 port on down ASA Task 2 Enable cluster config on down ASA to add it the cluster immediately Down ASA Enable ASA G0/2 port ! Re-join approriate ASA unit changeto system config terminal !Define cluster group cluster group fw enable !Wait for ASA to detect master. All rights reserved. finish sync. 44) UDP packets arriving? .16. LT RSEC-2740 © 2015 Cisco and/or its affiliates.30) Restart if needed… .2.30) ping still working? Ping 172.iperf Outside-host (IP 172.10.2. All rights reserved. and 3… Inside-host (IP 10.140.10.140.10./server.16.Task 2 Verify Test Connections are up Measure connection convergence of each test: 1A.2.44 …after locating ASA unit that owns your connections.44) ssh session still working? ssh -l user 10.140.iperf Outside-host (IP 172./client. 1B. Cisco Public 55 .16.30 Inside-host (IP 10. 2. Y flag means stub or backup conn Cisco Public Active UDP connection 56 Active TCP connection .2.30:22.Locate Owner ASA Locate conn owner ASA Task 2 You will next down ASA that owns most connections ASA1 !master/a/admin(config)# If UDP and TCP conns are on different ASAs.10.2. bytes 0.44:5001 inside 10.16. 18 most used Cluster stub connections: 1 in use. flags UIOB 10. flags y asa2:***************************************************************** 7 in use.30:22. 17 most used Cluster stub connections: 1 in use.16. idle 0:00:00.44:55505 inside 10. flags 10. 212 most used TCP outside 172.44:5001 inside TCP outside 172.30:38842.140.10. changeto context admin cluster exec sh conn asa1(LOCAL):********************************************************** 7 in use.16. All rights reserved.2. pick ASA with UDP conn as owner.30:38842.2.16. idle 0:00:00.140. flags –Y master/a/admin(config)# LT RSEC-2740 © 2015 Cisco and/or its affiliates. bytes 3910.10. bytes 883470. 2 most used UDP outside 172.140. and proceed to test.44:55505 inside UDP outside 172. idle 0:01:01.10.140. idle 0:01:01. bytes 0. detect master.Test 2 Simulate a crash on owner ASA Task 2 Observe and record if any packets were lost and if there was any impact on SSH session Owner ASA ! Write configs and simulate ASA crash write memory all crashinfo force page -fault !ASA will boot. All rights reserved. Cisco Public 57 Task 2 Lost Pkts/Secs . perform sync. and become a Slave unit !Cluster unit asa2 transitioned from DISABLED to SLAVE Simulate crash on owner ASA Protocol Crash owner ASA w/ CLI ping UDP iPerf ssh LT RSEC-2740 © 2015 Cisco and/or its affiliates. and record in your convergence table ASA detects that owner unit went down LT RSEC-2740 © 2015 Cisco and/or its affiliates.Measure Count how many UDP packets you lost Task 2 Count how many ping packets were lost Count (–nan%) UDP packets that were lost. All rights reserved. Cisco Public 58 . and becomes a slave unit 59 . All rights reserved. Cisco Public After reboot.Crashed ASA Re-joins LT RSEC-2740 © 2015 Cisco and/or its affiliates. syncs config. unit rejoins cluster Task 2 Detects master. iperf Outside-host (IP 172.2.16.30) ping still working? Ping 172.30 Inside-host (IP 10.2.2.10.30) Restart if needed .140.16. Cisco Public 60 .140. LT RSEC-2740 © 2015 Cisco and/or its affiliates.44) ssh session still working? ssh -l user 10.44) UDP packets arriving? . 2.44 …after locating ASA unit that owns your connections. All rights reserved./server.10.iperf Outside-host (IP 172.140.Task 2 Verify Test Connections are up Measure connection convergence of each test: 1A./client. and 3… Inside-host (IP 10.10. 1B.16. 2.44:5001 inside 10. idle 0:00:00.140. All rights reserved. bytes 3910.16. 18 most used Cluster stub connections: 1 in use.16.44:5001 inside TCP outside 172.Locate Owner ASA Locate conn owner ASA Task 2 You will next down ASA that owns most connections ASA1 !master/a/admin(config)# If UDP and TCP conns are on different ASAs.140.30:38842. Y flag means stub or backup conn Cisco Public Active UDP connection 61 Active TCP connection .10. and proceed to test.16. flags –Y master/a/admin(config)# LT RSEC-2740 © 2015 Cisco and/or its affiliates. pick ASA with UDP conn as owner.140. flags y asa2:***************************************************************** 7 in use. flags 10. 212 most used TCP outside 172.10.30:38842.10. bytes 0. bytes 0.44:55505 inside UDP outside 172. 17 most used Cluster stub connections: 1 in use. bytes 883470.2. 2 most used UDP outside 172.2. idle 0:00:00. idle 0:01:01. flags UIOB 10.16. idle 0:01:01.30:22.2.10.44:55505 inside 10. changeto context admin cluster exec sh conn asa1(LOCAL):********************************************************** 7 in use.140.30:22. 1.10.1.0 255.255.0 [110/1] via 1.200 to network 0.255. outside O 172.255.1. inside ospf dead-interval 3 O 172. All rights reserved.1.1 255.200. outside LT RSEC-2740 © 2015 Cisco and/or its affiliates.2. Cisco Public 0.ASA1 and ASA2 routes to CSRs Let’s try shorter dead-intervals CSR2 CSR1 !change spf dead-interval from 30sec to 3sec !change spf dead-interval from 30sec to 3sec !CSR1# !CSR1# interface GigabitEthernet1 interface GigabitEthernet1 ip ospf dead-interval 3 ip ospf dead-interval 3 ASA Master ASA Master !change spf dead-interval from 30sec to 3sec !Verify OSPF routes !master/a/asa1/admin(config)# !master/a/asa1/admin(config)# changeto context admin sh route ospf interface inside (snip) ospf dead-interval 3 Gateway of last resort is 1.0 [110/11] via 1.255.0 255.2.2.0 0.0. 00:15:19. 00:14:37. 00:15:19.2.255 [110/12] via 1.0 [110/11] via 1.2.0.200.3. 00:15:19.255.0.200.200. outside 62 Task 2 .0.140.16.1.0.16.255.0 ! O*E2 interface outside O 10.0.1. All rights reserved.Test 3 Protocol Shutdown the CCL port on owner ASA Observe and record if any packets were lost and if there was any impact on SSH session Task 3 Lost Pkts/Secs ping UDP iPerf ssh Disable ASA CCL port LT RSEC-2740 © 2015 Cisco and/or its affiliates. Task 2 Cisco Public 63 . Cisco Public 64 . All rights reserved.Measure Count how many UDP packets you lost Task 2 Count how many ping packets were lost Count (–nan%) UDP packets that were lost. and record in your convergence table Count the missed PINGs LT RSEC-2740 ASA detects that owner unit went down © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 65 . finish sync.Up the CCL port on down ASA Recover Down ASA Task 2 Enable cluster group to immediately add ASA to the cluster Down ASA ! Re-join approriate ASA unit changeto system config terminal !Define cluster group cluster group fw enable !Wait for ASA to detect master. and become a Slave unit !Cluster unit asa2 transitioned from DISABLED to SLAVE Enable ASA CCL port LT RSEC-2740 © 2015 Cisco and/or its affiliates. Task 3 CCL Task 3: L3 Cluster in IP SLA IP 1.1.1.2 Interna l IP 1.1.2.2 IP 1.1.1.3 Preview External ASA1 IP 1.1.2.3 CSR1 CSR2 Stay in L3 or individual interface mode and proceed to applying Task 3 CLI. ASA2 Remove OSPF config on ASA master only Tests Check IP SLA configs on CSRs Open test connections through cluster CSR1 and CSR2 still load-balancing but now via IP SLA tracks Down ASA that owns the connection Two paths still there with ASA1 and ASA2, still maintain state as L3/Individual cluster Verify IP SLA routes on CSR1 to outside Verify IP SLA routes on CSR2 to inside LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 66 Check when the connection state active Measure convergence Task 3 Individual Cluster Diagram ASA1 1.1.1.0/24 1.1.2.0/24 Master Inside VLAN 7 Po1.7 .1 (.2) Po1.8 .1 (.2) Master 10.10.140.0/24 Each ASA node has a unique IP on inside and outside VLANs. Outside VLAN 8 G0/3 .1 172.16.2.0/24 VLAN 15 VLAN 4 gig1 gig2 .30 Inside host 2.2.2.0/24 gig2 gig1 .200 .1 .200 .200 CCL VLAN 25 Outside host Internal CSR1 Po2.7 (.3) CSR2 G0/3 .2 Slave Po2.8 (.3) mgmt_pool Inside_pool Outside_pool 172.16.1.2-172.16.1.10 1.1.1.2-1.1.1.10 1.1.2.2-1.1.2.10 ASA2 LT RSEC-2740 .44 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 67 External CSR1 and CSR2 OSPF routes to two ASAs Verify CSR2 CSR1 !CSR1 OSPF routes !CSR2 OSPF routes !CSR1# !CSR2# sh ip route ospf sh ip route ospf (snip) (snip) Gateway of last resort is 1.1.1.3 to network 0.0.0.0 Gateway of last resort is 172.16.2.1 to network 0.0.0.0 O*E2 O*E2 0.0.0.0/0 [110/1] via 1.1.1.3, 00:00:28, GigabitEthernet1 [110/1] via 1.1.1.2, 00:10:23, GigabitEthernet1 O 1.1.2.0/24 [110/11] via 1.1.1.3, 00:00:28, GigabitEthernet1 [110/11] via 1.1.1.2, 00:10:23, GigabitEthernet1 10.0.0.0/24 is subnetted, 1 subnets O 172.16.2.0/24 [110/12] via 1.1.1.3, 00:00:28, GigabitEthernet1 172.16.0.0/16 is variably subnetted, 3 subnets, 2 masks 172.16.3.1/32 [110/13] via 1.1.1.3, 00:00:28, GigabitEthernet1 O [110/13] via 1.1.1.2, 00:10:23, GigabitEthernet1 CSR2# CSR1# LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 10.10.140.0 [110/12] via 1.1.2.3, 00:01:02, GigabitEthernet1 [110/12] via 1.1.2.2, 00:10:56, GigabitEthernet1 [110/12] via 1.1.1.2, 00:10:23, GigabitEthernet1 O 1.1.1.0/24 [110/11] via 1.1.2.3, 00:01:02, GigabitEthernet1 [110/11] via 1.1.2.2, 00:01:02, GigabitEthernet1 172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks O 0.0.0.0/0 [110/1] via 172.16.2.1, 3d14h, GigabitEthernet2 1.0.0.0/8 is variably subnetted, 3 subnets, 2 masks 1.0.0.0/8 is variably subnetted, 3 subnets, 2 masks O Task 3 68 172.16.3.1/32 [110/2] via 172.16.2.1, 3d14h, GigabitEthernet2 255.0 255.1. inside S* 0.1.1.On Master ASA remove OSPF CLI ASA Master ASA Master !ASA1 i n th is ca se is Mast er !Verify master routing relationships to host networks master/ a/as a1(co nfig) # sh clu i nf | i st ate !master/a/asa1/admin(config)# ! Thi s is "asa 1" in stat e MAS TER Uni t "a sa2" in st ate S LAVE show route Gateway of last resort is 1.0. 1.0 255.255.255.0.1.0.140.200.0.255.2.2.2.0 is directly connected. Cisco Public 69 Task 3 .1.0.16.2.1. mgmt changet o co ntext admi n S 10.255.1.0 [200/0] via 1.0 area 0 sh route networ k 1.0.0 [200/0] via 1. outside sh run rout er master/a/asa1/admin(config)# !ASA2 Slave router ospf 1 !slave/a/asa2/admin(config)# networ k 1.1. 255.200.200.255.0 master/ a/as a1(co nfig) # C 1.0 is directly connected.200 to network 0.0 [200/0] via 1.0 area 0 Gateway of last resort is 1.255.0.255.10.1.0 255.2.1. outside log-ad j-ch anges C 172.10.0 255.140. inside !Change to admin cont ext C 1.255.255.0 is directly connected.2 55.0 255.0 255.0.0.1.0 255.255. inside !master /a/a dmin( confi g)# S* 0.0.2 55. inside timers lsa -grou p-pac ing 1 C 1.0.1. All rights reserved.255.0 is directly connected.1.0 [200/0] via 1. mgmt S 10. 1. outside C 172.200.0 255.16.0 255.1.0 0.255.255. 255.2.1.200 to network 0.2.1.0 is directly connected.0.255.0 timers spf 1 1 C 1.0 is directly connected.0 0.255.0 255.1. outside master/ a/ad min(c onfig )# changet o co ntext admi n slave/a/asa2/admin(config)# no rout er o spf 1 LT RSEC-2740 © 2015 Cisco and/or its affiliates. 3 subnets.2.0.0/24 is directly connected. GigabitEthernet1 10.1.1/32 [110/2] via 172.2.1.0.0/8 is variably subnetted.0/16 is variably subnetted.0. 2 subnets.16.1. GigabitEthernet2 1.0.0/0 [110/1] via 172.2 1.0/24 is directly connected.2.0.10.0 S* O*E2 0.16.1. GigabitEthernet2 L 10.1.3 [200/0] via 1.2 S 1.0.1.10.140. GigabitEthernet1 L 0.16. GigabitEthernet2 CSR2# LT RSEC-2740 © 2015 Cisco and/or its affiliates.2.0/24 is subnetted.2.0.3.0.2.CSR1 and CSR2 static routes to two ASAs Verify CSR2 CSR1 !CSR1 IP SLA routes !CSR2 IP SLA routes !CSR1# !CSR2# sh ip route sh ip route (snip) (snip) Gateway of last resort is 1.16.1.2 172.10.140. All rights reserved.1.0.0.3 to network 0. 2 masks C Task 3 1.0/0 [200/0] via 1.0 Gateway of last resort is 172. 2 subnets.1/32 is directly connected.16.0.1.1.0/8 is variably subnetted.1.16.1.0. GigabitEthernet2 O 172.200/32 is directly connected.0/24 [200/0] via 1.1.0 [200/0] via 1. 1d03h. 2 masks C 172.3 [200/0] via 1. 2 masks L C 10. 1 subnets S CSR1# 10.1.2.2. GigabitEthernet1 C 10.0/24 is directly connected. 3 subnets.140.0.200/32 is directly connected.2. GigabitEthernet2 L 172.0/24 is directly connected.0.1. Cisco Public 70 .1.0.1.16.0/8 is variably subnetted. GigabitEthernet2 1.3 [200/0] via 1.0.0. GigabitEthernet1 1. 1d03h.200/32 is directly connected. 2 masks 1.1.2.1 to network 0.1.2.1. iperf Outside-host (IP 172. All rights reserved.16./client.10. and 3… Inside-host (IP 10.30 Inside-host (IP 10.10. LT RSEC-2740 © 2015 Cisco and/or its affiliates.30) ping still working? Ping 172.Task 3 Verify Test Connections are up Measure connection convergence of each test: 1A.140./server.16.140.44) ssh session still working? ssh -l user 10.44 …after locating ASA unit that owns your connections.iperf Outside-host (IP 172.10.140. Cisco Public 71 .16.30) Restart if needed .2.2. 1B. 2.2.44) UDP packets arriving? . 30:22. bytes 0.140.2. idle 0:02:39.16. flags Y asa2:***************************************************************** 7 in use. Task 3 Cisco Public 72 . bytes 4198. flags -Y master/a/admin(config)# LT RSEC-2740 © 2015 Cisco and/or its affiliates. 18 most used Cluster stub connections: 1 in use.10.140.30:60810.2.Locate conn owner ASA Locate Owner ASA You will then do test 1 with this owner ASA ASA1 !master/a/admin(config)# cluster exec sh conn asa1(LOCAL):********************************************************** 7 in use. flags 10.2. bytes 0.10. flags UIOB 10. idle 0:02:39.16. All rights reserved.44:5001 inside TCP outside 172.16.30:22. idle 0:00:00.30:60810.140.2. idle 0:00:14.10.140.44:58952 inside UDP outside 172.44:58952 inside 10. 16 most used Cluster stub connections: 1 in use. 50 most used UDP outside 172.44:5001 inside 10. bytes 170520.10. 696 most used TCP outside 172.16. Task 3 Testing Resiliency of ASA Cluster Designs Individual Mode (ECMP) ASA1 UP G0/2 Down or ASA2 UP G0/2 ASA1 G0/3 ASA1 Down UP Down or Down Test 3: Disable ASA node via cluster CLI or dow n CCL port Po1 Inside Host Outside Host CSR1 CSR2 Po2 Test 2: Simulate ASA crash w ith ‘crashinfo force page-fault’ CCL ASA2 (1) Determine the connection owner (2) Shut down the port on owner ASA © 2015 Cisco and/or its affiliates. All rights reserved. UP G0/3 CCL Test 1: Dow n 1st ASA port on the sw itch for unit that ow ns TCP/UDP conns LT RSEC-2740 ASA2 Cisco Public 73 . Remove the data port on owner ASA Test 1 Protocol Task 1 Lost Pkts/Secs Observe and record if any packets were lost and if there was any impact on SSH session Open IE/Firefox inside RDP To shutdown ASA2 ports on the switch.40/ ping UDP iPerf ssh Disable ASA G0/2 port LT RSEC-2740 Task 3 © 2015 Cisco and/or its affiliates. All rights reserved. use browser home page on jumpbox PC.2. pointing to link: http://172.16. Cisco Public 74 . All rights reserved. Cisco Public 75 . and record in your convergence table Count the missed PINGs LT RSEC-2740 ASA detects that owner unit went down © 2015 Cisco and/or its affiliates.Measure Count how many UDP packets you lost Task 3 Count how many ping packets were lost Count (–nan%) UDP packets that were lost. GigabitEthernet1 10.200/32 is directly connected. GigabitEthernet2 CSR2# LT RSEC-2740 1. All rights reserved.1.0.3 L 1.0.1.2.0/8 is variably subnetted.16.1.16.0 S* O*E2 0. 1 subnets © 2015 Cisco and/or its affiliates.1.2.16. GigabitEthernet2 L 10.0.0/16 is variably subnetted.1.1.0.1. 2 subnets. 2 masks C 1.1. 2 masks C 172.16.0.0.0.0.0/0 [200/0] via 1. 2 masks L C 10. GigabitEthernet2 1.200/32 is directly connected.0/8 is variably subnetted.140. Cisco Public 76 .0.0.3.0.16.1.10.2.CSR1 and CSR2 have one route to ASA Verify Task 3 CSR2 CSR1 !CSR1 IP SLA routes !CSR2 IP SLA routes !CSR1# !CSR2# sh ip route sh ip route (snip) (snip) Gateway of last resort is 1.1. GigabitEthernet1 10.2.0.10.2.2. GigabitEthernet1 S 1.3 172. GigabitEthernet2 L 172.0/24 is directly connected.0/24 is directly connected.0.2.0/24 is directly connected. 1d03h.2. 3 subnets.0. GigabitEthernet1 C 1. GigabitEthernet2 O 172.0.140.1 to network 0.16.3 to network 0. 2 subnets. 2 masks 0.0 [200/0] via 1. 3 subnets.1.1/32 [110/2] via 172.0.0/24 is subnetted.1.1/32 is directly connected.0/0 [110/1] via 172.16.0/8 is variably subnetted.3 1.10.0.1. 1d03h.0/24 [200/0] via 1.1.1.0 Gateway of last resort is 172.0/24 is directly connected. GigabitEthernet2 S CSR1# 10.1.2.140.200/32 is directly connected. Cisco Public 77 . All rights reserved.Recover Down ASA Up or ‘no shut’ G0/2 port on down ASA Task 3 Enable cluster config on down ASA to add it the cluster immediately Down ASA Enable ASA G0/2 port ! Re-join approriate ASA unit changeto system config terminal !Define cluster group cluster group fw enable !Wait for ASA to detect master. and become a Slave unit !Cluster unit asa2 transitioned from DISABLED to SLAVE LT RSEC-2740 © 2015 Cisco and/or its affiliates. finish sync. 18 most used Cluster stub connections: 1 in use.2. flags UIOB 10. bytes 170520. idle 0:02:39. bytes 4198.10.10. idle 0:02:39.16.30:22.140.16. flags -Y master/a/admin(config)# LT RSEC-2740 © 2015 Cisco and/or its affiliates.44:5001 inside TCP outside 172. 696 most used TCP outside 172. idle 0:00:14.44:58952 inside UDP outside 172.44:5001 inside 10.2.16.140. idle 0:00:00.2. flags Y asa2:***************************************************************** 7 in use. 16 most used Cluster stub connections: 1 in use. 50 most used UDP outside 172. bytes 0.10.16.140. Task 3 Cisco Public 78 .Locate conn owner ASA Locate Owner ASA You will then do test 2 with this owner ASA ASA1 !master/a/admin(config)# cluster exec sh conn asa1(LOCAL):********************************************************** 7 in use. flags 10. bytes 0.30:22. All rights reserved.2.10.44:58952 inside 10.30:60810.30:60810.140. All rights reserved. finish sync. detect master.Test 2 Simulate a crash on owner ASA Task 3 Observe and record if any packets were lost and if there was any impact on SSH session Owner ASA ! Write configs and simulate ASA crash changeto system write memory all crashinfo force page -fault !Wait for ASA to boot up. Cisco Public 79 Task 2 Lost Pkts/Secs . and become a Slave unit !Cluster unit asa2 transitioned from DISABLED to SLAVE Simulate crash on owner ASA Protocol Crash owner ASA w/ CLI ping UDP iPerf ssh LT RSEC-2740 © 2015 Cisco and/or its affiliates. Cisco Public 80 . and record in your convergence table ASA crashes Count the missed PINGs LT RSEC-2740 © 2015 Cisco and/or its affiliates.Measure Count how many UDP packets you lost Task 3 Count how many ping packets were lost Count (–nan%) UDP packets that were lost. All rights reserved. Crashed ASA Re-joins LT RSEC-2740 © 2015 Cisco and/or its affiliates. syncs config. unit rejoins cluster Task 3 Detects master. and becomes a slave unit 81 . Cisco Public After reboot. All rights reserved. 44 …after locating ASA unit that owns your connections.30) Restart if needed .44) UDP packets arriving? .30) ping still working? Ping 172.140. 1B.16.10. and 3… Inside-host (IP 10.30 Inside-host (IP 10. LT RSEC-2740 © 2015 Cisco and/or its affiliates.2.10.iperf Outside-host (IP 172.16.16.140.Task 3 Verify Test Connections are up Measure connection convergence of each test: 1A.10. 2./client.44) ssh session still working? ssh -l user 10.140.2. All rights reserved./server.iperf Outside-host (IP 172.2. Cisco Public 82 . 16.30:60810. bytes 4198.16.44:58952 inside UDP outside 172. idle 0:02:39.2.Locate conn owner ASA Locate Owner ASA You will then do test 3 with this owner ASA ASA1 !master/a/admin(config)# cluster exec sh conn asa1(LOCAL):********************************************************** 7 in use.30:60810.140. idle 0:00:14.2.10. All rights reserved.2.10.44:5001 inside TCP outside 172.10. flags UIOB 10.16. bytes 0. bytes 170520. 50 most used UDP outside 172.2. idle 0:00:00.30:22.140.44:5001 inside 10. flags 10. flags Y asa2:***************************************************************** 7 in use.140. Task 3 Cisco Public 83 . idle 0:02:39.44:58952 inside 10.140. 696 most used TCP outside 172.16.30:22. 16 most used Cluster stub connections: 1 in use. bytes 0. 18 most used Cluster stub connections: 1 in use.10. flags -Y master/a/admin(config)# LT RSEC-2740 © 2015 Cisco and/or its affiliates. Task 3 Cisco Public 84 .Test 3 Protocol Shutdown the CCL port on owner ASA Observe and record if any packets were lost and if there was any impact on SSH session Task 3 Lost Pkts/Secs ping UDP iPerf ssh Disable ASA CCL port LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 85 . and record in your convergence table ASA switches to Master role Count the missed PINGs LT RSEC-2740 © 2015 Cisco and/or its affiliates.Measure Count how many UDP packets you lost Task 3 Count how many ping packets were lost Count (–nan%) UDP packets that were lost. All rights reserved. All rights reserved. and become a Slave unit !Cluster unit asa2 transitioned from DISABLED to SLAVE LT RSEC-2740 © 2015 Cisco and/or its affiliates. finish sync. Cisco Public 86 .Recover Down ASA Up or ‘no shut’ CCL port on down ASA Task 3 Enable cluster config on down ASA to add it the cluster immediately Down ASA Enable ASA CCL port ! Re-join approriate ASA unit changeto system config terminal !Define cluster group cluster group fw enable !Wait for ASA to detect master. 1.1. Cisco Public No need to reopen the connection 87 .2 (optional) Interna l IP 1.1. Down ASA that owns the connection Open a ssh connection through cluster Check when connection state is active Verify IP SLA routes on CSR2 for PAT pool network LT RSEC-2740 © 2015 Cisco and/or its affiliates.2.3 Preview External ASA1 IP 1. Tests Add equal cost routes for new PAT network on CSR2. All rights reserved.2 IP 1.2.1.1.3 CSR1 CSR2 This is a bonus task that involves ASA and CSR configuration changes.Task 3* CCL Task 3 Bonus*: Add PAT IP 1.1. ASA2 Add Port Address Translation to outside interface of ASA L3 cluster with IP SLA. 200 object network inside-network [sudo] password for user : cisco nat (inside. user@lubuntu:~$ Cisco Public 88 Task 3* .140.3 changeto context admin [200/0] via 1.0/24 [200/0] via 1.1.3.255.3.1.1.255.3.0 255.3.0/24 gw 172.1.3 200 track 2 !ASA master CSR2# show ip route master/a/asa1(config)# (snip) S 1.0 1.3 object network inside-network subnet 10.0 255.ASA PAT config CLI CSR2 routes to PAT network on ASA CSR2 !CSR2 config terminal ASA Master ip route 1.1. All rights reserved.255.3.2 200 track 1 ip route 1.10.1.1.16.outside) dynamic pat -pool pat-ips LT RSEC-2740 © 2015 Cisco and/or its affiliates.0 255.255.2 config terminal object network pat-ips OutsideHost range 1.2.2.0 1.2.2 1.255.2.1.0 !Must add routed on outside linux to new network ! sudo route add -net 1.255.1.1.2.3. 30) .iperf Outside-host (IP 172.140.iperf Inside-host (IP 10. Cisco Public 89 .16.44) ./client.10.Setup Test Connections with Xlates Task 3* iPerf UDP packets sending from Inside to Outside Host Outside-host (IP 172.2.140.10.2.30) ping 172.2./server.16.44 Ping and SSH Inside to Outside LT RSEC-2740 © 2015 Cisco and/or its affiliates.16.16.44) Can not go to inside now without a static NAT Inside-host (IP 10.44 or Ssh
[email protected]. All rights reserved. 3/49741 flags ri idle 0:00:23 timeout 0:00:30 10.3. idle 0:00:00.3.3/6300 flags ri idle 0:00:05 timeout 0:00:30 10.10.10.140. flags –Y ICMP outside 172.2.44:5001 inside bytes 2072700. idle 0:00:00.44:0 inside 5432.140.10. bytes © 2015 Cisco and/or its affiliates. bytes asa1:***************************************************************** asa1:***************************************************************** TCP PAT from inside:10.3.1.10.3.140. All rights reserved.10.16.2.140.30/41221 to outside:1. master/a/asa1/admin(config)# 10.Show conns and xlates on ASA cluster Verify translations Task 3* ASA Master ASA Master ! You can also try ‘show conn detail’ to decode the flags master/a/asa1/admin(config)# changeto context admin cluster exec sh xlate cluster exec sh conn master/a/asa1/admin(config)# asa2(LOCAL):********************************************************** asa2(LOCAL):********************************************************** TCP outside 172.16. idle 0:00:29.16.10. flags - ICMP outside 172.140.2.30:49741.30:41221. idle 0:00:00.2.30/6300 to outside:1. UDP PAT from inside:10. idle 0:00:00.3/6300 flags ri idle 0:00:36 timeout 0:00:30 10.10.30/49741 to outside:1.30/41221 to outside:1.2.44:22 inside bytes 5286.16.10.2. flags UxIO UDP outside 172.2/41221 flags ri idle 0:00:41 timeout 0:00:30 0 in use.3.140.44:5001 inside bytes 0. 2 most used TCP outside 172.140.140. idle 0:00:29.10.140.44:22 inside bytes 0.10.30:6300.10.16.3/49741 flags ri idle 0:01:56 timeout 0:00:30 10. ICMP PAT from inside:10.1.10. flags Y LT RSEC-2740 UDP PAT from inside:10.2/41221 flags ri idle 0:00:11 timeout 0:00:30 10.30/6300 to outside:1.16. flags Y UDP outside 172. Cisco Public 90 . flags TCP PAT from inside:10.30:49741.1.140.30:6300.1.3.30/49741 to outside:1.44:0 inside 0.30:41221.1.140.1.140. ICMP PAT from inside:10. 1.1.2.2.255.2 200 track 1 object network inside-network no ip route 1.3. Task 3* Cisco Public 91 . All rights reserved.0 1.0 255.255.3.0 1.Remove PAT Remove PAT and route configs for now Later in spanned.1.255. you will again add new PAT config ASA Master CSR2 changeto context admin config terminal config terminal no ip route 1.3 200 track 2 no nat (inside.0 255.255.1.outside) dynamic pat -pool pat-ips exit write memory LT RSEC-2740 © 2015 Cisco and/or its affiliates. 2.1. All rights reserved.1.1. continue to maintain state in Routed Firewall CSR2 ASA2 Tests Open test connections through cluster Check when the connection state active Measure convergence Bring Up downed ASA Verify one IP route on CSR1 to outside Verify one IP route on CSR2 to inside Cisco Public One hop Down ASA that owns the connection Ensure dead-intervals match (should be 3sec) © 2015 Cisco and/or its affiliates. External ASA1 Switch to L2 or spanned interface mode by moving ASA port-channel to ports assigned for spanned mode and applying Task 4 CLI. LT RSEC-2740 IP 1.1 92 .Task 4 CCL Task 4: L2 Cluster in Routed IP 1.1 Interna l Preview CSR1 One path Switch now load-balances under one IP path Review CSR and ASA OSPF config ASA1 and ASA2 in L2/Spanned cluster. Outside VLAN 8 .2.1 gig1 gig2 .0/24 VLAN 4 G0/3 .2.16.1 Outside host Internal G0/3 .2.10 G0/0 G0/1 ASA2 Slave CCL VLAN 25 IP pool needed only for management interface LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved.1 .16.200 .0/24 Master Inside VLAN 7 G0/0 VLAN 15 gig1 gig2 Inside host G0/1 ASA1 10.1.Task 4 ASA Spanned / Routed Cluster Diagram Master 1.1.30 ASA cluster nodes share the same IP for inside and outside VLANs.200 Po4.0/24 1.200 172.1.1 Po4.7 . 2.2-172.1.0/24 Cisco Public 93 .8 .2 CSR1 CSR2 mgmt_pool 172.2.44 External .0/24 .1.140.10.16. .Disable clustering feature on both units CLI Task 4 And prep ASAs to change mode to Spanned cluster ASA2 ASA1 ! Disable clustring on ASA1 unit ! Disable clustring on ASA2 unit changeto system changeto system config terminal config terminal cluster group fw cluster group fw no enable no enable ! Cluster disable is performing cleanup. !All data interfaces have been shutdown due to clustering being disabled. To recover either enable clustering or remove cluster group configuration. Cluster unit asa1 transitioned from MASTER to DISABLED Cluster unit asa2 transitioned from SLAVE to DISABLED ClusterDisabled/a/asa1(cfg-cluster)# ClusterDisabled/a/asa2(cfg -cluster)# LT RSEC-2740 © 2015 Cisco and/or its affiliates.. Cisco Public 94 . To recover either enable clustering or remove cluster group configuration. !All data interfaces have been shutdown due to clustering being disabled.done. ! Cluster disable is performing cleanup.done. All rights reserved. finish sync.cfg running-config !MUST confirm Y for YES. and become a Slave unit !Cluster unit asa2 transitioned from DISABLED to SLAVE LT RSEC-2740 © 2015 Cisco and/or its affiliates. remove these commands and wait to finish sync !Define cluster group !Wait 1 min for ASA1 unit to become Master cluster group fw !Cluster unit asa1 transitioned from DISABLED to MASTER Execute ASA2 CLI after ASA1 loads config and becomes Master local-unit asa2 cluster-interface GigabitEthernet0/3 ip 2.255.cfg task4-admin.cfg no shut copy /noconfirm milan/task4 -system.255.2 255.0 priority 20 console-replicate health-check holdtime 3 clacp system-mac auto system-priority 1 enable !Wait for ASA2 to detect master.2. Cisco Public 95 .2.Clear then re-apply L2 cluster configs CLI Task 4 Review changes needed to move ASA2 ASA1 ! Execute CLI to convert to L2 or Spanned interface mode ! Clear ASA2 unit and convert it to L2 Spanned interface mode changeto system changeto system config term config terminal clear config all clear config all cluster interface-mode spanned force cluster interface-mode spanned force !WARNING: Cluster interface -mode is changed to 'spanned' without…(snip) !Bring up interface for CCL interface GigabitEthernet0/3 copy /noconfirm milan/task4 -admin. All rights reserved. 3(2) Serial No.1 CCL MAC : c464.3(2) Serial No.2. asa1(LOCAL):********************************************************** Group Port-channel Protocol Span-cluster Ports ------+-------------+---------+------------+--------------2 Po2(U) LACP Yes Gi0/0(P) Gi0/1(P) asa2:***************************************************************** Group Port-channel Protocol Span-cluster Ports ------+-------------+---------+------------+--------------2 Po2(U) LACP Yes Gi0/0(P) !master/a/asa1# !Notice that Non-Stop Forwarding is enabled for ASA now changeto context admin show run router Cisco Public 96 Gi0/1(P) .: FCH16097J8X CCL IP : 2.: FCH16097J78 CCL IP : 2.2 CCL MAC : c464.2.1339.1339.Review cluster state and port-channel Verify ASA1 Master ASA1 Master !master/a/asa1# !master/a/asa1# changeto system show cluster info cluster exec show port-channel summary Cluster fw: On Interface mode: spanned This is "asa1" in state MASTER ID : 0 Version : 9. All rights reserved.2.1481 Last join : 19:17:36 UTC Jan 14 2015 Last leave: N/A master/a/asa1(config)# LT RSEC-2740 Task 4 © 2015 Cisco and/or its affiliates.2.1841 Last join : 18:43:37 UTC Jan 14 2015 Last leave: N/A Other members in the cluster: Unit "asa2" in state SLAVE ID : 1 Version : 9. 1 subnets 10. GigabitEthernet1 O 172.0.10. 2 subnets.1 to network 0.140. GigabitEthernet1 1.1.0.0/24 is subnetted.0.1.16.0 Gateway of last resort is 1.3. GigabitEthernet2 172.0/24 is directly connected.0.0/8 is variably subnetted.2.0. 00:21:25.2. 2 masks C 1.2.200/32 is directly connected.Verify one IP path through cluster from CSRs Verify CSR Routes Where are my OSPF routes? Hmmm.0.16.0/24 [110/11] via 1.0/0 [110/1] via 1. GigabitEthernet1 10.200/32 is directly connected.0/24 is directly connected. 2 subnets.16.0.2.0/16 is variably subnetted.10.0. GigabitEthernet1 L 1.0.2.0.0 [110/12] via 1.0. GigabitEthernet1 O C L O CSR2# 10. GigabitEthernet2 172.1.140.2.1.1.1.1.16. GigabitEthernet2 1.0. 2 masks 172. 00:25:26.1/32 [110/13] via 1.3. 2 masks 1.1.16.1.0/8 is variably subnetted.1.10.0.1.2.0/24 is directly connected. GigabitEthernet1 O 1.1.1.0 O*E2 O*E2 0. GigabitEthernet1 1.1.1. GigabitEthernet1 CSR1# LT RSEC-2740 © 2015 Cisco and/or its affiliates.16.1. GigabitEthernet2 L 10.16.16. All rights reserved.1. 3 subnets. 00:25:31. Cisco Public 97 0.1. 2 masks C 10.1.140. GigabitEthernet2 .0.1 to network 0. GigabitEthernet2 172. 3 subnets. GigabitEthernet1 172.2.2.1. Do my dead-intervals match? CSR1 CSR2 CSR1# Task 4 Are your routes missing? Make sure to sync up Master’s OSPF dead-interval to what you setup on CSRs in the Task 2.0/24 [110/11] via 1.0/8 is variably subnetted.1.16.0.1/32 is directly connected.1.1/32 [110/2] via 172.2.1. 3 subnets. 00:21:20.200/32 is directly connected.0/0 [110/1] via 172. 00:25:26.0/24 [110/12] via 1.0.1.1.0/16 is variably subnetted. GigabitEthernet1 O C L 1. 3d00h. 3d00h.0.1.1.2.0/24 is directly connected. 00:25:26. 2 masks O 172.16. CSR2# sh ip route sh ip route Gateway of last resort is 172.0.1.1. 44) UDP packets arriving? .140.10. and manually Inside-host (IP 10.30) ping still working? Ping 172.140.16. LT RSEC-2740 © 2015 Cisco and/or its affiliates. Cisco Public 98 . observe and record packets lost for UDP and PING.2. All rights reserved.10.2.30) Still sending packets… .Task 4 Setup Test Connections For each Test.16./client.2.30 Inside-host (IP 10. 1B.44) ssh session still working? Type one char and wait ssh -l user 10.iperf Outside-host (IP 172. 2.iperf Outside-host (IP 172.16./server. and 3.140.10.44 Measure connection convergence of each test: 1A. 1B. and 3 Spanned Interface Mode (Ether-channel) ASA1 UP G0/0 Down or ASA2 UP G0/0 ASA1 ASA1 Down UP G0/1 or ASA2 UP Down G0/1 Down CCL Test 1A: Dow n 1st ASA port on the sw itch for unit that ow ns TCP/UDP conns Test 1B: Dow n 2nd ASA port on Sw itch (w orst-case scenario) Po4 Inside Host Outside Host CSR1 CSR2 Po4 G0/3 ASA1 UP Test 2: Simulate ASA crash w ith ‘crashinfo force page-fault’ CCL ASA2 (1) Determine the connection owner (2) Shut down the port on owner ASA LT RSEC-2740 © 2015 Cisco and/or its affiliates. Cisco Public 99 Down or ASA2 UP Test 3: Disable ASA node via cluster CLI or dow n CCL port G0/3 Down . All rights reserved.Task 4 Resiliency Tests: 1A. 2. 2. use browser home page on jumpbox PC.40/ ping UDP iPerf ssh Disable ASA G0/0 port LT RSEC-2740 Task 4 © 2015 Cisco and/or its affiliates. Cisco Public 100 .Test 1A Protocol Task 1A Lost Pkts/Secs Remove one of two data ports in ASA Port-Channel Observe and record if any packets were lost and if there was any impact on SSH session Open IE/Firefox inside RDP To shutdown ASA2 ports on the switch. All rights reserved.16. pointing to link: http://172. 40/ ping UDP iPerf ssh Disable ASA G0/1 port LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 101 .Test 1B Protocol Task 1B Lost Pkts/Secs Remove the 2 nd data port in ASA Port-Channel Task 4 Observe and record how many packets were lost and how quickly on SSH session recovered Open IE/Firefox inside RDP To shutdown ASA2 ports on the switch.16.2. pointing to link: http://172. use browser home page on jumpbox PC. pointing to link: http://172. Cisco Public 102 . All rights reserved. use browser home page on jumpbox PC. finish sync.40/ Up the ASA G0/1 port Up the ASA G0/0 port Down ASA ! Re-join approriate ASA unit changeto system config terminal !Define cluster group cluster group fw enable !Wait for ASA2 to detect master.16. and become a Slave unit !Cluster unit asa2 transitioned from DISABLED to SLAVE LT RSEC-2740 © 2015 Cisco and/or its affiliates.Recover ASA unit Open IE/Firefox inside RDP ‘no shut’ both ASA data ports on down ASA Task 4 Re-enable cluster CLI to allow ASA to re-join To shutdown ASA2 ports on the switch.2. finish sync. All rights reserved.Crash connection owner ASA Test 2 Protocol Task 4 Removing owner ASA from cluster Task 2 Lost Pkts/Secs Owner ASA ping ! Write configs and simulate ASA crash write memory all UDP iPerf crashinfo force page-fault ssh !Define cluster group cluster group fw enable !Wait for ASA2 to detect master. and become a Slave unit !Cluster unit asa2 transitioned from DISABLED to SLAVE Crash owner ASA w/ CLI LT RSEC-2740 © 2015 Cisco and/or its affiliates. Cisco Public 103 . you can simply disable clustering UDP iPerf cluster group fw ssh no enable !Or you can ‘down’ the CCL for owner ASA via web page !As shown below in the home web page… Down CCL on owner ASA LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved.Test 3 Protocol Take out owner ASA unit from the cluster Task 4 Removing owner ASA from cluster Task 3 Lost Pkts/Secs Owner ASA ping !You can do test 3 in two ways !In the CLI. Cisco Public 104 . Cluster unit asa1 transitioned from DISABLED to SLAVE Watch CSR consoles for route convergence logs LT RSEC-2740 © 2015 Cisco and/or its affiliates. to rejoin master Down ASA Bring UP CCL on owner ASA !Enable cluster on disabled Slave !ClusterDisabled/a/asa1/admin(config)# changeto context sys !ClusterDisabled/a/asa1(config)# cluster group fw Enable !Detected Cluster Master. Cisco Public 105 . (snip) End configuration replication from Master. All rights reserved.Recover down ASA No Shut ASA CCL on Switch with IE Task 4 Enable cluster on ASA cli. Cisco Public 106 . Tests Add equal cost routes for new PAT network on CSR2. One Hop Aw ay CSR2 ASA2 Add Port Address Translation to outside interface of ASA L2 cluster with OSPF.1.1 (optional) Interna l Preview External ASA1 CSR1 One path This is a bonus task that involves adding back PAT configuration to ASA master.1. All rights reserved.Task 4* CCL Task 4 Bonus*: Add PAT IP 1.1.1 IP 1. Disable ASA that owns connections Open test connections through cluster Check when connection state is active Verify route on CSR2 for PAT pool network Verify xlates for open connections LT RSEC-2740 © 2015 Cisco and/or its affiliates.2. 1.0 1.0 255.140. we need to SSH from inside to outside host NOTE: because we are translating inside subnet.3 (snip) object network inside-network S subnet 10.0 1.16. you will need pat -ips object changeto context admin ip route 1.1 object network inside-network InsideHost nat (inside.3. Cisco Public 107 . we need to test ssh from inside to outside %ASA-7-609001: Built local -host outside:172.2.10.10.3.1.16.140.1.16.2.2.10.1. You can test with ssh from inside to outside linux.30 %ASA-5-305013: Asymmetric NAT rules matched for forward and reverse flows.outside) dynamic pat -pool pat-ips ! Enable logging on master (this enables it on the slave too) logging on user@inside-lnx:~$ ssh -l user 172.2.255.255.2 1.1 object network pat-ips sh ip route range 1.1.2.44/34770 dst inside:10.44 ! Re-open your SSH connection to expose the translation info
[email protected]/24 [1/0] via 1.255.30/22 denied due to NAT reverse path failure LT RSEC-2740 © 2015 Cisco and/or its affiliates.1.0 255.3. All rights reserved.16.44 %ASA-7-609001: Built local -host inside:10.44's password: ! Notice NAT syslog now denying connection outside to inside user@inside-lnx:~$ ! Therefore.2. Task 4* Open connection inside to outside ASA1 CSR2 needs a static route to ASA cluster PAT subnet to redistribute into OSPF CSR2 ! If you skipped Task 3*. Connection for tcp src outside:172.3.140. inbound conns now need static NAT.255.CLI Add address translation cli Due to PAT for inside subnet. 30) ping 172.140.2.140.10.Setup Test Connections with Xlates Task 4* iPerf UDP packets sending from Inside to Outside Host Outside-host (IP 172.16.2. Cisco Public 108 .30) .44) .44 Ping and SSH Inside to Outside LT RSEC-2740 © 2015 Cisco and/or its affiliates.2./client.iperf Inside-host (IP 10.44) Can not go to inside now without a static NAT Inside-host (IP 10.iperf Outside-host (IP 172.10./server.44 Ssh
[email protected]. All rights reserved.16.16. 0. GigabitEthernet1 1.2.0.1.16. 00:42:31. 2 masks 10. GigabitEthernet2 10.0/0 [110/1] via 1.1.30:50511.0/24 is directly connected. GigabitEthernet1 1.1. 9 most used Cluster stub connections: 1 in use.0/16 is variably subnetted.0/24 [110/11] via 1.140. GigabitEthernet1 1. 00:42:27.10.16.1.3.1. flags UxIO 10. 00:22:50.10.3.1/32 [110/13] via 1. 4 subnets.140.16.1.1.1.1.1. master/a/asa1/admin(config)# LT RSEC-2740 Task 4* © 2015 Cisco and/or its affiliates.200/32 is directly connected.30:50511. GigabitEthernet1 1.0 TCP PAT from inside:10.0.1.1 to network 0. 2 subnets.1.0.140.3. 00:42:27.140.1.0.140.Verify xlate(s) through cluster and OSPF route on CSR1 Verify ASA1 CSR1 !master/a/asa1/admin(config)# !CSR1# sh ip route cluster exec show xlate asa1(LOCAL):********************************************************** Gateway of last resort is 1.44:22 inside bytes 0.1.2.0. 2 masks 1. idle 0:07:45.0/8 is variably subnetted.1.3.2.0.3/50511 flags ri idle 0:00:27 timeout 0:00: 30 O*E2 asa2:***************************************************************** C L O O E2 TCP PAT from inside:10.16.1.2. idle 0:07:45.140.0/24 is directly connected. GigabitEthernet1 10.10. GigabitEthernet1 172. 0 most used TCP outside 172.1. Cisco Public 109 0.3/50511 flags ri idle 0:25:46 timeout 0:00:30 master/a/asa1/admin(config)# cluster exec show conn asa1(LOCAL):********************************************************** C L 4 in use.1. 2 subnets. All rights reserved.10.0/8 is variably subnetted.0.1/32 is directly connected.1. 19 most used Cluster stub connections: 1 in use.1.1. 3 most used TCP outside 172.1.0/24 [110/12] via 1.10. 2 masks 172. flags y O O CSR1# 10. GigabitEthernet1 .1.30/50511 to outside:1. GigabitEthernet2 172. 00:42:27.1.1.30/50511 to outside:1.10.44:22 inside bytes 4102. asa2:***************************************************************** 1 in use.16.0/24 [110/20] via 1.0. All rights reserved.outside) dynamic pat -pool pat-ips write memory LT RSEC-2740 © 2015 Cisco and/or its affiliates.3.1 object network inside-network no nat (inside.0 1. Task 4* Cisco Public 110 . you will add PAT config ASA Master CSR2 changeto context admin config terminal config terminal no ip route 1.1.255.2.1.255.0 255.Remove PAT Remove PAT and route configs for now Later in spanned. Cisco Public 111 .1.200/16 Interna l Preview External ASA1 CSR1 One Subnet Change to Transparent mode in admin context.2. this clear ASA configuration Directly Connected CSR2 ASA2 Rebuild context configuration by applying Task 5 CLI to ASAs and CSRs. All rights reserved.1. to allow peering OSPF through ASA Down ASA that owns most connections Open test connections through cluster Change OSPF configs on CSRs Check when the connection state active Verify OSPF route on CSR1 to outside Measure convergence Verify OSPF route on CSR2 to inside LT RSEC-2740 © 2015 Cisco and/or its affiliates.Task 5 CCL Task 5: L2 Cluster in Transp IP 1.200/16 IP 1. Tests Change CSR IP addresses to /16 subnet.1. 0.140.0/16 Master Inside VLAN 7 G0/0 VLAN 15 Inside host VLAN 4 gig1 .2.0/24 0/3 .ASA Spanned / Transparent Cluster Diagram CSRs directly connected over 1.10.1.8 BVI1 Po4.200 .44 Outside host Internal 172.2 mgmt_pool G0/0 G0/1 ASA2 Slave 2.2.0.1 172.2.0.10 Task 5 CSR2 Inside and Outside interfaces Bridged by ASA cluster External .16.16.7 BVI1 CSR1 0/3 .1.2-172. Cisco Public 112 .0/16 1.1.0/24 CCL VLAN 25 LT RSEC-2740 © 2015 Cisco and/or its affiliates.200 .1.1.1 gig1 gig2 .1.200 Po4.0/16 subnet through L2 firewall Master 1. All rights reserved.2.0/24 gig2 .30 Outside VLAN 8 G0/1 ASA1 10.16. cfg LT RSEC-2740 mac 113 ..cfg interface bridge-group context admin .56bf. address ---------------------------------------------------------------------------------- config-url disk0:/task5-admin.cfg INFO: Admin context will take some time to come up .34b8 dynamic 5 1 inside 0016.Change context to Transparent FW mode CLI Task 5 Verify mac-addresses of CSRs ASA1 ASA1 !Install a transparent firewall context config for current admin context !master/a/asa1/admin(config -if)# config terminal sh mac-address-table changeto system copy /noconfirm milan/task5 -admin.cfg task5-admin.dbc2 dynamic 4 1 master/a/asa1/admin(config -if)# master/a/asa1(config -ctx)# © 2015 Cisco and/or its affiliates.9cd3.56bf. please wait... Cryptochecksum (unchanged): dcf70f21 bc4b86f6 c570e03f 2093dcd6 INFO: Context admin was created with URL disk0:/task5-admin. type Age(min) Cisco Public inside 0050. All rights reserved.b780 dynamic 4 1 outside 0050. 1/32 [110/3] via 1.16.2.0 config terminal interface GigabitEthernet1 ip address 1. 00:01:20.2.0. GigabitEthernet1 LT RSEC-2740 Task 5 © 2015 Cisco and/or its affiliates. once they can ping each other and peer directly ! Verify routes on CSRs.255. GigabitEthernet1 172.1. 2 subnets.2.Change CSRs to directly connected routers CLI CSR1 CSR2 !Change CSR subnet to /16 so they can peer through ASA cluster !Change CSR subnet to /16 so they can peer through ASA cluster config terminal interface GigabitEthernet1 ip address 1.2.0.1.255 area 0 network 1.1.1.1.1.3.0 router ospf 1 no network 1.0.2.0/0 [110/1] via 172.0.0.0. GigabitEthernet1 172.0 0.255.0 Gateway of last resort is 1.0.3. Cisco Public O O CSR2# 114 0.0.1.2.2.1.0. GigabitEthernet2 .0.1.0/0 [110/1] via 1.16.255 area 0 router ospf 1 no network 1. 00:01:20.0 0.200.0 0.0/24 is subnetted. 03:17:49. once they can ping each other and peer directly show ip route ospf show ip route ospf Gateway of last resort is 172.255 area 0 ! Verify routes on CSRs.16. 2 masks 172.0.0.1.0.0/16 is variably subnetted. 00:01:20.0.1.200.1.0. 00:01:29.0.0 [110/2] via 1.16.10.0.1/32 [110/2] via 172.2.0. GigabitEthernet1 172.16. 3 subnets.1 to network 0. 1 subnets 10.200.0.200 255. 2 masks 172.0 O*E2 O O CSR1# O*E2 0.1. 03:17:49.0.1.0. GigabitEthernet2 10.2.200.140.16.200 to network 0.16.1.0/24 [110/2] via 1.16.1. All rights reserved.0/16 is variably subnetted.2.0 0.0.200 255.255.255.255 area 0 network 1. 0.5. flags master/a/asa1/admin(config)# LT RSEC-2740 © 2015 Cisco and/or its affiliates. flags OSPF outside 1. 117 most used OSPF outside 224.200.Show OSPF connections through ASA cluster Verify ASA1 Master !master/a/asa1/admin(config)# cluster exec show conn asa1(LOCAL):********************************************************** 0 in use. 6 most used asa2:***************************************************************** 2 in use.0.5 inside 1. bytes 179984.1. bytes 181176.1. Cisco Public 115 Task 5 .0.2.1. idle 0:00:00. All rights reserved.0. 8 most used Cluster stub connections: 0 in use.200 inside 224. 19 most used Cluster stub connections: 0 in use. idle 0:00:00. and 3… Inside-host (IP 10.30) Restart if needed .140.10. Cisco Public 116 .44 …after locating ASA unit that owns your connections.140.2. LT RSEC-2740 © 2015 Cisco and/or its affiliates.10.30 Inside-host (IP 10.30) ping still working? Ping 172. 2.10./server.16. All rights reserved.16.iperf Outside-host (IP 172.16. 1B.Task 5 Setup Test Connections Again Measure connection convergence of each test: 1A.iperf Outside-host (IP 172.2./client.44) Restart ssh session… ssh -l user 10.2.140.44) UDP packets arriving? . 16. bytes 264.5 inside 1.2. 0 most used OSPF outside 224.30:2841.140.44:5001 inside 10. idle 0:00:00.30:2841. idle 0:02:05. bytes 364400.1.2.30:22. Cisco Public 117 .5.10.44:0 NP Identity Ifc OSPF outside 1.200 inside UDP outside 10. flags 172.0.10.16. 10 most used Cluster stub connections: 2 in use.44:0 inside TCP outside ICMP outside 172. bytes 1440600.30:2841 NP Identity Ifc OSPF inside 1.16.140.2.44:55501 inside 10.0. flags UIOB ICMP inside 10.10.2.2. 96 most used OSPF outside 1.2. idle 0:02:05.44:5001 inside 10. flags –y asa2:***************************************************************** 3 in use. flags z master/a/asa1/admin(config)# LT RSEC-2740 Task 5 © 2015 Cisco and/or its affiliates.16. idle 0:00:00.16.140.30:36188. All rights reserved.140.44:0. bytes 0. flags z 224. flags z 224. flags 172.200.5.0. bytes 159712.2. idle 0:00:02. bytes 0.30:36188.10. bytes 0.44:55501 inside 10.1.16.2. 3 most used Cluster stub connections: 4 in use.1.1.200 inside UDP outside 1. bytes 0. idle 0:00:00.10. bytes 4262.200. idle 0:00:00.10.140.1.30:2841. flags - ICMP outside 172. idle 0:00:00. bytes 363712. flags y 10.140.30:22.2.16. idle 0:00:10.10. bytes 0. idle 0:00:00. flags 172.10.2.140.0.5.0. flags 224.1.140. flags ICMP outside 172.Locate conn owner ASA Find Owner ASA Shut down ASA data port on the Switch with IE ASA1 !master/a/asa1/admin(config)# cluster exec show conn asa1(LOCAL):********************************************************** 4 in use. bytes 0. idle 0:00:00.1.1.0. idle 0:00:00.200 NP Identity Ifc TCP outside 172.0. bytes 160272. flags z 10.0. idle 0:00:00.1.44:0 inside OSPF outside 1.16.2.200 NP Identity Ifc 172. All rights reserved. 2. and 3 Spanned Interface Mode (Ether-channel) ASA1 UP G0/0 Down or ASA2 UP G0/0 ASA1 ASA1 Down UP G0/1 or ASA2 UP Down G0/1 Down CCL Test 1A: Dow n 1st ASA port on the sw itch for unit that ow ns TCP/UDP conns Test 1B: Dow n 2nd ASA port on Sw itch (w orst-case scenario) Po4 Inside Host Outside Host CSR1 CSR2 Po4 G0/3 ASA1 UP Test 2: Simulate ASA crash w ith ‘crashinfo force page-fault’ CCL ASA2 (1) Determine the connection owner (2) Shut down the port on owner ASA LT RSEC-2740 © 2015 Cisco and/or its affiliates. Cisco Public 118 Down or ASA2 UP Test 3: Disable ASA node via cluster CLI or dow n CCL port G0/3 Down .Task 5 Resiliency Tests: 1B. All rights reserved.Test 1B Protocol Task 1B Lost Pkts/Secs Remove both data ports in ASA Port-Channel Task 5 Observe and record if any packets were lost and if there was any impact on SSH session Open IE/Firefox inside RDP To shutdown ASA2 ports on the switch. pointing to link: http://172.40/ ping UDP iPerf ssh Disable ASA G0/0 port LT RSEC-2740 © 2015 Cisco and/or its affiliates.16.2. use browser home page on jumpbox PC. Disable ASA G0/1 port Cisco Public 119 . All rights reserved.Recover ASA unit ‘no shut’ both ASA data ports on down ASA Task 5 Re-enable cluster CLI to allow ASA to re-join Up the ASA G0/1 port Up the ASA G0/0 port Down ASA ! Re-join approriate ASA unit changeto system config terminal !Define cluster group cluster group fw enable !Wait for ASA2 to detect master. and become a Slave unit !Cluster unit asa2 transitioned from DISABLED to SLAVE LT RSEC-2740 © 2015 Cisco and/or its affiliates. finish sync. Cisco Public 120 . and become a Slave unit !Cluster unit asa2 transitioned from DISABLED to SLAVE Crash owner ASA w/ CLI LT RSEC-2740 © 2015 Cisco and/or its affiliates.Crash connection owner ASA Test 2 Protocol Task 5 Removing owner ASA from cluster Task 2 Lost Pkts/Secs Owner ASA ping ! Write configs and simulate ASA crash changeto system UDP iPerf write memory all ssh crashinfo force page-fault !Wait for ASA2 to detect master. finish sync. Cisco Public 121 . All rights reserved. All rights reserved. Cisco Public 122 . you can simply disable clustering UDP iPerf changeto system ssh cluster group fw no enable !Or you can ‘down’ the CCL for owner ASA via web page !As shown below in the home web page… Down CCL on owner ASA LT RSEC-2740 © 2015 Cisco and/or its affiliates.Test 3 Protocol Take out owner ASA unit from the cluster Task 5 Removing owner ASA from cluster Task 3 Lost Pkts/Secs Owner ASA ping !You can do test 3 in two ways !In the CLI. Cluster unit asa1 transitioned from DISABLED to SLAVE Watch CSR consoles for route convergence logs LT RSEC-2740 © 2015 Cisco and/or its affiliates.Recover down ASA No Shut ASA CCL on Switch with IE Task 5 Enable cluster on ASA cli. Cisco Public 123 . (snip) End configuration replication from Master. to rejoin master Down ASA Bring UP CCL on owner ASA !Enable cluster on disabled Slave !ClusterDisabled/a/asa1/admin(config)# changeto context system !ClusterDisabled/a/asa1(config)# cluster group fw enable !Detected Cluster Master. All rights reserved. 200/16 Interna l Preview External ASA1 CSR1 One Subnet This is a bonus task to add PAT configuration in transparent firewall mode on ASA master.1. Directly Connected CSR2 ASA2 Add Port Address Translation to outside interface inside admin context.200/16 IP 1. All rights reserved. Tests Remove older route for PAT network on CSR2.1. Cisco Public 124 .1.2. it is not needed as PAT and CSR interfaces are now in same network Down ASA that owns the connection Open test connections through cluster Check when connection state is active Verify xlates LT RSEC-2740 © 2015 Cisco and/or its affiliates.Task 5* CCL Task 5 Bonus*: Add PAT (optional) IP 1. 0/24 is subnetted. Cisco Public 125 . GigabitEthernet2 CSR2# LT RSEC-2740 © 2015 Cisco and/or its affiliates.0/24 is directly connected.0.0.1.0.2. 00:42:16. 3 subnets.16.1.0/16 is directly connected. All rights reserved.2 1.3.0/8 is variably subnetted.2.0/16 is variably subnetted.16. 08:02:20.200.1.3.1. GigabitEthernet1 172. 1 subnets nat (inside.16.2. GigabitEthernet2 clear local L 172.3.0.1.0.16.0. you will need pat -ips and inside-network objects !CSR2# object network pat-ips show ip route range 1.2.0 255.3 Gateway of last resort is 172.200/32 is directly connected. 3 subnets.0 [110/2] via 1.255.0 object network inside-network subnet 10.0.10.0.10.0 O*E2 0.2.1/32 [110/2] via 172.outside) dynamic pat -pool pat-ips O 10. GigabitEthernet2 1.1 to network 0.2. GigabitEthernet1 10.255.1.1.200/32 is directly connected.Introduce PAT CLI Task 5* Remove route on CSR2 CSR2 ASA1 ! If you skipped Task 3*.16.0. 2 masks ! You may need to clear existing conns to create an xlate C 172.1.0. GigabitEthernet2 O 172.16.140.140. 3 masks C changeto context admin L object network inside-network 1.0/0 [110/1] via 172. 08:02:20.16. GigabitEthernet1 1. Setup Test Connections with Xlates Task 5* iPerf UDP packets sending from Inside to Outside Host Outside-host (IP 172.10.44 Ping and SSH Inside to Outside LT RSEC-2740 © 2015 Cisco and/or its affiliates.2.2.16.44) .2.2.30) ping 172.16. so SSH from inside to outside Inside-host (IP 10.44 ssh
[email protected]) .iperf Inside-host (IP 10.16./server. All rights reserved.10.iperf Outside-host (IP 172. Cisco Public 126 .44) Can not go to inside now without a static NAT./client.140.140. 140.static. master/a/asa1/admin(config)# LT RSEC-2740 © 2015 Cisco and/or its affiliates. r .3.1.140.2.static. e .extended. All rights reserved.10.0.2.DNS. idle 0:00:06.30/50519 to outside:1.0.16.2. i .200. flags OSPF outside 1. flags y master/a/asa1/admin(config)# 10. Flags: D . bytes 132. Cisco Public 127 .twice. TCP PAT from inside:10.1.44:22 inside bytes 4166.140.net-to-net asa2:***************************************************************** TCP PAT from inside:10. s .10.portmap.1.1. 3 most used 1 in use. 2 most used OSPF outside 224.dynamic.44:22 inside bytes 0. i .DNS.5.2/50519 flags ri idle 0:03:03 timeout 0:00:30 3 in use.2.dynamic.twice. TCP outside 172.Re-open test connections Verify Task 5* Verify conn and xlates are created ASA1 ASA1 cluster exec show conn cluster exec show xlate asa1(LOCAL):********************************************************** asa1(LOCAL):********************************************************** 1 in use. I .identity.extended. bytes 159000.200 inside 1.1.2/50519 flags ri idle 0:00:29 timeout 0:00:30 s .200. 2 most used 1 in use. 2 most used Cluster stub connections: 0 in use.30:50519. OSPF outside 1.net-to-net TCP outside 172. 6 most used asa2:***************************************************************** Cluster stub connections: 1 in use. T . idle 0:00:06. 0 most used Flags: D .1.portmap.0. I .1. r . idle 0:00:54. flags UIO 10.5 inside flags 1. e . N .identity.30:50519. idle 0:00:00. N .30/50519 to outside:1.3.16. idle 0:00:00. T .1. bytes 158544.200 inside flags 224.0.140.10.10. CONGRATULATIONS…. on completing the LTRSEC-2740 lab 128 Call to Action • Visit the World of Solutions for – Cisco Campus – Visit Network and Content Security Booths – Technical Solution Clinics • Meet the Engineer – ASA experts from our team will be available to meet you • Lunch time Table Topics • DevNet zone related labs and sessions • Recommended Reading: for reading material and further resources for this session, please visit www.pearson-books.com/CLMilan2015 LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 129 Complete Your Online Session Evaluation • Please complete your online session evaluations after each session. Complete 4 session evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt. • All surveys can be completed via the Cisco Live Mobile App or the Communication Stations LT RSEC-2740 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 130 Additional Slides . ASA Cluster to Routers Data Plane – Individual Mode Single Attach IP 1.1.1 Interface Layer 3 mode • Dedicated IP/MAC addresses per ASA Interface • ECMP from both sides of ASA (outside and inside) • Improve convergence by tuning timers Dual Attach IP 1.2.1.2 IP 1.2.1 vPC IP 1. All rights reserved. Inside CCL Cisco Public 132 vPC .2 IP 1.1.1.1 IP 1.2.1.1 Po 200 IP 1.1.1.1.2 IP 1.4 Po 101 IP 1.1.3 Po 202 Po 203 IP 1.2.1.1.2.1.4 Po 103 IP 1.1.1.3 IP 1.2 Po 201 IP 1.4 IP 1.2.2.1.1.1.1.1.1.1.3 IP 1.1.4 Outside Outside Inside CCL LT RSEC-2740 Po 100 © 2015 Cisco and/or its affiliates.3 Po 102 IP 1.2.1.1. All rights reserved.ASA Cluster to Switch Data Plane – Spanned Mode Interface Layer 2 mode • One IP per Ether-channel interface shared by the cluster • A port ID on each ASA joins the a spanned port-channel • vPC extends the channel across two switches • Data Plane MUST use cLACP cLACP ASA Po 10 cLACP ASA Po 10 LACP vPC 100 Po 100 Classic Switch N7K/vPC Cat/VSS CCL LT RSEC-2740 © 2015 Cisco and/or its affiliates. CCL Cisco Public 133 . 134 .