Routing with NSX using multiple sites (with iBGP between the UDLR and the ESG's)
From Iwan
Routing with NSX using multiple sites (with iBGP between the UDLR and the ESG's)
Introduction
In one of my previous articles I demonstrated the routing between two Data Centers with using eBGP.
This article will be about doing the same, but now with iBGP between the DLR control VM and the ESG's of both sites.
Diagram
The following diagram will be used for our setup.
LAB100_-_NSX_Routing_based_on_iBGP.pdf
Components
I am still using the same components as in this previous article.
Use–Cases
The use-case that I will be testing is the same use-case as in the previous article (only now with iBGP)
Tenant one
Tenant one will have workloads in DC1 and in DC2 and because it is the routing protocol to determine the egress path the traffic will exit (based on the configuration) from the primary site. When the primary site is down the traffic should exit from the secondary site.
Routing protocol options
All possible routing options are described in the previous article.
The routing option in this article will be:
Option 2
- iBGP peering between the UDLR and the ESG’s
- eBGP peering between the ESG’s and the external routers
- eBGP peering between the external routers and the CORE
Option 2
Because of the many options available in the setup I want to start with the following: Tenant 1 workloads with the use of UDLR-01 and option 1 routing. The other options will be outlined in other articles.
I have only documented the changes below compared to the previous article. The "show" outputs are given fully again of all the routing tables and the route path verification as well.
UDLR configuration
DC1
The UDLR is in AS# 65530 and it will stay this way. We will change the ESG's to be in this same AS.
- Change the peering AS towards the ESG
DC2
The UDLR control VM does not exist in DC2.
ESG configuration
DC1
- Change the Local AS
- Configure the BGP peers towards the external routers and the UDLR.
- Disable BGP
- Enable BGP
DC2
- Change the Local AS
- Configure the BGP peers towards the external routers and the UDLR.
- Disable BGP
- Enable BGP
Cisco1000V configuration
DC1
! root ##bl##hostname rt-a-01 ! no router bgp 65511 router bgp 65511 bgp router-id 10.11.11.31 bgp log-neighbor-changes neighbor 10.11.11.253 remote-as 65510 neighbor 10.11.11.253 description CS01 neighbor 10.100.19.2 remote-as 65530 neighbor 10.100.19.2 description ESG-A ! address-family ipv4 neighbor 10.11.11.253 activate neighbor 10.100.19.2 activate exit-address-family ! ip route 10.200.19.0 255.255.255.0 10.11.11.253 ip route 10.200.21.0 255.255.255.0 10.11.11.253 !
! root ##bl##hostname rt-a-02 ! no router bgp 65511 router bgp 65511 bgp router-id 10.11.11.32 bgp log-neighbor-changes neighbor 10.11.11.253 remote-as 65510 neighbor 10.11.11.253 description CS01 neighbor 10.100.21.2 remote-as 65530 neighbor 10.100.21.2 description ESG-A ! address-family ipv4 neighbor 10.11.11.253 activate neighbor 10.100.21.2 activate exit-address-family ! ip route 10.200.19.0 255.255.255.0 10.11.11.253 ip route 10.200.21.0 255.255.255.0 10.11.11.253 !
DC2
! root ##bl##hostname rt-b-01 ! no router bgp 65512 router bgp 65512 bgp router-id 10.11.11.33 bgp log-neighbor-changes neighbor 10.11.11.253 remote-as 65510 neighbor 10.11.11.253 description CS01 neighbor 10.200.19.2 remote-as 65530 neighbor 10.200.19.2 description ESG-B ! address-family ipv4 neighbor 10.11.11.253 activate neighbor 10.200.19.2 activate exit-address-family ! ip route 10.100.19.0 255.255.255.0 10.11.11.253 ip route 10.100.21.0 255.255.255.0 10.11.11.253 !
! root ##bl##hostname rt-b-02 ! no router bgp 65512 router bgp 65512 bgp router-id 10.11.11.34 bgp log-neighbor-changes neighbor 10.11.11.253 remote-as 65510 neighbor 10.11.11.253 description CS01 neighbor 10.200.21.2 remote-as 65530 neighbor 10.200.21.2 description ESG-B ! address-family ipv4 neighbor 10.11.11.253 activate neighbor 10.200.21.2 activate exit-address-family ! ip route 10.100.19.0 255.255.255.0 10.11.11.253 ip route 10.100.21.0 255.255.255.0 10.11.11.253 !
Core configuration
Between the Core and the Cisco 1000V CSR's the AS did not change and we are still doing eBGP there.
UDLR route peering verification
DC1
root ##bl##REGX-UDLR01-0> show ip bgp neighbors BGP neighbor is 172.39.39.1, remote AS 65530, root ##y##BGP state = Established, up Hold time is 180, Keep alive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast:advertised and received Graceful restart Capability:none Restart remain time: 0 Received 10 messages, Sent 11 messages Default minimum time between advertisement runs is 30 seconds For Address family IPv4 Unicast:advertised and received Index 1 Identifier 0x57572d5c Route refresh request:received 0 sent 0 root ##y## Prefixes received 6 sent 7 advertised 7 Connections established 1, dropped 2 Local host: 172.39.39.13, Local port: 179 Remote host: 172.39.39.1, Remote port: 21815 BGP neighbor is 172.39.39.2, remote AS 65530, root ##y##BGP state = Established, up Hold time is 180, Keep alive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast:advertised and received Graceful restart Capability:none Restart remain time: 0 Received 10 messages, Sent 10 messages Default minimum time between advertisement runs is 30 seconds For Address family IPv4 Unicast:advertised and received Index 2 Identifier 0x57572d5c Route refresh request:received 0 sent 0 root ##y## Prefixes received 6 sent 7 advertised 7 Connections established 1, dropped 2 Local host: 172.39.39.13, Local port: 179 Remote host: 172.39.39.2, Remote port: 41244
DC2
The UDLR control VM does not exist in DC2.
ESG route peering verification
DC1
root ##bl##REGA-ESG01-0> show ip bgp neighbors BGP neighbor is 10.100.19.1, remote AS 65511, root ##y##BGP state = Established, up Hold time is 180, Keep alive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast:advertised and received Graceful restart Capability:none Restart remain time: 0 Received 16 messages, Sent 14 messages Default minimum time between advertisement runs is 30 seconds For Address family IPv4 Unicast:advertised and received Index 1 Identifier 0xc2a60eac Route refresh request:received 0 sent 0 root ##y## Prefixes received 2 sent 10 advertised 10 Connections established 1, dropped 1 Local host: 10.100.19.2, Local port: 61962 Remote host: 10.100.19.1, Remote port: 179 BGP neighbor is 10.100.21.1, remote AS 65511, root ##y##BGP state = Established, up Hold time is 180, Keep alive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast:advertised and received Graceful restart Capability:none Restart remain time: 0 Received 17 messages, Sent 14 messages Default minimum time between advertisement runs is 30 seconds For Address family IPv4 Unicast:advertised and received Index 2 Identifier 0xc2a60eac Route refresh request:received 0 sent 0 root ##y## Prefixes received 2 sent 10 advertised 10 Connections established 1, dropped 1 Local host: 10.100.21.2, Local port: 27456 Remote host: 10.100.21.1, Remote port: 179 BGP neighbor is 172.39.39.13, remote AS 65530, root ##y##BGP state = Established, up Hold time is 180, Keep alive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast:advertised and received Graceful restart Capability:none Restart remain time: 0 Received 15 messages, Sent 18 messages Default minimum time between advertisement runs is 30 seconds For Address family IPv4 Unicast:advertised and received Index 3 Identifier 0xc2a60eac Route refresh request:received 0 sent 0 root ##y## Prefixes received 7 sent 6 advertised 6 Connections established 2, dropped 2 Local host: 172.39.39.1, Local port: 21815 Remote host: 172.39.39.13, Remote port: 179
DC2
root ##bl##REGB-ESG01-0> show ip bgp neighbors BGP neighbor is 10.200.19.1, remote AS 65512, root ##y##BGP state = Established, up Hold time is 180, Keep alive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast:advertised and received Graceful restart Capability:none Restart remain time: 0 Received 18 messages, Sent 17 messages Default minimum time between advertisement runs is 30 seconds For Address family IPv4 Unicast:advertised and received Index 1 Identifier 0x359d76ec Route refresh request:received 0 sent 0 root ##y## Prefixes received 2 sent 10 advertised 10 Connections established 1, dropped 1 Local host: 10.200.19.2, Local port: 43262 Remote host: 10.200.19.1, Remote port: 179 BGP neighbor is 10.200.21.1, remote AS 65512, root ##y##BGP state = Established, up Hold time is 180, Keep alive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast:advertised and received Graceful restart Capability:none Restart remain time: 0 Received 18 messages, Sent 15 messages Default minimum time between advertisement runs is 30 seconds For Address family IPv4 Unicast:advertised and received Index 2 Identifier 0x359d76ec Route refresh request:received 0 sent 0 root ##y## Prefixes received 2 sent 10 advertised 10 Connections established 1, dropped 1 Local host: 10.200.21.2, Local port: 19183 Remote host: 10.200.21.1, Remote port: 179 BGP neighbor is 172.39.39.13, remote AS 65530, root ##y##BGP state = Established, up Hold time is 180, Keep alive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast:advertised and received Graceful restart Capability:none Restart remain time: 0 Received 18 messages, Sent 20 messages Default minimum time between advertisement runs is 30 seconds For Address family IPv4 Unicast:advertised and received Index 3 Identifier 0x359d76ec Route refresh request:received 0 sent 0 root ##y## Prefixes received 7 sent 6 advertised 6 Connections established 2, dropped 2 Local host: 172.39.39.2, Local port: 41244 Remote host: 172.39.39.13, Remote port: 179
Cisco1000V route peering verification
DC1
root ##bl##rt-a-01#show ip bgp summary BGP router identifier 10.11.11.31, local AS number 65511 BGP table version is 54, main routing table version 54 15 network entries using 3720 bytes of memory 15 path entries using 1800 bytes of memory 3/3 BGP path/bestpath attribute entries using 720 bytes of memory 3 BGP AS-PATH entries using 88 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 6328 total bytes of memory BGP activity 15/0 prefixes, 34/19 paths, scan interval 60 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd root ##y##10.11.11.253 4 65510 22 24 54 0 0 00:13:57 5 root ##y##10.100.19.2 4 65530 17 18 54 0 0 00:10:26 10 rt-a-01#
root ##bl##rt-a-02#show ip bgp summary BGP router identifier 10.11.11.32, local AS number 65511 BGP table version is 54, main routing table version 54 15 network entries using 3720 bytes of memory 15 path entries using 1800 bytes of memory 3/3 BGP path/bestpath attribute entries using 720 bytes of memory 3 BGP AS-PATH entries using 88 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 6328 total bytes of memory BGP activity 15/0 prefixes, 34/19 paths, scan interval 60 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd root ##y##10.11.11.253 4 65510 22 23 54 0 0 00:13:34 5 root ##y##10.100.21.2 4 65530 17 20 54 0 0 00:11:03 10 rt-a-02#
DC2
root ##bl##rt-b-01#show ip bgp summary rt-b-01#show ip bgp summary BGP router identifier 10.11.11.33, local AS number 65512 BGP table version is 48, main routing table version 48 15 network entries using 3720 bytes of memory 22 path entries using 2640 bytes of memory 3/3 BGP path/bestpath attribute entries using 720 bytes of memory 3 BGP AS-PATH entries using 88 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 7168 total bytes of memory BGP activity 15/0 prefixes, 38/16 paths, scan interval 60 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd root ##y##10.11.11.253 4 65510 22 24 48 0 0 00:13:39 12 root ##y##10.200.19.2 4 65530 19 20 48 0 0 00:12:11 10 rt-b-01#
root ##bl##rt-b-02#show ip bgp summary BGP router identifier 10.11.11.34, local AS number 65512 BGP table version is 28, main routing table version 28 15 network entries using 3720 bytes of memory 22 path entries using 2640 bytes of memory 3/3 BGP path/bestpath attribute entries using 720 bytes of memory 3 BGP AS-PATH entries using 88 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 7168 total bytes of memory BGP activity 15/0 prefixes, 38/16 paths, scan interval 60 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd root ##y##10.11.11.253 4 65510 21 21 28 0 0 00:13:29 12 root ##y##10.200.21.2 4 65530 19 21 28 0 0 00:12:42 10 rt-b-02#
Core route peering verification
root ##bl##cs-01#show ip bgp summary BGP router identifier 10.11.11.253, local AS number 65510 BGP table version is 402, main routing table version 402 15 network entries using 2040 bytes of memory 42 path entries using 2184 bytes of memory 3/3 BGP path/bestpath attribute entries using 372 bytes of memory 2 BGP AS-PATH entries using 48 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 4644 total bytes of memory BGP activity 63/48 prefixes, 644/602 paths, scan interval 60 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd root ##y##10.11.11.31 4 65511 27 25 402 0 0 00:16:20 10 root ##y##10.11.11.32 4 65511 25 24 402 0 0 00:15:20 10 root ##y##10.11.11.33 4 65512 25 23 402 0 0 00:14:43 10 root ##y##10.11.11.34 4 65512 21 22 402 0 0 00:14:02 10 cs-01#
UDLR routing tables
DC1
root ##bl##REGX-UDLR01-0> show ip route Codes: O - OSPF derived, i - IS-IS derived, B - BGP derived, C - connected, S - static, L1 - IS-IS level-1, L2 - IS-IS level-2, IA - OSPF inter area, E1 - OSPF external type 1, E2 - OSPF external type 2, N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 Total number of routes: 15 root ##y##B 10.11.11.0/24 [200/0] via 172.39.39.1 B 10.22.22.0/24 [200/0] via 172.39.39.1 B 10.100.19.0/24 [200/0] via 172.39.39.1 B 10.100.21.0/24 [200/0] via 172.39.39.1 B 10.200.19.0/24 [200/0] via 172.39.39.2 B 10.200.21.0/24 [200/0] via 172.39.39.2 root ##y##C 172.20.1.0/24 [0/0] via 172.20.1.254 root ##y##C 172.20.2.0/24 [0/0] via 172.20.2.254 root ##y##C 172.20.3.0/24 [0/0] via 172.20.3.254 C 172.20.8.0/24 [0/0] via 172.20.8.254 C 172.20.9.0/24 [0/0] via 172.20.9.254 C 172.20.10.0/24 [0/0] via 172.20.10.254 C 172.39.39.0/28 [0/0] via 172.39.39.13 B 172.39.39.16/28 [200/0] via 172.39.39.1 B 172.39.39.32/28 [200/0] via 172.39.39.2
DC2
The UDLR control VM does not exist in DC2.
ESG routing tables
DC1
root ##bl##REGA-ESG01-0> show ip route Codes: O - OSPF derived, i - IS-IS derived, B - BGP derived, C - connected, S - static, L1 - IS-IS level-1, L2 - IS-IS level-2, IA - OSPF inter area, E1 - OSPF external type 1, E2 - OSPF external type 2, N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 Total number of routes: 12 root ##y##B 10.11.11.0/24 [20/0] via 10.100.19.1 B 10.11.11.0/24 [20/0] via 10.100.21.1 B 10.22.22.0/24 [20/0] via 10.100.19.1 B 10.22.22.0/24 [20/0] via 10.100.21.1 C 10.100.19.0/24 [0/0] via 10.100.19.2 C 10.100.21.0/24 [0/0] via 10.100.21.2 root ##y##B 172.20.1.0/24 [200/0] via 172.39.39.14 root ##y##B 172.20.2.0/24 [200/0] via 172.39.39.14 root ##y##B 172.20.3.0/24 [200/0] via 172.39.39.14 B 172.20.8.0/24 [200/0] via 172.39.39.14 B 172.20.9.0/24 [200/0] via 172.39.39.14 B 172.20.10.0/24 [200/0] via 172.39.39.14 C 172.39.39.0/28 [0/0] via 172.39.39.1 C 172.39.39.16/28 [0/0] via 172.39.39.17
DC2
root ##bl##REGB-ESG01-0> show ip route Codes: O - OSPF derived, i - IS-IS derived, B - BGP derived, C - connected, S - static, L1 - IS-IS level-1, L2 - IS-IS level-2, IA - OSPF inter area, E1 - OSPF external type 1, E2 - OSPF external type 2, N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 Total number of routes: 12 root ##y##B 10.11.11.0/24 [20/0] via 10.200.19.1 B 10.11.11.0/24 [20/0] via 10.200.21.1 B 10.22.22.0/24 [20/0] via 10.200.19.1 B 10.22.22.0/24 [20/0] via 10.200.21.1 C 10.200.19.0/24 [0/0] via 10.200.19.2 C 10.200.21.0/24 [0/0] via 10.200.21.2 root ##y##B 172.20.1.0/24 [200/0] via 172.39.39.14 root ##y##B 172.20.2.0/24 [200/0] via 172.39.39.14 root ##y##B 172.20.3.0/24 [200/0] via 172.39.39.14 B 172.20.8.0/24 [200/0] via 172.39.39.14 B 172.20.9.0/24 [200/0] via 172.39.39.14 B 172.20.10.0/24 [200/0] via 172.39.39.14 C 172.39.39.0/28 [0/0] via 172.39.39.2 C 172.39.39.32/28 [0/0] via 172.39.39.33 REGB-ESG01-0>
Cisco1000V routing tables
DC1
root ##bl##rt-a-01#show ip route Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP a - application route + - replicated route, % - next hop override Gateway of last resort is not set 10.0.0.0/8 is variably subnetted, 10 subnets, 2 masks root ##y##C 10.11.11.0/24 is directly connected, GigabitEthernet2 L 10.11.11.31/32 is directly connected, GigabitEthernet2 B 10.22.22.0/24 [20/0] via 10.11.11.253, 00:17:32 C 10.100.1.0/24 is directly connected, GigabitEthernet1 L 10.100.1.101/32 is directly connected, GigabitEthernet1 C 10.100.19.0/24 is directly connected, GigabitEthernet3 L 10.100.19.1/32 is directly connected, GigabitEthernet3 B 10.100.21.0/24 [20/0] via 10.100.19.2, 00:14:45 S 10.200.19.0/24 [1/0] via 10.11.11.253 S 10.200.21.0/24 [1/0] via 10.11.11.253 172.20.0.0/24 is subnetted, 6 subnets root ##y##B 172.20.1.0 [20/0] via 10.100.19.2, 00:14:02 root ##y##B 172.20.2.0 [20/0] via 10.100.19.2, 00:14:02 root ##y##B 172.20.3.0 [20/0] via 10.100.19.2, 00:14:02 B 172.20.8.0 [20/0] via 10.100.19.2, 00:14:02 B 172.20.9.0 [20/0] via 10.100.19.2, 00:14:02 B 172.20.10.0 [20/0] via 10.100.19.2, 00:14:02 172.39.0.0/28 is subnetted, 3 subnets B 172.39.39.0 [20/0] via 10.100.19.2, 00:14:45 B 172.39.39.16 [20/0] via 10.100.19.2, 00:14:45 B 172.39.39.32 [20/0] via 10.11.11.253, 00:14:47 rt-a-01#
root ##bl##rt-a-02#show ip route Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP a - application route + - replicated route, % - next hop override Gateway of last resort is not set 10.0.0.0/8 is variably subnetted, 10 subnets, 2 masks root ##y##C 10.11.11.0/24 is directly connected, GigabitEthernet2 L 10.11.11.32/32 is directly connected, GigabitEthernet2 B 10.22.22.0/24 [20/0] via 10.11.11.253, 00:16:46 C 10.100.1.0/24 is directly connected, GigabitEthernet1 L 10.100.1.102/32 is directly connected, GigabitEthernet1 B 10.100.19.0/24 [20/0] via 10.100.21.2, 00:15:18 C 10.100.21.0/24 is directly connected, GigabitEthernet3 L 10.100.21.1/32 is directly connected, GigabitEthernet3 S 10.200.19.0/24 [1/0] via 10.11.11.253 S 10.200.21.0/24 [1/0] via 10.11.11.253 172.20.0.0/24 is subnetted, 6 subnets root ##y##B 172.20.1.0 [20/0] via 10.100.21.2, 00:14:35 root ##y##B 172.20.2.0 [20/0] via 10.100.21.2, 00:14:35 root ##y##B 172.20.3.0 [20/0] via 10.100.21.2, 00:14:35 B 172.20.8.0 [20/0] via 10.100.21.2, 00:14:35 B 172.20.9.0 [20/0] via 10.100.21.2, 00:14:35 B 172.20.10.0 [20/0] via 10.100.21.2, 00:14:35 172.39.0.0/28 is subnetted, 3 subnets B 172.39.39.0 [20/0] via 10.100.21.2, 00:15:18 B 172.39.39.16 [20/0] via 10.100.21.2, 00:15:18 B 172.39.39.32 [20/0] via 10.11.11.253, 00:15:21 rt-a-02#
DC2
root ##bl##rt-b-01#show ip route Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP a - application route + - replicated route, % - next hop override Gateway of last resort is not set 10.0.0.0/8 is variably subnetted, 10 subnets, 2 masks root ##y##C 10.11.11.0/24 is directly connected, GigabitEthernet2 L 10.11.11.33/32 is directly connected, GigabitEthernet2 B 10.22.22.0/24 [20/0] via 10.11.11.253, 00:16:51 S 10.100.19.0/24 [1/0] via 10.11.11.253 S 10.100.21.0/24 [1/0] via 10.11.11.253 C 10.200.1.0/24 is directly connected, GigabitEthernet1 L 10.200.1.101/32 is directly connected, GigabitEthernet1 C 10.200.19.0/24 is directly connected, GigabitEthernet3 L 10.200.19.1/32 is directly connected, GigabitEthernet3 B 10.200.21.0/24 [20/0] via 10.200.19.2, 00:16:14 172.20.0.0/24 is subnetted, 6 subnets root ##y##B 172.20.1.0 [20/0] via 10.200.19.2, 00:15:05 root ##y##B 172.20.2.0 [20/0] via 10.200.19.2, 00:15:05 root ##y##B 172.20.3.0 [20/0] via 10.200.19.2, 00:15:05 B 172.20.8.0 [20/0] via 10.200.19.2, 00:15:05 B 172.20.9.0 [20/0] via 10.200.19.2, 00:15:05 B 172.20.10.0 [20/0] via 10.200.19.2, 00:15:05 172.39.0.0/28 is subnetted, 3 subnets B 172.39.39.0 [20/0] via 10.200.19.2, 00:16:14 B 172.39.39.16 [20/0] via 10.11.11.253, 00:16:51 B 172.39.39.32 [20/0] via 10.200.19.2, 00:16:14 rt-b-01#
root ##bl##rt-b-02#show ip route Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP a - application route + - replicated route, % - next hop override Gateway of last resort is not set 10.0.0.0/8 is variably subnetted, 10 subnets, 2 masks root ##y##C 10.11.11.0/24 is directly connected, GigabitEthernet2 L 10.11.11.34/32 is directly connected, GigabitEthernet2 B 10.22.22.0/24 [20/0] via 10.11.11.253, 00:16:25 S 10.100.19.0/24 [1/0] via 10.11.11.253 S 10.100.21.0/24 [1/0] via 10.11.11.253 C 10.200.1.0/24 is directly connected, GigabitEthernet1 L 10.200.1.102/32 is directly connected, GigabitEthernet1 B 10.200.19.0/24 [20/0] via 10.200.21.2, 00:16:25 C 10.200.21.0/24 is directly connected, GigabitEthernet3 L 10.200.21.1/32 is directly connected, GigabitEthernet3 172.20.0.0/24 is subnetted, 6 subnets root ##y##B 172.20.1.0 [20/0] via 10.200.21.2, 00:15:40 root ##y##B 172.20.2.0 [20/0] via 10.200.21.2, 00:15:40 root ##y##B 172.20.3.0 [20/0] via 10.200.21.2, 00:15:40 B 172.20.8.0 [20/0] via 10.200.21.2, 00:15:40 B 172.20.9.0 [20/0] via 10.200.21.2, 00:15:40 B 172.20.10.0 [20/0] via 10.200.21.2, 00:15:40 172.39.0.0/28 is subnetted, 3 subnets B 172.39.39.0 [20/0] via 10.200.21.2, 00:16:25 B 172.39.39.16 [20/0] via 10.11.11.253, 00:16:25 B 172.39.39.32 [20/0] via 10.200.21.2, 00:16:25 rt-b-02#
Core routing table
root ##bl##cs-01#show ip route bgp Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP + - replicated route, % - next hop override Gateway of last resort is 10.11.11.254 to network 0.0.0.0 172.20.0.0/24 is subnetted, 6 subnets root ##y##B 172.20.1.0 [20/0] via 10.11.11.31, 00:16:19 root ##y##B 172.20.2.0 [20/0] via 10.11.11.31, 00:16:19 root ##y##B 172.20.3.0 [20/0] via 10.11.11.31, 00:16:19 B 172.20.8.0 [20/0] via 10.11.11.31, 00:16:19 B 172.20.9.0 [20/0] via 10.11.11.31, 00:16:19 B 172.20.10.0 [20/0] via 10.11.11.31, 00:16:19 172.39.0.0/28 is subnetted, 3 subnets B 172.39.39.0 [20/0] via 10.11.11.32, 00:16:50 B 172.39.39.16 [20/0] via 10.11.11.32, 00:16:50 B 172.39.39.32 [20/0] via 10.11.11.34, 00:16:51 cs-01#
UDLR BGP tables
DC1
root ##bl##REGX-UDLR01-0> show ip bgp Status codes: s - suppressed, d - damped, > - best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight AS Path root ##y## > 10.11.11.0/24 10.100.19.1 0 100 200 65511 65510 i root ##y## 10.11.11.0/24 10.200.19.1 0 100 30 65512 65510 i > 10.22.22.0/24 10.100.19.1 0 100 200 65511 65510 i 10.22.22.0/24 10.200.19.1 0 100 30 65512 65510 i > 10.100.19.0/24 172.39.39.1 0 100 200 ? > 10.100.21.0/24 172.39.39.1 0 100 200 ? > 10.200.19.0/24 172.39.39.2 0 100 30 ? > 10.200.21.0/24 172.39.39.2 0 100 30 ? root ##y## > 172.20.1.0/24 0.0.0.0 0 100 32768 ? root ##y## > 172.20.2.0/24 0.0.0.0 0 100 32768 ? root ##y## > 172.20.3.0/24 0.0.0.0 0 100 32768 ? > 172.20.8.0/24 0.0.0.0 0 100 32768 ? > 172.20.9.0/24 0.0.0.0 0 100 32768 ? > 172.20.10.0/24 0.0.0.0 0 100 32768 ? 172.39.39.0/28 172.39.39.1 0 100 200 ? 172.39.39.0/28 172.39.39.2 0 100 30 ? > 172.39.39.0/28 0.0.0.0 0 100 32768 ? > 172.39.39.16/28 172.39.39.1 0 100 200 ? > 172.39.39.32/28 172.39.39.2 0 100 30 ? REGX-UDLR01-0>
DC2
The UDLR control VM does not exist in DC2.
ESG BGP tables
DC1
root ##bl##REGA-ESG01-0> show ip bgp Status codes: s - suppressed, d - damped, > - best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight AS Path root ##y## > 10.11.11.0/24 10.100.19.1 0 100 60 65511 65510 i root ##y## 10.11.11.0/24 10.100.21.1 0 100 60 65511 65510 i > 10.22.22.0/24 10.100.19.1 0 100 60 65511 65510 i 10.22.22.0/24 10.100.21.1 0 100 60 65511 65510 i > 10.100.19.0/24 0.0.0.0 0 100 32768 ? > 10.100.21.0/24 0.0.0.0 0 100 32768 ? root ##y## > 172.20.1.0/24 172.39.39.14 0 100 60 ? root ##y## > 172.20.2.0/24 172.39.39.14 0 100 60 ? root ##y## > 172.20.3.0/24 172.39.39.14 0 100 60 ? > 172.20.8.0/24 172.39.39.14 0 100 60 ? > 172.20.9.0/24 172.39.39.14 0 100 60 ? > 172.20.10.0/24 172.39.39.14 0 100 60 ? 172.39.39.0/28 172.39.39.14 0 100 60 ? > 172.39.39.0/28 0.0.0.0 0 100 32768 ? > 172.39.39.16/28 0.0.0.0 0 100 32768 ? REGA-ESG01-0>
DC2
root ##bl##REGB-ESG01-0> show ip bgp Status codes: s - suppressed, d - damped, > - best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight AS Path root ##y## > 10.11.11.0/24 10.200.19.1 0 100 60 65512 65510 i root ##y## 10.11.11.0/24 10.200.21.1 0 100 60 65512 65510 i > 10.22.22.0/24 10.200.19.1 0 100 60 65512 65510 i 10.22.22.0/24 10.200.21.1 0 100 60 65512 65510 i > 10.200.19.0/24 0.0.0.0 0 100 32768 ? > 10.200.21.0/24 0.0.0.0 0 100 32768 ? root ##y## > 172.20.1.0/24 172.39.39.14 0 100 60 ? root ##y## > 172.20.2.0/24 172.39.39.14 0 100 60 ? root ##y## > 172.20.3.0/24 172.39.39.14 0 100 60 ? > 172.20.8.0/24 172.39.39.14 0 100 60 ? > 172.20.9.0/24 172.39.39.14 0 100 60 ? > 172.20.10.0/24 172.39.39.14 0 100 60 ? 172.39.39.0/28 172.39.39.14 0 100 60 ? > 172.39.39.0/28 0.0.0.0 0 100 32768 ? > 172.39.39.32/28 0.0.0.0 0 100 32768 ? REGB-ESG01-0>
Cisco1000V BGP tables
DC1
root ##bl##rt-a-01#show ip bgp BGP table version is 54, local router ID is 10.11.11.31 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path root ##y## r> 10.11.11.0/24 10.11.11.253 0 0 65510 i *> 10.22.22.0/24 10.11.11.253 0 0 65510 i r> 10.100.19.0/24 10.100.19.2 0 65530 ? *> 10.100.21.0/24 10.100.19.2 0 65530 ? r> 10.200.19.0/24 10.11.11.253 0 65510 65512 65530 ? r> 10.200.21.0/24 10.11.11.253 0 65510 65512 65530 ? root ##y## *> 172.20.1.0/24 10.100.19.2 0 65530 ? root ##y## *> 172.20.2.0/24 10.100.19.2 0 65530 ? root ##y## *> 172.20.3.0/24 10.100.19.2 0 65530 ? *> 172.20.8.0/24 10.100.19.2 0 65530 ? *> 172.20.9.0/24 10.100.19.2 0 65530 ? *> 172.20.10.0/24 10.100.19.2 0 65530 ? *> 172.39.39.0/28 10.100.19.2 0 65530 ? Network Next Hop Metric LocPrf Weight Path *> 172.39.39.16/28 10.100.19.2 0 65530 ? *> 172.39.39.32/28 10.11.11.253 0 65510 65512 65530 ? rt-a-01#
root ##bl##rt-a-02#show ip bgp BGP table version is 54, local router ID is 10.11.11.32 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path root ##y## r> 10.11.11.0/24 10.11.11.253 0 0 65510 i *> 10.22.22.0/24 10.11.11.253 0 0 65510 i *> 10.100.19.0/24 10.100.21.2 0 65530 ? r> 10.100.21.0/24 10.100.21.2 0 65530 ? r> 10.200.19.0/24 10.11.11.253 0 65510 65512 65530 ? r> 10.200.21.0/24 10.11.11.253 0 65510 65512 65530 ? root ##y## *> 172.20.1.0/24 10.100.21.2 0 65530 ? root ##y## *> 172.20.2.0/24 10.100.21.2 0 65530 ? root ##y## *> 172.20.3.0/24 10.100.21.2 0 65530 ? *> 172.20.8.0/24 10.100.21.2 0 65530 ? *> 172.20.9.0/24 10.100.21.2 0 65530 ? *> 172.20.10.0/24 10.100.21.2 0 65530 ? *> 172.39.39.0/28 10.100.21.2 0 65530 ? Network Next Hop Metric LocPrf Weight Path *> 172.39.39.16/28 10.100.21.2 0 65530 ? *> 172.39.39.32/28 10.11.11.253 0 65510 65512 65530 ? rt-a-02#
DC2
root ##bl##rt-b-01#show ip bgp BGP table version is 48, local router ID is 10.11.11.33 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path root ##y## r> 10.11.11.0/24 10.11.11.253 0 0 65510 i *> 10.22.22.0/24 10.11.11.253 0 0 65510 i r> 10.100.19.0/24 10.11.11.253 0 65510 65511 65530 ? r> 10.100.21.0/24 10.11.11.253 0 65510 65511 65530 ? r> 10.200.19.0/24 10.200.19.2 0 65530 ? *> 10.200.21.0/24 10.200.19.2 0 65530 ? *> 172.20.1.0/24 10.200.19.2 0 65530 ? * 10.11.11.253 0 65510 65511 65530 ? *> 172.20.2.0/24 10.200.19.2 0 65530 ? * 10.11.11.253 0 65510 65511 65530 ? Network Next Hop Metric LocPrf Weight Path root ##y## *> 172.20.3.0/24 10.200.19.2 0 65530 ? root ##y## * 10.11.11.253 0 65510 65511 65530 ? *> 172.20.8.0/24 10.200.19.2 0 65530 ? * 10.11.11.253 0 65510 65511 65530 ? *> 172.20.9.0/24 10.200.19.2 0 65530 ? * 10.11.11.253 0 65510 65511 65530 ? *> 172.20.10.0/24 10.200.19.2 0 65530 ? * 10.11.11.253 0 65510 65511 65530 ? *> 172.39.39.0/28 10.200.19.2 0 65530 ? * 10.11.11.253 0 65510 65511 65530 ? *> 172.39.39.16/28 10.11.11.253 0 65510 65511 65530 ? *> 172.39.39.32/28 10.200.19.2 0 65530 ? rt-b-01#
root ##bl##rt-b-02#show ip bgp rt-b-02#show ip bgp BGP table version is 28, local router ID is 10.11.11.34 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path root ##y## r> 10.11.11.0/24 10.11.11.253 0 0 65510 i *> 10.22.22.0/24 10.11.11.253 0 0 65510 i r> 10.100.19.0/24 10.11.11.253 0 65510 65511 65530 ? r> 10.100.21.0/24 10.11.11.253 0 65510 65511 65530 ? *> 10.200.19.0/24 10.200.21.2 0 65530 ? r> 10.200.21.0/24 10.200.21.2 0 65530 ? root ##y## *> 172.20.1.0/24 10.200.21.2 0 65530 ? root ##y## * 10.11.11.253 0 65510 65511 65530 ? root ##y## *> 172.20.2.0/24 10.200.21.2 0 65530 ? root ##y## * 10.11.11.253 0 65510 65511 65530 ? Network Next Hop Metric LocPrf Weight Path root ##y## *> 172.20.3.0/24 10.200.21.2 0 65530 ? root ##y## * 10.11.11.253 0 65510 65511 65530 ? *> 172.20.8.0/24 10.200.21.2 0 65530 ? * 10.11.11.253 0 65510 65511 65530 ? *> 172.20.9.0/24 10.200.21.2 0 65530 ? * 10.11.11.253 0 65510 65511 65530 ? *> 172.20.10.0/24 10.200.21.2 0 65530 ? * 10.11.11.253 0 65510 65511 65530 ? *> 172.39.39.0/28 10.200.21.2 0 65530 ? * 10.11.11.253 0 65510 65511 65530 ? *> 172.39.39.16/28 10.11.11.253 0 65510 65511 65530 ? *> 172.39.39.32/28 10.200.21.2 0 65530 ? rt-b-02#
Core BGP tables
root ##bl##cs-01#show ip bgp BGP table version is 402, local router ID is 10.11.11.253 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, x best-external Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path root ##y##*> 10.11.11.0/24 0.0.0.0 0 32768 i *> 10.22.22.0/24 0.0.0.0 0 32768 i r 10.100.19.0/24 10.11.11.31 0 65511 65530 ? r> 10.11.11.32 0 65511 65530 ? r 10.100.21.0/24 10.11.11.31 0 65511 65530 ? r> 10.11.11.32 0 65511 65530 ? r 10.200.19.0/24 10.11.11.33 0 65512 65530 ? r> 10.11.11.34 0 65512 65530 ? r 10.200.21.0/24 10.11.11.33 0 65512 65530 ? r> 10.11.11.34 0 65512 65530 ? root ##y##* 172.20.1.0/24 10.11.11.32 0 65511 65530 ? root ##y##* 10.11.11.33 0 65512 65530 ? root ##y##* 10.11.11.34 0 65512 65530 ? root ##y##*> 10.11.11.31 0 65511 65530 ? root ##y##* 172.20.2.0/24 10.11.11.32 0 65511 65530 ? root ##y##* 10.11.11.33 0 65512 65530 ? root ##y##* 10.11.11.34 0 65512 65530 ? root ##y##*> 10.11.11.31 0 65511 65530 ? root ##y##* 172.20.3.0/24 10.11.11.32 0 65511 65530 ? root ##y##* 10.11.11.33 0 65512 65530 ? root ##y##* 10.11.11.34 0 65512 65530 ? root ##y##*> 10.11.11.31 0 65511 65530 ? * 172.20.8.0/24 10.11.11.32 0 65511 65530 ? * 10.11.11.33 0 65512 65530 ? * 10.11.11.34 0 65512 65530 ? *> 10.11.11.31 0 65511 65530 ? * 172.20.9.0/24 10.11.11.32 0 65511 65530 ? * 10.11.11.33 0 65512 65530 ? * 10.11.11.34 0 65512 65530 ? *> 10.11.11.31 0 65511 65530 ? * 172.20.10.0/24 10.11.11.32 0 65511 65530 ? * 10.11.11.33 0 65512 65530 ? * 10.11.11.34 0 65512 65530 ? *> 10.11.11.31 0 65511 65530 ? * 172.39.39.0/28 10.11.11.31 0 65511 65530 ? * 10.11.11.33 0 65512 65530 ? * 10.11.11.34 0 65512 65530 ? *> 10.11.11.32 0 65511 65530 ? * 172.39.39.16/28 10.11.11.31 0 65511 65530 ? *> 10.11.11.32 0 65511 65530 ? * 172.39.39.32/28 10.11.11.33 0 65512 65530 ? *> 10.11.11.34 0 65512 65530 ? cs-01#
Routing path verifications
Routing path verification from the external client VM to T1–WEB–1 and T1–WEB–2
The expectation is that the traffic will route through DC1.
root ##bl##C:\Users\Administrator>tracert 172.20.1.1 Tracing route to 172.20.1.1 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.31 root ##y## 3 <1 ms <1 ms <1 ms 10.100.19.2 4 1 ms <1 ms <1 ms 172.39.39.14 5 2 ms <1 ms <1 ms 172.20.1.1 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.2 Tracing route to 172.20.1.2 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.31 root ##y## 3 <1 ms <1 ms <1 ms 10.100.19.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 6 ms <1 ms <1 ms 172.20.1.2 Trace complete.
Routing path verification from the T1–WEB–1 and T1–WEB–2 to the external client VM
root ##bl##root@Web01:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.184 ms 0.060 ms 0.149 ms 2 172.39.39.1 (172.39.39.1) 0.454 ms 0.408 ms 0.438 ms root ##y## 3 10.100.19.1 (10.100.19.1) 1.412 ms 1.450 ms 1.434 ms 4 10.11.11.50 (10.11.11.50) 1.454 ms * * root@Web01:~#
root ##bl##root@Web02:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.132 ms 0.105 ms 0.082 ms 2 172.39.39.1 (172.39.39.1) 0.922 ms 0.880 ms 0.771 ms root ##y## 3 10.100.19.1 (10.100.19.1) 1.110 ms 1.716 ms 1.704 ms 4 10.11.11.50 (10.11.11.50) 2.738 ms * *
We are seeing that the ingress AND egress paths that are taken is the path trough DC1. And this was expected...
Routing path verification from the external client VM to T1–WEB–3 and T1–WEB–4
The expectation is that the traffic will route through DC1.
root ##bl##C:\Users\Administrator>tracert 172.20.1.3 Tracing route to 172.20.1.3 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.31 root ##y## 3 <1 ms <1 ms <1 ms 10.100.21.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 3 ms 1 ms <1 ms 172.20.1.3 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.4 Tracing route to 172.20.1.4 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.31 root ##y## 3 <1 ms <1 ms <1 ms 10.100.19.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 6 ms 1 ms <1 ms 172.20.1.4 Trace complete.
Routing path verification from the T1–WEB–3 and T1–WEB–4 to the external client VM
The expectation is that the traffic will route through DC1.
root ##bl##root@Web03:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.133 ms 0.257 ms 0.137 ms 2 172.39.39.1 (172.39.39.1) 0.510 ms 0.460 ms 0.601 ms root ##y## 3 10.100.21.1 (10.100.21.1) 1.298 ms 1.319 ms 1.266 ms 4 10.11.11.50 (10.11.11.50) 1.380 ms * *
root ##bl##root@Web04:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.145 ms 0.101 ms 0.094 ms 2 172.39.39.1 (172.39.39.1) 0.557 ms 0.631 ms 0.818 ms root ##y## 3 10.100.19.1 (10.100.19.1) 1.287 ms 1.290 ms 1.315 ms 4 10.11.11.50 (10.11.11.50) 1.393 ms * *
The routing components in DC1 may fail or in case of a disaster the full site may go down. In that case, the routing should flow through DC2.
Let’s test this…
Turn RT-A-01 off. Verify paths again. The expectation is that the traffic will route through RT-A-02.
Routing path verification from the external client VM to T1–WEB–1, T1–WEB–2, T1–WEB–3 and T1–WEB–4
root ##bl##C:\Users\Administrator>tracert 172.20.1.1 Tracing route to 172.20.1.1 over a maximum of 30 hops 1 3 ms 1 ms 1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.32 root ##y## 3 <1 ms <1 ms <1 ms 10.100.21.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 3 ms <1 ms <1 ms 172.20.1.1 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.2 Tracing route to 172.20.1.2 over a maximum of 30 hops 1 1 ms 1 ms 4 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.32 root ##y## 3 <1 ms <1 ms <1 ms 10.100.21.2 4 <1 ms 1 ms <1 ms 172.39.39.14 5 3 ms <1 ms <1 ms 172.20.1.2 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.3 Tracing route to 172.20.1.3 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] root ##y## 3 <1 ms <1 ms <1 ms 10.100.21.2 4 1 ms <1 ms <1 ms 172.39.39.14 5 2 ms <1 ms <1 ms 172.20.1.3
root ##bl##C:\Users\Administrator>tracert 172.20.1.4 Tracing route to 172.20.1.4 over a maximum of 30 hops 1 1 ms 1 ms 1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.32 root ##y## 3 <1 ms <1 ms <1 ms 10.100.21.2 4 1 ms <1 ms <1 ms 172.39.39.14 5 1 ms 1 ms <1 ms 172.20.1.4 Trace complete.
Routing path verification from the T1–WEB–1, T1–WEB–2, T1–WEB–3 and T1–WEB–4 to the external client VM
root ##bl##root@Web01:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.273 ms 0.157 ms 0.323 ms 2 172.39.39.1 (172.39.39.1) 0.314 ms 0.248 ms 0.575 ms root ##y## 3 10.100.21.1 (10.100.21.1) 0.608 ms 0.594 ms 0.550 ms 4 10.11.11.50 (10.11.11.50) 0.596 ms * *
root ##bl##root@Web02:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.189 ms 0.090 ms 0.097 ms 2 172.39.39.1 (172.39.39.1) 0.408 ms 0.361 ms 0.441 ms root ##y## 3 10.100.21.1 (10.100.21.1) 0.668 ms 0.636 ms 0.774 ms 4 10.11.11.50 (10.11.11.50) 0.733 ms * *
root ##bl##root@Web03:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.160 ms 0.144 ms 0.124 ms 2 172.39.39.1 (172.39.39.1) 0.512 ms 0.500 ms 0.478 ms root ##y## 3 10.100.21.1 (10.100.21.1) 1.095 ms 1.115 ms 1.153 ms 4 10.11.11.50 (10.11.11.50) 1.907 ms * *
root ##bl##root@Web04:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.155 ms 0.144 ms 0.109 ms 2 172.39.39.1 (172.39.39.1) 4.136 ms 4.021 ms 3.921 ms root ##y## 3 10.100.21.1 (10.100.21.1) 6.588 ms 6.605 ms 6.631 ms 4 10.11.11.50 (10.11.11.50) 6.878 ms * *
Turn RT-A-02 off. Verify paths again. The expectation is that the traffic will route through DC2 as both upstream routers in DC1 are down.
Routing path verification from the external client VM to T1–WEB–1 , T1–WEB–2, T1–WEB–3 and T1–WEB–4
root ##bl##C:\Users\Administrator>tracert 172.20.1.1 Tracing route to 172.20.1.1 over a maximum of 30 hops 1 <1 ms 1 ms 2 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.33 root ##y## 3 <1 ms <1 ms <1 ms 10.200.19.2 4 1 ms <1 ms <1 ms 172.39.39.14 5 75 ms 1 ms 1 ms 172.20.1.1 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.2 Tracing route to 172.20.1.2 over a maximum of 30 hops 1 3 ms 1 ms 1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.33 root ##y## 3 1 ms <1 ms <1 ms 10.200.19.2 4 1 ms <1 ms <1 ms 172.39.39.14 5 4 ms 1 ms <1 ms 172.20.1.2 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.3 Tracing route to 172.20.1.3 over a maximum of 30 hops 1 4 ms 1 ms 1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.33 root ##y## 3 <1 ms <1 ms <1 ms 10.200.19.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 1 ms <1 ms <1 ms 172.20.1.3 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.4 Tracing route to 172.20.1.4 over a maximum of 30 hops 1 2 ms 1 ms 1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.33 root ##y## 3 1 ms <1 ms <1 ms 10.200.19.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 3 ms 1 ms <1 ms 172.20.1.4 Trace complete.
Routing path verification from the T1–WEB–1, T1–WEB–2, T1–WEB–3 and T1–WEB–4 to the external client VM
root ##bl##root@Web01:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.280 ms 0.217 ms 0.209 ms 2 172.39.39.2 (172.39.39.2) 0.646 ms 0.787 ms 0.767 ms root ##y## 3 10.200.19.1 (10.200.19.1) 1.025 ms 1.678 ms 1.604 ms 4 10.11.11.50 (10.11.11.50) 1.577 ms * *
root ##bl##root@Web02:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.135 ms 0.137 ms 0.056 ms 2 172.39.39.2 (172.39.39.2) 0.669 ms 0.651 ms 0.727 ms root ##y## 3 10.200.19.1 (10.200.19.1) 2.106 ms 2.067 ms 2.061 ms 4 10.11.11.50 (10.11.11.50) 2.195 ms * *
root ##bl##root@Web03:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.185 ms 0.150 ms 0.179 ms 2 172.39.39.2 (172.39.39.2) 0.267 ms 0.285 ms 0.209 ms root ##y## 3 10.200.19.1 (10.200.19.1) 0.871 ms 0.707 ms 0.746 ms 4 10.11.11.50 (10.11.11.50) 0.850 ms * *
root ##bl##root@Web04:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.119 ms 0.077 ms 0.079 ms 2 172.39.39.2 (172.39.39.2) 0.322 ms 0.382 ms 0.447 ms root ##y## 3 10.200.19.1 (10.200.19.1) 1.584 ms 1.540 ms 1.495 ms 4 10.11.11.50 (10.11.11.50) 1.479 ms * *
Turn RT-A-01 and RT-A-02 back on and turn ESG-A off. Verify paths again. The expectation is that the traffic will route through DC2 the only ESG in DC1 is down.
Before I turned off ESG-A I have verified if the traffic was flowing through DC1 again as an extra verification step. This is not shown below. For some reason the traffic kept flowing through DC1 so I had to turn BGP off/on again on the ESG in DC2 and the this caused the traffic to flow back trough DC1.
Routing path verification from the external client VM to T1–WEB–1 , T1–WEB–2, T1–WEB–3 and T1–WEB–4
With the ESG-A turned off...
root ##bl##C:\Users\Administrator>tracert 172.20.1.1 Tracing route to 172.20.1.1 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.33 root ##y## 3 <1 ms <1 ms <1 ms 10.200.19.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 254 ms 1 ms 1 ms 172.20.1.1 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.2 Tracing route to 172.20.1.2 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.33 root ##y## 3 <1 ms <1 ms <1 ms 10.200.19.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 1 ms 1 ms <1 ms 172.20.1.2 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.3 Tracing route to 172.20.1.3 over a maximum of 30 hops 1 3 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.33 root ##y## 3 <1 ms <1 ms <1 ms 10.200.19.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 1 ms <1 ms <1 ms 172.20.1.3 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.4 Tracing route to 172.20.1.4 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.33 root ##y## 3 <1 ms <1 ms <1 ms 10.200.19.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 5 ms <1 ms <1 ms 172.20.1.4 Trace complete.
Routing path verification from the T1–WEB–1, T1–WEB–2, T1–WEB–3 and T1–WEB–4 to the external client VM
root ##bl##root@Web01:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.154 ms 0.115 ms 0.102 ms 2 172.39.39.2 (172.39.39.2) 0.665 ms 0.660 ms 0.637 ms root ##y## 3 10.200.19.1 (10.200.19.1) 0.969 ms 0.998 ms 0.977 ms 4 10.11.11.50 (10.11.11.50) 1.196 ms * *
root ##bl##root@Web02:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 1.620 ms 1.536 ms 1.488 ms 2 172.39.39.2 (172.39.39.2) 0.452 ms 0.502 ms 0.510 ms root ##y## 3 10.200.19.1 (10.200.19.1) 0.763 ms 0.718 ms 0.686 ms 4 10.11.11.50 (10.11.11.50) 0.993 ms * *
root ##bl##root@Web03:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.171 ms 0.131 ms 0.143 ms 2 172.39.39.2 (172.39.39.2) 0.326 ms 0.362 ms 0.483 ms root ##y## 3 10.200.19.1 (10.200.19.1) 1.043 ms 1.063 ms 0.974 ms 4 10.11.11.50 (10.11.11.50) 1.055 ms * *
root ##bl##root@Web04:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.131 ms 3.804 ms 3.799 ms 2 172.39.39.2 (172.39.39.2) 0.443 ms 0.319 ms 0.333 ms root ##y## 3 10.200.19.1 (10.200.19.1) 0.575 ms 0.508 ms 0.772 ms 4 10.11.11.50 (10.11.11.50) 0.782 ms * *
Turn ESG-A back on. Verify paths again. The expectation is that everything is back to normal now.
For some reason after turning on the ESG on DC1 the routes kept flowing through DC2. The same “issue” we saw before. So, I turned BGP off/on again on the ESG in DC2. This is not shown below. Below you will see the outputs after the BGP “reset” on DC2.
Routing path verification from the external client VM to T1–WEB–1 , T1–WEB–2, T1–WEB–3 and T1–WEB–4
root ##bl##C:\Users\Administrator>tracert 172.20.1.1 Tracing route to 172.20.1.1 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.31 root ##y## 3 <1 ms <1 ms <1 ms 10.100.21.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 1 ms <1 ms <1 ms 172.20.1.1 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.2 Tracing route to 172.20.1.2 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.31 root ##y## 3 <1 ms <1 ms <1 ms 10.100.21.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 4 ms <1 ms <1 ms 172.20.1.2 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.3 Tracing route to 172.20.1.3 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.31 root ##y## 3 <1 ms <1 ms <1 ms 10.100.21.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 2 ms 1 ms <1 ms 172.20.1.3 Trace complete.
root ##bl##C:\Users\Administrator>tracert 172.20.1.4 Tracing route to 172.20.1.4 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms cs-01.home.local [10.11.11.253] 2 <1 ms <1 ms <1 ms 10.11.11.31 root ##y## 3 <1 ms <1 ms <1 ms 10.100.19.2 4 <1 ms <1 ms <1 ms 172.39.39.14 5 2 ms <1 ms <1 ms 172.20.1.4 Trace complete.
Routing path verification from the T1–WEB–1, T1–WEB–2, T1–WEB–3 and T1–WEB–4 to the external client VM
root ##bl##root@Web01:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.163 ms 0.203 ms 0.123 ms 2 172.39.39.1 (172.39.39.1) 0.214 ms 0.255 ms 0.266 ms root ##y## 3 10.100.21.1 (10.100.21.1) 0.647 ms 0.558 ms 0.401 ms 4 10.11.11.50 (10.11.11.50) 0.798 ms * *
root ##bl##root@Web02:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.183 ms 0.117 ms 0.119 ms 2 172.39.39.1 (172.39.39.1) 0.290 ms 0.241 ms 0.319 ms root ##y## 3 10.100.21.1 (10.100.21.1) 0.609 ms 0.555 ms 0.602 ms 4 10.11.11.50 (10.11.11.50) 0.737 ms * *
root ##bl##root@Web03:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.246 ms 0.102 ms 0.065 ms 2 172.39.39.1 (172.39.39.1) 0.465 ms 0.521 ms 0.568 ms root ##y## 3 10.100.21.1 (10.100.21.1) 0.686 ms 0.840 ms 0.812 ms 4 10.11.11.50 (10.11.11.50) 0.983 ms * *
root ##bl##root@Web02:~# traceroute 10.11.11.50 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets 1 172.20.1.254 (172.20.1.254) 0.165 ms 0.177 ms 0.211 ms 2 172.39.39.1 (172.39.39.1) 0.433 ms 0.500 ms 0.549 ms root ##y## 3 10.100.19.1 (10.100.19.1) 0.860 ms 0.956 ms 0.949 ms 4 10.11.11.50 (10.11.11.50) 1.073 ms * *
Use–Cases
The end conclusion is that routing with this topology (with iBGP between the DLR and the ESG) works fine. When DC1 becomes unavailable DC2 will take over. For some reason when DC1 is available again the switch-over does not happen automatically, and we need to "kick" it by disabling BGP on the ESG-B.