Routing with NSX using multiple sites using Locale ID (with eBGP all the way) and Subnet and IP address planning and registration using PHPIPAM: Difference between pages

From Iwan
(Difference between pages)
Jump to: navigation, search
(Import pages from iwan.wiki)
 
No edit summary
 
Line 1: Line 1:
Deploying an automated nested lab with the size and scale I am doing, requires proper IP address management and documentation.
Typically I would do this using excel sheets.
There are much smarter ways to do IP address management and there are paid solutions so-called IPAM that also offer other services like DNS for example.


While I was upgrading my lab and perform management I remembered that I have been using another web-based tool in the past called PHPIPAM.
The code can be downloaded [https://sourceforge.net/projects/phpipam/ here].


Routing with NSX using multiple sites using Locale ID (with eBGP all the way)
And the demo can be found [https://demo.phpipam.net/login/ here].


== Introduction ==
I decided to re-install and registered all my IP addresses including the ones I am using for the nested automated labs.


In one of my previous articles I demonstrated the routing between two Data Centers with using eBGP and iBGP.
Below you will see a screenshot of the page where I registered all subnets I used:
These previous articles focussed on using only one path northbound as the primary exit point.
So workloads that are in DC1 and DC2 will either choose the DC1 WAN or the DC2 WAN as the primary path (where DC1 is always prefered and DC2 is always secondary)


This article will be about doing almost the same, but now with the use of the NSX Locale ID feature.
[[File:screenshot_1550.png|1000px]]
With this feature the workloads that are in DC1 will exit trough the DC1 WAN and the workloads that are in DC2 will exit trough the DC2 WAN.


== Diagram ==
As an example I also included the registered IP addresses in the "LX-APP-MGMT-11" segment.


The following diagram will be used for our setup.
[[File:screenshot_1551.png|1000px]]


[[File:LAB100_-_NSX_Routing_based_on_eBGP_and_locale_ID.pdf]]
[[Category:Articles]]
 
[[Category:VMware]]
[[File:nsx-routing-blog-locale-1.png|600px]]
[[Category:NSX-T]]
 
[[Category:Networking]]
== Components ==
 
I am still using the same componenents as in this previous article but in this article we will actively use the UDLR-02 and leave the UDLR-1 alone.
 
== Use-Cases ==
 
I have thought of the following use-cases below.
 
As it is not possible to test active/passive and active/active with one UDLR I am using two UDLR instances here.
 
For testing purpose, I will use three tenants with each having their own Web, App and DB tiers.
 
Tenant one and two will be using the first UDLR (UDLR-01) with active/passive site egress. Tenant three will be using the second UDLR (UDLR-02) with active/active site egress.
 
=== Tenant three ===
 
Tenant three will have workloads in DC1 and in DC2 and because it is using local egress the traffic will exit from the Cisco CSR1000V routers local to that site. When the primary site is down the traffic should exit from the secondary site and when the secondary is down traffic should exit from the primary.
 
== Routing protocol options ==
 
All possible routing options are described in the previous article.
 
The routing option in this article will be:
 
Option 1
# eBGP peering between the UDLR and the ESG’s
# eBGP peering between the ESG’s and the external routers
# eBGP peering between the external routers and the CORE
 
== Option 1 ==
 
Because of the many options available in the setup I want to start with the following:
Tenant 3 workloads with the use of UDLR-02 and option 1 routing.
 
=== UDLR configuration ===
 
When Locale ID is used the UDLR Control VM needs to be deployed in both DC's.
After the deployment is done on the primary site we need to browse to the secondary NSX Manager and manually deploy an instance of the UDLR control VM in DC2.
When this is done some DC2 site specific configurations are needed.
 
# DC1 and DC2 UDLR-02 instances will have their own interfaces with own IP addresses
# DC1 and DC2 UDLR-02 instances will have their own eBGP peerings
# DC1 and DC2 UDLR-02 instances will have their own site (or NSX Manager) specific Locale ID
 
==== DC1 ====
 
Manage --> Firewall
 
# Make sure the firewall is disabled
 
[[File:nsx-routing-blog-locale-2.png|600px]]
 
# Get the Locale ID from the NSX Manager
 
[[File:nsx-routing-blog-locale-3.png|600px]]
 
Manage --> Routing --> Global Configuration
 
# Configure Locale ID and make sure this is the same as the NSX Manager
# Enable ECMP
# Make sure that the default gateway is not configured
# Configure a router ID
 
[[File:nsx-routing-blog-locale-4.png|600px]]
 
Manage --> Routing --> BGP
 
# Enable BGP
# Configure the Local AS
# Disable Graceful Restart
# Configure the BGP peers towards ESG-A
 
[[File:nsx-routing-blog-locale-5.png|600px]]
 
Manage --> Routing --> Route Redistribution
 
# Enable route redistribution for BGP
# Specify what you allow to redistribute
 
[[File:nsx-routing-blog-locale-6.png|600px]]
 
==== DC2 ====
 
# Make sure the firewall is disabled
 
[[File:nsx-routing-blog-locale-7.png|600px]]
 
# Get the Locale ID from the NSX Manager
 
[[File:nsx-routing-blog-locale-8.png|600px]]
 
Manage --> Routing --> Global Configuration
 
# Configure Locale ID and make sure this is the same as the NSX Manager
# Enable ECMP
# Make sure that the default gateway is not configured
# Configure a router ID
 
[[File:nsx-routing-blog-locale-9.png|600px]]
 
Manage --> Routing --> BGP
 
# Enable BGP
# Configure the Local AS
# Disable Graceful Restart
# Configure the BGP peers towards ESG-A
 
[[File:nsx-routing-blog-locale-10.png|600px]]
 
Manage --> Routing --> Route Redistribution
 
# Enable route redistribution for BGP
# Specify what you allow to redistribute
 
[[File:nsx-routing-blog-locale-11.png|600px]]
 
=== Compute Cluster configuration ===
 
The same locale ID (per site) also need to be configured on the Compute Clusters.
We need to verify this if this happened automatically and if this is not the case we need to configure this manually.
 
==== DC1 ====
 
[[File:nsx-routing-blog-locale-12.png|600px]]
 
==== DC2 ====
 
[[File:nsx-routing-blog-locale-13.png|600px]]
 
=== ESG configuration ===
 
==== DC1 ====
 
Manage --> Firewall
 
# Make sure the firewall is disabled
 
[[File:nsx-routing-blog-locale-14.png|600px]]
 
Manage --> Routing --> Global Configuration
 
# Enable ECMP
# Make sure that the default gateway is not configured
# Configure a router ID
 
[[File:nsx-routing-blog-locale-15.png|600px]]
 
Manage --> Routing --> BGP
 
# Enable BGP
# Configure the Local AS
# Disable Graceful Restart
# Configure the BGP peers towards the external routers and the UDLR.
 
[[File:nsx-routing-blog-locale-16.png|600px]]
 
Manage --> Routing --> Route Redistribution
 
# Enable route redistribution for BGP
# Specify what you allow to redistribute
 
[[File:nsx-routing-blog-locale-17.png|600px]]
 
==== DC2 ====
 
Manage --> Firewall
 
# Make sure the firewall is disabled
 
[[File:nsx-routing-blog-locale-18.png|600px]]
 
Manage --> Routing --> Global Configuration
 
# Enable ECMP
# Make sure that the default gateway is not configured
# Configure a router ID
 
[[File:nsx-routing-blog-locale-19.png|600px]]
 
Manage --> Routing --> BGP
 
# Enable BGP
# Configure the Local AS
# Disable Graceful Restart
# Configure the BGP peers towards the external routers and the UDLR
 
[[File:nsx-routing-blog-locale-20.png|600px]]
 
Manage --> Routing --> Route Redistribution
 
# Enable route redistribution for BGP
# Specify what you allow to redistribute
 
[[File:nsx-routing-blog-locale-21.png|600px]]
 
=== Cisco1000V configuration ===
 
==== DC1 ====
 
{{console|body=
!
##bl##hostname rt-a-01
!
router bgp 65511
bgp router-id 10.11.11.31
bgp log-neighbor-changes
neighbor 10.11.11.253 remote-as 65510
neighbor 10.11.11.253 description CS01
neighbor 10.100.19.2 remote-as 65521
neighbor 10.100.19.2 description ESG-A
!
address-family ipv4
  neighbor 10.11.11.253 activate
  neighbor 10.100.19.2 activate
exit-address-family
!
ip route 10.200.19.0 255.255.255.0 10.11.11.253
ip route 10.200.21.0 255.255.255.0 10.11.11.253
!
}}
 
{{console|body=
!
##bl##hostname rt-a-02
!
router bgp 65511
bgp router-id 10.11.11.32
bgp log-neighbor-changes
neighbor 10.11.11.253 remote-as 65510
neighbor 10.11.11.253 description CS01
neighbor 10.100.21.2 remote-as 65521
neighbor 10.100.21.2 description ESG-A
!
address-family ipv4
  neighbor 10.11.11.253 activate
  neighbor 10.100.21.2 activate
exit-address-family
!
ip route 10.200.19.0 255.255.255.0 10.11.11.253
ip route 10.200.21.0 255.255.255.0 10.11.11.253
!
}}
 
==== DC2 ====
 
{{console|body=
!
##bl##hostname rt-b-01
!
router bgp 65512
bgp router-id 10.11.11.33
bgp log-neighbor-changes
neighbor 10.11.11.253 remote-as 65510
neighbor 10.11.11.253 description CS01
neighbor 10.200.19.2 remote-as 65522
neighbor 10.200.19.2 description ESG-B
!
address-family ipv4
  neighbor 10.11.11.253 activate
  neighbor 10.200.19.2 activate
exit-address-family
!
ip route 10.100.19.0 255.255.255.0 10.11.11.253
ip route 10.100.21.0 255.255.255.0 10.11.11.253
!
}}
 
{{console|body=
!
##bl##hostname rt-b-02
!
router bgp 65512
bgp router-id 10.11.11.34
bgp log-neighbor-changes
neighbor 10.11.11.253 remote-as 65510
neighbor 10.11.11.253 description CS01
neighbor 10.200.21.2 remote-as 65522
neighbor 10.200.21.2 description ESG-B
!
address-family ipv4
  neighbor 10.11.11.253 activate
  neighbor 10.200.21.2 activate
exit-address-family
!
ip route 10.100.19.0 255.255.255.0 10.11.11.253
ip route 10.100.21.0 255.255.255.0 10.11.11.253
!
}}
 
=== Core configuration ===
 
{{console|body=
!
##bl##hostname cs-01
!
router bgp 65510
bgp router-id 10.11.11.253
bgp log-neighbor-changes
neighbor 10.11.11.31 remote-as 65511
neighbor 10.11.11.31 description RT-A-01
neighbor 10.11.11.32 remote-as 65511
neighbor 10.11.11.32 description RT-A-02
neighbor 10.11.11.33 remote-as 65512
neighbor 10.11.11.33 description RT-B-01
neighbor 10.11.11.34 remote-as 65512
neighbor 10.11.11.34 description RT-B-02
!
address-family ipv4
  network 10.11.11.0 mask 255.255.255.0
  neighbor 10.11.11.31 activate
  neighbor 10.11.11.31 next-hop-self
  neighbor 10.11.11.32 activate
  neighbor 10.11.11.32 next-hop-self
  neighbor 10.11.11.33 activate
  neighbor 10.11.11.33 next-hop-self
  neighbor 10.11.11.34 activate
  neighbor 10.11.11.34 next-hop-self
  no auto-summary
exit-address-family
!
}}
 
=== UDLR route peering verification ===
 
==== DC1 ====
 
[[File:nsx-routing-blog-locale-22.png|600px]]
 
==== DC2 ====
 
{{console|body=
##bl##NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0> show ip bgp neighbors
 
BGP neighbor is 172.39.39.33,  remote AS 65522,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
        Route refresh: advertised and received
        Address family IPv4 Unicast:advertised and received
        Graceful restart Capability:none
                Restart remain time: 0
Received 3194 messages, Sent 4728 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
        Index 1 Identifier 0x3740211c
        Route refresh request:received 0 sent 0
        Prefixes received 13 sent 5 advertised 5
Connections established 1, dropped 2468
Local host: 172.39.39.45, Local port: 179
Remote host: 172.39.39.33, Remote port: 53355
 
NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0>
 
}}
 
=== ESG route peering verification ===
 
==== DC1 ====
 
{{console|body=
##bl##REGA-ESG01-0> show ip bgp neighbors
 
BGP neighbor is 10.100.19.1,  remote AS 65511,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
        Route refresh: advertised and received
        Address family IPv4 Unicast:advertised and received
        Graceful restart Capability:none
                Restart remain time: 0
Received 2203 messages, Sent 2301 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
        Index 1 Identifier 0xa87fda9c
        Route refresh request:received 0 sent 0
        Prefixes received 3 sent 14 advertised 14
Connections established 1, dropped 1
Local host: 10.100.19.2, Local port: 12895
Remote host: 10.100.19.1, Remote port: 179
 
 
BGP neighbor is 10.100.21.1,  remote AS 65511,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
        Route refresh: advertised and received
        Address family IPv4 Unicast:advertised and received
        Graceful restart Capability:none
                Restart remain time: 0
Received 2201 messages, Sent 2300 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
        Index 2 Identifier 0xa87fda9c
        Route refresh request:received 0 sent 0
        Prefixes received 3 sent 14 advertised 14
Connections established 1, dropped 1
Local host: 10.100.21.2, Local port: 31006
Remote host: 10.100.21.1, Remote port: 179
 
 
BGP neighbor is 172.39.39.13,  remote AS 65530,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
        Route refresh: advertised and received
        Address family IPv4 Unicast:advertised and received
        Graceful restart Capability:none
                Restart remain time: 0
Received 2340 messages, Sent 2323 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
        Index 3 Identifier 0xa87fda9c
        Route refresh request:received 0 sent 0
        Prefixes received 7 sent 7 advertised 7
Connections established 2, dropped 42
Local host: 172.39.39.1, Local port: 13445
Remote host: 172.39.39.13, Remote port: 179
 
 
BGP neighbor is 172.39.39.29,  remote AS 65530,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
        Route refresh: advertised and received
        Address family IPv4 Unicast:advertised and received
        Graceful restart Capability:none
                Restart remain time: 0
Received 2286 messages, Sent 2282 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
        Index 4 Identifier 0xa87fda9c
        Route refresh request:received 0 sent 0
        Prefixes received 5 sent 7 advertised 7
Connections established 1, dropped 1
Local host: 172.39.39.17, Local port: 37920
Remote host: 172.39.39.29, Remote port: 179
 
REGA-ESG01-0>
}}
 
==== DC2 ====
 
{{console|body=
##bl##REGB-ESG01-0> show ip bgp neighbors
 
BGP neighbor is 10.200.19.1,  remote AS 65512,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
        Route refresh: advertised and received
        Address family IPv4 Unicast:advertised and received
        Graceful restart Capability:none
                Restart remain time: 0
Received 2192 messages, Sent 2287 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
        Index 1 Identifier 0x637a1aec
        Route refresh request:received 0 sent 0
        Prefixes received 9 sent 8 advertised 8
Connections established 1, dropped 1
Local host: 10.200.19.2, Local port: 57360
Remote host: 10.200.19.1, Remote port: 179
 
 
BGP neighbor is 10.200.21.1,  remote AS 65512,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
        Route refresh: advertised and received
        Address family IPv4 Unicast:advertised and received
        Graceful restart Capability:none
                Restart remain time: 0
Received 2191 messages, Sent 2280 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
        Index 2 Identifier 0x637a1aec
        Route refresh request:received 0 sent 0
        Prefixes received 9 sent 8 advertised 8
Connections established 1, dropped 1
Local host: 10.200.21.2, Local port: 17297
Remote host: 10.200.21.1, Remote port: 179
 
 
BGP neighbor is 172.39.39.13,  remote AS 65530,
BGP state = Idle, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
        Route refresh: None
        Address family none
        Graceful restart Capability:none
                Restart remain time: 0
Received 15751 messages, Sent 11421 messages
Default minimum time between advertisement runs is 30 seconds
For Address family none
        Index 3 Identifier 0x637a1aec
        Route refresh request:received 0 sent 0
        Prefixes received 0 sent 0 advertised 0
Connections established 0, dropped 15751
Local host: 10.200.21.2, Local port: 0
Remote host: 172.39.39.13, Remote port: 0
 
 
BGP neighbor is 172.39.39.45,  remote AS 65530,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
        Route refresh: advertised and received
        Address family IPv4 Unicast:advertised and received
        Graceful restart Capability:none
                Restart remain time: 0
Received 2278 messages, Sent 2279 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
        Index 4 Identifier 0x637a1aec
        Route refresh request:received 0 sent 0
        Prefixes received 5 sent 13 advertised 13
Connections established 1, dropped 1
Local host: 172.39.39.33, Local port: 53355
Remote host: 172.39.39.45, Remote port: 179
 
REGB-ESG01-0>
}}
 
=== Cisco1000V route peering verification ===
 
==== DC1 ====
 
{{console|body=
##bl##rt-a-01#show ip bgp summary
BGP router identifier 10.11.11.31, local AS number 65511
BGP table version is 54, main routing table version 54
17 network entries using 4216 bytes of memory
21 path entries using 2520 bytes of memory
3/3 BGP path/bestpath attribute entries using 720 bytes of memory
3 BGP AS-PATH entries using 88 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 7544 total bytes of memory
BGP activity 30/13 prefixes, 47/26 paths, scan interval 60 secs
 
Neighbor        V          AS MsgRcvd MsgSent  TblVer  InQ OutQ Up/Down  State/PfxRcd
10.11.11.253    4        65510    2218    2206      54    0    0 1d09h          7
10.100.19.2    4        65521    2304    2205      54    0    0 1d09h          14
rt-a-01#
}}
 
{{console|body=
##bl##rt-a-02#show ip bgp summary
BGP router identifier 10.11.11.32, local AS number 65511
BGP table version is 54, main routing table version 54
17 network entries using 4216 bytes of memory
21 path entries using 2520 bytes of memory
3/3 BGP path/bestpath attribute entries using 720 bytes of memory
3 BGP AS-PATH entries using 88 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 7544 total bytes of memory
BGP activity 35/18 prefixes, 47/26 paths, scan interval 60 secs
 
Neighbor        V          AS MsgRcvd MsgSent  TblVer  InQ OutQ Up/Down  State/PfxRcd
10.11.11.253    4        65510    2209    2210      54    0    0 1d09h          7
10.100.21.2    4        65521    2303    2204      54    0    0 1d09h          14
rt-a-02#
}}
 
==== DC2 ====
 
{{console|body=
##bl##rt-b-01#show ip bgp summary
BGP router identifier 10.11.11.33, local AS number 65512
BGP table version is 76, main routing table version 76
17 network entries using 4216 bytes of memory
19 path entries using 2280 bytes of memory
3/3 BGP path/bestpath attribute entries using 720 bytes of memory
3 BGP AS-PATH entries using 88 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 7304 total bytes of memory
BGP activity 30/13 prefixes, 50/31 paths, scan interval 60 secs
 
Neighbor        V          AS MsgRcvd MsgSent  TblVer  InQ OutQ Up/Down  State/PfxRcd
10.11.11.253    4        65510    5385    5387      76    0    0 3d09h          11
10.200.19.2    4        65522    2290    2194      76    0    0 1d09h          8
rt-b-01#
}}
 
{{console|body=
##bl##rt-b-02#show ip bgp summary
BGP router identifier 10.11.11.34, local AS number 65512
BGP table version is 49, main routing table version 49
17 network entries using 4216 bytes of memory
19 path entries using 2280 bytes of memory
3/3 BGP path/bestpath attribute entries using 720 bytes of memory
3 BGP AS-PATH entries using 88 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 7304 total bytes of memory
BGP activity 17/0 prefixes, 35/16 paths, scan interval 60 secs
 
Neighbor        V          AS MsgRcvd MsgSent  TblVer  InQ OutQ Up/Down  State/PfxRcd
10.11.11.253    4        65510    2210    2204      49    0    0 1d09h          11
10.200.21.2    4        65522    2284    2195      49    0    0 1d09h          8
rt-b-02#
}}
 
=== Core route peering verification ===
 
{{console|body=
##bl##cs-01#show ip bgp summary
BGP router identifier 10.11.11.253, local AS number 65510
BGP table version is 83, main routing table version 83
17 network entries using 2312 bytes of memory
45 path entries using 2340 bytes of memory
3/3 BGP path/bestpath attribute entries using 372 bytes of memory
2 BGP AS-PATH entries using 48 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 5072 total bytes of memory
BGP activity 30/13 prefixes, 109/64 paths, scan interval 60 secs
 
Neighbor        V          AS MsgRcvd MsgSent  TblVer  InQ OutQ Up/Down  State/PfxRcd
10.11.11.31    4        65511    2209    2221      83    0    0 1d09h          14
10.11.11.32    4        65511    2212    2212      83    0    0 1d09h          14
10.11.11.33    4        65512    5389    5387      83    0    0 3d09h          8
10.11.11.34    4        65512    2205    2210      83    0    0 1d09h          8
cs-01#
}}
 
=== UDLR routing tables ===
 
==== DC1 ====
 
[[File:nsx-routing-blog-locale-23.png|600px]]
 
==== DC2 ====
 
{{console|body=
##bl##NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0> show ip route
 
Codes: O - OSPF derived, i - IS-IS derived, B - BGP derived,
C - connected, S - static, L1 - IS-IS level-1, L2 - IS-IS level-2,
IA - OSPF inter area, E1 - OSPF external type 1, E2 - OSPF external type 2,
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
 
Total number of routes: 17
 
B      10.11.11.0/24        [20/0]        via 172.39.39.33
B      10.100.19.0/24      [20/0]        via 172.39.39.33
B      10.100.21.0/24      [20/0]        via 172.39.39.33
B      10.200.19.0/24      [20/0]        via 172.39.39.33
B      10.200.21.0/24      [20/0]        via 172.39.39.33
B      172.20.1.0/24        [20/0]        via 172.39.39.33
B      172.20.2.0/24        [20/0]        via 172.39.39.33
B      172.20.3.0/24        [20/0]        via 172.39.39.33
B      172.20.8.0/24        [20/0]        via 172.39.39.33
B      172.20.9.0/24        [20/0]        via 172.39.39.33
B      172.20.10.0/24      [20/0]        via 172.39.39.33
C      172.30.1.0/24        [0/0]        via 172.30.1.254
C      172.30.2.0/24        [0/0]        via 172.30.2.254
C      172.30.3.0/24        [0/0]        via 172.30.3.254
B      172.39.39.0/28      [20/0]        via 172.39.39.33
C      172.39.39.16/28      [0/0]        via 172.39.39.30
C      172.39.39.32/28      [0/0]        via 172.39.39.45
NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0>
}}
 
=== ESG routing tables ===
 
==== DC1 ====
 
{{console|body=
##bl##REGA-ESG01-0> show ip route
 
Codes: O - OSPF derived, i - IS-IS derived, B - BGP derived,
C - connected, S - static, L1 - IS-IS level-1, L2 - IS-IS level-2,
IA - OSPF inter area, E1 - OSPF external type 1, E2 - OSPF external type 2,
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
 
Total number of routes: 17
 
B      10.11.11.0/24        [20/0]        via 10.100.19.1
B      10.11.11.0/24        [20/0]        via 10.100.21.1
C      10.100.19.0/24      [0/0]        via 10.100.19.2
C      10.100.21.0/24      [0/0]        via 10.100.21.2
B      10.200.19.0/24      [20/0]        via 10.100.19.1
B      10.200.19.0/24      [20/0]        via 10.100.21.1
B      10.200.21.0/24      [20/0]        via 10.100.19.1
B      10.200.21.0/24      [20/0]        via 10.100.21.1
B      172.20.1.0/24        [20/0]        via 172.39.39.14
B      172.20.2.0/24        [20/0]        via 172.39.39.14
B      172.20.3.0/24        [20/0]        via 172.39.39.14
B      172.20.8.0/24        [20/0]       via 172.39.39.14
B      172.20.9.0/24        [20/0]       via 172.39.39.14
B      172.20.10.0/24      [20/0]        via 172.39.39.14
B      172.30.1.0/24        [20/0]        via 172.39.39.30
B      172.30.2.0/24        [20/0]        via 172.39.39.30
B      172.30.3.0/24        [20/0]        via 172.39.39.30
C      172.39.39.0/28      [0/0]        via 172.39.39.1
C      172.39.39.16/28      [0/0]        via 172.39.39.17
B      172.39.39.32/28      [20/0]        via 172.39.39.30
REGA-ESG01-0>
}}
 
==== DC2 ====
 
{{console|body=
##bl##REGB-ESG01-0> show ip route
 
Codes: O - OSPF derived, i - IS-IS derived, B - BGP derived,
C - connected, S - static, L1 - IS-IS level-1, L2 - IS-IS level-2,
IA - OSPF inter area, E1 - OSPF external type 1, E2 - OSPF external type 2,
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
 
Total number of routes: 17
 
B      10.11.11.0/24        [20/0]        via 10.200.19.1
B      10.11.11.0/24        [20/0]        via 10.200.21.1
B      10.100.19.0/24      [20/0]        via 10.200.19.1
B      10.100.19.0/24      [20/0]       via 10.200.21.1
B      10.100.21.0/24      [20/0]       via 10.200.19.1
B      10.100.21.0/24      [20/0]        via 10.200.21.1
C      10.200.19.0/24      [0/0]        via 10.200.19.2
C      10.200.21.0/24      [0/0]        via 10.200.21.2
B      172.20.1.0/24        [20/0]        via 10.200.19.1
B      172.20.1.0/24        [20/0]        via 10.200.21.1
B      172.20.2.0/24        [20/0]        via 10.200.19.1
B      172.20.2.0/24        [20/0]        via 10.200.21.1
B      172.20.3.0/24        [20/0]        via 10.200.19.1
B      172.20.3.0/24        [20/0]        via 10.200.21.1
B      172.20.8.0/24        [20/0]        via 10.200.19.1
B      172.20.8.0/24        [20/0]        via 10.200.21.1
B      172.20.9.0/24        [20/0]        via 10.200.19.1
B      172.20.9.0/24        [20/0]        via 10.200.21.1
B      172.20.10.0/24      [20/0]        via 10.200.19.1
B      172.20.10.0/24      [20/0]        via 10.200.21.1
B      172.30.1.0/24        [20/0]        via 172.39.39.46
B      172.30.2.0/24        [20/0]        via 172.39.39.46
B      172.30.3.0/24        [20/0]        via 172.39.39.46
C      172.39.39.0/28      [0/0]        via 172.39.39.2
B      172.39.39.16/28      [20/0]        via 172.39.39.46
C      172.39.39.32/28      [0/0]        via 172.39.39.33
REGB-ESG01-0>
}}
 
=== Cisco1000V routing tables ===
 
==== DC1 ====
 
{{console|body=
##bl##rt-a-01#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
      D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
      N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
      E1 - OSPF external type 1, E2 - OSPF external type 2
      i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
      ia - IS-IS inter area, * - candidate default, U - per-user static route
      o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
      a - application route
      + - replicated route, % - next hop override
 
Gateway of last resort is not set
 
      10.0.0.0/8 is variably subnetted, 9 subnets, 2 masks
C        10.11.11.0/24 is directly connected, GigabitEthernet2
L        10.11.11.31/32 is directly connected, GigabitEthernet2
C        10.100.1.0/24 is directly connected, GigabitEthernet1
L        10.100.1.101/32 is directly connected, GigabitEthernet1
C        10.100.19.0/24 is directly connected, GigabitEthernet3
L        10.100.19.1/32 is directly connected, GigabitEthernet3
B        10.100.21.0/24 [20/0] via 10.100.19.2, 1d09h
S        10.200.19.0/24 [1/0] via 10.11.11.253
S        10.200.21.0/24 [1/0] via 10.11.11.253
      172.20.0.0/24 is subnetted, 6 subnets
B        172.20.1.0 [20/0] via 10.100.19.2, 1d09h
B        172.20.2.0 [20/0] via 10.100.19.2, 1d09h
B        172.20.3.0 [20/0] via 10.100.19.2, 1d09h
B        172.20.8.0 [20/0] via 10.100.19.2, 1d09h
B        172.20.9.0 [20/0] via 10.100.19.2, 1d09h
B        172.20.10.0 [20/0] via 10.100.19.2, 1d09h
      172.30.0.0/24 is subnetted, 3 subnets
B        172.30.1.0 [20/0] via 10.100.19.2, 1d09h
B        172.30.2.0 [20/0] via 10.100.19.2, 1d09h
B        172.30.3.0 [20/0] via 10.100.19.2, 1d09h
      172.39.0.0/28 is subnetted, 3 subnets
B        172.39.39.0 [20/0] via 10.100.19.2, 1d09h
B        172.39.39.16 [20/0] via 10.100.19.2, 1d09h
B        172.39.39.32 [20/0] via 10.100.19.2, 1d09h
rt-a-01#
}}
 
{{console|body=
##bl##rt-a-02#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
      D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
      N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
      E1 - OSPF external type 1, E2 - OSPF external type 2
      i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
      ia - IS-IS inter area, * - candidate default, U - per-user static route
      o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
      a - application route
      + - replicated route, % - next hop override
 
Gateway of last resort is not set
 
      10.0.0.0/8 is variably subnetted, 9 subnets, 2 masks
C        10.11.11.0/24 is directly connected, GigabitEthernet2
L        10.11.11.32/32 is directly connected, GigabitEthernet2
C        10.100.1.0/24 is directly connected, GigabitEthernet1
L        10.100.1.102/32 is directly connected, GigabitEthernet1
B        10.100.19.0/24 [20/0] via 10.100.21.2, 1d09h
C        10.100.21.0/24 is directly connected, GigabitEthernet3
L        10.100.21.1/32 is directly connected, GigabitEthernet3
S        10.200.19.0/24 [1/0] via 10.11.11.253
S        10.200.21.0/24 [1/0] via 10.11.11.253
      172.20.0.0/24 is subnetted, 6 subnets
B        172.20.1.0 [20/0] via 10.100.21.2, 1d09h
B        172.20.2.0 [20/0] via 10.100.21.2, 1d09h
B        172.20.3.0 [20/0] via 10.100.21.2, 1d09h
B        172.20.8.0 [20/0] via 10.100.21.2, 1d09h
B        172.20.9.0 [20/0] via 10.100.21.2, 1d09h
B        172.20.10.0 [20/0] via 10.100.21.2, 1d09h
      172.30.0.0/24 is subnetted, 3 subnets
B        172.30.1.0 [20/0] via 10.100.21.2, 1d09h
B        172.30.2.0 [20/0] via 10.100.21.2, 1d09h
B        172.30.3.0 [20/0] via 10.100.21.2, 1d09h
      172.39.0.0/28 is subnetted, 3 subnets
B        172.39.39.0 [20/0] via 10.100.21.2, 1d09h
B        172.39.39.16 [20/0] via 10.100.21.2, 1d09h
B        172.39.39.32 [20/0] via 10.100.21.2, 1d09h
rt-a-02#
}}
 
==== DC2 ====
 
{{console|body=
##bl##rt-b-01#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
      D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
      N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
      E1 - OSPF external type 1, E2 - OSPF external type 2
      i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
      ia - IS-IS inter area, * - candidate default, U - per-user static route
      o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
      a - application route
      + - replicated route, % - next hop override
 
Gateway of last resort is not set
 
      10.0.0.0/8 is variably subnetted, 9 subnets, 2 masks
C        10.11.11.0/24 is directly connected, GigabitEthernet2
L        10.11.11.33/32 is directly connected, GigabitEthernet2
S        10.100.19.0/24 [1/0] via 10.11.11.253
S        10.100.21.0/24 [1/0] via 10.11.11.253
C        10.200.1.0/24 is directly connected, GigabitEthernet1
L        10.200.1.101/32 is directly connected, GigabitEthernet1
C        10.200.19.0/24 is directly connected, GigabitEthernet3
L        10.200.19.1/32 is directly connected, GigabitEthernet3
B        10.200.21.0/24 [20/0] via 10.200.19.2, 1d09h
      172.20.0.0/24 is subnetted, 6 subnets
B        172.20.1.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.2.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.3.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.8.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.9.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.10.0 [20/0] via 10.11.11.253, 1d09h
      172.30.0.0/24 is subnetted, 3 subnets
B        172.30.1.0 [20/0] via 10.200.19.2, 1d09h
B        172.30.2.0 [20/0] via 10.200.19.2, 1d09h
B        172.30.3.0 [20/0] via 10.200.19.2, 1d09h
      172.39.0.0/28 is subnetted, 3 subnets
B        172.39.39.0 [20/0] via 10.200.19.2, 1d09h
B        172.39.39.16 [20/0] via 10.200.19.2, 1d09h
B        172.39.39.32 [20/0] via 10.200.19.2, 1d09h
rt-b-01#
}}
 
{{console|body=
##bl##rt-b-02#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
      D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
      N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
      E1 - OSPF external type 1, E2 - OSPF external type 2
      i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
      ia - IS-IS inter area, * - candidate default, U - per-user static route
      o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
      a - application route
      + - replicated route, % - next hop override
 
Gateway of last resort is not set
 
      10.0.0.0/8 is variably subnetted, 9 subnets, 2 masks
C        10.11.11.0/24 is directly connected, GigabitEthernet2
L        10.11.11.34/32 is directly connected, GigabitEthernet2
S        10.100.19.0/24 [1/0] via 10.11.11.253
S        10.100.21.0/24 [1/0] via 10.11.11.253
C        10.200.1.0/24 is directly connected, GigabitEthernet1
L        10.200.1.102/32 is directly connected, GigabitEthernet1
B        10.200.19.0/24 [20/0] via 10.200.21.2, 1d09h
C        10.200.21.0/24 is directly connected, GigabitEthernet3
L        10.200.21.1/32 is directly connected, GigabitEthernet3
      172.20.0.0/24 is subnetted, 6 subnets
B        172.20.1.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.2.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.3.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.8.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.9.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.10.0 [20/0] via 10.11.11.253, 1d09h
      172.30.0.0/24 is subnetted, 3 subnets
B        172.30.1.0 [20/0] via 10.200.21.2, 1d09h
B        172.30.2.0 [20/0] via 10.200.21.2, 1d09h
B        172.30.3.0 [20/0] via 10.200.21.2, 1d09h
      172.39.0.0/28 is subnetted, 3 subnets
B        172.39.39.0 [20/0] via 10.200.21.2, 1d09h
B        172.39.39.16 [20/0] via 10.200.21.2, 1d09h
B        172.39.39.32 [20/0] via 10.200.21.2, 1d09h
rt-b-02#
}}
 
=== Core routing table ===
 
{{console|body=
##bl##cs-01#show ip route bgp
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
      D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
      N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
      E1 - OSPF external type 1, E2 - OSPF external type 2
      i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
      ia - IS-IS inter area, * - candidate default, U - per-user static route
      o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
      + - replicated route, % - next hop override
 
Gateway of last resort is 10.11.11.254 to network 0.0.0.0
 
      172.20.0.0/24 is subnetted, 6 subnets
B        172.20.1.0 [20/0] via 10.11.11.31, 1d09h
B        172.20.2.0 [20/0] via 10.11.11.31, 1d09h
B        172.20.3.0 [20/0] via 10.11.11.31, 1d09h
B        172.20.8.0 [20/0] via 10.11.11.31, 1d09h
B        172.20.9.0 [20/0] via 10.11.11.31, 1d09h
B        172.20.10.0 [20/0] via 10.11.11.31, 1d09h
      172.30.0.0/24 is subnetted, 3 subnets
B        172.30.1.0 [20/0] via 10.11.11.33, 1d09h
B        172.30.2.0 [20/0] via 10.11.11.33, 1d09h
B        172.30.3.0 [20/0] via 10.11.11.33, 1d09h
      172.39.0.0/28 is subnetted, 3 subnets
B        172.39.39.0 [20/0] via 10.11.11.31, 1d09h
B        172.39.39.16 [20/0] via 10.11.11.31, 1d09h
B        172.39.39.32 [20/0] via 10.11.11.34, 1d09h
cs-01#
}}
 
=== UDLR BGP tables ===
 
==== DC1 ====
 
[[File:nsx-routing-blog-locale-24.png|600px]]
 
==== DC2 ====
 
{{console|body=
##bl##NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0> show ip bgp
 
Status codes: s - suppressed, d - damped, > - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
 
    Network            Next Hop      Metric  LocPrf  Weight AS Path
  > 10.11.11.0/24      172.39.39.33      0    100      60  65522 i
  > 10.100.19.0/24    172.39.39.33      0    100      60  65522 ?
  > 10.100.21.0/24    172.39.39.33      0    100      60  65522 ?
  > 10.200.19.0/24    172.39.39.33      0    100      60  65522 ?
  > 10.200.21.0/24    172.39.39.33      0    100      60  65522 ?
  > 172.20.1.0/24      172.39.39.33      0    100      60  65522 ?
  > 172.20.2.0/24      172.39.39.33      0    100      60  65522 ?
  > 172.20.3.0/24      172.39.39.33      0    100      60  65522 ?
  > 172.20.8.0/24      172.39.39.33      0    100      60  65522 ?
  > 172.20.9.0/24      172.39.39.33      0    100      60  65522 ?
  > 172.20.10.0/24    172.39.39.33      0    100      60  65522 ?
  > 172.30.1.0/24      0.0.0.0          0    100  32768  ?
  > 172.30.2.0/24      0.0.0.0          0    100  32768  ?
  > 172.30.3.0/24      0.0.0.0          0    100  32768  ?
  > 172.39.39.0/28    172.39.39.33      0    100      60  65522 ?
  > 172.39.39.16/28    0.0.0.0          0    100  32768  ?
    172.39.39.32/28    172.39.39.33      0    100      60  65522 ?
  > 172.39.39.32/28    0.0.0.0          0    100  32768  ?
NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0>
}}
 
 
=== ESG BGP tables ===
 
==== DC1 ====
 
{{console|body=
##bl##REGA-ESG01-0> show ip bgp
 
Status codes: s - suppressed, d - damped, > - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
 
    Network            Next Hop      Metric  LocPrf  Weight AS Path
  > 10.11.11.0/24      10.100.19.1      0    100      60  65511 65510 i
    10.11.11.0/24      10.100.21.1      0    100      60  65511 65510 i
  > 10.100.19.0/24    0.0.0.0          0    100  32768  ?
  > 10.100.21.0/24    0.0.0.0          0    100  32768  ?
  > 10.200.19.0/24    10.100.19.1      0    100      60  65511 65510 65512 65522 ?
    10.200.19.0/24    10.100.21.1      0    100      60  65511 65510 65512 65522 ?
  > 10.200.21.0/24    10.100.19.1      0    100      60  65511 65510 65512 65522 ?
    10.200.21.0/24    10.100.21.1      0    100      60  65511 65510 65512 65522 ?
  > 172.20.1.0/24      172.39.39.14      0    100      60  65530 ?
  > 172.20.2.0/24      172.39.39.14      0    100      60  65530 ?
  > 172.20.3.0/24      172.39.39.14      0    100      60  65530 ?
  > 172.20.8.0/24      172.39.39.14      0    100      60  65530 ?
  > 172.20.9.0/24      172.39.39.14      0    100      60  65530 ?
  > 172.20.10.0/24    172.39.39.14      0    100      60  65530 ?
  > 172.30.1.0/24      172.39.39.30      0    100      60  65530 ?
  > 172.30.2.0/24      172.39.39.30      0    100      60  65530 ?
  > 172.30.3.0/24      172.39.39.30      0    100      60  65530 ?
    172.39.39.0/28    172.39.39.14      0    100      60  65530 ?
  > 172.39.39.0/28    0.0.0.0          0    100  32768  ?
    172.39.39.16/28    172.39.39.30      0    100      60  65530 ?
  > 172.39.39.16/28    0.0.0.0          0    100  32768  ?
  > 172.39.39.32/28    172.39.39.30      0    100      60  65530 ?
REGA-ESG01-0>
}}
 
==== DC2 ====
 
{{console|body=
##bl##REGB-ESG01-0> show ip bgp
 
Status codes: s - suppressed, d - damped, > - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
 
    Network            Next Hop      Metric  LocPrf  Weight AS Path
  > 10.11.11.0/24      10.200.19.1      0    100      60  65512 65510 i
    10.11.11.0/24      10.200.21.1      0    100      60  65512 65510 i
  > 10.100.19.0/24    10.200.19.1      0    100      60  65512 65510 65511 65521 ?
    10.100.19.0/24    10.200.21.1      0    100      60  65512 65510 65511 65521 ?
  > 10.100.21.0/24    10.200.19.1      0    100      60  65512 65510 65511 65521 ?
    10.100.21.0/24    10.200.21.1      0    100      60  65512 65510 65511 65521 ?
  > 10.200.19.0/24    0.0.0.0          0    100  32768  ?
  > 10.200.21.0/24    0.0.0.0          0    100  32768  ?
  > 172.20.1.0/24      10.200.19.1      0    100      60  65512 65510 65511 65521 ?
    172.20.1.0/24      10.200.21.1      0    100      60  65512 65510 65511 65521 ?
  > 172.20.2.0/24      10.200.19.1      0    100      60  65512 65510 65511 65521 ?
    172.20.2.0/24      10.200.21.1      0    100      60  65512 65510 65511 65521 ?
  > 172.20.3.0/24      10.200.19.1      0    100      60  65512 65510 65511 65521 ?
    172.20.3.0/24      10.200.21.1      0    100      60  65512 65510 65511 65521 ?
  > 172.20.8.0/24      10.200.19.1      0    100      60  65512 65510 65511 65521 ?
    172.20.8.0/24      10.200.21.1      0    100      60  65512 65510 65511 65521 ?
  > 172.20.9.0/24      10.200.19.1      0    100      60  65512 65510 65511 65521 ?
    172.20.9.0/24      10.200.21.1      0    100      60  65512 65510 65511 65521 ?
  > 172.20.10.0/24    10.200.19.1      0    100      60  65512 65510 65511 65521 ?
    172.20.10.0/24    10.200.21.1      0    100      60  65512 65510 65511 65521 ?
  > 172.30.1.0/24      172.39.39.46      0    100      60  65530 ?
  > 172.30.2.0/24      172.39.39.46      0    100      60  65530 ?
  > 172.30.3.0/24      172.39.39.46      0    100      60  65530 ?
  > 172.39.39.0/28    0.0.0.0          0    100  32768  ?
  > 172.39.39.16/28    172.39.39.46      0    100      60  65530 ?
    172.39.39.32/28    172.39.39.46      0    100      60  65530 ?
  > 172.39.39.32/28    0.0.0.0          0    100  32768  ?
REGB-ESG01-0>
}}
 
=== Cisco1000V BGP tables ===
 
==== DC1 ====
 
{{console|body=
##bl##rt-a-01#show ip bgp
BGP table version is 54, local router ID is 10.11.11.31
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
 
    Network          Next Hop            Metric LocPrf Weight Path
r>  10.11.11.0/24    10.11.11.253            0            0 65510 i
r>  10.100.19.0/24  10.100.19.2                            0 65521 ?
*>  10.100.21.0/24  10.100.19.2                            0 65521 ?
r>  10.200.19.0/24  10.11.11.253                          0 65510 65512 65522 ?
r>  10.200.21.0/24  10.11.11.253                          0 65510 65512 65522 ?
*>  172.20.1.0/24    10.100.19.2                            0 65521 ?
*>  172.20.2.0/24    10.100.19.2                            0 65521 ?
*>  172.20.3.0/24    10.100.19.2                            0 65521 ?
*>  172.20.8.0/24    10.100.19.2                            0 65521 ?
*>  172.20.9.0/24    10.100.19.2                            0 65521 ?
*>  172.20.10.0/24  10.100.19.2                            0 65521 ?
*>  172.30.1.0/24    10.100.19.2                            0 65521 ?
*                    10.11.11.253                          0 65510 65512 65522 ?
    Network          Next Hop            Metric LocPrf Weight Path
*>  172.30.2.0/24    10.100.19.2                            0 65521 ?
*                    10.11.11.253                          0 65510 65512 65522 ?
*>  172.30.3.0/24    10.100.19.2                            0 65521 ?
*                    10.11.11.253                          0 65510 65512 65522 ?
*>  172.39.39.0/28  10.100.19.2                            0 65521 ?
*>  172.39.39.16/28  10.100.19.2                            0 65521 ?
*>  172.39.39.32/28  10.100.19.2                            0 65521 ?
*                    10.11.11.253                          0 65510 65512 65522 ?
rt-a-01#
}}
 
{{console|body=
##bl##rt-a-02#show ip bgp
BGP table version is 54, local router ID is 10.11.11.32
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
 
    Network          Next Hop            Metric LocPrf Weight Path
r>  10.11.11.0/24    10.11.11.253            0            0 65510 i
*>  10.100.19.0/24  10.100.21.2                            0 65521 ?
r>  10.100.21.0/24  10.100.21.2                            0 65521 ?
r>  10.200.19.0/24  10.11.11.253                          0 65510 65512 65522 ?
r>  10.200.21.0/24  10.11.11.253                          0 65510 65512 65522 ?
*>  172.20.1.0/24    10.100.21.2                            0 65521 ?
*>  172.20.2.0/24    10.100.21.2                            0 65521 ?
*>  172.20.3.0/24    10.100.21.2                            0 65521 ?
*>  172.20.8.0/24    10.100.21.2                            0 65521 ?
*>  172.20.9.0/24    10.100.21.2                            0 65521 ?
*>  172.20.10.0/24  10.100.21.2                            0 65521 ?
*>  172.30.1.0/24    10.100.21.2                            0 65521 ?
*                    10.11.11.253                          0 65510 65512 65522 ?
    Network          Next Hop            Metric LocPrf Weight Path
*>  172.30.2.0/24    10.100.21.2                            0 65521 ?
*                    10.11.11.253                          0 65510 65512 65522 ?
*>  172.30.3.0/24    10.100.21.2                            0 65521 ?
*                    10.11.11.253                          0 65510 65512 65522 ?
*>  172.39.39.0/28  10.100.21.2                            0 65521 ?
*>  172.39.39.16/28  10.100.21.2                            0 65521 ?
*>  172.39.39.32/28  10.100.21.2                            0 65521 ?
*                    10.11.11.253                          0 65510 65512 65522 ?
rt-a-02#
}}
 
==== DC2 ====
 
{{console|body=
##bl##rt-b-01#show ip bgp
BGP table version is 76, local router ID is 10.11.11.33
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
 
    Network          Next Hop            Metric LocPrf Weight Path
r>  10.11.11.0/24    10.11.11.253            0            0 65510 i
r>  10.100.19.0/24  10.11.11.253                          0 65510 65511 65521 ?
r>  10.100.21.0/24  10.11.11.253                          0 65510 65511 65521 ?
r>  10.200.19.0/24  10.200.19.2                            0 65522 ?
*>  10.200.21.0/24  10.200.19.2                            0 65522 ?
*>  172.20.1.0/24    10.11.11.253                          0 65510 65511 65521 ?
*>  172.20.2.0/24    10.11.11.253                          0 65510 65511 65521 ?
*>  172.20.3.0/24    10.11.11.253                          0 65510 65511 65521 ?
*>  172.20.8.0/24    10.11.11.253                          0 65510 65511 65521 ?
    Network          Next Hop            Metric LocPrf Weight Path
*>  172.20.9.0/24    10.11.11.253                          0 65510 65511 65521 ?
*>  172.20.10.0/24  10.11.11.253                          0 65510 65511 65521 ?
*>  172.30.1.0/24    10.200.19.2                            0 65522 ?
*>  172.30.2.0/24    10.200.19.2                            0 65522 ?
*>  172.30.3.0/24    10.200.19.2                            0 65522 ?
*>  172.39.39.0/28  10.200.19.2                            0 65522 ?
*                    10.11.11.253                          0 65510 65511 65521 ?
*>  172.39.39.16/28  10.200.19.2                            0 65522 ?
*                    10.11.11.253                          0 65510 65511 65521 ?
*>  172.39.39.32/28  10.200.19.2                            0 65522 ?
rt-b-01#
}}
 
{{console|body=
##bl##rt-b-02#show ip bgp
BGP table version is 49, local router ID is 10.11.11.34
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
 
    Network          Next Hop            Metric LocPrf Weight Path
r>  10.11.11.0/24    10.11.11.253            0            0 65510 i
r>  10.100.19.0/24  10.11.11.253                          0 65510 65511 65521 ?
r>  10.100.21.0/24  10.11.11.253                          0 65510 65511 65521 ?
*>  10.200.19.0/24  10.200.21.2                            0 65522 ?
r>  10.200.21.0/24  10.200.21.2                            0 65522 ?
*>  172.20.1.0/24    10.11.11.253                          0 65510 65511 65521 ?
*>  172.20.2.0/24    10.11.11.253                          0 65510 65511 65521 ?
*>  172.20.3.0/24    10.11.11.253                          0 65510 65511 65521 ?
*>  172.20.8.0/24    10.11.11.253                          0 65510 65511 65521 ?
    Network          Next Hop            Metric LocPrf Weight Path
*>  172.20.9.0/24    10.11.11.253                          0 65510 65511 65521 ?
*>  172.20.10.0/24  10.11.11.253                          0 65510 65511 65521 ?
*>  172.30.1.0/24    10.200.21.2                            0 65522 ?
*>  172.30.2.0/24    10.200.21.2                            0 65522 ?
*>  172.30.3.0/24    10.200.21.2                            0 65522 ?
*>  172.39.39.0/28  10.200.21.2                            0 65522 ?
*                    10.11.11.253                          0 65510 65511 65521 ?
*>  172.39.39.16/28  10.200.21.2                            0 65522 ?
*                    10.11.11.253                          0 65510 65511 65521 ?
*>  172.39.39.32/28  10.200.21.2                            0 65522 ?
rt-b-02#
}}
 
=== Core BGP tables ===
 
{{console|body=
##bl##cs-01#show ip bgp
BGP table version is 83, local router ID is 10.11.11.253
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, x best-external
Origin codes: i - IGP, e - EGP, ? - incomplete
 
  Network          Next Hop            Metric LocPrf Weight Path
*> 10.11.11.0/24    0.0.0.0                  0        32768 i
r  10.100.19.0/24  10.11.11.32                            0 65511 65521 ?
r>                  10.11.11.31                            0 65511 65521 ?
r  10.100.21.0/24  10.11.11.32                            0 65511 65521 ?
r>                  10.11.11.31                            0 65511 65521 ?
r  10.200.19.0/24  10.11.11.33                            0 65512 65522 ?
r>                  10.11.11.34                            0 65512 65522 ?
r  10.200.21.0/24  10.11.11.33                            0 65512 65522 ?
r>                  10.11.11.34                            0 65512 65522 ?
*  172.20.1.0/24    10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.20.2.0/24    10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.20.3.0/24    10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.20.8.0/24    10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.20.9.0/24    10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.20.10.0/24  10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.30.1.0/24    10.11.11.31                            0 65511 65521 ?
*                  10.11.11.32                            0 65511 65521 ?
*                  10.11.11.34                            0 65512 65522 ?
*>                  10.11.11.33                            0 65512 65522 ?
*  172.30.2.0/24    10.11.11.31                            0 65511 65521 ?
*                  10.11.11.32                            0 65511 65521 ?
*                  10.11.11.34                            0 65512 65522 ?
*>                  10.11.11.33                            0 65512 65522 ?
*  172.30.3.0/24    10.11.11.31                            0 65511 65521 ?
*                  10.11.11.32                            0 65511 65521 ?
*                  10.11.11.34                            0 65512 65522 ?
*>                  10.11.11.33                            0 65512 65522 ?
*  172.39.39.0/28  10.11.11.33                            0 65512 65522 ?
*                  10.11.11.34                            0 65512 65522 ?
*                  10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.39.39.16/28  10.11.11.34                            0 65512 65522 ?
*                  10.11.11.33                            0 65512 65522 ?
*                  10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.39.39.32/28  10.11.11.31                            0 65511 65521 ?
*                  10.11.11.32                            0 65511 65521 ?
*                  10.11.11.33                            0 65512 65522 ?
*>                  10.11.11.34                            0 65512 65522 ?
cs-01#
}}
 
=== Routing path verifications ===
 
==== Routing path verification from the external client VM to T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 ====
 
This is an INGRESS trafficflow.
As we did not make any configuration changes to manipulate the INGRESS trafficflow the INGRESS traffic can either choose to select the DC1 path or the DC2 path.
This is currently determined by BGP.
 
{{console|body=
##bl##C:\Users\Administrator>tracert 172.30.1.1
 
Tracing route to 172.30.1.1 over a maximum of 30 hops
 
  1    1 ms    1 ms    1 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    1 ms    <1 ms    <1 ms  10.200.21.2
  4    1 ms    <1 ms    <1 ms  172.39.39.46
  5    1 ms    1 ms    1 ms  172.30.1.1
 
Trace complete.
}}
 
{{console|body=
##bl##C:\Users\Administrator>tracert 172.30.1.2
 
Tracing route to 172.30.1.2 over a maximum of 30 hops
 
  1    2 ms    1 ms    1 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    <1 ms    <1 ms    <1 ms  10.200.21.2
  4    <1 ms    <1 ms    <1 ms  172.39.39.46
  5  944 ms    1 ms    <1 ms  172.30.1.2
 
Trace complete.
}}
 
{{console|body=
##bl##C:\Users\Administrator>tracert 172.30.1.3
 
Tracing route to 172.30.1.3 over a maximum of 30 hops
 
  1    1 ms    1 ms    6 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    <1 ms    <1 ms    <1 ms  10.200.21.2
  4    <1 ms    <1 ms    <1 ms  172.39.39.46
  5  723 ms    1 ms    <1 ms  172.30.1.3
 
Trace complete.
}}
 
{{console|body=
##bl##C:\Users\Administrator>tracert 172.30.1.4
 
Tracing route to 172.30.1.4 over a maximum of 30 hops
 
  1    1 ms    1 ms    4 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    <1 ms    <1 ms    <1 ms  10.200.21.2
  4    <1 ms    <1 ms    <1 ms  172.39.39.46
  5    82 ms    1 ms    <1 ms  172.30.1.4
 
Trace complete.
}}
 
==== Routing path verification from the T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 to the external client VM ====
 
This is an EGRESS trafficflow.
The expectation is that the traffic towards T3-WEB-1 and T3-WEB-2 will fo trough the DC1 routers and the traffic towards T3-WEB-3 and T3-WEB-4 will fo trough the DC2 routers.
The reason for this is that Locale ID is configured.
 
{{console|body=
##bl##root@Web01:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
1  172.30.1.254 (172.30.1.254)  0.082 ms  2.689 ms  2.687 ms
2  172.39.39.17 (172.39.39.17)  0.666 ms  0.527 ms  0.524 ms
3  10.100.21.1 (10.100.21.1)  1.067 ms  1.091 ms  1.072 ms
4  10.11.11.50 (10.11.11.50)  1.347 ms * *
root@Web01:~#
}}
 
{{console|body=
##bl##root@Web02:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
1  172.30.1.254 (172.30.1.254)  0.172 ms  0.149 ms  0.131 ms
2  * * 172.39.39.17 (172.39.39.17)  0.455 ms
3  10.100.21.1 (10.100.21.1)  0.758 ms  0.599 ms  0.494 ms
4  10.11.11.50 (10.11.11.50)  0.918 ms * *
root@Web02:~#
}}
 
{{console|body=
##bl##root@Web03:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
1  172.30.1.254 (172.30.1.254)  0.120 ms  0.109 ms  0.060 ms
2  172.39.39.33 (172.39.39.33)  0.387 ms  0.345 ms  0.445 ms
3  10.200.21.1 (10.200.21.1)  1.249 ms  1.371 ms  1.383 ms
4  10.11.11.50 (10.11.11.50)  1.210 ms * *
root@Web03:~#
}}
 
{{console|body=
##bl##root@Web04:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
1  172.30.1.254 (172.30.1.254)  0.109 ms  0.181 ms  0.132 ms
2  172.39.39.33 (172.39.39.33)  0.471 ms  0.470 ms  0.466 ms
3  10.200.21.1 (10.200.21.1)  0.900 ms  0.871 ms  0.868 ms
4  10.11.11.50 (10.11.11.50)  1.063 ms * *
root@Web04:~#
}}
 
=== Change the Locale ID manually on DC2 to match DC1 ===
 
==== DC1 ====
 
The locale ID will not change here and these settings are kept te same
 
==== DC2 ====
 
===== Compute Cluster =====
 
[[File:nsx-routing-blog-locale-25.png|600px]]
 
=== Routing path verifications ===
 
==== Routing path verification from the external client VM to T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 ====
 
This is an INGRESS trafficflow.
As we did not make any configuration changes to manipulate the INGRESS trafficflow the INGRESS traffic can either choose to select the DC1 path or the DC2 path.
This is currently determined by BGP.
 
{{console|body=
##bl##C:\Users\Administrator>tracert 172.30.1.1
 
Tracing route to 172.30.1.1 over a maximum of 30 hops
 
  1    <1 ms    <1 ms    3 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    1 ms    <1 ms    <1 ms  10.200.21.2
  4  231 ms    <1 ms    <1 ms  172.39.39.30
  5  966 ms    1 ms    <1 ms  172.30.1.1
 
Trace complete.
}}
 
 
{{console|body=
##bl##C:\Users\Administrator>tracert 172.30.1.2
 
Tracing route to 172.30.1.2 over a maximum of 30 hops
 
  1    <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    <1 ms    <1 ms    <1 ms  10.200.21.2
  4    1 ms    <1 ms    <1 ms  172.39.39.30
  5    1 ms    1 ms    <1 ms  172.30.1.2
 
Trace complete.
}}
 
 
{{console|body=
##bl##C:\Users\Administrator>tracert 172.30.1.3
 
Tracing route to 172.30.1.3 over a maximum of 30 hops
 
  1    <1 ms    <1 ms    3 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    <1 ms    <1 ms    <1 ms  10.200.21.2
  4    1 ms    <1 ms    <1 ms  172.39.39.30
  5    2 ms    <1 ms    <1 ms  172.30.1.3
 
Trace complete.
}}
 
 
{{console|body=
##bl##C:\Users\Administrator>tracert 172.30.1.4
 
Tracing route to 172.30.1.4 over a maximum of 30 hops
 
  1    <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    <1 ms    <1 ms    <1 ms  10.200.21.2
  4    1 ms    <1 ms    <1 ms  172.39.39.30
  5  761 ms    1 ms    1 ms  172.30.1.4
 
Trace complete.
}}
 
==== Routing path verification from the T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 to the external client VM ====
 
This is an EGRESS trafficflow.
The expectation is that the traffic towards T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 will go trough the DC1 routers.
The reason for this is that Locale ID is configured and tweaked.
 
{{console|body=
##bl##root@Web01:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
1  172.30.1.254 (172.30.1.254)  0.209 ms  0.086 ms  0.091 ms
2  172.39.39.17 (172.39.39.17)  0.612 ms  0.603 ms  0.594 ms
3  10.100.21.1 (10.100.21.1)  1.129 ms  1.234 ms  1.184 ms
4  10.11.11.50 (10.11.11.50)  1.583 ms * *
root@Web01:~#
}}
 
{{console|body=
##bl##root@Web02:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
1  172.30.1.254 (172.30.1.254)  0.223 ms  0.098 ms  0.107 ms
2  172.39.39.17 (172.39.39.17)  0.559 ms  0.564 ms  0.546 ms
3  10.100.21.1 (10.100.21.1)  1.179 ms  1.226 ms  1.221 ms
4  10.11.11.50 (10.11.11.50)  2.281 ms * *
root@Web02:~#
}}
 
{{console|body=
##bl##root@Web03:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
1  172.30.1.254 (172.30.1.254)  0.250 ms  0.126 ms  0.130 ms
2  * * 172.39.39.17 (172.39.39.17)  0.991 ms
3  10.100.19.1 (10.100.19.1)  1.303 ms  1.323 ms  1.350 ms
4  10.11.11.50 (10.11.11.50)  1.345 ms * *
root@Web03:~#
}}
 
{{console|body=
##bl##root@Web04:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
1  172.30.1.254 (172.30.1.254)  0.147 ms  0.093 ms  0.079 ms
2  * * 172.39.39.17 (172.39.39.17)  1.076 ms
3  10.100.19.1 (10.100.19.1)  1.951 ms  1.948 ms  2.100 ms
4  10.11.11.50 (10.11.11.50)  1.799 ms * *
root@Web04:~#
}}
 
Lets now put the Locale ID back to how it was and have the egress traffic exiting locally again.
What happens in case of any failures on DC2.
Does DC1 takes over the path for the DC2 workloads when the ESG-B goes down automatically?
 
=== Take ESG-B down and verify the paths ===
 
After I have verified if the locale ID is working as it should I did a power-off on the ESG-B to see what happens.
The expectation is that all the traffic that was exiting the DC2 WAN routers will now go trough the DC1 WAN routers.
After the power-off of the ESG-B the route was not rerouted automatically.
A change of the Locale ID of the DC2 compute cluster (manually) forced a re-route to the DC1 WAN routers.

Revision as of 19:59, 12 January 2024

Deploying an automated nested lab with the size and scale I am doing, requires proper IP address management and documentation. Typically I would do this using excel sheets. There are much smarter ways to do IP address management and there are paid solutions so-called IPAM that also offer other services like DNS for example.

While I was upgrading my lab and perform management I remembered that I have been using another web-based tool in the past called PHPIPAM. The code can be downloaded here.

And the demo can be found here.

I decided to re-install and registered all my IP addresses including the ones I am using for the nested automated labs.

Below you will see a screenshot of the page where I registered all subnets I used:

Screenshot 1550.png

As an example I also included the registered IP addresses in the "LX-APP-MGMT-11" segment.

Screenshot 1551.png