Routing with NSX using multiple sites and Routing with NSX using multiple sites using Locale ID (with eBGP all the way): Difference between pages

Revision as of 15:44, 12 January 2024

Routing with NSX using multiple sites using Locale ID (with eBGP all the way)

Introduction

In one of my previous articles I demonstrated the routing between two Data Centers with using eBGP and iBGP. These previous articles focussed on using only one path northbound as the primary exit point. So workloads that are in DC1 and DC2 will either choose the DC1 WAN or the DC2 WAN as the primary path (where DC1 is always prefered and DC2 is always secondary)

This article will be about doing almost the same, but now with the use of the NSX Locale ID feature. With this feature the workloads that are in DC1 will exit trough the DC1 WAN and the workloads that are in DC2 will exit trough the DC2 WAN.

Diagram

The following diagram will be used for our setup.

File:LAB100 - NSX Routing based on eBGP and locale ID.pdf

Components

I am still using the same componenents as in this previous article but in this article we will actively use the UDLR-02 and leave the UDLR-1 alone.

Use-Cases

I have thought of the following use-cases below.

As it is not possible to test active/passive and active/active with one UDLR I am using two UDLR instances here.

For testing purpose, I will use three tenants with each having their own Web, App and DB tiers.

Tenant one and two will be using the first UDLR (UDLR-01) with active/passive site egress. Tenant three will be using the second UDLR (UDLR-02) with active/active site egress.

Tenant three

Tenant three will have workloads in DC1 and in DC2 and because it is using local egress the traffic will exit from the Cisco CSR1000V routers local to that site. When the primary site is down the traffic should exit from the secondary site and when the secondary is down traffic should exit from the primary.

Routing protocol options

All possible routing options are described in the previous article.

The routing option in this article will be:

Option 1

eBGP peering between the UDLR and the ESG’s
eBGP peering between the ESG’s and the external routers
eBGP peering between the external routers and the CORE

Option 1

Because of the many options available in the setup I want to start with the following: Tenant 3 workloads with the use of UDLR-02 and option 1 routing.

UDLR configuration

When Locale ID is used the UDLR Control VM needs to be deployed in both DC's. After the deployment is done on the primary site we need to browse to the secondary NSX Manager and manually deploy an instance of the UDLR control VM in DC2. When this is done some DC2 site specific configurations are needed.

DC1 and DC2 UDLR-02 instances will have their own interfaces with own IP addresses
DC1 and DC2 UDLR-02 instances will have their own eBGP peerings
DC1 and DC2 UDLR-02 instances will have their own site (or NSX Manager) specific Locale ID

DC1

Manage --> Firewall

Make sure the firewall is disabled

Get the Locale ID from the NSX Manager

Manage --> Routing --> Global Configuration

Configure Locale ID and make sure this is the same as the NSX Manager
Enable ECMP
Make sure that the default gateway is not configured
Configure a router ID

Manage --> Routing --> BGP

Enable BGP
Configure the Local AS
Disable Graceful Restart
Configure the BGP peers towards ESG-A

Manage --> Routing --> Route Redistribution

Enable route redistribution for BGP
Specify what you allow to redistribute

DC2

Make sure the firewall is disabled

Get the Locale ID from the NSX Manager

Manage --> Routing --> Global Configuration

Configure Locale ID and make sure this is the same as the NSX Manager
Enable ECMP
Make sure that the default gateway is not configured
Configure a router ID

Manage --> Routing --> BGP

Enable BGP
Configure the Local AS
Disable Graceful Restart
Configure the BGP peers towards ESG-A

Manage --> Routing --> Route Redistribution

Enable route redistribution for BGP
Specify what you allow to redistribute

Compute Cluster configuration

The same locale ID (per site) also need to be configured on the Compute Clusters. We need to verify this if this happened automatically and if this is not the case we need to configure this manually.

DC1

DC2

ESG configuration

DC1

Manage --> Firewall

Make sure the firewall is disabled

Manage --> Routing --> Global Configuration

Enable ECMP
Make sure that the default gateway is not configured
Configure a router ID

Manage --> Routing --> BGP

Enable BGP
Configure the Local AS
Disable Graceful Restart
Configure the BGP peers towards the external routers and the UDLR.

Manage --> Routing --> Route Redistribution

Enable route redistribution for BGP
Specify what you allow to redistribute

DC2

Manage --> Firewall

Make sure the firewall is disabled

Manage --> Routing --> Global Configuration

Enable ECMP
Make sure that the default gateway is not configured
Configure a router ID

Manage --> Routing --> BGP

Enable BGP
Configure the Local AS
Disable Graceful Restart
Configure the BGP peers towards the external routers and the UDLR

Manage --> Routing --> Route Redistribution

Enable route redistribution for BGP
Specify what you allow to redistribute

Cisco1000V configuration

DC1

!
root ##bl##hostname rt-a-01
!
router bgp 65511
 bgp router-id 10.11.11.31
 bgp log-neighbor-changes
 neighbor 10.11.11.253 remote-as 65510
 neighbor 10.11.11.253 description CS01
 neighbor 10.100.19.2 remote-as 65521
 neighbor 10.100.19.2 description ESG-A
 !
 address-family ipv4
  neighbor 10.11.11.253 activate
  neighbor 10.100.19.2 activate
 exit-address-family
!
ip route 10.200.19.0 255.255.255.0 10.11.11.253
ip route 10.200.21.0 255.255.255.0 10.11.11.253
!

!
root ##bl##hostname rt-a-02
!
router bgp 65511
 bgp router-id 10.11.11.32
 bgp log-neighbor-changes
 neighbor 10.11.11.253 remote-as 65510
 neighbor 10.11.11.253 description CS01
 neighbor 10.100.21.2 remote-as 65521
 neighbor 10.100.21.2 description ESG-A
 !
 address-family ipv4
  neighbor 10.11.11.253 activate
  neighbor 10.100.21.2 activate
 exit-address-family
!
ip route 10.200.19.0 255.255.255.0 10.11.11.253
ip route 10.200.21.0 255.255.255.0 10.11.11.253
!

DC2

!
root ##bl##hostname rt-b-01
!
router bgp 65512
 bgp router-id 10.11.11.33
 bgp log-neighbor-changes
 neighbor 10.11.11.253 remote-as 65510
 neighbor 10.11.11.253 description CS01
 neighbor 10.200.19.2 remote-as 65522
 neighbor 10.200.19.2 description ESG-B
 !
 address-family ipv4
  neighbor 10.11.11.253 activate
  neighbor 10.200.19.2 activate
 exit-address-family
!
ip route 10.100.19.0 255.255.255.0 10.11.11.253
ip route 10.100.21.0 255.255.255.0 10.11.11.253
!

!
root ##bl##hostname rt-b-02
!
router bgp 65512
 bgp router-id 10.11.11.34
 bgp log-neighbor-changes
 neighbor 10.11.11.253 remote-as 65510
 neighbor 10.11.11.253 description CS01
 neighbor 10.200.21.2 remote-as 65522
 neighbor 10.200.21.2 description ESG-B
 !
 address-family ipv4
  neighbor 10.11.11.253 activate
  neighbor 10.200.21.2 activate
 exit-address-family
!
ip route 10.100.19.0 255.255.255.0 10.11.11.253
ip route 10.100.21.0 255.255.255.0 10.11.11.253
!

Core configuration

!
root ##bl##hostname cs-01
!
router bgp 65510
 bgp router-id 10.11.11.253
 bgp log-neighbor-changes
 neighbor 10.11.11.31 remote-as 65511
 neighbor 10.11.11.31 description RT-A-01
 neighbor 10.11.11.32 remote-as 65511
 neighbor 10.11.11.32 description RT-A-02
 neighbor 10.11.11.33 remote-as 65512
 neighbor 10.11.11.33 description RT-B-01
 neighbor 10.11.11.34 remote-as 65512
 neighbor 10.11.11.34 description RT-B-02
 !
 address-family ipv4
  network 10.11.11.0 mask 255.255.255.0
  neighbor 10.11.11.31 activate
  neighbor 10.11.11.31 next-hop-self
  neighbor 10.11.11.32 activate
  neighbor 10.11.11.32 next-hop-self
  neighbor 10.11.11.33 activate
  neighbor 10.11.11.33 next-hop-self
  neighbor 10.11.11.34 activate
  neighbor 10.11.11.34 next-hop-self
  no auto-summary
 exit-address-family
!

UDLR route peering verification

DC1

DC2

root ##bl##NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0> show ip bgp neighbors

BGP neighbor is 172.39.39.33,   remote AS 65522,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
         Route refresh: advertised and received
         Address family IPv4 Unicast:advertised and received
         Graceful restart Capability:none
                 Restart remain time: 0
Received 3194 messages, Sent 4728 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
         Index 1 Identifier 0x3740211c
         Route refresh request:received 0 sent 0
         Prefixes received 13 sent 5 advertised 5
Connections established 1, dropped 2468
Local host: 172.39.39.45, Local port: 179
Remote host: 172.39.39.33, Remote port: 53355

NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0>

ESG route peering verification

DC1

root ##bl##REGA-ESG01-0> show ip bgp neighbors

BGP neighbor is 10.100.19.1,   remote AS 65511,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
         Route refresh: advertised and received
         Address family IPv4 Unicast:advertised and received
         Graceful restart Capability:none
                 Restart remain time: 0
Received 2203 messages, Sent 2301 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
         Index 1 Identifier 0xa87fda9c
         Route refresh request:received 0 sent 0
         Prefixes received 3 sent 14 advertised 14
Connections established 1, dropped 1
Local host: 10.100.19.2, Local port: 12895
Remote host: 10.100.19.1, Remote port: 179


BGP neighbor is 10.100.21.1,   remote AS 65511,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
         Route refresh: advertised and received
         Address family IPv4 Unicast:advertised and received
         Graceful restart Capability:none
                 Restart remain time: 0
Received 2201 messages, Sent 2300 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
         Index 2 Identifier 0xa87fda9c
         Route refresh request:received 0 sent 0
         Prefixes received 3 sent 14 advertised 14
Connections established 1, dropped 1
Local host: 10.100.21.2, Local port: 31006
Remote host: 10.100.21.1, Remote port: 179


BGP neighbor is 172.39.39.13,   remote AS 65530,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
         Route refresh: advertised and received
         Address family IPv4 Unicast:advertised and received
         Graceful restart Capability:none
                 Restart remain time: 0
Received 2340 messages, Sent 2323 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
         Index 3 Identifier 0xa87fda9c
         Route refresh request:received 0 sent 0
         Prefixes received 7 sent 7 advertised 7
Connections established 2, dropped 42
Local host: 172.39.39.1, Local port: 13445
Remote host: 172.39.39.13, Remote port: 179


BGP neighbor is 172.39.39.29,   remote AS 65530,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
         Route refresh: advertised and received
         Address family IPv4 Unicast:advertised and received
         Graceful restart Capability:none
                 Restart remain time: 0
Received 2286 messages, Sent 2282 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
         Index 4 Identifier 0xa87fda9c
         Route refresh request:received 0 sent 0
         Prefixes received 5 sent 7 advertised 7
Connections established 1, dropped 1
Local host: 172.39.39.17, Local port: 37920
Remote host: 172.39.39.29, Remote port: 179

REGA-ESG01-0>

DC2

root ##bl##REGB-ESG01-0> show ip bgp neighbors

BGP neighbor is 10.200.19.1,   remote AS 65512,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
         Route refresh: advertised and received
         Address family IPv4 Unicast:advertised and received
         Graceful restart Capability:none
                 Restart remain time: 0
Received 2192 messages, Sent 2287 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
         Index 1 Identifier 0x637a1aec
         Route refresh request:received 0 sent 0
         Prefixes received 9 sent 8 advertised 8
Connections established 1, dropped 1
Local host: 10.200.19.2, Local port: 57360
Remote host: 10.200.19.1, Remote port: 179


BGP neighbor is 10.200.21.1,   remote AS 65512,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
         Route refresh: advertised and received
         Address family IPv4 Unicast:advertised and received
         Graceful restart Capability:none
                 Restart remain time: 0
Received 2191 messages, Sent 2280 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
         Index 2 Identifier 0x637a1aec
         Route refresh request:received 0 sent 0
         Prefixes received 9 sent 8 advertised 8
Connections established 1, dropped 1
Local host: 10.200.21.2, Local port: 17297
Remote host: 10.200.21.1, Remote port: 179


BGP neighbor is 172.39.39.13,   remote AS 65530,
BGP state = Idle, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
         Route refresh: None
         Address family none
         Graceful restart Capability:none
                 Restart remain time: 0
Received 15751 messages, Sent 11421 messages
Default minimum time between advertisement runs is 30 seconds
For Address family none
         Index 3 Identifier 0x637a1aec
         Route refresh request:received 0 sent 0
         Prefixes received 0 sent 0 advertised 0
Connections established 0, dropped 15751
Local host: 10.200.21.2, Local port: 0
Remote host: 172.39.39.13, Remote port: 0


BGP neighbor is 172.39.39.45,   remote AS 65530,
BGP state = Established, up
Hold time is 180, Keep alive interval is 60 seconds
Neighbor capabilities:
         Route refresh: advertised and received
         Address family IPv4 Unicast:advertised and received
         Graceful restart Capability:none
                 Restart remain time: 0
Received 2278 messages, Sent 2279 messages
Default minimum time between advertisement runs is 30 seconds
For Address family IPv4 Unicast:advertised and received
         Index 4 Identifier 0x637a1aec
         Route refresh request:received 0 sent 0
         Prefixes received 5 sent 13 advertised 13
Connections established 1, dropped 1
Local host: 172.39.39.33, Local port: 53355
Remote host: 172.39.39.45, Remote port: 179

REGB-ESG01-0>

Cisco1000V route peering verification

DC1

root ##bl##rt-a-01#show ip bgp summary
BGP router identifier 10.11.11.31, local AS number 65511
BGP table version is 54, main routing table version 54
17 network entries using 4216 bytes of memory
21 path entries using 2520 bytes of memory
3/3 BGP path/bestpath attribute entries using 720 bytes of memory
3 BGP AS-PATH entries using 88 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 7544 total bytes of memory
BGP activity 30/13 prefixes, 47/26 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.11.11.253    4        65510    2218    2206       54    0    0 1d09h           7
10.100.19.2     4        65521    2304    2205       54    0    0 1d09h          14
rt-a-01#

root ##bl##rt-a-02#show ip bgp summary
BGP router identifier 10.11.11.32, local AS number 65511
BGP table version is 54, main routing table version 54
17 network entries using 4216 bytes of memory
21 path entries using 2520 bytes of memory
3/3 BGP path/bestpath attribute entries using 720 bytes of memory
3 BGP AS-PATH entries using 88 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 7544 total bytes of memory
BGP activity 35/18 prefixes, 47/26 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.11.11.253    4        65510    2209    2210       54    0    0 1d09h           7
10.100.21.2     4        65521    2303    2204       54    0    0 1d09h          14
rt-a-02#

DC2

root ##bl##rt-b-01#show ip bgp summary
BGP router identifier 10.11.11.33, local AS number 65512
BGP table version is 76, main routing table version 76
17 network entries using 4216 bytes of memory
19 path entries using 2280 bytes of memory
3/3 BGP path/bestpath attribute entries using 720 bytes of memory
3 BGP AS-PATH entries using 88 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 7304 total bytes of memory
BGP activity 30/13 prefixes, 50/31 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.11.11.253    4        65510    5385    5387       76    0    0 3d09h          11
10.200.19.2     4        65522    2290    2194       76    0    0 1d09h           8
rt-b-01#

root ##bl##rt-b-02#show ip bgp summary
BGP router identifier 10.11.11.34, local AS number 65512
BGP table version is 49, main routing table version 49
17 network entries using 4216 bytes of memory
19 path entries using 2280 bytes of memory
3/3 BGP path/bestpath attribute entries using 720 bytes of memory
3 BGP AS-PATH entries using 88 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 7304 total bytes of memory
BGP activity 17/0 prefixes, 35/16 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.11.11.253    4        65510    2210    2204       49    0    0 1d09h          11
10.200.21.2     4        65522    2284    2195       49    0    0 1d09h           8
rt-b-02#

Core route peering verification

root ##bl##cs-01#show ip bgp summary 
BGP router identifier 10.11.11.253, local AS number 65510
BGP table version is 83, main routing table version 83
17 network entries using 2312 bytes of memory
45 path entries using 2340 bytes of memory
3/3 BGP path/bestpath attribute entries using 372 bytes of memory
2 BGP AS-PATH entries using 48 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 5072 total bytes of memory
BGP activity 30/13 prefixes, 109/64 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.11.11.31     4        65511    2209    2221       83    0    0 1d09h          14
10.11.11.32     4        65511    2212    2212       83    0    0 1d09h          14
10.11.11.33     4        65512    5389    5387       83    0    0 3d09h           8
10.11.11.34     4        65512    2205    2210       83    0    0 1d09h           8
cs-01#

UDLR routing tables

DC1

DC2

root ##bl##NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0> show ip route

Codes: O - OSPF derived, i - IS-IS derived, B - BGP derived,
C - connected, S - static, L1 - IS-IS level-1, L2 - IS-IS level-2,
IA - OSPF inter area, E1 - OSPF external type 1, E2 - OSPF external type 2,
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

Total number of routes: 17

B       10.11.11.0/24        [20/0]        via 172.39.39.33
B       10.100.19.0/24       [20/0]        via 172.39.39.33
B       10.100.21.0/24       [20/0]        via 172.39.39.33
B       10.200.19.0/24       [20/0]        via 172.39.39.33
B       10.200.21.0/24       [20/0]        via 172.39.39.33
B       172.20.1.0/24        [20/0]        via 172.39.39.33
B       172.20.2.0/24        [20/0]        via 172.39.39.33
B       172.20.3.0/24        [20/0]        via 172.39.39.33
B       172.20.8.0/24        [20/0]        via 172.39.39.33
B       172.20.9.0/24        [20/0]        via 172.39.39.33
B       172.20.10.0/24       [20/0]        via 172.39.39.33
C       172.30.1.0/24        [0/0]         via 172.30.1.254
C       172.30.2.0/24        [0/0]         via 172.30.2.254
C       172.30.3.0/24        [0/0]         via 172.30.3.254
B       172.39.39.0/28       [20/0]        via 172.39.39.33
C       172.39.39.16/28      [0/0]         via 172.39.39.30
C       172.39.39.32/28      [0/0]         via 172.39.39.45
NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0>

ESG routing tables

DC1

root ##bl##REGA-ESG01-0> show ip route

Codes: O - OSPF derived, i - IS-IS derived, B - BGP derived,
C - connected, S - static, L1 - IS-IS level-1, L2 - IS-IS level-2,
IA - OSPF inter area, E1 - OSPF external type 1, E2 - OSPF external type 2,
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

Total number of routes: 17

B       10.11.11.0/24        [20/0]        via 10.100.19.1
B       10.11.11.0/24        [20/0]        via 10.100.21.1
C       10.100.19.0/24       [0/0]         via 10.100.19.2
C       10.100.21.0/24       [0/0]         via 10.100.21.2
B       10.200.19.0/24       [20/0]        via 10.100.19.1
B       10.200.19.0/24       [20/0]        via 10.100.21.1
B       10.200.21.0/24       [20/0]        via 10.100.19.1
B       10.200.21.0/24       [20/0]        via 10.100.21.1
B       172.20.1.0/24        [20/0]        via 172.39.39.14
B       172.20.2.0/24        [20/0]        via 172.39.39.14
B       172.20.3.0/24        [20/0]        via 172.39.39.14
B       172.20.8.0/24        [20/0]        via 172.39.39.14
B       172.20.9.0/24        [20/0]        via 172.39.39.14
B       172.20.10.0/24       [20/0]        via 172.39.39.14
B       172.30.1.0/24        [20/0]        via 172.39.39.30
B       172.30.2.0/24        [20/0]        via 172.39.39.30
B       172.30.3.0/24        [20/0]        via 172.39.39.30
C       172.39.39.0/28       [0/0]         via 172.39.39.1
C       172.39.39.16/28      [0/0]         via 172.39.39.17
B       172.39.39.32/28      [20/0]        via 172.39.39.30
REGA-ESG01-0>

DC2

root ##bl##REGB-ESG01-0> show ip route

Codes: O - OSPF derived, i - IS-IS derived, B - BGP derived,
C - connected, S - static, L1 - IS-IS level-1, L2 - IS-IS level-2,
IA - OSPF inter area, E1 - OSPF external type 1, E2 - OSPF external type 2,
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

Total number of routes: 17

B       10.11.11.0/24        [20/0]        via 10.200.19.1
B       10.11.11.0/24        [20/0]        via 10.200.21.1
B       10.100.19.0/24       [20/0]        via 10.200.19.1
B       10.100.19.0/24       [20/0]        via 10.200.21.1
B       10.100.21.0/24       [20/0]        via 10.200.19.1
B       10.100.21.0/24       [20/0]        via 10.200.21.1
C       10.200.19.0/24       [0/0]         via 10.200.19.2
C       10.200.21.0/24       [0/0]         via 10.200.21.2
B       172.20.1.0/24        [20/0]        via 10.200.19.1
B       172.20.1.0/24        [20/0]        via 10.200.21.1
B       172.20.2.0/24        [20/0]        via 10.200.19.1
B       172.20.2.0/24        [20/0]        via 10.200.21.1
B       172.20.3.0/24        [20/0]        via 10.200.19.1
B       172.20.3.0/24        [20/0]        via 10.200.21.1
B       172.20.8.0/24        [20/0]        via 10.200.19.1
B       172.20.8.0/24        [20/0]        via 10.200.21.1
B       172.20.9.0/24        [20/0]        via 10.200.19.1
B       172.20.9.0/24        [20/0]        via 10.200.21.1
B       172.20.10.0/24       [20/0]        via 10.200.19.1
B       172.20.10.0/24       [20/0]        via 10.200.21.1
B       172.30.1.0/24        [20/0]        via 172.39.39.46
B       172.30.2.0/24        [20/0]        via 172.39.39.46
B       172.30.3.0/24        [20/0]        via 172.39.39.46
C       172.39.39.0/28       [0/0]         via 172.39.39.2
B       172.39.39.16/28      [20/0]        via 172.39.39.46
C       172.39.39.32/28      [0/0]         via 172.39.39.33
REGB-ESG01-0>

Cisco1000V routing tables

DC1

root ##bl##rt-a-01#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 9 subnets, 2 masks
C        10.11.11.0/24 is directly connected, GigabitEthernet2
L        10.11.11.31/32 is directly connected, GigabitEthernet2
C        10.100.1.0/24 is directly connected, GigabitEthernet1
L        10.100.1.101/32 is directly connected, GigabitEthernet1
C        10.100.19.0/24 is directly connected, GigabitEthernet3
L        10.100.19.1/32 is directly connected, GigabitEthernet3
B        10.100.21.0/24 [20/0] via 10.100.19.2, 1d09h
S        10.200.19.0/24 [1/0] via 10.11.11.253
S        10.200.21.0/24 [1/0] via 10.11.11.253
      172.20.0.0/24 is subnetted, 6 subnets
B        172.20.1.0 [20/0] via 10.100.19.2, 1d09h
B        172.20.2.0 [20/0] via 10.100.19.2, 1d09h
B        172.20.3.0 [20/0] via 10.100.19.2, 1d09h
B        172.20.8.0 [20/0] via 10.100.19.2, 1d09h
B        172.20.9.0 [20/0] via 10.100.19.2, 1d09h
B        172.20.10.0 [20/0] via 10.100.19.2, 1d09h
      172.30.0.0/24 is subnetted, 3 subnets
B        172.30.1.0 [20/0] via 10.100.19.2, 1d09h
B        172.30.2.0 [20/0] via 10.100.19.2, 1d09h
B        172.30.3.0 [20/0] via 10.100.19.2, 1d09h
      172.39.0.0/28 is subnetted, 3 subnets
B        172.39.39.0 [20/0] via 10.100.19.2, 1d09h
B        172.39.39.16 [20/0] via 10.100.19.2, 1d09h
B        172.39.39.32 [20/0] via 10.100.19.2, 1d09h
rt-a-01#

root ##bl##rt-a-02#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 9 subnets, 2 masks
C        10.11.11.0/24 is directly connected, GigabitEthernet2
L        10.11.11.32/32 is directly connected, GigabitEthernet2
C        10.100.1.0/24 is directly connected, GigabitEthernet1
L        10.100.1.102/32 is directly connected, GigabitEthernet1
B        10.100.19.0/24 [20/0] via 10.100.21.2, 1d09h
C        10.100.21.0/24 is directly connected, GigabitEthernet3
L        10.100.21.1/32 is directly connected, GigabitEthernet3
S        10.200.19.0/24 [1/0] via 10.11.11.253
S        10.200.21.0/24 [1/0] via 10.11.11.253
      172.20.0.0/24 is subnetted, 6 subnets
B        172.20.1.0 [20/0] via 10.100.21.2, 1d09h
B        172.20.2.0 [20/0] via 10.100.21.2, 1d09h
B        172.20.3.0 [20/0] via 10.100.21.2, 1d09h
B        172.20.8.0 [20/0] via 10.100.21.2, 1d09h
B        172.20.9.0 [20/0] via 10.100.21.2, 1d09h
B        172.20.10.0 [20/0] via 10.100.21.2, 1d09h
      172.30.0.0/24 is subnetted, 3 subnets
B        172.30.1.0 [20/0] via 10.100.21.2, 1d09h
B        172.30.2.0 [20/0] via 10.100.21.2, 1d09h
B        172.30.3.0 [20/0] via 10.100.21.2, 1d09h
      172.39.0.0/28 is subnetted, 3 subnets
B        172.39.39.0 [20/0] via 10.100.21.2, 1d09h
B        172.39.39.16 [20/0] via 10.100.21.2, 1d09h
B        172.39.39.32 [20/0] via 10.100.21.2, 1d09h
rt-a-02#

DC2

root ##bl##rt-b-01#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 9 subnets, 2 masks
C        10.11.11.0/24 is directly connected, GigabitEthernet2
L        10.11.11.33/32 is directly connected, GigabitEthernet2
S        10.100.19.0/24 [1/0] via 10.11.11.253
S        10.100.21.0/24 [1/0] via 10.11.11.253
C        10.200.1.0/24 is directly connected, GigabitEthernet1
L        10.200.1.101/32 is directly connected, GigabitEthernet1
C        10.200.19.0/24 is directly connected, GigabitEthernet3
L        10.200.19.1/32 is directly connected, GigabitEthernet3
B        10.200.21.0/24 [20/0] via 10.200.19.2, 1d09h
      172.20.0.0/24 is subnetted, 6 subnets
B        172.20.1.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.2.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.3.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.8.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.9.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.10.0 [20/0] via 10.11.11.253, 1d09h
      172.30.0.0/24 is subnetted, 3 subnets
B        172.30.1.0 [20/0] via 10.200.19.2, 1d09h
B        172.30.2.0 [20/0] via 10.200.19.2, 1d09h
B        172.30.3.0 [20/0] via 10.200.19.2, 1d09h
      172.39.0.0/28 is subnetted, 3 subnets
B        172.39.39.0 [20/0] via 10.200.19.2, 1d09h
B        172.39.39.16 [20/0] via 10.200.19.2, 1d09h
B        172.39.39.32 [20/0] via 10.200.19.2, 1d09h
rt-b-01#

root ##bl##rt-b-02#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 9 subnets, 2 masks
C        10.11.11.0/24 is directly connected, GigabitEthernet2
L        10.11.11.34/32 is directly connected, GigabitEthernet2
S        10.100.19.0/24 [1/0] via 10.11.11.253
S        10.100.21.0/24 [1/0] via 10.11.11.253
C        10.200.1.0/24 is directly connected, GigabitEthernet1
L        10.200.1.102/32 is directly connected, GigabitEthernet1
B        10.200.19.0/24 [20/0] via 10.200.21.2, 1d09h
C        10.200.21.0/24 is directly connected, GigabitEthernet3
L        10.200.21.1/32 is directly connected, GigabitEthernet3
      172.20.0.0/24 is subnetted, 6 subnets
B        172.20.1.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.2.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.3.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.8.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.9.0 [20/0] via 10.11.11.253, 1d09h
B        172.20.10.0 [20/0] via 10.11.11.253, 1d09h
      172.30.0.0/24 is subnetted, 3 subnets
B        172.30.1.0 [20/0] via 10.200.21.2, 1d09h
B        172.30.2.0 [20/0] via 10.200.21.2, 1d09h
B        172.30.3.0 [20/0] via 10.200.21.2, 1d09h
      172.39.0.0/28 is subnetted, 3 subnets
B        172.39.39.0 [20/0] via 10.200.21.2, 1d09h
B        172.39.39.16 [20/0] via 10.200.21.2, 1d09h
B        172.39.39.32 [20/0] via 10.200.21.2, 1d09h
rt-b-02#

Core routing table

root ##bl##cs-01#show ip route bgp 
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override

Gateway of last resort is 10.11.11.254 to network 0.0.0.0

      172.20.0.0/24 is subnetted, 6 subnets
B        172.20.1.0 [20/0] via 10.11.11.31, 1d09h
B        172.20.2.0 [20/0] via 10.11.11.31, 1d09h
B        172.20.3.0 [20/0] via 10.11.11.31, 1d09h
B        172.20.8.0 [20/0] via 10.11.11.31, 1d09h
B        172.20.9.0 [20/0] via 10.11.11.31, 1d09h
B        172.20.10.0 [20/0] via 10.11.11.31, 1d09h
      172.30.0.0/24 is subnetted, 3 subnets
B        172.30.1.0 [20/0] via 10.11.11.33, 1d09h
B        172.30.2.0 [20/0] via 10.11.11.33, 1d09h
B        172.30.3.0 [20/0] via 10.11.11.33, 1d09h
      172.39.0.0/28 is subnetted, 3 subnets
B        172.39.39.0 [20/0] via 10.11.11.31, 1d09h
B        172.39.39.16 [20/0] via 10.11.11.31, 1d09h
B        172.39.39.32 [20/0] via 10.11.11.34, 1d09h
cs-01#

UDLR BGP tables

DC1

DC2

root ##bl##NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0> show ip bgp

Status codes: s - suppressed, d - damped, > - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop       Metric  LocPrf  Weight AS Path
  > 10.11.11.0/24      172.39.39.33      0     100      60   65522 i
  > 10.100.19.0/24     172.39.39.33      0     100      60   65522 ?
  > 10.100.21.0/24     172.39.39.33      0     100      60   65522 ?
  > 10.200.19.0/24     172.39.39.33      0     100      60   65522 ?
  > 10.200.21.0/24     172.39.39.33      0     100      60   65522 ?
  > 172.20.1.0/24      172.39.39.33      0     100      60   65522 ?
  > 172.20.2.0/24      172.39.39.33      0     100      60   65522 ?
  > 172.20.3.0/24      172.39.39.33      0     100      60   65522 ?
  > 172.20.8.0/24      172.39.39.33      0     100      60   65522 ?
  > 172.20.9.0/24      172.39.39.33      0     100      60   65522 ?
  > 172.20.10.0/24     172.39.39.33      0     100      60   65522 ?
  > 172.30.1.0/24      0.0.0.0           0     100   32768   ?
  > 172.30.2.0/24      0.0.0.0           0     100   32768   ?
  > 172.30.3.0/24      0.0.0.0           0     100   32768   ?
  > 172.39.39.0/28     172.39.39.33      0     100      60   65522 ?
  > 172.39.39.16/28    0.0.0.0           0     100   32768   ?
    172.39.39.32/28    172.39.39.33      0     100      60   65522 ?
  > 172.39.39.32/28    0.0.0.0           0     100   32768   ?
NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0>

ESG BGP tables

DC1

root ##bl##REGA-ESG01-0> show ip bgp

Status codes: s - suppressed, d - damped, > - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop       Metric  LocPrf  Weight AS Path
  > 10.11.11.0/24      10.100.19.1       0     100      60   65511 65510 i
    10.11.11.0/24      10.100.21.1       0     100      60   65511 65510 i
  > 10.100.19.0/24     0.0.0.0           0     100   32768   ?
  > 10.100.21.0/24     0.0.0.0           0     100   32768   ?
  > 10.200.19.0/24     10.100.19.1       0     100      60   65511 65510 65512 65522 ?
    10.200.19.0/24     10.100.21.1       0     100      60   65511 65510 65512 65522 ?
  > 10.200.21.0/24     10.100.19.1       0     100      60   65511 65510 65512 65522 ?
    10.200.21.0/24     10.100.21.1       0     100      60   65511 65510 65512 65522 ?
  > 172.20.1.0/24      172.39.39.14      0     100      60   65530 ?
  > 172.20.2.0/24      172.39.39.14      0     100      60   65530 ?
  > 172.20.3.0/24      172.39.39.14      0     100      60   65530 ?
  > 172.20.8.0/24      172.39.39.14      0     100      60   65530 ?
  > 172.20.9.0/24      172.39.39.14      0     100      60   65530 ?
  > 172.20.10.0/24     172.39.39.14      0     100      60   65530 ?
  > 172.30.1.0/24      172.39.39.30      0     100      60   65530 ?
  > 172.30.2.0/24      172.39.39.30      0     100      60   65530 ?
  > 172.30.3.0/24      172.39.39.30      0     100      60   65530 ?
    172.39.39.0/28     172.39.39.14      0     100      60   65530 ?
  > 172.39.39.0/28     0.0.0.0           0     100   32768   ?
    172.39.39.16/28    172.39.39.30      0     100      60   65530 ?
  > 172.39.39.16/28    0.0.0.0           0     100   32768   ?
  > 172.39.39.32/28    172.39.39.30      0     100      60   65530 ?
REGA-ESG01-0>

DC2

root ##bl##REGB-ESG01-0> show ip bgp

Status codes: s - suppressed, d - damped, > - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop       Metric  LocPrf  Weight AS Path
  > 10.11.11.0/24      10.200.19.1       0     100      60   65512 65510 i
    10.11.11.0/24      10.200.21.1       0     100      60   65512 65510 i
  > 10.100.19.0/24     10.200.19.1       0     100      60   65512 65510 65511 65521 ?
    10.100.19.0/24     10.200.21.1       0     100      60   65512 65510 65511 65521 ?
  > 10.100.21.0/24     10.200.19.1       0     100      60   65512 65510 65511 65521 ?
    10.100.21.0/24     10.200.21.1       0     100      60   65512 65510 65511 65521 ?
  > 10.200.19.0/24     0.0.0.0           0     100   32768   ?
  > 10.200.21.0/24     0.0.0.0           0     100   32768   ?
  > 172.20.1.0/24      10.200.19.1       0     100      60   65512 65510 65511 65521 ?
    172.20.1.0/24      10.200.21.1       0     100      60   65512 65510 65511 65521 ?
  > 172.20.2.0/24      10.200.19.1       0     100      60   65512 65510 65511 65521 ?
    172.20.2.0/24      10.200.21.1       0     100      60   65512 65510 65511 65521 ?
  > 172.20.3.0/24      10.200.19.1       0     100      60   65512 65510 65511 65521 ?
    172.20.3.0/24      10.200.21.1       0     100      60   65512 65510 65511 65521 ?
  > 172.20.8.0/24      10.200.19.1       0     100      60   65512 65510 65511 65521 ?
    172.20.8.0/24      10.200.21.1       0     100      60   65512 65510 65511 65521 ?
  > 172.20.9.0/24      10.200.19.1       0     100      60   65512 65510 65511 65521 ?
    172.20.9.0/24      10.200.21.1       0     100      60   65512 65510 65511 65521 ?
  > 172.20.10.0/24     10.200.19.1       0     100      60   65512 65510 65511 65521 ?
    172.20.10.0/24     10.200.21.1       0     100      60   65512 65510 65511 65521 ?
  > 172.30.1.0/24      172.39.39.46      0     100      60   65530 ?
  > 172.30.2.0/24      172.39.39.46      0     100      60   65530 ?
  > 172.30.3.0/24      172.39.39.46      0     100      60   65530 ?
  > 172.39.39.0/28     0.0.0.0           0     100   32768   ?
  > 172.39.39.16/28    172.39.39.46      0     100      60   65530 ?
    172.39.39.32/28    172.39.39.46      0     100      60   65530 ?
  > 172.39.39.32/28    0.0.0.0           0     100   32768   ?
REGB-ESG01-0>

Cisco1000V BGP tables

DC1

root ##bl##rt-a-01#show ip bgp
BGP table version is 54, local router ID is 10.11.11.31
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 r>  10.11.11.0/24    10.11.11.253             0             0 65510 i
 r>  10.100.19.0/24   10.100.19.2                            0 65521 ?
 *>  10.100.21.0/24   10.100.19.2                            0 65521 ?
 r>  10.200.19.0/24   10.11.11.253                           0 65510 65512 65522 ?
 r>  10.200.21.0/24   10.11.11.253                           0 65510 65512 65522 ?
 *>  172.20.1.0/24    10.100.19.2                            0 65521 ?
 *>  172.20.2.0/24    10.100.19.2                            0 65521 ?
 *>  172.20.3.0/24    10.100.19.2                            0 65521 ?
 *>  172.20.8.0/24    10.100.19.2                            0 65521 ?
 *>  172.20.9.0/24    10.100.19.2                            0 65521 ?
 *>  172.20.10.0/24   10.100.19.2                            0 65521 ?
 *>  172.30.1.0/24    10.100.19.2                            0 65521 ?
 *                    10.11.11.253                           0 65510 65512 65522 ?
     Network          Next Hop            Metric LocPrf Weight Path
 *>  172.30.2.0/24    10.100.19.2                            0 65521 ?
 *                    10.11.11.253                           0 65510 65512 65522 ?
 *>  172.30.3.0/24    10.100.19.2                            0 65521 ?
 *                    10.11.11.253                           0 65510 65512 65522 ?
 *>  172.39.39.0/28   10.100.19.2                            0 65521 ?
 *>  172.39.39.16/28  10.100.19.2                            0 65521 ?
 *>  172.39.39.32/28  10.100.19.2                            0 65521 ?
 *                    10.11.11.253                           0 65510 65512 65522 ?
rt-a-01#

root ##bl##rt-a-02#show ip bgp
BGP table version is 54, local router ID is 10.11.11.32
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 r>  10.11.11.0/24    10.11.11.253             0             0 65510 i
 *>  10.100.19.0/24   10.100.21.2                            0 65521 ?
 r>  10.100.21.0/24   10.100.21.2                            0 65521 ?
 r>  10.200.19.0/24   10.11.11.253                           0 65510 65512 65522 ?
 r>  10.200.21.0/24   10.11.11.253                           0 65510 65512 65522 ?
 *>  172.20.1.0/24    10.100.21.2                            0 65521 ?
 *>  172.20.2.0/24    10.100.21.2                            0 65521 ?
 *>  172.20.3.0/24    10.100.21.2                            0 65521 ?
 *>  172.20.8.0/24    10.100.21.2                            0 65521 ?
 *>  172.20.9.0/24    10.100.21.2                            0 65521 ?
 *>  172.20.10.0/24   10.100.21.2                            0 65521 ?
 *>  172.30.1.0/24    10.100.21.2                            0 65521 ?
 *                    10.11.11.253                           0 65510 65512 65522 ?
     Network          Next Hop            Metric LocPrf Weight Path
 *>  172.30.2.0/24    10.100.21.2                            0 65521 ?
 *                    10.11.11.253                           0 65510 65512 65522 ?
 *>  172.30.3.0/24    10.100.21.2                            0 65521 ?
 *                    10.11.11.253                           0 65510 65512 65522 ?
 *>  172.39.39.0/28   10.100.21.2                            0 65521 ?
 *>  172.39.39.16/28  10.100.21.2                            0 65521 ?
 *>  172.39.39.32/28  10.100.21.2                            0 65521 ?
 *                    10.11.11.253                           0 65510 65512 65522 ?
rt-a-02#

DC2

root ##bl##rt-b-01#show ip bgp
BGP table version is 76, local router ID is 10.11.11.33
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 r>  10.11.11.0/24    10.11.11.253             0             0 65510 i
 r>  10.100.19.0/24   10.11.11.253                           0 65510 65511 65521 ?
 r>  10.100.21.0/24   10.11.11.253                           0 65510 65511 65521 ?
 r>  10.200.19.0/24   10.200.19.2                            0 65522 ?
 *>  10.200.21.0/24   10.200.19.2                            0 65522 ?
 *>  172.20.1.0/24    10.11.11.253                           0 65510 65511 65521 ?
 *>  172.20.2.0/24    10.11.11.253                           0 65510 65511 65521 ?
 *>  172.20.3.0/24    10.11.11.253                           0 65510 65511 65521 ?
 *>  172.20.8.0/24    10.11.11.253                           0 65510 65511 65521 ?
     Network          Next Hop            Metric LocPrf Weight Path
 *>  172.20.9.0/24    10.11.11.253                           0 65510 65511 65521 ?
 *>  172.20.10.0/24   10.11.11.253                           0 65510 65511 65521 ?
 *>  172.30.1.0/24    10.200.19.2                            0 65522 ?
 *>  172.30.2.0/24    10.200.19.2                            0 65522 ?
 *>  172.30.3.0/24    10.200.19.2                            0 65522 ?
 *>  172.39.39.0/28   10.200.19.2                            0 65522 ?
 *                    10.11.11.253                           0 65510 65511 65521 ?
 *>  172.39.39.16/28  10.200.19.2                            0 65522 ?
 *                    10.11.11.253                           0 65510 65511 65521 ?
 *>  172.39.39.32/28  10.200.19.2                            0 65522 ?
rt-b-01#

root ##bl##rt-b-02#show ip bgp
BGP table version is 49, local router ID is 10.11.11.34
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 r>  10.11.11.0/24    10.11.11.253             0             0 65510 i
 r>  10.100.19.0/24   10.11.11.253                           0 65510 65511 65521 ?
 r>  10.100.21.0/24   10.11.11.253                           0 65510 65511 65521 ?
 *>  10.200.19.0/24   10.200.21.2                            0 65522 ?
 r>  10.200.21.0/24   10.200.21.2                            0 65522 ?
 *>  172.20.1.0/24    10.11.11.253                           0 65510 65511 65521 ?
 *>  172.20.2.0/24    10.11.11.253                           0 65510 65511 65521 ?
 *>  172.20.3.0/24    10.11.11.253                           0 65510 65511 65521 ?
 *>  172.20.8.0/24    10.11.11.253                           0 65510 65511 65521 ?
     Network          Next Hop            Metric LocPrf Weight Path
 *>  172.20.9.0/24    10.11.11.253                           0 65510 65511 65521 ?
 *>  172.20.10.0/24   10.11.11.253                           0 65510 65511 65521 ?
 *>  172.30.1.0/24    10.200.21.2                            0 65522 ?
 *>  172.30.2.0/24    10.200.21.2                            0 65522 ?
 *>  172.30.3.0/24    10.200.21.2                            0 65522 ?
 *>  172.39.39.0/28   10.200.21.2                            0 65522 ?
 *                    10.11.11.253                           0 65510 65511 65521 ?
 *>  172.39.39.16/28  10.200.21.2                            0 65522 ?
 *                    10.11.11.253                           0 65510 65511 65521 ?
 *>  172.39.39.32/28  10.200.21.2                            0 65522 ?
rt-b-02#

Core BGP tables

root ##bl##cs-01#show ip bgp
BGP table version is 83, local router ID is 10.11.11.253
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, x best-external
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 10.11.11.0/24    0.0.0.0                  0         32768 i
r  10.100.19.0/24   10.11.11.32                            0 65511 65521 ?
r>                  10.11.11.31                            0 65511 65521 ?
r  10.100.21.0/24   10.11.11.32                            0 65511 65521 ?
r>                  10.11.11.31                            0 65511 65521 ?
r  10.200.19.0/24   10.11.11.33                            0 65512 65522 ?
r>                  10.11.11.34                            0 65512 65522 ?
r  10.200.21.0/24   10.11.11.33                            0 65512 65522 ?
r>                  10.11.11.34                            0 65512 65522 ?
*  172.20.1.0/24    10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.20.2.0/24    10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.20.3.0/24    10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.20.8.0/24    10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.20.9.0/24    10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.20.10.0/24   10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.30.1.0/24    10.11.11.31                            0 65511 65521 ?
*                   10.11.11.32                            0 65511 65521 ?
*                   10.11.11.34                            0 65512 65522 ?
*>                  10.11.11.33                            0 65512 65522 ?
*  172.30.2.0/24    10.11.11.31                            0 65511 65521 ?
*                   10.11.11.32                            0 65511 65521 ?
*                   10.11.11.34                            0 65512 65522 ?
*>                  10.11.11.33                            0 65512 65522 ?
*  172.30.3.0/24    10.11.11.31                            0 65511 65521 ?
*                   10.11.11.32                            0 65511 65521 ?
*                   10.11.11.34                            0 65512 65522 ?
*>                  10.11.11.33                            0 65512 65522 ?
*  172.39.39.0/28   10.11.11.33                            0 65512 65522 ?
*                   10.11.11.34                            0 65512 65522 ?
*                   10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.39.39.16/28  10.11.11.34                            0 65512 65522 ?
*                   10.11.11.33                            0 65512 65522 ?
*                   10.11.11.32                            0 65511 65521 ?
*>                  10.11.11.31                            0 65511 65521 ?
*  172.39.39.32/28  10.11.11.31                            0 65511 65521 ?
*                   10.11.11.32                            0 65511 65521 ?
*                   10.11.11.33                            0 65512 65522 ?
*>                  10.11.11.34                            0 65512 65522 ?
cs-01#

Routing path verifications

Routing path verification from the external client VM to T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4

This is an INGRESS trafficflow. As we did not make any configuration changes to manipulate the INGRESS trafficflow the INGRESS traffic can either choose to select the DC1 path or the DC2 path. This is currently determined by BGP.

root ##bl##C:\Users\Administrator>tracert 172.30.1.1

Tracing route to 172.30.1.1 over a maximum of 30 hops

  1     1 ms     1 ms     1 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3     1 ms    <1 ms    <1 ms  10.200.21.2
  4     1 ms    <1 ms    <1 ms  172.39.39.46
  5     1 ms     1 ms     1 ms  172.30.1.1

Trace complete.

root ##bl##C:\Users\Administrator>tracert 172.30.1.2

Tracing route to 172.30.1.2 over a maximum of 30 hops

  1     2 ms     1 ms     1 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    <1 ms    <1 ms    <1 ms  10.200.21.2
  4    <1 ms    <1 ms    <1 ms  172.39.39.46
  5   944 ms     1 ms    <1 ms  172.30.1.2

Trace complete.

root ##bl##C:\Users\Administrator>tracert 172.30.1.3

Tracing route to 172.30.1.3 over a maximum of 30 hops

  1     1 ms     1 ms     6 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    <1 ms    <1 ms    <1 ms  10.200.21.2
  4    <1 ms    <1 ms    <1 ms  172.39.39.46
  5   723 ms     1 ms    <1 ms  172.30.1.3

Trace complete.

root ##bl##C:\Users\Administrator>tracert 172.30.1.4

Tracing route to 172.30.1.4 over a maximum of 30 hops

  1     1 ms     1 ms     4 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    <1 ms    <1 ms    <1 ms  10.200.21.2
  4    <1 ms    <1 ms    <1 ms  172.39.39.46
  5    82 ms     1 ms    <1 ms  172.30.1.4

Trace complete.

Routing path verification from the T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 to the external client VM

This is an EGRESS trafficflow. The expectation is that the traffic towards T3-WEB-1 and T3-WEB-2 will fo trough the DC1 routers and the traffic towards T3-WEB-3 and T3-WEB-4 will fo trough the DC2 routers. The reason for this is that Locale ID is configured.

root ##bl##root@Web01:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
 1  172.30.1.254 (172.30.1.254)  0.082 ms  2.689 ms  2.687 ms
 2  172.39.39.17 (172.39.39.17)  0.666 ms  0.527 ms  0.524 ms
 3  10.100.21.1 (10.100.21.1)  1.067 ms  1.091 ms  1.072 ms
 4  10.11.11.50 (10.11.11.50)  1.347 ms * *
root@Web01:~#

root ##bl##root@Web02:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
 1  172.30.1.254 (172.30.1.254)  0.172 ms  0.149 ms  0.131 ms
 2  * * 172.39.39.17 (172.39.39.17)  0.455 ms
 3  10.100.21.1 (10.100.21.1)  0.758 ms  0.599 ms  0.494 ms
 4  10.11.11.50 (10.11.11.50)  0.918 ms * *
root@Web02:~#

root ##bl##root@Web03:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
 1  172.30.1.254 (172.30.1.254)  0.120 ms  0.109 ms  0.060 ms
 2  172.39.39.33 (172.39.39.33)  0.387 ms  0.345 ms  0.445 ms
 3  10.200.21.1 (10.200.21.1)  1.249 ms  1.371 ms  1.383 ms
 4  10.11.11.50 (10.11.11.50)  1.210 ms * *
root@Web03:~#

root ##bl##root@Web04:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
 1  172.30.1.254 (172.30.1.254)  0.109 ms  0.181 ms  0.132 ms
 2  172.39.39.33 (172.39.39.33)  0.471 ms  0.470 ms  0.466 ms
 3  10.200.21.1 (10.200.21.1)  0.900 ms  0.871 ms  0.868 ms
 4  10.11.11.50 (10.11.11.50)  1.063 ms * *
root@Web04:~#

Change the Locale ID manually on DC2 to match DC1

DC1

The locale ID will not change here and these settings are kept te same

DC2

Compute Cluster

Routing path verifications

Routing path verification from the external client VM to T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4

This is an INGRESS trafficflow. As we did not make any configuration changes to manipulate the INGRESS trafficflow the INGRESS traffic can either choose to select the DC1 path or the DC2 path. This is currently determined by BGP.

root ##bl##C:\Users\Administrator>tracert 172.30.1.1

Tracing route to 172.30.1.1 over a maximum of 30 hops

  1    <1 ms    <1 ms     3 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3     1 ms    <1 ms    <1 ms  10.200.21.2
  4   231 ms    <1 ms    <1 ms  172.39.39.30
  5   966 ms     1 ms    <1 ms  172.30.1.1

Trace complete.

root ##bl##C:\Users\Administrator>tracert 172.30.1.2

Tracing route to 172.30.1.2 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    <1 ms    <1 ms    <1 ms  10.200.21.2
  4     1 ms    <1 ms    <1 ms  172.39.39.30
  5     1 ms     1 ms    <1 ms  172.30.1.2

Trace complete.

root ##bl##C:\Users\Administrator>tracert 172.30.1.3

Tracing route to 172.30.1.3 over a maximum of 30 hops

  1    <1 ms    <1 ms     3 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    <1 ms    <1 ms    <1 ms  10.200.21.2
  4     1 ms    <1 ms    <1 ms  172.39.39.30
  5     2 ms    <1 ms    <1 ms  172.30.1.3

Trace complete.

root ##bl##C:\Users\Administrator>tracert 172.30.1.4

Tracing route to 172.30.1.4 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
  2    <1 ms    <1 ms    <1 ms  10.11.11.33
  3    <1 ms    <1 ms    <1 ms  10.200.21.2
  4     1 ms    <1 ms    <1 ms  172.39.39.30
  5   761 ms     1 ms     1 ms  172.30.1.4

Trace complete.

Routing path verification from the T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 to the external client VM

This is an EGRESS trafficflow. The expectation is that the traffic towards T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 will go trough the DC1 routers. The reason for this is that Locale ID is configured and tweaked.

root ##bl##root@Web01:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
 1  172.30.1.254 (172.30.1.254)  0.209 ms  0.086 ms  0.091 ms
 2  172.39.39.17 (172.39.39.17)  0.612 ms  0.603 ms  0.594 ms
 3  10.100.21.1 (10.100.21.1)  1.129 ms  1.234 ms  1.184 ms
 4  10.11.11.50 (10.11.11.50)  1.583 ms * *
root@Web01:~#

root ##bl##root@Web02:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
 1  172.30.1.254 (172.30.1.254)  0.223 ms  0.098 ms  0.107 ms
 2  172.39.39.17 (172.39.39.17)  0.559 ms  0.564 ms  0.546 ms
 3  10.100.21.1 (10.100.21.1)  1.179 ms  1.226 ms  1.221 ms
 4  10.11.11.50 (10.11.11.50)  2.281 ms * *
root@Web02:~#

root ##bl##root@Web03:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
 1  172.30.1.254 (172.30.1.254)  0.250 ms  0.126 ms  0.130 ms
 2  * * 172.39.39.17 (172.39.39.17)  0.991 ms
 3  10.100.19.1 (10.100.19.1)  1.303 ms  1.323 ms  1.350 ms
 4  10.11.11.50 (10.11.11.50)  1.345 ms * *
root@Web03:~#

root ##bl##root@Web04:~# traceroute 10.11.11.50
traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
 1  172.30.1.254 (172.30.1.254)  0.147 ms  0.093 ms  0.079 ms
 2  * * 172.39.39.17 (172.39.39.17)  1.076 ms
 3  10.100.19.1 (10.100.19.1)  1.951 ms  1.948 ms  2.100 ms
 4  10.11.11.50 (10.11.11.50)  1.799 ms * *
root@Web04:~#

Lets now put the Locale ID back to how it was and have the egress traffic exiting locally again. What happens in case of any failures on DC2. Does DC1 takes over the path for the DC2 workloads when the ESG-B goes down automatically?

Take ESG-B down and verify the paths

After I have verified if the locale ID is working as it should I did a power-off on the ESG-B to see what happens. The expectation is that all the traffic that was exiting the DC2 WAN routers will now go trough the DC1 WAN routers. After the power-off of the ESG-B the route was not rerouted automatically. A change of the Locale ID of the DC2 compute cluster (manually) forced a re-route to the DC1 WAN routers.

@@ Line 1: / Line 1: @@
+Routing with NSX using multiple sites using Locale ID (with eBGP all the way)
 == Introduction ==
-This article will be about routing using different protocols and routing domain setups.
+In one of my previous articles I demonstrated the routing between two Data Centers with using eBGP and iBGP.
-As you know with routing we advertise network prefixes so that the networks are known on other places on the network. In this process, we can influence the traffic flow when multiple routing/network paths are available.
+These previous articles focussed on using only one path northbound as the primary exit point.
-This can be done egress (from inside to outside – typically south to north) and ingress (from outside to inside – typically from north to south).
+So workloads that are in DC1 and DC2 will either choose the DC1 WAN or the DC2 WAN as the primary path (where DC1 is always prefered and DC2 is always secondary)
-To influence the path that is taken there are different methods per protocol possible.
+This article will be about doing almost the same, but now with the use of the NSX Locale ID feature.
+With this feature the workloads that are in DC1 will exit trough the DC1 WAN and the workloads that are in DC2 will exit trough the DC2 WAN.
-In this article, I will only discuss OSPF cost and BGP weight because these are the only mechanisms that are supported by the Edges of NSX.
+== Diagram ==
-I will also only talk about egress routing as ingress routing is usually influenced on other devices with different attributes / mechanisms.
-The general rule for OSPF cost is that the lower the cost is the more preferred the route is.
+The following diagram will be used for our setup.
-For BGP this is the higher the weight the more preferred the route is.
-== NSX Multisite deployments ==
+[[File:LAB100_-_NSX_Routing_based_on_eBGP_and_locale_ID.pdf]]
-When we are using vSphere together with NSX Multisite deployments are supported.
+[[File:nsx-routing-blog-locale-1.png|600px]]
-The following options are possible:
-# Multisite with multiple vCenters
+== Components ==
-## With active/passive site egress (Routing Metric or Local Egress Utilized)
-## With active/active site egress (Local Egress Utilized)
-# Multisite with single vCenter (stretched storage required)
-## With active/passive site egress (Routing Metric or Local Egress Utilized)
-## With active/active site egress (Local Egress Utilized)
-I will only discuss 1a in this article with Routing Metric route manipulation.
+I am still using the same componenents as in this previous article but in this article we will actively use the UDLR-02 and leave the UDLR-1 alone.
-== Diagram ==
+== Use-Cases ==
+I have thought of the following use-cases below.
-The following diagram will be used for our setup.
+As it is not possible to test active/passive and active/active with one UDLR I am using two UDLR instances here.
-[[File:LAB100_-_NSX_Routing_based_on_eBGP_v2.pdf]]
+For testing purpose, I will use three tenants with each having their own Web, App and DB tiers.
-[[File:nsx-routing-blog-1.png|600px]]
+Tenant one and two will be using the first UDLR (UDLR-01) with active/passive site egress. Tenant three will be using the second UDLR (UDLR-02) with active/active site egress.
-== Components ==
+=== Tenant three ===
+Tenant three will have workloads in DC1 and in DC2 and because it is using local egress the traffic will exit from the Cisco CSR1000V routers local to that site. When the primary site is down the traffic should exit from the secondary site and when the secondary is down traffic should exit from the primary.
-I have used the following components:
+== Routing protocol options ==
-=== CORE ===
+All possible routing options are described in the previous article.
-* CS01 = Cisco 3550 L3 Core Switch
+The routing option in this article will be:
-=== DC1 ===
+Option 1
+# eBGP peering between the UDLR and the ESG’s
+# eBGP peering between the ESG’s and the external routers
+# eBGP peering between the external routers and the CORE
-* External PSC (6.5)
+== Option 1 ==
-* vCenter Server Appliance (6.5)
-* NSX Manager – Primary (6.3.1)
-* RT-A-01 = CSR1000V
-* RT-A-02 = CSR1000V
-* ESG-A = NSX ESG (standalone, no ECMP, no HA)
-* UDLR-01 (standalone, no HA)
-* UDLR-02 (local egress enabled)
-* 3-TIER APP
-=== DC2 ===
+Because of the many options available in the setup I want to start with the following:
+Tenant 3 workloads with the use of UDLR-02 and option 1 routing.
-* External PSC (6.5)
+=== UDLR configuration ===
-* vCenter Server Appliance (6.5)
-* NSX Manager – Secondary (6.3.1)
-* RT-B-01 = CSR1000V
-* RT-B-02 = CSR1000V
-* ESG-B = NSX ESG (standalone, no ECMP, no HA)
-* UDLR-02 (local egress enabled)
-* 3-TIER APP
-== Use-Cases ==
+When Locale ID is used the UDLR Control VM needs to be deployed in both DC's.
+After the deployment is done on the primary site we need to browse to the secondary NSX Manager and manually deploy an instance of the UDLR control VM in DC2.
+When this is done some DC2 site specific configurations are needed.
-I have thought of the following use-cases below.
+# DC1 and DC2 UDLR-02 instances will have their own interfaces with own IP addresses
+# DC1 and DC2 UDLR-02 instances will have their own eBGP peerings
+# DC1 and DC2 UDLR-02 instances will have their own site (or NSX Manager) specific Locale ID
-As it is not possible to test active/passive and active/active with one UDLR I am using two UDLR instances here.
+==== DC1 ====
-For testing purpose, I will use three tenants with each having their own Web, App and DB tiers.
+Manage --> Firewall
-Tenant one and two will be using the first UDLR (UDLR-01) with active/passive site egress.
+# Make sure the firewall is disabled
-Tenant three will be using the second UDLR (UDLR-02) with active/active site egress.
-=== Tenant one ===
+[[File:nsx-routing-blog-locale-2.png|600px]]
-Tenant one will have workloads in DC1 and in DC2 and because it is the routing protocol to determine the egress path the traffic will exit (based on the configuration) from the primary site.
-When the primary site is down the traffic should exit from the secondary site.
-=== Tenant two ===
+# Get the Locale ID from the NSX Manager
-Tenant two will have workloads in DC2 and because it is the routing protocol to determine the egress path the traffic will exit (based on the configuration) from the primary site.
-When the primary site is down the traffic should exit from the secondary site.
-This will be demonstrated in this article by using tenant one because tenant one has workloads in both sites.
+[[File:nsx-routing-blog-locale-3.png|600px]]
-=== Tenant three ===
+Manage --> Routing --> Global Configuration
-Tenant three will have workloads in DC1 and in DC2 and because it is using local egress the traffic will exit from the Cisco CSR1000V routers local to that site.
-When the primary site is down the traffic should exit from the secondary site and when the secondary is down traffic should exit from the primary.
-The following implementation options are available and I am going to deploy and test all of them.
+# Configure Locale ID and make sure this is the same as the NSX Manager
+# Enable ECMP
+# Make sure that the default gateway is not configured
+# Configure a router ID
-This will not be demonstrated in this article, but in another article.
+[[File:nsx-routing-blog-locale-4.png|600px]]
-== Routing protocol options ==
+Manage --> Routing --> BGP
-# Option 1
+# Enable BGP
-## eBGP peering between the UDLR and the ESG’s
+# Configure the Local AS
-## eBGP peering between the ESG’s and the external routers
+# Disable Graceful Restart
-## eBGP peering between the external routers and the CORE
+# Configure the BGP peers towards ESG-A
-# Option 2
-## iBGP peering between the UDLR and the ESG’s
-## eBGP peering between the ESG’s and the external routers
-## eBGP peering between the external routers and the CORE
-# Option 3
-## iBGP peering between the UDLR and the ESG’s
-## iBGP peering between the ESG’s and the external routers
-## eBGP peering between the external routers and the CORE
-# Option 4
-## OSPF peering between the UDLR and the ESG’s (where the ESG is an Area Border Router (ABR))
-## OSPF peering between the ESG’s and the external routers (where the ESG is an Area Border Router (ABR))
-## eBGP peering between the external routers and the CORE
-# Option 5
-## OSPF peering between the UDLR and the ESG’s (in the same area)
-## OSPF peering between the ESG’s and the external routers (in the same area)
-## eBGP peering between the external routers and the CORE
-== Option 1 ==
+[[File:nsx-routing-blog-locale-5.png|600px]]
-Because of the many options available in the setup I want to start with the following:
+Manage --> Routing --> Route Redistribution
-Tenant 1 workloads with the use of UDLR-01 and option 1 routing.
-The other options will be outlined in other articles.
-=== UDLR configuration ===
+# Enable route redistribution for BGP
+# Specify what you allow to redistribute
-==== DC1 ====
+[[File:nsx-routing-blog-locale-6.png|600px]]
-Manage --> Firewall
+==== DC2 ====
 # Make sure the firewall is disabled
-[[File:nsx-routing-blog-2.png|600px]]
+[[File:nsx-routing-blog-locale-7.png|600px]]
+# Get the Locale ID from the NSX Manager
+[[File:nsx-routing-blog-locale-8.png|600px]]
 Manage --> Routing --> Global Configuration
+# Configure Locale ID and make sure this is the same as the NSX Manager
 # Enable ECMP
 # Make sure that the default gateway is not configured
 # Configure a router ID
-[[File:nsx-routing-blog-3.png|600px]]
+[[File:nsx-routing-blog-locale-9.png|600px]]
 Manage --> Routing --> BGP
@@ Line 148: / Line 125: @@
 # Configure the Local AS
 # Disable Graceful Restart
-# Configure the BGP peers towards ESG-A and ESG-B
+# Configure the BGP peers towards ESG-A
-[[File:nsx-routing-blog-4.png|600px]]
+[[File:nsx-routing-blog-locale-10.png|600px]]
 Manage --> Routing --> Route Redistribution
@@ Line 157: / Line 134: @@
 # Specify what you allow to redistribute
-[[File:nsx-routing-blog-5.png|600px]]
+[[File:nsx-routing-blog-locale-11.png|600px]]
+=== Compute Cluster configuration ===
+The same locale ID (per site) also need to be configured on the Compute Clusters.
+We need to verify this if this happened automatically and if this is not the case we need to configure this manually.
+==== DC1 ====
+[[File:nsx-routing-blog-locale-12.png|600px]]
 ==== DC2 ====
-The UDLR control VM does not exist in DC2.
+[[File:nsx-routing-blog-locale-13.png|600px]]
 === ESG configuration ===
@@ Line 171: / Line 157: @@
 # Make sure the firewall is disabled
-[[File:nsx-routing-blog-6.png|600px]]
+[[File:nsx-routing-blog-locale-14.png|600px]]
 Manage --> Routing --> Global Configuration
@@ Line 179: / Line 165: @@
 # Configure a router ID
-[[File:nsx-routing-blog-7.png|600px]]
+[[File:nsx-routing-blog-locale-15.png|600px]]
 Manage --> Routing --> BGP
@@ Line 188: / Line 174: @@
 # Configure the BGP peers towards the external routers and the UDLR.
-[[File:nsx-routing-blog-8.png|600px]]
+[[File:nsx-routing-blog-locale-16.png|600px]]
 Manage --> Routing --> Route Redistribution
@@ Line 195: / Line 181: @@
 # Specify what you allow to redistribute
-[[File:nsx-routing-blog-9.png|600px]]
+[[File:nsx-routing-blog-locale-17.png|600px]]
 ==== DC2 ====
@@ Line 203: / Line 189: @@
 # Make sure the firewall is disabled
-[[File:nsx-routing-blog-10.png|600px]]
+[[File:nsx-routing-blog-locale-18.png|600px]]
 Manage --> Routing --> Global Configuration
@@ Line 211: / Line 197: @@
 # Configure a router ID
-[[File:nsx-routing-blog-11.png|600px]]
+[[File:nsx-routing-blog-locale-19.png|600px]]
 Manage --> Routing --> BGP
@@ Line 220: / Line 206: @@
 # Configure the BGP peers towards the external routers and the UDLR
-[[File:nsx-routing-blog-12.png|600px]]
+[[File:nsx-routing-blog-locale-20.png|600px]]
 Manage --> Routing --> Route Redistribution
@@ Line 227: / Line 213: @@
 # Specify what you allow to redistribute
-[[File:nsx-routing-blog-13.png|600px]]
+[[File:nsx-routing-blog-locale-21.png|600px]]
 === Cisco1000V configuration ===
@@ Line 267: / Line 253: @@
   neighbor 10.100.21.2 description ESG-A
   !
-<...> missing?
+ address-family ipv4
+  neighbor 10.11.11.253 activate
+  neighbor 10.100.21.2 activate
+ exit-address-family
 !
 ip route 10.200.19.0 255.255.255.0 10.11.11.253
@@ Line 356: / Line 345: @@
 ==== DC1 ====
+[[File:nsx-routing-blog-locale-22.png|600px]]
+==== DC2 ====
 {{console|body=
-##bl##REGX-UDLR01-0> show ip bgp neighbors
+##bl##NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0> show ip bgp neighbors
-##y##BGP neighbor is 172.39.39.1,   remote AS 65521,
+BGP neighbor is 172.39.39.33,   remote AS 65522,
-##y##BGP state = Established, up
+BGP state = Established, up
 Hold time is 180, Keep alive interval is 60 seconds
 Neighbor capabilities:
@@ Line 368: / Line 361: @@
           Graceful restart Capability:none
                   Restart remain time: 0
-Received 2886 messages, Sent 2903 messages
+Received 3194 messages, Sent 4728 messages
 Default minimum time between advertisement runs is 30 seconds
 For Address family IPv4 Unicast:advertised and received
-          Index 1 Identifier 0x576e9e8c
+          Index 1 Identifier 0x3740211c
           Route refresh request:received 0 sent 0
-          ##y##Prefixes received 4 sent 12 advertised 12
+          Prefixes received 13 sent 5 advertised 5
-Connections established 2, dropped 3
+Connections established 1, dropped 2468
-Local host: 172.39.39.13, Local port: 179
+Local host: 172.39.39.45, Local port: 179
-Remote host: 172.39.39.1, Remote port: 37933
+Remote host: 172.39.39.33, Remote port: 53355
+NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0>
-##y##BGP neighbor is 172.39.39.2,   remote AS 65522,
+}}
-##y##BGP state = Established, up
+=== ESG route peering verification ===
+==== DC1 ====
+{{console|body=
+##bl##REGA-ESG01-0> show ip bgp neighbors
+BGP neighbor is 10.100.19.1,   remote AS 65511,
+BGP state = Established, up
 Hold time is 180, Keep alive interval is 60 seconds
 Neighbor capabilities:
@@ Line 387: / Line 390: @@
           Graceful restart Capability:none
                   Restart remain time: 0
-Received 2903 messages, Sent 2905 messages
+Received 2203 messages, Sent 2301 messages
 Default minimum time between advertisement runs is 30 seconds
 For Address family IPv4 Unicast:advertised and received
-          Index 2 Identifier 0x576e9e8c
+          Index 1 Identifier 0xa87fda9c
           Route refresh request:received 0 sent 0
-          ##y##Prefixes received 6 sent 10 advertised 10
+          Prefixes received 3 sent 14 advertised 14
 Connections established 1, dropped 1
-Local host: 172.39.39.13, Local port: 55933
+Local host: 10.100.19.2, Local port: 12895
-Remote host: 172.39.39.2, Remote port: 179
+Remote host: 10.100.19.1, Remote port: 179
-REGX-UDLR01-0>
-}}
-==== DC2 ====
-The UDLR control VM does not exist in DC2.
-=== ESG route peering verification ===
-==== DC1 ====
+BGP neighbor is 10.100.21.1,   remote AS 65511,
+BGP state = Established, up
-{{console|body=
-##bl##REGA-ESG01-0> show ip bgp neighbors
-##y##BGP neighbor is 10.100.19.1,   remote AS 65511,
-##y##BGP state = Established, up
 Hold time is 180, Keep alive interval is 60 seconds
 Neighbor capabilities:
@@ Line 419: / Line 409: @@
           Graceful restart Capability:none
                   Restart remain time: 0
-Received 2758 messages, Sent 2869 messages
+Received 2201 messages, Sent 2300 messages
 Default minimum time between advertisement runs is 30 seconds
 For Address family IPv4 Unicast:advertised and received
-          Index 1 Identifier 0x9c4708ec
+          Index 2 Identifier 0xa87fda9c
           Route refresh request:received 0 sent 0
-          ##y##Prefixes received 2 sent 15 advertised 15
+          Prefixes received 3 sent 14 advertised 14
 Connections established 1, dropped 1
-Local host: 10.100.19.2, Local port: 25030
+Local host: 10.100.21.2, Local port: 31006
-Remote host: 10.100.19.1, Remote port: 179
+Remote host: 10.100.21.1, Remote port: 179
-##y##BGP neighbor is 10.100.21.1,   remote AS 65511,
+BGP neighbor is 172.39.39.13,   remote AS 65530,
-##y##BGP state = Established, up
+BGP state = Established, up
 Hold time is 180, Keep alive interval is 60 seconds
 Neighbor capabilities:
@@ Line 438: / Line 428: @@
           Graceful restart Capability:none
                   Restart remain time: 0
-Received 2754 messages, Sent 2873 messages
+Received 2340 messages, Sent 2323 messages
 Default minimum time between advertisement runs is 30 seconds
 For Address family IPv4 Unicast:advertised and received
-          Index 2 Identifier 0x9c4708ec
+          Index 3 Identifier 0xa87fda9c
           Route refresh request:received 0 sent 0
-          ##y##Prefixes received 2 sent 15 advertised 15
+          Prefixes received 7 sent 7 advertised 7
-Connections established 1, dropped 1
+Connections established 2, dropped 42
-Local host: 10.100.21.2, Local port: 31420
+Local host: 172.39.39.1, Local port: 13445
-Remote host: 10.100.21.1, Remote port: 179
+Remote host: 172.39.39.13, Remote port: 179
-##y##BGP neighbor is 172.39.39.13,   remote AS 65530,
+BGP neighbor is 172.39.39.29,   remote AS 65530,
-##y##BGP state = Established, up
+BGP state = Established, up
 Hold time is 180, Keep alive interval is 60 seconds
 Neighbor capabilities:
@@ Line 457: / Line 447: @@
           Graceful restart Capability:none
                   Restart remain time: 0
-Received 2877 messages, Sent 2859 messages
+Received 2286 messages, Sent 2282 messages
 Default minimum time between advertisement runs is 30 seconds
 For Address family IPv4 Unicast:advertised and received
-          Index 3 Identifier 0x9c4708ec
+          Index 4 Identifier 0xa87fda9c
           Route refresh request:received 0 sent 0
-          ##y##Prefixes received 12 sent 4 advertised 4
+          Prefixes received 5 sent 7 advertised 7
 Connections established 1, dropped 1
-Local host: 172.39.39.1, Local port: 37933
+Local host: 172.39.39.17, Local port: 37920
-Remote host: 172.39.39.13, Remote port: 179
+Remote host: 172.39.39.29, Remote port: 179
 REGA-ESG01-0>
@@ Line 475: / Line 465: @@
 ##bl##REGB-ESG01-0> show ip bgp neighbors
-##y##BGP neighbor is 10.200.19.1,   remote AS 65512,
+BGP neighbor is 10.200.19.1,   remote AS 65512,
-##y##BGP state = Established, up
+BGP state = Established, up
 Hold time is 180, Keep alive interval is 60 seconds
 Neighbor capabilities:
@@ Line 483: / Line 473: @@
           Graceful restart Capability:none
                   Restart remain time: 0
-Received 2804 messages, Sent 2936 messages
+Received 2192 messages, Sent 2287 messages
 Default minimum time between advertisement runs is 30 seconds
 For Address family IPv4 Unicast:advertised and received
-          Index 1 Identifier 0x359d82ac
+          Index 1 Identifier 0x637a1aec
           Route refresh request:received 0 sent 0
-          ##y##Prefixes received 2 sent 13 advertised 13
+          Prefixes received 9 sent 8 advertised 8
 Connections established 1, dropped 1
-Local host: 10.200.19.2, Local port: 43134
+Local host: 10.200.19.2, Local port: 57360
 Remote host: 10.200.19.1, Remote port: 179
-##y##BGP neighbor is 10.200.21.1,   remote AS 65512,
+BGP neighbor is 10.200.21.1,   remote AS 65512,
-##y##BGP state = Established, up
+BGP state = Established, up
 Hold time is 180, Keep alive interval is 60 seconds
 Neighbor capabilities:
@@ Line 502: / Line 492: @@
           Graceful restart Capability:none
                   Restart remain time: 0
-Received 2801 messages, Sent 2937 messages
+Received 2191 messages, Sent 2280 messages
 Default minimum time between advertisement runs is 30 seconds
 For Address family IPv4 Unicast:advertised and received
-          Index 2 Identifier 0x359d82ac
+          Index 2 Identifier 0x637a1aec
           Route refresh request:received 0 sent 0
-          ##y##Prefixes received 2 sent 13 advertised 13
+          Prefixes received 9 sent 8 advertised 8
 Connections established 1, dropped 1
-Local host: 10.200.21.2, Local port: 19055
+Local host: 10.200.21.2, Local port: 17297
 Remote host: 10.200.21.1, Remote port: 179
-##y##BGP neighbor is 172.39.39.13,   remote AS 65530,
+BGP neighbor is 172.39.39.13,   remote AS 65530,
-##y##BGP state = Established, up
+BGP state = Idle, up
+Hold time is 180, Keep alive interval is 60 seconds
+Neighbor capabilities:
+         Route refresh: None
+         Address family none
+         Graceful restart Capability:none
+                 Restart remain time: 0
+Received 15751 messages, Sent 11421 messages
+Default minimum time between advertisement runs is 30 seconds
+For Address family none
+         Index 3 Identifier 0x637a1aec
+         Route refresh request:received 0 sent 0
+         Prefixes received 0 sent 0 advertised 0
+Connections established 0, dropped 15751
+Local host: 10.200.21.2, Local port: 0
+Remote host: 172.39.39.13, Remote port: 0
+BGP neighbor is 172.39.39.45,   remote AS 65530,
+BGP state = Established, up
 Hold time is 180, Keep alive interval is 60 seconds
 Neighbor capabilities:
@@ Line 521: / Line 530: @@
           Graceful restart Capability:none
                   Restart remain time: 0
-Received 2926 messages, Sent 2937 messages
+Received 2278 messages, Sent 2279 messages
 Default minimum time between advertisement runs is 30 seconds
 For Address family IPv4 Unicast:advertised and received
-          Index 3 Identifier 0x359d82ac
+          Index 4 Identifier 0x637a1aec
           Route refresh request:received 0 sent 0
-          ##y##Prefixes received 10 sent 6 advertised 6
+          Prefixes received 5 sent 13 advertised 13
-Connections established 2, dropped 1
+Connections established 1, dropped 1
-Local host: 172.39.39.2, Local port: 179
+Local host: 172.39.39.33, Local port: 53355
-Remote host: 172.39.39.13, Remote port: 55933
+Remote host: 172.39.39.45, Remote port: 179
 REGB-ESG01-0>
@@ Line 541: / Line 550: @@
 ##bl##rt-a-01#show ip bgp summary
 BGP router identifier 10.11.11.31, local AS number 65511
-BGP table version is 145, main routing table version 145
+BGP table version is 54, main routing table version 54
-network entries using 3720 bytes of memory
+network entries using 4216 bytes of memory
-path entries using 3240 bytes of memory
+path entries using 2520 bytes of memory
-/2 BGP path/bestpath attribute entries using 960 bytes of memory
+/3 BGP path/bestpath attribute entries using 720 bytes of memory
 BGP AS-PATH entries using 88 bytes of memory
 BGP route-map cache entries using 0 bytes of memory
 BGP filter-list cache entries using 0 bytes of memory
-BGP using 8008 total bytes of memory
+BGP using 7544 total bytes of memory
-BGP activity 24/9 prefixes, 105/78 paths, scan interval 60 secs
+BGP activity 30/13 prefixes, 47/26 paths, scan interval 60 secs
 Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
-##y##10.11.11.253    4        65510   12586   12590      145    0    0 1w0d           12
+.11.11.253    4        65510    2218    2206       54    0    0 1d09h           7
-##y##10.100.19.2     4        65521    2880    2769      145    0    0 1d17h          15
+.100.19.2     4        65521    2304    2205       54    0    0 1d09h          14
 rt-a-01#
 }}
@@ Line 560: / Line 569: @@
 ##bl##rt-a-02#show ip bgp summary
 BGP router identifier 10.11.11.32, local AS number 65511
-BGP table version is 145, main routing table version 145
+BGP table version is 54, main routing table version 54
-network entries using 3720 bytes of memory
+network entries using 4216 bytes of memory
-path entries using 3240 bytes of memory
+path entries using 2520 bytes of memory
-/2 BGP path/bestpath attribute entries using 960 bytes of memory
+/3 BGP path/bestpath attribute entries using 720 bytes of memory
 BGP AS-PATH entries using 88 bytes of memory
 BGP route-map cache entries using 0 bytes of memory
 BGP filter-list cache entries using 0 bytes of memory
-BGP using 8008 total bytes of memory
+BGP using 7544 total bytes of memory
-BGP activity 31/16 prefixes, 105/78 paths, scan interval 60 secs
+BGP activity 35/18 prefixes, 47/26 paths, scan interval 60 secs
 Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
-##y##10.11.11.253    4        65510   12588   12583      145    0    0 1w0d           12
+.11.11.253    4        65510    2209    2210       54    0    0 1d09h           7
-##y##10.100.21.2     4        65521    2884    2766      145    0    0 1d17h          15
+.100.21.2     4        65521    2303    2204       54    0    0 1d09h          14
 rt-a-02#
 }}
@@ Line 581: / Line 590: @@
 ##bl##rt-b-01#show ip bgp summary
 BGP router identifier 10.11.11.33, local AS number 65512
-BGP table version is 152, main routing table version 152
+BGP table version is 76, main routing table version 76
-network entries using 3720 bytes of memory
+network entries using 4216 bytes of memory
-path entries using 2160 bytes of memory
+path entries using 2280 bytes of memory
-/2 BGP path/bestpath attribute entries using 720 bytes of memory
+/3 BGP path/bestpath attribute entries using 720 bytes of memory
 BGP AS-PATH entries using 88 bytes of memory
 BGP route-map cache entries using 0 bytes of memory
 BGP filter-list cache entries using 0 bytes of memory
-BGP using 6688 total bytes of memory
+BGP using 7304 total bytes of memory
-BGP activity 42/27 prefixes, 104/86 paths, scan interval 60 secs
+BGP activity 30/13 prefixes, 50/31 paths, scan interval 60 secs
 Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
-##y##10.11.11.253    4        65510   12575   12587      152    0    0 1w0d            5
+.11.11.253    4        65510    5385    5387       76    0    0 3d09h          11
-##y##10.200.19.2     4        65522    2946    2813      152    0    0 1d18h          13
+.200.19.2     4        65522    2290    2194       76    0    0 1d09h           8
 rt-b-01#
 }}
@@ Line 600: / Line 609: @@
 ##bl##rt-b-02#show ip bgp summary
 BGP router identifier 10.11.11.34, local AS number 65512
-BGP table version is 152, main routing table version 152
+BGP table version is 49, main routing table version 49
-network entries using 3720 bytes of memory
+network entries using 4216 bytes of memory
-path entries using 2160 bytes of memory
+path entries using 2280 bytes of memory
-/2 BGP path/bestpath attribute entries using 720 bytes of memory
+/3 BGP path/bestpath attribute entries using 720 bytes of memory
 BGP AS-PATH entries using 88 bytes of memory
 BGP route-map cache entries using 0 bytes of memory
 BGP filter-list cache entries using 0 bytes of memory
-BGP using 6688 total bytes of memory
+BGP using 7304 total bytes of memory
-BGP activity 44/29 prefixes, 104/86 paths, scan interval 60 secs
+BGP activity 17/0 prefixes, 35/16 paths, scan interval 60 secs
 Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
-##y##10.11.11.253    4        65510   12587   12587      152    0    0 1w0d            5
+.11.11.253    4        65510    2210    2204       49    0    0 1d09h          11
-##y##10.200.21.2     4        65522    2947    2812      152    0    0 1d18h          13
+.200.21.2     4        65522    2284    2195       49    0    0 1d09h           8
 rt-b-02#
 }}
@@ Line 621: / Line 630: @@
 ##bl##cs-01#show ip bgp summary
 BGP router identifier 10.11.11.253, local AS number 65510
-BGP table version is 153, main routing table version 153
+BGP table version is 83, main routing table version 83
-network entries using 2040 bytes of memory
+network entries using 2312 bytes of memory
-path entries using 2808 bytes of memory
+path entries using 2340 bytes of memory
 /3 BGP path/bestpath attribute entries using 372 bytes of memory
 BGP AS-PATH entries using 48 bytes of memory
 BGP route-map cache entries using 0 bytes of memory
 BGP filter-list cache entries using 0 bytes of memory
-BGP using 5268 total bytes of memory
+BGP using 5072 total bytes of memory
-BGP activity 32/17 prefixes, 256/202 paths, scan interval 60 secs
+BGP activity 30/13 prefixes, 109/64 paths, scan interval 60 secs
 Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
-##y##10.11.11.31     4        65511   12594   12590      153    0    0 1w0d           13
+.11.11.31     4        65511    2209    2221       83    0    0 1d09h          14
-##y##10.11.11.32     4        65511   12586   12591      153    0    0 1w0d           13
+.11.11.32     4        65511    2212    2212       83    0    0 1d09h          14
-##y##10.11.11.33     4        65512   12589   12577      153    0    0 1w0d           13
+.11.11.33     4        65512    5389    5387       83    0    0 3d09h           8
-##y##10.11.11.34     4        65512   12588   12587      153    0    0 1w0d           13
+.11.11.34     4        65512    2205    2210       83    0    0 1d09h           8
 cs-01#
 }}
@@ Line 642: / Line 651: @@
 ==== DC1 ====
+[[File:nsx-routing-blog-locale-23.png|600px]]
+==== DC2 ====
 {{console|body=
-##bl##REGX-UDLR01-0> show ip route
+##bl##NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0> show ip route
 Codes: O - OSPF derived, i - IS-IS derived, B - BGP derived,
@@ Line 651: / Line 664: @@
 N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
-Total number of routes: 15
+Total number of routes: 17
-##y##B       10.11.11.0/24        [20/0]        via 172.39.39.2
+B       10.11.11.0/24        [20/0]        via 172.39.39.33
-B       10.22.22.0/24        [20/0]        via 172.39.39.2
+B       10.100.19.0/24       [20/0]        via 172.39.39.33
-B       10.100.19.0/24       [20/0]        via 172.39.39.1
+B       10.100.21.0/24       [20/0]        via 172.39.39.33
-B       10.100.21.0/24       [20/0]        via 172.39.39.1
+B       10.200.19.0/24       [20/0]        via 172.39.39.33
-B       10.200.19.0/24       [20/0]        via 172.39.39.2
+B       10.200.21.0/24       [20/0]        via 172.39.39.33
-B       10.200.21.0/24       [20/0]        via 172.39.39.2
+B       172.20.1.0/24        [20/0]        via 172.39.39.33
-##y##C       172.20.1.0/24        [0/0]         via 172.20.1.254
+B       172.20.2.0/24        [20/0]        via 172.39.39.33
-##y##C       172.20.2.0/24        [0/0]         via 172.20.2.254
+B       172.20.3.0/24        [20/0]        via 172.39.39.33
-##y##C       172.20.3.0/24        [0/0]         via 172.20.3.254
+B       172.20.8.0/24        [20/0]        via 172.39.39.33
-C       172.20.8.0/24        [0/0]         via 172.20.8.254
+B       172.20.9.0/24        [20/0]        via 172.39.39.33
-C       172.20.9.0/24        [0/0]         via 172.20.9.254
+B       172.20.10.0/24       [20/0]        via 172.39.39.33
-C       172.20.10.0/24       [0/0]         via 172.20.10.254
+C       172.30.1.0/24        [0/0]         via 172.30.1.254
-C       172.39.39.0/28       [0/0]         via 172.39.39.13
+C       172.30.2.0/24        [0/0]         via 172.30.2.254
-B       172.39.39.16/28      [20/0]        via 172.39.39.1
+C       172.30.3.0/24        [0/0]         via 172.30.3.254
-B       172.39.39.32/28      [20/0]        via 172.39.39.2
+B       172.39.39.0/28       [20/0]        via 172.39.39.33
-REGX-UDLR01-0>
+C       172.39.39.16/28      [0/0]         via 172.39.39.30
+C       172.39.39.32/28      [0/0]         via 172.39.39.45
+NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0>
 }}
-==== DC2 ====
-The UDLR control VM does not exist in DC2.
 === ESG routing tables ===
@@ Line 687: / Line 698: @@
 N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
-Total number of routes: 15
+Total number of routes: 17
-##y##B       10.11.11.0/24        [20/0]        via 172.39.39.14
+B       10.11.11.0/24        [20/0]        via 10.100.19.1
-B       10.22.22.0/24        [20/0]        via 172.39.39.14
+B       10.11.11.0/24        [20/0]        via 10.100.21.1
 C       10.100.19.0/24       [0/0]         via 10.100.19.2
 C       10.100.21.0/24       [0/0]         via 10.100.21.2
-B       10.200.19.0/24       [20/0]        via 172.39.39.14
+B       10.200.19.0/24       [20/0]        via 10.100.19.1
-B       10.200.21.0/24       [20/0]        via 172.39.39.14
+B       10.200.19.0/24       [20/0]        via 10.100.21.1
-##y##B       172.20.1.0/24        [20/0]        via 172.39.39.14
+B       10.200.21.0/24       [20/0]        via 10.100.19.1
-##y##B       172.20.2.0/24        [20/0]        via 172.39.39.14
+B       10.200.21.0/24       [20/0]        via 10.100.21.1
-##y##B       172.20.3.0/24        [20/0]        via 172.39.39.14
+B       172.20.1.0/24        [20/0]        via 172.39.39.14
+B       172.20.2.0/24        [20/0]        via 172.39.39.14
+B       172.20.3.0/24        [20/0]        via 172.39.39.14
 B       172.20.8.0/24        [20/0]        via 172.39.39.14
 B       172.20.9.0/24        [20/0]        via 172.39.39.14
 B       172.20.10.0/24       [20/0]        via 172.39.39.14
+B       172.30.1.0/24        [20/0]        via 172.39.39.30
+B       172.30.2.0/24        [20/0]        via 172.39.39.30
+B       172.30.3.0/24        [20/0]        via 172.39.39.30
 C       172.39.39.0/28       [0/0]         via 172.39.39.1
 C       172.39.39.16/28      [0/0]         via 172.39.39.17
-B       172.39.39.32/28      [20/0]        via 172.39.39.14
+B       172.39.39.32/28      [20/0]        via 172.39.39.30
 REGA-ESG01-0>
 }}
@@ Line 717: / Line 733: @@
 N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
-Total number of routes: 15
+Total number of routes: 17
-##y##B       10.11.11.0/24        [20/0]        via 10.200.19.1
+B       10.11.11.0/24        [20/0]        via 10.200.19.1
-##y##B       10.11.11.0/24        [20/0]        via 10.200.21.1
+B       10.11.11.0/24        [20/0]        via 10.200.21.1
-B       10.22.22.0/24        [20/0]        via 10.200.19.1
+B       10.100.19.0/24       [20/0]        via 10.200.19.1
-B       10.22.22.0/24        [20/0]        via 10.200.21.1
+B       10.100.19.0/24       [20/0]        via 10.200.21.1
-B       10.100.19.0/24       [20/0]        via 172.39.39.14
+B       10.100.21.0/24       [20/0]        via 10.200.19.1
-B       10.100.21.0/24       [20/0]        via 172.39.39.14
+B       10.100.21.0/24       [20/0]        via 10.200.21.1
 C       10.200.19.0/24       [0/0]         via 10.200.19.2
 C       10.200.21.0/24       [0/0]         via 10.200.21.2
-##y##B       172.20.1.0/24        [20/0]        via 172.39.39.14
+B       172.20.1.0/24        [20/0]        via 10.200.19.1
-##y##B       172.20.2.0/24        [20/0]        via 172.39.39.14
+B       172.20.1.0/24        [20/0]        via 10.200.21.1
-##y##B       172.20.3.0/24        [20/0]        via 172.39.39.14
+B       172.20.2.0/24        [20/0]        via 10.200.19.1
-B       172.20.8.0/24        [20/0]        via 172.39.39.14
+B       172.20.2.0/24        [20/0]        via 10.200.21.1
-B       172.20.9.0/24        [20/0]        via 172.39.39.14
+B       172.20.3.0/24        [20/0]        via 10.200.19.1
-B       172.20.10.0/24       [20/0]        via 172.39.39.14
+B       172.20.3.0/24        [20/0]        via 10.200.21.1
+B       172.20.8.0/24        [20/0]        via 10.200.19.1
+B       172.20.8.0/24        [20/0]        via 10.200.21.1
+B       172.20.9.0/24        [20/0]        via 10.200.19.1
+B       172.20.9.0/24        [20/0]        via 10.200.21.1
+B       172.20.10.0/24       [20/0]        via 10.200.19.1
+B       172.20.10.0/24       [20/0]        via 10.200.21.1
+B       172.30.1.0/24        [20/0]        via 172.39.39.46
+B       172.30.2.0/24        [20/0]        via 172.39.39.46
+B       172.30.3.0/24        [20/0]        via 172.39.39.46
 C       172.39.39.0/28       [0/0]         via 172.39.39.2
-B       172.39.39.16/28      [20/0]        via 172.39.39.14
+B       172.39.39.16/28      [20/0]        via 172.39.39.46
 C       172.39.39.32/28      [0/0]         via 172.39.39.33
 REGB-ESG01-0>
@@ Line 757: / Line 782: @@
 Gateway of last resort is not set
-.0.0.0/8 is variably subnetted, 10 subnets, 2 masks
+.0.0.0/8 is variably subnetted, 9 subnets, 2 masks
 C        10.11.11.0/24 is directly connected, GigabitEthernet2
 L        10.11.11.31/32 is directly connected, GigabitEthernet2
-B        10.22.22.0/24 [20/0] via 10.11.11.253, 1d18h
 C        10.100.1.0/24 is directly connected, GigabitEthernet1
 L        10.100.1.101/32 is directly connected, GigabitEthernet1
 C        10.100.19.0/24 is directly connected, GigabitEthernet3
 L        10.100.19.1/32 is directly connected, GigabitEthernet3
-B        10.100.21.0/24 [20/0] via 10.100.19.2, 1d17h
+B        10.100.21.0/24 [20/0] via 10.100.19.2, 1d09h
 S        10.200.19.0/24 [1/0] via 10.11.11.253
 S        10.200.21.0/24 [1/0] via 10.11.11.253
 .20.0.0/24 is subnetted, 6 subnets
-##y##B        172.20.1.0 [20/0] via 10.100.19.2, 1d17h
+B        172.20.1.0 [20/0] via 10.100.19.2, 1d09h
-##y##B        172.20.2.0 [20/0] via 10.100.19.2, 1d17h
+B        172.20.2.0 [20/0] via 10.100.19.2, 1d09h
-##y##B        172.20.3.0 [20/0] via 10.100.19.2, 1d17h
+B        172.20.3.0 [20/0] via 10.100.19.2, 1d09h
-B        172.20.8.0 [20/0] via 10.100.19.2, 1d17h
+B        172.20.8.0 [20/0] via 10.100.19.2, 1d09h
-B        172.20.9.0 [20/0] via 10.100.19.2, 1d17h
+B        172.20.9.0 [20/0] via 10.100.19.2, 1d09h
-B        172.20.10.0 [20/0] via 10.100.19.2, 1d17h
+B        172.20.10.0 [20/0] via 10.100.19.2, 1d09h
+.30.0.0/24 is subnetted, 3 subnets
+B        172.30.1.0 [20/0] via 10.100.19.2, 1d09h
+B        172.30.2.0 [20/0] via 10.100.19.2, 1d09h
+B        172.30.3.0 [20/0] via 10.100.19.2, 1d09h
 .39.0.0/28 is subnetted, 3 subnets
-B        172.39.39.0 [20/0] via 10.100.19.2, 1d17h
+B        172.39.39.0 [20/0] via 10.100.19.2, 1d09h
-B        172.39.39.16 [20/0] via 10.100.19.2, 1d17h
+B        172.39.39.16 [20/0] via 10.100.19.2, 1d09h
-B        172.39.39.32 [20/0] via 10.100.19.2, 1d17h
+B        172.39.39.32 [20/0] via 10.100.19.2, 1d09h
 rt-a-01#
 }}
@@ Line 796: / Line 824: @@
 Gateway of last resort is not set
-.0.0.0/8 is variably subnetted, 10 subnets, 2 masks
+.0.0.0/8 is variably subnetted, 9 subnets, 2 masks
 C        10.11.11.0/24 is directly connected, GigabitEthernet2
 L        10.11.11.32/32 is directly connected, GigabitEthernet2
-B        10.22.22.0/24 [20/0] via 10.11.11.253, 1d18h
 C        10.100.1.0/24 is directly connected, GigabitEthernet1
 L        10.100.1.102/32 is directly connected, GigabitEthernet1
-B        10.100.19.0/24 [20/0] via 10.100.21.2, 1d17h
+B        10.100.19.0/24 [20/0] via 10.100.21.2, 1d09h
 C        10.100.21.0/24 is directly connected, GigabitEthernet3
 L        10.100.21.1/32 is directly connected, GigabitEthernet3
@@ Line 808: / Line 835: @@
 S        10.200.21.0/24 [1/0] via 10.11.11.253
 .20.0.0/24 is subnetted, 6 subnets
-##y##B        172.20.1.0 [20/0] via 10.100.21.2, 1d17h
+B        172.20.1.0 [20/0] via 10.100.21.2, 1d09h
-##y##B        172.20.2.0 [20/0] via 10.100.21.2, 1d17h
+B        172.20.2.0 [20/0] via 10.100.21.2, 1d09h
-##y##B        172.20.3.0 [20/0] via 10.100.21.2, 1d17h
+B        172.20.3.0 [20/0] via 10.100.21.2, 1d09h
-B        172.20.8.0 [20/0] via 10.100.21.2, 1d17h
+B        172.20.8.0 [20/0] via 10.100.21.2, 1d09h
-B        172.20.9.0 [20/0] via 10.100.21.2, 1d17h
+B        172.20.9.0 [20/0] via 10.100.21.2, 1d09h
-B        172.20.10.0 [20/0] via 10.100.21.2, 1d17h
+B        172.20.10.0 [20/0] via 10.100.21.2, 1d09h
+.30.0.0/24 is subnetted, 3 subnets
+B        172.30.1.0 [20/0] via 10.100.21.2, 1d09h
+B        172.30.2.0 [20/0] via 10.100.21.2, 1d09h
+B        172.30.3.0 [20/0] via 10.100.21.2, 1d09h
 .39.0.0/28 is subnetted, 3 subnets
-B        172.39.39.0 [20/0] via 10.100.21.2, 1d17h
+B        172.39.39.0 [20/0] via 10.100.21.2, 1d09h
-B        172.39.39.16 [20/0] via 10.100.21.2, 1d17h
+B        172.39.39.16 [20/0] via 10.100.21.2, 1d09h
-B        172.39.39.32 [20/0] via 10.100.21.2, 1d17h
+B        172.39.39.32 [20/0] via 10.100.21.2, 1d09h
 rt-a-02#
 }}
@@ Line 837: / Line 868: @@
 Gateway of last resort is not set
-.0.0.0/8 is variably subnetted, 10 subnets, 2 masks
+.0.0.0/8 is variably subnetted, 9 subnets, 2 masks
 C        10.11.11.0/24 is directly connected, GigabitEthernet2
 L        10.11.11.33/32 is directly connected, GigabitEthernet2
-B        10.22.22.0/24 [20/0] via 10.11.11.253, 1d18h
 S        10.100.19.0/24 [1/0] via 10.11.11.253
 S        10.100.21.0/24 [1/0] via 10.11.11.253
@@ Line 847: / Line 877: @@
 C        10.200.19.0/24 is directly connected, GigabitEthernet3
 L        10.200.19.1/32 is directly connected, GigabitEthernet3
-B        10.200.21.0/24 [20/0] via 10.200.19.2, 1d18h
+B        10.200.21.0/24 [20/0] via 10.200.19.2, 1d09h
 .20.0.0/24 is subnetted, 6 subnets
-##y##B        172.20.1.0 [20/0] via 10.200.19.2, 1d18h
+B        172.20.1.0 [20/0] via 10.11.11.253, 1d09h
-##y##B        172.20.2.0 [20/0] via 10.200.19.2, 1d18h
+B        172.20.2.0 [20/0] via 10.11.11.253, 1d09h
-##y##B        172.20.3.0 [20/0] via 10.200.19.2, 1d18h
+B        172.20.3.0 [20/0] via 10.11.11.253, 1d09h
-B        172.20.8.0 [20/0] via 10.200.19.2, 1d18h
+B        172.20.8.0 [20/0] via 10.11.11.253, 1d09h
-B        172.20.9.0 [20/0] via 10.200.19.2, 1d18h
+B        172.20.9.0 [20/0] via 10.11.11.253, 1d09h
-B        172.20.10.0 [20/0] via 10.200.19.2, 1d18h
+B        172.20.10.0 [20/0] via 10.11.11.253, 1d09h
+.30.0.0/24 is subnetted, 3 subnets
+B        172.30.1.0 [20/0] via 10.200.19.2, 1d09h
+B        172.30.2.0 [20/0] via 10.200.19.2, 1d09h
+B        172.30.3.0 [20/0] via 10.200.19.2, 1d09h
 .39.0.0/28 is subnetted, 3 subnets
-B        172.39.39.0 [20/0] via 10.200.19.2, 1d18h
+B        172.39.39.0 [20/0] via 10.200.19.2, 1d09h
-B        172.39.39.16 [20/0] via 10.200.19.2, 1d17h
+B        172.39.39.16 [20/0] via 10.200.19.2, 1d09h
-B        172.39.39.32 [20/0] via 10.200.19.2, 1d18h
+B        172.39.39.32 [20/0] via 10.200.19.2, 1d09h
 rt-b-01#
 }}
@@ Line 876: / Line 910: @@
 Gateway of last resort is not set
-.0.0.0/8 is variably subnetted, 10 subnets, 2 masks
+.0.0.0/8 is variably subnetted, 9 subnets, 2 masks
 C        10.11.11.0/24 is directly connected, GigabitEthernet2
 L        10.11.11.34/32 is directly connected, GigabitEthernet2
-B        10.22.22.0/24 [20/0] via 10.11.11.253, 1d18h
 S        10.100.19.0/24 [1/0] via 10.11.11.253
 S        10.100.21.0/24 [1/0] via 10.11.11.253
 C        10.200.1.0/24 is directly connected, GigabitEthernet1
 L        10.200.1.102/32 is directly connected, GigabitEthernet1
-B        10.200.19.0/24 [20/0] via 10.200.21.2, 1d18h
+B        10.200.19.0/24 [20/0] via 10.200.21.2, 1d09h
 C        10.200.21.0/24 is directly connected, GigabitEthernet3
 L        10.200.21.1/32 is directly connected, GigabitEthernet3
 .20.0.0/24 is subnetted, 6 subnets
-##y##B        172.20.1.0 [20/0] via 10.200.21.2, 1d18h
+B        172.20.1.0 [20/0] via 10.11.11.253, 1d09h
-##y##B        172.20.2.0 [20/0] via 10.200.21.2, 1d18h
+B        172.20.2.0 [20/0] via 10.11.11.253, 1d09h
-##y##B        172.20.3.0 [20/0] via 10.200.21.2, 1d18h
+B        172.20.3.0 [20/0] via 10.11.11.253, 1d09h
-B        172.20.8.0 [20/0] via 10.200.21.2, 1d18h
+B        172.20.8.0 [20/0] via 10.11.11.253, 1d09h
-B        172.20.9.0 [20/0] via 10.200.21.2, 1d18h
+B        172.20.9.0 [20/0] via 10.11.11.253, 1d09h
-B        172.20.10.0 [20/0] via 10.200.21.2, 1d18h
+B        172.20.10.0 [20/0] via 10.11.11.253, 1d09h
+.30.0.0/24 is subnetted, 3 subnets
+B        172.30.1.0 [20/0] via 10.200.21.2, 1d09h
+B        172.30.2.0 [20/0] via 10.200.21.2, 1d09h
+B        172.30.3.0 [20/0] via 10.200.21.2, 1d09h
 .39.0.0/28 is subnetted, 3 subnets
-B        172.39.39.0 [20/0] via 10.200.21.2, 1d18h
+B        172.39.39.0 [20/0] via 10.200.21.2, 1d09h
-B        172.39.39.16 [20/0] via 10.200.21.2, 1d17h
+B        172.39.39.16 [20/0] via 10.200.21.2, 1d09h
-B        172.39.39.32 [20/0] via 10.200.21.2, 1d18h
+B        172.39.39.32 [20/0] via 10.200.21.2, 1d09h
 rt-b-02#
 }}
@@ Line 917: / Line 954: @@
 .20.0.0/24 is subnetted, 6 subnets
-##y##B        172.20.1.0 [20/0] via 10.11.11.31, 2d02h
+B        172.20.1.0 [20/0] via 10.11.11.31, 1d09h
-##y##B        172.20.2.0 [20/0] via 10.11.11.31, 2d02h
+B        172.20.2.0 [20/0] via 10.11.11.31, 1d09h
-##y##B        172.20.3.0 [20/0] via 10.11.11.31, 2d02h
+B        172.20.3.0 [20/0] via 10.11.11.31, 1d09h
-B        172.20.8.0 [20/0] via 10.11.11.31, 2d02h
+B        172.20.8.0 [20/0] via 10.11.11.31, 1d09h
-B        172.20.9.0 [20/0] via 10.11.11.31, 2d02h
+B        172.20.9.0 [20/0] via 10.11.11.31, 1d09h
-B        172.20.10.0 [20/0] via 10.11.11.31, 2d02h
+B        172.20.10.0 [20/0] via 10.11.11.31, 1d09h
+.30.0.0/24 is subnetted, 3 subnets
+B        172.30.1.0 [20/0] via 10.11.11.33, 1d09h
+B        172.30.2.0 [20/0] via 10.11.11.33, 1d09h
+B        172.30.3.0 [20/0] via 10.11.11.33, 1d09h
 .39.0.0/28 is subnetted, 3 subnets
-B        172.39.39.0 [20/0] via 10.11.11.31, 2d02h
+B        172.39.39.0 [20/0] via 10.11.11.31, 1d09h
-B        172.39.39.16 [20/0] via 10.11.11.31, 2d02h
+B        172.39.39.16 [20/0] via 10.11.11.31, 1d09h
-B        172.39.39.32 [20/0] via 10.11.11.34, 2d02h
+B        172.39.39.32 [20/0] via 10.11.11.34, 1d09h
 cs-01#
 }}
@@ Line 934: / Line 975: @@
 ==== DC1 ====
-[[File:nsx-routing-blog-14.png|600px]]
+[[File:nsx-routing-blog-locale-24.png|600px]]
 ==== DC2 ====
-The UDLR control VM does not exist in DC2.
+{{console|body=
+##bl##NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0> show ip bgp
+Status codes: s - suppressed, d - damped, > - best, i - internal
+Origin codes: i - IGP, e - EGP, ? - incomplete
+    Network            Next Hop       Metric  LocPrf  Weight AS Path
+  > 10.11.11.0/24      172.39.39.33      0     100      60   65522 i
+  > 10.100.19.0/24     172.39.39.33      0     100      60   65522 ?
+  > 10.100.21.0/24     172.39.39.33      0     100      60   65522 ?
+  > 10.200.19.0/24     172.39.39.33      0     100      60   65522 ?
+  > 10.200.21.0/24     172.39.39.33      0     100      60   65522 ?
+  > 172.20.1.0/24      172.39.39.33      0     100      60   65522 ?
+  > 172.20.2.0/24      172.39.39.33      0     100      60   65522 ?
+  > 172.20.3.0/24      172.39.39.33      0     100      60   65522 ?
+  > 172.20.8.0/24      172.39.39.33      0     100      60   65522 ?
+  > 172.20.9.0/24      172.39.39.33      0     100      60   65522 ?
+  > 172.20.10.0/24     172.39.39.33      0     100      60   65522 ?
+  > 172.30.1.0/24      0.0.0.0           0     100   32768   ?
+  > 172.30.2.0/24      0.0.0.0           0     100   32768   ?
+  > 172.30.3.0/24      0.0.0.0           0     100   32768   ?
+  > 172.39.39.0/28     172.39.39.33      0     100      60   65522 ?
+  > 172.39.39.16/28    0.0.0.0           0     100   32768   ?
+.39.39.32/28    172.39.39.33      0     100      60   65522 ?
+  > 172.39.39.32/28    0.0.0.0           0     100   32768   ?
+NSX-edge-71b9e24f-9a47-423d-b8e7-392dc8b3e47a-0>
+}}
 === ESG BGP tables ===
@@ Line 951: / Line 1,019: @@
      Network            Next Hop       Metric  LocPrf  Weight AS Path
-##y##    10.11.11.0/24      10.100.19.1       0     100      60   65511 65510 i
+  > 10.11.11.0/24      10.100.19.1       0     100      60   65511 65510 i
-##y##    10.11.11.0/24      10.100.21.1       0     100      60   65511 65510 i
+.11.11.0/24      10.100.21.1       0     100      60   65511 65510 i
-##y##  > 10.11.11.0/24      172.39.39.14      0     100      60   65530 i
-.22.22.0/24      10.100.19.1       0     100      60   65511 65510 i
-.22.22.0/24      10.100.21.1       0     100      60   65511 65510 i
-  > 10.22.22.0/24      172.39.39.14      0     100      60   65530 i
    > 10.100.19.0/24     0.0.0.0           0     100   32768   ?
    > 10.100.21.0/24     0.0.0.0           0     100   32768   ?
-   > 10.200.19.0/24     172.39.39.14      0     100      60   65530 ?
+   > 10.200.19.0/24     10.100.19.1       0     100      60   65511 65510 65512 65522 ?
-   > 10.200.21.0/24     172.39.39.14      0     100      60   65530 ?
+.200.19.0/24     10.100.21.1       0     100      60   65511 65510 65512 65522 ?
-   ##y##> 172.20.1.0/24      172.39.39.14      0     100      60   65530 ?
+   > 10.200.21.0/24     10.100.19.1       0     100      60   65511 65510 65512 65522 ?
-   ##y##> 172.20.2.0/24      172.39.39.14      0     100      60   65530 ?
+.200.21.0/24     10.100.21.1       0     100      60   65511 65510 65512 65522 ?
-   ##y##> 172.20.3.0/24      172.39.39.14      0     100      60   65530 ?
+   > 172.20.1.0/24      172.39.39.14      0     100      60   65530 ?
+   > 172.20.2.0/24      172.39.39.14      0     100      60   65530 ?
+   > 172.20.3.0/24      172.39.39.14      0     100      60   65530 ?
    > 172.20.8.0/24      172.39.39.14      0     100      60   65530 ?
    > 172.20.9.0/24      172.39.39.14      0     100      60   65530 ?
    > 172.20.10.0/24     172.39.39.14      0     100      60   65530 ?
+  > 172.30.1.0/24      172.39.39.30      0     100      60   65530 ?
+  > 172.30.2.0/24      172.39.39.30      0     100      60   65530 ?
+  > 172.30.3.0/24      172.39.39.30      0     100      60   65530 ?
 .39.39.0/28     172.39.39.14      0     100      60   65530 ?
    > 172.39.39.0/28     0.0.0.0           0     100   32768   ?
+.39.39.16/28    172.39.39.30      0     100      60   65530 ?
    > 172.39.39.16/28    0.0.0.0           0     100   32768   ?
-   > 172.39.39.32/28    172.39.39.14      0     100      60   65530 ?
+   > 172.39.39.32/28    172.39.39.30      0     100      60   65530 ?
 REGA-ESG01-0>
 }}
@@ Line 983: / Line 1,053: @@
      Network            Next Hop       Metric  LocPrf  Weight AS Path
-##y##  > 10.11.11.0/24      10.200.19.1       0     100      60   65512 65510 i
+  > 10.11.11.0/24      10.200.19.1       0     100      60   65512 65510 i
-##y##    10.11.11.0/24      10.200.21.1       0     100      60   65512 65510 i
+.11.11.0/24      10.200.21.1       0     100      60   65512 65510 i
-   > 10.22.22.0/24      10.200.19.1       0     100      60   65512 65510 i
+   > 10.100.19.0/24     10.200.19.1       0     100      60   65512 65510 65511 65521 ?
-.22.22.0/24      10.200.21.1       0     100      60   65512 65510 i
+.100.19.0/24     10.200.21.1       0     100      60   65512 65510 65511 65521 ?
-   > 10.100.19.0/24     172.39.39.14      0     100      60   65530 ?
+   > 10.100.21.0/24     10.200.19.1       0     100      60   65512 65510 65511 65521 ?
-  > 10.100.21.0/24     172.39.39.14      0     100      60   65530 ?
+.100.21.0/24     10.200.21.1       0     100      60   65512 65510 65511 65521 ?
    > 10.200.19.0/24     0.0.0.0           0     100   32768   ?
    > 10.200.21.0/24     0.0.0.0           0     100   32768   ?
-##y##  > 172.20.1.0/24      172.39.39.14      0     100      60   65530 ?
+  > 172.20.1.0/24      10.200.19.1       0     100      60   65512 65510 65511 65521 ?
-##y##  > 172.20.2.0/24      172.39.39.14      0     100      60   65530 ?
+.20.1.0/24      10.200.21.1       0     100      60   65512 65510 65511 65521 ?
-##y##  > 172.20.3.0/24      172.39.39.14      0     100      60   65530 ?
+  > 172.20.2.0/24      10.200.19.1       0     100      60   65512 65510 65511 65521 ?
-   > 172.20.8.0/24      172.39.39.14      0     100      60   65530 ?
+.20.2.0/24      10.200.21.1       0     100      60   65512 65510 65511 65521 ?
-   > 172.20.9.0/24      172.39.39.14      0     100      60   65530 ?
+  > 172.20.3.0/24      10.200.19.1       0     100      60   65512 65510 65511 65521 ?
-   > 172.20.10.0/24     172.39.39.14      0     100      60   65530 ?
+.20.3.0/24      10.200.21.1       0     100      60   65512 65510 65511 65521 ?
-.39.39.0/28     172.39.39.14      0     100      60   65530 ?
+   > 172.20.8.0/24      10.200.19.1       0     100      60   65512 65510 65511 65521 ?
+.20.8.0/24      10.200.21.1       0     100      60   65512 65510 65511 65521 ?
+   > 172.20.9.0/24      10.200.19.1       0     100      60   65512 65510 65511 65521 ?
+.20.9.0/24      10.200.21.1       0     100      60   65512 65510 65511 65521 ?
+   > 172.20.10.0/24     10.200.19.1       0     100      60   65512 65510 65511 65521 ?
+.20.10.0/24     10.200.21.1       0     100      60   65512 65510 65511 65521 ?
+  > 172.30.1.0/24      172.39.39.46      0     100      60   65530 ?
+  > 172.30.2.0/24      172.39.39.46      0     100      60   65530 ?
+  > 172.30.3.0/24      172.39.39.46      0     100      60   65530 ?
    > 172.39.39.0/28     0.0.0.0           0     100   32768   ?
-   > 172.39.39.16/28    172.39.39.14      0     100      60   65530 ?
+   > 172.39.39.16/28    172.39.39.46      0     100      60   65530 ?
+.39.39.32/28    172.39.39.46      0     100      60   65530 ?
    > 172.39.39.32/28    0.0.0.0           0     100   32768   ?
 REGB-ESG01-0>
@@ Line 1,010: / Line 1,089: @@
 {{console|body=
 ##bl##rt-a-01#show ip bgp
-BGP table version is 145, local router ID is 10.11.11.31
+BGP table version is 54, local router ID is 10.11.11.31
 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
                r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
@@ Line 1,018: / Line 1,097: @@
       Network          Next Hop            Metric LocPrf Weight Path
-##y## r   10.11.11.0/24    10.100.19.2                            0 65521 i
+ r>  10.11.11.0/24    10.11.11.253             0             0 65510 i
-##y## r>                   10.11.11.253             0             0 65510 i
- *   10.22.22.0/24    10.100.19.2                            0 65521 i
- *>                   10.11.11.253             0             0 65510 i
   r>  10.100.19.0/24   10.100.19.2                            0 65521 ?
   *>  10.100.21.0/24   10.100.19.2                            0 65521 ?
-  r>  10.200.19.0/24   10.100.19.2                            0 65521 ?
+  r>  10.200.19.0/24   10.11.11.253                           0 65510 65512 65522 ?
- r                    10.11.11.253                           0 65510 65512 65522 ?
+  r>  10.200.21.0/24   10.11.11.253                           0 65510 65512 65522 ?
-  r>  10.200.21.0/24   10.100.19.2                            0 65521 ?
+ *>  172.20.1.0/24    10.100.19.2                            0 65521 ?
- r                    10.11.11.253                           0 65510 65512 65522 ?
+ *>  172.20.2.0/24    10.100.19.2                            0 65521 ?
-##y## *>  172.20.1.0/24    10.100.19.2                            0 65521 ?
+ *>  172.20.3.0/24    10.100.19.2                            0 65521 ?
-##y## *                    10.11.11.253                           0 65510 65512 65522 ?
-     Network          Next Hop            Metric LocPrf Weight Path
-##y## *>  172.20.2.0/24    10.100.19.2                            0 65521 ?
-##y## *                    10.11.11.253                           0 65510 65512 65522 ?
-##y## *>  172.20.3.0/24    10.100.19.2                            0 65521 ?
-##y## *                    10.11.11.253                           0 65510 65512 65522 ?
   *>  172.20.8.0/24    10.100.19.2                            0 65521 ?
+ *>  172.20.9.0/24    10.100.19.2                            0 65521 ?
+ *>  172.20.10.0/24   10.100.19.2                            0 65521 ?
+ *>  172.30.1.0/24    10.100.19.2                            0 65521 ?
   *                    10.11.11.253                           0 65510 65512 65522 ?
-  *>  172.20.9.0/24    10.100.19.2                            0 65521 ?
+     Network          Next Hop            Metric LocPrf Weight Path
+  *>  172.30.2.0/24    10.100.19.2                            0 65521 ?
   *                    10.11.11.253                           0 65510 65512 65522 ?
-  *>  172.20.10.0/24   10.100.19.2                            0 65521 ?
+  *>  172.30.3.0/24    10.100.19.2                            0 65521 ?
   *                    10.11.11.253                           0 65510 65512 65522 ?
   *>  172.39.39.0/28   10.100.19.2                            0 65521 ?
- *                    10.11.11.253                           0 65510 65512 65522 ?
   *>  172.39.39.16/28  10.100.19.2                            0 65521 ?
   *>  172.39.39.32/28  10.100.19.2                            0 65521 ?
@@ Line 1,051: / Line 1,124: @@
 {{console|body=
 ##bl##rt-a-02#show ip bgp
-BGP table version is 145, local router ID is 10.11.11.32
+BGP table version is 54, local router ID is 10.11.11.32
 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
                r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
@@ Line 1,059: / Line 1,132: @@
       Network          Next Hop            Metric LocPrf Weight Path
-  ##y##r   10.11.11.0/24    10.100.21.2                            0 65521 i
+  r>  10.11.11.0/24    10.11.11.253             0             0 65510 i
-  ##y##r>                   10.11.11.253             0             0 65510 i
- *  10.22.22.0/24    10.100.21.2                            0 65521 i
- *>                  10.11.11.253             0             0 65510 i
   *>  10.100.19.0/24   10.100.21.2                            0 65521 ?
   r>  10.100.21.0/24   10.100.21.2                            0 65521 ?
-  r>  10.200.19.0/24   10.100.21.2                            0 65521 ?
+  r>  10.200.19.0/24   10.11.11.253                           0 65510 65512 65522 ?
- r                    10.11.11.253                           0 65510 65512 65522 ?
+  r>  10.200.21.0/24   10.11.11.253                           0 65510 65512 65522 ?
-  r>  10.200.21.0/24   10.100.21.2                            0 65521 ?
+ *>  172.20.1.0/24    10.100.21.2                            0 65521 ?
- r                    10.11.11.253                           0 65510 65512 65522 ?
+ *>  172.20.2.0/24    10.100.21.2                            0 65521 ?
-##y## *>   172.20.1.0/24    10.100.21.2                            0 65521 ?
+ *>  172.20.3.0/24    10.100.21.2                            0 65521 ?
-##y## *                     10.11.11.253                           0 65510 65512 65522 ?
-     Network          Next Hop            Metric LocPrf Weight Path
-##y## *>   172.20.2.0/24    10.100.21.2                            0 65521 ?
-##y## *                    10.11.11.253                           0 65510 65512 65522 ?
-##y## *>  172.20.3.0/24    10.100.21.2                            0 65521 ?
-##y## *                    10.11.11.253                           0 65510 65512 65522 ?
   *>  172.20.8.0/24    10.100.21.2                            0 65521 ?
+ *>  172.20.9.0/24    10.100.21.2                            0 65521 ?
+ *>  172.20.10.0/24   10.100.21.2                            0 65521 ?
+ *>  172.30.1.0/24    10.100.21.2                            0 65521 ?
   *                    10.11.11.253                           0 65510 65512 65522 ?
-  *>  172.20.9.0/24    10.100.21.2                            0 65521 ?
+     Network          Next Hop            Metric LocPrf Weight Path
+  *>  172.30.2.0/24    10.100.21.2                            0 65521 ?
   *                    10.11.11.253                           0 65510 65512 65522 ?
-  *>  172.20.10.0/24   10.100.21.2                            0 65521 ?
+  *>  172.30.3.0/24    10.100.21.2                            0 65521 ?
   *                    10.11.11.253                           0 65510 65512 65522 ?
   *>  172.39.39.0/28   10.100.21.2                            0 65521 ?
- *                    10.11.11.253                           0 65510 65512 65522 ?
   *>  172.39.39.16/28  10.100.21.2                            0 65521 ?
   *>  172.39.39.32/28  10.100.21.2                            0 65521 ?
@@ Line 1,094: / Line 1,161: @@
 {{console|body=
 ##bl##rt-b-01#show ip bgp
-BGP table version is 152, local router ID is 10.11.11.33
+BGP table version is 76, local router ID is 10.11.11.33
 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
                r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
@@ Line 1,102: / Line 1,169: @@
       Network          Next Hop            Metric LocPrf Weight Path
-##y## r>  10.11.11.0/24    10.11.11.253             0             0 65510 i
+ r>  10.11.11.0/24    10.11.11.253             0             0 65510 i
-  *>  10.22.22.0/24    10.11.11.253             0             0 65510 i
+  r>  10.100.19.0/24   10.11.11.253                           0 65510 65511 65521 ?
- r   10.100.19.0/24   10.11.11.253                           0 65510 65511 65521 ?
+  r>  10.100.21.0/24   10.11.11.253                           0 65510 65511 65521 ?
-  r>                   10.200.19.2                            0 65522 ?
-  r   10.100.21.0/24   10.11.11.253                           0 65510 65511 65521 ?
- r>                   10.200.19.2                            0 65522 ?
   r>  10.200.19.0/24   10.200.19.2                            0 65522 ?
   *>  10.200.21.0/24   10.200.19.2                            0 65522 ?
-##y## *>  172.20.1.0/24    10.200.19.2                            0 65522 ?
+ *>  172.20.1.0/24    10.11.11.253                           0 65510 65511 65521 ?
-##y## *>  172.20.2.0/24    10.200.19.2                            0 65522 ?
+ *>  172.20.2.0/24    10.11.11.253                           0 65510 65511 65521 ?
-##y## *>  172.20.3.0/24    10.200.19.2                            0 65522 ?
+ *>  172.20.3.0/24    10.11.11.253                           0 65510 65511 65521 ?
-  *>  172.20.8.0/24    10.200.19.2                            0 65522 ?
+  *>  172.20.8.0/24    10.11.11.253                           0 65510 65511 65521 ?
       Network          Next Hop            Metric LocPrf Weight Path
-  *>  172.20.9.0/24    10.200.19.2                            0 65522 ?
+  *>  172.20.9.0/24    10.11.11.253                           0 65510 65511 65521 ?
-  *>  172.20.10.0/24   10.200.19.2                            0 65522 ?
+ *>  172.20.10.0/24   10.11.11.253                           0 65510 65511 65521 ?
+ *>  172.30.1.0/24    10.200.19.2                            0 65522 ?
+  *>  172.30.2.0/24    10.200.19.2                            0 65522 ?
+ *>  172.30.3.0/24    10.200.19.2                            0 65522 ?
   *>  172.39.39.0/28   10.200.19.2                            0 65522 ?
-  *   172.39.39.16/28  10.11.11.253                           0 65510 65511 65521 ?
+  *                    10.11.11.253                           0 65510 65511 65521 ?
-  *>                   10.200.19.2                            0 65522 ?
+  *>  172.39.39.16/28  10.200.19.2                            0 65522 ?
+ *                    10.11.11.253                           0 65510 65511 65521 ?
   *>  172.39.39.32/28  10.200.19.2                            0 65522 ?
 rt-b-01#
@@ Line 1,126: / Line 1,194: @@
 {{console|body=
 ##bl##rt-b-02#show ip bgp
-BGP table version is 152, local router ID is 10.11.11.34
+BGP table version is 49, local router ID is 10.11.11.34
 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
                r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
@@ Line 1,134: / Line 1,202: @@
       Network          Next Hop            Metric LocPrf Weight Path
-##y## r>  10.11.11.0/24    10.11.11.253             0             0 65510 i
+ r>  10.11.11.0/24    10.11.11.253             0             0 65510 i
-  *>  10.22.22.0/24    10.11.11.253             0             0 65510 i
+  r>  10.100.19.0/24   10.11.11.253                           0 65510 65511 65521 ?
- r   10.100.19.0/24   10.11.11.253                           0 65510 65511 65521 ?
+  r>  10.100.21.0/24   10.11.11.253                           0 65510 65511 65521 ?
-  r>                   10.200.21.2                            0 65522 ?
-  r   10.100.21.0/24   10.11.11.253                           0 65510 65511 65521 ?
- r>                   10.200.21.2                            0 65522 ?
   *>  10.200.19.0/24   10.200.21.2                            0 65522 ?
   r>  10.200.21.0/24   10.200.21.2                            0 65522 ?
-##y## *>  172.20.1.0/24    10.200.21.2                            0 65522 ?
+ *>  172.20.1.0/24    10.11.11.253                           0 65510 65511 65521 ?
-##y## *>  172.20.2.0/24    10.200.21.2                            0 65522 ?
+ *>  172.20.2.0/24    10.11.11.253                           0 65510 65511 65521 ?
-##y## *>  172.20.3.0/24    10.200.21.2                            0 65522 ?
+ *>  172.20.3.0/24    10.11.11.253                           0 65510 65511 65521 ?
-  *>  172.20.8.0/24    10.200.21.2                            0 65522 ?
+  *>  172.20.8.0/24    10.11.11.253                           0 65510 65511 65521 ?
       Network          Next Hop            Metric LocPrf Weight Path
-  *>  172.20.9.0/24    10.200.21.2                            0 65522 ?
+  *>  172.20.9.0/24    10.11.11.253                           0 65510 65511 65521 ?
-  *>  172.20.10.0/24   10.200.21.2                            0 65522 ?
+ *>  172.20.10.0/24   10.11.11.253                           0 65510 65511 65521 ?
+ *>  172.30.1.0/24    10.200.21.2                            0 65522 ?
+  *>  172.30.2.0/24    10.200.21.2                            0 65522 ?
+ *>  172.30.3.0/24    10.200.21.2                            0 65522 ?
   *>  172.39.39.0/28   10.200.21.2                            0 65522 ?
-  *   172.39.39.16/28  10.11.11.253                           0 65510 65511 65521 ?
+  *                    10.11.11.253                           0 65510 65511 65521 ?
-  *>                   10.200.21.2                            0 65522 ?
+  *>  172.39.39.16/28  10.200.21.2                            0 65522 ?
+ *                    10.11.11.253                           0 65510 65511 65521 ?
   *>  172.39.39.32/28  10.200.21.2                            0 65522 ?
 rt-b-02#
@@ Line 1,160: / Line 1,229: @@
 {{console|body=
 ##bl##cs-01#show ip bgp
-BGP table version is 153, local router ID is 10.11.11.253
+BGP table version is 83, local router ID is 10.11.11.253
 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
                r RIB-failure, S Stale, m multipath, b backup-path, x best-external
@@ Line 1,166: / Line 1,235: @@
     Network          Next Hop            Metric LocPrf Weight Path
-##y##*> 10.11.11.0/24    0.0.0.0                  0         32768 i
+*> 10.11.11.0/24    0.0.0.0                  0         32768 i
-##y##*> 10.22.22.0/24    0.0.0.0                  0         32768 i
 r  10.100.19.0/24   10.11.11.32                            0 65511 65521 ?
-r                   10.11.11.33                            0 65512 65522 ?
-r                   10.11.11.34                            0 65512 65522 ?
 r>                  10.11.11.31                            0 65511 65521 ?
 r  10.100.21.0/24   10.11.11.32                            0 65511 65521 ?
-r                   10.11.11.33                            0 65512 65522 ?
-r                   10.11.11.34                            0 65512 65522 ?
 r>                  10.11.11.31                            0 65511 65521 ?
-r  10.200.19.0/24   10.11.11.31                            0 65511 65521 ?
+r  10.200.19.0/24   10.11.11.33                            0 65512 65522 ?
-r                   10.11.11.32                            0 65511 65521 ?
-r                   10.11.11.33                            0 65512 65522 ?
 r>                  10.11.11.34                            0 65512 65522 ?
-r  10.200.21.0/24   10.11.11.31                            0 65511 65521 ?
+r  10.200.21.0/24   10.11.11.33                            0 65512 65522 ?
-r                   10.11.11.32                            0 65511 65521 ?
-r                   10.11.11.33                            0 65512 65522 ?
 r>                  10.11.11.34                            0 65512 65522 ?
-##y##*  172.20.1.0/24    10.11.11.31                            0 65511 65521 ?
+*  172.20.1.0/24    10.11.11.32                            0 65511 65521 ?
-##y##*                   10.11.11.32                            0 65511 65521 ?
+*>                  10.11.11.31                            0 65511 65521 ?
-##y##*>                  10.11.11.33                            0 65512 65522 ?
+*  172.20.2.0/24    10.11.11.32                            0 65511 65521 ?
-##y##*                   10.11.11.34                            0 65512 65522 ?
+*>                  10.11.11.31                            0 65511 65521 ?
-##y##*  172.20.2.0/24    10.11.11.31                            0 65511 65521 ?
+*  172.20.3.0/24    10.11.11.32                            0 65511 65521 ?
-##y##*                   10.11.11.32                            0 65511 65521 ?
+*>                  10.11.11.31                            0 65511 65521 ?
-##y##*>                  10.11.11.33                            0 65512 65522 ?
+*  172.20.8.0/24    10.11.11.32                            0 65511 65521 ?
-##y##*                   10.11.11.34                            0 65512 65522 ?
+*>                  10.11.11.31                            0 65511 65521 ?
-##y##*  172.20.3.0/24    10.11.11.31                            0 65511 65521 ?
+*  172.20.9.0/24    10.11.11.32                            0 65511 65521 ?
-##y##*                   10.11.11.32                            0 65511 65521 ?
+*>                  10.11.11.31                            0 65511 65521 ?
-##y##*>                  10.11.11.33                            0 65512 65522 ?
+*  172.20.10.0/24   10.11.11.32                            0 65511 65521 ?
-##y##*                   10.11.11.34                            0 65512 65522 ?
+*>                  10.11.11.31                            0 65511 65521 ?
-*  172.20.8.0/24    10.11.11.31                            0 65511 65521 ?
+*  172.30.1.0/24    10.11.11.31                            0 65511 65521 ?
 *                   10.11.11.32                            0 65511 65521 ?
+*                   10.11.11.34                            0 65512 65522 ?
 *>                  10.11.11.33                            0 65512 65522 ?
+*  172.30.2.0/24    10.11.11.31                            0 65511 65521 ?
+*                   10.11.11.32                            0 65511 65521 ?
 *                   10.11.11.34                            0 65512 65522 ?
-*  172.20.9.0/24    10.11.11.31                            0 65511 65521 ?
-*                   10.11.11.32                            0 65511 65521 ?
 *>                  10.11.11.33                            0 65512 65522 ?
-*                   10.11.11.34                            0 65512 65522 ?
+*  172.30.3.0/24    10.11.11.31                            0 65511 65521 ?
-*  172.20.10.0/24   10.11.11.31                            0 65511 65521 ?
 *                   10.11.11.32                            0 65511 65521 ?
-*>                  10.11.11.33                            0 65512 65522 ?
 *                   10.11.11.34                            0 65512 65522 ?
-*  172.39.39.0/28   10.11.11.32                            0 65511 65521 ?
-*                   10.11.11.31                            0 65511 65521 ?
 *>                  10.11.11.33                            0 65512 65522 ?
+*  172.39.39.0/28   10.11.11.33                            0 65512 65522 ?
 *                   10.11.11.34                            0 65512 65522 ?
-*  172.39.39.16/28  10.11.11.32                            0 65511 65521 ?
+*                   10.11.11.32                            0 65511 65521 ?
+*>                  10.11.11.31                            0 65511 65521 ?
+*  172.39.39.16/28  10.11.11.34                            0 65512 65522 ?
 *                   10.11.11.33                            0 65512 65522 ?
-*                   10.11.11.34                            0 65512 65522 ?
+*                   10.11.11.32                            0 65511 65521 ?
 *>                  10.11.11.31                            0 65511 65521 ?
 *  172.39.39.32/28  10.11.11.31                            0 65511 65521 ?
@@ Line 1,225: / Line 1,285: @@
 === Routing path verifications ===
-==== Routing path verification from the external client VM to T1-WEB-1 and T1-WEB-2 ====
+==== Routing path verification from the external client VM to T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 ====
-The expectation is that the traffic will route through DC1.
+This is an INGRESS trafficflow.
+As we did not make any configuration changes to manipulate the INGRESS trafficflow the INGRESS traffic can either choose to select the DC1 path or the DC2 path.
+This is currently determined by BGP.
 {{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.1
+##bl##C:\Users\Administrator>tracert 172.30.1.1
-Tracing route to 172.20.1.1 over a maximum of 30 hops
+Tracing route to 172.30.1.1 over a maximum of 30 hops
-     1 ms     1 ms     2 ms  cs-01.home.local [10.11.11.253]
+     1 ms     1 ms     1 ms  cs-01.home.local [10.11.11.253]
     <1 ms    <1 ms    <1 ms  10.11.11.33
-##y##  3    <1 ms    <1 ms    <1 ms  10.200.19.2
+    1 ms    <1 ms    <1 ms  10.200.21.2
-   <1 ms    <1 ms    <1 ms  172.39.39.14
+    1 ms    <1 ms    <1 ms  172.39.39.46
-     1 ms     1 ms     1 ms  172.20.1.1
+     1 ms     1 ms     1 ms  172.30.1.1
-Trace complete
+Trace complete.
 }}
 {{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.2
+##bl##C:\Users\Administrator>tracert 172.30.1.2
-Tracing route to 172.20.1.2 over a maximum of 30 hops
+Tracing route to 172.30.1.2 over a maximum of 30 hops
-   <1 ms     1 ms     1 ms  cs-01.home.local [10.11.11.253]
+    2 ms     1 ms     1 ms  cs-01.home.local [10.11.11.253]
     <1 ms    <1 ms    <1 ms  10.11.11.33
-##y##  3    <1 ms    <1 ms    <1 ms  10.200.19.2
+    <1 ms    <1 ms    <1 ms  10.200.21.2
-    <1 ms    <1 ms    <1 ms  172.39.39.14
+    <1 ms    <1 ms    <1 ms  172.39.39.46
-     1 ms    <1 ms    <1 ms  172.20.1.2
+  944 ms     1 ms    <1 ms  172.30.1.2
 Trace complete.
-}}
-==== Routing path verification from the T1-WEB-1 and T1-WEB-2 to the external client VM ====
-{{console|body=
-##bl##root@Web01:~# traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.713 ms  0.649 ms  0.613 ms
-  172.39.39.2 (172.39.39.2)  0.562 ms  0.599 ms  0.554 ms
- ##y##3  10.200.19.1 (10.200.19.1)  0.741 ms  0.893 ms  1.023 ms
-  10.11.11.50 (10.11.11.50)  1.231 ms * *
-root@Web01:~#
 }}
 {{console|body=
-##bl##root@Web02:~# traceroute 10.11.11.50
+##bl##C:\Users\Administrator>tracert 172.30.1.3
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.113 ms  0.080 ms  0.065 ms
-  172.39.39.2 (172.39.39.2)  0.488 ms  0.526 ms  0.481 ms
-##y## 3  10.200.19.1 (10.200.19.1)  0.861 ms  0.792 ms  0.729 ms
-  10.11.11.50 (10.11.11.50)  1.233 ms * *
-root@Web02:~#
-}}
-We are seeing that the ingress AND egress paths that are taken is the path trough DC2.
+Tracing route to 172.30.1.3 over a maximum of 30 hops
-I want this to be DC1 so I will have to make the weight higher on the UDLR towards EPG-A.
-[[File:nsx-routing-blog-15.png|600px]]
+    1 ms     1 ms     6 ms  cs-01.home.local [10.11.11.253]
+    <1 ms    <1 ms    <1 ms  10.11.11.33
-==== Routing path verification from the external client VM to T1-WEB-1 and T1-WEB-2 ====
+    <1 ms    <1 ms    <1 ms  10.200.21.2
+    <1 ms    <1 ms    <1 ms  172.39.39.46
-The expectation is that the traffic will route through DC1.
+  723 ms     1 ms    <1 ms  172.30.1.3
-{{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.1
-Tracing route to 172.20.1.1 over a maximum of 30 hops
-   <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.31
-##y##  3    <1 ms    <1 ms    <1 ms  10.100.21.2
-    <1 ms    <1 ms    <1 ms  172.39.39.14
-     1 ms    <1 ms    <1 ms  172.20.1.1
 Trace complete.
@@ Line 1,303: / Line 1,334: @@
 {{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.2
+##bl##C:\Users\Administrator>tracert 172.30.1.4
-Tracing route to 172.20.1.2 over a maximum of 30 hops
+Tracing route to 172.30.1.4 over a maximum of 30 hops
-   <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
+    1 ms     1 ms     4 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.31
+    <1 ms    <1 ms    <1 ms  10.11.11.33
-##y##  3    <1 ms    <1 ms    <1 ms  10.100.19.2
+    <1 ms    <1 ms    <1 ms  10.200.21.2
-    1 ms    <1 ms    <1 ms  172.39.39.14
+   <1 ms    <1 ms    <1 ms  172.39.39.46
-     5 ms    <1 ms    <1 ms  172.20.1.2
+   82 ms     1 ms    <1 ms  172.30.1.4
 Trace complete.
 }}
-==== Routing path verification from the T1-WEB-1 and T1-WEB-2 to the external client VM ====
+==== Routing path verification from the T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 to the external client VM ====
-The expectation is that the traffic will route through DC1.
+This is an EGRESS trafficflow.
+The expectation is that the traffic towards T3-WEB-1 and T3-WEB-2 will fo trough the DC1 routers and the traffic towards T3-WEB-3 and T3-WEB-4 will fo trough the DC2 routers.
+The reason for this is that Locale ID is configured.
 {{console|body=
 ##bl##root@Web01:~# traceroute 10.11.11.50
 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.169 ms  0.083 ms  0.096 ms
+  172.30.1.254 (172.30.1.254)  0.082 ms  2.689 ms  2.687 ms
-  172.39.39.1 (172.39.39.1)  0.195 ms  0.265 ms  0.273 ms
+  172.39.39.17 (172.39.39.17)  0.666 ms  0.527 ms  0.524 ms
-##y## 3  10.100.21.1 (10.100.21.1)  1.075 ms  0.964 ms  0.921 ms
+  10.100.21.1 (10.100.21.1)  1.067 ms  1.091 ms  1.072 ms
-  10.11.11.50 (10.11.11.50)  0.958 ms * *
+  10.11.11.50 (10.11.11.50)  1.347 ms * *
 root@Web01:~#
 }}
@@ Line 1,333: / Line 1,366: @@
 ##bl##root@Web02:~# traceroute 10.11.11.50
 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.190 ms  0.145 ms  0.160 ms
+  172.30.1.254 (172.30.1.254)  0.172 ms  0.149 ms  0.131 ms
-  172.39.39.1 (172.39.39.1)  0.445 ms  0.324 ms  0.389 ms
+  * * 172.39.39.17 (172.39.39.17)  0.455 ms
-##y## 3  10.100.19.1 (10.100.19.1)  0.679 ms  0.821 ms  0.744 ms
+  10.100.21.1 (10.100.21.1)  0.758 ms  0.599 ms  0.494 ms
-  10.11.11.50 (10.11.11.50)  0.938 ms * *
+  10.11.11.50 (10.11.11.50)  0.918 ms * *
 root@Web02:~#
 }}
-==== Routing path verification from the external client VM to T1-WEB-3 and T1-WEB-4 ====
-The expectation is that the traffic will route through DC1.
-{{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.3
-Tracing route to 172.20.1.3 over a maximum of 30 hops
-    <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.31
-##y##  3    <1 ms    <1 ms    <1 ms  10.100.21.2
-    <1 ms    <1 ms    <1 ms  172.39.39.14
-   202 ms     1 ms     1 ms  172.20.1.3
-Trace complete.
-}}
-{{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.4
-Tracing route to 172.20.1.4 over a maximum of 30 hops
-    <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.31
-##y##  3    <1 ms    <1 ms    <1 ms  10.100.21.2
-    <1 ms    <1 ms    <1 ms  172.39.39.14
-   779 ms     1 ms    <1 ms  172.20.1.4
-Trace complete.
-}}
-==== Routing path verification from the T1-WEB-3 and T1-WEB-4 to the external client VM ====
-The expectation is that the traffic will route through DC1.
 {{console|body=
 ##bl##root@Web03:~# traceroute 10.11.11.50
 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.255 ms  0.228 ms  0.190 ms
+  172.30.1.254 (172.30.1.254)  0.120 ms  0.109 ms  0.060 ms
-  172.39.39.1 (172.39.39.1)  0.608 ms  0.568 ms  0.605 ms
+  172.39.39.33 (172.39.39.33)  0.387 ms  0.345 ms  0.445 ms
-##y## 3  10.100.21.1 (10.100.21.1)  1.086 ms  1.197 ms  1.148 ms
+  10.200.21.1 (10.200.21.1)  1.249 ms  1.371 ms  1.383 ms
-  10.11.11.50 (10.11.11.50)  1.603 ms * *
+  10.11.11.50 (10.11.11.50)  1.210 ms * *
 root@Web03:~#
 }}
@@ Line 1,389: / Line 1,386: @@
 ##bl##root@Web04:~# traceroute 10.11.11.50
 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.127 ms  0.079 ms  0.058 ms
+  172.30.1.254 (172.30.1.254)  0.109 ms  0.181 ms  0.132 ms
-  172.39.39.1 (172.39.39.1)  0.682 ms  0.751 ms  0.702 ms
+  172.39.39.33 (172.39.39.33)  0.471 ms  0.470 ms  0.466 ms
-##y## 3  10.100.21.1 (10.100.21.1)  1.278 ms  1.348 ms  1.300 ms
+  10.200.21.1 (10.200.21.1)  0.900 ms  0.871 ms  0.868 ms
-  10.11.11.50 (10.11.11.50)  1.341 ms * *
+  10.11.11.50 (10.11.11.50)  1.063 ms * *
 root@Web04:~#
 }}
-The routing components in DC1 may fail or in case of a disaster the full site may go down.
+=== Change the Locale ID manually on DC2 to match DC1 ===
-In that case, the routing should flow through DC2.
-Let’s test this…
+==== DC1 ====
-Turn RT-A-01 off.
+The locale ID will not change here and these settings are kept te same
-Verify paths again.
-The expectation is that the traffic will route through RT-A-02.
-==== Routing path verification from the external client VM to T1-WEB-1, T1-WEB-2, T1-WEB-3 and T1-WEB-4 ====
+==== DC2 ====
-{{console|body=
+===== Compute Cluster =====
-##bl##C:\Users\Administrator>tracert 172.20.1.1
-Tracing route to 172.20.1.1 over a maximum of 30 hops
+[[File:nsx-routing-blog-locale-25.png|600px]]
-     1 ms     2 ms     1 ms  cs-01.home.local [10.11.11.253]
+=== Routing path verifications ===
-    <1 ms    <1 ms    <1 ms  10.11.11.32
-##y##  3    <1 ms    <1 ms    <1 ms  10.100.21.2
-    <1 ms    <1 ms    <1 ms  172.39.39.14
-     1 ms    <1 ms    <1 ms  172.20.1.1
-Trace complete.
+==== Routing path verification from the external client VM to T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 ====
-}}
-{{console|body=
+This is an INGRESS trafficflow.
-##bl##C:\Users\Administrator>tracert 172.20.1.2
+As we did not make any configuration changes to manipulate the INGRESS trafficflow the INGRESS traffic can either choose to select the DC1 path or the DC2 path.
+This is currently determined by BGP.
-Tracing route to 172.20.1.2 over a maximum of 30 hops
-    <1 ms     1 ms     1 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.32
-##y##  3    <1 ms    <1 ms    <1 ms  10.100.21.2
-    <1 ms    <1 ms    <1 ms  172.39.39.14
-     5 ms    <1 ms    <1 ms  172.20.1.2
-Trace complete.
-}}
 {{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.3
+##bl##C:\Users\Administrator>tracert 172.30.1.1
-Tracing route to 172.20.1.3 over a maximum of 30 hops
+Tracing route to 172.30.1.1 over a maximum of 30 hops
-    2 ms     1 ms     1 ms  cs-01.home.local [10.11.11.253]
+    <1 ms    <1 ms     3 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.32
-##y##  3    <1 ms    <1 ms    <1 ms  10.100.21.2
-    <1 ms    <1 ms    <1 ms  172.39.39.14
-     2 ms     1 ms     1 ms  172.20.1.3
-Trace complete.
-}}
-{{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.4
-Tracing route to 172.20.1.4 over a maximum of 30 hops
-     2 ms     1 ms     1 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.32
-##y##  3    <1 ms    <1 ms    <1 ms  10.100.21.2
-    <1 ms    <1 ms    <1 ms  172.39.39.14
-     1 ms    <1 ms     1 ms  172.20.1.4
-Trace complete.
-}}
-==== Routing path verification from the T1-WEB-1, T1-WEB-2, T1-WEB-3 and T1-WEB-4 to the external client VM ====
-{{console|body=
-##bl##root@Web01:~#  traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.141 ms  0.066 ms  0.118 ms
-  172.39.39.1 (172.39.39.1)  0.213 ms  0.246 ms  0.196 ms
-##y## 3  10.100.21.1 (10.100.21.1)  0.734 ms  0.662 ms  0.612 ms
-  10.11.11.50 (10.11.11.50)  0.831 ms * *
-root@Web01:~#
-}}
-{{console|body=
-##bl##root@Web02:~#  traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  1.098 ms  1.051 ms  1.055 ms
-  172.39.39.1 (172.39.39.1)  0.815 ms  0.799 ms  0.851 ms
-##y## 3  10.100.21.1 (10.100.21.1)  0.778 ms  0.765 ms  0.980 ms
-  10.11.11.50 (10.11.11.50)  0.846 ms * *
-root@Web02:~#
-}}
-{{console|body=
-##bl##root@Web03:~#  traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.524 ms  0.480 ms  0.435 ms
-  172.39.39.1 (172.39.39.1)  0.879 ms  0.945 ms  0.920 ms
-##y## 3  10.100.21.1 (10.100.21.1)  1.224 ms  1.401 ms  1.400 ms
-  10.11.11.50 (10.11.11.50)  2.391 ms * *
-root@Web03:~#
-}}
-{{console|body=
-##bl##root@Web04:~#  traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.197 ms  0.174 ms  0.194 ms
-  172.39.39.1 (172.39.39.1)  0.726 ms  0.699 ms  0.649 ms
-##y## 3  10.100.21.1 (10.100.21.1)  1.137 ms  1.258 ms  1.255 ms
-  10.11.11.50 (10.11.11.50)  1.297 ms * *
-root@Web04:~#
-}}
-Turn RT-A-02 off.
-Verify paths again.
-The expectation is that the traffic will route through DC2 as both upstream routers in DC1 are down.
-==== Routing path verification from the external client VM to T1-WEB-1 , T1-WEB-2, T1-WEB-3 and T1-WEB-4 ====
-{{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.1
-Tracing route to 172.20.1.1 over a maximum of 30 hops
-     2 ms     1 ms     1 ms  cs-01.home.local [10.11.11.253]
     <1 ms    <1 ms    <1 ms  10.11.11.33
-##y##  3    <1 ms    <1 ms    <1 ms  10.200.19.2
+    1 ms    <1 ms    <1 ms  10.200.21.2
-   <1 ms    <1 ms    <1 ms  172.39.39.14
+  231 ms    <1 ms    <1 ms  172.39.39.30
-    1 ms     1 ms    <1 ms  172.20.1.1
+  966 ms     1 ms    <1 ms  172.30.1.1
 Trace complete.
 }}
-{{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.2
-Tracing route to 172.20.1.2 over a maximum of 30 hops
-    <1 ms     1 ms     1 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.33
-##y##  3    <1 ms    <1 ms    <1 ms  10.200.19.2
-     1 ms    <1 ms    <1 ms  172.39.39.14
-     7 ms    <1 ms    <1 ms  172.20.1.2
-Trace complete.
-}}
 {{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.3
+##bl##C:\Users\Administrator>tracert 172.30.1.2
-Tracing route to 172.20.1.3 over a maximum of 30 hops
+Tracing route to 172.30.1.2 over a maximum of 30 hops
-    2 ms     1 ms     1 ms  cs-01.home.local [10.11.11.253]
+    <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.33
-##y##  3    <1 ms    <1 ms    <1 ms  10.200.19.2
-     1 ms    <1 ms    <1 ms  172.39.39.14
-     1 ms     1 ms     1 ms  172.20.1.3
-Trace complete.
-}}
-{{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.4
-Tracing route to 172.20.1.4 over a maximum of 30 hops
-     8 ms     3 ms     2 ms  cs-01.home.local [10.11.11.253]
     <1 ms    <1 ms    <1 ms  10.11.11.33
-##y##  3    <1 ms    <1 ms    <1 ms  10.200.19.2
+    <1 ms    <1 ms    <1 ms  10.200.21.2
-    <1 ms     4 ms    <1 ms  172.39.39.14
+    1 ms    <1 ms    <1 ms  172.39.39.30
-     4 ms    <1 ms    <1 ms  172.20.1.4
+     1 ms     1 ms    <1 ms  172.30.1.2
 Trace complete.
 }}
-==== Routing path verification from the T1-WEB-1, T1-WEB-2, T1-WEB-3 and T1-WEB-4 to the external client VM ====
-{{console|body=
-##bl##root@Web01:~#  traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.118 ms  0.079 ms  0.102 ms
-  172.39.39.2 (172.39.39.2)  0.767 ms  0.753 ms  0.691 ms
-##y## 3  10.200.19.1 (10.200.19.1)  0.855 ms  0.869 ms  0.972 ms
-  10.11.11.50 (10.11.11.50)  1.105 ms * *
-root@Web01:~#
-}}
-{{console|body=
-##bl##root@Web02:~#  traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.135 ms  0.085 ms  0.076 ms
-  172.39.39.2 (172.39.39.2)  0.919 ms  0.872 ms  0.830 ms
-##y## 3  10.200.19.1 (10.200.19.1)  1.138 ms  1.075 ms  1.034 ms
-  10.11.11.50 (10.11.11.50)  1.795 ms * *
-root@Web02:~#
-}}
-{{console|body=
-##bl##root@Web03:~#  traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.199 ms  0.155 ms  0.095 ms
-  172.39.39.2 (172.39.39.2)  0.353 ms  0.400 ms  0.639 ms
-##y## 3  10.200.19.1 (10.200.19.1)  0.722 ms  0.803 ms  0.818 ms
-  10.11.11.50 (10.11.11.50)  0.961 ms * *
-root@Web03:~#
-}}
-{{console|body=
-##bl##root@Web04:~#  traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.184 ms  0.181 ms  0.120 ms
-  172.39.39.2 (172.39.39.2)  0.435 ms  0.477 ms  0.587 ms
-##y## 3  10.200.19.1 (10.200.19.1)  0.867 ms  0.887 ms  0.819 ms
-  10.11.11.50 (10.11.11.50)  0.913 ms * *
-root@Web04:~#
-}}
-Turn RT-A-01 and RT-A-02 back on and turn ESG-A off.
-Verify paths again.
-The expectation is that the traffic will route through DC2 the only ESG in DC1 is down.
-Before I turned off ESG-A I have verified if the traffic was flowing through DC1 again as an extra verification step. This is not shown below.
-For some reason the traffic kept flowing through DC1 so I had to turn BGP off/on again on the ESG in DC2 and the this caused the traffic to flow back trough DC1.
-==== Routing path verification from the external client VM to T1-WEB-1 , T1-WEB-2, T1-WEB-3 and T1-WEB-4 ====
 {{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.1
+##bl##C:\Users\Administrator>tracert 172.30.1.3
-Tracing route to 172.20.1.1 over a maximum of 30 hops
+Tracing route to 172.30.1.3 over a maximum of 30 hops
-    <1 ms    <1 ms     1 ms  cs-01.home.local [10.11.11.253]
+    <1 ms    <1 ms     3 ms  cs-01.home.local [10.11.11.253]
     <1 ms    <1 ms    <1 ms  10.11.11.33
-##y##  3    <1 ms    <1 ms    <1 ms  10.200.19.2
+    <1 ms    <1 ms    <1 ms  10.200.21.2
-   <1 ms    <1 ms    <1 ms  172.39.39.14
+    1 ms    <1 ms    <1 ms  172.39.39.30
-     2 ms     1 ms    <1 ms  172.20.1.1
+     2 ms    <1 ms    <1 ms  172.30.1.3
 Trace complete.
 }}
-{{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.2
-Tracing route to 172.20.1.2 over a maximum of 30 hops
-    <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.33
-##y##  3    <1 ms    <1 ms    <1 ms  10.200.19.2
-    <1 ms    <1 ms    <1 ms  172.39.39.14
-     3 ms     1 ms     1 ms  172.20.1.2
-Trace complete.
-}}
 {{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.3
+##bl##C:\Users\Administrator>tracert 172.30.1.4
-Tracing route to 172.20.1.3 over a maximum of 30 hops
+Tracing route to 172.30.1.4 over a maximum of 30 hops
     <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
     <1 ms    <1 ms    <1 ms  10.11.11.33
-##y##  3    <1 ms    <1 ms    <1 ms  10.200.19.2
+    <1 ms    <1 ms    <1 ms  10.200.21.2
-   <1 ms    <1 ms    <1 ms  172.39.39.14
+    1 ms    <1 ms    <1 ms  172.39.39.30
-     1 ms    <1 ms    <1 ms  172.20.1.3
+  761 ms     1 ms     1 ms  172.30.1.4
 Trace complete.
 }}
-{{console|body=
+==== Routing path verification from the T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 to the external client VM ====
-##bl##C:\Users\Administrator>tracert 172.20.1.4
-Tracing route to 172.20.1.4 over a maximum of 30 hops
-    <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
-   <1 ms    <1 ms    <1 ms  10.11.11.33
-##y##  3    <1 ms    <1 ms    <1 ms  10.200.19.2
-    <1 ms    <1 ms    <1 ms  172.39.39.14
-     1 ms    <1 ms    <1 ms  172.20.1.4
-Trace complete.
+This is an EGRESS trafficflow.
-}}
+The expectation is that the traffic towards T3-WEB-1, T3-WEB-2, T3-WEB-3 and T3-WEB-4 will go trough the DC1 routers.
+The reason for this is that Locale ID is configured and tweaked.
-==== Routing path verification from the T1-WEB-1, T1-WEB-2, T1-WEB-3 and T1-WEB-4 to the external client VM ====
 {{console|body=
-##bl##root@Web01:~#  traceroute 10.11.11.50
+##bl##root@Web01:~# traceroute 10.11.11.50
 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.142 ms  0.082 ms  0.094 ms
+  172.30.1.254 (172.30.1.254)  0.209 ms  0.086 ms  0.091 ms
-  172.39.39.2 (172.39.39.2)  2.189 ms  2.164 ms  2.132 ms
+  172.39.39.17 (172.39.39.17)  0.612 ms  0.603 ms  0.594 ms
-##y## 3  10.200.19.1 (10.200.19.1)  2.081 ms  5.269 ms  5.250 ms
+  10.100.21.1 (10.100.21.1)  1.129 ms  1.234 ms  1.184 ms
-  10.11.11.50 (10.11.11.50)  5.254 ms * *
+  10.11.11.50 (10.11.11.50)  1.583 ms * *
 root@Web01:~#
 }}
 {{console|body=
-##bl##root@Web02:~#  traceroute 10.11.11.50
+##bl##root@Web02:~# traceroute 10.11.11.50
 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.186 ms  0.094 ms  0.110 ms
+  172.30.1.254 (172.30.1.254)  0.223 ms  0.098 ms  0.107 ms
-  172.39.39.2 (172.39.39.2)  0.623 ms  0.637 ms  0.547 ms
+  172.39.39.17 (172.39.39.17)  0.559 ms  0.564 ms  0.546 ms
-##y## 3  10.200.19.1 (10.200.19.1)  0.843 ms  0.773 ms  0.740 ms
+  10.100.21.1 (10.100.21.1)  1.179 ms  1.226 ms  1.221 ms
-  10.11.11.50 (10.11.11.50)  1.089 ms * *
+  10.11.11.50 (10.11.11.50)  2.281 ms * *
 root@Web02:~#
 }}
 {{console|body=
-##bl##root@Web03:~#  traceroute 10.11.11.50
+##bl##root@Web03:~# traceroute 10.11.11.50
 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.091 ms  0.108 ms  0.068 ms
+  172.30.1.254 (172.30.1.254)  0.250 ms  0.126 ms  0.130 ms
-  172.39.39.2 (172.39.39.2)  0.227 ms  0.274 ms  0.363 ms
+  * * 172.39.39.17 (172.39.39.17)  0.991 ms
-##y## 3  10.200.19.1 (10.200.19.1)  0.669 ms  0.645 ms  0.525 ms
+  10.100.19.1 (10.100.19.1)  1.303 ms  1.323 ms  1.350 ms
-  10.11.11.50 (10.11.11.50)  0.636 ms * *
+  10.11.11.50 (10.11.11.50)  1.345 ms * *
 root@Web03:~#
 }}
 {{console|body=
-##bl##root@Web04:~#  traceroute 10.11.11.50
+##bl##root@Web04:~# traceroute 10.11.11.50
 traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.149 ms  0.178 ms  0.127 ms
+  172.30.1.254 (172.30.1.254)  0.147 ms  0.093 ms  0.079 ms
-  172.39.39.2 (172.39.39.2)  0.331 ms  0.383 ms  0.300 ms
+  * * 172.39.39.17 (172.39.39.17)  1.076 ms
-##y## 3  10.200.19.1 (10.200.19.1)  0.513 ms  0.576 ms  0.536 ms
+  10.100.19.1 (10.100.19.1)  1.951 ms  1.948 ms  2.100 ms
-  10.11.11.50 (10.11.11.50)  0.794 ms * *
+  10.11.11.50 (10.11.11.50)  1.799 ms * *
 root@Web04:~#
 }}
-Turn RT-A-01, RT-A-02 and turn ESG-A back on.
+Lets now put the Locale ID back to how it was and have the egress traffic exiting locally again.
-Verify paths again.
+What happens in case of any failures on DC2.
-The expectation is that everything is back to normal now.
+Does DC1 takes over the path for the DC2 workloads when the ESG-B goes down automatically?
-For some reason after turning on the ESG on DC1 the routes kept flowing through DC2.
-The same “issue” we saw before.
-So, I turned BGP off/on again on the ESG in DC2. This is not shown below. Below you will see the outputs after the BGP “reset” on DC2.
-==== Routing path verification from the external client VM to T1-WEB-1 , T1-WEB-2, T1-WEB-3 and T1-WEB-4 ====
-{{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.1
-Tracing route to 172.20.1.1 over a maximum of 30 hops
-    <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.31
-##y##  3    <1 ms    <1 ms    <1 ms  10.100.21.2
-     1 ms    <1 ms    <1 ms  172.39.39.14
-     1 ms    <1 ms    <1 ms  172.20.1.1
-Trace complete.
-}}
-{{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.2
-Tracing route to 172.20.1.2 over a maximum of 30 hops
-     1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.31
-##y##  3    <1 ms    <1 ms    <1 ms  10.100.19.2
-    <1 ms    <1 ms    <1 ms  172.39.39.14
-     3 ms    <1 ms    <1 ms  172.20.1.2
-Trace complete.
-}}
-{{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.3
-Tracing route to 172.20.1.3 over a maximum of 30 hops
-    <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms    <1 ms  10.11.11.31
-##y##  3     1 ms    <1 ms    <1 ms  10.100.19.2
-     1 ms    <1 ms    <1 ms  172.39.39.14
-     5 ms     1 ms     2 ms  172.20.1.3
-Trace complete.
-}}
-{{console|body=
-##bl##C:\Users\Administrator>tracert 172.20.1.4
-Tracing route to 172.20.1.4 over a maximum of 30 hops
-    <1 ms    <1 ms    <1 ms  cs-01.home.local [10.11.11.253]
-    <1 ms    <1 ms     1 ms  10.11.11.31
-##y##  3    <1 ms    <1 ms    <1 ms  10.100.19.2
-    <1 ms    <1 ms    <1 ms  172.39.39.14
-     5 ms     1 ms    <1 ms  172.20.1.4
-Trace complete.
-}}
-==== Routing path verification from the T1-WEB-1, T1-WEB-2, T1-WEB-3 and T1-WEB-4 to the external client VM ====
-{{console|body=
-##bl##root@Web01:~#  traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.153 ms  0.118 ms  0.186 ms
-  172.39.39.1 (172.39.39.1)  0.239 ms  0.272 ms  0.292 ms
-##y## 3  10.100.21.1 (10.100.21.1)  0.643 ms  0.611 ms  0.581 ms
-  10.11.11.50 (10.11.11.50)  0.959 ms * *
-root@Web01:~#
-}}
-{{console|body=
-##bl##root@Web02:~#  traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.137 ms  0.083 ms  0.069 ms
-  172.39.39.1 (172.39.39.1)  0.285 ms  0.386 ms  0.330 ms
-##y## 3  10.100.19.1 (10.100.19.1)  0.639 ms  0.542 ms  0.519 ms
-  10.11.11.50 (10.11.11.50)  0.785 ms * *
-root@Web02:~#
-}}
-{{console|body=
-##bl##root@Web03:~#  traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.127 ms  0.072 ms  0.069 ms
-  172.39.39.1 (172.39.39.1)  0.579 ms  0.704 ms  0.671 ms
-##y## 3  10.100.19.1 (10.100.19.1)  0.631 ms  0.896 ms  1.064 ms
-  10.11.11.50 (10.11.11.50)  1.063 ms * *
-root@Web03:~#
-}}
-{{console|body=
-##bl##root@Web02:~#  traceroute 10.11.11.50
-traceroute to 10.11.11.50 (10.11.11.50), 30 hops max, 60 byte packets
-  172.20.1.254 (172.20.1.254)  0.142 ms  0.199 ms  0.163 ms
-  172.39.39.1 (172.39.39.1)  0.555 ms  0.450 ms  0.395 ms
-##y## 3  10.100.19.1 (10.100.19.1)  0.884 ms  0.866 ms  0.934 ms
-  10.11.11.50 (10.11.11.50)  1.103 ms * *
-root@Web02:~#
-}}
-=== Additional investigation on routing table problem ===
-I did some investigation “why” DC1 would not become the primary routing path when the DC1 full path is available again.
-In the BGP tables in the beginning (show ip bgp) the UDLR only showed ONE path to the 10.11.11.0/24 network.
-The UDLR selects 172.39.39.2 (the interface of the ESG in DC2) as the next hop.
-{{console|body=
-##bl##REGX-UDLR01-0> show ip bgp
-Status codes: s - suppressed, d - damped, > - best, i - internal
-Origin codes: i - IGP, e - EGP, ? - incomplete
-    Network            Next Hop       Metric  LocPrf  Weight AS Path
-##y##  > 10.11.11.0/24      172.39.39.2      0     100      30   65522 i
-REGA-ESG01-0>
-}}
-The ESG of DC1 showed three paths to the 10.11.11.0/24 network. (The output below is truncated to make it more readable)
-One is retrieved from RT-A-01 the other from RT-A-02 and the other one is retrieved from the UDLR on interface 172.39.39.14.
-{{console|body=
-##bl##REGA-ESG01-0> show ip bgp
-Status codes: s - suppressed, d - damped, > - best, i - internal
-Origin codes: i - IGP, e - EGP, ? - incomplete
-    Network            Next Hop       Metric  LocPrf  Weight AS Path
-##y##    10.11.11.0/24      10.100.19.1       0     100      60   65511 65510 i
-##y##    10.11.11.0/24      10.100.21.1       0     100      60   65511 65510 i
-##y##  > 10.11.11.0/24      172.39.39.14      0     100      60   65530 i
-REGA-ESG01-0>
-}}
-If we look at the ESG in DC2 we see only two paths to the 10.11.11.0/24 network.
-One is retrieved from RT-B-01 the other from RT-B-02. (The output below is truncated to make it more readable)
-{{console|body=
-##bl##REGB-ESG01-0> show ip bgp
-Status codes: s - suppressed, d - damped, > - best, i - internal
-Origin codes: i - IGP, e - EGP, ? - incomplete
-    Network            Next Hop       Metric  LocPrf  Weight AS Path
-##y##  > 10.11.11.0/24      10.200.19.1       0     100      60   65512 65510 i
-##y##    10.11.11.0/24      10.200.21.1       0     100      60   65512 65510 i
-REGB-ESG01-0>
-}}
-I did not find a real answer, other than the fact that the UDLR should display two routes towards the 10.11.11.0/24 network and it should show these routes in the BGP table.
+=== Take ESG-B down and verify the paths ===
-For some reason this is not happening and I have no explanation why.
-But in terms of operations this does not matter because the goal is to have two paths to the north (10.11.11.0.24) and if one goes down the other one should take over and this happens as we could see above.
-<br />
+After I have verified if the locale ID is working as it should I did a power-off on the ESG-B to see what happens.
-[[Category:Articles]]
+The expectation is that all the traffic that was exiting the DC2 WAN routers will now go trough the DC1 WAN routers.
-[[Category:VMware]]
+After the power-off of the ESG-B the route was not rerouted automatically.
+A change of the Locale ID of the DC2 compute cluster (manually) forced a re-route to the DC1 WAN routers.