The NSX PowerOPS Tool unleashed

Introduction

A few months back one of my colleagues and good friend Dominic did a presentation about NSX PowerOPS. Due to a busy schedule, I was not able to try it and test it up until last week.

Business Problem?

NSX has transformed customer’s approach to Networking & Security and has achieved a significant installed base. Operations is challenged with Day-2 understanding, documentation & health of their NSX deployments.

The main business problem that arises here is that one has all the tools suggested to operationalize NSX but the team is still at a loss when they have to use them to troubleshoot NSX”

For who is this?

The NSX Power Operations (Tool) is for people who are responsible for the installation or administrators of the NSX environment.

PowerOPS - What is it? And what does it do?

The NSX Power Operations (Tool) is a tool that is Day-2 focused tool to extend product usefulness & adoption, Automated documentation, focusing on daily operational activities.

The installation

The installation steps are outlined below.

Check version of PowerShell

Make sure version 5 or above of PowerShell is installed, and if it is installed already you can verify this with the following command:

PS C:\Users\Administrator> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.14393.0
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.0
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

PS C:\Users\Administrator>


Here you see that I already had version 5.x installed of PowerShell.

Change Execution Policy

PS C:\Users\Administrator> Set-ExecutionPolicy RemoteSigned

Execution Policy Change
The execution policy helps protect you from scripts that you do not trust. Changing the execution policy might expose
you to the security risks described in the about_Execution_Policies help topic at
http://go.microsoft.com/fwlink/?LinkID=135170. Do you want to change the execution policy?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): A
PS C:\Users\Administrator>


Install PowerCLI

PS C:\Users\Administrator> Find-Module VMware.PowerCLI


A quick verification tells us that PowerCLI in installed correctly.

PS C:\Users\Administrator> Find-Module VMware.PowerCLI

Version    Name                                Repository           Description
-------    ----                                ----------           -----------
10.2.0.... VMware.PowerCLI                     PSGallery            This Windows PowerShell module contains VMware.Po...


We can also verify what VMware PowerCLI modules are now available.

PS C:\Users\Administrator> Get-Module VMware* -ListAvailable

    Directory: C:\Users\Administrator\Documents\WindowsPowerShell\Modules

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Script     6.7.0.8... VMware.DeployAutomation             {Add-DeployRule, Add-ProxyServer, Add-ScriptBundle, Copy-De...
Script     6.7.0.8... VMware.ImageBuilder                 {Add-EsxSoftwareDepot, Add-EsxSoftwarePackage, Compare-EsxI...
Manifest   10.2.0.... VMware.PowerCLI
Script     6.7.0.9... VMware.Vim
Script     10.1.0.... VMware.VimAutomation.Cis.Core       {Connect-CisServer, Disconnect-CisServer, Get-CisService}
Script     10.0.0.... VMware.VimAutomation.Cloud          {Add-CIDatastore, Connect-CIServer, Disconnect-CIServer, Ge...
Script     10.1.0.... VMware.VimAutomation.Common
Script     10.1.0.... VMware.VimAutomation.Core           {Add-PassthroughDevice, Add-VirtualSwitchPhysicalNetworkAda...
Script     6.5.4.7... VMware.VimAutomation.HA             Get-DrmInfo
Script     7.5.0.8... VMware.VimAutomation.HorizonView    {Connect-HVServer, Disconnect-HVServer}
Script     10.0.0.... VMware.VimAutomation.License        Get-LicenseDataManager
Script     10.2.0.... VMware.VimAutomation.Nsxt           {Connect-NsxtServer, Disconnect-NsxtServer, Get-NsxtService}
Script     10.0.0.... VMware.VimAutomation.PCloud         {Connect-PIServer, Disconnect-PIServer, Get-PIComputeInstan...
Script     10.1.0.... VMware.VimAutomation.Sdk
Script     10.0.0.... VMware.VimAutomation.Srm            {Connect-SrmServer, Disconnect-SrmServer}
Script     10.1.0.... VMware.VimAutomation.Storage        {Add-KeyManagementServer, Copy-VDisk, Export-SpbmStoragePol...
Script     1.2.0.0    VMware.VimAutomation.StorageUtility Update-VmfsDatastore
Script     10.1.0.... VMware.VimAutomation.Vds            {Add-VDSwitchPhysicalNetworkAdapter, Add-VDSwitchVMHost, Ex...
Script     10.0.0.... VMware.VimAutomation.Vmc            {Connect-Vmc, Disconnect-Vmc, Get-VmcService, Connect-VmcSe...
Script     10.0.0.... VMware.VimAutomation.vROps          {Connect-OMServer, Disconnect-OMServer, Get-OMAlert, Get-OM...
Script     6.5.1.7... VMware.VumAutomation                {Add-EntityBaseline, Copy-Patch, Get-Baseline, Get-Complian...


Download GIT and Download PowerOPS with using a command windows (not powershell)

Download GIT from Media:https://git-scm.com/downloads this website Download PowerOPS with using a regular command window (NOT PowerShell).

C:\Users\Administrator>git clone https://github.com/vmware/nsx-powerops.git
Cloning into 'nsx-powerops'...
remote: Counting objects: 849, done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 849 (delta 23), reused 28 (delta 13), pack-reused 800R
Receiving objects: 100% (849/849), 1.14 MiB


Verify if the git repository is cloned properly.

C:\Users\Administrator>dir
 Volume in drive C has no label.
 Volume Serial Number is BA5F-76FD

 Directory of C:\Users\Administrator

09/18/2018  04:19 AM    <DIR>          .
09/18/2018  04:19 AM    <DIR>          ..
09/18/2018  02:19 AM    <DIR>          Contacts
09/18/2018  02:19 AM    <DIR>          Desktop
09/18/2018  02:34 AM    <DIR>          Documents
09/18/2018  04:12 AM    <DIR>          Downloads
09/18/2018  02:19 AM    <DIR>          Favorites
09/18/2018  02:19 AM    <DIR>          Links
09/18/2018  02:19 AM    <DIR>          Music
09/18/2018  04:19 AM    <DIR>          nsx-powerops
09/18/2018  02:19 AM    <DIR>          Pictures
09/18/2018  02:19 AM    <DIR>          Saved Games
09/18/2018  02:19 AM    <DIR>          Searches
09/18/2018  02:19 AM    <DIR>          Videos
               0 File(s)              0 bytes
              14 Dir(s)  89,933,549,568 bytes free

C:\Users\Administrator>


Run the PowerOPS script (after the installation)

PS C:\Users\Administrator\nsx-powerops> .\nsx-PowerOps.ps1


When you run PowerOPS for the first time you can set up "Connection Profiles" with hostnames and log in credentials for fifferent componenenets. Once these are initially set these can be changed later on in the menu if required.

Initial startup menu
NSX-PowerOps Version: 2.5.
*************************************************************************************************************************

NSX-PowerOps Main Menu

1 - Install NSX-PowerOps Dependancies    [  Installed  ]
2 - Connection Profiles                  [  Default: 10.11.11.122  ]
3 - Configure Scheduled Tasks            [  ENABLED  ]
4 - NSX-PowerOps Documentation Tools     [  ENABLED  ]
5 - NSX-PowerOps HealthChecks            [  ENABLED  ]

*************************************************************************************************************************
Default Connection Profile: 10.11.11.122
Current Connection Profile: 10.11.11.122
Connected: True
Output Directory: C:\Users\Administrator\nsx-powerops\Report\2018-09-18_23-56
*************************************************************************************************************************

Enter the number of your choice, h to get item specific help, x to go back or q to quit.


Select 4 for NSX-PowerOps Documentation Tools
NSX-PowerOps Version: 2.5.
*************************************************************************************************************************

NSX-PowerOps Main Menu > NSX-PowerOps Documentation Tools

Auto Run All
1 - Run All Following Documentation Menu Items                         [  ENABLED  ]

Environment Documentation
2 - Document All NSX Components                                        [  ENABLED  ]
3 - Document ESXi Host(s) Info                                         [  ENABLED  ]

Networking Documentation
4 - Document NSX Environment via Visio Diagramming Tool                [  ENABLED  ]
5 - Document Routing information                                       [  ENABLED  ]
6 - Document Load Balancing Information                                [  ENABLED  ]

Security Documentation
7 - Document NSX DFW info to Excel via DFW2Excel (small environments)  [  ENABLED  ]
8 - Document NSX DFW info to Excel via DFW2Excel (large environments)  [  ENABLED  ]

*************************************************************************************************************************
Default Connection Profile: 10.11.11.122
Current Connection Profile: 10.11.11.122
Connected: True
Output Directory: C:\Users\Administrator\nsx-powerops\Report\2018-09-18_23-56
*************************************************************************************************************************

Enter the number of your choice, h to get item specific help, x to go back or q to quit.


2 - Document All NSX Components

Generated File by PowerOPS Media:2._NSX-Components-2018-09-19_00-06.xlsx


3 - Document ESXi Host(s) Info

Output errors after selecting the option

out-event : A specified parameter was not correct: argument[0]
At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:426 char:17
+                 out-event -entrytype error $ErrorMessage
+                 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,out-event

Exception calling "Add" with "2" argument(s): "Item has already been added. Key in dictionary: 'VXLAN Installed'  Key being added: 'VXLAN Installed'"
At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:427 char:17
+ ...             $sshCommandOutputDataLogicalSwitch.Add("VXLAN Installed", ...
+                 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentException

out-event : A specified parameter was not correct: argument[0]
At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:426 char:17
+                 out-event -entrytype error $ErrorMessage
+                 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,out-event

Exception calling "Add" with "2" argument(s): "Item has already been added. Key in dictionary: 'VXLAN Installed'  Key being added: 'VXLAN Installed'"
At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:427 char:17
+ ...             $sshCommandOutputDataLogicalSwitch.Add("VXLAN Installed", ...
+                 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentException

out-event : A specified parameter was not correct: argument[0]
At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:426 char:17
+                 out-event -entrytype error $ErrorMessage
+                 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,out-event

Exception calling "Add" with "2" argument(s): "Item has already been added. Key in dictionary: 'VXLAN Installed'  Key being added: 'VXLAN Installed'"
At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:427 char:17
+ ...             $sshCommandOutputDataLogicalSwitch.Add("VXLAN Installed", ...
+                 16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentException


The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.

Generated File by PowerOPS Media:3._ESXi-Hosts-2018-09-19_00-07.xlsx

4 - Document NSX Environment via Visio Diagramming Tool

Output after selecting the option

PowerNSX Object Capture Script - Version 2.0.1

Getting NSX Objects
  Getting LogicalSwitches
  Getting DV PortGroups
  Getting VSS PortGroups
  Getting Logical Routers
  Getting Edges
  Getting NSX Controllers
  Getting VMs
  Getting IP and MAC details from Spoofguard
  Getting configured IP Set objects
  Getting configured Services
  Getting configured Service groups
  Getting configured Security groups
  Getting configured Security tags
  Getting configured DFW Rules
  Getting Security Policies

Creating Object Export Bundle

Capture Bundle created at C:\Users\Administrator\nsx-powerops\Report\2018-09-18_23-56\NSX-CaptureBundle-2018-09-19_00-10.zip
PowerNSX Object Diagram Script

Launching Microsoft Visio.

Building Diagram
  Adding DLR-01 to diagram with stencil Logical Router
  Adding LS-TRANSIT to diagram with stencil Logical Switch
  Connecting Logical Router with Logical Switch with text: uplink: 192.168.10.2
  Adding LS-PRODUCTION-01 to diagram with stencil Logical Switch
  Connecting Logical Router with Logical Switch.4 with text: internal: 172.16.10.1
  Adding LS-DHCPTEST to diagram with stencil Logical Switch
  Connecting Logical Router with Logical Switch.6 with text: internal: 172.16.11.1
  Adding LS-MARCEL to diagram with stencil Logical Switch
  Connecting Logical Router with Logical Switch.8 with text: internal: 10.11.30.1
  Adding RoutedNetwork-92c01f5a-e699-4568-8726-e1161fdf0963 to diagram with stencil Logical Switch
  Connecting Logical Router with Logical Switch.10 with text: internal: 172.16.30.129
  Adding RoutedNetwork-5213d324-af25-4fa1-a7c1-a19771fc08af to diagram with stencil Logical Switch
  Connecting Logical Router with Logical Switch.12 with text: internal: 172.16.30.161
  Adding LS-PRODUCTION-02 to diagram with stencil Logical Switch
  Connecting Logical Router with Logical Switch.14 with text: internal: 172.16.12.1
  Adding ESG-01 to diagram with stencil Edge
  Adding VLAN16-vDS-NSX to diagram with stencil PortGroup
  Connecting Edge with PortGroup with text: uplink: 10.11.16.1
  Connecting Edge with Logical Switch with text: internal: 192.168.10.1
  Adding Edge-DemoApp-99f68d14-4bb1-4a81-9594-62f73e6529fb to diagram with stencil Edge
  Connecting Edge.20 with Logical Switch.12 with text: uplink: 172.16.30.162 172.16.30.163

Saved diagram at C:\Users\Administrator\nsx-powerops\Report\2018-09-18_23-56\NSX-CaptureBundle-2018-09-19_00-10.vsdx
Done.  Hit any key to continue.


Generated Files by PowerOPS Media:4._NSX-CaptureBundle-2018-09-19_00-10.vsdx Media:4._NSX-CaptureBundle-2018-09-19_00-10.zip


5 - Document Routing information

Generated File by PowerOPS Media:5._NSX-Routing-2018-09-19_00-12.xlsx

6 - Document Load Balancing Information

Output errors after selecting the option

You typed an invalid name for a sheet or chart. Make sure that:
• The name that you type does not exceed 31 characters.
• The name does not contain any of the following characters:  :  \  /  ?  *  [  or  ]
• You did not leave the name blank.
At C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1:972 char:9
+         $wsStats.Name = $edge.name
+         16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)16:34, 24 September 2018 (BST)~
    + CategoryInfo          : OperationStopped: (:) [], COMException
    + FullyQualifiedErrorId : System.Runtime.InteropServices.COMException


The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.

Generated File by PowerOPS Media:6._LBToExcel-2018-09-19_00-13.xlsx

7 - Document NSX DFW info to Excel via DFW2Excel (small environments)

Output after selecting the option

Collection of IP Addresses Enabled
Collection of Security Tag VM Membership Enabled
Collection of Security Group VM Membership Enabled

Retrieving IP Addresses for ALL Virtual Machines in vCenter environment.
*** This may take a while ***.

Retrieving Services configured in NSX-v.

Retrieving Service Groups configured in NSX-v.

Retrieving MACSETS configured in NSX-v.

Retrieving IPSETS configured in NSX-v.

Retrieving Security Groups configured in NSX-v.

Retrieving Security Groups Effective Membership in NSX-v.
Collection of VM Sec Membership Enabled

Retrieving Security Tags configured in NSX-v.

Retrieving VMs in DFW Exclusion List

Retrieving DFW Redirect FW Rules

Retrieving DFW Layer 2 FW Rules

Retrieving DFW Layer 3 FW Rules

Retrieving Environment Summary


Generated File by PowerOPS Media:7._DfwToExcel-2018-09-19_00-15.xlsx


8 - Document NSX DFW info to Excel via DFW2Excel (large environments)

This output would be the same as option 7.

Select x to go back and select 5 for the health checks

Select 5 for NSX-PowerOps HealthChecks
NSX-PowerOps Version: 2.5.
*************************************************************************************************************************

NSX-PowerOps Main Menu > NSX-PowerOps HealthChecks

Non-Interactive Healthchecks Menu
1 - NSX Connectivity Test                         [  ENABLED  ]
2 - NSX Manager Test                              [  ENABLED  ]
3 - NSX Controllers Appliance Test                [  ENABLED  ]
4 - NSX Logical Switch Test                       [  ENABLED  ]
5 - NSX DFW Memory Test (requires SSH to ESXi)    [  ENABLED  ]
6 - Check VIB Version                             [  ENABLED  ]

Interactive Healthchecks Menu
7 - Check DLR Instance                            [  ENABLED  ]
8 - Check vTEP to vTEP connectivity               [  ENABLED  ]

Beta Healthchecks Menu
9 - Compare DLR and Hosts Routing Tables [BETA]   [  ENABLED  ]

*************************************************************************************************************************
Default Connection Profile: 10.11.11.122
Current Connection Profile: 10.11.11.122
Connected: True
Output Directory: C:\Users\Administrator\nsx-powerops\Report\2018-09-18_23-56
*************************************************************************************************************************

Enter the number of your choice, h to get item specific help, x to go back or q to quit.


1 - NSX Connectivity Test

Output after selecting the option

Using ESXi credentials from connection profile 10.11.11.122
Using NSX Controller credentials from connection profile 10.11.11.122
Using NSX Manager credentials from connection profile 10.11.11.122

Describing Basic System Connectivity and Time Tests
NSX Manager: 10.11.11.122
 [+] is connected to vCenter 585ms
 [-] is within 5 minutes of the local system time (Note:  Local system and NSX Manager must be in same TimeZone) 165ms
   Expected {419.986275675} to be less than {5}
   49:         $totalAbsMinsDrift


The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.

Generated File by PowerOPS Media:TestNSXConnections-2018-09-19_00-24.xml


2 - NSX Manager Test

Output after selecting the option

Using ESXi credentials from connection profile 10.11.11.122
Using NSX Controller credentials from connection profile 10.11.11.122
Using NSX Manager credentials from connection profile 10.11.11.122

Describing NSX Manager
 [+] performed an Inventory less than 30 minutes ago 648ms
 [+] has the NSX Manager Service in a running state 5.59s
 [+] has the vPostgres Service in a running state 81ms
 [+] has the RabbitMQ Service in a running state 22ms
 [-] has the NSX Replicator Service in a stopped state 150ms
   Expected: {} to match the expression {STOPPED}
   75:                 $_.name -match 'NSX Replicator'}).status


The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.

Generated File by PowerOPS Media:TestNSXManager-2018-09-19_00-26.xml


3 - NSX Controllers Appliance Test

Output after selecting the option

Using ESXi credentials from connection profile 10.11.11.122
Using NSX Controller credentials from connection profile 10.11.11.122
Using NSX Manager credentials from connection profile 10.11.11.122

Describing NSX Controllers
The NSX controller cluster
 [-] has the supported number of controller nodes 4.14s
   Expected: {3}
   But was:  {1}
   54:         It "has the supported number of controller nodes" {$NsxControllers.Count


The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.

Generated File by PowerOPS Media:TestNSXControllers-2018-09-19_00-33.xml


4 - NSX Logical Switch Test

Output after selecting the option

Using ESXi credentials from connection profile 10.11.11.122
Using NSX Controller credentials from connection profile 10.11.11.122
Using NSX Manager credentials from connection profile 10.11.11.122

Describing Logical Switches
the Logical Switch LS-TRANSIT
Logical Switch control plane tests
 [-] is found on all controllers 2.56s
   String lengths are both 26. Strings differ at index 13.
   Expected: {VNI found on 1 controllers}
   But was:  {VNI found on 0 controllers}
   ------------------------^
   115:                 "VNI found on $($CurrLsVniResults.count) controllers"


The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.

Generated File by PowerOPS Media:TestNSXLogicalSwitch-2018-09-19_00-28.xml


5 - NSX DFW Memory Test (requires SSH to ESXi)

Output after selecting the option

Using ESXi credentials from connection profile 10.11.11.122
Using NSX Controller credentials from connection profile 10.11.11.122
Using NSX Manager credentials from connection profile 10.11.11.122

Describing Distributed Firewall Memory heaps
 [+] esx-02.home.local is reachable via ssh 1.55s
 [+] esx-02.home.local returns system heaps 158ms
 [+] esx-02.home.local has not exceeded the 80% memory threshold on memory heap vsip-attr-0x4322ab5a3000/ for esx-02.home.local 86ms
 [+] esx-02.home.local has not exceeded the 80% memory threshold on memory heap vsip-flow-0x4310336f6000/ for esx-02.home.local 115ms
 [+] esx-02.home.local has not exceeded the 80% memory threshold on memory heap vsip-ipdiscovery-0x4310136f3000/ for esx-02.home.local 76ms
 [+] esx-02.home.local has not exceeded the 80% memory threshold on memory heap vsip-rules-0x4323f77a0000/ for esx-02.home.local 70ms
 [+] esx-02.home.local has not exceeded the 80% memory threshold on memory heap vsip-state-0x430f736f0000/ for esx-02.home.local 86ms
 [+] esx-02.home.local has not exceeded the 80% memory threshold on memory heap vsip-module-0x430ef36ed000/ for esx-02.home.local 71ms
 [+] esx-01.home.local is reachable via ssh 199ms
 [+] esx-01.home.local returns system heaps 139ms
 [+] esx-01.home.local has not exceeded the 80% memory threshold on memory heap vsip-attr-0x43251f631000/ for esx-01.home.local 73ms
 [+] esx-01.home.local has not exceeded the 80% memory threshold on memory heap vsip-flow-0x4310ec78d000/ for esx-01.home.local 85ms
 [+] esx-01.home.local has not exceeded the 80% memory threshold on memory heap vsip-ipdiscovery-0x430f1fae6000/ for esx-01.home.local 73ms
 [+] esx-01.home.local has not exceeded the 80% memory threshold on memory heap vsip-rules-0x43245f82e000/ for esx-01.home.local 84ms
 [+] esx-01.home.local has not exceeded the 80% memory threshold on memory heap vsip-state-0x43104c78a000/ for esx-01.home.local 69ms
 [+] esx-01.home.local has not exceeded the 80% memory threshold on memory heap vsip-module-0x430fcc787000/ for esx-01.home.local 78ms
 [+] esx-04.home.local is reachable via ssh 226ms
 [+] esx-04.home.local returns system heaps 137ms
 [+] esx-04.home.local has not exceeded the 80% memory threshold on memory heap vsip-attr-0x430f0df72000/ for esx-04.home.local 88ms
 [+] esx-04.home.local has not exceeded the 80% memory threshold on memory heap vsip-flow-0x430c1a068000/ for esx-04.home.local 77ms
 [+] esx-04.home.local has not exceeded the 80% memory threshold on memory heap vsip-ipdiscovery-0x430979eeb000/ for esx-04.home.local 78ms
 [+] esx-04.home.local has not exceeded the 80% memory threshold on memory heap vsip-rules-0x430eae06f000/ for esx-04.home.local 69ms
 [+] esx-04.home.local has not exceeded the 80% memory threshold on memory heap vsip-state-0x430e6e06c000/ for esx-04.home.local 87ms
 [+] esx-04.home.local has not exceeded the 80% memory threshold on memory heap vsip-module-0x430dee069000/ for esx-04.home.local 70ms
Tests completed in 3.85s
Passed: 24 Failed: 0 Skipped: 0 Pending: 0 Inconclusive: 0


Generated File by PowerOPS Media:TestNSXDistributedFirewallHeap-2018-09-19_00-36.xml


6 - Check VIB Version

Output after selecting the option


Using ESXi credentials from connection profile 10.11.11.122
Using NSX Controller credentials from connection profile 10.11.11.122
Using NSX Manager credentials from connection profile 10.11.11.122

 [-] Error occurred in test script 'C:\Users\Administrator\nsx-powerops/HealthCheck/testNSXVIBVersion.Tests.ps1' 640ms
   NotSupportedException: The response content cannot be parsed because the Internet Explorer engine is not available, or Internet Explorer's first-launch configuration is not complete. Specify the UseBasicParsing parameter and try again.
   at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\HealthCheck\testNSXVIBVersion.Tests.ps1: line 72
   at <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Pester.psm1: line 297
   at Invoke-Pester, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Pester.psm1: line 310
   at runNSXTest, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1322
   at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1715
   at Show-MenuV2, C:\Users\Administrator\nsx-powerops\util.ps1: line 328
   at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1667
   at Show-MenuV2, C:\Users\Administrator\nsx-powerops\util.ps1: line 328
   at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1915
   at <ScriptBlock>, <No file>: line 1
Tests completed in 640ms
Passed: 0 Failed: 1 Skipped: 0 Pending: 0 Inconclusive: 0


The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.

Generated File by PowerOPS Media:TestNSXVIBVersion-2018-09-19_00-37.xml

7 - Check DLR Instance

Output after selecting the option

Using ESXi credentials from connection profile 10.11.11.122
Using NSX Controller credentials from connection profile 10.11.11.122
Using NSX Manager credentials from connection profile 10.11.11.122

Please provide the Compute Cluster name [example: Compute Cluster A1]: Compute
Please provide Cluster's VDS name [example: ComputeA_VDS]: vDS-Compute
Please provide VDN ID [example: 10000]: 5001

Describing NSX VDR Port Tests
Cluster Compute
WARNING:
   The Get-EsxCli cmdlet provides a new interface to the ESXCLI functionality.
   Use the -V2 parameter to switch to the new cmdlet interface.
   Check the cmdlet help for more information.

   Scripts that use the old cmdlet interface may not be compatible across two different versions of the ESX server.
   The old cmdlet interface is deprecated and will be removed in a future version.
Checking Host: esx-02.home.local
Checking Host: esx-01.home.local
Checking Host: esx-04.home.local
 [+] got VDR deployed 322.1s
Tests completed in 322.1s
Passed: 1 Failed: 0 Skipped: 0 Pending: 0 Inconclusive: 0


Generated File by PowerOPS Media:TestNSXVDR-2018-09-19_00-38.xml


8 - Check vTEP to vTEP connectivity

Output after selecting the option

Using ESXi credentials from connection profile 10.11.11.122
Using NSX Controller credentials from connection profile 10.11.11.122
Using NSX Manager credentials from connection profile 10.11.11.122

Describing NSX Manager

>> Please provide the MTU size to test [Default: 1572]:

>> Run this test from 'one' host or 'all' [Default: all]:

 ******************************
 Pinging From Host: esx-01.home.local
 Pinging From VMKnic: vmk3
 Pinging From VMKnic IP: 10.11.15.7
 Ping MTU Size is: 1572
 ******************************

 Pinging To Host: esx-01.home.local
 Pinging To its VMKnic: vmk3
 Pinging To its IP: 10.11.15.7
 [+] Ping test result 17.25s
  [+] round-trip min/avg/max=0.041/0.044/0.049ms

 Pinging To Host: esx-02.home.local
 Pinging To its VMKnic: vmk3
 Pinging To its IP: 10.11.15.6
 [+] Ping test result 2.9s
  [+] round-trip min/avg/max=0.25/0.274/0.299ms

 Pinging To Host: esx-04.home.local
 Pinging To its VMKnic: vmk3
 Pinging To its IP: 10.11.15.8
 [+] Ping test result 2.7s
  [+] round-trip min/avg/max=0.367/0.426/0.49ms

 ******************************
 Pinging From Host: esx-02.home.local
 Pinging From VMKnic: vmk3
 Pinging From VMKnic IP: 10.11.15.6
 Ping MTU Size is: 1572
 ******************************

 Pinging To Host: esx-01.home.local
 Pinging To its VMKnic: vmk3
 Pinging To its IP: 10.11.15.7
 [+] Ping test result 2.79s
  [+] round-trip min/avg/max=0.189/0.234/0.268ms

 Pinging To Host: esx-02.home.local
 Pinging To its VMKnic: vmk3
 Pinging To its IP: 10.11.15.6
 [+] Ping test result 2.65s
  [+] round-trip min/avg/max=0.045/0.062/0.072ms

 Pinging To Host: esx-04.home.local
 Pinging To its VMKnic: vmk3
 Pinging To its IP: 10.11.15.8
 [+] Ping test result 2.73s
  [+] round-trip min/avg/max=0.414/0.436/0.451ms

 ******************************
 Pinging From Host: esx-04.home.local
 Pinging From VMKnic: vmk3
 Pinging From VMKnic IP: 10.11.15.8
 Ping MTU Size is: 1572
 ******************************

 Pinging To Host: esx-01.home.local
 Pinging To its VMKnic: vmk3
 Pinging To its IP: 10.11.15.7
 [+] Ping test result 2.81s
  [+] round-trip min/avg/max=0.26/0.304/0.37ms

 Pinging To Host: esx-02.home.local
 Pinging To its VMKnic: vmk3
 Pinging To its IP: 10.11.15.6
 [+] Ping test result 3.74s
  [+] round-trip min/avg/max=0.414/0.448/0.494ms

 Pinging To Host: esx-04.home.local
 Pinging To its VMKnic: vmk3
 Pinging To its IP: 10.11.15.8
 [+] Ping test result 2.93s
  [+] round-trip min/avg/max=0.109/0.115/0.12ms
Tests completed in 40.49s
Passed: 9 Failed: 0 Skipped: 0 Pending: 0 Inconclusive: 0


Generated Files by PowerOPS Media:VMKnicPingTestOutput-2018-09-19_00-46.xlsx Media:TestNSXMTUUnderlay-2018-09-19_00-45.xml

9 - Compare DLR and Hosts Routing Tables [BETA]

Output after selecting the option

Using ESXi credentials from connection profile 10.11.11.122
Using NSX Controller credentials from connection profile 10.11.11.122
Using NSX Manager credentials from connection profile 10.11.11.122

Using existing PowerCLI connection to 10.11.11.52
Found single logical router DLR-01

Collecting routing table from DLR-01
Collecting routing table from host esx-02.home.local
Collecting routing table from host esx-01.home.local
Collecting routing table from host esx-04.home.local

Validating routing table of ESXi server esx-02.home.local
 [-] Error occurred in test script 'C:\Users\Administrator\nsx-powerops/HealthCheck/testNSXDlrHostRoutingTable.Tests.ps1' 4.38s
   RuntimeException: The It command may only be used inside a Describe block.
   at Assert-DescribeInProgress, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Functions\Describe.ps1: line 125
   at ItImpl, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Functions\It.ps1: line 140
   at It, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Functions\It.ps1: line 117
   at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\HealthCheck\testNSXDlrHostRoutingTable.Tests.ps1: line 287
   at <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Pester.psm1: line 297
   at Invoke-Pester, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.0\Pester.psm1: line 310
   at runNSXTest, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1322
   at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1736
   at Show-MenuV2, C:\Users\Administrator\nsx-powerops\util.ps1: line 328
   at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1667
   at Show-MenuV2, C:\Users\Administrator\nsx-powerops\util.ps1: line 328
   at <ScriptBlock>, C:\Users\Administrator\nsx-powerops\nsx-PowerOps.ps1: line 1915
   at <ScriptBlock>, <No file>: line 1
Tests completed in 4.38s
Passed: 0 Failed: 1 Skipped: 0 Pending: 0 Inconclusive: 0


The errors displayed above are probably due to a compatibility problem of the combination of the version PowerNSX and the version NSX that I am using. I did not try to troubleshoot these as the goal of this exercise only was to demonstrate how the PowerOPS script worked.

Generated File by PowerOPS Media:TestNSXDlrHostRoutingTable-2018-09-19_00-47.xml