ForwardOrReverse

As a basis for an upcoming post on splitting vmkernel traffic across over layer 3 boundaries I wanted to describe how vmkernel traffic is routed on an ESX host. There seems to be a lot of confusion in this area and hopefully this will help to clear it up.

If you need a refresher on IP addresses, network masks, or subnets check out this Cisco article.

Directly Connected Networks

If a host is directly connected to a subnet it will use that interface to talk to devices in that subnet. For example if I have an interface with the IP 10.1.1.1 NETMASK 255.255.255.0, that interface will be used to talk to anything on the 10.1.1.0 network. This applies to every directly connected interface.

If I have three vmkernel port groups defined with the following IP information
vmk0: 10.1.0.1    255.255.255.0
vmk1: 10.1.1.1    255.255.255.0
vmk2: 10.1.2.1    255.255.255.0

Then vmk0 will be used to talk to everything on 10.1.0.0, vmk1 for 10.1.1.0, and vmk2 for 10.1.2.0.

Remote Networks

So, what happens when the device I am talking to is on a subnet that I am not directly connected to? This is where the routing table really comes into play so let’s take a look at it using:

vicfg-route –list

VMkernel Routes:
Network             Netmask             Gateway
10.1.0.0            255.255.255.0       Local Subnet
10.1.1.0            255.255.255.0       Local Subnet
10.1.2.0            255.255.255.0       Local Subnet
default             0.0.0.0             10.1.0.254

We see the directly connected networks with a Gateway of Local Subnet. This describes the direct communication that we discussed in Directly Connected Networks.

The last line is a result of our configuration of the “VMkernel Default Gateway” when setting up the vmkernel port group. What it says is send everything else to the router at 10.1.0.254.

The router is in the 10.1.0.0 network and since vmk0 is directly connected to that subnet we know that it will be used for all non local traffic.

A point of clarification

I have seen some confusing statements out there to the effect of “The vmkernel port group with the default gateway assigned will be used to send traffic.” As we have seen, this is not quite true.

All vmkernel ports use the same default gateway so there is no specific assignment per port group. The vmkernel port group that is directly connected to the specified gateway will be used. Unless specific routes are added that means the vmknic in the same subnet as the default gateway will be used for all routed vmkernel traffic.

The routing table can be customized using the vicfg-route command, but should be done rarely. I will discuss one reason you want to do that in my post on splitting vmkernel traffic when crossing layer 3 boundaries.

Side Note: Service Console vs. VMkernel
On the non ESXi versions of vSphere the service console and vmkernel each have their own TCP/IP stacks and therefore have their own IP configuration including routing tables. This means that any IP configuration of one has no effect on the other. The service console’s routing table can be viewed with the command “route” or “route -n”.

Here are a couple of functions to help you export and import OS customization specs in vCenter. Be aware that if you do this between vCenters or the certificate changes then you will have to set any encrypted passwords to their correct value after the import.

These functions and the standard *-OSCustomizationSpec cmdlets should allow you to replicate your customization specs between multiple vCenters fairly easily.

Function Export-OSCustomizationSpec {
    param (
        [string]$specName,
        [string]$exportFile = "$specname.xml"
    )
    $csmgr = Get-View CustomizationSpecManager
 
    if ($csmgr.DoesCustomizationSpecExist($specName)) {
        $spec = $csmgr.GetCustomizationSpec($specName)
        $csmgr.CustomizationSpecItemToXml($spec) | Out-File $exportFile
    }
    else {
        throw "Spec $specName not found"
    }
}
 
Function Import-OSCustomizationSpec {
    param (
        [string]$importFile,
        [string]$specName #Use to change the spec name from that defined in the file
    )
 
    $specXml = Get-Content $importFile
    $csmgr = Get-View CustomizationSpecManager
    $spec = $csmgr.XmlToCustomizationSpecItem($specXml)
 
    # Change the name if a new one was given.
    if ($specName) {
        $spec.Info.Name = $specName
    }
 
    if ($csmgr.DoesCustomizationSpecExist($spec.Info.Name)) {
        throw "Spec $specName already exists."
    }
    else {
        $csmgr.CreateCustomizationSpec($spec)
    }
}

Steve Jin recently posted How to Notify All the Active vSphere Users on his DoubleCloud blog. Below are some functions to make this easy to do via PowerCLI.

Function Get-VIServiceMessage {
    $sessionManager = Get-View SessionManager
    $sessionManager.Message
}
 
Function Set-VIServiceMessage {
    param (
        [String]$message = $(throw "Must specify a message.")
    )
 
    $sessionManager = Get-View SessionManager
    $sessionManager.UpdateServiceMessage($message)
}
 
Function Remove-VIServiceMessage {
    Set-ServiceMessage("")
}

Here is some simple code to generate a reference list of the PowerCLI cmdlet objects(nouns) and what actions(verbs) you can perform on them.

This can come in handy for people new to PowerCLI to show them what the cmdlets will handle for them.

$commandTable = @{}
$commandList = Get-Command -Module VMware.VimAutomation.Core -CommandType Cmdlet
 
foreach ($command in $commandList) {
 
    if (!$commandTable.ContainsKey($command.Noun)) {
        $commandTable[$command.Noun] = @()
    }
 
    $commandTable[$command.Noun] += $command.Verb
}
 
foreach ($command in $commandTable.Keys | Sort-Object) {
    $verbs = $commandTable[$command] | Sort-Object
    $command | Select-Object @{Name="Noun"; Expression={$command}}, 
                                 @{Name="Verbs"; Expression={[String]::Join(",", $verbs)}}
}

Here is the output for PowerCLI version 4 update 1.
Read more…