Scripting

Men and Mice API Client Gem for Ruby

Part of what I am doing these days is integrating Red Hat CloudForms / ManageIQ with the Men and Mice IP address management (IPAM) application. In order to reduce gem dependencies, reduce duplication, and handle some architecture issues I have been working on a client gem that uses the JSON-RPC API provided by Men and Mice.

If you are using Ruby and want to integrate with M&M please check out the gem and provide any feedback you have through Github issues.

The gem: https://rubygems.org/gems/mm_json_client
The source: https://github.com/ewannema/mm_json_client

Foreman ESXi Installation Never Completes

In our environment we want to use Foreman to build our OCP hardware. It was working well except that the installation would repeatedly PXE boot to reinstall because Foreman would not know that the PXE portion was complete.

The standard notification code below was failing for some reason.

%post --interpreter=busybox --ignorefailure=true 
wget -O /dev/null <%= foreman_url %>

After running the unattended install in a Fusion VM so I could have console access I reviewed the install log and a misleading message from wget about the URL being bad. After a little digging I determine that the problem was no DNS in the ESXi installer.

Per the documentation this is expected.
Deploying ESXi 5.x using the Scripted Install feature (2004582)

Note: When ESXi 5.x is being installed using a kickstart file via PXE with DHCP enabled; the DNS settings are not available and are not saved in the configuration file.

Following what I have seen in previous installation scripts I implemented a quick fix to the Foreman OS template to add DNS settings to the installer environment so the call works and the build proceeds as expected.

%post --interpreter=busybox --ignorefailure=true 
# Add temporary DNS resolution so the foreman call works
echo "nameserver <%= @host.subnet.dns_primary %>" >> /etc/resolv.conf
echo "nameserver <%= @host.subnet.dns_secondary %>" >> /etc/resolv.conf
wget -O /dev/null <%= foreman_url %>
echo "Done with Foreman call"

PowerShell: Alternate script behavior when dot sourced vs run directly

I use dot sourcing in my PowerShell scripts to reuse functions* and sometimes I want different behavior if they are included (dot sourced) or run directly. This concept is used in Python to provide an easy way to run tests against the code when run directly but not when it is imported.

You are not restricted to running tests using this approach, but I think that automated testing is useful so that is the example I provide below.
(more…)

VMware PowerCLI 5 Cmdlets by Noun

Generated using this post.

Noun Verbs
AdvancedSetting Get,New,Remove,Set
AlarmAction Get,New,Remove
AlarmActionTrigger Get,New,Remove
AlarmDefinition Get,Set
Annotation Get,Set
CDDrive Get,New,Remove,Set
Cluster Get,Move,New,Remove,Set
CustomAttribute Get,New,Remove,Set
CustomField New,Remove,Set
Datacenter Get,Move,New,Remove,Set
Datastore Get,New,Remove,Set
DatastoreCluster Get
DatastoreItem Copy
DrsRecommendation Apply,Get
DrsRule Get,New,Remove,Set
ErrorReport Get
EsxCli Get
EsxTop Get
FloppyDrive Get,New,Remove,Set
Folder Get,Move,New,Remove,Set
HAPrimaryVMHost Get
HardDisk Copy,Get,New,Remove,Set
Inventory Get,Move,Remove
IScsiHbaTarget Get,New,Remove,Set
Log Get
LogType Get
NetworkAdapter Get,New,Remove,Set
NicTeamingPolicy Get,Set
OSCustomizationNicMapping Get,New,Remove,Set
OSCustomizationSpec Get,New,Remove,Set
PassthroughDevice Add,Get,Remove
PowerCLIConfiguration Get,Set
PowerCLIVersion Get
ResourcePool Get,Move,New,Remove,Set
ScsiController Get,New,Set
ScsiLun Get,Set
ScsiLunPath Get,Set
Snapshot Get,New,Remove,Set
Stat Get
StatInterval Get,New,Remove,Set
StatType Get
Task Get,Stop,Wait
Template Get,Move,New,Remove,Set
Tools Dismount,Mount,Update,Wait
UsbDevice Get,Remove
VApp Export,Get,Import,Move,New,Remove,Set,Start,Stop
VIAccount Get
VICredentialStoreItem Get,New,Remove
VIEvent Get
View Get
VIObjectByVIView Get
VIPermission Get,New,Remove,Set
VIPrivilege Get
VIProperty Get,New,Remove
VIRole Get,New,Remove,Set
VirtualPortGroup Get,New,Remove,Set
VirtualSwitch Get,New,Remove,Set
VIServer Connect,Disconnect
VM Get,Move,New,Remove,Restart,Set,Start,Stop,Suspend
VMGuest Get,Restart,Shutdown,Suspend
VMGuestFile Copy
VMGuestNetworkInterface Get,Set
VMGuestRoute Get,New,Remove
VMHost Add,Get,Move,Remove,Restart,Set,Start,Stop,Suspend
VMHostAccount Get,New,Remove,Set
VMHostAdvancedConfiguration Get,Set
VMHostAuthentication Get,Set
VMHostAvailableTimeZone Get
VMHostDiagnosticPartition Get,Set
VMHostDisk Get
VMHostDiskPartition Format,Get
VMHostFirewallDefaultPolicy Get,Set
VMHostFirewallException Get,Set
VMHostFirmware Get,Set
VMHostHba Get,Set
VMHostModule Get,Set
VMHostNetwork Get,Set
VMHostNetworkAdapter Get,New,Remove,Set
VmHostNtpServer Add,Get,Remove
VMHostPatch Get,Install
VMHostProfile Apply,Export,Get,Import,New,Remove,Set
VMHostProfileCompliance Test
VMHostProfileRequiredInput Get
VMHostRoute Get,New,Remove,Set
VMHostService Get,Restart,Set,Start,Stop
VMHostSnmp Get,Set,Test
VMHostStartPolicy Get,Set
VMHostStorage Get,Set
VMHostSysLogServer Get,Set
VMQuestion Get,Set
VMResourceConfiguration Get,Set
VMScript Invoke
VMStartPolicy Get,Set

Simple XML output of a data object

I am doing some data collection using PowerShell and the output needs to be a simple XML document string.

The built in cmdlets do a great job when persisting data, but they produce some fairly complicated XML that provides more detail on object types than I can use.

Here is a single function to take a PSObject based data object and turn it into XML. I have also included a test function to show an example of building the data object.

 
Function Convert-CustomObjectToXml {
<#
.SYNOPSIS
 
Outputs a human readable simple text XML representation of a simple PS object.
 
.PARAMETER object
 
The input object to inspect and dump.
 
.PARAMETER depth
 
The number of levels deep to dump. Defaults to 1.
 
.PARAMETER rootEl
 
The name of the root element in the document. Defaults to "root"
 
.PARAMETER indentString
 
The string used to indent each level of XML. Defaults to two spaces.
Set to "" to remove indentation.
 
.DESCRIPTION
 
Outputs a human readable simple text XML representation of a simple PS object.
 
A PSObject with member types of NoteProperty will be dumped to XML.  Only
nested PSObjects up to the depth specified will be searched. All other
note properties will be ouput using their strings values.
 
The output consists of node with property names and text nodes containing the
proprty value.
 
#>
	param (
		[PSCustomObject]$object,
		[Int32]$depth = 1,
		[String]$rootEl = "root",
		[String]$indentString = "  ",
		[Int32]$indent = 1,
		[switch]$isRoot = $true
	)
 
	# Output the root element opening tag
	if ($isRoot) {
		"<{0}>" -f $rootEl
	}
 
	# Iterate through all of the note properties in the object.
	foreach ($prop in (Get-Member -InputObject $object -MemberType NoteProperty)) {
		$child = $object.($prop.Name)
 
		# Check if the property is an object and we want to dig into it
		if ($child.GetType().Name -eq "PSCustomObject" -and $depth -gt 1) {
			"{0}<{1}>" -f ($indentString * $indent), $prop.Name
				Convert-CustomObjectToXml $child -isRoot:$false -indent ($indent + 1) -depth ($depth - 1) -indentString $indentString
			"{0}</{1}>" -f ($indentString * $indent), $prop.Name
		}
		else {
			# output the element or elements in the case of an array
			foreach ($element in $child) {
				"{0}<{1}>{2}</{1}>" -f ($indentString * $indent), $prop.Name, $element
			}
		}
	}
 
	# Output the root element closing tag
	if ($isRoot) {
		"</{0}>" -f $rootEl
	}
}
 
Function Test-Convert-CustomObjectToXml {
 
	# Create an example object and then try to dump it to the screen.
	$a = New-Object PSObject
	$a | Add-Member -MemberType NoteProperty -Name "CPU" -Value 4
	$a | Add-Member -MemberType NoteProperty -Name "Memory" -Value 800
	$a | Add-Member -MemberType NoteProperty -Name "NICs" -Value (New-Object PSObject)
	$a.NICs | Add-Member -MemberType NoteProperty -Name "vmnic0" -Value "1000"
	$a.NICs | Add-Member -MemberType NoteProperty -Name "vmnic1" -Value "1000"
	$a.NICs | Add-Member -MemberType NoteProperty -Name "vmnic2" -Value "1000"
	$a.NICs | Add-Member -MemberType NoteProperty -Name "vmnic3" -Value "1000"
	$a | Add-Member -MemberType NoteProperty -Name "VMs" -Value (New-Object PSObject)
	$a.VMs | Add-Member -MemberType NoteProperty -Name "VM" -Value @()
	$a.VMs.VM += "foo"
	$a.VMs.VM += "bar"
	$a.VMs.VM += "bash"
	$a.VMs.VM += "bang"
 
	Convert-CustomObjectToXml $a
 
	Convert-CustomObjectToXml $a -depth 10 -rootEl "config"
}

Some sample output.

<root>
  <CPU>4</CPU>
  <Memory>800</Memory>
  <NICs>@{vmnic0=1000; vmnic1=1000; vmnic2=1000; vmnic3=1000}</NICs>
  <VMs>@{VM=System.Object[]}</VMs>
</root>
<config>
  <CPU>4</CPU>
  <Memory>800</Memory>
  <NICs>
    <vmnic0>1000</vmnic0>
    <vmnic1>1000</vmnic1>
    <vmnic2>1000</vmnic2>
    <vmnic3>1000</vmnic3>
  </NICs>
  <VMs>
    <VM>foo</VM>
    <VM>bar</VM>
    <VM>bash</VM>
    <VM>bang</VM>
  </VMs>
</config>

Testing TCP Network Firewall Ports from ESXi

As we all transition to ESXi there are certain activities that we need to figure out a new way of doing. One item that comes up in new deployments is making sure that all of the ESXi –> firewall rules are defined correctly.

I do not believe that there is a way to do this from the SDK so its off to Tech Support Mode. Unfortunately there is no telnet, netcat, or other network tool that I typically use.

Python to the rescue:

#!/bin/env python
import sys
import os
from socket import *
 
def usage():
    print "USAGE: %s <host> <port>" % (sys.argv[0],)
    sys.exit(1)
 
# Check the # of parameters
if not len(sys.argv) == 3:
    usage()
 
# Check that the host and the port are appropriate.
try:
    host = sys.argv[1]
    port = int(sys.argv[2])
except:
    usage()
 
print "Attempting a TCP connection to host '%s' on port '%s'.\n" % (host, port)
try:
    s = socket(AF_INET, SOCK_STREAM)
    s.connect((host, port))
except Exception, e:
    print "Connection Failed: %s." % (e,)
else:
    print "Connection successful."

Python can also send UDP packets fairly easily, but since there is no connection the only way to tell if the traffic worked is to monitor it on in the network or on the destination side. As a result it is just as easy to do the operation that requires the port and if it fails, set up the traffic monitor.

PowerShell Asynchronous Job Queueing

Here is some sample code to have a variable size work queue for asynchronous jobs in PowerShell. The sample code uses Start-Job, but it was written as a skeleton to be able to run multiple PowerCLI tasks at once.

The variable length is beneficial in case the work should be split up, but there is a chance of overwhelming the system if too many concurrent jobs are started at once.

Write-Host "Script Starting."
 
$concurrentTasks = 3 # How many tasks can run at once.
$waitTime = 500 # Duration in ms the script waits to check for job status changes.
 
# $jobs holds pertinent job information. This is a test script so these are
# just simple values to track script progress.
$jobs = @("a", "b", "c", "d", "e", "f", "g")
$jobIndex = 0 #Which job is up for running
$workItems = @{} # Items being worked on
$workComplete = $false # Is the script done with what it needs to do?
 
while (!$workComplete) {
	# Process any finished jobs.
	foreach ($key in @() + $workItems.Keys) {
		# Write-Host "Checking job $key."
		if ($workItems[$key].State -eq "Completed") {
			"$key is done."
			$result = Receive-Job $workItems[$key]
			$workItems.Remove($key)
			"Result: $result"
		}
	}
 
	# Start new jobs if there are open slots.
	while (($workItems.Count -lt $concurrentTasks) -and 
		   ($jobIndex -lt $jobs.Length)) {
			# These jobs don't do anything other than wait a variable amount of time
			# and print an output message.
			$workTime = Get-Random -Minimum 2000 -Maximum 10000
			$job = $jobs[$jobIndex]
			"Starting job {0}." -f $job
			$workItems[$job] = Start-Job -ArgumentList $workTime, $job -ScriptBlock {Start-Sleep -Milliseconds $args[0]; "{0} processed." -f $args[1]}
			$jobIndex += 1
	}
 
	# If all jobs have been processed we are done.
	if ($jobIndex -eq $jobs.Length -and $workItems.Count -eq 0) {
		$workComplete = $true
	}
 
	# Wait between status checks
	Start-Sleep -Milliseconds $waitTime
}
 
Write-Host "Script Finished."

Basic PowerShell Registry Value Helpers

Working with registry values in PowerShell can be a little cumbersome, so here are some example helpers that I am using to simplify some of my configuration scripts.

Function Get-RegistryValue {
    param (
        $key,
        $value
    )
 
    (Get-ItemProperty -Path $key -Name $value).$value
}
 
Function Test-RegistryValue {
    param (
        $key,
        $value
    )
 
    $data = Get-ItemProperty -Path $key -Name $value -ErrorAction SilentlyContinue
 
    if ($data) {
        $true
    }
    else {
        $false
    }
}
 
Function Set-RegistryValue {
    param (
        $key,
        $value,
        $data
    )
 
    Set-ItemProperty -Path $key -Name $value -Value $data
}
 
Function New-RegistryValue {
    param (
        $key,
        $value,
        $data = "",
        $propertyType = "String" #Default to string registy value types
    )
 
    New-ItemProperty -Path $key -Name $value -Value $data -PropertyType $propertyType | Out-Null
}
 
Function Remove-RegistryValue {
    param (
        $key,
        $value
    )
 
    Remove-ItemProperty -Path $key -Name $value | Out-Null
}
 
Function Rename-RegistryValue {
    param (
        $key,
        $value,
        $newValue
    )
 
    Rename-ItemProperty -Path $key -Name $value -NewName $newValue
}