Check root password with PowerCLI (Multi-threading!)

Keeping your root password similar in all of your ESXi hosts, is one of the virtual environment key methods to control and maintain large environments. It will make it easier to connect directly to a host in case of vCenter failure, access SSH for troubleshooting and control it from DCUI.

There is a great one-liner by Kelvin Wong, that allows you to get a list of all of the VMhosts that have different password than the standard one.

In this post, I’ll try to:

  1. Make it simpler for PowerCLI beginners to use this script
  2. Provide advanced users with methods of multi-threading in PowerShell 2.0

First, the script for getting a list of hosts with non standard password :

The file created by this script, contains a list of hosts with root password different than the one typed as input.

BUT, the script will test connection to each of the hosts one-by-one, which may take a while if you have more than 10 hosts. In an environment of 74 hosts, for example, it took the script 10:30 min to run (calculated with measure-command of course):


We will reduce the run time of the script, by using parallelism of the hosts check. Multi-threading in PowerShell.

The script will start the same way as the one above –

And now, adding the interesting part –


Function CanWeAddJob() is checking what is the running job count in your PowerShell session, and determine whether to add more job, or not. In this example, it does it to a maximum of 10 parallel jobs.

Foreach is here to split the long VMHosts list to many separated tasks. It will only add task to the running job queue, if the queue have less than 10 jobs.
PSSnapin was added to each of the new powershell.exe instances created, with RunAs32 parameter, to make it take less RAM of your server / workstation.

While the script is running, you should see something like this in your task manager:


Can you guess what was the run time of the multi-threaded script?


2:42 min, which saved me 75% of the original run time.

Detach All Unused Devices in ESXi Cluster

Removing storage devices (Datastores + RDMs) can be tricky task for the VI admin. He will have to:

  1. Map the datastores / RDMs in use (maybe with PowerCLI scripts, EMC VSI, or other storage plugins)
  2. Delete / Remove the specific devices from virtual environment side (Delete datastore, Remove RDM from VM)
  3. Detach the devices from ESXi hosts
  4. Let the storage admin know this devices can be removed from storage side

I’m here to help you with automation for the 3rd task. The amount of steps required for detaching 1 device off 10 hosts cluster, is 10 steps (or 10 detach operations). Detaching 20 devices off this cluster will result in 200 steps, which can be very tedious task.

This script is written in PowerCLI, VMware KB shows more ways of detaching disks.

Script initialization (Environment names, Snapin, log location, time function):

Adding some of the detach disk functions:

The power of the script comes here – the ability to map the devices in the cluster, and separate them to few groups:
$ClusterDSDevices – Devices that are used as datastores in the cluster
$ClusterRDMDevices – Devices that are used as RDMs in the cluster
$ClusterVMAXDevices – All Devices that are visible by the cluster, from a specific vendor. In this example, EMC VMAX devices will presented
$AllUseddevices – Both $ClusterDSDevices and $ClusterRDMDevices variables combined

Variables will contain a list of naa devices, and it is your job to determine what should be declared for detach by the end of the script.
In the end of the mapping script I added an example I used – filter for:All VMAX devices (both DS and RDM), that are not in use. It will list the required devices in $DevicesForDetach variable.

It is now time to use the list we created, ans start detaching disks, all tasks will be exported to a log.

Log file will look like that in the end of the process:


In this example, 19 devices were detached from 9 hosts, resulted in 171 detach commands to the ESXi hosts in the cluster. The script took less than an hour to complete it.

Configure TL-WR703N with OpenWRT for Wireless printing

Recently I wanted to configure a wireless printer in a university’s dorms room, which only have wired network. In order to do this change “stealthy”, I needed to use the smallest possible router with USB connection.

TP-Link WR703N is a small router being sold in China, and is capable of OpenWRT firmware. Beware: TL-WR702N is the version being sold globally, but isn’t OpenWRT capable. WR703N specs are:

  • Atheros AR7240 CPU (400Mhz)
  • Atheros AR9331 Chipset (integrated wireless)
  • 802.11 b/g/n 150Mbps
  • 4 MB flash memory, 32 MB RAM
  • USB 2.0 port
  • Powered via micro-USB socket
  • Tiny form factor: 5.7cm x 5.7cm x 1.8cm


Specifications and size makes it a great solution for travels also. This router can be found in eBay at sub-$30 price range, shipped. There are some hardware versions, and you should hope to get a compatible version with OpenWRT, since the default firmware is in Chinese. Hardware version is written at the bottom of the router, and I got v1.6. Connecting to the web interface, I could see more details regarding it – FW build 130321, original FW rel. 37153n. According this model’s OpenWRT page, version is supported for AA, which is the stable version of the firmware.

Starting the process is simple, connect to the router using Ethernet cable, go to in your browser (username: admin, password: admin). Click on last +, and then 3rd bulletin. You should see something similar to this screen:


Download and use the firmware in this page:

Continue configure the router to match your environment needs (Static IP / DHCP / PPPoE, Wifi). There are many good guides, including:

For the printer server part, you’ll need to connect using putty (SSH client) to the router’s address, installing a lightweight USB package –

If printer is already connected, you could monitor the connection status using command dmesg.

This command will direct you to the right device path, which should be either /dev/usb/lp0 or /dev/lp0. In my case it was /dev/usb/lp0. You could see it if you’ll cd to the /dev folder.

Once file is saved, you can start the service and enable service to start at boot time:

We’ll have to open printer ports in the firewall as well.

and restart firewall service:

Enjoy your new Router / Wireless Printer server!

ESXi 5.x – VM reset stuck at 95%

Few weeks ago we had an issue with freezing VM, and the only way of getting out of it was to perform hard reset to the VM. The procedure is well known – right click on the VM > Power > Reset.


The command took quite a while to actually initiate the reset. I really wondered what happened there, so I started investigating it.

vmkernel log didn’t show anything. The VM’s vmware.log log on the other hand, was quite interesting:

2014-09-25T03:44:35.209Z| vmx| VMMon_VSCSIStopVports: Invalid handle
2014-09-25T03:44:35.209Z| vmx| VMMon_VSCSIDestroyDev: Not found
2014-09-25T04:13:15.305Z| vmx| ide1:0: Command TEST UNIT READY took 2264.916 seconds (ok)
2014-09-25T04:13:15.305Z| vmx| SOCKET 9 (98) disconnecting VNC backend by request of remote manager
2014-09-25T04:13:15.307Z| vmx| MKS local poweroff

It looked like TEST UNIT READY took 37min to complete! Only after this command finished, the VM reset preparation continued. So what is this mysterious command, and what took it so long to complete?
TEST UNIT READY is a SCSI command sent to the target in order to get a response, and a status of the device. In the log file, ide1:0 is referred, but CD-ROM drive in this VM isn’t mounted according to vSphere client –


Just to make sure, I checked out the vmx file, which showed me something I didn’t see from the vSphere client side –


vmx file indicates of an iso mounted to this VM. Not just an iso, but the VMware tools iso. Checking again from the guest OS side – and the disk isn’t mounted. So what just happened here?

A bug.

This KB has a low rating of 2 stars right now, probably since the solution didn’t include PowerCLI, right? Here it is.

Mapping of the VMs with ghosted CD-ROM:

The actual removal of the CD-ROM from all VMs:

Works best on 64Bit PowerShell console.


iLO inventory in Multi-Domain ESXi environment

On enterprise companies, you might find a vCenter managing few DNS zones (a.serv.corp, b.serv.corp etc.). There might be a situation where there are few vCenters, each manages a different domain. Keeping track on HP iLO* IPs is easier when you register the iLO IP in the DNS, and use “ilo” as a prefix to each server – for example, esx01 iLO address will be ilo-esx01 and so on. It all works until it doesn’t, since host name changes, and maintaining it for more than just a few ESXi hosts can be demanding. Enters PowerShell.

I’ll assume you have some PowerShell / PowerCLI experience, and you know how to add the right snapin, and connect to the relevant vCenter(s).

First, add the Get-VMHostWSManInstance function, with prerequisites described here:


Using System.Net.DNS .Net class isn’t enough for us, since we want to query few domain DNS servers.

You’ll need the DNSShell module installed in order to reverse lookup in each of the domains:

Please note that this module might not work well on 32Bit PowerShell version, so use 64Bit PowerShell console.

From this point, the script itself is straight forward, gathers all the data required including: vCenter Name, VMHost Name, Domain, iLO-IP, iLO-DNS.


Output will look like:



* HP iLO is referenced here, script will work for any other IPMI technology such as Cisco CIMC, Dell DRAC, etc.