Scripting ESI PowerShell cmdlets and VMware

EMC offers a variety of tools to manage/enhance your virtual or physical environments–some free, some licensed. In most cases when you think of EMC tools for VMware one conjures up the free Virtual Storage Integrator which is more commonly referred to as VSI.

VSI is a great tool and continues to be improved through each version and allows you to provision storage, manage pathing, configure SRM etc. The one thing it does not have is a way to automate these tasks through an API or CLI. This is where another product comes in–one that many do not associate with VMware. The EMC Storage Integrator (ESI) is a lot of times seen as the Microsoft version of VSI–but that isn’t really true at all. While it might have started out that way and does indeed support Hyper-V and has a ton of Microsoft-specific features it is really the heterogeneous storage integrator. Importantly it  has a very handy and powerful feature–PowerShell cmdlets.

If you follow Alan Renoufs blog you might remember him posting about it in its infancy back in 2011:

http://www.virtu-al.net/2011/03/21/using-the-emc-cmdlets-with-powercli/

Since then it has been greatly expanded (and commands have changed) and EMC now offers a long list of specialized PowerShell cmdlets that can be used to easily bridge the gap between your compute environment and the storage environment. Something that programmatically can be difficult otherwise. This post is going to focus on ESI Powershell cmdlets and VMAX/VMware interaction. But it is important to note that just because I am only going to go into detail about VMAX and VMware here it does indeed support other hypervisors (Hyper-V, Xenserver etc.) and of course other EMC storage arrays.

First some links:

Download: EMC Storage Integrator 2.1

Download: ESI PowerShell cmdlets Online Help

The cmdlets include EMC storage operations like the simple New-EmcLun (creates a new lun):

New-EmcLun -Pool $pool -Name mylun_test_1 -Capacity 1024mb

Or getting datastore information from an ESXi host and giving you EMC-specific information:

Get-EmcDatastore -VMwareSystem $h

To get these cmdlets on a system all you need to do is install ESI and select the PowerShell option (on by default) when installing. Furthermore–the EMC SMI-S provider (this is what provides the VMAX VASA provider among other things) needs to be installed somewhere with access to the target array (needs some gatekeepers). Solutions Enabler has a build that includes SMI-S that you can install on a host but in my opinion the simplest option is the deploy the Solutions Enabler vApp (or the Unisphere one). They both include the SMI-S provider by default. So if you are already using the vApp you already have it. Once installed you can use these PowerShell cmdlets in Powershell by running:

import-module “C:Program FilesemcEMC Storage IntegratorESIPSToolKitESIPSToolkit”

The main point of this post besides of course advertising the cmdlets is to document something that I haven’t found documented elsewhere (yet). If you want to use these cmdlets you need two connections, one to a VMAX array via a SMI-S provider to perform storage related functions and a second to a host (vCenter, ESXi etc.). Typically easiest to just connect to a vCenter as it will provide the ESXi host connections as well. You can create these connections via the ESI gui, but for unattended scripts this isn’t very desirable–especially if you do not want the connections to remain after a script has been run. In the current documentation I have not found a way to do this, but do not fear it is possible!

The cmdlet get-emchostsystemcredential or get-emcstoragesystemcredential usually just pop-up a GUI/Java interface for a user to supply a credential. Once again, not useful for an unattended script.

You can add the credentials programmatically for a vCenter host like below.

First create and define a variable (I used the name vCCreds but it can be whatever) and then set it equal to array string like below. You will notice I use further variables for the user name, password and server name in this string as well.

$vCcreds = @{"UserFriendlyName"="MyHost";"UseCurrentDomainCredential"="False";"Username"="$viuser";"Password"="$vipassword";"IsCluster"="False";"Server"="$viserver"}

Once you have defined the credentials you can connect the vCenter server by passing it the vCenter credentials. This also stores the object connected (the vCenter) as a ESI cmdlet object in the variable $vCenter (any name you choose) that can be used to reference it later:

$vCenter = Connect-EMCSystem -SystemType "VMware" -CreationParameters $vCcreds

To disconnect you can pass that vCenter object to the disconnect command and add -force so it does not prompt the user:

$vCenter | Disconnect-EmcHostSystem -force

The VMAX connection is very similar in method. You need to supply the credentials to a SMI-S provider that has access to the desired VMAX array and the connection information. Once again I used PowerShell objects for many of the parameters. The most important thing to note is that the VMAX serial number must be the FULL SERIAL NUMBER. With leading zeroes! So do not enter 195701238 or 1238, it must be 00019570123. Running these commands will give you a connection via that SMI-S server to that VMAX. Note that if the SMI-S server has access to other arrays you will not have access to them after creating this connection–each array needs its own connection.

$SMISCreds = @{"UserFriendlyName"="Symmetrix";"SerialNumber"="$symmSN";"Host"="$SMISserver";"Port"="5989";"UseSSL"="True";"UseWindowsAuth"="False";"Username"="$SMISuser";"Password"="$SMISpwd";"IgnoreServerCACheck"="True";"IgnoreServerCNCheck"="True"}

$symmarray = Connect-EMCSystem -SystemType "VMAX" -CreationParameters $SMISCreds

$symmarray | Disconnect-EmcStorageSystem -force

So for an example this is what I did with all of this:

The nice thing about these cmdlets and well PowerShell in general is that it is very easy to learn and use. If you have a basic understanding of coding logic you can do quite a bit. Personally I have almost never used PowerShell before save a few simple scripts and it didn’t take me much time at all to get started. So I wanted to see what I could put together to demonstrate what can be done with these cmdlets.  I set out to write was a script that automatically provisioned storage when a datastore cluster got low on capacity:

1. Initiated by vCenter when a datastore cluster became a certain % full. When vCenter issues a capacity alert on a given datastore cluster the PowerShell script is called.
2. Connect PowerCLI to vCenter, connect ESI cmdlets to vCenter and VMAX array
3. Check Solutions Enabler server configuration
4. EMC Powershell cmdlets would query a current device in the datastore cluster for Symmetrix SN, thin pool, size, meta configuration (if any) and meta member size. Typically you want all datastores in a datastore cluster to be similarly configured, especially with SDRS enabled, so these lookups are important.
5. Create a new device on the array with the same configuration as the other datastores using the specs found in the previous steps
6. Mask the device to the cluster.
7. Format the device with VMFS
8. Move the device into the datastore cluster.
9. Disconnect PowerCLI to vCenter, disconnect ESI cmdlets to vCenter and VMAX array

Now while the ESI cmdlets are very useful they can’t do every thing that each array offers. Too many special features for it to have it all–but each new version of ESI adds to what it can do. As I alluded to earlier in many ways these ESI cmdlets are a great bridge between two worlds. They can take VMware information and relate it to EMC information with ease. Allowing you to connect things like PowerCLI to Solutions Enabler with ESI as the intermediary. Therefore this script will use four pieces; PowerShell (the one ring that rules them all), PowerCLI, ESI cmdlets and Solutions Enabler. PowerCLI will take care of the VMware-specific tasks (creating VMFS, moving into a cluster etc.), the cmdlets will take the VMware information and relate it to EMC information and do some of the storage tasks (identify Symmetrix information of existing datastores, mask the new device to the VMware environment etc.) and Solutions Enabler will finally do the very specific Symmetrix tasks (like identifying meta configuration).

Download it here if you want to check it out. Note that it is saved as a .doc file but can be renamed to .ps1 or .txt as it doesn’t contain any special formatting. WordPress doesn’t like those file types :/

PowerShell Script Download

Probably not perfect, but hopefully a good example on what can be somewhat easily achieved by combining these technologies.

A few notes:

1. The script creates a log file PS-Script-MMDDHHMMSS.log.
2. I set two of the important CLI variables in the script but neglected to include the SYMCLI path variable. I recommend always setting this globally regardless but you would need to add the SYMCLI bin folder to you path variable for this to work out of the box.
3. ESI cmdlets are still a little funky with naming. The current version (in the future this is changing) ESI requires a certain naming convention for initiator groups. This tells it which masking view to use etc. The convention is as such: esi_smi_ig_<name of host> So you need to rename the IG to the hosts to this or use Solutions Enabler to script the masking portion.
4. The script requires the local SYMAPI netcnfg file to be configured prior to running so it can contact a remote SE server if the server is not local to the Powershell host. The script does check for this and will fail if absent (in a remote config)
5. There are a few object values that must be pre-defined and are listed at the top of the script. Some of which are passwords, which are therefore hard-coded but it is probably better to make use of secure strings and store the passwords in a encrypted text file. Here is a helpful article for that by the way:

http://social.technet.microsoft.com/wiki/contents/articles/4546.working-with-passwords-secure-strings-and-credentials-in-windows-powershell.aspx

With great Power(Shell) comes great responsibility.