Enhanced UNMAP script using with PowerCLI and RESTful API

The most common request I get for scripts here at Pure Storage is an UNMAP script using PowerCLI. I have a basic one here that does the trick–UNMAPs Pure Storage volumes in a vCenter. That being said it is pretty dumb–doesn’t tell you much about what happened other than what volumes it is reclaiming (or not reclaiming) and moves on. A few requests have come in recently for something a little more in-depth. Most notably the ability to see how much space has been reclaimed. This information cannot be gathered from the VMware side of things–it has to come from the FlashArray.

There are two options here–either use our REST APIs or use our PowerShell toolkit to get this information (which just wraps the REST calls). For this script I chose to use the REST API directly from within PowerShell. What this script does is:

  1. Connects to the vCenter and FlashArray
  2. Finds all of the datastores and counts how many are actually Pure Storage volumes (NAA comparison)
  3. Iterates through all of the datastores
  4. Skips it if it is not Pure
  5. If it is, the current data reduction ratio is reported and so the is current physical written capacity on the FlashArray.
  6. Runs UNMAP on the datastore
  7. Reports the new data reduction and physical space after UNMAP completes and how much was reclaimed.
  8. Repeats for the rest of the volumes.

The script reports all of this to the console window, but it always throws it in a log file through add-content. If you don’t want it to return the info to the console, simply delete the write-host lines. If you don’t want it to log, delete the add-content lines.

There are a few required parameters–vCenter information (IP, username, password), FlashArray info (IP, username, password), UNMAP block count and a log file location. These are hard-coded parameters, but that can easily be changed by altering it to a read-host.

You may also note that after each UNMAP the script sleeps for 60 seconds–I do this so I make sure the FlashArray has time to update its information right after the UNMAP. 60 seconds is VERY conservative–probably 10 or so is fine, so feel free to mess with that number if you don’t like waiting. I also have another sleep at the end of each datastore operation to give a quick chance to review the latest results before it starts spewing the next datastore information on the screen (note this update didn’t make it into the video demo below–it doesn’t wait after each datastore).

See the script in action below. Essentially I am deleting a bunch of VMs across 4 datastores and then running the UNMAP. You can see the space get reclaimed on the FlashArray.

The script is in Word doc form here: unmapscript

Reclaiming in-guest capacity with VMware and Pure Storage

Reclaiming “dirty” or “dead” space is a topic that goes by my desk quite often these days–since the FlashArray is a data reduction array it is especially important that space is not wasted on the array–throws off the economics etc. Therefore UNMAP is an important VAAI feature to leverage in any AFA environment. Supporting UNMAP is definitely table stakes for AFAs.

Note–I am doing to use the terms “dead”, “dirty” and “stranded” to define space that needs to be reclaimed interchangeably. So anyways…

Unfortunately UNMAP in its current form does not satisfy all of the reclamation use cases. UNMAP will only reclaim space on any array when capacity is cleared from the VMFS volume–so when a VM (or virtual disk) is deleted or migrated elsewhere. It does not have the ability to reclaim space when data is “deleted” inside a virtual machine by the guest OS when using virtual disks. VMware does not know this capacity has been cleared and neither can the array. So until this virtual disk is deleted or moved the capacity cannot be reclaimed with UNMAP. So to be clear, UNMAP with vmkfstools (in ESXi 5.0/5.1) or esxcli (in ESXi 5.5) does not allow you to reclaim space that remains stranded inside of virtual disks.

Continue reading

PowerActions–The PowerCLI Plugin for the vSphere Web Client with UNMAP

The other day I stumbled upon a new VMware labs “Fling” called PowerActions. Basically it allows you to run in-context PowerShell/PowerCLI scripts right from within the vSphere Web Client. My mouth drooled at the promise of what this could deliver–and it really delivered! This is my new favorite tool by a landslide. See the announcement here from @alanrenouf http://www.virtu-al.net/2014/09/16/powercli-vsphere-web-clientannouncing-poweractions/. I’d recommend readin this first before you continue down with my post.

homepage

 

Continue reading

Site Recovery Manager with PowerCLI Automation Gotcha

Quick post here. So I have been reviewing some great posts from @vmKen and @BenMeadowcroft about automating Site Recovery Manager operations with PowerCLI and wanted to give it a try myself. They outlined the process rather clearly in their blogs so it was a breeze to get most of the stuff up and running. But when I went to actually execute a test recovery or a recovery etc. it kept failing! The PowerCLI command to start the recovery  was $VMrp.Start($RPmode)–the $VMrp being my recovery plan and the $RPMode being the recovery plan mode of a recovery. The command was accepted but the recovery plan never started.

I got the following error in vCenter:

Unable to start the requested operation. Another operation may be in progress. Please wait for it to finish and try again.

Hmm…weird. I could kick off a test from the GUI with no issue so nothing was “interfering” from what I could tell. I thought maybe since I was using Site Recovery Manager 5.8 maybe something had changed so I tried it with my 5.5 environment and got the same result.

srm55

webclient

After I was about to lose my mind it finally occurred to me that I was connecting to the protected vCenter and the protected SRM server (I did enter in remote credentials for the recovery SRM server though). While I could query the recovery plan etc without issue from here, maybe SRM didn’t allow a recovery plan to be started unless you directly connected to the recovery vCenter/SRM server.

So I reconnected to the recovery site and it worked! So I guess it makes a difference, so FYI. Now there might be a workaround to this and it is definitely possible I missed something that allows this but this seems to be what you need to do. If you find this isn’t true please let me know!

Thanks Ken and Ben for getting me started!! Cool stuff. Kens posts:

http://blogs.vmware.com/vsphere/2014/05/automate-failover-with-srm.html

http://blogs.vmware.com/vsphere/2014/05/srm-powercli-reporting.html

http://blogs.vmware.com/vsphere/2014/05/powercli-and-the-srm-api.html

Pure Storage Web Client Plugin version 1.1.13

I have done a few posts on here that involve the Pure Storage Plugin for the vSphere Web Client (here and here) since I joined. Well here is another. We just released a new version of the Web Client Plugin (I am going to refer to it as WCP for the rest of this post because I am a lazy typist). We bundle the WCP into Purity and therefore the WCP is installed, updated and uninstalled from our GUI/CLI to vCenter (yes we do also offer a mechanism to update it outside of upgrading Purity itself). Our latest release of Purity, 4.0.12, includes WCP version 1.1.13–while there is no new functionality there are two important fixes.

webclient_intro

Continue reading

The Pure Storage Content Pack 1.0 for VMware vCenter Log Insight

The Pure Storage Content Pack for VMware vCenter Log Insight is now live on the VMware Solution Exchange! Download it today for free. As past posts have shown I have done a decent amount of work with Log Insight here at Pure and in my previous job. A product I have really liked from VMware for a variety of reasons, a big one being that it is so very easy to use. We really improved our syslog feature on the FlashArray in the 4.0 Purity release, so it was the perfect time to create our first content pack!

purestoragecontentpack

Continue reading

VMware vCenter Site Recovery Manager 5.8

As people are probably aware, VMware just released the slew of new product updates that was announced at VMworld. vSphere 5.5 U2 at the core, but essentially all of the major products had updates–the one in particular interest to me was vCenter Site Recovery Manager. This latest release, numbered 5.8 is probably the biggest update to SRM since probably 5.0 moving to 5.1, arguably bigger. I am still playing around with it, but I wanted to share some of the things I found of interest in it.

schema_vmware_site_recovery_manager

Continue reading