Posted on by codyhosterman

Unattended VMFS UNMAP Script

I updated my UNMAP PowerCLI script a month or so ago and improved quite a few things–but I did remove hard-coded variables and replaced it with interactive input. Which is fine for some, but for many it was not.

Note: Move to VMFS-6 in vSphere 6.5 and you don’t have to worry about this UNMAP business anymore šŸ™‚

Essentially, quite a few people want to run it as a scheduled task in Windows, and if it requires input that just isn’t going to work out of the box. So I have created an unattended version of the script. For details read on.

Note: I will continue to update the script (bugs, features, etc.) but will note them on my other blog post about the script here:

Pure Storage FlashArray UNMAP PowerCLI Script for VMware ESXi

I will only update this post if the unattended version changes in a way that makes these instructions wrong.

First off, the script:

https://github.com/codyhosterman/powercli/blob/master/unmapsdkunattended.ps1

No difference in function with this, but a slight difference in how you run it.

The credentials handling is the biggest change. The kicker here is that the script needs credentials to run against vCenter and the various FlashArrays. And storing credentials in the script with plain text is not a good idea. And, as far as I am aware, the script cannot use the credentials the scheduled task is run with.

So there are a variety of options here. I went with the option described in this Stack Overflow post:

http://stackoverflow.com/questions/30492045/access-windows-task-credentials-in-the-powershell-script

I use two credential files. One for vCenter. One for all of your FlashArrays (this assumes one set of credentials are valid for all of your arrays–if not, we can talk about edits). To make it easier, I wrote a short script for you to use to create the proper credential files. You can get that here:

https://github.com/codyhosterman/powercli/blob/master/unattendedUnmapConfigurator.ps1

At a high level this is what you need to do:

  • Run the configurator to create the credential files
  • Run the script (either through task scheduler or manually) with the appropriate parameters.

Creating the Credential Files

The first step is creating the credential files. To do so, run the configurator script linked above.

The first step will be to choose a directory (or create one). This is where you credential files will go and the eventual logs for your UNMAP runs. I will use C:\UNMAP.

Then, a dialog box will appear asking for your FlashArray credentials. Enter those.

Then, a dialog box will appear asking for your vCenter credentials. Enter those.

In the folder, you will see two credential files appear:

faUnmapCreds.xml and vcUnmapCreads.xml. Each contains those credentials in an encrypted fashion. They are tied to the usernameĀ that actually created the files. So they can only be referenced and used by the user that created them. I created them with user account “cody”. In the above screen shot I can use those credentials. In the lower window which is running under “administrator” it cannot reference those credentials as seen in the error:

So now you can run the UNMAP script!

Running the Unattended UNMAP Script

The unattended UNMAP script still needs some input. The location of the credential files (which doubles as the directory to put its log files to), a vCenter IP/FQDN and one or more FlashArray IP/FQDNs.

These are configured through parameters that can be passed upon calling the script, so there is never a need to actually open and edit the script.

param( 
[Parameter(Position=0,mandatory=$true)]
[String[]] $flasharrays,
[Parameter(Position=1,mandatory=$true)] 
[String] $vcenter,
[Parameter(Position=2,mandatory=$true)] 
[String] $logfolder,
[String] $loginsightserver,
[String] $loginsightagentID
)

I have my log folder at C:\UNMAP, my vCenter is 10.21.10.32 and my FlashArrays are 10.21.88.7 and 10.21.88.244. So my command to run the script would look like so:

.\unmapsdkunattended.ps1 -flasharrays 10.21.88.7,10.21.88.244 -vcenter 10.21.10.32 -logfolder C:\folder\folder

Note that the FlashArrays are comma-separated. Theses three things are mandatory. The script will not run without them. Optionally, you can enter in a Log Insight IP/FQDN and an agent ID if you plan on logging to Log Insight the results.

Running the script as a Scheduled Task in Windows

So first create a new task in Windows Task Scheduler. On the general tab, enter in the name and description. Select the following:

  • Choose the user that you used to create the credential files (in my case “cody”). This will allow the script to use the authentication files.
  • Choose run whether the user is logged on or not (I guess you dont have to, but I imagine you want this)

Set you trigger however. Once a week, once a month, whatever you feel is right.

Then on the action, create a new one like so:

Powershell.exe as the program and your starter command like above Ā (with the full path for the script) in the “add arguments” box. Mine is like so:

c:\unmapsdkunattended.ps1 -flasharrays 10.21.88.7,10.21.88.244 -vcenter 10.21.10.32 -logfolder C:\UNMAP

Save and you are good to go!

32 Replies to “Unattended VMFS UNMAP Script”

  1. Hi Cody,

    Thanks for sharing the great script. The UnMAP procedure is just for Flash Array or they also applicable to any deduplicated array ?

    As for the impact on the storage system performance, how does the UnMAP process impacting the production environment ?

    Reply

    1. Albert, you’re welcome! Running UNMAP is a good idea on any array, data-reducing or not, in my opinion. I think it is more important on data reduction arrays for a variety of reasons, but generally the concept of dedupe can cause dead space to be a bit more of a problem on data reduction arrays. So you should have some type of practice of running UNMAP regardless to the storage vendor.

      The performance question is a bit different. In my testing, UNMAP running to a datastore with a large workload will only cause the UNMAP process to throttle and not affect the actual workload. That being said, the array model might be different. For the FlashArray, running UNMAP is a very lightweight process. It is processed extremely quickly. That is probably similar for most arrays, but there are some traditional arrays at least that still have problems, so it is a good idea to always ask the vendor first.

      Reply

  2. Cool, many thanks for the explanation Cody.

    In the post above, I was trying to find the script scheduled task execution time so that I know the best practice in running the UnMAP process.

    I guess Pure Storage FA420 is quite strong enough to handle both the Production workload and the UnMPA process at the same time, hence it doesn’t matter when the time to run it for 24×7 production environment.

    Reply

    1. Sure thing! Yeah it doesn’t really matter. Unless you are totally maxing out the FA, you can run it whenever. I have some customers run it 2x a day because they are constantly deleting and creating VMs (100s a day). Others (and more commonly) run it once a week or once a month.

      Reply

    1. Hmm. It is the same code, and it doesn’t really use the path, it basically looks to see if the PowerCLI initialization script is there and if it is it runs it. Then it checks to see if the module is loaded (which the script loads). So maybe neither is loading in that case it but it auto-loads when you launch PowerShell manually. Is is not installed in the default location? I see my script expects that (which is stupid-I will fix that). There is a post here about it: https://www.vmguru.com/2016/04/powershell-friday-load-powercli/

      I will change it to look for the reg key instead. That might be the issue.

      Reply

      1. Hello,

        This is where its installed:

        C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI

        I think thats the default location.

        Thanks!

        Reply

  4. I am receiving the same error as a task or as the manual run.
    PowerCLI not found. Please verify installation and retry.
    Terminating Script

    I have verified that I have PowerCLI 5.1 R1 and can log into my vCenter server from it fine.

    Thanks!

    Reply

  7. Any thoughts on why my 2012 R2 server would be giving me fits when working with certain drives that seem to be formatted correctly?

    When using the GUI it states ‘Optimization not available’ on all the ones that are set to a blocksize of 64K aka 65536.

    I am able to run this command and it runs successfully, but only through PS : Optimize-Volume -DriveLetter Z -Analyze -Defrag -Verbose

    If I run this command though I get different results: defrag /C /O

    The slab consolidation / trim operation cannot be performed because the volume alignment is invalid. (0x89000029)

    Invoking slab consolidation on ITT (Z:)…

    Reply

    1. Hmm that’s bananas. Usually 64 K works better than anything else. What version of vSphere are you running? Is the underlying storage Pure or something else?

      Reply

  10. It says optimization is not available in the GUI but I am able to run it with PowerShell’s Optimize-Volume, but not defrag.exe

    Optimize-Volume -DriveLetter L -Analyze -Defrag ā€“Verbose

    Reply

  12. If I run the analyze only, I get this:
    PS P:\> Optimize-Volume -DriveLetter V -Analyze -Verbose
    VERBOSE: Invoking slab consolidation on IDE_SHARED (V:)…
    VERBOSE: Slab Analysis: 0% complete…
    Optimize-Volume : A general error occurred that is not covered by a more specific error code.
    At line:1 char:1
    + Optimize-Volume -DriveLetter V -Analyze -Verbose
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (MSFT_Volume (Ob…c-a1f9-ed6e…):ROOT/Microsoft/…age/MSFT_Volume) [Optimize-Volume], CimException
    + FullyQualifiedErrorId : HRESULT 0x89000029,Optimize-Volume

    Reply

    1. This is likely a VMware issue. Something configured for these VMDKs that is blocking UNMAP. My guess is the process that works isn’t actually working but it simply not handling the error properly. But it is hard to say. Have you been able to confirm the PS cmdlet actually unmaps space? Also did you open a case with our support? I can ping the support person and help.

      Reply

  13. Here is the ticket number I have open: 00362179

    But we didn’t make much progress as other drives work fine if newly created using 64K.

    Ill open a ticket with VMware as well.

    Reply

  14. Do you think if the GUI is broken yet I can force a optimize with this command all should be fine and good?
    Optimize-Volume -DriveLetter L -Analyze -Defrag -Verbose

    Reply

  15. Hello,

    i tested the script in our environment and it works so far.
    Only problem is that powershell generates a error message for each volume at the end of the script (convert block) – itĀ“s only a cosmetic problem.

    Cannot convert value “3Ā 543” to type “System.Int32”. Error: “Input string was not in a correct format.”
    At unmapsdkunattended.ps1:408 char:5
    + $reclaimedvirtualspace = $virtualspace[$i] – $usedvolcap
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : InvalidArgument: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvalidCastFromStringToInteger

    Is this problem already fixed or what have I done wrong?

    Reply

    1. Hmm interesting. Looks like there is a space in the value causing it to not be a integer but instead a string. Not sure why that would happen. I will take a look at the code to see if there is a reason. This is the first I’ve seen this issue.

      Reply

  16. Is there a “just report the space reclaimable” – don’t execute – just report ?
    Ideally reporting would be a quick(er) operation ? (could be plugin for daily vCheck reports? – maybe it already is ?)

    thanks!

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.