Managing In-Guest UNMAP and Automatic VMFS-6 UNMAP with PowerCLI

Another UNMAP post.  I was working on updating my best practices script the other day and I realized a lot of UNMAP configuration from a PowerCLI standpoint was not well documented, especially for the vSphere 6.5 stuff which introduces automatic UNMAP to VMFS. Automatic UNMAP  is great. But what if someone turns it off? Or what if, for some reason, I want to disable it? Or I want to make  sure it is on? Well there are a lot ways to do this–so let’s look at PowerCLI.

So there are two things here. In-guest UNMAP and VMFS UNMAP. Let’s define both:

• In-guest UNMAP: When a guest deletes or moves a file on a virtual disk. This space is dead on the VMFS and the underlying volume.
• VMFS UNMAP: When you delete or move a VM or a virtual disk. This space becomes dead on the underlying volume.

If you are running ESXi 6.0 in-guest UNMAP is enabled by default for Windows VMs. In ESXi 6.5, Linux support was added. What this means is if a guest OS deletes or moves a files on a thin virtual disk, the thin virtual disk will be shrunk by the size of that file. If you enable the setting EnableBlockDelete, ESXi will take it one step further and issue UNMAP to the physical volume that hosts the VMFS which reclaims the space on the array. This second step, turned on by EnableBlockDelete is NOT enabled by default.

The second part is VMFS UNMAP. When you delete a VM or a virtual disk, UNMAP can be issued to the VMFS to reclaim the space on the physical volume. VMFS UNMAP is a manual procedure from ESXi 5.0 U1 through ESXi 6.0 U2.

Not until ESXi 6.5 and VMFS-6 is this automated. This is controlled by two settings:

1. VMFS Automatic UNMAP: Each VMFS-6 datastore has a setting for automatic UNMAP. It is either “None” meaning off, or “low”. For the time being, ignore the blog posts out there that say that it can be medium or high. In the initial release of ESXi 6.5 only none or low is enabled/acknowledged in the code.
2. ESXi Automatic UNMAP participation: This is an ESXi host setting that says if that host will participate in automatic UNMAP.

Let’s start with enabling in-guest and then we will look at the ESXi host participation.

Managing In-Guest UNMAP

For in-guest UNMAP in the VM you need to make sure of two things:

• In Windows, the NTFS must use the 64K allocation unit
• In Linux, the file system must be mounted with the discard option

Check these posts for more information:

• What’s new in ESXi 6.5 Storage Part I: UNMAP
• Allocation Unit Size and Automatic Windows In-Guest UNMAP on VMware
• Direct Guest OS UNMAP in vSphere 6.0

To enable the end-to-end UNMAP, you need to enable EnableBlockDelete. This can be done with PowerCLI.\

So, remember, this value is only applicable to ESXi 6.0+.

First, check if it is enabled. Since this is an advanced setting you can use the “get-advancedsetting” cmdlet. Though you can use the esxcli PowerCLI stuff, this is easier.

First get the ESXi host. get-vmhost with the name will do it, store it in a variable.

$esx = get-vmhost esxi3-pure.flashcloud.local

Then, get the value with this:

$enableblockdelete = $esx | Get-AdvancedSetting -Name VMFS3.EnableBlockDelete

You can check the value of the setting by looking at the “value” property. If it is set to 1 it is enabled, if it is set to 0 it is disabled.

So since mine reports as disabled, I want to set it to 1 in order to enable it. This can be done with the “set-advancedsetting” cmdlet by passing the previously stored variable into it:

$enableblockdelete |Set-AdvancedSetting -Value 1 -Confirm:$false

Adding -Confirm:$false automatically accepts the warning PowerCLI sends asking you to confirm the change.

Managing Automatic VMFS UNMAP

The next part is managing automatic UNMAP with vSphere 6.5 and VMFS-6. Let’s be clear about two things:

• This is entirely enabled by default–there is no need to turn this on for new environments. This is only if someone turned it off.
• Just because you are using vSphere 6.5 doesn’t mean you have automatic UNMAP. You must be using VMFS-6 volumes, which is NOT the default format for volumes in vSphere 6.5.

First, let’s check to see if a datastore has automatic UNMAP enabled. As I mentioned, it must be VMFS-6, which you can confirm easily with PowerCLI:

$datastore = get-datastore MyDatastore
$datastore.ExtensionData.info.vmfs.version

Make sure it is 6.x (current version is 6.81).

Then you can check the automatic setting with PowerCLI using the esxcli implementation (I am using the v2 implementation which I highly recommend). One thing to note as well, is that this is a datastore level setting–if automatic UNMAP is enabled on a datastore any ESXi host that is capable of/configured to issue automatic UNMAP they will. You only need to enable or disable it on datastore once.

There are four commands: obtain the esxcli information for any compatible host, build the variable properties with the .createargs method and then populate it with the datastore name, then query the datastore:

$esxcli=get-esxcli -VMHost $esx -v2
$unmapargs = $esxcli.storage.vmfs.reclaim.config.get.createargs()
$unmapargs.volumelabel = $datastore.name
$unmapresult = $esxcli.storage.vmfs.reclaim.config.get.invoke($unmapargs)

The value you want to check is the property called “ReclaimPriority”. If it is set to “none”, it is off. You want this to be set to “low”.

If you need to change it, the process is very similar:

$unmapsetargs = $esxcli.storage.vmfs.reclaim.config.set.createargs()
$unmapsetargs.volumelabel = $datastore.name
$unmapsetargs.reclaimpriority = "low"
$esxcli.storage.vmfs.reclaim.config.set.invoke($unmapsetargs)

If it worked, the last command will return “true”.

The second part is checking whether an ESXi 6.5 host is configured to participate in automatic UNMAP for the VMFS-6 datastores it has access to. This is controlled by a host setting called “EnableVMFS6Unmap”.

This setting is not as obvious as the datastore setting–it is not available in the GUI nor will you see it when you list advanced settings in the CLI–you have to specify it specifically by name to see it. One of the “hidden” settings similar to the XCOPY tunable “MaxHWTransferSize” for example. Anyways, to check its configuration you can use the same process we used as above for “EnableBlockDelete”.

$autounmap = $esx | Get-AdvancedSetting -Name VMFS3.EnableVMFS6Unmap

Mine is set to 0, so someone disabled it. I want to re-enable it with set-advancedsetting:

$autounmap |Set-AdvancedSetting -Value 1 -Confirm:$false

All set! Making sure these three settings are on will ensure that your UNMAP is working and your environment is as efficient from a space perspective as possible.

Below is an full PowerCLI script that check your full vCenter for all three of these settings and then fixes them if needed.

This script:

1. Connects to a vCenter via interactive dialog during execution of the script for vCenter and credentials
2. Checks all ESXi 6.0 and later hosts for EnableBlockDelete and EnableVMFS6Unmap to be enabled. If they are not enabled they are enabled by the script.
3. Checks all VMFS-6 datastores for automatic UNMAP. If it is not enabled the script enables it.
4. Logs everything to a file specified by the user during the script (interactive dialog)
5. Writes to the screen only hosts or datastores that were changed

Get it here:

https://github.com/codyhosterman/powercli/blob/master/checkandfixUNMAP.ps1