Changing an ESXi SATP Rule

One of the few hard requirements we make to configure best practices on ESXi for the FlashArray is to create a SATP rule. A SATP rule simply describes a certain configuration (mainly around multipathing) for a specific set of devices (usually devices from an array). For the FlashArray, this rule consists of making sure devices are using Round Robin and an I/O operations limit of 1.

esxcli storage nmp satp rule add -s VMW_SATP_ALUA -V PURE -M FlashArray -P “VMW_PSP_RR” -O iops=1 -e “FlashArray SATP Rule”

Increasingly a question I have received is how do I delete a rule? Seems like a simple request, and for the most part it is. But there are some umm peculiarities to it. So let’s say you created a rule like so:

esxcli storage nmp satp rule add -s VMW_SATP_ALUA -V PURE -M FlashArray -P “VMW_PSP_RR” -O iops=10 -e “FlashArray SATP Rule”

This says all Pure Storage FlashArray devices will use Round Robin and an I/O Operations Limit (IOPS) of ten. But no, I am perfectionist and I actually want it to be 1 as is recommended. You can’t simply change a SATP rule, you have to delete it and then create a new one that has the configuration you want.

Deleting a rule gets a bit weird. The first thing you should do is query the rules for the specifics of the rule you want to delete. There are a ton of default rules, so narrowing them helps, but as far as I can tell the only built in filter is by SATP, but of course grep is also your friend.

esxcli storage nmp satp rule list -s VMW_SATP_ALUA

This rule is for Pure Storage (vendor = PURE), FlashArray (model = FlashArray) the ALUA SATP, Round Robin PSP and I/O Operations as 10 (IOPS=10). There is also a description. Most everything else is blank.

To delete the rule, you need to supply the fields that cause the rule to be unique, such as the SATP, the PSP, the Vendor, the Model and any options. If you leave one out you get an error:

Error deleting SATP rule: Sysinfo error on operation returned status : Not found. Please see the VMkernel log for detailed error information

If you look at the VMkernel log you see something like this:

2015-06-29T23:14:04.273Z cpu4:34797 opID=581b5177)NMP: nmp_SatpRemoveSupportedID:1744: user rule Uid “(null)” Vendor “PURE” and Model “FlashArray” and Claim Options “(null)” Driver “(null)” not in the SATP “VMW_SATP_ALUA” Supported ID list.

You can see that it injects into the command claim options, user rule ID, and driver even if they are not specified. They are left as “null” though. What it doesn’t note is that that PSP options are required, so if you have a PSP option you must include it, the error in the vmkernel log will not express that you missed it or entered it wrong, so that makes it a bit tough to figure out at times. My rule has iops=10, so I need to add that. It is also important to note that these are all case senstive, so IOPS=10 is not iops=10. This will error out as well.

Note that you do not need to specify the “rule group” which is user or system or the description. The description is not deemed a unique indentifier so it is not required regardless to whether it exists. If you do add it for some reason it will accept it, but it will ignore it. Even if it is wrong the removal will succeed as long as everything else is right.

So to remove my rule from above? Just run:

esxcli storage nmp satp rule remove -s VMW_SATP_ALUA -V PURE -M FlashArray -P “VMW_PSP_RR” -O iops=10 -e “FlashArray SATP Rule”