Understanding VMware ESXi Queuing and the FlashArray

So I am in the middle of updating my best practices guide for vSphere on FlashArray and one of the topics I am looking into providing better guidance around is ESXi queue management. This breaks down to a few things:

  • Array volume queue depth limit
  • Datastore queue depth limit
  • Virtual Machine vSCSI Adapter queue depth limit
  • Virtual Disk queue depth limit

I have had more than a few questions lately about handling this–either just general queries or performance escalations. And generally from what I have found it comes down to fundamental understanding of how ESXi queuing works. And how the FlashArray plays with it. So I put a blog post together of a use case and walking through solving a performance problem. Explaining concepts along the way.

Please note:

  • This is a simple example to explain how queuing works in ESXi
  • Mileage will vary depending on your workload and configuration
  • This workload is targeted specifically to make relationships easier to understand
  • PLEASE do not make changes in your environment at least until you read my conclusion at the end. And frankly not without direct guidance from VMware support.

I am sorry, this is a long one. But hopefully informative!

If you prefer a video, here is my 1 hr VMworld session that goes into depth on what I write below:

Continue reading “Understanding VMware ESXi Queuing and the FlashArray”

Introducing the FlashArray Plugin for vRealize Orchestrator v1.0

This is a blog I have been waiting a long time to write. The past year and a half of my work has heavily focused on improving and building our VMware vRealize integration at Pure Storage. Log Insight and Operations Manager integration already existed (analytics etc.), so the next logical step is actually provisioning (orchestration). So vRealize Orchestrator and Automation. The first step I took was using the built-in REST plugin in vRO to build a workflow package that customers could use to actually manage the FlashArray without much work on their own part inside of vRO.

I started to realize that a workflow package was not enough. Especially when it comes to vRA Anything-As-A-Service integration. A big part of what is missing from a workflow package is custom objects and inventory management. Something that a plugin can easily achieve. So, without further ado–please meet the FlashArray vRO plugin! Downloadable at the VMware Solution Exchange and fully certified by VMware and Pure Storage:

FlashArray vRO Download

My vRO plugin white paper Continue reading “Introducing the FlashArray Plugin for vRealize Orchestrator v1.0”

VMFS Snapshots and the FlashArray Part VII: Restoring a VM

This is part 7 of this 7 part series. Questions around managing VMFS snapshots have been cropping up a lot lately and I realized I didn’t have a lot of specific Pure Storage and VMware resignaturing information out there. Especially around scripting all of this and the various options to do this. So I put a long series out here about how to do all of this.

The series being:

  1. Mounting an unresolved VMFS
  2. Why not force mount?
  3. Why might a VMFS resignature operation fail?
  4. How to correlate a VMFS and a FlashArray volume
  5. How to snapshot a VMFS on the FlashArray
  6. How to mount a VMFS FlashArray snapshot
  7. Restoring a single VM from a FlashArray snapshot

Continue reading “VMFS Snapshots and the FlashArray Part VII: Restoring a VM”

PowerShell GUI VMware and FlashArray Storage Management Tool

Here is my storage manager for the FlashArray and VMware. Based on PowerCLI, but uses a front end GUI. Enjoy!

NOTICE THIS HAS BEEN DEPRECATED IN FAVOR OF THE POWERSHELL MODULE HERE:
https://www.codyhosterman.com/scripts-and-tools/pure-storage-powershell-vmware-module/

There are a variety of methods of managing VMware objects (VMFS volumes, VMs, VMDKs and RDMs) and the underlying snapshots to recovery or clone them. But often I get asked if I have a PowerShell (PowerCLI) script to do one or all of them. I have a bunch on my GitHub, but I decided a week or so ago to put something a bit more robust together. At first I was making it a standard interactive script, but it morphed into a GUI, using combo-boxes etc:

Continue reading “PowerShell GUI VMware and FlashArray Storage Management Tool”

Detecting what FlashArray VMFS Volumes Have Dead Space

Another UNMAP post, are you shocked? A common question that came up was what volumes have dead space? What datastores should I run UNMAP on?

My usual response was, well it is hard to say. Dead space is introduced when you move a VM or you delete one. The array will not release the space until you either delete the physical volume, overwrite it, or issue UNMAP. Until vSphere 6.5, UNMAP for VMFS was not automatic. You had to run a CLI command to do it. So that leads back to the question, well I have 100 datastores, which ones should I run it on?

So to find out, you need to know two things:

  1. How much space the file system reports as currently being used.
  2. How much space the array is physically storing for the volume hosting that file system.

Continue reading “Detecting what FlashArray VMFS Volumes Have Dead Space”

VMFS Capacity Monitoring in a Data Reducing World

A question recently came up on the Pure Storage Community Forum about VMFS capacity alerts that said, to paraphrase:

“I am constantly getting capacity threshold (75%) alerts on my VMFS volumes but when I look at my FlashArray volume used capacity it is nowhere near that in used space. What can I do to make the VMware number closer to the FlashArray one so I don’t get these alerts?”

This comment really boils down to what is the difference between these numbers and how do I handle it? So, let’s dig into this. Continue reading “VMFS Capacity Monitoring in a Data Reducing World”

1.7 Release of the FlashArray Workflow Package for vRealize Orchestrator

Finally starting to catch up on work after VMworld. A lot of blog posts queued up in my head that I want to start getting out. Here is the first. I have completed an update of the FlashArray workflow package with some bug fixes and some new workflows. As always the workflow package can be found here:

https://github.com/codyhosterman/orchestrator/blob/master/com.purestorage.codyhosterman.package

17image

 

Continue reading “1.7 Release of the FlashArray Workflow Package for vRealize Orchestrator”

vRealize Orchestrator, TLS 1.2 and Certificate Importing

As I have blogged about before, TLS 1.0 and SSL v3 were deprecated in Purity 4.7, requiring all connections to use either TLS 1.1 or TLS 1.2. This affected a variety of integrations, some we updated, some you just had to alter their behavior. A few VMware products do not/did not use TLS 1.1/1.2 by default, so they either need to altered or upgraded. This almost invariably boiled down to the JDK version that was in use. vRealize Orchestrator is no exception.

vcologo

Continue reading “vRealize Orchestrator, TLS 1.2 and Certificate Importing”

Running UNMAP with vRealize Orchestrator

Let me start out with saying I’m embarrassed I have only been using vRO for 8 months or so. It is AWESOME.

The FlashArray Workflow Package for vRealize Orchestrator has been updated to include two new objects:

  1. Auto-expand datastore policy template
  2. Workflow to run UNMAP on a datastore

The creation of the first part is explained in this post. But if you are using the FlashArray it is all built into the package, so you have to do very little work. I’ll explain in a bit.

The UNMAP workflow is generic–it can be used with any VMFS datastore that supports UNMAP. So it is included in the workflow package and it is also standalone for those of you who don’t have a FlashArray. You can get the standalone here:

https://github.com/codyhosterman/orchestrator/blob/master/vmfsunmap.workflow

Continue reading “Running UNMAP with vRealize Orchestrator”

Automatic VMFS expansion with vCenter SNMP and vRealize Orchestrator

Virtual disk oversubscription is becoming increasingly common and so is allowing people to provision their own VMs. So increasing a datastore capacity is also an increasingly common operation. Because of the performance of flash, merged with ESXi features like VAAI ATS. Expanding a VMFS is easy. Expanding a storage volume these days is easy. But you still have to actually do it. What if I want to automate the process to respond to datastore capacity threshold limits? There are a variety of ways to achieve this. Let’s look at it via vCenter SNMP alerts and vRealize Orchestrator workflows. Continue reading “Automatic VMFS expansion with vCenter SNMP and vRealize Orchestrator”