This is a topic that I get asked about a lot and a recent internal email thread prompted me to write a post about it. On a Symmetrix array if you want a volume larger than 240 GB you need to create a metavolume. When creating a metavolume you have two configuration choices; concatenated or striped. There are many benefits to striped over concatenated (all of them performance-related) but one disadvantage. Due to their nature striped metavolumes are harder to expand. Until a few years ago thin striped metas couldn’t even be expanded online. So the decision was easy–do you think you will need to expand or not. In Enginuity 5875 and Solutions Enabler online expansion of striped metavolumes was allowed.
To expand one you need an identically configured metavolume configured as a BCV type and fully persistently allocated to a thin pool. During the process Enginuity copies the data to the BCV device and then adds the new device(s) to the original meta and then re-stripes all of the data back. Often I hear customers ask how to make sure to create an identical device and the answer is usually just “look at it and do the same thing”. Not ideal. But there is a better answer.
The Symmetrix configuration change command “symconfigure” offers the ability to create a new device by copying the configuration of an existing one. The symconfigure copy device command is in fact supported for use with metavolumes which is something that a lot of people don’t realize. I personally didn’t either for the longest time either, I was thrilled to find out that it was. Takes a lot of the guess work out of creating that BCV.
So the process:
Step 1: Identify the Symmetrix device ID of the metavolume you want to expand. This can be done a million ways and depends on your operating system etc. but for the case of VSI the easiest way is to use VSI. In the image below the device ID of my VMFS is 06CF.
Step 2: Create the BCV. Note this can be done in Unisphere or Solutions Enabler. SE first.
Using the symconfigure copy command I am going to create a new metavolume in one command identical to my source. In every way but one. I am going to override the config to indicate that the new volume will be a BCV instead of just a standard TDEV.
symconfigure -cmd "configure 1 devices copying dev 6CF overriding config=bcv+tdev;" commit -sid 38
This will create my device exactly how I need it. See Unisphere instructions via GIF below. Essentially just find the device and choose duplicate. The drawback is you cannot override the config. So it requires an extra step after to change it to a BCV. These processes will also bind the device to the same pool, so that’s not needed!
Step 3 Preallocate the device fully and persistently. SE or Unisphere.
symconfigure -cmd "allocate tdev 6D7 start_cyl=0, end_cyl=last_cyl, allocate_type=persistent;" commit -nop -sid 38
Or Unisphere:
Step 4. Expand the meta. Choose how much larger you want the device to be and add those new devices to the meta. Usually you want the meta to be a count of 4, 8, 16 or 32. So I making my 8 way into a 16 way. Doubling the capacity. The members, however many you add, just need to each be the same size as the original members. Use SE or Unisphere
symconfigure -cmd “add dev 6DF:6E6 to meta 6CF, protect_data=TRUE, bcv_meta_head=6D7;” -v commit -sid 38
Unisphere:
Note that this process can take a long time. Depends on how much data must be re-striped, what kind of workload is going to the device (especially the change rate) and the available VMAX resources. Recommend doing it when the device is not at its peak I/O period.
Step 4. Rescan your host and add the new capacity to the filesystem. For VMFS growing check out our techbook at page 69 here:
http://www.emc.com/collateral/hardware/solution-overview/h2529-vmware-esx-svr-w-symmetrix-wp-ldv.pdf
Once it is done go ahead and delete the BCV or re-purpose it for something else.
Wow, i thought the copy command was only supported for CKD devices, could not tell from SE man pages if it also binds to the same pool ? We have autometa enabled in our box, hopefully one will not step on another.
Yeah not well documented but definitely supports FBA too. Also binds to the same pool. Very handy, just found this all out recently.
Not sure about autometa though
Thanks. I like the ‘copying dev’ idea. This will save me some time.
Great! Glad you find it useful
The configure device with copying is very good tip, thanks Cody !
But don’t you agree that this should be implemented in the background for thin devices and be transparent to the user ?
You’re welcome! Agreed–yeah this process can be improved–Im sure it is being worked on
just tried it in my environment where we fully allocate thin devices, copying device that was fully allocated (allocate_type = persistent) resulted in device that was not fully allocated. Bummer i guess it does not copy all “attributes” of source device.
Yeah seems not to. Let me ping some people and see if I can find out exactly things the copy process actually copies. I am guessing they see allocation (persistent or not) as a non-static attribute so they don’t copy it. I will see what I can find out–if i get some type of a list I will update the post.
He Cody,
Thanks for this nice post,
did you ever get a reply on your ping ?
Now an unbind / bind fully alloc with persistent flag on action is still needed.
Thanks,
Ferry
He Cody,
did you ever hear something back on your ping ?
for now it seems i have to unbind an bind fully allocated with persistent flag on.
Allocation will not carry over with the copy; however I am unsure why you are doing an unbind and bind. Once the device is copied, simply run an allocation on the volume (full) and mark it as persistent.
Damos–I no longer work at EMC so I honestly don’t remember what came out of this. Sorry about that. Drew Tonnesen took over my work there so he’s the authority now
if you can get that list of attributes that would be great, Thank you Cody.
Hello Cody, I just had a question about devices 6DF:6E6. When were those devices created? Was it during the second step, or you’ve created those devices some other time? I just wasn’t sure if they get created when you did the step number 2 when you executed the copy.
Other thing that I was thinking around is the BCV. Does the size of the BCV device needs to be same or larger than original device or new expanded device?
Thanks a lot!
I created those prior to this process, I knew how much larger I wanted to make my meta so I created a few devices that were the same size as my current members so they would be valid new members. This step was skipped over in this process.
As far as the BCV it must be the same size, configuration, (member count, member size) as the original meta volume
Hi Cody, This is a nice process to expand striped meta on a symdev. The copy step and overriding the config type in a single syntax is really good. I’ve been using this in my client environment successfully. To add up, on SE 8.3 onwards, instead of symconfigure to preallocate we can use symdev syntax which is given below,
symdev allocate -devs -persistent -v -sid xxx