HPE Storage Users Group

A Storage Administrator Community




Post new topic Reply to topic  [ 19 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Understanding Space
PostPosted: Tue Oct 31, 2017 12:34 pm 

Joined: Thu Apr 17, 2014 11:58 am
Posts: 15
Hello all - I had how space worked explained to me when our 3par was installed, but that was 2013 and I was a bit overwhelmed. Fast forward to now, I'm a little mystified and was hoping for a bit of clarification.

We have a (I believe) 7000 series which shows 79% allocated and 21% free:

Image

If I go into my CPGs, here is how it lists:

Image

So I guess I have a few questions:

  1. How much space to i really have left:
  2. My CPG shows essentially that it is full - will it grow into the empty raw space?
  3. For my own curiousity, why would your CPG be set up to not use all your space in the first place (obviously I didn't do the setting up).
  4. My VVs are thinly provisioned and I'm using them with VM's that are thick provisioned, which is how I understand I should be doing it. Last week I initially thought I was getting low on space, so I got rid of some old VM's, but I cannot tell that it freed up space listed under CPGs - why would that be?

I thank you for any efforts to understand this. We are hoping to get some consulting done later this year to evaluate our space needs, but for now, we just want to make sure we are comfortable with what we have.


Top
 Profile  
Reply with quote  
 Post subject: Re: Understanding Space
PostPosted: Tue Oct 31, 2017 3:49 pm 

Joined: Tue Mar 29, 2016 9:32 pm
Posts: 136
Hiya,

Answer the simple question first:

You have 2,044,000GiB free

But ... that is just "raw" (normal) free space and doesn't account for anything like RAID levels, set size of volumes, reserved snapshot space for volumes, growth increments of CPGs etc.

Next .. a CPG isn't a container that has your volumes are in, it's not like a pool that you give disk space too, a CPG is just a template to say how you would like your volumes to be configured, so the space you see listed against the CPG is just the amount of space volumes are using that are using that CPGs template.

Working out how much space you really have left is awkward .. you basically have to guess! :-) It's a painful thing of a the 3PAR, the second picture you have shows the 3PARs guess/estimate as to how much usable space it would get if that (only only that) CPG was used for future volumes : 1,695,000GiB.

I personally like to work on the provisioned size of the volumes allocated to hosts (the size of how much each volume could potentially be if fully used).
I know how much "normal" space I have for volumes on each physical disks (either from the GUI with that first picture you have showing "Free:" or from CLI with "showpd -c" and looking at the total of the free normal chunklets column.
So long as my provisioned space is less than my normal space I'm happy, if I over provision that is me gambling no hosts go nuts and try to fully use all the space in their volumes. I'm an old school storage admin so over provisioning feels wrong to me :-)

Hope that helps.


Top
 Profile  
Reply with quote  
 Post subject: Re: Understanding Space
PostPosted: Tue Oct 31, 2017 3:52 pm 

Joined: Thu Jul 06, 2017 3:28 pm
Posts: 41
The first image you posted is raw space left on the array. This does not take into account any system overhead or raid settings applied on the cpg. The "free" space # is a bit misleading there. I found the best measurement of remaining space on a 3par is the "maximum cpg size" from the SSMC.

Deleting space from a VM hosted on the 3par will not initially free up any space on the 3par even though the data store will reflect the new space. You need to run scsi unmap commands from the esxi host to free up the chunklets on the 3par.

It is normal to see the CPG near full, it will auto expand as you fill it completely. My assumption is it expands as needed in case you add more disks in the meantime, the retune would take longer if everything was 100% allocated vs as needed.


Top
 Profile  
Reply with quote  
 Post subject: Re: Understanding Space
PostPosted: Tue Oct 31, 2017 9:33 pm 

Joined: Mon Jul 29, 2013 9:01 pm
Posts: 62
you need to run a compactcpg on your cpg...


Top
 Profile  
Reply with quote  
 Post subject: Re: Understanding Space
PostPosted: Wed Nov 01, 2017 4:51 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
jwood.mls wrote:
So I guess I have a few questions:

  1. How much space to i really have left:
  2. My CPG shows essentially that it is full - will it grow into the empty raw space?
  3. For my own curiousity, why would your CPG be set up to not use all your space in the first place (obviously I didn't do the setting up).
  4. My VVs are thinly provisioned and I'm using them with VM's that are thick provisioned, which is how I understand I should be doing it. Last week I initially thought I was getting low on space, so I got rid of some old VM's, but I cannot tell that it freed up space listed under CPGs - why would that be?


1. As others have stated, 2044 GiB of RAW capacity (pre-RAID) + whatever is available within your CPGs (I see 43GB net (post-RAID) in the CPG you've posted)
2. A healthy system has 99% used space within a CPG. When the CPG needs more space it will allocated from the system free space pool where you today have 2044GB free.
3. In case you want multiple CPGs (which is more common than uncommon) where you either have multiple tiers or different redundancy requirements on different parts of the system
4. Pre-VMFS6 you want Thin on 3PAR, Eager Zero Thick on Vmware. With VMFS6 you want thin on both as Vmware has finally enabled automatic unmap again. When you delete a VM on a VMFS5 datastore today, Vmware will only delete the pointers and leave all the data on the datastore. As 3PARs reclamation techology scans and reclaims zero blocks you would need to run unmap on each datastore in Vmware. With VMFS6 they finally enabled this automatically..... just like Microsoft did with Windows Server 2012. Unfortunately it seems that the zero-detect/reclaim on 3PAR is designed to handle day-to-day unmap but when you bulk up 2 years of Vmware unmap it will in many cases need to some time to get thru it.

Look at showvv -s after running unmap. Usr Used is what is actually written to the volume, while Usr Rsvd is how much is allocated. As time goes, Usr Rsvd will decline to almost the same number as Usr Rvsd.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Understanding Space
PostPosted: Wed Nov 01, 2017 5:34 am 

Joined: Wed Nov 19, 2014 5:14 am
Posts: 505
For a quick an dirty manual calculation you would take your Free Raw Space 2,044,000 GiB & multiply by the parity overhead of the CPG.

2044000 * 0.83 = 1,696,520 and that's how much free usable space you have to grow into, which maps roughly to the 1,695,000 the CPG is showing as Estimated Free System Space.

r5-ssz-3=0.66
r5-ssz-4=0.75
r5-ssz-5=0.80
r5-ssz-6=0.83
r5-ssz-7=0.85
r5-ssz-8=0.87
r5-ssz-9=0.88
r6-ssz-6=0.66
r6-ssz-8=0.75
r6-ssz-10=0.80
r6-12=0.83
r6-16=0.87

In this case the actual CPG calculated overhead is 1695000/2044000 = 0.8293.


Top
 Profile  
Reply with quote  
 Post subject: Re: Understanding Space
PostPosted: Wed Nov 01, 2017 11:49 am 

Joined: Thu Apr 17, 2014 11:58 am
Posts: 15
First off

THANK YOU all for your comments. The one time I had this explained to me was in a very brief "brain dump" windows and since I didn't have a good point of reference, some of it just came and went.

First off, thank you for reminding me that the CPG is a template. After you said that I did remember better about that.

Second off - one thing I was never told about was the unmap being needed (I'm running esxi 5.5). I'm certain I've deleted VM's, freed up space, etc, so I'm guessing it does need to be run. I've done a bit of reading about it and watched a video or two - One question I wanted to confirm is, it isn't destructive in any way is it? As near as I can tell it is kind of zeroing out space that was already marked as deleted, so I would think it should be fine, just wanted to be sure! I'm assuming since this is a more automatic process from what I can tell on newer versions of esxi, that it is probably something you want to run against all your VV's on a regular basis? (Also, side note, I ran the unmap on a vv that was iso files only where I had deleted some - in the management console, it shows 29% used user size, but a showvv -s command shows only 17.8% Usr used but the Rsvd size is more like the 29% - is that allowing for parity for the raid levels? ) . Edit: eventually the reserved size dropped a bit. Now after more clearing, I'm showing 15% used and 26% reserved.

Next question - sjm - compactcpg - I've seen reference to this but don't know much about it - Why do I need to do it? Does it need to be done regularly? Any potential ill effects?

Again, I really appreciate everything. Trying to let it sink in more as I try to learn (and relearn)!


Top
 Profile  
Reply with quote  
 Post subject: Re: Understanding Space
PostPosted: Wed Nov 01, 2017 3:38 pm 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
Unmap is not destructive. It may add some load to the host doing it as it writes zeros.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Understanding Space
PostPosted: Thu Nov 02, 2017 4:16 am 

Joined: Wed Nov 19, 2014 5:14 am
Posts: 505
When freeing space the following is happening under the covers.

You have a CPG which is template or policy for how volumes are layed out.
The volumes associated with the CPG are formed from logical disks.
These in turn are assembled from chunklets across disks based on the CPG policy.
Logical disks are shared across many volumes to provide wide striping of data across many disks.

When you signal space is free within a volume at the file system layer using unmap.
The space is returned synchronously to the underlying volume for reuse, this is referred to as User space.
However many of the blocks will remain allocated within the Logical disk, this referred to a Reserved space.
In order to return reserved space the system has to defrag this shared space into contiguos space.

This is required since the space is shared and scattered and needs to become contiguous to be freed.
There are thresholds below which a volumes reserved space will not fall in order to preserve performance and wide striping.
This space is returned asynchronously to the CPG once contiguous in order to support new logical disk growth for volumes associated with that CPG.

A compact CPG at this point will return the space to the system as a whole, for reuse in any similar CPG.
You can schedule this to happen periodically but doing so too often will just create more work for the system.


Top
 Profile  
Reply with quote  
 Post subject: Re: Understanding Space
PostPosted: Thu Nov 02, 2017 12:08 pm 

Joined: Thu Apr 17, 2014 11:58 am
Posts: 15
Alright, gotcha...so if you are running unmap and it does free up a lot of space, you probably should run compactcpg, but not something you would do all the time...

I did run unmap yesterday and it changed the used size on my VV's from 6% up to 23% (!!) in one case, so I'm guessing I do need to run the compact CPG to just get a clearer picture of where I am now.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 19 posts ]  Go to page 1, 2  Next


Who is online

Users browsing this forum: No registered users and 66 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group | DVGFX2 by: Matt