HPE Storage Users Group

A Storage Administrator Community




Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: Find dedup ratio pr. volume
PostPosted: Wed Feb 20, 2019 7:14 am 

Joined: Tue Nov 19, 2013 4:38 pm
Posts: 119
I am trying to convert a volume from dedup v2 to v3 but there is something I don't understand about the space utilized.

I am first converting it to TVV and then to TDVV on another CPG.

I used showvv -r to find out how much RAW space it used before and after. And the space used is increased by 50%.
I am skeptical about this large increase, but I know that the dedup ratio won’t be great as I only have done this with a few volumes

However I would like to get the math right. Can I somehow find the dedup ratio on a specific volume?


Top
 Profile  
Reply with quote  
 Post subject: Re: Find dedup ratio pr. volume
PostPosted: Wed Feb 20, 2019 8:44 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
If you can tell me how you would like it calculated it's easy :)

(btw, dedupe shown with showvv -r is not the inter- or intra-dedupe ratio of the vomume)

Also, what was the CPG settings on old and new volumes?

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Find dedup ratio pr. volume
PostPosted: Wed Feb 20, 2019 10:34 am 

Joined: Tue Nov 18, 2014 5:13 am
Posts: 17
hello,


you can ask for specific columns in showvv to be displayed with -showcols, and in this case the interesting value would be in Dedup .

since it's a regular showvv order, you an add some name/filters.

here is the complete command I used when I first deduped some VV in 3parOS322 :

showvv -showcols Name,Prov,Adm_Used_MB,Snp_Used_MB,Usr_Used_MB,VSize_MB,Dedup *MyVOLSregexp*


Top
 Profile  
Reply with quote  
 Post subject: Re: Find dedup ratio pr. volume
PostPosted: Wed Feb 20, 2019 12:02 pm 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
aDelaloy wrote:
hello,


you can ask for specific columns in showvv to be displayed with -showcols, and in this case the interesting value would be in Dedup .

since it's a regular showvv order, you an add some name/filters.

here is the complete command I used when I first deduped some VV in 3parOS322 :

showvv -showcols Name,Prov,Adm_Used_MB,Snp_Used_MB,Usr_Used_MB,VSize_MB,Dedup *MyVOLSregexp*


That is just a weighted number and is neither the dedupe ratio of that single volume if placed in its own CPG nor related to how much the CPG might shrink if the volume is removed.

If you want the exact data for just one single volume if it was placed in a CPG of its own you need to use the dedupe estimator.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Find dedup ratio pr. volume
PostPosted: Thu Feb 21, 2019 2:52 am 

Joined: Tue Nov 19, 2013 4:38 pm
Posts: 119
skumflum wrote:
I am trying to convert a volume from dedup v2 to v3 but there is something I don't understand about the space utilized.

I am first converting it to TVV and then to TDVV on another CPG.

I used showvv -r to find out how much RAW space it used before and after. And the space used is increased by 50%.
I am skeptical about this large increase, but I know that the dedup ratio won’t be great as I only have done this with a few volumes

However I would like to get the math right. Can I somehow find the dedup ratio on a specific volume?


Thank you :)

I will try to explain in another way.

Showvv -r RawRsvd shows the actual RAW space the VV has reserved right?
I moved a volume from a CPG with dedup v2 R5 7+1 to another CPG R6 12+2 with dedup v2.

I did the following:
1. Converted it to thin
2. Moved it the other CPG
3. Converted it back to dedup

The volume RawRsvd was: 1429650
After the convertion it is: 2209498

I don’t get that. R5 7+1 and R6 12+2 should have the same space penalty. My guess would be that it dedups with less efficiency and that’s why I asked for dedup stats pr. Volumes. However, it may be something else.

How would you monitor the space used when taking on this task?


Top
 Profile  
Reply with quote  
 Post subject: Re: Find dedup ratio pr. volume
PostPosted: Thu Feb 21, 2019 3:34 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
7+1 and 14+2 has the same penalty. Not 12+2.

For thin you are right about showvv - r, for dedupe you are not.

For a single TDVV in a CPG with multiple TDVVs, the number will never be correct. Because you have a spiderweb of inter and intra VV dedupe it just isn't possible to provide a dedupe ratio for a single VV.

Example: you have 100 totally identical volumes with no dedupe within the VV. CPG dedupe will be 100:1 but if that volume was the only one in the CPG you will get 1:1. What would you expect the dedupe ratio for that volume to be?

And to put it into the real world, some blocks dedupe within the volume and some blocks dedupe with other blocks in other volumes in the CPG. In this real world the "dedupe ratio" of a volume might decrease when you delete a totally different volume, or improve when you add another volume.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Find dedup ratio pr. volume
PostPosted: Thu Feb 21, 2019 10:49 am 

Joined: Wed Jan 16, 2019 4:26 pm
Posts: 37
Location: Florida
New guy trying to follow along. So when there are multiple TDVVs in a CPG, the dedupe occurs across all of those TDVVs? The process isn't localized to individual TDVVs?


Top
 Profile  
Reply with quote  
 Post subject: Re: Find dedup ratio pr. volume
PostPosted: Thu Feb 21, 2019 11:03 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
richard612 wrote:
New guy trying to follow along. So when there are multiple TDVVs in a CPG, the dedupe occurs across all of those TDVVs? The process isn't localized to individual TDVVs?


Correct.

Dedupe happens in the DDS (DeDupe Store). There is one DDS per CPG.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Find dedup ratio pr. volume
PostPosted: Wed Feb 27, 2019 9:21 am 

Joined: Tue Nov 19, 2013 4:38 pm
Posts: 119
MammaGutt wrote:
7+1 and 14+2 has the same penalty. Not 12+2.

For thin you are right about showvv - r, for dedupe you are not.

For a single TDVV in a CPG with multiple TDVVs, the number will never be correct. Because you have a spiderweb of inter and intra VV dedupe it just isn't possible to provide a dedupe ratio for a single VV.

Example: you have 100 totally identical volumes with no dedupe within the VV. CPG dedupe will be 100:1 but if that volume was the only one in the CPG you will get 1:1. What would you expect the dedupe ratio for that volume to be?

And to put it into the real world, some blocks dedupe within the volume and some blocks dedupe with other blocks in other volumes in the CPG. In this real world the "dedupe ratio" of a volume might decrease when you delete a totally different volume, or improve when you add another volume.


Thank you

Tt was a typo... the new CPG IS 14+2 so it should have the same penalty

I did some more investigation, and something is strange (to me).
The same pattern when migrating other volumes. Approximal 50% more uses raw space.
Then I looked at the compaction rate before and after a migration. As I understand it compaction ratio is zero’s, right? This should be the same.

A volume had 10:1 compaction ratio before I moved it and 6:1 after I moved it.

Do you have any ideas? :?


Top
 Profile  
Reply with quote  
 Post subject: Re: Find dedup ratio pr. volume
PostPosted: Thu Feb 28, 2019 5:30 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
Dedupe numbers are not updated in real-time per VV. Due to the reasons I've stated previously it isn't straight forward to get a dedupe ratio per volume so there are some calculations being done in the background to give estimates.

The only way to get an accurate detail about a single VV is to use the dedupe estimator for that single VV. Compaction ratio is the combined savings of all data reduction technologies (thin provision, zero detect, dedupe and compression). Basically how much data is written to disk vs the size of the volume.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 


Who is online

Users browsing this forum: Google [Bot] and 63 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group | DVGFX2 by: Matt