HPE Storage Users Group
https://3parug.com/

Find dedup ratio pr. volume
https://3parug.com/viewtopic.php?f=18&t=3103
Page 1 of 1

Author:  skumflum [ Wed Feb 20, 2019 7:14 am ]
Post subject:  Find dedup ratio pr. volume

I am trying to convert a volume from dedup v2 to v3 but there is something I don't understand about the space utilized.

I am first converting it to TVV and then to TDVV on another CPG.

I used showvv -r to find out how much RAW space it used before and after. And the space used is increased by 50%.
I am skeptical about this large increase, but I know that the dedup ratio won’t be great as I only have done this with a few volumes

However I would like to get the math right. Can I somehow find the dedup ratio on a specific volume?

Author:  MammaGutt [ Wed Feb 20, 2019 8:44 am ]
Post subject:  Re: Find dedup ratio pr. volume

If you can tell me how you would like it calculated it's easy :)

(btw, dedupe shown with showvv -r is not the inter- or intra-dedupe ratio of the vomume)

Also, what was the CPG settings on old and new volumes?

Author:  aDelaloy [ Wed Feb 20, 2019 10:34 am ]
Post subject:  Re: Find dedup ratio pr. volume

hello,


you can ask for specific columns in showvv to be displayed with -showcols, and in this case the interesting value would be in Dedup .

since it's a regular showvv order, you an add some name/filters.

here is the complete command I used when I first deduped some VV in 3parOS322 :

showvv -showcols Name,Prov,Adm_Used_MB,Snp_Used_MB,Usr_Used_MB,VSize_MB,Dedup *MyVOLSregexp*

Author:  MammaGutt [ Wed Feb 20, 2019 12:02 pm ]
Post subject:  Re: Find dedup ratio pr. volume

aDelaloy wrote:
hello,


you can ask for specific columns in showvv to be displayed with -showcols, and in this case the interesting value would be in Dedup .

since it's a regular showvv order, you an add some name/filters.

here is the complete command I used when I first deduped some VV in 3parOS322 :

showvv -showcols Name,Prov,Adm_Used_MB,Snp_Used_MB,Usr_Used_MB,VSize_MB,Dedup *MyVOLSregexp*


That is just a weighted number and is neither the dedupe ratio of that single volume if placed in its own CPG nor related to how much the CPG might shrink if the volume is removed.

If you want the exact data for just one single volume if it was placed in a CPG of its own you need to use the dedupe estimator.

Author:  skumflum [ Thu Feb 21, 2019 2:52 am ]
Post subject:  Re: Find dedup ratio pr. volume

skumflum wrote:
I am trying to convert a volume from dedup v2 to v3 but there is something I don't understand about the space utilized.

I am first converting it to TVV and then to TDVV on another CPG.

I used showvv -r to find out how much RAW space it used before and after. And the space used is increased by 50%.
I am skeptical about this large increase, but I know that the dedup ratio won’t be great as I only have done this with a few volumes

However I would like to get the math right. Can I somehow find the dedup ratio on a specific volume?


Thank you :)

I will try to explain in another way.

Showvv -r RawRsvd shows the actual RAW space the VV has reserved right?
I moved a volume from a CPG with dedup v2 R5 7+1 to another CPG R6 12+2 with dedup v2.

I did the following:
1. Converted it to thin
2. Moved it the other CPG
3. Converted it back to dedup

The volume RawRsvd was: 1429650
After the convertion it is: 2209498

I don’t get that. R5 7+1 and R6 12+2 should have the same space penalty. My guess would be that it dedups with less efficiency and that’s why I asked for dedup stats pr. Volumes. However, it may be something else.

How would you monitor the space used when taking on this task?

Author:  MammaGutt [ Thu Feb 21, 2019 3:34 am ]
Post subject:  Re: Find dedup ratio pr. volume

7+1 and 14+2 has the same penalty. Not 12+2.

For thin you are right about showvv - r, for dedupe you are not.

For a single TDVV in a CPG with multiple TDVVs, the number will never be correct. Because you have a spiderweb of inter and intra VV dedupe it just isn't possible to provide a dedupe ratio for a single VV.

Example: you have 100 totally identical volumes with no dedupe within the VV. CPG dedupe will be 100:1 but if that volume was the only one in the CPG you will get 1:1. What would you expect the dedupe ratio for that volume to be?

And to put it into the real world, some blocks dedupe within the volume and some blocks dedupe with other blocks in other volumes in the CPG. In this real world the "dedupe ratio" of a volume might decrease when you delete a totally different volume, or improve when you add another volume.

Author:  richard612 [ Thu Feb 21, 2019 10:49 am ]
Post subject:  Re: Find dedup ratio pr. volume

New guy trying to follow along. So when there are multiple TDVVs in a CPG, the dedupe occurs across all of those TDVVs? The process isn't localized to individual TDVVs?

Author:  MammaGutt [ Thu Feb 21, 2019 11:03 am ]
Post subject:  Re: Find dedup ratio pr. volume

richard612 wrote:
New guy trying to follow along. So when there are multiple TDVVs in a CPG, the dedupe occurs across all of those TDVVs? The process isn't localized to individual TDVVs?


Correct.

Dedupe happens in the DDS (DeDupe Store). There is one DDS per CPG.

Author:  skumflum [ Wed Feb 27, 2019 9:21 am ]
Post subject:  Re: Find dedup ratio pr. volume

MammaGutt wrote:
7+1 and 14+2 has the same penalty. Not 12+2.

For thin you are right about showvv - r, for dedupe you are not.

For a single TDVV in a CPG with multiple TDVVs, the number will never be correct. Because you have a spiderweb of inter and intra VV dedupe it just isn't possible to provide a dedupe ratio for a single VV.

Example: you have 100 totally identical volumes with no dedupe within the VV. CPG dedupe will be 100:1 but if that volume was the only one in the CPG you will get 1:1. What would you expect the dedupe ratio for that volume to be?

And to put it into the real world, some blocks dedupe within the volume and some blocks dedupe with other blocks in other volumes in the CPG. In this real world the "dedupe ratio" of a volume might decrease when you delete a totally different volume, or improve when you add another volume.


Thank you

Tt was a typo... the new CPG IS 14+2 so it should have the same penalty

I did some more investigation, and something is strange (to me).
The same pattern when migrating other volumes. Approximal 50% more uses raw space.
Then I looked at the compaction rate before and after a migration. As I understand it compaction ratio is zero’s, right? This should be the same.

A volume had 10:1 compaction ratio before I moved it and 6:1 after I moved it.

Do you have any ideas? :?

Author:  MammaGutt [ Thu Feb 28, 2019 5:30 am ]
Post subject:  Re: Find dedup ratio pr. volume

Dedupe numbers are not updated in real-time per VV. Due to the reasons I've stated previously it isn't straight forward to get a dedupe ratio per volume so there are some calculations being done in the background to give estimates.

The only way to get an accurate detail about a single VV is to use the dedupe estimator for that single VV. Compaction ratio is the combined savings of all data reduction technologies (thin provision, zero detect, dedupe and compression). Basically how much data is written to disk vs the size of the volume.

Page 1 of 1 All times are UTC - 5 hours
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/