HPE Storage Users Group https://3parug.com/ |
|
Find dedup ratio pr. volume https://3parug.com/viewtopic.php?f=18&t=3103 |
Page 1 of 1 |
Author: | skumflum [ Wed Feb 20, 2019 7:14 am ] |
Post subject: | Find dedup ratio pr. volume |
I am trying to convert a volume from dedup v2 to v3 but there is something I don't understand about the space utilized. I am first converting it to TVV and then to TDVV on another CPG. I used showvv -r to find out how much RAW space it used before and after. And the space used is increased by 50%. I am skeptical about this large increase, but I know that the dedup ratio won’t be great as I only have done this with a few volumes However I would like to get the math right. Can I somehow find the dedup ratio on a specific volume? |
Author: | MammaGutt [ Wed Feb 20, 2019 8:44 am ] |
Post subject: | Re: Find dedup ratio pr. volume |
If you can tell me how you would like it calculated it's easy (btw, dedupe shown with showvv -r is not the inter- or intra-dedupe ratio of the vomume) Also, what was the CPG settings on old and new volumes? |
Author: | aDelaloy [ Wed Feb 20, 2019 10:34 am ] |
Post subject: | Re: Find dedup ratio pr. volume |
hello, you can ask for specific columns in showvv to be displayed with -showcols, and in this case the interesting value would be in Dedup . since it's a regular showvv order, you an add some name/filters. here is the complete command I used when I first deduped some VV in 3parOS322 : showvv -showcols Name,Prov,Adm_Used_MB,Snp_Used_MB,Usr_Used_MB,VSize_MB,Dedup *MyVOLSregexp* |
Author: | MammaGutt [ Wed Feb 20, 2019 12:02 pm ] |
Post subject: | Re: Find dedup ratio pr. volume |
aDelaloy wrote: hello, you can ask for specific columns in showvv to be displayed with -showcols, and in this case the interesting value would be in Dedup . since it's a regular showvv order, you an add some name/filters. here is the complete command I used when I first deduped some VV in 3parOS322 : showvv -showcols Name,Prov,Adm_Used_MB,Snp_Used_MB,Usr_Used_MB,VSize_MB,Dedup *MyVOLSregexp* That is just a weighted number and is neither the dedupe ratio of that single volume if placed in its own CPG nor related to how much the CPG might shrink if the volume is removed. If you want the exact data for just one single volume if it was placed in a CPG of its own you need to use the dedupe estimator. |
Author: | skumflum [ Thu Feb 21, 2019 2:52 am ] |
Post subject: | Re: Find dedup ratio pr. volume |
skumflum wrote: I am trying to convert a volume from dedup v2 to v3 but there is something I don't understand about the space utilized. I am first converting it to TVV and then to TDVV on another CPG. I used showvv -r to find out how much RAW space it used before and after. And the space used is increased by 50%. I am skeptical about this large increase, but I know that the dedup ratio won’t be great as I only have done this with a few volumes However I would like to get the math right. Can I somehow find the dedup ratio on a specific volume? Thank you I will try to explain in another way. Showvv -r RawRsvd shows the actual RAW space the VV has reserved right? I moved a volume from a CPG with dedup v2 R5 7+1 to another CPG R6 12+2 with dedup v2. I did the following: 1. Converted it to thin 2. Moved it the other CPG 3. Converted it back to dedup The volume RawRsvd was: 1429650 After the convertion it is: 2209498 I don’t get that. R5 7+1 and R6 12+2 should have the same space penalty. My guess would be that it dedups with less efficiency and that’s why I asked for dedup stats pr. Volumes. However, it may be something else. How would you monitor the space used when taking on this task? |
Author: | MammaGutt [ Thu Feb 21, 2019 3:34 am ] |
Post subject: | Re: Find dedup ratio pr. volume |
7+1 and 14+2 has the same penalty. Not 12+2. For thin you are right about showvv - r, for dedupe you are not. For a single TDVV in a CPG with multiple TDVVs, the number will never be correct. Because you have a spiderweb of inter and intra VV dedupe it just isn't possible to provide a dedupe ratio for a single VV. Example: you have 100 totally identical volumes with no dedupe within the VV. CPG dedupe will be 100:1 but if that volume was the only one in the CPG you will get 1:1. What would you expect the dedupe ratio for that volume to be? And to put it into the real world, some blocks dedupe within the volume and some blocks dedupe with other blocks in other volumes in the CPG. In this real world the "dedupe ratio" of a volume might decrease when you delete a totally different volume, or improve when you add another volume. |
Author: | richard612 [ Thu Feb 21, 2019 10:49 am ] |
Post subject: | Re: Find dedup ratio pr. volume |
New guy trying to follow along. So when there are multiple TDVVs in a CPG, the dedupe occurs across all of those TDVVs? The process isn't localized to individual TDVVs? |
Author: | MammaGutt [ Thu Feb 21, 2019 11:03 am ] |
Post subject: | Re: Find dedup ratio pr. volume |
richard612 wrote: New guy trying to follow along. So when there are multiple TDVVs in a CPG, the dedupe occurs across all of those TDVVs? The process isn't localized to individual TDVVs? Correct. Dedupe happens in the DDS (DeDupe Store). There is one DDS per CPG. |
Author: | skumflum [ Wed Feb 27, 2019 9:21 am ] |
Post subject: | Re: Find dedup ratio pr. volume |
MammaGutt wrote: 7+1 and 14+2 has the same penalty. Not 12+2. For thin you are right about showvv - r, for dedupe you are not. For a single TDVV in a CPG with multiple TDVVs, the number will never be correct. Because you have a spiderweb of inter and intra VV dedupe it just isn't possible to provide a dedupe ratio for a single VV. Example: you have 100 totally identical volumes with no dedupe within the VV. CPG dedupe will be 100:1 but if that volume was the only one in the CPG you will get 1:1. What would you expect the dedupe ratio for that volume to be? And to put it into the real world, some blocks dedupe within the volume and some blocks dedupe with other blocks in other volumes in the CPG. In this real world the "dedupe ratio" of a volume might decrease when you delete a totally different volume, or improve when you add another volume. Thank you Tt was a typo... the new CPG IS 14+2 so it should have the same penalty I did some more investigation, and something is strange (to me). The same pattern when migrating other volumes. Approximal 50% more uses raw space. Then I looked at the compaction rate before and after a migration. As I understand it compaction ratio is zero’s, right? This should be the same. A volume had 10:1 compaction ratio before I moved it and 6:1 after I moved it. Do you have any ideas? |
Author: | MammaGutt [ Thu Feb 28, 2019 5:30 am ] |
Post subject: | Re: Find dedup ratio pr. volume |
Dedupe numbers are not updated in real-time per VV. Due to the reasons I've stated previously it isn't straight forward to get a dedupe ratio per volume so there are some calculations being done in the background to give estimates. The only way to get an accurate detail about a single VV is to use the dedupe estimator for that single VV. Compaction ratio is the combined savings of all data reduction technologies (thin provision, zero detect, dedupe and compression). Basically how much data is written to disk vs the size of the volume. |
Page 1 of 1 | All times are UTC - 5 hours |
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ |