HPE Storage Users Group
https://3parug.com/

Will ebay disks work?
https://3parug.com/viewtopic.php?f=18&t=3131		 Page 1 of 2
Author:  AxisNL [ Mon Mar 18, 2019 8:18 am ]
Post subject:  Will ebay disks work?
Hey guys and girls,

I just started out a new job as a sysadmin for a company that uses 3pars. I never worked with them, but I'm starting to learn really fast, and I am already growing quite fond of 3par.

I currently have an issue with one of our 8200's. Apparently there was no monitoring in place, so nobody saw the issue before it was too late.

Short story: cpg full, volume read-only, and no way to fix.

Long storage: We have a pool with 46x6TB NL disks, with a raid6 CPG on top of that (178TB max usable). On that CPG are 15 x 16TB volumes (thin provisioned) that are used by a single Windows VM. Using windows storage spaces those 15 volumes are aggregated into a single 240TB volume used by a file server. Nobody thought about the thin provisioning, until the file server filled up, and then the 178TB limit was hit..

I see two possible paths to get out of this situation: 1. remove data and shrink, 2. add disks.

Option 1: We want to remove data again, but we can't. Since one of the VV's is in the failed state (it cannot allocate from the CPG), the volume is reported to Windows in the failed state, and is read-only. So we cannot remove any data from the volume :(

Option 2, adding disks, is the best idea of course. But according to our supplier, HPE will take 3-4 weeks to deliver some disks (!).

One of the idea's I'm playing with, is buying two disks from ebay (pn 823123-001), to have the problem fixed within a week and then replacing them when the original new hp disks arrive.

Is this a possibility? Can I just plug in any compatible disk and will it work? Or will I run into licensing issues and such?
Author:  MammaGutt [ Mon Mar 18, 2019 9:13 am ]
Post subject:  Re: Will ebay disks work?
AxisNL wrote:
One of the idea's I'm playing with, is buying two disks from ebay (pn 823123-001), to have the problem fixed within a week and then replacing them when the original new hp disks arrive.

Is this a possibility? Can I just plug in any compatible disk and will it work? Or will I run into licensing issues and such?


The perfect answer it depends.

If you can find 3PAR-branded drives (which 823123-001 seems to be) it "should work". However you probably will have difficulties getting support on those drives.

It also depends on what type of licenses you got. If you have the old licenses, 46 drives and 8200 you are 2 drive licenses away from hitting the drive cap (so you need licenses). If you have the new All-Inclusive it doesn't count drives anymore. "showlicense" in CLI will say if it counts disks or not.

One thing that you've not looked into is (temporarily) reduce the amount of spare in the system.

I'm guessing you're doing default sparing which will have 2 drives worth of spares. It is really not ideal to reduce it, but from the looks of it you're way beyond the point of what is recommended. Download this guide ( https://support.hpe.com/hpsc/doc/public ... =c04204279 )and look at removespare command. This will rather instantly remove capacity from spare pool and allocate it to your capacity pool.

removespare 0:a will remove all spares on PD 0. Do that for all drives and you should get ~10TB of additional space to help you survive until you can source more drives and licenses. Once you add more drives it will auto-trigger "admithw" which will re-create spares .... Once new drives have balanced you should run "admithw" again to ensure you have the correct spare capacity again.

Keep in mind, doing this with the spare will put you in a position where you ONLY have RAID redundancy, so you need to replace failed drives quickly until you've gotten more drives!
Author:  AxisNL [ Mon Mar 18, 2019 9:58 am ]
Post subject:  Re: Will ebay disks work?
MammaGutt wrote:
AxisNL wrote:
One thing that you've not looked into is (temporarily) reduce the amount of spare in the system.

I'm guessing you're doing default sparing which will have 2 drives worth of spares. It is really not ideal to reduce it, but from the looks of it you're way beyond the point of what is recommended. Download this guide ( https://support.hpe.com/hpsc/doc/public ... =c04204279 )and look at removespare command. This will rather instantly remove capacity from spare pool and allocate it to your capacity pool.

removespare 0:a will remove all spares on PD 0. Do that for all drives and you should get ~10TB of additional space to help you survive until you can source more drives and licenses. Once you add more drives it will auto-trigger "admithw" which will re-create spares .... Once new drives have balanced you should run "admithw" again to ensure you have the correct spare capacity again.

Keep in mind, doing this with the spare will put you in a position where you ONLY have RAID redundancy, so you need to replace failed drives quickly until you've gotten more drives!

You are my hero! I disabled some spare blocks on a few drives, and this allowed the CPG and the volume to grow again. Now the server admin can log in and delete some data again, so we don't run into this issue again the next weeks. New drivers are ordered, I'll be sure to run the admithw command again to check the spares!
Author:  MammaGutt [ Mon Mar 18, 2019 10:24 am ]
Post subject:  Re: Will ebay disks work?
Not sure how storage spaces work, but with windows mount points unmap is not forwarded to the 3PAR. Only way to reclaim deleted capacity is running a script which writes zero blocks to empty parts of the disk. Also not sure how/if that would impact storage spaces.

But yeah, tl:dr, it might not help simply deleting files in VM (assuming hyper-v)
Author:  adam [ Tue Mar 19, 2019 1:44 am ]
Post subject:  Re: Will ebay disks work?
As recommended do a "showlicense" and check your licensing model.
I got caught out once by this from moving disks between two of our 3par 8200's.
plugged the 8 new disks in and disk license limit was hit doh!

I purchased an 8 disk license to add the disks but it took approx 2 weeks for the reseller to explain to HPE what i was trying to do.

but now we have transition to all inclusive licensing.
Author:  AxisNL [ Tue Mar 19, 2019 6:24 am ]
Post subject:  Re: Will ebay disks work?
MammaGutt wrote:
Not sure how storage spaces work, but with windows mount points unmap is not forwarded to the 3PAR. Only way to reclaim deleted capacity is running a script which writes zero blocks to empty parts of the disk. Also not sure how/if that would impact storage spaces.

But yeah, tl:dr, it might not help simply deleting files in VM (assuming hyper-v)

Thanks, I'm looking into trimming and unmap commands in windows!
Author:  nicDM77 [ Tue Mar 19, 2019 7:30 am ]
Post subject:  Re: Will ebay disks work?
another consideration:
we create a 1TB thick provisioned virtual volume on each of our 3PARs, as a sort of reserve.
If we ever do run into space allocation issues, we can delete such a volume quickly and create a little bit of buffer.
Author:  AxisNL [ Tue Mar 19, 2019 7:32 am ]
Post subject:  Re: Will ebay disks work?
nicDM77 wrote:
another consideration:
we create a 1TB thick provisioned virtual volume on each of our 3PARs, as a sort of reserve.
If we ever do run into space allocation issues, we can delete such a volume quickly and create a little bit of buffer.

Smart!
Author:  nicDM77 [ Tue Mar 19, 2019 7:34 am ]
Post subject:  Re: Will ebay disks work?
AxisNL wrote:
Thanks, I'm looking into trimming and unmap commands in windows!

Not sure if this is still applicable with "modern age" windows servers, but sdelete might be worth looking at (again, if still applicable :?: )

https://docs.microsoft.com/en-us/sysinternals/downloads/sdelete
Author:  MBILC [ Thu Mar 21, 2019 11:33 am ]
Post subject:  Re: Will ebay disks work?
AxisNL wrote:
nicDM77 wrote:
another consideration:
we create a 1TB thick provisioned virtual volume on each of our 3PARs, as a sort of reserve.
If we ever do run into space allocation issues, we can delete such a volume quickly and create a little bit of buffer.

Smart!


But also should not be needed if you are properly monitoring and maintaining your storage.

By chance have you force a comptactcpg -f -pat * run on the 3pars?

Even though that runs on its own, I have gained 20TB yesterday from running it on production...
Page 1 of 2 All times are UTC - 5 hours
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/