HPE Storage Users Group

A Storage Administrator Community




Post new topic Reply to topic  [ 19 posts ]  Go to page Previous  1, 2
Author Message
 Post subject: Re: 8440 3PAR enclosure design with 216 SSD disks
PostPosted: Mon Aug 07, 2017 10:24 pm 

Joined: Tue Mar 29, 2016 9:32 pm
Posts: 136
Darking wrote:
does that really matter tho?

I believe HP gives 7 years warranty on all SSDs bought after june 2015.

https://www.hpe.com/h20195/v2/getpdf.as ... f?ver=10.0



True .. but I wonder what will happen in 3 years time if all our drives have worn down at the same rate and all want to be replaced in the same week :-)
- yes our 3PARs are the el-cheapo model with the cMLCs .. sigh! :-)


Top
 Profile  
Reply with quote  
 Post subject: Re: 8440 3PAR enclosure design with 216 SSD disks
PostPosted: Sun Aug 20, 2017 1:09 pm 

Joined: Tue Jan 15, 2013 10:33 am
Posts: 41
So we have updated to 16 enclosures (14+2) plus eight more ssd disks. So we have would have 224 1,92TB disks. Now I am thinking about the raid level. I dont know if I should move to raid6 or keep raid5 (3+1) and cage ha.

I guess I have the following options:
raid 6 (6+2) cage ha 75% capacity efficiency
raid 5 (3+1) cage ha 75% capacity efficiency
raid 5 (7+1) cage ha 87,5% capacity efficiency

I am currently afraid about the write penalty which would be introduced with raid6. What do you think? Currently my two 8200 are running raid5 3+1 flawlessly with cage ha.

Okay, I've read the actual best practice guide for 3PAR and for SSDs it is recommended to have raid6 with 3+1 or 7+1 for systems which have the capacity. There must be something wrong, I haven't ever seen raid 6 with a set size of 3+1 or 7+1 :P.

Further down at the AO section, is the best practice for SSDs Raid 5 with set size 3+1.

I dont get it.....


Top
 Profile  
Reply with quote  
 Post subject: Re: 8440 3PAR enclosure design with 216 SSD disks
PostPosted: Mon Aug 21, 2017 8:02 am 

Joined: Wed Nov 19, 2014 5:14 am
Posts: 505
Might be a typo but from a parity overhead perspective 3+1 is the same as 6+2 and 7+1 the same as 14+2. Some of the documents may be referring to earlier recommendations, but as the drives get bigger it becomes more prudent to use Raid 6. It's also simpler to just recommend a blanket Raid 6 vs one based on the capacity or drive model etc. See below Raid 6 whitepaper.

https://www.hpe.com/h20195/V2/Getdocument.aspx?docname=a00000244enw


Top
 Profile  
Reply with quote  
 Post subject: Re: 8440 3PAR enclosure design with 216 SSD disks
PostPosted: Mon Aug 21, 2017 9:33 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
markinnz wrote:
Darking wrote:
does that really matter tho?

I believe HP gives 7 years warranty on all SSDs bought after june 2015.

https://www.hpe.com/h20195/v2/getpdf.as ... f?ver=10.0



True .. but I wonder what will happen in 3 years time if all our drives have worn down at the same rate and all want to be replaced in the same week :-)
- yes our 3PARs are the el-cheapo model with the cMLCs .. sigh! :-)


To be honest I would like to see that :)

I've looked at some really hammered systems and the SSD life % is still in the high 80-ies after a couple of years. Just looking at the basics and taking the absolute negative road.
1 DWPD over 3 years for a 1.92TB and 3.84TB SSD.
That's more than 2PB of data needed to be written to a 1.92TB and more than 4PB of data needed to be written to the 3.84TB SSD.
Over 7 years, that would yearly amount to about 300TB per 1.92TB SSD and 600TB for the 3.84TB SSD...
If my calculations are not completely off, that would be about a constant average of 10MB/s write for 1.92TB and 20MB/s for the 3.84TB for 7 years.
With 16kB blocks, thats 640 IOPS (for 1.92) and 1280 IOPS (for 3.84TB) of contant write per SSD in the system over 7 years.

And thats the absolutely most negative road with 3 years service life of 1 DWPD with 7 years until wear out and excludes all benefits of adaptive sparing. From what I've seen on hammered systems with the first gen 1.92TB SSDs, I'm really not worried about wear out on the SSDs.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: 8440 3PAR enclosure design with 216 SSD disks
PostPosted: Mon Aug 21, 2017 3:35 pm 

Joined: Wed Nov 19, 2014 5:14 am
Posts: 505
Agreed I did some calculations on this a while ago and you're very safe, in effect every time the capacity doubles on the drive so too does the projected life of the media at a constant level of throughput.
BTW that also goes for adding more SSD as with wide striping you're effectively offloading the existing media.

In order to hit the wear limits you would have to be performing very high and 100% sustained writes 24x7x365 over many years and that just isn't real world. In addition the drive manufacturer DWPD values don't take into account a lot of the technology the engineers continue to develop around wear leveling which only increases published numbers. e.g. the manufacturer published figure of 1 DWPD on any drive is meaningless in a 3PAR system as such the whole media life argument is pretty much a moot point these days.

From a purely practical perspective do you really think that any vendor would put such a no strings attached 5 year warranty with an option for 7 years with support if they really expected this to be an issue :-)


Last edited by JohnMH on Tue Aug 22, 2017 3:32 am, edited 3 times in total.

Top
 Profile  
Reply with quote  
 Post subject: Re: 8440 3PAR enclosure design with 216 SSD disks
PostPosted: Tue Aug 22, 2017 2:01 am 

Joined: Tue Jan 15, 2013 10:33 am
Posts: 41
Yesterday I thought a little about it. And my guess is Raid5 3+1 and Raid6 6+2 would require approximately the same backend IOPS and cpu performance. What do you think?

Thanks


Top
 Profile  
Reply with quote  
 Post subject: Re: 8440 3PAR enclosure design with 216 SSD disks
PostPosted: Tue Aug 22, 2017 3:25 am 

Joined: Wed Nov 19, 2014 5:14 am
Posts: 505
It depends on many factors such as read/write ratio, IO size, cache availability etc but in theory using update in place Raid 5 requires 4 backend I/O's for a write e.g.

1. Read old data
2. Read old parity
(calculate new parity)
3. Write new data
4. Write new parity

Raid 6 with an optimized layout requires 6 backend I/O's to account for the additional parity, however all of this assumes that this is direct IO to disk without the benefit or writeback cache and intelligent flushing which is designed to mask this process. In many cases the data will sit in cache until a full stripe is available and then be flushed and so reduce the number of I/O's required for a given amount of data. From a CPU perspective the parity calculations are handled by the ASIC, there may be a few more interrupts on the CPU but the difference should be negligible.

So it depends, in a heavily loaded system that is write heavy you will see a difference at max throughput, but the majority of systems I've seen tend to be biased more toward reads so the difference isn't huge. in reality the individual hosts are unlikely to see much of a difference but in the case of spinning disk you may need to add a few more disks, with SSD the controller becomes the bottleneck much sooner so having to do more backend IO means the array is likely to run out of bandwidth sooner but we're talking about running the system at sustained peak load which is unlikely and probably not advisable.


Top
 Profile  
Reply with quote  
 Post subject: Re: 8440 3PAR enclosure design with 216 SSD disks
PostPosted: Tue Aug 22, 2017 6:13 am 

Joined: Wed Nov 09, 2011 12:01 pm
Posts: 392
The rebuild times concerned me thou, according to the examples in that RAID6 whitepaper RAID6 disk rebuild is something like 4 x RAID5 on SSD. On top of the extra writes and slower performance throw in 4x rebuild time and it really doesn't sell RAID6 well.

For huge SSDs certainly adds some warm fuzziness but personally I've no intension of going above 3.8TB and 1.9TB is where I'd prefer to be at (once they can rebuild an 8TB in 2 hours I might change my mind). ;)


Top
 Profile  
Reply with quote  
 Post subject: Re: 8440 3PAR enclosure design with 216 SSD disks
PostPosted: Wed Aug 23, 2017 4:33 am 

Joined: Wed Nov 19, 2014 5:14 am
Posts: 505
Granted but even with slower rebuilds raid 6 is still many orders of magnitude more robust than raid 5, but as with most things it's a tradeoff. However with the abundant performance provided by flash media, you're typically in a much better position to accept the additional overhead.


Attachments:
raid.jpg
raid.jpg [ 51.67 KiB | Viewed 17121 times ]
Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 19 posts ]  Go to page Previous  1, 2


Who is online

Users browsing this forum: Google [Bot], Majestic-12 [Bot] and 46 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group | DVGFX2 by: Matt