Adaptive Optimization and full SSD tier

[skumflum]

I’ve just got this friendly E-mail from the 3PAR support engineers:

-------------------------
Please bear in mind that 3PAR HP Array S/N xxx issued the following notification:
“The system has used over 85% of the raw physical space. Available physical space required for auto-growth of logical capacity by CPGs with default LD characteristics may be at risk.”

Suggested actions:

- If these SSD drives are used for Adaptive Optimization (AO) then try to move the data back to NL and FC drives to gain more space on SSD drives.
- Compactcpg command should be used to trim the Volumes.
- Move some volumes to a CPG based on different type of drives (use: tunevv)
- Reclaim space by writing zeros to free areas of your TPVVs
- Make sure your TPVVs have zero detect policy enabled (use: showvv -pol to check)
- Delete old and unwanted snapshots as they use more and more space over time
- When Adaptive Optimization (AO) is used, data can be manually moved back to FC or NL
- Consider adding more capacity
---------------------------

From the technical white paper: Adaptive Optimization for HP 3PAR StoreServ Storage

"If the space in a tier exceeds the tier size (or the CPG warning limit), then the algorithm will first try to move regions out of that tier into any other tier with available space in an attempt to lower the tier’s size below the limit. If no other region has space, then the algorithm logs a warning and does nothing. (Note that if the warning limit for any CPG is exceeded, the array will generate an alert.) If space is available in a faster tier, it chooses the busiest regions to move to that tier. Similarly, if space is available in a slower tier, it chooses the most idle regions to move to that tier. The average tier service times and average tier access rates are ignored when data is being moved because the size limits of a tier have been exceeded"

And from the 3PAR Best Practices Guide:

“Best practice: If SSDs are used in Adaptive Optimization configurations, no thin provisioning volumes should be directly associated with SSD CPGs. The thin provisioning volumes should only be associated with FC CPGs. This will help ensure that SSD capacity is consumed by Adaptive Optimization and will allow this capacity to be safely used to 95 percent or even 100 percent.”

To my questions:
I use Adaptive Optimization and want to utilize all available SSD. I have NO TPVV on SSD.

1. Would the AO algorithm not take care of the moving? I can’t find anything in the documentation that suggest how to move data manual as the engineer suggested. [edit] yes dynamic optimization but this has little to do with AO [/edit]
2. Is it safe to continue? (i was sure until i got this email)
3. Whats with the 95% capacity? A golden rule?



Ps. (I have created some free space just in case)

[Richard Siemers]

I get the exact same alerts. I also have 0 VV's created in the SSD Tier (CPG).

I think this alert only applies if you have THIN provisioned volumes CREATED/Assigned to the SSD tier. Since you (nor I) do that, I believe we both can safely ignore that alert.

I had opened a ticket with HP requesting a filter or something similar be implemented to reduce the amount of pages I received, as well as cut down on the unnecessary ticketing and troubleshooting they do on their end. They successfully entered the following suppression filter into my service processor's notifications config to prevent my team from being paged directly, however, they still generate tickets on their end and send emails, but at a much less frequent interval.

Regex expression for filter:

Code: Select all

^Total SSD raw space

[skumflum]

Thanks

I was sure I understood correctly but this email caused me to doubt.
How long ago did you create the case? Maybe they have developed a more permanent solution.

[skumflum]

The answer from HP was as follows: “We know this is an issue but have no fix yet”. I just hope they don’t call a colleague and induce a panic attack

[Richard Siemers]

The case I created was 4646539843 from 10/31/2013. It did seam to work for a few months, as I do not see a new SSD Space% tickets logged until Dec 26th. Perhaps they did some system upgrades over Christmas that wiped out the filter they had put in place.

[Cleanur]

Assuming you have TP volumes created directly within the SSD CPG and are also running AO then potentially the two will be competing for space. There is nothing to stop AO promoting enough data that the TPVV could have an out of space condition when they attempt to grow (depends on the config). The warning is really there to prevent this out of space condition for those volumes.

Place a growth warning on the CPG to prevent AO from exceeding the 95% utilized and posting the
alert. AO should stop promoting data once it hits this threshold and should begin demoting data if you lower the threshold. But yes if you need to make wholesale changes to entire VV's then you would need to tune the volumes using DO,

You SHOULD also CHECK with SUPPORT whether "Setsys RawSpaceAlertSSD" would be of help in suppressing the alert (use at your own risk) assuming you have a large SSD tier and want to run at or near the edge.

AFAIK there is no golden rule but the array alerts automatically on specific thresholds, it's typically just common sense not to run most things in life at or very near 100% for long periods of time. If you decide to, then you need understand the potential implication and be willing to monitor things very carefully.

[Richard Siemers]

Thanks I will check into that setsys command. We have a pretty small SSD tier comparitively, about 1% of our total capacity... so we like to use absolutely as much of it as possible. Its frustrating enough that HP forced SSDs to have spare chunklets on them, I would have much rather preferred to use FC chunklets for SSD failures, fully understanding that when sparing occurs, the FC chunks will be slower than SSD.

This concept supports an/our AO policy of promoting hot FC chunks to SSD, and trying to use as much SSD as is available. I understand there are requirements that would justify hot sparing to SSD, but this is not one of them.

[Cleanur]

Check out the new adaptive sparing feature, sadly I don't think this can be backported to older SSD drives. http://www.techopsguys.com/tag/3par/

[skumflum]

I know traditional it is against common sense to run at 100% but SSD is premium price, and like Richard I want to use much of it as possible.

I noted in particular the following from 3PAR Best Practices Guide:

“Best practice: If SSDs are used in Adaptive Optimization configurations, no thin provisioning volumes should be directly associated with SSD CPGs. The thin provisioning volumes should only be associated with FC CPGs. This will help ensure that SSD capacity is consumed by Adaptive Optimization and will allow this capacity to be safely used to 95 percent or even 100 percent.”

I don't like the ambiguity in “….or even 100 percent”. Does this mean that one should be extra careful going from 95-100%?

[Cleanur]

If there are no TPVV's directly associated with the SSD tier, then all new writes will land in the FC tier. Overwrites will go to wherever the data currently resides so no chance of a volume being unable to grow as growth will occur in the FC CPG. It's just you will need to be aware and enforce that the SSD tier can only be used for AO and fully provisioned volumes, if you put TPVV's in there then you risk a out of space condition. However from memory the CPG will attempt to grow into another lower tier to avoid this issue but then you need to manually retune to fix.