3PAR Users Group

A Storage Administrator Community




Post new topic Reply to topic  [ 3 posts ] 
Author Message
 Post subject: Adaptive Flash Cache problems on 7400 with 3.2.2 (MU6)
PostPosted: Mon Apr 29, 2019 11:25 am 

Joined: Wed Jan 16, 2019 4:26 pm
Posts: 23
Location: Florida
I've posted this over on the HPE Community but one of their moderators moved it to the wrong forum where it's unlikely to get much attention.

We've recently turned on Adaptive Flash Cache for the first time. Now we're seeing two errors; the first is from Infosite: "High slab space usage detected." The second is in Events & Alerts: "Flash Cache I/O times are slow on node: 3"

I've looked at some performance reports in SSMC. Looking at "user time", node 3 has been busier than the others for days/weeks. However, when we enabled Flash Cache a few days ago the "system time" metric for node 3 spiked above the other nodes and addition to user time.

Looking at Physical Drive Performance, I see that the SSDs do have some IOPs spikes, with write latency spiking to 22ms during one stat sample. All 16 SSD drives chart identically.

The report "Adaptive Flash Cache Performance" shows all four nodes charting pretty much identically. Flash Cache used percentage peaked at 400% roughly 20 hours after enablement but it's now slowly wobbling up and down between 300 to 400%. I would expect the cache to remain full once filled. :?:

There are no other workloads on these SSDs (no CPGs).

Any thoughts?


Top
 Profile  
Reply with quote  
 Post subject: Re: Adaptive Flash Cache problems on 7400 with 3.2.2 (MU6)
PostPosted: Fri May 03, 2019 9:38 pm 
User avatar

Joined: Fri Jun 21, 2013 1:03 pm
Posts: 106
Location: USA
When assigning space for AFC, the high slab will show up if you don't leave a certain percentage space free. AFC does have a hit rate command, so you can see some statistics for how well it's picking up the load.

Write latency generally should be very low, there are some cpu background tasks that we've seen spike up the latency every once and awhile but not to the point of any real concern.

AFC only needs a few SSDs on the 7000 series (no need for 16 SSDs to be assigned). Play around with the space allocations a bit till you find what's best for your environment. On one setup with a 10/45/45 mix of SSD/10k/7k AFC started to plateau around 120GB. I saw better overall performance and then setup a small flash tier for some of the heavy hit databases.


Top
 Profile  
Reply with quote  
 Post subject: Re: Adaptive Flash Cache problems on 7400 with 3.2.2 (MU6)
PostPosted: Fri May 10, 2019 9:04 am 

Joined: Wed Jan 16, 2019 4:26 pm
Posts: 23
Location: Florida
Thanks. My post finally got HP's attention and they said the high swap usage and high slab usage alerts in Infosight were cosmetic (I'm paraphrasing) and not to worry about them.

I did cut the AFC size from 768GB to 128GB. The hit rate is just as high and the alerts went away. I suspect 3PAR's caching implementation is like many others in that it uses RAM to track the blocks in L2 cache and dropping the size freed up a bunch of RAM.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 


Who is online

Users browsing this forum: Google [Bot] and 23 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group | DVGFX2 by: Matt