3PAR Users Group

A Storage Administrator Community




Post new topic Reply to topic  [ 8 posts ] 
Author Message
 Post subject: 3PAR performance utilization rate - reffered to IOPS
PostPosted: Thu Apr 12, 2018 5:30 am 

Joined: Tue Mar 08, 2016 10:51 am
Posts: 6
We want to find out, how we are currently situated with the performance utilization of our 3PAR storages. I imagined, that for this, we have to compare the maximum possible IOPS number to the currently load of IOPS.

For the max. possible IOPS, I refer to the max front-end IOPS which is indicated by the NinjaSTARS for 3PAR Tool. We have three 3PAR Storages. Two identically configured StoreServ 7200 and one StoreServ 8200.

The StoreServ 8200 is provided with 6 x 1.92TB SSD SFF, 48x 1.2TB SAS 10k SFF and 24x 6+TB NL 7.2k LFF. With the configured R/W balance of 60% Read, Cache Hits of 75% Read (yes it’s really high, but I have verified it multiple times) and 5% for AFC, the Ninja Tool indicates me a number of max front-end IOPS of 90,583.
Attachment:
File comment: StoreServ 8200 - Ninja Tools
StoreServ8200.png
StoreServ8200.png [ 28.08 KiB | Viewed 854 times ]


Both StoreServ7200 have 4x 480GB SSD SFF (primary for AFC, but AFC does not need the whole space, so I have a small R5 volume too), 48x 600 SAS 10k SFF and 20x 1.2TB SAS 10k SFF. For these storages, the Ninja Tool (same settings as before) comes up with max front-end IOPS of 22,142. I used the 8200 model in the Ninja Tools for this configuration to, because in this version the 7200 model isn’t available and in the older version of the Ninja Tools it isn’t possible to configure only 4 SSD for AFC. I suppose due to the older controller model, the indicated IOPS number is probably a bit lower.
Attachment:
File comment: StoreServ 7200 - Ninja Tools
StoreServ7200.png
StoreServ7200.png [ 22.1 KiB | Viewed 854 times ]


So here come my questions:
Can I rely on the, from the Ninja Tool indicated number as max IOPS for my storages?
Which report do I have to use, to get the currently IOPS load of the whole storage array?

We are using also VMware vRealize Operations Manager with the HPE Storage Plug-in (former HPE StoreFront Analytics Management Pack), which monitors 3PAR data through the StoreServ SMI-S provider. This plugin gives me a pretty high number of Total IOPS. The only reports in the StoreServ Management Console, which shows me similar high values, are the Host Port Performance respectively the Enclosure Port Performance with all of the ports selected.
Is this the IOPS number I can compare to the max IOPS from the Ninja Tool?

Attachment:
File comment: vRealize Operations - Array Total IOPS
vrops_total_iops.png
vrops_total_iops.png [ 158.17 KiB | Viewed 854 times ]

Attachment:
File comment: SSMC Report - Enclosure Ports - Total IOPS
SSMC_enc_ports_total_iops.png
SSMC_enc_ports_total_iops.png [ 72.93 KiB | Viewed 854 times ]

Thank you for your time and help.


Last edited by m.rieder on Fri Apr 13, 2018 4:28 am, edited 1 time in total.

Top
 Profile  
Reply with quote  
 Post subject: Re: 3PAR performance utilization rate - reffered to IOPS
PostPosted: Thu Apr 12, 2018 7:46 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 623
Location: Europe
m.rieder wrote:
So here come my questions:
Can I rely on the, from the Ninja Tool indicated number as max IOPS for my storages?
Which report to I have to use, to get the currently IOPS load of the whole storage array?


TL;DR

From what I've been told Ninja Tool should be pretty accurate. Just keep in mind that higher IOsize = lower IOPS in many cases. If you are in a position where bandwidth your limiting factor, 2x IOsize = 1/2 IOPS. So your Ninja Tool output is for 8kB IOsize while your current IOsize seems to be around 20kB


Top
 Profile  
Reply with quote  
 Post subject: Re: 3PAR performance utilization rate - reffered to IOPS
PostPosted: Thu Apr 12, 2018 9:21 am 

Joined: Wed Nov 19, 2014 5:14 am
Posts: 494
It depends, the performance data in NinjaStars where possible is taken from tested data rather than simply modeled.

The max front end IOps figure is the cumulative number of each of those tiers running flat out based on the R/W ratio, IO size and response time dialed into the tool.

If you are tiering using AO then you'll get an average over the tiers based on capacity ratios and access density which will be less than the max FE IOps figure.

To get a similar output you'd want to look at the VLUN performance statistics as that represents Front End Performance as shown in NinjaStars.

If you're looking at headroom in the system a quick and dirty method would be to look at how saturated the backend disks are with statpd or a report from system reporter.


Top
 Profile  
Reply with quote  
 Post subject: Re: 3PAR performance utilization rate - reffered to IOPS
PostPosted: Thu Apr 12, 2018 9:34 pm 
Site Admin
User avatar

Joined: Tue Aug 18, 2009 10:35 pm
Posts: 1172
Location: Dallas, Texas
What are you trying to accomplish with this study? My guess is your want to migrate some more apps to the array and want to check if it will fit first?

For measuring how "saturated" a storage system is... I recommend looking at latency metrics too for an indication of stress. You can measure latency overall at the front end adapters, as well as in Vmware... and also at the backend disk level like JohnMH mentioned. I would break it into classes of backend reports: SSD latency, SAS latency, and NL latency.

_________________
Richard Siemers
The views and opinions expressed are my own and do not necessarily reflect those of my employer.


Top
 Profile  
Reply with quote  
 Post subject: Re: 3PAR performance utilization rate - reffered to IOPS
PostPosted: Fri Apr 13, 2018 7:57 am 

Joined: Tue Mar 08, 2016 10:51 am
Posts: 6
Thank you for your responses. I noticed, that I forgot to mention, that the graphical reports refer to our StoreServ 7200. For that one, the Ninja Tool indicates me the 22,000 max IOPS. Therefore, the peaks with values above 30,000 seem too high to me.

Richard Siemers wrote:
What are you trying to accomplish with this study? My guess is your want to migrate some more apps to the array and want to check if it will fit first?

My goal with this study is, that I want to have an appropriate argumentation basis, when I’m going to my boss to request an upgrade of the storage units. As we are a service hosting provider company, I can’t predict in advance the workloads coming with the next projects. And I don’t want to deal with the problems, after we receive customer complaints.
In general, I would like to better understand the significance of the indivdual performance reports to understand whether there are any bottlenecks.

I agree looking at the latencies is a good indicator how the storage is performing. However, I miss a reference of how much more load the storage can handle or rather I want to know on which point we are right now, concerning the workload saturation.

Please correct me if I’m wrong:
    The performance of the FC ports indicating the incoming IOPS from the connected server.
    The performance of the Disk ports indicating the IOPS send from the controller nodes down to the physical disks.

It makes no sense to me, to add these two numbers, but this is the value, which the Storage Plug-in is showing as the StoreServ total IOPS value. In my opinion, the total number of IOPS of all ports isn’t meaningful at all.
Does anyone else have experience with the HPE Storage Plug-in for vRealize Operations Manager?

JohnMH wrote:
To get a similar output you'd want to look at the VLUN performance statistics as that represents Front End Performance as shown in NinjaStars.

So I think the right way is to go with the Virtual Volume performance report as suggested by JohnMH.

Thank you


Top
 Profile  
Reply with quote  
 Post subject: Re: 3PAR performance utilization rate - reffered to IOPS
PostPosted: Fri Apr 13, 2018 11:39 am 

Joined: Wed Nov 19, 2014 5:14 am
Posts: 494
Just keep in mind that NinjaStars is a sizing tool only, it's numbers are conservative as it's designed to provide a quick idea of the available capacity and performance a given configuration could provide. Numbers in the real world with multiple concurrent workloads, cache hits, differing raid levels, block sizes and read write ratios can be much more complex and tend to fluctuate over time.

Quote:
The performance of the FC ports indicating the incoming IOps from the connected server.


Should give you a reasonable idea of the total front end IOps, incoming and outgoing for the entire system, but these likely include metadata as well as data transfers.

Quote:
The performance of the Disk ports indicating the IOps send from the controller nodes down to the physical disks.


These number are backend so will include raid and system overheads etc, which would not be particularly helpful unless you were trouble shooting a performance issue.

Statvlun will give you the frontend IOps and latency as experienced by the host, but unless you're seeing high latency it probably doesn't tell you anything about how heavily loaded the system is.

Hence using statpd, which will provide an idea of how heavily loaded the installed disks are and how much headroom is available, which will vary based on the current configuration and actual workload.

Unfortunately it's not an exact science :-)


Top
 Profile  
Reply with quote  
 Post subject: Re: 3PAR performance utilization rate - reffered to IOPS
PostPosted: Fri Apr 13, 2018 1:10 pm 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 623
Location: Europe
And not to forget ODX and VAAI which can create a lot of backend IOPS without any frontend.

If I'm not mistaken NinjaSTARS assume no cache hits (or I think you can set it). In your view it assumes random IO, sequential you can get a lot more (I've seen NL drives do 400 IOPS with doing large sequential IOs.

Depending on how your old 3PAR was procured I would go to you boss and say "This is the performance numbers we were quoted for the 3PAR when we bought it and we are now at <this and that> number. So with the current situation I can no longer guarantee the quality and performance of the storage service I'm delivering. Small changes on the load may tip us over the edge and into trouble."


Top
 Profile  
Reply with quote  
 Post subject: Re: 3PAR performance utilization rate - reffered to IOPS
PostPosted: Sat Apr 14, 2018 4:09 pm 
Site Admin
User avatar

Joined: Tue Aug 18, 2009 10:35 pm
Posts: 1172
Location: Dallas, Texas
Correct. you don't add the front end iops and back end iops together. You look at them separately, as well as the physical disk stats separately. Any 3 area can give you reason to justify an upgrade.

The ninjatool is calculating max iops based on the number and type of physical disks in the system.
There are general iops values for PDs used by storage vendors you can use to compare. Here is an example:

Device Type IOPS Interface
=========== ===== ================
7,200 rpm DISK ~75-100 IOPS
10,000 rpm DISK ~140 IOPS
15,000 rpm DISK ~175-210 IOPS
Value SSD ~4k - 6k IOPS
Enterprise SSD ~7k to 10K IOPS


For the purpose of evaluating storage performance to detect upgrade needs, I recommend analysing performance end-to-end, at the front end port, back end port, and at the PD level.

_________________
Richard Siemers
The views and opinions expressed are my own and do not necessarily reflect those of my employer.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 


Who is online

Users browsing this forum: Google [Bot] and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group | DVGFX2 by: Matt