HPE Storage Users Group

A Storage Administrator Community




Post new topic Reply to topic  [ 9 posts ] 
Author Message
 Post subject: Determine which host uses port bandwith?
PostPosted: Wed Nov 15, 2017 3:29 am 

Joined: Wed May 07, 2014 1:51 am
Posts: 267
Hi all,

I'm trying to find ouch which of our hosts isn't doing proper round robin, but stresses one of the four ports it's zoned to. I can't find the link between hosts and ports - the "ports compare by" - reports don't show any host specific data (no group_by hosts possible). srstathost is not implemented, and srstatport does not give me the info I want as well.

Am I missing something?

I know I could try checking all hosts...

_________________
When all else fails, read the instructions.


Top
 Profile  
Reply with quote  
 Post subject: Re: Determine which host uses port bandwith?
PostPosted: Wed Nov 15, 2017 3:57 am 

Joined: Wed May 07, 2014 1:51 am
Posts: 267
OK, found it, but it's not very comfortable:

Go to hosts in SSMC
select first host that uses the port you want to examine
select "Performance"
klick at point in time that you want to analyze in the iops-graph
change "groupby" from "Host name" to "Port node", "port slot" and "port number"
press "close"
sigh, swear
repeat all of the above, this time klick "apply"
look if this host is doing proper round robin (all ports more or less equally used)

repeat for all hosts that use the port.

If anybody knows a smarter way of doing this: any help would be highly appreciated!

_________________
When all else fails, read the instructions.


Top
 Profile  
Reply with quote  
 Post subject: Re: Determine which host uses port bandwith?
PostPosted: Wed Nov 15, 2017 4:56 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
statvlun will show performance per path.

I really like statvlun -hostsum when looking for which hosts are most active and statvlun -vvsum when looking for the most active VVs.

For your use case I would probably do "statvlun -sortcol 1 -d 30 -iter 1" which will give you the average per path over 30 seconds and sorted by VV name. 30 seconds should be enough to show imbalanced paths.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Determine which host uses port bandwith?
PostPosted: Wed Nov 15, 2017 6:42 am 

Joined: Wed May 07, 2014 1:51 am
Posts: 267
MammaGutt, thank you for you input. But I'm not looking for an imbalanced port, I already have one and try to find the hosts that are responsible for it.

I used srstatvlun -vv set:xxx -attime -groupby Potr_n... to find which cluster is responsible for the imbalance, now I want to see which of the clusters hosts it is!

_________________
When all else fails, read the instructions.


Top
 Profile  
Reply with quote  
 Post subject: Re: Determine which host uses port bandwith?
PostPosted: Wed Nov 15, 2017 7:57 am 

Joined: Fri Nov 25, 2016 7:07 am
Posts: 4
statport will show you load on FC ports, this will enable you to see which port is loaded more than the others.


statvlun -hostsum (host summary), this might be easier way to indication which host is top polluter. Will not tell you on what port that host is creating traffic, but you will use the combined information to identify where round-robin is missing.



statvlun is most likely what you are looking for : it displays DETAILED breakdown of "vv per host per port" communication statistics, but be prepared to get some LENGHTY output which is not easy to process. Every single LUN, accessed by every single host by every single path...

You can shrink down that output by limiting only the "offending port" you identified by using "ports" - ie. if you know you have heavy traffic on x:y:1, you don't need to display anything else. Use -ports 1 for statvlun and it will only display ALL ports X:Y:1



Example of my unbalanced configuration :
http://imageshost.eu/images/2017/11/15/3par.png

by using statport, I caught the problem to be on port 3:5:2 - this is not displayed here. By using statvlun with "-ports 3" I demonstrate there is not too much traffic on X:Y:3 ports. And there is brutal difference with output of command where I used "-ports 2" showing tremendous traffic on ports X:Y:2


Because I used HOSTSUM switch, it's very very easy to instantly identify host (actually two or three) being responsible for majority of traffic.

Apologies, I had to scramble the names of hosts due to company security policies.

As you see on screenshot, out of total 563117 kB/s two hosts (name ending with 02 and 06) were responsible for 264227 + 165724 = 429951 kB/s ; that is 76% of all traffic was generated by two hosts only. If I add the third largest consumer, that would be additional 80686 kB/s which is pushing me to 90%... three hosts responsible for 90% of all traffic on single port !

Sure, this is short-term data only, but you get the idea...


Top
 Profile  
Reply with quote  
 Post subject: Re: Determine which host uses port bandwith?
PostPosted: Wed Nov 15, 2017 8:42 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
apol wrote:
MammaGutt, thank you for you input. But I'm not looking for an imbalanced port, I already have one and try to find the hosts that are responsible for it.

I used srstatvlun -vv set:xxx -attime -groupby Potr_n... to find which cluster is responsible for the imbalance, now I want to see which of the clusters hosts it is!

I did not use the word "port" a single time in my post :)

statport will give you imbalanced PORT
statvlun will give you imbalanced PATH

As you probably know, a path is a 1:1 relationship between one host port on the 3PAR and one port on the HBA of the server. It was my impression that it was exactly this you were looking for.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Determine which host uses port bandwith?
PostPosted: Wed Nov 15, 2017 8:57 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
Just tested my command. If you do a short change to the sortcol parameter it will also sort by host and port in case you have the same volumes exported to many hosts over many paths.

statvlun -sortcol 1:2:3 -d 30 -iter 1


Statvlun does allow filtering on vv set, but if you know the exact vv you could add it like this:

statvlun -sortcol 1:2:3 -d 30 -iter 1 -v <vvname>

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Determine which host uses port bandwith?
PostPosted: Mon Nov 20, 2017 9:01 am 

Joined: Wed May 07, 2014 1:51 am
Posts: 267
Thank you all for your input!

I was so used to always use -hostsum or -vvsum after srstatvlun that I didn't realize that it does exactly what I was looking for :)

I wrote a script that issues

Code:
srstatvlun -vv <pattern to match cluster-vvs>  -host <hostname> -port <one of four ports a cluster uses>

for each path for each host. Result is a csv-File for each host, containing the total iops per path for the given host.

_________________
When all else fails, read the instructions.


Top
 Profile  
Reply with quote  
 Post subject: Re: Determine which host uses port bandwith?
PostPosted: Tue Nov 21, 2017 12:36 am 
Site Admin
User avatar

Joined: Tue Aug 18, 2009 10:35 pm
Posts: 1328
Location: Dallas, Texas
This used to be easier with off-array System Reporter. You could show front end port iops 8 or 16 ports at a time and look for any lines that were running solo and not working as part of a team. You could also view all ports and filter it by host and you could quickly see which ports a host was zoned to and if it was properly balanced.

I saw a "balance" report at one point on a version of SSMC, but it was unclear what it did. I am currently in between 3PARs and don't have access to one to check if there is still a balance report.

Purestorage has a simple but effective command line "purehost monitor --balance" that will watch traffic on a host for an interval and report back if the IO was balanced or not. Which is similiar to a "srsstatvlun -hostsum" with some extra code tacked on to evaluate the data to see if it's reasonably balanced.

_________________
Richard Siemers
The views and opinions expressed are my own and do not necessarily reflect those of my employer.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 9 posts ] 


Who is online

Users browsing this forum: Google [Bot] and 43 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group | DVGFX2 by: Matt