3PAR Users Group

A Storage Administrator Community




Post new topic Reply to topic  [ 4 posts ] 
Author Message
 Post subject: Is running system tune anywhere riskful?
PostPosted: Thu Jul 11, 2019 6:20 am 

Joined: Thu Jul 11, 2019 4:54 am
Posts: 1
Due to organizational issues I've inherited some 3PAR systems (8400/7400/7200) running production workloads for lots of customer. Fairly experienced in IT, I do not have a lot of experience with system administration of 3PAR systems itself. So, sorry for being noob in such an experienced User Group, I really hope anyone can answer some questions.

Recently I've expanded storage capacity of one of the 3PAR systems by adding 8x SSD in a 2 node system. There were already 20 SSD in each prior to the expansion.

Last week we were notified that there are latency issues on some parts of our virtualized environment (vSphere 5.5). Further investigation lead into a hypothesis that this could be due to unevenly distributed requests on the storage layer. Because the older drives are stuffed over 97% most of the new (write) requests will pass to only the new drives (8x).

Reading this forum and sites like https://d8tadude.com/2014/06/06/disk-load-and-tunesys/ hint me in doing a system tune, so data will be evenly balanced across the system. To add to the hypothesis: this tune will balanced data evenly, so requests will be routed more spread than the current high density on the new drives. This would lead to less latency problems.

That for a situational overview given I have to following questions:
  • Is there any risk for my production workload in running a system tune?
  • Can anyone confirm or correct my hypothesis with arguments, so I can learn something about the inner working?


Top
 Profile  
Reply with quote  
 Post subject: Re: Is running system tune anywhere riskful?
PostPosted: Thu Jul 11, 2019 6:43 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 931
Location: Europe
acropia wrote:
That for a situational overview given I have to following questions:
  • Is there any risk for my production workload in running a system tune?
  • Can anyone confirm or correct my hypothesis with arguments, so I can learn something about the inner working?


With tunesys you will read and re-write data back to the system for better distribution. Worst thing that could happen is that this added load puts the array over the top. If that happens you could just cancel tunesys task and it will stop moving data within a few minutes as the child tasks finish.

Given that the hardware is balanced, you are 100% correct with your hypothesis.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Is running system tune anywhere riskful?
PostPosted: Thu Jul 11, 2019 10:23 pm 

Joined: Mon Jul 03, 2017 9:05 pm
Posts: 20
Currently am also running tunesys in my environment with the help of HPE after adding some new SSD disks to our arrays and below are some of the key points to be remembered regarding tunesys.

Data Rebalance (tunesys)
The tunesys command is used to analyze and detect disk utilization across an entire storage system. The command runs a series of low level operations to re-balance resources on the system.
Syntax : ‘tunesys’ – This command would run as a background task & would initiate a task id which can be monitored by ‘showtask –d <tasknumber>’

NOTE:
a. Tunesys does a complete analysis of the system and makes all attempts to redistribute data chunklets evenly on all the PD’s.

b. This is programmed to run as a low priority task and duration of run time cannot be determined due to its dynamic nature and also depends on the load from time to time.
Based on past experiences it has been noted that tunesys runs for multiple days to week. In my current case, it is nearly a month and still not finished.

c. To rebalance data, tunesys requires to be run multiple times to balance the data completely.

d. All scheduled task will be suspended and resumed when tunesys is completed.

e. Dynamic Optimization license is required for all OS versions of 3.1.1 and below for tunesys/tunevv.

f. Dynamic Optimization license is required for all OS version 3.1.2 above to tune a volume (tunevv).

g. During the execution of tunesys if system identifies no tunes are required then we can perform alternate/additional option to tune the system depending on our arrays current configuration.
Note: We may also need to create a new CPG and tuneall the volumes to new CPG to distribute data evenly.

Recently I added new SSD disks and as part of that i am running now tunesys and it is noticed many times that SSD disk utilization touched 100% many times in between and this created lot of issue for me as client raised serious concerns that our array is going to crash.

But, It is the expected behavior of tunesys to make use of the free space to re-layout the data and later release the space. It does it several times, hence we are seeing 100% utilization of SSD disks in our environment.

Post tunevv, tunesys will trigger a compactcpg task automatically, which will release the space also. So if your disks shows 100% utilized, then you can manually cancel the tunesys and ran compactCPG manually too which I do often to make customer happy.


Top
 Profile  
Reply with quote  
 Post subject: Re: Is running system tune anywhere riskful?
PostPosted: Fri Jul 26, 2019 4:20 pm 

Joined: Thu Dec 04, 2014 2:12 am
Posts: 3
@Acropia, being fairly noob at administering 3PAR systems and serving a lot of customers doesn’t seem like a winning combination. Running tunesys after adding new disks is a fairly common task but should indeed be used with caution.

It might be a serious consideration to hire a 3PAR specialist to do these kind of tasks for you if you’re not up for it yourself. 3PAR is a very good and solid enterprise storage system but in no means plug & play.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 


Who is online

Users browsing this forum: No registered users and 32 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group | DVGFX2 by: Matt