HPE Storage Users Group

A Storage Administrator Community




Post new topic Reply to topic  [ 11 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6
PostPosted: Tue May 18, 2021 11:17 am 

Joined: Tue May 18, 2021 10:57 am
Posts: 7
Hi guys!
Help me to solve problem with update. Previously I never used any data storage systems.
I have 3par 7400 INFORM OS 3.2.1.426 (MU5) patch P49,P53,P55.
I installed Physical SP-4.4.0.GA-142 (MU9) for update to INFORM OS 3.2.2.709 (MU6).
First try:
Code:
No InSplore running
Terminated InSplores
No InSplore running
Release version 3.2.1.426 (MU5)
Patches  P49,P53,P55

Component Name                   Version                         
CLI Server                       3.2.1.483 (P53)                 
CLI Client                       3.2.1.483                       
System Manager                   3.2.1.488 (P55)                 
Kernel                           3.2.1.426 (MU5)                 
TPD Kernel Code                  3.2.1.426 (MU5)                 
TPD Kernel Patch                 3.2.1.488 (P55)                 
Selected upgrade type  simple

MESSAGE START State, No HPE 3PAR OS upgrade in progress
STATE0
 
StoreServ release version 3.2.1.426 + P49,P53,P55
Beginning upgrade checks
-- Test Name --------------------------------------------------------  RESULT  -
Check Defrag                                                           PASSED 
Check Local                                                            PASSED 
Check PD                                                               PASSED 
Check Upgrade                                                          PASSED 
Check Upgrade Offline                                                  PASSED 
Check VV                                                               PASSED 
Check Cluster Name Exists                                              PASSED 


3par001 is in an acceptable state to perform a software upgrade
   No InSplore currently running
Stopping spevent task for StoreServ 3par001
Stopping spcollect tasks for StoreServ 3par001
If running, stopping InSplore for StoreServ 3par001
 
 
Transferring software may be a long running operation. Do not abort the operation.
 
Installing update 3.2.2.709-MU6
Validating 3.2.2.709-MU6
Enter Simple Upgrade state(0)
auto doupdate OS-3.2.2.709-MU6
substituting package for staged
substituting package for staged
Beginning upgrade of system software.
Extracting...
-/-/-/-/-/-/.-/-/-/-/-/-/.-/-/-               Applying new package...
Calculating checksums...
-/.-/   Performing pre-update checks...
-  Following PD(s) exceeds the chunklet failure threshold and needs corrective action 5,6,7,2
Update failed.
System is not currently undergoing an online upgrade.

System is not currently undergoing an online upgrade.
First node to be rebooted is node 1
   Which will be followed by node 0
The upgrade has failed. Please view the messages above and contact HPE Customer Support (Error Code 1)
Password Processing Done


Second try:
Code:
No InSplore running
Terminated InSplores
No InSplore running
Release version 3.2.1.426 (MU5)
Patches  P49,P53,P55

Component Name                   Version                         
CLI Server                       3.2.1.483 (P53)                 
CLI Client                       3.2.1.483                       
System Manager                   3.2.1.488 (P55)                 
Kernel                           3.2.1.426 (MU5)                 
TPD Kernel Code                  3.2.1.426 (MU5)                 
TPD Kernel Patch                 3.2.1.488 (P55)                 
Selected upgrade type  simple

MESSAGE START State, No HPE 3PAR OS upgrade in progress
STATE0
 
StoreServ release version 3.2.1.426 + P49,P53,P55
Beginning upgrade checks
-- Test Name --------------------------------------------------------  RESULT  -
Check Defrag                                                           PASSED 
Check Local                                                            PASSED 
Check PD                                                               PASSED 
Check Upgrade                                                          PASSED 
Check Upgrade Offline                                                  PASSED 
Check VV                                                               PASSED 
Check Cluster Name Exists                                              PASSED 


3par001 is in an acceptable state to perform a software upgrade
   No InSplore currently running
Stopping spevent task for StoreServ 3par001
Stopping spcollect tasks for StoreServ 3par001
If running, stopping InSplore for StoreServ 3par001
 
 
Transferring software may be a long running operation. Do not abort the operation.
 
Installing update 3.2.2.709-MU6
Validating 3.2.2.709-MU6
Enter Simple Upgrade state(0)
auto doupdate OS-3.2.2.709-MU6


System is not currently undergoing an online upgrade.
First node to be rebooted is node 1
   Which will be followed by node 0
The upgrade has failed. Please view the messages above and contact HPE Customer Support (Error Code 1)
Password Processing Done
 
system-wide certificate already exists
 
Beginning post upgrade checks
HPE 3PAR OS version(altroot) = 3.2.1.292+P17,P18,P19,P23,P30,P50,P52, rc(0)
-- Test Name --------------------------------------------------------  RESULT  -
Check Defrag                                                           PASSED 
Health Check                                                           PASSED 


Post upgrade checks are passed but software upgrade failed on 3par001 due to incompatible state
Starting spcollect tasks for StoreServ 3par001
Starting spevent task for StoreServ 3par001
 
There was an issue upgrading your StoreServ. Please contact HPE Customer Service with the output
from the above operation to resolve this problem


Top
 Profile  
Reply with quote  
 Post subject: Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6
PostPosted: Wed May 19, 2021 3:39 am 

Joined: Mon Apr 14, 2014 9:07 am
Posts: 189
what do you get from a checkhealth -svc -detail

The first screenshot suggests you have 4 physical disks with issues. I'm assuming they're not actually failed disks? Might need to fix those disks first.

maybe try a showpd -degraded -failed

could also try a showpd -c and it will show you how many failed chunklets on all disks.


Top
 Profile  
Reply with quote  
 Post subject: Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6
PostPosted: Wed May 19, 2021 4:49 am 

Joined: Tue May 18, 2021 10:57 am
Posts: 7
checkhealth -svc -detail:
Code:
Checking alert
Checking ao
Checking cabling
Checking cage
Checking cert
Checking dar
Checking date
Checking file
Checking fs
Checking host
Checking ld
Checking license
Checking network
Checking node
Checking pd
Checking pdch
Checking port
Checking qos
Checking rc
Checking snmp
Checking task
Checking vlun
Checking vv
Checking sp
Component -------------------Description-------------------- Qty
Alert     New alerts                                           2
Node      Filesystems which are over 90% full                  1
Node      Filesystems which are over 80% full                  1
Host      Host ports not configured for virtual port support   1
License   Licenses which have expired                          1
pdch      LDs with chunklets on a remote disk                 14
RC        Structure Incompatibility                            1
vlun      Hosts not connected to a port                        2

Component ---Identifier--- -------------------------------------------------------------Description--------------------------------------------------------------
Alert     sw_os            An Update is Available                                                                                                               
Alert     sw_os            Update OS-3.2.2.709-MU6 has failed:   Following PD(s) exceeds the chunklet failure threshold and needs corrective action: 5,6,7,2-----
Node      node:0           Filesystem /dev/sda7 mounted on "/common" is over 90% full                                                                           
Node      node:1           Filesystem /dev/sda7 mounted on "/common" is over 80% full                                                                           
Host      Port:1:1:1       Port WWN not found on FC Fabric attached to Port:0:1:1                                                                               
License   Peer Motion      License has expired                                                                                                                   
pdch      ld:worker0.usr.3 LD has 2 remote chunklets                                                                                                             
pdch      ld:worker0.usr.1 LD has 3 remote chunklets                                                                                                             
pdch      ld:admin.usr.2   LD has 2 remote chunklets                                                                                                             
pdch      ld:master0.usr.5 LD has 16 remote chunklets                                                                                                           
pdch      ld:master1.usr.5 LD has 16 remote chunklets                                                                                                           
pdch      ld:.srdata.usr.1 LD has 13 remote chunklets                                                                                                           
pdch      ld:master2.usr.5 LD has 16 remote chunklets                                                                                                           
pdch      ld:master0.usr.3 LD has 2 remote chunklets                                                                                                             
pdch      ld:worker0.usr.5 LD has 16 remote chunklets                                                                                                           
pdch      ld:master0.usr.1 LD has 3 remote chunklets                                                                                                             
pdch      ld:master1.usr.1 LD has 4 remote chunklets                                                                                                             
pdch      ld:master1.usr.3 LD has 2 remote chunklets                                                                                                             
pdch      ld:master2.usr.3 LD has 2 remote chunklets                                                                                                             
pdch      ld:master2.usr.1 LD has 4 remote chunklets                                                                                                             
RC        Structure        Remote Copy internal structure is incompatible.                                                                                       
vlun      host:master0     Host iscsi_name:iqn.1994-05.com.redhat:3443780cbb is not connected to a port                                                         
vlun      host:master0     Host iscsi_name:iqn.1994-05.com.redhat:bfbbd4d9d631 is not connected to a port


showpd -degraded -failed:
Code:
No PDs listed


showpd -c:
Code:
                             ------- Normal Chunklets -------- ---- Spare Chunklets ----
                             - Used - -------- Unused -------- - Used - ---- Unused ----
Id CagePos Type State  Total  OK Fail Free Uninit Unavail Fail OK  Fail Free Uninit Fail
 0 0:0:0   SSD  normal  1787 120    0 1220      0       0    0  0     0  447      0    0
 1 0:1:0   SSD  normal  1787 159    0 1181      0       0    0  0     0  447      0    0
 2 0:2:0   SSD  normal  1787   0    0 1333      0       0    7  0     0  447      0    0
 3 0:3:0   SSD  normal  1787 131    0 1204      0       0    5  0     0  447      0    0
 4 0:4:0   SSD  normal  1787 120    0 1215      0       0    5  0     0  447      0    0
 5 0:5:0   SSD  normal  1787 131    0 1202      0       0    7  0     0  447      0    0
 6 0:6:0   SSD  normal  1787 119    0 1215      0       0    7  0     0  446      0    0
 7 0:7:0   SSD  normal  1787 167    0 1167      0       0    7  0     0  446      0    0
----------------------------------------------------------------------------------------
 8 total               14296 947    0 9737      0       0   38  0     0 3574      0    0


Top
 Profile  
Reply with quote  
 Post subject: Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6
PostPosted: Wed May 19, 2021 4:57 am 

Joined: Tue May 18, 2021 10:57 am
Posts: 7
I tryed fix failed chunklets on disk 2 in the following way:
1. connected disk to server
2. reformatted to 512 block size
3. blkdiscard
4. reformatted to 520 block size
5. returned disk to 3par

But this did not solve the problem with failed chunklets.


Top
 Profile  
Reply with quote  
 Post subject: Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6
PostPosted: Wed May 19, 2021 7:54 am 

Joined: Mon Apr 14, 2014 9:07 am
Posts: 189
so at least you can see the reason the upgrade failed is those PD's as per
Alert sw_os Update OS-3.2.2.709-MU6 has failed: Following PD(s) exceeds the chunklet failure threshold and needs corrective action: 5,6,7,2-----

Now I've never thought about taking a disk out of 3par and connecting it to a server and formatting it. But i know the 3pars have a database of failed disks, my worry is now you've removed it I think its failed and has now marked that disk as bad. Because in the last screen shot, i don't like the OK column shows 0 on disk 2.

I'm assuming this 3par isn't under support anymore? and that's why you haven't asked HPE to swap the disk.

run another showpd -c and see if the details on disk 2 have changed anymore? i don't know if it has to rescan the chunklets again.


Top
 Profile  
Reply with quote  
 Post subject: Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6
PostPosted: Wed May 19, 2021 9:56 am 

Joined: Tue May 18, 2021 10:57 am
Posts: 7
Yes, unfortunately there is no support for this storage.
Before disconnecting disk 2, I manually transferred all used chunklets, but after returning disk 2, they apparently did not come back automatically, so the value in the OK column is 0.
`showpd -c`:
Code:
                             ------- Normal Chunklets -------- ---- Spare Chunklets ----
                             - Used - -------- Unused -------- - Used - ---- Unused ----
Id CagePos Type State  Total  OK Fail Free Uninit Unavail Fail OK  Fail Free Uninit Fail
 0 0:0:0   SSD  normal  1787 120    0 1220      0       0    0  0     0  447      0    0
 1 0:1:0   SSD  normal  1787 159    0 1181      0       0    0  0     0  447      0    0
 2 0:2:0   SSD  normal  1787   0    0 1333      0       0    7  0     0  447      0    0
 3 0:3:0   SSD  normal  1787 131    0 1204      0       0    5  0     0  447      0    0
 4 0:4:0   SSD  normal  1787 120    0 1215      0       0    5  0     0  447      0    0
 5 0:5:0   SSD  normal  1787 131    0 1202      0       0    7  0     0  447      0    0
 6 0:6:0   SSD  normal  1787 119    0 1215      0       0    7  0     0  446      0    0
 7 0:7:0   SSD  normal  1787 167    0 1167      0       0    7  0     0  446      0    0
----------------------------------------------------------------------------------------
 8 total               14296 947    0 9737      0       0   38  0     0 3574      0    0


Top
 Profile  
Reply with quote  
 Post subject: Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6
PostPosted: Wed May 19, 2021 10:00 am 

Joined: Tue May 18, 2021 10:57 am
Posts: 7
Maybe I can somehow indicate not to use these failed chunklets or raise this threshold, for example, to 10?


Top
 Profile  
Reply with quote  
 Post subject: Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6
PostPosted: Thu May 20, 2021 7:03 am 

Joined: Tue May 18, 2021 10:57 am
Posts: 7
I recently found out that I can unmap failed chunklets and update 3parOS. Only it is not clear how to do it.


Top
 Profile  
Reply with quote  
 Post subject: Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6
PostPosted: Thu May 20, 2021 8:38 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
Are you using the upgrade tool?

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: Update failed 3par OS 3.2.1MU5 to 3.2.2MU6
PostPosted: Thu May 20, 2021 9:16 am 

Joined: Tue May 18, 2021 10:57 am
Posts: 7
I tried update via Service Processor (WebUI).


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 11 posts ]  Go to page 1, 2  Next


Who is online

Users browsing this forum: No registered users and 38 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group | DVGFX2 by: Matt