HPE Storage Users Group
https://3parug.com/

3par 7200 3.2.1 MU3 OS upgrade problem
https://3parug.com/viewtopic.php?f=18&t=2640
Page 1 of 1

Author:  Miho etf [ Wed Sep 06, 2017 2:22 am ]
Post subject:  3par 7200 3.2.1 MU3 OS upgrade problem

Hello community

I have problems trying to upgrade 3par 7200 OS 3.2.1 MU3 to 3.2.2 MU4 through SPOOC

The upgrade started normally and it looks like node0 upgraded, but after that the upgrade failed

(Rebooting node 0.

Error The cache server is waiting for the system manager to finish reporting the system configuration and status. Please try again later.
The upgrade has failed. Please view the messages above and contact HP Customer Support (Error Code 1))


Than I tried servicemag status command, and the output was :

{System manager is still initializing - try this command later.
Use 'showsysmgr' to see the current system state.} 0 0 0
cli% showsysmgr
Cannot reach TOC quorum. Use 'setsysmgr' to set system manager startup state.

Reason for being unable to automatically start:
TOC information preserved in powerfail save on node 0 but missing nodes from mask 0x03. Run "setsysmgr tocgen" to start despite this.

Than I ran setsysmgr tocgen and the system was started up but shownode command only shows node0.

This is upgrade status output:

upgradesys -status

Node 0: 3.2.1.MU3 (old)
Node 1: Unknown (rebooting)
Next node to be rebooted is node 0
3PAR cli

We tried to reseat node1 and halt whole 3par controller enclosure waiting for batteries to discharge, and after that the problem was still there

The storage system does not have production data on itself

Any iea wht could be wrong?

Thanks

Author:  mathiasz [ Wed Sep 06, 2017 7:37 am ]
Post subject:  Re: 3par 7200 3.2.1 MU3 OS upgrade problem

Hello,
I would recommend to connect with serial line to the node.
In the past I had an upgrade where the internal Card with boot image was dead....
Mathias

Author:  Miho etf [ Thu Sep 07, 2017 8:59 am ]
Post subject:  Re: 3par 7200 3.2.1 MU3 OS upgrade problem

Thank you Mathias for your answer, I have connected with serial cable tothe problematic node, and the checkhealth output is:

1. Out Of The Box Procedure
2. Re-enter network configuration
3. Update the CBIOS
4. Enable or disable CLI error injections
5. Perform a Node-to-Node rescue
6. Set up the system to wipe and rerun ootb
7. Cancel a wipe
8. Perform a deinstallation
9. Update the system for recently added hardware (admithw)
10. Check system health (checkhealth)
11. Exit
> 10
Could not open connection to CLI server process tpdtcl, it has not yet
started or has failed

Author:  mathiasz [ Wed Sep 13, 2017 6:11 am ]
Post subject:  Re: 3par 7200 3.2.1 MU3 OS upgrade problem

Hello Miho,

Hmmm....
For me it sounds like calling HP...
If it is not a productive 3par and has no support I can imagine that node-to-node rescue can be an option. May be...
Last post here: http://3parug.com/viewtopic.php?f=18&t=1554 sounds also interesting. Maybe a clean boot disk is the way....
But my preference is calling HP....
Good luck

Mathias

Author:  aberus [ Mon Sep 18, 2017 10:13 am ]
Post subject:  Re: 3par 7200 3.2.1 MU3 OS upgrade problem

I highly recommend that you call support, with that being said I have run into this issue twice, once on a 7200 and the other time on a 7400 and I had to perform a node rescue. If you choose to proceed with this on your own here are the procedures I followed:

• Connect to good node in NP Node via the serial cable
• Verify active network is on the good node in the NP (shownet) 
• If other node is the active then issue the setnet changenode <#>  
• Verify Node that you want to rescue has a network cable plugged into mgmt port
• Shutdown Node that needs rescued (shutdownnode halt #) 
• Verify Node is down with blue led 
• Start Node Rescue (startnoderescue -node #) 
• Move serial cable over to Node and watch rescue operation
• Once complete and at login prompt, move cable back over to master Node
• Verify rescued Node is in cluster (shownode) 
• Verify there are no space issue (checkhealth -svc -detail file) 

Please note that on the 7000 series arrays the node rescue is performed across the network interface.

Author:  Miho etf [ Tue Oct 03, 2017 7:52 am ]
Post subject:  Re: 3par 7200 3.2.1 MU3 OS upgrade problem

Thank you all for your advices and contribution,
We solved the problem by replacing problematic node,
Although we still do not know what is the problem with old one (LEDs and logs look OK, but it just does not work-it can not join cluster, and we tried a lot of things)
I suppose the problem is with node ssd, I will try to replace it with new one and see what happens
when I put the node in the storage and try to copy the OS from the good one

Thanks

Page 1 of 1 All times are UTC - 5 hours
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/