HPE Storage Users Group

A Storage Administrator Community




Post new topic Reply to topic  [ 9 posts ] 
Author Message
 Post subject: 3par 7200 double node failure after moving to new rack
PostPosted: Tue Nov 12, 2019 11:49 am 

Joined: Thu Jun 13, 2019 7:12 pm
Posts: 11
I re-racked a 3par 7200 we had shipped up from our DR datacenter to use for test/training environment.

All data cables match 3par documentation and are configured the same as pre-unracking DR config, documented with notes and photos.

Unit powers on, amber LED illuminates and no unit identifier number appears on the front. Both power supplies have BATTERY FAILURE led illuminated steady, not blinking. Serial console reports nothing.

Controllers themselves illuminate all three status LEDs, then HOT PLUG and NODE FAULT begin flashing. After a few seconds, controller 0 stops flashing for a litlte bit and then starts flashing again.

Any suggestions? Could it be two simultaneous failed batteries?


Top
 Profile  
Reply with quote  
 Post subject: Re: 3par 7200 double node failure after moving to new rack
PostPosted: Tue Nov 12, 2019 12:11 pm 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
If serial console reports nothing on either nodes, I would almost guess both nodes had failed.

Did you reboot nodes or power cycle the system before moving it to ensure there wasn't any latent hardware failures in the system?

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: 3par 7200 double node failure after moving to new rack
PostPosted: Tue Nov 12, 2019 12:15 pm 

Joined: Thu Jun 13, 2019 7:12 pm
Posts: 11
Nope. I verified there was nothing reported in the management console, that's all.

I don't care about the data on the unit; is it possible to recovery from a double node failure without HP direct involvement?


Top
 Profile  
Reply with quote  
 Post subject: Re: 3par 7200 double node failure after moving to new rack
PostPosted: Tue Nov 12, 2019 12:40 pm 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
If the nodes have failed, you need new nodes.

If my memory serves me right, a replaced node gets the system details from the remaining node. If both have failed, you have none working so there is no one to get the information from and you would have to get the information manually entered in the nodes. Secondly you would have to get the 3PAR OS loaded on the nodes, however this is also usually done from one of the other working nodes in the system.

So nothing is impossible, but I have no idea where to start :)

Edit: friendly advise for the future. Always test the hardware before moving anything, at least older storage systems. Reboot nodes one by one, and then full power cycle by flipping the circut breakers.

By doing that you can fix issues upfront and you also have proof towards transportation company if something breaks on the way from A to B.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: 3par 7200 double node failure after moving to new rack
PostPosted: Tue Nov 12, 2019 12:44 pm 

Joined: Thu Jun 13, 2019 7:12 pm
Posts: 11
That mirrors my line of thinking.

We shall see how this goes. We have third-party maintenance from a local company, I'll see what they have to say.

I think it's possible to failed PSU batteries might prevent system from booting, in order to protect data. I can't find any reliable information on that though.


Top
 Profile  
Reply with quote  
 Post subject: Re: 3par 7200 double node failure after moving to new rack
PostPosted: Wed Nov 13, 2019 4:05 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
keenerb wrote:
That mirrors my line of thinking.

We shall see how this goes. We have third-party maintenance from a local company, I'll see what they have to say.

I think it's possible to failed PSU batteries might prevent system from booting, in order to protect data. I can't find any reliable information on that though.


My assumption is that it would disable write-cache.

But, there was a patch released not that long ago that allowed 2-node systems to boot with a failed node. However, it you had one failed node, the other one should have had serial output.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
 Post subject: Re: 3par 7200 double node failure after moving to new rack
PostPosted: Wed Nov 13, 2019 5:03 pm 

Joined: Sat Nov 24, 2018 4:18 pm
Posts: 35
If boot drives are ok, then you can get two working nodes and just swap boot drive ssd into it


Top
 Profile  
Reply with quote  
 Post subject: Re: 3par 7200 double node failure after moving to new rack
PostPosted: Tue Nov 19, 2019 9:30 am 

Joined: Thu Jun 13, 2019 7:12 pm
Posts: 11
feldsam wrote:
If boot drives are ok, then you can get two working nodes and just swap boot drive ssd into it


I had considered this as well. Do you know if these are standard SATA drives? Can I inspect them with a SATA to USB adapter? What filesystem are they?


Top
 Profile  
Reply with quote  
 Post subject: Re: 3par 7200 double node failure after moving to new rack
PostPosted: Tue Nov 19, 2019 10:17 am 

Joined: Mon Sep 21, 2015 2:11 pm
Posts: 1570
Location: Europe
Google says that this is one of the models used : https://www.ebay.com/itm/LOT-OF-10-SanD ... 3350774864

So looks like SATA connector.

I would guess it is some type of Linux filesystem, but I would also expect it to be encrypted or something.

_________________
The views and opinions expressed are my own and do not necessarily reflect those of my current or previous employers.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 9 posts ] 


Who is online

Users browsing this forum: No registered users and 34 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group | DVGFX2 by: Matt