HPE Storage Users Group
https://3parug.com/

3par 7400 linux & oracle
https://3parug.com/viewtopic.php?f=18&t=1020		 Page 1 of 1
Author:  steveg [ Thu Nov 20, 2014 12:12 pm ]
Post subject:  3par 7400 linux & oracle
Hi,

We have just got our 1st 3par 7400 to run a oracle DB.

during our testing we never see any write merges on the multipath devices ( using deadline scheduler), for example:

Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await svctm %util
dm-3 0.00 0.00 0.00 2018.00 0.00 8.38 8.50 0.37 0.18 0.14 28.00
dm-5 0.00 0.00 0.00 1801.00 0.00 8.39 9.54 0.37 0.20 0.15 27.50
dm-11 0.00 0.00 0.00 3818.00 0.00 16.77 8.99 0.74 0.19 0.14 51.90

Has anyone else seen this behaviour ?

Thanks

Steve
Author:  Richard Siemers [ Tue Nov 25, 2014 10:53 am ]
Post subject:  Re: 3par 7400 linux & oracle
Linux is foreign to me, I suspect/believe "write merging" is the same thing as "write coalescing" in other vocabularies... which 3PAR ports have settings to enable/disable interupt coalescence, and in recent versions/patches have changed the default to be off. Maybe thats related?

Code:
showport -par 

N:S:P Connmode ConnType CfgRate MaxRate Class2   UniqNodeWwn VCN      IntCoal
4:0:1 disk     loop     auto    4Gbps   disabled disabled    disabled enabled
4:0:2 disk     loop     auto    4Gbps   disabled disabled    disabled enabled
4:0:3 disk     loop     auto    4Gbps   disabled disabled    disabled enabled
4:0:4 disk     loop     auto    4Gbps   disabled disabled    disabled enabled
4:5:1 host     point    auto    4Gbps   disabled disabled    disabled disabled
4:5:2 host     point    auto    4Gbps   disabled disabled    disabled disabled
4:5:3 host     point    auto    4Gbps   disabled disabled    disabled disabled
4:5:4 host     point    auto    4Gbps   disabled disabled    disabled disabled
Author:  steveg [ Wed Nov 26, 2014 11:48 am ]
Post subject:  Re: 3par 7400 linux & oracle
Hi,

thanks, i'll take a look. my current thinking is that the linux kernel & device mapper is now obeying FLUSH and FUA commands and sendning them down to the 3par where on our old EVA devices and kernels it was being ignored somewhere along the OS I/O path.

Cheers

Steve
Author:  steveg [ Tue Dec 23, 2014 8:16 am ]
Post subject:  Re: 3par 7400 linux & oracle
Hi,

just to let everyone know in case they stumble into the same problem. HP upgraded the 3par from 3.1.3 to 3.2.1 P04 and that disabled the WCE bit on the presented SCSI devices and now we are seeing write merges and its looking quite a bit better from early results

Issue ID 104168 in the 3.2.1 release notes.
Author:  afidel [ Tue Dec 23, 2014 9:12 am ]
Post subject:  Re: 3par 7400 linux & oracle
steveg wrote:
Hi,

just to let everyone know in case they stumble into the same problem. HP upgraded the 3par from 3.1.3 to 3.2.1 P04 and that disabled the WCE bit on the presented SCSI devices and now we are seeing write merges and its looking quite a bit better from early results

Issue ID 104168 in the 3.2.1 release notes.

Hmm, so they lie to the OS and say that they aren't caching writes. Not sure I'm a fan, sure it improves performance but it might lead to issues. I mean the write cache is supposed to be nonvolatile, but bugs happen and if the OS has failed to issue a flush because it's assuming there is no cache them that could lead to data corruption.
Author:  steveg [ Tue Dec 23, 2014 9:53 am ]
Post subject:  Re: 3par 7400 linux & oracle
I dont know if the particular version we had on the 3par was different to prior versions so it might just be a bug in that particular version of the 3par OS we hit.

Prior to 3.2.1 when running sdparm -g WCE against a 3par volume

WCE 1 [cha: n, def: 0, sav: 0]

now its

WCE 0 [cha: n, def: 0, sav: 0]

Our HP EVA's all report write cache disabled as well so maybe all vendors do it.
Author:  JohnMH [ Tue Dec 23, 2014 2:01 pm ]
Post subject:  Re: 3par 7400 linux & oracle
All vendors with a proper writeback cache solution will do this. The need to be able to Implement this is where the storage can't guarantee data and cache integrity through end to end integrity checks and a minimum of mirrored cache with destage to permanent media. i.e a non engineered roll your own solution.
If you were to obey this then you undermine much of 3PAR's design goal and may as well deploy some cheap JBOD storage.
Page 1 of 1 All times are UTC - 5 hours
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/