I have one query regarding the cage level redundancy. Suppose i am using set size of 3+1 in RAID 5 and i am having 5 disk enclosures behind each pair of nodes. So how it will write data to these 5 disk enclosures with 3+1 step size. What will happen to 5th disk enclosure. Will it write in round robin way ??? and what is best practice in this case??
If you go raid 4+1 then you will have an exact fit, one chunklet from each cage will be used to form a raidlet, multiple independent raidlets will form a logical disk.
If you use 3+1 then your first raidlet will use chunklets from cages 1,2,3,4 , the second raidlet will use chunklets from cages 5,1,2,3, the third 4,5,1,2 and so on with the raidlets continually wrapping around the cages.
The downside of this approach is you will inevitably have a few chunklets left as there won't be enough to form the final few stripes in the system. The upside is you have to buy fewer disks to maintain cage on upgrades and a smaller stripe size has better availability, less overhead in use and also rebuilds faster.
Basically if your cage numbers per node pair match or exceed your Raid 5 stripe size you're good to go. For Raid 6 you need to match or exceed half the stripe size since you can lose 2 chunklets from the same raidlet and survive. For Raid 1 with two copies you'll need two or more cages. As mentioned in the previous post all of this assumes your disk quantities are divisible by the stripe size and are distributed vertically to ensure all disk can be used for cage level.
Thanks a lot for replying . It almost clear my doubts. I can understand how no of cages and RAID set size correlates. But i cannot understand how RAID set size correlates with no of disks per cage.
I can understand that we should have equal no of disks across all the cages and to have best cage level redundancy. But how number of disks per cage correlates with RAID set size??
Lets say you have 5 cages and 20 FC drives that would provide balanced cage level availability for the following:-
20 disks / 4+1 set size = 4 LD's (2 per node) 20 disks / 8+2 set size = 2 LD's (1 per node)
With the above you could still provide storage for other set sizes but potentially you would end up with either a node imbalance or wasted space as the set size wouldn't divide equally into the disk qty. So It's probably not a good idea if you want 100% cage availability, although most tend to run a mixture of cage and mag and mag could utilise the remainder. When configuring the sizing tools will avoid these configurations because of the above and will match the disk qty to the set size.
Really you need the number of disks to be evenly divisible by the set size, NinjaStars and other tools will enforce this for you if you click the H/A button.
