Re: HAMMER and RAID 5

[Dmitri Nikulin <dnikulin@xxxxxxxxx>]

On Tue, Mar 3, 2009 at 1:08 PM, Mag Gam <magawake@gmail.com> wrote:
> I was wondering if HAMMER will ever have network based RAID 5. After
> researching several file systems it seems HAMMER is probably  the
> closest to achieve this problem and will make HAMMER a pioneer.

Intuitively I highly doubt network RAID5 is worth it. Even local disk
RAID5 is unusable for many work loads.

In contrast, check out some of the more flexible RAID10 modes
available in Linux:
http://en.wikipedia.org/wiki/Non-standard_RAID_levels#Linux_MD_RAID_10
You can get N/M effective space (N raw storage / M copies) with
RAID0-like striping for all of it. It performs very well and certainly
much better than the parity-based RAID5.

Imagine how RAID5 would work with network devices:

Read old data block from one server
Read parity block from another server
Generate new parity block
Write data block to one server
Write parity block to another server

All with NO atomicity guarantees, so HAMMER would have to pick up the
slack. Even in the best case you have 8x the latency of a single trip
to a machine (4 request/response pairs of 2 IOs each). All compared to
a one round trip (2 IOs) to write to a plain slave, or N round trips
for N redundant copies. What is an acceptable penalty on local disks
is pretty heavy for network storage.

If you really want, you can use vinum over iSCSI to get networked
RAID5, but it will not perform well.

-- 
Dmitri Nikulin

Centre for Synchrotron Science
Monash University
Victoria 3800, Australia