Re: HAMMER update 06-Feb-2008

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

:This sounds quite like LFS now.  LFS however split the volume in smaller =
:
:blocks which could be "used", "empty" or "open", IIRC.  Their background =
:
:cleaner then could push remaining data from used blocks to a currently=20
:open one, marking the block "empty" after that, allowing the FS to write =
:
:to the blocks again.
:
:They always had problems with the cleaner, I think.
:
:cheers
:   simon

    It's the 'pushing data around' action that causes the most problems
    with recovery and cleaning, or more specifically, recovery after you
    crash while cleaning.  It can get messy really fast, especially if you
    are trying to manage bitmaps.

    With the cluster idea I got stuck the moment a transaction spanned
    more the one cluster.  I suddenly had two recovery domains related
    to one transaction, and I couldn't find a solution to it.

    A filesystem-wide FIFO abstraction solves all three problems by
    providing exactly one synchronization point (the circular fifo's indices)
    for the entire filesystem.

    I really do think that the performance issues with cleaning HAMMER
    will be addressable through the use of a very-large-block blockmap,
    or by implementing a set of named very-large blocks.  I have some
    other ideas as well, such as treating the data associated with records
    as an out-of-band entity that is not made part of the FIFO itself.

    One last thing that must be considered is real time mirroring.  There's
    something to be said for having a single logical FIFO for the entire
    filesystem in a clustered environment.  It makes non-queued mirroring
    ridiculously easy to implement.

						-Matt