Re: HAMMER update 06-Feb-2008

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

:How about "Generational Garbarge Collection"? Assuming that there are 
:some files that will never be deleted this could give slighly better
:performance.
:
:Keep a "copy count" (a copy occurs if the cleaner has to copy data from 
:the left end to the right end of the FIFO). If that increases over, say 
:3, copy it into the old generation FIFO.
:
:One problem of course is how to dimension each generation, and how many
:to use.
:
:I think that's basically how LFS works.
:
:Regards,
:
:   Michael

    The idea I have is turn the physical layout of the disk into a
    logical linear layout.  That is, instead of actually laying the FIFO
    down linearly on the physical disk, instead create a very large backing
    blockmap that allows large (say 16MB) chunks of the disk to be shifted
    around simply by adjusting the block map.

    Now the FIFO copying can simply shift the underlying block map to move
    a 16MB block from the beginning of the FIFO to the end of the FIFO
    if no garbage collection is otherwise required.

    One can also have the idea of 'named very large blocks', where each one
    represents a separate recovery zone.   This is like a blockmap allocation
    mechanism that allows you to move the block from the beginning of the
    FIFO to the end without having to rewire the references to elements
    within that block.

    I think the initial implementation has to just work with the disk.  I
    am making sure my design is condusive to these sorts of optimizations
    but I am not going to try to implement a blockmap immediately.

    What is becoming very clear to me now that I'm actually coding this stuff,
    is that my original clustering idea was simply too highly integrated into
    the general filesystem operations, requiring complex code all over the
    place.   The new idea is far, FAR less complex and yet as we see there are
    plenty of layer-based solutions that can solve the copy case issues
    without requiring the filesystem core to have much knowledge about the
    additional layer.

					-Matt
					Matthew Dillon 
					<dillon@backplane.com>