Re: hammer prune explanation

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

:Hi,
:
:I don't understand the usage of
:
:   hammer prune from xxx to yyy every zzz
:
:Could someone enlighten me, what the "from" and "to" exactly means?
:
:Does it mean, that all deleted records with an age between xxx and yyy
:are considered for pruning? Starting from "xxx", just keep deleted
:records every "zzz"?
:
:Regards,
:
:   Michael

    You got it.   Note that 'deletions' also mean overwrites and changes.
    For example, if you chmod a file HAMMER will remember the old modes
    as a deleted record.

    So here's an example:

	hammer prune /mnt from 1d to 30d every 1d

    Anything between 1 day and 30 days old which has been deleted is subject
    to pruning.  But you are also saying 'every 1d', which means you are
    telling HAMMER to *RETAIN* deleted records on 1-day boundaries.  The
    'every' specification becomes your snapshot granularity.

    So take some file X and say it was modified like this over the period of
    a couple of days:

    [-------- day1 --------][-------- day2 --------][-------- day3 --------]
    aaaaaaaaaaaaaabbcccccdddddddddeeeeffffggghhhhiiiiiiijjjjjjjkkklllllllllllll

    The pruning run will physically remove and recover the space for
    records 'b', 'c', 'e', 'f', 'g', 'h', 'j', and 'k'.  I will expand the
    range of the other deleted records that were retained in order to
    maintain a continuum space for AS-OF lookups:

    [-------- day1 --------][-------- day2 --------][-------- day3 --------]
    aaaaaaaaaaaaaaaaaaaaaaaaddddddddddddddddddddddddiiiiiiiiiiiiiiiiiiiiiiiilll

    Now I want you to also note a side effect of this, which is that the
    timestamp range for the deleted records that were retained has been
    expanded.

    This expansion of the range, which is done in order to maintain a 
    consistent continuum for AS-OF lookups, can cause 'record creep'.

    This record creep is why the hammer prune function puts so many
    restrictions on the date ranges.  You can do this:

	hammer prune /mnt from 5m to 1h every 5m

    But you can't do this:

	hammer prune /mnt from 5m to 1h every 6m

    The reason is that if you do not specify boundaries that are integral
    multiples within the range being pruned, the 'record creep' becomes
    unbounded... it just keeps creeping.

    If you prune using a configuration file you can give multiple ranges,
    like this (/etc/hammer.conf).  Each successive range must be an integral
    multiples of the previous one to avoid record creep.

    prune /usr/obj from 15m to 12h every 15m
    prune /usr/obj from 12h to 7d every 12h
    prune /usr/obj from 7d to 35d every 1d
    prune /usr/obj from 35d to 350d every 7d

    I probably need to add some more features to the pruning directive,
    like a shortcut to tell HAMMER not to retain any data at all past a
    certain age.

					-Matt
					Matthew Dillon 
					<dillon@backplane.com>