DragonFly BSD

hammerscript

Notes for Justin and Samuel to pass back and forth.

Introducing HAMMER (sjg)

The problems everyone thinks they have, but really don't -or- The problems HAMMER was NOT designed to solve

The nail

How HAMMER compares to its peers and predecessors (sjg)

Traditional filesystems

Modern filesystem trends

Getting to know HAMMER (jcs)

DragonFly, at its inception in 2003, was intended to be a single-system-image operating system, able to run on multiple machines but presenting a single running operating system to the user. The increase in CPU cores has made that less desirable as a target, but being able to move data from volume to volume, no matter the location, is still useful.

Hammer is a filesystem designed to mirror disk volumes from server to server, retaining history and deduplicating data. Development started in 2007, and was first released with DragonFly 2.0. Deduplication was added in 2008 (??) and released with version 2.8. (??)

Hammer is a historical, fine-grained file system, saving the state of the disk every time the system commits to disk. That fine-grained history means that old versions of files can be accessed at any point in time, live, as long as you have the disk space to hold that history of changes. In addition, entire snapshots of Hammer volumes are saved, with read-only versions of the disk sitting for access. This happens automatically, even with deleted files. The amount of disk history retained is configurable, and the number of snapshots is limited only by total disk space.

DragonFly also is able to mirror volumes from one disk to another, or from one machine to another, or even one machine to many slave machines. It works over relatively slow links, too. A Hammer volume can be mirrored to another off-site system as a backup strategy, or even to another local disk and a remote one at the same time. The amount of history retention for each master and slave Hammer volume can be configured separately, so a large Hammer slave can be set to hold multiple months of disk activity, while the smaller Hammer master can keep much less, to conserve space.

Hammer supports a feature found in higher-end filesystems: deduplication. In general terms, a deduplicated system keeps track of all the data on a system. When 2 ranges of data match exactly, the system keeps only one of those ranges, but maintains two references to it. The total amount of space used on a disk can be greatly reduced, without changing the data stored.

Deduplication results vary depending on how much repeated data structures exist on a Hammer volume. The results tend to be a 10% to 40% reduction in disk usage.

Hammer offers a number of other modern benefits:

Deploying HAMMER

HAMMER recipes, a guide for the impatient. (jcs)

Are you impatient? Are you sold on Hammer and just want to try it out? Did you just delete something really, really important and need to get it back? Here's some simple use cases for Hammer to show just what you can do.

Simple case: I scrambled a file

The most simple case: you've scrambled a file. Maybe you rewrote several lines and saved it, or accidentally mashed the keyboard, but either way, the file is still present - just wrong.

By default, the undo tool will output the previous version of a file with a note about the timestamp for that last change, prefixed with >>>. 

undo filename

>>> filename 0000 0x00000001c059c3d0 20-Aug-2011 19:24:18

(previous version info here)

Other options exist, like using -i to iterate over all previous versions saved to disk, or way to generate a diff. What if you delete the file? It'll still work.

Help! I filled up the disk!

File history takes up space on disk. Remember, the amount of history data isn't determined by the quantity of files, but rather how frequently they change. Those changes are what makes up filesystem history. It's possible to fill a disk because of that history, yet the df command will still report less than 100% usage.

Each pseudo-file-system (PFS) on a Hammer volume is part of that volume's space. If a particular PFS is very busy, it can eat up all space on the disk and make every PFS unusable. You can prune older history entries from each volume using 'hammer prune /PFSname', or if you don't care about losing recent fine-grained history entries, 'hammer prune-everything'. That command may take a few minutes to complete.

For a long-term fix, you may want to change the amount of history retained in any given PFS with 'hammer viconfig'. That same config controls what 'hammer prune' will remove.

How to create a snapshot

By and large, you never need to create a snapshot manually. Hammer volumes are set up to do so, once a day by default, and saved for 60 days. Snapshots happen at the PFS level and automatic snapshots are configured with 'hammer viconfig', see the cleanup section of the hammer(8) manpage for details.

These snapshots can be found in /var/hammer, in directories named after the matching PFS. So, if you happen to delete an important log file in /usr, chances are good you can find an older version of it in /var/hammer/usr/. Each snapshot is named by the time at which it is taken, i.e. 'snap-YYYYMMDD-HHMM'. Treat them as read-only backup volume that you can copy from at any time.

If you do want to create a snapshot at an arbitrary time, 'hammer snap PATH' will take an immediate snapshot of the PFS that contains PATH.