Re: Description of the Journaling topology

[Gary Thorpe <gathorpe79@xxxxxxxxx>]

Matthew Dillon wrote:
[...]
>     When all is said and done the journaling mechanism is going to look
>     like this:
>
> 			      -> [ MEMORY FIFO ] -> [ worker thread ] -> STREAM
> 			     /	   e.g. 16MB	    [ secondary spool]
>     VFSOP -> [journal shim]--				e.g. 16GB
>             (transaction id) \
> 			      -> [filesystem VFS op]
>
>
> 		          ------> target (e.g. an off-site machine).
> 		    STREAM	     |
> 			  <----------+
> 	     [transid acks going back]
>
> 	     STREAM = generic file descriptor.  e.g. regular file, socket,
> 	     fifo, pipe, whatever.  Half or full duplex.

This ASCII art does not at all look clear on the reader I am using....?

[...]
>     Eventually (not in two weeks) the journaling layer will make these acked
>     transaction ids available to any journal-aware VFS filesystem allowing
>     the filesystem to leverage the kernel's journaling layer for its own use
>     and/or to control the underlying filesystem's own management of commits
>     to physical storage.  I also intend to use the journaling layer, with
>     suitable additional cache coherency protocols, to handle filesystem
>     synchronization in a clustered environment.  In particular, an ability
>     to do high-level cache-coherent replication that would be immune to 
>     catastrophic corruption rather then block-device-level replication which
>     tends to propogate corrupting events.  As you can see, I have *BIG* plans
>     for the journaling layer over the next few years.
>
> 						-Matt

I think referring to Journaling is a bit misleading as these features 
will not help the traditional case of recovery after power loss on a 
file system (all buffered data would be lost as it is in memory). It 
seems to be more of an integration of transactions into the VFS layer, 
so that the kernel will use a Transaction topology when reading/writting 
data. I am guessing that the actual file system/remote file system will 
do any 'journaling' and indicate when transactions are committed etc. 
How will non-journaling file systems work? Will this inter operate with 
anything else that is not DragonFlyBSD (if it doesn't it makes remote 
backup less useful).

Could do all of this from a user application (a lot of distributed file 
system's work from user land I think)?