Re: VFS ROADMAP (and vfs01.patch stage 1 available for testing)

[Joerg Sonnenberger <joerg@xxxxxxxxxxxxxxxxx>]

On Thu, Aug 12, 2004 at 06:19:40PM -0700, Matthew Dillon wrote:
>     Then I'll start working on stage 2 which will be to wrap all the 
>     VOP forwarding calls (the VCALL and VOCALL macros).
> 
>     That will give us the infrastructure necessary to implement a 
>     messaging interface in a later stage (probably around stage 15 :-)).

Do you want to keep the message API with the structure as argument or do
you want to switch to direct argument passing and marshalling in the
messaging layer? In the short term, that would make the calling more
readable, but might increase the overhead on the stack.

>     The really nasty stuff starts at stage 3.  Before I can implement the
>     messaging interface I have to:
> 
> 	* Lock namespaces via the namecache rather then via directory 
> 	  vnode locking (ultimately means that directories do not have
> 	  to be exclusively locked during create/delete/rename).  Otherwise
> 	  even the simplest, fully cached namespace operations will wind up
> 	  needing a message.

How does this play with remote and/or dynamically created filesystems?
Does the filesystem have to keep track of the namespace entries and
invalidate them? Moving away from an exclusive vnode lock for modifying
operations does fit in with internal range locking, because those could
be implemented very well e.g. in a tree-based FS.

> 	  This step alone will require major changes to the arguments passed
> 	  in just about every single VOP call because we will be switching
> 	  from passing directory vnodes to passing namecache pointers.
> 
> 	* Ranged data locks will replace the vnode lock for I/O atomicy 
> 	  guarentees (ultimately means that if program #1 is
> 	  blocked writing to a vnode program #2 can still read cached
> 	  data from that same vnode without blocking on program #1).
> 	  Otherwise the messaging latency will kill I/O performance.

Do you plan to move the data locking into the filesystem or should it
still be implemented in the VFS layer? Moving it down makes IMO more sense
because it would allow us to keep a simple locking for less important
filesystems and would allow us to better exploit the internal data structures.
E.g. if we have a special data structure to handle the byte ranges of a
file anyway, we could attach the locking on that level.

> 
> 	* vattr information will be cached in the vnode so it can be
> 	  accessed directly without having to enter the VFS layer.
> 
> 	* VM objects will become mandatory for all filesystems and will
> 	  also be made directly accessible to the kernel without having
> 	  to enter the VFS layer (ultimately this will result in greatly
> 	  improved read() and write() performance).

How does this effect not physically backed filesystems? If I want to
support compressed files in ext2, when do I have to decompress the actual
data?

> 	* Implement a bottom-up cache invalidation abstraction for 
> 	  namespace, attribute, and file data, so layered filesystems
> 	  work properly.

The invalidation is one problem, the locking another. The separation of
vnode and namespace locks should solve most issues though.

Let's discuss the rest later :)

Joerg