Re: Background fsck

[Gary Thorpe <gathorpe79@xxxxxxxxx>]

65264.cpressey@xxxxxxxxxxxxxxx>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Lines: 167
NNTP-Posting-Host: 149.99.114.90
X-Trace: 1074716458 crater_reader.dragonflybsd.org 181 149.99.114.90
Xref: crater_reader.dragonflybsd.org dragonfly.kernel:3374

Chris Pressey wrote:

> On Wed, 21 Jan 2004 11:10:52 -0500
> Gary Thorpe <gathorpe79@xxxxxxxxx> wrote:
> 
> 
>>Smarter and more self-contained devices that lie about caches being 
>>on/off and cannot even automatically relocate bad sectors properly?
>>
> 
> OK, "cleverer" (in quotes) might have been a better choice of word than
> "smarter". :)


There was a thread on tech-kern@xxxxxxxxxx regarding relocation of bad 
sectors and caching and the informal observations were that some IDE 
drives LIE about this when the cache is enabled. That's such a wonderful 
improvement don't you think? In fact, without hardware being able to 
reliably and truthful inform the OS on what is happening, no filesystem 
can guarantee anything. But ATA gets high sequential transfers so I 
guess thats all that matters.

> 
> 
>>Data consistency is still the responsibility of the OS
>>
> 
> I respectfully disagree - data consistency is the responsibility of the
> filesystem.


And a filesystem is a part of the OS, whether its in a monolithic kernel 
or exists as a user-space server. Critical functionality like that is by 
definition a part of the OS. Device drivers can run at least partially 
in user-space, but I would still consider them OS components.

> 
> There are two models you can follow here - roughly, the BSDs treat the
> filesystem as part of the OS while the Linuxes don't.  I'd rather DFBSD
> go on a path closer to the Linux model.  Instead of inventing a new
> journalled filesystem, why not use existing ones.  At least to start.


I agree. Instead of redesigning one, it would be better to use an 
existing one: ext3, reiserfs, xfs, jfs, bfs, vxfs - there are a lot of 
choices. All of the open sourced ones are GPLed however and some like 
XFS have almost as much source code as the rest of the linux kernel put 
together. Existing designs may be MORE complex and harder to maintain 
than softdeps: do you want that? If it is necessary, design one from 
scratch using principles the others explored/built on. Its not impossible.

> This means fixing VFS, but that's got to be done anyway.
> 
> Perhaps the filesystem will never migrate to the device itself, but on
> the other hand, there's no reason not to cut its umbilical cord.


Since the VFS is NOT the major obstacle to supporting jounraling (almost 
all discussions I have seen end with "lets get LFS working right instead 
" which implies that the people who will actually decide want to keep it 
a BSD-based) and the VFS systems in all the BSDs already support 
multiple file system, I don't see where you are going? Do you mean to 
make them more modular/flexible to allow module loading unloading and 
dynamic addition of filesystems? Do you honestly think Linux has a good 
design for this or is it a hack (I don't know I am asking)?

> 
> 
>>Right: so design a file system with atomicity.
>>
> 
> Right, or adopt one, or several.

> 
> 
>>The way this is most commonly do is with journaling.
>>
> 
> Sure.  But journalling != atomicity, and I don't care nearly as much
> about the former as I do the latter.


Yes journaling is atomicity: either a change makes it into the log or it 
doesn't, or at least thats the impression I get on HOW they _should_ be 
designed (with journaling commits being atomic). The other alternative 
is to try and get EVERY file system operation to be atomic, which will 
probably be infeasible or completely destroy performance. Disks can only 
guarantee that small blocks are read/written atomically, so could you 
please elabourate on how this would work?

> 
> 
>>Suppose the driver has a bug which cause the kernel to use an invalid 
>>pointer: since most OS's are still monolithic, you are more unsure
>>about what you may have just corrupted (including FS code).
>>
> 
> Or suppose the kernel just refuses to use the invalid pointer.


Or suppose it IS valid, but it points to the wrong data and you 
overwrite something and it is only caught later? What error handling can 
you do: the error is asynchronous as it will either go undetected 
immediately and be revealed later OR it will cause a trap. Unless you 
want to add exception handling to a kernel, there is not much else you 
can do if the error occurs in the same module as the core kernel (as in 
a monolithic and not in a microkernel, although faults within a 
microkernel and not one of the servers would have the same result).

> I guess the usual argument against that is that it's inefficient to
> determine the validity of every single pointer before using it, and to
> a great extent that's true.  There's always going to be a tradeoff
> between efficiency and robustness.


Something like that is what hardware is needed for: AS400's implement 
this I believe with hardware support (no FS journaling in hardware 
though). I suppose it would be interesting to people working on fault 
tolerance/corrections in things like space exploration, but I doubt 
there is enough will to get it working on even commercial systems.

> 
> So this brings me to something that's got me in a bit of a tizzy. 
> Basically, the website is good at describing the technical goals of
> DFly, but it's a bit light on overall philosophy.  The following design
> goals are mentioned on the main page: 
> 
>   - Scaleability
>   - Robustness
>   - Debuggability
>   - Compatibility (forwards and backwards)
>   - Upgradeability
> 
> The rest is nuts-and-bolts stuff, which is fine, but from it, you can't
> easily take Idea X for Improvement Y and see how well it fits in The
> Big Picture[tm].  That's why it's hard to evaluate stuff like
> internationalization and other suggestions that cross this list every so
> often - sure, they'd be great, lots of things would be great, but
> there's really nothing in the project that implies that it should (or
> shouldn't) get a high (or low) priority.
> 
> Another (although much more hackneyed) way to put this, to paraphrase
> Ulrich Spoerlein's sig: "FreeBSD is the most powerful OS.  NetBSD is the
> most portable OS.  OpenBSD is the most secure OS.  Linux is the most
> popular OS.  So what is DragonFlyBSD?"

> 
> Most scaleable?  Most robust?  Most debuggable?
> 
> Just-plain-best-designed?


Why does this matter? Who needs a cheap slogan anyway?

> 
> Can the design goals of DFly even *be* reduced to a sound bite that fits
> in a sig?  In a sense, I hope not, but in another sense, if they can't,
> the project's direction risks being unclear.
> 
> Not sure what can be done about this except to lay out DFly's
> philosophical goals more explicitly.
> 
> Anyway, sorry for ranting, take it with a grain of salt.
> 
> -Chris
>