Re: Preliminary restructuring layout (was Re: sys/ tree re-structuring proposal)

[Julian Elischer <julian@xxxxxxxxxxxx>]

Matthew Dillon wrote:
>      Well, excellent, it looks like I didn't mess things up too badly :-)
>
>      I'm going to be working on the module build today but I thought I would
>      also explain some of things that were done including some that might
>      look a little wierd.
>
> Additional cleanups everyone should do to reduce confusion:
>
>     # remove random non-archive garbage from your sys/ tree and then
>     # cvs update in order to allow cvs to remove dead directories.
>     # (using the suggested 'update -Pd' line in your ~/.cvsrc)
>     #
>     cd /usr/src/sys
>     find . -name '*.bak' -delete
>     find . -name '*.orig' -delete
>     cvs update
>
>
> Obvious changes:
>
>     netproto/{atalk,atm,ipsec,ipx,key,natm,ncp,ns,smb}
>
> 	Generally replaces top level directories netatalk, netatm, etc.  This
> 	change went fairly smoothly and I think it is an improvement.
>
>     net/{accf_data,accf_http,bridge,disc,dummynet,ef,faith,gif,gre,i4b,ip6fw,
> 	 ip_mroute,ipfilter,ipfw,ppp,ppp_layer,sl,sppp,stf,tap,tun,vlan}

[Warning.. my opinion only]

Since you already have a 'net' I would have put net proto as net/proto
"All networking is under 'net'" sounds more consitent than
"All networking is under net and netproto".

Also the two accf modules should be under net/accf/...
in fact I'm not sure that teh accept filters shouldn't be under

net/proto/tcp/accf

things that are IP based should be under ip somewhere I think rather than
under 'net' directly..

> 	Generally move 'internal' network interfaces and partitionable
> 	infrastructure to subdirectories.


Ok but more hierarchy can make things clearer..

> 	net/ipfw and net/ip6fw are experiments.  In particular, ip6fw is
> 	experimentally moved from netinet6.  I haven't moved other things
> 	from netinet6 because I really don't know how good an idea it is
> 	yet.
>
> 	i4b was the most complex thing to clean up, particularly note that
> 	the architecture-specific portions of i4b were moved from i386/include
> 	to net/i4b/include/i386.  This was done as an experiment to see how
> 	well things could be organizde with architecture-specific pieces broken
> 	out and associated with their subsystem rather then jumbled together
> 	in one directory.  It required creating an 'i4b_machine' directory
> 	in /usr/include in order for programs to access machine-dependant
> 	portions of i4b (since <machine/BLAH> no longer has those files).
>
>     dev/* -> dev/{agp,atm,crypto,disk,drm,hfa,misc,netif,powermng,raid,serial,
> 		sound,streams,usbmisc,video}/*

Im assuming that you went ahead and put the scsi adapters under 'disk'..
I still disagree with this.. they should be under dev/scsi.. possibly even
bus/scsi

> 	Moved the device drivers from the jumbled mess under dev/ to sub
> 	categories.

>     *fs,miscfs/* -> vfs/{coda,deadfs,fdesc,fifofs,gnu,hpfs,isofs,mfs,msdosfs,
> 			nfs,ntfs,nullfs,nwfs,portal,procfs,smbfs,specfs,
> 			ufs,umapfs,union}
>
> 	All the VFS subsystems have been moved to the vfs/ directory.  mfs is
> 	no longer under ufs and ufs and ffs have been combined together and
> 	is just named ufs.

yay (ish)

you are undoing the work that Kirk did to separate ufs and ffs.
when (if) lfs comes back, where does it go? (since it was also 'ufs').


> 	I really liked how this came out.
>
>     emulation/{ibcs2,linux,posix4,svr4}
>
> 	I moved the random top level emulations and ibcs2 (which was under
> 	i386) into emulation.  It may have been a mistake to move posix4 to
> 	emulation and I might move that back to the top level.
>
> 	Eventually emulation/ will be converted to a userland middle layer
> 	as previously discussed, so it was important to partition it into
> 	its own subdirectory.
>
> 	Architecture specific files were moved under their appropriate 
> 	emulation subdirectory.  e.g. instead of i386/linux we have
> 	emulation/linux/i386 for linux-specific i386-specific files.
>
>     bus/{cam,canbus,eisa,firewire,iicbus,isa,mca,pccard,pci,ppbus,smbus,usb}
>
> 	All the bus architectures were separated out and moved into their
> 	own subdirectory.
>
> 	I may have made a mistake moving cam to bus/cam.  It's hard to say 
> 	what 'cam' is but it seems to be referenced in enough disparate 
> 	places that leaving it at the top level is probably the better choice.
>
>     modules/* -> associated directories
>
> 	All the module Makefile's have been moved to their associated
> 	subsystems.  The 'modules' directory hierarchy has been deleted.
>
> 	The Makefile's are currently named 'Makefile.module' just so I know
> 	what I have to go in and fix.  They are not part of the build system
> 	yet.
>
>     netgraph/*.h -> netgraph/<subsystem>/*.h
>
> 	When I was reorganizing netgraph I had to reincorporate the
> 	modules/ Makefiles into it which is why I moved the subsystems
> 	into their own subdirectories.  I actually didn't mind all the
> 	subsystems being at the toplevel (in netgraph/*) but I didn't see
> 	an easy way to incorporate the modules Makefile's that way.
>
> 	This turned out to be a real hassle for /usr/include file generation
> 	since there are a lot of netgraph related userland utilities.  It
> 	might make sense to undo this part of the move but for now I am 
> 	going to concentrate on making the modules build work.


I have always wante to have Makefiles near the code for the modules.
netgraph can 'stand' these changes.. as for /usr/include..
well, there aren't THAT many userland utilities..

> Inobvious changes:
>
>     By removing the use of -I- I changed many of the absolute #include file
>     paths that were being used to reference header files in the same 
>     directory as the source file into relative #include file paths using
>     quotes, e.g. "blah.h" instead of <a/b/c/blah.h>.  This generally made
>     it a whole lot easier to move subsystems around and I also believe it
>     is a good idea just on principle.  This also makes inter-subsystem
>     dependancies far more apparent in the codebase.  I did not universally
>     'fix' all the #include's for locality of reference because it is an
>     aweful lot of work, but I made a good dent.
>
>     However, 'config' generates header files for devices named <DEVNAME>.h
>     in the compile/ directory, which conflicts with similarly named header
>     files in the source tree.  Originally the source code used "blah.h"
>     to get at the compile/ directory header file and <a/b/c/d/blah.h> to
>     get at the source-version.



>     To solve this I changed 'config' to generate use_<DEVNAME>.h instead of
>     <DEVNAME>.h.  I think this change will be well received because it really
>     makes things clear in the source modules and it fits well with the
>     opt_<SUBSYSTEMNAME>.h files that config already generations for options.

Why not put them in a subdirectory..
. .compile/GENERIC/opt/blah.
It would help clean up the build directory..

>     /usr/include generation required considerable work.  Anyone making further
>     changes to /usr/src/include/Makefile should run them by me before
>     comitting as there are some gotchas (which I documented in the Makefile).
>     I added a lot of 'compatibility' softlinks to reduce the amount of work
>     that had to be done in /usr/src.  Please examine your newly generated
>     /usr/include to see the new organization.

Many ecxported .h fiels don't need to be exported.
Phk was at one stage (I think it was him) trying to work on a way to 
indicate which .h files should be expoerted and which should not.


>     UFS used to be broken up into ufs/ufs, ufs/ffs, and there was also ufs/mfs.
>     Now we just have vfs/mfs and vfs/ufs, with all the files that were in
>     ufs/ufs and ufs/ffs now in vfs/ufs.  There were no conflicts in making
>     this change and its about time that we stopped abstracting ffs as a layer
>     on top of ufs, at least in regards to source file organization.

Have you asked Kirk about this? what about ifs?


> Final Note:
>
>     None of the changes are set in stone and I am sure we will be doing 
>     additional reorganization work, especially in net/, netinet/, and
>     netinet6/.  However, it isn't easy to reorganize files due to the huge
>     number of #include dependancies so I am going to let things settle for
>     at least a month, maybe longer, before any additional major rewiring.
>
> 						-Matt