Re: Packaging system effort

["Matthew D. Fuller" <fullermd@xxxxxxxxxxxxxxx>]

Just a few comments from my previous thinkings on related matter...


On Thu, Feb 26, 2004 at 01:36:43PM +0100 I heard the voice of
Simon 'corecode' Schubert, and lo! it spake thus:
> 
> The packaging system must provide the necessary infrastructure to
> hold descriptions for multiple versions of one ``program'' without
> lots of overhead and easy (read: easy, sane choice of versions)
> usage.

Clarification: This is an attribute of the package management system,
not of packages.  (I know we all know this; just think these sort of
splits need to be stated more explicitly)


> It is highly desirable to be able to install multiple versions of
> one program at the same time. Besides means to enable this in the
> filesystem (symlinks, variable symlinks, VFS voodoo, etc) - which
> might not be available on all target platforms - this also adds some
> more questions concerning the logic of newly installed packages.
> Imagine two perl versions installed: 5.6 and 5.8; which version
> should the newly installed spamassassin depend upon?

VFS Voodoo(tm) is almost certainly the Right Answer to this, but it's
a bear to implement (even with a good, clean layering system in the
kernel to hook into, there's so many edge cases to consider).

This requires an extra tool to map specific packages to each other for
dependancies.


> As the system should be easily usable and not just academic, easy
> tools are strongly needed. This includes tools for all sorts of
> maintenance: from updating descriptions over searching to upgrading
> installed packages. It is most desirable to also provide graphical
> tools (ncurses, X11, web), or at least provide infrastructure so
> that third parties can easily develop such.

One useful thing about pkg_* and apt-get and urpmi and such is that
you can configure a list of network servers to go to to get the
packages.  I think it'd be useful to design it from the bottom up to
look for a meta-server, and get its list of servers from there.  Makes
it easier to update and expand mirroring and load-balance and end-user
configure and all that jazz.


> Debian's way of providing -dev packages which have been splitted off
> is always a highly controversial point of discussion.

Hate.  Hate hate hate hate hate.  Big throbbing pots of hate.  Flaming
hate.

I'm not overly fond of this concept, I s'pose.


> It is desirable to have a way to import an individually defined set
> of packages for easy deployment of multiple systems.

A way (maybe a network server, maybe just a ssh-and-read-a-file)
inside the package tools to say "Give me everything that's on this
other machine", with options like a menu to make selections of those
packages, would be nifty.


> The system must both support building from source and installing
> from precompiled binary packages equally good and be able to use
> building from source as fallback method if binary packages are not
> available in an individual configuration. Furthermore it should be
> easy to build binary packages for installation on another system.

And especially don't try to outsmart me on source building.  I have
this nasty habit of running custom hacks in everything.  I don't like
systems that get in my way   8-)


> As a direct conclusion the system must have strong binary package
> distribution support. In the past a lot of people demanded a
> streaming binary format to have the ability to install packages
> straight whilst downloading without having to wait for the whole
> package. This needs to be discussed further as installing while
> downloading is even less atomic than installing after download which
> can lead to other major problems.

If we do a VFS Voodoo(tm) solution where each package is installed in
its own filesystem, we don't need to worry about atomicity   8-)


> If possible, a nice addition would be the optional integration of
> installation and build management of the base system.

This always makes me uncomfortable, even as I acknowledge that it's a
necessary step at some point.  There's so much foot-shooting that can
go on, and so much vein-popping.


> This update strategies need not only to be selected for all
> available packages as a whole, the user needs to have the ability to
> individually specify them for single packages or groups thereof.
> Ports doesn't provide this - versions are dictated by committers;
> portage provides this feature to some small amount (accept
> keywords).  Debian runs stable/testing/unstable versions.

This is another way of saying that we need to have more history
available, so you can more easily say "No, give me this version from 4
months ago instead"?  (at least on source builds; binaries start to
eat storage space if you try)


> It should be possible to build and install packages as an
> unprivileged user.  Sometimes local security policy or laziness of
> an admin demands the installation of a package into the user's home
> dir. A nice point would be native support for such in the packaging
> system. This doesn't mean that binary packages need to be
> relocatable into home dirs, but the system would need to provide an
> alternative (user home) location of package registry.

The system should be able to determine at least the install base
($PREFIX or some such) at install-time.  Of course, that gets
increasingly icky in programs that internally hardcode paths at
build-time.

What do you mean by "alternative package registry"?  Is that
'registry' a noun or a verb?


> An essential duty of a packaging system is the tracking of installed
> files. It must be an easy task to remove a package and thus all its
> installed files from the system. The system needs to provide
> collision management (same file installed by several packages, VFS
> voodoo?) and configuration file awareness (see below). Compare with
> portage (automatic list building) and ports (ugly manually generated
> plists).

VFS Voodoo(tm)!!  Then you just delete the filesystem (the vn(4)
backing file, or whatever it's called these days) to delete the
package.  No mess.  And you don't even have to worry about a plist;
`ls` will generate one for you.


> The packaging system descriptions shouldn't consume too much space
> in general and inodes in specific.

Are you referring to the build location (i.e., /usr/ports) or the
database location (i.e., /var/db/pkg)?  Either way, it's definately
true.


> This leads to patches. As the system should aim to be OS agnostic in
> most parts this also counts for patches. These should be specially
> crafted so that they at best don't interfere with the build process
> on other platforms/OSes.  This means extensive use of #if
> defined(__MyOS__) etc.

And/or, a registry where you define what platform (down even to what
version of the OS) each patch is necessary for.


> The system should provide support for bug tracking so that users can
> easily check for known bugs and report new ones or add followup
> information for existing ones.

I think an existing PR system is the best solution here, with some
glue to be able to associate PR's with packages.


> The system should be able to bootstrap itself. This means it
> shouldn't depend on the system tools be included with the host OS.

That means self-installing packages (sorta like self-extracting ZIP
files).


> Basic Principe: The last instance of decision is always the user -
> but she shouldn't have to be in most cases.

Amen.



-- 
Matthew Fuller     (MF4839)   |  fullermd@xxxxxxxxxxxxxxx
Systems/Network Administrator |  http://www.over-yonder.net/~fullermd/

"The only reason I'm burning my candle at both ends, is because I
      haven't figured out how to light the middle yet"