Re: The ports-system and userland in general.

[Andrew Hacking <ahacking@xxxxxxxxxxxxxxx>]

Hi all,

Ive been using Linux and FreeBSD for a number of years and now recently
DFly and Gentoo (for about a year), so I would like to share some
ideas/features which would be useful in the "replacement ports/package
system".

Firstly, I would love to see Matts "application environments" allowing
multiple packages to be installed on a system without conflict. This
would be a true breakthrough in software management.

Now to ports/packaging. Unfortunately there are a number of competing
requirements with the ports/packaging system which are probably
unresolvable but some compromise is possible.

On one hand it is nice to have pre-built packages rather than having to
wait for everything to compile.  On the other you end up with a
"GENERIC" package, with the default features/options enabled which may
be either more than what you want (ie drags in the kitchen sink), or
less than what you want (eg vim syntax highlighting wasn't compiled
in).  Contrast with a build from source approach where you have some
flexibility to customise the features you want in a package but have to
wait for everything to build.

Both FreeBSD ports and Gentoo portage give you some degree of control
over which features you want enabled when building from source.  IMO
from using both FreeBSD and Gentoo, Gentoo does a much better job when
it comes to being able to disable or enable specific features on a
system wide or an a per port basis. I am not saying Gentoo is better in
all respects (I still prefer DFly/FreeBSD), it just has some very nice
features which should not be ignored.

One of the great features of Gentoo is its notion of "system profile"
and fine grained USE variable tunables. This makes it possible to build
a system with "only what I want", eg  disable all X11 support, enable
ldap.  Gentoo also offers the ability to _easily_
install/upgrade/downgrade from multiple versions of a port which I find
somewhat lacking in the FreeBSD ports system.  There is also support for
"virtual" packages eg "X" which can be supplied from a number of
variants (X.org, XFree86, KDrive etc) as well as package "slots" which
allows more than one variant of a package to be installed on your system
(however this feature only works where variants play nice together). I
like the fact that I can use the Gentoo portage system for building
*entire systems* (kernel+userland) with just the features I want, and
that I have the ability to override the portage tree with my own shadow
tree when I really need to customise/hack a port.  This means I can use
it for building everything from a workstation to an embedded/hardened
network appliance, just by selecting an appropriate profile and tweaking
some USE variables.

Unfortunately the flexibility offered in building from source is in
opposition to providing binary packages. Perhaps a middle ground between
"binary install" and "build from source" could be achieved by supporting
a number of "binary profiles". This would result in the most appropriate
binary package for your "system profile" being installed.  The user has
the ability to get a binary package installed with a good match for
their system profile, but can still build from source if they really
must.  The "package server" could also gather statistics on what
profiles/USE variable combos are most popular so that the pre-built
binary collection can be tuned to best meet the needs of the user base
balanced against the cost of building and hosting profile specific
binary packages.

I would really like to see some of the features listed above make it
into whatever system DragonFly uses.  I'm also willing to contribute to
building such a system when the direction and approach becomes clear.  
It would be really nice to have Gentoo's "system profile" concept in
DFly.  It would be great if the profile concept could be applied not
just to ports but also to "buildworld/installworld" since that would
make building tailored mini/tiny/pico/embedded/trusted/hardened/whatever
systems work out of the box.

Ultimately though, one of the biggest impediments to change is being
able to support a large ports collection such as what FreeBSD and Debian
currently provide.  Throwing away the current system and starting afresh
is a _big_ decision.  It may be possible to migrate or wrap the current
ports tree to the "brave new world" in order to keep its breadth as
large as the FreeBSD ports tree, although keeping a large collection of
native "brave new world" ports maintained will require many hands...

Anyway thats enough from me.

- Andrew.

On Wed, 2004-12-15 at 03:20, Devon H. O'Dell wrote: 
> Weapon of Mass Deduction wrote:
> > Hello all,
> > 
> > I remember Matthew explaining he's looking for
> > a better alternative to NetBSD pkgsrc for the
> > ports-system, as a reaction to some people
> > proposing pkgsrc.
> > 
> > I'm currently thinking out a good userland
> > architecture, so I would like to know what
> > qualities he or others is/are especially looking
> > for then.
> > 
> > This also applies to the userland. Of course
> > something about this is already written down
> > at http://www.dragonflybsd.org/goals/userapi.cgi ,
> > but that article focusses on the programmatic
> > plans, not too much on the functional.
> > 
> > 
> There have been miscellaneous discussions / arguments / bikesheds about 
> this subject on IRC and on the lists. Simon (corecode) has done some 
> research into it; Will_D has as well, and there are obviously efforts of 
> other parties to get pkgsrc working with additional DragonFly-specific 
> functionality (as I understand it).
> 
> At this point, I can't say that anybody has definite goals. All people 
> have different ideas about what a perfect package manager should do. If 
> you're interested in making one, there is a link on the Wiki to some 
> discussion and research about this.
> 
> The concensus seems to be:
> 
>      o It shouldn't be ports
>      o It should be made ground-up
>      o It should allow multiple versions of a program to be installed
>      o It should allow for easy upgrades
>      o It should allow for source building and binary packages
> 
> Simon (corecode) has ideas about the format needed to store information 
> about the packages. I don't know much about this, but it's something 
> that seriously needs reworking.
> 
> There may be more points, but I don't know what they were. Something 
> that would be interesting would be a way to port files from pkgsrc or 
> ports (or other build systems) over to Whatever Build System, which 
> would reduce the need to manage $BIGNUM files with such a small team.
> 
> I think the best idea is to get started on something and, when you've 
> got something working, talk about it. If it works well, other people 
> will use it. Maybe it'll be incorporated as our ``official system''. 
> Maybe not. But at the least it'll be a good idea of what works and what 
> doesn't.
> 
> --Devon