Re: pkgsrc bulk build profile

[Bill Hacker <wbh@xxxxxxxxxxxxx>]

Justin C. Sherrill wrote:
> Something interesting I noticed while doing an initial bulk pkgsrc build
> on df.v12.su:
>
> Build start: 2009-02-13 04:26
> Build end:   2009-02-20 18:46
>
> This machine has 4 2.4G cpus, but I think this is the same time I'd get on
> single-CPU pkgbox under good circumstances.  My handwaving guess is that
> either the pbulk process combined with the BGL is I/O bound, not CPU
> bound.
>
> I recall Joerg had mentioned using VMWare to put multiple hosts on a
> single machine, and then using pbulk's internal distributed build method
> to spread work between those hosts; that may be possible on this system
> if/when Qemu works on DragonFly?
>
> Other ideas welcome; I'd like to use bulk pkgsrc builds as a sort of
> performance profile, plus anything that makes them faster is good.

I don't think you need to wait for Qemu on DragonFly.

Seems to me that vkernel would do a better job, and it is already there.

OTOH - I'm not at all au fait with how well DFLY is doing SMP these days...

And yes - storage I/O bound  - or 'contentious' - is almost a certainty.

Put an microphone on the HDD's and listen...

;-)

I'd look to separate the work over multiple HDD and controllers more 
than, or at least as much as, separate CPU.

For dependency reasons, each store would need 100% of the sources, libs, 
and toolchain, even if only 1/4 of the 'finals' were being asked to be 
built on that portion.

Bill