Re: Benchmarking

[Ivan Voras <ivoras@xxxxxxxxxxxxxx>]

On Sun, 28 Mar 2004, Matthew Dillon wrote:

> :It seems that DF could do a lot better but there's something holding it
> :back (especially in the web CMS test). I got a lot of 'too many database
> :connections' errors, like the db server doesn't get the chance to free
> :the connection when it's closed so that the next client can use it.
> :Also, under DF a bug in a program was reveiled: an object was not
> :protected by a mutex/lock like it should be. It is interesting that the
> :bug had not manifested on all the other tested systems, but on DF it was
> :consistent and frequent (maybe the process-switching rate in DF is higher?).
> :

>
>     'Too many database connections' errors sound like something that could
>     be tuned, but without more information it's hard to guess at what the
>     problem is.  Maybe a soft file descriptor limit is being reached or

Yes, it very easily could be, but that is not the intention in
benchmarking :)

>     something.  The CMS transaction rate issue is probably either related
>     to the reported error creating issues, or it is related to the over
>     active scheduler (which hopefully was fixed last night).  The other
>     CMS numbers look right.

No, I recorded only the runs without errors. My "gut feeling" about it is
related to what you said - the postgresql (forked) children seem to stall
occasionaly, or loose time in high context-switching rate.

>     expected FreeBSD-5 to win here because their PIPE code is totally
>     giant-free.  The shell script performance is rather odd, I'm not sure
>     I believe the FreeBSD-5.2-CUSTOM number there.

The results were consistent across several runs.

>     DFly verses FreeBSD-4.9 due to the serializing token overhead.  I'm
>     actually a bit surprised that DragonFly is beating out FreeBSD-5.x
>     there, perhaps FreeBSD-5.x is not being compiled with the same filesystem
>     optimizations (like UFS_DIRHASH).  I may have bumped up the cache limits

It was.

>     The PG TPS numbers look about right.  What you are seeing is SMP
>     overhead.  FreeBSD-4.9 is probably serializing/batching the operations

The machine is UP :)

>     more (which always makes raw TPS numbers look better), but if so this
>     is normally not observable unless you also measure the standard deviation
>     of the transaction latency.  That's why raw TPS numbers make for bad

Yes, at this stage I'm very sorry I didn't, but now it is lost because I
don't have the time to re-run all systems...

>     CMS: already covered.  I'd be interested in knowing whether rerunning
>     that test with the latest kernel (and after figuring out what is causing
>     the error messages) improves the numbers any.

I'll try it.

>     Buildworld tests?  Building the same world or each project's own worlds?
>     You can't really compare buildworld times because the projects have vastly

I know... That one's just for fun :)


-- 
Every sufficiently advanced magic is indistinguishable from technology
   - Arthur C Anticlarke