Benchmark - 2nd run

[Jean-Marc Zucconi <jmz@xxxxxxxxxxxxxx>]

Hi,

I have made another set of benchmarks, following several
suggestions. I benchmarked FreeBSD 4.9-R, 5.1-R, 5.2-R and
DragonFly. 

- I used the same kernel configuration file for the 4 systems. In
  particular I removed 'cpu 486', maxusers was set to 0, and I don't
  have INVARIANTS or WITNESS options.

- the kernels and the worlds were compiled with CFLAGS=-O -pipe.

- /etc/malloc.conf is always linked to aj

- I/O tests were done on the same filesystem (IDE UDMA33 in all cases)

- bytebench binaries were compiled static. I used the same binaries
  on FreeBSD 4.9-R and DragonFly; they were recompiled on
  FreeBSD-5.[12] as the compiler is not the same.

                                          FreeBSD-4.9    FreeBSD-5.1     FreeBSD-5.2       DragonFly
Dhrystone 2 without register variables   660050.1 lps   650148.4 lps    536903.0 lps    667712.8 lps  
Dhrystone 2 using register variables     660062.3 lps   650258.6 lps    536114.9 lps    665312.0 lps  
Arithmetic Test (type = arithoh)        1518178.8 lps  1527450.3 lps    859344.7 lps   1513511.1 lps 
Arithmetic Test (type = register)         89009.9 lps    91551.8 lps     74357.4 lps     87657.5 lps  
Arithmetic Test (type = short)            81709.0 lps    91413.9 lps     70952.4 lps     81487.4 lps  
Arithmetic Test (type = int)              89030.8 lps    91550.6 lps     74461.2 lps     88331.6 lps  
Arithmetic Test (type = long)             88999.0 lps    91544.6 lps     74446.8 lps     89063.4 lps  
Arithmetic Test (type = float)            88148.0 lps   105020.8 lps     85441.5 lps     87531.7 lps  
Arithmetic Test (type = double)           88148.2 lps   104999.4 lps     85508.8 lps     87891.6 lps  
System Call Overhead Test                107461.9 lps    82630.7 lps     39562.5 lps     89537.9 lps  
Pipe Throughput Test                     133588.2 lps    86297.9 lps     30748.0 lps    106999.4 lps  
Pipe-based Context Switching Test         38436.5 lps    23821.9 lps      7884.0 lps     18203.4 lps  
Process Creation Test                      2736.3 lps     1574.6 lps      1110.1 lps      1569.6 lps  
Execl Throughput Test                      1772.9 lps     1521.2 lps       864.7 lps      1497.6 lps  
File Read  (10 seconds)                  179477.0 KBps  167848.0 KBps    38635.0 KBps   179972.0 KBps 
File Write (10 seconds)                   18633.0 KBps   12800.0 KBps     7800.0 KBps    18433.0 KBps 
File Copy  (10 seconds)                   16298.0 KBps   11944.0 KBps     4436.0 KBps    16271.0 KBps 
File Read  (30 seconds)                  179598.0 KBps  169150.0 KBps    39068.0 KBps   177467.0 KBps 
File Write (30 seconds)                   18399.0 KBps   12589.0 KBps     7688.0 KBps    18277.0 KBps 
File Copy  (30 seconds)                   15297.0 KBps   11468.0 KBps     4375.0 KBps    15308.0 KBps 
C Compiler Test                             428.1 lpm      258.4 lpm       143.2 lpm       407.1 lpm  
Shell scripts (1 concurrent)                756.0 lpm      567.9 lpm       284.0 lpm       628.6 lpm  
Shell scripts (2 concurrent)                507.3 lpm      346.6 lpm       165.0 lpm       385.6 lpm  
Shell scripts (4 concurrent)                271.0 lpm      183.0 lpm        86.6 lpm       208.3 lpm  
Shell scripts (8 concurrent)                137.7 lpm       93.0 lpm        45.0 lpm       107.0 lpm  
Dc: sqrt(2) to 99 decimal places          83900.0 lpm    52378.5 lpm     40616.7 lpm     25753.6 lpm  
Recursion Test--Tower of Hanoi             7920.6 lps     8162.1 lps      6736.9 lps      7825.9 lps  


                     INDEX VALUES           --------- FreeBSD-4.9 --------  --------- FreeBSD-5.1 --------   --------- FreeBSD-5.2 --------    ---------- DragonFly ---------
TEST                                        BASELINE     RESULT      INDEX  BASELINE     RESULT      INDEX   BASELINE     RESULT      INDEX    BASELINE     RESULT      INDEX
                                                                                                                                                                             
Arithmetic Test (type = double)               2541.7    88148.2       34.7    2541.7   104999.4       41.3     2541.7    85508.8       33.6      2541.7    87891.6       34.6
Dhrystone 2 without register variables       22366.3   660050.1       29.5   22366.3   650148.4       29.1    22366.3   536903.0       24.0     22366.3   667712.8       29.9
Execl Throughput Test                           16.5     1772.9      107.4      16.5     1521.2       92.2       16.5      864.7       52.4        16.5     1497.6       90.8
File Copy  (30 seconds)                        179.0    15297.0       85.5     179.0    11468.0       64.1      179.0     4375.0       24.4       179.0    15308.0       85.5
Pipe-based Context Switching Test             1318.5    38436.5       29.2    1318.5    23821.9       18.1     1318.5     7884.0        6.0      1318.5    18203.4       13.8
Shell scripts (8 concurrent)                     4.0      137.7       34.4       4.0       93.0       23.2        4.0       45.0       11.2         4.0      107.0       26.8       
                                                                 =========                       =========                        =========                         =========
     SUM of  6 items                                                 320.7                           268.0                            151.7                             281.3
     AVERAGE                                                          53.4                           44.7                              25.3                              46.9



Finally to test if there is a problem with the compiler I used the lorenz
benchmark (it simply does raw math computations, and does not use much
memory):

Lorenz on 5.1
4.9binary: 15.77 real        15.73 user         0.01 sys
5.1binary: 16.52 real        16.51 user         0.00 sys
5.2binary: 16.36 real        16.32 user         0.02 sys

Lorenz on 5.2
4.9binary: 19.35 real        18.12 user         1.14 sys
5.1binary: 27.65 real        26.05 user         1.48 sys
5.2binary: 29.61 real        27.60 user         1.87 sys

The 5.1 results indicate that there is no problem with the compiler
generated code, but I don't know how to interpret the 5.2 results: I
would expect also a bad behavior with the 4.9 binary (all are static)

Then I booted a uniprocessor kernel and reran the latest test:

Lorenz on 5.2 UP
4.9binary: 15.80 real        15.71 user         0.02 sys
5.1binary: 22.57 real        22.45 user         0.03 sys
5.2binary: 24.16 real        24.05 user         0.03 sys

Jean-Marc

-- 
Jean-Marc Zucconi -- PGP Key: finger jmz@xxxxxxxxxxx [KeyID: 400B38E9]