Re: HAMMER changes in HEAD, also needs testing

[Thomas Nikolajsen <thomas.nikolajsen@xxxxxxx>]

> I´ve made some fairly straightforward changes to HAMMER in HEAD, but
> they need testing.

I´ve done some performance tests on a HEAD system from a few days before the change, and just after.
A ~100GB tar file was extracted to a local HAMMER file system, and a cpdup copy of the files was made,
result was traversed with find.
After updating system, to include HAMMER & cpdup changes of June 20th, same thing was done again.

Unpacking and cpdup´ing was slower after updating, but find was faster.
Also after updating traversing cpdup copies was faster for copies made before update.

This wasn´t what I expected from the description of the changes, any comments?

During tar file extraction after updating system, I noticed, looking at 'hammer iostats 10',
that inode-ops, file-rd & file-wr stalled for long periods of time, like 1-2 minutes;
dev-read & dev-write was still high; after stall inode-ops would rocket up, but only for half a minute,
then a new stall started. During stall, it seemed all file system operations was also stalled,
including processes that I only can imagine was using NFS (but this can't be true I guess).

Test system is atom 330 (dual core 1,6GHz), physmem=512MB
running SMP on local SATA disk using nata(4) (no ahci(4) support for onboard disk controller).
(system does have 2GB RAM, physmem limited to get smaller core dumps;
on other tests I´ve done, this didn´t seemed to make any difference)

Tar file used contains contains many smaller files (cvs repos for most BSDs & some checkouts for each),
and a good number of big files (e.g. ISOs & pictures).

I could make some more tests, but would like to use a smaller test set;
for now 100GB is too time consuming on my HW.
Any recommendations on test set size vs physmem size.
I could also test on ahci(4) or sili(4) (on other host).

 -thomas

- HEAD system from a few days before June 20th
root@bohr# df .
Filesystem 1K-blocks      Used     Avail Capacity  Mounted on
HAMMER     854704128 130220032 724484096    15%    /hammer
root@bohr# cd /hammer/test/pre
root@bohr# time tar xf /hammer/data/hammer.tar
    15996.71 real       207.43 user      1768.43 sys
root@bohr# cd ..
root@bohr# time cpdup pre pre.cpdup
    43937.88 real       206.70 user      4339.66 sys
root@bohr# df .
Filesystem 1K-blocks      Used     Avail Capacity  Mounted on
HAMMER     854704128 344834048 509870080    40%    /hammer
root@bohr# time find pre | gzip >pre.find.gz
      847.99 real        19.55 user       185.58 sys
root@bohr# time find pre -ls | gzip >pre.find.-ls.gz
     1171.63 real        88.60 user       501.59 sys

- HEAD system from June 20th, including HAMMER changes
root@bohr# cd /hammer/test/post
root@bohr# time tar xf /hammer/data/hammer.tar
    21740.77 real       203.77 user      1752.30 sys
root@bohr# df .
Filesystem 1K-blocks      Used     Avail Capacity  Mounted on
HAMMER     854704128 457375744 397328384    54%    /hammer
root@bohr# cd ..
root@bohr# time cpdup post post.cpdup
    72476.51 real       204.83 user      4041.41 sys
root@bohr# df .
Filesystem 1K-blocks      Used     Avail Capacity  Mounted on
HAMMER     854704128 564363264 290340864    66%    /hammer
 
root@bohr# time find post | gzip >post.find.gz
      488.17 real        13.37 user       126.37 sys
root@bohr# time find post -ls | gzip >post.find.-ls.gz
      854.08 real        65.07 user       397.89 sys

root@bohr# time find pre.cpdup | gzip >pre.cpdup.post.find..gz
      362.19 real        14.24 user       124.45 sys
root@bohr# time find pre.cpdup -ls | gzip >pre.cpdup.post.find.-ls.gz
      619.08 real        65.47 user       314.53 sys
root@bohr# time find post.cpdup | gzip >post.cpdup.post.find..gz
      450.81 real        14.18 user       134.78 sys
root@bohr# time find post.cpdup -ls | gzip >post.cpdup.post.find.-ls.gz
      813.99 real        68.53 user       408.46 sys