[DragonFlyBSD - Bug #2297] strange NFS (client) error messages / problems

[Stefan Krüger via Redmine <bugtracker-admin@xxxxxxxxxxxxxxxxxxxxx>]

Issue #2297 has been updated by Stefan Krüger.


Had a crash today:

# kgdb kern.6 vmcore.6

Unread portion of the kernel message buffer:
panic: lockmgr: locking against myself
cpuid = 0
Trace beginning at frame 0xffffffe0a2e97548
panic() at panic+0x1fb 0xffffffff80325f3d
panic() at panic+0x1fb 0xffffffff80325f3d
debuglockmgr() at debuglockmgr+0x47b 0xffffffff803135a7
getblk() at getblk+0x146 0xffffffff80388de3
breadnx() at breadnx+0x3f 0xffffffff803892c5
nvextendbuf() at nvextendbuf+0x8e 0xffffffff803a06e4
nfs_meta_setsize() at nfs_meta_setsize+0x56 0xffffffff80427fd9
nfs_getattrcache() at nfs_getattrcache+0x129 0xffffffff80438d0c
nfs_getattr() at nfs_getattr+0xaa 0xffffffff80440424
vop_getattr() at vop_getattr+0x86 0xffffffff803b1e02
vn_stat() at vn_stat+0x31 0xffffffff803af3bf
kern_stat() at kern_stat+0x5c 0xffffffff803ab6c0
sys_lstat() at sys_lstat+0x3e 0xffffffff803ab7e2
syscall2() at syscall2+0x41e 0xffffffff80540c03
Xfast_syscall() at Xfast_syscall+0xcb 0xffffffff8052a36b
boot() called on cpu#0
Uptime: 1d2h39m50s
Physical memory: 3971 MB
Dumping 1099 MB: 1084 1068 1052 1036 1020 1004 988 972 956 940 924 908 892 876 860 844 828 812 796 780 764 748 732 716 700 684 668 652 636 620 604 588 572 556 540 524 508 492 476 460 444 428 412 396 380 364 348 332 316 300 284 268 252 236 220 204 188 172 156 140 124 108 92 76 60 44 28 12
...

(kgdb) where
#0  _get_mycpu () at ./machine/thread.h:69
#1  md_dumpsys (di=<optimized out>)
    at /usr/src/sys/platform/pc64/x86_64/dump_machdep.c:263
#2  0xffffffff80325655 in dumpsys () at /usr/src/sys/kern/kern_shutdown.c:925
#3  0xffffffff80325cbb in boot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:387
#4  0xffffffff80325f72 in panic (
    fmt=0xffffffff8059ffb8 "lockmgr: locking against myself")
    at /usr/src/sys/kern/kern_shutdown.c:831
#5  0xffffffff803135a7 in debuglockmgr (lkp=0xffffffe00e892fe8,
    flags=33554466, name=<optimized out>, file=<optimized out>,
    line=<optimized out>) at /usr/src/sys/kern/kern_lock.c:385
#6  0xffffffff80388de3 in BUF_TIMELOCK (timo=<optimized out>,
    wmesg=<optimized out>, locktype=<optimized out>, bp=<optimized out>)
    at /usr/src/sys/sys/buf2.h:102
#7  getblk (vp=0xffffffe0310db980, loffset=0, size=<optimized out>,
    blkflags=<optimized out>, slptimeo=<optimized out>)
    at /usr/src/sys/kern/vfs_bio.c:2952
#8  0xffffffff803892c5 in breadnx (vp=0xffffffe0310db980,
    loffset=<unavailable>, size=<unavailable>, raoffset=0x0, rabsize=0x0,
    cnt=<unavailable>, bpp=0xffffffe0a2e97720)
    at /usr/src/sys/kern/vfs_bio.c:922
#9  0xffffffff803a06e4 in bread (bpp=<optimized out>, size=<optimized out>,
    loffset=<optimized out>, vp=<optimized out>) at /usr/src/sys/sys/buf2.h:360
#10 nvextendbuf (vp=0xffffffe0310db980, olength=0, nlength=<optimized out>,
    oblksize=32768, nblksize=<optimized out>, oboff=<optimized out>, nboff=-1,
    trivial=0) at /usr/src/sys/kern/vfs_vm.c:377
#11 0xffffffff80427fd9 in nfs_meta_setsize (vp=<unavailable>,
    td=<optimized out>, nsize=<unavailable>, trivial=<optimized out>)
    at /usr/src/sys/vfs/nfs/nfs_bio.c:1294
#12 0xffffffff80438d0c in nfs_getattrcache (vp=0xffffffe0310db980,
    vaper=0xffffffe0a2e978d8) at /usr/src/sys/vfs/nfs/nfs_subs.c:950
#13 0xffffffff80440424 in nfs_getattr (ap=0xffffffe0a2e97888)
    at /usr/src/sys/vfs/nfs/nfs_vnops.c:682
#14 0xffffffff803b1e02 in vop_getattr (ops=0xffffffe0a193dfc0,
    vp=<unavailable>, vap=<unavailable>) at /usr/src/sys/kern/vfs_vopops.c:355
#15 0xffffffff803af3bf in vn_stat (vp=0xffffffe0310db980,
    sb=0xffffffe0a2e97a28, cred=0xffffffe00df679e0)
    at /usr/src/sys/kern/vfs_vnops.c:762
#16 0xffffffff803ab6c0 in kern_stat (nd=0xffffffe0a2e97aa8,
    st=0xffffffe0a2e97a28) at /usr/src/sys/kern/vfs_syscalls.c:2644
#17 0xffffffff803ab7e2 in sys_lstat (uap=0xffffffe0a2e97b68)
    at /usr/src/sys/kern/vfs_syscalls.c:2698
#18 0xffffffff80540c03 in syscall2 (frame=0xffffffe0a2e97c18)
    at /usr/src/sys/platform/pc64/x86_64/trap.c:1231
#19 0xffffffff8052a36b in Xfast_syscall ()
    at /usr/src/sys/platform/pc64/x86_64/exception.S:323
#20 0x000000000000002b in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

I had this running:

while (1)
sleep 10
date
ls /home/cosmicdj # my NFS homedir
end

----------------------------------------
Bug #2297: strange NFS (client) error messages / problems
http://bugs.dragonflybsd.org/issues/2297

Author: Stefan Krüger
Status: New
Priority: Normal
Assignee: 
Category: 
Target version: 


Hi,

I have DragonFly mounting my home dir via NFS (server is running Fedora 16) and noticed the following warnings on the console:

dfly.my.lan kernel log messages:
> nfs send error 32 for server f16srv:/home/cosmicdj
> receive error 54 from nfs server f16srv:/home/cosmicdj
> nfs send error 32 for server f16srv:/home/cosmicdj
> receive error 54 from nfs server f16srv:/home/cosmicdj

+ tons more, always error 32 or 54

Today, I got this:

cosmicdj@dfly:/home/cosmicdj> ls
^T
load: 0.00  cmd: ls 3244 [getblk] 0.00u 0.00s 0% 2496k
^C
^T
load: 0.00  cmd: ls 3244 [getblk] 0.00u 0.00s 0% 2496k
^C
^Z
^T
load: 0.00  cmd: ls 3244 [getblk] 0.00u 0.00s 0% 2496k

Connection to dflyclosed. (<- killed my shell)

I tried to kill 'ls', even via -9. Didn't help. So I ended up killing the shell and tried to umount/remount my NFS homedir via console:


root@dfly # umount /home/cosmicdj
^T
load: 0.00  cmd: umount 3295 [clock] 0.00u 0.00s 0% 1444k
[diagnostic] cache_lock: blocked on 0xffffffe07d31e7f8 ".history_beastie"
^C
^Z
^T
load: 0.00  cmd: umount 3295 [clock] 0.00u 0.09s 0% 1444k
[diagnostic] cache_lock: blocked on 0xffffffe07d319158 ""
[diagnostic] cache_lock: blocked on 0xffffffe07d319158 ""
[diagnostic] cache_lock: blocked on 0xffffffe07d319158 ""


I get "[diagnostic] cache_lock: blocked on 0xffffffe07d319158 """ errors when I try to login via ssh again (I guess sshd tries to read my key...)

So it was working fine, but after, say 1 day uptime it doesn't work anymore :(


-- 
You have received this notification because you have either subscribed to it, or are involved in it.
To change your notification preferences, please click here: http://bugs.dragonflybsd.org/my/account