livelock in lwkt_setcpu_remote()

["Magliano Andre'" <masterblaster@xxxxxxxxxx>]

Hi all,

i'm currently porting conky (a nice system monitor utility) to dfly, and
when running the binary i'm constantly incurring in following livelock:

lwkt_setcpu_remote: td->td_flags 00800621
lwkt_setcpu_remote: td->td_flags 00800621
lwkt_setcpu_remote: td->td_flags 00800621
. ..

in /sys/kern/lwkt_thread.c lwkt_setcpu_remote() says

    while (td->td_flags & (TDF_RUNNING|TDF_PREEMPT_LOCK)) {
#ifdef SMP
        lwkt_process_ipiq();
#endif
        cpu_lfence();
        cpu_pause();
        if (--retry == 0) {
                kprintf("lwkt_setcpu_remote: td->td_flags %08x\n",
                        td->td_flags);
                retry = 10000000;
        }
    }

and it doesn't get out of 'while' anymore.


I'm runnning v2.9.1.890.g8e5e11-DEVELOPMENT on an atom N510 board (2
cores, 2 logical cpus each)

I reproduced the problem with a later v2.9.1.951.g6eaa5-DEVELOPMENT
(pulled some hours ago) on VitualBox with SMP kernel and 2 cpus
configured (it's just a matter of seconds).

The problem appears with VirtualBox even with non SMP kernel and 1 cpu
configured, so maybe isn't strictly SMP related.


A console stack trace from VirtualBox simulation and making '#if 1' out
of '#if 0' for debugging porpuses according to commit
cfaeae2ac9bf2ac78367308f7070d4a11c1680d2 (plus td->td_release %p printed
too)

cpu 0 ipiq maxed cscount 0 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 1 ipiq maxed cscount 1 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 0 ipiq maxed cscount 0 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 1 ipiq maxed cscount 1 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 0 ipiq maxed cscount 0 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 1 ipiq maxed cscount 1 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
lwkt_setcpu_remote: td->td_release 0, td->td_flags 00000621
cpu 0 ipiq maxed cscount 0 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 1 ipiq maxed cscount 1 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 0 ipiq maxed cscount 0 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 1 ipiq maxed cscount 1 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 0 ipiq maxed cscount 0 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 1 ipiq maxed cscount 1 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 0 ipiq maxed cscount 0 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
lwkt_setcpu_remote: td->td_release 0, td->td_flags 00000621
cpu 0 ipiq maxed cscount 0 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 1 ipiq maxed cscount 1 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 0 ipiq maxed cscount 0 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 1 ipiq maxed cscount 1 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 0 ipiq maxed cscount 0 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 1 ipiq maxed cscount 1 spin 0
 pmap_interlock_wait lwkt_setcpu_remote
cpu 0 ipiq maxed cscount 0 spin 0
panic: cpu 0 ipiq maxed
cpuid = 0
Trace beginning at frame 0xcc17ebb8
panic(ffffffff,0,c045dc7f,cc17ebf0,c9825818) at panic+0x1a2
panic(c045dc7f,0,0,0,cc17ec10) at panic+0x1a2
lwkt_process_ipiq_core(1,ff800110,c16bc000,cc019100,0) at
lwkt_process_ipiq_core+0xda
lwkt_process_ipiq(cc0ac9d8,1,cc218b54,c03d86c0,cc019100) at
lwkt_process_ipiq+0x7e
pmap_interlock_wait(cc019100) at pmap_interlock_wait+0x37
cpu_heavy_restore(ff809000,c01e8b3b,cc0ac9d8,0,0) at
cpu_heavy_restore+0x28
lwkt_setcpu_self(ff809000,0,d0c,0,0) at lwkt_setcpu_self+0xb9
sysctl_kern_proc(c04add00,0,0,cc218bf4,cc218bf4) at sysctl_kern_proc+0x294
sysctl_root(cc218bf4,cc218c28,4,cc0ac9d8,0) at sysctl_root+0x127
userland_sysctl(cc218c7c,3,0,bf6fbf48,0) at userland_sysctl+0x128
sys___sysctl(cc218cf0,cc218d00,18,cc0ac9d8,cc0ac9d8) at sys___sysctl+0x8f
syscall2(cc218d40) at syscall2+0x272
Xint0x80_syscall() at Xint0x80_syscall+0x36
Debugger("panic")

CPU0 stopping CPUs: 0x00000002
 stopped
Stopped at      Debugger+0x3f:  movb    $0,in_Debugger.7216

:(