Re: external module sched_in event

Avi Kivity
2007-12-23T17:37:40Z

Andrea Arcangeli wrote:
> On Sat, Dec 22, 2007 at 09:13:44PM +0200, Avi Kivity wrote:
>
>> Unfortunately, this fails badly on Intel i386:
>>
>
> Hmm ok there's a definitive bug that I forgot a int1 kernel->kernel
> switch on x86 has no special debug stack like on x86-64. This will
> have a better chance to work, hope I got all offsets right by
> memory.... At least the offset "32" in the leal and eax + fastcall
> should all be right or I doubt it could survive the double
> dereferencing. Likely the one-more-derefence didn't oops there because
> you likely have >=1g of ram and there was a 25% chance of crashing due
> the lack of sched-in and 75% chance of crashing in the
> one-more-dereference in a more meaningful way.
>
>

Now I see lots of

> BUG: warning at arch/i386/kernel/smp.c:701/smp_call_function_single()
> (Not tainted)
> [<f8c053bb>] __vcpu_clear+0x0/0x4a [kvm_intel]
> [<c0417ab9>] smp_call_function_single+0x90/0x10c
> [<c0403126>] __switch_to+0x174/0x18e
> [<f8c05614>] vcpu_clear+0x41/0x50 [kvm_intel]
> [<f8c058a5>] vmx_vcpu_load+0x2e/0x103 [kvm_intel]
> [<f8c0516d>] vmx_vcpu_put+0xc0/0xf3 [kvm_intel]
> [<f8c5f744>] kvm_arch_vcpu_load+0x9/0xa [kvm]
> [<f8c6b961>] preempt_notifier_trigger+0x5b/0xe1 [kvm]
> [<f8c6b79a>] pn_int1_handler+0x16/0x26 [kvm]
> [<c061fa14>] __mutex_lock_slowpath+0x45/0x77
> [<c061f8ff>] mutex_lock+0x26/0x29
> [<f8c6a465>] apic_update_ppr+0x17/0x3e [kvm]
> [<f8c650ed>] kvm_mmu_page_fault+0x14/0x9b [kvm]
> [<f8c6a55a>] kvm_get_apic_interrupt+0x3a/0x4f [kvm]
> [<f8c06cdb>] kvm_handle_exit+0x6a/0x86 [kvm_intel]
> [<f8c623cb>] kvm_arch_vcpu_ioctl_run+0x2a4/0x3aa [kvm]
> [<f8c5f246>] kvm_vcpu_ioctl+0xce/0x298 [kvm]
> [<c0420e83>] __activate_task+0x1c/0x29
> [<c0422645>] try_to_wake_up+0x3aa/0x3b4
> [<c06205b5>] _spin_unlock_irq+0x5/0x7
> [<c041fb40>] __wake_up_common+0x32/0x55
> [<c0420a39>] __wake_up+0x32/0x43
> [<c043b367>] wake_futex+0x42/0x4c
> [<c043b61a>] futex_wake+0xa6/0xb0
> [<c043c233>] do_futex+0x217/0xb7d
> [<f88626e5>] journal_stop+0x1cb/0x1d7 [jbd]
> [<c045addb>] mapping_tagged+0x2b/0x32
> [<f8c5ee89>] kvm_vm_ioctl+0x172/0x183 [kvm]
> [<c06205b5>] _spin_unlock_irq+0x5/0x7
> [<c061ef69>] __sched_text_start+0x999/0xa21
> [<c0419d4e>] smp_apic_timer_interrupt+0x76/0x80
> [<f8c5f178>] kvm_vcpu_ioctl+0x0/0x298 [kvm]
> [<c047c4a7>] do_ioctl+0x1f/0x62
> [<c047c72e>] vfs_ioctl+0x244/0x256
> [<c047c78c>] sys_ioctl+0x4c/0x64
> [<c0403f64>] syscall_call+0x7/0xb
> =======================

The sched_in notifier needs to enable interrupts (but it must disable
preemption to avoid recursion).

Eventually I got this:

BUG: spinlock lockup on CPU#3, qemu-system-x86/4425, c07001cc (Not tainted)
[<f8c053bb>] __vcpu_clear+0x0/0x4a [kvm_intel]
[<c04edec8>] _raw_spin_lock+0xb8/0xd9
[<c0417ac3>] smp_call_function_single+0x9a/0x10c
[<c0403126>] __switch_to+0x174/0x18e
[<f8c05614>] vcpu_clear+0x41/0x50 [kvm_intel]
[<f8c058a5>] vmx_vcpu_load+0x2e/0x103 [kvm_intel]
[<f8c0516d>] vmx_vcpu_put+0xc0/0xf3 [kvm_intel]
[<f8c5f744>] kvm_arch_vcpu_load+0x9/0xa [kvm]
[<f8c6b961>] preempt_notifier_trigger+0x5b/0xe1 [kvm]
[<f8c6b79a>] pn_int1_handler+0x16/0x26 [kvm]
[<c061fa14>] __mutex_lock_slowpath+0x45/0x77
[<c061f8ff>] mutex_lock+0x26/0x29
[<f8c6a465>] apic_update_ppr+0x17/0x3e [kvm]
[<f8c650ed>] kvm_mmu_page_fault+0x14/0x9b [kvm]
[<f8c6a55a>] kvm_get_apic_interrupt+0x3a/0x4f [kvm]
[<f8c06cdb>] kvm_handle_exit+0x6a/0x86 [kvm_intel]
[<f8c623cb>] kvm_arch_vcpu_ioctl_run+0x2a4/0x3aa [kvm]
[<c05e6d63>] tcp_sendmsg+0x913/0xa04
[<f8c5f246>] kvm_vcpu_ioctl+0xce/0x298 [kvm]
[<c0420e83>] __activate_task+0x1c/0x29
[<c0422645>] try_to_wake_up+0x3aa/0x3b4
[<c06205b5>] _spin_unlock_irq+0x5/0x7
[<c061ef69>] __sched_text_start+0x999/0xa21
[<c047d5e5>] core_sys_select+0x218/0x2f3
[<c043b61a>] futex_wake+0xa6/0xb0
[<c043c233>] do_futex+0x217/0xb7d
[<c042faa1>] __dequeue_signal+0xff/0x14e
[<c0430ca0>] dequeue_signal+0x36/0xae
[<f8c5ee89>] kvm_vm_ioctl+0x172/0x183 [kvm]
[<c043a03a>] ktime_get_ts+0x16/0x44
[<c043a07a>] ktime_get+0x12/0x34
[<c043698b>] common_timer_get+0xf4/0x130
[<f8c5f178>] kvm_vcpu_ioctl+0x0/0x298 [kvm]
[<c047c4a7>] do_ioctl+0x1f/0x62
[<c047c72e>] vfs_ioctl+0x244/0x256
[<c04ed2c0>] copy_to_user+0x3c/0x50
[<c047c78c>] sys_ioctl+0x4c/0x64
[<c0403f64>] syscall_call+0x7/0xb
=======================

followed by lockup of the qemu process, but it may be due to interrupts
being disabled.

--
error compiling committee.c: too many arguments to function