[PATCH v2] KVM: x86: Destroy the PIC and IOAPIC before destroying vCPUs

[PATCH v2] KVM: x86: Destroy the PIC and IOAPIC before destroying vCPUs

[Weiming Shi]

kvm_ioapic_eoi_inject_work() re-delivers a throttled level-triggered
interrupt via kvm_irq_delivery_to_apic(), which walks kvm->arch.apic_map
and dereferences the destination vCPU's APIC.  The work is cancelled only
in kvm_ioapic_destroy(), which runs after kvm_destroy_vcpus() has freed
the vCPUs and their APICs.  kvm_free_lapic() does not rebuild apic_map, so
the map is left with dangling pointers, and a work item that fires during
that window reads freed memory:

 BUG: KASAN: slab-use-after-free in __kvm_irq_delivery_to_apic_fast (arch/x86/kvm/lapic.c:1248)
 Read of size 8 by task kworker/3:1
 Workqueue: events kvm_ioapic_eoi_inject_work
  __kvm_irq_delivery_to_apic_fast (arch/x86/kvm/lapic.c:1248)
  __kvm_irq_delivery_to_apic (arch/x86/kvm/lapic.c:1343)
  ioapic_service (arch/x86/kvm/ioapic.c:492)
  kvm_ioapic_eoi_inject_work (arch/x86/kvm/ioapic.c:525)
  process_one_work

 Freed by task 153:
  kvm_arch_vcpu_destroy (arch/x86/kvm/x86.c:12871)
  kvm_destroy_vcpus (virt/kvm/kvm_main.c:489)
  kvm_arch_destroy_vm (arch/x86/kvm/x86.c:13402)
  kvm_destroy_vm (virt/kvm/kvm_main.c:1302)
  kvm_vm_release (virt/kvm/kvm_main.c:1363)

A guest arms the work by EOIing a level-triggered pin 10000 times in a
row, so the window is reachable from guest ring 0 whenever its VM is torn
down soon after.

Destroy the in-kernel PIC and IOAPIC in kvm_arch_pre_destroy_vm(),
before vCPUs are freed, so the eoi_inject work is cancelled while the
target APICs are still valid.  This also unregisters the PIC/IOAPIC
MMIO devices while the KVM buses still exist; kvm_destroy_vm() tears
the buses down right after kvm_free_irq_routing() and before
kvm_arch_destroy_vm(), so the previous kvm_io_bus_unregister_dev() in
kvm_ioapic_destroy() was a no-op.

Fixes: 184564efae4d ("kvm: ioapic: conditionally delay irq delivery duringeoi broadcast")
Link: https://lore.kernel.org/all/88ba60ad32ba851426a3f6590b0e402210991b4a.e33b58ce.0008.4e1c.aa62.c1024b242cbf@bytedance.com/
Suggested-by: Kai Huang <kai.huang@intel.com>
Reported-by: Zhong Wang <wangzhong.c0ss4ck@bytedance.com>
Reported-by: Xuanqing Shi <shixuanqing.11@bytedance.com>
Signed-off-by: Weiming Shi <bestswngs@gmail.com>
---
v1: https://lore.kernel.org/all/20260705045450.1325048-2-bestswngs@gmail.com/

v2:
 - Per Kai's suggestion, instead of adding a kvm_ioapic_pre_destroy()
   helper that only cancels the eoi_inject work, move
   kvm_pic_destroy()/kvm_ioapic_destroy() as a whole into
   kvm_arch_pre_destroy_vm().  This also fixes the stale
   kvm_io_bus_unregister_dev() Kai pointed out.
 arch/x86/kvm/x86.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index fd1c4a36b593..5925da351a9a 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -13370,10 +13370,14 @@ void kvm_arch_pre_destroy_vm(struct kvm *kvm)
 	 * Stop all background workers and kthreads before destroying vCPUs, as
 	 * iterating over vCPUs in a different task while vCPUs are being freed
 	 * is unsafe, i.e. will lead to use-after-free.  The PIT also needs to
-	 * be stopped before IRQ routing is freed.
+	 * be stopped before IRQ routing is freed.  The PIC and IOAPIC need to
+	 * be destroyed here too, for the same reason, and because they must
+	 * be destroyed before the KVM buses are torn down.
 	 */
 #ifdef CONFIG_KVM_IOAPIC
 	kvm_free_pit(kvm);
+	kvm_pic_destroy(kvm);
+	kvm_ioapic_destroy(kvm);
 #endif
 
 	kvm_mmu_pre_destroy_vm(kvm);
@@ -13400,10 +13404,6 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 		perf_release_mediated_pmu();
 	kvm_destroy_vcpus(kvm);
 	kvm_free_msr_filter(srcu_dereference_check(kvm->arch.msr_filter, &kvm->srcu, 1));
-#ifdef CONFIG_KVM_IOAPIC
-	kvm_pic_destroy(kvm);
-	kvm_ioapic_destroy(kvm);
-#endif
 	kvfree(rcu_dereference_check(kvm->arch.apic_map, 1));
 	kfree(srcu_dereference_check(kvm->arch.pmu_event_filter, &kvm->srcu, 1));
 	kvm_mmu_uninit_vm(kvm);
-- 
2.54.0

Re: [PATCH v2] KVM: x86: Destroy the PIC and IOAPIC before destroying vCPUs

[Sean Christopherson]

On Tue, Jul 07, 2026, Weiming Shi wrote:
> kvm_ioapic_eoi_inject_work() re-delivers a throttled level-triggered
> interrupt via kvm_irq_delivery_to_apic(), which walks kvm->arch.apic_map
> and dereferences the destination vCPU's APIC.  The work is cancelled only
> in kvm_ioapic_destroy(), which runs after kvm_destroy_vcpus() has freed
> the vCPUs and their APICs.  kvm_free_lapic() does not rebuild apic_map, so
> the map is left with dangling pointers, and a work item that fires during
> that window reads freed memory:
> 
>  BUG: KASAN: slab-use-after-free in __kvm_irq_delivery_to_apic_fast (arch/x86/kvm/lapic.c:1248)
>  Read of size 8 by task kworker/3:1
>  Workqueue: events kvm_ioapic_eoi_inject_work
>   __kvm_irq_delivery_to_apic_fast (arch/x86/kvm/lapic.c:1248)
>   __kvm_irq_delivery_to_apic (arch/x86/kvm/lapic.c:1343)
>   ioapic_service (arch/x86/kvm/ioapic.c:492)
>   kvm_ioapic_eoi_inject_work (arch/x86/kvm/ioapic.c:525)
>   process_one_work
> 
>  Freed by task 153:
>   kvm_arch_vcpu_destroy (arch/x86/kvm/x86.c:12871)
>   kvm_destroy_vcpus (virt/kvm/kvm_main.c:489)
>   kvm_arch_destroy_vm (arch/x86/kvm/x86.c:13402)
>   kvm_destroy_vm (virt/kvm/kvm_main.c:1302)
>   kvm_vm_release (virt/kvm/kvm_main.c:1363)
> 
> A guest arms the work by EOIing a level-triggered pin 10000 times in a
> row, so the window is reachable from guest ring 0 whenever its VM is torn
> down soon after.
> 
> Destroy the in-kernel PIC and IOAPIC in kvm_arch_pre_destroy_vm(),
> before vCPUs are freed, so the eoi_inject work is cancelled while the
> target APICs are still valid.  This also unregisters the PIC/IOAPIC
> MMIO devices while the KVM buses still exist; kvm_destroy_vm() tears
> the buses down right after kvm_free_irq_routing() and before
> kvm_arch_destroy_vm(), so the previous kvm_io_bus_unregister_dev() in
> kvm_ioapic_destroy() was a no-op.
> 
> Fixes: 184564efae4d ("kvm: ioapic: conditionally delay irq delivery duringeoi broadcast")
> Link: https://lore.kernel.org/all/88ba60ad32ba851426a3f6590b0e402210991b4a.e33b58ce.0008.4e1c.aa62.c1024b242cbf@bytedance.com/
> Suggested-by: Kai Huang <kai.huang@intel.com>
> Reported-by: Zhong Wang <wangzhong.c0ss4ck@bytedance.com>
> Reported-by: Xuanqing Shi <shixuanqing.11@bytedance.com>
> Signed-off-by: Weiming Shi <bestswngs@gmail.com>
> ---
> v1: https://lore.kernel.org/all/20260705045450.1325048-2-bestswngs@gmail.com/
> 
> v2:
>  - Per Kai's suggestion, instead of adding a kvm_ioapic_pre_destroy()
>    helper that only cancels the eoi_inject work, move
>    kvm_pic_destroy()/kvm_ioapic_destroy() as a whole into
>    kvm_arch_pre_destroy_vm().  This also fixes the stale
>    kvm_io_bus_unregister_dev() Kai pointed out.

Sadly, it creates an even easier-to-exploit NULL pointer deref.  Your v1 is what
I came up with idependently (the bug got reported off-list), though I eschewed a
helper.

Author:     Sean Christopherson <seanjc@google.com>
AuthorDate: Tue Jun 30 08:13:42 2026 -0700
Commit:     Sean Christopherson <seanjc@google.com>
CommitDate: Tue Jun 30 08:28:19 2026 -0700

    KVM: x86: Cancel delayed I/O APIC EOI handling before destroying vCPUs
    
    Cancel (and flush) the I/O APIC's delayed EOI handling work during the
    "pre VM destroy" phase, before vCPUs are destroyed, as processing the EOI
    broadcast will inject another IRQ if the line is asserted, i.e. will try
    to deliver an IRQ to the target vCPU(s).  Canceling the work after vCPUs
    are destroyed leads to UAF if the delayed work is processed after vCPUs are
    destroyed.
    
      BUG: KASAN: slab-use-after-free in __kvm_irq_delivery_to_apic_fast+0x9bf/0xa20 arch/x86/kvm/lapic.c:1250
      Read of size 8 at addr ffff8880499abea0 by task kworker/1:2/1218
    
      CPU: 1 UID: 0 PID: 1218 Comm: kworker/1:2 Not tainted 7.1.0-rc7 #5 PREEMPT(lazy)
      Hardware name: QEMU Ubuntu 25.10 PC v2 (i440FX + PIIX, + 10.1 machine, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
      Workqueue: events kvm_ioapic_eoi_inject_work
      Call Trace:
       <TASK>
       __dump_stack /root/linux/lib/dump_stack.c:94
       dump_stack_lvl+0x100/0x190 /root/linux/lib/dump_stack.c:120
       print_address_description /root/linux/mm/kasan/report.c:378
       print_report+0x139/0x4ad /root/linux/mm/kasan/report.c:482
       kasan_report+0xe4/0x1d0 /root/linux/mm/kasan/report.c:595
       __kvm_irq_delivery_to_apic_fast+0x9bf/0xa20 /root/linux/arch/x86/kvm/lapic.c:1250
       __kvm_irq_delivery_to_apic+0xd8/0xbf0 /root/linux/arch/x86/kvm/lapic.c:1345
       kvm_irq_delivery_to_apic /root/linux/arch/x86/kvm/lapic.h:129
       ioapic_service+0x308/0x590 /root/linux/arch/x86/kvm/ioapic.c:492
       kvm_ioapic_eoi_inject_work+0x13c/0x190 /root/linux/arch/x86/kvm/ioapic.c:532
       process_one_work+0xa59/0x19a0 /root/linux/kernel/workqueue.c:3314
       process_scheduled_works /root/linux/kernel/workqueue.c:3397
       worker_thread+0x5eb/0xe50 /root/linux/kernel/workqueue.c:3478
       kthread+0x370/0x450 /root/linux/kernel/kthread.c:436
       ret_from_fork+0x72b/0xd30 /root/linux/arch/x86/kernel/process.c:158
       ret_from_fork_asm+0x1a/0x30 /root/linux/arch/x86/entry/entry_64.S:245
       </TASK>
    
    Note, the VM is unreachable once kvm_destroy_vm() starts, and scheduling
    new work via kvm_ioapic_send_eoi() can only be done via KVM_RUN, i.e.
    requires a live vCPU.
    
    Alternatively, KVM could simply destroy the I/O APIC during the "pre" phase
    of VM destruction, but that gets more than a bit sketchy as KVM expects the
    I/O APIC to exist if ioapic_in_kernel() is true, and nested virtualization
    in particular has a bad habit of touching VM-scope state during vCPU
    destruction.  E.g. attempting to free the PIC during the pre phase would
    lead to a NULL pointer dereference in kvm_cpu_has_extint(), and it's not
    hard to imagine the I/O APIC having a similar flaw.
    
    Fixes: 17bcd7144263 ("KVM: x86: Free vCPUs before freeing VM state")
    Reported-by: <zdi-disclosures@trendmicro.com>
    Cc: stable@vger.kernel.org
    Signed-off-by: Sean Christopherson <seanjc@google.com>

diff --git arch/x86/kvm/x86.c arch/x86/kvm/x86.c
index 0626e835e9eb..a0cc74c8ded1 100644
--- arch/x86/kvm/x86.c
+++ arch/x86/kvm/x86.c
@@ -9942,6 +9942,8 @@ void kvm_arch_pre_destroy_vm(struct kvm *kvm)
         */
 #ifdef CONFIG_KVM_IOAPIC
        kvm_free_pit(kvm);
+       if (kvm->arch.vioapic)
+               cancel_delayed_work_sync(&kvm->arch.vioapic->eoi_inject);
 #endif
 
        kvm_mmu_pre_destroy_vm(kvm);

Re: [PATCH v2] KVM: x86: Destroy the PIC and IOAPIC before destroying vCPUs

[Huang, Kai]

>     Alternatively, KVM could simply destroy the I/O APIC during the "pre" phase
>     of VM destruction, but that gets more than a bit sketchy as KVM expects the
>     I/O APIC to exist if ioapic_in_kernel() is true, and nested virtualization
>     in particular has a bad habit of touching VM-scope state during vCPU
>     destruction.  E.g. attempting to free the PIC during the pre phase would
>     lead to a NULL pointer dereference in kvm_cpu_has_extint(), and it's not
>     hard to imagine the I/O APIC having a similar flaw.

Hmm seems vmx_vcpu_free() can eventually call into kvm_cpu_has_extint() via
nested_vmx_vmexit().  Thanks for pointing out.

>     
>     Fixes: 17bcd7144263 ("KVM: x86: Free vCPUs before freeing VM state")
>     Reported-by: <zdi-disclosures@trendmicro.com>
>     Cc: stable@vger.kernel.org
>     Signed-off-by: Sean Christopherson <seanjc@google.com>
> 
> diff --git arch/x86/kvm/x86.c arch/x86/kvm/x86.c
> index 0626e835e9eb..a0cc74c8ded1 100644
> --- arch/x86/kvm/x86.c
> +++ arch/x86/kvm/x86.c
> @@ -9942,6 +9942,8 @@ void kvm_arch_pre_destroy_vm(struct kvm *kvm)
>          */
>  #ifdef CONFIG_KVM_IOAPIC
>         kvm_free_pit(kvm);
> +       if (kvm->arch.vioapic)
> +               cancel_delayed_work_sync(&kvm->arch.vioapic->eoi_inject);

Maybe add a comment to explain why we cannot destroy IOAPIC and PIC here?

Re: [PATCH v2] KVM: x86: Destroy the PIC and IOAPIC before destroying vCPUs

[Sean Christopherson]

On Mon, Jul 06, 2026, Kai Huang wrote:
> 
> >     Alternatively, KVM could simply destroy the I/O APIC during the "pre" phase
> >     of VM destruction, but that gets more than a bit sketchy as KVM expects the
> >     I/O APIC to exist if ioapic_in_kernel() is true, and nested virtualization
> >     in particular has a bad habit of touching VM-scope state during vCPU
> >     destruction.  E.g. attempting to free the PIC during the pre phase would
> >     lead to a NULL pointer dereference in kvm_cpu_has_extint(), and it's not
> >     hard to imagine the I/O APIC having a similar flaw.
> 
> Hmm seems vmx_vcpu_free() can eventually call into kvm_cpu_has_extint() via
> nested_vmx_vmexit().  Thanks for pointing out.

Yeah, I found out the hard way :-)

> >     Fixes: 17bcd7144263 ("KVM: x86: Free vCPUs before freeing VM state")
> >     Reported-by: <zdi-disclosures@trendmicro.com>
> >     Cc: stable@vger.kernel.org
> >     Signed-off-by: Sean Christopherson <seanjc@google.com>
> > 
> > diff --git arch/x86/kvm/x86.c arch/x86/kvm/x86.c
> > index 0626e835e9eb..a0cc74c8ded1 100644
> > --- arch/x86/kvm/x86.c
> > +++ arch/x86/kvm/x86.c
> > @@ -9942,6 +9942,8 @@ void kvm_arch_pre_destroy_vm(struct kvm *kvm)
> >          */
> >  #ifdef CONFIG_KVM_IOAPIC
> >         kvm_free_pit(kvm);
> > +       if (kvm->arch.vioapic)
> > +               cancel_delayed_work_sync(&kvm->arch.vioapic->eoi_inject);
> 
> Maybe add a comment to explain why we cannot destroy IOAPIC and PIC here?

Hmm, agreed, the block comment above can/should be extended to explain why it's
ok to destroy the PIT, but not the PIC or I/O APIC.

Re: [PATCH v2] KVM: x86: Destroy the PIC and IOAPIC before destroying vCPUs

[Huang, Kai]

On Mon, 2026-07-06 at 15:10 -0700, Sean Christopherson wrote:
> On Mon, Jul 06, 2026, Kai Huang wrote:
> > 
> > >     Alternatively, KVM could simply destroy the I/O APIC during the "pre" phase
> > >     of VM destruction, but that gets more than a bit sketchy as KVM expects the
> > >     I/O APIC to exist if ioapic_in_kernel() is true, and nested virtualization
> > >     in particular has a bad habit of touching VM-scope state during vCPU
> > >     destruction.  E.g. attempting to free the PIC during the pre phase would
> > >     lead to a NULL pointer dereference in kvm_cpu_has_extint(), and it's not
> > >     hard to imagine the I/O APIC having a similar flaw.
> > 
> > Hmm seems vmx_vcpu_free() can eventually call into kvm_cpu_has_extint() via
> > nested_vmx_vmexit().  Thanks for pointing out.
> 
> Yeah, I found out the hard way :-)

:-)

Btw, I don't think the order of kvm_free_irq_routine() and destroying IOAPIC/PIC
is a concern here?  As you mentioned, once kvm_destroy_vm() starts, VM is not
touchable from userspace, therefore it should not be possible to have code path
which can reference vIOAPIC/vPIC after they got destroyed but before
kvm_free_irq_routine() through IRQ routine table?

Re: [PATCH v2] KVM: x86: Destroy the PIC and IOAPIC before destroying vCPUs

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Use-after-free on device error path in kvm_create_pit().
--

commit 738d33431d0b6932537e95c21eed6b24c1201219
Author: Weiming Shi <bestswngs@gmail.com>

KVM: x86: Destroy the PIC and IOAPIC before destroying vCPUs

This commit moves the destruction of the in-kernel PIC and IOAPIC to
kvm_arch_pre_destroy_vm(), before vCPUs are freed. This ensures that the
eoi_inject work is cancelled while target APICs remain valid. It prevents
a use-after-free where a delayed work item could dereference a freed vCPU's
APIC map.

Link: https://lore.kernel.org/all/88ba60ad32ba851426a3f6590b0e402210991b4a.e33b58ce.0008.4e1c.aa62.c1024b242cbf@bytedance.com/

> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index afcac1042947a..38ac0e0a3f769 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -13428,10 +13428,14 @@ void kvm_arch_pre_destroy_vm(struct kvm *kvm)
>  	 * Stop all background workers and kthreads before destroying vCPUs, as
>  	 * iterating over vCPUs in a different task while vCPUs are being freed
>  	 * is unsafe, i.e. will lead to use-after-free.  The PIT also needs to
> -	 * be stopped before IRQ routing is freed.
> +	 * be stopped before IRQ routing is freed.  The PIC and IOAPIC need to
> +	 * be destroyed here too, for the same reason, and because they must
> +	 * be destroyed before the KVM buses are torn down.
>  	 */
>  #ifdef CONFIG_KVM_IOAPIC
>  	kvm_free_pit(kvm);
[Severity: High]
This isn't a bug introduced by this patch, but since we are touching the PIT
and IOAPIC destruction paths here, is there a potential use-after-free on the
device error path in kvm_create_pit()?

If host userspace creates a VM, initializes the IRQCHIP, creates a vCPU,
and starts running it, the vCPU can perform PIO exits, calling
kvm_io_bus_read() and acquiring srcu_read_lock(&kvm->srcu).

If userspace concurrently calls KVM_CREATE_PIT2 with the
KVM_PIT_SPEAKER_DUMMY flag, the kernel registers pit->dev on the PIO bus.
The vCPU thread could pick this up.

If the subsequent registration of pit->speaker_dev fails, the error path
unregisters pit->dev but frees it without waiting for SRCU readers:

arch/x86/kvm/i8254.c:kvm_create_pit() {
    ...
fail_register_speaker:
	kvm_io_bus_unregister_dev(kvm, KVM_PIO_BUS, &pit->dev);
fail_register_pit:
	mutex_unlock(&kvm->slots_lock);
	kvm_pit_set_reinject(pit, false);
	kthread_destroy_worker(pit->worker);
fail_kthread:
	kfree(pit);
	return NULL;
}

Could the vCPU thread, while iterating over the old bus, dereference
pit->dev->ops and access the freed memory?

> +	kvm_pic_destroy(kvm);
> +	kvm_ioapic_destroy(kvm);
>  #endif

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260706180025.2735341-3-bestswngs@gmail.com?part=1