Kernel KVM virtualization development
 help / color / mirror / Atom feed
From: Igor Mammedov <imammedo@redhat.com>
To: kvm@vger.kernel.org
Cc: pbonzini@redhat.com, Sean Christopherson <seanjc@google.com>
Subject: Re: [kvm-unit-tests PATCH 0/2] x86/apic: fix false test_apic_change_mode failures on stalled vCPUs
Date: Mon, 5 Oct 2026 17:23:29 +0200	[thread overview]
Message-ID: <20261005172329.4ce13193@imammedo> (raw)
In-Reply-To: <20260428133524.3628482-1-imammedo@redhat.com>

On Tue, 28 Apr 2026 15:35:22 +0200
Igor Mammedov <imammedo@redhat.com> wrote:

> test_apic_change_mode sporadically fails in CI on both Intel and AMD
> hosts with errors like:
>   "FAIL: TMCCT should have a non-zero value"
>   "FAIL: TMCCT should be reset to the initial-count"
>   "FAIL: TMCCT should not be reset to TMICT value"
> 
> The root cause is that the APIC timer runs at wall clock time under KVM.
> With the default tmict=0x999999 (~10ms period at 1ns bus cycle).
> 
> A vCPU stall for sufficiently large portion of TMICT leads to false positives
> (reasons could be: host preemption, cross-socket migration, heavy CPU
> contention). It's basically not possible to reliably sample timer values
> while it's running.
> 
> This series adds retry logic with increasing timer periods (10ms, 60ms,
> 700ms) so that transient vCPU stalls don't cause false failures, while
> real bugs still get caught. (most of false failures are handled by 60ms
> timer, and 700ms is one pathological case observed in a week of testing)

looks like this was lost,
ping

> 
> Reproducer (requires 2+ NUMA nodes):
> 
>   stress-ng --cpu 128 --timer 32 --hrtimers 32 --quiet &
>   sleep 2
>   while true; do
>       /usr/libexec/qemu-kvm --no-reboot -nodefaults \
>           -global kvm-pit.lost_tick_policy=discard \
>           -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 \
>           -display none -serial stdio -device pci-testdev \
>           -machine q35 -kernel x86/apic.flat \
>           -smp 1 -cpu qemu64,+x2apic,+tsc-deadline \
>           >> apic_race.log 2>&1 &  
>       QEMU_PID=$!
>       while kill -0 $QEMU_PID 2>/dev/null; do
>           taskset -p -c 0 $QEMU_PID 2>/dev/null
>           sleep 0.001
>           taskset -p -c 1 $QEMU_PID 2>/dev/null
>           sleep 0.001
>       done
>       wait $QEMU_PID 2>/dev/null
>   done
> 
> patches reduce ~4% failure rate (8 FAILs / 216 PASSes in 2 minutes).
> to 0 FAILs over thousands of runs.
> 
> Igor Mammedov (2):
>   x86/apic: separate reporting from actual measurements
>   x86/apic: add retry logic to test_apic_change_mode
> 
>  x86/apic.c | 119 ++++++++++++++++++++++++++++++++---------------------
>  1 file changed, 72 insertions(+), 47 deletions(-)
> 


      parent reply	other threads:[~2026-10-05 15:23 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-04-28 13:35 [kvm-unit-tests PATCH 0/2] x86/apic: fix false test_apic_change_mode failures on stalled vCPUs Igor Mammedov
2026-04-28 13:35 ` [kvm-unit-tests PATCH 1/2] x86/apic: separate reporting from actual measurements Igor Mammedov
2026-04-28 13:35 ` [kvm-unit-tests PATCH 2/2] x86/apic: add retry logic to test_apic_change_mode Igor Mammedov
2026-10-06  7:15   ` Sean Christopherson
2026-10-05 15:23 ` Igor Mammedov [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261005172329.4ce13193@imammedo \
    --to=imammedo@redhat.com \
    --cc=kvm@vger.kernel.org \
    --cc=pbonzini@redhat.com \
    --cc=seanjc@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox