From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EDC0342DFEC for ; Tue, 21 Jul 2026 19:35:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.71 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784662518; cv=none; b=DbpQs7iZZvxN1cKTybnAu04ufFZZT0paRBZhjLitzBBxepMcc1V9h/1e2xARB46GTtrU0+1/P5DExVTqo3CXDoSr+0vNtmTYoImMrdv88TLuq7HzenLfkD7fRVq4P7Zv6otdGMMsAx4SuJngfNT1VdCW2/5DBIxKxqLKh7DhAak= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784662518; c=relaxed/simple; bh=0KW+K+vUu9nZ0xvz5mpgniIIwmFWQyKRDfB2Y7d3pZI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XYkJy3KUuUKIGajNZ5v1qEej2GwnuYwTQkkkSwBS04UtR7OK9X2EZ3olpOnyaWpxJ6JgP8VvHfnMaiW5dTvzwhqfTVIBTjHzBXux4X73YRGpz8OyABFyBpxfoIJgeZ5irHY3uzOkOE2h1qnLIt6LNdstlTUs+ZtC1eZ/HizlLXA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=vanHNgec; arc=none smtp.client-ip=209.85.216.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vanHNgec" Received: by mail-pj1-f71.google.com with SMTP id 98e67ed59e1d1-388cfc4848dso14744469a91.3 for ; Tue, 21 Jul 2026 12:35:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784662516; x=1785267316; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=sAOhZx900WbQ/bQqjsgqEg24fHfXOh/5sLZRpMa2xuQ=; b=vanHNgecd6lDdefFg8i9nfO2ZQK13KcU5v5Ov2Xbxua+WbJNGfr5adCCsOYr+/fbJe 9pleQDEedZGuDQOHIooixCoUu9UjlRzQJvINzpuR+2XJUXMf6Xnk6dM1MJBfWTbQ3hME UcxFJ7XvUKVdmepN5shohrC4LU+hY/Ir9Tx56zu42iqUGAgYUSbg0eCbey2fJaeqyZNP wlvRWnUo3nJpZLyNsbWB9AdmGLYoTcD+E279Kp4sx3Ki4JINnNAm/M8UzHaUfr9YX/js ysOYqsGqozBs7h1CLqQFHGxftEGnj04+qQSvjnPjq4N60vdBldioOabYYDYVLoHzfYXU hXDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784662516; x=1785267316; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=sAOhZx900WbQ/bQqjsgqEg24fHfXOh/5sLZRpMa2xuQ=; b=BKPAHOIlkmt4qc+QLDunTH62Vlqpa0tcrJZGhUNmxDZEkZoZR0Tj4J93yOUg1pEZbH dhLk3za5huMjlRQXYuyXJ1HpsRQjyiJUGFOaKZcFS8Md/wHTRlgfmxODDxgirSWq/Wzg f6/EXX4zddVIPkogBCVVPETMSKWua5Od/Et2TEsPgjKbuVuku0gePkV41jCwKFmzWgVu XVBsnwfKmOBgG+kpEneiiL4L21qRi+34znNIx4lX4psmo6tXdM1Lg7Ag/djoZ3NBxD7C pLdUNV0OuyKPhC8hl6SVITAjKMIv4MF80AK2Kt25K967AvojGe5i2wFaZ3OamalBIPU4 m62g== X-Gm-Message-State: AOJu0Yxiiiyx1cGYi6OpWftytVplyvqlrMorBb3Alj1m8rEPQeUa6/zN W9R7vYIs2Q5/ONuVvDWVGiwcetD9GmTrzurFFZWftewzX2rMrOz0kYy8UIV/HH4JxQK2bi3M9Or GBVrM+Q== X-Received: from pji3.prod.google.com ([2002:a17:90b:3fc3:b0:38a:4c07:6209]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:274a:b0:38d:f94d:4c6e with SMTP id 98e67ed59e1d1-38e4b43120bmr21045776a91.17.1784662515994; Tue, 21 Jul 2026 12:35:15 -0700 (PDT) Date: Tue, 21 Jul 2026 12:35:15 -0700 In-Reply-To: <20260715125905.104313-1-davydov-max@yandex-team.ru> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715125905.104313-1-davydov-max@yandex-team.ru> Message-ID: Subject: Re: [PATCH] clocksource: stop monitoring TSC in VMs From: Sean Christopherson To: Maksim Davydov Cc: linux-kernel@vger.kernel.org, tglx@kernel.org, mingo@redhat.com, peterz@infradead.org, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com Content-Type: text/plain; charset="us-ascii" On Wed, Jul 15, 2026, Maksim Davydov wrote: > It seems that in a virtual machine hardware clocks shouldn't be monitored > by software (paravirtualized or emulated) clocks because they have > different behaviour. For example, a VM can be live-migrated to another > host or it can be paused. Both of them can cause false positive marking of > TSC as unstable: > > w/ kvm-clock: > clocksource: Marking clocksource tsc unstable due to frequency skew > clocksource: Watchdog kvm-clock interval: 301338133ns > clocksource: Clocksource tsc interval: 620070548ns > tsc: Marking TSC unstable due to clocksource watchdog > clocksource: Switched to clocksource kvm-clock > > w/o kvm-clock: > clocksource: Marking clocksource tsc unstable due to frequency skew > clocksource: Watchdog hpet interval: 248411930ns > clocksource: Clocksource tsc interval: 480045119ns > tsc: Marking TSC unstable due to clocksource watchdog > TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'. > sched_clock: Marking unstable (302400491443, 145482655)<-(302687230264, -141256934) > clocksource: Switched to clocksource hpet > > Both examples were created in the VM with invariant TSC, but without > TSC_ADJUST MSR in order to fail check in check_system_tsc_reliable(). So, > it is stable enough to be the best clocksource but not reliable enough > to be without a watchdog. > > The reason why TSC can be marked as unstable is the different way of > saving and restoring state of clocks. In virtualized environment TSC is a > "hardware" clock that QEMU doesn't stop during VM pause. On the other hand, > QEMU stops hpet and kvm-clock. So live-migration to paused state (e.g. in > order to hot-plug some devices) or short stop+cont can cause divergence > of TSC and any other "software" clocks. (The point to have the same > behaviour for all clocks will be discussed later in qemu-devel.) > > Nevertheless, it is important to point out, kvm_clock_read() used to touch > watchdog and that prevents false positive marking TSC as unstable, > because a hypervisor usually notifies a VM about clocks interference and > a guest OS usually checks the appropriate flags. This behaviour was changed > in 8739c6811572. But still, this issue has also existed (and now exists) > with other clocks. > > Thus, it seems that the clocksource watchdog has to be disabled for TSC > in VMs unless the opposite is explicitly requested by `tsc=watchdog`. > "Hardware" stable TSC is more realible than "software" clocks that > usually use host's TSC inside. > > Fixes: 8739c6811572 ("sched/clock/x86: Mark sched_clock() noinstr") > Signed-off-by: Maksim Davydov > --- > arch/x86/kernel/tsc.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c > index ce10ae4b298b..11b24f0c19bd 100644 > --- a/arch/x86/kernel/tsc.c > +++ b/arch/x86/kernel/tsc.c > @@ -1553,7 +1553,8 @@ void __init tsc_init(void) > return; > } > > - if (tsc_clocksource_reliable || tsc_watchdog == TSC_WATCHDOG_OFF) > + if (tsc_clocksource_reliable || tsc_watchdog == TSC_WATCHDOG_OFF || > + boot_cpu_has(X86_FEATURE_HYPERVISOR)) > tsc_disable_clocksource_watchdog(); I've been working on fixing this issue, along with a whole pile of other TSC-related virtualization issues, for ~1.5 years (yikes!). I'm hoping to land the series "soon', ideally in 7.3. Any testing/review you can provide that series would be much appreciated! https://lore.kernel.org/all/20260701193212.749551-23-seanjc@google.com > > clocksource_register_khz(&clocksource_tsc_early, tsc_khz); > -- > 2.34.1 >