From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7484D26B2DA for ; Wed, 9 Jul 2025 07:04:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752044698; cv=none; b=RxgcQBnj9oo8bI0DNFqfOQUZhq+Pku5GROtONF+ols0VEkXGLUtdBwDtrOKYVHd6pgFEpEvW/MhsY/Nm4aY4byH/VpOk3JWQSoZXY639TV+jru70rRCZuJ8wbwIjFWAk1cvRtDvR1cTH88n79lYnYEEbLY6Qh1ID3ipNrqkLW8w= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752044698; c=relaxed/simple; bh=HQE/ReeVyazvpH4LrBHiy8IMnxQNSg2ydx4fd5xrWGw=; h=Date:Message-Id:Mime-Version:Subject:From:To:Cc:Content-Type; b=Z4lsOcHygXNTXFh+qjukLCHeHtKo4ozf3agCWj4H/1N6qhcmMI/+dR/invkbZOngBr2AVfK+ERdAU6ZmR6aKL3MiWx7E3JjwWkFKHmQEiSnhkR0PBUcqZtKOgJ6T+0ySAqz+5NNSLJ2/sAhBEWNKLp4UhBlF512r82Cx+p/Dtqk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--suleiman.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fOy/yLxe; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--suleiman.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fOy/yLxe" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e8973c943d6so1133222276.0 for ; Wed, 09 Jul 2025 00:04:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1752044695; x=1752649495; darn=vger.kernel.org; h=cc:to:from:subject:mime-version:message-id:date:from:to:cc:subject :date:message-id:reply-to; bh=qU47TFvcnC6XmmQdhr6RzfJRAjN1DEPce8n7XkqyXcU=; b=fOy/yLxeqwoU26qFO5UjRd47r5dRLmyETZLCizGdEnsGT/nHhFDzGNabWbXZNPf0bQ YX/fnPmpk6bHDZNzX0FmDRw0xtp17HyvJce7F6pkaUjTGv4bpj7s+gPM1f/7yBpc/S2/ JFQQkjdAOe8CThpUxqhPPczS+2Lg17+LQsljITjND2u8ZW7/6rMAqmivQDfTfAh5aQ02 YRxL1Jrir8hVs53ZT502QL9NK3Xufx5p+Luivm7eBIF3ygB1I5WZD3rpL7dW7BSys+8B yv/iacVMeUcfWDzX8ZZ80nHevVajRJsFuirU6PTSvT94Q+xcqqS68nMPMSM72lh8v5HK TC4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752044695; x=1752649495; h=cc:to:from:subject:mime-version:message-id:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=qU47TFvcnC6XmmQdhr6RzfJRAjN1DEPce8n7XkqyXcU=; b=s73CWSYC3Tm749EK+aQGJaneElEUhmpX/Z7yM7gZIAgjXGkYYSFHMleYnTIyU3DwJV Ik0N0g9pwhe2sTxN249ApXmMgfIOgTkkvFh9/ikAQOqd6ADbwhRY6vwEPYj9sZvWj3Ia MOcfrR2SvtGxqO4yf4/gGG4oUE68JEQdOfGaCF5bMkD4dVXvCt6BIVQ2Jr2bX4cT03UK 8AsxEAiD/GTMn/69n2n5Og34LtYk598aNYRJ0fxw7j8wnEeVjkgeLiyLoBH2+eUaljl5 bqJ5Yis/BU8R1ktgsSy6QbbeJgv3X11QD0NL5H8hnUn45/kr3ryYrBtqYZWpLQk5wdEx uzrA== X-Forwarded-Encrypted: i=1; AJvYcCWI2E0Z/7TYz7TagHvdw3a2gQg3EqP+wF62Fjitnn9KmKSyQgVG8eBPUXjfX34bIPlAFJf5o0PAgyuGVJg=@vger.kernel.org X-Gm-Message-State: AOJu0YxVcJ2amGaIxrlVvUgT7OZdwlhk61ipDwb1o0IYPeunRLzZJUyX 9+E1m6RzzZloWZVayo6KqK1rmpVBjz6uIOt54nSxYmQZ3U9v/cRDsDLnA9XvcyZMNfKh3iY/8oF Zd6wG06/UeNUr0Q== X-Google-Smtp-Source: AGHT+IG4MF1b0UhYNvoPcg3YGZ372MuwkqCpX7pOrOVrYzNo/IMRhknvGRb0k0FharKF+y8Ie0BQS790QHc/vQ== X-Received: from suleiman1.tok.corp.google.com ([2401:fa00:8f:203:a92c:694f:82fe:62a]) (user=suleiman job=sendgmr) by 2002:a05:6902:458a:b0:e86:3ad2:43c with SMTP id 3f1490d57ef6-e8b6f4fef3fmr765276.5.1752044695263; Wed, 09 Jul 2025 00:04:55 -0700 (PDT) Date: Wed, 9 Jul 2025 16:04:47 +0900 Message-Id: <20250709070450.473297-1-suleiman@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Subject: [PATCH v6 0/3] KVM: x86: Include host suspended time in steal time From: Suleiman Souhlal To: Paolo Bonzini , Sean Christopherson Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Chao Gao , David Woodhouse , Sergey Senozhatsky , Konrad Rzeszutek Wilk , Tzung-Bi Shih , John Stultz , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, ssouhlal@freebsd.org, Suleiman Souhlal Content-Type: text/plain; charset="UTF-8" This series makes it so that the time that the host is suspended is included in guests' steal time. When the host resumes from a suspend, the guest thinks any task that was running during the suspend ran for a long time, even though the effective run time was much shorter, which can end up having negative effects with scheduling. To mitigate this issue, include the time that the host was suspended in steal time, if the guest requests it, which lets the guest subtract the duration from the tasks' runtime. Add new ABI to make this behavior opt-in per-guest. In addition, make the guest TSC behavior consistent whether the host TSC went backwards or not. v6: - Use true/false for bools. - Indentation. - Remove superfluous flag. - Use atomic operations for accumulating suspend duration. - Reuse generic vcpu block/kick infrastructure instead of rolling our own. - Add ABI to make the behavior opt-in per-guest. - Add command line parameter to make guest use this. - Reword commit messages in imperative mood. v5: https://lore.kernel.org/kvm/20250325041350.1728373-1-suleiman@google.com/ - Fix grammar mistakes in commit message. v4: https://lore.kernel.org/kvm/20250221053927.486476-1-suleiman@google.com/ - Advance guest TSC on suspends where host TSC goes backwards. - Block vCPUs from running until resume notifier. - Move suspend duration accounting out of machine-independent kvm to x86. - Merge code and documentation patches. - Reworded documentation. v3: https://lore.kernel.org/kvm/20250107042202.2554063-1-suleiman@google.com/ - Use PM notifier instead of syscore ops (kvm_suspend()/kvm_resume()), because the latter doesn't get called on shallow suspend. - Don't call function under UACCESS. - Whitespace. v2: https://lore.kernel.org/kvm/20240820043543.837914-1-suleiman@google.com/ - Accumulate suspend time at machine-independent kvm layer and track per-VCPU instead of per-VM. - Document changes. v1: https://lore.kernel.org/kvm/20240710074410.770409-1-suleiman@google.com/ Suleiman Souhlal (3): KVM: x86: Advance guest TSC after deep suspend. KVM: x86: Include host suspended duration in steal time KVM: x86: Add "suspendsteal" cmdline to request host to add suspend duration in steal time .../admin-guide/kernel-parameters.txt | 5 + Documentation/virt/kvm/x86/cpuid.rst | 4 + Documentation/virt/kvm/x86/msr.rst | 15 +++ arch/x86/include/asm/kvm_host.h | 4 + arch/x86/include/uapi/asm/kvm_para.h | 2 + arch/x86/kernel/kvm.c | 15 +++ arch/x86/kvm/cpuid.c | 4 +- arch/x86/kvm/x86.c | 108 +++++++++++++++++- 8 files changed, 150 insertions(+), 7 deletions(-) -- 2.50.0.727.gbf7dc18ff4-goog