* Re: [PATCH v3 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
@ 2021-07-20 9:10 kernel test robot
0 siblings, 0 replies; 4+ messages in thread
From: kernel test robot @ 2021-07-20 9:10 UTC (permalink / raw)
To: kbuild
[-- Attachment #1: Type: text/plain, Size: 20728 bytes --]
CC: kbuild-all(a)lists.01.org
In-Reply-To: <20210719184949.1385910-2-oupton@google.com>
References: <20210719184949.1385910-2-oupton@google.com>
TO: Oliver Upton <oupton@google.com>
TO: kvm(a)vger.kernel.org
TO: kvmarm(a)lists.cs.columbia.edu
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: Sean Christopherson <seanjc@google.com>
CC: Marc Zyngier <maz@kernel.org>
CC: Peter Shier <pshier@google.com>
CC: Jim Mattson <jmattson@google.com>
CC: David Matlack <dmatlack@google.com>
CC: Ricardo Koller <ricarkol@google.com>
CC: Jing Zhang <jingzhangos@google.com>
Hi Oliver,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on kvm/queue]
[also build test WARNING on vhost/linux-next v5.14-rc2 next-20210720]
[cannot apply to kvmarm/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Oliver-Upton/KVM-Add-idempotent-controls-for-migrating-system-counter-state/20210720-151644
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
:::::: branch date: 2 hours ago
:::::: commit date: 2 hours ago
compiler: gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
reproduce:
cd tools/perf && ./check-headers.sh
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
perfheadercheck warnings: (new ones prefixed by >>)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 283> /* Flags that describe what fields in emulation_failure hold valid data. */
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 284> #define KVM_INTERNAL_ERROR_EMULATION_FLAG_INSTRUCTION_BYTES (1ULL << 0)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 285>
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 389> /*
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 390> * KVM_INTERNAL_ERROR_EMULATION
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 391> *
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 392> * "struct emulation_failure" is an overlay of "struct internal"
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 393> * that is used for the KVM_INTERNAL_ERROR_EMULATION sub-type of
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 394> * KVM_EXIT_INTERNAL_ERROR. Note, unlike other internal error
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 395> * sub-types, this struct is ABI! It also needs to be backwards
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 396> * compatible with "struct internal". Take special care that
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 397> * "ndata" is correct, that new fields are enumerated in "flags",
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 398> * and that each flag enumerates fields that are 64-bit aligned
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 399> * and sized (so that ndata+internal.data[] is valid/accurate).
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 400> */
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 401> struct {
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 402> __u32 suberror;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 403> __u32 ndata;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 404> __u64 flags;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 405> __u8 insn_size;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 406> __u8 insn_bytes[15];
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 407> } emulation_failure;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1108> #define KVM_CAP_HYPERV_ENFORCE_CPUID 199
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1109> #define KVM_CAP_SREGS2 200
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1110> #define KVM_CAP_EXIT_HYPERCALL 201
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1111> #define KVM_CAP_PPC_RPT_INVALIDATE 202
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1112> #define KVM_CAP_BINARY_STATS_FD 203
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1113> #define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1114> #define KVM_CAP_ARM_MTE 205
>> Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1226> #define KVM_CLOCK_REALTIME (1 << 2)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1227> #define KVM_CLOCK_HOST_TSC (1 << 3)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1201< __u32 pad[9];
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1232> __u32 pad0;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1233> __u64 realtime;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1234> __u64 host_tsc;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1235> __u32 pad[4];
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1465> #define KVM_ARM_MTE_COPY_TAGS _IOR(KVMIO, 0xb4, struct kvm_arm_copy_mte_tags)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1659> #define KVM_GET_SREGS2 _IOR(KVMIO, 0xcc, struct kvm_sregs2)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1660> #define KVM_SET_SREGS2 _IOW(KVMIO, 0xcd, struct kvm_sregs2)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1661>
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1939>
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1940> /**
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1941> * struct kvm_stats_header - Header of per vm/vcpu binary statistics data.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1942> * @flags: Some extra information for header, always 0 for now.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1943> * @name_size: The size in bytes of the memory which contains statistics
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1944> * name string including trailing '\0'. The memory is allocated
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1945> * at the send of statistics descriptor.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1946> * @num_desc: The number of statistics the vm or vcpu has.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1947> * @id_offset: The offset of the vm/vcpu stats' id string in the file pointed
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1948> * by vm/vcpu stats fd.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1949> * @desc_offset: The offset of the vm/vcpu stats' descriptor block in the file
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1950> * pointd by vm/vcpu stats fd.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1951> * @data_offset: The offset of the vm/vcpu stats' data block in the file
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1952> * pointed by vm/vcpu stats fd.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1953> *
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1954> * This is the header userspace needs to read from stats fd before any other
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1955> * readings. It is used by userspace to discover all the information about the
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1956> * vm/vcpu's binary statistics.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1957> * Userspace reads this header from the start of the vm/vcpu's stats fd.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1958> */
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1959> struct kvm_stats_header {
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1960> __u32 flags;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1961> __u32 name_size;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1962> __u32 num_desc;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1963> __u32 id_offset;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1964> __u32 desc_offset;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1965> __u32 data_offset;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1966> };
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1967>
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1968> #define KVM_STATS_TYPE_SHIFT 0
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1969> #define KVM_STATS_TYPE_MASK (0xF << KVM_STATS_TYPE_SHIFT)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1970> #define KVM_STATS_TYPE_CUMULATIVE (0x0 << KVM_STATS_TYPE_SHIFT)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1971> #define KVM_STATS_TYPE_INSTANT (0x1 << KVM_STATS_TYPE_SHIFT)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1972> #define KVM_STATS_TYPE_PEAK (0x2 << KVM_STATS_TYPE_SHIFT)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1973> #define KVM_STATS_TYPE_MAX KVM_STATS_TYPE_PEAK
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1974>
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1975> #define KVM_STATS_UNIT_SHIFT 4
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1976> #define KVM_STATS_UNIT_MASK (0xF << KVM_STATS_UNIT_SHIFT)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1977> #define KVM_STATS_UNIT_NONE (0x0 << KVM_STATS_UNIT_SHIFT)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1978> #define KVM_STATS_UNIT_BYTES (0x1 << KVM_STATS_UNIT_SHIFT)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1979> #define KVM_STATS_UNIT_SECONDS (0x2 << KVM_STATS_UNIT_SHIFT)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1980> #define KVM_STATS_UNIT_CYCLES (0x3 << KVM_STATS_UNIT_SHIFT)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1981> #define KVM_STATS_UNIT_MAX KVM_STATS_UNIT_CYCLES
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1982>
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1983> #define KVM_STATS_BASE_SHIFT 8
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1984> #define KVM_STATS_BASE_MASK (0xF << KVM_STATS_BASE_SHIFT)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1985> #define KVM_STATS_BASE_POW10 (0x0 << KVM_STATS_BASE_SHIFT)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1986> #define KVM_STATS_BASE_POW2 (0x1 << KVM_STATS_BASE_SHIFT)
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version@'include/uapi/linux/kvm.h': 1987> #define KVM_STATS_BASE_MAX KVM_STATS_BASE_POW2
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1988>
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1989> /**
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1990> * struct kvm_stats_desc - Descriptor of a KVM statistics.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1991> * @flags: Annotations of the stats, like type, unit, etc.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1992> * @exponent: Used together with @flags to determine the unit.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1993> * @size: The number of data items for this stats.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1994> * Every data item is of type __u64.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1995> * @offset: The offset of the stats to the start of stat structure in
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1996> * struture kvm or kvm_vcpu.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1997> * @unused: Unused field for future usage. Always 0 for now.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1998> * @name: The name string for the stats. Its size is indicated by the
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 1999> * &kvm_stats_header->name_size.
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 2000> */
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 2001> struct kvm_stats_desc {
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 2002> __u32 flags;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 2003> __s16 exponent;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 2004> __u16 size;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 2005> __u32 offset;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 2006> __u32 unused;
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 2007> char name[];
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 2008> };
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 2009>
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h': 2010> #define KVM_GET_STATS_FD _IO(KVMIO, 0xce)
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
^ permalink raw reply [flat|nested] 4+ messages in thread* [PATCH v3 00/12] KVM: Add idempotent controls for migrating system counter state
@ 2021-07-19 18:49 Oliver Upton
2021-07-19 18:49 ` Oliver Upton
0 siblings, 1 reply; 4+ messages in thread
From: Oliver Upton @ 2021-07-19 18:49 UTC (permalink / raw)
To: kvm, kvmarm
Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier,
Sean Christopherson, David Matlack, Paolo Bonzini,
linux-arm-kernel, Jim Mattson
KVM's current means of saving/restoring system counters is plagued with
temporal issues. At least on ARM64 and x86, we migrate the guest's
system counter by-value through the respective guest system register
values (cntvct_el0, ia32_tsc). Restoring system counters by-value is
brittle as the state is not idempotent: the host system counter is still
oscillating between the attempted save and restore. Furthermore, VMMs
may wish to transparently live migrate guest VMs, meaning that they
include the elapsed time due to live migration blackout in the guest
system counter view. The VMM thread could be preempted for any number of
reasons (scheduler, L0 hypervisor under nested) between the time that
it calculates the desired guest counter value and when KVM actually sets
this counter state.
Despite the value-based interface that we present to userspace, KVM
actually has idempotent guest controls by way of system counter offsets.
We can avoid all of the issues associated with a value-based interface
by abstracting these offset controls in new ioctls. This series
introduces new vCPU device attributes to provide userspace access to the
vCPU's system counter offset.
Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK
ioctls to provide userspace with a (host_tsc, realtime) instant. This is
essential for a VMM to perform precise migration of the guest's system
counters.
Patches 2-3 add support for x86 by shoehorning the new controls into the
pre-existing synchronization heuristics.
Patches 4-5 implement a test for the new additions to
KVM_{GET,SET}_CLOCK.
Patches 6-7 implement at test for the tsc offset attribute introduced in
patch 3.
Patch 8 adds a device attribute for the arm64 virtual counter-timer
offset.
Patch 9 extends the test from patch 7 to cover the arm64 virtual
counter-timer offset.
Patch 10 adds a device attribute for the arm64 physical counter-timer
offset. Currently, this is implemented as a synthetic register, forcing
the guest to trap to the host and emulating the offset in the fast exit
path. Later down the line we will have hardware with FEAT_ECV, which
allows the hypervisor to perform physical counter-timer offsetting in
hardware (CNTPOFF_EL2).
Patch 11 extends the test from patch 7 to cover the arm64 physical
counter-timer offset.
Patch 12 introduces a benchmark to measure the overhead of emulation in
patch 10.
Physical counter benchmark
--------------------------
The following data was collected by running 10000 iterations of the
benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S
machine with 2 80-core Ampere Altra SoCs. Measurements were collected
for both VHE and nVHE operation using the `kvm-arm.mode=` command-line
parameter.
nVHE
----
+--------------------+--------+---------+
| Metric | Native | Trapped |
+--------------------+--------+---------+
| Average | 54ns | 148ns |
| Standard Deviation | 124ns | 122ns |
| 95th Percentile | 258ns | 348ns |
+--------------------+--------+---------+
VHE
---
+--------------------+--------+---------+
| Metric | Native | Trapped |
+--------------------+--------+---------+
| Average | 53ns | 152ns |
| Standard Deviation | 92ns | 94ns |
| 95th Percentile | 204ns | 307ns |
+--------------------+--------+---------+
This series applies cleanly to the following commit:
1889228d80fe ("KVM: selftests: smm_test: Test SMM enter from L2")
v1 -> v2:
- Reimplemented as vCPU device attributes instead of a distinct ioctl.
- Added the (realtime, host_tsc) instant support to KVM_{GET,SET}_CLOCK
- Changed the arm64 implementation to broadcast counter
offset values to all vCPUs in a guest. This upholds the
architectural expectations of a consistent counter-timer across CPUs.
- Fixed a bug with traps in VHE mode. We now configure traps on every
transition into a guest to handle differing VMs (trapped, emulated).
v2 -> v3:
- Added documentation for additions to KVM_{GET,SET}_CLOCK
- Added documentation for all new vCPU attributes
- Added documentation for suggested algorithm to migrate a guest's
TSC(s)
- Bug fixes throughout series
- Rename KVM_CLOCK_REAL_TIME -> KVM_CLOCK_REALTIME
v1: https://lore.kernel.org/kvm/20210608214742.1897483-1-oupton@google.com/
v2: https://lore.kernel.org/r/20210716212629.2232756-1-oupton@google.com
Oliver Upton (12):
KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
KVM: x86: Refactor tsc synchronization code
KVM: x86: Expose TSC offset controls to userspace
tools: arch: x86: pull in pvclock headers
selftests: KVM: Add test for KVM_{GET,SET}_CLOCK
selftests: KVM: Add helpers for vCPU device attributes
selftests: KVM: Introduce system counter offset test
KVM: arm64: Allow userspace to configure a vCPU's virtual offset
selftests: KVM: Add support for aarch64 to system_counter_offset_test
KVM: arm64: Provide userspace access to the physical counter offset
selftests: KVM: Test physical counter offsetting
selftests: KVM: Add counter emulation benchmark
Documentation/virt/kvm/api.rst | 42 +-
Documentation/virt/kvm/devices/vcpu.rst | 101 +++++
Documentation/virt/kvm/locking.rst | 11 +
arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/include/asm/kvm_hyp.h | 2 -
arch/arm64/include/asm/sysreg.h | 1 +
arch/arm64/include/uapi/asm/kvm.h | 2 +
arch/arm64/kvm/arch_timer.c | 118 ++++-
arch/arm64/kvm/arm.c | 4 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +
arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++
arch/arm64/kvm/hyp/nvhe/switch.c | 2 -
arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +-
arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 ++
arch/x86/include/asm/kvm_host.h | 4 +
arch/x86/include/uapi/asm/kvm.h | 4 +
arch/x86/kvm/x86.c | 422 ++++++++++++++----
include/kvm/arm_arch_timer.h | 2 -
include/uapi/linux/kvm.h | 7 +-
tools/arch/x86/include/asm/pvclock-abi.h | 48 ++
tools/arch/x86/include/asm/pvclock.h | 103 +++++
tools/testing/selftests/kvm/.gitignore | 3 +
tools/testing/selftests/kvm/Makefile | 4 +
.../kvm/aarch64/counter_emulation_benchmark.c | 215 +++++++++
.../selftests/kvm/include/aarch64/processor.h | 24 +
.../testing/selftests/kvm/include/kvm_util.h | 11 +
tools/testing/selftests/kvm/lib/kvm_util.c | 38 ++
.../kvm/system_counter_offset_test.c | 206 +++++++++
.../selftests/kvm/x86_64/kvm_clock_test.c | 210 +++++++++
29 files changed, 1549 insertions(+), 133 deletions(-)
create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.h
create mode 100644 tools/arch/x86/include/asm/pvclock-abi.h
create mode 100644 tools/arch/x86/include/asm/pvclock.h
create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
--
2.32.0.402.g57bb445576-goog
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 4+ messages in thread* [PATCH v3 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK 2021-07-19 18:49 [PATCH v3 00/12] KVM: Add idempotent controls for migrating system counter state Oliver Upton 2021-07-19 18:49 ` Oliver Upton @ 2021-07-19 18:49 ` Oliver Upton 0 siblings, 0 replies; 4+ messages in thread From: Oliver Upton @ 2021-07-19 18:49 UTC (permalink / raw) To: kvm, kvmarm Cc: Marc Zyngier, Raghavendra Rao Anata, Peter Shier, Sean Christopherson, David Matlack, Paolo Bonzini, linux-arm-kernel, Jim Mattson Handling the migration of TSCs correctly is difficult, in part because Linux does not provide userspace with the ability to retrieve a (TSC, realtime) clock pair for a single instant in time. In lieu of a more convenient facility, KVM can report similar information in the kvm_clock structure. Provide userspace with a host TSC & realtime pair iff the realtime clock is based on the TSC. If userspace provides KVM_SET_CLOCK with a valid realtime value, advance the KVM clock by the amount of elapsed time. Do not step the KVM clock backwards, though, as it is a monotonic oscillator. Signed-off-by: Oliver Upton <oupton@google.com> --- Documentation/virt/kvm/api.rst | 42 +++++++-- arch/x86/include/asm/kvm_host.h | 3 + arch/x86/kvm/x86.c | 149 ++++++++++++++++++++------------ include/uapi/linux/kvm.h | 7 +- 4 files changed, 137 insertions(+), 64 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index b9ddce5638f5..70ccf68cd574 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -993,20 +993,34 @@ such as migration. When KVM_CAP_ADJUST_CLOCK is passed to KVM_CHECK_EXTENSION, it returns the set of bits that KVM can return in struct kvm_clock_data's flag member. -The only flag defined now is KVM_CLOCK_TSC_STABLE. If set, the returned -value is the exact kvmclock value seen by all VCPUs at the instant -when KVM_GET_CLOCK was called. If clear, the returned value is simply -CLOCK_MONOTONIC plus a constant offset; the offset can be modified -with KVM_SET_CLOCK. KVM will try to make all VCPUs follow this clock, -but the exact value read by each VCPU could differ, because the host -TSC is not stable. +FLAGS: + +KVM_CLOCK_TSC_STABLE. If set, the returned value is the exact kvmclock +value seen by all VCPUs at the instant when KVM_GET_CLOCK was called. +If clear, the returned value is simply CLOCK_MONOTONIC plus a constant +offset; the offset can be modified with KVM_SET_CLOCK. KVM will try +to make all VCPUs follow this clock, but the exact value read by each +VCPU could differ, because the host TSC is not stable. + +KVM_CLOCK_REALTIME. If set, the `realtime` field in the kvm_clock_data +structure is populated with the value of the host's real time +clocksource at the instant when KVM_GET_CLOCK was called. If clear, +the `realtime` field does not contain a value. + +KVM_CLOCK_HOST_TSC. If set, the `host_tsc` field in the kvm_clock_data +structure is populated with the value of the host's timestamp counter (TSC) +at the instant when KVM_GET_CLOCK was called. If clear, the `host_tsc` field +does not contain a value. :: struct kvm_clock_data { __u64 clock; /* kvmclock current value */ __u32 flags; - __u32 pad[9]; + __u32 pad0; + __u64 realtime; + __u64 host_tsc; + __u32 pad[4]; }; @@ -1023,12 +1037,22 @@ Sets the current timestamp of kvmclock to the value specified in its parameter. In conjunction with KVM_GET_CLOCK, it is used to ensure monotonicity on scenarios such as migration. +FLAGS: + +KVM_CLOCK_REALTIME. If set, KVM will compare the value of the `realtime` field +with the value of the host's real time clocksource at the instant when +KVM_SET_CLOCK was called. The difference in elapsed time is added to the final +kvmclock value that will be provided to guests. + :: struct kvm_clock_data { __u64 clock; /* kvmclock current value */ __u32 flags; - __u32 pad[9]; + __u32 pad0; + __u64 realtime; + __u64 host_tsc; + __u32 pad[4]; }; diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 974cbfb1eefe..3fb2b9270d01 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1936,4 +1936,7 @@ int kvm_cpu_dirty_log_size(void); int alloc_all_memslots_rmaps(struct kvm *kvm); +#define KVM_CLOCK_VALID_FLAGS \ + (KVM_CLOCK_TSC_STABLE | KVM_CLOCK_REALTIME | KVM_CLOCK_HOST_TSC) + #endif /* _ASM_X86_KVM_HOST_H */ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c471b1375f36..bff78168d4a2 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2780,17 +2780,24 @@ static void kvm_gen_update_masterclock(struct kvm *kvm) #endif } -u64 get_kvmclock_ns(struct kvm *kvm) +/* + * Returns true if realtime and TSC values were written back to the caller. + * Returns false if a clock triplet cannot be obtained, such as if the host's + * realtime clock is not based on the TSC. + */ +static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns, + u64 *realtime_ns, u64 *tsc) { struct kvm_arch *ka = &kvm->arch; struct pvclock_vcpu_time_info hv_clock; unsigned long flags; - u64 ret; + bool ret = false; spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags); if (!ka->use_master_clock) { spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags); - return get_kvmclock_base_ns() + ka->kvmclock_offset; + *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset; + return false; } hv_clock.tsc_timestamp = ka->master_cycle_now; @@ -2801,18 +2808,36 @@ u64 get_kvmclock_ns(struct kvm *kvm) get_cpu(); if (__this_cpu_read(cpu_tsc_khz)) { + struct timespec64 ts; + u64 tsc_val; + kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL, &hv_clock.tsc_shift, &hv_clock.tsc_to_system_mul); - ret = __pvclock_read_cycles(&hv_clock, rdtsc()); + + if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) { + *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec; + *tsc = tsc_val; + ret = true; + } + + *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val); } else - ret = get_kvmclock_base_ns() + ka->kvmclock_offset; + *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset; put_cpu(); return ret; } +u64 get_kvmclock_ns(struct kvm *kvm) +{ + u64 kvmclock_ns, realtime_ns, tsc; + + get_kvmclock_and_realtime(kvm, &kvmclock_ns, &realtime_ns, &tsc); + return kvmclock_ns; +} + static void kvm_setup_pvclock_page(struct kvm_vcpu *v, struct gfn_to_hva_cache *cache, unsigned int offset) @@ -4031,7 +4056,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r = KVM_SYNC_X86_VALID_FIELDS; break; case KVM_CAP_ADJUST_CLOCK: - r = KVM_CLOCK_TSC_STABLE; + r = KVM_CLOCK_VALID_FLAGS; break; case KVM_CAP_X86_DISABLE_EXITS: r |= KVM_X86_DISABLE_EXITS_HLT | KVM_X86_DISABLE_EXITS_PAUSE | @@ -5806,6 +5831,68 @@ int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state) } #endif /* CONFIG_HAVE_KVM_PM_NOTIFIER */ +static int kvm_vm_ioctl_get_clock(struct kvm *kvm, + void __user *argp) +{ + struct kvm_clock_data data; + + memset(&data, 0, sizeof(data)); + + if (get_kvmclock_and_realtime(kvm, &data.clock, &data.realtime, + &data.host_tsc)) + data.flags |= KVM_CLOCK_REALTIME | KVM_CLOCK_HOST_TSC; + + if (kvm->arch.use_master_clock) + data.flags |= KVM_CLOCK_TSC_STABLE; + + if (copy_to_user(argp, &data, sizeof(data))) + return -EFAULT; + + return 0; +} + +static int kvm_vm_ioctl_set_clock(struct kvm *kvm, + void __user *argp) +{ + struct kvm_arch *ka = &kvm->arch; + struct kvm_clock_data data; + u64 now_raw_ns; + + if (copy_from_user(&data, argp, sizeof(data))) + return -EFAULT; + + if (data.flags & ~KVM_CLOCK_REALTIME) + return -EINVAL; + + /* + * TODO: userspace has to take care of races with VCPU_RUN, so + * kvm_gen_update_masterclock() can be cut down to locked + * pvclock_update_vm_gtod_copy(). + */ + kvm_gen_update_masterclock(kvm); + + spin_lock_irq(&ka->pvclock_gtod_sync_lock); + if (data.flags & KVM_CLOCK_REALTIME) { + u64 now_real_ns = ktime_get_real_ns(); + + /* + * Avoid stepping the kvmclock backwards. + */ + if (now_real_ns > data.realtime) + data.clock += now_real_ns - data.realtime; + } + + if (ka->use_master_clock) + now_raw_ns = ka->master_kernel_ns; + else + now_raw_ns = get_kvmclock_base_ns(); + ka->kvmclock_offset = data.clock - now_raw_ns; + spin_unlock_irq(&ka->pvclock_gtod_sync_lock); + + kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE); + return 0; +} + long kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { @@ -6050,57 +6137,11 @@ long kvm_arch_vm_ioctl(struct file *filp, } #endif case KVM_SET_CLOCK: { - struct kvm_arch *ka = &kvm->arch; - struct kvm_clock_data user_ns; - u64 now_ns; - - r = -EFAULT; - if (copy_from_user(&user_ns, argp, sizeof(user_ns))) - goto out; - - r = -EINVAL; - if (user_ns.flags) - goto out; - - r = 0; - /* - * TODO: userspace has to take care of races with VCPU_RUN, so - * kvm_gen_update_masterclock() can be cut down to locked - * pvclock_update_vm_gtod_copy(). - */ - kvm_gen_update_masterclock(kvm); - - /* - * This pairs with kvm_guest_time_update(): when masterclock is - * in use, we use master_kernel_ns + kvmclock_offset to set - * unsigned 'system_time' so if we use get_kvmclock_ns() (which - * is slightly ahead) here we risk going negative on unsigned - * 'system_time' when 'user_ns.clock' is very small. - */ - spin_lock_irq(&ka->pvclock_gtod_sync_lock); - if (kvm->arch.use_master_clock) - now_ns = ka->master_kernel_ns; - else - now_ns = get_kvmclock_base_ns(); - ka->kvmclock_offset = user_ns.clock - now_ns; - spin_unlock_irq(&ka->pvclock_gtod_sync_lock); - - kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE); + r = kvm_vm_ioctl_set_clock(kvm, argp); break; } case KVM_GET_CLOCK: { - struct kvm_clock_data user_ns; - u64 now_ns; - - now_ns = get_kvmclock_ns(kvm); - user_ns.clock = now_ns; - user_ns.flags = kvm->arch.use_master_clock ? KVM_CLOCK_TSC_STABLE : 0; - memset(&user_ns.pad, 0, sizeof(user_ns.pad)); - - r = -EFAULT; - if (copy_to_user(argp, &user_ns, sizeof(user_ns))) - goto out; - r = 0; + r = kvm_vm_ioctl_get_clock(kvm, argp); break; } case KVM_MEMORY_ENCRYPT_OP: { diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index d9e4aabcb31a..53a49cb8616a 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1223,11 +1223,16 @@ struct kvm_irqfd { /* Do not use 1, KVM_CHECK_EXTENSION returned it before we had flags. */ #define KVM_CLOCK_TSC_STABLE 2 +#define KVM_CLOCK_REALTIME (1 << 2) +#define KVM_CLOCK_HOST_TSC (1 << 3) struct kvm_clock_data { __u64 clock; __u32 flags; - __u32 pad[9]; + __u32 pad0; + __u64 realtime; + __u64 host_tsc; + __u32 pad[4]; }; /* For KVM_CAP_SW_TLB */ -- 2.32.0.402.g57bb445576-goog _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm ^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH v3 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK @ 2021-07-19 18:49 ` Oliver Upton 0 siblings, 0 replies; 4+ messages in thread From: Oliver Upton @ 2021-07-19 18:49 UTC (permalink / raw) To: kvm, kvmarm Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier, Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang, Raghavendra Rao Anata, James Morse, Alexandru Elisei, Suzuki K Poulose, linux-arm-kernel, Oliver Upton Handling the migration of TSCs correctly is difficult, in part because Linux does not provide userspace with the ability to retrieve a (TSC, realtime) clock pair for a single instant in time. In lieu of a more convenient facility, KVM can report similar information in the kvm_clock structure. Provide userspace with a host TSC & realtime pair iff the realtime clock is based on the TSC. If userspace provides KVM_SET_CLOCK with a valid realtime value, advance the KVM clock by the amount of elapsed time. Do not step the KVM clock backwards, though, as it is a monotonic oscillator. Signed-off-by: Oliver Upton <oupton@google.com> --- Documentation/virt/kvm/api.rst | 42 +++++++-- arch/x86/include/asm/kvm_host.h | 3 + arch/x86/kvm/x86.c | 149 ++++++++++++++++++++------------ include/uapi/linux/kvm.h | 7 +- 4 files changed, 137 insertions(+), 64 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index b9ddce5638f5..70ccf68cd574 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -993,20 +993,34 @@ such as migration. When KVM_CAP_ADJUST_CLOCK is passed to KVM_CHECK_EXTENSION, it returns the set of bits that KVM can return in struct kvm_clock_data's flag member. -The only flag defined now is KVM_CLOCK_TSC_STABLE. If set, the returned -value is the exact kvmclock value seen by all VCPUs at the instant -when KVM_GET_CLOCK was called. If clear, the returned value is simply -CLOCK_MONOTONIC plus a constant offset; the offset can be modified -with KVM_SET_CLOCK. KVM will try to make all VCPUs follow this clock, -but the exact value read by each VCPU could differ, because the host -TSC is not stable. +FLAGS: + +KVM_CLOCK_TSC_STABLE. If set, the returned value is the exact kvmclock +value seen by all VCPUs at the instant when KVM_GET_CLOCK was called. +If clear, the returned value is simply CLOCK_MONOTONIC plus a constant +offset; the offset can be modified with KVM_SET_CLOCK. KVM will try +to make all VCPUs follow this clock, but the exact value read by each +VCPU could differ, because the host TSC is not stable. + +KVM_CLOCK_REALTIME. If set, the `realtime` field in the kvm_clock_data +structure is populated with the value of the host's real time +clocksource at the instant when KVM_GET_CLOCK was called. If clear, +the `realtime` field does not contain a value. + +KVM_CLOCK_HOST_TSC. If set, the `host_tsc` field in the kvm_clock_data +structure is populated with the value of the host's timestamp counter (TSC) +at the instant when KVM_GET_CLOCK was called. If clear, the `host_tsc` field +does not contain a value. :: struct kvm_clock_data { __u64 clock; /* kvmclock current value */ __u32 flags; - __u32 pad[9]; + __u32 pad0; + __u64 realtime; + __u64 host_tsc; + __u32 pad[4]; }; @@ -1023,12 +1037,22 @@ Sets the current timestamp of kvmclock to the value specified in its parameter. In conjunction with KVM_GET_CLOCK, it is used to ensure monotonicity on scenarios such as migration. +FLAGS: + +KVM_CLOCK_REALTIME. If set, KVM will compare the value of the `realtime` field +with the value of the host's real time clocksource at the instant when +KVM_SET_CLOCK was called. The difference in elapsed time is added to the final +kvmclock value that will be provided to guests. + :: struct kvm_clock_data { __u64 clock; /* kvmclock current value */ __u32 flags; - __u32 pad[9]; + __u32 pad0; + __u64 realtime; + __u64 host_tsc; + __u32 pad[4]; }; diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 974cbfb1eefe..3fb2b9270d01 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1936,4 +1936,7 @@ int kvm_cpu_dirty_log_size(void); int alloc_all_memslots_rmaps(struct kvm *kvm); +#define KVM_CLOCK_VALID_FLAGS \ + (KVM_CLOCK_TSC_STABLE | KVM_CLOCK_REALTIME | KVM_CLOCK_HOST_TSC) + #endif /* _ASM_X86_KVM_HOST_H */ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c471b1375f36..bff78168d4a2 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2780,17 +2780,24 @@ static void kvm_gen_update_masterclock(struct kvm *kvm) #endif } -u64 get_kvmclock_ns(struct kvm *kvm) +/* + * Returns true if realtime and TSC values were written back to the caller. + * Returns false if a clock triplet cannot be obtained, such as if the host's + * realtime clock is not based on the TSC. + */ +static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns, + u64 *realtime_ns, u64 *tsc) { struct kvm_arch *ka = &kvm->arch; struct pvclock_vcpu_time_info hv_clock; unsigned long flags; - u64 ret; + bool ret = false; spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags); if (!ka->use_master_clock) { spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags); - return get_kvmclock_base_ns() + ka->kvmclock_offset; + *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset; + return false; } hv_clock.tsc_timestamp = ka->master_cycle_now; @@ -2801,18 +2808,36 @@ u64 get_kvmclock_ns(struct kvm *kvm) get_cpu(); if (__this_cpu_read(cpu_tsc_khz)) { + struct timespec64 ts; + u64 tsc_val; + kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL, &hv_clock.tsc_shift, &hv_clock.tsc_to_system_mul); - ret = __pvclock_read_cycles(&hv_clock, rdtsc()); + + if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) { + *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec; + *tsc = tsc_val; + ret = true; + } + + *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val); } else - ret = get_kvmclock_base_ns() + ka->kvmclock_offset; + *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset; put_cpu(); return ret; } +u64 get_kvmclock_ns(struct kvm *kvm) +{ + u64 kvmclock_ns, realtime_ns, tsc; + + get_kvmclock_and_realtime(kvm, &kvmclock_ns, &realtime_ns, &tsc); + return kvmclock_ns; +} + static void kvm_setup_pvclock_page(struct kvm_vcpu *v, struct gfn_to_hva_cache *cache, unsigned int offset) @@ -4031,7 +4056,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r = KVM_SYNC_X86_VALID_FIELDS; break; case KVM_CAP_ADJUST_CLOCK: - r = KVM_CLOCK_TSC_STABLE; + r = KVM_CLOCK_VALID_FLAGS; break; case KVM_CAP_X86_DISABLE_EXITS: r |= KVM_X86_DISABLE_EXITS_HLT | KVM_X86_DISABLE_EXITS_PAUSE | @@ -5806,6 +5831,68 @@ int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state) } #endif /* CONFIG_HAVE_KVM_PM_NOTIFIER */ +static int kvm_vm_ioctl_get_clock(struct kvm *kvm, + void __user *argp) +{ + struct kvm_clock_data data; + + memset(&data, 0, sizeof(data)); + + if (get_kvmclock_and_realtime(kvm, &data.clock, &data.realtime, + &data.host_tsc)) + data.flags |= KVM_CLOCK_REALTIME | KVM_CLOCK_HOST_TSC; + + if (kvm->arch.use_master_clock) + data.flags |= KVM_CLOCK_TSC_STABLE; + + if (copy_to_user(argp, &data, sizeof(data))) + return -EFAULT; + + return 0; +} + +static int kvm_vm_ioctl_set_clock(struct kvm *kvm, + void __user *argp) +{ + struct kvm_arch *ka = &kvm->arch; + struct kvm_clock_data data; + u64 now_raw_ns; + + if (copy_from_user(&data, argp, sizeof(data))) + return -EFAULT; + + if (data.flags & ~KVM_CLOCK_REALTIME) + return -EINVAL; + + /* + * TODO: userspace has to take care of races with VCPU_RUN, so + * kvm_gen_update_masterclock() can be cut down to locked + * pvclock_update_vm_gtod_copy(). + */ + kvm_gen_update_masterclock(kvm); + + spin_lock_irq(&ka->pvclock_gtod_sync_lock); + if (data.flags & KVM_CLOCK_REALTIME) { + u64 now_real_ns = ktime_get_real_ns(); + + /* + * Avoid stepping the kvmclock backwards. + */ + if (now_real_ns > data.realtime) + data.clock += now_real_ns - data.realtime; + } + + if (ka->use_master_clock) + now_raw_ns = ka->master_kernel_ns; + else + now_raw_ns = get_kvmclock_base_ns(); + ka->kvmclock_offset = data.clock - now_raw_ns; + spin_unlock_irq(&ka->pvclock_gtod_sync_lock); + + kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE); + return 0; +} + long kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { @@ -6050,57 +6137,11 @@ long kvm_arch_vm_ioctl(struct file *filp, } #endif case KVM_SET_CLOCK: { - struct kvm_arch *ka = &kvm->arch; - struct kvm_clock_data user_ns; - u64 now_ns; - - r = -EFAULT; - if (copy_from_user(&user_ns, argp, sizeof(user_ns))) - goto out; - - r = -EINVAL; - if (user_ns.flags) - goto out; - - r = 0; - /* - * TODO: userspace has to take care of races with VCPU_RUN, so - * kvm_gen_update_masterclock() can be cut down to locked - * pvclock_update_vm_gtod_copy(). - */ - kvm_gen_update_masterclock(kvm); - - /* - * This pairs with kvm_guest_time_update(): when masterclock is - * in use, we use master_kernel_ns + kvmclock_offset to set - * unsigned 'system_time' so if we use get_kvmclock_ns() (which - * is slightly ahead) here we risk going negative on unsigned - * 'system_time' when 'user_ns.clock' is very small. - */ - spin_lock_irq(&ka->pvclock_gtod_sync_lock); - if (kvm->arch.use_master_clock) - now_ns = ka->master_kernel_ns; - else - now_ns = get_kvmclock_base_ns(); - ka->kvmclock_offset = user_ns.clock - now_ns; - spin_unlock_irq(&ka->pvclock_gtod_sync_lock); - - kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE); + r = kvm_vm_ioctl_set_clock(kvm, argp); break; } case KVM_GET_CLOCK: { - struct kvm_clock_data user_ns; - u64 now_ns; - - now_ns = get_kvmclock_ns(kvm); - user_ns.clock = now_ns; - user_ns.flags = kvm->arch.use_master_clock ? KVM_CLOCK_TSC_STABLE : 0; - memset(&user_ns.pad, 0, sizeof(user_ns.pad)); - - r = -EFAULT; - if (copy_to_user(argp, &user_ns, sizeof(user_ns))) - goto out; - r = 0; + r = kvm_vm_ioctl_get_clock(kvm, argp); break; } case KVM_MEMORY_ENCRYPT_OP: { diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index d9e4aabcb31a..53a49cb8616a 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1223,11 +1223,16 @@ struct kvm_irqfd { /* Do not use 1, KVM_CHECK_EXTENSION returned it before we had flags. */ #define KVM_CLOCK_TSC_STABLE 2 +#define KVM_CLOCK_REALTIME (1 << 2) +#define KVM_CLOCK_HOST_TSC (1 << 3) struct kvm_clock_data { __u64 clock; __u32 flags; - __u32 pad[9]; + __u32 pad0; + __u64 realtime; + __u64 host_tsc; + __u32 pad[4]; }; /* For KVM_CAP_SW_TLB */ -- 2.32.0.402.g57bb445576-goog ^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH v3 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK @ 2021-07-19 18:49 ` Oliver Upton 0 siblings, 0 replies; 4+ messages in thread From: Oliver Upton @ 2021-07-19 18:49 UTC (permalink / raw) To: kvm, kvmarm Cc: Paolo Bonzini, Sean Christopherson, Marc Zyngier, Peter Shier, Jim Mattson, David Matlack, Ricardo Koller, Jing Zhang, Raghavendra Rao Anata, James Morse, Alexandru Elisei, Suzuki K Poulose, linux-arm-kernel, Oliver Upton Handling the migration of TSCs correctly is difficult, in part because Linux does not provide userspace with the ability to retrieve a (TSC, realtime) clock pair for a single instant in time. In lieu of a more convenient facility, KVM can report similar information in the kvm_clock structure. Provide userspace with a host TSC & realtime pair iff the realtime clock is based on the TSC. If userspace provides KVM_SET_CLOCK with a valid realtime value, advance the KVM clock by the amount of elapsed time. Do not step the KVM clock backwards, though, as it is a monotonic oscillator. Signed-off-by: Oliver Upton <oupton@google.com> --- Documentation/virt/kvm/api.rst | 42 +++++++-- arch/x86/include/asm/kvm_host.h | 3 + arch/x86/kvm/x86.c | 149 ++++++++++++++++++++------------ include/uapi/linux/kvm.h | 7 +- 4 files changed, 137 insertions(+), 64 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index b9ddce5638f5..70ccf68cd574 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -993,20 +993,34 @@ such as migration. When KVM_CAP_ADJUST_CLOCK is passed to KVM_CHECK_EXTENSION, it returns the set of bits that KVM can return in struct kvm_clock_data's flag member. -The only flag defined now is KVM_CLOCK_TSC_STABLE. If set, the returned -value is the exact kvmclock value seen by all VCPUs at the instant -when KVM_GET_CLOCK was called. If clear, the returned value is simply -CLOCK_MONOTONIC plus a constant offset; the offset can be modified -with KVM_SET_CLOCK. KVM will try to make all VCPUs follow this clock, -but the exact value read by each VCPU could differ, because the host -TSC is not stable. +FLAGS: + +KVM_CLOCK_TSC_STABLE. If set, the returned value is the exact kvmclock +value seen by all VCPUs at the instant when KVM_GET_CLOCK was called. +If clear, the returned value is simply CLOCK_MONOTONIC plus a constant +offset; the offset can be modified with KVM_SET_CLOCK. KVM will try +to make all VCPUs follow this clock, but the exact value read by each +VCPU could differ, because the host TSC is not stable. + +KVM_CLOCK_REALTIME. If set, the `realtime` field in the kvm_clock_data +structure is populated with the value of the host's real time +clocksource at the instant when KVM_GET_CLOCK was called. If clear, +the `realtime` field does not contain a value. + +KVM_CLOCK_HOST_TSC. If set, the `host_tsc` field in the kvm_clock_data +structure is populated with the value of the host's timestamp counter (TSC) +at the instant when KVM_GET_CLOCK was called. If clear, the `host_tsc` field +does not contain a value. :: struct kvm_clock_data { __u64 clock; /* kvmclock current value */ __u32 flags; - __u32 pad[9]; + __u32 pad0; + __u64 realtime; + __u64 host_tsc; + __u32 pad[4]; }; @@ -1023,12 +1037,22 @@ Sets the current timestamp of kvmclock to the value specified in its parameter. In conjunction with KVM_GET_CLOCK, it is used to ensure monotonicity on scenarios such as migration. +FLAGS: + +KVM_CLOCK_REALTIME. If set, KVM will compare the value of the `realtime` field +with the value of the host's real time clocksource at the instant when +KVM_SET_CLOCK was called. The difference in elapsed time is added to the final +kvmclock value that will be provided to guests. + :: struct kvm_clock_data { __u64 clock; /* kvmclock current value */ __u32 flags; - __u32 pad[9]; + __u32 pad0; + __u64 realtime; + __u64 host_tsc; + __u32 pad[4]; }; diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 974cbfb1eefe..3fb2b9270d01 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1936,4 +1936,7 @@ int kvm_cpu_dirty_log_size(void); int alloc_all_memslots_rmaps(struct kvm *kvm); +#define KVM_CLOCK_VALID_FLAGS \ + (KVM_CLOCK_TSC_STABLE | KVM_CLOCK_REALTIME | KVM_CLOCK_HOST_TSC) + #endif /* _ASM_X86_KVM_HOST_H */ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c471b1375f36..bff78168d4a2 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2780,17 +2780,24 @@ static void kvm_gen_update_masterclock(struct kvm *kvm) #endif } -u64 get_kvmclock_ns(struct kvm *kvm) +/* + * Returns true if realtime and TSC values were written back to the caller. + * Returns false if a clock triplet cannot be obtained, such as if the host's + * realtime clock is not based on the TSC. + */ +static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns, + u64 *realtime_ns, u64 *tsc) { struct kvm_arch *ka = &kvm->arch; struct pvclock_vcpu_time_info hv_clock; unsigned long flags; - u64 ret; + bool ret = false; spin_lock_irqsave(&ka->pvclock_gtod_sync_lock, flags); if (!ka->use_master_clock) { spin_unlock_irqrestore(&ka->pvclock_gtod_sync_lock, flags); - return get_kvmclock_base_ns() + ka->kvmclock_offset; + *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset; + return false; } hv_clock.tsc_timestamp = ka->master_cycle_now; @@ -2801,18 +2808,36 @@ u64 get_kvmclock_ns(struct kvm *kvm) get_cpu(); if (__this_cpu_read(cpu_tsc_khz)) { + struct timespec64 ts; + u64 tsc_val; + kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL, &hv_clock.tsc_shift, &hv_clock.tsc_to_system_mul); - ret = __pvclock_read_cycles(&hv_clock, rdtsc()); + + if (kvm_get_walltime_and_clockread(&ts, &tsc_val)) { + *realtime_ns = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec; + *tsc = tsc_val; + ret = true; + } + + *kvmclock_ns = __pvclock_read_cycles(&hv_clock, tsc_val); } else - ret = get_kvmclock_base_ns() + ka->kvmclock_offset; + *kvmclock_ns = get_kvmclock_base_ns() + ka->kvmclock_offset; put_cpu(); return ret; } +u64 get_kvmclock_ns(struct kvm *kvm) +{ + u64 kvmclock_ns, realtime_ns, tsc; + + get_kvmclock_and_realtime(kvm, &kvmclock_ns, &realtime_ns, &tsc); + return kvmclock_ns; +} + static void kvm_setup_pvclock_page(struct kvm_vcpu *v, struct gfn_to_hva_cache *cache, unsigned int offset) @@ -4031,7 +4056,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r = KVM_SYNC_X86_VALID_FIELDS; break; case KVM_CAP_ADJUST_CLOCK: - r = KVM_CLOCK_TSC_STABLE; + r = KVM_CLOCK_VALID_FLAGS; break; case KVM_CAP_X86_DISABLE_EXITS: r |= KVM_X86_DISABLE_EXITS_HLT | KVM_X86_DISABLE_EXITS_PAUSE | @@ -5806,6 +5831,68 @@ int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state) } #endif /* CONFIG_HAVE_KVM_PM_NOTIFIER */ +static int kvm_vm_ioctl_get_clock(struct kvm *kvm, + void __user *argp) +{ + struct kvm_clock_data data; + + memset(&data, 0, sizeof(data)); + + if (get_kvmclock_and_realtime(kvm, &data.clock, &data.realtime, + &data.host_tsc)) + data.flags |= KVM_CLOCK_REALTIME | KVM_CLOCK_HOST_TSC; + + if (kvm->arch.use_master_clock) + data.flags |= KVM_CLOCK_TSC_STABLE; + + if (copy_to_user(argp, &data, sizeof(data))) + return -EFAULT; + + return 0; +} + +static int kvm_vm_ioctl_set_clock(struct kvm *kvm, + void __user *argp) +{ + struct kvm_arch *ka = &kvm->arch; + struct kvm_clock_data data; + u64 now_raw_ns; + + if (copy_from_user(&data, argp, sizeof(data))) + return -EFAULT; + + if (data.flags & ~KVM_CLOCK_REALTIME) + return -EINVAL; + + /* + * TODO: userspace has to take care of races with VCPU_RUN, so + * kvm_gen_update_masterclock() can be cut down to locked + * pvclock_update_vm_gtod_copy(). + */ + kvm_gen_update_masterclock(kvm); + + spin_lock_irq(&ka->pvclock_gtod_sync_lock); + if (data.flags & KVM_CLOCK_REALTIME) { + u64 now_real_ns = ktime_get_real_ns(); + + /* + * Avoid stepping the kvmclock backwards. + */ + if (now_real_ns > data.realtime) + data.clock += now_real_ns - data.realtime; + } + + if (ka->use_master_clock) + now_raw_ns = ka->master_kernel_ns; + else + now_raw_ns = get_kvmclock_base_ns(); + ka->kvmclock_offset = data.clock - now_raw_ns; + spin_unlock_irq(&ka->pvclock_gtod_sync_lock); + + kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE); + return 0; +} + long kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { @@ -6050,57 +6137,11 @@ long kvm_arch_vm_ioctl(struct file *filp, } #endif case KVM_SET_CLOCK: { - struct kvm_arch *ka = &kvm->arch; - struct kvm_clock_data user_ns; - u64 now_ns; - - r = -EFAULT; - if (copy_from_user(&user_ns, argp, sizeof(user_ns))) - goto out; - - r = -EINVAL; - if (user_ns.flags) - goto out; - - r = 0; - /* - * TODO: userspace has to take care of races with VCPU_RUN, so - * kvm_gen_update_masterclock() can be cut down to locked - * pvclock_update_vm_gtod_copy(). - */ - kvm_gen_update_masterclock(kvm); - - /* - * This pairs with kvm_guest_time_update(): when masterclock is - * in use, we use master_kernel_ns + kvmclock_offset to set - * unsigned 'system_time' so if we use get_kvmclock_ns() (which - * is slightly ahead) here we risk going negative on unsigned - * 'system_time' when 'user_ns.clock' is very small. - */ - spin_lock_irq(&ka->pvclock_gtod_sync_lock); - if (kvm->arch.use_master_clock) - now_ns = ka->master_kernel_ns; - else - now_ns = get_kvmclock_base_ns(); - ka->kvmclock_offset = user_ns.clock - now_ns; - spin_unlock_irq(&ka->pvclock_gtod_sync_lock); - - kvm_make_all_cpus_request(kvm, KVM_REQ_CLOCK_UPDATE); + r = kvm_vm_ioctl_set_clock(kvm, argp); break; } case KVM_GET_CLOCK: { - struct kvm_clock_data user_ns; - u64 now_ns; - - now_ns = get_kvmclock_ns(kvm); - user_ns.clock = now_ns; - user_ns.flags = kvm->arch.use_master_clock ? KVM_CLOCK_TSC_STABLE : 0; - memset(&user_ns.pad, 0, sizeof(user_ns.pad)); - - r = -EFAULT; - if (copy_to_user(argp, &user_ns, sizeof(user_ns))) - goto out; - r = 0; + r = kvm_vm_ioctl_get_clock(kvm, argp); break; } case KVM_MEMORY_ENCRYPT_OP: { diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index d9e4aabcb31a..53a49cb8616a 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1223,11 +1223,16 @@ struct kvm_irqfd { /* Do not use 1, KVM_CHECK_EXTENSION returned it before we had flags. */ #define KVM_CLOCK_TSC_STABLE 2 +#define KVM_CLOCK_REALTIME (1 << 2) +#define KVM_CLOCK_HOST_TSC (1 << 3) struct kvm_clock_data { __u64 clock; __u32 flags; - __u32 pad[9]; + __u32 pad0; + __u64 realtime; + __u64 host_tsc; + __u32 pad[4]; }; /* For KVM_CAP_SW_TLB */ -- 2.32.0.402.g57bb445576-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-07-20 9:10 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-07-20 9:10 [PATCH v3 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK kernel test robot -- strict thread matches above, loose matches on Subject: below -- 2021-07-19 18:49 [PATCH v3 00/12] KVM: Add idempotent controls for migrating system counter state Oliver Upton 2021-07-19 18:49 ` [PATCH v3 01/12] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK Oliver Upton 2021-07-19 18:49 ` Oliver Upton 2021-07-19 18:49 ` Oliver Upton
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.