Re: [PATCH RFC 0/6] x86/time: PVCLOCK_TSC_STABLE_BIT support

["Haozhong Zhang" <haozhong.zhang@intel.com>]

On 12/29/15 14:58, Andrew Cooper wrote:
> On 28/12/2015 16:59, Joao Martins wrote:
> >Hey!
> >
> >I've been working on pvclock vdso support on Linux guests, and came
> >across Xen lacking support for PVCLOCK_TSC_STABLE_BIT flag which is
> >required for vdso as of the latest pvclock ABI shared with KVM.
> 
> , and originally borrowed from Xen.
> 
> Please be aware that all of this was originally the Xen ABI (c/s 1271b793,
> 2005) and was added to Linux (c/s 7af192c, 2008) for join use with KVM.  In
> particular, Linux c/s 424c32f1a (which introduces 'flags') and every
> subsequent change in pvclock-abi.h violates the comment at the top,
> reminding people that the hypervisors must be kept in sync.
> 
> By the looks of things, the structures are still compatible, and having the
> two in sync is in everyones best interest. The first steps here need to be
> Linux upstreaming their local modifications, and further efforts made to
> ensuring that ABI changes don't go unnoticed as far as Xen is concerned
> (entry in the maintainers file with xen-devel listed?)
> 
> >In addition, I've found some problems which aren't necessarily visible
> >to kernel as the pvclock algorithm in linux keeps the highest pvti
> >time read among all cpus. But as is, a process using vdso gettimeofday
> >observes a significant amount of warps (i.e. time going backwards) and
> >it could be due to 1) time calibration skew in xen rendezvous
> >algorithm, 2) xen clocksource not in sync with TSC and 3) in
> >situations when guests unaware of VCPUS moving to a different PCPU.
> >The warps are seen more frequently on PV guests (potentially because
> >vcpu time infos are only updated when guest is in kernel mode, and
> >perhaps lack of tsc_offset?), and in rare ocasions on HVM guests. And
> >it is worth noting that with guests VCPUs pinned, only PV guests see
> >these warps. But on HVM guests specifically: such warps only occur
> >when one of guest VCPUs is pinned to CPU0.
> 
> These are all concerning findings (especially the pinning on cpu0). Which
> version of Xen have you been developing on?
> 
> Haozhong Zhang (CC'd) found and fixed several timing related bugs as part of
> his VMX TSC Scaling support series (Message root at
> <1449435529-12989-1-git-send-email-haozhong.zhang@intel.com>). I would be
> surprised if your identified bugs and his identified bugs didn't at least
> partially overlap.  (Note that most of the series has yet to be applied).
> 

My VMX TSC scaling support patch series takes more concern on bugs
that can cause problems when TSC scaling is used for HVM guests. I
didn't test much for PV guests and HVM guests with pinned vcpu, so I
think Joao's patch series is fixing different timing bugs.

Haozhong