Re: xen-4.1: PV domain hanging at startup, jiffies stopped

[Marek Marczykowski <marmarek@mimuw.edu.pl>]

[-- Attachment #1.1.1: Type: text/plain, Size: 1950 bytes --]

On 31.08.2011 23:01, Keir Fraser wrote:
> On 31/08/2011 21:49, "Marek Marczykowski" <marmarek@mimuw.edu.pl> wrote:
> 
>> xen_vcpuop_set_next_event schedules event by getting current time
>> (xen_clocksource_read()) (*1) adding delta (expires-now) and programming
>> event with VCPUOP_set_singleshot_timer hypercall. Then xen gets current
>> time (*2) and in some rare cases this time is after expected timer
>> expiration... Even after VCPUOP_set_singleshot_timer hypercal,
>> xen_clocksource_read() reports time slightly in the past comparing to
>> xen time (reported by NOW() macro).
>>
>> I think this is because "current" time is calculated different way in *1
>> and *2. The *1 way is controlled by tsc_mode, which is described here:
>> http://lxr.xensource.com/lxr/source/docs/misc/tscmode.txt. Default
>> tsc_mode=0 is "smart" and I think because of that can be slightly before
>> NOW() time. tsc_mode=2 looks almost the same as NOW() macro works.
>>
>> Is this reasoning correct?
> 
> They really ought to work out to the same thing. This will trivially be the
> case with tsc_mode=2 because both guest and hypervisor will see the same
> (real) values from RDTSC, and use the same offsets and sacle factors to turn
> that into a current system time. When using emulated TSC in the guest
> (tsc_mode=0,1) then the TSC values it sees, and the offsets and scale
> factors it applies, are different. It is intended that it should result in
> the same values being computed for NOW(), but I suppose something could be
> going wrong there. 

NOW() calls get_s_time() which doesn't look to be depended on tsc_mode
setting. Have I missed something?

> By how much have you seen guest and hypervisor disagree?

Adding printks in domU and hypervisor side using attached patches.

-- 
Pozdrawiam / Best Regards,
Marek Marczykowski         | RLU #390519
marmarek at mimuw edu pl   | xmpp:marmarek at staszic waw pl

[-- Attachment #1.1.2: debug-kernel-domU.diff --]
[-- Type: text/plain, Size: 967 bytes --]

diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c
index 5158c50..0976e44 100644
--- a/arch/x86/xen/time.c
+++ b/arch/x86/xen/time.c
@@ -332,16 +332,22 @@ static int xen_vcpuop_set_next_event(unsigned long delta,
 	int cpu = smp_processor_id();
 	struct vcpu_set_singleshot_timer single;
 	int ret;
+	s64 base;
 
 	WARN_ON(evt->mode != CLOCK_EVT_MODE_ONESHOT);
 
-	single.timeout_abs_ns = get_abs_timeout(delta);
-	single.flags = VCPU_SSHOTTMR_future;
+	base = xen_clocksource_read();
+	single.timeout_abs_ns = base + delta;
+	single.flags = VCPU_SSHOTTMR_future & 0;
 
 	ret = HYPERVISOR_vcpu_op(VCPUOP_set_singleshot_timer, cpu, &single);
 
 	BUG_ON(ret != 0 && ret != -ETIME);
 
+	if (ret == -ETIME) {
+		printk(KERN_WARNING "hypercall VCPUOP_set_singleshot_timer failed with -ETIME on %d CPU with params: timeout=%lld, flags=%d, base-pre: %lld, base-post: %lld\n", cpu, single.timeout_abs_ns, single.flags, base, get_abs_timeout(0));
+	}
+
 	return ret;
 }
 

[-- Attachment #1.1.3: debug-xen.diff --]
[-- Type: text/plain, Size: 956 bytes --]

diff -r 227130622561 xen/common/domain.c
--- a/xen/common/domain.c	Thu Aug 25 12:03:14 2011 +0100
+++ b/xen/common/domain.c	Wed Aug 31 23:05:31 2011 +0200
@@ -896,6 +896,7 @@ long do_vcpu_op(int cmd, int vcpuid, XEN
     case VCPUOP_set_singleshot_timer:
     {
         struct vcpu_set_singleshot_timer set;
+		s_time_t now;
 
         if ( v != current )
             return -EINVAL;
@@ -903,9 +904,12 @@ long do_vcpu_op(int cmd, int vcpuid, XEN
         if ( copy_from_guest(&set, arg, 1) )
             return -EFAULT;
 
+		now = NOW();
         if ( (set.flags & VCPU_SSHOTTMR_future) &&
-             (set.timeout_abs_ns < NOW()) )
+             (set.timeout_abs_ns < now) ) {
+			gdprintk(XENLOG_INFO, "VCPUOP_set_singleshot_timer: time in past: %lu < %ld\n", set.timeout_abs_ns, now);
             return -ETIME;
+		}
 
         migrate_timer(&v->singleshot_timer, smp_processor_id());
         set_timer(&v->singleshot_timer, set.timeout_abs_ns);

[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5842 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel