[Qemu-devel] [patch] option -no-tsc for i386 with speedstep

[Qemu-devel] [patch] option -no-tsc for i386 with speedstep

[Massimo Dal Zotto]

[-- Attachment #1: Type: text/plain, Size: 1036 bytes --]

When qemu runs on an i386 cpu with speedstep enabled the clock of the
guest os is not in sync with the clock on the host os because the
vm_timer used for irq 0 generates interrupts at wrong rate when
the host cpu frequency changes.

The problem is that the vm_timer uses the rdtsc instruction and the value
of ticks_per_sec, computed at start time, for calculating the expire time
of vm_timers. While ticks_per_sec is constant the values returned by
rdtsc are dependent on the current cpu clock, which is not constant if
speedstep is used.

The problem can be cleary observed by running "xclock -update 1" in the
guest os and observing how the clock speed varies with the cpu freqency.

The following patch fixes the problem by adding a new option -no-tsc for
the i386 architecture which can be used to disable rdtsc when running on
a cpu with speedstep enabled.

The patch works for me but I don't know if this is the best way of fixing
this bug. If anyone has a better suggestion it is welcome.

-- 
Massimo Dal Zotto <dz@debian.org>

[-- Attachment #2: qemu-notsc.patch --]
[-- Type: text/plain, Size: 1759 bytes --]

--- qemu-0.6.2cvs20050425.orig/vl.c	2005-04-08 00:20:28.000000000 +0200
+++ qemu-0.6.2cvs20050425/vl.c	2005-04-25 11:44:41.119819832 +0200
@@ -135,6 +135,10 @@
 int prep_enabled = 0;
 int rtc_utc = 1;
 int cirrus_vga_enabled = 1;
+#ifdef __i386__
+static int notsc = 0;
+extern int64_t get_clock(void);
+#endif
 #ifdef TARGET_SPARC
 int graphic_width = 1024;
 int graphic_height = 768;
@@ -502,6 +506,9 @@
 int64_t cpu_get_real_ticks(void)
 {
     int64_t val;
+    if (notsc) {
+	return get_clock();
+    }
     asm volatile ("rdtsc" : "=A" (val));
     return val;
 }
@@ -2792,6 +2799,9 @@
 #ifdef USE_CODE_COPY
            "-no-code-copy   disable code copy acceleration\n"
 #endif
+#ifdef __i386__
+           "-no-tsc         disable tsc as clock source\n"
+#endif
 #ifdef TARGET_I386
            "-isa            simulate an ISA-only system (default is PCI system)\n"
            "-std-vga        simulate a standard VGA card with VESA Bochs Extensions\n"
@@ -2863,6 +2873,7 @@
     QEMU_OPTION_hdachs,
     QEMU_OPTION_L,
     QEMU_OPTION_no_code_copy,
+    QEMU_OPTION_no_tsc,
     QEMU_OPTION_pci,
     QEMU_OPTION_isa,
     QEMU_OPTION_prep,
@@ -2931,6 +2942,9 @@
 #ifdef USE_KQEMU
     { "no-kqemu", 0, QEMU_OPTION_no_kqemu },
 #endif
+#ifdef __i386__
+    { "no-tsc", 0, QEMU_OPTION_no_tsc },
+#endif
 #ifdef TARGET_PPC
     { "prep", 0, QEMU_OPTION_prep },
 #endif
@@ -3216,6 +3230,11 @@
             case QEMU_OPTION_no_code_copy:
                 code_copy_enabled = 0;
                 break;
+#ifdef __i386__
+            case QEMU_OPTION_no_tsc:
+                notsc = 1;
+                break;
+#endif
             case QEMU_OPTION_nics:
                 nb_nics = atoi(optarg);
                 if (nb_nics < 0 || nb_nics > MAX_NICS) {

[Qemu-devel] Re: [patch] option -no-tsc for i386 with speedstep

[Heike C. Zimmerer]

Massimo Dal Zotto <dz@debian.org> writes:

> When qemu runs on an i386 cpu with speedstep enabled the clock of the
> guest os is not in sync with the clock on the host os because the
> vm_timer used for irq 0 generates interrupts at wrong rate when
> the host cpu frequency changes.
>
> The following patch fixes the problem by adding a new option -no-tsc for
> the i386 architecture which can be used to disable rdtsc when running on
> a cpu with speedstep enabled.

Unfortunately this patch, effectively replacing a single assembly
instruction by a gettimeofday() call, slows some operations down to a
degree that makes a WinXP guest on GNU/Linux host nearly unusable at
network and file system operations (at least here).

- Heike

Re: [Qemu-devel] [patch] option -no-tsc for i386 with speedstep

[Jim C. Brown]

On Mon, Apr 25, 2005 at 01:15:32PM +0200, Massimo Dal Zotto wrote:
> The patch works for me but I don't know if this is the best way of fixing
> this bug. If anyone has a better suggestion it is welcome.
> 
> -- 
> Massimo Dal Zotto <dz@debian.org>

I just want to point out that your patches break qemu for almost every platform
other than i386.

> +#ifdef __i386__
> +static int notsc = 0;
> +extern int64_t get_clock(void);
> +#endif

> @@ -502,6 +506,9 @@
>  int64_t cpu_get_real_ticks(void)
>  {
>      int64_t val;
> +    if (notsc) {
> +	return get_clock();
> +    }
>      asm volatile ("rdtsc" : "=A" (val));
>      return val;
>  }

You probably want to do this, because notsc is only declared for the i386
platform.

 int64_t cpu_get_real_ticks(void)
 {
     int64_t val;
+#ifdef __i386__
+    if (notsc) {
+	return get_clock();
+    }
+#endif
     asm volatile ("rdtsc" : "=A" (val));
     return val;
 }

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

Re: [Qemu-devel] [patch] option -no-tsc for i386 with speedstep

[Paul Brook]

On Monday 25 April 2005 18:07, Jim C. Brown wrote:
> On Mon, Apr 25, 2005 at 01:15:32PM +0200, Massimo Dal Zotto wrote:
> > The patch works for me but I don't know if this is the best way of fixing
> > this bug. If anyone has a better suggestion it is welcome.
> >
> > --
> > Massimo Dal Zotto <dz@debian.org>
>
> I just want to point out that your patches break qemu for almost every
> platform other than i386.

Rubbish.

<snip>
> You probably want to do this, because notsc is only declared for the i386
> platform.
>
>  int64_t cpu_get_real_ticks(void)
>  {
>      int64_t val;
> +#ifdef __i386__
> +    if (notsc) {
> +	return get_clock();
> +    }
> +#endif
>      asm volatile ("rdtsc" : "=A" (val));
>      return val;
>  }

This is already inside a #if defined(__i386__) block

Paul

Re: [Qemu-devel] [patch] option -no-tsc for i386 with speedstep

[Massimo Dal Zotto]

On Mon, Apr 25, 2005 at 01:07:52PM -0400, Jim C. Brown wrote:
> 
> I just want to point out that your patches break qemu for almost every
> platform other than i386.
> 
> You probably want to do this, because notsc is only declared for the i386
> platform.
> 
>  int64_t cpu_get_real_ticks(void)
>  {
>      int64_t val;
> +#ifdef __i386__
> +    if (notsc) {
> +	return get_clock();
> +    }
> +#endif
>      asm volatile ("rdtsc" : "=A" (val));
>      return val;
>  }
> 

Correct, I forgot the #ifdef inside that function.

I made also some more tests and it turned out that my patch works but
not as expected. The clock now runs at constant rate at any speedstep
frequency, which is what I wanted, but unfortunately it is always ~20%
slower than wall clock.

I tried also to force different values of ticks_per_sec but it seems
that it has no effects on irq frequency, so there must be something that
I don't understand in the timer code.

Any idea on how to fix this bug?

-- 
Massimo Dal Zotto <dz@debian.org>

[Qemu-devel] Re: [patch] option -no-tsc for i386 with speedstep

[Heike C. Zimmerer]

"Jim C. Brown" <jma5@umd.edu> writes:

> I just want to point out that your patches break qemu for almost every platform
> other than i386.
> [..]
> You probably want to do this, because notsc is only declared for the i386
> platform.
>
>  int64_t cpu_get_real_ticks(void)
>  {
>      int64_t val;
> +#ifdef __i386__
> +    if (notsc) {
> +	return get_clock();
> +    }
> +#endif
>      asm volatile ("rdtsc" : "=A" (val));
>      return val;
>  }

His patch gets inserted after an '#elif defined(__i386__)' directive
(at least the part cited above).

- Heike

Re: [Qemu-devel] [patch] option -no-tsc for i386 with speedstep

[Fabrice Bellard]

Massimo Dal Zotto wrote:
> When qemu runs on an i386 cpu with speedstep enabled the clock of the
> guest os is not in sync with the clock on the host os because the
> vm_timer used for irq 0 generates interrupts at wrong rate when
> the host cpu frequency changes.
> 
> The problem is that the vm_timer uses the rdtsc instruction and the value
> of ticks_per_sec, computed at start time, for calculating the expire time
> of vm_timers. While ticks_per_sec is constant the values returned by
> rdtsc are dependent on the current cpu clock, which is not constant if
> speedstep is used.
> 
> The problem can be cleary observed by running "xclock -update 1" in the
> guest os and observing how the clock speed varies with the cpu freqency.
> 
> The following patch fixes the problem by adding a new option -no-tsc for
> the i386 architecture which can be used to disable rdtsc when running on
> a cpu with speedstep enabled.
> 
> The patch works for me but I don't know if this is the best way of fixing
> this bug. If anyone has a better suggestion it is welcome.

This is not the best way to fix the bug. I see 4 solutions:

- Compute ticks_per_sec every n seconds and update the timers accordingly.

- Use a virtual cycle counter and update ticks_per_sec dynamically. This 
solution has the advantage of not needing any precise host timer (= very 
portable), but it will slow down QEMU a bit.

- Use a specific driver to get the time efficiently (= without system 
call) or to know when the host frequency changes

- Disable speedstep or use a CPU for which the rdtsc period does not 
vary even if the CPU frequency changes (the P4 has this feature if I 
remember correctly and all well designed CPUs should have such a feature !).

Fabrice.

Re: [Qemu-devel] [patch] option -no-tsc for i386 with speedstep

[Massimo Dal Zotto]

On Mon, Apr 25, 2005 at 08:58:06PM +0200, Fabrice Bellard wrote:
> 
> This is not the best way to fix the bug. I see 4 solutions:
> 
> - Compute ticks_per_sec every n seconds and update the timers accordingly.

This won't work well if an userpace scaling governor is continuously
changing the cpu frequency depending on system load (which is often
generated by qemu itself) as happens on my laptop.

> - Use a virtual cycle counter and update ticks_per_sec dynamically. This 
> solution has the advantage of not needing any precise host timer (= very 
> portable), but it will slow down QEMU a bit.

How could a virtual cycle counter measure real time with accuracy?

> - Use a specific driver to get the time efficiently (= without system 
> call) or to know when the host frequency changes
> 
> - Disable speedstep or use a CPU for which the rdtsc period does not 
> vary even if the CPU frequency changes (the P4 has this feature if I 
> remember correctly and all well designed CPUs should have such a feature !).

Speedstep is needed to save battery charge and my cpu, which is a
recent centrino, returns a variable rdtsc result. We are condemned
to use whichever cpu laptop makers decide to put in our laptops.

The tsc timer code in kernel 2.6 has an algorithm to compute a constant
clock rate from a variable tsc but I don't know how this could be ported
in user space. Could this code be reused, maybe with the help of kqemu?

In the meantime until we find a better solution could you give us some
explanation on why using a microseconds clock from gettimeofday instead
of rdtsc the guest os clock runs always 20% slower?

Having an almost precise system clock is a normal requirement and I
believe than even a sub-optimal solution would be better than a
constantly wrong clock like many of us are experiencing.

-- 
Massimo Dal Zotto

Re: [Qemu-devel] [patch] option -no-tsc for i386 with speedstep

[Jonas Maebe]

On 25 Apr 2005, at 22:17, Massimo Dal Zotto wrote:

> In the meantime until we find a better solution could you give us some
> explanation on why using a microseconds clock from gettimeofday instead
> of rdtsc the guest os clock runs always 20% slower?

Because a system call (which gettimeofday is) is very slow (two context 
switches).


Jonas

Re: [Qemu-devel] [patch] option -no-tsc for i386 with speedstep

[Filip Navara]

Jonas Maebe wrote:

> On 25 Apr 2005, at 22:17, Massimo Dal Zotto wrote:
>
>> In the meantime until we find a better solution could you give us some
>> explanation on why using a microseconds clock from gettimeofday instead
>> of rdtsc the guest os clock runs always 20% slower?
>
> Because a system call (which gettimeofday is) is very slow (two 
> context switches).

I remember there was a Linux patch which changed the gettimeofday 
function to read the info from user mode shared page (which is updated 
by the kernel) and thus avoided the syscall overhead. Not sure if/when 
it was integrated into the official sources...

(Once again Windows NT were ahead of time with this idea implemented way 
earlier ;-)

- Filip

Re: [Qemu-devel] [patch] option -no-tsc for i386 with speedstep

[Lionel Ulmer]

On Mon, Apr 25, 2005 at 10:38:42PM +0200, Filip Navara wrote:
> (Once again Windows NT were ahead of time with this idea implemented way 
> earlier ;-)

Strange, I got report from people testing stuff in Wine (about the same
RTDSC problem with laptops) finding out that 'gettimeofday' was taking less
time than 'QueryPerformanceCounter' in Windows (ie more calls per seconds).

             Lionel

-- 
		 Lionel Ulmer - http://www.bbrox.org/

Re: [Qemu-devel] [patch] option -no-tsc for i386 with speedstep

[Taras Glek]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Filip Navara wrote:

> Jonas Maebe wrote:
>
>> On 25 Apr 2005, at 22:17, Massimo Dal Zotto wrote:
>>
>>> In the meantime until we find a better solution could you give
>>> us some explanation on why using a microseconds clock from
>>> gettimeofday instead of rdtsc the guest os clock runs always
>>> 20% slower?
>>
>>
>> Because a system call (which gettimeofday is) is very slow (two
>> context switches).
>
>
> I remember there was a Linux patch which changed the gettimeofday
> function to read the info from user mode shared page (which is
> updated by the kernel) and thus avoided the syscall overhead. Not
> sure if/when it was integrated into the official sources...
>
> (Once again Windows NT were ahead of time with this idea
> implemented way earlier ;-)

Another *solution* is to use kernel mode linux =). See
http://web.yl.is.s.u-tokyo.ac.jp/~tosh/kml/
It makes already low Linux syscall overhead much lower.

Tara
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCbXMfmzwUPloTW30RAg+7AKCh0jGnYhvRp0/qtwm36wXDrCxd2ACgpdMi
LBQsxo56m98yPvkAPFJ0PAQ=
=jpKR
-----END PGP SIGNATURE-----

Re: [Qemu-devel] [patch] option -no-tsc for i386 with speedstep (solved)

[Massimo Dal Zotto]

[-- Attachment #1: Type: text/plain, Size: 1185 bytes --]

On Mon, Apr 25, 2005 at 10:24:45PM +0200, Jonas Maebe wrote:
> 
> On 25 Apr 2005, at 22:17, Massimo Dal Zotto wrote:
> 
> >In the meantime until we find a better solution could you give us some
> >explanation on why using a microseconds clock from gettimeofday instead
> >of rdtsc the guest os clock runs always 20% slower?
> 
> Because a system call (which gettimeofday is) is very slow (two context 
> switches).
> 

No, it has nothing to do with the speed of gettimeofday, which on my pc
takes only a few microsecons to execute.

The problem seems in (rounding?) errors in computations of ticks_per_sec
and/or next_transition_time and was solved (om my 600/1800 laptop) by
replacing the following instruction in my previous patch

    return get_clock();

with

    return get_clock()<<12;

It seems that simply changing the scale of ticks values reduces the
computation errors in the timer code, which must be somewhere even
if I haven't been able to find them.

The clock in the guest os is now running at the correct speed and the
sleep(1) command takes the exact real time to execute at any cpu speed.

The following patch should be then the correct one.

-- 
Massimo Dal Zotto

[-- Attachment #2: qemu-notsc.patch --]
[-- Type: text/plain, Size: 2035 bytes --]

--- qemu-0.6.2cvs20050425.orig/vl.c	2005-04-08 00:20:28.000000000 +0200
+++ qemu-0.6.2cvs20050425/vl.c	2005-04-25 23:03:57.096969536 +0200
@@ -135,6 +135,10 @@
 int prep_enabled = 0;
 int rtc_utc = 1;
 int cirrus_vga_enabled = 1;
+#ifdef __i386__
+static int notsc = 0;
+extern int64_t get_clock(void);
+#endif
 #ifdef TARGET_SPARC
 int graphic_width = 1024;
 int graphic_height = 768;
@@ -502,6 +506,9 @@
 int64_t cpu_get_real_ticks(void)
 {
     int64_t val;
+    if (notsc) {
+	return get_clock()<<12;
+    }
     asm volatile ("rdtsc" : "=A" (val));
     return val;
 }
@@ -590,6 +597,7 @@
     usec = get_clock() - usec;
     ticks = cpu_get_real_ticks() - ticks;
     ticks_per_sec = (ticks * 1000000LL + (usec >> 1)) / usec;
+    //printf("ticks_per_sec=%lld\n",ticks_per_sec);
 }
 
 /* compute with 96 bit intermediate result: (a*b)/c */
@@ -2792,6 +2800,9 @@
 #ifdef USE_CODE_COPY
            "-no-code-copy   disable code copy acceleration\n"
 #endif
+#ifdef __i386__
+           "-no-tsc         disable tsc as clock source\n"
+#endif
 #ifdef TARGET_I386
            "-isa            simulate an ISA-only system (default is PCI system)\n"
            "-std-vga        simulate a standard VGA card with VESA Bochs Extensions\n"
@@ -2863,6 +2874,7 @@
     QEMU_OPTION_hdachs,
     QEMU_OPTION_L,
     QEMU_OPTION_no_code_copy,
+    QEMU_OPTION_no_tsc,
     QEMU_OPTION_pci,
     QEMU_OPTION_isa,
     QEMU_OPTION_prep,
@@ -2931,6 +2943,9 @@
 #ifdef USE_KQEMU
     { "no-kqemu", 0, QEMU_OPTION_no_kqemu },
 #endif
+#ifdef __i386__
+    { "no-tsc", 0, QEMU_OPTION_no_tsc },
+#endif
 #ifdef TARGET_PPC
     { "prep", 0, QEMU_OPTION_prep },
 #endif
@@ -3216,6 +3231,11 @@
             case QEMU_OPTION_no_code_copy:
                 code_copy_enabled = 0;
                 break;
+#ifdef __i386__
+            case QEMU_OPTION_no_tsc:
+                notsc = 1;
+                break;
+#endif
             case QEMU_OPTION_nics:
                 nb_nics = atoi(optarg);
                 if (nb_nics < 0 || nb_nics > MAX_NICS) {

Re: [Qemu-devel] [patch] option -no-tsc for i386 with speedstep (solved)

[Jonas Maebe]

On 25 apr 2005, at 23:46, Massimo Dal Zotto wrote:

> No, it has nothing to do with the speed of gettimeofday, which on my pc
> takes only a few microsecons to execute.

Sorry, I had misread the original remark: I thought it was asking why 
qemu suddenly ran (supposedly) 20% slower when calling gettimeofday all 
the time, not that it was asking why the clock got skewed.


Jonas

Re: [Qemu-devel] [patch] option -no-tsc for i386 with speedstep

[Kyle Hayes]

On Monday 25 April 2005 04:15, Massimo Dal Zotto wrote:
> When qemu runs on an i386 cpu with speedstep enabled the clock of the
> guest os is not in sync with the clock on the host os because the
> vm_timer used for irq 0 generates interrupts at wrong rate when
> the host cpu frequency changes.
>
> The problem is that the vm_timer uses the rdtsc instruction and the
> value of ticks_per_sec, computed at start time, for calculating the
> expire time of vm_timers. While ticks_per_sec is constant the values
> returned by rdtsc are dependent on the current cpu clock, which is not
> constant if speedstep is used.

There is a discussion about this problem on either x86-secret.com or 
Sudhian.  I can't remember which :-(  In the latest series of P4 chips 
(both Xeon and desktop I think), the counter used for the rdtsc 
instruction stays at the same frequency while the rest of the CPU changes.  
This led to some "interesting" results with programs like CPU-Z.

Intel's response was that they now hold the counter at a stable frequency 
so that this works.  Unfortunately, there are years of processors which do 
not do that, so other workarounds must be done.

Best,
Kyle