Re: FreeBSD timing issues and qemu (was: Re: [Qemu-devel] Re: Breakage with local APIC routing)

[John Baldwin <jhb@freebsd.org>]

On Thursday 10 September 2009 3:08:00 pm Luigi Rizzo wrote:
> On Thu, Sep 10, 2009 at 07:46:40PM +0200, Juergen Lock wrote:
> > On Wed, Sep 09, 2009 at 10:46:16PM +0200, Luigi Rizzo wrote:
> > > On Mon, Sep 07, 2009 at 10:59:55PM +0200, Juergen Lock wrote:
> > > > [I'm copying freebsd-current@FreeBSD.org because ppl there might know
> > > > more about this...]
> > > > 
> > > >  qemu on FreeBSD hosts used to be able to run a (FreeBSD at least) 
guest
> > > > with the same HZ as the host (like, 1000) with (mostly) proper timing
> > > > once, but no longer. :(  It seems there are two problems involved:
> > > > 
> > > >  a) use of apic seems to cause the clock irq rate to be doubled to 2 * 
HZ
> > > > (can anyone explain why?), i.e. a FreeBSD 7 guest on a FreeBSD 7 host
> > > > only gets proper timing after setting hint.apic.0.disabled=1 via the
> > > > loader.  (as can be verified by `vmstat -i' and `time sleep 2' in an
> > > > installed guest or via the fixit->cdrom/dvd shell on a FreeBSD livefs
> > > > or dvd1 iso.)
> > > > 
> > > >  b) qemu running on FreeBSD 8 hosts (and most likely head) has the
> > > > additional problem of running its timers only at HZ/2 when using
> > > > setitimer(2) (called `-clock unix' in qemu), as seen below.  (as also
> > > 
> > > this problem in 8.x is caused by the bug i described here yesterday:
> > > 
> > > 
http://lists.freebsd.org/pipermail/freebsd-current/2009-September/011393.html
> > > 
> > > In qeumu, the setitimer call (in file vl.c) has a timeout of 1 tick
> > > which maps to callout_reset(..., 1, ...) and because (due to the bug)
> > > 8.x processes callouts 1 tick late, this effectively halves the clock 
rate.
> > > 
> > Thanx for the pointer!
> > 
> >  The proposed patch in that post didn't make a different here tho,
> > guest still sees only half host HZ clock irq rate. (i.e. ~500 Hz.)
> > 
> >  Here is the patch I used, to make sure I patched what you meant...
> > 
> > Index: sys/kern/kern_timeout.c
> > @@ -323,7 +323,7 @@ softclock(void *arg)
> >  	steps = 0;
> >  	cc = (struct callout_cpu *)arg;
> >  	CC_LOCK(cc);
> > -	while (cc->cc_softticks != ticks) {
> > +	while (cc->cc_softticks-1 != ticks) {
> >  		/*
> >  		 * cc_softticks may be modified by hard clock, so cache
> >  		 * it while we work on a given bucket.
> > 
> 
> as mentioned in the followup message in that thread,
> you also need this change in callout_tick()
> 
>         mtx_lock_spin_flags(&cc->cc_lock, MTX_QUIET);
>      -  for (; (cc->cc_softticks - ticks) < 0; cc->cc_softticks++) {
>      +  for (; (cc->cc_softticks - ticks) <= 0; cc->cc_softticks++) {
>                 bucket = cc->cc_softticks & callwheelmask;

I would fix the style in the first hunk (spaces around '-') but I think you 
should commit this and get it into 8.0.  I think a per-CPU ticks might prove 
very problematic as 'ticks' is rather widely used (though I would find that 
cleaner perhaps).

-- 
John Baldwin