Re: [PATCH] Add a timer mode that disables pending missed ticks

[Dave Winchell <dwinchell@virtualiron.com>]

Hi Keir,

I've added comments below.
See my next mail on some interesting performance numbers.

thanks,
Dave

Keir Fraser wrote:

>On 7/11/07 19:38, "Dave Winchell" <dwinchell@virtualiron.com> wrote:
>
>  
>
>>My feeling is that we should go full SYNC. Yes, in theory the
>>guests should be able to handle ASYNC, but in reality it appears that
>>some do not. Since it is easy for us to give them SYNC,
>>lets just do it and not stress them out.
>>    
>>
>
>One problem with pure SYNC is there's a fair chance you won't deliver any
>ticks at all for a long time, if the guest only runs in short bursts (e.g.,
>I/O bound) and happens not to be running on any tick boundary. I'm not sure
>how much that matters. It could cause time goes backwards if the time
>extrapolation via the TSC is not perfectly accurate, or cause problems if
>there are any assumptions that TSC delta since last tick fits in 32 bits
>(less likely in x64 code I suppose). Anyway, my point is that only testing
>VCPUs under full load may cause us to optimise in ways that have nasty
>unexpected effects for other workloads.
>  
>
I agree that this could be a problem. I have an idea that could give us full
SYNC and eliminate the long periods without clock interrupts.
In pt_process_missed_ticks() when missed_ticks > 0 set pt->run_timer = 1.
In pt_save_timer():

    list_for_each_entry ( pt, head, list )
        if(!pt->run_timer)
             stop_timer(&pt->timer);

And in pt_timer_fn():

    pt->run_timer = 0;

So, for a guest that misses a tick, we will interrupt him once from the
descheduled state and then leave him alone in the descheduled state.

>  
>
>>For default mode as checked into unstable is now,
>>64 bit guests should run quite fast as missed is calculated and then a bunch
>>of additional interrupts are delivered. On the other hand
>>32bit guests very well in default mode.
>>
>>For the original code, before we put in the constant tsc offset business,
>>64bit guests run poorly and 32bit quests very well time-wise.
>>    
>>
>
>The default mode hasn't changed. Are you under the impression that
>missed-ticks-but-no-delay-of-tsc is the default mode now? I know x64 guests
>run badly with that because they treat every one of the missed ticks they
>receive as a full tick.
>  
>
Sorry, I was confused.
However, the default mode will still run poorly for 64 bit guests because
of the pending_nr's accumulated while the guest has interrupts disabled.
As I recall, the effect is quite large, on the order of 10% error.
I'll get you a number later today.

> -- Keir
>
>  
>
>>>Or is the lack of
>>>synchronization of TSCs across VCPUs causing issues that you're trying to
>>>avoid?
>>>      
>>>
>>This does cause issues, but its not the only contributor to poor timing.
>>Having TSCs synchronized across vcpus will help some of the time going
>>backwards problems we have seen, I think.
>>
>>Regards,
>>Dave
>>
>>Keir Fraser wrote:
>>
>>    
>>
>>>On 7/11/07 17:29, "Keir Fraser" <Keir.Fraser@cl.cam.ac.uk> wrote:
>>>
>>> 
>>>
>>>      
>>>
>>>>So, you can see we send an interrupt immediately (and ASYNC) if any ticks
>>>>have been missed, but then successive ticks are delivered 'on the beat'. A
>>>>possible middleground? Or perhaps we should just go with SYNC after all...
>>>>   
>>>>
>>>>        
>>>>
>>>How do these Linux x64 guests fare with the original and default timer mode,
>>>by the way? I would expect that time should be accounted pretty accurately
>>>in that mode, albeit with more interrupts than you'd like. Or is the lack of
>>>synchronisation of TSCs across VCPUs causing issues that you're trying to
>>>avoid?
>>>
>>>-- Keir
>>>
>>>
>>> 
>>>
>>>      
>>>
>
>
>  
>