timer interrupts, virqs, irq balance questions

timer interrupts, virqs, irq balance questions

[Ryan Harper]

I've been looking into bug [1]#195 and I have a couple of questions on
how timer interrupts and virqs are handled.  Is it possible for dom0
linux to irq balance timer interrupts to different cpus?  That is, if
xen sends a VIRQ_TIMER to vcpu0, backed by cpu0, is it possible for that
interrupt to be handled by vcpu1, backed by cpu1 ?  

After putting in some debug code in to timer_interrupt in linux, I can
see that when we get a "Time went backwards" message, that
shadow->system_timestamp+offset is significantly behind the previous
value of processed_system_time, and that the previous value was set by
the OTHER vcpu.  


Timer ISR/0: Time went backwards:
delta=-44611438 cpu_delta=15388562 shadow=69081255546
off=254172405 processed=69380039389 cpu_processed=69320039389
last_modified=CPU1, prev_st=69291796314, prev_offset=92519230
 0: 69320039389
 1: 69380000000
Timer ISR/0: Time went backwards:
delta=-1374402 cpu_delta=528625598 shadow=69081255546
off=777409441 processed=69860039389 cpu_processed=69330039389
last_modified=CPU1, prev_st=69291796314, prev_offset=571453258
 0: 69330039389
 1: 69860000000
Timer ISR/1: Time went backwards:
delta=-150020322 cpu_delta=70019067 shadow=69291796314
off=798222753 processed=70240039389 cpu_processed=70020000000
last_modified=CPU0, prev_st=70243918726, prev_offset=14792
 0: 70240039389
 1: 70020000000

Of note to me was the other cpu was the last one to modified the
processed_system_time value.  I believe that 1) either the
system_timestamp from each cpus are wildly out of sync or 2) some how,
an interrupt destined for a particular vcpu, is actually handled on a
different cpu, which causes problems since the handler using
smp_processor_id() to index into the shared page for current values

Any thoughts or comments on this would be most helpful. 


1. http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=195

-- 
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
ryanh@us.ibm.com

Re: timer interrupts, virqs, irq balance questions

[Keir Fraser]

On 20 Sep 2005, at 16:06, Ryan Harper wrote:

> I've been looking into bug [1]#195 and I have a couple of questions on
> how timer interrupts and virqs are handled.  Is it possible for dom0
> linux to irq balance timer interrupts to different cpus?  That is, if
> xen sends a VIRQ_TIMER to vcpu0, backed by cpu0, is it possible for 
> that
> interrupt to be handled by vcpu1, backed by cpu1 ?

No, event channels bound to VIRQs cannot have their cpu affinity 
changed. So vcpu0 VIRQ_TIMER will always call back to vcpu0, and so on.

  -- Keir