Re: [xen-devel] create irq failed due to move_cleanup_count always being set

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 04/01/12 04:37, Liuyongan wrote:
> Hi, all
>    
>     I'm using xen-4.0 to do a test. And when I create a domain, it failed due to create_irq() failure. As only 33 domains were successfully created and destroyed before I got the continuous failures, and the domain just before the failure was properly destroyed(at least destroy_irq() was properly called, which will clear move_in_progress, according to the prink-message). So I can conclude for certain that __assign_irq_vector failed due to move_cleanup_count always being set.

Is it always 33 domains it takes to cause the problem, or does it vary? 
If it varies, then I think you want this patch
http://xenbits.xensource.com/hg/xen-unstable.hg/rev/68b903bb1b01 which
corrects the logic which works out which moved vectors it should clean
up.  Without it, stale irq numbers build up in the per-cpu irq_vector
tables leading to __assign_irq_vector failing with -ENOSPC as it find
find a vector to allocate.

> //this is the normal case when create and destroy domain whose id is 31;
> (XEN) irq.c:1232:d0 bind pirq 79, irq 77, share flag:0
> (XEN) irq.c:1377: dom31: pirq 79, irq 77 force unbind
> (XEN) irq.c:1593: dom31: forcing unbind of pirq 79
> (XEN) irq.c:223, destroy irq 77
>
> //domain id 32 is created and destroyed correctly also.
> (XEN) irq.c:1232:d0 bind pirq 79, irq 77, share flag:0
> (XEN) irq.c:1377: dom32: pirq 79, irq 77 force unbind
> (XEN) irq.c:1593: dom32: forcing unbind of pirq 79
> (XEN) irq.c:223, destroy irq 77
>
> //all the subsequent domain creation failed, below lists only 3 times:
> (XEN) physdev.c:88: dom33: can't create irq for msi!
> (XEN) physdev.c:88: dom34: can't create irq for msi!
> (XEN) physdev.c:88: dom35: can't create irq for msi!
>
>      I think this might be a bug and might have fixed, so I compare my code with 4.1.2 and search the mail list for potential patches. (http://xen.markmail.org/search/?q=move_cleanup_count#query:move_cleanup_count+page:6+mid:fpkrafqbeyiauvhs+state:results) submit a patch which add locks in __assign_irq_vector. Can anybody explain why this lock is needed? Or is there a patch that might fix my bug? Thx.

This patch fixes a problem where IOAPIC line level interrupts cease for
a while.  It has nothing to do with MSI interrupts.  (Also, there are no
locks altered, and xen-4.0-testing seems to have gained an additional
hunk in hvm/vmx code unrelated to the original patch.)

>     Addition message: my board is arch-x86, no domains left when failed to create new ones, create_irq failure lasted one day until I reboot the board, and the irq number allocated is used certainly for a msi dev.
>
> Yong an Liu
> 2012.1.4
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com