Re: [PATCH 0/5] Add MSI support to XEN

[Keir Fraser <keir.fraser@eu.citrix.com>]

[-- Attachment #1.1: Type: text/plain, Size: 3693 bytes --]

Thanks,

I¹ll have to look at the patches regarding the per-domain pirq changes. That
sounds like it probably makes sense, but I seem to remember there were big
changes to the irq architecture and irq naming in the hypervisor in previous
iterations of these patches, which I didn¹t understand.

This IRQ storm issue still needs properly resolving. Noone has yet explained
how a message-based interrupt source can cause an irq storm. Storms are
inherently a property of level-triggered sources, where ACK/EOI immediately
causes re-sampling of the interrupt line and re-assertion of the interrupt
at the CPU. How can anything similar happen with MSI? You (Intel) are
probably uniquely placed to answer this question, since you manufacture the
chipset and NIC which exhibit this problem.

 -- Keir

On 27/3/08 06:55, "Shan, Haitao" <haitao.shan@intel.com> wrote:

> The basic idea including:
> 1) Keep vector global resource owned by xen, while split pirq into per-domain
> information.
> 2) Domain0 kernel will operate msi resource for domain0/domU, while QEMU will
> operate MSI resource for HVM domain.
> 3) Xen will do EOI for MSI interrupt.
> 
> Signed-off-by: Yunhong Jiang <yunhong.jiang@intel.com
> <mailto:yunhong.jiang@intel.com> >
>  
>     There are no much changes made compared with the original patches. But
> there do have some issues that we need your kind comments.
>     1> ACK-NEW method is necessary to avoid IRQ storm. But it causes the
> deadlock. 
>          During my tests, I do find there can be deadlock with patches
> applied. When assigned a NIC device to HVM domain, the scenario is: Dom0 is
> waiting to IDE interrupt (vector 0x21); HVM domain is waiting for qemu¹s IDE
> emulation and thus blocked; NIC interrupt (MSI vector 0x31) is waiting for
> injection to HVM domain since it is blocked now; IDE interrupt is waiting for
> NIC interrupt since NIC interrupt is of high priority but not ACKed by XEN
> now. When IDE interrupt and NIC interrupt are delivered to the same CPU, and
> when guest OS is Vista, the phenomenon is easy to be observed.
>     2> Without ACK-NEW, some naughty NIC devices as we observed will bring IRQ
> storms. For this phenomenon, I think Yunhong can comment more. Basically,
> writing EOI without mask the source of MSI will bring IRQ storm. Although the
> reason is under investigation, XEN should anyhow handle such bogous device,
> right?
>     3> Using ACK-OLD and masking the MSI when writing EOI can be solution.
> However, XEN does not own PCI configuration spaces.
>  
>     We also tried some work arounds.
>     One work around might be using a timer to force a EOI within some time
> interval. This method is already implemented in VT-D¹s code. However, with
> this approach, if the timer is fired and EOI is written, this is essentially
> the same apporach as option 2.
>     Another approach is to never deliver these two IRQs to the same CPU. But
> this is really ugly and can not be applied to UP.
>     We have also considered using VT-D 2 interrupt remapping feature.
> According to the spec, there is no bit in the remapping table to mask the
> interrupt. Therefore, this can not be combined with option 2 to solve the
> issue. Masking the interrupt still needs accessing PCI configuration spaces.
>  
>     We think the most clean method may be to move ownership from dom0 to VMM.
> However, this is a great change. This should be well discussed in community
> and need your comments.
>  
>     These patch series sent out can be served as a discussion materials. What
> is your comments on the patches and the issues, Keir?
>     
> Thanks!



[-- Attachment #1.2: Type: text/html, Size: 4565 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel