Re: [PATCH 0/8][v2] MSI-X mask emulation support for assigned device

Sheng Yang
2010-10-21T07:40:08Z

On Wednesday 20 October 2010 17:51:01 Avi Kivity wrote:
> On 10/20/2010 10:26 AM, Sheng Yang wrote:
> > Here is v2.
> >
> > Changelog:
> >
> > v1->v2
> >
> > The major change from v1 is I've added the in-kernel MSI-X mask emulation
> > support, as well as adding shortcuts for reading MSI-X table.
> >
> > I've taken Michael's advice to use mask/unmask directly, but unsure about
> > exporting irq_to_desc() for module...
> >
> > Also add flush_work() according to Marcelo's comments.
>
> Any performance numbers? What are the affected guests? just RHEL 4, or
> any others?

At least current RHEL5 series would be affected. I have done an simple benchmark on
RHEL5u5 guest with 512m memory and 1 cpu. Device is one 10G NIC with SRIOV. One VF
was assigned to guest to communicate the PF in the host. 3 threads had been used
in iperf of guest to push the CPU utilization to 100%. In this condition, QEmu
method's bandwidth is about 20% lower than in-kernel one(~7.5G vs ~9G). Interrupt
rate in this condition is about 20k/sec.

The reason is 2.6.18 kernel used mask_msi in ack() for MSI chip, caused significant
mask bit operations when interrupt rate is high.

We have also reproduced the issue on some large scale benchmark on the guest with
newer kernel like 2.6.30 on Xen, under very high interrupt rate, due to some
interrupt rate limitation mechanism in kernel.
>
> Alex, Michael, how would you do this with vfio?

--
regards
Yang, Sheng