PCI Passthrough & iperf Throughput Skew

PCI Passthrough & iperf Throughput Skew

[Jambunathan K]

I am seeing a strange behaviour with Xen-3.0.4 which seems to suggest
that Dom0 has either higher Scheduling affinity or Interrupt affinity.

I am in the initial stages of investigation and it is possible that my
hunch is totally wrong. I would like to hear comments or questions
from the experts.

I have a multifunction NIC card, with one function being used by Dom0
and another one exported to DomU using PCI passthrough

Dom0 and DomU have almost the same memory allocated to each
(~1.3G). The ports themselves are operating at the same speed and
share the same IRQ.

iperf (as a client) is started on Dom0 in the same way as it is
started on DomU.

iperf results are skewed in favour of Dom0 in the ratio 10:1.

Given all other things are similar wouldn't it be normal to expect
throughputs in the ratio of 1:1.

(ps:- The 2 ports on the Xen machine is connected
"point-to-point-wise" to 2 ports on another machine running Linux
natively)


Regards,
Jambunathan K.

Re: PCI Passthrough & iperf Throughput Skew

[Jambunathan K]

Additional Data Point:

If I just exercise iperf in Dom0 or DomU *separately*, I get identical
throughput .  The 10:1 skew happens only when the iperf is running
concurrently on the Dom0 and DomU.

Regards,
Jambunathan K.

Jambunathan K wrote:
> I am seeing a strange behaviour with Xen-3.0.4 which seems to suggest
> that Dom0 has either higher Scheduling affinity or Interrupt affinity.
>
> I am in the initial stages of investigation and it is possible that my
> hunch is totally wrong. I would like to hear comments or questions
> from the experts.
>
> I have a multifunction NIC card, with one function being used by Dom0
> and another one exported to DomU using PCI passthrough
>
> Dom0 and DomU have almost the same memory allocated to each
> (~1.3G). The ports themselves are operating at the same speed and
> share the same IRQ.
>
> iperf (as a client) is started on Dom0 in the same way as it is
> started on DomU.
>
> iperf results are skewed in favour of Dom0 in the ratio 10:1.
>
> Given all other things are similar wouldn't it be normal to expect
> throughputs in the ratio of 1:1.
>
> (ps:- The 2 ports on the Xen machine is connected
> "point-to-point-wise" to 2 ports on another machine running Linux
> natively)
>
>
> Regards,
> Jambunathan K.
>
>
>
>
>
>
>
>