NIC: Queue assignment to guest OS

NIC: Queue assignment to guest OS

[Kallol Biswas]

[-- Attachment #1.1: Type: text/plain, Size: 572 bytes --]

Hi,

    I have a requirement to assign the transmit and receive queues of a NIC
card to different guest operating systems so that the guests can directly
submit transmit requests to controller and receive packets from NIC
controller without going through a middle layer.  The NIC controller is not
SRIOV capable and there are a few MSI-X vectors available.

 

What would be the best way to implement the requirement in Xen? I am new to
Xen but have experience in a similar virtualization framework.

 

Any input or pointer to document will be appreciated.

 

Kallol 


[-- Attachment #1.2: Type: text/html, Size: 2403 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: NIC: Queue assignment to guest OS

[Konrad Rzeszutek Wilk]

On Thu, Apr 14, 2011 at 06:17:17PM -0700, Kallol Biswas wrote:
> Hi,
> 
>     I have a requirement to assign the transmit and receive queues of a NIC
> card to different guest operating systems so that the guests can directly
> submit transmit requests to controller and receive packets from NIC
> controller without going through a middle layer.  The NIC controller is not
> SRIOV capable and there are a few MSI-X vectors available.

OK. So you want to "export" the rx and tx queue (where each queue is associated
with an MSI-X) in the guest.

> 
>  
> 
> What would be the best way to implement the requirement in Xen? I am new to
> Xen but have experience in a similar virtualization framework.

How did you do this in other virtualization frameworks?

Re: NIC: Queue assignment to guest OS

[Kallol Biswas]

Unfortunately other frameworks are not open source, and we can not use the design and expose to all.

On Apr 19, 2011, at 11:47 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

> On Thu, Apr 14, 2011 at 06:17:17PM -0700, Kallol Biswas wrote:
>> Hi,
>> 
>>    I have a requirement to assign the transmit and receive queues of a NIC
>> card to different guest operating systems so that the guests can directly
>> submit transmit requests to controller and receive packets from NIC
>> controller without going through a middle layer.  The NIC controller is not
>> SRIOV capable and there are a few MSI-X vectors available.
> 
> OK. So you want to "export" the rx and tx queue (where each queue is associated
> with an MSI-X) in the guest.
> 
>> 
>> 
>> 
>> What would be the best way to implement the requirement in Xen? I am new to
>> Xen but have experience in a similar virtualization framework.
> 
> How did you do this in other virtualization frameworks?

Re: NIC: Queue assignment to guest OS

[Konrad Rzeszutek Wilk]

On Tue, Apr 19, 2011 at 11:50:35AM -0700, Kallol Biswas wrote:
> Unfortunately other frameworks are not open source, and we can not use the design and expose to all.

Please don't top post.
> 
> On Apr 19, 2011, at 11:47 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> 
> > On Thu, Apr 14, 2011 at 06:17:17PM -0700, Kallol Biswas wrote:
> >> Hi,
> >> 
> >>    I have a requirement to assign the transmit and receive queues of a NIC
> >> card to different guest operating systems so that the guests can directly
> >> submit transmit requests to controller and receive packets from NIC
> >> controller without going through a middle layer.  The NIC controller is not
> >> SRIOV capable and there are a few MSI-X vectors available.
> > 
> > OK. So you want to "export" the rx and tx queue (where each queue is associated
> > with an MSI-X) in the guest.
> > 
> >> 
> >> 
> >> 
> >> What would be the best way to implement the requirement in Xen? I am new to
> >> Xen but have experience in a similar virtualization framework.
> > 
> > How did you do this in other virtualization frameworks?

I am still at loss here how you guys managed this without ripping the network
driver of a NIC card and passed the MSI-X to a guest and the full PCI device
to _all_ of them and managed to negotiate some form of locking.

RE: NIC: Queue assignment to guest OS

[Kallol Biswas]

If it is ok to modify host the network driver how should we do it in Xen?
What modules/driver should be changed.
The idea is to avoid backend/frontend implementation, and give direct HW
access to guest for maximum performance. There are certain issues with
guests accessing HW directly, one of them is guest to guest communication,
and another is VM migration.

Sorry again, legal agreement prevents us to disclose anything in previous
implementation; we overcame these issues in a certain way. Xen NIC folks
should be able to address the issues.

> > 
> > How did you do this in other virtualization frameworks?

I am still at loss here how you guys managed this without ripping the
network
driver of a NIC card and passed the MSI-X to a guest and the full PCI device
to _all_ of them and managed to negotiate some form of locking.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel