From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Kallol Biswas" Subject: NIC: Queue assignment to guest OS Date: Thu, 14 Apr 2011 18:17:17 -0700 Message-ID: <003801cbfb0a$de67a410$9b36ec30$@com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2098220243==" Return-path: Content-Language: en-us List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --===============2098220243== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0039_01CBFAD0.3208CC10" Content-Language: en-us This is a multi-part message in MIME format. ------=_NextPart_000_0039_01CBFAD0.3208CC10 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Hi, I have a requirement to assign the transmit and receive queues of a NIC card to different guest operating systems so that the guests can directly submit transmit requests to controller and receive packets from NIC controller without going through a middle layer. The NIC controller is not SRIOV capable and there are a few MSI-X vectors available. What would be the best way to implement the requirement in Xen? I am new to Xen but have experience in a similar virtualization framework. Any input or pointer to document will be appreciated. Kallol ------=_NextPart_000_0039_01CBFAD0.3208CC10 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

I have a requirement to assign the = transmit and receive queues of a NIC card to different guest operating = systems so that the guests can directly submit transmit requests to = controller and receive packets from NIC controller without going through = a middle layer. The NIC controller is not SRIOV capable and there = are a few MSI-X vectors available.

What would = be the best way to implement the requirement in Xen? I am new to Xen but = have experience in a similar virtualization framework.

Any input or = pointer to document will be appreciated.

Kallol =

------=_NextPart_000_0039_01CBFAD0.3208CC10-- --===============2098220243== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============2098220243==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: NIC: Queue assignment to guest OS Date: Tue, 19 Apr 2011 14:47:50 -0400 Message-ID: <20110419184750.GB7427@dumpdata.com> References: <003801cbfb0a$de67a410$9b36ec30$@com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <003801cbfb0a$de67a410$9b36ec30$@com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Kallol Biswas Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org On Thu, Apr 14, 2011 at 06:17:17PM -0700, Kallol Biswas wrote: > Hi, > > I have a requirement to assign the transmit and receive queues of a NIC > card to different guest operating systems so that the guests can directly > submit transmit requests to controller and receive packets from NIC > controller without going through a middle layer. The NIC controller is not > SRIOV capable and there are a few MSI-X vectors available. OK. So you want to "export" the rx and tx queue (where each queue is associated with an MSI-X) in the guest. > > > > What would be the best way to implement the requirement in Xen? I am new to > Xen but have experience in a similar virtualization framework. How did you do this in other virtualization frameworks? From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kallol Biswas Subject: Re: NIC: Queue assignment to guest OS Date: Tue, 19 Apr 2011 11:50:35 -0700 Message-ID: <8DE07D65-675C-4AF7-8B4F-AA643173250F@nucleodyne.com> References: <003801cbfb0a$de67a410$9b36ec30$@com> <20110419184750.GB7427@dumpdata.com> Mime-Version: 1.0 (iPhone Mail 8C148a) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <20110419184750.GB7427@dumpdata.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Konrad Rzeszutek Wilk Cc: "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org Unfortunately other frameworks are not open source, and we can not use the d= esign and expose to all. On Apr 19, 2011, at 11:47 AM, Konrad Rzeszutek Wilk = wrote: > On Thu, Apr 14, 2011 at 06:17:17PM -0700, Kallol Biswas wrote: >> Hi, >>=20 >> I have a requirement to assign the transmit and receive queues of a NI= C >> card to different guest operating systems so that the guests can directly= >> submit transmit requests to controller and receive packets from NIC >> controller without going through a middle layer. The NIC controller is n= ot >> SRIOV capable and there are a few MSI-X vectors available. >=20 > OK. So you want to "export" the rx and tx queue (where each queue is assoc= iated > with an MSI-X) in the guest. >=20 >>=20 >>=20 >>=20 >> What would be the best way to implement the requirement in Xen? I am new t= o >> Xen but have experience in a similar virtualization framework. >=20 > How did you do this in other virtualization frameworks? From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: NIC: Queue assignment to guest OS Date: Tue, 19 Apr 2011 16:45:32 -0400 Message-ID: <20110419204532.GA13003@dumpdata.com> References: <003801cbfb0a$de67a410$9b36ec30$@com> <20110419184750.GB7427@dumpdata.com> <8DE07D65-675C-4AF7-8B4F-AA643173250F@nucleodyne.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <8DE07D65-675C-4AF7-8B4F-AA643173250F@nucleodyne.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Kallol Biswas Cc: "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org On Tue, Apr 19, 2011 at 11:50:35AM -0700, Kallol Biswas wrote: > Unfortunately other frameworks are not open source, and we can not use the design and expose to all. Please don't top post. > > On Apr 19, 2011, at 11:47 AM, Konrad Rzeszutek Wilk wrote: > > > On Thu, Apr 14, 2011 at 06:17:17PM -0700, Kallol Biswas wrote: > >> Hi, > >> > >> I have a requirement to assign the transmit and receive queues of a NIC > >> card to different guest operating systems so that the guests can directly > >> submit transmit requests to controller and receive packets from NIC > >> controller without going through a middle layer. The NIC controller is not > >> SRIOV capable and there are a few MSI-X vectors available. > > > > OK. So you want to "export" the rx and tx queue (where each queue is associated > > with an MSI-X) in the guest. > > > >> > >> > >> > >> What would be the best way to implement the requirement in Xen? I am new to > >> Xen but have experience in a similar virtualization framework. > > > > How did you do this in other virtualization frameworks? I am still at loss here how you guys managed this without ripping the network driver of a NIC card and passed the MSI-X to a guest and the full PCI device to _all_ of them and managed to negotiate some form of locking. From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Kallol Biswas" Subject: RE: NIC: Queue assignment to guest OS Date: Tue, 19 Apr 2011 16:24:59 -0700 Message-ID: <005801cbfee9$00ef50f0$02cdf2d0$@com> References: <003801cbfb0a$de67a410$9b36ec30$@com> <20110419184750.GB7427@dumpdata.com> <8DE07D65-675C-4AF7-8B4F-AA643173250F@nucleodyne.com> <20110419204532.GA13003@dumpdata.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20110419204532.GA13003@dumpdata.com> Content-Language: en-us List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: 'Konrad Rzeszutek Wilk' Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org If it is ok to modify host the network driver how should we do it in Xen? What modules/driver should be changed. The idea is to avoid backend/frontend implementation, and give direct HW access to guest for maximum performance. There are certain issues with guests accessing HW directly, one of them is guest to guest communication, and another is VM migration. Sorry again, legal agreement prevents us to disclose anything in previous implementation; we overcame these issues in a certain way. Xen NIC folks should be able to address the issues. > > > > How did you do this in other virtualization frameworks? I am still at loss here how you guys managed this without ripping the network driver of a NIC card and passed the MSI-X to a guest and the full PCI device to _all_ of them and managed to negotiate some form of locking. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel