From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ritu kaur Subject: Re: Shared memory and event channel Date: Mon, 22 Feb 2010 09:36:37 -0800 Message-ID: <29b32d341002220936q2f6f3cdaif3cbb766d1e644d1@mail.gmail.com> References: <29b32d341002211058l7e283336pa4fdfd0dc0b7124b@mail.gmail.com> <1266787199.24577.18.camel@agari.van.xensource.com> <29b32d341002211533k4956a129ifff18281cfa92e41@mail.gmail.com> <1266825344.4996.183.camel@ramone.somacoma.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0621185793==" Return-path: In-Reply-To: <1266825344.4996.183.camel@ramone.somacoma.net> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Daniel Stodden Cc: "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org --===============0621185793== Content-Type: multipart/alternative; boundary=000e0cd106c010c6d5048033e0b6 --000e0cd106c010c6d5048033e0b6 Content-Type: text/plain; charset=ISO-8859-1 Hi Daniel, Thanks once again and reply/question's inline... On Sun, Feb 21, 2010 at 11:55 PM, Daniel Stodden wrote: > On Sun, 2010-02-21 at 18:33 -0500, Ritu kaur wrote: > > Hi Daniel, > > > > Thanks for the explanation, however, my main question is still > > unanswered > > > > "My understanding is one has to use xenbus(registering and monitoring > > for device creation) mechanism to setup shared mem rings and event > > channel between dom's and there is no other way to do it." > > Anything capable of passing two integers around could give you a shared > memory connection. > I want to know more about "anything capable". I have read documents from xen.org and so far my understanding is that the only mechanism of setting up shared mem rings is via xenbus(since PV drivers are the only users currently), pointers or example code which uses alternate mechanism will be helpful. > > All I need is to setup NIC register reads and writes from domUs(ioctl > > is one such application which I have been discussing in another > > thread) and to implement this I considered an option of using shared > > memory rings. If answer to the above question is "Yes" then probably I > > will not take that route. > > You need to understand that netback and the interface corresponding to > your hardware NIC have no direct association. Netback just provides a > kernel network interface, not the hardware controller. As any good > network citizen, it passes packet buffers around, without any > assumptions were they go. Not even an implicit assumption that somewhere > in there is a physical NIC involved at all. > > There's a galaxy of layer 2/3 stuff between netback and the hardware. > Bridging, routing, NAT etc., all in different variants. For XCP it's > typically bridged. Netback won't know, because it doesn't have to. > > And least of all it wants to learn about your NIC. > > > It will be really helpful if you can elaborate on "why not just write > > an auxiliary driver adding only added functionality but remaining > > separate from the base networking stack" > > You would not even have to take down the vifs to prevent domU access to > a NIC. The aren't bound to the NIC anyway. > > For low-level access to the NIC, you also don't necessarily need to set > up message passing. Even if you would, none of that belongs into the PV > interface. > > I'm not sure right now how easy the control plane in XCP will make it > without other domU's notice, but maybe consider something like: > > 1. Take the physical NIC out of the virtual network. > 2. Take the driver down. > 3. Pass access to the NIC to a domU. > 4. Let domU do the unspeakable. > 5.-7. Revert 3,2,1 to normal. > > This won't mess with the the PV drivers. Get PCI passthrough to work for > 3 and 4 and you save yourself a tedious ring protocol design. If not, > consider doing the hardware programming in dom0, because there's not > much left for domU anyway. > > You need a split toolstack to get the dom0 network control steps on > behalf of domU done. Might be just a scripted agent, accessible to domU > via a couple RPCs. Could also turn out to be as simple as talking > through the primary vif, because the connection between domU and dom0 > could remain unaffected. > > PCI passthrough is via config changes and no code changes, if that's the case I am not sure how it would solve multiple domU accesses. For the second paragraph, do you have recommended readings? frankly I don't completely understand the solution any pointers appreciated. In addition, registers in NIC are memory mapped(ioremap function is used, and in ioctls memcpy_toio and memcpy_fromio is used to write/read registers) and wanted to know if its possible to map memory from dom0 into domU's? I haven't looked into details of issues that will comeup with mapping, but thought of checking. ioctl is one application which uses register reads/writes and there are other modules(in kernel I believe) is being developed which need register read/write functionality as well. Thanks Daniel > > > > > > > --000e0cd106c010c6d5048033e0b6 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Daniel,

Thanks once again and reply/question's inline...
<= br>
On Sun, Feb 21, 2010 at 11:55 PM, Daniel Stod= den <dani= el.stodden@citrix.com> wrote:
On Sun, 2010-02-21 at 18:33 -0500, Ritu kaur wrote:
> Hi Daniel,
>
> Thanks for the explanation, however, my main question is still
> unanswered
>
> "My understanding is one has to use xenbus(registering and monito= ring
> for device creation) mechanism to setup shared mem rings and event
> channel between dom's and there is no other way to do it."
Anything capable of passing two integers around cou= ld give you a shared
memory connection.

I want to know more about= "anything capable". I have read documents from xen.org and so far my understanding is that the only mechanis= m of setting up shared mem rings is via xenbus(since PV drivers are the onl= y users currently),=A0 pointers or example code which uses alternate mechan= ism will be helpful.


> All I need is to setup NIC register reads and = writes from domUs(ioctl
> is one such application which I have been discussing in another
> thread) and to implement this I considered an option of using shared > memory rings. If answer to the above question is "Yes" then = probably I
> will not take that route.

You need to understand that netback and the interfa= ce corresponding to
your hardware NIC have no direct association. Netback just provides a
kernel network interface, not the hardware controller. As any good
network citizen, it passes packet buffers around, without any
assumptions were they go. Not even an implicit assumption that somewhere in there is a physical NIC involved at all.

There's a galaxy of layer 2/3 stuff between netback and the hardware. Bridging, routing, NAT etc., all in different variants. For XCP it's typically bridged. Netback won't know, because it doesn't have to.<= br>
And least of all it wants to learn about your NIC.

> It will be really helpful if you can elaborate= on "why not just write
> an auxiliary driver adding only added functionality but remaining
> separate from the base networking stack"

You would not even have to take dow= n the vifs to prevent domU access to
a NIC. The aren't bound to the NIC anyway.

For low-level access to the NIC, you also don't necessarily need to set=
up message passing. Even if you would, none of that belongs into the PV
interface.

I'm not sure right now how easy the control plane in XCP will make it without other domU's notice, but maybe consider something like:

=A01. Take the physical NIC out of the virtual network.
=A02. Take the driver down.
=A03. Pass access to the NIC to a domU.
=A04. Let domU do the unspeakable.
=A05.-7. Revert 3,2,1 to normal.

This won't mess with the the PV drivers. Get PCI passthrough to work fo= r
3 and 4 and you save yourself a tedious ring protocol design. If not,
consider doing the hardware programming in dom0, because there's not much left for domU anyway.

You need a split toolstack to get the dom0 network control steps on
behalf of domU done. Might be just a scripted agent, accessible to domU
via a couple RPCs. Could also turn out to be as simple as talking
through the primary vif, because the connection between domU and dom0
could remain unaffected.


PCI passthrough is via config changes= and no code changes, if that's the case I am not sure how it would sol= ve multiple domU accesses. For the second paragraph, do you have recommende= d readings? frankly I don't completely understand the solution any poin= ters appreciated.

In addition, registers in NIC are memory mapped(ioremap function is use= d, and in ioctls memcpy_toio and memcpy_fromio is used to write/read regist= ers) and wanted=A0 to know if its possible to map memory from dom0 into dom= U's? I haven't looked into details of issues that will comeup with = mapping, but thought of checking. ioctl is one application which uses regis= ter reads/writes and there are other modules(in kernel I believe) is being = developed which need register read/write functionality as well.

Thanks

Daniel







--000e0cd106c010c6d5048033e0b6-- --===============0621185793== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============0621185793==--