From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ritu kaur Subject: Re: Shared memory and event channel Date: Mon, 22 Feb 2010 14:16:30 -0800 Message-ID: <29b32d341002221416t4e00b899q18e07a69ad24b07f@mail.gmail.com> References: <29b32d341002211058l7e283336pa4fdfd0dc0b7124b@mail.gmail.com> <1266787199.24577.18.camel@agari.van.xensource.com> <29b32d341002211533k4956a129ifff18281cfa92e41@mail.gmail.com> <1266825344.4996.183.camel@ramone.somacoma.net> <29b32d341002220936q2f6f3cdaif3cbb766d1e644d1@mail.gmail.com> <1266874463.27288.57.camel@agari.van.xensource.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1045754906==" Return-path: In-Reply-To: <1266874463.27288.57.camel@agari.van.xensource.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Daniel Stodden Cc: "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org --===============1045754906== Content-Type: multipart/alternative; boundary=000e0cd1a0aaf7dbe0048037c8ea --000e0cd1a0aaf7dbe0048037c8ea Content-Type: text/plain; charset=ISO-8859-1 Hi Daniel, Please see inline... On Mon, Feb 22, 2010 at 1:34 PM, Daniel Stodden wrote: > On Mon, 2010-02-22 at 12:36 -0500, Ritu kaur wrote: > > > > > I'm not sure right now how easy the control plane in XCP will > > make it > > without other domU's notice, but maybe consider something > > like: > > > > 1. Take the physical NIC out of the virtual network. > > 2. Take the driver down. > > 3. Pass access to the NIC to a domU. > > 4. Let domU do the unspeakable. > > 5.-7. Revert 3,2,1 to normal. > > > > This won't mess with the the PV drivers. Get PCI passthrough > > to work for > > 3 and 4 and you save yourself a tedious ring protocol design. > > If not, > > consider doing the hardware programming in dom0, because > > there's not > > much left for domU anyway. > > > > You need a split toolstack to get the dom0 network control > > steps on > > behalf of domU done. Might be just a scripted agent, > > accessible to domU > > via a couple RPCs. Could also turn out to be as simple as > > talking > > through the primary vif, because the connection between domU > > and dom0 > > could remain unaffected. > > > > > > > > PCI passthrough is via config changes and no code changes, if that's > > the case I am not sure how it would solve multiple domU accesses. > > My understanding after catching up a little on the past of this thread > was that you want the network controller in some maintenance mode. Is > this correct? > All I need to is access NIC registers via domU's(network controller will still be working normally). Using PCI passthrough solves the problem for a domU, however, it doesn't solve when multiple domU's wanting to read NIC registers(ex. statistics). > > To get it there you will need to temporarily remove it from the virtual > network topology. > > The PCI passthrough mode might solve your second problem, which is how > the domU is supposed to access the device once it's been pulled off the > data path. > > > For the second paragraph, do you have recommended readings? frankly I > > don't completely understand the solution any pointers appreciated. > > > In addition, registers in NIC are memory mapped(ioremap function is > > used, and in ioctls memcpy_toio and memcpy_fromio is used to > > write/read registers) and wanted to know if its possible to map > > memory from dom0 into domU's? > > Yes. This is the third problem, which is how to program a device. I'd > recommend "Linux Device Drivers" on that subject. There are also free > books like http://tldp.org/LDP/tlk/tlk-title.html. Examples likely > outdate, but the concepts remain. > > If the device is memory mapped, it doesn't mean it's in memory. It means > it's in the machine memory address space. The difference should become > clear once you're done with understanding your driver. > > Is this the reason why you are so concerned about the memory sharing > mechanism? No not really. I wanted to use shared memory between dom's as a solution for multiple domU access(since pci passthrough doesn't solve it). The clarification I wanted here(NIC registers are memory mapped), can I take "machine memory address space(which is in dom0)" and remap it to domU's such that I can get multiple domU access. To summarize, 1. PCI passthrough mechanism works for single domU 2. Shared memory rings between dom's as a solution to have multiple domU access, not a workable solution though 3. Take mapped machine address in dom0 and remap it into domU's(just another thought, not sure it works) and wanted clarification here. Thanks > The good news is now you won't need to bother, that's only > for memory. :) > > Daniel > > > --000e0cd1a0aaf7dbe0048037c8ea Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Daniel,

Please see inline...

On= Mon, Feb 22, 2010 at 1:34 PM, Daniel Stodden <daniel.stodden@citrix.com> wrote:
<= div class=3D"h5">On Mon, 2010-02-22 at 12:36 -0500, Ritu kaur wrote:

>
> =A0 =A0 =A0 =A0 I'm not sure right now how easy the control plane = in XCP will
> =A0 =A0 =A0 =A0 make it
> =A0 =A0 =A0 =A0 without other domU's notice, but maybe consider so= mething
> =A0 =A0 =A0 =A0 like:
>
> =A0 =A0 =A0 =A0 =A01. Take the physical NIC out of the virtual network= .
> =A0 =A0 =A0 =A0 =A02. Take the driver down.
> =A0 =A0 =A0 =A0 =A03. Pass access to the NIC to a domU.
> =A0 =A0 =A0 =A0 =A04. Let domU do the unspeakable.
> =A0 =A0 =A0 =A0 =A05.-7. Revert 3,2,1 to normal.
>
> =A0 =A0 =A0 =A0 This won't mess with the the PV drivers. Get PCI p= assthrough
> =A0 =A0 =A0 =A0 to work for
> =A0 =A0 =A0 =A0 3 and 4 and you save yourself a tedious ring protocol = design.
> =A0 =A0 =A0 =A0 If not,
> =A0 =A0 =A0 =A0 consider doing the hardware programming in dom0, becau= se
> =A0 =A0 =A0 =A0 there's not
> =A0 =A0 =A0 =A0 much left for domU anyway.
>
> =A0 =A0 =A0 =A0 You need a split toolstack to get the dom0 network con= trol
> =A0 =A0 =A0 =A0 steps on
> =A0 =A0 =A0 =A0 behalf of domU done. Might be just a scripted agent, > =A0 =A0 =A0 =A0 accessible to domU
> =A0 =A0 =A0 =A0 via a couple RPCs. Could also turn out to be as simple= as
> =A0 =A0 =A0 =A0 talking
> =A0 =A0 =A0 =A0 through the primary vif, because the connection betwee= n domU
> =A0 =A0 =A0 =A0 and dom0
> =A0 =A0 =A0 =A0 could remain unaffected.
>
>
>
> PCI passthrough is via config changes and no code changes, if that'= ;s
> the case I am not sure how it would solve multiple domU accesses.

My understanding after catching up a little on the past of this= thread
was that you want the network controller in some maintenance mode. Is
this correct?
=A0
All I need to=A0 is access= NIC registers via domU's(network controller will still be working norm= ally). Using PCI passthrough solves the problem for a domU, however, it doesn't solve when multiple domU's wanting to read NIC registers(ex. statistics).=A0 =

To get it there you will need to temporarily remove it from the virtual
network topology.

The PCI passthrough mode might solve your second problem, which is how
the domU is supposed to access the device once it's been pulled off the=
data path.=A0

> For the second paragraph, do you have recommended readings? frankly I<= br> > don't completely understand the solution any pointers appreciated.=

> In addition, registers in NIC are memory mapped(ioremap function is > used, and in ioctls memcpy_toio and memcpy_fromio is used to
> write/read registers) and wanted =A0to know if its possible to map
> memory from dom0 into domU's?

Yes. This is the third problem, which is how to program a device. I&#= 39;d
recommend "Linux Device Drivers" on that subject. There are also = free
books like http://tldp.org/LDP/tlk/tlk-title.html. Examples likely
outdate, but the concepts remain.

If the device is memory mapped, it doesn't mean it's in memory. It = means
it's in the machine memory address space. The difference should become<= br> clear once you're done with understanding your driver.

Is this the reason why you are so concerned about the memory sharing
mechanism?

No not really. I wanted to use shared memo= ry between dom's as a solution for multiple domU access(since pci passt= hrough doesn't solve it).

The clarification I wanted here(NIC r= egisters are memory mapped), can I take "machine memory address space(= which is in dom0)" and remap it to domU's such that I can get mult= iple domU access.

To summarize,

1. PCI passthrough mechanism works for single domU=
2. Shared memory rings between dom's as a solution to have multiple= domU access, not a workable solution though
3. Take mapped machine addr= ess in dom0 and remap it into domU's(just another thought, not sure it = works) and wanted clarification here.

Thanks
=A0
The good news is now you won't need to bother, that's onl= y
for memory. :)

Daniel



--000e0cd1a0aaf7dbe0048037c8ea-- --===============1045754906== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============1045754906==--