RE: VP problematic for backend drivers on IA64?

RE: VP problematic for backend drivers on IA64?

[Magenheimer, Dan (HP Labs Fort Collins)]

> -----Original Message-----
> From: Muli Ben-Yehuda [mailto:mulix@mulix.org] 
> Sent: Friday, January 20, 2006 3:17 PM
> To: Magenheimer, Dan (HP Labs Fort Collins)
> Cc: xen-devel; okrieg@us.ibm.com; Jon Mason
> Subject: Re: VP problematic for backend drivers on IA64?
> 
> On Fri, Jan 20, 2006 at 09:08:21AM -0800, Magenheimer, Dan 
> (HP Labs Fort Collins) wrote:
> > Hi Muli --
> > 
> > I'm cc'ing the xen-ia64-devel list as many of the
> > Xen/ia64 team don't keep up with xen-devel...
> 
> Actually, you didn't :-)

Oops!  For anyone on xen-ia64-devel wanting to catch up on this thread:

http://lists.xensource.com/archives/html/xen-devel/2006-01/msg00492.html
http://lists.xensource.com/archives/html/xen-devel/2006-01/msg00507.html



> > The backend drivers have a lot of code that assume P2M.
> > Blkback has been "ported" to handle P==M but netback
> > never was.  Neither has been "ported" to VP yet so there
> > is some work to do.  It may turn out to be easy (e.g.
> > #define'ing a few macros to be no-ops).  However, there's
> > likely to be some subtle changes too as there was for P==M.
> 
> Where can I find the diff for blkback to work P==M? is this integrated
> into xen-unstable or is it in the IA64 tree?

It is all checked in to xen-unstable.  (The xen-ia64-unstable tree is
sync'ed roughly weekly with xen-unstable.)

> > But the real problem is not really in the backend drivers,
> > it is in the lower layers of the driver stack that the
> > backend drivers sit on top of.  VP means that the machine
> > addresses are hidden to the domain.  But domain0 (and
> > future driver domains) still need to program DMA-capable
> > devices, both for any domain0 I/O and for I/O on behalf
> > of domU's (via blkfront/blkback).  Thus, domain0 cannot
> > really be fully VP.
> 
> Linux provides the DMA-API abstraction, so that drivers do not need to
> be aware of the deails of translating from a guest-physical address to
> a bus address (akak machine address). Theoretically, a DMA-API
> implementation is the only part of the dom0 Linux kernel that would
> need to know to read the P2M table (P2M) or do nothing (P=M) or call
> into Xen to get the tanslation (VP without IOMMU) or call into Xen to
> establish an IOMMU mapping (VP w/ IOMMU).

Yes, unless there are legacy drivers/devices that circumvent the
DMA interface.  I don't know if this is the case on some/many/all
Linux/ia64 configurations... perhaps someone with more familiarity
with a broad range of Linux/ia64 configurations can comment?  I
would be concerned with, for example, IDE, GART, VGA, console...?

> > I think what we discussed at the summit was a modified form
> > of VP which is somewhere between VP and P2M.  All RAM
> > addressing is VP, but all device addressing needs to be
> > P2M.   It was observed that since an IOMMU intercepts all
> > device addressing (and only device addressing), by ensuring
> > that domain0 (and any driver domain) only has device
> > addressing via a "software IOMMU", the problem should be
> > solved.
> 
> Unless the machine has a real HW IOMMU, the device must see bus
> addresses, which means the driver must pass it bus addresses. The
> "virtual IOMMU" therefore becomes a DMA-API implementation which calls
> into Xen for P->Bus translation.

OK.

> > That just about exhausts my expertise in this area, so
> > others can feel free to jump in (and please correct my
> > mistakes).
> 
> I think it makes sense. Does IA64 already implement VP dom0? are there
> any plans for x86(-64) VP dom0?

No, Xen/ia64 domain0 has always been P==M, though some hypervisor
code written prior to booting on hardware (back when it only ran
on a simulator) under an ifdef may be resurrected that supports
VP dom0.

> Cheers,
> Muli
> -- 
> Muli Ben-Yehuda

Thanks!
Dan

Re: RE: VP problematic for backend drivers on IA64?

[Alex Williamson]

On Sun, 2006-01-22 at 14:45 -0800, Magenheimer, Dan (HP Labs Fort
Collins) wrote:
> 
> Yes, unless there are legacy drivers/devices that circumvent the
> DMA interface.  I don't know if this is the case on some/many/all
> Linux/ia64 configurations... perhaps someone with more familiarity
> with a broad range of Linux/ia64 configurations can comment?  I
> would be concerned with, for example, IDE, GART, VGA, console...?

   VGA and serial consoles don't typically do DMA-like operations AFAIK.
They may live in legacy address spaces, but I think their programming
model is entirely reads and writes.  All IDE chips can operate in
standard PCI mode these days, so they should be covered by the DMA-API.
GARTs are similar to IOMMUs, they'll need to be modified to understand
the translation.

	Alex

-- 
Alex Williamson                             HP Linux & Open Source Lab