Re: [RFC 0 PATCH 3/3] PVH dom0: construct_dom0 changes

[Mukesh Rathor <mukesh.rathor@oracle.com>]

On Fri, 04 Oct 2013 07:53:20 +0100
"Jan Beulich" <JBeulich@suse.com> wrote:

> >>> On 03.10.13 at 02:53, Mukesh Rathor <mukesh.rathor@oracle.com>
> >>> wrote:
> > On Fri, 27 Sep 2013 07:54:39 +0100
> > "Jan Beulich" <JBeulich@suse.com> wrote:
> > 
> >> >>> On 27.09.13 at 02:17, Mukesh Rathor <mukesh.rathor@oracle.com>
> >> >>> wrote:
> >> > On Thu, 26 Sep 2013 09:02:41 +0100 "Jan Beulich"
> >> > <JBeulich@suse.com> wrote:
> >> >> >>> On 25.09.13 at 23:03, Mukesh Rathor
> >> >> >>> <mukesh.rathor@oracle.com> wrote:
> >> >> > +/*
> >> >> > + * Set the 1:1 map for all non-RAM regions for dom 0. Thus,
> >> >> > dom0 will have
> >> >> > + * the entire io region mapped in the EPT/NPT.
> >> >> > + *
> >> >> > + * PVH FIXME: The following doesn't map MMIO ranges when they
> >> >> > sit above the
> >> >> > + *            highest E820 covered address.
> >> >> 
> >> >> This absolutely needs fixing before this can go in.
> >> > 
> >> > Any suggestions on how to fix it? Mapping all the way to end
> >> > could result in a huge hap table. 
> >> 
> >> You'll probably need a call down from Dom0 telling you where it
> >> finds/puts MMIO resources. Or perhaps that could be mapped
> >> in on demand from the EPT fault handler (since these regions
> >> shouldn't be subject to DMA, and hence IOMMU faults shouldn't
> >> occur - perhaps that's even a reason to not share page tables
> >> at least in dom0-strict mode)?
> > 
> > Thinking about mapping in on demand from the EPT fault handler, how
> > would I know if the access beyond last e820 entry is genuine and
> > not a faulty pte in a buggy guest? Could I consult the mmconfig
> > table (?) or the ACPI table in xen? Any pointers would be
> > helpful... my knowledge runs out quickly here.
> 
> You'd have to inspect all the BARs of the devices the domain owns.
> Hence the thought of having Dom0 tell you about those resource
> assignments.
> 
> > FWIW, at present pv-ops linux doesn't allow any mmio access beyond
> > the last e820 entry. So, we'd need a fix there too. In my very orig
> > patch, I was updating all IO mappings on demand by putting hook
> > in linux native_pte_update if it was _PAGE_BIT_IOMAP. Another 
> > possibility would be do that for any mappings above the last
> > e820 entry. What do you think?
> 
> Special casing IOMAP page table creation might be an option, but
> has the downside of allowing kernel bugs to propagate into Xen's
> view of the world.
> 
> > For testing purposes, do you have reference for hardware? I don't
> > see any here with such configuration.
> 
> Nothing specific, but I know that SR-IOV virtual functions easily
> cause kernels to run out of MMIO space below 4G (namely when
> the hole is only around 1Gb or even less), and Intel must have
> knowledge of graphics cards having so huge a frame buffer that
> it can only be mapped above 4G.

In that case, I don't see why this is a MUST for the patch. Combined
with the fact that at present pv-ops doesn't even allow for mapping
above the last e820 entry, I think this can be left FIXME/bug-fix for 
near future that can be done relatively quickly by an expert in that
area if learning that takes me a long time.

thanks
Mukesh