Re: [Xen-devel] [PATCH] Persistent grant maps for xen blk drivers

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Fri, Sep 21, 2012 at 01:23:48PM +0100, David Vrabel wrote:
> On 20/09/12 12:48, Jan Beulich wrote:
> >>>> On 20.09.12 at 13:30, Oliver Chick <oliver.chick@citrix.com> wrote:
> >> The memory overhead, and fallback mode points are related:
> >> -Firstly, it turns out that the overhead is actually 2.75MB, not 11MB
> >> per device. I made a mistake (pointed out by Jan) as the maximum number
> >> of requests that can fit into a single-page ring is 64, not 256.
> >> -Clearly, this still scales linearly. So the problem of memory footprint
> >> will occur with more VMs, or block devices.
> >> -Whilst 2.75MB per device is probably acceptable (?), if we start using
> >> multipage rings, then we might not want to have
> >> BLKIF_MAX_PERS_REQUESTS_PER_DEVICE==__RING_SIZE, as this will cause the
> >> memory overhead to increase. This is why I have implemented the
> >> 'fallback' mode. With a multipage ring, it seems reasonable to want the
> >> first $x$ grefs seen by blkback to be treated as persistent, and any
> >> later ones to be non-persistent. Does that seem sensible?
> > 
> >>From a resource usage pov, perhaps. But this will get the guest
> > entirely unpredictable performance. Plus I don't think 11Mb of
> > _virtual_ space is unacceptable overhead in a 64-bit kernel. If
> > you really want/need this in a 32-bit one, then perhaps some
> > other alternatives would be needed (and persistent grants may
> > not be the right approach there in the first place).
> 
> It's not just virtual space.  blkback in pvops kernels allocates its
> pages from the balloon and if there aren't enough ballooned out pages it
> has to allocate real pages (releasing the MFN back to Xen).
> 
> Classic kernels didn't need to do this as there was an API for allocate
> new "empty" struct page's that get mapped into kernel space.

Can we ressurect/implement that in the new pvops? We could use the memory
hotplug mechanism to allocate "non-existent" memory. 
> 
> David