Re: [Xen-devel] [PATCH] Persistent grant maps for xen blk drivers

[David Vrabel <david.vrabel@citrix.com>]

On 20/09/12 12:48, Jan Beulich wrote:
>>>> On 20.09.12 at 13:30, Oliver Chick <oliver.chick@citrix.com> wrote:
>> The memory overhead, and fallback mode points are related:
>> -Firstly, it turns out that the overhead is actually 2.75MB, not 11MB
>> per device. I made a mistake (pointed out by Jan) as the maximum number
>> of requests that can fit into a single-page ring is 64, not 256.
>> -Clearly, this still scales linearly. So the problem of memory footprint
>> will occur with more VMs, or block devices.
>> -Whilst 2.75MB per device is probably acceptable (?), if we start using
>> multipage rings, then we might not want to have
>> BLKIF_MAX_PERS_REQUESTS_PER_DEVICE==__RING_SIZE, as this will cause the
>> memory overhead to increase. This is why I have implemented the
>> 'fallback' mode. With a multipage ring, it seems reasonable to want the
>> first $x$ grefs seen by blkback to be treated as persistent, and any
>> later ones to be non-persistent. Does that seem sensible?
> 
>>From a resource usage pov, perhaps. But this will get the guest
> entirely unpredictable performance. Plus I don't think 11Mb of
> _virtual_ space is unacceptable overhead in a 64-bit kernel. If
> you really want/need this in a 32-bit one, then perhaps some
> other alternatives would be needed (and persistent grants may
> not be the right approach there in the first place).

It's not just virtual space.  blkback in pvops kernels allocates its
pages from the balloon and if there aren't enough ballooned out pages it
has to allocate real pages (releasing the MFN back to Xen).

Classic kernels didn't need to do this as there was an API for allocate
new "empty" struct page's that get mapped into kernel space.

David