Re: [Hackathon minutes] PV block improvements

["Roger Pau Monné" <roger.pau@citrix.com>]

Hello,

While working on further block improvements I've found an issue with
persistent grants in blkfront.

Persistent grants basically allocate grants and then they are never
released, so both blkfront and blkback keep using the same memory pages
for all the transactions.

This is not a problem in blkback, because we can dynamically choose how
many grants we want to map. On the other hand, blkfront cannot remove
the access to those grants at any point, because blkfront doesn't know
if blkback has this grants mapped persistently or not.

So if for example we start expanding the number of segments in indirect
requests, to a value like 512 segments per requests, blkfront will
probably try to persistently map 512*32+512 = 16896 grants per device,
that's much more grants that the current default, which is 32*256 = 8192
(if using grant tables v2). This can cause serious problems to other
interfaces inside the DomU, since blkfront basically starts hoarding all
possible grants, leaving other interfaces completely locked.

I've been thinking about different ways to solve this, but so far I
haven't been able to found a nice solution:

1. Limit the number of persistent grants a blkfront instance can use,
let's say that only the first X used grants will be persistently mapped
by both blkfront and blkback, and if more grants are needed the previous
map/unmap will be used.

2. Switch to grant copy in blkback, and get rid of persistent grants (I
have not benchmarked this solution, but I'm quite sure it will involve a
performance regression, specially when scaling to a high number of domains).

3. Increase the size of the grant_table or the size of a single grant
(from 4k to 2M) (this is from Stefano Stabellini).

4. Introduce a new request type that we can use to request blkback to
unmap certain grefs so we can free them in blkfront.

So far none of them looks like a suitable solution.