Re: [Hackathon minutes] PV block improvements

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Fri, Jun 21, 2013 at 07:10:59PM +0200, Roger Pau Monné wrote:
> Hello,
> 
> While working on further block improvements I've found an issue with
> persistent grants in blkfront.
> 
> Persistent grants basically allocate grants and then they are never
> released, so both blkfront and blkback keep using the same memory pages
> for all the transactions.
> 
> This is not a problem in blkback, because we can dynamically choose how
> many grants we want to map. On the other hand, blkfront cannot remove
> the access to those grants at any point, because blkfront doesn't know
> if blkback has this grants mapped persistently or not.
> 
> So if for example we start expanding the number of segments in indirect
> requests, to a value like 512 segments per requests, blkfront will
> probably try to persistently map 512*32+512 = 16896 grants per device,
> that's much more grants that the current default, which is 32*256 = 8192
> (if using grant tables v2). This can cause serious problems to other
> interfaces inside the DomU, since blkfront basically starts hoarding all
> possible grants, leaving other interfaces completely locked.
> 
> I've been thinking about different ways to solve this, but so far I
> haven't been able to found a nice solution:
> 
> 1. Limit the number of persistent grants a blkfront instance can use,
> let's say that only the first X used grants will be persistently mapped
> by both blkfront and blkback, and if more grants are needed the previous
> map/unmap will be used.
> 
> 2. Switch to grant copy in blkback, and get rid of persistent grants (I
> have not benchmarked this solution, but I'm quite sure it will involve a
> performance regression, specially when scaling to a high number of domains).
> 
> 3. Increase the size of the grant_table or the size of a single grant
> (from 4k to 2M) (this is from Stefano Stabellini).
> 
> 4. Introduce a new request type that we can use to request blkback to
> unmap certain grefs so we can free them in blkfront.


5). Lift the limit of grant pages a domain can have.

6). Have an outstanding of grant pools that are mapped to a guest and
recycle them? That way both netfront and blkfront could use them as needed?

> 
> So far none of them looks like a suitable solution.
>