From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: [Hackathon minutes] PV block improvements Date: Fri, 21 Jun 2013 16:16:25 -0400 Message-ID: <20130621201625.GA18283@phenom.dumpdata.com> References: <519F81E1.8030203@citrix.com> <51C48923.1040808@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline In-Reply-To: <51C48923.1040808@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Cc: xen-devel List-Id: xen-devel@lists.xenproject.org On Fri, Jun 21, 2013 at 07:10:59PM +0200, Roger Pau Monn=E9 wrote: > Hello, > = > While working on further block improvements I've found an issue with > persistent grants in blkfront. > = > Persistent grants basically allocate grants and then they are never > released, so both blkfront and blkback keep using the same memory pages > for all the transactions. > = > This is not a problem in blkback, because we can dynamically choose how > many grants we want to map. On the other hand, blkfront cannot remove > the access to those grants at any point, because blkfront doesn't know > if blkback has this grants mapped persistently or not. > = > So if for example we start expanding the number of segments in indirect > requests, to a value like 512 segments per requests, blkfront will > probably try to persistently map 512*32+512 =3D 16896 grants per device, > that's much more grants that the current default, which is 32*256 =3D 8192 > (if using grant tables v2). This can cause serious problems to other > interfaces inside the DomU, since blkfront basically starts hoarding all > possible grants, leaving other interfaces completely locked. > = > I've been thinking about different ways to solve this, but so far I > haven't been able to found a nice solution: > = > 1. Limit the number of persistent grants a blkfront instance can use, > let's say that only the first X used grants will be persistently mapped > by both blkfront and blkback, and if more grants are needed the previous > map/unmap will be used. > = > 2. Switch to grant copy in blkback, and get rid of persistent grants (I > have not benchmarked this solution, but I'm quite sure it will involve a > performance regression, specially when scaling to a high number of domain= s). > = > 3. Increase the size of the grant_table or the size of a single grant > (from 4k to 2M) (this is from Stefano Stabellini). > = > 4. Introduce a new request type that we can use to request blkback to > unmap certain grefs so we can free them in blkfront. 5). Lift the limit of grant pages a domain can have. 6). Have an outstanding of grant pools that are mapped to a guest and recycle them? That way both netfront and blkfront could use them as needed? > = > So far none of them looks like a suitable solution. > =