[GIT PULL] (xen) stable/for-jens-3.2 - blkback and blkfront patches for 3.2

[GIT PULL] (xen) stable/for-jens-3.2 - blkback and blkfront patches for 3.2

[Konrad Rzeszutek Wilk]

Hey Jens,

Please pull the following git tree in your 3.2 drivers tree:

 git pull git://oss.oracle.com/git/kwilk/xen.git stable/for-jens-3.2

which has an exciting new feature - the discard ('feature-discard')
aka TRIM/UNMAP. Now they can be passed from the guest to the backend.

There is also a tiny cleanup patch.

 drivers/block/xen-blkback/blkback.c |   86 +++++++++++++++++++++++-----
 drivers/block/xen-blkback/common.h  |   95 ++++++++++++++++++++++++------
 drivers/block/xen-blkback/xenbus.c  |   62 +++++++++++++++++++-
 drivers/block/xen-blkfront.c        |  109 +++++++++++++++++++++++++++-------
 include/xen/interface/io/blkif.h    |   36 ++++++++++++
 5 files changed, 331 insertions(+), 57 deletions(-)

Joe Jin (1):
      xen-blkback: fixed indentation and comments

Li Dongyang (4):
      xen-blkfront: add BLKIF_OP_DISCARD and discard request struct
      xen-blkback: Implement discard requests ('feature-discard')
      xen-blkfront: Handle discard requests.
      xen-blkfront: fix a deadlock while handling discard response

Re: [GIT PULL] (xen) stable/for-jens-3.2 - blkback and blkfront patches for 3.2

[Christoph Hellwig]

On Wed, Sep 21, 2011 at 11:09:55AM -0400, Konrad Rzeszutek Wilk wrote:
> Hey Jens,
> 
> Please pull the following git tree in your 3.2 drivers tree:
> 
>  git pull git://oss.oracle.com/git/kwilk/xen.git stable/for-jens-3.2
> 
> which has an exciting new feature - the discard ('feature-discard')
> aka TRIM/UNMAP. Now they can be passed from the guest to the backend.

What is the design of them?  Do you provide the 'discard zeroes data'
flag (mandatory for any kind of relible operation) and 'discard
granularity / alignment' tunables (very useful)?

Re: [GIT PULL] (xen) stable/for-jens-3.2 - blkback and blkfront patches for 3.2

[Konrad Rzeszutek Wilk]

On Wed, Sep 21, 2011 at 11:17:32AM -0400, Christoph Hellwig wrote:
> On Wed, Sep 21, 2011 at 11:09:55AM -0400, Konrad Rzeszutek Wilk wrote:
> > Hey Jens,
> > 
> > Please pull the following git tree in your 3.2 drivers tree:
> > 
> >  git pull git://oss.oracle.com/git/kwilk/xen.git stable/for-jens-3.2
> > 
> > which has an exciting new feature - the discard ('feature-discard')
> > aka TRIM/UNMAP. Now they can be passed from the guest to the backend.
> 
> What is the design of them?  Do you provide the 'discard zeroes data'
> flag (mandatory for any kind of relible operation) and 'discard

Hmm, not sure what that is. Is that REQ_SECURE, err BLKDEV_DISCARD_SECURE?
If so then not in this patchset - but it does not seem like it would dificult
to add that in? Li, what do you think?

> granularity / alignment' tunables (very useful)?

The discard granularity/alignment are certainly there. But they are just
one way (backend tells the guest what they are). Should they be
more tunable (frontend tells the backend what the values ought to be?)

Li, did I miss anything in my response?

Re: [GIT PULL] (xen) stable/for-jens-3.2 - blkback and blkfront patches for 3.2

[Konrad Rzeszutek Wilk]

On Thu, Sep 22, 2011 at 09:05:17PM +1000, Li Dongyang wrote:
> On Thursday, September 22, 2011, Konrad Rzeszutek Wilk <
> konrad.wilk@oracle.com> wrote:
> > On Wed, Sep 21, 2011 at 11:17:32AM -0400, Christoph Hellwig wrote:
> >> On Wed, Sep 21, 2011 at 11:09:55AM -0400, Konrad Rzeszutek Wilk wrote:
> >> > Hey Jens,
> >> >
> >> > Please pull the following git tree in your 3.2 drivers tree:
> >> >
> >> >  git pull git://oss.oracle.com/git/kwilk/xen.git stable/for-jens-3.2
> >> >
> >> > which has an exciting new feature - the discard ('feature-discard')
> >> > aka TRIM/UNMAP. Now they can be passed from the guest to the backend.
> >>
> >> What is the design of them?  Do you provide the 'discard zeroes data'
> >> flag (mandatory for any kind of relible operation) and 'discard
> >
> > Hmm, not sure what that is. Is that REQ_SECURE, err BLKDEV_DISCARD_SECURE?
> > If so then not in this patchset - but it does not seem like it would
> dificult
> > to add that in? Li, what do you think?
> We deal the discard with 2 ways in the backend depend on what the guest is
> using:
> punch the hole on the image, to save some disk usage,
> or redirect the discard request to phy device.
> for the hole punching case, we are already DISCARD_SECURE,
> for the phy dev case, we could add the flag and pass it from blkfront to
> blkback, and yes
> That's not difficult, could be added later.

Li, could you implement it please  - on top of the 'stable/for-jens-3.2' patch queue ?
If it would not be too much trouble of course.

> >
> >> granularity / alignment' tunables (very useful)?
> >
> > The discard granularity/alignment are certainly there. But they are just
> > one way (backend tells the guest what they are). Should they be
> > more tunable (frontend tells the backend what the values ought to be?)
> >
> Yes, plz tell more about tunable granularity and alignment, and we can add
> that later,
> Thanks a lot
> > Li, did I miss anything in my response?
> >

Re: [GIT PULL] (xen) stable/for-jens-3.2 - blkback and blkfront patches for 3.2

[Li Dongyang]

seems my mail did not reach the mailing list?

On Fri, Sep 23, 2011 at 11:38 PM, Li Dongyang <lidongyang@novell.com> wrote:
> On Friday, September 23, 2011, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
>> On Thu, Sep 22, 2011 at 09:05:17PM +1000, Li Dongyang wrote:
>>> On Thursday, September 22, 2011, Konrad Rzeszutek Wilk <
>>> konrad.wilk@oracle.com> wrote:
>>> > On Wed, Sep 21, 2011 at 11:17:32AM -0400, Christoph Hellwig wrote:
>>> >> On Wed, Sep 21, 2011 at 11:09:55AM -0400, Konrad Rzeszutek Wilk wrote:
>>> >> > Hey Jens,
>>> >> >
>>> >> > Please pull the following git tree in your 3.2 drivers tree:
>>> >> >
>>> >> >  git pull git://oss.oracle.com/git/kwilk/xen.git stable/for-jens-3.2
>>> >> >
>>> >> > which has an exciting new feature - the discard ('feature-discard')
>>> >> > aka TRIM/UNMAP. Now they can be passed from the guest to the
>>> >> > backend.
>>> >>
>>> >> What is the design of them?  Do you provide the 'discard zeroes data'
>>> >> flag (mandatory for any kind of relible operation) and 'discard
>>> >
>>> > Hmm, not sure what that is. Is that REQ_SECURE, err
>>> > BLKDEV_DISCARD_SECURE?
>>> > If so then not in this patchset - but it does not seem like it would
>>> dificult
>>> > to add that in? Li, what do you think?
>>> We deal the discard with 2 ways in the backend depend on what the guest
>>> is
>>> using:
>>> punch the hole on the image, to save some disk usage,
>>> or redirect the discard request to phy device.
>>> for the hole punching case, we are already DISCARD_SECURE,
>>> for the phy dev case, we could add the flag and pass it from blkfront to
>>> blkback, and yes
>>> That's not difficult, could be added later.
>>
>> Li, could you implement it please  - on top of the 'stable/for-jens-3.2'
>> patch queue ?
>> If it would not be too much trouble of course.
> Yeah I'd like to, but am on a vacation to Australia, nothing with me except
> an iPad,
> and I'll be back on 7th next month, so I gonna sent a patch after that,
> Thanks.
>>
>>> >
>>> >> granularity / alignment' tunables (very useful)?
>>> >
>>> > The discard granularity/alignment are certainly there. But they are
>>> > just
>>> > one way (backend tells the guest what they are). Should they be
>>> > more tunable (frontend tells the backend what the values ought to be?)
>>> >
>>> Yes, plz tell more about tunable granularity and alignment, and we can
>>> add
>>> that later,
>>> Thanks a lot
>>> > Li, did I miss anything in my response?
>>> >
>>

[GIT PULL] (xen) stable/for-jens-3.2 - blkback and blkfront patches for 3.2

[Konrad Rzeszutek Wilk]

Hey Jens,

Please pull the following git tree in your 3.2 drivers tree:

 git pull git://oss.oracle.com/git/kwilk/xen.git stable/for-jens-3.2

which has a couple of bug fixes and two new features for Xen blkfront
and blkback.

The bug-fixes are the normal variety - on error path we did not the proper
thing (leak memory), on entry to some functions we did not check whether we
should just ignore the request (b/c we might not support it), and we also
forgot to increment the write sector count.

There are also two new features:
 - discard support, aka trim/unmap. We export from the backend the
   discard alignment, granularity and do normal REQ_DISCARD. We do have
   patches for the REQ_SECURE type - but not the hardware to test it so
   holding on the secure discard until I get my hands on it.

 - barrier (pre 2.6.38 kernel) support. The backend supports the FLUSH
   type operation but to support older kernels that did barriers we have
   implemented a drain technique that counts all of the bio's in flight and
   waits until they are completed and then sends a FLUSH.

In regards to testing, I've been testing this on Intel SSD, Western Digital
SSD. But sadly all of them export the block size as 512 - is there a good
SSD or disk where it would export it as something more than 512?

And here is the credit list:

Jan Beulich (1):
      xen-blkback: use kzalloc() in favor of kmalloc()+memset()

Joe Jin (1):
      xen-blkback: fixed indentation and comments

Konrad Rzeszutek Wilk (4):
      xen-blkfront: If no barrier or flush is supported, use invalid operation.
      xen/blkback: Support 'feature-barrier' aka old-style BARRIER requests.
      xen/blkback: Report VBD_WSECT (wr_sect) properly.
      xen/blkback: Fix the inhibition to map pages when discarding sector ranges.

Laszlo Ersek (1):
      xen-blkfront: plug device number leak in xlblk_init() error path

Li Dongyang (4):
      xen-blkfront: add BLKIF_OP_DISCARD and discard request struct
      xen-blkback: Implement discard requests ('feature-discard')
      xen-blkfront: Handle discard requests.
      xen-blkfront: fix a deadlock while handling discard response

The diffstat:

 drivers/block/xen-blkback/blkback.c |  130 +++++++++++++++++++++++++++++------
 drivers/block/xen-blkback/common.h  |  100 ++++++++++++++++++++++-----
 drivers/block/xen-blkback/xenbus.c  |   80 +++++++++++++++++++++-
 drivers/block/xen-blkfront.c        |  123 ++++++++++++++++++++++++++-------
 include/xen/interface/io/blkif.h    |   36 ++++++++++
 5 files changed, 403 insertions(+), 66 deletions(-)

Re: [GIT PULL] (xen) stable/for-jens-3.2 - blkback and blkfront patches for 3.2

[Konrad Rzeszutek Wilk]

>       xen/blkback: Fix the inhibition to map pages when discarding sector ranges.

And I accidently messed that up. I put in a patch that had == instead of !=.

So on top of these patches I've also stuck this in the git tree:

git 53289f32632cca41c81b71b928f3d126c5564342
Author: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Date:   Fri Oct 14 12:13:05 2011 -0400

    xen/blkback: Check for proper operation.
    
    The patch titled: "xen/blkback: Fix the inhibition to map pages
    when discarding sector ranges." had the right idea except that
    it used the wrong comparison operator. It had == instead of !=.
    
    This fixes the bug where all operations would have been stalled.
    
    Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>



Please make sure you have that when you pull.

Thanks!