[PATCH 0 of 3] [PATCH] Patches to change BLKIF_OP_TRIM to BLKIF_OP_DISCARD. (v4)

[PATCH 0 of 3] [PATCH] Patches to change BLKIF_OP_TRIM to BLKIF_OP_DISCARD. (v4)

[Konrad Rzeszutek Wilk]

The v4 of these patches. Includes the changes mentioned to by Ian in the
last review.

Please apply.

[PATCH 1 of 3] interface: rename of trim to discard in blkif.h

[Konrad Rzeszutek Wilk]

# HG changeset patch
# User Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
# Date 1319662851 14400
# Node ID 92e266ce9c1c1600998aeacee2996b8de2a8743e
# Parent  c681dd5aecf3da3c6fd0e4d8a760a9cd18617033
interface: rename of trim to discard in blkif.h

Just a simple sed s/trim/discard/. We are ignoring the comments
which are incorrect.

The reason for the name change is that TRIM is specific to ATA
while the operation can be done on top of SCSI interfaces too.
Hence the rename to something more generic.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

diff -r c681dd5aecf3 -r 92e266ce9c1c xen/include/public/io/blkif.h
--- a/xen/include/public/io/blkif.h	Tue Oct 25 19:22:09 2011 +0100
+++ b/xen/include/public/io/blkif.h	Wed Oct 26 17:00:51 2011 -0400
@@ -82,25 +82,25 @@
  */
 #define BLKIF_OP_RESERVED_1        4
 /*
- * Recognised only if "feature-trim" is present in backend xenbus info.
- * The "feature-trim" node contains a boolean indicating whether trim
- * requests are likely to succeed or fail. Either way, a trim request
+ * Recognised only if "feature-discard" is present in backend xenbus info.
+ * The "feature-discard" node contains a boolean indicating whether discard
+ * requests are likely to succeed or fail. Either way, a discard request
  * may fail at any time with BLKIF_RSP_EOPNOTSUPP if it is unsupported by
  * the underlying block-device hardware. The boolean simply indicates whether
- * or not it is worthwhile for the frontend to attempt trim requests.
- * If a backend does not recognise BLKIF_OP_TRIM, it should *not*
- * create the "feature-trim" node!
+ * or not it is worthwhile for the frontend to attempt discard requests.
+ * If a backend does not recognise BLKIF_OP_DISCARD, it should *not*
+ * create the "feature-discard" node!
  * 
- * Trim operation is a request for the underlying block device to mark
- * extents to be erased. Trim operations are passed with sector_number as the
- * sector index to begin trim operations at and nr_sectors as the number of
- * sectors to be trimmed. The specified sectors should be trimmed if the
- * underlying block device supports trim operations, or a BLKIF_RSP_EOPNOTSUPP
- * should be returned. More information about trim operations at:
+ * Discard operation is a request for the underlying block device to mark
+ * extents to be erased. Discard operations are passed with sector_number as the
+ * sector index to begin discard operations at and nr_sectors as the number of
+ * sectors to be discarded. The specified sectors should be discarded if the
+ * underlying block device supports discard operations, or a BLKIF_RSP_EOPNOTSUPP
+ * should be returned. More information about discard operations at:
  * http://t13.org/Documents/UploadedDocuments/docs2008/
  *     e07154r6-Data_Set_Management_Proposal_for_ATA-ACS2.doc
  */
-#define BLKIF_OP_TRIM              5
+#define BLKIF_OP_DISCARD           5
 
 /*
  * Maximum scatter/gather segments per request.
@@ -135,17 +135,17 @@ typedef struct blkif_request blkif_reque
 
 /*
  * Cast to this structure when blkif_request.operation == BLKIF_OP_TRIM
- * sizeof(struct blkif_request_trim) <= sizeof(struct blkif_request)
+ * sizeof(struct blkif_request_discard) <= sizeof(struct blkif_request)
  */
-struct blkif_request_trim {
-    uint8_t        operation;    /* BLKIF_OP_TRIM                        */
+struct blkif_request_discard {
+    uint8_t        operation;    /* BLKIF_OP_DISCARD                     */
     uint8_t        reserved;     /*                                      */
     blkif_vdev_t   handle;       /* same as for read/write requests      */
     uint64_t       id;           /* private guest value, echoed in resp  */
     blkif_sector_t sector_number;/* start sector idx on disk             */
-    uint64_t       nr_sectors;   /* number of contiguous sectors to trim */
+    uint64_t       nr_sectors;   /* number of contiguous sectors to discard*/
 };
-typedef struct blkif_request_trim blkif_request_trim_t;
+typedef struct blkif_request_discard blkif_request_discard_t;
 
 struct blkif_response {
     uint64_t        id;              /* copied from request */

[PATCH 2 of 3] interface: Flesh out the BLKIF_OP_DISCARD description

[Konrad Rzeszutek Wilk]

# HG changeset patch
# User Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
# Date 1318457227 14400
# Node ID 14793d6c4adb38cc57a6d55a8907a18d5ca18634
# Parent  92e266ce9c1c1600998aeacee2996b8de2a8743e
interface: Flesh out the BLKIF_OP_DISCARD description.

We flesh out details on what is expected of 'feature-flush' and
what are some of the extra parameters that the frontend can read
from the backend. Those extra parameters are: : discard-aligment,
and discard-granularity.

Acked-by: Jan Beulich <JBeulich@suse.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

diff -r 92e266ce9c1c -r 14793d6c4adb xen/include/public/io/blkif.h
--- a/xen/include/public/io/blkif.h	Wed Oct 26 17:00:51 2011 -0400
+++ b/xen/include/public/io/blkif.h	Wed Oct 12 18:07:07 2011 -0400
@@ -83,22 +83,43 @@
 #define BLKIF_OP_RESERVED_1        4
 /*
  * Recognised only if "feature-discard" is present in backend xenbus info.
- * The "feature-discard" node contains a boolean indicating whether discard
- * requests are likely to succeed or fail. Either way, a discard request
+ * The "feature-discard" node contains a boolean indicating whether trim
+ * (ATA) or unmap (SCSI) - conviently called discard requests are likely
+ * to succeed or fail. Either way, a discard request
  * may fail at any time with BLKIF_RSP_EOPNOTSUPP if it is unsupported by
  * the underlying block-device hardware. The boolean simply indicates whether
  * or not it is worthwhile for the frontend to attempt discard requests.
  * If a backend does not recognise BLKIF_OP_DISCARD, it should *not*
  * create the "feature-discard" node!
- * 
+ *
  * Discard operation is a request for the underlying block device to mark
- * extents to be erased. Discard operations are passed with sector_number as the
+ * extents to be erased. However, discard does not guarantee that the blocks
+ * will be erased from the device - it is just a hint to the device
+ * controller that these blocks are no longer in use. What the device
+ * controller does with that information is left to the controller.
+ * Discard operations are passed with sector_number as the
  * sector index to begin discard operations at and nr_sectors as the number of
  * sectors to be discarded. The specified sectors should be discarded if the
- * underlying block device supports discard operations, or a BLKIF_RSP_EOPNOTSUPP
- * should be returned. More information about discard operations at:
+ * underlying block device supports trim (ATA) or unmap (SCSI) operations,
+ * or a BLKIF_RSP_EOPNOTSUPP  should be returned.
+ * More information about trim/unmap operations at:
  * http://t13.org/Documents/UploadedDocuments/docs2008/
  *     e07154r6-Data_Set_Management_Proposal_for_ATA-ACS2.doc
+ * http://www.seagate.com/staticfiles/support/disc/manuals/
+ *     Interface%20manuals/100293068c.pdf
+ * The backend can optionally provide these extra XenBus attributes to
+ * further optimize the discard functionality:
+ * 'discard-aligment' - Devices that support discard functionality may
+ * internally allocate space in units that are bigger than the exported
+ * logical block size. The discard-alignment parameter indicates how many bytes
+ * the beginning of the partition is offset from the internal allocation unit's
+ * natural alignment. Do not confuse this with natural disk alignment offset.
+ * 'discard-granularity'  - Devices that support discard functionality may
+ * internally allocate space using units that are bigger than the logical block
+ * size. The discard-granularity parameter indicates the size of the internal
+ * allocation unit in bytes if reported by the device. Otherwise the
+ * discard-granularity will be set to match the device's physical block size.
+ * It is the minimum size you can discard.
  */
 #define BLKIF_OP_DISCARD           5

[PATCH 3 of 3] interface: add 'discard-secure' and BLKIF_DISCARD_SECURE

[Konrad Rzeszutek Wilk]

# HG changeset patch
# User Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
# Date 1318457231 14400
# Node ID ef78e170a9008346578458abf541075310cb299b
# Parent  14793d6c4adb38cc57a6d55a8907a18d5ca18634
interface: add 'discard-secure' and BLKIF_DISCARD_SECURE

Alter the 'reserved' uint8_t to be used a 'flag'. We use only for
one flag: BLKIF_DISCARD_SECURE.

That flag can only be set if the backend has set 'discard-secure' to one.
If backend has not set 'discard-secure' to one, that flag will have no
effect.

Acked-by: Jan Beulich <JBeulich@suse.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

diff -r 14793d6c4adb -r ef78e170a900 xen/include/public/io/blkif.h
--- a/xen/include/public/io/blkif.h	Wed Oct 12 18:07:07 2011 -0400
+++ b/xen/include/public/io/blkif.h	Wed Oct 12 18:07:11 2011 -0400
@@ -120,6 +120,9 @@
  * allocation unit in bytes if reported by the device. Otherwise the
  * discard-granularity will be set to match the device's physical block size.
  * It is the minimum size you can discard.
+ * 'discard-secure' - All copies of the discarded sectors (potentially created
+ * by garbage collection) must also be erased.  To use this feature, the flag
+ * BLKIF_DISCARD_SECURE must be set in the blkif_request_discard.
  */
 #define BLKIF_OP_DISCARD           5
 
@@ -160,7 +163,8 @@ typedef struct blkif_request blkif_reque
  */
 struct blkif_request_discard {
     uint8_t        operation;    /* BLKIF_OP_DISCARD                     */
-    uint8_t        reserved;     /*                                      */
+    uint8_t        flag;         /* BLKIF_DISCARD_SECURE or zero         */
+#define BLKIF_DISCARD_SECURE (1<<0)  /* ignored if discard-secure=0      */
     blkif_vdev_t   handle;       /* same as for read/write requests      */
     uint64_t       id;           /* private guest value, echoed in resp  */
     blkif_sector_t sector_number;/* start sector idx on disk             */

[PATCH 0 of 3] Patches to alter BLKIF_OP_TRIM to BLKIF_OP_DISCARD (v4)

[Konrad Rzeszutek Wilk]

This is the v4 patches which is broken in three parts and does:

 1). rename trim->discard
 2). flesh out the description
 3). add the secure-discard option.

[PATCH 3 of 3] interface: add 'discard-secure' and BLKIF_DISCARD_SECURE

[Konrad Rzeszutek Wilk]

# HG changeset patch
# User Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
# Date 1318457231 14400
# Node ID 50850daec7f0486ee7ca69b3d4cb58b4d340a5a7
# Parent  15c2d70dbac3e31c2d74b6700e1bb5f8a7d8256e
interface: add 'discard-secure' and BLKIF_DISCARD_SECURE

Alter the 'reserved' uint8_t to be used a 'flag'. We use only for
one flag: BLKIF_DISCARD_SECURE.

That flag can only be set if the backend has set 'discard-secure' to one.
If backend has not set 'discard-secure' to one, that flag will have no
effect.

Acked-by: Jan Beulich <JBeulich@suse.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

diff -r 15c2d70dbac3 -r 50850daec7f0 xen/include/public/io/blkif.h
--- a/xen/include/public/io/blkif.h	Wed Oct 12 18:07:07 2011 -0400
+++ b/xen/include/public/io/blkif.h	Wed Oct 12 18:07:11 2011 -0400
@@ -107,7 +107,7 @@
  *     e07154r6-Data_Set_Management_Proposal_for_ATA-ACS2.doc
  * http://www.seagate.com/staticfiles/support/disc/manuals/
  *     Interface%20manuals/100293068c.pdf
- * The backend can optionally provide two extra XenBus attributes to
+ * The backend can optionally provide three extra XenBus attributes to
  * further optimize the discard functionality:
  * 'discard-aligment' - Devices that support discard functionality may
  * internally allocate space in units that are bigger than the exported
@@ -119,6 +119,9 @@
  * size. The discard-granularity parameter indicates the size of the internal
  * allocation unit in bytes if reported by the device. Otherwise the
  * discard-granularity will be set to match the device's physical block size.
+ * 'discard-secure' - All copies of the discarded sectors (potentially created
+ * by garbage collection) must also be erased.  To use this feature, the flag
+ * BLKIF_DISCARD_SECURE must be set in the blkif_request_trim.
  */
 #define BLKIF_OP_DISCARD           5
 
@@ -159,7 +162,8 @@ typedef struct blkif_request blkif_reque
  */
 struct blkif_request_discard {
     uint8_t        operation;    /* BLKIF_OP_DISCARD                     */
-    uint8_t        reserved;     /*                                      */
+    uint8_t        flag;         /* BLKIF_DISCARD_SECURE or zero         */
+#define BLKIF_DISCARD_SECURE (1<<0)  /* ignored if discard-secure=0      */
     blkif_vdev_t   handle;       /* same as for read/write requests      */
     uint64_t       id;           /* private guest value, echoed in resp  */
     blkif_sector_t sector_number;/* start sector idx on disk             */

Re: [PATCH 3 of 3] interface: add 'discard-secure' and BLKIF_DISCARD_SECURE

[Ian Campbell]

On Wed, 2011-10-12 at 23:12 +0100, Konrad Rzeszutek Wilk wrote:
> # HG changeset patch
> # User Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> # Date 1318457231 14400
> # Node ID 50850daec7f0486ee7ca69b3d4cb58b4d340a5a7
> # Parent  15c2d70dbac3e31c2d74b6700e1bb5f8a7d8256e
> interface: add 'discard-secure' and BLKIF_DISCARD_SECURE
> 
> Alter the 'reserved' uint8_t to be used a 'flag'. We use only for
> one flag: BLKIF_DISCARD_SECURE.
> 
> That flag can only be set if the backend has set 'discard-secure' to one.
> If backend has not set 'discard-secure' to one, that flag will have no
> effect.
> 
> Acked-by: Jan Beulich <JBeulich@suse.com>
> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> 
> diff -r 15c2d70dbac3 -r 50850daec7f0 xen/include/public/io/blkif.h
> --- a/xen/include/public/io/blkif.h	Wed Oct 12 18:07:07 2011 -0400
> +++ b/xen/include/public/io/blkif.h	Wed Oct 12 18:07:11 2011 -0400
> @@ -107,7 +107,7 @@
>   *     e07154r6-Data_Set_Management_Proposal_for_ATA-ACS2.doc
>   * http://www.seagate.com/staticfiles/support/disc/manuals/
>   *     Interface%20manuals/100293068c.pdf
> - * The backend can optionally provide two extra XenBus attributes to
> + * The backend can optionally provide three extra XenBus attributes to
                                         these

avoids patching (or more likely forgetting to patch) this line every
time we add an attribute.

>   * further optimize the discard functionality:
>   * 'discard-aligment' - Devices that support discard functionality may
>   * internally allocate space in units that are bigger than the exported
> @@ -119,6 +119,9 @@
>   * size. The discard-granularity parameter indicates the size of the internal
>   * allocation unit in bytes if reported by the device. Otherwise the
>   * discard-granularity will be set to match the device's physical block size.
> + * 'discard-secure' - All copies of the discarded sectors (potentially created
> + * by garbage collection) must also be erased.  To use this feature, the flag
> + * BLKIF_DISCARD_SECURE must be set in the blkif_request_trim.

Stray "trim" here.

>   */
>  #define BLKIF_OP_DISCARD           5

It just occurred to me that if reusing the reserved field is going to
prove a problem we could have had BLKIF_OP_DISCARD_SECURE. I think we've
got things under control though.

>  
> @@ -159,7 +162,8 @@ typedef struct blkif_request blkif_reque
>   */
>  struct blkif_request_discard {
>      uint8_t        operation;    /* BLKIF_OP_DISCARD                     */
> -    uint8_t        reserved;     /*                                      */
> +    uint8_t        flag;         /* BLKIF_DISCARD_SECURE or zero         */
> +#define BLKIF_DISCARD_SECURE (1<<0)  /* ignored if discard-secure=0      */
>      blkif_vdev_t   handle;       /* same as for read/write requests      */
>      uint64_t       id;           /* private guest value, echoed in resp  */
>      blkif_sector_t sector_number;/* start sector idx on disk             */
> 
> 

Re: [PATCH 3 of 3] interface: add 'discard-secure' and BLKIF_DISCARD_SECURE

[Konrad Rzeszutek Wilk]

On Thu, Oct 13, 2011 at 08:59:28AM +0100, Ian Campbell wrote:
> On Wed, 2011-10-12 at 23:12 +0100, Konrad Rzeszutek Wilk wrote:
> > # HG changeset patch
> > # User Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > # Date 1318457231 14400
> > # Node ID 50850daec7f0486ee7ca69b3d4cb58b4d340a5a7
> > # Parent  15c2d70dbac3e31c2d74b6700e1bb5f8a7d8256e
> > interface: add 'discard-secure' and BLKIF_DISCARD_SECURE
> > 
> > Alter the 'reserved' uint8_t to be used a 'flag'. We use only for
> > one flag: BLKIF_DISCARD_SECURE.
> > 
> > That flag can only be set if the backend has set 'discard-secure' to one.
> > If backend has not set 'discard-secure' to one, that flag will have no
> > effect.
> > 
> > Acked-by: Jan Beulich <JBeulich@suse.com>
> > Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > 
> > diff -r 15c2d70dbac3 -r 50850daec7f0 xen/include/public/io/blkif.h
> > --- a/xen/include/public/io/blkif.h	Wed Oct 12 18:07:07 2011 -0400
> > +++ b/xen/include/public/io/blkif.h	Wed Oct 12 18:07:11 2011 -0400
> > @@ -107,7 +107,7 @@
> >   *     e07154r6-Data_Set_Management_Proposal_for_ATA-ACS2.doc
> >   * http://www.seagate.com/staticfiles/support/disc/manuals/
> >   *     Interface%20manuals/100293068c.pdf
> > - * The backend can optionally provide two extra XenBus attributes to
> > + * The backend can optionally provide three extra XenBus attributes to
>                                          these
> 
> avoids patching (or more likely forgetting to patch) this line every
> time we add an attribute.
> 
> >   * further optimize the discard functionality:
> >   * 'discard-aligment' - Devices that support discard functionality may
> >   * internally allocate space in units that are bigger than the exported
> > @@ -119,6 +119,9 @@
> >   * size. The discard-granularity parameter indicates the size of the internal
> >   * allocation unit in bytes if reported by the device. Otherwise the
> >   * discard-granularity will be set to match the device's physical block size.
> > + * 'discard-secure' - All copies of the discarded sectors (potentially created
> > + * by garbage collection) must also be erased.  To use this feature, the flag
> > + * BLKIF_DISCARD_SECURE must be set in the blkif_request_trim.
> 
> Stray "trim" here.

Duh!
> 
> >   */
> >  #define BLKIF_OP_DISCARD           5
> 
> It just occurred to me that if reusing the reserved field is going to
> prove a problem we could have had BLKIF_OP_DISCARD_SECURE. I think we've
> got things under control though.

Yeah, I think the reserved->flag attribute worked out nicely.
> 
> >  
> > @@ -159,7 +162,8 @@ typedef struct blkif_request blkif_reque
> >   */
> >  struct blkif_request_discard {
> >      uint8_t        operation;    /* BLKIF_OP_DISCARD                     */
> > -    uint8_t        reserved;     /*                                      */
> > +    uint8_t        flag;         /* BLKIF_DISCARD_SECURE or zero         */
> > +#define BLKIF_DISCARD_SECURE (1<<0)  /* ignored if discard-secure=0      */
> >      blkif_vdev_t   handle;       /* same as for read/write requests      */
> >      uint64_t       id;           /* private guest value, echoed in resp  */
> >      blkif_sector_t sector_number;/* start sector idx on disk             */
> > 
> > 
>