[PATCH v3 0/2] ubi: add possibility to skip CRC check for static UBI volumes

[PATCH v3 0/2] ubi: add possibility to skip CRC check for static UBI volumes

[Quentin Schulz]

Some users of static UBI volumes implement their own integrity check, thus
making the volume CRC check done at open time useless. For instance, this
is the case when one use the ubiblock + dm-verity + squashfs combination,
where dm-verity already checks integrity of the block device but this time
at the block granularity instead of verifying the whole volume.

Skipping this test drastically improves the boot-time.

Patches for mtd-utils will be available as well.

Thanks,
Quentin

v3:
  - fix a few typos,

v2:
  - use volume flags instead of arguments on the kernel command line as
  suggested by Richard,

Quentin Schulz (2):
  ubi: provide a way to skip CRC checks
  ubi: expose the volume CRC check skip flag

 drivers/mtd/ubi/cdev.c      |  4 ++++
 drivers/mtd/ubi/kapi.c      |  2 +-
 drivers/mtd/ubi/ubi-media.h |  6 ++++++
 drivers/mtd/ubi/ubi.h       |  4 ++++
 drivers/mtd/ubi/vmt.c       | 12 ++++++++++++
 drivers/mtd/ubi/vtbl.c      |  3 +++
 include/uapi/mtd/ubi-user.h | 16 ++++++++++++++--
 7 files changed, 44 insertions(+), 3 deletions(-)

base-commit: 6dfa2c9bfb5f4b33861de2d8ea9c58086ce17215
-- 
git-series 0.9.1

[PATCH v3 1/2] ubi: provide a way to skip CRC checks

[Quentin Schulz]

Some users of static UBI volumes implement their own integrity check,
thus making the volume CRC check done at open time useless. For
instance, this is the case when one use the ubiblock + dm-verity +
squashfs combination, where dm-verity already checks integrity of the
block device but this time at the block granularity instead of verifying
the whole volume.

Skipping this test drastically improves the boot-time.

Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>
---
 drivers/mtd/ubi/kapi.c      |  2 +-
 drivers/mtd/ubi/ubi-media.h |  6 ++++++
 drivers/mtd/ubi/ubi.h       |  4 ++++
 drivers/mtd/ubi/vmt.c       |  9 +++++++++
 drivers/mtd/ubi/vtbl.c      |  3 +++
 5 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/drivers/mtd/ubi/kapi.c b/drivers/mtd/ubi/kapi.c
index d4b2e87..e9e9ecb 100644
--- a/drivers/mtd/ubi/kapi.c
+++ b/drivers/mtd/ubi/kapi.c
@@ -202,7 +202,7 @@ struct ubi_volume_desc *ubi_open_volume(int ubi_num, int vol_id, int mode)
 	desc->mode = mode;
 
 	mutex_lock(&ubi->ckvol_mutex);
-	if (!vol->checked) {
+	if (!vol->checked && !vol->skip_check) {
 		/* This is the first open - check the volume */
 		err = ubi_check_volume(ubi, vol_id);
 		if (err < 0) {
diff --git a/drivers/mtd/ubi/ubi-media.h b/drivers/mtd/ubi/ubi-media.h
index 195ff8c..b5fe8f8 100644
--- a/drivers/mtd/ubi/ubi-media.h
+++ b/drivers/mtd/ubi/ubi-media.h
@@ -45,6 +45,11 @@ enum {
  * Volume flags used in the volume table record.
  *
  * @UBI_VTBL_AUTORESIZE_FLG: auto-resize this volume
+ * @UBI_VTBL_SKIP_CRC_CHECK_FLG: skip the CRC check done on a static volume at
+ *				 open time. Should only be set on volumes that
+ *				 are used by upper layers doing this kind of
+ *				 check. Main use-case for this flag is
+ *				 boot-time reduction
  *
  * %UBI_VTBL_AUTORESIZE_FLG flag can be set only for one volume in the volume
  * table. UBI automatically re-sizes the volume which has this flag and makes
@@ -76,6 +81,7 @@ enum {
  */
 enum {
 	UBI_VTBL_AUTORESIZE_FLG = 0x01,
+	UBI_VTBL_SKIP_CRC_CHECK_FLG = 0x02,
 };
 
 /*
diff --git a/drivers/mtd/ubi/ubi.h b/drivers/mtd/ubi/ubi.h
index f5ba97c..d47b9e4 100644
--- a/drivers/mtd/ubi/ubi.h
+++ b/drivers/mtd/ubi/ubi.h
@@ -327,6 +327,9 @@ struct ubi_eba_leb_desc {
  *           atomic LEB change
  *
  * @eba_tbl: EBA table of this volume (LEB->PEB mapping)
+ * @skip_check: %1 if CRC check of this static volume should be skipped.
+ *		Directly reflects the presence of the
+ *		%UBI_VTBL_SKIP_CRC_CHECK_FLG flag in the vtbl entry
  * @checked: %1 if this static volume was checked
  * @corrupted: %1 if the volume is corrupted (static volumes only)
  * @upd_marker: %1 if the update marker is set for this volume
@@ -374,6 +377,7 @@ struct ubi_volume {
 	void *upd_buf;
 
 	struct ubi_eba_table *eba_tbl;
+	unsigned int skip_check:1;
 	unsigned int checked:1;
 	unsigned int corrupted:1;
 	unsigned int upd_marker:1;
diff --git a/drivers/mtd/ubi/vmt.c b/drivers/mtd/ubi/vmt.c
index 0be5167..e2606a4 100644
--- a/drivers/mtd/ubi/vmt.c
+++ b/drivers/mtd/ubi/vmt.c
@@ -299,6 +299,10 @@ int ubi_create_volume(struct ubi_device *ubi, struct ubi_mkvol_req *req)
 		vtbl_rec.vol_type = UBI_VID_DYNAMIC;
 	else
 		vtbl_rec.vol_type = UBI_VID_STATIC;
+
+	if (vol->skip_check)
+		vtbl_rec.flags |= UBI_VTBL_SKIP_CRC_CHECK_FLG;
+
 	memcpy(vtbl_rec.name, vol->name, vol->name_len);
 
 	err = ubi_change_vtbl_record(ubi, vol_id, &vtbl_rec);
@@ -733,6 +737,11 @@ static int self_check_volume(struct ubi_device *ubi, int vol_id)
 			ubi_err(ubi, "bad used_bytes");
 			goto fail;
 		}
+
+		if (vol->skip_check) {
+			ubi_err(ubi, "bad skip_check");
+			goto fail;
+		}
 	} else {
 		if (vol->used_ebs < 0 || vol->used_ebs > vol->reserved_pebs) {
 			ubi_err(ubi, "bad used_ebs");
diff --git a/drivers/mtd/ubi/vtbl.c b/drivers/mtd/ubi/vtbl.c
index 94d7a86..2c133cd 100644
--- a/drivers/mtd/ubi/vtbl.c
+++ b/drivers/mtd/ubi/vtbl.c
@@ -560,6 +560,9 @@ static int init_volumes(struct ubi_device *ubi,
 		vol->name[vol->name_len] = '\0';
 		vol->vol_id = i;
 
+		if (vtbl[i].flags & UBI_VTBL_SKIP_CRC_CHECK_FLG)
+			vol->skip_check = 1;
+
 		if (vtbl[i].flags & UBI_VTBL_AUTORESIZE_FLG) {
 			/* Auto re-size flag may be set only for one volume */
 			if (ubi->autoresize_vol_id != -1) {
-- 
git-series 0.9.1

Re: [PATCH v3 1/2] ubi: provide a way to skip CRC checks

[Richard Weinberger]

Am Donnerstag, 28. Juni 2018, 09:40:52 CEST schrieb Quentin Schulz:
> Some users of static UBI volumes implement their own integrity check,
> thus making the volume CRC check done at open time useless. For
> instance, this is the case when one use the ubiblock + dm-verity +
> squashfs combination, where dm-verity already checks integrity of the
> block device but this time at the block granularity instead of verifying
> the whole volume.
> 
> Skipping this test drastically improves the boot-time.
> 
> Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
> Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
> Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>

Reviewed-by: Richard Weinberger <richard@nod.at>

Thanks,
//richard

Re: [PATCH v3 1/2] ubi: provide a way to skip CRC checks

[Artem Bityutskiy]

Hi,

On Thu, 2018-06-28 at 09:40 +0200, Quentin Schulz wrote:
> diff --git a/drivers/mtd/ubi/kapi.c b/drivers/mtd/ubi/kapi.c
> index d4b2e87..e9e9ecb 100644
> --- a/drivers/mtd/ubi/kapi.c
> +++ b/drivers/mtd/ubi/kapi.c
> @@ -202,7 +202,7 @@ struct ubi_volume_desc *ubi_open_volume(int ubi_num, int vol_id, int mode)
>  	desc->mode = mode;
>  
>  	mutex_lock(&ubi->ckvol_mutex);
> -	if (!vol->checked) {
> +	if (!vol->checked && !vol->skip_check) {
>  		/* This is the first open - check the volume */
>  		err = ubi_check_volume(ubi, vol_id);
>  		if (err < 0) {

Did you deliberately did not add a similar check to 'vol_cdev_write()' ?
You want to skip checking on load but do have the checking after volume update ?
Looks a bit inconsistent to me. At the very least deserves a comment in
'vol_cdev_write()' about why 'skip_check' flag is ignored there.

Re: [PATCH v3 1/2] ubi: provide a way to skip CRC checks

[Richard Weinberger]

Artem,

Am Montag, 2. Juli 2018, 09:30:25 CEST schrieb Artem Bityutskiy:
> Hi,
> 
> On Thu, 2018-06-28 at 09:40 +0200, Quentin Schulz wrote:
> > diff --git a/drivers/mtd/ubi/kapi.c b/drivers/mtd/ubi/kapi.c
> > index d4b2e87..e9e9ecb 100644
> > --- a/drivers/mtd/ubi/kapi.c
> > +++ b/drivers/mtd/ubi/kapi.c
> > @@ -202,7 +202,7 @@ struct ubi_volume_desc *ubi_open_volume(int ubi_num, int vol_id, int mode)
> >  	desc->mode = mode;
> >  
> >  	mutex_lock(&ubi->ckvol_mutex);
> > -	if (!vol->checked) {
> > +	if (!vol->checked && !vol->skip_check) {
> >  		/* This is the first open - check the volume */
> >  		err = ubi_check_volume(ubi, vol_id);
> >  		if (err < 0) {
> 
> Did you deliberately did not add a similar check to 'vol_cdev_write()' ?
> You want to skip checking on load but do have the checking after volume update ?
> Looks a bit inconsistent to me. At the very least deserves a comment in
> 'vol_cdev_write()' about why 'skip_check' flag is ignored there.

Well, the idea is skipping the check, not the crc32 on the medium.
That way we can later, if needed, offer a way to drop the flag but
and don't have to rewrite with crc32-enabled.

Thanks,
//richard

Re: [PATCH v3 1/2] ubi: provide a way to skip CRC checks

[Artem Bityutskiy]

On Mon, 2018-07-02 at 09:43 +0200, Richard Weinberger wrote:
> Artem,
> 
> Am Montag, 2. Juli 2018, 09:30:25 CEST schrieb Artem Bityutskiy:
> > Hi,
> > 
> > On Thu, 2018-06-28 at 09:40 +0200, Quentin Schulz wrote:
> > > diff --git a/drivers/mtd/ubi/kapi.c b/drivers/mtd/ubi/kapi.c
> > > index d4b2e87..e9e9ecb 100644
> > > --- a/drivers/mtd/ubi/kapi.c
> > > +++ b/drivers/mtd/ubi/kapi.c
> > > @@ -202,7 +202,7 @@ struct ubi_volume_desc *ubi_open_volume(int
> > > ubi_num, int vol_id, int mode)
> > >  	desc->mode = mode;
> > >  
> > >  	mutex_lock(&ubi->ckvol_mutex);
> > > -	if (!vol->checked) {
> > > +	if (!vol->checked && !vol->skip_check) {
> > >  		/* This is the first open - check the volume */
> > >  		err = ubi_check_volume(ubi, vol_id);
> > >  		if (err < 0) {
> > 
> > Did you deliberately did not add a similar check to
> > 'vol_cdev_write()' ?
> > You want to skip checking on load but do have the checking after
> > volume update ?
> > Looks a bit inconsistent to me. At the very least deserves a
> > comment in
> > 'vol_cdev_write()' about why 'skip_check' flag is ignored there.
> 
> Well, the idea is skipping the check, not the crc32 on the medium.
> That way we can later, if needed, offer a way to drop the flag but
> and don't have to rewrite with crc32-enabled.

I understand that. I am talking about cdev.c line 370. We will check
the CRC after the update regardless of 'skip_check'. So I point out
that this is not very consistent with what we do in
'ubi_open_volume()'.

Is this deliberate or not? If this is deliberate, we should at least
add an explanation comment in 'vol_cdev_write()'.

Re: [PATCH v3 1/2] ubi: provide a way to skip CRC checks

[Boris Brezillon]

Hi Artem,

On Mon, 02 Jul 2018 10:30:25 +0300
Artem Bityutskiy <dedekind1@gmail.com> wrote:

> Hi,
> 
> On Thu, 2018-06-28 at 09:40 +0200, Quentin Schulz wrote:
> > diff --git a/drivers/mtd/ubi/kapi.c b/drivers/mtd/ubi/kapi.c
> > index d4b2e87..e9e9ecb 100644
> > --- a/drivers/mtd/ubi/kapi.c
> > +++ b/drivers/mtd/ubi/kapi.c
> > @@ -202,7 +202,7 @@ struct ubi_volume_desc *ubi_open_volume(int ubi_num, int vol_id, int mode)
> >  	desc->mode = mode;
> >  
> >  	mutex_lock(&ubi->ckvol_mutex);
> > -	if (!vol->checked) {
> > +	if (!vol->checked && !vol->skip_check) {
> >  		/* This is the first open - check the volume */
> >  		err = ubi_check_volume(ubi, vol_id);
> >  		if (err < 0) {  
> 
> Did you deliberately did not add a similar check to 'vol_cdev_write()' ?
> You want to skip checking on load but do have the checking after volume update ?

Yep, it's on purpose, I asked Quentin to keep the test on the update
volume path.

> Looks a bit inconsistent to me. At the very least deserves a comment in
> 'vol_cdev_write()' about why 'skip_check' flag is ignored there.

Well, I thought checking the CRC just after updating the volume made
sense, just to make sure things were written correctly on the medium.
Let's add a comment explaining why we keep the check here, unless you
see a strong reason to get rid of this check in the update path.

Regards,

Boris

Re: [PATCH v3 1/2] ubi: provide a way to skip CRC checks

[Richard Weinberger]

Am Montag, 2. Juli 2018, 09:51:21 CEST schrieb Boris Brezillon:
> Hi Artem,
> 
> On Mon, 02 Jul 2018 10:30:25 +0300
> Artem Bityutskiy <dedekind1@gmail.com> wrote:
> 
> > Hi,
> > 
> > On Thu, 2018-06-28 at 09:40 +0200, Quentin Schulz wrote:
> > > diff --git a/drivers/mtd/ubi/kapi.c b/drivers/mtd/ubi/kapi.c
> > > index d4b2e87..e9e9ecb 100644
> > > --- a/drivers/mtd/ubi/kapi.c
> > > +++ b/drivers/mtd/ubi/kapi.c
> > > @@ -202,7 +202,7 @@ struct ubi_volume_desc *ubi_open_volume(int ubi_num, int vol_id, int mode)
> > >  	desc->mode = mode;
> > >  
> > >  	mutex_lock(&ubi->ckvol_mutex);
> > > -	if (!vol->checked) {
> > > +	if (!vol->checked && !vol->skip_check) {
> > >  		/* This is the first open - check the volume */
> > >  		err = ubi_check_volume(ubi, vol_id);
> > >  		if (err < 0) {  
> > 
> > Did you deliberately did not add a similar check to 'vol_cdev_write()' ?
> > You want to skip checking on load but do have the checking after volume update ?
> 
> Yep, it's on purpose, I asked Quentin to keep the test on the update
> volume path.
> 
> > Looks a bit inconsistent to me. At the very least deserves a comment in
> > 'vol_cdev_write()' about why 'skip_check' flag is ignored there.
> 
> Well, I thought checking the CRC just after updating the volume made
> sense, just to make sure things were written correctly on the medium.
> Let's add a comment explaining why we keep the check here, unless you
> see a strong reason to get rid of this check in the update path.

+1

I also vote for keeping the check.
vol->skip_check is really just "skip the check upon volume open".

Thanks,
//richard

Re: [PATCH v3 1/2] ubi: provide a way to skip CRC checks

[Artem Bityutskiy]

On Mon, 2018-07-02 at 09:51 +0200, Boris Brezillon wrote:
> Well, I thought checking the CRC just after updating the volume made
> sense, just to make sure things were written correctly on the medium.
> Let's add a comment explaining why we keep the check here, unless you
> see a strong reason to get rid of this check in the update path.

I am fine with this, thanks.

[PATCH v3 2/2] ubi: expose the volume CRC check skip flag

[Quentin Schulz]

Now that we have the logic for skipping CRC check for static UBI volumes
in the core, let's expose it to users.

This makes use of a padding byte in the volume description data
structure as a flag. This flag only tell for now whether we should skip
the CRC check of a volume.

This checks the UBI volume for which we are trying to skip the CRC check
is static.

Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>
---
 drivers/mtd/ubi/cdev.c      |  4 ++++
 drivers/mtd/ubi/vmt.c       |  3 +++
 include/uapi/mtd/ubi-user.h | 16 ++++++++++++++--
 3 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c
index 45c3296..3eea1df 100644
--- a/drivers/mtd/ubi/cdev.c
+++ b/drivers/mtd/ubi/cdev.c
@@ -622,6 +622,10 @@ static int verify_mkvol_req(const struct ubi_device *ubi,
 	    req->vol_type != UBI_STATIC_VOLUME)
 		goto bad;
 
+	if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG &&
+	    req->vol_type != UBI_STATIC_VOLUME)
+		goto bad;
+
 	if (req->alignment > ubi->leb_size)
 		goto bad;
 
diff --git a/drivers/mtd/ubi/vmt.c b/drivers/mtd/ubi/vmt.c
index e2606a4..729588b 100644
--- a/drivers/mtd/ubi/vmt.c
+++ b/drivers/mtd/ubi/vmt.c
@@ -174,6 +174,9 @@ int ubi_create_volume(struct ubi_device *ubi, struct ubi_mkvol_req *req)
 	vol->dev.class = &ubi_class;
 	vol->dev.groups = volume_dev_groups;
 
+	if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG)
+		vol->skip_check = 1;
+
 	spin_lock(&ubi->volumes_lock);
 	if (vol_id == UBI_VOL_NUM_AUTO) {
 		/* Find unused volume ID */
diff --git a/include/uapi/mtd/ubi-user.h b/include/uapi/mtd/ubi-user.h
index 5b04a49..6e5ded5 100644
--- a/include/uapi/mtd/ubi-user.h
+++ b/include/uapi/mtd/ubi-user.h
@@ -285,6 +285,18 @@ struct ubi_attach_req {
 	__s8 padding[10];
 };
 
+/*
+ * UBI volume flags.
+ *
+ * @UBI_VOL_SKIP_CRC_CHECK_FLG: skip the CRC check done on a static volume at
+ *				open time. Only valid for static volumes and
+ *				should only be used if the volume user has a
+ *				way to verify data integrity
+ */
+enum {
+	UBI_VOL_SKIP_CRC_CHECK_FLG = 0x1,
+};
+
 /**
  * struct ubi_mkvol_req - volume description data structure used in
  *                        volume creation requests.
@@ -292,7 +304,7 @@ struct ubi_attach_req {
  * @alignment: volume alignment
  * @bytes: volume size in bytes
  * @vol_type: volume type (%UBI_DYNAMIC_VOLUME or %UBI_STATIC_VOLUME)
- * @padding1: reserved for future, not used, has to be zeroed
+ * @flags: volume flags (%UBI_VOL_SKIP_CRC_CHECK_FLG)
  * @name_len: volume name length
  * @padding2: reserved for future, not used, has to be zeroed
  * @name: volume name
@@ -321,7 +333,7 @@ struct ubi_mkvol_req {
 	__s32 alignment;
 	__s64 bytes;
 	__s8 vol_type;
-	__s8 padding1;
+	__u8 flags;
 	__s16 name_len;
 	__s8 padding2[4];
 	char name[UBI_MAX_VOLUME_NAME + 1];
-- 
git-series 0.9.1

Re: [PATCH v3 2/2] ubi: expose the volume CRC check skip flag

[Richard Weinberger]

Quentin,

Am Donnerstag, 28. Juni 2018, 09:40:53 CEST schrieb Quentin Schulz:
> Now that we have the logic for skipping CRC check for static UBI volumes
> in the core, let's expose it to users.
> 
> This makes use of a padding byte in the volume description data
> structure as a flag. This flag only tell for now whether we should skip
> the CRC check of a volume.
> 
> This checks the UBI volume for which we are trying to skip the CRC check
> is static.
> 
> Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
> Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
> Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>
> ---
>  drivers/mtd/ubi/cdev.c      |  4 ++++
>  drivers/mtd/ubi/vmt.c       |  3 +++
>  include/uapi/mtd/ubi-user.h | 16 ++++++++++++++--
>  3 files changed, 21 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c
> index 45c3296..3eea1df 100644
> --- a/drivers/mtd/ubi/cdev.c
> +++ b/drivers/mtd/ubi/cdev.c
> @@ -622,6 +622,10 @@ static int verify_mkvol_req(const struct ubi_device *ubi,
>  	    req->vol_type != UBI_STATIC_VOLUME)
>  		goto bad;
>  
> +	if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG &&
> +	    req->vol_type != UBI_STATIC_VOLUME)
> +		goto bad;

We should also reject unknown flags here.

Thanks,
//richard

Re: [PATCH v3 2/2] ubi: expose the volume CRC check skip flag

[Boris Brezillon]

On Sun, 01 Jul 2018 21:35:57 +0200
Richard Weinberger <richard@nod.at> wrote:

> Quentin,
> 
> Am Donnerstag, 28. Juni 2018, 09:40:53 CEST schrieb Quentin Schulz:
> > Now that we have the logic for skipping CRC check for static UBI volumes
> > in the core, let's expose it to users.
> > 
> > This makes use of a padding byte in the volume description data
> > structure as a flag. This flag only tell for now whether we should skip
> > the CRC check of a volume.
> > 
> > This checks the UBI volume for which we are trying to skip the CRC check
> > is static.
> > 
> > Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
> > Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
> > Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>
> > ---
> >  drivers/mtd/ubi/cdev.c      |  4 ++++
> >  drivers/mtd/ubi/vmt.c       |  3 +++
> >  include/uapi/mtd/ubi-user.h | 16 ++++++++++++++--
> >  3 files changed, 21 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c
> > index 45c3296..3eea1df 100644
> > --- a/drivers/mtd/ubi/cdev.c
> > +++ b/drivers/mtd/ubi/cdev.c
> > @@ -622,6 +622,10 @@ static int verify_mkvol_req(const struct ubi_device *ubi,
> >  	    req->vol_type != UBI_STATIC_VOLUME)
> >  		goto bad;
> >  
> > +	if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG &&

Oops, missed that req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG check was
missing parens (checkpatch --strict should complain about that).

> > +	    req->vol_type != UBI_STATIC_VOLUME)
> > +		goto bad;  
> 
> We should also reject unknown flags here.

I agree. Talking about missing checks, it seems that none of the
padding sections are checked (I hope all mkvol users are zero-ing the
struct as requested in ubi-user.h). And we should probably also
check that vtbl->flags does not contain unknown flags.

Re: [PATCH v3 2/2] ubi: expose the volume CRC check skip flag

[Richard Weinberger]

Am Sonntag, 1. Juli 2018, 22:33:47 CEST schrieb Boris Brezillon:
> On Sun, 01 Jul 2018 21:35:57 +0200
> Richard Weinberger <richard@nod.at> wrote:
> 
> > Quentin,
> > 
> > Am Donnerstag, 28. Juni 2018, 09:40:53 CEST schrieb Quentin Schulz:
> > > Now that we have the logic for skipping CRC check for static UBI volumes
> > > in the core, let's expose it to users.
> > > 
> > > This makes use of a padding byte in the volume description data
> > > structure as a flag. This flag only tell for now whether we should skip
> > > the CRC check of a volume.
> > > 
> > > This checks the UBI volume for which we are trying to skip the CRC check
> > > is static.
> > > 
> > > Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
> > > Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
> > > Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>
> > > ---
> > >  drivers/mtd/ubi/cdev.c      |  4 ++++
> > >  drivers/mtd/ubi/vmt.c       |  3 +++
> > >  include/uapi/mtd/ubi-user.h | 16 ++++++++++++++--
> > >  3 files changed, 21 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c
> > > index 45c3296..3eea1df 100644
> > > --- a/drivers/mtd/ubi/cdev.c
> > > +++ b/drivers/mtd/ubi/cdev.c
> > > @@ -622,6 +622,10 @@ static int verify_mkvol_req(const struct ubi_device *ubi,
> > >  	    req->vol_type != UBI_STATIC_VOLUME)
> > >  		goto bad;
> > >  
> > > +	if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG &&
> 
> Oops, missed that req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG check was
> missing parens (checkpatch --strict should complain about that).

Latest when building my local branch or in linux-next we had noticed.
No need to worry.
 
> > > +	    req->vol_type != UBI_STATIC_VOLUME)
> > > +		goto bad;  
> > 
> > We should also reject unknown flags here.
> 
> I agree. Talking about missing checks, it seems that none of the
> padding sections are checked (I hope all mkvol users are zero-ing the
> struct as requested in ubi-user.h). And we should probably also
> check that vtbl->flags does not contain unknown flags.

At least mtd-utils does zeros the request struct before using it.
So it does the right thing.

Adding checks for non-zero padding and more checking for vtbl->flags
is a good idea.

Thanks,
//richard

Re: [PATCH v3 2/2] ubi: expose the volume CRC check skip flag

[Quentin Schulz]

[-- Attachment #1: Type: text/plain, Size: 2186 bytes --]

Hi Richard, Boris,

On Sun, Jul 01, 2018 at 10:50:41PM +0200, Richard Weinberger wrote:
> Am Sonntag, 1. Juli 2018, 22:33:47 CEST schrieb Boris Brezillon:
> > On Sun, 01 Jul 2018 21:35:57 +0200
> > Richard Weinberger <richard@nod.at> wrote:
> > 
> > > Quentin,
> > > 
> > > Am Donnerstag, 28. Juni 2018, 09:40:53 CEST schrieb Quentin Schulz:
> > > > Now that we have the logic for skipping CRC check for static UBI volumes
> > > > in the core, let's expose it to users.
> > > > 
> > > > This makes use of a padding byte in the volume description data
> > > > structure as a flag. This flag only tell for now whether we should skip
> > > > the CRC check of a volume.
> > > > 
> > > > This checks the UBI volume for which we are trying to skip the CRC check
> > > > is static.
> > > > 
> > > > Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
> > > > Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
> > > > Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>
> > > > ---
> > > >  drivers/mtd/ubi/cdev.c      |  4 ++++
> > > >  drivers/mtd/ubi/vmt.c       |  3 +++
> > > >  include/uapi/mtd/ubi-user.h | 16 ++++++++++++++--
> > > >  3 files changed, 21 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c
> > > > index 45c3296..3eea1df 100644
> > > > --- a/drivers/mtd/ubi/cdev.c
> > > > +++ b/drivers/mtd/ubi/cdev.c
> > > > @@ -622,6 +622,10 @@ static int verify_mkvol_req(const struct ubi_device *ubi,
> > > >  	    req->vol_type != UBI_STATIC_VOLUME)
> > > >  		goto bad;
> > > >  
> > > > +	if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG &&
> > 
> > Oops, missed that req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG check was
> > missing parens (checkpatch --strict should complain about that).
> 
> Latest when building my local branch or in linux-next we had noticed.
> No need to worry.
>  
> > > > +	    req->vol_type != UBI_STATIC_VOLUME)
> > > > +		goto bad;  
> > > 
> > > We should also reject unknown flags here.
> > 
> > I agree.

Should I send another version of my patches for it? Same for
parenthesis around the flags masking above?

Quentin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH v3 2/2] ubi: expose the volume CRC check skip flag

[Boris Brezillon]

On Mon, 2 Jul 2018 08:44:33 +0200
Quentin Schulz <quentin.schulz@bootlin.com> wrote:

> Hi Richard, Boris,
> 
> On Sun, Jul 01, 2018 at 10:50:41PM +0200, Richard Weinberger wrote:
> > Am Sonntag, 1. Juli 2018, 22:33:47 CEST schrieb Boris Brezillon:  
> > > On Sun, 01 Jul 2018 21:35:57 +0200
> > > Richard Weinberger <richard@nod.at> wrote:
> > >   
> > > > Quentin,
> > > > 
> > > > Am Donnerstag, 28. Juni 2018, 09:40:53 CEST schrieb Quentin Schulz:  
> > > > > Now that we have the logic for skipping CRC check for static UBI volumes
> > > > > in the core, let's expose it to users.
> > > > > 
> > > > > This makes use of a padding byte in the volume description data
> > > > > structure as a flag. This flag only tell for now whether we should skip
> > > > > the CRC check of a volume.
> > > > > 
> > > > > This checks the UBI volume for which we are trying to skip the CRC check
> > > > > is static.
> > > > > 
> > > > > Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
> > > > > Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
> > > > > Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>
> > > > > ---
> > > > >  drivers/mtd/ubi/cdev.c      |  4 ++++
> > > > >  drivers/mtd/ubi/vmt.c       |  3 +++
> > > > >  include/uapi/mtd/ubi-user.h | 16 ++++++++++++++--
> > > > >  3 files changed, 21 insertions(+), 2 deletions(-)
> > > > > 
> > > > > diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c
> > > > > index 45c3296..3eea1df 100644
> > > > > --- a/drivers/mtd/ubi/cdev.c
> > > > > +++ b/drivers/mtd/ubi/cdev.c
> > > > > @@ -622,6 +622,10 @@ static int verify_mkvol_req(const struct ubi_device *ubi,
> > > > >  	    req->vol_type != UBI_STATIC_VOLUME)
> > > > >  		goto bad;
> > > > >  
> > > > > +	if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG &&  
> > > 
> > > Oops, missed that req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG check was
> > > missing parens (checkpatch --strict should complain about that).  
> > 
> > Latest when building my local branch or in linux-next we had noticed.
> > No need to worry.
> >    
> > > > > +	    req->vol_type != UBI_STATIC_VOLUME)
> > > > > +		goto bad;    
> > > > 
> > > > We should also reject unknown flags here.  
> > > 
> > > I agree.  
> 
> Should I send another version of my patches for it?

Yes please, respin your series with this additional check. Just define

#define UBI_VOL_VALID_FLGS	(UBI_VOL_SKIP_CRC_CHECK_FLG)

and then, in verify_mkvol_req() add

	if (req->flags & ~UBI_VOL_VALID_FLGS)
		goto bad;

> Same for
> parenthesis around the flags masking above?

No need to fix that one (unless Richard cares), as it seems I had it
wrong.

Re: [PATCH v3 2/2] ubi: expose the volume CRC check skip flag

[Richard Weinberger]

Am Montag, 2. Juli 2018, 08:52:27 CEST schrieb Boris Brezillon:
> On Mon, 2 Jul 2018 08:44:33 +0200
> Quentin Schulz <quentin.schulz@bootlin.com> wrote:
> 
> > Hi Richard, Boris,
> > 
> > On Sun, Jul 01, 2018 at 10:50:41PM +0200, Richard Weinberger wrote:
> > > Am Sonntag, 1. Juli 2018, 22:33:47 CEST schrieb Boris Brezillon:  
> > > > On Sun, 01 Jul 2018 21:35:57 +0200
> > > > Richard Weinberger <richard@nod.at> wrote:
> > > >   
> > > > > Quentin,
> > > > > 
> > > > > Am Donnerstag, 28. Juni 2018, 09:40:53 CEST schrieb Quentin Schulz:  
> > > > > > Now that we have the logic for skipping CRC check for static UBI volumes
> > > > > > in the core, let's expose it to users.
> > > > > > 
> > > > > > This makes use of a padding byte in the volume description data
> > > > > > structure as a flag. This flag only tell for now whether we should skip
> > > > > > the CRC check of a volume.
> > > > > > 
> > > > > > This checks the UBI volume for which we are trying to skip the CRC check
> > > > > > is static.
> > > > > > 
> > > > > > Suggested-by: Boris Brezillon <boris.brezillon@bootlin.com>
> > > > > > Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
> > > > > > Reviewed-by: Boris Brezillon <boris.brezillon@bootlin.com>
> > > > > > ---
> > > > > >  drivers/mtd/ubi/cdev.c      |  4 ++++
> > > > > >  drivers/mtd/ubi/vmt.c       |  3 +++
> > > > > >  include/uapi/mtd/ubi-user.h | 16 ++++++++++++++--
> > > > > >  3 files changed, 21 insertions(+), 2 deletions(-)
> > > > > > 
> > > > > > diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c
> > > > > > index 45c3296..3eea1df 100644
> > > > > > --- a/drivers/mtd/ubi/cdev.c
> > > > > > +++ b/drivers/mtd/ubi/cdev.c
> > > > > > @@ -622,6 +622,10 @@ static int verify_mkvol_req(const struct ubi_device *ubi,
> > > > > >  	    req->vol_type != UBI_STATIC_VOLUME)
> > > > > >  		goto bad;
> > > > > >  
> > > > > > +	if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG &&  
> > > > 
> > > > Oops, missed that req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG check was
> > > > missing parens (checkpatch --strict should complain about that).  
> > > 
> > > Latest when building my local branch or in linux-next we had noticed.
> > > No need to worry.
> > >    
> > > > > > +	    req->vol_type != UBI_STATIC_VOLUME)
> > > > > > +		goto bad;    
> > > > > 
> > > > > We should also reject unknown flags here.  
> > > > 
> > > > I agree.  
> > 
> > Should I send another version of my patches for it?

Yes. Please.

> Yes please, respin your series with this additional check. Just define
> 
> #define UBI_VOL_VALID_FLGS	(UBI_VOL_SKIP_CRC_CHECK_FLG)
> 
> and then, in verify_mkvol_req() add
> 
> 	if (req->flags & ~UBI_VOL_VALID_FLGS)
> 		goto bad;

Yep.
 
> > Same for
> > parenthesis around the flags masking above?
> 
> No need to fix that one (unless Richard cares), as it seems I had it
> wrong.

Nah. If both gcc and checkpatch don't complain, let's keep it as-is.

Thanks,
//richard

Re: [PATCH v3 2/2] ubi: expose the volume CRC check skip flag

[Joe Perches]

On Sun, 2018-07-01 at 22:33 +0200, Boris Brezillon wrote:
> On Sun, 01 Jul 2018 21:35:57 +0200 Richard Weinberger <richard@nod.at> wrote:
> > Am Donnerstag, 28. Juni 2018, 09:40:53 CEST schrieb Quentin Schulz:
> > > Now that we have the logic for skipping CRC check for static UBI volumes
> > > in the core, let's expose it to users.
[]
> > > diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c
[]
> > > @@ -622,6 +622,10 @@ static int verify_mkvol_req(const struct ubi_device *ubi,
> > >  	    req->vol_type != UBI_STATIC_VOLUME)
> > >  		goto bad;
> > >  
> > > +	if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG &&
> 
> Oops, missed that req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG check was
> missing parens (checkpatch --strict should complain about that).

Why should checkpatch complain?
& has higher precedence than &&.

Re: [PATCH v3 2/2] ubi: expose the volume CRC check skip flag

[Richard Weinberger]

Am Sonntag, 1. Juli 2018, 22:54:32 CEST schrieb Joe Perches:
> On Sun, 2018-07-01 at 22:33 +0200, Boris Brezillon wrote:
> > On Sun, 01 Jul 2018 21:35:57 +0200 Richard Weinberger <richard@nod.at> wrote:
> > > Am Donnerstag, 28. Juni 2018, 09:40:53 CEST schrieb Quentin Schulz:
> > > > Now that we have the logic for skipping CRC check for static UBI volumes
> > > > in the core, let's expose it to users.
> []
> > > > diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c
> []
> > > > @@ -622,6 +622,10 @@ static int verify_mkvol_req(const struct ubi_device *ubi,
> > > >  	    req->vol_type != UBI_STATIC_VOLUME)
> > > >  		goto bad;
> > > >  
> > > > +	if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG &&
> > 
> > Oops, missed that req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG check was
> > missing parens (checkpatch --strict should complain about that).
> 
> Why should checkpatch complain?
> & has higher precedence than &&.

The code is more readable.

Thanks,
//richard

Re: [PATCH v3 2/2] ubi: expose the volume CRC check skip flag

[Joe Perches]

On Sun, 2018-07-01 at 23:01 +0200, Richard Weinberger wrote:
> Am Sonntag, 1. Juli 2018, 22:54:32 CEST schrieb Joe Perches:
> > On Sun, 2018-07-01 at 22:33 +0200, Boris Brezillon wrote:
> > > On Sun, 01 Jul 2018 21:35:57 +0200 Richard Weinberger <richard@nod.at> wrote:
> > > > Am Donnerstag, 28. Juni 2018, 09:40:53 CEST schrieb Quentin Schulz:
> > > > > Now that we have the logic for skipping CRC check for static UBI volumes
> > > > > in the core, let's expose it to users.
> > []
> > > > > diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c
> > []
> > > > > @@ -622,6 +622,10 @@ static int verify_mkvol_req(const struct ubi_device *ubi,
> > > > >  	    req->vol_type != UBI_STATIC_VOLUME)
> > > > >  		goto bad;
> > > > >  
> > > > > +	if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG &&
> > > 
> > > Oops, missed that req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG check was
> > > missing parens (checkpatch --strict should complain about that).
> > 
> > Why should checkpatch complain?
> > & has higher precedence than &&.
> 
> The code is more readable.

IYO.  checkpatch doesn't care and I think it's unnecessary.

Just fyi:

checkpatch does suggest parenthesis removal with --strict
when using
	== or !=
with
	&& or ||

e.g.:

$ cat -n foo.c
     1	bool function(void)
     2	{
     3		if (foo & 1 && bar & 2)
     4			return true;
     5		if ((foo & 1) && (bar && 2))
     6			return true;
     7		if (foo == 1 && bar != 2)
     8			return true;
     9		if ((foo == 1) && (bar != 2))
    10			return true;
    11		return false;
    12	}

$ ./scripts/checkpatch.pl -f --strict foo.c
WARNING: Missing or malformed SPDX-License-Identifier tag in line 1
#1: FILE: foo.c:1:
+bool function(void)

CHECK: Unnecessary parentheses around 'foo == 1'
#9: FILE: foo.c:9:
+	if ((foo == 1) && (bar != 2))

CHECK: Unnecessary parentheses around 'bar != 2'
#9: FILE: foo.c:9:
+	if ((foo == 1) && (bar != 2))

total: 0 errors, 1 warnings, 2 checks, 12 lines checked

NOTE: For some of the reported defects, checkpatch may be able to
      mechanically convert to the typical style using --fix or --fix-inplace.

foo.c has style problems, please review.

NOTE: If any of the errors are false positives, please report
      them to the maintainer, see CHECKPATCH in MAINTAINERS.

Re: [PATCH v3 2/2] ubi: expose the volume CRC check skip flag

[Boris Brezillon]

On Sun, 01 Jul 2018 13:54:32 -0700
Joe Perches <joe@perches.com> wrote:

> On Sun, 2018-07-01 at 22:33 +0200, Boris Brezillon wrote:
> > On Sun, 01 Jul 2018 21:35:57 +0200 Richard Weinberger <richard@nod.at> wrote:  
> > > Am Donnerstag, 28. Juni 2018, 09:40:53 CEST schrieb Quentin Schulz:  
> > > > Now that we have the logic for skipping CRC check for static UBI volumes
> > > > in the core, let's expose it to users.  
> []
> > > > diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c  
> []
> > > > @@ -622,6 +622,10 @@ static int verify_mkvol_req(const struct ubi_device *ubi,
> > > >  	    req->vol_type != UBI_STATIC_VOLUME)
> > > >  		goto bad;
> > > >  
> > > > +	if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG &&  
> > 
> > Oops, missed that req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG check was
> > missing parens (checkpatch --strict should complain about that).  
> 
> Why should checkpatch complain?
> & has higher precedence than &&.
> 

Yes, I know, but I remember checkpatch complaining about that in one
of my patch (maybe it was a slightly different case though). Anyway, I
double checked and, as you report, checkpatch does not complain, so
please ignore this comment (sorry for the noise).