[RFC] mtd: ubi: skip programming unused bits in ubi headers

[RFC] mtd: ubi: skip programming unused bits in ubi headers

[Cheng Ming Lin]

From: Cheng Ming Lin <chengminglin@mxic.com.tw>

This patch prevents unnecessary programming of bits in ec_hdr and
vid_hdr that are not used or read during normal UBI operation. These
unused bits are typically already set to 1 in erased flash and do not
need to be explicitly programmed to 0 if they are not used.

Programming such unused areas offers no functional benefit and may
result in unnecessary flash wear, reducing the overall lifetime of the
device. By skipping these writes, we preserve the flash state as much
as possible and minimize wear caused by redundant operations.

This change ensures that only necessary fields are written when
preparing UBI headers, improving flash efficiency without affecting
functionality.

Signed-off-by: Cheng Ming Lin <chengminglin@mxic.com.tw>
---
 drivers/mtd/ubi/io.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/mtd/ubi/io.c b/drivers/mtd/ubi/io.c
index a4999bce435f..ccab4691b338 100644
--- a/drivers/mtd/ubi/io.c
+++ b/drivers/mtd/ubi/io.c
@@ -868,6 +868,7 @@ int ubi_io_write_ec_hdr(struct ubi_device *ubi, int pnum,
 		return -EROFS;
 	}
 
+	memset((char *)ec_hdr + UBI_EC_HDR_SIZE, 0xFF, ubi->ec_hdr_alsize - UBI_EC_HDR_SIZE);
 	err = ubi_io_write(ubi, ec_hdr, pnum, 0, ubi->ec_hdr_alsize);
 	return err;
 }
@@ -1150,6 +1151,7 @@ int ubi_io_write_vid_hdr(struct ubi_device *ubi, int pnum,
 		return -EROFS;
 	}
 
+	memset((char *)p + ubi->vid_hdr_shift, 0xFF, ubi->vid_hdr_alsize - UBI_VID_HDR_SIZE);
 	err = ubi_io_write(ubi, p, pnum, ubi->vid_hdr_aloffset,
 			   ubi->vid_hdr_alsize);
 	return err;
-- 
2.25.1

Re: [RFC] mtd: ubi: skip programming unused bits in ubi headers

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Cheng Ming Lin" <linchengming884@gmail.com>
> An: "richard" <richard@nod.at>, "chengzhihao1" <chengzhihao1@huawei.com>, "Miquel Raynal" <miquel.raynal@bootlin.com>,
> "Vignesh Raghavendra" <vigneshr@ti.com>, "linux-mtd" <linux-mtd@lists.infradead.org>, "linux-kernel"
> <linux-kernel@vger.kernel.org>
> CC: "Alvin Zhou" <alvinzhou@mxic.com.tw>, leoyu@mxic.com.tw, "Cheng Ming Lin" <chengminglin@mxic.com.tw>
> Gesendet: Dienstag, 12. August 2025 07:19:49
> Betreff: [RFC] mtd: ubi: skip programming unused bits in ubi headers

> From: Cheng Ming Lin <chengminglin@mxic.com.tw>
> 
> This patch prevents unnecessary programming of bits in ec_hdr and
> vid_hdr that are not used or read during normal UBI operation. These
> unused bits are typically already set to 1 in erased flash and do not
> need to be explicitly programmed to 0 if they are not used.
> 
> Programming such unused areas offers no functional benefit and may
> result in unnecessary flash wear, reducing the overall lifetime of the
> device. By skipping these writes, we preserve the flash state as much
> as possible and minimize wear caused by redundant operations.

We talk about programming a single (sub)page, right?

> This change ensures that only necessary fields are written when
> preparing UBI headers, improving flash efficiency without affecting
> functionality.

I have been told that writing 0xFF bytes to NAND should be avoided.
This is also why UBI initializes them to 0x00.

Do you have data which proves the opposite?

Thanks,
//richard

Re: [RFC] mtd: ubi: skip programming unused bits in ubi headers

[Cheng Ming Lin]

Hi Richard,

Richard Weinberger <richard@nod.at> 於 2025年8月12日 週二 下午2:57寫道：
>
> ----- Ursprüngliche Mail -----
> > Von: "Cheng Ming Lin" <linchengming884@gmail.com>
> > An: "richard" <richard@nod.at>, "chengzhihao1" <chengzhihao1@huawei.com>, "Miquel Raynal" <miquel.raynal@bootlin.com>,
> > "Vignesh Raghavendra" <vigneshr@ti.com>, "linux-mtd" <linux-mtd@lists.infradead.org>, "linux-kernel"
> > <linux-kernel@vger.kernel.org>
> > CC: "Alvin Zhou" <alvinzhou@mxic.com.tw>, leoyu@mxic.com.tw, "Cheng Ming Lin" <chengminglin@mxic.com.tw>
> > Gesendet: Dienstag, 12. August 2025 07:19:49
> > Betreff: [RFC] mtd: ubi: skip programming unused bits in ubi headers
>
> > From: Cheng Ming Lin <chengminglin@mxic.com.tw>
> >
> > This patch prevents unnecessary programming of bits in ec_hdr and
> > vid_hdr that are not used or read during normal UBI operation. These
> > unused bits are typically already set to 1 in erased flash and do not
> > need to be explicitly programmed to 0 if they are not used.
> >
> > Programming such unused areas offers no functional benefit and may
> > result in unnecessary flash wear, reducing the overall lifetime of the
> > device. By skipping these writes, we preserve the flash state as much
> > as possible and minimize wear caused by redundant operations.
>
> We talk about programming a single (sub)page, right?

Since the UBI header is mainly written into a single page, the "unused
areas" here refer to the space within a single (sub)page that is outside
the header content.

>
> > This change ensures that only necessary fields are written when
> > preparing UBI headers, improving flash efficiency without affecting
> > functionality.
>
> I have been told that writing 0xFF bytes to NAND should be avoided.
> This is also why UBI initializes them to 0x00.

Normally, after a NAND flash block is erased, all bits are in the 0xFF
state. Programming 0xFF in this case will not actually change the NAND
cells, as the device can inhibit programming—either by raising the
bitline voltage or using a self-boosted program inhibit mechanism.
Therefore, programming 0xFF is a normal and harmless operation for NAND
flash.

>
> Do you have data which proves the opposite?

I am currently collecting relevant data, and will present it once it is
ready. Thank you.

>
> Thanks,
> //richard

Thanks,
Cheng Ming Lin

Re: [RFC] mtd: ubi: skip programming unused bits in ubi headers

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Cheng Ming Lin" <linchengming884@gmail.com>
>> I have been told that writing 0xFF bytes to NAND should be avoided.
>> This is also why UBI initializes them to 0x00.
> 
> Normally, after a NAND flash block is erased, all bits are in the 0xFF
> state. Programming 0xFF in this case will not actually change the NAND
> cells, as the device can inhibit programming—either by raising the
> bitline voltage or using a self-boosted program inhibit mechanism.
> Therefore, programming 0xFF is a normal and harmless operation for NAND
> flash.

That's the crucial question. Is this true for all NAND chips?
Do avoid larger writes of 0xFF patterns file systems such as UBIFS
also use compression a lot.
At least that's what I have been told when I asked 10+ years ago
how UBI works.

Thanks,
//richard

Re: [RFC] mtd: ubi: skip programming unused bits in ubi headers

[Cheng Ming Lin]

Hi Richard,

Richard Weinberger <richard@nod.at> 於 2025年8月14日 週四 下午7:30寫道：
>
> ----- Ursprüngliche Mail -----
> > Von: "Cheng Ming Lin" <linchengming884@gmail.com>
> >> I have been told that writing 0xFF bytes to NAND should be avoided.
> >> This is also why UBI initializes them to 0x00.
> >
> > Normally, after a NAND flash block is erased, all bits are in the 0xFF
> > state. Programming 0xFF in this case will not actually change the NAND
> > cells, as the device can inhibit programming—either by raising the
> > bitline voltage or using a self-boosted program inhibit mechanism.
> > Therefore, programming 0xFF is a normal and harmless operation for NAND
> > flash.
>
> That's the crucial question. Is this true for all NAND chips?
> Do avoid larger writes of 0xFF patterns file systems such as UBIFS
> also use compression a lot.
> At least that's what I have been told when I asked 10+ years ago
> how UBI works.

You are correct that there is an issue when blank data pages are repeatedly
erased. This can lead to cells entering a too deep VT state, which may
cause program failures in subsequent operations.

To address this, newer NAND generations typically perform a pre-program
step before executing the actual erase. This keeps the VT distribution from
going too deep, thereby avoiding the blank-page degradation problem.
Such pre-PGM techniques have been widely adopted for roughly the past
10–15 years.

Additionally, JESD22-A117E (page 6, section 4.1.2.1) specifies examples of
acceptable data patterns, including solid-programmed, checkerboard/inverse-
checkerboard, and checkerboard-with-fill patterns. According to
qualification knowledge bases, using 0x00 can sometimes trigger special
failure modes, whereas writing 0xFF is more robust.

Additionally, the Kioxia TC58NVG1S3HTA00 datasheet (page 63) also notes
that continuous program/erase cycling with a high percentage of ‘0'
bits in the data pattern can accelerate block endurance degradation.
This further supports avoiding large 0x00 patterns.

Link: https://europe.kioxia.com/content/dam/kioxia/newidr/productinfo/datasheet/201910/DST_TC58NVG1S3HTA00-TDE_EN_31442.pdf

>
> Thanks,
> //richard
>

Thanks,
Cheng-Ming Lin

Re: [RFC] mtd: ubi: skip programming unused bits in ubi headers

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Cheng Ming Lin" <linchengming884@gmail.com>
>> That's the crucial question. Is this true for all NAND chips?
>> Do avoid larger writes of 0xFF patterns file systems such as UBIFS
>> also use compression a lot.
>> At least that's what I have been told when I asked 10+ years ago
>> how UBI works.
> 
> You are correct that there is an issue when blank data pages are repeatedly
> erased. This can lead to cells entering a too deep VT state, which may
> cause program failures in subsequent operations.
> 
> To address this, newer NAND generations typically perform a pre-program
> step before executing the actual erase. This keeps the VT distribution from
> going too deep, thereby avoiding the blank-page degradation problem.
> Such pre-PGM techniques have been widely adopted for roughly the past
> 10–15 years.
> 
> Additionally, JESD22-A117E (page 6, section 4.1.2.1) specifies examples of
> acceptable data patterns, including solid-programmed, checkerboard/inverse-
> checkerboard, and checkerboard-with-fill patterns. According to
> qualification knowledge bases, using 0x00 can sometimes trigger special
> failure modes, whereas writing 0xFF is more robust.
> 
> Additionally, the Kioxia TC58NVG1S3HTA00 datasheet (page 63) also notes
> that continuous program/erase cycling with a high percentage of ‘0'
> bits in the data pattern can accelerate block endurance degradation.
> This further supports avoiding large 0x00 patterns.

Thanks for the detailed answer!
So, yes let's apply this change.

Thanks,
//richard

Re: [RFC] mtd: ubi: skip programming unused bits in ubi headers

[Cheng Ming Lin]

Hi Richard,

Richard Weinberger <richard@nod.at> 於 2025年9月12日 週五 下午3:57寫道：
>
> ----- Ursprüngliche Mail -----
> > Von: "Cheng Ming Lin" <linchengming884@gmail.com>
> >> That's the crucial question. Is this true for all NAND chips?
> >> Do avoid larger writes of 0xFF patterns file systems such as UBIFS
> >> also use compression a lot.
> >> At least that's what I have been told when I asked 10+ years ago
> >> how UBI works.
> >
> > You are correct that there is an issue when blank data pages are repeatedly
> > erased. This can lead to cells entering a too deep VT state, which may
> > cause program failures in subsequent operations.
> >
> > To address this, newer NAND generations typically perform a pre-program
> > step before executing the actual erase. This keeps the VT distribution from
> > going too deep, thereby avoiding the blank-page degradation problem.
> > Such pre-PGM techniques have been widely adopted for roughly the past
> > 10–15 years.
> >
> > Additionally, JESD22-A117E (page 6, section 4.1.2.1) specifies examples of
> > acceptable data patterns, including solid-programmed, checkerboard/inverse-
> > checkerboard, and checkerboard-with-fill patterns. According to
> > qualification knowledge bases, using 0x00 can sometimes trigger special
> > failure modes, whereas writing 0xFF is more robust.
> >
> > Additionally, the Kioxia TC58NVG1S3HTA00 datasheet (page 63) also notes
> > that continuous program/erase cycling with a high percentage of ‘0'
> > bits in the data pattern can accelerate block endurance degradation.
> > This further supports avoiding large 0x00 patterns.
>
> Thanks for the detailed answer!
> So, yes let's apply this change.

Sorry, I need to adjust the code a bit, since I just noticed my
condition on vid_hdr shift was incorrect. I will send a v2 version.

>
> Thanks,
> //richard

Thanks,
Cheng Ming Lin

Re: [RFC] mtd: ubi: skip programming unused bits in ubi headers

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Cheng Ming Lin" <linchengming884@gmail.com>
>> Thanks for the detailed answer!
>> So, yes let's apply this change.
> 
> Sorry, I need to adjust the code a bit, since I just noticed my
> condition on vid_hdr shift was incorrect. I will send a v2 version.

I expected anyway that you'll send a non-RFC version of the patch. :)
Just in case, you ran ubi tests with various different nands?
nandsim can help.
Especially with and without subpage support.

Thanks,
//richard

Re: [RFC] mtd: ubi: skip programming unused bits in ubi headers

[Cheng Ming Lin]

Hi Richard,

Richard Weinberger <richard@nod.at> 於 2025年9月12日 週五 下午4:04寫道：
>
> ----- Ursprüngliche Mail -----
> > Von: "Cheng Ming Lin" <linchengming884@gmail.com>
> >> Thanks for the detailed answer!
> >> So, yes let's apply this change.
> >
> > Sorry, I need to adjust the code a bit, since I just noticed my
> > condition on vid_hdr shift was incorrect. I will send a v2 version.
>
> I expected anyway that you'll send a non-RFC version of the patch. :)
> Just in case, you ran ubi tests with various different nands?
> nandsim can help.
> Especially with and without subpage support.

I have already run ubi-utils such as ubiformat, ubiattach,
ubimkvol, and mounting UBIFS on NAND flash.

I will also run ubi-tests with nandsim on different NANDs before
submitting the formal patch, and cover both cases you mentioned: with
and without subpage support.

>
> Thanks,
> //richard

Thanks,
Cheng Ming Lin