[PATCH v3 01/10] ubi: Expose mean erase counter in sysfs

[PATCH v3 01/10] ubi: Expose mean erase counter in sysfs

[Rickard Andersson]

Currently, only "max_ec" can be read from sysfs, which provides a
limited view of the flash device’s wear. In certain cases, such as
bugs in the wear-leveling algorithm, specific blocks can be worn down
more than others, resulting in uneven wear distribution. Providing a
mean erase counter value gives a better understanding of the overall
flash wear.
There exists more detailed info in debugfs, but this information is
only available for debug builds.

This patch calculates the mean value only when it is requested. This
calculation typically takes 2.7 ms for an UBI device with 4091 blocks
running on a single core Cortex-A9 at 792 MHz.

Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: Rickard Andersson <rickard.andersson@axis.com>
---
 drivers/mtd/ubi/build.c | 46 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
index f5140a78a4a8..231ca288f397 100644
--- a/drivers/mtd/ubi/build.c
+++ b/drivers/mtd/ubi/build.c
@@ -131,6 +131,8 @@ static struct device_attribute dev_volumes_count =
 	__ATTR(volumes_count, S_IRUGO, dev_attribute_show, NULL);
 static struct device_attribute dev_max_ec =
 	__ATTR(max_ec, S_IRUGO, dev_attribute_show, NULL);
+static struct device_attribute dev_mean_ec =
+	__ATTR(mean_ec, S_IRUGO, dev_attribute_show, NULL);
 static struct device_attribute dev_reserved_for_bad =
 	__ATTR(reserved_for_bad, S_IRUGO, dev_attribute_show, NULL);
 static struct device_attribute dev_bad_peb_count =
@@ -337,6 +339,47 @@ int ubi_major2num(int major)
 	return ubi_num;
 }
 
+/**
+ * ubi_calc_mean_ec - calculate mean erase counter value.
+ * @ubi: UBI device description object
+ * @start_peb: First PEB in the range
+ * @end_peb: End PEB in the half-open range
+ *
+ * Returns the mean value of all non-bad blocks of the device that are
+ * in the range. Range is half-open i.e end_peb is not actually included.
+ */
+static int ubi_calc_mean_ec(struct ubi_device *ubi, int start_peb, int end_peb)
+{
+	struct ubi_wl_entry *wl;
+	int peb;
+	int ec_count = 0;
+	int mean_ec = 0;
+	uint64_t ec_sum = 0;
+
+	for (peb = start_peb; peb < end_peb; peb++) {
+		int err;
+
+		err = ubi_io_is_bad(ubi, peb);
+		if (err)
+			continue;
+
+		spin_lock(&ubi->wl_lock);
+
+		wl = ubi->lookuptbl[peb];
+		if (wl) {
+			ec_sum += wl->ec;
+			ec_count++;
+		}
+
+		spin_unlock(&ubi->wl_lock);
+	}
+
+	if (ec_count > 0)
+		mean_ec = div_u64(ec_sum, ec_count);
+
+	return mean_ec;
+}
+
 /* "Show" method for files in '/<sysfs>/class/ubi/ubiX/' */
 static ssize_t dev_attribute_show(struct device *dev,
 				  struct device_attribute *attr, char *buf)
@@ -366,6 +409,8 @@ static ssize_t dev_attribute_show(struct device *dev,
 		ret = sprintf(buf, "%d\n", ubi->vol_count - UBI_INT_VOL_COUNT);
 	else if (attr == &dev_max_ec)
 		ret = sprintf(buf, "%d\n", ubi->max_ec);
+	else if (attr == &dev_mean_ec)
+		ret = sprintf(buf, "%d\n", ubi_calc_mean_ec(ubi, 0, ubi->peb_count));
 	else if (attr == &dev_reserved_for_bad)
 		ret = sprintf(buf, "%d\n", ubi->beb_rsvd_pebs);
 	else if (attr == &dev_bad_peb_count)
@@ -392,6 +437,7 @@ static struct attribute *ubi_dev_attrs[] = {
 	&dev_total_eraseblocks.attr,
 	&dev_volumes_count.attr,
 	&dev_max_ec.attr,
+	&dev_mean_ec.attr,
 	&dev_reserved_for_bad.attr,
 	&dev_bad_peb_count.attr,
 	&dev_max_vol_count.attr,
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH v3 02/10] ubi: Expose mean erase counter for fastmap in sysfs

[Rickard Andersson]

Since the fastmap area has its own wear levelling it is valuable to
provide a mean value for that area. This value can be used in order
to estimate life expectancy of the flash.

Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: Rickard Andersson <rickard.andersson@axis.com>
---
 drivers/mtd/ubi/build.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
index 231ca288f397..1c531d623a62 100644
--- a/drivers/mtd/ubi/build.c
+++ b/drivers/mtd/ubi/build.c
@@ -133,6 +133,10 @@ static struct device_attribute dev_max_ec =
 	__ATTR(max_ec, S_IRUGO, dev_attribute_show, NULL);
 static struct device_attribute dev_mean_ec =
 	__ATTR(mean_ec, S_IRUGO, dev_attribute_show, NULL);
+#ifdef CONFIG_MTD_UBI_FASTMAP
+static struct device_attribute dev_mean_ec_fastmap =
+	__ATTR(mean_ec_fastmap, S_IRUGO, dev_attribute_show, NULL);
+#endif
 static struct device_attribute dev_reserved_for_bad =
 	__ATTR(reserved_for_bad, S_IRUGO, dev_attribute_show, NULL);
 static struct device_attribute dev_bad_peb_count =
@@ -411,6 +415,10 @@ static ssize_t dev_attribute_show(struct device *dev,
 		ret = sprintf(buf, "%d\n", ubi->max_ec);
 	else if (attr == &dev_mean_ec)
 		ret = sprintf(buf, "%d\n", ubi_calc_mean_ec(ubi, 0, ubi->peb_count));
+#ifdef CONFIG_MTD_UBI_FASTMAP
+	else if (attr == &dev_mean_ec_fastmap)
+		ret = sprintf(buf, "%d\n", ubi_calc_mean_ec(ubi, 0, UBI_FM_MAX_START));
+#endif
 	else if (attr == &dev_reserved_for_bad)
 		ret = sprintf(buf, "%d\n", ubi->beb_rsvd_pebs);
 	else if (attr == &dev_bad_peb_count)
@@ -438,6 +446,9 @@ static struct attribute *ubi_dev_attrs[] = {
 	&dev_volumes_count.attr,
 	&dev_max_ec.attr,
 	&dev_mean_ec.attr,
+#ifdef CONFIG_MTD_UBI_FASTMAP
+	&dev_mean_ec_fastmap.attr,
+#endif
 	&dev_reserved_for_bad.attr,
 	&dev_bad_peb_count.attr,
 	&dev_max_vol_count.attr,
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v3 02/10] ubi: Expose mean erase counter for fastmap in sysfs

[Richard Weinberger]

Rickard,

----- Ursprüngliche Mail -----
> Von: "Rickard X Andersson" <rickard.andersson@axis.com>
> An: "richard" <richard@nod.at>, "chengzhihao1" <chengzhihao1@huawei.com>, "linux-mtd" <linux-mtd@lists.infradead.org>,
> "rickard314 andersson" <rickard314.andersson@gmail.com>
> CC: "Rickard X Andersson" <rickard.andersson@axis.com>, "kernel" <kernel@axis.com>
> Gesendet: Freitag, 11. Oktober 2024 14:58:56
> Betreff: [PATCH v3 02/10] ubi: Expose mean erase counter for fastmap in sysfs

> Since the fastmap area has its own wear levelling it is valuable to
> provide a mean value for that area. This value can be used in order
> to estimate life expectancy of the flash.
> 
> Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
> Signed-off-by: Rickard Andersson <rickard.andersson@axis.com>
> ---
> drivers/mtd/ubi/build.c | 11 +++++++++++
> 1 file changed, 11 insertions(+)
> 
> diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
> index 231ca288f397..1c531d623a62 100644
> --- a/drivers/mtd/ubi/build.c
> +++ b/drivers/mtd/ubi/build.c
> @@ -133,6 +133,10 @@ static struct device_attribute dev_max_ec =
> 	__ATTR(max_ec, S_IRUGO, dev_attribute_show, NULL);
> static struct device_attribute dev_mean_ec =
> 	__ATTR(mean_ec, S_IRUGO, dev_attribute_show, NULL);
> +#ifdef CONFIG_MTD_UBI_FASTMAP
> +static struct device_attribute dev_mean_ec_fastmap =
> +	__ATTR(mean_ec_fastmap, S_IRUGO, dev_attribute_show, NULL);
> +#endif

Do you really need distinct sysfs attributes for the fastmap and the
non-fastmap case? Userspace does not care whether fastmap is used or
not, all userspace is interested in is the mean erase counter value.

IMHO, the mean_ec attribute should combine the mean EC values from
the fastmap and the data area.

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v3 02/10] ubi: Expose mean erase counter for fastmap in sysfs

[Rickard x Andersson]

On 11/6/24 21:20, Richard Weinberger wrote:

> 
> Do you really need distinct sysfs attributes for the fastmap and the
> non-fastmap case? Userspace does not care whether fastmap is used or
> not, all userspace is interested in is the mean erase counter value.
> 
> IMHO, the mean_ec attribute should combine the mean EC values from
> the fastmap and the data area.
> 
> Thanks,
> //richard

Hi and thanks for answering!

One of my devices have the following status:

/sys/class/ubi/ubi1 # cat max_ec_data
4672
/sys/class/ubi/ubi1 # cat mean_ec_fastmap
8869

If you combine the mean EC values you will not be able to tell that the 
fastmap area is much more worn down. For example the wear of your 
fastmap area could be on the verge of breaking the flash but that will 
not be seen on a mean value that includes both the fastmap and data area.

If fastmap is not enabled on your system then mean_ec_fastmap will not 
be visible in sysfs.

Thanks,
Rickard A.


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v3 02/10] ubi: Expose mean erase counter for fastmap in sysfs

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Rickard x Andersson" <rickaran@axis.com>
> An: "richard" <richard@nod.at>, "Rickard X Andersson" <rickard.andersson@axis.com>
> One of my devices have the following status:
> 
> /sys/class/ubi/ubi1 # cat max_ec_data
> 4672
> /sys/class/ubi/ubi1 # cat mean_ec_fastmap
> 8869
> 
> If you combine the mean EC values you will not be able to tell that the
> fastmap area is much more worn down. For example the wear of your
> fastmap area could be on the verge of breaking the flash but that will
> not be seen on a mean value that includes both the fastmap and data area.
> 
> If fastmap is not enabled on your system then mean_ec_fastmap will not
> be visible in sysfs.

But does the user application care?
Let me ask differently, what is the purpose of exposing these numbers?
If you just want to know how much your flash is aged, combined max/mean
ec counters will do it.

For debugging fastmap specific issues, the detailed info via debugfs will do it.

I'm totally fine with exposing more counters, we just need to be clear
about their meaning and use case. Especially in sysfs we must not make a mess.

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v3 02/10] ubi: Expose mean erase counter for fastmap in sysfs

[Rickard x Andersson]

On 11/7/24 12:31, Richard Weinberger wrote:
> ----- Ursprüngliche Mail -----
>> Von: "Rickard x Andersson" <rickaran@axis.com>
>> An: "richard" <richard@nod.at>, "Rickard X Andersson" <rickard.andersson@axis.com>
>> One of my devices have the following status:
>>
>> /sys/class/ubi/ubi1 # cat max_ec_data
>> 4672
>> /sys/class/ubi/ubi1 # cat mean_ec_fastmap
>> 8869
>>
>> If you combine the mean EC values you will not be able to tell that the
>> fastmap area is much more worn down. For example the wear of your
>> fastmap area could be on the verge of breaking the flash but that will
>> not be seen on a mean value that includes both the fastmap and data area.
>>
>> If fastmap is not enabled on your system then mean_ec_fastmap will not
>> be visible in sysfs.
> 
> But does the user application care?
> Let me ask differently, what is the purpose of exposing these numbers?
> If you just want to know how much your flash is aged, combined max/mean
> ec counters will do it.
> 
> For debugging fastmap specific issues, the detailed info via debugfs will do it.
> 
> I'm totally fine with exposing more counters, we just need to be clear
> about their meaning and use case. Especially in sysfs we must not make a mess.
> 
> Thanks,
> //richard

It is for diagnostics. For example, if you have 100 devices running you 
might want to know if some of them are about to be worn down. If you 
know that you can take action before they actually break down. One 
action could be to for example replace the device in that case.
Since the devices are operate in the "field" it is recommended avoid 
using debugfs due to security concerns.

A device might break down due to heavy wear of just the fastmap area.

Thanks,
Rickard A.


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH v3 03/10] ubi: Expose max erase counter for fastmap in sysfs

[Rickard Andersson]

Since the fastmap area has its own wear levelling it is valuable to
provide a max erase counter value specifically for that area.

Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: Rickard Andersson <rickard.andersson@axis.com>
---
 drivers/mtd/ubi/build.c | 43 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
index 1c531d623a62..a5a2e6e3be38 100644
--- a/drivers/mtd/ubi/build.c
+++ b/drivers/mtd/ubi/build.c
@@ -134,6 +134,8 @@ static struct device_attribute dev_max_ec =
 static struct device_attribute dev_mean_ec =
 	__ATTR(mean_ec, S_IRUGO, dev_attribute_show, NULL);
 #ifdef CONFIG_MTD_UBI_FASTMAP
+static struct device_attribute dev_max_ec_fastmap =
+	__ATTR(max_ec_fastmap, S_IRUGO, dev_attribute_show, NULL);
 static struct device_attribute dev_mean_ec_fastmap =
 	__ATTR(mean_ec_fastmap, S_IRUGO, dev_attribute_show, NULL);
 #endif
@@ -384,6 +386,44 @@ static int ubi_calc_mean_ec(struct ubi_device *ubi, int start_peb, int end_peb)
 	return mean_ec;
 }
 
+/**
+ * ubi_calc_max_ec_fastmap - calculate max erase counter value.
+ * @ubi: UBI device description object
+ * @start_peb: First PEB in the range
+ * @end_peb: End PEB in the half-open range
+ *
+ * Returns the max erase counter value that can be found in the range.
+ * Range is half-open i.e end_peb is not actually included.
+ */
+#ifdef CONFIG_MTD_UBI_FASTMAP
+static int ubi_calc_max_ec(struct ubi_device *ubi, int start_peb, int end_peb)
+{
+	struct ubi_wl_entry *wl;
+	int peb;
+	int max_ec = 0;
+
+	for (peb = start_peb; peb < end_peb; peb++) {
+		int err;
+
+		err = ubi_io_is_bad(ubi, peb);
+		if (err)
+			continue;
+
+		spin_lock(&ubi->wl_lock);
+
+		wl = ubi->lookuptbl[peb];
+		if (wl) {
+			if (max_ec < wl->ec)
+				max_ec = wl->ec;
+		}
+
+		spin_unlock(&ubi->wl_lock);
+	}
+
+	return max_ec;
+}
+#endif
+
 /* "Show" method for files in '/<sysfs>/class/ubi/ubiX/' */
 static ssize_t dev_attribute_show(struct device *dev,
 				  struct device_attribute *attr, char *buf)
@@ -416,6 +456,8 @@ static ssize_t dev_attribute_show(struct device *dev,
 	else if (attr == &dev_mean_ec)
 		ret = sprintf(buf, "%d\n", ubi_calc_mean_ec(ubi, 0, ubi->peb_count));
 #ifdef CONFIG_MTD_UBI_FASTMAP
+	else if (attr == &dev_max_ec_fastmap)
+		ret = sprintf(buf, "%d\n", ubi_calc_max_ec(ubi, 0, UBI_FM_MAX_START));
 	else if (attr == &dev_mean_ec_fastmap)
 		ret = sprintf(buf, "%d\n", ubi_calc_mean_ec(ubi, 0, UBI_FM_MAX_START));
 #endif
@@ -447,6 +489,7 @@ static struct attribute *ubi_dev_attrs[] = {
 	&dev_max_ec.attr,
 	&dev_mean_ec.attr,
 #ifdef CONFIG_MTD_UBI_FASTMAP
+	&dev_max_ec_fastmap.attr,
 	&dev_mean_ec_fastmap.attr,
 #endif
 	&dev_reserved_for_bad.attr,
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH v3 04/10] ubi: Expose mean erase counter for data in sysfs

[Rickard Andersson]

Expose mean erase counter for data area in sysfs. Data area includes
all blocks except for the fastmap area.

This value can be used in order to estimate life expectancy of the
flash.

Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: Rickard Andersson <rickard.andersson@axis.com>
---
 drivers/mtd/ubi/build.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
index a5a2e6e3be38..9cc7bc74f0d2 100644
--- a/drivers/mtd/ubi/build.c
+++ b/drivers/mtd/ubi/build.c
@@ -138,6 +138,8 @@ static struct device_attribute dev_max_ec_fastmap =
 	__ATTR(max_ec_fastmap, S_IRUGO, dev_attribute_show, NULL);
 static struct device_attribute dev_mean_ec_fastmap =
 	__ATTR(mean_ec_fastmap, S_IRUGO, dev_attribute_show, NULL);
+static struct device_attribute dev_mean_ec_data =
+	__ATTR(mean_ec_data, S_IRUGO, dev_attribute_show, NULL);
 #endif
 static struct device_attribute dev_reserved_for_bad =
 	__ATTR(reserved_for_bad, S_IRUGO, dev_attribute_show, NULL);
@@ -460,6 +462,8 @@ static ssize_t dev_attribute_show(struct device *dev,
 		ret = sprintf(buf, "%d\n", ubi_calc_max_ec(ubi, 0, UBI_FM_MAX_START));
 	else if (attr == &dev_mean_ec_fastmap)
 		ret = sprintf(buf, "%d\n", ubi_calc_mean_ec(ubi, 0, UBI_FM_MAX_START));
+	else if (attr == &dev_mean_ec_data)
+		ret = sprintf(buf, "%d\n", ubi_calc_mean_ec(ubi, UBI_FM_MAX_START, ubi->peb_count));
 #endif
 	else if (attr == &dev_reserved_for_bad)
 		ret = sprintf(buf, "%d\n", ubi->beb_rsvd_pebs);
@@ -491,6 +495,7 @@ static struct attribute *ubi_dev_attrs[] = {
 #ifdef CONFIG_MTD_UBI_FASTMAP
 	&dev_max_ec_fastmap.attr,
 	&dev_mean_ec_fastmap.attr,
+	&dev_mean_ec_data.attr,
 #endif
 	&dev_reserved_for_bad.attr,
 	&dev_bad_peb_count.attr,
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH v3 05/10] ubi: Expose max erase counter for data in sysfs

[Rickard Andersson]

Expose max erase counter for data area in sysfs. Data area includes
all blocks except for the fastmap area.

This value can be used in order to estimate life expectancy of the
flash.

Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: Rickard Andersson <rickard.andersson@axis.com>
---
 drivers/mtd/ubi/build.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
index 9cc7bc74f0d2..fa1ff0146572 100644
--- a/drivers/mtd/ubi/build.c
+++ b/drivers/mtd/ubi/build.c
@@ -138,6 +138,8 @@ static struct device_attribute dev_max_ec_fastmap =
 	__ATTR(max_ec_fastmap, S_IRUGO, dev_attribute_show, NULL);
 static struct device_attribute dev_mean_ec_fastmap =
 	__ATTR(mean_ec_fastmap, S_IRUGO, dev_attribute_show, NULL);
+static struct device_attribute dev_max_ec_data =
+	__ATTR(max_ec_data, S_IRUGO, dev_attribute_show, NULL);
 static struct device_attribute dev_mean_ec_data =
 	__ATTR(mean_ec_data, S_IRUGO, dev_attribute_show, NULL);
 #endif
@@ -462,6 +464,8 @@ static ssize_t dev_attribute_show(struct device *dev,
 		ret = sprintf(buf, "%d\n", ubi_calc_max_ec(ubi, 0, UBI_FM_MAX_START));
 	else if (attr == &dev_mean_ec_fastmap)
 		ret = sprintf(buf, "%d\n", ubi_calc_mean_ec(ubi, 0, UBI_FM_MAX_START));
+	else if (attr == &dev_max_ec_data)
+		ret = sprintf(buf, "%d\n", ubi_calc_max_ec(ubi, UBI_FM_MAX_START, ubi->peb_count));
 	else if (attr == &dev_mean_ec_data)
 		ret = sprintf(buf, "%d\n", ubi_calc_mean_ec(ubi, UBI_FM_MAX_START, ubi->peb_count));
 #endif
@@ -495,6 +499,7 @@ static struct attribute *ubi_dev_attrs[] = {
 #ifdef CONFIG_MTD_UBI_FASTMAP
 	&dev_max_ec_fastmap.attr,
 	&dev_mean_ec_fastmap.attr,
+	&dev_max_ec_data.attr,
 	&dev_mean_ec_data.attr,
 #endif
 	&dev_reserved_for_bad.attr,
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH v3 06/10] ubi: Add mean erase counter sysfs attribute

[Rickard Andersson]

A mean erase counter gives a better understanding of the flash
wear.

Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: Rickard Andersson <rickard.andersson@axis.com>
---
 Documentation/ABI/stable/sysfs-class-ubi | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/Documentation/ABI/stable/sysfs-class-ubi b/Documentation/ABI/stable/sysfs-class-ubi
index a6b324014692..535a3e6dab12 100644
--- a/Documentation/ABI/stable/sysfs-class-ubi
+++ b/Documentation/ABI/stable/sysfs-class-ubi
@@ -219,3 +219,10 @@ Description:
 		Logical eraseblock size of this volume. Equivalent to logical
 		eraseblock size of the device aligned on the volume alignment
 		value.
+
+What:		/sys/class/ubi/ubiX/mean_ec
+Date:		October 2024
+KernelVersion:	6.X
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		Mean physical eraseblock erase counter value.
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH v3 07/10] ubi: Add mean erase counter sysfs attribute for fastmap area

[Rickard Andersson]

A mean erase counter gives a better understanding of the flash
wear of the fastmap area.

Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: Rickard Andersson <rickard.andersson@axis.com>
---
 Documentation/ABI/stable/sysfs-class-ubi | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Documentation/ABI/stable/sysfs-class-ubi b/Documentation/ABI/stable/sysfs-class-ubi
index 535a3e6dab12..05a701b03a9a 100644
--- a/Documentation/ABI/stable/sysfs-class-ubi
+++ b/Documentation/ABI/stable/sysfs-class-ubi
@@ -226,3 +226,11 @@ KernelVersion:	6.X
 Contact:	linux-mtd@lists.infradead.org
 Description:
 		Mean physical eraseblock erase counter value.
+
+What:		/sys/class/ubi/ubiX/mean_ec_fastmap
+Date:		October 2024
+KernelVersion:	6.X
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		Mean physical eraseblock erase counter value for the fastmap
+		area.
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH v3 08/10] ubi: Add max erase counter sysfs attribute for fastmap area

[Rickard Andersson]

A max erase counter specifically for the fastmap area gives a better
understanding of the flash wear.

Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: Rickard Andersson <rickard.andersson@axis.com>
---
 Documentation/ABI/stable/sysfs-class-ubi | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Documentation/ABI/stable/sysfs-class-ubi b/Documentation/ABI/stable/sysfs-class-ubi
index 05a701b03a9a..62c93fda0bc4 100644
--- a/Documentation/ABI/stable/sysfs-class-ubi
+++ b/Documentation/ABI/stable/sysfs-class-ubi
@@ -78,6 +78,14 @@ Contact:	Artem Bityutskiy <dedekind@infradead.org>
 Description:
 		Maximum physical eraseblock erase counter value.
 
+What:		/sys/class/ubi/ubiX/max_ec_fastmap
+Date:		October 2024
+KernelVersion:	6.X
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		Maximum physical eraseblock erase counter value for a block
+		in the fastmap area.
+
 What:		/sys/class/ubi/ubiX/max_vol_count
 Date:		July 2006
 KernelVersion:	2.6.22
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH v3 09/10] ubi: Add mean erase counter sysfs attribute for data area

[Rickard Andersson]

Data area includes all blocks except for the fastmap area.

Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: Rickard Andersson <rickard.andersson@axis.com>
---
 Documentation/ABI/stable/sysfs-class-ubi | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Documentation/ABI/stable/sysfs-class-ubi b/Documentation/ABI/stable/sysfs-class-ubi
index 62c93fda0bc4..100e1c0698a3 100644
--- a/Documentation/ABI/stable/sysfs-class-ubi
+++ b/Documentation/ABI/stable/sysfs-class-ubi
@@ -242,3 +242,11 @@ Contact:	linux-mtd@lists.infradead.org
 Description:
 		Mean physical eraseblock erase counter value for the fastmap
 		area.
+
+What:		/sys/class/ubi/ubiX/mean_ec_data
+Date:		October 2024
+KernelVersion:	6.X
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		Mean physical eraseblock erase counter value for data area.
+		Data area includes all blocks except for the fastmap area.
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH v3 10/10] ubi: Add max erase counter sysfs attribute for data area

[Rickard Andersson]

Data area includes all blocks except for the fastmap area.

Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: Rickard Andersson <rickard.andersson@axis.com>
---
 Documentation/ABI/stable/sysfs-class-ubi | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Documentation/ABI/stable/sysfs-class-ubi b/Documentation/ABI/stable/sysfs-class-ubi
index 100e1c0698a3..ff453557b0db 100644
--- a/Documentation/ABI/stable/sysfs-class-ubi
+++ b/Documentation/ABI/stable/sysfs-class-ubi
@@ -86,6 +86,14 @@ Description:
 		Maximum physical eraseblock erase counter value for a block
 		in the fastmap area.
 
+What:		/sys/class/ubi/ubiX/max_ec_data
+Date:		October 2024
+KernelVersion:	6.X
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		Max physical eraseblock erase counter value for data area.
+		Data area includes all blocks except for the fastmap area.
+
 What:		/sys/class/ubi/ubiX/max_vol_count
 Date:		July 2006
 KernelVersion:	2.6.22
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v3 01/10] ubi: Expose mean erase counter in sysfs

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Rickard X Andersson" <rickard.andersson@axis.com>
> There exists more detailed info in debugfs, but this information is
> only available for debug builds.

If the detailed info is useful for regular userspace we can also think
of adding a non-debugfs interface for it.
E.g. an ioctl() that returns this data in binary form.

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v3 01/10] ubi: Expose mean erase counter in sysfs

[Zhihao Cheng]

在 2024/11/7 4:34, Richard Weinberger 写道:
> ----- Ursprüngliche Mail -----
>> Von: "Rickard X Andersson" <rickard.andersson@axis.com>
>> There exists more detailed info in debugfs, but this information is
>> only available for debug builds.
> 
> If the detailed info is useful for regular userspace we can also think
> of adding a non-debugfs interface for it.
> E.g. an ioctl() that returns this data in binary form.

This method sounds better.
> 
> Thanks,
> //richard
> 
> .
> 


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v3 01/10] ubi: Expose mean erase counter in sysfs

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "chengzhihao1" <chengzhihao1@huawei.com>
> An: "richard" <richard@nod.at>, "Rickard X Andersson" <rickard.andersson@axis.com>
> CC: "linux-mtd" <linux-mtd@lists.infradead.org>, "rickard314 andersson" <rickard314.andersson@gmail.com>, "kernel"
> <kernel@axis.com>
> Gesendet: Donnerstag, 7. November 2024 02:39:45
> Betreff: Re: [PATCH v3 01/10] ubi: Expose mean erase counter in sysfs

> 在 2024/11/7 4:34, Richard Weinberger 写道:
>> ----- Ursprüngliche Mail -----
>>> Von: "Rickard X Andersson" <rickard.andersson@axis.com>
>>> There exists more detailed info in debugfs, but this information is
>>> only available for debug builds.
>> 
>> If the detailed info is useful for regular userspace we can also think
>> of adding a non-debugfs interface for it.
>> E.g. an ioctl() that returns this data in binary form.
> 
> This method sounds better.

Rickard, what do you think about this option?

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v3 01/10] ubi: Expose mean erase counter in sysfs

[Rickard x Andersson]

On 11/14/24 19:50, Richard Weinberger wrote:
> ----- Ursprüngliche Mail -----
>> Von: "chengzhihao1" <chengzhihao1@huawei.com>
>> An: "richard" <richard@nod.at>, "Rickard X Andersson" <rickard.andersson@axis.com>
>> CC: "linux-mtd" <linux-mtd@lists.infradead.org>, "rickard314 andersson" <rickard314.andersson@gmail.com>, "kernel"
>> <kernel@axis.com>
>> Gesendet: Donnerstag, 7. November 2024 02:39:45
>> Betreff: Re: [PATCH v3 01/10] ubi: Expose mean erase counter in sysfs
> 
>> 在 2024/11/7 4:34, Richard Weinberger 写道:
>>> ----- Ursprüngliche Mail -----
>>>> Von: "Rickard X Andersson" <rickard.andersson@axis.com>
>>>> There exists more detailed info in debugfs, but this information is
>>>> only available for debug builds.
>>>
>>> If the detailed info is useful for regular userspace we can also think
>>> of adding a non-debugfs interface for it.
>>> E.g. an ioctl() that returns this data in binary form.
>>
>> This method sounds better.
> 
> Rickard, what do you think about this option?
> 
> Thanks,
> //richard

Hi,

I would prefer to keep it in sysfs. We already have the following in 
sysfs (/sys/class/ubi/ubiX) that is related to flash wear:

-r--r--r--    1 root root      4096 Oct 24 08:54 avail_eraseblocks
-r--r--r--    1 root root      4096 Nov 15 08:26 bad_peb_count
-r--r--r--    1 root root      4096 Nov 15 08:26 max_ec
-r--r--r--    1 root root      4096 Oct 24 08:54 reserved_for_bad

With my patch series the following will be added for normal builds:

-r--r--r--    1 root root      4096 Nov 15 08:26 mean_ec

If fastmap is enabled (CONFIG_MTD_UBI_FASTMAP=y) also the following is 
added:

-r--r--r--    1 root root      4096 Nov 15 08:26 max_ec_data
-r--r--r--    1 root root      4096 Nov 15 08:26 max_ec_fastmap
-r--r--r--    1 root root      4096 Nov 15 08:26 mean_ec_data
-r--r--r--    1 root root      4096 Nov 15 08:26 mean_ec_fastmap

If you think the above is too "bloated" in the fastmap case 
(CONFIG_MTD_UBI_FASTMAP=y) we could reduce the additional files in the 
fastmap case to the following (More similar to my V1 version of the 
patch series):

-r--r--r--    1 root root      4096 Nov 15 08:26 mean_ec_fastmap

Thanks,
Rickard A.


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v3 01/10] ubi: Expose mean erase counter in sysfs

[Zhihao Cheng]

在 2024/11/15 17:00, Rickard x Andersson 写道:
> On 11/14/24 19:50, Richard Weinberger wrote:
>> ----- Ursprüngliche Mail -----
>>> Von: "chengzhihao1" <chengzhihao1@huawei.com>
>>> An: "richard" <richard@nod.at>, "Rickard X Andersson" 
>>> <rickard.andersson@axis.com>
>>> CC: "linux-mtd" <linux-mtd@lists.infradead.org>, "rickard314 
>>> andersson" <rickard314.andersson@gmail.com>, "kernel"
>>> <kernel@axis.com>
>>> Gesendet: Donnerstag, 7. November 2024 02:39:45
>>> Betreff: Re: [PATCH v3 01/10] ubi: Expose mean erase counter in sysfs
>>
>>> 在 2024/11/7 4:34, Richard Weinberger 写道:
>>>> ----- Ursprüngliche Mail -----
>>>>> Von: "Rickard X Andersson" <rickard.andersson@axis.com>
>>>>> There exists more detailed info in debugfs, but this information is
>>>>> only available for debug builds.
>>>>
>>>> If the detailed info is useful for regular userspace we can also think
>>>> of adding a non-debugfs interface for it.
>>>> E.g. an ioctl() that returns this data in binary form.
>>>
>>> This method sounds better.
>>
>> Rickard, what do you think about this option?
>>
>> Thanks,
>> //richard
> 
> Hi,
> 
> I would prefer to keep it in sysfs. We already have the following in 
> sysfs (/sys/class/ubi/ubiX) that is related to flash wear:
> 
> -r--r--r--    1 root root      4096 Oct 24 08:54 avail_eraseblocks
> -r--r--r--    1 root root      4096 Nov 15 08:26 bad_peb_count
> -r--r--r--    1 root root      4096 Nov 15 08:26 max_ec
> -r--r--r--    1 root root      4096 Oct 24 08:54 reserved_for_bad
> 
> With my patch series the following will be added for normal builds:
> 
> -r--r--r--    1 root root      4096 Nov 15 08:26 mean_ec
> 
> If fastmap is enabled (CONFIG_MTD_UBI_FASTMAP=y) also the following is 
> added:
> 
> -r--r--r--    1 root root      4096 Nov 15 08:26 max_ec_data
> -r--r--r--    1 root root      4096 Nov 15 08:26 max_ec_fastmap
> -r--r--r--    1 root root      4096 Nov 15 08:26 mean_ec_data
> -r--r--r--    1 root root      4096 Nov 15 08:26 mean_ec_fastmap
> 
> If you think the above is too "bloated" in the fastmap case 
> (CONFIG_MTD_UBI_FASTMAP=y) we could reduce the additional files in the 
> fastmap case to the following (More similar to my V1 version of the 
> patch series):
> 
> -r--r--r--    1 root root      4096 Nov 15 08:26 mean_ec_fastmap
> 
> Thanks,
> Rickard A.
> 
> .

Hi Rickard. Let's reorganize our thoughts.

1. We need to know the overall flash wear, including:
   1) Is the flash over/approaching lifetime? - the 'max_ec' could tell us
   2) How worn down the flash is in different areas? the 'mean_ec' could 
tell us the overall situation, the 'mean_ec_data' and 'mean_ec_fastmap' 
could tell us more details in different areas.
   3) Does the wear-leveling algorithm work correctly? the 
'max_ec_data', 'min_ec_data' could be used to judge whether PEBs usage 
in data area is balanced，the 'max_ec_fastmap', 'min_ec_fastmap' could be 
used to judge whether PEBs usage in fastmap area is balanced.
2. Kernel only provides mechanisms, user program implements logic to 
satisfy what user wants.
3. I think points 1-1)~3) belongs to user requirements, they can be 
implemented according to the raw ec statistics(debugfs interface, or 
according to Richard's suggestion providing a new ioctl).
Why do I think points 1-1)~3) belong to user logic? They are customized 
and can only be used in specific scenarios, not like 'max_ec', 
'bad_peb_count' and 'leb_size' which are general features of UBI and be 
widely used. For example, if someday one user wants to known the mediam 
ec count in data and fastmap areas, two interfaces will be added in 
sysfs, which looks bloated(There are too many bespoke interfaces under 
sysfs).


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v3 01/10] ubi: Expose mean erase counter in sysfs

[Rickard X Andersson]

On 11/16/24 03:55, Zhihao Cheng wrote:
> 在 2024/11/15 17:00, Rickard x Andersson 写道:
>> On 11/14/24 19:50, Richard Weinberger wrote:
>>> ----- Ursprüngliche Mail -----
>>>> Von: "chengzhihao1" <chengzhihao1@huawei.com>
>>>> An: "richard" <richard@nod.at>, "Rickard X Andersson" 
>>>> <rickard.andersson@axis.com>
>>>> CC: "linux-mtd" <linux-mtd@lists.infradead.org>, "rickard314 
>>>> andersson" <rickard314.andersson@gmail.com>, "kernel"
>>>> <kernel@axis.com>
>>>> Gesendet: Donnerstag, 7. November 2024 02:39:45
>>>> Betreff: Re: [PATCH v3 01/10] ubi: Expose mean erase counter in sysfs
>>>
>>>> 在 2024/11/7 4:34, Richard Weinberger 写道:
>>>>> ----- Ursprüngliche Mail -----
>>>>>> Von: "Rickard X Andersson" <rickard.andersson@axis.com>
>>>>>> There exists more detailed info in debugfs, but this information is
>>>>>> only available for debug builds.
>>>>>
>>>>> If the detailed info is useful for regular userspace we can also think
>>>>> of adding a non-debugfs interface for it.
>>>>> E.g. an ioctl() that returns this data in binary form.
>>>>
>>>> This method sounds better.
>>>
>>> Rickard, what do you think about this option?
>>>
>>> Thanks,
>>> //richard
>>
>> Hi,
>>
>> I would prefer to keep it in sysfs. We already have the following in 
>> sysfs (/sys/class/ubi/ubiX) that is related to flash wear:
>>
>> -r--r--r--    1 root root      4096 Oct 24 08:54 avail_eraseblocks
>> -r--r--r--    1 root root      4096 Nov 15 08:26 bad_peb_count
>> -r--r--r--    1 root root      4096 Nov 15 08:26 max_ec
>> -r--r--r--    1 root root      4096 Oct 24 08:54 reserved_for_bad
>>
>> With my patch series the following will be added for normal builds:
>>
>> -r--r--r--    1 root root      4096 Nov 15 08:26 mean_ec
>>
>> If fastmap is enabled (CONFIG_MTD_UBI_FASTMAP=y) also the following is 
>> added:
>>
>> -r--r--r--    1 root root      4096 Nov 15 08:26 max_ec_data
>> -r--r--r--    1 root root      4096 Nov 15 08:26 max_ec_fastmap
>> -r--r--r--    1 root root      4096 Nov 15 08:26 mean_ec_data
>> -r--r--r--    1 root root      4096 Nov 15 08:26 mean_ec_fastmap
>>
>> If you think the above is too "bloated" in the fastmap case 
>> (CONFIG_MTD_UBI_FASTMAP=y) we could reduce the additional files in the 
>> fastmap case to the following (More similar to my V1 version of the 
>> patch series):
>>
>> -r--r--r--    1 root root      4096 Nov 15 08:26 mean_ec_fastmap
>>
>> Thanks,
>> Rickard A.
>>
>> .
> 
> Hi Rickard. Let's reorganize our thoughts.

Thanks for feedback.

> 
> 1. We need to know the overall flash wear, including:
>    1) Is the flash over/approaching lifetime? - the 'max_ec' could tell us

On many devices max_ec is practically unusable. The reason for this 
being previous bugs in UBI causing single fastmap blocks to be written 
many many times. For example a situation where single fastmap block is 
written e.g. 60000 times and the rest of the fastmap blocks being 
written around 2000 times.

>    2) How worn down the flash is in different areas? the 'mean_ec' could 
> tell us the overall situation, the 'mean_ec_data' and 'mean_ec_fastmap' 
> could tell us more details in different areas.
>    3) Does the wear-leveling algorithm work correctly? the 
> 'max_ec_data', 'min_ec_data' could be used to judge whether PEBs usage 
> in data area is balanced，the 'max_ec_fastmap', 'min_ec_fastmap' could 
> be used to judge whether PEBs usage in fastmap area is balanced.
> 2. Kernel only provides mechanisms, user program implements logic to 
> satisfy what user wants.
> 3. I think points 1-1)~3) belongs to user requirements, they can be 
> implemented according to the raw ec statistics(debugfs interface, or 
> according to Richard's suggestion providing a new ioctl).
> Why do I think points 1-1)~3) belong to user logic? They are customized 
> and can only be used in specific scenarios, not like 'max_ec', 
> 'bad_peb_count' and 'leb_size' which are general features of UBI and be 
> widely used. For example, if someday one user wants to known the mediam 
> ec count in data and fastmap areas, two interfaces will be added in 
> sysfs, which looks bloated(There are too many bespoke interfaces under 
> sysfs).
> 

@Richard Weinberger, do you also prefer an IOCTL solution? Should I 
resend my patch series using IOCTL.

Thanks,
Rickard A.


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/