UBI Fastmap updates (v7++)

UBI Fastmap updates (v7++)

[Richard Weinberger]

Here are some updates for UBI Fastmap.

Beside of mostly minor changes it addresses also two major issues:

1. While creating the fastmap we have to see all PEBs, no PEB has to be in flight.
To ensure this fastmap calls ubi_wl_flush(). But after running more tests I encountered
the problem that sometimes PEBs are still missing. It turned out that  ubi_wl_flush() itself
can create new work which will be processed after ubi_wl_flush() returns.
To fix this I've changed wear_leveling_worker() such that it does not call schedule_erase()
and instead calls the erase worker direly.
Furthermore the erase worker has been changed to omit the call to ensure_wear_leveling(),
which produces again work.
With this changes ubi_wl_flush() really returns only if all work is done and fastmap sees als PEBs.
Artem, I know this is a sledgehammer solution. What do you prefer?

2. wear_leveling_worker() does not use ubi_wl_get_peb(),
therefore some changes are not visible to fastmap.
To address this issue a second small pool has been added.
It contains PEBs to be used by wear_leveling_worker().
For now this pool get refilled and checked by any call to ubi_wl_get_peb().
We cannot refill this pool upon it's empty.
This could trigger a call to ubi_update_fastmap().
As described above  ubi_update_fastmap() calls ubi_wl_flush()
which could call ubi_update_fastmap() again...

Later I'll release some test-scripts and patches to proof the correctness of fastmap.
The patches are mostly very hacky, 
I try to rewrite them in a sane manner such that we can add them into the fastmap code.
E.g. We could cal scan_all() after attaching from a fastmap to proof that fastmap reconstructs
the EBA table correctly.

Happy testing!
//richard

[PATCH 01/21] UBI: Fastmap: enhance find_early_wl_entry()
[PATCH 02/21] UBI: Fastmap: Add PEB leak detection
[PATCH 03/21] UBI: Fastmap: Remove useless code
[PATCH 04/21] UBI: Fastmap: Check for duplicated PEBs in add_aeb()
[PATCH 05/21] UBI: Fastmap: Rewrite scan_pool()
[PATCH 06/21] UBI: Fastmap: Ensure that new_fm->e[0]->ec is always
[PATCH 07/21] UBI: Fastmap: Prevent PEB leak
[PATCH 08/21] UBI: Fastmap: Add scrub support to add_aeb()
[PATCH 09/21] UBI: Fastmap: Make ubi_wl_get_fm_peb() return a
[PATCH 10/21] UBI: Fastmap: Detect whether a PEB has to be unmapped.
[PATCH 11/21] UBI: Fastmap: Adjust comment above ubi_wl_flush()
[PATCH 12/21] UBI: Fastmap: Handle unused "used" PEBs better
[PATCH 13/21] UBI: Fastmap: Introduce WL pool Signed-off-by: Richard
[PATCH 14/21] UBI: Fastmap: Erase PEBs synchronous in
[PATCH 15/21] UBI: Fastmap: Fix NULL-pointer bug
[PATCH 16/21] UBI: Fastmap: Fix loglevel
[PATCH 17/21] UBI: Fastmap: Relax EC self check
[PATCH 18/21] UBI: Fastmap: Store magic values in be32
[PATCH 19/21] UBI: Fastmap: Minor fixes
[PATCH 20/21] UBI: Fastmap: Torture fastmap PEBs that showed bit
[PATCH 21/21] UBI: Fastmap: make checkpatch.pl happy Signed-off-by:

[PATCH 01/21] UBI: Fastmap: enhance find_early_wl_entry()

[Richard Weinberger]

Select always the PEB with the lowest pnum and ec.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/wl.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index 2325d3d..8f6321a 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -418,11 +418,13 @@ static struct ubi_wl_entry *find_early_wl_entry(struct rb_root *root,
 {
 	struct rb_node *p;
 	struct ubi_wl_entry *e, *victim = NULL;
+	int max_ec = UBI_MAX_ERASECOUNTER;
 
 	ubi_rb_for_each_entry(p, e, root, u.rb) {
-		if (e->pnum < max_pnum) {
+		if (e->pnum < max_pnum && e->ec < max_ec) {
 			victim = e;
 			max_pnum = e->pnum;
+			max_ec = e->ec;
 		}
 	}
 
-- 
1.7.6.5

[PATCH 02/21] UBI: Fastmap: Add PEB leak detection

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/wl.c |   15 ++++++++++++++-
 1 files changed, 14 insertions(+), 1 deletions(-)

diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index 8f6321a..ea0ab86 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -1557,7 +1557,7 @@ static void cancel_pending(struct ubi_device *ubi)
  */
 int ubi_wl_init(struct ubi_device *ubi, struct ubi_attach_info *ai)
 {
-	int err, i;
+	int err, i, found_pebs = 0;
 	struct rb_node *rb1, *rb2;
 	struct ubi_ainf_volume *av;
 	struct ubi_ainf_peb *aeb, *tmp;
@@ -1596,6 +1596,8 @@ int ubi_wl_init(struct ubi_device *ubi, struct ubi_attach_info *ai)
 			kmem_cache_free(ubi_wl_entry_slab, e);
 			goto out_free;
 		}
+
+		found_pebs++;
 	}
 
 	list_for_each_entry(aeb, &ai->free, u.list) {
@@ -1613,6 +1615,8 @@ int ubi_wl_init(struct ubi_device *ubi, struct ubi_attach_info *ai)
 		wl_tree_add(e, &ubi->free);
 
 		ubi->lookuptbl[e->pnum] = e;
+
+		found_pebs++;
 	}
 
 	ubi_rb_for_each_entry(rb1, av, &ai->volumes, rb) {
@@ -1638,9 +1642,18 @@ int ubi_wl_init(struct ubi_device *ubi, struct ubi_attach_info *ai)
 				       e->pnum, e->ec);
 				wl_tree_add(e, &ubi->scrub);
 			}
+
+			found_pebs++;
 		}
 	}
 
+	dbg_wl("found %i PEBs", found_pebs);
+
+	if (ai->fm)
+		ubi_assert(ubi->peb_count == found_pebs + ai->fm->used_blocks);
+	else
+		ubi_assert(ubi->peb_count == found_pebs);
+
 	if (ubi->avail_pebs < WL_RESERVED_PEBS) {
 		ubi_err("no enough physical eraseblocks (%d, need %d)",
 			ubi->avail_pebs, WL_RESERVED_PEBS);
-- 
1.7.6.5

[PATCH 03/21] UBI: Fastmap: Remove useless code

[Richard Weinberger]

This case can not happen. No need to check for it.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/wl.c |    2 --
 1 files changed, 0 insertions(+), 2 deletions(-)

diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index ea0ab86..52548e7 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -1630,8 +1630,6 @@ int ubi_wl_init(struct ubi_device *ubi, struct ubi_attach_info *ai)
 			e->pnum = aeb->pnum;
 			e->ec = aeb->ec;
 			ubi->lookuptbl[e->pnum] = e;
-			if (ubi_is_fm_block(ubi, aeb->pnum))
-				continue;
 
 			if (!aeb->scrub) {
 				dbg_wl("add PEB %d EC %d to the used tree",
-- 
1.7.6.5

[PATCH 04/21] UBI: Fastmap: Check for duplicated PEBs in add_aeb()

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index d446fc3..97d694f 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -53,6 +53,10 @@ static int add_aeb(struct ubi_attach_info *ai, struct list_head *list,
 {
 	struct ubi_ainf_peb *aeb;
 
+	list_for_each_entry(aeb, list, u.list)
+		if (aeb->pnum == pnum)
+			return 0;
+
 	aeb = kmem_cache_alloc(ai->aeb_slab_cache, GFP_KERNEL);
 	if (!aeb)
 		return -ENOMEM;
-- 
1.7.6.5

[PATCH 05/21] UBI: Fastmap: Rewrite scan_pool()

[Richard Weinberger]

Rewrite scan_pool() and make sure that free PEBs are added
to the free list.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |   51 +++++++++++++++++++-------------------------
 1 files changed, 22 insertions(+), 29 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 97d694f..a82a604 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -349,6 +349,8 @@ static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 	 * after the creation of the fastmap
 	 */
 	for (i = 0; i < pool_size; i++) {
+		int scrub = 0;
+
 		pnum = be32_to_cpu(pebs[i]);
 
 		if (ubi_io_is_bad(ubi, pnum)) {
@@ -358,42 +360,33 @@ static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 			goto out;
 		}
 
+		err = ubi_io_read_ec_hdr(ubi, pnum, ech, 0);
+		if (err && err != UBI_IO_BITFLIPS) {
+			dbg_bld("unable to read EC header!");
+			ret = err > 0 ? UBI_BAD_FASTMAP : err;
+			goto out;
+		} else if (ret == UBI_IO_BITFLIPS)
+			scrub = 1;
+
+		if (be32_to_cpu(ech->image_seq) != ubi->image_seq) {
+			dbg_bld("image seq mismatch!");
+			err = UBI_BAD_FASTMAP;
+			goto out;
+		}
+
 		err = ubi_io_read_vid_hdr(ubi, pnum, vh, 0);
+		if (err == UBI_IO_FF || err == UBI_IO_FF_BITFLIPS) {
+			scrub = 1;
+			//TODO SCRUB!
+			add_aeb(ai, &ai->free, pnum, be64_to_cpu(ech->ec));
 
-		/* TODO: so here again - calling the function and checking what
-		 * it returns I consider to be one logical block which should
-		 * not contain newlines inbetween. You are inconsistent about
-		 * this and in some places you have a newline, in some you
-		 * don't. Could we please remove them globally in cases like
-		 * this?
-		 * Again my disclaimer: sorry, I hope this is not too annoying
-		 * for you. */
-		if (err == UBI_IO_FF || err == UBI_IO_FF_BITFLIPS)
 			continue;
-		else if (err == 0 || err == UBI_IO_BITFLIPS) {
-			int scrub = 0;
-
-			ubi_msg("Found non empty PEB:%i in pool", pnum);
+		} else if (err == 0 || err == UBI_IO_BITFLIPS) {
+			dbg_bld("Found non empty PEB:%i in pool", pnum);
 
 			if (err == UBI_IO_BITFLIPS)
 				scrub = 1;
 
-			err = ubi_io_read_ec_hdr(ubi, pnum, ech, 0);
-			if (err && err != UBI_IO_BITFLIPS) {
-				dbg_bld("unable to read EC header!");
-				ret = err > 0 ? UBI_BAD_FASTMAP : err;
-
-				goto out;
-			} else if (ret == UBI_IO_BITFLIPS)
-				scrub = 1;
-
-			if (be32_to_cpu(ech->image_seq) != ubi->image_seq) {
-				dbg_bld("image seq mismatch!");
-				err = UBI_BAD_FASTMAP;
-
-				goto out;
-			}
-
 			found_orphan = 0;
 			list_for_each_entry(tmp_aeb, eba_orphans, u.list) {
 				if (tmp_aeb->pnum == pnum) {
-- 
1.7.6.5

[PATCH 06/21] UBI: Fastmap: Ensure that new_fm->e[0]->ec is always set

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index a82a604..c2a1c18 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -1310,7 +1310,6 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 			}
 
 			new_fm->e[0]->pnum = old_fm->e[0]->pnum;
-			new_fm->e[0]->ec = old_fm->e[0]->ec;
 		} else {
 			/* we've got a new early PEB, return the old one */
 			ubi_wl_put_fm_peb(ubi, old_fm->e[0], 0);
@@ -1326,9 +1325,10 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 
 			goto err;
 		}
-		new_fm->e[0]->ec = get_ec(ubi, new_fm->e[0]->pnum);
 	}
 
+	new_fm->e[0]->ec = get_ec(ubi, new_fm->e[0]->pnum);
+
 	if (new_fm->used_blocks > UBI_FM_MAX_BLOCKS) {
 		ubi_err("fastmap too large");
 		ret = -ENOSPC;
-- 
1.7.6.5

[PATCH 07/21] UBI: Fastmap: Prevent PEB leak

[Richard Weinberger]

Before writing the new fastmap we have to ensure that no
PEBs are in flight.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index c2a1c18..82a011d 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -1366,6 +1366,14 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 		kfree(old_fm);
 	}
 
+	/* Ensure that the PEBs of the old fastmap got erased and added to the
+	 * free list before we write the fastmap. Otherwise fastmp does not
+	 * see these PEBs and we leak them.
+	 * FIXME: Rewrite ubi_wl_flush() such that we can flush only the
+	 * erase work instead of all work.
+	 */
+	ubi_wl_flush(ubi);
+
 	ret = ubi_write_fastmap(ubi, new_fm);
 out_unlock:
 	mutex_unlock(&ubi->fm_mutex);
-- 
1.7.6.5

[PATCH 08/21] UBI: Fastmap: Add scrub support to add_aeb()

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |   15 +++++++--------
 1 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 82a011d..1d3f724 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -47,9 +47,10 @@ out:
  * @list: the target list
  * @pnum: PEB number of the new attach erase block
  * @ec: erease counter of the new LEB
+ * @scrub: scrub this PEB after attaching
  */
 static int add_aeb(struct ubi_attach_info *ai, struct list_head *list,
-		   int pnum, int ec)
+		   int pnum, int ec, int scrub)
 {
 	struct ubi_ainf_peb *aeb;
 
@@ -64,7 +65,8 @@ static int add_aeb(struct ubi_attach_info *ai, struct list_head *list,
 	aeb->pnum = pnum;
 	aeb->ec = ec;
 	aeb->lnum = -1;
-	aeb->scrub = aeb->copy_flag = aeb->sqnum = 0;
+	aeb->scrub = scrub;
+	aeb->copy_flag = aeb->sqnum = 0;
 
 	ai->ec_sum += aeb->ec;
 	ai->ec_count++;
@@ -376,10 +378,7 @@ static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 
 		err = ubi_io_read_vid_hdr(ubi, pnum, vh, 0);
 		if (err == UBI_IO_FF || err == UBI_IO_FF_BITFLIPS) {
-			scrub = 1;
-			//TODO SCRUB!
-			add_aeb(ai, &ai->free, pnum, be64_to_cpu(ech->ec));
-
+			add_aeb(ai, &ai->free, pnum, be64_to_cpu(ech->ec), 1);
 			continue;
 		} else if (err == 0 || err == UBI_IO_BITFLIPS) {
 			dbg_bld("Found non empty PEB:%i in pool", pnum);
@@ -530,7 +529,7 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 			goto fail_bad;
 
 		add_aeb(ai, &ai->free, be32_to_cpu(fmec->pnum),
-			be32_to_cpu(fmec->ec));
+			be32_to_cpu(fmec->ec), 0);
 	}
 
 	/* read EC values from used list */
@@ -541,7 +540,7 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 			goto fail_bad;
 
 		add_aeb(ai, &used, be32_to_cpu(fmec->pnum),
-			be32_to_cpu(fmec->ec));
+			be32_to_cpu(fmec->ec), 0);
 	}
 
 	ai->mean_ec = div_u64(ai->ec_sum, ai->ec_count);
-- 
1.7.6.5

[PATCH 09/21] UBI: Fastmap: Make ubi_wl_get_fm_peb() return a ubi_wl_entry

[Richard Weinberger]

...makes life easier

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |   36 +++++++++++-------------------------
 drivers/mtd/ubi/ubi.h     |    2 +-
 drivers/mtd/ubi/wl.c      |    8 +++-----
 3 files changed, 15 insertions(+), 31 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 1d3f724..fb5dae5 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -1191,24 +1191,6 @@ out:
 }
 
 /**
- * get_ec - returns the erase counter of a given PEB
- * @ubi: UBI device object
- * @pnum: PEB number
- */
-static int get_ec(struct ubi_device *ubi, int pnum)
-{
-	struct ubi_wl_entry *e;
-
-	e = ubi->lookuptbl[pnum];
-
-	/* can this really happen? */
-	if (!e)
-		return ubi->mean_ec ?: 1;
-	else
-		return e->ec;
-}
-
-/**
  * ubi_update_fastmap - will be called by UBI if a volume changes or
  * a fastmap pool becomes full.
  * @ubi: UBI device object
@@ -1217,6 +1199,7 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 {
 	int ret, i;
 	struct ubi_fastmap_layout *new_fm, *old_fm;
+	struct ubi_wl_entry *tmp_e;
 
 	if (ubi->ro_mode)
 		return 0;
@@ -1252,12 +1235,12 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 	ubi->fm = NULL;
 
 	spin_lock(&ubi->wl_lock);
-	new_fm->e[0]->pnum = ubi_wl_get_fm_peb(ubi, UBI_FM_MAX_START);
+	tmp_e = ubi_wl_get_fm_peb(ubi, UBI_FM_MAX_START);
 	spin_unlock(&ubi->wl_lock);
 
 	if (old_fm) {
 		/* no fresh early PEB was found, reuse the old one */
-		if (new_fm->e[0]->pnum < 0) {
+		if (!tmp_e) {
 			struct ubi_ec_hdr *ec_hdr;
 			long long ec;
 
@@ -1309,6 +1292,7 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 			}
 
 			new_fm->e[0]->pnum = old_fm->e[0]->pnum;
+			new_fm->e[0]->ec = old_fm->e[0]->ec;
 		} else {
 			/* we've got a new early PEB, return the old one */
 			ubi_wl_put_fm_peb(ubi, old_fm->e[0], 0);
@@ -1318,7 +1302,7 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 		for (i = 1; i < old_fm->used_blocks; i++)
 			ubi_wl_put_fm_peb(ubi, old_fm->e[i], 0);
 	} else {
-		if (new_fm->e[0]->pnum < 0) {
+		if (!tmp_e) {
 			ubi_err("could not find an early PEB");
 			ret = -ENOSPC;
 
@@ -1326,7 +1310,8 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 		}
 	}
 
-	new_fm->e[0]->ec = get_ec(ubi, new_fm->e[0]->pnum);
+	new_fm->e[0]->pnum = tmp_e->pnum;
+	new_fm->e[0]->ec = tmp_e->ec;
 
 	if (new_fm->used_blocks > UBI_FM_MAX_BLOCKS) {
 		ubi_err("fastmap too large");
@@ -1340,10 +1325,10 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 
 	for (i = 1; i < new_fm->used_blocks; i++) {
 		spin_lock(&ubi->wl_lock);
-		new_fm->e[i]->pnum = ubi_wl_get_fm_peb(ubi, -1);
+		tmp_e = ubi_wl_get_fm_peb(ubi, -1);
 		spin_unlock(&ubi->wl_lock);
 
-		if (new_fm->e[i]->pnum < 0) {
+		if (!new_fm->e[i]) {
 			ubi_err("could not get any free erase block");
 
 			while (i--) {
@@ -1355,7 +1340,8 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 			goto err;
 		}
 
-		new_fm->e[i]->ec = get_ec(ubi, new_fm->e[i]->pnum);
+		new_fm->e[i]->pnum = tmp_e->pnum;
+		new_fm->e[i]->ec = tmp_e->ec;
 	}
 
 	if (old_fm) {
diff --git a/drivers/mtd/ubi/ubi.h b/drivers/mtd/ubi/ubi.h
index 0db97b0..be1933b 100644
--- a/drivers/mtd/ubi/ubi.h
+++ b/drivers/mtd/ubi/ubi.h
@@ -730,7 +730,7 @@ int ubi_wl_scrub_peb(struct ubi_device *ubi, int pnum);
 int ubi_wl_init(struct ubi_device *ubi, struct ubi_attach_info *ai);
 void ubi_wl_close(struct ubi_device *ubi);
 int ubi_thread(void *u);
-int ubi_wl_get_fm_peb(struct ubi_device *ubi, int max_pnum);
+struct ubi_wl_entry *ubi_wl_get_fm_peb(struct ubi_device *ubi, int max_pnum);
 int ubi_wl_put_fm_peb(struct ubi_device *ubi, struct ubi_wl_entry *used_e, int torture);
 
 /* io.c */
diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index 52548e7..15e4895 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -441,10 +441,9 @@ static struct ubi_wl_entry *find_early_wl_entry(struct rb_root *root,
  * If max_pnum is negative a PEB with a mean EC will be selected.
  * Must be called with wl_lock held!
  */
-int ubi_wl_get_fm_peb(struct ubi_device *ubi, int max_pnum)
+struct ubi_wl_entry *ubi_wl_get_fm_peb(struct ubi_device *ubi, int max_pnum)
 {
-	int ret = -ENOSPC;
-	struct ubi_wl_entry *e;
+	struct ubi_wl_entry *e = NULL;
 
 	if (!ubi->free.rb_node) {
 		ubi_err("no free eraseblocks");
@@ -461,13 +460,12 @@ int ubi_wl_get_fm_peb(struct ubi_device *ubi, int max_pnum)
 		goto out;
 
 	self_check_in_wl_tree(ubi, e, &ubi->free);
-	ret = e->pnum;
 
 	/* remove it from the free list,
 	 * the wl subsystem does no longer know this erase block */
 	rb_erase(&e->u.rb, &ubi->free);
 out:
-	return ret;
+	return e;
 }
 
 /**
-- 
1.7.6.5

[PATCH 10/21] UBI: Fastmap: Detect whether a PEB has to be unmapped.

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |   35 ++++++++++++++++++++++++++++++++++-
 1 files changed, 34 insertions(+), 1 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index fb5dae5..b42f81c 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -313,6 +313,34 @@ static int process_pool_aeb(struct ubi_device *ubi, struct ubi_attach_info *ai,
 }
 
 /**
+ * unmap_peb - unmap a PEB.
+ * If fastmap detects a free PEB in the pool it has to check whether
+ * this PEB has been unmapped after writing the fastmap.
+ *
+ * @ubi: UBI device object
+ * @pnum: The PEB to be unmapped
+ */
+static void unmap_peb(struct ubi_attach_info *ai, int pnum)
+{
+	struct ubi_ainf_volume *av;
+	struct rb_node *node, *node2;
+	struct ubi_ainf_peb *aeb;
+
+	for (node = rb_first(&ai->volumes); node; node = rb_next(node)) {
+		av = rb_entry(node, struct ubi_ainf_volume, rb);
+
+		for (node2 = rb_first(&av->root); node2; node2 = rb_next(node2)) {
+			aeb = rb_entry(node2, struct ubi_ainf_peb, u.rb);
+			if (aeb->pnum == pnum) {
+				rb_erase(&aeb->u.rb, &av->root);
+				kmem_cache_free(ai->aeb_slab_cache, aeb);
+				return;
+			}
+		}
+	}
+}
+
+/**
  * scan_pool - scans a pool for changed (no longer empty PEBs)
  * @ubi: UBI device object
  * @ai: attach info object
@@ -378,7 +406,12 @@ static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 
 		err = ubi_io_read_vid_hdr(ubi, pnum, vh, 0);
 		if (err == UBI_IO_FF || err == UBI_IO_FF_BITFLIPS) {
-			add_aeb(ai, &ai->free, pnum, be64_to_cpu(ech->ec), 1);
+			unmap_peb(ai, pnum);
+			ubi_msg("Adding PEB to free: %i", pnum);
+			if (err == UBI_IO_FF_BITFLIPS)
+				add_aeb(ai, &ai->free, pnum, be64_to_cpu(ech->ec), 1);
+			else
+				add_aeb(ai, &ai->free, pnum, be64_to_cpu(ech->ec), 0);
 			continue;
 		} else if (err == 0 || err == UBI_IO_BITFLIPS) {
 			dbg_bld("Found non empty PEB:%i in pool", pnum);
-- 
1.7.6.5

[PATCH 11/21] UBI: Fastmap: Adjust comment above ubi_wl_flush()

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index b42f81c..20b016e 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -1387,8 +1387,8 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 	/* Ensure that the PEBs of the old fastmap got erased and added to the
 	 * free list before we write the fastmap. Otherwise fastmp does not
 	 * see these PEBs and we leak them.
-	 * FIXME: Rewrite ubi_wl_flush() such that we can flush only the
-	 * erase work instead of all work.
+	 * We need the flush also to ensure that no to be scrubbed PEBs are in
+	 * flight.
 	 */
 	ubi_wl_flush(ubi);
 
-- 
1.7.6.5

[PATCH 12/21] UBI: Fastmap: Handle unused "used" PEBs better

[Richard Weinberger]

If fastmap detects that some PEBs in the used list are no longer
used it has to erase them.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 20b016e..1c2e906 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -712,6 +712,13 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 	if (max_sqnum > ai->max_sqnum)
 		ai->max_sqnum = max_sqnum;
 
+	list_for_each_entry_safe(tmp_aeb, _tmp_aeb, &used, u.list) {
+		list_del(&tmp_aeb->u.list);
+		dbg_bld("adding PEB from used to erase list: %i", tmp_aeb->pnum);
+		add_aeb(ai, &ai->erase, tmp_aeb->pnum, tmp_aeb->ec, 0);
+		kmem_cache_free(ai->aeb_slab_cache, tmp_aeb);
+	}
+
 	return 0;
 
 fail_bad:
-- 
1.7.6.5

[PATCH 13/21] UBI: Fastmap: Introduce WL pool Signed-off-by: Richard Weinberger <richard@nod.at>

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/build.c     |    4 ++
 drivers/mtd/ubi/fastmap.c   |   51 ++++++++++++++++----------
 drivers/mtd/ubi/ubi-media.h |    2 +
 drivers/mtd/ubi/ubi.h       |    1 +
 drivers/mtd/ubi/wl.c        |   82 ++++++++++++++++++++++++++++++++++++++----
 5 files changed, 112 insertions(+), 28 deletions(-)

diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
index 2acfe5b..0ad6789 100644
--- a/drivers/mtd/ubi/build.c
+++ b/drivers/mtd/ubi/build.c
@@ -887,6 +887,7 @@ int ubi_attach_mtd_dev(struct mtd_info *mtd, int ubi_num, int vid_hdr_offset)
 	ubi->autoresize_vol_id = -1;
 
 	ubi->fm_pool.used = ubi->fm_pool.size = 0;
+	ubi->fm_wl_pool.used = ubi->fm_wl_pool.size = 0;
 
 	/*
 	 * fm_pool.max_size is 5% of the total number of PEBs but it's also
@@ -897,7 +898,10 @@ int ubi_attach_mtd_dev(struct mtd_info *mtd, int ubi_num, int vid_hdr_offset)
 	if (ubi->fm_pool.max_size < UBI_FM_MIN_POOL_SIZE)
 		ubi->fm_pool.max_size = UBI_FM_MIN_POOL_SIZE;
 
+	ubi->fm_wl_pool.max_size = UBI_FM_WL_POOL_SIZE;
+
 	ubi_msg("fastmap pool size: %d", ubi->fm_pool.max_size);
+	ubi_msg("fastmap WL pool size: %d", ubi->fm_wl_pool.max_size);
 
 	mutex_init(&ubi->buf_mutex);
 	mutex_init(&ubi->ckvol_mutex);
diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 1c2e906..29e6d69 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -500,7 +500,7 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 
 	struct ubi_fm_sb *fmsb;
 	struct ubi_fm_hdr *fmhdr;
-	struct ubi_fm_scan_pool *fmpl;
+	struct ubi_fm_scan_pool *fmpl1, *fmpl2;
 	struct ubi_fm_ec *fmec;
 	struct ubi_fm_volhdr *fmvhdr;
 	struct ubi_fm_eba *fm_eba;
@@ -547,11 +547,18 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 	if (fmhdr->magic != UBI_FM_HDR_MAGIC)
 		goto fail_bad;
 
-	fmpl = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
-	fm_pos += sizeof(*fmpl);
+	fmpl1 = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
+	fm_pos += sizeof(*fmpl1);
 	if (fm_pos >= fm_size)
 		goto fail_bad;
-	if (fmpl->magic != UBI_FM_POOL_MAGIC)
+	if (fmpl1->magic != UBI_FM_POOL_MAGIC)
+		goto fail_bad;
+
+	fmpl2 = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
+	fm_pos += sizeof(*fmpl2);
+	if (fm_pos >= fm_size)
+		goto fail_bad;
+	if (fmpl2->magic != UBI_FM_POOL_MAGIC)
 		goto fail_bad;
 
 	/* read EC values from free list */
@@ -694,20 +701,15 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 		kfree(ech);
 	}
 
-	/*
-	 * The remainning PEBs in the used list are not used.
-	 * They lived in the fastmap pool but got never used.
-	 */
-	list_for_each_entry_safe(tmp_aeb, _tmp_aeb, &used, u.list) {
-		list_del(&tmp_aeb->u.list);
-		list_add_tail(&tmp_aeb->u.list, &ai->free);
-	}
-
-	ret = scan_pool(ubi, ai, fmpl->pebs, be32_to_cpu(fmpl->size),
+	ret = scan_pool(ubi, ai, fmpl1->pebs, be32_to_cpu(fmpl1->size),
 		&max_sqnum, &eba_orphans);
 	if (ret)
 		goto fail;
 
+	ret = scan_pool(ubi, ai, fmpl2->pebs, be32_to_cpu(fmpl2->size),
+		&max_sqnum, &eba_orphans);
+	if (ret)
+		goto fail;
 
 	if (max_sqnum > ai->max_sqnum)
 		ai->max_sqnum = max_sqnum;
@@ -1024,7 +1026,7 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 	char *fm_raw;
 	struct ubi_fm_sb *fmsb;
 	struct ubi_fm_hdr *fmh;
-	struct ubi_fm_scan_pool *fmpl;
+	struct ubi_fm_scan_pool *fmpl1, *fmpl2;
 	struct ubi_fm_ec *fec;
 	struct ubi_fm_volhdr *fvh;
 	struct ubi_fm_eba *feba;
@@ -1100,13 +1102,21 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 	used_peb_count = 0;
 	vol_count = 0;
 
-	fmpl = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
-	fm_pos += sizeof(*fmpl);
-	fmpl->magic = UBI_FM_POOL_MAGIC;
-	fmpl->size = cpu_to_be32(ubi->fm_pool.size);
+	fmpl1 = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
+	fm_pos += sizeof(*fmpl1);
+	fmpl1->magic = UBI_FM_POOL_MAGIC;
+	fmpl1->size = cpu_to_be32(ubi->fm_pool.size);
 
 	for (i = 0; i < ubi->fm_pool.size; i++)
-		fmpl->pebs[i] = cpu_to_be32(ubi->fm_pool.pebs[i]);
+		fmpl1->pebs[i] = cpu_to_be32(ubi->fm_pool.pebs[i]);
+
+	fmpl2 = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
+	fm_pos += sizeof(*fmpl2);
+	fmpl2->magic = UBI_FM_POOL_MAGIC;
+	fmpl2->size = cpu_to_be32(ubi->fm_wl_pool.size);
+
+	for (i = 0; i < ubi->fm_wl_pool.size; i++)
+		fmpl2->pebs[i] = cpu_to_be32(ubi->fm_wl_pool.pebs[i]);
 
 	for (node = rb_first(&ubi->free); node; node = rb_next(node)) {
 		wl_e = rb_entry(node, struct ubi_wl_entry, u.rb);
@@ -1250,6 +1260,7 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 
 	new_fm->size = sizeof(struct ubi_fm_hdr) + \
 			sizeof(struct ubi_fm_scan_pool) + \
+			sizeof(struct ubi_fm_scan_pool) + \
 			(ubi->peb_count * sizeof(struct ubi_fm_ec)) + \
 			(sizeof(struct ubi_fm_eba) + \
 			(ubi->peb_count * sizeof(__be32))) + \
diff --git a/drivers/mtd/ubi/ubi-media.h b/drivers/mtd/ubi/ubi-media.h
index b4ecd1b..a36748c 100644
--- a/drivers/mtd/ubi/ubi-media.h
+++ b/drivers/mtd/ubi/ubi-media.h
@@ -403,6 +403,8 @@ struct ubi_vtbl_record {
 #define UBI_FM_MIN_POOL_SIZE	8
 #define UBI_FM_MAX_POOL_SIZE	256
 
+#define UBI_FM_WL_POOL_SIZE	25
+
 /**
  * struct ubi_fm_sb - UBI fastmap super block
  * @magic: fastmap super block magic number (%UBI_FM_SB_MAGIC)
diff --git a/drivers/mtd/ubi/ubi.h b/drivers/mtd/ubi/ubi.h
index be1933b..ccd0da7 100644
--- a/drivers/mtd/ubi/ubi.h
+++ b/drivers/mtd/ubi/ubi.h
@@ -480,6 +480,7 @@ struct ubi_device {
 	/* Fastmap stuff */
 	struct ubi_fastmap_layout *fm;
 	struct ubi_fm_pool fm_pool;
+	struct ubi_fm_pool fm_wl_pool;
 	struct mutex fm_mutex;
 	struct mutex fm_pool_mutex;
 	int attached_by_scanning;
diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index 15e4895..8fb8b41 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -507,7 +507,6 @@ retry:
 	 */
 	rb_erase(&e->u.rb, &ubi->free);
 	dbg_wl("PEB %d EC %d", e->pnum, e->ec);
-	prot_queue_add(ubi, e);
 	spin_unlock(&ubi->wl_lock);
 
 	err = ubi_self_check_all_ff(ubi, e->pnum, ubi->vid_hdr_aloffset,
@@ -520,6 +519,44 @@ retry:
 	return e->pnum;
 }
 
+static int refill_wl_pool(struct ubi_device *ubi)
+{
+	int ret, i;
+	struct ubi_fm_pool *pool = &ubi->fm_wl_pool;
+	struct ubi_wl_entry *e;
+
+	spin_lock(&ubi->wl_lock);
+	if (pool->used != pool->size && pool->size) {
+		spin_unlock(&ubi->wl_lock);
+		return 0;
+	}
+
+	for (i = 0; i < pool->max_size; i++) {
+		if (!ubi->free.rb_node) {
+			spin_unlock(&ubi->wl_lock);
+			break;
+		}
+
+		e = find_wl_entry(&ubi->free, WL_FREE_MAX_DIFF);
+		self_check_in_wl_tree(ubi, e, &ubi->free);
+		rb_erase(&e->u.rb, &ubi->free);
+
+		pool->pebs[i] = e->pnum;
+	}
+	pool->size = i;
+	spin_unlock(&ubi->wl_lock);
+
+	ret = ubi_update_fastmap(ubi);
+	if (ret) {
+		ubi_ro_mode(ubi);
+
+		return ret > 0 ? -EINVAL : ret;
+	}
+	pool->used = 0;
+
+	return pool->size ? 0 : -ENOSPC;
+}
+
 /* ubi_wl_get_peb - works exaclty like __ubi_wl_get_peb but keeps track of
  * the fastmap pool.
  */
@@ -530,6 +567,8 @@ int ubi_wl_get_peb(struct ubi_device *ubi)
 
 	mutex_lock(&ubi->fm_pool_mutex);
 
+	refill_wl_pool(ubi);
+
 	/* pool contains no free blocks, create a new one
 	 * and write a fastmap */
 	if (pool->used == pool->size || !pool->size) {
@@ -549,13 +588,37 @@ int ubi_wl_get_peb(struct ubi_device *ubi)
 			return ret > 0 ? -EINVAL : ret;
 		}
 	}
-	mutex_unlock(&ubi->fm_pool_mutex);
 
 	/* we got not a single free PEB */
 	if (!pool->size)
-		return -ENOSPC;
+		ret = -ENOSPC;
+	else {
+		spin_lock(&ubi->wl_lock);
+		ret = pool->pebs[pool->used++];
+		prot_queue_add(ubi, ubi->lookuptbl[ret]);
+		spin_unlock(&ubi->wl_lock);
+	}
+
+	mutex_unlock(&ubi->fm_pool_mutex);
 
-	return pool->pebs[pool->used++];
+	return ret;
+}
+
+/* get_peb_for_wl - returns a PEB to be used internally by the WL sub-system
+ *
+ * @ubi: UBI device description object
+ */
+static struct ubi_wl_entry *get_peb_for_wl(struct ubi_device *ubi)
+{
+	struct ubi_fm_pool *pool = &ubi->fm_wl_pool;
+	int pnum;
+
+	if (pool->used == pool->size || !pool->size) {
+		return NULL;
+	} else {
+		pnum = pool->pebs[pool->used++];
+		return ubi->lookuptbl[pnum];
+	}
 }
 
 /**
@@ -830,7 +893,9 @@ static int wear_leveling_worker(struct ubi_device *ubi, struct ubi_work *wrk,
 		 * counters differ much enough, start wear-leveling.
 		 */
 		e1 = rb_entry(rb_first(&ubi->used), struct ubi_wl_entry, u.rb);
-		e2 = find_wl_entry(&ubi->free, WL_FREE_MAX_DIFF);
+		e2 = get_peb_for_wl(ubi);
+		if (!e2)
+			goto out_cancel;
 
 		if (!(e2->ec - e1->ec >= UBI_WL_THRESHOLD)) {
 			dbg_wl("no WL needed: min used EC %d, max free EC %d",
@@ -845,14 +910,15 @@ static int wear_leveling_worker(struct ubi_device *ubi, struct ubi_work *wrk,
 		/* Perform scrubbing */
 		scrubbing = 1;
 		e1 = rb_entry(rb_first(&ubi->scrub), struct ubi_wl_entry, u.rb);
-		e2 = find_wl_entry(&ubi->free, WL_FREE_MAX_DIFF);
+		e2 = get_peb_for_wl(ubi);
+		if (!e2)
+			goto out_cancel;
+
 		self_check_in_wl_tree(ubi, e1, &ubi->scrub);
 		rb_erase(&e1->u.rb, &ubi->scrub);
 		dbg_wl("scrub PEB %d to PEB %d", e1->pnum, e2->pnum);
 	}
 
-	self_check_in_wl_tree(ubi, e2, &ubi->free);
-	rb_erase(&e2->u.rb, &ubi->free);
 	ubi->move_from = e1;
 	ubi->move_to = e2;
 	spin_unlock(&ubi->wl_lock);
-- 
1.7.6.5

Re: [PATCH 13/21] UBI: Fastmap: Introduce WL pool Signed-off-by: Richard Weinberger <richard@nod.at>

[Artem Bityutskiy]

[-- Attachment #1: Type: text/plain, Size: 790 bytes --]

On Wed, 2012-06-13 at 12:42 +0200, Richard Weinberger wrote:
> Signed-off-by: Richard Weinberger <richard@nod.at>
> ---
>  drivers/mtd/ubi/build.c     |    4 ++
>  drivers/mtd/ubi/fastmap.c   |   51 ++++++++++++++++----------
>  drivers/mtd/ubi/ubi-media.h |    2 +
>  drivers/mtd/ubi/ubi.h       |    1 +
>  drivers/mtd/ubi/wl.c        |   82 ++++++++++++++++++++++++++++++++++++++----
>  5 files changed, 112 insertions(+), 28 deletions(-)

Hi, I've rebased the fastmap branch against v3.5-rc4, and your patches
do not apply cleanly anymore, because there are collisions with Joel's
changes which were merged recently. I've applied everything up to this
patch and pushed out. Could you please re-base and re-send your queue?
Thanks!

-- 
Best Regards,
Artem Bityutskiy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 13/21] UBI: Fastmap: Introduce WL pool Signed-off-by: Richard Weinberger <richard@nod.at>

[Richard Weinberger]

[-- Attachment #1: Type: text/plain, Size: 1313 bytes --]

Am 27.06.2012 06:13, schrieb Artem Bityutskiy:
> On Wed, 2012-06-13 at 12:42 +0200, Richard Weinberger wrote:
>> Signed-off-by: Richard Weinberger <richard@nod.at>
>> ---
>>  drivers/mtd/ubi/build.c     |    4 ++
>>  drivers/mtd/ubi/fastmap.c   |   51 ++++++++++++++++----------
>>  drivers/mtd/ubi/ubi-media.h |    2 +
>>  drivers/mtd/ubi/ubi.h       |    1 +
>>  drivers/mtd/ubi/wl.c        |   82 ++++++++++++++++++++++++++++++++++++++----
>>  5 files changed, 112 insertions(+), 28 deletions(-)
> 
> Hi, I've rebased the fastmap branch against v3.5-rc4, and your patches
> do not apply cleanly anymore, because there are collisions with Joel's
> changes which were merged recently. I've applied everything up to this
> patch and pushed out. Could you please re-base and re-send your queue?
> Thanks!
> 

I'm confused.
schedule_erase() needs now vol_id and lnum as parameters but
the erase worker does not use them at all.
Am I missing something?

Fastmap is using schedule_erase() in ubi_wl_put_fm_peb() too,
but ubi_wl_put_fm_peb() does not know anything about volume IDs or
LEBs.
It's used to return PEBs used by fastmap itself.
Fastmap's volume headers have always lnum = 0 and
vol_id = UBI_FM_SB_VOLUME_ID or UBI_FM_DATA_VOLUME_ID.
Is this a problem?

Thanks,
//richard


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [PATCH 13/21] UBI: Fastmap: Introduce WL pool Signed-off-by: Richard Weinberger <richard@nod.at>

[Artem Bityutskiy]

[-- Attachment #1: Type: text/plain, Size: 353 bytes --]

On Wed, 2012-06-27 at 10:20 +0200, Richard Weinberger wrote:
> I'm confused.
> schedule_erase() needs now vol_id and lnum as parameters but
> the erase worker does not use them at all.
> Am I missing something?

The wl flush function needs them to force erasure of a all PEBs
belonging to a specific LEB.

-- 
Best Regards,
Artem Bityutskiy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 14/21] UBI: Fastmap: Erase PEBs synchronous in wear_leveling_worker()

[Richard Weinberger]

Fastmap calls ubi_wl_flush() to ensure that no PEBs are in flight.
But if ubi_wl_flush() runs the WL-worker the worker may inject
new erase work which runs later.
This PEBs are not visible to fastmap and leak.
So solve this erase PEBs synchronous in wear_leveling_worker().

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/wl.c |   23 ++++++++++++++++++++---
 1 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index 8fb8b41..def6a88 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -804,6 +804,23 @@ static int schedule_erase(struct ubi_device *ubi, struct ubi_wl_entry *e,
 	return 0;
 }
 
+static int do_sync_erase(struct ubi_device *ubi, struct ubi_wl_entry *e,
+			 int torture)
+{
+	struct ubi_work *wl_wrk;
+
+	ubi_msg("sync erase of PEB %i", e->pnum);
+
+	wl_wrk = kmalloc(sizeof(struct ubi_work), GFP_NOFS);
+	if (!wl_wrk)
+		return -ENOMEM;
+
+	wl_wrk->e = e;
+	wl_wrk->torture = torture;
+
+	return erase_worker(ubi, wl_wrk, 0);
+}
+
 /**
  * ubi_wl_put_fm_peb - returns a PEB used in a fastmap to the wear-leveling
  * sub-system.
@@ -1035,7 +1052,7 @@ static int wear_leveling_worker(struct ubi_device *ubi, struct ubi_work *wrk,
 	ubi->move_to_put = ubi->wl_scheduled = 0;
 	spin_unlock(&ubi->wl_lock);
 
-	err = schedule_erase(ubi, e1, 0);
+	err = do_sync_erase(ubi, e1, 0);
 	if (err) {
 		kmem_cache_free(ubi_wl_entry_slab, e1);
 		if (e2)
@@ -1050,7 +1067,7 @@ static int wear_leveling_worker(struct ubi_device *ubi, struct ubi_work *wrk,
 		 */
 		dbg_wl("PEB %d (LEB %d:%d) was put meanwhile, erase",
 		       e2->pnum, vol_id, lnum);
-		err = schedule_erase(ubi, e2, 0);
+		err = do_sync_erase(ubi, e2, 0);
 		if (err) {
 			kmem_cache_free(ubi_wl_entry_slab, e2);
 			goto out_ro;
@@ -1089,7 +1106,7 @@ out_not_moved:
 	spin_unlock(&ubi->wl_lock);
 
 	ubi_free_vid_hdr(ubi, vid_hdr);
-	err = schedule_erase(ubi, e2, torture);
+	err = do_sync_erase(ubi, e2, torture);
 	if (err) {
 		kmem_cache_free(ubi_wl_entry_slab, e2);
 		goto out_ro;
-- 
1.7.6.5

[PATCH 15/21] UBI: Fastmap: Fix NULL-pointer bug

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |    4 ++--
 drivers/mtd/ubi/wl.c      |    3 ---
 2 files changed, 2 insertions(+), 5 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 29e6d69..65cb3f1 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -1342,8 +1342,8 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 				goto err;
 			}
 
-			new_fm->e[0]->pnum = old_fm->e[0]->pnum;
-			new_fm->e[0]->ec = old_fm->e[0]->ec;
+			tmp_e->pnum = old_fm->e[0]->pnum;
+			tmp_e->ec = old_fm->e[0]->ec;
 		} else {
 			/* we've got a new early PEB, return the old one */
 			ubi_wl_put_fm_peb(ubi, old_fm->e[0], 0);
diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index def6a88..004ce89 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -1251,9 +1251,6 @@ static int erase_worker(struct ubi_device *ubi, struct ubi_work *wl_wrk,
 		 * protected physical eraseblocks.
 		 */
 		serve_prot_queue(ubi);
-
-		/* And take care about wear-leveling */
-		err = ensure_wear_leveling(ubi);
 		return err;
 	}
 
-- 
1.7.6.5

[PATCH 16/21] UBI: Fastmap: Fix loglevel

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 65cb3f1..4a16570 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -407,7 +407,7 @@ static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 		err = ubi_io_read_vid_hdr(ubi, pnum, vh, 0);
 		if (err == UBI_IO_FF || err == UBI_IO_FF_BITFLIPS) {
 			unmap_peb(ai, pnum);
-			ubi_msg("Adding PEB to free: %i", pnum);
+			dbg_bld("Adding PEB to free: %i", pnum);
 			if (err == UBI_IO_FF_BITFLIPS)
 				add_aeb(ai, &ai->free, pnum, be64_to_cpu(ech->ec), 1);
 			else
-- 
1.7.6.5

[PATCH 17/21] UBI: Fastmap: Relax EC self check

[Richard Weinberger]

Using fastmap the EC value can be differ by 1.
FIXME: Find a way to solve this issue

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/wl.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index 004ce89..c4d8273 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -1821,7 +1821,7 @@ static int self_check_ec(struct ubi_device *ubi, int pnum, int ec)
 	}
 
 	read_ec = be64_to_cpu(ec_hdr->ec);
-	if (ec != read_ec) {
+	if (ec != read_ec && read_ec - ec > 1) {
 		ubi_err("self-check failed for PEB %d", pnum);
 		ubi_err("read EC is %lld, should be %d", read_ec, ec);
 		dump_stack();
-- 
1.7.6.5

[PATCH 18/21] UBI: Fastmap: Store magic values in be32

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |   24 ++++++++++++------------
 1 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 4a16570..446dc0e 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -544,21 +544,21 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 	if (fm_pos >= fm_size)
 		goto fail_bad;
 
-	if (fmhdr->magic != UBI_FM_HDR_MAGIC)
+	if (be32_to_cpu(fmhdr->magic) != UBI_FM_HDR_MAGIC)
 		goto fail_bad;
 
 	fmpl1 = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
 	fm_pos += sizeof(*fmpl1);
 	if (fm_pos >= fm_size)
 		goto fail_bad;
-	if (fmpl1->magic != UBI_FM_POOL_MAGIC)
+	if (be32_to_cpu(fmpl1->magic) != UBI_FM_POOL_MAGIC)
 		goto fail_bad;
 
 	fmpl2 = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
 	fm_pos += sizeof(*fmpl2);
 	if (fm_pos >= fm_size)
 		goto fail_bad;
-	if (fmpl2->magic != UBI_FM_POOL_MAGIC)
+	if (be32_to_cpu(fmpl2->magic) != UBI_FM_POOL_MAGIC)
 		goto fail_bad;
 
 	/* read EC values from free list */
@@ -593,7 +593,7 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 		if (fm_pos >= fm_size)
 			goto fail_bad;
 
-		if (fmvhdr->magic != UBI_FM_VHDR_MAGIC)
+		if (be32_to_cpu(fmvhdr->magic) != UBI_FM_VHDR_MAGIC)
 			goto fail_bad;
 
 		av = add_vol(ai, be32_to_cpu(fmvhdr->vol_id),
@@ -614,7 +614,7 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 		if (fm_pos >= fm_size)
 			goto fail_bad;
 
-		if (fm_eba->magic != UBI_FM_EBA_MAGIC)
+		if (be32_to_cpu(fm_eba->magic) != UBI_FM_EBA_MAGIC)
 			goto fail_bad;
 
 		for (j = 0; j < be32_to_cpu(fm_eba->reserved_pebs); j++) {
@@ -831,7 +831,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	/* TODO: please, use 'be32_to_cpu()' _every_ time you access a __be32 /
 	 * etc field. Please, look how things are done in io.c. Please, check
 	 * and fix globally. */
-	if (fmsb->magic != UBI_FM_SB_MAGIC) {
+	if (be32_to_cpu(fmsb->magic) != UBI_FM_SB_MAGIC) {
 		/* TODO: not urgent, but examine all the error messages and
 		 * print more information there. Here you should print what was
 		 * read and what was expected. See io.c and do similarly or
@@ -1091,20 +1091,20 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 	fm_pos += sizeof(*fmh);
 	ubi_assert(fm_pos <= new_fm->size);
 
-	fmsb->magic = UBI_FM_SB_MAGIC;
+	fmsb->magic = cpu_to_be32(UBI_FM_SB_MAGIC);
 	fmsb->version = UBI_FM_FMT_VERSION;
 	fmsb->used_blocks = cpu_to_be32(new_fm->used_blocks);
 	/* the max sqnum will be filled in while *reading* the fastmap */
 	fmsb->sqnum = 0;
 
-	fmh->magic = UBI_FM_HDR_MAGIC;
+	fmh->magic = cpu_to_be32(UBI_FM_HDR_MAGIC);
 	free_peb_count = 0;
 	used_peb_count = 0;
 	vol_count = 0;
 
 	fmpl1 = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
 	fm_pos += sizeof(*fmpl1);
-	fmpl1->magic = UBI_FM_POOL_MAGIC;
+	fmpl1->magic = cpu_to_be32(UBI_FM_POOL_MAGIC);
 	fmpl1->size = cpu_to_be32(ubi->fm_pool.size);
 
 	for (i = 0; i < ubi->fm_pool.size; i++)
@@ -1112,7 +1112,7 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 
 	fmpl2 = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
 	fm_pos += sizeof(*fmpl2);
-	fmpl2->magic = UBI_FM_POOL_MAGIC;
+	fmpl2->magic = cpu_to_be32(UBI_FM_POOL_MAGIC);
 	fmpl2->size = cpu_to_be32(ubi->fm_wl_pool.size);
 
 	for (i = 0; i < ubi->fm_wl_pool.size; i++)
@@ -1156,7 +1156,7 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 		fm_pos += sizeof(*fvh);
 		ubi_assert(fm_pos <= new_fm->size);
 
-		fvh->magic = UBI_FM_VHDR_MAGIC;
+		fvh->magic = cpu_to_be32(UBI_FM_VHDR_MAGIC);
 		fvh->vol_id = cpu_to_be32(vol->vol_id);
 		fvh->vol_type = vol->vol_type;
 		fvh->used_ebs = cpu_to_be32(vol->used_ebs);
@@ -1174,7 +1174,7 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 			feba->pnum[j] = cpu_to_be32(vol->eba_tbl[j]);
 
 		feba->reserved_pebs = cpu_to_be32(j);
-		feba->magic = UBI_FM_EBA_MAGIC;
+		feba->magic = cpu_to_be32(UBI_FM_EBA_MAGIC);
 	}
 	fmh->vol_count = cpu_to_be32(vol_count);
 	fmh->bad_peb_count = cpu_to_be32(ubi->bad_peb_count);
-- 
1.7.6.5

[PATCH 19/21] UBI: Fastmap: Minor fixes

[Richard Weinberger]

Fix some TODOs

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |  124 +++++++++++----------------------------------
 1 files changed, 29 insertions(+), 95 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 446dc0e..8b09c29 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -329,7 +329,8 @@ static void unmap_peb(struct ubi_attach_info *ai, int pnum)
 	for (node = rb_first(&ai->volumes); node; node = rb_next(node)) {
 		av = rb_entry(node, struct ubi_ainf_volume, rb);
 
-		for (node2 = rb_first(&av->root); node2; node2 = rb_next(node2)) {
+		for (node2 = rb_first(&av->root); node2;
+		     node2 = rb_next(node2)) {
 			aeb = rb_entry(node2, struct ubi_ainf_peb, u.rb);
 			if (aeb->pnum == pnum) {
 				rb_erase(&aeb->u.rb, &av->root);
@@ -356,11 +357,7 @@ static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 	struct ubi_vid_hdr *vh;
 	struct ubi_ec_hdr *ech;
 	struct ubi_ainf_peb *new_aeb, *tmp_aeb;
-	int i;
-	int pnum;
-	int err;
-	int ret = 0;
-	int found_orphan;
+	int i, pnum, err, found_orphan, ret = 0;
 
 	ech = kzalloc(ubi->ec_hdr_alsize, GFP_KERNEL);
 	if (!ech)
@@ -386,7 +383,6 @@ static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 		if (ubi_io_is_bad(ubi, pnum)) {
 			dbg_bld("bad PEB in fastmap pool!");
 			ret = UBI_BAD_FASTMAP;
-
 			goto out;
 		}
 
@@ -431,17 +427,10 @@ static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 				list_del(&tmp_aeb->u.list);
 			}
 
-			/* TODO: could you please follow UBI style of how we
-			 * split lines? Notice that we aling arguments WRT the
-			 * bracket. We use few tabs, and then at the and use
-			 * few spaces for fine-tuning the alignment. Please,
-			 * let's be consistent. Take a look at any UBI file for
-			 * example. */
 			new_aeb = kmem_cache_alloc(ai->aeb_slab_cache,
-				GFP_KERNEL);
+						   GFP_KERNEL);
 			if (!new_aeb) {
 				ret = -ENOMEM;
-
 				goto out;
 			}
 
@@ -458,14 +447,12 @@ static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 			err = process_pool_aeb(ubi, ai, vh, new_aeb);
 			if (err) {
 				ret = err > 0 ? UBI_BAD_FASTMAP : err;
-
 				goto out;
 			}
 		} else {
 			/* We are paranoid and fall back to scanning mode */
 			ubi_err("fastmap pool PEBs contains damaged PEBs!");
 			ret = err > 0 ? UBI_BAD_FASTMAP : err;
-
 			goto out;
 		}
 
@@ -474,7 +461,6 @@ static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 out:
 	ubi_free_vid_hdr(ubi, vh);
 	kfree(ech);
-
 	return ret;
 }
 
@@ -484,29 +470,20 @@ out:
  * @fm_raw: the fastmap it self as byte array
  * @fm_size: size of the fastmap in bytes
  */
-/* TODO: please, make all pointers like 'fm_raw' of type void. It is indeed a
- * pointer to data blob, it is not a pointer to a string of characters. Note,
- * that in pointer arithmetics void * is the same as char *. */
 static int ubi_attach_fastmap(struct ubi_device *ubi,
 			      struct ubi_attach_info *ai,
-			      char *fm_raw, size_t fm_size)
+			      void *fm_raw, size_t fm_size)
 {
-	struct list_head used;
-	struct list_head eba_orphans;
-	/* TODO: please, try to declare variables of the same time on one line */
+	struct list_head used, eba_orphans;
 	struct ubi_ainf_volume *av;
 	struct ubi_ainf_peb *aeb, *tmp_aeb, *_tmp_aeb;
 	struct ubi_ec_hdr *ech;
-
 	struct ubi_fm_sb *fmsb;
 	struct ubi_fm_hdr *fmhdr;
 	struct ubi_fm_scan_pool *fmpl1, *fmpl2;
 	struct ubi_fm_ec *fmec;
 	struct ubi_fm_volhdr *fmvhdr;
 	struct ubi_fm_eba *fm_eba;
-
-	/* TODO: no blank lines in the local variable declaration block
-	 * please. */
 	int ret, i, j;
 	size_t fm_pos = 0;
 	unsigned long long max_sqnum = 0;
@@ -521,11 +498,10 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 	ai->min_ec = UBI_MAX_ERASECOUNTER;
 
 	ai->aeb_slab_cache = kmem_cache_create("ubi_ainf_peb_slab",
-					      sizeof(struct ubi_ainf_peb),
-					      0, 0, NULL);
+					       sizeof(struct ubi_ainf_peb),
+					       0, 0, NULL);
 	if (!ai->aeb_slab_cache) {
 		ret = -ENOMEM;
-
 		goto fail;
 	}
 
@@ -597,9 +573,9 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 			goto fail_bad;
 
 		av = add_vol(ai, be32_to_cpu(fmvhdr->vol_id),
-			be32_to_cpu(fmvhdr->used_ebs),
-			be32_to_cpu(fmvhdr->data_pad),
-			fmvhdr->vol_type, be32_to_cpu(fmvhdr->last_eb_bytes));
+			     be32_to_cpu(fmvhdr->used_ebs),
+			     be32_to_cpu(fmvhdr->data_pad),
+			     fmvhdr->vol_type, be32_to_cpu(fmvhdr->last_eb_bytes));
 
 		if (!av)
 			goto fail_bad;
@@ -618,14 +594,14 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 			goto fail_bad;
 
 		for (j = 0; j < be32_to_cpu(fm_eba->reserved_pebs); j++) {
+			int pnum = be32_to_cpu(fm_eba->pnum[j]);
 
 			if ((int)be32_to_cpu(fm_eba->pnum[j]) < 0)
 				continue;
 
 			aeb = NULL;
 			list_for_each_entry(tmp_aeb, &used, u.list) {
-				if (tmp_aeb->pnum == \
-					be32_to_cpu(fm_eba->pnum[j]))
+				if (tmp_aeb->pnum == pnum)
 					aeb = tmp_aeb;
 			}
 
@@ -638,7 +614,7 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 			 */
 			if (!aeb) {
 				aeb = kmem_cache_alloc(ai->aeb_slab_cache,
-					GFP_KERNEL);
+						       GFP_KERNEL);
 				if (!aeb) {
 					ret = -ENOMEM;
 
@@ -649,9 +625,7 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 				aeb->pnum = be32_to_cpu(fm_eba->pnum[j]);
 				aeb->ec = -1;
 				aeb->scrub = aeb->copy_flag = aeb->sqnum = 0;
-
 				list_add_tail(&aeb->u.list, &eba_orphans);
-
 				continue;
 			}
 
@@ -669,18 +643,16 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 		ech = kzalloc(ubi->ec_hdr_alsize, GFP_KERNEL);
 		if (!ech) {
 			ret = -ENOMEM;
-
 			goto fail;
 		}
 
 		list_for_each_entry_safe(tmp_aeb, _tmp_aeb, &eba_orphans,
-			u.list) {
+					 u.list) {
 			int err;
 
 			if (ubi_io_is_bad(ubi, tmp_aeb->pnum)) {
 				ret = UBI_BAD_FASTMAP;
 				kfree(ech);
-
 				goto fail;
 			}
 
@@ -702,12 +674,12 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 	}
 
 	ret = scan_pool(ubi, ai, fmpl1->pebs, be32_to_cpu(fmpl1->size),
-		&max_sqnum, &eba_orphans);
+			&max_sqnum, &eba_orphans);
 	if (ret)
 		goto fail;
 
 	ret = scan_pool(ubi, ai, fmpl2->pebs, be32_to_cpu(fmpl2->size),
-		&max_sqnum, &eba_orphans);
+			&max_sqnum, &eba_orphans);
 	if (ret)
 		goto fail;
 
@@ -751,10 +723,13 @@ static int ubi_find_fastmap(struct ubi_device *ubi, int *fm_start)
 
 	*fm_start = -1;
 	for (i = 0; i < UBI_FM_MAX_START; i++) {
+		if (ubi_io_is_bad(ubi, i))
+			continue;
+
 		ret = ubi_io_read_vid_hdr(ubi, i, vhdr, 0);
 		if (ret < 0)
 			goto out;
-		else if (ret > 0)
+		else if (ret > 0 && ret != UBI_IO_BITFLIPS)
 			continue;
 
 		if (be32_to_cpu(vhdr->vol_id) == UBI_FM_SB_VOLUME_ID) {
@@ -798,7 +773,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	struct ubi_vid_hdr *vh;
 	struct ubi_ec_hdr *ech;
 	int ret, i, used_blocks, pnum, sb_pnum = 0;
-	char *fm_raw;
+	void *fm_raw;
 	size_t fm_size;
 	__be32 crc, tmp_crc;
 	unsigned long long sqnum = 0;
@@ -821,16 +796,10 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	 * care of? */
 	ret = ubi_io_read(ubi, fmsb, sb_pnum, ubi->leb_start, sizeof(*fmsb));
 	if (ret && ret != UBI_IO_BITFLIPS) {
-		/* TODO: what are the error codes > 0 ? Why is this check? */
-		if (ret > 0)
-			ret = UBI_BAD_FASTMAP;
 		kfree(fmsb);
 		goto out;
 	}
 
-	/* TODO: please, use 'be32_to_cpu()' _every_ time you access a __be32 /
-	 * etc field. Please, look how things are done in io.c. Please, check
-	 * and fix globally. */
 	if (be32_to_cpu(fmsb->magic) != UBI_FM_SB_MAGIC) {
 		/* TODO: not urgent, but examine all the error messages and
 		 * print more information there. Here you should print what was
@@ -913,11 +882,9 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 		}
 
 		ret = ubi_io_read_vid_hdr(ubi, pnum, vh, 0);
-		if (ret) {
+		if (ret && ret != UBI_IO_BITFLIPS) {
 			ubi_err("unable to read fastmap block# %i (PEB: %i)",
 				i, pnum);
-			if (ret > 0)
-				ret = UBI_BAD_FASTMAP;
 			kfree(fmsb);
 			goto free_hdr;
 		}
@@ -941,13 +908,10 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 			sqnum = be64_to_cpu(vh->sqnum);
 
 		ret = ubi_io_read(ubi, fm_raw + (ubi->leb_size * i), pnum,
-			ubi->leb_start, ubi->leb_size);
-
-		if (ret) {
+				  ubi->leb_start, ubi->leb_size);
+		if (ret && ret != UBI_IO_BITFLIPS) {
 			ubi_err("unable to read fastmap block# %i (PEB: %i)",
 				i, pnum);
-			if (ret > 0)
-				ret = UBI_BAD_FASTMAP;
 			kfree(fmsb);
 			goto free_hdr;
 		}
@@ -1023,7 +987,7 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 			     struct ubi_fastmap_layout *new_fm)
 {
 	size_t fm_pos = 0;
-	char *fm_raw;
+	void *fm_raw;
 	struct ubi_fm_sb *fmsb;
 	struct ubi_fm_hdr *fmh;
 	struct ubi_fm_scan_pool *fmpl1, *fmpl2;
@@ -1039,44 +1003,18 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 	fm_raw = vzalloc(new_fm->size);
 	if (!fm_raw) {
 		ret = -ENOMEM;
-
-		/* TODO: nitpick, sorry for being annoying, but this should not
-		 * be difficult to fix. I find it very irritating when there
-		 * are useless blank lines like this - they only make less code
-		 * fit my screen and make large functions even larger.
-		 *
-		 * Why we use blank lines inside a single function? To make
-		 * code more readable. How we make it more readable? By
-		 * separating logically different blocks of code with a
-		 * newline.
-		 *
-		 * What is the perbose of these newlines before goto's? This is
-		 * a single piece of error-handling code - you assing the
-		 * return value and go to the further error processing. These
-		 * newlines serve no purpose just blow the lines number in the
-		 * code. Could we please kill them globally?
-		 *
-		 * I am again sorry - probably this is not completely
-		 * technical, but I gave the explanation why these newlines are
-		 * annying. And as a person who spent a lot of personal
-		 * (non-paid) time maintaining this code and who will keep
-		 * doing this - I think I have right to require to do such
-		 * cosmetic things :-) If you do not agree with my reasoning,
-		 * may be this is better - you'll keep me happier :-) */
 		goto out;
 	}
 
 	avhdr = new_fm_vhdr(ubi, UBI_FM_SB_VOLUME_ID);
 	if (!avhdr) {
 		ret = -ENOMEM;
-
 		goto out_vfree;
 	}
 
 	dvhdr = new_fm_vhdr(ubi, UBI_FM_DATA_VOLUME_ID);
 	if (!dvhdr) {
 		ret = -ENOMEM;
-
 		goto out_kfree;
 	}
 
@@ -1189,7 +1127,6 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 	ret = ubi_io_write_vid_hdr(ubi, new_fm->e[0]->pnum, avhdr);
 	if (ret) {
 		ubi_err("unable to write vid_hdr to fastmap SB!");
-
 		goto out_kfree;
 	}
 
@@ -1210,7 +1147,6 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 		if (ret) {
 			ubi_err("unable to write vid_hdr to PEB %i!",
 				new_fm->e[i]->pnum);
-
 			goto out_kfree;
 		}
 	}
@@ -1221,7 +1157,6 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 		if (ret) {
 			ubi_err("unable to write fastmap to PEB %i!",
 				new_fm->e[i]->pnum);
-
 			goto out_kfree;
 		}
 	}
@@ -1303,9 +1238,8 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 			}
 
 			/* we have to erase the block by hand */
-
 			ret = ubi_io_read_ec_hdr(ubi, old_fm->e[0]->pnum,
-				ec_hdr, 0);
+						 ec_hdr, 0);
 			if (ret && ret != UBI_IO_BITFLIPS) {
 				ubi_err("unable to read EC header");
 				kfree(ec_hdr);
@@ -1314,7 +1248,7 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 			}
 
 			ret = ubi_io_sync_erase(ubi, old_fm->e[0]->pnum,
-				0);
+						0);
 			if (ret < 0) {
 				ubi_err("unable to erase old SB");
 				kfree(ec_hdr);
@@ -1334,7 +1268,7 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 
 			ec_hdr->ec = cpu_to_be64(ec);
 			ret = ubi_io_write_ec_hdr(ubi, old_fm->e[0]->pnum,
-				ec_hdr);
+						  ec_hdr);
 			kfree(ec_hdr);
 			if (ret) {
 				ubi_err("unable to write new EC header");
-- 
1.7.6.5

[PATCH 20/21] UBI: Fastmap: Torture fastmap PEBs that showed bit flips

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/attach.c  |    8 +++++-
 drivers/mtd/ubi/fastmap.c |   46 ++++++++++++++++++++++++++++----------------
 drivers/mtd/ubi/ubi.h     |    5 ++-
 3 files changed, 38 insertions(+), 21 deletions(-)

diff --git a/drivers/mtd/ubi/attach.c b/drivers/mtd/ubi/attach.c
index 2a0c1ba..4c6b97b 100644
--- a/drivers/mtd/ubi/attach.c
+++ b/drivers/mtd/ubi/attach.c
@@ -1371,8 +1371,12 @@ void ubi_destroy_ai(struct ubi_device *ubi, struct ubi_attach_info *ai)
 
 	/* Return all PEBs back to the WL sub-system */
 	if (ai->fm) {
-		while(ai->fm->used_blocks--)
-			ubi_wl_put_fm_peb(ubi, ai->fm->e[ai->fm->used_blocks], 0);
+		int i, torture;
+
+		for (i = 0; i < ai->fm->used_blocks; i++) {
+			torture = ai->fm->to_be_tortured[i];
+			ubi_wl_put_fm_peb(ubi, ai->fm->e[i], torture);
+		}
 
 		kfree(ai->fm);
 	}
diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 8b09c29..ecc313d 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -713,7 +713,7 @@ fail:
  */
 static int ubi_find_fastmap(struct ubi_device *ubi, int *fm_start)
 {
-	int i, ret;
+	int i, ret = -ENOENT;
 	struct ubi_vid_hdr *vhdr;
 	unsigned long long max_sqnum = 0, sqnum;
 
@@ -772,8 +772,8 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	struct ubi_fm_sb *fmsb;
 	struct ubi_vid_hdr *vh;
 	struct ubi_ec_hdr *ech;
-	int ret, i, used_blocks, pnum, sb_pnum = 0;
-	void *fm_raw;
+	int i, used_blocks, pnum, sb_pnum = 0, ret = 0;
+	void *fm_raw = NULL;
 	size_t fm_size;
 	__be32 crc, tmp_crc;
 	unsigned long long sqnum = 0;
@@ -790,15 +790,20 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 		goto out;
 	}
 
-	/* TODO: If 'ubi_io_read()' returns you UBI_IO_BITFLIP, this means that
-	 * the PEB has a bit-flip and has to be scrubbed. How will the
-	 * superblock be scrubbed or how is the bit-flip guaranteed to be taken
-	 * care of? */
+	ai->fm = kzalloc(sizeof(*ai->fm), GFP_KERNEL);
+	if (!ai->fm) {
+		ret = -ENOMEM;
+		kfree(fmsb);
+		goto free_raw;
+	}
+
 	ret = ubi_io_read(ubi, fmsb, sb_pnum, ubi->leb_start, sizeof(*fmsb));
 	if (ret && ret != UBI_IO_BITFLIPS) {
 		kfree(fmsb);
+		kfree(ai->fm);
 		goto out;
-	}
+	} else if (ret == UBI_IO_BITFLIPS)
+		ai->fm->to_be_tortured[0] = 1;
 
 	if (be32_to_cpu(fmsb->magic) != UBI_FM_SB_MAGIC) {
 		/* TODO: not urgent, but examine all the error messages and
@@ -811,6 +816,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 		ubi_err("super block magic does not match");
 		ret = UBI_BAD_FASTMAP;
 		kfree(fmsb);
+		kfree(ai->fm);
 		goto out;
 	}
 
@@ -818,6 +824,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 		ubi_err("unknown fastmap format version!");
 		ret = UBI_BAD_FASTMAP;
 		kfree(fmsb);
+		kfree(ai->fm);
 		goto out;
 	}
 
@@ -826,6 +833,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 		ubi_err("number of fastmap blocks is invalid");
 		ret = UBI_BAD_FASTMAP;
 		kfree(fmsb);
+		kfree(ai->fm);
 		goto out;
 	}
 
@@ -835,6 +843,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	if (!fm_raw) {
 		ret = -ENOMEM;
 		kfree(fmsb);
+		kfree(ai->fm);
 		goto out;
 	}
 
@@ -842,6 +851,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	if (!ech) {
 		ret = -ENOMEM;
 		kfree(fmsb);
+		kfree(ai->fm);
 		goto free_raw;
 	}
 
@@ -849,6 +859,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	if (!vh) {
 		ret = -ENOMEM;
 		kfree(fmsb);
+		kfree(ai->fm);
 		kfree(ech);
 		goto free_raw;
 	}
@@ -858,6 +869,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 
 		if (ubi_io_is_bad(ubi, pnum)) {
 			ret = UBI_BAD_FASTMAP;
+			kfree(ai->fm);
 			kfree(fmsb);
 			goto free_hdr;
 		}
@@ -869,8 +881,10 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 			if (ret > 0)
 				ret = UBI_BAD_FASTMAP;
 			kfree(fmsb);
+			kfree(ai->fm);
 			goto free_hdr;
-		}
+		} else if (ret == UBI_IO_BITFLIPS)
+			ai->fm->to_be_tortured[i] = 1;
 
 		if (!ubi->image_seq)
 			ubi->image_seq = be32_to_cpu(ech->image_seq);
@@ -878,6 +892,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 		if (be32_to_cpu(ech->image_seq) != ubi->image_seq) {
 			ret = UBI_BAD_FASTMAP;
 			kfree(fmsb);
+			kfree(ai->fm);
 			goto free_hdr;
 		}
 
@@ -886,6 +901,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 			ubi_err("unable to read fastmap block# %i (PEB: %i)",
 				i, pnum);
 			kfree(fmsb);
+			kfree(ai->fm);
 			goto free_hdr;
 		}
 
@@ -893,7 +909,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 			if (be32_to_cpu(vh->vol_id) != UBI_FM_SB_VOLUME_ID) {
 				ret = UBI_BAD_FASTMAP;
 				kfree(fmsb);
-
+				kfree(ai->fm);
 				goto free_hdr;
 			}
 		} else {
@@ -913,6 +929,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 			ubi_err("unable to read fastmap block# %i (PEB: %i)",
 				i, pnum);
 			kfree(fmsb);
+			kfree(ai->fm);
 			goto free_hdr;
 		}
 	}
@@ -926,7 +943,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	if (crc != tmp_crc) {
 		ubi_err("fastmap data CRC is invalid");
 		ret = UBI_BAD_FASTMAP;
-
+		kfree(ai->fm);
 		goto free_hdr;
 	}
 
@@ -936,12 +953,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	if (ret) {
 		if (ret > 0)
 			ret = UBI_BAD_FASTMAP;
-		goto free_hdr;
-	}
-
-	ai->fm = kzalloc(sizeof(*ai->fm), GFP_KERNEL);
-	if (!ai->fm) {
-		ret = -ENOMEM;
+		kfree(ai->fm);
 		goto free_hdr;
 	}
 
diff --git a/drivers/mtd/ubi/ubi.h b/drivers/mtd/ubi/ubi.h
index ccd0da7..ba6bfd1 100644
--- a/drivers/mtd/ubi/ubi.h
+++ b/drivers/mtd/ubi/ubi.h
@@ -214,13 +214,14 @@ struct ubi_volume_desc;
 
 /**
  * struct ubi_fastmap_layout - in-memory fastmap data structure.
- * @peb: PEBs used by the current fastmap
- * @ec: the erase counter of each used PEB
+ * @e: PEBs used by the current fastmap
+ * @to_be_tortured: if non-zero tortured this PEB
  * @size: size of the fastmap in bytes
  * @used_blocks: number of used PEBs
  */
 struct ubi_fastmap_layout {
 	struct ubi_wl_entry *e[UBI_FM_MAX_BLOCKS];
+	int to_be_tortured[UBI_FM_MAX_BLOCKS];
 	size_t size;
 	int used_blocks;
 };
-- 
1.7.6.5

[PATCH 21/21] UBI: Fastmap: make checkpatch.pl happy Signed-off-by: Richard Weinberger <richard@nod.at>

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |   10 ++++++----
 drivers/mtd/ubi/wl.c      |    3 ++-
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index ecc313d..8b033e5 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -402,12 +402,13 @@ static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 
 		err = ubi_io_read_vid_hdr(ubi, pnum, vh, 0);
 		if (err == UBI_IO_FF || err == UBI_IO_FF_BITFLIPS) {
+			unsigned long long ec = be64_to_cpu(ech->ec);
 			unmap_peb(ai, pnum);
 			dbg_bld("Adding PEB to free: %i", pnum);
 			if (err == UBI_IO_FF_BITFLIPS)
-				add_aeb(ai, &ai->free, pnum, be64_to_cpu(ech->ec), 1);
+				add_aeb(ai, &ai->free, pnum, ec, 1);
 			else
-				add_aeb(ai, &ai->free, pnum, be64_to_cpu(ech->ec), 0);
+				add_aeb(ai, &ai->free, pnum, ec, 0);
 			continue;
 		} else if (err == 0 || err == UBI_IO_BITFLIPS) {
 			dbg_bld("Found non empty PEB:%i in pool", pnum);
@@ -575,7 +576,8 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 		av = add_vol(ai, be32_to_cpu(fmvhdr->vol_id),
 			     be32_to_cpu(fmvhdr->used_ebs),
 			     be32_to_cpu(fmvhdr->data_pad),
-			     fmvhdr->vol_type, be32_to_cpu(fmvhdr->last_eb_bytes));
+			     fmvhdr->vol_type,
+			     be32_to_cpu(fmvhdr->last_eb_bytes));
 
 		if (!av)
 			goto fail_bad;
@@ -688,7 +690,7 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 
 	list_for_each_entry_safe(tmp_aeb, _tmp_aeb, &used, u.list) {
 		list_del(&tmp_aeb->u.list);
-		dbg_bld("adding PEB from used to erase list: %i", tmp_aeb->pnum);
+		dbg_bld("moving PEB from used to erase: %i", tmp_aeb->pnum);
 		add_aeb(ai, &ai->erase, tmp_aeb->pnum, tmp_aeb->ec, 0);
 		kmem_cache_free(ai->aeb_slab_cache, tmp_aeb);
 	}
diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index c4d8273..a64cd83 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -827,7 +827,8 @@ static int do_sync_erase(struct ubi_device *ubi, struct ubi_wl_entry *e,
  *
  * see: ubi_wl_put_peb()
  */
-int ubi_wl_put_fm_peb(struct ubi_device *ubi, struct ubi_wl_entry *used_e, int torture)
+int ubi_wl_put_fm_peb(struct ubi_device *ubi, struct ubi_wl_entry *used_e,
+    int torture)
 {
 	struct ubi_wl_entry *e;
 	int pnum = used_e->pnum;
-- 
1.7.6.5