UBI fastmap updates

UBI fastmap updates

[Richard Weinberger]

This is the next round of UBI fastmap updates.
Most changes are coding style and kernel doc fixes.

The highlights are:
	- Fastmap stores now the pool size in it's anchor PEB
	  such that users can set custom pool sizes. (E.g using ubinize)

I've addressed all TODOs.
I did not remove this point from the TODO list:
"2. Implement power-cut emulation infrastructure similar to what is in UBIFS and
   test UBI + fastmap with it."

Artem, do we really need a new power-cut emulation infrastructure?
I did some tests using integck (integck -p -m 0 /mnt).
Nothing bad happened. :-)
Ff you disable writing the fastmap at detach time you can also simulate
a power cut. I did that very often. We written fastmap has to be valid at any time!
Also if a power cut happens while writing the fastmap it's CRC will be
bad and fastmap falls back to scanning mode.
So, I'm not 100% sure what fastmap specific power cut tests need to be done.

Another point:
UBI_FM_MAX_POOL_SIZE is currently 256.
A fastmap pool cannot be greater than this value.
Are we fine with this upper limit?

If you want to test fastmap you can use my git repo:
git://git.kernel.org/pub/scm/linux/kernel/git/rw/ubi2.git ubi2/v14

Enjoy!
//richard

[PATCH 01/16] UBI: Fastmap: Check find_mean_wl_entry()'s return
[PATCH 02/16] UBI: Fastmap: Kernel doc updates
[PATCH 03/16] UBI: Fastmap: Fix license version
[PATCH 04/16] UBI: Fastmap: Rename self_check_fastmap()
[PATCH 05/16] UBI: Fastmap: Address a TODO
[PATCH 06/16] UBI: Fastmap: Address another TODO
[PATCH 07/16] UBI: Fastmap: Address jet another TOOD
[PATCH 08/16] UBI: Fastmap: Address another TOOD
[PATCH 09/16] UBI: Fastmap: Be more verbose on fastmap failure
[PATCH 10/16] UBI: Fastmap: More kernel doc updates
[PATCH 11/16] UBI: Fastmap: Store pool sizes in fastmap
[PATCH 12/16] UBI: Fastmap: Make checkpatch.pl happy
[PATCH 13/16] UBI: Fastmap: Remove one point from TODO list
[PATCH 14/16] UBI: Fastmap: Remove one point from TODO list
[PATCH 15/16] UBI: Fastmap: Remove one point from TODO list
[PATCH 16/16] UBI: Fastmap: Remove one point from TODO list

[PATCH 01/16] UBI: Fastmap: Check find_mean_wl_entry()'s return value

[Richard Weinberger]

find_mean_wl_entry() is allowed to return NULL.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/wl.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index b44cba0..85a3373 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -509,6 +509,10 @@ retry:
 	}
 
 	e = find_mean_wl_entry(ubi, &ubi->free);
+	if (!e) {
+		ubi_err("no free eraseblocks");
+		return -ENOSPC;
+	}
 
 	self_check_in_wl_tree(ubi, e, &ubi->free);
 
-- 
1.7.6.5

[PATCH 02/16] UBI: Fastmap: Kernel doc updates

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/attach.c    |    2 +-
 drivers/mtd/ubi/eba.c       |    8 ++++++++
 drivers/mtd/ubi/fastmap.c   |   16 +++++++++-------
 drivers/mtd/ubi/ubi-media.h |    5 +++--
 drivers/mtd/ubi/ubi.h       |    2 ++
 drivers/mtd/ubi/wl.c        |   32 +++++++++++++++++++++++++-------
 6 files changed, 48 insertions(+), 17 deletions(-)

diff --git a/drivers/mtd/ubi/attach.c b/drivers/mtd/ubi/attach.c
index 00e58dd..021c7d3 100644
--- a/drivers/mtd/ubi/attach.c
+++ b/drivers/mtd/ubi/attach.c
@@ -750,7 +750,7 @@ struct ubi_ainf_peb *ubi_early_get_peb(struct ubi_device *ubi,
 /**
  * check_corruption - check the data area of PEB.
  * @ubi: UBI device description object
- * @vid_hrd: the (corrupted) VID header of this PEB
+ * @vid_hdr: the (corrupted) VID header of this PEB
  * @pnum: the physical eraseblock number to check
  *
  * This is a helper function which is used to distinguish between VID header
diff --git a/drivers/mtd/ubi/eba.c b/drivers/mtd/ubi/eba.c
index cb30e7a..cb0139c 100644
--- a/drivers/mtd/ubi/eba.c
+++ b/drivers/mtd/ubi/eba.c
@@ -1178,6 +1178,7 @@ out_unlock_leb:
 /**
  * print_rsvd_warning - warn about not having enough reserved PEBs.
  * @ubi: UBI device description object
+ * @ai: UBI attach info object
  *
  * This is a helper function for 'ubi_eba_init()' which is called when UBI
  * cannot reserve enough PEBs for bad block handling. This function makes a
@@ -1216,6 +1217,13 @@ static void print_rsvd_warning(struct ubi_device *ubi,
 			ubi->corr_peb_count);
 }
 
+/**
+ * self_check_eba - run a self check on the EBA table construected by fastmap.
+ *
+ * @ubi: UBI device description object
+ * @ai_fastmap: UBI attach info object created by fastmap
+ * @ai_scan: UBI attach info object created by scanning
+ */
 int self_check_eba(struct ubi_device *ubi, struct ubi_attach_info *ai_fastmap,
 		   struct ubi_attach_info *ai_scan)
 {
diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index c6927a9..988f620 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -259,7 +259,7 @@ static int update_vol(struct ubi_device *ubi, struct ubi_attach_info *ai,
 }
 
 /**
- * process_pool_aeb - we found a non-empty PEB in a pool
+ * process_pool_aeb - we found a non-empty PEB in a pool.
  * @ubi: UBI device object
  * @ai: attach info object
  * @new_vh: the volume header derived from new_aeb
@@ -313,7 +313,7 @@ static int process_pool_aeb(struct ubi_device *ubi, struct ubi_attach_info *ai,
  * If fastmap detects a free PEB in the pool it has to check whether
  * this PEB has been unmapped after writing the fastmap.
  *
- * @ubi: UBI device object
+ * @ai: UBI attach info object
  * @pnum: The PEB to be unmapped
  */
 static void unmap_peb(struct ubi_attach_info *ai, int pnum)
@@ -338,13 +338,14 @@ static void unmap_peb(struct ubi_attach_info *ai, int pnum)
 }
 
 /**
- * scan_pool - scans a pool for changed (no longer empty PEBs)
+ * scan_pool - scans a pool for changed (no longer empty PEBs).
  * @ubi: UBI device object
  * @ai: attach info object
  * @pebs: an array of all PEB numbers in the to be scanned pool
  * @pool_size: size of the pool (number of entries in @pebs)
  * @max_sqnum: pointer to the maximal sequence number
  * @eba_orphans: list of PEBs which need to be scanned
+ * @free: list of PEBs which are most likely free (and go into @ai->free)
  */
 static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 		     int *pebs, int pool_size, unsigned long long *max_sqnum,
@@ -484,6 +485,7 @@ static int self_check_fastmap(struct ubi_attach_info *ai)
 /**
  * ubi_attach_fastmap - creates ubi_attach_info from a fastmap.
  * @ubi: UBI device object
+ * @ai: UBI attach info object
  * @fm_raw: the fastmap it self as byte array
  * @fm_size: size of the fastmap in bytes
  */
@@ -791,7 +793,7 @@ out:
 }
 
 /**
- * ubi_scan_fastmap - scan the fastmap
+ * ubi_scan_fastmap - scan the fastmap.
  * @ubi: UBI device object
  * @ai: UBI attach info to be filled
  *
@@ -1047,7 +1049,7 @@ out:
 }
 
 /**
- * ubi_write_fastmap - writes a fastmap
+ * ubi_write_fastmap - writes a fastmap.
  * @ubi: UBI device object
  * @new_fm: the to be written fastmap
  */
@@ -1279,7 +1281,7 @@ out:
 }
 
 /**
- * erase_block - Manually erase a PEB
+ * erase_block - Manually erase a PEB.
  * @ubi: UBI device object
  * @pnum: PEB to be erased
  */
@@ -1324,7 +1326,7 @@ out:
 }
 
 /**
- * invalidate_fastmap - destroys a fastmap
+ * invalidate_fastmap - destroys a fastmap.
  * @ubi: UBI device object
  * @fm: the fastmap to be destroyed
  */
diff --git a/drivers/mtd/ubi/ubi-media.h b/drivers/mtd/ubi/ubi-media.h
index a0648c1..eaf81a2 100644
--- a/drivers/mtd/ubi/ubi-media.h
+++ b/drivers/mtd/ubi/ubi-media.h
@@ -432,7 +432,8 @@ struct ubi_fm_sb {
  * struct ubi_fm_hdr - header of the fastmap data set
  * @magic: fastmap header magic number (%UBI_FM_HDR_MAGIC)
  * @free_peb_count: number of free PEBs known by this fastmap
- * @free_peb_count: number of used PEBs known by this fastmap
+ * @used_peb_count: number of used PEBs known by this fastmap
+ * @scrub_peb_count: number of to be scrubbed PEBs known by this fastmap
  * @bad_peb_count: number of bad PEBs known by this fastmap
  * @erase_peb_count: number of bad PEBs which have to be erased
  * @vol_count: number of UBI volumes known by this fastmap
@@ -500,7 +501,7 @@ struct ubi_fm_volhdr {
 
 /**
  * struct ubi_fm_eba - denotes an association beween a PEB and LEB
- * @magic EBA table magic number
+ * @magic: EBA table magic number
  * @reserved_pebs: number of table entries
  * @pnum: PEB number of LEB (LEB is the index)
  */
diff --git a/drivers/mtd/ubi/ubi.h b/drivers/mtd/ubi/ubi.h
index 0ca01eb..b60818d 100644
--- a/drivers/mtd/ubi/ubi.h
+++ b/drivers/mtd/ubi/ubi.h
@@ -382,6 +382,8 @@ struct ubi_wl_entry;
  *
  * @fm: in-memory data structure of the currently used fastmap
  * @fm_pool: in-memory data structure of the fastmap pool
+ * @fm_wl_pool: in-memory data structure of the fastmap pool used by the WL
+ * 		sub-system
  * @fm_mutex: serializes ubi_update_fastmap()
  * @fm_sem: allows ubi_update_fastmap() to block EBA table changes
  * @fm_work: fastmap work queue
diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index 85a3373..06bf985 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -483,13 +483,13 @@ out:
 }
 
 /**
- * __ubi_wl_get_peb - get a physical eraseblock.
+ * __wl_get_peb - get a physical eraseblock.
  * @ubi: UBI device description object
  *
  * This function returns a physical eraseblock in case of success and a
  * negative error code in case of failure. Might sleep.
  */
-static int __ubi_wl_get_peb(struct ubi_device *ubi)
+static int __wl_get_peb(struct ubi_device *ubi)
 {
 	int err;
 	struct ubi_wl_entry *e;
@@ -585,7 +585,7 @@ static void refill_wl_user_pool(struct ubi_device *ubi)
 	return_unused_pool_pebs(ubi, pool);
 
 	for (pool->size = 0; pool->size < pool->max_size; pool->size++) {
-		pool->pebs[pool->size] = __ubi_wl_get_peb(ubi);
+		pool->pebs[pool->size] = __wl_get_peb(ubi);
 		if (pool->pebs[pool->size] < 0)
 			break;
 	}
@@ -604,7 +604,7 @@ void ubi_refill_pools(struct ubi_device *ubi)
 	spin_unlock(&ubi->wl_lock);
 }
 
-/* ubi_wl_get_peb - works exaclty like __ubi_wl_get_peb but keeps track of
+/* ubi_wl_get_peb - works exaclty like __wl_get_peb but keeps track of
  * the fastmap pool.
  */
 int ubi_wl_get_peb(struct ubi_device *ubi)
@@ -630,7 +630,7 @@ int ubi_wl_get_peb(struct ubi_device *ubi)
 	return ret;
 }
 
-/* get_peb_for_wl - returns a PEB to be used internally by the WL sub-system
+/* get_peb_for_wl - returns a PEB to be used internally by the WL sub-system.
  *
  * @ubi: UBI device description object
  */
@@ -818,7 +818,7 @@ static int erase_worker(struct ubi_device *ubi, struct ubi_work *wl_wrk,
 			int cancel);
 
 /**
- * ubi_is_erase_work - checks whether a work is erase work
+ * ubi_is_erase_work - checks whether a work is erase work.
  * @wrk: The work object to be checked
  */
 int ubi_is_erase_work(struct ubi_work *wrk)
@@ -862,6 +862,15 @@ static int schedule_erase(struct ubi_device *ubi, struct ubi_wl_entry *e,
 	return 0;
 }
 
+/**
+ * do_sync_erase - run the erase worker synchronously.
+ * @ubi: UBI device description object
+ * @e: the WL entry of the physical eraseblock to erase
+ * @vol_id: the volume ID that last used this PEB
+ * @lnum: the last used logical eraseblock number for the PEB
+ * @torture: if the physical eraseblock has to be tortured
+ *
+ */
 static int do_sync_erase(struct ubi_device *ubi, struct ubi_wl_entry *e,
 			 int vol_id, int lnum, int torture)
 {
@@ -884,8 +893,12 @@ static int do_sync_erase(struct ubi_device *ubi, struct ubi_wl_entry *e,
 /**
  * ubi_wl_put_fm_peb - returns a PEB used in a fastmap to the wear-leveling
  * sub-system.
- *
  * see: ubi_wl_put_peb()
+ *
+ * @ubi: UBI device description object
+ * @fm_e: physical eraseblock to return
+ * @lnum: the last used logical eraseblock number for the PEB
+ * @torture: if this physical eraseblock has to be tortured
  */
 int ubi_wl_put_fm_peb(struct ubi_device *ubi, struct ubi_wl_entry *fm_e,
 		      int lnum, int torture)
@@ -1229,6 +1242,7 @@ out_cancel:
 /**
  * ensure_wear_leveling - schedule wear-leveling if it is needed.
  * @ubi: UBI device description object
+ * @nested: set to non-zero if this function is called from UBI worker
  *
  * This function checks if it is time to start wear-leveling and schedules it
  * if yes. This function returns zero in case of success and a negative error
@@ -1295,6 +1309,10 @@ out_unlock:
 	return err;
 }
 
+/**
+ * ubi_ensure_anchor_pebs - schedule wear-leveling to produce an anchor PEB.
+ * @ubi: UBI device description object
+ */
 int ubi_ensure_anchor_pebs(struct ubi_device *ubi)
 {
 	struct ubi_work *wrk;
-- 
1.7.6.5

[PATCH 03/16] UBI: Fastmap: Fix license version

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 988f620..ba547e2 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -4,7 +4,7 @@
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License.
+ * the Free Software Foundation; version 2.
  *
  * This program is distributed in the hope that it will be useful,
  * but WITHOUT ANY WARRANTY; without even the implied warranty of
-- 
1.7.6.5

[PATCH 04/16] UBI: Fastmap: Rename self_check_fastmap()

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |    8 ++++++--
 1 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index ba547e2..a596cef 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -462,7 +462,11 @@ out:
 	return ret;
 }
 
-static int self_check_fastmap(struct ubi_attach_info *ai)
+/**
+ * count_fastmap_pebs - Counts the PEBs found by fastmap.
+ * @ai: The UBI attach info object
+ */
+static int count_fastmap_pebs(struct ubi_attach_info *ai)
 {
 	struct ubi_ainf_peb *aeb;
 	struct ubi_ainf_volume *av;
@@ -1006,7 +1010,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	 * We do this here because in ubi_wl_init() it's too late
 	 * and we cannot fall back to scanning.
 	 */
-	if (WARN_ON(self_check_fastmap(ai) != ubi->peb_count -
+	if (WARN_ON(count_fastmap_pebs(ai) != ubi->peb_count -
 		    ai->bad_peb_count - used_blocks)) {
 		ret = UBI_BAD_FASTMAP;
 		kfree(fm);
-- 
1.7.6.5

Re: [PATCH 04/16] UBI: Fastmap: Rename self_check_fastmap()

[Artem Bityutskiy]

[-- Attachment #1: Type: text/plain, Size: 7624 bytes --]

On Wed, 2012-06-27 at 17:57 +0200, Richard Weinberger wrote:
> Signed-off-by: Richard Weinberger <richard@nod.at>

Hi, pushed this and added a couple of TODOs. Could you please take a
look? Most TODOs are non-essential and easy, but one is more important -
the one about regression in attach time of non-fastmap images.

Thanks!

From b43ceb01a33b322898232fb05e5cc7f754656421 Mon Sep 17 00:00:00 2001
From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Date: Thu, 28 Jun 2012 15:08:29 +0300
Subject: [PATCH 1/2] UBI: Fastmap: rename new_ai to alloc_ia

Just more consistent, we do not use "new" in, e.g., 'ubi_zalloc_vid_hdr()'.
Also move the code to avoid forward references, which we also do not use.

Also get rid of unneeded goto in alloc_ia(). Nothing major.

Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
---
 drivers/mtd/ubi/attach.c |   41 +++++++++++++++++++----------------------
 1 file changed, 19 insertions(+), 22 deletions(-)

diff --git a/drivers/mtd/ubi/attach.c b/drivers/mtd/ubi/attach.c
index 7552d25..8204b2d 100644
--- a/drivers/mtd/ubi/attach.c
+++ b/drivers/mtd/ubi/attach.c
@@ -91,7 +91,6 @@
 
 static int self_check_ai(struct ubi_device *ubi, struct ubi_attach_info *ai);
 static void destroy_ai(struct ubi_device *ubi, struct ubi_attach_info *ai);
-static struct ubi_attach_info *new_ai(void);
 
 /* Temporary variables used during scanning */
 static struct ubi_ec_hdr *ech;
@@ -1216,6 +1215,22 @@ out_ai:
 	return err;
 }
 
+static struct ubi_attach_info *alloc_ai(void)
+{
+	static struct ubi_attach_info *ai;
+
+	ai = kzalloc(sizeof(struct ubi_attach_info), GFP_KERNEL);
+	if (ai) {
+		INIT_LIST_HEAD(&ai->corr);
+		INIT_LIST_HEAD(&ai->free);
+		INIT_LIST_HEAD(&ai->erase);
+		INIT_LIST_HEAD(&ai->alien);
+		ai->volumes = RB_ROOT;
+	}
+
+	return ai;
+}
+
 /**
  * ubi_attach - attach an MTD device.
  * @ubi: UBI device descriptor
@@ -1229,7 +1244,7 @@ int ubi_attach(struct ubi_device *ubi, int force_scan)
 	int err;
 	struct ubi_attach_info *ai;
 
-	ai = new_ai();
+	ai = alloc_ai();
 	if (!ai)
 		return -ENOMEM;
 
@@ -1239,7 +1254,7 @@ int ubi_attach(struct ubi_device *ubi, int force_scan)
 		err = ubi_scan_fastmap(ubi, ai);
 		if (err > 0) {
 			destroy_ai(ubi, ai);
-			ai = new_ai();
+			ai = alloc_ai();
 			if (!ai)
 				return -ENOMEM;
 
@@ -1279,7 +1294,7 @@ int ubi_attach(struct ubi_device *ubi, int force_scan)
 
 	if (ubi->fm && ubi->dbg->chk_gen) {
 		struct ubi_attach_info *scan_ai;
-		scan_ai = new_ai();
+		scan_ai = alloc_ai();
 		if (!scan_ai)
 			goto out_ai;
 
@@ -1395,24 +1410,6 @@ static void destroy_ai(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	kfree(ai);
 }
 
-static struct ubi_attach_info *new_ai(void)
-{
-	static struct ubi_attach_info *ai;
-
-	ai = kzalloc(sizeof(struct ubi_attach_info), GFP_KERNEL);
-	if (!ai)
-		goto out;
-
-	INIT_LIST_HEAD(&ai->corr);
-	INIT_LIST_HEAD(&ai->free);
-	INIT_LIST_HEAD(&ai->erase);
-	INIT_LIST_HEAD(&ai->alien);
-	ai->volumes = RB_ROOT;
-
-out:
-	return ai;
-}
-
 /**
  * self_check_ai - check the attaching information.
  * @ubi: UBI device description object
-- 
1.7.10


From 400a911b104d88bbf6cc593e8bb24c9865a1c449 Mon Sep 17 00:00:00 2001
From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Date: Thu, 28 Jun 2012 15:34:17 +0300
Subject: [PATCH 2/2] UBI: fastmap: some todo and random changes

Add few consmetic changes and a bunch of TODOs.

Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
---
 drivers/mtd/ubi/attach.c |   29 +++++++++++++++++++++++++++++
 drivers/mtd/ubi/eba.c    |    7 ++++---
 2 files changed, 33 insertions(+), 3 deletions(-)

diff --git a/drivers/mtd/ubi/attach.c b/drivers/mtd/ubi/attach.c
index 8204b2d..3d9be42 100644
--- a/drivers/mtd/ubi/attach.c
+++ b/drivers/mtd/ubi/attach.c
@@ -89,6 +89,13 @@
 #include <linux/random.h>
 #include "ubi.h"
 
+/*
+ * TODO: please, no forward declarations. We do not use them in UBI code.
+ * Actually initially I did use them a lot, but when upstreaming, I was asked
+ * to remove. Please, follow this convention as well. Please, change globally.
+ * I mean, I am already used to that _all_ the code is upside-down, let's keep
+ * it that way, or re-structure all the code. :-)
+ */
 static int self_check_ai(struct ubi_device *ubi, struct ubi_attach_info *ai);
 static void destroy_ai(struct ubi_device *ubi, struct ubi_attach_info *ai);
 
@@ -1170,6 +1177,7 @@ static int scan_all(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	if (ai->ec_count)
 		ai->mean_ec = div_u64(ai->ec_sum, ai->ec_count);
 
+	/* TODO: if we attach by fastmap, we do not execute this? */
 	err = late_analysis(ubi, ai);
 	if (err)
 		goto out_vidh;
@@ -1251,6 +1259,25 @@ int ubi_attach(struct ubi_device *ubi, int force_scan)
 	if (force_scan)
 		err = scan_all(ubi, ai);
 	else {
+		/* TODO: this is a regression. If I have an old image, and I do
+		 * not want to use fastmap, I will be forced to waste time for
+		 * useless scan of 64 first eraseblocks. Not good.
+		 *
+		 * Can you teach ubi_scan_fastmap() to use 'scan_peb()'
+		 * function for scanning and build normal ai information? If it
+		 * finds fastmap - it can destroy the collected ai. If it does
+		 * not find, it returns ai. Then you just confinue scanning.
+		 *
+		 * I buess what we'll need is:
+		 * 1. scan_all() -> scan_range(..., int pnum1, int pnum2);
+		 * 2. ubi_scan_fastmap() returns the pnum of the last scanned
+		 *    eraseblock if fastmap was not found;
+		 *    Also 'ubi_scan_fastmap()' uses scan_peb() for scanning.
+		 * 3. You call 'scan_range(..., pnum, c->peb_cnt - 1)' and
+		 *    it continues.
+		 *
+		 * And no regressions.
+		 */
 		err = ubi_scan_fastmap(ubi, ai);
 		if (err > 0) {
 			destroy_ai(ubi, ai);
@@ -1276,6 +1303,7 @@ int ubi_attach(struct ubi_device *ubi, int force_scan)
 	if (err)
 		goto out_ai;
 
+	/* TODO: Hmm why this code is not hidden in 'ubi_scan_fastmap()' ? */
 	if (ubi->fm) {
 		ubi->fm_pool.max_size = ubi->fm->max_pool_size;
 		ubi->fm_wl_pool.max_size = ubi->fm->max_wl_pool_size;
@@ -1294,6 +1322,7 @@ int ubi_attach(struct ubi_device *ubi, int force_scan)
 
 	if (ubi->fm && ubi->dbg->chk_gen) {
 		struct ubi_attach_info *scan_ai;
+
 		scan_ai = alloc_ai();
 		if (!scan_ai)
 			goto out_ai;
diff --git a/drivers/mtd/ubi/eba.c b/drivers/mtd/ubi/eba.c
index cb0139c..6d6e301 100644
--- a/drivers/mtd/ubi/eba.c
+++ b/drivers/mtd/ubi/eba.c
@@ -1218,11 +1218,12 @@ static void print_rsvd_warning(struct ubi_device *ubi,
 }
 
 /**
- * self_check_eba - run a self check on the EBA table construected by fastmap.
- *
+ * self_check_eba - run a self check on the EBA table constructed by fastmap.
  * @ubi: UBI device description object
  * @ai_fastmap: UBI attach info object created by fastmap
  * @ai_scan: UBI attach info object created by scanning
+ *
+ * TODO: what we do and what return.
  */
 int self_check_eba(struct ubi_device *ubi, struct ubi_attach_info *ai_fastmap,
 		   struct ubi_attach_info *ai_scan)
@@ -1290,6 +1291,7 @@ int self_check_eba(struct ubi_device *ubi, struct ubi_attach_info *ai_fastmap,
 
 				ubi_err("LEB:%i:%i is PEB:%i instead of %i!",
 					vol->vol_id, i, fm_eba[i][j], scan_eba[i][j]);
+				/* TODO: no, please, return error instead */
 				BUG();
 			}
 		}
@@ -1306,7 +1308,6 @@ out_free:
 
 	kfree(scan_eba);
 	kfree(fm_eba);
-
 	return ret;
 }
 
-- 
1.7.10

-- 
Best Regards,
Artem Bityutskiy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 05/16] UBI: Fastmap: Address a TODO

[Richard Weinberger]

We cannot use an aggregate data structure because fastmaps
internals parts do not have fixed positions.
They depends on the number of avaliable PEBs.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |    4 ----
 1 files changed, 0 insertions(+), 4 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index a596cef..c2434cd 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -535,10 +535,6 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 	if (fm_pos >= fm_size)
 		goto fail_bad;
 
-	/* TODO: this is difficult to read. Can we please have instead an
-	 * aggregate data structure? I did not think hard on it may be you have
-	 * a good reason for this difficult style, but on the first glance it
-	 * does not look like. And where are all the endiness stuff?  */
 	fmhdr = (struct ubi_fm_hdr *)(fm_raw + fm_pos);
 	fm_pos += sizeof(*fmhdr);
 	if (fm_pos >= fm_size)
-- 
1.7.6.5

[PATCH 06/16] UBI: Fastmap: Address another TODO

[Richard Weinberger]

Fixed.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |    7 -------
 1 files changed, 0 insertions(+), 7 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index c2434cd..4d8ef9e 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -797,13 +797,6 @@ out:
  * @ubi: UBI device object
  * @ai: UBI attach info to be filled
  *
- * TODO: not urgent, but at some point - check the code with kernel doc and fix
- * its complaints.
- *
- * TODO: not urgent, but for consistency, follow the UBI/UBIFS style and put a
- * dot at the end of the first short description sentence (globally):
- *    ubi_scan_fastmap - scan the fastmap. (<-dot).
- *
  * TODO: not urgent, but it is desireble to document error codes in the header
  * comments and probably describe what the function does, if there is something
  * to say (globally).
-- 
1.7.6.5

[PATCH 07/16] UBI: Fastmap: Address jet another TOOD

[Richard Weinberger]

Writing the fastmap upon attach time if ai->is_empty=1
does not make sense. At this stage the erase worker is busy
with erasing all PEBs and fastmap won't get a free anchor PEB.
While detaching we write anyway a fastmap.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/attach.c |    8 --------
 1 files changed, 0 insertions(+), 8 deletions(-)

diff --git a/drivers/mtd/ubi/attach.c b/drivers/mtd/ubi/attach.c
index 021c7d3..068f11a 100644
--- a/drivers/mtd/ubi/attach.c
+++ b/drivers/mtd/ubi/attach.c
@@ -1301,14 +1301,6 @@ int ubi_attach(struct ubi_device *ubi, int force_scan)
 	}
 
 	destroy_ai(ubi, ai);
-
-	/* TODO: UBI auto formats the flash if it is empty (see ubi->is_empty).
-	 * It is currently done so that every sub-system writes initializes its
-	 * own stuff. Well, now it is only the vtbl sub-system - it creates
-	 * empty volume table. And this is why we have "early" function for
-	 * getting free PEBs. Fastmap should do the same - so I guess it is
-	 * good to do it somewhere here. Also, we need to re-create the fastmap
-	 * on-flash data-structures if they were corrupted. */
 	return 0;
 
 out_wl:
-- 
1.7.6.5

[PATCH 08/16] UBI: Fastmap: Address another TOOD

[Richard Weinberger]

This will be "fixed" upon ubinize has fastmap support.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/attach.c |   15 ---------------
 1 files changed, 0 insertions(+), 15 deletions(-)

diff --git a/drivers/mtd/ubi/attach.c b/drivers/mtd/ubi/attach.c
index 068f11a..1ac58ec 100644
--- a/drivers/mtd/ubi/attach.c
+++ b/drivers/mtd/ubi/attach.c
@@ -1250,21 +1250,6 @@ int ubi_attach(struct ubi_device *ubi, int force_scan)
 	if (err)
 		goto out_ai;
 
-	/* TODO: currently the fastmap code assumes that the fastmap data
-	 * structures are created only by the kernel when the kernel attaches
-	 * an fastmap-less image. However, this assumption is too limiting and
-	 * for sure people will want to pre-create UBI images with fastmap
-	 * using the ubinize tool. Then they wont have to waste a lot of time
-	 * waiting for full scan and fastmap initializetion during the first
-	 * boot. This is a very important feature for the factory production
-	 * line where every additional minute per device costs a lot.
-	 *
-	 * When you are attaching an MTD device which contains an image
-	 * generated by ubinize with a fastmap, you will not know the
-	 * 'bad_peb_count' value. Most probably it will contain something like
-	 * -1. The same is true for the per-PEB information in the fastmap - it
-	 * won't tell which PEBs are bad. So we need to detect this and iterate
-	 * over all PEBs, find out which are bad, and update 'ai' here. */
 	ubi->bad_peb_count = ai->bad_peb_count;
 	ubi->good_peb_count = ubi->peb_count - ubi->bad_peb_count;
 	ubi->corr_peb_count = ai->corr_peb_count;
-- 
1.7.6.5

[PATCH 09/16] UBI: Fastmap: Be more verbose on fastmap failure

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |   67 ++++++++++++++++++++++++++++++---------------
 1 files changed, 45 insertions(+), 22 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 4d8ef9e..07ac06e 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -299,7 +299,6 @@ static int process_pool_aeb(struct ubi_device *ubi, struct ubi_attach_info *ai,
 		av = tmp_av;
 	else {
 		ubi_err("orphaned volume in fastmap pool!");
-
 		return UBI_BAD_FASTMAP;
 	}
 
@@ -378,21 +377,23 @@ static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 		pnum = be32_to_cpu(pebs[i]);
 
 		if (ubi_io_is_bad(ubi, pnum)) {
-			dbg_bld("bad PEB in fastmap pool!");
+			ubi_err("bad PEB in fastmap pool!");
 			ret = UBI_BAD_FASTMAP;
 			goto out;
 		}
 
 		err = ubi_io_read_ec_hdr(ubi, pnum, ech, 0);
 		if (err && err != UBI_IO_BITFLIPS) {
-			dbg_bld("unable to read EC header!");
+			ubi_err("unable to read EC header! PEB:%i err:%i",
+				pnum, err);
 			ret = err > 0 ? UBI_BAD_FASTMAP : err;
 			goto out;
 		} else if (ret == UBI_IO_BITFLIPS)
 			scrub = 1;
 
 		if (be32_to_cpu(ech->image_seq) != ubi->image_seq) {
-			dbg_bld("image seq mismatch!");
+			ubi_err("bad image seq: 0x%x, expected: 0x%x",
+				be32_to_cpu(ech->image_seq), ubi->image_seq);
 			err = UBI_BAD_FASTMAP;
 			goto out;
 		}
@@ -540,22 +541,31 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 	if (fm_pos >= fm_size)
 		goto fail_bad;
 
-	if (be32_to_cpu(fmhdr->magic) != UBI_FM_HDR_MAGIC)
+	if (be32_to_cpu(fmhdr->magic) != UBI_FM_HDR_MAGIC) {
+		ubi_err("bad fastmap header magic: 0x%x, expected: 0x%x",
+			be32_to_cpu(fmhdr->magic), UBI_FM_HDR_MAGIC);
 		goto fail_bad;
+	}
 
 	fmpl1 = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
 	fm_pos += sizeof(*fmpl1);
 	if (fm_pos >= fm_size)
 		goto fail_bad;
-	if (be32_to_cpu(fmpl1->magic) != UBI_FM_POOL_MAGIC)
+	if (be32_to_cpu(fmpl1->magic) != UBI_FM_POOL_MAGIC) {
+		ubi_err("bad fastmap pool magic: 0x%x, expected: 0x%x",
+			be32_to_cpu(fmpl1->magic), UBI_FM_POOL_MAGIC);
 		goto fail_bad;
+	}
 
 	fmpl2 = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
 	fm_pos += sizeof(*fmpl2);
 	if (fm_pos >= fm_size)
 		goto fail_bad;
-	if (be32_to_cpu(fmpl2->magic) != UBI_FM_POOL_MAGIC)
+	if (be32_to_cpu(fmpl2->magic) != UBI_FM_POOL_MAGIC) {
+		ubi_err("bad fastmap pool magic: 0x%x, expected: 0x%x",
+			be32_to_cpu(fmpl2->magic), UBI_FM_POOL_MAGIC);
 		goto fail_bad;
+	}
 
 	/* read EC values from free list */
 	for (i = 0; i < be32_to_cpu(fmhdr->free_peb_count); i++) {
@@ -611,8 +621,12 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 		if (fm_pos >= fm_size)
 			goto fail_bad;
 
-		if (be32_to_cpu(fmvhdr->magic) != UBI_FM_VHDR_MAGIC)
+		if (be32_to_cpu(fmvhdr->magic) != UBI_FM_VHDR_MAGIC) {
+			ubi_err("bad fastmap vol header magic: 0x%x, "
+				"expected: 0x%x",
+				be32_to_cpu(fmvhdr->magic), UBI_FM_VHDR_MAGIC);
 			goto fail_bad;
+		}
 
 		av = add_vol(ai, be32_to_cpu(fmvhdr->vol_id),
 			     be32_to_cpu(fmvhdr->used_ebs),
@@ -633,8 +647,12 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 		if (fm_pos >= fm_size)
 			goto fail_bad;
 
-		if (be32_to_cpu(fm_eba->magic) != UBI_FM_EBA_MAGIC)
+		if (be32_to_cpu(fm_eba->magic) != UBI_FM_EBA_MAGIC) {
+			ubi_err("bad fastmap EBA header magic: 0x%x, "
+				"expected: 0x%x",
+				be32_to_cpu(fm_eba->magic), UBI_FM_EBA_MAGIC);
 			goto fail_bad;
+		}
 
 		for (j = 0; j < be32_to_cpu(fm_eba->reserved_pebs); j++) {
 			int pnum = be32_to_cpu(fm_eba->pnum[j]);
@@ -694,6 +712,7 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 			int err;
 
 			if (ubi_io_is_bad(ubi, tmp_aeb->pnum)) {
+				ubi_err("bad PEB in fastmap EBA orphan list");
 				ret = UBI_BAD_FASTMAP;
 				kfree(ech);
 				goto fail;
@@ -701,7 +720,8 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 
 			err = ubi_io_read_ec_hdr(ubi, tmp_aeb->pnum, ech, 0);
 			if (err && err != UBI_IO_BITFLIPS) {
-				dbg_bld("unable to read EC header!");
+				ubi_err("unable to read EC header! PEB:%i "
+					"err:%i", tmp_aeb->pnum, err);
 				ret = err > 0 ? UBI_BAD_FASTMAP : err;
 				kfree(ech);
 
@@ -841,14 +861,8 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 		fm->to_be_tortured[0] = 1;
 
 	if (be32_to_cpu(fmsb->magic) != UBI_FM_SB_MAGIC) {
-		/* TODO: not urgent, but examine all the error messages and
-		 * print more information there. Here you should print what was
-		 * read and what was expected. See io.c and do similarly or
-		 * better.
-		 * Please, change globally. E.g., when you print about bad
-		 * version - print what was expected and what was actually
-		 * found. */
-		ubi_err("super block magic does not match");
+		ubi_err("bad super block magic: 0x%x, expected: 0x%x",
+			be32_to_cpu(fmsb->magic), UBI_FM_SB_MAGIC);
 		ret = UBI_BAD_FASTMAP;
 		kfree(fmsb);
 		kfree(fm);
@@ -856,7 +870,8 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 	}
 
 	if (fmsb->version != UBI_FM_FMT_VERSION) {
-		ubi_err("unknown fastmap format version!");
+		ubi_err("bad fastmap version: %i, expected: %i",
+			fmsb->version, UBI_FM_FMT_VERSION);
 		ret = UBI_BAD_FASTMAP;
 		kfree(fmsb);
 		kfree(fm);
@@ -865,7 +880,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 
 	used_blocks = be32_to_cpu(fmsb->used_blocks);
 	if (used_blocks > UBI_FM_MAX_BLOCKS || used_blocks < 1) {
-		ubi_err("number of fastmap blocks is invalid");
+		ubi_err("number of fastmap blocks is invalid: %i", used_blocks);
 		ret = UBI_BAD_FASTMAP;
 		kfree(fmsb);
 		kfree(fm);
@@ -942,6 +957,10 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 
 		if (i == 0) {
 			if (be32_to_cpu(vh->vol_id) != UBI_FM_SB_VOLUME_ID) {
+				ubi_err("bad fastmap anchor vol_id: 0x%x,"
+					" expected: 0x%x",
+					be32_to_cpu(vh->vol_id),
+					UBI_FM_SB_VOLUME_ID);
 				ret = UBI_BAD_FASTMAP;
 				kfree(fmsb);
 				kfree(fm);
@@ -949,6 +968,10 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 			}
 		} else {
 			if (be32_to_cpu(vh->vol_id) != UBI_FM_DATA_VOLUME_ID) {
+				ubi_err("bad fastmap data vol_id: 0x%x,"
+					" expected: 0x%x",
+					be32_to_cpu(vh->vol_id),
+					UBI_FM_DATA_VOLUME_ID);
 				ret = UBI_BAD_FASTMAP;
 				kfree(fmsb);
 				goto free_hdr;
@@ -961,8 +984,8 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 		ret = ubi_io_read(ubi, fm_raw + (ubi->leb_size * i), pnum,
 				  ubi->leb_start, ubi->leb_size);
 		if (ret && ret != UBI_IO_BITFLIPS) {
-			ubi_err("unable to read fastmap block# %i (PEB: %i)",
-				i, pnum);
+			ubi_err("unable to read fastmap block# %i (PEB: %i, "
+				"err: %i)", i, pnum, ret);
 			kfree(fmsb);
 			kfree(fm);
 			goto free_hdr;
-- 
1.7.6.5

[PATCH 10/16] UBI: Fastmap: More kernel doc updates

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |   34 ++++++++++++++++++++++++++++++----
 1 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 07ac06e..c234d94 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -20,6 +20,9 @@
  * new_fm_vhdr - allocate a new volume header for fastmap usage.
  * @ubi: UBI device description object
  * @vol_id: the VID of the new header
+ *
+ * Returns a new struct ubi_vid_hdr on success.
+ * NULL indicates out of memory.
  */
 static struct ubi_vid_hdr *new_fm_vhdr(struct ubi_device *ubi, int vol_id)
 {
@@ -48,6 +51,8 @@ out:
  * @pnum: PEB number of the new attach erase block
  * @ec: erease counter of the new LEB
  * @scrub: scrub this PEB after attaching
+ *
+ * Returns 0 on success, < 0 indicates an internal error.
  */
 static int add_aeb(struct ubi_attach_info *ai, struct list_head *list,
 		   int pnum, int ec, int scrub)
@@ -79,13 +84,16 @@ static int add_aeb(struct ubi_attach_info *ai, struct list_head *list,
 }
 
 /**
- * add_vol - create and add a new scan volume to ubi_attach_info.
+ * add_vol - create and add a new volume to ubi_attach_info.
  * @ai: ubi_attach_info object
  * @vol_id: VID of the new volume
  * @used_ebs: number of used EBS
  * @data_pad: data padding value of the new volume
  * @vol_type: volume type
  * @last_eb_bytes: number of bytes in the last LEB
+ *
+ * Returns the new struct ubi_ainf_volume on success.
+ * NULL indicates an error.
  */
 static struct ubi_ainf_volume *add_vol(struct ubi_attach_info *ai, int vol_id,
 				       int used_ebs, int data_pad, u8 vol_type,
@@ -170,6 +178,8 @@ static void assign_aeb_to_av(struct ubi_attach_info *ai,
  * @av: the volume this LEB belongs to
  * @new_vh: the volume header derived from new_aeb
  * @new_aeb: the AEB to be examined
+ *
+ * Returns 0 on success, < 0 indicates an internal error.
  */
 static int update_vol(struct ubi_device *ubi, struct ubi_attach_info *ai,
 		      struct ubi_ainf_volume *av, struct ubi_vid_hdr *new_vh,
@@ -264,6 +274,8 @@ static int update_vol(struct ubi_device *ubi, struct ubi_attach_info *ai,
  * @ai: attach info object
  * @new_vh: the volume header derived from new_aeb
  * @new_aeb: the AEB to be examined
+ *
+ * Returns 0 on success, < 0 indicates an internal error.
  */
 static int process_pool_aeb(struct ubi_device *ubi, struct ubi_attach_info *ai,
 			    struct ubi_vid_hdr *new_vh,
@@ -345,6 +357,9 @@ static void unmap_peb(struct ubi_attach_info *ai, int pnum)
  * @max_sqnum: pointer to the maximal sequence number
  * @eba_orphans: list of PEBs which need to be scanned
  * @free: list of PEBs which are most likely free (and go into @ai->free)
+ *
+ * Returns 0 on success, if the pool is unusable UBI_BAD_FASTMAP is returned.
+ * < 0 indicates an internal error.
  */
 static int scan_pool(struct ubi_device *ubi, struct ubi_attach_info *ai,
 		     int *pebs, int pool_size, unsigned long long *max_sqnum,
@@ -493,6 +508,9 @@ static int count_fastmap_pebs(struct ubi_attach_info *ai)
  * @ai: UBI attach info object
  * @fm_raw: the fastmap it self as byte array
  * @fm_size: size of the fastmap in bytes
+ *
+ * Returns 0 on success, UBI_BAD_FASTMAP if the found fastmap was unusable.
+ * < 0 indicates an internal error.
  */
 static int ubi_attach_fastmap(struct ubi_device *ubi,
 			      struct ubi_attach_info *ai,
@@ -817,9 +835,9 @@ out:
  * @ubi: UBI device object
  * @ai: UBI attach info to be filled
  *
- * TODO: not urgent, but it is desireble to document error codes in the header
- * comments and probably describe what the function does, if there is something
- * to say (globally).
+ * Returns 0 on success, UBI_NO_FASTMAP if no fastmap was found,
+ * UBI_BAD_FASTMAP if one was found but is not usable.
+ * < 0 indicates an internal error.
  */
 int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 {
@@ -1068,6 +1086,8 @@ out:
  * ubi_write_fastmap - writes a fastmap.
  * @ubi: UBI device object
  * @new_fm: the to be written fastmap
+ *
+ * Returns 0 on success, < 0 indicates an internal error.
  */
 static int ubi_write_fastmap(struct ubi_device *ubi,
 			     struct ubi_fastmap_layout *new_fm)
@@ -1300,6 +1320,8 @@ out:
  * erase_block - Manually erase a PEB.
  * @ubi: UBI device object
  * @pnum: PEB to be erased
+ *
+ * Returns the new EC value on success, < 0 indicates an internal error.
  */
 static int erase_block(struct ubi_device *ubi, int pnum)
 {
@@ -1345,6 +1367,8 @@ out:
  * invalidate_fastmap - destroys a fastmap.
  * @ubi: UBI device object
  * @fm: the fastmap to be destroyed
+ *
+ * Returns 0 on success, < 0 indicates an internal error.
  */
 static int invalidate_fastmap(struct ubi_device *ubi,
 			      struct ubi_fastmap_layout *fm)
@@ -1376,6 +1400,8 @@ static int invalidate_fastmap(struct ubi_device *ubi,
  * ubi_update_fastmap - will be called by UBI if a volume changes or
  * a fastmap pool becomes full.
  * @ubi: UBI device object
+ *
+ * Returns 0 on success, < 0 indicates an internal error.
  */
 int ubi_update_fastmap(struct ubi_device *ubi)
 {
-- 
1.7.6.5

[PATCH 11/16] UBI: Fastmap: Store pool sizes in fastmap

[Richard Weinberger]

Later this can be used by ubinize to allow custom
pool sizes.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/attach.c    |    8 ++++
 drivers/mtd/ubi/build.c     |    4 +-
 drivers/mtd/ubi/fastmap.c   |   84 ++++++++++++++++++++++++++++--------------
 drivers/mtd/ubi/ubi-media.h |    4 ++-
 drivers/mtd/ubi/ubi.h       |    6 +++
 5 files changed, 75 insertions(+), 31 deletions(-)

diff --git a/drivers/mtd/ubi/attach.c b/drivers/mtd/ubi/attach.c
index 1ac58ec..7552d25 100644
--- a/drivers/mtd/ubi/attach.c
+++ b/drivers/mtd/ubi/attach.c
@@ -1261,6 +1261,14 @@ int ubi_attach(struct ubi_device *ubi, int force_scan)
 	if (err)
 		goto out_ai;
 
+	if (ubi->fm) {
+		ubi->fm_pool.max_size = ubi->fm->max_pool_size;
+		ubi->fm_wl_pool.max_size = ubi->fm->max_wl_pool_size;
+
+		ubi_msg("fastmap pool size: %d", ubi->fm_pool.max_size);
+		ubi_msg("fastmap WL pool size: %d", ubi->fm_wl_pool.max_size);
+	}
+
 	err = ubi_wl_init(ubi, ai);
 	if (err)
 		goto out_vtbl;
diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
index d00101e..7094550 100644
--- a/drivers/mtd/ubi/build.c
+++ b/drivers/mtd/ubi/build.c
@@ -900,8 +900,8 @@ int ubi_attach_mtd_dev(struct mtd_info *mtd, int ubi_num, int vid_hdr_offset)
 
 	ubi->fm_wl_pool.max_size = UBI_FM_WL_POOL_SIZE;
 
-	ubi_msg("fastmap pool size: %d", ubi->fm_pool.max_size);
-	ubi_msg("fastmap WL pool size: %d", ubi->fm_wl_pool.max_size);
+	ubi_msg("default fastmap pool size: %d", ubi->fm_pool.max_size);
+	ubi_msg("default fastmap WL pool size: %d", ubi->fm_wl_pool.max_size);
 
 	mutex_init(&ubi->buf_mutex);
 	mutex_init(&ubi->ckvol_mutex);
diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index c234d94..e285e3b 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -506,15 +506,14 @@ static int count_fastmap_pebs(struct ubi_attach_info *ai)
  * ubi_attach_fastmap - creates ubi_attach_info from a fastmap.
  * @ubi: UBI device object
  * @ai: UBI attach info object
- * @fm_raw: the fastmap it self as byte array
- * @fm_size: size of the fastmap in bytes
+ * @fm: the fastmap to be attached
  *
  * Returns 0 on success, UBI_BAD_FASTMAP if the found fastmap was unusable.
  * < 0 indicates an internal error.
  */
 static int ubi_attach_fastmap(struct ubi_device *ubi,
 			      struct ubi_attach_info *ai,
-			      void *fm_raw, size_t fm_size)
+			      struct ubi_fastmap_layout *fm)
 {
 	struct list_head used, eba_orphans, free;
 	struct ubi_ainf_volume *av;
@@ -526,9 +525,10 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 	struct ubi_fm_ec *fmec;
 	struct ubi_fm_volhdr *fmvhdr;
 	struct ubi_fm_eba *fm_eba;
-	int ret, i, j;
-	size_t fm_pos = 0;
+	int ret, i, j, pool_size, wl_pool_size;
+	size_t fm_pos = 0, fm_size = fm->size;
 	unsigned long long max_sqnum = 0;
+	void *fm_raw = fm->raw;
 
 	INIT_LIST_HEAD(&used);
 	INIT_LIST_HEAD(&free);
@@ -585,6 +585,34 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 		goto fail_bad;
 	}
 
+	pool_size = be16_to_cpu(fmpl1->size);
+	wl_pool_size = be16_to_cpu(fmpl2->size);
+	fm->max_pool_size = be16_to_cpu(fmpl1->max_size);
+	fm->max_wl_pool_size = be16_to_cpu(fmpl2->max_size);
+
+	if (pool_size > UBI_FM_MAX_POOL_SIZE || pool_size < 0) {
+		ubi_err("bad pool size: %i", pool_size);
+		goto fail_bad;
+	}
+
+	if (wl_pool_size > UBI_FM_MAX_POOL_SIZE || wl_pool_size < 0) {
+		ubi_err("bad WL pool size: %i", wl_pool_size);
+		goto fail_bad;
+	}
+
+
+	if (fm->max_pool_size > UBI_FM_MAX_POOL_SIZE ||
+	    fm->max_pool_size < 0) {
+		ubi_err("bad maximal pool size: %i", fm->max_pool_size);
+		goto fail_bad;
+	}
+
+	if (fm->max_wl_pool_size > UBI_FM_MAX_POOL_SIZE ||
+	    fm->max_wl_pool_size < 0) {
+		ubi_err("bad maximal WL pool size: %i", fm->max_wl_pool_size);
+		goto fail_bad;
+	}
+
 	/* read EC values from free list */
 	for (i = 0; i < be32_to_cpu(fmhdr->free_peb_count); i++) {
 		fmec = (struct ubi_fm_ec *)(fm_raw + fm_pos);
@@ -754,13 +782,13 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 		kfree(ech);
 	}
 
-	ret = scan_pool(ubi, ai, fmpl1->pebs, be32_to_cpu(fmpl1->size),
-			&max_sqnum, &eba_orphans, &free);
+	ret = scan_pool(ubi, ai, fmpl1->pebs, pool_size, &max_sqnum,
+			&eba_orphans, &free);
 	if (ret)
 		goto fail;
 
-	ret = scan_pool(ubi, ai, fmpl2->pebs, be32_to_cpu(fmpl2->size),
-			&max_sqnum, &eba_orphans, &free);
+	ret = scan_pool(ubi, ai, fmpl2->pebs, wl_pool_size, &max_sqnum,
+			&eba_orphans, &free);
 	if (ret)
 		goto fail;
 
@@ -772,6 +800,16 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 		list_add_tail(&tmp_aeb->u.list, &ai->free);
 	}
 
+	/*
+	 * If fastmap is leaking PEBs (must not happen), raise a
+	 * fat warning and fall back to scanning mode.
+	 * We do this here because in ubi_wl_init() it's too late
+	 * and we cannot fall back to scanning.
+	 */
+	if (WARN_ON(count_fastmap_pebs(ai) != ubi->peb_count -
+		    ai->bad_peb_count - fm->used_blocks))
+		goto fail_bad;
+
 	return 0;
 
 fail_bad:
@@ -1026,7 +1064,11 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 
 	fmsb->sqnum = sqnum;
 
-	ret = ubi_attach_fastmap(ubi, ai, fm_raw, fm_size);
+	fm->size = fm_size;
+	fm->used_blocks = used_blocks;
+	fm->raw = fm_raw;
+
+	ret = ubi_attach_fastmap(ubi, ai, fm);
 	if (ret) {
 		if (ret > 0)
 			ret = UBI_BAD_FASTMAP;
@@ -1034,22 +1076,6 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 		goto free_hdr;
 	}
 
-	/*
-	 * If fastmap is leaking PEBs (must not happen), raise a
-	 * fat warning and fall back to scanning mode.
-	 * We do this here because in ubi_wl_init() it's too late
-	 * and we cannot fall back to scanning.
-	 */
-	if (WARN_ON(count_fastmap_pebs(ai) != ubi->peb_count -
-		    ai->bad_peb_count - used_blocks)) {
-		ret = UBI_BAD_FASTMAP;
-		kfree(fm);
-		goto free_hdr;
-	}
-
-	fm->size = fm_size;
-	fm->used_blocks = used_blocks;
-
 	for (i = 0; i < used_blocks; i++) {
 		struct ubi_wl_entry *e;
 
@@ -1153,7 +1179,8 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 	fmpl1 = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
 	fm_pos += sizeof(*fmpl1);
 	fmpl1->magic = cpu_to_be32(UBI_FM_POOL_MAGIC);
-	fmpl1->size = cpu_to_be32(ubi->fm_pool.size);
+	fmpl1->size = cpu_to_be16(ubi->fm_pool.size);
+	fmpl1->max_size = cpu_to_be16(ubi->fm_pool.max_size);
 
 	for (i = 0; i < ubi->fm_pool.size; i++)
 		fmpl1->pebs[i] = cpu_to_be32(ubi->fm_pool.pebs[i]);
@@ -1161,7 +1188,8 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 	fmpl2 = (struct ubi_fm_scan_pool *)(fm_raw + fm_pos);
 	fm_pos += sizeof(*fmpl2);
 	fmpl2->magic = cpu_to_be32(UBI_FM_POOL_MAGIC);
-	fmpl2->size = cpu_to_be32(ubi->fm_wl_pool.size);
+	fmpl2->size = cpu_to_be16(ubi->fm_wl_pool.size);
+	fmpl2->max_size = cpu_to_be16(ubi->fm_wl_pool.max_size);
 
 	for (i = 0; i < ubi->fm_wl_pool.size; i++)
 		fmpl2->pebs[i] = cpu_to_be32(ubi->fm_wl_pool.pebs[i]);
diff --git a/drivers/mtd/ubi/ubi-media.h b/drivers/mtd/ubi/ubi-media.h
index eaf81a2..f1b85a4f 100644
--- a/drivers/mtd/ubi/ubi-media.h
+++ b/drivers/mtd/ubi/ubi-media.h
@@ -455,11 +455,13 @@ struct ubi_fm_hdr {
  * struct ubi_fm_scan_pool - Fastmap pool PEBs to be scanned while attaching
  * @magic: pool magic numer (%UBI_FM_POOL_MAGIC)
  * @size: current pool size
+ * @max_size: maximal pool size
  * @pebs: an array containing the location of all PEBs in this pool
  */
 struct ubi_fm_scan_pool {
 	__be32 magic;
-	__be32 size;
+	__be16 size;
+	__be16 max_size;
 	__be32 pebs[UBI_FM_MAX_POOL_SIZE];
 	__be32 padding[4];
 } __packed;
diff --git a/drivers/mtd/ubi/ubi.h b/drivers/mtd/ubi/ubi.h
index b60818d..8e2592d 100644
--- a/drivers/mtd/ubi/ubi.h
+++ b/drivers/mtd/ubi/ubi.h
@@ -218,12 +218,18 @@ struct ubi_volume_desc;
  * @to_be_tortured: if non-zero tortured this PEB
  * @size: size of the fastmap in bytes
  * @used_blocks: number of used PEBs
+ * @max_pool_size: maximal size of the user pool
+ * @max_wl_pool_size: maximal size of the pooly used by the WL sub-system
+ * @raw: the fastmap itself as byte array (only valid while attaching)
  */
 struct ubi_fastmap_layout {
 	struct ubi_wl_entry *e[UBI_FM_MAX_BLOCKS];
 	int to_be_tortured[UBI_FM_MAX_BLOCKS];
 	size_t size;
 	int used_blocks;
+	int max_pool_size;
+	int max_wl_pool_size;
+	void *raw;
 };
 
 /**
-- 
1.7.6.5

[PATCH 12/16] UBI: Fastmap: Make checkpatch.pl happy

[Richard Weinberger]

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 drivers/mtd/ubi/fastmap.c |   21 ++++++++++++---------
 drivers/mtd/ubi/wl.c      |   17 ++++++++---------
 2 files changed, 20 insertions(+), 18 deletions(-)

diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index e285e3b..2c7c350 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -668,7 +668,7 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 			goto fail_bad;
 
 		if (be32_to_cpu(fmvhdr->magic) != UBI_FM_VHDR_MAGIC) {
-			ubi_err("bad fastmap vol header magic: 0x%x, "
+			ubi_err("bad fastmap vol header magic: 0x%x, " \
 				"expected: 0x%x",
 				be32_to_cpu(fmvhdr->magic), UBI_FM_VHDR_MAGIC);
 			goto fail_bad;
@@ -694,7 +694,7 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 			goto fail_bad;
 
 		if (be32_to_cpu(fm_eba->magic) != UBI_FM_EBA_MAGIC) {
-			ubi_err("bad fastmap EBA header magic: 0x%x, "
+			ubi_err("bad fastmap EBA header magic: 0x%x, " \
 				"expected: 0x%x",
 				be32_to_cpu(fm_eba->magic), UBI_FM_EBA_MAGIC);
 			goto fail_bad;
@@ -766,7 +766,7 @@ static int ubi_attach_fastmap(struct ubi_device *ubi,
 
 			err = ubi_io_read_ec_hdr(ubi, tmp_aeb->pnum, ech, 0);
 			if (err && err != UBI_IO_BITFLIPS) {
-				ubi_err("unable to read EC header! PEB:%i "
+				ubi_err("unable to read EC header! PEB:%i " \
 					"err:%i", tmp_aeb->pnum, err);
 				ret = err > 0 ? UBI_BAD_FASTMAP : err;
 				kfree(ech);
@@ -1013,7 +1013,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 
 		if (i == 0) {
 			if (be32_to_cpu(vh->vol_id) != UBI_FM_SB_VOLUME_ID) {
-				ubi_err("bad fastmap anchor vol_id: 0x%x,"
+				ubi_err("bad fastmap anchor vol_id: 0x%x," \
 					" expected: 0x%x",
 					be32_to_cpu(vh->vol_id),
 					UBI_FM_SB_VOLUME_ID);
@@ -1024,7 +1024,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 			}
 		} else {
 			if (be32_to_cpu(vh->vol_id) != UBI_FM_DATA_VOLUME_ID) {
-				ubi_err("bad fastmap data vol_id: 0x%x,"
+				ubi_err("bad fastmap data vol_id: 0x%x," \
 					" expected: 0x%x",
 					be32_to_cpu(vh->vol_id),
 					UBI_FM_DATA_VOLUME_ID);
@@ -1040,7 +1040,7 @@ int ubi_scan_fastmap(struct ubi_device *ubi, struct ubi_attach_info *ai)
 		ret = ubi_io_read(ubi, fm_raw + (ubi->leb_size * i), pnum,
 				  ubi->leb_start, ubi->leb_size);
 		if (ret && ret != UBI_IO_BITFLIPS) {
-			ubi_err("unable to read fastmap block# %i (PEB: %i, "
+			ubi_err("unable to read fastmap block# %i (PEB: %i, " \
 				"err: %i)", i, pnum, ret);
 			kfree(fmsb);
 			kfree(fm);
@@ -1305,7 +1305,8 @@ static int ubi_write_fastmap(struct ubi_device *ubi,
 	}
 
 	fmsb->data_crc = 0;
-	fmsb->data_crc = cpu_to_be32(crc32(UBI_CRC32_INIT, fm_raw, new_fm->size));
+	fmsb->data_crc = cpu_to_be32(crc32(UBI_CRC32_INIT, fm_raw,
+					   new_fm->size));
 
 	for (i = 1; i < new_fm->used_blocks; i++) {
 		dvhdr->sqnum = cpu_to_be64(ubi_next_sqnum(ubi));
@@ -1510,7 +1511,8 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 				int j;
 
 				for (j = 1; j < i; j++)
-					ubi_wl_put_fm_peb(ubi, new_fm->e[j], j, 0);
+					ubi_wl_put_fm_peb(ubi, new_fm->e[j],
+							  j, 0);
 
 				ubi_err("could not erase old fastmap PEB");
 				goto err;
@@ -1541,7 +1543,8 @@ int ubi_update_fastmap(struct ubi_device *ubi)
 				ubi_err("could not erase old anchor PEB");
 
 				for (i = 1; i < new_fm->used_blocks; i++)
-					ubi_wl_put_fm_peb(ubi, new_fm->e[i], i, 0);
+					ubi_wl_put_fm_peb(ubi, new_fm->e[i],
+							  i, 0);
 				goto err;
 			}
 
diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index 06bf985..28385d2 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -349,8 +349,8 @@ static void prot_queue_add(struct ubi_device *ubi, struct ubi_wl_entry *e)
  * This function looks for a wear leveling entry with erase counter closest to
  * min + @diff, where min is the smallest erase counter.
  */
-static struct ubi_wl_entry *find_wl_entry(struct ubi_device *ubi, struct rb_root *root,
-					  int diff)
+static struct ubi_wl_entry *find_wl_entry(struct ubi_device *ubi,
+					  struct rb_root *root, int diff)
 {
 	struct rb_node *p;
 	struct ubi_wl_entry *e, *prev_e = NULL;
@@ -403,12 +403,12 @@ static struct ubi_wl_entry *find_mean_wl_entry(struct ubi_device *ubi,
 		e = rb_entry(root->rb_node, struct ubi_wl_entry, u.rb);
 
 		/* If no fastmap has been written and this WL entry can be used
-		 * as anchor PEB, hold it back and return the second best WL entry
-		 * such that fastmap can use the anchor PEB later. */
+		 * as anchor PEB, hold it back and return the second best
+		 * WL entry such that fastmap can use the anchor PEB later. */
 		if (e && !ubi->fm && e->pnum < UBI_FM_MAX_START)
-			e = rb_entry(rb_next(root->rb_node), struct ubi_wl_entry, u.rb);
-	}
-	else
+			e = rb_entry(rb_next(root->rb_node),
+				     struct ubi_wl_entry, u.rb);
+	} else
 		e = find_wl_entry(ubi, root, WL_FREE_MAX_DIFF/2);
 
 	return e;
@@ -997,8 +997,7 @@ static int wear_leveling_worker(struct ubi_device *ubi, struct ubi_work *wrk,
 		self_check_in_wl_tree(ubi, e1, &ubi->used);
 		rb_erase(&e1->u.rb, &ubi->used);
 		dbg_wl("anchor-move PEB %d to PEB %d", e1->pnum, e2->pnum);
-	}
-	else if (!ubi->scrub.rb_node) {
+	} else if (!ubi->scrub.rb_node) {
 		/*
 		 * Now pick the least worn-out used physical eraseblock and a
 		 * highly worn-out free physical eraseblock. If the erase
-- 
1.7.6.5

[PATCH 13/16] UBI: Fastmap: Remove one point from TODO list

[Richard Weinberger]

fastmap works fine with ro mtd devices.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 TODO |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/TODO b/TODO
index a944159..b1ae872 100644
--- a/TODO
+++ b/TODO
@@ -4,7 +4,6 @@ it is merged.
 For all or most of the items there we need to write a testcase. We can put it
 to the ubi-utils.git repository, to a separate branch at the beginning
 
-1. Test with a R/O MTD device: !(mtd->flags & MTD_WRITEABLE)
 2. Implement power-cut emulation infrastructure similar to what is in UBIFS and
    test UBI + fastmap with it.
 3. Test the autoresize feature
-- 
1.7.6.5

[PATCH 14/16] UBI: Fastmap: Remove one point from TODO list

[Richard Weinberger]

autoresize works too.
It was tested with ubirsvol and ubinize which triggered
an autoresize.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 TODO |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/TODO b/TODO
index b1ae872..7386d3b 100644
--- a/TODO
+++ b/TODO
@@ -6,7 +6,6 @@ to the ubi-utils.git repository, to a separate branch at the beginning
 
 2. Implement power-cut emulation infrastructure similar to what is in UBIFS and
    test UBI + fastmap with it.
-3. Test the autoresize feature
 4. Test 'ubi_flush()'
 5. Test that the same UBI image works fine on both LE and BE machines. I guess
    we can do this using sime kind of emulators?
-- 
1.7.6.5

[PATCH 15/16] UBI: Fastmap: Remove one point from TODO list

[Richard Weinberger]

ubi_flush() ahas been tested more than once.
In early versions fastmap called it itself

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 TODO |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/TODO b/TODO
index 7386d3b..267b8d3 100644
--- a/TODO
+++ b/TODO
@@ -6,6 +6,5 @@ to the ubi-utils.git repository, to a separate branch at the beginning
 
 2. Implement power-cut emulation infrastructure similar to what is in UBIFS and
    test UBI + fastmap with it.
-4. Test 'ubi_flush()'
 5. Test that the same UBI image works fine on both LE and BE machines. I guess
    we can do this using sime kind of emulators?
-- 
1.7.6.5

[PATCH 16/16] UBI: Fastmap: Remove one point from TODO list

[Richard Weinberger]

BE/LE has been tested using qemu-ppc and an x86 box.

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 TODO |    2 --
 1 files changed, 0 insertions(+), 2 deletions(-)

diff --git a/TODO b/TODO
index 267b8d3..12a019a 100644
--- a/TODO
+++ b/TODO
@@ -6,5 +6,3 @@ to the ubi-utils.git repository, to a separate branch at the beginning
 
 2. Implement power-cut emulation infrastructure similar to what is in UBIFS and
    test UBI + fastmap with it.
-5. Test that the same UBI image works fine on both LE and BE machines. I guess
-   we can do this using sime kind of emulators?
-- 
1.7.6.5