[Buildroot] [pull request v4] Pull request for branch yem-ext234

[Buildroot] [pull request v4] Pull request for branch yem-ext234

[Yann E. MORIN]

Hello All!

This patch series enhances the ext2 fs backend to also build ext3 and ext4
filesystems, and differentiate between ext2 rev0 and ext2 rev1 filesystems.


Changes v3 -> v4:
  - rebased ontop master for Peter

Changes v2 -> v3:
  - cleanup ext2 dependencies  (Thomas)
  - explain UUID removal  (Thomas)
  - use post-hook to create "ext# -> ext2" symlink  (Thomas, Arnout)
  - drop patch to let fs name the image file extension  (Thomas, Arnout)

Changes v1 -> v2:
  - enhance the ext2/3/4 generation  (Arnout, Peter)
  - allow fs generators to specify the image extension  (Arnout)
  - remove ext2 count- and time-based fsck  (Arnout)


The following changes since commit 2c4ff68e08e063152245d0a1b211df7a4360f925:

  system/securetty: Add ttyPS[0-1] (2013-04-11 23:32:11 +0200)

are available in the git repository at:
  git://gitorious.org/buildroot/buildroot.git yem-ext234

Yann E. MORIN (5):
      fs/ext2: enclose the ext2 options in if...endif
      fs/ext2: add ability to build ext3/4 too
      fs/ext2: add a symlink 'rootfs.ext#' -> 'rootfs.ext2'
      fs/ext2: add support for ext2 rev0 and rev1
      fs/ext2: remove count- and time-based fsck

 fs/ext2/Config.in                |   57 +++++++++++++++++++++-----
 fs/ext2/ext2.mk                  |   14 ++++++-
 fs/ext2/genext2fs.sh             |   82 +++++++++++++++++++++++++++++++++++++-
 package/e2fsprogs/Config.in.host |    2 +-
 4 files changed, 140 insertions(+), 15 deletions(-)

Regards,
Yann E. MORIN

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/5] fs/ext2: enclose the ext2 options in if...endif

[Yann E. MORIN]

Use a 'if...endif' construct instead of repeating the
'depends on' for each symbols.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: Arnout Vandecappelle <arnout@mind.be>
Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 fs/ext2/Config.in |    7 +++----
 1 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/fs/ext2/Config.in b/fs/ext2/Config.in
index 2af44a8..cb4beed 100644
--- a/fs/ext2/Config.in
+++ b/fs/ext2/Config.in
@@ -3,25 +3,23 @@ config BR2_TARGET_ROOTFS_EXT2
 	help
 	  Build an ext2 root filesystem
 
+if BR2_TARGET_ROOTFS_EXT2
+
 config BR2_TARGET_ROOTFS_EXT2_BLOCKS
 	int "size in blocks (leave at 0 for auto calculation)"
-	depends on BR2_TARGET_ROOTFS_EXT2
 	default 0
 
 config BR2_TARGET_ROOTFS_EXT2_INODES
 	int "inodes (leave at 0 for auto calculation)"
-	depends on BR2_TARGET_ROOTFS_EXT2
 	default 0
 
 config BR2_TARGET_ROOTFS_EXT2_RESBLKS
 	int "reserved blocks percentage"
-	depends on BR2_TARGET_ROOTFS_EXT2
 	default 0
 
 choice
 	prompt "Compression method"
 	default BR2_TARGET_ROOTFS_EXT2_NONE
-	depends on BR2_TARGET_ROOTFS_EXT2
 	help
 	  Select compressor for ext2 filesystem of the root filesystem
 
@@ -47,3 +45,4 @@ config BR2_TARGET_ROOTFS_EXT2_LZMA
 
 endchoice
 
+endif # BR2_TARGET_ROOTFS_EXT2
-- 
1.7.2.5

[Buildroot] [PATCH 2/5] fs/ext2: add ability to build ext3/4 too

[Yann E. MORIN]

Use the host-e2fsprogs to tune2fs the generated rootfs.ext2 image,
and upgrade it to either one of ext2, ext3 or ext4.

Since calling tune2fs may require running e2fsck (tune2fs will warn
to do so when certain FS options are changed), we systematically call
e2fsck. This makes the code path simpler, and as a side-effect checks
that genext2fs did not generate garbage.

In turn, e2fsck will unconditionally add a UUID to the filesystem,
which is bad for reproducibility, so we call tune2fs again to remove
the UUID. This does not require checking the filesystem.

To ensure compatibility of Buildroot's .config, leave ext2 as the
default. Boards' .config can override this at will.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <jacmet@uclibc.org>

---
Changes v3 -> v2:
  - quiesce e2fsck a bit

Changes v2 -> v3:
  - fix dependencies (Thomas)

Changes v1->v2: (all after comments by Arnout)
  - menuconfig prompt rework (& Peter)
  - cleanup ext2.mk
  - use environment to pass ext generation/variant
  - add rational for 1081 additional blocks
  - explain the reason we run e2fsck
---
 fs/ext2/Config.in                |   36 +++++++++++++++----
 fs/ext2/ext2.mk                  |    6 ++-
 fs/ext2/genext2fs.sh             |   73 +++++++++++++++++++++++++++++++++++++-
 package/e2fsprogs/Config.in.host |    2 +-
 4 files changed, 106 insertions(+), 11 deletions(-)

diff --git a/fs/ext2/Config.in b/fs/ext2/Config.in
index cb4beed..c8320e2 100644
--- a/fs/ext2/Config.in
+++ b/fs/ext2/Config.in
@@ -1,10 +1,32 @@
 config BR2_TARGET_ROOTFS_EXT2
-	bool "ext2 root filesystem"
+	bool "ext2/3/4 root filesystem"
+	select BR2_PACKAGE_HOST_E2FSPROGS
 	help
-	  Build an ext2 root filesystem
+	  Build an ext2/3/4 root filesystem
 
 if BR2_TARGET_ROOTFS_EXT2
 
+choice
+	bool "ext2/3/4 variant"
+	default BR2_TARGET_ROOTFS_EXT2_2
+
+config BR2_TARGET_ROOTFS_EXT2_2
+	bool "ext2"
+
+config BR2_TARGET_ROOTFS_EXT2_3
+	bool "ext3"
+
+config BR2_TARGET_ROOTFS_EXT2_4
+	bool "ext4"
+
+endchoice
+
+config BR2_TARGET_ROOTFS_EXT2_GEN
+	int
+	default 2 if BR2_TARGET_ROOTFS_EXT2_2
+	default 3 if BR2_TARGET_ROOTFS_EXT2_3
+	default 4 if BR2_TARGET_ROOTFS_EXT2_4
+
 config BR2_TARGET_ROOTFS_EXT2_BLOCKS
 	int "size in blocks (leave at 0 for auto calculation)"
 	default 0
@@ -21,27 +43,27 @@ choice
 	prompt "Compression method"
 	default BR2_TARGET_ROOTFS_EXT2_NONE
 	help
-	  Select compressor for ext2 filesystem of the root filesystem
+	  Select compressor for ext2/3/4 filesystem of the root filesystem
 
 config BR2_TARGET_ROOTFS_EXT2_NONE
 	bool "no compression"
 	help
-	  Do not compress the ext2 filesystem.
+	  Do not compress the ext2/3/4 filesystem.
 
 config BR2_TARGET_ROOTFS_EXT2_GZIP
 	bool "gzip"
 	help
-	  Do compress the ext2 filesystem with gzip.
+	  Do compress the ext2/3/4 filesystem with gzip.
 
 config BR2_TARGET_ROOTFS_EXT2_BZIP2
 	bool "bzip2"
 	help
-	  Do compress the ext2 filesystem with bzip2.
+	  Do compress the ext2/3/4 filesystem with bzip2.
 
 config BR2_TARGET_ROOTFS_EXT2_LZMA
 	bool "lzma"
 	help
-	  Do compress the ext2 filesystem with lzma.
+	  Do compress the ext2/3/4 filesystem with lzma.
 
 endchoice
 
diff --git a/fs/ext2/ext2.mk b/fs/ext2/ext2.mk
index 7b71592..018a5a3 100644
--- a/fs/ext2/ext2.mk
+++ b/fs/ext2/ext2.mk
@@ -18,10 +18,12 @@ ifneq ($(strip $(BR2_TARGET_ROOTFS_EXT2_RESBLKS)),0)
 EXT2_OPTS += -m $(BR2_TARGET_ROOTFS_EXT2_RESBLKS)
 endif
 
-ROOTFS_EXT2_DEPENDENCIES = host-genext2fs
+ROOTFS_EXT2_DEPENDENCIES = host-genext2fs host-e2fsprogs
+
+EXT2_ENV = GEN=$(BR2_TARGET_ROOTFS_EXT2_GEN)
 
 define ROOTFS_EXT2_CMD
-	PATH=$(TARGET_PATH) fs/ext2/genext2fs.sh -d $(TARGET_DIR) $(EXT2_OPTS) $@
+	PATH=$(TARGET_PATH) $(EXT2_ENV) fs/ext2/genext2fs.sh -d $(TARGET_DIR) $(EXT2_OPTS) $@
 endef
 
 $(eval $(call ROOTFS_TARGET,ext2))
diff --git a/fs/ext2/genext2fs.sh b/fs/ext2/genext2fs.sh
index 7a518ae..c2049f5 100755
--- a/fs/ext2/genext2fs.sh
+++ b/fs/ext2/genext2fs.sh
@@ -1,10 +1,13 @@
 #!/bin/sh
 # genext2fs wrapper calculating needed blocks/inodes values if not specified
+set -e
 
 export LC_ALL=C
 
 CALC_BLOCKS=1
 CALC_INODES=1
+EXT_OPTS=
+EXT_OPTS_O=
 
 while getopts x:d:D:b:i:N:m:g:e:zfqUPhVv f
 do
@@ -14,6 +17,7 @@ do
 	d) TARGET_DIR=$OPTARG ;;
     esac
 done
+eval IMG="\"\${${OPTIND}}\""
 
 # calculate needed inodes
 if [ $CALC_INODES -eq 1 ];
@@ -30,7 +34,74 @@ then
     # we scale inodes / blocks with 10% to compensate for bitmaps size + slack
     BLOCKS=$(du -s -c -k $TARGET_DIR | grep total | sed -e "s/total//")
     BLOCKS=$(expr 500 + \( $BLOCKS + $INODES / 8 \) \* 11 / 10)
+    # we add 1081 blocks (a bit more than 1 MiB, assuming 1KiB blocks) for
+    # the journal if ext3/4
+    # Note: I came to 1081 blocks after trial-and-error checks. With 1080 or
+    # less additional blocks, and tune2fs would refuse to add the journal;
+    # with 1081 additional blocks or above, tune2fs wil happily add a journal.
+    # YMMV.
+    if [ ${GEN} -ge 3 ]; then
+        BLOCKS=$(expr 1081 + $BLOCKS )
+    fi
     set -- $@ -b $BLOCKS
 fi
 
-exec genext2fs $@
+e2tunefsck() {
+    # Upgrade the file system
+    if [ $# -ne 0 ]; then
+        tune2fs "$@" "${IMG}"
+    fi
+
+    # After changing filesystem options, running fsck is required
+    # (see: man tune2fs). Running e2fsck in other cases will ensure
+    # coherency of the filesystem, although it is not required.
+    # 'e2fsck -pDf' means:
+    #  - automatically repair
+    #  - optimise and check for duplicate entries
+    #  - force checking
+    # Sending output to oblivion, as e2fsck can be *very* verbose,
+    # especially with filesystems generated by genext2fs.
+    # Exit codes 1 & 2 are OK, it means fs errors were successfully
+    # corrected, hence our little trick with $ret.
+    ret=0
+    e2fsck -pDf "${IMG}" >/dev/null || ret=$?
+    case ${ret} in
+       0|1|2) ;;
+       *)   exit ${ret};;
+    esac
+    printf "\ne2fsck was successfully run on '%s' (ext%d)\n\n"  \
+           "${IMG##*/}" "${GEN}"
+
+    # e2fsck will force a *random* UUID, which is bad
+    # for reproducibility, so we do not want it
+    tune2fs -U clear "${IMG}"
+}
+
+# Check we know what generation to generate
+case "${GEN}" in
+    2|3|4)
+	;;
+    *)
+	printf "%s: unknown ext generation to generate\n" "${0##*/}" >&2
+	exit 1
+	;;
+esac
+
+# Add a journal for ext3 and above
+if [ ${GEN} -ge 3 ]; then
+    EXT_OPTS="${EXT_OPTS} -j -J size=1"
+fi
+
+# Add ext4 specific features
+if [ ${GEN} -ge 4 ]; then
+    EXT_OPTS_O="${EXT_OPTS_O},extents,uninit_bg,dir_index"
+fi
+
+# Add our -O options (there will be at most one leading comma, remove it)
+if [ -n "${EXT_OPTS_O}" ]; then
+    EXT_OPTS="${EXT_OPTS} -O ${EXT_OPTS_O#,}"
+fi
+
+# Generate and upgrade the filesystem
+genext2fs "$@"
+e2tunefsck ${EXT_OPTS}
diff --git a/package/e2fsprogs/Config.in.host b/package/e2fsprogs/Config.in.host
index ea6a0bd..0c001c2 100644
--- a/package/e2fsprogs/Config.in.host
+++ b/package/e2fsprogs/Config.in.host
@@ -1,6 +1,6 @@
 config BR2_PACKAGE_HOST_E2FSPROGS
 	bool "host e2fsprogs"
 	help
-	  The EXT2 file system utilities.
+	  The EXT2/3/4 file system utilities.
 	  
 	  http://e2fsprogs.sf.net
-- 
1.7.2.5

[Buildroot] [PATCH 3/5] fs/ext2: add a symlink 'rootfs.ext#' -> 'rootfs.ext2'

[Yann E. MORIN]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
---
 fs/ext2/ext2.mk |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/fs/ext2/ext2.mk b/fs/ext2/ext2.mk
index 018a5a3..ffb7257 100644
--- a/fs/ext2/ext2.mk
+++ b/fs/ext2/ext2.mk
@@ -26,4 +26,11 @@ define ROOTFS_EXT2_CMD
 	PATH=$(TARGET_PATH) $(EXT2_ENV) fs/ext2/genext2fs.sh -d $(TARGET_DIR) $(EXT2_OPTS) $@
 endef
 
+define ROOTFS_EXT2_HOOK_SYMLINK
+	ln -sf rootfs.ext2 $(BINARIES_DIR)/rootfs.ext$(BR2_TARGET_ROOTFS_EXT2_GEN)
+endef
+ifneq ($(BR2_TARGET_ROOTFS_EXT2_GEN),2)
+ROOTFS_EXT2_POST_GEN_HOOKS += ROOTFS_EXT2_HOOK_SYMLINK
+endif
+
 $(eval $(call ROOTFS_TARGET,ext2))
-- 
1.7.2.5

[Buildroot] [PATCH 4/5] fs/ext2: add support for ext2 rev0 and rev1

[Yann E. MORIN]

Some bootloaders have a buggy ext2 support, and require ext2 rev1
instead of the traditional ext2 rev0 that genext2fs produces.

tune2fs accepts only one '-O list' at a time, so we need to construct
a list of -O options.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

---
Changes v1->v2:
  - separate GEN/REV and use two environment variables (Arnout)
  - all are revision 1, except ext2r0 (Arnout)
---
 fs/ext2/Config.in    |   20 +++++++++++++++++---
 fs/ext2/ext2.mk      |    3 ++-
 fs/ext2/genext2fs.sh |   12 +++++++++---
 3 files changed, 28 insertions(+), 7 deletions(-)

diff --git a/fs/ext2/Config.in b/fs/ext2/Config.in
index c8320e2..097242e 100644
--- a/fs/ext2/Config.in
+++ b/fs/ext2/Config.in
@@ -6,12 +6,20 @@ config BR2_TARGET_ROOTFS_EXT2
 
 if BR2_TARGET_ROOTFS_EXT2
 
+config BR2_TARGET_ROOTFS_EXT2_2
+	bool
+
 choice
 	bool "ext2/3/4 variant"
-	default BR2_TARGET_ROOTFS_EXT2_2
+	default BR2_TARGET_ROOTFS_EXT2_2r0
 
-config BR2_TARGET_ROOTFS_EXT2_2
-	bool "ext2"
+config BR2_TARGET_ROOTFS_EXT2_2r0
+	bool "ext2 (rev0)"
+	select BR2_TARGET_ROOTFS_EXT2_2
+
+config BR2_TARGET_ROOTFS_EXT2_2r1
+	bool "ext2 (rev1)"
+	select BR2_TARGET_ROOTFS_EXT2_2
 
 config BR2_TARGET_ROOTFS_EXT2_3
 	bool "ext3"
@@ -27,6 +35,12 @@ config BR2_TARGET_ROOTFS_EXT2_GEN
 	default 3 if BR2_TARGET_ROOTFS_EXT2_3
 	default 4 if BR2_TARGET_ROOTFS_EXT2_4
 
+# All ext generations are revision 1, except ext2r0, which is revision 0
+config BR2_TARGET_ROOTFS_EXT2_REV
+    int
+    default 0   if BR2_TARGET_ROOTFS_EXT2_2r0
+    default 1   if !BR2_TARGET_ROOTFS_EXT2_2r0
+
 config BR2_TARGET_ROOTFS_EXT2_BLOCKS
 	int "size in blocks (leave at 0 for auto calculation)"
 	default 0
diff --git a/fs/ext2/ext2.mk b/fs/ext2/ext2.mk
index ffb7257..d74f31e 100644
--- a/fs/ext2/ext2.mk
+++ b/fs/ext2/ext2.mk
@@ -20,7 +20,8 @@ endif
 
 ROOTFS_EXT2_DEPENDENCIES = host-genext2fs host-e2fsprogs
 
-EXT2_ENV = GEN=$(BR2_TARGET_ROOTFS_EXT2_GEN)
+EXT2_ENV  = GEN=$(BR2_TARGET_ROOTFS_EXT2_GEN)
+EXT2_ENV += REV=$(BR2_TARGET_ROOTFS_EXT2_REV)
 
 define ROOTFS_EXT2_CMD
 	PATH=$(TARGET_PATH) $(EXT2_ENV) fs/ext2/genext2fs.sh -d $(TARGET_DIR) $(EXT2_OPTS) $@
diff --git a/fs/ext2/genext2fs.sh b/fs/ext2/genext2fs.sh
index c2049f5..76f454c 100755
--- a/fs/ext2/genext2fs.sh
+++ b/fs/ext2/genext2fs.sh
@@ -78,15 +78,21 @@ e2tunefsck() {
 }
 
 # Check we know what generation to generate
-case "${GEN}" in
-    2|3|4)
+case "${GEN}:${REV}" in
+    2:0|2:1|3:1|4:1)
 	;;
     *)
-	printf "%s: unknown ext generation to generate\n" "${0##*/}" >&2
+	printf "%s: unknown ext generation '%s' and/or revision '%s'\n" \
+	       "${0##*/}" "${GEN}" "${REV}" >&2
 	exit 1
 	;;
 esac
 
+# Upgrade to rev1 if needed
+if [ ${REV} -ge 1 ]; then
+    EXT_OPTS_O="${EXT_OPTS_O},filetype"
+fi
+
 # Add a journal for ext3 and above
 if [ ${GEN} -ge 3 ]; then
     EXT_OPTS="${EXT_OPTS} -j -J size=1"
-- 
1.7.2.5

[Buildroot] [PATCH 5/5] fs/ext2: remove count- and time-based fsck

[Yann E. MORIN]

Set the count- and time-based checks intervals to 0, thus effectively
disabling automatic checks at boot (after a suggestion by Arnout).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 fs/ext2/genext2fs.sh |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/fs/ext2/genext2fs.sh b/fs/ext2/genext2fs.sh
index 76f454c..a4eb351 100755
--- a/fs/ext2/genext2fs.sh
+++ b/fs/ext2/genext2fs.sh
@@ -73,8 +73,11 @@ e2tunefsck() {
            "${IMG##*/}" "${GEN}"
 
     # e2fsck will force a *random* UUID, which is bad
-    # for reproducibility, so we do not want it
-    tune2fs -U clear "${IMG}"
+    # for reproducibility, so we do not want it.
+    # Remove count- and time-based checks, they are not welcome
+    # on embedded devices, where they can cause serious boot-time
+    # issues by tremendously slowing down the boot.
+    tune2fs -U clear -c 0 -i 0 "${IMG}"
 }
 
 # Check we know what generation to generate
-- 
1.7.2.5

[Buildroot] [pull request v4] Pull request for branch yem-ext234

[Peter Korsgaard]

>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:

 Yann> Hello All!

 Yann> This patch series enhances the ext2 fs backend to also build ext3
 Yann> and ext4 filesystems, and differentiate between ext2 rev0 and
 Yann> ext2 rev1 filesystems.

Committed series, thanks.

Notice: I did a quick test with a smallish ext4 (3.5MB) filesystem,
where the 1081 blocks extra for the journal wasn't enough, so I've
increased it to 1300 instead.

-- 
Bye, Peter Korsgaard

[Buildroot] [pull request v4] Pull request for branch yem-ext234

[Yann E. MORIN]

Peter, All,

On Fri, Apr 12, 2013 at 03:00:45PM +0200, Peter Korsgaard wrote:
> >>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:
>  Yann> This patch series enhances the ext2 fs backend to also build ext3
>  Yann> and ext4 filesystems, and differentiate between ext2 rev0 and
>  Yann> ext2 rev1 filesystems.
[--SNIP--]
> Notice: I did a quick test with a smallish ext4 (3.5MB) filesystem,
> where the 1081 blocks extra for the journal wasn't enough, so I've
> increased it to 1300 instead.

Strange, Thomas had the same issue, which I was notable to reproduce
here with a basic busybox-only file system (roughly 3.5MiB here too).

Anyway, thanks for the fix up. Indeed, systems going for ext3/4 will
probably not be much impacted by the few extra sectors.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'