[PATCH v3 0/3] Rewrite sdcard_image.bbclass

[PATCH v3 0/3] Rewrite sdcard_image.bbclass

[Otavio Salvador]

The required fix for dosfstools to work has been merged on OE-Core and
this allows this changes to be tested and merged now.

The following changes since commit 018824ece4a04245b4277d5c8afcccfd6ca75df0:

  matrix-tui.inc: drop 'PRIORITY = "optional"' as it is the default (2011-12-14 01:05:59 +0000)

are available in the git repository at:
  git://github.com/OSSystems/meta-ti master
  https://github.com/OSSystems/meta-ti/tree/master

Otavio Salvador (3):
  sdcard_image: rewrote to avoid using loopback and mount/unmount
  ti-hw-bringup-image: adapt to newer sdcard_image class
  cloud9-image: adapt to newer sdcard_image class

 classes/sdcard_image.bbclass               |  180 ++++++++++------------------
 recipes-misc/images/cloud9-image.bb        |    6 +-
 recipes-misc/images/ti-hw-bringup-image.bb |    3 +
 3 files changed, 69 insertions(+), 120 deletions(-)

-- 
1.7.2.5

Re: [PATCH v3 0/3] Rewrite sdcard_image.bbclass

[Otavio Salvador]

[-- Attachment #1: Type: text/plain, Size: 533 bytes --]

On Thu, Dec 15, 2011 at 13:05, Otavio Salvador <otavio@ossystems.com.br>wrote:

> The required fix for dosfstools to work has been merged on OE-Core and
> this allows this changes to be tested and merged now.
>

I am converting those to use mtools and avoid the dosfstools -d usage.

Please hold the merging until I do that.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

[-- Attachment #2: Type: text/html, Size: 1012 bytes --]

[PATCH v3 0/3] Rewrite sdcard_image.bbclass

[Otavio Salvador]

This version of patches uses mtools to copy the files onto the fat
partition avoiding the problems from dosfstools in previous versions.

The following changes since commit c00b5df3b85b3badbd268e376c0a19fab19cc4f3:

  linux-ti335x-psp 3.1: disable whiteheat firmware to unbreak build with make 3.85 (2011-12-21 15:33:36 +0100)

are available in the git repository at:
  git://github.com/OSSystems/meta-ti master
  https://github.com/OSSystems/meta-ti/tree/master

Otavio Salvador (3):
  sdcard_image: rewrote to avoid using loopback and mount/unmount
  ti-hw-bringup-image: adapt to newer sdcard_image class
  cloud9-image: adapt to newer sdcard_image class

 classes/sdcard_image.bbclass               |  181 ++++++++++------------------
 recipes-misc/images/cloud9-image.bb        |    6 +-
 recipes-misc/images/ti-hw-bringup-image.bb |    3 +
 3 files changed, 70 insertions(+), 120 deletions(-)

-- 
1.7.2.5

[PATCH v3 1/3] sdcard_image: rewrote to avoid using loopback and mount/unmount

[Otavio Salvador]

Allow generation by regular users without fancy setup requirements and
use of loopback and mount/unmount. Besides, this also includes
following improvements:

 * use of megabytes when specifying SDIMG_SIZE
 * remove restriction of filesystem to rootfs

Images that use this class needs to generate the rootfs filesystem and
set ROOTFS variable according.

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
---
 classes/sdcard_image.bbclass |  181 +++++++++++++++---------------------------
 1 files changed, 64 insertions(+), 117 deletions(-)

diff --git a/classes/sdcard_image.bbclass b/classes/sdcard_image.bbclass
index 519bd9a..831a1f7 100644
--- a/classes/sdcard_image.bbclass
+++ b/classes/sdcard_image.bbclass
@@ -1,99 +1,42 @@
+#
+# Create an image that can by written onto a SD card using dd.
+#
+# External variables needed:
+#   ${ROOTFS} - the rootfs image to incorporate
+
 inherit image
 
 # Add the fstypes we need
-IMAGE_FSTYPES_append = " tar.bz2 sdimg"
+IMAGE_FSTYPES += "sdimg"
 
 # Ensure required utilities are present
-IMAGE_DEPENDS_sdimg = "genext2fs-native e2fsprogs-native"
-
-# Change this to match your host distro
-LOSETUP ?= "/sbin/losetup"
-
-# Since these need to go in /etc/fstab we can hardcode them
-# Since the vars are weakly assigned, you can override them from your local.conf
-LOOPDEV ?= "/dev/loop1"
-LOOPDEV_BOOT ?= "/dev/loop2"
-LOOPDEV_FS ?= "/dev/loop3"
+IMAGE_DEPENDS_sdimg = "dosfstools-native mtools-native parted-native"
 
-# Default to 4GiB images
-SDIMG_SIZE ?= "444" 
+# Default to 3.4GiB images
+SDIMG_SIZE ?= "3400"
 
-# FS type for rootfs
-ROOTFSTYPE ?= "ext3"
+# Boot partition volume id
+BOOTDD_VOLUME_ID_beaglebone = "BEAGLE_BONE"
+BOOTDD_VOLUME_ID ?= "${MACHINE}"
 
-BOOTPARTNAME_beaglebone = "BEAGLE_BONE"
-BOOTPARTNAME ?= "${MACHINE}"
-
-IMAGEDATESTAMP = "${@time.strftime('%Y.%m.%d',time.gmtime())}"
+# Addional space for boot partition
+BOOTDD_EXTRA_SPACE ?= "16384"
 
 # Files and/or directories to be copied into the vfat partition
 FATPAYLOAD ?= ""
 
 IMAGE_CMD_sdimg () {
-	SDIMG=${WORKDIR}/sd.img
-
-	# sanity check fstab entry for boot partition mounting
-	if [ "x$(cat /etc/fstab | grep ${LOOPDEV_BOOT} | grep ${WORKDIR}/tmp-mnt-boot | grep user || true)" = "x" ]; then
-		echo "/etc/fstab entries need to be created with the user flag for the loop devices like:"
-		echo "${LOOPDEV_BOOT} ${WORKDIR}/tmp-mnt-boot vfat user 0 0"
-        false
-	fi
-
-	# cleanup loops
-	for loop in ${LOOPDEV} ${LOOPDEV_BOOT} ${LOOPDEV_FS} ; do
-		${LOSETUP} -d $loop || true
-	done
-
-	# If an SD image is already present, reuse and reformat it
-	if [ ! -e ${SDIMG} ] ; then
-		dd if=/dev/zero of=${SDIMG} bs=$(echo '255 * 63 * 512' | bc) count=${SDIMG_SIZE}
-	fi
-
-	${LOSETUP} ${LOOPDEV} ${SDIMG}
-
-	# Create partition table
-	dd if=/dev/zero of=${LOOPDEV} bs=1024 count=1024
-	SIZE=$(/sbin/fdisk -l ${LOOPDEV} | grep Disk | grep bytes | awk '{print $5}')
-	CYLINDERS=$(echo $SIZE/255/63/512 | bc)
-	{
-	echo ,9,0x0C,*
-	echo ,,,-
-	} | /sbin/sfdisk -D -H 255 -S 63 -C ${CYLINDERS} ${LOOPDEV}
-
-	# Prepare loop devices for boot and filesystem partitions
-	BOOT_OFFSET=32256
-	FS_OFFSET_SECT=$(/sbin/fdisk -l -u ${LOOPDEV} 2>&1 | grep Linux | perl -p -i -e "s/\s+/ /"|cut -d " " -f 2)
-	FS_OFFSET=$(echo "$FS_OFFSET_SECT * 512" | bc)
-	FS_SIZE_BLOCKS=$(/sbin/fdisk -l -u ${LOOPDEV} 2>&1 | grep Linux | perl -p -i -e "s/\s+/ /g" \ 
-	|cut -d " " -f 4 | cut -d "+" -f 1)
- 
-	LOOPDEV_BLOCKS=$(/sbin/fdisk -l -u ${LOOPDEV} 2>&1 | grep FAT | perl -p -i -e "s/\s+/ /g"|cut -d " " -f 5)
-	LOOPDEV_BYTES=$(echo "$LOOPDEV_BLOCKS * 1024" | bc)
-
-	${LOSETUP} -d ${LOOPDEV}
-
-	${LOSETUP} ${LOOPDEV_BOOT} ${SDIMG} -o ${BOOT_OFFSET} 
-
-	/sbin/mkfs.vfat ${LOOPDEV_BOOT} -n ${BOOTPARTNAME} $LOOPDEV_BLOCKS
-
-	# Prepare filesystem partition
-	# Copy ubi used by flashing scripts
-	if [ -e  ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ubi ] ; then
-		echo "Copying UBIFS image to file system"
-		cp ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ubi ${IMAGE_ROOTFS}/boot/fs.ubi
-	fi
+	TMP=${WORKDIR}/tmp
+	SDIMG=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.sdimg
 
-	# Prepare boot partion. First mount the boot partition, and copy the boot loader and supporting files
-	# from the root filesystem
-
-	mkdir -p ${WORKDIR}/tmp-mnt-boot
-	mount $LOOPDEV_BOOT ${WORKDIR}/tmp-mnt-boot
+	# Prepare boot filesystem
+	install -d $TMP/boot
 
 	echo "Copying bootloaders into the boot partition"
 	if [ -e ${IMAGE_ROOTFS}/boot/MLO ] ; then
-		cp -v ${IMAGE_ROOTFS}/boot/MLO ${WORKDIR}/tmp-mnt-boot 
+		cp -v ${IMAGE_ROOTFS}/boot/MLO $TMP/boot
 	else
-		cp -v ${DEPLOY_DIR_IMAGE}/MLO ${WORKDIR}/tmp-mnt-boot
+		cp -v ${DEPLOY_DIR_IMAGE}/MLO $TMP/boot
 	fi
 
 	# Check for u-boot SPL
@@ -103,54 +46,58 @@ IMAGE_CMD_sdimg () {
 		suffix=bin
 	fi
 
-	cp -v ${IMAGE_ROOTFS}/boot/uEnv.txt ${WORKDIR}/tmp-mnt-boot || true
-	cp -v ${IMAGE_ROOTFS}/boot/user.txt ${WORKDIR}/tmp-mnt-boot || true
-	cp -v ${IMAGE_ROOTFS}/boot/uImage ${WORKDIR}/tmp-mnt-boot || true
-
 	if [ -e ${IMAGE_ROOTFS}/boot/u-boot.$suffix ] ; then
-		cp -v ${IMAGE_ROOTFS}/boot/{u-boot.$suffix} ${WORKDIR}/tmp-mnt-boot || true
+		cp -v ${IMAGE_ROOTFS}/boot/u-boot.$suffix $TMP/boot || true
 	else
-		cp -v ${DEPLOY_DIR_IMAGE}/u-boot-${MACHINE}.$suffix ${WORKDIR}/tmp-mnt-boot/u-boot.$suffix 
+		cp -v ${DEPLOY_DIR_IMAGE}/u-boot-${MACHINE}.$suffix $TMP/boot/u-boot.$suffix
+	fi
+
+	if [ -e ${IMAGE_ROOTFS}/boot/uImage ]; then
+		cp -v ${IMAGE_ROOTFS}/boot/uImage $TMP/boot || true
+	else
+		cp -v ${DEPLOY_DIR_IMAGE}/uImage-${MACHINE}.bin $TMP/boot/uImage
+	fi
+
+	cp -v ${IMAGE_ROOTFS}/boot/uEnv.txt $TMP/boot || true
+	cp -v ${IMAGE_ROOTFS}/boot/user.txt $TMP/boot || true
+
+	# Copy ubi used by flashing scripts
+	if [ -e  ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ubi ] ; then
+		echo "Copying UBIFS image to file system"
+		cp ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ubi ${IMAGE_ROOTFS}/boot/fs.ubi
 	fi
 
 	if [ -n ${FATPAYLOAD} ] ; then
 		echo "Copying payload into VFAT"
 		for entry in ${FATPAYLOAD} ; do
 				# add the || true to stop aborting on vfat issues like not supporting .~lock files
-				cp -av ${IMAGE_ROOTFS}$entry ${WORKDIR}/tmp-mnt-boot || true
+				cp -av ${IMAGE_ROOTFS}$entry $TMP/boot || true
 		done
 	fi
 
-	echo "${IMAGE_NAME}-${IMAGEDATESTAMP}" > ${IMAGE_ROOTFS}/etc/image-version-info
-
-	# Cleanup VFAT mount
-	echo "Cleaning up VFAT mount"
-	umount ${WORKDIR}/tmp-mnt-boot
-	${LOSETUP} -d ${LOOPDEV_BOOT} || true
-
-	# Prepare rootfs parition
-	echo "Creating rootfs loopback"
-	${LOSETUP} ${LOOPDEV_FS} ${SDIMG} -o ${FS_OFFSET}
-
-	FS_NUM_INODES=$(echo $FS_SIZE_BLOCKS / 4 | bc)
-
-	case "${ROOTFSTYPE}" in
-		ext3)
-				genext2fs -z -N $FS_NUM_INODES -b $FS_SIZE_BLOCKS -d ${IMAGE_ROOTFS} ${LOOPDEV_FS}
-				tune2fs -L ${IMAGE_NAME} -j ${LOOPDEV_FS}
-				;;
-		ext4)
-				genext2fs -z -N $FS_NUM_INODES -b $FS_SIZE_BLOCKS -d ${IMAGE_ROOTFS} ${LOOPDEV_FS}
-				tune2fs -L ${IMAGE_NAME} -j -O extents,uninit_bg,dir_index ${LOOPDEV_FS}
-				;;
-		*)
-				echo "Please set ROOTFSTYPE to something supported"
-				exit 1
-				;;
-	esac
-
-	${LOSETUP} -d ${LOOPDEV_FS} || true
-
-	gzip -c ${WORKDIR}/sd.img > ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}-${IMAGEDATESTAMP}.img.gz
-	rm -f ${WORKDIR}/sd.img
+	dd if=/dev/zero of=${SDIMG} bs=$(expr 1000 \* 1000) count=${SDIMG_SIZE}
+
+	# Create the boot filesystem
+	BOOT_OFFSET=32256
+	BOOT_BLOCKS=$(du --apparent-size -ks $TMP/boot | cut -f 1)
+	BOOT_SIZE=$(expr $BOOT_BLOCKS + ${BOOTDD_EXTRA_SPACE})
+	mkfs.vfat -n ${BOOTDD_VOLUME_ID} -S 512 -C $TMP/boot.img $BOOT_SIZE
+	mcopy -i $TMP/boot.img -s $TMP/boot/* ::/
+
+	# Create partition table
+	END1=$(expr $BOOT_SIZE \* 1024)
+	END2=$(expr $END1 + 512)
+	parted -s ${SDIMG} mklabel msdos
+	parted -s ${SDIMG} mkpart primary fat16 ${BOOT_OFFSET}B ${END1}B
+	parted -s ${SDIMG} mkpart primary ext3 ${END2}B 100%
+	parted -s ${SDIMG} set 1 boot on
+	parted ${SDIMG} print
+
+	OFFSET1=$(expr $BOOT_OFFSET / 512)
+	OFFSET2=$(expr $END2 / 512)
+	dd if=$TMP/boot.img of=${SDIMG} conv=notrunc seek=$OFFSET1 bs=512
+	dd if=${ROOTFS} of=${SDIMG} conv=notrunc seek=$OFFSET2 bs=512
+
+	cd ${DEPLOY_DIR_IMAGE}
+	ln -sf ${IMAGE_NAME}.sdimg ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.sdimg
 }
-- 
1.7.2.5

[PATCH v3 2/3] ti-hw-bringup-image: adapt to newer sdcard_image class

[Otavio Salvador]

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
---
 recipes-misc/images/ti-hw-bringup-image.bb |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/recipes-misc/images/ti-hw-bringup-image.bb b/recipes-misc/images/ti-hw-bringup-image.bb
index 4e708bb..4c16301 100644
--- a/recipes-misc/images/ti-hw-bringup-image.bb
+++ b/recipes-misc/images/ti-hw-bringup-image.bb
@@ -38,4 +38,7 @@ IMAGE_LOGIN_MANAGER = "tinylogin shadow"
 
 export IMAGE_BASENAME = "TI-hw-bringup"
 
+ROOTFS = "${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3"
+IMAGE_FSTYPES = "ext3"
+
 inherit sdcard_image
-- 
1.7.2.5

[PATCH v3 3/3] cloud9-image: adapt to newer sdcard_image class

[Otavio Salvador]

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
---
 recipes-misc/images/cloud9-image.bb |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/recipes-misc/images/cloud9-image.bb b/recipes-misc/images/cloud9-image.bb
index 826dc93..7214d91 100644
--- a/recipes-misc/images/cloud9-image.bb
+++ b/recipes-misc/images/cloud9-image.bb
@@ -4,8 +4,6 @@ require ti-hw-bringup-image.bb
 
 FATPAYLOAD = "${datadir}/beaglebone-getting-started/*"
 
-ROOTFSTYPE_beaglebone = "ext4"
-
 IMAGE_INSTALL += " \
 	cloud9 \
 	task-sdk-target \
@@ -27,5 +25,7 @@ IMAGE_INSTALL += " \
 	nodejs-dev \
 "
 
-export IMAGE_BASENAME = "Cloud9-IDE"
+ROOTFS_beaglebone = "${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext4"
+IMAGE_FSTYPES_beaglebone= "ext4"
 
+export IMAGE_BASENAME = "Cloud9-IDE"
-- 
1.7.2.5

Re: [PATCH v3 0/3] Rewrite sdcard_image.bbclass

[Peter Bigot]

On Thu, Dec 22, 2011 at 7:00 AM, Otavio Salvador
<otavio@ossystems.com.br> wrote:
> This version of patches uses mtools to copy the files onto the fat
> partition avoiding the problems from dosfstools in previous versions.

For my cloud9-image build the mcopy aborts with:

 Total number of sectors (84030) not a multiple of sectors per track (32)!
 Add mtools_skip_check=1 to your .mtoolsrc file to skip this test

Changing the calculation of BOOT_SIZE to round up to a multiple of 32
gets past that.

The disk geometry is bizarre.

The image doesn't boot an xM (just prints 60 to the console and
hangs).  I thought this might be because MLO isn't the first entry in
the vfat partition (the recursive mcopy puts some documentation files
first), but correcting that doesn't help.

Peter

>
> The following changes since commit c00b5df3b85b3badbd268e376c0a19fab19cc4f3:
>
>  linux-ti335x-psp 3.1: disable whiteheat firmware to unbreak build with make 3.85 (2011-12-21 15:33:36 +0100)
>
> are available in the git repository at:
>  git://github.com/OSSystems/meta-ti master
>  https://github.com/OSSystems/meta-ti/tree/master
>
> Otavio Salvador (3):
>  sdcard_image: rewrote to avoid using loopback and mount/unmount
>  ti-hw-bringup-image: adapt to newer sdcard_image class
>  cloud9-image: adapt to newer sdcard_image class
>
>  classes/sdcard_image.bbclass               |  181 ++++++++++------------------
>  recipes-misc/images/cloud9-image.bb        |    6 +-
>  recipes-misc/images/ti-hw-bringup-image.bb |    3 +
>  3 files changed, 70 insertions(+), 120 deletions(-)
>
> --
> 1.7.2.5
>
> _______________________________________________
> meta-ti mailing list
> meta-ti@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-ti