* [RFC 0/2] IMAGE_ROOTFS_SIZE Cleanup
@ 2011-05-24 6:38 Saul Wold
2011-05-24 6:38 ` [RFC 1/2] " Saul Wold
2011-05-24 6:38 ` [RFC 2/2] image_types: add IMAGE_ROOTFS_EXTRA_SPACE Saul Wold
0 siblings, 2 replies; 20+ messages in thread
From: Saul Wold @ 2011-05-24 6:38 UTC (permalink / raw)
To: openembedded-core
This RFC is still in the works, I would like to have folks look at it.
Previously, IMAGE_ROOTFS_SIZE was being set with _ext2 or _ext3 in
various places, this caused problem, because it was not easily overriden.
By removing all the overrides, it allows the default mechanism to work
correctly.
Currently that default mechanism takes a du of the rootfs image and then
adds in an overhead factor (IMAGE_OVERHEAD_FACTOR) or 1.3x of the du size,
in most cases this should be sufficent. I am still testing this, and I
heard that there might be a post-install issue with RPM DB being larger.
This additional addresses a couple of other issues, by clearing all the bad
overrides, it allows Image or Layer overrides to work correctly either by
setting IMAGE_ROOTFS_SIZE to some arbitray number or by allowing some extra
space be added above the existing overhead via the IMAGE_ROOTFS_EXTRA_SPACE
option.
The base default is now 64M (maybe I need to only set the one in
bitbake.conf, comments abou tthat?) with the minimal images base as 8M
(currently qemux86 sizes up to 9.9M used on an 11M rootfs).
There will be a corresponding patch for meta-yocto to remove the
IMAGE_ROOTFS_SIZE defaults in atom-pc.conf and routerstationpro.conf
I welcome comments.
Thanks
Sau!
The following changes since commit ad72ffadca975166cd8a416e4bf7b7fe33616d39:
Merge branch 'master', remote-tracking branch 'origin/master' (2011-05-23 23:00:20 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib.git sgw/rootfs_size
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=sgw/rootfs_size
Saul Wold (2):
IMAGE_ROOTFS_SIZE Cleanup
image_types: add IMAGE_ROOTFS_EXTRA_SPACE
meta/classes/image_types.bbclass | 9 ++++++---
meta/conf/bitbake.conf | 4 ++++
meta/conf/distro/include/default-distrovars.inc | 2 --
meta/conf/machine/include/qemu.inc | 2 --
.../images/core-image-minimal-initramfs.bb | 2 ++
meta/recipes-core/images/core-image-minimal.bb | 2 ++
6 files changed, 14 insertions(+), 7 deletions(-)
--
1.7.3.4
^ permalink raw reply [flat|nested] 20+ messages in thread
* [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-24 6:38 [RFC 0/2] IMAGE_ROOTFS_SIZE Cleanup Saul Wold
@ 2011-05-24 6:38 ` Saul Wold
2011-05-26 18:04 ` Joshua Lock
2011-05-24 6:38 ` [RFC 2/2] image_types: add IMAGE_ROOTFS_EXTRA_SPACE Saul Wold
1 sibling, 1 reply; 20+ messages in thread
From: Saul Wold @ 2011-05-24 6:38 UTC (permalink / raw)
To: openembedded-core
This basic cleanup removes the _ext2/3 overrides from places they
no longer belong since they did not allow further overrides. In doing
this the core-image-minimal* recipes can now set a reasonably small
rootfs so that it's a realistic size for minimal.
The new default for minimal is 8M and will be adujsted upward by the
IMAGE_OVERHEAD_FACTOR (of 1.2).
This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
Signed-off-by: Saul Wold <sgw@linux.intel.com>
---
meta/classes/image_types.bbclass | 7 +++++--
meta/conf/distro/include/default-distrovars.inc | 2 --
meta/conf/machine/include/qemu.inc | 2 --
.../images/core-image-minimal-initramfs.bb | 2 ++
meta/recipes-core/images/core-image-minimal.bb | 2 ++
5 files changed, 9 insertions(+), 6 deletions(-)
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index ec0cafd..69f859e 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -21,22 +21,25 @@ runimagecmd () {
}
IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
+
IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
+
IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
IMAGE_CMD_ext2.gz () {
rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
- genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
+ genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
}
+
IMAGE_CMD_ext3 () {
genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
}
IMAGE_CMD_ext3.gz () {
rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
- genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
+ genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
diff --git a/meta/conf/distro/include/default-distrovars.inc b/meta/conf/distro/include/default-distrovars.inc
index 1aa45c8..4b68a0a 100644
--- a/meta/conf/distro/include/default-distrovars.inc
+++ b/meta/conf/distro/include/default-distrovars.inc
@@ -1,7 +1,5 @@
QA_LOGFILE = "${TMPDIR}/qa.log"
-IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
-
OEINCLUDELOGS ?= "yes"
KERNEL_CONSOLE ?= "ttyS0"
diff --git a/meta/conf/machine/include/qemu.inc b/meta/conf/machine/include/qemu.inc
index 4122a88..9ef242f 100644
--- a/meta/conf/machine/include/qemu.inc
+++ b/meta/conf/machine/include/qemu.inc
@@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
IMAGE_FSTYPES ?= "tar.bz2 ext3"
ROOT_FLASH_SIZE = "280"
-IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
-IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
# Don't include kernels in standard images
RDEPENDS_kernel-base = ""
diff --git a/meta/recipes-core/images/core-image-minimal-initramfs.bb b/meta/recipes-core/images/core-image-minimal-initramfs.bb
index 21aaa6c..3246d5c 100644
--- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
+++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
@@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
LICENSE = "MIT"
inherit core-image
+
+IMAGE_ROOTFS_SIZE = "8192"
diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
index aa00e67..743e121 100644
--- a/meta/recipes-core/images/core-image-minimal.bb
+++ b/meta/recipes-core/images/core-image-minimal.bb
@@ -9,5 +9,7 @@ LICENSE = "MIT"
inherit core-image
+IMAGE_ROOTFS_SIZE = "8192"
+
# remove not needed ipkg informations
ROOTFS_POSTPROCESS_COMMAND += "remove_packaging_data_files ; "
--
1.7.3.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [RFC 2/2] image_types: add IMAGE_ROOTFS_EXTRA_SPACE
2011-05-24 6:38 [RFC 0/2] IMAGE_ROOTFS_SIZE Cleanup Saul Wold
2011-05-24 6:38 ` [RFC 1/2] " Saul Wold
@ 2011-05-24 6:38 ` Saul Wold
2011-05-26 18:04 ` Joshua Lock
1 sibling, 1 reply; 20+ messages in thread
From: Saul Wold @ 2011-05-24 6:38 UTC (permalink / raw)
To: openembedded-core
[YOCTO #886]
This address 2 needs after the IMAGE_ROOTFS_SIZE is cleaned up, by
removing some of the _ext2/3 overrides it allows for a cleaner override
using IMAGE_ROOTFS_SIZE to create a large rootfs, or by setting the
IMAGE_ROOTFS_EXTRA_SPACE, will allow for extra space allocated in Kilobytes
above the base size (determined by du of the rootfs * IMAGE_OVERHEAD_FACTOR,
default of 1.3).
Signed-off-by: Saul Wold <sgw@linux.intel.com>
---
meta/classes/image_types.bbclass | 2 +-
meta/conf/bitbake.conf | 4 ++++
2 files changed, 5 insertions(+), 1 deletions(-)
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 69f859e..601a5fb 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -13,7 +13,7 @@ def get_imagecmds(d):
runimagecmd () {
# Image generation code for image type ${type}
- ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR}; print (size > ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
+ ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR} + ${IMAGE_ROOTFS_EXTRA_SPACE}; print (size > ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
${cmd}
cd ${DEPLOY_DIR_IMAGE}/
rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.${type}
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index a0af672..fa3618c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -350,6 +350,10 @@ IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
# This option allows for a precentage overage of the actaul image size rather than a
# fixed extra space
IMAGE_OVERHEAD_FACTOR ?= 1.3
+# This option allow for adding addition space in K above and beyond what the
+# IMAGE_OVERHEAD_FACTOR might add. To set a fixed size then overrideing IMAGE_ROOTFS_SIZE
+# with the max size one wants should do the trick
+IMAGE_OVERHEAD_EXTRA_SPACE = "0"
EXTRA_IMAGEDEPENDS = ""
--
1.7.3.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-24 6:38 ` [RFC 1/2] " Saul Wold
@ 2011-05-26 18:04 ` Joshua Lock
2011-05-26 18:28 ` Saul Wold
0 siblings, 1 reply; 20+ messages in thread
From: Joshua Lock @ 2011-05-26 18:04 UTC (permalink / raw)
To: openembedded-core
On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
> This basic cleanup removes the _ext2/3 overrides from places they
> no longer belong since they did not allow further overrides. In doing
> this the core-image-minimal* recipes can now set a reasonably small
> rootfs so that it's a realistic size for minimal.
Awesome. Thanks for tackling this!
Few questions below.
>
> The new default for minimal is 8M and will be adujsted upward by the
> IMAGE_OVERHEAD_FACTOR (of 1.2).
>
> This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
>
> Signed-off-by: Saul Wold <sgw@linux.intel.com>
> ---
> meta/classes/image_types.bbclass | 7 +++++--
> meta/conf/distro/include/default-distrovars.inc | 2 --
> meta/conf/machine/include/qemu.inc | 2 --
> .../images/core-image-minimal-initramfs.bb | 2 ++
> meta/recipes-core/images/core-image-minimal.bb | 2 ++
> 5 files changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
> index ec0cafd..69f859e 100644
> --- a/meta/classes/image_types.bbclass
> +++ b/meta/classes/image_types.bbclass
> @@ -21,22 +21,25 @@ runimagecmd () {
> }
>
> IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
> +
> IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
> +
> IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
> IMAGE_CMD_ext2.gz () {
> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
> rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> }
> +
> IMAGE_CMD_ext3 () {
> genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
> tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
> }
> IMAGE_CMD_ext3.gz () {
> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
> diff --git a/meta/conf/distro/include/default-distrovars.inc b/meta/conf/distro/include/default-distrovars.inc
> index 1aa45c8..4b68a0a 100644
> --- a/meta/conf/distro/include/default-distrovars.inc
> +++ b/meta/conf/distro/include/default-distrovars.inc
> @@ -1,7 +1,5 @@
> QA_LOGFILE = "${TMPDIR}/qa.log"
>
> -IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
> -
> OEINCLUDELOGS ?= "yes"
> KERNEL_CONSOLE ?= "ttyS0"
>
> diff --git a/meta/conf/machine/include/qemu.inc b/meta/conf/machine/include/qemu.inc
> index 4122a88..9ef242f 100644
> --- a/meta/conf/machine/include/qemu.inc
> +++ b/meta/conf/machine/include/qemu.inc
> @@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
> IMAGE_FSTYPES ?= "tar.bz2 ext3"
>
> ROOT_FLASH_SIZE = "280"
> -IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
> -IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
>
> # Don't include kernels in standard images
> RDEPENDS_kernel-base = ""
> diff --git a/meta/recipes-core/images/core-image-minimal-initramfs.bb b/meta/recipes-core/images/core-image-minimal-initramfs.bb
> index 21aaa6c..3246d5c 100644
> --- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
> +++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
> @@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
> LICENSE = "MIT"
>
> inherit core-image
> +
> +IMAGE_ROOTFS_SIZE = "8192"
I'm not really sure about this, an initramfs that's the same size as a
more fully featured rootfs?
> diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
> index aa00e67..743e121 100644
> --- a/meta/recipes-core/images/core-image-minimal.bb
> +++ b/meta/recipes-core/images/core-image-minimal.bb
> @@ -9,5 +9,7 @@ LICENSE = "MIT"
>
> inherit core-image
>
> +IMAGE_ROOTFS_SIZE = "8192"
> +
In your cover letter you stated that the minimal image is currently
9.9M, which means we *need* the overhead to contain the entire image
contents. Correct? That seems a little unwise.
> # remove not needed ipkg informations
> ROOTFS_POSTPROCESS_COMMAND += "remove_packaging_data_files ; "
--
Joshua Lock
Yocto Project Build Monkey
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 2/2] image_types: add IMAGE_ROOTFS_EXTRA_SPACE
2011-05-24 6:38 ` [RFC 2/2] image_types: add IMAGE_ROOTFS_EXTRA_SPACE Saul Wold
@ 2011-05-26 18:04 ` Joshua Lock
2011-05-26 18:22 ` Saul Wold
0 siblings, 1 reply; 20+ messages in thread
From: Joshua Lock @ 2011-05-26 18:04 UTC (permalink / raw)
To: openembedded-core
On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
> [YOCTO #886]
>
> This address 2 needs after the IMAGE_ROOTFS_SIZE is cleaned up, by
> removing some of the _ext2/3 overrides it allows for a cleaner override
> using IMAGE_ROOTFS_SIZE to create a large rootfs, or by setting the
> IMAGE_ROOTFS_EXTRA_SPACE, will allow for extra space allocated in Kilobytes
> above the base size (determined by du of the rootfs * IMAGE_OVERHEAD_FACTOR,
> default of 1.3).
>
> Signed-off-by: Saul Wold <sgw@linux.intel.com>
> ---
> meta/classes/image_types.bbclass | 2 +-
> meta/conf/bitbake.conf | 4 ++++
> 2 files changed, 5 insertions(+), 1 deletions(-)
>
> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
> index 69f859e..601a5fb 100644
> --- a/meta/classes/image_types.bbclass
> +++ b/meta/classes/image_types.bbclass
> @@ -13,7 +13,7 @@ def get_imagecmds(d):
>
> runimagecmd () {
> # Image generation code for image type ${type}
> - ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR}; print (size > ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
> + ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR} + ${IMAGE_ROOTFS_EXTRA_SPACE}; print (size > ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
> ${cmd}
> cd ${DEPLOY_DIR_IMAGE}/
> rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.${type}
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index a0af672..fa3618c 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -350,6 +350,10 @@ IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
> # This option allows for a precentage overage of the actaul image size rather than a
> # fixed extra space
> IMAGE_OVERHEAD_FACTOR ?= 1.3
> +# This option allow for adding addition space in K above and beyond what the
> +# IMAGE_OVERHEAD_FACTOR might add. To set a fixed size then overrideing IMAGE_ROOTFS_SIZE
> +# with the max size one wants should do the trick
> +IMAGE_OVERHEAD_EXTRA_SPACE = "0"
This appears to be a different variable to the IMAGE_ROOTFS_EXTRA_SPACE
used in the command above? Also, you've done a hard assignment here.
Shouldn't it be a soft (?=) assignment?
Cheers,
Joshua
--
Joshua Lock
Yocto Project Build Monkey
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 2/2] image_types: add IMAGE_ROOTFS_EXTRA_SPACE
2011-05-26 18:04 ` Joshua Lock
@ 2011-05-26 18:22 ` Saul Wold
2011-05-26 20:58 ` Darren Hart
0 siblings, 1 reply; 20+ messages in thread
From: Saul Wold @ 2011-05-26 18:22 UTC (permalink / raw)
To: openembedded-core
On 05/26/2011 11:04 AM, Joshua Lock wrote:
> On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
>> [YOCTO #886]
>>
>> This address 2 needs after the IMAGE_ROOTFS_SIZE is cleaned up, by
>> removing some of the _ext2/3 overrides it allows for a cleaner override
>> using IMAGE_ROOTFS_SIZE to create a large rootfs, or by setting the
>> IMAGE_ROOTFS_EXTRA_SPACE, will allow for extra space allocated in Kilobytes
>> above the base size (determined by du of the rootfs * IMAGE_OVERHEAD_FACTOR,
>> default of 1.3).
>>
>> Signed-off-by: Saul Wold<sgw@linux.intel.com>
>> ---
>> meta/classes/image_types.bbclass | 2 +-
>> meta/conf/bitbake.conf | 4 ++++
>> 2 files changed, 5 insertions(+), 1 deletions(-)
>>
>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
>> index 69f859e..601a5fb 100644
>> --- a/meta/classes/image_types.bbclass
>> +++ b/meta/classes/image_types.bbclass
>> @@ -13,7 +13,7 @@ def get_imagecmds(d):
>>
>> runimagecmd () {
>> # Image generation code for image type ${type}
>> - ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR}; print (size> ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
>> + ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR} + ${IMAGE_ROOTFS_EXTRA_SPACE}; print (size> ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
>> ${cmd}
>> cd ${DEPLOY_DIR_IMAGE}/
>> rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.${type}
>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>> index a0af672..fa3618c 100644
>> --- a/meta/conf/bitbake.conf
>> +++ b/meta/conf/bitbake.conf
>> @@ -350,6 +350,10 @@ IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
>> # This option allows for a precentage overage of the actaul image size rather than a
>> # fixed extra space
>> IMAGE_OVERHEAD_FACTOR ?= 1.3
>> +# This option allow for adding addition space in K above and beyond what the
>> +# IMAGE_OVERHEAD_FACTOR might add. To set a fixed size then overrideing IMAGE_ROOTFS_SIZE
>> +# with the max size one wants should do the trick
>> +IMAGE_OVERHEAD_EXTRA_SPACE = "0"
>
> This appears to be a different variable to the IMAGE_ROOTFS_EXTRA_SPACE
> used in the command above? Also, you've done a hard assignment here.
> Shouldn't it be a soft (?=) assignment?
>
Yes, I fixed the OVERHEAD vs ROOTFS already, noticed that during post
RFC testing. I did not see the soft (?=) problem, but agree it needs to
be fixed.
Sau!
> Cheers,
> Joshua
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-26 18:04 ` Joshua Lock
@ 2011-05-26 18:28 ` Saul Wold
2011-05-26 19:54 ` Joshua Lock
0 siblings, 1 reply; 20+ messages in thread
From: Saul Wold @ 2011-05-26 18:28 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
On 05/26/2011 11:04 AM, Joshua Lock wrote:
> On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
>> This basic cleanup removes the _ext2/3 overrides from places they
>> no longer belong since they did not allow further overrides. In doing
>> this the core-image-minimal* recipes can now set a reasonably small
>> rootfs so that it's a realistic size for minimal.
>
> Awesome. Thanks for tackling this!
>
> Few questions below.
>
>>
>> The new default for minimal is 8M and will be adujsted upward by the
>> IMAGE_OVERHEAD_FACTOR (of 1.2).
>>
>> This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
>>
>> Signed-off-by: Saul Wold<sgw@linux.intel.com>
>> ---
>> meta/classes/image_types.bbclass | 7 +++++--
>> meta/conf/distro/include/default-distrovars.inc | 2 --
>> meta/conf/machine/include/qemu.inc | 2 --
>> .../images/core-image-minimal-initramfs.bb | 2 ++
>> meta/recipes-core/images/core-image-minimal.bb | 2 ++
>> 5 files changed, 9 insertions(+), 6 deletions(-)
>>
>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
>> index ec0cafd..69f859e 100644
>> --- a/meta/classes/image_types.bbclass
>> +++ b/meta/classes/image_types.bbclass
>> @@ -21,22 +21,25 @@ runimagecmd () {
>> }
>>
>> IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
>> +
>> IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
>> +
>> IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
>> IMAGE_CMD_ext2.gz () {
>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
>> rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>> }
>> +
>> IMAGE_CMD_ext3 () {
>> genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
>> tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
>> }
>> IMAGE_CMD_ext3.gz () {
>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>> tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
>> diff --git a/meta/conf/distro/include/default-distrovars.inc b/meta/conf/distro/include/default-distrovars.inc
>> index 1aa45c8..4b68a0a 100644
>> --- a/meta/conf/distro/include/default-distrovars.inc
>> +++ b/meta/conf/distro/include/default-distrovars.inc
>> @@ -1,7 +1,5 @@
>> QA_LOGFILE = "${TMPDIR}/qa.log"
>>
>> -IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
>> -
>> OEINCLUDELOGS ?= "yes"
>> KERNEL_CONSOLE ?= "ttyS0"
>>
>> diff --git a/meta/conf/machine/include/qemu.inc b/meta/conf/machine/include/qemu.inc
>> index 4122a88..9ef242f 100644
>> --- a/meta/conf/machine/include/qemu.inc
>> +++ b/meta/conf/machine/include/qemu.inc
>> @@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
>> IMAGE_FSTYPES ?= "tar.bz2 ext3"
>>
>> ROOT_FLASH_SIZE = "280"
>> -IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
>> -IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
>>
>> # Don't include kernels in standard images
>> RDEPENDS_kernel-base = ""
>> diff --git a/meta/recipes-core/images/core-image-minimal-initramfs.bb b/meta/recipes-core/images/core-image-minimal-initramfs.bb
>> index 21aaa6c..3246d5c 100644
>> --- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
>> +++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
>> @@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
>> LICENSE = "MIT"
>>
>> inherit core-image
>> +
>> +IMAGE_ROOTFS_SIZE = "8192"
>
> I'm not really sure about this, an initramfs that's the same size as a
> more fully featured rootfs?
>
That may be, then we need to increase the size slightly, but this will
trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR,
rather than the 64M which would be the default, this was to ensure it
could get smaller. I don't have a current initramfs size, I will build
and verify.
>> diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
>> index aa00e67..743e121 100644
>> --- a/meta/recipes-core/images/core-image-minimal.bb
>> +++ b/meta/recipes-core/images/core-image-minimal.bb
>> @@ -9,5 +9,7 @@ LICENSE = "MIT"
>>
>> inherit core-image
>>
>> +IMAGE_ROOTFS_SIZE = "8192"
>> +
>
> In your cover letter you stated that the minimal image is currently
> 9.9M, which means we *need* the overhead to contain the entire image
> contents. Correct? That seems a little unwise.
>
Right, that's currently, we want to see the image get smaller even, so
the 8M is an appropriate setting. Currently the actual ext3 image is
13M with 10M of contents.
So are you suggested that the 8M size is unwise or the overhead, not
sure I am catching your meaning here.
>> # remove not needed ipkg informations
>> ROOTFS_POSTPROCESS_COMMAND += "remove_packaging_data_files ; "
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-26 18:28 ` Saul Wold
@ 2011-05-26 19:54 ` Joshua Lock
2011-05-26 20:55 ` Darren Hart
0 siblings, 1 reply; 20+ messages in thread
From: Joshua Lock @ 2011-05-26 19:54 UTC (permalink / raw)
To: Saul Wold; +Cc: Patches and discussions about the oe-core layer
On Thu, 2011-05-26 at 11:28 -0700, Saul Wold wrote:
> On 05/26/2011 11:04 AM, Joshua Lock wrote:
> > On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
> >> This basic cleanup removes the _ext2/3 overrides from places they
> >> no longer belong since they did not allow further overrides. In doing
> >> this the core-image-minimal* recipes can now set a reasonably small
> >> rootfs so that it's a realistic size for minimal.
> >
> > Awesome. Thanks for tackling this!
> >
> > Few questions below.
> >
> >>
> >> The new default for minimal is 8M and will be adujsted upward by the
> >> IMAGE_OVERHEAD_FACTOR (of 1.2).
> >>
> >> This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
> >>
> >> Signed-off-by: Saul Wold<sgw@linux.intel.com>
> >> ---
> >> meta/classes/image_types.bbclass | 7 +++++--
> >> meta/conf/distro/include/default-distrovars.inc | 2 --
> >> meta/conf/machine/include/qemu.inc | 2 --
> >> .../images/core-image-minimal-initramfs.bb | 2 ++
> >> meta/recipes-core/images/core-image-minimal.bb | 2 ++
> >> 5 files changed, 9 insertions(+), 6 deletions(-)
> >>
> >> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
> >> index ec0cafd..69f859e 100644
> >> --- a/meta/classes/image_types.bbclass
> >> +++ b/meta/classes/image_types.bbclass
> >> @@ -21,22 +21,25 @@ runimagecmd () {
> >> }
> >>
> >> IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
> >> +
> >> IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
> >> +
> >> IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
> >> IMAGE_CMD_ext2.gz () {
> >> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> >> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> >> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> >> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> >> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
> >> rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> >> }
> >> +
> >> IMAGE_CMD_ext3 () {
> >> genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
> >> tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
> >> }
> >> IMAGE_CMD_ext3.gz () {
> >> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> >> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >> tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
> >> diff --git a/meta/conf/distro/include/default-distrovars.inc b/meta/conf/distro/include/default-distrovars.inc
> >> index 1aa45c8..4b68a0a 100644
> >> --- a/meta/conf/distro/include/default-distrovars.inc
> >> +++ b/meta/conf/distro/include/default-distrovars.inc
> >> @@ -1,7 +1,5 @@
> >> QA_LOGFILE = "${TMPDIR}/qa.log"
> >>
> >> -IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
> >> -
> >> OEINCLUDELOGS ?= "yes"
> >> KERNEL_CONSOLE ?= "ttyS0"
> >>
> >> diff --git a/meta/conf/machine/include/qemu.inc b/meta/conf/machine/include/qemu.inc
> >> index 4122a88..9ef242f 100644
> >> --- a/meta/conf/machine/include/qemu.inc
> >> +++ b/meta/conf/machine/include/qemu.inc
> >> @@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
> >> IMAGE_FSTYPES ?= "tar.bz2 ext3"
> >>
> >> ROOT_FLASH_SIZE = "280"
> >> -IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
> >> -IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
> >>
> >> # Don't include kernels in standard images
> >> RDEPENDS_kernel-base = ""
> >> diff --git a/meta/recipes-core/images/core-image-minimal-initramfs.bb b/meta/recipes-core/images/core-image-minimal-initramfs.bb
> >> index 21aaa6c..3246d5c 100644
> >> --- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
> >> +++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
> >> @@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
> >> LICENSE = "MIT"
> >>
> >> inherit core-image
> >> +
> >> +IMAGE_ROOTFS_SIZE = "8192"
> >
> > I'm not really sure about this, an initramfs that's the same size as a
> > more fully featured rootfs?
> >
> That may be, then we need to increase the size slightly, but this will
> trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR,
> rather than the 64M which would be the default, this was to ensure it
> could get smaller. I don't have a current initramfs size, I will build
> and verify.
>
>
> >> diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
> >> index aa00e67..743e121 100644
> >> --- a/meta/recipes-core/images/core-image-minimal.bb
> >> +++ b/meta/recipes-core/images/core-image-minimal.bb
> >> @@ -9,5 +9,7 @@ LICENSE = "MIT"
> >>
> >> inherit core-image
> >>
> >> +IMAGE_ROOTFS_SIZE = "8192"
> >> +
> >
> > In your cover letter you stated that the minimal image is currently
> > 9.9M, which means we *need* the overhead to contain the entire image
> > contents. Correct? That seems a little unwise.
> >
> Right, that's currently, we want to see the image get smaller even, so
> the 8M is an appropriate setting. Currently the actual ext3 image is
> 13M with 10M of contents.
>
> So are you suggested that the 8M size is unwise or the overhead, not
> sure I am catching your meaning here.
I'm just nervous about relying on the overhead, what if someone sets it
lower and then the image doesn't fit?
Having the goal of a smaller minimal image is good but in my opinion we
should adjust the rootfs size when the image is smaller, not before.
Cheers,
Joshua
--
Joshua Lock
Yocto Project Build Monkey
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-26 19:54 ` Joshua Lock
@ 2011-05-26 20:55 ` Darren Hart
2011-05-26 21:01 ` Joshua Lock
0 siblings, 1 reply; 20+ messages in thread
From: Darren Hart @ 2011-05-26 20:55 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
On 05/26/2011 12:54 PM, Joshua Lock wrote:
> On Thu, 2011-05-26 at 11:28 -0700, Saul Wold wrote:
>> On 05/26/2011 11:04 AM, Joshua Lock wrote:
>>> On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
>>>> This basic cleanup removes the _ext2/3 overrides from places they
>>>> no longer belong since they did not allow further overrides. In doing
>>>> this the core-image-minimal* recipes can now set a reasonably small
>>>> rootfs so that it's a realistic size for minimal.
>>>
>>> Awesome. Thanks for tackling this!
>>>
>>> Few questions below.
>>>
>>>>
>>>> The new default for minimal is 8M and will be adujsted upward by the
>>>> IMAGE_OVERHEAD_FACTOR (of 1.2).
>>>>
>>>> This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
>>>>
>>>> Signed-off-by: Saul Wold<sgw@linux.intel.com>
>>>> ---
>>>> meta/classes/image_types.bbclass | 7 +++++--
>>>> meta/conf/distro/include/default-distrovars.inc | 2 --
>>>> meta/conf/machine/include/qemu.inc | 2 --
>>>> .../images/core-image-minimal-initramfs.bb | 2 ++
>>>> meta/recipes-core/images/core-image-minimal.bb | 2 ++
>>>> 5 files changed, 9 insertions(+), 6 deletions(-)
>>>>
>>>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
>>>> index ec0cafd..69f859e 100644
>>>> --- a/meta/classes/image_types.bbclass
>>>> +++ b/meta/classes/image_types.bbclass
>>>> @@ -21,22 +21,25 @@ runimagecmd () {
>>>> }
>>>>
>>>> IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
>>>> +
>>>> IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
>>>> +
>>>> IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
>>>> IMAGE_CMD_ext2.gz () {
>>>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>>>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>>>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>>>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>>>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
>>>> rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>>>> }
>>>> +
>>>> IMAGE_CMD_ext3 () {
>>>> genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
>>>> tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
>>>> }
>>>> IMAGE_CMD_ext3.gz () {
>>>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>>>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>> tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
>>>> diff --git a/meta/conf/distro/include/default-distrovars.inc b/meta/conf/distro/include/default-distrovars.inc
>>>> index 1aa45c8..4b68a0a 100644
>>>> --- a/meta/conf/distro/include/default-distrovars.inc
>>>> +++ b/meta/conf/distro/include/default-distrovars.inc
>>>> @@ -1,7 +1,5 @@
>>>> QA_LOGFILE = "${TMPDIR}/qa.log"
>>>>
>>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
>>>> -
>>>> OEINCLUDELOGS ?= "yes"
>>>> KERNEL_CONSOLE ?= "ttyS0"
>>>>
>>>> diff --git a/meta/conf/machine/include/qemu.inc b/meta/conf/machine/include/qemu.inc
>>>> index 4122a88..9ef242f 100644
>>>> --- a/meta/conf/machine/include/qemu.inc
>>>> +++ b/meta/conf/machine/include/qemu.inc
>>>> @@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
>>>> IMAGE_FSTYPES ?= "tar.bz2 ext3"
>>>>
>>>> ROOT_FLASH_SIZE = "280"
>>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
>>>> -IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
>>>>
>>>> # Don't include kernels in standard images
>>>> RDEPENDS_kernel-base = ""
>>>> diff --git a/meta/recipes-core/images/core-image-minimal-initramfs.bb b/meta/recipes-core/images/core-image-minimal-initramfs.bb
>>>> index 21aaa6c..3246d5c 100644
>>>> --- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
>>>> +++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
>>>> @@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
>>>> LICENSE = "MIT"
>>>>
>>>> inherit core-image
>>>> +
>>>> +IMAGE_ROOTFS_SIZE = "8192"
>>>
>>> I'm not really sure about this, an initramfs that's the same size as a
>>> more fully featured rootfs?
>>>
>> That may be, then we need to increase the size slightly, but this will
>> trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR,
>> rather than the 64M which would be the default, this was to ensure it
>> could get smaller. I don't have a current initramfs size, I will build
>> and verify.
>>
>>
>>>> diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
>>>> index aa00e67..743e121 100644
>>>> --- a/meta/recipes-core/images/core-image-minimal.bb
>>>> +++ b/meta/recipes-core/images/core-image-minimal.bb
>>>> @@ -9,5 +9,7 @@ LICENSE = "MIT"
>>>>
>>>> inherit core-image
>>>>
>>>> +IMAGE_ROOTFS_SIZE = "8192"
>>>> +
>>>
>>> In your cover letter you stated that the minimal image is currently
>>> 9.9M, which means we *need* the overhead to contain the entire image
>>> contents. Correct? That seems a little unwise.
>>>
>> Right, that's currently, we want to see the image get smaller even, so
>> the 8M is an appropriate setting. Currently the actual ext3 image is
>> 13M with 10M of contents.
>>
>> So are you suggested that the 8M size is unwise or the overhead, not
>> sure I am catching your meaning here.
>
> I'm just nervous about relying on the overhead, what if someone sets it
> lower and then the image doesn't fit?
>
> Having the goal of a smaller minimal image is good but in my opinion we
> should adjust the rootfs size when the image is smaller, not before.
Every argument I've heard in favor of the IMAGE_OVERHEAD was centered
around ensuring there is enough space for the user to actually use the
system after it was installed and booted. Using this single metric to
try and both make up for oversized images and provide space for user
data seems like to cause maintenance issues going forward.
While our goal should be to reduce the image size, the default image
size in the recipes should default to the largest of the images built
for the supported machines. The overhead should be used strictly for
user data.
--
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 2/2] image_types: add IMAGE_ROOTFS_EXTRA_SPACE
2011-05-26 18:22 ` Saul Wold
@ 2011-05-26 20:58 ` Darren Hart
0 siblings, 0 replies; 20+ messages in thread
From: Darren Hart @ 2011-05-26 20:58 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer; +Cc: Saul Wold
On 05/26/2011 11:22 AM, Saul Wold wrote:
> On 05/26/2011 11:04 AM, Joshua Lock wrote:
>> On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
>>> [YOCTO #886]
>>>
>>> This address 2 needs after the IMAGE_ROOTFS_SIZE is cleaned up, by
>>> removing some of the _ext2/3 overrides it allows for a cleaner override
>>> using IMAGE_ROOTFS_SIZE to create a large rootfs, or by setting the
>>> IMAGE_ROOTFS_EXTRA_SPACE, will allow for extra space allocated in Kilobytes
>>> above the base size (determined by du of the rootfs * IMAGE_OVERHEAD_FACTOR,
>>> default of 1.3).
>>>
>>> Signed-off-by: Saul Wold<sgw@linux.intel.com>
>>> ---
>>> meta/classes/image_types.bbclass | 2 +-
>>> meta/conf/bitbake.conf | 4 ++++
>>> 2 files changed, 5 insertions(+), 1 deletions(-)
>>>
>>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
>>> index 69f859e..601a5fb 100644
>>> --- a/meta/classes/image_types.bbclass
>>> +++ b/meta/classes/image_types.bbclass
>>> @@ -13,7 +13,7 @@ def get_imagecmds(d):
>>>
>>> runimagecmd () {
>>> # Image generation code for image type ${type}
>>> - ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR}; print (size> ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
>>> + ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR} + ${IMAGE_ROOTFS_EXTRA_SPACE}; print (size> ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
>>> ${cmd}
>>> cd ${DEPLOY_DIR_IMAGE}/
>>> rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.${type}
>>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>>> index a0af672..fa3618c 100644
>>> --- a/meta/conf/bitbake.conf
>>> +++ b/meta/conf/bitbake.conf
>>> @@ -350,6 +350,10 @@ IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
>>> # This option allows for a precentage overage of the actaul image size rather than a
contextual fix while you're at it:
s/precentage/percentage/
s/actaul/actual/
>>> # fixed extra space
>>> IMAGE_OVERHEAD_FACTOR ?= 1.3
>>> +# This option allow for adding addition space in K above and beyond what the
>>> +# IMAGE_OVERHEAD_FACTOR might add. To set a fixed size then overrideing IMAGE_ROOTFS_SIZE
>>> +# with the max size one wants should do the trick
>>> +IMAGE_OVERHEAD_EXTRA_SPACE = "0"
>>
>> This appears to be a different variable to the IMAGE_ROOTFS_EXTRA_SPACE
>> used in the command above? Also, you've done a hard assignment here.
>> Shouldn't it be a soft (?=) assignment?
>>
> Yes, I fixed the OVERHEAD vs ROOTFS already, noticed that during post
> RFC testing. I did not see the soft (?=) problem, but agree it needs to
> be fixed.
>
Hrm, I didn't realize there were two variables. If
IMAGE_ROOTFS_EXTRA_SPACE is used to ensure the user has some space for
their data, what then is the idea driving IMAGE_OVERHEAD_FACTOR?
--
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-26 20:55 ` Darren Hart
@ 2011-05-26 21:01 ` Joshua Lock
2011-05-26 21:03 ` Darren Hart
0 siblings, 1 reply; 20+ messages in thread
From: Joshua Lock @ 2011-05-26 21:01 UTC (permalink / raw)
To: Darren Hart; +Cc: Patches and discussions about the oe-core layer
On Thu, 2011-05-26 at 13:55 -0700, Darren Hart wrote:
>
> On 05/26/2011 12:54 PM, Joshua Lock wrote:
> > On Thu, 2011-05-26 at 11:28 -0700, Saul Wold wrote:
> >> On 05/26/2011 11:04 AM, Joshua Lock wrote:
> >>> On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
> >>>> This basic cleanup removes the _ext2/3 overrides from places they
> >>>> no longer belong since they did not allow further overrides. In doing
> >>>> this the core-image-minimal* recipes can now set a reasonably small
> >>>> rootfs so that it's a realistic size for minimal.
> >>>
> >>> Awesome. Thanks for tackling this!
> >>>
> >>> Few questions below.
> >>>
> >>>>
> >>>> The new default for minimal is 8M and will be adujsted upward by the
> >>>> IMAGE_OVERHEAD_FACTOR (of 1.2).
> >>>>
> >>>> This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
> >>>>
> >>>> Signed-off-by: Saul Wold<sgw@linux.intel.com>
> >>>> ---
> >>>> meta/classes/image_types.bbclass | 7 +++++--
> >>>> meta/conf/distro/include/default-distrovars.inc | 2 --
> >>>> meta/conf/machine/include/qemu.inc | 2 --
> >>>> .../images/core-image-minimal-initramfs.bb | 2 ++
> >>>> meta/recipes-core/images/core-image-minimal.bb | 2 ++
> >>>> 5 files changed, 9 insertions(+), 6 deletions(-)
> >>>>
> >>>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
> >>>> index ec0cafd..69f859e 100644
> >>>> --- a/meta/classes/image_types.bbclass
> >>>> +++ b/meta/classes/image_types.bbclass
> >>>> @@ -21,22 +21,25 @@ runimagecmd () {
> >>>> }
> >>>>
> >>>> IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
> >>>> +
> >>>> IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
> >>>> +
> >>>> IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
> >>>> IMAGE_CMD_ext2.gz () {
> >>>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> >>>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> >>>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> >>>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> >>>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
> >>>> rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> >>>> }
> >>>> +
> >>>> IMAGE_CMD_ext3 () {
> >>>> genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
> >>>> tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
> >>>> }
> >>>> IMAGE_CMD_ext3.gz () {
> >>>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> >>>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >>>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >>>> tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >>>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >>>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
> >>>> diff --git a/meta/conf/distro/include/default-distrovars.inc b/meta/conf/distro/include/default-distrovars.inc
> >>>> index 1aa45c8..4b68a0a 100644
> >>>> --- a/meta/conf/distro/include/default-distrovars.inc
> >>>> +++ b/meta/conf/distro/include/default-distrovars.inc
> >>>> @@ -1,7 +1,5 @@
> >>>> QA_LOGFILE = "${TMPDIR}/qa.log"
> >>>>
> >>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
> >>>> -
> >>>> OEINCLUDELOGS ?= "yes"
> >>>> KERNEL_CONSOLE ?= "ttyS0"
> >>>>
> >>>> diff --git a/meta/conf/machine/include/qemu.inc b/meta/conf/machine/include/qemu.inc
> >>>> index 4122a88..9ef242f 100644
> >>>> --- a/meta/conf/machine/include/qemu.inc
> >>>> +++ b/meta/conf/machine/include/qemu.inc
> >>>> @@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
> >>>> IMAGE_FSTYPES ?= "tar.bz2 ext3"
> >>>>
> >>>> ROOT_FLASH_SIZE = "280"
> >>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
> >>>> -IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
> >>>>
> >>>> # Don't include kernels in standard images
> >>>> RDEPENDS_kernel-base = ""
> >>>> diff --git a/meta/recipes-core/images/core-image-minimal-initramfs.bb b/meta/recipes-core/images/core-image-minimal-initramfs.bb
> >>>> index 21aaa6c..3246d5c 100644
> >>>> --- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
> >>>> +++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
> >>>> @@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
> >>>> LICENSE = "MIT"
> >>>>
> >>>> inherit core-image
> >>>> +
> >>>> +IMAGE_ROOTFS_SIZE = "8192"
> >>>
> >>> I'm not really sure about this, an initramfs that's the same size as a
> >>> more fully featured rootfs?
> >>>
> >> That may be, then we need to increase the size slightly, but this will
> >> trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR,
> >> rather than the 64M which would be the default, this was to ensure it
> >> could get smaller. I don't have a current initramfs size, I will build
> >> and verify.
> >>
> >>
> >>>> diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
> >>>> index aa00e67..743e121 100644
> >>>> --- a/meta/recipes-core/images/core-image-minimal.bb
> >>>> +++ b/meta/recipes-core/images/core-image-minimal.bb
> >>>> @@ -9,5 +9,7 @@ LICENSE = "MIT"
> >>>>
> >>>> inherit core-image
> >>>>
> >>>> +IMAGE_ROOTFS_SIZE = "8192"
> >>>> +
> >>>
> >>> In your cover letter you stated that the minimal image is currently
> >>> 9.9M, which means we *need* the overhead to contain the entire image
> >>> contents. Correct? That seems a little unwise.
> >>>
> >> Right, that's currently, we want to see the image get smaller even, so
> >> the 8M is an appropriate setting. Currently the actual ext3 image is
> >> 13M with 10M of contents.
> >>
> >> So are you suggested that the 8M size is unwise or the overhead, not
> >> sure I am catching your meaning here.
> >
> > I'm just nervous about relying on the overhead, what if someone sets it
> > lower and then the image doesn't fit?
> >
> > Having the goal of a smaller minimal image is good but in my opinion we
> > should adjust the rootfs size when the image is smaller, not before.
>
> Every argument I've heard in favor of the IMAGE_OVERHEAD was centered
> around ensuring there is enough space for the user to actually use the
> system after it was installed and booted. Using this single metric to
> try and both make up for oversized images and provide space for user
> data seems like to cause maintenance issues going forward.
>
> While our goal should be to reduce the image size, the default image
> size in the recipes should default to the largest of the images built
> for the supported machines. The overhead should be used strictly for
> user data.
>
So it turns out that if IMAGE_ROOTFS is smaller than the contents the
size of the contents is used. Once this if test has been performed the
overhead factor is applied.
I found this to be a tad confusing (hence all the back and forth here)
so have filed a documentation bug about it:
http://bugzilla.pokylinux.org/show_bug.cgi?id=1110
Maybe we can have some comments in the code near where those variables
are set too?
--
Joshua Lock
Yocto Project Build Monkey
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-26 21:01 ` Joshua Lock
@ 2011-05-26 21:03 ` Darren Hart
2011-05-26 21:13 ` Joshua Lock
` (2 more replies)
0 siblings, 3 replies; 20+ messages in thread
From: Darren Hart @ 2011-05-26 21:03 UTC (permalink / raw)
To: Joshua Lock; +Cc: Patches and discussions about the oe-core layer
On 05/26/2011 02:01 PM, Joshua Lock wrote:
> On Thu, 2011-05-26 at 13:55 -0700, Darren Hart wrote:
>>
>> On 05/26/2011 12:54 PM, Joshua Lock wrote:
>>> On Thu, 2011-05-26 at 11:28 -0700, Saul Wold wrote:
>>>> On 05/26/2011 11:04 AM, Joshua Lock wrote:
>>>>> On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
>>>>>> This basic cleanup removes the _ext2/3 overrides from places they
>>>>>> no longer belong since they did not allow further overrides. In doing
>>>>>> this the core-image-minimal* recipes can now set a reasonably small
>>>>>> rootfs so that it's a realistic size for minimal.
>>>>>
>>>>> Awesome. Thanks for tackling this!
>>>>>
>>>>> Few questions below.
>>>>>
>>>>>>
>>>>>> The new default for minimal is 8M and will be adujsted upward by the
>>>>>> IMAGE_OVERHEAD_FACTOR (of 1.2).
>>>>>>
>>>>>> This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
>>>>>>
>>>>>> Signed-off-by: Saul Wold<sgw@linux.intel.com>
>>>>>> ---
>>>>>> meta/classes/image_types.bbclass | 7 +++++--
>>>>>> meta/conf/distro/include/default-distrovars.inc | 2 --
>>>>>> meta/conf/machine/include/qemu.inc | 2 --
>>>>>> .../images/core-image-minimal-initramfs.bb | 2 ++
>>>>>> meta/recipes-core/images/core-image-minimal.bb | 2 ++
>>>>>> 5 files changed, 9 insertions(+), 6 deletions(-)
>>>>>>
>>>>>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
>>>>>> index ec0cafd..69f859e 100644
>>>>>> --- a/meta/classes/image_types.bbclass
>>>>>> +++ b/meta/classes/image_types.bbclass
>>>>>> @@ -21,22 +21,25 @@ runimagecmd () {
>>>>>> }
>>>>>>
>>>>>> IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
>>>>>> +
>>>>>> IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
>>>>>> +
>>>>>> IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
>>>>>> IMAGE_CMD_ext2.gz () {
>>>>>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>>>>>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>>>>>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>>>>>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>>>>>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
>>>>>> rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>>>>>> }
>>>>>> +
>>>>>> IMAGE_CMD_ext3 () {
>>>>>> genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
>>>>>> tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
>>>>>> }
>>>>>> IMAGE_CMD_ext3.gz () {
>>>>>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>>>>>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>>>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>>>> tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>>>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>>>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
>>>>>> diff --git a/meta/conf/distro/include/default-distrovars.inc b/meta/conf/distro/include/default-distrovars.inc
>>>>>> index 1aa45c8..4b68a0a 100644
>>>>>> --- a/meta/conf/distro/include/default-distrovars.inc
>>>>>> +++ b/meta/conf/distro/include/default-distrovars.inc
>>>>>> @@ -1,7 +1,5 @@
>>>>>> QA_LOGFILE = "${TMPDIR}/qa.log"
>>>>>>
>>>>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
>>>>>> -
>>>>>> OEINCLUDELOGS ?= "yes"
>>>>>> KERNEL_CONSOLE ?= "ttyS0"
>>>>>>
>>>>>> diff --git a/meta/conf/machine/include/qemu.inc b/meta/conf/machine/include/qemu.inc
>>>>>> index 4122a88..9ef242f 100644
>>>>>> --- a/meta/conf/machine/include/qemu.inc
>>>>>> +++ b/meta/conf/machine/include/qemu.inc
>>>>>> @@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
>>>>>> IMAGE_FSTYPES ?= "tar.bz2 ext3"
>>>>>>
>>>>>> ROOT_FLASH_SIZE = "280"
>>>>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
>>>>>> -IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
>>>>>>
>>>>>> # Don't include kernels in standard images
>>>>>> RDEPENDS_kernel-base = ""
>>>>>> diff --git a/meta/recipes-core/images/core-image-minimal-initramfs.bb b/meta/recipes-core/images/core-image-minimal-initramfs.bb
>>>>>> index 21aaa6c..3246d5c 100644
>>>>>> --- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
>>>>>> +++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
>>>>>> @@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
>>>>>> LICENSE = "MIT"
>>>>>>
>>>>>> inherit core-image
>>>>>> +
>>>>>> +IMAGE_ROOTFS_SIZE = "8192"
>>>>>
>>>>> I'm not really sure about this, an initramfs that's the same size as a
>>>>> more fully featured rootfs?
>>>>>
>>>> That may be, then we need to increase the size slightly, but this will
>>>> trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR,
>>>> rather than the 64M which would be the default, this was to ensure it
>>>> could get smaller. I don't have a current initramfs size, I will build
>>>> and verify.
>>>>
>>>>
>>>>>> diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
>>>>>> index aa00e67..743e121 100644
>>>>>> --- a/meta/recipes-core/images/core-image-minimal.bb
>>>>>> +++ b/meta/recipes-core/images/core-image-minimal.bb
>>>>>> @@ -9,5 +9,7 @@ LICENSE = "MIT"
>>>>>>
>>>>>> inherit core-image
>>>>>>
>>>>>> +IMAGE_ROOTFS_SIZE = "8192"
>>>>>> +
>>>>>
>>>>> In your cover letter you stated that the minimal image is currently
>>>>> 9.9M, which means we *need* the overhead to contain the entire image
>>>>> contents. Correct? That seems a little unwise.
>>>>>
>>>> Right, that's currently, we want to see the image get smaller even, so
>>>> the 8M is an appropriate setting. Currently the actual ext3 image is
>>>> 13M with 10M of contents.
>>>>
>>>> So are you suggested that the 8M size is unwise or the overhead, not
>>>> sure I am catching your meaning here.
>>>
>>> I'm just nervous about relying on the overhead, what if someone sets it
>>> lower and then the image doesn't fit?
>>>
>>> Having the goal of a smaller minimal image is good but in my opinion we
>>> should adjust the rootfs size when the image is smaller, not before.
>>
>> Every argument I've heard in favor of the IMAGE_OVERHEAD was centered
>> around ensuring there is enough space for the user to actually use the
>> system after it was installed and booted. Using this single metric to
>> try and both make up for oversized images and provide space for user
>> data seems like to cause maintenance issues going forward.
>>
>> While our goal should be to reduce the image size, the default image
>> size in the recipes should default to the largest of the images built
>> for the supported machines. The overhead should be used strictly for
>> user data.
>>
>
> So it turns out that if IMAGE_ROOTFS is smaller than the contents the
> size of the contents is used. Once this if test has been performed the
> overhead factor is applied.
Ah Good!
>
> I found this to be a tad confusing (hence all the back and forth here)
> so have filed a documentation bug about it:
> http://bugzilla.pokylinux.org/show_bug.cgi?id=1110
>
> Maybe we can have some comments in the code near where those variables
> are set too?
>
Yeah, this wasn't clear to me either. And my question in 2/2 still
stands - what is the goal of the overhead factor?
--
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-26 21:03 ` Darren Hart
@ 2011-05-26 21:13 ` Joshua Lock
2011-05-26 21:15 ` Saul Wold
2011-05-26 21:14 ` Paul Eggleton
2011-06-02 1:46 ` Tom Rini
2 siblings, 1 reply; 20+ messages in thread
From: Joshua Lock @ 2011-05-26 21:13 UTC (permalink / raw)
To: Darren Hart; +Cc: Patches and discussions about the oe-core layer
On Thu, 2011-05-26 at 14:03 -0700, Darren Hart wrote:
>
> On 05/26/2011 02:01 PM, Joshua Lock wrote:
> > On Thu, 2011-05-26 at 13:55 -0700, Darren Hart wrote:
> >>
> >> On 05/26/2011 12:54 PM, Joshua Lock wrote:
> >>> On Thu, 2011-05-26 at 11:28 -0700, Saul Wold wrote:
> >>>> On 05/26/2011 11:04 AM, Joshua Lock wrote:
> >>>>> On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
> >>>>>> This basic cleanup removes the _ext2/3 overrides from places they
> >>>>>> no longer belong since they did not allow further overrides. In doing
> >>>>>> this the core-image-minimal* recipes can now set a reasonably small
> >>>>>> rootfs so that it's a realistic size for minimal.
> >>>>>
> >>>>> Awesome. Thanks for tackling this!
> >>>>>
> >>>>> Few questions below.
> >>>>>
> >>>>>>
> >>>>>> The new default for minimal is 8M and will be adujsted upward by the
> >>>>>> IMAGE_OVERHEAD_FACTOR (of 1.2).
> >>>>>>
> >>>>>> This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
> >>>>>>
> >>>>>> Signed-off-by: Saul Wold<sgw@linux.intel.com>
> >>>>>> ---
> >>>>>> meta/classes/image_types.bbclass | 7 +++++--
> >>>>>> meta/conf/distro/include/default-distrovars.inc | 2 --
> >>>>>> meta/conf/machine/include/qemu.inc | 2 --
> >>>>>> .../images/core-image-minimal-initramfs.bb | 2 ++
> >>>>>> meta/recipes-core/images/core-image-minimal.bb | 2 ++
> >>>>>> 5 files changed, 9 insertions(+), 6 deletions(-)
> >>>>>>
> >>>>>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
> >>>>>> index ec0cafd..69f859e 100644
> >>>>>> --- a/meta/classes/image_types.bbclass
> >>>>>> +++ b/meta/classes/image_types.bbclass
> >>>>>> @@ -21,22 +21,25 @@ runimagecmd () {
> >>>>>> }
> >>>>>>
> >>>>>> IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
> >>>>>> +
> >>>>>> IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
> >>>>>> +
> >>>>>> IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
> >>>>>> IMAGE_CMD_ext2.gz () {
> >>>>>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> >>>>>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> >>>>>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> >>>>>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> >>>>>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
> >>>>>> rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> >>>>>> }
> >>>>>> +
> >>>>>> IMAGE_CMD_ext3 () {
> >>>>>> genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
> >>>>>> tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
> >>>>>> }
> >>>>>> IMAGE_CMD_ext3.gz () {
> >>>>>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> >>>>>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >>>>>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >>>>>> tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >>>>>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >>>>>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
> >>>>>> diff --git a/meta/conf/distro/include/default-distrovars.inc b/meta/conf/distro/include/default-distrovars.inc
> >>>>>> index 1aa45c8..4b68a0a 100644
> >>>>>> --- a/meta/conf/distro/include/default-distrovars.inc
> >>>>>> +++ b/meta/conf/distro/include/default-distrovars.inc
> >>>>>> @@ -1,7 +1,5 @@
> >>>>>> QA_LOGFILE = "${TMPDIR}/qa.log"
> >>>>>>
> >>>>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
> >>>>>> -
> >>>>>> OEINCLUDELOGS ?= "yes"
> >>>>>> KERNEL_CONSOLE ?= "ttyS0"
> >>>>>>
> >>>>>> diff --git a/meta/conf/machine/include/qemu.inc b/meta/conf/machine/include/qemu.inc
> >>>>>> index 4122a88..9ef242f 100644
> >>>>>> --- a/meta/conf/machine/include/qemu.inc
> >>>>>> +++ b/meta/conf/machine/include/qemu.inc
> >>>>>> @@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
> >>>>>> IMAGE_FSTYPES ?= "tar.bz2 ext3"
> >>>>>>
> >>>>>> ROOT_FLASH_SIZE = "280"
> >>>>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
> >>>>>> -IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
> >>>>>>
> >>>>>> # Don't include kernels in standard images
> >>>>>> RDEPENDS_kernel-base = ""
> >>>>>> diff --git a/meta/recipes-core/images/core-image-minimal-initramfs.bb b/meta/recipes-core/images/core-image-minimal-initramfs.bb
> >>>>>> index 21aaa6c..3246d5c 100644
> >>>>>> --- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
> >>>>>> +++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
> >>>>>> @@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
> >>>>>> LICENSE = "MIT"
> >>>>>>
> >>>>>> inherit core-image
> >>>>>> +
> >>>>>> +IMAGE_ROOTFS_SIZE = "8192"
> >>>>>
> >>>>> I'm not really sure about this, an initramfs that's the same size as a
> >>>>> more fully featured rootfs?
> >>>>>
> >>>> That may be, then we need to increase the size slightly, but this will
> >>>> trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR,
> >>>> rather than the 64M which would be the default, this was to ensure it
> >>>> could get smaller. I don't have a current initramfs size, I will build
> >>>> and verify.
> >>>>
> >>>>
> >>>>>> diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
> >>>>>> index aa00e67..743e121 100644
> >>>>>> --- a/meta/recipes-core/images/core-image-minimal.bb
> >>>>>> +++ b/meta/recipes-core/images/core-image-minimal.bb
> >>>>>> @@ -9,5 +9,7 @@ LICENSE = "MIT"
> >>>>>>
> >>>>>> inherit core-image
> >>>>>>
> >>>>>> +IMAGE_ROOTFS_SIZE = "8192"
> >>>>>> +
> >>>>>
> >>>>> In your cover letter you stated that the minimal image is currently
> >>>>> 9.9M, which means we *need* the overhead to contain the entire image
> >>>>> contents. Correct? That seems a little unwise.
> >>>>>
> >>>> Right, that's currently, we want to see the image get smaller even, so
> >>>> the 8M is an appropriate setting. Currently the actual ext3 image is
> >>>> 13M with 10M of contents.
> >>>>
> >>>> So are you suggested that the 8M size is unwise or the overhead, not
> >>>> sure I am catching your meaning here.
> >>>
> >>> I'm just nervous about relying on the overhead, what if someone sets it
> >>> lower and then the image doesn't fit?
> >>>
> >>> Having the goal of a smaller minimal image is good but in my opinion we
> >>> should adjust the rootfs size when the image is smaller, not before.
> >>
> >> Every argument I've heard in favor of the IMAGE_OVERHEAD was centered
> >> around ensuring there is enough space for the user to actually use the
> >> system after it was installed and booted. Using this single metric to
> >> try and both make up for oversized images and provide space for user
> >> data seems like to cause maintenance issues going forward.
> >>
> >> While our goal should be to reduce the image size, the default image
> >> size in the recipes should default to the largest of the images built
> >> for the supported machines. The overhead should be used strictly for
> >> user data.
> >>
> >
> > So it turns out that if IMAGE_ROOTFS is smaller than the contents the
> > size of the contents is used. Once this if test has been performed the
> > overhead factor is applied.
>
> Ah Good!
>
> >
> > I found this to be a tad confusing (hence all the back and forth here)
> > so have filed a documentation bug about it:
> > http://bugzilla.pokylinux.org/show_bug.cgi?id=1110
> >
> > Maybe we can have some comments in the code near where those variables
> > are set too?
> >
>
> Yeah, this wasn't clear to me either. And my question in 2/2 still
> stands - what is the goal of the overhead factor?
>
My understanding is that it's for system overhead such as postinstalls
whereas ROOTFS_IMAGE_EXTRA_SPACE is for user and data space that you
want above the standard image size.
So if you want to ensure all of your images have 2GB space for data
you'd set the ROOTFS_IMAGE_EXTRA_SPACE to 2GB.
Cheers,
Joshua
--
Joshua Lock
Yocto Project Build Monkey
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-26 21:03 ` Darren Hart
2011-05-26 21:13 ` Joshua Lock
@ 2011-05-26 21:14 ` Paul Eggleton
2011-05-27 4:07 ` Darren Hart
2011-06-02 1:46 ` Tom Rini
2 siblings, 1 reply; 20+ messages in thread
From: Paul Eggleton @ 2011-05-26 21:14 UTC (permalink / raw)
To: openembedded-core; +Cc: Darren Hart
On Thursday 26 May 2011 22:03:02 Darren Hart wrote:
> Yeah, this wasn't clear to me either. And my question in 2/2 still
> stands - what is the goal of the overhead factor?
I think the thinking was that the more software you have installed to begin
with the more user data you're likely to need to store. Personally I think it
would be simpler if we just set some additional overhead as an absolute value
rather than a factor - if it needs to be different per image type then it
easily can be.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-26 21:13 ` Joshua Lock
@ 2011-05-26 21:15 ` Saul Wold
2011-05-26 21:31 ` Joshua Lock
0 siblings, 1 reply; 20+ messages in thread
From: Saul Wold @ 2011-05-26 21:15 UTC (permalink / raw)
To: Joshua Lock; +Cc: Darren Hart, Patches and discussions about the oe-core layer
On 05/26/2011 02:13 PM, Joshua Lock wrote:
> On Thu, 2011-05-26 at 14:03 -0700, Darren Hart wrote:
>>
>> On 05/26/2011 02:01 PM, Joshua Lock wrote:
>>> On Thu, 2011-05-26 at 13:55 -0700, Darren Hart wrote:
>>>>
>>>> On 05/26/2011 12:54 PM, Joshua Lock wrote:
>>>>> On Thu, 2011-05-26 at 11:28 -0700, Saul Wold wrote:
>>>>>> On 05/26/2011 11:04 AM, Joshua Lock wrote:
>>>>>>> On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
>>>>>>>> This basic cleanup removes the _ext2/3 overrides from places they
>>>>>>>> no longer belong since they did not allow further overrides. In doing
>>>>>>>> this the core-image-minimal* recipes can now set a reasonably small
>>>>>>>> rootfs so that it's a realistic size for minimal.
>>>>>>>
>>>>>>> Awesome. Thanks for tackling this!
>>>>>>>
>>>>>>> Few questions below.
>>>>>>>
>>>>>>>>
>>>>>>>> The new default for minimal is 8M and will be adujsted upward by the
>>>>>>>> IMAGE_OVERHEAD_FACTOR (of 1.2).
>>>>>>>>
>>>>>>>> This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
>>>>>>>>
>>>>>>>> Signed-off-by: Saul Wold<sgw@linux.intel.com>
>>>>>>>> ---
>>>>>>>> meta/classes/image_types.bbclass | 7 +++++--
>>>>>>>> meta/conf/distro/include/default-distrovars.inc | 2 --
>>>>>>>> meta/conf/machine/include/qemu.inc | 2 --
>>>>>>>> .../images/core-image-minimal-initramfs.bb | 2 ++
>>>>>>>> meta/recipes-core/images/core-image-minimal.bb | 2 ++
>>>>>>>> 5 files changed, 9 insertions(+), 6 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
>>>>>>>> index ec0cafd..69f859e 100644
>>>>>>>> --- a/meta/classes/image_types.bbclass
>>>>>>>> +++ b/meta/classes/image_types.bbclass
>>>>>>>> @@ -21,22 +21,25 @@ runimagecmd () {
>>>>>>>> }
>>>>>>>>
>>>>>>>> IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
>>>>>>>> +
>>>>>>>> IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
>>>>>>>> +
>>>>>>>> IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
>>>>>>>> IMAGE_CMD_ext2.gz () {
>>>>>>>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>>>>>>>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>>>>>>>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>>>>>>>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>>>>>>>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
>>>>>>>> rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>>>>>>>> }
>>>>>>>> +
>>>>>>>> IMAGE_CMD_ext3 () {
>>>>>>>> genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
>>>>>>>> tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
>>>>>>>> }
>>>>>>>> IMAGE_CMD_ext3.gz () {
>>>>>>>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>>>>>>>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>>>>>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>>>>>> tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>>>>>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>>>>>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
>>>>>>>> diff --git a/meta/conf/distro/include/default-distrovars.inc b/meta/conf/distro/include/default-distrovars.inc
>>>>>>>> index 1aa45c8..4b68a0a 100644
>>>>>>>> --- a/meta/conf/distro/include/default-distrovars.inc
>>>>>>>> +++ b/meta/conf/distro/include/default-distrovars.inc
>>>>>>>> @@ -1,7 +1,5 @@
>>>>>>>> QA_LOGFILE = "${TMPDIR}/qa.log"
>>>>>>>>
>>>>>>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
>>>>>>>> -
>>>>>>>> OEINCLUDELOGS ?= "yes"
>>>>>>>> KERNEL_CONSOLE ?= "ttyS0"
>>>>>>>>
>>>>>>>> diff --git a/meta/conf/machine/include/qemu.inc b/meta/conf/machine/include/qemu.inc
>>>>>>>> index 4122a88..9ef242f 100644
>>>>>>>> --- a/meta/conf/machine/include/qemu.inc
>>>>>>>> +++ b/meta/conf/machine/include/qemu.inc
>>>>>>>> @@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
>>>>>>>> IMAGE_FSTYPES ?= "tar.bz2 ext3"
>>>>>>>>
>>>>>>>> ROOT_FLASH_SIZE = "280"
>>>>>>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
>>>>>>>> -IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
>>>>>>>>
>>>>>>>> # Don't include kernels in standard images
>>>>>>>> RDEPENDS_kernel-base = ""
>>>>>>>> diff --git a/meta/recipes-core/images/core-image-minimal-initramfs.bb b/meta/recipes-core/images/core-image-minimal-initramfs.bb
>>>>>>>> index 21aaa6c..3246d5c 100644
>>>>>>>> --- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
>>>>>>>> +++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
>>>>>>>> @@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
>>>>>>>> LICENSE = "MIT"
>>>>>>>>
>>>>>>>> inherit core-image
>>>>>>>> +
>>>>>>>> +IMAGE_ROOTFS_SIZE = "8192"
>>>>>>>
>>>>>>> I'm not really sure about this, an initramfs that's the same size as a
>>>>>>> more fully featured rootfs?
>>>>>>>
>>>>>> That may be, then we need to increase the size slightly, but this will
>>>>>> trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR,
>>>>>> rather than the 64M which would be the default, this was to ensure it
>>>>>> could get smaller. I don't have a current initramfs size, I will build
>>>>>> and verify.
>>>>>>
>>>>>>
>>>>>>>> diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
>>>>>>>> index aa00e67..743e121 100644
>>>>>>>> --- a/meta/recipes-core/images/core-image-minimal.bb
>>>>>>>> +++ b/meta/recipes-core/images/core-image-minimal.bb
>>>>>>>> @@ -9,5 +9,7 @@ LICENSE = "MIT"
>>>>>>>>
>>>>>>>> inherit core-image
>>>>>>>>
>>>>>>>> +IMAGE_ROOTFS_SIZE = "8192"
>>>>>>>> +
>>>>>>>
>>>>>>> In your cover letter you stated that the minimal image is currently
>>>>>>> 9.9M, which means we *need* the overhead to contain the entire image
>>>>>>> contents. Correct? That seems a little unwise.
>>>>>>>
>>>>>> Right, that's currently, we want to see the image get smaller even, so
>>>>>> the 8M is an appropriate setting. Currently the actual ext3 image is
>>>>>> 13M with 10M of contents.
>>>>>>
>>>>>> So are you suggested that the 8M size is unwise or the overhead, not
>>>>>> sure I am catching your meaning here.
>>>>>
>>>>> I'm just nervous about relying on the overhead, what if someone sets it
>>>>> lower and then the image doesn't fit?
>>>>>
>>>>> Having the goal of a smaller minimal image is good but in my opinion we
>>>>> should adjust the rootfs size when the image is smaller, not before.
>>>>
>>>> Every argument I've heard in favor of the IMAGE_OVERHEAD was centered
>>>> around ensuring there is enough space for the user to actually use the
>>>> system after it was installed and booted. Using this single metric to
>>>> try and both make up for oversized images and provide space for user
>>>> data seems like to cause maintenance issues going forward.
>>>>
>>>> While our goal should be to reduce the image size, the default image
>>>> size in the recipes should default to the largest of the images built
>>>> for the supported machines. The overhead should be used strictly for
>>>> user data.
>>>>
>>>
>>> So it turns out that if IMAGE_ROOTFS is smaller than the contents the
>>> size of the contents is used. Once this if test has been performed the
>>> overhead factor is applied.
>>
>> Ah Good!
>>
>>>
>>> I found this to be a tad confusing (hence all the back and forth here)
>>> so have filed a documentation bug about it:
>>> http://bugzilla.pokylinux.org/show_bug.cgi?id=1110
>>>
>>> Maybe we can have some comments in the code near where those variables
>>> are set too?
>>>
>>
>> Yeah, this wasn't clear to me either. And my question in 2/2 still
>> stands - what is the goal of the overhead factor?
>>
>
> My understanding is that it's for system overhead such as postinstalls
> whereas ROOTFS_IMAGE_EXTRA_SPACE is for user and data space that you
> want above the standard image size.
>
Correct
> So if you want to ensure all of your images have 2GB space for data
> you'd set the ROOTFS_IMAGE_EXTRA_SPACE to 2GB.
>
That's one way, another is if you want all your images to be 2G
including the core contents set ROOTFS_IMAGE_SIZE to 2G and the images
will all be 2G (remember to set in K, since we don't parse the M or G).
Sau!
> Cheers,
> Joshua
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-26 21:15 ` Saul Wold
@ 2011-05-26 21:31 ` Joshua Lock
0 siblings, 0 replies; 20+ messages in thread
From: Joshua Lock @ 2011-05-26 21:31 UTC (permalink / raw)
To: Saul Wold; +Cc: Darren Hart, Patches, the oe-core layer
On Thu, 2011-05-26 at 14:15 -0700, Saul Wold wrote:
> On 05/26/2011 02:13 PM, Joshua Lock wrote:
> > On Thu, 2011-05-26 at 14:03 -0700, Darren Hart wrote:
> >>
> >> On 05/26/2011 02:01 PM, Joshua Lock wrote:
> >>> On Thu, 2011-05-26 at 13:55 -0700, Darren Hart wrote:
> >>>>
> >>>> On 05/26/2011 12:54 PM, Joshua Lock wrote:
> >>>>> On Thu, 2011-05-26 at 11:28 -0700, Saul Wold wrote:
> >>>>>> On 05/26/2011 11:04 AM, Joshua Lock wrote:
> >>>>>>> On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
> >>>>>>>> This basic cleanup removes the _ext2/3 overrides from places they
> >>>>>>>> no longer belong since they did not allow further overrides. In doing
> >>>>>>>> this the core-image-minimal* recipes can now set a reasonably small
> >>>>>>>> rootfs so that it's a realistic size for minimal.
> >>>>>>>
> >>>>>>> Awesome. Thanks for tackling this!
> >>>>>>>
> >>>>>>> Few questions below.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> The new default for minimal is 8M and will be adujsted upward by the
> >>>>>>>> IMAGE_OVERHEAD_FACTOR (of 1.2).
> >>>>>>>>
> >>>>>>>> This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
> >>>>>>>>
> >>>>>>>> Signed-off-by: Saul Wold<sgw@linux.intel.com>
> >>>>>>>> ---
> >>>>>>>> meta/classes/image_types.bbclass | 7 +++++--
> >>>>>>>> meta/conf/distro/include/default-distrovars.inc | 2 --
> >>>>>>>> meta/conf/machine/include/qemu.inc | 2 --
> >>>>>>>> .../images/core-image-minimal-initramfs.bb | 2 ++
> >>>>>>>> meta/recipes-core/images/core-image-minimal.bb | 2 ++
> >>>>>>>> 5 files changed, 9 insertions(+), 6 deletions(-)
> >>>>>>>>
> >>>>>>>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
> >>>>>>>> index ec0cafd..69f859e 100644
> >>>>>>>> --- a/meta/classes/image_types.bbclass
> >>>>>>>> +++ b/meta/classes/image_types.bbclass
> >>>>>>>> @@ -21,22 +21,25 @@ runimagecmd () {
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>> IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
> >>>>>>>> +
> >>>>>>>> IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
> >>>>>>>> +
> >>>>>>>> IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
> >>>>>>>> IMAGE_CMD_ext2.gz () {
> >>>>>>>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> >>>>>>>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> >>>>>>>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> >>>>>>>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
> >>>>>>>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
> >>>>>>>> rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> >>>>>>>> }
> >>>>>>>> +
> >>>>>>>> IMAGE_CMD_ext3 () {
> >>>>>>>> genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
> >>>>>>>> tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
> >>>>>>>> }
> >>>>>>>> IMAGE_CMD_ext3.gz () {
> >>>>>>>> rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&& mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
> >>>>>>>> - genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >>>>>>>> + genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >>>>>>>> tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >>>>>>>> gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
> >>>>>>>> mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
> >>>>>>>> diff --git a/meta/conf/distro/include/default-distrovars.inc b/meta/conf/distro/include/default-distrovars.inc
> >>>>>>>> index 1aa45c8..4b68a0a 100644
> >>>>>>>> --- a/meta/conf/distro/include/default-distrovars.inc
> >>>>>>>> +++ b/meta/conf/distro/include/default-distrovars.inc
> >>>>>>>> @@ -1,7 +1,5 @@
> >>>>>>>> QA_LOGFILE = "${TMPDIR}/qa.log"
> >>>>>>>>
> >>>>>>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
> >>>>>>>> -
> >>>>>>>> OEINCLUDELOGS ?= "yes"
> >>>>>>>> KERNEL_CONSOLE ?= "ttyS0"
> >>>>>>>>
> >>>>>>>> diff --git a/meta/conf/machine/include/qemu.inc b/meta/conf/machine/include/qemu.inc
> >>>>>>>> index 4122a88..9ef242f 100644
> >>>>>>>> --- a/meta/conf/machine/include/qemu.inc
> >>>>>>>> +++ b/meta/conf/machine/include/qemu.inc
> >>>>>>>> @@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
> >>>>>>>> IMAGE_FSTYPES ?= "tar.bz2 ext3"
> >>>>>>>>
> >>>>>>>> ROOT_FLASH_SIZE = "280"
> >>>>>>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
> >>>>>>>> -IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
> >>>>>>>>
> >>>>>>>> # Don't include kernels in standard images
> >>>>>>>> RDEPENDS_kernel-base = ""
> >>>>>>>> diff --git a/meta/recipes-core/images/core-image-minimal-initramfs.bb b/meta/recipes-core/images/core-image-minimal-initramfs.bb
> >>>>>>>> index 21aaa6c..3246d5c 100644
> >>>>>>>> --- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
> >>>>>>>> +++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
> >>>>>>>> @@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
> >>>>>>>> LICENSE = "MIT"
> >>>>>>>>
> >>>>>>>> inherit core-image
> >>>>>>>> +
> >>>>>>>> +IMAGE_ROOTFS_SIZE = "8192"
> >>>>>>>
> >>>>>>> I'm not really sure about this, an initramfs that's the same size as a
> >>>>>>> more fully featured rootfs?
> >>>>>>>
> >>>>>> That may be, then we need to increase the size slightly, but this will
> >>>>>> trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR,
> >>>>>> rather than the 64M which would be the default, this was to ensure it
> >>>>>> could get smaller. I don't have a current initramfs size, I will build
> >>>>>> and verify.
> >>>>>>
> >>>>>>
> >>>>>>>> diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
> >>>>>>>> index aa00e67..743e121 100644
> >>>>>>>> --- a/meta/recipes-core/images/core-image-minimal.bb
> >>>>>>>> +++ b/meta/recipes-core/images/core-image-minimal.bb
> >>>>>>>> @@ -9,5 +9,7 @@ LICENSE = "MIT"
> >>>>>>>>
> >>>>>>>> inherit core-image
> >>>>>>>>
> >>>>>>>> +IMAGE_ROOTFS_SIZE = "8192"
> >>>>>>>> +
> >>>>>>>
> >>>>>>> In your cover letter you stated that the minimal image is currently
> >>>>>>> 9.9M, which means we *need* the overhead to contain the entire image
> >>>>>>> contents. Correct? That seems a little unwise.
> >>>>>>>
> >>>>>> Right, that's currently, we want to see the image get smaller even, so
> >>>>>> the 8M is an appropriate setting. Currently the actual ext3 image is
> >>>>>> 13M with 10M of contents.
> >>>>>>
> >>>>>> So are you suggested that the 8M size is unwise or the overhead, not
> >>>>>> sure I am catching your meaning here.
> >>>>>
> >>>>> I'm just nervous about relying on the overhead, what if someone sets it
> >>>>> lower and then the image doesn't fit?
> >>>>>
> >>>>> Having the goal of a smaller minimal image is good but in my opinion we
> >>>>> should adjust the rootfs size when the image is smaller, not before.
> >>>>
> >>>> Every argument I've heard in favor of the IMAGE_OVERHEAD was centered
> >>>> around ensuring there is enough space for the user to actually use the
> >>>> system after it was installed and booted. Using this single metric to
> >>>> try and both make up for oversized images and provide space for user
> >>>> data seems like to cause maintenance issues going forward.
> >>>>
> >>>> While our goal should be to reduce the image size, the default image
> >>>> size in the recipes should default to the largest of the images built
> >>>> for the supported machines. The overhead should be used strictly for
> >>>> user data.
> >>>>
> >>>
> >>> So it turns out that if IMAGE_ROOTFS is smaller than the contents the
> >>> size of the contents is used. Once this if test has been performed the
> >>> overhead factor is applied.
> >>
> >> Ah Good!
> >>
> >>>
> >>> I found this to be a tad confusing (hence all the back and forth here)
> >>> so have filed a documentation bug about it:
> >>> http://bugzilla.pokylinux.org/show_bug.cgi?id=1110
> >>>
> >>> Maybe we can have some comments in the code near where those variables
> >>> are set too?
> >>>
> >>
> >> Yeah, this wasn't clear to me either. And my question in 2/2 still
> >> stands - what is the goal of the overhead factor?
> >>
> >
> > My understanding is that it's for system overhead such as postinstalls
> > whereas ROOTFS_IMAGE_EXTRA_SPACE is for user and data space that you
> > want above the standard image size.
> >
> Correct
>
> > So if you want to ensure all of your images have 2GB space for data
> > you'd set the ROOTFS_IMAGE_EXTRA_SPACE to 2GB.
> >
> That's one way, another is if you want all your images to be 2G
> including the core contents set ROOTFS_IMAGE_SIZE to 2G and the images
> will all be 2G (remember to set in K, since we don't parse the M or G).
We probably should though. Shall I file a feature request? :-)
Cheers,
Joshua
--
Joshua Lock
Yocto Project Build Monkey
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-26 21:14 ` Paul Eggleton
@ 2011-05-27 4:07 ` Darren Hart
2011-05-27 4:29 ` Saul Wold
0 siblings, 1 reply; 20+ messages in thread
From: Darren Hart @ 2011-05-27 4:07 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
On 05/26/2011 02:14 PM, Paul Eggleton wrote:
> On Thursday 26 May 2011 22:03:02 Darren Hart wrote:
>> Yeah, this wasn't clear to me either. And my question in 2/2 still
>> stands - what is the goal of the overhead factor?
>
> I think the thinking was that the more software you have installed to begin
> with the more user data you're likely to need to store. Personally I think it
> would be simpler if we just set some additional overhead as an absolute value
I had the same thought, so "seconded".
--
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-27 4:07 ` Darren Hart
@ 2011-05-27 4:29 ` Saul Wold
2011-05-27 5:22 ` Darren Hart
0 siblings, 1 reply; 20+ messages in thread
From: Saul Wold @ 2011-05-27 4:29 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
Cc: Paul Eggleton, Darren Hart
On 05/26/2011 09:07 PM, Darren Hart wrote:
>
>
> On 05/26/2011 02:14 PM, Paul Eggleton wrote:
>> On Thursday 26 May 2011 22:03:02 Darren Hart wrote:
>>> Yeah, this wasn't clear to me either. And my question in 2/2 still
>>> stands - what is the goal of the overhead factor?
>>
>> I think the thinking was that the more software you have installed to begin
>> with the more user data you're likely to need to store. Personally I think it
>> would be simpler if we just set some additional overhead as an absolute value
>
> I had the same thought, so "seconded".
>
It used to be that way and failed in unpredictable ways, what would work
for one image would be way over kill for another or be too little, so
the overhead factor works better.
If you want a known amount of space then you can use the
IMAGE_ROOTFS_EXTRA_SPACE.
Sau!
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-27 4:29 ` Saul Wold
@ 2011-05-27 5:22 ` Darren Hart
0 siblings, 0 replies; 20+ messages in thread
From: Darren Hart @ 2011-05-27 5:22 UTC (permalink / raw)
To: Saul Wold; +Cc: Paul Eggleton, Patches and discussions about the oe-core layer
On 05/26/2011 09:29 PM, Saul Wold wrote:
> On 05/26/2011 09:07 PM, Darren Hart wrote:
>>
>>
>> On 05/26/2011 02:14 PM, Paul Eggleton wrote:
>>> On Thursday 26 May 2011 22:03:02 Darren Hart wrote:
>>>> Yeah, this wasn't clear to me either. And my question in 2/2 still
>>>> stands - what is the goal of the overhead factor?
>>>
>>> I think the thinking was that the more software you have installed to begin
>>> with the more user data you're likely to need to store. Personally I think it
>>> would be simpler if we just set some additional overhead as an absolute value
>>
>> I had the same thought, so "seconded".
>>
> It used to be that way and failed in unpredictable ways, what would work
> for one image would be way over kill for another or be too little, so
> the overhead factor works better.
>
Seems to me this is just slightly more complicated, but every bit as
likely to fail in unexpected ways in the future. Perhaps the simplest,
most robust, and most explicit way to address this would be to set the
IMAGE_ROOTFS_EXTRA_SPACE in each IMAGE file, just as we set the base size?
This still ensures you get at least IMAGE_ROOTFS_EXTRA_SPACE freespace
since the base size will expand to be at least the size the image requires.
My final 0.02 USD on the matter.
> If you want a known amount of space then you can use the
> IMAGE_ROOTFS_EXTRA_SPACE.
>
> Sau!
--
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup
2011-05-26 21:03 ` Darren Hart
2011-05-26 21:13 ` Joshua Lock
2011-05-26 21:14 ` Paul Eggleton
@ 2011-06-02 1:46 ` Tom Rini
2 siblings, 0 replies; 20+ messages in thread
From: Tom Rini @ 2011-06-02 1:46 UTC (permalink / raw)
To: openembedded-core
On 05/26/2011 02:03 PM, Darren Hart wrote:
[snip]
> Yeah, this wasn't clear to me either. And my question in 2/2 still
> stands - what is the goal of the overhead factor?
I know I'm late to the thread, sorry, but the problem is that the 'du'
estimation we use for the common case of "image is larger than our
default" sucks pretty bad. Without some additional fudge factors (both
the multiplier and additional room) most images won't actually end up
fitting into an ext* image and genext2fs chokes. I know the rpmdb was
mentioned before but really it's everything. Take any image, poke at
what du says things take and what df shows it's taking and cry :(
--
Tom Rini
Mentor Graphics Corporation
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2011-06-02 1:50 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-05-24 6:38 [RFC 0/2] IMAGE_ROOTFS_SIZE Cleanup Saul Wold
2011-05-24 6:38 ` [RFC 1/2] " Saul Wold
2011-05-26 18:04 ` Joshua Lock
2011-05-26 18:28 ` Saul Wold
2011-05-26 19:54 ` Joshua Lock
2011-05-26 20:55 ` Darren Hart
2011-05-26 21:01 ` Joshua Lock
2011-05-26 21:03 ` Darren Hart
2011-05-26 21:13 ` Joshua Lock
2011-05-26 21:15 ` Saul Wold
2011-05-26 21:31 ` Joshua Lock
2011-05-26 21:14 ` Paul Eggleton
2011-05-27 4:07 ` Darren Hart
2011-05-27 4:29 ` Saul Wold
2011-05-27 5:22 ` Darren Hart
2011-06-02 1:46 ` Tom Rini
2011-05-24 6:38 ` [RFC 2/2] image_types: add IMAGE_ROOTFS_EXTRA_SPACE Saul Wold
2011-05-26 18:04 ` Joshua Lock
2011-05-26 18:22 ` Saul Wold
2011-05-26 20:58 ` Darren Hart
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox