Re: [RFC 1/2] IMAGE_ROOTFS_SIZE Cleanup

[Darren Hart <dvhart@linux.intel.com>]

On 05/26/2011 02:01 PM, Joshua Lock wrote:
> On Thu, 2011-05-26 at 13:55 -0700, Darren Hart wrote:
>>
>> On 05/26/2011 12:54 PM, Joshua Lock wrote:
>>> On Thu, 2011-05-26 at 11:28 -0700, Saul Wold wrote:
>>>> On 05/26/2011 11:04 AM, Joshua Lock wrote:
>>>>> On Mon, 2011-05-23 at 23:38 -0700, Saul Wold wrote:
>>>>>> This basic cleanup removes the _ext2/3 overrides from places they
>>>>>> no longer belong since they did not allow further overrides. In doing
>>>>>> this the core-image-minimal* recipes can now set a reasonably small
>>>>>> rootfs so that it's a realistic size for minimal.
>>>>>
>>>>> Awesome. Thanks for tackling this!
>>>>>
>>>>> Few questions below.
>>>>>
>>>>>>
>>>>>> The new default for minimal is 8M and will be adujsted upward by the
>>>>>> IMAGE_OVERHEAD_FACTOR (of 1.2).
>>>>>>
>>>>>> This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype>  code
>>>>>>
>>>>>> Signed-off-by: Saul Wold<sgw@linux.intel.com>
>>>>>> ---
>>>>>>   meta/classes/image_types.bbclass                   |    7 +++++--
>>>>>>   meta/conf/distro/include/default-distrovars.inc    |    2 --
>>>>>>   meta/conf/machine/include/qemu.inc                 |    2 --
>>>>>>   .../images/core-image-minimal-initramfs.bb         |    2 ++
>>>>>>   meta/recipes-core/images/core-image-minimal.bb     |    2 ++
>>>>>>   5 files changed, 9 insertions(+), 6 deletions(-)
>>>>>>
>>>>>> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
>>>>>> index ec0cafd..69f859e 100644
>>>>>> --- a/meta/classes/image_types.bbclass
>>>>>> +++ b/meta/classes/image_types.bbclass
>>>>>> @@ -21,22 +21,25 @@ runimagecmd () {
>>>>>>   }
>>>>>>
>>>>>>   IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
>>>>>> +
>>>>>>   IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
>>>>>> +
>>>>>>   IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
>>>>>>   IMAGE_CMD_ext2.gz () {
>>>>>>   	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>>>>>> -	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>>>>>> +	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>>>>>>   	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
>>>>>>   	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
>>>>>>   	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>>>>>>   }
>>>>>> +
>>>>>>   IMAGE_CMD_ext3 () {
>>>>>>   	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
>>>>>>   	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
>>>>>>   }
>>>>>>   IMAGE_CMD_ext3.gz () {
>>>>>>   	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
>>>>>> -	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>>>> +	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>>>>   	tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>>>>   	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
>>>>>>   	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
>>>>>> diff --git a/meta/conf/distro/include/default-distrovars.inc b/meta/conf/distro/include/default-distrovars.inc
>>>>>> index 1aa45c8..4b68a0a 100644
>>>>>> --- a/meta/conf/distro/include/default-distrovars.inc
>>>>>> +++ b/meta/conf/distro/include/default-distrovars.inc
>>>>>> @@ -1,7 +1,5 @@
>>>>>>   QA_LOGFILE = "${TMPDIR}/qa.log"
>>>>>>
>>>>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
>>>>>> -
>>>>>>   OEINCLUDELOGS ?= "yes"
>>>>>>   KERNEL_CONSOLE ?= "ttyS0"
>>>>>>
>>>>>> diff --git a/meta/conf/machine/include/qemu.inc b/meta/conf/machine/include/qemu.inc
>>>>>> index 4122a88..9ef242f 100644
>>>>>> --- a/meta/conf/machine/include/qemu.inc
>>>>>> +++ b/meta/conf/machine/include/qemu.inc
>>>>>> @@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
>>>>>>   IMAGE_FSTYPES ?= "tar.bz2 ext3"
>>>>>>
>>>>>>   ROOT_FLASH_SIZE = "280"
>>>>>> -IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
>>>>>> -IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
>>>>>>
>>>>>>   # Don't include kernels in standard images
>>>>>>   RDEPENDS_kernel-base = ""
>>>>>> diff --git a/meta/recipes-core/images/core-image-minimal-initramfs.bb b/meta/recipes-core/images/core-image-minimal-initramfs.bb
>>>>>> index 21aaa6c..3246d5c 100644
>>>>>> --- a/meta/recipes-core/images/core-image-minimal-initramfs.bb
>>>>>> +++ b/meta/recipes-core/images/core-image-minimal-initramfs.bb
>>>>>> @@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
>>>>>>   LICENSE = "MIT"
>>>>>>
>>>>>>   inherit core-image
>>>>>> +
>>>>>> +IMAGE_ROOTFS_SIZE = "8192"
>>>>>
>>>>> I'm not really sure about this, an initramfs that's the same size as a
>>>>> more fully featured rootfs?
>>>>>
>>>> That may be, then we need to increase the size slightly, but this will 
>>>> trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR, 
>>>> rather than the 64M which would be the default, this was to ensure it 
>>>> could get smaller.  I don't have a current initramfs size, I will build 
>>>> and verify.
>>>>
>>>>
>>>>>> diff --git a/meta/recipes-core/images/core-image-minimal.bb b/meta/recipes-core/images/core-image-minimal.bb
>>>>>> index aa00e67..743e121 100644
>>>>>> --- a/meta/recipes-core/images/core-image-minimal.bb
>>>>>> +++ b/meta/recipes-core/images/core-image-minimal.bb
>>>>>> @@ -9,5 +9,7 @@ LICENSE = "MIT"
>>>>>>
>>>>>>   inherit core-image
>>>>>>
>>>>>> +IMAGE_ROOTFS_SIZE = "8192"
>>>>>> +
>>>>>
>>>>> In your cover letter you stated that the minimal image is currently
>>>>> 9.9M, which means we *need* the overhead to contain the entire image
>>>>> contents. Correct? That seems a little unwise.
>>>>>
>>>> Right, that's currently, we want to see the image get smaller even, so 
>>>> the 8M is an appropriate setting.  Currently the actual ext3 image is 
>>>> 13M with 10M of contents.
>>>>
>>>> So are you suggested that the 8M size is unwise or the overhead, not 
>>>> sure I am catching your meaning here.
>>>
>>> I'm just nervous about relying on the overhead, what if someone sets it
>>> lower and then the image doesn't fit?
>>>
>>> Having the goal of a smaller minimal image is good but in my opinion we
>>> should adjust the rootfs size when the image is smaller, not before.
>>
>> Every argument I've heard in favor of the IMAGE_OVERHEAD was centered
>> around ensuring there is enough space for the user to actually use the
>> system after it was installed and booted. Using this single metric to
>> try and both make up for oversized images and provide space for user
>> data seems like to cause maintenance issues going forward.
>>
>> While our goal should be to reduce the image size, the default image
>> size in the recipes should default to the largest of the images built
>> for the supported machines. The overhead should be used strictly for
>> user data.
>>
> 
> So it turns out that if IMAGE_ROOTFS is smaller than the contents the
> size of the contents is used. Once this if test has been performed the
> overhead factor is applied.

Ah Good!

> 
> I found this to be a tad confusing (hence all the back and forth here)
> so have filed a documentation bug about it:
> http://bugzilla.pokylinux.org/show_bug.cgi?id=1110
> 
> Maybe we can have some comments in the code near where those variables
> are set too?
> 

Yeah, this wasn't clear to me either. And my question in 2/2 still
stands - what is the goal of the overhead factor?

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel