[PATCH] Gracefully abort if rootfs.img gets to big

[PATCH] Gracefully abort if rootfs.img gets to big

[Chris Hallinan]

See https://bugzilla.yoctoproject.org/show_bug.cgi?id=7541

Current implementation of live image type cannot support image size
greater than 4GB. If the image gets too large, the build simply 
hangs silently, without any clues or debug output to indicate why.
This patch prevents the build hanging on rootfs.img getting too big.

---
 meta/classes/bootimg.bbclass | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/meta/classes/bootimg.bbclass b/meta/classes/bootimg.bbclass
index b1c03ba..e143e24 100644
--- a/meta/classes/bootimg.bbclass
+++ b/meta/classes/bootimg.bbclass
@@ -84,6 +84,13 @@ populate() {
 	fi
 
 	if [ -n "${ROOTFS}" ] && [ -s "${ROOTFS}" ]; then
+		ROOTFS_FILE_SIZE=`stat -L -c "%s" ${ROOTFS}`
+		bbnote "do_bootimg(): rootfs.img size = ${ROOTFS_FILE_SIZE}"
+		# Actual limit is difficult to determine
+		# This leaves ~128MiB headroom for initrd and boot files
+		if [ ${ROOTFS_FILE_SIZE} -gt 4160800000 ]; then
+			bbfatal "Root fs (rootfs.img) is too big for live image"
+		fi
 		install -m 0644 ${ROOTFS} ${DEST}/rootfs.img
 	fi
 
-- 
1.9.1

Re: [PATCH] Gracefully abort if rootfs.img gets to big

[akuster808]

Chris,

Should the hardcoded value '4160800000' be a variable so it can be 
overridden?

- armin

On 03/30/2015 01:50 PM, Chris Hallinan wrote:
> See https://bugzilla.yoctoproject.org/show_bug.cgi?id=7541
>
> Current implementation of live image type cannot support image size
> greater than 4GB. If the image gets too large, the build simply
> hangs silently, without any clues or debug output to indicate why.
> This patch prevents the build hanging on rootfs.img getting too big.
>
> ---
>   meta/classes/bootimg.bbclass | 7 +++++++
>   1 file changed, 7 insertions(+)
>
> diff --git a/meta/classes/bootimg.bbclass b/meta/classes/bootimg.bbclass
> index b1c03ba..e143e24 100644
> --- a/meta/classes/bootimg.bbclass
> +++ b/meta/classes/bootimg.bbclass
> @@ -84,6 +84,13 @@ populate() {
>   	fi
>
>   	if [ -n "${ROOTFS}" ] && [ -s "${ROOTFS}" ]; then
> +		ROOTFS_FILE_SIZE=`stat -L -c "%s" ${ROOTFS}`
> +		bbnote "do_bootimg(): rootfs.img size = ${ROOTFS_FILE_SIZE}"
> +		# Actual limit is difficult to determine
> +		# This leaves ~128MiB headroom for initrd and boot files
> +		if [ ${ROOTFS_FILE_SIZE} -gt 4160800000 ]; then
> +			bbfatal "Root fs (rootfs.img) is too big for live image"
> +		fi
>   		install -m 0644 ${ROOTFS} ${DEST}/rootfs.img
>   	fi
>
>

Re: [PATCH] Gracefully abort if rootfs.img gets to big

[Chris Hallinan]

Hi Armin,

Great catching up with you last week, and thanks for looking at my
patch.  I supposed it's a double edged sword.  If we make it a
variable, it can be set wrong and the build hang again.  OTOH, 128MB
is probably way more space than we really need to reserve, and I
supposed if someone were just banging up against the limit, they might
want to tweak it.  I'll defer to the collective judgement of the
group, and resubmit if that's the consensus.

Oh, and silly me...I figured the mail list daemon would put the
[Yocto] tag on the patch. Since it seems to tag our posts, that is a
mystery to me.  Maybe Michael can enlighten me off list. :-P

-Chris

On Mon, Mar 30, 2015 at 6:51 PM, akuster808 <akuster808@gmail.com> wrote:
> Chris,
>
> Should the hardcoded value '4160800000' be a variable so it can be
> overridden?
>
> - armin
>
>
> On 03/30/2015 01:50 PM, Chris Hallinan wrote:
>>
>> See https://bugzilla.yoctoproject.org/show_bug.cgi?id=7541
>>
>> Current implementation of live image type cannot support image size
>> greater than 4GB. If the image gets too large, the build simply
>> hangs silently, without any clues or debug output to indicate why.
>> This patch prevents the build hanging on rootfs.img getting too big.
>>
>> ---
>>   meta/classes/bootimg.bbclass | 7 +++++++
>>   1 file changed, 7 insertions(+)
>>
>> diff --git a/meta/classes/bootimg.bbclass b/meta/classes/bootimg.bbclass
>> index b1c03ba..e143e24 100644
>> --- a/meta/classes/bootimg.bbclass
>> +++ b/meta/classes/bootimg.bbclass
>> @@ -84,6 +84,13 @@ populate() {
>>         fi
>>
>>         if [ -n "${ROOTFS}" ] && [ -s "${ROOTFS}" ]; then
>> +               ROOTFS_FILE_SIZE=`stat -L -c "%s" ${ROOTFS}`
>> +               bbnote "do_bootimg(): rootfs.img size =
>> ${ROOTFS_FILE_SIZE}"
>> +               # Actual limit is difficult to determine
>> +               # This leaves ~128MiB headroom for initrd and boot files
>> +               if [ ${ROOTFS_FILE_SIZE} -gt 4160800000 ]; then
>> +                       bbfatal "Root fs (rootfs.img) is too big for live
>> image"
>> +               fi
>>                 install -m 0644 ${ROOTFS} ${DEST}/rootfs.img
>>         fi
>>
>>
>



-- 
Life is like Linux - it never stands still.

Re: [PATCH] Gracefully abort if rootfs.img gets to big

[Paul Eggleton]

Hi Chris,

Great to see you at ELC!

On Monday 30 March 2015 19:17:46 Chris Hallinan wrote:
> Great catching up with you last week, and thanks for looking at my
> patch.  I supposed it's a double edged sword.  If we make it a
> variable, it can be set wrong and the build hang again.  OTOH, 128MB
> is probably way more space than we really need to reserve, and I
> supposed if someone were just banging up against the limit, they might
> want to tweak it.  I'll defer to the collective judgement of the
> group, and resubmit if that's the consensus.

Is there any practical way we can measure the actual size of these files?

> Oh, and silly me...I figured the mail list daemon would put the
> [Yocto] tag on the patch. Since it seems to tag our posts, that is a
> mystery to me.

The mailing list always prepends the list name to the subject of posted 
messages.

A few other notes:

1) This patch should be sent to the openembedded-core@lists.openembedded.org 
mailing list rather than here. 

2) The YP bugzilla reference should be of the format [YOCTO #7541] .

3) The shortlog (first line of the commit message / subject) should be prefixed 
with the area/file the patch is changing i.e. for this one it should be  
"bootimg.bbclass: gracefully abort if rootfs.img gets to big".

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [PATCH] Gracefully abort if rootfs.img gets to big

[Chris Hallinan]

On Tue, Mar 31, 2015 at 9:17 AM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> Hi Chris,
>
> Great to see you at ELC!
>
> On Monday 30 March 2015 19:17:46 Chris Hallinan wrote:
>> Great catching up with you last week, and thanks for looking at my
>> patch.  I supposed it's a double edged sword.  If we make it a
>> variable, it can be set wrong and the build hang again.  OTOH, 128MB
>> is probably way more space than we really need to reserve, and I
>> supposed if someone were just banging up against the limit, they might
>> want to tweak it.  I'll defer to the collective judgement of the
>> group, and resubmit if that's the consensus.
>
> Is there any practical way we can measure the actual size of these files?

I gave this some thought after Armin's comment, and yes, maybe there
is a way to estimate the size available. I'll look further into it.

>
>> Oh, and silly me...I figured the mail list daemon would put the
>> [Yocto] tag on the patch. Since it seems to tag our posts, that is a
>> mystery to me.
>
> The mailing list always prepends the list name to the subject of posted
> messages.

That what I thought, too.  Wonder why it didn't prepend the list name
to this patch?


> A few other notes:
>
> 1) This patch should be sent to the openembedded-core@lists.openembedded.org
> mailing list rather than here.

Ah, wasn't sure about that, thanks!

>
> 2) The YP bugzilla reference should be of the format [YOCTO #7541] .
>
> 3) The shortlog (first line of the commit message / subject) should be prefixed
> with the area/file the patch is changing i.e. for this one it should be
> "bootimg.bbclass: gracefully abort if rootfs.img gets to big".

Thanks for the guidance. As you can see, I'm not experienced at
sending patches!  I'll investigate and re-send to oe-core@

Regards,

Chris

Re: [PATCH] Gracefully abort if rootfs.img gets to big

[Paul Eggleton]

On Tuesday 31 March 2015 10:16:52 Chris Hallinan wrote:
> On Tue, Mar 31, 2015 at 9:17 AM, Paul Eggleton
> <paul.eggleton@linux.intel.com> wrote:
> > On Monday 30 March 2015 19:17:46 Chris Hallinan wrote:
> >> Great catching up with you last week, and thanks for looking at my
> >> patch.  I supposed it's a double edged sword.  If we make it a
> >> variable, it can be set wrong and the build hang again.  OTOH, 128MB
> >> is probably way more space than we really need to reserve, and I
> >> supposed if someone were just banging up against the limit, they might
> >> want to tweak it.  I'll defer to the collective judgement of the
> >> group, and resubmit if that's the consensus.
> > 
> > Is there any practical way we can measure the actual size of these files?
> 
> I gave this some thought after Armin's comment, and yes, maybe there
> is a way to estimate the size available. I'll look further into it.
> 
> >> Oh, and silly me...I figured the mail list daemon would put the
> >> [Yocto] tag on the patch. Since it seems to tag our posts, that is a
> >> mystery to me.
> > 
> > The mailing list always prepends the list name to the subject of posted
> > messages.
> 
> That what I thought, too.  Wonder why it didn't prepend the list name
> to this patch?

It did - see:
https://lists.yoctoproject.org/pipermail/yocto/2015-March/024294.html
 
> > A few other notes:
> > 
> > 1) This patch should be sent to the
> > openembedded-core@lists.openembedded.org mailing list rather than here.
> 
> Ah, wasn't sure about that, thanks!
> 
> > 2) The YP bugzilla reference should be of the format [YOCTO #7541] .
> > 
> > 3) The shortlog (first line of the commit message / subject) should be
> > prefixed with the area/file the patch is changing i.e. for this one it
> > should be "bootimg.bbclass: gracefully abort if rootfs.img gets to big".
> 
> Thanks for the guidance. As you can see, I'm not experienced at
> sending patches!  I'll investigate and re-send to oe-core@

Great!

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre