Is the x86-64 kernel size limit real?

Is the x86-64 kernel size limit real?

[Olivier Galibert]

I get bitched at by the build process because the kernel I get is
around 4.5Mb compressed.  i386 does not have that limitation.
Interestingly, a diff between the two build.c gives:

--- ../../../i386/boot/tools/build.c	2006-06-22 20:19:33.000000000 +0200
+++ build.c	2006-06-22 20:19:33.000000000 +0200
@@ -70,8 +70,7 @@
 
 int main(int argc, char ** argv)
 {
-	unsigned int i, sz, setup_sectors;
-	int c;
+	unsigned int i, c, sz, setup_sectors;
 	u32 sys_size;
 	byte major_root, minor_root;
 	struct stat sb;
@@ -150,8 +149,10 @@
 	sz = sb.st_size;
 	fprintf (stderr, "System is %d kB\n", sz/1024);
 	sys_size = (sz + 15) / 16;
-	if (!is_big_kernel && sys_size > DEF_SYSSIZE)
-		die("System is too big. Try using bzImage or modules.");
+	/* 0x40000*16 = 4.0 MB, reasonable estimate for the current maximum */
+	if (sys_size > (is_big_kernel ? 0x40000 : DEF_SYSSIZE))
+		die("System is too big. Try using %smodules.",
+			is_big_kernel ? "" : "bzImage or ");
 	while (sz > 0) {
 		int l, n;
 

which shows two things:
1- a8f5034540195307362d071a8b387226b410469f should have a x86-64 version
2- the limit looks entirely artificial

So, is removing the limit prone to bite me?

  OG.

PS: Please do not turn this thread into a pro/against modules ones, TYVM.

Re: Is the x86-64 kernel size limit real?

[H. Peter Anvin]

Followup to:  <20060622204627.GA47994@dspnet.fr.eu.org>
By author:    Olivier Galibert <galibert@pobox.com>
In newsgroup: linux.dev.kernel
>
> I get bitched at by the build process because the kernel I get is
> around 4.5Mb compressed.  i386 does not have that limitation.
> Interestingly, a diff between the two build.c gives:
> 
> --- ../../../i386/boot/tools/build.c	2006-06-22 20:19:33.000000000 +0200
> +++ build.c	2006-06-22 20:19:33.000000000 +0200
> @@ -70,8 +70,7 @@
>  
>  int main(int argc, char ** argv)
>  {
> -	unsigned int i, sz, setup_sectors;
> -	int c;
> +	unsigned int i, c, sz, setup_sectors;
>  	u32 sys_size;
>  	byte major_root, minor_root;
>  	struct stat sb;
> @@ -150,8 +149,10 @@
>  	sz = sb.st_size;
>  	fprintf (stderr, "System is %d kB\n", sz/1024);
>  	sys_size = (sz + 15) / 16;
> -	if (!is_big_kernel && sys_size > DEF_SYSSIZE)
> -		die("System is too big. Try using bzImage or modules.");
> +	/* 0x40000*16 = 4.0 MB, reasonable estimate for the current maximum */
> +	if (sys_size > (is_big_kernel ? 0x40000 : DEF_SYSSIZE))
> +		die("System is too big. Try using %smodules.",
> +			is_big_kernel ? "" : "bzImage or ");
>  	while (sz > 0) {
>  		int l, n;
>  
> 
> which shows two things:
> 1- a8f5034540195307362d071a8b387226b410469f should have a x86-64 version
> 2- the limit looks entirely artificial
> 
> So, is removing the limit prone to bite me?
> 

It turns out x86-64, unlike i386, does still have a hardcoded limit,
but the limit in build.c is wrong:

kernel/head.S:
        /* 40MB kernel mapping. The kernel code cannot be bigger than that.
           When you change this change KERNEL_TEXT_SIZE in page.h too. */
        /* (2^48-(2*1024*1024*1024)-((2^39)*511)-((2^30)*510)) = 0 */

So this should be replaced by KERNEL_TEXT_SIZE in page.h, or better,
this should be done dynamically in x86-64 too.

	-hpa

Re: Is the x86-64 kernel size limit real?

[Olivier Galibert]

On Thu, Jun 22, 2006 at 02:38:02PM -0700, H. Peter Anvin wrote:
> It turns out x86-64, unlike i386, does still have a hardcoded limit,
> but the limit in build.c is wrong:
> 
> kernel/head.S:
>         /* 40MB kernel mapping. The kernel code cannot be bigger than that.
>            When you change this change KERNEL_TEXT_SIZE in page.h too. */
>         /* (2^48-(2*1024*1024*1024)-((2^39)*511)-((2^30)*510)) = 0 */
> 
> So this should be replaced by KERNEL_TEXT_SIZE in page.h, or better,
> this should be done dynamically in x86-64 too.

Interesting.  KERNEL_TEXT_SIZE wouldn't work though, since that's the
decompressed size while the 4Mb limit is on the compressed size.  As a
datapoint, though, the uncompressed image is 15.7Mb, for a 4.5Mb
compressed image.

  OG.

Re: Is the x86-64 kernel size limit real?

[H. Peter Anvin]

Olivier Galibert wrote:
> On Thu, Jun 22, 2006 at 02:38:02PM -0700, H. Peter Anvin wrote:
>> It turns out x86-64, unlike i386, does still have a hardcoded limit,
>> but the limit in build.c is wrong:
>>
>> kernel/head.S:
>>         /* 40MB kernel mapping. The kernel code cannot be bigger than that.
>>            When you change this change KERNEL_TEXT_SIZE in page.h too. */
>>         /* (2^48-(2*1024*1024*1024)-((2^39)*511)-((2^30)*510)) = 0 */
>>
>> So this should be replaced by KERNEL_TEXT_SIZE in page.h, or better,
>> this should be done dynamically in x86-64 too.
> 
> Interesting.  KERNEL_TEXT_SIZE wouldn't work though, since that's the
> decompressed size while the 4Mb limit is on the compressed size.  As a
> datapoint, though, the uncompressed image is 15.7Mb, for a 4.5Mb
> compressed image.
> 

Oh, right.  In fact, the 4 MB "limit" for i386 was actually an 8 MB uncompressed limit, 
with a 2:1 ratio assumed... not very accurate.

The limit should be removed from the boot tools; since we're talking uncompressed limits 
those should be tested in the linker script if anywhere.

	-hpa

Re: Is the x86-64 kernel size limit real?

[Olivier Galibert]

On Thu, Jun 22, 2006 at 03:45:41PM -0700, H. Peter Anvin wrote:
> The limit should be removed from the boot tools; since we're talking 
> uncompressed limits those should be tested in the linker script if anywhere.

Probably yeah.  And the two build.c files should become one too.
There is version drift already.  I'm not a x86-64 maintainer though :-)

  OG.

Re: Is the x86-64 kernel size limit real?

[Eduard-Gabriel Munteanu]

*This message was transferred with a trial version of CommuniGate(r) Pro*
Olivier Galibert wrote:

> 
> which shows two things:
> 1- a8f5034540195307362d071a8b387226b410469f should have a x86-64 version
> 2- the limit looks entirely artificial
> 
> So, is removing the limit prone to bite me?
> 
>   OG.

The build system merely tries to warn you it's not going to fit on a 
floppy disk. "bzImage" means "Big zImage", not "bz2-compressed Image", 
so unless you're building a floppy disk, don't use zImage.

Re: Is the x86-64 kernel size limit real?

[H. Peter Anvin]

Eduard-Gabriel Munteanu wrote:
> *This message was transferred with a trial version of CommuniGate(r) Pro*
> Olivier Galibert wrote:
> 
>>
>> which shows two things:
>> 1- a8f5034540195307362d071a8b387226b410469f should have a x86-64 version
>> 2- the limit looks entirely artificial
>>
>> So, is removing the limit prone to bite me?
>>
>>   OG.
> 
> The build system merely tries to warn you it's not going to fit on a 
> floppy disk. "bzImage" means "Big zImage", not "bz2-compressed Image", 
> so unless you're building a floppy disk, don't use zImage.
> 

He's talking about the bzImage limit, not the zImage limit.  The bzImage limit in x86-64 
is real (in the sense it exists) but incorrect (in the sense that it has the wrong value); 
see my other post.

	-hpa

Re: Is the x86-64 kernel size limit real?

[Olivier Galibert]

On Fri, Jun 23, 2006 at 12:27:08AM +0000, Eduard-Gabriel Munteanu wrote:
> *This message was transferred with a trial version of CommuniGate(r) Pro*
> Olivier Galibert wrote:
> 
> >
> >which shows two things:
> >1- a8f5034540195307362d071a8b387226b410469f should have a x86-64 version
> >2- the limit looks entirely artificial
> >
> >So, is removing the limit prone to bite me?
> >
> >  OG.
> 
> The build system merely tries to warn you it's not going to fit on a 
> floppy disk. "bzImage" means "Big zImage", not "bz2-compressed Image", 
> so unless you're building a floppy disk, don't use zImage.

You failed to notice the "is_big_kernel ? 0x40000 : ..." part, which
means the 4Mb limit is for bzImage.  And the "die(...)" part, which
means it's not a warning but an error.

  OG.

Re: Is the x86-64 kernel size limit real?

[Eduard-Gabriel Munteanu]

*This message was transferred with a trial version of CommuniGate(r) Pro*
Olivier Galibert wrote:
> *This message was transferred with a trial version of CommuniGate(r) Pro*
> *This message was transferred with a trial version of CommuniGate(r) Pro*
> On Fri, Jun 23, 2006 at 12:27:08AM +0000, Eduard-Gabriel Munteanu wrote:
> 
>>*This message was transferred with a trial version of CommuniGate(r) Pro*
>>Olivier Galibert wrote:
>>
>>
>>>which shows two things:
>>>1- a8f5034540195307362d071a8b387226b410469f should have a x86-64 version
>>>2- the limit looks entirely artificial
>>>
>>>So, is removing the limit prone to bite me?
>>>
>>> OG.
>>
>>The build system merely tries to warn you it's not going to fit on a 
>>floppy disk. "bzImage" means "Big zImage", not "bz2-compressed Image", 
>>so unless you're building a floppy disk, don't use zImage.
> 
> 
> You failed to notice the "is_big_kernel ? 0x40000 : ..." part, which
> means the 4Mb limit is for bzImage.  And the "die(...)" part, which
> means it's not a warning but an error.
> 

Sorry, I thought it was you who made that patch.

Re: Is the x86-64 kernel size limit real?

[Andi Kleen]

Olivier Galibert <galibert@pobox.com> writes:

> I get bitched at by the build process because the kernel I get is
> around 4.5Mb compressed.  i386 does not have that limitation.
> Interestingly, a diff between the two build.c gives:

A patch to fix it is already queued for 2.6.18

Also long term it might be completely dropped when the uncompressor
moves to long mode.

-Andi

Re: Is the x86-64 kernel size limit real?

[H. Peter Anvin]

Followup to:  <p73hd2cnik6.fsf@verdi.suse.de>
By author:    Andi Kleen <ak@suse.de>
In newsgroup: linux.dev.kernel
>
> Olivier Galibert <galibert@pobox.com> writes:
> 
> > I get bitched at by the build process because the kernel I get is
> > around 4.5Mb compressed.  i386 does not have that limitation.
> > Interestingly, a diff between the two build.c gives:
> 
> A patch to fix it is already queued for 2.6.18
> 
> Also long term it might be completely dropped when the uncompressor
> moves to long mode.
> 

It can be completely dropped now (and the directories unified); the
size limitation on the uncompressed size can be enforced in the linker
script.

The uncompressor only needs to be in long mode to support > 4 GB.

	-hpa