[PATCH] libxc: arm: Load the zImage to the rambase address + 2MB

[PATCH] libxc: arm: Load the zImage to the rambase address + 2MB

[Julien Grall]

Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000).

Kernel are usually using 1MB (or 2MB) mapping for the early page table. If
the kernel doesn't relocate itself this would require to virtual address
starting at 0xXXXX8000. This is not part of the zImage protocol, but a
convenience for Linux after the decompressor stage. Linux is able to
load at any address in the memory and it will relocate itself to respect
it own constraint.

Load the zImage at rambase address + 2MB to make life easier for other
kernel (such as FreeBSD, Mini-OS). FWIW, this is already the case for DOM0.

Signed-off-by: Julien Grall <julien.grall@linaro.org>

---

This patch is candidate for backporting. Xen 4.4 is suitable to boot any
guest OS, but without this patch it will require some modification in the
guest kernel.

Hence, with this patch DOM0 and the guest will have the same requirement
for booting.
---
 tools/libxc/xc_dom_armzimageloader.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxc/xc_dom_armzimageloader.c b/tools/libxc/xc_dom_armzimageloader.c
index 2b28781..2cf95a5 100644
--- a/tools/libxc/xc_dom_armzimageloader.c
+++ b/tools/libxc/xc_dom_armzimageloader.c
@@ -87,7 +87,7 @@ static int xc_dom_parse_zimage32_kernel(struct xc_dom_image *dom)
     zimage = (uint32_t *)dom->kernel_blob;
 
     /* Do not load kernel at the very first RAM address */
-    v_start = rambase + 0x8000;
+    v_start = rambase + 0x200000;
 
     if ( dom->kernel_size > UINT64_MAX - v_start )
     {
-- 
1.7.10.4

Re: [PATCH] libxc: arm: Load the zImage to the rambase address + 2MB

[Ian Campbell]

On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote:
> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000).
> 
> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If
> the kernel doesn't relocate itself this would require to virtual address
> starting at 0xXXXX8000. This is not part of the zImage protocol, but a
> convenience for Linux after the decompressor stage. Linux is able to
> load at any address in the memory and it will relocate itself to respect
> it own constraint.

Yes, because the boot protocol does not guarantee 2MB alignment, so if
the OS wants to rely on that then it is obliged to take care of
relocating itself.

> Load the zImage at rambase address + 2MB to make life easier for other
> kernel (such as FreeBSD, Mini-OS).

Whether or not it is easier these OSes *must* be prepared to be loaded
at any 4k aligned address. It sounds to me like you are hoping that
these OSes can *rely* on 2MB alignment, which is not the case. If they
make that assumption then they are buggy, regardless of whether it
happens to currently work with some loader or not.

> FWIW, this is already the case for DOM0.

On that basis I'm more inclined to change dom0 to use 32kb alignment.

Ian.

Re: [PATCH] libxc: arm: Load the zImage to the rambase address + 2MB

[Julien Grall]

On 07/03/2014 11:01 AM, Ian Campbell wrote:
> On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote:
>> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000).
>>
>> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If
>> the kernel doesn't relocate itself this would require to virtual address
>> starting at 0xXXXX8000. This is not part of the zImage protocol, but a
>> convenience for Linux after the decompressor stage. Linux is able to
>> load at any address in the memory and it will relocate itself to respect
>> it own constraint.
> 
> Yes, because the boot protocol does not guarantee 2MB alignment, so if
> the OS wants to rely on that then it is obliged to take care of
> relocating itself.
> 
>> Load the zImage at rambase address + 2MB to make life easier for other
>> kernel (such as FreeBSD, Mini-OS).
> 
> Whether or not it is easier these OSes *must* be prepared to be loaded
> at any 4k aligned address. It sounds to me like you are hoping that
> these OSes can *rely* on 2MB alignment, which is not the case. If they
> make that assumption then they are buggy, regardless of whether it
> happens to currently work with some loader or not.

Using 4K alignment make impossible to use 1MB or 2MB mapping for early
page table. Which make the code (usually in assembly) even harder to
write or to impose relocation in the assembly code.

>> FWIW, this is already the case for DOM0.
> 
> On that basis I'm more inclined to change dom0 to use 32kb alignment.

It's not really 32kb alignment but an address in 0xXXXX8000. If the
kernel doesn't want to do relocation (because it's a pain to write in
assembly code), it has to compile the kernel with a 0x8000 offset (see
mini-os patch series).

This solution, make the kernel more tight to Xen changes. I would prefer
if we relax this change allowing easier support of new kernel to 2MB
alignment.

Regards,

-- 
Julien Grall

Re: [PATCH] libxc: arm: Load the zImage to the rambase address + 2MB

[Thomas Leonard]

On 03/07/14 11:12, Julien Grall wrote:
> On 07/03/2014 11:01 AM, Ian Campbell wrote:
>> On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote:
>>> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000).
>>>
>>> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If
>>> the kernel doesn't relocate itself this would require to virtual address
>>> starting at 0xXXXX8000. This is not part of the zImage protocol, but a
>>> convenience for Linux after the decompressor stage. Linux is able to
>>> load at any address in the memory and it will relocate itself to respect
>>> it own constraint.
>>
>> Yes, because the boot protocol does not guarantee 2MB alignment, so if
>> the OS wants to rely on that then it is obliged to take care of
>> relocating itself.
>>
>>> Load the zImage at rambase address + 2MB to make life easier for other
>>> kernel (such as FreeBSD, Mini-OS).
>>
>> Whether or not it is easier these OSes *must* be prepared to be loaded
>> at any 4k aligned address. It sounds to me like you are hoping that
>> these OSes can *rely* on 2MB alignment, which is not the case. If they
>> make that assumption then they are buggy, regardless of whether it
>> happens to currently work with some loader or not.
>
> Using 4K alignment make impossible to use 1MB or 2MB mapping for early
> page table. Which make the code (usually in assembly) even harder to
> write or to impose relocation in the assembly code.

This isn't the case for Mini-OS, at least. We already use a 1MB mapping 
with the 0x8000 offset just fine. The translation table (using 1MB 
sections) is 16K, which would fit nicely in the 0x8000 gap (although 
currently we don't use that space).

Starting at 2MB would create an inconvenient 2MB of free memory just 
before the kernel, with no obvious way to tell xmalloc about it.

>>> FWIW, this is already the case for DOM0.
>>
>> On that basis I'm more inclined to change dom0 to use 32kb alignment.
>
> It's not really 32kb alignment but an address in 0xXXXX8000. If the
> kernel doesn't want to do relocation (because it's a pain to write in
> assembly code), it has to compile the kernel with a 0x8000 offset (see
> mini-os patch series).
>
> This solution, make the kernel more tight to Xen changes. I would prefer
> if we relax this change allowing easier support of new kernel to 2MB
> alignment.
>
> Regards,
>

Re: [PATCH] libxc: arm: Load the zImage to the rambase address + 2MB

[Julien Grall]

On 07/03/2014 11:37 AM, Thomas Leonard wrote:
>> Using 4K alignment make impossible to use 1MB or 2MB mapping for early
>> page table. Which make the code (usually in assembly) even harder to
>> write or to impose relocation in the assembly code.
> 
> This isn't the case for Mini-OS, at least. We already use a 1MB mapping
> with the 0x8000 offset just fine. The translation table (using 1MB
> sections) is 16K, which would fit nicely in the 0x8000 gap (although
> currently we don't use that space).

As I said in Mini-OS series, the spec doesn't say anything about the
start offset of the guest kernel for the Linux boot protocol.

0x8000 offset is very inconvenient for most of the other OS than Linux.
As Linux is able to relocate itself, I don't see any known issue about
starting at an 2MB-aligned address.

> Starting at 2MB would create an inconvenient 2MB of free memory just
> before the kernel, with no obvious way to tell xmalloc about it.

That's a bug in Mini-OS in this case. Why can't you use the whole memory?

Regards,

-- 
Julien Grall

Re: [PATCH] libxc: arm: Load the zImage to the rambase address + 2MB

[Ian Campbell]

On Thu, 2014-07-03 at 12:12 +0100, Julien Grall wrote:
> On 07/03/2014 11:37 AM, Thomas Leonard wrote:
> >> Using 4K alignment make impossible to use 1MB or 2MB mapping for early
> >> page table. Which make the code (usually in assembly) even harder to
> >> write or to impose relocation in the assembly code.
> > 
> > This isn't the case for Mini-OS, at least. We already use a 1MB mapping
> > with the 0x8000 offset just fine. The translation table (using 1MB
> > sections) is 16K, which would fit nicely in the 0x8000 gap (although
> > currently we don't use that space).
> 
> As I said in Mini-OS series, the spec doesn't say anything about the
> start offset of the guest kernel for the Linux boot protocol.
> 
> 0x8000 offset is very inconvenient for most of the other OS than Linux.
> As Linux is able to relocate itself, I don't see any known issue about
> starting at an 2MB-aligned address.

The known issue is that you cannot rely going forward on the address
always being 2MB aligned. You *cannot* assume this.

Ian.

Re: [PATCH] libxc: arm: Load the zImage to the rambase address + 2MB

[Thomas Leonard]

On 03/07/14 11:12, Julien Grall wrote:
> On 07/03/2014 11:01 AM, Ian Campbell wrote:
>> On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote:
>>> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000).
>>>
>>> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If
>>> the kernel doesn't relocate itself this would require to virtual address
>>> starting at 0xXXXX8000. This is not part of the zImage protocol, but a
>>> convenience for Linux after the decompressor stage. Linux is able to
>>> load at any address in the memory and it will relocate itself to respect
>>> it own constraint.
>>
>> Yes, because the boot protocol does not guarantee 2MB alignment, so if
>> the OS wants to rely on that then it is obliged to take care of
>> relocating itself.
>>
>>> Load the zImage at rambase address + 2MB to make life easier for other
>>> kernel (such as FreeBSD, Mini-OS).
>>
>> Whether or not it is easier these OSes *must* be prepared to be loaded
>> at any 4k aligned address. It sounds to me like you are hoping that
>> these OSes can *rely* on 2MB alignment, which is not the case. If they
>> make that assumption then they are buggy, regardless of whether it
>> happens to currently work with some loader or not.
>
> Using 4K alignment make impossible to use 1MB or 2MB mapping for early
> page table. Which make the code (usually in assembly) even harder to
> write or to impose relocation in the assembly code.

This isn't the case for Mini-OS, at least. We already use a 1MB mapping 
with the 0x8000 offset just fine. The translation table (using 1MB 
sections) is 16K, which would fit nicely in the 0x8000 gap (although 
currently we don't use that space).

Starting at 2MB would create an inconvenient 2MB of free memory just 
before the kernel, with no obvious way to tell xmalloc about it.

>>> FWIW, this is already the case for DOM0.
>>
>> On that basis I'm more inclined to change dom0 to use 32kb alignment.
>
> It's not really 32kb alignment but an address in 0xXXXX8000. If the
> kernel doesn't want to do relocation (because it's a pain to write in
> assembly code), it has to compile the kernel with a 0x8000 offset (see
> mini-os patch series).
>
> This solution, make the kernel more tight to Xen changes. I would prefer
> if we relax this change allowing easier support of new kernel to 2MB
> alignment.
>
> Regards,
>


-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

Re: [PATCH] libxc: arm: Load the zImage to the rambase address + 2MB

[Ian Campbell]

On Thu, 2014-07-03 at 11:39 +0100, Thomas Leonard wrote:
> On 03/07/14 11:12, Julien Grall wrote:
> > On 07/03/2014 11:01 AM, Ian Campbell wrote:
> >> On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote:
> >>> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000).
> >>>
> >>> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If
> >>> the kernel doesn't relocate itself this would require to virtual address
> >>> starting at 0xXXXX8000. This is not part of the zImage protocol, but a
> >>> convenience for Linux after the decompressor stage. Linux is able to
> >>> load at any address in the memory and it will relocate itself to respect
> >>> it own constraint.
> >>
> >> Yes, because the boot protocol does not guarantee 2MB alignment, so if
> >> the OS wants to rely on that then it is obliged to take care of
> >> relocating itself.
> >>
> >>> Load the zImage at rambase address + 2MB to make life easier for other
> >>> kernel (such as FreeBSD, Mini-OS).
> >>
> >> Whether or not it is easier these OSes *must* be prepared to be loaded
> >> at any 4k aligned address. It sounds to me like you are hoping that
> >> these OSes can *rely* on 2MB alignment, which is not the case. If they
> >> make that assumption then they are buggy, regardless of whether it
> >> happens to currently work with some loader or not.
> >
> > Using 4K alignment make impossible to use 1MB or 2MB mapping for early
> > page table. Which make the code (usually in assembly) even harder to
> > write or to impose relocation in the assembly code.
> 
> This isn't the case for Mini-OS, at least. We already use a 1MB mapping 
> with the 0x8000 offset just fine. The translation table (using 1MB 
> sections) is 16K, which would fit nicely in the 0x8000 gap (although 
> currently we don't use that space).
> 
> Starting at 2MB would create an inconvenient 2MB of free memory just 
> before the kernel, with no obvious way to tell xmalloc about it.

As far as I know you need to cope with arbitrary 4K alignment I'm
afraid, that's all which is guaranteed.

You can't assume that just because we use 0x8000 or 2MB offsets today it
will always be that way.

Ian.

Re: [PATCH] libxc: arm: Load the zImage to the rambase address + 2MB

[Julien Grall]

On 07/03/2014 12:18 PM, Ian Campbell wrote:
> On Thu, 2014-07-03 at 11:39 +0100, Thomas Leonard wrote:
>> On 03/07/14 11:12, Julien Grall wrote:
>>> On 07/03/2014 11:01 AM, Ian Campbell wrote:
>>>> On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote:
>>>>> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000).
>>>>>
>>>>> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If
>>>>> the kernel doesn't relocate itself this would require to virtual address
>>>>> starting at 0xXXXX8000. This is not part of the zImage protocol, but a
>>>>> convenience for Linux after the decompressor stage. Linux is able to
>>>>> load at any address in the memory and it will relocate itself to respect
>>>>> it own constraint.
>>>>
>>>> Yes, because the boot protocol does not guarantee 2MB alignment, so if
>>>> the OS wants to rely on that then it is obliged to take care of
>>>> relocating itself.
>>>>
>>>>> Load the zImage at rambase address + 2MB to make life easier for other
>>>>> kernel (such as FreeBSD, Mini-OS).
>>>>
>>>> Whether or not it is easier these OSes *must* be prepared to be loaded
>>>> at any 4k aligned address. It sounds to me like you are hoping that
>>>> these OSes can *rely* on 2MB alignment, which is not the case. If they
>>>> make that assumption then they are buggy, regardless of whether it
>>>> happens to currently work with some loader or not.
>>>
>>> Using 4K alignment make impossible to use 1MB or 2MB mapping for early
>>> page table. Which make the code (usually in assembly) even harder to
>>> write or to impose relocation in the assembly code.
>>
>> This isn't the case for Mini-OS, at least. We already use a 1MB mapping 
>> with the 0x8000 offset just fine. The translation table (using 1MB 
>> sections) is 16K, which would fit nicely in the 0x8000 gap (although 
>> currently we don't use that space).
>>
>> Starting at 2MB would create an inconvenient 2MB of free memory just 
>> before the kernel, with no obvious way to tell xmalloc about it.
> 
> As far as I know you need to cope with arbitrary 4K alignment I'm
> afraid, that's all which is guaranteed.

I think the zImage is also be able to cope on loading at 0xXXXXX200.
There is no 4k alignment requirements :).

> You can't assume that just because we use 0x8000 or 2MB offsets today it
> will always be that way.

That's all which is guaranteed if we are loading the kernel on a any
bootloader. Even though, on u-boot you can choose your loading address
(via u-boot image). That's why Xen is boot perfectly even if we have a
2MB-align address.

I don't find mad to define a standard for booting on Xen saying the
address will always be N MB align.

For Mini-OS it will help to have a simpler code which doesn't need
relocation.

Regards,

-- 
Julien Grall

Re: [PATCH] libxc: arm: Load the zImage to the rambase address + 2MB

[Ian Campbell]

On Thu, 2014-07-03 at 13:21 +0100, Julien Grall wrote:
> On 07/03/2014 12:18 PM, Ian Campbell wrote:
> > On Thu, 2014-07-03 at 11:39 +0100, Thomas Leonard wrote:
> >> On 03/07/14 11:12, Julien Grall wrote:
> >>> On 07/03/2014 11:01 AM, Ian Campbell wrote:
> >>>> On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote:
> >>>>> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000).
> >>>>>
> >>>>> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If
> >>>>> the kernel doesn't relocate itself this would require to virtual address
> >>>>> starting at 0xXXXX8000. This is not part of the zImage protocol, but a
> >>>>> convenience for Linux after the decompressor stage. Linux is able to
> >>>>> load at any address in the memory and it will relocate itself to respect
> >>>>> it own constraint.
> >>>>
> >>>> Yes, because the boot protocol does not guarantee 2MB alignment, so if
> >>>> the OS wants to rely on that then it is obliged to take care of
> >>>> relocating itself.
> >>>>
> >>>>> Load the zImage at rambase address + 2MB to make life easier for other
> >>>>> kernel (such as FreeBSD, Mini-OS).
> >>>>
> >>>> Whether or not it is easier these OSes *must* be prepared to be loaded
> >>>> at any 4k aligned address. It sounds to me like you are hoping that
> >>>> these OSes can *rely* on 2MB alignment, which is not the case. If they
> >>>> make that assumption then they are buggy, regardless of whether it
> >>>> happens to currently work with some loader or not.
> >>>
> >>> Using 4K alignment make impossible to use 1MB or 2MB mapping for early
> >>> page table. Which make the code (usually in assembly) even harder to
> >>> write or to impose relocation in the assembly code.
> >>
> >> This isn't the case for Mini-OS, at least. We already use a 1MB mapping 
> >> with the 0x8000 offset just fine. The translation table (using 1MB 
> >> sections) is 16K, which would fit nicely in the 0x8000 gap (although 
> >> currently we don't use that space).
> >>
> >> Starting at 2MB would create an inconvenient 2MB of free memory just 
> >> before the kernel, with no obvious way to tell xmalloc about it.
> > 
> > As far as I know you need to cope with arbitrary 4K alignment I'm
> > afraid, that's all which is guaranteed.
> 
> I think the zImage is also be able to cope on loading at 0xXXXXX200.
> There is no 4k alignment requirements :).

Then it's even more flexible than I thought. Fine (this is the problem
with de facto standards).

> > You can't assume that just because we use 0x8000 or 2MB offsets today it
> > will always be that way.
> 
> That's all which is guaranteed if we are loading the kernel on a any
> bootloader.

No, nothing is of that sort is guaranteed or required by anything.

>  Even though, on u-boot you can choose your loading address
> (via u-boot image). That's why Xen is boot perfectly even if we have a
> 2MB-align address.

This is a bug in Xen. We currently get away with it. But if a bootloader
comes along which doesn't load us at a 2MB boundary then that is *our*
bug, not the bootloader's.

> I don't find mad to define a standard for booting on Xen saying the
> address will always be N MB align.

We are using someonelse's standard, which does not say this. We are not
going to deviate from that (admittedly somewhat de facto) standard,
sorry.

You are of course free to go and have FreeBSD make whatever assumptions
about alignment which you like about load addresses. But those are not
guaranteed by the zImage protocol and when the behaviour of Xen changes
within that spec and breaks FreeBSD then that will be a FreeBSD bug, not
a Xen bug.

If you want to define your own standard then go ahead and pick your own
magic number and write down what the bootloader must guarantee and what
the kernel must be capable of dealing with when that protocol isused.
But the Linux zImage protocol is not ours to mess with.

Ian.

Re: [PATCH] libxc: arm: Load the zImage to the rambase address + 2MB

[Ian Campbell]

On Thu, 2014-07-03 at 11:12 +0100, Julien Grall wrote:
> On 07/03/2014 11:01 AM, Ian Campbell wrote:
> > On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote:
> >> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000).
> >>
> >> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If
> >> the kernel doesn't relocate itself this would require to virtual address
> >> starting at 0xXXXX8000. This is not part of the zImage protocol, but a
> >> convenience for Linux after the decompressor stage. Linux is able to
> >> load at any address in the memory and it will relocate itself to respect
> >> it own constraint.
> > 
> > Yes, because the boot protocol does not guarantee 2MB alignment, so if
> > the OS wants to rely on that then it is obliged to take care of
> > relocating itself.
> > 
> >> Load the zImage at rambase address + 2MB to make life easier for other
> >> kernel (such as FreeBSD, Mini-OS).
> > 
> > Whether or not it is easier these OSes *must* be prepared to be loaded
> > at any 4k aligned address. It sounds to me like you are hoping that
> > these OSes can *rely* on 2MB alignment, which is not the case. If they
> > make that assumption then they are buggy, regardless of whether it
> > happens to currently work with some loader or not.
> 
> Using 4K alignment make impossible to use 1MB or 2MB mapping for early
> page table. Which make the code (usually in assembly) even harder to
> write or to impose relocation in the assembly code.

yes, but that is what the protocol requires you to do, or to handle
realigning yourself early.

> >> FWIW, this is already the case for DOM0.
> > 
> > On that basis I'm more inclined to change dom0 to use 32kb alignment.
> 
> It's not really 32kb alignment but an address in 0xXXXX8000.

I meant 32kb alignment.

>  If the
> kernel doesn't want to do relocation (because it's a pain to write in
> assembly code), it has to compile the kernel with a 0x8000 offset (see
> mini-os patch series).

Yes, if a kernel doesn't want to do relocaiton then it must give an
address.

Conversely if a kernel does do relocation then it must do it properly,
not assuming 2MB alignment.

> This solution, make the kernel more tight to Xen changes. I would prefer
> if we relax this change allowing easier support of new kernel to 2MB
> alignment.

Easier isn't relevant here. The protocol requires you to deal with this
if you claim to be relocatable.
> 
> Regards,
>