[U-Boot] armv8 relocation questions

[Darwin Rambo <drambo@broadcom.com>]

On 14-05-16 06:47 AM, fenghua at phytium.com.cn wrote:
>
> hi Darwin,
>        It's a little late.
>>
>> I'm hoping someone can help answer these questions about armv8 
>> relocation.
>>
>> The CONFIG_SYS_TEXT_BASE seems to be be usually setup to a decent 
>> amount of alignment. For the purposes of this discussion, let's say 
>> it would normally be 0x88000000 and all is well. The relocation 
>> address moves to near the end of memory, to say, 0xfffa8000. So far 
>> so good.
>>
>> Now let's say I want to shift the image a bit so that I can add a 
>> small 32-byte header required by a previous bootloader. So I set 
>> CONFIG_SYS_TEXT_BASE to 0x88000020, and the relocated address is 
>> still 0xfffa8000 and the relocated vectors should be at 0xfffa9000. 
>> The image crashes so after some debugging, I find that the code 
>> appears to be relocated fine, but some sections have symbols that are 
>> not relocated properly. The vectors try to relocate to 0xfffa8fe0 and 
>> rodata.str1.1 printf format strings are also 0x20 off. There are 
>> likely other offset sections with issues as well.
>>
>> The relocation offset is 0x77fa7fe0 due to the calculations in 
>> arch/arm/lib/board.c. Simplifying, they look like this:
>>
>> addr = CONFIG_SYS_SDRAM_BASE + gd->ram_size;
>>
>>         /* round down to next 4 kB limit */
>>         addr &= ~(4096 - 1);
>>         debug("Top of RAM usable for U-Boot at: %08lx\n", addr);
>>
>>         /*
>>          * reserve memory for U-Boot code, data & bss
>>          * round down to next 4 kB limit
>>          */
>>         addr -= gd->mon_len;
>>         addr &= ~(4096 - 1);
>>
>>         addr += 0x20; // hack to adjust relocaddr to aligned address...
>>
>> <snip>
>>
>> gd->relocaddr = addr;
>>         gd->start_addr_sp = addr_sp;
>>         gd->reloc_off = addr - _TEXT_BASE;
>>         debug("relocation Offset is: %08lx\n", gd->reloc_off);
>>
>>
>> Since _TEXT_BASE is 0x88000020 and addr is 0xfffa8000, the reloc_off 
>> is a number like 0x77fa7fe0.
>>
>> Now if I add 0x20 to 'addr' above just before the <snip>, relocaddr 
>> becomes 0x77fa8000 and the relocation works perfectly and no more 
>> crashes happen.
>>
>> So my question - is the CONFIG_SYS_TEXT_BASE alignment requirement 
>> related to to any assumptions in the linker itself about image base 
>> alignment, specifically referring to creation of the rela.dyn 
>> sections and their use for image relocation?
>>
>> A related question is if CONFIG_SYS_TEXT_BASE needs to be at a 
>> specific alignment. The maximum alignment in the armv8 code base is 
>> ".align 11" which I believe means 0x800 or 2048.
>>
>> Note that an armv7 target appears to relocate properly with smaller 
>> offsets such as 0x20.
>>
>> Thanks.
>>
>>
>        I traced the problem you described and found it is caused by 
> 'adrp' instruction. 'adrp' instruction produces 4kb aligned address of 
> a label. So, if CONFIG_SYS_TEXT_BASE is not 4kb aligned the address 
> produced by 'adrp' and following 'add' instruction will be incorrect.
>       For example, if CONFIG_SYS_TEXT_BASE = 0x20 then address of 
> '_start' is 0x20 and address of '_end_ofs' is 0x30, where u-boot 
> access variable '_end_ofs' gcc generate code as following:
>      adrp  x0,  ...
>      add   x0, x0, 0x30
>      We noticed that 0x30 is added to 'x0' to produce the address 
> of '_end_ofs'. So when CONFIG_SYS_TEXT_BASE=0x20 and relocated 
> destination address is not 0x****020 register x0 contain incorrect 
> address of '_end_ofs'.
Thank you David. I agree that the adrp will cause problems if the string 
sections and label used are not relocated to correct boundaries. The 
adrp used before the relocation works because the symbols are on aligned 
boundaries.

I think I have a way to explain this problem better now.

1. Working generic code:
CONFIG_SYS_TEXT_BASE = 0x88000000
unrelocated vectors = 0x888001000
relocation address = 0xfffa8000 = 0xfffa8000 - 0x88000000
relocation offset = 77fa8000
relocated vectors = 0xfffa9000  (0x800 alignment, but pushed to fffa9000 
because of code after 0xfffa8800)

Now in this case, the .align directive for the vectors section wants the 
vectors sitting at 0x800 alignment, which they are.
When the symbols are relocated, the vectors are now at 0xfffa9000 which 
is aligned properly.

2. Failing offset case:
CONFIG_SYS_TEXT_BASE = 0x88000020
unrelocated vectors = 0x888001000  (respecting .align 11 directive)
relocation address = 0xfffa8000
relocation offset = 77fa7fe0 = 0xfffa8000 - 0x88000020
relocated vectors = 0xfffa8fe0 (BAD ALIGNMENT)
Now the relocated rodata.str1.1 string tables are not aligned, which I 
believe breaks
the adrp instruction. The strings are offset by 0x20 and the printf fails.

3. Fixed offset case:
CONFIG_SYS_TEXT_BASE = 0x88000020
unrelocated vectors = 0x888001000  (respecting .align 11 directive)
relocation address = 0xfffa8020
relocation offset = 77fa8000 = 0xfffa8020 - 0x88000020
relocated vectors = 0xfffa9000 (GOOD ALIGNMENT, respects .align 11 after 
relocation)

So in this fixed offset case, the adrp label is aligned to a 4K page size.


General comments:

Any section in the image that requires a particular alignment must have 
that alignment respected after relocation. You cannot relocate to an 
arbitrary address if it breaks the maximum image alignment requirement 
after relocation.

If the vectors want an 0x800 alignment, and the relocation puts them on 
a 0x1000 boundary, then that will work fine.
If the string section wants a 0x1000 alignment and are relocated to a 
0x1000 boundary, that also works.

But if for some reason, the hardware ever required a 0x2000 (.align 13) 
alignment, then the generic code's 0x1000 (.align 12) relocation 
alignment would not work because the alignment after relocation would 
not respect the .align 13 directive. We just haven't run into this issue 
yet and may never do so, but it is important to understand the 
limitations of relocation relative to image alignment requirements. The 
current hardcoded 4096 (0x1000) image relocation alignment just happens 
to work and looks nice, that's all, but not by consideration of image 
alignment.

And if any text base alignment is less than the image's maximum 
alignment requirement, the load will fail, and then we likely scratch 
our heads and set the CONFIG_SYS_TEXT_BASE alignment higher until it works.

But since most people just use higher alignments naturally, this issues 
remain mostly hidden I think.

Best regards,
Darwin

>
> David.
>
>