[Xenomai] ARM/BeagleBone Thumb2 Kernels

[Xenomai] ARM/BeagleBone Thumb2 Kernels

[Charles Steinkuehler]

So when I try to build a Xenomai patched kernel for the BeagleBone using
Thumb2 instructions, I get the following linker errors:

>   LD      init/built-in.o
> arch/arm/kernel/built-in.o: In function `__irq_usr':
> :(.kprobes.text+0x3c2): relocation truncated to fit: R_ARM_THM_JUMP19
> against symbol `__ipipe_ret_to_user_irqs_disabled' defined in .text
> section in arch/arm/kernel/built-in.o
> arch/arm/kernel/built-in.o: In function `ret_from_exception':
> :(.kprobes.text+0x600): relocation truncated to fit: R_ARM_THM_JUMP19
> against symbol `__ipipe_ret_to_user_irqs_disabled' defined in .text
> section in arch/arm/kernel/built-in.o
> make: *** [vmlinux] Error 1
> 
> The link error goes away if I disable CONFIG_THUMB2_KERNEL, which wasn't
> set in the original kernel config.

This has recently become more of a problem, as the default BeagleBone
kernel has switched to enabling CONFIG_THUMB2_KERNEL.

A bit of Googling turned up the following patch to fix a similar problem
on the Tegra (not Xenomai related):

  http://patchwork.ozlabs.org/patch/236760/

I applied a similar if-then patch (see below) to the two Xenomai calls
that generate the linker errors, and the result is a working kernel.

Can someone more familiar with iPipeand ARM assembly review the attached
minor patch and advise on whether or not it seems OK?

As I understand things, the two bne calls in the default ipipe patch
don't work when building a kernel using the thumb instructions due to
the limited range of the conditional branch.  Using the if-then
construct when THUMB2 is enabled allows for a larger addressing range in
the branch, thus avoiding the linker errors.  If this seems OK, perhaps
it should be applied to the master ipipe patch set?  If not, is there a
better way to avoid the linker errors when building with the thumb
instructions?

-- 
Charles Steinkuehler
charles@steinkuehler.net
-------------- next part --------------
diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S
index e2bc263..6f4d9f0 100644
--- a/arch/arm/kernel/entry-armv.S
+++ b/arch/arm/kernel/entry-armv.S
@@ -469,6 +469,7 @@ __irq_usr:
 	kuser_cmpxchg_check
 	irq_handler
 #ifdef CONFIG_IPIPE
+THUMB(	it	ne )
 	bne	__ipipe_ret_to_user_irqs_disabled
 #endif	/* CONFIG_IPIPE */
 	get_thread_info tsk
@@ -767,6 +768,7 @@ ENTRY(ret_from_exception)
 	disable_irq
 	bl	__ipipe_check_root
 	cmp     r0, #1
+THUMB(	it	ne )
 	bne	__ipipe_ret_to_user_irqs_disabled  @ Fast exit path over non-root domains
 #endif /* CONFIG_IPIPE */
 	get_thread_info tsk
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://www.xenomai.org/pipermail/xenomai/attachments/20140123/8019559b/attachment.sig>

Re: [Xenomai] ARM/BeagleBone Thumb2 Kernels

[Gilles Chanteperdrix]

On 01/24/2014 03:56 AM, Charles Steinkuehler wrote:
> So when I try to build a Xenomai patched kernel for the BeagleBone using
> Thumb2 instructions, I get the following linker errors:
> 
>>   LD      init/built-in.o
>> arch/arm/kernel/built-in.o: In function `__irq_usr':
>> :(.kprobes.text+0x3c2): relocation truncated to fit: R_ARM_THM_JUMP19
>> against symbol `__ipipe_ret_to_user_irqs_disabled' defined in .text
>> section in arch/arm/kernel/built-in.o
>> arch/arm/kernel/built-in.o: In function `ret_from_exception':
>> :(.kprobes.text+0x600): relocation truncated to fit: R_ARM_THM_JUMP19
>> against symbol `__ipipe_ret_to_user_irqs_disabled' defined in .text
>> section in arch/arm/kernel/built-in.o
>> make: *** [vmlinux] Error 1
>>
>> The link error goes away if I disable CONFIG_THUMB2_KERNEL, which wasn't
>> set in the original kernel config.
> 
> This has recently become more of a problem, as the default BeagleBone
> kernel has switched to enabling CONFIG_THUMB2_KERNEL.
> 
> A bit of Googling turned up the following patch to fix a similar problem
> on the Tegra (not Xenomai related):
> 
>   http://patchwork.ozlabs.org/patch/236760/
> 
> I applied a similar if-then patch (see below) to the two Xenomai calls
> that generate the linker errors, and the result is a working kernel.
> 
> Can someone more familiar with iPipeand ARM assembly review the attached
> minor patch and advise on whether or not it seems OK?
> 
> As I understand things, the two bne calls in the default ipipe patch
> don't work when building a kernel using the thumb instructions due to
> the limited range of the conditional branch.  Using the if-then
> construct when THUMB2 is enabled allows for a larger addressing range in
> the branch, thus avoiding the linker errors.  If this seems OK, perhaps
> it should be applied to the master ipipe patch set?  If not, is there a
> better way to avoid the linker errors when building with the thumb
> instructions?

This looks fine, but I have one objection: why does it work for me with
the toolchain I use (linaro 2012.09)?

-- 
                                                                Gilles.

Re: [Xenomai] ARM/BeagleBone Thumb2 Kernels

[Gilles Chanteperdrix]

On 01/24/2014 03:56 AM, Charles Steinkuehler wrote:
> So when I try to build a Xenomai patched kernel for the BeagleBone using
> Thumb2 instructions, I get the following linker errors:
> 
>>   LD      init/built-in.o
>> arch/arm/kernel/built-in.o: In function `__irq_usr':
>> :(.kprobes.text+0x3c2): relocation truncated to fit: R_ARM_THM_JUMP19
>> against symbol `__ipipe_ret_to_user_irqs_disabled' defined in .text
>> section in arch/arm/kernel/built-in.o
>> arch/arm/kernel/built-in.o: In function `ret_from_exception':
>> :(.kprobes.text+0x600): relocation truncated to fit: R_ARM_THM_JUMP19
>> against symbol `__ipipe_ret_to_user_irqs_disabled' defined in .text
>> section in arch/arm/kernel/built-in.o
>> make: *** [vmlinux] Error 1
>>
>> The link error goes away if I disable CONFIG_THUMB2_KERNEL, which wasn't
>> set in the original kernel config.
> 
> This has recently become more of a problem, as the default BeagleBone
> kernel has switched to enabling CONFIG_THUMB2_KERNEL.
> 
> A bit of Googling turned up the following patch to fix a similar problem
> on the Tegra (not Xenomai related):
> 
>   http://patchwork.ozlabs.org/patch/236760/
> 
> I applied a similar if-then patch (see below) to the two Xenomai calls
> that generate the linker errors, and the result is a working kernel.
> 
> Can someone more familiar with iPipeand ARM assembly review the attached
> minor patch and advise on whether or not it seems OK?
> 
> As I understand things, the two bne calls in the default ipipe patch
> don't work when building a kernel using the thumb instructions due to
> the limited range of the conditional branch.  Using the if-then
> construct when THUMB2 is enabled allows for a larger addressing range in
> the branch, thus avoiding the linker errors.  If this seems OK, perhaps
> it should be applied to the master ipipe patch set?  If not, is there a
> better way to avoid the linker errors when building with the thumb
> instructions?
> 

Could you provide me with the details (kernel version or git url, config
file used, toolchain used) allowing to reproduce this issue?

I have been compiling thumb2 kernels for omap3 and omap4 for a long
time, and never observed this issue. So, there must be something
different in your case.

-- 
                                                                Gilles.

Re: [Xenomai] ARM/BeagleBone Thumb2 Kernels

[Charles Steinkuehler]

On 1/24/2014 5:22 AM, Gilles Chanteperdrix wrote:
> 
> Could you provide me with the details (kernel version or git url, config
> file used, toolchain used) allowing to reproduce this issue?

Tool chain: gcc-linaro-arm-linux-gnueabihf-4.7-2013.04-20130415_linux

I'm building kernels for the BeagleBone, maintained as a set of patches
and scripts that apply to a clean kernel source tree.  The scripts can
be found here:

  https://github.com/cdsteinkuehler/linux-dev

Checkout the 3.8.13-bone37-xenomai branch.  Copy system.sh.sample to
system.sh and edit as needed for your system.  If you have a local copy
of linaro and the Linus kernel source, you can set CC= and LINUX_GIT= to
save a bunch of download time and drive space, otherwise the scripts
will install both for you.

If you set AUTO_BUILD=1 in the script, it will build a kernel
unattended.  If you don't set this, you'll be taken to menuconfig with
the default configuration loaded.

> I have been compiling thumb2 kernels for omap3 and omap4 for a long
> time, and never observed this issue. So, there must be something
> different in your case.

As I understand it, the issue arises as the kernel grows larger and the
distance between the branch and it's target increases.  My vmlinux for
the (fixed Thumb2 version) BeagleBone kernel is currently weighing in at
13225004 bytes.

-- 
Charles Steinkuehler
charles@steinkuehler.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://www.xenomai.org/pipermail/xenomai/attachments/20140124/b5ded526/attachment.sig>

Re: [Xenomai] ARM/BeagleBone Thumb2 Kernels

[Gilles Chanteperdrix]

On 01/24/2014 12:38 PM, Charles Steinkuehler wrote:
> As I understand it, the issue arises as the kernel grows larger and the
> distance between the branch and it's target increases.  My vmlinux for
> the (fixed Thumb2 version) BeagleBone kernel is currently weighing in at
> 13225004 bytes.
>
That is precisely the thing I do not understand: the jump goes from 
entry-armv.S to entry-common.S which should remain close since they 
follow each other on the link-edit command line. Since the source files 
are assembly, the distance should more or less be the same, whatever the 
compilation options. And if they are not close, this is where the bug 
is, because this assembly is used for every syscall and every interrupt.

-- 
					    Gilles.

Re: [Xenomai] ARM/BeagleBone Thumb2 Kernels

[Charles Steinkuehler]

On 1/24/2014 6:18 AM, Gilles Chanteperdrix wrote:
> On 01/24/2014 12:38 PM, Charles Steinkuehler wrote:
>> As I understand it, the issue arises as the kernel grows larger and the
>> distance between the branch and it's target increases.  My vmlinux for
>> the (fixed Thumb2 version) BeagleBone kernel is currently weighing in at
>> 13225004 bytes.
>>
> That is precisely the thing I do not understand: the jump goes from
> entry-armv.S to entry-common.S which should remain close since they
> follow each other on the link-edit command line. Since the source files
> are assembly, the distance should more or less be the same, whatever the
> compilation options. And if they are not close, this is where the bug
> is, because this assembly is used for every syscall and every interrupt.

Well I don't pretend to know what goes on in the kernel memory map.  Is
there anything I can send from my kernel builds that would shed light on
this (a linker map or something)?

-- 
Charles Steinkuehler
charles@steinkuehler.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://www.xenomai.org/pipermail/xenomai/attachments/20140124/c58ddf2b/attachment.sig>

Re: [Xenomai] ARM/BeagleBone Thumb2 Kernels

[Gilles Chanteperdrix]

On 01/24/2014 12:53 PM, Charles Steinkuehler wrote:
> On 1/24/2014 6:18 AM, Gilles Chanteperdrix wrote:
>> On 01/24/2014 12:38 PM, Charles Steinkuehler wrote:
>>> As I understand it, the issue arises as the kernel grows larger and the
>>> distance between the branch and it's target increases.  My vmlinux for
>>> the (fixed Thumb2 version) BeagleBone kernel is currently weighing in at
>>> 13225004 bytes.
>>>
>> That is precisely the thing I do not understand: the jump goes from
>> entry-armv.S to entry-common.S which should remain close since they
>> follow each other on the link-edit command line. Since the source files
>> are assembly, the distance should more or less be the same, whatever the
>> compilation options. And if they are not close, this is where the bug
>> is, because this assembly is used for every syscall and every interrupt.
>
> Well I don't pretend to know what goes on in the kernel memory map.  Is
> there anything I can send from my kernel builds that would shed light on
> this (a linker map or something)?
>

I have no doubt that in the kernel you build, entry-arm.S and 
entry-common.S end up far from each other, which a linker map would 
show. What I am interested in is why this happens, for that, I need to 
build the kernel the same way you build it, and see what happens. You 
have given me all I need.

-- 
					    Gilles.

Re: [Xenomai] ARM/BeagleBone Thumb2 Kernels

[Gilles Chanteperdrix]

On 01/24/2014 03:56 AM, Charles Steinkuehler wrote:
> So when I try to build a Xenomai patched kernel for the BeagleBone using
> Thumb2 instructions, I get the following linker errors:
> 
>>   LD      init/built-in.o
>> arch/arm/kernel/built-in.o: In function `__irq_usr':
>> :(.kprobes.text+0x3c2): relocation truncated to fit: R_ARM_THM_JUMP19
>> against symbol `__ipipe_ret_to_user_irqs_disabled' defined in .text
>> section in arch/arm/kernel/built-in.o
>> arch/arm/kernel/built-in.o: In function `ret_from_exception':
>> :(.kprobes.text+0x600): relocation truncated to fit: R_ARM_THM_JUMP19
>> against symbol `__ipipe_ret_to_user_irqs_disabled' defined in .text
>> section in arch/arm/kernel/built-in.o
>> make: *** [vmlinux] Error 1
>>
>> The link error goes away if I disable CONFIG_THUMB2_KERNEL, which wasn't
>> set in the original kernel config.
> 
> This has recently become more of a problem, as the default BeagleBone
> kernel has switched to enabling CONFIG_THUMB2_KERNEL.
> 
> A bit of Googling turned up the following patch to fix a similar problem
> on the Tegra (not Xenomai related):
> 
>   http://patchwork.ozlabs.org/patch/236760/
> 
> I applied a similar if-then patch (see below) to the two Xenomai calls
> that generate the linker errors, and the result is a working kernel.

Applied, thanks, I could reproduce the issue, and thinking about it
having entry-common.o close to entry-arm.o should not bring any
performance improvement in fact.

Regards.

-- 
                                                                Gilles.

Re: [Xenomai] ARM/BeagleBone Thumb2 Kernels

[Charles Steinkuehler]

On 1/25/2014 12:14 PM, Gilles Chanteperdrix wrote:
> On 01/24/2014 03:56 AM, Charles Steinkuehler wrote:
>>
>> I applied a similar if-then patch (see below) to the two Xenomai calls
>> that generate the linker errors, and the result is a working kernel.
> 
> Applied, thanks, I could reproduce the issue, and thinking about it
> having entry-common.o close to entry-arm.o should not bring any
> performance improvement in fact.

Thanks for looking into this!

I've been running the Thumb2 kernel a while with no apparent problems,
and it does seem to have improved the latency numbers even with all the
other suggested tweaks.  Thanks again for all the advice!

-- 
Charles Steinkuehler
charles@steinkuehler.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://www.xenomai.org/pipermail/xenomai/attachments/20140125/a404d09d/attachment.sig>