Re: Building Kernel with -O0

Re: Building Kernel with -O0

[Robert Hancock]

Keith A. Prickett wrote:
> I was looking at some changes to the 2.6.26.y kernel in "compiler-gcc.h"
> and had some questions.
> ( http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.26.y.git;a=commitdiff;h=ae3a0064e6d69068b1c9fd075095da062430bda9 )
> 
> I was attempting to cross-compile Linux for a Marvell core with
> Optimizations turned off and ran across an error due to the
> __attribute__((always_inline)) attribute.
> 
> It looks like the default behavior used to always have this feature on,
> but Ingo committed some changes that allows this behavior to be turned
> off (With the correct Kconfig file changes).
> 
> I'm VERY new to Linux but was wondering if there was a way (config
> option) to turn this inline attribute off in previous versions of the
> kernel besides these new macros.  It seems building with optimization
> level 0 should be a natural thing for kernel driver developers to do for
> debugging purposes.
> 
> In summary: I want to compile with optimizations off and the compile is
> failing now when I try this.  How can this be resolved?

I don't think this has ever been supported. Even in current kernels 
where the compiler is not always forced to inline all functions, I 
believe there are some functions that need to be inlined and still use 
forced inlining, and if no optimizations are enabled, no inlining can be 
done.

Re: Building Kernel with -O0

[Bodo Eggert]

linux-os (Dick Johnson) <linux-os@analogic.com> wrote:

> If you have to single-step through kernel procedures, you
> are not writing proper code for a kernel. Furthermore, even
> when using the kernel debugger, you are not executing the
> exact same path that would be executed without the debugger
> attached.

I sometimes single-step in order to verify my (userspace) code, instead of
just throwing random test data at it. It's easier and makes me feel better.
OTOH, -O2 never was a problem if I used it, so I don't strongly vote
for enabling -O0.

Re: Building Kernel with -O0

[emin ak]

2008/9/10 Bodo Eggert <7eggert@gmx.de>:
> linux-os (Dick Johnson) <linux-os@analogic.com> wrote:
>
>> If you have to single-step through kernel procedures, you
>> are not writing proper code for a kernel. Furthermore, even
>> when using the kernel debugger, you are not executing the
>> exact same path that would be executed without the debugger
>> attached.
>
> I sometimes single-step in order to verify my (userspace) code, instead of
> just throwing random test data at it. It's easier and makes me feel better.
> OTOH, -O2 never was a problem if I used it, so I don't strongly vote
> for enabling -O0.
>
> --

I had compiled linux-2.6.23 with -O0 for kgdb only with little
modifications, it's  running with no error for a long time. I'am not a
linux expert, I think talented kernel hackers can make kernel to
compile with -O0 easly but for some reason they dont want to do it.
Debug is much understandable for me (and for most people I think) with
-O0. Linux kernel programming ability is not a gift, of course you can
learn the code with printks and reading, but which age are we living
in?

Emin

Re: Building Kernel with -O0

[linux-os (Dick Johnson)]

On Fri, 12 Sep 2008, emin ak wrote:

> 2008/9/10 Bodo Eggert <7eggert@gmx.de>:
>> linux-os (Dick Johnson) <linux-os@analogic.com> wrote:
>>
>>> If you have to single-step through kernel procedures, you
>>> are not writing proper code for a kernel. Furthermore, even
>>> when using the kernel debugger, you are not executing the
>>> exact same path that would be executed without the debugger
>>> attached.
>>
>> I sometimes single-step in order to verify my (userspace) code, instead of
>> just throwing random test data at it. It's easier and makes me feel better.
>> OTOH, -O2 never was a problem if I used it, so I don't strongly vote
>> for enabling -O0.
>>
>> --
>
> I had compiled linux-2.6.23 with -O0 for kgdb only with little
> modifications, it's  running with no error for a long time. I'am not a
> linux expert, I think talented kernel hackers can make kernel to
> compile with -O0 easly but for some reason they dont want to do it.
> Debug is much understandable for me (and for most people I think) with
> -O0. Linux kernel programming ability is not a gift, of course you can
> learn the code with printks and reading, but which age are we living
> in?
>
> Emin
>

Well, to use the ** kernel ** to single-
step through the ** kernel ** is
really not going to be much more than an
exercise. It's not going to find the bugs
that are likely to exist in kernel code.


Cheers,
Dick Johnson
Penguin : Linux version 2.6.25.17 on an i686 machine (4786.38 BogoMips).
My book : http://www.AbominableFirebug.com/
_


****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Building Kernel with -O0

[linux-os (Dick Johnson)]

On Fri, 12 Sep 2008, linux-os (Dick Johnson) wrote:

>
> On Fri, 12 Sep 2008, emin ak wrote:
>
>> 2008/9/10 Bodo Eggert <7eggert@gmx.de>:
>>> linux-os (Dick Johnson) <linux-os@analogic.com> wrote:
>>>
>>>> If you have to single-step through kernel procedures, you
>>>> are not writing proper code for a kernel. Furthermore, even
>>>> when using the kernel debugger, you are not executing the
>>>> exact same path that would be executed without the debugger
>>>> attached.
>>>
>>> I sometimes single-step in order to verify my (userspace) code, instead of
>>> just throwing random test data at it. It's easier and makes me feel better.
>>> OTOH, -O2 never was a problem if I used it, so I don't strongly vote
>>> for enabling -O0.
>>>

Let me steal a bit of bandwidth and explain what
I mean about writing kernel code.

What is often lost to new people who want to develop
kernel code is that you can write and test many
pieces of kernel code within a "friendly" user-mode
environment.

Suppose you need to write a driver for a complex
I/O subsystem that is accessed over the PCI bus.

This seems difficult until you understand that the
starting piece of code will be something to find
your device and access its registers. This can
usually be (mostly) copied from another existing
driver.

Once its memory-mapped registers have been
uncovered, you can implement mmap() in your
driver and then access them from a user-space
program you write, reading/writing setting
bits and observing the hardware behavior
well before you write actual driver code.
Parts of the code that you develop in user-
space will eventually be cut and pasted
into your kernel driver code.

You are not going to be able to write your
interrupt service routines in user-mode but
they can still be in your driver code in which
you implemented mmap(). They just do a printk()
and attempt to reset your hardware for the next
interrupt. If it doesn't work, you can experiment
in user-mode, writing to registers, until you
have satisfied the hardware so the next interrupt
will properly occur.

If you write drivers in this manner, you will not
have code that doesn't work in the kernel code. It
may not work exactly as you want, but it will work
well enough so that a few printks will allow you
to get its real-time operation working as expected.

Warning! Do NOT leave hardware registers exposed to
user-mode access like Windows does. Your initial memory-
mapping was to help develop kernel code, not to create
a user-mode driver! Such code would be called a "layering
violation..."


Cheers,
Dick Johnson
Penguin : Linux version 2.6.25.17 on an i686 machine (4786.38 BogoMips).
My book : http://www.AbominableFirebug.com/
_


****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Building Kernel with -O0

[Keith A. Prickett]

I was looking at some changes to the 2.6.26.y kernel in "compiler-gcc.h"
and had some questions.
( http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.26.y.git;a=commitdiff;h=ae3a0064e6d69068b1c9fd075095da062430bda9 )

I was attempting to cross-compile Linux for a Marvell core with
Optimizations turned off and ran across an error due to the
__attribute__((always_inline)) attribute.

It looks like the default behavior used to always have this feature on,
but Ingo committed some changes that allows this behavior to be turned
off (With the correct Kconfig file changes).

I'm VERY new to Linux but was wondering if there was a way (config
option) to turn this inline attribute off in previous versions of the
kernel besides these new macros.  It seems building with optimization
level 0 should be a natural thing for kernel driver developers to do for
debugging purposes.

In summary: I want to compile with optimizations off and the compile is
failing now when I try this.  How can this be resolved?


For reference:
- Stable Kernel 2.6.26
- I'm using arm-marvell-eabi-gcc 4.1.1
- Compiling for ARM architecture
- Neither CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING nor
CONFIG_OPTIMIZE_INLINING are on (or even options in the ARM
architecture)
- The error I receive is:
 "sorry, unimplemented: inlining failed in call to '<inline function>':
function body not available"


Thanks in advance,
--
Keith Prickett

Re: Building Kernel with -O0

[Adrian Bunk]

On Thu, Sep 04, 2008 at 10:50:53AM -0700, Keith A. Prickett wrote:
>...
> It seems building with optimization
> level 0 should be a natural thing for kernel driver developers to do for
> debugging purposes.
> 
> In summary: I want to compile with optimizations off and the compile is
> failing now when I try this.  How can this be resolved?
>...

There are several places in the kernel where we e.g. rely on gcc 
removing dead code, and otherwise the linking of the kernel fails.

Compiling with -O0 never worked, and is not likely to ever work.

> Thanks in advance,
> Keith Prickett

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Building Kernel with -O0

[Keith A. Prickett]

On Tue, 2008-09-09 at 19:04 +0300, Adrian Bunk wrote:
> On Thu, Sep 04, 2008 at 10:50:53AM -0700, Keith A. Prickett wrote:
> >...
> > It seems building with optimization
> > level 0 should be a natural thing for kernel driver developers to do for
> > debugging purposes.
> > 
> > In summary: I want to compile with optimizations off and the compile is
> > failing now when I try this.  How can this be resolved?
> >...
> 
> There are several places in the kernel where we e.g. rely on gcc 
> removing dead code, and otherwise the linking of the kernel fails.
> 
> Compiling with -O0 never worked, and is not likely to ever work.
I am currently running the kernel compiled with -O0.  The final problem
I ran into was some NULL pointer dereference and compiling ONLY files in
the mm directory with -O2 fixed the issue.

To build with -O0 I had to change a few things (for my ARM
architecture):
1. Ensure the __attribute__((always_inline)) was turned off in the
"inline" #define (compiler.h)

2. Fix some inline assembly in kprobes (ARM only) that, due to no
optimization, needed more registers than the compiler had available.
(Similar to this issue:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=13850)

3. Implement a fix as described here:
(http://lists.arm.linux.org.uk/lurker/message/20050201.210458.5cc93c10.en.html)

4. Change -O2 to -O0 in the root Makefile

5. Add extra cflags in the mm Makefile so it builds with -O2 only in
that directory.

All of this could, potentially be changed by a configuration parameter
and added into the kernel release.  Does someone have a primer on why
this kind of behavior is not desired or "possible"?

Doing this has given me SO much visibility into stack variables data
values while single-stepping the code.  I feel that the time invested to
make this work is well worth the return.  In the book "Writing Solid
Code" (http://www.amazon.com/Writing-Solid-Code-Microsofts-Programming/dp/1556155514) one technique discussed is: stepping through your code.  In order to be able to do this effectively you need optimizations turned off.  

Re: Building Kernel with -O0

[linux-os (Dick Johnson)]

On Tue, 9 Sep 2008, Keith A. Prickett wrote:

> On Tue, 2008-09-09 at 19:04 +0300, Adrian Bunk wrote:
>> On Thu, Sep 04, 2008 at 10:50:53AM -0700, Keith A. Prickett wrote:

Microsoft and "Writing Solid Code" are mutually exclusive.

> make this work is well worth the return.  In the book "Writing Solid
> Code" (http://www.amazon.com/Writing-Solid-Code-Microsofts-Programming/dp/1556155514) one technique discussed is: stepping through your code.  In order to be able to do this effectively you need optimizations turned off.

Sorry, I couldn't resist after being required to write code in
VC++/MFC, where memory leaks are "normal," and pointers passed
to "methods" don't remain valid if there is any other activity
besides yours occurring in the machine.

If you have to single-step through kernel procedures, you
are not writing proper code for a kernel. Furthermore, even
when using the kernel debugger, you are not executing the
exact same path that would be executed without the debugger
attached.

These are some of the reasons why any attempt to compile
with no optimization will usually fall upon deaf ears. There
is a lot of in-line assembly that can't work with -O0 and if
you change the code so that it does, it will not be the same
as "normal" code so whatever it is that you are trying to
accomplish will probably fail to work as planned.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.25.17 on an i686 machine (4786.16 BogoMips).
My book : http://www.AbominableFirebug.com/
_


****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Building Kernel with -O0

[Andi Kleen]

"Keith A. Prickett" <keithp@marvell.com> writes:
>
> All of this could, potentially be changed by a configuration parameter
> and added into the kernel release.  Does someone have a primer on why
> this kind of behavior is not desired or "possible"?

Traditionally it was not allowed because -O0 didn't inline and
the kernel relied on inlining in some cases. But with always_inline
that is obsolete -- all functions that rely on inlining should be 
marked __always_inline.

I think a few more cases crept in where it was common to write
build time asserts as

      if (some condition the compiler evaluates at runtime)
         __error_condition_xyz_is_false(); 
 
and this obviously relies on the optimizer to build. But these
are all slowly moving over the BUILD_BUG_ON() which also doesn't
rely on the optimizer, so it's also obsolete.

Then there's the issue that some of the kernel macros will
generate absolutely terrible code without optimizer
(good examples are the macros in asm-x86/uaccess.h). But I guess
that could be tolerated too.

Then another reason was that traditionally no source level
debugger was included and if you do assembly level debugging
optimized code is typically easier to read and debug because
the unoptimized gcc code is so terrible.

And with kgdb now being a standard option that has also changed.
And I agree with you that sometimes stepping through code
is very educational. And gdb tends to be a lot happier 
with -O0 code indeed.

So if you can make it all work cleanly, ideally tested on
other architectures too and a bit broader (e.g. with allyesconfig) 
and track down whatever the mm/* issue 
is correctly I don't see any real reason to not submit
a mainline patch that enables this as a CONFIG. Of course
it has to be very clean and not contain hacks.

If you're not familiar with submitting kernel patches
please read Documentation/{SubmittingPatches,SubmitChecklist} 
first.

Hope this helps,

-Andi

-- 
ak@linux.intel.com

Re: Building Kernel with -O0

[Adrian Bunk]

On Tue, Sep 09, 2008 at 09:24:58PM +0200, Andi Kleen wrote:
> "Keith A. Prickett" <keithp@marvell.com> writes:
> >
> > All of this could, potentially be changed by a configuration parameter
> > and added into the kernel release.  Does someone have a primer on why
> > this kind of behavior is not desired or "possible"?
> 
> Traditionally it was not allowed because -O0 didn't inline and
> the kernel relied on inlining in some cases. But with always_inline
> that is obsolete -- all functions that rely on inlining should be 
> marked __always_inline.
> 
> I think a few more cases crept in where it was common to write
> build time asserts as
> 
>       if (some condition the compiler evaluates at runtime)
                                                     ^^^^^^^
buildtime

>          __error_condition_xyz_is_false(); 
>  
> and this obviously relies on the optimizer to build. But these
> are all slowly moving over the BUILD_BUG_ON() which also doesn't
> rely on the optimizer, so it's also obsolete.
>...

The reason why it works for Keith at all seems to be that even with -O0 
gcc already does this kind of optimizations.

Check e.g. the assembler generated for the following
(userspace) program with -O0:

<--  snip  -->

int foobar;

int main()
{ 
	if (1) return 0;

	foobar = 12345;

	return 42;
}

<--  snip  -->

> Hope this helps,
> 
> -Andi

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Building Kernel with -O0

[Andi Kleen]

> > I think a few more cases crept in where it was common to write
> > build time asserts as
> > 
> >       if (some condition the compiler evaluates at runtime)
>                                                      ^^^^^^^
> buildtime

Thank for the correction.

> 
> >          __error_condition_xyz_is_false(); 
> >  
> > and this obviously relies on the optimizer to build. But these
> > are all slowly moving over the BUILD_BUG_ON() which also doesn't
> > rely on the optimizer, so it's also obsolete.
> >...
> 
> The reason why it works for Keith at all seems to be that even with -O0 
> gcc already does this kind of optimizations.

What I mean is global optimization over the whole function, like
constant propagation. That is required for some of the more complex
macros like the *_user() ones.

Some expression evaluation is required by the C standard (e.g.
otherwise you couldn't generally declare global arrays with expressions)
While that's technically not required in the body of the function
outside declarations the compiler does it normally. But only inside
a statement. 

-Andi

Re: Building Kernel with -O0

[David Howells]

Adrian Bunk <bunk@kernel.org> wrote:

> Compiling with -O0 never worked, and is not likely to ever work.

Compiling with -O0 has worked previously, but not for a while.

David

Re: Building Kernel with -O0

[Adrian Bunk]

On Tue, Sep 09, 2008 at 11:24:25PM +0100, David Howells wrote:
> Adrian Bunk <bunk@kernel.org> wrote:
> 
> > Compiling with -O0 never worked, and is not likely to ever work.
> 
> Compiling with -O0 has worked previously, but not for a while.

You can call me surprised.

When did it work?

> David

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Building Kernel with -O0

[David Howells]

Adrian Bunk <bunk@kernel.org> wrote:

> When did it work?

When 2.4 was the cutting edge kernel.  I don't remember the dates or versions
exactly.

David