[U-Boot] Building under Cygwin - "-ansi" flag?

[U-Boot] Building under Cygwin - "-ansi" flag?

[Vasili Galka]

Hi,

I'm building u-boot (am335x_evm config) using arm-none-eabi toolchain
under Cygwin. When trying to build unmodified v2014.04 tag I get some
errors. However, if I remove the "-ansi" flag from the Makefile,
everything builds fine:

-ifeq ($(HOSTOS),cygwin)
-HOSTCFLAGS    += -ansi
-endif

I tried tracking down where those lines came from and why building
under Cygwin requires separate treatment. I found our this originates
from the following commit (dated Jan 2008):

  SHA-1: 883e3925d99a8dd69c5b0201cba5b1887f88f95c

  * Fix build problems under Cygwin

  This patch allows u-boot to build without error in a cygwin
  environment.  Cygwin does not define __u64 in it's
  include/asm/types.h file.  The -idirafter flag in the u-boot
  build causes the inclusion of the cygwin types.h file as opposed
  to u-bot/include/asm/types.h file which does define __u64.
  Subsequently, sha1.c compile fails due to unknown symbol.

  Signed-off-by: Brian Miller <raptorbrino@netscape.net>

This comment does not really explain why "-ansi" flag is required.
Personally, I don't see any reason for it.

Maybe it should be removed at all?

Best,
Vasili

[U-Boot] Building under Cygwin - "-ansi" flag?

[Tom Rini]

On Thu, May 08, 2014 at 02:49:35PM +0300, Vasili Galka wrote:
> Hi,
> 
> I'm building u-boot (am335x_evm config) using arm-none-eabi toolchain
> under Cygwin. When trying to build unmodified v2014.04 tag I get some
> errors. However, if I remove the "-ansi" flag from the Makefile,
> everything builds fine:
> 
> -ifeq ($(HOSTOS),cygwin)
> -HOSTCFLAGS    += -ansi
> -endif
> 
> I tried tracking down where those lines came from and why building
> under Cygwin requires separate treatment. I found our this originates
> from the following commit (dated Jan 2008):
> 
>   SHA-1: 883e3925d99a8dd69c5b0201cba5b1887f88f95c
> 
>   * Fix build problems under Cygwin
> 
>   This patch allows u-boot to build without error in a cygwin
>   environment.  Cygwin does not define __u64 in it's
>   include/asm/types.h file.  The -idirafter flag in the u-boot
>   build causes the inclusion of the cygwin types.h file as opposed
>   to u-bot/include/asm/types.h file which does define __u64.
>   Subsequently, sha1.c compile fails due to unknown symbol.
> 
>   Signed-off-by: Brian Miller <raptorbrino@netscape.net>
> 
> This comment does not really explain why "-ansi" flag is required.

-ansi changes include order perhaps?

> Personally, I don't see any reason for it.
> 
> Maybe it should be removed at all?

Yes, but can you please test a few more builds?  If you're able to use
MAKEALL, please run MAKEALL -a arm

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140509/2ba29205/attachment.pgp>

[U-Boot] Building under Cygwin - "-ansi" flag?

[Vasili Galka]

On Sun, May 11, 2014 at 1:27 PM, Vasili Galka <vvv444@gmail.com> wrote:
>
> On Fri, May 9, 2014 at 6:08 PM, Tom Rini <trini@ti.com> wrote:
>>
>> On Thu, May 08, 2014 at 02:49:35PM +0300, Vasili Galka wrote:
>> > Hi,
>> >
>> > I'm building u-boot (am335x_evm config) using arm-none-eabi toolchain
>> > under Cygwin. When trying to build unmodified v2014.04 tag I get some
>> > errors. However, if I remove the "-ansi" flag from the Makefile,
>> > everything builds fine:
>> >
>> > -ifeq ($(HOSTOS),cygwin)
>> > -HOSTCFLAGS    += -ansi
>> > -endif
>> >
>> > I tried tracking down where those lines came from and why building
>> > under Cygwin requires separate treatment. I found our this originates
>> > from the following commit (dated Jan 2008):
>> >
>> >   SHA-1: 883e3925d99a8dd69c5b0201cba5b1887f88f95c
>> >
>> >   * Fix build problems under Cygwin
>> >
>> >   This patch allows u-boot to build without error in a cygwin
>> >   environment.  Cygwin does not define __u64 in it's
>> >   include/asm/types.h file.  The -idirafter flag in the u-boot
>> >   build causes the inclusion of the cygwin types.h file as opposed
>> >   to u-bot/include/asm/types.h file which does define __u64.
>> >   Subsequently, sha1.c compile fails due to unknown symbol.
>> >
>> >   Signed-off-by: Brian Miller <raptorbrino@netscape.net>
>> >
>> > This comment does not really explain why "-ansi" flag is required.
>>
>> -ansi changes include order perhaps?
>>
>> > Personally, I don't see any reason for it.
>> >
>> > Maybe it should be removed at all?
>>
>> Yes, but can you please test a few more builds?  If you're able to use
>> MAKEALL, please run MAKEALL -a arm
>>
>> --
>> Tom
>
>
> Hi Tom,
>
> There are indeed some errors, I look into it later and keep you updated.
>
> Best,
> Vasili


Dear Tom,

I've finally found time to look deeper into this problem. Let me outline
the situation:
1. The problems with Cygwin build arise in code built for the "host"
   architecture i.e. stuff in "tools" sub-directory.
2. Along the years, there were several commits targeting the problems.
   All were related to include search paths/their order:
    a. e1a3f6b (July 2004): U-Boot include paths for "tools" build were
       changed to "-idirafter", i.e. host headers were given higher
       priority than headers in U-Boot tree.
    b. 883e392 (Jan 2008): The above patch caused "asm/types.h" shipped
       with Cygwin to be included. Turned out Cygwin does not define
       __u64, so this patch tried to workaround it by adding the "-ansi"
       flag. This flag dropped some include search paths effectively
       returning the use of "asm/types.h" from U-Boot tree.
    c. 669dfc2 (March 2013): Yet another header was included from the
       host paths and caused problems: "libfdt_env.h". This time it was
       worked around by adding the following flag to override it:
       "-include $(SRCTREE)/include/libfdt_env.h"
3. Currently all builds fine on Linux. However, on Cygwin some configs
   fail to build since not all the code complies with the ANSI standard.
   On the other hand, removing the "-ansi" flag causes build errors in
   other configs due to reasons explained above.

Now my thoughts on the situation:
1. Requesting compliance of U-Boot code with the ANSI standard is an
   overshoot. It was used as a workaround to a problem and a different
   solution shall be found. There is already a code that does not
   compile with it.
   Speaking of it, is there a policy about U-Boot code compliance to
   some standard? C99? GNU99? If not, I think it's important to define
   one and stick to it. This will eliminate uncertainty and simplify
   things in the future.
2. I see a reoccurring pattern of some headers being used from the host,
   causing mismatches and arising build errors. Then some way is sought
   to include headers from U-Boot tree prior to them which solves the
   problem. This happens for both Linux and Cygwin. This pattern is a
   result of the original decision from 2004 to prioritize the host
   include paths over the paths from U-Boot tree. I see this happening
   again and again with different headers in the future. So here comes
   the question, is it really the right thing prioritize the include
   paths this way? Why do host paths MUST come first?
   I'll try reverting this locally and looking what breaks and what
   alternative solutions exist.

P.S. I'm building all with arm-none-eabi toolchain. I set up
powerpc-none-eabi toolchain under Linux and got some build errors.
I'll open a separate thread about it.

Best regards,
Vasili

[U-Boot] Building under Cygwin - "-ansi" flag?

[Jeroen Hofstee]

Dear Vasili,

On ma, 2014-05-19 at 12:45 +0300, Vasili Galka wrote:
[...]

> 2. I see a reoccurring pattern of some headers being used from the host,
>    causing mismatches and arising build errors. Then some way is sought
>    to include headers from U-Boot tree prior to them which solves the
>    problem. This happens for both Linux and Cygwin. 

you can add FreeBSD to the list as well, see [1]

>    This pattern is a
>    result of the original decision from 2004 to prioritize the host
>    include paths over the paths from U-Boot tree. 

any reference?

> I see this happening
>    again and again with different headers in the future. So here comes
>    the question, is it really the right thing prioritize the include
>    paths this way? Why do host paths MUST come first?
>    I'll try reverting this locally and looking what breaks and what
>    alternative solutions exist.

I have no idea why it is the way it is, but keep in mind that e.g. stdio
headers in u-boot is quite something different then stdio for the target
userland.

Regards,
Jeroen

[1] http://lists.denx.de/pipermail/u-boot/2014-May/179301.html

[U-Boot] Building under Cygwin - "-ansi" flag?

[Vasili Galka]

Hi Jeroen,

On Tue, May 20, 2014 at 9:06 PM, Jeroen Hofstee <dasuboot@myspectrum.nl> wrote:
> Dear Vasili,
>
> On ma, 2014-05-19 at 12:45 +0300, Vasili Galka wrote:
> [...]
>
>> 2. I see a reoccurring pattern of some headers being used from the host,
>>    causing mismatches and arising build errors. Then some way is sought
>>    to include headers from U-Boot tree prior to them which solves the
>>    problem. This happens for both Linux and Cygwin.
>
> you can add FreeBSD to the list as well, see [1]

Thanks! This supports my point that we have a reoccurring problem here that
requires a more general solution.

>>    This pattern is a
>>    result of the original decision from 2004 to prioritize the host
>>    include paths over the paths from U-Boot tree.
>
> any reference?

This decision is a part of the above mentioned commit: e1a3f6b (July 2004)
I don't know how much the original committer was aware of its long
term implications.

>> I see this happening
>>    again and again with different headers in the future. So here comes
>>    the question, is it really the right thing prioritize the include
>>    paths this way? Why do host paths MUST come first?
>>    I'll try reverting this locally and looking what breaks and what
>>    alternative solutions exist.
>
> I have no idea why it is the way it is, but keep in mind that e.g. stdio
> headers in u-boot is quite something different then stdio for the target
> userland.

Sure. I'll keep it in mind while I'm designing a solution here. I'm
afraid there is
no easy way to fix it though.

> Regards,
> Jeroen
>
> [1] http://lists.denx.de/pipermail/u-boot/2014-May/179301.html
>

Best,
Vasili

[U-Boot] Building under Cygwin - "-ansi" flag?

[Jeroen Hofstee]

Hello Vasili,

On wo, 2014-05-21 at 11:50 +0300, Vasili Galka wrote:

> >>    This pattern is a
> >>    result of the original decision from 2004 to prioritize the host
> >>    include paths over the paths from U-Boot tree.
> >
> > any reference?
> 
> This decision is a part of the above mentioned commit: e1a3f6b (July 2004)
> I don't know how much the original committer was aware of its long
> term implications.

If the only valid reason is "Fix host tools building in Cygwin
environment" as mentioned in the original commit, then I am all in favor
of dropping it and finding a decent solution for cygwin.

> >> I see this happening
> >>    again and again with different headers in the future. So here comes
> >>    the question, is it really the right thing prioritize the include
> >>    paths this way? Why do host paths MUST come first?
> >>    I'll try reverting this locally and looking what breaks and what
> >>    alternative solutions exist.
> >
> > I have no idea why it is the way it is, but keep in mind that e.g. stdio
> > headers in u-boot is quite something different then stdio for the target
> > userland.
> 
> Sure. I'll keep it in mind while I'm designing a solution here. I'm
> afraid there is no easy way to fix it though.
> 
This is easier than it sounds. U-boot is build with -nostdinc for the
binary itself. And it tries to get the compiler related includes back
with "isystem $(shell $(CC) -print-file-name=include". (and the printf
declaration and friends are actually in common.h for the loader, which
makes it even harder to do it wrong by accident).

Can you check what "arm-none-eabi-gcc -print-file-name=include" returns
on cygwin?

mmm, this one might be also be a challenge for cygwin: 
"dirname `arm-none-eabi-gcc -print-libgcc-file-name`", but you will
likely get linker errors if that contains spaces / backslashes or simply
fails.

But perhaps even easier, can you post the problems you encounter if you
remove the idirafter. Likely easier then guessing what can go wrong in a
cygwin build.

Regards,
Jeroen

[U-Boot] Building under Cygwin - "-ansi" flag?

[Vasili Galka]

Hi Jeroen,

On Thu, May 22, 2014 at 9:27 PM, Jeroen Hofstee <dasuboot@myspectrum.nl> wrote:
> Hello Vasili,
>
> On wo, 2014-05-21 at 11:50 +0300, Vasili Galka wrote:
>
>> >>    This pattern is a
>> >>    result of the original decision from 2004 to prioritize the host
>> >>    include paths over the paths from U-Boot tree.
>> >
>> > any reference?
>>
>> This decision is a part of the above mentioned commit: e1a3f6b (July 2004)
>> I don't know how much the original committer was aware of its long
>> term implications.
>
> If the only valid reason is "Fix host tools building in Cygwin
> environment" as mentioned in the original commit, then I am all in favor
> of dropping it and finding a decent solution for cygwin.
>
>> >> I see this happening
>> >>    again and again with different headers in the future. So here comes
>> >>    the question, is it really the right thing prioritize the include
>> >>    paths this way? Why do host paths MUST come first?
>> >>    I'll try reverting this locally and looking what breaks and what
>> >>    alternative solutions exist.
>> >
>> > I have no idea why it is the way it is, but keep in mind that e.g. stdio
>> > headers in u-boot is quite something different then stdio for the target
>> > userland.
>>
>> Sure. I'll keep it in mind while I'm designing a solution here. I'm
>> afraid there is no easy way to fix it though.
>>
> This is easier than it sounds. U-boot is build with -nostdinc for the
> binary itself. And it tries to get the compiler related includes back
> with "isystem $(shell $(CC) -print-file-name=include". (and the printf
> declaration and friends are actually in common.h for the loader, which
> makes it even harder to do it wrong by accident).
>
> Can you check what "arm-none-eabi-gcc -print-file-name=include" returns
> on cygwin?
>
> mmm, this one might be also be a challenge for cygwin:
> "dirname `arm-none-eabi-gcc -print-libgcc-file-name`", but you will
> likely get linker errors if that contains spaces / backslashes or simply
> fails.
>
> But perhaps even easier, can you post the problems you encounter if you
> remove the idirafter. Likely easier then guessing what can go wrong in a
> cygwin build.
>
> Regards,
> Jeroen
>
>

Let's suppose there is an application in tools directory that requires
both "libfdt_env.h" (from uboot tree) and "errno.h" (from host
environment). However, as both headers exist at both places, in the
uboot tree and on the host environment, there is no way to solve this
just by altering the include paths order. For any reshuffling both
headers will be taken either from uboot or from host - which is
undesired.

I can only think of one bulletproof solution that would fix the above
problem. Unfortunately it's conceptual and requires much work:
* Currently, almost all include derectives in in uboot code use <>
style. I propose to modify this concept.
* For each #include directive the developer shall decide from where
the header file is obtained - U-Boot source tree or the host
environment.
* Depending on that, "" shall be used only for headers from U-Boot
tree, <> shall be used only for all other (host provided in tools
folder)
* The -iquote compiler flag can be used for adding "" search paths.
The -I flag for adding <> paths.

Of course that's just a draft of the idea. What do you think?

Best,
Vasili

[U-Boot] Building under Cygwin - "-ansi" flag?

[Masahiro Yamada]

Hi.


> 
> I've finally found time to look deeper into this problem. Let me outline
> the situation:
> 1. The problems with Cygwin build arise in code built for the "host"
>    architecture i.e. stuff in "tools" sub-directory.
> 2. Along the years, there were several commits targeting the problems.
>    All were related to include search paths/their order:
>     a. e1a3f6b (July 2004): U-Boot include paths for "tools" build were
>        changed to "-idirafter", i.e. host headers were given higher
>        priority than headers in U-Boot tree.
>     b. 883e392 (Jan 2008): The above patch caused "asm/types.h" shipped
>        with Cygwin to be included. Turned out Cygwin does not define
>        __u64, so this patch tried to workaround it by adding the "-ansi"
>        flag. This flag dropped some include search paths effectively
>        returning the use of "asm/types.h" from U-Boot tree.

So, with Sandbox configuration,  host-tools include arch/sandbox/include/asm/types.h
With beagle bone configuration,  they include arch/arm/include/asm/type.h

How amazing!



>     c. 669dfc2 (March 2013): Yet another header was included from the
>        host paths and caused problems: "libfdt_env.h". This time it was
>        worked around by adding the following flag to override it:
>        "-include $(SRCTREE)/include/libfdt_env.h"

Seems people are only adding new features with ridiculous workarounds,
without refactrong the code,
without thinking what the proper implementing is.
Sadly.


Best Regards
Masahiro Yamada

[U-Boot] Building under Cygwin - "-ansi" flag?

[Masahiro Yamada]

Hi.

> >
> 
> Let's suppose there is an application in tools directory that requires
> both "libfdt_env.h" (from uboot tree) and "errno.h" (from host
> environment). However, as both headers exist at both places, in the
> uboot tree and on the host environment, there is no way to solve this
> just by altering the include paths order. For any reshuffling both
> headers will be taken either from uboot or from host - which is
> undesired.

True.
What we should do is not to tweak the inclusion order.


I guess we should follow what happended in Linux Kernel world:

They split out the user-space API content in separated headers
and put them into  new   include/uapi/  and  arch/$(ARCH)/include/uapi/
directories.


So, the right direction we should go is:

- Define the exact tools-API  which should be exported to
  host program space. (For ex.  image structure)

- Put the tool-API headers into a new directory.
    (I have no idea about a good dir name.   include/host-api/ ? )
 This directory is added to include-path for  both U-Boot and host programs.


               include                                          include
 U-Boot  ---------->   include/host-api/    <--------   Host programs


U-boot  includes:  include/host-api/  +  include/   + arch/$ARCH/include/

Host programs include:  include/host-api/ + headers provided by environments





Comments are welcome !


Best Regards
Masahiro Yamada

[U-Boot] Building under Cygwin - "-ansi" flag?

[Masahiro Yamada]

Hi.


On Thu, 8 May 2014 14:49:35 +0300
Vasili Galka <vvv444@gmail.com> wrote:

> Hi,
> 
> I'm building u-boot (am335x_evm config) using arm-none-eabi toolchain
> under Cygwin. When trying to build unmodified v2014.04 tag I get some
> errors. However, if I remove the "-ansi" flag from the Makefile,
> everything builds fine:
> 
> -ifeq ($(HOSTOS),cygwin)
> -HOSTCFLAGS    += -ansi
> -endif
> 
> I tried tracking down where those lines came from and why building
> under Cygwin requires separate treatment. I found our this originates
> from the following commit (dated Jan 2008):
> 
>   SHA-1: 883e3925d99a8dd69c5b0201cba5b1887f88f95c
> 
>   * Fix build problems under Cygwin
> 
>   This patch allows u-boot to build without error in a cygwin
>   environment.  Cygwin does not define __u64 in it's
>   include/asm/types.h file.  The -idirafter flag in the u-boot
>   build causes the inclusion of the cygwin types.h file as opposed
>   to u-bot/include/asm/types.h file which does define __u64.
>   Subsequently, sha1.c compile fails due to unknown symbol.
> 
>   Signed-off-by: Brian Miller <raptorbrino@netscape.net>
> 
> This comment does not really explain why "-ansi" flag is required.
> Personally, I don't see any reason for it.
> 
> Maybe it should be removed at all?
> 


As far as I tested,  missing __u64 error occurred
when I tried to build host tools on Cygwin,
regardless  with/without  -ansi option.


This is my build log:


  GEN     include/autoconf.mk.dep
  GEN     include/autoconf.mk
  CHK     include/config/uboot.release
  CHK     include/generated/version_autogenerated.h
  CHK     include/generated/timestamp_autogenerated.h
  UPD     include/generated/timestamp_autogenerated.h
  HOSTCC  scripts/basic/fixdep
scripts/basic/fixdep.c: In function 'parse_config_file':
scripts/basic/fixdep.c:255:4: warning: array subscript has type 'char' [-Wchar-subscripts]
    if (!(isalnum(*q) || *q == '_'))
    ^
  CC      lib/asm-offsets.s
  GEN     include/generated/generic-asm-offsets.h
  HOSTCC  tools/img2srec
In file included from /home/yamada/workspace/u-boot/include/libfdt_env.h:13:0,
                 from <command-line>:0:
/home/yamada/workspace/u-boot/include/linux/types.h:135:1: error: unknown type name '__u64'
 typedef __u64 __bitwise __le64;
 ^
/home/yamada/workspace/u-boot/include/linux/types.h:136:1: error: unknown type name '__u64'
 typedef __u64 __bitwise __be64;
 ^
scripts/Makefile.host:118: recipe for target 'tools/img2srec' failed
make[1]: *** [tools/img2srec] Error 1
Makefile:1022: recipe for target 'tools' failed
make: *** [tools] Error 2







/usr/include/asm/types.h  on my Cygwin is like this:

------------------------------->8----------------------
/* asm/types.h

   Copyright 1998, 2000, 2001 Red Hat, Inc.

This file is part of Cygwin.

This software is a copyrighted work licensed under the terms of the
Cygwin license.  Please consult the file "CYGWIN_LICENSE" for
details. */

#ifndef _ASM_TYPES_H
#define _ASM_TYPES_H

typedef __signed__ char __s8;
typedef unsigned char __u8;

typedef __signed__ short __s16;
typedef unsigned short __u16;

typedef __signed__ int __s32;
typedef unsigned int __u32;

#endif /* _ASM_TYPES_H */
-----------------------------8<-----------------------------


Best Regards
Masahiro Yamada

[U-Boot] Building under Cygwin - "-ansi" flag?

[Vasili Galka]

Hi Masahiro,

>> Let's suppose there is an application in tools directory that requires
>> both "libfdt_env.h" (from uboot tree) and "errno.h" (from host
>> environment). However, as both headers exist at both places, in the
>> uboot tree and on the host environment, there is no way to solve this
>> just by altering the include paths order. For any reshuffling both
>> headers will be taken either from uboot or from host - which is
>> undesired.
>
> True.
> What we should do is not to tweak the inclusion order.
>
>
> I guess we should follow what happended in Linux Kernel world:
>
> They split out the user-space API content in separated headers
> and put them into  new   include/uapi/  and  arch/$(ARCH)/include/uapi/
> directories.
>
>
> So, the right direction we should go is:
>
> - Define the exact tools-API  which should be exported to
>   host program space. (For ex.  image structure)
>
> - Put the tool-API headers into a new directory.
>     (I have no idea about a good dir name.   include/host-api/ ? )
>  This directory is added to include-path for  both U-Boot and host programs.
>
>
>                include                                          include
>  U-Boot  ---------->   include/host-api/    <--------   Host programs
>
>
> U-boot  includes:  include/host-api/  +  include/   + arch/$ARCH/include/
>
> Host programs include:  include/host-api/ + headers provided by environments

What you propose sounds good. But are you sure that no tools require a
really deep details of U-Boot internals? In such case separating the
details into an interface won't be possible. If (hopefully) that's not
the case, interfaces seem to be the right direction. Although
implementing them requires great work.
In any case, as a new comer, I don't think I know U-Boot enough for
such design decision. I would be glad to hear the opinion of other
major developers.

As about the solution I initially thought of, here is a more detailed design:
https://docs.google.com/document/d/1D86EedRrViJHsTZrf9bEQB-DYsEweHDKaJyZYI2Yr3I/edit?usp=sharing
It does not contradict what you propose and these two can be used
together. My proposal does not target only the specific compilation
issues descussed here but the general ease of code understanding and
navigation. I think it is much easier to implement and can provide a
good short term solution. Since re-factoring host-api requires much
effort and deep understanding of existing code. The only drawback of
my proposal I see is the large patch footprint. But maybe you see
other weaknesses. I'll be glad for your opinion.

Best regards,
Vasili