error when compiling dts preprocessed with -g3

error when compiling dts preprocessed with -g3

[Adrian Fiergolski]

Hi,

I am using yocto to build my custom distribution targetting arm64 on
Ubuntu 18.04.4.

I have an issue compiling dts which was preprocessed with gcc version
7.5.0 (I haven't tried other version) and '-g3' flag. The error message:

    ./scripts/dtc/dtc -O dtb -o
    arch/arm64/boot/dts/xilinx/zynqmp-falcon.dtb -b 0
    -i/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/
    -i/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/scripts/dtc/include-prefixes
    -Wno-unit_address_vs_reg -Wno-unit_address_format
    -Wno-avoid_unnecessary_addr_size -Wno-alias_paths
    -Wno-graph_child_address -Wno-simple_bus_reg
    -Wno-unique_unit_address -Wno-pci_device_reg  -d
    arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.d.dtc.tmp
    arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.dts.tmp

    Error: <built-in>:1.1-2 syntax error
    FATAL ERROR: Unable to parse input tree

The part of a preprocessed dts causing issue:

    # 1
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
    # 1
    "/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/linux-falcon_zynqmp-standard-build//"
    # 1 "<built-in>"
    #define __STDC__ 1
    #define __ASSEMBLER__ 1
    #define __STDC_HOSTED__ 1
    # 1 "<command-line>"
    #define __DTS__ 1
    # 1
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
    # 11
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
    /dts-v1/;

    # 1
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp.dtsi"
    1
    # 15
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp.dtsi"
    # 1
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/scripts/dtc/include-prefixes/dt-bindings/power/xlnx-zynqmp-power.h"
    1

The full command preprocessing dts:

    gcc -O0 -g3 -fno-omit-frame-pointer
    -L/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/recipe-sysroot-native/usr/lib
    -L/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/recipe-sysroot-native/lib
    -Wl,--enable-new-dtags
    -Wl,-rpath-link,/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/recipe-sysroot-native/usr/lib
    -Wl,-rpath-link,/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/recipe-sysroot-native/lib
    -Wl,-rpath,/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/recipe-sysroot-native/usr/lib
    -Wl,-rpath,/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/recipe-sysroot-native/lib
    -Wl,-O1 -Wl,--allow-shlib-undefined
    -Wl,--dynamic-linker=/home/afiergol/fastree/falcon/poky/build/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
    -E $(dtc_cpp_flags) -x assembler-with-cpp -o
    arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.dts.tmp

Once the '-g3' flag is removed, the part of the preprocessed DTS looks
like this:

    # 1
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
    # 1 "<built-in>"
    # 1 "<command-line>"
    # 1
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
    # 11
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
    /dts-v1/;

    # 1
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp.dtsi"
    1
    # 15
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp.dtsi"
    # 1
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/scripts/dtc/include-prefixes/dt-bindings/power/xlnx-zynqmp-power.h"
    1
    # 16
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp.dtsi"
    2
    # 1
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/scripts/dtc/include-prefixes/dt-bindings/reset/xlnx-zynqmp-resets.h"
    1
    # 17
    "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp.dtsi"
    2

And the device compiler succeeds.

Is it a known behaviour?

Regards,

Adrian

Re: error when compiling dts preprocessed with -g3

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 6091 bytes --]

On Fri, Aug 14, 2020 at 08:13:22PM +0200, Adrian Fiergolski wrote:
> Hi,
> 
> I am using yocto to build my custom distribution targetting arm64 on
> Ubuntu 18.04.4.
> 
> I have an issue compiling dts which was preprocessed with gcc version
> 7.5.0 (I haven't tried other version) and '-g3' flag. The error message:

Huh.  It's surprising to me that -g options affect the preprocessor
behavior at all.

>     ./scripts/dtc/dtc -O dtb -o
>     arch/arm64/boot/dts/xilinx/zynqmp-falcon.dtb -b 0
>     -i/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/
>     -i/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/scripts/dtc/include-prefixes
>     -Wno-unit_address_vs_reg -Wno-unit_address_format
>     -Wno-avoid_unnecessary_addr_size -Wno-alias_paths
>     -Wno-graph_child_address -Wno-simple_bus_reg
>     -Wno-unique_unit_address -Wno-pci_device_reg  -d
>     arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.d.dtc.tmp
>     arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.dts.tmp
> 
>     Error: <built-in>:1.1-2 syntax error
>     FATAL ERROR: Unable to parse input tree
> 
> The part of a preprocessed dts causing issue:
> 
>     # 1
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
>     # 1
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/linux-falcon_zynqmp-standard-build//"
>     # 1 "<built-in>"
>     #define __STDC__ 1

If it's preprocessed already, why does it still have #defines?

This is the cause of the problem - the dtc lexer can hande the lines
in the preprocessor output with file/linue number information, but it
doesn't expect there to be remaining preprocessor directives.

>     #define __ASSEMBLER__ 1
>     #define __STDC_HOSTED__ 1
>     # 1 "<command-line>"
>     #define __DTS__ 1
>     # 1
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
>     # 11
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
>     /dts-v1/;
> 
>     # 1
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp.dtsi"
>     1
>     # 15
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp.dtsi"
>     # 1
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/scripts/dtc/include-prefixes/dt-bindings/power/xlnx-zynqmp-power.h"
>     1
> 
> The full command preprocessing dts:
> 
>     gcc -O0 -g3 -fno-omit-frame-pointer
>     -L/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/recipe-sysroot-native/usr/lib
>     -L/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/recipe-sysroot-native/lib
>     -Wl,--enable-new-dtags
>     -Wl,-rpath-link,/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/recipe-sysroot-native/usr/lib
>     -Wl,-rpath-link,/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/recipe-sysroot-native/lib
>     -Wl,-rpath,/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/recipe-sysroot-native/usr/lib
>     -Wl,-rpath,/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/recipe-sysroot-native/lib
>     -Wl,-O1 -Wl,--allow-shlib-undefined
>     -Wl,--dynamic-linker=/home/afiergol/fastree/falcon/poky/build/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
>     -E $(dtc_cpp_flags) -x assembler-with-cpp -o
>     arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.dts.tmp
> 
> Once the '-g3' flag is removed, the part of the preprocessed DTS looks
> like this:
> 
>     # 1
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
>     # 1 "<built-in>"
>     # 1 "<command-line>"
>     # 1
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
>     # 11
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
>     /dts-v1/;
> 
>     # 1
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp.dtsi"
>     1
>     # 15
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp.dtsi"
>     # 1
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/scripts/dtc/include-prefixes/dt-bindings/power/xlnx-zynqmp-power.h"
>     1
>     # 16
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp.dtsi"
>     2
>     # 1
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/scripts/dtc/include-prefixes/dt-bindings/reset/xlnx-zynqmp-resets.h"
>     1
>     # 17
>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp.dtsi"
>     2
> 
> And the device compiler succeeds.
> 
> Is it a known behaviour?

Not to me.  I don't think I've ever tried using -g3.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: error when compiling dts preprocessed with -g3

[Adrian Fiergolski]

Hi David,

Thank you for your reply.

On 20.08.2020 12:27, David Gibson wrote:
> On Fri, Aug 14, 2020 at 08:13:22PM +0200, Adrian Fiergolski wrote:
>> Hi,
>>
>> I am using yocto to build my custom distribution targetting arm64 on
>> Ubuntu 18.04.4.
>>
>> I have an issue compiling dts which was preprocessed with gcc version
>> 7.5.0 (I haven't tried other version) and '-g3' flag. The error message:
> Huh.  It's surprising to me that -g options affect the preprocessor
> behavior at all.
>
>>     ./scripts/dtc/dtc -O dtb -o
>>     arch/arm64/boot/dts/xilinx/zynqmp-falcon.dtb -b 0
>>     -i/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/
>>     -i/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/scripts/dtc/include-prefixes
>>     -Wno-unit_address_vs_reg -Wno-unit_address_format
>>     -Wno-avoid_unnecessary_addr_size -Wno-alias_paths
>>     -Wno-graph_child_address -Wno-simple_bus_reg
>>     -Wno-unique_unit_address -Wno-pci_device_reg  -d
>>     arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.d.dtc.tmp
>>     arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.dts.tmp
>>
>>     Error: <built-in>:1.1-2 syntax error
>>     FATAL ERROR: Unable to parse input tree
>>
>> The part of a preprocessed dts causing issue:
>>
>>     # 1
>>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
>>     # 1
>>     "/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/linux-falcon_zynqmp-standard-build//"
>>     # 1 "<built-in>"
>>     #define __STDC__ 1
> If it's preprocessed already, why does it still have #defines?
>
> This is the cause of the problem - the dtc lexer can hande the lines
> in the preprocessor output with file/linue number information, but it
> doesn't expect there to be remaining preprocessor directives.

Yes, it's preprocessed already by gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0.

Are you able to reproduce the issue (any dts including dtsi should give
similar results)?

Regards,

Adrian

Re: error when compiling dts preprocessed with -g3

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3091 bytes --]

On Thu, Aug 20, 2020 at 01:32:17PM +0200, Adrian Fiergolski wrote:
> Hi David,
> 
> Thank you for your reply.
> 
> On 20.08.2020 12:27, David Gibson wrote:
> > On Fri, Aug 14, 2020 at 08:13:22PM +0200, Adrian Fiergolski wrote:
> >> Hi,
> >>
> >> I am using yocto to build my custom distribution targetting arm64 on
> >> Ubuntu 18.04.4.
> >>
> >> I have an issue compiling dts which was preprocessed with gcc version
> >> 7.5.0 (I haven't tried other version) and '-g3' flag. The error message:
> > Huh.  It's surprising to me that -g options affect the preprocessor
> > behavior at all.
> >
> >>     ./scripts/dtc/dtc -O dtb -o
> >>     arch/arm64/boot/dts/xilinx/zynqmp-falcon.dtb -b 0
> >>     -i/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/
> >>     -i/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/scripts/dtc/include-prefixes
> >>     -Wno-unit_address_vs_reg -Wno-unit_address_format
> >>     -Wno-avoid_unnecessary_addr_size -Wno-alias_paths
> >>     -Wno-graph_child_address -Wno-simple_bus_reg
> >>     -Wno-unique_unit_address -Wno-pci_device_reg  -d
> >>     arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.d.dtc.tmp
> >>     arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.dts.tmp
> >>
> >>     Error: <built-in>:1.1-2 syntax error
> >>     FATAL ERROR: Unable to parse input tree
> >>
> >> The part of a preprocessed dts causing issue:
> >>
> >>     # 1
> >>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
> >>     # 1
> >>     "/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/linux-falcon_zynqmp-standard-build//"
> >>     # 1 "<built-in>"
> >>     #define __STDC__ 1
> > If it's preprocessed already, why does it still have #defines?
> >
> > This is the cause of the problem - the dtc lexer can hande the lines
> > in the preprocessor output with file/linue number information, but it
> > doesn't expect there to be remaining preprocessor directives.
> 
> Yes, it's preprocessed already by gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0.
> 
> Are you able to reproduce the issue (any dts including dtsi should give
> similar results)?

No, I haven't been able to reproduce this problem.  I did hit a
different problem in the area, where cpp was generating line number
information in a format the dtc lexer didn't expect.  I've just pushed
a fix for that.

But leaving in #defines in preprocessed code really looks like a
problem with how the preprocessor is invoked, not with dtc.  If I'm
going to debug this without more messing around than I have time for,
please give me *specific* dts/dtsi files to attempt this, and
*exactly* how the preprocessor is being invoked.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: error when compiling dts preprocessed with -g3

[Adrian Fiergolski]

Hi David,

Thanks for your message.

I found a workaround for this issue and as I raised the problem almost a
year ago, I would need time to reproduce it, which unfortunately I can't
afford these days. Thus I can't support you in debugging this anymore.

Regards,

Adrian

On 08.06.2021 06:18, David Gibson wrote:
> On Thu, Aug 20, 2020 at 01:32:17PM +0200, Adrian Fiergolski wrote:
>> Hi David,
>>
>> Thank you for your reply.
>>
>> On 20.08.2020 12:27, David Gibson wrote:
>>> On Fri, Aug 14, 2020 at 08:13:22PM +0200, Adrian Fiergolski wrote:
>>>> Hi,
>>>>
>>>> I am using yocto to build my custom distribution targetting arm64 on
>>>> Ubuntu 18.04.4.
>>>>
>>>> I have an issue compiling dts which was preprocessed with gcc version
>>>> 7.5.0 (I haven't tried other version) and '-g3' flag. The error message:
>>> Huh.  It's surprising to me that -g options affect the preprocessor
>>> behavior at all.
>>>
>>>>     ./scripts/dtc/dtc -O dtb -o
>>>>     arch/arm64/boot/dts/xilinx/zynqmp-falcon.dtb -b 0
>>>>     -i/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/
>>>>     -i/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/scripts/dtc/include-prefixes
>>>>     -Wno-unit_address_vs_reg -Wno-unit_address_format
>>>>     -Wno-avoid_unnecessary_addr_size -Wno-alias_paths
>>>>     -Wno-graph_child_address -Wno-simple_bus_reg
>>>>     -Wno-unique_unit_address -Wno-pci_device_reg  -d
>>>>     arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.d.dtc.tmp
>>>>     arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.dts.tmp
>>>>
>>>>     Error: <built-in>:1.1-2 syntax error
>>>>     FATAL ERROR: Unable to parse input tree
>>>>
>>>> The part of a preprocessed dts causing issue:
>>>>
>>>>     # 1
>>>>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
>>>>     # 1
>>>>     "/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/linux-falcon_zynqmp-standard-build//"
>>>>     # 1 "<built-in>"
>>>>     #define __STDC__ 1
>>> If it's preprocessed already, why does it still have #defines?
>>>
>>> This is the cause of the problem - the dtc lexer can hande the lines
>>> in the preprocessor output with file/linue number information, but it
>>> doesn't expect there to be remaining preprocessor directives.
>> Yes, it's preprocessed already by gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0.
>>
>> Are you able to reproduce the issue (any dts including dtsi should give
>> similar results)?
> No, I haven't been able to reproduce this problem.  I did hit a
> different problem in the area, where cpp was generating line number
> information in a format the dtc lexer didn't expect.  I've just pushed
> a fix for that.
>
> But leaving in #defines in preprocessed code really looks like a
> problem with how the preprocessor is invoked, not with dtc.  If I'm
> going to debug this without more messing around than I have time for,
> please give me *specific* dts/dtsi files to attempt this, and
> *exactly* how the preprocessor is being invoked.
>

Re: error when compiling dts preprocessed with -g3

[Adrian Fiergolski]

Hi David,

I am being hit by this problem again. This time on Ubuntu 20.04 LTS with
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04), yocto (hardknott) and
kernel v5.10.

As it's standard kernel built, I am using scripts/Makefile.lib from
kernel sources. This line [1] causes the problem. As you indicated, it
calls the preprocessor and afterwards dtc.

I use the custom hardware (and .dts), however, I think you can reproduce
the problem by calling:

    git clone --depth 1 --single-branch --branch v5.10 
https://github.com/torvalds/linux.git
    cd linux
    gcc -g3 -O0 -fno-omit-frame-pointer -E -nostdinc
-Iscripts/dtc/include-prefixes -undef -D__DTS__ -x assembler-with-cpp -o
zynqmp-zcu102-revB.dts.tmp arch/arm64/boot/dts/xilinx/zynqmp-zcu102-revB.dts

The last command I extracted and simplified from [1], specific for my
yocto built. With '-g3' option, there are preprocessors directives left
in the produced zynqmp-zcu102-revB.dts.tmp which cause dts to fail.

Regards,

Adrian

[1]
https://github.com/torvalds/linux/blob/2c85ebc57b3e1817b6ce1a6b703928e113a90442/scripts/Makefile.lib#L319

On 08.06.2021 12:47, Adrian Fiergolski wrote:
> Hi David,
>
> Thanks for your message.
>
> I found a workaround for this issue and as I raised the problem almost a
> year ago, I would need time to reproduce it, which unfortunately I can't
> afford these days. Thus I can't support you in debugging this anymore.
>
> Regards,
>
> Adrian
>
> On 08.06.2021 06:18, David Gibson wrote:
>> On Thu, Aug 20, 2020 at 01:32:17PM +0200, Adrian Fiergolski wrote:
>>> Hi David,
>>>
>>> Thank you for your reply.
>>>
>>> On 20.08.2020 12:27, David Gibson wrote:
>>>> On Fri, Aug 14, 2020 at 08:13:22PM +0200, Adrian Fiergolski wrote:
>>>>> Hi,
>>>>>
>>>>> I am using yocto to build my custom distribution targetting arm64 on
>>>>> Ubuntu 18.04.4.
>>>>>
>>>>> I have an issue compiling dts which was preprocessed with gcc version
>>>>> 7.5.0 (I haven't tried other version) and '-g3' flag. The error message:
>>>> Huh.  It's surprising to me that -g options affect the preprocessor
>>>> behavior at all.
>>>>
>>>>>     ./scripts/dtc/dtc -O dtb -o
>>>>>     arch/arm64/boot/dts/xilinx/zynqmp-falcon.dtb -b 0
>>>>>     -i/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/
>>>>>     -i/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/scripts/dtc/include-prefixes
>>>>>     -Wno-unit_address_vs_reg -Wno-unit_address_format
>>>>>     -Wno-avoid_unnecessary_addr_size -Wno-alias_paths
>>>>>     -Wno-graph_child_address -Wno-simple_bus_reg
>>>>>     -Wno-unique_unit_address -Wno-pci_device_reg  -d
>>>>>     arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.d.dtc.tmp
>>>>>     arch/arm64/boot/dts/xilinx/.zynqmp-falcon.dtb.dts.tmp
>>>>>
>>>>>     Error: <built-in>:1.1-2 syntax error
>>>>>     FATAL ERROR: Unable to parse input tree
>>>>>
>>>>> The part of a preprocessed dts causing issue:
>>>>>
>>>>>     # 1
>>>>>     "/home/afiergol/fastree/falcon/poky/build/tmp/work-shared/falcon-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-falcon.dts"
>>>>>     # 1
>>>>>     "/home/afiergol/fastree/falcon/poky/build/tmp/work/falcon_zynqmp-poky-linux/linux-xlnx/5.4+gitAUTOINC+22b71b4162-r0/linux-falcon_zynqmp-standard-build//"
>>>>>     # 1 "<built-in>"
>>>>>     #define __STDC__ 1
>>>> If it's preprocessed already, why does it still have #defines?
>>>>
>>>> This is the cause of the problem - the dtc lexer can hande the lines
>>>> in the preprocessor output with file/linue number information, but it
>>>> doesn't expect there to be remaining preprocessor directives.
>>> Yes, it's preprocessed already by gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0.
>>>
>>> Are you able to reproduce the issue (any dts including dtsi should give
>>> similar results)?
>> No, I haven't been able to reproduce this problem.  I did hit a
>> different problem in the area, where cpp was generating line number
>> information in a format the dtc lexer didn't expect.  I've just pushed
>> a fix for that.
>>
>> But leaving in #defines in preprocessed code really looks like a
>> problem with how the preprocessor is invoked, not with dtc.  If I'm
>> going to debug this without more messing around than I have time for,
>> please give me *specific* dts/dtsi files to attempt this, and
>> *exactly* how the preprocessor is being invoked.
>>

Re: error when compiling dts preprocessed with -g3

[Rob Herring]

On Tue, Oct 12, 2021 at 8:16 AM Adrian Fiergolski
<adrian.fiergolski-/A8SZzjb9pj2eFz/2MeuCQ@public.gmane.org> wrote:
>
> Hi David,
>
> I am being hit by this problem again. This time on Ubuntu 20.04 LTS with
> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04), yocto (hardknott) and
> kernel v5.10.
>
> As it's standard kernel built, I am using scripts/Makefile.lib from
> kernel sources. This line [1] causes the problem. As you indicated, it
> calls the preprocessor and afterwards dtc.

How is '-g3' getting introduced here? You are externally overriding
HOSTCC with 'gcc -g3' or something like that?

Just add '-g0' to the command line in [1]. Debug info is never useful
here. The documentation for -g3 clearly says it keeps #defines which
is fundamentally incompatible with our usage of the preprocessor.

Rob

Re: error when compiling dts preprocessed with -g3

[Adrian Fiergolski]

On 12.10.2021 15:38, Rob Herring wrote:
> On Tue, Oct 12, 2021 at 8:16 AM Adrian Fiergolski
> <adrian.fiergolski-/A8SZzjb9pj2eFz/2MeuCQ@public.gmane.org> wrote:
>> Hi David,
>>
>> I am being hit by this problem again. This time on Ubuntu 20.04 LTS with
>> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04), yocto (hardknott) and
>> kernel v5.10.
>>
>> As it's standard kernel built, I am using scripts/Makefile.lib from
>> kernel sources. This line [1] causes the problem. As you indicated, it
>> calls the preprocessor and afterwards dtc.
> How is '-g3' getting introduced here? You are externally overriding
> HOSTCC with 'gcc -g3' or something like that?

Yes, that's the way yocto introduces kernel build flags [1].

[1]
https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/kernel.bbclass#n229

Adrian

Re: error when compiling dts preprocessed with -g3

[Rob Herring]

On Tue, Oct 12, 2021 at 9:47 AM Adrian Fiergolski
<adrian.fiergolski-/A8SZzjb9pj2eFz/2MeuCQ@public.gmane.org> wrote:
>
> On 12.10.2021 15:38, Rob Herring wrote:
> > On Tue, Oct 12, 2021 at 8:16 AM Adrian Fiergolski
> > <adrian.fiergolski-/A8SZzjb9pj2eFz/2MeuCQ@public.gmane.org> wrote:
> >> Hi David,
> >>
> >> I am being hit by this problem again. This time on Ubuntu 20.04 LTS with
> >> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04), yocto (hardknott) and
> >> kernel v5.10.
> >>
> >> As it's standard kernel built, I am using scripts/Makefile.lib from
> >> kernel sources. This line [1] causes the problem. As you indicated, it
> >> calls the preprocessor and afterwards dtc.
> > How is '-g3' getting introduced here? You are externally overriding
> > HOSTCC with 'gcc -g3' or something like that?
>
> Yes, that's the way yocto introduces kernel build flags [1].

HOSTCC is supposed to be set to just the compiler. HOSTCFLAGS and
others should be used to set the flags.

Rob

>
> [1]
> https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/kernel.bbclass#n229
>
> Adrian
>

Re: error when compiling dts preprocessed with -g3

[Adrian Fiergolski]

I let myself add to this thread Wenzong, author of the change in the
yocto class using HOSTCC, and Ross and Richard who signed off the change.

Wenzong, Ross and Richard, there is a problem in the way you use HOSTCC,
once one adds '-g3' debug flag and DTS compiler. All the details
regarding the problem can be found in the archives [1].

Regards,

Adrian

[1] https://www.spinics.net/lists/devicetree-compiler/index.html#03281

On 12.10.2021 18:50, Rob Herring wrote:
> On Tue, Oct 12, 2021 at 9:47 AM Adrian Fiergolski
> <adrian.fiergolski-/A8SZzjb9pj2eFz/2MeuCQ@public.gmane.org> wrote:
>> On 12.10.2021 15:38, Rob Herring wrote:
>>> On Tue, Oct 12, 2021 at 8:16 AM Adrian Fiergolski
>>> <adrian.fiergolski-/A8SZzjb9pj2eFz/2MeuCQ@public.gmane.org> wrote:
>>>> Hi David,
>>>>
>>>> I am being hit by this problem again. This time on Ubuntu 20.04 LTS with
>>>> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04), yocto (hardknott) and
>>>> kernel v5.10.
>>>>
>>>> As it's standard kernel built, I am using scripts/Makefile.lib from
>>>> kernel sources. This line [1] causes the problem. As you indicated, it
>>>> calls the preprocessor and afterwards dtc.
>>> How is '-g3' getting introduced here? You are externally overriding
>>> HOSTCC with 'gcc -g3' or something like that?
>> Yes, that's the way yocto introduces kernel build flags [1].
> HOSTCC is supposed to be set to just the compiler. HOSTCFLAGS and
> others should be used to set the flags.
>
> Rob
>
>> [1]
>> https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/kernel.bbclass#n229
>>
>> Adrian
>>

Re: error when compiling dts preprocessed with -g3

[Richard Purdie]

On Thu, 2021-10-14 at 11:39 +0200, Adrian Fiergolski wrote:
> 
> On 12.10.2021 18:50, Rob Herring wrote:
> > On Tue, Oct 12, 2021 at 9:47 AM Adrian Fiergolski
> > <adrian.fiergolski-/A8SZzjb9pj2eFz/2MeuCQ@public.gmane.org> wrote:
> > > On 12.10.2021 15:38, Rob Herring wrote:
> > > > On Tue, Oct 12, 2021 at 8:16 AM Adrian Fiergolski
> > > > <adrian.fiergolski-/A8SZzjb9pj2eFz/2MeuCQ@public.gmane.org> wrote:
> > > > > Hi David,
> > > > > 
> > > > > I am being hit by this problem again. This time on Ubuntu 20.04 LTS with
> > > > > gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04), yocto (hardknott) and
> > > > > kernel v5.10.
> > > > > 
> > > > > As it's standard kernel built, I am using scripts/Makefile.lib from
> > > > > kernel sources. This line [1] causes the problem. As you indicated, it
> > > > > calls the preprocessor and afterwards dtc.
> > > > How is '-g3' getting introduced here? You are externally overriding
> > > > HOSTCC with 'gcc -g3' or something like that?
> > > Yes, that's the way yocto introduces kernel build flags [1].
> > HOSTCC is supposed to be set to just the compiler. HOSTCFLAGS and
> > others should be used to set the flags.
> > 
> > Rob
> > 
> > > [1]
> > > https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/kernel.bbclass#n229
> > > 
> > > Adrian
> > > 
> 
> I let myself add to this thread Wenzong, author of the change in the
> yocto class using HOSTCC, and Ross and Richard who signed off the change.
> 
> Wenzong, Ross and Richard, there is a problem in the way you use HOSTCC,
> once one adds '-g3' debug flag and DTS compiler. All the details
> regarding the problem can be found in the archives [1].
> 
> 
> [1] https://www.spinics.net/lists/devicetree-compiler/index.html#03281

This isn't my area of expertise but I took a look and discussed it with our
kernel people and we agree this doesn't look right. I've queued a change for
testing which makes this look like:

EXTRA_OEMAKE = " HOSTCC="${BUILD_CC}" HOSTCFLAGS="${BUILD_CFLAGS}" HOST_LDFLAGS="${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}""
EXTRA_OEMAKE += " HOSTCXX="${BUILD_CXX}" HOSTCXXFLAGS="${BUILD_CXXFLAGS}""
 
which I think is what you're asking for. Whether this breaks things, we'll see.

Cheers,

Richard