Re: [PATCH dwarves] pahole: Avoid generating artificial inlined functions for BTF

[Alan Maguire <alan.maguire@oracle.com>]

On 20/10/2025 21:44, David Faust wrote:
> 
> 
> On 10/20/25 13:11, Alan Maguire wrote:
>> On 20/10/2025 17:01, Yonghong Song wrote:
>>>
>>>
>>> On 10/20/25 3:53 AM, Alan Maguire wrote:
>>>> On 03/10/2025 18:36, Yonghong Song wrote:
>>>>> In llvm pull request [1], the dwarf is changed to accommodate functions
>>>>> whose signatures are different from source level although they have
>>>>> the same name. Other non-source functions are also included in dwarf.
>>>>>
>>>>> The following is an example:
>>>>>
>>>>> The source:
>>>>> ====
>>>>>    $ cat test.c
>>>>>    struct t { int a; };
>>>>>    char *tar(struct t *a, struct t *d);
>>>>>    __attribute__((noinline)) static char * foo(struct t *a, struct t
>>>>> *d, int b)
>>>>>    {
>>>>>      return tar(a, d);
>>>>>    }
>>>>>    char *bar(struct t *a, struct t *d)
>>>>>    {
>>>>>      return foo(a, d, 1);
>>>>>    }
>>>>> ====
>>>>>
>>>>> Part of generated dwarf:
>>>>> ====
>>>>> 0x0000005c:   DW_TAG_subprogram
>>>>>                  DW_AT_low_pc    (0x0000000000000010)
>>>>>                  DW_AT_high_pc   (0x0000000000000015)
>>>>>                  DW_AT_frame_base        (DW_OP_reg7 RSP)
>>>>>                  DW_AT_linkage_name      ("foo")
>>>>>                  DW_AT_name      ("foo")
>>>>>                  DW_AT_decl_file ("/home/yhs/tests/sig-change/
>>>>> deadarg/test.c")
>>>>>                  DW_AT_decl_line (3)
>>>>>                  DW_AT_type      (0x000000bb "char *")
>>>>>                  DW_AT_artificial        (true)
>>>>>                  DW_AT_external  (true)
>>>>>
>>>>> 0x0000006c:     DW_TAG_formal_parameter
>>>>>                    DW_AT_location        (DW_OP_reg5 RDI)
>>>>>                    DW_AT_decl_file       ("/home/yhs/tests/sig-
>>>>> change/deadarg/test.c")
>>>>>                    DW_AT_decl_line       (3)
>>>>>                    DW_AT_type    (0x000000c4 "t *")
>>>>>
>>>>> 0x00000075:     DW_TAG_formal_parameter
>>>>>                    DW_AT_location        (DW_OP_reg4 RSI)
>>>>>                    DW_AT_decl_file       ("/home/yhs/tests/sig-
>>>>> change/deadarg/test.c")
>>>>>                    DW_AT_decl_line       (3)
>>>>>                    DW_AT_type    (0x000000c4 "t *")
>>>>>
>>>>> 0x0000007e:     DW_TAG_inlined_subroutine
>>>>>                    DW_AT_abstract_origin (0x0000009a "foo")
>>>>>                    DW_AT_low_pc  (0x0000000000000010)
>>>>>                    DW_AT_high_pc (0x0000000000000015)
>>>>>                    DW_AT_call_file       ("/home/yhs/tests/sig-
>>>>> change/deadarg/test.c")
>>>>>                    DW_AT_call_line       (0)
>>>>>
>>>>> 0x0000008a:       DW_TAG_formal_parameter
>>>>>                      DW_AT_location      (DW_OP_reg5 RDI)
>>>>>                      DW_AT_abstract_origin       (0x000000a2 "a")
>>>>>
>>>>> 0x00000091:       DW_TAG_formal_parameter
>>>>>                      DW_AT_location      (DW_OP_reg4 RSI)
>>>>>                      DW_AT_abstract_origin       (0x000000aa "d")
>>>>>
>>>>> 0x00000098:       NULL
>>>>>
>>>>> 0x00000099:     NULL
>>>>>
>>>>> 0x0000009a:   DW_TAG_subprogram
>>>>>                  DW_AT_name      ("foo")
>>>>>                  DW_AT_decl_file ("/home/yhs/tests/sig-change/
>>>>> deadarg/test.c")
>>>>>                  DW_AT_decl_line (3)
>>>>>                  DW_AT_prototyped        (true)
>>>>>                  DW_AT_type      (0x000000bb "char *")
>>>>>                  DW_AT_inline    (DW_INL_inlined)
>>>>>
>>>>> 0x000000a2:     DW_TAG_formal_parameter
>>>>>                    DW_AT_name    ("a")
>>>>>                    DW_AT_decl_file       ("/home/yhs/tests/sig-
>>>>> change/deadarg/test.c")
>>>>>                    DW_AT_decl_line       (3)
>>>>>                    DW_AT_type    (0x000000c4 "t *")
>>>>>
>>>>> 0x000000aa:     DW_TAG_formal_parameter
>>>>>                    DW_AT_name    ("d")
>>>>>                    DW_AT_decl_file       ("/home/yhs/tests/sig-
>>>>> change/deadarg/test.c")
>>>>>                    DW_AT_decl_line       (3)
>>>>>                    DW_AT_type    (0x000000c4 "t *")
>>>>>
>>>>> 0x000000b2:     DW_TAG_formal_parameter
>>>>>                    DW_AT_name    ("b")
>>>>>                    DW_AT_decl_file       ("/home/yhs/tests/sig-
>>>>> change/deadarg/test.c")
>>>>>                    DW_AT_decl_line       (3)
>>>>>                    DW_AT_type    (0x000000d8 "int")
>>>>>
>>>>> 0x000000ba:     NULL
>>>>> ====
>>>>>
>>>>> In the above, there are two subprograms with the same name 'foo'.
>>>>> Currently btf encoder will consider both functions as ELF functions.
>>>>> Since two subprograms have different signature, the funciton will
>>>>> be ignored.
>>>>>
>>>>> But actually, one of function 'foo' is marked as DW_INL_inlined which
>>>>> means
>>>>> we should not treat it as an elf funciton. The patch fixed this issue
>>>>> by filtering subprograms if the corresponding function__inlined() is
>>>>> true.
>>>>>
>>>>> This will fix the issue for [1]. But it should work fine without [1]
>>>>> too.
>>>>>
>>>>>    [1] https://github.com/llvm/llvm-project/pull/157349
>>>> The change itself looks fine on the surface but it has some odd
>>>> consequences that we need to find a solution for.
>>>>
>>>> Specifically in CI I was seeing an error in BTF-to-DWARF function
>>>> comparison:
>>>>
>>>> https://github.com/alan-maguire/dwarves/actions/runs/18376819644/
>>>> job/52352757287#step:7:40
>>>>
>>>> 1: Validation of BTF encoding of functions; this may take some time:
>>>> ERROR: mismatch : BTF '__be32 ip6_make_flowlabel(struct net *, struct
>>>> sk_buff *, __be32, struct flowi6 *, bool);' not found; DWARF ''
>>>>
>>>> Further investigation reveals the problem; there is a constprop variant
>>>> of ip6_make_flowlabel():
>>>>
>>>> ffffffff81ecf390 t ip6_make_flowlabel.constprop.0
>>>>
>>>> ..and the problem is it has a different function signature:
>>>>
>>>> __be32 ip6_make_flowlabel(struct net *, struct sk_buff *, __be32, struct
>>>> flowi6 *, bool);
>>>>
>>>> The "real" function (that was inlined, other than the constprop variant)
>>>> looks like this:
>>>>
>>>> static inline __be32 ip6_make_flowlabel(struct net *net, struct sk_buff
>>>> *skb,
>>>>                       __be32 flowlabel, bool autolabel,
>>>>                       struct flowi6 *fl6);
>>>>
>>>> i.e. the last two parameters are in a different order.
>>>
>>> It is interesting that gcc optimization may change parameter orders...
>>>
>>
>> Yeah, I'm checking into this because I sort of wonder if it's a bug in
>> pahole processing and that the bool was in fact constant-propagated and
>> the struct fl6 * was actually the last ip6_make_flowlabel.constprop
>> parameter. Might be an issue in how we handle abstract origin cases.
> 
> Yeah, I think most likely 'autolabel' was const-propagated and *fl6 is
> the last real arg as you suggest.
> 
> I'm not an expert on the IPA optimization passes, but I don't know of
> any that would reorder parameters like that. 
> 
> OTOH, I see a few places in kernel sources where ip6_make_flowlabel is
> passed a simple 'true' for autolabel.  That sort of thing will almost
> certainly be optimized by the IPA-cprop pass.
> 
> Note that you may have _both_ the "real" version and the .constprop
> version of the function.  IPA-cprop can create specialized versions
> of functions so places where a parameter is a known constant can use
> the .constprop version (where 'autolabel' has been dropped) while
> other places where it may be variable use the original.
> 
> IPA-SRA (.isra suffix) can also change function parameters and return
> values, but afaiu it does not reorder existing parameters.
> 

Thanks for the additional info!

Looking at the specific case, here's one instance of the inlined
function's representation:

 <1><be25126>: Abbrev Number: 35 (DW_TAG_subprogram)
    <be25127>   DW_AT_name        : (indirect string, offset: 0x3bce6f):
ip6_make_flowlabel
    <be2512b>   DW_AT_decl_file   : 3
    <be2512c>   DW_AT_decl_line   : 952
    <be2512e>   DW_AT_decl_column : 22
    <be2512f>   DW_AT_prototyped  : 1
    <be2512f>   DW_AT_type        : <0xbdef11c>
    <be25133>   DW_AT_inline      : 3   (declared as inline and inlined)
    <be25134>   DW_AT_sibling     : <0xbe25187>
 <2><be25138>: Abbrev Number: 20 (DW_TAG_formal_parameter)
    <be25139>   DW_AT_name        : net
    <be2513d>   DW_AT_decl_file   : 3
    <be2513e>   DW_AT_decl_line   : 952
    <be25140>   DW_AT_decl_column : 53
    <be25141>   DW_AT_type        : <0xbe019b0>
 <2><be25145>: Abbrev Number: 20 (DW_TAG_formal_parameter)
    <be25146>   DW_AT_name        : skb
    <be2514a>   DW_AT_decl_file   : 3
    <be2514b>   DW_AT_decl_line   : 952
    <be2514d>   DW_AT_decl_column : 74
    <be2514e>   DW_AT_type        : <0xbdfd253>
 <2><be25152>: Abbrev Number: 40 (DW_TAG_formal_parameter)
    <be25153>   DW_AT_name        : (indirect string, offset: 0x10853):
flowlabel
    <be25157>   DW_AT_decl_file   : 3
    <be25158>   DW_AT_decl_line   : 953
    <be2515a>   DW_AT_decl_column : 13
    <be2515b>   DW_AT_type        : <0xbdef11c>
 <2><be2515f>: Abbrev Number: 40 (DW_TAG_formal_parameter)
    <be25160>   DW_AT_name        : (indirect string, offset: 0x3bcc9e):
autolabel
    <be25164>   DW_AT_decl_file   : 3
    <be25165>   DW_AT_decl_line   : 953
    <be25167>   DW_AT_decl_column : 29
    <be25168>   DW_AT_type        : <0xbdef194>
 <2><be2516c>: Abbrev Number: 20 (DW_TAG_formal_parameter)
    <be2516d>   DW_AT_name        : fl6
    <be25171>   DW_AT_decl_file   : 3
    <be25172>   DW_AT_decl_line   : 954
    <be25174>   DW_AT_decl_column : 21
    <be25175>   DW_AT_type        : <0xbe100ac>


And here's the abstract origin reference to it which I believe causes
the trouble:

 <1><be2708c>: Abbrev Number: 205 (DW_TAG_subprogram)
    <be2708e>   DW_AT_abstract_origin: <0xbe25126>
    <be27092>   DW_AT_low_pc      : 0xffffffff81ecf390
    <be2709a>   DW_AT_high_pc     : 0xa2
    <be270a2>   DW_AT_frame_base  : 1 byte block: 9c
(DW_OP_call_frame_cfa)
    <be270a4>   DW_AT_call_all_calls: 1
    <be270a4>   DW_AT_sibling     : <0xbe27268>
 <2><be270a8>: Abbrev Number: 7 (DW_TAG_formal_parameter)
    <be270a9>   DW_AT_abstract_origin: <0xbe25138>
    <be270ad>   DW_AT_location    : 0x18ed328 (location list)
    <be270b1>   DW_AT_GNU_locviews: 0x18ed31c
 <2><be270b5>: Abbrev Number: 7 (DW_TAG_formal_parameter)
    <be270b6>   DW_AT_abstract_origin: <0xbe25145>
    <be270ba>   DW_AT_location    : 0x18ed363 (location list)
    <be270be>   DW_AT_GNU_locviews: 0x18ed359
 <2><be270c2>: Abbrev Number: 7 (DW_TAG_formal_parameter)
    <be270c3>   DW_AT_abstract_origin: <0xbe25152>
    <be270c7>   DW_AT_location    : 0x18ed399 (location list)
    <be270cb>   DW_AT_GNU_locviews: 0x18ed38f
 <2><be270cf>: Abbrev Number: 7 (DW_TAG_formal_parameter)
    <be270d0>   DW_AT_abstract_origin: <0xbe2516c>
    <be270d4>   DW_AT_location    : 0x18ed3cb (location list)
    <be270d8>   DW_AT_GNU_locviews: 0x18ed3c3
 <2><be270dc>: Abbrev Number: 16 (DW_TAG_variable)
    <be270dd>   DW_AT_abstract_origin: <0xbe25179>
    <be270e1>   DW_AT_location    : 0x18ed3f6 (location list)
    <be270e5>   DW_AT_GNU_locviews: 0x18ed3f0
 <2><be270e9>: Abbrev Number: 55 (DW_TAG_formal_parameter)
    <be270ea>   DW_AT_abstract_origin: <0xbe2515f>

So what you see above is two things. First the order of parameters is
not preserved; specifically the original function and inlined function
representation it is

net, skb, flowlabel, autolabel, fl6

...while the non-inlined references via abstract origin has order

net, skb, flowlabel, fl6, and finally autolabel (with a DW_TAG_variable
inbetween).

And secondly what's interesting here is that the other parameters all
specify locations while autolabel does not.

The problem we have is that

1. pahole does not attach any significance to reordering like this and
does not detect it as far as I can see (I've also observed similar
patterns in inline site representations where order differs from the
original abstract origin function)

2. pahole also does not enforce the need for location info for a
parameter (implicit assumption being that if no location is present it
is in the usual calling-convention-dictated place)

The combination of 1 and 2 leads to the problem observed.

The DWARF spec appears to mandate source code order for parameters but I
couldn't find any equivalent mention of abstract origin parameter
references.

From the above empirical case and others it _seems_ like the ordering
_is_ meaningful in cases like this. How we extract that meaning without
breaking other things is always the challenge though.

I've started experimenting with detecting location misordering in
abstract origin references in the work-in-progress location code since
it does more extensive parameter location handling. There are a fair few
instances of misordering detected, especially for inline expansions it
seems (likely due to more frequent argument omissions at inline sites).
I'm hoping detecting misordering combined with enforcing location info
for misordered cases might be enough to detect and handle cases like
this, but as always the worry is other stuff gets broken as a
consequence. I'll report back when I have more data.

Alan