Re: llvm bpf debug info. Re: [RFC PATCH v4 3/3] bpf: Introduce function for outputing data to perf event

[Alexei Starovoitov <ast@plumgrid.com>]

On 7/29/15 2:38 AM, He Kuang wrote:
> Hi, Alexei
>
> On 2015/7/28 10:18, Alexei Starovoitov wrote:
>> On 7/25/15 3:04 AM, He Kuang wrote:
>>> I noticed that for 64-bit elf format, the reloc sections have
>>> 'Addend' in the entry, but there's no 'Addend' info in bpf elf
>>> file(64bit). I think there must be something wrong in the process
>>> of .s -> .o, which related to 64bit/32bit. Anyway, we can parse out the
>>> AT_name now, DW_AT_LOCATION still missed and need your help.
>>
>> looks like objdump/llvm-dwarfdump can only read known EM,
>> but that that shouldn't be the problem for your dwarf reader right?
>> It should be able to recognize id-s of ELF::R_X86_64_* relo used right?
>> As far as AT_location for testprog.c it seems there is no info for
>> local variables because they were optimized away.
>> With -O0 I see AT_location being emitted.
>> Also line number info seems to be good in both cases.
>> But in our case, we don't need this anyway, no? we need to see
>> the types of structs mainly or you have some other use cases?
>> I think line number info would be great to correlate the error reported
>> by verifier into specific line in C.
>>
>
> Yes, without AT_location, we can lookup the user output data type
> by line number, but there're some issues when we look deep.
>
> There're two steps of work that should be done in user space,
> first we embed data type into bpf output record, then we use this
> type, or index or some other identifier to lookup the type from
> dwarf info, so we got a few plans.
>
> * Plan A. Use line number to identify the user data type
>
> Predefined macros:
>
>    #define DEFINE_BPF_OUTPUT_DATA(type,
> var)                               \
>            const int BPF_OUTPUT_LINE__##var = __LINE__; type var;
>
>    #define BPF_OUTPUT_TRACE_DATA(data, size)       \
>            __bpf_output_trace_data(BPF_OUTPUT_LINE__##data, &data, size)
>
> User defined BPF code:
>
>    struct user_define_struct {
>           ...
>    };
>
>    int testprog(int myvar_a, long myvar_b)
>    {
>            DEFINE_BPF_OUTPUT_DATA(struct user_define_struct, myvar_c);
>
>            BPF_OUTPUT_TRACE_DATA(myvar_c, sizeof(myvar_c));
>
>    ...
>
> We use macros to embed linenum implicitly, which leads an extra
> restriction that user should not define multiple variables in the
> same line and not split the macro over multiple lines, like this:
>
>    22 DEFINE_BPF_OUTPUT_DATA(struct xxtype, a);
> DEFINE_BPF_OUTPUT_DATA(struct xxtype, b);
>
> Or
>
>    22 DEFINE_BPF_OUTPUT_DATA(struct user_define_struct,
>    23                               myvar_c);
>
> DW_AT_decl_line = 22, while __LINE__ = 23
>
> So we should add verifier in the llvm BPF backend to warn on the
> above codes.
>
> * Plan B. Lookup variable type from dwarf AT_location info
>
> We can make use of the output data variable's address, for bpf is
> a minus offset to frame base. Then lookup matched offset from
> location info(e.g. "DW_OP_fbreg: -32") to identify the variable
> type.
>
> For getting the frame base address, we can use builtin functions
> like __builtin_frame_base() and __builtin_dwarf_cfa() which
> returns the call frame base address. Currently those builtin
> functions are not implemented in BPF lower operation yet, so we
> tested our bpf program by using a variable tag on frame base, as
> following:
>
>    struct user_define_struct {
>       ...
>    };
>
>    typedef struct {} frame_base_tag;
>
>    int testprog(void)
>    {
>        frame_base_tag BPF_FRAME_BASE;
>        struct user_define_struct myvar_a;
>
>        __bpf_trace_output_data((void *)&myvar_a - (void *)&BPF_FRAME_BASE,
>                                &myvar_a, sizeof(myvar_a));
>    ...
>
> The first argument of __bpf_trace_output_data() will be caculated
> and it's easy to traverse the variable DIEs in dwarf info and
> check each DW_AT_location attribute to find the corresponding
> variable type.
>
> The things let us worry about is the opimization may reuse the
> stack space which can cause different variables share the same
> address, by some rough tests that kind of optimization does not
> appear.
>
> * Comparison
>
> Plan A needs less effort and easy to implement, but requires more
> check to ensure user not use multiple definition in the same line
> and not use macro cross lines.
>
> The advantages of plan B is that we do not need introduce macros
> showed in above example and all the things are done implicitly,
> but the AT_location info is the prerequisite of this plan, I'm
> not sure whether we can guarantee this info in dwarf or not.
>
> Another way we can think of is adding new builtin functions to
> indicate the compilier to generate codes return the dwarf type
> index directly:
>
>    __bpf_trace_output_data(__builtin_dwarf_type(myvar_a), &myvar_a, size);
>

probably both A and B won't really work when programs get bigger
and optimizations will start moving lines around.
the builtin_dwarf_type idea is actually quite interesting.
Potentially that builtin can stringify type name and later we can
search it in dwarf. Please take a look how to add such builtin.
There are few similar builtins that deal with exception handling
and need type info. May be they can be reused. Like:
int_eh_typeid_for and int_eh_dwarf_cfa