Re: elfutils DWARF problem was: Re: Problem with BTF generation on mips64el

[Tony Ambardar <tony.ambardar@gmail.com>]

On Tue, Jun 11, 2024 at 03:07:29PM +0200, Mark Wielaard wrote:
> Hi,
> 
> Adding elfutils-devel to CC to keep everyone up to date on the state of
> the patches.
> 
> On Mon, 2024-06-10 at 23:36 -0700, Tony Ambardar wrote:
> > On Mon, Jun 03, 2024 at 08:47:24PM -0700, Tony Ambardar wrote:
> > > On Mon, Jun 03, 2024 at 09:18:33PM +0200, Mark Wielaard wrote:
> > > > On Mon, Jun 03, 2024 at 02:40:45PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > > Couldn't find a way to ask eu-readelf for more verbose output, where we
> > > > > could perhaps get some clue as to why it produces nothing while binutils
> > > > > readelf manages to grok it, Mark, do you know some other way to ask
> > > > > eu-readelf to produce more debug output?
> > > > > 
> > > > > I'm unsure if the netdevsim.ko file was left in a semi encoded BTF state
> > > > > that then made eu-readelf to not be able to process it while pahole,
> > > > > that uses eltuils' libraries, was able to process the first two CUs for
> > > > > a kernel module and all the CUs for the vmlinux file :-\
> > > > > 
> > > > > Mark, the whole thread is available at:
> > > > > 
> > > > > https://lore.kernel.org/all/Zl3Zp5r9m6X_i_J4@x1/T/#u
> > > > 
> > > > I haven't looked at the vmlinux file. But for the .ko file the issue
> > > > is that the elfutils MIPS backend isn't complete. Specifically MIPS
> > > > relocations aren't recognized (and so cannot be applied). There are
> > > > some pending patches which try to fix that:
> > > > 
> > > > https://patchwork.sourceware.org/project/elfutils/list/?series=31601
> > > 
> > > Earlier in the thread, Hengqi Chen pointed out the latest elfutils backend
> > > work for MIPS, and I locally rebuilt elfutils and then pahole from their
> > > respective next/main branches. For elfutils, main (935ee131cf7c) includes
> > > 
> > >   e259f126 Support Mips architecture
> > >   f2acb069 stack: Fix stack unwind failure on mips
> > >   db33cb0c backends: Add register_info, return_value_location, core_note mips
> > > 
> > > which partially applies the patchwork series but leaves out the support for
> > > readelf, strip, and elflint.
> > > 
> > > I believe this means the vmlinux and .ko files I shared are OK, or is there
> > > more backend work needed for MIPS?
> > > 
> > > The bits missing in eu-readelf would explain the blank output both Arnaldo
> > > and I see from "$ eu-readelf -winfo vmlinux". I tried rebuilding with the
> > > patchwork readelf patch locally but ran into merge conflicts.
> > 
> > A short update, starting with answering my own question.
> > 
> > No, apparently the above commits *do not* complete the backend work. Ying
> > Huang submitted additional related patches since March 5: [1][2]
> > 
> >     strip: Adapt src/strip -o -f on mips
> >     readelf: Adapt src/readelf -h/-S/-r/-w/-l/-d/-a on mips
> >     elflint: adapt src/elflint --gnu src/nm on mips
> >     test: Add mips in run-allregs.sh and run-readelf-mixed-corenote.sh
> > 
> > Despite the titles, these patches do include core backend changes for MIPS.
> > I resolved the various merge conflicts [3], rebuilt elfutils, and retested
> > kernel builds to now find:
> > 
> >   - pahole is able to read DWARF[45] info and create .BTF for modules
> >   - resolve_btfids can successfully patch .BTF_ids in modules
> >   - kernel successfully loads modules with BTF and kfuncs (tested 6.6 LTS)
> > 
> > Huzzah!
> > 
> > 
> > Ying:
> > 
> > Thank you for developing these MIPS patches. In your view, are the MIPS
> > changes now complete, or do you plan further updates that might improve or
> > impact parsing DWARF debug/reloc info in apps like pahole?
> > 
> > 
> > Mark:
> > 
> > Given that BTF usage on Linux/MIPS is basically broken without these
> > patches, could I request some of your review time for them to be merged? If
> > it's helpful, my branch [3] includes all patches with conflicts fixed, and
> > I also successfully ran the elfutils self-tests (including MIPS from Ying).
> > Please feel free to add for these patches:
> > 
> >     Tested-by: Tony Ambardar <Tony.Ambardar@gmail.com>
> 
> Yes, I would very much like to integrate the rest of these patches. But
> I keep running out of time. The main issues were that, as you noticed,
> the patches mix backend and frontend tool changes a bit. I don't have
> access to a MIPS system to test them on. There are a couple of
> different MIPS abis (I believe all combinations of 32/64 bit and
> big/little endianness), but people have only tested on mips64le (maybe
> that is the only relevant one these days?) And finally the way MIPS
> represents relocations is slightly different than any other ELF
> architecture does. So we have to translate that somewhere to make the
> standards functions work. I have to convince myself that doing that in
> elf_getdata as the patches do is the right place.
> 

Glad to hear there's strong interest. Not much I can add to the code
structure discussion but I *can* confirm my testing included both
mips64el and mips32be, specifically to improve word-size/endianness
coverage. The former is supported by Debian, while the latter can still
be found on many embedded devices like consumer routers. If you need
additional testing for review/merge I can help with that (using
OpenWrt/QEMU primarily).

> > Many thanks everyone for your help,
> > Tony
> > 
> > [1]: https://patchwork.sourceware.org/project/elfutils/list/?series=31601
> > [2]: https://patchwork.sourceware.org/project/elfutils/list/?series=34310
> > [3]:
> > https://github.com/guidosarducci/elfutils/commits/main-fix-mips-support-reloc/
>