Re: [PATCH 0/3] elf: load the main program from AT_EXECFD

[Florian Weimer <fweimer@redhat.com>]

Summary for kernel list: We are looking for ways to make executables not
loaded by the kernel more compatible with the rest of the system.  That
includes proper /proc/self/exe and auxv values.  The discussion was
triggered by Christian's binfmt_misc patch, which happens to share many
of the problems when /usr/bin/ld.so is used to load programs (instead of
the kernel).

* Christian Brauner:

> On 2026-07-16 17:37:21+02:00, Florian Weimer wrote:
>> * Christian Brauner:
>> 
>> >> What does /proc/self/exe look like for such processes?  Does GDB work?
>> >
>> > Right, I checked that.
>> >
>> > /proc/self/exe is ld.so. The kernel exec'd ld.so, so it names ld.so
>> > whether the program arrived via AT_EXECFD, via --program-fd, or via a
>> > plain "ld.so PROG" command line.
>> 
>> Yeah, and that causes problems with binaries that try to be relocatable.
>> We don't have a very convenient way to get the correct path in those
>> scenarios, so a lot of code uses /proc/self/exe instead.
>> 
>> I'm wondering if we can use the checkpoint-restore facilities (the
>> restore parts) to fix this: load ld.so another time, transfer control to
>> it, and instruct it to unmap the first copy and then invoke the
>> necessary prctls to make the process look exactly like a directly
>> invoked process.
>
> At first I was very confused about this proposal but I think I
> understand what you are after now. Your point is that we shouldn't just
> fix argv like in my proposal but actually even fix the exe file and
> remap.

Not sure about remap.  But I think to turn this into an it-just-works
solution, we need to fix /proc/self/exe and the auxiliary vector.

> So I think doing this purely in userspace isn't doable. The restore
> parts can fix everything except the exe link. PR_SET_MM_MAP requires
> capabilities in the relevant user namespace for the exe file. That makes
> it pretty useless for us. And dropping that capability requirement isn't
> feasible, I think. LSMs and audit trust the exe link, so an uncapped
> exe file would let any process masquerade as an arbitrary executable.

Is the latter really a problem?

Today, it's possible to stop a process that is SUID on disk before it
runs any code: 

$ ls -l /proc/163710/exe
lrwxrwxrwx. 1 fweimer fweimer 0 Jul 17 10:11 /proc/163710/exe -> /usr/bin/su

(gdb) info thread
  Id   Target Id           Frame 
* 1    process 163710 "su" 0x00007f5ae2f24e40 in _start ()
   from /lib64/ld-linux-x86-64.so.2

This is with kernel.yama.ptrace_scope=1.  I assume this gives me full
control over a process that is nominally running /usr/bin/su.  Even
AT_SECURE is set to 1:

Breakpoint 2, main (argc=1, argv=0x7ffd77cfb2e8) at login-utils/su.c:5
5	{
(gdb) print __libc_enable_secure
$1 = 1
(gdb) print (int) getuid ()
$2 = 1000
(gdb) print (int) geteuid ()
$3 = 1000
(gdb) print (long) getauxval(23)
$4 = 1

Of course, the AT_SECURE transition did not actually happen, and the
process is running with the original user privileges.

> Everything else (the entire saved auxv, the start/end_code/data, brk and
> stack markers) is validated but requires no capability at all. So an
> unprivileged ld.so can already repair /proc/<pid>/auxv and the
> stat/statm code accounting, but never /proc/self/exe.

It's good for us if AT_SECURE does not need protecting.  We would like
to use a fake 1 value in our test suite.

> Unmapping the first copy is also not really feasible. Even with
> privilege making this work would be very ugly: The kernel's
> replace_mm_exe_file() refuses with -EBUSY while any vma still maps the
> old exe file. From my research, CRIU works around exactly this by
> copying its restorer blob into an anonymous mapping before it unmaps the
> old address space.

Unmapping the first copy was only my idea to make this work with the
existing kernel facilities.  It's not required for the binfmt_misc case
and most /usr/bin/ld.so scenarios, and actually drives up complexity
considerably.

> So I think the exe link should be fixed up at exec time.
> begin_new_exec() sets mm->exe_file to bprm->file which after the
> binfmt_misc handoff is the interpreter. But bprm->executable is the file
> the kernel access-checked and kept open for AT_EXECFD. We have it right
> there. This is the file that would_dump() uses for it's decision and
> binfmt_misc's 'C' flag derive credentials from.
>
> We simply need an extension to binfmt_misc that sets mm->exe_file to
> bprm->executable. Then it is correct from the start and there's no
> window and no privilege question and existing 'O'/'C' users (qemu-user
> registrations) are unaffected. It then raises AT_FLAGS_PRESERVE_ARGV.
>
> On the userspace side, ld.so now sees this AT_* flag and in response
> issues one uncapped PR_SET_MM_MAP (that's available completely
> unprivileged) to retarget AT_PHDR/AT_ENTRY/AT_BASE and drop the stale
> AT_EXECFD from saved_auxv.
>
> With both in place, attaching gdb to a dispatched process is fully
> correct and /proc/self/exe-based self-location works. As a bonus the
> program file gets the same exe_file write-denial a directly executed
> binary has. Today it is ld.so that gets pinned and the running program's
> file stays writable while its text is mapped.
>
> After this, only uninteresting differences should be left. This would be
> a follow-up series I'm happy to do. Does that sound reasonable?

The proposal looks quite straightforward.  It's a shame that it does not
fix the /usr/bin/ld.so case.  Is there anything we can do to address
that?

I wouldn't mind if we had a system call that triggered the kernel ELF
loader for the main executable.

Thanks,
Florian