[LSF/MM/BPF TOPIC] Namespace-aware upcalls from kernel filesystems

[LSF/MM/BPF TOPIC] Namespace-aware upcalls from kernel filesystems

[Shyam Prasad N]

Kernel filesystems sometimes need to upcall to userspace to get some
work done, which cannot be achieved in kernel code (or rather it is
better to be done in userspace). Some examples are DNS resolutions,
user authentication, ID mapping etc.

Filesystems like SMB and NFS clients use the kernel keys subsystem for
some of these, which has an upcall facility that can exec a binary in
userspace. However, this upcall mechanism is not namespace aware and
upcalls to the host namespaces (namespaces of the init process).

This can be an inconvenience or a blocker for container services,
which run most code from containers and do not like to host any
binaries in the host namespace. They now need to host an upcall
handler in the host namespace, which can switch to the appropriate
namespaces based on the parameters sent before getting the work done.

I tried to prototype a namespace aware upcall mechanism for kernel keys here:
https://www.spinics.net/lists/keyrings/msg17581.html
But it has not been successful so far. I'm seeking reviews on this
approach from security point of view.

Another option that I could think of is to host a device file in
devfs. The mount could register with keys subsystem by keeping an FD
open from inside a container. The keys subsystem could then upcall on
the "right" FD based on some parameter supplied to it.

Looking forward to hearing if there is a better approach to solving
this problem.

-- 
Regards,
Shyam

Re: [LSF/MM/BPF TOPIC] Namespace-aware upcalls from kernel filesystems

[Chuck Lever]

On Sat, Feb 14, 2026, at 5:06 AM, Shyam Prasad N wrote:
> Kernel filesystems sometimes need to upcall to userspace to get some
> work done, which cannot be achieved in kernel code (or rather it is
> better to be done in userspace). Some examples are DNS resolutions,
> user authentication, ID mapping etc.
>
> Filesystems like SMB and NFS clients use the kernel keys subsystem for
> some of these, which has an upcall facility that can exec a binary in
> userspace. However, this upcall mechanism is not namespace aware and
> upcalls to the host namespaces (namespaces of the init process).

Hello Shyam, we've been introducing netlink control interfaces, which
are namespace-aware. The kernel TLS handshake mechanism now uses
this approach, as does the new NFSD netlink protocol.


-- 
Chuck Lever

Re: [LSF/MM/BPF TOPIC] Namespace-aware upcalls from kernel filesystems

[Shyam Prasad N]

On Sat, Feb 14, 2026 at 9:10 PM Chuck Lever <cel@kernel.org> wrote:
>
>
> On Sat, Feb 14, 2026, at 5:06 AM, Shyam Prasad N wrote:
> > Kernel filesystems sometimes need to upcall to userspace to get some
> > work done, which cannot be achieved in kernel code (or rather it is
> > better to be done in userspace). Some examples are DNS resolutions,
> > user authentication, ID mapping etc.
> >
> > Filesystems like SMB and NFS clients use the kernel keys subsystem for
> > some of these, which has an upcall facility that can exec a binary in
> > userspace. However, this upcall mechanism is not namespace aware and
> > upcalls to the host namespaces (namespaces of the init process).
>
> Hello Shyam, we've been introducing netlink control interfaces, which
> are namespace-aware. The kernel TLS handshake mechanism now uses
> this approach, as does the new NFSD netlink protocol.
>
>
> --
> Chuck Lever

Hi Chuck,

Interesting. Let me explore this a bit more.
I'm assuming that this is the file that I should be looking into:
fs/nfsd/nfsctl.c
And that there would be a corresponding handler in nfs-utils?

-- 
Regards,
Shyam

Re: [LSF/MM/BPF TOPIC] Namespace-aware upcalls from kernel filesystems

[Chuck Lever]

On Mon, Feb 16, 2026, at 11:14 PM, Shyam Prasad N wrote:
> On Sat, Feb 14, 2026 at 9:10 PM Chuck Lever <cel@kernel.org> wrote:
>>
>>
>> On Sat, Feb 14, 2026, at 5:06 AM, Shyam Prasad N wrote:
>> > Kernel filesystems sometimes need to upcall to userspace to get some
>> > work done, which cannot be achieved in kernel code (or rather it is
>> > better to be done in userspace). Some examples are DNS resolutions,
>> > user authentication, ID mapping etc.
>> >
>> > Filesystems like SMB and NFS clients use the kernel keys subsystem for
>> > some of these, which has an upcall facility that can exec a binary in
>> > userspace. However, this upcall mechanism is not namespace aware and
>> > upcalls to the host namespaces (namespaces of the init process).
>>
>> Hello Shyam, we've been introducing netlink control interfaces, which
>> are namespace-aware. The kernel TLS handshake mechanism now uses
>> this approach, as does the new NFSD netlink protocol.
>>
>>
>> --
>> Chuck Lever
>
> Hi Chuck,
>
> Interesting. Let me explore this a bit more.
> I'm assuming that this is the file that I should be looking into:
> fs/nfsd/nfsctl.c

Yes, clustered towards the end of the file. NFSD's use of netlink
is as a downcall-style administrative control plane.

net/handshake/netlink.c uses netlink as an upcall for driving
kernel-initiated TLS handshake requests up to a user daemon. This
mechanism has been adopted by NFSD, the NFS client, and the NVMe
over TCP drivers. An in-kernel QUIC implementation is planned and
will also be using this.


> And that there would be a corresponding handler in nfs-utils?

For NFSD, nfs-utils has a new tool called nfsdctl.

The TLS handshake user space components are in ktls-utils. See:
https://github.com/oracle/ktls-utils

-- 
Chuck Lever

Re: [LSF/MM/BPF TOPIC] Namespace-aware upcalls from kernel filesystems

[Jeff Layton]

On Tue, 2026-02-17 at 09:21 -0500, Chuck Lever wrote:
> 
> On Mon, Feb 16, 2026, at 11:14 PM, Shyam Prasad N wrote:
> > On Sat, Feb 14, 2026 at 9:10 PM Chuck Lever <cel@kernel.org> wrote:
> > > 
> > > 
> > > On Sat, Feb 14, 2026, at 5:06 AM, Shyam Prasad N wrote:
> > > > Kernel filesystems sometimes need to upcall to userspace to get some
> > > > work done, which cannot be achieved in kernel code (or rather it is
> > > > better to be done in userspace). Some examples are DNS resolutions,
> > > > user authentication, ID mapping etc.
> > > > 
> > > > Filesystems like SMB and NFS clients use the kernel keys subsystem for
> > > > some of these, which has an upcall facility that can exec a binary in
> > > > userspace. However, this upcall mechanism is not namespace aware and
> > > > upcalls to the host namespaces (namespaces of the init process).
> > > 
> > > Hello Shyam, we've been introducing netlink control interfaces, which
> > > are namespace-aware. The kernel TLS handshake mechanism now uses
> > > this approach, as does the new NFSD netlink protocol.
> > > 
> > > 
> > > --
> > > Chuck Lever
> > 
> > Hi Chuck,
> > 
> > Interesting. Let me explore this a bit more.
> > I'm assuming that this is the file that I should be looking into:
> > fs/nfsd/nfsctl.c
> 
> Yes, clustered towards the end of the file. NFSD's use of netlink
> is as a downcall-style administrative control plane.
> 
> net/handshake/netlink.c uses netlink as an upcall for driving
> kernel-initiated TLS handshake requests up to a user daemon. This
> mechanism has been adopted by NFSD, the NFS client, and the NVMe
> over TCP drivers. An in-kernel QUIC implementation is planned and
> will also be using this.
> 
> 
> > And that there would be a corresponding handler in nfs-utils?
> 
> For NFSD, nfs-utils has a new tool called nfsdctl.
> 
> The TLS handshake user space components are in ktls-utils. See:
> https://github.com/oracle/ktls-utils


I think the consensus at this point is to move away from usermodehelper
as an upcall mechanism. The Linux kernel lacks a container object that
allows you to associate namespaces with one another, so you need an
already-running userspace process to do that association in userland.

netlink upcalls are bound to a network namespace. That works in the
above examples because they are also bound to a network namespace.
netlink upcalls require a running daemon in that namespace, which is
what ties that network namespace to other sorts of namespaces.

So, a related discussion we should have is whether and how we should
deprecate the old usermodehelper upcalls, given that they are
problematic in this way. 
-- 
Jeff Layton <jlayton@kernel.org>

Re: [LSF/MM/BPF TOPIC] Namespace-aware upcalls from kernel filesystems

[Shyam Prasad N]

On Tue, Feb 17, 2026 at 8:48 PM Jeff Layton <jlayton@kernel.org> wrote:
>
> On Tue, 2026-02-17 at 09:21 -0500, Chuck Lever wrote:
> >
> > On Mon, Feb 16, 2026, at 11:14 PM, Shyam Prasad N wrote:
> > > On Sat, Feb 14, 2026 at 9:10 PM Chuck Lever <cel@kernel.org> wrote:
> > > >
> > > >
> > > > On Sat, Feb 14, 2026, at 5:06 AM, Shyam Prasad N wrote:
> > > > > Kernel filesystems sometimes need to upcall to userspace to get some
> > > > > work done, which cannot be achieved in kernel code (or rather it is
> > > > > better to be done in userspace). Some examples are DNS resolutions,
> > > > > user authentication, ID mapping etc.
> > > > >
> > > > > Filesystems like SMB and NFS clients use the kernel keys subsystem for
> > > > > some of these, which has an upcall facility that can exec a binary in
> > > > > userspace. However, this upcall mechanism is not namespace aware and
> > > > > upcalls to the host namespaces (namespaces of the init process).
> > > >
> > > > Hello Shyam, we've been introducing netlink control interfaces, which
> > > > are namespace-aware. The kernel TLS handshake mechanism now uses
> > > > this approach, as does the new NFSD netlink protocol.
> > > >
> > > >
> > > > --
> > > > Chuck Lever
> > >
> > > Hi Chuck,
> > >
> > > Interesting. Let me explore this a bit more.
> > > I'm assuming that this is the file that I should be looking into:
> > > fs/nfsd/nfsctl.c
> >
> > Yes, clustered towards the end of the file. NFSD's use of netlink
> > is as a downcall-style administrative control plane.
> >
> > net/handshake/netlink.c uses netlink as an upcall for driving
> > kernel-initiated TLS handshake requests up to a user daemon. This
> > mechanism has been adopted by NFSD, the NFS client, and the NVMe
> > over TCP drivers. An in-kernel QUIC implementation is planned and
> > will also be using this.
> >
> >
> > > And that there would be a corresponding handler in nfs-utils?
> >
> > For NFSD, nfs-utils has a new tool called nfsdctl.
> >
> > The TLS handshake user space components are in ktls-utils. See:
> > https://github.com/oracle/ktls-utils
>
>
> I think the consensus at this point is to move away from usermodehelper
> as an upcall mechanism. The Linux kernel lacks a container object that
> allows you to associate namespaces with one another, so you need an
> already-running userspace process to do that association in userland.
>
> netlink upcalls are bound to a network namespace. That works in the
> above examples because they are also bound to a network namespace.
> netlink upcalls require a running daemon in that namespace, which is
> what ties that network namespace to other sorts of namespaces.

As long as the "connection" is initiated from the userspace, I think
it should be aware of all namespaces (not just net namespace).
As you said, this will also mean that there's a need for a daemon to
watch this fd.

>
> So, a related discussion we should have is whether and how we should
> deprecate the old usermodehelper upcalls, given that they are
> problematic in this way.

I see a few other users of UMH: coredump, initrd, kmod and kobject.
https://elixir.bootlin.com/linux/v6.19.3/C/ident/call_usermodehelper_setup
Based on the descriptions, most of these are used in early boot or are
done in subsystems that don't need namespace awareness.
To me, request_key is the odd one out here. I think we should have
upcall handlers in the different namespaces to watch for upcalls.

> --
> Jeff Layton <jlayton@kernel.org>



-- 
Regards,
Shyam

Re: [LSF/MM/BPF TOPIC] Namespace-aware upcalls from kernel filesystems

[Christian Brauner]

On Tue, Feb 17, 2026 at 10:18:39AM -0500, Jeff Layton wrote:
> On Tue, 2026-02-17 at 09:21 -0500, Chuck Lever wrote:
> > 
> > On Mon, Feb 16, 2026, at 11:14 PM, Shyam Prasad N wrote:
> > > On Sat, Feb 14, 2026 at 9:10 PM Chuck Lever <cel@kernel.org> wrote:
> > > > 
> > > > 
> > > > On Sat, Feb 14, 2026, at 5:06 AM, Shyam Prasad N wrote:
> > > > > Kernel filesystems sometimes need to upcall to userspace to get some
> > > > > work done, which cannot be achieved in kernel code (or rather it is
> > > > > better to be done in userspace). Some examples are DNS resolutions,
> > > > > user authentication, ID mapping etc.
> > > > > 
> > > > > Filesystems like SMB and NFS clients use the kernel keys subsystem for
> > > > > some of these, which has an upcall facility that can exec a binary in
> > > > > userspace. However, this upcall mechanism is not namespace aware and
> > > > > upcalls to the host namespaces (namespaces of the init process).
> > > > 
> > > > Hello Shyam, we've been introducing netlink control interfaces, which
> > > > are namespace-aware. The kernel TLS handshake mechanism now uses
> > > > this approach, as does the new NFSD netlink protocol.
> > > > 
> > > > 
> > > > --
> > > > Chuck Lever
> > > 
> > > Hi Chuck,
> > > 
> > > Interesting. Let me explore this a bit more.
> > > I'm assuming that this is the file that I should be looking into:
> > > fs/nfsd/nfsctl.c
> > 
> > Yes, clustered towards the end of the file. NFSD's use of netlink
> > is as a downcall-style administrative control plane.
> > 
> > net/handshake/netlink.c uses netlink as an upcall for driving
> > kernel-initiated TLS handshake requests up to a user daemon. This
> > mechanism has been adopted by NFSD, the NFS client, and the NVMe
> > over TCP drivers. An in-kernel QUIC implementation is planned and
> > will also be using this.
> > 
> > 
> > > And that there would be a corresponding handler in nfs-utils?
> > 
> > For NFSD, nfs-utils has a new tool called nfsdctl.
> > 
> > The TLS handshake user space components are in ktls-utils. See:
> > https://github.com/oracle/ktls-utils
> 
> 
> I think the consensus at this point is to move away from usermodehelper
> as an upcall mechanism. The Linux kernel lacks a container object that
> allows you to associate namespaces with one another, so you need an
> already-running userspace process to do that association in userland.
> 
> netlink upcalls are bound to a network namespace. That works in the
> above examples because they are also bound to a network namespace.
> netlink upcalls require a running daemon in that namespace, which is
> what ties that network namespace to other sorts of namespaces.
> 
> So, a related discussion we should have is whether and how we should
> deprecate the old usermodehelper upcalls, given that they are
> problematic in this way. 

Yes, I want usermodehelpers to eventually go away with the very few
legitimate cases being ideally replaced by user workers I added a few
years ago. User workers are spawned by the kernel but are actual
children of the caller and inherit its settings just like a fork()
would.

They're not just problematic because they're namespace unaware. That's
even something we could fix, I'm sure. It's that they run with full
kernel privileges. As evidenced by the old coredump usermodehelper this
is just an invitation for security issues. It's also completely opaque
to userspace when a process suddenly pops into the process table that is
not a child and more or less out of its control that then executes
binaries placed in some location in userspace. The mechanism just
doesn't cut it anymore imho.

There's a couple of avenues we have here to redo this so having that
discussion might be useful.

Re: [LSF/MM/BPF TOPIC] Namespace-aware upcalls from kernel filesystems

[Shyam Prasad N]

On Tue, Feb 17, 2026 at 7:51 PM Chuck Lever <cel@kernel.org> wrote:
>
>
>
> On Mon, Feb 16, 2026, at 11:14 PM, Shyam Prasad N wrote:
> > On Sat, Feb 14, 2026 at 9:10 PM Chuck Lever <cel@kernel.org> wrote:
> >>
> >>
> >> On Sat, Feb 14, 2026, at 5:06 AM, Shyam Prasad N wrote:
> >> > Kernel filesystems sometimes need to upcall to userspace to get some
> >> > work done, which cannot be achieved in kernel code (or rather it is
> >> > better to be done in userspace). Some examples are DNS resolutions,
> >> > user authentication, ID mapping etc.
> >> >
> >> > Filesystems like SMB and NFS clients use the kernel keys subsystem for
> >> > some of these, which has an upcall facility that can exec a binary in
> >> > userspace. However, this upcall mechanism is not namespace aware and
> >> > upcalls to the host namespaces (namespaces of the init process).
> >>
> >> Hello Shyam, we've been introducing netlink control interfaces, which
> >> are namespace-aware. The kernel TLS handshake mechanism now uses
> >> this approach, as does the new NFSD netlink protocol.
> >>
> >>
> >> --
> >> Chuck Lever
> >
> > Hi Chuck,
> >
> > Interesting. Let me explore this a bit more.
> > I'm assuming that this is the file that I should be looking into:
> > fs/nfsd/nfsctl.c
>
> Yes, clustered towards the end of the file. NFSD's use of netlink
> is as a downcall-style administrative control plane.
>
> net/handshake/netlink.c uses netlink as an upcall for driving
> kernel-initiated TLS handshake requests up to a user daemon. This
> mechanism has been adopted by NFSD, the NFS client, and the NVMe
> over TCP drivers. An in-kernel QUIC implementation is planned and
> will also be using this.
>
>
> > And that there would be a corresponding handler in nfs-utils?
>
> For NFSD, nfs-utils has a new tool called nfsdctl.
>
> The TLS handshake user space components are in ktls-utils. See:
> https://github.com/oracle/ktls-utils
>
> --
> Chuck Lever

Thanks Chuck. Will explore this in more detail.

-- 
Regards,
Shyam

Re: [LSF/MM/BPF TOPIC] Namespace-aware upcalls from kernel filesystems

[David Leadbeater]

On Sat, Feb 14, 2026 at 03:36:22PM +0530, Shyam Prasad N wrote:
> I tried to prototype a namespace aware upcall mechanism for kernel keys here:
> https://www.spinics.net/lists/keyrings/msg17581.html
> But it has not been successful so far. I'm seeking reviews on this
> approach from security point of view.

I have more context from the containers side, but to me this doesn't
appear safe. Entering the right namespaces isn't enough to safely run
code within a container. The container runtime may have set up seccomp
or other limits which this upcall won't respect.

I would like to see a solution to this though, we currently have custom
callback code to make this work. I'm not familiar enough with the
interfaces but an approach where something registers also seems
desirable because it is able to preserve backwards compatibility, which
changing the namespace the upcall runs in doesn't.

David

Re: [LSF/MM/BPF TOPIC] Namespace-aware upcalls from kernel filesystems

[Shyam Prasad N]

On Mon, Feb 16, 2026 at 6:25 AM David Leadbeater <dgl@dgl.cx> wrote:
>
> On Sat, Feb 14, 2026 at 03:36:22PM +0530, Shyam Prasad N wrote:
> > I tried to prototype a namespace aware upcall mechanism for kernel keys here:
> > https://www.spinics.net/lists/keyrings/msg17581.html
> > But it has not been successful so far. I'm seeking reviews on this
> > approach from security point of view.
>
> I have more context from the containers side, but to me this doesn't
> appear safe. Entering the right namespaces isn't enough to safely run
> code within a container. The container runtime may have set up seccomp
> or other limits which this upcall won't respect.

Hi David,
Thanks for these comments.
Let me look into seccomp to see if kernel will have any visibility into it.

>
> I would like to see a solution to this though, we currently have custom
> callback code to make this work. I'm not familiar enough with the
> interfaces but an approach where something registers also seems
> desirable because it is able to preserve backwards compatibility, which
> changing the namespace the upcall runs in doesn't.

Ack. Will explore the options and get back to this thread.

>
> David
>


-- 
Regards,
Shyam