[bug report] [regression?] bpf lsm breaks /proc/*/attr/current with security= on commandline

[bug report] [regression?] bpf lsm breaks /proc/*/attr/current with security= on commandline

[Filip Hejsek]

Hello,

TLDR: because of bpf lsm, putting security=selinux on commandline
      results in /proc/*/attr/current returning errors.

When the legacy security= commandline option is used, the specified lsm
is added to the end of the lsm list. For example, security=apparmor
results in the following order of security modules:

   capability,landlock,lockdown,yama,bpf,apparmor

In particular, the bpf lsm will be ordered before the chosen major lsm.

This causes reads and writes of /proc/*/attr/current to fail, because
the bpf hook overrides the apparmor/selinux hook.

As you can see in the code below, only the first registered hook is
called (when reading attr/current, lsmid is 0):

int security_getprocattr(struct task_struct *p, int lsmid, const char *name,
			 char **value)
{
	struct lsm_static_call *scall;

	lsm_for_each_hook(scall, getprocattr) {
		if (lsmid != 0 && lsmid != scall->hl->lsmid->id)
			continue;
		return scall->hl->hook.getprocattr(p, name, value);
	}
	return LSM_RET_DEFAULT(getprocattr);
}

Even though the bpf lsm doesn't allow attaching bpf programs to this
hook, it still prevents the other hooks from being called.

This is maybe a regression, because with the same commandline, reading
from /proc/*/attr/current probably worked before the introduction of
bpf lsm.

Regards,
Filip Hejsek

Re: [bug report] [regression?] bpf lsm breaks /proc/*/attr/current with security= on commandline

[Paul Moore]

On Sat, Sep 13, 2025 at 1:01 PM Filip Hejsek <filip.hejsek@gmail.com> wrote:
>
> Hello,
>
> TLDR: because of bpf lsm, putting security=selinux on commandline
>       results in /proc/*/attr/current returning errors.
>
> When the legacy security= commandline option is used, the specified lsm
> is added to the end of the lsm list. For example, security=apparmor
> results in the following order of security modules:
>
>    capability,landlock,lockdown,yama,bpf,apparmor
>
> In particular, the bpf lsm will be ordered before the chosen major lsm.
>
> This causes reads and writes of /proc/*/attr/current to fail, because
> the bpf hook overrides the apparmor/selinux hook.

What kernel are you using?  Things appear to work correctly on my
kernel that is tracking upstream (Fedora Rawhide + some unrelated
bits):

% uname -a
Linux dev-rawhide-1.lan 6.17.0-0.rc7.250923gd1ab3.57.1.secnext.fc44.x86_64 #1 SM
P PREEMPT_DYNAMIC Tue Sep 23 10:07:14 EDT 2025 x86_64 GNU/Linux
% cat /proc/cmdline
BOOT_IMAGE=(hd0,gpt4)/boot/vmlinuz-6.17.0-0.rc7.250923gd1ab3.57.1.secnext.fc44.x
86_64 root=UUID=285029fa-4431-45e9-af1b-298ab0caf16a ro console=ttyS0 mitigation
s=off security=selinux
% cat /sys/kernel/security/lsm; echo ""
lockdown,capability,yama,selinux,bpf,landlock,ipe,ima,evm
% id -Z
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
% cat /proc/self/attr/current; echo ""
unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

I even ran it against the LSM initialization rework that has been
proposed, but has not yet been accepted/merged, and that worked the
same as above.

Is this a distro kernel with a lot of "special" patches which aren't
present upstream?

-- 
paul-moore.com

Re: [bug report] [regression?] bpf lsm breaks /proc/*/attr/current with security= on commandline

[Filip Hejsek]

On Wed, 2025-09-24 at 17:24 -0400, Paul Moore wrote:
> On Sat, Sep 13, 2025 at 1:01 PM Filip Hejsek <filip.hejsek@gmail.com> wrote:
> > 
> > Hello,
> > 
> > TLDR: because of bpf lsm, putting security=selinux on commandline
> >       results in /proc/*/attr/current returning errors.
> > 
> > When the legacy security= commandline option is used, the specified lsm
> > is added to the end of the lsm list. For example, security=apparmor
> > results in the following order of security modules:
> > 
> >    capability,landlock,lockdown,yama,bpf,apparmor
> > 
> > In particular, the bpf lsm will be ordered before the chosen major lsm.
> > 
> > This causes reads and writes of /proc/*/attr/current to fail, because
> > the bpf hook overrides the apparmor/selinux hook.
> 
> What kernel are you using?

I'm using Arch Linux kernel, which is very close to mainline. I have
also tested my own build from git sources (I used a stripped down
config which I based on config from Arch). Example in QEMU:

$ qemu-system-x86_64 -nodefaults -accel kvm -cpu host -smp cpus=2 -m 1G -display none -kernel ~/git/linux/arch/x86/boot/bzImage -initrd ./initramfs.img -serial mon:stdio -append 'console=ttyS0 security=selinux'
:: mounting '' on real root
mount: /new_root: no valid filesystem type specified.
ERROR: Failed to mount '' on real root
You are now being dropped into an emergency shell.
sh: can't access tty; job control turned off
[rootfs ~]# uname -a
Linux archlinux 6.17.0-rc7-00020-gcec1e6e5d1ab #3 SMP PREEMPT_DYNAMIC Thu Sep 25 16:28:02 CEST 2025 x86_64 GNU/Linux
[rootfs ~]# mount -t securityfs securityfs /sys/kernel/security
[rootfs ~]# cat /proc/cmdline
console=ttyS0 security=selinux
[rootfs ~]# cat /sys/kernel/security/lsm; echo
capability,landlock,lockdown,yama,bpf,selinux
[rootfs ~]# cat /proc/self/attr/current
cat: read error: Invalid argument

(Note: In this example, uname reports archlinux, but that's only
because I based the config on Arch config, it's not actually an Arch
kernel.)

Maybe the different behavior is caused by a different config? You can
find the Arch config at [1]. Based on Fedora package sources, I think
their config has

   CONFIG_LSM="lockdown,yama,integrity,selinux,bpf,landlock,ipe"

while the Arch config has

   CONFIG_LSM="landlock,lockdown,yama,integrity,bpf"

.

[1]: https://gitlab.archlinux.org/archlinux/packaging/packages/linux/-/raw/main/config?ref_type=heads

>   Things appear to work correctly on my
> kernel that is tracking upstream (Fedora Rawhide + some unrelated
> bits):
> 
> % uname -a
> Linux dev-rawhide-1.lan 6.17.0-0.rc7.250923gd1ab3.57.1.secnext.fc44.x86_64 #1 SM
> P PREEMPT_DYNAMIC Tue Sep 23 10:07:14 EDT 2025 x86_64 GNU/Linux
> % cat /proc/cmdline
> BOOT_IMAGE=(hd0,gpt4)/boot/vmlinuz-6.17.0-0.rc7.250923gd1ab3.57.1.secnext.fc44.x
> 86_64 root=UUID=285029fa-4431-45e9-af1b-298ab0caf16a ro console=ttyS0 mitigation
> s=off security=selinux
> % cat /sys/kernel/security/lsm; echo ""
> lockdown,capability,yama,selinux,bpf,landlock,ipe,ima,evm
> % id -Z
> unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
> % cat /proc/self/attr/current; echo ""
> unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
> 
> I even ran it against the LSM initialization rework that has been
> proposed, but has not yet been accepted/merged, and that worked the
> same as above.
> 
> Is this a distro kernel with a lot of "special" patches which aren't
> present upstream?

Re: [bug report] [regression?] bpf lsm breaks /proc/*/attr/current with security= on commandline

[Paul Moore]

On Thu, Sep 25, 2025 at 10:56 AM Filip Hejsek <filip.hejsek@gmail.com> wrote:
> On Wed, 2025-09-24 at 17:24 -0400, Paul Moore wrote:
> > On Sat, Sep 13, 2025 at 1:01 PM Filip Hejsek <filip.hejsek@gmail.com> wrote:
> > >
> > > Hello,
> > >
> > > TLDR: because of bpf lsm, putting security=selinux on commandline
> > >       results in /proc/*/attr/current returning errors.
> > >
> > > When the legacy security= commandline option is used, the specified lsm
> > > is added to the end of the lsm list. For example, security=apparmor
> > > results in the following order of security modules:
> > >
> > >    capability,landlock,lockdown,yama,bpf,apparmor
> > >
> > > In particular, the bpf lsm will be ordered before the chosen major lsm.
> > >
> > > This causes reads and writes of /proc/*/attr/current to fail, because
> > > the bpf hook overrides the apparmor/selinux hook.
> >
> > What kernel are you using?
>
> I'm using Arch Linux kernel, which is very close to mainline. I have
> also tested my own build from git sources (I used a stripped down
> config which I based on config from Arch). Example in QEMU:
>
> $ qemu-system-x86_64 -nodefaults -accel kvm -cpu host -smp cpus=2 -m 1G -display none -kernel ~/git/linux/arch/x86/boot/bzImage -initrd ./initramfs.img -serial mon:stdio -append 'console=ttyS0 security=selinux'
> :: mounting '' on real root
> mount: /new_root: no valid filesystem type specified.
> ERROR: Failed to mount '' on real root
> You are now being dropped into an emergency shell.
> sh: can't access tty; job control turned off
> [rootfs ~]# uname -a
> Linux archlinux 6.17.0-rc7-00020-gcec1e6e5d1ab #3 SMP PREEMPT_DYNAMIC Thu Sep 25 16:28:02 CEST 2025 x86_64 GNU/Linux
> [rootfs ~]# mount -t securityfs securityfs /sys/kernel/security
> [rootfs ~]# cat /proc/cmdline
> console=ttyS0 security=selinux
> [rootfs ~]# cat /sys/kernel/security/lsm; echo
> capability,landlock,lockdown,yama,bpf,selinux
> [rootfs ~]# cat /proc/self/attr/current
> cat: read error: Invalid argument
>
> (Note: In this example, uname reports archlinux, but that's only
> because I based the config on Arch config, it's not actually an Arch
> kernel.)
>
> Maybe the different behavior is caused by a different config? You can
> find the Arch config at [1]. Based on Fedora package sources, I think
> their config has
>
>    CONFIG_LSM="lockdown,yama,integrity,selinux,bpf,landlock,ipe"
>
> while the Arch config has
>
>    CONFIG_LSM="landlock,lockdown,yama,integrity,bpf"

That's interesting, you're running a LSM that isn't normally run in
your distro and you're not properly initializing it (no policy load).
Both are acceptable, but you're definitely operating in the
corner-iest of corner cases ;)

I'd have to look at the relevant code, but I suspect that with
"selinux" missing from the CONFIG_LSM list and you manually specifying
it on the kernel command line with "security=selinux" you are getting
it placed at the very end as opposed to what I saw (I have "selinux"
in my CONFIG_LSM list).  It's further complicated by the fact that the
procfs call into the LSM's security_getprocattr() hook is going to
pass a 0/zero into the hook as the @lsmid which means "first
available".

Considering that the "security=" parameter is a legacy option, I'd
encourage you to try the "lsm=" parameter (make sure you specify the
full list of LSMs you want, in order) to see if that works.  The
"security=" option predates both the concept of multiple simultaneous
LSMs as well as the uniqueness that is the BPF LSM.  Assuming that
"lsm=" works for you, and I would expect it to work, I think that is
the right solution here; new or unusual systems really shouldn't be
using "security=" at this point.

-- 
paul-moore.com

Re: [bug report] [regression?] bpf lsm breaks /proc/*/attr/current with security= on commandline

[Filip Hejsek]

On Thu, 2025-09-25 at 11:28 -0400, Paul Moore wrote:
> On Thu, Sep 25, 2025 at 10:56 AM Filip Hejsek <filip.hejsek@gmail.com> wrote:
> > On Wed, 2025-09-24 at 17:24 -0400, Paul Moore wrote:
> > > On Sat, Sep 13, 2025 at 1:01 PM Filip Hejsek <filip.hejsek@gmail.com> wrote:
> > > > 
> > > > Hello,
> > > > 
> > > > TLDR: because of bpf lsm, putting security=selinux on commandline
> > > >       results in /proc/*/attr/current returning errors.
> > > > 
> > > > When the legacy security= commandline option is used, the specified lsm
> > > > is added to the end of the lsm list. For example, security=apparmor
> > > > results in the following order of security modules:
> > > > 
> > > >    capability,landlock,lockdown,yama,bpf,apparmor
> > > > 
> > > > In particular, the bpf lsm will be ordered before the chosen major lsm.
> > > > 
> > > > This causes reads and writes of /proc/*/attr/current to fail, because
> > > > the bpf hook overrides the apparmor/selinux hook.
> > > 
> > > What kernel are you using?
> > 
> > I'm using Arch Linux kernel, which is very close to mainline. I have
> > also tested my own build from git sources (I used a stripped down
> > config which I based on config from Arch). Example in QEMU:
> > 
> > $ qemu-system-x86_64 -nodefaults -accel kvm -cpu host -smp cpus=2 -m 1G -display none -kernel ~/git/linux/arch/x86/boot/bzImage -initrd ./initramfs.img -serial mon:stdio -append 'console=ttyS0 security=selinux'
> > :: mounting '' on real root
> > mount: /new_root: no valid filesystem type specified.
> > ERROR: Failed to mount '' on real root
> > You are now being dropped into an emergency shell.
> > sh: can't access tty; job control turned off
> > [rootfs ~]# uname -a
> > Linux archlinux 6.17.0-rc7-00020-gcec1e6e5d1ab #3 SMP PREEMPT_DYNAMIC Thu Sep 25 16:28:02 CEST 2025 x86_64 GNU/Linux
> > [rootfs ~]# mount -t securityfs securityfs /sys/kernel/security
> > [rootfs ~]# cat /proc/cmdline
> > console=ttyS0 security=selinux
> > [rootfs ~]# cat /sys/kernel/security/lsm; echo
> > capability,landlock,lockdown,yama,bpf,selinux
> > [rootfs ~]# cat /proc/self/attr/current
> > cat: read error: Invalid argument
> > 
> > (Note: In this example, uname reports archlinux, but that's only
> > because I based the config on Arch config, it's not actually an Arch
> > kernel.)
> > 
> > Maybe the different behavior is caused by a different config? You can
> > find the Arch config at [1]. Based on Fedora package sources, I think
> > their config has
> > 
> >    CONFIG_LSM="lockdown,yama,integrity,selinux,bpf,landlock,ipe"
> > 
> > while the Arch config has
> > 
> >    CONFIG_LSM="landlock,lockdown,yama,integrity,bpf"
> 
> That's interesting, you're running a LSM that isn't normally run in
> your distro and you're not properly initializing it (no policy load).
> Both are acceptable, but you're definitely operating in the
> corner-iest of corner cases ;)
> 
> I'd have to look at the relevant code, but I suspect that with
> "selinux" missing from the CONFIG_LSM list and you manually specifying
> it on the kernel command line with "security=selinux" you are getting
> it placed at the very end as opposed to what I saw (I have "selinux"
> in my CONFIG_LSM list).  It's further complicated by the fact that the
> procfs call into the LSM's security_getprocattr() hook is going to
> pass a 0/zero into the hook as the @lsmid which means "first
> available".
> 
> Considering that the "security=" parameter is a legacy option, I'd
> encourage you to try the "lsm=" parameter (make sure you specify the
> full list of LSMs you want, in order) to see if that works.

Yes, that works.

The problem isn't that there wouldn't be any working configuration. The
problem is that a userspace program (in my case CRIU) was broken and I
had to spend time figuring out what the cause of the issue was. I'm not
the only one who encountered this issue [1]. I know that at least
Manjaro Linux used to ship a grub config with security=apparmor at some
point (and maybe still does). I no longer use Manjaro Linux, but I
still had this parameter left in my grub config.

[1]: https://github.com/checkpoint-restore/criu/issues/2033

So in reporting this issue, I was just hoping to help future users
avoid the same problem. If you think this is a waste of time, feel free
to ignore this (and sorry I didn't make this clear in the first email).

Lastly, I will offer a few thoughts:
 * The fact that the security parameter can break programs like this is
   highly non-obvious and undocumented.
 * The BPF LSM hook which causes this breakage is useless, because a
   BPF program cannot be attached to it. I think it would make sense to
   just remove it.
 * Switching to using /proc/*/attr/<lsm>/* solves the problem from the
   userspace side. Unfortunately, selinux does not have its
   subdirectory in attr.

Kind regards,
Filip Hejsek

Re: [bug report] [regression?] bpf lsm breaks /proc/*/attr/current with security= on commandline

[Paul Moore]

On Thu, Sep 25, 2025 at 12:25 PM Filip Hejsek <filip.hejsek@gmail.com> wrote:
> On Thu, 2025-09-25 at 11:28 -0400, Paul Moore wrote:
> > On Thu, Sep 25, 2025 at 10:56 AM Filip Hejsek <filip.hejsek@gmail.com> wrote:
> > > On Wed, 2025-09-24 at 17:24 -0400, Paul Moore wrote:
> > > > On Sat, Sep 13, 2025 at 1:01 PM Filip Hejsek <filip.hejsek@gmail.com> wrote:
> > > > >
> > > > > Hello,
> > > > >
> > > > > TLDR: because of bpf lsm, putting security=selinux on commandline
> > > > >       results in /proc/*/attr/current returning errors.
> > > > >
> > > > > When the legacy security= commandline option is used, the specified lsm
> > > > > is added to the end of the lsm list. For example, security=apparmor
> > > > > results in the following order of security modules:
> > > > >
> > > > >    capability,landlock,lockdown,yama,bpf,apparmor
> > > > >
> > > > > In particular, the bpf lsm will be ordered before the chosen major lsm.
> > > > >
> > > > > This causes reads and writes of /proc/*/attr/current to fail, because
> > > > > the bpf hook overrides the apparmor/selinux hook.
> > > >
> > > > What kernel are you using?
> > >
> > > I'm using Arch Linux kernel, which is very close to mainline. I have
> > > also tested my own build from git sources (I used a stripped down
> > > config which I based on config from Arch). Example in QEMU:
> > >
> > > $ qemu-system-x86_64 -nodefaults -accel kvm -cpu host -smp cpus=2 -m 1G -display none -kernel ~/git/linux/arch/x86/boot/bzImage -initrd ./initramfs.img -serial mon:stdio -append 'console=ttyS0 security=selinux'
> > > :: mounting '' on real root
> > > mount: /new_root: no valid filesystem type specified.
> > > ERROR: Failed to mount '' on real root
> > > You are now being dropped into an emergency shell.
> > > sh: can't access tty; job control turned off
> > > [rootfs ~]# uname -a
> > > Linux archlinux 6.17.0-rc7-00020-gcec1e6e5d1ab #3 SMP PREEMPT_DYNAMIC Thu Sep 25 16:28:02 CEST 2025 x86_64 GNU/Linux
> > > [rootfs ~]# mount -t securityfs securityfs /sys/kernel/security
> > > [rootfs ~]# cat /proc/cmdline
> > > console=ttyS0 security=selinux
> > > [rootfs ~]# cat /sys/kernel/security/lsm; echo
> > > capability,landlock,lockdown,yama,bpf,selinux
> > > [rootfs ~]# cat /proc/self/attr/current
> > > cat: read error: Invalid argument
> > >
> > > (Note: In this example, uname reports archlinux, but that's only
> > > because I based the config on Arch config, it's not actually an Arch
> > > kernel.)
> > >
> > > Maybe the different behavior is caused by a different config? You can
> > > find the Arch config at [1]. Based on Fedora package sources, I think
> > > their config has
> > >
> > >    CONFIG_LSM="lockdown,yama,integrity,selinux,bpf,landlock,ipe"
> > >
> > > while the Arch config has
> > >
> > >    CONFIG_LSM="landlock,lockdown,yama,integrity,bpf"
> >
> > That's interesting, you're running a LSM that isn't normally run in
> > your distro and you're not properly initializing it (no policy load).
> > Both are acceptable, but you're definitely operating in the
> > corner-iest of corner cases ;)
> >
> > I'd have to look at the relevant code, but I suspect that with
> > "selinux" missing from the CONFIG_LSM list and you manually specifying
> > it on the kernel command line with "security=selinux" you are getting
> > it placed at the very end as opposed to what I saw (I have "selinux"
> > in my CONFIG_LSM list).  It's further complicated by the fact that the
> > procfs call into the LSM's security_getprocattr() hook is going to
> > pass a 0/zero into the hook as the @lsmid which means "first
> > available".
> >
> > Considering that the "security=" parameter is a legacy option, I'd
> > encourage you to try the "lsm=" parameter (make sure you specify the
> > full list of LSMs you want, in order) to see if that works.
>
> Yes, that works.
>
> The problem isn't that there wouldn't be any working configuration. The
> problem is that a userspace program (in my case CRIU) was broken and I
> had to spend time figuring out what the cause of the issue was. I'm not
> the only one who encountered this issue [1].

Arguably, when a Linux distro that is enabling new features, e.g.
multiple LSMs and/or the BPF LSM, they have an obligation to migrate
away from legacy/deprecated configuration knobs to properly use those
new features.

With the "security=" command line option only supporting one LSM, it's
going to be hard to do the right thing in all situations; someone is
always going to be upset with the placement in a multi-LSM system.
Perhaps it's time to make a push to properly deprecate and eventually
remove the "security=" command line option; it would definitely
simplify the kernel code.

> So in reporting this issue, I was just hoping to help future users
> avoid the same problem. If you think this is a waste of time, feel free
> to ignore this (and sorry I didn't make this clear in the first email).
>
> Lastly, I will offer a few thoughts:
>  * The fact that the security parameter can break programs like this is
>    highly non-obvious and undocumented.

https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html

I'll agree that the documentation probably should carry a stronger
warning, but it is marked as deprecated.

>  * The BPF LSM hook which causes this breakage is useless, because a
>    BPF program cannot be attached to it. I think it would make sense to
>    just remove it.

That would be a decision the BPF LSM maintainer would need to make.
There is precedent for doing things like this in the BPF LSM, and I
would tend to agree that this hook probably has little use for the
usual BPF LSMs.

>  * Switching to using /proc/*/attr/<lsm>/* solves the problem from the
>    userspace side. Unfortunately, selinux does not have its
>    subdirectory in attr.

This is a legacy carryover from the early days of the LSM framework
and SELinux.  By the time there was a need for LSM specific entries in
procfs there was already a significant amount of userspace that relied
on getting SELinux info from /proc/*/attr; migrating to
/proc/*/attr/selinux (or any other path for that matter) would have
broken too many things.

The good news is that there are a number of reasons why we want to get
away from using procfs for these things anyway, and we recently added
a few LSM syscalls to both solve the procfs problems and better
support multiple LSMs.  LWN.net wrote a nice article on the syscalls
which you can find at the link below:

* https://lwn.net/Articles/919059

-- 
paul-moore.com