BPF LSM prevent program unload

[Frederick Lawler <fred@cloudflare.com>]

Hi,

IIUC, LSMs are supposed to give us the ability to design policy around
unprivileged users and in addition to privileged users. As we expand
our usage of BPF LSM's, there are cases where we want to restrict
privileged users from unloading our progs. For instance, any privileged
user that wants to remove restrictions we've placed on privileged users.

We currently have a loader application doesn't leverage BPF skeletons. We
instead load BPF object files, and then pin the progs to a mount point that
is a bpf filesystem. On next run, if we have new policies, load in new
policies, and finally unload the old.

Here are some conditions a privileged user may unload programs:
	
	umount /sys/fs/bpf
	rm -rf /sys/fs/bpf/lsm
	rm /sys/fs/bpf/lsm/some_prog
	unlink /sys/fs/bpf/lsm/some_prog

This works because once we remove the last reference, the programs and
pinned maps are cleaned up.

Moving individual pins or moving the mount entirely with mount --move
do not perform any clean up operations. Lastly, bpftool doesn't currently
have the ability to unload LSM's AFAIK.

The few ideas I have floating around are:

1. Leverage some LSM hooks (BPF or otherwise) to restrict on the functions
   security_sb_umount(), security_path_unlink(), security_inode_unlink().

   Both security_path_unlink() and security_inode_unlink() handle the
   unlink/remove case, but not the umount case.

3. Leverage SELinux/Apparmor to possibly handle these cases.

4. Introduce a security_bpf_prog_unload() to target hopefully the
   umount and unlink cases at the same time.

5. Possible moonshot idea: introduce a interface to pin _specifically_
   BPF LSM's to the kernel, and avoid the bpf sysfs problems all
   together.

We're making the assumption this problem has been thought about before,
and are wondering if there's anything obvious we're missing here.

Fred