LOCK_DOWN_FORCE_INTEGRITY_FOR_FUZZING?

LOCK_DOWN_FORCE_INTEGRITY_FOR_FUZZING?

[Dmitry Vyukov]

Hi Lockdown maintainers,

We don't enable Lockdown integrity mode on syzbot during fuzzing, but
we would like to. The main problem is the restriction of debugfs,
which we need for fuzzing. But we do duplicate some of its
restrictions (e.g. DEVKMEM=n).

It come up again recently in the context of discussion of memory
corruptions when a mounted blocked device is being written to by root:
https://lore.kernel.org/all/CACT4Y+b1vGfe0Uvp6YmKahK4GfCfvdBLCh0SAQzGgWN1s6A+0Q@mail.gmail.com/
It looks like this restriction of writing to mounted block devices
should be part of integrity lockdown (but I am not an expert).

What do you think about the addition of a new level that is integrity
but with debug fs access?
LOCKDOWN_RTAS_ERROR_INJECTION also looks like it's in the same bucket
of "fine for testing".

At least for us it is OK if it can be enabled only via kernel config
(no cmd line) and named accordingly
(TEST_ONLY_DONT_ENABLE_IN_PRODUCTION).

If we have it, we could restrict writing to mounted devices in
integrity mode and enable this mode on syzbot.

Thanks

Re: LOCK_DOWN_FORCE_INTEGRITY_FOR_FUZZING?

[Nathan Lynch]

Hi Dmitry,

Dmitry Vyukov <dvyukov@google.com> writes:
> Hi Lockdown maintainers,

I'm not a lockdown maintainer, but I feel like I can respond to a couple
things here.

> We don't enable Lockdown integrity mode on syzbot during fuzzing, but
> we would like to. The main problem is the restriction of debugfs,
> which we need for fuzzing. But we do duplicate some of its
> restrictions (e.g. DEVKMEM=n).
>
> It come up again recently in the context of discussion of memory
> corruptions when a mounted blocked device is being written to by root:
> https://lore.kernel.org/all/CACT4Y+b1vGfe0Uvp6YmKahK4GfCfvdBLCh0SAQzGgWN1s6A+0Q@mail.gmail.com/
> It looks like this restriction of writing to mounted block devices
> should be part of integrity lockdown (but I am not an expert).

I'm not sure, I'll leave that question to others.

> What do you think about the addition of a new level that is integrity
> but with debug fs access?
> LOCKDOWN_RTAS_ERROR_INJECTION also looks like it's in the same bucket
> of "fine for testing".

Thanks for checking. Error injection via RTAS (pseries partition
firmware) can cause unrecoverable cache and memory corruption. Right now
I don't think including LOCKDOWN_RTAS_ERROR_INJECTION in a relaxed
integrity mode for fuzzing would yield useful results.

> At least for us it is OK if it can be enabled only via kernel config
> (no cmd line) and named accordingly
> (TEST_ONLY_DONT_ENABLE_IN_PRODUCTION).
>
> If we have it, we could restrict writing to mounted devices in
> integrity mode and enable this mode on syzbot.

So I understand the proposal to involve something like:

--- a/include/linux/security.h
+++ b/include/linux/security.h
@@ -120,11 +120,12 @@ enum lockdown_reason {
 	LOCKDOWN_TIOCSSERIAL,
 	LOCKDOWN_MODULE_PARAMETERS,
 	LOCKDOWN_MMIOTRACE,
-	LOCKDOWN_DEBUGFS,
 	LOCKDOWN_XMON_WR,
 	LOCKDOWN_BPF_WRITE_USER,
 	LOCKDOWN_DBG_WRITE_KERNEL,
 	LOCKDOWN_RTAS_ERROR_INJECTION,
+	LOCKDOWN_INTEGRITY_FUZZING_MAX,
+	LOCKDOWN_DEBUGFS,
 	LOCKDOWN_INTEGRITY_MAX,
 	LOCKDOWN_KCORE,
 	LOCKDOWN_KPROBES,

Is that right?

I don't have a specific example at hand, but I suspect that enabling all
of debugfs would allow syzbot to reach code that integrity mode
ordinarily would prevent. Which doesn't seem like what you would want?

I wonder if the debugfs-hosted facilities needed by syzbot could be
identified in a finer-grained way that could be incorporated into the
lockdown reason list for this purpose. So that way only the things
syzbot needs would be exposed.

Re: LOCK_DOWN_FORCE_INTEGRITY_FOR_FUZZING?

[Dmitry Vyukov]

On Fri, 17 Mar 2023 at 16:21, Nathan Lynch <nathanl@linux.ibm.com> wrote:
>
> Hi Dmitry,
>
> Dmitry Vyukov <dvyukov@google.com> writes:
> > Hi Lockdown maintainers,
>
> I'm not a lockdown maintainer, but I feel like I can respond to a couple
> things here.
>
> > We don't enable Lockdown integrity mode on syzbot during fuzzing, but
> > we would like to. The main problem is the restriction of debugfs,
> > which we need for fuzzing. But we do duplicate some of its
> > restrictions (e.g. DEVKMEM=n).
> >
> > It come up again recently in the context of discussion of memory
> > corruptions when a mounted blocked device is being written to by root:
> > https://lore.kernel.org/all/CACT4Y+b1vGfe0Uvp6YmKahK4GfCfvdBLCh0SAQzGgWN1s6A+0Q@mail.gmail.com/
> > It looks like this restriction of writing to mounted block devices
> > should be part of integrity lockdown (but I am not an expert).
>
> I'm not sure, I'll leave that question to others.
>
> > What do you think about the addition of a new level that is integrity
> > but with debug fs access?
> > LOCKDOWN_RTAS_ERROR_INJECTION also looks like it's in the same bucket
> > of "fine for testing".
>
> Thanks for checking. Error injection via RTAS (pseries partition
> firmware) can cause unrecoverable cache and memory corruption. Right now
> I don't think including LOCKDOWN_RTAS_ERROR_INJECTION in a relaxed
> integrity mode for fuzzing would yield useful results.
>
> > At least for us it is OK if it can be enabled only via kernel config
> > (no cmd line) and named accordingly
> > (TEST_ONLY_DONT_ENABLE_IN_PRODUCTION).
> >
> > If we have it, we could restrict writing to mounted devices in
> > integrity mode and enable this mode on syzbot.
>
> So I understand the proposal to involve something like:
>
> --- a/include/linux/security.h
> +++ b/include/linux/security.h
> @@ -120,11 +120,12 @@ enum lockdown_reason {
>         LOCKDOWN_TIOCSSERIAL,
>         LOCKDOWN_MODULE_PARAMETERS,
>         LOCKDOWN_MMIOTRACE,
> -       LOCKDOWN_DEBUGFS,
>         LOCKDOWN_XMON_WR,
>         LOCKDOWN_BPF_WRITE_USER,
>         LOCKDOWN_DBG_WRITE_KERNEL,
>         LOCKDOWN_RTAS_ERROR_INJECTION,
> +       LOCKDOWN_INTEGRITY_FUZZING_MAX,
> +       LOCKDOWN_DEBUGFS,
>         LOCKDOWN_INTEGRITY_MAX,
>         LOCKDOWN_KCORE,
>         LOCKDOWN_KPROBES,
>
> Is that right?

Hi Nathan,

Thanks for the feedback.
Yes, something like this + config to enable this mode.

> I don't have a specific example at hand, but I suspect that enabling all
> of debugfs would allow syzbot to reach code that integrity mode
> ordinarily would prevent. Which doesn't seem like what you would want?
>
> I wonder if the debugfs-hosted facilities needed by syzbot could be
> identified in a finer-grained way that could be incorporated into the
> lockdown reason list for this purpose. So that way only the things
> syzbot needs would be exposed.

Debugfs also contains lots of code that needs to be tested.
It's also required to test production code, e.g. set up some stub devices, etc.
We generally can restrict fuzzer access based on paths (it's also
required to avoid writing garbage into /dev/sda, etc). So far having
debugfs accessible did not cause any problems.

Re: LOCK_DOWN_FORCE_INTEGRITY_FOR_FUZZING?

[Tetsuo Handa]

On 2023/03/14 19:02, Dmitry Vyukov wrote:
> At least for us it is OK if it can be enabled only via kernel config
> (no cmd line) and named accordingly
> (TEST_ONLY_DONT_ENABLE_IN_PRODUCTION).

3 years ago, there were several discussions on whether we can use build-time (i.e.
kernel config) options. There was a strong suggestion to use boot-time (i.e. kernel
command line) options for switching because Linux 5.6+ can handle very long kernel
command line using a boot-config file. 

  https://lkml.kernel.org/r/CAHk-=wiVQUo_RJAaivHU5MFdznNOX4GKgJH1xrFc83e9oLnuvQ@mail.gmail.com

But I think that the reality remains that build-time options is the better.
Since I think that syzbot/syzkaller got enough awareness and past record, maybe
it is time to retry persuading Linus about use of build-time options for fuzzing.

Re: LOCK_DOWN_FORCE_INTEGRITY_FOR_FUZZING?

[Paul Moore]

On Tue, Mar 14, 2023 at 6:02 AM Dmitry Vyukov <dvyukov@google.com> wrote:
>
> Hi Lockdown maintainers,
>
> We don't enable Lockdown integrity mode on syzbot during fuzzing, but
> we would like to. The main problem is the restriction of debugfs,
> which we need for fuzzing. But we do duplicate some of its
> restrictions (e.g. DEVKMEM=n).
>
> It come up again recently in the context of discussion of memory
> corruptions when a mounted blocked device is being written to by root:
> https://lore.kernel.org/all/CACT4Y+b1vGfe0Uvp6YmKahK4GfCfvdBLCh0SAQzGgWN1s6A+0Q@mail.gmail.com/
> It looks like this restriction of writing to mounted block devices
> should be part of integrity lockdown (but I am not an expert).

The first question I would ask is are there legitimate reasons to
write directly to a block device that would not impact the integrity
of the device?  Of course the first thing I think of when I think of
block devices is a storage device with a filesystem on top, but I'm
not comfortable enough with my block device knowledge to know what
other classes of block devices might exist and how one might normally
interact with them.

If storage devices are the only instances of block devices that we
support, we would also need to be careful that any calls to
security_locked_down() happen only when userspace is attempting a
direct write; we wouldn't want to bump into a lockdown access denial
when the filesystem attempts to write to the block device.

> What do you think about the addition of a new level that is integrity
> but with debug fs access?
> LOCKDOWN_RTAS_ERROR_INJECTION also looks like it's in the same bucket
> of "fine for testing".

It's been a little while, and I can't say I fully understood the
ppc/pseries RTAS stuff at the time (or now for that matter), but I
didn't think that the RTAS error injection mechanism was strictly for
testing, is it?

Regardless of RTAS, if the goal is testing lockdown with syzbot, which
requires debugfs, why not test a lockdown level that still allows
debugfs?

> At least for us it is OK if it can be enabled only via kernel config
> (no cmd line) and named accordingly
> (TEST_ONLY_DONT_ENABLE_IN_PRODUCTION).
>
> If we have it, we could restrict writing to mounted devices in
> integrity mode and enable this mode on syzbot.

Regardless of what we decide wrt to a new level, I think we want it to
always be present and not dependent on a config knob, either runtime
or buildtime.

-- 
paul-moore.com

Re: LOCK_DOWN_FORCE_INTEGRITY_FOR_FUZZING?

[Dmitry Vyukov]

On Mon, 20 Mar 2023 at 21:36, Paul Moore <paul@paul-moore.com> wrote:
>
> On Tue, Mar 14, 2023 at 6:02 AM Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > Hi Lockdown maintainers,
> >
> > We don't enable Lockdown integrity mode on syzbot during fuzzing, but
> > we would like to. The main problem is the restriction of debugfs,
> > which we need for fuzzing. But we do duplicate some of its
> > restrictions (e.g. DEVKMEM=n).
> >
> > It come up again recently in the context of discussion of memory
> > corruptions when a mounted blocked device is being written to by root:
> > https://lore.kernel.org/all/CACT4Y+b1vGfe0Uvp6YmKahK4GfCfvdBLCh0SAQzGgWN1s6A+0Q@mail.gmail.com/
> > It looks like this restriction of writing to mounted block devices
> > should be part of integrity lockdown (but I am not an expert).
>
> The first question I would ask is are there legitimate reasons to
> write directly to a block device that would not impact the integrity
> of the device?  Of course the first thing I think of when I think of
> block devices is a storage device with a filesystem on top, but I'm
> not comfortable enough with my block device knowledge to know what
> other classes of block devices might exist and how one might normally
> interact with them.

That thread mentions tune2fs utility that does this now.


> If storage devices are the only instances of block devices that we
> support, we would also need to be careful that any calls to
> security_locked_down() happen only when userspace is attempting a
> direct write; we wouldn't want to bump into a lockdown access denial
> when the filesystem attempts to write to the block device.
>
> > What do you think about the addition of a new level that is integrity
> > but with debug fs access?
> > LOCKDOWN_RTAS_ERROR_INJECTION also looks like it's in the same bucket
> > of "fine for testing".
>
> It's been a little while, and I can't say I fully understood the
> ppc/pseries RTAS stuff at the time (or now for that matter), but I
> didn't think that the RTAS error injection mechanism was strictly for
> testing, is it?
>
> Regardless of RTAS, if the goal is testing lockdown with syzbot, which
> requires debugfs, why not test a lockdown level that still allows
> debugfs?

My understanding is that only possible to enable these 3 levels:
https://elixir.bootlin.com/linux/v6.3-rc3/source/security/lockdown/lockdown.c#L19
and we want something in-between "none" and "integrity".


> > At least for us it is OK if it can be enabled only via kernel config
> > (no cmd line) and named accordingly
> > (TEST_ONLY_DONT_ENABLE_IN_PRODUCTION).
> >
> > If we have it, we could restrict writing to mounted devices in
> > integrity mode and enable this mode on syzbot.
>
> Regardless of what we decide wrt to a new level, I think we want it to
> always be present and not dependent on a config knob, either runtime
> or buildtime.

This will work for testing scenarios, but I was worried about production.
It's not supposed to be enabled in production builds, so making it
available can open the door for misconfiguration.

Re: LOCK_DOWN_FORCE_INTEGRITY_FOR_FUZZING?

[Paul Moore]

On Tue, Mar 21, 2023 at 5:57 AM Dmitry Vyukov <dvyukov@google.com> wrote:
> On Mon, 20 Mar 2023 at 21:36, Paul Moore <paul@paul-moore.com> wrote:
> > On Tue, Mar 14, 2023 at 6:02 AM Dmitry Vyukov <dvyukov@google.com> wrote:
> > >
> > > Hi Lockdown maintainers,
> > >
> > > We don't enable Lockdown integrity mode on syzbot during fuzzing, but
> > > we would like to. The main problem is the restriction of debugfs,
> > > which we need for fuzzing. But we do duplicate some of its
> > > restrictions (e.g. DEVKMEM=n).
> > >
> > > It come up again recently in the context of discussion of memory
> > > corruptions when a mounted blocked device is being written to by root:
> > > https://lore.kernel.org/all/CACT4Y+b1vGfe0Uvp6YmKahK4GfCfvdBLCh0SAQzGgWN1s6A+0Q@mail.gmail.com/
> > > It looks like this restriction of writing to mounted block devices
> > > should be part of integrity lockdown (but I am not an expert).
> >
> > The first question I would ask is are there legitimate reasons to
> > write directly to a block device that would not impact the integrity
> > of the device?  Of course the first thing I think of when I think of
> > block devices is a storage device with a filesystem on top, but I'm
> > not comfortable enough with my block device knowledge to know what
> > other classes of block devices might exist and how one might normally
> > interact with them.
>
> That thread mentions tune2fs utility that does this now.

In which case it would seem that this isn't something we could
restrict on the grounds of integrity.  Regardless, thanks for
mentioning it.

> > If storage devices are the only instances of block devices that we
> > support, we would also need to be careful that any calls to
> > security_locked_down() happen only when userspace is attempting a
> > direct write; we wouldn't want to bump into a lockdown access denial
> > when the filesystem attempts to write to the block device.
> >
> > > What do you think about the addition of a new level that is integrity
> > > but with debug fs access?
> > > LOCKDOWN_RTAS_ERROR_INJECTION also looks like it's in the same bucket
> > > of "fine for testing".
> >
> > It's been a little while, and I can't say I fully understood the
> > ppc/pseries RTAS stuff at the time (or now for that matter), but I
> > didn't think that the RTAS error injection mechanism was strictly for
> > testing, is it?
> >
> > Regardless of RTAS, if the goal is testing lockdown with syzbot, which
> > requires debugfs, why not test a lockdown level that still allows
> > debugfs?
>
> My understanding is that only possible to enable these 3 levels:
> https://elixir.bootlin.com/linux/v6.3-rc3/source/security/lockdown/lockdown.c#L19
> and we want something in-between "none" and "integrity".

Sorry, my apologies, most of my interactions with the lockdown LSM
have focused on the enforcement side and not the userspace API in
securityfs; while lockdown has multiple different levels/reasons, for
some reason only three are actually available to userspace.  I'm
guessing that the ordering of levels beyond the basic
none/integrity/confidentiality values is an area of debate and it is
easier to offer a limited number of options.

> > > At least for us it is OK if it can be enabled only via kernel config
> > > (no cmd line) and named accordingly
> > > (TEST_ONLY_DONT_ENABLE_IN_PRODUCTION).
> > >
> > > If we have it, we could restrict writing to mounted devices in
> > > integrity mode and enable this mode on syzbot.
> >
> > Regardless of what we decide wrt to a new level, I think we want it to
> > always be present and not dependent on a config knob, either runtime
> > or buildtime.
>
> This will work for testing scenarios, but I was worried about production.
> It's not supposed to be enabled in production builds, so making it
> available can open the door for misconfiguration.

I'm also concerned about the difference between test and production
and the potential for "special" test configurations making their way
into production systems, this is why I'm not in favor of special
configurations which are only enabled for testing.  I also have
concerns about testing "special" configurations as opposed to what
people actually run.

-- 
paul-moore.com