An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

[Andy Lutomirski]

Since this thread has devolved horribly, I'm going to propose a solution.

1. Split the "lockdown" state into three levels:  (please don't
bikeshed about the names right now.)

LOCKDOWN_NONE: normal behavior

LOCKDOWN_PROTECT_INTEGREITY: kernel tries to keep root from writing to
kernel memory

LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY: kernel tries to keep root from
reading or writing kernel memory.

2. The kexec protocol gets a new flag min_lockdown_level.  A kexeced
kernel will boot with at least that lockdown level regardless of its
configuration.  kexec sets min_lockdown_level to the running kernels'
lockdown_level.  Some future API could allow kexec with a higher
min_lockdown_level.  An even fancier future API could allow a
LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY kernel to kexec with
min_lockdown_level == LOCKDOWN_PROTECT_INTEGRITY if there's some
mechanism that guarantees that memory gets zeroed in the process.

3. All the bpf and tracing stuf, etc, gets changed so it only takes
effect when LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY is set.  This
removes a giant annoyance on distro kernels that are likely to want to
enable LOCKDOWN_PROTECT_INTEGRITY.  If you load a key into the kernel,
and you want to keep that key safe, you can enable
LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY at that time.  After all, if
root is compromised before that, root can just remember a copu of the
key in user memory or email it to someone.

4. There's a kernel config option for the default lockdown level.
This operates completely independently of secure boot.

5. There's a command line option to increase the lockdown level above
the default level.  No particular authentication is needed for this
option to work.

6. There's a way to *decrease* the lockdown level below the configured
value.  (This ability itself may be gated by a config option.)
Choices include a UEFI protected variable, an authenticated flag
passed by the bootloader, and even just some special flag in the boot
handoff protocol.  It would be really quite useful for a user to be
able to ask their bootloader to reduce the lockdown level for the
purpose of a particular boot for debugging.  I read the docs on
mokutil --disable-validation, and it's quite messy.  Let's have a way
to do this that is mostly independent of the particular firmware in
use.

I can imagine a grub option that decreases lockdown level along with a
rule that grub will *not* load that option from its config, for
example.

7. kexec does not attempt to think about "secure boot" at all.
They're totally separate.

What do you all think?  I think that this checks basically all the
boxes, is a lot more user friendly than the current patchset or what
distros do, and actually makes some sense from a security perspective.

--Andy

Re: An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

[David Howells]

Andy Lutomirski <luto@kernel.org> wrote:

> Since this thread has devolved horribly, I'm going to propose a solution.
> 
> 1. Split the "lockdown" state into three levels:  (please don't
> bikeshed about the names right now.)
> 
> LOCKDOWN_NONE: normal behavior
> 
> LOCKDOWN_PROTECT_INTEGREITY: kernel tries to keep root from writing to
> kernel memory
> 
> LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY: kernel tries to keep root from
> reading or writing kernel memory.

In theory, it's good idea, but in practice it's not as easy to implement as I
think you think.

Let me list here the things that currently get restricted by lockdown:

 (1) Manipulation of devices to access the kernel image:

	- Ioports & ioperm
	- /dev/ports
	- /dev/mem
	- PCI Bar
	- Some debugfs files
	- pcmcia_cis
	- Driver hardware parameters
	  - ISA drivers
	  - TIOCSSERIAL
	  - SCSI EATA driver
	- testmmiotrace
	- firmware

 (2) Direct kernel memory modification:

	- /dev/mem
	- /dev/kmem
	- bpf
	- kprobes

 (3) Direct kernel memory reading:

	- /dev/mem
	- /dev/kmem
	- /dev/kcore
	- bpf
	- kprobes
	- perf

 (4) Indirect kernel access:

	- Modules
	- MSRs
	- Suspend to disk
	- ACPI (custom_method, RSDP, table override, error injection)

 (5) Kexec.

Note that /dev/mem can be used to access MMIO devices (I'm not sure about
/dev/kmem, though).  Even reads through /dev/mem can do this.  I'm not sure
whether that's sufficient to actually affect a modification, though.

Debugfs is particularly icky.  It contains at least a couple of thousand
files, a few of which provide direct access to hardware, some of which change
driver behaviour and some of which just give information.  Auditing that pile
isn't something I really want to have to do.  I'd rather just turn the whole
lot off, but got persuaded otherwise by people who have been using it to
provide mechanisms that programs rely on - hence the only allow files that
have 0444 (and even that is iffy as some of these files have side effects and
write ops anyway).

> 2. The kexec protocol gets a new flag min_lockdown_level.  A kexeced
> kernel will boot with at least that lockdown level regardless of its
> configuration.  kexec sets min_lockdown_level to the running kernels'
> lockdown_level.  Some future API could allow kexec with a higher
> min_lockdown_level.  An even fancier future API could allow a
> LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY kernel to kexec with
> min_lockdown_level == LOCKDOWN_PROTECT_INTEGRITY if there's some
> mechanism that guarantees that memory gets zeroed in the process.

Note that on x86 we already have an allocated flag for passing the secure boot
flag from kexec/bootloader to the kernel being booted, and what you're
proposing ought to be redundant.  See boot_params::secure_boot.

> 3. All the bpf and tracing stuf, etc, gets changed so it only takes
> effect when LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY is set.

Uh, no.  bpf, for example, can be used to modify kernel memory.  I think only
the following are safe from being able to talk directly to devices:

	/dev/kmem (O_RDONLY only)
	/dev/kcore
	perf
	some debugfs files

> This removes a giant annoyance on distro kernels that are likely to want to
> enable LOCKDOWN_PROTECT_INTEGRITY.

*shrug* Distros have been running with the full set for a while.  I haven't
seen many complaints.

> If you load a key into the kernel, and you want to keep that key safe, you
> can enable LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY at that time.  After all,
> if root is compromised before that, root can just remember a copu of the key
> in user memory or email it to someone.

If your key needs to be truly protected, it should never be seen unencrypted
in userspace, rather it should be decrypted in the TPM and then retained in
kernel memory only.

> ...
> 6. There's a way to *decrease* the lockdown level below the configured
> value.  (This ability itself may be gated by a config option.)
> Choices include a UEFI protected variable,

By turning secure boot off, maybe?

> an authenticated flag passed by the bootloader,

Authenticated how?  How do you stop the running system from passing this to
the bootloader next time it is run?  I guess you're thinking of a bootloader
"command" that can only be passed by someone sat at a keyboard and never read
from the config file.

> and even just some special flag in the boot handoff protocol.

We already have that with secure boot.

> It would be really quite useful for a user to be able to ask their
> bootloader to reduce the lockdown level for the purpose of a particular boot
> for debugging.

This I shall grant you - but you have to be able to prevent an attacker inside
the system from making use of it.  There's a SysRq provided to drop out of
lockdown mode - in theory only usable if you're sat at a wired keyboard
connected to the computer.

> 7. kexec does not attempt to think about "secure boot" at all.
> They're totally separate.

Except that they're not.  The boot_params flag must be set to something by
kexec on x86 for transfer along the chain.  However, we can leave secure boot
out for now - that can be treated as a separate issue.

Note that whether you like it or not, distribution kernels already work this
way and propagate the boot_params flag over kexec.

> What do you all think?  I think that this checks basically all the
> boxes, is a lot more user friendly than the current patchset or what
> distros do, and actually makes some sense from a security perspective.

You haven't really defined well what's allowed in _INTEGRITY mode verses
_SECRECY mode.

I'm currently in the throes of modifying my patchset to make it more inline
with what Linus mandates: dropping the enablement from secure boot; dropping
the propagation of the flag over kexec; providing a config option to make it
mandatory.  There is already a command line option to trigger it.

David

Re: An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

[Jann Horn]

+ast@kernel.org

On Wed, Apr 4, 2018 at 6:17 PM, David Howells <dhowells@redhat.com> wrote:
> Andy Lutomirski <luto@kernel.org> wrote:
[...]
>> 3. All the bpf and tracing stuf, etc, gets changed so it only takes
>> effect when LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY is set.
>
> Uh, no.  bpf, for example, can be used to modify kernel memory.

I'm pretty sure bpf isn't supposed to be able to modify arbitrary
kernel memory. AFAIU if you can use BPF to write to arbitrary kernel
memory, that's a bug; with CAP_SYS_ADMIN, you can read from userspace,
write to userspace, and read from kernelspace, but you shouldn't be
able to write to kernelspace.

Re: An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

[Andy Lutomirski]

On Wed, Apr 4, 2018 at 9:17 AM, David Howells <dhowells@redhat.com> wrote:
> Andy Lutomirski <luto@kernel.org> wrote:
>
>> Since this thread has devolved horribly, I'm going to propose a solution.
>>
>> 1. Split the "lockdown" state into three levels:  (please don't
>> bikeshed about the names right now.)
>>
>> LOCKDOWN_NONE: normal behavior
>>
>> LOCKDOWN_PROTECT_INTEGREITY: kernel tries to keep root from writing to
>> kernel memory
>>
>> LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY: kernel tries to keep root from
>> reading or writing kernel memory.
>
> In theory, it's good idea, but in practice it's not as easy to implement as I
> think you think.

[...]

so do your best.  The whole lockdown patchset is a best-effort thing,
not a "we did it and it's done" thing.

> Note that /dev/mem can be used to access MMIO devices (I'm not sure about
> /dev/kmem, though).  Even reads through /dev/mem can do this.  I'm not sure
> whether that's sufficient to actually affect a modification, though.

So turn it off in both modes.  /dev/mem is basically obsolete anyway.
Or restrict it to actual memory, or whatever.

>
>> 2. The kexec protocol gets a new flag min_lockdown_level.  A kexeced
>> kernel will boot with at least that lockdown level regardless of its
>> configuration.  kexec sets min_lockdown_level to the running kernels'
>> lockdown_level.  Some future API could allow kexec with a higher
>> min_lockdown_level.  An even fancier future API could allow a
>> LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY kernel to kexec with
>> min_lockdown_level == LOCKDOWN_PROTECT_INTEGRITY if there's some
>> mechanism that guarantees that memory gets zeroed in the process.
>
> Note that on x86 we already have an allocated flag for passing the secure boot
> flag from kexec/bootloader to the kernel being booted, and what you're
> proposing ought to be redundant.  See boot_params::secure_boot.

Incorrect.  Linus is rejecting the idea that secure boot implies
lockdown.  I strongly agree with him.  I also think that trying to
make security decisions in a kexeced kernel based on whether the
previous kernel was secure booted is a bad idea, so I'm suggesting a
new feature.  I don't really care what value is passed in
boot_params::secure_boot for a kexeced kernel, but I think it should
not be used for lockdown.

So I stand by my proposal.

>
>> 3. All the bpf and tracing stuf, etc, gets changed so it only takes
>> effect when LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY is set.
>
> Uh, no.  bpf, for example, can be used to modify kernel memory.

Really?  If that is, in fact, true, then turn *that* feature off when
LOCKDOWN_PROTECT_INTEGRITY is set.

>> This removes a giant annoyance on distro kernels that are likely to want to
>> enable LOCKDOWN_PROTECT_INTEGRITY.
>
> *shrug* Distros have been running with the full set for a while.  I haven't
> seen many complaints.

As Linus noted, that's because a lot of users who care have already
turned off secure boot.  Once this whole thing gets sorted out, the
hackish dependence of lockdown and secure boot can go away, so it's
time to think about making lockdown less annoying.

>
>> If you load a key into the kernel, and you want to keep that key safe, you
>> can enable LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY at that time.  After all,
>> if root is compromised before that, root can just remember a copu of the key
>> in user memory or email it to someone.
>
> If your key needs to be truly protected, it should never be seen unencrypted
> in userspace, rather it should be decrypted in the TPM and then retained in
> kernel memory only.

You keep making claims like this with no justification whatsoever.

>
>> ...
>> 6. There's a way to *decrease* the lockdown level below the configured
>> value.  (This ability itself may be gated by a config option.)
>> Choices include a UEFI protected variable,
>
> By turning secure boot off, maybe?

No.

>
>> an authenticated flag passed by the bootloader,
>
> Authenticated how?  How do you stop the running system from passing this to
> the bootloader next time it is run?  I guess you're thinking of a bootloader
> "command" that can only be passed by someone sat at a keyboard and never read
> from the config file.

What I meant was some channel that the bootloader can use to securely
communicate options to the running kernel.  The concept of "secure"
here is obviously debatable.  Something in boot_params would work.
Some specially namespaced command line option could work.  Some whole
new "authenticated data from bootloader" block could work.

>
>> and even just some special flag in the boot handoff protocol.
>
> We already have that with secure boot.

No.  That has been rejected over and over in this thread.  I'm
proposing an alternative here.

>
>> 7. kexec does not attempt to think about "secure boot" at all.
>> They're totally separate.
>
> Except that they're not.  The boot_params flag must be set to something by
> kexec on x86 for transfer along the chain.

How about setting it unconditionally to false?

>> What do you all think?  I think that this checks basically all the
>> boxes, is a lot more user friendly than the current patchset or what
>> distros do, and actually makes some sense from a security perspective.
>
> You haven't really defined well what's allowed in _INTEGRITY mode verses
> _SECRECY mode.

_INTEGRITY means "make a best effort to keep root from writing kernel
memory, talking directly to non-IOMMU-protected devices, writing MSRs,
or otherwise corrupting state".  _INTEGRITY_AND_SECRECY means "do all
that and also make a best effort to keep root from reading kernel
memory".

Re: An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

[David Howells]

Jann Horn <jannh@google.com> wrote:

> > Uh, no.  bpf, for example, can be used to modify kernel memory.
> 
> I'm pretty sure bpf isn't supposed to be able to modify arbitrary
> kernel memory. AFAIU if you can use BPF to write to arbitrary kernel
> memory, that's a bug; with CAP_SYS_ADMIN, you can read from userspace,
> write to userspace, and read from kernelspace, but you shouldn't be
> able to write to kernelspace.

Ah - you may be right.  I seem to have misremembered what Joey Lee wrote in
his patch description.

David

Re: An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

[joeyli]

On Wed, Apr 04, 2018 at 11:19:27PM +0100, David Howells wrote:
> Jann Horn <jannh@google.com> wrote:
> 
> > > Uh, no.  bpf, for example, can be used to modify kernel memory.
> > 
> > I'm pretty sure bpf isn't supposed to be able to modify arbitrary
> > kernel memory. AFAIU if you can use BPF to write to arbitrary kernel
> > memory, that's a bug; with CAP_SYS_ADMIN, you can read from userspace,
> > write to userspace, and read from kernelspace, but you shouldn't be
> > able to write to kernelspace.
> 
> Ah - you may be right.  I seem to have misremembered what Joey Lee wrote in
> his patch description.
>

Sorry for it's my fault to misunderstood the behavoir of bpf with
CAP_SYS_ADMIN.

Joey Lee

Re: An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

[James Morris]

On Wed, 4 Apr 2018, David Howells wrote:

> > 6. There's a way to *decrease* the lockdown level below the configured
> > value.  (This ability itself may be gated by a config option.)
> > Choices include a UEFI protected variable,
> 
> By turning secure boot off, maybe?

It's surely reasonable to allow an already secure-booted system to be 
debugged without needing to be rebooted.



- James
-- 
James Morris
<jmorris@namei.org>

Re: An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

[Matthew Garrett]

On Wed, Apr 4, 2018 at 4:25 PM James Morris <jmorris@namei.org> wrote:
> It's surely reasonable to allow an already secure-booted system to be
> debugged without needing to be rebooted.

alt-sysrq-x from a physical console will do that.

Re: An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

[joeyli]

Hi David, 

On Wed, Apr 04, 2018 at 05:17:24PM +0100, David Howells wrote:
> Andy Lutomirski <luto@kernel.org> wrote:
> 
> > Since this thread has devolved horribly, I'm going to propose a solution.
> > 
> > 1. Split the "lockdown" state into three levels:  (please don't
> > bikeshed about the names right now.)
> > 
> > LOCKDOWN_NONE: normal behavior
> > 
> > LOCKDOWN_PROTECT_INTEGREITY: kernel tries to keep root from writing to
> > kernel memory
> > 
> > LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY: kernel tries to keep root from
> > reading or writing kernel memory.
> 
> In theory, it's good idea, but in practice it's not as easy to implement as I
> think you think.
> 
> Let me list here the things that currently get restricted by lockdown:
> 
[...snip]
>  (5) Kexec.
>

About IMA with kernel module signing and kexec(not on x86_64 yet)...

Because IMA can be used to verify the integrity of kernel module or even
the image for kexec. I think that the
IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY must be enabled at runtime
when kernel is locked-down.

Because the root can enroll master key to keyring then IMA trusts the ima key
derived from master key. It causes that the arbitrary signed module can be loaded
when the root compromised.

Thanks
Joey Lee

Re: An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

[Mimi Zohar]

On Thu, 2018-04-05 at 10:16 +0800, joeyli wrote:
> Hi David, 
> 
> On Wed, Apr 04, 2018 at 05:17:24PM +0100, David Howells wrote:
> > Andy Lutomirski <luto@kernel.org> wrote:
> > 
> > > Since this thread has devolved horribly, I'm going to propose a solution.
> > > 
> > > 1. Split the "lockdown" state into three levels:  (please don't
> > > bikeshed about the names right now.)
> > > 
> > > LOCKDOWN_NONE: normal behavior
> > > 
> > > LOCKDOWN_PROTECT_INTEGREITY: kernel tries to keep root from writing to
> > > kernel memory
> > > 
> > > LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY: kernel tries to keep root from
> > > reading or writing kernel memory.
> > 
> > In theory, it's good idea, but in practice it's not as easy to implement as I
> > think you think.
> > 
> > Let me list here the things that currently get restricted by lockdown:
> > 
> [...snip]
> >  (5) Kexec.
> >
> 
> About IMA with kernel module signing and kexec(not on x86_64 yet)...

Only carrying the measurement list across kexec is architecture
specific, but everything else should work.  

> Because IMA can be used to verify the integrity of kernel module or even
> the image for kexec. I think that the
> IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY must be enabled at runtime
> when kernel is locked-down.

I think we need to understand the problem a bit better.  Is the
problem that you're using the secondary keyring and loading the UEFI
keys onto the secondary keyring?

> Because the root can enroll master key to keyring then IMA trusts the ima key
> derived from master key. It causes that the arbitrary signed module can be loaded
> when the root compromised.

With only the builtin keyring, only keys signed by a builtin key can
be added to the IMA keyring.

Mimi

Re: An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

[jlee]

Hi Mimi,

On Thu, Apr 05, 2018 at 10:01:09AM -0400, Mimi Zohar wrote:
> On Thu, 2018-04-05 at 10:16 +0800, joeyli wrote:
> > Hi David, 
> > 
> > On Wed, Apr 04, 2018 at 05:17:24PM +0100, David Howells wrote:
> > > Andy Lutomirski <luto@kernel.org> wrote:
> > > 
> > > > Since this thread has devolved horribly, I'm going to propose a solution.
> > > > 
> > > > 1. Split the "lockdown" state into three levels:  (please don't
> > > > bikeshed about the names right now.)
> > > > 
> > > > LOCKDOWN_NONE: normal behavior
> > > > 
> > > > LOCKDOWN_PROTECT_INTEGREITY: kernel tries to keep root from writing to
> > > > kernel memory
> > > > 
> > > > LOCKDOWN_PROTECT_INTEGRITY_AND_SECRECY: kernel tries to keep root from
> > > > reading or writing kernel memory.
> > > 
> > > In theory, it's good idea, but in practice it's not as easy to implement as I
> > > think you think.
> > > 
> > > Let me list here the things that currently get restricted by lockdown:
> > > 
> > [...snip]
> > >  (5) Kexec.
> > >
> > 
> > About IMA with kernel module signing and kexec(not on x86_64 yet)...
> 
> Only carrying the measurement list across kexec is architecture
> specific, but everything else should work.  
> 
> > Because IMA can be used to verify the integrity of kernel module or even
> > the image for kexec. I think that the
> > IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY must be enabled at runtime
> > when kernel is locked-down.
> 
> I think we need to understand the problem a bit better.  Is the
> problem that you're using the secondary keyring and loading the UEFI
> keys onto the secondary keyring?
>

Sorry for my mistake. I want to write INTEGRITY_TRUSTED_KEYRING in
above but not IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY.

My brain is not clear when writing the mail.
 
> > Because the root can enroll master key to keyring then IMA trusts the ima key
> > derived from master key. It causes that the arbitrary signed module can be loaded
> > when the root compromised.
> 
> With only the builtin keyring, only keys signed by a builtin key can
> be added to the IMA keyring.
> 

Thanks for your description. I saw that the IMA_LOAD_X509 already depends
on IMA_TRUSTED_KEYRING (INTEGRITY_TRUSTED_KEYRING). Please ignore my concern.

Thanks a lot!
Joey Lee

Re: An actual suggestion (Re: [GIT PULL] Kernel lockdown for secure boot)

[joeyli]

Hi Andy,

On Wed, Apr 04, 2018 at 07:49:12AM -0700, Andy Lutomirski wrote:
> Since this thread has devolved horribly, I'm going to propose a solution.
...
> 6. There's a way to *decrease* the lockdown level below the configured
> value.  (This ability itself may be gated by a config option.)
> Choices include a UEFI protected variable, an authenticated flag
> passed by the bootloader, and even just some special flag in the boot
> handoff protocol.  It would be really quite useful for a user to be
> able to ask their bootloader to reduce the lockdown level for the
> purpose of a particular boot for debugging.  I read the docs on

The "mokutil --disable-validation" done a similar bahvior as above.
Just it lets kernel to ignore the secure boot. 

> mokutil --disable-validation, and it's quite messy.  Let's have a way
> to do this that is mostly independent of the particular firmware in
> use.
>

Why the disabl-validation is messy?   
The mokutil is shim specific but not dependent on particular firmware. 
 
> I can imagine a grub option that decreases lockdown level along with a
> rule that grub will *not* load that option from its config, for
> example.
>

The root can modify the grub config to decrease lockdown level in next
boot without physcial accessing. The mokutil's interactive UI is used
to deal with user to confirm the physcial accessing.

Thanks
Joey Lee