AF_ALG hardening

[Demi Marie Obenour <demiobenour@gmail.com>]

[-- Attachment #1.1.1: Type: text/plain, Size: 5269 bytes --]

On 5/1/26 23:35, Eric Biggers wrote:
> On Fri, May 01, 2026 at 08:21:27PM -0400, Demi Marie Obenour wrote:
>> I think the single biggest hardening win for AF_ALG would be to move
>> to the crypto library.  The recent CVEs you mentioned mostly seem
>> to relate to the crypto API, and with a hard-coded list of allowed
>> algorithms there's no need to use the crypto API anymore.  I'm not
>> familiar enough with kernel code to do this easily, but for anyone
>> with basic knowledge of the existing code it should (hopefully) be
>> straightforward.
>>
>> In the meantime, only using synchronous algorithms and not using
>> hardware drivers would also be a useful simplification.  The latter
>> would make it especially clear that AF_ALG is deprecated, because
>> its one potential advantage (being able to use hardware acceleration)
>> would no longer be present.
> 
> The kernel's crypto library
> (https://docs.kernel.org/crypto/libcrypto.html) does greatly simplify a
> lot of kernel code that needs to use crypto algorithms.  Yes, AF_ALG
> doesn't use it directly yet.  Currently AF_ALG puts all the data in
> (zero-copy) scatterlists, then invokes the "traditional crypto API"
> which is very complex and has full scatterlist support, asynchronous
> execution support, an algorithm template system, etc.  In some cases the
> crypto library is then used internally, but it's not called directly.
> 
> So the idea would be something along the lines of:
> 
> - Add an algorithm allowlist to AF_ALG.  It would include only what the
>   small set of userspace programs that uses it actually needs.  Bizarre
>   stuff like "authencesn" wouldn't be included.
> 
> - Change AF_ALG to make it copy any data written to an AF_ALG file
>   descriptor into an internal kernel buffer.  Put the output in another
>   internal kernel buffer, then copy it to userspace.  No zero-copy, and
>   no scatterlists.  Both restrictions would greatly reduce the chance of
>   bugs: the actual crypto algorithms would operate only on these
>   internal buffers, not on pagecache data (e.g. the contents of 'su') or
>   buffers that userspace can concurrently modify.  The use of simple
>   virtual addresses would eliminate all the scatterlist complexity.
> 
> - AF_ALG would implement each algorithm by invoking the corresponding
>   the crypto library functions
>   (https://docs.kernel.org/crypto/libcrypto.html#api-documentation).  No
>   asynchronous execution, no buggy hardware crypto drivers, etc.

Yup!  That's exactly the idea.

> It sounds good to me.  For people who feel like the su binary on their
> system is a bit too restrictive and would like to fix that, these
> changes might not be all that great for them.  But for the rest of us,
> they should work rather well.
> 
> Of course, it'll also be a fair a bit of work, and unfortunately I also
> expect pushback from people who (incorrectly IMO) think that AF_ALG
> performance is important, even moreso than security.

If one cares about crypto offload performance, they would be better
served by creating a better interface to it than AF_ALG.  AF_ALG is
a horrible API with (presumably) tons of overhead.  I know the QAT
driver and an Nvidia BlueField DPU accelerator driver both bypass it.

Furthermore, AF_ALG only supports symmetric algorithms.  These
algorithms are inexpensive in software, so the cost of going to an
accelerator and back is enormous compared to the cost of a single
operation.  For offload to even a very fast accelerator to make sense,
one must be able to deeply pipeline requests.  However, this creates
a huge amount of additional complexity for software.

On the other hand, asymmetric cryptography performs far more work per
operation.  This might (no benchmarks!) mean that offloading asymmetric
algorithms makes more sense than offloading symmetric ones.  The cost
of sending the work to the accelerator and waiting for completion
is less than the time needed to perform the operation, so even a
synchronous interface could still be faster than running the algorithm
on the CPU.  Furthermore, long-term keys are very often asymmetric,
so DPA and EMA protections are much more likely to be relevant here.
Asymmetric accelerators also don't have a better alternative in the
form of inline encryption hardware.

I think a high performance interface to hardware cryptography (and,
more importantly, compression) would look much more like RDMA.
There would be a kernel driver that did the bare minimum to provide
isolation between userspace programs, and a userspace driver that
was responsible for abstracting over the hardware.

> Either way, the first step will be to create the algorithm allowlist,
> which should happen anyway, regardless of the other changes.

The simplest changes I can see are:

1. Get rid of zero-copy support (splice()).
2. Get rid of AIO support.
3. Only allow software implementations.

All of these are really simple.  I can send patches, but be warned
that they would only be compile-tested, as I don't know how to test
the code.

I removed oss-security from CC as this is now a Linux kernel development
discussion.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]