[PATCH 00/12] arm64: crypto: prepare for new kernel mode NEON policy

[ard.biesheuvel@linaro.org (Ard Biesheuvel)]

TL;DR: preparatory work for expected changes in arm64's handling of kernel
       mode SIMD

@Herbert: The arm64 maintainers may want to take this through the arm64 tree,
          and if not, we need their acks on patch #1. Thanks.

Currently, arm64 allows kernel mode NEON (KMN) in process, softirq or hardirq
context. In the process case, we preserve/restore the NEON context lazily,
but in the softirq/hardirq cases, we eagerly stash a slice of the NEON
register file, and immediately restore it when kernel_neon_end() is called.

Given the above, arm64 actually does not use the generic may_use_simd() API
at all*, which was added to allow async wrappers of synchronous SIMD routines
to be implemented in a generic manner. (On x86, kernel mode SIMD may be used
in process context or while serving an interrupt taken from user space. On ARM,
SIMD may only be used in process context)

When adding support for the SVE architecture extension, which shared part of
the NEON register file with the SIMD and crypto extensions, the eager preserve/
restore in interrupt context is becoming a problem: it should either preserve
and restore the entire SVE state (which may be up to 8 KB in size), or it
should not be allowed to interrupt the lazy preserve, which does need to deal
with the large SVE state anyway. Otherwise, such an interruption would corrupt
the NEON state the lazy preserve sees after the interruption.

Given how
a) KMN is never actually used in hardirq context,
b) KMN is only used in softirq context by mac80211 code running on behalf of
   WiFi devices that don't perform the crypto in hardware,
b) KMN in softirq context is statistically unlikely to interrupt the kernel
   while it is doing kernel mode NEON in process context,

the unconditional eager preserve/restore typically executes when no KMN in
process context is in progress, and we can simplify things substantially by
disallowing nested KMN, i.e., disallow KMN in hardirq context, and allow KMN
in softirq only if no KMN in process context is already in progress.

The no-nesting rule leaves only the outer SVE-aware lazy preserve/restore,
which needs to execute with bottom halves disabled, but other than that, no
intrusive changes should be needed to deal with the SVE payloads.

Given that the no-nesting rule implies that SIMD is no longer allowed in any
context, the KMN users need to be made aware of this. This series updates the
current KMN users in the arm64 tree to take may_use_simd() into account. Since
at this time, SIMD is still allowed in any context, an implementation of
may_use_simd() is added that simply returns true (#1). It will be updated in
the future when the no-nesting modifications are made.

* may_use_simd() is only used as a hint in the SHA256 NEON code, since on some
  microarchitectures, it is only marginally faster, and the eager preserve and
  restore could actually make it slower.

Ard Biesheuvel (12):
  arm64: neon: replace generic definition of may_use_simd()
  crypto: arm64/ghash-ce - add non-SIMD scalar fallback
  crypto: arm64/crct10dif - add non-SIMD generic fallback
  crypto: arm64/crc32 - add non-SIMD scalar fallback
  crypto: arm64/sha1-ce - add non-SIMD generic fallback
  crypto: arm64/sha2-ce - add non-SIMD scalar fallback
  crypto: arm64/aes-ce-cipher - match round key endianness with generic
    code
  crypto: arm64/aes-ce-cipher: add non-SIMD generic fallback
  crypto: arm64/aes-ce-ccm: add non-SIMD generic fallback
  crypto: arm64/aes-blk - add a non-SIMD fallback for synchronous CTR
  crypto: arm64/chacha20 - take may_use_simd() into account
  crypto: arm64/aes-bs - implement non-SIMD fallback for AES-CTR

 arch/arm64/crypto/Kconfig              |  22 ++-
 arch/arm64/crypto/aes-ce-ccm-core.S    |  30 ++--
 arch/arm64/crypto/aes-ce-ccm-glue.c    | 152 +++++++++++++++-----
 arch/arm64/crypto/aes-ce-cipher.c      |  55 ++++---
 arch/arm64/crypto/aes-ce.S             |  12 +-
 arch/arm64/crypto/aes-ctr-fallback.h   |  55 +++++++
 arch/arm64/crypto/aes-glue.c           |  17 ++-
 arch/arm64/crypto/aes-neonbs-glue.c    |  48 ++++++-
 arch/arm64/crypto/chacha20-neon-glue.c |   5 +-
 arch/arm64/crypto/crc32-ce-glue.c      |  11 +-
 arch/arm64/crypto/crct10dif-ce-glue.c  |  13 +-
 arch/arm64/crypto/ghash-ce-glue.c      |  49 +++++--
 arch/arm64/crypto/sha1-ce-glue.c       |  18 ++-
 arch/arm64/crypto/sha2-ce-glue.c       |  30 +++-
 arch/arm64/crypto/sha256-glue.c        |   1 +
 arch/arm64/include/asm/Kbuild          |   1 -
 arch/arm64/include/asm/simd.h          |  24 ++++
 17 files changed, 420 insertions(+), 123 deletions(-)
 create mode 100644 arch/arm64/crypto/aes-ctr-fallback.h
 create mode 100644 arch/arm64/include/asm/simd.h

-- 
2.7.4