Re: [PATCH 14/21] crypto: x86 - add some helper macros for ECB and CBC modes

[Eric Biggers <ebiggers@kernel.org>]

On Thu, Dec 31, 2020 at 06:23:30PM +0100, Ard Biesheuvel wrote:
> The x86 glue helper module has started to show its age:
> - It relies heavily on function pointers to invoke asm helper functions that
>   operate on fixed input sizes that are relatively small. This means the
>   performance is severely impacted by retpolines.
> - It goes to great lengths to amortize the cost of kernel_fpu_begin()/end()
>   over as much work as possible, which is no longer necessary now that FPU
>   save/restore is done lazily, and doing so may cause unbounded scheduling
>   blackouts due to the fact that enabling the FPU in kernel mode disables
>   preemption.
> - The CBC mode decryption helper makes backward strides through the input, in
>   order to avoid a single block size memcpy() between chunks. Consuming the
>   input in this manner is highly likely to defeat any hardware prefetchers,
>   so it is better to go through the data linearly, and perform the extra
>   memcpy() where needed (which is turned into direct loads and stores by the
>   compiler anyway). Note that benchmarks won't show this effect, given that
>   the memory they use is always cache hot.
> 
> GCC does not seem to be smart enough to elide the indirect calls when the
> function pointers are passed as arguments to static inline helper routines
> modeled after the existing ones. So instead, let's create some CPP macros
> that encapsulate the core of the ECB and CBC processing, so we can wire
> them up for existing users of the glue helper module, i.e., Camellia,
> Serpent, Twofish and CAST6.
> 
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> ---
>  arch/x86/crypto/ecb_cbc_helpers.h | 71 ++++++++++++++++++++
>  1 file changed, 71 insertions(+)

Acked-by: Eric Biggers <ebiggers@google.com>