From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDD7BC10F03 for ; Fri, 22 Mar 2019 06:31:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 96F3721917 for ; Fri, 22 Mar 2019 06:31:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727717AbfCVGb2 (ORCPT ); Fri, 22 Mar 2019 02:31:28 -0400 Received: from orcrist.hmeau.com ([104.223.48.154]:44096 "EHLO deadmen.hmeau.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726029AbfCVGb2 (ORCPT ); Fri, 22 Mar 2019 02:31:28 -0400 Received: from gondobar.mordor.me.apana.org.au ([192.168.128.4] helo=gondobar) by deadmen.hmeau.com with esmtps (Exim 4.89 #2 (Debian)) id 1h7DgU-0002YL-CL; Fri, 22 Mar 2019 14:29:42 +0800 Received: from herbert by gondobar with local (Exim 4.89) (envelope-from ) id 1h7DgU-0001Hl-4u; Fri, 22 Mar 2019 14:29:42 +0800 Subject: [PATCH 5/17] zinc: Add x86 accelerated ChaCha20 References: <20190322062740.nrwfx2rvmt7lzotj@gondor.apana.org.au> To: Linus Torvalds , "David S. Miller" , "Jason A. Donenfeld" , Eric Biggers , Ard Biesheuvel , Linux Crypto Mailing List , linux-fscrypt@vger.kernel.org, linux-arm-kernel@lists.infradead.org, LKML , Paul Crowley , Greg Kaiser , Samuel Neves , Tomer Ashur , Martin Willi Message-Id: From: Herbert Xu Date: Fri, 22 Mar 2019 14:29:42 +0800 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org This patch exposes the crypto API x86 chacha20 implementation through zinc. Signed-off-by: Herbert Xu --- lib/zinc/Kconfig | 1 lib/zinc/chacha20/chacha20-x86_64-glue.c | 55 +++++++++++++++++++++++++++++++ lib/zinc/chacha20/chacha20.c | 4 ++ 3 files changed, 60 insertions(+) diff --git a/lib/zinc/Kconfig b/lib/zinc/Kconfig index 1fffd0a1a74c..010547fa6c9d 100644 --- a/lib/zinc/Kconfig +++ b/lib/zinc/Kconfig @@ -1,6 +1,7 @@ config ZINC_CHACHA20 tristate select CRYPTO_CHACHA20 + select CRYPTO_CHACHA20_X86_64 if ZINC_ARCH_X86_64 config ZINC_SELFTEST bool "Zinc cryptography library self-tests" diff --git a/lib/zinc/chacha20/chacha20-x86_64-glue.c b/lib/zinc/chacha20/chacha20-x86_64-glue.c new file mode 100644 index 000000000000..e0b387b09c04 --- /dev/null +++ b/lib/zinc/chacha20/chacha20-x86_64-glue.c @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/* + * Copyright (C) 2015-2018 Jason A. Donenfeld . All Rights Reserved. + */ + +#include +#include +#include +#include +#include + +static bool chacha20_use_ssse3 __ro_after_init; +static bool *const chacha20_nobs[] __initconst = { + &chacha20_use_ssse3 }; + +static void __init chacha20_fpu_init(void) +{ + chacha20_use_ssse3 = boot_cpu_has(X86_FEATURE_SSSE3); +} + +static inline bool chacha20_arch(struct chacha20_ctx *ctx, u8 *dst, + const u8 *src, size_t len, + simd_context_t *simd_context) +{ + /* SIMD disables preemption, so relax after processing each page. */ + BUILD_BUG_ON(PAGE_SIZE < CHACHA20_BLOCK_SIZE || + PAGE_SIZE % CHACHA20_BLOCK_SIZE); + + if (!IS_ENABLED(CONFIG_AS_SSSE3) || !chacha20_use_ssse3 || + len <= CHACHA20_BLOCK_SIZE || !simd_use(simd_context)) + return false; + + for (;;) { + const size_t bytes = min_t(size_t, len, PAGE_SIZE); + + crypto_chacha_dosimd(ctx->state, dst, src, bytes, 20); + + len -= bytes; + if (!len) + break; + dst += bytes; + src += bytes; + simd_relax(simd_context); + } + + return true; +} + +static inline bool hchacha20_arch(u32 derived_key[CHACHA20_KEY_WORDS], + const u8 nonce[HCHACHA20_NONCE_SIZE], + const u8 key[HCHACHA20_KEY_SIZE], + simd_context_t *simd_context) +{ + return false; +} diff --git a/lib/zinc/chacha20/chacha20.c b/lib/zinc/chacha20/chacha20.c index cfbdaf4384dc..c84fe504623f 100644 --- a/lib/zinc/chacha20/chacha20.c +++ b/lib/zinc/chacha20/chacha20.c @@ -17,6 +17,9 @@ #include // For crypto_xor_cpy. #include +#if defined(CONFIG_ZINC_ARCH_X86_64) +#include "chacha20-x86_64-glue.c" +#else static bool *const chacha20_nobs[] __initconst = { }; static void __init chacha20_fpu_init(void) { @@ -34,6 +37,7 @@ static inline bool hchacha20_arch(u32 derived_key[CHACHA20_KEY_WORDS], { return false; } +#endif void chacha20(struct chacha20_ctx *ctx, u8 *dst, const u8 *src, u32 len, simd_context_t *simd_context)