From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99A2F1CAA92; Thu, 17 Apr 2025 06:49:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744872588; cv=none; b=K3OvcPE09mZNtgfIQ6hJceFMqFONS9eDiFQ+nZaXvJeoAbzdrjMpCQ85B9dXzT3B/baqBQspwJPruFNGnEJz6f9sR1JWvKad6knW/GcNTlpPCoJlbbVjIaRk5/D3Dbm+7xIDXD06JTGfOCXkUMgMHI2JUYLcXv4XhJlZDbOuGf0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744872588; c=relaxed/simple; bh=KAmtj40+mb5153rX4LnofGPDdYa1AOTgOJ9c8iLLZZg=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type; b=SdsW7621oBBl2pYHceRfuAl9QPhNDb5iXWbVlsT2xxyN5B1fDt1tT6u+Kw0Yocz8Fiv0bZ0JScQjGZZ9HL28utISZOHMxO4/RduJnGUvcdGH60C/MvzqhdV71wKKU78VBq653coQmZzmUq/IwykwmWHetcx+DmyURC309ojkc6s= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ML+jS5jL; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ML+jS5jL" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-225df540edcso17021625ad.0; Wed, 16 Apr 2025 23:49:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1744872586; x=1745477386; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=9YMfDtiPIBm1mjpyWsSE2L94WCWtJEZ4yISF+jitXmY=; b=ML+jS5jLCQkPaSGt5iNbyBZqQFI4ssRJJNAmtw8SLseQ4YVdsakvlyqzdVPkjVAM5t enI7avhN797F13LoarawIzRdNjppOAUCMt/RXlsIvzcXW6YELxYh16bWqJ1NwP4523NK DfWg3u86EaKsDOzKk4HGvCizxV9eibkxiUWZOEfU/mT1Vf40Sx6DwplIURZWOKuTgBcB DMJpGfz8FyeZQE0Jw19cwCVjeJZkfy87obxSo4PQw7moTY04qaCpQ2+n3wAeus5Qj3CK W7K0eUjdlErufzLFricnU/fP/nLUJcFz5OIi9eLd5mdc4J6yZcMp8vMO4F2+UuHh6h/M +nZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744872586; x=1745477386; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9YMfDtiPIBm1mjpyWsSE2L94WCWtJEZ4yISF+jitXmY=; b=vk9vUu24B11GM6yYvjoor+x+4BF6BasjQUfrXGKLC0mEXkAAwZUo4DvFP5idhw6L+w bC+nEmDTnG2/gzp1hYw8WNYQe8n7JZqQtJOQnSq8egE6OnpagwQOR50oEN17YPUmu5LI Z3DPV7EmOpr8j5ecIbHdaKU1Nfc09QxtICbd9Qpb5AW1Zcnca7iFhi/83U2K6ySp9M+Z ybpr0rAA7Ha/a2Jjci76b2B8TtU3fyX7k+mPm92Blr3MocfzKZbc2AYh3Q09xONKxmha t12TWjXMz96Yk6KV5DxR2z4gIya3ozFABmjkWMH8CJx8StkZqrAuAkyTOI+lPR4S+37T u4/A== X-Forwarded-Encrypted: i=1; AJvYcCV4ckiZOGn6UpNg6aJWmC9TK3xK1XPINaV+QIZpELKjm2hMbnHVl/bQKjbQhAoE7nigIy+Q3skl0I92nXP5@vger.kernel.org, AJvYcCVTCWlTIKk/Zz1v4jGX0LRhiGT7J4PQYkgpJeFIHRIgzL1QLDNUZZCPP4HZA9eqq1pjcIy0ygh+YbIlcpk=@vger.kernel.org X-Gm-Message-State: AOJu0Yzofi9GNMMTRI3ZZGAYIaWJgs5dDvAqGoFHEoNoSW22rPAlpbiO 3jmKJXAXrsvxhs1LEIHT4bg82mWxeAwI4MCHn6elSr8LKxJHoXkL X-Gm-Gg: ASbGncvAxefo9iENSDlSB5MlnmWNV0xt1987+/4m0SfSgYzhGg12ERlpvwnLdFWwAhB UedX+YeZCjQyW7j435d/+D2Ngvtq+Dj2AvvNqshVLKs8arKypPOU4kBr0HZCzWBQo+VJNUMF4Do mIptPTn5XFYQWWubyHb3XnSaSVQEIAVb3tqbeob/nfJ9lKiqwFL5hfNPp0/fYEuB6WDunRxgyU5 iP+3ca0pZUKVUCBRyPT2TUI+bwumaULpBw6O+nRaGm2E0Dn8h2/z43ee+zD2jw/nS3g4D5XmPrs QLMUAkIRxvD89YOQ7xqERRjSUVq+yRtaTYa9/Ux6oSAA X-Google-Smtp-Source: AGHT+IH06tnUkG6buwymrXxr/yQbpbdQ3t87juqs5171jY1Q6arc8q1JJROlVIT02INqgtBqb707oA== X-Received: by 2002:a17:902:e5c3:b0:215:44fe:163d with SMTP id d9443c01a7336-22c41d6836dmr24028295ad.17.1744872585488; Wed, 16 Apr 2025 23:49:45 -0700 (PDT) Received: from gmail.com ([116.237.135.88]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22c33ef0f41sm25978565ad.41.2025.04.16.23.49.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Apr 2025 23:49:44 -0700 (PDT) From: Qingfang Deng To: Herbert Xu , "David S. Miller" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , linux-crypto@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Christoph=20M=C3=BCllner?= , Heiko Stuebner , Qingfang Deng Subject: [RFC PATCH] crypto: riscv: scalar accelerated GHASH Date: Thu, 17 Apr 2025 14:49:38 +0800 Message-ID: <20250417064940.68469-1-dqfext@gmail.com> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From: Qingfang Deng Add a scalar implementation of GHASH for RISC-V using the Zbc (carry-less multiplication) and Zbb (bit-manipulation) extensions. This implementation is adapted from OpenSSL but rewritten in plain C for clarity. Unlike the OpenSSL one that rely on bit-reflection of the data, this version uses a pre-computed (reflected and multiplied) key, inspired by the approach used in Intel's CLMUL driver, to avoid reflections during runtime. Signed-off-by: Qingfang Deng --- arch/riscv/crypto/Kconfig | 16 +- arch/riscv/crypto/Makefile | 2 + arch/riscv/crypto/ghash-riscv64-clmul.c | 270 ++++++++++++++++++++++++ 3 files changed, 287 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/crypto/ghash-riscv64-clmul.c diff --git a/arch/riscv/crypto/Kconfig b/arch/riscv/crypto/Kconfig index 6392e1e11bc9..03b74d4116cb 100644 --- a/arch/riscv/crypto/Kconfig +++ b/arch/riscv/crypto/Kconfig @@ -26,7 +26,7 @@ config CRYPTO_CHACHA_RISCV64 default CRYPTO_LIB_CHACHA_INTERNAL config CRYPTO_GHASH_RISCV64 - tristate "Hash functions: GHASH" + tristate "Hash functions: GHASH (vector accelarated)" depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO select CRYPTO_GCM help @@ -35,6 +35,20 @@ config CRYPTO_GHASH_RISCV64 Architecture: riscv64 using: - Zvkg vector crypto extension +config CRYPTO_GHASH_RISCV64_CLMUL + tristate "Hash functions: GHASH (CLMUL scalar accelerated)" + depends on 64BIT && TOOLCHAIN_HAS_ZBB && TOOLCHAIN_HAS_ZBC + select CRYPTO_GCM + help + GCM GHASH function (NIST SP 800-38D) + + Architecture: riscv64 using: + - Zbb Bitmanipulation extension + - Zbc Carry-less multiplication + OR + - Zbkb Bit-manipulation for Cryptography + - Zbkc Carry-less multiplication for Cryptography + config CRYPTO_SHA256_RISCV64 tristate "Hash functions: SHA-224 and SHA-256" depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO diff --git a/arch/riscv/crypto/Makefile b/arch/riscv/crypto/Makefile index 247c7bc7288c..b5dc497d398c 100644 --- a/arch/riscv/crypto/Makefile +++ b/arch/riscv/crypto/Makefile @@ -10,6 +10,8 @@ chacha-riscv64-y := chacha-riscv64-glue.o chacha-riscv64-zvkb.o obj-$(CONFIG_CRYPTO_GHASH_RISCV64) += ghash-riscv64.o ghash-riscv64-y := ghash-riscv64-glue.o ghash-riscv64-zvkg.o +obj-$(CONFIG_CRYPTO_GHASH_RISCV64_CLMUL) += ghash-riscv64-clmul.o + obj-$(CONFIG_CRYPTO_SHA256_RISCV64) += sha256-riscv64.o sha256-riscv64-y := sha256-riscv64-glue.o sha256-riscv64-zvknha_or_zvknhb-zvkb.o diff --git a/arch/riscv/crypto/ghash-riscv64-clmul.c b/arch/riscv/crypto/ghash-riscv64-clmul.c new file mode 100644 index 000000000000..4777aa8e94cb --- /dev/null +++ b/arch/riscv/crypto/ghash-riscv64-clmul.c @@ -0,0 +1,270 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * GHASH using the RISC-V Zbc/Zbkc (CLMUL) extension + * + * Copyright (C) 2023 VRULL GmbH + * Author: Christoph Müllner + * + * Copyright (C) 2025 Siflower Communications Ltd + * Author: Qingfang Deng + */ + +#include +#include +#include +#include +#include +#include + +#define GHASH_MOD_POLY 0xc200000000000000 + +struct riscv64_clmul_ghash_ctx { + __uint128_t key; +}; + +struct riscv64_clmul_ghash_desc_ctx { + __uint128_t shash; + u8 buffer[GHASH_DIGEST_SIZE]; + int bytes; +}; + +static __always_inline u64 riscv_zbb_swab64(u64 val) +{ + asm (".option push\n" + ".option arch,+zbb\n" + "rev8 %0, %1\n" + ".option pop\n" + : "=r" (val) : "r" (val)); + return val; +} + +static __always_inline __uint128_t get_unaligned_be128(const u8 *p) +{ + __uint128_t val; +#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS + val = *(__uint128_t *)p; + val = riscv_zbb_swab64(val >> 64) | (__uint128_t)riscv_zbb_swab64(val) << 64; +#else + val = (__uint128_t)p[0] << 120; + val |= (__uint128_t)p[1] << 112; + val |= (__uint128_t)p[2] << 104; + val |= (__uint128_t)p[3] << 96; + val |= (__uint128_t)p[4] << 88; + val |= (__uint128_t)p[5] << 80; + val |= (__uint128_t)p[6] << 72; + val |= (__uint128_t)p[7] << 64; + val |= (__uint128_t)p[8] << 56; + val |= (__uint128_t)p[9] << 48; + val |= (__uint128_t)p[10] << 40; + val |= (__uint128_t)p[11] << 32; + val |= (__uint128_t)p[12] << 24; + val |= (__uint128_t)p[13] << 16; + val |= (__uint128_t)p[14] << 8; + val |= (__uint128_t)p[15]; +#endif + return val; +} + +static __always_inline void put_unaligned_be128(__uint128_t val, u8 *p) +{ +#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS + *(__uint128_t *)p = riscv_zbb_swab64(val >> 64) | (__uint128_t)riscv_zbb_swab64(val) << 64; +#else + p[0] = val >> 120; + p[1] = val >> 112; + p[2] = val >> 104; + p[3] = val >> 96; + p[4] = val >> 88; + p[5] = val >> 80; + p[6] = val >> 72; + p[7] = val >> 64; + p[8] = val >> 56; + p[9] = val >> 48; + p[10] = val >> 40; + p[11] = val >> 32; + p[12] = val >> 24; + p[13] = val >> 16; + p[14] = val >> 8; + p[15] = val; +#endif +} + +static __always_inline __attribute_const__ +__uint128_t clmul128(u64 a, u64 b) +{ + u64 hi, lo; + + asm(".option push\n" + ".option arch,+zbc\n" + "clmul %0, %2, %3\n" + "clmulh %1, %2, %3\n" + ".option pop\n" + : "=&r" (lo), "=&r" (hi) : "r" (a), "r" (b)); + return (__uint128_t)hi << 64 | lo; +} + +static int riscv64_clmul_ghash_init(struct shash_desc *desc) +{ + struct riscv64_clmul_ghash_desc_ctx *dctx = shash_desc_ctx(desc); + + dctx->bytes = 0; + dctx->shash = 0; + return 0; +} + +/* Compute GMULT (Xi*H mod f) using the Zbc (clmul) extensions. + * Using the no-Karatsuba approach and clmul for the final reduction. + * This results in an implementation with minimized number of instructions. + * HW with clmul latencies higher than 2 cycles might observe a performance + * improvement with Karatsuba. HW with clmul latencies higher than 6 cycles + * might observe a performance improvement with additionally converting the + * reduction to shift&xor. For a full discussion of this estimates see + * https://github.com/riscv/riscv-crypto/blob/master/doc/supp/gcm-mode-cmul.adoc + */ +static void gcm_ghash_rv64i_zbc(__uint128_t *Xi, __uint128_t k, const u8 *inp, size_t len) +{ + u64 k_hi = k >> 64, k_lo = k, p_hi, p_lo; + __uint128_t hash = *Xi, p; + + do { + __uint128_t t0, t1, t2, t3, lo, mid, hi; + + /* Load the input data, byte-reverse them, and XOR them with Xi */ + p = get_unaligned_be128(inp); + + inp += GHASH_BLOCK_SIZE; + len -= GHASH_BLOCK_SIZE; + + p ^= hash; + p_hi = p >> 64; + p_lo = p; + + /* Multiplication (without Karatsuba) */ + t0 = clmul128(p_lo, k_lo); + t1 = clmul128(p_lo, k_hi); + t2 = clmul128(p_hi, k_lo); + t3 = clmul128(p_hi, k_hi); + mid = t1 ^ t2; + lo = t0 ^ (mid << 64); + hi = t3 ^ (mid >> 64); + + /* Reduction with clmul */ + mid = clmul128(lo, GHASH_MOD_POLY); + lo ^= mid << 64; + hi ^= lo ^ (mid >> 64); + hi ^= clmul128(lo >> 64, GHASH_MOD_POLY); + hash = hi; + } while (len); + + *Xi = hash; +} + +static int riscv64_clmul_ghash_setkey(struct crypto_shash *tfm, const u8 *key, unsigned int keylen) +{ + struct riscv64_clmul_ghash_ctx *ctx = crypto_shash_ctx(tfm); + __uint128_t k; + + if (keylen != GHASH_BLOCK_SIZE) + return -EINVAL; + + k = get_unaligned_be128(key); + k = (k << 1 | k >> 127) ^ (k >> 127 ? (__uint128_t)GHASH_MOD_POLY << 64 : 0); + ctx->key = k; + + return 0; +} + +static int riscv64_clmul_ghash_update(struct shash_desc *desc, const u8 *src, unsigned int srclen) +{ + struct riscv64_clmul_ghash_ctx *ctx = crypto_shash_ctx(desc->tfm); + struct riscv64_clmul_ghash_desc_ctx *dctx = shash_desc_ctx(desc); + unsigned int len; + + if (dctx->bytes) { + if (dctx->bytes + srclen < GHASH_DIGEST_SIZE) { + memcpy(dctx->buffer + dctx->bytes, src, srclen); + dctx->bytes += srclen; + return 0; + } + memcpy(dctx->buffer + dctx->bytes, src, GHASH_DIGEST_SIZE - dctx->bytes); + + gcm_ghash_rv64i_zbc(&dctx->shash, ctx->key, dctx->buffer, GHASH_DIGEST_SIZE); + + src += GHASH_DIGEST_SIZE - dctx->bytes; + srclen -= GHASH_DIGEST_SIZE - dctx->bytes; + dctx->bytes = 0; + } + + len = round_down(srclen, GHASH_BLOCK_SIZE); + if (len) { + gcm_ghash_rv64i_zbc(&dctx->shash, ctx->key, src, len); + src += len; + srclen -= len; + } + + if (srclen) { + memcpy(dctx->buffer, src, srclen); + dctx->bytes = srclen; + } + return 0; +} + +static int riscv64_clmul_ghash_final(struct shash_desc *desc, u8 out[GHASH_DIGEST_SIZE]) +{ + struct riscv64_clmul_ghash_ctx *ctx = crypto_shash_ctx(desc->tfm); + struct riscv64_clmul_ghash_desc_ctx *dctx = shash_desc_ctx(desc); + int i; + + if (dctx->bytes) { + for (i = dctx->bytes; i < GHASH_DIGEST_SIZE; i++) + dctx->buffer[i] = 0; + gcm_ghash_rv64i_zbc(&dctx->shash, ctx->key, dctx->buffer, GHASH_DIGEST_SIZE); + dctx->bytes = 0; + } + put_unaligned_be128(dctx->shash, out); + return 0; +} + +struct shash_alg riscv64_clmul_ghash_alg = { + .init = riscv64_clmul_ghash_init, + .update = riscv64_clmul_ghash_update, + .final = riscv64_clmul_ghash_final, + .setkey = riscv64_clmul_ghash_setkey, + .descsize = sizeof(struct riscv64_clmul_ghash_desc_ctx), + .digestsize = GHASH_DIGEST_SIZE, + .base = { + .cra_blocksize = GHASH_BLOCK_SIZE, + .cra_ctxsize = sizeof(struct riscv64_clmul_ghash_ctx), + .cra_priority = 250, + .cra_name = "ghash", + .cra_driver_name = "ghash-riscv64-clmul", + .cra_module = THIS_MODULE, + }, +}; + +static int __init riscv64_clmul_ghash_mod_init(void) +{ + bool has_clmul, has_rev8; + + has_clmul = riscv_isa_extension_available(NULL, ZBC) || + riscv_isa_extension_available(NULL, ZBKC); + has_rev8 = riscv_isa_extension_available(NULL, ZBB) || + riscv_isa_extension_available(NULL, ZBKB); + if (has_clmul && has_rev8) + return crypto_register_shash(&riscv64_clmul_ghash_alg); + + return -ENODEV; +} + +static void __exit riscv64_clmul_ghash_mod_fini(void) +{ + crypto_unregister_shash(&riscv64_clmul_ghash_alg); +} + +module_init(riscv64_clmul_ghash_mod_init); +module_exit(riscv64_clmul_ghash_mod_fini); + +MODULE_DESCRIPTION("GHASH (RISC-V CLMUL accelerated)"); +MODULE_AUTHOR("Qingfang Deng "); +MODULE_LICENSE("GPL"); +MODULE_ALIAS_CRYPTO("ghash"); -- 2.43.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 01ECFC369B2 for ; Thu, 17 Apr 2025 06:56:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=Xs95knkyk88ydOH55hGDUcNUtVMGHCBZ0R/xyODckJg=; b=ZOfhsitNqhddIg v6DP/xEM4vkLWlWhR1hLPQBnU+fj+Uh55WfcCz9fowdCsim/9y7dFsm3SV5NE32N963hpbMapexjH SqE1LPHrAIhlSEdgt7frXkUoDB875jyZlA7B5JRSXBMg/AibyGEf/EcdhwOGzgbzAXb6mp6qk3NfT XnN2ZJz2+C5/8PXlm8KhmXvsQVpo18dxq/yTyu3QNOEutlwlfz9xl+LLaEYj3n78bQ8o2/wvaWO5R HwXH3oKjmUKUusyBF1cG+n+vGJ1vHBRtEcKEVtVoJiatPTgC2rQM7t16yKY2v0jw3wjAtAVhAutJq YNnLc0FK0EP5Mihm9m9w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1u5JAU-0000000Bydj-45wv; Thu, 17 Apr 2025 06:56:14 +0000 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1u5J4E-0000000BxAx-46lS for linux-riscv@lists.infradead.org; Thu, 17 Apr 2025 06:49:48 +0000 Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-227914acd20so12844795ad.1 for ; Wed, 16 Apr 2025 23:49:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1744872586; x=1745477386; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=9YMfDtiPIBm1mjpyWsSE2L94WCWtJEZ4yISF+jitXmY=; b=f0CzjD/elgqQG1Js/e3bX1QcPKHXq4YvXM7eu8CygJyJvCQdD1M1kbCMnJhGghW7Tp 0k9XzUG0QdDC9RIfk4VcREWg7ivXTPXGMfTkA+uuIDrAnDv5ylf3HhkGITLc1ZQaDlXw 7EKd7oBpK3Hy1N8XLn0yazah/cYSJaYB1o9ZATjafVpTocIz0eoufBEqaS2hAcowekXv a0jKMPAmBhTqYR66GAH2j6RPolf9RYUO8s2Fkbd8FPn7rP2OUzqfmDT0SeXqBCMkZ1+s UL3dtrZ6wjEhQJOJV6sXan5pEQsVaypapYAM+ebiSh7VHyyM5J0nJ2+n4AjRnkzBy+YF Qk1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744872586; x=1745477386; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9YMfDtiPIBm1mjpyWsSE2L94WCWtJEZ4yISF+jitXmY=; b=VeFk5tuY5zJNmRFJMXiTP6P08jFnLh/6+1FHPpmq4HHzmIDCCozSmrBuwgBPyHrtwL dtde7Be0d3Tk2x0o7hJbjVAxwiZAN+s/YYJ05Yr+LE9LtxKdXtzVZZiwwVlhGORPWOmQ ydxJf1apGqMs85QVJXiOmktBzMg8wsbMmwUBLnKnXVaW3TR3Zx/AgK++mrRezkytt+83 NPDZq340vaNDdLxy7kJFpiQwd9g2HYtLsxNn7I+QyySCBfT93c1li8RiQnsvFtoLQIQv xWpT7USatYr+FpthTpeE9AyZmjyRUjFwpZ6lmapg4h/4uMHYfw+91IsxqCuzwVI5y+IU DqKQ== X-Forwarded-Encrypted: i=1; AJvYcCVEPdMKylqIQ5BRnYg5gXDuzCItU11abj0+NkXaDB7psISBkPV2Saj0a+CVBQHASmWzhht+fko/Xz3A1Q==@lists.infradead.org X-Gm-Message-State: AOJu0YxuCsq1vhglHWy0J+aoDiRgponB/pyjt2d3LQd6H3iZc8JUaSqp 03tkEY1hAt33REDjn3sYZFPwWxaliKxp+9I6Dbophkdype0TfycC X-Gm-Gg: ASbGnctlY/d7GCydXyg1nSvsYhrcWCtH8HljD+EpTqtsY1hL/EQjJiNQ5d/UyD2AJBv 0TkbzXCkNr2eDhScAWAEZFMk2UQqbYVsaivXMg2J9orP0Wef1S26qB58V4o5DYZu9YFZIwLqu6U AMQrEZ9DiVnxs1IbwXdd/XOGz0kvLSgDCSkI8SoHXCqBig0j4J/bs7LAn8D9DBCn+akREZZgtm+ 6Bd0d1Ig1abLvwhWXu6xXKEUyit150ETTFDzXOIpPjhzFrfohyWgb5r2d2fDTkNHrjqVeAJFyMo MK1dPoePEwVS99kpnpQ7hslF+tSeKgw204HZo/MWkwKm X-Google-Smtp-Source: AGHT+IH06tnUkG6buwymrXxr/yQbpbdQ3t87juqs5171jY1Q6arc8q1JJROlVIT02INqgtBqb707oA== X-Received: by 2002:a17:902:e5c3:b0:215:44fe:163d with SMTP id d9443c01a7336-22c41d6836dmr24028295ad.17.1744872585488; Wed, 16 Apr 2025 23:49:45 -0700 (PDT) Received: from gmail.com ([116.237.135.88]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22c33ef0f41sm25978565ad.41.2025.04.16.23.49.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Apr 2025 23:49:44 -0700 (PDT) From: Qingfang Deng To: Herbert Xu , "David S. Miller" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , linux-crypto@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Christoph=20M=C3=BCllner?= , Heiko Stuebner , Qingfang Deng Subject: [RFC PATCH] crypto: riscv: scalar accelerated GHASH Date: Thu, 17 Apr 2025 14:49:38 +0800 Message-ID: <20250417064940.68469-1-dqfext@gmail.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250416_234947_044576_C680CF1F X-CRM114-Status: GOOD ( 21.53 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org RnJvbTogUWluZ2ZhbmcgRGVuZyA8cWluZ2ZhbmcuZGVuZ0BzaWZsb3dlci5jb20uY24+CgpBZGQg YSBzY2FsYXIgaW1wbGVtZW50YXRpb24gb2YgR0hBU0ggZm9yIFJJU0MtViB1c2luZyB0aGUgWmJj IChjYXJyeS1sZXNzCm11bHRpcGxpY2F0aW9uKSBhbmQgWmJiIChiaXQtbWFuaXB1bGF0aW9uKSBl eHRlbnNpb25zLiBUaGlzIGltcGxlbWVudGF0aW9uCmlzIGFkYXB0ZWQgZnJvbSBPcGVuU1NMIGJ1 dCByZXdyaXR0ZW4gaW4gcGxhaW4gQyBmb3IgY2xhcml0eS4KClVubGlrZSB0aGUgT3BlblNTTCBv bmUgdGhhdCByZWx5IG9uIGJpdC1yZWZsZWN0aW9uIG9mIHRoZSBkYXRhLCB0aGlzCnZlcnNpb24g dXNlcyBhIHByZS1jb21wdXRlZCAocmVmbGVjdGVkIGFuZCBtdWx0aXBsaWVkKSBrZXksIGluc3Bp cmVkIGJ5CnRoZSBhcHByb2FjaCB1c2VkIGluIEludGVsJ3MgQ0xNVUwgZHJpdmVyLCB0byBhdm9p ZCByZWZsZWN0aW9ucyBkdXJpbmcKcnVudGltZS4KClNpZ25lZC1vZmYtYnk6IFFpbmdmYW5nIERl bmcgPHFpbmdmYW5nLmRlbmdAc2lmbG93ZXIuY29tLmNuPgotLS0KIGFyY2gvcmlzY3YvY3J5cHRv L0tjb25maWcgICAgICAgICAgICAgICB8ICAxNiArLQogYXJjaC9yaXNjdi9jcnlwdG8vTWFrZWZp bGUgICAgICAgICAgICAgIHwgICAyICsKIGFyY2gvcmlzY3YvY3J5cHRvL2doYXNoLXJpc2N2NjQt Y2xtdWwuYyB8IDI3MCArKysrKysrKysrKysrKysrKysrKysrKysKIDMgZmlsZXMgY2hhbmdlZCwg Mjg3IGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkKIGNyZWF0ZSBtb2RlIDEwMDY0NCBhcmNo L3Jpc2N2L2NyeXB0by9naGFzaC1yaXNjdjY0LWNsbXVsLmMKCmRpZmYgLS1naXQgYS9hcmNoL3Jp c2N2L2NyeXB0by9LY29uZmlnIGIvYXJjaC9yaXNjdi9jcnlwdG8vS2NvbmZpZwppbmRleCA2Mzky ZTFlMTFiYzkuLjAzYjc0ZDQxMTZjYiAxMDA2NDQKLS0tIGEvYXJjaC9yaXNjdi9jcnlwdG8vS2Nv bmZpZworKysgYi9hcmNoL3Jpc2N2L2NyeXB0by9LY29uZmlnCkBAIC0yNiw3ICsyNiw3IEBAIGNv bmZpZyBDUllQVE9fQ0hBQ0hBX1JJU0NWNjQKIAlkZWZhdWx0IENSWVBUT19MSUJfQ0hBQ0hBX0lO VEVSTkFMCiAKIGNvbmZpZyBDUllQVE9fR0hBU0hfUklTQ1Y2NAotCXRyaXN0YXRlICJIYXNoIGZ1 bmN0aW9uczogR0hBU0giCisJdHJpc3RhdGUgIkhhc2ggZnVuY3Rpb25zOiBHSEFTSCAodmVjdG9y IGFjY2VsYXJhdGVkKSIKIAlkZXBlbmRzIG9uIDY0QklUICYmIFJJU0NWX0lTQV9WICYmIFRPT0xD SEFJTl9IQVNfVkVDVE9SX0NSWVBUTwogCXNlbGVjdCBDUllQVE9fR0NNCiAJaGVscApAQCAtMzUs NiArMzUsMjAgQEAgY29uZmlnIENSWVBUT19HSEFTSF9SSVNDVjY0CiAJICBBcmNoaXRlY3R1cmU6 IHJpc2N2NjQgdXNpbmc6CiAJICAtIFp2a2cgdmVjdG9yIGNyeXB0byBleHRlbnNpb24KIAorY29u ZmlnIENSWVBUT19HSEFTSF9SSVNDVjY0X0NMTVVMCisJdHJpc3RhdGUgIkhhc2ggZnVuY3Rpb25z OiBHSEFTSCAoQ0xNVUwgc2NhbGFyIGFjY2VsZXJhdGVkKSIKKwlkZXBlbmRzIG9uIDY0QklUICYm IFRPT0xDSEFJTl9IQVNfWkJCICYmIFRPT0xDSEFJTl9IQVNfWkJDCisJc2VsZWN0IENSWVBUT19H Q00KKwloZWxwCisJICBHQ00gR0hBU0ggZnVuY3Rpb24gKE5JU1QgU1AgODAwLTM4RCkKKworCSAg QXJjaGl0ZWN0dXJlOiByaXNjdjY0IHVzaW5nOgorCSAgLSBaYmIgQml0bWFuaXB1bGF0aW9uIGV4 dGVuc2lvbgorCSAgLSBaYmMgQ2FycnktbGVzcyBtdWx0aXBsaWNhdGlvbgorCSAgICBPUgorCSAg LSBaYmtiIEJpdC1tYW5pcHVsYXRpb24gZm9yIENyeXB0b2dyYXBoeQorCSAgLSBaYmtjIENhcnJ5 LWxlc3MgbXVsdGlwbGljYXRpb24gZm9yIENyeXB0b2dyYXBoeQorCiBjb25maWcgQ1JZUFRPX1NI QTI1Nl9SSVNDVjY0CiAJdHJpc3RhdGUgIkhhc2ggZnVuY3Rpb25zOiBTSEEtMjI0IGFuZCBTSEEt MjU2IgogCWRlcGVuZHMgb24gNjRCSVQgJiYgUklTQ1ZfSVNBX1YgJiYgVE9PTENIQUlOX0hBU19W RUNUT1JfQ1JZUFRPCmRpZmYgLS1naXQgYS9hcmNoL3Jpc2N2L2NyeXB0by9NYWtlZmlsZSBiL2Fy Y2gvcmlzY3YvY3J5cHRvL01ha2VmaWxlCmluZGV4IDI0N2M3YmM3Mjg4Yy4uYjVkYzQ5N2QzOThj IDEwMDY0NAotLS0gYS9hcmNoL3Jpc2N2L2NyeXB0by9NYWtlZmlsZQorKysgYi9hcmNoL3Jpc2N2 L2NyeXB0by9NYWtlZmlsZQpAQCAtMTAsNiArMTAsOCBAQCBjaGFjaGEtcmlzY3Y2NC15IDo9IGNo YWNoYS1yaXNjdjY0LWdsdWUubyBjaGFjaGEtcmlzY3Y2NC16dmtiLm8KIG9iai0kKENPTkZJR19D UllQVE9fR0hBU0hfUklTQ1Y2NCkgKz0gZ2hhc2gtcmlzY3Y2NC5vCiBnaGFzaC1yaXNjdjY0LXkg Oj0gZ2hhc2gtcmlzY3Y2NC1nbHVlLm8gZ2hhc2gtcmlzY3Y2NC16dmtnLm8KIAorb2JqLSQoQ09O RklHX0NSWVBUT19HSEFTSF9SSVNDVjY0X0NMTVVMKSArPSBnaGFzaC1yaXNjdjY0LWNsbXVsLm8K Kwogb2JqLSQoQ09ORklHX0NSWVBUT19TSEEyNTZfUklTQ1Y2NCkgKz0gc2hhMjU2LXJpc2N2NjQu bwogc2hhMjU2LXJpc2N2NjQteSA6PSBzaGEyNTYtcmlzY3Y2NC1nbHVlLm8gc2hhMjU2LXJpc2N2 NjQtenZrbmhhX29yX3p2a25oYi16dmtiLm8KIApkaWZmIC0tZ2l0IGEvYXJjaC9yaXNjdi9jcnlw dG8vZ2hhc2gtcmlzY3Y2NC1jbG11bC5jIGIvYXJjaC9yaXNjdi9jcnlwdG8vZ2hhc2gtcmlzY3Y2 NC1jbG11bC5jCm5ldyBmaWxlIG1vZGUgMTAwNjQ0CmluZGV4IDAwMDAwMDAwMDAwMC4uNDc3N2Fh OGU5NGNiCi0tLSAvZGV2L251bGwKKysrIGIvYXJjaC9yaXNjdi9jcnlwdG8vZ2hhc2gtcmlzY3Y2 NC1jbG11bC5jCkBAIC0wLDAgKzEsMjcwIEBACisvLyBTUERYLUxpY2Vuc2UtSWRlbnRpZmllcjog R1BMLTIuMAorLyoKKyAqIEdIQVNIIHVzaW5nIHRoZSBSSVNDLVYgWmJjL1pia2MgKENMTVVMKSBl eHRlbnNpb24KKyAqCisgKiBDb3B5cmlnaHQgKEMpIDIwMjMgVlJVTEwgR21iSAorICogQXV0aG9y OiBDaHJpc3RvcGggTcO8bGxuZXIgPGNocmlzdG9waC5tdWVsbG5lckB2cnVsbC5ldT4KKyAqCisg KiBDb3B5cmlnaHQgKEMpIDIwMjUgU2lmbG93ZXIgQ29tbXVuaWNhdGlvbnMgTHRkCisgKiBBdXRo b3I6IFFpbmdmYW5nIERlbmcgPHFpbmdmYW5nLmRlbmdAc2lmbG93ZXIuY29tLmNuPgorICovCisK KyNpbmNsdWRlIDxsaW51eC9jcnlwdG8uaD4KKyNpbmNsdWRlIDxsaW51eC9lcnIuaD4KKyNpbmNs dWRlIDxsaW51eC9tb2R1bGUuaD4KKyNpbmNsdWRlIDxsaW51eC90eXBlcy5oPgorI2luY2x1ZGUg PGNyeXB0by9naGFzaC5oPgorI2luY2x1ZGUgPGNyeXB0by9pbnRlcm5hbC9oYXNoLmg+CisKKyNk ZWZpbmUgR0hBU0hfTU9EX1BPTFkJMHhjMjAwMDAwMDAwMDAwMDAwCisKK3N0cnVjdCByaXNjdjY0 X2NsbXVsX2doYXNoX2N0eCB7CisJX191aW50MTI4X3Qga2V5OworfTsKKworc3RydWN0IHJpc2N2 NjRfY2xtdWxfZ2hhc2hfZGVzY19jdHggeworCV9fdWludDEyOF90IHNoYXNoOworCXU4IGJ1ZmZl cltHSEFTSF9ESUdFU1RfU0laRV07CisJaW50IGJ5dGVzOworfTsKKworc3RhdGljIF9fYWx3YXlz X2lubGluZSB1NjQgcmlzY3ZfemJiX3N3YWI2NCh1NjQgdmFsKQoreworCWFzbSAoIi5vcHRpb24g cHVzaFxuIgorCSAgICAgIi5vcHRpb24gYXJjaCwremJiXG4iCisJICAgICAicmV2OCAlMCwgJTFc biIKKwkgICAgICIub3B0aW9uIHBvcFxuIgorCSAgICAgOiAiPXIiICh2YWwpIDogInIiICh2YWwp KTsKKwlyZXR1cm4gdmFsOworfQorCitzdGF0aWMgX19hbHdheXNfaW5saW5lIF9fdWludDEyOF90 IGdldF91bmFsaWduZWRfYmUxMjgoY29uc3QgdTggKnApCit7CisJX191aW50MTI4X3QgdmFsOwor I2lmZGVmIENPTkZJR19IQVZFX0VGRklDSUVOVF9VTkFMSUdORURfQUNDRVNTCisJdmFsID0gKihf X3VpbnQxMjhfdCAqKXA7CisJdmFsID0gcmlzY3ZfemJiX3N3YWI2NCh2YWwgPj4gNjQpIHwgKF9f dWludDEyOF90KXJpc2N2X3piYl9zd2FiNjQodmFsKSA8PCA2NDsKKyNlbHNlCisJdmFsID0gKF9f dWludDEyOF90KXBbMF0gPDwgMTIwOworCXZhbCB8PSAoX191aW50MTI4X3QpcFsxXSA8PCAxMTI7 CisJdmFsIHw9IChfX3VpbnQxMjhfdClwWzJdIDw8IDEwNDsKKwl2YWwgfD0gKF9fdWludDEyOF90 KXBbM10gPDwgOTY7CisJdmFsIHw9IChfX3VpbnQxMjhfdClwWzRdIDw8IDg4OworCXZhbCB8PSAo X191aW50MTI4X3QpcFs1XSA8PCA4MDsKKwl2YWwgfD0gKF9fdWludDEyOF90KXBbNl0gPDwgNzI7 CisJdmFsIHw9IChfX3VpbnQxMjhfdClwWzddIDw8IDY0OworCXZhbCB8PSAoX191aW50MTI4X3Qp cFs4XSA8PCA1NjsKKwl2YWwgfD0gKF9fdWludDEyOF90KXBbOV0gPDwgNDg7CisJdmFsIHw9IChf X3VpbnQxMjhfdClwWzEwXSA8PCA0MDsKKwl2YWwgfD0gKF9fdWludDEyOF90KXBbMTFdIDw8IDMy OworCXZhbCB8PSAoX191aW50MTI4X3QpcFsxMl0gPDwgMjQ7CisJdmFsIHw9IChfX3VpbnQxMjhf dClwWzEzXSA8PCAxNjsKKwl2YWwgfD0gKF9fdWludDEyOF90KXBbMTRdIDw8IDg7CisJdmFsIHw9 IChfX3VpbnQxMjhfdClwWzE1XTsKKyNlbmRpZgorCXJldHVybiB2YWw7Cit9CisKK3N0YXRpYyBf X2Fsd2F5c19pbmxpbmUgdm9pZCBwdXRfdW5hbGlnbmVkX2JlMTI4KF9fdWludDEyOF90IHZhbCwg dTggKnApCit7CisjaWZkZWYgQ09ORklHX0hBVkVfRUZGSUNJRU5UX1VOQUxJR05FRF9BQ0NFU1MK KwkqKF9fdWludDEyOF90ICopcCA9IHJpc2N2X3piYl9zd2FiNjQodmFsID4+IDY0KSB8IChfX3Vp bnQxMjhfdClyaXNjdl96YmJfc3dhYjY0KHZhbCkgPDwgNjQ7CisjZWxzZQorCXBbMF0gPSB2YWwg Pj4gMTIwOworCXBbMV0gPSB2YWwgPj4gMTEyOworCXBbMl0gPSB2YWwgPj4gMTA0OworCXBbM10g PSB2YWwgPj4gOTY7CisJcFs0XSA9IHZhbCA+PiA4ODsKKwlwWzVdID0gdmFsID4+IDgwOworCXBb Nl0gPSB2YWwgPj4gNzI7CisJcFs3XSA9IHZhbCA+PiA2NDsKKwlwWzhdID0gdmFsID4+IDU2Owor CXBbOV0gPSB2YWwgPj4gNDg7CisJcFsxMF0gPSB2YWwgPj4gNDA7CisJcFsxMV0gPSB2YWwgPj4g MzI7CisJcFsxMl0gPSB2YWwgPj4gMjQ7CisJcFsxM10gPSB2YWwgPj4gMTY7CisJcFsxNF0gPSB2 YWwgPj4gODsKKwlwWzE1XSA9IHZhbDsKKyNlbmRpZgorfQorCitzdGF0aWMgX19hbHdheXNfaW5s aW5lIF9fYXR0cmlidXRlX2NvbnN0X18KK19fdWludDEyOF90IGNsbXVsMTI4KHU2NCBhLCB1NjQg YikKK3sKKwl1NjQgaGksIGxvOworCisJYXNtKCIub3B0aW9uIHB1c2hcbiIKKwkgICAgIi5vcHRp b24gYXJjaCwremJjXG4iCisJICAgICJjbG11bAklMCwgJTIsICUzXG4iCisJICAgICJjbG11bGgJ JTEsICUyLCAlM1xuIgorCSAgICAiLm9wdGlvbiBwb3BcbiIKKwkgICAgOiAiPSZyIiAobG8pLCAi PSZyIiAoaGkpIDogInIiIChhKSwgInIiIChiKSk7CisJcmV0dXJuIChfX3VpbnQxMjhfdCloaSA8 PCA2NCB8IGxvOworfQorCitzdGF0aWMgaW50IHJpc2N2NjRfY2xtdWxfZ2hhc2hfaW5pdChzdHJ1 Y3Qgc2hhc2hfZGVzYyAqZGVzYykKK3sKKwlzdHJ1Y3QgcmlzY3Y2NF9jbG11bF9naGFzaF9kZXNj X2N0eCAqZGN0eCA9IHNoYXNoX2Rlc2NfY3R4KGRlc2MpOworCisJZGN0eC0+Ynl0ZXMgPSAwOwor CWRjdHgtPnNoYXNoID0gMDsKKwlyZXR1cm4gMDsKK30KKworLyogQ29tcHV0ZSBHTVVMVCAoWGkq SCBtb2QgZikgdXNpbmcgdGhlIFpiYyAoY2xtdWwpIGV4dGVuc2lvbnMuCisgKiBVc2luZyB0aGUg bm8tS2FyYXRzdWJhIGFwcHJvYWNoIGFuZCBjbG11bCBmb3IgdGhlIGZpbmFsIHJlZHVjdGlvbi4K KyAqIFRoaXMgcmVzdWx0cyBpbiBhbiBpbXBsZW1lbnRhdGlvbiB3aXRoIG1pbmltaXplZCBudW1i ZXIgb2YgaW5zdHJ1Y3Rpb25zLgorICogSFcgd2l0aCBjbG11bCBsYXRlbmNpZXMgaGlnaGVyIHRo YW4gMiBjeWNsZXMgbWlnaHQgb2JzZXJ2ZSBhIHBlcmZvcm1hbmNlCisgKiBpbXByb3ZlbWVudCB3 aXRoIEthcmF0c3ViYS4gSFcgd2l0aCBjbG11bCBsYXRlbmNpZXMgaGlnaGVyIHRoYW4gNiBjeWNs ZXMKKyAqIG1pZ2h0IG9ic2VydmUgYSBwZXJmb3JtYW5jZSBpbXByb3ZlbWVudCB3aXRoIGFkZGl0 aW9uYWxseSBjb252ZXJ0aW5nIHRoZQorICogcmVkdWN0aW9uIHRvIHNoaWZ0Jnhvci4gRm9yIGEg ZnVsbCBkaXNjdXNzaW9uIG9mIHRoaXMgZXN0aW1hdGVzIHNlZQorICogaHR0cHM6Ly9naXRodWIu Y29tL3Jpc2N2L3Jpc2N2LWNyeXB0by9ibG9iL21hc3Rlci9kb2Mvc3VwcC9nY20tbW9kZS1jbXVs LmFkb2MKKyAqLworc3RhdGljIHZvaWQgZ2NtX2doYXNoX3J2NjRpX3piYyhfX3VpbnQxMjhfdCAq WGksIF9fdWludDEyOF90IGssIGNvbnN0IHU4ICppbnAsIHNpemVfdCBsZW4pCit7CisJdTY0IGtf aGkgPSBrID4+IDY0LCBrX2xvID0gaywgcF9oaSwgcF9sbzsKKwlfX3VpbnQxMjhfdCBoYXNoID0g KlhpLCBwOworCisJZG8geworCQlfX3VpbnQxMjhfdCB0MCwgdDEsIHQyLCB0MywgbG8sIG1pZCwg aGk7CisKKwkJLyogTG9hZCB0aGUgaW5wdXQgZGF0YSwgYnl0ZS1yZXZlcnNlIHRoZW0sIGFuZCBY T1IgdGhlbSB3aXRoIFhpICovCisJCXAgPSBnZXRfdW5hbGlnbmVkX2JlMTI4KGlucCk7CisKKwkJ aW5wICs9IEdIQVNIX0JMT0NLX1NJWkU7CisJCWxlbiAtPSBHSEFTSF9CTE9DS19TSVpFOworCisJ CXAgXj0gaGFzaDsKKwkJcF9oaSA9IHAgPj4gNjQ7CisJCXBfbG8gPSBwOworCisJCS8qIE11bHRp cGxpY2F0aW9uICh3aXRob3V0IEthcmF0c3ViYSkgKi8KKwkJdDAgPSBjbG11bDEyOChwX2xvLCBr X2xvKTsKKwkJdDEgPSBjbG11bDEyOChwX2xvLCBrX2hpKTsKKwkJdDIgPSBjbG11bDEyOChwX2hp LCBrX2xvKTsKKwkJdDMgPSBjbG11bDEyOChwX2hpLCBrX2hpKTsKKwkJbWlkID0gdDEgXiB0MjsK KwkJbG8gPSB0MCBeIChtaWQgPDwgNjQpOworCQloaSA9IHQzIF4gKG1pZCA+PiA2NCk7CisKKwkJ LyogUmVkdWN0aW9uIHdpdGggY2xtdWwgKi8KKwkJbWlkID0gY2xtdWwxMjgobG8sIEdIQVNIX01P RF9QT0xZKTsKKwkJbG8gXj0gbWlkIDw8IDY0OworCQloaSBePSBsbyBeIChtaWQgPj4gNjQpOwor CQloaSBePSBjbG11bDEyOChsbyA+PiA2NCwgR0hBU0hfTU9EX1BPTFkpOworCQloYXNoID0gaGk7 CisJfSB3aGlsZSAobGVuKTsKKworCSpYaSA9IGhhc2g7Cit9CisKK3N0YXRpYyBpbnQgcmlzY3Y2 NF9jbG11bF9naGFzaF9zZXRrZXkoc3RydWN0IGNyeXB0b19zaGFzaCAqdGZtLCBjb25zdCB1OCAq a2V5LCB1bnNpZ25lZCBpbnQga2V5bGVuKQoreworCXN0cnVjdCByaXNjdjY0X2NsbXVsX2doYXNo X2N0eCAqY3R4ID0gY3J5cHRvX3NoYXNoX2N0eCh0Zm0pOworCV9fdWludDEyOF90IGs7CisKKwlp ZiAoa2V5bGVuICE9IEdIQVNIX0JMT0NLX1NJWkUpCisJCXJldHVybiAtRUlOVkFMOworCisJayA9 IGdldF91bmFsaWduZWRfYmUxMjgoa2V5KTsKKwlrID0gKGsgPDwgMSB8IGsgPj4gMTI3KSBeIChr ID4+IDEyNyA/IChfX3VpbnQxMjhfdClHSEFTSF9NT0RfUE9MWSA8PCA2NCA6IDApOworCWN0eC0+ a2V5ID0gazsKKworCXJldHVybiAwOworfQorCitzdGF0aWMgaW50IHJpc2N2NjRfY2xtdWxfZ2hh c2hfdXBkYXRlKHN0cnVjdCBzaGFzaF9kZXNjICpkZXNjLCBjb25zdCB1OCAqc3JjLCB1bnNpZ25l ZCBpbnQgc3JjbGVuKQoreworCXN0cnVjdCByaXNjdjY0X2NsbXVsX2doYXNoX2N0eCAqY3R4ID0g Y3J5cHRvX3NoYXNoX2N0eChkZXNjLT50Zm0pOworCXN0cnVjdCByaXNjdjY0X2NsbXVsX2doYXNo X2Rlc2NfY3R4ICpkY3R4ID0gc2hhc2hfZGVzY19jdHgoZGVzYyk7CisJdW5zaWduZWQgaW50IGxl bjsKKworCWlmIChkY3R4LT5ieXRlcykgeworCQlpZiAoZGN0eC0+Ynl0ZXMgKyBzcmNsZW4gPCBH SEFTSF9ESUdFU1RfU0laRSkgeworCQkJbWVtY3B5KGRjdHgtPmJ1ZmZlciArIGRjdHgtPmJ5dGVz LCBzcmMsIHNyY2xlbik7CisJCQlkY3R4LT5ieXRlcyArPSBzcmNsZW47CisJCQlyZXR1cm4gMDsK KwkJfQorCQltZW1jcHkoZGN0eC0+YnVmZmVyICsgZGN0eC0+Ynl0ZXMsIHNyYywgR0hBU0hfRElH RVNUX1NJWkUgLSBkY3R4LT5ieXRlcyk7CisKKwkJZ2NtX2doYXNoX3J2NjRpX3piYygmZGN0eC0+ c2hhc2gsIGN0eC0+a2V5LCBkY3R4LT5idWZmZXIsIEdIQVNIX0RJR0VTVF9TSVpFKTsKKworCQlz cmMgKz0gR0hBU0hfRElHRVNUX1NJWkUgLSBkY3R4LT5ieXRlczsKKwkJc3JjbGVuIC09IEdIQVNI X0RJR0VTVF9TSVpFIC0gZGN0eC0+Ynl0ZXM7CisJCWRjdHgtPmJ5dGVzID0gMDsKKwl9CisKKwls ZW4gPSByb3VuZF9kb3duKHNyY2xlbiwgR0hBU0hfQkxPQ0tfU0laRSk7CisJaWYgKGxlbikgewor CQlnY21fZ2hhc2hfcnY2NGlfemJjKCZkY3R4LT5zaGFzaCwgY3R4LT5rZXksIHNyYywgbGVuKTsK KwkJc3JjICs9IGxlbjsKKwkJc3JjbGVuIC09IGxlbjsKKwl9CisKKwlpZiAoc3JjbGVuKSB7CisJ CW1lbWNweShkY3R4LT5idWZmZXIsIHNyYywgc3JjbGVuKTsKKwkJZGN0eC0+Ynl0ZXMgPSBzcmNs ZW47CisJfQorCXJldHVybiAwOworfQorCitzdGF0aWMgaW50IHJpc2N2NjRfY2xtdWxfZ2hhc2hf ZmluYWwoc3RydWN0IHNoYXNoX2Rlc2MgKmRlc2MsIHU4IG91dFtHSEFTSF9ESUdFU1RfU0laRV0p Cit7CisJc3RydWN0IHJpc2N2NjRfY2xtdWxfZ2hhc2hfY3R4ICpjdHggPSBjcnlwdG9fc2hhc2hf Y3R4KGRlc2MtPnRmbSk7CisJc3RydWN0IHJpc2N2NjRfY2xtdWxfZ2hhc2hfZGVzY19jdHggKmRj dHggPSBzaGFzaF9kZXNjX2N0eChkZXNjKTsKKwlpbnQgaTsKKworCWlmIChkY3R4LT5ieXRlcykg eworCQlmb3IgKGkgPSBkY3R4LT5ieXRlczsgaSA8IEdIQVNIX0RJR0VTVF9TSVpFOyBpKyspCisJ CQlkY3R4LT5idWZmZXJbaV0gPSAwOworCQlnY21fZ2hhc2hfcnY2NGlfemJjKCZkY3R4LT5zaGFz aCwgY3R4LT5rZXksIGRjdHgtPmJ1ZmZlciwgR0hBU0hfRElHRVNUX1NJWkUpOworCQlkY3R4LT5i eXRlcyA9IDA7CisJfQorCXB1dF91bmFsaWduZWRfYmUxMjgoZGN0eC0+c2hhc2gsIG91dCk7CisJ cmV0dXJuIDA7Cit9CisKK3N0cnVjdCBzaGFzaF9hbGcgcmlzY3Y2NF9jbG11bF9naGFzaF9hbGcg PSB7CisJLmluaXQgPSByaXNjdjY0X2NsbXVsX2doYXNoX2luaXQsCisJLnVwZGF0ZSA9IHJpc2N2 NjRfY2xtdWxfZ2hhc2hfdXBkYXRlLAorCS5maW5hbCA9IHJpc2N2NjRfY2xtdWxfZ2hhc2hfZmlu YWwsCisJLnNldGtleSA9IHJpc2N2NjRfY2xtdWxfZ2hhc2hfc2V0a2V5LAorCS5kZXNjc2l6ZSA9 IHNpemVvZihzdHJ1Y3QgcmlzY3Y2NF9jbG11bF9naGFzaF9kZXNjX2N0eCksCisJLmRpZ2VzdHNp emUgPSBHSEFTSF9ESUdFU1RfU0laRSwKKwkuYmFzZSA9IHsKKwkJIC5jcmFfYmxvY2tzaXplID0g R0hBU0hfQkxPQ0tfU0laRSwKKwkJIC5jcmFfY3R4c2l6ZSA9IHNpemVvZihzdHJ1Y3QgcmlzY3Y2 NF9jbG11bF9naGFzaF9jdHgpLAorCQkgLmNyYV9wcmlvcml0eSA9IDI1MCwKKwkJIC5jcmFfbmFt ZSA9ICJnaGFzaCIsCisJCSAuY3JhX2RyaXZlcl9uYW1lID0gImdoYXNoLXJpc2N2NjQtY2xtdWwi LAorCQkgLmNyYV9tb2R1bGUgPSBUSElTX01PRFVMRSwKKwl9LAorfTsKKworc3RhdGljIGludCBf X2luaXQgcmlzY3Y2NF9jbG11bF9naGFzaF9tb2RfaW5pdCh2b2lkKQoreworCWJvb2wgaGFzX2Ns bXVsLCBoYXNfcmV2ODsKKworCWhhc19jbG11bCA9IHJpc2N2X2lzYV9leHRlbnNpb25fYXZhaWxh YmxlKE5VTEwsIFpCQykgfHwKKwkJICAgIHJpc2N2X2lzYV9leHRlbnNpb25fYXZhaWxhYmxlKE5V TEwsIFpCS0MpOworCWhhc19yZXY4ID0gcmlzY3ZfaXNhX2V4dGVuc2lvbl9hdmFpbGFibGUoTlVM TCwgWkJCKSB8fAorCQkgICByaXNjdl9pc2FfZXh0ZW5zaW9uX2F2YWlsYWJsZShOVUxMLCBaQktC KTsKKwlpZiAoaGFzX2NsbXVsICYmIGhhc19yZXY4KQorCQlyZXR1cm4gY3J5cHRvX3JlZ2lzdGVy X3NoYXNoKCZyaXNjdjY0X2NsbXVsX2doYXNoX2FsZyk7CisKKwlyZXR1cm4gLUVOT0RFVjsKK30K Kworc3RhdGljIHZvaWQgX19leGl0IHJpc2N2NjRfY2xtdWxfZ2hhc2hfbW9kX2Zpbmkodm9pZCkK K3sKKwljcnlwdG9fdW5yZWdpc3Rlcl9zaGFzaCgmcmlzY3Y2NF9jbG11bF9naGFzaF9hbGcpOwor fQorCittb2R1bGVfaW5pdChyaXNjdjY0X2NsbXVsX2doYXNoX21vZF9pbml0KTsKK21vZHVsZV9l eGl0KHJpc2N2NjRfY2xtdWxfZ2hhc2hfbW9kX2ZpbmkpOworCitNT0RVTEVfREVTQ1JJUFRJT04o IkdIQVNIIChSSVNDLVYgQ0xNVUwgYWNjZWxlcmF0ZWQpIik7CitNT0RVTEVfQVVUSE9SKCJRaW5n ZmFuZyBEZW5nIDxkcWZleHRAZ21haWwuY29tPiIpOworTU9EVUxFX0xJQ0VOU0UoIkdQTCIpOwor TU9EVUxFX0FMSUFTX0NSWVBUTygiZ2hhc2giKTsKLS0gCjIuNDMuMAoKCl9fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LXJpc2N2IG1haWxpbmcgbGlz dApsaW51eC1yaXNjdkBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQu b3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtcmlzY3YK