From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1BBE381B9; Fri, 5 Jan 2024 18:51:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XatPJLOQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CA4E2C433C9; Fri, 5 Jan 2024 18:51:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704480693; bh=KWqoN9K4c+36n+xRQWs4t4tC8x2ENilxLlOhbN7ScK4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XatPJLOQBbuNUYQayOldPhAd+Y9CDrMh2wy7VucsS6AB4k8R0yrgdJFCRe2PeU1YW JJLealACQihSwGZFEg+74xAiUSHz2HMbKN38ucE0pNaDg8UMDDUFmfjwHLsgf6/c2T wCsZpsXiol+JdIFUNl6n/kscYYJXV5YzZSpAfTOCqXnvnMdvgcq2xiTMbe8C4T0p9g MX4SqsiezYNLadU36ELkt2DIT1oAr1TKGqxP1pSyXd5t55BHAXj7a3/ISRFrmjRcYw wjYAWndmhruS9jAsJeKcFVKL+zXZzLQR5g0poZRSIA6mHIn0jZmEYDZDJeuCNwLl1F qRJa/HrWHlsfg== From: Eric Biggers To: linux-crypto@vger.kernel.org, linux-riscv@lists.infradead.org, Jerry Shih Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Heiko Stuebner , Phoebe Chen , hongrong.hsu@sifive.com, Paul Walmsley , Palmer Dabbelt , Albert Ou , Andy Chiu , Charalampos Mitrodimas , Heiko Stuebner Subject: [PATCH v2 10/12] crypto: riscv - add vector crypto accelerated SHA-{512,384} Date: Fri, 5 Jan 2024 10:49:46 -0800 Message-ID: <20240105184950.43181-11-ebiggers@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240105184950.43181-1-ebiggers@kernel.org> References: <20240105184950.43181-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From: Jerry Shih Add an implementation of SHA-512 and SHA-384 using the Zvknhb extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and greatly reducing code duplication. Co-developed-by: Charalampos Mitrodimas Signed-off-by: Charalampos Mitrodimas Co-developed-by: Heiko Stuebner Signed-off-by: Heiko Stuebner Co-developed-by: Phoebe Chen Signed-off-by: Phoebe Chen Signed-off-by: Jerry Shih Co-developed-by: Eric Biggers Signed-off-by: Eric Biggers --- arch/riscv/crypto/Kconfig | 11 + arch/riscv/crypto/Makefile | 3 + arch/riscv/crypto/sha512-riscv64-glue.c | 133 ++++++++++++ .../riscv/crypto/sha512-riscv64-zvknhb-zvkb.S | 203 ++++++++++++++++++ 4 files changed, 350 insertions(+) create mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha512-riscv64-zvknhb-zvkb.S diff --git a/arch/riscv/crypto/Kconfig b/arch/riscv/crypto/Kconfig index 687dbb71f7d50..0fd0bf46c9099 100644 --- a/arch/riscv/crypto/Kconfig +++ b/arch/riscv/crypto/Kconfig @@ -43,11 +43,22 @@ config CRYPTO_SHA256_RISCV64 tristate "Hash functions: SHA-224 and SHA-256" depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO select CRYPTO_SHA256 help SHA-224 and SHA-256 secure hash algorithm (FIPS 180) Architecture: riscv64 using: - Zvknha or Zvknhb vector crypto extensions - Zvkb vector crypto extension +config CRYPTO_SHA512_RISCV64 + tristate "Hash functions: SHA-384 and SHA-512" + depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO + select CRYPTO_SHA512 + help + SHA-384 and SHA-512 secure hash algorithm (FIPS 180) + + Architecture: riscv64 using: + - Zvknhb vector crypto extension + - Zvkb vector crypto extension + endmenu diff --git a/arch/riscv/crypto/Makefile b/arch/riscv/crypto/Makefile index 8674e9829cdb1..8dbfda1918990 100644 --- a/arch/riscv/crypto/Makefile +++ b/arch/riscv/crypto/Makefile @@ -8,10 +8,13 @@ aes-riscv64-y := aes-riscv64-glue.o aes-riscv64-zvkned.o \ aes-riscv64-zvkned-zvbb-zvkg.o aes-riscv64-zvkned-zvkb.o obj-$(CONFIG_CRYPTO_CHACHA_RISCV64) += chacha-riscv64.o chacha-riscv64-y := chacha-riscv64-glue.o chacha-riscv64-zvkb.o obj-$(CONFIG_CRYPTO_GHASH_RISCV64) += ghash-riscv64.o ghash-riscv64-y := ghash-riscv64-glue.o ghash-riscv64-zvkg.o obj-$(CONFIG_CRYPTO_SHA256_RISCV64) += sha256-riscv64.o sha256-riscv64-y := sha256-riscv64-glue.o sha256-riscv64-zvknha_or_zvknhb-zvkb.o + +obj-$(CONFIG_CRYPTO_SHA512_RISCV64) += sha512-riscv64.o +sha512-riscv64-y := sha512-riscv64-glue.o sha512-riscv64-zvknhb-zvkb.o diff --git a/arch/riscv/crypto/sha512-riscv64-glue.c b/arch/riscv/crypto/sha512-riscv64-glue.c new file mode 100644 index 0000000000000..43b56a08aeb51 --- /dev/null +++ b/arch/riscv/crypto/sha512-riscv64-glue.c @@ -0,0 +1,133 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * SHA-512 and SHA-384 using the RISC-V vector crypto extensions + * + * Copyright (C) 2023 VRULL GmbH + * Author: Heiko Stuebner + * + * Copyright (C) 2023 SiFive, Inc. + * Author: Jerry Shih + */ + +#include +#include +#include +#include +#include +#include +#include + +/* + * Note: the asm function only uses the 'state' field of struct sha512_state. + * It is assumed to be the first field. + */ +asmlinkage void sha512_transform_zvknhb_zvkb( + struct sha512_state *state, const u8 *data, int num_blocks); + +static int riscv64_sha512_update(struct shash_desc *desc, const u8 *data, + unsigned int len) +{ + /* + * Ensure struct sha512_state begins directly with the SHA-512 + * 512-bit internal state, as this is what the asm function expects. + */ + BUILD_BUG_ON(offsetof(struct sha512_state, state) != 0); + + if (crypto_simd_usable()) { + kernel_vector_begin(); + sha512_base_do_update(desc, data, len, + sha512_transform_zvknhb_zvkb); + kernel_vector_end(); + } else { + crypto_sha512_update(desc, data, len); + } + return 0; +} + +static int riscv64_sha512_finup(struct shash_desc *desc, const u8 *data, + unsigned int len, u8 *out) +{ + if (crypto_simd_usable()) { + kernel_vector_begin(); + if (len) + sha512_base_do_update(desc, data, len, + sha512_transform_zvknhb_zvkb); + sha512_base_do_finalize(desc, sha512_transform_zvknhb_zvkb); + kernel_vector_end(); + + return sha512_base_finish(desc, out); + } + + return crypto_sha512_finup(desc, data, len, out); +} + +static int riscv64_sha512_final(struct shash_desc *desc, u8 *out) +{ + return riscv64_sha512_finup(desc, NULL, 0, out); +} + +static int riscv64_sha512_digest(struct shash_desc *desc, const u8 *data, + unsigned int len, u8 *out) +{ + return sha512_base_init(desc) ?: + riscv64_sha512_finup(desc, data, len, out); +} + +static struct shash_alg riscv64_sha512_algs[] = { + { + .init = sha512_base_init, + .update = riscv64_sha512_update, + .final = riscv64_sha512_final, + .finup = riscv64_sha512_finup, + .digest = riscv64_sha512_digest, + .descsize = sizeof(struct sha512_state), + .digestsize = SHA512_DIGEST_SIZE, + .base = { + .cra_blocksize = SHA512_BLOCK_SIZE, + .cra_priority = 300, + .cra_name = "sha512", + .cra_driver_name = "sha512-riscv64-zvknhb-zvkb", + .cra_module = THIS_MODULE, + }, + }, { + .init = sha384_base_init, + .update = riscv64_sha512_update, + .final = riscv64_sha512_final, + .finup = riscv64_sha512_finup, + .descsize = sizeof(struct sha512_state), + .digestsize = SHA384_DIGEST_SIZE, + .base = { + .cra_blocksize = SHA384_BLOCK_SIZE, + .cra_priority = 300, + .cra_name = "sha384", + .cra_driver_name = "sha384-riscv64-zvknhb-zvkb", + .cra_module = THIS_MODULE, + }, + }, +}; + +static int __init riscv64_sha512_mod_init(void) +{ + if (riscv_isa_extension_available(NULL, ZVKNHB) && + riscv_isa_extension_available(NULL, ZVKB) && + riscv_vector_vlen() >= 128) + return crypto_register_shashes(riscv64_sha512_algs, + ARRAY_SIZE(riscv64_sha512_algs)); + + return -ENODEV; +} + +static void __exit riscv64_sha512_mod_exit(void) +{ + crypto_unregister_shashes(riscv64_sha512_algs, + ARRAY_SIZE(riscv64_sha512_algs)); +} + +module_init(riscv64_sha512_mod_init); +module_exit(riscv64_sha512_mod_exit); + +MODULE_DESCRIPTION("SHA-512 (RISC-V accelerated)"); +MODULE_AUTHOR("Heiko Stuebner "); +MODULE_LICENSE("GPL"); +MODULE_ALIAS_CRYPTO("sha512"); +MODULE_ALIAS_CRYPTO("sha384"); diff --git a/arch/riscv/crypto/sha512-riscv64-zvknhb-zvkb.S b/arch/riscv/crypto/sha512-riscv64-zvknhb-zvkb.S new file mode 100644 index 0000000000000..3a9ae210f9158 --- /dev/null +++ b/arch/riscv/crypto/sha512-riscv64-zvknhb-zvkb.S @@ -0,0 +1,203 @@ +/* SPDX-License-Identifier: Apache-2.0 OR BSD-2-Clause */ +// +// This file is dual-licensed, meaning that you can use it under your +// choice of either of the following two licenses: +// +// Copyright 2023 The OpenSSL Project Authors. All Rights Reserved. +// +// Licensed under the Apache License 2.0 (the "License"). You can obtain +// a copy in the file LICENSE in the source distribution or at +// https://www.openssl.org/source/license.html +// +// or +// +// Copyright (c) 2023, Christoph Müllner +// Copyright (c) 2023, Phoebe Chen +// Copyright 2024 Google LLC +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// The generated code of this file depends on the following RISC-V extensions: +// - RV64I +// - RISC-V Vector ('V') with VLEN >= 128 +// - RISC-V Vector SHA-2 Secure Hash extension ('Zvknhb') +// - RISC-V Vector Cryptography Bit-manipulation extension ('Zvkb') + +#include + +.text +.option arch, +zvknhb, +zvkb + +#define STATEP a0 +#define DATA a1 +#define NUM_BLOCKS a2 + +#define STATEP_C a3 +#define K a4 + +#define MASK v0 +#define INDICES v1 +#define W0 v10 // LMUL=2 +#define W1 v12 // LMUL=2 +#define W2 v14 // LMUL=2 +#define W3 v16 // LMUL=2 +#define VTMP v20 // LMUL=2 +#define FEBA v22 // LMUL=2 +#define HGDC v24 // LMUL=2 +#define PREV_FEBA v26 // LMUL=2 +#define PREV_HGDC v28 // LMUL=2 + +// Do 4 rounds of SHA-512. w0 contains the current 4 message schedule words. +// +// If not all the message schedule words have been computed yet, then this also +// computes 4 more message schedule words. w1-w3 contain the next 3 groups of 4 +// message schedule words; this macro computes the group after w3 and writes it +// to w0. This means that the next (w0, w1, w2, w3) is the current (w1, w2, w3, +// w0), so the caller must cycle through the registers accordingly. +.macro sha512_4rounds last, w0, w1, w2, w3 + vle64.v VTMP, (K) + addi K, K, 32 + vadd.vv VTMP, VTMP, \w0 + vsha2cl.vv HGDC, FEBA, VTMP + vsha2ch.vv FEBA, HGDC, VTMP +.if !\last + vmerge.vvm VTMP, \w2, \w1, MASK + vsha2ms.vv \w0, VTMP, \w3 +.endif +.endm + +.macro sha512_16rounds last + sha512_4rounds \last, W0, W1, W2, W3 + sha512_4rounds \last, W1, W2, W3, W0 + sha512_4rounds \last, W2, W3, W0, W1 + sha512_4rounds \last, W3, W0, W1, W2 +.endm + +// void sha512_transform_zvknhb_zvkb(u64 state[8], const u8 *data, +// int num_blocks); +SYM_TYPED_FUNC_START(sha512_transform_zvknhb_zvkb) + + // Setup mask for the vmerge to replace the first word (idx==0) in + // message scheduling. There are 4 words, so an 8-bit mask suffices. + vsetivli zero, 1, e8, m1, ta, ma + vmv.v.i MASK, 0x01 + + // Load the state. The state is stored as {a,b,c,d,e,f,g,h}, but we + // need {f,e,b,a},{h,g,d,c}. The dst vtype is e64m2 and the index vtype + // is e8mf4. We use index-load with the i8 indices {40, 32, 8, 0}, + // loaded using the 32-bit little endian value 0x00082028. + li t0, 0x00082028 + vsetivli zero, 1, e32, m1, ta, ma + vmv.v.x INDICES, t0 + addi STATEP_C, STATEP, 16 + vsetivli zero, 4, e64, m2, ta, ma + vluxei8.v FEBA, (STATEP), INDICES + vluxei8.v HGDC, (STATEP_C), INDICES + +.Lnext_block: + la K, K512 + addi NUM_BLOCKS, NUM_BLOCKS, -1 + + // Save the previous state, as it's needed later. + vmv.v.v PREV_FEBA, FEBA + vmv.v.v PREV_HGDC, HGDC + + // Load the next 1024-bit message block and endian-swap each 64-bit word + vle64.v W0, (DATA) + vrev8.v W0, W0 + addi DATA, DATA, 32 + vle64.v W1, (DATA) + vrev8.v W1, W1 + addi DATA, DATA, 32 + vle64.v W2, (DATA) + vrev8.v W2, W2 + addi DATA, DATA, 32 + vle64.v W3, (DATA) + vrev8.v W3, W3 + addi DATA, DATA, 32 + + // Do the 80 rounds of SHA-512. + sha512_16rounds 0 + sha512_16rounds 0 + sha512_16rounds 0 + sha512_16rounds 0 + sha512_16rounds 1 + + // Add the previous state. + vadd.vv FEBA, FEBA, PREV_FEBA + vadd.vv HGDC, HGDC, PREV_HGDC + + // Repeat if more blocks remain. + bnez NUM_BLOCKS, .Lnext_block + + // Store the new state and return. + vsuxei8.v FEBA, (STATEP), INDICES + vsuxei8.v HGDC, (STATEP_C), INDICES + ret +SYM_FUNC_END(sha512_transform_zvknhb_zvkb) + +.section ".rodata" +.p2align 3 +.type K512, @object +K512: + .dword 0x428a2f98d728ae22, 0x7137449123ef65cd + .dword 0xb5c0fbcfec4d3b2f, 0xe9b5dba58189dbbc + .dword 0x3956c25bf348b538, 0x59f111f1b605d019 + .dword 0x923f82a4af194f9b, 0xab1c5ed5da6d8118 + .dword 0xd807aa98a3030242, 0x12835b0145706fbe + .dword 0x243185be4ee4b28c, 0x550c7dc3d5ffb4e2 + .dword 0x72be5d74f27b896f, 0x80deb1fe3b1696b1 + .dword 0x9bdc06a725c71235, 0xc19bf174cf692694 + .dword 0xe49b69c19ef14ad2, 0xefbe4786384f25e3 + .dword 0x0fc19dc68b8cd5b5, 0x240ca1cc77ac9c65 + .dword 0x2de92c6f592b0275, 0x4a7484aa6ea6e483 + .dword 0x5cb0a9dcbd41fbd4, 0x76f988da831153b5 + .dword 0x983e5152ee66dfab, 0xa831c66d2db43210 + .dword 0xb00327c898fb213f, 0xbf597fc7beef0ee4 + .dword 0xc6e00bf33da88fc2, 0xd5a79147930aa725 + .dword 0x06ca6351e003826f, 0x142929670a0e6e70 + .dword 0x27b70a8546d22ffc, 0x2e1b21385c26c926 + .dword 0x4d2c6dfc5ac42aed, 0x53380d139d95b3df + .dword 0x650a73548baf63de, 0x766a0abb3c77b2a8 + .dword 0x81c2c92e47edaee6, 0x92722c851482353b + .dword 0xa2bfe8a14cf10364, 0xa81a664bbc423001 + .dword 0xc24b8b70d0f89791, 0xc76c51a30654be30 + .dword 0xd192e819d6ef5218, 0xd69906245565a910 + .dword 0xf40e35855771202a, 0x106aa07032bbd1b8 + .dword 0x19a4c116b8d2d0c8, 0x1e376c085141ab53 + .dword 0x2748774cdf8eeb99, 0x34b0bcb5e19b48a8 + .dword 0x391c0cb3c5c95a63, 0x4ed8aa4ae3418acb + .dword 0x5b9cca4f7763e373, 0x682e6ff3d6b2b8a3 + .dword 0x748f82ee5defb2fc, 0x78a5636f43172f60 + .dword 0x84c87814a1f0ab72, 0x8cc702081a6439ec + .dword 0x90befffa23631e28, 0xa4506cebde82bde9 + .dword 0xbef9a3f7b2c67915, 0xc67178f2e372532b + .dword 0xca273eceea26619c, 0xd186b8c721c0c207 + .dword 0xeada7dd6cde0eb1e, 0xf57d4f7fee6ed178 + .dword 0x06f067aa72176fba, 0x0a637dc5a2c898a6 + .dword 0x113f9804bef90dae, 0x1b710b35131c471b + .dword 0x28db77f523047d84, 0x32caab7b40c72493 + .dword 0x3c9ebe0a15c9bebc, 0x431d67c49c100d4c + .dword 0x4cc5d4becb3e42b6, 0x597f299cfc657e2a + .dword 0x5fcb6fab3ad6faec, 0x6c44198c4a475817 +.size K512, . - K512 -- 2.43.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69217C47DA7 for ; Fri, 5 Jan 2024 18:51:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=XrcW77qPrxA0C0InjYdGiNAJN3p/HGvnFSbnK5/H8jo=; b=RF/acwn00Sjco5 Ey4frFzinQ5XV53Pk69OaNEaLycnDvHrlSF6btJ6gw8DfOS7TUOpEKO9I6x0uOZmdjJuuZ6UAaOlT sOIW0p60fdrzlDFlor0CobreCkEjpT72mYuJ2+CMrwAU1GQOImMoJDI3VmGLylSx8Ect6zTB9oddl glBacGSo3Y/EqdlnPxg6kYF3KNfERHtQ9V5fvFZZ3frVlIRqJLxMKhC7smNOmIsIAtRdjuX6MV/DL 9teUPAmEX0cABmi+C0jatldPSuxZzLPeKiqigrPiq7XDY03L5uDR3q+nBx2yijhBTZgPL5ZFbBbfL U1Q/d/mTH4G34Dkq1xtA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rLpIG-0001Q3-0N; Fri, 05 Jan 2024 18:51:44 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rLpI6-0001J3-0X for linux-riscv@lists.infradead.org; Fri, 05 Jan 2024 18:51:39 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 8EDFE61BE9; Fri, 5 Jan 2024 18:51:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CA4E2C433C9; Fri, 5 Jan 2024 18:51:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704480693; bh=KWqoN9K4c+36n+xRQWs4t4tC8x2ENilxLlOhbN7ScK4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XatPJLOQBbuNUYQayOldPhAd+Y9CDrMh2wy7VucsS6AB4k8R0yrgdJFCRe2PeU1YW JJLealACQihSwGZFEg+74xAiUSHz2HMbKN38ucE0pNaDg8UMDDUFmfjwHLsgf6/c2T wCsZpsXiol+JdIFUNl6n/kscYYJXV5YzZSpAfTOCqXnvnMdvgcq2xiTMbe8C4T0p9g MX4SqsiezYNLadU36ELkt2DIT1oAr1TKGqxP1pSyXd5t55BHAXj7a3/ISRFrmjRcYw wjYAWndmhruS9jAsJeKcFVKL+zXZzLQR5g0poZRSIA6mHIn0jZmEYDZDJeuCNwLl1F qRJa/HrWHlsfg== From: Eric Biggers To: linux-crypto@vger.kernel.org, linux-riscv@lists.infradead.org, Jerry Shih Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Heiko Stuebner , Phoebe Chen , hongrong.hsu@sifive.com, Paul Walmsley , Palmer Dabbelt , Albert Ou , Andy Chiu , Charalampos Mitrodimas , Heiko Stuebner Subject: [PATCH v2 10/12] crypto: riscv - add vector crypto accelerated SHA-{512,384} Date: Fri, 5 Jan 2024 10:49:46 -0800 Message-ID: <20240105184950.43181-11-ebiggers@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240105184950.43181-1-ebiggers@kernel.org> References: <20240105184950.43181-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240105_105134_304020_C6A24B80 X-CRM114-Status: GOOD ( 31.10 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org RnJvbTogSmVycnkgU2hpaCA8amVycnkuc2hpaEBzaWZpdmUuY29tPgoKQWRkIGFuIGltcGxlbWVu dGF0aW9uIG9mIFNIQS01MTIgYW5kIFNIQS0zODQgdXNpbmcgdGhlIFp2a25oYiBleHRlbnNpb24u ClRoZSBhc3NlbWJseSBjb2RlIGlzIGRlcml2ZWQgZnJvbSBPcGVuU1NMIGNvZGUgKG9wZW5zc2wv b3BlbnNzbCMyMTkyMykKdGhhdCB3YXMgZHVhbC1saWNlbnNlZCBzbyB0aGF0IGl0IGNvdWxkIGJl IHJldXNlZCBpbiB0aGUga2VybmVsLgpOZXZlcnRoZWxlc3MsIHRoZSBhc3NlbWJseSBoYXMgYmVl biBzaWduaWZpY2FudGx5IHJld29ya2VkIGZvcgppbnRlZ3JhdGlvbiB3aXRoIHRoZSBrZXJuZWws IGZvciBleGFtcGxlIGJ5IHVzaW5nIGEgcmVndWxhciAuUyBmaWxlCmluc3RlYWQgb2YgdGhlIHNv LWNhbGxlZCBwZXJsYXNtLCB1c2luZyB0aGUgYXNzZW1ibGVyIGluc3RlYWQgb2YgYmFyZQonLmlu c3QnLCBhbmQgZ3JlYXRseSByZWR1Y2luZyBjb2RlIGR1cGxpY2F0aW9uLgoKQ28tZGV2ZWxvcGVk LWJ5OiBDaGFyYWxhbXBvcyBNaXRyb2RpbWFzIDxjaGFyYWxhbXBvcy5taXRyb2RpbWFzQHZydWxs LmV1PgpTaWduZWQtb2ZmLWJ5OiBDaGFyYWxhbXBvcyBNaXRyb2RpbWFzIDxjaGFyYWxhbXBvcy5t aXRyb2RpbWFzQHZydWxsLmV1PgpDby1kZXZlbG9wZWQtYnk6IEhlaWtvIFN0dWVibmVyIDxoZWlr by5zdHVlYm5lckB2cnVsbC5ldT4KU2lnbmVkLW9mZi1ieTogSGVpa28gU3R1ZWJuZXIgPGhlaWtv LnN0dWVibmVyQHZydWxsLmV1PgpDby1kZXZlbG9wZWQtYnk6IFBob2ViZSBDaGVuIDxwaG9lYmUu Y2hlbkBzaWZpdmUuY29tPgpTaWduZWQtb2ZmLWJ5OiBQaG9lYmUgQ2hlbiA8cGhvZWJlLmNoZW5A c2lmaXZlLmNvbT4KU2lnbmVkLW9mZi1ieTogSmVycnkgU2hpaCA8amVycnkuc2hpaEBzaWZpdmUu Y29tPgpDby1kZXZlbG9wZWQtYnk6IEVyaWMgQmlnZ2VycyA8ZWJpZ2dlcnNAZ29vZ2xlLmNvbT4K U2lnbmVkLW9mZi1ieTogRXJpYyBCaWdnZXJzIDxlYmlnZ2Vyc0Bnb29nbGUuY29tPgotLS0KIGFy Y2gvcmlzY3YvY3J5cHRvL0tjb25maWcgICAgICAgICAgICAgICAgICAgICB8ICAxMSArCiBhcmNo L3Jpc2N2L2NyeXB0by9NYWtlZmlsZSAgICAgICAgICAgICAgICAgICAgfCAgIDMgKwogYXJjaC9y aXNjdi9jcnlwdG8vc2hhNTEyLXJpc2N2NjQtZ2x1ZS5jICAgICAgIHwgMTMzICsrKysrKysrKysr KwogLi4uL3Jpc2N2L2NyeXB0by9zaGE1MTItcmlzY3Y2NC16dmtuaGItenZrYi5TIHwgMjAzICsr KysrKysrKysrKysrKysrKwogNCBmaWxlcyBjaGFuZ2VkLCAzNTAgaW5zZXJ0aW9ucygrKQogY3Jl YXRlIG1vZGUgMTAwNjQ0IGFyY2gvcmlzY3YvY3J5cHRvL3NoYTUxMi1yaXNjdjY0LWdsdWUuYwog Y3JlYXRlIG1vZGUgMTAwNjQ0IGFyY2gvcmlzY3YvY3J5cHRvL3NoYTUxMi1yaXNjdjY0LXp2a25o Yi16dmtiLlMKCmRpZmYgLS1naXQgYS9hcmNoL3Jpc2N2L2NyeXB0by9LY29uZmlnIGIvYXJjaC9y aXNjdi9jcnlwdG8vS2NvbmZpZwppbmRleCA2ODdkYmI3MWY3ZDUwLi4wZmQwYmY0NmM5MDk5IDEw MDY0NAotLS0gYS9hcmNoL3Jpc2N2L2NyeXB0by9LY29uZmlnCisrKyBiL2FyY2gvcmlzY3YvY3J5 cHRvL0tjb25maWcKQEAgLTQzLDExICs0MywyMiBAQCBjb25maWcgQ1JZUFRPX1NIQTI1Nl9SSVND VjY0CiAJdHJpc3RhdGUgIkhhc2ggZnVuY3Rpb25zOiBTSEEtMjI0IGFuZCBTSEEtMjU2IgogCWRl cGVuZHMgb24gNjRCSVQgJiYgUklTQ1ZfSVNBX1YgJiYgVE9PTENIQUlOX0hBU19WRUNUT1JfQ1JZ UFRPCiAJc2VsZWN0IENSWVBUT19TSEEyNTYKIAloZWxwCiAJICBTSEEtMjI0IGFuZCBTSEEtMjU2 IHNlY3VyZSBoYXNoIGFsZ29yaXRobSAoRklQUyAxODApCiAKIAkgIEFyY2hpdGVjdHVyZTogcmlz Y3Y2NCB1c2luZzoKIAkgIC0gWnZrbmhhIG9yIFp2a25oYiB2ZWN0b3IgY3J5cHRvIGV4dGVuc2lv bnMKIAkgIC0gWnZrYiB2ZWN0b3IgY3J5cHRvIGV4dGVuc2lvbgogCitjb25maWcgQ1JZUFRPX1NI QTUxMl9SSVNDVjY0CisJdHJpc3RhdGUgIkhhc2ggZnVuY3Rpb25zOiBTSEEtMzg0IGFuZCBTSEEt NTEyIgorCWRlcGVuZHMgb24gNjRCSVQgJiYgUklTQ1ZfSVNBX1YgJiYgVE9PTENIQUlOX0hBU19W RUNUT1JfQ1JZUFRPCisJc2VsZWN0IENSWVBUT19TSEE1MTIKKwloZWxwCisJICBTSEEtMzg0IGFu ZCBTSEEtNTEyIHNlY3VyZSBoYXNoIGFsZ29yaXRobSAoRklQUyAxODApCisKKwkgIEFyY2hpdGVj dHVyZTogcmlzY3Y2NCB1c2luZzoKKwkgIC0gWnZrbmhiIHZlY3RvciBjcnlwdG8gZXh0ZW5zaW9u CisJICAtIFp2a2IgdmVjdG9yIGNyeXB0byBleHRlbnNpb24KKwogZW5kbWVudQpkaWZmIC0tZ2l0 IGEvYXJjaC9yaXNjdi9jcnlwdG8vTWFrZWZpbGUgYi9hcmNoL3Jpc2N2L2NyeXB0by9NYWtlZmls ZQppbmRleCA4Njc0ZTk4MjljZGIxLi44ZGJmZGExOTE4OTkwIDEwMDY0NAotLS0gYS9hcmNoL3Jp c2N2L2NyeXB0by9NYWtlZmlsZQorKysgYi9hcmNoL3Jpc2N2L2NyeXB0by9NYWtlZmlsZQpAQCAt OCwxMCArOCwxMyBAQCBhZXMtcmlzY3Y2NC15IDo9IGFlcy1yaXNjdjY0LWdsdWUubyBhZXMtcmlz Y3Y2NC16dmtuZWQubyBcCiAJCSBhZXMtcmlzY3Y2NC16dmtuZWQtenZiYi16dmtnLm8gYWVzLXJp c2N2NjQtenZrbmVkLXp2a2IubwogCiBvYmotJChDT05GSUdfQ1JZUFRPX0NIQUNIQV9SSVNDVjY0 KSArPSBjaGFjaGEtcmlzY3Y2NC5vCiBjaGFjaGEtcmlzY3Y2NC15IDo9IGNoYWNoYS1yaXNjdjY0 LWdsdWUubyBjaGFjaGEtcmlzY3Y2NC16dmtiLm8KIAogb2JqLSQoQ09ORklHX0NSWVBUT19HSEFT SF9SSVNDVjY0KSArPSBnaGFzaC1yaXNjdjY0Lm8KIGdoYXNoLXJpc2N2NjQteSA6PSBnaGFzaC1y aXNjdjY0LWdsdWUubyBnaGFzaC1yaXNjdjY0LXp2a2cubwogCiBvYmotJChDT05GSUdfQ1JZUFRP X1NIQTI1Nl9SSVNDVjY0KSArPSBzaGEyNTYtcmlzY3Y2NC5vCiBzaGEyNTYtcmlzY3Y2NC15IDo9 IHNoYTI1Ni1yaXNjdjY0LWdsdWUubyBzaGEyNTYtcmlzY3Y2NC16dmtuaGFfb3JfenZrbmhiLXp2 a2IubworCitvYmotJChDT05GSUdfQ1JZUFRPX1NIQTUxMl9SSVNDVjY0KSArPSBzaGE1MTItcmlz Y3Y2NC5vCitzaGE1MTItcmlzY3Y2NC15IDo9IHNoYTUxMi1yaXNjdjY0LWdsdWUubyBzaGE1MTIt cmlzY3Y2NC16dmtuaGItenZrYi5vCmRpZmYgLS1naXQgYS9hcmNoL3Jpc2N2L2NyeXB0by9zaGE1 MTItcmlzY3Y2NC1nbHVlLmMgYi9hcmNoL3Jpc2N2L2NyeXB0by9zaGE1MTItcmlzY3Y2NC1nbHVl LmMKbmV3IGZpbGUgbW9kZSAxMDA2NDQKaW5kZXggMDAwMDAwMDAwMDAwMC4uNDNiNTZhMDhhZWI1 MQotLS0gL2Rldi9udWxsCisrKyBiL2FyY2gvcmlzY3YvY3J5cHRvL3NoYTUxMi1yaXNjdjY0LWds dWUuYwpAQCAtMCwwICsxLDEzMyBAQAorLy8gU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0y LjAtb3ItbGF0ZXIKKy8qCisgKiBTSEEtNTEyIGFuZCBTSEEtMzg0IHVzaW5nIHRoZSBSSVNDLVYg dmVjdG9yIGNyeXB0byBleHRlbnNpb25zCisgKgorICogQ29weXJpZ2h0IChDKSAyMDIzIFZSVUxM IEdtYkgKKyAqIEF1dGhvcjogSGVpa28gU3R1ZWJuZXIgPGhlaWtvLnN0dWVibmVyQHZydWxsLmV1 PgorICoKKyAqIENvcHlyaWdodCAoQykgMjAyMyBTaUZpdmUsIEluYy4KKyAqIEF1dGhvcjogSmVy cnkgU2hpaCA8amVycnkuc2hpaEBzaWZpdmUuY29tPgorICovCisKKyNpbmNsdWRlIDxhc20vc2lt ZC5oPgorI2luY2x1ZGUgPGFzbS92ZWN0b3IuaD4KKyNpbmNsdWRlIDxjcnlwdG8vaW50ZXJuYWwv aGFzaC5oPgorI2luY2x1ZGUgPGNyeXB0by9pbnRlcm5hbC9zaW1kLmg+CisjaW5jbHVkZSA8Y3J5 cHRvL3NoYTUxMl9iYXNlLmg+CisjaW5jbHVkZSA8bGludXgvbGlua2FnZS5oPgorI2luY2x1ZGUg PGxpbnV4L21vZHVsZS5oPgorCisvKgorICogTm90ZTogdGhlIGFzbSBmdW5jdGlvbiBvbmx5IHVz ZXMgdGhlICdzdGF0ZScgZmllbGQgb2Ygc3RydWN0IHNoYTUxMl9zdGF0ZS4KKyAqIEl0IGlzIGFz c3VtZWQgdG8gYmUgdGhlIGZpcnN0IGZpZWxkLgorICovCithc21saW5rYWdlIHZvaWQgc2hhNTEy X3RyYW5zZm9ybV96dmtuaGJfenZrYigKKwlzdHJ1Y3Qgc2hhNTEyX3N0YXRlICpzdGF0ZSwgY29u c3QgdTggKmRhdGEsIGludCBudW1fYmxvY2tzKTsKKworc3RhdGljIGludCByaXNjdjY0X3NoYTUx Ml91cGRhdGUoc3RydWN0IHNoYXNoX2Rlc2MgKmRlc2MsIGNvbnN0IHU4ICpkYXRhLAorCQkJCSB1 bnNpZ25lZCBpbnQgbGVuKQoreworCS8qCisJICogRW5zdXJlIHN0cnVjdCBzaGE1MTJfc3RhdGUg YmVnaW5zIGRpcmVjdGx5IHdpdGggdGhlIFNIQS01MTIKKwkgKiA1MTItYml0IGludGVybmFsIHN0 YXRlLCBhcyB0aGlzIGlzIHdoYXQgdGhlIGFzbSBmdW5jdGlvbiBleHBlY3RzLgorCSAqLworCUJV SUxEX0JVR19PTihvZmZzZXRvZihzdHJ1Y3Qgc2hhNTEyX3N0YXRlLCBzdGF0ZSkgIT0gMCk7CisK KwlpZiAoY3J5cHRvX3NpbWRfdXNhYmxlKCkpIHsKKwkJa2VybmVsX3ZlY3Rvcl9iZWdpbigpOwor CQlzaGE1MTJfYmFzZV9kb191cGRhdGUoZGVzYywgZGF0YSwgbGVuLAorCQkJCSAgICAgIHNoYTUx Ml90cmFuc2Zvcm1fenZrbmhiX3p2a2IpOworCQlrZXJuZWxfdmVjdG9yX2VuZCgpOworCX0gZWxz ZSB7CisJCWNyeXB0b19zaGE1MTJfdXBkYXRlKGRlc2MsIGRhdGEsIGxlbik7CisJfQorCXJldHVy biAwOworfQorCitzdGF0aWMgaW50IHJpc2N2NjRfc2hhNTEyX2ZpbnVwKHN0cnVjdCBzaGFzaF9k ZXNjICpkZXNjLCBjb25zdCB1OCAqZGF0YSwKKwkJCQl1bnNpZ25lZCBpbnQgbGVuLCB1OCAqb3V0 KQoreworCWlmIChjcnlwdG9fc2ltZF91c2FibGUoKSkgeworCQlrZXJuZWxfdmVjdG9yX2JlZ2lu KCk7CisJCWlmIChsZW4pCisJCQlzaGE1MTJfYmFzZV9kb191cGRhdGUoZGVzYywgZGF0YSwgbGVu LAorCQkJCQkgICAgICBzaGE1MTJfdHJhbnNmb3JtX3p2a25oYl96dmtiKTsKKwkJc2hhNTEyX2Jh c2VfZG9fZmluYWxpemUoZGVzYywgc2hhNTEyX3RyYW5zZm9ybV96dmtuaGJfenZrYik7CisJCWtl cm5lbF92ZWN0b3JfZW5kKCk7CisKKwkJcmV0dXJuIHNoYTUxMl9iYXNlX2ZpbmlzaChkZXNjLCBv dXQpOworCX0KKworCXJldHVybiBjcnlwdG9fc2hhNTEyX2ZpbnVwKGRlc2MsIGRhdGEsIGxlbiwg b3V0KTsKK30KKworc3RhdGljIGludCByaXNjdjY0X3NoYTUxMl9maW5hbChzdHJ1Y3Qgc2hhc2hf ZGVzYyAqZGVzYywgdTggKm91dCkKK3sKKwlyZXR1cm4gcmlzY3Y2NF9zaGE1MTJfZmludXAoZGVz YywgTlVMTCwgMCwgb3V0KTsKK30KKworc3RhdGljIGludCByaXNjdjY0X3NoYTUxMl9kaWdlc3Qo c3RydWN0IHNoYXNoX2Rlc2MgKmRlc2MsIGNvbnN0IHU4ICpkYXRhLAorCQkJCSB1bnNpZ25lZCBp bnQgbGVuLCB1OCAqb3V0KQoreworCXJldHVybiBzaGE1MTJfYmFzZV9pbml0KGRlc2MpID86CisJ ICAgICAgIHJpc2N2NjRfc2hhNTEyX2ZpbnVwKGRlc2MsIGRhdGEsIGxlbiwgb3V0KTsKK30KKwor c3RhdGljIHN0cnVjdCBzaGFzaF9hbGcgcmlzY3Y2NF9zaGE1MTJfYWxnc1tdID0geworCXsKKwkJ LmluaXQgPSBzaGE1MTJfYmFzZV9pbml0LAorCQkudXBkYXRlID0gcmlzY3Y2NF9zaGE1MTJfdXBk YXRlLAorCQkuZmluYWwgPSByaXNjdjY0X3NoYTUxMl9maW5hbCwKKwkJLmZpbnVwID0gcmlzY3Y2 NF9zaGE1MTJfZmludXAsCisJCS5kaWdlc3QgPSByaXNjdjY0X3NoYTUxMl9kaWdlc3QsCisJCS5k ZXNjc2l6ZSA9IHNpemVvZihzdHJ1Y3Qgc2hhNTEyX3N0YXRlKSwKKwkJLmRpZ2VzdHNpemUgPSBT SEE1MTJfRElHRVNUX1NJWkUsCisJCS5iYXNlID0geworCQkJLmNyYV9ibG9ja3NpemUgPSBTSEE1 MTJfQkxPQ0tfU0laRSwKKwkJCS5jcmFfcHJpb3JpdHkgPSAzMDAsCisJCQkuY3JhX25hbWUgPSAi c2hhNTEyIiwKKwkJCS5jcmFfZHJpdmVyX25hbWUgPSAic2hhNTEyLXJpc2N2NjQtenZrbmhiLXp2 a2IiLAorCQkJLmNyYV9tb2R1bGUgPSBUSElTX01PRFVMRSwKKwkJfSwKKwl9LCB7CisJCS5pbml0 ID0gc2hhMzg0X2Jhc2VfaW5pdCwKKwkJLnVwZGF0ZSA9IHJpc2N2NjRfc2hhNTEyX3VwZGF0ZSwK KwkJLmZpbmFsID0gcmlzY3Y2NF9zaGE1MTJfZmluYWwsCisJCS5maW51cCA9IHJpc2N2NjRfc2hh NTEyX2ZpbnVwLAorCQkuZGVzY3NpemUgPSBzaXplb2Yoc3RydWN0IHNoYTUxMl9zdGF0ZSksCisJ CS5kaWdlc3RzaXplID0gU0hBMzg0X0RJR0VTVF9TSVpFLAorCQkuYmFzZSA9IHsKKwkJCS5jcmFf YmxvY2tzaXplID0gU0hBMzg0X0JMT0NLX1NJWkUsCisJCQkuY3JhX3ByaW9yaXR5ID0gMzAwLAor CQkJLmNyYV9uYW1lID0gInNoYTM4NCIsCisJCQkuY3JhX2RyaXZlcl9uYW1lID0gInNoYTM4NC1y aXNjdjY0LXp2a25oYi16dmtiIiwKKwkJCS5jcmFfbW9kdWxlID0gVEhJU19NT0RVTEUsCisJCX0s CisJfSwKK307CisKK3N0YXRpYyBpbnQgX19pbml0IHJpc2N2NjRfc2hhNTEyX21vZF9pbml0KHZv aWQpCit7CisJaWYgKHJpc2N2X2lzYV9leHRlbnNpb25fYXZhaWxhYmxlKE5VTEwsIFpWS05IQikg JiYKKwkgICAgcmlzY3ZfaXNhX2V4dGVuc2lvbl9hdmFpbGFibGUoTlVMTCwgWlZLQikgJiYKKwkg ICAgcmlzY3ZfdmVjdG9yX3ZsZW4oKSA+PSAxMjgpCisJCXJldHVybiBjcnlwdG9fcmVnaXN0ZXJf c2hhc2hlcyhyaXNjdjY0X3NoYTUxMl9hbGdzLAorCQkJCQkgICAgICAgQVJSQVlfU0laRShyaXNj djY0X3NoYTUxMl9hbGdzKSk7CisKKwlyZXR1cm4gLUVOT0RFVjsKK30KKworc3RhdGljIHZvaWQg X19leGl0IHJpc2N2NjRfc2hhNTEyX21vZF9leGl0KHZvaWQpCit7CisJY3J5cHRvX3VucmVnaXN0 ZXJfc2hhc2hlcyhyaXNjdjY0X3NoYTUxMl9hbGdzLAorCQkJCSAgQVJSQVlfU0laRShyaXNjdjY0 X3NoYTUxMl9hbGdzKSk7Cit9CisKK21vZHVsZV9pbml0KHJpc2N2NjRfc2hhNTEyX21vZF9pbml0 KTsKK21vZHVsZV9leGl0KHJpc2N2NjRfc2hhNTEyX21vZF9leGl0KTsKKworTU9EVUxFX0RFU0NS SVBUSU9OKCJTSEEtNTEyIChSSVNDLVYgYWNjZWxlcmF0ZWQpIik7CitNT0RVTEVfQVVUSE9SKCJI ZWlrbyBTdHVlYm5lciA8aGVpa28uc3R1ZWJuZXJAdnJ1bGwuZXU+Iik7CitNT0RVTEVfTElDRU5T RSgiR1BMIik7CitNT0RVTEVfQUxJQVNfQ1JZUFRPKCJzaGE1MTIiKTsKK01PRFVMRV9BTElBU19D UllQVE8oInNoYTM4NCIpOwpkaWZmIC0tZ2l0IGEvYXJjaC9yaXNjdi9jcnlwdG8vc2hhNTEyLXJp c2N2NjQtenZrbmhiLXp2a2IuUyBiL2FyY2gvcmlzY3YvY3J5cHRvL3NoYTUxMi1yaXNjdjY0LXp2 a25oYi16dmtiLlMKbmV3IGZpbGUgbW9kZSAxMDA2NDQKaW5kZXggMDAwMDAwMDAwMDAwMC4uM2E5 YWUyMTBmOTE1OAotLS0gL2Rldi9udWxsCisrKyBiL2FyY2gvcmlzY3YvY3J5cHRvL3NoYTUxMi1y aXNjdjY0LXp2a25oYi16dmtiLlMKQEAgLTAsMCArMSwyMDMgQEAKKy8qIFNQRFgtTGljZW5zZS1J ZGVudGlmaWVyOiBBcGFjaGUtMi4wIE9SIEJTRC0yLUNsYXVzZSAqLworLy8KKy8vIFRoaXMgZmls ZSBpcyBkdWFsLWxpY2Vuc2VkLCBtZWFuaW5nIHRoYXQgeW91IGNhbiB1c2UgaXQgdW5kZXIgeW91 cgorLy8gY2hvaWNlIG9mIGVpdGhlciBvZiB0aGUgZm9sbG93aW5nIHR3byBsaWNlbnNlczoKKy8v CisvLyBDb3B5cmlnaHQgMjAyMyBUaGUgT3BlblNTTCBQcm9qZWN0IEF1dGhvcnMuIEFsbCBSaWdo dHMgUmVzZXJ2ZWQuCisvLworLy8gTGljZW5zZWQgdW5kZXIgdGhlIEFwYWNoZSBMaWNlbnNlIDIu MCAodGhlICJMaWNlbnNlIikuIFlvdSBjYW4gb2J0YWluCisvLyBhIGNvcHkgaW4gdGhlIGZpbGUg TElDRU5TRSBpbiB0aGUgc291cmNlIGRpc3RyaWJ1dGlvbiBvciBhdAorLy8gaHR0cHM6Ly93d3cu b3BlbnNzbC5vcmcvc291cmNlL2xpY2Vuc2UuaHRtbAorLy8KKy8vIG9yCisvLworLy8gQ29weXJp Z2h0IChjKSAyMDIzLCBDaHJpc3RvcGggTcO8bGxuZXIgPGNocmlzdG9waC5tdWVsbG5lckB2cnVs bC5ldT4KKy8vIENvcHlyaWdodCAoYykgMjAyMywgUGhvZWJlIENoZW4gPHBob2ViZS5jaGVuQHNp Zml2ZS5jb20+CisvLyBDb3B5cmlnaHQgMjAyNCBHb29nbGUgTExDCisvLyBBbGwgcmlnaHRzIHJl c2VydmVkLgorLy8KKy8vIFJlZGlzdHJpYnV0aW9uIGFuZCB1c2UgaW4gc291cmNlIGFuZCBiaW5h cnkgZm9ybXMsIHdpdGggb3Igd2l0aG91dAorLy8gbW9kaWZpY2F0aW9uLCBhcmUgcGVybWl0dGVk IHByb3ZpZGVkIHRoYXQgdGhlIGZvbGxvd2luZyBjb25kaXRpb25zCisvLyBhcmUgbWV0OgorLy8g MS4gUmVkaXN0cmlidXRpb25zIG9mIHNvdXJjZSBjb2RlIG11c3QgcmV0YWluIHRoZSBhYm92ZSBj b3B5cmlnaHQKKy8vICAgIG5vdGljZSwgdGhpcyBsaXN0IG9mIGNvbmRpdGlvbnMgYW5kIHRoZSBm b2xsb3dpbmcgZGlzY2xhaW1lci4KKy8vIDIuIFJlZGlzdHJpYnV0aW9ucyBpbiBiaW5hcnkgZm9y bSBtdXN0IHJlcHJvZHVjZSB0aGUgYWJvdmUgY29weXJpZ2h0CisvLyAgICBub3RpY2UsIHRoaXMg bGlzdCBvZiBjb25kaXRpb25zIGFuZCB0aGUgZm9sbG93aW5nIGRpc2NsYWltZXIgaW4gdGhlCisv LyAgICBkb2N1bWVudGF0aW9uIGFuZC9vciBvdGhlciBtYXRlcmlhbHMgcHJvdmlkZWQgd2l0aCB0 aGUgZGlzdHJpYnV0aW9uLgorLy8KKy8vIFRISVMgU09GVFdBUkUgSVMgUFJPVklERUQgQlkgVEhF IENPUFlSSUdIVCBIT0xERVJTIEFORCBDT05UUklCVVRPUlMKKy8vICJBUyBJUyIgQU5EIEFOWSBF WFBSRVNTIE9SIElNUExJRUQgV0FSUkFOVElFUywgSU5DTFVESU5HLCBCVVQgTk9UCisvLyBMSU1J VEVEIFRPLCBUSEUgSU1QTElFRCBXQVJSQU5USUVTIE9GIE1FUkNIQU5UQUJJTElUWSBBTkQgRklU TkVTUyBGT1IKKy8vIEEgUEFSVElDVUxBUiBQVVJQT1NFIEFSRSBESVNDTEFJTUVELiBJTiBOTyBF VkVOVCBTSEFMTCBUSEUgQ09QWVJJR0hUCisvLyBPV05FUiBPUiBDT05UUklCVVRPUlMgQkUgTElB QkxFIEZPUiBBTlkgRElSRUNULCBJTkRJUkVDVCwgSU5DSURFTlRBTCwKKy8vIFNQRUNJQUwsIEVY RU1QTEFSWSwgT1IgQ09OU0VRVUVOVElBTCBEQU1BR0VTIChJTkNMVURJTkcsIEJVVCBOT1QKKy8v IExJTUlURUQgVE8sIFBST0NVUkVNRU5UIE9GIFNVQlNUSVRVVEUgR09PRFMgT1IgU0VSVklDRVM7 IExPU1MgT0YgVVNFLAorLy8gREFUQSwgT1IgUFJPRklUUzsgT1IgQlVTSU5FU1MgSU5URVJSVVBU SU9OKSBIT1dFVkVSIENBVVNFRCBBTkQgT04gQU5ZCisvLyBUSEVPUlkgT0YgTElBQklMSVRZLCBX SEVUSEVSIElOIENPTlRSQUNULCBTVFJJQ1QgTElBQklMSVRZLCBPUiBUT1JUCisvLyAoSU5DTFVE SU5HIE5FR0xJR0VOQ0UgT1IgT1RIRVJXSVNFKSBBUklTSU5HIElOIEFOWSBXQVkgT1VUIE9GIFRI RSBVU0UKKy8vIE9GIFRISVMgU09GVFdBUkUsIEVWRU4gSUYgQURWSVNFRCBPRiBUSEUgUE9TU0lC SUxJVFkgT0YgU1VDSCBEQU1BR0UuCisKKy8vIFRoZSBnZW5lcmF0ZWQgY29kZSBvZiB0aGlzIGZp bGUgZGVwZW5kcyBvbiB0aGUgZm9sbG93aW5nIFJJU0MtViBleHRlbnNpb25zOgorLy8gLSBSVjY0 SQorLy8gLSBSSVNDLVYgVmVjdG9yICgnVicpIHdpdGggVkxFTiA+PSAxMjgKKy8vIC0gUklTQy1W IFZlY3RvciBTSEEtMiBTZWN1cmUgSGFzaCBleHRlbnNpb24gKCdadmtuaGInKQorLy8gLSBSSVND LVYgVmVjdG9yIENyeXB0b2dyYXBoeSBCaXQtbWFuaXB1bGF0aW9uIGV4dGVuc2lvbiAoJ1p2a2In KQorCisjaW5jbHVkZSA8bGludXgvY2ZpX3R5cGVzLmg+CisKKy50ZXh0Cisub3B0aW9uIGFyY2gs ICt6dmtuaGIsICt6dmtiCisKKyNkZWZpbmUgU1RBVEVQCQlhMAorI2RlZmluZSBEQVRBCQlhMQor I2RlZmluZSBOVU1fQkxPQ0tTCWEyCisKKyNkZWZpbmUgU1RBVEVQX0MJYTMKKyNkZWZpbmUgSwkJ YTQKKworI2RlZmluZSBNQVNLCQl2MAorI2RlZmluZSBJTkRJQ0VTCQl2MQorI2RlZmluZSBXMAkJ djEwCS8vIExNVUw9MgorI2RlZmluZSBXMQkJdjEyCS8vIExNVUw9MgorI2RlZmluZSBXMgkJdjE0 CS8vIExNVUw9MgorI2RlZmluZSBXMwkJdjE2CS8vIExNVUw9MgorI2RlZmluZSBWVE1QCQl2MjAJ Ly8gTE1VTD0yCisjZGVmaW5lIEZFQkEJCXYyMgkvLyBMTVVMPTIKKyNkZWZpbmUgSEdEQwkJdjI0 CS8vIExNVUw9MgorI2RlZmluZSBQUkVWX0ZFQkEJdjI2CS8vIExNVUw9MgorI2RlZmluZSBQUkVW X0hHREMJdjI4CS8vIExNVUw9MgorCisvLyBEbyA0IHJvdW5kcyBvZiBTSEEtNTEyLiAgdzAgY29u dGFpbnMgdGhlIGN1cnJlbnQgNCBtZXNzYWdlIHNjaGVkdWxlIHdvcmRzLgorLy8KKy8vIElmIG5v dCBhbGwgdGhlIG1lc3NhZ2Ugc2NoZWR1bGUgd29yZHMgaGF2ZSBiZWVuIGNvbXB1dGVkIHlldCwg dGhlbiB0aGlzIGFsc28KKy8vIGNvbXB1dGVzIDQgbW9yZSBtZXNzYWdlIHNjaGVkdWxlIHdvcmRz LiAgdzEtdzMgY29udGFpbiB0aGUgbmV4dCAzIGdyb3VwcyBvZiA0CisvLyBtZXNzYWdlIHNjaGVk dWxlIHdvcmRzOyB0aGlzIG1hY3JvIGNvbXB1dGVzIHRoZSBncm91cCBhZnRlciB3MyBhbmQgd3Jp dGVzIGl0CisvLyB0byB3MC4gIFRoaXMgbWVhbnMgdGhhdCB0aGUgbmV4dCAodzAsIHcxLCB3Miwg dzMpIGlzIHRoZSBjdXJyZW50ICh3MSwgdzIsIHczLAorLy8gdzApLCBzbyB0aGUgY2FsbGVyIG11 c3QgY3ljbGUgdGhyb3VnaCB0aGUgcmVnaXN0ZXJzIGFjY29yZGluZ2x5LgorLm1hY3JvCXNoYTUx Ml80cm91bmRzCWxhc3QsIHcwLCB3MSwgdzIsIHczCisJdmxlNjQudgkJVlRNUCwgKEspCisJYWRk aQkJSywgSywgMzIKKwl2YWRkLnZ2CQlWVE1QLCBWVE1QLCBcdzAKKwl2c2hhMmNsLnZ2CUhHREMs IEZFQkEsIFZUTVAKKwl2c2hhMmNoLnZ2CUZFQkEsIEhHREMsIFZUTVAKKy5pZiAhXGxhc3QKKwl2 bWVyZ2UudnZtCVZUTVAsIFx3MiwgXHcxLCBNQVNLCisJdnNoYTJtcy52dglcdzAsIFZUTVAsIFx3 MworLmVuZGlmCisuZW5kbQorCisubWFjcm8Jc2hhNTEyXzE2cm91bmRzCWxhc3QKKwlzaGE1MTJf NHJvdW5kcwlcbGFzdCwgVzAsIFcxLCBXMiwgVzMKKwlzaGE1MTJfNHJvdW5kcwlcbGFzdCwgVzEs IFcyLCBXMywgVzAKKwlzaGE1MTJfNHJvdW5kcwlcbGFzdCwgVzIsIFczLCBXMCwgVzEKKwlzaGE1 MTJfNHJvdW5kcwlcbGFzdCwgVzMsIFcwLCBXMSwgVzIKKy5lbmRtCisKKy8vIHZvaWQgc2hhNTEy X3RyYW5zZm9ybV96dmtuaGJfenZrYih1NjQgc3RhdGVbOF0sIGNvbnN0IHU4ICpkYXRhLAorLy8J CQkJICAgICBpbnQgbnVtX2Jsb2Nrcyk7CitTWU1fVFlQRURfRlVOQ19TVEFSVChzaGE1MTJfdHJh bnNmb3JtX3p2a25oYl96dmtiKQorCisJLy8gU2V0dXAgbWFzayBmb3IgdGhlIHZtZXJnZSB0byBy ZXBsYWNlIHRoZSBmaXJzdCB3b3JkIChpZHg9PTApIGluCisJLy8gbWVzc2FnZSBzY2hlZHVsaW5n LiAgVGhlcmUgYXJlIDQgd29yZHMsIHNvIGFuIDgtYml0IG1hc2sgc3VmZmljZXMuCisJdnNldGl2 bGkJemVybywgMSwgZTgsIG0xLCB0YSwgbWEKKwl2bXYudi5pCQlNQVNLLCAweDAxCisKKwkvLyBM b2FkIHRoZSBzdGF0ZS4gIFRoZSBzdGF0ZSBpcyBzdG9yZWQgYXMge2EsYixjLGQsZSxmLGcsaH0s IGJ1dCB3ZQorCS8vIG5lZWQge2YsZSxiLGF9LHtoLGcsZCxjfS4gIFRoZSBkc3QgdnR5cGUgaXMg ZTY0bTIgYW5kIHRoZSBpbmRleCB2dHlwZQorCS8vIGlzIGU4bWY0LiAgV2UgdXNlIGluZGV4LWxv YWQgd2l0aCB0aGUgaTggaW5kaWNlcyB7NDAsIDMyLCA4LCAwfSwKKwkvLyBsb2FkZWQgdXNpbmcg dGhlIDMyLWJpdCBsaXR0bGUgZW5kaWFuIHZhbHVlIDB4MDAwODIwMjguCisJbGkJCXQwLCAweDAw MDgyMDI4CisJdnNldGl2bGkJemVybywgMSwgZTMyLCBtMSwgdGEsIG1hCisJdm12LnYueAkJSU5E SUNFUywgdDAKKwlhZGRpCQlTVEFURVBfQywgU1RBVEVQLCAxNgorCXZzZXRpdmxpCXplcm8sIDQs IGU2NCwgbTIsIHRhLCBtYQorCXZsdXhlaTgudglGRUJBLCAoU1RBVEVQKSwgSU5ESUNFUworCXZs dXhlaTgudglIR0RDLCAoU1RBVEVQX0MpLCBJTkRJQ0VTCisKKy5MbmV4dF9ibG9jazoKKwlsYQkJ SywgSzUxMgorCWFkZGkJCU5VTV9CTE9DS1MsIE5VTV9CTE9DS1MsIC0xCisKKwkvLyBTYXZlIHRo ZSBwcmV2aW91cyBzdGF0ZSwgYXMgaXQncyBuZWVkZWQgbGF0ZXIuCisJdm12LnYudgkJUFJFVl9G RUJBLCBGRUJBCisJdm12LnYudgkJUFJFVl9IR0RDLCBIR0RDCisKKwkvLyBMb2FkIHRoZSBuZXh0 IDEwMjQtYml0IG1lc3NhZ2UgYmxvY2sgYW5kIGVuZGlhbi1zd2FwIGVhY2ggNjQtYml0IHdvcmQK Kwl2bGU2NC52CQlXMCwgKERBVEEpCisJdnJldjgudgkJVzAsIFcwCisJYWRkaQkJREFUQSwgREFU QSwgMzIKKwl2bGU2NC52CQlXMSwgKERBVEEpCisJdnJldjgudgkJVzEsIFcxCisJYWRkaQkJREFU QSwgREFUQSwgMzIKKwl2bGU2NC52CQlXMiwgKERBVEEpCisJdnJldjgudgkJVzIsIFcyCisJYWRk aQkJREFUQSwgREFUQSwgMzIKKwl2bGU2NC52CQlXMywgKERBVEEpCisJdnJldjgudgkJVzMsIFcz CisJYWRkaQkJREFUQSwgREFUQSwgMzIKKworCS8vIERvIHRoZSA4MCByb3VuZHMgb2YgU0hBLTUx Mi4KKwlzaGE1MTJfMTZyb3VuZHMgMAorCXNoYTUxMl8xNnJvdW5kcyAwCisJc2hhNTEyXzE2cm91 bmRzIDAKKwlzaGE1MTJfMTZyb3VuZHMgMAorCXNoYTUxMl8xNnJvdW5kcyAxCisKKwkvLyBBZGQg dGhlIHByZXZpb3VzIHN0YXRlLgorCXZhZGQudnYJCUZFQkEsIEZFQkEsIFBSRVZfRkVCQQorCXZh ZGQudnYJCUhHREMsIEhHREMsIFBSRVZfSEdEQworCisJLy8gUmVwZWF0IGlmIG1vcmUgYmxvY2tz IHJlbWFpbi4KKwlibmV6CQlOVU1fQkxPQ0tTLCAuTG5leHRfYmxvY2sKKworCS8vIFN0b3JlIHRo ZSBuZXcgc3RhdGUgYW5kIHJldHVybi4KKwl2c3V4ZWk4LnYJRkVCQSwgKFNUQVRFUCksIElORElD RVMKKwl2c3V4ZWk4LnYJSEdEQywgKFNUQVRFUF9DKSwgSU5ESUNFUworCXJldAorU1lNX0ZVTkNf RU5EKHNoYTUxMl90cmFuc2Zvcm1fenZrbmhiX3p2a2IpCisKKy5zZWN0aW9uICIucm9kYXRhIgor LnAyYWxpZ24gMworLnR5cGUgSzUxMiwgQG9iamVjdAorSzUxMjoKKwkuZHdvcmQJCTB4NDI4YTJm OThkNzI4YWUyMiwgMHg3MTM3NDQ5MTIzZWY2NWNkCisJLmR3b3JkCQkweGI1YzBmYmNmZWM0ZDNi MmYsIDB4ZTliNWRiYTU4MTg5ZGJiYworCS5kd29yZAkJMHgzOTU2YzI1YmYzNDhiNTM4LCAweDU5 ZjExMWYxYjYwNWQwMTkKKwkuZHdvcmQJCTB4OTIzZjgyYTRhZjE5NGY5YiwgMHhhYjFjNWVkNWRh NmQ4MTE4CisJLmR3b3JkCQkweGQ4MDdhYTk4YTMwMzAyNDIsIDB4MTI4MzViMDE0NTcwNmZiZQor CS5kd29yZAkJMHgyNDMxODViZTRlZTRiMjhjLCAweDU1MGM3ZGMzZDVmZmI0ZTIKKwkuZHdvcmQJ CTB4NzJiZTVkNzRmMjdiODk2ZiwgMHg4MGRlYjFmZTNiMTY5NmIxCisJLmR3b3JkCQkweDliZGMw NmE3MjVjNzEyMzUsIDB4YzE5YmYxNzRjZjY5MjY5NAorCS5kd29yZAkJMHhlNDliNjljMTllZjE0 YWQyLCAweGVmYmU0Nzg2Mzg0ZjI1ZTMKKwkuZHdvcmQJCTB4MGZjMTlkYzY4YjhjZDViNSwgMHgy NDBjYTFjYzc3YWM5YzY1CisJLmR3b3JkCQkweDJkZTkyYzZmNTkyYjAyNzUsIDB4NGE3NDg0YWE2 ZWE2ZTQ4MworCS5kd29yZAkJMHg1Y2IwYTlkY2JkNDFmYmQ0LCAweDc2Zjk4OGRhODMxMTUzYjUK KwkuZHdvcmQJCTB4OTgzZTUxNTJlZTY2ZGZhYiwgMHhhODMxYzY2ZDJkYjQzMjEwCisJLmR3b3Jk CQkweGIwMDMyN2M4OThmYjIxM2YsIDB4YmY1OTdmYzdiZWVmMGVlNAorCS5kd29yZAkJMHhjNmUw MGJmMzNkYTg4ZmMyLCAweGQ1YTc5MTQ3OTMwYWE3MjUKKwkuZHdvcmQJCTB4MDZjYTYzNTFlMDAz ODI2ZiwgMHgxNDI5Mjk2NzBhMGU2ZTcwCisJLmR3b3JkCQkweDI3YjcwYTg1NDZkMjJmZmMsIDB4 MmUxYjIxMzg1YzI2YzkyNgorCS5kd29yZAkJMHg0ZDJjNmRmYzVhYzQyYWVkLCAweDUzMzgwZDEz OWQ5NWIzZGYKKwkuZHdvcmQJCTB4NjUwYTczNTQ4YmFmNjNkZSwgMHg3NjZhMGFiYjNjNzdiMmE4 CisJLmR3b3JkCQkweDgxYzJjOTJlNDdlZGFlZTYsIDB4OTI3MjJjODUxNDgyMzUzYgorCS5kd29y ZAkJMHhhMmJmZThhMTRjZjEwMzY0LCAweGE4MWE2NjRiYmM0MjMwMDEKKwkuZHdvcmQJCTB4YzI0 YjhiNzBkMGY4OTc5MSwgMHhjNzZjNTFhMzA2NTRiZTMwCisJLmR3b3JkCQkweGQxOTJlODE5ZDZl ZjUyMTgsIDB4ZDY5OTA2MjQ1NTY1YTkxMAorCS5kd29yZAkJMHhmNDBlMzU4NTU3NzEyMDJhLCAw eDEwNmFhMDcwMzJiYmQxYjgKKwkuZHdvcmQJCTB4MTlhNGMxMTZiOGQyZDBjOCwgMHgxZTM3NmMw ODUxNDFhYjUzCisJLmR3b3JkCQkweDI3NDg3NzRjZGY4ZWViOTksIDB4MzRiMGJjYjVlMTliNDhh OAorCS5kd29yZAkJMHgzOTFjMGNiM2M1Yzk1YTYzLCAweDRlZDhhYTRhZTM0MThhY2IKKwkuZHdv cmQJCTB4NWI5Y2NhNGY3NzYzZTM3MywgMHg2ODJlNmZmM2Q2YjJiOGEzCisJLmR3b3JkCQkweDc0 OGY4MmVlNWRlZmIyZmMsIDB4NzhhNTYzNmY0MzE3MmY2MAorCS5kd29yZAkJMHg4NGM4NzgxNGEx ZjBhYjcyLCAweDhjYzcwMjA4MWE2NDM5ZWMKKwkuZHdvcmQJCTB4OTBiZWZmZmEyMzYzMWUyOCwg MHhhNDUwNmNlYmRlODJiZGU5CisJLmR3b3JkCQkweGJlZjlhM2Y3YjJjNjc5MTUsIDB4YzY3MTc4 ZjJlMzcyNTMyYgorCS5kd29yZAkJMHhjYTI3M2VjZWVhMjY2MTljLCAweGQxODZiOGM3MjFjMGMy MDcKKwkuZHdvcmQJCTB4ZWFkYTdkZDZjZGUwZWIxZSwgMHhmNTdkNGY3ZmVlNmVkMTc4CisJLmR3 b3JkCQkweDA2ZjA2N2FhNzIxNzZmYmEsIDB4MGE2MzdkYzVhMmM4OThhNgorCS5kd29yZAkJMHgx MTNmOTgwNGJlZjkwZGFlLCAweDFiNzEwYjM1MTMxYzQ3MWIKKwkuZHdvcmQJCTB4MjhkYjc3ZjUy MzA0N2Q4NCwgMHgzMmNhYWI3YjQwYzcyNDkzCisJLmR3b3JkCQkweDNjOWViZTBhMTVjOWJlYmMs IDB4NDMxZDY3YzQ5YzEwMGQ0YworCS5kd29yZAkJMHg0Y2M1ZDRiZWNiM2U0MmI2LCAweDU5N2Yy OTljZmM2NTdlMmEKKwkuZHdvcmQJCTB4NWZjYjZmYWIzYWQ2ZmFlYywgMHg2YzQ0MTk4YzRhNDc1 ODE3Cisuc2l6ZSBLNTEyLCAuIC0gSzUxMgotLSAKMi40My4wCgoKX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtcmlzY3YgbWFpbGluZyBsaXN0Cmxp bnV4LXJpc2N2QGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcv bWFpbG1hbi9saXN0aW5mby9saW51eC1yaXNjdgo=