From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3FDE8374F3; Fri, 5 Jan 2024 18:51:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bIBqIaiE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 37B6DC433D9; Fri, 5 Jan 2024 18:51:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704480692; bh=Q9KorLkt9z4ETNv86tVce30LmXb337KOzYRmXS1ZQGc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bIBqIaiEYWEWx02zwyaHirNUpJcm10b4KacO6EhFXhcVN2GwLyTzQa4x3Ql32LNCa Mav+8JgMm1Tyth5+KS/96pluhr9QvSJkk4iviO2L52SgxCFwkWtJF7sEwg86Ulhmlp vNSlBAAmMJFDIWwi13ZJduR4T5f/CyyYZZeAAOz4ZykLxXJhDjDIR8OlHwKHhR4wLi I6JrkM/40hDv6zD2Qbb9/ytXA2WFfbouADlSmhLvb9B34xoU9y/GFd3dfG7oGFn0l/ fDZTTI9LOmG8EZHILx0kaf/acU4nGeRaOGX7rokiSyhwK/VWQSnoX6fBWQKG8BuIU9 pMMjDzVexMROw== From: Eric Biggers To: linux-crypto@vger.kernel.org, linux-riscv@lists.infradead.org, Jerry Shih Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Heiko Stuebner , Phoebe Chen , hongrong.hsu@sifive.com, Paul Walmsley , Palmer Dabbelt , Albert Ou , Andy Chiu , Charalampos Mitrodimas , Heiko Stuebner Subject: [PATCH v2 09/12] crypto: riscv - add vector crypto accelerated SHA-{256,224} Date: Fri, 5 Jan 2024 10:49:45 -0800 Message-ID: <20240105184950.43181-10-ebiggers@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240105184950.43181-1-ebiggers@kernel.org> References: <20240105184950.43181-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From: Jerry Shih Add an implementation of SHA-256 and SHA-224 using the Zvknha or Zvknhb extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and greatly reducing code duplication. Co-developed-by: Charalampos Mitrodimas Signed-off-by: Charalampos Mitrodimas Co-developed-by: Heiko Stuebner Signed-off-by: Heiko Stuebner Co-developed-by: Phoebe Chen Signed-off-by: Phoebe Chen Signed-off-by: Jerry Shih Co-developed-by: Eric Biggers Signed-off-by: Eric Biggers --- arch/riscv/crypto/Kconfig | 11 + arch/riscv/crypto/Makefile | 3 + arch/riscv/crypto/sha256-riscv64-glue.c | 137 +++++++++++ .../sha256-riscv64-zvknha_or_zvknhb-zvkb.S | 225 ++++++++++++++++++ 4 files changed, 376 insertions(+) create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha_or_zvknhb-zvkb.S diff --git a/arch/riscv/crypto/Kconfig b/arch/riscv/crypto/Kconfig index 676ba5af8f55a..687dbb71f7d50 100644 --- a/arch/riscv/crypto/Kconfig +++ b/arch/riscv/crypto/Kconfig @@ -32,11 +32,22 @@ config CRYPTO_CHACHA_RISCV64 config CRYPTO_GHASH_RISCV64 tristate "Hash functions: GHASH" depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO select CRYPTO_GCM help GCM GHASH function (NIST SP 800-38D) Architecture: riscv64 using: - Zvkg vector crypto extension +config CRYPTO_SHA256_RISCV64 + tristate "Hash functions: SHA-224 and SHA-256" + depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO + select CRYPTO_SHA256 + help + SHA-224 and SHA-256 secure hash algorithm (FIPS 180) + + Architecture: riscv64 using: + - Zvknha or Zvknhb vector crypto extensions + - Zvkb vector crypto extension + endmenu diff --git a/arch/riscv/crypto/Makefile b/arch/riscv/crypto/Makefile index 9666a27d76862..8674e9829cdb1 100644 --- a/arch/riscv/crypto/Makefile +++ b/arch/riscv/crypto/Makefile @@ -5,10 +5,13 @@ obj-$(CONFIG_CRYPTO_AES_RISCV64) += aes-riscv64.o aes-riscv64-y := aes-riscv64-glue.o aes-riscv64-zvkned.o \ aes-riscv64-zvkned-zvbb-zvkg.o aes-riscv64-zvkned-zvkb.o obj-$(CONFIG_CRYPTO_CHACHA_RISCV64) += chacha-riscv64.o chacha-riscv64-y := chacha-riscv64-glue.o chacha-riscv64-zvkb.o obj-$(CONFIG_CRYPTO_GHASH_RISCV64) += ghash-riscv64.o ghash-riscv64-y := ghash-riscv64-glue.o ghash-riscv64-zvkg.o + +obj-$(CONFIG_CRYPTO_SHA256_RISCV64) += sha256-riscv64.o +sha256-riscv64-y := sha256-riscv64-glue.o sha256-riscv64-zvknha_or_zvknhb-zvkb.o diff --git a/arch/riscv/crypto/sha256-riscv64-glue.c b/arch/riscv/crypto/sha256-riscv64-glue.c new file mode 100644 index 0000000000000..71e051e40a64f --- /dev/null +++ b/arch/riscv/crypto/sha256-riscv64-glue.c @@ -0,0 +1,137 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * SHA-256 and SHA-224 using the RISC-V vector crypto extensions + * + * Copyright (C) 2022 VRULL GmbH + * Author: Heiko Stuebner + * + * Copyright (C) 2023 SiFive, Inc. + * Author: Jerry Shih + */ + +#include +#include +#include +#include +#include +#include +#include + +/* + * Note: the asm function only uses the 'state' field of struct sha256_state. + * It is assumed to be the first field. + */ +asmlinkage void sha256_transform_zvknha_or_zvknhb_zvkb( + struct sha256_state *state, const u8 *data, int num_blocks); + +static int riscv64_sha256_update(struct shash_desc *desc, const u8 *data, + unsigned int len) +{ + /* + * Ensure struct sha256_state begins directly with the SHA-256 + * 256-bit internal state, as this is what the asm function expects. + */ + BUILD_BUG_ON(offsetof(struct sha256_state, state) != 0); + + if (crypto_simd_usable()) { + kernel_vector_begin(); + sha256_base_do_update(desc, data, len, + sha256_transform_zvknha_or_zvknhb_zvkb); + kernel_vector_end(); + } else { + crypto_sha256_update(desc, data, len); + } + return 0; +} + +static int riscv64_sha256_finup(struct shash_desc *desc, const u8 *data, + unsigned int len, u8 *out) +{ + if (crypto_simd_usable()) { + kernel_vector_begin(); + if (len) + sha256_base_do_update( + desc, data, len, + sha256_transform_zvknha_or_zvknhb_zvkb); + sha256_base_do_finalize( + desc, sha256_transform_zvknha_or_zvknhb_zvkb); + kernel_vector_end(); + + return sha256_base_finish(desc, out); + } + + return crypto_sha256_finup(desc, data, len, out); +} + +static int riscv64_sha256_final(struct shash_desc *desc, u8 *out) +{ + return riscv64_sha256_finup(desc, NULL, 0, out); +} + +static int riscv64_sha256_digest(struct shash_desc *desc, const u8 *data, + unsigned int len, u8 *out) +{ + return sha256_base_init(desc) ?: + riscv64_sha256_finup(desc, data, len, out); +} + +static struct shash_alg riscv64_sha256_algs[] = { + { + .init = sha256_base_init, + .update = riscv64_sha256_update, + .final = riscv64_sha256_final, + .finup = riscv64_sha256_finup, + .digest = riscv64_sha256_digest, + .descsize = sizeof(struct sha256_state), + .digestsize = SHA256_DIGEST_SIZE, + .base = { + .cra_blocksize = SHA256_BLOCK_SIZE, + .cra_priority = 300, + .cra_name = "sha256", + .cra_driver_name = "sha256-riscv64-zvknha_or_zvknhb-zvkb", + .cra_module = THIS_MODULE, + }, + }, { + .init = sha224_base_init, + .update = riscv64_sha256_update, + .final = riscv64_sha256_final, + .finup = riscv64_sha256_finup, + .descsize = sizeof(struct sha256_state), + .digestsize = SHA224_DIGEST_SIZE, + .base = { + .cra_blocksize = SHA224_BLOCK_SIZE, + .cra_priority = 300, + .cra_name = "sha224", + .cra_driver_name = "sha224-riscv64-zvknha_or_zvknhb-zvkb", + .cra_module = THIS_MODULE, + }, + }, +}; + +static int __init riscv64_sha256_mod_init(void) +{ + /* Both zvknha and zvknhb provide the SHA-256 instructions. */ + if ((riscv_isa_extension_available(NULL, ZVKNHA) || + riscv_isa_extension_available(NULL, ZVKNHB)) && + riscv_isa_extension_available(NULL, ZVKB) && + riscv_vector_vlen() >= 128) + return crypto_register_shashes(riscv64_sha256_algs, + ARRAY_SIZE(riscv64_sha256_algs)); + + return -ENODEV; +} + +static void __exit riscv64_sha256_mod_exit(void) +{ + crypto_unregister_shashes(riscv64_sha256_algs, + ARRAY_SIZE(riscv64_sha256_algs)); +} + +module_init(riscv64_sha256_mod_init); +module_exit(riscv64_sha256_mod_exit); + +MODULE_DESCRIPTION("SHA-256 (RISC-V accelerated)"); +MODULE_AUTHOR("Heiko Stuebner "); +MODULE_LICENSE("GPL"); +MODULE_ALIAS_CRYPTO("sha256"); +MODULE_ALIAS_CRYPTO("sha224"); diff --git a/arch/riscv/crypto/sha256-riscv64-zvknha_or_zvknhb-zvkb.S b/arch/riscv/crypto/sha256-riscv64-zvknha_or_zvknhb-zvkb.S new file mode 100644 index 0000000000000..8ebcc17de4dce --- /dev/null +++ b/arch/riscv/crypto/sha256-riscv64-zvknha_or_zvknhb-zvkb.S @@ -0,0 +1,225 @@ +/* SPDX-License-Identifier: Apache-2.0 OR BSD-2-Clause */ +// +// This file is dual-licensed, meaning that you can use it under your +// choice of either of the following two licenses: +// +// Copyright 2023 The OpenSSL Project Authors. All Rights Reserved. +// +// Licensed under the Apache License 2.0 (the "License"). You can obtain +// a copy in the file LICENSE in the source distribution or at +// https://www.openssl.org/source/license.html +// +// or +// +// Copyright (c) 2023, Christoph Müllner +// Copyright (c) 2023, Phoebe Chen +// Copyright 2024 Google LLC +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// The generated code of this file depends on the following RISC-V extensions: +// - RV64I +// - RISC-V Vector ('V') with VLEN >= 128 +// - RISC-V Vector SHA-2 Secure Hash extension ('Zvknha' or 'Zvknhb') +// - RISC-V Vector Cryptography Bit-manipulation extension ('Zvkb') + +#include + +.text +.option arch, +zvknha, +zvkb + +#define STATEP a0 +#define DATA a1 +#define NUM_BLOCKS a2 + +#define STATEP_C a3 + +#define MASK v0 +#define INDICES v1 +#define W0 v2 +#define W1 v3 +#define W2 v4 +#define W3 v5 +#define VTMP v6 +#define FEBA v7 +#define HGDC v8 +#define K0 v10 +#define K1 v11 +#define K2 v12 +#define K3 v13 +#define K4 v14 +#define K5 v15 +#define K6 v16 +#define K7 v17 +#define K8 v18 +#define K9 v19 +#define K10 v20 +#define K11 v21 +#define K12 v22 +#define K13 v23 +#define K14 v24 +#define K15 v25 +#define PREV_FEBA v26 +#define PREV_HGDC v27 + +// Do 4 rounds of SHA-256. w0 contains the current 4 message schedule words. +// +// If not all the message schedule words have been computed yet, then this also +// computes 4 more message schedule words. w1-w3 contain the next 3 groups of 4 +// message schedule words; this macro computes the group after w3 and writes it +// to w0. This means that the next (w0, w1, w2, w3) is the current (w1, w2, w3, +// w0), so the caller must cycle through the registers accordingly. +.macro sha256_4rounds last, k, w0, w1, w2, w3 + vadd.vv VTMP, \k, \w0 + vsha2cl.vv HGDC, FEBA, VTMP + vsha2ch.vv FEBA, HGDC, VTMP +.if !\last + vmerge.vvm VTMP, \w2, \w1, MASK + vsha2ms.vv \w0, VTMP, \w3 +.endif +.endm + +.macro sha256_16rounds last, k0, k1, k2, k3 + sha256_4rounds \last, \k0, W0, W1, W2, W3 + sha256_4rounds \last, \k1, W1, W2, W3, W0 + sha256_4rounds \last, \k2, W2, W3, W0, W1 + sha256_4rounds \last, \k3, W3, W0, W1, W2 +.endm + +// void sha256_transform_zvknha_or_zvknhb_zvkb(u32 state[8], const u8 *data, +// int num_blocks); +SYM_TYPED_FUNC_START(sha256_transform_zvknha_or_zvknhb_zvkb) + + // Load the round constants into K0-K15. + vsetivli zero, 4, e32, m1, ta, ma + la t0, K256 + vle32.v K0, (t0) + addi t0, t0, 16 + vle32.v K1, (t0) + addi t0, t0, 16 + vle32.v K2, (t0) + addi t0, t0, 16 + vle32.v K3, (t0) + addi t0, t0, 16 + vle32.v K4, (t0) + addi t0, t0, 16 + vle32.v K5, (t0) + addi t0, t0, 16 + vle32.v K6, (t0) + addi t0, t0, 16 + vle32.v K7, (t0) + addi t0, t0, 16 + vle32.v K8, (t0) + addi t0, t0, 16 + vle32.v K9, (t0) + addi t0, t0, 16 + vle32.v K10, (t0) + addi t0, t0, 16 + vle32.v K11, (t0) + addi t0, t0, 16 + vle32.v K12, (t0) + addi t0, t0, 16 + vle32.v K13, (t0) + addi t0, t0, 16 + vle32.v K14, (t0) + addi t0, t0, 16 + vle32.v K15, (t0) + + // Setup mask for the vmerge to replace the first word (idx==0) in + // message scheduling. There are 4 words, so an 8-bit mask suffices. + vsetivli zero, 1, e8, m1, ta, ma + vmv.v.i MASK, 0x01 + + // Load the state. The state is stored as {a,b,c,d,e,f,g,h}, but we + // need {f,e,b,a},{h,g,d,c}. The dst vtype is e32m1 and the index vtype + // is e8mf4. We use index-load with the i8 indices {20, 16, 4, 0}, + // loaded using the 32-bit little endian value 0x00041014. + li t0, 0x00041014 + vsetivli zero, 1, e32, m1, ta, ma + vmv.v.x INDICES, t0 + addi STATEP_C, STATEP, 8 + vsetivli zero, 4, e32, m1, ta, ma + vluxei8.v FEBA, (STATEP), INDICES + vluxei8.v HGDC, (STATEP_C), INDICES + +.Lnext_block: + addi NUM_BLOCKS, NUM_BLOCKS, -1 + + // Save the previous state, as it's needed later. + vmv.v.v PREV_FEBA, FEBA + vmv.v.v PREV_HGDC, HGDC + + // Load the next 512-bit message block and endian-swap each 32-bit word. + vle32.v W0, (DATA) + vrev8.v W0, W0 + addi DATA, DATA, 16 + vle32.v W1, (DATA) + vrev8.v W1, W1 + addi DATA, DATA, 16 + vle32.v W2, (DATA) + vrev8.v W2, W2 + addi DATA, DATA, 16 + vle32.v W3, (DATA) + vrev8.v W3, W3 + addi DATA, DATA, 16 + + // Do the 64 rounds of SHA-256. + sha256_16rounds 0, K0, K1, K2, K3 + sha256_16rounds 0, K4, K5, K6, K7 + sha256_16rounds 0, K8, K9, K10, K11 + sha256_16rounds 1, K12, K13, K14, K15 + + // Add the previous state. + vadd.vv FEBA, FEBA, PREV_FEBA + vadd.vv HGDC, HGDC, PREV_HGDC + + // Repeat if more blocks remain. + bnez NUM_BLOCKS, .Lnext_block + + // Store the new state and return. + vsuxei8.v FEBA, (STATEP), INDICES + vsuxei8.v HGDC, (STATEP_C), INDICES + ret +SYM_FUNC_END(sha256_transform_zvknha_or_zvknhb_zvkb) + +.section ".rodata" +.p2align 2 +.type K256, @object +K256: + .word 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5 + .word 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5 + .word 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3 + .word 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174 + .word 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc + .word 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da + .word 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7 + .word 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967 + .word 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13 + .word 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85 + .word 0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3 + .word 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070 + .word 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5 + .word 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3 + .word 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208 + .word 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2 +.size K256, . - K256 -- 2.43.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0B9E6C47DA2 for ; Fri, 5 Jan 2024 18:51:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=FWLWrO8Y+D1Fkz5GxYi9ZvHJdr+K50iQzukyGJUiVUQ=; b=Tl2KElNckH1y0o d+nbiFSOA0gaRBUvbKugrISFOtSPNH0xGZFFwLLGnhiMPrGVHCWS1lABqVHXX304TLADczs/be76r vt6vL+vlQlhgu9p/SyLDsccqk7nCqFMxq8O3qoeXmkqJxjTUW8ygtr6S6kKmN+vnzbdlhUXKxMd0+ gMMhEDVbN4mpPNb7fRSV44caxCVHCCRqYIKpZluwnbWwpR5x1B7vkFmduQSv8e1WB3HUXHVWL3pnR DFx8KIgRGDTOAKr1C4gHSoXTgrxP6AMEk79KR/wtiUdhUplNHnIniwjpjY+hh9CU+PgTufrGokpFR qE0q8OcgEw82PPIRYfFg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rLpIE-0001Oh-27; Fri, 05 Jan 2024 18:51:42 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rLpI5-0001Is-2X for linux-riscv@lists.infradead.org; Fri, 05 Jan 2024 18:51:37 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 05C8F61B76; Fri, 5 Jan 2024 18:51:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 37B6DC433D9; Fri, 5 Jan 2024 18:51:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704480692; bh=Q9KorLkt9z4ETNv86tVce30LmXb337KOzYRmXS1ZQGc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bIBqIaiEYWEWx02zwyaHirNUpJcm10b4KacO6EhFXhcVN2GwLyTzQa4x3Ql32LNCa Mav+8JgMm1Tyth5+KS/96pluhr9QvSJkk4iviO2L52SgxCFwkWtJF7sEwg86Ulhmlp vNSlBAAmMJFDIWwi13ZJduR4T5f/CyyYZZeAAOz4ZykLxXJhDjDIR8OlHwKHhR4wLi I6JrkM/40hDv6zD2Qbb9/ytXA2WFfbouADlSmhLvb9B34xoU9y/GFd3dfG7oGFn0l/ fDZTTI9LOmG8EZHILx0kaf/acU4nGeRaOGX7rokiSyhwK/VWQSnoX6fBWQKG8BuIU9 pMMjDzVexMROw== From: Eric Biggers To: linux-crypto@vger.kernel.org, linux-riscv@lists.infradead.org, Jerry Shih Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Heiko Stuebner , Phoebe Chen , hongrong.hsu@sifive.com, Paul Walmsley , Palmer Dabbelt , Albert Ou , Andy Chiu , Charalampos Mitrodimas , Heiko Stuebner Subject: [PATCH v2 09/12] crypto: riscv - add vector crypto accelerated SHA-{256,224} Date: Fri, 5 Jan 2024 10:49:45 -0800 Message-ID: <20240105184950.43181-10-ebiggers@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240105184950.43181-1-ebiggers@kernel.org> References: <20240105184950.43181-1-ebiggers@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240105_105133_933680_C7DD0EE8 X-CRM114-Status: GOOD ( 30.92 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org RnJvbTogSmVycnkgU2hpaCA8amVycnkuc2hpaEBzaWZpdmUuY29tPgoKQWRkIGFuIGltcGxlbWVu dGF0aW9uIG9mIFNIQS0yNTYgYW5kIFNIQS0yMjQgdXNpbmcgdGhlIFp2a25oYSBvciBadmtuaGIK ZXh0ZW5zaW9uLiAgVGhlIGFzc2VtYmx5IGNvZGUgaXMgZGVyaXZlZCBmcm9tIE9wZW5TU0wgY29k ZQoob3BlbnNzbC9vcGVuc3NsIzIxOTIzKSB0aGF0IHdhcyBkdWFsLWxpY2Vuc2VkIHNvIHRoYXQg aXQgY291bGQgYmUKcmV1c2VkIGluIHRoZSBrZXJuZWwuICBOZXZlcnRoZWxlc3MsIHRoZSBhc3Nl bWJseSBoYXMgYmVlbiBzaWduaWZpY2FudGx5CnJld29ya2VkIGZvciBpbnRlZ3JhdGlvbiB3aXRo IHRoZSBrZXJuZWwsIGZvciBleGFtcGxlIGJ5IHVzaW5nIGEgcmVndWxhcgouUyBmaWxlIGluc3Rl YWQgb2YgdGhlIHNvLWNhbGxlZCBwZXJsYXNtLCB1c2luZyB0aGUgYXNzZW1ibGVyIGluc3RlYWQg b2YKYmFyZSAnLmluc3QnLCBhbmQgZ3JlYXRseSByZWR1Y2luZyBjb2RlIGR1cGxpY2F0aW9uLgoK Q28tZGV2ZWxvcGVkLWJ5OiBDaGFyYWxhbXBvcyBNaXRyb2RpbWFzIDxjaGFyYWxhbXBvcy5taXRy b2RpbWFzQHZydWxsLmV1PgpTaWduZWQtb2ZmLWJ5OiBDaGFyYWxhbXBvcyBNaXRyb2RpbWFzIDxj aGFyYWxhbXBvcy5taXRyb2RpbWFzQHZydWxsLmV1PgpDby1kZXZlbG9wZWQtYnk6IEhlaWtvIFN0 dWVibmVyIDxoZWlrby5zdHVlYm5lckB2cnVsbC5ldT4KU2lnbmVkLW9mZi1ieTogSGVpa28gU3R1 ZWJuZXIgPGhlaWtvLnN0dWVibmVyQHZydWxsLmV1PgpDby1kZXZlbG9wZWQtYnk6IFBob2ViZSBD aGVuIDxwaG9lYmUuY2hlbkBzaWZpdmUuY29tPgpTaWduZWQtb2ZmLWJ5OiBQaG9lYmUgQ2hlbiA8 cGhvZWJlLmNoZW5Ac2lmaXZlLmNvbT4KU2lnbmVkLW9mZi1ieTogSmVycnkgU2hpaCA8amVycnku c2hpaEBzaWZpdmUuY29tPgpDby1kZXZlbG9wZWQtYnk6IEVyaWMgQmlnZ2VycyA8ZWJpZ2dlcnNA Z29vZ2xlLmNvbT4KU2lnbmVkLW9mZi1ieTogRXJpYyBCaWdnZXJzIDxlYmlnZ2Vyc0Bnb29nbGUu Y29tPgotLS0KIGFyY2gvcmlzY3YvY3J5cHRvL0tjb25maWcgICAgICAgICAgICAgICAgICAgICB8 ICAxMSArCiBhcmNoL3Jpc2N2L2NyeXB0by9NYWtlZmlsZSAgICAgICAgICAgICAgICAgICAgfCAg IDMgKwogYXJjaC9yaXNjdi9jcnlwdG8vc2hhMjU2LXJpc2N2NjQtZ2x1ZS5jICAgICAgIHwgMTM3 ICsrKysrKysrKysrCiAuLi4vc2hhMjU2LXJpc2N2NjQtenZrbmhhX29yX3p2a25oYi16dmtiLlMg ICAgfCAyMjUgKysrKysrKysrKysrKysrKysrCiA0IGZpbGVzIGNoYW5nZWQsIDM3NiBpbnNlcnRp b25zKCspCiBjcmVhdGUgbW9kZSAxMDA2NDQgYXJjaC9yaXNjdi9jcnlwdG8vc2hhMjU2LXJpc2N2 NjQtZ2x1ZS5jCiBjcmVhdGUgbW9kZSAxMDA2NDQgYXJjaC9yaXNjdi9jcnlwdG8vc2hhMjU2LXJp c2N2NjQtenZrbmhhX29yX3p2a25oYi16dmtiLlMKCmRpZmYgLS1naXQgYS9hcmNoL3Jpc2N2L2Ny eXB0by9LY29uZmlnIGIvYXJjaC9yaXNjdi9jcnlwdG8vS2NvbmZpZwppbmRleCA2NzZiYTVhZjhm NTVhLi42ODdkYmI3MWY3ZDUwIDEwMDY0NAotLS0gYS9hcmNoL3Jpc2N2L2NyeXB0by9LY29uZmln CisrKyBiL2FyY2gvcmlzY3YvY3J5cHRvL0tjb25maWcKQEAgLTMyLDExICszMiwyMiBAQCBjb25m aWcgQ1JZUFRPX0NIQUNIQV9SSVNDVjY0CiBjb25maWcgQ1JZUFRPX0dIQVNIX1JJU0NWNjQKIAl0 cmlzdGF0ZSAiSGFzaCBmdW5jdGlvbnM6IEdIQVNIIgogCWRlcGVuZHMgb24gNjRCSVQgJiYgUklT Q1ZfSVNBX1YgJiYgVE9PTENIQUlOX0hBU19WRUNUT1JfQ1JZUFRPCiAJc2VsZWN0IENSWVBUT19H Q00KIAloZWxwCiAJICBHQ00gR0hBU0ggZnVuY3Rpb24gKE5JU1QgU1AgODAwLTM4RCkKIAogCSAg QXJjaGl0ZWN0dXJlOiByaXNjdjY0IHVzaW5nOgogCSAgLSBadmtnIHZlY3RvciBjcnlwdG8gZXh0 ZW5zaW9uCiAKK2NvbmZpZyBDUllQVE9fU0hBMjU2X1JJU0NWNjQKKwl0cmlzdGF0ZSAiSGFzaCBm dW5jdGlvbnM6IFNIQS0yMjQgYW5kIFNIQS0yNTYiCisJZGVwZW5kcyBvbiA2NEJJVCAmJiBSSVND Vl9JU0FfViAmJiBUT09MQ0hBSU5fSEFTX1ZFQ1RPUl9DUllQVE8KKwlzZWxlY3QgQ1JZUFRPX1NI QTI1NgorCWhlbHAKKwkgIFNIQS0yMjQgYW5kIFNIQS0yNTYgc2VjdXJlIGhhc2ggYWxnb3JpdGht IChGSVBTIDE4MCkKKworCSAgQXJjaGl0ZWN0dXJlOiByaXNjdjY0IHVzaW5nOgorCSAgLSBadmtu aGEgb3IgWnZrbmhiIHZlY3RvciBjcnlwdG8gZXh0ZW5zaW9ucworCSAgLSBadmtiIHZlY3RvciBj cnlwdG8gZXh0ZW5zaW9uCisKIGVuZG1lbnUKZGlmZiAtLWdpdCBhL2FyY2gvcmlzY3YvY3J5cHRv L01ha2VmaWxlIGIvYXJjaC9yaXNjdi9jcnlwdG8vTWFrZWZpbGUKaW5kZXggOTY2NmEyN2Q3Njg2 Mi4uODY3NGU5ODI5Y2RiMSAxMDA2NDQKLS0tIGEvYXJjaC9yaXNjdi9jcnlwdG8vTWFrZWZpbGUK KysrIGIvYXJjaC9yaXNjdi9jcnlwdG8vTWFrZWZpbGUKQEAgLTUsMTAgKzUsMTMgQEAKIAogb2Jq LSQoQ09ORklHX0NSWVBUT19BRVNfUklTQ1Y2NCkgKz0gYWVzLXJpc2N2NjQubwogYWVzLXJpc2N2 NjQteSA6PSBhZXMtcmlzY3Y2NC1nbHVlLm8gYWVzLXJpc2N2NjQtenZrbmVkLm8gXAogCQkgYWVz LXJpc2N2NjQtenZrbmVkLXp2YmItenZrZy5vIGFlcy1yaXNjdjY0LXp2a25lZC16dmtiLm8KIAog b2JqLSQoQ09ORklHX0NSWVBUT19DSEFDSEFfUklTQ1Y2NCkgKz0gY2hhY2hhLXJpc2N2NjQubwog Y2hhY2hhLXJpc2N2NjQteSA6PSBjaGFjaGEtcmlzY3Y2NC1nbHVlLm8gY2hhY2hhLXJpc2N2NjQt enZrYi5vCiAKIG9iai0kKENPTkZJR19DUllQVE9fR0hBU0hfUklTQ1Y2NCkgKz0gZ2hhc2gtcmlz Y3Y2NC5vCiBnaGFzaC1yaXNjdjY0LXkgOj0gZ2hhc2gtcmlzY3Y2NC1nbHVlLm8gZ2hhc2gtcmlz Y3Y2NC16dmtnLm8KKworb2JqLSQoQ09ORklHX0NSWVBUT19TSEEyNTZfUklTQ1Y2NCkgKz0gc2hh MjU2LXJpc2N2NjQubworc2hhMjU2LXJpc2N2NjQteSA6PSBzaGEyNTYtcmlzY3Y2NC1nbHVlLm8g c2hhMjU2LXJpc2N2NjQtenZrbmhhX29yX3p2a25oYi16dmtiLm8KZGlmZiAtLWdpdCBhL2FyY2gv cmlzY3YvY3J5cHRvL3NoYTI1Ni1yaXNjdjY0LWdsdWUuYyBiL2FyY2gvcmlzY3YvY3J5cHRvL3No YTI1Ni1yaXNjdjY0LWdsdWUuYwpuZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAwMDAwMDAwMDAw MDAwLi43MWUwNTFlNDBhNjRmCi0tLSAvZGV2L251bGwKKysrIGIvYXJjaC9yaXNjdi9jcnlwdG8v c2hhMjU2LXJpc2N2NjQtZ2x1ZS5jCkBAIC0wLDAgKzEsMTM3IEBACisvLyBTUERYLUxpY2Vuc2Ut SWRlbnRpZmllcjogR1BMLTIuMC1vci1sYXRlcgorLyoKKyAqIFNIQS0yNTYgYW5kIFNIQS0yMjQg dXNpbmcgdGhlIFJJU0MtViB2ZWN0b3IgY3J5cHRvIGV4dGVuc2lvbnMKKyAqCisgKiBDb3B5cmln aHQgKEMpIDIwMjIgVlJVTEwgR21iSAorICogQXV0aG9yOiBIZWlrbyBTdHVlYm5lciA8aGVpa28u c3R1ZWJuZXJAdnJ1bGwuZXU+CisgKgorICogQ29weXJpZ2h0IChDKSAyMDIzIFNpRml2ZSwgSW5j LgorICogQXV0aG9yOiBKZXJyeSBTaGloIDxqZXJyeS5zaGloQHNpZml2ZS5jb20+CisgKi8KKwor I2luY2x1ZGUgPGFzbS9zaW1kLmg+CisjaW5jbHVkZSA8YXNtL3ZlY3Rvci5oPgorI2luY2x1ZGUg PGNyeXB0by9pbnRlcm5hbC9oYXNoLmg+CisjaW5jbHVkZSA8Y3J5cHRvL2ludGVybmFsL3NpbWQu aD4KKyNpbmNsdWRlIDxjcnlwdG8vc2hhMjU2X2Jhc2UuaD4KKyNpbmNsdWRlIDxsaW51eC9saW5r YWdlLmg+CisjaW5jbHVkZSA8bGludXgvbW9kdWxlLmg+CisKKy8qCisgKiBOb3RlOiB0aGUgYXNt IGZ1bmN0aW9uIG9ubHkgdXNlcyB0aGUgJ3N0YXRlJyBmaWVsZCBvZiBzdHJ1Y3Qgc2hhMjU2X3N0 YXRlLgorICogSXQgaXMgYXNzdW1lZCB0byBiZSB0aGUgZmlyc3QgZmllbGQuCisgKi8KK2FzbWxp bmthZ2Ugdm9pZCBzaGEyNTZfdHJhbnNmb3JtX3p2a25oYV9vcl96dmtuaGJfenZrYigKKwlzdHJ1 Y3Qgc2hhMjU2X3N0YXRlICpzdGF0ZSwgY29uc3QgdTggKmRhdGEsIGludCBudW1fYmxvY2tzKTsK Kworc3RhdGljIGludCByaXNjdjY0X3NoYTI1Nl91cGRhdGUoc3RydWN0IHNoYXNoX2Rlc2MgKmRl c2MsIGNvbnN0IHU4ICpkYXRhLAorCQkJCSB1bnNpZ25lZCBpbnQgbGVuKQoreworCS8qCisJICog RW5zdXJlIHN0cnVjdCBzaGEyNTZfc3RhdGUgYmVnaW5zIGRpcmVjdGx5IHdpdGggdGhlIFNIQS0y NTYKKwkgKiAyNTYtYml0IGludGVybmFsIHN0YXRlLCBhcyB0aGlzIGlzIHdoYXQgdGhlIGFzbSBm dW5jdGlvbiBleHBlY3RzLgorCSAqLworCUJVSUxEX0JVR19PTihvZmZzZXRvZihzdHJ1Y3Qgc2hh MjU2X3N0YXRlLCBzdGF0ZSkgIT0gMCk7CisKKwlpZiAoY3J5cHRvX3NpbWRfdXNhYmxlKCkpIHsK KwkJa2VybmVsX3ZlY3Rvcl9iZWdpbigpOworCQlzaGEyNTZfYmFzZV9kb191cGRhdGUoZGVzYywg ZGF0YSwgbGVuLAorCQkJCSAgICAgIHNoYTI1Nl90cmFuc2Zvcm1fenZrbmhhX29yX3p2a25oYl96 dmtiKTsKKwkJa2VybmVsX3ZlY3Rvcl9lbmQoKTsKKwl9IGVsc2UgeworCQljcnlwdG9fc2hhMjU2 X3VwZGF0ZShkZXNjLCBkYXRhLCBsZW4pOworCX0KKwlyZXR1cm4gMDsKK30KKworc3RhdGljIGlu dCByaXNjdjY0X3NoYTI1Nl9maW51cChzdHJ1Y3Qgc2hhc2hfZGVzYyAqZGVzYywgY29uc3QgdTgg KmRhdGEsCisJCQkJdW5zaWduZWQgaW50IGxlbiwgdTggKm91dCkKK3sKKwlpZiAoY3J5cHRvX3Np bWRfdXNhYmxlKCkpIHsKKwkJa2VybmVsX3ZlY3Rvcl9iZWdpbigpOworCQlpZiAobGVuKQorCQkJ c2hhMjU2X2Jhc2VfZG9fdXBkYXRlKAorCQkJCWRlc2MsIGRhdGEsIGxlbiwKKwkJCQlzaGEyNTZf dHJhbnNmb3JtX3p2a25oYV9vcl96dmtuaGJfenZrYik7CisJCXNoYTI1Nl9iYXNlX2RvX2ZpbmFs aXplKAorCQkJZGVzYywgc2hhMjU2X3RyYW5zZm9ybV96dmtuaGFfb3JfenZrbmhiX3p2a2IpOwor CQlrZXJuZWxfdmVjdG9yX2VuZCgpOworCisJCXJldHVybiBzaGEyNTZfYmFzZV9maW5pc2goZGVz Yywgb3V0KTsKKwl9CisKKwlyZXR1cm4gY3J5cHRvX3NoYTI1Nl9maW51cChkZXNjLCBkYXRhLCBs ZW4sIG91dCk7Cit9CisKK3N0YXRpYyBpbnQgcmlzY3Y2NF9zaGEyNTZfZmluYWwoc3RydWN0IHNo YXNoX2Rlc2MgKmRlc2MsIHU4ICpvdXQpCit7CisJcmV0dXJuIHJpc2N2NjRfc2hhMjU2X2ZpbnVw KGRlc2MsIE5VTEwsIDAsIG91dCk7Cit9CisKK3N0YXRpYyBpbnQgcmlzY3Y2NF9zaGEyNTZfZGln ZXN0KHN0cnVjdCBzaGFzaF9kZXNjICpkZXNjLCBjb25zdCB1OCAqZGF0YSwKKwkJCQkgdW5zaWdu ZWQgaW50IGxlbiwgdTggKm91dCkKK3sKKwlyZXR1cm4gc2hhMjU2X2Jhc2VfaW5pdChkZXNjKSA/ OgorCSAgICAgICByaXNjdjY0X3NoYTI1Nl9maW51cChkZXNjLCBkYXRhLCBsZW4sIG91dCk7Cit9 CisKK3N0YXRpYyBzdHJ1Y3Qgc2hhc2hfYWxnIHJpc2N2NjRfc2hhMjU2X2FsZ3NbXSA9IHsKKwl7 CisJCS5pbml0ID0gc2hhMjU2X2Jhc2VfaW5pdCwKKwkJLnVwZGF0ZSA9IHJpc2N2NjRfc2hhMjU2 X3VwZGF0ZSwKKwkJLmZpbmFsID0gcmlzY3Y2NF9zaGEyNTZfZmluYWwsCisJCS5maW51cCA9IHJp c2N2NjRfc2hhMjU2X2ZpbnVwLAorCQkuZGlnZXN0ID0gcmlzY3Y2NF9zaGEyNTZfZGlnZXN0LAor CQkuZGVzY3NpemUgPSBzaXplb2Yoc3RydWN0IHNoYTI1Nl9zdGF0ZSksCisJCS5kaWdlc3RzaXpl ID0gU0hBMjU2X0RJR0VTVF9TSVpFLAorCQkuYmFzZSA9IHsKKwkJCS5jcmFfYmxvY2tzaXplID0g U0hBMjU2X0JMT0NLX1NJWkUsCisJCQkuY3JhX3ByaW9yaXR5ID0gMzAwLAorCQkJLmNyYV9uYW1l ID0gInNoYTI1NiIsCisJCQkuY3JhX2RyaXZlcl9uYW1lID0gInNoYTI1Ni1yaXNjdjY0LXp2a25o YV9vcl96dmtuaGItenZrYiIsCisJCQkuY3JhX21vZHVsZSA9IFRISVNfTU9EVUxFLAorCQl9LAor CX0sIHsKKwkJLmluaXQgPSBzaGEyMjRfYmFzZV9pbml0LAorCQkudXBkYXRlID0gcmlzY3Y2NF9z aGEyNTZfdXBkYXRlLAorCQkuZmluYWwgPSByaXNjdjY0X3NoYTI1Nl9maW5hbCwKKwkJLmZpbnVw ID0gcmlzY3Y2NF9zaGEyNTZfZmludXAsCisJCS5kZXNjc2l6ZSA9IHNpemVvZihzdHJ1Y3Qgc2hh MjU2X3N0YXRlKSwKKwkJLmRpZ2VzdHNpemUgPSBTSEEyMjRfRElHRVNUX1NJWkUsCisJCS5iYXNl ID0geworCQkJLmNyYV9ibG9ja3NpemUgPSBTSEEyMjRfQkxPQ0tfU0laRSwKKwkJCS5jcmFfcHJp b3JpdHkgPSAzMDAsCisJCQkuY3JhX25hbWUgPSAic2hhMjI0IiwKKwkJCS5jcmFfZHJpdmVyX25h bWUgPSAic2hhMjI0LXJpc2N2NjQtenZrbmhhX29yX3p2a25oYi16dmtiIiwKKwkJCS5jcmFfbW9k dWxlID0gVEhJU19NT0RVTEUsCisJCX0sCisJfSwKK307CisKK3N0YXRpYyBpbnQgX19pbml0IHJp c2N2NjRfc2hhMjU2X21vZF9pbml0KHZvaWQpCit7CisJLyogQm90aCB6dmtuaGEgYW5kIHp2a25o YiBwcm92aWRlIHRoZSBTSEEtMjU2IGluc3RydWN0aW9ucy4gKi8KKwlpZiAoKHJpc2N2X2lzYV9l eHRlbnNpb25fYXZhaWxhYmxlKE5VTEwsIFpWS05IQSkgfHwKKwkgICAgIHJpc2N2X2lzYV9leHRl bnNpb25fYXZhaWxhYmxlKE5VTEwsIFpWS05IQikpICYmCisJICAgIHJpc2N2X2lzYV9leHRlbnNp b25fYXZhaWxhYmxlKE5VTEwsIFpWS0IpICYmCisJICAgIHJpc2N2X3ZlY3Rvcl92bGVuKCkgPj0g MTI4KQorCQlyZXR1cm4gY3J5cHRvX3JlZ2lzdGVyX3NoYXNoZXMocmlzY3Y2NF9zaGEyNTZfYWxn cywKKwkJCQkJICAgICAgIEFSUkFZX1NJWkUocmlzY3Y2NF9zaGEyNTZfYWxncykpOworCisJcmV0 dXJuIC1FTk9ERVY7Cit9CisKK3N0YXRpYyB2b2lkIF9fZXhpdCByaXNjdjY0X3NoYTI1Nl9tb2Rf ZXhpdCh2b2lkKQoreworCWNyeXB0b191bnJlZ2lzdGVyX3NoYXNoZXMocmlzY3Y2NF9zaGEyNTZf YWxncywKKwkJCQkgIEFSUkFZX1NJWkUocmlzY3Y2NF9zaGEyNTZfYWxncykpOworfQorCittb2R1 bGVfaW5pdChyaXNjdjY0X3NoYTI1Nl9tb2RfaW5pdCk7Cittb2R1bGVfZXhpdChyaXNjdjY0X3No YTI1Nl9tb2RfZXhpdCk7CisKK01PRFVMRV9ERVNDUklQVElPTigiU0hBLTI1NiAoUklTQy1WIGFj Y2VsZXJhdGVkKSIpOworTU9EVUxFX0FVVEhPUigiSGVpa28gU3R1ZWJuZXIgPGhlaWtvLnN0dWVi bmVyQHZydWxsLmV1PiIpOworTU9EVUxFX0xJQ0VOU0UoIkdQTCIpOworTU9EVUxFX0FMSUFTX0NS WVBUTygic2hhMjU2Iik7CitNT0RVTEVfQUxJQVNfQ1JZUFRPKCJzaGEyMjQiKTsKZGlmZiAtLWdp dCBhL2FyY2gvcmlzY3YvY3J5cHRvL3NoYTI1Ni1yaXNjdjY0LXp2a25oYV9vcl96dmtuaGItenZr Yi5TIGIvYXJjaC9yaXNjdi9jcnlwdG8vc2hhMjU2LXJpc2N2NjQtenZrbmhhX29yX3p2a25oYi16 dmtiLlMKbmV3IGZpbGUgbW9kZSAxMDA2NDQKaW5kZXggMDAwMDAwMDAwMDAwMC4uOGViY2MxN2Rl NGRjZQotLS0gL2Rldi9udWxsCisrKyBiL2FyY2gvcmlzY3YvY3J5cHRvL3NoYTI1Ni1yaXNjdjY0 LXp2a25oYV9vcl96dmtuaGItenZrYi5TCkBAIC0wLDAgKzEsMjI1IEBACisvKiBTUERYLUxpY2Vu c2UtSWRlbnRpZmllcjogQXBhY2hlLTIuMCBPUiBCU0QtMi1DbGF1c2UgKi8KKy8vCisvLyBUaGlz IGZpbGUgaXMgZHVhbC1saWNlbnNlZCwgbWVhbmluZyB0aGF0IHlvdSBjYW4gdXNlIGl0IHVuZGVy IHlvdXIKKy8vIGNob2ljZSBvZiBlaXRoZXIgb2YgdGhlIGZvbGxvd2luZyB0d28gbGljZW5zZXM6 CisvLworLy8gQ29weXJpZ2h0IDIwMjMgVGhlIE9wZW5TU0wgUHJvamVjdCBBdXRob3JzLiBBbGwg UmlnaHRzIFJlc2VydmVkLgorLy8KKy8vIExpY2Vuc2VkIHVuZGVyIHRoZSBBcGFjaGUgTGljZW5z ZSAyLjAgKHRoZSAiTGljZW5zZSIpLiBZb3UgY2FuIG9idGFpbgorLy8gYSBjb3B5IGluIHRoZSBm aWxlIExJQ0VOU0UgaW4gdGhlIHNvdXJjZSBkaXN0cmlidXRpb24gb3IgYXQKKy8vIGh0dHBzOi8v d3d3Lm9wZW5zc2wub3JnL3NvdXJjZS9saWNlbnNlLmh0bWwKKy8vCisvLyBvcgorLy8KKy8vIENv cHlyaWdodCAoYykgMjAyMywgQ2hyaXN0b3BoIE3DvGxsbmVyIDxjaHJpc3RvcGgubXVlbGxuZXJA dnJ1bGwuZXU+CisvLyBDb3B5cmlnaHQgKGMpIDIwMjMsIFBob2ViZSBDaGVuIDxwaG9lYmUuY2hl bkBzaWZpdmUuY29tPgorLy8gQ29weXJpZ2h0IDIwMjQgR29vZ2xlIExMQworLy8gQWxsIHJpZ2h0 cyByZXNlcnZlZC4KKy8vCisvLyBSZWRpc3RyaWJ1dGlvbiBhbmQgdXNlIGluIHNvdXJjZSBhbmQg YmluYXJ5IGZvcm1zLCB3aXRoIG9yIHdpdGhvdXQKKy8vIG1vZGlmaWNhdGlvbiwgYXJlIHBlcm1p dHRlZCBwcm92aWRlZCB0aGF0IHRoZSBmb2xsb3dpbmcgY29uZGl0aW9ucworLy8gYXJlIG1ldDoK Ky8vIDEuIFJlZGlzdHJpYnV0aW9ucyBvZiBzb3VyY2UgY29kZSBtdXN0IHJldGFpbiB0aGUgYWJv dmUgY29weXJpZ2h0CisvLyAgICBub3RpY2UsIHRoaXMgbGlzdCBvZiBjb25kaXRpb25zIGFuZCB0 aGUgZm9sbG93aW5nIGRpc2NsYWltZXIuCisvLyAyLiBSZWRpc3RyaWJ1dGlvbnMgaW4gYmluYXJ5 IGZvcm0gbXVzdCByZXByb2R1Y2UgdGhlIGFib3ZlIGNvcHlyaWdodAorLy8gICAgbm90aWNlLCB0 aGlzIGxpc3Qgb2YgY29uZGl0aW9ucyBhbmQgdGhlIGZvbGxvd2luZyBkaXNjbGFpbWVyIGluIHRo ZQorLy8gICAgZG9jdW1lbnRhdGlvbiBhbmQvb3Igb3RoZXIgbWF0ZXJpYWxzIHByb3ZpZGVkIHdp dGggdGhlIGRpc3RyaWJ1dGlvbi4KKy8vCisvLyBUSElTIFNPRlRXQVJFIElTIFBST1ZJREVEIEJZ IFRIRSBDT1BZUklHSFQgSE9MREVSUyBBTkQgQ09OVFJJQlVUT1JTCisvLyAiQVMgSVMiIEFORCBB TlkgRVhQUkVTUyBPUiBJTVBMSUVEIFdBUlJBTlRJRVMsIElOQ0xVRElORywgQlVUIE5PVAorLy8g TElNSVRFRCBUTywgVEhFIElNUExJRUQgV0FSUkFOVElFUyBPRiBNRVJDSEFOVEFCSUxJVFkgQU5E IEZJVE5FU1MgRk9SCisvLyBBIFBBUlRJQ1VMQVIgUFVSUE9TRSBBUkUgRElTQ0xBSU1FRC4gSU4g Tk8gRVZFTlQgU0hBTEwgVEhFIENPUFlSSUdIVAorLy8gT1dORVIgT1IgQ09OVFJJQlVUT1JTIEJF IExJQUJMRSBGT1IgQU5ZIERJUkVDVCwgSU5ESVJFQ1QsIElOQ0lERU5UQUwsCisvLyBTUEVDSUFM LCBFWEVNUExBUlksIE9SIENPTlNFUVVFTlRJQUwgREFNQUdFUyAoSU5DTFVESU5HLCBCVVQgTk9U CisvLyBMSU1JVEVEIFRPLCBQUk9DVVJFTUVOVCBPRiBTVUJTVElUVVRFIEdPT0RTIE9SIFNFUlZJ Q0VTOyBMT1NTIE9GIFVTRSwKKy8vIERBVEEsIE9SIFBST0ZJVFM7IE9SIEJVU0lORVNTIElOVEVS UlVQVElPTikgSE9XRVZFUiBDQVVTRUQgQU5EIE9OIEFOWQorLy8gVEhFT1JZIE9GIExJQUJJTElU WSwgV0hFVEhFUiBJTiBDT05UUkFDVCwgU1RSSUNUIExJQUJJTElUWSwgT1IgVE9SVAorLy8gKElO Q0xVRElORyBORUdMSUdFTkNFIE9SIE9USEVSV0lTRSkgQVJJU0lORyBJTiBBTlkgV0FZIE9VVCBP RiBUSEUgVVNFCisvLyBPRiBUSElTIFNPRlRXQVJFLCBFVkVOIElGIEFEVklTRUQgT0YgVEhFIFBP U1NJQklMSVRZIE9GIFNVQ0ggREFNQUdFLgorCisvLyBUaGUgZ2VuZXJhdGVkIGNvZGUgb2YgdGhp cyBmaWxlIGRlcGVuZHMgb24gdGhlIGZvbGxvd2luZyBSSVNDLVYgZXh0ZW5zaW9uczoKKy8vIC0g UlY2NEkKKy8vIC0gUklTQy1WIFZlY3RvciAoJ1YnKSB3aXRoIFZMRU4gPj0gMTI4CisvLyAtIFJJ U0MtViBWZWN0b3IgU0hBLTIgU2VjdXJlIEhhc2ggZXh0ZW5zaW9uICgnWnZrbmhhJyBvciAnWnZr bmhiJykKKy8vIC0gUklTQy1WIFZlY3RvciBDcnlwdG9ncmFwaHkgQml0LW1hbmlwdWxhdGlvbiBl eHRlbnNpb24gKCdadmtiJykKKworI2luY2x1ZGUgPGxpbnV4L2NmaV90eXBlcy5oPgorCisudGV4 dAorLm9wdGlvbiBhcmNoLCArenZrbmhhLCArenZrYgorCisjZGVmaW5lIFNUQVRFUAkJYTAKKyNk ZWZpbmUgREFUQQkJYTEKKyNkZWZpbmUgTlVNX0JMT0NLUwlhMgorCisjZGVmaW5lIFNUQVRFUF9D CWEzCisKKyNkZWZpbmUgTUFTSwkJdjAKKyNkZWZpbmUgSU5ESUNFUwkJdjEKKyNkZWZpbmUgVzAJ CXYyCisjZGVmaW5lIFcxCQl2MworI2RlZmluZSBXMgkJdjQKKyNkZWZpbmUgVzMJCXY1CisjZGVm aW5lIFZUTVAJCXY2CisjZGVmaW5lIEZFQkEJCXY3CisjZGVmaW5lIEhHREMJCXY4CisjZGVmaW5l IEswCQl2MTAKKyNkZWZpbmUgSzEJCXYxMQorI2RlZmluZSBLMgkJdjEyCisjZGVmaW5lIEszCQl2 MTMKKyNkZWZpbmUgSzQJCXYxNAorI2RlZmluZSBLNQkJdjE1CisjZGVmaW5lIEs2CQl2MTYKKyNk ZWZpbmUgSzcJCXYxNworI2RlZmluZSBLOAkJdjE4CisjZGVmaW5lIEs5CQl2MTkKKyNkZWZpbmUg SzEwCQl2MjAKKyNkZWZpbmUgSzExCQl2MjEKKyNkZWZpbmUgSzEyCQl2MjIKKyNkZWZpbmUgSzEz CQl2MjMKKyNkZWZpbmUgSzE0CQl2MjQKKyNkZWZpbmUgSzE1CQl2MjUKKyNkZWZpbmUgUFJFVl9G RUJBCXYyNgorI2RlZmluZSBQUkVWX0hHREMJdjI3CisKKy8vIERvIDQgcm91bmRzIG9mIFNIQS0y NTYuICB3MCBjb250YWlucyB0aGUgY3VycmVudCA0IG1lc3NhZ2Ugc2NoZWR1bGUgd29yZHMuCisv LworLy8gSWYgbm90IGFsbCB0aGUgbWVzc2FnZSBzY2hlZHVsZSB3b3JkcyBoYXZlIGJlZW4gY29t cHV0ZWQgeWV0LCB0aGVuIHRoaXMgYWxzbworLy8gY29tcHV0ZXMgNCBtb3JlIG1lc3NhZ2Ugc2No ZWR1bGUgd29yZHMuICB3MS13MyBjb250YWluIHRoZSBuZXh0IDMgZ3JvdXBzIG9mIDQKKy8vIG1l c3NhZ2Ugc2NoZWR1bGUgd29yZHM7IHRoaXMgbWFjcm8gY29tcHV0ZXMgdGhlIGdyb3VwIGFmdGVy IHczIGFuZCB3cml0ZXMgaXQKKy8vIHRvIHcwLiAgVGhpcyBtZWFucyB0aGF0IHRoZSBuZXh0ICh3 MCwgdzEsIHcyLCB3MykgaXMgdGhlIGN1cnJlbnQgKHcxLCB3MiwgdzMsCisvLyB3MCksIHNvIHRo ZSBjYWxsZXIgbXVzdCBjeWNsZSB0aHJvdWdoIHRoZSByZWdpc3RlcnMgYWNjb3JkaW5nbHkuCisu bWFjcm8Jc2hhMjU2XzRyb3VuZHMJbGFzdCwgaywgdzAsIHcxLCB3MiwgdzMKKwl2YWRkLnZ2CQlW VE1QLCBcaywgXHcwCisJdnNoYTJjbC52dglIR0RDLCBGRUJBLCBWVE1QCisJdnNoYTJjaC52dglG RUJBLCBIR0RDLCBWVE1QCisuaWYgIVxsYXN0CisJdm1lcmdlLnZ2bQlWVE1QLCBcdzIsIFx3MSwg TUFTSworCXZzaGEybXMudnYJXHcwLCBWVE1QLCBcdzMKKy5lbmRpZgorLmVuZG0KKworLm1hY3Jv CXNoYTI1Nl8xNnJvdW5kcwlsYXN0LCBrMCwgazEsIGsyLCBrMworCXNoYTI1Nl80cm91bmRzCVxs YXN0LCBcazAsIFcwLCBXMSwgVzIsIFczCisJc2hhMjU2XzRyb3VuZHMJXGxhc3QsIFxrMSwgVzEs IFcyLCBXMywgVzAKKwlzaGEyNTZfNHJvdW5kcwlcbGFzdCwgXGsyLCBXMiwgVzMsIFcwLCBXMQor CXNoYTI1Nl80cm91bmRzCVxsYXN0LCBcazMsIFczLCBXMCwgVzEsIFcyCisuZW5kbQorCisvLyB2 b2lkIHNoYTI1Nl90cmFuc2Zvcm1fenZrbmhhX29yX3p2a25oYl96dmtiKHUzMiBzdGF0ZVs4XSwg Y29uc3QgdTggKmRhdGEsCisvLwkJCQkJICAgICAgIGludCBudW1fYmxvY2tzKTsKK1NZTV9UWVBF RF9GVU5DX1NUQVJUKHNoYTI1Nl90cmFuc2Zvcm1fenZrbmhhX29yX3p2a25oYl96dmtiKQorCisJ Ly8gTG9hZCB0aGUgcm91bmQgY29uc3RhbnRzIGludG8gSzAtSzE1LgorCXZzZXRpdmxpCXplcm8s IDQsIGUzMiwgbTEsIHRhLCBtYQorCWxhCQl0MCwgSzI1NgorCXZsZTMyLnYJCUswLCAodDApCisJ YWRkaQkJdDAsIHQwLCAxNgorCXZsZTMyLnYJCUsxLCAodDApCisJYWRkaQkJdDAsIHQwLCAxNgor CXZsZTMyLnYJCUsyLCAodDApCisJYWRkaQkJdDAsIHQwLCAxNgorCXZsZTMyLnYJCUszLCAodDAp CisJYWRkaQkJdDAsIHQwLCAxNgorCXZsZTMyLnYJCUs0LCAodDApCisJYWRkaQkJdDAsIHQwLCAx NgorCXZsZTMyLnYJCUs1LCAodDApCisJYWRkaQkJdDAsIHQwLCAxNgorCXZsZTMyLnYJCUs2LCAo dDApCisJYWRkaQkJdDAsIHQwLCAxNgorCXZsZTMyLnYJCUs3LCAodDApCisJYWRkaQkJdDAsIHQw LCAxNgorCXZsZTMyLnYJCUs4LCAodDApCisJYWRkaQkJdDAsIHQwLCAxNgorCXZsZTMyLnYJCUs5 LCAodDApCisJYWRkaQkJdDAsIHQwLCAxNgorCXZsZTMyLnYJCUsxMCwgKHQwKQorCWFkZGkJCXQw LCB0MCwgMTYKKwl2bGUzMi52CQlLMTEsICh0MCkKKwlhZGRpCQl0MCwgdDAsIDE2CisJdmxlMzIu dgkJSzEyLCAodDApCisJYWRkaQkJdDAsIHQwLCAxNgorCXZsZTMyLnYJCUsxMywgKHQwKQorCWFk ZGkJCXQwLCB0MCwgMTYKKwl2bGUzMi52CQlLMTQsICh0MCkKKwlhZGRpCQl0MCwgdDAsIDE2CisJ dmxlMzIudgkJSzE1LCAodDApCisKKwkvLyBTZXR1cCBtYXNrIGZvciB0aGUgdm1lcmdlIHRvIHJl cGxhY2UgdGhlIGZpcnN0IHdvcmQgKGlkeD09MCkgaW4KKwkvLyBtZXNzYWdlIHNjaGVkdWxpbmcu ICBUaGVyZSBhcmUgNCB3b3Jkcywgc28gYW4gOC1iaXQgbWFzayBzdWZmaWNlcy4KKwl2c2V0aXZs aQl6ZXJvLCAxLCBlOCwgbTEsIHRhLCBtYQorCXZtdi52LmkJCU1BU0ssIDB4MDEKKworCS8vIExv YWQgdGhlIHN0YXRlLiAgVGhlIHN0YXRlIGlzIHN0b3JlZCBhcyB7YSxiLGMsZCxlLGYsZyxofSwg YnV0IHdlCisJLy8gbmVlZCB7ZixlLGIsYX0se2gsZyxkLGN9LiAgVGhlIGRzdCB2dHlwZSBpcyBl MzJtMSBhbmQgdGhlIGluZGV4IHZ0eXBlCisJLy8gaXMgZThtZjQuICBXZSB1c2UgaW5kZXgtbG9h ZCB3aXRoIHRoZSBpOCBpbmRpY2VzIHsyMCwgMTYsIDQsIDB9LAorCS8vIGxvYWRlZCB1c2luZyB0 aGUgMzItYml0IGxpdHRsZSBlbmRpYW4gdmFsdWUgMHgwMDA0MTAxNC4KKwlsaQkJdDAsIDB4MDAw NDEwMTQKKwl2c2V0aXZsaQl6ZXJvLCAxLCBlMzIsIG0xLCB0YSwgbWEKKwl2bXYudi54CQlJTkRJ Q0VTLCB0MAorCWFkZGkJCVNUQVRFUF9DLCBTVEFURVAsIDgKKwl2c2V0aXZsaQl6ZXJvLCA0LCBl MzIsIG0xLCB0YSwgbWEKKwl2bHV4ZWk4LnYJRkVCQSwgKFNUQVRFUCksIElORElDRVMKKwl2bHV4 ZWk4LnYJSEdEQywgKFNUQVRFUF9DKSwgSU5ESUNFUworCisuTG5leHRfYmxvY2s6CisJYWRkaQkJ TlVNX0JMT0NLUywgTlVNX0JMT0NLUywgLTEKKworCS8vIFNhdmUgdGhlIHByZXZpb3VzIHN0YXRl LCBhcyBpdCdzIG5lZWRlZCBsYXRlci4KKwl2bXYudi52CQlQUkVWX0ZFQkEsIEZFQkEKKwl2bXYu di52CQlQUkVWX0hHREMsIEhHREMKKworCS8vIExvYWQgdGhlIG5leHQgNTEyLWJpdCBtZXNzYWdl IGJsb2NrIGFuZCBlbmRpYW4tc3dhcCBlYWNoIDMyLWJpdCB3b3JkLgorCXZsZTMyLnYJCVcwLCAo REFUQSkKKwl2cmV2OC52CQlXMCwgVzAKKwlhZGRpCQlEQVRBLCBEQVRBLCAxNgorCXZsZTMyLnYJ CVcxLCAoREFUQSkKKwl2cmV2OC52CQlXMSwgVzEKKwlhZGRpCQlEQVRBLCBEQVRBLCAxNgorCXZs ZTMyLnYJCVcyLCAoREFUQSkKKwl2cmV2OC52CQlXMiwgVzIKKwlhZGRpCQlEQVRBLCBEQVRBLCAx NgorCXZsZTMyLnYJCVczLCAoREFUQSkKKwl2cmV2OC52CQlXMywgVzMKKwlhZGRpCQlEQVRBLCBE QVRBLCAxNgorCisJLy8gRG8gdGhlIDY0IHJvdW5kcyBvZiBTSEEtMjU2LgorCXNoYTI1Nl8xNnJv dW5kcwkwLCBLMCwgSzEsIEsyLCBLMworCXNoYTI1Nl8xNnJvdW5kcwkwLCBLNCwgSzUsIEs2LCBL NworCXNoYTI1Nl8xNnJvdW5kcwkwLCBLOCwgSzksIEsxMCwgSzExCisJc2hhMjU2XzE2cm91bmRz CTEsIEsxMiwgSzEzLCBLMTQsIEsxNQorCisJLy8gQWRkIHRoZSBwcmV2aW91cyBzdGF0ZS4KKwl2 YWRkLnZ2CQlGRUJBLCBGRUJBLCBQUkVWX0ZFQkEKKwl2YWRkLnZ2CQlIR0RDLCBIR0RDLCBQUkVW X0hHREMKKworCS8vIFJlcGVhdCBpZiBtb3JlIGJsb2NrcyByZW1haW4uCisJYm5legkJTlVNX0JM T0NLUywgLkxuZXh0X2Jsb2NrCisKKwkvLyBTdG9yZSB0aGUgbmV3IHN0YXRlIGFuZCByZXR1cm4u CisJdnN1eGVpOC52CUZFQkEsIChTVEFURVApLCBJTkRJQ0VTCisJdnN1eGVpOC52CUhHREMsIChT VEFURVBfQyksIElORElDRVMKKwlyZXQKK1NZTV9GVU5DX0VORChzaGEyNTZfdHJhbnNmb3JtX3p2 a25oYV9vcl96dmtuaGJfenZrYikKKworLnNlY3Rpb24gIi5yb2RhdGEiCisucDJhbGlnbiAyCisu dHlwZSBLMjU2LCBAb2JqZWN0CitLMjU2OgorCS53b3JkCQkweDQyOGEyZjk4LCAweDcxMzc0NDkx LCAweGI1YzBmYmNmLCAweGU5YjVkYmE1CisJLndvcmQJCTB4Mzk1NmMyNWIsIDB4NTlmMTExZjEs IDB4OTIzZjgyYTQsIDB4YWIxYzVlZDUKKwkud29yZAkJMHhkODA3YWE5OCwgMHgxMjgzNWIwMSwg MHgyNDMxODViZSwgMHg1NTBjN2RjMworCS53b3JkCQkweDcyYmU1ZDc0LCAweDgwZGViMWZlLCAw eDliZGMwNmE3LCAweGMxOWJmMTc0CisJLndvcmQJCTB4ZTQ5YjY5YzEsIDB4ZWZiZTQ3ODYsIDB4 MGZjMTlkYzYsIDB4MjQwY2ExY2MKKwkud29yZAkJMHgyZGU5MmM2ZiwgMHg0YTc0ODRhYSwgMHg1 Y2IwYTlkYywgMHg3NmY5ODhkYQorCS53b3JkCQkweDk4M2U1MTUyLCAweGE4MzFjNjZkLCAweGIw MDMyN2M4LCAweGJmNTk3ZmM3CisJLndvcmQJCTB4YzZlMDBiZjMsIDB4ZDVhNzkxNDcsIDB4MDZj YTYzNTEsIDB4MTQyOTI5NjcKKwkud29yZAkJMHgyN2I3MGE4NSwgMHgyZTFiMjEzOCwgMHg0ZDJj NmRmYywgMHg1MzM4MGQxMworCS53b3JkCQkweDY1MGE3MzU0LCAweDc2NmEwYWJiLCAweDgxYzJj OTJlLCAweDkyNzIyYzg1CisJLndvcmQJCTB4YTJiZmU4YTEsIDB4YTgxYTY2NGIsIDB4YzI0Yjhi NzAsIDB4Yzc2YzUxYTMKKwkud29yZAkJMHhkMTkyZTgxOSwgMHhkNjk5MDYyNCwgMHhmNDBlMzU4 NSwgMHgxMDZhYTA3MAorCS53b3JkCQkweDE5YTRjMTE2LCAweDFlMzc2YzA4LCAweDI3NDg3NzRj LCAweDM0YjBiY2I1CisJLndvcmQJCTB4MzkxYzBjYjMsIDB4NGVkOGFhNGEsIDB4NWI5Y2NhNGYs IDB4NjgyZTZmZjMKKwkud29yZAkJMHg3NDhmODJlZSwgMHg3OGE1NjM2ZiwgMHg4NGM4NzgxNCwg MHg4Y2M3MDIwOAorCS53b3JkCQkweDkwYmVmZmZhLCAweGE0NTA2Y2ViLCAweGJlZjlhM2Y3LCAw eGM2NzE3OGYyCisuc2l6ZSBLMjU2LCAuIC0gSzI1NgotLSAKMi40My4wCgoKX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtcmlzY3YgbWFpbGluZyBs aXN0CmxpbnV4LXJpc2N2QGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVh ZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1yaXNjdgo=