* Re: pegged softirq and NAPI race (?)
From: Rik van Riel @ 2018-09-18 16:31 UTC (permalink / raw)
To: Song Liu, Eric Dumazet
Cc: netdev, Jeff Kirsher, Alexander Duyck, michael.chan@broadcom.com,
Kernel Team
In-Reply-To: <6FA4008E-CEEB-4EAB-BAD8-267D41574248@fb.com>
On Tue, 2018-09-18 at 12:19 -0400, Song Liu wrote:
> > On Sep 18, 2018, at 6:45 AM, Eric Dumazet <edumazet@google.com>
> > wrote:
> >
> > On Tue, Sep 18, 2018 at 1:41 AM Song Liu <songliubraving@fb.com>
> > wrote:
> > >
> > > We are debugging this issue that netconsole message triggers
> > > pegged softirq
> > > (ksoftirqd taking 100% CPU for many seconds). We found this issue
> > > in
> > > production with both bnxt and ixgbe, on a 4.11 based kernel. This
> > > is easily
> > > reproducible with ixgbe on 4.11, and latest net/net-next (see [1]
> > > for more
> > > detail).
> > >
> > > After debugging for some time, we found that this issue is likely
> > > related
> > > to 39e6c8208d7b ("net: solve a NAPI race"). After reverting this
> > > commit,
> > > the steps described in [1] cannot reproduce the issue on ixgbe.
> > > Reverting
> > > this commit also reduces the chances we hit the issue with bnxt
> > > (it still
> > > happens with a lower rate).
> > >
> > > I tried to fix this issue with relaxed variant (or older version)
> > > of
> > > napi_schedule_prep() in netpoll, just like the one on
> > > napi_watchdog().
> > > However, my tests do not always go as expected.
> > >
> > > Please share your comments/suggestions on which direction shall
> > > we try
> > > to fix this.
> > >
> > > Thanks in advance!
> > > Song
> > >
> > >
> > > [1]
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.spinics.net_lists_netdev_msg522328.html&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=iSaOapj1kxjhGYLgQr0Qd8mQCzVdobmgT1L4JwFvzxs&s=lCEhrz6wQJUUaJOkxFmtOszAgkf3Jh4reX_i1GbI5RI&e=
> >
> > You have not traced ixgbe to understand why driver hits
> > "clean_complete=false" all the time ?
>
> The trace showed that we got "clean_complete=false" because
> ixgbe_clean_rx_irq() used all budget (64). It feels like the driver
> is tricked to process old data on the rx_ring for one more time.
>
> Have you seen similar issue?
A quick reading of the code suggests that means
polling cannot keep up with the rate of incoming
packets.
That should not be a surprise, given that polling
appears to happen on just one CPU, while interrupt
driven packet delivery was fanned out across a
larger number of CPUs.
Does the NAPI code have any way in which it
periodically force-returns to IRQ mode, because
multiple CPUs in IRQ mode can keep up with packets
better than a single CPU in polling mode?
Alternatively, is NAPI with multi-queue network
adapters supposed to be polling on multiple CPUs,
but simply failing to do so in this case?
^ permalink raw reply
* [PATCH net-next v5 13/20] zinc: BLAKE2s x86_64 implementation
From: Jason A. Donenfeld @ 2018-09-18 16:16 UTC (permalink / raw)
To: linux-kernel, netdev, linux-crypto, davem, gregkh
Cc: Jason A. Donenfeld, Samuel Neves, Andy Lutomirski,
Jean-Philippe Aumasson, Thomas Gleixner, Ingo Molnar, x86
In-Reply-To: <20180918161646.19105-1-Jason@zx2c4.com>
These implementations from Samuel Neves support AVX and AVX-512VL.
Originally this used AVX-512F, but Skylake thermal throttling made
AVX-512VL more attractive and possible to do with negligable difference.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Samuel Neves <sneves@dei.uc.pt>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: x86@kernel.org
---
lib/zinc/Makefile | 1 +
lib/zinc/blake2s/blake2s-x86_64-glue.h | 59 +++
lib/zinc/blake2s/blake2s-x86_64.S | 685 +++++++++++++++++++++++++
lib/zinc/blake2s/blake2s.c | 4 +-
4 files changed, 748 insertions(+), 1 deletion(-)
create mode 100644 lib/zinc/blake2s/blake2s-x86_64-glue.h
create mode 100644 lib/zinc/blake2s/blake2s-x86_64.S
diff --git a/lib/zinc/Makefile b/lib/zinc/Makefile
index 0e0020f58d27..47607486c39d 100644
--- a/lib/zinc/Makefile
+++ b/lib/zinc/Makefile
@@ -22,4 +22,5 @@ zinc_chacha20poly1305-y := chacha20poly1305.o
obj-$(CONFIG_ZINC_CHACHA20POLY1305) += zinc_chacha20poly1305.o
zinc_blake2s-y := blake2s/blake2s.o
+zinc_blake2s-$(CONFIG_ZINC_ARCH_X86_64) += blake2s/blake2s-x86_64.o
obj-$(CONFIG_ZINC_BLAKE2S) += zinc_blake2s.o
diff --git a/lib/zinc/blake2s/blake2s-x86_64-glue.h b/lib/zinc/blake2s/blake2s-x86_64-glue.h
new file mode 100644
index 000000000000..22900ef8e7fe
--- /dev/null
+++ b/lib/zinc/blake2s/blake2s-x86_64-glue.h
@@ -0,0 +1,59 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ */
+
+#include <asm/cpufeature.h>
+#include <asm/processor.h>
+#include <asm/fpu/api.h>
+#include <asm/simd.h>
+
+#ifdef CONFIG_AS_AVX
+asmlinkage void blake2s_compress_avx(struct blake2s_state *state,
+ const u8 *block, const size_t nblocks,
+ const u32 inc);
+#endif
+#ifdef CONFIG_AS_AVX512
+asmlinkage void blake2s_compress_avx512(struct blake2s_state *state,
+ const u8 *block, const size_t nblocks,
+ const u32 inc);
+#endif
+
+static bool blake2s_use_avx __ro_after_init;
+static bool blake2s_use_avx512 __ro_after_init;
+
+static void __init blake2s_fpu_init(void)
+{
+ blake2s_use_avx =
+ boot_cpu_has(X86_FEATURE_AVX) &&
+ cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL);
+ blake2s_use_avx512 =
+ boot_cpu_has(X86_FEATURE_AVX) &&
+ boot_cpu_has(X86_FEATURE_AVX2) &&
+ boot_cpu_has(X86_FEATURE_AVX512F) &&
+ boot_cpu_has(X86_FEATURE_AVX512VL) &&
+ cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM |
+ XFEATURE_MASK_AVX512, NULL);
+}
+
+static inline bool blake2s_arch(struct blake2s_state *state, const u8 *block,
+ size_t nblocks, const u32 inc)
+{
+#ifdef CONFIG_AS_AVX512
+ if (blake2s_use_avx512 && irq_fpu_usable()) {
+ kernel_fpu_begin();
+ blake2s_compress_avx512(state, block, nblocks, inc);
+ kernel_fpu_end();
+ return true;
+ }
+#endif
+#ifdef CONFIG_AS_AVX
+ if (blake2s_use_avx && irq_fpu_usable()) {
+ kernel_fpu_begin();
+ blake2s_compress_avx(state, block, nblocks, inc);
+ kernel_fpu_end();
+ return true;
+ }
+#endif
+ return false;
+}
diff --git a/lib/zinc/blake2s/blake2s-x86_64.S b/lib/zinc/blake2s/blake2s-x86_64.S
new file mode 100644
index 000000000000..360be4818d06
--- /dev/null
+++ b/lib/zinc/blake2s/blake2s-x86_64.S
@@ -0,0 +1,685 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ * Copyright (C) 2017 Samuel Neves <sneves@dei.uc.pt>. All Rights Reserved.
+ */
+
+#include <linux/linkage.h>
+
+.section .rodata.cst32.BLAKE2S_IV, "aM", @progbits, 32
+.align 32
+IV: .octa 0xA54FF53A3C6EF372BB67AE856A09E667
+ .octa 0x5BE0CD191F83D9AB9B05688C510E527F
+.section .rodata.cst16.ROT16, "aM", @progbits, 16
+.align 16
+ROT16: .octa 0x0D0C0F0E09080B0A0504070601000302
+.section .rodata.cst16.ROR328, "aM", @progbits, 16
+.align 16
+ROR328: .octa 0x0C0F0E0D080B0A090407060500030201
+#ifdef CONFIG_AS_AVX512
+.section .rodata.cst64.BLAKE2S_SIGMA, "aM", @progbits, 640
+.align 64
+SIGMA:
+.long 0, 2, 4, 6, 1, 3, 5, 7, 8, 10, 12, 14, 9, 11, 13, 15
+.long 11, 2, 12, 14, 9, 8, 15, 3, 4, 0, 13, 6, 10, 1, 7, 5
+.long 10, 12, 11, 6, 5, 9, 13, 3, 4, 15, 14, 2, 0, 7, 8, 1
+.long 10, 9, 7, 0, 11, 14, 1, 12, 6, 2, 15, 3, 13, 8, 5, 4
+.long 4, 9, 8, 13, 14, 0, 10, 11, 7, 3, 12, 1, 5, 6, 15, 2
+.long 2, 10, 4, 14, 13, 3, 9, 11, 6, 5, 7, 12, 15, 1, 8, 0
+.long 4, 11, 14, 8, 13, 10, 12, 5, 2, 1, 15, 3, 9, 7, 0, 6
+.long 6, 12, 0, 13, 15, 2, 1, 10, 4, 5, 11, 14, 8, 3, 9, 7
+.long 14, 5, 4, 12, 9, 7, 3, 10, 2, 0, 6, 15, 11, 1, 13, 8
+.long 11, 7, 13, 10, 12, 14, 0, 15, 4, 5, 6, 9, 2, 1, 8, 3
+#endif /* CONFIG_AS_AVX512 */
+
+.text
+#ifdef CONFIG_AS_AVX
+ENTRY(blake2s_compress_avx)
+ movl %ecx, %ecx
+ testq %rdx, %rdx
+ je .Lendofloop
+ .align 32
+.Lbeginofloop:
+ addq %rcx, 32(%rdi)
+ vmovdqu IV+16(%rip), %xmm1
+ vmovdqu (%rsi), %xmm4
+ vpxor 32(%rdi), %xmm1, %xmm1
+ vmovdqu 16(%rsi), %xmm3
+ vshufps $136, %xmm3, %xmm4, %xmm6
+ vmovdqa ROT16(%rip), %xmm7
+ vpaddd (%rdi), %xmm6, %xmm6
+ vpaddd 16(%rdi), %xmm6, %xmm6
+ vpxor %xmm6, %xmm1, %xmm1
+ vmovdqu IV(%rip), %xmm8
+ vpshufb %xmm7, %xmm1, %xmm1
+ vmovdqu 48(%rsi), %xmm5
+ vpaddd %xmm1, %xmm8, %xmm8
+ vpxor 16(%rdi), %xmm8, %xmm9
+ vmovdqu 32(%rsi), %xmm2
+ vpblendw $12, %xmm3, %xmm5, %xmm13
+ vshufps $221, %xmm5, %xmm2, %xmm12
+ vpunpckhqdq %xmm2, %xmm4, %xmm14
+ vpslld $20, %xmm9, %xmm0
+ vpsrld $12, %xmm9, %xmm9
+ vpxor %xmm0, %xmm9, %xmm0
+ vshufps $221, %xmm3, %xmm4, %xmm9
+ vpaddd %xmm9, %xmm6, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vmovdqa ROR328(%rip), %xmm6
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm8, %xmm8
+ vpxor %xmm8, %xmm0, %xmm0
+ vpshufd $147, %xmm1, %xmm1
+ vpshufd $78, %xmm8, %xmm8
+ vpslld $25, %xmm0, %xmm10
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm10, %xmm0, %xmm0
+ vshufps $136, %xmm5, %xmm2, %xmm10
+ vpshufd $57, %xmm0, %xmm0
+ vpaddd %xmm10, %xmm9, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpaddd %xmm12, %xmm9, %xmm9
+ vpblendw $12, %xmm2, %xmm3, %xmm12
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm8, %xmm8
+ vpxor %xmm8, %xmm0, %xmm10
+ vpslld $20, %xmm10, %xmm0
+ vpsrld $12, %xmm10, %xmm10
+ vpxor %xmm0, %xmm10, %xmm0
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm8, %xmm8
+ vpxor %xmm8, %xmm0, %xmm0
+ vpshufd $57, %xmm1, %xmm1
+ vpshufd $78, %xmm8, %xmm8
+ vpslld $25, %xmm0, %xmm10
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm10, %xmm0, %xmm0
+ vpslldq $4, %xmm5, %xmm10
+ vpblendw $240, %xmm10, %xmm12, %xmm12
+ vpshufd $147, %xmm0, %xmm0
+ vpshufd $147, %xmm12, %xmm12
+ vpaddd %xmm9, %xmm12, %xmm12
+ vpaddd %xmm0, %xmm12, %xmm12
+ vpxor %xmm12, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm8, %xmm8
+ vpxor %xmm8, %xmm0, %xmm11
+ vpslld $20, %xmm11, %xmm9
+ vpsrld $12, %xmm11, %xmm11
+ vpxor %xmm9, %xmm11, %xmm0
+ vpshufd $8, %xmm2, %xmm9
+ vpblendw $192, %xmm5, %xmm3, %xmm11
+ vpblendw $240, %xmm11, %xmm9, %xmm9
+ vpshufd $177, %xmm9, %xmm9
+ vpaddd %xmm12, %xmm9, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm11
+ vpxor %xmm11, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm8, %xmm8
+ vpxor %xmm8, %xmm0, %xmm9
+ vpshufd $147, %xmm1, %xmm1
+ vpshufd $78, %xmm8, %xmm8
+ vpslld $25, %xmm9, %xmm0
+ vpsrld $7, %xmm9, %xmm9
+ vpxor %xmm0, %xmm9, %xmm0
+ vpslldq $4, %xmm3, %xmm9
+ vpblendw $48, %xmm9, %xmm2, %xmm9
+ vpblendw $240, %xmm9, %xmm4, %xmm9
+ vpshufd $57, %xmm0, %xmm0
+ vpshufd $177, %xmm9, %xmm9
+ vpaddd %xmm11, %xmm9, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm8, %xmm11
+ vpxor %xmm11, %xmm0, %xmm0
+ vpslld $20, %xmm0, %xmm8
+ vpsrld $12, %xmm0, %xmm0
+ vpxor %xmm8, %xmm0, %xmm0
+ vpunpckhdq %xmm3, %xmm4, %xmm8
+ vpblendw $12, %xmm10, %xmm8, %xmm12
+ vpshufd $177, %xmm12, %xmm12
+ vpaddd %xmm9, %xmm12, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm11
+ vpxor %xmm11, %xmm0, %xmm0
+ vpshufd $57, %xmm1, %xmm1
+ vpshufd $78, %xmm11, %xmm11
+ vpslld $25, %xmm0, %xmm12
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm12, %xmm0, %xmm0
+ vpunpckhdq %xmm5, %xmm2, %xmm12
+ vpshufd $147, %xmm0, %xmm0
+ vpblendw $15, %xmm13, %xmm12, %xmm12
+ vpslldq $8, %xmm5, %xmm13
+ vpshufd $210, %xmm12, %xmm12
+ vpaddd %xmm9, %xmm12, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm11
+ vpxor %xmm11, %xmm0, %xmm0
+ vpslld $20, %xmm0, %xmm12
+ vpsrld $12, %xmm0, %xmm0
+ vpxor %xmm12, %xmm0, %xmm0
+ vpunpckldq %xmm4, %xmm2, %xmm12
+ vpblendw $240, %xmm4, %xmm12, %xmm12
+ vpblendw $192, %xmm13, %xmm12, %xmm12
+ vpsrldq $12, %xmm3, %xmm13
+ vpaddd %xmm12, %xmm9, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm11
+ vpxor %xmm11, %xmm0, %xmm0
+ vpshufd $147, %xmm1, %xmm1
+ vpshufd $78, %xmm11, %xmm11
+ vpslld $25, %xmm0, %xmm12
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm12, %xmm0, %xmm0
+ vpblendw $60, %xmm2, %xmm4, %xmm12
+ vpblendw $3, %xmm13, %xmm12, %xmm12
+ vpshufd $57, %xmm0, %xmm0
+ vpshufd $78, %xmm12, %xmm12
+ vpaddd %xmm9, %xmm12, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm11
+ vpxor %xmm11, %xmm0, %xmm12
+ vpslld $20, %xmm12, %xmm13
+ vpsrld $12, %xmm12, %xmm0
+ vpblendw $51, %xmm3, %xmm4, %xmm12
+ vpxor %xmm13, %xmm0, %xmm0
+ vpblendw $192, %xmm10, %xmm12, %xmm10
+ vpslldq $8, %xmm2, %xmm12
+ vpshufd $27, %xmm10, %xmm10
+ vpaddd %xmm9, %xmm10, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm11
+ vpxor %xmm11, %xmm0, %xmm0
+ vpshufd $57, %xmm1, %xmm1
+ vpshufd $78, %xmm11, %xmm11
+ vpslld $25, %xmm0, %xmm10
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm10, %xmm0, %xmm0
+ vpunpckhdq %xmm2, %xmm8, %xmm10
+ vpshufd $147, %xmm0, %xmm0
+ vpblendw $12, %xmm5, %xmm10, %xmm10
+ vpshufd $210, %xmm10, %xmm10
+ vpaddd %xmm9, %xmm10, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm11
+ vpxor %xmm11, %xmm0, %xmm10
+ vpslld $20, %xmm10, %xmm0
+ vpsrld $12, %xmm10, %xmm10
+ vpxor %xmm0, %xmm10, %xmm0
+ vpblendw $12, %xmm4, %xmm5, %xmm10
+ vpblendw $192, %xmm12, %xmm10, %xmm10
+ vpunpckldq %xmm2, %xmm4, %xmm12
+ vpshufd $135, %xmm10, %xmm10
+ vpaddd %xmm9, %xmm10, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm13
+ vpxor %xmm13, %xmm0, %xmm0
+ vpshufd $147, %xmm1, %xmm1
+ vpshufd $78, %xmm13, %xmm13
+ vpslld $25, %xmm0, %xmm10
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm10, %xmm0, %xmm0
+ vpblendw $15, %xmm3, %xmm4, %xmm10
+ vpblendw $192, %xmm5, %xmm10, %xmm10
+ vpshufd $57, %xmm0, %xmm0
+ vpshufd $198, %xmm10, %xmm10
+ vpaddd %xmm9, %xmm10, %xmm10
+ vpaddd %xmm0, %xmm10, %xmm10
+ vpxor %xmm10, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm13, %xmm13
+ vpxor %xmm13, %xmm0, %xmm9
+ vpslld $20, %xmm9, %xmm0
+ vpsrld $12, %xmm9, %xmm9
+ vpxor %xmm0, %xmm9, %xmm0
+ vpunpckhdq %xmm2, %xmm3, %xmm9
+ vpunpcklqdq %xmm12, %xmm9, %xmm15
+ vpunpcklqdq %xmm12, %xmm8, %xmm12
+ vpblendw $15, %xmm5, %xmm8, %xmm8
+ vpaddd %xmm15, %xmm10, %xmm15
+ vpaddd %xmm0, %xmm15, %xmm15
+ vpxor %xmm15, %xmm1, %xmm1
+ vpshufd $141, %xmm8, %xmm8
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm13, %xmm13
+ vpxor %xmm13, %xmm0, %xmm0
+ vpshufd $57, %xmm1, %xmm1
+ vpshufd $78, %xmm13, %xmm13
+ vpslld $25, %xmm0, %xmm10
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm10, %xmm0, %xmm0
+ vpunpcklqdq %xmm2, %xmm3, %xmm10
+ vpshufd $147, %xmm0, %xmm0
+ vpblendw $51, %xmm14, %xmm10, %xmm14
+ vpshufd $135, %xmm14, %xmm14
+ vpaddd %xmm15, %xmm14, %xmm14
+ vpaddd %xmm0, %xmm14, %xmm14
+ vpxor %xmm14, %xmm1, %xmm1
+ vpunpcklqdq %xmm3, %xmm4, %xmm15
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm13, %xmm13
+ vpxor %xmm13, %xmm0, %xmm0
+ vpslld $20, %xmm0, %xmm11
+ vpsrld $12, %xmm0, %xmm0
+ vpxor %xmm11, %xmm0, %xmm0
+ vpunpckhqdq %xmm5, %xmm3, %xmm11
+ vpblendw $51, %xmm15, %xmm11, %xmm11
+ vpunpckhqdq %xmm3, %xmm5, %xmm15
+ vpaddd %xmm11, %xmm14, %xmm11
+ vpaddd %xmm0, %xmm11, %xmm11
+ vpxor %xmm11, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm13, %xmm13
+ vpxor %xmm13, %xmm0, %xmm0
+ vpshufd $147, %xmm1, %xmm1
+ vpshufd $78, %xmm13, %xmm13
+ vpslld $25, %xmm0, %xmm14
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm14, %xmm0, %xmm14
+ vpunpckhqdq %xmm4, %xmm2, %xmm0
+ vpshufd $57, %xmm14, %xmm14
+ vpblendw $51, %xmm15, %xmm0, %xmm15
+ vpaddd %xmm15, %xmm11, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm13, %xmm13
+ vpxor %xmm13, %xmm14, %xmm14
+ vpslld $20, %xmm14, %xmm11
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm11, %xmm14, %xmm14
+ vpblendw $3, %xmm2, %xmm4, %xmm11
+ vpslldq $8, %xmm11, %xmm0
+ vpblendw $15, %xmm5, %xmm0, %xmm0
+ vpshufd $99, %xmm0, %xmm0
+ vpaddd %xmm15, %xmm0, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm1, %xmm0
+ vpaddd %xmm12, %xmm15, %xmm15
+ vpshufb %xmm6, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm13, %xmm13
+ vpxor %xmm13, %xmm14, %xmm14
+ vpshufd $57, %xmm0, %xmm0
+ vpshufd $78, %xmm13, %xmm13
+ vpslld $25, %xmm14, %xmm1
+ vpsrld $7, %xmm14, %xmm14
+ vpxor %xmm1, %xmm14, %xmm14
+ vpblendw $3, %xmm5, %xmm4, %xmm1
+ vpshufd $147, %xmm14, %xmm14
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpshufb %xmm7, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm13, %xmm13
+ vpxor %xmm13, %xmm14, %xmm14
+ vpslld $20, %xmm14, %xmm12
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm12, %xmm14, %xmm14
+ vpsrldq $4, %xmm2, %xmm12
+ vpblendw $60, %xmm12, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm15, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpblendw $12, %xmm4, %xmm3, %xmm1
+ vpshufb %xmm6, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm13, %xmm13
+ vpxor %xmm13, %xmm14, %xmm14
+ vpshufd $147, %xmm0, %xmm0
+ vpshufd $78, %xmm13, %xmm13
+ vpslld $25, %xmm14, %xmm12
+ vpsrld $7, %xmm14, %xmm14
+ vpxor %xmm12, %xmm14, %xmm14
+ vpsrldq $4, %xmm5, %xmm12
+ vpblendw $48, %xmm12, %xmm1, %xmm1
+ vpshufd $33, %xmm5, %xmm12
+ vpshufd $57, %xmm14, %xmm14
+ vpshufd $108, %xmm1, %xmm1
+ vpblendw $51, %xmm12, %xmm10, %xmm12
+ vpaddd %xmm15, %xmm1, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpaddd %xmm12, %xmm15, %xmm15
+ vpshufb %xmm7, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm13, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpslld $20, %xmm14, %xmm13
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm13, %xmm14, %xmm14
+ vpslldq $12, %xmm3, %xmm13
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpshufb %xmm6, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpshufd $57, %xmm0, %xmm0
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm14, %xmm12
+ vpsrld $7, %xmm14, %xmm14
+ vpxor %xmm12, %xmm14, %xmm14
+ vpblendw $51, %xmm5, %xmm4, %xmm12
+ vpshufd $147, %xmm14, %xmm14
+ vpblendw $192, %xmm13, %xmm12, %xmm12
+ vpaddd %xmm12, %xmm15, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpsrldq $4, %xmm3, %xmm12
+ vpshufb %xmm7, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpslld $20, %xmm14, %xmm13
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm13, %xmm14, %xmm14
+ vpblendw $48, %xmm2, %xmm5, %xmm13
+ vpblendw $3, %xmm12, %xmm13, %xmm13
+ vpshufd $156, %xmm13, %xmm13
+ vpaddd %xmm15, %xmm13, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpshufb %xmm6, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpshufd $147, %xmm0, %xmm0
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm14, %xmm13
+ vpsrld $7, %xmm14, %xmm14
+ vpxor %xmm13, %xmm14, %xmm14
+ vpunpcklqdq %xmm2, %xmm4, %xmm13
+ vpshufd $57, %xmm14, %xmm14
+ vpblendw $12, %xmm12, %xmm13, %xmm12
+ vpshufd $180, %xmm12, %xmm12
+ vpaddd %xmm15, %xmm12, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpshufb %xmm7, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpslld $20, %xmm14, %xmm12
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm12, %xmm14, %xmm14
+ vpunpckhqdq %xmm9, %xmm4, %xmm12
+ vpshufd $198, %xmm12, %xmm12
+ vpaddd %xmm15, %xmm12, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpaddd %xmm15, %xmm8, %xmm15
+ vpshufb %xmm6, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpshufd $57, %xmm0, %xmm0
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm14, %xmm12
+ vpsrld $7, %xmm14, %xmm14
+ vpxor %xmm12, %xmm14, %xmm14
+ vpsrldq $4, %xmm4, %xmm12
+ vpshufd $147, %xmm14, %xmm14
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpshufb %xmm7, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpslld $20, %xmm14, %xmm8
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm14, %xmm8, %xmm14
+ vpblendw $48, %xmm5, %xmm2, %xmm8
+ vpblendw $3, %xmm12, %xmm8, %xmm8
+ vpunpckhqdq %xmm5, %xmm4, %xmm12
+ vpshufd $75, %xmm8, %xmm8
+ vpblendw $60, %xmm10, %xmm12, %xmm10
+ vpaddd %xmm15, %xmm8, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm0, %xmm15, %xmm0
+ vpshufd $45, %xmm10, %xmm10
+ vpshufb %xmm6, %xmm0, %xmm0
+ vpaddd %xmm15, %xmm10, %xmm15
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpshufd $147, %xmm0, %xmm0
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm14, %xmm8
+ vpsrld $7, %xmm14, %xmm14
+ vpxor %xmm14, %xmm8, %xmm8
+ vpshufd $57, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm15, %xmm15
+ vpxor %xmm0, %xmm15, %xmm0
+ vpshufb %xmm7, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm8, %xmm1, %xmm8
+ vpslld $20, %xmm8, %xmm10
+ vpsrld $12, %xmm8, %xmm8
+ vpxor %xmm8, %xmm10, %xmm10
+ vpunpckldq %xmm3, %xmm4, %xmm8
+ vpunpcklqdq %xmm9, %xmm8, %xmm9
+ vpaddd %xmm9, %xmm15, %xmm9
+ vpaddd %xmm10, %xmm9, %xmm9
+ vpxor %xmm0, %xmm9, %xmm8
+ vpshufb %xmm6, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm1, %xmm1
+ vpxor %xmm1, %xmm10, %xmm10
+ vpshufd $57, %xmm8, %xmm8
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm10, %xmm12
+ vpsrld $7, %xmm10, %xmm10
+ vpxor %xmm10, %xmm12, %xmm10
+ vpblendw $48, %xmm4, %xmm3, %xmm12
+ vpshufd $147, %xmm10, %xmm0
+ vpunpckhdq %xmm5, %xmm3, %xmm10
+ vpshufd $78, %xmm12, %xmm12
+ vpunpcklqdq %xmm4, %xmm10, %xmm10
+ vpblendw $192, %xmm2, %xmm10, %xmm10
+ vpshufhw $78, %xmm10, %xmm10
+ vpaddd %xmm10, %xmm9, %xmm10
+ vpaddd %xmm0, %xmm10, %xmm10
+ vpxor %xmm8, %xmm10, %xmm8
+ vpshufb %xmm7, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm1, %xmm1
+ vpxor %xmm0, %xmm1, %xmm9
+ vpslld $20, %xmm9, %xmm0
+ vpsrld $12, %xmm9, %xmm9
+ vpxor %xmm9, %xmm0, %xmm0
+ vpunpckhdq %xmm5, %xmm4, %xmm9
+ vpblendw $240, %xmm9, %xmm2, %xmm13
+ vpshufd $39, %xmm13, %xmm13
+ vpaddd %xmm10, %xmm13, %xmm10
+ vpaddd %xmm0, %xmm10, %xmm10
+ vpxor %xmm8, %xmm10, %xmm8
+ vpblendw $12, %xmm4, %xmm2, %xmm13
+ vpshufb %xmm6, %xmm8, %xmm8
+ vpslldq $4, %xmm13, %xmm13
+ vpblendw $15, %xmm5, %xmm13, %xmm13
+ vpaddd %xmm8, %xmm1, %xmm1
+ vpxor %xmm1, %xmm0, %xmm0
+ vpaddd %xmm13, %xmm10, %xmm13
+ vpshufd $147, %xmm8, %xmm8
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm0, %xmm14
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm0, %xmm14, %xmm14
+ vpshufd $57, %xmm14, %xmm14
+ vpaddd %xmm14, %xmm13, %xmm13
+ vpxor %xmm8, %xmm13, %xmm8
+ vpaddd %xmm13, %xmm12, %xmm12
+ vpshufb %xmm7, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm1, %xmm1
+ vpxor %xmm14, %xmm1, %xmm14
+ vpslld $20, %xmm14, %xmm10
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm14, %xmm10, %xmm10
+ vpaddd %xmm10, %xmm12, %xmm12
+ vpxor %xmm8, %xmm12, %xmm8
+ vpshufb %xmm6, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm1, %xmm1
+ vpxor %xmm1, %xmm10, %xmm0
+ vpshufd $57, %xmm8, %xmm8
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm0, %xmm10
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm0, %xmm10, %xmm10
+ vpblendw $48, %xmm2, %xmm3, %xmm0
+ vpblendw $15, %xmm11, %xmm0, %xmm0
+ vpshufd $147, %xmm10, %xmm10
+ vpshufd $114, %xmm0, %xmm0
+ vpaddd %xmm12, %xmm0, %xmm0
+ vpaddd %xmm10, %xmm0, %xmm0
+ vpxor %xmm8, %xmm0, %xmm8
+ vpshufb %xmm7, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm1, %xmm1
+ vpxor %xmm10, %xmm1, %xmm10
+ vpslld $20, %xmm10, %xmm11
+ vpsrld $12, %xmm10, %xmm10
+ vpxor %xmm10, %xmm11, %xmm10
+ vpslldq $4, %xmm4, %xmm11
+ vpblendw $192, %xmm11, %xmm3, %xmm3
+ vpunpckldq %xmm5, %xmm4, %xmm4
+ vpshufd $99, %xmm3, %xmm3
+ vpaddd %xmm0, %xmm3, %xmm3
+ vpaddd %xmm10, %xmm3, %xmm3
+ vpxor %xmm8, %xmm3, %xmm11
+ vpunpckldq %xmm5, %xmm2, %xmm0
+ vpblendw $192, %xmm2, %xmm5, %xmm2
+ vpshufb %xmm6, %xmm11, %xmm11
+ vpunpckhqdq %xmm0, %xmm9, %xmm0
+ vpblendw $15, %xmm4, %xmm2, %xmm4
+ vpaddd %xmm11, %xmm1, %xmm1
+ vpxor %xmm1, %xmm10, %xmm10
+ vpshufd $147, %xmm11, %xmm11
+ vpshufd $201, %xmm0, %xmm0
+ vpslld $25, %xmm10, %xmm8
+ vpsrld $7, %xmm10, %xmm10
+ vpxor %xmm10, %xmm8, %xmm10
+ vpshufd $78, %xmm1, %xmm1
+ vpaddd %xmm3, %xmm0, %xmm0
+ vpshufd $27, %xmm4, %xmm4
+ vpshufd $57, %xmm10, %xmm10
+ vpaddd %xmm10, %xmm0, %xmm0
+ vpxor %xmm11, %xmm0, %xmm11
+ vpaddd %xmm0, %xmm4, %xmm0
+ vpshufb %xmm7, %xmm11, %xmm7
+ vpaddd %xmm7, %xmm1, %xmm1
+ vpxor %xmm10, %xmm1, %xmm10
+ vpslld $20, %xmm10, %xmm8
+ vpsrld $12, %xmm10, %xmm10
+ vpxor %xmm10, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm0, %xmm0
+ vpxor %xmm7, %xmm0, %xmm7
+ vpshufb %xmm6, %xmm7, %xmm6
+ vpaddd %xmm6, %xmm1, %xmm1
+ vpxor %xmm1, %xmm8, %xmm8
+ vpshufd $78, %xmm1, %xmm1
+ vpshufd $57, %xmm6, %xmm6
+ vpslld $25, %xmm8, %xmm2
+ vpsrld $7, %xmm8, %xmm8
+ vpxor %xmm8, %xmm2, %xmm8
+ vpxor (%rdi), %xmm1, %xmm1
+ vpshufd $147, %xmm8, %xmm8
+ vpxor %xmm0, %xmm1, %xmm0
+ vmovups %xmm0, (%rdi)
+ vpxor 16(%rdi), %xmm8, %xmm0
+ vpxor %xmm6, %xmm0, %xmm6
+ vmovups %xmm6, 16(%rdi)
+ addq $64, %rsi
+ decq %rdx
+ jnz .Lbeginofloop
+.Lendofloop:
+ ret
+ENDPROC(blake2s_compress_avx)
+#endif /* CONFIG_AS_AVX */
+
+#ifdef CONFIG_AS_AVX512
+ENTRY(blake2s_compress_avx512)
+ vmovdqu (%rdi),%xmm0
+ vmovdqu 0x10(%rdi),%xmm1
+ vmovdqu 0x20(%rdi),%xmm4
+ vmovq %rcx,%xmm5
+ vmovdqa IV(%rip),%xmm14
+ vmovdqa IV+16(%rip),%xmm15
+ jmp .Lblake2s_compress_avx512_mainloop
+.align 32
+.Lblake2s_compress_avx512_mainloop:
+ vmovdqa %xmm0,%xmm10
+ vmovdqa %xmm1,%xmm11
+ vpaddq %xmm5,%xmm4,%xmm4
+ vmovdqa %xmm14,%xmm2
+ vpxor %xmm15,%xmm4,%xmm3
+ vmovdqu (%rsi),%ymm6
+ vmovdqu 0x20(%rsi),%ymm7
+ addq $0x40,%rsi
+ leaq SIGMA(%rip),%rax
+ movb $0xa,%cl
+.Lblake2s_compress_avx512_roundloop:
+ addq $0x40,%rax
+ vmovdqa -0x40(%rax),%ymm8
+ vmovdqa -0x20(%rax),%ymm9
+ vpermi2d %ymm7,%ymm6,%ymm8
+ vpermi2d %ymm7,%ymm6,%ymm9
+ vmovdqa %ymm8,%ymm6
+ vmovdqa %ymm9,%ymm7
+ vpaddd %xmm8,%xmm0,%xmm0
+ vpaddd %xmm1,%xmm0,%xmm0
+ vpxor %xmm0,%xmm3,%xmm3
+ vprord $0x10,%xmm3,%xmm3
+ vpaddd %xmm3,%xmm2,%xmm2
+ vpxor %xmm2,%xmm1,%xmm1
+ vprord $0xc,%xmm1,%xmm1
+ vextracti128 $0x1,%ymm8,%xmm8
+ vpaddd %xmm8,%xmm0,%xmm0
+ vpaddd %xmm1,%xmm0,%xmm0
+ vpxor %xmm0,%xmm3,%xmm3
+ vprord $0x8,%xmm3,%xmm3
+ vpaddd %xmm3,%xmm2,%xmm2
+ vpxor %xmm2,%xmm1,%xmm1
+ vprord $0x7,%xmm1,%xmm1
+ vpshufd $0x39,%xmm1,%xmm1
+ vpshufd $0x4e,%xmm2,%xmm2
+ vpshufd $0x93,%xmm3,%xmm3
+ vpaddd %xmm9,%xmm0,%xmm0
+ vpaddd %xmm1,%xmm0,%xmm0
+ vpxor %xmm0,%xmm3,%xmm3
+ vprord $0x10,%xmm3,%xmm3
+ vpaddd %xmm3,%xmm2,%xmm2
+ vpxor %xmm2,%xmm1,%xmm1
+ vprord $0xc,%xmm1,%xmm1
+ vextracti128 $0x1,%ymm9,%xmm9
+ vpaddd %xmm9,%xmm0,%xmm0
+ vpaddd %xmm1,%xmm0,%xmm0
+ vpxor %xmm0,%xmm3,%xmm3
+ vprord $0x8,%xmm3,%xmm3
+ vpaddd %xmm3,%xmm2,%xmm2
+ vpxor %xmm2,%xmm1,%xmm1
+ vprord $0x7,%xmm1,%xmm1
+ vpshufd $0x93,%xmm1,%xmm1
+ vpshufd $0x4e,%xmm2,%xmm2
+ vpshufd $0x39,%xmm3,%xmm3
+ decb %cl
+ jne .Lblake2s_compress_avx512_roundloop
+ vpxor %xmm10,%xmm0,%xmm0
+ vpxor %xmm11,%xmm1,%xmm1
+ vpxor %xmm2,%xmm0,%xmm0
+ vpxor %xmm3,%xmm1,%xmm1
+ decq %rdx
+ jne .Lblake2s_compress_avx512_mainloop
+ vmovdqu %xmm0,(%rdi)
+ vmovdqu %xmm1,0x10(%rdi)
+ vmovdqu %xmm4,0x20(%rdi)
+ vzeroupper
+ retq
+ENDPROC(blake2s_compress_avx512)
+#endif /* CONFIG_AS_AVX512 */
diff --git a/lib/zinc/blake2s/blake2s.c b/lib/zinc/blake2s/blake2s.c
index 0b62313267fb..c54f017fccfe 100644
--- a/lib/zinc/blake2s/blake2s.c
+++ b/lib/zinc/blake2s/blake2s.c
@@ -113,7 +113,9 @@ void blake2s_init_key(struct blake2s_state *state, const size_t outlen,
}
EXPORT_SYMBOL(blake2s_init_key);
-#ifndef HAVE_BLAKE2S_ARCH_IMPLEMENTATION
+#if defined(CONFIG_ZINC_ARCH_X86_64)
+#include "blake2s-x86_64-glue.h"
+#else
static void __init blake2s_fpu_init(void)
{
}
--
2.19.0
^ permalink raw reply related
* Re: pegged softirq and NAPI race (?)
From: Song Liu @ 2018-09-18 16:19 UTC (permalink / raw)
To: Eric Dumazet
Cc: netdev, Jeff Kirsher, Alexander Duyck, michael.chan@broadcom.com,
Kernel Team
In-Reply-To: <CANn89iLAPvPn2C9J+sSHsQ5EeF0hq47iMiiAfwoQkdZurQiyzA@mail.gmail.com>
> On Sep 18, 2018, at 6:45 AM, Eric Dumazet <edumazet@google.com> wrote:
>
> On Tue, Sep 18, 2018 at 1:41 AM Song Liu <songliubraving@fb.com> wrote:
>>
>> We are debugging this issue that netconsole message triggers pegged softirq
>> (ksoftirqd taking 100% CPU for many seconds). We found this issue in
>> production with both bnxt and ixgbe, on a 4.11 based kernel. This is easily
>> reproducible with ixgbe on 4.11, and latest net/net-next (see [1] for more
>> detail).
>>
>> After debugging for some time, we found that this issue is likely related
>> to 39e6c8208d7b ("net: solve a NAPI race"). After reverting this commit,
>> the steps described in [1] cannot reproduce the issue on ixgbe. Reverting
>> this commit also reduces the chances we hit the issue with bnxt (it still
>> happens with a lower rate).
>>
>> I tried to fix this issue with relaxed variant (or older version) of
>> napi_schedule_prep() in netpoll, just like the one on napi_watchdog().
>> However, my tests do not always go as expected.
>>
>> Please share your comments/suggestions on which direction shall we try
>> to fix this.
>>
>> Thanks in advance!
>> Song
>>
>>
>> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__www.spinics.net_lists_netdev_msg522328.html&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=iSaOapj1kxjhGYLgQr0Qd8mQCzVdobmgT1L4JwFvzxs&s=lCEhrz6wQJUUaJOkxFmtOszAgkf3Jh4reX_i1GbI5RI&e=
>
> You have not traced ixgbe to understand why driver hits
> "clean_complete=false" all the time ?
The trace showed that we got "clean_complete=false" because
ixgbe_clean_rx_irq() used all budget (64). It feels like the driver
is tricked to process old data on the rx_ring for one more time.
Have you seen similar issue?
Thanks,
Song
^ permalink raw reply
* [PATCH net-next v5 06/20] zinc: ChaCha20 MIPS32r2 implementation
From: Jason A. Donenfeld @ 2018-09-18 16:16 UTC (permalink / raw)
To: linux-kernel, netdev, linux-crypto, davem, gregkh
Cc: Jason A. Donenfeld, René van Dorst, Samuel Neves,
Andy Lutomirski, Jean-Philippe Aumasson, Ralf Baechle,
Paul Burton, James Hogan, linux-mips
In-Reply-To: <20180918161646.19105-1-Jason@zx2c4.com>
This MIPS32r2 implementation comes from René van Dorst and me and
results in a nice speedup on the usual OpenWRT targets.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: René van Dorst <opensource@vdorst.com>
Cc: Samuel Neves <sneves@dei.uc.pt>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Cc: James Hogan <jhogan@kernel.org>
Cc: linux-mips@linux-mips.org
---
lib/zinc/Makefile | 1 +
lib/zinc/chacha20/chacha20-mips-glue.h | 24 ++
lib/zinc/chacha20/chacha20-mips.S | 474 +++++++++++++++++++++++++
lib/zinc/chacha20/chacha20.c | 2 +
4 files changed, 501 insertions(+)
create mode 100644 lib/zinc/chacha20/chacha20-mips-glue.h
create mode 100644 lib/zinc/chacha20/chacha20-mips.S
diff --git a/lib/zinc/Makefile b/lib/zinc/Makefile
index 36b61145666d..5c7b452d6a19 100644
--- a/lib/zinc/Makefile
+++ b/lib/zinc/Makefile
@@ -7,4 +7,5 @@ zinc_chacha20-y := chacha20/chacha20.o
zinc_chacha20-$(CONFIG_ZINC_ARCH_X86_64) += chacha20/chacha20-x86_64.o
zinc_chacha20-$(CONFIG_ZINC_ARCH_ARM) += chacha20/chacha20-arm.o
zinc_chacha20-$(CONFIG_ZINC_ARCH_ARM64) += chacha20/chacha20-arm64.o
+zinc_chacha20-$(CONFIG_ZINC_ARCH_MIPS) += chacha20/chacha20-mips.o
obj-$(CONFIG_ZINC_CHACHA20) += zinc_chacha20.o
diff --git a/lib/zinc/chacha20/chacha20-mips-glue.h b/lib/zinc/chacha20/chacha20-mips-glue.h
new file mode 100644
index 000000000000..518deb5e6c34
--- /dev/null
+++ b/lib/zinc/chacha20/chacha20-mips-glue.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ */
+
+asmlinkage void chacha20_mips(u8 *out, const u8 *in, const size_t len,
+ const u32 key[8], const u32 counter[4]);
+static void __init chacha20_fpu_init(void)
+{
+}
+
+static inline bool chacha20_arch(u8 *dst, const u8 *src, const size_t len,
+ const u32 key[8], const u32 counter[4],
+ simd_context_t *simd_context)
+{
+ chacha20_mips(dst, src, len, key, counter);
+ return true;
+}
+
+static inline bool hchacha20_arch(u8 *derived_key, const u8 *nonce,
+ const u8 *key, simd_context_t *simd_context)
+{
+ return false;
+}
diff --git a/lib/zinc/chacha20/chacha20-mips.S b/lib/zinc/chacha20/chacha20-mips.S
new file mode 100644
index 000000000000..1415a77fd0c0
--- /dev/null
+++ b/lib/zinc/chacha20/chacha20-mips.S
@@ -0,0 +1,474 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2016-2018 René van Dorst <opensource@vdorst.com>. All Rights Reserved.
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ */
+
+#define MASK_U32 0x3c
+#define MASK_BYTES 0x03
+#define CHACHA20_BLOCK_SIZE 64
+#define STACK_SIZE 4*16
+
+#define X0 $t0
+#define X1 $t1
+#define X2 $t2
+#define X3 $t3
+#define X4 $t4
+#define X5 $t5
+#define X6 $t6
+#define X7 $t7
+#define X8 $v1
+#define X9 $fp
+#define X10 $s7
+#define X11 $s6
+#define X12 $s5
+#define X13 $s4
+#define X14 $s3
+#define X15 $s2
+/* Use regs which are overwritten on exit for Tx so we don't leak clear data. */
+#define T0 $s1
+#define T1 $s0
+#define T(n) T ## n
+#define X(n) X ## n
+
+/* Input arguments */
+#define OUT $a0
+#define IN $a1
+#define BYTES $a2
+/* KEY and NONCE argument must be u32 aligned */
+#define KEY $a3
+/* NONCE pointer is given via stack */
+#define NONCE $t9
+
+/* Output argument */
+/* NONCE[0] is kept in a register and not in memory.
+ * We don't want to touch original value in memory.
+ * Must be incremented every loop iteration.
+ */
+#define NONCE_0 $v0
+
+/* SAVED_X and SAVED_CA are set in the jump table.
+ * Use regs which are overwritten on exit else we don't leak clear data.
+ * They are used to handling the last bytes which are not multiple of 4.
+ */
+#define SAVED_X X15
+#define SAVED_CA $ra
+
+#define PTR_LAST_ROUND $t8
+
+/* ChaCha20 constants and stack location */
+#define CONSTANT_OFS_SP 48
+#define UNALIGNED_OFS_SP 40
+
+#define CONSTANT_1 0x61707865
+#define CONSTANT_2 0x3320646e
+#define CONSTANT_3 0x79622d32
+#define CONSTANT_4 0x6b206574
+
+#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+#define MSB 0
+#define LSB 3
+#define ROTx rotl
+#define ROTR(n) rotr n, 24
+#define CPU_TO_LE32(n) \
+ wsbh n; \
+ rotr n, 16;
+#else
+#define MSB 3
+#define LSB 0
+#define ROTx rotr
+#define CPU_TO_LE32(n)
+#define ROTR(n)
+#endif
+
+#define STORE_UNALIGNED(x, a, s, o) \
+.Lchacha20_mips_xor_unaligned_ ## x ## _b: ; \
+ .if ((s != NONCE) || (o != 0)); \
+ lw T0, o(s); \
+ .endif; \
+ lwl T1, x-4+MSB ## (IN); \
+ lwr T1, x-4+LSB ## (IN); \
+ .if ((s == NONCE) && (o == 0)); \
+ addu X ## a, NONCE_0; \
+ .else; \
+ addu X ## a, T0; \
+ .endif; \
+ CPU_TO_LE32(X ## a); \
+ xor X ## a, T1; \
+ swl X ## a, x-4+MSB ## (OUT); \
+ swr X ## a, x-4+LSB ## (OUT);
+
+#define STORE_ALIGNED(x, a, s, o) \
+.Lchacha20_mips_xor_aligned_ ## x ## _b: ; \
+ .if ((s != NONCE) || (o != 0)); \
+ lw T0, o(s); \
+ .endif; \
+ lw T1, x-4 ## (IN); \
+ .if ((s == NONCE) && (o == 0)); \
+ addu X ## a, NONCE_0; \
+ .else; \
+ addu X ## a, T0; \
+ .endif; \
+ CPU_TO_LE32(X ## a); \
+ xor X ## a, T1; \
+ sw X ## a, x-4 ## (OUT);
+
+/* Jump table macro.
+ * Used for setup and handling the last bytes, which are not multiple of 4.
+ * X15 is free to store Xn
+ * Every jumptable entry must be equal in size.
+ */
+#define JMPTBL_ALIGNED(x, a, s, o) \
+.Lchacha20_mips_jmptbl_aligned_ ## a: ; \
+ .if ((s == NONCE) && (o == 0)); \
+ move SAVED_CA, NONCE_0; \
+ .else; \
+ lw SAVED_CA, o(s);\
+ .endif; \
+ b .Lchacha20_mips_xor_aligned_ ## x ## _b; \
+ move SAVED_X, X ## a;
+
+#define JMPTBL_UNALIGNED(x, a, s, o) \
+.Lchacha20_mips_jmptbl_unaligned_ ## a: ; \
+ .if ((s == NONCE) && (o == 0)); \
+ move SAVED_CA, NONCE_0; \
+ .else; \
+ lw SAVED_CA, o(s);\
+ .endif; \
+ b .Lchacha20_mips_xor_unaligned_ ## x ## _b; \
+ move SAVED_X, X ## a;
+
+#define AXR(A, B, C, D, K, L, M, N, V, W, Y, Z, S) \
+ addu X(A), X(K); \
+ addu X(B), X(L); \
+ addu X(C), X(M); \
+ addu X(D), X(N); \
+ xor X(V), X(A); \
+ xor X(W), X(B); \
+ xor X(Y), X(C); \
+ xor X(Z), X(D); \
+ rotl X(V), S; \
+ rotl X(W), S; \
+ rotl X(Y), S; \
+ rotl X(Z), S;
+
+.text
+.set reorder
+.set noat
+.globl chacha20_mips
+.ent chacha20_mips
+chacha20_mips:
+ .frame $sp, STACK_SIZE, $ra
+ /* This is in the fifth argument */
+ lw NONCE, 16($sp)
+
+ /* Return bytes = 0. */
+ .set noreorder
+ beqz BYTES, .Lchacha20_mips_end
+ addiu $sp, -STACK_SIZE
+ .set reorder
+
+ /* Calculate PTR_LAST_ROUND */
+ addiu PTR_LAST_ROUND, BYTES, -1
+ ins PTR_LAST_ROUND, $zero, 0, 6
+ addu PTR_LAST_ROUND, OUT
+
+ /* Save s0-s7, fp, ra. */
+ sw $ra, 0($sp)
+ sw $fp, 4($sp)
+ sw $s0, 8($sp)
+ sw $s1, 12($sp)
+ sw $s2, 16($sp)
+ sw $s3, 20($sp)
+ sw $s4, 24($sp)
+ sw $s5, 28($sp)
+ sw $s6, 32($sp)
+ sw $s7, 36($sp)
+
+ lw NONCE_0, 0(NONCE)
+ /* Test IN or OUT is unaligned.
+ * UNALIGNED (T1) = ( IN | OUT ) & 0x00000003
+ */
+ or T1, IN, OUT
+ andi T1, 0x3
+
+ /* Load constant */
+ lui X0, %hi(CONSTANT_1)
+ lui X1, %hi(CONSTANT_2)
+ lui X2, %hi(CONSTANT_3)
+ lui X3, %hi(CONSTANT_4)
+ ori X0, %lo(CONSTANT_1)
+ ori X1, %lo(CONSTANT_2)
+ ori X2, %lo(CONSTANT_3)
+ ori X3, %lo(CONSTANT_4)
+
+ /* Store constant on stack. */
+ sw X0, 0+CONSTANT_OFS_SP($sp)
+ sw X1, 4+CONSTANT_OFS_SP($sp)
+ sw X2, 8+CONSTANT_OFS_SP($sp)
+ sw X3, 12+CONSTANT_OFS_SP($sp)
+
+ sw T1, UNALIGNED_OFS_SP($sp)
+
+ .set noreorder
+ b .Lchacha20_rounds_start
+ andi BYTES, (CHACHA20_BLOCK_SIZE-1)
+ .set reorder
+
+.align 4
+.Loop_chacha20_rounds:
+ addiu IN, CHACHA20_BLOCK_SIZE
+ addiu OUT, CHACHA20_BLOCK_SIZE
+ addiu NONCE_0, 1
+
+ lw X0, 0+CONSTANT_OFS_SP($sp)
+ lw X1, 4+CONSTANT_OFS_SP($sp)
+ lw X2, 8+CONSTANT_OFS_SP($sp)
+ lw X3, 12+CONSTANT_OFS_SP($sp)
+ lw T1, UNALIGNED_OFS_SP($sp)
+
+.Lchacha20_rounds_start:
+ lw X4, 0(KEY)
+ lw X5, 4(KEY)
+ lw X6, 8(KEY)
+ lw X7, 12(KEY)
+ lw X8, 16(KEY)
+ lw X9, 20(KEY)
+ lw X10, 24(KEY)
+ lw X11, 28(KEY)
+
+ move X12, NONCE_0
+ lw X13, 4(NONCE)
+ lw X14, 8(NONCE)
+ lw X15, 12(NONCE)
+
+ li $at, 9
+.Loop_chacha20_xor_rounds:
+ AXR( 0, 1, 2, 3, 4, 5, 6, 7, 12,13,14,15, 16);
+ AXR( 8, 9,10,11, 12,13,14,15, 4, 5, 6, 7, 12);
+ AXR( 0, 1, 2, 3, 4, 5, 6, 7, 12,13,14,15, 8);
+ AXR( 8, 9,10,11, 12,13,14,15, 4, 5, 6, 7, 7);
+ AXR( 0, 1, 2, 3, 5, 6, 7, 4, 15,12,13,14, 16);
+ AXR(10,11, 8, 9, 15,12,13,14, 5, 6, 7, 4, 12);
+ AXR( 0, 1, 2, 3, 5, 6, 7, 4, 15,12,13,14, 8);
+ AXR(10,11, 8, 9, 15,12,13,14, 5, 6, 7, 4, 7);
+ .set noreorder
+ bnez $at, .Loop_chacha20_xor_rounds
+ addiu $at, -1
+
+ /* Unaligned? Jump */
+ bnez T1, .Loop_chacha20_unaligned
+ andi $at, BYTES, MASK_U32
+
+ /* Last round? No jump */
+ bne OUT, PTR_LAST_ROUND, .Lchacha20_mips_xor_aligned_64_b
+ /* Load upper half of jump table addr */
+ lui T0, %hi(.Lchacha20_mips_jmptbl_aligned_0)
+
+ /* Full block? Jump */
+ beqz BYTES, .Lchacha20_mips_xor_aligned_64_b
+ /* Calculate lower half jump table addr and offset */
+ ins T0, $at, 2, 6
+
+ subu T0, $at
+ addiu T0, %lo(.Lchacha20_mips_jmptbl_aligned_0)
+
+ jr T0
+ /* Delay slot */
+ nop
+
+ .set reorder
+
+.Loop_chacha20_unaligned:
+ .set noreorder
+
+ /* Last round? no jump */
+ bne OUT, PTR_LAST_ROUND, .Lchacha20_mips_xor_unaligned_64_b
+ /* Load upper half of jump table addr */
+ lui T0, %hi(.Lchacha20_mips_jmptbl_unaligned_0)
+
+ /* Full block? Jump */
+ beqz BYTES, .Lchacha20_mips_xor_unaligned_64_b
+
+ /* Calculate lower half jump table addr and offset */
+ ins T0, $at, 2, 6
+ subu T0, $at
+ addiu T0, %lo(.Lchacha20_mips_jmptbl_unaligned_0)
+
+ jr T0
+ /* Delay slot */
+ nop
+
+ .set reorder
+
+/* Aligned code path
+ */
+.align 4
+ STORE_ALIGNED(64, 15, NONCE,12)
+ STORE_ALIGNED(60, 14, NONCE, 8)
+ STORE_ALIGNED(56, 13, NONCE, 4)
+ STORE_ALIGNED(52, 12, NONCE, 0)
+ STORE_ALIGNED(48, 11, KEY, 28)
+ STORE_ALIGNED(44, 10, KEY, 24)
+ STORE_ALIGNED(40, 9, KEY, 20)
+ STORE_ALIGNED(36, 8, KEY, 16)
+ STORE_ALIGNED(32, 7, KEY, 12)
+ STORE_ALIGNED(28, 6, KEY, 8)
+ STORE_ALIGNED(24, 5, KEY, 4)
+ STORE_ALIGNED(20, 4, KEY, 0)
+ STORE_ALIGNED(16, 3, $sp, 12+CONSTANT_OFS_SP)
+ STORE_ALIGNED(12, 2, $sp, 8+CONSTANT_OFS_SP)
+ STORE_ALIGNED( 8, 1, $sp, 4+CONSTANT_OFS_SP)
+.Lchacha20_mips_xor_aligned_4_b:
+ /* STORE_ALIGNED( 4, 0, $sp, 0+CONSTANT_OFS_SP) */
+ lw T0, 0+CONSTANT_OFS_SP($sp)
+ lw T1, 0(IN)
+ addu X0, T0
+ CPU_TO_LE32(X0)
+ xor X0, T1
+ .set noreorder
+ bne OUT, PTR_LAST_ROUND, .Loop_chacha20_rounds
+ sw X0, 0(OUT)
+ .set reorder
+
+ .set noreorder
+ bne $at, BYTES, .Lchacha20_mips_xor_bytes
+ /* Empty delayslot, Increase NONCE_0, return NONCE_0 value */
+ addiu NONCE_0, 1
+ .set noreorder
+
+.Lchacha20_mips_xor_done:
+ /* Restore used registers */
+ lw $ra, 0($sp)
+ lw $fp, 4($sp)
+ lw $s0, 8($sp)
+ lw $s1, 12($sp)
+ lw $s2, 16($sp)
+ lw $s3, 20($sp)
+ lw $s4, 24($sp)
+ lw $s5, 28($sp)
+ lw $s6, 32($sp)
+ lw $s7, 36($sp)
+.Lchacha20_mips_end:
+ .set noreorder
+ jr $ra
+ addiu $sp, STACK_SIZE
+ .set reorder
+
+ .set noreorder
+ /* Start jump table */
+ JMPTBL_ALIGNED( 0, 0, $sp, 0+CONSTANT_OFS_SP)
+ JMPTBL_ALIGNED( 4, 1, $sp, 4+CONSTANT_OFS_SP)
+ JMPTBL_ALIGNED( 8, 2, $sp, 8+CONSTANT_OFS_SP)
+ JMPTBL_ALIGNED(12, 3, $sp, 12+CONSTANT_OFS_SP)
+ JMPTBL_ALIGNED(16, 4, KEY, 0)
+ JMPTBL_ALIGNED(20, 5, KEY, 4)
+ JMPTBL_ALIGNED(24, 6, KEY, 8)
+ JMPTBL_ALIGNED(28, 7, KEY, 12)
+ JMPTBL_ALIGNED(32, 8, KEY, 16)
+ JMPTBL_ALIGNED(36, 9, KEY, 20)
+ JMPTBL_ALIGNED(40, 10, KEY, 24)
+ JMPTBL_ALIGNED(44, 11, KEY, 28)
+ JMPTBL_ALIGNED(48, 12, NONCE, 0)
+ JMPTBL_ALIGNED(52, 13, NONCE, 4)
+ JMPTBL_ALIGNED(56, 14, NONCE, 8)
+ JMPTBL_ALIGNED(60, 15, NONCE,12)
+ /* End jump table */
+ .set reorder
+
+/* Unaligned code path
+ */
+ STORE_UNALIGNED(64, 15, NONCE,12)
+ STORE_UNALIGNED(60, 14, NONCE, 8)
+ STORE_UNALIGNED(56, 13, NONCE, 4)
+ STORE_UNALIGNED(52, 12, NONCE, 0)
+ STORE_UNALIGNED(48, 11, KEY, 28)
+ STORE_UNALIGNED(44, 10, KEY, 24)
+ STORE_UNALIGNED(40, 9, KEY, 20)
+ STORE_UNALIGNED(36, 8, KEY, 16)
+ STORE_UNALIGNED(32, 7, KEY, 12)
+ STORE_UNALIGNED(28, 6, KEY, 8)
+ STORE_UNALIGNED(24, 5, KEY, 4)
+ STORE_UNALIGNED(20, 4, KEY, 0)
+ STORE_UNALIGNED(16, 3, $sp, 12+CONSTANT_OFS_SP)
+ STORE_UNALIGNED(12, 2, $sp, 8+CONSTANT_OFS_SP)
+ STORE_UNALIGNED( 8, 1, $sp, 4+CONSTANT_OFS_SP)
+.Lchacha20_mips_xor_unaligned_4_b:
+ /* STORE_UNALIGNED( 4, 0, $sp, 0+CONSTANT_OFS_SP) */
+ lw T0, 0+CONSTANT_OFS_SP($sp)
+ lwl T1, 0+MSB(IN)
+ lwr T1, 0+LSB(IN)
+ addu X0, T0
+ CPU_TO_LE32(X0)
+ xor X0, T1
+ swl X0, 0+MSB(OUT)
+ .set noreorder
+ bne OUT, PTR_LAST_ROUND, .Loop_chacha20_rounds
+ swr X0, 0+LSB(OUT)
+ .set reorder
+
+ /* Fall through to byte handling */
+ .set noreorder
+ beq $at, BYTES, .Lchacha20_mips_xor_done
+ /* Empty delayslot, increase NONCE_0, return NONCE_0 value */
+.Lchacha20_mips_xor_unaligned_0_b:
+.Lchacha20_mips_xor_aligned_0_b:
+ addiu NONCE_0, 1
+ .set reorder
+
+.Lchacha20_mips_xor_bytes:
+ addu OUT, $at
+ addu IN, $at
+ addu SAVED_X, SAVED_CA
+ /* First byte */
+ lbu T1, 0(IN)
+ andi $at, BYTES, 2
+ CPU_TO_LE32(SAVED_X)
+ ROTR(SAVED_X)
+ xor T1, SAVED_X
+ .set noreorder
+ beqz $at, .Lchacha20_mips_xor_done
+ sb T1, 0(OUT)
+ .set reorder
+ /* Second byte */
+ lbu T1, 1(IN)
+ andi $at, BYTES, 1
+ ROTx SAVED_X, 8
+ xor T1, SAVED_X
+ .set noreorder
+ beqz $at, .Lchacha20_mips_xor_done
+ sb T1, 1(OUT)
+ .set reorder
+ /* Third byte */
+ lbu T1, 2(IN)
+ ROTx SAVED_X, 8
+ xor T1, SAVED_X
+ .set noreorder
+ b .Lchacha20_mips_xor_done
+ sb T1, 2(OUT)
+ .set reorder
+.set noreorder
+
+.Lchacha20_mips_jmptbl_unaligned:
+ /* Start jump table */
+ JMPTBL_UNALIGNED( 0, 0, $sp, 0+CONSTANT_OFS_SP)
+ JMPTBL_UNALIGNED( 4, 1, $sp, 4+CONSTANT_OFS_SP)
+ JMPTBL_UNALIGNED( 8, 2, $sp, 8+CONSTANT_OFS_SP)
+ JMPTBL_UNALIGNED(12, 3, $sp, 12+CONSTANT_OFS_SP)
+ JMPTBL_UNALIGNED(16, 4, KEY, 0)
+ JMPTBL_UNALIGNED(20, 5, KEY, 4)
+ JMPTBL_UNALIGNED(24, 6, KEY, 8)
+ JMPTBL_UNALIGNED(28, 7, KEY, 12)
+ JMPTBL_UNALIGNED(32, 8, KEY, 16)
+ JMPTBL_UNALIGNED(36, 9, KEY, 20)
+ JMPTBL_UNALIGNED(40, 10, KEY, 24)
+ JMPTBL_UNALIGNED(44, 11, KEY, 28)
+ JMPTBL_UNALIGNED(48, 12, NONCE, 0)
+ JMPTBL_UNALIGNED(52, 13, NONCE, 4)
+ JMPTBL_UNALIGNED(56, 14, NONCE, 8)
+ JMPTBL_UNALIGNED(60, 15, NONCE,12)
+ /* End jump table */
+.set reorder
+
+.end chacha20_mips
+.set at
diff --git a/lib/zinc/chacha20/chacha20.c b/lib/zinc/chacha20/chacha20.c
index 385e7350adc3..0cf490ca23be 100644
--- a/lib/zinc/chacha20/chacha20.c
+++ b/lib/zinc/chacha20/chacha20.c
@@ -18,6 +18,8 @@
#include "chacha20-x86_64-glue.h"
#elif defined(CONFIG_ZINC_ARCH_ARM) || defined(CONFIG_ZINC_ARCH_ARM64)
#include "chacha20-arm-glue.h"
+#elif defined(CONFIG_ZINC_ARCH_MIPS)
+#include "chacha20-mips-glue.h"
#else
void __init chacha20_fpu_init(void)
{
--
2.19.0
^ permalink raw reply related
* [PATCH net-next v5 03/20] zinc: ChaCha20 generic C implementation and selftest
From: Jason A. Donenfeld @ 2018-09-18 16:16 UTC (permalink / raw)
To: linux-kernel, netdev, linux-crypto, davem, gregkh
Cc: Jason A. Donenfeld, Samuel Neves, Andy Lutomirski,
Jean-Philippe Aumasson
In-Reply-To: <20180918161646.19105-1-Jason@zx2c4.com>
This implements the ChaCha20 permutation as a single C statement, by way
of the comma operator, which the compiler is able to simplify
terrifically.
Information: https://cr.yp.to/chacha.html
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Cc: Samuel Neves <sneves@dei.uc.pt>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
---
include/zinc/chacha20.h | 52 ++
lib/zinc/Kconfig | 4 +
lib/zinc/Makefile | 3 +
lib/zinc/chacha20/chacha20.c | 193 ++++++
lib/zinc/selftest/chacha20.h | 1182 ++++++++++++++++++++++++++++++++++
5 files changed, 1434 insertions(+)
create mode 100644 include/zinc/chacha20.h
create mode 100644 lib/zinc/chacha20/chacha20.c
create mode 100644 lib/zinc/selftest/chacha20.h
diff --git a/include/zinc/chacha20.h b/include/zinc/chacha20.h
new file mode 100644
index 000000000000..afad69409f5f
--- /dev/null
+++ b/include/zinc/chacha20.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ */
+
+#ifndef _ZINC_CHACHA20_H
+#define _ZINC_CHACHA20_H
+
+#include <asm/unaligned.h>
+#include <linux/simd.h>
+#include <linux/kernel.h>
+#include <linux/types.h>
+
+enum {
+ CHACHA20_IV_SIZE = 16,
+ CHACHA20_KEY_SIZE = 32,
+ CHACHA20_BLOCK_SIZE = 64,
+ CHACHA20_BLOCK_WORDS = CHACHA20_BLOCK_SIZE / sizeof(u32),
+ HCHACHA20_KEY_SIZE = 32,
+ HCHACHA20_NONCE_SIZE = 16
+};
+
+struct chacha20_ctx {
+ u32 key[8];
+ u32 counter[4];
+} __aligned(32);
+
+static inline void chacha20_init(struct chacha20_ctx *state,
+ const u8 key[CHACHA20_KEY_SIZE],
+ const u64 nonce)
+{
+ state->key[0] = get_unaligned_le32(key + 0);
+ state->key[1] = get_unaligned_le32(key + 4);
+ state->key[2] = get_unaligned_le32(key + 8);
+ state->key[3] = get_unaligned_le32(key + 12);
+ state->key[4] = get_unaligned_le32(key + 16);
+ state->key[5] = get_unaligned_le32(key + 20);
+ state->key[6] = get_unaligned_le32(key + 24);
+ state->key[7] = get_unaligned_le32(key + 28);
+ state->counter[0] = state->counter[1] = 0;
+ state->counter[2] = nonce & U32_MAX;
+ state->counter[3] = nonce >> 32;
+}
+void chacha20(struct chacha20_ctx *state, u8 *dst, const u8 *src, u32 len,
+ simd_context_t *simd_context);
+
+/* Derived key should be 32-bit aligned */
+void hchacha20(u8 derived_key[CHACHA20_KEY_SIZE],
+ const u8 nonce[HCHACHA20_NONCE_SIZE],
+ const u8 key[HCHACHA20_KEY_SIZE], simd_context_t *simd_context);
+
+#endif /* _ZINC_CHACHA20_H */
diff --git a/lib/zinc/Kconfig b/lib/zinc/Kconfig
index 4e2e59126a67..1ca1ae1e9ea9 100644
--- a/lib/zinc/Kconfig
+++ b/lib/zinc/Kconfig
@@ -1,3 +1,7 @@
+config ZINC_CHACHA20
+ tristate
+ select CRYPTO_ALGAPI
+
config ZINC_DEBUG
bool "Zinc cryptography library debugging and self-tests"
help
diff --git a/lib/zinc/Makefile b/lib/zinc/Makefile
index 83dfd63988c0..83b320f1ace2 100644
--- a/lib/zinc/Makefile
+++ b/lib/zinc/Makefile
@@ -2,3 +2,6 @@ ccflags-y := -O3
ccflags-y += -Wframe-larger-than=$(if (CONFIG_KASAN),16384,8192)
ccflags-y += -D'pr_fmt(fmt)="zinc: " fmt'
ccflags-$(CONFIG_ZINC_DEBUG) += -DDEBUG
+
+zinc_chacha20-y := chacha20/chacha20.o
+obj-$(CONFIG_ZINC_CHACHA20) += zinc_chacha20.o
diff --git a/lib/zinc/chacha20/chacha20.c b/lib/zinc/chacha20/chacha20.c
new file mode 100644
index 000000000000..3f00e1edd4c8
--- /dev/null
+++ b/lib/zinc/chacha20/chacha20.c
@@ -0,0 +1,193 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ *
+ * Implementation of the ChaCha20 stream cipher.
+ *
+ * Information: https://cr.yp.to/chacha.html
+ */
+
+#include <zinc/chacha20.h>
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <crypto/algapi.h>
+
+#ifndef HAVE_CHACHA20_ARCH_IMPLEMENTATION
+void __init chacha20_fpu_init(void)
+{
+}
+static inline bool chacha20_arch(u8 *out, const u8 *in, const size_t len,
+ const u32 key[8], const u32 counter[4],
+ simd_context_t *simd_context)
+{
+ return false;
+}
+static inline bool hchacha20_arch(u8 *derived_key, const u8 *nonce,
+ const u8 *key, simd_context_t *simd_context)
+{
+ return false;
+}
+#endif
+
+#define EXPAND_32_BYTE_K 0x61707865U, 0x3320646eU, 0x79622d32U, 0x6b206574U
+
+#define QUARTER_ROUND(x, a, b, c, d) ( \
+ x[a] += x[b], \
+ x[d] = rol32((x[d] ^ x[a]), 16), \
+ x[c] += x[d], \
+ x[b] = rol32((x[b] ^ x[c]), 12), \
+ x[a] += x[b], \
+ x[d] = rol32((x[d] ^ x[a]), 8), \
+ x[c] += x[d], \
+ x[b] = rol32((x[b] ^ x[c]), 7) \
+)
+
+#define C(i, j) (i * 4 + j)
+
+#define DOUBLE_ROUND(x) ( \
+ /* Column Round */ \
+ QUARTER_ROUND(x, C(0, 0), C(1, 0), C(2, 0), C(3, 0)), \
+ QUARTER_ROUND(x, C(0, 1), C(1, 1), C(2, 1), C(3, 1)), \
+ QUARTER_ROUND(x, C(0, 2), C(1, 2), C(2, 2), C(3, 2)), \
+ QUARTER_ROUND(x, C(0, 3), C(1, 3), C(2, 3), C(3, 3)), \
+ /* Diagonal Round */ \
+ QUARTER_ROUND(x, C(0, 0), C(1, 1), C(2, 2), C(3, 3)), \
+ QUARTER_ROUND(x, C(0, 1), C(1, 2), C(2, 3), C(3, 0)), \
+ QUARTER_ROUND(x, C(0, 2), C(1, 3), C(2, 0), C(3, 1)), \
+ QUARTER_ROUND(x, C(0, 3), C(1, 0), C(2, 1), C(3, 2)) \
+)
+
+#define TWENTY_ROUNDS(x) ( \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x) \
+)
+
+static void chacha20_block_generic(__le32 *stream, u32 *state)
+{
+ u32 x[CHACHA20_BLOCK_WORDS];
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(x); ++i)
+ x[i] = state[i];
+
+ TWENTY_ROUNDS(x);
+
+ for (i = 0; i < ARRAY_SIZE(x); ++i)
+ stream[i] = cpu_to_le32(x[i] + state[i]);
+
+ ++state[12];
+}
+
+static void chacha20_generic(u8 *out, const u8 *in, u32 len, const u32 key[8],
+ const u32 counter[4])
+{
+ __le32 buf[CHACHA20_BLOCK_WORDS];
+ u32 x[] = {
+ EXPAND_32_BYTE_K,
+ key[0], key[1], key[2], key[3],
+ key[4], key[5], key[6], key[7],
+ counter[0], counter[1], counter[2], counter[3]
+ };
+
+ if (out != in)
+ memmove(out, in, len);
+
+ while (len >= CHACHA20_BLOCK_SIZE) {
+ chacha20_block_generic(buf, x);
+ crypto_xor(out, (u8 *)buf, CHACHA20_BLOCK_SIZE);
+ len -= CHACHA20_BLOCK_SIZE;
+ out += CHACHA20_BLOCK_SIZE;
+ }
+ if (len) {
+ chacha20_block_generic(buf, x);
+ crypto_xor(out, (u8 *)buf, len);
+ }
+}
+
+void chacha20(struct chacha20_ctx *state, u8 *dst, const u8 *src, u32 len,
+ simd_context_t *simd_context)
+{
+ if (!chacha20_arch(dst, src, len, state->key, state->counter,
+ simd_context))
+ chacha20_generic(dst, src, len, state->key, state->counter);
+ state->counter[0] += (len + 63) / 64;
+}
+EXPORT_SYMBOL(chacha20);
+
+static void hchacha20_generic(u8 derived_key[CHACHA20_KEY_SIZE],
+ const u8 nonce[HCHACHA20_NONCE_SIZE],
+ const u8 key[HCHACHA20_KEY_SIZE])
+{
+ __le32 *out = (__force __le32 *)derived_key;
+ u32 x[] = { EXPAND_32_BYTE_K,
+ get_unaligned_le32(key + 0),
+ get_unaligned_le32(key + 4),
+ get_unaligned_le32(key + 8),
+ get_unaligned_le32(key + 12),
+ get_unaligned_le32(key + 16),
+ get_unaligned_le32(key + 20),
+ get_unaligned_le32(key + 24),
+ get_unaligned_le32(key + 28),
+ get_unaligned_le32(nonce + 0),
+ get_unaligned_le32(nonce + 4),
+ get_unaligned_le32(nonce + 8),
+ get_unaligned_le32(nonce + 12)
+ };
+
+ TWENTY_ROUNDS(x);
+
+ out[0] = cpu_to_le32(x[0]);
+ out[1] = cpu_to_le32(x[1]);
+ out[2] = cpu_to_le32(x[2]);
+ out[3] = cpu_to_le32(x[3]);
+ out[4] = cpu_to_le32(x[12]);
+ out[5] = cpu_to_le32(x[13]);
+ out[6] = cpu_to_le32(x[14]);
+ out[7] = cpu_to_le32(x[15]);
+}
+
+/* Derived key should be 32-bit aligned */
+void hchacha20(u8 derived_key[CHACHA20_KEY_SIZE],
+ const u8 nonce[HCHACHA20_NONCE_SIZE],
+ const u8 key[HCHACHA20_KEY_SIZE], simd_context_t *simd_context)
+{
+ if (!hchacha20_arch(derived_key, nonce, key, simd_context))
+ hchacha20_generic(derived_key, nonce, key);
+}
+EXPORT_SYMBOL(hchacha20);
+
+#include "../selftest/chacha20.h"
+
+static bool nosimd __initdata = false;
+
+static int __init mod_init(void)
+{
+ if (!nosimd)
+ chacha20_fpu_init();
+#ifdef DEBUG
+ if (!chacha20_selftest())
+ return -ENOTRECOVERABLE;
+#endif
+ return 0;
+}
+
+static void __exit mod_exit(void)
+{
+}
+
+module_param(nosimd, bool, 0);
+module_init(mod_init);
+module_exit(mod_exit);
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("ChaCha20 stream cipher");
+MODULE_AUTHOR("Jason A. Donenfeld <Jason@zx2c4.com>");
diff --git a/lib/zinc/selftest/chacha20.h b/lib/zinc/selftest/chacha20.h
new file mode 100644
index 000000000000..f591460a854a
--- /dev/null
+++ b/lib/zinc/selftest/chacha20.h
@@ -0,0 +1,1182 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ */
+
+#ifdef DEBUG
+
+enum { MAXIMUM_TEST_BUFFER_LEN = 800 };
+
+struct chacha20_testvec {
+ u8 key[CHACHA20_KEY_SIZE];
+ u64 nonce;
+ u8 input[MAXIMUM_TEST_BUFFER_LEN];
+ u8 result[MAXIMUM_TEST_BUFFER_LEN];
+ size_t ilen;
+};
+
+/*
+ * #!/usr/bin/env python3
+ *
+ * import chacha20
+ * import os
+ * import struct
+ *
+ * def encode_blob(blob):
+ * a = ""
+ * x = 0
+ * for i in blob:
+ * a += ('0x%02x' % i) + ","
+ * x += 1
+ * if x % 8 == 0:
+ * a += "\n\t\t "
+ * else:
+ * a += " "
+ * if x % 8 == 0:
+ * return a[:len(a) - 8]
+ * return a[:len(a) - 2]
+ *
+ * enc = [ ]
+ * dec = [ ]
+ *
+ * def make_vector(plen):
+ * key = os.urandom(32)
+ * nonce = os.urandom(8)
+ * p = os.urandom(plen)
+ * c = chacha20.chacha20_encrypt(p, key, nonce)
+ *
+ * out = "{\n"
+ * out += "\t.key\t= { " + encode_blob(key) + " },\n"
+ * out += "\t.nonce\t= " + hex(struct.unpack("<Q", nonce)[0]) + "ULL,\n"
+ * out += "\t.input\t= { " + encode_blob(p) + " },\n"
+ * out += "\t.ilen\t= " + str(len(p)) + ",\n"
+ * out += "\t.result\t= { " + encode_blob(c) + " }\n"
+ * out += "}"
+ * enc.append(out)
+ *
+ * block_size = 64
+ *
+ * make_vector(1)
+ * make_vector(63)
+ * make_vector(64)
+ * make_vector(127)
+ * make_vector(250)
+ * make_vector(256)
+ * make_vector(280)
+ * make_vector(511)
+ * make_vector(512)
+ * make_vector(513)
+ * make_vector(514)
+ * make_vector(769)
+ *
+ * print(", ".join(enc))
+ */
+
+static const struct chacha20_testvec
+chacha20_vectors[] __initconst = {{
+ .key = { 0x8e, 0xee, 0x50, 0x40, 0xb2, 0x76, 0x80, 0xeb,
+ 0xd8, 0x7e, 0xf2, 0x5a, 0xcf, 0x25, 0xd6, 0x6c,
+ 0x9e, 0xc8, 0x3d, 0x0c, 0xf7, 0x05, 0x00, 0x40,
+ 0xa0, 0x75, 0x66, 0x86, 0xdf, 0x8d, 0xac, 0x77 },
+ .nonce = 0x37d4e0484f294097ULL,
+ .input = { 0xe1 },
+ .ilen = 1,
+ .result = { 0xf1 }
+}, {
+ .key = { 0x23, 0xe2, 0x3b, 0x3a, 0x22, 0xd2, 0xfb, 0x69,
+ 0x92, 0xcc, 0x0a, 0xda, 0x48, 0x21, 0x33, 0x8f,
+ 0x57, 0x58, 0x08, 0x17, 0xca, 0xe5, 0x56, 0x08,
+ 0x71, 0x86, 0x91, 0xc7, 0xae, 0x64, 0x4d, 0xfd },
+ .nonce = 0xe0c627c7e1a4b012ULL,
+ .input = { 0x83, 0x95, 0x1c, 0xc6, 0x8a, 0x53, 0x92, 0x6a,
+ 0x3f, 0x1e, 0x67, 0x17, 0x0b, 0x2b, 0x45, 0x04,
+ 0xaf, 0x5a, 0x75, 0x59, 0xb7, 0xab, 0x80, 0xde,
+ 0xa4, 0x8f, 0xda, 0x03, 0xfd, 0xda, 0xe1, 0x7e,
+ 0x75, 0x99, 0x44, 0x6f, 0xea, 0x75, 0x26, 0x68,
+ 0xcc, 0x20, 0xe0, 0x7c, 0xf1, 0x09, 0x31, 0x44,
+ 0xba, 0xb9, 0xe8, 0x7f, 0xce, 0x5e, 0x89, 0x6d,
+ 0x0d, 0x4d, 0x20, 0x93, 0x52, 0x91, 0x85 },
+ .ilen = 63,
+ .result = { 0xa1, 0x32, 0xec, 0xab, 0x86, 0x35, 0x99, 0x88,
+ 0xc0, 0xef, 0xae, 0x69, 0x64, 0xa3, 0xba, 0xcb,
+ 0xb9, 0x44, 0x24, 0x92, 0x9f, 0x33, 0xdb, 0xc0,
+ 0x7f, 0xe5, 0x39, 0xa8, 0x57, 0x2a, 0x29, 0x5f,
+ 0xdc, 0xc4, 0xee, 0x7f, 0xcd, 0xd9, 0x5f, 0x0b,
+ 0x49, 0xc2, 0x9a, 0xa2, 0x5e, 0x5a, 0x37, 0x3d,
+ 0x7a, 0x1e, 0x66, 0xc6, 0xd1, 0xb9, 0xcb, 0x98,
+ 0xc0, 0xd1, 0xc8, 0xc5, 0xf9, 0x3d, 0xed }
+}, {
+ .key = { 0x09, 0x5c, 0x71, 0xde, 0x41, 0x76, 0x2d, 0x48,
+ 0x4f, 0x58, 0xec, 0x0e, 0x62, 0x00, 0x73, 0x04,
+ 0x03, 0x47, 0x04, 0x8a, 0x4c, 0xd7, 0x52, 0x5a,
+ 0x7b, 0x84, 0x36, 0xbf, 0x8a, 0xdb, 0xeb, 0xf5 },
+ .nonce = 0x45a98ae9d6bf5373ULL,
+ .input = { 0x40, 0x3d, 0x31, 0x7d, 0x63, 0xba, 0xec, 0xd8,
+ 0xa6, 0x73, 0xc4, 0x8c, 0x40, 0xc3, 0xaa, 0x3b,
+ 0xca, 0x03, 0x52, 0xa0, 0xe8, 0x3f, 0xb6, 0x21,
+ 0x3f, 0x07, 0xb1, 0xb3, 0x14, 0x94, 0xf8, 0x36,
+ 0xb3, 0x06, 0x33, 0x1e, 0xb1, 0x05, 0x94, 0x53,
+ 0x1f, 0xee, 0x16, 0x8e, 0xcd, 0x0f, 0x38, 0x41,
+ 0x1f, 0x4d, 0x12, 0x60, 0xec, 0x88, 0x42, 0x2a,
+ 0xca, 0x3e, 0x84, 0x33, 0x13, 0x17, 0xeb, 0x04 },
+ .ilen = 64,
+ .result = { 0x48, 0x0e, 0x7f, 0x54, 0xe6, 0x3e, 0x98, 0x81,
+ 0xc6, 0x1d, 0x5a, 0x1b, 0x4d, 0xaf, 0x27, 0x3c,
+ 0xce, 0xd7, 0x98, 0x05, 0x92, 0xdb, 0xb9, 0x24,
+ 0xef, 0xbc, 0x6a, 0x01, 0xb9, 0x61, 0xb9, 0xf9,
+ 0x66, 0xee, 0xc9, 0x30, 0xf5, 0x48, 0xb1, 0xde,
+ 0xcd, 0xe9, 0x4d, 0x05, 0x2f, 0xea, 0x2c, 0xdf,
+ 0x32, 0xc1, 0x43, 0x50, 0x31, 0x1d, 0x0a, 0x3a,
+ 0x4e, 0x24, 0xd3, 0x69, 0xc6, 0x18, 0x23, 0xdb }
+}, {
+ .key = { 0x5d, 0x01, 0xdf, 0xe5, 0xbc, 0xf6, 0xaf, 0x04,
+ 0x11, 0xda, 0xef, 0x43, 0x6b, 0xdf, 0xdd, 0xb8,
+ 0xa6, 0xc6, 0x4a, 0xbd, 0xa4, 0xb3, 0xba, 0xf6,
+ 0xce, 0x2f, 0xf8, 0x34, 0xbb, 0xdd, 0x22, 0x42 },
+ .nonce = 0x70faa08a4184e385ULL,
+ .input = { 0xb8, 0x06, 0x92, 0x20, 0xc7, 0x7c, 0xb2, 0x01,
+ 0x3e, 0xf6, 0x73, 0x6c, 0x74, 0x48, 0x72, 0xec,
+ 0x5a, 0x38, 0xfa, 0x36, 0x39, 0x39, 0xff, 0x3c,
+ 0x81, 0x91, 0x99, 0x0b, 0x29, 0xec, 0xed, 0x4c,
+ 0x53, 0x9a, 0x24, 0xce, 0x4b, 0xcf, 0x8d, 0x36,
+ 0x2b, 0xf1, 0x8b, 0x0a, 0x36, 0xdc, 0xf8, 0xdc,
+ 0x5a, 0x77, 0x64, 0xca, 0x5f, 0x06, 0xf8, 0xb8,
+ 0xc2, 0xa3, 0x69, 0x0d, 0x2f, 0x2f, 0xa5, 0x39,
+ 0x94, 0xc6, 0x83, 0xe3, 0x20, 0x48, 0x8b, 0xb7,
+ 0x58, 0x26, 0xd0, 0x39, 0x2a, 0x4f, 0x7b, 0x66,
+ 0xda, 0x52, 0x0d, 0xa9, 0x5b, 0xb3, 0x44, 0x57,
+ 0x38, 0x86, 0x6a, 0xba, 0xa7, 0x86, 0x44, 0x82,
+ 0xfb, 0xa8, 0xe8, 0x9d, 0x08, 0x1c, 0x3a, 0xce,
+ 0x92, 0x02, 0x61, 0xb6, 0x3e, 0x81, 0xcb, 0x44,
+ 0x12, 0x44, 0xf8, 0x2f, 0x53, 0x62, 0xbd, 0xfb,
+ 0x3e, 0x9f, 0x65, 0x19, 0xae, 0xc8, 0x4f },
+ .ilen = 127,
+ .result = { 0x0e, 0x37, 0xd1, 0x08, 0x6c, 0x55, 0xc5, 0x88,
+ 0x33, 0x0a, 0xbd, 0x6e, 0x24, 0xb1, 0x00, 0xdd,
+ 0xb0, 0x43, 0x1f, 0x5a, 0x20, 0x06, 0xfb, 0xa5,
+ 0x00, 0x7c, 0xb4, 0x86, 0x58, 0x46, 0x55, 0x6c,
+ 0xa2, 0x28, 0xd7, 0x55, 0x51, 0xd5, 0xf1, 0xa8,
+ 0x43, 0x06, 0xc4, 0x50, 0xd3, 0x6f, 0x44, 0x21,
+ 0x9c, 0xce, 0x29, 0x89, 0xc2, 0x79, 0x20, 0x8f,
+ 0x76, 0xfe, 0x3a, 0xef, 0xdd, 0x59, 0x21, 0x94,
+ 0xe4, 0xe2, 0xa3, 0x3c, 0x08, 0x74, 0xe7, 0xb5,
+ 0x99, 0xf3, 0x8a, 0x88, 0xad, 0xb1, 0x98, 0x31,
+ 0xd2, 0x2a, 0x36, 0xda, 0xdf, 0x2c, 0x4c, 0xcd,
+ 0xd6, 0xed, 0xe2, 0xca, 0x54, 0xa0, 0x72, 0x0d,
+ 0xce, 0x16, 0xe3, 0xa7, 0x2d, 0x93, 0x2b, 0x07,
+ 0x18, 0x3f, 0xdb, 0x14, 0xd0, 0x00, 0x26, 0xe4,
+ 0x48, 0xa6, 0xe4, 0x59, 0xb7, 0xec, 0xbc, 0xee,
+ 0xba, 0x5e, 0x7e, 0xae, 0xf2, 0xd8, 0x91 }
+}, {
+ .key = { 0x92, 0x58, 0xaa, 0xf5, 0x07, 0x8f, 0xa3, 0xdd,
+ 0xc6, 0x97, 0x32, 0x34, 0xd6, 0x96, 0xca, 0xc1,
+ 0x6d, 0xf3, 0xef, 0xfe, 0x44, 0x32, 0x82, 0xa4,
+ 0x52, 0x26, 0xf2, 0x69, 0xfb, 0xbf, 0x24, 0xc8 },
+ .nonce = 0xf66f76ba3ce150e4ULL,
+ .input = { 0x69, 0x3d, 0xd7, 0xa9, 0x86, 0xf7, 0x64, 0xac,
+ 0x5b, 0x0d, 0xd3, 0xa8, 0x4a, 0xf8, 0xc9, 0x12,
+ 0xa9, 0x2a, 0xc2, 0x28, 0x14, 0x02, 0x48, 0x7a,
+ 0x64, 0xeb, 0x5b, 0x5a, 0xce, 0x79, 0xca, 0x86,
+ 0xa4, 0x60, 0xe6, 0xbe, 0x56, 0x29, 0x50, 0x2d,
+ 0xbd, 0x44, 0x88, 0xf8, 0x92, 0xf7, 0x60, 0xe7,
+ 0xb1, 0xf4, 0x8b, 0x44, 0xbf, 0x25, 0x4d, 0xc8,
+ 0x6b, 0x2e, 0xdc, 0x9e, 0xe8, 0x0f, 0x38, 0x3b,
+ 0x48, 0x8c, 0x9c, 0x30, 0xd8, 0x18, 0xf5, 0xcd,
+ 0x85, 0x69, 0x71, 0x99, 0xe3, 0xb0, 0x23, 0x11,
+ 0x26, 0xf5, 0x84, 0x15, 0x3f, 0xb8, 0xfb, 0xf4,
+ 0xee, 0x9a, 0xc2, 0x79, 0xe6, 0x92, 0x6f, 0x4d,
+ 0x8a, 0x62, 0xcb, 0x38, 0x37, 0x44, 0x84, 0x09,
+ 0xc6, 0xf7, 0xa1, 0x48, 0xf1, 0xa1, 0xf7, 0x1b,
+ 0x87, 0x8e, 0xd0, 0xef, 0xa2, 0x1e, 0x32, 0x8e,
+ 0x9b, 0xdc, 0x0e, 0xa4, 0xf1, 0x94, 0x25, 0x1b,
+ 0x9b, 0x3c, 0x50, 0xdd, 0x41, 0x99, 0xd0, 0x96,
+ 0x60, 0xde, 0xbd, 0xed, 0x0b, 0x9a, 0xa4, 0xcb,
+ 0x2d, 0x2e, 0xd6, 0xdf, 0x97, 0xe0, 0xea, 0x87,
+ 0x12, 0xe1, 0x9b, 0x30, 0xe8, 0x6f, 0x01, 0x25,
+ 0x0c, 0xa4, 0xc1, 0xd4, 0x0e, 0xca, 0x07, 0x46,
+ 0xd9, 0x4b, 0x5d, 0x03, 0xcb, 0x8d, 0xcf, 0x55,
+ 0xe2, 0x62, 0x61, 0x10, 0x86, 0xf9, 0xf5, 0xe8,
+ 0xfe, 0x2f, 0xc5, 0xbc, 0x27, 0x2a, 0x1b, 0x1b,
+ 0x43, 0x4e, 0xb7, 0xd7, 0x18, 0x7b, 0xcf, 0x7a,
+ 0x5b, 0xa5, 0x18, 0x72, 0xa4, 0xa7, 0x24, 0x9f,
+ 0x6a, 0x12, 0xe7, 0x44, 0xb5, 0x45, 0x8c, 0xd5,
+ 0x53, 0xba, 0x2c, 0xaf, 0xaf, 0x3f, 0xe7, 0x12,
+ 0x45, 0xbd, 0x61, 0x68, 0xde, 0xc3, 0x38, 0xa8,
+ 0x5b, 0xbe, 0xc8, 0x7b, 0xab, 0xab, 0x21, 0x75,
+ 0x95, 0x8e, 0xb7, 0x98, 0xb8, 0x76, 0x69, 0xf8,
+ 0xf4, 0x70 },
+ .ilen = 250,
+ .result = { 0xd6, 0xfc, 0x21, 0x55, 0x85, 0x12, 0xa2, 0xd2,
+ 0x1c, 0x22, 0x96, 0xd6, 0xfe, 0x61, 0x24, 0x75,
+ 0x0c, 0xaa, 0x90, 0xe9, 0x0b, 0xe1, 0x72, 0xc8,
+ 0xb5, 0xcb, 0x17, 0x95, 0x9e, 0x30, 0x7c, 0x23,
+ 0x69, 0x5f, 0x7a, 0x71, 0x34, 0xac, 0x47, 0x18,
+ 0x0a, 0x9e, 0x39, 0x71, 0x2a, 0xff, 0x5a, 0xba,
+ 0x35, 0x1b, 0x60, 0xc2, 0xe8, 0x10, 0x2d, 0xbf,
+ 0x75, 0xf8, 0x2f, 0x2a, 0x0e, 0x01, 0x67, 0x4e,
+ 0xaf, 0xb5, 0xf3, 0x1c, 0xa1, 0x62, 0x7e, 0x22,
+ 0x2d, 0x17, 0x3c, 0x73, 0x03, 0x4f, 0xff, 0xb2,
+ 0x47, 0xae, 0x68, 0x63, 0x95, 0xcf, 0x66, 0x0e,
+ 0x22, 0xdf, 0x6c, 0x8b, 0xb4, 0xee, 0x3a, 0xb3,
+ 0x5d, 0x0d, 0xd3, 0x33, 0x16, 0x13, 0x45, 0x59,
+ 0x42, 0x74, 0x9f, 0xc5, 0xfb, 0xc6, 0x20, 0x88,
+ 0x33, 0x41, 0x55, 0x28, 0x61, 0x33, 0x9f, 0x80,
+ 0x57, 0x39, 0x26, 0xe4, 0x3b, 0x98, 0xcd, 0x6b,
+ 0xef, 0x10, 0x33, 0x20, 0xee, 0x18, 0x3a, 0x59,
+ 0x89, 0x4a, 0x1a, 0xc3, 0xdf, 0xb2, 0x5b, 0x0b,
+ 0x08, 0x1f, 0xaa, 0xaa, 0x8e, 0x54, 0xd3, 0x10,
+ 0xf7, 0xc2, 0xce, 0x68, 0x06, 0x06, 0x3e, 0x39,
+ 0x13, 0x0f, 0x43, 0xfd, 0x3b, 0x61, 0xc7, 0xfd,
+ 0x92, 0xf2, 0x0b, 0xe7, 0xa2, 0x60, 0xd8, 0xd8,
+ 0x00, 0xe1, 0x18, 0x77, 0x13, 0xb8, 0xa4, 0xa8,
+ 0xc0, 0x60, 0x55, 0x96, 0x2d, 0x6a, 0x69, 0x72,
+ 0x05, 0xa7, 0x27, 0xe5, 0x05, 0x20, 0x22, 0xcf,
+ 0xcf, 0xd5, 0x26, 0x70, 0x1d, 0x76, 0x97, 0xd6,
+ 0x93, 0x71, 0xd4, 0x0a, 0xe2, 0x5e, 0x47, 0x54,
+ 0x42, 0x6e, 0xb0, 0xea, 0xa9, 0x8b, 0x73, 0x2c,
+ 0xc7, 0xbb, 0x2d, 0x64, 0xfd, 0xcf, 0x23, 0x33,
+ 0xb1, 0x93, 0x0c, 0x32, 0x66, 0x1e, 0x26, 0x7a,
+ 0x8c, 0xb6, 0xfa, 0x85, 0x44, 0x35, 0x02, 0xce,
+ 0x05, 0xf6 }
+}, {
+ .key = { 0x56, 0x5b, 0x3a, 0xc9, 0xa6, 0x55, 0x34, 0xa2,
+ 0xb3, 0xaf, 0x33, 0xed, 0x9d, 0xed, 0x97, 0xfa,
+ 0x13, 0x1b, 0x06, 0x50, 0x09, 0x02, 0xa1, 0x93,
+ 0xe4, 0xa8, 0x3b, 0xd5, 0x8f, 0xc9, 0xa0, 0xc2 },
+ .nonce = 0x88663f30a2aa338ULL,
+ .input = { 0x9a, 0x2a, 0x04, 0xa0, 0x09, 0x55, 0x05, 0xfc,
+ 0x17, 0x80, 0x12, 0x6a, 0x78, 0x63, 0xe6, 0x55,
+ 0x13, 0xb7, 0x1e, 0x42, 0x67, 0x86, 0x01, 0x61,
+ 0x42, 0x74, 0x9a, 0x08, 0x46, 0xac, 0x69, 0x3a,
+ 0x10, 0x24, 0x18, 0x97, 0x3b, 0x85, 0xfb, 0x40,
+ 0x5a, 0x77, 0x5b, 0xf3, 0x6b, 0x77, 0xf0, 0x1e,
+ 0x41, 0xba, 0x38, 0x7d, 0xa1, 0x23, 0x9b, 0xd3,
+ 0x58, 0x59, 0x58, 0x34, 0xa9, 0x03, 0xb3, 0x43,
+ 0x9f, 0x5b, 0xe3, 0x1d, 0xf5, 0x9f, 0x8a, 0xdc,
+ 0x99, 0x99, 0x45, 0xb0, 0x72, 0xe0, 0xa0, 0x81,
+ 0xed, 0x7e, 0x2c, 0x21, 0x3d, 0xf2, 0xad, 0xfd,
+ 0xde, 0x9a, 0xf0, 0x40, 0xe7, 0xce, 0x1f, 0xc2,
+ 0x34, 0x2b, 0xf9, 0xc9, 0x52, 0x71, 0x0c, 0xa6,
+ 0x8a, 0x64, 0x90, 0xd0, 0xcf, 0x28, 0x8d, 0xc6,
+ 0x1c, 0x44, 0xd3, 0xdc, 0x24, 0x2a, 0x57, 0x79,
+ 0xc9, 0xa1, 0xb2, 0x38, 0x33, 0x20, 0xa2, 0xd0,
+ 0x59, 0x6e, 0xb9, 0x27, 0x06, 0xdb, 0xaf, 0x7d,
+ 0xed, 0x74, 0x8d, 0x34, 0xbc, 0x3f, 0x89, 0x42,
+ 0xe0, 0x28, 0x59, 0x2b, 0x8a, 0x3f, 0x42, 0xc5,
+ 0x69, 0x55, 0x1d, 0x8e, 0xa2, 0xba, 0xda, 0x01,
+ 0x68, 0x7f, 0x5d, 0xbd, 0x06, 0xa6, 0x95, 0xa7,
+ 0xad, 0x60, 0xf4, 0x5a, 0x43, 0x62, 0x8f, 0x69,
+ 0x9a, 0xc3, 0xd1, 0x89, 0xd3, 0xfe, 0xc8, 0x2d,
+ 0x4d, 0xdd, 0x1c, 0x07, 0xfa, 0x3f, 0xaa, 0x7e,
+ 0xfd, 0x90, 0xff, 0x6b, 0x35, 0x5e, 0x23, 0x0d,
+ 0x38, 0x16, 0x00, 0xc7, 0x00, 0xa0, 0xb2, 0x37,
+ 0x1e, 0x18, 0xdf, 0xa1, 0x18, 0xbc, 0x21, 0x88,
+ 0x80, 0x8f, 0x8c, 0xf8, 0x0c, 0x83, 0x69, 0xd6,
+ 0x89, 0x0e, 0xe9, 0xbc, 0x69, 0x11, 0x3b, 0x17,
+ 0xa1, 0x44, 0x4c, 0xf4, 0x02, 0x9e, 0x3a, 0x8f,
+ 0xaf, 0xbc, 0xcc, 0x4f, 0x46, 0x42, 0x7d, 0x63,
+ 0x9d, 0x95, 0x8d, 0xa7, 0x9d, 0xf9, 0x73, 0xc0 },
+ .ilen = 256,
+ .result = { 0x79, 0x75, 0xce, 0x01, 0x53, 0x9b, 0x03, 0x24,
+ 0x1b, 0x28, 0x10, 0x70, 0x98, 0xd8, 0x43, 0x4d,
+ 0x48, 0xba, 0xa2, 0xfb, 0x44, 0x08, 0xff, 0xac,
+ 0x59, 0x91, 0xb2, 0x4c, 0x8b, 0x76, 0x99, 0x79,
+ 0x56, 0x9b, 0x61, 0x69, 0x3c, 0x75, 0xef, 0xbf,
+ 0x4b, 0x22, 0x91, 0x58, 0xd9, 0x54, 0xaa, 0xba,
+ 0xe7, 0x20, 0xf5, 0xba, 0x4a, 0xe0, 0xbc, 0x23,
+ 0x3d, 0x2d, 0x39, 0x82, 0xad, 0x74, 0x01, 0x4d,
+ 0x2d, 0xd8, 0x42, 0x3d, 0x94, 0x42, 0x48, 0x31,
+ 0x40, 0x81, 0xc0, 0x24, 0xeb, 0xf8, 0xa3, 0xa0,
+ 0x51, 0x2e, 0x70, 0x7a, 0x6e, 0xfe, 0xa2, 0xae,
+ 0x49, 0x4b, 0x60, 0x76, 0x1c, 0x82, 0x5a, 0xe5,
+ 0xcd, 0x0e, 0x17, 0xd4, 0x0e, 0x50, 0x22, 0x7f,
+ 0xd9, 0xd8, 0x5e, 0x60, 0x45, 0xd7, 0x98, 0xa6,
+ 0xc1, 0x01, 0xc8, 0x45, 0x18, 0x37, 0x9f, 0xd1,
+ 0x26, 0x61, 0xd4, 0x74, 0x7f, 0x44, 0xd3, 0xad,
+ 0xf0, 0x20, 0xa1, 0x75, 0x8d, 0xaa, 0xf0, 0x26,
+ 0xad, 0xf7, 0x83, 0x45, 0xb5, 0x6f, 0xd0, 0xf9,
+ 0xa3, 0xa9, 0x7b, 0xdf, 0xfd, 0x90, 0x30, 0xa9,
+ 0x66, 0x25, 0x19, 0xd6, 0xcb, 0x16, 0xfb, 0xb1,
+ 0x8b, 0x33, 0x0b, 0x78, 0x8f, 0x40, 0xe0, 0x58,
+ 0x99, 0x3d, 0xd7, 0xfe, 0xd6, 0x2c, 0x84, 0xfd,
+ 0x74, 0x99, 0xea, 0x0c, 0x24, 0x18, 0x2a, 0x1c,
+ 0x46, 0xc0, 0xdc, 0xdf, 0xd0, 0x62, 0x03, 0x10,
+ 0x68, 0x37, 0xb1, 0x3f, 0x45, 0x97, 0xf3, 0xe5,
+ 0x2e, 0xd3, 0xe1, 0x9d, 0x84, 0x57, 0x1c, 0x71,
+ 0x3e, 0xb9, 0x5b, 0x42, 0xc4, 0xaa, 0x84, 0xf0,
+ 0x47, 0x42, 0x5d, 0xed, 0xde, 0x80, 0x3e, 0xde,
+ 0xed, 0xac, 0xeb, 0x3f, 0x13, 0xaa, 0x6f, 0x8a,
+ 0x10, 0xaa, 0x97, 0x64, 0x28, 0x06, 0x2c, 0xaf,
+ 0xfc, 0x71, 0x55, 0x45, 0x8b, 0xd1, 0x31, 0xc7,
+ 0x29, 0xdf, 0xd2, 0x93, 0x43, 0xad, 0xe9, 0x75 }
+}, {
+ .key = { 0x70, 0x11, 0x81, 0x4d, 0x0f, 0xf3, 0x69, 0xe9,
+ 0x16, 0x2e, 0x24, 0x94, 0x7a, 0x7b, 0xeb, 0x6c,
+ 0x8d, 0xfa, 0xc1, 0x89, 0xaf, 0x79, 0x67, 0xec,
+ 0xe3, 0x0c, 0x74, 0x3d, 0xeb, 0x0e, 0x7c, 0x52 },
+ .nonce = 0x8a3a75f2adc47f8fULL,
+ .input = { 0xfa, 0xc3, 0xca, 0xf1, 0xdd, 0x0c, 0xf3, 0xbb,
+ 0xbe, 0x51, 0x0f, 0xea, 0x1f, 0x09, 0xda, 0x68,
+ 0xfa, 0xb2, 0x0d, 0xa9, 0xc6, 0xb3, 0x2b, 0x0c,
+ 0x8a, 0xd6, 0x3f, 0xd6, 0x2e, 0x9d, 0x0e, 0x01,
+ 0xdf, 0x58, 0xf2, 0x02, 0x2d, 0x16, 0x54, 0x69,
+ 0x0e, 0xff, 0x23, 0x01, 0x84, 0x9c, 0xa7, 0xfc,
+ 0x3e, 0xd6, 0xb7, 0x32, 0x4e, 0xa9, 0x5d, 0xc1,
+ 0xb7, 0x62, 0x2d, 0x52, 0xda, 0x95, 0x17, 0x56,
+ 0xf2, 0xa7, 0x81, 0x16, 0x12, 0x5a, 0xab, 0x0b,
+ 0x65, 0x5e, 0xad, 0xe5, 0xbc, 0xee, 0xe9, 0xd7,
+ 0x03, 0xe3, 0x18, 0x50, 0xed, 0xcc, 0xef, 0xd8,
+ 0x59, 0xaa, 0x83, 0x01, 0xbc, 0x61, 0x9c, 0x5f,
+ 0xa3, 0x72, 0x4a, 0x47, 0xce, 0x14, 0x7c, 0x13,
+ 0xb9, 0xe9, 0xc3, 0xba, 0xb0, 0xc6, 0x4d, 0xb8,
+ 0xb2, 0x02, 0xc1, 0xbf, 0x6a, 0xb6, 0xcb, 0x65,
+ 0x05, 0xaa, 0xdb, 0x35, 0x4e, 0x78, 0x5c, 0xbd,
+ 0x9f, 0xef, 0x58, 0x75, 0x55, 0x49, 0x93, 0xb4,
+ 0xd0, 0x21, 0x44, 0x8e, 0x05, 0xb8, 0xbe, 0xa9,
+ 0x72, 0x33, 0x8e, 0x9b, 0xa3, 0xf7, 0x38, 0x97,
+ 0xf7, 0x25, 0xb6, 0x4b, 0x44, 0xbb, 0xc6, 0x45,
+ 0x55, 0xb9, 0x64, 0xbf, 0x43, 0xf6, 0x51, 0x0d,
+ 0xde, 0xa3, 0xc5, 0xda, 0x45, 0x03, 0x7c, 0x7b,
+ 0xf4, 0xd2, 0x1c, 0xce, 0x38, 0xa9, 0x0e, 0x2d,
+ 0x90, 0x80, 0x1d, 0x5d, 0xca, 0x18, 0xdc, 0x53,
+ 0x31, 0x42, 0xaa, 0x92, 0xfb, 0x59, 0xac, 0x8b,
+ 0x57, 0x25, 0xa8, 0xf0, 0xc1, 0x55, 0x4a, 0x16,
+ 0xdb, 0xa4, 0xf7, 0xfd, 0xe1, 0x49, 0x31, 0x6f,
+ 0xb5, 0x5c, 0x69, 0x9d, 0xc2, 0x98, 0x3e, 0x9c,
+ 0xf5, 0x51, 0x52, 0xdc, 0x2f, 0x0e, 0x90, 0x5f,
+ 0x41, 0x88, 0x3a, 0x69, 0x1c, 0x91, 0x69, 0x97,
+ 0xcb, 0xa5, 0x86, 0x52, 0xf7, 0x98, 0x64, 0x04,
+ 0x47, 0x60, 0x6f, 0x32, 0x3b, 0xe2, 0x7b, 0x33,
+ 0x0a, 0x79, 0x53, 0xad, 0x97, 0xa5, 0xc9, 0x1b,
+ 0x8b, 0x5d, 0xfd, 0x1e, 0x51, 0x36, 0x2a, 0x7e,
+ 0x0c, 0xdd, 0x0d, 0x0a, 0xc1, 0xf8, 0xb9, 0x77 },
+ .ilen = 280,
+ .result = { 0x9b, 0x42, 0x0b, 0x62, 0x0c, 0x90, 0x82, 0x01,
+ 0x6d, 0x87, 0x1d, 0x33, 0xb2, 0x9d, 0x54, 0x8b,
+ 0x1b, 0xc1, 0xe5, 0x3b, 0x50, 0xcf, 0x2a, 0x49,
+ 0x3d, 0xcf, 0x7b, 0x70, 0xd5, 0xa2, 0x6c, 0x9b,
+ 0x55, 0x72, 0x3b, 0x70, 0xbb, 0x99, 0xf3, 0xfd,
+ 0x34, 0x87, 0xa6, 0x72, 0x77, 0x56, 0x36, 0x05,
+ 0x5f, 0xde, 0x34, 0x9c, 0xb9, 0x26, 0x24, 0x6e,
+ 0x28, 0x64, 0x30, 0x99, 0xc2, 0xfd, 0xc1, 0x6f,
+ 0x27, 0x30, 0x8a, 0x1d, 0xf5, 0x68, 0xd1, 0x99,
+ 0xc8, 0x79, 0xb5, 0x40, 0xbc, 0xd2, 0x5d, 0xfc,
+ 0x6d, 0x57, 0x65, 0x35, 0x77, 0x9a, 0x1f, 0x3c,
+ 0xba, 0x73, 0x76, 0xf3, 0xdb, 0xe9, 0x18, 0x64,
+ 0xe6, 0x7e, 0xc9, 0xf3, 0xa6, 0xb2, 0xbf, 0xf0,
+ 0xa1, 0xcd, 0xd5, 0x03, 0xdd, 0x1e, 0x00, 0x07,
+ 0xba, 0x76, 0x7d, 0x46, 0x48, 0xf1, 0x7d, 0xb7,
+ 0x78, 0xd5, 0x9a, 0xdf, 0x80, 0x18, 0xd2, 0xad,
+ 0x97, 0xc7, 0xbd, 0xeb, 0x5a, 0xab, 0x17, 0x92,
+ 0x4b, 0x93, 0xd3, 0x39, 0x18, 0xef, 0xbb, 0xfe,
+ 0x89, 0x6d, 0x7d, 0x1d, 0xd3, 0x86, 0x4d, 0xfd,
+ 0x3f, 0x67, 0xfc, 0x9b, 0xfe, 0x4e, 0x9e, 0xe7,
+ 0xfb, 0x9f, 0x45, 0x8a, 0xd4, 0xd9, 0x0c, 0x30,
+ 0x90, 0xb8, 0xef, 0xc7, 0x18, 0x04, 0x09, 0x67,
+ 0xdf, 0x57, 0xe7, 0x30, 0x45, 0x91, 0xe3, 0xa6,
+ 0x73, 0x4a, 0x83, 0x6e, 0x92, 0x1e, 0xe5, 0xad,
+ 0x56, 0x1b, 0x86, 0x5c, 0xc3, 0x52, 0x58, 0x01,
+ 0xe7, 0xb9, 0x2e, 0xdc, 0x96, 0xee, 0x4c, 0x65,
+ 0x5a, 0xd7, 0x19, 0x6b, 0xfa, 0xa3, 0x0b, 0xf4,
+ 0xec, 0xab, 0xb5, 0xb2, 0xf8, 0xdd, 0x23, 0xff,
+ 0xe6, 0xa6, 0x55, 0x75, 0xcb, 0xb1, 0xa7, 0xed,
+ 0x6a, 0xa8, 0x60, 0x1a, 0x61, 0x64, 0x2a, 0xbc,
+ 0x73, 0xed, 0xa7, 0x54, 0x0d, 0x79, 0xc9, 0xc6,
+ 0x14, 0x6c, 0x78, 0xb1, 0x14, 0x10, 0x47, 0x80,
+ 0x76, 0xb7, 0x81, 0xac, 0x09, 0xf5, 0x12, 0x2d,
+ 0x21, 0x9c, 0x79, 0xd9, 0x37, 0xae, 0x2a, 0x51,
+ 0x41, 0x1e, 0x9e, 0x7f, 0x0d, 0x4b, 0x17, 0x87 }
+}, {
+ .key = { 0x23, 0xa3, 0x7e, 0x3a, 0xe4, 0x66, 0xb7, 0x86,
+ 0x7b, 0xdb, 0xf1, 0xdf, 0x76, 0x65, 0x4a, 0x70,
+ 0xc4, 0x93, 0x4d, 0xf4, 0xc1, 0x0e, 0xc1, 0x69,
+ 0xce, 0x43, 0xed, 0x9f, 0x48, 0x36, 0xa3, 0x03 },
+ .nonce = 0xbc7df621eb115af2ULL,
+ .input = { 0x11, 0x32, 0xa9, 0x18, 0x65, 0x15, 0x25, 0x2e,
+ 0x9a, 0xbb, 0x0c, 0x25, 0x96, 0x1e, 0xbf, 0x89,
+ 0x71, 0x7f, 0x18, 0x3a, 0x33, 0xf2, 0x9d, 0x85,
+ 0x23, 0x4d, 0x1f, 0x91, 0x44, 0x75, 0x08, 0xee,
+ 0xe0, 0x2d, 0xab, 0x1d, 0x8f, 0xab, 0x0e, 0x3f,
+ 0xe7, 0x3d, 0x43, 0x3d, 0x19, 0x49, 0xfc, 0xb2,
+ 0xf1, 0xc3, 0x53, 0x41, 0x0b, 0x35, 0xda, 0xd1,
+ 0x8c, 0x18, 0x9e, 0xb7, 0x1b, 0xa2, 0x5a, 0xfd,
+ 0x8a, 0xa4, 0xc2, 0xf4, 0x8d, 0x98, 0x01, 0x53,
+ 0xb8, 0x82, 0xf5, 0x53, 0x5c, 0xfe, 0x6d, 0x7d,
+ 0x63, 0x21, 0x46, 0x16, 0x29, 0x9a, 0xcf, 0x20,
+ 0xb3, 0xf7, 0xd2, 0x7f, 0xb4, 0x30, 0x90, 0xbd,
+ 0x95, 0xc2, 0xd2, 0xc0, 0x14, 0x84, 0xd3, 0x4a,
+ 0xf0, 0x12, 0xa6, 0xa8, 0xe8, 0x55, 0x97, 0xb2,
+ 0x0c, 0xee, 0x25, 0xf7, 0x94, 0xef, 0xae, 0xf7,
+ 0xbb, 0xe2, 0x64, 0x7d, 0xe2, 0xdc, 0x0f, 0x6b,
+ 0x7c, 0xa6, 0xe4, 0x71, 0x38, 0x44, 0x3d, 0xcd,
+ 0x0d, 0x94, 0xe9, 0x52, 0x6a, 0xb0, 0x8e, 0xd5,
+ 0x20, 0xe8, 0x60, 0x9c, 0xde, 0x92, 0xfc, 0x67,
+ 0x4e, 0xc3, 0x02, 0x6c, 0x6e, 0xd7, 0x56, 0x0a,
+ 0x00, 0x81, 0xc8, 0xd0, 0x26, 0x3c, 0x9a, 0x97,
+ 0x98, 0xfa, 0x10, 0x0e, 0xad, 0xeb, 0x7e, 0xdc,
+ 0x41, 0x58, 0x68, 0xf8, 0xa4, 0x70, 0x95, 0x2d,
+ 0x8f, 0x00, 0xd2, 0xcf, 0x9e, 0x57, 0x9e, 0xa7,
+ 0xe8, 0x11, 0x4c, 0xfc, 0x64, 0x51, 0x64, 0x11,
+ 0x85, 0xef, 0x47, 0xf5, 0xd3, 0x98, 0xda, 0x48,
+ 0x7e, 0x04, 0x55, 0x95, 0x4b, 0xc1, 0xc8, 0x42,
+ 0x66, 0x7c, 0xc9, 0x2c, 0xb6, 0xde, 0xc9, 0x3b,
+ 0x3d, 0x2b, 0x94, 0x88, 0xbf, 0x07, 0xd4, 0xe9,
+ 0x88, 0x61, 0x12, 0xb6, 0x3e, 0x2c, 0x14, 0x7f,
+ 0xea, 0x6d, 0x0a, 0xd9, 0x64, 0xf0, 0x3d, 0x57,
+ 0x82, 0x4a, 0xab, 0x57, 0x65, 0xba, 0x34, 0xca,
+ 0x22, 0x4d, 0xa8, 0xb5, 0xfe, 0xc2, 0x0a, 0xb4,
+ 0x0a, 0xb2, 0x05, 0x8d, 0x28, 0xc3, 0x5d, 0xee,
+ 0x40, 0xae, 0x48, 0x16, 0xeb, 0x60, 0x07, 0x59,
+ 0x53, 0x7d, 0x8d, 0xf6, 0x7c, 0x15, 0x32, 0x86,
+ 0x0c, 0x58, 0x86, 0x2c, 0xab, 0x92, 0xe8, 0xc8,
+ 0x7b, 0x97, 0x73, 0xd2, 0x11, 0x53, 0xed, 0xab,
+ 0x7d, 0xc2, 0xeb, 0x07, 0x5b, 0xcf, 0xc6, 0x5f,
+ 0xcb, 0xa1, 0x88, 0x9f, 0x4f, 0x74, 0xa9, 0x26,
+ 0xdc, 0x63, 0x7c, 0x5b, 0x60, 0x4b, 0x54, 0x02,
+ 0xa7, 0xaf, 0x7b, 0xf7, 0x34, 0x76, 0xd3, 0xbc,
+ 0x74, 0x51, 0x3c, 0x74, 0x57, 0xe4, 0x06, 0x22,
+ 0x25, 0xc3, 0x5f, 0x41, 0x40, 0x87, 0xf8, 0xf9,
+ 0xed, 0x26, 0x36, 0x7a, 0xb2, 0x5c, 0xde, 0xdb,
+ 0x85, 0x1b, 0x70, 0x87, 0xc8, 0x0e, 0x0e, 0x9c,
+ 0x1e, 0x8d, 0x86, 0x6a, 0x52, 0x3a, 0x64, 0x7c,
+ 0x50, 0xa3, 0xfe, 0x57, 0x48, 0x3a, 0xf7, 0x35,
+ 0x71, 0xea, 0x90, 0xd3, 0xc5, 0xf0, 0x9d, 0x08,
+ 0x94, 0x81, 0x6a, 0x90, 0xa6, 0xf1, 0x70, 0x53,
+ 0x3b, 0x46, 0x10, 0x5f, 0x2a, 0x90, 0x3c, 0xa7,
+ 0x06, 0x54, 0xa8, 0xd5, 0x7d, 0x8d, 0x3e, 0x59,
+ 0xaa, 0x19, 0xa3, 0x48, 0x40, 0xc3, 0xf4, 0xb4,
+ 0x70, 0x77, 0xed, 0x05, 0x0c, 0x41, 0x34, 0x53,
+ 0x7a, 0xd3, 0x1a, 0xc9, 0xbb, 0xee, 0xd3, 0xc3,
+ 0xcb, 0xd6, 0xe7, 0xed, 0x56, 0x17, 0x9d, 0x47,
+ 0x5f, 0x9a, 0xe6, 0x83, 0x41, 0x70, 0x1e, 0xf1,
+ 0x65, 0x98, 0xbf, 0xea, 0x00, 0x3b, 0xfb, 0x0e,
+ 0x91, 0x82, 0x30, 0xa0, 0xa5, 0xbb, 0xff, 0x3c,
+ 0x04, 0x52, 0x8f, 0xc2, 0x16, 0xa8, 0xe2, 0x78,
+ 0x55, 0xaf, 0x91, 0x20, 0xa1, 0xbe, 0xde, 0x90,
+ 0xb3, 0xeb, 0xf1, 0x18, 0x84, 0x25, 0xf0, 0xd4,
+ 0x42, 0x0f, 0x0b, 0xfc, 0x16, 0x47, 0x0c, 0x94,
+ 0x99, 0xeb, 0x96, 0xb5, 0x21, 0x76, 0x03 },
+ .ilen = 511,
+ .result = { 0x93, 0xf6, 0xa1, 0x01, 0xf0, 0x82, 0x18, 0x8d,
+ 0xf2, 0xd1, 0x29, 0x36, 0x6b, 0x1f, 0xcb, 0x7b,
+ 0xa2, 0xde, 0x37, 0x8f, 0x31, 0x46, 0x9b, 0x73,
+ 0xd4, 0x9e, 0x71, 0x7b, 0x49, 0x79, 0xc5, 0x2c,
+ 0x29, 0x1f, 0x60, 0xfd, 0x94, 0x9c, 0x00, 0xc0,
+ 0xe5, 0x49, 0xd3, 0x99, 0xe8, 0xb8, 0x7a, 0x9a,
+ 0x35, 0xb3, 0x58, 0xa1, 0x2c, 0x4b, 0xef, 0x7c,
+ 0x05, 0xf0, 0x07, 0x35, 0x4e, 0x02, 0x36, 0xae,
+ 0x29, 0xe4, 0x7c, 0x21, 0x35, 0xba, 0x77, 0xe0,
+ 0x80, 0x59, 0xd4, 0x19, 0xcd, 0xb0, 0xc6, 0xff,
+ 0x4f, 0x71, 0x91, 0x04, 0x90, 0x75, 0xe7, 0xea,
+ 0xd0, 0xc6, 0xa5, 0x10, 0xd2, 0x4b, 0x9d, 0x0f,
+ 0x3e, 0x08, 0x33, 0xf8, 0x7d, 0xe0, 0xcf, 0xb9,
+ 0x4e, 0xae, 0xd2, 0xe7, 0xe0, 0x2e, 0x98, 0x11,
+ 0xdc, 0xb9, 0x95, 0x25, 0xb2, 0x97, 0x16, 0x5b,
+ 0x75, 0x40, 0xa0, 0x6c, 0xcb, 0x67, 0xfb, 0x55,
+ 0x35, 0x20, 0xd1, 0xf0, 0x58, 0x3c, 0x67, 0xc3,
+ 0x77, 0x4e, 0xee, 0xaf, 0xbb, 0x5c, 0x15, 0x1b,
+ 0x05, 0xf8, 0xd3, 0x00, 0x8a, 0x33, 0x4c, 0x8f,
+ 0x01, 0x78, 0x00, 0xb0, 0x54, 0x9c, 0x63, 0x0c,
+ 0x09, 0x37, 0xf0, 0x0a, 0x4a, 0xae, 0x82, 0x2c,
+ 0x14, 0x99, 0xa2, 0x32, 0x3e, 0x6a, 0x90, 0x73,
+ 0x70, 0x6c, 0x01, 0xb1, 0x60, 0xf3, 0x15, 0xc6,
+ 0xc9, 0x05, 0x9a, 0xdc, 0xda, 0x69, 0x1b, 0x31,
+ 0xe7, 0xcf, 0x53, 0x40, 0x3f, 0x42, 0xdf, 0x35,
+ 0x38, 0x6a, 0xc2, 0x00, 0x11, 0xcb, 0x2c, 0x37,
+ 0x81, 0xdd, 0xa8, 0x27, 0xcb, 0x85, 0x89, 0x68,
+ 0x79, 0x46, 0x0c, 0xe3, 0x8e, 0xf4, 0x05, 0x9e,
+ 0xfc, 0xbd, 0x49, 0xbb, 0x4c, 0xe1, 0x05, 0xdf,
+ 0x8c, 0xca, 0xef, 0xbc, 0x3e, 0xb9, 0xa7, 0x02,
+ 0x0b, 0xc6, 0x91, 0xe4, 0x48, 0x9d, 0x3a, 0x58,
+ 0x4a, 0xf6, 0x49, 0xb0, 0xd9, 0xf0, 0xed, 0x39,
+ 0xe5, 0xf2, 0xe2, 0x47, 0xd0, 0x9e, 0xfa, 0xe1,
+ 0xb3, 0x7b, 0xb8, 0x1c, 0xef, 0x92, 0xf2, 0xf0,
+ 0xc0, 0xd5, 0xf1, 0x89, 0xee, 0xba, 0x52, 0xea,
+ 0x39, 0x08, 0x4c, 0xec, 0x92, 0x72, 0x0d, 0x92,
+ 0x3e, 0x57, 0x09, 0x5b, 0x4d, 0x6b, 0xbc, 0x31,
+ 0x7a, 0xcf, 0x28, 0x30, 0x68, 0x29, 0x3a, 0x99,
+ 0x84, 0xa8, 0xb7, 0x36, 0xc7, 0xa7, 0x76, 0x76,
+ 0x10, 0xd5, 0x6d, 0x35, 0xb0, 0xe0, 0x86, 0xa0,
+ 0x12, 0x19, 0x78, 0x0b, 0xbb, 0x7b, 0x03, 0xae,
+ 0x61, 0xf6, 0xe9, 0xb8, 0x7d, 0x7b, 0xf2, 0xb4,
+ 0x4e, 0x1c, 0x57, 0x37, 0xa8, 0xa6, 0x9c, 0x6a,
+ 0xc2, 0xae, 0xc2, 0xfa, 0x10, 0xd7, 0x77, 0x6f,
+ 0x80, 0x91, 0xfe, 0xa9, 0xd2, 0x49, 0xc6, 0x13,
+ 0xad, 0x18, 0x35, 0x94, 0x95, 0xe1, 0x5a, 0x18,
+ 0x0e, 0x31, 0x49, 0xa5, 0x31, 0xa9, 0xb1, 0xd2,
+ 0xf3, 0x36, 0x0a, 0x72, 0x46, 0x45, 0xc1, 0xd0,
+ 0x22, 0x6d, 0x3d, 0xe2, 0x73, 0x74, 0xad, 0x0a,
+ 0x0c, 0xfd, 0x1d, 0x67, 0xab, 0xcf, 0xe4, 0x4b,
+ 0x52, 0xa2, 0xba, 0x87, 0xe8, 0x6b, 0xc3, 0xf9,
+ 0xb2, 0xb0, 0x05, 0xf7, 0xe9, 0xac, 0x96, 0x3f,
+ 0xbb, 0xe5, 0xae, 0xdb, 0xd3, 0x4f, 0x88, 0x3c,
+ 0x29, 0x69, 0x90, 0xc6, 0x2e, 0x56, 0x89, 0x5d,
+ 0xda, 0x57, 0xe2, 0x3e, 0x21, 0x72, 0x50, 0x2d,
+ 0x65, 0xe9, 0xbd, 0x53, 0x4c, 0xe4, 0x3f, 0x75,
+ 0xc7, 0x46, 0x0c, 0xb7, 0x35, 0x25, 0x9b, 0x29,
+ 0x6e, 0xb5, 0x91, 0xa1, 0x37, 0xca, 0xa1, 0x26,
+ 0x09, 0x3f, 0x7e, 0xbd, 0x8f, 0xf0, 0x2f, 0x59,
+ 0x69, 0xa5, 0x74, 0xe2, 0x31, 0xe9, 0x04, 0x36,
+ 0x78, 0x14, 0xf0, 0x30, 0x39, 0x6d, 0xfb, 0xae,
+ 0xed, 0xb5, 0x23, 0x54, 0x28, 0x3d, 0x31, 0x23,
+ 0x49, 0x43, 0xab, 0xf2, 0xf8, 0xc4, 0xe7, 0x5f,
+ 0xef, 0x51, 0xa9, 0x25, 0xb9, 0x87, 0x4c }
+}, {
+ .key = { 0xa6, 0xa1, 0xb9, 0x92, 0xe4, 0x20, 0x25, 0xe9,
+ 0x7e, 0x51, 0x70, 0x50, 0xc9, 0xa3, 0x9c, 0x7f,
+ 0x48, 0x9f, 0xca, 0x2f, 0x16, 0x8a, 0xaf, 0xdb,
+ 0x25, 0xc0, 0xd3, 0x28, 0x4d, 0x19, 0xd4, 0x57 },
+ .nonce = 0xf81deedff5b091e0ULL,
+ .input = { 0x2a, 0xdb, 0xe1, 0x1d, 0x93, 0xd9, 0x63, 0x17,
+ 0x09, 0x06, 0xed, 0x92, 0x67, 0x1a, 0x35, 0x00,
+ 0x1a, 0x61, 0xc1, 0xce, 0xd1, 0xd6, 0x6b, 0x78,
+ 0x09, 0x32, 0xad, 0x36, 0xf8, 0x2b, 0xc7, 0x6f,
+ 0xc7, 0x3b, 0x3a, 0x73, 0x33, 0x00, 0xb0, 0x5c,
+ 0x38, 0x8c, 0x48, 0x82, 0x85, 0x09, 0x8a, 0x4a,
+ 0x0b, 0x32, 0x53, 0xed, 0x80, 0x66, 0x39, 0xa9,
+ 0xa9, 0xf7, 0x11, 0x3b, 0xac, 0x3a, 0xbc, 0xbe,
+ 0xda, 0x20, 0x83, 0xc4, 0x42, 0xcf, 0xf6, 0x21,
+ 0xb7, 0x70, 0x33, 0x5b, 0x6c, 0xcd, 0x20, 0xd0,
+ 0x48, 0x95, 0x36, 0x17, 0xd4, 0x24, 0x55, 0xc7,
+ 0x77, 0xc1, 0x32, 0xe2, 0xa5, 0xd2, 0xd4, 0xc1,
+ 0x6c, 0xe5, 0x77, 0xe4, 0x8c, 0xda, 0x20, 0x03,
+ 0x4b, 0x75, 0x97, 0x64, 0x10, 0x40, 0xf8, 0x5c,
+ 0x94, 0xf8, 0xf7, 0x0a, 0x7f, 0x43, 0x56, 0x9a,
+ 0xcb, 0xe2, 0x8d, 0xe4, 0x81, 0x4c, 0x6f, 0x89,
+ 0x4c, 0x0c, 0x02, 0xf6, 0x00, 0x89, 0x0e, 0xbf,
+ 0x3f, 0xd5, 0x73, 0x83, 0x7e, 0x42, 0x49, 0xf6,
+ 0xdd, 0x21, 0x2f, 0x46, 0x66, 0x07, 0x59, 0xc5,
+ 0x19, 0xee, 0x29, 0x80, 0xc9, 0x3e, 0xa0, 0xfc,
+ 0x78, 0x73, 0xcc, 0x38, 0xed, 0xe8, 0xc1, 0x95,
+ 0xd7, 0x9e, 0x02, 0x03, 0x77, 0x03, 0x19, 0x2a,
+ 0x0d, 0x0b, 0x50, 0x40, 0x64, 0x07, 0x91, 0x6f,
+ 0x82, 0x55, 0x1e, 0x8d, 0x9d, 0xe3, 0x83, 0x75,
+ 0x06, 0xba, 0x4f, 0x42, 0x7c, 0x73, 0x6b, 0x6c,
+ 0x77, 0x31, 0x57, 0xfb, 0x1f, 0x54, 0xe0, 0x05,
+ 0x22, 0x22, 0x05, 0x32, 0x6c, 0x56, 0x2e, 0xc7,
+ 0x7a, 0x8c, 0xc9, 0xbb, 0x7c, 0x89, 0x6a, 0xb4,
+ 0x2e, 0xbb, 0xad, 0xf6, 0xd2, 0xf2, 0x38, 0xf6,
+ 0xcb, 0xab, 0xcc, 0x57, 0x98, 0xd4, 0x5f, 0x99,
+ 0x4c, 0xa0, 0xe1, 0x62, 0x25, 0x9e, 0x31, 0x34,
+ 0xcc, 0xce, 0xd9, 0x79, 0x6a, 0x13, 0xb9, 0x1d,
+ 0x0b, 0x9b, 0x40, 0x73, 0x0f, 0x9e, 0xa5, 0xfd,
+ 0xae, 0x63, 0x3a, 0xa0, 0x52, 0x1e, 0x29, 0x65,
+ 0x0f, 0x62, 0xcc, 0xda, 0x7f, 0x3c, 0x06, 0x91,
+ 0xde, 0xea, 0x77, 0xe5, 0x88, 0x6d, 0x7d, 0x26,
+ 0xc4, 0xf3, 0xc3, 0xa0, 0xf5, 0xd6, 0x51, 0xda,
+ 0x55, 0xcd, 0x38, 0x6f, 0x71, 0x0f, 0x5b, 0x2e,
+ 0x3d, 0xab, 0x7c, 0xf1, 0xae, 0xd5, 0xb5, 0xef,
+ 0x21, 0x51, 0xb6, 0xb5, 0xe9, 0x91, 0xd0, 0xcd,
+ 0x30, 0x11, 0x6d, 0x62, 0x23, 0xfe, 0x5b, 0xda,
+ 0xce, 0xef, 0x82, 0x68, 0x7b, 0x3e, 0xd1, 0x7e,
+ 0x3b, 0xdf, 0x9f, 0xf6, 0xd1, 0x98, 0xc7, 0xfd,
+ 0xdd, 0xc5, 0xc7, 0xb4, 0x79, 0x91, 0x9e, 0x43,
+ 0x93, 0xf1, 0xdb, 0x13, 0xb3, 0x46, 0x6d, 0x69,
+ 0xa2, 0xce, 0xab, 0xf5, 0x23, 0xbc, 0xdf, 0x99,
+ 0x84, 0x5d, 0xba, 0x42, 0x84, 0xaf, 0xf6, 0x70,
+ 0xaf, 0x36, 0x5f, 0xc7, 0x32, 0x56, 0x95, 0x9f,
+ 0x4a, 0x81, 0x2f, 0x1f, 0xa7, 0x58, 0x9f, 0x98,
+ 0x9c, 0x1d, 0x61, 0x98, 0x32, 0xcb, 0xf6, 0xba,
+ 0x3e, 0xda, 0x5d, 0x66, 0x26, 0x9b, 0x5d, 0x03,
+ 0x6e, 0xa2, 0x66, 0xe8, 0x9e, 0x14, 0x23, 0xae,
+ 0x02, 0x24, 0xcd, 0xc5, 0xa7, 0x81, 0x02, 0x0f,
+ 0x7b, 0xf5, 0x0c, 0x6e, 0x0b, 0x88, 0x12, 0x94,
+ 0xaa, 0x86, 0xff, 0x02, 0xef, 0xaa, 0x36, 0xa5,
+ 0x1c, 0x12, 0x23, 0x61, 0x66, 0xfe, 0x05, 0xc1,
+ 0x73, 0x43, 0x4e, 0x2b, 0xc9, 0xda, 0xbb, 0x2f,
+ 0x67, 0x50, 0x0a, 0x1f, 0xd2, 0x65, 0x8c, 0x75,
+ 0x9c, 0xf6, 0xcb, 0xab, 0x33, 0xea, 0xa2, 0x20,
+ 0x58, 0x95, 0x05, 0x9f, 0x76, 0xa4, 0x67, 0x68,
+ 0xc0, 0x77, 0x57, 0x0b, 0xe9, 0x9e, 0x93, 0x72,
+ 0x22, 0x77, 0x9f, 0x8d, 0xf9, 0x2c, 0xe5, 0xa0,
+ 0x32, 0x6b, 0xe1, 0xb5, 0x60, 0xa7, 0xa0, 0x7a,
+ 0xe2, 0x1e, 0x5c, 0x74, 0x1a, 0x1c, 0x69, 0xc1 },
+ .ilen = 512,
+ .result = { 0xf4, 0x3e, 0xcf, 0xe1, 0xf7, 0x21, 0xb2, 0x55,
+ 0xc9, 0xf5, 0xd3, 0x54, 0x40, 0xd0, 0x53, 0x28,
+ 0x62, 0xfd, 0xb6, 0x7b, 0x84, 0x51, 0xfd, 0x4d,
+ 0x86, 0x95, 0x9a, 0xfe, 0xcf, 0x10, 0xe1, 0xec,
+ 0x9f, 0x28, 0x2e, 0x24, 0x0f, 0xd4, 0x27, 0x12,
+ 0x7a, 0x36, 0xd4, 0x75, 0x52, 0xa3, 0x1b, 0xb6,
+ 0x18, 0xe4, 0x2a, 0x35, 0xd2, 0x30, 0xf3, 0xc7,
+ 0xdd, 0xf9, 0x3a, 0xfa, 0x90, 0xfd, 0xc3, 0xe6,
+ 0xe5, 0xd4, 0x0c, 0xab, 0xff, 0xd1, 0x9d, 0xfc,
+ 0x7c, 0x50, 0xcc, 0x22, 0x68, 0xd0, 0x81, 0xff,
+ 0x41, 0x7d, 0x28, 0x75, 0xd0, 0x22, 0xa9, 0x45,
+ 0x24, 0x53, 0xf7, 0xc6, 0xed, 0x43, 0x74, 0x14,
+ 0xa8, 0xbf, 0x19, 0x22, 0xfd, 0x46, 0x2e, 0x8d,
+ 0x96, 0xd2, 0x5e, 0x2f, 0xea, 0x52, 0x29, 0x61,
+ 0x48, 0xd6, 0x57, 0xa7, 0x78, 0xaf, 0xdb, 0x0b,
+ 0x21, 0x62, 0x12, 0x7c, 0xaa, 0x40, 0xff, 0xdc,
+ 0xdc, 0x8f, 0x21, 0x73, 0x08, 0x6a, 0x05, 0xdf,
+ 0x3b, 0xd3, 0xd5, 0xdf, 0x63, 0xa7, 0xaa, 0xff,
+ 0x67, 0xb2, 0xd7, 0xff, 0x4f, 0xba, 0xf4, 0x35,
+ 0xef, 0x3f, 0xb2, 0x84, 0xe2, 0x04, 0xd1, 0xb2,
+ 0x6c, 0x53, 0x5e, 0xb7, 0xdb, 0x78, 0x3f, 0x54,
+ 0x63, 0x77, 0xf4, 0xda, 0x50, 0xf4, 0x21, 0xb8,
+ 0xb4, 0xf1, 0x0b, 0x97, 0x23, 0x7b, 0x5f, 0x84,
+ 0xea, 0x20, 0xf2, 0xa2, 0xc4, 0xb8, 0x59, 0xb3,
+ 0x50, 0x67, 0xf7, 0xaf, 0xbe, 0xf1, 0x8c, 0xa0,
+ 0x37, 0x44, 0x8a, 0xd0, 0x70, 0x77, 0x22, 0x24,
+ 0xd2, 0xd6, 0xa1, 0x3a, 0x74, 0x61, 0xa7, 0x38,
+ 0x5a, 0xc4, 0xdc, 0x42, 0x6f, 0xdc, 0x6e, 0xc8,
+ 0xc7, 0x82, 0x0b, 0x16, 0x72, 0xc5, 0x5f, 0xd4,
+ 0x1f, 0x63, 0x13, 0x25, 0xb6, 0xa5, 0x0e, 0xef,
+ 0xa0, 0x75, 0xa4, 0x4e, 0x68, 0xaf, 0x9b, 0x6d,
+ 0x6f, 0xc9, 0x84, 0x5e, 0xc6, 0x64, 0x50, 0x70,
+ 0x81, 0x94, 0xd6, 0x41, 0x6b, 0x17, 0x35, 0x2c,
+ 0x15, 0xd8, 0xf0, 0x42, 0x7a, 0x20, 0x42, 0xca,
+ 0xe0, 0x5a, 0x24, 0xbc, 0xff, 0x90, 0x9e, 0x1d,
+ 0x05, 0x3d, 0x16, 0x35, 0x7d, 0xdc, 0xc6, 0xe2,
+ 0x0c, 0xf0, 0xa7, 0xe9, 0xa5, 0x09, 0x28, 0x6c,
+ 0xd9, 0x80, 0x80, 0xe8, 0xca, 0x48, 0x26, 0x68,
+ 0xb6, 0x84, 0xcb, 0x87, 0x76, 0x13, 0x51, 0x25,
+ 0x47, 0xc4, 0x8a, 0xcd, 0x8d, 0x1a, 0x07, 0x14,
+ 0x5a, 0xa7, 0x1a, 0xbb, 0x86, 0xe6, 0x76, 0xa0,
+ 0x29, 0x50, 0xae, 0xcb, 0x52, 0xfe, 0x0e, 0x27,
+ 0xd6, 0x48, 0x40, 0xfb, 0x3e, 0xb3, 0xdb, 0x67,
+ 0x2a, 0x3a, 0x9e, 0x62, 0xfc, 0x57, 0x5a, 0x0b,
+ 0x2c, 0xe5, 0x6d, 0x3d, 0x33, 0x55, 0xfc, 0xad,
+ 0x5d, 0x96, 0x68, 0xd9, 0xea, 0x92, 0xc1, 0x92,
+ 0x40, 0x3e, 0x72, 0xe6, 0xcc, 0x51, 0xc0, 0x40,
+ 0x70, 0x39, 0x61, 0x8a, 0xcd, 0xef, 0xde, 0x1d,
+ 0x44, 0x7d, 0x86, 0x66, 0x9f, 0xaa, 0x22, 0x9e,
+ 0xb9, 0xd1, 0x02, 0x05, 0x6b, 0xbd, 0xda, 0x29,
+ 0x60, 0x45, 0x77, 0x35, 0xeb, 0xae, 0x8b, 0xd8,
+ 0x79, 0x95, 0x8d, 0x1c, 0x3d, 0xe6, 0xf1, 0x37,
+ 0x70, 0x9a, 0x63, 0xab, 0xdb, 0x21, 0x4d, 0x7b,
+ 0x74, 0x00, 0x02, 0x0c, 0x72, 0x9d, 0x04, 0x39,
+ 0x39, 0xfc, 0x9c, 0x7d, 0x9b, 0x42, 0xf0, 0x94,
+ 0x72, 0x1e, 0x88, 0x58, 0xbf, 0xc9, 0x28, 0x9f,
+ 0xa0, 0x77, 0x65, 0x2e, 0x9d, 0xe9, 0x00, 0x33,
+ 0xc2, 0xd1, 0x83, 0x2a, 0x27, 0xb1, 0xa4, 0x69,
+ 0xc8, 0x61, 0x99, 0x09, 0xe1, 0xb3, 0xdf, 0x8a,
+ 0x11, 0xd4, 0xd4, 0x26, 0xc7, 0x31, 0x83, 0x67,
+ 0x1f, 0xd5, 0x00, 0x39, 0x49, 0xf1, 0xa6, 0xaf,
+ 0xbc, 0x6e, 0xe9, 0xc5, 0xe0, 0xa7, 0xee, 0x0c,
+ 0x7c, 0x9e, 0x08, 0x7f, 0x47, 0x9f, 0xed, 0x85,
+ 0x20, 0x23, 0x14, 0x0b, 0x3e, 0x58, 0xc3, 0x59 }
+}, {
+ .key = { 0x24, 0x30, 0xaa, 0x5c, 0xad, 0xee, 0xc9, 0xe5,
+ 0x15, 0x8d, 0x04, 0xd4, 0x1e, 0xd1, 0xac, 0xd5,
+ 0xa2, 0xc2, 0xd9, 0xd2, 0x55, 0xd9, 0x34, 0x09,
+ 0x44, 0xf4, 0x29, 0x1c, 0x85, 0xd1, 0x5f, 0xeb },
+ .nonce = 0xb9f185373d1a89baULL,
+ .input = { 0xe5, 0x1a, 0xcf, 0x39, 0x07, 0x21, 0x2b, 0x01,
+ 0x7a, 0x9a, 0x13, 0x54, 0x57, 0x8e, 0x45, 0x64,
+ 0xa5, 0xb4, 0x0f, 0x17, 0x72, 0x31, 0x92, 0x6b,
+ 0x18, 0x5e, 0x79, 0x51, 0x51, 0x56, 0xe5, 0x34,
+ 0x6d, 0x70, 0x34, 0x21, 0x00, 0x09, 0x97, 0x3f,
+ 0x9a, 0x56, 0xed, 0x10, 0x45, 0x20, 0xb5, 0x99,
+ 0x53, 0x09, 0xad, 0x22, 0xc0, 0xbb, 0x13, 0x1f,
+ 0xeb, 0xe1, 0xa2, 0x31, 0x9e, 0xab, 0xa8, 0x3c,
+ 0xf5, 0xc4, 0xa6, 0xa0, 0xb2, 0xde, 0xd3, 0x58,
+ 0x3a, 0x46, 0x88, 0x74, 0x0a, 0x8f, 0x1b, 0x67,
+ 0x55, 0x8a, 0xe0, 0xd1, 0x32, 0x9e, 0x07, 0x32,
+ 0x87, 0x4f, 0x04, 0xc1, 0xaf, 0xcb, 0x82, 0xdb,
+ 0x84, 0x15, 0x52, 0xdc, 0x31, 0xfe, 0x99, 0xee,
+ 0x4c, 0x8a, 0xd9, 0x2b, 0x2e, 0xe5, 0xaf, 0x4f,
+ 0x95, 0x4b, 0x46, 0x2b, 0x28, 0xe9, 0xd9, 0xf8,
+ 0x04, 0xc3, 0xbd, 0x8c, 0xb3, 0x2b, 0x50, 0x7d,
+ 0x2a, 0x3a, 0x9d, 0xfd, 0xca, 0x79, 0x91, 0x98,
+ 0xc8, 0xe4, 0xdb, 0xe0, 0xe3, 0x51, 0x67, 0xc8,
+ 0x99, 0x8e, 0x26, 0x63, 0xaa, 0xf5, 0x51, 0x71,
+ 0x90, 0xa2, 0x09, 0x57, 0x6d, 0x30, 0xfe, 0xa1,
+ 0x8e, 0xd6, 0x20, 0x32, 0xbb, 0x84, 0x50, 0xc6,
+ 0xde, 0xc7, 0x7e, 0xd0, 0x5c, 0xd0, 0x31, 0x93,
+ 0x1a, 0x09, 0x18, 0x90, 0x66, 0xc7, 0xc2, 0x05,
+ 0x25, 0xb9, 0x2f, 0x8c, 0xd6, 0x38, 0x11, 0x10,
+ 0x4e, 0x71, 0x26, 0x41, 0x7a, 0x27, 0x6a, 0x55,
+ 0xfc, 0x58, 0x2a, 0x67, 0xd6, 0x1f, 0x9f, 0x7f,
+ 0x57, 0x7c, 0xda, 0x7d, 0xa7, 0x78, 0x84, 0xdc,
+ 0x7a, 0xd7, 0x4d, 0xdc, 0xb6, 0x70, 0x30, 0xea,
+ 0xb9, 0xff, 0x49, 0x00, 0x66, 0x15, 0x22, 0x9e,
+ 0x97, 0x53, 0xd8, 0x16, 0x8c, 0xe1, 0x58, 0xb8,
+ 0x6e, 0x04, 0xe0, 0x79, 0xf9, 0xf4, 0x2f, 0x75,
+ 0x46, 0x06, 0xcf, 0x5b, 0x78, 0xd6, 0xc9, 0xd8,
+ 0xa2, 0xfd, 0x1b, 0x9b, 0x9a, 0x82, 0x16, 0x34,
+ 0xc6, 0x9c, 0x03, 0x95, 0x89, 0xc2, 0x7f, 0xe7,
+ 0x1e, 0xb4, 0x45, 0x43, 0xea, 0xa5, 0x91, 0x3d,
+ 0x26, 0xfe, 0x8c, 0xa8, 0x0f, 0x8a, 0xb8, 0x13,
+ 0x8e, 0x24, 0xc3, 0x75, 0xa5, 0x9f, 0xb3, 0x65,
+ 0x0c, 0x3e, 0x7a, 0x69, 0x5c, 0x01, 0x2c, 0x74,
+ 0xb7, 0x23, 0x1b, 0x1a, 0x07, 0x61, 0x57, 0x39,
+ 0xd7, 0x8d, 0xdd, 0x15, 0x68, 0x1d, 0xdd, 0x4f,
+ 0x3a, 0xc6, 0x48, 0x71, 0x3f, 0x1e, 0x69, 0xb4,
+ 0xb5, 0xfc, 0x4f, 0x25, 0x4a, 0xbe, 0x3f, 0x1d,
+ 0xed, 0x02, 0x7b, 0x82, 0x43, 0x9c, 0x4c, 0xee,
+ 0x5d, 0x16, 0x6e, 0xb6, 0xb3, 0xaa, 0x8f, 0x6b,
+ 0x94, 0x98, 0xe9, 0x83, 0x71, 0xa6, 0x4f, 0xc9,
+ 0xcc, 0x0e, 0xf2, 0x53, 0x85, 0x49, 0xea, 0xda,
+ 0x43, 0x5c, 0xdc, 0xf4, 0x63, 0x9e, 0xa3, 0x98,
+ 0x1d, 0x17, 0x10, 0x43, 0xe0, 0xea, 0x30, 0x34,
+ 0x75, 0x77, 0xc2, 0x3c, 0x56, 0xdc, 0x0c, 0x1b,
+ 0x57, 0xfe, 0x73, 0x02, 0xe9, 0x3e, 0x1a, 0xe3,
+ 0x44, 0xdd, 0xa1, 0x70, 0xed, 0xd9, 0x8c, 0x69,
+ 0xdb, 0xbb, 0xc9, 0x7d, 0x96, 0x38, 0x8b, 0x3a,
+ 0x7b, 0x99, 0x25, 0xaa, 0xab, 0xe9, 0x10, 0x2a,
+ 0xa7, 0xdb, 0xad, 0x0a, 0x33, 0xdc, 0x1d, 0x0a,
+ 0xc4, 0x63, 0x6b, 0x0b, 0x58, 0x67, 0xb5, 0x7a,
+ 0x93, 0xee, 0x5f, 0x74, 0xac, 0xa8, 0x50, 0x29,
+ 0x03, 0x91, 0xf7, 0x10, 0xb6, 0xbe, 0xc7, 0xd1,
+ 0xef, 0xc9, 0xa2, 0x61, 0x89, 0x6f, 0x5e, 0x04,
+ 0xb0, 0xfb, 0x46, 0x9e, 0x18, 0xe7, 0x52, 0x38,
+ 0xdd, 0x64, 0x76, 0xd7, 0x4f, 0xbb, 0xaa, 0x20,
+ 0x36, 0x8d, 0x86, 0xf4, 0x5d, 0x8b, 0x52, 0x66,
+ 0x5c, 0x17, 0x98, 0xc0, 0xc5, 0x5b, 0xc8, 0x00,
+ 0xa4, 0x99, 0x3d, 0x0f, 0x0b, 0x5b, 0x41, 0x96,
+ 0x48, 0xf3, 0xa5, 0x8c, 0x8e, 0xbb, 0x37, 0x3e,
+ 0x13 },
+ .ilen = 513,
+ .result = { 0x60, 0x64, 0xf1, 0xf3, 0x13, 0x5c, 0xf5, 0xd4,
+ 0xdc, 0xb9, 0x7b, 0xf8, 0x85, 0xf8, 0xac, 0x46,
+ 0x8c, 0x3b, 0x74, 0xa3, 0x6d, 0x15, 0x36, 0x01,
+ 0x42, 0x62, 0xfb, 0xef, 0x54, 0x37, 0x92, 0x4f,
+ 0xb6, 0x1a, 0x62, 0xa1, 0x0a, 0xb0, 0xf0, 0x52,
+ 0x26, 0x0d, 0xd5, 0x0e, 0x4e, 0x98, 0x1a, 0xd8,
+ 0x82, 0x60, 0x8b, 0x50, 0x50, 0xfd, 0xbb, 0xc8,
+ 0x90, 0xb7, 0x41, 0x26, 0x7f, 0x28, 0x4c, 0x42,
+ 0x50, 0xc1, 0x5c, 0x93, 0x66, 0x0c, 0xbc, 0x73,
+ 0x46, 0x59, 0xc3, 0x7d, 0x16, 0xfd, 0x72, 0x6d,
+ 0x9b, 0x09, 0x1e, 0xd2, 0xa7, 0xca, 0x4f, 0x96,
+ 0xca, 0x58, 0x12, 0x75, 0xdb, 0xbf, 0x1b, 0xfe,
+ 0xa3, 0x7d, 0x17, 0xf5, 0x93, 0x52, 0x0d, 0xbe,
+ 0xc3, 0x6b, 0x06, 0x82, 0x08, 0xec, 0x02, 0xa6,
+ 0x9c, 0x54, 0x74, 0x2f, 0x3f, 0x6e, 0x9e, 0x37,
+ 0x5f, 0x35, 0xd9, 0xd5, 0x39, 0x88, 0xd6, 0x0e,
+ 0xd9, 0x64, 0x17, 0xf2, 0xb2, 0x66, 0x08, 0xc2,
+ 0x0b, 0xa5, 0xd4, 0xc2, 0x01, 0xfa, 0x73, 0x4d,
+ 0x69, 0x48, 0xa0, 0x9d, 0x2b, 0xde, 0x2d, 0xbd,
+ 0xb7, 0xec, 0xc7, 0x55, 0x01, 0x4e, 0xfe, 0x0a,
+ 0xa8, 0xee, 0xb1, 0x5c, 0xad, 0x90, 0x3b, 0x94,
+ 0x41, 0x3b, 0xa3, 0x24, 0xe5, 0xc8, 0x5c, 0xb9,
+ 0x5f, 0x5e, 0x8e, 0x9c, 0x87, 0x57, 0xea, 0xb9,
+ 0x93, 0xbb, 0x68, 0xc1, 0xad, 0xc3, 0xcc, 0xdc,
+ 0x54, 0xc3, 0xab, 0xe8, 0xee, 0x63, 0xe3, 0x78,
+ 0x47, 0x52, 0x36, 0x7e, 0xf0, 0x04, 0xd7, 0xcd,
+ 0x23, 0xa9, 0xd6, 0x7b, 0x6b, 0x39, 0xe6, 0xf1,
+ 0xf1, 0xf4, 0xe6, 0x43, 0x43, 0x31, 0x2f, 0xe6,
+ 0x15, 0x24, 0x0e, 0x02, 0x56, 0x3a, 0xa2, 0x1f,
+ 0x82, 0xf6, 0xd1, 0x23, 0x96, 0x09, 0xf8, 0x7a,
+ 0x55, 0x66, 0x39, 0x4c, 0x5e, 0x77, 0xff, 0x20,
+ 0xf5, 0xef, 0x9e, 0x34, 0x24, 0xbc, 0x33, 0x31,
+ 0x27, 0xb5, 0x0b, 0x20, 0x3e, 0x78, 0x31, 0xa3,
+ 0x48, 0x9d, 0x51, 0xfe, 0x60, 0xc6, 0xb2, 0xec,
+ 0x76, 0x6b, 0x6c, 0xde, 0xfc, 0x53, 0x20, 0xa3,
+ 0x83, 0x0f, 0x7b, 0x82, 0xd9, 0x01, 0xf4, 0x03,
+ 0xba, 0x90, 0x57, 0x72, 0x8e, 0x7b, 0xa1, 0x17,
+ 0xed, 0x61, 0xe9, 0x57, 0x20, 0x37, 0x18, 0xc2,
+ 0x99, 0xa8, 0x62, 0x26, 0x9d, 0x7d, 0x6b, 0x4f,
+ 0xc1, 0xe5, 0x92, 0x77, 0x3f, 0x44, 0xfd, 0xcd,
+ 0xa4, 0x1d, 0x59, 0x51, 0xac, 0x8b, 0x76, 0xc5,
+ 0x64, 0xc7, 0x4d, 0xf6, 0x84, 0x42, 0xc5, 0x73,
+ 0x4d, 0xe6, 0x6f, 0xa7, 0xda, 0x15, 0x43, 0x32,
+ 0xd0, 0x2e, 0x5a, 0xee, 0x36, 0x42, 0x44, 0x77,
+ 0x92, 0xa9, 0xc0, 0xdc, 0xa9, 0x23, 0x59, 0xce,
+ 0x51, 0xf3, 0x67, 0x7a, 0xca, 0x06, 0xed, 0xc6,
+ 0xaf, 0x29, 0x0e, 0xa5, 0xe6, 0x3b, 0xeb, 0x1e,
+ 0x53, 0x48, 0x1a, 0x58, 0x13, 0x01, 0x51, 0x7f,
+ 0xb6, 0x54, 0x8a, 0xab, 0x02, 0x48, 0x73, 0xac,
+ 0x24, 0x6b, 0x5e, 0x5a, 0x2e, 0xf6, 0x63, 0x50,
+ 0xb6, 0x32, 0x0e, 0xcf, 0x93, 0xa1, 0x48, 0x7a,
+ 0xdb, 0x65, 0xab, 0x2e, 0x8a, 0x22, 0x96, 0x71,
+ 0x5a, 0xb9, 0xf7, 0xe4, 0x99, 0x25, 0xa4, 0x80,
+ 0xb5, 0x62, 0xe5, 0x4d, 0x2f, 0xcb, 0xce, 0x73,
+ 0x61, 0x8f, 0x00, 0xa3, 0xd5, 0x40, 0xec, 0xbb,
+ 0x4a, 0xa3, 0xe3, 0x95, 0x20, 0x8b, 0x6a, 0xd6,
+ 0x89, 0xa0, 0x91, 0x42, 0xb4, 0xe6, 0x4e, 0x49,
+ 0xfa, 0x73, 0xab, 0x1e, 0x22, 0x20, 0x30, 0x5e,
+ 0xe1, 0x94, 0x62, 0x48, 0x4d, 0x16, 0x4f, 0xea,
+ 0x44, 0x4c, 0x19, 0xea, 0x00, 0x29, 0x5c, 0xce,
+ 0x99, 0x32, 0xc0, 0xb3, 0x51, 0xe8, 0x39, 0x7b,
+ 0x1c, 0x8f, 0x24, 0xe3, 0x01, 0x5e, 0x65, 0xb2,
+ 0x5e, 0x80, 0x15, 0xdf, 0x17, 0x11, 0x1b, 0x40,
+ 0xe6, 0xdb, 0xe3, 0x60, 0x7b, 0x2f, 0xa8, 0xa7,
+ 0xb6 }
+}, {
+ .key = { 0xb2, 0xd2, 0x69, 0x85, 0x45, 0xb5, 0x4c, 0x3c,
+ 0xba, 0xb1, 0x01, 0xd4, 0x3d, 0xbe, 0x75, 0x91,
+ 0x98, 0x4c, 0x7b, 0x1d, 0xb1, 0xb0, 0x52, 0xf1,
+ 0xdd, 0xc3, 0x58, 0x1f, 0x2e, 0x9a, 0x72, 0x4f },
+ .nonce = 0x9aba2243ec59aca0ULL,
+ .input = { 0x6b, 0xf4, 0xfa, 0xf5, 0x28, 0x04, 0xfa, 0xd3,
+ 0x03, 0xab, 0x62, 0x69, 0x9f, 0x5e, 0x14, 0x83,
+ 0xe9, 0xf2, 0xe5, 0x98, 0xd9, 0x7e, 0x05, 0x22,
+ 0xf1, 0xf7, 0x0f, 0x76, 0x4a, 0x33, 0x9d, 0x49,
+ 0xa2, 0x16, 0xcf, 0x70, 0xd2, 0x94, 0x7e, 0xe5,
+ 0xcc, 0xe3, 0x01, 0x4d, 0x21, 0x14, 0x98, 0xef,
+ 0xbf, 0x18, 0x0e, 0x07, 0x00, 0x8a, 0x82, 0xcb,
+ 0x12, 0x65, 0xab, 0xd5, 0x2a, 0xe2, 0xe6, 0x07,
+ 0xe3, 0x3a, 0x78, 0x22, 0xb0, 0xd8, 0xce, 0x96,
+ 0xde, 0x47, 0x6f, 0xa8, 0xfb, 0x44, 0x00, 0x7b,
+ 0x33, 0x4e, 0xb4, 0x5d, 0x99, 0x9e, 0x14, 0x49,
+ 0xca, 0x90, 0x80, 0x11, 0xac, 0xa9, 0x9b, 0x6d,
+ 0x01, 0x24, 0x9b, 0x05, 0x6d, 0x57, 0x35, 0x2a,
+ 0xde, 0xd2, 0xb6, 0x54, 0x93, 0x55, 0x08, 0xef,
+ 0xfb, 0x19, 0xd0, 0x3e, 0xf2, 0x95, 0xda, 0x75,
+ 0x0e, 0x27, 0x38, 0x14, 0xcf, 0xd2, 0x4e, 0xb7,
+ 0x27, 0xd1, 0xbd, 0x52, 0x08, 0x7e, 0x6c, 0xe4,
+ 0xfd, 0xc5, 0xc3, 0x2a, 0x76, 0x84, 0xf8, 0x4a,
+ 0x4e, 0x10, 0x90, 0xb0, 0x0c, 0x32, 0x25, 0x9f,
+ 0x5f, 0x75, 0x2a, 0x24, 0x57, 0x45, 0xd5, 0x64,
+ 0x32, 0xf7, 0x30, 0x5a, 0x13, 0xbd, 0x79, 0x1c,
+ 0x50, 0x46, 0xf7, 0xbd, 0xdc, 0x32, 0x3c, 0x43,
+ 0x1c, 0x84, 0xe6, 0x73, 0x78, 0x01, 0x94, 0xe2,
+ 0x49, 0x34, 0x32, 0xd9, 0x86, 0x2c, 0xff, 0xa6,
+ 0x6a, 0x46, 0x9c, 0xfc, 0xf3, 0x22, 0x31, 0xef,
+ 0x60, 0xb8, 0xdc, 0x2e, 0xcc, 0x0e, 0x8c, 0x3a,
+ 0xbb, 0xba, 0xb0, 0x36, 0xa0, 0x97, 0x78, 0x09,
+ 0x3a, 0x87, 0xa5, 0xef, 0xc6, 0x50, 0x87, 0x02,
+ 0xfa, 0xb8, 0x3e, 0x55, 0x2d, 0x78, 0xc8, 0xa8,
+ 0x8a, 0xa6, 0x63, 0xba, 0x31, 0x72, 0x56, 0x39,
+ 0x7d, 0xb8, 0x4a, 0xe4, 0x1c, 0xe9, 0x7b, 0x8c,
+ 0x97, 0x25, 0x3f, 0x25, 0xea, 0x8a, 0x6e, 0x2a,
+ 0x09, 0x83, 0x17, 0x12, 0x9b, 0x3b, 0xb8, 0xe6,
+ 0xf8, 0x0f, 0x48, 0x68, 0x9a, 0xcf, 0x73, 0x95,
+ 0xf1, 0x4c, 0x1b, 0x9b, 0x2e, 0x64, 0x17, 0x6c,
+ 0xb1, 0xcd, 0x90, 0x7a, 0xf6, 0x90, 0xf5, 0xa1,
+ 0x22, 0x9a, 0x6b, 0x16, 0xfc, 0xc8, 0x86, 0x5a,
+ 0xe7, 0x68, 0x53, 0x2b, 0x31, 0xd0, 0x06, 0x63,
+ 0x81, 0xdf, 0xd2, 0x9b, 0x7f, 0x60, 0x46, 0xf7,
+ 0xf3, 0xef, 0xb5, 0x71, 0xa4, 0x96, 0xf7, 0xe2,
+ 0xd1, 0x9b, 0x9d, 0x71, 0x0e, 0x06, 0xb5, 0x6e,
+ 0x6e, 0x14, 0x4b, 0xcf, 0xdd, 0x9e, 0x53, 0x71,
+ 0x54, 0xb9, 0xfa, 0x4f, 0xb4, 0x25, 0x48, 0xa4,
+ 0x8a, 0xf8, 0xca, 0x1e, 0x56, 0xc6, 0xa6, 0x19,
+ 0x11, 0x67, 0x9f, 0xb6, 0x59, 0xa1, 0x80, 0x97,
+ 0xb5, 0xdd, 0x50, 0x95, 0x94, 0xbe, 0x8f, 0x65,
+ 0x05, 0xe3, 0xa2, 0xba, 0x22, 0x36, 0x04, 0x1b,
+ 0x44, 0xdf, 0xd8, 0x2c, 0x08, 0xb4, 0x6b, 0x40,
+ 0xec, 0x5d, 0x7f, 0xbb, 0x94, 0x25, 0x3c, 0xf3,
+ 0x73, 0x52, 0xd4, 0xc5, 0x63, 0x54, 0xde, 0x0f,
+ 0x4d, 0xd6, 0xfe, 0x5e, 0xf9, 0x02, 0x84, 0x31,
+ 0x28, 0xee, 0xfd, 0x0e, 0xbe, 0xaf, 0xe0, 0x75,
+ 0x1c, 0xfd, 0xcc, 0xcf, 0x4e, 0x67, 0xd6, 0x86,
+ 0x9c, 0x9d, 0x3c, 0x87, 0xb2, 0x89, 0x79, 0x9b,
+ 0xd7, 0x8e, 0xca, 0x90, 0xb6, 0xdd, 0x4a, 0xf1,
+ 0x0a, 0xed, 0xaa, 0xc1, 0x80, 0xf9, 0x96, 0x49,
+ 0xce, 0xa3, 0x12, 0x63, 0xae, 0xc6, 0xcb, 0x3b,
+ 0xf1, 0xe7, 0xb9, 0x1d, 0x0b, 0x0d, 0xd4, 0x1c,
+ 0x48, 0x30, 0x53, 0xab, 0xba, 0x94, 0x45, 0xc4,
+ 0x5c, 0xaf, 0xe0, 0xe7, 0x81, 0x61, 0x2f, 0xa1,
+ 0x02, 0x12, 0x55, 0x5c, 0xde, 0x3a, 0x48, 0x7c,
+ 0xe0, 0x3b, 0xab, 0x37, 0x18, 0xf1, 0x1b, 0x8b,
+ 0xba, 0x6c, 0x4a, 0xef, 0x5b, 0x95, 0xdb, 0xd8,
+ 0xab, 0x8d, 0xde, 0x27, 0xdf, 0x44, 0xcd, 0xb1,
+ 0x8d, 0x97 },
+ .ilen = 514,
+ .result = { 0x20, 0xaf, 0x08, 0x43, 0x2c, 0x9b, 0xff, 0x20,
+ 0x30, 0xc3, 0xec, 0xec, 0x81, 0x08, 0x9b, 0x8d,
+ 0x6e, 0x2a, 0x48, 0xef, 0x65, 0x01, 0xaf, 0x1a,
+ 0xe9, 0x46, 0x32, 0x74, 0x28, 0x50, 0xd4, 0x8a,
+ 0xc9, 0xf3, 0x2c, 0x6c, 0x6b, 0xad, 0x79, 0xa3,
+ 0x61, 0x2c, 0x7e, 0x41, 0xf0, 0xef, 0x74, 0x90,
+ 0xc2, 0xc2, 0xfb, 0xc1, 0x41, 0x48, 0xcb, 0xa2,
+ 0xc3, 0x37, 0x31, 0x63, 0x97, 0x28, 0xd8, 0x7a,
+ 0x6e, 0xa7, 0x57, 0xfb, 0xb2, 0x7d, 0xfc, 0xe1,
+ 0x78, 0xe9, 0x32, 0x2c, 0x74, 0xed, 0x29, 0xf7,
+ 0xb7, 0x4c, 0x65, 0x04, 0x09, 0x47, 0x22, 0x2a,
+ 0x16, 0x3e, 0xe4, 0xff, 0x53, 0x2a, 0x4f, 0xea,
+ 0x7c, 0x31, 0x12, 0x0d, 0xc0, 0x55, 0xc8, 0x6b,
+ 0x97, 0x7f, 0x2e, 0x64, 0xc1, 0x7b, 0x46, 0xb9,
+ 0x9a, 0xfa, 0xcc, 0x36, 0x94, 0x8e, 0x44, 0x06,
+ 0xe2, 0xa6, 0xf2, 0xef, 0x8b, 0x6a, 0x45, 0x44,
+ 0x2e, 0xc8, 0x6a, 0x3c, 0x1b, 0xa5, 0xa8, 0x01,
+ 0xf5, 0x27, 0x23, 0x27, 0xb0, 0xe0, 0x97, 0x69,
+ 0x73, 0x08, 0xa4, 0x3e, 0x71, 0x79, 0xce, 0xa7,
+ 0xce, 0x38, 0xd8, 0x1c, 0xb8, 0xf3, 0x7f, 0x03,
+ 0x6a, 0x4f, 0x19, 0x05, 0xf0, 0x09, 0xe8, 0xb2,
+ 0x19, 0xd4, 0x12, 0xcd, 0x46, 0x87, 0xc2, 0xc5,
+ 0xd2, 0xe3, 0x04, 0x98, 0x97, 0x16, 0xda, 0xd6,
+ 0x39, 0x3a, 0xfe, 0xa8, 0x77, 0x69, 0xc7, 0xe9,
+ 0xb2, 0xe7, 0x65, 0x64, 0x06, 0x22, 0xe7, 0xc2,
+ 0xd4, 0x26, 0x10, 0xf5, 0xd3, 0x01, 0x9f, 0x3d,
+ 0x32, 0x90, 0xbf, 0xd0, 0x19, 0x80, 0xbd, 0xe5,
+ 0xde, 0xe8, 0x15, 0xb3, 0xe9, 0x13, 0x2e, 0x5b,
+ 0x59, 0x2f, 0x3a, 0x24, 0xb2, 0xce, 0xfa, 0x94,
+ 0xe2, 0xc7, 0x69, 0x05, 0xa5, 0xc4, 0xb2, 0x30,
+ 0xfb, 0x48, 0xd9, 0x75, 0x94, 0xf1, 0x85, 0xb5,
+ 0xff, 0xe3, 0xe9, 0xd5, 0xa6, 0xcf, 0x12, 0x46,
+ 0xc5, 0x9a, 0x70, 0xf3, 0xec, 0xbe, 0x2c, 0x24,
+ 0xd2, 0xa2, 0x17, 0x67, 0x7a, 0x46, 0x8f, 0xb4,
+ 0x96, 0xa4, 0xab, 0xf6, 0xab, 0xb4, 0x7b, 0x31,
+ 0x6d, 0xf0, 0x37, 0x8f, 0x98, 0xda, 0x3c, 0x56,
+ 0xd3, 0x5b, 0xe1, 0xd0, 0x57, 0x04, 0xf5, 0xcc,
+ 0x0c, 0x20, 0x4b, 0xd2, 0x17, 0xed, 0x3c, 0x4b,
+ 0x32, 0xa4, 0xd8, 0x87, 0xaf, 0xae, 0xf9, 0x45,
+ 0x26, 0x9d, 0xc5, 0xe4, 0x61, 0x31, 0xf0, 0xde,
+ 0x7b, 0x55, 0x1b, 0x7e, 0xc9, 0x01, 0x91, 0x07,
+ 0xb0, 0x45, 0xf1, 0x1c, 0x75, 0xa9, 0xf5, 0x58,
+ 0xbd, 0x4c, 0x49, 0x4e, 0xd8, 0x73, 0x37, 0x08,
+ 0x92, 0x05, 0x87, 0xba, 0xa3, 0xc8, 0x14, 0xa0,
+ 0xee, 0x21, 0xd7, 0xf6, 0x03, 0x37, 0xe3, 0xe0,
+ 0x4f, 0xea, 0x85, 0x94, 0x1b, 0x45, 0x92, 0x0a,
+ 0xa0, 0xea, 0xba, 0xc2, 0xb9, 0x88, 0x29, 0xe8,
+ 0xbd, 0x81, 0xda, 0x95, 0x45, 0xf5, 0x5f, 0x20,
+ 0x94, 0x80, 0x21, 0x36, 0x10, 0x6a, 0xb7, 0x90,
+ 0xc0, 0xa9, 0x9e, 0x2a, 0x20, 0xa9, 0xd5, 0x61,
+ 0xf4, 0x72, 0x56, 0xaf, 0x00, 0x9a, 0x33, 0xf6,
+ 0x51, 0x12, 0xfc, 0x84, 0x15, 0x36, 0xc5, 0x27,
+ 0x4f, 0x44, 0xdf, 0x64, 0xde, 0xd7, 0xe1, 0xe9,
+ 0xc8, 0x08, 0xa3, 0x42, 0x60, 0xec, 0x1f, 0xbe,
+ 0x9f, 0x7b, 0x79, 0x47, 0xff, 0xea, 0x6f, 0x4f,
+ 0xc6, 0xea, 0x34, 0x54, 0x09, 0xb2, 0xa1, 0xde,
+ 0xb4, 0x54, 0xb9, 0x82, 0x9e, 0x41, 0xad, 0xa6,
+ 0x4f, 0x8f, 0xac, 0x3b, 0x01, 0x04, 0xbb, 0x9c,
+ 0x6a, 0xac, 0x24, 0xc1, 0xa8, 0x91, 0xb7, 0x5b,
+ 0x88, 0x88, 0xf3, 0x42, 0x17, 0xd4, 0xb7, 0xfc,
+ 0xc8, 0x80, 0x1b, 0x54, 0x91, 0x47, 0xd8, 0x1b,
+ 0xbc, 0x12, 0x7e, 0x4a, 0x03, 0x3b, 0x78, 0x83,
+ 0x2c, 0x92, 0x92, 0xa4, 0x31, 0xf2, 0xe8, 0x13,
+ 0x36, 0x0e, 0xc6, 0xb4, 0xed, 0x5c, 0x22, 0x71,
+ 0xa0, 0xa9 }
+}, {
+ .key = { 0x61, 0x93, 0x3b, 0xc8, 0x4b, 0x53, 0x4f, 0xc1,
+ 0x9e, 0x74, 0x24, 0x7f, 0x00, 0xcb, 0xc8, 0x33,
+ 0x42, 0x55, 0x6f, 0x77, 0xf1, 0x35, 0x52, 0xb0,
+ 0x67, 0xe7, 0x1d, 0x6e, 0xf1, 0xf6, 0x1c, 0x4d },
+ .nonce = 0xa3030d135287782eULL,
+ .input = { 0x83, 0x01, 0x49, 0xa9, 0x1b, 0x38, 0x85, 0x91,
+ 0x26, 0x41, 0x91, 0x20, 0xb2, 0x9e, 0x98, 0xaf,
+ 0x27, 0x30, 0xe1, 0xa6, 0x44, 0x4d, 0x2a, 0xaa,
+ 0xb3, 0x50, 0x7b, 0x27, 0x12, 0x86, 0x8b, 0xba,
+ 0x18, 0xaf, 0xc8, 0x8c, 0x68, 0x1b, 0x51, 0x1e,
+ 0x99, 0x89, 0x15, 0xcb, 0xd1, 0x5e, 0xaa, 0xe6,
+ 0x7f, 0x0d, 0x78, 0x7a, 0x66, 0xfc, 0xb5, 0x09,
+ 0x0d, 0x6e, 0xeb, 0xc1, 0x8b, 0x92, 0x5f, 0x94,
+ 0x83, 0xb5, 0x3e, 0x5a, 0x73, 0x28, 0xca, 0xb4,
+ 0x99, 0x9d, 0xe1, 0x17, 0x64, 0x12, 0x59, 0xb7,
+ 0xbe, 0x50, 0xf1, 0x52, 0xd3, 0x34, 0x93, 0x3c,
+ 0xa3, 0xbc, 0xc2, 0x00, 0x86, 0x7e, 0x40, 0x9e,
+ 0xd6, 0x65, 0xf8, 0xf3, 0x8a, 0x60, 0x7c, 0x63,
+ 0x01, 0xc7, 0x1a, 0xbd, 0xe5, 0x0c, 0x91, 0xfe,
+ 0x6b, 0x6c, 0xae, 0xca, 0x9b, 0x79, 0xa6, 0x07,
+ 0xa3, 0x00, 0xc0, 0x63, 0x3b, 0xfb, 0xc6, 0xa5,
+ 0xeb, 0x36, 0xd5, 0xaa, 0xc5, 0x28, 0xed, 0x37,
+ 0x04, 0x90, 0xd2, 0x4d, 0x08, 0x5e, 0x23, 0x8b,
+ 0xe7, 0xf3, 0x93, 0xf4, 0x91, 0xc6, 0xcf, 0x13,
+ 0x24, 0x0a, 0x2d, 0x6a, 0x81, 0x3b, 0xaa, 0xd9,
+ 0x4c, 0xe0, 0x57, 0xe1, 0x11, 0x62, 0xf5, 0x58,
+ 0x7f, 0xcd, 0x7b, 0xe6, 0xa8, 0xc3, 0x5a, 0xd5,
+ 0xc3, 0xd0, 0x32, 0xcf, 0x14, 0x10, 0x4c, 0x79,
+ 0x2d, 0x67, 0x4d, 0x1c, 0xe7, 0x93, 0x2e, 0x1f,
+ 0x9c, 0x51, 0x31, 0xb9, 0x3c, 0x8b, 0x00, 0x72,
+ 0x95, 0x16, 0xb8, 0x5e, 0xf6, 0x6c, 0xfa, 0xe6,
+ 0x7c, 0xbb, 0xa6, 0xef, 0x73, 0x4b, 0xcd, 0x10,
+ 0x32, 0x6b, 0x0d, 0xcb, 0x82, 0x3c, 0x1a, 0x2d,
+ 0xde, 0x7a, 0xc8, 0xcc, 0x23, 0xce, 0x06, 0x6a,
+ 0x92, 0x65, 0xe7, 0x00, 0xe3, 0xe5, 0xf1, 0x46,
+ 0x9e, 0x6a, 0xe8, 0xfb, 0x8c, 0x2c, 0x52, 0x4b,
+ 0x9a, 0x0a, 0x66, 0x26, 0xdc, 0x6f, 0x25, 0x86,
+ 0x2b, 0x63, 0x6b, 0x57, 0x4f, 0x62, 0xb2, 0xe6,
+ 0x5c, 0xbf, 0x55, 0xd9, 0x65, 0x88, 0x8e, 0xb5,
+ 0xc1, 0xfb, 0x0e, 0x80, 0xb5, 0x76, 0x0b, 0x1f,
+ 0xe6, 0x49, 0x8b, 0x60, 0xef, 0xad, 0xb6, 0x47,
+ 0xc2, 0x0e, 0x4c, 0xc7, 0xf8, 0xe6, 0xcb, 0x83,
+ 0x41, 0xf8, 0xd6, 0x72, 0x17, 0xfc, 0x62, 0x03,
+ 0x9f, 0x42, 0x38, 0x3d, 0xe4, 0x26, 0x2b, 0x8c,
+ 0xf8, 0xa4, 0x1d, 0x05, 0x7b, 0x03, 0x3f, 0x64,
+ 0xec, 0x64, 0x77, 0x29, 0xdd, 0xb3, 0x02, 0xf9,
+ 0xb3, 0x0c, 0x60, 0x3d, 0xcb, 0x4d, 0x61, 0x8c,
+ 0xf2, 0x07, 0x2b, 0x0d, 0x07, 0x67, 0xf7, 0xd6,
+ 0x7e, 0xf1, 0xe8, 0x0e, 0xbb, 0x93, 0x9b, 0x3a,
+ 0xe5, 0xe1, 0x03, 0x75, 0x66, 0xe0, 0x10, 0x7e,
+ 0x7b, 0x35, 0x54, 0xc1, 0xd4, 0xec, 0xfa, 0x66,
+ 0x55, 0x22, 0x22, 0x6a, 0xee, 0x30, 0x7c, 0x8f,
+ 0xc9, 0x5d, 0xb9, 0x1f, 0x86, 0x5c, 0x7d, 0x71,
+ 0xc5, 0x2b, 0x74, 0x0a, 0x24, 0x15, 0x8f, 0x91,
+ 0x7b, 0xb0, 0x6b, 0x0b, 0x02, 0x84, 0x92, 0x4a,
+ 0x68, 0x22, 0x63, 0xb4, 0xf1, 0x4f, 0x94, 0x69,
+ 0x25, 0x24, 0x87, 0xef, 0x12, 0x36, 0xb9, 0x75,
+ 0x96, 0x42, 0x65, 0xb6, 0x6d, 0x8b, 0x5d, 0x7b,
+ 0x99, 0x0c, 0xc8, 0xbb, 0x87, 0x83, 0x21, 0xe4,
+ 0xb0, 0xe4, 0xa5, 0x76, 0x0c, 0xe6, 0x2d, 0x70,
+ 0x8e, 0xb2, 0xf9, 0xd0, 0xa8, 0xf2, 0x93, 0xfa,
+ 0x7d, 0xf4, 0x08, 0x58, 0x1d, 0x53, 0xee, 0x1c,
+ 0x74, 0xdf, 0x83, 0x16, 0xbb, 0x84, 0x30, 0xe0,
+ 0x06, 0xf6, 0x4b, 0x05, 0x03, 0xd8, 0x7d, 0xbc,
+ 0x04, 0xc8, 0x44, 0x2a, 0x5d, 0xe1, 0xdc, 0x06,
+ 0xea, 0x61, 0xb7, 0xde, 0xea, 0x27, 0x0f, 0x74,
+ 0x9a, 0x6d, 0xed, 0x1f, 0x03, 0x11, 0xd3, 0xca,
+ 0xf0, 0x6c, 0x3c, 0x62, 0x5c, 0x6a, 0x64, 0xa2,
+ 0x1d, 0x1b, 0xec, 0x21, 0xd0, 0xaa, 0x92, 0xdb,
+ 0x89, 0xac, 0x19, 0xd2, 0xd6, 0xd3, 0x52, 0xa4,
+ 0x36, 0x5b, 0x66, 0xe2, 0xea, 0xa3, 0xe8, 0x6a,
+ 0x1f, 0x8f, 0xaf, 0x6d, 0xfe, 0xcc, 0x5d, 0x9f,
+ 0x88, 0x70, 0xe7, 0x41, 0xa1, 0xb5, 0x3e, 0x6a,
+ 0x2a, 0x94, 0xc5, 0x33, 0x71, 0xb9, 0xea, 0x74,
+ 0xf4, 0x46, 0x8f, 0x71, 0x0e, 0x8b, 0x84, 0x1b,
+ 0xaf, 0xc8, 0x50, 0x74, 0x60, 0xe1, 0xf2, 0x44,
+ 0xc1, 0xac, 0x77, 0x50, 0x93, 0x4e, 0xb1, 0x3c,
+ 0xd3, 0xcf, 0x0c, 0xdf, 0xc1, 0xb9, 0x28, 0x4f,
+ 0xe1, 0x6b, 0xd6, 0xeb, 0x88, 0x57, 0x1c, 0x98,
+ 0xa6, 0xc7, 0x4c, 0xf6, 0xb6, 0x27, 0x9d, 0xfb,
+ 0xc5, 0xc2, 0x7c, 0xf3, 0xa7, 0x29, 0x72, 0xca,
+ 0x3f, 0xf3, 0x56, 0x42, 0xca, 0x50, 0xaf, 0x76,
+ 0x88, 0xd1, 0x7d, 0x88, 0x91, 0x0d, 0x80, 0x05,
+ 0x4f, 0xc1, 0xa4, 0xe4, 0xd5, 0x17, 0xd6, 0xa7,
+ 0xd3, 0xb7, 0x85, 0x44, 0x80, 0x70, 0xee, 0x6a,
+ 0x27, 0x4d, 0x52, 0xd8, 0xfb, 0x7b, 0x51, 0x6c,
+ 0x44, 0x8f, 0x44, 0x4d, 0x74, 0x11, 0x65, 0xac,
+ 0x04, 0x4a, 0x4e, 0xef, 0x98, 0x89, 0x20, 0x09,
+ 0x34, 0x19, 0xfe, 0x38, 0x29, 0xd4, 0x53, 0xee,
+ 0xf9, 0x37, 0x1f, 0xc0, 0x2a, 0x5e, 0xa4, 0x3e,
+ 0x3f, 0x30, 0xd4, 0xaf, 0xa5, 0x3c, 0x77, 0xe4,
+ 0xf5, 0xf7, 0xad, 0x55, 0xd8, 0xc9, 0x90, 0x00,
+ 0x03, 0xa3, 0x9e, 0xb1, 0x84, 0xb0, 0xbb, 0x46,
+ 0x9a, 0xdd, 0xe7, 0xdd, 0xaa, 0x8a, 0xa2, 0xd7,
+ 0xa8, 0x03, 0x63, 0xf4, 0x16, 0x7c, 0x3e, 0xa0,
+ 0xa8, 0x32, 0x9e, 0x3f, 0x16, 0xf0, 0xa5, 0x7a,
+ 0xff, 0x44, 0xfc, 0x60, 0xb1, 0x65, 0xf4, 0xf5,
+ 0xb1, 0x11, 0x7c, 0xe5, 0x0d, 0x78, 0xdc, 0x1a,
+ 0xc3, 0xe0, 0x3f, 0x8c, 0x5e, 0x5a, 0x6b, 0x2f,
+ 0x11, 0xbe, 0xc9, 0xff, 0xe6, 0x84, 0xbf, 0x62,
+ 0x93, 0xdb, 0xe7, 0xe6, 0x30, 0x56, 0x99, 0x4e,
+ 0x85 },
+ .ilen = 769,
+ .result = { 0x49, 0xbd, 0x98, 0x31, 0xb8, 0x44, 0x59, 0x02,
+ 0x7e, 0x1e, 0x7a, 0x4f, 0x77, 0x71, 0xd8, 0x5d,
+ 0xef, 0x8d, 0xcb, 0xa4, 0x61, 0x76, 0xab, 0xd8,
+ 0x07, 0x75, 0x86, 0x51, 0x5b, 0xcf, 0xc3, 0xd8,
+ 0x6c, 0x36, 0xe9, 0x1f, 0x24, 0x48, 0xe1, 0xca,
+ 0xff, 0x73, 0x0d, 0x30, 0xc1, 0xff, 0x8f, 0x6c,
+ 0x51, 0xf0, 0x58, 0x7f, 0x1c, 0xba, 0x94, 0x16,
+ 0x59, 0x81, 0x7f, 0x04, 0xcf, 0x58, 0xa0, 0x5f,
+ 0x64, 0x35, 0x55, 0x3e, 0x5d, 0x2e, 0x45, 0x55,
+ 0x71, 0xd1, 0xc9, 0x43, 0x9d, 0x3a, 0xf2, 0x93,
+ 0xed, 0x05, 0x25, 0x2e, 0xc0, 0x8c, 0x09, 0x0b,
+ 0x06, 0x3a, 0x89, 0x26, 0xd3, 0x32, 0x94, 0xc7,
+ 0x66, 0xd1, 0x13, 0xb0, 0x12, 0xd9, 0x99, 0x37,
+ 0x08, 0x14, 0xe4, 0xd5, 0x32, 0x07, 0x3d, 0xcc,
+ 0xa6, 0xc4, 0x25, 0x2f, 0x49, 0x60, 0xa9, 0x78,
+ 0x4c, 0x1c, 0x42, 0xdb, 0x46, 0xa0, 0x6e, 0x84,
+ 0x09, 0x01, 0xe8, 0x69, 0xb0, 0x6e, 0xc4, 0x3c,
+ 0xd9, 0x53, 0x5a, 0x8e, 0xf0, 0xe5, 0xc6, 0x73,
+ 0x7d, 0x46, 0x8c, 0x49, 0x6e, 0xfd, 0x7e, 0x7f,
+ 0xb1, 0xb6, 0xd6, 0x4e, 0x94, 0xce, 0xe1, 0x55,
+ 0x9b, 0x8f, 0x96, 0x46, 0x62, 0x43, 0x59, 0x9c,
+ 0x8f, 0x07, 0x85, 0xad, 0xac, 0x53, 0x54, 0xda,
+ 0xce, 0x99, 0xb8, 0x8d, 0x23, 0x5f, 0x15, 0xcd,
+ 0x13, 0x00, 0xf8, 0xc4, 0x73, 0x74, 0xef, 0x6e,
+ 0x69, 0x9d, 0x10, 0x7e, 0x32, 0xaa, 0xab, 0x72,
+ 0xd9, 0xa0, 0x35, 0x96, 0xbf, 0xbe, 0xae, 0xf5,
+ 0x1d, 0xbc, 0x73, 0x2b, 0x07, 0x0c, 0x1d, 0xb5,
+ 0xfe, 0x9d, 0xc2, 0xda, 0x57, 0x0e, 0xcb, 0x2c,
+ 0x6a, 0x47, 0xab, 0x27, 0xd5, 0x02, 0xc4, 0x4c,
+ 0xc4, 0xb2, 0x1d, 0xca, 0x0d, 0x13, 0xa3, 0x2c,
+ 0x93, 0x23, 0x7d, 0xc9, 0xe9, 0x6b, 0x39, 0xaa,
+ 0xba, 0x24, 0xaf, 0x09, 0x42, 0x5c, 0xfe, 0x63,
+ 0xa2, 0x0e, 0x15, 0x0b, 0x06, 0xf8, 0x1a, 0xfc,
+ 0x9c, 0x03, 0x94, 0x90, 0x0d, 0x2b, 0xdc, 0x7a,
+ 0x08, 0xf3, 0xdd, 0xc6, 0x34, 0xac, 0x2a, 0xfb,
+ 0xd1, 0x5b, 0x04, 0x27, 0x4f, 0x63, 0x76, 0xbb,
+ 0x03, 0x73, 0xb5, 0x9c, 0x49, 0x8b, 0xb5, 0xe7,
+ 0x23, 0xb9, 0x79, 0xf6, 0x4b, 0x48, 0xe4, 0x74,
+ 0x7c, 0xa4, 0xb0, 0x6b, 0xbf, 0x0c, 0x61, 0x9e,
+ 0x19, 0x57, 0xaa, 0x38, 0x40, 0xbf, 0x7b, 0x33,
+ 0x28, 0xeb, 0x96, 0x3d, 0xc2, 0xb6, 0x2e, 0x3e,
+ 0xc7, 0xc9, 0xb5, 0x7b, 0x3e, 0x2c, 0x79, 0x94,
+ 0x10, 0xb5, 0xcd, 0x5b, 0x52, 0xef, 0x57, 0x41,
+ 0x98, 0xae, 0x51, 0x6b, 0x0d, 0x4c, 0x42, 0xe3,
+ 0x9e, 0x62, 0x25, 0xd1, 0x71, 0x19, 0x73, 0x4f,
+ 0x5e, 0x52, 0x76, 0x8e, 0x5f, 0xd7, 0xa2, 0xa9,
+ 0xd2, 0x60, 0x44, 0x81, 0x87, 0xe8, 0x2c, 0xb2,
+ 0xf1, 0xa7, 0x2a, 0x2b, 0x40, 0xf9, 0x0a, 0x01,
+ 0x83, 0xfc, 0x8d, 0xcc, 0x83, 0x28, 0xf4, 0xad,
+ 0x6e, 0xcb, 0xe1, 0x02, 0x89, 0xd7, 0x14, 0x73,
+ 0xd7, 0x56, 0x1e, 0x41, 0xf3, 0x0b, 0xa4, 0xf2,
+ 0x02, 0x8b, 0xc1, 0x9e, 0xe6, 0xd1, 0x89, 0xbd,
+ 0x20, 0xf3, 0x1a, 0xfc, 0x8e, 0x7c, 0xde, 0xb9,
+ 0x2a, 0x51, 0x67, 0xeb, 0x0e, 0x32, 0x3f, 0x88,
+ 0xc1, 0x26, 0xd9, 0xb8, 0x6e, 0xb0, 0x20, 0x4c,
+ 0xfa, 0x4b, 0xfa, 0x21, 0xd9, 0xd3, 0x52, 0xdd,
+ 0xbc, 0x20, 0x91, 0xe7, 0x45, 0x03, 0xe7, 0x3e,
+ 0x40, 0xb0, 0x76, 0xc7, 0xb5, 0xec, 0xb3, 0x67,
+ 0x8f, 0xba, 0xfd, 0xf3, 0x15, 0x81, 0xdc, 0x2f,
+ 0x03, 0x8d, 0xf0, 0x69, 0x9f, 0xd6, 0xda, 0x8f,
+ 0x98, 0xa0, 0x04, 0xb3, 0x73, 0x7b, 0xf2, 0xbd,
+ 0xf2, 0x9e, 0xc8, 0x30, 0x0f, 0x40, 0x65, 0x1f,
+ 0x67, 0xda, 0x7d, 0x56, 0x53, 0x66, 0xff, 0xe9,
+ 0x15, 0xe4, 0x4b, 0x61, 0x26, 0x62, 0x8f, 0x5a,
+ 0xd0, 0xe7, 0xc4, 0x53, 0x5f, 0x96, 0x4d, 0x82,
+ 0x50, 0x61, 0x64, 0x1b, 0xe3, 0xe3, 0xbb, 0x6b,
+ 0x4f, 0x1d, 0x58, 0xf2, 0xde, 0xfd, 0x59, 0x0c,
+ 0x62, 0xa9, 0xc3, 0x65, 0x45, 0xd8, 0x82, 0x45,
+ 0xaa, 0x06, 0x4c, 0xc1, 0x6e, 0x95, 0x9a, 0x7a,
+ 0x2b, 0x1c, 0x91, 0xc1, 0x87, 0x45, 0x47, 0x71,
+ 0xca, 0xea, 0x0c, 0xa6, 0x70, 0x08, 0x2d, 0x1b,
+ 0x26, 0x38, 0xaa, 0x41, 0x0b, 0xe7, 0xa1, 0xe6,
+ 0x6e, 0xa4, 0x7c, 0x8d, 0xf1, 0x1a, 0xd8, 0x05,
+ 0xb3, 0x69, 0x0b, 0x06, 0xc4, 0x49, 0x84, 0x2d,
+ 0x08, 0xe6, 0x57, 0xe9, 0x64, 0x02, 0xd2, 0xc0,
+ 0x73, 0xf1, 0x6d, 0x88, 0x07, 0x48, 0x47, 0x7c,
+ 0xe4, 0x42, 0x4b, 0x5a, 0x65, 0xb4, 0xc3, 0x65,
+ 0xcc, 0xd6, 0xc6, 0x5d, 0xc7, 0x21, 0xc8, 0x65,
+ 0x28, 0x96, 0xd6, 0xad, 0x2b, 0xc7, 0x49, 0xbe,
+ 0x98, 0xb3, 0x43, 0xb8, 0x9a, 0xd1, 0xe9, 0xa0,
+ 0xed, 0x8d, 0x8c, 0x2c, 0x00, 0xa6, 0x1e, 0x3a,
+ 0xb5, 0x0d, 0xeb, 0xcf, 0x12, 0xe6, 0x0e, 0x4f,
+ 0x7d, 0x87, 0xca, 0x3d, 0x37, 0xaf, 0xb4, 0xa3,
+ 0xdc, 0xd4, 0xf7, 0xe2, 0x7e, 0xfc, 0xa7, 0x1b,
+ 0x92, 0x79, 0x63, 0x10, 0xce, 0x7b, 0x59, 0x87,
+ 0x89, 0x49, 0xb7, 0x50, 0x86, 0x7c, 0xa7, 0x5f,
+ 0x3f, 0x50, 0x31, 0x46, 0xd4, 0xfa, 0x6e, 0xcd,
+ 0xf4, 0xbe, 0xce, 0xbe, 0xd0, 0x71, 0xef, 0x2c,
+ 0xfe, 0xf1, 0x3c, 0xf0, 0xfd, 0xef, 0x96, 0x73,
+ 0xa3, 0xa7, 0x23, 0x35, 0x6c, 0x27, 0xef, 0x78,
+ 0xec, 0xdd, 0x46, 0xb7, 0xe4, 0xce, 0x37, 0x43,
+ 0x35, 0xf2, 0x38, 0x35, 0xf5, 0xb5, 0xc2, 0x9f,
+ 0xf0, 0x97, 0x52, 0x23, 0x0c, 0x74, 0x03, 0xd1,
+ 0x53, 0xd9, 0xf0, 0x73, 0xb6, 0xae, 0x4c, 0xa3,
+ 0xac, 0x6f, 0x4c, 0x93, 0x72, 0x47, 0x7b, 0x44,
+ 0x8d, 0xf9, 0xee, 0x24, 0x08, 0x95, 0x62, 0xbe,
+ 0x31 }
+}};
+
+static bool __init chacha20_selftest(void)
+{
+ size_t i;
+ u8 offset_input[MAXIMUM_TEST_BUFFER_LEN + 1]
+ __aligned(__alignof__(unsigned long));
+ u8 offset_key[CHACHA20_KEY_SIZE + 1]
+ __aligned(__alignof__(unsigned long));
+ u8 computed_result[MAXIMUM_TEST_BUFFER_LEN + 1]
+ __aligned(__alignof__(unsigned long));
+ struct chacha20_ctx state;
+ bool success = true;
+ simd_context_t simd_context;
+
+ simd_get(&simd_context);
+ for (i = 0; i < ARRAY_SIZE(chacha20_vectors); ++i) {
+ memset(computed_result, 0, sizeof(computed_result));
+ memset(&state, 0, sizeof(state));
+ chacha20_init(&state, chacha20_vectors[i].key,
+ chacha20_vectors[i].nonce);
+ chacha20(&state, computed_result, chacha20_vectors[i].input,
+ chacha20_vectors[i].ilen, &simd_context);
+ if (memcmp(computed_result, chacha20_vectors[i].result,
+ chacha20_vectors[i].ilen)) {
+ pr_info("chacha20 self-test %zu: FAIL\n", i + 1);
+ success = false;
+ }
+ memset(computed_result, 0, sizeof(computed_result));
+ memset(&state, 0, sizeof(state));
+ memcpy(offset_input + 1, chacha20_vectors[i].input,
+ chacha20_vectors[i].ilen);
+ memcpy(offset_key + 1, chacha20_vectors[i].key,
+ CHACHA20_KEY_SIZE);
+ chacha20_init(&state, offset_key + 1, chacha20_vectors[i].nonce);
+ chacha20(&state, computed_result + 1, offset_input + 1,
+ chacha20_vectors[i].ilen, &simd_context);
+ if (memcmp(computed_result + 1, chacha20_vectors[i].result,
+ chacha20_vectors[i].ilen)) {
+ pr_info("chacha20 self-test %zu (unaligned): FAIL\n", i + 1);
+ success = false;
+ }
+ }
+ simd_put(&simd_context);
+ if (success)
+ pr_info("chacha20 self-tests: pass\n");
+ return success;
+}
+#endif
--
2.19.0
^ permalink raw reply related
* [bpf PATCH v4 3/3] bpf: test_maps, only support ESTABLISHED socks
From: John Fastabend @ 2018-09-18 16:01 UTC (permalink / raw)
To: edumazet, ast, daniel; +Cc: netdev
In-Reply-To: <20180918155826.22728.71788.stgit@john-Precision-Tower-5810>
Ensure that sockets added to a sock{map|hash} that is not in the
ESTABLISHED state is rejected.
Fixes: 1aa12bdf1bfb ("bpf: sockmap, add sock close() hook to remove socks")
Signed-off-by: John Fastabend <john.fastabend@gmail.com>
Acked-by: Yonghong Song <yhs@fb.com>
---
tools/testing/selftests/bpf/test_maps.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c
index 6f54f84..9b552c0 100644
--- a/tools/testing/selftests/bpf/test_maps.c
+++ b/tools/testing/selftests/bpf/test_maps.c
@@ -580,7 +580,11 @@ static void test_sockmap(int tasks, void *data)
/* Test update without programs */
for (i = 0; i < 6; i++) {
err = bpf_map_update_elem(fd, &i, &sfd[i], BPF_ANY);
- if (err) {
+ if (i < 2 && !err) {
+ printf("Allowed update sockmap '%i:%i' not in ESTABLISHED\n",
+ i, sfd[i]);
+ goto out_sockmap;
+ } else if (i >= 2 && err) {
printf("Failed noprog update sockmap '%i:%i'\n",
i, sfd[i]);
goto out_sockmap;
@@ -741,7 +745,7 @@ static void test_sockmap(int tasks, void *data)
}
/* Test map update elem afterwards fd lives in fd and map_fd */
- for (i = 0; i < 6; i++) {
+ for (i = 2; i < 6; i++) {
err = bpf_map_update_elem(map_fd_rx, &i, &sfd[i], BPF_ANY);
if (err) {
printf("Failed map_fd_rx update sockmap %i '%i:%i'\n",
@@ -845,7 +849,7 @@ static void test_sockmap(int tasks, void *data)
}
/* Delete the elems without programs */
- for (i = 0; i < 6; i++) {
+ for (i = 2; i < 6; i++) {
err = bpf_map_delete_elem(fd, &i);
if (err) {
printf("Failed delete sockmap %i '%i:%i'\n",
^ permalink raw reply related
* [bpf PATCH v4 2/3] bpf: sockmap, fix transition through disconnect without close
From: John Fastabend @ 2018-09-18 16:01 UTC (permalink / raw)
To: edumazet, ast, daniel; +Cc: netdev
In-Reply-To: <20180918155826.22728.71788.stgit@john-Precision-Tower-5810>
It is possible (via shutdown()) for TCP socks to go trough TCP_CLOSE
state via tcp_disconnect() without actually calling tcp_close which
would then call our bpf_tcp_close() callback. Because of this a user
could disconnect a socket then put it in a LISTEN state which would
break our assumptions about sockets always being ESTABLISHED state.
To resolve this rely on the unhash hook, which is called in the
disconnect case, to remove the sock from the sockmap.
Reported-by: Eric Dumazet <edumazet@google.com>
Fixes: 1aa12bdf1bfb ("bpf: sockmap, add sock close() hook to remove socks")
Signed-off-by: John Fastabend <john.fastabend@gmail.com>
Acked-by: Yonghong Song <yhs@fb.com>
---
kernel/bpf/sockmap.c | 60 ++++++++++++++++++++++++++++++++++----------------
1 file changed, 41 insertions(+), 19 deletions(-)
diff --git a/kernel/bpf/sockmap.c b/kernel/bpf/sockmap.c
index 1f97b55..0a0f2ec 100644
--- a/kernel/bpf/sockmap.c
+++ b/kernel/bpf/sockmap.c
@@ -132,6 +132,7 @@ struct smap_psock {
struct work_struct gc_work;
struct proto *sk_proto;
+ void (*save_unhash)(struct sock *sk);
void (*save_close)(struct sock *sk, long timeout);
void (*save_data_ready)(struct sock *sk);
void (*save_write_space)(struct sock *sk);
@@ -143,6 +144,7 @@ static int bpf_tcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
static int bpf_tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size);
static int bpf_tcp_sendpage(struct sock *sk, struct page *page,
int offset, size_t size, int flags);
+static void bpf_tcp_unhash(struct sock *sk);
static void bpf_tcp_close(struct sock *sk, long timeout);
static inline struct smap_psock *smap_psock_sk(const struct sock *sk)
@@ -184,6 +186,7 @@ static void build_protos(struct proto prot[SOCKMAP_NUM_CONFIGS],
struct proto *base)
{
prot[SOCKMAP_BASE] = *base;
+ prot[SOCKMAP_BASE].unhash = bpf_tcp_unhash;
prot[SOCKMAP_BASE].close = bpf_tcp_close;
prot[SOCKMAP_BASE].recvmsg = bpf_tcp_recvmsg;
prot[SOCKMAP_BASE].stream_memory_read = bpf_tcp_stream_read;
@@ -217,6 +220,7 @@ static int bpf_tcp_init(struct sock *sk)
return -EBUSY;
}
+ psock->save_unhash = sk->sk_prot->unhash;
psock->save_close = sk->sk_prot->close;
psock->sk_proto = sk->sk_prot;
@@ -305,30 +309,12 @@ static struct smap_psock_map_entry *psock_map_pop(struct sock *sk,
return e;
}
-static void bpf_tcp_close(struct sock *sk, long timeout)
+static void bpf_tcp_remove(struct sock *sk, struct smap_psock *psock)
{
- void (*close_fun)(struct sock *sk, long timeout);
struct smap_psock_map_entry *e;
struct sk_msg_buff *md, *mtmp;
- struct smap_psock *psock;
struct sock *osk;
- lock_sock(sk);
- rcu_read_lock();
- psock = smap_psock_sk(sk);
- if (unlikely(!psock)) {
- rcu_read_unlock();
- release_sock(sk);
- return sk->sk_prot->close(sk, timeout);
- }
-
- /* The psock may be destroyed anytime after exiting the RCU critial
- * section so by the time we use close_fun the psock may no longer
- * be valid. However, bpf_tcp_close is called with the sock lock
- * held so the close hook and sk are still valid.
- */
- close_fun = psock->save_close;
-
if (psock->cork) {
free_start_sg(psock->sock, psock->cork, true);
kfree(psock->cork);
@@ -379,6 +365,42 @@ static void bpf_tcp_close(struct sock *sk, long timeout)
kfree(e);
e = psock_map_pop(sk, psock);
}
+}
+
+static void bpf_tcp_unhash(struct sock *sk)
+{
+ void (*unhash_fun)(struct sock *sk);
+ struct smap_psock *psock;
+
+ rcu_read_lock();
+ psock = smap_psock_sk(sk);
+ if (unlikely(!psock)) {
+ rcu_read_unlock();
+ if (sk->sk_prot->unhash)
+ sk->sk_prot->unhash(sk);
+ return;
+ }
+ unhash_fun = psock->save_unhash;
+ bpf_tcp_remove(sk, psock);
+ rcu_read_unlock();
+ unhash_fun(sk);
+}
+
+static void bpf_tcp_close(struct sock *sk, long timeout)
+{
+ void (*close_fun)(struct sock *sk, long timeout);
+ struct smap_psock *psock;
+
+ lock_sock(sk);
+ rcu_read_lock();
+ psock = smap_psock_sk(sk);
+ if (unlikely(!psock)) {
+ rcu_read_unlock();
+ release_sock(sk);
+ return sk->sk_prot->close(sk, timeout);
+ }
+ close_fun = psock->save_close;
+ bpf_tcp_remove(sk, psock);
rcu_read_unlock();
release_sock(sk);
close_fun(sk, timeout);
^ permalink raw reply related
* [bpf PATCH v4 1/3] bpf: sockmap only allow ESTABLISHED sock state
From: John Fastabend @ 2018-09-18 16:01 UTC (permalink / raw)
To: edumazet, ast, daniel; +Cc: netdev
In-Reply-To: <20180918155826.22728.71788.stgit@john-Precision-Tower-5810>
After this patch we only allow socks that are in ESTABLISHED state or
are being added via a sock_ops event that is transitioning into an
ESTABLISHED state. By allowing sock_ops events we allow users to
manage sockmaps directly from sock ops programs. The two supported
sock_ops ops are BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB and
BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB.
Similar to TLS ULP this ensures sk_user_data is correct.
Reported-by: Eric Dumazet <edumazet@google.com>
Fixes: 1aa12bdf1bfb ("bpf: sockmap, add sock close() hook to remove socks")
Signed-off-by: John Fastabend <john.fastabend@gmail.com>
Acked-by: Yonghong Song <yhs@fb.com>
---
kernel/bpf/sockmap.c | 31 ++++++++++++++++++++++++++++++-
1 file changed, 30 insertions(+), 1 deletion(-)
diff --git a/kernel/bpf/sockmap.c b/kernel/bpf/sockmap.c
index 488ef96..1f97b55 100644
--- a/kernel/bpf/sockmap.c
+++ b/kernel/bpf/sockmap.c
@@ -2097,8 +2097,12 @@ static int sock_map_update_elem(struct bpf_map *map,
return -EINVAL;
}
+ /* ULPs are currently supported only for TCP sockets in ESTABLISHED
+ * state.
+ */
if (skops.sk->sk_type != SOCK_STREAM ||
- skops.sk->sk_protocol != IPPROTO_TCP) {
+ skops.sk->sk_protocol != IPPROTO_TCP ||
+ skops.sk->sk_state != TCP_ESTABLISHED) {
fput(socket->file);
return -EOPNOTSUPP;
}
@@ -2453,6 +2457,16 @@ static int sock_hash_update_elem(struct bpf_map *map,
return -EINVAL;
}
+ /* ULPs are currently supported only for TCP sockets in ESTABLISHED
+ * state.
+ */
+ if (skops.sk->sk_type != SOCK_STREAM ||
+ skops.sk->sk_protocol != IPPROTO_TCP ||
+ skops.sk->sk_state != TCP_ESTABLISHED) {
+ fput(socket->file);
+ return -EOPNOTSUPP;
+ }
+
lock_sock(skops.sk);
preempt_disable();
rcu_read_lock();
@@ -2543,10 +2557,22 @@ struct sock *__sock_hash_lookup_elem(struct bpf_map *map, void *key)
.map_check_btf = map_check_no_btf,
};
+static bool bpf_is_valid_sock_op(struct bpf_sock_ops_kern *ops)
+{
+ return ops->op == BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB ||
+ ops->op == BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB;
+}
BPF_CALL_4(bpf_sock_map_update, struct bpf_sock_ops_kern *, bpf_sock,
struct bpf_map *, map, void *, key, u64, flags)
{
WARN_ON_ONCE(!rcu_read_lock_held());
+
+ /* ULPs are currently supported only for TCP sockets in ESTABLISHED
+ * state. This checks that the sock ops triggering the update is
+ * one indicating we are (or will be soon) in an ESTABLISHED state.
+ */
+ if (!bpf_is_valid_sock_op(bpf_sock))
+ return -EOPNOTSUPP;
return sock_map_ctx_update_elem(bpf_sock, map, key, flags);
}
@@ -2565,6 +2591,9 @@ struct sock *__sock_hash_lookup_elem(struct bpf_map *map, void *key)
struct bpf_map *, map, void *, key, u64, flags)
{
WARN_ON_ONCE(!rcu_read_lock_held());
+
+ if (!bpf_is_valid_sock_op(bpf_sock))
+ return -EOPNOTSUPP;
return sock_hash_ctx_update_elem(bpf_sock, map, key, flags);
}
^ permalink raw reply related
* [bpf PATCH v4 0/3] bpf, sockmap ESTABLISHED state only
From: John Fastabend @ 2018-09-18 16:01 UTC (permalink / raw)
To: edumazet, ast, daniel; +Cc: netdev
Eric noted that using the close callback is not sufficient
to catch all transitions from ESTABLISHED state to a LISTEN
state. So this series does two things. First, only allow
adding socks in ESTABLISH state and second use unhash callback
to catch tcp_disconnect() transitions.
v2: added check for ESTABLISH state in hash update sockmap as well
v3: Do not release lock from unhash in error path, no lock was
used in the first place. And drop not so useful code comments
v4: convert,
if (unhash()) return unhash(); return
to if (unhash()) unhash(); return;
Thanks for reviewing Yonghong I carried your ACKs forward.
---
John Fastabend (3):
bpf: sockmap only allow ESTABLISHED sock state
bpf: sockmap, fix transition through disconnect without close
bpf: test_maps, only support ESTABLISHED socks
kernel/bpf/sockmap.c | 91 ++++++++++++++++++++++++-------
tools/testing/selftests/bpf/test_maps.c | 10 ++-
2 files changed, 78 insertions(+), 23 deletions(-)
^ permalink raw reply
* Re: [PATCH v3] PCI: Reprogram bridge prefetch registers on resume
From: Bjorn Helgaas @ 2018-09-18 21:32 UTC (permalink / raw)
To: Daniel Drake
Cc: andy.shevchenko-VuQAYsv1563Yd54FQh9/CA,
linux-pm-u79uwXL29TY76Z2rM5mHXA, linux-pci-u79uwXL29TY76Z2rM5mHXA,
rafael.j.wysocki-ral2JQCrhuEAvxtiuMwx3w,
nic_swsd-Rasf1IRRPZFBDgjK7y7TUQ,
keith.busch-ral2JQCrhuEAvxtiuMwx3w, netdev-u79uwXL29TY76Z2rM5mHXA,
nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
bhelgaas-hpIqsD4AKlfQT0dZR+AlfA, rchang-eYqpPyKDWXRBDgjK7y7TUQ,
linux-6IF/jdPJHihWk0Htik3J/w, davem-fT/PcQaiUtIeIZ0/mPfg9Q,
jonathan.derrick-ral2JQCrhuEAvxtiuMwx3w,
hkallweit1-Re5JQEeQqe8AvxtiuMwx3w
In-Reply-To: <20180913033745.11178-1-drake-6IF/jdPJHihWk0Htik3J/w@public.gmane.org>
On Thu, Sep 13, 2018 at 11:37:45AM +0800, Daniel Drake wrote:
> On 38+ Intel-based Asus products, the nvidia GPU becomes unusable
> after S3 suspend/resume. The affected products include multiple
> generations of nvidia GPUs and Intel SoCs. After resume, nouveau logs
> many errors such as:
>
> fifo: fault 00 [READ] at 0000005555555000 engine 00 [GR] client 04
> [HUB/FE] reason 4a [] on channel -1 [007fa91000 unknown]
> DRM: failed to idle channel 0 [DRM]
>
> Similarly, the nvidia proprietary driver also fails after resume
> (black screen, 100% CPU usage in Xorg process). We shipped a sample
> to Nvidia for diagnosis, and their response indicated that it's a
> problem with the parent PCI bridge (on the Intel SoC), not the GPU.
>
> Runtime suspend/resume works fine, only S3 suspend is affected.
>
> We found a workaround: on resume, rewrite the Intel PCI bridge
> 'Prefetchable Base Upper 32 Bits' register (PCI_PREF_BASE_UPPER32). In
> the cases that I checked, this register has value 0 and we just have to
> rewrite that value.
>
> Linux already saves and restores PCI config space during suspend/resume,
> but this register was being skipped because upon resume, it already
> has value 0 (the correct, pre-suspend value).
>
> Intel appear to have previously acknowledged this behaviour and the
> requirement to rewrite this register.
> https://bugzilla.kernel.org/show_bug.cgi?id=116851#c23
>
> Based on that, rewrite the prefetch register values even when that
> appears unnecessary.
>
> We have confirmed this solution on all the affected models we have
> in-hands (X542UQ, UX533FD, X530UN, V272UN).
>
> Additionally, this solves an issue where r8169 MSI-X interrupts were
> broken after S3 suspend/resume on Asus X441UAR. This issue was recently
> worked around in commit 7bb05b85bc2d ("r8169: don't use MSI-X on
> RTL8106e"). It also fixes the same issue on RTL6186evl/8111evl on an
> Aimfor-tech laptop that we had not yet patched. I suspect it will also
> fix the issue that was worked around in commit 7c53a722459c ("r8169:
> don't use MSI-X on RTL8168g").
>
> Thomas Martitz reports that this change also solves an issue where
> the AMD Radeon Polaris 10 GPU on the HP Zbook 14u G5 is unresponsive
> after S3 suspend/resume.
>
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=201069
> Signed-off-by: Daniel Drake <drake@endlessm.com>
Applied with Rafael's and Peter's reviewed-by to pci/enumeration for v4.20.
Thanks for the the huge investigative effort!
> ---
> drivers/pci/pci.c | 25 +++++++++++++++++--------
> 1 file changed, 17 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 29ff9619b5fa..5d58220b6997 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -1289,12 +1289,12 @@ int pci_save_state(struct pci_dev *dev)
> EXPORT_SYMBOL(pci_save_state);
>
> static void pci_restore_config_dword(struct pci_dev *pdev, int offset,
> - u32 saved_val, int retry)
> + u32 saved_val, int retry, bool force)
> {
> u32 val;
>
> pci_read_config_dword(pdev, offset, &val);
> - if (val == saved_val)
> + if (!force && val == saved_val)
> return;
>
> for (;;) {
> @@ -1313,25 +1313,34 @@ static void pci_restore_config_dword(struct pci_dev *pdev, int offset,
> }
>
> static void pci_restore_config_space_range(struct pci_dev *pdev,
> - int start, int end, int retry)
> + int start, int end, int retry,
> + bool force)
> {
> int index;
>
> for (index = end; index >= start; index--)
> pci_restore_config_dword(pdev, 4 * index,
> pdev->saved_config_space[index],
> - retry);
> + retry, force);
> }
>
> static void pci_restore_config_space(struct pci_dev *pdev)
> {
> if (pdev->hdr_type == PCI_HEADER_TYPE_NORMAL) {
> - pci_restore_config_space_range(pdev, 10, 15, 0);
> + pci_restore_config_space_range(pdev, 10, 15, 0, false);
> /* Restore BARs before the command register. */
> - pci_restore_config_space_range(pdev, 4, 9, 10);
> - pci_restore_config_space_range(pdev, 0, 3, 0);
> + pci_restore_config_space_range(pdev, 4, 9, 10, false);
> + pci_restore_config_space_range(pdev, 0, 3, 0, false);
> + } else if (pdev->hdr_type == PCI_HEADER_TYPE_BRIDGE) {
> + pci_restore_config_space_range(pdev, 12, 15, 0, false);
> + /* Force rewriting of prefetch registers to avoid
> + * S3 resume issues on Intel PCI bridges that occur when
> + * these registers are not explicitly written.
> + */
> + pci_restore_config_space_range(pdev, 9, 11, 0, true);
> + pci_restore_config_space_range(pdev, 0, 8, 0, false);
> } else {
> - pci_restore_config_space_range(pdev, 0, 15, 0);
> + pci_restore_config_space_range(pdev, 0, 15, 0, false);
> }
> }
>
> --
> 2.17.1
>
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau
^ permalink raw reply
* Re: [PATCH bpf-next] samples/bpf: fix a compilation failure
From: Daniel Borkmann @ 2018-09-18 15:51 UTC (permalink / raw)
To: Yonghong Song, ast, netdev; +Cc: kernel-team
In-Reply-To: <20180918050813.2138154-1-yhs@fb.com>
On 09/18/2018 07:08 AM, Yonghong Song wrote:
> samples/bpf build failed with the following errors:
>
> $ make samples/bpf/
> ...
> HOSTCC samples/bpf/sockex3_user.o
> /data/users/yhs/work/net-next/samples/bpf/sockex3_user.c:16:8: error: redefinition of ‘struct bpf_flow_keys’
> struct bpf_flow_keys {
> ^
> In file included from /data/users/yhs/work/net-next/samples/bpf/sockex3_user.c:4:0:
> ./usr/include/linux/bpf.h:2338:9: note: originally defined here
> struct bpf_flow_keys *flow_keys;
> ^
> make[3]: *** [samples/bpf/sockex3_user.o] Error 1
>
> Commit d58e468b1112d ("flow_dissector: implements flow dissector BPF hook")
> introduced struct bpf_flow_keys in include/uapi/linux/bpf.h and hence
> caused the naming conflict with samples/bpf/sockex3_user.c.
>
> The fix is to rename struct bpf_flow_keys in samples/bpf/sockex3_user.c
> to flow_keys to avoid the conflict.
>
> Signed-off-by: Yonghong Song <yhs@fb.com>
Applied to bpf-next, thanks!
^ permalink raw reply
* [PATCH net] sfp: fix oops with ethtool -m
From: Russell King @ 2018-09-18 15:48 UTC (permalink / raw)
To: David S. Miller; +Cc: Andrew Lunn, Florian Fainelli, netdev
If a network interface is created prior to the SFP socket being
available, ethtool can request module information. This unfortunately
leads to an oops:
Unable to handle kernel NULL pointer dereference at virtual address 00000008
pgd = (ptrval)
[00000008] *pgd=7c400831, *pte=00000000, *ppte=00000000
Internal error: Oops: 17 [#1] SMP ARM
Modules linked in:
CPU: 0 PID: 1480 Comm: ethtool Not tainted 4.19.0-rc3 #138
Hardware name: Broadcom Northstar Plus SoC
PC is at sfp_get_module_info+0x8/0x10
LR is at dev_ethtool+0x218c/0x2afc
Fix this by not filling in the network device's SFP bus pointer until
SFP is fully bound, thereby avoiding the core calling into the SFP bus
code.
Fixes: ce0aa27ff3f6 ("sfp: add sfp-bus to bridge between network devices and sfp cages")
Reported-by: Florian Fainelli <f.fainelli@gmail.com>
Tested-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
---
drivers/net/phy/sfp-bus.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/phy/sfp-bus.c b/drivers/net/phy/sfp-bus.c
index 740655261e5b..83060fb349f4 100644
--- a/drivers/net/phy/sfp-bus.c
+++ b/drivers/net/phy/sfp-bus.c
@@ -349,6 +349,7 @@ static int sfp_register_bus(struct sfp_bus *bus)
}
if (bus->started)
bus->socket_ops->start(bus->sfp);
+ bus->netdev->sfp_bus = bus;
bus->registered = true;
return 0;
}
@@ -357,6 +358,7 @@ static void sfp_unregister_bus(struct sfp_bus *bus)
{
const struct sfp_upstream_ops *ops = bus->upstream_ops;
+ bus->netdev->sfp_bus = NULL;
if (bus->registered) {
if (bus->started)
bus->socket_ops->stop(bus->sfp);
@@ -438,7 +440,6 @@ static void sfp_upstream_clear(struct sfp_bus *bus)
{
bus->upstream_ops = NULL;
bus->upstream = NULL;
- bus->netdev->sfp_bus = NULL;
bus->netdev = NULL;
}
@@ -467,7 +468,6 @@ struct sfp_bus *sfp_register_upstream(struct fwnode_handle *fwnode,
bus->upstream_ops = ops;
bus->upstream = upstream;
bus->netdev = ndev;
- ndev->sfp_bus = bus;
if (bus->sfp) {
ret = sfp_register_bus(bus);
--
2.7.4
^ permalink raw reply related
* Re: [PATCH bpf-next v2] tools/bpf: bpftool: improve output format for bpftool net
From: Daniel Borkmann @ 2018-09-18 15:48 UTC (permalink / raw)
To: Yonghong Song, ast, netdev; +Cc: kernel-team
In-Reply-To: <20180917231300.1188382-1-yhs@fb.com>
On 09/18/2018 01:13 AM, Yonghong Song wrote:
> This is a followup patch for Commit f6f3bac08ff9
> ("tools/bpf: bpftool: add net support").
> Some improvements are made for the bpftool net output.
> Specially, plain output is more concise such that
> per attachment should nicely fit in one line.
> Compared to previous output, the prog tag is removed
> since it can be easily obtained with program id.
> Similar to xdp attachments, the device name is added
> to tc attachments.
>
> The bpf program attached through shared block
> mechanism is supported as well.
> $ ip link add dev v1 type veth peer name v2
> $ tc qdisc add dev v1 ingress_block 10 egress_block 20 clsact
> $ tc qdisc add dev v2 ingress_block 10 egress_block 20 clsact
> $ tc filter add block 10 protocol ip prio 25 bpf obj bpf_shared.o sec ingress flowid 1:1
> $ tc filter add block 20 protocol ip prio 30 bpf obj bpf_cyclic.o sec classifier flowid 1:1
> $ bpftool net
> xdp:
>
> tc:
> v2(7) clsact/ingress bpf_shared.o:[ingress] id 23
> v2(7) clsact/egress bpf_cyclic.o:[classifier] id 24
> v1(8) clsact/ingress bpf_shared.o:[ingress] id 23
> v1(8) clsact/egress bpf_cyclic.o:[classifier] id 24
>
> The documentation and "bpftool net help" are updated
> to make it clear that current implementation only
> supports xdp and tc attachments. For programs
> attached to cgroups, "bpftool cgroup" can be used
> to dump attachments. For other programs e.g.
> sk_{filter,skb,msg,reuseport} and lwt/seg6,
> iproute2 tools should be used.
>
> The new output:
> $ bpftool net
> xdp:
> eth0(2) driver id 198
>
> tc:
> eth0(2) clsact/ingress fbflow_icmp id 335 act [{icmp_action id 336}]
> eth0(2) clsact/egress fbflow_egress id 334
> $ bpftool -jp net
> [{
> "xdp": [{
> "devname": "eth0",
> "ifindex": 2,
> "mode": "driver",
> "id": 198
> }
> ],
> "tc": [{
> "devname": "eth0",
> "ifindex": 2,
> "kind": "clsact/ingress",
> "name": "fbflow_icmp",
> "id": 335,
> "act": [{
> "name": "icmp_action",
> "id": 336
> }
> ]
> },{
> "devname": "eth0",
> "ifindex": 2,
> "kind": "clsact/egress",
> "name": "fbflow_egress",
> "id": 334
> }
> ]
> }
> ]
>
> Signed-off-by: Yonghong Song <yhs@fb.com>
Applied to bpf-next, thanks!
^ permalink raw reply
* Re: [PATCH net-next v5 00/20] WireGuard: Secure Network Tunnel
From: Jason A. Donenfeld @ 2018-09-18 21:01 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Linux Kernel Mailing List, <netdev@vger.kernel.org>,
open list:HARDWARE RANDOM NUMBER GENERATOR CORE, David S. Miller,
Greg Kroah-Hartman
In-Reply-To: <CAKv+Gu8doWS+BYzqoT7499CJX0hLm1jBSMfVxJoG3wsWFO7t1w@mail.gmail.com>
Hi Ard,
On Tue, Sep 18, 2018 at 11:28:50AM -0700, Ard Biesheuvel wrote:
> On 18 September 2018 at 09:16, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> > - While I initially wasn't going to do this for the initial
> > patchset, it was just so simple to do: now there's a nosimd
> > module parameter that can be used to disable simd instructions
> > for debugging and testing, or on weird systems.
> >
>
> I was going to respond in the other thread but it is probably better
> to move the discussion here.
>
> My concern about the monolithic nature of each algo module is not only
> about SIMD, and it has nothing to do with weird systems. It has to do
> with micro-architectural differences which are more common on ARM than
> on other architectures *, I suppose. But generalizing from that, it
> has to do with policy which is currently owned by userland and not by
> the kernel. This will also be important for choosing between the time
> variant but less safe table based scalar AES and the much slower time
> invariant version (which is substantially slower, especially on
> decryption) once we move AES into this library.
>
> So a command line option for the kernel is not the solution here. If
> we can't have separate modules, could we at least have per-module
> options that put the policy decisions back into userland?
>
> * as an example, the SHA256 NEON code I collaborated on with Andy
> Polyakov 2 years ago is significantly faster on some cores and not on
> others
Interesting concern. There are micro-architectural quirks on x86 too
that the current code actually already considers. Notably, we use an
AVX-512VL path for Skylake-X but an AVX-512F path for Knights Landing
and Coffee Lake and others, due to thermal throttling when touching the
zmm registers on Skylake-X. So, in the code, we have it automatically
select the right thing based on the micro-architecture.
Is the same thing not possible with ARM? Do you not have access to this
information already, such that the module can just always do the right
thing and not require any user intervention?
If so, that would be ideal. If not (and I'm curious to learn why not
exactly), then indeed we could add some runtime nobs in /sys/module/
{algo}/parameters/{nob}, or the like. This would be super easy to do,
should we ever encounter a situation where we're unable to auto-detect
the correct thing.
Regards,
Jason
^ permalink raw reply
* Re: [PATCH net-next v3 02/17] zinc: introduce minimal cryptography library
From: Jason A. Donenfeld @ 2018-09-18 20:36 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Andrew Lutomirski, David Miller, Andrew Lunn, Eric Biggers,
Greg Kroah-Hartman, LKML, Netdev, Samuel Neves,
Jean-Philippe Aumasson, Linux Crypto Mailing List
In-Reply-To: <CAKv+Gu8OKyVvvgLy48GDV2KJt-UmmHbf5x0qNxQC_SDiddb9-g@mail.gmail.com>
Hi Ard,
On Tue, Sep 18, 2018 at 11:53:11AM -0700, Ard Biesheuvel wrote:
> On 17 September 2018 at 08:52, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> > Hi Ard,
> >
>
> Given that you show no interest whatsoever in gaining an understanding
> of the underlying requirements that we have to deal with in the crypto
> API, the only way to get my point across is by repeatedly stating it
Sorry if I've come across that way, but I am certainly interested in
gaining such an understanding of said requirements.
> I have pointed out to you numerous times (as has Eric) that the
> ChaCha20 ARM code you are importing from the OpenSSL project has been
> found to be slower on Cortex-A7, which represents the vast majority of
> devices expected to be in the field in 1~2 years.
I mentioned in the other thread that I intend immanently to begin
benching on a variety of ARM boards, and I should have numbers and
results and conclusions somewhat soon for the list.
My initial notion here was that it'd be better to go with AndyP's code
no matter what, and then later work with him on contributing said
improvements, and then port these back to the kernel. However, you and
Andy made a compelling point about code replacement -- that it's okay to
replace all in one go only if there are positive benchmark results. So I
think what I'll do to appease this is -- if the benchmarks are indeed
how Eric suggested -- stick with your faster code, and then follow up
with replacement plans after the merge. (I feel a bit more comfortable
with varying ChaCha code, because implementations tend to be pretty
straight forward and harder to screw up in subtle ways than, say,
poly1305 or curve25519.)
> have asked you more than once to split out your changes to the
> upstream OpenSSL code into separate patches so we can more easily
> track them, but v5 now puts them in the commit log (again) [but in a
> corrupted way - the preprocessor directives are filtered out by
> git-commit], which means we cannot use git diff/blame etc to look at
> them.
Didnt't realize this was so important to you. It's trivial to do, so
I'll do that for AndyP's implementations for the next revision.
> Upstreaming code is about taking an interest in other people's use
> cases, and about choosing your battles. It is unfortunate that we have
> spent all this time talking about a couple of crypto routines, while
> the actual meat of your submission is in WireGuard itself, which I'm
> sure you much rather talk about.
I don't find it unfortunate; getting the crypto right is of the utmost
importance.
Regards,
Jason
^ permalink raw reply
* RE: [Intel-wired-lan] [PATCH net-next v2 02/14] iavf: diet and reformat
From: Bowers, AndrewX @ 2018-09-18 14:57 UTC (permalink / raw)
To: Brandeburg, Jesse, netdev@vger.kernel.org,
intel-wired-lan@lists.osuosl.org
In-Reply-To: <20180915003757.169108-3-jesse.brandeburg@intel.com>
> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces@osuosl.org] On
> Behalf Of Jesse Brandeburg
> Sent: Friday, September 14, 2018 5:38 PM
> To: netdev@vger.kernel.org; intel-wired-lan@lists.osuosl.org
> Subject: [Intel-wired-lan] [PATCH net-next v2 02/14] iavf: diet and reformat
>
> Remove a bunch of unused code and reformat a few lines. Also
> remove some now un-necessary files.
>
> Signed-off-by: Jesse Brandeburg <jesse.brandeburg@intel.com>
> ---
> drivers/net/ethernet/intel/iavf/i40e_adminq.c | 27 -
> drivers/net/ethernet/intel/iavf/i40e_adminq_cmd.h | 2277 +------------------
> --
> drivers/net/ethernet/intel/iavf/i40e_common.c | 337 ---
> drivers/net/ethernet/intel/iavf/i40e_hmc.h | 215 --
> drivers/net/ethernet/intel/iavf/i40e_lan_hmc.h | 158 --
> drivers/net/ethernet/intel/iavf/i40e_prototype.h | 65 +-
> drivers/net/ethernet/intel/iavf/i40e_register.h | 245 ---
> drivers/net/ethernet/intel/iavf/i40e_type.h | 783 +------
> 8 files changed, 50 insertions(+), 4057 deletions(-)
> delete mode 100644 drivers/net/ethernet/intel/iavf/i40e_hmc.h
> delete mode 100644 drivers/net/ethernet/intel/iavf/i40e_lan_hmc.h
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
^ permalink raw reply
* Re: [PATCH net-next v5 06/20] zinc: ChaCha20 MIPS32r2 implementation
From: Paul Burton @ 2018-09-18 20:25 UTC (permalink / raw)
To: Jason A. Donenfeld
Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
linux-crypto@vger.kernel.org, davem@davemloft.net,
gregkh@linuxfoundation.org, René van Dorst, Samuel Neves,
Andy Lutomirski, Jean-Philippe Aumasson, Ralf Baechle,
James Hogan, linux-mips@linux-mips.org
In-Reply-To: <20180918161646.19105-7-Jason@zx2c4.com>
Hi Jason,
On Tue, Sep 18, 2018 at 06:16:32PM +0200, Jason A. Donenfeld wrote:
> +.Lchacha20_mips_xor_aligned_4_b:
> + /* STORE_ALIGNED( 4, 0, $sp, 0+CONSTANT_OFS_SP) */
> + lw T0, 0+CONSTANT_OFS_SP($sp)
> + lw T1, 0(IN)
> + addu X0, T0
> + CPU_TO_LE32(X0)
> + xor X0, T1
> + .set noreorder
> + bne OUT, PTR_LAST_ROUND, .Loop_chacha20_rounds
> + sw X0, 0(OUT)
> + .set reorder
> +
> + .set noreorder
> + bne $at, BYTES, .Lchacha20_mips_xor_bytes
> + /* Empty delayslot, Increase NONCE_0, return NONCE_0 value */
> + addiu NONCE_0, 1
> + .set noreorder
Should this be .set reorder?
Even better - could we not just place the addiu before the bne & drop
the .set noreorder, allowing the assembler to fill the delay slot with
the addiu? Likewise in many other places throughout the patch.
That would be more future proof - particularly if we ever want to adjust
this for use with the nanoMIPS ISA which has no delay slots. It may also
allow the assembler the choice to use compact branches (ie. branches
without visible delay slots) when targeting MIPS32r6. I know neither of
these will currently build this code, but I think avoiding all the
noreorder blocks would be a nice cleanup just for the sake of
readability anyway.
Thanks,
Paul
^ permalink raw reply
* Re: linux-next: Tree for Sep 18 (kernel/bpf/syscall)
From: Willem de Bruijn @ 2018-09-18 20:22 UTC (permalink / raw)
To: Daniel Borkmann
Cc: rdunlap, Stephen Rothwell, Linux-Next Mailing List, LKML,
Network Development, Alexei Starovoitov, Petar Penkov,
Willem de Bruijn
In-Reply-To: <CAF=yD-Kv7ig9UEo8Ci3eZOV=spa9cbGA5Mfa5Too8TgUzVA3fA@mail.gmail.com>
On Tue, Sep 18, 2018 at 12:28 PM Willem de Bruijn
<willemdebruijn.kernel@gmail.com> wrote:
>
> On Tue, Sep 18, 2018 at 11:44 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >
> > [ Cc Petar and Willem ]
> >
> > On 09/18/2018 05:37 PM, Randy Dunlap wrote:
> > > On 9/17/18 10:12 PM, Stephen Rothwell wrote:
> > >> Hi all,
> > >>
> > >> Changes since 20180913:
> > >
> > > on i386 or x86_64: (in 6 of 20 randconfigs)
> > >
> > > kernel/bpf/syscall.o: In function `__x64_sys_bpf':
> > > syscall.c:(.text+0x3278): undefined reference to `skb_flow_dissector_bpf_prog_attach'
> > > syscall.c:(.text+0x3310): undefined reference to `skb_flow_dissector_bpf_prog_detach'
> > > kernel/bpf/syscall.o:(.rodata+0x3f0): undefined reference to `flow_dissector_prog_ops'
> > > kernel/bpf/verifier.o:(.rodata+0x250): undefined reference to `flow_dissector_verifier_ops'
> > >
> > > Full randconfig file is attached.
> >
> > Looks like we need a wrapper for these, config had:
> >
> > # CONFIG_NET is not set
>
> Thanks for forwarding the report.
>
> For the functions, I think we can use a similar static inline stub as
> used e.g., for bpf_map_offload_map_alloc.
>
> The _ops references are from a macro that includes linux/bpf_types.h,
> so indeed a CONFIG looks needed.
>
> I need to look it over before I send out, but tentative patch:
Sent http://patchwork.ozlabs.org/patch/971280/
^ permalink raw reply
* Re: [PATCH] net: apm: xgene: force XGene enet driver to re-balance IRQ usage
From: Al Stone @ 2018-09-18 20:21 UTC (permalink / raw)
To: David Miller; +Cc: netdev, linux-kernel, isubramanian, kchudgar, qnguyen
In-Reply-To: <20180917.193531.1424852853827678152.davem@davemloft.net>
On 09/17/2018 08:35 PM, David Miller wrote:
> From: Al Stone <ahs3@redhat.com>
> Date: Mon, 17 Sep 2018 17:35:33 -0600
>
>> @@ -866,8 +866,11 @@ static int xgene_enet_napi(struct napi_struct *napi, const int budget)
>> processed = xgene_enet_process_ring(ring, budget);
>>
>> if (processed != budget) {
>> + struct irq_desc *desc = irq_to_desc(ring->irq);
>> +
>> napi_complete_done(napi, processed);
>> - enable_irq(ring->irq);
>> + if (desc && desc->depth > 0)
>> + enable_irq(ring->irq);
>
> I really can't accept a patch that grovels into IRQ layer internals
> to work around a driver's IRQ enable/disable usage problem.
>
> Sorry.
No worries. I hesitated even sending it, actually. The rewrite of the driver
that is needed is just something no one seems to have any interest in, or any
time for, but it really needs it to fix this properly (a colleague has found at
least one other issue in the structure of this driver, for example). The cost
of doing that rewrite/restructure, though, far outweighs the benefit for this
device -- especially when the workaround is "don't run tuned".
I'll put a proper fix back on the list of Interesting Things to Work On When
Time Allows, for now.
Thanks for the quick -- and kind :) -- response. I really appreciate it.
--
ciao,
al
-----------------------------------
Al Stone
Software Engineer
Red Hat, Inc.
ahs3@redhat.com
-----------------------------------
^ permalink raw reply
* [PATCH 4/4] media: bpf: add bpf function to report mouse movement
From: Sean Young @ 2018-09-18 14:29 UTC (permalink / raw)
To: linux-media; +Cc: netdev
In-Reply-To: <20180918142930.6686-1-sean@mess.org>
Some IR remotes have a directional pad or other pointer-like thing that
can be used as a mouse. Make it possible to decode these types of IR
protocols in BPF.
Cc: netdev@vger.kernel.org
Signed-off-by: Sean Young <sean@mess.org>
---
drivers/media/rc/bpf-lirc.c | 24 +++++++++++++++++++++++
include/uapi/linux/bpf.h | 17 +++++++++++++++-
tools/testing/selftests/bpf/bpf_helpers.h | 2 ++
3 files changed, 42 insertions(+), 1 deletion(-)
diff --git a/drivers/media/rc/bpf-lirc.c b/drivers/media/rc/bpf-lirc.c
index 8b97fd1f0cea..390a722e6211 100644
--- a/drivers/media/rc/bpf-lirc.c
+++ b/drivers/media/rc/bpf-lirc.c
@@ -59,6 +59,28 @@ static const struct bpf_func_proto rc_keydown_proto = {
.arg4_type = ARG_ANYTHING,
};
+BPF_CALL_3(bpf_rc_pointer_rel, u32*, sample, s32, rel_x, s32, rel_y)
+{
+ struct ir_raw_event_ctrl *ctrl;
+
+ ctrl = container_of(sample, struct ir_raw_event_ctrl, bpf_sample);
+
+ input_report_rel(ctrl->dev->input_dev, REL_X, rel_x);
+ input_report_rel(ctrl->dev->input_dev, REL_Y, rel_y);
+ input_sync(ctrl->dev->input_dev);
+
+ return 0;
+}
+
+static const struct bpf_func_proto rc_pointer_rel_proto = {
+ .func = bpf_rc_pointer_rel,
+ .gpl_only = true,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_CTX,
+ .arg2_type = ARG_ANYTHING,
+ .arg3_type = ARG_ANYTHING,
+};
+
static const struct bpf_func_proto *
lirc_mode2_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
{
@@ -67,6 +89,8 @@ lirc_mode2_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
return &rc_repeat_proto;
case BPF_FUNC_rc_keydown:
return &rc_keydown_proto;
+ case BPF_FUNC_rc_pointer_rel:
+ return &rc_pointer_rel_proto;
case BPF_FUNC_map_lookup_elem:
return &bpf_map_lookup_elem_proto;
case BPF_FUNC_map_update_elem:
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 66917a4eba27..20d6750b687c 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -2141,6 +2141,20 @@ union bpf_attr {
* request in the skb.
* Return
* 0 on success, or a negative error in case of failure.
+ *
+ * int bpf_rc_pointer_rel(void *ctx, s32 rel_x, s32 rel_y)
+ * Description
+ * This helper is used in programs implementing IR decoding, to
+ * report a successfully decoded pointer movement.
+ *
+ * The *ctx* should point to the lirc sample as passed into
+ * the program.
+ *
+ * This helper is only available is the kernel was compiled with
+ * the **CONFIG_BPF_LIRC_MODE2** configuration option set to
+ * "**y**".
+ * Return
+ * 0
*/
#define __BPF_FUNC_MAPPER(FN) \
FN(unspec), \
@@ -2226,7 +2240,8 @@ union bpf_attr {
FN(get_current_cgroup_id), \
FN(get_local_storage), \
FN(sk_select_reuseport), \
- FN(skb_ancestor_cgroup_id),
+ FN(skb_ancestor_cgroup_id), \
+ FN(rc_pointer_rel),
/* integer value in 'imm' field of BPF_CALL instruction selects which helper
* function eBPF program intends to call
diff --git a/tools/testing/selftests/bpf/bpf_helpers.h b/tools/testing/selftests/bpf/bpf_helpers.h
index e4be7730222d..edda498c1231 100644
--- a/tools/testing/selftests/bpf/bpf_helpers.h
+++ b/tools/testing/selftests/bpf/bpf_helpers.h
@@ -143,6 +143,8 @@ static unsigned long long (*bpf_skb_cgroup_id)(void *ctx) =
(void *) BPF_FUNC_skb_cgroup_id;
static unsigned long long (*bpf_skb_ancestor_cgroup_id)(void *ctx, int level) =
(void *) BPF_FUNC_skb_ancestor_cgroup_id;
+static int (*bpf_rc_pointer_rel)(void *ctx, int rel_x, int rel_y) =
+ (void *) BPF_FUNC_rc_pointer_rel;
/* llvm builtin functions that eBPF C program may use to
* emit BPF_LD_ABS and BPF_LD_IND instructions
--
2.17.1
^ permalink raw reply related
* Re: [PATCH][net-next] netlink: avoid to allocate full skb when sending to many devices
From: Eric Dumazet @ 2018-09-18 14:08 UTC (permalink / raw)
To: Li RongQing, netdev
In-Reply-To: <1537248415-15534-1-git-send-email-lirongqing@baidu.com>
On 09/17/2018 10:26 PM, Li RongQing wrote:
> if skb->head is vmalloc address, when this skb is delivered, full
> allocation for this skb is required, if there are many devices,
> the full allocation will be called for every devices
>
> now using the first time allocated skb when iterate other devices
> to send, reduce full allocation and speedup deliver.
>
> Signed-off-by: Zhang Yu <zhangyu31@baidu.com>
> Signed-off-by: Li RongQing <lirongqing@baidu.com>
> ---
> net/netlink/af_netlink.c | 14 ++++++++------
> 1 file changed, 8 insertions(+), 6 deletions(-)
>
>
This looks very broken to me.
Only the original skb (given as an argument to __netlink_deliver_tap()) is guaranteed to not
disappear while the loop is performed.
(There is no skb_clone() after the first netlink_to_full_skb())
^ permalink raw reply
* [PATCH net 3/5] net/smc: enable fallback for connection abort in state INIT
From: Ursula Braun @ 2018-09-18 13:46 UTC (permalink / raw)
To: davem; +Cc: netdev, linux-s390, schwidefsky, heiko.carstens, raspl,
linux-kernel
In-Reply-To: <20180918134638.90271-1-ubraun@linux.ibm.com>
If a linkgroup is terminated abnormally already due to failing
LLC CONFIRM LINK or LLC ADD LINK, fallback to TCP is still possible.
In this case do not switch to state SMC_PEERABORTWAIT and do not set
sk_err.
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
---
net/smc/smc_close.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/net/smc/smc_close.c b/net/smc/smc_close.c
index ac961dfb1ea1..ea2b87f29469 100644
--- a/net/smc/smc_close.c
+++ b/net/smc/smc_close.c
@@ -100,15 +100,14 @@ static void smc_close_active_abort(struct smc_sock *smc)
struct smc_cdc_conn_state_flags *txflags =
&smc->conn.local_tx_ctrl.conn_state_flags;
- sk->sk_err = ECONNABORTED;
- if (smc->clcsock && smc->clcsock->sk) {
- smc->clcsock->sk->sk_err = ECONNABORTED;
- smc->clcsock->sk->sk_state_change(smc->clcsock->sk);
+ if (sk->sk_state != SMC_INIT && smc->clcsock && smc->clcsock->sk) {
+ sk->sk_err = ECONNABORTED;
+ if (smc->clcsock && smc->clcsock->sk) {
+ smc->clcsock->sk->sk_err = ECONNABORTED;
+ smc->clcsock->sk->sk_state_change(smc->clcsock->sk);
+ }
}
switch (sk->sk_state) {
- case SMC_INIT:
- sk->sk_state = SMC_PEERABORTWAIT;
- break;
case SMC_ACTIVE:
sk->sk_state = SMC_PEERABORTWAIT;
release_sock(sk);
@@ -143,6 +142,7 @@ static void smc_close_active_abort(struct smc_sock *smc)
case SMC_PEERFINCLOSEWAIT:
sock_put(sk); /* passive closing */
break;
+ case SMC_INIT:
case SMC_PEERABORTWAIT:
case SMC_CLOSED:
break;
--
2.16.4
^ permalink raw reply related
* [PATCH net 2/5] net/smc: remove duplicate mutex_unlock
From: Ursula Braun @ 2018-09-18 13:46 UTC (permalink / raw)
To: davem; +Cc: netdev, linux-s390, schwidefsky, heiko.carstens, raspl,
linux-kernel
In-Reply-To: <20180918134638.90271-1-ubraun@linux.ibm.com>
For a failing smc_listen_rdma_finish() smc_listen_decline() is
called. If fallback is possible, the new socket is already enqueued
to be accepted in smc_listen_decline(). Avoid enqueuing a second time
afterwards in this case, otherwise the smc_create_lgr_pending lock
is released twice:
[ 373.463976] WARNING: bad unlock balance detected!
[ 373.463978] 4.18.0-rc7+ #123 Tainted: G O
[ 373.463979] -------------------------------------
[ 373.463980] kworker/1:1/30 is trying to release lock (smc_create_lgr_pending) at:
[ 373.463990] [<000003ff801205fc>] smc_listen_work+0x22c/0x5d0 [smc]
[ 373.463991] but there are no more locks to release!
[ 373.463991]
other info that might help us debug this:
[ 373.463993] 2 locks held by kworker/1:1/30:
[ 373.463994] #0: 00000000772cbaed ((wq_completion)"events"){+.+.}, at: process_one_work+0x1ec/0x6b0
[ 373.464000] #1: 000000003ad0894a ((work_completion)(&new_smc->smc_listen_work)){+.+.}, at: process_one_work+0x1ec/0x6b0
[ 373.464003]
stack backtrace:
[ 373.464005] CPU: 1 PID: 30 Comm: kworker/1:1 Kdump: loaded Tainted: G O 4.18.0-rc7uschi+ #123
[ 373.464007] Hardware name: IBM 2827 H43 738 (LPAR)
[ 373.464010] Workqueue: events smc_listen_work [smc]
[ 373.464011] Call Trace:
[ 373.464015] ([<0000000000114100>] show_stack+0x60/0xd8)
[ 373.464019] [<0000000000a8c9bc>] dump_stack+0x9c/0xd8
[ 373.464021] [<00000000001dcaf8>] print_unlock_imbalance_bug+0xf8/0x108
[ 373.464022] [<00000000001e045c>] lock_release+0x114/0x4f8
[ 373.464025] [<0000000000aa87fa>] __mutex_unlock_slowpath+0x4a/0x300
[ 373.464027] [<000003ff801205fc>] smc_listen_work+0x22c/0x5d0 [smc]
[ 373.464029] [<0000000000197a68>] process_one_work+0x2a8/0x6b0
[ 373.464030] [<0000000000197ec2>] worker_thread+0x52/0x410
[ 373.464033] [<000000000019fd0e>] kthread+0x15e/0x178
[ 373.464035] [<0000000000aaf58a>] kernel_thread_starter+0x6/0xc
[ 373.464052] [<0000000000aaf584>] kernel_thread_starter+0x0/0xc
[ 373.464054] INFO: lockdep is turned off.
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
---
net/smc/af_smc.c | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c
index 9c3976bcde46..5c6d30eb4a71 100644
--- a/net/smc/af_smc.c
+++ b/net/smc/af_smc.c
@@ -1153,9 +1153,9 @@ static int smc_listen_rdma_reg(struct smc_sock *new_smc, int local_contact)
}
/* listen worker: finish RDMA setup */
-static void smc_listen_rdma_finish(struct smc_sock *new_smc,
- struct smc_clc_msg_accept_confirm *cclc,
- int local_contact)
+static int smc_listen_rdma_finish(struct smc_sock *new_smc,
+ struct smc_clc_msg_accept_confirm *cclc,
+ int local_contact)
{
struct smc_link *link = &new_smc->conn.lgr->lnk[SMC_SINGLE_LINK];
int reason_code = 0;
@@ -1178,11 +1178,12 @@ static void smc_listen_rdma_finish(struct smc_sock *new_smc,
if (reason_code)
goto decline;
}
- return;
+ return 0;
decline:
mutex_unlock(&smc_create_lgr_pending);
smc_listen_decline(new_smc, reason_code, local_contact);
+ return reason_code;
}
/* setup for RDMA connection of server */
@@ -1279,8 +1280,10 @@ static void smc_listen_work(struct work_struct *work)
}
/* finish worker */
- if (!ism_supported)
- smc_listen_rdma_finish(new_smc, &cclc, local_contact);
+ if (!ism_supported) {
+ if (smc_listen_rdma_finish(new_smc, &cclc, local_contact))
+ return;
+ }
smc_conn_save_peer_info(new_smc, &cclc);
mutex_unlock(&smc_create_lgr_pending);
smc_listen_out_connected(new_smc);
--
2.16.4
^ permalink raw reply related
* [PATCH net 1/5] net/smc: fix non-blocking connect problem
From: Ursula Braun @ 2018-09-18 13:46 UTC (permalink / raw)
To: davem; +Cc: netdev, linux-s390, schwidefsky, heiko.carstens, raspl,
linux-kernel
In-Reply-To: <20180918134638.90271-1-ubraun@linux.ibm.com>
In state SMC_INIT smc_poll() delegates polling to the internal
CLC socket. This means, once the connect worker has finished
its kernel_connect() step, the poll wake-up may occur. This is not
intended. The wake-up should occur from the wake up call in
smc_connect_work() after __smc_connect() has finished.
Thus in state SMC_INIT this patch now calls sock_poll_wait() on the
main SMC socket.
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
---
net/smc/af_smc.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c
index 2d8a1e15e4f9..9c3976bcde46 100644
--- a/net/smc/af_smc.c
+++ b/net/smc/af_smc.c
@@ -742,7 +742,10 @@ static void smc_connect_work(struct work_struct *work)
smc->sk.sk_err = -rc;
out:
- smc->sk.sk_state_change(&smc->sk);
+ if (smc->sk.sk_err)
+ smc->sk.sk_state_change(&smc->sk);
+ else
+ smc->sk.sk_write_space(&smc->sk);
kfree(smc->connect_info);
smc->connect_info = NULL;
release_sock(&smc->sk);
@@ -1529,7 +1532,7 @@ static __poll_t smc_poll(struct file *file, struct socket *sock,
return EPOLLNVAL;
smc = smc_sk(sock->sk);
- if ((sk->sk_state == SMC_INIT) || smc->use_fallback) {
+ if (smc->use_fallback) {
/* delegate to CLC child sock */
mask = smc->clcsock->ops->poll(file, smc->clcsock, wait);
sk->sk_err = smc->clcsock->sk->sk_err;
--
2.16.4
^ permalink raw reply related
* [PATCH net 0/5] net/smc: fixes 2018-09-18
From: Ursula Braun @ 2018-09-18 13:46 UTC (permalink / raw)
To: davem; +Cc: netdev, linux-s390, schwidefsky, heiko.carstens, raspl,
linux-kernel
Dave,
here are some fixes in different areas of the smc code for the net
tree.
Thanks, Ursula
Karsten Graul (1):
net/smc: no urgent data check for listen sockets
Ursula Braun (3):
net/smc: fix non-blocking connect problem
net/smc: remove duplicate mutex_unlock
net/smc: enable fallback for connection abort in state INIT
YueHaibing (1):
net/smc: fix sizeof to int comparison
net/smc/af_smc.c | 26 ++++++++++++++++----------
net/smc/smc_clc.c | 14 ++++++--------
net/smc/smc_close.c | 14 +++++++-------
3 files changed, 29 insertions(+), 25 deletions(-)
--
2.16.4
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox