* Re: [PATCH net-next v5 00/20] WireGuard: Secure Network Tunnel
From: Jason A. Donenfeld @ 2018-09-18 21:01 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Linux Kernel Mailing List, <netdev@vger.kernel.org>,
open list:HARDWARE RANDOM NUMBER GENERATOR CORE, David S. Miller,
Greg Kroah-Hartman
In-Reply-To: <CAKv+Gu8doWS+BYzqoT7499CJX0hLm1jBSMfVxJoG3wsWFO7t1w@mail.gmail.com>
Hi Ard,
On Tue, Sep 18, 2018 at 11:28:50AM -0700, Ard Biesheuvel wrote:
> On 18 September 2018 at 09:16, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> > - While I initially wasn't going to do this for the initial
> > patchset, it was just so simple to do: now there's a nosimd
> > module parameter that can be used to disable simd instructions
> > for debugging and testing, or on weird systems.
> >
>
> I was going to respond in the other thread but it is probably better
> to move the discussion here.
>
> My concern about the monolithic nature of each algo module is not only
> about SIMD, and it has nothing to do with weird systems. It has to do
> with micro-architectural differences which are more common on ARM than
> on other architectures *, I suppose. But generalizing from that, it
> has to do with policy which is currently owned by userland and not by
> the kernel. This will also be important for choosing between the time
> variant but less safe table based scalar AES and the much slower time
> invariant version (which is substantially slower, especially on
> decryption) once we move AES into this library.
>
> So a command line option for the kernel is not the solution here. If
> we can't have separate modules, could we at least have per-module
> options that put the policy decisions back into userland?
>
> * as an example, the SHA256 NEON code I collaborated on with Andy
> Polyakov 2 years ago is significantly faster on some cores and not on
> others
Interesting concern. There are micro-architectural quirks on x86 too
that the current code actually already considers. Notably, we use an
AVX-512VL path for Skylake-X but an AVX-512F path for Knights Landing
and Coffee Lake and others, due to thermal throttling when touching the
zmm registers on Skylake-X. So, in the code, we have it automatically
select the right thing based on the micro-architecture.
Is the same thing not possible with ARM? Do you not have access to this
information already, such that the module can just always do the right
thing and not require any user intervention?
If so, that would be ideal. If not (and I'm curious to learn why not
exactly), then indeed we could add some runtime nobs in /sys/module/
{algo}/parameters/{nob}, or the like. This would be super easy to do,
should we ever encounter a situation where we're unable to auto-detect
the correct thing.
Regards,
Jason
^ permalink raw reply
* Re: [PATCH bpf-next v2] tools/bpf: bpftool: improve output format for bpftool net
From: Daniel Borkmann @ 2018-09-18 15:48 UTC (permalink / raw)
To: Yonghong Song, ast, netdev; +Cc: kernel-team
In-Reply-To: <20180917231300.1188382-1-yhs@fb.com>
On 09/18/2018 01:13 AM, Yonghong Song wrote:
> This is a followup patch for Commit f6f3bac08ff9
> ("tools/bpf: bpftool: add net support").
> Some improvements are made for the bpftool net output.
> Specially, plain output is more concise such that
> per attachment should nicely fit in one line.
> Compared to previous output, the prog tag is removed
> since it can be easily obtained with program id.
> Similar to xdp attachments, the device name is added
> to tc attachments.
>
> The bpf program attached through shared block
> mechanism is supported as well.
> $ ip link add dev v1 type veth peer name v2
> $ tc qdisc add dev v1 ingress_block 10 egress_block 20 clsact
> $ tc qdisc add dev v2 ingress_block 10 egress_block 20 clsact
> $ tc filter add block 10 protocol ip prio 25 bpf obj bpf_shared.o sec ingress flowid 1:1
> $ tc filter add block 20 protocol ip prio 30 bpf obj bpf_cyclic.o sec classifier flowid 1:1
> $ bpftool net
> xdp:
>
> tc:
> v2(7) clsact/ingress bpf_shared.o:[ingress] id 23
> v2(7) clsact/egress bpf_cyclic.o:[classifier] id 24
> v1(8) clsact/ingress bpf_shared.o:[ingress] id 23
> v1(8) clsact/egress bpf_cyclic.o:[classifier] id 24
>
> The documentation and "bpftool net help" are updated
> to make it clear that current implementation only
> supports xdp and tc attachments. For programs
> attached to cgroups, "bpftool cgroup" can be used
> to dump attachments. For other programs e.g.
> sk_{filter,skb,msg,reuseport} and lwt/seg6,
> iproute2 tools should be used.
>
> The new output:
> $ bpftool net
> xdp:
> eth0(2) driver id 198
>
> tc:
> eth0(2) clsact/ingress fbflow_icmp id 335 act [{icmp_action id 336}]
> eth0(2) clsact/egress fbflow_egress id 334
> $ bpftool -jp net
> [{
> "xdp": [{
> "devname": "eth0",
> "ifindex": 2,
> "mode": "driver",
> "id": 198
> }
> ],
> "tc": [{
> "devname": "eth0",
> "ifindex": 2,
> "kind": "clsact/ingress",
> "name": "fbflow_icmp",
> "id": 335,
> "act": [{
> "name": "icmp_action",
> "id": 336
> }
> ]
> },{
> "devname": "eth0",
> "ifindex": 2,
> "kind": "clsact/egress",
> "name": "fbflow_egress",
> "id": 334
> }
> ]
> }
> ]
>
> Signed-off-by: Yonghong Song <yhs@fb.com>
Applied to bpf-next, thanks!
^ permalink raw reply
* [PATCH net] sfp: fix oops with ethtool -m
From: Russell King @ 2018-09-18 15:48 UTC (permalink / raw)
To: David S. Miller; +Cc: Andrew Lunn, Florian Fainelli, netdev
If a network interface is created prior to the SFP socket being
available, ethtool can request module information. This unfortunately
leads to an oops:
Unable to handle kernel NULL pointer dereference at virtual address 00000008
pgd = (ptrval)
[00000008] *pgd=7c400831, *pte=00000000, *ppte=00000000
Internal error: Oops: 17 [#1] SMP ARM
Modules linked in:
CPU: 0 PID: 1480 Comm: ethtool Not tainted 4.19.0-rc3 #138
Hardware name: Broadcom Northstar Plus SoC
PC is at sfp_get_module_info+0x8/0x10
LR is at dev_ethtool+0x218c/0x2afc
Fix this by not filling in the network device's SFP bus pointer until
SFP is fully bound, thereby avoiding the core calling into the SFP bus
code.
Fixes: ce0aa27ff3f6 ("sfp: add sfp-bus to bridge between network devices and sfp cages")
Reported-by: Florian Fainelli <f.fainelli@gmail.com>
Tested-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
---
drivers/net/phy/sfp-bus.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/phy/sfp-bus.c b/drivers/net/phy/sfp-bus.c
index 740655261e5b..83060fb349f4 100644
--- a/drivers/net/phy/sfp-bus.c
+++ b/drivers/net/phy/sfp-bus.c
@@ -349,6 +349,7 @@ static int sfp_register_bus(struct sfp_bus *bus)
}
if (bus->started)
bus->socket_ops->start(bus->sfp);
+ bus->netdev->sfp_bus = bus;
bus->registered = true;
return 0;
}
@@ -357,6 +358,7 @@ static void sfp_unregister_bus(struct sfp_bus *bus)
{
const struct sfp_upstream_ops *ops = bus->upstream_ops;
+ bus->netdev->sfp_bus = NULL;
if (bus->registered) {
if (bus->started)
bus->socket_ops->stop(bus->sfp);
@@ -438,7 +440,6 @@ static void sfp_upstream_clear(struct sfp_bus *bus)
{
bus->upstream_ops = NULL;
bus->upstream = NULL;
- bus->netdev->sfp_bus = NULL;
bus->netdev = NULL;
}
@@ -467,7 +468,6 @@ struct sfp_bus *sfp_register_upstream(struct fwnode_handle *fwnode,
bus->upstream_ops = ops;
bus->upstream = upstream;
bus->netdev = ndev;
- ndev->sfp_bus = bus;
if (bus->sfp) {
ret = sfp_register_bus(bus);
--
2.7.4
^ permalink raw reply related
* Re: [PATCH bpf-next] samples/bpf: fix a compilation failure
From: Daniel Borkmann @ 2018-09-18 15:51 UTC (permalink / raw)
To: Yonghong Song, ast, netdev; +Cc: kernel-team
In-Reply-To: <20180918050813.2138154-1-yhs@fb.com>
On 09/18/2018 07:08 AM, Yonghong Song wrote:
> samples/bpf build failed with the following errors:
>
> $ make samples/bpf/
> ...
> HOSTCC samples/bpf/sockex3_user.o
> /data/users/yhs/work/net-next/samples/bpf/sockex3_user.c:16:8: error: redefinition of ‘struct bpf_flow_keys’
> struct bpf_flow_keys {
> ^
> In file included from /data/users/yhs/work/net-next/samples/bpf/sockex3_user.c:4:0:
> ./usr/include/linux/bpf.h:2338:9: note: originally defined here
> struct bpf_flow_keys *flow_keys;
> ^
> make[3]: *** [samples/bpf/sockex3_user.o] Error 1
>
> Commit d58e468b1112d ("flow_dissector: implements flow dissector BPF hook")
> introduced struct bpf_flow_keys in include/uapi/linux/bpf.h and hence
> caused the naming conflict with samples/bpf/sockex3_user.c.
>
> The fix is to rename struct bpf_flow_keys in samples/bpf/sockex3_user.c
> to flow_keys to avoid the conflict.
>
> Signed-off-by: Yonghong Song <yhs@fb.com>
Applied to bpf-next, thanks!
^ permalink raw reply
* Re: [PATCH v3] PCI: Reprogram bridge prefetch registers on resume
From: Bjorn Helgaas @ 2018-09-18 21:32 UTC (permalink / raw)
To: Daniel Drake
Cc: andy.shevchenko-VuQAYsv1563Yd54FQh9/CA,
linux-pm-u79uwXL29TY76Z2rM5mHXA, linux-pci-u79uwXL29TY76Z2rM5mHXA,
rafael.j.wysocki-ral2JQCrhuEAvxtiuMwx3w,
nic_swsd-Rasf1IRRPZFBDgjK7y7TUQ,
keith.busch-ral2JQCrhuEAvxtiuMwx3w, netdev-u79uwXL29TY76Z2rM5mHXA,
nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
bhelgaas-hpIqsD4AKlfQT0dZR+AlfA, rchang-eYqpPyKDWXRBDgjK7y7TUQ,
linux-6IF/jdPJHihWk0Htik3J/w, davem-fT/PcQaiUtIeIZ0/mPfg9Q,
jonathan.derrick-ral2JQCrhuEAvxtiuMwx3w,
hkallweit1-Re5JQEeQqe8AvxtiuMwx3w
In-Reply-To: <20180913033745.11178-1-drake-6IF/jdPJHihWk0Htik3J/w@public.gmane.org>
On Thu, Sep 13, 2018 at 11:37:45AM +0800, Daniel Drake wrote:
> On 38+ Intel-based Asus products, the nvidia GPU becomes unusable
> after S3 suspend/resume. The affected products include multiple
> generations of nvidia GPUs and Intel SoCs. After resume, nouveau logs
> many errors such as:
>
> fifo: fault 00 [READ] at 0000005555555000 engine 00 [GR] client 04
> [HUB/FE] reason 4a [] on channel -1 [007fa91000 unknown]
> DRM: failed to idle channel 0 [DRM]
>
> Similarly, the nvidia proprietary driver also fails after resume
> (black screen, 100% CPU usage in Xorg process). We shipped a sample
> to Nvidia for diagnosis, and their response indicated that it's a
> problem with the parent PCI bridge (on the Intel SoC), not the GPU.
>
> Runtime suspend/resume works fine, only S3 suspend is affected.
>
> We found a workaround: on resume, rewrite the Intel PCI bridge
> 'Prefetchable Base Upper 32 Bits' register (PCI_PREF_BASE_UPPER32). In
> the cases that I checked, this register has value 0 and we just have to
> rewrite that value.
>
> Linux already saves and restores PCI config space during suspend/resume,
> but this register was being skipped because upon resume, it already
> has value 0 (the correct, pre-suspend value).
>
> Intel appear to have previously acknowledged this behaviour and the
> requirement to rewrite this register.
> https://bugzilla.kernel.org/show_bug.cgi?id=116851#c23
>
> Based on that, rewrite the prefetch register values even when that
> appears unnecessary.
>
> We have confirmed this solution on all the affected models we have
> in-hands (X542UQ, UX533FD, X530UN, V272UN).
>
> Additionally, this solves an issue where r8169 MSI-X interrupts were
> broken after S3 suspend/resume on Asus X441UAR. This issue was recently
> worked around in commit 7bb05b85bc2d ("r8169: don't use MSI-X on
> RTL8106e"). It also fixes the same issue on RTL6186evl/8111evl on an
> Aimfor-tech laptop that we had not yet patched. I suspect it will also
> fix the issue that was worked around in commit 7c53a722459c ("r8169:
> don't use MSI-X on RTL8168g").
>
> Thomas Martitz reports that this change also solves an issue where
> the AMD Radeon Polaris 10 GPU on the HP Zbook 14u G5 is unresponsive
> after S3 suspend/resume.
>
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=201069
> Signed-off-by: Daniel Drake <drake@endlessm.com>
Applied with Rafael's and Peter's reviewed-by to pci/enumeration for v4.20.
Thanks for the the huge investigative effort!
> ---
> drivers/pci/pci.c | 25 +++++++++++++++++--------
> 1 file changed, 17 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 29ff9619b5fa..5d58220b6997 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -1289,12 +1289,12 @@ int pci_save_state(struct pci_dev *dev)
> EXPORT_SYMBOL(pci_save_state);
>
> static void pci_restore_config_dword(struct pci_dev *pdev, int offset,
> - u32 saved_val, int retry)
> + u32 saved_val, int retry, bool force)
> {
> u32 val;
>
> pci_read_config_dword(pdev, offset, &val);
> - if (val == saved_val)
> + if (!force && val == saved_val)
> return;
>
> for (;;) {
> @@ -1313,25 +1313,34 @@ static void pci_restore_config_dword(struct pci_dev *pdev, int offset,
> }
>
> static void pci_restore_config_space_range(struct pci_dev *pdev,
> - int start, int end, int retry)
> + int start, int end, int retry,
> + bool force)
> {
> int index;
>
> for (index = end; index >= start; index--)
> pci_restore_config_dword(pdev, 4 * index,
> pdev->saved_config_space[index],
> - retry);
> + retry, force);
> }
>
> static void pci_restore_config_space(struct pci_dev *pdev)
> {
> if (pdev->hdr_type == PCI_HEADER_TYPE_NORMAL) {
> - pci_restore_config_space_range(pdev, 10, 15, 0);
> + pci_restore_config_space_range(pdev, 10, 15, 0, false);
> /* Restore BARs before the command register. */
> - pci_restore_config_space_range(pdev, 4, 9, 10);
> - pci_restore_config_space_range(pdev, 0, 3, 0);
> + pci_restore_config_space_range(pdev, 4, 9, 10, false);
> + pci_restore_config_space_range(pdev, 0, 3, 0, false);
> + } else if (pdev->hdr_type == PCI_HEADER_TYPE_BRIDGE) {
> + pci_restore_config_space_range(pdev, 12, 15, 0, false);
> + /* Force rewriting of prefetch registers to avoid
> + * S3 resume issues on Intel PCI bridges that occur when
> + * these registers are not explicitly written.
> + */
> + pci_restore_config_space_range(pdev, 9, 11, 0, true);
> + pci_restore_config_space_range(pdev, 0, 8, 0, false);
> } else {
> - pci_restore_config_space_range(pdev, 0, 15, 0);
> + pci_restore_config_space_range(pdev, 0, 15, 0, false);
> }
> }
>
> --
> 2.17.1
>
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau
^ permalink raw reply
* [bpf PATCH v4 0/3] bpf, sockmap ESTABLISHED state only
From: John Fastabend @ 2018-09-18 16:01 UTC (permalink / raw)
To: edumazet, ast, daniel; +Cc: netdev
Eric noted that using the close callback is not sufficient
to catch all transitions from ESTABLISHED state to a LISTEN
state. So this series does two things. First, only allow
adding socks in ESTABLISH state and second use unhash callback
to catch tcp_disconnect() transitions.
v2: added check for ESTABLISH state in hash update sockmap as well
v3: Do not release lock from unhash in error path, no lock was
used in the first place. And drop not so useful code comments
v4: convert,
if (unhash()) return unhash(); return
to if (unhash()) unhash(); return;
Thanks for reviewing Yonghong I carried your ACKs forward.
---
John Fastabend (3):
bpf: sockmap only allow ESTABLISHED sock state
bpf: sockmap, fix transition through disconnect without close
bpf: test_maps, only support ESTABLISHED socks
kernel/bpf/sockmap.c | 91 ++++++++++++++++++++++++-------
tools/testing/selftests/bpf/test_maps.c | 10 ++-
2 files changed, 78 insertions(+), 23 deletions(-)
^ permalink raw reply
* [bpf PATCH v4 1/3] bpf: sockmap only allow ESTABLISHED sock state
From: John Fastabend @ 2018-09-18 16:01 UTC (permalink / raw)
To: edumazet, ast, daniel; +Cc: netdev
In-Reply-To: <20180918155826.22728.71788.stgit@john-Precision-Tower-5810>
After this patch we only allow socks that are in ESTABLISHED state or
are being added via a sock_ops event that is transitioning into an
ESTABLISHED state. By allowing sock_ops events we allow users to
manage sockmaps directly from sock ops programs. The two supported
sock_ops ops are BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB and
BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB.
Similar to TLS ULP this ensures sk_user_data is correct.
Reported-by: Eric Dumazet <edumazet@google.com>
Fixes: 1aa12bdf1bfb ("bpf: sockmap, add sock close() hook to remove socks")
Signed-off-by: John Fastabend <john.fastabend@gmail.com>
Acked-by: Yonghong Song <yhs@fb.com>
---
kernel/bpf/sockmap.c | 31 ++++++++++++++++++++++++++++++-
1 file changed, 30 insertions(+), 1 deletion(-)
diff --git a/kernel/bpf/sockmap.c b/kernel/bpf/sockmap.c
index 488ef96..1f97b55 100644
--- a/kernel/bpf/sockmap.c
+++ b/kernel/bpf/sockmap.c
@@ -2097,8 +2097,12 @@ static int sock_map_update_elem(struct bpf_map *map,
return -EINVAL;
}
+ /* ULPs are currently supported only for TCP sockets in ESTABLISHED
+ * state.
+ */
if (skops.sk->sk_type != SOCK_STREAM ||
- skops.sk->sk_protocol != IPPROTO_TCP) {
+ skops.sk->sk_protocol != IPPROTO_TCP ||
+ skops.sk->sk_state != TCP_ESTABLISHED) {
fput(socket->file);
return -EOPNOTSUPP;
}
@@ -2453,6 +2457,16 @@ static int sock_hash_update_elem(struct bpf_map *map,
return -EINVAL;
}
+ /* ULPs are currently supported only for TCP sockets in ESTABLISHED
+ * state.
+ */
+ if (skops.sk->sk_type != SOCK_STREAM ||
+ skops.sk->sk_protocol != IPPROTO_TCP ||
+ skops.sk->sk_state != TCP_ESTABLISHED) {
+ fput(socket->file);
+ return -EOPNOTSUPP;
+ }
+
lock_sock(skops.sk);
preempt_disable();
rcu_read_lock();
@@ -2543,10 +2557,22 @@ struct sock *__sock_hash_lookup_elem(struct bpf_map *map, void *key)
.map_check_btf = map_check_no_btf,
};
+static bool bpf_is_valid_sock_op(struct bpf_sock_ops_kern *ops)
+{
+ return ops->op == BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB ||
+ ops->op == BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB;
+}
BPF_CALL_4(bpf_sock_map_update, struct bpf_sock_ops_kern *, bpf_sock,
struct bpf_map *, map, void *, key, u64, flags)
{
WARN_ON_ONCE(!rcu_read_lock_held());
+
+ /* ULPs are currently supported only for TCP sockets in ESTABLISHED
+ * state. This checks that the sock ops triggering the update is
+ * one indicating we are (or will be soon) in an ESTABLISHED state.
+ */
+ if (!bpf_is_valid_sock_op(bpf_sock))
+ return -EOPNOTSUPP;
return sock_map_ctx_update_elem(bpf_sock, map, key, flags);
}
@@ -2565,6 +2591,9 @@ struct sock *__sock_hash_lookup_elem(struct bpf_map *map, void *key)
struct bpf_map *, map, void *, key, u64, flags)
{
WARN_ON_ONCE(!rcu_read_lock_held());
+
+ if (!bpf_is_valid_sock_op(bpf_sock))
+ return -EOPNOTSUPP;
return sock_hash_ctx_update_elem(bpf_sock, map, key, flags);
}
^ permalink raw reply related
* [bpf PATCH v4 2/3] bpf: sockmap, fix transition through disconnect without close
From: John Fastabend @ 2018-09-18 16:01 UTC (permalink / raw)
To: edumazet, ast, daniel; +Cc: netdev
In-Reply-To: <20180918155826.22728.71788.stgit@john-Precision-Tower-5810>
It is possible (via shutdown()) for TCP socks to go trough TCP_CLOSE
state via tcp_disconnect() without actually calling tcp_close which
would then call our bpf_tcp_close() callback. Because of this a user
could disconnect a socket then put it in a LISTEN state which would
break our assumptions about sockets always being ESTABLISHED state.
To resolve this rely on the unhash hook, which is called in the
disconnect case, to remove the sock from the sockmap.
Reported-by: Eric Dumazet <edumazet@google.com>
Fixes: 1aa12bdf1bfb ("bpf: sockmap, add sock close() hook to remove socks")
Signed-off-by: John Fastabend <john.fastabend@gmail.com>
Acked-by: Yonghong Song <yhs@fb.com>
---
kernel/bpf/sockmap.c | 60 ++++++++++++++++++++++++++++++++++----------------
1 file changed, 41 insertions(+), 19 deletions(-)
diff --git a/kernel/bpf/sockmap.c b/kernel/bpf/sockmap.c
index 1f97b55..0a0f2ec 100644
--- a/kernel/bpf/sockmap.c
+++ b/kernel/bpf/sockmap.c
@@ -132,6 +132,7 @@ struct smap_psock {
struct work_struct gc_work;
struct proto *sk_proto;
+ void (*save_unhash)(struct sock *sk);
void (*save_close)(struct sock *sk, long timeout);
void (*save_data_ready)(struct sock *sk);
void (*save_write_space)(struct sock *sk);
@@ -143,6 +144,7 @@ static int bpf_tcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
static int bpf_tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size);
static int bpf_tcp_sendpage(struct sock *sk, struct page *page,
int offset, size_t size, int flags);
+static void bpf_tcp_unhash(struct sock *sk);
static void bpf_tcp_close(struct sock *sk, long timeout);
static inline struct smap_psock *smap_psock_sk(const struct sock *sk)
@@ -184,6 +186,7 @@ static void build_protos(struct proto prot[SOCKMAP_NUM_CONFIGS],
struct proto *base)
{
prot[SOCKMAP_BASE] = *base;
+ prot[SOCKMAP_BASE].unhash = bpf_tcp_unhash;
prot[SOCKMAP_BASE].close = bpf_tcp_close;
prot[SOCKMAP_BASE].recvmsg = bpf_tcp_recvmsg;
prot[SOCKMAP_BASE].stream_memory_read = bpf_tcp_stream_read;
@@ -217,6 +220,7 @@ static int bpf_tcp_init(struct sock *sk)
return -EBUSY;
}
+ psock->save_unhash = sk->sk_prot->unhash;
psock->save_close = sk->sk_prot->close;
psock->sk_proto = sk->sk_prot;
@@ -305,30 +309,12 @@ static struct smap_psock_map_entry *psock_map_pop(struct sock *sk,
return e;
}
-static void bpf_tcp_close(struct sock *sk, long timeout)
+static void bpf_tcp_remove(struct sock *sk, struct smap_psock *psock)
{
- void (*close_fun)(struct sock *sk, long timeout);
struct smap_psock_map_entry *e;
struct sk_msg_buff *md, *mtmp;
- struct smap_psock *psock;
struct sock *osk;
- lock_sock(sk);
- rcu_read_lock();
- psock = smap_psock_sk(sk);
- if (unlikely(!psock)) {
- rcu_read_unlock();
- release_sock(sk);
- return sk->sk_prot->close(sk, timeout);
- }
-
- /* The psock may be destroyed anytime after exiting the RCU critial
- * section so by the time we use close_fun the psock may no longer
- * be valid. However, bpf_tcp_close is called with the sock lock
- * held so the close hook and sk are still valid.
- */
- close_fun = psock->save_close;
-
if (psock->cork) {
free_start_sg(psock->sock, psock->cork, true);
kfree(psock->cork);
@@ -379,6 +365,42 @@ static void bpf_tcp_close(struct sock *sk, long timeout)
kfree(e);
e = psock_map_pop(sk, psock);
}
+}
+
+static void bpf_tcp_unhash(struct sock *sk)
+{
+ void (*unhash_fun)(struct sock *sk);
+ struct smap_psock *psock;
+
+ rcu_read_lock();
+ psock = smap_psock_sk(sk);
+ if (unlikely(!psock)) {
+ rcu_read_unlock();
+ if (sk->sk_prot->unhash)
+ sk->sk_prot->unhash(sk);
+ return;
+ }
+ unhash_fun = psock->save_unhash;
+ bpf_tcp_remove(sk, psock);
+ rcu_read_unlock();
+ unhash_fun(sk);
+}
+
+static void bpf_tcp_close(struct sock *sk, long timeout)
+{
+ void (*close_fun)(struct sock *sk, long timeout);
+ struct smap_psock *psock;
+
+ lock_sock(sk);
+ rcu_read_lock();
+ psock = smap_psock_sk(sk);
+ if (unlikely(!psock)) {
+ rcu_read_unlock();
+ release_sock(sk);
+ return sk->sk_prot->close(sk, timeout);
+ }
+ close_fun = psock->save_close;
+ bpf_tcp_remove(sk, psock);
rcu_read_unlock();
release_sock(sk);
close_fun(sk, timeout);
^ permalink raw reply related
* [bpf PATCH v4 3/3] bpf: test_maps, only support ESTABLISHED socks
From: John Fastabend @ 2018-09-18 16:01 UTC (permalink / raw)
To: edumazet, ast, daniel; +Cc: netdev
In-Reply-To: <20180918155826.22728.71788.stgit@john-Precision-Tower-5810>
Ensure that sockets added to a sock{map|hash} that is not in the
ESTABLISHED state is rejected.
Fixes: 1aa12bdf1bfb ("bpf: sockmap, add sock close() hook to remove socks")
Signed-off-by: John Fastabend <john.fastabend@gmail.com>
Acked-by: Yonghong Song <yhs@fb.com>
---
tools/testing/selftests/bpf/test_maps.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c
index 6f54f84..9b552c0 100644
--- a/tools/testing/selftests/bpf/test_maps.c
+++ b/tools/testing/selftests/bpf/test_maps.c
@@ -580,7 +580,11 @@ static void test_sockmap(int tasks, void *data)
/* Test update without programs */
for (i = 0; i < 6; i++) {
err = bpf_map_update_elem(fd, &i, &sfd[i], BPF_ANY);
- if (err) {
+ if (i < 2 && !err) {
+ printf("Allowed update sockmap '%i:%i' not in ESTABLISHED\n",
+ i, sfd[i]);
+ goto out_sockmap;
+ } else if (i >= 2 && err) {
printf("Failed noprog update sockmap '%i:%i'\n",
i, sfd[i]);
goto out_sockmap;
@@ -741,7 +745,7 @@ static void test_sockmap(int tasks, void *data)
}
/* Test map update elem afterwards fd lives in fd and map_fd */
- for (i = 0; i < 6; i++) {
+ for (i = 2; i < 6; i++) {
err = bpf_map_update_elem(map_fd_rx, &i, &sfd[i], BPF_ANY);
if (err) {
printf("Failed map_fd_rx update sockmap %i '%i:%i'\n",
@@ -845,7 +849,7 @@ static void test_sockmap(int tasks, void *data)
}
/* Delete the elems without programs */
- for (i = 0; i < 6; i++) {
+ for (i = 2; i < 6; i++) {
err = bpf_map_delete_elem(fd, &i);
if (err) {
printf("Failed delete sockmap %i '%i:%i'\n",
^ permalink raw reply related
* [PATCH net-next v5 03/20] zinc: ChaCha20 generic C implementation and selftest
From: Jason A. Donenfeld @ 2018-09-18 16:16 UTC (permalink / raw)
To: linux-kernel, netdev, linux-crypto, davem, gregkh
Cc: Jason A. Donenfeld, Samuel Neves, Andy Lutomirski,
Jean-Philippe Aumasson
In-Reply-To: <20180918161646.19105-1-Jason@zx2c4.com>
This implements the ChaCha20 permutation as a single C statement, by way
of the comma operator, which the compiler is able to simplify
terrifically.
Information: https://cr.yp.to/chacha.html
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Cc: Samuel Neves <sneves@dei.uc.pt>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
---
include/zinc/chacha20.h | 52 ++
lib/zinc/Kconfig | 4 +
lib/zinc/Makefile | 3 +
lib/zinc/chacha20/chacha20.c | 193 ++++++
lib/zinc/selftest/chacha20.h | 1182 ++++++++++++++++++++++++++++++++++
5 files changed, 1434 insertions(+)
create mode 100644 include/zinc/chacha20.h
create mode 100644 lib/zinc/chacha20/chacha20.c
create mode 100644 lib/zinc/selftest/chacha20.h
diff --git a/include/zinc/chacha20.h b/include/zinc/chacha20.h
new file mode 100644
index 000000000000..afad69409f5f
--- /dev/null
+++ b/include/zinc/chacha20.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ */
+
+#ifndef _ZINC_CHACHA20_H
+#define _ZINC_CHACHA20_H
+
+#include <asm/unaligned.h>
+#include <linux/simd.h>
+#include <linux/kernel.h>
+#include <linux/types.h>
+
+enum {
+ CHACHA20_IV_SIZE = 16,
+ CHACHA20_KEY_SIZE = 32,
+ CHACHA20_BLOCK_SIZE = 64,
+ CHACHA20_BLOCK_WORDS = CHACHA20_BLOCK_SIZE / sizeof(u32),
+ HCHACHA20_KEY_SIZE = 32,
+ HCHACHA20_NONCE_SIZE = 16
+};
+
+struct chacha20_ctx {
+ u32 key[8];
+ u32 counter[4];
+} __aligned(32);
+
+static inline void chacha20_init(struct chacha20_ctx *state,
+ const u8 key[CHACHA20_KEY_SIZE],
+ const u64 nonce)
+{
+ state->key[0] = get_unaligned_le32(key + 0);
+ state->key[1] = get_unaligned_le32(key + 4);
+ state->key[2] = get_unaligned_le32(key + 8);
+ state->key[3] = get_unaligned_le32(key + 12);
+ state->key[4] = get_unaligned_le32(key + 16);
+ state->key[5] = get_unaligned_le32(key + 20);
+ state->key[6] = get_unaligned_le32(key + 24);
+ state->key[7] = get_unaligned_le32(key + 28);
+ state->counter[0] = state->counter[1] = 0;
+ state->counter[2] = nonce & U32_MAX;
+ state->counter[3] = nonce >> 32;
+}
+void chacha20(struct chacha20_ctx *state, u8 *dst, const u8 *src, u32 len,
+ simd_context_t *simd_context);
+
+/* Derived key should be 32-bit aligned */
+void hchacha20(u8 derived_key[CHACHA20_KEY_SIZE],
+ const u8 nonce[HCHACHA20_NONCE_SIZE],
+ const u8 key[HCHACHA20_KEY_SIZE], simd_context_t *simd_context);
+
+#endif /* _ZINC_CHACHA20_H */
diff --git a/lib/zinc/Kconfig b/lib/zinc/Kconfig
index 4e2e59126a67..1ca1ae1e9ea9 100644
--- a/lib/zinc/Kconfig
+++ b/lib/zinc/Kconfig
@@ -1,3 +1,7 @@
+config ZINC_CHACHA20
+ tristate
+ select CRYPTO_ALGAPI
+
config ZINC_DEBUG
bool "Zinc cryptography library debugging and self-tests"
help
diff --git a/lib/zinc/Makefile b/lib/zinc/Makefile
index 83dfd63988c0..83b320f1ace2 100644
--- a/lib/zinc/Makefile
+++ b/lib/zinc/Makefile
@@ -2,3 +2,6 @@ ccflags-y := -O3
ccflags-y += -Wframe-larger-than=$(if (CONFIG_KASAN),16384,8192)
ccflags-y += -D'pr_fmt(fmt)="zinc: " fmt'
ccflags-$(CONFIG_ZINC_DEBUG) += -DDEBUG
+
+zinc_chacha20-y := chacha20/chacha20.o
+obj-$(CONFIG_ZINC_CHACHA20) += zinc_chacha20.o
diff --git a/lib/zinc/chacha20/chacha20.c b/lib/zinc/chacha20/chacha20.c
new file mode 100644
index 000000000000..3f00e1edd4c8
--- /dev/null
+++ b/lib/zinc/chacha20/chacha20.c
@@ -0,0 +1,193 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ *
+ * Implementation of the ChaCha20 stream cipher.
+ *
+ * Information: https://cr.yp.to/chacha.html
+ */
+
+#include <zinc/chacha20.h>
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <crypto/algapi.h>
+
+#ifndef HAVE_CHACHA20_ARCH_IMPLEMENTATION
+void __init chacha20_fpu_init(void)
+{
+}
+static inline bool chacha20_arch(u8 *out, const u8 *in, const size_t len,
+ const u32 key[8], const u32 counter[4],
+ simd_context_t *simd_context)
+{
+ return false;
+}
+static inline bool hchacha20_arch(u8 *derived_key, const u8 *nonce,
+ const u8 *key, simd_context_t *simd_context)
+{
+ return false;
+}
+#endif
+
+#define EXPAND_32_BYTE_K 0x61707865U, 0x3320646eU, 0x79622d32U, 0x6b206574U
+
+#define QUARTER_ROUND(x, a, b, c, d) ( \
+ x[a] += x[b], \
+ x[d] = rol32((x[d] ^ x[a]), 16), \
+ x[c] += x[d], \
+ x[b] = rol32((x[b] ^ x[c]), 12), \
+ x[a] += x[b], \
+ x[d] = rol32((x[d] ^ x[a]), 8), \
+ x[c] += x[d], \
+ x[b] = rol32((x[b] ^ x[c]), 7) \
+)
+
+#define C(i, j) (i * 4 + j)
+
+#define DOUBLE_ROUND(x) ( \
+ /* Column Round */ \
+ QUARTER_ROUND(x, C(0, 0), C(1, 0), C(2, 0), C(3, 0)), \
+ QUARTER_ROUND(x, C(0, 1), C(1, 1), C(2, 1), C(3, 1)), \
+ QUARTER_ROUND(x, C(0, 2), C(1, 2), C(2, 2), C(3, 2)), \
+ QUARTER_ROUND(x, C(0, 3), C(1, 3), C(2, 3), C(3, 3)), \
+ /* Diagonal Round */ \
+ QUARTER_ROUND(x, C(0, 0), C(1, 1), C(2, 2), C(3, 3)), \
+ QUARTER_ROUND(x, C(0, 1), C(1, 2), C(2, 3), C(3, 0)), \
+ QUARTER_ROUND(x, C(0, 2), C(1, 3), C(2, 0), C(3, 1)), \
+ QUARTER_ROUND(x, C(0, 3), C(1, 0), C(2, 1), C(3, 2)) \
+)
+
+#define TWENTY_ROUNDS(x) ( \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x), \
+ DOUBLE_ROUND(x) \
+)
+
+static void chacha20_block_generic(__le32 *stream, u32 *state)
+{
+ u32 x[CHACHA20_BLOCK_WORDS];
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(x); ++i)
+ x[i] = state[i];
+
+ TWENTY_ROUNDS(x);
+
+ for (i = 0; i < ARRAY_SIZE(x); ++i)
+ stream[i] = cpu_to_le32(x[i] + state[i]);
+
+ ++state[12];
+}
+
+static void chacha20_generic(u8 *out, const u8 *in, u32 len, const u32 key[8],
+ const u32 counter[4])
+{
+ __le32 buf[CHACHA20_BLOCK_WORDS];
+ u32 x[] = {
+ EXPAND_32_BYTE_K,
+ key[0], key[1], key[2], key[3],
+ key[4], key[5], key[6], key[7],
+ counter[0], counter[1], counter[2], counter[3]
+ };
+
+ if (out != in)
+ memmove(out, in, len);
+
+ while (len >= CHACHA20_BLOCK_SIZE) {
+ chacha20_block_generic(buf, x);
+ crypto_xor(out, (u8 *)buf, CHACHA20_BLOCK_SIZE);
+ len -= CHACHA20_BLOCK_SIZE;
+ out += CHACHA20_BLOCK_SIZE;
+ }
+ if (len) {
+ chacha20_block_generic(buf, x);
+ crypto_xor(out, (u8 *)buf, len);
+ }
+}
+
+void chacha20(struct chacha20_ctx *state, u8 *dst, const u8 *src, u32 len,
+ simd_context_t *simd_context)
+{
+ if (!chacha20_arch(dst, src, len, state->key, state->counter,
+ simd_context))
+ chacha20_generic(dst, src, len, state->key, state->counter);
+ state->counter[0] += (len + 63) / 64;
+}
+EXPORT_SYMBOL(chacha20);
+
+static void hchacha20_generic(u8 derived_key[CHACHA20_KEY_SIZE],
+ const u8 nonce[HCHACHA20_NONCE_SIZE],
+ const u8 key[HCHACHA20_KEY_SIZE])
+{
+ __le32 *out = (__force __le32 *)derived_key;
+ u32 x[] = { EXPAND_32_BYTE_K,
+ get_unaligned_le32(key + 0),
+ get_unaligned_le32(key + 4),
+ get_unaligned_le32(key + 8),
+ get_unaligned_le32(key + 12),
+ get_unaligned_le32(key + 16),
+ get_unaligned_le32(key + 20),
+ get_unaligned_le32(key + 24),
+ get_unaligned_le32(key + 28),
+ get_unaligned_le32(nonce + 0),
+ get_unaligned_le32(nonce + 4),
+ get_unaligned_le32(nonce + 8),
+ get_unaligned_le32(nonce + 12)
+ };
+
+ TWENTY_ROUNDS(x);
+
+ out[0] = cpu_to_le32(x[0]);
+ out[1] = cpu_to_le32(x[1]);
+ out[2] = cpu_to_le32(x[2]);
+ out[3] = cpu_to_le32(x[3]);
+ out[4] = cpu_to_le32(x[12]);
+ out[5] = cpu_to_le32(x[13]);
+ out[6] = cpu_to_le32(x[14]);
+ out[7] = cpu_to_le32(x[15]);
+}
+
+/* Derived key should be 32-bit aligned */
+void hchacha20(u8 derived_key[CHACHA20_KEY_SIZE],
+ const u8 nonce[HCHACHA20_NONCE_SIZE],
+ const u8 key[HCHACHA20_KEY_SIZE], simd_context_t *simd_context)
+{
+ if (!hchacha20_arch(derived_key, nonce, key, simd_context))
+ hchacha20_generic(derived_key, nonce, key);
+}
+EXPORT_SYMBOL(hchacha20);
+
+#include "../selftest/chacha20.h"
+
+static bool nosimd __initdata = false;
+
+static int __init mod_init(void)
+{
+ if (!nosimd)
+ chacha20_fpu_init();
+#ifdef DEBUG
+ if (!chacha20_selftest())
+ return -ENOTRECOVERABLE;
+#endif
+ return 0;
+}
+
+static void __exit mod_exit(void)
+{
+}
+
+module_param(nosimd, bool, 0);
+module_init(mod_init);
+module_exit(mod_exit);
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("ChaCha20 stream cipher");
+MODULE_AUTHOR("Jason A. Donenfeld <Jason@zx2c4.com>");
diff --git a/lib/zinc/selftest/chacha20.h b/lib/zinc/selftest/chacha20.h
new file mode 100644
index 000000000000..f591460a854a
--- /dev/null
+++ b/lib/zinc/selftest/chacha20.h
@@ -0,0 +1,1182 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ */
+
+#ifdef DEBUG
+
+enum { MAXIMUM_TEST_BUFFER_LEN = 800 };
+
+struct chacha20_testvec {
+ u8 key[CHACHA20_KEY_SIZE];
+ u64 nonce;
+ u8 input[MAXIMUM_TEST_BUFFER_LEN];
+ u8 result[MAXIMUM_TEST_BUFFER_LEN];
+ size_t ilen;
+};
+
+/*
+ * #!/usr/bin/env python3
+ *
+ * import chacha20
+ * import os
+ * import struct
+ *
+ * def encode_blob(blob):
+ * a = ""
+ * x = 0
+ * for i in blob:
+ * a += ('0x%02x' % i) + ","
+ * x += 1
+ * if x % 8 == 0:
+ * a += "\n\t\t "
+ * else:
+ * a += " "
+ * if x % 8 == 0:
+ * return a[:len(a) - 8]
+ * return a[:len(a) - 2]
+ *
+ * enc = [ ]
+ * dec = [ ]
+ *
+ * def make_vector(plen):
+ * key = os.urandom(32)
+ * nonce = os.urandom(8)
+ * p = os.urandom(plen)
+ * c = chacha20.chacha20_encrypt(p, key, nonce)
+ *
+ * out = "{\n"
+ * out += "\t.key\t= { " + encode_blob(key) + " },\n"
+ * out += "\t.nonce\t= " + hex(struct.unpack("<Q", nonce)[0]) + "ULL,\n"
+ * out += "\t.input\t= { " + encode_blob(p) + " },\n"
+ * out += "\t.ilen\t= " + str(len(p)) + ",\n"
+ * out += "\t.result\t= { " + encode_blob(c) + " }\n"
+ * out += "}"
+ * enc.append(out)
+ *
+ * block_size = 64
+ *
+ * make_vector(1)
+ * make_vector(63)
+ * make_vector(64)
+ * make_vector(127)
+ * make_vector(250)
+ * make_vector(256)
+ * make_vector(280)
+ * make_vector(511)
+ * make_vector(512)
+ * make_vector(513)
+ * make_vector(514)
+ * make_vector(769)
+ *
+ * print(", ".join(enc))
+ */
+
+static const struct chacha20_testvec
+chacha20_vectors[] __initconst = {{
+ .key = { 0x8e, 0xee, 0x50, 0x40, 0xb2, 0x76, 0x80, 0xeb,
+ 0xd8, 0x7e, 0xf2, 0x5a, 0xcf, 0x25, 0xd6, 0x6c,
+ 0x9e, 0xc8, 0x3d, 0x0c, 0xf7, 0x05, 0x00, 0x40,
+ 0xa0, 0x75, 0x66, 0x86, 0xdf, 0x8d, 0xac, 0x77 },
+ .nonce = 0x37d4e0484f294097ULL,
+ .input = { 0xe1 },
+ .ilen = 1,
+ .result = { 0xf1 }
+}, {
+ .key = { 0x23, 0xe2, 0x3b, 0x3a, 0x22, 0xd2, 0xfb, 0x69,
+ 0x92, 0xcc, 0x0a, 0xda, 0x48, 0x21, 0x33, 0x8f,
+ 0x57, 0x58, 0x08, 0x17, 0xca, 0xe5, 0x56, 0x08,
+ 0x71, 0x86, 0x91, 0xc7, 0xae, 0x64, 0x4d, 0xfd },
+ .nonce = 0xe0c627c7e1a4b012ULL,
+ .input = { 0x83, 0x95, 0x1c, 0xc6, 0x8a, 0x53, 0x92, 0x6a,
+ 0x3f, 0x1e, 0x67, 0x17, 0x0b, 0x2b, 0x45, 0x04,
+ 0xaf, 0x5a, 0x75, 0x59, 0xb7, 0xab, 0x80, 0xde,
+ 0xa4, 0x8f, 0xda, 0x03, 0xfd, 0xda, 0xe1, 0x7e,
+ 0x75, 0x99, 0x44, 0x6f, 0xea, 0x75, 0x26, 0x68,
+ 0xcc, 0x20, 0xe0, 0x7c, 0xf1, 0x09, 0x31, 0x44,
+ 0xba, 0xb9, 0xe8, 0x7f, 0xce, 0x5e, 0x89, 0x6d,
+ 0x0d, 0x4d, 0x20, 0x93, 0x52, 0x91, 0x85 },
+ .ilen = 63,
+ .result = { 0xa1, 0x32, 0xec, 0xab, 0x86, 0x35, 0x99, 0x88,
+ 0xc0, 0xef, 0xae, 0x69, 0x64, 0xa3, 0xba, 0xcb,
+ 0xb9, 0x44, 0x24, 0x92, 0x9f, 0x33, 0xdb, 0xc0,
+ 0x7f, 0xe5, 0x39, 0xa8, 0x57, 0x2a, 0x29, 0x5f,
+ 0xdc, 0xc4, 0xee, 0x7f, 0xcd, 0xd9, 0x5f, 0x0b,
+ 0x49, 0xc2, 0x9a, 0xa2, 0x5e, 0x5a, 0x37, 0x3d,
+ 0x7a, 0x1e, 0x66, 0xc6, 0xd1, 0xb9, 0xcb, 0x98,
+ 0xc0, 0xd1, 0xc8, 0xc5, 0xf9, 0x3d, 0xed }
+}, {
+ .key = { 0x09, 0x5c, 0x71, 0xde, 0x41, 0x76, 0x2d, 0x48,
+ 0x4f, 0x58, 0xec, 0x0e, 0x62, 0x00, 0x73, 0x04,
+ 0x03, 0x47, 0x04, 0x8a, 0x4c, 0xd7, 0x52, 0x5a,
+ 0x7b, 0x84, 0x36, 0xbf, 0x8a, 0xdb, 0xeb, 0xf5 },
+ .nonce = 0x45a98ae9d6bf5373ULL,
+ .input = { 0x40, 0x3d, 0x31, 0x7d, 0x63, 0xba, 0xec, 0xd8,
+ 0xa6, 0x73, 0xc4, 0x8c, 0x40, 0xc3, 0xaa, 0x3b,
+ 0xca, 0x03, 0x52, 0xa0, 0xe8, 0x3f, 0xb6, 0x21,
+ 0x3f, 0x07, 0xb1, 0xb3, 0x14, 0x94, 0xf8, 0x36,
+ 0xb3, 0x06, 0x33, 0x1e, 0xb1, 0x05, 0x94, 0x53,
+ 0x1f, 0xee, 0x16, 0x8e, 0xcd, 0x0f, 0x38, 0x41,
+ 0x1f, 0x4d, 0x12, 0x60, 0xec, 0x88, 0x42, 0x2a,
+ 0xca, 0x3e, 0x84, 0x33, 0x13, 0x17, 0xeb, 0x04 },
+ .ilen = 64,
+ .result = { 0x48, 0x0e, 0x7f, 0x54, 0xe6, 0x3e, 0x98, 0x81,
+ 0xc6, 0x1d, 0x5a, 0x1b, 0x4d, 0xaf, 0x27, 0x3c,
+ 0xce, 0xd7, 0x98, 0x05, 0x92, 0xdb, 0xb9, 0x24,
+ 0xef, 0xbc, 0x6a, 0x01, 0xb9, 0x61, 0xb9, 0xf9,
+ 0x66, 0xee, 0xc9, 0x30, 0xf5, 0x48, 0xb1, 0xde,
+ 0xcd, 0xe9, 0x4d, 0x05, 0x2f, 0xea, 0x2c, 0xdf,
+ 0x32, 0xc1, 0x43, 0x50, 0x31, 0x1d, 0x0a, 0x3a,
+ 0x4e, 0x24, 0xd3, 0x69, 0xc6, 0x18, 0x23, 0xdb }
+}, {
+ .key = { 0x5d, 0x01, 0xdf, 0xe5, 0xbc, 0xf6, 0xaf, 0x04,
+ 0x11, 0xda, 0xef, 0x43, 0x6b, 0xdf, 0xdd, 0xb8,
+ 0xa6, 0xc6, 0x4a, 0xbd, 0xa4, 0xb3, 0xba, 0xf6,
+ 0xce, 0x2f, 0xf8, 0x34, 0xbb, 0xdd, 0x22, 0x42 },
+ .nonce = 0x70faa08a4184e385ULL,
+ .input = { 0xb8, 0x06, 0x92, 0x20, 0xc7, 0x7c, 0xb2, 0x01,
+ 0x3e, 0xf6, 0x73, 0x6c, 0x74, 0x48, 0x72, 0xec,
+ 0x5a, 0x38, 0xfa, 0x36, 0x39, 0x39, 0xff, 0x3c,
+ 0x81, 0x91, 0x99, 0x0b, 0x29, 0xec, 0xed, 0x4c,
+ 0x53, 0x9a, 0x24, 0xce, 0x4b, 0xcf, 0x8d, 0x36,
+ 0x2b, 0xf1, 0x8b, 0x0a, 0x36, 0xdc, 0xf8, 0xdc,
+ 0x5a, 0x77, 0x64, 0xca, 0x5f, 0x06, 0xf8, 0xb8,
+ 0xc2, 0xa3, 0x69, 0x0d, 0x2f, 0x2f, 0xa5, 0x39,
+ 0x94, 0xc6, 0x83, 0xe3, 0x20, 0x48, 0x8b, 0xb7,
+ 0x58, 0x26, 0xd0, 0x39, 0x2a, 0x4f, 0x7b, 0x66,
+ 0xda, 0x52, 0x0d, 0xa9, 0x5b, 0xb3, 0x44, 0x57,
+ 0x38, 0x86, 0x6a, 0xba, 0xa7, 0x86, 0x44, 0x82,
+ 0xfb, 0xa8, 0xe8, 0x9d, 0x08, 0x1c, 0x3a, 0xce,
+ 0x92, 0x02, 0x61, 0xb6, 0x3e, 0x81, 0xcb, 0x44,
+ 0x12, 0x44, 0xf8, 0x2f, 0x53, 0x62, 0xbd, 0xfb,
+ 0x3e, 0x9f, 0x65, 0x19, 0xae, 0xc8, 0x4f },
+ .ilen = 127,
+ .result = { 0x0e, 0x37, 0xd1, 0x08, 0x6c, 0x55, 0xc5, 0x88,
+ 0x33, 0x0a, 0xbd, 0x6e, 0x24, 0xb1, 0x00, 0xdd,
+ 0xb0, 0x43, 0x1f, 0x5a, 0x20, 0x06, 0xfb, 0xa5,
+ 0x00, 0x7c, 0xb4, 0x86, 0x58, 0x46, 0x55, 0x6c,
+ 0xa2, 0x28, 0xd7, 0x55, 0x51, 0xd5, 0xf1, 0xa8,
+ 0x43, 0x06, 0xc4, 0x50, 0xd3, 0x6f, 0x44, 0x21,
+ 0x9c, 0xce, 0x29, 0x89, 0xc2, 0x79, 0x20, 0x8f,
+ 0x76, 0xfe, 0x3a, 0xef, 0xdd, 0x59, 0x21, 0x94,
+ 0xe4, 0xe2, 0xa3, 0x3c, 0x08, 0x74, 0xe7, 0xb5,
+ 0x99, 0xf3, 0x8a, 0x88, 0xad, 0xb1, 0x98, 0x31,
+ 0xd2, 0x2a, 0x36, 0xda, 0xdf, 0x2c, 0x4c, 0xcd,
+ 0xd6, 0xed, 0xe2, 0xca, 0x54, 0xa0, 0x72, 0x0d,
+ 0xce, 0x16, 0xe3, 0xa7, 0x2d, 0x93, 0x2b, 0x07,
+ 0x18, 0x3f, 0xdb, 0x14, 0xd0, 0x00, 0x26, 0xe4,
+ 0x48, 0xa6, 0xe4, 0x59, 0xb7, 0xec, 0xbc, 0xee,
+ 0xba, 0x5e, 0x7e, 0xae, 0xf2, 0xd8, 0x91 }
+}, {
+ .key = { 0x92, 0x58, 0xaa, 0xf5, 0x07, 0x8f, 0xa3, 0xdd,
+ 0xc6, 0x97, 0x32, 0x34, 0xd6, 0x96, 0xca, 0xc1,
+ 0x6d, 0xf3, 0xef, 0xfe, 0x44, 0x32, 0x82, 0xa4,
+ 0x52, 0x26, 0xf2, 0x69, 0xfb, 0xbf, 0x24, 0xc8 },
+ .nonce = 0xf66f76ba3ce150e4ULL,
+ .input = { 0x69, 0x3d, 0xd7, 0xa9, 0x86, 0xf7, 0x64, 0xac,
+ 0x5b, 0x0d, 0xd3, 0xa8, 0x4a, 0xf8, 0xc9, 0x12,
+ 0xa9, 0x2a, 0xc2, 0x28, 0x14, 0x02, 0x48, 0x7a,
+ 0x64, 0xeb, 0x5b, 0x5a, 0xce, 0x79, 0xca, 0x86,
+ 0xa4, 0x60, 0xe6, 0xbe, 0x56, 0x29, 0x50, 0x2d,
+ 0xbd, 0x44, 0x88, 0xf8, 0x92, 0xf7, 0x60, 0xe7,
+ 0xb1, 0xf4, 0x8b, 0x44, 0xbf, 0x25, 0x4d, 0xc8,
+ 0x6b, 0x2e, 0xdc, 0x9e, 0xe8, 0x0f, 0x38, 0x3b,
+ 0x48, 0x8c, 0x9c, 0x30, 0xd8, 0x18, 0xf5, 0xcd,
+ 0x85, 0x69, 0x71, 0x99, 0xe3, 0xb0, 0x23, 0x11,
+ 0x26, 0xf5, 0x84, 0x15, 0x3f, 0xb8, 0xfb, 0xf4,
+ 0xee, 0x9a, 0xc2, 0x79, 0xe6, 0x92, 0x6f, 0x4d,
+ 0x8a, 0x62, 0xcb, 0x38, 0x37, 0x44, 0x84, 0x09,
+ 0xc6, 0xf7, 0xa1, 0x48, 0xf1, 0xa1, 0xf7, 0x1b,
+ 0x87, 0x8e, 0xd0, 0xef, 0xa2, 0x1e, 0x32, 0x8e,
+ 0x9b, 0xdc, 0x0e, 0xa4, 0xf1, 0x94, 0x25, 0x1b,
+ 0x9b, 0x3c, 0x50, 0xdd, 0x41, 0x99, 0xd0, 0x96,
+ 0x60, 0xde, 0xbd, 0xed, 0x0b, 0x9a, 0xa4, 0xcb,
+ 0x2d, 0x2e, 0xd6, 0xdf, 0x97, 0xe0, 0xea, 0x87,
+ 0x12, 0xe1, 0x9b, 0x30, 0xe8, 0x6f, 0x01, 0x25,
+ 0x0c, 0xa4, 0xc1, 0xd4, 0x0e, 0xca, 0x07, 0x46,
+ 0xd9, 0x4b, 0x5d, 0x03, 0xcb, 0x8d, 0xcf, 0x55,
+ 0xe2, 0x62, 0x61, 0x10, 0x86, 0xf9, 0xf5, 0xe8,
+ 0xfe, 0x2f, 0xc5, 0xbc, 0x27, 0x2a, 0x1b, 0x1b,
+ 0x43, 0x4e, 0xb7, 0xd7, 0x18, 0x7b, 0xcf, 0x7a,
+ 0x5b, 0xa5, 0x18, 0x72, 0xa4, 0xa7, 0x24, 0x9f,
+ 0x6a, 0x12, 0xe7, 0x44, 0xb5, 0x45, 0x8c, 0xd5,
+ 0x53, 0xba, 0x2c, 0xaf, 0xaf, 0x3f, 0xe7, 0x12,
+ 0x45, 0xbd, 0x61, 0x68, 0xde, 0xc3, 0x38, 0xa8,
+ 0x5b, 0xbe, 0xc8, 0x7b, 0xab, 0xab, 0x21, 0x75,
+ 0x95, 0x8e, 0xb7, 0x98, 0xb8, 0x76, 0x69, 0xf8,
+ 0xf4, 0x70 },
+ .ilen = 250,
+ .result = { 0xd6, 0xfc, 0x21, 0x55, 0x85, 0x12, 0xa2, 0xd2,
+ 0x1c, 0x22, 0x96, 0xd6, 0xfe, 0x61, 0x24, 0x75,
+ 0x0c, 0xaa, 0x90, 0xe9, 0x0b, 0xe1, 0x72, 0xc8,
+ 0xb5, 0xcb, 0x17, 0x95, 0x9e, 0x30, 0x7c, 0x23,
+ 0x69, 0x5f, 0x7a, 0x71, 0x34, 0xac, 0x47, 0x18,
+ 0x0a, 0x9e, 0x39, 0x71, 0x2a, 0xff, 0x5a, 0xba,
+ 0x35, 0x1b, 0x60, 0xc2, 0xe8, 0x10, 0x2d, 0xbf,
+ 0x75, 0xf8, 0x2f, 0x2a, 0x0e, 0x01, 0x67, 0x4e,
+ 0xaf, 0xb5, 0xf3, 0x1c, 0xa1, 0x62, 0x7e, 0x22,
+ 0x2d, 0x17, 0x3c, 0x73, 0x03, 0x4f, 0xff, 0xb2,
+ 0x47, 0xae, 0x68, 0x63, 0x95, 0xcf, 0x66, 0x0e,
+ 0x22, 0xdf, 0x6c, 0x8b, 0xb4, 0xee, 0x3a, 0xb3,
+ 0x5d, 0x0d, 0xd3, 0x33, 0x16, 0x13, 0x45, 0x59,
+ 0x42, 0x74, 0x9f, 0xc5, 0xfb, 0xc6, 0x20, 0x88,
+ 0x33, 0x41, 0x55, 0x28, 0x61, 0x33, 0x9f, 0x80,
+ 0x57, 0x39, 0x26, 0xe4, 0x3b, 0x98, 0xcd, 0x6b,
+ 0xef, 0x10, 0x33, 0x20, 0xee, 0x18, 0x3a, 0x59,
+ 0x89, 0x4a, 0x1a, 0xc3, 0xdf, 0xb2, 0x5b, 0x0b,
+ 0x08, 0x1f, 0xaa, 0xaa, 0x8e, 0x54, 0xd3, 0x10,
+ 0xf7, 0xc2, 0xce, 0x68, 0x06, 0x06, 0x3e, 0x39,
+ 0x13, 0x0f, 0x43, 0xfd, 0x3b, 0x61, 0xc7, 0xfd,
+ 0x92, 0xf2, 0x0b, 0xe7, 0xa2, 0x60, 0xd8, 0xd8,
+ 0x00, 0xe1, 0x18, 0x77, 0x13, 0xb8, 0xa4, 0xa8,
+ 0xc0, 0x60, 0x55, 0x96, 0x2d, 0x6a, 0x69, 0x72,
+ 0x05, 0xa7, 0x27, 0xe5, 0x05, 0x20, 0x22, 0xcf,
+ 0xcf, 0xd5, 0x26, 0x70, 0x1d, 0x76, 0x97, 0xd6,
+ 0x93, 0x71, 0xd4, 0x0a, 0xe2, 0x5e, 0x47, 0x54,
+ 0x42, 0x6e, 0xb0, 0xea, 0xa9, 0x8b, 0x73, 0x2c,
+ 0xc7, 0xbb, 0x2d, 0x64, 0xfd, 0xcf, 0x23, 0x33,
+ 0xb1, 0x93, 0x0c, 0x32, 0x66, 0x1e, 0x26, 0x7a,
+ 0x8c, 0xb6, 0xfa, 0x85, 0x44, 0x35, 0x02, 0xce,
+ 0x05, 0xf6 }
+}, {
+ .key = { 0x56, 0x5b, 0x3a, 0xc9, 0xa6, 0x55, 0x34, 0xa2,
+ 0xb3, 0xaf, 0x33, 0xed, 0x9d, 0xed, 0x97, 0xfa,
+ 0x13, 0x1b, 0x06, 0x50, 0x09, 0x02, 0xa1, 0x93,
+ 0xe4, 0xa8, 0x3b, 0xd5, 0x8f, 0xc9, 0xa0, 0xc2 },
+ .nonce = 0x88663f30a2aa338ULL,
+ .input = { 0x9a, 0x2a, 0x04, 0xa0, 0x09, 0x55, 0x05, 0xfc,
+ 0x17, 0x80, 0x12, 0x6a, 0x78, 0x63, 0xe6, 0x55,
+ 0x13, 0xb7, 0x1e, 0x42, 0x67, 0x86, 0x01, 0x61,
+ 0x42, 0x74, 0x9a, 0x08, 0x46, 0xac, 0x69, 0x3a,
+ 0x10, 0x24, 0x18, 0x97, 0x3b, 0x85, 0xfb, 0x40,
+ 0x5a, 0x77, 0x5b, 0xf3, 0x6b, 0x77, 0xf0, 0x1e,
+ 0x41, 0xba, 0x38, 0x7d, 0xa1, 0x23, 0x9b, 0xd3,
+ 0x58, 0x59, 0x58, 0x34, 0xa9, 0x03, 0xb3, 0x43,
+ 0x9f, 0x5b, 0xe3, 0x1d, 0xf5, 0x9f, 0x8a, 0xdc,
+ 0x99, 0x99, 0x45, 0xb0, 0x72, 0xe0, 0xa0, 0x81,
+ 0xed, 0x7e, 0x2c, 0x21, 0x3d, 0xf2, 0xad, 0xfd,
+ 0xde, 0x9a, 0xf0, 0x40, 0xe7, 0xce, 0x1f, 0xc2,
+ 0x34, 0x2b, 0xf9, 0xc9, 0x52, 0x71, 0x0c, 0xa6,
+ 0x8a, 0x64, 0x90, 0xd0, 0xcf, 0x28, 0x8d, 0xc6,
+ 0x1c, 0x44, 0xd3, 0xdc, 0x24, 0x2a, 0x57, 0x79,
+ 0xc9, 0xa1, 0xb2, 0x38, 0x33, 0x20, 0xa2, 0xd0,
+ 0x59, 0x6e, 0xb9, 0x27, 0x06, 0xdb, 0xaf, 0x7d,
+ 0xed, 0x74, 0x8d, 0x34, 0xbc, 0x3f, 0x89, 0x42,
+ 0xe0, 0x28, 0x59, 0x2b, 0x8a, 0x3f, 0x42, 0xc5,
+ 0x69, 0x55, 0x1d, 0x8e, 0xa2, 0xba, 0xda, 0x01,
+ 0x68, 0x7f, 0x5d, 0xbd, 0x06, 0xa6, 0x95, 0xa7,
+ 0xad, 0x60, 0xf4, 0x5a, 0x43, 0x62, 0x8f, 0x69,
+ 0x9a, 0xc3, 0xd1, 0x89, 0xd3, 0xfe, 0xc8, 0x2d,
+ 0x4d, 0xdd, 0x1c, 0x07, 0xfa, 0x3f, 0xaa, 0x7e,
+ 0xfd, 0x90, 0xff, 0x6b, 0x35, 0x5e, 0x23, 0x0d,
+ 0x38, 0x16, 0x00, 0xc7, 0x00, 0xa0, 0xb2, 0x37,
+ 0x1e, 0x18, 0xdf, 0xa1, 0x18, 0xbc, 0x21, 0x88,
+ 0x80, 0x8f, 0x8c, 0xf8, 0x0c, 0x83, 0x69, 0xd6,
+ 0x89, 0x0e, 0xe9, 0xbc, 0x69, 0x11, 0x3b, 0x17,
+ 0xa1, 0x44, 0x4c, 0xf4, 0x02, 0x9e, 0x3a, 0x8f,
+ 0xaf, 0xbc, 0xcc, 0x4f, 0x46, 0x42, 0x7d, 0x63,
+ 0x9d, 0x95, 0x8d, 0xa7, 0x9d, 0xf9, 0x73, 0xc0 },
+ .ilen = 256,
+ .result = { 0x79, 0x75, 0xce, 0x01, 0x53, 0x9b, 0x03, 0x24,
+ 0x1b, 0x28, 0x10, 0x70, 0x98, 0xd8, 0x43, 0x4d,
+ 0x48, 0xba, 0xa2, 0xfb, 0x44, 0x08, 0xff, 0xac,
+ 0x59, 0x91, 0xb2, 0x4c, 0x8b, 0x76, 0x99, 0x79,
+ 0x56, 0x9b, 0x61, 0x69, 0x3c, 0x75, 0xef, 0xbf,
+ 0x4b, 0x22, 0x91, 0x58, 0xd9, 0x54, 0xaa, 0xba,
+ 0xe7, 0x20, 0xf5, 0xba, 0x4a, 0xe0, 0xbc, 0x23,
+ 0x3d, 0x2d, 0x39, 0x82, 0xad, 0x74, 0x01, 0x4d,
+ 0x2d, 0xd8, 0x42, 0x3d, 0x94, 0x42, 0x48, 0x31,
+ 0x40, 0x81, 0xc0, 0x24, 0xeb, 0xf8, 0xa3, 0xa0,
+ 0x51, 0x2e, 0x70, 0x7a, 0x6e, 0xfe, 0xa2, 0xae,
+ 0x49, 0x4b, 0x60, 0x76, 0x1c, 0x82, 0x5a, 0xe5,
+ 0xcd, 0x0e, 0x17, 0xd4, 0x0e, 0x50, 0x22, 0x7f,
+ 0xd9, 0xd8, 0x5e, 0x60, 0x45, 0xd7, 0x98, 0xa6,
+ 0xc1, 0x01, 0xc8, 0x45, 0x18, 0x37, 0x9f, 0xd1,
+ 0x26, 0x61, 0xd4, 0x74, 0x7f, 0x44, 0xd3, 0xad,
+ 0xf0, 0x20, 0xa1, 0x75, 0x8d, 0xaa, 0xf0, 0x26,
+ 0xad, 0xf7, 0x83, 0x45, 0xb5, 0x6f, 0xd0, 0xf9,
+ 0xa3, 0xa9, 0x7b, 0xdf, 0xfd, 0x90, 0x30, 0xa9,
+ 0x66, 0x25, 0x19, 0xd6, 0xcb, 0x16, 0xfb, 0xb1,
+ 0x8b, 0x33, 0x0b, 0x78, 0x8f, 0x40, 0xe0, 0x58,
+ 0x99, 0x3d, 0xd7, 0xfe, 0xd6, 0x2c, 0x84, 0xfd,
+ 0x74, 0x99, 0xea, 0x0c, 0x24, 0x18, 0x2a, 0x1c,
+ 0x46, 0xc0, 0xdc, 0xdf, 0xd0, 0x62, 0x03, 0x10,
+ 0x68, 0x37, 0xb1, 0x3f, 0x45, 0x97, 0xf3, 0xe5,
+ 0x2e, 0xd3, 0xe1, 0x9d, 0x84, 0x57, 0x1c, 0x71,
+ 0x3e, 0xb9, 0x5b, 0x42, 0xc4, 0xaa, 0x84, 0xf0,
+ 0x47, 0x42, 0x5d, 0xed, 0xde, 0x80, 0x3e, 0xde,
+ 0xed, 0xac, 0xeb, 0x3f, 0x13, 0xaa, 0x6f, 0x8a,
+ 0x10, 0xaa, 0x97, 0x64, 0x28, 0x06, 0x2c, 0xaf,
+ 0xfc, 0x71, 0x55, 0x45, 0x8b, 0xd1, 0x31, 0xc7,
+ 0x29, 0xdf, 0xd2, 0x93, 0x43, 0xad, 0xe9, 0x75 }
+}, {
+ .key = { 0x70, 0x11, 0x81, 0x4d, 0x0f, 0xf3, 0x69, 0xe9,
+ 0x16, 0x2e, 0x24, 0x94, 0x7a, 0x7b, 0xeb, 0x6c,
+ 0x8d, 0xfa, 0xc1, 0x89, 0xaf, 0x79, 0x67, 0xec,
+ 0xe3, 0x0c, 0x74, 0x3d, 0xeb, 0x0e, 0x7c, 0x52 },
+ .nonce = 0x8a3a75f2adc47f8fULL,
+ .input = { 0xfa, 0xc3, 0xca, 0xf1, 0xdd, 0x0c, 0xf3, 0xbb,
+ 0xbe, 0x51, 0x0f, 0xea, 0x1f, 0x09, 0xda, 0x68,
+ 0xfa, 0xb2, 0x0d, 0xa9, 0xc6, 0xb3, 0x2b, 0x0c,
+ 0x8a, 0xd6, 0x3f, 0xd6, 0x2e, 0x9d, 0x0e, 0x01,
+ 0xdf, 0x58, 0xf2, 0x02, 0x2d, 0x16, 0x54, 0x69,
+ 0x0e, 0xff, 0x23, 0x01, 0x84, 0x9c, 0xa7, 0xfc,
+ 0x3e, 0xd6, 0xb7, 0x32, 0x4e, 0xa9, 0x5d, 0xc1,
+ 0xb7, 0x62, 0x2d, 0x52, 0xda, 0x95, 0x17, 0x56,
+ 0xf2, 0xa7, 0x81, 0x16, 0x12, 0x5a, 0xab, 0x0b,
+ 0x65, 0x5e, 0xad, 0xe5, 0xbc, 0xee, 0xe9, 0xd7,
+ 0x03, 0xe3, 0x18, 0x50, 0xed, 0xcc, 0xef, 0xd8,
+ 0x59, 0xaa, 0x83, 0x01, 0xbc, 0x61, 0x9c, 0x5f,
+ 0xa3, 0x72, 0x4a, 0x47, 0xce, 0x14, 0x7c, 0x13,
+ 0xb9, 0xe9, 0xc3, 0xba, 0xb0, 0xc6, 0x4d, 0xb8,
+ 0xb2, 0x02, 0xc1, 0xbf, 0x6a, 0xb6, 0xcb, 0x65,
+ 0x05, 0xaa, 0xdb, 0x35, 0x4e, 0x78, 0x5c, 0xbd,
+ 0x9f, 0xef, 0x58, 0x75, 0x55, 0x49, 0x93, 0xb4,
+ 0xd0, 0x21, 0x44, 0x8e, 0x05, 0xb8, 0xbe, 0xa9,
+ 0x72, 0x33, 0x8e, 0x9b, 0xa3, 0xf7, 0x38, 0x97,
+ 0xf7, 0x25, 0xb6, 0x4b, 0x44, 0xbb, 0xc6, 0x45,
+ 0x55, 0xb9, 0x64, 0xbf, 0x43, 0xf6, 0x51, 0x0d,
+ 0xde, 0xa3, 0xc5, 0xda, 0x45, 0x03, 0x7c, 0x7b,
+ 0xf4, 0xd2, 0x1c, 0xce, 0x38, 0xa9, 0x0e, 0x2d,
+ 0x90, 0x80, 0x1d, 0x5d, 0xca, 0x18, 0xdc, 0x53,
+ 0x31, 0x42, 0xaa, 0x92, 0xfb, 0x59, 0xac, 0x8b,
+ 0x57, 0x25, 0xa8, 0xf0, 0xc1, 0x55, 0x4a, 0x16,
+ 0xdb, 0xa4, 0xf7, 0xfd, 0xe1, 0x49, 0x31, 0x6f,
+ 0xb5, 0x5c, 0x69, 0x9d, 0xc2, 0x98, 0x3e, 0x9c,
+ 0xf5, 0x51, 0x52, 0xdc, 0x2f, 0x0e, 0x90, 0x5f,
+ 0x41, 0x88, 0x3a, 0x69, 0x1c, 0x91, 0x69, 0x97,
+ 0xcb, 0xa5, 0x86, 0x52, 0xf7, 0x98, 0x64, 0x04,
+ 0x47, 0x60, 0x6f, 0x32, 0x3b, 0xe2, 0x7b, 0x33,
+ 0x0a, 0x79, 0x53, 0xad, 0x97, 0xa5, 0xc9, 0x1b,
+ 0x8b, 0x5d, 0xfd, 0x1e, 0x51, 0x36, 0x2a, 0x7e,
+ 0x0c, 0xdd, 0x0d, 0x0a, 0xc1, 0xf8, 0xb9, 0x77 },
+ .ilen = 280,
+ .result = { 0x9b, 0x42, 0x0b, 0x62, 0x0c, 0x90, 0x82, 0x01,
+ 0x6d, 0x87, 0x1d, 0x33, 0xb2, 0x9d, 0x54, 0x8b,
+ 0x1b, 0xc1, 0xe5, 0x3b, 0x50, 0xcf, 0x2a, 0x49,
+ 0x3d, 0xcf, 0x7b, 0x70, 0xd5, 0xa2, 0x6c, 0x9b,
+ 0x55, 0x72, 0x3b, 0x70, 0xbb, 0x99, 0xf3, 0xfd,
+ 0x34, 0x87, 0xa6, 0x72, 0x77, 0x56, 0x36, 0x05,
+ 0x5f, 0xde, 0x34, 0x9c, 0xb9, 0x26, 0x24, 0x6e,
+ 0x28, 0x64, 0x30, 0x99, 0xc2, 0xfd, 0xc1, 0x6f,
+ 0x27, 0x30, 0x8a, 0x1d, 0xf5, 0x68, 0xd1, 0x99,
+ 0xc8, 0x79, 0xb5, 0x40, 0xbc, 0xd2, 0x5d, 0xfc,
+ 0x6d, 0x57, 0x65, 0x35, 0x77, 0x9a, 0x1f, 0x3c,
+ 0xba, 0x73, 0x76, 0xf3, 0xdb, 0xe9, 0x18, 0x64,
+ 0xe6, 0x7e, 0xc9, 0xf3, 0xa6, 0xb2, 0xbf, 0xf0,
+ 0xa1, 0xcd, 0xd5, 0x03, 0xdd, 0x1e, 0x00, 0x07,
+ 0xba, 0x76, 0x7d, 0x46, 0x48, 0xf1, 0x7d, 0xb7,
+ 0x78, 0xd5, 0x9a, 0xdf, 0x80, 0x18, 0xd2, 0xad,
+ 0x97, 0xc7, 0xbd, 0xeb, 0x5a, 0xab, 0x17, 0x92,
+ 0x4b, 0x93, 0xd3, 0x39, 0x18, 0xef, 0xbb, 0xfe,
+ 0x89, 0x6d, 0x7d, 0x1d, 0xd3, 0x86, 0x4d, 0xfd,
+ 0x3f, 0x67, 0xfc, 0x9b, 0xfe, 0x4e, 0x9e, 0xe7,
+ 0xfb, 0x9f, 0x45, 0x8a, 0xd4, 0xd9, 0x0c, 0x30,
+ 0x90, 0xb8, 0xef, 0xc7, 0x18, 0x04, 0x09, 0x67,
+ 0xdf, 0x57, 0xe7, 0x30, 0x45, 0x91, 0xe3, 0xa6,
+ 0x73, 0x4a, 0x83, 0x6e, 0x92, 0x1e, 0xe5, 0xad,
+ 0x56, 0x1b, 0x86, 0x5c, 0xc3, 0x52, 0x58, 0x01,
+ 0xe7, 0xb9, 0x2e, 0xdc, 0x96, 0xee, 0x4c, 0x65,
+ 0x5a, 0xd7, 0x19, 0x6b, 0xfa, 0xa3, 0x0b, 0xf4,
+ 0xec, 0xab, 0xb5, 0xb2, 0xf8, 0xdd, 0x23, 0xff,
+ 0xe6, 0xa6, 0x55, 0x75, 0xcb, 0xb1, 0xa7, 0xed,
+ 0x6a, 0xa8, 0x60, 0x1a, 0x61, 0x64, 0x2a, 0xbc,
+ 0x73, 0xed, 0xa7, 0x54, 0x0d, 0x79, 0xc9, 0xc6,
+ 0x14, 0x6c, 0x78, 0xb1, 0x14, 0x10, 0x47, 0x80,
+ 0x76, 0xb7, 0x81, 0xac, 0x09, 0xf5, 0x12, 0x2d,
+ 0x21, 0x9c, 0x79, 0xd9, 0x37, 0xae, 0x2a, 0x51,
+ 0x41, 0x1e, 0x9e, 0x7f, 0x0d, 0x4b, 0x17, 0x87 }
+}, {
+ .key = { 0x23, 0xa3, 0x7e, 0x3a, 0xe4, 0x66, 0xb7, 0x86,
+ 0x7b, 0xdb, 0xf1, 0xdf, 0x76, 0x65, 0x4a, 0x70,
+ 0xc4, 0x93, 0x4d, 0xf4, 0xc1, 0x0e, 0xc1, 0x69,
+ 0xce, 0x43, 0xed, 0x9f, 0x48, 0x36, 0xa3, 0x03 },
+ .nonce = 0xbc7df621eb115af2ULL,
+ .input = { 0x11, 0x32, 0xa9, 0x18, 0x65, 0x15, 0x25, 0x2e,
+ 0x9a, 0xbb, 0x0c, 0x25, 0x96, 0x1e, 0xbf, 0x89,
+ 0x71, 0x7f, 0x18, 0x3a, 0x33, 0xf2, 0x9d, 0x85,
+ 0x23, 0x4d, 0x1f, 0x91, 0x44, 0x75, 0x08, 0xee,
+ 0xe0, 0x2d, 0xab, 0x1d, 0x8f, 0xab, 0x0e, 0x3f,
+ 0xe7, 0x3d, 0x43, 0x3d, 0x19, 0x49, 0xfc, 0xb2,
+ 0xf1, 0xc3, 0x53, 0x41, 0x0b, 0x35, 0xda, 0xd1,
+ 0x8c, 0x18, 0x9e, 0xb7, 0x1b, 0xa2, 0x5a, 0xfd,
+ 0x8a, 0xa4, 0xc2, 0xf4, 0x8d, 0x98, 0x01, 0x53,
+ 0xb8, 0x82, 0xf5, 0x53, 0x5c, 0xfe, 0x6d, 0x7d,
+ 0x63, 0x21, 0x46, 0x16, 0x29, 0x9a, 0xcf, 0x20,
+ 0xb3, 0xf7, 0xd2, 0x7f, 0xb4, 0x30, 0x90, 0xbd,
+ 0x95, 0xc2, 0xd2, 0xc0, 0x14, 0x84, 0xd3, 0x4a,
+ 0xf0, 0x12, 0xa6, 0xa8, 0xe8, 0x55, 0x97, 0xb2,
+ 0x0c, 0xee, 0x25, 0xf7, 0x94, 0xef, 0xae, 0xf7,
+ 0xbb, 0xe2, 0x64, 0x7d, 0xe2, 0xdc, 0x0f, 0x6b,
+ 0x7c, 0xa6, 0xe4, 0x71, 0x38, 0x44, 0x3d, 0xcd,
+ 0x0d, 0x94, 0xe9, 0x52, 0x6a, 0xb0, 0x8e, 0xd5,
+ 0x20, 0xe8, 0x60, 0x9c, 0xde, 0x92, 0xfc, 0x67,
+ 0x4e, 0xc3, 0x02, 0x6c, 0x6e, 0xd7, 0x56, 0x0a,
+ 0x00, 0x81, 0xc8, 0xd0, 0x26, 0x3c, 0x9a, 0x97,
+ 0x98, 0xfa, 0x10, 0x0e, 0xad, 0xeb, 0x7e, 0xdc,
+ 0x41, 0x58, 0x68, 0xf8, 0xa4, 0x70, 0x95, 0x2d,
+ 0x8f, 0x00, 0xd2, 0xcf, 0x9e, 0x57, 0x9e, 0xa7,
+ 0xe8, 0x11, 0x4c, 0xfc, 0x64, 0x51, 0x64, 0x11,
+ 0x85, 0xef, 0x47, 0xf5, 0xd3, 0x98, 0xda, 0x48,
+ 0x7e, 0x04, 0x55, 0x95, 0x4b, 0xc1, 0xc8, 0x42,
+ 0x66, 0x7c, 0xc9, 0x2c, 0xb6, 0xde, 0xc9, 0x3b,
+ 0x3d, 0x2b, 0x94, 0x88, 0xbf, 0x07, 0xd4, 0xe9,
+ 0x88, 0x61, 0x12, 0xb6, 0x3e, 0x2c, 0x14, 0x7f,
+ 0xea, 0x6d, 0x0a, 0xd9, 0x64, 0xf0, 0x3d, 0x57,
+ 0x82, 0x4a, 0xab, 0x57, 0x65, 0xba, 0x34, 0xca,
+ 0x22, 0x4d, 0xa8, 0xb5, 0xfe, 0xc2, 0x0a, 0xb4,
+ 0x0a, 0xb2, 0x05, 0x8d, 0x28, 0xc3, 0x5d, 0xee,
+ 0x40, 0xae, 0x48, 0x16, 0xeb, 0x60, 0x07, 0x59,
+ 0x53, 0x7d, 0x8d, 0xf6, 0x7c, 0x15, 0x32, 0x86,
+ 0x0c, 0x58, 0x86, 0x2c, 0xab, 0x92, 0xe8, 0xc8,
+ 0x7b, 0x97, 0x73, 0xd2, 0x11, 0x53, 0xed, 0xab,
+ 0x7d, 0xc2, 0xeb, 0x07, 0x5b, 0xcf, 0xc6, 0x5f,
+ 0xcb, 0xa1, 0x88, 0x9f, 0x4f, 0x74, 0xa9, 0x26,
+ 0xdc, 0x63, 0x7c, 0x5b, 0x60, 0x4b, 0x54, 0x02,
+ 0xa7, 0xaf, 0x7b, 0xf7, 0x34, 0x76, 0xd3, 0xbc,
+ 0x74, 0x51, 0x3c, 0x74, 0x57, 0xe4, 0x06, 0x22,
+ 0x25, 0xc3, 0x5f, 0x41, 0x40, 0x87, 0xf8, 0xf9,
+ 0xed, 0x26, 0x36, 0x7a, 0xb2, 0x5c, 0xde, 0xdb,
+ 0x85, 0x1b, 0x70, 0x87, 0xc8, 0x0e, 0x0e, 0x9c,
+ 0x1e, 0x8d, 0x86, 0x6a, 0x52, 0x3a, 0x64, 0x7c,
+ 0x50, 0xa3, 0xfe, 0x57, 0x48, 0x3a, 0xf7, 0x35,
+ 0x71, 0xea, 0x90, 0xd3, 0xc5, 0xf0, 0x9d, 0x08,
+ 0x94, 0x81, 0x6a, 0x90, 0xa6, 0xf1, 0x70, 0x53,
+ 0x3b, 0x46, 0x10, 0x5f, 0x2a, 0x90, 0x3c, 0xa7,
+ 0x06, 0x54, 0xa8, 0xd5, 0x7d, 0x8d, 0x3e, 0x59,
+ 0xaa, 0x19, 0xa3, 0x48, 0x40, 0xc3, 0xf4, 0xb4,
+ 0x70, 0x77, 0xed, 0x05, 0x0c, 0x41, 0x34, 0x53,
+ 0x7a, 0xd3, 0x1a, 0xc9, 0xbb, 0xee, 0xd3, 0xc3,
+ 0xcb, 0xd6, 0xe7, 0xed, 0x56, 0x17, 0x9d, 0x47,
+ 0x5f, 0x9a, 0xe6, 0x83, 0x41, 0x70, 0x1e, 0xf1,
+ 0x65, 0x98, 0xbf, 0xea, 0x00, 0x3b, 0xfb, 0x0e,
+ 0x91, 0x82, 0x30, 0xa0, 0xa5, 0xbb, 0xff, 0x3c,
+ 0x04, 0x52, 0x8f, 0xc2, 0x16, 0xa8, 0xe2, 0x78,
+ 0x55, 0xaf, 0x91, 0x20, 0xa1, 0xbe, 0xde, 0x90,
+ 0xb3, 0xeb, 0xf1, 0x18, 0x84, 0x25, 0xf0, 0xd4,
+ 0x42, 0x0f, 0x0b, 0xfc, 0x16, 0x47, 0x0c, 0x94,
+ 0x99, 0xeb, 0x96, 0xb5, 0x21, 0x76, 0x03 },
+ .ilen = 511,
+ .result = { 0x93, 0xf6, 0xa1, 0x01, 0xf0, 0x82, 0x18, 0x8d,
+ 0xf2, 0xd1, 0x29, 0x36, 0x6b, 0x1f, 0xcb, 0x7b,
+ 0xa2, 0xde, 0x37, 0x8f, 0x31, 0x46, 0x9b, 0x73,
+ 0xd4, 0x9e, 0x71, 0x7b, 0x49, 0x79, 0xc5, 0x2c,
+ 0x29, 0x1f, 0x60, 0xfd, 0x94, 0x9c, 0x00, 0xc0,
+ 0xe5, 0x49, 0xd3, 0x99, 0xe8, 0xb8, 0x7a, 0x9a,
+ 0x35, 0xb3, 0x58, 0xa1, 0x2c, 0x4b, 0xef, 0x7c,
+ 0x05, 0xf0, 0x07, 0x35, 0x4e, 0x02, 0x36, 0xae,
+ 0x29, 0xe4, 0x7c, 0x21, 0x35, 0xba, 0x77, 0xe0,
+ 0x80, 0x59, 0xd4, 0x19, 0xcd, 0xb0, 0xc6, 0xff,
+ 0x4f, 0x71, 0x91, 0x04, 0x90, 0x75, 0xe7, 0xea,
+ 0xd0, 0xc6, 0xa5, 0x10, 0xd2, 0x4b, 0x9d, 0x0f,
+ 0x3e, 0x08, 0x33, 0xf8, 0x7d, 0xe0, 0xcf, 0xb9,
+ 0x4e, 0xae, 0xd2, 0xe7, 0xe0, 0x2e, 0x98, 0x11,
+ 0xdc, 0xb9, 0x95, 0x25, 0xb2, 0x97, 0x16, 0x5b,
+ 0x75, 0x40, 0xa0, 0x6c, 0xcb, 0x67, 0xfb, 0x55,
+ 0x35, 0x20, 0xd1, 0xf0, 0x58, 0x3c, 0x67, 0xc3,
+ 0x77, 0x4e, 0xee, 0xaf, 0xbb, 0x5c, 0x15, 0x1b,
+ 0x05, 0xf8, 0xd3, 0x00, 0x8a, 0x33, 0x4c, 0x8f,
+ 0x01, 0x78, 0x00, 0xb0, 0x54, 0x9c, 0x63, 0x0c,
+ 0x09, 0x37, 0xf0, 0x0a, 0x4a, 0xae, 0x82, 0x2c,
+ 0x14, 0x99, 0xa2, 0x32, 0x3e, 0x6a, 0x90, 0x73,
+ 0x70, 0x6c, 0x01, 0xb1, 0x60, 0xf3, 0x15, 0xc6,
+ 0xc9, 0x05, 0x9a, 0xdc, 0xda, 0x69, 0x1b, 0x31,
+ 0xe7, 0xcf, 0x53, 0x40, 0x3f, 0x42, 0xdf, 0x35,
+ 0x38, 0x6a, 0xc2, 0x00, 0x11, 0xcb, 0x2c, 0x37,
+ 0x81, 0xdd, 0xa8, 0x27, 0xcb, 0x85, 0x89, 0x68,
+ 0x79, 0x46, 0x0c, 0xe3, 0x8e, 0xf4, 0x05, 0x9e,
+ 0xfc, 0xbd, 0x49, 0xbb, 0x4c, 0xe1, 0x05, 0xdf,
+ 0x8c, 0xca, 0xef, 0xbc, 0x3e, 0xb9, 0xa7, 0x02,
+ 0x0b, 0xc6, 0x91, 0xe4, 0x48, 0x9d, 0x3a, 0x58,
+ 0x4a, 0xf6, 0x49, 0xb0, 0xd9, 0xf0, 0xed, 0x39,
+ 0xe5, 0xf2, 0xe2, 0x47, 0xd0, 0x9e, 0xfa, 0xe1,
+ 0xb3, 0x7b, 0xb8, 0x1c, 0xef, 0x92, 0xf2, 0xf0,
+ 0xc0, 0xd5, 0xf1, 0x89, 0xee, 0xba, 0x52, 0xea,
+ 0x39, 0x08, 0x4c, 0xec, 0x92, 0x72, 0x0d, 0x92,
+ 0x3e, 0x57, 0x09, 0x5b, 0x4d, 0x6b, 0xbc, 0x31,
+ 0x7a, 0xcf, 0x28, 0x30, 0x68, 0x29, 0x3a, 0x99,
+ 0x84, 0xa8, 0xb7, 0x36, 0xc7, 0xa7, 0x76, 0x76,
+ 0x10, 0xd5, 0x6d, 0x35, 0xb0, 0xe0, 0x86, 0xa0,
+ 0x12, 0x19, 0x78, 0x0b, 0xbb, 0x7b, 0x03, 0xae,
+ 0x61, 0xf6, 0xe9, 0xb8, 0x7d, 0x7b, 0xf2, 0xb4,
+ 0x4e, 0x1c, 0x57, 0x37, 0xa8, 0xa6, 0x9c, 0x6a,
+ 0xc2, 0xae, 0xc2, 0xfa, 0x10, 0xd7, 0x77, 0x6f,
+ 0x80, 0x91, 0xfe, 0xa9, 0xd2, 0x49, 0xc6, 0x13,
+ 0xad, 0x18, 0x35, 0x94, 0x95, 0xe1, 0x5a, 0x18,
+ 0x0e, 0x31, 0x49, 0xa5, 0x31, 0xa9, 0xb1, 0xd2,
+ 0xf3, 0x36, 0x0a, 0x72, 0x46, 0x45, 0xc1, 0xd0,
+ 0x22, 0x6d, 0x3d, 0xe2, 0x73, 0x74, 0xad, 0x0a,
+ 0x0c, 0xfd, 0x1d, 0x67, 0xab, 0xcf, 0xe4, 0x4b,
+ 0x52, 0xa2, 0xba, 0x87, 0xe8, 0x6b, 0xc3, 0xf9,
+ 0xb2, 0xb0, 0x05, 0xf7, 0xe9, 0xac, 0x96, 0x3f,
+ 0xbb, 0xe5, 0xae, 0xdb, 0xd3, 0x4f, 0x88, 0x3c,
+ 0x29, 0x69, 0x90, 0xc6, 0x2e, 0x56, 0x89, 0x5d,
+ 0xda, 0x57, 0xe2, 0x3e, 0x21, 0x72, 0x50, 0x2d,
+ 0x65, 0xe9, 0xbd, 0x53, 0x4c, 0xe4, 0x3f, 0x75,
+ 0xc7, 0x46, 0x0c, 0xb7, 0x35, 0x25, 0x9b, 0x29,
+ 0x6e, 0xb5, 0x91, 0xa1, 0x37, 0xca, 0xa1, 0x26,
+ 0x09, 0x3f, 0x7e, 0xbd, 0x8f, 0xf0, 0x2f, 0x59,
+ 0x69, 0xa5, 0x74, 0xe2, 0x31, 0xe9, 0x04, 0x36,
+ 0x78, 0x14, 0xf0, 0x30, 0x39, 0x6d, 0xfb, 0xae,
+ 0xed, 0xb5, 0x23, 0x54, 0x28, 0x3d, 0x31, 0x23,
+ 0x49, 0x43, 0xab, 0xf2, 0xf8, 0xc4, 0xe7, 0x5f,
+ 0xef, 0x51, 0xa9, 0x25, 0xb9, 0x87, 0x4c }
+}, {
+ .key = { 0xa6, 0xa1, 0xb9, 0x92, 0xe4, 0x20, 0x25, 0xe9,
+ 0x7e, 0x51, 0x70, 0x50, 0xc9, 0xa3, 0x9c, 0x7f,
+ 0x48, 0x9f, 0xca, 0x2f, 0x16, 0x8a, 0xaf, 0xdb,
+ 0x25, 0xc0, 0xd3, 0x28, 0x4d, 0x19, 0xd4, 0x57 },
+ .nonce = 0xf81deedff5b091e0ULL,
+ .input = { 0x2a, 0xdb, 0xe1, 0x1d, 0x93, 0xd9, 0x63, 0x17,
+ 0x09, 0x06, 0xed, 0x92, 0x67, 0x1a, 0x35, 0x00,
+ 0x1a, 0x61, 0xc1, 0xce, 0xd1, 0xd6, 0x6b, 0x78,
+ 0x09, 0x32, 0xad, 0x36, 0xf8, 0x2b, 0xc7, 0x6f,
+ 0xc7, 0x3b, 0x3a, 0x73, 0x33, 0x00, 0xb0, 0x5c,
+ 0x38, 0x8c, 0x48, 0x82, 0x85, 0x09, 0x8a, 0x4a,
+ 0x0b, 0x32, 0x53, 0xed, 0x80, 0x66, 0x39, 0xa9,
+ 0xa9, 0xf7, 0x11, 0x3b, 0xac, 0x3a, 0xbc, 0xbe,
+ 0xda, 0x20, 0x83, 0xc4, 0x42, 0xcf, 0xf6, 0x21,
+ 0xb7, 0x70, 0x33, 0x5b, 0x6c, 0xcd, 0x20, 0xd0,
+ 0x48, 0x95, 0x36, 0x17, 0xd4, 0x24, 0x55, 0xc7,
+ 0x77, 0xc1, 0x32, 0xe2, 0xa5, 0xd2, 0xd4, 0xc1,
+ 0x6c, 0xe5, 0x77, 0xe4, 0x8c, 0xda, 0x20, 0x03,
+ 0x4b, 0x75, 0x97, 0x64, 0x10, 0x40, 0xf8, 0x5c,
+ 0x94, 0xf8, 0xf7, 0x0a, 0x7f, 0x43, 0x56, 0x9a,
+ 0xcb, 0xe2, 0x8d, 0xe4, 0x81, 0x4c, 0x6f, 0x89,
+ 0x4c, 0x0c, 0x02, 0xf6, 0x00, 0x89, 0x0e, 0xbf,
+ 0x3f, 0xd5, 0x73, 0x83, 0x7e, 0x42, 0x49, 0xf6,
+ 0xdd, 0x21, 0x2f, 0x46, 0x66, 0x07, 0x59, 0xc5,
+ 0x19, 0xee, 0x29, 0x80, 0xc9, 0x3e, 0xa0, 0xfc,
+ 0x78, 0x73, 0xcc, 0x38, 0xed, 0xe8, 0xc1, 0x95,
+ 0xd7, 0x9e, 0x02, 0x03, 0x77, 0x03, 0x19, 0x2a,
+ 0x0d, 0x0b, 0x50, 0x40, 0x64, 0x07, 0x91, 0x6f,
+ 0x82, 0x55, 0x1e, 0x8d, 0x9d, 0xe3, 0x83, 0x75,
+ 0x06, 0xba, 0x4f, 0x42, 0x7c, 0x73, 0x6b, 0x6c,
+ 0x77, 0x31, 0x57, 0xfb, 0x1f, 0x54, 0xe0, 0x05,
+ 0x22, 0x22, 0x05, 0x32, 0x6c, 0x56, 0x2e, 0xc7,
+ 0x7a, 0x8c, 0xc9, 0xbb, 0x7c, 0x89, 0x6a, 0xb4,
+ 0x2e, 0xbb, 0xad, 0xf6, 0xd2, 0xf2, 0x38, 0xf6,
+ 0xcb, 0xab, 0xcc, 0x57, 0x98, 0xd4, 0x5f, 0x99,
+ 0x4c, 0xa0, 0xe1, 0x62, 0x25, 0x9e, 0x31, 0x34,
+ 0xcc, 0xce, 0xd9, 0x79, 0x6a, 0x13, 0xb9, 0x1d,
+ 0x0b, 0x9b, 0x40, 0x73, 0x0f, 0x9e, 0xa5, 0xfd,
+ 0xae, 0x63, 0x3a, 0xa0, 0x52, 0x1e, 0x29, 0x65,
+ 0x0f, 0x62, 0xcc, 0xda, 0x7f, 0x3c, 0x06, 0x91,
+ 0xde, 0xea, 0x77, 0xe5, 0x88, 0x6d, 0x7d, 0x26,
+ 0xc4, 0xf3, 0xc3, 0xa0, 0xf5, 0xd6, 0x51, 0xda,
+ 0x55, 0xcd, 0x38, 0x6f, 0x71, 0x0f, 0x5b, 0x2e,
+ 0x3d, 0xab, 0x7c, 0xf1, 0xae, 0xd5, 0xb5, 0xef,
+ 0x21, 0x51, 0xb6, 0xb5, 0xe9, 0x91, 0xd0, 0xcd,
+ 0x30, 0x11, 0x6d, 0x62, 0x23, 0xfe, 0x5b, 0xda,
+ 0xce, 0xef, 0x82, 0x68, 0x7b, 0x3e, 0xd1, 0x7e,
+ 0x3b, 0xdf, 0x9f, 0xf6, 0xd1, 0x98, 0xc7, 0xfd,
+ 0xdd, 0xc5, 0xc7, 0xb4, 0x79, 0x91, 0x9e, 0x43,
+ 0x93, 0xf1, 0xdb, 0x13, 0xb3, 0x46, 0x6d, 0x69,
+ 0xa2, 0xce, 0xab, 0xf5, 0x23, 0xbc, 0xdf, 0x99,
+ 0x84, 0x5d, 0xba, 0x42, 0x84, 0xaf, 0xf6, 0x70,
+ 0xaf, 0x36, 0x5f, 0xc7, 0x32, 0x56, 0x95, 0x9f,
+ 0x4a, 0x81, 0x2f, 0x1f, 0xa7, 0x58, 0x9f, 0x98,
+ 0x9c, 0x1d, 0x61, 0x98, 0x32, 0xcb, 0xf6, 0xba,
+ 0x3e, 0xda, 0x5d, 0x66, 0x26, 0x9b, 0x5d, 0x03,
+ 0x6e, 0xa2, 0x66, 0xe8, 0x9e, 0x14, 0x23, 0xae,
+ 0x02, 0x24, 0xcd, 0xc5, 0xa7, 0x81, 0x02, 0x0f,
+ 0x7b, 0xf5, 0x0c, 0x6e, 0x0b, 0x88, 0x12, 0x94,
+ 0xaa, 0x86, 0xff, 0x02, 0xef, 0xaa, 0x36, 0xa5,
+ 0x1c, 0x12, 0x23, 0x61, 0x66, 0xfe, 0x05, 0xc1,
+ 0x73, 0x43, 0x4e, 0x2b, 0xc9, 0xda, 0xbb, 0x2f,
+ 0x67, 0x50, 0x0a, 0x1f, 0xd2, 0x65, 0x8c, 0x75,
+ 0x9c, 0xf6, 0xcb, 0xab, 0x33, 0xea, 0xa2, 0x20,
+ 0x58, 0x95, 0x05, 0x9f, 0x76, 0xa4, 0x67, 0x68,
+ 0xc0, 0x77, 0x57, 0x0b, 0xe9, 0x9e, 0x93, 0x72,
+ 0x22, 0x77, 0x9f, 0x8d, 0xf9, 0x2c, 0xe5, 0xa0,
+ 0x32, 0x6b, 0xe1, 0xb5, 0x60, 0xa7, 0xa0, 0x7a,
+ 0xe2, 0x1e, 0x5c, 0x74, 0x1a, 0x1c, 0x69, 0xc1 },
+ .ilen = 512,
+ .result = { 0xf4, 0x3e, 0xcf, 0xe1, 0xf7, 0x21, 0xb2, 0x55,
+ 0xc9, 0xf5, 0xd3, 0x54, 0x40, 0xd0, 0x53, 0x28,
+ 0x62, 0xfd, 0xb6, 0x7b, 0x84, 0x51, 0xfd, 0x4d,
+ 0x86, 0x95, 0x9a, 0xfe, 0xcf, 0x10, 0xe1, 0xec,
+ 0x9f, 0x28, 0x2e, 0x24, 0x0f, 0xd4, 0x27, 0x12,
+ 0x7a, 0x36, 0xd4, 0x75, 0x52, 0xa3, 0x1b, 0xb6,
+ 0x18, 0xe4, 0x2a, 0x35, 0xd2, 0x30, 0xf3, 0xc7,
+ 0xdd, 0xf9, 0x3a, 0xfa, 0x90, 0xfd, 0xc3, 0xe6,
+ 0xe5, 0xd4, 0x0c, 0xab, 0xff, 0xd1, 0x9d, 0xfc,
+ 0x7c, 0x50, 0xcc, 0x22, 0x68, 0xd0, 0x81, 0xff,
+ 0x41, 0x7d, 0x28, 0x75, 0xd0, 0x22, 0xa9, 0x45,
+ 0x24, 0x53, 0xf7, 0xc6, 0xed, 0x43, 0x74, 0x14,
+ 0xa8, 0xbf, 0x19, 0x22, 0xfd, 0x46, 0x2e, 0x8d,
+ 0x96, 0xd2, 0x5e, 0x2f, 0xea, 0x52, 0x29, 0x61,
+ 0x48, 0xd6, 0x57, 0xa7, 0x78, 0xaf, 0xdb, 0x0b,
+ 0x21, 0x62, 0x12, 0x7c, 0xaa, 0x40, 0xff, 0xdc,
+ 0xdc, 0x8f, 0x21, 0x73, 0x08, 0x6a, 0x05, 0xdf,
+ 0x3b, 0xd3, 0xd5, 0xdf, 0x63, 0xa7, 0xaa, 0xff,
+ 0x67, 0xb2, 0xd7, 0xff, 0x4f, 0xba, 0xf4, 0x35,
+ 0xef, 0x3f, 0xb2, 0x84, 0xe2, 0x04, 0xd1, 0xb2,
+ 0x6c, 0x53, 0x5e, 0xb7, 0xdb, 0x78, 0x3f, 0x54,
+ 0x63, 0x77, 0xf4, 0xda, 0x50, 0xf4, 0x21, 0xb8,
+ 0xb4, 0xf1, 0x0b, 0x97, 0x23, 0x7b, 0x5f, 0x84,
+ 0xea, 0x20, 0xf2, 0xa2, 0xc4, 0xb8, 0x59, 0xb3,
+ 0x50, 0x67, 0xf7, 0xaf, 0xbe, 0xf1, 0x8c, 0xa0,
+ 0x37, 0x44, 0x8a, 0xd0, 0x70, 0x77, 0x22, 0x24,
+ 0xd2, 0xd6, 0xa1, 0x3a, 0x74, 0x61, 0xa7, 0x38,
+ 0x5a, 0xc4, 0xdc, 0x42, 0x6f, 0xdc, 0x6e, 0xc8,
+ 0xc7, 0x82, 0x0b, 0x16, 0x72, 0xc5, 0x5f, 0xd4,
+ 0x1f, 0x63, 0x13, 0x25, 0xb6, 0xa5, 0x0e, 0xef,
+ 0xa0, 0x75, 0xa4, 0x4e, 0x68, 0xaf, 0x9b, 0x6d,
+ 0x6f, 0xc9, 0x84, 0x5e, 0xc6, 0x64, 0x50, 0x70,
+ 0x81, 0x94, 0xd6, 0x41, 0x6b, 0x17, 0x35, 0x2c,
+ 0x15, 0xd8, 0xf0, 0x42, 0x7a, 0x20, 0x42, 0xca,
+ 0xe0, 0x5a, 0x24, 0xbc, 0xff, 0x90, 0x9e, 0x1d,
+ 0x05, 0x3d, 0x16, 0x35, 0x7d, 0xdc, 0xc6, 0xe2,
+ 0x0c, 0xf0, 0xa7, 0xe9, 0xa5, 0x09, 0x28, 0x6c,
+ 0xd9, 0x80, 0x80, 0xe8, 0xca, 0x48, 0x26, 0x68,
+ 0xb6, 0x84, 0xcb, 0x87, 0x76, 0x13, 0x51, 0x25,
+ 0x47, 0xc4, 0x8a, 0xcd, 0x8d, 0x1a, 0x07, 0x14,
+ 0x5a, 0xa7, 0x1a, 0xbb, 0x86, 0xe6, 0x76, 0xa0,
+ 0x29, 0x50, 0xae, 0xcb, 0x52, 0xfe, 0x0e, 0x27,
+ 0xd6, 0x48, 0x40, 0xfb, 0x3e, 0xb3, 0xdb, 0x67,
+ 0x2a, 0x3a, 0x9e, 0x62, 0xfc, 0x57, 0x5a, 0x0b,
+ 0x2c, 0xe5, 0x6d, 0x3d, 0x33, 0x55, 0xfc, 0xad,
+ 0x5d, 0x96, 0x68, 0xd9, 0xea, 0x92, 0xc1, 0x92,
+ 0x40, 0x3e, 0x72, 0xe6, 0xcc, 0x51, 0xc0, 0x40,
+ 0x70, 0x39, 0x61, 0x8a, 0xcd, 0xef, 0xde, 0x1d,
+ 0x44, 0x7d, 0x86, 0x66, 0x9f, 0xaa, 0x22, 0x9e,
+ 0xb9, 0xd1, 0x02, 0x05, 0x6b, 0xbd, 0xda, 0x29,
+ 0x60, 0x45, 0x77, 0x35, 0xeb, 0xae, 0x8b, 0xd8,
+ 0x79, 0x95, 0x8d, 0x1c, 0x3d, 0xe6, 0xf1, 0x37,
+ 0x70, 0x9a, 0x63, 0xab, 0xdb, 0x21, 0x4d, 0x7b,
+ 0x74, 0x00, 0x02, 0x0c, 0x72, 0x9d, 0x04, 0x39,
+ 0x39, 0xfc, 0x9c, 0x7d, 0x9b, 0x42, 0xf0, 0x94,
+ 0x72, 0x1e, 0x88, 0x58, 0xbf, 0xc9, 0x28, 0x9f,
+ 0xa0, 0x77, 0x65, 0x2e, 0x9d, 0xe9, 0x00, 0x33,
+ 0xc2, 0xd1, 0x83, 0x2a, 0x27, 0xb1, 0xa4, 0x69,
+ 0xc8, 0x61, 0x99, 0x09, 0xe1, 0xb3, 0xdf, 0x8a,
+ 0x11, 0xd4, 0xd4, 0x26, 0xc7, 0x31, 0x83, 0x67,
+ 0x1f, 0xd5, 0x00, 0x39, 0x49, 0xf1, 0xa6, 0xaf,
+ 0xbc, 0x6e, 0xe9, 0xc5, 0xe0, 0xa7, 0xee, 0x0c,
+ 0x7c, 0x9e, 0x08, 0x7f, 0x47, 0x9f, 0xed, 0x85,
+ 0x20, 0x23, 0x14, 0x0b, 0x3e, 0x58, 0xc3, 0x59 }
+}, {
+ .key = { 0x24, 0x30, 0xaa, 0x5c, 0xad, 0xee, 0xc9, 0xe5,
+ 0x15, 0x8d, 0x04, 0xd4, 0x1e, 0xd1, 0xac, 0xd5,
+ 0xa2, 0xc2, 0xd9, 0xd2, 0x55, 0xd9, 0x34, 0x09,
+ 0x44, 0xf4, 0x29, 0x1c, 0x85, 0xd1, 0x5f, 0xeb },
+ .nonce = 0xb9f185373d1a89baULL,
+ .input = { 0xe5, 0x1a, 0xcf, 0x39, 0x07, 0x21, 0x2b, 0x01,
+ 0x7a, 0x9a, 0x13, 0x54, 0x57, 0x8e, 0x45, 0x64,
+ 0xa5, 0xb4, 0x0f, 0x17, 0x72, 0x31, 0x92, 0x6b,
+ 0x18, 0x5e, 0x79, 0x51, 0x51, 0x56, 0xe5, 0x34,
+ 0x6d, 0x70, 0x34, 0x21, 0x00, 0x09, 0x97, 0x3f,
+ 0x9a, 0x56, 0xed, 0x10, 0x45, 0x20, 0xb5, 0x99,
+ 0x53, 0x09, 0xad, 0x22, 0xc0, 0xbb, 0x13, 0x1f,
+ 0xeb, 0xe1, 0xa2, 0x31, 0x9e, 0xab, 0xa8, 0x3c,
+ 0xf5, 0xc4, 0xa6, 0xa0, 0xb2, 0xde, 0xd3, 0x58,
+ 0x3a, 0x46, 0x88, 0x74, 0x0a, 0x8f, 0x1b, 0x67,
+ 0x55, 0x8a, 0xe0, 0xd1, 0x32, 0x9e, 0x07, 0x32,
+ 0x87, 0x4f, 0x04, 0xc1, 0xaf, 0xcb, 0x82, 0xdb,
+ 0x84, 0x15, 0x52, 0xdc, 0x31, 0xfe, 0x99, 0xee,
+ 0x4c, 0x8a, 0xd9, 0x2b, 0x2e, 0xe5, 0xaf, 0x4f,
+ 0x95, 0x4b, 0x46, 0x2b, 0x28, 0xe9, 0xd9, 0xf8,
+ 0x04, 0xc3, 0xbd, 0x8c, 0xb3, 0x2b, 0x50, 0x7d,
+ 0x2a, 0x3a, 0x9d, 0xfd, 0xca, 0x79, 0x91, 0x98,
+ 0xc8, 0xe4, 0xdb, 0xe0, 0xe3, 0x51, 0x67, 0xc8,
+ 0x99, 0x8e, 0x26, 0x63, 0xaa, 0xf5, 0x51, 0x71,
+ 0x90, 0xa2, 0x09, 0x57, 0x6d, 0x30, 0xfe, 0xa1,
+ 0x8e, 0xd6, 0x20, 0x32, 0xbb, 0x84, 0x50, 0xc6,
+ 0xde, 0xc7, 0x7e, 0xd0, 0x5c, 0xd0, 0x31, 0x93,
+ 0x1a, 0x09, 0x18, 0x90, 0x66, 0xc7, 0xc2, 0x05,
+ 0x25, 0xb9, 0x2f, 0x8c, 0xd6, 0x38, 0x11, 0x10,
+ 0x4e, 0x71, 0x26, 0x41, 0x7a, 0x27, 0x6a, 0x55,
+ 0xfc, 0x58, 0x2a, 0x67, 0xd6, 0x1f, 0x9f, 0x7f,
+ 0x57, 0x7c, 0xda, 0x7d, 0xa7, 0x78, 0x84, 0xdc,
+ 0x7a, 0xd7, 0x4d, 0xdc, 0xb6, 0x70, 0x30, 0xea,
+ 0xb9, 0xff, 0x49, 0x00, 0x66, 0x15, 0x22, 0x9e,
+ 0x97, 0x53, 0xd8, 0x16, 0x8c, 0xe1, 0x58, 0xb8,
+ 0x6e, 0x04, 0xe0, 0x79, 0xf9, 0xf4, 0x2f, 0x75,
+ 0x46, 0x06, 0xcf, 0x5b, 0x78, 0xd6, 0xc9, 0xd8,
+ 0xa2, 0xfd, 0x1b, 0x9b, 0x9a, 0x82, 0x16, 0x34,
+ 0xc6, 0x9c, 0x03, 0x95, 0x89, 0xc2, 0x7f, 0xe7,
+ 0x1e, 0xb4, 0x45, 0x43, 0xea, 0xa5, 0x91, 0x3d,
+ 0x26, 0xfe, 0x8c, 0xa8, 0x0f, 0x8a, 0xb8, 0x13,
+ 0x8e, 0x24, 0xc3, 0x75, 0xa5, 0x9f, 0xb3, 0x65,
+ 0x0c, 0x3e, 0x7a, 0x69, 0x5c, 0x01, 0x2c, 0x74,
+ 0xb7, 0x23, 0x1b, 0x1a, 0x07, 0x61, 0x57, 0x39,
+ 0xd7, 0x8d, 0xdd, 0x15, 0x68, 0x1d, 0xdd, 0x4f,
+ 0x3a, 0xc6, 0x48, 0x71, 0x3f, 0x1e, 0x69, 0xb4,
+ 0xb5, 0xfc, 0x4f, 0x25, 0x4a, 0xbe, 0x3f, 0x1d,
+ 0xed, 0x02, 0x7b, 0x82, 0x43, 0x9c, 0x4c, 0xee,
+ 0x5d, 0x16, 0x6e, 0xb6, 0xb3, 0xaa, 0x8f, 0x6b,
+ 0x94, 0x98, 0xe9, 0x83, 0x71, 0xa6, 0x4f, 0xc9,
+ 0xcc, 0x0e, 0xf2, 0x53, 0x85, 0x49, 0xea, 0xda,
+ 0x43, 0x5c, 0xdc, 0xf4, 0x63, 0x9e, 0xa3, 0x98,
+ 0x1d, 0x17, 0x10, 0x43, 0xe0, 0xea, 0x30, 0x34,
+ 0x75, 0x77, 0xc2, 0x3c, 0x56, 0xdc, 0x0c, 0x1b,
+ 0x57, 0xfe, 0x73, 0x02, 0xe9, 0x3e, 0x1a, 0xe3,
+ 0x44, 0xdd, 0xa1, 0x70, 0xed, 0xd9, 0x8c, 0x69,
+ 0xdb, 0xbb, 0xc9, 0x7d, 0x96, 0x38, 0x8b, 0x3a,
+ 0x7b, 0x99, 0x25, 0xaa, 0xab, 0xe9, 0x10, 0x2a,
+ 0xa7, 0xdb, 0xad, 0x0a, 0x33, 0xdc, 0x1d, 0x0a,
+ 0xc4, 0x63, 0x6b, 0x0b, 0x58, 0x67, 0xb5, 0x7a,
+ 0x93, 0xee, 0x5f, 0x74, 0xac, 0xa8, 0x50, 0x29,
+ 0x03, 0x91, 0xf7, 0x10, 0xb6, 0xbe, 0xc7, 0xd1,
+ 0xef, 0xc9, 0xa2, 0x61, 0x89, 0x6f, 0x5e, 0x04,
+ 0xb0, 0xfb, 0x46, 0x9e, 0x18, 0xe7, 0x52, 0x38,
+ 0xdd, 0x64, 0x76, 0xd7, 0x4f, 0xbb, 0xaa, 0x20,
+ 0x36, 0x8d, 0x86, 0xf4, 0x5d, 0x8b, 0x52, 0x66,
+ 0x5c, 0x17, 0x98, 0xc0, 0xc5, 0x5b, 0xc8, 0x00,
+ 0xa4, 0x99, 0x3d, 0x0f, 0x0b, 0x5b, 0x41, 0x96,
+ 0x48, 0xf3, 0xa5, 0x8c, 0x8e, 0xbb, 0x37, 0x3e,
+ 0x13 },
+ .ilen = 513,
+ .result = { 0x60, 0x64, 0xf1, 0xf3, 0x13, 0x5c, 0xf5, 0xd4,
+ 0xdc, 0xb9, 0x7b, 0xf8, 0x85, 0xf8, 0xac, 0x46,
+ 0x8c, 0x3b, 0x74, 0xa3, 0x6d, 0x15, 0x36, 0x01,
+ 0x42, 0x62, 0xfb, 0xef, 0x54, 0x37, 0x92, 0x4f,
+ 0xb6, 0x1a, 0x62, 0xa1, 0x0a, 0xb0, 0xf0, 0x52,
+ 0x26, 0x0d, 0xd5, 0x0e, 0x4e, 0x98, 0x1a, 0xd8,
+ 0x82, 0x60, 0x8b, 0x50, 0x50, 0xfd, 0xbb, 0xc8,
+ 0x90, 0xb7, 0x41, 0x26, 0x7f, 0x28, 0x4c, 0x42,
+ 0x50, 0xc1, 0x5c, 0x93, 0x66, 0x0c, 0xbc, 0x73,
+ 0x46, 0x59, 0xc3, 0x7d, 0x16, 0xfd, 0x72, 0x6d,
+ 0x9b, 0x09, 0x1e, 0xd2, 0xa7, 0xca, 0x4f, 0x96,
+ 0xca, 0x58, 0x12, 0x75, 0xdb, 0xbf, 0x1b, 0xfe,
+ 0xa3, 0x7d, 0x17, 0xf5, 0x93, 0x52, 0x0d, 0xbe,
+ 0xc3, 0x6b, 0x06, 0x82, 0x08, 0xec, 0x02, 0xa6,
+ 0x9c, 0x54, 0x74, 0x2f, 0x3f, 0x6e, 0x9e, 0x37,
+ 0x5f, 0x35, 0xd9, 0xd5, 0x39, 0x88, 0xd6, 0x0e,
+ 0xd9, 0x64, 0x17, 0xf2, 0xb2, 0x66, 0x08, 0xc2,
+ 0x0b, 0xa5, 0xd4, 0xc2, 0x01, 0xfa, 0x73, 0x4d,
+ 0x69, 0x48, 0xa0, 0x9d, 0x2b, 0xde, 0x2d, 0xbd,
+ 0xb7, 0xec, 0xc7, 0x55, 0x01, 0x4e, 0xfe, 0x0a,
+ 0xa8, 0xee, 0xb1, 0x5c, 0xad, 0x90, 0x3b, 0x94,
+ 0x41, 0x3b, 0xa3, 0x24, 0xe5, 0xc8, 0x5c, 0xb9,
+ 0x5f, 0x5e, 0x8e, 0x9c, 0x87, 0x57, 0xea, 0xb9,
+ 0x93, 0xbb, 0x68, 0xc1, 0xad, 0xc3, 0xcc, 0xdc,
+ 0x54, 0xc3, 0xab, 0xe8, 0xee, 0x63, 0xe3, 0x78,
+ 0x47, 0x52, 0x36, 0x7e, 0xf0, 0x04, 0xd7, 0xcd,
+ 0x23, 0xa9, 0xd6, 0x7b, 0x6b, 0x39, 0xe6, 0xf1,
+ 0xf1, 0xf4, 0xe6, 0x43, 0x43, 0x31, 0x2f, 0xe6,
+ 0x15, 0x24, 0x0e, 0x02, 0x56, 0x3a, 0xa2, 0x1f,
+ 0x82, 0xf6, 0xd1, 0x23, 0x96, 0x09, 0xf8, 0x7a,
+ 0x55, 0x66, 0x39, 0x4c, 0x5e, 0x77, 0xff, 0x20,
+ 0xf5, 0xef, 0x9e, 0x34, 0x24, 0xbc, 0x33, 0x31,
+ 0x27, 0xb5, 0x0b, 0x20, 0x3e, 0x78, 0x31, 0xa3,
+ 0x48, 0x9d, 0x51, 0xfe, 0x60, 0xc6, 0xb2, 0xec,
+ 0x76, 0x6b, 0x6c, 0xde, 0xfc, 0x53, 0x20, 0xa3,
+ 0x83, 0x0f, 0x7b, 0x82, 0xd9, 0x01, 0xf4, 0x03,
+ 0xba, 0x90, 0x57, 0x72, 0x8e, 0x7b, 0xa1, 0x17,
+ 0xed, 0x61, 0xe9, 0x57, 0x20, 0x37, 0x18, 0xc2,
+ 0x99, 0xa8, 0x62, 0x26, 0x9d, 0x7d, 0x6b, 0x4f,
+ 0xc1, 0xe5, 0x92, 0x77, 0x3f, 0x44, 0xfd, 0xcd,
+ 0xa4, 0x1d, 0x59, 0x51, 0xac, 0x8b, 0x76, 0xc5,
+ 0x64, 0xc7, 0x4d, 0xf6, 0x84, 0x42, 0xc5, 0x73,
+ 0x4d, 0xe6, 0x6f, 0xa7, 0xda, 0x15, 0x43, 0x32,
+ 0xd0, 0x2e, 0x5a, 0xee, 0x36, 0x42, 0x44, 0x77,
+ 0x92, 0xa9, 0xc0, 0xdc, 0xa9, 0x23, 0x59, 0xce,
+ 0x51, 0xf3, 0x67, 0x7a, 0xca, 0x06, 0xed, 0xc6,
+ 0xaf, 0x29, 0x0e, 0xa5, 0xe6, 0x3b, 0xeb, 0x1e,
+ 0x53, 0x48, 0x1a, 0x58, 0x13, 0x01, 0x51, 0x7f,
+ 0xb6, 0x54, 0x8a, 0xab, 0x02, 0x48, 0x73, 0xac,
+ 0x24, 0x6b, 0x5e, 0x5a, 0x2e, 0xf6, 0x63, 0x50,
+ 0xb6, 0x32, 0x0e, 0xcf, 0x93, 0xa1, 0x48, 0x7a,
+ 0xdb, 0x65, 0xab, 0x2e, 0x8a, 0x22, 0x96, 0x71,
+ 0x5a, 0xb9, 0xf7, 0xe4, 0x99, 0x25, 0xa4, 0x80,
+ 0xb5, 0x62, 0xe5, 0x4d, 0x2f, 0xcb, 0xce, 0x73,
+ 0x61, 0x8f, 0x00, 0xa3, 0xd5, 0x40, 0xec, 0xbb,
+ 0x4a, 0xa3, 0xe3, 0x95, 0x20, 0x8b, 0x6a, 0xd6,
+ 0x89, 0xa0, 0x91, 0x42, 0xb4, 0xe6, 0x4e, 0x49,
+ 0xfa, 0x73, 0xab, 0x1e, 0x22, 0x20, 0x30, 0x5e,
+ 0xe1, 0x94, 0x62, 0x48, 0x4d, 0x16, 0x4f, 0xea,
+ 0x44, 0x4c, 0x19, 0xea, 0x00, 0x29, 0x5c, 0xce,
+ 0x99, 0x32, 0xc0, 0xb3, 0x51, 0xe8, 0x39, 0x7b,
+ 0x1c, 0x8f, 0x24, 0xe3, 0x01, 0x5e, 0x65, 0xb2,
+ 0x5e, 0x80, 0x15, 0xdf, 0x17, 0x11, 0x1b, 0x40,
+ 0xe6, 0xdb, 0xe3, 0x60, 0x7b, 0x2f, 0xa8, 0xa7,
+ 0xb6 }
+}, {
+ .key = { 0xb2, 0xd2, 0x69, 0x85, 0x45, 0xb5, 0x4c, 0x3c,
+ 0xba, 0xb1, 0x01, 0xd4, 0x3d, 0xbe, 0x75, 0x91,
+ 0x98, 0x4c, 0x7b, 0x1d, 0xb1, 0xb0, 0x52, 0xf1,
+ 0xdd, 0xc3, 0x58, 0x1f, 0x2e, 0x9a, 0x72, 0x4f },
+ .nonce = 0x9aba2243ec59aca0ULL,
+ .input = { 0x6b, 0xf4, 0xfa, 0xf5, 0x28, 0x04, 0xfa, 0xd3,
+ 0x03, 0xab, 0x62, 0x69, 0x9f, 0x5e, 0x14, 0x83,
+ 0xe9, 0xf2, 0xe5, 0x98, 0xd9, 0x7e, 0x05, 0x22,
+ 0xf1, 0xf7, 0x0f, 0x76, 0x4a, 0x33, 0x9d, 0x49,
+ 0xa2, 0x16, 0xcf, 0x70, 0xd2, 0x94, 0x7e, 0xe5,
+ 0xcc, 0xe3, 0x01, 0x4d, 0x21, 0x14, 0x98, 0xef,
+ 0xbf, 0x18, 0x0e, 0x07, 0x00, 0x8a, 0x82, 0xcb,
+ 0x12, 0x65, 0xab, 0xd5, 0x2a, 0xe2, 0xe6, 0x07,
+ 0xe3, 0x3a, 0x78, 0x22, 0xb0, 0xd8, 0xce, 0x96,
+ 0xde, 0x47, 0x6f, 0xa8, 0xfb, 0x44, 0x00, 0x7b,
+ 0x33, 0x4e, 0xb4, 0x5d, 0x99, 0x9e, 0x14, 0x49,
+ 0xca, 0x90, 0x80, 0x11, 0xac, 0xa9, 0x9b, 0x6d,
+ 0x01, 0x24, 0x9b, 0x05, 0x6d, 0x57, 0x35, 0x2a,
+ 0xde, 0xd2, 0xb6, 0x54, 0x93, 0x55, 0x08, 0xef,
+ 0xfb, 0x19, 0xd0, 0x3e, 0xf2, 0x95, 0xda, 0x75,
+ 0x0e, 0x27, 0x38, 0x14, 0xcf, 0xd2, 0x4e, 0xb7,
+ 0x27, 0xd1, 0xbd, 0x52, 0x08, 0x7e, 0x6c, 0xe4,
+ 0xfd, 0xc5, 0xc3, 0x2a, 0x76, 0x84, 0xf8, 0x4a,
+ 0x4e, 0x10, 0x90, 0xb0, 0x0c, 0x32, 0x25, 0x9f,
+ 0x5f, 0x75, 0x2a, 0x24, 0x57, 0x45, 0xd5, 0x64,
+ 0x32, 0xf7, 0x30, 0x5a, 0x13, 0xbd, 0x79, 0x1c,
+ 0x50, 0x46, 0xf7, 0xbd, 0xdc, 0x32, 0x3c, 0x43,
+ 0x1c, 0x84, 0xe6, 0x73, 0x78, 0x01, 0x94, 0xe2,
+ 0x49, 0x34, 0x32, 0xd9, 0x86, 0x2c, 0xff, 0xa6,
+ 0x6a, 0x46, 0x9c, 0xfc, 0xf3, 0x22, 0x31, 0xef,
+ 0x60, 0xb8, 0xdc, 0x2e, 0xcc, 0x0e, 0x8c, 0x3a,
+ 0xbb, 0xba, 0xb0, 0x36, 0xa0, 0x97, 0x78, 0x09,
+ 0x3a, 0x87, 0xa5, 0xef, 0xc6, 0x50, 0x87, 0x02,
+ 0xfa, 0xb8, 0x3e, 0x55, 0x2d, 0x78, 0xc8, 0xa8,
+ 0x8a, 0xa6, 0x63, 0xba, 0x31, 0x72, 0x56, 0x39,
+ 0x7d, 0xb8, 0x4a, 0xe4, 0x1c, 0xe9, 0x7b, 0x8c,
+ 0x97, 0x25, 0x3f, 0x25, 0xea, 0x8a, 0x6e, 0x2a,
+ 0x09, 0x83, 0x17, 0x12, 0x9b, 0x3b, 0xb8, 0xe6,
+ 0xf8, 0x0f, 0x48, 0x68, 0x9a, 0xcf, 0x73, 0x95,
+ 0xf1, 0x4c, 0x1b, 0x9b, 0x2e, 0x64, 0x17, 0x6c,
+ 0xb1, 0xcd, 0x90, 0x7a, 0xf6, 0x90, 0xf5, 0xa1,
+ 0x22, 0x9a, 0x6b, 0x16, 0xfc, 0xc8, 0x86, 0x5a,
+ 0xe7, 0x68, 0x53, 0x2b, 0x31, 0xd0, 0x06, 0x63,
+ 0x81, 0xdf, 0xd2, 0x9b, 0x7f, 0x60, 0x46, 0xf7,
+ 0xf3, 0xef, 0xb5, 0x71, 0xa4, 0x96, 0xf7, 0xe2,
+ 0xd1, 0x9b, 0x9d, 0x71, 0x0e, 0x06, 0xb5, 0x6e,
+ 0x6e, 0x14, 0x4b, 0xcf, 0xdd, 0x9e, 0x53, 0x71,
+ 0x54, 0xb9, 0xfa, 0x4f, 0xb4, 0x25, 0x48, 0xa4,
+ 0x8a, 0xf8, 0xca, 0x1e, 0x56, 0xc6, 0xa6, 0x19,
+ 0x11, 0x67, 0x9f, 0xb6, 0x59, 0xa1, 0x80, 0x97,
+ 0xb5, 0xdd, 0x50, 0x95, 0x94, 0xbe, 0x8f, 0x65,
+ 0x05, 0xe3, 0xa2, 0xba, 0x22, 0x36, 0x04, 0x1b,
+ 0x44, 0xdf, 0xd8, 0x2c, 0x08, 0xb4, 0x6b, 0x40,
+ 0xec, 0x5d, 0x7f, 0xbb, 0x94, 0x25, 0x3c, 0xf3,
+ 0x73, 0x52, 0xd4, 0xc5, 0x63, 0x54, 0xde, 0x0f,
+ 0x4d, 0xd6, 0xfe, 0x5e, 0xf9, 0x02, 0x84, 0x31,
+ 0x28, 0xee, 0xfd, 0x0e, 0xbe, 0xaf, 0xe0, 0x75,
+ 0x1c, 0xfd, 0xcc, 0xcf, 0x4e, 0x67, 0xd6, 0x86,
+ 0x9c, 0x9d, 0x3c, 0x87, 0xb2, 0x89, 0x79, 0x9b,
+ 0xd7, 0x8e, 0xca, 0x90, 0xb6, 0xdd, 0x4a, 0xf1,
+ 0x0a, 0xed, 0xaa, 0xc1, 0x80, 0xf9, 0x96, 0x49,
+ 0xce, 0xa3, 0x12, 0x63, 0xae, 0xc6, 0xcb, 0x3b,
+ 0xf1, 0xe7, 0xb9, 0x1d, 0x0b, 0x0d, 0xd4, 0x1c,
+ 0x48, 0x30, 0x53, 0xab, 0xba, 0x94, 0x45, 0xc4,
+ 0x5c, 0xaf, 0xe0, 0xe7, 0x81, 0x61, 0x2f, 0xa1,
+ 0x02, 0x12, 0x55, 0x5c, 0xde, 0x3a, 0x48, 0x7c,
+ 0xe0, 0x3b, 0xab, 0x37, 0x18, 0xf1, 0x1b, 0x8b,
+ 0xba, 0x6c, 0x4a, 0xef, 0x5b, 0x95, 0xdb, 0xd8,
+ 0xab, 0x8d, 0xde, 0x27, 0xdf, 0x44, 0xcd, 0xb1,
+ 0x8d, 0x97 },
+ .ilen = 514,
+ .result = { 0x20, 0xaf, 0x08, 0x43, 0x2c, 0x9b, 0xff, 0x20,
+ 0x30, 0xc3, 0xec, 0xec, 0x81, 0x08, 0x9b, 0x8d,
+ 0x6e, 0x2a, 0x48, 0xef, 0x65, 0x01, 0xaf, 0x1a,
+ 0xe9, 0x46, 0x32, 0x74, 0x28, 0x50, 0xd4, 0x8a,
+ 0xc9, 0xf3, 0x2c, 0x6c, 0x6b, 0xad, 0x79, 0xa3,
+ 0x61, 0x2c, 0x7e, 0x41, 0xf0, 0xef, 0x74, 0x90,
+ 0xc2, 0xc2, 0xfb, 0xc1, 0x41, 0x48, 0xcb, 0xa2,
+ 0xc3, 0x37, 0x31, 0x63, 0x97, 0x28, 0xd8, 0x7a,
+ 0x6e, 0xa7, 0x57, 0xfb, 0xb2, 0x7d, 0xfc, 0xe1,
+ 0x78, 0xe9, 0x32, 0x2c, 0x74, 0xed, 0x29, 0xf7,
+ 0xb7, 0x4c, 0x65, 0x04, 0x09, 0x47, 0x22, 0x2a,
+ 0x16, 0x3e, 0xe4, 0xff, 0x53, 0x2a, 0x4f, 0xea,
+ 0x7c, 0x31, 0x12, 0x0d, 0xc0, 0x55, 0xc8, 0x6b,
+ 0x97, 0x7f, 0x2e, 0x64, 0xc1, 0x7b, 0x46, 0xb9,
+ 0x9a, 0xfa, 0xcc, 0x36, 0x94, 0x8e, 0x44, 0x06,
+ 0xe2, 0xa6, 0xf2, 0xef, 0x8b, 0x6a, 0x45, 0x44,
+ 0x2e, 0xc8, 0x6a, 0x3c, 0x1b, 0xa5, 0xa8, 0x01,
+ 0xf5, 0x27, 0x23, 0x27, 0xb0, 0xe0, 0x97, 0x69,
+ 0x73, 0x08, 0xa4, 0x3e, 0x71, 0x79, 0xce, 0xa7,
+ 0xce, 0x38, 0xd8, 0x1c, 0xb8, 0xf3, 0x7f, 0x03,
+ 0x6a, 0x4f, 0x19, 0x05, 0xf0, 0x09, 0xe8, 0xb2,
+ 0x19, 0xd4, 0x12, 0xcd, 0x46, 0x87, 0xc2, 0xc5,
+ 0xd2, 0xe3, 0x04, 0x98, 0x97, 0x16, 0xda, 0xd6,
+ 0x39, 0x3a, 0xfe, 0xa8, 0x77, 0x69, 0xc7, 0xe9,
+ 0xb2, 0xe7, 0x65, 0x64, 0x06, 0x22, 0xe7, 0xc2,
+ 0xd4, 0x26, 0x10, 0xf5, 0xd3, 0x01, 0x9f, 0x3d,
+ 0x32, 0x90, 0xbf, 0xd0, 0x19, 0x80, 0xbd, 0xe5,
+ 0xde, 0xe8, 0x15, 0xb3, 0xe9, 0x13, 0x2e, 0x5b,
+ 0x59, 0x2f, 0x3a, 0x24, 0xb2, 0xce, 0xfa, 0x94,
+ 0xe2, 0xc7, 0x69, 0x05, 0xa5, 0xc4, 0xb2, 0x30,
+ 0xfb, 0x48, 0xd9, 0x75, 0x94, 0xf1, 0x85, 0xb5,
+ 0xff, 0xe3, 0xe9, 0xd5, 0xa6, 0xcf, 0x12, 0x46,
+ 0xc5, 0x9a, 0x70, 0xf3, 0xec, 0xbe, 0x2c, 0x24,
+ 0xd2, 0xa2, 0x17, 0x67, 0x7a, 0x46, 0x8f, 0xb4,
+ 0x96, 0xa4, 0xab, 0xf6, 0xab, 0xb4, 0x7b, 0x31,
+ 0x6d, 0xf0, 0x37, 0x8f, 0x98, 0xda, 0x3c, 0x56,
+ 0xd3, 0x5b, 0xe1, 0xd0, 0x57, 0x04, 0xf5, 0xcc,
+ 0x0c, 0x20, 0x4b, 0xd2, 0x17, 0xed, 0x3c, 0x4b,
+ 0x32, 0xa4, 0xd8, 0x87, 0xaf, 0xae, 0xf9, 0x45,
+ 0x26, 0x9d, 0xc5, 0xe4, 0x61, 0x31, 0xf0, 0xde,
+ 0x7b, 0x55, 0x1b, 0x7e, 0xc9, 0x01, 0x91, 0x07,
+ 0xb0, 0x45, 0xf1, 0x1c, 0x75, 0xa9, 0xf5, 0x58,
+ 0xbd, 0x4c, 0x49, 0x4e, 0xd8, 0x73, 0x37, 0x08,
+ 0x92, 0x05, 0x87, 0xba, 0xa3, 0xc8, 0x14, 0xa0,
+ 0xee, 0x21, 0xd7, 0xf6, 0x03, 0x37, 0xe3, 0xe0,
+ 0x4f, 0xea, 0x85, 0x94, 0x1b, 0x45, 0x92, 0x0a,
+ 0xa0, 0xea, 0xba, 0xc2, 0xb9, 0x88, 0x29, 0xe8,
+ 0xbd, 0x81, 0xda, 0x95, 0x45, 0xf5, 0x5f, 0x20,
+ 0x94, 0x80, 0x21, 0x36, 0x10, 0x6a, 0xb7, 0x90,
+ 0xc0, 0xa9, 0x9e, 0x2a, 0x20, 0xa9, 0xd5, 0x61,
+ 0xf4, 0x72, 0x56, 0xaf, 0x00, 0x9a, 0x33, 0xf6,
+ 0x51, 0x12, 0xfc, 0x84, 0x15, 0x36, 0xc5, 0x27,
+ 0x4f, 0x44, 0xdf, 0x64, 0xde, 0xd7, 0xe1, 0xe9,
+ 0xc8, 0x08, 0xa3, 0x42, 0x60, 0xec, 0x1f, 0xbe,
+ 0x9f, 0x7b, 0x79, 0x47, 0xff, 0xea, 0x6f, 0x4f,
+ 0xc6, 0xea, 0x34, 0x54, 0x09, 0xb2, 0xa1, 0xde,
+ 0xb4, 0x54, 0xb9, 0x82, 0x9e, 0x41, 0xad, 0xa6,
+ 0x4f, 0x8f, 0xac, 0x3b, 0x01, 0x04, 0xbb, 0x9c,
+ 0x6a, 0xac, 0x24, 0xc1, 0xa8, 0x91, 0xb7, 0x5b,
+ 0x88, 0x88, 0xf3, 0x42, 0x17, 0xd4, 0xb7, 0xfc,
+ 0xc8, 0x80, 0x1b, 0x54, 0x91, 0x47, 0xd8, 0x1b,
+ 0xbc, 0x12, 0x7e, 0x4a, 0x03, 0x3b, 0x78, 0x83,
+ 0x2c, 0x92, 0x92, 0xa4, 0x31, 0xf2, 0xe8, 0x13,
+ 0x36, 0x0e, 0xc6, 0xb4, 0xed, 0x5c, 0x22, 0x71,
+ 0xa0, 0xa9 }
+}, {
+ .key = { 0x61, 0x93, 0x3b, 0xc8, 0x4b, 0x53, 0x4f, 0xc1,
+ 0x9e, 0x74, 0x24, 0x7f, 0x00, 0xcb, 0xc8, 0x33,
+ 0x42, 0x55, 0x6f, 0x77, 0xf1, 0x35, 0x52, 0xb0,
+ 0x67, 0xe7, 0x1d, 0x6e, 0xf1, 0xf6, 0x1c, 0x4d },
+ .nonce = 0xa3030d135287782eULL,
+ .input = { 0x83, 0x01, 0x49, 0xa9, 0x1b, 0x38, 0x85, 0x91,
+ 0x26, 0x41, 0x91, 0x20, 0xb2, 0x9e, 0x98, 0xaf,
+ 0x27, 0x30, 0xe1, 0xa6, 0x44, 0x4d, 0x2a, 0xaa,
+ 0xb3, 0x50, 0x7b, 0x27, 0x12, 0x86, 0x8b, 0xba,
+ 0x18, 0xaf, 0xc8, 0x8c, 0x68, 0x1b, 0x51, 0x1e,
+ 0x99, 0x89, 0x15, 0xcb, 0xd1, 0x5e, 0xaa, 0xe6,
+ 0x7f, 0x0d, 0x78, 0x7a, 0x66, 0xfc, 0xb5, 0x09,
+ 0x0d, 0x6e, 0xeb, 0xc1, 0x8b, 0x92, 0x5f, 0x94,
+ 0x83, 0xb5, 0x3e, 0x5a, 0x73, 0x28, 0xca, 0xb4,
+ 0x99, 0x9d, 0xe1, 0x17, 0x64, 0x12, 0x59, 0xb7,
+ 0xbe, 0x50, 0xf1, 0x52, 0xd3, 0x34, 0x93, 0x3c,
+ 0xa3, 0xbc, 0xc2, 0x00, 0x86, 0x7e, 0x40, 0x9e,
+ 0xd6, 0x65, 0xf8, 0xf3, 0x8a, 0x60, 0x7c, 0x63,
+ 0x01, 0xc7, 0x1a, 0xbd, 0xe5, 0x0c, 0x91, 0xfe,
+ 0x6b, 0x6c, 0xae, 0xca, 0x9b, 0x79, 0xa6, 0x07,
+ 0xa3, 0x00, 0xc0, 0x63, 0x3b, 0xfb, 0xc6, 0xa5,
+ 0xeb, 0x36, 0xd5, 0xaa, 0xc5, 0x28, 0xed, 0x37,
+ 0x04, 0x90, 0xd2, 0x4d, 0x08, 0x5e, 0x23, 0x8b,
+ 0xe7, 0xf3, 0x93, 0xf4, 0x91, 0xc6, 0xcf, 0x13,
+ 0x24, 0x0a, 0x2d, 0x6a, 0x81, 0x3b, 0xaa, 0xd9,
+ 0x4c, 0xe0, 0x57, 0xe1, 0x11, 0x62, 0xf5, 0x58,
+ 0x7f, 0xcd, 0x7b, 0xe6, 0xa8, 0xc3, 0x5a, 0xd5,
+ 0xc3, 0xd0, 0x32, 0xcf, 0x14, 0x10, 0x4c, 0x79,
+ 0x2d, 0x67, 0x4d, 0x1c, 0xe7, 0x93, 0x2e, 0x1f,
+ 0x9c, 0x51, 0x31, 0xb9, 0x3c, 0x8b, 0x00, 0x72,
+ 0x95, 0x16, 0xb8, 0x5e, 0xf6, 0x6c, 0xfa, 0xe6,
+ 0x7c, 0xbb, 0xa6, 0xef, 0x73, 0x4b, 0xcd, 0x10,
+ 0x32, 0x6b, 0x0d, 0xcb, 0x82, 0x3c, 0x1a, 0x2d,
+ 0xde, 0x7a, 0xc8, 0xcc, 0x23, 0xce, 0x06, 0x6a,
+ 0x92, 0x65, 0xe7, 0x00, 0xe3, 0xe5, 0xf1, 0x46,
+ 0x9e, 0x6a, 0xe8, 0xfb, 0x8c, 0x2c, 0x52, 0x4b,
+ 0x9a, 0x0a, 0x66, 0x26, 0xdc, 0x6f, 0x25, 0x86,
+ 0x2b, 0x63, 0x6b, 0x57, 0x4f, 0x62, 0xb2, 0xe6,
+ 0x5c, 0xbf, 0x55, 0xd9, 0x65, 0x88, 0x8e, 0xb5,
+ 0xc1, 0xfb, 0x0e, 0x80, 0xb5, 0x76, 0x0b, 0x1f,
+ 0xe6, 0x49, 0x8b, 0x60, 0xef, 0xad, 0xb6, 0x47,
+ 0xc2, 0x0e, 0x4c, 0xc7, 0xf8, 0xe6, 0xcb, 0x83,
+ 0x41, 0xf8, 0xd6, 0x72, 0x17, 0xfc, 0x62, 0x03,
+ 0x9f, 0x42, 0x38, 0x3d, 0xe4, 0x26, 0x2b, 0x8c,
+ 0xf8, 0xa4, 0x1d, 0x05, 0x7b, 0x03, 0x3f, 0x64,
+ 0xec, 0x64, 0x77, 0x29, 0xdd, 0xb3, 0x02, 0xf9,
+ 0xb3, 0x0c, 0x60, 0x3d, 0xcb, 0x4d, 0x61, 0x8c,
+ 0xf2, 0x07, 0x2b, 0x0d, 0x07, 0x67, 0xf7, 0xd6,
+ 0x7e, 0xf1, 0xe8, 0x0e, 0xbb, 0x93, 0x9b, 0x3a,
+ 0xe5, 0xe1, 0x03, 0x75, 0x66, 0xe0, 0x10, 0x7e,
+ 0x7b, 0x35, 0x54, 0xc1, 0xd4, 0xec, 0xfa, 0x66,
+ 0x55, 0x22, 0x22, 0x6a, 0xee, 0x30, 0x7c, 0x8f,
+ 0xc9, 0x5d, 0xb9, 0x1f, 0x86, 0x5c, 0x7d, 0x71,
+ 0xc5, 0x2b, 0x74, 0x0a, 0x24, 0x15, 0x8f, 0x91,
+ 0x7b, 0xb0, 0x6b, 0x0b, 0x02, 0x84, 0x92, 0x4a,
+ 0x68, 0x22, 0x63, 0xb4, 0xf1, 0x4f, 0x94, 0x69,
+ 0x25, 0x24, 0x87, 0xef, 0x12, 0x36, 0xb9, 0x75,
+ 0x96, 0x42, 0x65, 0xb6, 0x6d, 0x8b, 0x5d, 0x7b,
+ 0x99, 0x0c, 0xc8, 0xbb, 0x87, 0x83, 0x21, 0xe4,
+ 0xb0, 0xe4, 0xa5, 0x76, 0x0c, 0xe6, 0x2d, 0x70,
+ 0x8e, 0xb2, 0xf9, 0xd0, 0xa8, 0xf2, 0x93, 0xfa,
+ 0x7d, 0xf4, 0x08, 0x58, 0x1d, 0x53, 0xee, 0x1c,
+ 0x74, 0xdf, 0x83, 0x16, 0xbb, 0x84, 0x30, 0xe0,
+ 0x06, 0xf6, 0x4b, 0x05, 0x03, 0xd8, 0x7d, 0xbc,
+ 0x04, 0xc8, 0x44, 0x2a, 0x5d, 0xe1, 0xdc, 0x06,
+ 0xea, 0x61, 0xb7, 0xde, 0xea, 0x27, 0x0f, 0x74,
+ 0x9a, 0x6d, 0xed, 0x1f, 0x03, 0x11, 0xd3, 0xca,
+ 0xf0, 0x6c, 0x3c, 0x62, 0x5c, 0x6a, 0x64, 0xa2,
+ 0x1d, 0x1b, 0xec, 0x21, 0xd0, 0xaa, 0x92, 0xdb,
+ 0x89, 0xac, 0x19, 0xd2, 0xd6, 0xd3, 0x52, 0xa4,
+ 0x36, 0x5b, 0x66, 0xe2, 0xea, 0xa3, 0xe8, 0x6a,
+ 0x1f, 0x8f, 0xaf, 0x6d, 0xfe, 0xcc, 0x5d, 0x9f,
+ 0x88, 0x70, 0xe7, 0x41, 0xa1, 0xb5, 0x3e, 0x6a,
+ 0x2a, 0x94, 0xc5, 0x33, 0x71, 0xb9, 0xea, 0x74,
+ 0xf4, 0x46, 0x8f, 0x71, 0x0e, 0x8b, 0x84, 0x1b,
+ 0xaf, 0xc8, 0x50, 0x74, 0x60, 0xe1, 0xf2, 0x44,
+ 0xc1, 0xac, 0x77, 0x50, 0x93, 0x4e, 0xb1, 0x3c,
+ 0xd3, 0xcf, 0x0c, 0xdf, 0xc1, 0xb9, 0x28, 0x4f,
+ 0xe1, 0x6b, 0xd6, 0xeb, 0x88, 0x57, 0x1c, 0x98,
+ 0xa6, 0xc7, 0x4c, 0xf6, 0xb6, 0x27, 0x9d, 0xfb,
+ 0xc5, 0xc2, 0x7c, 0xf3, 0xa7, 0x29, 0x72, 0xca,
+ 0x3f, 0xf3, 0x56, 0x42, 0xca, 0x50, 0xaf, 0x76,
+ 0x88, 0xd1, 0x7d, 0x88, 0x91, 0x0d, 0x80, 0x05,
+ 0x4f, 0xc1, 0xa4, 0xe4, 0xd5, 0x17, 0xd6, 0xa7,
+ 0xd3, 0xb7, 0x85, 0x44, 0x80, 0x70, 0xee, 0x6a,
+ 0x27, 0x4d, 0x52, 0xd8, 0xfb, 0x7b, 0x51, 0x6c,
+ 0x44, 0x8f, 0x44, 0x4d, 0x74, 0x11, 0x65, 0xac,
+ 0x04, 0x4a, 0x4e, 0xef, 0x98, 0x89, 0x20, 0x09,
+ 0x34, 0x19, 0xfe, 0x38, 0x29, 0xd4, 0x53, 0xee,
+ 0xf9, 0x37, 0x1f, 0xc0, 0x2a, 0x5e, 0xa4, 0x3e,
+ 0x3f, 0x30, 0xd4, 0xaf, 0xa5, 0x3c, 0x77, 0xe4,
+ 0xf5, 0xf7, 0xad, 0x55, 0xd8, 0xc9, 0x90, 0x00,
+ 0x03, 0xa3, 0x9e, 0xb1, 0x84, 0xb0, 0xbb, 0x46,
+ 0x9a, 0xdd, 0xe7, 0xdd, 0xaa, 0x8a, 0xa2, 0xd7,
+ 0xa8, 0x03, 0x63, 0xf4, 0x16, 0x7c, 0x3e, 0xa0,
+ 0xa8, 0x32, 0x9e, 0x3f, 0x16, 0xf0, 0xa5, 0x7a,
+ 0xff, 0x44, 0xfc, 0x60, 0xb1, 0x65, 0xf4, 0xf5,
+ 0xb1, 0x11, 0x7c, 0xe5, 0x0d, 0x78, 0xdc, 0x1a,
+ 0xc3, 0xe0, 0x3f, 0x8c, 0x5e, 0x5a, 0x6b, 0x2f,
+ 0x11, 0xbe, 0xc9, 0xff, 0xe6, 0x84, 0xbf, 0x62,
+ 0x93, 0xdb, 0xe7, 0xe6, 0x30, 0x56, 0x99, 0x4e,
+ 0x85 },
+ .ilen = 769,
+ .result = { 0x49, 0xbd, 0x98, 0x31, 0xb8, 0x44, 0x59, 0x02,
+ 0x7e, 0x1e, 0x7a, 0x4f, 0x77, 0x71, 0xd8, 0x5d,
+ 0xef, 0x8d, 0xcb, 0xa4, 0x61, 0x76, 0xab, 0xd8,
+ 0x07, 0x75, 0x86, 0x51, 0x5b, 0xcf, 0xc3, 0xd8,
+ 0x6c, 0x36, 0xe9, 0x1f, 0x24, 0x48, 0xe1, 0xca,
+ 0xff, 0x73, 0x0d, 0x30, 0xc1, 0xff, 0x8f, 0x6c,
+ 0x51, 0xf0, 0x58, 0x7f, 0x1c, 0xba, 0x94, 0x16,
+ 0x59, 0x81, 0x7f, 0x04, 0xcf, 0x58, 0xa0, 0x5f,
+ 0x64, 0x35, 0x55, 0x3e, 0x5d, 0x2e, 0x45, 0x55,
+ 0x71, 0xd1, 0xc9, 0x43, 0x9d, 0x3a, 0xf2, 0x93,
+ 0xed, 0x05, 0x25, 0x2e, 0xc0, 0x8c, 0x09, 0x0b,
+ 0x06, 0x3a, 0x89, 0x26, 0xd3, 0x32, 0x94, 0xc7,
+ 0x66, 0xd1, 0x13, 0xb0, 0x12, 0xd9, 0x99, 0x37,
+ 0x08, 0x14, 0xe4, 0xd5, 0x32, 0x07, 0x3d, 0xcc,
+ 0xa6, 0xc4, 0x25, 0x2f, 0x49, 0x60, 0xa9, 0x78,
+ 0x4c, 0x1c, 0x42, 0xdb, 0x46, 0xa0, 0x6e, 0x84,
+ 0x09, 0x01, 0xe8, 0x69, 0xb0, 0x6e, 0xc4, 0x3c,
+ 0xd9, 0x53, 0x5a, 0x8e, 0xf0, 0xe5, 0xc6, 0x73,
+ 0x7d, 0x46, 0x8c, 0x49, 0x6e, 0xfd, 0x7e, 0x7f,
+ 0xb1, 0xb6, 0xd6, 0x4e, 0x94, 0xce, 0xe1, 0x55,
+ 0x9b, 0x8f, 0x96, 0x46, 0x62, 0x43, 0x59, 0x9c,
+ 0x8f, 0x07, 0x85, 0xad, 0xac, 0x53, 0x54, 0xda,
+ 0xce, 0x99, 0xb8, 0x8d, 0x23, 0x5f, 0x15, 0xcd,
+ 0x13, 0x00, 0xf8, 0xc4, 0x73, 0x74, 0xef, 0x6e,
+ 0x69, 0x9d, 0x10, 0x7e, 0x32, 0xaa, 0xab, 0x72,
+ 0xd9, 0xa0, 0x35, 0x96, 0xbf, 0xbe, 0xae, 0xf5,
+ 0x1d, 0xbc, 0x73, 0x2b, 0x07, 0x0c, 0x1d, 0xb5,
+ 0xfe, 0x9d, 0xc2, 0xda, 0x57, 0x0e, 0xcb, 0x2c,
+ 0x6a, 0x47, 0xab, 0x27, 0xd5, 0x02, 0xc4, 0x4c,
+ 0xc4, 0xb2, 0x1d, 0xca, 0x0d, 0x13, 0xa3, 0x2c,
+ 0x93, 0x23, 0x7d, 0xc9, 0xe9, 0x6b, 0x39, 0xaa,
+ 0xba, 0x24, 0xaf, 0x09, 0x42, 0x5c, 0xfe, 0x63,
+ 0xa2, 0x0e, 0x15, 0x0b, 0x06, 0xf8, 0x1a, 0xfc,
+ 0x9c, 0x03, 0x94, 0x90, 0x0d, 0x2b, 0xdc, 0x7a,
+ 0x08, 0xf3, 0xdd, 0xc6, 0x34, 0xac, 0x2a, 0xfb,
+ 0xd1, 0x5b, 0x04, 0x27, 0x4f, 0x63, 0x76, 0xbb,
+ 0x03, 0x73, 0xb5, 0x9c, 0x49, 0x8b, 0xb5, 0xe7,
+ 0x23, 0xb9, 0x79, 0xf6, 0x4b, 0x48, 0xe4, 0x74,
+ 0x7c, 0xa4, 0xb0, 0x6b, 0xbf, 0x0c, 0x61, 0x9e,
+ 0x19, 0x57, 0xaa, 0x38, 0x40, 0xbf, 0x7b, 0x33,
+ 0x28, 0xeb, 0x96, 0x3d, 0xc2, 0xb6, 0x2e, 0x3e,
+ 0xc7, 0xc9, 0xb5, 0x7b, 0x3e, 0x2c, 0x79, 0x94,
+ 0x10, 0xb5, 0xcd, 0x5b, 0x52, 0xef, 0x57, 0x41,
+ 0x98, 0xae, 0x51, 0x6b, 0x0d, 0x4c, 0x42, 0xe3,
+ 0x9e, 0x62, 0x25, 0xd1, 0x71, 0x19, 0x73, 0x4f,
+ 0x5e, 0x52, 0x76, 0x8e, 0x5f, 0xd7, 0xa2, 0xa9,
+ 0xd2, 0x60, 0x44, 0x81, 0x87, 0xe8, 0x2c, 0xb2,
+ 0xf1, 0xa7, 0x2a, 0x2b, 0x40, 0xf9, 0x0a, 0x01,
+ 0x83, 0xfc, 0x8d, 0xcc, 0x83, 0x28, 0xf4, 0xad,
+ 0x6e, 0xcb, 0xe1, 0x02, 0x89, 0xd7, 0x14, 0x73,
+ 0xd7, 0x56, 0x1e, 0x41, 0xf3, 0x0b, 0xa4, 0xf2,
+ 0x02, 0x8b, 0xc1, 0x9e, 0xe6, 0xd1, 0x89, 0xbd,
+ 0x20, 0xf3, 0x1a, 0xfc, 0x8e, 0x7c, 0xde, 0xb9,
+ 0x2a, 0x51, 0x67, 0xeb, 0x0e, 0x32, 0x3f, 0x88,
+ 0xc1, 0x26, 0xd9, 0xb8, 0x6e, 0xb0, 0x20, 0x4c,
+ 0xfa, 0x4b, 0xfa, 0x21, 0xd9, 0xd3, 0x52, 0xdd,
+ 0xbc, 0x20, 0x91, 0xe7, 0x45, 0x03, 0xe7, 0x3e,
+ 0x40, 0xb0, 0x76, 0xc7, 0xb5, 0xec, 0xb3, 0x67,
+ 0x8f, 0xba, 0xfd, 0xf3, 0x15, 0x81, 0xdc, 0x2f,
+ 0x03, 0x8d, 0xf0, 0x69, 0x9f, 0xd6, 0xda, 0x8f,
+ 0x98, 0xa0, 0x04, 0xb3, 0x73, 0x7b, 0xf2, 0xbd,
+ 0xf2, 0x9e, 0xc8, 0x30, 0x0f, 0x40, 0x65, 0x1f,
+ 0x67, 0xda, 0x7d, 0x56, 0x53, 0x66, 0xff, 0xe9,
+ 0x15, 0xe4, 0x4b, 0x61, 0x26, 0x62, 0x8f, 0x5a,
+ 0xd0, 0xe7, 0xc4, 0x53, 0x5f, 0x96, 0x4d, 0x82,
+ 0x50, 0x61, 0x64, 0x1b, 0xe3, 0xe3, 0xbb, 0x6b,
+ 0x4f, 0x1d, 0x58, 0xf2, 0xde, 0xfd, 0x59, 0x0c,
+ 0x62, 0xa9, 0xc3, 0x65, 0x45, 0xd8, 0x82, 0x45,
+ 0xaa, 0x06, 0x4c, 0xc1, 0x6e, 0x95, 0x9a, 0x7a,
+ 0x2b, 0x1c, 0x91, 0xc1, 0x87, 0x45, 0x47, 0x71,
+ 0xca, 0xea, 0x0c, 0xa6, 0x70, 0x08, 0x2d, 0x1b,
+ 0x26, 0x38, 0xaa, 0x41, 0x0b, 0xe7, 0xa1, 0xe6,
+ 0x6e, 0xa4, 0x7c, 0x8d, 0xf1, 0x1a, 0xd8, 0x05,
+ 0xb3, 0x69, 0x0b, 0x06, 0xc4, 0x49, 0x84, 0x2d,
+ 0x08, 0xe6, 0x57, 0xe9, 0x64, 0x02, 0xd2, 0xc0,
+ 0x73, 0xf1, 0x6d, 0x88, 0x07, 0x48, 0x47, 0x7c,
+ 0xe4, 0x42, 0x4b, 0x5a, 0x65, 0xb4, 0xc3, 0x65,
+ 0xcc, 0xd6, 0xc6, 0x5d, 0xc7, 0x21, 0xc8, 0x65,
+ 0x28, 0x96, 0xd6, 0xad, 0x2b, 0xc7, 0x49, 0xbe,
+ 0x98, 0xb3, 0x43, 0xb8, 0x9a, 0xd1, 0xe9, 0xa0,
+ 0xed, 0x8d, 0x8c, 0x2c, 0x00, 0xa6, 0x1e, 0x3a,
+ 0xb5, 0x0d, 0xeb, 0xcf, 0x12, 0xe6, 0x0e, 0x4f,
+ 0x7d, 0x87, 0xca, 0x3d, 0x37, 0xaf, 0xb4, 0xa3,
+ 0xdc, 0xd4, 0xf7, 0xe2, 0x7e, 0xfc, 0xa7, 0x1b,
+ 0x92, 0x79, 0x63, 0x10, 0xce, 0x7b, 0x59, 0x87,
+ 0x89, 0x49, 0xb7, 0x50, 0x86, 0x7c, 0xa7, 0x5f,
+ 0x3f, 0x50, 0x31, 0x46, 0xd4, 0xfa, 0x6e, 0xcd,
+ 0xf4, 0xbe, 0xce, 0xbe, 0xd0, 0x71, 0xef, 0x2c,
+ 0xfe, 0xf1, 0x3c, 0xf0, 0xfd, 0xef, 0x96, 0x73,
+ 0xa3, 0xa7, 0x23, 0x35, 0x6c, 0x27, 0xef, 0x78,
+ 0xec, 0xdd, 0x46, 0xb7, 0xe4, 0xce, 0x37, 0x43,
+ 0x35, 0xf2, 0x38, 0x35, 0xf5, 0xb5, 0xc2, 0x9f,
+ 0xf0, 0x97, 0x52, 0x23, 0x0c, 0x74, 0x03, 0xd1,
+ 0x53, 0xd9, 0xf0, 0x73, 0xb6, 0xae, 0x4c, 0xa3,
+ 0xac, 0x6f, 0x4c, 0x93, 0x72, 0x47, 0x7b, 0x44,
+ 0x8d, 0xf9, 0xee, 0x24, 0x08, 0x95, 0x62, 0xbe,
+ 0x31 }
+}};
+
+static bool __init chacha20_selftest(void)
+{
+ size_t i;
+ u8 offset_input[MAXIMUM_TEST_BUFFER_LEN + 1]
+ __aligned(__alignof__(unsigned long));
+ u8 offset_key[CHACHA20_KEY_SIZE + 1]
+ __aligned(__alignof__(unsigned long));
+ u8 computed_result[MAXIMUM_TEST_BUFFER_LEN + 1]
+ __aligned(__alignof__(unsigned long));
+ struct chacha20_ctx state;
+ bool success = true;
+ simd_context_t simd_context;
+
+ simd_get(&simd_context);
+ for (i = 0; i < ARRAY_SIZE(chacha20_vectors); ++i) {
+ memset(computed_result, 0, sizeof(computed_result));
+ memset(&state, 0, sizeof(state));
+ chacha20_init(&state, chacha20_vectors[i].key,
+ chacha20_vectors[i].nonce);
+ chacha20(&state, computed_result, chacha20_vectors[i].input,
+ chacha20_vectors[i].ilen, &simd_context);
+ if (memcmp(computed_result, chacha20_vectors[i].result,
+ chacha20_vectors[i].ilen)) {
+ pr_info("chacha20 self-test %zu: FAIL\n", i + 1);
+ success = false;
+ }
+ memset(computed_result, 0, sizeof(computed_result));
+ memset(&state, 0, sizeof(state));
+ memcpy(offset_input + 1, chacha20_vectors[i].input,
+ chacha20_vectors[i].ilen);
+ memcpy(offset_key + 1, chacha20_vectors[i].key,
+ CHACHA20_KEY_SIZE);
+ chacha20_init(&state, offset_key + 1, chacha20_vectors[i].nonce);
+ chacha20(&state, computed_result + 1, offset_input + 1,
+ chacha20_vectors[i].ilen, &simd_context);
+ if (memcmp(computed_result + 1, chacha20_vectors[i].result,
+ chacha20_vectors[i].ilen)) {
+ pr_info("chacha20 self-test %zu (unaligned): FAIL\n", i + 1);
+ success = false;
+ }
+ }
+ simd_put(&simd_context);
+ if (success)
+ pr_info("chacha20 self-tests: pass\n");
+ return success;
+}
+#endif
--
2.19.0
^ permalink raw reply related
* [PATCH net-next v5 06/20] zinc: ChaCha20 MIPS32r2 implementation
From: Jason A. Donenfeld @ 2018-09-18 16:16 UTC (permalink / raw)
To: linux-kernel, netdev, linux-crypto, davem, gregkh
Cc: Jason A. Donenfeld, René van Dorst, Samuel Neves,
Andy Lutomirski, Jean-Philippe Aumasson, Ralf Baechle,
Paul Burton, James Hogan, linux-mips
In-Reply-To: <20180918161646.19105-1-Jason@zx2c4.com>
This MIPS32r2 implementation comes from René van Dorst and me and
results in a nice speedup on the usual OpenWRT targets.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: René van Dorst <opensource@vdorst.com>
Cc: Samuel Neves <sneves@dei.uc.pt>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Cc: James Hogan <jhogan@kernel.org>
Cc: linux-mips@linux-mips.org
---
lib/zinc/Makefile | 1 +
lib/zinc/chacha20/chacha20-mips-glue.h | 24 ++
lib/zinc/chacha20/chacha20-mips.S | 474 +++++++++++++++++++++++++
lib/zinc/chacha20/chacha20.c | 2 +
4 files changed, 501 insertions(+)
create mode 100644 lib/zinc/chacha20/chacha20-mips-glue.h
create mode 100644 lib/zinc/chacha20/chacha20-mips.S
diff --git a/lib/zinc/Makefile b/lib/zinc/Makefile
index 36b61145666d..5c7b452d6a19 100644
--- a/lib/zinc/Makefile
+++ b/lib/zinc/Makefile
@@ -7,4 +7,5 @@ zinc_chacha20-y := chacha20/chacha20.o
zinc_chacha20-$(CONFIG_ZINC_ARCH_X86_64) += chacha20/chacha20-x86_64.o
zinc_chacha20-$(CONFIG_ZINC_ARCH_ARM) += chacha20/chacha20-arm.o
zinc_chacha20-$(CONFIG_ZINC_ARCH_ARM64) += chacha20/chacha20-arm64.o
+zinc_chacha20-$(CONFIG_ZINC_ARCH_MIPS) += chacha20/chacha20-mips.o
obj-$(CONFIG_ZINC_CHACHA20) += zinc_chacha20.o
diff --git a/lib/zinc/chacha20/chacha20-mips-glue.h b/lib/zinc/chacha20/chacha20-mips-glue.h
new file mode 100644
index 000000000000..518deb5e6c34
--- /dev/null
+++ b/lib/zinc/chacha20/chacha20-mips-glue.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ */
+
+asmlinkage void chacha20_mips(u8 *out, const u8 *in, const size_t len,
+ const u32 key[8], const u32 counter[4]);
+static void __init chacha20_fpu_init(void)
+{
+}
+
+static inline bool chacha20_arch(u8 *dst, const u8 *src, const size_t len,
+ const u32 key[8], const u32 counter[4],
+ simd_context_t *simd_context)
+{
+ chacha20_mips(dst, src, len, key, counter);
+ return true;
+}
+
+static inline bool hchacha20_arch(u8 *derived_key, const u8 *nonce,
+ const u8 *key, simd_context_t *simd_context)
+{
+ return false;
+}
diff --git a/lib/zinc/chacha20/chacha20-mips.S b/lib/zinc/chacha20/chacha20-mips.S
new file mode 100644
index 000000000000..1415a77fd0c0
--- /dev/null
+++ b/lib/zinc/chacha20/chacha20-mips.S
@@ -0,0 +1,474 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2016-2018 René van Dorst <opensource@vdorst.com>. All Rights Reserved.
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ */
+
+#define MASK_U32 0x3c
+#define MASK_BYTES 0x03
+#define CHACHA20_BLOCK_SIZE 64
+#define STACK_SIZE 4*16
+
+#define X0 $t0
+#define X1 $t1
+#define X2 $t2
+#define X3 $t3
+#define X4 $t4
+#define X5 $t5
+#define X6 $t6
+#define X7 $t7
+#define X8 $v1
+#define X9 $fp
+#define X10 $s7
+#define X11 $s6
+#define X12 $s5
+#define X13 $s4
+#define X14 $s3
+#define X15 $s2
+/* Use regs which are overwritten on exit for Tx so we don't leak clear data. */
+#define T0 $s1
+#define T1 $s0
+#define T(n) T ## n
+#define X(n) X ## n
+
+/* Input arguments */
+#define OUT $a0
+#define IN $a1
+#define BYTES $a2
+/* KEY and NONCE argument must be u32 aligned */
+#define KEY $a3
+/* NONCE pointer is given via stack */
+#define NONCE $t9
+
+/* Output argument */
+/* NONCE[0] is kept in a register and not in memory.
+ * We don't want to touch original value in memory.
+ * Must be incremented every loop iteration.
+ */
+#define NONCE_0 $v0
+
+/* SAVED_X and SAVED_CA are set in the jump table.
+ * Use regs which are overwritten on exit else we don't leak clear data.
+ * They are used to handling the last bytes which are not multiple of 4.
+ */
+#define SAVED_X X15
+#define SAVED_CA $ra
+
+#define PTR_LAST_ROUND $t8
+
+/* ChaCha20 constants and stack location */
+#define CONSTANT_OFS_SP 48
+#define UNALIGNED_OFS_SP 40
+
+#define CONSTANT_1 0x61707865
+#define CONSTANT_2 0x3320646e
+#define CONSTANT_3 0x79622d32
+#define CONSTANT_4 0x6b206574
+
+#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+#define MSB 0
+#define LSB 3
+#define ROTx rotl
+#define ROTR(n) rotr n, 24
+#define CPU_TO_LE32(n) \
+ wsbh n; \
+ rotr n, 16;
+#else
+#define MSB 3
+#define LSB 0
+#define ROTx rotr
+#define CPU_TO_LE32(n)
+#define ROTR(n)
+#endif
+
+#define STORE_UNALIGNED(x, a, s, o) \
+.Lchacha20_mips_xor_unaligned_ ## x ## _b: ; \
+ .if ((s != NONCE) || (o != 0)); \
+ lw T0, o(s); \
+ .endif; \
+ lwl T1, x-4+MSB ## (IN); \
+ lwr T1, x-4+LSB ## (IN); \
+ .if ((s == NONCE) && (o == 0)); \
+ addu X ## a, NONCE_0; \
+ .else; \
+ addu X ## a, T0; \
+ .endif; \
+ CPU_TO_LE32(X ## a); \
+ xor X ## a, T1; \
+ swl X ## a, x-4+MSB ## (OUT); \
+ swr X ## a, x-4+LSB ## (OUT);
+
+#define STORE_ALIGNED(x, a, s, o) \
+.Lchacha20_mips_xor_aligned_ ## x ## _b: ; \
+ .if ((s != NONCE) || (o != 0)); \
+ lw T0, o(s); \
+ .endif; \
+ lw T1, x-4 ## (IN); \
+ .if ((s == NONCE) && (o == 0)); \
+ addu X ## a, NONCE_0; \
+ .else; \
+ addu X ## a, T0; \
+ .endif; \
+ CPU_TO_LE32(X ## a); \
+ xor X ## a, T1; \
+ sw X ## a, x-4 ## (OUT);
+
+/* Jump table macro.
+ * Used for setup and handling the last bytes, which are not multiple of 4.
+ * X15 is free to store Xn
+ * Every jumptable entry must be equal in size.
+ */
+#define JMPTBL_ALIGNED(x, a, s, o) \
+.Lchacha20_mips_jmptbl_aligned_ ## a: ; \
+ .if ((s == NONCE) && (o == 0)); \
+ move SAVED_CA, NONCE_0; \
+ .else; \
+ lw SAVED_CA, o(s);\
+ .endif; \
+ b .Lchacha20_mips_xor_aligned_ ## x ## _b; \
+ move SAVED_X, X ## a;
+
+#define JMPTBL_UNALIGNED(x, a, s, o) \
+.Lchacha20_mips_jmptbl_unaligned_ ## a: ; \
+ .if ((s == NONCE) && (o == 0)); \
+ move SAVED_CA, NONCE_0; \
+ .else; \
+ lw SAVED_CA, o(s);\
+ .endif; \
+ b .Lchacha20_mips_xor_unaligned_ ## x ## _b; \
+ move SAVED_X, X ## a;
+
+#define AXR(A, B, C, D, K, L, M, N, V, W, Y, Z, S) \
+ addu X(A), X(K); \
+ addu X(B), X(L); \
+ addu X(C), X(M); \
+ addu X(D), X(N); \
+ xor X(V), X(A); \
+ xor X(W), X(B); \
+ xor X(Y), X(C); \
+ xor X(Z), X(D); \
+ rotl X(V), S; \
+ rotl X(W), S; \
+ rotl X(Y), S; \
+ rotl X(Z), S;
+
+.text
+.set reorder
+.set noat
+.globl chacha20_mips
+.ent chacha20_mips
+chacha20_mips:
+ .frame $sp, STACK_SIZE, $ra
+ /* This is in the fifth argument */
+ lw NONCE, 16($sp)
+
+ /* Return bytes = 0. */
+ .set noreorder
+ beqz BYTES, .Lchacha20_mips_end
+ addiu $sp, -STACK_SIZE
+ .set reorder
+
+ /* Calculate PTR_LAST_ROUND */
+ addiu PTR_LAST_ROUND, BYTES, -1
+ ins PTR_LAST_ROUND, $zero, 0, 6
+ addu PTR_LAST_ROUND, OUT
+
+ /* Save s0-s7, fp, ra. */
+ sw $ra, 0($sp)
+ sw $fp, 4($sp)
+ sw $s0, 8($sp)
+ sw $s1, 12($sp)
+ sw $s2, 16($sp)
+ sw $s3, 20($sp)
+ sw $s4, 24($sp)
+ sw $s5, 28($sp)
+ sw $s6, 32($sp)
+ sw $s7, 36($sp)
+
+ lw NONCE_0, 0(NONCE)
+ /* Test IN or OUT is unaligned.
+ * UNALIGNED (T1) = ( IN | OUT ) & 0x00000003
+ */
+ or T1, IN, OUT
+ andi T1, 0x3
+
+ /* Load constant */
+ lui X0, %hi(CONSTANT_1)
+ lui X1, %hi(CONSTANT_2)
+ lui X2, %hi(CONSTANT_3)
+ lui X3, %hi(CONSTANT_4)
+ ori X0, %lo(CONSTANT_1)
+ ori X1, %lo(CONSTANT_2)
+ ori X2, %lo(CONSTANT_3)
+ ori X3, %lo(CONSTANT_4)
+
+ /* Store constant on stack. */
+ sw X0, 0+CONSTANT_OFS_SP($sp)
+ sw X1, 4+CONSTANT_OFS_SP($sp)
+ sw X2, 8+CONSTANT_OFS_SP($sp)
+ sw X3, 12+CONSTANT_OFS_SP($sp)
+
+ sw T1, UNALIGNED_OFS_SP($sp)
+
+ .set noreorder
+ b .Lchacha20_rounds_start
+ andi BYTES, (CHACHA20_BLOCK_SIZE-1)
+ .set reorder
+
+.align 4
+.Loop_chacha20_rounds:
+ addiu IN, CHACHA20_BLOCK_SIZE
+ addiu OUT, CHACHA20_BLOCK_SIZE
+ addiu NONCE_0, 1
+
+ lw X0, 0+CONSTANT_OFS_SP($sp)
+ lw X1, 4+CONSTANT_OFS_SP($sp)
+ lw X2, 8+CONSTANT_OFS_SP($sp)
+ lw X3, 12+CONSTANT_OFS_SP($sp)
+ lw T1, UNALIGNED_OFS_SP($sp)
+
+.Lchacha20_rounds_start:
+ lw X4, 0(KEY)
+ lw X5, 4(KEY)
+ lw X6, 8(KEY)
+ lw X7, 12(KEY)
+ lw X8, 16(KEY)
+ lw X9, 20(KEY)
+ lw X10, 24(KEY)
+ lw X11, 28(KEY)
+
+ move X12, NONCE_0
+ lw X13, 4(NONCE)
+ lw X14, 8(NONCE)
+ lw X15, 12(NONCE)
+
+ li $at, 9
+.Loop_chacha20_xor_rounds:
+ AXR( 0, 1, 2, 3, 4, 5, 6, 7, 12,13,14,15, 16);
+ AXR( 8, 9,10,11, 12,13,14,15, 4, 5, 6, 7, 12);
+ AXR( 0, 1, 2, 3, 4, 5, 6, 7, 12,13,14,15, 8);
+ AXR( 8, 9,10,11, 12,13,14,15, 4, 5, 6, 7, 7);
+ AXR( 0, 1, 2, 3, 5, 6, 7, 4, 15,12,13,14, 16);
+ AXR(10,11, 8, 9, 15,12,13,14, 5, 6, 7, 4, 12);
+ AXR( 0, 1, 2, 3, 5, 6, 7, 4, 15,12,13,14, 8);
+ AXR(10,11, 8, 9, 15,12,13,14, 5, 6, 7, 4, 7);
+ .set noreorder
+ bnez $at, .Loop_chacha20_xor_rounds
+ addiu $at, -1
+
+ /* Unaligned? Jump */
+ bnez T1, .Loop_chacha20_unaligned
+ andi $at, BYTES, MASK_U32
+
+ /* Last round? No jump */
+ bne OUT, PTR_LAST_ROUND, .Lchacha20_mips_xor_aligned_64_b
+ /* Load upper half of jump table addr */
+ lui T0, %hi(.Lchacha20_mips_jmptbl_aligned_0)
+
+ /* Full block? Jump */
+ beqz BYTES, .Lchacha20_mips_xor_aligned_64_b
+ /* Calculate lower half jump table addr and offset */
+ ins T0, $at, 2, 6
+
+ subu T0, $at
+ addiu T0, %lo(.Lchacha20_mips_jmptbl_aligned_0)
+
+ jr T0
+ /* Delay slot */
+ nop
+
+ .set reorder
+
+.Loop_chacha20_unaligned:
+ .set noreorder
+
+ /* Last round? no jump */
+ bne OUT, PTR_LAST_ROUND, .Lchacha20_mips_xor_unaligned_64_b
+ /* Load upper half of jump table addr */
+ lui T0, %hi(.Lchacha20_mips_jmptbl_unaligned_0)
+
+ /* Full block? Jump */
+ beqz BYTES, .Lchacha20_mips_xor_unaligned_64_b
+
+ /* Calculate lower half jump table addr and offset */
+ ins T0, $at, 2, 6
+ subu T0, $at
+ addiu T0, %lo(.Lchacha20_mips_jmptbl_unaligned_0)
+
+ jr T0
+ /* Delay slot */
+ nop
+
+ .set reorder
+
+/* Aligned code path
+ */
+.align 4
+ STORE_ALIGNED(64, 15, NONCE,12)
+ STORE_ALIGNED(60, 14, NONCE, 8)
+ STORE_ALIGNED(56, 13, NONCE, 4)
+ STORE_ALIGNED(52, 12, NONCE, 0)
+ STORE_ALIGNED(48, 11, KEY, 28)
+ STORE_ALIGNED(44, 10, KEY, 24)
+ STORE_ALIGNED(40, 9, KEY, 20)
+ STORE_ALIGNED(36, 8, KEY, 16)
+ STORE_ALIGNED(32, 7, KEY, 12)
+ STORE_ALIGNED(28, 6, KEY, 8)
+ STORE_ALIGNED(24, 5, KEY, 4)
+ STORE_ALIGNED(20, 4, KEY, 0)
+ STORE_ALIGNED(16, 3, $sp, 12+CONSTANT_OFS_SP)
+ STORE_ALIGNED(12, 2, $sp, 8+CONSTANT_OFS_SP)
+ STORE_ALIGNED( 8, 1, $sp, 4+CONSTANT_OFS_SP)
+.Lchacha20_mips_xor_aligned_4_b:
+ /* STORE_ALIGNED( 4, 0, $sp, 0+CONSTANT_OFS_SP) */
+ lw T0, 0+CONSTANT_OFS_SP($sp)
+ lw T1, 0(IN)
+ addu X0, T0
+ CPU_TO_LE32(X0)
+ xor X0, T1
+ .set noreorder
+ bne OUT, PTR_LAST_ROUND, .Loop_chacha20_rounds
+ sw X0, 0(OUT)
+ .set reorder
+
+ .set noreorder
+ bne $at, BYTES, .Lchacha20_mips_xor_bytes
+ /* Empty delayslot, Increase NONCE_0, return NONCE_0 value */
+ addiu NONCE_0, 1
+ .set noreorder
+
+.Lchacha20_mips_xor_done:
+ /* Restore used registers */
+ lw $ra, 0($sp)
+ lw $fp, 4($sp)
+ lw $s0, 8($sp)
+ lw $s1, 12($sp)
+ lw $s2, 16($sp)
+ lw $s3, 20($sp)
+ lw $s4, 24($sp)
+ lw $s5, 28($sp)
+ lw $s6, 32($sp)
+ lw $s7, 36($sp)
+.Lchacha20_mips_end:
+ .set noreorder
+ jr $ra
+ addiu $sp, STACK_SIZE
+ .set reorder
+
+ .set noreorder
+ /* Start jump table */
+ JMPTBL_ALIGNED( 0, 0, $sp, 0+CONSTANT_OFS_SP)
+ JMPTBL_ALIGNED( 4, 1, $sp, 4+CONSTANT_OFS_SP)
+ JMPTBL_ALIGNED( 8, 2, $sp, 8+CONSTANT_OFS_SP)
+ JMPTBL_ALIGNED(12, 3, $sp, 12+CONSTANT_OFS_SP)
+ JMPTBL_ALIGNED(16, 4, KEY, 0)
+ JMPTBL_ALIGNED(20, 5, KEY, 4)
+ JMPTBL_ALIGNED(24, 6, KEY, 8)
+ JMPTBL_ALIGNED(28, 7, KEY, 12)
+ JMPTBL_ALIGNED(32, 8, KEY, 16)
+ JMPTBL_ALIGNED(36, 9, KEY, 20)
+ JMPTBL_ALIGNED(40, 10, KEY, 24)
+ JMPTBL_ALIGNED(44, 11, KEY, 28)
+ JMPTBL_ALIGNED(48, 12, NONCE, 0)
+ JMPTBL_ALIGNED(52, 13, NONCE, 4)
+ JMPTBL_ALIGNED(56, 14, NONCE, 8)
+ JMPTBL_ALIGNED(60, 15, NONCE,12)
+ /* End jump table */
+ .set reorder
+
+/* Unaligned code path
+ */
+ STORE_UNALIGNED(64, 15, NONCE,12)
+ STORE_UNALIGNED(60, 14, NONCE, 8)
+ STORE_UNALIGNED(56, 13, NONCE, 4)
+ STORE_UNALIGNED(52, 12, NONCE, 0)
+ STORE_UNALIGNED(48, 11, KEY, 28)
+ STORE_UNALIGNED(44, 10, KEY, 24)
+ STORE_UNALIGNED(40, 9, KEY, 20)
+ STORE_UNALIGNED(36, 8, KEY, 16)
+ STORE_UNALIGNED(32, 7, KEY, 12)
+ STORE_UNALIGNED(28, 6, KEY, 8)
+ STORE_UNALIGNED(24, 5, KEY, 4)
+ STORE_UNALIGNED(20, 4, KEY, 0)
+ STORE_UNALIGNED(16, 3, $sp, 12+CONSTANT_OFS_SP)
+ STORE_UNALIGNED(12, 2, $sp, 8+CONSTANT_OFS_SP)
+ STORE_UNALIGNED( 8, 1, $sp, 4+CONSTANT_OFS_SP)
+.Lchacha20_mips_xor_unaligned_4_b:
+ /* STORE_UNALIGNED( 4, 0, $sp, 0+CONSTANT_OFS_SP) */
+ lw T0, 0+CONSTANT_OFS_SP($sp)
+ lwl T1, 0+MSB(IN)
+ lwr T1, 0+LSB(IN)
+ addu X0, T0
+ CPU_TO_LE32(X0)
+ xor X0, T1
+ swl X0, 0+MSB(OUT)
+ .set noreorder
+ bne OUT, PTR_LAST_ROUND, .Loop_chacha20_rounds
+ swr X0, 0+LSB(OUT)
+ .set reorder
+
+ /* Fall through to byte handling */
+ .set noreorder
+ beq $at, BYTES, .Lchacha20_mips_xor_done
+ /* Empty delayslot, increase NONCE_0, return NONCE_0 value */
+.Lchacha20_mips_xor_unaligned_0_b:
+.Lchacha20_mips_xor_aligned_0_b:
+ addiu NONCE_0, 1
+ .set reorder
+
+.Lchacha20_mips_xor_bytes:
+ addu OUT, $at
+ addu IN, $at
+ addu SAVED_X, SAVED_CA
+ /* First byte */
+ lbu T1, 0(IN)
+ andi $at, BYTES, 2
+ CPU_TO_LE32(SAVED_X)
+ ROTR(SAVED_X)
+ xor T1, SAVED_X
+ .set noreorder
+ beqz $at, .Lchacha20_mips_xor_done
+ sb T1, 0(OUT)
+ .set reorder
+ /* Second byte */
+ lbu T1, 1(IN)
+ andi $at, BYTES, 1
+ ROTx SAVED_X, 8
+ xor T1, SAVED_X
+ .set noreorder
+ beqz $at, .Lchacha20_mips_xor_done
+ sb T1, 1(OUT)
+ .set reorder
+ /* Third byte */
+ lbu T1, 2(IN)
+ ROTx SAVED_X, 8
+ xor T1, SAVED_X
+ .set noreorder
+ b .Lchacha20_mips_xor_done
+ sb T1, 2(OUT)
+ .set reorder
+.set noreorder
+
+.Lchacha20_mips_jmptbl_unaligned:
+ /* Start jump table */
+ JMPTBL_UNALIGNED( 0, 0, $sp, 0+CONSTANT_OFS_SP)
+ JMPTBL_UNALIGNED( 4, 1, $sp, 4+CONSTANT_OFS_SP)
+ JMPTBL_UNALIGNED( 8, 2, $sp, 8+CONSTANT_OFS_SP)
+ JMPTBL_UNALIGNED(12, 3, $sp, 12+CONSTANT_OFS_SP)
+ JMPTBL_UNALIGNED(16, 4, KEY, 0)
+ JMPTBL_UNALIGNED(20, 5, KEY, 4)
+ JMPTBL_UNALIGNED(24, 6, KEY, 8)
+ JMPTBL_UNALIGNED(28, 7, KEY, 12)
+ JMPTBL_UNALIGNED(32, 8, KEY, 16)
+ JMPTBL_UNALIGNED(36, 9, KEY, 20)
+ JMPTBL_UNALIGNED(40, 10, KEY, 24)
+ JMPTBL_UNALIGNED(44, 11, KEY, 28)
+ JMPTBL_UNALIGNED(48, 12, NONCE, 0)
+ JMPTBL_UNALIGNED(52, 13, NONCE, 4)
+ JMPTBL_UNALIGNED(56, 14, NONCE, 8)
+ JMPTBL_UNALIGNED(60, 15, NONCE,12)
+ /* End jump table */
+.set reorder
+
+.end chacha20_mips
+.set at
diff --git a/lib/zinc/chacha20/chacha20.c b/lib/zinc/chacha20/chacha20.c
index 385e7350adc3..0cf490ca23be 100644
--- a/lib/zinc/chacha20/chacha20.c
+++ b/lib/zinc/chacha20/chacha20.c
@@ -18,6 +18,8 @@
#include "chacha20-x86_64-glue.h"
#elif defined(CONFIG_ZINC_ARCH_ARM) || defined(CONFIG_ZINC_ARCH_ARM64)
#include "chacha20-arm-glue.h"
+#elif defined(CONFIG_ZINC_ARCH_MIPS)
+#include "chacha20-mips-glue.h"
#else
void __init chacha20_fpu_init(void)
{
--
2.19.0
^ permalink raw reply related
* Re: pegged softirq and NAPI race (?)
From: Song Liu @ 2018-09-18 16:19 UTC (permalink / raw)
To: Eric Dumazet
Cc: netdev, Jeff Kirsher, Alexander Duyck, michael.chan@broadcom.com,
Kernel Team
In-Reply-To: <CANn89iLAPvPn2C9J+sSHsQ5EeF0hq47iMiiAfwoQkdZurQiyzA@mail.gmail.com>
> On Sep 18, 2018, at 6:45 AM, Eric Dumazet <edumazet@google.com> wrote:
>
> On Tue, Sep 18, 2018 at 1:41 AM Song Liu <songliubraving@fb.com> wrote:
>>
>> We are debugging this issue that netconsole message triggers pegged softirq
>> (ksoftirqd taking 100% CPU for many seconds). We found this issue in
>> production with both bnxt and ixgbe, on a 4.11 based kernel. This is easily
>> reproducible with ixgbe on 4.11, and latest net/net-next (see [1] for more
>> detail).
>>
>> After debugging for some time, we found that this issue is likely related
>> to 39e6c8208d7b ("net: solve a NAPI race"). After reverting this commit,
>> the steps described in [1] cannot reproduce the issue on ixgbe. Reverting
>> this commit also reduces the chances we hit the issue with bnxt (it still
>> happens with a lower rate).
>>
>> I tried to fix this issue with relaxed variant (or older version) of
>> napi_schedule_prep() in netpoll, just like the one on napi_watchdog().
>> However, my tests do not always go as expected.
>>
>> Please share your comments/suggestions on which direction shall we try
>> to fix this.
>>
>> Thanks in advance!
>> Song
>>
>>
>> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__www.spinics.net_lists_netdev_msg522328.html&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=iSaOapj1kxjhGYLgQr0Qd8mQCzVdobmgT1L4JwFvzxs&s=lCEhrz6wQJUUaJOkxFmtOszAgkf3Jh4reX_i1GbI5RI&e=
>
> You have not traced ixgbe to understand why driver hits
> "clean_complete=false" all the time ?
The trace showed that we got "clean_complete=false" because
ixgbe_clean_rx_irq() used all budget (64). It feels like the driver
is tricked to process old data on the rx_ring for one more time.
Have you seen similar issue?
Thanks,
Song
^ permalink raw reply
* [PATCH net-next v5 13/20] zinc: BLAKE2s x86_64 implementation
From: Jason A. Donenfeld @ 2018-09-18 16:16 UTC (permalink / raw)
To: linux-kernel, netdev, linux-crypto, davem, gregkh
Cc: Jason A. Donenfeld, Samuel Neves, Andy Lutomirski,
Jean-Philippe Aumasson, Thomas Gleixner, Ingo Molnar, x86
In-Reply-To: <20180918161646.19105-1-Jason@zx2c4.com>
These implementations from Samuel Neves support AVX and AVX-512VL.
Originally this used AVX-512F, but Skylake thermal throttling made
AVX-512VL more attractive and possible to do with negligable difference.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Samuel Neves <sneves@dei.uc.pt>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: x86@kernel.org
---
lib/zinc/Makefile | 1 +
lib/zinc/blake2s/blake2s-x86_64-glue.h | 59 +++
lib/zinc/blake2s/blake2s-x86_64.S | 685 +++++++++++++++++++++++++
lib/zinc/blake2s/blake2s.c | 4 +-
4 files changed, 748 insertions(+), 1 deletion(-)
create mode 100644 lib/zinc/blake2s/blake2s-x86_64-glue.h
create mode 100644 lib/zinc/blake2s/blake2s-x86_64.S
diff --git a/lib/zinc/Makefile b/lib/zinc/Makefile
index 0e0020f58d27..47607486c39d 100644
--- a/lib/zinc/Makefile
+++ b/lib/zinc/Makefile
@@ -22,4 +22,5 @@ zinc_chacha20poly1305-y := chacha20poly1305.o
obj-$(CONFIG_ZINC_CHACHA20POLY1305) += zinc_chacha20poly1305.o
zinc_blake2s-y := blake2s/blake2s.o
+zinc_blake2s-$(CONFIG_ZINC_ARCH_X86_64) += blake2s/blake2s-x86_64.o
obj-$(CONFIG_ZINC_BLAKE2S) += zinc_blake2s.o
diff --git a/lib/zinc/blake2s/blake2s-x86_64-glue.h b/lib/zinc/blake2s/blake2s-x86_64-glue.h
new file mode 100644
index 000000000000..22900ef8e7fe
--- /dev/null
+++ b/lib/zinc/blake2s/blake2s-x86_64-glue.h
@@ -0,0 +1,59 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ */
+
+#include <asm/cpufeature.h>
+#include <asm/processor.h>
+#include <asm/fpu/api.h>
+#include <asm/simd.h>
+
+#ifdef CONFIG_AS_AVX
+asmlinkage void blake2s_compress_avx(struct blake2s_state *state,
+ const u8 *block, const size_t nblocks,
+ const u32 inc);
+#endif
+#ifdef CONFIG_AS_AVX512
+asmlinkage void blake2s_compress_avx512(struct blake2s_state *state,
+ const u8 *block, const size_t nblocks,
+ const u32 inc);
+#endif
+
+static bool blake2s_use_avx __ro_after_init;
+static bool blake2s_use_avx512 __ro_after_init;
+
+static void __init blake2s_fpu_init(void)
+{
+ blake2s_use_avx =
+ boot_cpu_has(X86_FEATURE_AVX) &&
+ cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL);
+ blake2s_use_avx512 =
+ boot_cpu_has(X86_FEATURE_AVX) &&
+ boot_cpu_has(X86_FEATURE_AVX2) &&
+ boot_cpu_has(X86_FEATURE_AVX512F) &&
+ boot_cpu_has(X86_FEATURE_AVX512VL) &&
+ cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM |
+ XFEATURE_MASK_AVX512, NULL);
+}
+
+static inline bool blake2s_arch(struct blake2s_state *state, const u8 *block,
+ size_t nblocks, const u32 inc)
+{
+#ifdef CONFIG_AS_AVX512
+ if (blake2s_use_avx512 && irq_fpu_usable()) {
+ kernel_fpu_begin();
+ blake2s_compress_avx512(state, block, nblocks, inc);
+ kernel_fpu_end();
+ return true;
+ }
+#endif
+#ifdef CONFIG_AS_AVX
+ if (blake2s_use_avx && irq_fpu_usable()) {
+ kernel_fpu_begin();
+ blake2s_compress_avx(state, block, nblocks, inc);
+ kernel_fpu_end();
+ return true;
+ }
+#endif
+ return false;
+}
diff --git a/lib/zinc/blake2s/blake2s-x86_64.S b/lib/zinc/blake2s/blake2s-x86_64.S
new file mode 100644
index 000000000000..360be4818d06
--- /dev/null
+++ b/lib/zinc/blake2s/blake2s-x86_64.S
@@ -0,0 +1,685 @@
+/* SPDX-License-Identifier: MIT
+ *
+ * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ * Copyright (C) 2017 Samuel Neves <sneves@dei.uc.pt>. All Rights Reserved.
+ */
+
+#include <linux/linkage.h>
+
+.section .rodata.cst32.BLAKE2S_IV, "aM", @progbits, 32
+.align 32
+IV: .octa 0xA54FF53A3C6EF372BB67AE856A09E667
+ .octa 0x5BE0CD191F83D9AB9B05688C510E527F
+.section .rodata.cst16.ROT16, "aM", @progbits, 16
+.align 16
+ROT16: .octa 0x0D0C0F0E09080B0A0504070601000302
+.section .rodata.cst16.ROR328, "aM", @progbits, 16
+.align 16
+ROR328: .octa 0x0C0F0E0D080B0A090407060500030201
+#ifdef CONFIG_AS_AVX512
+.section .rodata.cst64.BLAKE2S_SIGMA, "aM", @progbits, 640
+.align 64
+SIGMA:
+.long 0, 2, 4, 6, 1, 3, 5, 7, 8, 10, 12, 14, 9, 11, 13, 15
+.long 11, 2, 12, 14, 9, 8, 15, 3, 4, 0, 13, 6, 10, 1, 7, 5
+.long 10, 12, 11, 6, 5, 9, 13, 3, 4, 15, 14, 2, 0, 7, 8, 1
+.long 10, 9, 7, 0, 11, 14, 1, 12, 6, 2, 15, 3, 13, 8, 5, 4
+.long 4, 9, 8, 13, 14, 0, 10, 11, 7, 3, 12, 1, 5, 6, 15, 2
+.long 2, 10, 4, 14, 13, 3, 9, 11, 6, 5, 7, 12, 15, 1, 8, 0
+.long 4, 11, 14, 8, 13, 10, 12, 5, 2, 1, 15, 3, 9, 7, 0, 6
+.long 6, 12, 0, 13, 15, 2, 1, 10, 4, 5, 11, 14, 8, 3, 9, 7
+.long 14, 5, 4, 12, 9, 7, 3, 10, 2, 0, 6, 15, 11, 1, 13, 8
+.long 11, 7, 13, 10, 12, 14, 0, 15, 4, 5, 6, 9, 2, 1, 8, 3
+#endif /* CONFIG_AS_AVX512 */
+
+.text
+#ifdef CONFIG_AS_AVX
+ENTRY(blake2s_compress_avx)
+ movl %ecx, %ecx
+ testq %rdx, %rdx
+ je .Lendofloop
+ .align 32
+.Lbeginofloop:
+ addq %rcx, 32(%rdi)
+ vmovdqu IV+16(%rip), %xmm1
+ vmovdqu (%rsi), %xmm4
+ vpxor 32(%rdi), %xmm1, %xmm1
+ vmovdqu 16(%rsi), %xmm3
+ vshufps $136, %xmm3, %xmm4, %xmm6
+ vmovdqa ROT16(%rip), %xmm7
+ vpaddd (%rdi), %xmm6, %xmm6
+ vpaddd 16(%rdi), %xmm6, %xmm6
+ vpxor %xmm6, %xmm1, %xmm1
+ vmovdqu IV(%rip), %xmm8
+ vpshufb %xmm7, %xmm1, %xmm1
+ vmovdqu 48(%rsi), %xmm5
+ vpaddd %xmm1, %xmm8, %xmm8
+ vpxor 16(%rdi), %xmm8, %xmm9
+ vmovdqu 32(%rsi), %xmm2
+ vpblendw $12, %xmm3, %xmm5, %xmm13
+ vshufps $221, %xmm5, %xmm2, %xmm12
+ vpunpckhqdq %xmm2, %xmm4, %xmm14
+ vpslld $20, %xmm9, %xmm0
+ vpsrld $12, %xmm9, %xmm9
+ vpxor %xmm0, %xmm9, %xmm0
+ vshufps $221, %xmm3, %xmm4, %xmm9
+ vpaddd %xmm9, %xmm6, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vmovdqa ROR328(%rip), %xmm6
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm8, %xmm8
+ vpxor %xmm8, %xmm0, %xmm0
+ vpshufd $147, %xmm1, %xmm1
+ vpshufd $78, %xmm8, %xmm8
+ vpslld $25, %xmm0, %xmm10
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm10, %xmm0, %xmm0
+ vshufps $136, %xmm5, %xmm2, %xmm10
+ vpshufd $57, %xmm0, %xmm0
+ vpaddd %xmm10, %xmm9, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpaddd %xmm12, %xmm9, %xmm9
+ vpblendw $12, %xmm2, %xmm3, %xmm12
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm8, %xmm8
+ vpxor %xmm8, %xmm0, %xmm10
+ vpslld $20, %xmm10, %xmm0
+ vpsrld $12, %xmm10, %xmm10
+ vpxor %xmm0, %xmm10, %xmm0
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm8, %xmm8
+ vpxor %xmm8, %xmm0, %xmm0
+ vpshufd $57, %xmm1, %xmm1
+ vpshufd $78, %xmm8, %xmm8
+ vpslld $25, %xmm0, %xmm10
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm10, %xmm0, %xmm0
+ vpslldq $4, %xmm5, %xmm10
+ vpblendw $240, %xmm10, %xmm12, %xmm12
+ vpshufd $147, %xmm0, %xmm0
+ vpshufd $147, %xmm12, %xmm12
+ vpaddd %xmm9, %xmm12, %xmm12
+ vpaddd %xmm0, %xmm12, %xmm12
+ vpxor %xmm12, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm8, %xmm8
+ vpxor %xmm8, %xmm0, %xmm11
+ vpslld $20, %xmm11, %xmm9
+ vpsrld $12, %xmm11, %xmm11
+ vpxor %xmm9, %xmm11, %xmm0
+ vpshufd $8, %xmm2, %xmm9
+ vpblendw $192, %xmm5, %xmm3, %xmm11
+ vpblendw $240, %xmm11, %xmm9, %xmm9
+ vpshufd $177, %xmm9, %xmm9
+ vpaddd %xmm12, %xmm9, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm11
+ vpxor %xmm11, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm8, %xmm8
+ vpxor %xmm8, %xmm0, %xmm9
+ vpshufd $147, %xmm1, %xmm1
+ vpshufd $78, %xmm8, %xmm8
+ vpslld $25, %xmm9, %xmm0
+ vpsrld $7, %xmm9, %xmm9
+ vpxor %xmm0, %xmm9, %xmm0
+ vpslldq $4, %xmm3, %xmm9
+ vpblendw $48, %xmm9, %xmm2, %xmm9
+ vpblendw $240, %xmm9, %xmm4, %xmm9
+ vpshufd $57, %xmm0, %xmm0
+ vpshufd $177, %xmm9, %xmm9
+ vpaddd %xmm11, %xmm9, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm8, %xmm11
+ vpxor %xmm11, %xmm0, %xmm0
+ vpslld $20, %xmm0, %xmm8
+ vpsrld $12, %xmm0, %xmm0
+ vpxor %xmm8, %xmm0, %xmm0
+ vpunpckhdq %xmm3, %xmm4, %xmm8
+ vpblendw $12, %xmm10, %xmm8, %xmm12
+ vpshufd $177, %xmm12, %xmm12
+ vpaddd %xmm9, %xmm12, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm11
+ vpxor %xmm11, %xmm0, %xmm0
+ vpshufd $57, %xmm1, %xmm1
+ vpshufd $78, %xmm11, %xmm11
+ vpslld $25, %xmm0, %xmm12
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm12, %xmm0, %xmm0
+ vpunpckhdq %xmm5, %xmm2, %xmm12
+ vpshufd $147, %xmm0, %xmm0
+ vpblendw $15, %xmm13, %xmm12, %xmm12
+ vpslldq $8, %xmm5, %xmm13
+ vpshufd $210, %xmm12, %xmm12
+ vpaddd %xmm9, %xmm12, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm11
+ vpxor %xmm11, %xmm0, %xmm0
+ vpslld $20, %xmm0, %xmm12
+ vpsrld $12, %xmm0, %xmm0
+ vpxor %xmm12, %xmm0, %xmm0
+ vpunpckldq %xmm4, %xmm2, %xmm12
+ vpblendw $240, %xmm4, %xmm12, %xmm12
+ vpblendw $192, %xmm13, %xmm12, %xmm12
+ vpsrldq $12, %xmm3, %xmm13
+ vpaddd %xmm12, %xmm9, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm11
+ vpxor %xmm11, %xmm0, %xmm0
+ vpshufd $147, %xmm1, %xmm1
+ vpshufd $78, %xmm11, %xmm11
+ vpslld $25, %xmm0, %xmm12
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm12, %xmm0, %xmm0
+ vpblendw $60, %xmm2, %xmm4, %xmm12
+ vpblendw $3, %xmm13, %xmm12, %xmm12
+ vpshufd $57, %xmm0, %xmm0
+ vpshufd $78, %xmm12, %xmm12
+ vpaddd %xmm9, %xmm12, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm11
+ vpxor %xmm11, %xmm0, %xmm12
+ vpslld $20, %xmm12, %xmm13
+ vpsrld $12, %xmm12, %xmm0
+ vpblendw $51, %xmm3, %xmm4, %xmm12
+ vpxor %xmm13, %xmm0, %xmm0
+ vpblendw $192, %xmm10, %xmm12, %xmm10
+ vpslldq $8, %xmm2, %xmm12
+ vpshufd $27, %xmm10, %xmm10
+ vpaddd %xmm9, %xmm10, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm11
+ vpxor %xmm11, %xmm0, %xmm0
+ vpshufd $57, %xmm1, %xmm1
+ vpshufd $78, %xmm11, %xmm11
+ vpslld $25, %xmm0, %xmm10
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm10, %xmm0, %xmm0
+ vpunpckhdq %xmm2, %xmm8, %xmm10
+ vpshufd $147, %xmm0, %xmm0
+ vpblendw $12, %xmm5, %xmm10, %xmm10
+ vpshufd $210, %xmm10, %xmm10
+ vpaddd %xmm9, %xmm10, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm11
+ vpxor %xmm11, %xmm0, %xmm10
+ vpslld $20, %xmm10, %xmm0
+ vpsrld $12, %xmm10, %xmm10
+ vpxor %xmm0, %xmm10, %xmm0
+ vpblendw $12, %xmm4, %xmm5, %xmm10
+ vpblendw $192, %xmm12, %xmm10, %xmm10
+ vpunpckldq %xmm2, %xmm4, %xmm12
+ vpshufd $135, %xmm10, %xmm10
+ vpaddd %xmm9, %xmm10, %xmm9
+ vpaddd %xmm0, %xmm9, %xmm9
+ vpxor %xmm9, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm11, %xmm13
+ vpxor %xmm13, %xmm0, %xmm0
+ vpshufd $147, %xmm1, %xmm1
+ vpshufd $78, %xmm13, %xmm13
+ vpslld $25, %xmm0, %xmm10
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm10, %xmm0, %xmm0
+ vpblendw $15, %xmm3, %xmm4, %xmm10
+ vpblendw $192, %xmm5, %xmm10, %xmm10
+ vpshufd $57, %xmm0, %xmm0
+ vpshufd $198, %xmm10, %xmm10
+ vpaddd %xmm9, %xmm10, %xmm10
+ vpaddd %xmm0, %xmm10, %xmm10
+ vpxor %xmm10, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm13, %xmm13
+ vpxor %xmm13, %xmm0, %xmm9
+ vpslld $20, %xmm9, %xmm0
+ vpsrld $12, %xmm9, %xmm9
+ vpxor %xmm0, %xmm9, %xmm0
+ vpunpckhdq %xmm2, %xmm3, %xmm9
+ vpunpcklqdq %xmm12, %xmm9, %xmm15
+ vpunpcklqdq %xmm12, %xmm8, %xmm12
+ vpblendw $15, %xmm5, %xmm8, %xmm8
+ vpaddd %xmm15, %xmm10, %xmm15
+ vpaddd %xmm0, %xmm15, %xmm15
+ vpxor %xmm15, %xmm1, %xmm1
+ vpshufd $141, %xmm8, %xmm8
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm13, %xmm13
+ vpxor %xmm13, %xmm0, %xmm0
+ vpshufd $57, %xmm1, %xmm1
+ vpshufd $78, %xmm13, %xmm13
+ vpslld $25, %xmm0, %xmm10
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm10, %xmm0, %xmm0
+ vpunpcklqdq %xmm2, %xmm3, %xmm10
+ vpshufd $147, %xmm0, %xmm0
+ vpblendw $51, %xmm14, %xmm10, %xmm14
+ vpshufd $135, %xmm14, %xmm14
+ vpaddd %xmm15, %xmm14, %xmm14
+ vpaddd %xmm0, %xmm14, %xmm14
+ vpxor %xmm14, %xmm1, %xmm1
+ vpunpcklqdq %xmm3, %xmm4, %xmm15
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm13, %xmm13
+ vpxor %xmm13, %xmm0, %xmm0
+ vpslld $20, %xmm0, %xmm11
+ vpsrld $12, %xmm0, %xmm0
+ vpxor %xmm11, %xmm0, %xmm0
+ vpunpckhqdq %xmm5, %xmm3, %xmm11
+ vpblendw $51, %xmm15, %xmm11, %xmm11
+ vpunpckhqdq %xmm3, %xmm5, %xmm15
+ vpaddd %xmm11, %xmm14, %xmm11
+ vpaddd %xmm0, %xmm11, %xmm11
+ vpxor %xmm11, %xmm1, %xmm1
+ vpshufb %xmm6, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm13, %xmm13
+ vpxor %xmm13, %xmm0, %xmm0
+ vpshufd $147, %xmm1, %xmm1
+ vpshufd $78, %xmm13, %xmm13
+ vpslld $25, %xmm0, %xmm14
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm14, %xmm0, %xmm14
+ vpunpckhqdq %xmm4, %xmm2, %xmm0
+ vpshufd $57, %xmm14, %xmm14
+ vpblendw $51, %xmm15, %xmm0, %xmm15
+ vpaddd %xmm15, %xmm11, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm1, %xmm1
+ vpshufb %xmm7, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm13, %xmm13
+ vpxor %xmm13, %xmm14, %xmm14
+ vpslld $20, %xmm14, %xmm11
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm11, %xmm14, %xmm14
+ vpblendw $3, %xmm2, %xmm4, %xmm11
+ vpslldq $8, %xmm11, %xmm0
+ vpblendw $15, %xmm5, %xmm0, %xmm0
+ vpshufd $99, %xmm0, %xmm0
+ vpaddd %xmm15, %xmm0, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm1, %xmm0
+ vpaddd %xmm12, %xmm15, %xmm15
+ vpshufb %xmm6, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm13, %xmm13
+ vpxor %xmm13, %xmm14, %xmm14
+ vpshufd $57, %xmm0, %xmm0
+ vpshufd $78, %xmm13, %xmm13
+ vpslld $25, %xmm14, %xmm1
+ vpsrld $7, %xmm14, %xmm14
+ vpxor %xmm1, %xmm14, %xmm14
+ vpblendw $3, %xmm5, %xmm4, %xmm1
+ vpshufd $147, %xmm14, %xmm14
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpshufb %xmm7, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm13, %xmm13
+ vpxor %xmm13, %xmm14, %xmm14
+ vpslld $20, %xmm14, %xmm12
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm12, %xmm14, %xmm14
+ vpsrldq $4, %xmm2, %xmm12
+ vpblendw $60, %xmm12, %xmm1, %xmm1
+ vpaddd %xmm1, %xmm15, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpblendw $12, %xmm4, %xmm3, %xmm1
+ vpshufb %xmm6, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm13, %xmm13
+ vpxor %xmm13, %xmm14, %xmm14
+ vpshufd $147, %xmm0, %xmm0
+ vpshufd $78, %xmm13, %xmm13
+ vpslld $25, %xmm14, %xmm12
+ vpsrld $7, %xmm14, %xmm14
+ vpxor %xmm12, %xmm14, %xmm14
+ vpsrldq $4, %xmm5, %xmm12
+ vpblendw $48, %xmm12, %xmm1, %xmm1
+ vpshufd $33, %xmm5, %xmm12
+ vpshufd $57, %xmm14, %xmm14
+ vpshufd $108, %xmm1, %xmm1
+ vpblendw $51, %xmm12, %xmm10, %xmm12
+ vpaddd %xmm15, %xmm1, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpaddd %xmm12, %xmm15, %xmm15
+ vpshufb %xmm7, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm13, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpslld $20, %xmm14, %xmm13
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm13, %xmm14, %xmm14
+ vpslldq $12, %xmm3, %xmm13
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpshufb %xmm6, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpshufd $57, %xmm0, %xmm0
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm14, %xmm12
+ vpsrld $7, %xmm14, %xmm14
+ vpxor %xmm12, %xmm14, %xmm14
+ vpblendw $51, %xmm5, %xmm4, %xmm12
+ vpshufd $147, %xmm14, %xmm14
+ vpblendw $192, %xmm13, %xmm12, %xmm12
+ vpaddd %xmm12, %xmm15, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpsrldq $4, %xmm3, %xmm12
+ vpshufb %xmm7, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpslld $20, %xmm14, %xmm13
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm13, %xmm14, %xmm14
+ vpblendw $48, %xmm2, %xmm5, %xmm13
+ vpblendw $3, %xmm12, %xmm13, %xmm13
+ vpshufd $156, %xmm13, %xmm13
+ vpaddd %xmm15, %xmm13, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpshufb %xmm6, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpshufd $147, %xmm0, %xmm0
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm14, %xmm13
+ vpsrld $7, %xmm14, %xmm14
+ vpxor %xmm13, %xmm14, %xmm14
+ vpunpcklqdq %xmm2, %xmm4, %xmm13
+ vpshufd $57, %xmm14, %xmm14
+ vpblendw $12, %xmm12, %xmm13, %xmm12
+ vpshufd $180, %xmm12, %xmm12
+ vpaddd %xmm15, %xmm12, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpshufb %xmm7, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpslld $20, %xmm14, %xmm12
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm12, %xmm14, %xmm14
+ vpunpckhqdq %xmm9, %xmm4, %xmm12
+ vpshufd $198, %xmm12, %xmm12
+ vpaddd %xmm15, %xmm12, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpaddd %xmm15, %xmm8, %xmm15
+ vpshufb %xmm6, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpshufd $57, %xmm0, %xmm0
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm14, %xmm12
+ vpsrld $7, %xmm14, %xmm14
+ vpxor %xmm12, %xmm14, %xmm14
+ vpsrldq $4, %xmm4, %xmm12
+ vpshufd $147, %xmm14, %xmm14
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm15, %xmm0, %xmm0
+ vpshufb %xmm7, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpslld $20, %xmm14, %xmm8
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm14, %xmm8, %xmm14
+ vpblendw $48, %xmm5, %xmm2, %xmm8
+ vpblendw $3, %xmm12, %xmm8, %xmm8
+ vpunpckhqdq %xmm5, %xmm4, %xmm12
+ vpshufd $75, %xmm8, %xmm8
+ vpblendw $60, %xmm10, %xmm12, %xmm10
+ vpaddd %xmm15, %xmm8, %xmm15
+ vpaddd %xmm14, %xmm15, %xmm15
+ vpxor %xmm0, %xmm15, %xmm0
+ vpshufd $45, %xmm10, %xmm10
+ vpshufb %xmm6, %xmm0, %xmm0
+ vpaddd %xmm15, %xmm10, %xmm15
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm1, %xmm14, %xmm14
+ vpshufd $147, %xmm0, %xmm0
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm14, %xmm8
+ vpsrld $7, %xmm14, %xmm14
+ vpxor %xmm14, %xmm8, %xmm8
+ vpshufd $57, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm15, %xmm15
+ vpxor %xmm0, %xmm15, %xmm0
+ vpshufb %xmm7, %xmm0, %xmm0
+ vpaddd %xmm0, %xmm1, %xmm1
+ vpxor %xmm8, %xmm1, %xmm8
+ vpslld $20, %xmm8, %xmm10
+ vpsrld $12, %xmm8, %xmm8
+ vpxor %xmm8, %xmm10, %xmm10
+ vpunpckldq %xmm3, %xmm4, %xmm8
+ vpunpcklqdq %xmm9, %xmm8, %xmm9
+ vpaddd %xmm9, %xmm15, %xmm9
+ vpaddd %xmm10, %xmm9, %xmm9
+ vpxor %xmm0, %xmm9, %xmm8
+ vpshufb %xmm6, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm1, %xmm1
+ vpxor %xmm1, %xmm10, %xmm10
+ vpshufd $57, %xmm8, %xmm8
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm10, %xmm12
+ vpsrld $7, %xmm10, %xmm10
+ vpxor %xmm10, %xmm12, %xmm10
+ vpblendw $48, %xmm4, %xmm3, %xmm12
+ vpshufd $147, %xmm10, %xmm0
+ vpunpckhdq %xmm5, %xmm3, %xmm10
+ vpshufd $78, %xmm12, %xmm12
+ vpunpcklqdq %xmm4, %xmm10, %xmm10
+ vpblendw $192, %xmm2, %xmm10, %xmm10
+ vpshufhw $78, %xmm10, %xmm10
+ vpaddd %xmm10, %xmm9, %xmm10
+ vpaddd %xmm0, %xmm10, %xmm10
+ vpxor %xmm8, %xmm10, %xmm8
+ vpshufb %xmm7, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm1, %xmm1
+ vpxor %xmm0, %xmm1, %xmm9
+ vpslld $20, %xmm9, %xmm0
+ vpsrld $12, %xmm9, %xmm9
+ vpxor %xmm9, %xmm0, %xmm0
+ vpunpckhdq %xmm5, %xmm4, %xmm9
+ vpblendw $240, %xmm9, %xmm2, %xmm13
+ vpshufd $39, %xmm13, %xmm13
+ vpaddd %xmm10, %xmm13, %xmm10
+ vpaddd %xmm0, %xmm10, %xmm10
+ vpxor %xmm8, %xmm10, %xmm8
+ vpblendw $12, %xmm4, %xmm2, %xmm13
+ vpshufb %xmm6, %xmm8, %xmm8
+ vpslldq $4, %xmm13, %xmm13
+ vpblendw $15, %xmm5, %xmm13, %xmm13
+ vpaddd %xmm8, %xmm1, %xmm1
+ vpxor %xmm1, %xmm0, %xmm0
+ vpaddd %xmm13, %xmm10, %xmm13
+ vpshufd $147, %xmm8, %xmm8
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm0, %xmm14
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm0, %xmm14, %xmm14
+ vpshufd $57, %xmm14, %xmm14
+ vpaddd %xmm14, %xmm13, %xmm13
+ vpxor %xmm8, %xmm13, %xmm8
+ vpaddd %xmm13, %xmm12, %xmm12
+ vpshufb %xmm7, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm1, %xmm1
+ vpxor %xmm14, %xmm1, %xmm14
+ vpslld $20, %xmm14, %xmm10
+ vpsrld $12, %xmm14, %xmm14
+ vpxor %xmm14, %xmm10, %xmm10
+ vpaddd %xmm10, %xmm12, %xmm12
+ vpxor %xmm8, %xmm12, %xmm8
+ vpshufb %xmm6, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm1, %xmm1
+ vpxor %xmm1, %xmm10, %xmm0
+ vpshufd $57, %xmm8, %xmm8
+ vpshufd $78, %xmm1, %xmm1
+ vpslld $25, %xmm0, %xmm10
+ vpsrld $7, %xmm0, %xmm0
+ vpxor %xmm0, %xmm10, %xmm10
+ vpblendw $48, %xmm2, %xmm3, %xmm0
+ vpblendw $15, %xmm11, %xmm0, %xmm0
+ vpshufd $147, %xmm10, %xmm10
+ vpshufd $114, %xmm0, %xmm0
+ vpaddd %xmm12, %xmm0, %xmm0
+ vpaddd %xmm10, %xmm0, %xmm0
+ vpxor %xmm8, %xmm0, %xmm8
+ vpshufb %xmm7, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm1, %xmm1
+ vpxor %xmm10, %xmm1, %xmm10
+ vpslld $20, %xmm10, %xmm11
+ vpsrld $12, %xmm10, %xmm10
+ vpxor %xmm10, %xmm11, %xmm10
+ vpslldq $4, %xmm4, %xmm11
+ vpblendw $192, %xmm11, %xmm3, %xmm3
+ vpunpckldq %xmm5, %xmm4, %xmm4
+ vpshufd $99, %xmm3, %xmm3
+ vpaddd %xmm0, %xmm3, %xmm3
+ vpaddd %xmm10, %xmm3, %xmm3
+ vpxor %xmm8, %xmm3, %xmm11
+ vpunpckldq %xmm5, %xmm2, %xmm0
+ vpblendw $192, %xmm2, %xmm5, %xmm2
+ vpshufb %xmm6, %xmm11, %xmm11
+ vpunpckhqdq %xmm0, %xmm9, %xmm0
+ vpblendw $15, %xmm4, %xmm2, %xmm4
+ vpaddd %xmm11, %xmm1, %xmm1
+ vpxor %xmm1, %xmm10, %xmm10
+ vpshufd $147, %xmm11, %xmm11
+ vpshufd $201, %xmm0, %xmm0
+ vpslld $25, %xmm10, %xmm8
+ vpsrld $7, %xmm10, %xmm10
+ vpxor %xmm10, %xmm8, %xmm10
+ vpshufd $78, %xmm1, %xmm1
+ vpaddd %xmm3, %xmm0, %xmm0
+ vpshufd $27, %xmm4, %xmm4
+ vpshufd $57, %xmm10, %xmm10
+ vpaddd %xmm10, %xmm0, %xmm0
+ vpxor %xmm11, %xmm0, %xmm11
+ vpaddd %xmm0, %xmm4, %xmm0
+ vpshufb %xmm7, %xmm11, %xmm7
+ vpaddd %xmm7, %xmm1, %xmm1
+ vpxor %xmm10, %xmm1, %xmm10
+ vpslld $20, %xmm10, %xmm8
+ vpsrld $12, %xmm10, %xmm10
+ vpxor %xmm10, %xmm8, %xmm8
+ vpaddd %xmm8, %xmm0, %xmm0
+ vpxor %xmm7, %xmm0, %xmm7
+ vpshufb %xmm6, %xmm7, %xmm6
+ vpaddd %xmm6, %xmm1, %xmm1
+ vpxor %xmm1, %xmm8, %xmm8
+ vpshufd $78, %xmm1, %xmm1
+ vpshufd $57, %xmm6, %xmm6
+ vpslld $25, %xmm8, %xmm2
+ vpsrld $7, %xmm8, %xmm8
+ vpxor %xmm8, %xmm2, %xmm8
+ vpxor (%rdi), %xmm1, %xmm1
+ vpshufd $147, %xmm8, %xmm8
+ vpxor %xmm0, %xmm1, %xmm0
+ vmovups %xmm0, (%rdi)
+ vpxor 16(%rdi), %xmm8, %xmm0
+ vpxor %xmm6, %xmm0, %xmm6
+ vmovups %xmm6, 16(%rdi)
+ addq $64, %rsi
+ decq %rdx
+ jnz .Lbeginofloop
+.Lendofloop:
+ ret
+ENDPROC(blake2s_compress_avx)
+#endif /* CONFIG_AS_AVX */
+
+#ifdef CONFIG_AS_AVX512
+ENTRY(blake2s_compress_avx512)
+ vmovdqu (%rdi),%xmm0
+ vmovdqu 0x10(%rdi),%xmm1
+ vmovdqu 0x20(%rdi),%xmm4
+ vmovq %rcx,%xmm5
+ vmovdqa IV(%rip),%xmm14
+ vmovdqa IV+16(%rip),%xmm15
+ jmp .Lblake2s_compress_avx512_mainloop
+.align 32
+.Lblake2s_compress_avx512_mainloop:
+ vmovdqa %xmm0,%xmm10
+ vmovdqa %xmm1,%xmm11
+ vpaddq %xmm5,%xmm4,%xmm4
+ vmovdqa %xmm14,%xmm2
+ vpxor %xmm15,%xmm4,%xmm3
+ vmovdqu (%rsi),%ymm6
+ vmovdqu 0x20(%rsi),%ymm7
+ addq $0x40,%rsi
+ leaq SIGMA(%rip),%rax
+ movb $0xa,%cl
+.Lblake2s_compress_avx512_roundloop:
+ addq $0x40,%rax
+ vmovdqa -0x40(%rax),%ymm8
+ vmovdqa -0x20(%rax),%ymm9
+ vpermi2d %ymm7,%ymm6,%ymm8
+ vpermi2d %ymm7,%ymm6,%ymm9
+ vmovdqa %ymm8,%ymm6
+ vmovdqa %ymm9,%ymm7
+ vpaddd %xmm8,%xmm0,%xmm0
+ vpaddd %xmm1,%xmm0,%xmm0
+ vpxor %xmm0,%xmm3,%xmm3
+ vprord $0x10,%xmm3,%xmm3
+ vpaddd %xmm3,%xmm2,%xmm2
+ vpxor %xmm2,%xmm1,%xmm1
+ vprord $0xc,%xmm1,%xmm1
+ vextracti128 $0x1,%ymm8,%xmm8
+ vpaddd %xmm8,%xmm0,%xmm0
+ vpaddd %xmm1,%xmm0,%xmm0
+ vpxor %xmm0,%xmm3,%xmm3
+ vprord $0x8,%xmm3,%xmm3
+ vpaddd %xmm3,%xmm2,%xmm2
+ vpxor %xmm2,%xmm1,%xmm1
+ vprord $0x7,%xmm1,%xmm1
+ vpshufd $0x39,%xmm1,%xmm1
+ vpshufd $0x4e,%xmm2,%xmm2
+ vpshufd $0x93,%xmm3,%xmm3
+ vpaddd %xmm9,%xmm0,%xmm0
+ vpaddd %xmm1,%xmm0,%xmm0
+ vpxor %xmm0,%xmm3,%xmm3
+ vprord $0x10,%xmm3,%xmm3
+ vpaddd %xmm3,%xmm2,%xmm2
+ vpxor %xmm2,%xmm1,%xmm1
+ vprord $0xc,%xmm1,%xmm1
+ vextracti128 $0x1,%ymm9,%xmm9
+ vpaddd %xmm9,%xmm0,%xmm0
+ vpaddd %xmm1,%xmm0,%xmm0
+ vpxor %xmm0,%xmm3,%xmm3
+ vprord $0x8,%xmm3,%xmm3
+ vpaddd %xmm3,%xmm2,%xmm2
+ vpxor %xmm2,%xmm1,%xmm1
+ vprord $0x7,%xmm1,%xmm1
+ vpshufd $0x93,%xmm1,%xmm1
+ vpshufd $0x4e,%xmm2,%xmm2
+ vpshufd $0x39,%xmm3,%xmm3
+ decb %cl
+ jne .Lblake2s_compress_avx512_roundloop
+ vpxor %xmm10,%xmm0,%xmm0
+ vpxor %xmm11,%xmm1,%xmm1
+ vpxor %xmm2,%xmm0,%xmm0
+ vpxor %xmm3,%xmm1,%xmm1
+ decq %rdx
+ jne .Lblake2s_compress_avx512_mainloop
+ vmovdqu %xmm0,(%rdi)
+ vmovdqu %xmm1,0x10(%rdi)
+ vmovdqu %xmm4,0x20(%rdi)
+ vzeroupper
+ retq
+ENDPROC(blake2s_compress_avx512)
+#endif /* CONFIG_AS_AVX512 */
diff --git a/lib/zinc/blake2s/blake2s.c b/lib/zinc/blake2s/blake2s.c
index 0b62313267fb..c54f017fccfe 100644
--- a/lib/zinc/blake2s/blake2s.c
+++ b/lib/zinc/blake2s/blake2s.c
@@ -113,7 +113,9 @@ void blake2s_init_key(struct blake2s_state *state, const size_t outlen,
}
EXPORT_SYMBOL(blake2s_init_key);
-#ifndef HAVE_BLAKE2S_ARCH_IMPLEMENTATION
+#if defined(CONFIG_ZINC_ARCH_X86_64)
+#include "blake2s-x86_64-glue.h"
+#else
static void __init blake2s_fpu_init(void)
{
}
--
2.19.0
^ permalink raw reply related
* Re: pegged softirq and NAPI race (?)
From: Rik van Riel @ 2018-09-18 16:31 UTC (permalink / raw)
To: Song Liu, Eric Dumazet
Cc: netdev, Jeff Kirsher, Alexander Duyck, michael.chan@broadcom.com,
Kernel Team
In-Reply-To: <6FA4008E-CEEB-4EAB-BAD8-267D41574248@fb.com>
On Tue, 2018-09-18 at 12:19 -0400, Song Liu wrote:
> > On Sep 18, 2018, at 6:45 AM, Eric Dumazet <edumazet@google.com>
> > wrote:
> >
> > On Tue, Sep 18, 2018 at 1:41 AM Song Liu <songliubraving@fb.com>
> > wrote:
> > >
> > > We are debugging this issue that netconsole message triggers
> > > pegged softirq
> > > (ksoftirqd taking 100% CPU for many seconds). We found this issue
> > > in
> > > production with both bnxt and ixgbe, on a 4.11 based kernel. This
> > > is easily
> > > reproducible with ixgbe on 4.11, and latest net/net-next (see [1]
> > > for more
> > > detail).
> > >
> > > After debugging for some time, we found that this issue is likely
> > > related
> > > to 39e6c8208d7b ("net: solve a NAPI race"). After reverting this
> > > commit,
> > > the steps described in [1] cannot reproduce the issue on ixgbe.
> > > Reverting
> > > this commit also reduces the chances we hit the issue with bnxt
> > > (it still
> > > happens with a lower rate).
> > >
> > > I tried to fix this issue with relaxed variant (or older version)
> > > of
> > > napi_schedule_prep() in netpoll, just like the one on
> > > napi_watchdog().
> > > However, my tests do not always go as expected.
> > >
> > > Please share your comments/suggestions on which direction shall
> > > we try
> > > to fix this.
> > >
> > > Thanks in advance!
> > > Song
> > >
> > >
> > > [1]
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.spinics.net_lists_netdev_msg522328.html&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=iSaOapj1kxjhGYLgQr0Qd8mQCzVdobmgT1L4JwFvzxs&s=lCEhrz6wQJUUaJOkxFmtOszAgkf3Jh4reX_i1GbI5RI&e=
> >
> > You have not traced ixgbe to understand why driver hits
> > "clean_complete=false" all the time ?
>
> The trace showed that we got "clean_complete=false" because
> ixgbe_clean_rx_irq() used all budget (64). It feels like the driver
> is tricked to process old data on the rx_ring for one more time.
>
> Have you seen similar issue?
A quick reading of the code suggests that means
polling cannot keep up with the rate of incoming
packets.
That should not be a surprise, given that polling
appears to happen on just one CPU, while interrupt
driven packet delivery was fanned out across a
larger number of CPUs.
Does the NAPI code have any way in which it
periodically force-returns to IRQ mode, because
multiple CPUs in IRQ mode can keep up with packets
better than a single CPU in polling mode?
Alternatively, is NAPI with multi-queue network
adapters supposed to be polling on multiple CPUs,
but simply failing to do so in this case?
^ permalink raw reply
* Re: pegged softirq and NAPI race (?)
From: Eric Dumazet @ 2018-09-18 16:33 UTC (permalink / raw)
To: songliubraving
Cc: netdev, Jeff Kirsher, Alexander Duyck, michael.chan, kernel-team
In-Reply-To: <6FA4008E-CEEB-4EAB-BAD8-267D41574248@fb.com>
On Tue, Sep 18, 2018 at 9:19 AM Song Liu <songliubraving@fb.com> wrote:
>
>
>
> > On Sep 18, 2018, at 6:45 AM, Eric Dumazet <edumazet@google.com> wrote:
> >
> > On Tue, Sep 18, 2018 at 1:41 AM Song Liu <songliubraving@fb.com> wrote:
> >>
> >> We are debugging this issue that netconsole message triggers pegged softirq
> >> (ksoftirqd taking 100% CPU for many seconds). We found this issue in
> >> production with both bnxt and ixgbe, on a 4.11 based kernel. This is easily
> >> reproducible with ixgbe on 4.11, and latest net/net-next (see [1] for more
> >> detail).
> >>
> >> After debugging for some time, we found that this issue is likely related
> >> to 39e6c8208d7b ("net: solve a NAPI race"). After reverting this commit,
> >> the steps described in [1] cannot reproduce the issue on ixgbe. Reverting
> >> this commit also reduces the chances we hit the issue with bnxt (it still
> >> happens with a lower rate).
> >>
> >> I tried to fix this issue with relaxed variant (or older version) of
> >> napi_schedule_prep() in netpoll, just like the one on napi_watchdog().
> >> However, my tests do not always go as expected.
> >>
> >> Please share your comments/suggestions on which direction shall we try
> >> to fix this.
> >>
> >> Thanks in advance!
> >> Song
> >>
> >>
> >> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__www.spinics.net_lists_netdev_msg522328.html&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=iSaOapj1kxjhGYLgQr0Qd8mQCzVdobmgT1L4JwFvzxs&s=lCEhrz6wQJUUaJOkxFmtOszAgkf3Jh4reX_i1GbI5RI&e=
> >
> > You have not traced ixgbe to understand why driver hits
> > "clean_complete=false" all the time ?
>
> The trace showed that we got "clean_complete=false" because
> ixgbe_clean_rx_irq() used all budget (64). It feels like the driver
> is tricked to process old data on the rx_ring for one more time.
Process old data ???? That would be quite an horrible bug !
Probably ASAN would help here, detecting use-after-free or things like that.
^ permalink raw reply
* Re: [PATCH 1/2] netlink: add NLA_REJECT policy type
From: Johannes Berg @ 2018-09-18 16:42 UTC (permalink / raw)
To: Jamal Hadi Salim, Marcelo Ricardo Leitner, Michal Kubecek
Cc: linux-wireless, netdev, jbenc
In-Reply-To: <1a1e0b60-24d9-a313-f46e-afb590a99b3a@mojatatu.com>
On Tue, 2018-09-18 at 09:12 -0400, Jamal Hadi Salim wrote:
> Not very familiar with how wifi scan gets initiated. I am guessing
> you issue some GET or SET to start a scan - and you get an async
> response when it is complete (and it would include the time it took)?
> Or maybe you get an immediate response and event notification later
> and the time spent is in that notification?
There isn't really a GET or SET. It's just an arbitrary generic netlink
command that you send down. Not everything is a GET/SET model :-)
But yes, you issue a command via generic netlink to start a scan, with
some attributes, and then eventually get an async notification when it's
done (or was aborted for some reason.)
For purposes of my example, that time attribute would be in the
notification, yes.
> I would still see that as a read-only attribute.
I suppose you could, but you never really get to see it anywhere else.
> And the utility of "execute" bit is only in blocking another scan
> from being initiated when one is in progress, if that is a desired
> goal.
Not sure I understand that. I'm not really talking about some sort of
generic "SET_VALUE" command with a "SCAN_NOW" attribute?
> > I dunno. I think we already bloated the policies too much by including
> > the validation_data pointer, and would hate to add more to that :-)
>
> Your mileage may vary. NLA_REJECT may work acls offer more fine grained
> controls.
Fair point. We do have padding in the policy (at least on 64-bit
platforms) that we could use for more bits ;-)
johannes
^ permalink raw reply
* Re: pegged softirq and NAPI race (?)
From: Song Liu @ 2018-09-18 16:49 UTC (permalink / raw)
To: Eric Dumazet
Cc: netdev, Jeff Kirsher, Alexander Duyck, michael.chan@broadcom.com,
Kernel Team
In-Reply-To: <CANn89iK4O+4RgFkaF0b9N6AFzZCGL8FjsNWBYNcr6MA2CaSRXw@mail.gmail.com>
> On Sep 18, 2018, at 9:33 AM, Eric Dumazet <edumazet@google.com> wrote:
>
> On Tue, Sep 18, 2018 at 9:19 AM Song Liu <songliubraving@fb.com> wrote:
>>
>>
>>
>>> On Sep 18, 2018, at 6:45 AM, Eric Dumazet <edumazet@google.com> wrote:
>>>
>>> On Tue, Sep 18, 2018 at 1:41 AM Song Liu <songliubraving@fb.com> wrote:
>>>>
>>>> We are debugging this issue that netconsole message triggers pegged softirq
>>>> (ksoftirqd taking 100% CPU for many seconds). We found this issue in
>>>> production with both bnxt and ixgbe, on a 4.11 based kernel. This is easily
>>>> reproducible with ixgbe on 4.11, and latest net/net-next (see [1] for more
>>>> detail).
>>>>
>>>> After debugging for some time, we found that this issue is likely related
>>>> to 39e6c8208d7b ("net: solve a NAPI race"). After reverting this commit,
>>>> the steps described in [1] cannot reproduce the issue on ixgbe. Reverting
>>>> this commit also reduces the chances we hit the issue with bnxt (it still
>>>> happens with a lower rate).
>>>>
>>>> I tried to fix this issue with relaxed variant (or older version) of
>>>> napi_schedule_prep() in netpoll, just like the one on napi_watchdog().
>>>> However, my tests do not always go as expected.
>>>>
>>>> Please share your comments/suggestions on which direction shall we try
>>>> to fix this.
>>>>
>>>> Thanks in advance!
>>>> Song
>>>>
>>>>
>>>> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__www.spinics.net_lists_netdev_msg522328.html&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=iSaOapj1kxjhGYLgQr0Qd8mQCzVdobmgT1L4JwFvzxs&s=lCEhrz6wQJUUaJOkxFmtOszAgkf3Jh4reX_i1GbI5RI&e=
>>>
>>> You have not traced ixgbe to understand why driver hits
>>> "clean_complete=false" all the time ?
>>
>> The trace showed that we got "clean_complete=false" because
>> ixgbe_clean_rx_irq() used all budget (64). It feels like the driver
>> is tricked to process old data on the rx_ring for one more time.
>
> Process old data ???? That would be quite an horrible bug !
>
> Probably ASAN would help here, detecting use-after-free or things like that.
I have tried KASAN, unfortunately, it doesn't yield any useful data. I think
it is not a use-after-free of some skb. It is more like bugs in the handling
of the rx/tx ring.
Thanks,
Song
^ permalink raw reply
* Re: [PATCH v3 net-next] ravb: do not write 1 to reserved bits
From: Sergei Shtylyov @ 2018-09-18 16:54 UTC (permalink / raw)
To: Simon Horman, David Miller; +Cc: Magnus Damm, netdev, linux-renesas-soc
In-Reply-To: <20180918102226.8017-1-horms+renesas@verge.net.au>
On 09/18/2018 01:22 PM, Simon Horman wrote:
> From: Kazuya Mizuguchi <kazuya.mizuguchi.ks@renesas.com>
>
> EtherAVB hardware requires 0 to be written to status register bits in
> order to clear them, however, care must be taken not to:
>
> 1. Clear other bits, by writing zero to them
> 2. Write one to reserved bits
>
> This patch corrects the ravb driver with respect to the second point above.
> This is done by defining reserved bit masks for the affected registers and,
> after auditing the code, ensure all sites that may write a one to a
> reserved bit use are suitably masked.
>
> Signed-off-by: Kazuya Mizuguchi <kazuya.mizuguchi.ks@renesas.com>
> Signed-off-by: Simon Horman <horms+renesas@verge.net.au>
[...]
Reviewed-by: Sergei Shtylyov <sergei.shtylyov@cogentembedded.com>
> diff --git a/drivers/net/ethernet/renesas/ravb.h b/drivers/net/ethernet/renesas/ravb.h
> index 1470fc12282b..9b6bf557a2f5 100644
> --- a/drivers/net/ethernet/renesas/ravb.h
> +++ b/drivers/net/ethernet/renesas/ravb.h
> @@ -428,6 +428,7 @@ enum EIS_BIT {
> EIS_CULF1 = 0x00000080,
> EIS_TFFF = 0x00000100,
> EIS_QFS = 0x00010000,
> + EIS_RESERVED = (GENMASK(31, 17) | GENMASK(15, 11)),
Well, I'm not a big fan of BIT() and GENMASK() -- they still lack a macro
to #define a bit/field value. But if you prefer to use them, OK, let's be so...
[...]
MBR, Sergei
^ permalink raw reply
* Re: [PATCH v3 net-next] ravb: do not write 1 to reserved bits
From: Sergei Shtylyov @ 2018-09-18 16:56 UTC (permalink / raw)
To: Simon Horman, David Miller; +Cc: Magnus Damm, netdev, linux-renesas-soc
In-Reply-To: <20180918102226.8017-1-horms+renesas@verge.net.au>
On 09/18/2018 01:22 PM, Simon Horman wrote:
> From: Kazuya Mizuguchi <kazuya.mizuguchi.ks@renesas.com>
>
> EtherAVB hardware requires 0 to be written to status register bits in
> order to clear them, however, care must be taken not to:
>
> 1. Clear other bits, by writing zero to them
> 2. Write one to reserved bits
>
> This patch corrects the ravb driver with respect to the second point above.
> This is done by defining reserved bit masks for the affected registers and,
> after auditing the code, ensure all sites that may write a one to a
> reserved bit use are suitably masked.
>
> Signed-off-by: Kazuya Mizuguchi <kazuya.mizuguchi.ks@renesas.com>
> Signed-off-by: Simon Horman <horms+renesas@verge.net.au>
BTW, perhaps this should be merged into net.git instead? DaveM, your call? :-)
MBR, Sergei
^ permalink raw reply
* Re: kernel 4.18.5 Realtek 8111G network adapter stops responding under high system load
From: Maciej S. Szmigiero @ 2018-09-18 22:30 UTC (permalink / raw)
To: David Arendt; +Cc: linux-kernel, nic_swsd, netdev, Heiner Kallweit
In-Reply-To: <20180918102349.Horde.yHctatUod-_SC696oYKmySa@cloud.luxmemories.lu>
Hi,
On 18.09.2018 12:23, David Arendt wrote:
> Hi,
>
> Today I had the network adapter problems again.
> So the patch doesn't seem to change anything regarding this problem.
> This week my time is unfortunately very limited, but I will try to
> find some time next weekend to look a bit more into the issue.
If the problem is caused by missing TXCFG_AUTO_FIFO bit in TxConfig,
as the register difference would suggest, then you can try applying
the following patch (hack) on top of 4.18.8 that is already patched
with commit f74dd480cf4e:
--- a/drivers/net/ethernet/realtek/r8169.c
+++ b/drivers/net/ethernet/realtek/r8169.c
@@ -5043,7 +5043,8 @@
{
/* Set DMA burst size and Interframe Gap Time */
RTL_W32(tp, TxConfig, (TX_DMA_BURST << TxDMAShift) |
- (InterFrameGap << TxInterFrameGapShift));
+ (InterFrameGap << TxInterFrameGapShift)
+ | TXCFG_AUTO_FIFO);
}
static void rtl_set_rx_max_size(struct rtl8169_private *tp)
This hack will probably only work properly on RTL_GIGA_MAC_VER_40 or
later NICs.
Before running any tests please verify with "ethtool -d enp3s0" that
TxConfig register now contains 0x4f000f80, as it did in the old,
working driver version.
If this does not help then a bisection will most likely be needed.
> Thanks in advance,
> David Arendt
Maciej
^ permalink raw reply
* Re: [PATCH net-next v5 19/20] security/keys: rewrite big_key crypto to use Zinc
From: David Howells @ 2018-09-18 17:01 UTC (permalink / raw)
To: Jason A. Donenfeld
Cc: dhowells, linux-kernel, netdev, linux-crypto, davem, gregkh,
Samuel Neves, Andy Lutomirski, Jean-Philippe Aumasson,
Eric Biggers
In-Reply-To: <20180918161646.19105-20-Jason@zx2c4.com>
Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> A while back, I noticed that the crypto and crypto API usage in big_keys
> were entirely broken in multiple ways, so I rewrote it. Now, I'm
> rewriting it again, but this time using Zinc's ChaCha20Poly1305
> function.
warthog>git grep chacha20poly1305_decrypt net-next/master
warthog1>
Where do I find this?
David
^ permalink raw reply
* [PATCH bpf] tools: bpf: fix license for a compat header file
From: Jakub Kicinski @ 2018-09-18 17:13 UTC (permalink / raw)
To: alexei.starovoitov, daniel; +Cc: netdev, oss-drivers, Jakub Kicinski
libc_compat.h is used by libbpf so make sure it's licensed under
LGPL or BSD license. The license change should be OK, I'm the only
author of the file.
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: Quentin Monnet <quentin.monnet@netronome.com>
---
tools/include/tools/libc_compat.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/include/tools/libc_compat.h b/tools/include/tools/libc_compat.h
index 664ced8cb1b0..e907ba6f15e5 100644
--- a/tools/include/tools/libc_compat.h
+++ b/tools/include/tools/libc_compat.h
@@ -1,4 +1,4 @@
-// SPDX-License-Identifier: GPL-2.0+
+// SPDX-License-Identifier: (LGPL-2.0+ OR BSD-2-Clause)
/* Copyright (C) 2018 Netronome Systems, Inc. */
#ifndef __TOOLS_LIBC_COMPAT_H
--
2.17.1
^ permalink raw reply related
* Re: [RFC 2/5] netlink: set extack error message in nla_validate()
From: David Ahern @ 2018-09-18 17:18 UTC (permalink / raw)
To: Johannes Berg, netdev; +Cc: Johannes Berg
In-Reply-To: <20180918131212.20266-2-johannes@sipsolutions.net>
On 9/18/18 6:12 AM, Johannes Berg wrote:
> diff --git a/lib/nlattr.c b/lib/nlattr.c
> index 120ad569e13d..efbd6c1aff29 100644
> --- a/lib/nlattr.c
> +++ b/lib/nlattr.c
> @@ -181,9 +181,13 @@ int nla_validate(const struct nlattr *head, int len, int maxtype,
> int rem;
>
> nla_for_each_attr(nla, head, len, rem) {
> - int err = validate_nla(nla, maxtype, policy, NULL);
> + static const char _msg[] = "Attribute failed policy validation";
> + const char *msg = _msg;
> + int err = validate_nla(nla, maxtype, policy, &msg);
>
> if (err < 0) {
> + if (extack)
> + extack->_msg = msg;
> NL_SET_BAD_ATTR(extack, nla);
> return err;
> }
>
I take it this set is on top of another set - the NLA_REJECT?
^ permalink raw reply
* Re: [RFC 1/5] netlink: remove NLA_NESTED_COMPAT
From: David Ahern @ 2018-09-18 17:18 UTC (permalink / raw)
To: Johannes Berg, netdev; +Cc: Johannes Berg
In-Reply-To: <20180918131212.20266-1-johannes@sipsolutions.net>
On 9/18/18 6:12 AM, Johannes Berg wrote:
> From: Johannes Berg <johannes.berg@intel.com>
>
> This isn't used anywhere, so we might as well get rid of it.
>
> Signed-off-by: Johannes Berg <johannes.berg@intel.com>
> ---
> include/net/netlink.h | 2 --
> lib/nlattr.c | 11 -----------
> 2 files changed, 13 deletions(-)
>
Reviewed-by: David Ahern <dsahern@gmail.com>
^ permalink raw reply
* Re: [PATCH net-next v5 09/20] zinc: Poly1305 ARM and ARM64 implementations
From: Eric Biggers @ 2018-09-18 22:55 UTC (permalink / raw)
To: Jason A. Donenfeld
Cc: linux-kernel, netdev, linux-crypto, davem, gregkh, Samuel Neves,
Andy Lutomirski, Jean-Philippe Aumasson, Andy Polyakov,
Russell King, linux-arm-kernel
In-Reply-To: <20180918161646.19105-10-Jason@zx2c4.com>
On Tue, Sep 18, 2018 at 06:16:35PM +0200, Jason A. Donenfeld wrote:
> diff --git a/lib/zinc/poly1305/poly1305-arm-glue.h b/lib/zinc/poly1305/poly1305-arm-glue.h
> new file mode 100644
> index 000000000000..dd3fa5a38c62
> --- /dev/null
> +++ b/lib/zinc/poly1305/poly1305-arm-glue.h
> @@ -0,0 +1,65 @@
> +/* SPDX-License-Identifier: MIT
> + *
> + * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
> + */
> +
> +#include <asm/hwcap.h>
> +#include <asm/neon.h>
> +
> +asmlinkage void poly1305_init_arm(void *ctx, const u8 key[16]);
> +asmlinkage void poly1305_blocks_arm(void *ctx, const u8 *inp, const size_t len,
> + const u32 padbit);
> +asmlinkage void poly1305_emit_arm(void *ctx, u8 mac[16], const u32 nonce[4]);
> +#if IS_ENABLED(CONFIG_KERNEL_MODE_NEON) && \
> + (defined(CONFIG_64BIT) || __LINUX_ARM_ARCH__ >= 7)
> +#define ARM_USE_NEON
> +asmlinkage void poly1305_blocks_neon(void *ctx, const u8 *inp, const size_t len,
> + const u32 padbit);
> +asmlinkage void poly1305_emit_neon(void *ctx, u8 mac[16], const u32 nonce[4]);
> +#endif
> +
> +static bool poly1305_use_neon __ro_after_init;
> +
> +static void __init poly1305_fpu_init(void)
> +{
> +#if defined(CONFIG_ARM64)
> + poly1305_use_neon = elf_hwcap & HWCAP_ASIMD;
> +#elif defined(CONFIG_ARM)
> + poly1305_use_neon = elf_hwcap & HWCAP_NEON;
> +#endif
> +}
> +
> +static inline bool poly1305_init_arch(void *ctx,
> + const u8 key[POLY1305_KEY_SIZE])
> +{
> + poly1305_init_arm(ctx, key);
> + return true;
> +}
> +
> +static inline bool poly1305_blocks_arch(void *ctx, const u8 *inp,
> + const size_t len, const u32 padbit,
> + simd_context_t *simd_context)
> +{
> +#if defined(ARM_USE_NEON)
> + if (poly1305_use_neon && simd_use(simd_context)) {
> + poly1305_blocks_neon(ctx, inp, len, padbit);
> + return true;
> + }
> +#endif
> + poly1305_blocks_arm(ctx, inp, len, padbit);
> + return true;
> +}
This will compute the wrong digest if called with simd_context=HAVE_FULL_SIMD
and then later with simd_context=HAVE_NO_SIMD, since poly1305_blocks_neon()
converts the accumulator from base 32 to base 26, whereas poly1305_blocks_arm()
assumes it is still in base 32. Is that intentional? I'm sure this is a rare
case, but my understanding is that the existing crypto API doesn't preclude
calling successive steps in different contexts. And I'm concerned that it could
be relevant in some cases, e.g. especially if people are importing a hash state
that was exported earlier. Handling it by silently computing the wrong digest
is not a great idea...
- Eric
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox