* Re: [PATCH] net/nfp: fix Vlan offload flags check
From: Ferruh Yigit @ 2017-01-13 12:03 UTC (permalink / raw)
To: Olivier Matz, dev, alejandro.lucero
In-Reply-To: <1484226296-540-1-git-send-email-olivier.matz@6wind.com>
On 1/12/2017 1:04 PM, Olivier Matz wrote:
> Fix typo when checking that no Vlan offload flags are passed at port
> initialization.
>
> By the way, also fix a typo in the log.
>
> Fixes: d4a27a3b092a ("nfp: add basic features")
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> ---
> drivers/net/nfp/nfp_net.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
> index 65ba09f..ed3c9b8 100644
> --- a/drivers/net/nfp/nfp_net.c
> +++ b/drivers/net/nfp/nfp_net.c
> @@ -2174,8 +2174,8 @@ nfp_net_vlan_offload_set(struct rte_eth_dev *dev, int mask)
> new_ctrl = 0;
>
> if ((mask & ETH_VLAN_FILTER_OFFLOAD) ||
> - (mask & ETH_VLAN_FILTER_OFFLOAD))
> - RTE_LOG(INFO, PMD, "Not support for ETH_VLAN_FILTER_OFFLOAD or"
> + (mask & ETH_VLAN_FILTER_EXTEND))
I can't find "ETH_VLAN_FILTER_EXTEND",
do you mean "ETH_VLAN_EXTEND_OFFLOAD" ?
> + RTE_LOG(INFO, PMD, "No support for ETH_VLAN_FILTER_OFFLOAD or"
> " ETH_VLAN_FILTER_EXTEND");
>
> /* Enable vlan strip if it is not configured yet */
>
^ permalink raw reply
* [PATCH v1] doc: add guidelines on stable and lts releases
From: John McNamara @ 2017-01-13 13:06 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, thomas.monjalon, John McNamara
Add document explaining the current Stable and LTS process.
Signed-off-by: John McNamara <john.mcnamara@intel.com>
---
V1: For background see previous discussions on Stable and LTS releses:
http://dpdk.org/ml/archives/dev/2016-July/044848.html
http://dpdk.org/ml/archives/dev/2016-June/040256.html
doc/guides/contributing/index.rst | 1 +
doc/guides/contributing/stable.rst | 99 ++++++++++++++++++++++++++++++++++++++
2 files changed, 100 insertions(+)
create mode 100644 doc/guides/contributing/stable.rst
diff --git a/doc/guides/contributing/index.rst b/doc/guides/contributing/index.rst
index f6af317..329b678 100644
--- a/doc/guides/contributing/index.rst
+++ b/doc/guides/contributing/index.rst
@@ -10,4 +10,5 @@ Contributor's Guidelines
versioning
documentation
patches
+ stable
cheatsheet
diff --git a/doc/guides/contributing/stable.rst b/doc/guides/contributing/stable.rst
new file mode 100644
index 0000000..735e116
--- /dev/null
+++ b/doc/guides/contributing/stable.rst
@@ -0,0 +1,99 @@
+.. stable_lts_releases:
+
+DPDK Stable Releases and Long Term Support
+==========================================
+
+This section sets out the guidelines for the DPDK Stable Releases and the DPDK
+Long Term Support releases (LTS).
+
+
+Introduction
+------------
+
+The purpose of the DPDK Stable Releases is to maintain releases of DPDK with
+backported fixes over an extended period of time. This provides downstream
+consumers of DPDK with a stable target on which to base applications or
+packages.
+
+The Long Term Support release (LTS) is a designation applied to a Stable
+Release to indicate longer term support.
+
+
+Stable Releases
+---------------
+
+Any major release of DPDK can be designated as a Stable Release if a
+maintainer volunteers to maintain it.
+
+A Stable Release is used to backport fixes from an ``N`` release back to an
+``N-1`` release, for example, from 16.11 to 16.07.
+
+The duration of a stable is one complete release cycle (3 months). It can be
+longer, up to 1 year, if a maintainer continues to support the stable branch,
+or if users supply backported fixes, however the explicit commitment should be
+for one release cycle.
+
+The release cadence is determined by the maintainer based on the number of
+bugfixes and the criticality of the bugs. Releases should be coordinated with
+the validation engineers to ensure that a tagged release has been tested.
+
+
+LTS Release
+-----------
+
+A stable release can be designated as an LTS release based on community
+agreement and a commitment from a maintainer. An LTS release will have a
+maintenance duration of 2 years.
+
+The current DPDK LTS release is 16.11.
+
+It is anticipated that there will be at least 4 releases per year of the LTS
+or approximately 1 every 3 months. However, the cadence can be shorter or
+longer depending on the number and criticality of the backported
+fixes. Releases should be coordinated with the validation engineers to ensure
+that a tagged release has been tested.
+
+
+What changes should be backported
+---------------------------------
+
+Backporting should be limited to bug fixes.
+
+Features should not be backported to stable releases. It may be acceptable, in
+limited cases, to back port features for the LTS release where:
+
+* There is a justifiable use case (for example a new PMD).
+* The change is non-invasive.
+* The work of preparing the backport is done by the proposer.
+* There is support within the community.
+
+
+The Stable Mailing List
+-----------------------
+
+The Stable and LTS release are coordinated on the stable@dpdk.org mailing
+list.
+
+All fix patches to the master branch that are candidates for backporting
+should also be CCed to the `stable@dpdk.org <http://dpdk.org/ml/listinfo/stable>`_
+mailing list.
+
+
+Releasing
+---------
+
+A Stable Release will be released by:
+
+* Tagging the release with YY.MM.n (year, month, number).
+* Uploading a tarball of the release to dpdk.org.
+* Sending an announcement to the `announce@dpdk.org <http://dpdk.org/ml/listinfo/announce>`_
+ list.
+
+Stable release are available on the `dpdk.org download page <http://dpdk.org/download>`_.
+
+
+ABI
+---
+
+The Stable Release should not be seen as a way of breaking or circumventing
+the DPDK ABI policy.
--
2.7.4
^ permalink raw reply related
* Re: [PATCH v1] doc: add guidelines on stable and lts releases
From: Mcnamara, John @ 2017-01-13 13:14 UTC (permalink / raw)
To: dev@dpdk.org; +Cc: Liu, Yuanhan, thomas.monjalon@6wind.com
In-Reply-To: <1484312782-21120-1-git-send-email-john.mcnamara@intel.com>
> -----Original Message-----
> From: Mcnamara, John
> Sent: Friday, January 13, 2017 1:06 PM
> To: dev@dpdk.org
> Cc: Liu, Yuanhan <yuanhan.liu@intel.com>; thomas.monjalon@6wind.com;
> Mcnamara, John <john.mcnamara@intel.com>
> Subject: [PATCH v1] doc: add guidelines on stable and lts releases
>
> Add document explaining the current Stable and LTS process.
Hi,
Just a reminder that DPDK 16.07 was a stable release, 16.11 is a stable
release and it will also become the 2 year LTS release.
Some links:
http://dpdk.org/ml/archives/stable/ - Mailing list
http://dpdk.org/download - Download page for mainline and stable release
http://fast.dpdk.org/rel/ - Stable release tarballs
If you are a downstream consumer of DPDK and would like to influence how the
Stable/LTS releases are working, or should work, then let us know.
John
^ permalink raw reply
* [PATCH v4 7/8] net/virtio_user: enable multiqueue with vhost kernel
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
In-Reply-To: <1484309921-116526-1-git-send-email-jianfeng.tan@intel.com>
With vhost kernel, to enable multiqueue, we need backend device
in kernel support multiqueue feature. Specifically, with tap
as the backend, as linux/Documentation/networking/tuntap.txt shows,
we check if tap supports IFF_MULTI_QUEUE feature.
And for vhost kernel, each queue pair has a vhost fd, and with a tap
fd binding this vhost fd. All tap fds are set with the same tap
interface name.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost_kernel.c | 68 ++++++++++++++++++++---
drivers/net/virtio/virtio_user/vhost_kernel_tap.c | 5 +-
drivers/net/virtio/virtio_user/vhost_kernel_tap.h | 2 +-
drivers/net/virtio/virtio_user/virtio_user_dev.c | 1 +
4 files changed, 67 insertions(+), 9 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index a5d3e1e..0e45dd6 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -188,6 +188,29 @@ prepare_vhost_memory_kernel(void)
(1ULL << VIRTIO_NET_F_CSUM))
static int
+tap_supporte_mq(void)
+{
+ int tapfd;
+ unsigned int tap_features;
+
+ tapfd = open(PATH_NET_TUN, O_RDWR);
+ if (tapfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s: %s",
+ PATH_NET_TUN, strerror(errno));
+ return -1;
+ }
+
+ if (ioctl(tapfd, TUNGETFEATURES, &tap_features) == -1) {
+ PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
+ close(tapfd);
+ return -1;
+ }
+
+ close(tapfd);
+ return tap_features & IFF_MULTI_QUEUE;
+}
+
+static int
vhost_kernel_ioctl(struct virtio_user_dev *dev,
enum vhost_user_request req,
void *arg)
@@ -196,6 +219,8 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
unsigned int i;
uint64_t req_kernel;
struct vhost_memory_kernel *vm = NULL;
+ int vhostfd;
+ unsigned int queue_sel;
PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
@@ -215,15 +240,37 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
/* VHOST kernel does not know about below flags */
*(uint64_t *)arg &= ~VHOST_KERNEL_GUEST_OFFLOADS_MASK;
*(uint64_t *)arg &= ~VHOST_KERNEL_HOST_OFFLOADS_MASK;
+
+ *(uint64_t *)arg &= ~(1ULL << VIRTIO_NET_F_MQ);
}
- for (i = 0; i < dev->max_queue_pairs; ++i) {
- if (dev->vhostfds[i] < 0)
- continue;
+ switch (req_kernel) {
+ case VHOST_SET_VRING_NUM:
+ case VHOST_SET_VRING_ADDR:
+ case VHOST_SET_VRING_BASE:
+ case VHOST_GET_VRING_BASE:
+ case VHOST_SET_VRING_KICK:
+ case VHOST_SET_VRING_CALL:
+ queue_sel = *(unsigned int *)arg;
+ vhostfd = dev->vhostfds[queue_sel / 2];
+ *(unsigned int *)arg = queue_sel % 2;
+ PMD_DRV_LOG(DEBUG, "vhostfd=%d, index=%u",
+ vhostfd, *(unsigned int *)arg);
+ break;
+ default:
+ vhostfd = -1;
+ }
+ if (vhostfd == -1) {
+ for (i = 0; i < dev->max_queue_pairs; ++i) {
+ if (dev->vhostfds[i] < 0)
+ continue;
- ret = ioctl(dev->vhostfds[i], req_kernel, arg);
- if (ret < 0)
- break;
+ ret = ioctl(dev->vhostfds[i], req_kernel, arg);
+ if (ret < 0)
+ break;
+ }
+ } else {
+ ret = ioctl(vhostfd, req_kernel, arg);
}
if (!ret && req_kernel == VHOST_GET_FEATURES) {
@@ -233,6 +280,12 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
*/
*((uint64_t *)arg) |= VHOST_KERNEL_GUEST_OFFLOADS_MASK;
*((uint64_t *)arg) |= VHOST_KERNEL_HOST_OFFLOADS_MASK;
+
+ /* vhost_kernel will not declare this feature, but it does
+ * support multi-queue.
+ */
+ if (tap_supporte_mq())
+ *(uint64_t *)arg |= (1ull << VIRTIO_NET_F_MQ);
}
if (vm)
@@ -305,6 +358,7 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
int hdr_size;
int vhostfd;
int tapfd;
+ int req_mq = (dev->max_queue_pairs > 1);
vhostfd = dev->vhostfds[pair_idx];
@@ -324,7 +378,7 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
else
hdr_size = sizeof(struct virtio_net_hdr);
- tapfd = vhost_kernel_open_tap(&dev->ifname, hdr_size);
+ tapfd = vhost_kernel_open_tap(&dev->ifname, hdr_size, req_mq);
if (tapfd < 0) {
PMD_DRV_LOG(ERR, "fail to open tap for vhost kernel");
return -1;
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
index cdb5c3c..f585de8 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
@@ -44,7 +44,7 @@
#include "../virtio_logs.h"
int
-vhost_kernel_open_tap(char **p_ifname, int hdr_size)
+vhost_kernel_open_tap(char **p_ifname, int hdr_size, int req_mq)
{
unsigned int tap_features;
int sndbuf = INT_MAX;
@@ -91,6 +91,9 @@ vhost_kernel_open_tap(char **p_ifname, int hdr_size)
goto error;
}
+ if (req_mq)
+ ifr.ifr_flags |= IFF_MULTI_QUEUE;
+
if (*p_ifname)
strncpy(ifr.ifr_name, *p_ifname, IFNAMSIZ);
else
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
index 8535116..eae340c 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
@@ -64,4 +64,4 @@
/* Constants */
#define PATH_NET_TUN "/dev/net/tun"
-int vhost_kernel_open_tap(char **p_ifname, int hdr_size);
+int vhost_kernel_open_tap(char **p_ifname, int hdr_size, int req_mq);
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index ae0824c..6617bc8 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -93,6 +93,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
state.num = vring->num;
dev->ops->send_request(dev, VHOST_USER_SET_VRING_NUM, &state);
+ state.index = queue_sel;
state.num = 0; /* no reservation */
dev->ops->send_request(dev, VHOST_USER_SET_VRING_BASE, &state);
--
2.7.4
^ permalink raw reply related
* [PATCH v4 6/8] net/virtio_user: enable offloading
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
In-Reply-To: <1484309921-116526-1-git-send-email-jianfeng.tan@intel.com>
When used with vhost kernel backend, we can offload at both directions.
- From vhost kernel to virtio_user, the offload is enabled so that
DPDK app can trust the flow is checksum-correct; and if DPDK app
sends it through another port, the checksum needs to be
recalculated or offloaded. It also applies to TSO.
- From virtio_user to vhost_kernel, the offload is enabled so that
kernel can trust the flow is L4-checksum-correct, no need to verify
it; if kernel will consume it, DPDK app should make sure the
l3-checksum is correctly set.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost_kernel.c | 40 +++++++++++++++++++++--
drivers/net/virtio/virtio_user/vhost_kernel_tap.c | 14 ++++++++
drivers/net/virtio/virtio_user/vhost_kernel_tap.h | 7 ++++
3 files changed, 59 insertions(+), 2 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index 9301f96..a5d3e1e 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -165,6 +165,28 @@ prepare_vhost_memory_kernel(void)
return vm;
}
+/* with below features, vhost kernel does not need to do the checksum and TSO,
+ * these info will be passed to virtio_user through virtio net header.
+ */
+#define VHOST_KERNEL_GUEST_OFFLOADS_MASK \
+ ((1ULL << VIRTIO_NET_F_GUEST_CSUM) | \
+ (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \
+ (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \
+ (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
+ (1ULL << VIRTIO_NET_F_GUEST_UFO))
+
+/* with below features, when flows from virtio_user to vhost kernel
+ * (1) if flows goes up through the kernel networking stack, it does not need
+ * to verify checksum, which can save CPU cycles;
+ * (2) if flows goes through a Linux bridge and outside from an interface
+ * (kernel driver), checksum and TSO will be done by GSO in kernel or even
+ * offloaded into real physical device.
+ */
+#define VHOST_KERNEL_HOST_OFFLOADS_MASK \
+ ((1ULL << VIRTIO_NET_F_HOST_TSO4) | \
+ (1ULL << VIRTIO_NET_F_HOST_TSO6) | \
+ (1ULL << VIRTIO_NET_F_CSUM))
+
static int
vhost_kernel_ioctl(struct virtio_user_dev *dev,
enum vhost_user_request req,
@@ -186,10 +208,15 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
arg = (void *)vm;
}
- /* We don't need memory protection here */
- if (req_kernel == VHOST_SET_FEATURES)
+ if (req_kernel == VHOST_SET_FEATURES) {
+ /* We don't need memory protection here */
*(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
+ /* VHOST kernel does not know about below flags */
+ *(uint64_t *)arg &= ~VHOST_KERNEL_GUEST_OFFLOADS_MASK;
+ *(uint64_t *)arg &= ~VHOST_KERNEL_HOST_OFFLOADS_MASK;
+ }
+
for (i = 0; i < dev->max_queue_pairs; ++i) {
if (dev->vhostfds[i] < 0)
continue;
@@ -199,6 +226,15 @@ vhost_kernel_ioctl(struct virtio_user_dev *dev,
break;
}
+ if (!ret && req_kernel == VHOST_GET_FEATURES) {
+ /* with tap as the backend, all these features are supported
+ * but not claimed by vhost-net, so we add them back when
+ * reporting to upper layer.
+ */
+ *((uint64_t *)arg) |= VHOST_KERNEL_GUEST_OFFLOADS_MASK;
+ *((uint64_t *)arg) |= VHOST_KERNEL_HOST_OFFLOADS_MASK;
+ }
+
if (vm)
free(vm);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
index 797713b..cdb5c3c 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
@@ -50,6 +50,12 @@ vhost_kernel_open_tap(char **p_ifname, int hdr_size)
int sndbuf = INT_MAX;
struct ifreq ifr;
int tapfd;
+ unsigned int offload =
+ TUN_F_CSUM |
+ TUN_F_TSO4 |
+ TUN_F_TSO6 |
+ TUN_F_TSO_ECN |
+ TUN_F_UFO;
/* TODO:
* 1. verify we can get/set vnet_hdr_len, tap_probe_vnet_hdr_len
@@ -106,6 +112,14 @@ vhost_kernel_open_tap(char **p_ifname, int hdr_size)
goto error;
}
+ /* TODO: before set the offload capabilities, we'd better (1) check
+ * negotiated features to see if necessary to offload; (2) query tap
+ * to see if it supports the offload capabilities.
+ */
+ if (ioctl(tapfd, TUNSETOFFLOAD, offload) != 0)
+ PMD_DRV_LOG(ERR, "TUNSETOFFLOAD ioctl() failed: %s",
+ strerror(errno));
+
if (!(*p_ifname))
*p_ifname = strdup(ifr.ifr_name);
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
index 1fc58ff..8535116 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
@@ -54,6 +54,13 @@
#define IFF_ATTACH_QUEUE 0x0200
#define IFF_DETACH_QUEUE 0x0400
+/* Features for GSO (TUNSETOFFLOAD). */
+#define TUN_F_CSUM 0x01 /* You can hand me unchecksummed packets. */
+#define TUN_F_TSO4 0x02 /* I can handle TSO for IPv4 packets */
+#define TUN_F_TSO6 0x04 /* I can handle TSO for IPv6 packets */
+#define TUN_F_TSO_ECN 0x08 /* I can handle TSO with ECN bits. */
+#define TUN_F_UFO 0x10 /* I can handle UFO packets */
+
/* Constants */
#define PATH_NET_TUN "/dev/net/tun"
--
2.7.4
^ permalink raw reply related
* [PATCH v4 8/8] doc: add guide to use virtio_user as exceptional path
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
In-Reply-To: <1484309921-116526-1-git-send-email-jianfeng.tan@intel.com>
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
.../img/virtio_user_as_excpetional_path.png | Bin 0 -> 38600 bytes
doc/guides/prog_guide/index.rst | 1 +
.../prog_guide/virtio_user_as_exceptional_path.rst | 104 +++++++++++++++++++++
3 files changed, 105 insertions(+)
create mode 100644 doc/guides/prog_guide/img/virtio_user_as_excpetional_path.png
create mode 100644 doc/guides/prog_guide/virtio_user_as_exceptional_path.rst
diff --git a/doc/guides/prog_guide/img/virtio_user_as_excpetional_path.png b/doc/guides/prog_guide/img/virtio_user_as_excpetional_path.png
new file mode 100644
index 0000000000000000000000000000000000000000..9fd45e13eefc2ef39ac2c389f407d60ab4144c59
GIT binary patch
literal 38600
zcmeFZdpy(qA2&W76`>TJkPbu+9mM6hkRnDo=QL5tVO(L(+mv$(Q4t~M97nNb*gBA;
znuUfnD`#ee88+K~AL_cUuIqcae~0`2J?=m5KN9o#ybrJA^Ywha-xAK7pWU)~_ht|X
zw8g~uv?U0%&Hw~j(=50F_(Yg<KNk3JorgX|9|WpN7Fxcv9{73F@5UDcKp@GjEB^%E
z$<2X5pa)(ir}eEvTo_!Tt}AvIM>*vi-TI8vT#u~x5|-A<j&eQlZMUTfTP;0M!!@i~
zh<W|d+IRu|7y8=U4T4(<Pc%gmBE!ry)U5q$OE);WyBk<L2nnCdcUpg2!_vgqvfeKB
zoTb8=N&cMA!U=lm#Q-WR)4QsP#YVnayi?d8JY6?}d*3t?(O<W~+YJI8*K+Z{dpY?s
z_Hx`hV7(xa>k&q(l2h%zHLE{W-)EUd$ev#N^M`@`f-z!0zOOqRz4}dL?_;qa&t8o(
z5M2EzW6NgQ)o)zY{}(*;+u^)FCcKo~EUC{rw>lRH^QkS>`=G<RUU^N^9}D{N`TxYP
zk@bMIMiklnAG?F7-d}#N2$MPzTF{)cAihm_RH@DFzQ=?ph#8dH?D9IN0z76cl*WqK
zd=GfxMRdfueDTSqcGO9VY|UuTocROS-_WT&yw6-wpKZ!(oDTLL=v99BPP5F&m2T%j
z7uD)JR5NNNC;+@GH}v@Q##uWhzKsRnt1L!z&oavF)jHP@zqB!4KVez2`eKCsse<))
z6`r4RgSnJsl%rAXFNk*3%DWlsxb2mYbxc(poXOawwH4b+ha%^dkMHa++D07>lNYJ>
zT&CZ=vYC1EeP6>z?}rknnDq`~D{mg%$_^D*-`YW3ev*$(p=2M}%{5hgSA*T}slTpA
zjfMMqVm)GNtm9PztX#qxcphYSI&0xAhthhI9lGOi`>peAiCwOpi{y>%+k3K==#BT!
z($2Y4%48m_xRC4qK#*V5(xRg{Q#FUz#Mkodz_vbC;<FCjI^T0T2+8)*;`$u$F2{d_
z7sa@T<0bB`EXnM2H&%L#PluW?g4qJ4^oXWH52#IPd-s^L2pifoIMkA=B7-Y~lFQql
zZi<(HG5Parq4Vm&h$9Kekpx6|49{FfhBlv1@22$CNjZ#>_!G{O2q8U9Y&QW>>es-e
z>%qd|;BiM7Q~3gfHhW?IWFDJ8qt}JqGZtFZ&Itc_*f~~O+?jWB8&#Ce5387e{w9}E
zJ!O|UajbLhTzYvs*}a(a#Mjw*SVvVoAv;$u%ba+`+nY*RQ$AMjMnrSYdFy)AMwYf-
zgr1$RGa-8D_R~AaM>(<53{jjJqczM0zU0N|vfD@}GM$*CSII%F-u^{6b&RSJc02GP
zo9%|~?z2Vs(EJoWskfrtKQGMJDb+#C`}$5pb0_ED_s>3|SYxZI{Id@L;&MDPFC8i+
zOo_HCh^o_D3z1qJKmF-}-42)`B>^vDJ+G<tyi?!DYY7KCv+*YTgjA%Wb8?j`GAUW)
zT!Xn;T#v2PHOd@20#c(I)<WW5)VM(6UQMn;<co`Ii@+~>fEB6fS}7LFo$|g7w2$ia
zZj;A3XORzWNbvEY@^aaaiICL32auA{+jVsx4(g6@PIuD_%(b}Qo_i5@yeJyAgkHM6
zGgHTVE{rjKjWlP)(JuqHwhU?^u;kBZ?uGHXClCT@6s~N1xGCtG{((B`^;$-sv^yA!
zQ75tLG?BZhabae-cZeilt3Jv{HrSK)NeesXU2@hs`s~WZz?2ekCr-J&j$n6!kCc=#
z#eE*_!m>MgM}>&IiW=@>QKsv3*>=Y`*eXX#9=LOi&sme8Qc@|4u~}kO9qGU7jGih9
z>2Nx^32Sge&u(J}GMCe^J5ZOEZp>-YMYl(vOCQcflQ3S|fl^v2$?{IP<ol3Ef!UB{
z=)7{!`#>2_%U0!_?cKX38hZ}n84n<%$?aOGLs-4MQPeHG`4X?F)rm+CaYG|Yr%u{(
zvU2=*n!9!Vm?9e>%N%v?hg?qB#yH<N9szSPy_8Y{gOyLS*km{&N1~(xvfDXl9(9LD
z;JrtpN!*U0n`kbXkhFAV%sXQqLfb$^hdGl*`-qe9l3Tv%Dy`Cfq+HL*c_sL5IJov$
zmnIM(tM5nLIJNPd*5TPGe6fIwc-G--f%Uk?GJUUzxL9+`va!MzX%CVBw7a<r^(1dg
zM4c`!sZ5}LN;KTu+422(N?C=nR-1jCOfB8j9!u<GUCTOYBW{rO``FiBIc?k)tsA$}
zJpws{+)u?JkBva#NoVBO+R!`P=^Gc>c6Q$7yv8K+pvlnP&VJNrdrllQy{g1_O?m8!
z*VOfTbS@1-;mGb9@;0vDEopB@yo8jFuHU2%nOvoK{gMOO=45C$>^q{Ql+ZFt(8Ciw
z#OA3keB6}K_P)3Y_LVPwecSz*@Ymq8oTn-!xA0Xx*E`5y-3_cSI7BgrCK<=%!)ew{
z8^(z93%m^Pyuyzuqd@L@Vf#too{UZ}-<vHQe_y13`DytEWRA$E3Ms8q<zai`2}RBY
zYw5U6r6Rv`2jLy|8r8NqovicTjuXQ5mG9(eaPeX-SieI75C8@$VYOGT)k0txtEIv2
z+AR*u>edOKpO*O<N)?e5)K_>)$k=9b#+h8Oyv|a7hfS;=uyI%kvA8qf0;P1cU8}-k
z?6hu6k~_88iC9#Y@FYhFxsA#?*-Pl}p2sel-@zH2+JnGqAf3t0>+dV#(32rv6a8}H
z({_y4koAxvF1czG{`>hl8ep#>cYMRM<_|L#c=a}ZX}j2afk@hOj)!)Tkor0=B}RYz
z;Xz51yHrJ8QOm9@SL-cLxCd=16t$OKFKcyOI)gZY=RJv~nJFSomG5}|rEACGi`?#@
zeN*GSx>4_1!B22a9y-M)3J(5_Y;uYWUrb0Tp1}AMa^M)F@tIGfwmZ(BdnB3FjOq61
z>|JM>Y+;AD7|Rw@2ofeUusq*<Tms^j>S0?1FCq6Dc?Yv^qM?ZgKi7#7bYo`N-^icB
zeO>SdM#X!U;?x%#HaAqA9ruT8GndQ4xI2Gt<@$|NRtt7a%%p0RmaC7VW#QPVzP_}~
zLbeIK6ZZNY?J4yNCot8USdQxs8H05TRC8VAH`BkT%~nNS8D5AIc~7pt-)CW&$-7xK
z?jILRN{G^hTGdrza^P5_lRGR{baSV!+aqmUuP&%+E_0M~+U63GrrR;UlRa#=!(n2K
zT|uyc<tjtd1{qrs7`r9%d>J`j4j+;o_8}?pG5!1cb&y-A56;1k)>Nd@_0@272;BQr
z;O<rGChV*q_M*q{#mm`>xW){AJE|KxsKd8=rV`vRHuNGVJ<~e9$Ci);=X;HAY-H1_
zj5bh%ybZHnS?y?Q&`-}3FV4t)T+yI<pML~ve#Q-nLeJUtsjo=<P2a?`wg^(i#K$}>
z<SXa-%mJYfVN<~hB4f?gxmaIW$VF6VzSDaGrqWR?e!kab7)mBnuOf1k55T3K!N-uW
z0EY3*dcmgh!!2kJD$MgZb2V(#`G^(PP=067=wCD+Pol#R8$6Ilvoe<{jz(o0@@n9G
z%W-+3(x7`iwqV&QhZo-?jv01dnH>ew^NG-;y2GyR3CU4>@88q0cbg+fN8o1ud!NZ2
z0#b!LY`*t0{<|&w>{%(4iXG(1{N4q04O^?LSd?Bv4mg|JIqJKnAwWNU#~g)L@_f9q
zq3Mkand9^9DcwFIJ&Mp^&Kbik`v-=**MGJuY5uX+ic_xZ*K>t%TdJ4@=c$~dJk?pv
zZK9vx)i8p$HpnUX#h9O;Io0_n6Qjt-FX^pcQzY(1kpytg?^&;6A(I^5a1Vcnt_vFC
z(kFo}Q`<5L(=FX(rn0i$G_M7@xnR{oc91wixn`8S!)`v9SCd&hIG(t@e%k_8J-D{Q
zJ)2>}gCnT>8%bmZ=cNAo-^S^=l?BsOS9mUED3PqWP-!=?>W&WT3am5(uzi<_JDD=e
zsIT9~ulr2<)IXxE!&9X@%irUv-0Ip#LZ%-nC74kt=}h~f#~h+X7rY)41<$f)B4p}!
zD5%ZwYIB2?ZCfg?6;LwV=q3<X{~GwvJFQ@)dk=u+dItcq<vg~@7Q=oM%G}McvWbB}
z3)}8zOefO&>#R%KS&DpI|L8}JD=i<Niw#g}&1c%54Hs7F^A;-ueCsa!Sn4^gt#tzD
zbrhIu&q#g8a!v0-i_O&nt9vzO_EGtD5I!M6rJ#LKSNTy-fcqhgO>gChChc`43Q>3S
zM)CNR42ND~p2Pl*{Sc0XvYI$A`?GQtpA0!!+_xNUDV_=5eNp`)MO({q*--|!eJb$g
zbYe<hi4~odOTDKh^V#G*h5(cGkb+?WCxbH#>9br_q~k4=?sPzKHuUU7re!eoZP5?q
z-w*uizP7Xr_6qs9D5_U|?OCvXhsjql9Ivi&yedh{R>`=@xu6-N$Z4Mw%bh?6@9tS=
z%g@UEo#XhK({FTX{r%@p3h32A%-(d|UXJ(g-TwX+CFZ4bZk_f5?>;Rkn;Xu_T<yJ|
zTU+9yO&?qOmUy%@pjk3M4p(rB^H>G-78_rxPAOlehvkZHWVOoS)5_>|1-A&^$GdF_
zIeJFfT8BOYRbgahVbtk8#x_)iZyT$%u0LUW=Xi)OsGt-W1{*(pE>>sEU?YRC_-!i9
zR1yZ_Z(sj{?`=i=`FojN<NGK`+Gy2wF1=b=9<FgFDdA!k5mY@d9QuC6`_B-?Idp<V
zV=UVz+G%oUrB(He(EwbYffp-iSwplKiR(+YJv!<5#i!gi{65N=jBHfl$nAhQ^~~p%
zUx0t`eogCr&?mt^8=IE|O$Y#LL)RlqYxKKaU4*vS3U}1jA9;C8UQ5Bt>dDx<4S6v6
zqv}UpJYKm6AGL-^#P-`lqj$~*?Dx^OY9)(4hi`lKF=!&dC$*y8m>TNVT1u>B+YWVD
zLqvRZ>At3W409#Nh`C3clkVm;giSGZ<CY6`cxJ-1X{%g)Qo3)HGNH^h!;#8TggV({
z$LK<NrTYsdlqVPH_><=7`<UBhWJN+w-e!nl?keVOmy6ECy=)Yjm_(Ok7E`!f*h{+b
zQ+T9Zt6UdsdT+Hs63^(V5LF%{WwZ7}9=6*;I(E4nO3hY9?m0;}vy32BeyE(G3r})6
zvnOphnCW9yTD!{VClF*(pvz-0pG_XF5<!g`bb4Qs0^fyD1un&8mc6<pg@~9iUq3kg
zaU#Iuv?PS$ji&FWj#T+6kYJdbRmBq-eRXEj@#8pX9g!0>LBF~KNRa)~1#~a=mn2!@
zswG^BdZLcHLB8>bdaklG>&rOk;{)oI>e8s=9MdrBECXEw<A1u!NCN+cPT;pgyDwiF
zgU$Ov9Tu8Kv@Vusjg71K=s_Tp%gw}8cB1bNzDrQ{Y3nU`?!f~3>9^DJi7Kis=(C;k
z<yv+7iWr?&tQq7yq7ZYG*xp55tDtqP4xi7j)T#=bnxI>CYd%?a{A4+LSM_F)!t-mi
zE^?qQxO}r0?_&av9E<VyK{M=3(_ht9(P;JdcE<C;&+C+nk1(=$RXZRPB3gsVxe`XW
z#+-3vE!=Z!y%#|w^e03Y>*Ct9cfL%Msrbx2L=shpzvvFsh!T0TJY+3@xAbMMh#cb>
zOzxIeNL_1iQ~L`)L+a&APhd89!5WbUQ-fs3?wL=UZdV?dPj|@5pda@M4VzM9@I(Cb
ziJqCliv(*rzCye8o-~w()COnuZx|*Qcv=aQOwfeHQ)HWkMp<CLC8PN$UGFeN#~f9<
zyfiapLesPL0ujXxHX~P#M(drSoNr<0oDr%oy%Opm=JO1~n<_27qTW{1)Q7Fm#bpO+
z#HgT4&ley%s;;n><?u4zFY`*zm%^%HOb&LOaT2KuOO$2fE>X}V@(rB3+E1t*SUt4S
z0=7grHST&&{+>->J<I-%xjkI07-iGFDTik#->+rQT>A`nBbP6RD>ixhNuWYk?_?W#
zP+kub)ecl7z5B49ZqP%>hIkcXU(pQeTZLkL@48d`@<~Hch9*u#S2Zw{4dr50$QI_z
zdYGKRJHnka1?`{QzbzD1on5G>rQ@7Ge^y69k{0gxCekz{vJa0<FT2B=ATe7`c7@*1
zS$5uxq0NU-YRK*dqY1n(U~^(y+5Y>|8vGDOgM%80e8jm}8b#A!(rg9>G9<(+2;<YW
z7URe3P|dKeR-2kROA5&@iP7D<L~At)W@xZ>KPC+p)8n!AWxws{d=;mvevQL~pH0TP
zoM0|sq-hIqM4uif*jd)Pg-(KfLilms<nuqnn91E^>`a%9v07(=q7IYFPYfXw1eb0+
z7TE`{wlUiH04et69*z=u9~#^V-F&}_CJnW#H6h(7)w9_605;Tbdu6R%c1UHrWMEl7
z#op#g^`?Y6eac>o@AQ054ZYB1;|xo3sCew)sg3M+ePwhXL$bJqUS_^gBF0GSWOL|^
zh=Es*R?I~WDH&ZF;$^iWu0qb4lf}v^-k%MbnkQ&7sV>lzrx-dtoPsZ<d?`LUHI8KX
zPcq=|p;3KKR=BFO1!ze&j>>|&#{Y<#<INW~R%OCBoOaNu8Kr|Nd|wo&ok5r_+rbk>
zg5ZyPt^$zys-s_)e!jT@V+Mi+x7I?~jhHmA`vH#kC1h&1ApD@81sYV2bj!gkZ1M~i
z9_p&}i)zh0aVQq4ou7f-vazX!;BJ>8O6i|X5^CBqipN#!av;{s(9-#9%;)3KEcz`Z
z+&@?Ht!c7%cfAj(p%j-x)OrYkjLdsDLxbn*%qSTSIcHEMqlhfBk+SjFQVu665{|Va
zBEl=C-TO-B>(E|Vbzr!{&cYsVcP}ZnRx$UpOfYK8gg=CIWlVEux<9_4y^CC1Mjn*Y
zT#03%%f)LA8V)`8zV7Cw4s(R(!x)|?##zB4<Ix9<Hg;p)%VIlOSIFAoxKAeWv>dqn
zA^1qw#^N_gcbbgkT9=C3s|T~)w@q-qbZ{?8Ib-dFPO!AB=0o^>MZ0cJa||U9alIpX
zDBmpPN>~ByG3AF{6Ter5%~^YDPf*Fd{<f7*K?R>2M=QmSH$#;LUg#N~rR`p21q(hc
zv^vUOj>pD>GBgm*HRPR$rWIx>@<OjNot<|Bl#<o&c{L^lFSB{e$|y~q0;lB<1B6)Q
z7iN}ug;+jcbmij$lAl!drd!DJwV<Km;2S-3VVdvC_^#GGx}?Qt8PKV#Jn!q2h;EgY
zr$Kx4!OYqH3COv)e*Id*AM|AP{bdbd&qc*m>I_s}_kWnyRZ;z$FE2Wh(tpsSy|OF1
zS}9)}4hl&s$NzZPr--`^DoFa{hv$L@vOmadALakfL{B_A)O*AvTRvLMKv353Y^$v5
zeGn)GGI4Wl1|qdf6ZkM%%u;dVA;6&1a6KS7ICAB|=zAg{Q19<~`q{s`0CV0OYL(pv
z0zFMoJ??bvYs?&=-Ny&PB6n9tin#mDV6EySN#L<VOV{Jqfg*hd5``sqhkR=UK04f%
zY<LhDTO4}%dB+Q(QJ`aR$^^N0zW``o*5dxxF&kWdQw1J7H#;b{37D~2a+iwQ_gA5S
z2+pU<#2JG?Z$}<S3vDTj{&mg|(g}pOllMV#kM{gBtN8<`10<pMdTkfCy@(b7xQ?F_
zfe!B%KK0ier?y@Kt?>aG2K85KJcv1c5ffIq4!Sr(-e--GpuIXFzt4AG#p~jPK|3>7
z7XZ2?xK_-yvyoY(I=^=4jn3RdqEq8yi>k|HU!8XiiqGq|?4v)$4<>_=^;Jizp1aLY
zdb#_y_BVywfh&t!w(%dOnQN7H&w(DRhEp#B`+ppN6eOSyA+NbfOp?-;@C&C<Y)!p9
zsPoyOBM<A=CmzEVls?WgN_qBC>xd4o%>sGcNDS1{<4sfqHem$p3}CaPdCPM|^lgd#
zOqI<jJx6oBrJ-R*-M;U1PD1Wt!Vpdk9oAj`pnaZBT4T^nIsfKM5-vYh!Y?qJWlxQT
zPn{Bog!j__HXweB@@c2A@uIV4Qzu)c!+l1v)c^t>$WK(!a?^EqZ`|9==(rgtp*MLw
z%uWzkP*JaU$6pJ=Nr*KjBPQL?Zr@fL{}^1}StpiXd-NOmuA-bzU(gtFV83xURS9(T
zV7QdSUn0m|bMqFMZvg&UJ4HGFSt0r`U;SYhmf-U#XpAub`aENSZ4cU+*d$R2*xUKj
zJ0n3^#eM?%Ek^6OlgWy734{K{HyW{{3_#TLov+oeXxADS$l$#QokT4KlSB>nOEfRw
zg39x+H;3?_b9?QUY^jxQ1dAOpECwariL3>+a`MuBvA35_`fDHl+r{pg+s-Q=!tLdv
zdD>T~2Jz-dMkxC}wg%E8lAS@%KFA7y1t_pX<#{UiteFfjPZ^BNwUrX$H2I=QZZXLD
zYOcx-<qhMb4YgN@d%}PTjJQDe?5EPJ*4+H0ec0zBTat6nS|D!=AEFvIddg>#(Lp>m
zW0X<_ZEaBDHf~2rnr4krPkLn=#P{DB#6DACYbw~Z93)DEnPHKdx7b2}asM#mN9)&R
zn^B%Jr=ssYkL-wrR>__GCKaH~Oj_5FcsdQ|enpTSB*DK?(YFC+xBOshlhA3(5ZJ>=
zh72gEy&6Ty-g*7Px%7e<321W%q9?F|QDcbSm~nu9pd2u^?OQkOdr>?i&fdG6y{D9z
z{hjW8!n1o*b?&4s!nZYpa*1RRjW;UV&RyQ>h?UT^%te124_YKT%=i_RYDtu~PDmYw
zK-X5ZDDojXeR$kx&tR;J=+U#E!M1>ZuXtr7B&{LxE4Wkk<go>MCl-7y@4ya(>APN!
zTq)^tl8JveMdr;HkvkIlOvbUIF=uJaR8<-Z)f!V&D&{uM{)Xo5{ES$vfW}>YG7kzE
z;yeIC;!i<Ot+`1U+-I{hUPusD;3rVp5Oo<4a)a$TL}EgJ?<Kzg(i*eQ?0pI|qqk}m
zqQ@xKmcgxEq&0V_!+Eeu#mn~{fOX<gCM)qLq|GQ*H<g+ywHj8cJ^dxcm~(r5LQ9G7
z#vBAmSlu!IOu08%)lx;bWO=?R@M&a+!3_yvA5GY1S$hA>z`~Vojq!Vb4R|%Z8G0G$
zK4$#x3(ArU7SN}Ca%=Cz;+vP*6aBjf2_toPj$vP;Tub}!pAQ~AC&Ip?hQaH_Nb-YH
ziw)u>LO&N7DRCZ2awqTKv!n}*yk(OC`@Q@O`YL9F4G=kfGQX@}#}~+P=!jpMA6D9V
zyIzX@c{|@WEiX~_Ge*y7?J$kkX)Q?74q#V{s#SUglmfE5%=O;!Ykcofs}~Z)?@P&a
zvM3V!1aOhQ>$0M~nJRbl@J-EAr$~HOUNCMEW?VyC;DSCC4_d8w(#2f+^}j}cb;!qr
z0*STEC>8ic_L8{5bphFQz>Ab47O`!@Ho`S%7Q<1Hgl6Bt%3dW*a~|%&5HhC(cdhu$
zMLrNFen}+2P-|}H+SAFz;=;{>nm+G>np?!0o}CstqB$JqeVhL~so21njDB{fmwFtM
zyY={rn>^*!ZT>Z-_+$dk0tG!`S9dCGH(RMYe3O<Kdw0im4;J*94x$T=(7LLyZj
z$6#$aX%mz5i*)}?Ux4~|h%FB_`W|l>k4bT&Y?U-*&eRVrvYna&is<CgD#ySXntuUD
zqo@m5!Ymi))1FeFF^U!1(g<YIEV;7+m&jGlmv&`#VJk8qD$zszB3$)d2+l2C6^~=h
zQw}7z*s78v%8%_E)K<(VqM#j*;Jg~G97E016pB8teSdcv`nOEnqlVZl$hM6ZQhB~l
z2inQJl%8cTxXWLgIkf>QJkGiF_)NNYaNwhe*|u!Gd)=rFw~8RLr#tJoJW4Rr(W`6`
zel2#@H2qd=T@L^D_q-dDAB9{yE38g+m|*c+Uh>xXk=m}MK@_9!^@$*kB%=15t8#n0
zhqm~p+DtjNw*JIvEV*w$NDUGr#<y?EDjYvy5lCaB#eMrqSkLpwz>4RSZT>#?;-)6^
z4lB|othxG0)`Q!>v7f7-+iozxas=`ekU<CH-Qw|EN>r7%EiyXx-$WW;E7Qg4ar&I^
zV4&D;9@SbfK?WTo$=#c@Xucg`dQ0!4WIFSiWoCbwTe8%u^~eBu_E+n<liPt7S61vo
zJUCO~Es$5K;M01(MU0OTlL3rTTS>`5%+gs%_T2R01fUI7$RYCI=fkd9J*sXMZIX6P
z%IE!EiN&S@`g|?4_=W5cI&_@;Fr>=r)HQcS$~SO|jXcX?Z<{`Np<qHqvK@$Hh-Qta
z6;^^5a0Di;bbq4J^v5yFv6|5>tK%Zq{%RaAQfeRPwMZ%Z+t>DNBh(cthcjl^Z!QQf
z7;{!bI)}-(YSG+&_}M?4BT}J;TxES@f62cySddvlQ_uxLf#UWbaU$US&o}|Hazi_B
zH;5en=_iwS7ZcyRY;t{7FYwP_JyNPalP?M2#b*a2H?0)|Wo!#)<cqB_OP@yrnq37*
z|KT$ZuFq7pDR4gR(rN%)1%CeykNpYvehdbROkMu%^dDLOPy89V{}pH@#Q%VvtLX`t
z{s-#(5%~WEoU1_Tze2$OH&dp3m`P1M7=EzfkoFHr{ISqe>J0~Z`wP}@eGwf1oFw@J
z9DhJ=lH6^*>EEK>a}5Kxj!4L72d08mHvu60(}^$`wa%M7;nmQak6HT*vMyru)V$73
zTS4$`_uP|pgWz>kp9>8FfB9?Tr)AvA7^xaoVs*QIG5y1^_wco#>N~|juzAKR@c#p@
z7u~7%1=t5tS5+mx>DP#oUwe$a1@uBav#+ijYxk?=-%}sK34nUeST$hSD@RiP3>S%?
z-Z*K1K$L4<WLuicuQ>ifUj}^x$aN2d(KkUk@+;^+w<mxKw6qIKogES3E7ivzA=b##
zy=o6A5~fz?UG9hewVe2$vYCyCoUWZe^hokw_TPJ8M)F}3kmCai{zU@vrd4;x)?dvX
zQUd$~RQQkIuVT}G3TMTTQ~oKJ70_n+50R|8!#}05V!p=z5X7p5EBr&Dt4{Fuy|`*O
zeLbduczr}VoG$M&(?gjGYbjXJ^_?HIMGlcTIVwbL=B?H*tQ#lrSBbBg9;%e<a~kc$
zDbC&2FJa$i_4k~^a?VPJx631kYOpSR6zW62yB)Y{2kSniU;T@oo|m&#FSO__{<&IR
zZsfyF|J-GDlMAnh;<8*o3*#8@IJ4c!4kVLuz3D5-_4Bpe$%ms4o)`4AAeMF^*Ibno
zKK1vVzhISRJ+U*PZpl9!z4wlRarjeJ;Cz+uKZS<LGHXW3;TsBWK_;z|OT_%uHp?=A
zAot*3l;fk3m4baZL~-;i)~P8Gd;0J)^t_<JmmfM5hAm7{&psN2gnko(J`Wkoi<215
zMImPwN*{#`BqT;{6s`LFT#$%ZXrMK*O`hRKYpJo%r68ZF8VfWOe?Y+^^xZ(TxVgPk
zIHRt-2<}ctzTVSc&E8b)96@`y2JnPxZr_(3ZfRi%r->6r0|`y2#v+Et-Lb+Rutlf!
z(dv21;Gjv}j#JCWl_BlEiqVU#2|Y7dL(AuG57apOV7q92-N3y=da*Vz&%Ucfzg#0E
z<_qBxO>O%z%l7ZE4w{DY+Kd}VL8~@lm}Mo~T=D@&rfTr&>D!O{Z?}wujB-A1Uu2z&
zWRORuqHzuS_n@(eBx9W!XFbA5)vXZA+;e62R#?68M)RHkNrQ;HFI5Z57Qf1nJy31#
z=`krXp_7%<mEjdi;TOM_nL5vuf0zkrarZ^QoMj6=cj2TQIxfP*W5nJ&?~GhA{~LM3
zjdPD-H{72X7<Z~aLa=(<TQ6o#iR~)SxI~|i+2KlA#Q(M)h&Ha$w&J8AyK#0xlbKIi
zutHBWZ%1+ycW{<wI8*r9f@RhIU3|p)l0i;*v$JHmjB0N*4eVEkM`&Gua<8>*-jAT|
zcBhonEBoZXg(hr5=5h~#MLGMCIWwGRo~RN4g)fx4-R5&=k0k`vI>+iQ0M<;Iwu1>B
zJ@C{0V%19w;_KNH<&v#?1K!VS!dp`CoU7RkYuoDj*>^|mzyg@$oWTbQ-+`Dp=PK^^
z@R6-~CS9%!e>5eNspxoine*D{Mxg8)>pDXomF;_TZ;TlDCAy&?A0m4>n_EQ5>Jxp#
zAojt`c7)vX1KK)LHCsqHe<gQQDb;%8r)Qr!J~dpE@u%Y4WlvLsD6quAB<(6zC}QDG
zWkI_~n*H5k`shQDRRN;?_^QzKF7FSs)_*D2u5|%IExq(*sNreG1)H7KKqt{C!+xU1
zIR_0Jvl0viG{mjPq_N5vHV9Vwa`;8T(mof841B$ydy53${s^;HOWL*~p)bRIm!sET
zwa5<*egNvi<!sr~237ybtg7JPp23F6J+8NyI#{zCKs3%s#+lSCH?n;19IZh=Z+Q9E
zJxLK$h_r3#8%sQHd(^(}S&$}Tvn+0TSZg<iRI3<WQBSCqvWXGlAkr6aws>XNwNkov
zc(Ya9O&RF`y>$O~m!I?2i{<;&myV9~XxRFA&V|Z+*>Y2pguk-tMn`2X8Q?M>an?kt
z>J(OJ*Gy||!ps-;6|`d#nwC-i+p6f4ScCXNe@=uW<;^s)ON(A8oeS(v@h6;J%}+;b
z>m>~)`thmWG@z1%>8aIK9bW6RN0+Q@^%Y1>BSzDpyPL-SQ>iRpF&n$b-z~-ayNG3l
zQ-TR?(+~W%keY)bOoixcr2Pwy?qQSYckZcS=8};1g)+>s*5erKU-PQ#wE)%V9&E$d
z+#nV`)TaS^;J$tr{<vg79^%yxUAlZJWP-<0>1jNP;8V<pJ&xb<yGKoGBYe80Byos(
zJ*AWVN%qursJ903i~{xCt*LGla+i8HbC)dQK~2#g{Mu?lG<;?fM(Jv5DewktZ%8Hj
zNf~kGZJl35kLAJxNs7GYX3H1b7x5=nO$h0;&D--t-qeX2QKQ6IZ9$w_M)@r<&CRmx
z6L!L-K6CalVpy;@meePSeVIcGoa7D;!0Vi2>tyS0EARNp;H~E8ox5XFCW1!4XkaR=
zIb4mN@AvC7glNIVF95{wp14)~i$rkL;qZ@I;^D)Is-;)P<|Y?rnjKK`MDT#f*6@hN
z3a1#cD=`nPv$}vO54BfDgr{h^p};H*txQ|GG~k6QYAGYu9XE9xbN=swMI;Kkkb&?r
z*oYftQe(^fyxQvh`XVGuvP7-dV`6gVemd>tv@RTBBpe|mtC+L1WE-|s^1#4bVzQ#d
zkW%zvsM@W>W9||L)>+((SS0lAlXpj?1L)r6m<x3mW2dlPLB6GbaFZ+P=|#ukXf?t~
zxmNrI%oe0BJomyHpA)*HKkdfz<OO~NCBZYvApYrqzRpG8XHNXZGVP20k_Pwup3yKh
zLb7FEC|j86tt#}y5}Y#Dt^68EtG%tf^=|_0`ywO@m$A%j5KCn%Iy-KJK1{49&GoOE
z%P*DrDmoK}O~r_X`%b_V>V;(Aq3%}^_7t|`L6{pk3qQ91D|iHO9$=3u(P}&P#~zOE
zahAoIaZoG9Sokxr{Ojl+W5hT<DP`Cq;R}K@&?3(AC>YF{y;0f`K>H8tNs19e$;TH@
z7@mte9DUfaDz-uYsNLs{YOAjG3)TBWoFhO5C$ujU=jXMn3^;@F(r(19DgyjyC1d_l
zu&-jr*pX@TYiYOyPESmVif4fQnU4tZXN+%00*=|?r&mMLKhdsXmZNN;X^k7;I!M^G
zsmpBY=j4r;JJZ^KtR-rjirSNHAO97!{#60}D{M_WL^fuwd1m4NVBOLGA21xytO07B
z|ILBs|JOzK?)}$n_-n25K?R`suj#Kl`t8d<p#Q1wZV64lMVy*FYGRIDlAI8|TI|a~
zoq4*2bCuAHc@0o&t7+wzVrAb>-1!>iG%mawM&MK1=QoO9p?qEY@16@gj{or-zi1nW
z>dB5ucnbS2VD_8p-;pXvOjvgp?Liw4SBBy5IhdO=Mx0L7F+XmDLFh$p8vSP#b5mAm
z2l6Snc~--j%CZuC=@dQpk3!4!YDA5pyZE$9+DzG%h-X9iZemsX-Xj-y%CdSk^l{v>
zR_UmYv;VdE6Q-2m(l3<UupEA)1b+@PO7U4;-W**HC=reO&Gfsjf=+C;s2FTC`DA_h
zNoI#*|ACq{QP49^zg0KWF9t8c2ll_7M)L@Y@LcEPT`bP4_pMZZYpLR1SgLh{fH>e9
zNxCe1l=(KG6LiqMc%Rpa$X(YmU$mO+*o9MhjpF)d85rjdlTD~x(mm^Am)|m`KfQtQ
zG1=Bxt1knT`<kCGTe-}a*;Q6yvXs3vB9$dAo>~R(XcnJ;{<fp`_TrqX+DtT6)jfU~
zK>VA;QF<ngyys$SOjaxGcKUR(1!YZ#{c?~!Nc<zfcmH4nT-%pS0w>Hx2W2EJ1}I7g
zR5moXe^2AeK7h8MHEh-$zh3iN(loXoiNczc-GLLSYM;$zUi{KK>T%=E9aDqTx-3uj
zPCXR{=OxozEaw}X+R>j_n%8AtOKUS(<+%O#^2Kh#w*E40;!?xjiK*E0$fpeX3Fimj
z<T=-0J}whW@m{itxSKbw$*cMNCJzfO`b_WE4eqqpM*MBjiSUQbb~}XoR1V+^_N%ND
zZd$&lLM+A2xsNdxh~*B*?f{Lub)6fJ{?akS*z44zaak{DTk>bp7?}3&!OG3zY+M^M
zY1l3yQ5^1fWTs^UUvu9fEV#}Yvi<?YoxxnO0)P)$?WIL3kIFA*hU}k~|DM@(xsiT+
zKy*6{N6(v{EpIjwhByhE=*qe9Qz+Bq+`g(}1lH3>W)bM8{n^EP@nFu6aV-sab^3Uw
z97nS+;}g6-v&;E_p)Ua%G*Ny%i$yKgJY2(d{oAa&;%V${ql4SBZredHSr2BUO}eA;
zo=?$iCBnJJ{#-{c%4VWp>jYqP!^N0(aPa1s6y0qU=?_17ryor}{D@M5i1i>;%z!7`
z2X;|>h+BLnr^d2PRFBzw-ys(AksGb2au|@JV(I82CeXQGj6$n*hAmKwf4k^^=QO3k
zBA>kTMNd6`cPb!`d*51RQ-@ET<jKkoy&YrlGNk#vQm{lRP^`F%o#`JWjHNwT_HxfP
z6a`>l$~LZfv6xRTJ;2i!EEOqkl{6R-{cZhumk@|DtTjpM5;Z#IxF)N|-A=c9@2#BZ
zdmrxI%DF%h<*$8lH-ewy8n%pW@?n!dt_cuxu`*rp%6q1s+jTc7<Q}_~Lyzx6EXshA
zhB$HSI=7*6HpYm#MtnP2w?Zkr|B%7`fX))LQ~_9=P=`Ak&&_fTZ+gxE+4#IYXmB&S
zbfwUD^?ftx2T&v#S*7zDG?%v8-J<taMgw(NtSk`Vdjd{eMXkV=FJ_g5CDys(?vTd^
zY2bsPNaQPTuN6`uBkLH40uVDJSG~9&uy)5Q->f~f!~^=yOmSevtFPYfr2(AIQ4K4)
zAHFHo^Q83Qwgu5mpvSr2GaXh4jFGGFc`GGf{X+-T9OJ%O`OGO3JPPR7a-+#zE40T^
z4R;oHbyHpJEI;(+pW9gj8k27<ez5xLu4jbR{S<$uMT8&f^wK$>IxVvebPD{7Jbvt_
zjyFO%^3ZnhMk*j+5Qx3$f4L|+CVy@F{P!(A?o~;c)&R~=nEj@)3UI#sd+9}9<s4#4
z0d(MMa`;b5=un!{U`heq^VOX9pLCho!i~kRwB7%kNt}B2oYoNx^y^ns&5C*bJ8@o<
zSlGiB0ooI*m010M(8`Aaq#+Gx4g9F3{)2KKU;N@XrVaG(R{_+2vJtgJ&KAT*pz;6T
zg>{{fHw~N^_`ff#^xE)+sp@RMkk@-`!Hg*$XBw(AtYC5R(eJM|C3>Tqf`Bfuf8|C>
z+s7>szzxC$A&40a#n*9f@R#d};ubpBuYdpgvfJXu@jnc5S<W-;(kteQYy3?s>28DD
zC?E9(({RVkKh9SS0=;-qdeu3KNuCNlMm6HwKwg}*1NX8n#$w`@0Y1b!9hT9-W~|UO
zqIsV9nh!BzZ1%_;Wu+zqTnm||)^zj|d&RQ<rV7){*HW@Dxw8+7G1;qJigW1j{qo$3
zR@T|1w!6dz8h5JuSxJkZ;p$$t8;HmdaUJB=Z`ezlSKVn@E>G+!_g{uBxt7-Yo4RIM
z)^D{6T|Md(?(F5F=M4Wov&Z)!JxW%OzOq^AfP2QBl8wgW+afq6Kurfk2*1=~NjbCJ
zX_-_xXn3DzN%rsT@e=-AH{pinEe<3LAGIVn+JeKgyU{9h?S$p_L@1{Y>iFe)tJF6S
zxPL{sxq9gB&a3`~Re1J9%dytIoxk0vopYlSwsR8fsi%t>LvQUlSq~Z#d{Ea`%);=o
zm;sb_83SG|ApqCc*c6k(80Ewr9QHm*j2Jl~1Anv2W10U^XYM_BmX^pEWrJL=4j`?%
zgO6H#YqXD((8kZ+M=Rl%hgDTq&H%(Tj;Q1;G>$4G=j~AN>l_A_EODA$74D146U(3Q
zjZ2n4OJ6SPVWlY-_@dz1dlhJJRET@B7@|K+WxK%`J?lLy?&?2e8<Gd|UlBxw#Hqkj
zRhomhPpI6b>6*>L%UcBk{LfpGgOB=sYaDwgsV^d>RlB&9%U>Gyt}+tZPnbPIc)HhY
zVsPN5aW0%l!fZE~kf!rDb4&?L({}zeJKfEZN8n=lL7E9|x|){wY&!1{&=f2$0#b$U
zy3Ml8c}Ov=ZlweQVk_<7b^{EcoNaYVkva@d3r_KKklWo{eyq-jecS6p+YOIugdfDy
zew>+cUflmSN>uOp^JzRs0?c^^4sZ5G7nSXMI)yW>X){YdX4D-}Cbw>eIJi(kf1%BL
zXhE58p1gj&&&=1dugb2Qd%s8S2*#P+1DxSzarkBBRSrB<H?9kpcHjy{=RRS(x1%x;
zmZt8(iF^#Arc5j~3`w5zpki7PYQClv6Fmg?8hxrgZxQpy>erP@N+hB#ovTUO+L}9Q
zlPy1QAW178J`vQ~vw@^nsT^q^H1(z4QJRx{$U%75+{ctFq}Y^KOEjP;t0hjREt{H5
zT)&(HWCe%E*Y*H~C~#mPK%X~=T25{wyP7(Q&CvU;oi`F~(xQuV>Bxju5p{*J<8M4O
z3$5>j4g^Y})hfw7P`mOsS226{*m;I$y}6#F3r$X!Nj}L)#<Y^lGbeo&sph7DqfzML
zJS-z|rk+&WuRa<=!(c-zJ)`i*=V<bWrA!tvB|>|J1X*#IoAAxDFz9*a>8c5x>#a|`
z<BCk>TS?1!AH~(}EPb;w*BqYo&>7stpXenbSpgQ>j9Qc|kvn=Y`^3lJ2CuM}6r;Dl
zYe^MNyFNQ44-UN)%B4eT3g~buT>_0@cuRtLgJAq(jSQ!zCDDbMKji)U-Xrlnj5nE(
z?qTyzT*CG4p_fSC(mjP(PgCl_7W9$sGu{N<;`UQ`-=;Uo4P}_l2kzo0C?o1=lHLRw
zSY4$&44K<@H+LQ6o2YZ&kNpnm@Uz(YDCH1hc)$3AUAgke>{9hzG*(%DOm39gJnDni
zXk$KFMlTj)MiW7qH8jSK6w?1x>_LdC*&D)6Yu(R;$B)7$O56EwvRerqy8I=gXtG82
zB2V|KO|t$83M#V;lqF{Hc4RaTITj~Dn^mT5b0KP6qzD93*X2zFBXvI2E~~pgDw|(S
zFzGC8YInawODgVyd3YT!V0i8(D|kntd?!3yDDI9eeWtR7{TX!IdcgW(#2n43Qj*Bj
z>@ddICzkb%7lEObrG`*&gz%)t%=ON}@2TmHh4aqp^`2QuD+Iwxq%q0TzlqW}Y5E#6
zBXsdgD6X+%7k|z17@^VP8bry8i~B;hR1ZEvvd4oT1@(E_O@FDMBM$MwiuIN`9C>#m
zGU3}2q!%iga_24o86))bvz+;f5?zL;>OSghSBW0Lgoj>8--}wF5o$lHH+%Q<()T!U
zW6v`W;((zHyJKLsT@||g*^m2qU<W$HX!c~!D2Kr3Nb(s#G-%;*ImscqWGuy}Ox;~*
zkEk`*hFrz9iAhn9;=_exA>x(W9nG_@-<fsKDpOM~FUw}l;Wn9duj_adVMCwN)-2Pi
z$}*;SuI8faJdJrHF!)JB=WFZiI6MP3D$I%_KWAVdFc<HM4zegV*os5Iw5@PHf8;3}
zc%v52Jn8w|q9m@^eEMY5i~P(fjrIf5Gx5`@n_jBEZRRF*MN(#O19rVRbXg&l51D18
zwi-(-FFm+PnrFo4Gxi<4-&!?WsX{wrY{Y~^0^IIRU%UG~GmHu+!*98G@JIZ3S7*{u
zOX-PVZYS6o50RVSU9p6p=HQ1Zm-OaKEtGjv(>&iAKN|jk3(<<E_{=1811ZSakB!h_
zjUAW1wtMhNS6G?c8F*hsdwCyTWjRAJhdDk=?@MLVpFV}5GjFr9_u6ocLc~3_R|A#o
zM}9Gl6Hx!5{zCT~cwZH>@iyi2a9yzog8My)a@>k?OwK#)&&Us80<VdKHWi(%*(Fs6
z$Tg$g@W4U<BalRIM6k&yE;)Bzmo}$6-B%yrF*|3F!(*19jHQD!huTup4kf2Yj0!G6
zH<c#jr0da)j3Kqv^f?Q061kxm0M5Xn#9}Kwz&TX)fMbSAyk#OkVx|?f+?q&k965Qm
ziHHBXUez8MKY`*;^AXoucGxNDu}R4)Lh=sAK#XG=7*iMXKb&mxnmvCKZ|rIH$hfkk
zU8_+gZMlp=&-VrQO<Pi2-5a{*CXglmkR)}_ZPrGA8Ut22x@+EsC@n*@SUw$|y*TGj
z<%$)cWt^Xra=mdMRpZ<kq(1`oieJ-%*3g}1=E_{2_l+HY@!}Ey|7)wS<A$K7$nW1o
zQHw`%zDLV9`i*|fURb=b*qOts1*g*ZGN~TMOihb>IyV4!|3<b}_6cnB0CReWrKH*2
zF(~F)RBgz7IaJCem_JY8krQ*-rMV7`bJ^2QP7yOLi3?5Bh{Sf42A?HNrUi=K+3sBn
zT_~0`u<qWO|6yH$*KEfXOA;11veZZ#4y9MU_Rf4ere#TZHe>C@*MJyv%#t!6G<gR8
zsdvj5G4I#785ogt)+TA##}SIX;>hlYcc%NbuXN6eyB8s!n6GMmM`1uOBjD9&n11FS
zpqApBJ++!e9|Z4PA{<1A^t_g4^~vje+cZt94;ZEHIWI_RuvIE0yxHcKz|WN7w}9bx
zAlV0tN&6ua9}9l_c5;52&zp8UOY6_~OC;TVE3Y?`aOH*}LH7(g&JQ`4mA*bDxQXtZ
z=-d#b+|TJ>P(dyPqk@-0_XC#{M!)?2dJC^EVp~i~*Ukb)bjd^}?TS*z%9hu36v3RO
z9b4qYZBR~65e#gBr@<ADS$Y=EyMQ?KHb<LPSfczBKr}DWFk+urqD3~zL=xJp>I#Wm
z&cQLVaIhmU`yQ`!U4X)pV3O&Q@tLs>5|aWq51JCZm<%9{Y^4bg)FTB_g8?2o`#Lq<
z1bKQ|EObCTH>6FW<k2j0DR><FMn~nHNqJ#^fkpfq6ebEWVmL9kPmr`oOeT$ayeri6
zD^>u6Fp}jrb#ISf<Mo>0>HcBLW#EwCs?J-cmp@J`3CR#<_7G~8vB>zc{!RIQ*(C4r
zQN4xwVTv1*Jx*>KHxd(1KH&3}qE_`JQz!7Db%mo~%`vm|P2rt|`#)dNF6%iy(O2p&
zPvELQ@Cq}xp?sE-WOF+`vP?f=99ALbBR&Li%ns<9VC>pRXLlf)`sEtc$8f8>{%RiR
z6iRD)9H{JMA6KauJvQAZy#2N=+V^Ob8ejohi)2Ey*Nws6)X{S;2LUA}9s)mSV<pbz
z>?gH8;8l0|xeVt%j}VtV8G5gMlS-?&W!l-Q*<@|G#^U|}qHblt0yS853P-PMsl@i<
zp7YrB^z}(AV9as(rOfinfweM=F9rYr!>?He&WE+%<~h+s=CCLZHc=HhlbA^HBOtC%
zH!ZNqvu)cAmVYyx_Vo-`Mz0D&?ap;;uTZ-1dkSxTS72|FqK&&U@y&JiP7VrTPTg7I
zTTeGn@_BV~1d3?*#QK{onz2m`9YanFa4<Q^%PvW;Y>fWlv$LHF&i8Nb@G&t@HK<v>
zmhYVIRSEA>A-)86a7Xf^P|DZFpq=yPqRPM@GZ=yze{>kmHO-MD(Y(K%me15$<ezD~
z9y7NkT5U+iNb^C8#1i{(YoKumL^p8kJY8q;YgFP!=@8e8+ezMeE&=mVR$x|5vNCVT
zf_HhB^7d;2r)dTSE!&hc$H*i7!8AnsC<ktlu*F{)wxPi$ElwrvR`Ic<GKU<$fX?W$
zc6UfR<B8=0*~vntg!6SM&B8?w@2O;?p)js>X=czELg7B!RPiAb;3}~15I`nuS!qcz
zO6d!4B!ve8w9d-dFy~(fLJN`oaOe7qSttL74sLi)MB+R05-i3R7hs$Bu-4>1{IovM
zst50ABoA`v=pa0zlSvbf7PVjVg5$5v7=p!5)3ge5iFtk+9sA=Y4hH6b|9&%cr>yr7
zO7`-d2<O&lvNiWn5@u;OozS@WDIH(Cx0X(oWy7C`QMu&=cKLAJlZnyDvC;T3j~k0?
z;`Ll|ONa~foNeG!z9%#<{MbE!k8s0FGiSYC?GRn2)V2Yt-%gk+uDP2#Sr^)Phhynp
zixGbiLaV(vT|5I*1<KK^N24e!==+O;7x0YIfI3fnKAUyH5X#?(hasVhp(FO_tr+Bo
zUjpIfWubH3?v)PXRyKkpNfmBjM7YyLi|o{U1T=o+9gC)@HAXgw;thY1JY74MC$r{>
zUkvc~M<|z=3<dU1<(p#%Pn?w9sR%U{l--m0J+sWuVtku&{Iv^))50W!I0+qJ+<5UY
z1<E{enz{R-V&vEya|ubc>I^y$E0Ys-mfHb)^BbgnE{cf@lzOp{y|*`{%yapw&+X46
zqUY6l=OD)^Q=+*T*cUbAo)rgoshT4M%AQ`TH+3#!yVCT$#^`a~%Fh^emjv-rf2xyW
z5bs5itu3Iq(>KfZed|*SD3k1fY-LzTHGAjj3+)L@#8PGR{T`Bt1J%Uwjx}uu4zk0K
z+#CxKaGe79G&_g))!K%&JN>`DxG_wX-OUviYm%ea9aD+CW}OR9hF}?*#87iOJ+yl$
zuf1^pS=r{E!GQ|x`nHprUfS1dXKb@>La5E<4Eyt)SNCJumN6?Vj1@bF@4nr@Y(lgy
zvp6Amb?IlH!r7w>)K<jOU+CI-vuyE1O9$bx{ivs6bVk{gBzR(%dIGETDZ$I@`s*n%
z(#<$z|4qJ+2ufZMIJCdG5unzN9Pzq#*Kp$<Rxhwx6!&5o$c<gG=hVMYI=RoPY#>aa
z<ZezB+V8;po=dtkon0ch1{4=V?t<IjFie!%tsK$Zk_?oZ?5aJd!v9J5<(}9qI}89L
zlbW0HJB*ZACtQ7*m2Q$IEaM<kW<VeaAR9T~jgoN#ap~SvQ0AR~P>#rthokQS(-&^J
zC6m$Myp87f<IwcKr>T}7T@X|t0vzT4h-D-0ib<LM7gprPzwlna4EX<;iTaNe?Ej(d
zR*B}nv1$Js2}|VMnD!s7;r~QnKkk(V*xv9%K$Gvxe<Ra7CNH<0+{QS;Ip|@sV(R~6
zUOf}4AzSG5gB1O@)ID%7rvjxbAHDsYXQ{fFK`e3mX;{wRS#jszU#{oPKN7Rtw{io>
zcC|y%`5xfgo`rN3NZ((UdUyut=gn>9?~AtHxY{yUA1QmD`RJ-Nf3a0Hhuu{ohdQec
zcHgc`-dd-<WD^&iQm}TFvtI)mp_fCs6UU)T@9ZS7{bBThP=FH{cz=TE6tVbP#wB8@
z9V$ZD=nq`wY9luuV++U*jX51$Z7SFafaZ#fM3h4!#!aCPja-HwR6<`3IONeT87CsO
zxkYTB{OWD8A4hs4LuZXr09<(N%NDn#jyS#3$JRxCU%el;dg%p}A&=00Zz!r;8}jwW
zMhQumkZ)U!#6GmUu%5Oc7N}UIdt>ejvHK){9dW6qL0>xJ`^y#vmWB_R3djw&mOUM?
zQa_1E>pWM=I4`KL^6TN`xq{%3v25wE*48TLhE!m5(d8GSN>k4VlZG8HKR>dresSfd
z+s`w{?-L1ohE|B=2-*Sy`c%-ZesLgC5;!6V4vyKk=htO~dWLJn4^KWLy5JG94F)NR
zf88(xy@&{G+SB~XS@!LZD|0_iFuUs2#^<#ND|^+}|C?V**M{U72`gXy0wBsnjizx(
zq70WL<GgZL&3XB|DF6FF3xF0^_4M@nj34g<;UZ8+qAxCUfC5qy%%1}jLMW7e%;I|w
zy=Tj1&QqK9p+7I@4IcUtsDq3_;v4pXB11a{V(`jm2qFtr4O0$I!Blvn1#&hmQ4}#|
zn7BaTa}xn_RW}iUt};#%2FR&Vr0+}?VJRz-#%ZECE|2H(#s|=RHd+$<#lL;v#f`P?
zL`l-k&DAPDt`Yvy>j#0}M>Q#*nV#rCH*;S};;R=ITLGHuHF)`T3&4u_DXY{ErM8H2
zJ4BU*WC`De2-P9e;j{*&=m=(kjB^BUE*JVGuoIFPKHZcK{Jk}epA-Cf6x&sLz~C)#
zXG}5|uyzR<-uJll^~tMsWjbMbxZU8bWB=@f7C%1APnGvol;?WA+s010s(vK1<%0&T
zF~W39j2LtHpZ6h;irNBKf!<8g2&n6WcQjOl_M&DaQ8lZq&I+N^2oU2g5(X<w^l<im
zHfAT+CuOB2ah+a?RiW)RlwF7Tm&=pyqa;+ACk$#+^44!@Ix-Zy$JZXoi$mJR<JXVO
zqfy+aVzVqP0TC+ycC~<Zyarrs!z$(U#1&Zz>dXA&lI5GAK|W%)?Bz(%F=eM~0)|4n
zMD-TGKHcd*njNeH<_t~)q>4l?z*R_sm(sxXO3_Y%B%^lXdJ>j6mc0)^+Y7Ttf*Dbk
z3O65%{*dUOH$UA?AzuY$xoZpUfg3Es^#H5<fVQu|FKuZ>|1B1LriH4IEPt4=oDL8;
z%q%y6n`N4d0cddU@Xg1n|6wp|l0od5#N9z2Z(&#}Mk%6k9OCC7VGzXVtBauA?my-h
zeA9kww1FV|A2&yjVY=2OiG2c}5^p+jT4=AsSO-z49;mJiWrqe5RbBcI<>w0J%T#&g
z3auIX`gbk5`IK(jD_e+)lne8SLu6O=FXaVCJ__y+1#IF_y3^C;NinXESd(`&w)tng
z`s?=VA2LwO6T4Zn8XW$*f3nlj>;Ee4%LAcK|9~f5qRo`ht?8f?I>ib#6mpd_p{dlS
zqS=Zj=cuTVBXUGchiDMF@0n^7RyoQorj;VsxHZi2e!epdO26Iref?E@zt8u1KKK0y
zXbOIle4rhN?hDDxJ7q3df%k<6AO5cgt1~GM=7QNhuY@;x9NY3EHbcT+EX0n?wShKZ
z>)xrovl9@&31)XB=5FwKu;dp&-cj-DhPYxJ)c*};FUD?}^{dB{u|+Sh&xML$#P=)_
zxFzp_5IVzIXk1vG9B(U`{PBuwnwEkUM5uN_bGteAC^$otM5-Cm`F2}&=hd|y10v;s
z!Z*!4JSZXAd-*Lqg*j*S&miT>TPRwcfTkZI&)Qs&QGjFa6T&RH%@?Nu$xo#I^ax<j
z2#*%RNZ?$0Q_vTi99s_wzC1S3STfBXkgBVbd@=x#h;u>0)l2;ScnM>6HC_39$TJ|f
zJI4nrn9|QzPTTm@x5<O+&*su8P=E^pO9zYLa<Djd6{xKOP+uCDS#=X?Wa3zsg4y3<
zBK%}LK03BWF2)8#PIIb*f-evM{~ttK3@evE@ctC0WoMEjh_(4|xXs##<|(Q*6jj7(
zaZq}Oo&iX_&0J7v_~Ru$DRR3O-uz63h=%3^At(l0G+2W(z)aTP{J)CI+PZw%gfkb%
zc4xOnA;Q(g7}MgX-2DK3t#gcE3U%ZHJMVOcbOxZHSAhS$6P^L;Hy3zfi@t>7+~6fh
z^~R#FC5*i^(TZcuM<s-?fEE0(f^(o81Zj7vlm7>MUFM%WK6B3VMBgr$+N@r{2f_xA
zW1D})f`Kv~opq_FfeinVWFO~~;Y4@LWLh+{a;O4YELCa+6!K_sqCapNg5<Z{PY5SI
zRZb&)fHX&kx|AQGQy`><7(4nvWIret{)vCv<na-d<YMeW9^?GH<N6-Y0E_e4r!EXu
znE@Mr8UyfnKO=v?a{zWPe*rr?IOvjJy|~{}kO8SJP&avP`{beO>gBIQqHh+u4|40r
zL3YA|2Y<ETf8EJ=8_+mYrychUm^pXB3KPNXq55+YnT?m1d!kRaj?oIgHHc483Tf!R
zC-SamfVT&$ErL7)M!R$Ij3us{1Fmz~wdgBi{pY{v47$I5hVBw~KB#VpVbf|CN?>7Z
z>uF%_h@Y9e9_WATG=c#}4lm&?&h0H4r>bcl-&OFDX?<F@`$}O%2uF{ldip@3kM6EL
z%gNXxrPlkYpxa#stALCIrwX^2kcgwb#Lbk8oU31kTo~QTp&WgcM7!=Q9aM|mSKdiW
zh*%D)`Hxt2JKX8|t$YH~<MJYl{3V0r`sM67_RB7bJOxHpMs|ml8r8Bi6i*{m(k~|Y
z*y7z`Uh=W|Tvp<X5WTG;jV_HBod>9{^!HCUdDvV`<UZ+w;ab&5ca5ePcEJk8J(yYM
zTw>-cf?ZV~M<kDiFdb`lh+%hRSMSS!V}g@nCLy9Sa1aDrh{btOv^;Ub?bAXK5#e0F
z7%O;2s2q^dP35ppVNTMU@{6%W57nW3d9ullkjK|mLYQ@$?h3G-p$5DGh^zeJqoDcz
zo+2!>7FCF>tP)w)I83LgF-GHzRp=qR*rXCt2a#1axPA9P>oShxs*%CWVI1RSK_h3-
zETp7<I7h;>u+g~=zj@f#ZPRU*u60Mk5i_fSXFHA@Q|g&Zx7g324m6yrK9hMqRW&y8
znUijJ{(wpezAn3!#W};ESALFrH0s7Fz+1_E{)9W4Zt2J7L{i<{`g&FE{F8|KWYh%r
z@lU^Lq3oQe@>ZP8Vp{B@pbMe~Zynyp4^Y{yzt2-Bxpq^7KG0*o=se+SWks=LV~$}(
zah_}A`M=Gm_BPqwm6tBX>g3AiygNYE&*~mdRwr_@-w3>Zr*_%Y#z?G@aKutu1@~d-
zdrXwF{dA}fzSR_bY#?cvT=+V<*-Jm=GtES=kELNwU-n$P@1+5=xUW>inYj|rB9s(U
z-cZE$`z#*X?kl!AuKLSkf^_ZY+v2X>qn~v`D>%aga)Pq+qeFr>+10O%O++{QXy%j+
zB<J9s^*rWOxvo+(5S7?$i|g20Jyf@Ki_=@9(b%rum20_7fVhf48{i*_fI>r0rN&@y
zeN&UT=L<Dyst<!s1<Tww=I&7XYtHSM%=5LgaMD#|n%T3GNBhu?QjZ(%g}?47vz6Rn
zMIt(DcJD}64$i^eqBv+35YpO`-*?qyW=Cge67jQzRaWKrRel~gePqYBBfGsNGfR7A
z#UA*A@&f10N*t^D{iTSZ;LFuq!8U=b;GE2y*U4pFPn|aDRjs2L+%<{Fd^jNM(Pp6p
z?YA2nODnA}Mw1)K#voR1Jx=@niP&vqQkksmoMuz;wg13C*FMu(AIg|LREyT{AwrXw
z_T;6!mNmtziq97>KzbSxqsOwiHd55-v_|{zvdDuCjdOc7Y7KW54~w3%+F#Z3G1dU{
znwI_k$*1yk#pn9%^;W?!ji7zxpZE-7&SF_xPTST>S6ek4_cQyAFehmgSHAI5G->7m
z!hoBWx<>er!w;V8Rj<;skC$S&Jxai*_E6&wkg&H%&tsbw7zLk7K0SE(j-02r9Qho<
zq&{!wel^-Uwv|c3ZzCTo>G93Ny7e(V{X6~eK;*Bv+FfojR*uT-dkp66m2tmfyaN|e
z<yr5OMHW=8!7`P7#RS_aD$!>H>C%7OQB|T1FdD*h;?7#MzaJPPQIPMoLlcvlFPri%
z5_36WaKY-)`i6{(vK5CYJt<to{RXx*^lJrr1~_+T-vk?Q8aE`-KB4CUv3D5nTwc3!
zEt$sPQc`wT<_Bmbs@)ZnuzlljL->d*tuj4V#sS*rou717pU`h?FtX(g{3E=tV+GA{
zM!gLff-e2<CSW<<r0;`hfG4Rg7DerL;=MOL6&htvki%O>y;7P?J`v4=p&a1I)2$sY
zm%N$|9Zzc{7Y=weUNAJN)Pm_5k1CQ@4dDp!h1=4TSnP_=aV3`0PN%gFe_d9XX*`K>
zCx-MOwwLdn%2s!qtuJ?o%>e;g@AmYxpE9;OTY0Z^UrcwRw}Un9kH-Zbm2{(INJQVP
zJDZlD<I>=kudFd_&CF6Al@4~`V(S+M!Sc1g->EK6O~K!&D9$_^?r{#j5$?S)L>1~^
z#qZ~oZ6|w+T_5Q7uB4B?$%qFEbCA$JYhhH8t7^}X8N4jjpw=6y*6~N=XA2oFm!CV@
z9}ZkjAPy7wT>is(#5lg2GTApnWjVNSoY+<L)Qz5K*4IVprE{a|8{BhTgmpq|PY2jD
z-iVL7zn7C_xBDC+utyl0%u>H(qO(pzo5i2a_-0TrWF0YV(?HBFh)9H5UIbHe{;QwO
zp{(Rb1d-ZV_1*$@^rN+RvA2lI2BDe6!*{UXJk$omC>3k2oiVcPD}3BxVV)NDc;MMi
zc56;k+!}2wdun@s+O5z2Ydz7rE2p=$vbI|9oAcqa00oZVn;)s6Ci3`Qt7)5cXX^I2
zkb9ekGQ?Q;53K>Ujj~7FwNw<`8hrepNe^}@u}ewc7x){)6t(1oN_=KYqDFzKJZAok
zPklm6(2rrgc&~4LLxFM4ua|hDWtY`vKDD{}xI5l{kYGCNjKz{RQJ6L}G6a1A>J7&V
zgv4l74<Q+zwkL2T@_dlC*fo1jTZRk^-?(y1PD0ONrB!)5=QO+#!iend*<*)kc{x`y
z8hy=n8tZPk@(_mXFxx{h0%n)kTz%Q?eem~tHOl+a{(Mk@y{^|2>~_nBj8~}`bk3nZ
zvl<Bz-}H8aY)EwXoL<jGvWM$3>P~xa$bY=fEoz^?r|X|_?>RN6+9@?6;Z?5Z2v;|-
zIF${KV41Xok}p-J?~X%X{Ghi>wwp5A4J1^8V{Zd%aLjX<0*=~>T=6um@`38#NP3$*
z4%-{zNnVFeMdHK0EpOb9yXAEzB7Mspr^J1shtlnG@Zo+Wc9Vp0oXwplXU)L?bGX2B
z3XvcK{CC4N{<i|gitbllcy7jew<!GwgRS8io?`hC>XJ#4<@ZKht8(HV;akfK8&<+Z
zU&!S&w%X2L*(iDBZFxpp<ECY@x@|SzGwecP#@@k)WmUO*ZF3F%pLxrv(^jSCJ#G;t
zF6us5A!1}#U&xa+e9Lp1;KhXbIn_@%dFcc}IUyo0_efFCj0t1(hZPj0T}@m|=Pgy`
zvZe)Xq#EG*N?0DSy?4yyUvMC+{J{Z5c|_9OdFSP3x}Vh6>KYxQb3<gZ-b&SMmfyw=
zzrPuG&nAqVJqlA>_4&R$V!c7&@^?Tigy$eWc6qAS=LS>Mg_j@sq<E$06GYaK!0A(L
z5!73SQo*;hoT`_(p^N=-=tr~BV3qSGl})wEdw@BvyasWy%M;!5mFq`L2`)@y2XprT
zH;laD#igb`4sJM769S<qMl0zT9yMa%7f9gpHhYW_)DH$e5~NGp3lXD<Xgc2;Uw#3j
zL6zGj=h0Tv4&c$B3Z8-)e;$tWj~1BbZMg;Re;K;26MF#e<bMcxm{0X=B<8+2PF4*R
zDhE$+YS<%qs%LZowIZZ_?X(81&&*HHjU}~b+x!eW%N3}bPe!zO3SpFnnEEZ!Zq3rb
z>O8Y=b;Q&_H)8NNqZBt+X@ct2g7`>oCt@msjrL@MQZH*W_w5&60~sKM1hW)T3v>qW
zOjVn#u_i-ANB_L_sV+Z%|BuP&i>$#KqjBn-s;&4qc<AYy1d$UT2!Y6shfeacn0+NM
zTmdP#_8Fy9Nnn!n{u~$NBg+>;GY&OYn$w2|_*j{6^7!*PxR%s(M;>pPMfjJ$*F$E4
zj&&tR_wz$Q{1Jahq;O(8U<V_0#Nt?pjI02b^5A*VQ*538495vNm=ZpjAchNL=IIv2
zX(TQm{VCX@{NW}@EFkVO)Y4CcRCr$KR1wFAS%|y`q&Q?jp8wC)A=vSL8a?Ney74kN
zqaRP39d-QDE*+sRjf|Nv5{I?|;n1L9y$eHir{(o`_r5NGh^(1f-Ta6gYP}Xhei;zx
z-N8H0yS==*Gdx^HT*GP+7GwLkL^}DYi7|>?vbPly!ZaQdESBdIzqc?3Y8i?7n>}_9
zXHTa7sdE;QKw*~R)K5TuMS{}fc?zQec@6cc!bYw%<^lTsO3wf$S7^uzVK%&*>PRbQ
ze;l8NFxWB-y%jiq0|$eig+9T_NW0h5;Y{~e`Wm)2s<q~hVdkm^DUP!aR8qvLu(NYS
z+A}r{x^TV&iT~b_X~!MP!6byb);kajMvs0->m*P|8%u~C84Z`}SNLS`_OZ!hh2o!|
zo?>rBbvT^|4%BDwG#MK}gm^4IK}HR^zuR843$i;^#$wSptS^nT*n|VpT_z<N%4kDs
z=9d_u9T%MKlzMY=oBlaiv#Q~;Rdoc7`Sm2`@w@)?)dx)kzook21d~swhq}gv51BO#
zF<M8kw=&|-*12*;l*JK222}nSTL^MfDE<|N0<PcA#gEPULj<*)zP`fCXSq6ipGfs6
zWLMYXSk~m$bn{rfN353YGb!EA`fGEbFU{VfrY;IY!jH-fD$A!E_{vhd33m0ZBzi;D
zSH*-)T<rEiU=N=k3`SmTQji?9$&JN)vlO!#%HoCzpB8c`-{fXK@%N(rzK~iFbKqm5
zg6Kt~UCI7+mtPZpf1RMxJ5uOI5nGYd`QeI{!)SbW=_+~EqYj>-_s8LBle`wR38$kB
zLMqa(z7AtE|G45|>9wZpJK<V2Vv@E3IFmhxdSB=2TwODG*y6O&*61#$oHWe>LRLbm
zN{1@$N%!+_hJQXif~z5!RSeZxKeKXZ)m*^U8RDMkyHi!G<$cWImuuJwzBwl}Bb1jm
zV{hsCeI7K{)?;=pn5|cJv0ndEZiY1b`TJy2(#6gXaja7%8LURB3|nV8Fs&g3epfu-
z573TGKIHPm{C(0z>_l`#k*S#I2>3kuenz6iQ=$YbVk5oPSny1)LCr>wG&@pFx>d&k
zE*FBLCNktXU%-Zds{W9l(gSj(2W^|r1|5&I?@%Sn*u!wJGZqgH+O0ZQ;xGB(vWivx
z<K{K5YRk-eFT@+AQxetiAkAWAOavivcPYdZ?fMSfjgM2U5uferHn}n1%d-qqUVM6n
zOdIoaP~Dt<AXIsUVZH6ml%dxh%SIWdf7+xP#8;bSQgc=$(MJ;LLUxbAb3x?LfJ5Eb
zR8AxfA#qwu!LiD<Vh6(t%1UFjDxL&9^bGiZ!72*x?<+h%N@ag$W?b90>VDSN_=clU
z0Jb%QnMmE9(FqL=r;u#v%3i4a30ns9YWol4=6;4o{B!_s%cCUP6iQQ1&BcbRN}PZp
zn|-3A-w7EXuQc10_4?yl7u>v?(dF15LA$^CaGEPy_-N0yqh$uqXhI8|^5U9GjnfVE
zKCn|9qP?v%O5<R<*~>~u;GY>;);^hIx9^Bttk$YjcE)9shRMQL@iHDuM)TF?m=GSh
zKmmZo?r+)#zc(j`SEfWr4Vf5r^>&nN6XODohewf|miuRxj<`bU*dy;J-@2nRMoau0
z?^kXG^O!u+Zbr@NoLessct%3wkQcFULxXp)Fh6E8;h>IM3f_z0nk6L?*z;7@Pyyte
z{lS^%hTH>ue>89Wv06E#*E8xT>4l%O9|bdZ3t_&wEt+csd8Y%syi;9g;CG+f;u_F$
z?8`GZ#+N!EhCRw`mVoT5B&$txqsMCHG4(K(3!kFOGvGi_MnHf(#;3VzTK{4gOt9Yc
zD%JwB3cUHbQ@sVg4&Nz;g>*I8AOEUL5sL<LH)GKqFvg;syTQYD-Cch3e;Enzf%6o`
zAN)Y=^aji%bvPxT%{Kxjp<{B|gKx7<!o&FR;whHRGq7x=iG9+LJk^oQhHU3pws3ko
z2zgo$?*v1Y#&U80kGlrtU4tX4z%slj^)Hr5K=-|v33V-b{iP5!+tq0Qp}9Mkew1s*
zBh@*JvE{r3#V?Rp;2E&|eAZRaB4Y&+N*gA!YJ7vp6r8Tf$=4tAz|+BU>c~NEI+ss6
zhk)0ox+h%EX_JYdV~Ism3a@m5A1|BgmH|sNjZ|mK31gsc2*XeDO!2iQXo9Z?xSrD&
zVGNYL)Ii=Bij(f}V}4Uoo)$-H%LNK2lI4)N7^2gtOhCn0xxv&=LH~iQJ3pbtjr*eX
z7BIUg1u%Zis-%rGvsR7*Zl-@0jAf5OU%Njc`B8f8PEQFrH@e3qh=<R8ZpF9%xZB<%
zy6M_sl&!=R14|Iew?A(LRwv|rUJ1^mJiA$bsyFN|1w$dbLS#IUehYX4lF)oFfmiSp
z4inTeAU!ro_s&8J5o*u#nH~W;`T=n1v0Q{k@Og+YZAEC@HznsigS%3M7?xX_)kvU$
zv1xg+c>P24#lL>rwOJU63cWy@X@Low#%$%OGLg4gaS*UjISeEWaTIAl>gTb%-TGdM
zr7%Gq=7jGN6~er@4Ek=|<{llJ1NhS)#<?7&2b1R-d?&>0ID5~48V>>6OyJa%5;+U-
zSX72BLCTa)4fxIGL7I<~iDd1+eDN9I2{h#r`Ggb5J8TTk`vc6V41Y|`VtkE2lnMw*
zpm?5A;WMi+4+G|ymhy+{093}8P)SH|a@eDw3xt1geR$hC2Y3e}hqZYA_m6AwdGE(<
zfrt0xheR`qi(U)h{x891z;Q_N$b=H&i(B4@A%L2g(;f%n{r;s=e#&mbQT{h{WT8{=
zG*$2%@A8xae{4LDGAZbb&@CY)jLC$HQzlUeKeIcN`<xpD<l5tt7I+5aI}Zb2jH6)c
z_#Xi-1*ky^;7f=Fgc1&pOQTgVOaERL^t;Z-oXox0nOq2`Mw*p_sM71K!N~U3a=kGh
zQ+!YmbJFc08Y{#d48H4t;q`@BKOs*v_G_rMpVA*-Qgo$#2!wZx<d6v#sEGA2+wi$7
zMgQXMmW6?uIus*e%rXWiXdOqR@3+94+7wf;JYfE&7glhK@M0|T?>+FwA1{S4v<*<?
zK{4mu7Vu)|F<Rj5H7G%mgWs-_uD}*uSj4+N0O99j;t(XWSZ4yd^|q=wvPuRhqGRH4
zBSe88q~9asNw6yX`Z)_&BNei-$TN$PN8tbP;|@b%-wN=q&pilI#9R?@58ctUr=9^Y
z1pnJW75D<kgSBqRU&n6M^&a2#i<U{_@lcW66G*`3`?54JEz8#<2Yd0RetpF}ALyS(
zl`lm;OfiKSKqi=$ibb*>E7o$a%~g2s<|Xg3#c{@Sap3vQhv8i~Q-G!x@@;otF^Rol
zd8os}W9KDl>BFv3s2C}^Yy5XWz@bn<5|f_fzFX|$vs(8kD!~)kF-Uf3N@kpq^>Bgi
ztYM`-JOasMp9}V(rQ}i70P!$SkVvtSm+PyJ2s*wGiW*qNfM6hiuNN*E4w68fgYELb
zwZ-wmLXTo$_jLI19}mgs6m>4bY}2v?U$xABaRk1eiUv^pC5B}`V=9AN^%*v(aKLib
z@jiiL5NdEYhe{26AQU@s4)b13nDlD7v7qB^5c~jmn@Jx6y$yHg=1qE#i&EXAla>Vx
z_H{yeA@MAx#hE(nj9U*{2*9QA!$TL_$+2udaV)en-z5b?%@^!%IsE~A)V&GWJ#Xcv
zRbKMfpb32sls>qd-x9`11CMPyt!@9HeQu&hz+V;YUPCD0M`E}(v^wdEbMsO5AXvg?
zZ}MQS)waU4bO03u1TQdVMf;;K+7DbN!W3V)ubE4^hXR2{4;Qap4}LTb`r!lgcK@^1
zBO1o>3L${TLaE7+k%7u!=0=YP9$PAA{>dGRf_eZE+!vcc1>3yR5aCX6LA|)2clbmS
z5#~w>RfxPD_0Xts4sE#*xRKm|S6jQ(nxwyoNAt&aT;6+BkA+%-Ojp?=ZLL>NHNQxL
zr}JA57vHVy+QrEY?GQN#I%JD*ux9|$1I*dW)`M7BvT)zo_PH;<!(<6}rKRWIv+av9
zUWtTu&wwjbE16_kWQ^LLoyfkRAjR>`_Nd<YBZ7B0Y;R@+*jm;SEmf5w?!7FyW#!Vg
z>gYxZtm^8D8GUz+yE4htFR_Z_2kJbLA3ic;3o@7Gm>e_O$|wcBt@?CO%fU?A1`m6v
zKMR`;0@K$j!OTbv^#+;mGVqV|cU3zY`yblHim%z^vie&M0ZwjbRhC8OsU12I<mlSH
zE1PI0CYzJt9GTe?ol!+IKS8#@zopeZOm5y+bvmrFD>JS&U1?P+X!@|(I6<Yoi-2xT
zfF`dCN*qIJ!bBSZRL4^p_t#{n<ysM~vEFD3y63_$=R$L?XtRL(KIw0io@C9w(0ESG
zYl3e=L-z{L02&UN8x&$;QvN(oGjL>nP!21`-+cJWTkt}TiKjh(ID3I>WF@vJ^UREO
zG9D|aBVcWv6qBRy#_XVS&W0Nodih|iq9hHvjR+dX1rqDpr3R!ABz*c5l-g)QN<I`@
zl5xbaAV9S+9-7BD8aeBfuSTf$6QodkfJ|u_BTBHOgk0YElDFu$v{+AQwBiRr?S(qF
zCsrEVXEo0KTWwc5vvTE9%JWas-7ronsezM7){8U9tzoV3jj=P)>+&W_CaJNRB6^n3
zpVs5!zPUVAUX?n5341;!(!>nHYwp!%+*?w(Z#FSh-G!juXkpeo&$Wu)d}43ApRa0H
z=^;U~^HHl&*Ic97tHdO0GDh_npYNQtZoG7QOcQKbmfMyX<A%BoeVZto5`(N<-8MbT
z4O(I=Zj)Y&&PyX*i4|I82RG(rHIss5#0Ts$iEbua*F1^KPtFQB?%(ykJIA4lmDsMh
zR0y+>PgRd{$EoUptQy&>^*kD<bw)<eSnoNj%&6e1gVX@Km0avZyFA)TsKdLNJM5bu
zTrY1WWacLMy5p>c;5Vn;e{eSlA{6O3U~9^HAj1LZRm3DWq+qRXA7SU<%wyapotW#v
zKI=w&*B~ReFSSUgtBYgfy8paIjFMSUDJBx<R<pyL-q4MVTF9s}Ze&zIS8G~g9P2B|
zES;J2d8^L28v1)<EOP*aS;GjustmD0CPC)iWo;KKv~aFqKswv;+-*|Wxj3me6u7_S
zvow=DSfkh0A=RZ9G)U<>+S-Nb&8So<%cM!1Z5Gh3TDjDu%hX}?Sy^uwgfe+@q;mp5
zI~wF4(pRV?S4T@_Ds1(qN8OTsLv9__&VM6;%F)eh&~Izdtsc$D{pJ^Rx02%LgMq@B
z68{D<PE)3y5#cRVA&`|%wMQv~JI$4swD1ic_n&j=`-w!zDbxyNDwF#k7pV6x(RvBH
z(zJKa=<Z{Fw_B%&KSYiQGh-!ooy<4Lb=~^N4zrFT{6x=?;A?NC-@uAtrWoNFXJqca
z<;t>fuoUFL98e2{K#Hl;aq~GmNeBaJBC>}gzsp^VGuCr!*sZoRd8pu4CA~vvy-CoJ
zT+pCHLzghdj<#x1=5VVi2M6UUr8~qfG_57e*i?!skF4;oLu>+U1h(k$I=C#OMjtHY
zH*#;|$&#qgFtmy{p@_YgnIWum)^LSRexrubuH!?Z^q5@9di}PJsG#@hjU5z~b?MJ2
z&wA<M(e-wAi492(qp{-vw>hqk>j!^^c85sXz>xByD0F<L=nI@YF7b6-Nb;r<EeWTN
zM%yLN6a`gO>x9d_)LLp}pvD>?1#{TUR1&Kb=YmXXDCwh#oWW?gU4BQ7jT$-6U}=^q
ze!i`C$3>&>vTKvf9(jc;hsQ-ex0~zwLk3o>p-k-MEW|Ae5!*)TCL6f6XOiGHpSkHh
z6;jHC?%kSEvG^wy%IX_#JdjwvJ7Jt9xkHs+mRB<@7wV!wmYq8@L)~YoNGS7hq%wg`
zB|p-<k4yro#*b#;Ka>e8{W0jW=_%@5^2@)*wFj@0YW_y{ZV@4j1>)QVhY+r5bza)L
zijh^Q(ye{^{)IPo<mQ`Y+ewVK{a<=Ve%m%upHzT`R*#>Lf_XMNBR&iTglw7uSB-3=
z9C}348p?v16=Z6djN4&-fR{2<=w?a(Z7sRhF<QfCgG;2G)4b~@Jju?3G}rj4J#Zwe
zT;NTOPTGUs!^OqWy3|?v0pML~yilVbN<JB8Ef;8O84Kp&VtqEhI_6+89`rwdq1j7X
z4r$97`8M#82#qDNFvZD7RA2G<g|76PM7JC(1=r=S{Ao#Cqbb7>w7Y^=)3@Y)XG*Jc
zQ?Sgn&X3ubIMDdipVYE7R*PcPOw|v?3Ra8HiEp@uEwV?DrH@Rj;_vo3w5?+w5j=M9
zP3aG__G7RC2Tdy_uu<P%OCQ}#B__6P3=CR#SyK@2yI21B^Bma?9=gc*wvN}xC@!uQ
z_Us8HeBXq9UPoKQkSpH4p40AS;L^-#^;vyni6it#vDN!>iE@sC8uZl_%Gv{di&Qs@
z<$Jtr&v<~|2L7A-Gf`v|7w<$N`6~O>dW@;FziJc*t>-i&BNcS?VMI*BGc+;|hB33R
zUqUNtJ%hIHh+>6+x42;%S|CAozG>!c&wy`WTiZq=)4#aZ>hr43L?~<t)6`*;Ta1f^
zk!CO4(|rL5aq0NGw5-Kp)J7#1$=+?7`{e9D9{AgP#U~$d#a&ou01UC(v*cVX`<@0(
z;L;GnGWpvyvvk=wwRI-+ITcthaow~(Up1~SS~GbQD$uwIw*@?aknoG@eu8K1Y0uqO
zeIE<e*uAZiFa(7Y%j7IwBpdMOF2NMxnbpbWGai>jD-Pa|!xka;0A*D5-Po0Pe6Zni
zsTzm*IOL%tGL79yBR}gmZiZ~v%_sF|%wvxQ)JqPfUST{L((JZd;hoW{E#lm{jXGRp
z7NspnE_0;y<Z(JSii{3*q?hex9H%+8miW1THF&a0QxF5K7y`D+M6;saMXc48^pLi>
zu~8e9ton?-#I>@P&#LpB*Ctg9{$Pr%opg)B!eLSRm1cZN!hr}c!iQ~}$**ptC`%fa
zdHKvyhXK<eNgI>ZVh%!e*x%4>HF`&0=>&Q|T&NasVU6`HR6}7mn3Q{1(u-J5ma;co
zk~GvLG-!+Zp8@O<00@be>y>|bw#bCw_MJp+RiV#okTCe0Wwo$(+iqfwdiw=8rSQr|
zw5eiG!xvJhlYU~R?{`)&GM_hXdy}LY&ULUENY|X5*u}mwKGWWms`OUk_xA1ubuF21
zTv`S)*UdeF+$FJ5a;XujR`s&kEw`23(9|rxl$962G3kq3l&{lrC=k!Zh>&QtUd2#i
zKxN}<897~Jz3O19(c?t-hvb~xmz65jLNm!3l$Q1K+8Oc$%<Fp7Oto<D?N&oKW#bcB
z5x|l@lI+2jJ-3l7<xcfq6)mkQlN5L!nU>YN#jB~e*MB{2T}9?An?o8ttDQ27wl3<P
zv)A<aiQJ9#x_)+njrKS9H%9&z6W8<Jc%61~v#zD>Q<?h8rH|0>{~#W==or#8TKKxw
zO3I`l6fV;FNMx<_At`phEs)>c>{-&b`ZjtSiRmxub~iv<+Tp0dwn9?wXo@BUi@!4$
z*}-wRKL{FxB%KJi_S0uQ6uh=~vvfwd-g|p9t*Vn&I~%Le@!<)(^qP(0q=u}zqqnFp
z#vY)HJR$sLZ1<NGjCNj`id=nyYH5YLP{o(Xy4ehH(ghz;A;OgKN&_EMsTW{(0=Lug
z{D|w@RgyH>LuR&Qqd|ow7Ev+pB9}UhzVKd@k*CO3vKqcAt9<^k{MUpa;~BRQhYW>!
z=(%7M3=Q~8V$X(at8=$hH0&@Tc<vC}028Hi@*?+DsmRMoRW@2}HH9hs#Ds`SHqViM
zk^E}zZ1j8?-2nx?vw<J~FvRV=lk|Q^FWpH~D?-3qUdA|myV;%9!|b*rOX46g3twuZ
zTP0CO6A1ZVoALu^*X4A&w#L&PzqKD+8w8)#=(WPwd5dOCu{&I(;HvO+Up><7W=Z9i
z#zofM{tzaF72MfPjBw|n^-|?0b=DLPe;R+)?U4)8HYeSt7xdLqloNy9=Ph==UR*e?
ziTzmN8M=8`GZgPzmu#(h3Sv>np;=F)%1-8{{9akK4h~?#%wDF4H!0rPYv$Id7sSaa
zY>(VbZZ&|)#e{sj`WCgq==LyPFk+-%^eeS~)Sx)zwPa1z!~IgnA7zr6k%D$oZ<VCk
zao_rPyNvGWZCp!P{fSs(e6sE}WSd6UntXcRTBGuY>*%A9B`Q6_b+@y#5*O^;HcvEP
z&y>)rv3dA4j`3)pRb2NnvNOA!pff*N&1Gw}oF0B7H)MbsXqA3Tp11SfAIu9@E6<)>
zB;^#LEB&_2vGmdry^xz>-y`A#_C%DSlLymWn}BBCa_<8gI$Xav0fHuQ<_BtO$Ju$h
z4~j4Iw?6li7I%y$1S(pj-;#2NPNuByEp%{rdPh5~ZH6LuK<2wCS4GZ*&6oW>E8JF-
zzYCs|vL|({XC8%`^HusSaigeB%F3~|mOK|2TWd%MR3K_GJV0W+;^@mDa|{B$pLUhp
zt}(XWocQtexcUUO;e|n%nyjczpr4!h_4RCXTA}?rFIw-);N-&ZpK}ld&NrRGH%uSW
z!9{l`Z6O|zDqpvfWvZ*gOr`jL3pbwe`h@@ZZW6|KgAAFRfqH-fxU|sJt{NkOOH6Os
zh-%C^!`Wybl{aH3elEm=6K|9x%3Xlz!4TbcRqEUBf%5<S!e@BlW{}K|`N5SV@qtd1
zf>odWJGcaVcs4IMz-!k>?U~{BiYY-HZp7@mKQz*meCml*r&Z87^Njln+A74S(bc%%
z%6{3QB}7ZJo6GKbyp(ZYJ25j7!Vhqpg<9ymv@atcj@Bq=Bvg9jpSvhq67oTc+$!>4
zzU`8(`#=_VrMknZFeEtqIYBS+RNck;b%V@@fk%iYhGm(x*8c-nNh1x?H!S-HYKHt|
zH|;!554S8y`I4sH&QM>U&W?+Y!vAl)=}MJ{l)2FThbO&l`~@0#F|d0vYLjuU%fIb_
zgRBux9j(f@E;*oAn=4h5Vyk3=#G%y<%Oqvo^2wJSeAN=;)qQne9TVew9$*TgP_w_!
z>nE|_1z}bDzDstjv@ku}Oxu|qa;k*<H*{-Y|I_=Lrv&U6<z7jhlU_d)RM~&?z*&}S
z^RIF4@&w>6=`yg%8G-xv{{;8zi;!J=GGN!t!C?=}p{JL)Z~r%}jTuX*7cYRI*Zz_|
zkB!r3ByuyCEB9g|6lzHru-Rd$@=N}lp%CMT$De*5jH&+;G?}pZQ7_bs{e7ZeRce^2
zb%jvL99O=@nCWEH)u*jO<Fh4S;6PY&EgVR)WR_F6BixdMHT1mku(>AUY^XsyL)`iH
z80yBuXCT>@7aN4JPLAIAaFq6M1b-z(Q&4*@yi=<MT&BipFUHi6hI0Om%c3dm0T2R&
z_qdW$G7$JnxXj6lUYlYlI+R$Pz8U7=MMtR_8+Ifen6%5L4PMeN9sz4F-9Z!}_nSAj
zUCsCp`~i<c0NNmo^QoV<yzd^J)jN5QaZo<A8evHUE@m}cNjUxn5jUCsnd9^ijnY5R
zFZViT6rf({-X4pe2_DjlKQIjch3h2e14TFWT~2Y&&*7clgaU&AJP{!LfLjR80^CSj
zaB`wHC37WNe@{NiG%}5Qkni(SNw&Cur)TcNpQPW0x(+RF4*W^HLOd3JK)|cq+f$~(
zZm^h!nE1z<x$SC{Gjx8Tk#&9)HV{ca{*x-Cu}We~RoSJtr(uAZDEHq|bskBa^?_;$
z9*-FUpwBCI4Nj3vN>_T0X@Pu%netU~qzTau_mzbq*D(L{=r^8DP(~<+SpRQw8QCa=
z{3Yyox`4)h`vFVoNvIi|BvPjl$iQ}gz21kGs|WoI@N`RYefI&^FbzMAa?J-wCM|~~
z#-9~)cnXvbn{)Z<Ow1$Lnv=bVH7!R%WMgI6slQK^nE-1AUCUYgGg*+_oQkSwtadI<
zuz7ke?mHfjsJb5{3Ib43iM?4iGr2mRiqN_LZg$~hJr&2bC_vbH<M44-s^nIS&aGTK
z3%G9!WFveOctU>k<Rl^es);A#y{X&Rj`7qaS@YwTX{v*_14L9uf+u6qF3^d6Q2QAb
zhQ$Bki{}CNa?X|6|95XWpAyhpAZ=o^M20+QwtmoM%qaO0`tJ->gR-K9e)~bm{)AH`
z%ZzeA4q2mdJX0}<ZN)JSMXaz7JIMmK#tf4mpv@tYu85P1uFjl`Z6*!6>efUHtsWD`
zKLE(ZOZR~`GPZM)U5C)*f&<9M4}#jjZH8}_BJu@U>MfdUn9V@FxB<7~?F&He2i&qn
z?Re8*uhE#9jU1QP;(C~lw78;Bh#fjy#J5Aor2su--i|uIDdLb*g4-QotIai8_>a9u
z9?L@PNA9mc&eA3f)^T~o0Y}&Ohc{D89Z5K5s}7F(-GJy;|CCWiXWc7MCiackFRyvq
zevm6~+zFJgQP;`E{VV!QNsRu2eek3W!$8uN-E5VVyS6Sv)qLey?uUC4&1kGQl`W1T
zjBivNbEHbSutYL4=)4(;`TeSo>CI)_r`|U1OZrGO+^~!}K$B<F;t77CYlSh+W_pNH
zftwMb@{w5&uR@lQ`-?5>aDtAl1{_wiS<>&WY0P>K({23Tc16fJn<XN@l^K6J8Zuda
z@%oaoRBkd4mYBa$&LfXyefo9?!KH03f;RYZ{|6f1oQdrhO7gm_PkX#pOC`p-Rkv=%
z$08{@__?uI0^2?6iMRL~s&;w57IXV3q5ZV$s(yv>Z)?G~ui`i%r{tKa+X44=R3o*H
z{xBT-FX+P=TSHU=yXqrcT!m*%egvjca6jRWfdUV8GndcXOcToaJK!auQ7;}qDX_Yv
z@*hh#nc9r6zGVF42cfWq@%K(#u5M#lCyfk*cY7RO0X2Yz%(aI+YCU)^8Vbe8@7OtP
p8NNZ|b_9YUlm6#hDPPaAIq4$}3IaQp97A@ay-!~w5qs>~{{d1>&&2=$
literal 0
HcmV?d00001
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index ed7f770..008a43b 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -55,6 +55,7 @@ Programmer's Guide
pdump_lib
multi_proc_support
kernel_nic_interface
+ virtio_user_as_exception_path
thread_safety_dpdk_functions
qos_framework
power_man
diff --git a/doc/guides/prog_guide/virtio_user_as_exceptional_path.rst b/doc/guides/prog_guide/virtio_user_as_exceptional_path.rst
new file mode 100644
index 0000000..5cfeb10
--- /dev/null
+++ b/doc/guides/prog_guide/virtio_user_as_exceptional_path.rst
@@ -0,0 +1,104 @@
+.. BSD LICENSE
+ Copyright(c) 2016 Intel Corporation. All rights reserved.
+ All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+.. _virtio_user:
+
+Virtio_user as Exceptional Path
+===============================
+
+The virtual device, virtio_user, was originally introduced with vhost-user backend, as a high performance solution for IPC (Inter-Process Communication) and user space container networking.
+
+Virtio_user with vhost-kernel backend is a solution for exceptional path, such as KNI which exchanges packets with kernel networking stack. This solution is very promising in:
+
+* Maintenance: vhost and vhost-net (kernel) is upstreamed and extensively used kernel module.
+
+* Features: vhost-net is born to be a networking solution, which has lots of networking related featuers, like multi queue, tso, multi-seg mbuf, etc.
+
+* Performance: similar to KNI, this solution would use one or more kthreads to send/receive packets from user space DPDK applications, which has little impact on user space polling thread (except that it might enter into kernel space to wake up those kthreads if necessary).
+
+The overview of an application using virtio_user as exceptional path is shown in :numref:`figure_virtio_user_as_exceptional_path`.
+
+.. _figure_virtio_user_as_exceptional_path:
+
+.. figure:: img/virtio_user_as_exceptional_path.*
+
+ Overview of a DPDK app using virtio_user as excpetional path
+
+
+Sample Usage
+------------
+
+As a prerequisite, the vhost/vhost-net kernel CONFIG should be chosen before compiling the kernel and those kernel modules should be inserted.
+
+#. Compile the DPDK and bind a physical NIC (for communicating with outside) to igb_uio/uio_pci_generic/vfio-pci.
+
+#. Run testpmd:
+
+ .. code-block:: console
+
+ examples/kni/build/app/kni -c -0xf0 -n 4 -- -p 0x3 -P --config="(0,4,6),(1,5,7)"
+ $(testpmd) -c 0xc -n 4 --vdev=virtio_user0,path=/dev/vhost-net,queue_size=1024 -- -i --txqflags=0x0 --disable-hw-vlan --enable-lro --crc-strip --enable-rx-cksum --rxd=1024 --txd=1024
+
+ This command runs testpmd with two ports, one physical NIC to communicate with outside, and one virtio_user to communicate with kernel.
+
+* ``--enable-lro`` This is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from kernel can be transmitted DPDK application and further TSOed by physical NIC.
+
+* ``--enable-rx-cksum`` This is used to negotiate VIRTIO_NET_F_GUEST_CSUM so that packets from kernel can be deemed as valid Rx checksumed.
+
+* ``queue_size`` of virtio_user, 256 by default. To avoid shortage of descriptors, we can increase it to 1024.
+
+* ``queues`` of virtio_user to specify multi-queues. Each qeueue will be served by a kthread:
+
+ .. code-block:: console
+
+ $(testpmd) -c 0xc -n 4 --vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 -- -i --txqflags=0x0 --disable-hw-vlan --enable-lro --crc-strip --enable-rx-cksum --txq=2 --rxq=2 --rxd=1024 --txd=1024
+
+#. Start testpmd:
+
+ .. code-block:: console
+
+ (testpmd) start
+
+#. Configure IP address and start tap:
+
+ .. code-block:: console
+
+ ifconfig tap0 1.1.1.1/24 up
+
+.. note::
+
+ The tap device will be named tap0, tap1, etc, by kernel.
+
+Then, all traffic from physical NIC can be forwarded into kernel stack, and all traffic on the tap0 can be sent out from physical NIC.
+
+Limitations
+-----------
+
+This solution is only available on Linux systems.
--
2.7.4
^ permalink raw reply related
* [PATCH v4 5/8] net/virtio_user: add vhost kernel support
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
In-Reply-To: <1484309921-116526-1-git-send-email-jianfeng.tan@intel.com>
This patch add support vhost kernel as the backend for virtio_user.
Three main hook functions are added:
- vhost_kernel_setup() to open char device, each vq pair needs one
vhostfd;
- vhost_kernel_ioctl() to communicate control messages with vhost
kernel module;
- vhost_kernel_enable_queue_pair() to open tap device and set it
as the backend of corresonding vhost fd (that is to say, vq pair).
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
doc/guides/rel_notes/release_17_02.rst | 20 ++
drivers/net/virtio/Makefile | 2 +
drivers/net/virtio/virtio_user/vhost.h | 2 +
drivers/net/virtio/virtio_user/vhost_kernel.c | 311 ++++++++++++++++++++++
drivers/net/virtio/virtio_user/vhost_kernel_tap.c | 116 ++++++++
drivers/net/virtio/virtio_user/vhost_kernel_tap.h | 60 +++++
drivers/net/virtio/virtio_user/virtio_user_dev.c | 36 ++-
drivers/net/virtio/virtio_user/virtio_user_dev.h | 5 +
8 files changed, 549 insertions(+), 3 deletions(-)
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel.c
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel_tap.c
create mode 100644 drivers/net/virtio/virtio_user/vhost_kernel_tap.h
diff --git a/doc/guides/rel_notes/release_17_02.rst b/doc/guides/rel_notes/release_17_02.rst
index 180af82..7354df5 100644
--- a/doc/guides/rel_notes/release_17_02.rst
+++ b/doc/guides/rel_notes/release_17_02.rst
@@ -52,6 +52,26 @@ New Features
See the :ref:`Generic flow API <Generic_flow_API>` documentation for more
information.
+* **virtio_user with vhost-kernel as another exceptional path.**
+
+ Previously, we upstreamed a virtual device, virtio_user with vhost-user
+ as the backend, as a way for IPC (Inter-Process Communication) and user
+ space container networking.
+
+ Virtio_user with vhost-kernel as the backend is a solution for exceptional
+ path, such as KNI, which exchanges packets with kernel networking stack.
+ This solution is very promising in:
+
+ * maintenance: vhost and vhost-net (kernel) is upstreamed and extensively
+ used kernel module.
+ * features: vhost-net is born to be a networking solution, which has
+ lots of networking related featuers, like multi queue, tso, multi-seg
+ mbuf, etc.
+ * performance: similar to KNI, this solution would use one or more
+ kthreads to send/receive packets from user space DPDK applications,
+ which has little impact on user space polling thread (except that
+ it might enter into kernel space to wake up those kthreads if
+ necessary).
Resolved Issues
---------------
diff --git a/drivers/net/virtio/Makefile b/drivers/net/virtio/Makefile
index 97972a6..8b5b5d6 100644
--- a/drivers/net/virtio/Makefile
+++ b/drivers/net/virtio/Makefile
@@ -60,6 +60,8 @@ endif
ifeq ($(CONFIG_RTE_VIRTIO_USER),y)
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_user.c
+SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_kernel.c
+SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/vhost_kernel_tap.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user/virtio_user_dev.c
SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_user_ethdev.c
endif
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index 515e4fc..5c983bd 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -118,4 +118,6 @@ struct virtio_user_backend_ops {
};
struct virtio_user_backend_ops ops_user;
+struct virtio_user_backend_ops ops_kernel;
+
#endif
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
new file mode 100644
index 0000000..9301f96
--- /dev/null
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -0,0 +1,311 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <unistd.h>
+
+#include <rte_memory.h>
+#include <rte_eal_memconfig.h>
+
+#include "vhost.h"
+#include "virtio_user_dev.h"
+#include "vhost_kernel_tap.h"
+
+struct vhost_memory_kernel {
+ uint32_t nregions;
+ uint32_t padding;
+ struct vhost_memory_region regions[0];
+};
+
+/* vhost kernel ioctls */
+#define VHOST_VIRTIO 0xAF
+#define VHOST_GET_FEATURES _IOR(VHOST_VIRTIO, 0x00, __u64)
+#define VHOST_SET_FEATURES _IOW(VHOST_VIRTIO, 0x00, __u64)
+#define VHOST_SET_OWNER _IO(VHOST_VIRTIO, 0x01)
+#define VHOST_RESET_OWNER _IO(VHOST_VIRTIO, 0x02)
+#define VHOST_SET_MEM_TABLE _IOW(VHOST_VIRTIO, 0x03, struct vhost_memory_kernel)
+#define VHOST_SET_LOG_BASE _IOW(VHOST_VIRTIO, 0x04, __u64)
+#define VHOST_SET_LOG_FD _IOW(VHOST_VIRTIO, 0x07, int)
+#define VHOST_SET_VRING_NUM _IOW(VHOST_VIRTIO, 0x10, struct vhost_vring_state)
+#define VHOST_SET_VRING_ADDR _IOW(VHOST_VIRTIO, 0x11, struct vhost_vring_addr)
+#define VHOST_SET_VRING_BASE _IOW(VHOST_VIRTIO, 0x12, struct vhost_vring_state)
+#define VHOST_GET_VRING_BASE _IOWR(VHOST_VIRTIO, 0x12, struct vhost_vring_state)
+#define VHOST_SET_VRING_KICK _IOW(VHOST_VIRTIO, 0x20, struct vhost_vring_file)
+#define VHOST_SET_VRING_CALL _IOW(VHOST_VIRTIO, 0x21, struct vhost_vring_file)
+#define VHOST_SET_VRING_ERR _IOW(VHOST_VIRTIO, 0x22, struct vhost_vring_file)
+#define VHOST_NET_SET_BACKEND _IOW(VHOST_VIRTIO, 0x30, struct vhost_vring_file)
+
+static uint64_t max_regions = 64;
+
+static void
+get_vhost_kernel_max_regions(void)
+{
+ int fd;
+ char buf[20] = {'\0'};
+
+ fd = open("/sys/module/vhost/parameters/max_mem_regions", O_RDONLY);
+ if (fd < 0)
+ return;
+
+ if (read(fd, buf, 20-1) > 0)
+ max_regions = strtoull(buf, NULL, 10);
+
+ close(fd);
+}
+
+static uint64_t vhost_req_user_to_kernel[] = {
+ [VHOST_USER_SET_OWNER] = VHOST_SET_OWNER,
+ [VHOST_USER_RESET_OWNER] = VHOST_RESET_OWNER,
+ [VHOST_USER_SET_FEATURES] = VHOST_SET_FEATURES,
+ [VHOST_USER_GET_FEATURES] = VHOST_GET_FEATURES,
+ [VHOST_USER_SET_VRING_CALL] = VHOST_SET_VRING_CALL,
+ [VHOST_USER_SET_VRING_NUM] = VHOST_SET_VRING_NUM,
+ [VHOST_USER_SET_VRING_BASE] = VHOST_SET_VRING_BASE,
+ [VHOST_USER_GET_VRING_BASE] = VHOST_GET_VRING_BASE,
+ [VHOST_USER_SET_VRING_ADDR] = VHOST_SET_VRING_ADDR,
+ [VHOST_USER_SET_VRING_KICK] = VHOST_SET_VRING_KICK,
+ [VHOST_USER_SET_MEM_TABLE] = VHOST_SET_MEM_TABLE,
+};
+
+/* By default, vhost kernel module allows 64 regions, but DPDK allows
+ * 256 segments. As a relief, below function merges those virtually
+ * adjacent memsegs into one region.
+ */
+static struct vhost_memory_kernel *
+prepare_vhost_memory_kernel(void)
+{
+ uint32_t i, j, k = 0;
+ struct rte_memseg *seg;
+ struct vhost_memory_region *mr;
+ struct vhost_memory_kernel *vm;
+
+ vm = malloc(sizeof(struct vhost_memory_kernel) +
+ max_regions *
+ sizeof(struct vhost_memory_region));
+
+ for (i = 0; i < RTE_MAX_MEMSEG; ++i) {
+ seg = &rte_eal_get_configuration()->mem_config->memseg[i];
+ if (!seg->addr)
+ break;
+
+ int new_region = 1;
+
+ for (j = 0; j < k; ++j) {
+ mr = &vm->regions[j];
+
+ if (mr->userspace_addr + mr->memory_size ==
+ (uint64_t)(uintptr_t)seg->addr) {
+ mr->memory_size += seg->len;
+ new_region = 0;
+ break;
+ }
+
+ if ((uint64_t)(uintptr_t)seg->addr + seg->len ==
+ mr->userspace_addr) {
+ mr->guest_phys_addr =
+ (uint64_t)(uintptr_t)seg->addr;
+ mr->userspace_addr =
+ (uint64_t)(uintptr_t)seg->addr;
+ mr->memory_size += seg->len;
+ new_region = 0;
+ break;
+ }
+ }
+
+ if (new_region == 0)
+ continue;
+
+ mr = &vm->regions[k++];
+ /* use vaddr here! */
+ mr->guest_phys_addr = (uint64_t)(uintptr_t)seg->addr;
+ mr->userspace_addr = (uint64_t)(uintptr_t)seg->addr;
+ mr->memory_size = seg->len;
+ mr->mmap_offset = 0;
+
+ if (k >= max_regions) {
+ free(vm);
+ return NULL;
+ }
+ }
+
+ vm->nregions = k;
+ vm->padding = 0;
+ return vm;
+}
+
+static int
+vhost_kernel_ioctl(struct virtio_user_dev *dev,
+ enum vhost_user_request req,
+ void *arg)
+{
+ int ret = -1;
+ unsigned int i;
+ uint64_t req_kernel;
+ struct vhost_memory_kernel *vm = NULL;
+
+ PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
+
+ req_kernel = vhost_req_user_to_kernel[req];
+
+ if (req_kernel == VHOST_SET_MEM_TABLE) {
+ vm = prepare_vhost_memory_kernel();
+ if (!vm)
+ return -1;
+ arg = (void *)vm;
+ }
+
+ /* We don't need memory protection here */
+ if (req_kernel == VHOST_SET_FEATURES)
+ *(uint64_t *)arg &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
+
+ for (i = 0; i < dev->max_queue_pairs; ++i) {
+ if (dev->vhostfds[i] < 0)
+ continue;
+
+ ret = ioctl(dev->vhostfds[i], req_kernel, arg);
+ if (ret < 0)
+ break;
+ }
+
+ if (vm)
+ free(vm);
+
+ if (ret < 0)
+ PMD_DRV_LOG(ERR, "%s failed: %s",
+ vhost_msg_strings[req], strerror(errno));
+
+ return ret;
+}
+
+/**
+ * Set up environment to talk with a vhost kernel backend.
+ *
+ * @return
+ * - (-1) if fail to set up;
+ * - (>=0) if successful.
+ */
+static int
+vhost_kernel_setup(struct virtio_user_dev *dev)
+{
+ int vhostfd;
+ uint32_t i;
+
+ get_vhost_kernel_max_regions();
+
+ for (i = 0; i < dev->max_queue_pairs; ++i) {
+ vhostfd = open(dev->path, O_RDWR);
+ if (vhostfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s, %s",
+ dev->path, strerror(errno));
+ return -1;
+ }
+
+ dev->vhostfds[i] = vhostfd;
+ }
+
+ return 0;
+}
+
+static int
+vhost_kernel_set_backend(int vhostfd, int tapfd)
+{
+ struct vhost_vring_file f;
+
+ f.fd = tapfd;
+ f.index = 0;
+ if (ioctl(vhostfd, VHOST_NET_SET_BACKEND, &f) < 0) {
+ PMD_DRV_LOG(ERR, "VHOST_NET_SET_BACKEND fails, %s",
+ strerror(errno));
+ return -1;
+ }
+
+ f.index = 1;
+ if (ioctl(vhostfd, VHOST_NET_SET_BACKEND, &f) < 0) {
+ PMD_DRV_LOG(ERR, "VHOST_NET_SET_BACKEND fails, %s",
+ strerror(errno));
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev,
+ uint16_t pair_idx,
+ int enable)
+{
+ int hdr_size;
+ int vhostfd;
+ int tapfd;
+
+ vhostfd = dev->vhostfds[pair_idx];
+
+ if (!enable) {
+ if (dev->tapfds[pair_idx]) {
+ close(dev->tapfds[pair_idx]);
+ dev->tapfds[pair_idx] = -1;
+ }
+ return vhost_kernel_set_backend(vhostfd, -1);
+ } else if (dev->tapfds[pair_idx] >= 0) {
+ return 0;
+ }
+
+ if ((dev->features & (1ULL << VIRTIO_NET_F_MRG_RXBUF)) ||
+ (dev->features & (1ULL << VIRTIO_F_VERSION_1)))
+ hdr_size = sizeof(struct virtio_net_hdr_mrg_rxbuf);
+ else
+ hdr_size = sizeof(struct virtio_net_hdr);
+
+ tapfd = vhost_kernel_open_tap(&dev->ifname, hdr_size);
+ if (tapfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open tap for vhost kernel");
+ return -1;
+ }
+
+ if (vhost_kernel_set_backend(vhostfd, tapfd) < 0) {
+ PMD_DRV_LOG(ERR, "fail to set backend for vhost kernel");
+ close(tapfd);
+ return -1;
+ }
+
+ dev->tapfds[pair_idx] = tapfd;
+ return 0;
+}
+
+struct virtio_user_backend_ops ops_kernel = {
+ .setup = vhost_kernel_setup,
+ .send_request = vhost_kernel_ioctl,
+ .enable_qp = vhost_kernel_enable_queue_pair
+};
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.c b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
new file mode 100644
index 0000000..797713b
--- /dev/null
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.c
@@ -0,0 +1,116 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <net/if.h>
+#include <errno.h>
+#include <string.h>
+#include <limits.h>
+
+#include "vhost_kernel_tap.h"
+#include "../virtio_logs.h"
+
+int
+vhost_kernel_open_tap(char **p_ifname, int hdr_size)
+{
+ unsigned int tap_features;
+ int sndbuf = INT_MAX;
+ struct ifreq ifr;
+ int tapfd;
+
+ /* TODO:
+ * 1. verify we can get/set vnet_hdr_len, tap_probe_vnet_hdr_len
+ * 2. get number of memory regions from vhost module parameter
+ * max_mem_regions, supported in newer version linux kernel
+ */
+ tapfd = open(PATH_NET_TUN, O_RDWR);
+ if (tapfd < 0) {
+ PMD_DRV_LOG(ERR, "fail to open %s: %s",
+ PATH_NET_TUN, strerror(errno));
+ return -1;
+ }
+
+ /* Construct ifr */
+ memset(&ifr, 0, sizeof(ifr));
+ ifr.ifr_flags = IFF_TAP | IFF_NO_PI;
+
+ if (ioctl(tapfd, TUNGETFEATURES, &tap_features) == -1) {
+ PMD_DRV_LOG(ERR, "TUNGETFEATURES failed: %s", strerror(errno));
+ goto error;
+ }
+ if (tap_features & IFF_ONE_QUEUE)
+ ifr.ifr_flags |= IFF_ONE_QUEUE;
+
+ /* Let tap instead of vhost-net handle vnet header, as the latter does
+ * not support offloading. And in this case, we should not set feature
+ * bit VHOST_NET_F_VIRTIO_NET_HDR.
+ */
+ if (tap_features & IFF_VNET_HDR) {
+ ifr.ifr_flags |= IFF_VNET_HDR;
+ } else {
+ PMD_DRV_LOG(ERR, "TAP does not support IFF_VNET_HDR");
+ goto error;
+ }
+
+ if (*p_ifname)
+ strncpy(ifr.ifr_name, *p_ifname, IFNAMSIZ);
+ else
+ strncpy(ifr.ifr_name, "tap%d", IFNAMSIZ);
+ if (ioctl(tapfd, TUNSETIFF, (void *)&ifr) == -1) {
+ PMD_DRV_LOG(ERR, "TUNSETIFF failed: %s", strerror(errno));
+ goto error;
+ }
+
+ fcntl(tapfd, F_SETFL, O_NONBLOCK);
+
+ if (ioctl(tapfd, TUNSETVNETHDRSZ, &hdr_size) < 0) {
+ PMD_DRV_LOG(ERR, "TUNSETVNETHDRSZ failed: %s", strerror(errno));
+ goto error;
+ }
+
+ if (ioctl(tapfd, TUNSETSNDBUF, &sndbuf) < 0) {
+ PMD_DRV_LOG(ERR, "TUNSETSNDBUF failed: %s", strerror(errno));
+ goto error;
+ }
+
+ if (!(*p_ifname))
+ *p_ifname = strdup(ifr.ifr_name);
+
+ return tapfd;
+error:
+ close(tapfd);
+ return -1;
+}
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel_tap.h b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
new file mode 100644
index 0000000..1fc58ff
--- /dev/null
+++ b/drivers/net/virtio/virtio_user/vhost_kernel_tap.h
@@ -0,0 +1,60 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/ioctl.h>
+
+/* TUN ioctls */
+#define TUNSETIFF _IOW('T', 202, int)
+#define TUNGETFEATURES _IOR('T', 207, unsigned int)
+#define TUNSETOFFLOAD _IOW('T', 208, unsigned int)
+#define TUNGETIFF _IOR('T', 210, unsigned int)
+#define TUNSETSNDBUF _IOW('T', 212, int)
+#define TUNGETVNETHDRSZ _IOR('T', 215, int)
+#define TUNSETVNETHDRSZ _IOW('T', 216, int)
+#define TUNSETQUEUE _IOW('T', 217, int)
+#define TUNSETVNETLE _IOW('T', 220, int)
+#define TUNSETVNETBE _IOW('T', 222, int)
+
+/* TUNSETIFF ifr flags */
+#define IFF_TAP 0x0002
+#define IFF_NO_PI 0x1000
+#define IFF_ONE_QUEUE 0x2000
+#define IFF_VNET_HDR 0x4000
+#define IFF_MULTI_QUEUE 0x0100
+#define IFF_ATTACH_QUEUE 0x0200
+#define IFF_DETACH_QUEUE 0x0400
+
+/* Constants */
+#define PATH_NET_TUN "/dev/net/tun"
+
+int vhost_kernel_open_tap(char **p_ifname, int hdr_size);
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 32039a1..ae0824c 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -192,6 +192,9 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
for (i = 0; i < dev->max_queue_pairs; ++i)
dev->ops->enable_qp(dev, i, 0);
+ free(dev->ifname);
+ dev->ifname = NULL;
+
return 0;
}
@@ -230,7 +233,7 @@ is_vhost_user_by_type(const char *path)
static int
virtio_user_dev_setup(struct virtio_user_dev *dev)
{
- uint32_t i;
+ uint32_t i, q;
dev->vhostfd = -1;
for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
@@ -238,12 +241,28 @@ virtio_user_dev_setup(struct virtio_user_dev *dev)
dev->callfds[i] = -1;
}
+ dev->vhostfds = NULL;
+ dev->tapfds = NULL;
+
if (is_vhost_user_by_type(dev->path)) {
dev->ops = &ops_user;
- return dev->ops->setup(dev);
+ } else {
+ dev->ops = &ops_kernel;
+
+ dev->vhostfds = malloc(dev->max_queue_pairs * sizeof(int));
+ dev->tapfds = malloc(dev->max_queue_pairs * sizeof(int));
+ if (!dev->vhostfds || !dev->tapfds) {
+ PMD_INIT_LOG(ERR, "Failed to malloc");
+ return -1;
+ }
+
+ for (q = 0; q < dev->max_queue_pairs; ++q) {
+ dev->vhostfds[q] = -1;
+ dev->tapfds[q] = -1;
+ }
}
- return -1;
+ return dev->ops->setup(dev);
}
int
@@ -295,7 +314,18 @@ virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
void
virtio_user_dev_uninit(struct virtio_user_dev *dev)
{
+ uint32_t i;
+
+ virtio_user_stop_device(dev);
+
close(dev->vhostfd);
+
+ if (dev->vhostfds) {
+ for (i = 0; i < dev->max_queue_pairs; ++i)
+ close(dev->vhostfds[i]);
+ free(dev->vhostfds);
+ free(dev->tapfds);
+ }
}
static uint8_t
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 9f2f82e..0d39f40 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -43,6 +43,11 @@ struct virtio_user_dev {
/* for vhost_user backend */
int vhostfd;
+ /* for vhost_kernel backend */
+ char *ifname;
+ int *vhostfds;
+ int *tapfds;
+
/* for both vhost_user and vhost_kernel */
int callfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int kickfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
--
2.7.4
^ permalink raw reply related
* Re: [PATCH] net/nfp: fix Vlan offload flags check
From: Ferruh Yigit @ 2017-01-13 13:17 UTC (permalink / raw)
To: Olivier Matz; +Cc: dev, alejandro.lucero
In-Reply-To: <20170113134824.6542fc5e@platinum>
On 1/13/2017 12:48 PM, Olivier Matz wrote:
> On Fri, 13 Jan 2017 12:03:01 +0000, Ferruh Yigit
> <ferruh.yigit@intel.com> wrote:
>> On 1/12/2017 1:04 PM, Olivier Matz wrote:
>>> Fix typo when checking that no Vlan offload flags are passed at port
>>> initialization.
>>>
>>> By the way, also fix a typo in the log.
>>>
>>> Fixes: d4a27a3b092a ("nfp: add basic features")
>>>
>>> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
>>> ---
<...>
>
>
> I'll submit another patch with ETH_VLAN_EXTEND_OFFLOAD.
If this will be the only change, I can update it if you want:
- (mask & ETH_VLAN_FILTER_OFFLOAD))
- RTE_LOG(INFO, PMD, "Not support for ETH_VLAN_FILTER_OFFLOAD or"
- " ETH_VLAN_FILTER_EXTEND");
+ (mask & ETH_VLAN_EXTEND_OFFLOAD))
+ RTE_LOG(INFO, PMD, "No support for ETH_VLAN_FILTER_OFFLOAD or"
+ " ETH_VLAN_EXTEND_OFFLOAD");
> Thanks Ferruh.
>
> Olivier
>
^ permalink raw reply
* [PATCH v4 4/8] net/virtio_user: abstract virtio user backend ops
From: Jianfeng Tan @ 2017-01-13 12:18 UTC (permalink / raw)
To: dev; +Cc: yuanhan.liu, ferruh.yigit, cunming.liang, Jianfeng Tan
In-Reply-To: <1484309921-116526-1-git-send-email-jianfeng.tan@intel.com>
Add a struct virtio_user_backend_ops to abstract three kinds of backend
operations:
- setup, create the unix socket connection;
- send_request, sync messages with backend;
- enable_qp, enable some queue pair.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_user/vhost.h | 17 +++++-
drivers/net/virtio/virtio_user/vhost_user.c | 66 +++++++++++---------
drivers/net/virtio/virtio_user/virtio_user_dev.c | 77 +++++++++++++++---------
drivers/net/virtio/virtio_user/virtio_user_dev.h | 5 ++
4 files changed, 106 insertions(+), 59 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/vhost.h b/drivers/net/virtio/virtio_user/vhost.h
index e54ac35..515e4fc 100644
--- a/drivers/net/virtio/virtio_user/vhost.h
+++ b/drivers/net/virtio/virtio_user/vhost.h
@@ -96,6 +96,8 @@ enum vhost_user_request {
VHOST_USER_MAX
};
+const char * const vhost_msg_strings[VHOST_USER_MAX];
+
struct vhost_memory_region {
uint64_t guest_phys_addr;
uint64_t memory_size; /* bytes */
@@ -103,8 +105,17 @@ struct vhost_memory_region {
uint64_t mmap_offset;
};
-int vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg);
-int vhost_user_setup(const char *path);
-int vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable);
+struct virtio_user_dev;
+
+struct virtio_user_backend_ops {
+ int (*setup)(struct virtio_user_dev *dev);
+ int (*send_request)(struct virtio_user_dev *dev,
+ enum vhost_user_request req,
+ void *arg);
+ int (*enable_qp)(struct virtio_user_dev *dev,
+ uint16_t pair_idx,
+ int enable);
+};
+struct virtio_user_backend_ops ops_user;
#endif
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c b/drivers/net/virtio/virtio_user/vhost_user.c
index 295ce16..4ad7b21 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -41,6 +41,7 @@
#include <errno.h>
#include "vhost.h"
+#include "virtio_user_dev.h"
/* The version of the protocol we support */
#define VHOST_USER_VERSION 0x1
@@ -255,24 +256,25 @@ prepare_vhost_memory_user(struct vhost_user_msg *msg, int fds[])
static struct vhost_user_msg m;
-static const char * const vhost_msg_strings[] = {
- [VHOST_USER_SET_OWNER] = "VHOST_USER_SET_OWNER",
- [VHOST_USER_RESET_OWNER] = "VHOST_USER_RESET_OWNER",
- [VHOST_USER_SET_FEATURES] = "VHOST_USER_SET_FEATURES",
- [VHOST_USER_GET_FEATURES] = "VHOST_USER_GET_FEATURES",
- [VHOST_USER_SET_VRING_CALL] = "VHOST_USER_SET_VRING_CALL",
- [VHOST_USER_SET_VRING_NUM] = "VHOST_USER_SET_VRING_NUM",
- [VHOST_USER_SET_VRING_BASE] = "VHOST_USER_SET_VRING_BASE",
- [VHOST_USER_GET_VRING_BASE] = "VHOST_USER_GET_VRING_BASE",
- [VHOST_USER_SET_VRING_ADDR] = "VHOST_USER_SET_VRING_ADDR",
- [VHOST_USER_SET_VRING_KICK] = "VHOST_USER_SET_VRING_KICK",
- [VHOST_USER_SET_MEM_TABLE] = "VHOST_USER_SET_MEM_TABLE",
- [VHOST_USER_SET_VRING_ENABLE] = "VHOST_USER_SET_VRING_ENABLE",
- NULL,
+const char * const vhost_msg_strings[] = {
+ [VHOST_USER_SET_OWNER] = "VHOST_SET_OWNER",
+ [VHOST_USER_RESET_OWNER] = "VHOST_RESET_OWNER",
+ [VHOST_USER_SET_FEATURES] = "VHOST_SET_FEATURES",
+ [VHOST_USER_GET_FEATURES] = "VHOST_GET_FEATURES",
+ [VHOST_USER_SET_VRING_CALL] = "VHOST_SET_VRING_CALL",
+ [VHOST_USER_SET_VRING_NUM] = "VHOST_SET_VRING_NUM",
+ [VHOST_USER_SET_VRING_BASE] = "VHOST_SET_VRING_BASE",
+ [VHOST_USER_GET_VRING_BASE] = "VHOST_GET_VRING_BASE",
+ [VHOST_USER_SET_VRING_ADDR] = "VHOST_SET_VRING_ADDR",
+ [VHOST_USER_SET_VRING_KICK] = "VHOST_SET_VRING_KICK",
+ [VHOST_USER_SET_MEM_TABLE] = "VHOST_SET_MEM_TABLE",
+ [VHOST_USER_SET_VRING_ENABLE] = "VHOST_SET_VRING_ENABLE",
};
-int
-vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
+static int
+vhost_user_sock(struct virtio_user_dev *dev,
+ enum vhost_user_request req,
+ void *arg)
{
struct vhost_user_msg msg;
struct vhost_vring_file *file = 0;
@@ -280,9 +282,9 @@ vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
int fds[VHOST_MEMORY_MAX_NREGIONS];
int fd_num = 0;
int i, len;
+ int vhostfd = dev->vhostfd;
RTE_SET_USED(m);
- RTE_SET_USED(vhost_msg_strings);
PMD_DRV_LOG(INFO, "%s", vhost_msg_strings[req]);
@@ -403,15 +405,13 @@ vhost_user_sock(int vhostfd, enum vhost_user_request req, void *arg)
/**
* Set up environment to talk with a vhost user backend.
- * @param path
- * - The path to vhost user unix socket file.
*
* @return
- * - (-1) if fail to set up;
- * - (>=0) if successful, and it is the fd to vhostfd.
+ * - (-1) if fail;
+ * - (0) if succeed.
*/
-int
-vhost_user_setup(const char *path)
+static int
+vhost_user_setup(struct virtio_user_dev *dev)
{
int fd;
int flag;
@@ -429,18 +429,21 @@ vhost_user_setup(const char *path)
memset(&un, 0, sizeof(un));
un.sun_family = AF_UNIX;
- snprintf(un.sun_path, sizeof(un.sun_path), "%s", path);
+ snprintf(un.sun_path, sizeof(un.sun_path), "%s", dev->path);
if (connect(fd, (struct sockaddr *)&un, sizeof(un)) < 0) {
PMD_DRV_LOG(ERR, "connect error, %s", strerror(errno));
close(fd);
return -1;
}
- return fd;
+ dev->vhostfd = fd;
+ return 0;
}
-int
-vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable)
+static int
+vhost_user_enable_queue_pair(struct virtio_user_dev *dev,
+ uint16_t pair_idx,
+ int enable)
{
int i;
@@ -450,10 +453,15 @@ vhost_user_enable_queue_pair(int vhostfd, uint16_t pair_idx, int enable)
.num = enable,
};
- if (vhost_user_sock(vhostfd,
- VHOST_USER_SET_VRING_ENABLE, &state))
+ if (vhost_user_sock(dev, VHOST_USER_SET_VRING_ENABLE, &state))
return -1;
}
return 0;
}
+
+struct virtio_user_backend_ops ops_user = {
+ .setup = vhost_user_setup,
+ .send_request = vhost_user_sock,
+ .enable_qp = vhost_user_enable_queue_pair
+};
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 8dd563a..32039a1 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -39,6 +39,9 @@
#include <sys/mman.h>
#include <unistd.h>
#include <sys/eventfd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
#include "vhost.h"
#include "virtio_user_dev.h"
@@ -64,7 +67,7 @@ virtio_user_create_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
}
file.index = queue_sel;
file.fd = callfd;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_CALL, &file);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_CALL, &file);
dev->callfds[queue_sel] = callfd;
return 0;
@@ -88,12 +91,12 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
state.index = queue_sel;
state.num = vring->num;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_NUM, &state);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_NUM, &state);
state.num = 0; /* no reservation */
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_BASE, &state);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_BASE, &state);
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_ADDR, &addr);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_ADDR, &addr);
/* Of all per virtqueue MSGs, make sure VHOST_USER_SET_VRING_KICK comes
* lastly because vhost depends on this msg to judge if
@@ -106,7 +109,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
}
file.index = queue_sel;
file.fd = kickfd;
- vhost_user_sock(dev->vhostfd, VHOST_USER_SET_VRING_KICK, &file);
+ dev->ops->send_request(dev, VHOST_USER_SET_VRING_KICK, &file);
dev->kickfds[queue_sel] = kickfd;
return 0;
@@ -146,20 +149,19 @@ virtio_user_start_device(struct virtio_user_dev *dev)
if (virtio_user_queue_setup(dev, virtio_user_create_queue) < 0)
goto error;
- /* Step 1: set features
- * Make sure VHOST_USER_F_PROTOCOL_FEATURES is added if mq is enabled,
- * VIRTIO_NET_F_MAC and VIRTIO_NET_F_CTRL_VQ is stripped.
- */
+ /* Step 1: set features */
features = dev->features;
+ /* Strip VIRTIO_NET_F_MAC, as MAC address is handled in vdev init */
features &= ~(1ull << VIRTIO_NET_F_MAC);
+ /* Strip VIRTIO_NET_F_CTRL_VQ, as devices do not really need to know */
features &= ~(1ull << VIRTIO_NET_F_CTRL_VQ);
- ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_FEATURES, &features);
+ ret = dev->ops->send_request(dev, VHOST_USER_SET_FEATURES, &features);
if (ret < 0)
goto error;
PMD_DRV_LOG(INFO, "set features: %" PRIx64, features);
/* Step 2: share memory regions */
- ret = vhost_user_sock(dev->vhostfd, VHOST_USER_SET_MEM_TABLE, NULL);
+ ret = dev->ops->send_request(dev, VHOST_USER_SET_MEM_TABLE, NULL);
if (ret < 0)
goto error;
@@ -170,7 +172,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
/* Step 4: enable queues
* we enable the 1st queue pair by default.
*/
- vhost_user_enable_queue_pair(dev->vhostfd, 0, 1);
+ dev->ops->enable_qp(dev, 0, 1);
return 0;
error:
@@ -188,7 +190,7 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
}
for (i = 0; i < dev->max_queue_pairs; ++i)
- vhost_user_enable_queue_pair(dev->vhostfd, i, 0);
+ dev->ops->enable_qp(dev, i, 0);
return 0;
}
@@ -214,36 +216,57 @@ parse_mac(struct virtio_user_dev *dev, const char *mac)
}
}
+static int
+is_vhost_user_by_type(const char *path)
+{
+ struct stat sb;
+
+ if (stat(path, &sb) == -1)
+ return 0;
+
+ return S_ISSOCK(sb.st_mode);
+}
+
+static int
+virtio_user_dev_setup(struct virtio_user_dev *dev)
+{
+ uint32_t i;
+
+ dev->vhostfd = -1;
+ for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
+ dev->kickfds[i] = -1;
+ dev->callfds[i] = -1;
+ }
+
+ if (is_vhost_user_by_type(dev->path)) {
+ dev->ops = &ops_user;
+ return dev->ops->setup(dev);
+ }
+
+ return -1;
+}
+
int
virtio_user_dev_init(struct virtio_user_dev *dev, char *path, int queues,
int cq, int queue_size, const char *mac)
{
- uint32_t i;
-
snprintf(dev->path, PATH_MAX, "%s", path);
dev->max_queue_pairs = queues;
dev->queue_pairs = 1; /* mq disabled by default */
dev->queue_size = queue_size;
dev->mac_specified = 0;
parse_mac(dev, mac);
- dev->vhostfd = -1;
-
- for (i = 0; i < VIRTIO_MAX_VIRTQUEUES * 2 + 1; ++i) {
- dev->kickfds[i] = -1;
- dev->callfds[i] = -1;
- }
- dev->vhostfd = vhost_user_setup(dev->path);
- if (dev->vhostfd < 0) {
+ if (virtio_user_dev_setup(dev) < 0) {
PMD_INIT_LOG(ERR, "backend set up fails");
return -1;
}
- if (vhost_user_sock(dev->vhostfd, VHOST_USER_SET_OWNER, NULL) < 0) {
+ if (dev->ops->send_request(dev, VHOST_USER_SET_OWNER, NULL) < 0) {
PMD_INIT_LOG(ERR, "set_owner fails: %s", strerror(errno));
return -1;
}
- if (vhost_user_sock(dev->vhostfd, VHOST_USER_GET_FEATURES,
+ if (dev->ops->send_request(dev, VHOST_USER_GET_FEATURES,
&dev->device_features) < 0) {
PMD_INIT_LOG(ERR, "get_features failed: %s", strerror(errno));
return -1;
@@ -288,9 +311,9 @@ virtio_user_handle_mq(struct virtio_user_dev *dev, uint16_t q_pairs)
}
for (i = 0; i < q_pairs; ++i)
- ret |= vhost_user_enable_queue_pair(dev->vhostfd, i, 1);
+ ret |= dev->ops->enable_qp(dev, i, 1);
for (i = q_pairs; i < dev->max_queue_pairs; ++i)
- ret |= vhost_user_enable_queue_pair(dev->vhostfd, i, 0);
+ ret |= dev->ops->enable_qp(dev, i, 0);
dev->queue_pairs = q_pairs;
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h b/drivers/net/virtio/virtio_user/virtio_user_dev.h
index 28fc788..9f2f82e 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.h
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h
@@ -37,9 +37,13 @@
#include <limits.h>
#include "../virtio_pci.h"
#include "../virtio_ring.h"
+#include "vhost.h"
struct virtio_user_dev {
+ /* for vhost_user backend */
int vhostfd;
+
+ /* for both vhost_user and vhost_kernel */
int callfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int kickfds[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
int mac_specified;
@@ -54,6 +58,7 @@ struct virtio_user_dev {
uint8_t mac_addr[ETHER_ADDR_LEN];
char path[PATH_MAX];
struct vring vrings[VIRTIO_MAX_VIRTQUEUES * 2 + 1];
+ struct virtio_user_backend_ops *ops;
};
int virtio_user_start_device(struct virtio_user_dev *dev);
--
2.7.4
^ permalink raw reply related
* Re: [PATCH] net/nfp: fix Vlan offload flags check
From: Olivier Matz @ 2017-01-13 14:48 UTC (permalink / raw)
To: Ferruh Yigit; +Cc: dev, alejandro.lucero
In-Reply-To: <0fe55cec-99fe-8f61-9aa9-02cb51e9f990@intel.com>
On Fri, 13 Jan 2017 13:17:17 +0000, Ferruh Yigit
<ferruh.yigit@intel.com> wrote:
> On 1/13/2017 12:48 PM, Olivier Matz wrote:
> > On Fri, 13 Jan 2017 12:03:01 +0000, Ferruh Yigit
> > <ferruh.yigit@intel.com> wrote:
> >> On 1/12/2017 1:04 PM, Olivier Matz wrote:
> >>> Fix typo when checking that no Vlan offload flags are passed at
> >>> port initialization.
> >>>
> >>> By the way, also fix a typo in the log.
> >>>
> >>> Fixes: d4a27a3b092a ("nfp: add basic features")
> >>>
> >>> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> >>> ---
>
> <...>
>
> >
> >
> > I'll submit another patch with ETH_VLAN_EXTEND_OFFLOAD.
>
> If this will be the only change, I can update it if you want:
>
> - (mask & ETH_VLAN_FILTER_OFFLOAD))
> - RTE_LOG(INFO, PMD, "Not support for ETH_VLAN_FILTER_OFFLOAD
> or"
> - " ETH_VLAN_FILTER_EXTEND");
> + (mask & ETH_VLAN_EXTEND_OFFLOAD))
> + RTE_LOG(INFO, PMD, "No support for ETH_VLAN_FILTER_OFFLOAD or"
> + " ETH_VLAN_EXTEND_OFFLOAD");
>
Looks good to me, thank you.
Olivier
^ permalink raw reply
* Re: [PATCH] net/nfp: fix Vlan offload flags check
From: Alejandro Lucero @ 2017-01-13 14:57 UTC (permalink / raw)
To: Olivier Matz; +Cc: Ferruh Yigit, dev
In-Reply-To: <20170113154802.6166e529@platinum>
I was assuming in my acked answer, that the patch had followed usual path
and automatic compilation did not fail.
But I forgot about the NFP PMD not being enabled by default :-)
Hopefully, this will change if this just sent patch is accepted:
http://www.dpdk.org/ml/archives/dev/2017-January/054890.html
By other hand, I have compiled the NFP PMD with this patch and it works.
So acked again.
Thanks
On Fri, Jan 13, 2017 at 2:48 PM, Olivier Matz <olivier.matz@6wind.com>
wrote:
> On Fri, 13 Jan 2017 13:17:17 +0000, Ferruh Yigit
> <ferruh.yigit@intel.com> wrote:
> > On 1/13/2017 12:48 PM, Olivier Matz wrote:
> > > On Fri, 13 Jan 2017 12:03:01 +0000, Ferruh Yigit
> > > <ferruh.yigit@intel.com> wrote:
> > >> On 1/12/2017 1:04 PM, Olivier Matz wrote:
> > >>> Fix typo when checking that no Vlan offload flags are passed at
> > >>> port initialization.
> > >>>
> > >>> By the way, also fix a typo in the log.
> > >>>
> > >>> Fixes: d4a27a3b092a ("nfp: add basic features")
> > >>>
> > >>> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > >>> ---
> >
> > <...>
> >
> > >
> > >
> > > I'll submit another patch with ETH_VLAN_EXTEND_OFFLOAD.
> >
> > If this will be the only change, I can update it if you want:
> >
> > - (mask & ETH_VLAN_FILTER_OFFLOAD))
> > - RTE_LOG(INFO, PMD, "Not support for ETH_VLAN_FILTER_OFFLOAD
> > or"
> > - " ETH_VLAN_FILTER_EXTEND");
> > + (mask & ETH_VLAN_EXTEND_OFFLOAD))
> > + RTE_LOG(INFO, PMD, "No support for ETH_VLAN_FILTER_OFFLOAD or"
> > + " ETH_VLAN_EXTEND_OFFLOAD");
> >
>
> Looks good to me, thank you.
>
> Olivier
>
^ permalink raw reply
* Re: [PATCH v3 15/29] crypto/qat: use eal I/O device memory read/write API
From: Jerin Jacob @ 2017-01-13 14:57 UTC (permalink / raw)
To: Ferruh Yigit
Cc: dev, konstantin.ananyev, thomas.monjalon, bruce.richardson,
jianbo.liu, viktorin, santosh.shukla, John Griffin, Fiona Trahe,
Deepak Kumar Jain
In-Reply-To: <b528e477-194e-ad1a-2b5e-3422c6d49721@intel.com>
On Fri, Jan 13, 2017 at 11:32:29AM +0000, Ferruh Yigit wrote:
> On 1/13/2017 8:17 AM, Jerin Jacob wrote:
> > On Thu, Jan 12, 2017 at 07:09:22PM +0000, Ferruh Yigit wrote:
> >> Hi Jerin,
> >>
> >> On 1/12/2017 9:17 AM, Jerin Jacob wrote:
> >> <...>
> >>
> >>> +#include <rte_io.h>
> >>> +
> >>> /* CSR write macro */
> >>> -#define ADF_CSR_WR(csrAddr, csrOffset, val) \
> >>> - (void)((*((volatile uint32_t *)(((uint8_t *)csrAddr) + csrOffset)) \
> >>> - = (val)))
> >>> +#define ADF_CSR_WR(csrAddr, csrOffset, val) \
> >>> + rte_write32(val, (((uint8_t *)csrAddr) + csrOffset))
> >>
> >> For IA, this update introduces an extra compiler barrier (rte_io_wmb()),
> >> which is indeed not a must, is this correct?
> >
> > AFAIK, Compiler barrier is required for IA. I am not an IA expert, if
> > someone thinks it needs to changed then I can fix it in following commit
> > in this patch series by making rte_io_wmb() and rte_io_rmb() as empty.
> >
> > Let me know.
> >
> > AFAIK, Linux kernel code has a barrier in readl/writel for IA.
> >
> > Typically we don't use any non relaxed versions in fast path.In fast
> > typically all the drivers has explicit write barrier for doorbell write
> > and followed by a relaxed version of write. IMO, In any event, it won't
> > generate performance regression.
> >
> > [dpdk-master] $ git show
> > 70c343bdc8c33a51a9db23cd58122bdfc120a58f
> > commit 70c343bdc8c33a51a9db23cd58122bdfc120a58f
> > Author: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > Date: Mon Dec 5 06:36:49 2016 +0530
> >
> > eal/x86: define I/O device memory barriers for IA
> >
> > The patch does not provide any functional change for IA.
> > I/O barriers are mapped to existing smp barriers.
> >
> > CC: Bruce Richardson <bruce.richardson@intel.com>
> > CC: Konstantin Ananyev <konstantin.ananyev@intel.com>
> > Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> >
> > diff --git a/lib/librte_eal/common/include/arch/x86/rte_atomic.h
> > b/lib/librte_eal/common/include/arch/x86/rte_atomic.h
> > index 00b1cdf..4eac666 100644
> > --- a/lib/librte_eal/common/include/arch/x86/rte_atomic.h
> > +++ b/lib/librte_eal/common/include/arch/x86/rte_atomic.h
> > @@ -61,6 +61,12 @@ extern "C" {
> >
> > #define rte_smp_rmb() rte_compiler_barrier()
> >
> > +#define rte_io_mb() rte_mb()
> > +
> > +#define rte_io_wmb() rte_compiler_barrier()
> > +
> > +#define rte_io_rmb() rte_compiler_barrier()
> > +
> > /*------------------------- 16 bit atomic operations
> > * -------------------------*/
> >
> > #ifndef RTE_FORCE_INTRINSICS
> >
> >>
> >> If so, does it make sense to override these functions for x86, and make
> >> rte_writeX = rte_writeX_relaxed
> >> rte_readX = rte_readX_relaxed
> >>
> >>>
> >>> /* CSR read macro */
> >>> -#define ADF_CSR_RD(csrAddr, csrOffset) \
> >>> - (*((volatile uint32_t *)(((uint8_t *)csrAddr) + csrOffset)))
> >>> +#define ADF_CSR_RD(csrAddr, csrOffset) \
> >>> + rte_read32((((uint8_t *)csrAddr) + csrOffset))
> >>
> >> This patchset both introduces new rte_readX/rte_writeX functions, also
> >> applies them into drivers.
> >>
> >> While applying them, it changes the behavior.
> >> Like above code was doing a read, but after update it does read and
> >> read_memory_barrier.
> >>
> >> What do you think this patchset updates usage in a manner that keeps
> >> behavior exact same. Like using rte_read32_relaxed for this case.
> >> And doing architecture related updates in a different patchset?
> >
> > Need to use rte_read32 at this commit otherwise it will break for ARM.
> > That's was all point for this patchset.
>
> Why it breaks the ARM, is it because rte_*mb() updated for ARM in this
> patchset (patch 7/29) ?
Yes.
>
> I believe it is good to make these modifications in two phase:
It is in two phases only. First introduced the API with implementation and
enabled in each driver. Why did you think other-way around it is better?
I can rework and test if there is any value addition. If you concerned
about git bisect ability then I don't think we are loosing that in this
model.
Thoughts?
> - First replace old usage with rte_readX/rte_writeX while keeping exact
> same behavior
>
> - Second, do architecture specific changes. Both in eal and drivers
> level if required.
>
> Thanks,
> ferruh
>
> > For performance regression, we can always verify by taking delta
> > between this changeset and the previous changeset. If you think, I need
> > to make rte_io_wmb()/rte_io_rmb() as empty for IA then I could do that
> > as well.
> >
> >
> >>
> >> This both makes easy to see architecture specific updates, and makes
> >> easy to trace any possible performance issues by this patchset.
> >>
> >>>
> >>> #define ADF_BANK_INT_SRC_SEL_MASK_0 0x4444444CUL
> >>> #define ADF_BANK_INT_SRC_SEL_MASK_X 0x44444444UL
> >>>
> >>
>
^ permalink raw reply
* Re: [RFC PATCH 00/11] generalise rte_ring to allow different datatypes
From: Bruce Richardson @ 2017-01-13 15:00 UTC (permalink / raw)
To: Olivier Matz; +Cc: dev
In-Reply-To: <20170113152334.5efda35a@platinum>
On Fri, Jan 13, 2017 at 03:23:34PM +0100, Olivier Matz wrote:
> Hi Bruce,
>
> On Wed, 11 Jan 2017 15:05:14 +0000, Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> > The rte_ring library in DPDK provides an excellent high-performance
> > mechanism which can be used for passing pointers between cores and
> > for other tasks such as buffering. However, it does have a number
> > of limitations:
> >
> > * type information of pointers is lost, as it works with void pointers
> > * typecasting is needed when using enqueue/dequeue burst functions,
> > since arrays of other types cannot be automatically cast to void **
> > * the data to be passed through the ring itself must be no bigger than
> > a pointer
> >
> > While the first two limitations are an inconvenience, the final one is
> > one that can prevent use of rte_rings in cases where their
> > functionality is needed. The use-case which has inspired the patchset
> > is that of eventdev. When working with rte_events, each event is a
> > 16-byte structure consisting of a pointer and some metadata e.g.
> > priority and type. For these events, what is passed around between
> > cores is not pointers to events, but the events themselves. This
> > makes existing rings unsuitable for use by applications working with
> > rte_events, and also for use internally inside any software
> > implementation of an eventdev.
> >
> > For rings to handle events or other similarly sized structures, e.g.
> > NIC descriptors, etc., we then have two options - duplicate rte_ring
> > code to create new ring implementations for each of those types, or
> > generalise the existing code using macros so that the data type
> > handled by each rings is a compile time paramter. This patchset takes
> > the latter approach, and once applied would allow us to add an
> > rte_event_ring type to DPDK using a header file containing:
> >
> > #define RING_TYPE struct rte_event
> > #define RING_TYPE_NAME rte_event
> > #include <rte_typed_ring.h>
> > #undef RING_TYPE_NAME
> > #undef RING_TYPE
> >
> > [NOTE: the event_ring is not defined in this set, since it depends on
> > the eventdev implementation not present in the main tree]
> >
> > If we want to elimiate some of the typecasting on our code when
> > enqueuing and dequeuing mbuf pointers, an rte_mbuf_ring type can be
> > similarly created using the same number of lines of code.
> >
> > The downside of this generalisation is that the code for the rings now
> > has far more use of macros in it. However, I do not feel that overall
> > readability suffers much from this change, the since the changes are
> > pretty much just search-replace onces. There should also be no ABI
> > compatibility issues with this change, since the existing rte_ring
> > structures remain the same.
>
> I didn't dive deeply in the patches, just had a quick look. I
> understand the need, and even if I really don't like the "#define +
> #include" way to create a new specific ring (for readability,
> grepability), that may be a solution to your problem.
>
> I think using a similar approach than in sys/queue.h would be even
> worse in terms of readability.
>
>
> What do you think about the following approach?
>
> - add a new elt_size in rte_ring structure
>
> - update create/enqueue/dequeue/... functions to manage the elt size
>
> - change:
> rte_ring_enqueue_bulk(struct rte_ring *r,
> void * const *obj_table, unsigned n)
> to:
> rte_ring_enqueue_bulk(struct rte_ring *r, void *obj_table,
> unsigned n)
>
> This relaxes the type for the API in the function. In the caller,
> the type of obj_table would be:
> - (void **) in case of a ring of pointers
> - (uint8_t *) in case of a ring of uint8_t
> - (struct rte_event *) in case of a ring of rte_event
> ...
>
> I think (I have not tested it) it won't break compilation since
> any type can be implicitly casted into a void *. Also, I'd say it
> is possible to avoid breaking the ABI.
>
> - deprecate or forbid calls to:
> rte_ring_mp_enqueue(struct rte_ring *r, void *obj)
> (and similar)
>
> Because with a ring of pointers, obj is the pointer, passed by value.
> For other types, we would need
> rte_ring_mp_enqueue(struct rte_ring *r, <TYPE> obj)
>
> Maybe we could consider using a macro here.
>
>
> The drawbacks I see are:
> - a dynamic elt_size may slightly decrease performance
> - it still uses casts to (void *), so there is no type checking
>
Hi Olivier,
Thanks for the feedback.
Yes, I thought about that parameterized sizes solution too, but I did
not pursue it primarily because I was worried about the performance
hits. It would mean that the actual copies of the data elements would
have to be done via memcpy calls - or switches based on size - rather
than assignments, as now. Given that all these calls to enqueue/dequeue
are inlined, that could really hurt performance, as the size of the
elements to be copied are unknown to the compiler at compile time - as
the size is stored in the struct, and not available from the API call.
The compiler type-checking, I really like, being a believer in having the
compiler do as much work as possible for us, but it is a nice-to-have
rather than a mandatory requirement. :-)
Am I right in assuming that the main issue that you see with the patch
is the use of macros may lead to problems with maintainability with the
code?
For me, while macros may not be the nicest solution to the problem:
* it does keep everything in rte_ring exactly as it was before - no API
and ABI issues
* it should be completely hidden from the end user - most applications
should never need to use the typed ring directly. Instead apps should
instead use rte_ring and rte_event_ring headers.
* The majority of the code is still regular C, and the macros don't
effect readability much IMHO. Also, it's comparatively rare that there
are changes being made to the ring library. [Though I have a few
follow-on ideas myself!].
* It gives us the maximum support from the compiler for type checking
and error reporting based on that
This patchset is not for 17.02 so we have some time to consider our
options, though I would like some resolution on this early in the 17.05
timeframe so as to reuse any solution inside any software eventdevs we
create.
Regards,
/Bruce
^ permalink raw reply
* [PATCH] net/nfp: fix VLAN offload flags check
From: Ferruh Yigit @ 2017-01-13 15:16 UTC (permalink / raw)
To: dev; +Cc: stable, Alejandro Lucero, Ferruh Yigit, Olivier Matz
In-Reply-To: <CAD+H993rSwxCyKQBF=PQdp_SCe-tyx72yprNXYJO=qtTNjk9JA@mail.gmail.com>
From: Olivier Matz <olivier.matz@6wind.com>
Fix typo when checking that no VLAN offload flags are passed at port
initialization.
By the way, also fix a typo in the log.
Fixes: d4a27a3b092a ("nfp: add basic features")
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Alejandro Lucero <alejandro.lucero@netronome.com>
---
drivers/net/nfp/nfp_net.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
index 31e72e2..ab2d331 100644
--- a/drivers/net/nfp/nfp_net.c
+++ b/drivers/net/nfp/nfp_net.c
@@ -2174,9 +2174,9 @@ nfp_net_vlan_offload_set(struct rte_eth_dev *dev, int mask)
new_ctrl = 0;
if ((mask & ETH_VLAN_FILTER_OFFLOAD) ||
- (mask & ETH_VLAN_FILTER_OFFLOAD))
- RTE_LOG(INFO, PMD, "Not support for ETH_VLAN_FILTER_OFFLOAD or"
- " ETH_VLAN_FILTER_EXTEND");
+ (mask & ETH_VLAN_EXTEND_OFFLOAD))
+ RTE_LOG(INFO, PMD, "No support for ETH_VLAN_FILTER_OFFLOAD or"
+ " ETH_VLAN_EXTEND_OFFLOAD");
/* Enable vlan strip if it is not configured yet */
if ((mask & ETH_VLAN_STRIP_OFFLOAD) &&
--
2.9.3
^ permalink raw reply related
* Re: [PATCH] net/nfp: fix VLAN offload flags check
From: Ferruh Yigit @ 2017-01-13 15:18 UTC (permalink / raw)
To: dev; +Cc: stable, Alejandro Lucero, Olivier Matz
In-Reply-To: <20170113151625.18656-1-ferruh.yigit@intel.com>
On 1/13/2017 3:16 PM, Ferruh Yigit wrote:
> From: Olivier Matz <olivier.matz@6wind.com>
>
> Fix typo when checking that no VLAN offload flags are passed at port
> initialization.
>
> By the way, also fix a typo in the log.
>
> Fixes: d4a27a3b092a ("nfp: add basic features")
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> Acked-by: Alejandro Lucero <alejandro.lucero@netronome.com>
Applied to dpdk-next-net/master, thanks.
(I forget about stable tree, that is why need to re-send latest patch)
^ permalink raw reply
* Re: [PATCH v4 1/6] lib: distributor performance enhancements
From: Bruce Richardson @ 2017-01-13 15:19 UTC (permalink / raw)
To: David Hunt; +Cc: dev
In-Reply-To: <1483948248-91364-2-git-send-email-david.hunt@intel.com>
On Mon, Jan 09, 2017 at 07:50:43AM +0000, David Hunt wrote:
> Now sends bursts of up to 8 mbufs to each worker, and tracks
> the in-flight flow-ids (atomic scheduling)
>
> New file with a new api, similar to the old API except with _burst
> at the end of the function names
>
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---
> lib/librte_distributor/Makefile | 2 +
> lib/librte_distributor/rte_distributor.c | 72 +--
> lib/librte_distributor/rte_distributor_burst.c | 558 +++++++++++++++++++++
> lib/librte_distributor/rte_distributor_burst.h | 255 ++++++++++
> lib/librte_distributor/rte_distributor_priv.h | 189 +++++++
> lib/librte_distributor/rte_distributor_version.map | 9 +
> 6 files changed, 1014 insertions(+), 71 deletions(-)
> create mode 100644 lib/librte_distributor/rte_distributor_burst.c
> create mode 100644 lib/librte_distributor/rte_distributor_burst.h
> create mode 100644 lib/librte_distributor/rte_distributor_priv.h
>
Run a documentation sanity check after this patch throws up a few
warnings:
--- /dev/null 2017-01-10 10:26:01.206201474 +0000
+++ /tmp/doc-check/doc.txt 2017-01-13 15:19:50.717102848 +0000
@@ -0,0 +1,6 @@
+/home/bruce/dpdk-clean/lib/librte_distributor/rte_distributor_burst.h:187:
warning: argument 'mbuf' of command @param is not found in the argument
list of rte_distributor_return_pkt_burst(struct rte_distributor_burst
*d, unsigned int worker_id, struct rte_mbuf **oldpkt, int num)
+/home/bruce/dpdk-clean/lib/librte_distributor/rte_distributor_burst.h:199:
warning: The following parameters of
rte_distributor_return_pkt_burst(struct rte_distributor_burst *d,
unsigned int worker_id, struct rte_mbuf **oldpkt, int num) are not
documented:
+ parameter 'oldpkt'
+ parameter 'num'
+/home/bruce/dpdk-clean/lib/librte_distributor/rte_distributor_priv.h:73:
warning: Found unknown command `\in_flight_bitmask'
+/home/bruce/dpdk-clean/lib/librte_distributor/rte_distributor_priv.h:73:
warning: Found unknown command `\rte_distributor_process'
Regards,
/Bruce
^ permalink raw reply
* Re: [PATCH v4] mempool: use cache in single producer or consumer mode
From: Olivier Matz @ 2017-01-13 15:23 UTC (permalink / raw)
To: Wenfeng Liu; +Cc: konstantin.ananyev, dev
In-Reply-To: <1484101528-8144-1-git-send-email-liuwf@arraynetworks.com.cn>
On Wed, 11 Jan 2017 02:25:28 +0000, Wenfeng Liu
<liuwf@arraynetworks.com.cn> wrote:
> Currently we will check mempool flags when we put/get objects from
> mempool. However, this makes cache useless when mempool is SC|SP,
> SC|MP, MC|SP cases.
> This patch makes cache available in above cases and improves
> performance.
>
> Signed-off-by: Wenfeng Liu <liuwf@arraynetworks.com.cn>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
^ permalink raw reply
* [PATCH v5 0/3] Chained Mbufs support in SW PMDs
From: Tomasz Kulasek @ 2017-01-13 15:23 UTC (permalink / raw)
To: dev
In-Reply-To: <1483634768-35012-1-git-send-email-tomaszx.kulasek@intel.com>
This patch set adds support of scattered-gather list for SW PMDs.
As of now, application needs to reserve continuous block of memory for
mbufs which is not always the case. Hence needed to support chaining of
mbufs which are smaller in size but can be used if chained.
Above work involves:
--------------------
a) Create mbuf functions to coalesce mbuf chains into a single mbuf.
b) For each software poll mode driver code to detect chained mbufs
support and coalesce these before preforming crypto.
c) Add relevant unit tests to test the functionality.
Known limitations for openssl PMD:
----------------------------------
While libcrypto library expects continuous destination buffer for
output of cipher operations, implementation of openssl PMD is limited
the same way, and requires contiguous destination mbuf.
changes in v5:
- rebased to the master of dpdk-next-crypto
changes in v4:
- separated "rte_pktmbuf_linearize" implementation from this patch set
and sent as new patch for better reviewing,
changes in v3:
- rebased to dpdk-next-crypto
- reused tests for AES GCM SGL support in opensll from "app/test: add
SGL tests to cryptodev QAT suite"
changes in v2:
- add support for sgl in openssl PMD
- rte_pktmbuf_coalesce replaced with rte_pktmbuf_linearize
- extended test vector data for aes gcm from 60 to 2048 bytes
Tomasz Kulasek (3):
crypto: add sgl support in sw PMDs
crypto: add sgl support in openssl PMD
test: add sgl unit tests for crypto devices
app/test/test_cryptodev.c | 386 ++++++++++++++++++-
app/test/test_cryptodev.h | 138 +++++++
app/test/test_cryptodev_aes_test_vectors.h | 52 +++
app/test/test_cryptodev_blockcipher.c | 90 ++---
app/test/test_cryptodev_blockcipher.h | 1 +
app/test/test_cryptodev_gcm_test_vectors.h | 553 ++++++++++++++++++++++++++++
doc/guides/cryptodevs/openssl.rst | 3 +-
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c | 14 +
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 19 +-
drivers/crypto/kasumi/rte_kasumi_pmd.c | 13 +
drivers/crypto/null/null_crypto_pmd.c | 3 +-
drivers/crypto/openssl/rte_openssl_pmd.c | 329 +++++++++++++----
drivers/crypto/snow3g/rte_snow3g_pmd.c | 15 +
drivers/crypto/zuc/rte_zuc_pmd.c | 13 +
lib/librte_cryptodev/rte_cryptodev.c | 4 +-
lib/librte_cryptodev/rte_cryptodev.h | 2 +
16 files changed, 1507 insertions(+), 128 deletions(-)
--
1.7.9.5
^ permalink raw reply
* Re: [PATCH v4 2/6] lib: add distributor vector flow matching
From: Bruce Richardson @ 2017-01-13 15:26 UTC (permalink / raw)
To: David Hunt; +Cc: dev
In-Reply-To: <1483948248-91364-3-git-send-email-david.hunt@intel.com>
On Mon, Jan 09, 2017 at 07:50:44AM +0000, David Hunt wrote:
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---
> lib/librte_distributor/Makefile | 4 +
> lib/librte_distributor/rte_distributor_burst.c | 11 +-
> lib/librte_distributor/rte_distributor_match_sse.c | 113 +++++++++++++++++++++
> lib/librte_distributor/rte_distributor_priv.h | 6 ++
> 4 files changed, 133 insertions(+), 1 deletion(-)
> create mode 100644 lib/librte_distributor/rte_distributor_match_sse.c
>
> diff --git a/lib/librte_distributor/Makefile b/lib/librte_distributor/Makefile
> index 2acc54d..a725aaf 100644
> --- a/lib/librte_distributor/Makefile
> +++ b/lib/librte_distributor/Makefile
> @@ -44,6 +44,10 @@ LIBABIVER := 1
> # all source are stored in SRCS-y
> SRCS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) := rte_distributor.c
> SRCS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += rte_distributor_burst.c
> +ifeq ($(CONFIG_RTE_ARCH_X86),y)
> +SRCS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += rte_distributor_match_sse.c
> +endif
> +
>
I believe some of the intrinsics used in the vector code are SSE4.2
instructions, so you need to pass that flag for the compilation for e.g.
the "default" target for packaging into distros.
> # install this header file
> SYMLINK-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR)-include := rte_distributor.h
> diff --git a/lib/librte_distributor/rte_distributor_burst.c b/lib/librte_distributor/rte_distributor_burst.c
> index ae7cf9d..35044c4 100644
> --- a/lib/librte_distributor/rte_distributor_burst.c
> +++ b/lib/librte_distributor/rte_distributor_burst.c
> @@ -352,6 +352,9 @@ rte_distributor_process_burst(struct rte_distributor_burst *d,
> }
>
> switch (d->dist_match_fn) {
> + case RTE_DIST_MATCH_VECTOR:
> + find_match_vec(d, &flows[0], &matches[0]);
> + break;
> default:
> find_match_scalar(d, &flows[0], &matches[0]);
> }
> @@ -538,7 +541,13 @@ rte_distributor_create_burst(const char *name,
> snprintf(d->name, sizeof(d->name), "%s", name);
> d->num_workers = num_workers;
>
> - d->dist_match_fn = RTE_DIST_MATCH_SCALAR;
> +#if defined(RTE_ARCH_X86)
> + if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_SSE2)) {
> + d->dist_match_fn = RTE_DIST_MATCH_VECTOR;
> + } else {
> +#endif
> + d->dist_match_fn = RTE_DIST_MATCH_SCALAR;
> + }
>
Two issues here:
1) the check needs to be for SSE4.2, not SSE2 [minimum for DPDK on x86
is SSE3 anyway, so no need for any checks for SSE2]
2) The closing brace should be ifdefed out to fix compilation on non-x86
platforms. A simpler/better solution might actually be to remove the
braces since only a single line is involved in each branch.
Regards,
/Bruce
^ permalink raw reply
* [PATCH v5 1/3] crypto: add sgl support in sw PMDs
From: Tomasz Kulasek @ 2017-01-13 15:23 UTC (permalink / raw)
To: dev
In-Reply-To: <1484320997-1224-1-git-send-email-tomaszx.kulasek@intel.com>
This patch introduces RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER feature flag
informing that selected crypto device supports segmented mbufs natively
and doesn't need to be coalesced before crypto operation.
While using segmented buffers in crypto devices may have unpredictable
results, for PMDs which doesn't support it natively, additional check is
made for debug compilation.
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
---
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c | 14 ++++++++++++++
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 19 ++++++++++++++++---
drivers/crypto/kasumi/rte_kasumi_pmd.c | 13 +++++++++++++
drivers/crypto/null/null_crypto_pmd.c | 3 ++-
drivers/crypto/snow3g/rte_snow3g_pmd.c | 15 +++++++++++++++
drivers/crypto/zuc/rte_zuc_pmd.c | 13 +++++++++++++
lib/librte_cryptodev/rte_cryptodev.c | 4 ++--
lib/librte_cryptodev/rte_cryptodev.h | 2 ++
8 files changed, 77 insertions(+), 6 deletions(-)
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index af3d60f..5af22f7 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -377,6 +377,20 @@
break;
}
+#ifdef RTE_LIBRTE_PMD_AESNI_GCM_DEBUG
+ if (!rte_pktmbuf_is_contiguous(ops[i]->sym->m_src) ||
+ (ops[i]->sym->m_dst != NULL &&
+ !rte_pktmbuf_is_contiguous(
+ ops[i]->sym->m_dst))) {
+ ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+ GCM_LOG_ERR("PMD supports only contiguous mbufs, "
+ "op (%p) provides noncontiguous mbuf as "
+ "source/destination buffer.\n", ops[i]);
+ qp->qp_stats.enqueue_err_count++;
+ break;
+ }
+#endif
+
retval = process_gcm_crypto_op(qp, ops[i]->sym, sess);
if (retval < 0) {
ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
index 6d27d75..25f681b 100644
--- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
+++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
@@ -571,15 +571,28 @@
int i, processed_jobs = 0;
for (i = 0; i < nb_ops; i++) {
-#ifdef RTE_LIBRTE_AESNI_MB_DEBUG
- if (unlikely(op->type != RTE_CRYPTO_OP_TYPE_SYMMETRIC)) {
+#ifdef RTE_LIBRTE_PMD_AESNI_MB_DEBUG
+ if (unlikely(ops[i]->type != RTE_CRYPTO_OP_TYPE_SYMMETRIC)) {
MB_LOG_ERR("PMD only supports symmetric crypto "
"operation requests, op (%p) is not a "
- "symmetric operation.", op);
+ "symmetric operation.", ops[i]);
+ qp->stats.enqueue_err_count++;
+ goto flush_jobs;
+ }
+
+ if (!rte_pktmbuf_is_contiguous(ops[i]->sym->m_src) ||
+ (ops[i]->sym->m_dst != NULL &&
+ !rte_pktmbuf_is_contiguous(
+ ops[i]->sym->m_dst))) {
+ MB_LOG_ERR("PMD supports only contiguous mbufs, "
+ "op (%p) provides noncontiguous mbuf as "
+ "source/destination buffer.\n", ops[i]);
+ ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
qp->stats.enqueue_err_count++;
goto flush_jobs;
}
#endif
+
sess = get_session(qp, ops[i]);
if (unlikely(sess == NULL)) {
qp->stats.enqueue_err_count++;
diff --git a/drivers/crypto/kasumi/rte_kasumi_pmd.c b/drivers/crypto/kasumi/rte_kasumi_pmd.c
index b119da2..4bdd7bb 100644
--- a/drivers/crypto/kasumi/rte_kasumi_pmd.c
+++ b/drivers/crypto/kasumi/rte_kasumi_pmd.c
@@ -455,6 +455,19 @@
for (i = 0; i < nb_ops; i++) {
curr_c_op = ops[i];
+#ifdef RTE_LIBRTE_PMD_KASUMI_DEBUG
+ if (!rte_pktmbuf_is_contiguous(curr_c_op->sym->m_src) ||
+ (curr_c_op->sym->m_dst != NULL &&
+ !rte_pktmbuf_is_contiguous(
+ curr_c_op->sym->m_dst))) {
+ KASUMI_LOG_ERR("PMD supports only contiguous mbufs, "
+ "op (%p) provides noncontiguous mbuf as "
+ "source/destination buffer.\n", curr_c_op);
+ curr_c_op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+ break;
+ }
+#endif
+
/* Set status as enqueued (not processed yet) by default. */
curr_c_op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED;
diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c
index c69606b..c37d3d6 100644
--- a/drivers/crypto/null/null_crypto_pmd.c
+++ b/drivers/crypto/null/null_crypto_pmd.c
@@ -216,7 +216,8 @@
dev->enqueue_burst = null_crypto_pmd_enqueue_burst;
dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
- RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING;
+ RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
+ RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER;
internals = dev->data->dev_private;
diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c
index 3b4292a..9a6f16d 100644
--- a/drivers/crypto/snow3g/rte_snow3g_pmd.c
+++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c
@@ -330,6 +330,21 @@
unsigned i;
unsigned enqueued_ops, processed_ops;
+#ifdef RTE_LIBRTE_PMD_SNOW3G_DEBUG
+ for (i = 0; i < num_ops; i++) {
+ if (!rte_pktmbuf_is_contiguous(ops[i]->sym->m_src) ||
+ (ops[i]->sym->m_dst != NULL &&
+ !rte_pktmbuf_is_contiguous(
+ ops[i]->sym->m_dst))) {
+ SNOW3G_LOG_ERR("PMD supports only contiguous mbufs, "
+ "op (%p) provides noncontiguous mbuf as "
+ "source/destination buffer.\n", ops[i]);
+ ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+ return 0;
+ }
+ }
+#endif
+
switch (session->op) {
case SNOW3G_OP_ONLY_CIPHER:
processed_ops = process_snow3g_cipher_op(ops,
diff --git a/drivers/crypto/zuc/rte_zuc_pmd.c b/drivers/crypto/zuc/rte_zuc_pmd.c
index 3849119..bf53f76 100644
--- a/drivers/crypto/zuc/rte_zuc_pmd.c
+++ b/drivers/crypto/zuc/rte_zuc_pmd.c
@@ -211,6 +211,19 @@
break;
}
+#ifdef RTE_LIBRTE_PMD_ZUC_DEBUG
+ if (!rte_pktmbuf_is_contiguous(ops[i]->sym->m_src) ||
+ (ops[i]->sym->m_dst != NULL &&
+ !rte_pktmbuf_is_contiguous(
+ ops[i]->sym->m_dst))) {
+ ZUC_LOG_ERR("PMD supports only contiguous mbufs, "
+ "op (%p) provides noncontiguous mbuf as "
+ "source/destination buffer.\n", ops[i]);
+ ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+ break;
+ }
+#endif
+
src[i] = rte_pktmbuf_mtod(ops[i]->sym->m_src, uint8_t *) +
(ops[i]->sym->cipher.data.offset >> 3);
dst[i] = ops[i]->sym->m_dst ?
diff --git a/lib/librte_cryptodev/rte_cryptodev.c b/lib/librte_cryptodev/rte_cryptodev.c
index 54e95d5..bbab4b3 100644
--- a/lib/librte_cryptodev/rte_cryptodev.c
+++ b/lib/librte_cryptodev/rte_cryptodev.c
@@ -211,13 +211,13 @@ struct rte_cryptodev_callback {
return "CPU_AESNI";
case RTE_CRYPTODEV_FF_HW_ACCELERATED:
return "HW_ACCELERATED";
-
+ case RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER:
+ return "MBUF_SCATTER_GATHER";
default:
return NULL;
}
}
-
int
rte_cryptodev_create_vdev(const char *name, const char *args)
{
diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
index 29d8eec..fa311a9 100644
--- a/lib/librte_cryptodev/rte_cryptodev.h
+++ b/lib/librte_cryptodev/rte_cryptodev.h
@@ -227,6 +227,8 @@ struct rte_cryptodev_capabilities {
/**< Operations are off-loaded to an external hardware accelerator */
#define RTE_CRYPTODEV_FF_CPU_AVX512 (1ULL << 8)
/**< Utilises CPU SIMD AVX512 instructions */
+#define RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER (1ULL << 9)
+/**< Scatter-gather mbufs are supported */
/**
--
1.7.9.5
^ permalink raw reply related
* [PATCH v5 2/3] crypto: add sgl support in openssl PMD
From: Tomasz Kulasek @ 2017-01-13 15:23 UTC (permalink / raw)
To: dev
In-Reply-To: <1484320997-1224-1-git-send-email-tomaszx.kulasek@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
---
doc/guides/cryptodevs/openssl.rst | 3 +-
drivers/crypto/openssl/rte_openssl_pmd.c | 329 +++++++++++++++++++++++-------
2 files changed, 259 insertions(+), 73 deletions(-)
diff --git a/doc/guides/cryptodevs/openssl.rst b/doc/guides/cryptodevs/openssl.rst
index f1c39ba..f6ed6ea 100644
--- a/doc/guides/cryptodevs/openssl.rst
+++ b/doc/guides/cryptodevs/openssl.rst
@@ -112,6 +112,7 @@ Limitations
-----------
* Maximum number of sessions is 2048.
-* Chained mbufs are not supported.
+* Chained mbufs are supported only for source mbuf (destination must be
+ contiguous).
* Hash only is not supported for GCM and GMAC.
* Cipher only is not supported for GCM and GMAC.
diff --git a/drivers/crypto/openssl/rte_openssl_pmd.c b/drivers/crypto/openssl/rte_openssl_pmd.c
index 312154a..426e407 100644
--- a/drivers/crypto/openssl/rte_openssl_pmd.c
+++ b/drivers/crypto/openssl/rte_openssl_pmd.c
@@ -484,24 +484,112 @@
* Process Operations
*------------------------------------------------------------------------------
*/
+static inline int
+process_openssl_encryption_update(struct rte_mbuf *mbuf_src, int offset,
+ uint8_t **dst, int srclen, EVP_CIPHER_CTX *ctx)
+{
+ struct rte_mbuf *m;
+ int dstlen;
+ int l, n = srclen;
+ uint8_t *src;
+
+ for (m = mbuf_src; m != NULL && offset > rte_pktmbuf_data_len(m);
+ m = m->next)
+ offset -= rte_pktmbuf_data_len(m);
+
+ if (m == 0)
+ return -1;
+
+ src = rte_pktmbuf_mtod_offset(m, uint8_t *, offset);
+
+ l = rte_pktmbuf_data_len(m) - offset;
+ if (srclen <= l) {
+ if (EVP_EncryptUpdate(ctx, *dst, &dstlen, src, srclen) <= 0)
+ return -1;
+ *dst += l;
+ return 0;
+ }
+
+ if (EVP_EncryptUpdate(ctx, *dst, &dstlen, src, l) <= 0)
+ return -1;
+
+ *dst += dstlen;
+ n -= l;
+
+ for (m = m->next; (m != NULL) && (n > 0); m = m->next) {
+ src = rte_pktmbuf_mtod(m, uint8_t *);
+ l = rte_pktmbuf_data_len(m) < n ? rte_pktmbuf_data_len(m) : n;
+ if (EVP_EncryptUpdate(ctx, *dst, &dstlen, src, l) <= 0)
+ return -1;
+ *dst += dstlen;
+ n -= l;
+ }
+
+ return 0;
+}
+
+static inline int
+process_openssl_decryption_update(struct rte_mbuf *mbuf_src, int offset,
+ uint8_t **dst, int srclen, EVP_CIPHER_CTX *ctx)
+{
+ struct rte_mbuf *m;
+ int dstlen;
+ int l, n = srclen;
+ uint8_t *src;
+
+ for (m = mbuf_src; m != NULL && offset > rte_pktmbuf_data_len(m);
+ m = m->next)
+ offset -= rte_pktmbuf_data_len(m);
+
+ if (m == 0)
+ return -1;
+
+ src = rte_pktmbuf_mtod_offset(m, uint8_t *, offset);
+
+ l = rte_pktmbuf_data_len(m) - offset;
+ if (srclen <= l) {
+ if (EVP_DecryptUpdate(ctx, *dst, &dstlen, src, srclen) <= 0)
+ return -1;
+ *dst += l;
+ return 0;
+ }
+
+ if (EVP_DecryptUpdate(ctx, *dst, &dstlen, src, l) <= 0)
+ return -1;
+
+ *dst += dstlen;
+ n -= l;
+
+ for (m = m->next; (m != NULL) && (n > 0); m = m->next) {
+ src = rte_pktmbuf_mtod(m, uint8_t *);
+ l = rte_pktmbuf_data_len(m) < n ? rte_pktmbuf_data_len(m) : n;
+ if (EVP_DecryptUpdate(ctx, *dst, &dstlen, src, l) <= 0)
+ return -1;
+ *dst += dstlen;
+ n -= l;
+ }
+
+ return 0;
+}
/** Process standard openssl cipher encryption */
static int
-process_openssl_cipher_encrypt(uint8_t *src, uint8_t *dst,
- uint8_t *iv, uint8_t *key, int srclen,
+process_openssl_cipher_encrypt(struct rte_mbuf *mbuf_src, uint8_t *dst,
+ int offset, uint8_t *iv, uint8_t *key, int srclen,
EVP_CIPHER_CTX *ctx, const EVP_CIPHER *algo)
{
- int dstlen, totlen;
+ int totlen;
if (EVP_EncryptInit_ex(ctx, algo, NULL, key, iv) <= 0)
goto process_cipher_encrypt_err;
EVP_CIPHER_CTX_set_padding(ctx, 0);
- if (EVP_EncryptUpdate(ctx, dst, &dstlen, src, srclen) <= 0)
+ if (process_openssl_encryption_update(mbuf_src, offset, &dst,
+ srclen, ctx))
goto process_cipher_encrypt_err;
- if (EVP_EncryptFinal_ex(ctx, dst + dstlen, &totlen) <= 0)
+ if (EVP_EncryptFinal_ex(ctx, dst, &totlen) <= 0)
goto process_cipher_encrypt_err;
return 0;
@@ -513,23 +601,23 @@
/** Process standard openssl cipher decryption */
static int
-process_openssl_cipher_decrypt(uint8_t *src, uint8_t *dst,
- uint8_t *iv, uint8_t *key, int srclen,
+process_openssl_cipher_decrypt(struct rte_mbuf *mbuf_src, uint8_t *dst,
+ int offset, uint8_t *iv, uint8_t *key, int srclen,
EVP_CIPHER_CTX *ctx, const EVP_CIPHER *algo)
{
- int dstlen, totlen;
+ int totlen;
if (EVP_DecryptInit_ex(ctx, algo, NULL, key, iv) <= 0)
goto process_cipher_decrypt_err;
EVP_CIPHER_CTX_set_padding(ctx, 0);
- if (EVP_DecryptUpdate(ctx, dst, &dstlen, src, srclen) <= 0)
+ if (process_openssl_decryption_update(mbuf_src, offset, &dst,
+ srclen, ctx))
goto process_cipher_decrypt_err;
- if (EVP_DecryptFinal_ex(ctx, dst + dstlen, &totlen) <= 0)
+ if (EVP_DecryptFinal_ex(ctx, dst, &totlen) <= 0)
goto process_cipher_decrypt_err;
-
return 0;
process_cipher_decrypt_err:
@@ -539,11 +627,25 @@
/** Process cipher des 3 ctr encryption, decryption algorithm */
static int
-process_openssl_cipher_des3ctr(uint8_t *src, uint8_t *dst,
- uint8_t *iv, uint8_t *key, int srclen, EVP_CIPHER_CTX *ctx)
+process_openssl_cipher_des3ctr(struct rte_mbuf *mbuf_src, uint8_t *dst,
+ int offset, uint8_t *iv, uint8_t *key, int srclen,
+ EVP_CIPHER_CTX *ctx)
{
uint8_t ebuf[8], ctr[8];
int unused, n;
+ struct rte_mbuf *m;
+ uint8_t *src;
+ int l;
+
+ for (m = mbuf_src; m != NULL && offset > rte_pktmbuf_data_len(m);
+ m = m->next)
+ offset -= rte_pktmbuf_data_len(m);
+
+ if (m == 0)
+ goto process_cipher_des3ctr_err;
+
+ src = rte_pktmbuf_mtod_offset(m, uint8_t *, offset);
+ l = rte_pktmbuf_data_len(m) - offset;
/* We use 3DES encryption also for decryption.
* IV is not important for 3DES ecb
@@ -552,9 +654,8 @@
goto process_cipher_des3ctr_err;
memcpy(ctr, iv, 8);
- n = 0;
- while (n < srclen) {
+ for (n = 0; n < srclen; n++) {
if (n % 8 == 0) {
if (EVP_EncryptUpdate(ctx,
(unsigned char *)&ebuf, &unused,
@@ -562,8 +663,16 @@
goto process_cipher_des3ctr_err;
ctr_inc(ctr);
}
- dst[n] = src[n] ^ ebuf[n % 8];
- n++;
+ dst[n] = *(src++) ^ ebuf[n % 8];
+
+ l--;
+ if (!l) {
+ m = m->next;
+ if (m) {
+ src = rte_pktmbuf_mtod(m, uint8_t *);
+ l = rte_pktmbuf_data_len(m);
+ }
+ }
}
return 0;
@@ -575,9 +684,9 @@
/** Process auth/encription aes-gcm algorithm */
static int
-process_openssl_auth_encryption_gcm(uint8_t *src, int srclen,
- uint8_t *aad, int aadlen, uint8_t *iv, int ivlen,
- uint8_t *key, uint8_t *dst, uint8_t *tag,
+process_openssl_auth_encryption_gcm(struct rte_mbuf *mbuf_src, int offset,
+ int srclen, uint8_t *aad, int aadlen, uint8_t *iv, int ivlen,
+ uint8_t *key, uint8_t *dst, uint8_t *tag,
EVP_CIPHER_CTX *ctx, const EVP_CIPHER *algo)
{
int len = 0, unused = 0;
@@ -592,20 +701,20 @@
if (EVP_EncryptInit_ex(ctx, NULL, NULL, key, iv) <= 0)
goto process_auth_encryption_gcm_err;
- if (aadlen > 0) {
+ if (aadlen > 0)
if (EVP_EncryptUpdate(ctx, NULL, &len, aad, aadlen) <= 0)
goto process_auth_encryption_gcm_err;
- /* Workaround open ssl bug in version less then 1.0.1f */
- if (EVP_EncryptUpdate(ctx, empty, &unused, empty, 0) <= 0)
- goto process_auth_encryption_gcm_err;
- }
-
if (srclen > 0)
- if (EVP_EncryptUpdate(ctx, dst, &len, src, srclen) <= 0)
+ if (process_openssl_encryption_update(mbuf_src, offset, &dst,
+ srclen, ctx))
goto process_auth_encryption_gcm_err;
- if (EVP_EncryptFinal_ex(ctx, dst + len, &len) <= 0)
+ /* Workaround open ssl bug in version less then 1.0.1f */
+ if (EVP_EncryptUpdate(ctx, empty, &unused, empty, 0) <= 0)
+ goto process_auth_encryption_gcm_err;
+
+ if (EVP_EncryptFinal_ex(ctx, dst, &len) <= 0)
goto process_auth_encryption_gcm_err;
if (EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_GET_TAG, 16, tag) <= 0)
@@ -619,10 +728,10 @@
}
static int
-process_openssl_auth_decryption_gcm(uint8_t *src, int srclen,
- uint8_t *aad, int aadlen, uint8_t *iv, int ivlen,
- uint8_t *key, uint8_t *dst, uint8_t *tag,
- EVP_CIPHER_CTX *ctx, const EVP_CIPHER *algo)
+process_openssl_auth_decryption_gcm(struct rte_mbuf *mbuf_src, int offset,
+ int srclen, uint8_t *aad, int aadlen, uint8_t *iv, int ivlen,
+ uint8_t *key, uint8_t *dst, uint8_t *tag, EVP_CIPHER_CTX *ctx,
+ const EVP_CIPHER *algo)
{
int len = 0, unused = 0;
uint8_t empty[] = {};
@@ -639,20 +748,20 @@
if (EVP_DecryptInit_ex(ctx, NULL, NULL, key, iv) <= 0)
goto process_auth_decryption_gcm_err;
- if (aadlen > 0) {
+ if (aadlen > 0)
if (EVP_DecryptUpdate(ctx, NULL, &len, aad, aadlen) <= 0)
goto process_auth_decryption_gcm_err;
- /* Workaround open ssl bug in version less then 1.0.1f */
- if (EVP_DecryptUpdate(ctx, empty, &unused, empty, 0) <= 0)
- goto process_auth_decryption_gcm_err;
- }
-
if (srclen > 0)
- if (EVP_DecryptUpdate(ctx, dst, &len, src, srclen) <= 0)
+ if (process_openssl_decryption_update(mbuf_src, offset, &dst,
+ srclen, ctx))
goto process_auth_decryption_gcm_err;
- if (EVP_DecryptFinal_ex(ctx, dst + len, &len) <= 0)
+ /* Workaround open ssl bug in version less then 1.0.1f */
+ if (EVP_DecryptUpdate(ctx, empty, &unused, empty, 0) <= 0)
+ goto process_auth_decryption_gcm_err;
+
+ if (EVP_DecryptFinal_ex(ctx, dst, &len) <= 0)
goto process_auth_decryption_gcm_final_err;
return 0;
@@ -667,21 +776,50 @@
/** Process standard openssl auth algorithms */
static int
-process_openssl_auth(uint8_t *src, uint8_t *dst,
+process_openssl_auth(struct rte_mbuf *mbuf_src, uint8_t *dst, int offset,
__rte_unused uint8_t *iv, __rte_unused EVP_PKEY * pkey,
int srclen, EVP_MD_CTX *ctx, const EVP_MD *algo)
{
size_t dstlen;
+ struct rte_mbuf *m;
+ int l, n = srclen;
+ uint8_t *src;
+
+ for (m = mbuf_src; m != NULL && offset > rte_pktmbuf_data_len(m);
+ m = m->next)
+ offset -= rte_pktmbuf_data_len(m);
+
+ if (m == 0)
+ goto process_auth_err;
if (EVP_DigestInit_ex(ctx, algo, NULL) <= 0)
goto process_auth_err;
- if (EVP_DigestUpdate(ctx, (char *)src, srclen) <= 0)
+ src = rte_pktmbuf_mtod_offset(m, uint8_t *, offset);
+
+ l = rte_pktmbuf_data_len(m) - offset;
+ if (srclen <= l) {
+ if (EVP_DigestUpdate(ctx, (char *)src, srclen) <= 0)
+ goto process_auth_err;
+ goto process_auth_final;
+ }
+
+ if (EVP_DigestUpdate(ctx, (char *)src, l) <= 0)
goto process_auth_err;
+ n -= l;
+
+ for (m = m->next; (m != NULL) && (n > 0); m = m->next) {
+ src = rte_pktmbuf_mtod(m, uint8_t *);
+ l = rte_pktmbuf_data_len(m) < n ? rte_pktmbuf_data_len(m) : n;
+ if (EVP_DigestUpdate(ctx, (char *)src, l) <= 0)
+ goto process_auth_err;
+ n -= l;
+ }
+
+process_auth_final:
if (EVP_DigestFinal_ex(ctx, dst, (unsigned int *)&dstlen) <= 0)
goto process_auth_err;
-
return 0;
process_auth_err:
@@ -691,18 +829,48 @@
/** Process standard openssl auth algorithms with hmac */
static int
-process_openssl_auth_hmac(uint8_t *src, uint8_t *dst,
+process_openssl_auth_hmac(struct rte_mbuf *mbuf_src, uint8_t *dst, int offset,
__rte_unused uint8_t *iv, EVP_PKEY *pkey,
- int srclen, EVP_MD_CTX *ctx, const EVP_MD *algo)
+ int srclen, EVP_MD_CTX *ctx, const EVP_MD *algo)
{
size_t dstlen;
+ struct rte_mbuf *m;
+ int l, n = srclen;
+ uint8_t *src;
+
+ for (m = mbuf_src; m != NULL && offset > rte_pktmbuf_data_len(m);
+ m = m->next)
+ offset -= rte_pktmbuf_data_len(m);
+
+ if (m == 0)
+ goto process_auth_err;
if (EVP_DigestSignInit(ctx, NULL, algo, NULL, pkey) <= 0)
goto process_auth_err;
- if (EVP_DigestSignUpdate(ctx, (char *)src, srclen) <= 0)
+ src = rte_pktmbuf_mtod_offset(m, uint8_t *, offset);
+
+ l = rte_pktmbuf_data_len(m) - offset;
+ if (srclen <= l) {
+ if (EVP_DigestSignUpdate(ctx, (char *)src, srclen) <= 0)
+ goto process_auth_err;
+ goto process_auth_final;
+ }
+
+ if (EVP_DigestSignUpdate(ctx, (char *)src, l) <= 0)
goto process_auth_err;
+ n -= l;
+
+ for (m = m->next; (m != NULL) && (n > 0); m = m->next) {
+ src = rte_pktmbuf_mtod(m, uint8_t *);
+ l = rte_pktmbuf_data_len(m) < n ? rte_pktmbuf_data_len(m) : n;
+ if (EVP_DigestSignUpdate(ctx, (char *)src, l) <= 0)
+ goto process_auth_err;
+ n -= l;
+ }
+
+process_auth_final:
if (EVP_DigestSignFinal(ctx, dst, &dstlen) <= 0)
goto process_auth_err;
@@ -722,9 +890,18 @@
struct rte_mbuf *mbuf_src, struct rte_mbuf *mbuf_dst)
{
/* cipher */
- uint8_t *src = NULL, *dst = NULL, *iv, *tag, *aad;
+ uint8_t *dst = NULL, *iv, *tag, *aad;
int srclen, ivlen, aadlen, status = -1;
+ /*
+ * Segmented destination buffer is not supported for
+ * encryption/decryption
+ */
+ if (!rte_pktmbuf_is_contiguous(mbuf_dst)) {
+ op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+ return;
+ }
+
iv = op->sym->cipher.iv.data;
ivlen = op->sym->cipher.iv.length;
aad = op->sym->auth.aad.data;
@@ -740,22 +917,22 @@
srclen = 0;
else {
srclen = op->sym->cipher.data.length;
- src = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
- op->sym->cipher.data.offset);
dst = rte_pktmbuf_mtod_offset(mbuf_dst, uint8_t *,
op->sym->cipher.data.offset);
}
if (sess->cipher.direction == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
status = process_openssl_auth_encryption_gcm(
- src, srclen, aad, aadlen, iv, ivlen,
- sess->cipher.key.data, dst, tag,
- sess->cipher.ctx, sess->cipher.evp_algo);
+ mbuf_src, op->sym->cipher.data.offset, srclen,
+ aad, aadlen, iv, ivlen, sess->cipher.key.data,
+ dst, tag, sess->cipher.ctx,
+ sess->cipher.evp_algo);
else
status = process_openssl_auth_decryption_gcm(
- src, srclen, aad, aadlen, iv, ivlen,
- sess->cipher.key.data, dst, tag,
- sess->cipher.ctx, sess->cipher.evp_algo);
+ mbuf_src, op->sym->cipher.data.offset, srclen,
+ aad, aadlen, iv, ivlen, sess->cipher.key.data,
+ dst, tag, sess->cipher.ctx,
+ sess->cipher.evp_algo);
if (status != 0) {
if (status == (-EFAULT) &&
@@ -773,12 +950,19 @@
(struct rte_crypto_op *op, struct openssl_session *sess,
struct rte_mbuf *mbuf_src, struct rte_mbuf *mbuf_dst)
{
- uint8_t *src, *dst, *iv;
+ uint8_t *dst, *iv;
int srclen, status;
+ /*
+ * Segmented destination buffer is not supported for
+ * encryption/decryption
+ */
+ if (!rte_pktmbuf_is_contiguous(mbuf_dst)) {
+ op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+ return;
+ }
+
srclen = op->sym->cipher.data.length;
- src = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
- op->sym->cipher.data.offset);
dst = rte_pktmbuf_mtod_offset(mbuf_dst, uint8_t *,
op->sym->cipher.data.offset);
@@ -786,17 +970,20 @@
if (sess->cipher.mode == OPENSSL_CIPHER_LIB)
if (sess->cipher.direction == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
- status = process_openssl_cipher_encrypt(src, dst, iv,
+ status = process_openssl_cipher_encrypt(mbuf_src, dst,
+ op->sym->cipher.data.offset, iv,
sess->cipher.key.data, srclen,
sess->cipher.ctx,
sess->cipher.evp_algo);
else
- status = process_openssl_cipher_decrypt(src, dst, iv,
+ status = process_openssl_cipher_decrypt(mbuf_src, dst,
+ op->sym->cipher.data.offset, iv,
sess->cipher.key.data, srclen,
sess->cipher.ctx,
sess->cipher.evp_algo);
else
- status = process_openssl_cipher_des3ctr(src, dst, iv,
+ status = process_openssl_cipher_des3ctr(mbuf_src, dst,
+ op->sym->cipher.data.offset, iv,
sess->cipher.key.data, srclen,
sess->cipher.ctx);
@@ -810,12 +997,10 @@
(struct rte_crypto_op *op, struct openssl_session *sess,
struct rte_mbuf *mbuf_src, struct rte_mbuf *mbuf_dst)
{
- uint8_t *src, *dst;
+ uint8_t *dst;
int srclen, status;
srclen = op->sym->auth.data.length;
- src = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
- op->sym->auth.data.offset);
if (sess->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY)
dst = (uint8_t *)rte_pktmbuf_append(mbuf_src,
@@ -830,13 +1015,14 @@
switch (sess->auth.mode) {
case OPENSSL_AUTH_AS_AUTH:
- status = process_openssl_auth(src, dst,
- NULL, NULL, srclen,
+ status = process_openssl_auth(mbuf_src, dst,
+ op->sym->auth.data.offset, NULL, NULL, srclen,
sess->auth.auth.ctx, sess->auth.auth.evp_algo);
break;
case OPENSSL_AUTH_AS_HMAC:
- status = process_openssl_auth_hmac(src, dst,
- NULL, sess->auth.hmac.pkey, srclen,
+ status = process_openssl_auth_hmac(mbuf_src, dst,
+ op->sym->auth.data.offset, NULL,
+ sess->auth.hmac.pkey, srclen,
sess->auth.hmac.ctx, sess->auth.hmac.evp_algo);
break;
default:
@@ -850,8 +1036,7 @@
op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
}
/* Trim area used for digest from mbuf. */
- rte_pktmbuf_trim(mbuf_src,
- op->sym->auth.digest.length);
+ rte_pktmbuf_trim(mbuf_src, op->sym->auth.digest.length);
}
if (status != 0)
@@ -902,7 +1087,6 @@
op->sym->session = NULL;
}
-
if (op->status == RTE_CRYPTO_OP_STATUS_NOT_PROCESSED)
op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
@@ -996,7 +1180,8 @@
dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
- RTE_CRYPTODEV_FF_CPU_AESNI;
+ RTE_CRYPTODEV_FF_CPU_AESNI |
+ RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER;
/* Set vector instructions mode supported */
internals = dev->data->dev_private;
--
1.7.9.5
^ permalink raw reply related
* [PATCH v5 3/3] test: add sgl unit tests for crypto devices
From: Tomasz Kulasek @ 2017-01-13 15:23 UTC (permalink / raw)
To: dev; +Cc: Daniel Mrzyglod
In-Reply-To: <1484320997-1224-1-git-send-email-tomaszx.kulasek@intel.com>
This patch provides unit tests for set of cipher/hash combinations covering
currently implemented crypto PMD's and allowing to verify scatter gather
support.
Signed-off-by: Daniel Mrzyglod <danielx.t.mrzyglod@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
---
app/test/test_cryptodev.c | 386 ++++++++++++++++++-
app/test/test_cryptodev.h | 138 +++++++
app/test/test_cryptodev_aes_test_vectors.h | 52 +++
app/test/test_cryptodev_blockcipher.c | 87 ++---
app/test/test_cryptodev_blockcipher.h | 1 +
app/test/test_cryptodev_gcm_test_vectors.h | 553 ++++++++++++++++++++++++++++
6 files changed, 1168 insertions(+), 49 deletions(-)
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 3eaf1b7..5786fde 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -1736,6 +1736,10 @@ struct crypto_unittest_params {
TEST_ASSERT_NOT_NULL(sym_op->cipher.iv.data, "no room to prepend iv");
+ /* For OOP operation both buffers must have the same size */
+ if (ut_params->obuf)
+ rte_pktmbuf_prepend(ut_params->obuf, iv_pad_len);
+
memset(sym_op->cipher.iv.data, 0, iv_pad_len);
sym_op->cipher.iv.phys_addr = rte_pktmbuf_mtophys(ut_params->ibuf);
sym_op->cipher.iv.length = iv_pad_len;
@@ -2557,6 +2561,83 @@ struct crypto_unittest_params {
}
static int
+test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
+{
+ struct crypto_testsuite_params *ts_params = &testsuite_params;
+ struct crypto_unittest_params *ut_params = &unittest_params;
+
+ int retval;
+
+ unsigned int plaintext_pad_len;
+ unsigned int plaintext_len;
+
+ uint8_t buffer[10000];
+ const uint8_t *ciphertext;
+
+ struct rte_cryptodev_info dev_info;
+
+ rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+ if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER)) {
+ printf("Device doesn't support scatter-gather. "
+ "Test Skipped.\n");
+ return 0;
+ }
+
+ /* Create KASUMI session */
+ retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
+ RTE_CRYPTO_CIPHER_OP_ENCRYPT,
+ RTE_CRYPTO_CIPHER_KASUMI_F8,
+ tdata->key.data, tdata->key.len);
+ if (retval < 0)
+ return retval;
+
+ plaintext_len = ceil_byte_length(tdata->plaintext.len);
+
+
+ /* Append data which is padded to a multiple */
+ /* of the algorithms block size */
+ plaintext_pad_len = RTE_ALIGN_CEIL(plaintext_len, 8);
+
+ ut_params->ibuf = create_segmented_mbuf(ts_params->mbuf_pool,
+ plaintext_pad_len, 10, 0);
+
+ pktmbuf_write(ut_params->ibuf, 0, plaintext_len, tdata->plaintext.data);
+
+ /* Create KASUMI operation */
+ retval = create_wireless_algo_cipher_operation(tdata->iv.data,
+ tdata->iv.len,
+ tdata->plaintext.len,
+ tdata->validCipherOffsetLenInBits.len,
+ RTE_CRYPTO_CIPHER_KASUMI_F8);
+ if (retval < 0)
+ return retval;
+
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+ TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
+
+ ut_params->obuf = ut_params->op->sym->m_dst;
+
+ if (ut_params->obuf)
+ ciphertext = rte_pktmbuf_read(ut_params->obuf, tdata->iv.len,
+ plaintext_len, buffer);
+ else
+ ciphertext = rte_pktmbuf_read(ut_params->ibuf, tdata->iv.len,
+ plaintext_len, buffer);
+
+ /* Validate obuf */
+ TEST_HEXDUMP(stdout, "ciphertext:", ciphertext, plaintext_len);
+
+ /* Validate obuf */
+ TEST_ASSERT_BUFFERS_ARE_EQUAL_BIT(
+ ciphertext,
+ tdata->ciphertext.data,
+ tdata->validCipherLenInBits.len,
+ "KASUMI Ciphertext data not as expected");
+ return 0;
+}
+
+static int
test_kasumi_encryption_oop(const struct kasumi_test_data *tdata)
{
struct crypto_testsuite_params *ts_params = &testsuite_params;
@@ -2625,6 +2706,81 @@ struct crypto_unittest_params {
}
static int
+test_kasumi_encryption_oop_sgl(const struct kasumi_test_data *tdata)
+{
+ struct crypto_testsuite_params *ts_params = &testsuite_params;
+ struct crypto_unittest_params *ut_params = &unittest_params;
+
+ int retval;
+ unsigned int plaintext_pad_len;
+ unsigned int plaintext_len;
+
+ const uint8_t *ciphertext;
+ uint8_t buffer[2048];
+
+ struct rte_cryptodev_info dev_info;
+
+ rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+ if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER)) {
+ printf("Device doesn't support scatter-gather. "
+ "Test Skipped.\n");
+ return 0;
+ }
+
+ /* Create KASUMI session */
+ retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
+ RTE_CRYPTO_CIPHER_OP_ENCRYPT,
+ RTE_CRYPTO_CIPHER_KASUMI_F8,
+ tdata->key.data, tdata->key.len);
+ if (retval < 0)
+ return retval;
+
+ plaintext_len = ceil_byte_length(tdata->plaintext.len);
+ /* Append data which is padded to a multiple */
+ /* of the algorithms block size */
+ plaintext_pad_len = RTE_ALIGN_CEIL(plaintext_len, 8);
+
+ ut_params->ibuf = create_segmented_mbuf(ts_params->mbuf_pool,
+ plaintext_pad_len, 10, 0);
+ ut_params->obuf = create_segmented_mbuf(ts_params->mbuf_pool,
+ plaintext_pad_len, 3, 0);
+
+ /* Append data which is padded to a multiple */
+ /* of the algorithms block size */
+ pktmbuf_write(ut_params->ibuf, 0, plaintext_len, tdata->plaintext.data);
+
+ /* Create KASUMI operation */
+ retval = create_wireless_algo_cipher_operation_oop(tdata->iv.data,
+ tdata->iv.len,
+ tdata->plaintext.len,
+ tdata->validCipherOffsetLenInBits.len,
+ RTE_CRYPTO_CIPHER_KASUMI_F8);
+ if (retval < 0)
+ return retval;
+
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+ TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
+
+ ut_params->obuf = ut_params->op->sym->m_dst;
+ if (ut_params->obuf)
+ ciphertext = rte_pktmbuf_read(ut_params->obuf, tdata->iv.len,
+ plaintext_pad_len, buffer);
+ else
+ ciphertext = rte_pktmbuf_read(ut_params->ibuf, tdata->iv.len,
+ plaintext_pad_len, buffer);
+
+ /* Validate obuf */
+ TEST_ASSERT_BUFFERS_ARE_EQUAL_BIT(
+ ciphertext,
+ tdata->ciphertext.data,
+ tdata->validCipherLenInBits.len,
+ "KASUMI Ciphertext data not as expected");
+ return 0;
+}
+
+
+static int
test_kasumi_decryption_oop(const struct kasumi_test_data *tdata)
{
struct crypto_testsuite_params *ts_params = &testsuite_params;
@@ -2897,6 +3053,85 @@ struct crypto_unittest_params {
return 0;
}
+static int
+test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata)
+{
+ struct crypto_testsuite_params *ts_params = &testsuite_params;
+ struct crypto_unittest_params *ut_params = &unittest_params;
+
+ int retval;
+ unsigned int plaintext_pad_len;
+ unsigned int plaintext_len;
+ uint8_t buffer[10000];
+ const uint8_t *ciphertext;
+
+ struct rte_cryptodev_info dev_info;
+
+ rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+ if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER)) {
+ printf("Device doesn't support scatter-gather. "
+ "Test Skipped.\n");
+ return 0;
+ }
+
+ /* Create SNOW 3G session */
+ retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
+ RTE_CRYPTO_CIPHER_OP_ENCRYPT,
+ RTE_CRYPTO_CIPHER_SNOW3G_UEA2,
+ tdata->key.data, tdata->key.len);
+ if (retval < 0)
+ return retval;
+
+ plaintext_len = ceil_byte_length(tdata->plaintext.len);
+ /* Append data which is padded to a multiple of */
+ /* the algorithms block size */
+ plaintext_pad_len = RTE_ALIGN_CEIL(plaintext_len, 16);
+
+ ut_params->ibuf = create_segmented_mbuf(ts_params->mbuf_pool,
+ plaintext_pad_len, 10, 0);
+ ut_params->obuf = create_segmented_mbuf(ts_params->mbuf_pool,
+ plaintext_pad_len, 3, 0);
+
+ TEST_ASSERT_NOT_NULL(ut_params->ibuf,
+ "Failed to allocate input buffer in mempool");
+ TEST_ASSERT_NOT_NULL(ut_params->obuf,
+ "Failed to allocate output buffer in mempool");
+
+ pktmbuf_write(ut_params->ibuf, 0, plaintext_len, tdata->plaintext.data);
+
+ /* Create SNOW 3G operation */
+ retval = create_wireless_algo_cipher_operation_oop(tdata->iv.data,
+ tdata->iv.len,
+ tdata->validCipherLenInBits.len,
+ tdata->validCipherOffsetLenInBits.len,
+ RTE_CRYPTO_CIPHER_SNOW3G_UEA2);
+ if (retval < 0)
+ return retval;
+
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+ TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
+
+ ut_params->obuf = ut_params->op->sym->m_dst;
+ if (ut_params->obuf)
+ ciphertext = rte_pktmbuf_read(ut_params->obuf, tdata->iv.len,
+ plaintext_len, buffer);
+ else
+ ciphertext = rte_pktmbuf_read(ut_params->ibuf, tdata->iv.len,
+ plaintext_len, buffer);
+
+ TEST_HEXDUMP(stdout, "ciphertext:", ciphertext, plaintext_len);
+
+ /* Validate obuf */
+ TEST_ASSERT_BUFFERS_ARE_EQUAL_BIT(
+ ciphertext,
+ tdata->ciphertext.data,
+ tdata->validDataLenInBits.len,
+ "SNOW 3G Ciphertext data not as expected");
+
+ return 0;
+}
+
/* Shift right a buffer by "offset" bits, "offset" < 8 */
static void
buffer_shift_right(uint8_t *buffer, uint32_t length, uint8_t offset)
@@ -3552,6 +3787,84 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
}
static int
+test_zuc_encryption_sgl(const struct zuc_test_data *tdata)
+{
+ struct crypto_testsuite_params *ts_params = &testsuite_params;
+ struct crypto_unittest_params *ut_params = &unittest_params;
+
+ int retval;
+
+ unsigned int plaintext_pad_len;
+ unsigned int plaintext_len;
+ const uint8_t *ciphertext;
+ uint8_t ciphertext_buffer[2048];
+ struct rte_cryptodev_info dev_info;
+
+ rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
+ if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER)) {
+ printf("Device doesn't support scatter-gather. "
+ "Test Skipped.\n");
+ return 0;
+ }
+
+ plaintext_len = ceil_byte_length(tdata->plaintext.len);
+
+ /* Append data which is padded to a multiple */
+ /* of the algorithms block size */
+ plaintext_pad_len = RTE_ALIGN_CEIL(plaintext_len, 8);
+
+ ut_params->ibuf = create_segmented_mbuf(ts_params->mbuf_pool,
+ plaintext_pad_len, 10, 0);
+
+ pktmbuf_write(ut_params->ibuf, 0, plaintext_len,
+ tdata->plaintext.data);
+
+ /* Create ZUC session */
+ retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
+ RTE_CRYPTO_CIPHER_OP_ENCRYPT,
+ RTE_CRYPTO_CIPHER_ZUC_EEA3,
+ tdata->key.data, tdata->key.len);
+ if (retval < 0)
+ return retval;
+
+ /* Clear mbuf payload */
+
+ pktmbuf_write(ut_params->ibuf, 0, plaintext_len, tdata->plaintext.data);
+
+ /* Create ZUC operation */
+ retval = create_wireless_algo_cipher_operation(tdata->iv.data,
+ tdata->iv.len, tdata->plaintext.len,
+ tdata->validCipherOffsetLenInBits.len,
+ RTE_CRYPTO_CIPHER_ZUC_EEA3);
+ if (retval < 0)
+ return retval;
+
+ ut_params->op = process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op);
+ TEST_ASSERT_NOT_NULL(ut_params->op, "failed to retrieve obuf");
+
+ ut_params->obuf = ut_params->op->sym->m_dst;
+ if (ut_params->obuf)
+ ciphertext = rte_pktmbuf_read(ut_params->obuf,
+ tdata->iv.len, plaintext_len, ciphertext_buffer);
+ else
+ ciphertext = rte_pktmbuf_read(ut_params->ibuf,
+ tdata->iv.len, plaintext_len, ciphertext_buffer);
+
+ /* Validate obuf */
+ TEST_HEXDUMP(stdout, "ciphertext:", ciphertext, plaintext_len);
+
+ /* Validate obuf */
+ TEST_ASSERT_BUFFERS_ARE_EQUAL_BIT(
+ ciphertext,
+ tdata->ciphertext.data,
+ tdata->validCipherLenInBits.len,
+ "ZUC Ciphertext data not as expected");
+
+ return 0;
+}
+
+static int
test_zuc_authentication(const struct zuc_hash_test_data *tdata)
{
struct crypto_testsuite_params *ts_params = &testsuite_params;
@@ -3619,12 +3932,24 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
}
static int
+test_kasumi_encryption_test_case_1_sgl(void)
+{
+ return test_kasumi_encryption_sgl(&kasumi_test_case_1);
+}
+
+static int
test_kasumi_encryption_test_case_1_oop(void)
{
return test_kasumi_encryption_oop(&kasumi_test_case_1);
}
static int
+test_kasumi_encryption_test_case_1_oop_sgl(void)
+{
+ return test_kasumi_encryption_oop_sgl(&kasumi_test_case_1);
+}
+
+static int
test_kasumi_encryption_test_case_2(void)
{
return test_kasumi_encryption(&kasumi_test_case_2);
@@ -3696,6 +4021,13 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
}
static int
+test_snow3g_encryption_test_case_1_oop_sgl(void)
+{
+ return test_snow3g_encryption_oop_sgl(&snow3g_test_case_1);
+}
+
+
+static int
test_snow3g_encryption_test_case_1_offset_oop(void)
{
return test_snow3g_encryption_offset_oop(&snow3g_test_case_1);
@@ -3815,6 +4147,12 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
}
static int
+test_zuc_encryption_test_case_6_sgl(void)
+{
+ return test_zuc_encryption_sgl(&zuc_test_case_1);
+}
+
+static int
test_zuc_hash_generate_test_case_1(void)
{
return test_zuc_authentication(&zuc_hash_test_case_1);
@@ -3998,12 +4336,21 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
struct rte_crypto_sym_op *sym_op = ut_params->op->sym;
- sym_op->auth.digest.data = (uint8_t *)rte_pktmbuf_append(
- ut_params->ibuf, auth_tag_len);
- TEST_ASSERT_NOT_NULL(sym_op->auth.digest.data,
- "no room to append digest");
- sym_op->auth.digest.phys_addr = rte_pktmbuf_mtophys_offset(
- ut_params->ibuf, data_pad_len);
+ if (ut_params->obuf) {
+ sym_op->auth.digest.data = (uint8_t *)rte_pktmbuf_append(
+ ut_params->obuf, auth_tag_len);
+ TEST_ASSERT_NOT_NULL(sym_op->auth.digest.data,
+ "no room to append digest");
+ sym_op->auth.digest.phys_addr = pktmbuf_mtophys_offset(
+ ut_params->obuf, data_pad_len);
+ } else {
+ sym_op->auth.digest.data = (uint8_t *)rte_pktmbuf_append(
+ ut_params->ibuf, auth_tag_len);
+ TEST_ASSERT_NOT_NULL(sym_op->auth.digest.data,
+ "no room to append digest");
+ sym_op->auth.digest.phys_addr = pktmbuf_mtophys_offset(
+ ut_params->ibuf, data_pad_len);
+ }
sym_op->auth.digest.length = auth_tag_len;
if (op == RTE_CRYPTO_CIPHER_OP_DECRYPT) {
@@ -4050,6 +4397,11 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
TEST_HEXDUMP(stdout, "aad:",
sym_op->auth.aad.data, aad_len);
+ if (ut_params->obuf) {
+ rte_pktmbuf_prepend(ut_params->obuf, iv_pad_len);
+ rte_pktmbuf_prepend(ut_params->obuf, aad_buffer_len);
+ }
+
sym_op->cipher.data.length = data_len;
sym_op->cipher.data.offset = aad_buffer_len + iv_pad_len;
@@ -6312,6 +6664,14 @@ struct test_crypto_vector {
}
static int
+test_AES_GCM_auth_encrypt_SGL_out_of_place_400B_1seg(void)
+{
+ return test_AES_GCM_authenticated_encryption_SGL(
+ &gcm_test_case_8, OUT_OF_PLACE, 400,
+ gcm_test_case_8.plaintext.len);
+}
+
+static int
test_AES_GCM_auth_encrypt_SGL_in_place_1500B(void)
{
@@ -6683,6 +7043,10 @@ struct test_crypto_vector {
TEST_CASE_ST(ut_setup, ut_teardown,
test_AES_GMAC_authentication_verify_test_case_4),
+ /** Scatter-Gather */
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_AES_GCM_auth_encrypt_SGL_out_of_place_400B_1seg),
+
/** Negative tests */
TEST_CASE_ST(ut_setup, ut_teardown,
authentication_verify_HMAC_SHA1_fail_data_corrupt),
@@ -6751,6 +7115,8 @@ struct test_crypto_vector {
TEST_CASE_ST(ut_setup, ut_teardown,
test_kasumi_encryption_test_case_1),
TEST_CASE_ST(ut_setup, ut_teardown,
+ test_kasumi_encryption_test_case_1_sgl),
+ TEST_CASE_ST(ut_setup, ut_teardown,
test_kasumi_encryption_test_case_2),
TEST_CASE_ST(ut_setup, ut_teardown,
test_kasumi_encryption_test_case_3),
@@ -6773,6 +7139,10 @@ struct test_crypto_vector {
TEST_CASE_ST(ut_setup, ut_teardown,
test_kasumi_encryption_test_case_1_oop),
TEST_CASE_ST(ut_setup, ut_teardown,
+ test_kasumi_encryption_test_case_1_oop_sgl),
+
+
+ TEST_CASE_ST(ut_setup, ut_teardown,
test_kasumi_decryption_test_case_1_oop),
/** KASUMI hash only (UIA1) */
@@ -6825,6 +7195,8 @@ struct test_crypto_vector {
TEST_CASE_ST(ut_setup, ut_teardown,
test_snow3g_encryption_test_case_1_oop),
TEST_CASE_ST(ut_setup, ut_teardown,
+ test_snow3g_encryption_test_case_1_oop_sgl),
+ TEST_CASE_ST(ut_setup, ut_teardown,
test_snow3g_decryption_test_case_1_oop),
TEST_CASE_ST(ut_setup, ut_teardown,
@@ -6902,6 +7274,8 @@ struct test_crypto_vector {
test_zuc_hash_generate_test_case_4),
TEST_CASE_ST(ut_setup, ut_teardown,
test_zuc_hash_generate_test_case_5),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_zuc_encryption_test_case_6_sgl),
TEST_CASES_END() /**< NULL terminate unit test array */
}
};
diff --git a/app/test/test_cryptodev.h b/app/test/test_cryptodev.h
index a9089aa..67354a9 100644
--- a/app/test/test_cryptodev.h
+++ b/app/test/test_cryptodev.h
@@ -71,4 +71,142 @@
#define TRUNCATED_DIGEST_BYTE_LENGTH_SHA384 (24)
#define TRUNCATED_DIGEST_BYTE_LENGTH_SHA512 (32)
+/**
+ * Write (spread) data from buffer to mbuf data
+ *
+ * @param mbuf
+ * Destination mbuf
+ * @param offset
+ * Start offset in mbuf
+ * @param len
+ * Number of bytes to copy
+ * @param buffer
+ * Continuous source buffer
+ */
+static inline void
+pktmbuf_write(struct rte_mbuf *mbuf, int offset, int len, const uint8_t *buffer)
+{
+ int n = len;
+ int l;
+ struct rte_mbuf *m;
+ char *dst;
+
+ for (m = mbuf; (m != NULL) && (offset > m->data_len); m = m->next)
+ offset -= m->data_len;
+
+ l = m->data_len - offset;
+
+ /* copy data from first segment */
+ dst = rte_pktmbuf_mtod_offset(m, char *, offset);
+ if (len <= l) {
+ rte_memcpy(dst, buffer, len);
+ return;
+ }
+
+ rte_memcpy(dst, buffer, l);
+ buffer += l;
+ n -= l;
+
+ for (m = m->next; (m != NULL) && (n > 0); m = m->next) {
+ dst = rte_pktmbuf_mtod(m, char *);
+ l = m->data_len;
+ if (n < l) {
+ rte_memcpy(dst, buffer, n);
+ return;
+ }
+ rte_memcpy(dst, buffer, l);
+ buffer += l;
+ n -= l;
+ }
+}
+
+static inline uint8_t *
+pktmbuf_mtod_offset(struct rte_mbuf *mbuf, int offset) {
+ struct rte_mbuf *m;
+
+ for (m = mbuf; (m != NULL) && (offset > m->data_len); m = m->next)
+ offset -= m->data_len;
+
+ if (m == NULL) {
+ printf("pktmbuf_mtod_offset: offset out of buffer\n");
+ return NULL;
+ }
+ return rte_pktmbuf_mtod_offset(m, uint8_t *, offset);
+}
+
+static inline phys_addr_t
+pktmbuf_mtophys_offset(struct rte_mbuf *mbuf, int offset) {
+ struct rte_mbuf *m;
+
+ for (m = mbuf; (m != NULL) && (offset > m->data_len); m = m->next)
+ offset -= m->data_len;
+
+ if (m == NULL) {
+ printf("pktmbuf_mtophys_offset: offset out of buffer\n");
+ return 0;
+ }
+ return rte_pktmbuf_mtophys_offset(m, offset);
+}
+
+static inline struct rte_mbuf *
+create_segmented_mbuf(struct rte_mempool *mbuf_pool, int pkt_len,
+ int nb_segs, uint8_t pattern) {
+
+ struct rte_mbuf *m = NULL, *mbuf = NULL;
+ uint8_t *dst;
+ int data_len = 0;
+ int i, size;
+ int t_len;
+
+ if (pkt_len < 1) {
+ printf("Packet size must be 1 or more (is %d)\n", pkt_len);
+ return NULL;
+ }
+
+ if (nb_segs < 1) {
+ printf("Number of segments must be 1 or more (is %d)\n",
+ nb_segs);
+ return NULL;
+ }
+
+ t_len = pkt_len >= nb_segs ? pkt_len / nb_segs : 1;
+ size = pkt_len;
+
+ /* Create chained mbuf_src and fill it generated data */
+ for (i = 0; size > 0; i++) {
+
+ m = rte_pktmbuf_alloc(mbuf_pool);
+ if (i == 0)
+ mbuf = m;
+
+ if (m == NULL) {
+ printf("Cannot create segment for source mbuf");
+ goto fail;
+ }
+
+ /* Make sure if tailroom is zeroed */
+ memset(m->buf_addr, pattern, m->buf_len);
+
+ data_len = size > t_len ? t_len : size;
+ dst = (uint8_t *)rte_pktmbuf_append(m, data_len);
+ if (dst == NULL) {
+ printf("Cannot append %d bytes to the mbuf\n",
+ data_len);
+ goto fail;
+ }
+
+ if (mbuf != m)
+ rte_pktmbuf_chain(mbuf, m);
+
+ size -= data_len;
+
+ }
+ return mbuf;
+
+fail:
+ if (mbuf)
+ rte_pktmbuf_free(mbuf);
+ return NULL;
+}
+
#endif /* TEST_CRYPTODEV_H_ */
diff --git a/app/test/test_cryptodev_aes_test_vectors.h b/app/test/test_cryptodev_aes_test_vectors.h
index 898aae1..e566548 100644
--- a/app/test/test_cryptodev_aes_test_vectors.h
+++ b/app/test/test_cryptodev_aes_test_vectors.h
@@ -858,6 +858,16 @@
BLOCKCIPHER_TEST_TARGET_PMD_QAT
},
{
+ .test_descr = "AES-192-CTR XCBC Decryption Digest Verify "
+ "Scatter Gather",
+ .test_data = &aes_test_data_2,
+ .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC,
+ .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG |
+ BLOCKCIPHER_TEST_FEATURE_OOP,
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_MB |
+ BLOCKCIPHER_TEST_TARGET_PMD_QAT
+ },
+ {
.test_descr = "AES-256-CTR HMAC-SHA1 Encryption Digest",
.test_data = &aes_test_data_3,
.op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN,
@@ -883,6 +893,18 @@
BLOCKCIPHER_TEST_TARGET_PMD_QAT
},
{
+ .test_descr = "AES-128-CBC HMAC-SHA1 Encryption Digest "
+ "Scatter Gather",
+ .test_data = &aes_test_data_4,
+ .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN,
+ .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG |
+ BLOCKCIPHER_TEST_FEATURE_OOP,
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_MB |
+ BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL |
+ BLOCKCIPHER_TEST_TARGET_PMD_QAT
+
+ },
+ {
.test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest "
"Verify",
.test_data = &aes_test_data_4,
@@ -926,6 +948,17 @@
BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL
},
{
+ .test_descr = "AES-128-CBC HMAC-SHA512 Encryption Digest "
+ "Scatter Gather Sessionless",
+ .test_data = &aes_test_data_6,
+ .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN,
+ .feature_mask = BLOCKCIPHER_TEST_FEATURE_SESSIONLESS |
+ BLOCKCIPHER_TEST_FEATURE_SG |
+ BLOCKCIPHER_TEST_FEATURE_OOP,
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_MB |
+ BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL
+ },
+ {
.test_descr = "AES-128-CBC HMAC-SHA512 Decryption Digest "
"Verify",
.test_data = &aes_test_data_6,
@@ -935,6 +968,17 @@
BLOCKCIPHER_TEST_TARGET_PMD_QAT
},
{
+ .test_descr = "AES-128-CBC HMAC-SHA512 Decryption Digest "
+ "Verify Scatter Gather",
+ .test_data = &aes_test_data_6,
+ .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC,
+ .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG |
+ BLOCKCIPHER_TEST_FEATURE_OOP,
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_MB |
+ BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL |
+ BLOCKCIPHER_TEST_TARGET_PMD_QAT
+ },
+ {
.test_descr = "AES-128-CBC XCBC Encryption Digest",
.test_data = &aes_test_data_7,
.op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN,
@@ -1045,6 +1089,14 @@
BLOCKCIPHER_TEST_TARGET_PMD_MB
},
{
+ .test_descr = "AES-192-CBC Encryption Scater gather",
+ .test_data = &aes_test_data_10,
+ .op_mask = BLOCKCIPHER_TEST_OP_ENCRYPT,
+ .feature_mask = BLOCKCIPHER_TEST_FEATURE_SG |
+ BLOCKCIPHER_TEST_FEATURE_OOP,
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL
+ },
+ {
.test_descr = "AES-192-CBC Decryption",
.test_data = &aes_test_data_10,
.op_mask = BLOCKCIPHER_TEST_OP_DECRYPT,
diff --git a/app/test/test_cryptodev_blockcipher.c b/app/test/test_cryptodev_blockcipher.c
index f1fe624..01aef3b 100644
--- a/app/test/test_cryptodev_blockcipher.c
+++ b/app/test/test_cryptodev_blockcipher.c
@@ -41,6 +41,7 @@
#include <rte_cryptodev_pmd.h>
#include "test.h"
+#include "test_cryptodev.h"
#include "test_cryptodev_blockcipher.h"
#include "test_cryptodev_aes_test_vectors.h"
#include "test_cryptodev_des_test_vectors.h"
@@ -64,6 +65,7 @@
struct rte_crypto_sym_op *sym_op = NULL;
struct rte_crypto_op *op = NULL;
struct rte_cryptodev_sym_session *sess = NULL;
+ struct rte_cryptodev_info dev_info;
int status = TEST_SUCCESS;
const struct blockcipher_test_data *tdata = t->test_data;
@@ -77,6 +79,19 @@
uint8_t tmp_src_buf[MBUF_SIZE];
uint8_t tmp_dst_buf[MBUF_SIZE];
+ int nb_segs = 1;
+
+ if (t->feature_mask & BLOCKCIPHER_TEST_FEATURE_SG) {
+ rte_cryptodev_info_get(dev_id, &dev_info);
+ if (!(dev_info.feature_flags &
+ RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER)) {
+ printf("Device doesn't support scatter-gather. "
+ "Test Skipped.\n");
+ return 0;
+ }
+ nb_segs = 3;
+ }
+
if (tdata->cipher_key.len)
memcpy(cipher_key, tdata->cipher_key.data,
tdata->cipher_key.len);
@@ -101,49 +116,38 @@
}
/* preparing data */
- ibuf = rte_pktmbuf_alloc(mbuf_pool);
- if (!ibuf) {
- snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
- "line %u FAILED: %s",
- __LINE__, "Allocation of rte_mbuf failed");
- status = TEST_FAILED;
- goto error_exit;
- }
- memset(ibuf->buf_addr, src_pattern, ibuf->buf_len);
-
if (t->op_mask & BLOCKCIPHER_TEST_OP_CIPHER)
buf_len += tdata->iv.len;
if (t->op_mask & BLOCKCIPHER_TEST_OP_AUTH)
buf_len += digest_len;
- buf_p = rte_pktmbuf_append(ibuf, buf_len);
- if (!buf_p) {
+ /* for contiguous mbuf, nb_segs is 1 */
+ ibuf = create_segmented_mbuf(mbuf_pool,
+ tdata->ciphertext.len, nb_segs, src_pattern);
+ if (ibuf == NULL) {
snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN,
"line %u FAILED: %s",
- __LINE__, "No room to append mbuf");
+ __LINE__, "Cannot create source mbuf");
status = TEST_FAILED;
goto error_exit;
}
- if (t->op_mask & BLOCKCIPHER_TEST_OP_CIPHER) {
- rte_memcpy(buf_p, tdata->iv.data, tdata->iv.len);
- buf_p += tdata->iv.len;
- }
-
/* only encryption requires plaintext.data input,
* decryption/(digest gen)/(digest verify) use ciphertext.data
* to be computed
*/
- if (t->op_mask & BLOCKCIPHER_TEST_OP_ENCRYPT) {
- rte_memcpy(buf_p, tdata->plaintext.data,
- tdata->plaintext.len);
- buf_p += tdata->plaintext.len;
- } else {
- rte_memcpy(buf_p, tdata->ciphertext.data,
- tdata->ciphertext.len);
- buf_p += tdata->ciphertext.len;
- }
+ if (t->op_mask & BLOCKCIPHER_TEST_OP_ENCRYPT)
+ pktmbuf_write(ibuf, 0, tdata->plaintext.len,
+ tdata->plaintext.data);
+ else
+ pktmbuf_write(ibuf, 0, tdata->ciphertext.len,
+ tdata->ciphertext.data);
+ if (t->op_mask & BLOCKCIPHER_TEST_OP_CIPHER) {
+ rte_memcpy(rte_pktmbuf_prepend(ibuf, tdata->iv.len),
+ tdata->iv.data, tdata->iv.len);
+ }
+ buf_p = rte_pktmbuf_append(ibuf, digest_len);
if (t->op_mask & BLOCKCIPHER_TEST_OP_AUTH_VERIFY)
rte_memcpy(buf_p, tdata->digest.data, digest_len);
else
@@ -314,17 +318,17 @@
if (t->op_mask & BLOCKCIPHER_TEST_OP_AUTH_GEN) {
auth_xform->auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
- sym_op->auth.digest.data = rte_pktmbuf_mtod_offset
- (iobuf, uint8_t *, digest_offset);
+ sym_op->auth.digest.data = pktmbuf_mtod_offset
+ (iobuf, digest_offset);
sym_op->auth.digest.phys_addr =
- rte_pktmbuf_mtophys_offset(iobuf,
+ pktmbuf_mtophys_offset(iobuf,
digest_offset);
} else {
auth_xform->auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
- sym_op->auth.digest.data = rte_pktmbuf_mtod_offset
- (sym_op->m_src, uint8_t *, digest_offset);
+ sym_op->auth.digest.data = pktmbuf_mtod_offset
+ (sym_op->m_src, digest_offset);
sym_op->auth.digest.phys_addr =
- rte_pktmbuf_mtophys_offset(sym_op->m_src,
+ pktmbuf_mtophys_offset(sym_op->m_src,
digest_offset);
}
@@ -403,13 +407,10 @@
}
if (t->op_mask & BLOCKCIPHER_TEST_OP_CIPHER) {
- uint8_t *crypto_res;
+ uint8_t buffer[2048];
const uint8_t *compare_ref;
uint32_t compare_len;
- crypto_res = rte_pktmbuf_mtod_offset(iobuf, uint8_t *,
- tdata->iv.len);
-
if (t->op_mask & BLOCKCIPHER_TEST_OP_ENCRYPT) {
compare_ref = tdata->ciphertext.data;
compare_len = tdata->ciphertext.len;
@@ -418,7 +419,8 @@
compare_len = tdata->plaintext.len;
}
- if (memcmp(crypto_res, compare_ref, compare_len)) {
+ if (memcmp(rte_pktmbuf_read(iobuf, tdata->iv.len, compare_len,
+ buffer), compare_ref, compare_len)) {
snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN, "line %u "
"FAILED: %s", __LINE__,
"Crypto data not as expected");
@@ -431,12 +433,11 @@
uint8_t *auth_res;
if (t->op_mask & BLOCKCIPHER_TEST_OP_CIPHER)
- auth_res = rte_pktmbuf_mtod_offset(iobuf,
- uint8_t *,
- tdata->iv.len + tdata->ciphertext.len);
+ auth_res = pktmbuf_mtod_offset(iobuf,
+ tdata->iv.len + tdata->ciphertext.len);
else
- auth_res = rte_pktmbuf_mtod_offset(iobuf,
- uint8_t *, tdata->ciphertext.len);
+ auth_res = pktmbuf_mtod_offset(iobuf,
+ tdata->ciphertext.len);
if (memcmp(auth_res, tdata->digest.data, digest_len)) {
snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN, "line %u "
diff --git a/app/test/test_cryptodev_blockcipher.h b/app/test/test_cryptodev_blockcipher.h
index fe97e4c..7256f6b 100644
--- a/app/test/test_cryptodev_blockcipher.h
+++ b/app/test/test_cryptodev_blockcipher.h
@@ -45,6 +45,7 @@
#define BLOCKCIPHER_TEST_FEATURE_OOP 0x01
#define BLOCKCIPHER_TEST_FEATURE_SESSIONLESS 0x02
#define BLOCKCIPHER_TEST_FEATURE_STOPPER 0x04 /* stop upon failing */
+#define BLOCKCIPHER_TEST_FEATURE_SG 0x08 /* Scatter Gather */
#define BLOCKCIPHER_TEST_TARGET_PMD_MB 0x0001 /* Multi-buffer flag */
#define BLOCKCIPHER_TEST_TARGET_PMD_QAT 0x0002 /* QAT flag */
diff --git a/app/test/test_cryptodev_gcm_test_vectors.h b/app/test/test_cryptodev_gcm_test_vectors.h
index df984fc..45ea3d4 100644
--- a/app/test/test_cryptodev_gcm_test_vectors.h
+++ b/app/test/test_cryptodev_gcm_test_vectors.h
@@ -450,6 +450,559 @@ struct gmac_test_data {
}
};
+static const struct gcm_test_data gcm_test_case_8 = {
+ .key = {
+ .data = {
+ 0xfe, 0xff, 0xe9, 0x92, 0x86, 0x65, 0x73, 0x1c,
+ 0x6d, 0x6a, 0x8f, 0x94, 0x67, 0x30, 0x83, 0x08
+ },
+ .len = 16
+ },
+ .iv = {
+ .data = {
+ 0xca, 0xfe, 0xba, 0xbe, 0xfa, 0xce, 0xdb, 0xad,
+ 0xde, 0xca, 0xf8, 0x88
+ },
+ .len = 12
+ },
+ .aad = {
+ .data = {
+ 0xfe, 0xed, 0xfa, 0xce, 0xde, 0xad, 0xbe, 0xef,
+ 0xfe, 0xed, 0xfa, 0xce
+ },
+ .len = 12
+ },
+ .plaintext = {
+ .data = {
+ 0xC5, 0x34, 0x2E, 0x83, 0xEB, 0x4C, 0x02, 0x03,
+ 0xF7, 0xB2, 0x57, 0x35, 0x26, 0x81, 0x63, 0xAE,
+ 0x1F, 0xCD, 0x2D, 0x02, 0x91, 0x5A, 0xDB, 0x3A,
+ 0xF1, 0x38, 0xD8, 0x75, 0x86, 0x20, 0xCC, 0x1E,
+ 0xE6, 0xDC, 0xFF, 0xB5, 0xEA, 0x0E, 0x18, 0x7A,
+ 0x86, 0x6C, 0xAB, 0x39, 0x2D, 0x90, 0xAC, 0x77,
+ 0x5D, 0xED, 0x65, 0xB3, 0x05, 0x29, 0xBB, 0x09,
+ 0xD0, 0x21, 0x74, 0x6A, 0x67, 0x1C, 0x95, 0x42,
+ 0x55, 0xAD, 0xC8, 0x91, 0x28, 0xFE, 0x16, 0x9A,
+ 0xE1, 0xCB, 0xCD, 0x68, 0x3B, 0xDF, 0x3E, 0x3A,
+ 0x34, 0xFE, 0x9B, 0xFB, 0xF5, 0x15, 0x2A, 0x29,
+ 0x18, 0x99, 0x24, 0xBF, 0xB6, 0x43, 0xDB, 0xD1,
+ 0x69, 0x26, 0x1E, 0x31, 0x2C, 0x8C, 0x3C, 0x6B,
+ 0x7F, 0x06, 0xA6, 0x03, 0xE2, 0x1A, 0x50, 0xFE,
+ 0x7C, 0x69, 0xE5, 0x5F, 0x35, 0x93, 0xE9, 0x20,
+ 0x14, 0xB1, 0xCA, 0x61, 0xE7, 0x9C, 0x89, 0x08,
+ 0xD6, 0xB1, 0xC2, 0x63, 0x1B, 0x86, 0x5E, 0xF1,
+ 0xF5, 0x23, 0x0E, 0x9B, 0xE5, 0xBD, 0x5D, 0x04,
+ 0xF7, 0xEF, 0x8E, 0x46, 0xB0, 0x11, 0x4F, 0x69,
+ 0x62, 0x35, 0x51, 0xB7, 0x24, 0xA2, 0x31, 0xD0,
+ 0x32, 0x4E, 0xB8, 0x44, 0xC7, 0x59, 0xDE, 0x25,
+ 0xEA, 0x2D, 0x00, 0x0E, 0xF1, 0x07, 0xBA, 0xBB,
+ 0x9A, 0xBC, 0x4F, 0x57, 0xB7, 0x13, 0x57, 0xEF,
+ 0xD9, 0xF6, 0x80, 0x69, 0xEA, 0xE8, 0x47, 0x9C,
+ 0x51, 0x71, 0xE6, 0x8F, 0x69, 0x29, 0xB4, 0x60,
+ 0xE8, 0x50, 0xE5, 0xD0, 0x9B, 0xD2, 0x62, 0x6F,
+ 0x09, 0x5C, 0xD1, 0x4B, 0x85, 0xE2, 0xFD, 0xD3,
+ 0xEB, 0x28, 0x55, 0x77, 0x97, 0xCA, 0xD6, 0xA8,
+ 0xDC, 0x35, 0x68, 0xF7, 0x6A, 0xCF, 0x48, 0x3F,
+ 0x49, 0x31, 0x00, 0x65, 0xB7, 0x31, 0x1A, 0x49,
+ 0x75, 0xDE, 0xCE, 0x7F, 0x18, 0xB5, 0x31, 0x9A,
+ 0x64, 0x6D, 0xE5, 0x49, 0x1D, 0x6D, 0xF2, 0x21,
+ 0x9F, 0xF5, 0xFF, 0x7C, 0x41, 0x30, 0x33, 0x06,
+ 0x7B, 0xA4, 0xD8, 0x99, 0xF6, 0xCC, 0xDF, 0xC4,
+ 0x3F, 0xF3, 0xCD, 0xE7, 0x74, 0xC4, 0x4A, 0x19,
+ 0x5C, 0xCA, 0x42, 0x31, 0xF1, 0x3B, 0x65, 0x1C,
+ 0x3D, 0x56, 0x08, 0xBE, 0x15, 0x37, 0x23, 0x50,
+ 0xD6, 0xA3, 0x57, 0x64, 0x25, 0xBE, 0xDA, 0xC2,
+ 0x4E, 0xF5, 0x1A, 0xAD, 0x6F, 0x43, 0x78, 0x21,
+ 0xF9, 0x36, 0x39, 0x1F, 0x5F, 0xF7, 0x1B, 0xA0,
+ 0xEE, 0x8B, 0x4F, 0x8A, 0x9D, 0xD8, 0xED, 0x37,
+ 0xCE, 0x0D, 0x70, 0xE0, 0x3F, 0xE7, 0x11, 0x30,
+ 0x17, 0x1D, 0x03, 0x5E, 0xA0, 0x3D, 0x3F, 0x9E,
+ 0xF5, 0xD3, 0x74, 0x2E, 0xC1, 0xD6, 0xFF, 0xF7,
+ 0x2E, 0xE7, 0x80, 0x88, 0xCF, 0x0E, 0x7F, 0x12,
+ 0x71, 0x62, 0xC7, 0xF1, 0xC4, 0x2B, 0x64, 0x5D,
+ 0x1C, 0x9A, 0xB4, 0xCB, 0xB8, 0x24, 0xB3, 0x0B,
+ 0x33, 0xF2, 0x8A, 0x8F, 0x76, 0xC8, 0x81, 0xDA,
+ 0x1A, 0x10, 0xB5, 0xA9, 0xCD, 0xDC, 0x1A, 0x02,
+ 0xC1, 0xAE, 0x4F, 0x02, 0x1B, 0x13, 0x96, 0x5A,
+ 0x2E, 0x03, 0xA2, 0x68, 0xB2, 0x29, 0xAC, 0x28,
+ 0xB8, 0xDC, 0xD5, 0x27, 0x55, 0xEC, 0x43, 0xDC,
+ 0xB7, 0x49, 0x1D, 0xE1, 0x30, 0x25, 0x81, 0xA6,
+ 0x90, 0x1F, 0x75, 0xBA, 0x19, 0x1E, 0xF7, 0xC5,
+ 0x77, 0x35, 0xEE, 0x68, 0x71, 0x22, 0xA0, 0xB4,
+ 0xCC, 0x99, 0x86, 0x1B, 0x1B, 0xC8, 0x27, 0xFC,
+ 0x6D, 0x8D, 0xE7, 0x8B, 0xC3, 0x40, 0x3D, 0xA8,
+ 0xCB, 0x9B, 0xC4, 0x12, 0x07, 0xDD, 0xA1, 0x92,
+ 0xE5, 0x80, 0x7A, 0xF4, 0xDB, 0x4C, 0xE6, 0xEE,
+ 0xF9, 0xD5, 0x1C, 0x20, 0x18, 0xD3, 0x8F, 0xDF,
+ 0x1C, 0xD3, 0x51, 0x4E, 0x0E, 0xED, 0x06, 0x61,
+ 0xF7, 0xBA, 0x81, 0x3A, 0x2F, 0xEA, 0xED, 0x70,
+ 0xA9, 0xD9, 0x54, 0x4D, 0xFC, 0x1D, 0x19, 0xEA,
+ 0xA6, 0x39, 0x8C, 0x6C, 0x78, 0xA8, 0x05, 0xEB,
+ 0xF2, 0xB5, 0xDE, 0x06, 0x9D, 0x8A, 0x78, 0x2A,
+ 0xF5, 0x50, 0xA4, 0xBD, 0x9B, 0xDA, 0xCA, 0x66,
+ 0xC0, 0x23, 0xAB, 0xE8, 0x95, 0x7E, 0xC9, 0xD2,
+ 0x6F, 0x09, 0xF2, 0x9A, 0x17, 0x89, 0xDA, 0x47,
+ 0x65, 0x8C, 0x20, 0xFA, 0x4E, 0x86, 0x18, 0xEB,
+ 0x7C, 0x08, 0xEC, 0x8A, 0x05, 0x54, 0x96, 0xD2,
+ 0x7A, 0x8A, 0x81, 0x58, 0x75, 0x8C, 0x7B, 0x02,
+ 0xEE, 0x1F, 0x51, 0x88, 0xD0, 0xD1, 0x90, 0x99,
+ 0x0C, 0xAE, 0x51, 0x2E, 0x54, 0x3E, 0xB1, 0x7D,
+ 0xBC, 0xE8, 0x54, 0x93, 0x6D, 0x10, 0x3C, 0xC6,
+ 0x71, 0xF6, 0xF5, 0x0B, 0x07, 0x0A, 0x6E, 0x59,
+ 0x20, 0x45, 0x21, 0x7D, 0x37, 0x64, 0x92, 0x09,
+ 0xA7, 0xE2, 0x34, 0x6F, 0xFC, 0xCC, 0x66, 0x0E,
+ 0x88, 0x1B, 0x19, 0x86, 0x11, 0xD7, 0x81, 0x25,
+ 0xF1, 0x8A, 0x03, 0xB7, 0x7A, 0xF0, 0x98, 0x4A,
+ 0x5C, 0xA1, 0x6D, 0x85, 0xA4, 0x8C, 0x4B, 0x65,
+ 0x9F, 0x72, 0x64, 0x14, 0xBA, 0x74, 0xEE, 0xA3,
+ 0x88, 0xFE, 0x1B, 0xCF, 0x11, 0x4F, 0xD1, 0xAC,
+ 0xFA, 0x14, 0xC3, 0xA7, 0xDD, 0x06, 0x85, 0x4E,
+ 0x64, 0x06, 0x92, 0x9C, 0xDF, 0x06, 0x09, 0xF1,
+ 0x4D, 0xE8, 0xF8, 0x2F, 0x69, 0xB6, 0x8A, 0xAF,
+ 0x25, 0x21, 0xB5, 0x48, 0x59, 0xF8, 0x9D, 0x60,
+ 0xAE, 0x42, 0x11, 0x7A, 0x68, 0x4D, 0x7E, 0x76,
+ 0xB0, 0xD2, 0xE3, 0xD9, 0x24, 0x16, 0x20, 0x0A,
+ 0xEB, 0xE0, 0x68, 0xCB, 0xBC, 0xAB, 0x67, 0xE4,
+ 0xF3, 0x25, 0x1F, 0xD3, 0x85, 0xA7, 0x1D, 0x7E,
+ 0x3C, 0x63, 0xCB, 0xC2, 0x50, 0x90, 0x0F, 0x4B,
+ 0x6E, 0x68, 0x06, 0x84, 0x65, 0xF7, 0xD0, 0xD4,
+ 0x12, 0xED, 0xFA, 0xC9, 0x40, 0xE2, 0xC0, 0xC9,
+ 0x46, 0x22, 0x47, 0x5E, 0x6D, 0xC1, 0x63, 0xDB,
+ 0x51, 0x98, 0xDA, 0x1A, 0xC4, 0xB9, 0xED, 0xE9,
+ 0x09, 0xB9, 0xCF, 0x91, 0x04, 0x1C, 0x63, 0xD8,
+ 0xC5, 0xA5, 0xAE, 0x53, 0x7B, 0xA1, 0x29, 0x83,
+ 0x37, 0xFB, 0xBF, 0x96, 0xBB, 0x24, 0x3D, 0x77,
+ 0x8C, 0x0F, 0xB3, 0x4B, 0x66, 0x9C, 0x54, 0xBB,
+ 0xF6, 0xDD, 0xD1, 0xB4, 0xD2, 0xF6, 0xAA, 0xED,
+ 0x18, 0x56, 0x63, 0x3E, 0x0B, 0xCA, 0xAB, 0x70,
+ 0xBB, 0x63, 0xEA, 0xB1, 0x00, 0x65, 0x90, 0x18,
+ 0xB8, 0x63, 0xA2, 0xF2, 0xB6, 0x1E, 0x61, 0x7B,
+ 0xD5, 0x01, 0xD9, 0x4D, 0xC9, 0x9D, 0x99, 0xC1,
+ 0x57, 0x9D, 0x6F, 0xAE, 0x64, 0xE4, 0x0C, 0x7E,
+ 0xFA, 0x15, 0x5E, 0xB6, 0x43, 0xB8, 0x8B, 0x89,
+ 0x87, 0xCD, 0x4F, 0xAD, 0x30, 0x1E, 0xA5, 0x03,
+ 0x7A, 0xC2, 0x10, 0x42, 0x14, 0x88, 0xD6, 0x7A,
+ 0x6D, 0x56, 0x52, 0x2E, 0x8D, 0x1B, 0x5D, 0x36,
+ 0x27, 0xA0, 0x21, 0x4B, 0x64, 0xF0, 0xC5, 0x41,
+ 0xAD, 0x05, 0x4A, 0x24, 0xE4, 0x70, 0x88, 0x63,
+ 0x12, 0xD0, 0xBC, 0x05, 0x38, 0xD9, 0x41, 0x68,
+ 0x9F, 0x16, 0x9A, 0x54, 0x09, 0x21, 0x64, 0x36,
+ 0x63, 0x97, 0x3A, 0xB5, 0xE0, 0x25, 0x43, 0x8A,
+ 0x6A, 0x59, 0x97, 0xC1, 0x31, 0xA5, 0x66, 0xD2,
+ 0xF0, 0x1C, 0xDF, 0x97, 0x51, 0xD0, 0x61, 0xBA,
+ 0x55, 0x5F, 0xD7, 0x0D, 0xD4, 0x75, 0x8E, 0x79,
+ 0x04, 0x75, 0x00, 0xB9, 0xC0, 0x7A, 0x66, 0x05,
+ 0x9F, 0x2B, 0x44, 0x42, 0x75, 0x0F, 0xD5, 0x15,
+ 0xD6, 0x16, 0x8F, 0x6C, 0x6E, 0xD4, 0x37, 0xCF,
+ 0xB4, 0xDA, 0x93, 0x00, 0x11, 0xFB, 0xBE, 0xEE,
+ 0x3B, 0x6D, 0x1D, 0xBA, 0x33, 0xD1, 0x52, 0x8B,
+ 0x16, 0x39, 0x42, 0x27, 0xE6, 0x56, 0x4C, 0x41,
+ 0x91, 0xB0, 0x98, 0xAE, 0x9B, 0x2D, 0x9B, 0x23,
+ 0x80, 0x4C, 0xEA, 0x98, 0x57, 0x95, 0x28, 0x94,
+ 0x43, 0xD3, 0x88, 0x12, 0xDF, 0x89, 0x5A, 0x7B,
+ 0xC5, 0xCB, 0x36, 0x54, 0x65, 0x74, 0xB8, 0x4E,
+ 0xE2, 0x4D, 0x01, 0xD5, 0x9C, 0x82, 0xB9, 0x1A,
+ 0x09, 0xD2, 0xCE, 0x04, 0x36, 0xD8, 0x41, 0xAC,
+ 0x4C, 0xAD, 0xC6, 0x52, 0x91, 0x1A, 0x06, 0x6D,
+ 0xFC, 0xAB, 0x29, 0x93, 0x87, 0x88, 0xB9, 0x8C,
+ 0xFA, 0x57, 0x2B, 0x05, 0x03, 0xD0, 0x18, 0xED,
+ 0x7A, 0x7B, 0x81, 0x6A, 0x97, 0x65, 0x5B, 0x90,
+ 0xDE, 0xA9, 0xFC, 0x8F, 0xFC, 0xBB, 0x98, 0xD8,
+ 0xFA, 0x32, 0x3F, 0x3F, 0x7F, 0x74, 0x65, 0x38,
+ 0xC4, 0x28, 0xEC, 0x27, 0x1F, 0x28, 0x01, 0xB1,
+ 0xAF, 0x2B, 0x8A, 0x05, 0x38, 0x7B, 0x77, 0xC9,
+ 0x61, 0x77, 0x34, 0x2C, 0x22, 0xE5, 0xEB, 0xDC,
+ 0x9D, 0x18, 0x6E, 0x23, 0x25, 0x52, 0x69, 0xB7,
+ 0x05, 0xDB, 0x66, 0x5D, 0xEA, 0x76, 0x83, 0x82,
+ 0x97, 0x39, 0xAF, 0xC0, 0x50, 0x81, 0x18, 0x0D,
+ 0x22, 0xFA, 0xB7, 0x44, 0x5C, 0x3F, 0x69, 0xF3,
+ 0xAC, 0xC5, 0x63, 0x9F, 0xD8, 0x72, 0x7E, 0x9A,
+ 0xC2, 0xEB, 0x79, 0xD0, 0x74, 0x65, 0xE8, 0xCA,
+ 0xFD, 0xA8, 0x7D, 0x23, 0x07, 0x99, 0x3E, 0xAF,
+ 0xDB, 0x67, 0x10, 0xC0, 0xE5, 0x61, 0x77, 0xC6,
+ 0x8D, 0xC4, 0x0E, 0xAA, 0x55, 0xE3, 0xC0, 0xC7,
+ 0xA5, 0x36, 0x28, 0x61, 0xDB, 0x16, 0x96, 0x5E,
+ 0x01, 0x47, 0x82, 0xE3, 0xEB, 0x20, 0x3F, 0x10,
+ 0xFA, 0x5A, 0xBC, 0xD3, 0xF9, 0xCE, 0x04, 0x87,
+ 0x51, 0x07, 0xF9, 0xD0, 0xE7, 0x6D, 0xCB, 0xCC,
+ 0xC4, 0x15, 0x00, 0xE2, 0xDC, 0x8E, 0x7B, 0x5C,
+ 0x9A, 0xF2, 0x78, 0x70, 0x4D, 0xA1, 0xAA, 0xB5,
+ 0x13, 0xCC, 0x71, 0x66, 0x5A, 0x79, 0x13, 0x3B,
+ 0x12, 0xCD, 0x40, 0x30, 0x5A, 0x49, 0xD4, 0x20,
+ 0xED, 0xCF, 0x4A, 0x75, 0xE6, 0xD5, 0xDD, 0x0F,
+ 0xD4, 0xBE, 0x98, 0x9F, 0xD7, 0x1F, 0xC0, 0x02,
+ 0x31, 0xFA, 0x67, 0x37, 0x25, 0x86, 0x56, 0x85,
+ 0x2B, 0xA2, 0x57, 0xCD, 0x8E, 0x74, 0xE7, 0x69,
+ 0xEE, 0x33, 0x5A, 0x3F, 0xCD, 0x1E, 0xE3, 0xB9,
+ 0xAA, 0x52, 0xF5, 0x22, 0x4E, 0xE3, 0xFF, 0xC8,
+ 0xE3, 0x13, 0xA3, 0x9A, 0x63, 0x23, 0xC3, 0xD7,
+ 0xE5, 0x88, 0x3E, 0x0A, 0x4B, 0xA5, 0x01, 0xE6,
+ 0x13, 0xCF, 0xED, 0xEE, 0x2A, 0x58, 0x09, 0x3F,
+ 0x2F, 0x28, 0xE7, 0xC4, 0x6B, 0xEC, 0x49, 0x51,
+ 0x79, 0x8F, 0xD5, 0x19, 0x5D, 0xA5, 0x10, 0xCE,
+ 0x8E, 0xF6, 0x26, 0x78, 0x7A, 0xA8, 0x11, 0x52,
+ 0x5F, 0x97, 0x14, 0xC9, 0x29, 0x87, 0xB8, 0xA0,
+ 0x2D, 0xE6, 0xA7, 0x2A, 0xD4, 0xFF, 0xEB, 0xBA,
+ 0xFD, 0x58, 0x39, 0x33, 0xB1, 0xCE, 0x0E, 0x78,
+ 0x67, 0x1E, 0xA1, 0x92, 0x77, 0x63, 0xF8, 0xC0,
+ 0x02, 0x49, 0x73, 0xC0, 0xA1, 0x26, 0x83, 0x04,
+ 0x9A, 0x5D, 0x85, 0x68, 0x2A, 0x2F, 0xCB, 0x88,
+ 0x8D, 0x14, 0xB1, 0x33, 0xFA, 0xFB, 0xE9, 0x05,
+ 0xBE, 0x24, 0x1A, 0x6B, 0x29, 0x2B, 0x3F, 0x52,
+ 0x8F, 0xFB, 0xE6, 0x02, 0x77, 0x50, 0x71, 0xDB,
+ 0xE9, 0x92, 0x3F, 0xE1, 0x20, 0x62, 0x80, 0xAE,
+ 0xA4, 0x98, 0xC6, 0xCD, 0xE0, 0xB1, 0xC3, 0x33,
+ 0xB1, 0xC5, 0x91, 0x3C, 0x19, 0x34, 0xA8, 0xD9,
+ 0xB3, 0x25, 0x69, 0xE3, 0x9C, 0x5F, 0x78, 0xD0,
+ 0x83, 0x1F, 0xAB, 0x85, 0x13, 0x56, 0x69, 0xB5,
+ 0x06, 0x47, 0x62, 0x37, 0x27, 0x15, 0x14, 0x05,
+ 0x4A, 0xF4, 0x6A, 0x68, 0x2A, 0x6A, 0xC3, 0x5A,
+ 0xDF, 0xB5, 0xAE, 0x2F, 0x8D, 0x8F, 0x21, 0xDB,
+ 0x33, 0x00, 0x9B, 0xD4, 0xC4, 0x08, 0x3B, 0x81,
+ 0x63, 0x4C, 0xB0, 0x39, 0x4C, 0x0A, 0xD5, 0x71,
+ 0x3E, 0x5A, 0x50, 0x58, 0x9C, 0x07, 0x89, 0x79,
+ 0x79, 0x2F, 0x0B, 0xD9, 0x50, 0xBC, 0xCF, 0x46,
+ 0x7A, 0x68, 0x5C, 0xBF, 0x1E, 0x49, 0x77, 0x92,
+ 0x85, 0x11, 0x39, 0xA6, 0x2F, 0xDA, 0x7B, 0xFA,
+ 0x72, 0x87, 0x06, 0xCD, 0x84, 0x41, 0x20, 0x1B,
+ 0x66, 0x3F, 0x42, 0x0C, 0x9E, 0x19, 0xD3, 0x18,
+ 0x57, 0xA0, 0xEE, 0x16, 0x3A, 0xC7, 0xF9, 0xD3,
+ 0x8B, 0xC9, 0x24, 0x70, 0x70, 0x51, 0x7C, 0x06,
+ 0x68, 0xD3, 0x29, 0xC9, 0x85, 0x9A, 0x1C, 0xE6,
+ 0x8C, 0x17, 0xF4, 0x88, 0xDF, 0xEA, 0xFF, 0x44,
+ 0x8D, 0x54, 0xBE, 0x22, 0x07, 0xA5, 0x7C, 0x0C,
+ 0xF4, 0x8D, 0xB1, 0x0C, 0x07, 0xED, 0xBD, 0x28,
+ 0x19, 0xDA, 0x07, 0x71, 0xA8, 0xA1, 0xE0, 0xDD,
+ 0xEE, 0x08, 0x18, 0xA5, 0xBD, 0xDD, 0x32, 0x0B,
+ 0x70, 0x1C, 0xD9, 0xEE, 0x19, 0xC2, 0xAE, 0x5C,
+ 0xE3, 0x02, 0x74, 0x70, 0x96, 0x61, 0xB1, 0x73,
+ 0x3B, 0xD6, 0x74, 0xC0, 0x82, 0xA9, 0x1F, 0xE0,
+ 0xF1, 0x22, 0x50, 0xF3, 0x9F, 0xE5, 0x13, 0x92,
+ 0xFC, 0x0A, 0x1A, 0x3C, 0xB4, 0x46, 0xFB, 0x81,
+ 0x00, 0x84, 0xA4, 0x5E, 0x6B, 0x8C, 0x25, 0x6E,
+ 0xD7, 0xB7, 0x3B, 0x01, 0x65, 0xFB, 0x0B, 0x46,
+ 0x67, 0x27, 0x2D, 0x51, 0xAD, 0xB5, 0xE0, 0x85,
+ 0xC2, 0x95, 0xA3, 0xE3, 0x68, 0x4D, 0x9E, 0x8C,
+ 0x11, 0x53, 0xF0, 0xB2, 0x85, 0xFA, 0x52, 0x4E,
+ 0xEC, 0xF9, 0xB7, 0x3C, 0x89, 0x2C, 0x4D, 0x32,
+ 0x9A, 0xCB, 0x17, 0xF3, 0x16, 0xBF, 0x44, 0x40,
+ 0xE9, 0x5E, 0x51, 0x8C, 0x1E, 0x52, 0x0A, 0xC2,
+ 0xCD, 0xA5, 0xAA, 0x03, 0x27, 0xB0, 0x8F, 0x64,
+ 0xDB, 0xD7, 0x03, 0x01, 0x8A, 0x24, 0x28, 0x7E,
+ 0x53, 0x6F, 0x24, 0xFD, 0xAA, 0xE3, 0x78, 0xB6,
+ 0xA5, 0x5D, 0x5A, 0x67, 0x20, 0xE2, 0xBE, 0x3A,
+ 0x2B, 0xE7, 0x86, 0x11, 0xDD, 0x96, 0xCB, 0x09,
+ 0x65, 0xA0, 0x36, 0xF9, 0xB0, 0x20, 0x21, 0x8E,
+ 0xDB, 0xC0, 0x73, 0xC7, 0x79, 0xD8, 0xDA, 0xC2,
+ 0x66, 0x13, 0x64, 0x34, 0x0C, 0xE1, 0x22, 0x24,
+ 0x61, 0x67, 0x08, 0x39, 0x97, 0x3F, 0x33, 0x96,
+ 0xF2, 0x44, 0x18, 0x75, 0xBB, 0xF5, 0x6A, 0x5C,
+ 0x2C, 0xAE, 0x2A, 0x79, 0x3D, 0x47, 0x19, 0x53,
+ 0x50, 0x6C, 0x9F, 0xB3, 0x82, 0x55, 0x09, 0x78,
+ 0x7B, 0xAD, 0xBC, 0x05, 0x6F, 0xC8, 0x3D, 0xB6,
+ 0x7B, 0x30, 0xE6, 0xBB, 0x8B, 0xD0, 0x2F, 0xA6,
+ 0x15, 0xCC, 0x77, 0x8C, 0x21, 0xBA, 0x03, 0xED,
+ 0x56, 0x85, 0x82, 0x4F, 0x97, 0x8C, 0x59, 0x4F,
+ 0x53, 0x5A, 0xD2, 0x70, 0xD9, 0x07, 0xB3, 0xBD,
+ 0x1D, 0x3E, 0x97, 0xD4, 0x7D, 0x93, 0x35, 0xA4,
+ 0x82, 0x6E, 0xEA, 0x4B, 0xC8, 0x6C, 0xF5, 0xE6,
+ 0xEB, 0xAF, 0x11, 0xB0, 0xB4, 0x71, 0x8F, 0x7B,
+ 0xC4, 0x8C, 0xE2, 0x66, 0x51, 0x31, 0x99, 0x01,
+ 0x5B, 0xE7, 0x48, 0xF8, 0x4C, 0xE3, 0x9A, 0x77,
+ 0xF1, 0xC6, 0x09, 0xDE, 0x76, 0xD4, 0xE3, 0x5C,
+ 0xDF, 0xA3, 0xEC, 0x3C, 0x86, 0x7C, 0xA5, 0x3F,
+ 0x8D, 0x2A, 0xF3, 0x0B, 0x54, 0xB7, 0x54, 0xA2,
+ 0xC1, 0x69, 0xC0, 0x6F, 0x1C, 0x1C, 0x76, 0xD8,
+ 0x9F, 0x7A, 0x32, 0xB0, 0xA1, 0xA6, 0x9B, 0xB7,
+ 0x21, 0x56, 0x28, 0x2D, 0xB6, 0x97, 0x03, 0x5E,
+ 0x65, 0xE3, 0x74, 0x9A, 0x96, 0x7A, 0xF9, 0xF5,
+ 0xDD, 0x85, 0xCA, 0x4C, 0xB4, 0x03, 0x6A, 0xCD,
+ 0xB6, 0x01, 0xDC, 0x8B, 0xD8, 0x73, 0x8F, 0x4D,
+ 0x7F, 0xD6, 0x71, 0xEC, 0xD7, 0xC6, 0x0B, 0x5F,
+ 0x09, 0x21, 0xB2, 0x78, 0xA8, 0xAF, 0xAD, 0x2C,
+ 0xD4, 0x93, 0x9F, 0x71, 0xF7, 0x05, 0x89, 0x42,
+ 0xC9, 0x15, 0x6F, 0x2D, 0xE0, 0xBA, 0xC3, 0xD6,
+ 0xBF, 0xAC, 0xF8, 0x24, 0x58, 0x79, 0xA9, 0xC4,
+ 0xB4, 0x49, 0x3E, 0x0B, 0x9E, 0x5E, 0xE4, 0xA6,
+ 0x8B, 0xE8, 0xDE, 0xFB, 0x4A, 0xF1, 0x69, 0x9D,
+ 0x4F, 0x77, 0x83, 0x78, 0x55, 0x19, 0x42, 0x45,
+ 0xBF, 0xBD, 0xBD, 0x12, 0x0F, 0xEF, 0x8D, 0x04,
+ 0xD8, 0x5C, 0xF2, 0xC9, 0xF1, 0xA6, 0xE0, 0x3E,
+ 0x22, 0xA8, 0xA2, 0x5E, 0x66, 0xE9, 0xAB, 0xB4,
+ 0x71, 0xBE, 0x4B, 0x3F, 0xBE, 0xC4, 0xBA, 0x4A
+ },
+ .len = 2048
+ },
+ .ciphertext = {
+ .data = {
+ 0x5E, 0x86, 0x02, 0x64, 0x32, 0xBF, 0x70, 0xC2,
+ 0x19, 0x99, 0x7F, 0x47, 0x0D, 0xA4, 0x91, 0xA8,
+ 0x7A, 0xC0, 0xA5, 0x7E, 0xA8, 0x6C, 0x88, 0x00,
+ 0xEA, 0xB5, 0x96, 0x6B, 0x25, 0xBD, 0xE7, 0x42,
+ 0xDB, 0x35, 0xE7, 0x92, 0x2B, 0x00, 0x82, 0x35,
+ 0xD4, 0x2C, 0xCF, 0x47, 0xC8, 0xB2, 0xB3, 0x57,
+ 0xF7, 0x24, 0x83, 0x7F, 0xC5, 0x2E, 0xF1, 0xC9,
+ 0x57, 0x1A, 0xEF, 0xC2, 0x3A, 0x8C, 0x1E, 0x92,
+ 0x88, 0x05, 0xAF, 0x55, 0xE6, 0x0C, 0xA7, 0x6B,
+ 0x59, 0x62, 0x32, 0x21, 0xF1, 0xFF, 0xB5, 0x5B,
+ 0x22, 0x26, 0x6F, 0x0A, 0x36, 0xDC, 0x0D, 0x16,
+ 0x3B, 0x4E, 0x7C, 0xA3, 0x75, 0x30, 0x3F, 0xB0,
+ 0x99, 0x38, 0x42, 0x8E, 0x89, 0xA3, 0x7C, 0x99,
+ 0x2F, 0x0A, 0xA1, 0xC7, 0xFD, 0x2D, 0x21, 0x8F,
+ 0xBD, 0xD4, 0x11, 0xEA, 0x55, 0xF5, 0x6A, 0x50,
+ 0x90, 0x3B, 0x60, 0x57, 0xE1, 0x86, 0x1E, 0x50,
+ 0x28, 0x67, 0x3F, 0xD2, 0xF3, 0xBD, 0xFA, 0xEE,
+ 0xD6, 0x5A, 0x38, 0x30, 0xA3, 0xDD, 0x78, 0xC4,
+ 0x37, 0x59, 0x52, 0xC0, 0x92, 0x54, 0xC7, 0x53,
+ 0xF0, 0xE6, 0xA9, 0x63, 0x1F, 0x9B, 0x97, 0xFB,
+ 0x40, 0x23, 0xFE, 0x52, 0x6A, 0xF0, 0x3A, 0x94,
+ 0xEB, 0x6A, 0x9E, 0x8F, 0xC5, 0x05, 0x9C, 0x04,
+ 0x1B, 0x00, 0x34, 0x96, 0x12, 0xDA, 0x60, 0xC6,
+ 0xAA, 0x1A, 0x3E, 0xEB, 0x70, 0x17, 0x10, 0xBC,
+ 0xF5, 0xC2, 0xE2, 0x71, 0xF3, 0xB8, 0x1D, 0xCE,
+ 0x47, 0x94, 0x21, 0x71, 0x34, 0x8C, 0xCC, 0xDD,
+ 0x27, 0xCE, 0x6F, 0x68, 0xFF, 0x91, 0x4E, 0xC4,
+ 0xA0, 0xCA, 0xB0, 0x4F, 0x17, 0x53, 0x73, 0x92,
+ 0x6C, 0xA8, 0x16, 0x06, 0xE3, 0xD9, 0x92, 0x99,
+ 0xBE, 0xB0, 0x7D, 0x56, 0xF2, 0x72, 0x30, 0xDA,
+ 0xC4, 0x4E, 0xF4, 0xA6, 0x8F, 0xD2, 0xC7, 0x8A,
+ 0xA2, 0xFC, 0xF5, 0x63, 0x17, 0x48, 0x56, 0x4D,
+ 0xBE, 0x94, 0xFE, 0xF5, 0xB1, 0xA9, 0x96, 0xAB,
+ 0x3F, 0x2D, 0xD4, 0x15, 0xEE, 0x4F, 0xFA, 0x2C,
+ 0xBE, 0x91, 0xB7, 0xBC, 0x18, 0xC8, 0xDB, 0x02,
+ 0x20, 0x29, 0xF1, 0xC1, 0x88, 0x8C, 0x8D, 0xD1,
+ 0xB3, 0x4E, 0x93, 0x96, 0xDD, 0x22, 0xAB, 0x55,
+ 0xB5, 0x9F, 0x8B, 0x20, 0xAE, 0xC6, 0x0E, 0x26,
+ 0xC6, 0xFE, 0x2D, 0x5F, 0x95, 0x89, 0x06, 0x15,
+ 0x3D, 0x88, 0x16, 0xEC, 0x9B, 0x4A, 0x1B, 0x5D,
+ 0x2E, 0xB2, 0x13, 0x56, 0x9F, 0x33, 0xB3, 0x45,
+ 0xBF, 0x5F, 0x25, 0x7E, 0x75, 0x22, 0xD2, 0xE6,
+ 0x9F, 0xAC, 0x2D, 0xFD, 0x99, 0xC2, 0x9B, 0xFC,
+ 0xD7, 0x7A, 0x9B, 0x05, 0x30, 0x0F, 0xB7, 0x4A,
+ 0xFE, 0x24, 0xDD, 0x39, 0x9B, 0xBB, 0x2F, 0xDD,
+ 0xF9, 0xFB, 0xCA, 0x6C, 0x87, 0xBA, 0x73, 0xD4,
+ 0x85, 0x7B, 0xB2, 0x6F, 0x5C, 0xD8, 0xFB, 0xE9,
+ 0x41, 0x24, 0x3A, 0x3B, 0x4F, 0x91, 0x77, 0xA2,
+ 0x35, 0x78, 0xE5, 0x4C, 0xFE, 0x8B, 0x04, 0x03,
+ 0xD3, 0x84, 0xA9, 0x1C, 0xA7, 0x7C, 0x45, 0x13,
+ 0x7D, 0xC5, 0x0A, 0x2F, 0x02, 0xF8, 0x56, 0xD5,
+ 0x5F, 0x35, 0xED, 0x06, 0xBF, 0x67, 0xBA, 0x51,
+ 0x02, 0x95, 0x36, 0xF2, 0x9A, 0xBA, 0x9D, 0xF6,
+ 0xD6, 0x77, 0x50, 0xC9, 0xFC, 0x1E, 0x32, 0xB5,
+ 0x2F, 0xEA, 0x3C, 0x76, 0xB4, 0xE1, 0xCC, 0x42,
+ 0xEB, 0x71, 0x79, 0xD3, 0x7D, 0xB7, 0xC0, 0x88,
+ 0x25, 0x81, 0xE8, 0xC0, 0xB8, 0x38, 0x7E, 0x7B,
+ 0xFD, 0x18, 0xAB, 0x08, 0xB2, 0x71, 0xA5, 0xAD,
+ 0xA7, 0xBE, 0x48, 0x5F, 0x86, 0xE2, 0x41, 0x3D,
+ 0x7C, 0x37, 0x7A, 0xAB, 0xDB, 0xE0, 0x3B, 0x3D,
+ 0xB6, 0xE8, 0x23, 0x7C, 0xF1, 0x8F, 0xBA, 0xB7,
+ 0xE9, 0x78, 0x0B, 0xCA, 0x67, 0xA8, 0x10, 0x36,
+ 0xEB, 0x72, 0xED, 0xDD, 0xF0, 0x5C, 0x74, 0x8E,
+ 0xE5, 0x2A, 0xAE, 0x6E, 0xC4, 0xF1, 0xFC, 0xD8,
+ 0xEE, 0x56, 0x07, 0x88, 0x02, 0xDC, 0x9D, 0xB7,
+ 0xF9, 0x13, 0xE1, 0xE1, 0x9D, 0x89, 0x26, 0x0B,
+ 0x23, 0x74, 0x4A, 0x43, 0xAA, 0xA0, 0xA8, 0x97,
+ 0x85, 0x15, 0x58, 0xAB, 0x2B, 0xB5, 0xDA, 0x1A,
+ 0xBA, 0x29, 0x62, 0xCF, 0xDD, 0xA3, 0xBA, 0x9D,
+ 0x7D, 0x83, 0xA5, 0x18, 0xD4, 0x03, 0x0F, 0x61,
+ 0x9F, 0xB1, 0x7E, 0xEC, 0xD2, 0x6E, 0xAF, 0xCF,
+ 0x1E, 0xC1, 0x88, 0x97, 0x99, 0xD6, 0xBF, 0x47,
+ 0xB9, 0x0A, 0x69, 0x11, 0x3A, 0x55, 0x8B, 0x1D,
+ 0x2D, 0xFF, 0x78, 0xC8, 0xDE, 0x82, 0x29, 0xD6,
+ 0x08, 0x3C, 0xC4, 0xCB, 0x2F, 0x01, 0xD0, 0xE8,
+ 0xB1, 0x75, 0x5E, 0x23, 0xE0, 0x37, 0x7C, 0x1C,
+ 0xB6, 0xD9, 0x47, 0xDE, 0x23, 0x87, 0xD3, 0x68,
+ 0x47, 0x46, 0x78, 0xF3, 0xBF, 0x54, 0xA3, 0xB9,
+ 0x54, 0xD5, 0xC5, 0x0A, 0x7C, 0x92, 0x2A, 0xC2,
+ 0x14, 0x76, 0xA6, 0x5C, 0x6D, 0x0B, 0x94, 0x56,
+ 0x00, 0x6B, 0x5C, 0x27, 0xDE, 0x77, 0x9B, 0xF1,
+ 0xB1, 0x8C, 0xA7, 0x49, 0x77, 0xFC, 0x4E, 0x29,
+ 0x23, 0x8F, 0x2F, 0xF7, 0x83, 0x8D, 0x36, 0xD9,
+ 0xAB, 0x0E, 0x78, 0xF5, 0x90, 0x05, 0xB9, 0x79,
+ 0x70, 0x88, 0x59, 0x6F, 0xE2, 0xC5, 0xD7, 0x80,
+ 0x95, 0x04, 0x29, 0xE0, 0xFA, 0x37, 0xE8, 0x8B,
+ 0xC5, 0x21, 0x51, 0x1A, 0x62, 0xCE, 0x93, 0xAF,
+ 0x1A, 0xFE, 0xC3, 0x6F, 0x86, 0x94, 0x5E, 0x13,
+ 0xA6, 0x9A, 0x26, 0xF0, 0xB5, 0x7C, 0x41, 0x9A,
+ 0x80, 0xB8, 0x84, 0x5A, 0x55, 0xA9, 0xB0, 0x6A,
+ 0xFA, 0xEB, 0x46, 0x32, 0x0B, 0xE2, 0x9C, 0x65,
+ 0x86, 0x11, 0x39, 0x7E, 0xAF, 0x93, 0x19, 0x09,
+ 0x70, 0x40, 0x80, 0x14, 0xBA, 0x1D, 0xB3, 0x62,
+ 0x5B, 0xF3, 0x9A, 0x21, 0x98, 0x7E, 0x63, 0xB6,
+ 0x1A, 0xBD, 0x65, 0x98, 0x35, 0x2A, 0xA9, 0x76,
+ 0x29, 0x59, 0x84, 0x25, 0x81, 0xB8, 0xDE, 0x25,
+ 0x32, 0x10, 0x50, 0xB7, 0xD3, 0xB3, 0x69, 0xC8,
+ 0xE1, 0x33, 0xCB, 0x9E, 0x9C, 0x7A, 0x7C, 0xD2,
+ 0x6C, 0x92, 0x97, 0xA9, 0xFA, 0xAF, 0x30, 0xBA,
+ 0x9A, 0xB3, 0x3D, 0x9A, 0xE5, 0x0A, 0x9B, 0x8D,
+ 0x89, 0xE2, 0x2B, 0xB8, 0xBC, 0xF0, 0x23, 0xFF,
+ 0x7B, 0x0D, 0x00, 0x36, 0xEE, 0x79, 0xCB, 0xA5,
+ 0x70, 0x4C, 0x66, 0x02, 0x79, 0x2E, 0x5B, 0x83,
+ 0xCE, 0x55, 0x8B, 0x89, 0xD6, 0xE3, 0x71, 0x63,
+ 0xBC, 0xB1, 0x5F, 0x67, 0xB4, 0x7E, 0x05, 0x0D,
+ 0xAC, 0x6D, 0x4E, 0x2C, 0xA5, 0xF4, 0x47, 0x89,
+ 0xAC, 0x5E, 0xBE, 0x2F, 0xFC, 0x9B, 0x2F, 0x0B,
+ 0xBE, 0x63, 0x54, 0x97, 0xBB, 0x23, 0x27, 0xCD,
+ 0xB9, 0xB2, 0x28, 0x0D, 0xA4, 0x78, 0x2C, 0xAB,
+ 0xD1, 0xC9, 0x94, 0x40, 0x54, 0xF2, 0x35, 0x61,
+ 0x49, 0x01, 0x87, 0x55, 0xA5, 0xB5, 0x1E, 0x84,
+ 0x92, 0x9E, 0xC1, 0xA4, 0x0B, 0x66, 0x2B, 0xF8,
+ 0xAF, 0xC3, 0x1E, 0xAF, 0x66, 0x3F, 0x6F, 0x5F,
+ 0x70, 0xEC, 0x25, 0x29, 0xE4, 0x65, 0xB2, 0x04,
+ 0x47, 0xF6, 0x3C, 0xB5, 0x5F, 0x66, 0x9F, 0xA4,
+ 0x1B, 0xFC, 0xA2, 0xD5, 0x3E, 0x84, 0xBA, 0x88,
+ 0x0D, 0xF1, 0x6A, 0xF2, 0xF6, 0x1D, 0xF1, 0xA3,
+ 0x45, 0xB2, 0x51, 0xD8, 0xA2, 0x8F, 0x55, 0xA6,
+ 0x89, 0xC4, 0x15, 0xD5, 0x73, 0xA8, 0xB1, 0x31,
+ 0x66, 0x9E, 0xC1, 0x43, 0xE1, 0x5D, 0x4E, 0x04,
+ 0x84, 0x8F, 0xF2, 0xBC, 0xE1, 0x4E, 0x4D, 0x60,
+ 0x81, 0xCA, 0x53, 0x34, 0x95, 0x17, 0x3B, 0xAE,
+ 0x8F, 0x95, 0xA7, 0xC6, 0x47, 0xC6, 0xAC, 0x32,
+ 0x12, 0x39, 0xCA, 0xEF, 0xE0, 0x07, 0xBF, 0x17,
+ 0x4F, 0xDC, 0x1B, 0x4E, 0x3C, 0x84, 0xF1, 0x9F,
+ 0x43, 0x70, 0x19, 0xE6, 0xF3, 0x8B, 0x8B, 0x5D,
+ 0xDB, 0xD2, 0x9D, 0xD4, 0xB2, 0x30, 0x45, 0x55,
+ 0xA2, 0x67, 0xA2, 0x76, 0x4A, 0x74, 0xAD, 0x88,
+ 0x71, 0xE6, 0x3E, 0x13, 0x06, 0x30, 0x17, 0xE1,
+ 0xEF, 0xAC, 0x71, 0xFB, 0x43, 0xCD, 0xF6, 0xFA,
+ 0x0E, 0x4C, 0x4E, 0x16, 0xF6, 0x6A, 0x09, 0x86,
+ 0x6B, 0xEA, 0x47, 0x6C, 0x70, 0xE7, 0xAD, 0xA2,
+ 0xE0, 0xFD, 0x7F, 0xF0, 0x5C, 0x21, 0x53, 0x0F,
+ 0x28, 0xA1, 0x43, 0xE1, 0x06, 0xCA, 0x0B, 0x31,
+ 0x88, 0x22, 0xA6, 0xE6, 0x34, 0x5B, 0xE6, 0xCF,
+ 0x25, 0x81, 0x63, 0xFF, 0x78, 0x66, 0x85, 0x19,
+ 0xE2, 0x0A, 0x7E, 0x81, 0x8A, 0x17, 0x1A, 0x18,
+ 0x8A, 0x5F, 0x5D, 0x9E, 0x82, 0x13, 0x10, 0xB9,
+ 0xD3, 0xE6, 0x93, 0x1C, 0xE4, 0x2C, 0xCB, 0x49,
+ 0x1E, 0xB6, 0x36, 0x13, 0xBF, 0x28, 0xEE, 0xCC,
+ 0x49, 0xF5, 0x79, 0xFC, 0x20, 0x65, 0xBD, 0xE8,
+ 0xF0, 0x1B, 0x4E, 0xC0, 0x0D, 0x3E, 0x89, 0x91,
+ 0xCC, 0x64, 0x10, 0xC0, 0x2A, 0x2B, 0xA3, 0xFA,
+ 0x60, 0x3D, 0xC3, 0x52, 0x2F, 0x93, 0xDE, 0xB7,
+ 0x6E, 0x8A, 0xDF, 0x6C, 0x08, 0xCC, 0x8B, 0x3B,
+ 0xC8, 0x50, 0xEF, 0x58, 0x64, 0x9A, 0x3D, 0x16,
+ 0x70, 0x94, 0x11, 0xD8, 0x94, 0x2B, 0x70, 0x91,
+ 0x10, 0x70, 0x88, 0xF0, 0x40, 0x75, 0x9A, 0x2B,
+ 0x39, 0xA1, 0x27, 0x3F, 0x2E, 0x91, 0xEA, 0xA1,
+ 0xCC, 0x12, 0xC1, 0x7F, 0x73, 0x8C, 0x5C, 0x6B,
+ 0xFC, 0xC5, 0x6A, 0x1C, 0x05, 0xF1, 0x3D, 0x30,
+ 0x82, 0x4A, 0x65, 0x35, 0xCE, 0x80, 0x10, 0xBB,
+ 0x41, 0x94, 0xFB, 0x84, 0x80, 0x7B, 0x91, 0xC4,
+ 0x4D, 0xA3, 0x5F, 0xB9, 0xFB, 0xF9, 0xC9, 0x1D,
+ 0x4F, 0x99, 0x1C, 0x1F, 0x47, 0x44, 0x89, 0x0E,
+ 0xED, 0x6D, 0xB5, 0x85, 0x41, 0x94, 0xEF, 0xF9,
+ 0x2E, 0xA0, 0xC8, 0xCA, 0xFB, 0x44, 0x02, 0xC6,
+ 0xBF, 0x96, 0x87, 0x80, 0x1D, 0xEF, 0x2A, 0x81,
+ 0xAB, 0xB2, 0x56, 0xDF, 0x54, 0x8B, 0xAB, 0xAF,
+ 0xFE, 0x18, 0x8C, 0xAA, 0xD4, 0x00, 0x17, 0xBE,
+ 0xCF, 0x06, 0xE5, 0xA6, 0xBF, 0x5A, 0x52, 0x3B,
+ 0x4E, 0xF5, 0x65, 0x60, 0x95, 0xDE, 0x8A, 0x25,
+ 0x88, 0xA5, 0x24, 0x96, 0x29, 0x13, 0x0D, 0x19,
+ 0x45, 0x95, 0x91, 0x08, 0xD2, 0x9C, 0x4C, 0x34,
+ 0x42, 0xF0, 0xA5, 0x72, 0xEB, 0xFB, 0x5E, 0xAA,
+ 0x68, 0x80, 0x82, 0xAC, 0x34, 0xAD, 0x89, 0xF6,
+ 0xAF, 0x54, 0x82, 0xCF, 0x98, 0x8C, 0x75, 0x63,
+ 0x8D, 0xBD, 0x1C, 0x2A, 0xD7, 0x00, 0xA7, 0x8E,
+ 0xB9, 0x33, 0xB6, 0x3B, 0x95, 0x9A, 0x59, 0x1D,
+ 0x3F, 0x23, 0x6B, 0x18, 0xF8, 0x4F, 0x1A, 0x8D,
+ 0xC0, 0x26, 0x9F, 0x87, 0x61, 0xB6, 0xC6, 0x60,
+ 0x38, 0x22, 0x73, 0x1C, 0x99, 0x23, 0xEF, 0xD9,
+ 0xFD, 0xCB, 0x54, 0x74, 0xBB, 0x77, 0x14, 0xA3,
+ 0xA9, 0xE6, 0x7C, 0x7E, 0x03, 0x3A, 0x13, 0x6E,
+ 0x1D, 0x6F, 0x64, 0xB3, 0xFA, 0xFB, 0x52, 0xDE,
+ 0xDF, 0x08, 0xFB, 0x6F, 0xC5, 0xFA, 0x51, 0x6A,
+ 0x69, 0x29, 0x9B, 0x96, 0xE8, 0x16, 0xC8, 0xD1,
+ 0xE4, 0x19, 0xBD, 0x14, 0x74, 0x27, 0xE7, 0x10,
+ 0xF0, 0xC3, 0xE2, 0xA7, 0x60, 0x48, 0xBF, 0xDD,
+ 0xC4, 0x0D, 0xD0, 0xF2, 0xEF, 0xA6, 0xC9, 0xA2,
+ 0x73, 0xD1, 0xCF, 0x41, 0xE1, 0x3B, 0xE5, 0x49,
+ 0x91, 0x5D, 0x09, 0xFD, 0x1D, 0x95, 0x29, 0xDB,
+ 0x52, 0x48, 0xEB, 0xF5, 0x1D, 0xF8, 0x06, 0x67,
+ 0x75, 0xF2, 0x57, 0xA4, 0x20, 0x60, 0xEA, 0xB0,
+ 0x85, 0x93, 0x7C, 0xDD, 0x52, 0x01, 0xD4, 0x57,
+ 0xA8, 0x31, 0x2D, 0xF9, 0x0A, 0xD2, 0x2A, 0xD1,
+ 0x34, 0x18, 0x35, 0x16, 0xB6, 0x8B, 0x0F, 0x0B,
+ 0xCF, 0x50, 0x80, 0xFE, 0x76, 0xCC, 0x4F, 0x30,
+ 0x98, 0x19, 0x16, 0x3D, 0x01, 0xEA, 0x8D, 0x8A,
+ 0x3D, 0xDC, 0xFB, 0x1F, 0x77, 0x8D, 0x72, 0x76,
+ 0x02, 0x3C, 0x5D, 0xEE, 0x55, 0x13, 0x5B, 0x6E,
+ 0x5A, 0x2D, 0xD5, 0x77, 0xD7, 0x01, 0x84, 0x7D,
+ 0x21, 0x8C, 0xDD, 0x94, 0x7D, 0x31, 0x3D, 0xF0,
+ 0xE7, 0x28, 0xF5, 0x72, 0x36, 0x60, 0xE0, 0x59,
+ 0x5F, 0xFE, 0x38, 0xF8, 0x2F, 0xDB, 0x9E, 0x55,
+ 0x5A, 0xD6, 0xBA, 0x6C, 0x87, 0xF3, 0xC0, 0x76,
+ 0x5F, 0xA3, 0x0A, 0xC3, 0xA3, 0x8D, 0x0E, 0x52,
+ 0xA8, 0xDA, 0x26, 0x3A, 0xF9, 0x3E, 0x36, 0xB1,
+ 0x06, 0xF8, 0x20, 0x2D, 0x1C, 0x0B, 0x93, 0xBB,
+ 0xD3, 0x64, 0x77, 0xCE, 0x11, 0xFC, 0xA2, 0x0E,
+ 0x1B, 0x5B, 0x9E, 0x13, 0x9F, 0x20, 0x8B, 0xAA,
+ 0xCD, 0x72, 0xD7, 0xA6, 0xF3, 0x1E, 0x4F, 0x72,
+ 0xC6, 0x49, 0x0F, 0x7B, 0xF0, 0x4C, 0x61, 0x1F,
+ 0x43, 0x0D, 0x4F, 0x0D, 0x33, 0x13, 0xED, 0x63,
+ 0xE5, 0xDB, 0x71, 0xAB, 0xA4, 0x83, 0xEF, 0xDC,
+ 0x86, 0x9D, 0x4B, 0xBD, 0x1B, 0x8A, 0xFE, 0x39,
+ 0xA8, 0x8B, 0xBA, 0x4C, 0x85, 0x28, 0xFC, 0xB3,
+ 0x62, 0x85, 0xD2, 0xF0, 0x38, 0xD0, 0x4B, 0xA4,
+ 0xD1, 0x3B, 0xD4, 0xD0, 0x2C, 0x78, 0x6C, 0x6A,
+ 0xC2, 0x64, 0x2C, 0x31, 0x4A, 0xD8, 0x69, 0x24,
+ 0xED, 0x77, 0x7D, 0x68, 0x9A, 0xA1, 0x78, 0x81,
+ 0xD9, 0x7E, 0x6C, 0xFE, 0x0A, 0x0D, 0x76, 0xF7,
+ 0x4B, 0x58, 0xE7, 0xC9, 0xB5, 0x11, 0x07, 0x87,
+ 0x88, 0x6A, 0x9F, 0x3D, 0xE0, 0xEE, 0xCC, 0x60,
+ 0x6B, 0x6B, 0xE6, 0xB5, 0x54, 0x8B, 0x32, 0x1F,
+ 0x04, 0x1D, 0x0E, 0x9E, 0xFA, 0x6D, 0xB0, 0xE0,
+ 0x6D, 0xF9, 0x79, 0xB4, 0xAB, 0x5E, 0xDF, 0x23,
+ 0x7F, 0x95, 0xAD, 0x80, 0x17, 0x23, 0x90, 0x1F,
+ 0xF0, 0xC3, 0xD9, 0x2D, 0xAC, 0x3F, 0x63, 0xF5,
+ 0x77, 0xC5, 0x05, 0xAC, 0x06, 0xB6, 0xA1, 0xB4,
+ 0xA2, 0x40, 0xB3, 0x99, 0x34, 0x7D, 0x31, 0xD4,
+ 0xB1, 0xD4, 0xC1, 0xBB, 0x71, 0x1E, 0xDA, 0x3F,
+ 0xA9, 0x12, 0x68, 0xFA, 0x5B, 0x20, 0x24, 0x6D,
+ 0x4D, 0x72, 0x43, 0x18, 0xBF, 0x66, 0x71, 0x69,
+ 0x26, 0x7D, 0x77, 0x78, 0xF8, 0xE5, 0x20, 0xAE,
+ 0x56, 0x6C, 0x0F, 0x72, 0x94, 0x42, 0x85, 0x4F,
+ 0xE4, 0xFB, 0x32, 0x26, 0x1B, 0x1C, 0x6E, 0x0B,
+ 0xF0, 0xB8, 0x58, 0x00, 0xD2, 0x36, 0x64, 0xAD,
+ 0xA9, 0x00, 0xCE, 0x35, 0x3C, 0x88, 0x79, 0x94,
+ 0x0C, 0x0C, 0x9B, 0xF2, 0xDA, 0xBD, 0xCA, 0x93,
+ 0x37, 0x26, 0xD3, 0x08, 0x54, 0xD2, 0x0D, 0xBC,
+ 0x5D, 0x43, 0x5F, 0xCF, 0x28, 0xB5, 0xAA, 0x15,
+ 0x28, 0x46, 0x45, 0x6B, 0xE8, 0xDF, 0xE8, 0xCE,
+ 0x8F, 0xC0, 0x1A, 0x53, 0x63, 0x3B, 0x53, 0x75,
+ 0xDD, 0x43, 0x1F, 0x07, 0x0A, 0xD5, 0xA1, 0x2A,
+ 0x6E, 0x28, 0xE1, 0xD7, 0xD0, 0x09, 0xCF, 0x62,
+ 0xC1, 0x5F, 0x21, 0xDB, 0xC5, 0x40, 0x99, 0x48,
+ 0x87, 0x6E, 0x11, 0xF5, 0x5A, 0x4E, 0xBC, 0xF9,
+ 0xA8, 0x02, 0x7C, 0x47, 0x39, 0xA5, 0xD8, 0x52,
+ 0xB1, 0x80, 0xDC, 0xFE, 0x08, 0x4B, 0x5D, 0x09,
+ 0xDE, 0x06, 0xF3, 0x2A, 0xAD, 0x14, 0x76, 0x40,
+ 0x2F, 0x82, 0x28, 0x6A, 0xB6, 0x43, 0xEF, 0x71,
+ 0x63, 0xC2, 0x56, 0xEB, 0x3B, 0x4B, 0x52, 0x2F,
+ 0x93, 0xD3, 0x18, 0x3E, 0x18, 0xA8, 0xF7, 0x58,
+ 0xFC, 0x8B, 0x3D, 0x4D, 0x4B, 0x72, 0xBD, 0xF7,
+ 0x04, 0xC9, 0xB8, 0xD7, 0x6C, 0x8C, 0x67, 0xBB,
+ 0x4C, 0x9B, 0x57, 0xF7, 0x22, 0x4E, 0x41, 0xB6,
+ 0xFD, 0xD9, 0xF8, 0x41, 0x62, 0x0F, 0xFF, 0xAA,
+ 0xC6, 0x87, 0x95, 0xFF, 0xFD, 0x58, 0xD9, 0xB2,
+ 0xBA, 0x47, 0x61, 0x24, 0xEA, 0x92, 0x6E, 0x74,
+ 0xB3, 0xDA, 0xE5, 0x83, 0x99, 0x24, 0xB1, 0x71,
+ 0x2A, 0x33, 0xB2, 0xD5, 0x8F, 0xF0, 0x32, 0xCE,
+ 0x37, 0xCF, 0xC7, 0x1C, 0xE8, 0xDE, 0x46, 0x78,
+ 0x96, 0x97, 0xF6, 0x73, 0x90, 0xE5, 0x71, 0x05,
+ 0xEA, 0x0D, 0xC2, 0x1D, 0x9E, 0x43, 0x34, 0xBC,
+ 0x8F, 0x45, 0xE5, 0x08, 0xCA, 0x20, 0x0C, 0x84
+ },
+ .len = 2048
+ },
+ .auth_tag = {
+ .data = {
+ 0xD0, 0x62, 0x1F, 0x20, 0x1C, 0xE8, 0xDD, 0x36,
+ 0x00, 0x74, 0xF6, 0xD7, 0xFD, 0x2C, 0xA0, 0xAF
+ },
+ .len = 16
+ }
+};
+
/** GMAC Test Vectors */
static uint8_t gmac_plaintext[GMAC_LARGE_PLAINTEXT_LENGTH] = {
0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08,
--
1.7.9.5
^ permalink raw reply related
* Re: [PATCH v5] mbuf: add a function to linearize a packet
From: Olivier Matz @ 2017-01-13 15:32 UTC (permalink / raw)
To: Tomasz Kulasek; +Cc: dev, Pablo de Lara
In-Reply-To: <1484214044-9432-1-git-send-email-tomaszx.kulasek@intel.com>
On Thu, 12 Jan 2017 10:40:44 +0100, Tomasz Kulasek
<tomaszx.kulasek@intel.com> wrote:
> This patch adds function rte_pktmbuf_linearize to let crypto PMD
> coalesce chained mbuf before crypto operation and extend their
> capabilities to support segmented mbufs when device cannot handle
> them natively.
>
> Included unit tests for rte_pktmbuf_linearize functionality:
>
> 1) Creates banch of segmented mbufs with different size and number of
> segments.
> 2) Fills noncontigouos mbuf with sequential values.
> 3) Uses rte_pktmbuf_linearize to coalesce segmented buffer into one
> contiguous.
> 4) Verifies data in linearized buffer.
>
> Dependencies:
>
> This patch is rebased to the dpdk-next-crypto and should be applied
> before "Chained Mbufs support in SW PMDs" patchset.
>
> changes in v5:
> - name of patch changed,
> - improved coding style,
>
> changes in v4:
> - separated from "Chained Mbufs support in SW PMDs" patch set for
> better reviewing,
> - merged "rte_pktmbuf_linearize" implementation with unit tests,
>
> changes in v3:
> - rebased to dpdk-next-crypto
>
> changes in v2:
> - rte_pktmbuf_coalesce replaced with rte_pktmbuf_linearize
>
> Cc: Pablo de Lara <pablo.de.lara.guarch@intel.com>
> Cc: Olivier Matz <olivier.matz@6wind.com>
> Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Just one comment about the changelog for next time: as it's not
something we need to keep in the git history, you can put it after the
'---', so it will be dropped when applying the patch with 'git am'.
Thanks,
Olivier
^ permalink raw reply
* Re: [PATCH v4 3/6] test: unit tests for new distributor burst api
From: Bruce Richardson @ 2017-01-13 15:33 UTC (permalink / raw)
To: David Hunt; +Cc: dev
In-Reply-To: <1483948248-91364-4-git-send-email-david.hunt@intel.com>
On Mon, Jan 09, 2017 at 07:50:45AM +0000, David Hunt wrote:
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---
> app/test/test_distributor.c | 501 ++++++++++++++++++++++++++++++++++----------
> 1 file changed, 392 insertions(+), 109 deletions(-)
>
check-git-log script complains about your capitalization of the commit
title, API vs api:
Wrong headline lowercase:
test: unit tests for new distributor burst api
This also applies to a few more of the patches.
/Bruce
^ permalink raw reply
* Re: [PATCH v4] mempool: use cache in single producer or consumer mode
From: Thomas Monjalon @ 2017-01-13 15:34 UTC (permalink / raw)
To: Wenfeng Liu; +Cc: dev, Olivier Matz, konstantin.ananyev
In-Reply-To: <20170113162327.467538f9@platinum>
2017-01-13 16:23, Olivier Matz:
> On Wed, 11 Jan 2017 02:25:28 +0000, Wenfeng Liu
> <liuwf@arraynetworks.com.cn> wrote:
> > Currently we will check mempool flags when we put/get objects from
> > mempool. However, this makes cache useless when mempool is SC|SP,
> > SC|MP, MC|SP cases.
> > This patch makes cache available in above cases and improves
> > performance.
> >
> > Signed-off-by: Wenfeng Liu <liuwf@arraynetworks.com.cn>
>
> Acked-by: Olivier Matz <olivier.matz@6wind.com>
Applied, thanks
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox