Linux virtualization list
 help / color / mirror / Atom feed
* Re: virtio gpu sparse warning
From: Gerd Hoffmann @ 2016-11-28  7:50 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: David Airlie, Jiri Slaby, Linux kernel mailing list, dri-devel,
	virtualization
In-Reply-To: <20161124045101-mutt-send-email-mst@kernel.org>

On Do, 2016-11-24 at 04:57 +0200, Michael S. Tsirkin wrote:
> sparse produces these warnings:
> 
> drivers/gpu/drm/virtio/virtgpu_fb.c:340:27: warning: incorrect type in
> assignment (different address spaces)
> drivers/gpu/drm/virtio/virtgpu_fb.c:340:27:    expected char [noderef]
> <asn:2>*screen_base
> drivers/gpu/drm/virtio/virtgpu_fb.c:340:27:    got void *vmap
> 
> This is because the expected type is void __iomem *, while
> virtio gpu object is void *vmap.
> 
> We could just cast the warning away but I'm not sure this
> is not a symptom of an actual problem. For example, might
> some code call iounmap on this address?

Nobody is ever going to unmap that, the kernel will simply use given
address to access the framebuffer.

Actually it looks like this (in include/linux/fb.h):

        union {
                char __iomem *screen_base;      /* Virtual address */
                char *screen_buffer;
        };

and given that the virtio always uses normal ram as backing storage for
the framebuffer we should simply s/screen_base/screen_buffer/.  I'll go
prepare a patch.

cheers,
  Gerd

^ permalink raw reply

* RE: [virtio-dev] Re: [PATCH v2 2/2] crypto: add virtio-crypto driver
From: Gonglei (Arei) @ 2016-11-28  5:38 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: virtio-dev@lists.oasis-open.org, Huangweidong (C),
	Claudio Fontana, Hanweidong (Randy), qemu-devel@nongnu.org,
	Luonengjun, linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	salvatore.benedetto@intel.com, Xuquan (Quan Xu),
	linux-crypto@vger.kernel.org, stefanha@redhat.com,
	Zhoujian (jay, Euler), longpeng, arei.gonglei@hotmail.com,
	davem@davemloft.net, Wubin (H)
In-Reply-To: <20161128070829-mutt-send-email-mst@kernel.org>


> 
> From: Michael S. Tsirkin [mailto:mst@redhat.com]
> Sent: Monday, November 28, 2016 1:09 PM
> To: Gonglei (Arei)
> Subject: Re: [virtio-dev] Re: [PATCH v2 2/2] crypto: add virtio-crypto driver
> 
> On Mon, Nov 28, 2016 at 04:47:21AM +0000, Gonglei (Arei) wrote:
> > Michael, I'd like to add virtio-crypto stuff to your maintaining part likes
> > the virtio-net/blk parts so that the corresponding patches
> > can be CC'ed to you too because the virtio-crypto doesn't lay in
> > driver/virtio directory. Will you?
> >
> > Thanks!
> > -Gonglei
> 
> Sure, that's fine.
> 
Will do in the next version.  :)

Thanks,
-Gonglei

^ permalink raw reply

* Re: [virtio-dev] Re: [PATCH v2 2/2] crypto: add virtio-crypto driver
From: Michael S. Tsirkin @ 2016-11-28  5:09 UTC (permalink / raw)
  To: Gonglei (Arei)
  Cc: virtio-dev@lists.oasis-open.org, Huangweidong (C),
	Claudio Fontana, Hanweidong (Randy), qemu-devel@nongnu.org,
	Luonengjun, linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	salvatore.benedetto@intel.com, Xuquan (Quan Xu),
	linux-crypto@vger.kernel.org, stefanha@redhat.com,
	Zhoujian (jay, Euler), longpeng, arei.gonglei@hotmail.com,
	davem@davemloft.net, Wubin (H)
In-Reply-To: <33183CC9F5247A488A2544077AF19020D97A10EC@SZXEMA503-MBS.china.huawei.com>

On Mon, Nov 28, 2016 at 04:47:21AM +0000, Gonglei (Arei) wrote:
> Michael, I'd like to add virtio-crypto stuff to your maintaining part likes
> the virtio-net/blk parts so that the corresponding patches
> can be CC'ed to you too because the virtio-crypto doesn't lay in 
> driver/virtio directory. Will you?
> 
> Thanks!
> -Gonglei

Sure, that's fine.

-- 
MST

^ permalink raw reply

* RE: [virtio-dev] Re: [PATCH v2 2/2] crypto: add virtio-crypto driver
From: Gonglei (Arei) @ 2016-11-28  4:47 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: virtio-dev@lists.oasis-open.org, Huangweidong (C),
	Claudio Fontana, Hanweidong (Randy), qemu-devel@nongnu.org,
	Luonengjun, linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	salvatore.benedetto@intel.com, Xuquan (Quan Xu),
	linux-crypto@vger.kernel.org, stefanha@redhat.com,
	Zhoujian (jay, Euler), longpeng, arei.gonglei@hotmail.com,
	davem@davemloft.net, Wubin (H)
In-Reply-To: <20161127045532-mutt-send-email-mst@kernel.org>

Hi Michael,

>
> Subject: [virtio-dev] Re: [PATCH v2 2/2] crypto: add virtio-crypto driver
> 
> On Tue, Nov 22, 2016 at 04:10:23PM +0800, Gonglei wrote:
> > This patch introduces virtio-crypto driver for Linux Kernel.
> >
> > The virtio crypto device is a virtual cryptography device
> > as well as a kind of virtual hardware accelerator for
> > virtual machines. The encryption anddecryption requests
> > are placed in the data queue and are ultimately handled by
> > thebackend crypto accelerators. The second queue is the
> > control queue used to create or destroy sessions for
> > symmetric algorithms and will control some advanced features
> > in the future. The virtio crypto device provides the following
> > cryptoservices: CIPHER, MAC, HASH, and AEAD.
> >
> > For more information about virtio-crypto device, please see:
> >   http://qemu-project.org/Features/VirtioCrypto
> >
> > CC: Michael S. Tsirkin <mst@redhat.com>
> > CC: Cornelia Huck <cornelia.huck@de.ibm.com>
> > CC: Stefan Hajnoczi <stefanha@redhat.com>
> > CC: Herbert Xu <herbert@gondor.apana.org.au>
> > CC: Halil Pasic <pasic@linux.vnet.ibm.com>
> > CC: David S. Miller <davem@davemloft.net>
> > CC: Zeng Xin <xin.zeng@intel.com>
> > Signed-off-by: Gonglei <arei.gonglei@huawei.com>
> > ---
> >  MAINTAINERS                                  |   8 +
> >  drivers/crypto/Kconfig                       |   2 +
> >  drivers/crypto/Makefile                      |   1 +
> >  drivers/crypto/virtio/Kconfig                |  10 +
> >  drivers/crypto/virtio/Makefile               |   5 +
> >  drivers/crypto/virtio/virtio_crypto.c        | 444
> +++++++++++++++++++++++
> >  drivers/crypto/virtio/virtio_crypto_algs.c   | 524
> +++++++++++++++++++++++++++
> >  drivers/crypto/virtio/virtio_crypto_common.h | 124 +++++++
> >  drivers/crypto/virtio/virtio_crypto_mgr.c    | 258 +++++++++++++
> >  include/uapi/linux/Kbuild                    |   1 +
> >  include/uapi/linux/virtio_crypto.h           | 435
> ++++++++++++++++++++++
> >  include/uapi/linux/virtio_ids.h              |   1 +
> >  12 files changed, 1813 insertions(+)
> >  create mode 100644 drivers/crypto/virtio/Kconfig
> >  create mode 100644 drivers/crypto/virtio/Makefile
> >  create mode 100644 drivers/crypto/virtio/virtio_crypto.c
> >  create mode 100644 drivers/crypto/virtio/virtio_crypto_algs.c
> >  create mode 100644 drivers/crypto/virtio/virtio_crypto_common.h
> >  create mode 100644 drivers/crypto/virtio/virtio_crypto_mgr.c
> >  create mode 100644 include/uapi/linux/virtio_crypto.h
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 411e3b8..d6b18bb 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -12844,6 +12844,14 @@ S:	Maintained
> >  F:	drivers/virtio/virtio_input.c
> >  F:	include/uapi/linux/virtio_input.h
> >
> > +VIRTIO CRYPTO DRIVER
> > +M:  Gonglei <arei.gonglei@huawei.com>
> > +L:  virtualization@lists.linux-foundation.org
> > +L:  linux-crypto@vger.kernel.org
> > +S:  Maintained
> > +F:  drivers/crypto/virtio/
> > +F:  include/uapi/linux/virtio_crypto.h
> > +
> >  VIA RHINE NETWORK DRIVER
> >  S:	Orphan
> >  F:	drivers/net/ethernet/via/via-rhine.c
> > diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
> > index 4d2b81f..7956478 100644
> > --- a/drivers/crypto/Kconfig
> > +++ b/drivers/crypto/Kconfig
> > @@ -555,4 +555,6 @@ config CRYPTO_DEV_ROCKCHIP
> >
> >  source "drivers/crypto/chelsio/Kconfig"
> >
> > +source "drivers/crypto/virtio/Kconfig"
> > +
> >  endif # CRYPTO_HW
> > diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile
> > index ad7250f..bc53cb8 100644
> > --- a/drivers/crypto/Makefile
> > +++ b/drivers/crypto/Makefile
> > @@ -32,3 +32,4 @@ obj-$(CONFIG_CRYPTO_DEV_VMX) += vmx/
> >  obj-$(CONFIG_CRYPTO_DEV_SUN4I_SS) += sunxi-ss/
> >  obj-$(CONFIG_CRYPTO_DEV_ROCKCHIP) += rockchip/
> >  obj-$(CONFIG_CRYPTO_DEV_CHELSIO) += chelsio/
> > +obj-$(CONFIG_CRYPTO_DEV_VIRTIO) += virtio/
> > diff --git a/drivers/crypto/virtio/Kconfig b/drivers/crypto/virtio/Kconfig
> > new file mode 100644
> > index 0000000..ceae88c
> > --- /dev/null
> > +++ b/drivers/crypto/virtio/Kconfig
> > @@ -0,0 +1,10 @@
> > +config CRYPTO_DEV_VIRTIO
> > +	tristate "VirtIO crypto driver"
> > +	depends on VIRTIO
> > +    select CRYPTO_AEAD
> > +    select CRYPTO_AUTHENC
> > +    select CRYPTO_BLKCIPHER
> > +	default m
> > +	help
> > +	  This driver provides support for virtio crypto device. If you
> > +	  choose 'M' here, this module will be called virtio-crypto.
> > diff --git a/drivers/crypto/virtio/Makefile b/drivers/crypto/virtio/Makefile
> > new file mode 100644
> > index 0000000..a316e92
> > --- /dev/null
> > +++ b/drivers/crypto/virtio/Makefile
> > @@ -0,0 +1,5 @@
> > +obj-$(CONFIG_CRYPTO_DEV_VIRTIO) += virtio-crypto.o
> > +virtio-crypto-objs := \
> > +	virtio_crypto_algs.o \
> > +	virtio_crypto_mgr.o \
> > +	virtio_crypto.o
> > diff --git a/drivers/crypto/virtio/virtio_crypto.c
> b/drivers/crypto/virtio/virtio_crypto.c
> > new file mode 100644
> > index 0000000..56fdfed
> > --- /dev/null
> > +++ b/drivers/crypto/virtio/virtio_crypto.c
> > @@ -0,0 +1,444 @@
> > + /* Driver for Virtio crypto device.
> > +  *
> > +  * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
> > +  *
> > +  * This program is free software; you can redistribute it and/or modify
> > +  * it under the terms of the GNU General Public License as published by
> > +  * the Free Software Foundation; either version 2 of the License, or
> > +  * (at your option) any later version.
> > +  *
> > +  * This program is distributed in the hope that it will be useful,
> > +  * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +  * GNU General Public License for more details.
> > +  *
> > +  * You should have received a copy of the GNU General Public License
> > +  * along with this program; if not, see <http://www.gnu.org/licenses/>.
> > +  */
> > +
> > +#include <linux/err.h>
> > +#include <linux/module.h>
> > +#include <linux/virtio_config.h>
> > +#include <linux/cpu.h>
> > +
> > +#include <uapi/linux/virtio_crypto.h>
> > +#include "virtio_crypto_common.h"
> > +
> > +
> > +static void virtcrypto_dataq_callback(struct virtqueue *vq)
> > +{
> > +	struct virtio_crypto *vcrypto = vq->vdev->priv;
> > +	struct virtio_crypto_request *vc_req;
> > +	unsigned long flags;
> > +	unsigned int len;
> > +	struct ablkcipher_request *ablk_req;
> > +	int error;
> > +
> > +	spin_lock_irqsave(&vcrypto->lock, flags);
> > +	do {
> > +		virtqueue_disable_cb(vq);
> > +		while ((vc_req = virtqueue_get_buf(vq, &len)) != NULL) {
> > +			if (vc_req->type == VIRTIO_CRYPTO_SYM_OP_CIPHER) {
> > +				switch (vc_req->status) {
> > +				case VIRTIO_CRYPTO_OK:
> > +					error = 0;
> > +					break;
> > +				case VIRTIO_CRYPTO_INVSESS:
> > +				case VIRTIO_CRYPTO_ERR:
> > +					error = -EINVAL;
> > +					break;
> > +				case VIRTIO_CRYPTO_BADMSG:
> > +					error = -EBADMSG;
> > +					break;
> > +				default:
> > +					error = -EIO;
> > +					break;
> > +				}
> > +				ablk_req = vc_req->ablkcipher_req;
> > +				/* Finish the encrypt or decrypt process */
> > +				ablk_req->base.complete(&ablk_req->base, error);
> > +			}
> > +
> > +			kfree(vc_req->req_data);
> > +			kfree(vc_req->sgs);
> > +		}
> > +	} while (!virtqueue_enable_cb(vq));
> > +	spin_unlock_irqrestore(&vcrypto->lock, flags);
> > +}
> > +
> > +static int virtcrypto_find_vqs(struct virtio_crypto *vi)
> > +{
> > +	vq_callback_t **callbacks;
> > +	struct virtqueue **vqs;
> > +	int ret = -ENOMEM;
> > +	int i, total_vqs;
> > +	const char **names;
> > +
> > +	/* We expect 1 data virtqueue, followed by
> > +	 * possible N-1 data queues used in multiqueue mode, followed by
> > +	 * control vq.
> > +	 */
> > +	total_vqs = vi->max_data_queues + 1;
> > +
> > +	/* Allocate space for find_vqs parameters */
> > +	vqs = kcalloc(total_vqs, sizeof(*vqs), GFP_KERNEL);
> > +	if (!vqs)
> > +		goto err_vq;
> > +	callbacks = kcalloc(total_vqs, sizeof(*callbacks), GFP_KERNEL);
> > +	if (!callbacks)
> > +		goto err_callback;
> > +	names = kcalloc(total_vqs, sizeof(*names), GFP_KERNEL);
> > +	if (!names)
> > +		goto err_names;
> > +
> > +	/* Parameters for control virtqueue */
> > +	callbacks[total_vqs - 1] = NULL;
> > +	names[total_vqs - 1] = "controlq";
> > +
> > +	/* Allocate/initialize parameters for data virtqueues */
> > +	for (i = 0; i < vi->max_data_queues; i++) {
> > +		callbacks[i] = virtcrypto_dataq_callback;
> > +		snprintf(vi->data_vq[i].name, sizeof(vi->data_vq[i].name),
> > +				"dataq.%d", i);
> > +		names[i] = vi->data_vq[i].name;
> > +	}
> > +
> > +	ret = vi->vdev->config->find_vqs(vi->vdev, total_vqs, vqs, callbacks,
> > +					 names);
> > +	if (ret)
> > +		goto err_find;
> > +
> > +	vi->ctrl_vq = vqs[total_vqs - 1];
> > +
> > +	for (i = 0; i < vi->max_data_queues; i++)
> > +		vi->data_vq[i].vq = vqs[i];
> > +
> > +	kfree(names);
> > +	kfree(callbacks);
> > +	kfree(vqs);
> > +
> > +	return 0;
> > +
> > +err_find:
> > +	kfree(names);
> > +err_names:
> > +	kfree(callbacks);
> > +err_callback:
> > +	kfree(vqs);
> > +err_vq:
> > +	return ret;
> > +}
> > +
> > +static int virtcrypto_alloc_queues(struct virtio_crypto *vi)
> > +{
> > +	vi->data_vq = kcalloc(vi->max_data_queues, sizeof(*vi->data_vq),
> > +				GFP_KERNEL);
> > +	if (!vi->data_vq)
> > +		return -ENOMEM;
> > +
> > +	return 0;
> > +}
> > +
> > +static void virtcrypto_clean_affinity(struct virtio_crypto *vi, long hcpu)
> > +{
> > +	int i;
> > +
> > +	if (vi->affinity_hint_set) {
> > +		for (i = 0; i < vi->max_data_queues; i++)
> > +			virtqueue_set_affinity(vi->data_vq[i].vq, -1);
> > +
> > +		vi->affinity_hint_set = false;
> > +	}
> > +}
> > +
> > +static void virtcrypto_set_affinity(struct virtio_crypto *vi)
> > +{
> > +	int i;
> > +	int cpu;
> > +
> > +	/*
> > +	 * In multiqueue mode, when the number of cpu is equal to the number
> > +	 * of queue, we let the queue to be private to one cpu by
> > +	 * setting the affinity hint to eliminate the contention.
> > +	 */
> > +	if (vi->curr_queue == 1 ||
> > +	    vi->max_data_queues != num_online_cpus()) {
> > +		virtcrypto_clean_affinity(vi, -1);
> > +		return;
> > +	}
> > +
> 
> So how can you handle cpu hotplug then?
> I would imagine making max_data_queues > num_online_cpus
> and initializing just the required number would be
> a saner way.
> 

It makes sense.

> > +	i = 0;
> > +	for_each_online_cpu(cpu) {
> > +		virtqueue_set_affinity(vi->data_vq[i].vq, cpu);
> > +		i++;
> > +	}
> > +
> > +	vi->affinity_hint_set = true;
> > +}
> > +
> 
> Above is racy in prsence of cpu hotplug. Is there a userspace
> interface to fix up affinity after hotplug?

Not yet.

> If not you need to get notified after hotplug and
> fix it up yourself.
> 
I'll inspect it.

> > +static void virtcrypto_free_queues(struct virtio_crypto *vi)
> > +{
> > +	kfree(vi->data_vq);
> > +}
> > +
> > +static int virtcrypto_init_vqs(struct virtio_crypto *vi)
> > +{
> > +	int ret;
> > +
> > +	/* Allocate send & receive queues */
> > +	ret = virtcrypto_alloc_queues(vi);
> > +	if (ret)
> > +		goto err;
> > +
> > +	ret = virtcrypto_find_vqs(vi);
> > +	if (ret)
> > +		goto err_free;
> > +
> > +	get_online_cpus();
> > +	virtcrypto_set_affinity(vi);
> > +	put_online_cpus();
> > +
> > +	return 0;
> > +
> > +err_free:
> > +	virtcrypto_free_queues(vi);
> > +err:
> > +	return ret;
> > +}
> > +
> > +static int virtcrypto_update_status(struct virtio_crypto *vcrypto)
> > +{
> > +	__le32 status_le;
> > +	u32 status;
> > +	int err;
> > +
> > +	status_le = virtio_cread32_le(vcrypto->vdev,
> > +			offsetof(struct virtio_crypto_config, status));
> > +	status = le32_to_cpu(status_le);
> > +
> > +	/* Ignore unknown (future) status bits */
> > +	status &= VIRTIO_CRYPTO_S_HW_READY;
> > +
> > +	if (vcrypto->status == status)
> > +		return 0;
> > +
> > +	vcrypto->status = status;
> > +
> > +	if (vcrypto->status & VIRTIO_CRYPTO_S_HW_READY) {
> > +		err = virtcrypto_dev_start(vcrypto);
> > +		if (err) {
> > +			dev_err(&vcrypto->vdev->dev,
> > +				"Failed to start virtio crypto device.\n");
> > +			virtcrypto_dev_stop(vcrypto);
> > +			return -1;
> > +		}
> > +		dev_info(&vcrypto->vdev->dev, "Accelerator is ready\n");
> > +	} else {
> > +		virtcrypto_dev_stop(vcrypto);
> > +		dev_info(&vcrypto->vdev->dev, "Accelerator is not ready\n");
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void virtcrypto_del_vqs(struct virtio_crypto *vcrypto)
> > +{
> > +	struct virtio_device *vdev = vcrypto->vdev;
> > +
> > +	virtcrypto_clean_affinity(vcrypto, -1);
> > +
> > +	vdev->config->del_vqs(vdev);
> > +
> > +	virtcrypto_free_queues(vcrypto);
> > +}
> > +
> > +static int virtcrypto_probe(struct virtio_device *vdev)
> > +{
> > +	int err = -EFAULT;
> > +	struct virtio_crypto *vcrypto;
> > +	__le32 max_data_queues_le = 0, max_cipher_key_len_le = 0;
> > +	__le32 max_auth_key_len_le = 0;
> > +	__le64 max_size_le = 0;
> > +
> 
> Pls validate that it's a modern device (has VERSION_1 set).
> We should find a way to do it in a central place,
> but we don't have that now.
> 
So what I will do is add a VERSION_1 feature bit checking at the
beginning of the probe function:

if (!virtio_has_feature(vdev, VIRTIO_F_VERSION_1))
	return -ENODEV;

> > +	if (!vdev->config->get) {
> > +		dev_err(&vdev->dev, "%s failure: config access disabled\n",
> > +			__func__);
> > +		return -EINVAL;
> > +	}
> > +
> > +	if (num_possible_nodes() > 1 && dev_to_node(&vdev->dev) < 0) {
> > +		/*
> > +		 * If the accelerator is connected to a node with no memory
> > +		 * there is no point in using the accelerator since the remote
> > +		 * memory transaction will be very slow.
> > +		 */
> > +		dev_err(&vdev->dev, "Invalid NUMA configuration.\n");
> > +		return -EINVAL;
> > +	}
> > +
> > +	vcrypto = kzalloc_node(sizeof(*vcrypto), GFP_KERNEL,
> > +					dev_to_node(&vdev->dev));
> > +	if (!vcrypto)
> > +		return -ENOMEM;
> > +
> > +	max_data_queues_le = virtio_cread32_le(vdev,
> > +			offsetof(struct virtio_crypto_config, max_dataqueues));
> > +	vcrypto->max_data_queues = le32_to_cpu(max_data_queues_le);
> > +	if (vcrypto->max_data_queues < 1)
> > +		vcrypto->max_data_queues = 1;
> > +
> > +	dev_info(&vdev->dev, "max_queues: %u\n", vcrypto->max_data_queues);
> > +
> > +	max_cipher_key_len_le = virtio_cread32_le(vdev,
> > +		offsetof(struct virtio_crypto_config, max_cipher_key_len));
> > +	max_auth_key_len_le = virtio_cread32_le(vdev,
> > +		offsetof(struct virtio_crypto_config, max_auth_key_len));
> > +	max_size_le = virtio_cread64_le(vdev,
> > +		offsetof(struct virtio_crypto_config, max_size));
> > +
> > +	/* Add virtio crypto device to global table */
> > +	err = virtcrypto_devmgr_add_dev(vcrypto);
> > +	if (err) {
> > +		dev_err(&vdev->dev, "Failed to add new virtio crypto device.\n");
> > +		goto free;
> > +	}
> > +	vcrypto->owner = THIS_MODULE;
> > +	vcrypto = vdev->priv = vcrypto;
> > +	vcrypto->vdev = vdev;
> > +	spin_lock_init(&vcrypto->lock);
> > +	spin_lock_init(&vcrypto->ctrl_lock);
> > +
> > +	/* Use sigle data queue as default */
> > +	vcrypto->curr_queue = 1;
> > +	vcrypto->max_cipher_key_len = le32_to_cpu(max_cipher_key_len_le);
> > +	vcrypto->max_auth_key_len = le32_to_cpu(max_auth_key_len_le);
> > +	vcrypto->max_size = le64_to_cpu(max_size_le);
> > +
> > +	err = virtcrypto_init_vqs(vcrypto);
> > +	if (err) {
> > +		dev_err(&vdev->dev, "Failed to initialize vqs.\n");
> > +		goto free_dev;
> > +	}
> > +	virtio_device_ready(vdev);
> > +
> > +	err = virtcrypto_update_status(vcrypto);
> > +	if (err) {
> > +		err = -EFAULT;
> 
> Why EFAULT?
> 
I wanted to keep the same with err's initial value.
Maybe I shouldn't change the err here but directly
assign value in virtcrypto_update_status().

EPERM is better.


> > +		goto free_vqs;
> > +	}
> > +
> > +	return 0;
> > +
> > +free_vqs:
> > +	virtcrypto_del_vqs(vcrypto);
> 
> as you called virtio_device_ready this is likely unsafe
> unless you first reset device or make sure there are
> no interrupts in some other way.
> 
Good catch. Will add reset operations.

> > +free_dev:
> > +	virtcrypto_devmgr_rm_dev(vcrypto);
> > +free:
> > +	kfree(vcrypto);
> > +	return err;
> > +}
> > +
> > +static void virtcrypto_free_unused_reqs(struct virtio_crypto *vcrypto)
> > +{
> > +	struct virtio_crypto_request *vc_req;
> > +	int i;
> > +	struct virtqueue *vq;
> > +
> > +	for (i = 0; i < vcrypto->max_data_queues; i++) {
> > +		vq = vcrypto->data_vq[i].vq;
> > +		while ((vc_req = virtqueue_detach_unused_buf(vq)) != NULL) {
> > +			kfree(vc_req->req_data);
> > +			kfree(vc_req->sgs);
> > +		}
> > +	}
> > +}
> > +
> > +static void virtcrypto_remove(struct virtio_device *vdev)
> > +{
> > +	struct virtio_crypto *vcrypto = vdev->priv;
> > +
> > +	dev_info(&vdev->dev, "Start virtcrypto_remove.\n");
> > +
> > +	if (virtcrypto_dev_started(vcrypto))
> > +		virtcrypto_dev_stop(vcrypto);
> > +	vdev->config->reset(vdev);
> > +	virtcrypto_free_unused_reqs(vcrypto);
> > +	virtcrypto_del_vqs(vcrypto);
> > +	virtcrypto_devmgr_rm_dev(vcrypto);
> > +	kfree(vcrypto);
> > +}
> > +
> > +static void virtcrypto_config_changed(struct virtio_device *vdev)
> > +{
> > +	struct virtio_crypto *vcrypto = vdev->priv;
> > +
> > +	virtcrypto_update_status(vcrypto);
> > +}
> > +
> > +#ifdef CONFIG_PM_SLEEP
> > +static int virtcrypto_freeze(struct virtio_device *vdev)
> > +{
> > +	struct virtio_crypto *vcrypto = vdev->priv;
> > +	unsigned long flags;
> > +
> > +	virtcrypto_free_unused_reqs(vcrypto);
> > +	spin_lock_irqsave(&vcrypto->lock, flags);
> > +	if (virtcrypto_dev_started(vcrypto))
> > +		virtcrypto_dev_stop(vcrypto);
> 
> This is taking mutex locks under a spinlock.
> Highly unlikely to work. Try this with lockdep
> enabled.
> 
Good catch, will do and inspect the necessity of 
the spin_lock here as well.  
> 
> > +	spin_unlock_irqrestore(&vcrypto->lock, flags);
> > +
> > +	virtcrypto_del_vqs(vcrypto);
> > +	return 0;
> > +}
> > +
> > +static int virtcrypto_restore(struct virtio_device *vdev)
> > +{
> > +	struct virtio_crypto *vcrypto = vdev->priv;
> > +	int err;
> > +
> > +	err = virtcrypto_init_vqs(vcrypto);
> > +	if (err)
> > +		return err;
> > +
> > +	virtio_device_ready(vdev);
> > +	err = virtcrypto_dev_start(vcrypto);
> > +	if (err) {
> > +		dev_err(&vdev->dev, "Failed to start virtio crypto device.\n");
> > +		return -EFAULT;
> > +	}
> > +
> > +	return 0;
> > +}
> > +#endif
> > +
> > +
> > +static unsigned int features[] = {
> > +	/* none */
> > +};
> > +
> > +static struct virtio_device_id id_table[] = {
> > +	{ VIRTIO_ID_CRYPTO, VIRTIO_DEV_ANY_ID },
> > +	{ 0 },
> > +};
> > +
> > +static struct virtio_driver virtio_crypto_driver = {
> > +	.driver.name         = KBUILD_MODNAME,
> > +	.driver.owner        = THIS_MODULE,
> > +	.feature_table       = features,
> > +	.feature_table_size  = ARRAY_SIZE(features),
> > +	.id_table            = id_table,
> > +	.probe               = virtcrypto_probe,
> > +	.remove              = virtcrypto_remove,
> > +	.config_changed = virtcrypto_config_changed,
> > +#ifdef CONFIG_PM_SLEEP
> > +	.freeze = virtcrypto_freeze,
> > +	.restore = virtcrypto_restore,
> > +#endif
> > +};
> > +
> > +module_virtio_driver(virtio_crypto_driver);
> > +
> > +MODULE_DEVICE_TABLE(virtio, id_table);
> > +MODULE_DESCRIPTION("virtio crypto device driver");
> > +MODULE_LICENSE("GPL");
> > +MODULE_AUTHOR("Gonglei <arei.gonglei@huawei.com>");
> > diff --git a/drivers/crypto/virtio/virtio_crypto_algs.c
> b/drivers/crypto/virtio/virtio_crypto_algs.c
> > new file mode 100644
> > index 0000000..7b57584
> > --- /dev/null
> > +++ b/drivers/crypto/virtio/virtio_crypto_algs.c
> > @@ -0,0 +1,524 @@
> > + /* Algorithms supported by virtio crypto device
> > +  *
> > +  * Authors: Gonglei <arei.gonglei@huawei.com>
> > +  *
> > +  * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
> > +  *
> > +  * This program is free software; you can redistribute it and/or modify
> > +  * it under the terms of the GNU General Public License as published by
> > +  * the Free Software Foundation; either version 2 of the License, or
> > +  * (at your option) any later version.
> > +  *
> > +  * This program is distributed in the hope that it will be useful,
> > +  * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +  * GNU General Public License for more details.
> > +  *
> > +  * You should have received a copy of the GNU General Public License
> > +  * along with this program; if not, see <http://www.gnu.org/licenses/>.
> > +  */
> > +
> > +#include <linux/scatterlist.h>
> > +#include <crypto/algapi.h>
> > +#include <linux/err.h>
> > +#include <crypto/scatterwalk.h>
> > +#include <linux/atomic.h>
> > +
> > +#include <uapi/linux/virtio_crypto.h>
> > +#include "virtio_crypto_common.h"
> > +
> > +static DEFINE_MUTEX(algs_lock);
> > +static unsigned int virtio_crypto_active_devs;
> > +
> > +static u64 virtio_crypto_alg_sg_nents_length(struct scatterlist *sg)
> > +{
> > +	u64 total = 0;
> > +
> > +	for (total = 0; sg; sg = sg_next(sg))
> > +		total += sg->length;
> > +
> > +	return total;
> > +}
> > +
> > +static int virtio_crypto_alg_validate_key(int key_len, int *alg)
> > +{
> > +	switch (key_len) {
> > +	case AES_KEYSIZE_128:
> > +	case AES_KEYSIZE_192:
> > +	case AES_KEYSIZE_256:
> > +		*alg = VIRTIO_CRYPTO_CIPHER_AES_CBC;
> > +		break;
> > +	default:
> > +		return -EINVAL;
> > +	}
> > +	return 0;
> > +}
> > +
> > +static int virtio_crypto_alg_ablkcipher_init_session(
> > +		struct virtio_crypto_ablkcipher_ctx *ctx,
> > +		int alg, const uint8_t *key,
> > +		unsigned int keylen,
> > +		int encrypt)
> > +{
> > +	struct scatterlist outhdr, key_sg, inhdr, *sgs[3];
> > +	unsigned int tmp;
> > +	struct virtio_crypto *vcrypto = ctx->vcrypto;
> > +	int op = encrypt ? VIRTIO_CRYPTO_OP_ENCRYPT :
> VIRTIO_CRYPTO_OP_DECRYPT;
> > +	int err;
> > +	unsigned int num_out = 0, num_in = 0;
> > +
> > +	/*
> > +	 * Avoid to do DMA from the stack, switch to using
> > +	 * dynamically-allocated for the key
> > +	 */
> > +	uint8_t *cipher_key = kmalloc(keylen, GFP_ATOMIC);
> 
> Is all crypto using GFP_ATOMIC like this?
> Is there code that recovers if this fails?
> 
The crypto subsystem has a flag named CRYPTO_TFM_REQ_MAY_SLEEP to
decide to use GFP_KERENL or GFP_ATOMIC. We don't set the flag
for virtio-crypto device, so I use GFP_ATOMIC likes qat_alg.c

> 
> > +
> > +	if (!cipher_key)
> > +		return -ENOMEM;
> > +
> > +	memcpy(cipher_key, key, keylen);
> > +
> > +	spin_lock(&vcrypto->ctrl_lock);
> > +	/* Pad ctrl header */
> > +	vcrypto->ctrl.header.opcode =
> > +		cpu_to_le32(VIRTIO_CRYPTO_CIPHER_CREATE_SESSION);
> > +	vcrypto->ctrl.header.algo = cpu_to_le32((uint32_t)alg);
> 
> why cast here?
> 
Will change the type of alg at the beginning of definition,
remove the cast as well.

> > +	/* Set the default dataqueue id to 0 */
> > +	vcrypto->ctrl.header.queue_id = 0;
> > +
> > +	vcrypto->input.status = cpu_to_le32(VIRTIO_CRYPTO_ERR);
> > +	/* Pad cipher's parameters */
> > +	vcrypto->ctrl.u.sym_create_session.op_type =
> > +		cpu_to_le32(VIRTIO_CRYPTO_SYM_OP_CIPHER);
> > +	vcrypto->ctrl.u.sym_create_session.u.cipher.para.algo =
> > +		vcrypto->ctrl.header.algo;
> > +	vcrypto->ctrl.u.sym_create_session.u.cipher.para.keylen =
> > +		cpu_to_le32(keylen);
> > +	vcrypto->ctrl.u.sym_create_session.u.cipher.para.op =
> > +		cpu_to_le32(op);
> > +
> > +	sg_init_one(&outhdr, &vcrypto->ctrl, sizeof(vcrypto->ctrl));
> > +	sgs[num_out++] = &outhdr;
> > +
> > +	/* Set key */
> > +	sg_init_one(&key_sg, cipher_key, keylen);
> > +	sgs[num_out++] = &key_sg;
> > +
> > +	/* Return status and session id back */
> > +	sg_init_one(&inhdr, &vcrypto->input, sizeof(vcrypto->input));
> > +	sgs[num_out + num_in++] = &inhdr;
> > +
> > +	err = virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out,
> > +				num_in, vcrypto, GFP_ATOMIC);
> > +	if (err < 0) {
> > +		spin_unlock(&vcrypto->ctrl_lock);
> > +		kfree(cipher_key);
> > +		return err;
> > +	}
> > +	virtqueue_kick(vcrypto->ctrl_vq);
> > +
> > +	/*
> > +	 * Spin for a response, the kick causes an ioport write, trapping
> > +	 * into the hypervisor, so the request should be handled immediately.
> > +	 */
> > +	while (!virtqueue_get_buf(vcrypto->ctrl_vq, &tmp) &&
> > +	       !virtqueue_is_broken(vcrypto->ctrl_vq))
> > +		cpu_relax();
> > +
> > +	if (le32_to_cpu(vcrypto->input.status) != VIRTIO_CRYPTO_OK) {
> > +		spin_unlock(&vcrypto->ctrl_lock);
> > +		pr_err("virtio_crypto: Create session failed status: %u\n",
> > +			le32_to_cpu(vcrypto->input.status));
> > +		kfree(cipher_key);
> > +		return -EINVAL;
> > +	}
> > +	spin_unlock(&vcrypto->ctrl_lock);
> > +
> > +	spin_lock(&ctx->lock);
> > +	if (encrypt)
> > +		ctx->enc_sess_info.session_id =
> > +			le64_to_cpu(vcrypto->input.session_id);
> > +	else
> > +		ctx->dec_sess_info.session_id =
> > +			le64_to_cpu(vcrypto->input.session_id);
> > +	spin_unlock(&ctx->lock);
> > +
> > +	kfree(cipher_key);
> > +	return 0;
> > +}
> > +
> > +static int virtio_crypto_alg_ablkcipher_close_session(
> > +		struct virtio_crypto_ablkcipher_ctx *ctx,
> > +		int encrypt)
> > +{
> > +	struct scatterlist outhdr, status_sg, *sgs[2];
> > +	unsigned int tmp;
> > +	struct virtio_crypto_destroy_session_req *destroy_session;
> > +	struct virtio_crypto *vcrypto = ctx->vcrypto;
> > +	int err;
> > +	unsigned int num_out = 0, num_in = 0;
> > +
> > +	spin_lock(&vcrypto->ctrl_lock);
> > +	vcrypto->ctrl_status.status = VIRTIO_CRYPTO_ERR;
> > +	/* Pad ctrl header */
> > +	vcrypto->ctrl.header.opcode =
> > +		cpu_to_le32(VIRTIO_CRYPTO_CIPHER_DESTROY_SESSION);
> > +	/* Set the default virtqueue id to 0 */
> > +	vcrypto->ctrl.header.queue_id = 0;
> > +
> > +	destroy_session = &vcrypto->ctrl.u.destroy_session;
> > +
> > +	if (encrypt)
> > +		destroy_session->session_id =
> > +			cpu_to_le64(ctx->enc_sess_info.session_id);
> > +	else
> > +		destroy_session->session_id =
> > +			cpu_to_le64(ctx->dec_sess_info.session_id);
> > +
> > +	sg_init_one(&outhdr, &vcrypto->ctrl, sizeof(vcrypto->ctrl));
> > +	sgs[num_out++] = &outhdr;
> > +
> > +	/* Return status and session id back */
> > +	sg_init_one(&status_sg, &vcrypto->ctrl_status.status,
> > +		sizeof(vcrypto->ctrl_status.status));
> > +	sgs[num_out + num_in++] = &status_sg;
> > +
> > +	err = virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out,
> > +			num_in, vcrypto, GFP_ATOMIC);
> > +	if (err < 0) {
> > +		spin_unlock(&vcrypto->ctrl_lock);
> > +		return err;
> > +	}
> > +	virtqueue_kick(vcrypto->ctrl_vq);
> > +
> > +	while (!virtqueue_get_buf(vcrypto->ctrl_vq, &tmp) &&
> > +	       !virtqueue_is_broken(vcrypto->ctrl_vq))
> > +		cpu_relax();
> > +
> > +	if (vcrypto->ctrl_status.status != VIRTIO_CRYPTO_OK) {
> > +		spin_unlock(&vcrypto->ctrl_lock);
> > +		pr_err("virtio_crypto: Close session failed status: %u, session_id:
> 0x%llx\n",
> > +			vcrypto->ctrl_status.status,
> > +			destroy_session->session_id);
> > +
> > +		return -EINVAL;
> > +	}
> > +	spin_unlock(&vcrypto->ctrl_lock);
> > +
> > +	return 0;
> > +}
> > +
> > +static int virtio_crypto_alg_ablkcipher_init_sessions(
> > +		struct virtio_crypto_ablkcipher_ctx *ctx,
> > +		const uint8_t *key, unsigned int keylen)
> > +{
> > +	int alg;
> > +	int ret;
> > +	struct virtio_crypto *vcrypto = ctx->vcrypto;
> > +
> > +	if (keylen > vcrypto->max_cipher_key_len) {
> > +		pr_err("virtio_crypto: the key is too long\n");
> > +		goto bad_key;
> > +	}
> > +
> > +	if (virtio_crypto_alg_validate_key(keylen, &alg))
> > +		goto bad_key;
> > +
> > +	/* Create encryption session */
> > +	ret = virtio_crypto_alg_ablkcipher_init_session(ctx,
> > +			alg, key, keylen, 1);
> > +	if (ret)
> > +		return ret;
> > +	/* Create decryption session */
> > +	ret = virtio_crypto_alg_ablkcipher_init_session(ctx,
> > +			alg, key, keylen, 0);
> > +	if (ret) {
> > +		virtio_crypto_alg_ablkcipher_close_session(ctx, 1);
> > +		return ret;
> > +	}
> > +	return 0;
> > +
> > +bad_key:
> > +	crypto_tfm_set_flags(ctx->tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
> > +	return -EINVAL;
> > +}
> > +
> > +/* Note: kernel crypto API realization */
> > +static int virtio_crypto_ablkcipher_setkey(struct crypto_ablkcipher *tfm,
> > +					 const uint8_t *key,
> > +					 unsigned int keylen)
> > +{
> > +	struct virtio_crypto_ablkcipher_ctx *ctx = crypto_ablkcipher_ctx(tfm);
> > +	int ret;
> > +
> > +	spin_lock(&ctx->lock);
> > +
> > +	if (!ctx->vcrypto) {
> > +		/* New key */
> > +		int node = virtio_crypto_get_current_node();
> > +		struct virtio_crypto *vcrypto =
> > +				      virtcrypto_get_dev_node(node);
> > +		if (!vcrypto) {
> > +			vcrypto = virtcrypto_devmgr_get_first();
> > +			if (!vcrypto) {
> > +				pr_err("virtio_crypto: Could not find a virtio device in the
> system");
> > +				spin_unlock(&ctx->lock);
> > +				return -ENODEV;
> > +			}
> > +		}
> > +
> > +		ctx->vcrypto = vcrypto;
> > +	}
> > +	spin_unlock(&ctx->lock);
> > +
> > +	ret = virtio_crypto_alg_ablkcipher_init_sessions(ctx, key, keylen);
> > +	if (ret) {
> > +		virtcrypto_dev_put(ctx->vcrypto);
> > +		ctx->vcrypto = NULL;
> > +
> > +		return ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int
> > +__virtio_crypto_ablkcipher_do_req(struct virtio_crypto_request *vc_req,
> > +		struct ablkcipher_request *req,
> > +		struct data_queue *data_vq,
> > +		__u8 op)
> > +{
> > +	struct virtio_crypto_ablkcipher_ctx *ctx = vc_req->ablkcipher_ctx;
> > +	struct virtio_crypto *vcrypto = ctx->vcrypto;
> > +	struct virtio_crypto_op_data_req *req_data;
> > +	int src_nents, dst_nents;
> > +	int err;
> > +	unsigned long flags;
> > +	struct scatterlist outhdr, iv_sg, status_sg, **sgs;
> > +	int i;
> > +	u64 dst_len;
> > +	unsigned int num_out = 0, num_in = 0;
> > +	int sg_total;
> > +
> > +	src_nents = sg_nents_for_len(req->src, req->nbytes);
> > +	dst_nents = sg_nents(req->dst);
> > +
> > +	pr_debug("virtio_crypto: Number of sgs (src_nents: %d,
> dst_nents: %d)\n",
> > +			src_nents, dst_nents);
> > +
> > +	/* Why 3?  outhdr + iv + inhdr */
> > +	sg_total = src_nents + dst_nents + 3;
> > +	sgs = kzalloc_node(sg_total * sizeof(*sgs), GFP_ATOMIC,
> > +				dev_to_node(&vcrypto->vdev->dev));
> > +	if (!sgs)
> > +		return -ENOMEM;
> > +
> > +	req_data = kzalloc_node(sizeof(*req_data), GFP_ATOMIC,
> > +				dev_to_node(&vcrypto->vdev->dev));
> > +	if (!req_data) {
> > +		kfree(sgs);
> > +		return -ENOMEM;
> > +	}
> > +
> > +	vc_req->req_data = req_data;
> > +	vc_req->type = VIRTIO_CRYPTO_SYM_OP_CIPHER;
> > +	/* Head of operation */
> > +	if (op) {
> > +		req_data->header.session_id =
> > +			cpu_to_le64(ctx->enc_sess_info.session_id);
> > +		req_data->header.opcode =
> > +			cpu_to_le32(VIRTIO_CRYPTO_CIPHER_ENCRYPT);
> > +	} else {
> > +		req_data->header.session_id =
> > +			cpu_to_le64(ctx->dec_sess_info.session_id);
> > +	    req_data->header.opcode =
> > +			cpu_to_le32(VIRTIO_CRYPTO_CIPHER_DECRYPT);
> > +	}
> > +	req_data->u.sym_req.op_type =
> cpu_to_le32(VIRTIO_CRYPTO_SYM_OP_CIPHER);
> > +	req_data->u.sym_req.u.cipher.para.iv_len =
> cpu_to_le32(AES_BLOCK_SIZE);
> > +	req_data->u.sym_req.u.cipher.para.src_data_len =
> > +			cpu_to_le32(req->nbytes);
> > +
> > +	dst_len = virtio_crypto_alg_sg_nents_length(req->dst);
> > +	if (unlikely(dst_len > U32_MAX)) {
> > +		pr_err("virtio_crypto: The dst_len is beyond U32_MAX\n");
> > +		err = -EINVAL;
> > +		goto free;
> > +	}
> > +
> > +	pr_debug("virtio_crypto: src_len: %u, dst_len: %llu\n",
> > +			req->nbytes, dst_len);
> > +
> > +	if (unlikely(req->nbytes + dst_len + AES_BLOCK_SIZE +
> > +		sizeof(vc_req->status) > vcrypto->max_size)) {
> > +		pr_err("virtio_crypto: The length is too big\n");
> > +		err = -EINVAL;
> > +		goto free;
> > +	}
> > +
> > +	req_data->u.sym_req.u.cipher.para.dst_data_len =
> > +			cpu_to_le32((uint32_t)dst_len);
> > +
> > +	/* Outhdr */
> > +	sg_init_one(&outhdr, req_data, sizeof(*req_data));
> > +	sgs[num_out++] = &outhdr;
> > +
> > +	/* IV */
> > +	sg_init_one(&iv_sg, req->info, AES_BLOCK_SIZE);
> > +	sgs[num_out++] = &iv_sg;
> > +
> > +	/* Source data */
> > +	for (i = 0; i < src_nents; i++)
> > +		sgs[num_out++] = &req->src[i];
> > +
> > +	/* Destination data */
> > +	for (i = 0; i < dst_nents; i++)
> > +		sgs[num_out + num_in++] = &req->dst[i];
> > +
> > +	/* Status */
> > +	sg_init_one(&status_sg, &vc_req->status, sizeof(vc_req->status));
> > +	sgs[num_out + num_in++] = &status_sg;
> > +
> > +	vc_req->sgs = sgs;
> > +
> > +	spin_lock_irqsave(&vcrypto->lock, flags);
> > +	err = virtqueue_add_sgs(data_vq->vq, sgs, num_out,
> > +				num_in, vc_req, GFP_ATOMIC);
> > +	spin_unlock_irqrestore(&vcrypto->lock, flags);
> > +	if (unlikely(err < 0))
> > +		goto free;
> > +
> > +	return 0;
> > +
> > +free:
> > +	kfree(req_data);
> > +	kfree(sgs);
> > +	return err;
> > +}
> > +
> > +static int virtio_crypto_ablkcipher_encrypt(struct ablkcipher_request *req)
> > +{
> > +	struct crypto_ablkcipher *atfm = crypto_ablkcipher_reqtfm(req);
> > +	struct virtio_crypto_ablkcipher_ctx *ctx = crypto_ablkcipher_ctx(atfm);
> > +	struct virtio_crypto_request *vc_req = ablkcipher_request_ctx(req);
> > +	struct virtio_crypto *vcrypto = ctx->vcrypto;
> > +	int ret;
> > +	/* Use the first data virtqueue as default */
> > +	struct data_queue *data_vq = &vcrypto->data_vq[0];
> > +
> > +	vc_req->ablkcipher_ctx = ctx;
> > +	vc_req->ablkcipher_req = req;
> > +	ret = __virtio_crypto_ablkcipher_do_req(vc_req, req, data_vq, 1);
> > +	if (ret < 0) {
> > +		pr_err("virtio_crypto: Encryption failed!\n");
> > +		return ret;
> > +	}
> > +	virtqueue_kick(data_vq->vq);
> > +
> > +	return -EINPROGRESS;
> > +}
> > +
> > +static int virtio_crypto_ablkcipher_decrypt(struct ablkcipher_request *req)
> > +{
> > +	struct crypto_ablkcipher *atfm = crypto_ablkcipher_reqtfm(req);
> > +	struct virtio_crypto_ablkcipher_ctx *ctx = crypto_ablkcipher_ctx(atfm);
> > +	struct virtio_crypto_request *vc_req = ablkcipher_request_ctx(req);
> > +	struct virtio_crypto *vcrypto = ctx->vcrypto;
> > +	int ret;
> > +	/* Use the first data virtqueue as default */
> > +	struct data_queue *data_vq = &vcrypto->data_vq[0];
> > +
> > +	vc_req->ablkcipher_ctx = ctx;
> > +	vc_req->ablkcipher_req = req;
> > +
> > +	ret = __virtio_crypto_ablkcipher_do_req(vc_req, req, data_vq, 0);
> > +	if (ret < 0) {
> > +		pr_err("virtio_crypto: Decryption failed!\n");
> > +		return ret;
> > +	}
> > +	virtqueue_kick(data_vq->vq);
> > +
> > +	return -EINPROGRESS;
> > +}
> > +
> > +static int virtio_crypto_ablkcipher_init(struct crypto_tfm *tfm)
> > +{
> > +	struct virtio_crypto_ablkcipher_ctx *ctx = crypto_tfm_ctx(tfm);
> > +
> > +	spin_lock_init(&ctx->lock);
> > +	tfm->crt_ablkcipher.reqsize = sizeof(struct virtio_crypto_request);
> > +	ctx->tfm = tfm;
> > +
> > +	return 0;
> > +}
> > +
> > +static void virtio_crypto_ablkcipher_exit(struct crypto_tfm *tfm)
> > +{
> > +	struct virtio_crypto_ablkcipher_ctx *ctx = crypto_tfm_ctx(tfm);
> > +
> > +	if (!ctx->vcrypto)
> > +		return;
> > +
> > +	virtio_crypto_alg_ablkcipher_close_session(ctx, 1);
> > +	virtio_crypto_alg_ablkcipher_close_session(ctx, 0);
> > +	virtcrypto_dev_put(ctx->vcrypto);
> > +	ctx->vcrypto = NULL;
> > +}
> > +
> > +static struct crypto_alg virtio_crypto_algs[] = { {
> > +	.cra_name = "cbc(aes)",
> > +	.cra_driver_name = "virtio_crypto_aes_cbc",
> > +	.cra_priority = 4001,
> > +	.cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER | CRYPTO_ALG_ASYNC,
> > +	.cra_blocksize = AES_BLOCK_SIZE,
> > +	.cra_ctxsize  = sizeof(struct virtio_crypto_ablkcipher_ctx),
> > +	.cra_alignmask = 0,
> > +	.cra_module = THIS_MODULE,
> > +	.cra_type = &crypto_ablkcipher_type,
> > +	.cra_init = virtio_crypto_ablkcipher_init,
> > +	.cra_exit = virtio_crypto_ablkcipher_exit,
> > +	.cra_u = {
> > +	   .ablkcipher = {
> > +			.setkey = virtio_crypto_ablkcipher_setkey,
> > +			.decrypt = virtio_crypto_ablkcipher_decrypt,
> > +			.encrypt = virtio_crypto_ablkcipher_encrypt,
> > +			.min_keysize = AES_MIN_KEY_SIZE,
> > +			.max_keysize = AES_MAX_KEY_SIZE,
> > +			.ivsize = AES_BLOCK_SIZE,
> > +		},
> > +	},
> > +} };
> > +
> > +int virtio_crypto_algs_register(void)
> > +{
> > +	int ret = 0, i;
> > +
> > +	mutex_lock(&algs_lock);
> > +	if (++virtio_crypto_active_devs != 1)
> > +		goto unlock;
> > +
> > +	for (i = 0; i < ARRAY_SIZE(virtio_crypto_algs); i++) {
> > +		virtio_crypto_algs[i].cra_flags =
> > +			     CRYPTO_ALG_TYPE_ABLKCIPHER | CRYPTO_ALG_ASYNC;
> > +	}
> > +
> > +	ret = crypto_register_algs(virtio_crypto_algs,
> > +			ARRAY_SIZE(virtio_crypto_algs));
> > +
> > +unlock:
> > +	mutex_unlock(&algs_lock);
> > +	return ret;
> > +}
> > +
> > +void virtio_crypto_algs_unregister(void)
> > +{
> > +	mutex_lock(&algs_lock);
> > +	if (--virtio_crypto_active_devs != 0)
> > +		goto unlock;
> > +
> > +	crypto_unregister_algs(virtio_crypto_algs,
> > +			ARRAY_SIZE(virtio_crypto_algs));
> > +
> > +unlock:
> > +	mutex_unlock(&algs_lock);
> > +}
> > diff --git a/drivers/crypto/virtio/virtio_crypto_common.h
> b/drivers/crypto/virtio/virtio_crypto_common.h
> > new file mode 100644
> > index 0000000..a599733
> > --- /dev/null
> > +++ b/drivers/crypto/virtio/virtio_crypto_common.h
> > @@ -0,0 +1,124 @@
> > +/* Common header for Virtio crypto device.
> > + *
> > + * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License as published by
> > + * the Free Software Foundation; either version 2 of the License, or
> > + * (at your option) any later version.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU General Public License
> > + * along with this program; if not, see <http://www.gnu.org/licenses/>.
> > + */
> > +
> > +#ifndef _VIRITO_CRYPTO_COMMON_H
> > +#define _VIRITO_CRYPTO_COMMON_H
> > +
> > +#include <linux/virtio.h>
> > +#include <linux/crypto.h>
> > +#include <linux/spinlock.h>
> > +#include <crypto/aead.h>
> > +#include <crypto/aes.h>
> > +#include <crypto/authenc.h>
> > +
> > +
> > +/* Internal representation of a data virtqueue */
> > +struct data_queue {
> > +	/* Virtqueue associated with this send _queue */
> > +	struct virtqueue *vq;
> > +
> > +	/* Name of the tx queue: dataq.$index */
> > +	char name[32];
> > +};
> > +
> > +struct virtio_crypto {
> > +	struct virtio_device *vdev;
> > +	struct virtqueue *ctrl_vq;
> > +	struct data_queue *data_vq;
> > +
> > +	/* To protect the vq operations for the dataq */
> > +	spinlock_t lock;
> > +
> > +	/* To protect the vq operations for the controlq */
> > +	spinlock_t ctrl_lock;
> > +
> > +	/* Maximum of data queues supported by the device */
> > +	u32 max_data_queues;
> > +
> > +	/* Number of queue currently used by the driver */
> > +	u32 curr_queue;
> > +
> > +	/* Maximum length of cipher key */
> > +	u32 max_cipher_key_len;
> > +	/* Maximum length of authenticated key */
> > +	u32 max_auth_key_len;
> > +	/* Maximum size of per request */
> > +	u64 max_size;
> > +
> > +	/* Control VQ buffers: protected by the ctrl_lock */
> > +	struct virtio_crypto_op_ctrl_req ctrl;
> > +	struct virtio_crypto_session_input input;
> > +	struct virtio_crypto_inhdr ctrl_status;
> > +
> > +	unsigned long status;
> > +	atomic_t ref_count;
> > +	struct list_head list;
> > +	struct module *owner;
> > +	uint8_t dev_id;
> > +
> > +	/* Does the affinity hint is set for virtqueues? */
> > +	bool affinity_hint_set;
> > +};
> > +
> > +struct virtio_crypto_sym_session_info {
> > +	/* Backend session id, which come from the host side */
> > +	__u64 session_id;
> > +};
> > +
> > +struct virtio_crypto_ablkcipher_ctx {
> > +	struct virtio_crypto *vcrypto;
> > +	struct crypto_tfm *tfm;
> > +
> > +	struct virtio_crypto_sym_session_info enc_sess_info;
> > +	struct virtio_crypto_sym_session_info dec_sess_info;
> > +
> > +	/* Protects virtio_crypto_ablkcipher_ctx struct */
> > +	spinlock_t lock;
> > +};
> > +
> > +struct virtio_crypto_request {
> > +	/* Cipher or aead */
> > +	uint32_t type;
> > +	uint8_t status;
> > +	struct virtio_crypto_ablkcipher_ctx *ablkcipher_ctx;
> > +	struct ablkcipher_request *ablkcipher_req;
> > +	struct virtio_crypto_op_data_req *req_data;
> > +	struct scatterlist **sgs;
> > +};
> > +
> > +int virtcrypto_devmgr_add_dev(struct virtio_crypto *vcrypto_dev);
> > +struct list_head *virtcrypto_devmgr_get_head(void);
> > +void virtcrypto_devmgr_rm_dev(struct virtio_crypto *vcrypto_dev);
> > +struct virtio_crypto *virtcrypto_devmgr_get_first(void);
> > +int virtcrypto_dev_in_use(struct virtio_crypto *vcrypto_dev);
> > +int virtcrypto_dev_get(struct virtio_crypto *vcrypto_dev);
> > +void virtcrypto_dev_put(struct virtio_crypto *vcrypto_dev);
> > +int virtcrypto_dev_started(struct virtio_crypto *vcrypto_dev);
> > +struct virtio_crypto *virtcrypto_get_dev_node(int node);
> > +int virtcrypto_dev_start(struct virtio_crypto *vcrypto);
> > +void virtcrypto_dev_stop(struct virtio_crypto *vcrypto);
> > +
> > +static inline int virtio_crypto_get_current_node(void)
> > +{
> > +	return topology_physical_package_id(smp_processor_id());
> > +}
> > +
> > +int virtio_crypto_algs_register(void);
> > +void virtio_crypto_algs_unregister(void);
> > +
> > +#endif /* _VIRITO_CRYPTO_COMMON_H */
> > diff --git a/drivers/crypto/virtio/virtio_crypto_mgr.c
> b/drivers/crypto/virtio/virtio_crypto_mgr.c
> > new file mode 100644
> > index 0000000..5b7260c
> > --- /dev/null
> > +++ b/drivers/crypto/virtio/virtio_crypto_mgr.c
> > @@ -0,0 +1,258 @@
> > + /* Management for virtio crypto devices (refer to adf_dev_mgr.c)
> > +  *
> > +  * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
> > +  *
> > +  * This program is free software; you can redistribute it and/or modify
> > +  * it under the terms of the GNU General Public License as published by
> > +  * the Free Software Foundation; either version 2 of the License, or
> > +  * (at your option) any later version.
> > +  *
> > +  * This program is distributed in the hope that it will be useful,
> > +  * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +  * GNU General Public License for more details.
> > +  *
> > +  * You should have received a copy of the GNU General Public License
> > +  * along with this program; if not, see <http://www.gnu.org/licenses/>.
> > +  */
> > +
> > +#include <linux/mutex.h>
> > +#include <linux/list.h>
> > +#include <linux/module.h>
> > +
> > +#include <uapi/linux/virtio_crypto.h>
> > +#include "virtio_crypto_common.h"
> > +
> > +static LIST_HEAD(virtio_crypto_table);
> > +static DEFINE_MUTEX(table_lock);
> > +static uint32_t num_devices;
> > +
> > +#define VIRTIO_CRYPTO_MAX_DEVICES 32
> > +
> > +
> > +/*
> > + * virtcrypto_devmgr_add_dev() - Add vcrypto_dev to the acceleration
> > + * framework.
> > + * @vcrypto_dev:  Pointer to virtio crypto device.
> > + *
> > + * Function adds virtio crypto device to the global list.
> > + * To be used by virtio crypto device specific drivers.
> > + *
> > + * Return: 0 on success, error code othewise.
> > + */
> > +int virtcrypto_devmgr_add_dev(struct virtio_crypto *vcrypto_dev)
> > +{
> > +	struct list_head *itr;
> > +
> > +	if (num_devices == VIRTIO_CRYPTO_MAX_DEVICES) {
> > +		pr_info("virtio_crypto: only support up to %d devices\n",
> > +			    VIRTIO_CRYPTO_MAX_DEVICES);
> > +		return -EFAULT;
> > +	}
> > +
> > +	mutex_lock(&table_lock);
> > +	list_for_each(itr, &virtio_crypto_table) {
> > +		struct virtio_crypto *ptr =
> > +				list_entry(itr, struct virtio_crypto, list);
> > +
> > +		if (ptr == vcrypto_dev) {
> > +			mutex_unlock(&table_lock);
> > +			return -EEXIST;
> > +		}
> > +	}
> > +	atomic_set(&vcrypto_dev->ref_count, 0);
> > +	list_add_tail(&vcrypto_dev->list, &virtio_crypto_table);
> > +	vcrypto_dev->dev_id = num_devices++;
> > +	mutex_unlock(&table_lock);
> > +	return 0;
> > +}
> > +
> > +struct list_head *virtcrypto_devmgr_get_head(void)
> > +{
> > +	return &virtio_crypto_table;
> > +}
> > +
> > +/*
> > + * virtcrypto_devmgr_rm_dev() - Remove vcrypto_dev from the acceleration
> > + * framework.
> > + * @vcrypto_dev:  Pointer to virtio crypto device.
> > + *
> > + * Function removes virtio crypto device from the acceleration framework.
> > + * To be used by virtio crypto device specific drivers.
> > + *
> > + * Return: void
> > + */
> > +void virtcrypto_devmgr_rm_dev(struct virtio_crypto *vcrypto_dev)
> > +{
> > +	mutex_lock(&table_lock);
> > +	list_del(&vcrypto_dev->list);
> > +	num_devices--;
> > +	mutex_unlock(&table_lock);
> > +}
> > +
> > +/*
> > + * virtcrypto_devmgr_get_first()
> > + *
> > + * Function returns the first virtio crypto device from the acceleration
> > + * framework.
> > + *
> > + * To be used by virtio crypto device specific drivers.
> > + *
> > + * Return: pointer to vcrypto_dev or NULL if not found.
> > + */
> > +struct virtio_crypto *virtcrypto_devmgr_get_first(void)
> > +{
> > +	struct virtio_crypto *dev = NULL;
> > +
> > +	if (!list_empty(&virtio_crypto_table))
> > +		dev = list_first_entry(&virtio_crypto_table,
> > +					struct virtio_crypto,
> > +				    list);
> > +	return dev;
> > +}
> > +
> > +/*
> > + * virtcrypto_dev_in_use() - Check whether vcrypto_dev is currently in use
> > + * @vcrypto_dev: Pointer to virtio crypto device.
> > + *
> > + * To be used by virtio crypto device specific drivers.
> > + *
> > + * Return: 1 when device is in use, 0 otherwise.
> > + */
> > +int virtcrypto_dev_in_use(struct virtio_crypto *vcrypto_dev)
> > +{
> > +	return atomic_read(&vcrypto_dev->ref_count) != 0;
> > +}
> > +
> > +/*
> > + * virtcrypto_dev_get() - Increment vcrypto_dev reference count
> > + * @vcrypto_dev: Pointer to virtio crypto device.
> > + *
> > + * Increment the vcrypto_dev refcount and if this is the first time
> > + * incrementing it during this period the vcrypto_dev is in use,
> > + * increment the module refcount too.
> > + * To be used by virtio crypto device specific drivers.
> > + *
> > + * Return: 0 when successful, EFAULT when fail to bump module refcount
> > + */
> > +int virtcrypto_dev_get(struct virtio_crypto *vcrypto_dev)
> > +{
> > +	if (atomic_add_return(1, &vcrypto_dev->ref_count) == 1)
> > +		if (!try_module_get(vcrypto_dev->owner))
> > +			return -EFAULT;
> > +	return 0;
> > +}
> > +
> > +/*
> > + * virtcrypto_dev_put() - Decrement vcrypto_dev reference count
> > + * @vcrypto_dev: Pointer to virtio crypto device.
> > + *
> > + * Decrement the vcrypto_dev refcount and if this is the last time
> > + * decrementing it during this period the vcrypto_dev is in use,
> > + * decrement the module refcount too.
> > + * To be used by virtio crypto device specific drivers.
> > + *
> > + * Return: void
> > + */
> > +void virtcrypto_dev_put(struct virtio_crypto *vcrypto_dev)
> > +{
> > +	if (atomic_sub_return(1, &vcrypto_dev->ref_count) == 0)
> > +		module_put(vcrypto_dev->owner);
> > +}
> > +
> > +/*
> > + * virtcrypto_dev_started() - Check whether device has started
> > + * @vcrypto_dev: Pointer to virtio crypto device.
> > + *
> > + * To be used by virtio crypto device specific drivers.
> > + *
> > + * Return: 1 when the device has started, 0 otherwise
> > + */
> > +int virtcrypto_dev_started(struct virtio_crypto *vcrypto_dev)
> > +{
> > +	return (vcrypto_dev->status & VIRTIO_CRYPTO_S_HW_READY);
> > +}
> > +
> > +/*
> > + * virtcrypto_get_dev_node() - Get vcrypto_dev on the node.
> > + * @node:  Node id the driver works.
> > + *
> > + * Function returns the virtio crypto device used fewest on the node.
> > + *
> > + * To be used by virtio crypto device specific drivers.
> > + *
> > + * Return: pointer to vcrypto_dev or NULL if not found.
> > + */
> > +struct virtio_crypto *virtcrypto_get_dev_node(int node)
> > +{
> > +	struct virtio_crypto *vcrypto_dev = NULL, *tmp_dev;
> > +	unsigned long best = ~0;
> > +	unsigned long ctr;
> > +
> > +	list_for_each_entry(tmp_dev, virtcrypto_devmgr_get_head(), list) {
> > +
> > +		if ((node == dev_to_node(&tmp_dev->vdev->dev) ||
> > +		     dev_to_node(&tmp_dev->vdev->dev) < 0) &&
> > +		    virtcrypto_dev_started(tmp_dev)) {
> > +			ctr = atomic_read(&tmp_dev->ref_count);
> > +			if (best > ctr) {
> > +				vcrypto_dev = tmp_dev;
> > +				best = ctr;
> > +			}
> > +		}
> > +	}
> > +
> > +	if (!vcrypto_dev) {
> > +		pr_info("virtio_crypto: Could not find a device on node %d\n",
> > +				node);
> > +		/* Get any started device */
> > +		list_for_each_entry(tmp_dev,
> > +				virtcrypto_devmgr_get_head(), list) {
> > +			if (virtcrypto_dev_started(tmp_dev)) {
> > +				vcrypto_dev = tmp_dev;
> > +				break;
> > +			}
> > +		}
> > +	}
> > +
> > +	if (!vcrypto_dev)
> > +		return NULL;
> > +
> > +	virtcrypto_dev_get(vcrypto_dev);
> > +	return vcrypto_dev;
> > +}
> > +
> > +/*
> > + * virtcrypto_dev_start() - Start virtio crypto device
> > + * @vcrypto:    Pointer to virtio crypto device.
> > + *
> > + * Function notifies all the registered services that the virtio crypto device
> > + * is ready to be used.
> > + * To be used by virtio crypto device specific drivers.
> > + *
> > + * Return: 0 on success, EFAULT when fail to register algorithms
> > + */
> > +int virtcrypto_dev_start(struct virtio_crypto *vcrypto)
> > +{
> > +	if (virtio_crypto_algs_register()) {
> > +		pr_err("virtio_crypto: Failed to register crypto algs\n");
> > +		return -EFAULT;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +/*
> > + * virtcrypto_dev_stop() - Stop virtio crypto device
> > + * @vcrypto:    Pointer to virtio crypto device.
> > + *
> > + * Function notifies all the registered services that the virtio crypto device
> > + * is ready to be used.
> > + * To be used by virtio crypto device specific drivers.
> > + *
> > + * Return: void
> > + */
> > +void virtcrypto_dev_stop(struct virtio_crypto *vcrypto)
> > +{
> > +	virtio_crypto_algs_unregister();
> > +}
> > diff --git a/include/uapi/linux/Kbuild b/include/uapi/linux/Kbuild
> > index cd2be1c..4bdb84c 100644
> > --- a/include/uapi/linux/Kbuild
> > +++ b/include/uapi/linux/Kbuild
> > @@ -460,6 +460,7 @@ header-y += virtio_rng.h
> >  header-y += virtio_scsi.h
> >  header-y += virtio_types.h
> >  header-y += virtio_vsock.h
> > +header-y += virtio_crypto.h
> >  header-y += vm_sockets.h
> >  header-y += vt.h
> >  header-y += vtpm_proxy.h
> > diff --git a/include/uapi/linux/virtio_crypto.h
> b/include/uapi/linux/virtio_crypto.h
> > new file mode 100644
> > index 0000000..abc2cf5
> > --- /dev/null
> > +++ b/include/uapi/linux/virtio_crypto.h
> > @@ -0,0 +1,435 @@
> > +#ifndef _VIRTIO_CRYPTO_H
> > +#define _VIRTIO_CRYPTO_H
> > +/* This header is BSD licensed so anyone can use the definitions to
> implement
> > + * compatible drivers/servers.
> > + *
> > + * Redistribution and use in source and binary forms, with or without
> > + * modification, are permitted provided that the following conditions
> > + * are met:
> > + * 1. Redistributions of source code must retain the above copyright
> > + *    notice, this list of conditions and the following disclaimer.
> > + * 2. Redistributions in binary form must reproduce the above copyright
> > + *    notice, this list of conditions and the following disclaimer in the
> > + *    documentation and/or other materials provided with the distribution.
> > + * 3. Neither the name of IBM nor the names of its contributors
> > + *    may be used to endorse or promote products derived from this
> software
> > + *    without specific prior written permission.
> > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> CONTRIBUTORS
> > + * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT
> NOT
> > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
> FITNESS
> > + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL
> IBM OR
> > + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> NOT
> > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
> OF
> > + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
> CAUSED AND
> > + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
> LIABILITY,
> > + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY
> WAY OUT
> > + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> > + * SUCH DAMAGE.
> > + */
> > +#include <linux/types.h>
> > +#include <linux/virtio_types.h>
> > +#include <linux/virtio_ids.h>
> > +#include <linux/virtio_config.h>
> > +
> > +
> > +#define VIRTIO_CRYPTO_SERVICE_CIPHER 0
> > +#define VIRTIO_CRYPTO_SERVICE_HASH   1
> > +#define VIRTIO_CRYPTO_SERVICE_MAC    2
> > +#define VIRTIO_CRYPTO_SERVICE_AEAD   3
> > +
> > +#define VIRTIO_CRYPTO_OPCODE(service, op)   (((service) << 8) | (op))
> > +
> > +struct virtio_crypto_ctrl_header {
> > +#define VIRTIO_CRYPTO_CIPHER_CREATE_SESSION \
> > +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_CIPHER, 0x02)
> > +#define VIRTIO_CRYPTO_CIPHER_DESTROY_SESSION \
> > +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_CIPHER, 0x03)
> > +#define VIRTIO_CRYPTO_HASH_CREATE_SESSION \
> > +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_HASH, 0x02)
> > +#define VIRTIO_CRYPTO_HASH_DESTROY_SESSION \
> > +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_HASH, 0x03)
> > +#define VIRTIO_CRYPTO_MAC_CREATE_SESSION \
> > +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_MAC, 0x02)
> > +#define VIRTIO_CRYPTO_MAC_DESTROY_SESSION \
> > +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_MAC, 0x03)
> > +#define VIRTIO_CRYPTO_AEAD_CREATE_SESSION \
> > +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x02)
> > +#define VIRTIO_CRYPTO_AEAD_DESTROY_SESSION \
> > +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x03)
> > +	__le32 opcode;
> > +	__le32 algo;
> > +	__le32 flag;
> > +	/* data virtqueue id */
> > +	__le32 queue_id;
> > +};
> > +
> > +struct virtio_crypto_cipher_session_para {
> > +#define VIRTIO_CRYPTO_NO_CIPHER                 0
> > +#define VIRTIO_CRYPTO_CIPHER_ARC4               1
> > +#define VIRTIO_CRYPTO_CIPHER_AES_ECB            2
> > +#define VIRTIO_CRYPTO_CIPHER_AES_CBC            3
> > +#define VIRTIO_CRYPTO_CIPHER_AES_CTR            4
> > +#define VIRTIO_CRYPTO_CIPHER_DES_ECB            5
> > +#define VIRTIO_CRYPTO_CIPHER_DES_CBC            6
> > +#define VIRTIO_CRYPTO_CIPHER_3DES_ECB           7
> > +#define VIRTIO_CRYPTO_CIPHER_3DES_CBC           8
> > +#define VIRTIO_CRYPTO_CIPHER_3DES_CTR           9
> > +#define VIRTIO_CRYPTO_CIPHER_KASUMI_F8          10
> > +#define VIRTIO_CRYPTO_CIPHER_SNOW3G_UEA2        11
> > +#define VIRTIO_CRYPTO_CIPHER_AES_F8             12
> > +#define VIRTIO_CRYPTO_CIPHER_AES_XTS            13
> > +#define VIRTIO_CRYPTO_CIPHER_ZUC_EEA3           14
> > +	__le32 algo;
> > +	/* length of key */
> > +	__le32 keylen;
> > +
> > +#define VIRTIO_CRYPTO_OP_ENCRYPT  1
> > +#define VIRTIO_CRYPTO_OP_DECRYPT  2
> > +	/* encrypt or decrypt */
> > +	__le32 op;
> > +	__le32 padding;
> > +};
> > +
> > +struct virtio_crypto_session_input {
> > +	/* Device-writable part */
> > +	__le64 session_id;
> > +	__le32 status;
> > +	__le32 padding;
> > +};
> > +
> > +struct virtio_crypto_cipher_session_req {
> > +	struct virtio_crypto_cipher_session_para para;
> > +};
> > +
> > +struct virtio_crypto_hash_session_para {
> > +#define VIRTIO_CRYPTO_NO_HASH            0
> > +#define VIRTIO_CRYPTO_HASH_MD5           1
> > +#define VIRTIO_CRYPTO_HASH_SHA1          2
> > +#define VIRTIO_CRYPTO_HASH_SHA_224       3
> > +#define VIRTIO_CRYPTO_HASH_SHA_256       4
> > +#define VIRTIO_CRYPTO_HASH_SHA_384       5
> > +#define VIRTIO_CRYPTO_HASH_SHA_512       6
> > +#define VIRTIO_CRYPTO_HASH_SHA3_224      7
> > +#define VIRTIO_CRYPTO_HASH_SHA3_256      8
> > +#define VIRTIO_CRYPTO_HASH_SHA3_384      9
> > +#define VIRTIO_CRYPTO_HASH_SHA3_512      10
> > +#define VIRTIO_CRYPTO_HASH_SHA3_SHAKE128      11
> > +#define VIRTIO_CRYPTO_HASH_SHA3_SHAKE256      12
> > +	__le32 algo;
> > +	/* hash result length */
> > +	__le32 hash_result_len;
> > +};
> > +
> > +struct virtio_crypto_hash_create_session_req {
> > +	struct virtio_crypto_hash_session_para para;
> > +};
> > +
> > +struct virtio_crypto_mac_session_para {
> > +#define VIRTIO_CRYPTO_NO_MAC                       0
> > +#define VIRTIO_CRYPTO_MAC_HMAC_MD5                 1
> > +#define VIRTIO_CRYPTO_MAC_HMAC_SHA1                2
> > +#define VIRTIO_CRYPTO_MAC_HMAC_SHA_224             3
> > +#define VIRTIO_CRYPTO_MAC_HMAC_SHA_256             4
> > +#define VIRTIO_CRYPTO_MAC_HMAC_SHA_384             5
> > +#define VIRTIO_CRYPTO_MAC_HMAC_SHA_512             6
> > +#define VIRTIO_CRYPTO_MAC_CMAC_3DES                25
> > +#define VIRTIO_CRYPTO_MAC_CMAC_AES                 26
> > +#define VIRTIO_CRYPTO_MAC_KASUMI_F9                27
> > +#define VIRTIO_CRYPTO_MAC_SNOW3G_UIA2              28
> > +#define VIRTIO_CRYPTO_MAC_GMAC_AES                 41
> > +#define VIRTIO_CRYPTO_MAC_GMAC_TWOFISH             42
> > +#define VIRTIO_CRYPTO_MAC_CBCMAC_AES               49
> > +#define VIRTIO_CRYPTO_MAC_CBCMAC_KASUMI_F9         50
> > +#define VIRTIO_CRYPTO_MAC_XCBC_AES                 53
> > +	__le32 algo;
> > +	/* hash result length */
> > +	__le32 hash_result_len;
> > +	/* length of authenticated key */
> > +	__le32 auth_key_len;
> > +	__le32 padding;
> > +};
> > +
> > +struct virtio_crypto_mac_create_session_req {
> > +	struct virtio_crypto_mac_session_para para;
> > +};
> > +
> > +struct virtio_crypto_aead_session_para {
> > +#define VIRTIO_CRYPTO_NO_AEAD     0
> > +#define VIRTIO_CRYPTO_AEAD_GCM    1
> > +#define VIRTIO_CRYPTO_AEAD_CCM    2
> > +#define VIRTIO_CRYPTO_AEAD_CHACHA20_POLY1305  3
> > +	__le32 algo;
> > +	/* length of key */
> > +	__le32 key_len;
> > +	/* hash result length */
> > +	__le32 hash_result_len;
> > +	/* length of the additional authenticated data (AAD) in bytes */
> > +	__le32 aad_len;
> > +	/* encrypt or decrypt, See above VIRTIO_CRYPTO_OP_* */
> > +	__le32 op;
> > +	__le32 padding;
> > +};
> > +
> > +struct virtio_crypto_aead_create_session_req {
> > +	struct virtio_crypto_aead_session_para para;
> > +};
> > +
> > +struct virtio_crypto_alg_chain_session_para {
> > +#define VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER
> 1
> > +#define VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH
> 2
> > +	__le32 alg_chain_order;
> > +/* Plain hash */
> > +#define VIRTIO_CRYPTO_SYM_HASH_MODE_PLAIN    1
> > +/* Authenticated hash (mac) */
> > +#define VIRTIO_CRYPTO_SYM_HASH_MODE_AUTH     2
> > +/* Nested hash */
> > +#define VIRTIO_CRYPTO_SYM_HASH_MODE_NESTED   3
> > +	__le32 hash_mode;
> > +	struct virtio_crypto_cipher_session_para cipher_param;
> > +	union {
> > +		struct virtio_crypto_hash_session_para hash_param;
> > +		struct virtio_crypto_mac_session_para mac_param;
> > +	} u;
> 
> I personally dislike the use of unions of different size
> in these structures. For example, using hash_param,
> there's no way to initialize the trailing part of the
> structure. Also, the size of the structure is hard to
> figure out. I would like to see padding of all structures
> to be same size, and adding a union member that's just padding.
> 
> Same applies to other unions.
> 
Good suggestion :)  Let me rework them.

Michael, I'd like to add virtio-crypto stuff to your maintaining part likes
the virtio-net/blk parts so that the corresponding patches
can be CC'ed to you too because the virtio-crypto doesn't lay in 
driver/virtio directory. Will you?

Thanks!
-Gonglei

^ permalink raw reply

* RE: [virtio-dev] Re: [PATCH v2 1/2] virtio: introduce little edian functions for virtio_cread/write# family
From: Gonglei (Arei) @ 2016-11-28  2:57 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: virtio-dev@lists.oasis-open.org, Huangweidong (C),
	Claudio Fontana, Hanweidong (Randy), qemu-devel@nongnu.org,
	Luonengjun, linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	salvatore.benedetto@intel.com, Xuquan (Quan Xu),
	linux-crypto@vger.kernel.org, stefanha@redhat.com,
	Zhoujian (jay, Euler), longpeng, arei.gonglei@hotmail.com,
	davem@davemloft.net, Wubin (H)
In-Reply-To: <20161128044845-mutt-send-email-mst@kernel.org>

>
> Subject: Re: [virtio-dev] Re: [PATCH v2 1/2] virtio: introduce little edian
> functions for virtio_cread/write# family
> 
> On Mon, Nov 28, 2016 at 04:34:56AM +0200, Michael S. Tsirkin wrote:
> > On Mon, Nov 28, 2016 at 01:56:04AM +0000, Gonglei (Arei) wrote:
> > > Hi Michael,
> > >
> > > Thanks for your feedback firstly!
> > >
> > > > -----Original Message-----
> > > > From: virtio-dev@lists.oasis-open.org
> [mailto:virtio-dev@lists.oasis-open.org]
> > > > On Behalf Of Michael S. Tsirkin
> > > > Sent: Sunday, November 27, 2016 11:33 AM
> > > > To: Gonglei (Arei)
> > > > Subject: [virtio-dev] Re: [PATCH v2 1/2] virtio: introduce little edian
> functions for
> > > > virtio_cread/write# family
> > > >
> > > > On Tue, Nov 22, 2016 at 04:10:22PM +0800, Gonglei wrote:
> > > > > Virtio modern devices are always little edian, let's introduce
> > > > > the LE functions for read/write configuration space for
> > > > > virtio modern devices, which avoid complaint by Sparse when
> > > > > we use the virtio_creaed/virtio_cwrite in VIRTIO_1 devices.
> > > > >
> > > > > Signed-off-by: Gonglei <arei.gonglei@huawei.com>
> > > > > ---
> > > > >  include/linux/virtio_config.h | 45
> > > > +++++++++++++++++++++++++++++++++++++++++++
> > > > >  1 file changed, 45 insertions(+)
> > > > >
> > > > > diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> > > > > index 26c155b..de05707 100644
> > > > > --- a/include/linux/virtio_config.h
> > > > > +++ b/include/linux/virtio_config.h
> > > > > @@ -414,4 +414,49 @@ static inline void virtio_cwrite64(struct
> virtio_device
> > > > *vdev,
> > > > >  		_r;							\
> > > > >  	})
> > > > >
> > > > > +static inline __le16 virtio_cread16_le(struct virtio_device *vdev,
> > > > > +				 unsigned int offset)
> > > > > +{
> > > > > +	__le16 ret;
> > > > > +
> > > > > +	vdev->config->get(vdev, offset, &ret, sizeof(ret));
> > > > > +	return ret;
> > > > > +}
> > > > > +
> > > > > +static inline void virtio_cwrite16_le(struct virtio_device *vdev,
> > > > > +				   unsigned int offset, __le16 val)
> > > > > +{
> > > > > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > > > > +}
> > > > > +
> > > > > +static inline __le32 virtio_cread32_le(struct virtio_device *vdev,
> > > > > +				 unsigned int offset)
> > > > > +{
> > > > > +	__le32 ret;
> > > > > +
> > > > > +	vdev->config->get(vdev, offset, &ret, sizeof(ret));
> > > > > +	return ret;
> > > > > +}
> > > > > +
> > > > > +static inline void virtio_cwrite32_le(struct virtio_device *vdev,
> > > > > +				   unsigned int offset, __le32 val)
> > > > > +{
> > > > > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > > > > +}
> > > > > +
> > > > > +static inline __le64 virtio_cread64_le(struct virtio_device *vdev,
> > > > > +				 unsigned int offset)
> > > > > +{
> > > > > +	__le64 ret;
> > > > > +
> > > > > +	__virtio_cread_many(vdev, offset, &ret, 1, sizeof(ret));
> > > > > +	return ret;
> > > > > +}
> > > > > +
> > > > > +static inline void virtio_cwrite64_le(struct virtio_device *vdev,
> > > > > +				   unsigned int offset, __le64 val)
> > > > > +{
> > > > > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > > > > +}
> > > > > +
> > > > >  #endif /* _LINUX_VIRTIO_CONFIG_H */
> > > >
> > > > Could you please better explain what is the issue you are facing?
> > > > virtio_cwrite/virtio_cread all accept and return native types.
> > > >
> > > virtio_cwrite/virtio_cread are used to write/read configuration
> > > space for virtio devices. For virtio-crypto device, I used __le32/64 directly
> > > in struct virtio_crypto_config. The sparse reports warnings if I use
> virtio_cread()
> > > for virtio-crypto device.
> >
> > I suspect that's because you are doing cread into an le32 variable.
> >
> >
> >
> > > Furthermore, it means the warnings exist for all VIRTIO_1 devices because
> > > they are definitely LE, which it's not necessary to use
> virtio_to_cpu/cpu_to_virtio.
> > >
> > >
> > > PS: I googled a discussion about this topic for virtio-input device, pls see:
> > >
> http://linux.kernel.narkive.com/3argfbWz/patch-1-1-add-virtio-input-driver
> > >
> > > Regards,
> > > -Gonglei
> >
> > Looks like we changed the macros since - at least ATM
> > virtio_console_config uses __u16 fields (which is probably a bug -
> > I'll look into fixing it up) and they
> > do not seem to trigger warnings.
> 
> 
> Sorry, I recall now. We use u32 and similar types in the config
> space, and endian-ness is implicit.
> This works fine if you are using virtio_cread interfaces which will swap
> to/from LE for you.
> 
> What doesn't work correctly is shadowing the whole config space
> using ->get since that does not byte-swap. so just don't do it.
> 
> I would say just do what everyone does in this version and
> we can revisit this later.
> 

OK, it's fair. :) 

Thanks!
-Gonglei

> >
> > > > If you want it in LE format, swap it!
> > > >
> > > >
> > > >
> > > > > --
> > > > > 1.8.3.1
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> > > > For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

^ permalink raw reply

* Re: [virtio-dev] Re: [PATCH v2 1/2] virtio: introduce little edian functions for virtio_cread/write# family
From: Michael S. Tsirkin @ 2016-11-28  2:54 UTC (permalink / raw)
  To: Gonglei (Arei)
  Cc: virtio-dev@lists.oasis-open.org, Huangweidong (C),
	Claudio Fontana, Hanweidong (Randy), qemu-devel@nongnu.org,
	Luonengjun, linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	salvatore.benedetto@intel.com, Xuquan (Quan Xu),
	linux-crypto@vger.kernel.org, stefanha@redhat.com,
	Zhoujian (jay, Euler), longpeng, arei.gonglei@hotmail.com,
	davem@davemloft.net, Wubin (H)
In-Reply-To: <20161128042142-mutt-send-email-mst@kernel.org>

On Mon, Nov 28, 2016 at 04:34:56AM +0200, Michael S. Tsirkin wrote:
> On Mon, Nov 28, 2016 at 01:56:04AM +0000, Gonglei (Arei) wrote:
> > Hi Michael,
> > 
> > Thanks for your feedback firstly!
> > 
> > > -----Original Message-----
> > > From: virtio-dev@lists.oasis-open.org [mailto:virtio-dev@lists.oasis-open.org]
> > > On Behalf Of Michael S. Tsirkin
> > > Sent: Sunday, November 27, 2016 11:33 AM
> > > To: Gonglei (Arei)
> > > Subject: [virtio-dev] Re: [PATCH v2 1/2] virtio: introduce little edian functions for
> > > virtio_cread/write# family
> > > 
> > > On Tue, Nov 22, 2016 at 04:10:22PM +0800, Gonglei wrote:
> > > > Virtio modern devices are always little edian, let's introduce
> > > > the LE functions for read/write configuration space for
> > > > virtio modern devices, which avoid complaint by Sparse when
> > > > we use the virtio_creaed/virtio_cwrite in VIRTIO_1 devices.
> > > >
> > > > Signed-off-by: Gonglei <arei.gonglei@huawei.com>
> > > > ---
> > > >  include/linux/virtio_config.h | 45
> > > +++++++++++++++++++++++++++++++++++++++++++
> > > >  1 file changed, 45 insertions(+)
> > > >
> > > > diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> > > > index 26c155b..de05707 100644
> > > > --- a/include/linux/virtio_config.h
> > > > +++ b/include/linux/virtio_config.h
> > > > @@ -414,4 +414,49 @@ static inline void virtio_cwrite64(struct virtio_device
> > > *vdev,
> > > >  		_r;							\
> > > >  	})
> > > >
> > > > +static inline __le16 virtio_cread16_le(struct virtio_device *vdev,
> > > > +				 unsigned int offset)
> > > > +{
> > > > +	__le16 ret;
> > > > +
> > > > +	vdev->config->get(vdev, offset, &ret, sizeof(ret));
> > > > +	return ret;
> > > > +}
> > > > +
> > > > +static inline void virtio_cwrite16_le(struct virtio_device *vdev,
> > > > +				   unsigned int offset, __le16 val)
> > > > +{
> > > > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > > > +}
> > > > +
> > > > +static inline __le32 virtio_cread32_le(struct virtio_device *vdev,
> > > > +				 unsigned int offset)
> > > > +{
> > > > +	__le32 ret;
> > > > +
> > > > +	vdev->config->get(vdev, offset, &ret, sizeof(ret));
> > > > +	return ret;
> > > > +}
> > > > +
> > > > +static inline void virtio_cwrite32_le(struct virtio_device *vdev,
> > > > +				   unsigned int offset, __le32 val)
> > > > +{
> > > > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > > > +}
> > > > +
> > > > +static inline __le64 virtio_cread64_le(struct virtio_device *vdev,
> > > > +				 unsigned int offset)
> > > > +{
> > > > +	__le64 ret;
> > > > +
> > > > +	__virtio_cread_many(vdev, offset, &ret, 1, sizeof(ret));
> > > > +	return ret;
> > > > +}
> > > > +
> > > > +static inline void virtio_cwrite64_le(struct virtio_device *vdev,
> > > > +				   unsigned int offset, __le64 val)
> > > > +{
> > > > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > > > +}
> > > > +
> > > >  #endif /* _LINUX_VIRTIO_CONFIG_H */
> > > 
> > > Could you please better explain what is the issue you are facing?
> > > virtio_cwrite/virtio_cread all accept and return native types.
> > > 
> > virtio_cwrite/virtio_cread are used to write/read configuration
> > space for virtio devices. For virtio-crypto device, I used __le32/64 directly
> > in struct virtio_crypto_config. The sparse reports warnings if I use virtio_cread()
> > for virtio-crypto device.
> 
> I suspect that's because you are doing cread into an le32 variable.
> 
> 
> 
> > Furthermore, it means the warnings exist for all VIRTIO_1 devices because
> > they are definitely LE, which it's not necessary to use virtio_to_cpu/cpu_to_virtio.
> > 
> > 
> > PS: I googled a discussion about this topic for virtio-input device, pls see:
> >  http://linux.kernel.narkive.com/3argfbWz/patch-1-1-add-virtio-input-driver
> > 
> > Regards,
> > -Gonglei
> 
> Looks like we changed the macros since - at least ATM
> virtio_console_config uses __u16 fields (which is probably a bug -
> I'll look into fixing it up) and they
> do not seem to trigger warnings.


Sorry, I recall now. We use u32 and similar types in the config
space, and endian-ness is implicit.
This works fine if you are using virtio_cread interfaces which will swap
to/from LE for you.

What doesn't work correctly is shadowing the whole config space
using ->get since that does not byte-swap. so just don't do it.

I would say just do what everyone does in this version and
we can revisit this later.

> 
> > > If you want it in LE format, swap it!
> > > 
> > > 
> > > 
> > > > --
> > > > 1.8.3.1
> > > >
> > > 
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> > > For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

^ permalink raw reply

* RE: [virtio-dev] Re: [PATCH v2 1/2] virtio: introduce little edian functions for virtio_cread/write# family
From: Gonglei (Arei) @ 2016-11-28  2:53 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: virtio-dev@lists.oasis-open.org, Huangweidong (C),
	Claudio Fontana, Hanweidong (Randy), qemu-devel@nongnu.org,
	Luonengjun, linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	salvatore.benedetto@intel.com, Xuquan (Quan Xu),
	linux-crypto@vger.kernel.org, stefanha@redhat.com,
	Zhoujian (jay, Euler), longpeng, arei.gonglei@hotmail.com,
	davem@davemloft.net, Wubin (H)
In-Reply-To: <20161128042142-mutt-send-email-mst@kernel.org>

>
> Subject: Re: [virtio-dev] Re: [PATCH v2 1/2] virtio: introduce little edian
> functions for virtio_cread/write# family
> 
> On Mon, Nov 28, 2016 at 01:56:04AM +0000, Gonglei (Arei) wrote:
> > Hi Michael,
> >
> > Thanks for your feedback firstly!
> >
> > > -----Original Message-----
> > > From: virtio-dev@lists.oasis-open.org
> [mailto:virtio-dev@lists.oasis-open.org]
> > > On Behalf Of Michael S. Tsirkin
> > > Sent: Sunday, November 27, 2016 11:33 AM
> > > To: Gonglei (Arei)
> > > Subject: [virtio-dev] Re: [PATCH v2 1/2] virtio: introduce little edian functions
> for
> > > virtio_cread/write# family
> > >
> > > On Tue, Nov 22, 2016 at 04:10:22PM +0800, Gonglei wrote:
> > > > Virtio modern devices are always little edian, let's introduce
> > > > the LE functions for read/write configuration space for
> > > > virtio modern devices, which avoid complaint by Sparse when
> > > > we use the virtio_creaed/virtio_cwrite in VIRTIO_1 devices.
> > > >
> > > > Signed-off-by: Gonglei <arei.gonglei@huawei.com>
> > > > ---
> > > >  include/linux/virtio_config.h | 45
> > > +++++++++++++++++++++++++++++++++++++++++++
> > > >  1 file changed, 45 insertions(+)
> > > >
> > > > diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> > > > index 26c155b..de05707 100644
> > > > --- a/include/linux/virtio_config.h
> > > > +++ b/include/linux/virtio_config.h
> > > > @@ -414,4 +414,49 @@ static inline void virtio_cwrite64(struct
> virtio_device
> > > *vdev,
> > > >  		_r;							\
> > > >  	})
> > > >
> > > > +static inline __le16 virtio_cread16_le(struct virtio_device *vdev,
> > > > +				 unsigned int offset)
> > > > +{
> > > > +	__le16 ret;
> > > > +
> > > > +	vdev->config->get(vdev, offset, &ret, sizeof(ret));
> > > > +	return ret;
> > > > +}
> > > > +
> > > > +static inline void virtio_cwrite16_le(struct virtio_device *vdev,
> > > > +				   unsigned int offset, __le16 val)
> > > > +{
> > > > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > > > +}
> > > > +
> > > > +static inline __le32 virtio_cread32_le(struct virtio_device *vdev,
> > > > +				 unsigned int offset)
> > > > +{
> > > > +	__le32 ret;
> > > > +
> > > > +	vdev->config->get(vdev, offset, &ret, sizeof(ret));
> > > > +	return ret;
> > > > +}
> > > > +
> > > > +static inline void virtio_cwrite32_le(struct virtio_device *vdev,
> > > > +				   unsigned int offset, __le32 val)
> > > > +{
> > > > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > > > +}
> > > > +
> > > > +static inline __le64 virtio_cread64_le(struct virtio_device *vdev,
> > > > +				 unsigned int offset)
> > > > +{
> > > > +	__le64 ret;
> > > > +
> > > > +	__virtio_cread_many(vdev, offset, &ret, 1, sizeof(ret));
> > > > +	return ret;
> > > > +}
> > > > +
> > > > +static inline void virtio_cwrite64_le(struct virtio_device *vdev,
> > > > +				   unsigned int offset, __le64 val)
> > > > +{
> > > > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > > > +}
> > > > +
> > > >  #endif /* _LINUX_VIRTIO_CONFIG_H */
> > >
> > > Could you please better explain what is the issue you are facing?
> > > virtio_cwrite/virtio_cread all accept and return native types.
> > >
> > virtio_cwrite/virtio_cread are used to write/read configuration
> > space for virtio devices. For virtio-crypto device, I used __le32/64 directly
> > in struct virtio_crypto_config. The sparse reports warnings if I use
> virtio_cread()
> > for virtio-crypto device.
> 
> I suspect that's because you are doing cread into an le32 variable.
> 
> 
Yes.

> 
> > Furthermore, it means the warnings exist for all VIRTIO_1 devices because
> > they are definitely LE, which it's not necessary to use
> virtio_to_cpu/cpu_to_virtio.
> >
> >
> > PS: I googled a discussion about this topic for virtio-input device, pls see:
> >  http://linux.kernel.narkive.com/3argfbWz/patch-1-1-add-virtio-input-driver
> >
> > Regards,
> > -Gonglei
> 
> Looks like we changed the macros since - at least ATM
> virtio_console_config uses __u16 fields (which is probably a bug -
> I'll look into fixing it up) and they
> do not seem to trigger warnings.
> 
Cool.

 __u16/32/64 are fine, but __le16/32/64 will trigger warnings.

Regards,
-Gonglei

> 
> > > If you want it in LE format, swap it!
> > >
> > >
> > >
> > > > --
> > > > 1.8.3.1
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> > > For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

^ permalink raw reply

* Re: [virtio-dev] Re: [PATCH v2 1/2] virtio: introduce little edian functions for virtio_cread/write# family
From: Michael S. Tsirkin @ 2016-11-28  2:34 UTC (permalink / raw)
  To: Gonglei (Arei)
  Cc: virtio-dev@lists.oasis-open.org, Huangweidong (C),
	Claudio Fontana, Hanweidong (Randy), qemu-devel@nongnu.org,
	Luonengjun, linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	salvatore.benedetto@intel.com, Xuquan (Quan Xu),
	linux-crypto@vger.kernel.org, stefanha@redhat.com,
	Zhoujian (jay, Euler), longpeng, arei.gonglei@hotmail.com,
	davem@davemloft.net, Wubin (H)
In-Reply-To: <33183CC9F5247A488A2544077AF19020D97A103C@SZXEMA503-MBS.china.huawei.com>

On Mon, Nov 28, 2016 at 01:56:04AM +0000, Gonglei (Arei) wrote:
> Hi Michael,
> 
> Thanks for your feedback firstly!
> 
> > -----Original Message-----
> > From: virtio-dev@lists.oasis-open.org [mailto:virtio-dev@lists.oasis-open.org]
> > On Behalf Of Michael S. Tsirkin
> > Sent: Sunday, November 27, 2016 11:33 AM
> > To: Gonglei (Arei)
> > Subject: [virtio-dev] Re: [PATCH v2 1/2] virtio: introduce little edian functions for
> > virtio_cread/write# family
> > 
> > On Tue, Nov 22, 2016 at 04:10:22PM +0800, Gonglei wrote:
> > > Virtio modern devices are always little edian, let's introduce
> > > the LE functions for read/write configuration space for
> > > virtio modern devices, which avoid complaint by Sparse when
> > > we use the virtio_creaed/virtio_cwrite in VIRTIO_1 devices.
> > >
> > > Signed-off-by: Gonglei <arei.gonglei@huawei.com>
> > > ---
> > >  include/linux/virtio_config.h | 45
> > +++++++++++++++++++++++++++++++++++++++++++
> > >  1 file changed, 45 insertions(+)
> > >
> > > diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> > > index 26c155b..de05707 100644
> > > --- a/include/linux/virtio_config.h
> > > +++ b/include/linux/virtio_config.h
> > > @@ -414,4 +414,49 @@ static inline void virtio_cwrite64(struct virtio_device
> > *vdev,
> > >  		_r;							\
> > >  	})
> > >
> > > +static inline __le16 virtio_cread16_le(struct virtio_device *vdev,
> > > +				 unsigned int offset)
> > > +{
> > > +	__le16 ret;
> > > +
> > > +	vdev->config->get(vdev, offset, &ret, sizeof(ret));
> > > +	return ret;
> > > +}
> > > +
> > > +static inline void virtio_cwrite16_le(struct virtio_device *vdev,
> > > +				   unsigned int offset, __le16 val)
> > > +{
> > > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > > +}
> > > +
> > > +static inline __le32 virtio_cread32_le(struct virtio_device *vdev,
> > > +				 unsigned int offset)
> > > +{
> > > +	__le32 ret;
> > > +
> > > +	vdev->config->get(vdev, offset, &ret, sizeof(ret));
> > > +	return ret;
> > > +}
> > > +
> > > +static inline void virtio_cwrite32_le(struct virtio_device *vdev,
> > > +				   unsigned int offset, __le32 val)
> > > +{
> > > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > > +}
> > > +
> > > +static inline __le64 virtio_cread64_le(struct virtio_device *vdev,
> > > +				 unsigned int offset)
> > > +{
> > > +	__le64 ret;
> > > +
> > > +	__virtio_cread_many(vdev, offset, &ret, 1, sizeof(ret));
> > > +	return ret;
> > > +}
> > > +
> > > +static inline void virtio_cwrite64_le(struct virtio_device *vdev,
> > > +				   unsigned int offset, __le64 val)
> > > +{
> > > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > > +}
> > > +
> > >  #endif /* _LINUX_VIRTIO_CONFIG_H */
> > 
> > Could you please better explain what is the issue you are facing?
> > virtio_cwrite/virtio_cread all accept and return native types.
> > 
> virtio_cwrite/virtio_cread are used to write/read configuration
> space for virtio devices. For virtio-crypto device, I used __le32/64 directly
> in struct virtio_crypto_config. The sparse reports warnings if I use virtio_cread()
> for virtio-crypto device.

I suspect that's because you are doing cread into an le32 variable.



> Furthermore, it means the warnings exist for all VIRTIO_1 devices because
> they are definitely LE, which it's not necessary to use virtio_to_cpu/cpu_to_virtio.
> 
> 
> PS: I googled a discussion about this topic for virtio-input device, pls see:
>  http://linux.kernel.narkive.com/3argfbWz/patch-1-1-add-virtio-input-driver
> 
> Regards,
> -Gonglei

Looks like we changed the macros since - at least ATM
virtio_console_config uses __u16 fields (which is probably a bug -
I'll look into fixing it up) and they
do not seem to trigger warnings.


> > If you want it in LE format, swap it!
> > 
> > 
> > 
> > > --
> > > 1.8.3.1
> > >
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> > For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

^ permalink raw reply

* RE: [virtio-dev] Re: [PATCH v2 1/2] virtio: introduce little edian functions for virtio_cread/write# family
From: Gonglei (Arei) @ 2016-11-28  1:56 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: virtio-dev@lists.oasis-open.org, Huangweidong (C),
	Claudio Fontana, Hanweidong (Randy), qemu-devel@nongnu.org,
	Luonengjun, linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	salvatore.benedetto@intel.com, Xuquan (Quan Xu),
	linux-crypto@vger.kernel.org, stefanha@redhat.com,
	Zhoujian (jay, Euler), longpeng, arei.gonglei@hotmail.com,
	davem@davemloft.net, Wubin (H)
In-Reply-To: <20161127052802-mutt-send-email-mst@kernel.org>

Hi Michael,

Thanks for your feedback firstly!

> -----Original Message-----
> From: virtio-dev@lists.oasis-open.org [mailto:virtio-dev@lists.oasis-open.org]
> On Behalf Of Michael S. Tsirkin
> Sent: Sunday, November 27, 2016 11:33 AM
> To: Gonglei (Arei)
> Subject: [virtio-dev] Re: [PATCH v2 1/2] virtio: introduce little edian functions for
> virtio_cread/write# family
> 
> On Tue, Nov 22, 2016 at 04:10:22PM +0800, Gonglei wrote:
> > Virtio modern devices are always little edian, let's introduce
> > the LE functions for read/write configuration space for
> > virtio modern devices, which avoid complaint by Sparse when
> > we use the virtio_creaed/virtio_cwrite in VIRTIO_1 devices.
> >
> > Signed-off-by: Gonglei <arei.gonglei@huawei.com>
> > ---
> >  include/linux/virtio_config.h | 45
> +++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 45 insertions(+)
> >
> > diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> > index 26c155b..de05707 100644
> > --- a/include/linux/virtio_config.h
> > +++ b/include/linux/virtio_config.h
> > @@ -414,4 +414,49 @@ static inline void virtio_cwrite64(struct virtio_device
> *vdev,
> >  		_r;							\
> >  	})
> >
> > +static inline __le16 virtio_cread16_le(struct virtio_device *vdev,
> > +				 unsigned int offset)
> > +{
> > +	__le16 ret;
> > +
> > +	vdev->config->get(vdev, offset, &ret, sizeof(ret));
> > +	return ret;
> > +}
> > +
> > +static inline void virtio_cwrite16_le(struct virtio_device *vdev,
> > +				   unsigned int offset, __le16 val)
> > +{
> > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > +}
> > +
> > +static inline __le32 virtio_cread32_le(struct virtio_device *vdev,
> > +				 unsigned int offset)
> > +{
> > +	__le32 ret;
> > +
> > +	vdev->config->get(vdev, offset, &ret, sizeof(ret));
> > +	return ret;
> > +}
> > +
> > +static inline void virtio_cwrite32_le(struct virtio_device *vdev,
> > +				   unsigned int offset, __le32 val)
> > +{
> > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > +}
> > +
> > +static inline __le64 virtio_cread64_le(struct virtio_device *vdev,
> > +				 unsigned int offset)
> > +{
> > +	__le64 ret;
> > +
> > +	__virtio_cread_many(vdev, offset, &ret, 1, sizeof(ret));
> > +	return ret;
> > +}
> > +
> > +static inline void virtio_cwrite64_le(struct virtio_device *vdev,
> > +				   unsigned int offset, __le64 val)
> > +{
> > +	vdev->config->set(vdev, offset, &val, sizeof(val));
> > +}
> > +
> >  #endif /* _LINUX_VIRTIO_CONFIG_H */
> 
> Could you please better explain what is the issue you are facing?
> virtio_cwrite/virtio_cread all accept and return native types.
> 
virtio_cwrite/virtio_cread are used to write/read configuration
space for virtio devices. For virtio-crypto device, I used __le32/64 directly
in struct virtio_crypto_config. The sparse reports warnings if I use virtio_cread()
for virtio-crypto device.

Furthermore, it means the warnings exist for all VIRTIO_1 devices because
they are definitely LE, which it's not necessary to use virtio_to_cpu/cpu_to_virtio.


PS: I googled a discussion about this topic for virtio-input device, pls see:
 http://linux.kernel.narkive.com/3argfbWz/patch-1-1-add-virtio-input-driver

Regards,
-Gonglei

> If you want it in LE format, swap it!
> 
> 
> 
> > --
> > 1.8.3.1
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

^ permalink raw reply

* Re: automatic IRQ affinity for virtio
From: Michael S. Tsirkin @ 2016-11-27  3:37 UTC (permalink / raw)
  To: Christoph Hellwig; +Cc: axboe, linux-block, linux-kernel, virtualization
In-Reply-To: <20161125072538.GA732@lst.de>

On Fri, Nov 25, 2016 at 08:25:38AM +0100, Christoph Hellwig wrote:
> Btw, what's the best way to get any response to this series?
> But this and the predecessor seem to have completly fallen on deaf
> ears.

I'm sorry, I intend to review soon and if OK merge it.

The fact that it depends on tip means I can't put
it in my first pull request (until tip is merged)
so I deferred review somewhat.

^ permalink raw reply

* Re: [PATCH v2 1/2] virtio: introduce little edian functions for virtio_cread/write# family
From: Michael S. Tsirkin @ 2016-11-27  3:32 UTC (permalink / raw)
  To: Gonglei
  Cc: virtio-dev, weidong.huang, claudio.fontana, hanweidong,
	qemu-devel, luonengjun, linux-kernel, virtualization,
	salvatore.benedetto, xuquan8, linux-crypto, stefanha,
	jianjay.zhou, longpeng2, arei.gonglei, davem, wu.wubin, herbert
In-Reply-To: <1479802223-121104-2-git-send-email-arei.gonglei@huawei.com>

On Tue, Nov 22, 2016 at 04:10:22PM +0800, Gonglei wrote:
> Virtio modern devices are always little edian, let's introduce
> the LE functions for read/write configuration space for
> virtio modern devices, which avoid complaint by Sparse when
> we use the virtio_creaed/virtio_cwrite in VIRTIO_1 devices.
> 
> Signed-off-by: Gonglei <arei.gonglei@huawei.com>
> ---
>  include/linux/virtio_config.h | 45 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 45 insertions(+)
> 
> diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> index 26c155b..de05707 100644
> --- a/include/linux/virtio_config.h
> +++ b/include/linux/virtio_config.h
> @@ -414,4 +414,49 @@ static inline void virtio_cwrite64(struct virtio_device *vdev,
>  		_r;							\
>  	})
>  
> +static inline __le16 virtio_cread16_le(struct virtio_device *vdev,
> +				 unsigned int offset)
> +{
> +	__le16 ret;
> +
> +	vdev->config->get(vdev, offset, &ret, sizeof(ret));
> +	return ret;
> +}
> +
> +static inline void virtio_cwrite16_le(struct virtio_device *vdev,
> +				   unsigned int offset, __le16 val)
> +{
> +	vdev->config->set(vdev, offset, &val, sizeof(val));
> +}
> +
> +static inline __le32 virtio_cread32_le(struct virtio_device *vdev,
> +				 unsigned int offset)
> +{
> +	__le32 ret;
> +
> +	vdev->config->get(vdev, offset, &ret, sizeof(ret));
> +	return ret;
> +}
> +
> +static inline void virtio_cwrite32_le(struct virtio_device *vdev,
> +				   unsigned int offset, __le32 val)
> +{
> +	vdev->config->set(vdev, offset, &val, sizeof(val));
> +}
> +
> +static inline __le64 virtio_cread64_le(struct virtio_device *vdev,
> +				 unsigned int offset)
> +{
> +	__le64 ret;
> +
> +	__virtio_cread_many(vdev, offset, &ret, 1, sizeof(ret));
> +	return ret;
> +}
> +
> +static inline void virtio_cwrite64_le(struct virtio_device *vdev,
> +				   unsigned int offset, __le64 val)
> +{
> +	vdev->config->set(vdev, offset, &val, sizeof(val));
> +}
> +
>  #endif /* _LINUX_VIRTIO_CONFIG_H */

Could you please better explain what is the issue you are facing?
virtio_cwrite/virtio_cread all accept and return native types.

If you want it in LE format, swap it!



> -- 
> 1.8.3.1
> 

^ permalink raw reply

* Re: [PATCH v2 2/2] crypto: add virtio-crypto driver
From: Michael S. Tsirkin @ 2016-11-27  3:27 UTC (permalink / raw)
  To: Gonglei
  Cc: virtio-dev, weidong.huang, claudio.fontana, hanweidong,
	qemu-devel, luonengjun, linux-kernel, virtualization,
	salvatore.benedetto, xuquan8, linux-crypto, stefanha,
	jianjay.zhou, longpeng2, arei.gonglei, davem, wu.wubin, herbert
In-Reply-To: <1479802223-121104-3-git-send-email-arei.gonglei@huawei.com>

On Tue, Nov 22, 2016 at 04:10:23PM +0800, Gonglei wrote:
> This patch introduces virtio-crypto driver for Linux Kernel.
> 
> The virtio crypto device is a virtual cryptography device
> as well as a kind of virtual hardware accelerator for
> virtual machines. The encryption anddecryption requests
> are placed in the data queue and are ultimately handled by
> thebackend crypto accelerators. The second queue is the
> control queue used to create or destroy sessions for
> symmetric algorithms and will control some advanced features
> in the future. The virtio crypto device provides the following
> cryptoservices: CIPHER, MAC, HASH, and AEAD.
> 
> For more information about virtio-crypto device, please see:
>   http://qemu-project.org/Features/VirtioCrypto
> 
> CC: Michael S. Tsirkin <mst@redhat.com>
> CC: Cornelia Huck <cornelia.huck@de.ibm.com>
> CC: Stefan Hajnoczi <stefanha@redhat.com>
> CC: Herbert Xu <herbert@gondor.apana.org.au>
> CC: Halil Pasic <pasic@linux.vnet.ibm.com>
> CC: David S. Miller <davem@davemloft.net>
> CC: Zeng Xin <xin.zeng@intel.com>
> Signed-off-by: Gonglei <arei.gonglei@huawei.com>
> ---
>  MAINTAINERS                                  |   8 +
>  drivers/crypto/Kconfig                       |   2 +
>  drivers/crypto/Makefile                      |   1 +
>  drivers/crypto/virtio/Kconfig                |  10 +
>  drivers/crypto/virtio/Makefile               |   5 +
>  drivers/crypto/virtio/virtio_crypto.c        | 444 +++++++++++++++++++++++
>  drivers/crypto/virtio/virtio_crypto_algs.c   | 524 +++++++++++++++++++++++++++
>  drivers/crypto/virtio/virtio_crypto_common.h | 124 +++++++
>  drivers/crypto/virtio/virtio_crypto_mgr.c    | 258 +++++++++++++
>  include/uapi/linux/Kbuild                    |   1 +
>  include/uapi/linux/virtio_crypto.h           | 435 ++++++++++++++++++++++
>  include/uapi/linux/virtio_ids.h              |   1 +
>  12 files changed, 1813 insertions(+)
>  create mode 100644 drivers/crypto/virtio/Kconfig
>  create mode 100644 drivers/crypto/virtio/Makefile
>  create mode 100644 drivers/crypto/virtio/virtio_crypto.c
>  create mode 100644 drivers/crypto/virtio/virtio_crypto_algs.c
>  create mode 100644 drivers/crypto/virtio/virtio_crypto_common.h
>  create mode 100644 drivers/crypto/virtio/virtio_crypto_mgr.c
>  create mode 100644 include/uapi/linux/virtio_crypto.h
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 411e3b8..d6b18bb 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -12844,6 +12844,14 @@ S:	Maintained
>  F:	drivers/virtio/virtio_input.c
>  F:	include/uapi/linux/virtio_input.h
>  
> +VIRTIO CRYPTO DRIVER
> +M:  Gonglei <arei.gonglei@huawei.com>
> +L:  virtualization@lists.linux-foundation.org
> +L:  linux-crypto@vger.kernel.org
> +S:  Maintained
> +F:  drivers/crypto/virtio/
> +F:  include/uapi/linux/virtio_crypto.h
> +
>  VIA RHINE NETWORK DRIVER
>  S:	Orphan
>  F:	drivers/net/ethernet/via/via-rhine.c
> diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
> index 4d2b81f..7956478 100644
> --- a/drivers/crypto/Kconfig
> +++ b/drivers/crypto/Kconfig
> @@ -555,4 +555,6 @@ config CRYPTO_DEV_ROCKCHIP
>  
>  source "drivers/crypto/chelsio/Kconfig"
>  
> +source "drivers/crypto/virtio/Kconfig"
> +
>  endif # CRYPTO_HW
> diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile
> index ad7250f..bc53cb8 100644
> --- a/drivers/crypto/Makefile
> +++ b/drivers/crypto/Makefile
> @@ -32,3 +32,4 @@ obj-$(CONFIG_CRYPTO_DEV_VMX) += vmx/
>  obj-$(CONFIG_CRYPTO_DEV_SUN4I_SS) += sunxi-ss/
>  obj-$(CONFIG_CRYPTO_DEV_ROCKCHIP) += rockchip/
>  obj-$(CONFIG_CRYPTO_DEV_CHELSIO) += chelsio/
> +obj-$(CONFIG_CRYPTO_DEV_VIRTIO) += virtio/
> diff --git a/drivers/crypto/virtio/Kconfig b/drivers/crypto/virtio/Kconfig
> new file mode 100644
> index 0000000..ceae88c
> --- /dev/null
> +++ b/drivers/crypto/virtio/Kconfig
> @@ -0,0 +1,10 @@
> +config CRYPTO_DEV_VIRTIO
> +	tristate "VirtIO crypto driver"
> +	depends on VIRTIO
> +    select CRYPTO_AEAD
> +    select CRYPTO_AUTHENC
> +    select CRYPTO_BLKCIPHER
> +	default m
> +	help
> +	  This driver provides support for virtio crypto device. If you
> +	  choose 'M' here, this module will be called virtio-crypto.
> diff --git a/drivers/crypto/virtio/Makefile b/drivers/crypto/virtio/Makefile
> new file mode 100644
> index 0000000..a316e92
> --- /dev/null
> +++ b/drivers/crypto/virtio/Makefile
> @@ -0,0 +1,5 @@
> +obj-$(CONFIG_CRYPTO_DEV_VIRTIO) += virtio-crypto.o
> +virtio-crypto-objs := \
> +	virtio_crypto_algs.o \
> +	virtio_crypto_mgr.o \
> +	virtio_crypto.o
> diff --git a/drivers/crypto/virtio/virtio_crypto.c b/drivers/crypto/virtio/virtio_crypto.c
> new file mode 100644
> index 0000000..56fdfed
> --- /dev/null
> +++ b/drivers/crypto/virtio/virtio_crypto.c
> @@ -0,0 +1,444 @@
> + /* Driver for Virtio crypto device.
> +  *
> +  * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
> +  *
> +  * This program is free software; you can redistribute it and/or modify
> +  * it under the terms of the GNU General Public License as published by
> +  * the Free Software Foundation; either version 2 of the License, or
> +  * (at your option) any later version.
> +  *
> +  * This program is distributed in the hope that it will be useful,
> +  * but WITHOUT ANY WARRANTY; without even the implied warranty of
> +  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +  * GNU General Public License for more details.
> +  *
> +  * You should have received a copy of the GNU General Public License
> +  * along with this program; if not, see <http://www.gnu.org/licenses/>.
> +  */
> +
> +#include <linux/err.h>
> +#include <linux/module.h>
> +#include <linux/virtio_config.h>
> +#include <linux/cpu.h>
> +
> +#include <uapi/linux/virtio_crypto.h>
> +#include "virtio_crypto_common.h"
> +
> +
> +static void virtcrypto_dataq_callback(struct virtqueue *vq)
> +{
> +	struct virtio_crypto *vcrypto = vq->vdev->priv;
> +	struct virtio_crypto_request *vc_req;
> +	unsigned long flags;
> +	unsigned int len;
> +	struct ablkcipher_request *ablk_req;
> +	int error;
> +
> +	spin_lock_irqsave(&vcrypto->lock, flags);
> +	do {
> +		virtqueue_disable_cb(vq);
> +		while ((vc_req = virtqueue_get_buf(vq, &len)) != NULL) {
> +			if (vc_req->type == VIRTIO_CRYPTO_SYM_OP_CIPHER) {
> +				switch (vc_req->status) {
> +				case VIRTIO_CRYPTO_OK:
> +					error = 0;
> +					break;
> +				case VIRTIO_CRYPTO_INVSESS:
> +				case VIRTIO_CRYPTO_ERR:
> +					error = -EINVAL;
> +					break;
> +				case VIRTIO_CRYPTO_BADMSG:
> +					error = -EBADMSG;
> +					break;
> +				default:
> +					error = -EIO;
> +					break;
> +				}
> +				ablk_req = vc_req->ablkcipher_req;
> +				/* Finish the encrypt or decrypt process */
> +				ablk_req->base.complete(&ablk_req->base, error);
> +			}
> +
> +			kfree(vc_req->req_data);
> +			kfree(vc_req->sgs);
> +		}
> +	} while (!virtqueue_enable_cb(vq));
> +	spin_unlock_irqrestore(&vcrypto->lock, flags);
> +}
> +
> +static int virtcrypto_find_vqs(struct virtio_crypto *vi)
> +{
> +	vq_callback_t **callbacks;
> +	struct virtqueue **vqs;
> +	int ret = -ENOMEM;
> +	int i, total_vqs;
> +	const char **names;
> +
> +	/* We expect 1 data virtqueue, followed by
> +	 * possible N-1 data queues used in multiqueue mode, followed by
> +	 * control vq.
> +	 */
> +	total_vqs = vi->max_data_queues + 1;
> +
> +	/* Allocate space for find_vqs parameters */
> +	vqs = kcalloc(total_vqs, sizeof(*vqs), GFP_KERNEL);
> +	if (!vqs)
> +		goto err_vq;
> +	callbacks = kcalloc(total_vqs, sizeof(*callbacks), GFP_KERNEL);
> +	if (!callbacks)
> +		goto err_callback;
> +	names = kcalloc(total_vqs, sizeof(*names), GFP_KERNEL);
> +	if (!names)
> +		goto err_names;
> +
> +	/* Parameters for control virtqueue */
> +	callbacks[total_vqs - 1] = NULL;
> +	names[total_vqs - 1] = "controlq";
> +
> +	/* Allocate/initialize parameters for data virtqueues */
> +	for (i = 0; i < vi->max_data_queues; i++) {
> +		callbacks[i] = virtcrypto_dataq_callback;
> +		snprintf(vi->data_vq[i].name, sizeof(vi->data_vq[i].name),
> +				"dataq.%d", i);
> +		names[i] = vi->data_vq[i].name;
> +	}
> +
> +	ret = vi->vdev->config->find_vqs(vi->vdev, total_vqs, vqs, callbacks,
> +					 names);
> +	if (ret)
> +		goto err_find;
> +
> +	vi->ctrl_vq = vqs[total_vqs - 1];
> +
> +	for (i = 0; i < vi->max_data_queues; i++)
> +		vi->data_vq[i].vq = vqs[i];
> +
> +	kfree(names);
> +	kfree(callbacks);
> +	kfree(vqs);
> +
> +	return 0;
> +
> +err_find:
> +	kfree(names);
> +err_names:
> +	kfree(callbacks);
> +err_callback:
> +	kfree(vqs);
> +err_vq:
> +	return ret;
> +}
> +
> +static int virtcrypto_alloc_queues(struct virtio_crypto *vi)
> +{
> +	vi->data_vq = kcalloc(vi->max_data_queues, sizeof(*vi->data_vq),
> +				GFP_KERNEL);
> +	if (!vi->data_vq)
> +		return -ENOMEM;
> +
> +	return 0;
> +}
> +
> +static void virtcrypto_clean_affinity(struct virtio_crypto *vi, long hcpu)
> +{
> +	int i;
> +
> +	if (vi->affinity_hint_set) {
> +		for (i = 0; i < vi->max_data_queues; i++)
> +			virtqueue_set_affinity(vi->data_vq[i].vq, -1);
> +
> +		vi->affinity_hint_set = false;
> +	}
> +}
> +
> +static void virtcrypto_set_affinity(struct virtio_crypto *vi)
> +{
> +	int i;
> +	int cpu;
> +
> +	/*
> +	 * In multiqueue mode, when the number of cpu is equal to the number
> +	 * of queue, we let the queue to be private to one cpu by
> +	 * setting the affinity hint to eliminate the contention.
> +	 */
> +	if (vi->curr_queue == 1 ||
> +	    vi->max_data_queues != num_online_cpus()) {
> +		virtcrypto_clean_affinity(vi, -1);
> +		return;
> +	}
> +

So how can you handle cpu hotplug then?
I would imagine making max_data_queues > num_online_cpus
and initializing just the required number would be
a saner way.

> +	i = 0;
> +	for_each_online_cpu(cpu) {
> +		virtqueue_set_affinity(vi->data_vq[i].vq, cpu);
> +		i++;
> +	}
> +
> +	vi->affinity_hint_set = true;
> +}
> +

Above is racy in prsence of cpu hotplug. Is there a userspace
interface to fix up affinity after hotplug?
If not you need to get notified after hotplug and
fix it up yourself.

> +static void virtcrypto_free_queues(struct virtio_crypto *vi)
> +{
> +	kfree(vi->data_vq);
> +}
> +
> +static int virtcrypto_init_vqs(struct virtio_crypto *vi)
> +{
> +	int ret;
> +
> +	/* Allocate send & receive queues */
> +	ret = virtcrypto_alloc_queues(vi);
> +	if (ret)
> +		goto err;
> +
> +	ret = virtcrypto_find_vqs(vi);
> +	if (ret)
> +		goto err_free;
> +
> +	get_online_cpus();
> +	virtcrypto_set_affinity(vi);
> +	put_online_cpus();
> +
> +	return 0;
> +
> +err_free:
> +	virtcrypto_free_queues(vi);
> +err:
> +	return ret;
> +}
> +
> +static int virtcrypto_update_status(struct virtio_crypto *vcrypto)
> +{
> +	__le32 status_le;
> +	u32 status;
> +	int err;
> +
> +	status_le = virtio_cread32_le(vcrypto->vdev,
> +			offsetof(struct virtio_crypto_config, status));
> +	status = le32_to_cpu(status_le);
> +
> +	/* Ignore unknown (future) status bits */
> +	status &= VIRTIO_CRYPTO_S_HW_READY;
> +
> +	if (vcrypto->status == status)
> +		return 0;
> +
> +	vcrypto->status = status;
> +
> +	if (vcrypto->status & VIRTIO_CRYPTO_S_HW_READY) {
> +		err = virtcrypto_dev_start(vcrypto);
> +		if (err) {
> +			dev_err(&vcrypto->vdev->dev,
> +				"Failed to start virtio crypto device.\n");
> +			virtcrypto_dev_stop(vcrypto);
> +			return -1;
> +		}
> +		dev_info(&vcrypto->vdev->dev, "Accelerator is ready\n");
> +	} else {
> +		virtcrypto_dev_stop(vcrypto);
> +		dev_info(&vcrypto->vdev->dev, "Accelerator is not ready\n");
> +	}
> +
> +	return 0;
> +}
> +
> +static void virtcrypto_del_vqs(struct virtio_crypto *vcrypto)
> +{
> +	struct virtio_device *vdev = vcrypto->vdev;
> +
> +	virtcrypto_clean_affinity(vcrypto, -1);
> +
> +	vdev->config->del_vqs(vdev);
> +
> +	virtcrypto_free_queues(vcrypto);
> +}
> +
> +static int virtcrypto_probe(struct virtio_device *vdev)
> +{
> +	int err = -EFAULT;
> +	struct virtio_crypto *vcrypto;
> +	__le32 max_data_queues_le = 0, max_cipher_key_len_le = 0;
> +	__le32 max_auth_key_len_le = 0;
> +	__le64 max_size_le = 0;
> +

Pls validate that it's a modern device (has VERSION_1 set).
We should find a way to do it in a central place,
but we don't have that now.

> +	if (!vdev->config->get) {
> +		dev_err(&vdev->dev, "%s failure: config access disabled\n",
> +			__func__);
> +		return -EINVAL;
> +	}
> +
> +	if (num_possible_nodes() > 1 && dev_to_node(&vdev->dev) < 0) {
> +		/*
> +		 * If the accelerator is connected to a node with no memory
> +		 * there is no point in using the accelerator since the remote
> +		 * memory transaction will be very slow.
> +		 */
> +		dev_err(&vdev->dev, "Invalid NUMA configuration.\n");
> +		return -EINVAL;
> +	}
> +
> +	vcrypto = kzalloc_node(sizeof(*vcrypto), GFP_KERNEL,
> +					dev_to_node(&vdev->dev));
> +	if (!vcrypto)
> +		return -ENOMEM;
> +
> +	max_data_queues_le = virtio_cread32_le(vdev,
> +			offsetof(struct virtio_crypto_config, max_dataqueues));
> +	vcrypto->max_data_queues = le32_to_cpu(max_data_queues_le);
> +	if (vcrypto->max_data_queues < 1)
> +		vcrypto->max_data_queues = 1;
> +
> +	dev_info(&vdev->dev, "max_queues: %u\n", vcrypto->max_data_queues);
> +
> +	max_cipher_key_len_le = virtio_cread32_le(vdev,
> +		offsetof(struct virtio_crypto_config, max_cipher_key_len));
> +	max_auth_key_len_le = virtio_cread32_le(vdev,
> +		offsetof(struct virtio_crypto_config, max_auth_key_len));
> +	max_size_le = virtio_cread64_le(vdev,
> +		offsetof(struct virtio_crypto_config, max_size));
> +
> +	/* Add virtio crypto device to global table */
> +	err = virtcrypto_devmgr_add_dev(vcrypto);
> +	if (err) {
> +		dev_err(&vdev->dev, "Failed to add new virtio crypto device.\n");
> +		goto free;
> +	}
> +	vcrypto->owner = THIS_MODULE;
> +	vcrypto = vdev->priv = vcrypto;
> +	vcrypto->vdev = vdev;
> +	spin_lock_init(&vcrypto->lock);
> +	spin_lock_init(&vcrypto->ctrl_lock);
> +
> +	/* Use sigle data queue as default */
> +	vcrypto->curr_queue = 1;
> +	vcrypto->max_cipher_key_len = le32_to_cpu(max_cipher_key_len_le);
> +	vcrypto->max_auth_key_len = le32_to_cpu(max_auth_key_len_le);
> +	vcrypto->max_size = le64_to_cpu(max_size_le);
> +
> +	err = virtcrypto_init_vqs(vcrypto);
> +	if (err) {
> +		dev_err(&vdev->dev, "Failed to initialize vqs.\n");
> +		goto free_dev;
> +	}
> +	virtio_device_ready(vdev);
> +
> +	err = virtcrypto_update_status(vcrypto);
> +	if (err) {
> +		err = -EFAULT;

Why EFAULT?

> +		goto free_vqs;
> +	}
> +
> +	return 0;
> +
> +free_vqs:
> +	virtcrypto_del_vqs(vcrypto);

as you called virtio_device_ready this is likely unsafe
unless you first reset device or make sure there are
no interrupts in some other way.

> +free_dev:
> +	virtcrypto_devmgr_rm_dev(vcrypto);
> +free:
> +	kfree(vcrypto);
> +	return err;
> +}
> +
> +static void virtcrypto_free_unused_reqs(struct virtio_crypto *vcrypto)
> +{
> +	struct virtio_crypto_request *vc_req;
> +	int i;
> +	struct virtqueue *vq;
> +
> +	for (i = 0; i < vcrypto->max_data_queues; i++) {
> +		vq = vcrypto->data_vq[i].vq;
> +		while ((vc_req = virtqueue_detach_unused_buf(vq)) != NULL) {
> +			kfree(vc_req->req_data);
> +			kfree(vc_req->sgs);
> +		}
> +	}
> +}
> +
> +static void virtcrypto_remove(struct virtio_device *vdev)
> +{
> +	struct virtio_crypto *vcrypto = vdev->priv;
> +
> +	dev_info(&vdev->dev, "Start virtcrypto_remove.\n");
> +
> +	if (virtcrypto_dev_started(vcrypto))
> +		virtcrypto_dev_stop(vcrypto);
> +	vdev->config->reset(vdev);
> +	virtcrypto_free_unused_reqs(vcrypto);
> +	virtcrypto_del_vqs(vcrypto);
> +	virtcrypto_devmgr_rm_dev(vcrypto);
> +	kfree(vcrypto);
> +}
> +
> +static void virtcrypto_config_changed(struct virtio_device *vdev)
> +{
> +	struct virtio_crypto *vcrypto = vdev->priv;
> +
> +	virtcrypto_update_status(vcrypto);
> +}
> +
> +#ifdef CONFIG_PM_SLEEP
> +static int virtcrypto_freeze(struct virtio_device *vdev)
> +{
> +	struct virtio_crypto *vcrypto = vdev->priv;
> +	unsigned long flags;
> +
> +	virtcrypto_free_unused_reqs(vcrypto);
> +	spin_lock_irqsave(&vcrypto->lock, flags);
> +	if (virtcrypto_dev_started(vcrypto))
> +		virtcrypto_dev_stop(vcrypto);

This is taking mutex locks under a spinlock.
Highly unlikely to work. Try this with lockdep
enabled.


> +	spin_unlock_irqrestore(&vcrypto->lock, flags);
> +
> +	virtcrypto_del_vqs(vcrypto);
> +	return 0;
> +}
> +
> +static int virtcrypto_restore(struct virtio_device *vdev)
> +{
> +	struct virtio_crypto *vcrypto = vdev->priv;
> +	int err;
> +
> +	err = virtcrypto_init_vqs(vcrypto);
> +	if (err)
> +		return err;
> +
> +	virtio_device_ready(vdev);
> +	err = virtcrypto_dev_start(vcrypto);
> +	if (err) {
> +		dev_err(&vdev->dev, "Failed to start virtio crypto device.\n");
> +		return -EFAULT;
> +	}
> +
> +	return 0;
> +}
> +#endif
> +
> +
> +static unsigned int features[] = {
> +	/* none */
> +};
> +
> +static struct virtio_device_id id_table[] = {
> +	{ VIRTIO_ID_CRYPTO, VIRTIO_DEV_ANY_ID },
> +	{ 0 },
> +};
> +
> +static struct virtio_driver virtio_crypto_driver = {
> +	.driver.name         = KBUILD_MODNAME,
> +	.driver.owner        = THIS_MODULE,
> +	.feature_table       = features,
> +	.feature_table_size  = ARRAY_SIZE(features),
> +	.id_table            = id_table,
> +	.probe               = virtcrypto_probe,
> +	.remove              = virtcrypto_remove,
> +	.config_changed = virtcrypto_config_changed,
> +#ifdef CONFIG_PM_SLEEP
> +	.freeze = virtcrypto_freeze,
> +	.restore = virtcrypto_restore,
> +#endif
> +};
> +
> +module_virtio_driver(virtio_crypto_driver);
> +
> +MODULE_DEVICE_TABLE(virtio, id_table);
> +MODULE_DESCRIPTION("virtio crypto device driver");
> +MODULE_LICENSE("GPL");
> +MODULE_AUTHOR("Gonglei <arei.gonglei@huawei.com>");
> diff --git a/drivers/crypto/virtio/virtio_crypto_algs.c b/drivers/crypto/virtio/virtio_crypto_algs.c
> new file mode 100644
> index 0000000..7b57584
> --- /dev/null
> +++ b/drivers/crypto/virtio/virtio_crypto_algs.c
> @@ -0,0 +1,524 @@
> + /* Algorithms supported by virtio crypto device
> +  *
> +  * Authors: Gonglei <arei.gonglei@huawei.com>
> +  *
> +  * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
> +  *
> +  * This program is free software; you can redistribute it and/or modify
> +  * it under the terms of the GNU General Public License as published by
> +  * the Free Software Foundation; either version 2 of the License, or
> +  * (at your option) any later version.
> +  *
> +  * This program is distributed in the hope that it will be useful,
> +  * but WITHOUT ANY WARRANTY; without even the implied warranty of
> +  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +  * GNU General Public License for more details.
> +  *
> +  * You should have received a copy of the GNU General Public License
> +  * along with this program; if not, see <http://www.gnu.org/licenses/>.
> +  */
> +
> +#include <linux/scatterlist.h>
> +#include <crypto/algapi.h>
> +#include <linux/err.h>
> +#include <crypto/scatterwalk.h>
> +#include <linux/atomic.h>
> +
> +#include <uapi/linux/virtio_crypto.h>
> +#include "virtio_crypto_common.h"
> +
> +static DEFINE_MUTEX(algs_lock);
> +static unsigned int virtio_crypto_active_devs;
> +
> +static u64 virtio_crypto_alg_sg_nents_length(struct scatterlist *sg)
> +{
> +	u64 total = 0;
> +
> +	for (total = 0; sg; sg = sg_next(sg))
> +		total += sg->length;
> +
> +	return total;
> +}
> +
> +static int virtio_crypto_alg_validate_key(int key_len, int *alg)
> +{
> +	switch (key_len) {
> +	case AES_KEYSIZE_128:
> +	case AES_KEYSIZE_192:
> +	case AES_KEYSIZE_256:
> +		*alg = VIRTIO_CRYPTO_CIPHER_AES_CBC;
> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +	return 0;
> +}
> +
> +static int virtio_crypto_alg_ablkcipher_init_session(
> +		struct virtio_crypto_ablkcipher_ctx *ctx,
> +		int alg, const uint8_t *key,
> +		unsigned int keylen,
> +		int encrypt)
> +{
> +	struct scatterlist outhdr, key_sg, inhdr, *sgs[3];
> +	unsigned int tmp;
> +	struct virtio_crypto *vcrypto = ctx->vcrypto;
> +	int op = encrypt ? VIRTIO_CRYPTO_OP_ENCRYPT : VIRTIO_CRYPTO_OP_DECRYPT;
> +	int err;
> +	unsigned int num_out = 0, num_in = 0;
> +
> +	/*
> +	 * Avoid to do DMA from the stack, switch to using
> +	 * dynamically-allocated for the key
> +	 */
> +	uint8_t *cipher_key = kmalloc(keylen, GFP_ATOMIC);

Is all crypto using GFP_ATOMIC like this?
Is there code that recovers if this fails?


> +
> +	if (!cipher_key)
> +		return -ENOMEM;
> +
> +	memcpy(cipher_key, key, keylen);
> +
> +	spin_lock(&vcrypto->ctrl_lock);
> +	/* Pad ctrl header */
> +	vcrypto->ctrl.header.opcode =
> +		cpu_to_le32(VIRTIO_CRYPTO_CIPHER_CREATE_SESSION);
> +	vcrypto->ctrl.header.algo = cpu_to_le32((uint32_t)alg);

why cast here?

> +	/* Set the default dataqueue id to 0 */
> +	vcrypto->ctrl.header.queue_id = 0;
> +
> +	vcrypto->input.status = cpu_to_le32(VIRTIO_CRYPTO_ERR);
> +	/* Pad cipher's parameters */
> +	vcrypto->ctrl.u.sym_create_session.op_type =
> +		cpu_to_le32(VIRTIO_CRYPTO_SYM_OP_CIPHER);
> +	vcrypto->ctrl.u.sym_create_session.u.cipher.para.algo =
> +		vcrypto->ctrl.header.algo;
> +	vcrypto->ctrl.u.sym_create_session.u.cipher.para.keylen =
> +		cpu_to_le32(keylen);
> +	vcrypto->ctrl.u.sym_create_session.u.cipher.para.op =
> +		cpu_to_le32(op);
> +
> +	sg_init_one(&outhdr, &vcrypto->ctrl, sizeof(vcrypto->ctrl));
> +	sgs[num_out++] = &outhdr;
> +
> +	/* Set key */
> +	sg_init_one(&key_sg, cipher_key, keylen);
> +	sgs[num_out++] = &key_sg;
> +
> +	/* Return status and session id back */
> +	sg_init_one(&inhdr, &vcrypto->input, sizeof(vcrypto->input));
> +	sgs[num_out + num_in++] = &inhdr;
> +
> +	err = virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out,
> +				num_in, vcrypto, GFP_ATOMIC);
> +	if (err < 0) {
> +		spin_unlock(&vcrypto->ctrl_lock);
> +		kfree(cipher_key);
> +		return err;
> +	}
> +	virtqueue_kick(vcrypto->ctrl_vq);
> +
> +	/*
> +	 * Spin for a response, the kick causes an ioport write, trapping
> +	 * into the hypervisor, so the request should be handled immediately.
> +	 */
> +	while (!virtqueue_get_buf(vcrypto->ctrl_vq, &tmp) &&
> +	       !virtqueue_is_broken(vcrypto->ctrl_vq))
> +		cpu_relax();
> +
> +	if (le32_to_cpu(vcrypto->input.status) != VIRTIO_CRYPTO_OK) {
> +		spin_unlock(&vcrypto->ctrl_lock);
> +		pr_err("virtio_crypto: Create session failed status: %u\n",
> +			le32_to_cpu(vcrypto->input.status));
> +		kfree(cipher_key);
> +		return -EINVAL;
> +	}
> +	spin_unlock(&vcrypto->ctrl_lock);
> +
> +	spin_lock(&ctx->lock);
> +	if (encrypt)
> +		ctx->enc_sess_info.session_id =
> +			le64_to_cpu(vcrypto->input.session_id);
> +	else
> +		ctx->dec_sess_info.session_id =
> +			le64_to_cpu(vcrypto->input.session_id);
> +	spin_unlock(&ctx->lock);
> +
> +	kfree(cipher_key);
> +	return 0;
> +}
> +
> +static int virtio_crypto_alg_ablkcipher_close_session(
> +		struct virtio_crypto_ablkcipher_ctx *ctx,
> +		int encrypt)
> +{
> +	struct scatterlist outhdr, status_sg, *sgs[2];
> +	unsigned int tmp;
> +	struct virtio_crypto_destroy_session_req *destroy_session;
> +	struct virtio_crypto *vcrypto = ctx->vcrypto;
> +	int err;
> +	unsigned int num_out = 0, num_in = 0;
> +
> +	spin_lock(&vcrypto->ctrl_lock);
> +	vcrypto->ctrl_status.status = VIRTIO_CRYPTO_ERR;
> +	/* Pad ctrl header */
> +	vcrypto->ctrl.header.opcode =
> +		cpu_to_le32(VIRTIO_CRYPTO_CIPHER_DESTROY_SESSION);
> +	/* Set the default virtqueue id to 0 */
> +	vcrypto->ctrl.header.queue_id = 0;
> +
> +	destroy_session = &vcrypto->ctrl.u.destroy_session;
> +
> +	if (encrypt)
> +		destroy_session->session_id =
> +			cpu_to_le64(ctx->enc_sess_info.session_id);
> +	else
> +		destroy_session->session_id =
> +			cpu_to_le64(ctx->dec_sess_info.session_id);
> +
> +	sg_init_one(&outhdr, &vcrypto->ctrl, sizeof(vcrypto->ctrl));
> +	sgs[num_out++] = &outhdr;
> +
> +	/* Return status and session id back */
> +	sg_init_one(&status_sg, &vcrypto->ctrl_status.status,
> +		sizeof(vcrypto->ctrl_status.status));
> +	sgs[num_out + num_in++] = &status_sg;
> +
> +	err = virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out,
> +			num_in, vcrypto, GFP_ATOMIC);
> +	if (err < 0) {
> +		spin_unlock(&vcrypto->ctrl_lock);
> +		return err;
> +	}
> +	virtqueue_kick(vcrypto->ctrl_vq);
> +
> +	while (!virtqueue_get_buf(vcrypto->ctrl_vq, &tmp) &&
> +	       !virtqueue_is_broken(vcrypto->ctrl_vq))
> +		cpu_relax();
> +
> +	if (vcrypto->ctrl_status.status != VIRTIO_CRYPTO_OK) {
> +		spin_unlock(&vcrypto->ctrl_lock);
> +		pr_err("virtio_crypto: Close session failed status: %u, session_id: 0x%llx\n",
> +			vcrypto->ctrl_status.status,
> +			destroy_session->session_id);
> +
> +		return -EINVAL;
> +	}
> +	spin_unlock(&vcrypto->ctrl_lock);
> +
> +	return 0;
> +}
> +
> +static int virtio_crypto_alg_ablkcipher_init_sessions(
> +		struct virtio_crypto_ablkcipher_ctx *ctx,
> +		const uint8_t *key, unsigned int keylen)
> +{
> +	int alg;
> +	int ret;
> +	struct virtio_crypto *vcrypto = ctx->vcrypto;
> +
> +	if (keylen > vcrypto->max_cipher_key_len) {
> +		pr_err("virtio_crypto: the key is too long\n");
> +		goto bad_key;
> +	}
> +
> +	if (virtio_crypto_alg_validate_key(keylen, &alg))
> +		goto bad_key;
> +
> +	/* Create encryption session */
> +	ret = virtio_crypto_alg_ablkcipher_init_session(ctx,
> +			alg, key, keylen, 1);
> +	if (ret)
> +		return ret;
> +	/* Create decryption session */
> +	ret = virtio_crypto_alg_ablkcipher_init_session(ctx,
> +			alg, key, keylen, 0);
> +	if (ret) {
> +		virtio_crypto_alg_ablkcipher_close_session(ctx, 1);
> +		return ret;
> +	}
> +	return 0;
> +
> +bad_key:
> +	crypto_tfm_set_flags(ctx->tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
> +	return -EINVAL;
> +}
> +
> +/* Note: kernel crypto API realization */
> +static int virtio_crypto_ablkcipher_setkey(struct crypto_ablkcipher *tfm,
> +					 const uint8_t *key,
> +					 unsigned int keylen)
> +{
> +	struct virtio_crypto_ablkcipher_ctx *ctx = crypto_ablkcipher_ctx(tfm);
> +	int ret;
> +
> +	spin_lock(&ctx->lock);
> +
> +	if (!ctx->vcrypto) {
> +		/* New key */
> +		int node = virtio_crypto_get_current_node();
> +		struct virtio_crypto *vcrypto =
> +				      virtcrypto_get_dev_node(node);
> +		if (!vcrypto) {
> +			vcrypto = virtcrypto_devmgr_get_first();
> +			if (!vcrypto) {
> +				pr_err("virtio_crypto: Could not find a virtio device in the system");
> +				spin_unlock(&ctx->lock);
> +				return -ENODEV;
> +			}
> +		}
> +
> +		ctx->vcrypto = vcrypto;
> +	}
> +	spin_unlock(&ctx->lock);
> +
> +	ret = virtio_crypto_alg_ablkcipher_init_sessions(ctx, key, keylen);
> +	if (ret) {
> +		virtcrypto_dev_put(ctx->vcrypto);
> +		ctx->vcrypto = NULL;
> +
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int
> +__virtio_crypto_ablkcipher_do_req(struct virtio_crypto_request *vc_req,
> +		struct ablkcipher_request *req,
> +		struct data_queue *data_vq,
> +		__u8 op)
> +{
> +	struct virtio_crypto_ablkcipher_ctx *ctx = vc_req->ablkcipher_ctx;
> +	struct virtio_crypto *vcrypto = ctx->vcrypto;
> +	struct virtio_crypto_op_data_req *req_data;
> +	int src_nents, dst_nents;
> +	int err;
> +	unsigned long flags;
> +	struct scatterlist outhdr, iv_sg, status_sg, **sgs;
> +	int i;
> +	u64 dst_len;
> +	unsigned int num_out = 0, num_in = 0;
> +	int sg_total;
> +
> +	src_nents = sg_nents_for_len(req->src, req->nbytes);
> +	dst_nents = sg_nents(req->dst);
> +
> +	pr_debug("virtio_crypto: Number of sgs (src_nents: %d, dst_nents: %d)\n",
> +			src_nents, dst_nents);
> +
> +	/* Why 3?  outhdr + iv + inhdr */
> +	sg_total = src_nents + dst_nents + 3;
> +	sgs = kzalloc_node(sg_total * sizeof(*sgs), GFP_ATOMIC,
> +				dev_to_node(&vcrypto->vdev->dev));
> +	if (!sgs)
> +		return -ENOMEM;
> +
> +	req_data = kzalloc_node(sizeof(*req_data), GFP_ATOMIC,
> +				dev_to_node(&vcrypto->vdev->dev));
> +	if (!req_data) {
> +		kfree(sgs);
> +		return -ENOMEM;
> +	}
> +
> +	vc_req->req_data = req_data;
> +	vc_req->type = VIRTIO_CRYPTO_SYM_OP_CIPHER;
> +	/* Head of operation */
> +	if (op) {
> +		req_data->header.session_id =
> +			cpu_to_le64(ctx->enc_sess_info.session_id);
> +		req_data->header.opcode =
> +			cpu_to_le32(VIRTIO_CRYPTO_CIPHER_ENCRYPT);
> +	} else {
> +		req_data->header.session_id =
> +			cpu_to_le64(ctx->dec_sess_info.session_id);
> +	    req_data->header.opcode =
> +			cpu_to_le32(VIRTIO_CRYPTO_CIPHER_DECRYPT);
> +	}
> +	req_data->u.sym_req.op_type = cpu_to_le32(VIRTIO_CRYPTO_SYM_OP_CIPHER);
> +	req_data->u.sym_req.u.cipher.para.iv_len = cpu_to_le32(AES_BLOCK_SIZE);
> +	req_data->u.sym_req.u.cipher.para.src_data_len =
> +			cpu_to_le32(req->nbytes);
> +
> +	dst_len = virtio_crypto_alg_sg_nents_length(req->dst);
> +	if (unlikely(dst_len > U32_MAX)) {
> +		pr_err("virtio_crypto: The dst_len is beyond U32_MAX\n");
> +		err = -EINVAL;
> +		goto free;
> +	}
> +
> +	pr_debug("virtio_crypto: src_len: %u, dst_len: %llu\n",
> +			req->nbytes, dst_len);
> +
> +	if (unlikely(req->nbytes + dst_len + AES_BLOCK_SIZE +
> +		sizeof(vc_req->status) > vcrypto->max_size)) {
> +		pr_err("virtio_crypto: The length is too big\n");
> +		err = -EINVAL;
> +		goto free;
> +	}
> +
> +	req_data->u.sym_req.u.cipher.para.dst_data_len =
> +			cpu_to_le32((uint32_t)dst_len);
> +
> +	/* Outhdr */
> +	sg_init_one(&outhdr, req_data, sizeof(*req_data));
> +	sgs[num_out++] = &outhdr;
> +
> +	/* IV */
> +	sg_init_one(&iv_sg, req->info, AES_BLOCK_SIZE);
> +	sgs[num_out++] = &iv_sg;
> +
> +	/* Source data */
> +	for (i = 0; i < src_nents; i++)
> +		sgs[num_out++] = &req->src[i];
> +
> +	/* Destination data */
> +	for (i = 0; i < dst_nents; i++)
> +		sgs[num_out + num_in++] = &req->dst[i];
> +
> +	/* Status */
> +	sg_init_one(&status_sg, &vc_req->status, sizeof(vc_req->status));
> +	sgs[num_out + num_in++] = &status_sg;
> +
> +	vc_req->sgs = sgs;
> +
> +	spin_lock_irqsave(&vcrypto->lock, flags);
> +	err = virtqueue_add_sgs(data_vq->vq, sgs, num_out,
> +				num_in, vc_req, GFP_ATOMIC);
> +	spin_unlock_irqrestore(&vcrypto->lock, flags);
> +	if (unlikely(err < 0))
> +		goto free;
> +
> +	return 0;
> +
> +free:
> +	kfree(req_data);
> +	kfree(sgs);
> +	return err;
> +}
> +
> +static int virtio_crypto_ablkcipher_encrypt(struct ablkcipher_request *req)
> +{
> +	struct crypto_ablkcipher *atfm = crypto_ablkcipher_reqtfm(req);
> +	struct virtio_crypto_ablkcipher_ctx *ctx = crypto_ablkcipher_ctx(atfm);
> +	struct virtio_crypto_request *vc_req = ablkcipher_request_ctx(req);
> +	struct virtio_crypto *vcrypto = ctx->vcrypto;
> +	int ret;
> +	/* Use the first data virtqueue as default */
> +	struct data_queue *data_vq = &vcrypto->data_vq[0];
> +
> +	vc_req->ablkcipher_ctx = ctx;
> +	vc_req->ablkcipher_req = req;
> +	ret = __virtio_crypto_ablkcipher_do_req(vc_req, req, data_vq, 1);
> +	if (ret < 0) {
> +		pr_err("virtio_crypto: Encryption failed!\n");
> +		return ret;
> +	}
> +	virtqueue_kick(data_vq->vq);
> +
> +	return -EINPROGRESS;
> +}
> +
> +static int virtio_crypto_ablkcipher_decrypt(struct ablkcipher_request *req)
> +{
> +	struct crypto_ablkcipher *atfm = crypto_ablkcipher_reqtfm(req);
> +	struct virtio_crypto_ablkcipher_ctx *ctx = crypto_ablkcipher_ctx(atfm);
> +	struct virtio_crypto_request *vc_req = ablkcipher_request_ctx(req);
> +	struct virtio_crypto *vcrypto = ctx->vcrypto;
> +	int ret;
> +	/* Use the first data virtqueue as default */
> +	struct data_queue *data_vq = &vcrypto->data_vq[0];
> +
> +	vc_req->ablkcipher_ctx = ctx;
> +	vc_req->ablkcipher_req = req;
> +
> +	ret = __virtio_crypto_ablkcipher_do_req(vc_req, req, data_vq, 0);
> +	if (ret < 0) {
> +		pr_err("virtio_crypto: Decryption failed!\n");
> +		return ret;
> +	}
> +	virtqueue_kick(data_vq->vq);
> +
> +	return -EINPROGRESS;
> +}
> +
> +static int virtio_crypto_ablkcipher_init(struct crypto_tfm *tfm)
> +{
> +	struct virtio_crypto_ablkcipher_ctx *ctx = crypto_tfm_ctx(tfm);
> +
> +	spin_lock_init(&ctx->lock);
> +	tfm->crt_ablkcipher.reqsize = sizeof(struct virtio_crypto_request);
> +	ctx->tfm = tfm;
> +
> +	return 0;
> +}
> +
> +static void virtio_crypto_ablkcipher_exit(struct crypto_tfm *tfm)
> +{
> +	struct virtio_crypto_ablkcipher_ctx *ctx = crypto_tfm_ctx(tfm);
> +
> +	if (!ctx->vcrypto)
> +		return;
> +
> +	virtio_crypto_alg_ablkcipher_close_session(ctx, 1);
> +	virtio_crypto_alg_ablkcipher_close_session(ctx, 0);
> +	virtcrypto_dev_put(ctx->vcrypto);
> +	ctx->vcrypto = NULL;
> +}
> +
> +static struct crypto_alg virtio_crypto_algs[] = { {
> +	.cra_name = "cbc(aes)",
> +	.cra_driver_name = "virtio_crypto_aes_cbc",
> +	.cra_priority = 4001,
> +	.cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER | CRYPTO_ALG_ASYNC,
> +	.cra_blocksize = AES_BLOCK_SIZE,
> +	.cra_ctxsize  = sizeof(struct virtio_crypto_ablkcipher_ctx),
> +	.cra_alignmask = 0,
> +	.cra_module = THIS_MODULE,
> +	.cra_type = &crypto_ablkcipher_type,
> +	.cra_init = virtio_crypto_ablkcipher_init,
> +	.cra_exit = virtio_crypto_ablkcipher_exit,
> +	.cra_u = {
> +	   .ablkcipher = {
> +			.setkey = virtio_crypto_ablkcipher_setkey,
> +			.decrypt = virtio_crypto_ablkcipher_decrypt,
> +			.encrypt = virtio_crypto_ablkcipher_encrypt,
> +			.min_keysize = AES_MIN_KEY_SIZE,
> +			.max_keysize = AES_MAX_KEY_SIZE,
> +			.ivsize = AES_BLOCK_SIZE,
> +		},
> +	},
> +} };
> +
> +int virtio_crypto_algs_register(void)
> +{
> +	int ret = 0, i;
> +
> +	mutex_lock(&algs_lock);
> +	if (++virtio_crypto_active_devs != 1)
> +		goto unlock;
> +
> +	for (i = 0; i < ARRAY_SIZE(virtio_crypto_algs); i++) {
> +		virtio_crypto_algs[i].cra_flags =
> +			     CRYPTO_ALG_TYPE_ABLKCIPHER | CRYPTO_ALG_ASYNC;
> +	}
> +
> +	ret = crypto_register_algs(virtio_crypto_algs,
> +			ARRAY_SIZE(virtio_crypto_algs));
> +
> +unlock:
> +	mutex_unlock(&algs_lock);
> +	return ret;
> +}
> +
> +void virtio_crypto_algs_unregister(void)
> +{
> +	mutex_lock(&algs_lock);
> +	if (--virtio_crypto_active_devs != 0)
> +		goto unlock;
> +
> +	crypto_unregister_algs(virtio_crypto_algs,
> +			ARRAY_SIZE(virtio_crypto_algs));
> +
> +unlock:
> +	mutex_unlock(&algs_lock);
> +}
> diff --git a/drivers/crypto/virtio/virtio_crypto_common.h b/drivers/crypto/virtio/virtio_crypto_common.h
> new file mode 100644
> index 0000000..a599733
> --- /dev/null
> +++ b/drivers/crypto/virtio/virtio_crypto_common.h
> @@ -0,0 +1,124 @@
> +/* Common header for Virtio crypto device.
> + *
> + * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program; if not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#ifndef _VIRITO_CRYPTO_COMMON_H
> +#define _VIRITO_CRYPTO_COMMON_H
> +
> +#include <linux/virtio.h>
> +#include <linux/crypto.h>
> +#include <linux/spinlock.h>
> +#include <crypto/aead.h>
> +#include <crypto/aes.h>
> +#include <crypto/authenc.h>
> +
> +
> +/* Internal representation of a data virtqueue */
> +struct data_queue {
> +	/* Virtqueue associated with this send _queue */
> +	struct virtqueue *vq;
> +
> +	/* Name of the tx queue: dataq.$index */
> +	char name[32];
> +};
> +
> +struct virtio_crypto {
> +	struct virtio_device *vdev;
> +	struct virtqueue *ctrl_vq;
> +	struct data_queue *data_vq;
> +
> +	/* To protect the vq operations for the dataq */
> +	spinlock_t lock;
> +
> +	/* To protect the vq operations for the controlq */
> +	spinlock_t ctrl_lock;
> +
> +	/* Maximum of data queues supported by the device */
> +	u32 max_data_queues;
> +
> +	/* Number of queue currently used by the driver */
> +	u32 curr_queue;
> +
> +	/* Maximum length of cipher key */
> +	u32 max_cipher_key_len;
> +	/* Maximum length of authenticated key */
> +	u32 max_auth_key_len;
> +	/* Maximum size of per request */
> +	u64 max_size;
> +
> +	/* Control VQ buffers: protected by the ctrl_lock */
> +	struct virtio_crypto_op_ctrl_req ctrl;
> +	struct virtio_crypto_session_input input;
> +	struct virtio_crypto_inhdr ctrl_status;
> +
> +	unsigned long status;
> +	atomic_t ref_count;
> +	struct list_head list;
> +	struct module *owner;
> +	uint8_t dev_id;
> +
> +	/* Does the affinity hint is set for virtqueues? */
> +	bool affinity_hint_set;
> +};
> +
> +struct virtio_crypto_sym_session_info {
> +	/* Backend session id, which come from the host side */
> +	__u64 session_id;
> +};
> +
> +struct virtio_crypto_ablkcipher_ctx {
> +	struct virtio_crypto *vcrypto;
> +	struct crypto_tfm *tfm;
> +
> +	struct virtio_crypto_sym_session_info enc_sess_info;
> +	struct virtio_crypto_sym_session_info dec_sess_info;
> +
> +	/* Protects virtio_crypto_ablkcipher_ctx struct */
> +	spinlock_t lock;
> +};
> +
> +struct virtio_crypto_request {
> +	/* Cipher or aead */
> +	uint32_t type;
> +	uint8_t status;
> +	struct virtio_crypto_ablkcipher_ctx *ablkcipher_ctx;
> +	struct ablkcipher_request *ablkcipher_req;
> +	struct virtio_crypto_op_data_req *req_data;
> +	struct scatterlist **sgs;
> +};
> +
> +int virtcrypto_devmgr_add_dev(struct virtio_crypto *vcrypto_dev);
> +struct list_head *virtcrypto_devmgr_get_head(void);
> +void virtcrypto_devmgr_rm_dev(struct virtio_crypto *vcrypto_dev);
> +struct virtio_crypto *virtcrypto_devmgr_get_first(void);
> +int virtcrypto_dev_in_use(struct virtio_crypto *vcrypto_dev);
> +int virtcrypto_dev_get(struct virtio_crypto *vcrypto_dev);
> +void virtcrypto_dev_put(struct virtio_crypto *vcrypto_dev);
> +int virtcrypto_dev_started(struct virtio_crypto *vcrypto_dev);
> +struct virtio_crypto *virtcrypto_get_dev_node(int node);
> +int virtcrypto_dev_start(struct virtio_crypto *vcrypto);
> +void virtcrypto_dev_stop(struct virtio_crypto *vcrypto);
> +
> +static inline int virtio_crypto_get_current_node(void)
> +{
> +	return topology_physical_package_id(smp_processor_id());
> +}
> +
> +int virtio_crypto_algs_register(void);
> +void virtio_crypto_algs_unregister(void);
> +
> +#endif /* _VIRITO_CRYPTO_COMMON_H */
> diff --git a/drivers/crypto/virtio/virtio_crypto_mgr.c b/drivers/crypto/virtio/virtio_crypto_mgr.c
> new file mode 100644
> index 0000000..5b7260c
> --- /dev/null
> +++ b/drivers/crypto/virtio/virtio_crypto_mgr.c
> @@ -0,0 +1,258 @@
> + /* Management for virtio crypto devices (refer to adf_dev_mgr.c)
> +  *
> +  * Copyright 2016 HUAWEI TECHNOLOGIES CO., LTD.
> +  *
> +  * This program is free software; you can redistribute it and/or modify
> +  * it under the terms of the GNU General Public License as published by
> +  * the Free Software Foundation; either version 2 of the License, or
> +  * (at your option) any later version.
> +  *
> +  * This program is distributed in the hope that it will be useful,
> +  * but WITHOUT ANY WARRANTY; without even the implied warranty of
> +  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +  * GNU General Public License for more details.
> +  *
> +  * You should have received a copy of the GNU General Public License
> +  * along with this program; if not, see <http://www.gnu.org/licenses/>.
> +  */
> +
> +#include <linux/mutex.h>
> +#include <linux/list.h>
> +#include <linux/module.h>
> +
> +#include <uapi/linux/virtio_crypto.h>
> +#include "virtio_crypto_common.h"
> +
> +static LIST_HEAD(virtio_crypto_table);
> +static DEFINE_MUTEX(table_lock);
> +static uint32_t num_devices;
> +
> +#define VIRTIO_CRYPTO_MAX_DEVICES 32
> +
> +
> +/*
> + * virtcrypto_devmgr_add_dev() - Add vcrypto_dev to the acceleration
> + * framework.
> + * @vcrypto_dev:  Pointer to virtio crypto device.
> + *
> + * Function adds virtio crypto device to the global list.
> + * To be used by virtio crypto device specific drivers.
> + *
> + * Return: 0 on success, error code othewise.
> + */
> +int virtcrypto_devmgr_add_dev(struct virtio_crypto *vcrypto_dev)
> +{
> +	struct list_head *itr;
> +
> +	if (num_devices == VIRTIO_CRYPTO_MAX_DEVICES) {
> +		pr_info("virtio_crypto: only support up to %d devices\n",
> +			    VIRTIO_CRYPTO_MAX_DEVICES);
> +		return -EFAULT;
> +	}
> +
> +	mutex_lock(&table_lock);
> +	list_for_each(itr, &virtio_crypto_table) {
> +		struct virtio_crypto *ptr =
> +				list_entry(itr, struct virtio_crypto, list);
> +
> +		if (ptr == vcrypto_dev) {
> +			mutex_unlock(&table_lock);
> +			return -EEXIST;
> +		}
> +	}
> +	atomic_set(&vcrypto_dev->ref_count, 0);
> +	list_add_tail(&vcrypto_dev->list, &virtio_crypto_table);
> +	vcrypto_dev->dev_id = num_devices++;
> +	mutex_unlock(&table_lock);
> +	return 0;
> +}
> +
> +struct list_head *virtcrypto_devmgr_get_head(void)
> +{
> +	return &virtio_crypto_table;
> +}
> +
> +/*
> + * virtcrypto_devmgr_rm_dev() - Remove vcrypto_dev from the acceleration
> + * framework.
> + * @vcrypto_dev:  Pointer to virtio crypto device.
> + *
> + * Function removes virtio crypto device from the acceleration framework.
> + * To be used by virtio crypto device specific drivers.
> + *
> + * Return: void
> + */
> +void virtcrypto_devmgr_rm_dev(struct virtio_crypto *vcrypto_dev)
> +{
> +	mutex_lock(&table_lock);
> +	list_del(&vcrypto_dev->list);
> +	num_devices--;
> +	mutex_unlock(&table_lock);
> +}
> +
> +/*
> + * virtcrypto_devmgr_get_first()
> + *
> + * Function returns the first virtio crypto device from the acceleration
> + * framework.
> + *
> + * To be used by virtio crypto device specific drivers.
> + *
> + * Return: pointer to vcrypto_dev or NULL if not found.
> + */
> +struct virtio_crypto *virtcrypto_devmgr_get_first(void)
> +{
> +	struct virtio_crypto *dev = NULL;
> +
> +	if (!list_empty(&virtio_crypto_table))
> +		dev = list_first_entry(&virtio_crypto_table,
> +					struct virtio_crypto,
> +				    list);
> +	return dev;
> +}
> +
> +/*
> + * virtcrypto_dev_in_use() - Check whether vcrypto_dev is currently in use
> + * @vcrypto_dev: Pointer to virtio crypto device.
> + *
> + * To be used by virtio crypto device specific drivers.
> + *
> + * Return: 1 when device is in use, 0 otherwise.
> + */
> +int virtcrypto_dev_in_use(struct virtio_crypto *vcrypto_dev)
> +{
> +	return atomic_read(&vcrypto_dev->ref_count) != 0;
> +}
> +
> +/*
> + * virtcrypto_dev_get() - Increment vcrypto_dev reference count
> + * @vcrypto_dev: Pointer to virtio crypto device.
> + *
> + * Increment the vcrypto_dev refcount and if this is the first time
> + * incrementing it during this period the vcrypto_dev is in use,
> + * increment the module refcount too.
> + * To be used by virtio crypto device specific drivers.
> + *
> + * Return: 0 when successful, EFAULT when fail to bump module refcount
> + */
> +int virtcrypto_dev_get(struct virtio_crypto *vcrypto_dev)
> +{
> +	if (atomic_add_return(1, &vcrypto_dev->ref_count) == 1)
> +		if (!try_module_get(vcrypto_dev->owner))
> +			return -EFAULT;
> +	return 0;
> +}
> +
> +/*
> + * virtcrypto_dev_put() - Decrement vcrypto_dev reference count
> + * @vcrypto_dev: Pointer to virtio crypto device.
> + *
> + * Decrement the vcrypto_dev refcount and if this is the last time
> + * decrementing it during this period the vcrypto_dev is in use,
> + * decrement the module refcount too.
> + * To be used by virtio crypto device specific drivers.
> + *
> + * Return: void
> + */
> +void virtcrypto_dev_put(struct virtio_crypto *vcrypto_dev)
> +{
> +	if (atomic_sub_return(1, &vcrypto_dev->ref_count) == 0)
> +		module_put(vcrypto_dev->owner);
> +}
> +
> +/*
> + * virtcrypto_dev_started() - Check whether device has started
> + * @vcrypto_dev: Pointer to virtio crypto device.
> + *
> + * To be used by virtio crypto device specific drivers.
> + *
> + * Return: 1 when the device has started, 0 otherwise
> + */
> +int virtcrypto_dev_started(struct virtio_crypto *vcrypto_dev)
> +{
> +	return (vcrypto_dev->status & VIRTIO_CRYPTO_S_HW_READY);
> +}
> +
> +/*
> + * virtcrypto_get_dev_node() - Get vcrypto_dev on the node.
> + * @node:  Node id the driver works.
> + *
> + * Function returns the virtio crypto device used fewest on the node.
> + *
> + * To be used by virtio crypto device specific drivers.
> + *
> + * Return: pointer to vcrypto_dev or NULL if not found.
> + */
> +struct virtio_crypto *virtcrypto_get_dev_node(int node)
> +{
> +	struct virtio_crypto *vcrypto_dev = NULL, *tmp_dev;
> +	unsigned long best = ~0;
> +	unsigned long ctr;
> +
> +	list_for_each_entry(tmp_dev, virtcrypto_devmgr_get_head(), list) {
> +
> +		if ((node == dev_to_node(&tmp_dev->vdev->dev) ||
> +		     dev_to_node(&tmp_dev->vdev->dev) < 0) &&
> +		    virtcrypto_dev_started(tmp_dev)) {
> +			ctr = atomic_read(&tmp_dev->ref_count);
> +			if (best > ctr) {
> +				vcrypto_dev = tmp_dev;
> +				best = ctr;
> +			}
> +		}
> +	}
> +
> +	if (!vcrypto_dev) {
> +		pr_info("virtio_crypto: Could not find a device on node %d\n",
> +				node);
> +		/* Get any started device */
> +		list_for_each_entry(tmp_dev,
> +				virtcrypto_devmgr_get_head(), list) {
> +			if (virtcrypto_dev_started(tmp_dev)) {
> +				vcrypto_dev = tmp_dev;
> +				break;
> +			}
> +		}
> +	}
> +
> +	if (!vcrypto_dev)
> +		return NULL;
> +
> +	virtcrypto_dev_get(vcrypto_dev);
> +	return vcrypto_dev;
> +}
> +
> +/*
> + * virtcrypto_dev_start() - Start virtio crypto device
> + * @vcrypto:    Pointer to virtio crypto device.
> + *
> + * Function notifies all the registered services that the virtio crypto device
> + * is ready to be used.
> + * To be used by virtio crypto device specific drivers.
> + *
> + * Return: 0 on success, EFAULT when fail to register algorithms
> + */
> +int virtcrypto_dev_start(struct virtio_crypto *vcrypto)
> +{
> +	if (virtio_crypto_algs_register()) {
> +		pr_err("virtio_crypto: Failed to register crypto algs\n");
> +		return -EFAULT;
> +	}
> +
> +	return 0;
> +}
> +
> +/*
> + * virtcrypto_dev_stop() - Stop virtio crypto device
> + * @vcrypto:    Pointer to virtio crypto device.
> + *
> + * Function notifies all the registered services that the virtio crypto device
> + * is ready to be used.
> + * To be used by virtio crypto device specific drivers.
> + *
> + * Return: void
> + */
> +void virtcrypto_dev_stop(struct virtio_crypto *vcrypto)
> +{
> +	virtio_crypto_algs_unregister();
> +}
> diff --git a/include/uapi/linux/Kbuild b/include/uapi/linux/Kbuild
> index cd2be1c..4bdb84c 100644
> --- a/include/uapi/linux/Kbuild
> +++ b/include/uapi/linux/Kbuild
> @@ -460,6 +460,7 @@ header-y += virtio_rng.h
>  header-y += virtio_scsi.h
>  header-y += virtio_types.h
>  header-y += virtio_vsock.h
> +header-y += virtio_crypto.h
>  header-y += vm_sockets.h
>  header-y += vt.h
>  header-y += vtpm_proxy.h
> diff --git a/include/uapi/linux/virtio_crypto.h b/include/uapi/linux/virtio_crypto.h
> new file mode 100644
> index 0000000..abc2cf5
> --- /dev/null
> +++ b/include/uapi/linux/virtio_crypto.h
> @@ -0,0 +1,435 @@
> +#ifndef _VIRTIO_CRYPTO_H
> +#define _VIRTIO_CRYPTO_H
> +/* This header is BSD licensed so anyone can use the definitions to implement
> + * compatible drivers/servers.
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + * 1. Redistributions of source code must retain the above copyright
> + *    notice, this list of conditions and the following disclaimer.
> + * 2. Redistributions in binary form must reproduce the above copyright
> + *    notice, this list of conditions and the following disclaimer in the
> + *    documentation and/or other materials provided with the distribution.
> + * 3. Neither the name of IBM nor the names of its contributors
> + *    may be used to endorse or promote products derived from this software
> + *    without specific prior written permission.
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL IBM OR
> + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
> + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
> + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
> + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
> + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> + * SUCH DAMAGE.
> + */
> +#include <linux/types.h>
> +#include <linux/virtio_types.h>
> +#include <linux/virtio_ids.h>
> +#include <linux/virtio_config.h>
> +
> +
> +#define VIRTIO_CRYPTO_SERVICE_CIPHER 0
> +#define VIRTIO_CRYPTO_SERVICE_HASH   1
> +#define VIRTIO_CRYPTO_SERVICE_MAC    2
> +#define VIRTIO_CRYPTO_SERVICE_AEAD   3
> +
> +#define VIRTIO_CRYPTO_OPCODE(service, op)   (((service) << 8) | (op))
> +
> +struct virtio_crypto_ctrl_header {
> +#define VIRTIO_CRYPTO_CIPHER_CREATE_SESSION \
> +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_CIPHER, 0x02)
> +#define VIRTIO_CRYPTO_CIPHER_DESTROY_SESSION \
> +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_CIPHER, 0x03)
> +#define VIRTIO_CRYPTO_HASH_CREATE_SESSION \
> +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_HASH, 0x02)
> +#define VIRTIO_CRYPTO_HASH_DESTROY_SESSION \
> +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_HASH, 0x03)
> +#define VIRTIO_CRYPTO_MAC_CREATE_SESSION \
> +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_MAC, 0x02)
> +#define VIRTIO_CRYPTO_MAC_DESTROY_SESSION \
> +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_MAC, 0x03)
> +#define VIRTIO_CRYPTO_AEAD_CREATE_SESSION \
> +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x02)
> +#define VIRTIO_CRYPTO_AEAD_DESTROY_SESSION \
> +	   VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x03)
> +	__le32 opcode;
> +	__le32 algo;
> +	__le32 flag;
> +	/* data virtqueue id */
> +	__le32 queue_id;
> +};
> +
> +struct virtio_crypto_cipher_session_para {
> +#define VIRTIO_CRYPTO_NO_CIPHER                 0
> +#define VIRTIO_CRYPTO_CIPHER_ARC4               1
> +#define VIRTIO_CRYPTO_CIPHER_AES_ECB            2
> +#define VIRTIO_CRYPTO_CIPHER_AES_CBC            3
> +#define VIRTIO_CRYPTO_CIPHER_AES_CTR            4
> +#define VIRTIO_CRYPTO_CIPHER_DES_ECB            5
> +#define VIRTIO_CRYPTO_CIPHER_DES_CBC            6
> +#define VIRTIO_CRYPTO_CIPHER_3DES_ECB           7
> +#define VIRTIO_CRYPTO_CIPHER_3DES_CBC           8
> +#define VIRTIO_CRYPTO_CIPHER_3DES_CTR           9
> +#define VIRTIO_CRYPTO_CIPHER_KASUMI_F8          10
> +#define VIRTIO_CRYPTO_CIPHER_SNOW3G_UEA2        11
> +#define VIRTIO_CRYPTO_CIPHER_AES_F8             12
> +#define VIRTIO_CRYPTO_CIPHER_AES_XTS            13
> +#define VIRTIO_CRYPTO_CIPHER_ZUC_EEA3           14
> +	__le32 algo;
> +	/* length of key */
> +	__le32 keylen;
> +
> +#define VIRTIO_CRYPTO_OP_ENCRYPT  1
> +#define VIRTIO_CRYPTO_OP_DECRYPT  2
> +	/* encrypt or decrypt */
> +	__le32 op;
> +	__le32 padding;
> +};
> +
> +struct virtio_crypto_session_input {
> +	/* Device-writable part */
> +	__le64 session_id;
> +	__le32 status;
> +	__le32 padding;
> +};
> +
> +struct virtio_crypto_cipher_session_req {
> +	struct virtio_crypto_cipher_session_para para;
> +};
> +
> +struct virtio_crypto_hash_session_para {
> +#define VIRTIO_CRYPTO_NO_HASH            0
> +#define VIRTIO_CRYPTO_HASH_MD5           1
> +#define VIRTIO_CRYPTO_HASH_SHA1          2
> +#define VIRTIO_CRYPTO_HASH_SHA_224       3
> +#define VIRTIO_CRYPTO_HASH_SHA_256       4
> +#define VIRTIO_CRYPTO_HASH_SHA_384       5
> +#define VIRTIO_CRYPTO_HASH_SHA_512       6
> +#define VIRTIO_CRYPTO_HASH_SHA3_224      7
> +#define VIRTIO_CRYPTO_HASH_SHA3_256      8
> +#define VIRTIO_CRYPTO_HASH_SHA3_384      9
> +#define VIRTIO_CRYPTO_HASH_SHA3_512      10
> +#define VIRTIO_CRYPTO_HASH_SHA3_SHAKE128      11
> +#define VIRTIO_CRYPTO_HASH_SHA3_SHAKE256      12
> +	__le32 algo;
> +	/* hash result length */
> +	__le32 hash_result_len;
> +};
> +
> +struct virtio_crypto_hash_create_session_req {
> +	struct virtio_crypto_hash_session_para para;
> +};
> +
> +struct virtio_crypto_mac_session_para {
> +#define VIRTIO_CRYPTO_NO_MAC                       0
> +#define VIRTIO_CRYPTO_MAC_HMAC_MD5                 1
> +#define VIRTIO_CRYPTO_MAC_HMAC_SHA1                2
> +#define VIRTIO_CRYPTO_MAC_HMAC_SHA_224             3
> +#define VIRTIO_CRYPTO_MAC_HMAC_SHA_256             4
> +#define VIRTIO_CRYPTO_MAC_HMAC_SHA_384             5
> +#define VIRTIO_CRYPTO_MAC_HMAC_SHA_512             6
> +#define VIRTIO_CRYPTO_MAC_CMAC_3DES                25
> +#define VIRTIO_CRYPTO_MAC_CMAC_AES                 26
> +#define VIRTIO_CRYPTO_MAC_KASUMI_F9                27
> +#define VIRTIO_CRYPTO_MAC_SNOW3G_UIA2              28
> +#define VIRTIO_CRYPTO_MAC_GMAC_AES                 41
> +#define VIRTIO_CRYPTO_MAC_GMAC_TWOFISH             42
> +#define VIRTIO_CRYPTO_MAC_CBCMAC_AES               49
> +#define VIRTIO_CRYPTO_MAC_CBCMAC_KASUMI_F9         50
> +#define VIRTIO_CRYPTO_MAC_XCBC_AES                 53
> +	__le32 algo;
> +	/* hash result length */
> +	__le32 hash_result_len;
> +	/* length of authenticated key */
> +	__le32 auth_key_len;
> +	__le32 padding;
> +};
> +
> +struct virtio_crypto_mac_create_session_req {
> +	struct virtio_crypto_mac_session_para para;
> +};
> +
> +struct virtio_crypto_aead_session_para {
> +#define VIRTIO_CRYPTO_NO_AEAD     0
> +#define VIRTIO_CRYPTO_AEAD_GCM    1
> +#define VIRTIO_CRYPTO_AEAD_CCM    2
> +#define VIRTIO_CRYPTO_AEAD_CHACHA20_POLY1305  3
> +	__le32 algo;
> +	/* length of key */
> +	__le32 key_len;
> +	/* hash result length */
> +	__le32 hash_result_len;
> +	/* length of the additional authenticated data (AAD) in bytes */
> +	__le32 aad_len;
> +	/* encrypt or decrypt, See above VIRTIO_CRYPTO_OP_* */
> +	__le32 op;
> +	__le32 padding;
> +};
> +
> +struct virtio_crypto_aead_create_session_req {
> +	struct virtio_crypto_aead_session_para para;
> +};
> +
> +struct virtio_crypto_alg_chain_session_para {
> +#define VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER  1
> +#define VIRTIO_CRYPTO_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH  2
> +	__le32 alg_chain_order;
> +/* Plain hash */
> +#define VIRTIO_CRYPTO_SYM_HASH_MODE_PLAIN    1
> +/* Authenticated hash (mac) */
> +#define VIRTIO_CRYPTO_SYM_HASH_MODE_AUTH     2
> +/* Nested hash */
> +#define VIRTIO_CRYPTO_SYM_HASH_MODE_NESTED   3
> +	__le32 hash_mode;
> +	struct virtio_crypto_cipher_session_para cipher_param;
> +	union {
> +		struct virtio_crypto_hash_session_para hash_param;
> +		struct virtio_crypto_mac_session_para mac_param;
> +	} u;

I personally dislike the use of unions of different size
in these structures. For example, using hash_param,
there's no way to initialize the trailing part of the
structure. Also, the size of the structure is hard to
figure out. I would like to see padding of all structures
to be same size, and adding a union member that's just padding.

Same applies to other unions.

> +	/* length of the additional authenticated data (AAD) in bytes */
> +	__le32 aad_len;
> +	__le32 padding;
> +};
> +
> +struct virtio_crypto_alg_chain_session_req {
> +	struct virtio_crypto_alg_chain_session_para para;
> +};
> +
> +struct virtio_crypto_sym_create_session_req {
> +	union {
> +		struct virtio_crypto_cipher_session_req cipher;
> +		struct virtio_crypto_alg_chain_session_req chain;
> +	} u;
> +
> +	/* Device-readable part */
> +
> +/* No operation */
> +#define VIRTIO_CRYPTO_SYM_OP_NONE  0
> +/* Cipher only operation on the data */
> +#define VIRTIO_CRYPTO_SYM_OP_CIPHER  1
> +/*
> + * Chain any cipher with any hash or mac operation. The order
> + * depends on the value of alg_chain_order param
> + */
> +#define VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING  2
> +	__le32 op_type;
> +	__le32 padding;
> +};
> +
> +struct virtio_crypto_destroy_session_req {
> +	/* Device-readable part */
> +	__le64  session_id;
> +};
> +
> +/* The request of the control viritqueue's packet */
> +struct virtio_crypto_op_ctrl_req {
> +	struct virtio_crypto_ctrl_header header;
> +
> +	union {
> +		struct virtio_crypto_sym_create_session_req
> +			sym_create_session;
> +		struct virtio_crypto_hash_create_session_req
> +			hash_create_session;
> +		struct virtio_crypto_mac_create_session_req
> +			mac_create_session;
> +		struct virtio_crypto_aead_create_session_req
> +			aead_create_session;
> +		struct virtio_crypto_destroy_session_req
> +			destroy_session;
> +	} u;
> +};
> +
> +struct virtio_crypto_op_header {
> +#define VIRTIO_CRYPTO_CIPHER_ENCRYPT \
> +	VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_CIPHER, 0x00)
> +#define VIRTIO_CRYPTO_CIPHER_DECRYPT \
> +	VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_CIPHER, 0x01)
> +#define VIRTIO_CRYPTO_HASH \
> +	VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_HASH, 0x00)
> +#define VIRTIO_CRYPTO_MAC \
> +	VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_MAC, 0x00)
> +#define VIRTIO_CRYPTO_AEAD_ENCRYPT \
> +	VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x00)
> +#define VIRTIO_CRYPTO_AEAD_DECRYPT \
> +	VIRTIO_CRYPTO_OPCODE(VIRTIO_CRYPTO_SERVICE_AEAD, 0x01)
> +	__le32 opcode;
> +	/* algo should be service-specific algorithms */
> +	__le32 algo;
> +	/* session_id should be service-specific algorithms */
> +	__le64 session_id;
> +	/* control flag to control the request */
> +	__le32 flag;
> +	__le32 padding;
> +};
> +
> +struct virtio_crypto_cipher_para {
> +	/*
> +	 * Byte Length of valid IV/Counter
> +	 *
> +	 * For block ciphers in CBC or F8 mode, or for Kasumi in F8 mode, or for
> +	 *   SNOW3G in UEA2 mode, this is the length of the IV (which
> +	 *   must be the same as the block length of the cipher).
> +	 * For block ciphers in CTR mode, this is the length of the counter
> +	 *   (which must be the same as the block length of the cipher).
> +	 * For AES-XTS, this is the 128bit tweak, i, from IEEE Std 1619-2007.
> +	 *
> +	 * The IV/Counter will be updated after every partial cryptographic
> +	 * operation.
> +	 */
> +	__le32 iv_len;
> +	/* length of source data */
> +	__le32 src_data_len;
> +	/* length of dst data */
> +	__le32 dst_data_len;
> +	__le32 padding;
> +};
> +
> +struct virtio_crypto_hash_para {
> +	/* length of source data */
> +	__le32 src_data_len;
> +	/* hash result length */
> +	__le32 hash_result_len;
> +};
> +
> +struct virtio_crypto_mac_para {
> +	struct virtio_crypto_hash_para hash;
> +};
> +
> +struct virtio_crypto_aead_para {
> +	/*
> +	 * Byte Length of valid IV data pointed to by the below iv_addr
> +	 * parameter.
> +	 *
> +	 * For GCM mode, this is either 12 (for 96-bit IVs) or 16, in which
> +	 *   case iv_addr points to J0.
> +	 * For CCM mode, this is the length of the nonce, which can be in the
> +	 *   range 7 to 13 inclusive.
> +	 */
> +	__le32 iv_len;
> +	/* length of additional auth data */
> +	__le32 aad_len;
> +	/* length of source data */
> +	__le32 src_data_len;
> +	/* length of dst data */
> +	__le32 dst_data_len;
> +};
> +
> +struct virtio_crypto_cipher_data_req {
> +	/* Device-readable part */
> +	struct virtio_crypto_cipher_para para;
> +};
> +
> +struct virtio_crypto_hash_data_req {
> +	/* Device-readable part */
> +	struct virtio_crypto_hash_para para;
> +};
> +
> +struct virtio_crypto_mac_data_req {
> +	/* Device-readable part */
> +	struct virtio_crypto_mac_para para;
> +};
> +
> +struct virtio_crypto_alg_chain_data_para {
> +	__le32 iv_len;
> +	/* Length of source data */
> +	__le32 src_data_len;
> +	/* Length of destination data */
> +	__le32 dst_data_len;
> +	/* Starting point for cipher processing in source data */
> +	__le32 cipher_start_src_offset;
> +	/* Length of the source data that the cipher will be computed on */
> +	__le32 len_to_cipher;
> +	/* Starting point for hash processing in source data */
> +	__le32 hash_start_src_offset;
> +	/* Length of the source data that the hash will be computed on */
> +	__le32 len_to_hash;
> +	/* Length of the additional auth data */
> +	__le32 aad_len;
> +	/* Length of the hash result */
> +	__le32 hash_result_len;
> +	__le32 reserved;
> +};
> +
> +struct virtio_crypto_alg_chain_data_req {
> +	/* Device-readable part */
> +	struct virtio_crypto_alg_chain_data_para para;
> +};
> +
> +struct virtio_crypto_sym_data_req {
> +	union {
> +		struct virtio_crypto_cipher_data_req cipher;
> +		struct virtio_crypto_alg_chain_data_req chain;
> +	} u;
> +
> +	/* See above VIRTIO_CRYPTO_SYM_OP_* */
> +	__le32 op_type;
> +	__le32 padding;
> +};
> +
> +struct virtio_crypto_aead_data_req {
> +	/* Device-readable part */
> +	struct virtio_crypto_aead_para para;
> +};
> +
> +/* The request of the data viritqueue's packet */
> +struct virtio_crypto_op_data_req {
> +	struct virtio_crypto_op_header header;
> +
> +	union {
> +		struct virtio_crypto_sym_data_req  sym_req;
> +		struct virtio_crypto_hash_data_req hash_req;
> +		struct virtio_crypto_mac_data_req mac_req;
> +		struct virtio_crypto_aead_data_req aead_req;
> +	} u;
> +};
> +
> +#define VIRTIO_CRYPTO_OK        0
> +#define VIRTIO_CRYPTO_ERR       1
> +#define VIRTIO_CRYPTO_BADMSG    2
> +#define VIRTIO_CRYPTO_NOTSUPP   3
> +#define VIRTIO_CRYPTO_INVSESS   4 /* Invaild session id */
> +
> +/* The accelerator hardware is ready */
> +#define VIRTIO_CRYPTO_S_HW_READY  (1 << 0)
> +
> +struct virtio_crypto_config {
> +	/* See VIRTIO_CRYPTO_OP_* above */
> +	__le32  status;
> +
> +	/*
> +	 * Maximum number of data queue legal values are between 1 and 0x8000
> +	 */
> +	__le32  max_dataqueues;
> +
> +	/*
> +	 * Specifies the services mask which the devcie support,
> +	 * see VIRTIO_CRYPTO_SERVICE_* above
> +	 */
> +	__le32 crypto_services;
> +
> +	/* Detailed algorithms mask */
> +	__le32 cipher_algo_l;
> +	__le32 cipher_algo_h;
> +	__le32 hash_algo;
> +	__le32 mac_algo_l;
> +	__le32 mac_algo_h;
> +	__le32 aead_algo;
> +	/* Maximum length of cipher key */
> +	__le32 max_cipher_key_len;
> +	/* Maximum length of authenticated key */
> +	__le32 max_auth_key_len;
> +	__le32 reserve;
> +	/* Maximum size of each crypto request's content */
> +	__le64 max_size;
> +};
> +
> +struct virtio_crypto_inhdr {
> +	/* See VIRTIO_CRYPTO_* above */
> +	__u8 status;
> +};
> +#endif
> diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h
> index 3228d58..6d5c3b2 100644
> --- a/include/uapi/linux/virtio_ids.h
> +++ b/include/uapi/linux/virtio_ids.h
> @@ -42,5 +42,6 @@
>  #define VIRTIO_ID_GPU          16 /* virtio GPU */
>  #define VIRTIO_ID_INPUT        18 /* virtio input */
>  #define VIRTIO_ID_VSOCK        19 /* virtio vsock transport */
> +#define VIRTIO_ID_CRYPTO       20 /* virtio crypto */
>  
>  #endif /* _LINUX_VIRTIO_IDS_H */
> -- 
> 1.8.3.1
> 

^ permalink raw reply

* Re: [PATCH 0/2] virtio: fix complaint by sparse
From: Michael S. Tsirkin @ 2016-11-27  2:53 UTC (permalink / raw)
  To: Gonglei (Arei)
  Cc: linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org
In-Reply-To: <33183CC9F5247A488A2544077AF19020D979FA1E@SZXEMA503-MBS.china.huawei.com>

this is in my tree, will be in the next pull request.

On Sat, Nov 26, 2016 at 09:36:50AM +0000, Gonglei (Arei) wrote:
> Ping...?
> 
> > -----Original Message-----
> > From: Gonglei (Arei)
> > Sent: Tuesday, November 22, 2016 1:52 PM
> > To: virtualization@lists.linux-foundation.org; linux-kernel@vger.kernel.org
> > Cc: mst@redhat.com; jasowang@redhat.com; Gonglei (Arei)
> > Subject: [PATCH 0/2] virtio: fix complaint by sparse
> > 
> > I found some warnings reported by sparse in the virtio code
> > when I checked virtio-crypto's driver stuff. Let's fix them.
> > 
> > Gonglei (2):
> >   virtio_pci_modern: fix complaint by sparse
> >   virtio_ring: fix complaint by sparse
> > 
> >  drivers/virtio/virtio_pci_modern.c | 8 ++++----
> >  drivers/virtio/virtio_ring.c       | 4 ++--
> >  2 files changed, 6 insertions(+), 6 deletions(-)
> > 
> > --
> > 1.8.3.1
> > 

^ permalink raw reply

* WorldCIST'2017 - Submission deadline: November 30
From: ML @ 2016-11-26 11:51 UTC (permalink / raw)
  To: virtualization

[-- Attachment #1: Type: text/plain, Size: 8001 bytes --]

* Best papers published in several SCI/SSCI-indexed journals
** Proceedings by Springer, indexed by ISI, Scopus, DBLP, EI-Compendex, etc.

---------------------------------------------------------------------------------
WorldCIST'17 - 5th World Conference on Information Systems and Technologies 
Porto Santo Island, Madeira, Portugal
11th-13th of April 2017
http://www.worldcist.org/
-------------------------------------------------------------------------


SCOPE

The WorldCist'17 - 5th World Conference on Information Systems and Technologies, to be held at Porto Santo Island, Madeira, Portugal, 11 - 13 April 2017, is a global forum for researchers and practitioners to present and discuss the most recent innovations, trends, results, experiences and concerns in the several perspectives of Information Systems and Technologies.

We are pleased to invite you to submit your papers to WorldCist'17 (http://www.worldcist.org/). All submissions will be reviewed on the basis of relevance, originality, importance and clarity.


THEMES

Submitted papers should be related with one or more of the main themes proposed for the Conference:

A) Information and Knowledge Management (IKM);
B) Organizational Models and Information Systems (OMIS);
C) Software and Systems Modeling (SSM);
D) Software Systems, Architectures, Applications and Tools (SSAAT);
E) Multimedia Systems and Applications (MSA);
F) Computer Networks, Mobility and Pervasive Systems (CNMPS);
G) Intelligent and Decision Support Systems (IDSS);
H) Big Data Analytics and Applications (BDAA);
I) Human-Computer Interaction (HCI);
J) Ethics, Computers and Security (ECS)
K) Health Informatics (HIS);
L) Information Technologies in Education (ITE);
M) Information Technologies in Radiocommunications (ITR).


TYPES of SUBMISSIONS AND DECISIONS

Four types of papers can be submitted:

- Full paper: Finished or consolidated R&D works, to be included in one of the Conference themes. These papers are assigned a 10-page limit.

- Short paper: Ongoing works with relevant preliminary results, open to discussion. These papers are assigned a 7-page limit.

- Poster paper: Initial work with relevant ideas, open to discussion. These papers are assigned to a 4-page limit.

- Company paper: Companies' papers that show practical experience, R & D, tools, etc., focused on some topics of the conference. These papers are assigned to a 4-page limit.

Submitted papers must comply with the format of Advances in Intelligent Systems and Computing Series (see Instructions for Authors at Springer Website or download a DOC example) be written in English, must not have been published before, not be under review for any other conference or publication and not include any information leading to the authors’ identification. Therefore, the authors’ names, affiliations and bibliographic references should not be included in the version for evaluation by the Program Committee. This information should only be included in the camera-ready version, saved in Word or Latex format and also in PDF format. These files must be accompanied by the Consent to Publication form filled out, in a ZIP file, and uploaded at the conference management system.

All papers will be subjected to a “double-blind review” by at least two members of the Program Committee.

Based on Program Committee evaluation, a paper can be rejected or accepted by the Conference Chairs. In the later case, it can be accepted as the type originally submitted or as another type. Thus, full papers can be accepted as short papers or poster papers only. Similarly, short papers can be accepted as poster papers only. In these cases, the authors will be allowed to maintain the original number of pages in the camera-ready version.

The authors of accepted poster papers must also build and print a poster to be exhibited during the Conference. This poster must follow an A1 or A2 vertical format. The Conference can includes Work Sessions where these posters are presented and orally discussed, with a 5 minute limit per poster.

The authors of accepted full papers will have 15 minutes to present their work in a Conference Work Session; approximately 5 minutes of discussion will follow each presentation. The authors of accepted short papers and company papers will have 11 minutes to present their work in a Conference Work Session; approximately 4 minutes of discussion will follow each presentation.


PUBLICATION & INDEXING

To ensure that a full paper, short paper, poster paper or company paper is published in the Proceedings, at least one of the authors must be fully registered by the 8th of January 2017, and the paper must comply with the suggested layout and page-limit. Additionally, all recommended changes must be addressed by the authors before they submit the camera-ready version.

No more than one paper per registration will be published in the Conference Proceedings. An extra fee must be paid for publication of additional papers, with a maximum of one additional paper per registration. One registration permits only the participation of one author in the conference.

Full and short papers will be published in Proceedings by Springer, in Advances in Intelligent Systems and Computing Series. Poster and company papers will be published by AISTI.

Published full and short papers will be submitted for indexation by ISI, EI-Compendex, SCOPUS, DBLP and Google Scholar, among others, and will be available in the SpringerLink Digital Library.

The authors of the best selected papers will be invited to extend them for publication in international journals indexed by ISI/SCI, SCOPUS and DBLP, among others, such as:

- International Journal of Neural Systems (IF: 6.085 / Q1)
- Integrated Computer-Aided Engineering (IF: 4.981 / Q1)
- International Journal of Information Management (IF: 2.692 / Q1)
- Telematics & Informatics (IF: 2.261 / Q1) - Special Issue on "Disruption of Higher Education in the 21st Century due to ICTs"
- Electronic Commerce Research and Applications (IF: 2.139 / Q1)
- Computers, Environment and Urban Systems (IF: 2.092 / Q1)
- Data Mining and Knowledge Discovery (IF: 1.759 / Q1)
- Journal of Medical Systems (IF: 2.213 / Q2)
- Journal of Business Research (IF: 2.129 / Q2) - Special Issue on "Strategic Knowledge Management in the Digital Age"
- Pervasive and Mobile Computing (IF: 1.719 / Q2)
- Knowledge and Information Systems (IF: 1.702 / Q2)
- Journal of Grid Computing (IF: 1.561 / Q2) - Special Issue on "Big Data"
- Cluster Computing (IF:1.514 / Q2) - Special Issue on "Advanced Machine Learning in Parallel and Distributed Knowledge Discovery"
- International Journal of Critical Infrastructure Protection (IF: 1.351 / Q2)
- Expert Systems - Journal of Knowledge Engineering (IF: 0.947 / Q3)
- Concurrency and Computation: Practice and Experience (IF: 0.942 / Q3)
- Ethics and Information Technology (IF: 0.739 / Q3)
- Annals of Telecommunications (IF: 0.722 / Q3)
- Engineering Computations (IF: 0.691 / Q3)
- Advances in Complex Systems (IF: 0.461 / Q3)
- Computing and Informatics (IF: 0.504 / Q4)
- AI Communications (IF: 0.364 / Q4)
- Journal of Hospitality and Tourism Technology (SR: 0.672 / Q2)
- Transforming Government: People, Process and Policy (SR: 0.642 / Q2)
- TEM Journal - Technology, Education, Management, Informatics (ISI - Emerging Sources Citation Index)
- Computer Methods in Biomechanics and Biomedical Engineering - Imaging & Visualization (ISI - Emerging Sources Citation Index)
- Journal of Information Systems Engineering & Management


IMPORTANT DATES

Paper Submission: November 27, 2016

Notification of Acceptance: December 25, 20156

Payment of Registration, to ensure the inclusion of an accepted paper in the conference proceedings: January 8, 2017.

Camera-ready Submission: January 8, 2017


-

Website of the WorldCIST'17
http://www.worldcist.org/








[-- Attachment #2: Type: text/plain, Size: 183 bytes --]

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply

* RE: [PATCH v2 0/2] virtio-crypto: add Linux driver
From: Gonglei (Arei) @ 2016-11-26  9:38 UTC (permalink / raw)
  To: Gonglei (Arei), linux-kernel@vger.kernel.org,
	qemu-devel@nongnu.org, virtio-dev@lists.oasis-open.org,
	virtualization@lists.linux-foundation.org,
	linux-crypto@vger.kernel.org
  Cc: Huangweidong (C), Claudio Fontana, mst@redhat.com, Luonengjun,
	Hanweidong (Randy), Xuquan (Quan Xu),
	salvatore.benedetto@intel.com, stefanha@redhat.com,
	Zhoujian (jay, Euler), longpeng, arei.gonglei@hotmail.com,
	davem@davemloft.net, Wubin (H), herbert@gondor.apana.org.au
In-Reply-To: <1479802223-121104-1-git-send-email-arei.gonglei@huawei.com>

Hi,

> -----Original Message-----
> From: Gonglei (Arei)
> Sent: Tuesday, November 22, 2016 4:10 PM
> To: linux-kernel@vger.kernel.org; qemu-devel@nongnu.org;
> virtio-dev@lists.oasis-open.org; virtualization@lists.linux-foundation.org;
> linux-crypto@vger.kernel.org
> Subject: [PATCH v2 0/2] virtio-crypto: add Linux driver
> 
> The virtio crypto device is a virtual cryptography device
> as well as a kind of virtual hardware accelerator for
> virtual machines. The encryption anddecryption requests
> are placed in the data queue and are ultimately handled by
> thebackend crypto accelerators. The second queue is the
> control queue used to create or destroy sessions for
> symmetric algorithms and will control some advanced features
> in the future. The virtio crypto device provides the following
> cryptoservices: CIPHER, MAC, HASH, and AEAD.
> 
> For more information about virtio-crypto device, please see:
>   http://qemu-project.org/Features/VirtioCrypto
> 
> For better reviewing:
> 
> Patch 1 introduces the little edian functions for VIRTIO_1
> devices.
> 
> Patch 2 mainly includes five files:
>  1) virtio_crypto.h is the header file for virtio-crypto device,
> which is based on the virtio-crypto specification.
>  2) virtio_crypto.c is the entry of the driver module,
> which is similar with other virtio devices, such as virtio-net,
> virtio-input etc.
>  3) virtio_crypto_mgr.c is used to manage the virtio
> crypto devices in the system. We support up to 32 virtio-crypto
> devices currently. I use a global list to store the virtio crypto
> devices which refer to Intel QAT driver. Meanwhile, the file
> includs the functions of add/del/search/start/stop for virtio
> crypto devices.
>  4) virtio_crypto_common.h is a private header file for virtio
> crypto driver, includes structure definations, and function declarations.
>  5) virtio_crypto_algs.c is the realization of algs based on Linux Crypto
> Framwork,
> which can register different crypto algorithms. Currently it's only support
> AES-CBC.
> The Crypto guys can mainly focus to this file.
> 
> Actually I have no idea the virtio-crypto driver should be gone in whose
> tree, Michael's or Herbert's?
> 
> Would you give me a feedback? Thanks a lot!
> 
Ping?

Any ideas? Thanks.

Regards,
-Gonglei

> 
> v2:
>  - stop doing DMA from the stack, CONFIG_VMAP_STACK=y [Salvatore]
>  - convert __virtio32/64 to __le32/64 in virtio_crypto.h
>  - remove VIRTIO_CRYPTO_S_STARTED based on the lastest virtio crypto spec.
>  - introduces the little edian functions for VIRTIO_1 devices in patch 1.
> 
> Gonglei (2):
>   virtio: introduce little edian functions for virtio_cread/write#
>     family
>   crypto: add virtio-crypto driver
> 
>  MAINTAINERS                                  |   8 +
>  drivers/crypto/Kconfig                       |   2 +
>  drivers/crypto/Makefile                      |   1 +
>  drivers/crypto/virtio/Kconfig                |  10 +
>  drivers/crypto/virtio/Makefile               |   5 +
>  drivers/crypto/virtio/virtio_crypto.c        | 444
> +++++++++++++++++++++++
>  drivers/crypto/virtio/virtio_crypto_algs.c   | 524
> +++++++++++++++++++++++++++
>  drivers/crypto/virtio/virtio_crypto_common.h | 124 +++++++
>  drivers/crypto/virtio/virtio_crypto_mgr.c    | 258 +++++++++++++
>  include/linux/virtio_config.h                |  45 +++
>  include/uapi/linux/Kbuild                    |   1 +
>  include/uapi/linux/virtio_crypto.h           | 435
> ++++++++++++++++++++++
>  include/uapi/linux/virtio_ids.h              |   1 +
>  13 files changed, 1858 insertions(+)
>  create mode 100644 drivers/crypto/virtio/Kconfig
>  create mode 100644 drivers/crypto/virtio/Makefile
>  create mode 100644 drivers/crypto/virtio/virtio_crypto.c
>  create mode 100644 drivers/crypto/virtio/virtio_crypto_algs.c
>  create mode 100644 drivers/crypto/virtio/virtio_crypto_common.h
>  create mode 100644 drivers/crypto/virtio/virtio_crypto_mgr.c
>  create mode 100644 include/uapi/linux/virtio_crypto.h
> 
> --
> 1.8.3.1
> 

^ permalink raw reply

* RE: [PATCH 0/2] virtio: fix complaint by sparse
From: Gonglei (Arei) @ 2016-11-26  9:36 UTC (permalink / raw)
  To: Gonglei (Arei), virtualization@lists.linux-foundation.org,
	linux-kernel@vger.kernel.org
  Cc: mst@redhat.com
In-Reply-To: <1479793910-120188-1-git-send-email-arei.gonglei@huawei.com>

Ping...?

> -----Original Message-----
> From: Gonglei (Arei)
> Sent: Tuesday, November 22, 2016 1:52 PM
> To: virtualization@lists.linux-foundation.org; linux-kernel@vger.kernel.org
> Cc: mst@redhat.com; jasowang@redhat.com; Gonglei (Arei)
> Subject: [PATCH 0/2] virtio: fix complaint by sparse
> 
> I found some warnings reported by sparse in the virtio code
> when I checked virtio-crypto's driver stuff. Let's fix them.
> 
> Gonglei (2):
>   virtio_pci_modern: fix complaint by sparse
>   virtio_ring: fix complaint by sparse
> 
>  drivers/virtio/virtio_pci_modern.c | 8 ++++----
>  drivers/virtio/virtio_ring.c       | 4 ++--
>  2 files changed, 6 insertions(+), 6 deletions(-)
> 
> --
> 1.8.3.1
> 

^ permalink raw reply

* Re: [PATCH 0/3] virtio/vringh: kill off ACCESS_ONCE()
From: Christian Borntraeger @ 2016-11-25 21:45 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Mark Rutland, Davidlohr Bueso, KVM list, dbueso, Peter Zijlstra,
	netdev, Boqun Feng, LKML, virtualization, Paul McKenney,
	Linus Torvalds, Dmitry Vyukov
In-Reply-To: <20161125230735-mutt-send-email-mst@kernel.org>

On 11/25/2016 10:08 PM, Michael S. Tsirkin wrote:
> On Fri, Nov 25, 2016 at 05:49:45PM +0100, Christian Borntraeger wrote:
>> On 11/25/2016 05:17 PM, Peter Zijlstra wrote:
>>> On Fri, Nov 25, 2016 at 04:10:04PM +0000, Mark Rutland wrote:
>>>> On Fri, Nov 25, 2016 at 04:21:39PM +0100, Dmitry Vyukov wrote:
>>>
>>>>> What are use cases for such primitive that won't be OK with "read once
>>>>> _and_ atomically"?
>>>>
>>>> I have none to hand.
>>>
>>> Whatever triggers the __builtin_memcpy() paths, and even the size==8
>>> paths on 32bit.
>>>
>>> You could put a WARN in there to easily find them.
>>
>> There were several cases that I found during writing the *ONCE stuff.
>> For example there are some 32bit ppc variants with 64bit PTEs. Some for
>> others (I think sparc). And the mm/ code is perfectly fine with these
>> PTE accesses being done NOT atomic.
> 
> In that case do we even need _ONCE at all?

Yes. For example look at gup_pmd_range. Here several checks are made on the pmd.
It is important the the check for pmd_none is made on the same value than
the check for pmd_trans_huge, but it is not important that the value is still up
to date. 
And there are really cases where we cannot read the  thing atomically, e.g. on 
m68k and sparc(32bit) pmd_t is defined as array of longs.

Another problem is that a compiler can implement the following code as 2 memory
reads (e.g. if you have compare instructions that work on memory) instead of a 
memory read and 2 compares

int check(unsigned long *value_p) {
	unsigned long value = *value_p;
	if (condition_a(value))
		return 1;
	if (condition_b(value))
		return 2;
	return 3;
}

With READ_ONCE you forbid that. In past times you would have used barrier() after 
the assignment to achieve the same goal.


> Are there assumptions these are two 32 bit reads?

It depends on the code. Some places (e.g. in gup) assumes that the access via
READ_ONCE is atomic (which it is for sane compilers as long as the pointer
is <= word size). In some others places just one bit is tested.
> 
> 
>>
>>>
>>> The advantage of introducing the SINGLE_{LOAD,STORE}() helpers is that
>>> they compiletime validate this the size is 'right' and can runtime check
>>> alignment constraints.
>>>
>>> IE, they are strictly stronger than {READ,WRITE}_ONCE().
>>>
> 

^ permalink raw reply

* Re: [PATCH 0/3] virtio/vringh: kill off ACCESS_ONCE()
From: Michael S. Tsirkin @ 2016-11-25 21:08 UTC (permalink / raw)
  To: Christian Borntraeger
  Cc: Mark Rutland, Davidlohr Bueso, KVM list, dbueso, Peter Zijlstra,
	netdev, Boqun Feng, LKML, virtualization, Paul McKenney,
	Linus Torvalds, Dmitry Vyukov
In-Reply-To: <d7f3740b-e343-68fc-4996-f712dd8c07f3@de.ibm.com>

On Fri, Nov 25, 2016 at 05:49:45PM +0100, Christian Borntraeger wrote:
> On 11/25/2016 05:17 PM, Peter Zijlstra wrote:
> > On Fri, Nov 25, 2016 at 04:10:04PM +0000, Mark Rutland wrote:
> >> On Fri, Nov 25, 2016 at 04:21:39PM +0100, Dmitry Vyukov wrote:
> > 
> >>> What are use cases for such primitive that won't be OK with "read once
> >>> _and_ atomically"?
> >>
> >> I have none to hand.
> > 
> > Whatever triggers the __builtin_memcpy() paths, and even the size==8
> > paths on 32bit.
> > 
> > You could put a WARN in there to easily find them.
> 
> There were several cases that I found during writing the *ONCE stuff.
> For example there are some 32bit ppc variants with 64bit PTEs. Some for
> others (I think sparc). And the mm/ code is perfectly fine with these
> PTE accesses being done NOT atomic.

In that case do we even need _ONCE at all?
Are there assumptions these are two 32 bit reads?


> 
> > 
> > The advantage of introducing the SINGLE_{LOAD,STORE}() helpers is that
> > they compiletime validate this the size is 'right' and can runtime check
> > alignment constraints.
> > 
> > IE, they are strictly stronger than {READ,WRITE}_ONCE().
> > 

^ permalink raw reply

* Re: [PATCH 0/3] virtio/vringh: kill off ACCESS_ONCE()
From: Linus Torvalds @ 2016-11-25 18:47 UTC (permalink / raw)
  To: Mark Rutland
  Cc: Davidlohr Bueso, KVM list, Michael S. Tsirkin, Peter Zijlstra,
	netdev, Boqun Feng, Davidlohr Bueso, LKML, virtualization,
	Paul McKenney, Dmitry Vyukov
In-Reply-To: <20161125180649.GC30811@leverpostej>

On Fri, Nov 25, 2016 at 10:07 AM, Mark Rutland <mark.rutland@arm.com> wrote:
> On Fri, Nov 25, 2016 at 09:52:50AM -0800, Linus Torvalds wrote:
>> READ/WRITE_ONCE() are atomic *WHEN*THAT*IS*POSSIBLE*.
>
>> But sometimes it's not going to be atomic.
>
> That's the problem.

It has never really been much of a problem, and quite frankly, the
solution would never be to add _another_ crazy new function that will
just confuse everybody.

If you have code that depends on atomicity of READ_ONCE() and friends,
then you should add the appropriate built-time assert to *your* code.
Not to some random generic function that others care about and that
others do _not_ have problems with.

So if you have a data structure in virtio that is
architecture-dependent and might not be a word size, you add the

   BUILD_BUG_ON(sizeof(mytype) > sizeof(long));

or whatever. With a big comment saying "this needs to actually fit in
a single register so that we can do atomic accesses".

You do not screw it up for everybody else.

                Linus

^ permalink raw reply

* Re: [PATCH 0/3] virtio/vringh: kill off ACCESS_ONCE()
From: Christian Borntraeger @ 2016-11-25 18:46 UTC (permalink / raw)
  To: Mark Rutland
  Cc: Davidlohr Bueso, KVM list, Michael S. Tsirkin, Peter Zijlstra,
	netdev, Boqun Feng, dbueso, LKML, virtualization, Paul McKenney,
	Linus Torvalds, Dmitry Vyukov
In-Reply-To: <20161125172624.GA30811@leverpostej>

On 11/25/2016 06:28 PM, Mark Rutland wrote:
> On Fri, Nov 25, 2016 at 05:49:45PM +0100, Christian Borntraeger wrote:
>> On 11/25/2016 05:17 PM, Peter Zijlstra wrote:
>>> On Fri, Nov 25, 2016 at 04:10:04PM +0000, Mark Rutland wrote:
>>>> On Fri, Nov 25, 2016 at 04:21:39PM +0100, Dmitry Vyukov wrote:
>>>
>>>>> What are use cases for such primitive that won't be OK with "read once
>>>>> _and_ atomically"?
>>>>
>>>> I have none to hand.
>>>
>>> Whatever triggers the __builtin_memcpy() paths, and even the size==8
>>> paths on 32bit.
>>>
>>> You could put a WARN in there to easily find them.
>>
>> There were several cases that I found during writing the *ONCE stuff.
>> For example there are some 32bit ppc variants with 64bit PTEs. Some for
>> others (I think sparc).
> 
> We have similar on 32-bit ARM w/ LPAE. LPAE implies that a naturally
> aligned 64-bit access is single-copy atomic, which is what makes that
> ok.
> 
>> And the mm/ code is perfectly fine with these PTE accesses being done
>> NOT atomic.
> 
> That strikes me as surprising. Is there some mutual exclusion that
> prevents writes from occuring wherever a READ_ONCE() happens to a PTE?

See for example mm/memory.c handle_pte_fault.

---snip----

                /*
                 * some architectures can have larger ptes than wordsize,
                 * e.g.ppc44x-defconfig has CONFIG_PTE_64BIT=y and
                 * CONFIG_32BIT=y, so READ_ONCE or ACCESS_ONCE cannot guarantee
                 * atomic accesses.  The code below just needs a consistent
                 * view for the ifs and we later double check anyway with the
                 * ptl lock held. So here a barrier will do.
                 */
---snip---

The trick is that the code only does a specific check, but all other accesses are under
the pte lock.

^ permalink raw reply

* Re: [PATCH 0/3] virtio/vringh: kill off ACCESS_ONCE()
From: Mark Rutland @ 2016-11-25 18:07 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Davidlohr Bueso, KVM list, Michael S. Tsirkin, Peter Zijlstra,
	netdev, Boqun Feng, Davidlohr Bueso, LKML, virtualization,
	Paul McKenney, Dmitry Vyukov
In-Reply-To: <CA+55aFwpUxwdgKdbwfSW59VzPCJojNg7aqBdWio5MM6RJnG5Uw@mail.gmail.com>

On Fri, Nov 25, 2016 at 09:52:50AM -0800, Linus Torvalds wrote:
> READ/WRITE_ONCE() are atomic *WHEN*THAT*IS*POSSIBLE*.

> But sometimes it's not going to be atomic.

That's the problem.

Common code may rely on something being atomic when that's only true on
a subset of platforms. On others, it's silently "fixed" into something
that isn't atomic, and we get no diagnostic. The bug lurks beneath the
surface.

Thanks,
Mark.

^ permalink raw reply

* Re: [PATCH 0/3] virtio/vringh: kill off ACCESS_ONCE()
From: Linus Torvalds @ 2016-11-25 17:52 UTC (permalink / raw)
  To: Dmitry Vyukov
  Cc: Mark Rutland, Davidlohr Bueso, KVM list, Michael S. Tsirkin,
	Peter Zijlstra, netdev, Boqun Feng, Davidlohr Bueso, LKML,
	virtualization, Paul McKenney
In-Reply-To: <CACT4Y+YJUUjU_FCR1FOcF2DmkkcjiRFpnU7Q4=yzbgvKv0VhyQ@mail.gmail.com>

On Fri, Nov 25, 2016 at 9:28 AM, Dmitry Vyukov <dvyukov@google.com> wrote:
On Fri, Nov 25, 2016 at 5:17 PM, Peter Zijlstra <peterz@infradead.org> wrote:
>>
>> IE, they are strictly stronger than {READ,WRITE}_ONCE().

No, they are strictly bullshit.

Stop this idiocy. We went through this once already.

> Uh, so, READ/WRITE_ONCE are non-atomic now. I missed that.

No.

READ/WRITE_ONCE() are atomic *WHEN*THAT*IS*POSSIBLE*. So for something
that fits in a register, it will read it in one atomic access. For
something that fits in a register and is _possible_ to write
atomically, it will do so.

But sometimes it's not going to be atomic. We do not for a moment try
to make multi-word accesses be atomic. Not even if you could try to
use some magic cmpxchg16b thing. It's not "atomic" in that sense: it
will be doing multiple accesses.

Similarly, if you try to write a 8- or 16-bit word on alpha with
WRITE_ONCE() or you try to do other things, you have what's coming to
you.

And they just force some "copy to stable storage" when it isn't (ie a
"memcpy()" is not necessarily a single access and might be done as
multiple overlapping reads, but the end result is stable).

So trying to make anything else out of them is f*cking stupid.

READ_ONCE() and friends do the right thing. Trying to limit them is
*wrong*, because the restrictions would simply make them less useful.
And trying to make up something new is pointless and stupid.

So leave this code alone. Don't add some stupid "SINGLE_LOAD()" crap.
That's just moronic. READ_ONCE() is that, and so much more.

               Linus

^ permalink raw reply

* Re: [PATCH 0/3] virtio/vringh: kill off ACCESS_ONCE()
From: Mark Rutland @ 2016-11-25 17:43 UTC (permalink / raw)
  To: Dmitry Vyukov
  Cc: Davidlohr Bueso, KVM list, Michael S. Tsirkin, Peter Zijlstra,
	netdev, Boqun Feng, dbueso, LKML, virtualization, Paul McKenney,
	Linus Torvalds
In-Reply-To: <CACT4Y+YJUUjU_FCR1FOcF2DmkkcjiRFpnU7Q4=yzbgvKv0VhyQ@mail.gmail.com>

On Fri, Nov 25, 2016 at 06:28:53PM +0100, Dmitry Vyukov wrote:
> On Fri, Nov 25, 2016 at 5:17 PM, Peter Zijlstra <peterz@infradead.org> wrote:
> >> > What are use cases for such primitive that won't be OK with "read once
> >> > _and_ atomically"?
> >>
> >> I have none to hand.
> >
> > Whatever triggers the __builtin_memcpy() paths, and even the size==8
> > paths on 32bit.
> >
> > You could put a WARN in there to easily find them.
> >
> > The advantage of introducing the SINGLE_{LOAD,STORE}() helpers is that
> > they compiletime validate this the size is 'right' and can runtime check
> > alignment constraints.
> >
> > IE, they are strictly stronger than {READ,WRITE}_ONCE().
> 
> Uh, so, READ/WRITE_ONCE are non-atomic now. I missed that.

Yes, but *only* for types larger than word size. That has *always* been
the case.

It's still assumed that *_ONCE are single-copy-atomic for word size (or
smaller). Some architectures may also provide that guarnatee for
accesses larger than word size (e.g. 32-bit ARM w/ LPAE).

... It's just that as things stand we can't put checks in *_ONCE() for
the access size, since they're *also* used for larger accesses that
don't need atomicity.

> If READ/WRITE_ONCE are non-atomic, half of kernel is broken. All these
> loads of flags, ringbuffer positions, pointers, etc are broken.

Most of these will be fine, as above.

> What about restoring READ/WRITE_ONCE as atomic, and introducing
> separate primitives for _non_ atomic loads/stores?

Having a separate *_ONCE_TEARABLE() would certainly limit the number of
things we have to fix up, and would also make it clear that atomicity is
not expected.

... but we might have to go with SINGLE_*() if we can't convince Linus.

Thanks,
Mark.

^ permalink raw reply

* Re: [PATCH 0/3] virtio/vringh: kill off ACCESS_ONCE()
From: Peter Zijlstra @ 2016-11-25 17:42 UTC (permalink / raw)
  To: Mark Rutland
  Cc: Davidlohr Bueso, KVM list, Michael S. Tsirkin, netdev, Boqun Feng,
	dbueso, LKML, virtualization, Paul McKenney, Linus Torvalds,
	Dmitry Vyukov
In-Reply-To: <20161125172624.GA30811@leverpostej>

On Fri, Nov 25, 2016 at 05:28:01PM +0000, Mark Rutland wrote:
> On Fri, Nov 25, 2016 at 05:49:45PM +0100, Christian Borntraeger wrote:
> > On 11/25/2016 05:17 PM, Peter Zijlstra wrote:

> > There were several cases that I found during writing the *ONCE stuff.
> > For example there are some 32bit ppc variants with 64bit PTEs. Some for
> > others (I think sparc).
> 
> We have similar on 32-bit ARM w/ LPAE. LPAE implies that a naturally
> aligned 64-bit access is single-copy atomic, which is what makes that
> ok.
> 
> > And the mm/ code is perfectly fine with these PTE accesses being done
> > NOT atomic.
> 
> That strikes me as surprising. Is there some mutual exclusion that
> prevents writes from occuring wherever a READ_ONCE() happens to a PTE?
> 
> Otherwise, how is tearing not a problem? Does it have some pattern like
> the lockref cmpxchg?

On x86 PAE we play silly games, see arch/x86/mm/gup.c:gup_get_ptr().

Those two loads really should be READ_ONCE()/LOAD_SINGLE().

^ permalink raw reply

* Re: [PATCH 0/3] virtio/vringh: kill off ACCESS_ONCE()
From: Dmitry Vyukov via Virtualization @ 2016-11-25 17:28 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Mark Rutland, Davidlohr Bueso, KVM list, Michael S. Tsirkin,
	netdev, Boqun Feng, dbueso, LKML, virtualization, Paul McKenney,
	Linus Torvalds
In-Reply-To: <20161125161709.GQ3092@twins.programming.kicks-ass.net>

On Fri, Nov 25, 2016 at 5:17 PM, Peter Zijlstra <peterz@infradead.org> wrote:
>> > What are use cases for such primitive that won't be OK with "read once
>> > _and_ atomically"?
>>
>> I have none to hand.
>
> Whatever triggers the __builtin_memcpy() paths, and even the size==8
> paths on 32bit.
>
> You could put a WARN in there to easily find them.
>
> The advantage of introducing the SINGLE_{LOAD,STORE}() helpers is that
> they compiletime validate this the size is 'right' and can runtime check
> alignment constraints.
>
> IE, they are strictly stronger than {READ,WRITE}_ONCE().


Uh, so, READ/WRITE_ONCE are non-atomic now. I missed that.

If READ/WRITE_ONCE are non-atomic, half of kernel is broken. All these
loads of flags, ringbuffer positions, pointers, etc are broken.

What about restoring READ/WRITE_ONCE as atomic, and introducing
separate primitives for _non_ atomic loads/stores?
It seems to me that there is just a dozen of cases that don't need
atomicity and where performance is any important (though, some of
these should probably try to write to shared memory less frequently
and save hundreds of cycles, rather than try to save few cycles on
local instructions).

I've compiled kernel with restored size checks in
READ/WRITE/ACCESS_ONCE and the following places seem to expect that
access is actually atomic (while it is not).
But if we don't guarantee that word-sized READ/WRITE_ONCE are atomic,
then I am sure we can find a hundred more of broken places.


arch/x86/entry/vdso/vdso32/../vclock_gettime.c:297:18: note: in
expansion of macro ‘ACCESS_ONCE’
  time_t result = ACCESS_ONCE(gtod->wall_time_sec);

kernel/events/ring_buffer.c:160:10: error: call to
‘__compiletime_assert_160’ declared with attribute error: Need native
word sized stores/loads for atomicity.
   tail = READ_ONCE(rb->user_page->data_tail);

kernel/events/core.c:5145:16: error: call to
‘__compiletime_assert_5145’ declared with attribute error: Need native
word sized stores/loads for atomicity.
   aux_offset = ACCESS_ONCE(rb->user_page->aux_offset);
                ^
kernel/events/core.c:5146:14: error: call to
‘__compiletime_assert_5146’ declared with attribute error: Need native
word sized stores/loads for atomicity.
   aux_size = ACCESS_ONCE(rb->user_page->aux_size);

drivers/cpufreq/cpufreq_governor.c:283:8: error: call to
‘__compiletime_assert_283’ declared with attribute error: Need native
word sized stores/loads for atomicity.
  lst = READ_ONCE(policy_dbs->last_sample_time);
        ^
drivers/cpufreq/cpufreq_governor.c:301:7: error: call to
‘__compiletime_assert_301’ declared with attribute error: Need native
word sized stores/loads for atomicity.
   if (unlikely(lst != READ_ONCE(policy_dbs->last_sample_time))) {

net/core/gen_estimator.c:136:3: error: call to
‘__compiletime_assert_136’ declared with attribute error: Need native
word sized stores/loads for atomicity.
   WRITE_ONCE(e->rate_est->bps, (e->avbps + 0xF) >> 5);
   ^
net/core/gen_estimator.c:142:3: error: call to
‘__compiletime_assert_142’ declared with attribute error: Need native
word sized stores/loads for atomicity.
   WRITE_ONCE(e->rate_est->pps, (e->avpps + 0xF) >> 5);

fs/proc_namespace.c:28:10: error: call to ‘__compiletime_assert_28’
declared with attribute error: Need native word sized stores/loads for
atomicity.
  event = ACCESS_ONCE(ns->event);

drivers/md/dm-stats.c:700:32: error: call to
‘__compiletime_assert_700’ declared with attribute error: Need native
word sized stores/loads for atomicity.
   shared->tmp.sectors[READ] += ACCESS_ONCE(p->sectors[READ]);
                                ^
drivers/md/dm-stats.c:701:33: error: call to
‘__compiletime_assert_701’ declared with attribute error: Need native
word sized stores/loads for atomicity.
   shared->tmp.sectors[WRITE] += ACCESS_ONCE(p->sectors[WRITE]);
                                 ^
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox