Linux virtualization list
 help / color / mirror / Atom feed
* Re: [PATCH 2/2] vhost: disentangle vring endianness stuff from the core code
From: Greg Kurz @ 2016-02-10 13:08 UTC (permalink / raw)
  To: Michael S. Tsirkin; +Cc: netdev, linux-kernel, kvm, virtualization
In-Reply-To: <20160210132240-mutt-send-email-mst@redhat.com>

On Wed, 10 Feb 2016 13:48:09 +0200
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Wed, Jan 13, 2016 at 06:09:47PM +0100, Greg Kurz wrote:
> > The way vring endianness is being handled currently obfuscates
> > the code in vhost_init_used().
> > 
> > This patch tries to fix that by doing the following:
> > - move the the code that adjusts endianness to a dedicated helper
> > - export this helper so that backends explicitely call it
> > 
> > No behaviour change.
> > 
> > Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > ---
> >  drivers/vhost/net.c   |    3 +++
> >  drivers/vhost/scsi.c  |    3 +++
> >  drivers/vhost/test.c  |    2 ++
> >  drivers/vhost/vhost.c |   16 +++++++++++-----
> >  drivers/vhost/vhost.h |    1 +
> >  5 files changed, 20 insertions(+), 5 deletions(-)
> > 
> > diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> > index 9eda69e40678..df01c939cd00 100644
> > --- a/drivers/vhost/net.c
> > +++ b/drivers/vhost/net.c
> > @@ -917,6 +917,9 @@ static long vhost_net_set_backend(struct vhost_net *n, unsigned index, int fd)
> >  
> >  		vhost_net_disable_vq(n, vq);
> >  		vq->private_data = sock;
> > +
> > +		vhost_adjust_vring_endian(vq);
> > +
> >  		r = vhost_init_used(vq);
> >  		if (r)
> >  			goto err_used;  
> 
> 
> This is in fact a bug in existing code: if vhost_init_used
> fails, it preferably should not have side-effects.
> It's best to update it last thing.
> 

I'm afraid we can't because the following path needs the
vring endianness:

vhost_init_used()->vhost_update_used_flags()->cpu_to_vhost16()

But you are right, there is a bug: we should rollback if vhost_init_used()
fails. Something like below:

 err_used:
        vq->private_data = oldsock;
        vhost_net_enable_vq(n, vq);
+       vhost_adjust_vring_endian(vq);
        if (ubufs)
                vhost_net_ubuf_put_wait_and_free(ubufs);
 err_ubufs:

> > diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
> > index 29cfc57d496e..5a8363bfcb74 100644
> > --- a/drivers/vhost/scsi.c
> > +++ b/drivers/vhost/scsi.c
> > @@ -1274,6 +1274,9 @@ vhost_scsi_set_endpoint(struct vhost_scsi *vs,
> >  			vq = &vs->vqs[i].vq;
> >  			mutex_lock(&vq->mutex);
> >  			vq->private_data = vs_tpg;
> > +
> > +			vhost_adjust_vring_endian(vq);
> > +
> >  			vhost_init_used(vq);
> >  			mutex_unlock(&vq->mutex);
> >  		}
> > diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c
> > index f2882ac98726..75e3e0e9f5a8 100644
> > --- a/drivers/vhost/test.c
> > +++ b/drivers/vhost/test.c
> > @@ -196,6 +196,8 @@ static long vhost_test_run(struct vhost_test *n, int test)
> >  		oldpriv = vq->private_data;
> >  		vq->private_data = priv;
> >  
> > +		vhost_adjust_vring_endian(vq);
> > +
> >  		r = vhost_init_used(&n->vqs[index]);
> >  
> >  		mutex_unlock(&vq->mutex);
> > diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
> > index e02e06755ab7..b0a00340309e 100644
> > --- a/drivers/vhost/vhost.c
> > +++ b/drivers/vhost/vhost.c
> > @@ -123,6 +123,15 @@ static void vhost_disable_is_le(struct vhost_virtqueue *vq)
> >  	vq->is_le = virtio_legacy_is_little_endian();
> >  }
> >  
> > +void vhost_adjust_vring_endian(struct vhost_virtqueue *vq)
> > +{
> > +	if (!vq->private_data)
> > +		vhost_disable_is_le(vq);
> > +	else
> > +		vhost_enable_is_le(vq);
> > +}
> > +EXPORT_SYMBOL_GPL(vhost_adjust_vring_endian);
> > +
> >  static void vhost_poll_func(struct file *file, wait_queue_head_t *wqh,
> >  			    poll_table *pt)
> >  {  
> 
> I'd prefer "vhost_update_is_le" here. "endian" might also mean
> "user_be". But see below pls.
> 

Ok, I will rename if this patch survives the review.

> 
> > @@ -1166,12 +1175,9 @@ int vhost_init_used(struct vhost_virtqueue *vq)
> >  {
> >  	__virtio16 last_used_idx;
> >  	int r;
> > -	if (!vq->private_data) {
> > -		vhost_disable_is_le(vq);
> > -		return 0;
> > -	}
> >  
> > -	vhost_enable_is_le(vq);
> > +	if (!vq->private_data)
> > +		return 0;
> >  
> >  	r = vhost_update_used_flags(vq);
> >  	if (r)  
> 
> Looking at how callers use this, maybe we should just rename init_used
> to vhost_vq_init_access. The _used suffix was a hint that we
> access the vq used ring. But maybe what callers care about is
> that it must be called after access_ok.
> 

And so we would keep the current logic where it is up to the core
code to update is_le... that is basically getting rid of this patch :)

So IIUC you're asking for:
- fix the side-effect in vhost_init_used()
- rename vhost_init_used() to vhost_vq_init_access()

> > diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
> > index d3f767448a72..88d86f45f756 100644
> > --- a/drivers/vhost/vhost.h
> > +++ b/drivers/vhost/vhost.h
> > @@ -162,6 +162,7 @@ bool vhost_enable_notify(struct vhost_dev *, struct vhost_virtqueue *);
> >  
> >  int vhost_log_write(struct vhost_virtqueue *vq, struct vhost_log *log,
> >  		    unsigned int log_num, u64 len);
> > +void vhost_adjust_vring_endian(struct vhost_virtqueue *vq);
> >  
> >  #define vq_err(vq, fmt, ...) do {                                  \
> >  		pr_debug(pr_fmt(fmt), ##__VA_ARGS__);       \  
> 

^ permalink raw reply

* Re: [PATCH 1/2] vhost: helpers to enable/disable vring endianness
From: Greg Kurz @ 2016-02-10 12:11 UTC (permalink / raw)
  To: Michael S. Tsirkin; +Cc: netdev, linux-kernel, kvm, virtualization
In-Reply-To: <20160210131723-mutt-send-email-mst@redhat.com>

On Wed, 10 Feb 2016 13:21:22 +0200
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Wed, Jan 13, 2016 at 06:09:41PM +0100, Greg Kurz wrote:
> > The default use case for vhost is when the host and the vring have the
> > same endianness (default native endianness). But there are cases where
> > they differ and vhost should byteswap when accessing the vring:
> > - the host is big endian and the vring comes from a virtio 1.0 device
> >   which is always little endian
> > - the architecture is bi-endian and the vring comes from a legacy virtio
> >   device with a different endianness than the endianness of the host (aka
> >   legacy cross-endian)
> > 
> > These cases are handled by the vq->is_le and the optional vq->user_be,
> > with the following logic:
> > - if none of the fields is enabled, vhost access the vring without byteswap
> > - if the vring is virtio 1.0 and the host is big endian, vq->is_le is
> >   enabled to enforce little endian access to the vring
> > - if the vring is legacy cross-endian, userspace enables vq->user_be
> >   to inform vhost about the vring endianness. This endianness is then
> >   enforced for vring accesses through vq->is_le again
> > 
> > The logic is unclear in the current code.
> > 
> > This patch introduces helpers with explicit enable and disable semantics,
> > for better clarity.
> > 
> > No behaviour change.
> > 
> > Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > ---
> >  drivers/vhost/vhost.c |   28 +++++++++++++++++++---------
> >  1 file changed, 19 insertions(+), 9 deletions(-)
> > 
> > diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
> > index ad2146a9ab2d..e02e06755ab7 100644
> > --- a/drivers/vhost/vhost.c
> > +++ b/drivers/vhost/vhost.c
> > @@ -43,11 +43,16 @@ enum {
> >  #define vhost_avail_event(vq) ((__virtio16 __user *)&vq->used->ring[vq->num])
> >  
> >  #ifdef CONFIG_VHOST_CROSS_ENDIAN_LEGACY
> > -static void vhost_vq_reset_user_be(struct vhost_virtqueue *vq)
> > +static void vhost_disable_user_be(struct vhost_virtqueue *vq)
> >  {
> >  	vq->user_be = !virtio_legacy_is_little_endian();
> >  }
> >    
> 
> Hmm this doesn't look like an improvement to me.
> What does it mean to disable big endian? Make it little endian?

The default behavior for the device is native endian.

The SET_VRING_ENDIAN ioctl is used to make the device big endian
on little endian hosts, hence "enabling" cross-endian mode...

> Existing reset seems to make sense.
> 

... and we "disable" cross-endian mode on reset.

> > +static void vhost_enable_user_be(struct vhost_virtqueue *vq, bool user_be)
> > +{
> > +	vq->user_be = user_be;
> > +}
> > +  
> 
> And this is maybe "init_user_be"?
> 

Anyway I don't mind changing the names to reset/init_user_be if you think it
is clearer.

> >  static long vhost_set_vring_endian(struct vhost_virtqueue *vq, int __user *argp)
> >  {
> >  	struct vhost_vring_state s;
> > @@ -62,7 +67,7 @@ static long vhost_set_vring_endian(struct vhost_virtqueue *vq, int __user *argp)
> >  	    s.num != VHOST_VRING_BIG_ENDIAN)
> >  		return -EINVAL;
> >  
> > -	vq->user_be = s.num;
> > +	vhost_enable_user_be(vq, !!s.num);
> >  
> >  	return 0;
> >  }
> > @@ -81,7 +86,7 @@ static long vhost_get_vring_endian(struct vhost_virtqueue *vq, u32 idx,
> >  	return 0;
> >  }
> >  
> > -static void vhost_init_is_le(struct vhost_virtqueue *vq)
> > +static void vhost_enable_is_le(struct vhost_virtqueue *vq)
> >  {
> >  	/* Note for legacy virtio: user_be is initialized at reset time
> >  	 * according to the host endianness. If userspace does not set an  
> 
> Same thing really. I'd rather add "reset_is_le".
> 
> > @@ -91,7 +96,7 @@ static void vhost_init_is_le(struct vhost_virtqueue *vq)
> >  	vq->is_le = vhost_has_feature(vq, VIRTIO_F_VERSION_1) || !vq->user_be;
> >  }
> >  #else
> > -static void vhost_vq_reset_user_be(struct vhost_virtqueue *vq)
> > +static void vhost_disable_user_be(struct vhost_virtqueue *vq)
> >  {
> >  }
> >  
> > @@ -106,13 +111,18 @@ static long vhost_get_vring_endian(struct vhost_virtqueue *vq, u32 idx,
> >  	return -ENOIOCTLCMD;
> >  }
> >  
> > -static void vhost_init_is_le(struct vhost_virtqueue *vq)
> > +static void vhost_enable_is_le(struct vhost_virtqueue *vq)
> >  {
> >  	if (vhost_has_feature(vq, VIRTIO_F_VERSION_1))
> >  		vq->is_le = true;
> >  }
> >  #endif /* CONFIG_VHOST_CROSS_ENDIAN_LEGACY */
> >  
> > +static void vhost_disable_is_le(struct vhost_virtqueue *vq)
> > +{
> > +	vq->is_le = virtio_legacy_is_little_endian();
> > +}
> > +
> >  static void vhost_poll_func(struct file *file, wait_queue_head_t *wqh,
> >  			    poll_table *pt)
> >  {
> > @@ -276,8 +286,8 @@ static void vhost_vq_reset(struct vhost_dev *dev,
> >  	vq->call = NULL;
> >  	vq->log_ctx = NULL;
> >  	vq->memory = NULL;
> > -	vq->is_le = virtio_legacy_is_little_endian();
> > -	vhost_vq_reset_user_be(vq);
> > +	vhost_disable_is_le(vq);
> > +	vhost_disable_user_be(vq);
> >  }
> >  
> >  static int vhost_worker(void *data)
> > @@ -1157,11 +1167,11 @@ int vhost_init_used(struct vhost_virtqueue *vq)
> >  	__virtio16 last_used_idx;
> >  	int r;
> >  	if (!vq->private_data) {
> > -		vq->is_le = virtio_legacy_is_little_endian();
> > +		vhost_disable_is_le(vq);
> >  		return 0;
> >  	}
> >  
> > -	vhost_init_is_le(vq);
> > +	vhost_enable_is_le(vq);
> >  
> >  	r = vhost_update_used_flags(vq);
> >  	if (r)  
> 

^ permalink raw reply

* Re: [PATCH 2/2] vhost: disentangle vring endianness stuff from the core code
From: Michael S. Tsirkin @ 2016-02-10 11:48 UTC (permalink / raw)
  To: Greg Kurz; +Cc: netdev, linux-kernel, kvm, virtualization
In-Reply-To: <20160113170947.23705.95216.stgit@bahia.huguette.org>

On Wed, Jan 13, 2016 at 06:09:47PM +0100, Greg Kurz wrote:
> The way vring endianness is being handled currently obfuscates
> the code in vhost_init_used().
> 
> This patch tries to fix that by doing the following:
> - move the the code that adjusts endianness to a dedicated helper
> - export this helper so that backends explicitely call it
> 
> No behaviour change.
> 
> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> ---
>  drivers/vhost/net.c   |    3 +++
>  drivers/vhost/scsi.c  |    3 +++
>  drivers/vhost/test.c  |    2 ++
>  drivers/vhost/vhost.c |   16 +++++++++++-----
>  drivers/vhost/vhost.h |    1 +
>  5 files changed, 20 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> index 9eda69e40678..df01c939cd00 100644
> --- a/drivers/vhost/net.c
> +++ b/drivers/vhost/net.c
> @@ -917,6 +917,9 @@ static long vhost_net_set_backend(struct vhost_net *n, unsigned index, int fd)
>  
>  		vhost_net_disable_vq(n, vq);
>  		vq->private_data = sock;
> +
> +		vhost_adjust_vring_endian(vq);
> +
>  		r = vhost_init_used(vq);
>  		if (r)
>  			goto err_used;


This is in fact a bug in existing code: if vhost_init_used
fails, it preferably should not have side-effects.
It's best to update it last thing.

> diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
> index 29cfc57d496e..5a8363bfcb74 100644
> --- a/drivers/vhost/scsi.c
> +++ b/drivers/vhost/scsi.c
> @@ -1274,6 +1274,9 @@ vhost_scsi_set_endpoint(struct vhost_scsi *vs,
>  			vq = &vs->vqs[i].vq;
>  			mutex_lock(&vq->mutex);
>  			vq->private_data = vs_tpg;
> +
> +			vhost_adjust_vring_endian(vq);
> +
>  			vhost_init_used(vq);
>  			mutex_unlock(&vq->mutex);
>  		}
> diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c
> index f2882ac98726..75e3e0e9f5a8 100644
> --- a/drivers/vhost/test.c
> +++ b/drivers/vhost/test.c
> @@ -196,6 +196,8 @@ static long vhost_test_run(struct vhost_test *n, int test)
>  		oldpriv = vq->private_data;
>  		vq->private_data = priv;
>  
> +		vhost_adjust_vring_endian(vq);
> +
>  		r = vhost_init_used(&n->vqs[index]);
>  
>  		mutex_unlock(&vq->mutex);
> diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
> index e02e06755ab7..b0a00340309e 100644
> --- a/drivers/vhost/vhost.c
> +++ b/drivers/vhost/vhost.c
> @@ -123,6 +123,15 @@ static void vhost_disable_is_le(struct vhost_virtqueue *vq)
>  	vq->is_le = virtio_legacy_is_little_endian();
>  }
>  
> +void vhost_adjust_vring_endian(struct vhost_virtqueue *vq)
> +{
> +	if (!vq->private_data)
> +		vhost_disable_is_le(vq);
> +	else
> +		vhost_enable_is_le(vq);
> +}
> +EXPORT_SYMBOL_GPL(vhost_adjust_vring_endian);
> +
>  static void vhost_poll_func(struct file *file, wait_queue_head_t *wqh,
>  			    poll_table *pt)
>  {

I'd prefer "vhost_update_is_le" here. "endian" might also mean
"user_be". But see below pls.


> @@ -1166,12 +1175,9 @@ int vhost_init_used(struct vhost_virtqueue *vq)
>  {
>  	__virtio16 last_used_idx;
>  	int r;
> -	if (!vq->private_data) {
> -		vhost_disable_is_le(vq);
> -		return 0;
> -	}
>  
> -	vhost_enable_is_le(vq);
> +	if (!vq->private_data)
> +		return 0;
>  
>  	r = vhost_update_used_flags(vq);
>  	if (r)

Looking at how callers use this, maybe we should just rename init_used
to vhost_vq_init_access. The _used suffix was a hint that we
access the vq used ring. But maybe what callers care about is
that it must be called after access_ok.

> diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
> index d3f767448a72..88d86f45f756 100644
> --- a/drivers/vhost/vhost.h
> +++ b/drivers/vhost/vhost.h
> @@ -162,6 +162,7 @@ bool vhost_enable_notify(struct vhost_dev *, struct vhost_virtqueue *);
>  
>  int vhost_log_write(struct vhost_virtqueue *vq, struct vhost_log *log,
>  		    unsigned int log_num, u64 len);
> +void vhost_adjust_vring_endian(struct vhost_virtqueue *vq);
>  
>  #define vq_err(vq, fmt, ...) do {                                  \
>  		pr_debug(pr_fmt(fmt), ##__VA_ARGS__);       \

^ permalink raw reply

* Re: [PATCH 1/2] vhost: helpers to enable/disable vring endianness
From: Michael S. Tsirkin @ 2016-02-10 11:21 UTC (permalink / raw)
  To: Greg Kurz; +Cc: netdev, linux-kernel, kvm, virtualization
In-Reply-To: <20160113170941.23705.93915.stgit@bahia.huguette.org>

On Wed, Jan 13, 2016 at 06:09:41PM +0100, Greg Kurz wrote:
> The default use case for vhost is when the host and the vring have the
> same endianness (default native endianness). But there are cases where
> they differ and vhost should byteswap when accessing the vring:
> - the host is big endian and the vring comes from a virtio 1.0 device
>   which is always little endian
> - the architecture is bi-endian and the vring comes from a legacy virtio
>   device with a different endianness than the endianness of the host (aka
>   legacy cross-endian)
> 
> These cases are handled by the vq->is_le and the optional vq->user_be,
> with the following logic:
> - if none of the fields is enabled, vhost access the vring without byteswap
> - if the vring is virtio 1.0 and the host is big endian, vq->is_le is
>   enabled to enforce little endian access to the vring
> - if the vring is legacy cross-endian, userspace enables vq->user_be
>   to inform vhost about the vring endianness. This endianness is then
>   enforced for vring accesses through vq->is_le again
> 
> The logic is unclear in the current code.
> 
> This patch introduces helpers with explicit enable and disable semantics,
> for better clarity.
> 
> No behaviour change.
> 
> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> ---
>  drivers/vhost/vhost.c |   28 +++++++++++++++++++---------
>  1 file changed, 19 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
> index ad2146a9ab2d..e02e06755ab7 100644
> --- a/drivers/vhost/vhost.c
> +++ b/drivers/vhost/vhost.c
> @@ -43,11 +43,16 @@ enum {
>  #define vhost_avail_event(vq) ((__virtio16 __user *)&vq->used->ring[vq->num])
>  
>  #ifdef CONFIG_VHOST_CROSS_ENDIAN_LEGACY
> -static void vhost_vq_reset_user_be(struct vhost_virtqueue *vq)
> +static void vhost_disable_user_be(struct vhost_virtqueue *vq)
>  {
>  	vq->user_be = !virtio_legacy_is_little_endian();
>  }
>  

Hmm this doesn't look like an improvement to me.
What does it mean to disable big endian? Make it little endian?
Existing reset seems to make sense.

> +static void vhost_enable_user_be(struct vhost_virtqueue *vq, bool user_be)
> +{
> +	vq->user_be = user_be;
> +}
> +

And this is maybe "init_user_be"?

>  static long vhost_set_vring_endian(struct vhost_virtqueue *vq, int __user *argp)
>  {
>  	struct vhost_vring_state s;
> @@ -62,7 +67,7 @@ static long vhost_set_vring_endian(struct vhost_virtqueue *vq, int __user *argp)
>  	    s.num != VHOST_VRING_BIG_ENDIAN)
>  		return -EINVAL;
>  
> -	vq->user_be = s.num;
> +	vhost_enable_user_be(vq, !!s.num);
>  
>  	return 0;
>  }
> @@ -81,7 +86,7 @@ static long vhost_get_vring_endian(struct vhost_virtqueue *vq, u32 idx,
>  	return 0;
>  }
>  
> -static void vhost_init_is_le(struct vhost_virtqueue *vq)
> +static void vhost_enable_is_le(struct vhost_virtqueue *vq)
>  {
>  	/* Note for legacy virtio: user_be is initialized at reset time
>  	 * according to the host endianness. If userspace does not set an

Same thing really. I'd rather add "reset_is_le".

> @@ -91,7 +96,7 @@ static void vhost_init_is_le(struct vhost_virtqueue *vq)
>  	vq->is_le = vhost_has_feature(vq, VIRTIO_F_VERSION_1) || !vq->user_be;
>  }
>  #else
> -static void vhost_vq_reset_user_be(struct vhost_virtqueue *vq)
> +static void vhost_disable_user_be(struct vhost_virtqueue *vq)
>  {
>  }
>  
> @@ -106,13 +111,18 @@ static long vhost_get_vring_endian(struct vhost_virtqueue *vq, u32 idx,
>  	return -ENOIOCTLCMD;
>  }
>  
> -static void vhost_init_is_le(struct vhost_virtqueue *vq)
> +static void vhost_enable_is_le(struct vhost_virtqueue *vq)
>  {
>  	if (vhost_has_feature(vq, VIRTIO_F_VERSION_1))
>  		vq->is_le = true;
>  }
>  #endif /* CONFIG_VHOST_CROSS_ENDIAN_LEGACY */
>  
> +static void vhost_disable_is_le(struct vhost_virtqueue *vq)
> +{
> +	vq->is_le = virtio_legacy_is_little_endian();
> +}
> +
>  static void vhost_poll_func(struct file *file, wait_queue_head_t *wqh,
>  			    poll_table *pt)
>  {
> @@ -276,8 +286,8 @@ static void vhost_vq_reset(struct vhost_dev *dev,
>  	vq->call = NULL;
>  	vq->log_ctx = NULL;
>  	vq->memory = NULL;
> -	vq->is_le = virtio_legacy_is_little_endian();
> -	vhost_vq_reset_user_be(vq);
> +	vhost_disable_is_le(vq);
> +	vhost_disable_user_be(vq);
>  }
>  
>  static int vhost_worker(void *data)
> @@ -1157,11 +1167,11 @@ int vhost_init_used(struct vhost_virtqueue *vq)
>  	__virtio16 last_used_idx;
>  	int r;
>  	if (!vq->private_data) {
> -		vq->is_le = virtio_legacy_is_little_endian();
> +		vhost_disable_is_le(vq);
>  		return 0;
>  	}
>  
> -	vhost_init_is_le(vq);
> +	vhost_enable_is_le(vq);
>  
>  	r = vhost_update_used_flags(vq);
>  	if (r)

^ permalink raw reply

* CFP: CISTI'2016 - 11th Iberian Conference on IST; Deadline: February 21
From: Maria Lemos @ 2016-02-09 17:42 UTC (permalink / raw)
  To: virtualization

[-- Attachment #1: Type: text/plain, Size: 4880 bytes --]

--
--------------------------------                             
CISTI'2016 - 11th Iberian Conference on Information Systems and Technologies
           June 15 - 18, 2016, Gran Canaria, Canary Islands, Spain
               http://www.aisti.eu/cisti2016/index.php/en
------------------------------------------------------------------------------------

We are pleased to invite the academic and business community to submit their papers to CISTI'2016 - 11th Iberian Conference on Information Systems and Technologies, to be held in Gran Canaria, Canary Islands, Spain, between the 15th and 18th of June 2016. Authors are encouraged to submit original scientific contributions such as state-of-art reviews and new research perspectives, groundbreaking ideas and/or architectures, solutions and/or applications for real problems, empirical and/or evaluation works, case studies, etc., in conformity with the themes of this Conference.

Four types of papers can be submitted:

Full paper: Finished or consolidated R&D works, to be included in one of the Conference themes. These papers are assigned  a 6-page limit.

Short paper: Ongoing works with relevant preliminary results, opened to discussion. These papers are assigned a 4-page limit.

Poster paper: Initial work with relevant ideas, opened to discussion. These papers are assigned a 2-page limit.

Company paper: Companies' papers  that show practical experience, R & D, tools, etc., focused in some topics of the conference. These articles are abstracts with a maximum of 2 pages.

Papers submitted for the Scientific Committee’s evaluation must not include any information leading to the authors’ identification. Therefore, the authors’ names, affiliations and bibliographic references should not be included in the early version. This information should only be included in the final version.

Submitted papers must not have been published and must not be under review for any other conference and national or international publication.

Papers must comply with the format standard and be written in Portuguese, Spanish or English.

All papers will be subjected to a “blind review” by at least two members of the Scientific Committee.

Full papers can be accepted as short papers or poster papers only. Similarly, short papers can be accepted as poster papers only. In these two cases, the authors will be allowed to maintain the original number of pages in the proceedings publication.

The authors of accepted poster papers must also build and print a poster to be exhibited during the Conference. This poster must follow an A1 or A2 vertical format. The Conference includes Work Sessions where these posters are presented and orally discussed, with a 5-minute limit per poster.

The authors of accepted full papers will dispose of a 15-minute presentation in the Conference Work Session, and approximately 5 minutes of discussion will follow each presentation. The authors of accepted short papers and company papers will dispose of an 11-minute presentation in the Conference Work Session, and approximately 4 minutes of discussion will follow each presentation.


THEMES

Submitted papers must follow the main themes proposed for the Conference (the topics proposed in each theme constitute a mere framework reference; they are not intended as restrictive):

A) OMIS - Organizational Models and Information Systems

B) KMDSS - Knowledge Management and Decision Support Systems

C) SSAAT - Software Systems, Architectures, Applications and Tools

D) CNMPS - Computer Networks, Mobility and Pervasive Systems

E) HCC - Human Centered Computing

F) HIS - Health Informatics

G) ITE - Information Technologies in Education

H) AEC – Architecture and Engineering of Construction


PUBLICATION & INDEXING

To ensure that the contribution (full paper, short paper, poster paper or company paper) is published in the Proceedings, at least one of the authors must be fully registered by the 11th of April, and the paper must comply with the suggested layout and page-limit. Additionally, all recommended modifications must be addressed by the authors before they submit the final version.

No more than one paper per registration will be published in the Conference Proceedings. An extra fee must be paid for publication of additional papers, with a maximum of one additional paper per registration.

Published full and short papers will be sent to EI, IEEE XPlore, INSPEC, ISI, SCOPUS and Google Scholar.


IMPORTANT DATES

Paper submission: February 21, 2016

Notification of acceptance: March 27, 2016

Submission of accepted papers: April 10, 2016

Payment of registration, to ensure the inclusion of an accepted paper in the conference proceedings: April 8, 2016


Best regards,

CISTI'2016 Team
http://www.aisti.eu/cisti2016/index.php/en


[-- Attachment #2: Type: text/plain, Size: 183 bytes --]

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

^ permalink raw reply

* Re: [PATCH 6/6] VMware balloon: Update vmw_balloon.c to use the VMW_PORT macro
From: Greg KH @ 2016-02-08 19:41 UTC (permalink / raw)
  To: Sinclair Yeh
  Cc: pv-drivers, Xavier Deguillard, x86, linux-kernel, virtualization
In-Reply-To: <1453239965-1466-7-git-send-email-syeh@vmware.com>

On Tue, Jan 19, 2016 at 01:46:05PM -0800, Sinclair Yeh wrote:
> Updated VMWARE_BALLOON_CMD to use the common VMW_PORT macro.
> Doing this rather than replacing all instances of VMWARE_BALLOON_CMD
> to minimize code change.
> 
> Signed-off-by: Sinclair Yeh <syeh@vmware.com>
> Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
> Reviewed-by: Alok N Kataria <akataria@vmware.com>
> Acked-by: Xavier Deguillard <xdeguillard@vmware.com>
> Cc: pv-drivers@vmware.com
> Cc: Xavier Deguillard <xdeguillard@vmware.com>
> Cc: linux-kernel@vger.kernel.org
> Cc: virtualization@lists.linux-foundation.org
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>


Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

> -#define VMWARE_BALLOON_CMD(cmd, arg1, arg2, result)		\
> -({								\
> -	unsigned long __status, __dummy1, __dummy2, __dummy3;	\
> -	__asm__ __volatile__ ("inl %%dx" :			\
> -		"=a"(__status),					\
> -		"=c"(__dummy1),					\
> -		"=d"(__dummy2),					\
> -		"=b"(result),					\
> -		"=S" (__dummy3) :				\
> -		"0"(VMW_BALLOON_HV_MAGIC),			\
> -		"1"(VMW_BALLOON_CMD_##cmd),			\
> -		"2"(VMW_BALLOON_HV_PORT),			\
> -		"3"(arg1),					\
> -		"4" (arg2) :					\
> -		"memory");					\
> -	if (VMW_BALLOON_CMD_##cmd == VMW_BALLOON_CMD_START)	\
> -		result = __dummy1;				\
> -	result &= -1UL;						\
> -	__status & -1UL;					\
> +#define VMWARE_BALLOON_CMD(cmd, arg1, arg2, result)			\
> +({									\
> +	unsigned long __status, __dummy1, __dummy2;			\
> +	unsigned long __si, __di;					\
> +	VMW_PORT(VMW_BALLOON_CMD_##cmd, arg1, arg2, 0,			\
> +		 VMW_BALLOON_HV_PORT, VMW_BALLOON_HV_MAGIC,		\
> +		 __status, result, __dummy1, __dummy2, __si, __di);	\
> +	if (VMW_BALLOON_CMD_##cmd == VMW_BALLOON_CMD_START)		\
> +		result = __dummy1;					\
> +	result &= -1UL;							\
> +	__status & -1UL;						\
>  })

Honestly, it doesn't look anymore "readable" to me, but it's your code
to maintain, not mine... :)

^ permalink raw reply

* Re: [PATCH v7 9/9] vring: Use the DMA API on Xen
From: Andy Lutomirski @ 2016-02-04 17:49 UTC (permalink / raw)
  To: David Vrabel
  Cc: linux-s390, Joerg Roedel, KVM, Michael S. Tsirkin,
	Benjamin Herrenschmidt, Stefano Stabellini, Sebastian Ott,
	linux-kernel@vger.kernel.org, Christoph Hellwig,
	Christian Borntraeger, Andy Lutomirski,
	xen-devel@lists.xenproject.org, sparclinux, Paolo Bonzini,
	Linux Virtualization, David Woodhouse, David S. Miller,
	Martin Schwidefsky
In-Reply-To: <56B1CD18.1050509@citrix.com>

On Wed, Feb 3, 2016 at 1:49 AM, David Vrabel <david.vrabel@citrix.com> wrote:
> On 03/02/16 05:46, Andy Lutomirski wrote:
>> Signed-off-by: Andy Lutomirski <luto@kernel.org>
>
> You forgot the previous Reviewed-by tags.

Whoops.  If I send another version, they'll be there.

>
> David



-- 
Andy Lutomirski
AMA Capital Management, LLC

^ permalink raw reply

* Re: [PATCH net-next] virtio_net: add ethtool support for set and get of settings
From: Stephen Hemminger @ 2016-02-03 23:53 UTC (permalink / raw)
  To: Nikolay Aleksandrov
  Cc: Michael S. Tsirkin, Nikolay Aleksandrov, netdev, Roopa Prabhu,
	virtualization, davem
In-Reply-To: <1454417480-28672-1-git-send-email-razor@blackwall.org>

On Tue,  2 Feb 2016 13:51:20 +0100
Nikolay Aleksandrov <razor@blackwall.org> wrote:

> +static bool virtnet_validate_speed(u32 speed)
> +{
> +	switch (speed) {
> +	case SPEED_10:
> +	case SPEED_100:
> +	case SPEED_1000:
> +	case SPEED_2500:
> +	case SPEED_5000:
> +	case SPEED_10000:
> +	case SPEED_20000:
> +	case SPEED_25000:
> +	case SPEED_40000:
> +	case SPEED_50000:
> +	case SPEED_56000:
> +	case SPEED_100000:
> +	case SPEED_UNKNOWN:
> +		return true;
> +	}
> +
> +	return false;
> +}

Why limit to only known values. This switch() will get out of
date when some vendor introduces 64G or some other weird value.

^ permalink raw reply

* Re: [PATCH v7 5/9] virtio_ring: Support DMA APIs
From: Andy Lutomirski @ 2016-02-03 17:53 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: linux-s390, sparclinux, KVM, Stefano Stabellini,
	Benjamin Herrenschmidt, Sebastian Ott,
	linux-kernel@vger.kernel.org, Christoph Hellwig,
	Christian Borntraeger, Joerg Roedel, David Vrabel, Paolo Bonzini,
	Martin Schwidefsky, xen-devel@lists.xenproject.org,
	Linux Virtualization, David Woodhouse, David S. Miller
In-Reply-To: <20160203154633-mutt-send-email-mst@redhat.com>

On Feb 3, 2016 5:52 AM, "Michael S. Tsirkin" <mst@redhat.com> wrote:
>
> On Tue, Feb 02, 2016 at 09:46:36PM -0800, Andy Lutomirski wrote:
> > virtio_ring currently sends the device (usually a hypervisor)
> > physical addresses of its I/O buffers.  This is okay when DMA
> > addresses and physical addresses are the same thing, but this isn't
> > always the case.  For example, this never works on Xen guests, and
> > it is likely to fail if a physical "virtio" device ever ends up
> > behind an IOMMU or swiotlb.
> >
> > The immediate use case for me is to enable virtio on Xen guests.
> > For that to work, we need vring to support DMA address translation
> > as well as a corresponding change to virtio_pci or to another
> > driver.
> >
> > Signed-off-by: Andy Lutomirski <luto@kernel.org>
> > ---
> >  drivers/virtio/Kconfig           |   2 +-
> >  drivers/virtio/virtio_ring.c     | 200 ++++++++++++++++++++++++++++++++-------
> >  tools/virtio/linux/dma-mapping.h |  17 ++++
> >  3 files changed, 183 insertions(+), 36 deletions(-)
> >  create mode 100644 tools/virtio/linux/dma-mapping.h
> >
> > diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig
> > index cab9f3f63a38..77590320d44c 100644
> > --- a/drivers/virtio/Kconfig
> > +++ b/drivers/virtio/Kconfig
> > @@ -60,7 +60,7 @@ config VIRTIO_INPUT
> >
> >   config VIRTIO_MMIO
> >       tristate "Platform bus driver for memory mapped virtio devices"
> > -     depends on HAS_IOMEM
> > +     depends on HAS_IOMEM && HAS_DMA
> >       select VIRTIO
> >       ---help---
> >        This drivers provides support for memory mapped virtio
>
> What's this chunk doing here btw? Should be part of the mmio patch?
>

IIRC it was deliberate.  Making virtio depend on HAS_DMA didn't work
right because kconfig doesn't propagate dependencies through select
intelligently.  This patch makes core virtio depend on HAS_DMA, so I
added the dependency here, too.

--Andy

^ permalink raw reply

* Re: [PATCH v7 0/9] virtio DMA API, yet again
From: Stefano Stabellini @ 2016-02-03 17:52 UTC (permalink / raw)
  To: Andy Lutomirski
  Cc: linux-s390, Joerg Roedel, KVM, Michael S. Tsirkin,
	Benjamin Herrenschmidt, Stefano Stabellini, Sebastian Ott,
	linux-kernel@vger.kernel.org, Christoph Hellwig,
	Christian Borntraeger, David Vrabel, xen-devel, sparclinux,
	Paolo Bonzini, Linux Virtualization, David Woodhouse,
	David S. Miller, Martin Schwidefsky
In-Reply-To: <cover.1454477782.git.luto@kernel.org>

On Tue, 2 Feb 2016, Andy Lutomirski wrote:
> This switches virtio to use the DMA API on Xen and if requested by
> module option.
> 
> This fixes virtio on Xen, and it should break anything because it's
> off by default on everything except Xen PV on x86.
> 
> To the Xen people: is this okay?  If it doesn't work on other Xen
> variants (PVH? HVM?), can you submit follow-up patches to fix it?

I have been waiting for something like this for a long time: up to now
it wasn't possible to use Xen inside a VM with virtio devices.

You can add my:

Tested-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>


> To everyone else: we've waffled on this for way too long.  I think
> we should to get DMA API implementation in with a conservative
> policy like this rather than waiting until we achieve perfection.
> I'm tired of carrying these patches around.
> 
> I changed queue allocation around a bit in this version.  Per Michael's
> request, we no longer use dma_zalloc_coherent in the !dma_api case.
> Instead we use alloc_pages_exact, just like the current code does.
> This simplifies the ring address accessors, because they can always
> load from the dma addr rather than depending on vring_use_dma_api
> themselves.
> 
> There's an odd warning in here if the ring's physical address
> doesn't fit in a dma_addr_t.  This could only possible happen on
> really weird configurations in which phys_addr_t is wider than
> dma_addr_t.  AFAICT this is only possible on i386 PAE systems and on
> MIPS, and even there it only happens if highmem is off.  But that
> means we're safe, since we should never end up with high allocations
> on non-highmem systems unless we explicitly ask for them, which we
> don't.
> 
> If this is too scary, I can add yet more cruft to avoid it, but
> it seems harmless enough to me, and it means that the driver will
> be totally clean once all the vring_use_dma_api calls go away.
> 
> Michael, if these survive review, can you stage these in your tree?
> Can you also take a look at tools/virtio?  I probably broke it, but I
> couldn't get it to build without these patches either, so I'm stuck.
> 
> Changes from v6:
>  - Remove HAVE_DMA_ATTRS and add Acked-by (Cornelia)
>  - Add some missing signed-off-by lines from me (whoops)
>  - Rework queue allocation (Michael)
> 
> Changes from v5:
>  - Typo fixes (David Woodhouse)
>  - Use xen_domain() to detect Xen (David Vrabel)
>  - Pass struct vring_virtqueue * into vring_use_dma_api for future proofing
>  - Removed module parameter (Michael)
> 
> Changes from v4:
>  - Bake vring_use_dma_api in from the beginning.
>  - Automatically enable only on Xen.
>  - Add module parameter.
>  - Add s390 and alpha DMA API implementations.
>  - Rebase to 4.5-rc1.
> 
> Changes from v3:
>  - More big-endian fixes.
>  - Added better virtio-ring APIs that handle allocation and use them in
>    virtio-mmio and virtio-pci.
>  - Switch to Michael's virtio-net patch.
> 
> Changes from v2:
>  - Fix vring_mapping_error incorrect argument
> 
> Changes from v1:
>  - Fix an endian conversion error causing a BUG to hit.
>  - Fix a DMA ordering issue (swiotlb=force works now).
>  - Minor cleanups.
> 
> Andy Lutomirski (6):
>   vring: Introduce vring_use_dma_api()
>   virtio_ring: Support DMA APIs
>   virtio: Add improved queue allocation API
>   virtio_mmio: Use the DMA API if enabled
>   virtio_pci: Use the DMA API if enabled
>   vring: Use the DMA API on Xen
> 
> Christian Borntraeger (3):
>   dma: Provide simple noop dma ops
>   alpha/dma: use common noop dma ops
>   s390/dma: Allow per device dma ops
> 
>  arch/alpha/kernel/pci-noop.c        |  46 +---
>  arch/s390/Kconfig                   |   5 +-
>  arch/s390/include/asm/device.h      |   6 +-
>  arch/s390/include/asm/dma-mapping.h |   6 +-
>  arch/s390/pci/pci.c                 |   1 +
>  arch/s390/pci/pci_dma.c             |   4 +-
>  drivers/virtio/Kconfig              |   2 +-
>  drivers/virtio/virtio_mmio.c        |  67 ++----
>  drivers/virtio/virtio_pci_common.h  |   6 -
>  drivers/virtio/virtio_pci_legacy.c  |  42 ++--
>  drivers/virtio/virtio_pci_modern.c  |  61 ++---
>  drivers/virtio/virtio_ring.c        | 439 +++++++++++++++++++++++++++++++-----
>  include/linux/dma-mapping.h         |   2 +
>  include/linux/virtio.h              |  23 +-
>  include/linux/virtio_ring.h         |  35 +++
>  lib/Makefile                        |   1 +
>  lib/dma-noop.c                      |  75 ++++++
>  tools/virtio/linux/dma-mapping.h    |  17 ++
>  18 files changed, 594 insertions(+), 244 deletions(-)
>  create mode 100644 lib/dma-noop.c
>  create mode 100644 tools/virtio/linux/dma-mapping.h
> 
> -- 
> 2.5.0
> 

^ permalink raw reply

* Re: [PATCH v7 5/9] virtio_ring: Support DMA APIs
From: Michael S. Tsirkin @ 2016-02-03 13:52 UTC (permalink / raw)
  To: Andy Lutomirski
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, David Vrabel, xen-devel,
	sparclinux, Paolo Bonzini, Linux Virtualization, David Woodhouse,
	David S. Miller, Martin Schwidefsky
In-Reply-To: <a6c1a2d0b3ea60bfb27c810896179bd553e7e240.1454477782.git.luto@kernel.org>

On Tue, Feb 02, 2016 at 09:46:36PM -0800, Andy Lutomirski wrote:
> virtio_ring currently sends the device (usually a hypervisor)
> physical addresses of its I/O buffers.  This is okay when DMA
> addresses and physical addresses are the same thing, but this isn't
> always the case.  For example, this never works on Xen guests, and
> it is likely to fail if a physical "virtio" device ever ends up
> behind an IOMMU or swiotlb.
> 
> The immediate use case for me is to enable virtio on Xen guests.
> For that to work, we need vring to support DMA address translation
> as well as a corresponding change to virtio_pci or to another
> driver.
> 
> Signed-off-by: Andy Lutomirski <luto@kernel.org>
> ---
>  drivers/virtio/Kconfig           |   2 +-
>  drivers/virtio/virtio_ring.c     | 200 ++++++++++++++++++++++++++++++++-------
>  tools/virtio/linux/dma-mapping.h |  17 ++++
>  3 files changed, 183 insertions(+), 36 deletions(-)
>  create mode 100644 tools/virtio/linux/dma-mapping.h
> 
> diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig
> index cab9f3f63a38..77590320d44c 100644
> --- a/drivers/virtio/Kconfig
> +++ b/drivers/virtio/Kconfig
> @@ -60,7 +60,7 @@ config VIRTIO_INPUT
>  
>   config VIRTIO_MMIO
>  	tristate "Platform bus driver for memory mapped virtio devices"
> -	depends on HAS_IOMEM
> +	depends on HAS_IOMEM && HAS_DMA
>   	select VIRTIO
>   	---help---
>   	 This drivers provides support for memory mapped virtio

What's this chunk doing here btw? Should be part of the mmio patch?

> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
> index ab0be6c084f6..9abc008ff7ea 100644
> --- a/drivers/virtio/virtio_ring.c
> +++ b/drivers/virtio/virtio_ring.c
> @@ -24,6 +24,7 @@
>  #include <linux/module.h>
>  #include <linux/hrtimer.h>
>  #include <linux/kmemleak.h>
> +#include <linux/dma-mapping.h>
>  
>  #ifdef DEBUG
>  /* For development, we want to crash whenever the ring is screwed. */
> @@ -54,6 +55,11 @@
>  #define END_USE(vq)
>  #endif
>  
> +struct vring_desc_state {
> +	void *data;			/* Data for callback. */
> +	struct vring_desc *indir_desc;	/* Indirect descriptor, if any. */
> +};
> +
>  struct vring_virtqueue {
>  	struct virtqueue vq;
>  
> @@ -98,8 +104,8 @@ struct vring_virtqueue {
>  	ktime_t last_add_time;
>  #endif
>  
> -	/* Tokens for callbacks. */
> -	void *data[];
> +	/* Per-descriptor state. */
> +	struct vring_desc_state desc_state[];
>  };
>  
>  #define to_vvq(_vq) container_of(_vq, struct vring_virtqueue, vq)
> @@ -128,6 +134,79 @@ static bool vring_use_dma_api(struct virtio_device *vdev)
>  	return false;
>  }
>  
> +/*
> + * The DMA ops on various arches are rather gnarly right now, and
> + * making all of the arch DMA ops work on the vring device itself
> + * is a mess.  For now, we use the parent device for DMA ops.
> + */
> +struct device *vring_dma_dev(const struct vring_virtqueue *vq)
> +{
> +	return vq->vq.vdev->dev.parent;
> +}
> +
> +/* Map one sg entry. */
> +static dma_addr_t vring_map_one_sg(const struct vring_virtqueue *vq,
> +				   struct scatterlist *sg,
> +				   enum dma_data_direction direction)
> +{
> +	if (!vring_use_dma_api(vq->vq.vdev))
> +		return (dma_addr_t)sg_phys(sg);
> +
> +	/*
> +	 * We can't use dma_map_sg, because we don't use scatterlists in
> +	 * the way it expects (we don't guarantee that the scatterlist
> +	 * will exist for the lifetime of the mapping).
> +	 */
> +	return dma_map_page(vring_dma_dev(vq),
> +			    sg_page(sg), sg->offset, sg->length,
> +			    direction);
> +}
> +
> +static dma_addr_t vring_map_single(const struct vring_virtqueue *vq,
> +				   void *cpu_addr, size_t size,
> +				   enum dma_data_direction direction)
> +{
> +	if (!vring_use_dma_api(vq->vq.vdev))
> +		return (dma_addr_t)virt_to_phys(cpu_addr);
> +
> +	return dma_map_single(vring_dma_dev(vq),
> +			      cpu_addr, size, direction);
> +}
> +
> +static void vring_unmap_one(const struct vring_virtqueue *vq,
> +			    struct vring_desc *desc)
> +{
> +	u16 flags;
> +
> +	if (!vring_use_dma_api(vq->vq.vdev))
> +		return;
> +
> +	flags = virtio16_to_cpu(vq->vq.vdev, desc->flags);
> +
> +	if (flags & VRING_DESC_F_INDIRECT) {
> +		dma_unmap_single(vring_dma_dev(vq),
> +				 virtio64_to_cpu(vq->vq.vdev, desc->addr),
> +				 virtio32_to_cpu(vq->vq.vdev, desc->len),
> +				 (flags & VRING_DESC_F_WRITE) ?
> +				 DMA_FROM_DEVICE : DMA_TO_DEVICE);
> +	} else {
> +		dma_unmap_page(vring_dma_dev(vq),
> +			       virtio64_to_cpu(vq->vq.vdev, desc->addr),
> +			       virtio32_to_cpu(vq->vq.vdev, desc->len),
> +			       (flags & VRING_DESC_F_WRITE) ?
> +			       DMA_FROM_DEVICE : DMA_TO_DEVICE);
> +	}
> +}
> +
> +static int vring_mapping_error(const struct vring_virtqueue *vq,
> +			       dma_addr_t addr)
> +{
> +	if (!vring_use_dma_api(vq->vq.vdev))
> +		return 0;
> +
> +	return dma_mapping_error(vring_dma_dev(vq), addr);
> +}
> +
>  static struct vring_desc *alloc_indirect(struct virtqueue *_vq,
>  					 unsigned int total_sg, gfp_t gfp)
>  {
> @@ -161,7 +240,7 @@ static inline int virtqueue_add(struct virtqueue *_vq,
>  	struct vring_virtqueue *vq = to_vvq(_vq);
>  	struct scatterlist *sg;
>  	struct vring_desc *desc;
> -	unsigned int i, n, avail, descs_used, uninitialized_var(prev);
> +	unsigned int i, n, avail, descs_used, uninitialized_var(prev), err_idx;
>  	int head;
>  	bool indirect;
>  
> @@ -201,21 +280,15 @@ static inline int virtqueue_add(struct virtqueue *_vq,
>  
>  	if (desc) {
>  		/* Use a single buffer which doesn't continue */
> -		vq->vring.desc[head].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_INDIRECT);
> -		vq->vring.desc[head].addr = cpu_to_virtio64(_vq->vdev, virt_to_phys(desc));
> -		/* avoid kmemleak false positive (hidden by virt_to_phys) */
> -		kmemleak_ignore(desc);
> -		vq->vring.desc[head].len = cpu_to_virtio32(_vq->vdev, total_sg * sizeof(struct vring_desc));
> -
> +		indirect = true;
>  		/* Set up rest to use this indirect table. */
>  		i = 0;
>  		descs_used = 1;
> -		indirect = true;
>  	} else {
> +		indirect = false;
>  		desc = vq->vring.desc;
>  		i = head;
>  		descs_used = total_sg;
> -		indirect = false;
>  	}
>  
>  	if (vq->vq.num_free < descs_used) {
> @@ -230,13 +303,14 @@ static inline int virtqueue_add(struct virtqueue *_vq,
>  		return -ENOSPC;
>  	}
>  
> -	/* We're about to use some buffers from the free list. */
> -	vq->vq.num_free -= descs_used;
> -
>  	for (n = 0; n < out_sgs; n++) {
>  		for (sg = sgs[n]; sg; sg = sg_next(sg)) {
> +			dma_addr_t addr = vring_map_one_sg(vq, sg, DMA_TO_DEVICE);
> +			if (vring_mapping_error(vq, addr))
> +				goto unmap_release;
> +
>  			desc[i].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_NEXT);
> -			desc[i].addr = cpu_to_virtio64(_vq->vdev, sg_phys(sg));
> +			desc[i].addr = cpu_to_virtio64(_vq->vdev, addr);
>  			desc[i].len = cpu_to_virtio32(_vq->vdev, sg->length);
>  			prev = i;
>  			i = virtio16_to_cpu(_vq->vdev, desc[i].next);
> @@ -244,8 +318,12 @@ static inline int virtqueue_add(struct virtqueue *_vq,
>  	}
>  	for (; n < (out_sgs + in_sgs); n++) {
>  		for (sg = sgs[n]; sg; sg = sg_next(sg)) {
> +			dma_addr_t addr = vring_map_one_sg(vq, sg, DMA_FROM_DEVICE);
> +			if (vring_mapping_error(vq, addr))
> +				goto unmap_release;
> +
>  			desc[i].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_NEXT | VRING_DESC_F_WRITE);
> -			desc[i].addr = cpu_to_virtio64(_vq->vdev, sg_phys(sg));
> +			desc[i].addr = cpu_to_virtio64(_vq->vdev, addr);
>  			desc[i].len = cpu_to_virtio32(_vq->vdev, sg->length);
>  			prev = i;
>  			i = virtio16_to_cpu(_vq->vdev, desc[i].next);
> @@ -254,14 +332,33 @@ static inline int virtqueue_add(struct virtqueue *_vq,
>  	/* Last one doesn't continue. */
>  	desc[prev].flags &= cpu_to_virtio16(_vq->vdev, ~VRING_DESC_F_NEXT);
>  
> +	if (indirect) {
> +		/* Now that the indirect table is filled in, map it. */
> +		dma_addr_t addr = vring_map_single(
> +			vq, desc, total_sg * sizeof(struct vring_desc),
> +			DMA_TO_DEVICE);
> +		if (vring_mapping_error(vq, addr))
> +			goto unmap_release;
> +
> +		vq->vring.desc[head].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_INDIRECT);
> +		vq->vring.desc[head].addr = cpu_to_virtio64(_vq->vdev, addr);
> +
> +		vq->vring.desc[head].len = cpu_to_virtio32(_vq->vdev, total_sg * sizeof(struct vring_desc));
> +	}
> +
> +	/* We're using some buffers from the free list. */
> +	vq->vq.num_free -= descs_used;
> +
>  	/* Update free pointer */
>  	if (indirect)
>  		vq->free_head = virtio16_to_cpu(_vq->vdev, vq->vring.desc[head].next);
>  	else
>  		vq->free_head = i;
>  
> -	/* Set token. */
> -	vq->data[head] = data;
> +	/* Store token and indirect buffer state. */
> +	vq->desc_state[head].data = data;
> +	if (indirect)
> +		vq->desc_state[head].indir_desc = desc;
>  
>  	/* Put entry in available array (but don't update avail->idx until they
>  	 * do sync). */
> @@ -284,6 +381,24 @@ static inline int virtqueue_add(struct virtqueue *_vq,
>  		virtqueue_kick(_vq);
>  
>  	return 0;
> +
> +unmap_release:
> +	err_idx = i;
> +	i = head;
> +
> +	for (n = 0; n < total_sg; n++) {
> +		if (i == err_idx)
> +			break;
> +		vring_unmap_one(vq, &desc[i]);
> +		i = vq->vring.desc[i].next;
> +	}
> +
> +	vq->vq.num_free += total_sg;
> +
> +	if (indirect)
> +		kfree(desc);
> +
> +	return -EIO;
>  }
>  
>  /**
> @@ -454,27 +569,43 @@ EXPORT_SYMBOL_GPL(virtqueue_kick);
>  
>  static void detach_buf(struct vring_virtqueue *vq, unsigned int head)
>  {
> -	unsigned int i;
> +	unsigned int i, j;
> +	u16 nextflag = cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_NEXT);
>  
>  	/* Clear data ptr. */
> -	vq->data[head] = NULL;
> +	vq->desc_state[head].data = NULL;
>  
> -	/* Put back on free list: find end */
> +	/* Put back on free list: unmap first-level descriptors and find end */
>  	i = head;
>  
> -	/* Free the indirect table */
> -	if (vq->vring.desc[i].flags & cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_INDIRECT))
> -		kfree(phys_to_virt(virtio64_to_cpu(vq->vq.vdev, vq->vring.desc[i].addr)));
> -
> -	while (vq->vring.desc[i].flags & cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_NEXT)) {
> +	while (vq->vring.desc[i].flags & nextflag) {
> +		vring_unmap_one(vq, &vq->vring.desc[i]);
>  		i = virtio16_to_cpu(vq->vq.vdev, vq->vring.desc[i].next);
>  		vq->vq.num_free++;
>  	}
>  
> +	vring_unmap_one(vq, &vq->vring.desc[i]);
>  	vq->vring.desc[i].next = cpu_to_virtio16(vq->vq.vdev, vq->free_head);
>  	vq->free_head = head;
> +
>  	/* Plus final descriptor */
>  	vq->vq.num_free++;
> +
> +	/* Free the indirect table, if any, now that it's unmapped. */
> +	if (vq->desc_state[head].indir_desc) {
> +		struct vring_desc *indir_desc = vq->desc_state[head].indir_desc;
> +		u32 len = virtio32_to_cpu(vq->vq.vdev, vq->vring.desc[head].len);
> +
> +		BUG_ON(!(vq->vring.desc[head].flags &
> +			 cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_INDIRECT)));
> +		BUG_ON(len == 0 || len % sizeof(struct vring_desc));
> +
> +		for (j = 0; j < len / sizeof(struct vring_desc); j++)
> +			vring_unmap_one(vq, &indir_desc[j]);
> +
> +		kfree(vq->desc_state[head].indir_desc);
> +		vq->desc_state[head].indir_desc = NULL;
> +	}
>  }
>  
>  static inline bool more_used(const struct vring_virtqueue *vq)
> @@ -529,13 +660,13 @@ void *virtqueue_get_buf(struct virtqueue *_vq, unsigned int *len)
>  		BAD_RING(vq, "id %u out of range\n", i);
>  		return NULL;
>  	}
> -	if (unlikely(!vq->data[i])) {
> +	if (unlikely(!vq->desc_state[i].data)) {
>  		BAD_RING(vq, "id %u is not a head!\n", i);
>  		return NULL;
>  	}
>  
>  	/* detach_buf clears data, so grab it now. */
> -	ret = vq->data[i];
> +	ret = vq->desc_state[i].data;
>  	detach_buf(vq, i);
>  	vq->last_used_idx++;
>  	/* If we expect an interrupt for the next entry, tell host
> @@ -709,10 +840,10 @@ void *virtqueue_detach_unused_buf(struct virtqueue *_vq)
>  	START_USE(vq);
>  
>  	for (i = 0; i < vq->vring.num; i++) {
> -		if (!vq->data[i])
> +		if (!vq->desc_state[i].data)
>  			continue;
>  		/* detach_buf clears data, so grab it now. */
> -		buf = vq->data[i];
> +		buf = vq->desc_state[i].data;
>  		detach_buf(vq, i);
>  		vq->avail_idx_shadow--;
>  		vq->vring.avail->idx = cpu_to_virtio16(_vq->vdev, vq->avail_idx_shadow);
> @@ -766,7 +897,8 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
>  		return NULL;
>  	}
>  
> -	vq = kmalloc(sizeof(*vq) + sizeof(void *)*num, GFP_KERNEL);
> +	vq = kmalloc(sizeof(*vq) + num * sizeof(struct vring_desc_state),
> +		     GFP_KERNEL);
>  	if (!vq)
>  		return NULL;
>  
> @@ -800,11 +932,9 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
>  
>  	/* Put everything in free lists. */
>  	vq->free_head = 0;
> -	for (i = 0; i < num-1; i++) {
> +	for (i = 0; i < num-1; i++)
>  		vq->vring.desc[i].next = cpu_to_virtio16(vdev, i + 1);
> -		vq->data[i] = NULL;
> -	}
> -	vq->data[i] = NULL;
> +	memset(vq->desc_state, 0, num * sizeof(struct vring_desc_state));
>  
>  	return &vq->vq;
>  }
> diff --git a/tools/virtio/linux/dma-mapping.h b/tools/virtio/linux/dma-mapping.h
> new file mode 100644
> index 000000000000..4f93af89ae16
> --- /dev/null
> +++ b/tools/virtio/linux/dma-mapping.h
> @@ -0,0 +1,17 @@
> +#ifndef _LINUX_DMA_MAPPING_H
> +#define _LINUX_DMA_MAPPING_H
> +
> +#ifdef CONFIG_HAS_DMA
> +# error Virtio userspace code does not support CONFIG_HAS_DMA
> +#endif
> +
> +#define PCI_DMA_BUS_IS_PHYS 1
> +
> +enum dma_data_direction {
> +	DMA_BIDIRECTIONAL = 0,
> +	DMA_TO_DEVICE = 1,
> +	DMA_FROM_DEVICE = 2,
> +	DMA_NONE = 3,
> +};
> +
> +#endif
> -- 
> 2.5.0

^ permalink raw reply

* Re: [PATCH v7 9/9] vring: Use the DMA API on Xen
From: David Vrabel @ 2016-02-03  9:49 UTC (permalink / raw)
  To: Andy Lutomirski, Michael S. Tsirkin
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, xen-devel, sparclinux,
	Paolo Bonzini, Linux Virtualization, David Woodhouse,
	David S. Miller, Martin Schwidefsky
In-Reply-To: <a5b072806984a0fceb4f4d33cbeb1cea7d881e8b.1454477782.git.luto@kernel.org>

On 03/02/16 05:46, Andy Lutomirski wrote:
> Signed-off-by: Andy Lutomirski <luto@kernel.org>

You forgot the previous Reviewed-by tags.

David

^ permalink raw reply

* [PATCH v7 9/9] vring: Use the DMA API on Xen
From: Andy Lutomirski @ 2016-02-03  5:46 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, David Vrabel,
	Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
	Linux Virtualization, David Woodhouse, David S. Miller,
	Martin Schwidefsky
In-Reply-To: <cover.1454477782.git.luto@kernel.org>

Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 drivers/virtio/virtio_ring.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index e46d08107a50..5c802d47892c 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -25,6 +25,7 @@
 #include <linux/hrtimer.h>
 #include <linux/kmemleak.h>
 #include <linux/dma-mapping.h>
+#include <xen/xen.h>
 
 #ifdef DEBUG
 /* For development, we want to crash whenever the ring is screwed. */
@@ -136,6 +137,17 @@ struct vring_virtqueue {
 
 static bool vring_use_dma_api(struct virtio_device *vdev)
 {
+	/*
+	 * In theory, it's possible to have a buggy QEMU-supposed
+	 * emulated Q35 IOMMU and Xen enabled at the same time.  On
+	 * such a configuration, virtio has never worked and will
+	 * not work without an even larger kludge.  Instead, enable
+	 * the DMA API if we're a Xen guest, which at least allows
+	 * all of the sensible Xen configurations to work correctly.
+	 */
+	if (xen_domain())
+		return true;
+
 	return false;
 }
 
-- 
2.5.0

^ permalink raw reply related

* [PATCH v7 8/9] virtio_pci: Use the DMA API if enabled
From: Andy Lutomirski @ 2016-02-03  5:46 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, David Vrabel,
	Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
	Linux Virtualization, David Woodhouse, David S. Miller,
	Martin Schwidefsky
In-Reply-To: <cover.1454477782.git.luto@kernel.org>

This switches to vring_create_virtqueue, simplifying the driver and
adding DMA API support.

This fixes virtio-pci on platforms and busses that have IOMMUs.  This
will break the experimental QEMU Q35 IOMMU support until QEMU is
fixed.  In exchange, it fixes physical virtio hardware as well as
virtio-pci running under Xen.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 drivers/virtio/virtio_pci_common.h |  6 ----
 drivers/virtio/virtio_pci_legacy.c | 42 +++++++++++---------------
 drivers/virtio/virtio_pci_modern.c | 61 ++++++++++----------------------------
 3 files changed, 33 insertions(+), 76 deletions(-)

diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h
index 2cc252270b2d..28263200ed42 100644
--- a/drivers/virtio/virtio_pci_common.h
+++ b/drivers/virtio/virtio_pci_common.h
@@ -35,12 +35,6 @@ struct virtio_pci_vq_info {
 	/* the actual virtqueue */
 	struct virtqueue *vq;
 
-	/* the number of entries in the queue */
-	int num;
-
-	/* the virtual address of the ring queue */
-	void *queue;
-
 	/* the list node for the virtqueues list */
 	struct list_head node;
 
diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c
index 48bc9797e530..8c4e61783441 100644
--- a/drivers/virtio/virtio_pci_legacy.c
+++ b/drivers/virtio/virtio_pci_legacy.c
@@ -119,7 +119,6 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 				  u16 msix_vec)
 {
 	struct virtqueue *vq;
-	unsigned long size;
 	u16 num;
 	int err;
 
@@ -131,27 +130,19 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	if (!num || ioread32(vp_dev->ioaddr + VIRTIO_PCI_QUEUE_PFN))
 		return ERR_PTR(-ENOENT);
 
-	info->num = num;
 	info->msix_vector = msix_vec;
 
-	size = PAGE_ALIGN(vring_size(num, VIRTIO_PCI_VRING_ALIGN));
-	info->queue = alloc_pages_exact(size, GFP_KERNEL|__GFP_ZERO);
-	if (info->queue == NULL)
+	/* create the vring */
+	vq = vring_create_virtqueue(index, num,
+				    VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev,
+				    true, false, vp_notify, callback, name);
+	if (!vq)
 		return ERR_PTR(-ENOMEM);
 
 	/* activate the queue */
-	iowrite32(virt_to_phys(info->queue) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT,
+	iowrite32(virtqueue_get_desc_addr(vq) >> VIRTIO_PCI_QUEUE_ADDR_SHIFT,
 		  vp_dev->ioaddr + VIRTIO_PCI_QUEUE_PFN);
 
-	/* create the vring */
-	vq = vring_new_virtqueue(index, info->num,
-				 VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev,
-				 true, info->queue, vp_notify, callback, name);
-	if (!vq) {
-		err = -ENOMEM;
-		goto out_activate_queue;
-	}
-
 	vq->priv = (void __force *)vp_dev->ioaddr + VIRTIO_PCI_QUEUE_NOTIFY;
 
 	if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
@@ -159,17 +150,15 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 		msix_vec = ioread16(vp_dev->ioaddr + VIRTIO_MSI_QUEUE_VECTOR);
 		if (msix_vec == VIRTIO_MSI_NO_VECTOR) {
 			err = -EBUSY;
-			goto out_assign;
+			goto out_deactivate;
 		}
 	}
 
 	return vq;
 
-out_assign:
-	vring_del_virtqueue(vq);
-out_activate_queue:
+out_deactivate:
 	iowrite32(0, vp_dev->ioaddr + VIRTIO_PCI_QUEUE_PFN);
-	free_pages_exact(info->queue, size);
+	vring_del_virtqueue(vq);
 	return ERR_PTR(err);
 }
 
@@ -177,7 +166,6 @@ static void del_vq(struct virtio_pci_vq_info *info)
 {
 	struct virtqueue *vq = info->vq;
 	struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev);
-	unsigned long size;
 
 	iowrite16(vq->index, vp_dev->ioaddr + VIRTIO_PCI_QUEUE_SEL);
 
@@ -188,13 +176,10 @@ static void del_vq(struct virtio_pci_vq_info *info)
 		ioread8(vp_dev->ioaddr + VIRTIO_PCI_ISR);
 	}
 
-	vring_del_virtqueue(vq);
-
 	/* Select and deactivate the queue */
 	iowrite32(0, vp_dev->ioaddr + VIRTIO_PCI_QUEUE_PFN);
 
-	size = PAGE_ALIGN(vring_size(info->num, VIRTIO_PCI_VRING_ALIGN));
-	free_pages_exact(info->queue, size);
+	vring_del_virtqueue(vq);
 }
 
 static const struct virtio_config_ops virtio_pci_config_ops = {
@@ -227,6 +212,13 @@ int virtio_pci_legacy_probe(struct virtio_pci_device *vp_dev)
 		return -ENODEV;
 	}
 
+	rc = dma_set_mask_and_coherent(&pci_dev->dev, DMA_BIT_MASK(64));
+	if (rc)
+		rc = dma_set_mask_and_coherent(&pci_dev->dev,
+						DMA_BIT_MASK(32));
+	if (rc)
+		dev_warn(&pci_dev->dev, "Failed to enable 64-bit or 32-bit DMA.  Trying to continue, but this might not work.\n");
+
 	rc = pci_request_region(pci_dev, 0, "virtio-pci-legacy");
 	if (rc)
 		return rc;
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index c0c11fad4611..0b4a4f440b85 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -287,31 +287,6 @@ static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector)
 	return vp_ioread16(&vp_dev->common->msix_config);
 }
 
-static size_t vring_pci_size(u16 num)
-{
-	/* We only need a cacheline separation. */
-	return PAGE_ALIGN(vring_size(num, SMP_CACHE_BYTES));
-}
-
-static void *alloc_virtqueue_pages(int *num)
-{
-	void *pages;
-
-	/* TODO: allocate each queue chunk individually */
-	for (; *num && vring_pci_size(*num) > PAGE_SIZE; *num /= 2) {
-		pages = alloc_pages_exact(vring_pci_size(*num),
-					  GFP_KERNEL|__GFP_ZERO|__GFP_NOWARN);
-		if (pages)
-			return pages;
-	}
-
-	if (!*num)
-		return NULL;
-
-	/* Try to get a single page. You are my only hope! */
-	return alloc_pages_exact(vring_pci_size(*num), GFP_KERNEL|__GFP_ZERO);
-}
-
 static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 				  struct virtio_pci_vq_info *info,
 				  unsigned index,
@@ -343,29 +318,22 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev,
 	/* get offset of notification word for this vq */
 	off = vp_ioread16(&cfg->queue_notify_off);
 
-	info->num = num;
 	info->msix_vector = msix_vec;
 
-	info->queue = alloc_virtqueue_pages(&info->num);
-	if (info->queue == NULL)
-		return ERR_PTR(-ENOMEM);
-
 	/* create the vring */
-	vq = vring_new_virtqueue(index, info->num,
-				 SMP_CACHE_BYTES, &vp_dev->vdev,
-				 true, info->queue, vp_notify, callback, name);
-	if (!vq) {
-		err = -ENOMEM;
-		goto err_new_queue;
-	}
+	vq = vring_create_virtqueue(index, num,
+				    SMP_CACHE_BYTES, &vp_dev->vdev,
+				    true, true, vp_notify, callback, name);
+	if (!vq)
+		return ERR_PTR(-ENOMEM);
 
 	/* activate the queue */
-	vp_iowrite16(num, &cfg->queue_size);
-	vp_iowrite64_twopart(virt_to_phys(info->queue),
+	vp_iowrite16(virtqueue_get_vring_size(vq), &cfg->queue_size);
+	vp_iowrite64_twopart(virtqueue_get_desc_addr(vq),
 			     &cfg->queue_desc_lo, &cfg->queue_desc_hi);
-	vp_iowrite64_twopart(virt_to_phys(virtqueue_get_avail(vq)),
+	vp_iowrite64_twopart(virtqueue_get_avail_addr(vq),
 			     &cfg->queue_avail_lo, &cfg->queue_avail_hi);
-	vp_iowrite64_twopart(virt_to_phys(virtqueue_get_used(vq)),
+	vp_iowrite64_twopart(virtqueue_get_used_addr(vq),
 			     &cfg->queue_used_lo, &cfg->queue_used_hi);
 
 	if (vp_dev->notify_base) {
@@ -410,8 +378,6 @@ err_assign_vector:
 		pci_iounmap(vp_dev->pci_dev, (void __iomem __force *)vq->priv);
 err_map_notify:
 	vring_del_virtqueue(vq);
-err_new_queue:
-	free_pages_exact(info->queue, vring_pci_size(info->num));
 	return ERR_PTR(err);
 }
 
@@ -456,8 +422,6 @@ static void del_vq(struct virtio_pci_vq_info *info)
 		pci_iounmap(vp_dev->pci_dev, (void __force __iomem *)vq->priv);
 
 	vring_del_virtqueue(vq);
-
-	free_pages_exact(info->queue, vring_pci_size(info->num));
 }
 
 static const struct virtio_config_ops virtio_pci_config_nodev_ops = {
@@ -641,6 +605,13 @@ int virtio_pci_modern_probe(struct virtio_pci_device *vp_dev)
 		return -EINVAL;
 	}
 
+	err = dma_set_mask_and_coherent(&pci_dev->dev, DMA_BIT_MASK(64));
+	if (err)
+		err = dma_set_mask_and_coherent(&pci_dev->dev,
+						DMA_BIT_MASK(32));
+	if (err)
+		dev_warn(&pci_dev->dev, "Failed to enable 64-bit or 32-bit DMA.  Trying to continue, but this might not work.\n");
+
 	/* Device capability is only mandatory for devices that have
 	 * device-specific configuration.
 	 */
-- 
2.5.0

^ permalink raw reply related

* [PATCH v7 7/9] virtio_mmio: Use the DMA API if enabled
From: Andy Lutomirski @ 2016-02-03  5:46 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, David Vrabel,
	Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
	Linux Virtualization, David Woodhouse, David S. Miller,
	Martin Schwidefsky
In-Reply-To: <cover.1454477782.git.luto@kernel.org>

This switches to vring_create_virtqueue, simplifying the driver and
adding DMA API support.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 drivers/virtio/virtio_mmio.c | 67 ++++++++++----------------------------------
 1 file changed, 15 insertions(+), 52 deletions(-)

diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 745c6ee1bb3e..48bfea91dbca 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -99,12 +99,6 @@ struct virtio_mmio_vq_info {
 	/* the actual virtqueue */
 	struct virtqueue *vq;
 
-	/* the number of entries in the queue */
-	unsigned int num;
-
-	/* the virtual address of the ring queue */
-	void *queue;
-
 	/* the list node for the virtqueues list */
 	struct list_head node;
 };
@@ -322,15 +316,13 @@ static void vm_del_vq(struct virtqueue *vq)
 {
 	struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vq->vdev);
 	struct virtio_mmio_vq_info *info = vq->priv;
-	unsigned long flags, size;
+	unsigned long flags;
 	unsigned int index = vq->index;
 
 	spin_lock_irqsave(&vm_dev->lock, flags);
 	list_del(&info->node);
 	spin_unlock_irqrestore(&vm_dev->lock, flags);
 
-	vring_del_virtqueue(vq);
-
 	/* Select and deactivate the queue */
 	writel(index, vm_dev->base + VIRTIO_MMIO_QUEUE_SEL);
 	if (vm_dev->version == 1) {
@@ -340,8 +332,8 @@ static void vm_del_vq(struct virtqueue *vq)
 		WARN_ON(readl(vm_dev->base + VIRTIO_MMIO_QUEUE_READY));
 	}
 
-	size = PAGE_ALIGN(vring_size(info->num, VIRTIO_MMIO_VRING_ALIGN));
-	free_pages_exact(info->queue, size);
+	vring_del_virtqueue(vq);
+
 	kfree(info);
 }
 
@@ -356,8 +348,6 @@ static void vm_del_vqs(struct virtio_device *vdev)
 	free_irq(platform_get_irq(vm_dev->pdev, 0), vm_dev);
 }
 
-
-
 static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
 				  void (*callback)(struct virtqueue *vq),
 				  const char *name)
@@ -365,7 +355,8 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
 	struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
 	struct virtio_mmio_vq_info *info;
 	struct virtqueue *vq;
-	unsigned long flags, size;
+	unsigned long flags;
+	unsigned int num;
 	int err;
 
 	if (!name)
@@ -388,66 +379,40 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned index,
 		goto error_kmalloc;
 	}
 
-	/* Allocate pages for the queue - start with a queue as big as
-	 * possible (limited by maximum size allowed by device), drop down
-	 * to a minimal size, just big enough to fit descriptor table
-	 * and two rings (which makes it "alignment_size * 2")
-	 */
-	info->num = readl(vm_dev->base + VIRTIO_MMIO_QUEUE_NUM_MAX);
-
-	/* If the device reports a 0 entry queue, we won't be able to
-	 * use it to perform I/O, and vring_new_virtqueue() can't create
-	 * empty queues anyway, so don't bother to set up the device.
-	 */
-	if (info->num == 0) {
+	num = readl(vm_dev->base + VIRTIO_MMIO_QUEUE_NUM_MAX);
+	if (num == 0) {
 		err = -ENOENT;
-		goto error_alloc_pages;
-	}
-
-	while (1) {
-		size = PAGE_ALIGN(vring_size(info->num,
-				VIRTIO_MMIO_VRING_ALIGN));
-		/* Did the last iter shrink the queue below minimum size? */
-		if (size < VIRTIO_MMIO_VRING_ALIGN * 2) {
-			err = -ENOMEM;
-			goto error_alloc_pages;
-		}
-
-		info->queue = alloc_pages_exact(size, GFP_KERNEL | __GFP_ZERO);
-		if (info->queue)
-			break;
-
-		info->num /= 2;
+		goto error_new_virtqueue;
 	}
 
 	/* Create the vring */
-	vq = vring_new_virtqueue(index, info->num, VIRTIO_MMIO_VRING_ALIGN, vdev,
-				 true, info->queue, vm_notify, callback, name);
+	vq = vring_create_virtqueue(index, num, VIRTIO_MMIO_VRING_ALIGN, vdev,
+				 true, true, vm_notify, callback, name);
 	if (!vq) {
 		err = -ENOMEM;
 		goto error_new_virtqueue;
 	}
 
 	/* Activate the queue */
-	writel(info->num, vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
+	writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
 	if (vm_dev->version == 1) {
 		writel(PAGE_SIZE, vm_dev->base + VIRTIO_MMIO_QUEUE_ALIGN);
-		writel(virt_to_phys(info->queue) >> PAGE_SHIFT,
+		writel(virtqueue_get_desc_addr(vq) >> PAGE_SHIFT,
 				vm_dev->base + VIRTIO_MMIO_QUEUE_PFN);
 	} else {
 		u64 addr;
 
-		addr = virt_to_phys(info->queue);
+		addr = virtqueue_get_desc_addr(vq);
 		writel((u32)addr, vm_dev->base + VIRTIO_MMIO_QUEUE_DESC_LOW);
 		writel((u32)(addr >> 32),
 				vm_dev->base + VIRTIO_MMIO_QUEUE_DESC_HIGH);
 
-		addr = virt_to_phys(virtqueue_get_avail(vq));
+		addr = virtqueue_get_avail_addr(vq);
 		writel((u32)addr, vm_dev->base + VIRTIO_MMIO_QUEUE_AVAIL_LOW);
 		writel((u32)(addr >> 32),
 				vm_dev->base + VIRTIO_MMIO_QUEUE_AVAIL_HIGH);
 
-		addr = virt_to_phys(virtqueue_get_used(vq));
+		addr = virtqueue_get_used_addr(vq);
 		writel((u32)addr, vm_dev->base + VIRTIO_MMIO_QUEUE_USED_LOW);
 		writel((u32)(addr >> 32),
 				vm_dev->base + VIRTIO_MMIO_QUEUE_USED_HIGH);
@@ -471,8 +436,6 @@ error_new_virtqueue:
 		writel(0, vm_dev->base + VIRTIO_MMIO_QUEUE_READY);
 		WARN_ON(readl(vm_dev->base + VIRTIO_MMIO_QUEUE_READY));
 	}
-	free_pages_exact(info->queue, size);
-error_alloc_pages:
 	kfree(info);
 error_kmalloc:
 error_available:
-- 
2.5.0

^ permalink raw reply related

* [PATCH v7 6/9] virtio: Add improved queue allocation API
From: Andy Lutomirski @ 2016-02-03  5:46 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, David Vrabel,
	Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
	Linux Virtualization, David Woodhouse, David S. Miller,
	Martin Schwidefsky
In-Reply-To: <cover.1454477782.git.luto@kernel.org>

This leaves vring_new_virtqueue alone for compatbility, but it
adds two new improved APIs:

vring_create_virtqueue: Creates a virtqueue backed by automatically
allocated coherent memory.  (Some day it this could be extended to
support non-coherent memory, too, if there ends up being a platform
on which it's worthwhile.)

__vring_new_virtqueue: Creates a virtqueue with a manually-specified
layout.  This should allow mic_virtio to work much more cleanly.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 drivers/virtio/virtio_ring.c | 209 ++++++++++++++++++++++++++++++++++++-------
 include/linux/virtio.h       |  23 ++++-
 include/linux/virtio_ring.h  |  35 ++++++++
 3 files changed, 235 insertions(+), 32 deletions(-)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 9abc008ff7ea..e46d08107a50 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -95,6 +95,11 @@ struct vring_virtqueue {
 	/* How to notify other side. FIXME: commonalize hcalls! */
 	bool (*notify)(struct virtqueue *vq);
 
+	/* DMA, allocation, and size information */
+	bool we_own_ring;
+	size_t queue_size_in_bytes;
+	dma_addr_t queue_dma_addr;
+
 #ifdef DEBUG
 	/* They're supposed to lock for us. */
 	unsigned int in_use;
@@ -878,36 +883,31 @@ irqreturn_t vring_interrupt(int irq, void *_vq)
 }
 EXPORT_SYMBOL_GPL(vring_interrupt);
 
-struct virtqueue *vring_new_virtqueue(unsigned int index,
-				      unsigned int num,
-				      unsigned int vring_align,
-				      struct virtio_device *vdev,
-				      bool weak_barriers,
-				      void *pages,
-				      bool (*notify)(struct virtqueue *),
-				      void (*callback)(struct virtqueue *),
-				      const char *name)
+struct virtqueue *__vring_new_virtqueue(unsigned int index,
+					struct vring vring,
+					struct virtio_device *vdev,
+					bool weak_barriers,
+					bool (*notify)(struct virtqueue *),
+					void (*callback)(struct virtqueue *),
+					const char *name)
 {
-	struct vring_virtqueue *vq;
 	unsigned int i;
+	struct vring_virtqueue *vq;
 
-	/* We assume num is a power of 2. */
-	if (num & (num - 1)) {
-		dev_warn(&vdev->dev, "Bad virtqueue length %u\n", num);
-		return NULL;
-	}
-
-	vq = kmalloc(sizeof(*vq) + num * sizeof(struct vring_desc_state),
+	vq = kmalloc(sizeof(*vq) + vring.num * sizeof(struct vring_desc_state),
 		     GFP_KERNEL);
 	if (!vq)
 		return NULL;
 
-	vring_init(&vq->vring, num, pages, vring_align);
+	vq->vring = vring;
 	vq->vq.callback = callback;
 	vq->vq.vdev = vdev;
 	vq->vq.name = name;
-	vq->vq.num_free = num;
+	vq->vq.num_free = vring.num;
 	vq->vq.index = index;
+	vq->we_own_ring = false;
+	vq->queue_dma_addr = 0;
+	vq->queue_size_in_bytes = 0;
 	vq->notify = notify;
 	vq->weak_barriers = weak_barriers;
 	vq->broken = false;
@@ -932,18 +932,145 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
-	for (i = 0; i < num-1; i++)
+	for (i = 0; i < vring.num-1; i++)
 		vq->vring.desc[i].next = cpu_to_virtio16(vdev, i + 1);
-	memset(vq->desc_state, 0, num * sizeof(struct vring_desc_state));
+	memset(vq->desc_state, 0, vring.num * sizeof(struct vring_desc_state));
 
 	return &vq->vq;
 }
+EXPORT_SYMBOL_GPL(__vring_new_virtqueue);
+
+static void *vring_alloc_queue(struct virtio_device *vdev, size_t size,
+			      dma_addr_t *dma_handle, gfp_t flag)
+{
+	if (vring_use_dma_api(vdev)) {
+		return dma_alloc_coherent(vdev->dev.parent, size,
+					  dma_handle, flag);
+	} else {
+		void *queue = alloc_pages_exact(PAGE_ALIGN(size), flag);
+		if (queue) {
+			phys_addr_t phys_addr = virt_to_phys(queue);
+			*dma_handle = (dma_addr_t)phys_addr;
+
+			/*
+			 * Sanity check: make sure we dind't truncate
+			 * the address.  The only arches I can find that
+			 * have 64-bit phys_addr_t but 32-bit dma_addr_t
+			 * are certain non-highmem MIPS and x86
+			 * configurations, but these configurations
+			 * should never allocate physical pages above 32
+			 * bits, so this is fine.  Just in case, throw a
+			 * warning and abort if we end up with an
+			 * unrepresentable address.
+			 */
+			if (WARN_ON_ONCE(*dma_handle != phys_addr)) {
+				free_pages_exact(queue, PAGE_ALIGN(size));
+				return NULL;
+			}
+		}
+		return queue;
+	}
+}
+
+static void vring_free_queue(struct virtio_device *vdev, size_t size,
+			     void *queue, dma_addr_t dma_handle)
+{
+	if (vring_use_dma_api(vdev)) {
+		dma_free_coherent(vdev->dev.parent, size, queue, dma_handle);
+	} else {
+		free_pages_exact(queue, PAGE_ALIGN(size));
+	}
+}
+
+struct virtqueue *vring_create_virtqueue(
+	unsigned int index,
+	unsigned int num,
+	unsigned int vring_align,
+	struct virtio_device *vdev,
+	bool weak_barriers,
+	bool may_reduce_num,
+	bool (*notify)(struct virtqueue *),
+	void (*callback)(struct virtqueue *),
+	const char *name)
+{
+	struct virtqueue *vq;
+	void *queue;
+	dma_addr_t dma_addr;
+	size_t queue_size_in_bytes;
+	struct vring vring;
+
+	/* We assume num is a power of 2. */
+	if (num & (num - 1)) {
+		dev_warn(&vdev->dev, "Bad virtqueue length %u\n", num);
+		return NULL;
+	}
+
+	/* TODO: allocate each queue chunk individually */
+	for (; num && vring_size(num, vring_align) > PAGE_SIZE; num /= 2) {
+		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
+					  &dma_addr,
+					  GFP_KERNEL|__GFP_NOWARN|__GFP_ZERO);
+		if (queue)
+			break;
+	}
+
+	if (!num)
+		return NULL;
+
+	if (!queue) {
+		/* Try to get a single page. You are my only hope! */
+		queue = vring_alloc_queue(vdev, vring_size(num, vring_align),
+					  &dma_addr, GFP_KERNEL|__GFP_ZERO);
+	}
+	if (!queue)
+		return NULL;
+
+	queue_size_in_bytes = vring_size(num, vring_align);
+	vring_init(&vring, num, queue, vring_align);
+
+	vq = __vring_new_virtqueue(index, vring, vdev, weak_barriers,
+				   notify, callback, name);
+	if (!vq) {
+		vring_free_queue(vdev, queue_size_in_bytes, queue,
+				 dma_addr);
+		return NULL;
+	}
+
+	to_vvq(vq)->queue_dma_addr = dma_addr;
+	to_vvq(vq)->queue_size_in_bytes = queue_size_in_bytes;
+	to_vvq(vq)->we_own_ring = true;
+
+	return vq;
+}
+EXPORT_SYMBOL_GPL(vring_create_virtqueue);
+
+struct virtqueue *vring_new_virtqueue(unsigned int index,
+				      unsigned int num,
+				      unsigned int vring_align,
+				      struct virtio_device *vdev,
+				      bool weak_barriers,
+				      void *pages,
+				      bool (*notify)(struct virtqueue *vq),
+				      void (*callback)(struct virtqueue *vq),
+				      const char *name)
+{
+	struct vring vring;
+	vring_init(&vring, num, pages, vring_align);
+	return __vring_new_virtqueue(index, vring, vdev, weak_barriers,
+				     notify, callback, name);
+}
 EXPORT_SYMBOL_GPL(vring_new_virtqueue);
 
-void vring_del_virtqueue(struct virtqueue *vq)
+void vring_del_virtqueue(struct virtqueue *_vq)
 {
-	list_del(&vq->list);
-	kfree(to_vvq(vq));
+	struct vring_virtqueue *vq = to_vvq(_vq);
+
+	if (vq->we_own_ring) {
+		vring_free_queue(vq->vq.vdev, vq->queue_size_in_bytes,
+				 vq->vring.desc, vq->queue_dma_addr);
+	}
+	list_del(&_vq->list);
+	kfree(vq);
 }
 EXPORT_SYMBOL_GPL(vring_del_virtqueue);
 
@@ -1007,20 +1134,42 @@ void virtio_break_device(struct virtio_device *dev)
 }
 EXPORT_SYMBOL_GPL(virtio_break_device);
 
-void *virtqueue_get_avail(struct virtqueue *_vq)
+dma_addr_t virtqueue_get_desc_addr(struct virtqueue *_vq)
 {
 	struct vring_virtqueue *vq = to_vvq(_vq);
 
-	return vq->vring.avail;
+	BUG_ON(!vq->we_own_ring);
+
+	return vq->queue_dma_addr;
 }
-EXPORT_SYMBOL_GPL(virtqueue_get_avail);
+EXPORT_SYMBOL_GPL(virtqueue_get_desc_addr);
 
-void *virtqueue_get_used(struct virtqueue *_vq)
+dma_addr_t virtqueue_get_avail_addr(struct virtqueue *_vq)
 {
 	struct vring_virtqueue *vq = to_vvq(_vq);
 
-	return vq->vring.used;
+	BUG_ON(!vq->we_own_ring);
+
+	return vq->queue_dma_addr +
+		((char *)vq->vring.avail - (char *)vq->vring.desc);
+}
+EXPORT_SYMBOL_GPL(virtqueue_get_avail_addr);
+
+dma_addr_t virtqueue_get_used_addr(struct virtqueue *_vq)
+{
+	struct vring_virtqueue *vq = to_vvq(_vq);
+
+	BUG_ON(!vq->we_own_ring);
+
+	return vq->queue_dma_addr +
+		((char *)vq->vring.used - (char *)vq->vring.desc);
+}
+EXPORT_SYMBOL_GPL(virtqueue_get_used_addr);
+
+const struct vring *virtqueue_get_vring(struct virtqueue *vq)
+{
+	return &to_vvq(vq)->vring;
 }
-EXPORT_SYMBOL_GPL(virtqueue_get_used);
+EXPORT_SYMBOL_GPL(virtqueue_get_vring);
 
 MODULE_LICENSE("GPL");
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index 8f4d4bfa6d46..d5eb5479a425 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -75,8 +75,27 @@ unsigned int virtqueue_get_vring_size(struct virtqueue *vq);
 
 bool virtqueue_is_broken(struct virtqueue *vq);
 
-void *virtqueue_get_avail(struct virtqueue *vq);
-void *virtqueue_get_used(struct virtqueue *vq);
+const struct vring *virtqueue_get_vring(struct virtqueue *vq);
+dma_addr_t virtqueue_get_desc_addr(struct virtqueue *vq);
+dma_addr_t virtqueue_get_avail_addr(struct virtqueue *vq);
+dma_addr_t virtqueue_get_used_addr(struct virtqueue *vq);
+
+/*
+ * Legacy accessors -- in almost all cases, these are the wrong functions
+ * to use.
+ */
+static inline void *virtqueue_get_desc(struct virtqueue *vq)
+{
+	return virtqueue_get_vring(vq)->desc;
+}
+static inline void *virtqueue_get_avail(struct virtqueue *vq)
+{
+	return virtqueue_get_vring(vq)->avail;
+}
+static inline void *virtqueue_get_used(struct virtqueue *vq)
+{
+	return virtqueue_get_vring(vq)->used;
+}
 
 /**
  * virtio_device - representation of a device using virtio
diff --git a/include/linux/virtio_ring.h b/include/linux/virtio_ring.h
index a156e2b6ccfe..e8d36938f09a 100644
--- a/include/linux/virtio_ring.h
+++ b/include/linux/virtio_ring.h
@@ -59,6 +59,35 @@ static inline void virtio_store_mb(bool weak_barriers,
 struct virtio_device;
 struct virtqueue;
 
+/*
+ * Creates a virtqueue and allocates the descriptor ring.  If
+ * may_reduce_num is set, then this may allocate a smaller ring than
+ * expected.  The caller should query virtqueue_get_ring_size to learn
+ * the actual size of the ring.
+ */
+struct virtqueue *vring_create_virtqueue(unsigned int index,
+					 unsigned int num,
+					 unsigned int vring_align,
+					 struct virtio_device *vdev,
+					 bool weak_barriers,
+					 bool may_reduce_num,
+					 bool (*notify)(struct virtqueue *vq),
+					 void (*callback)(struct virtqueue *vq),
+					 const char *name);
+
+/* Creates a virtqueue with a custom layout. */
+struct virtqueue *__vring_new_virtqueue(unsigned int index,
+					struct vring vring,
+					struct virtio_device *vdev,
+					bool weak_barriers,
+					bool (*notify)(struct virtqueue *),
+					void (*callback)(struct virtqueue *),
+					const char *name);
+
+/*
+ * Creates a virtqueue with a standard layout but a caller-allocated
+ * ring.
+ */
 struct virtqueue *vring_new_virtqueue(unsigned int index,
 				      unsigned int num,
 				      unsigned int vring_align,
@@ -68,7 +97,13 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
 				      bool (*notify)(struct virtqueue *vq),
 				      void (*callback)(struct virtqueue *vq),
 				      const char *name);
+
+/*
+ * Destroys a virtqueue.  If created with vring_create_virtqueue, this
+ * also frees the ring.
+ */
 void vring_del_virtqueue(struct virtqueue *vq);
+
 /* Filter out transport-specific feature bits. */
 void vring_transport_features(struct virtio_device *vdev);
 
-- 
2.5.0

^ permalink raw reply related

* [PATCH v7 5/9] virtio_ring: Support DMA APIs
From: Andy Lutomirski @ 2016-02-03  5:46 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, David Vrabel,
	Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
	Linux Virtualization, David Woodhouse, David S. Miller,
	Martin Schwidefsky
In-Reply-To: <cover.1454477782.git.luto@kernel.org>

virtio_ring currently sends the device (usually a hypervisor)
physical addresses of its I/O buffers.  This is okay when DMA
addresses and physical addresses are the same thing, but this isn't
always the case.  For example, this never works on Xen guests, and
it is likely to fail if a physical "virtio" device ever ends up
behind an IOMMU or swiotlb.

The immediate use case for me is to enable virtio on Xen guests.
For that to work, we need vring to support DMA address translation
as well as a corresponding change to virtio_pci or to another
driver.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 drivers/virtio/Kconfig           |   2 +-
 drivers/virtio/virtio_ring.c     | 200 ++++++++++++++++++++++++++++++++-------
 tools/virtio/linux/dma-mapping.h |  17 ++++
 3 files changed, 183 insertions(+), 36 deletions(-)
 create mode 100644 tools/virtio/linux/dma-mapping.h

diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig
index cab9f3f63a38..77590320d44c 100644
--- a/drivers/virtio/Kconfig
+++ b/drivers/virtio/Kconfig
@@ -60,7 +60,7 @@ config VIRTIO_INPUT
 
  config VIRTIO_MMIO
 	tristate "Platform bus driver for memory mapped virtio devices"
-	depends on HAS_IOMEM
+	depends on HAS_IOMEM && HAS_DMA
  	select VIRTIO
  	---help---
  	 This drivers provides support for memory mapped virtio
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index ab0be6c084f6..9abc008ff7ea 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -24,6 +24,7 @@
 #include <linux/module.h>
 #include <linux/hrtimer.h>
 #include <linux/kmemleak.h>
+#include <linux/dma-mapping.h>
 
 #ifdef DEBUG
 /* For development, we want to crash whenever the ring is screwed. */
@@ -54,6 +55,11 @@
 #define END_USE(vq)
 #endif
 
+struct vring_desc_state {
+	void *data;			/* Data for callback. */
+	struct vring_desc *indir_desc;	/* Indirect descriptor, if any. */
+};
+
 struct vring_virtqueue {
 	struct virtqueue vq;
 
@@ -98,8 +104,8 @@ struct vring_virtqueue {
 	ktime_t last_add_time;
 #endif
 
-	/* Tokens for callbacks. */
-	void *data[];
+	/* Per-descriptor state. */
+	struct vring_desc_state desc_state[];
 };
 
 #define to_vvq(_vq) container_of(_vq, struct vring_virtqueue, vq)
@@ -128,6 +134,79 @@ static bool vring_use_dma_api(struct virtio_device *vdev)
 	return false;
 }
 
+/*
+ * The DMA ops on various arches are rather gnarly right now, and
+ * making all of the arch DMA ops work on the vring device itself
+ * is a mess.  For now, we use the parent device for DMA ops.
+ */
+struct device *vring_dma_dev(const struct vring_virtqueue *vq)
+{
+	return vq->vq.vdev->dev.parent;
+}
+
+/* Map one sg entry. */
+static dma_addr_t vring_map_one_sg(const struct vring_virtqueue *vq,
+				   struct scatterlist *sg,
+				   enum dma_data_direction direction)
+{
+	if (!vring_use_dma_api(vq->vq.vdev))
+		return (dma_addr_t)sg_phys(sg);
+
+	/*
+	 * We can't use dma_map_sg, because we don't use scatterlists in
+	 * the way it expects (we don't guarantee that the scatterlist
+	 * will exist for the lifetime of the mapping).
+	 */
+	return dma_map_page(vring_dma_dev(vq),
+			    sg_page(sg), sg->offset, sg->length,
+			    direction);
+}
+
+static dma_addr_t vring_map_single(const struct vring_virtqueue *vq,
+				   void *cpu_addr, size_t size,
+				   enum dma_data_direction direction)
+{
+	if (!vring_use_dma_api(vq->vq.vdev))
+		return (dma_addr_t)virt_to_phys(cpu_addr);
+
+	return dma_map_single(vring_dma_dev(vq),
+			      cpu_addr, size, direction);
+}
+
+static void vring_unmap_one(const struct vring_virtqueue *vq,
+			    struct vring_desc *desc)
+{
+	u16 flags;
+
+	if (!vring_use_dma_api(vq->vq.vdev))
+		return;
+
+	flags = virtio16_to_cpu(vq->vq.vdev, desc->flags);
+
+	if (flags & VRING_DESC_F_INDIRECT) {
+		dma_unmap_single(vring_dma_dev(vq),
+				 virtio64_to_cpu(vq->vq.vdev, desc->addr),
+				 virtio32_to_cpu(vq->vq.vdev, desc->len),
+				 (flags & VRING_DESC_F_WRITE) ?
+				 DMA_FROM_DEVICE : DMA_TO_DEVICE);
+	} else {
+		dma_unmap_page(vring_dma_dev(vq),
+			       virtio64_to_cpu(vq->vq.vdev, desc->addr),
+			       virtio32_to_cpu(vq->vq.vdev, desc->len),
+			       (flags & VRING_DESC_F_WRITE) ?
+			       DMA_FROM_DEVICE : DMA_TO_DEVICE);
+	}
+}
+
+static int vring_mapping_error(const struct vring_virtqueue *vq,
+			       dma_addr_t addr)
+{
+	if (!vring_use_dma_api(vq->vq.vdev))
+		return 0;
+
+	return dma_mapping_error(vring_dma_dev(vq), addr);
+}
+
 static struct vring_desc *alloc_indirect(struct virtqueue *_vq,
 					 unsigned int total_sg, gfp_t gfp)
 {
@@ -161,7 +240,7 @@ static inline int virtqueue_add(struct virtqueue *_vq,
 	struct vring_virtqueue *vq = to_vvq(_vq);
 	struct scatterlist *sg;
 	struct vring_desc *desc;
-	unsigned int i, n, avail, descs_used, uninitialized_var(prev);
+	unsigned int i, n, avail, descs_used, uninitialized_var(prev), err_idx;
 	int head;
 	bool indirect;
 
@@ -201,21 +280,15 @@ static inline int virtqueue_add(struct virtqueue *_vq,
 
 	if (desc) {
 		/* Use a single buffer which doesn't continue */
-		vq->vring.desc[head].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_INDIRECT);
-		vq->vring.desc[head].addr = cpu_to_virtio64(_vq->vdev, virt_to_phys(desc));
-		/* avoid kmemleak false positive (hidden by virt_to_phys) */
-		kmemleak_ignore(desc);
-		vq->vring.desc[head].len = cpu_to_virtio32(_vq->vdev, total_sg * sizeof(struct vring_desc));
-
+		indirect = true;
 		/* Set up rest to use this indirect table. */
 		i = 0;
 		descs_used = 1;
-		indirect = true;
 	} else {
+		indirect = false;
 		desc = vq->vring.desc;
 		i = head;
 		descs_used = total_sg;
-		indirect = false;
 	}
 
 	if (vq->vq.num_free < descs_used) {
@@ -230,13 +303,14 @@ static inline int virtqueue_add(struct virtqueue *_vq,
 		return -ENOSPC;
 	}
 
-	/* We're about to use some buffers from the free list. */
-	vq->vq.num_free -= descs_used;
-
 	for (n = 0; n < out_sgs; n++) {
 		for (sg = sgs[n]; sg; sg = sg_next(sg)) {
+			dma_addr_t addr = vring_map_one_sg(vq, sg, DMA_TO_DEVICE);
+			if (vring_mapping_error(vq, addr))
+				goto unmap_release;
+
 			desc[i].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_NEXT);
-			desc[i].addr = cpu_to_virtio64(_vq->vdev, sg_phys(sg));
+			desc[i].addr = cpu_to_virtio64(_vq->vdev, addr);
 			desc[i].len = cpu_to_virtio32(_vq->vdev, sg->length);
 			prev = i;
 			i = virtio16_to_cpu(_vq->vdev, desc[i].next);
@@ -244,8 +318,12 @@ static inline int virtqueue_add(struct virtqueue *_vq,
 	}
 	for (; n < (out_sgs + in_sgs); n++) {
 		for (sg = sgs[n]; sg; sg = sg_next(sg)) {
+			dma_addr_t addr = vring_map_one_sg(vq, sg, DMA_FROM_DEVICE);
+			if (vring_mapping_error(vq, addr))
+				goto unmap_release;
+
 			desc[i].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_NEXT | VRING_DESC_F_WRITE);
-			desc[i].addr = cpu_to_virtio64(_vq->vdev, sg_phys(sg));
+			desc[i].addr = cpu_to_virtio64(_vq->vdev, addr);
 			desc[i].len = cpu_to_virtio32(_vq->vdev, sg->length);
 			prev = i;
 			i = virtio16_to_cpu(_vq->vdev, desc[i].next);
@@ -254,14 +332,33 @@ static inline int virtqueue_add(struct virtqueue *_vq,
 	/* Last one doesn't continue. */
 	desc[prev].flags &= cpu_to_virtio16(_vq->vdev, ~VRING_DESC_F_NEXT);
 
+	if (indirect) {
+		/* Now that the indirect table is filled in, map it. */
+		dma_addr_t addr = vring_map_single(
+			vq, desc, total_sg * sizeof(struct vring_desc),
+			DMA_TO_DEVICE);
+		if (vring_mapping_error(vq, addr))
+			goto unmap_release;
+
+		vq->vring.desc[head].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_INDIRECT);
+		vq->vring.desc[head].addr = cpu_to_virtio64(_vq->vdev, addr);
+
+		vq->vring.desc[head].len = cpu_to_virtio32(_vq->vdev, total_sg * sizeof(struct vring_desc));
+	}
+
+	/* We're using some buffers from the free list. */
+	vq->vq.num_free -= descs_used;
+
 	/* Update free pointer */
 	if (indirect)
 		vq->free_head = virtio16_to_cpu(_vq->vdev, vq->vring.desc[head].next);
 	else
 		vq->free_head = i;
 
-	/* Set token. */
-	vq->data[head] = data;
+	/* Store token and indirect buffer state. */
+	vq->desc_state[head].data = data;
+	if (indirect)
+		vq->desc_state[head].indir_desc = desc;
 
 	/* Put entry in available array (but don't update avail->idx until they
 	 * do sync). */
@@ -284,6 +381,24 @@ static inline int virtqueue_add(struct virtqueue *_vq,
 		virtqueue_kick(_vq);
 
 	return 0;
+
+unmap_release:
+	err_idx = i;
+	i = head;
+
+	for (n = 0; n < total_sg; n++) {
+		if (i == err_idx)
+			break;
+		vring_unmap_one(vq, &desc[i]);
+		i = vq->vring.desc[i].next;
+	}
+
+	vq->vq.num_free += total_sg;
+
+	if (indirect)
+		kfree(desc);
+
+	return -EIO;
 }
 
 /**
@@ -454,27 +569,43 @@ EXPORT_SYMBOL_GPL(virtqueue_kick);
 
 static void detach_buf(struct vring_virtqueue *vq, unsigned int head)
 {
-	unsigned int i;
+	unsigned int i, j;
+	u16 nextflag = cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_NEXT);
 
 	/* Clear data ptr. */
-	vq->data[head] = NULL;
+	vq->desc_state[head].data = NULL;
 
-	/* Put back on free list: find end */
+	/* Put back on free list: unmap first-level descriptors and find end */
 	i = head;
 
-	/* Free the indirect table */
-	if (vq->vring.desc[i].flags & cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_INDIRECT))
-		kfree(phys_to_virt(virtio64_to_cpu(vq->vq.vdev, vq->vring.desc[i].addr)));
-
-	while (vq->vring.desc[i].flags & cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_NEXT)) {
+	while (vq->vring.desc[i].flags & nextflag) {
+		vring_unmap_one(vq, &vq->vring.desc[i]);
 		i = virtio16_to_cpu(vq->vq.vdev, vq->vring.desc[i].next);
 		vq->vq.num_free++;
 	}
 
+	vring_unmap_one(vq, &vq->vring.desc[i]);
 	vq->vring.desc[i].next = cpu_to_virtio16(vq->vq.vdev, vq->free_head);
 	vq->free_head = head;
+
 	/* Plus final descriptor */
 	vq->vq.num_free++;
+
+	/* Free the indirect table, if any, now that it's unmapped. */
+	if (vq->desc_state[head].indir_desc) {
+		struct vring_desc *indir_desc = vq->desc_state[head].indir_desc;
+		u32 len = virtio32_to_cpu(vq->vq.vdev, vq->vring.desc[head].len);
+
+		BUG_ON(!(vq->vring.desc[head].flags &
+			 cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_INDIRECT)));
+		BUG_ON(len == 0 || len % sizeof(struct vring_desc));
+
+		for (j = 0; j < len / sizeof(struct vring_desc); j++)
+			vring_unmap_one(vq, &indir_desc[j]);
+
+		kfree(vq->desc_state[head].indir_desc);
+		vq->desc_state[head].indir_desc = NULL;
+	}
 }
 
 static inline bool more_used(const struct vring_virtqueue *vq)
@@ -529,13 +660,13 @@ void *virtqueue_get_buf(struct virtqueue *_vq, unsigned int *len)
 		BAD_RING(vq, "id %u out of range\n", i);
 		return NULL;
 	}
-	if (unlikely(!vq->data[i])) {
+	if (unlikely(!vq->desc_state[i].data)) {
 		BAD_RING(vq, "id %u is not a head!\n", i);
 		return NULL;
 	}
 
 	/* detach_buf clears data, so grab it now. */
-	ret = vq->data[i];
+	ret = vq->desc_state[i].data;
 	detach_buf(vq, i);
 	vq->last_used_idx++;
 	/* If we expect an interrupt for the next entry, tell host
@@ -709,10 +840,10 @@ void *virtqueue_detach_unused_buf(struct virtqueue *_vq)
 	START_USE(vq);
 
 	for (i = 0; i < vq->vring.num; i++) {
-		if (!vq->data[i])
+		if (!vq->desc_state[i].data)
 			continue;
 		/* detach_buf clears data, so grab it now. */
-		buf = vq->data[i];
+		buf = vq->desc_state[i].data;
 		detach_buf(vq, i);
 		vq->avail_idx_shadow--;
 		vq->vring.avail->idx = cpu_to_virtio16(_vq->vdev, vq->avail_idx_shadow);
@@ -766,7 +897,8 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
 		return NULL;
 	}
 
-	vq = kmalloc(sizeof(*vq) + sizeof(void *)*num, GFP_KERNEL);
+	vq = kmalloc(sizeof(*vq) + num * sizeof(struct vring_desc_state),
+		     GFP_KERNEL);
 	if (!vq)
 		return NULL;
 
@@ -800,11 +932,9 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
 
 	/* Put everything in free lists. */
 	vq->free_head = 0;
-	for (i = 0; i < num-1; i++) {
+	for (i = 0; i < num-1; i++)
 		vq->vring.desc[i].next = cpu_to_virtio16(vdev, i + 1);
-		vq->data[i] = NULL;
-	}
-	vq->data[i] = NULL;
+	memset(vq->desc_state, 0, num * sizeof(struct vring_desc_state));
 
 	return &vq->vq;
 }
diff --git a/tools/virtio/linux/dma-mapping.h b/tools/virtio/linux/dma-mapping.h
new file mode 100644
index 000000000000..4f93af89ae16
--- /dev/null
+++ b/tools/virtio/linux/dma-mapping.h
@@ -0,0 +1,17 @@
+#ifndef _LINUX_DMA_MAPPING_H
+#define _LINUX_DMA_MAPPING_H
+
+#ifdef CONFIG_HAS_DMA
+# error Virtio userspace code does not support CONFIG_HAS_DMA
+#endif
+
+#define PCI_DMA_BUS_IS_PHYS 1
+
+enum dma_data_direction {
+	DMA_BIDIRECTIONAL = 0,
+	DMA_TO_DEVICE = 1,
+	DMA_FROM_DEVICE = 2,
+	DMA_NONE = 3,
+};
+
+#endif
-- 
2.5.0

^ permalink raw reply related

* [PATCH v7 4/9] vring: Introduce vring_use_dma_api()
From: Andy Lutomirski @ 2016-02-03  5:46 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, David Vrabel,
	Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
	Linux Virtualization, David Woodhouse, David S. Miller,
	Martin Schwidefsky
In-Reply-To: <cover.1454477782.git.luto@kernel.org>

This is a kludge, but no one has come up with a a better idea yet.
We'll introduce DMA API support guarded by vring_use_dma_api().
Eventually we may be able to return true on more and more systems,
and hopefully we can get rid of vring_use_dma_api() entirely some
day.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 drivers/virtio/virtio_ring.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index e12e385f7ac3..ab0be6c084f6 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -104,6 +104,30 @@ struct vring_virtqueue {
 
 #define to_vvq(_vq) container_of(_vq, struct vring_virtqueue, vq)
 
+/*
+ * The interaction between virtio and a possible IOMMU is a mess.
+ *
+ * On most systems with virtio, physical addresses match bus addresses,
+ * and it doesn't particularly matter whether we use the DMA API.
+ *
+ * On some systems, including Xen and any system with a physical device
+ * that speaks virtio behind a physical IOMMU, we must use the DMA API
+ * for virtio DMA to work at all.
+ *
+ * On other systems, including SPARC and PPC64, virtio-pci devices are
+ * enumerated as though they are behind an IOMMU, but the virtio host
+ * ignores the IOMMU, so we must either pretend that the IOMMU isn't
+ * there or somehow map everything as the identity.
+ *
+ * For the time being, we preserve historic behavior and bypass the DMA
+ * API.
+ */
+
+static bool vring_use_dma_api(struct virtio_device *vdev)
+{
+	return false;
+}
+
 static struct vring_desc *alloc_indirect(struct virtqueue *_vq,
 					 unsigned int total_sg, gfp_t gfp)
 {
-- 
2.5.0

^ permalink raw reply related

* [PATCH v7 3/9] s390/dma: Allow per device dma ops
From: Andy Lutomirski @ 2016-02-03  5:46 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, David Vrabel,
	Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
	Linux Virtualization, David Woodhouse, David S. Miller,
	Martin Schwidefsky
In-Reply-To: <cover.1454477782.git.luto@kernel.org>

From: Christian Borntraeger <borntraeger@de.ibm.com>

As virtio-ccw will have dma ops, we can no longer default to the
zPCI ones. Make use of dev_archdata to keep the dma_ops per device.
The pci devices now use that to override the default, and the
default is changed to use the noop ops for everything that does not
specify a device specific one.
To compile without PCI support we will enable HAS_DMA all the time,
via the default config in lib/Kconfig.

Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>
Acked-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Acked-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 arch/s390/Kconfig                   | 5 +----
 arch/s390/include/asm/device.h      | 6 +++++-
 arch/s390/include/asm/dma-mapping.h | 6 ++++--
 arch/s390/pci/pci.c                 | 1 +
 arch/s390/pci/pci_dma.c             | 4 ++--
 5 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 3be9c832dec1..b29c66e77e32 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -124,6 +124,7 @@ config S390
 	select HAVE_CMPXCHG_DOUBLE
 	select HAVE_CMPXCHG_LOCAL
 	select HAVE_DEBUG_KMEMLEAK
+	select HAVE_DMA_API_DEBUG
 	select HAVE_DYNAMIC_FTRACE
 	select HAVE_DYNAMIC_FTRACE_WITH_REGS
 	select HAVE_FTRACE_MCOUNT_RECORD
@@ -619,10 +620,6 @@ config HAS_IOMEM
 config IOMMU_HELPER
 	def_bool PCI
 
-config HAS_DMA
-	def_bool PCI
-	select HAVE_DMA_API_DEBUG
-
 config NEED_SG_DMA_LENGTH
 	def_bool PCI
 
diff --git a/arch/s390/include/asm/device.h b/arch/s390/include/asm/device.h
index d8f9872b0e2d..4a9f35e0973f 100644
--- a/arch/s390/include/asm/device.h
+++ b/arch/s390/include/asm/device.h
@@ -3,5 +3,9 @@
  *
  * This file is released under the GPLv2
  */
-#include <asm-generic/device.h>
+struct dev_archdata {
+	struct dma_map_ops *dma_ops;
+};
 
+struct pdev_archdata {
+};
diff --git a/arch/s390/include/asm/dma-mapping.h b/arch/s390/include/asm/dma-mapping.h
index e64bfcb9702f..3249b7464889 100644
--- a/arch/s390/include/asm/dma-mapping.h
+++ b/arch/s390/include/asm/dma-mapping.h
@@ -11,11 +11,13 @@
 
 #define DMA_ERROR_CODE		(~(dma_addr_t) 0x0)
 
-extern struct dma_map_ops s390_dma_ops;
+extern struct dma_map_ops s390_pci_dma_ops;
 
 static inline struct dma_map_ops *get_dma_ops(struct device *dev)
 {
-	return &s390_dma_ops;
+	if (dev && dev->archdata.dma_ops)
+		return dev->archdata.dma_ops;
+	return &dma_noop_ops;
 }
 
 static inline void dma_cache_sync(struct device *dev, void *vaddr, size_t size,
diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c
index 11d4f277e9f6..f5931135b9ae 100644
--- a/arch/s390/pci/pci.c
+++ b/arch/s390/pci/pci.c
@@ -649,6 +649,7 @@ int pcibios_add_device(struct pci_dev *pdev)
 
 	zdev->pdev = pdev;
 	pdev->dev.groups = zpci_attr_groups;
+	pdev->dev.archdata.dma_ops = &s390_pci_dma_ops;
 	zpci_map_resources(pdev);
 
 	for (i = 0; i < PCI_BAR_COUNT; i++) {
diff --git a/arch/s390/pci/pci_dma.c b/arch/s390/pci/pci_dma.c
index 4638b93c7632..a79173ec54b9 100644
--- a/arch/s390/pci/pci_dma.c
+++ b/arch/s390/pci/pci_dma.c
@@ -544,7 +544,7 @@ static int __init dma_debug_do_init(void)
 }
 fs_initcall(dma_debug_do_init);
 
-struct dma_map_ops s390_dma_ops = {
+struct dma_map_ops s390_pci_dma_ops = {
 	.alloc		= s390_dma_alloc,
 	.free		= s390_dma_free,
 	.map_sg		= s390_dma_map_sg,
@@ -555,7 +555,7 @@ struct dma_map_ops s390_dma_ops = {
 	.is_phys	= 0,
 	/* dma_supported is unconditionally true without a callback */
 };
-EXPORT_SYMBOL_GPL(s390_dma_ops);
+EXPORT_SYMBOL_GPL(s390_pci_dma_ops);
 
 static int __init s390_iommu_setup(char *str)
 {
-- 
2.5.0

^ permalink raw reply related

* [PATCH v7 2/9] alpha/dma: use common noop dma ops
From: Andy Lutomirski @ 2016-02-03  5:46 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, David Vrabel,
	Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
	Linux Virtualization, David Woodhouse, David S. Miller,
	Martin Schwidefsky
In-Reply-To: <cover.1454477782.git.luto@kernel.org>

From: Christian Borntraeger <borntraeger@de.ibm.com>

Some of the alpha pci noop dma ops are identical to the common ones.
Use them.

Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 arch/alpha/kernel/pci-noop.c | 46 ++++----------------------------------------
 1 file changed, 4 insertions(+), 42 deletions(-)

diff --git a/arch/alpha/kernel/pci-noop.c b/arch/alpha/kernel/pci-noop.c
index 2b1f4a1e9272..8e735b5e56bd 100644
--- a/arch/alpha/kernel/pci-noop.c
+++ b/arch/alpha/kernel/pci-noop.c
@@ -123,44 +123,6 @@ static void *alpha_noop_alloc_coherent(struct device *dev, size_t size,
 	return ret;
 }
 
-static void alpha_noop_free_coherent(struct device *dev, size_t size,
-				     void *cpu_addr, dma_addr_t dma_addr,
-				     struct dma_attrs *attrs)
-{
-	free_pages((unsigned long)cpu_addr, get_order(size));
-}
-
-static dma_addr_t alpha_noop_map_page(struct device *dev, struct page *page,
-				      unsigned long offset, size_t size,
-				      enum dma_data_direction dir,
-				      struct dma_attrs *attrs)
-{
-	return page_to_pa(page) + offset;
-}
-
-static int alpha_noop_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
-			     enum dma_data_direction dir, struct dma_attrs *attrs)
-{
-	int i;
-	struct scatterlist *sg;
-
-	for_each_sg(sgl, sg, nents, i) {
-		void *va;
-
-		BUG_ON(!sg_page(sg));
-		va = sg_virt(sg);
-		sg_dma_address(sg) = (dma_addr_t)virt_to_phys(va);
-		sg_dma_len(sg) = sg->length;
-	}
-
-	return nents;
-}
-
-static int alpha_noop_mapping_error(struct device *dev, dma_addr_t dma_addr)
-{
-	return 0;
-}
-
 static int alpha_noop_supported(struct device *dev, u64 mask)
 {
 	return mask < 0x00ffffffUL ? 0 : 1;
@@ -168,10 +130,10 @@ static int alpha_noop_supported(struct device *dev, u64 mask)
 
 struct dma_map_ops alpha_noop_ops = {
 	.alloc			= alpha_noop_alloc_coherent,
-	.free			= alpha_noop_free_coherent,
-	.map_page		= alpha_noop_map_page,
-	.map_sg			= alpha_noop_map_sg,
-	.mapping_error		= alpha_noop_mapping_error,
+	.free			= dma_noop_free_coherent,
+	.map_page		= dma_noop_map_page,
+	.map_sg			= dma_noop_map_sg,
+	.mapping_error		= dma_noop_mapping_error,
 	.dma_supported		= alpha_noop_supported,
 };
 
-- 
2.5.0

^ permalink raw reply related

* [PATCH v7 1/9] dma: Provide simple noop dma ops
From: Andy Lutomirski @ 2016-02-03  5:46 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, David Vrabel,
	Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
	Linux Virtualization, David Woodhouse, David S. Miller,
	Martin Schwidefsky
In-Reply-To: <cover.1454477782.git.luto@kernel.org>

From: Christian Borntraeger <borntraeger@de.ibm.com>

We are going to require dma_ops for several common drivers, even for
systems that do have an identity mapping. Lets provide some minimal
no-op dma_ops that can be used for that purpose.

Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 include/linux/dma-mapping.h |  2 ++
 lib/Makefile                |  1 +
 lib/dma-noop.c              | 75 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 78 insertions(+)
 create mode 100644 lib/dma-noop.c

diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 75857cda38e9..c0b27ff2c784 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -70,6 +70,8 @@ struct dma_map_ops {
 	int is_phys;
 };
 
+extern struct dma_map_ops dma_noop_ops;
+
 #define DMA_BIT_MASK(n)	(((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
 
 #define DMA_MASK_NONE	0x0ULL
diff --git a/lib/Makefile b/lib/Makefile
index a7c26a41a738..a572b86a1b1d 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -18,6 +18,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
 obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
 lib-$(CONFIG_MMU) += ioremap.o
 lib-$(CONFIG_SMP) += cpumask.o
+lib-$(CONFIG_HAS_DMA) += dma-noop.o
 
 lib-y	+= kobject.o klist.o
 obj-y	+= lockref.o
diff --git a/lib/dma-noop.c b/lib/dma-noop.c
new file mode 100644
index 000000000000..72145646857e
--- /dev/null
+++ b/lib/dma-noop.c
@@ -0,0 +1,75 @@
+/*
+ *	lib/dma-noop.c
+ *
+ * Simple DMA noop-ops that map 1:1 with memory
+ */
+#include <linux/export.h>
+#include <linux/mm.h>
+#include <linux/dma-mapping.h>
+#include <linux/scatterlist.h>
+
+static void *dma_noop_alloc(struct device *dev, size_t size,
+			    dma_addr_t *dma_handle, gfp_t gfp,
+			    struct dma_attrs *attrs)
+{
+	void *ret;
+
+	ret = (void *)__get_free_pages(gfp, get_order(size));
+	if (ret)
+		*dma_handle = virt_to_phys(ret);
+	return ret;
+}
+
+static void dma_noop_free(struct device *dev, size_t size,
+			  void *cpu_addr, dma_addr_t dma_addr,
+			  struct dma_attrs *attrs)
+{
+	free_pages((unsigned long)cpu_addr, get_order(size));
+}
+
+static dma_addr_t dma_noop_map_page(struct device *dev, struct page *page,
+				      unsigned long offset, size_t size,
+				      enum dma_data_direction dir,
+				      struct dma_attrs *attrs)
+{
+	return page_to_phys(page) + offset;
+}
+
+static int dma_noop_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
+			     enum dma_data_direction dir, struct dma_attrs *attrs)
+{
+	int i;
+	struct scatterlist *sg;
+
+	for_each_sg(sgl, sg, nents, i) {
+		void *va;
+
+		BUG_ON(!sg_page(sg));
+		va = sg_virt(sg);
+		sg_dma_address(sg) = (dma_addr_t)virt_to_phys(va);
+		sg_dma_len(sg) = sg->length;
+	}
+
+	return nents;
+}
+
+static int dma_noop_mapping_error(struct device *dev, dma_addr_t dma_addr)
+{
+	return 0;
+}
+
+static int dma_noop_supported(struct device *dev, u64 mask)
+{
+	return 1;
+}
+
+struct dma_map_ops dma_noop_ops = {
+	.alloc			= dma_noop_alloc,
+	.free			= dma_noop_free,
+	.map_page		= dma_noop_map_page,
+	.map_sg			= dma_noop_map_sg,
+	.mapping_error		= dma_noop_mapping_error,
+	.dma_supported		= dma_noop_supported,
+};
+
+EXPORT_SYMBOL(dma_noop_ops);
-- 
2.5.0

^ permalink raw reply related

* [PATCH v7 0/9] virtio DMA API, yet again
From: Andy Lutomirski @ 2016-02-03  5:46 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, David Vrabel,
	Andy Lutomirski, xen-devel, sparclinux, Paolo Bonzini,
	Linux Virtualization, David Woodhouse, David S. Miller,
	Martin Schwidefsky

This switches virtio to use the DMA API on Xen and if requested by
module option.

This fixes virtio on Xen, and it should break anything because it's
off by default on everything except Xen PV on x86.

To the Xen people: is this okay?  If it doesn't work on other Xen
variants (PVH? HVM?), can you submit follow-up patches to fix it?

To everyone else: we've waffled on this for way too long.  I think
we should to get DMA API implementation in with a conservative
policy like this rather than waiting until we achieve perfection.
I'm tired of carrying these patches around.

I changed queue allocation around a bit in this version.  Per Michael's
request, we no longer use dma_zalloc_coherent in the !dma_api case.
Instead we use alloc_pages_exact, just like the current code does.
This simplifies the ring address accessors, because they can always
load from the dma addr rather than depending on vring_use_dma_api
themselves.

There's an odd warning in here if the ring's physical address
doesn't fit in a dma_addr_t.  This could only possible happen on
really weird configurations in which phys_addr_t is wider than
dma_addr_t.  AFAICT this is only possible on i386 PAE systems and on
MIPS, and even there it only happens if highmem is off.  But that
means we're safe, since we should never end up with high allocations
on non-highmem systems unless we explicitly ask for them, which we
don't.

If this is too scary, I can add yet more cruft to avoid it, but
it seems harmless enough to me, and it means that the driver will
be totally clean once all the vring_use_dma_api calls go away.

Michael, if these survive review, can you stage these in your tree?
Can you also take a look at tools/virtio?  I probably broke it, but I
couldn't get it to build without these patches either, so I'm stuck.

Changes from v6:
 - Remove HAVE_DMA_ATTRS and add Acked-by (Cornelia)
 - Add some missing signed-off-by lines from me (whoops)
 - Rework queue allocation (Michael)

Changes from v5:
 - Typo fixes (David Woodhouse)
 - Use xen_domain() to detect Xen (David Vrabel)
 - Pass struct vring_virtqueue * into vring_use_dma_api for future proofing
 - Removed module parameter (Michael)

Changes from v4:
 - Bake vring_use_dma_api in from the beginning.
 - Automatically enable only on Xen.
 - Add module parameter.
 - Add s390 and alpha DMA API implementations.
 - Rebase to 4.5-rc1.

Changes from v3:
 - More big-endian fixes.
 - Added better virtio-ring APIs that handle allocation and use them in
   virtio-mmio and virtio-pci.
 - Switch to Michael's virtio-net patch.

Changes from v2:
 - Fix vring_mapping_error incorrect argument

Changes from v1:
 - Fix an endian conversion error causing a BUG to hit.
 - Fix a DMA ordering issue (swiotlb=force works now).
 - Minor cleanups.

Andy Lutomirski (6):
  vring: Introduce vring_use_dma_api()
  virtio_ring: Support DMA APIs
  virtio: Add improved queue allocation API
  virtio_mmio: Use the DMA API if enabled
  virtio_pci: Use the DMA API if enabled
  vring: Use the DMA API on Xen

Christian Borntraeger (3):
  dma: Provide simple noop dma ops
  alpha/dma: use common noop dma ops
  s390/dma: Allow per device dma ops

 arch/alpha/kernel/pci-noop.c        |  46 +---
 arch/s390/Kconfig                   |   5 +-
 arch/s390/include/asm/device.h      |   6 +-
 arch/s390/include/asm/dma-mapping.h |   6 +-
 arch/s390/pci/pci.c                 |   1 +
 arch/s390/pci/pci_dma.c             |   4 +-
 drivers/virtio/Kconfig              |   2 +-
 drivers/virtio/virtio_mmio.c        |  67 ++----
 drivers/virtio/virtio_pci_common.h  |   6 -
 drivers/virtio/virtio_pci_legacy.c  |  42 ++--
 drivers/virtio/virtio_pci_modern.c  |  61 ++---
 drivers/virtio/virtio_ring.c        | 439 +++++++++++++++++++++++++++++++-----
 include/linux/dma-mapping.h         |   2 +
 include/linux/virtio.h              |  23 +-
 include/linux/virtio_ring.h         |  35 +++
 lib/Makefile                        |   1 +
 lib/dma-noop.c                      |  75 ++++++
 tools/virtio/linux/dma-mapping.h    |  17 ++
 18 files changed, 594 insertions(+), 244 deletions(-)
 create mode 100644 lib/dma-noop.c
 create mode 100644 tools/virtio/linux/dma-mapping.h

-- 
2.5.0

^ permalink raw reply

* Re: [PATCH v6 6/9] virtio: Add improved queue allocation API
From: Andy Lutomirski @ 2016-02-02 18:26 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Joerg Roedel, KVM, linux-s390, Benjamin Herrenschmidt,
	Stefano Stabellini, Sebastian Ott, linux-kernel@vger.kernel.org,
	Christoph Hellwig, Christian Borntraeger, David Vrabel,
	Andy Lutomirski, xen-devel@lists.xenproject.org, sparclinux,
	Paolo Bonzini, Linux Virtualization, David Woodhouse,
	David S. Miller, Martin Schwidefsky
In-Reply-To: <20160202132151-mutt-send-email-mst@redhat.com>

On Tue, Feb 2, 2016 at 3:25 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
> On Mon, Feb 01, 2016 at 10:00:56AM -0800, Andy Lutomirski wrote:
>> This leaves vring_new_virtqueue alone for compatbility, but it
>> adds two new improved APIs:
>>
>> vring_create_virtqueue: Creates a virtqueue backed by automatically
>> allocated coherent memory.  (Some day it this could be extended to
>> support non-coherent memory, too, if there ends up being a platform
>> on which it's worthwhile.)
>>
>> __vring_new_virtqueue: Creates a virtqueue with a manually-specified
>> layout.  This should allow mic_virtio to work much more cleanly.
>>
>> Signed-off-by: Andy Lutomirski <luto@kernel.org>
>> ---
>>  drivers/virtio/virtio_ring.c | 178 +++++++++++++++++++++++++++++++++++--------
>>  include/linux/virtio.h       |  23 +++++-
>>  include/linux/virtio_ring.h  |  35 +++++++++
>>  3 files changed, 204 insertions(+), 32 deletions(-)
>>
>> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
>> index 2f621e96b9ff..cf2840c7e500 100644
>> --- a/drivers/virtio/virtio_ring.c
>> +++ b/drivers/virtio/virtio_ring.c
>> @@ -95,6 +95,11 @@ struct vring_virtqueue {
>>       /* How to notify other side. FIXME: commonalize hcalls! */
>>       bool (*notify)(struct virtqueue *vq);
>>
>> +     /* DMA, allocation, and size information */
>> +     bool we_own_ring;
>> +     size_t queue_size_in_bytes;
>> +     dma_addr_t queue_dma_addr;
>> +
>>  #ifdef DEBUG
>>       /* They're supposed to lock for us. */
>>       unsigned int in_use;
>> @@ -878,36 +883,31 @@ irqreturn_t vring_interrupt(int irq, void *_vq)
>>  }
>>  EXPORT_SYMBOL_GPL(vring_interrupt);
>>
>> -struct virtqueue *vring_new_virtqueue(unsigned int index,
>> -                                   unsigned int num,
>> -                                   unsigned int vring_align,
>> -                                   struct virtio_device *vdev,
>> -                                   bool weak_barriers,
>> -                                   void *pages,
>> -                                   bool (*notify)(struct virtqueue *),
>> -                                   void (*callback)(struct virtqueue *),
>> -                                   const char *name)
>> +struct virtqueue *__vring_new_virtqueue(unsigned int index,
>> +                                     struct vring vring,
>> +                                     struct virtio_device *vdev,
>> +                                     bool weak_barriers,
>> +                                     bool (*notify)(struct virtqueue *),
>> +                                     void (*callback)(struct virtqueue *),
>> +                                     const char *name)
>>  {
>> -     struct vring_virtqueue *vq;
>>       unsigned int i;
>> +     struct vring_virtqueue *vq;
>>
>> -     /* We assume num is a power of 2. */
>> -     if (num & (num - 1)) {
>> -             dev_warn(&vdev->dev, "Bad virtqueue length %u\n", num);
>> -             return NULL;
>> -     }
>> -
>> -     vq = kmalloc(sizeof(*vq) + num * sizeof(struct vring_desc_state),
>> +     vq = kmalloc(sizeof(*vq) + vring.num * sizeof(struct vring_desc_state),
>>                    GFP_KERNEL);
>>       if (!vq)
>>               return NULL;
>>
>> -     vring_init(&vq->vring, num, pages, vring_align);
>> +     vq->vring = vring;
>>       vq->vq.callback = callback;
>>       vq->vq.vdev = vdev;
>>       vq->vq.name = name;
>> -     vq->vq.num_free = num;
>> +     vq->vq.num_free = vring.num;
>>       vq->vq.index = index;
>> +     vq->we_own_ring = false;
>> +     vq->queue_dma_addr = 0;
>> +     vq->queue_size_in_bytes = 0;
>>       vq->notify = notify;
>>       vq->weak_barriers = weak_barriers;
>>       vq->broken = false;
>> @@ -932,18 +932,105 @@ struct virtqueue *vring_new_virtqueue(unsigned int index,
>>
>>       /* Put everything in free lists. */
>>       vq->free_head = 0;
>> -     for (i = 0; i < num-1; i++)
>> +     for (i = 0; i < vring.num-1; i++)
>>               vq->vring.desc[i].next = cpu_to_virtio16(vdev, i + 1);
>> -     memset(vq->desc_state, 0, num * sizeof(struct vring_desc_state));
>> +     memset(vq->desc_state, 0, vring.num * sizeof(struct vring_desc_state));
>>
>>       return &vq->vq;
>>  }
>> +EXPORT_SYMBOL_GPL(__vring_new_virtqueue);
>> +
>> +struct virtqueue *vring_create_virtqueue(
>> +     unsigned int index,
>> +     unsigned int num,
>> +     unsigned int vring_align,
>> +     struct virtio_device *vdev,
>> +     bool weak_barriers,
>> +     bool may_reduce_num,
>> +     bool (*notify)(struct virtqueue *),
>> +     void (*callback)(struct virtqueue *),
>> +     const char *name)
>> +{
>> +     struct virtqueue *vq;
>> +     void *queue;
>> +     dma_addr_t dma_addr;
>> +     size_t queue_size_in_bytes;
>> +     struct vring vring;
>> +
>> +     /* We assume num is a power of 2. */
>> +     if (num & (num - 1)) {
>> +             dev_warn(&vdev->dev, "Bad virtqueue length %u\n", num);
>> +             return NULL;
>> +     }
>> +
>> +     /* TODO: allocate each queue chunk individually */
>> +     for (; num && vring_size(num, vring_align) > PAGE_SIZE; num /= 2) {
>> +             queue = dma_zalloc_coherent(
>> +                     vdev->dev.parent, vring_size(num, vring_align),
>> +                     &dma_addr, GFP_KERNEL|__GFP_NOWARN);
>
> I think that we should teach this one to use regular kmalloc
> if vring_use_dma_api is cleared.
> Not a must but it seems cleaner at this stage.

Done.  It arguably makes the code simpler, too, since I can just set
dma_addr to virt_to_phys(queue).

--Andy

^ permalink raw reply

* Re: [PATCH v6 3/9] s390/dma: Allow per device dma ops
From: Cornelia Huck @ 2016-02-02 15:06 UTC (permalink / raw)
  To: Andy Lutomirski
  Cc: linux-s390, Joerg Roedel, KVM, Michael S. Tsirkin,
	Benjamin Herrenschmidt, Stefano Stabellini, Sebastian Ott,
	linux-kernel@vger.kernel.org, Christoph Hellwig,
	Christian Borntraeger, David Vrabel, xen-devel, sparclinux,
	Paolo Bonzini, Linux Virtualization, David Woodhouse,
	David S. Miller, Martin Schwidefsky
In-Reply-To: <454ab46cd61644851aa437c73e6c8c7da79bfb30.1454349471.git.luto@kernel.org>

On Mon,  1 Feb 2016 10:00:53 -0800
Andy Lutomirski <luto@kernel.org> wrote:

> From: Christian Borntraeger <borntraeger@de.ibm.com>
> 
> As virtio-ccw will have dma ops, we can no longer default to the
> zPCI ones. Make use of dev_archdata to keep the dma_ops per device.
> The pci devices now use that to override the default, and the
> default is changed to use the noop ops for everything that does not
> specify a device specific one.
> To compile without PCI support we will enable HAS_DMA all the time,
> via the default config in lib/Kconfig.
> 
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> Reviewed-by: Joerg Roedel <jroedel@suse.de>
> Acked-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
> ---
>  arch/s390/Kconfig                   | 6 ++----
>  arch/s390/include/asm/device.h      | 6 +++++-
>  arch/s390/include/asm/dma-mapping.h | 6 ++++--
>  arch/s390/pci/pci.c                 | 1 +
>  arch/s390/pci/pci_dma.c             | 4 ++--
>  5 files changed, 14 insertions(+), 9 deletions(-)
> 
> diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
> index 3be9c832dec1..5b22a26337b2 100644
> --- a/arch/s390/Kconfig
> +++ b/arch/s390/Kconfig
> @@ -124,6 +124,8 @@ config S390
>  	select HAVE_CMPXCHG_DOUBLE
>  	select HAVE_CMPXCHG_LOCAL
>  	select HAVE_DEBUG_KMEMLEAK
> +	select HAVE_DMA_ATTRS

HAVE_DMA_ATTRS no longer exists in the current tree.

> +	select HAVE_DMA_API_DEBUG
>  	select HAVE_DYNAMIC_FTRACE
>  	select HAVE_DYNAMIC_FTRACE_WITH_REGS
>  	select HAVE_FTRACE_MCOUNT_RECORD

With the line above removed:

Acked-by: Cornelia Huck <cornelia.huck@de.ibm.com>

^ permalink raw reply

* Re: [PATCH net-next] virtio_net: add ethtool support for set and get of settings
From: Michael S. Tsirkin @ 2016-02-02 14:10 UTC (permalink / raw)
  To: Nikolay Aleksandrov
  Cc: netdev, Nikolay Aleksandrov, Roopa Prabhu, davem, virtualization
In-Reply-To: <56B0B49E.8020902@cumulusnetworks.com>

On Tue, Feb 02, 2016 at 02:52:30PM +0100, Nikolay Aleksandrov wrote:
> On 02/02/2016 02:23 PM, Michael S. Tsirkin wrote:
> > On Tue, Feb 02, 2016 at 01:51:20PM +0100, Nikolay Aleksandrov wrote:
> >> From: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
> >>
> >> This patch allows the user to set and retrieve speed and duplex of the
> >> virtio_net device via ethtool. Having this functionality is very helpful
> >> for simulating different environments and also enables the virtio_net
> >> device to participate in operations where proper speed and duplex are
> >> required (e.g. currently bonding lacp mode requires full duplex). Custom
> >> speed and duplex are not allowed, the user-supplied settings are validated
> >> before applying. Only full and unknown duplex are allowed to be set.
> > 
> > Why isn't half duplex legal?
> > 
> 
> It is legal, I'm just not sure it makes sense. Isn't the virtual channel
> always "full duplex" ?

Not if behind it there's a backend that is half duplex.

> At first I had allowed it, but then I decided against and removed it. I
> don't mind adding it back though, the argument against is not strong.
> 
> >>
> >> Example:
> >> $ ethtool eth1
> >> Settings for eth1:
> >> ...
> >> 	Speed: Unknown!
> >> 	Duplex: Unknown! (255)
> >> $ ethtool -s eth1 speed 1000 duplex full
> >> $ ethtool eth1
> >> Settings for eth1:
> >> ...
> >> 	Speed: 1000Mb/s
> >> 	Duplex: Full
> >>
> >> Based on a patch by Roopa Prabhu.
> >>
> >> CC: Roopa Prabhu <roopa@cumulusnetworks.com>
> >> CC: Michael S. Tsirkin <mst@redhat.com>
> >> CC: virtualization@lists.linux-foundation.org
> >> Signed-off-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
> > 
> > Thanks!
> > Looks ok with one comment, see below.
> > 
> >> ---
> >> Allowed to set unknown speed/duplex if the user wants to reset them, though
> >> the user-space ethtool tool currently doesn't allow setting them.
> >>
> >>  drivers/net/virtio_net.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++++
> >>  1 file changed, 74 insertions(+)
> >>
> >> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> >> index 767ab11a6e9f..722ade567ee5 100644
> >> --- a/drivers/net/virtio_net.c
> >> +++ b/drivers/net/virtio_net.c
> >> @@ -146,6 +146,10 @@ struct virtnet_info {
> >>  	virtio_net_ctrl_ack ctrl_status;
> >>  	u8 ctrl_promisc;
> >>  	u8 ctrl_allmulti;
> >> +
> >> +	/* Ethtool settings */
> >> +	u8 duplex;
> >> +	u32 speed;
> >>  };
> >>  
> >>  struct padded_vnet_hdr {
> >> @@ -1376,6 +1380,72 @@ static void virtnet_get_channels(struct net_device *dev,
> >>  	channels->other_count = 0;
> >>  }
> >>  
> >> +static bool virtnet_validate_speed(u32 speed)
> >> +{
> >> +	switch (speed) {
> >> +	case SPEED_10:
> >> +	case SPEED_100:
> >> +	case SPEED_1000:
> >> +	case SPEED_2500:
> >> +	case SPEED_5000:
> >> +	case SPEED_10000:
> >> +	case SPEED_20000:
> >> +	case SPEED_25000:
> >> +	case SPEED_40000:
> >> +	case SPEED_50000:
> >> +	case SPEED_56000:
> >> +	case SPEED_100000:
> >> +	case SPEED_UNKNOWN:
> >> +		return true;
> >> +	}
> >> +
> >> +	return false;
> >> +}
> >> +
> >> +static bool virtnet_validate_duplex(u8 duplex)
> >> +{
> >> +	switch (duplex) {
> >> +	case DUPLEX_FULL:
> >> +	case DUPLEX_UNKNOWN:
> >> +		return true;
> >> +	}
> >> +
> >> +	return false;
> >> +}
> > 
> > Let's put the validating functions near where the
> > enums are defined so people remember to extend these
> > when adding new speeds?
> > Will help e.g. tun reuse this, too.
> > 
> 
> Right, very good suggestion. I'll split this patch in two and will
> add them for everyone to use.
> 
> Thank you,
>  Nik
> 
> >> +
> >> +static int virtnet_set_settings(struct net_device *dev, struct ethtool_cmd *cmd)
> >> +{
> >> +	struct virtnet_info *vi = netdev_priv(dev);
> >> +	u32 speed = ethtool_cmd_speed(cmd);
> >> +
> >> +	/* don't allow custom speed and duplex */
> >> +	if (!virtnet_validate_speed(speed) ||
> >> +	    !virtnet_validate_duplex(cmd->duplex))
> >> +		return -EINVAL;
> >> +	vi->speed = speed;
> >> +	vi->duplex = cmd->duplex;
> >> +
> >> +	return 0;
> >> +}
> >> +
> >> +static int virtnet_get_settings(struct net_device *dev, struct ethtool_cmd *cmd)
> >> +{
> >> +	struct virtnet_info *vi = netdev_priv(dev);
> >> +
> >> +	ethtool_cmd_speed_set(cmd, vi->speed);
> >> +	cmd->duplex = vi->duplex;
> >> +
> >> +	return 0;
> >> +}
> >> +
> >> +static void virtnet_init_settings(struct net_device *dev)
> >> +{
> >> +	struct virtnet_info *vi = netdev_priv(dev);
> >> +
> >> +	vi->speed = SPEED_UNKNOWN;
> >> +	vi->duplex = DUPLEX_UNKNOWN;
> >> +}
> >> +
> >>  static const struct ethtool_ops virtnet_ethtool_ops = {
> >>  	.get_drvinfo = virtnet_get_drvinfo,
> >>  	.get_link = ethtool_op_get_link,
> >> @@ -1383,6 +1453,8 @@ static const struct ethtool_ops virtnet_ethtool_ops = {
> >>  	.set_channels = virtnet_set_channels,
> >>  	.get_channels = virtnet_get_channels,
> >>  	.get_ts_info = ethtool_op_get_ts_info,
> >> +	.get_settings = virtnet_get_settings,
> >> +	.set_settings = virtnet_set_settings,
> >>  };
> >>  
> >>  #define MIN_MTU 68
> >> @@ -1855,6 +1927,8 @@ static int virtnet_probe(struct virtio_device *vdev)
> >>  	netif_set_real_num_tx_queues(dev, vi->curr_queue_pairs);
> >>  	netif_set_real_num_rx_queues(dev, vi->curr_queue_pairs);
> >>  
> >> +	virtnet_init_settings(dev);
> >> +
> >>  	err = register_netdev(dev);
> >>  	if (err) {
> >>  		pr_debug("virtio_net: registering device failed\n");
> >> -- 
> >> 2.4.3

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox