* [PATCH 00/16 V2] vhost: fix scsi cmd handling and IOPs
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
The following patches were made over Michael's vhost branch here:
https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git/log/?h=vhost
The patches also apply to Linus's or Martin's trees if you apply
https://patchwork.kernel.org/patch/11790681/
which was merged into mst's tree already.
The following patches are a follow up to this post:
https://patchwork.kernel.org/cover/11790763/
which originally was fixing how vhost-scsi handled cmds so we would
not get IO errors when sending more than 256 cmds.
In that patchset I needed to detect if a vq was in use and for this
patch:
https://patchwork.kernel.org/patch/11790685/
it was suggested to add support for VHOST_RING_ENABLE. While doing
that though I hit a couple problems:
1. The patches moved how vhost-scsi allocated cmds from per lio
session to per vhost vq. To support both VHOST_RING_ENABLE and
where userspace didn't support it, I would have to keep around the
old per session/device cmd allocator/completion and then also maintain
the new code. Or, I would still have to use this patch
patchwork.kernel.org/cover/11790763/ for the compat case so there
adding the new ioctl would not help much.
2. For vhost-scsi I also wanted to prevent where we allocate iovecs
for 128 vqs even though we normally use a couple. To do this, I needed
something similar to #1, but the problem is that the VHOST_RING_ENABLE
call would come too late.
To try and balance #1 and #2, these patches just allow vhost-scsi
to setup a vq when userspace starts to config it. This allows the
driver to only fully setup (we still waste some memory to support older
setups but do not have to preallocate everything like before) what
is used plus I do not need to maintain 2 code paths.
Note that in this posting I am also including additional patches
that create multiple vhost worker threads, because I wanted to see
if people felt that maybe to support that and for this enablement
issue we want a completely a new ioctl.
V2:
- fix use before set cpu var errors
- drop vhost_vq_is_setup
- include patches to do a worker thread per scsi IO vq
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH 01/16] vhost scsi: add lun parser helper
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
Move code to parse lun from req's lun_buf to helper, so tmf code
can use it in the next patch.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
drivers/vhost/scsi.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index b22adf0..0ea78d0 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -907,6 +907,11 @@ static void vhost_scsi_submission_work(struct work_struct *work)
return ret;
}
+static u16 vhost_buf_to_lun(u8 *lun_buf)
+{
+ return ((lun_buf[2] << 8) | lun_buf[3]) & 0x3FFF;
+}
+
static void
vhost_scsi_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq)
{
@@ -1045,12 +1050,12 @@ static void vhost_scsi_submission_work(struct work_struct *work)
tag = vhost64_to_cpu(vq, v_req_pi.tag);
task_attr = v_req_pi.task_attr;
cdb = &v_req_pi.cdb[0];
- lun = ((v_req_pi.lun[2] << 8) | v_req_pi.lun[3]) & 0x3FFF;
+ lun = vhost_buf_to_lun(v_req_pi.lun);
} else {
tag = vhost64_to_cpu(vq, v_req.tag);
task_attr = v_req.task_attr;
cdb = &v_req.cdb[0];
- lun = ((v_req.lun[2] << 8) | v_req.lun[3]) & 0x3FFF;
+ lun = vhost_buf_to_lun(v_req.lun);
}
/*
* Check that the received CDB size does not exceeded our
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 01/16] vhost scsi: add lun parser helper
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
Move code to parse lun from req's lun_buf to helper, so tmf code
can use it in the next patch.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
drivers/vhost/scsi.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index b22adf0..0ea78d0 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -907,6 +907,11 @@ static void vhost_scsi_submission_work(struct work_struct *work)
return ret;
}
+static u16 vhost_buf_to_lun(u8 *lun_buf)
+{
+ return ((lun_buf[2] << 8) | lun_buf[3]) & 0x3FFF;
+}
+
static void
vhost_scsi_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq)
{
@@ -1045,12 +1050,12 @@ static void vhost_scsi_submission_work(struct work_struct *work)
tag = vhost64_to_cpu(vq, v_req_pi.tag);
task_attr = v_req_pi.task_attr;
cdb = &v_req_pi.cdb[0];
- lun = ((v_req_pi.lun[2] << 8) | v_req_pi.lun[3]) & 0x3FFF;
+ lun = vhost_buf_to_lun(v_req_pi.lun);
} else {
tag = vhost64_to_cpu(vq, v_req.tag);
task_attr = v_req.task_attr;
cdb = &v_req.cdb[0];
- lun = ((v_req.lun[2] << 8) | v_req.lun[3]) & 0x3FFF;
+ lun = vhost_buf_to_lun(v_req.lun);
}
/*
* Check that the received CDB size does not exceeded our
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 02/16] vhost: remove work arg from vhost_work_flush
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
vhost_work_flush doesn't do anything with the work arg. This patch drops
it and then renames vhost_work_flush to vhost_work_dev_flush to reflect
that the function flushes all the works in the dev and not just a
specific queue or work item.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
drivers/vhost/scsi.c | 4 ++--
drivers/vhost/vhost.c | 8 ++++----
drivers/vhost/vhost.h | 2 +-
drivers/vhost/vsock.c | 2 +-
4 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 0ea78d0..86617bb 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1370,8 +1370,8 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
/* Flush both the vhost poll and vhost work */
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++)
vhost_scsi_flush_vq(vs, i);
- vhost_work_flush(&vs->dev, &vs->vs_completion_work);
- vhost_work_flush(&vs->dev, &vs->vs_event_work);
+ vhost_work_dev_flush(&vs->dev);
+ vhost_work_dev_flush(&vs->dev);
/* Wait for all reqs issued before the flush to be finished */
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++)
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index 9ad45e1..fbb66f6 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -231,7 +231,7 @@ void vhost_poll_stop(struct vhost_poll *poll)
}
EXPORT_SYMBOL_GPL(vhost_poll_stop);
-void vhost_work_flush(struct vhost_dev *dev, struct vhost_work *work)
+void vhost_work_dev_flush(struct vhost_dev *dev)
{
struct vhost_flush_struct flush;
@@ -243,13 +243,13 @@ void vhost_work_flush(struct vhost_dev *dev, struct vhost_work *work)
wait_for_completion(&flush.wait_event);
}
}
-EXPORT_SYMBOL_GPL(vhost_work_flush);
+EXPORT_SYMBOL_GPL(vhost_work_dev_flush);
/* Flush any work that has been scheduled. When calling this, don't hold any
* locks that are also used by the callback. */
void vhost_poll_flush(struct vhost_poll *poll)
{
- vhost_work_flush(poll->dev, &poll->work);
+ vhost_work_dev_flush(poll->dev);
}
EXPORT_SYMBOL_GPL(vhost_poll_flush);
@@ -533,7 +533,7 @@ static int vhost_attach_cgroups(struct vhost_dev *dev)
attach.owner = current;
vhost_work_init(&attach.work, vhost_attach_cgroups_work);
vhost_work_queue(dev, &attach.work);
- vhost_work_flush(dev, &attach.work);
+ vhost_work_dev_flush(dev);
return attach.ret;
}
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 9032d3c..11db183 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -46,7 +46,7 @@ void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn,
void vhost_poll_stop(struct vhost_poll *poll);
void vhost_poll_flush(struct vhost_poll *poll);
void vhost_poll_queue(struct vhost_poll *poll);
-void vhost_work_flush(struct vhost_dev *dev, struct vhost_work *work);
+void vhost_work_dev_flush(struct vhost_dev *dev);
long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *argp);
struct vhost_log {
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index a483cec..f40205f 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -652,7 +652,7 @@ static void vhost_vsock_flush(struct vhost_vsock *vsock)
for (i = 0; i < ARRAY_SIZE(vsock->vqs); i++)
if (vsock->vqs[i].handle_kick)
vhost_poll_flush(&vsock->vqs[i].poll);
- vhost_work_flush(&vsock->dev, &vsock->send_pkt_work);
+ vhost_work_dev_flush(&vsock->dev);
}
static void vhost_vsock_reset_orphans(struct sock *sk)
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 02/16] vhost: remove work arg from vhost_work_flush
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
vhost_work_flush doesn't do anything with the work arg. This patch drops
it and then renames vhost_work_flush to vhost_work_dev_flush to reflect
that the function flushes all the works in the dev and not just a
specific queue or work item.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
drivers/vhost/scsi.c | 4 ++--
drivers/vhost/vhost.c | 8 ++++----
drivers/vhost/vhost.h | 2 +-
drivers/vhost/vsock.c | 2 +-
4 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 0ea78d0..86617bb 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1370,8 +1370,8 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
/* Flush both the vhost poll and vhost work */
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++)
vhost_scsi_flush_vq(vs, i);
- vhost_work_flush(&vs->dev, &vs->vs_completion_work);
- vhost_work_flush(&vs->dev, &vs->vs_event_work);
+ vhost_work_dev_flush(&vs->dev);
+ vhost_work_dev_flush(&vs->dev);
/* Wait for all reqs issued before the flush to be finished */
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++)
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index 9ad45e1..fbb66f6 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -231,7 +231,7 @@ void vhost_poll_stop(struct vhost_poll *poll)
}
EXPORT_SYMBOL_GPL(vhost_poll_stop);
-void vhost_work_flush(struct vhost_dev *dev, struct vhost_work *work)
+void vhost_work_dev_flush(struct vhost_dev *dev)
{
struct vhost_flush_struct flush;
@@ -243,13 +243,13 @@ void vhost_work_flush(struct vhost_dev *dev, struct vhost_work *work)
wait_for_completion(&flush.wait_event);
}
}
-EXPORT_SYMBOL_GPL(vhost_work_flush);
+EXPORT_SYMBOL_GPL(vhost_work_dev_flush);
/* Flush any work that has been scheduled. When calling this, don't hold any
* locks that are also used by the callback. */
void vhost_poll_flush(struct vhost_poll *poll)
{
- vhost_work_flush(poll->dev, &poll->work);
+ vhost_work_dev_flush(poll->dev);
}
EXPORT_SYMBOL_GPL(vhost_poll_flush);
@@ -533,7 +533,7 @@ static int vhost_attach_cgroups(struct vhost_dev *dev)
attach.owner = current;
vhost_work_init(&attach.work, vhost_attach_cgroups_work);
vhost_work_queue(dev, &attach.work);
- vhost_work_flush(dev, &attach.work);
+ vhost_work_dev_flush(dev);
return attach.ret;
}
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 9032d3c..11db183 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -46,7 +46,7 @@ void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn,
void vhost_poll_stop(struct vhost_poll *poll);
void vhost_poll_flush(struct vhost_poll *poll);
void vhost_poll_queue(struct vhost_poll *poll);
-void vhost_work_flush(struct vhost_dev *dev, struct vhost_work *work);
+void vhost_work_dev_flush(struct vhost_dev *dev);
long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *argp);
struct vhost_log {
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index a483cec..f40205f 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -652,7 +652,7 @@ static void vhost_vsock_flush(struct vhost_vsock *vsock)
for (i = 0; i < ARRAY_SIZE(vsock->vqs); i++)
if (vsock->vqs[i].handle_kick)
vhost_poll_flush(&vsock->vqs[i].poll);
- vhost_work_flush(&vsock->dev, &vsock->send_pkt_work);
+ vhost_work_dev_flush(&vsock->dev);
}
static void vhost_vsock_reset_orphans(struct sock *sk)
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 03/16] vhost net: use goto error handling in open
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
In the next patches vhost_dev_init will be able to fail. This patch has
vhost_net_open use goto error handling like is done in the other vhost
code to make handling vhost_dev_init failures easier to handle and
extend in the future.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/net.c | 29 ++++++++++++++---------------
1 file changed, 14 insertions(+), 15 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index 531a00d..831d824 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -1286,27 +1286,18 @@ static int vhost_net_open(struct inode *inode, struct file *f)
if (!n)
return -ENOMEM;
vqs = kmalloc_array(VHOST_NET_VQ_MAX, sizeof(*vqs), GFP_KERNEL);
- if (!vqs) {
- kvfree(n);
- return -ENOMEM;
- }
+ if (!vqs)
+ goto err_vqs;
queue = kmalloc_array(VHOST_NET_BATCH, sizeof(void *),
GFP_KERNEL);
- if (!queue) {
- kfree(vqs);
- kvfree(n);
- return -ENOMEM;
- }
+ if (!queue)
+ goto err_queue;
n->vqs[VHOST_NET_VQ_RX].rxq.queue = queue;
xdp = kmalloc_array(VHOST_NET_BATCH, sizeof(*xdp), GFP_KERNEL);
- if (!xdp) {
- kfree(vqs);
- kvfree(n);
- kfree(queue);
- return -ENOMEM;
- }
+ if (!xdp)
+ goto err_xdp;
n->vqs[VHOST_NET_VQ_TX].xdp = xdp;
dev = &n->dev;
@@ -1338,6 +1329,14 @@ static int vhost_net_open(struct inode *inode, struct file *f)
n->refcnt_bias = 0;
return 0;
+
+err_xdp:
+ kfree(queue);
+err_queue:
+ kfree(vqs);
+err_vqs:
+ kvfree(n);
+ return -ENOMEM;
}
static struct socket *vhost_net_stop_vq(struct vhost_net *n,
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 03/16] vhost net: use goto error handling in open
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
In the next patches vhost_dev_init will be able to fail. This patch has
vhost_net_open use goto error handling like is done in the other vhost
code to make handling vhost_dev_init failures easier to handle and
extend in the future.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/net.c | 29 ++++++++++++++---------------
1 file changed, 14 insertions(+), 15 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index 531a00d..831d824 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -1286,27 +1286,18 @@ static int vhost_net_open(struct inode *inode, struct file *f)
if (!n)
return -ENOMEM;
vqs = kmalloc_array(VHOST_NET_VQ_MAX, sizeof(*vqs), GFP_KERNEL);
- if (!vqs) {
- kvfree(n);
- return -ENOMEM;
- }
+ if (!vqs)
+ goto err_vqs;
queue = kmalloc_array(VHOST_NET_BATCH, sizeof(void *),
GFP_KERNEL);
- if (!queue) {
- kfree(vqs);
- kvfree(n);
- return -ENOMEM;
- }
+ if (!queue)
+ goto err_queue;
n->vqs[VHOST_NET_VQ_RX].rxq.queue = queue;
xdp = kmalloc_array(VHOST_NET_BATCH, sizeof(*xdp), GFP_KERNEL);
- if (!xdp) {
- kfree(vqs);
- kvfree(n);
- kfree(queue);
- return -ENOMEM;
- }
+ if (!xdp)
+ goto err_xdp;
n->vqs[VHOST_NET_VQ_TX].xdp = xdp;
dev = &n->dev;
@@ -1338,6 +1329,14 @@ static int vhost_net_open(struct inode *inode, struct file *f)
n->refcnt_bias = 0;
return 0;
+
+err_xdp:
+ kfree(queue);
+err_queue:
+ kfree(vqs);
+err_vqs:
+ kvfree(n);
+ return -ENOMEM;
}
static struct socket *vhost_net_stop_vq(struct vhost_net *n,
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
This is just a prep patch to get vhost_dev_init callers ready to handle
the next patch where the function can fail. In this patch vhost_dev_init
just returns 0, but I think it's easier to check for goto/error handling
errors separated from the next patch.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/net.c | 11 +++++++----
drivers/vhost/scsi.c | 7 +++++--
drivers/vhost/test.c | 9 +++++++--
drivers/vhost/vdpa.c | 6 ++++--
drivers/vhost/vhost.c | 14 ++++++++------
drivers/vhost/vhost.h | 10 +++++-----
drivers/vhost/vsock.c | 9 ++++++---
7 files changed, 42 insertions(+), 24 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index 831d824..fd30b53 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -1316,10 +1316,11 @@ static int vhost_net_open(struct inode *inode, struct file *f)
n->vqs[i].rx_ring = NULL;
vhost_net_buf_init(&n->vqs[i].rxq);
}
- vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX,
- UIO_MAXIOV + VHOST_NET_BATCH,
- VHOST_NET_PKT_WEIGHT, VHOST_NET_WEIGHT, true,
- NULL);
+ if (vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX,
+ UIO_MAXIOV + VHOST_NET_BATCH,
+ VHOST_NET_PKT_WEIGHT, VHOST_NET_WEIGHT, true,
+ NULL))
+ goto err_dev_init;
vhost_poll_init(n->poll + VHOST_NET_VQ_TX, handle_tx_net, EPOLLOUT, dev);
vhost_poll_init(n->poll + VHOST_NET_VQ_RX, handle_rx_net, EPOLLIN, dev);
@@ -1330,6 +1331,8 @@ static int vhost_net_open(struct inode *inode, struct file *f)
return 0;
+err_dev_init:
+ kfree(xdp);
err_xdp:
kfree(queue);
err_queue:
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 86617bb..63ba363 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1632,14 +1632,17 @@ static int vhost_scsi_open(struct inode *inode, struct file *f)
vqs[i] = &vs->vqs[i].vq;
vs->vqs[i].vq.handle_kick = vhost_scsi_handle_kick;
}
- vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, UIO_MAXIOV,
- VHOST_SCSI_WEIGHT, 0, true, NULL);
+ if (vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, UIO_MAXIOV,
+ VHOST_SCSI_WEIGHT, 0, true, NULL))
+ goto err_dev_init;
vhost_scsi_init_inflight(vs, NULL);
f->private_data = vs;
return 0;
+err_dev_init:
+ kfree(vqs);
err_vqs:
kvfree(vs);
err_vs:
diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c
index a09dedc..c255ae5 100644
--- a/drivers/vhost/test.c
+++ b/drivers/vhost/test.c
@@ -119,12 +119,17 @@ static int vhost_test_open(struct inode *inode, struct file *f)
dev = &n->dev;
vqs[VHOST_TEST_VQ] = &n->vqs[VHOST_TEST_VQ];
n->vqs[VHOST_TEST_VQ].handle_kick = handle_vq_kick;
- vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV,
- VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT, true, NULL);
+ if (vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV,
+ VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT, true, NULL)
+ goto err_dev_init;
f->private_data = n;
return 0;
+
+err_dev_init:
+ kfree(vqs);
+ return -ENOMEM;
}
static void *vhost_test_stop_vq(struct vhost_test *n,
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index 62a9bb0..d413ceb 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -817,8 +817,9 @@ static int vhost_vdpa_open(struct inode *inode, struct file *filep)
vqs[i] = &v->vqs[i];
vqs[i]->handle_kick = handle_vq_kick;
}
- vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
- vhost_vdpa_process_iotlb_msg);
+ if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
+ vhost_vdpa_process_iotlb_msg))
+ goto err_dev_init;
dev->iotlb = vhost_iotlb_alloc(0, 0);
if (!dev->iotlb) {
@@ -836,6 +837,7 @@ static int vhost_vdpa_open(struct inode *inode, struct file *filep)
err_init_iotlb:
vhost_dev_cleanup(&v->vdev);
+err_dev_init:
kfree(vqs);
err:
atomic_dec(&v->opened);
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index fbb66f6..b05e690 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -459,12 +459,12 @@ static size_t vhost_get_desc_size(struct vhost_virtqueue *vq,
return sizeof(*vq->desc) * num;
}
-void vhost_dev_init(struct vhost_dev *dev,
- struct vhost_virtqueue **vqs, int nvqs,
- int iov_limit, int weight, int byte_weight,
- bool use_worker,
- int (*msg_handler)(struct vhost_dev *dev,
- struct vhost_iotlb_msg *msg))
+int vhost_dev_init(struct vhost_dev *dev,
+ struct vhost_virtqueue **vqs, int nvqs,
+ int iov_limit, int weight, int byte_weight,
+ bool use_worker,
+ int (*msg_handler)(struct vhost_dev *dev,
+ struct vhost_iotlb_msg *msg))
{
struct vhost_virtqueue *vq;
int i;
@@ -501,6 +501,8 @@ void vhost_dev_init(struct vhost_dev *dev,
vhost_poll_init(&vq->poll, vq->handle_kick,
EPOLLIN, dev);
}
+
+ return 0;
}
EXPORT_SYMBOL_GPL(vhost_dev_init);
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 11db183..a053318 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -167,11 +167,11 @@ struct vhost_dev {
};
bool vhost_exceeds_weight(struct vhost_virtqueue *vq, int pkts, int total_len);
-void vhost_dev_init(struct vhost_dev *, struct vhost_virtqueue **vqs,
- int nvqs, int iov_limit, int weight, int byte_weight,
- bool use_worker,
- int (*msg_handler)(struct vhost_dev *dev,
- struct vhost_iotlb_msg *msg));
+int vhost_dev_init(struct vhost_dev *dev, struct vhost_virtqueue **vqs,
+ int nvqs, int iov_limit, int weight, int byte_weight,
+ bool use_worker,
+ int (*msg_handler)(struct vhost_dev *dev,
+ struct vhost_iotlb_msg *msg));
long vhost_dev_set_owner(struct vhost_dev *dev);
bool vhost_dev_has_owner(struct vhost_dev *dev);
long vhost_dev_check_owner(struct vhost_dev *);
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index f40205f..a1a35e1 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -630,9 +630,10 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file)
vsock->vqs[VSOCK_VQ_TX].handle_kick = vhost_vsock_handle_tx_kick;
vsock->vqs[VSOCK_VQ_RX].handle_kick = vhost_vsock_handle_rx_kick;
- vhost_dev_init(&vsock->dev, vqs, ARRAY_SIZE(vsock->vqs),
- UIO_MAXIOV, VHOST_VSOCK_PKT_WEIGHT,
- VHOST_VSOCK_WEIGHT, true, NULL);
+ if (vhost_dev_init(&vsock->dev, vqs, ARRAY_SIZE(vsock->vqs),
+ UIO_MAXIOV, VHOST_VSOCK_PKT_WEIGHT,
+ VHOST_VSOCK_WEIGHT, true, NULL))
+ goto err_dev_init;
file->private_data = vsock;
spin_lock_init(&vsock->send_pkt_list_lock);
@@ -640,6 +641,8 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file)
vhost_work_init(&vsock->send_pkt_work, vhost_transport_send_pkt_work);
return 0;
+err_dev_init:
+ kfree(vqs);
out:
vhost_vsock_free(vsock);
return ret;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
This is just a prep patch to get vhost_dev_init callers ready to handle
the next patch where the function can fail. In this patch vhost_dev_init
just returns 0, but I think it's easier to check for goto/error handling
errors separated from the next patch.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/net.c | 11 +++++++----
drivers/vhost/scsi.c | 7 +++++--
drivers/vhost/test.c | 9 +++++++--
drivers/vhost/vdpa.c | 6 ++++--
drivers/vhost/vhost.c | 14 ++++++++------
drivers/vhost/vhost.h | 10 +++++-----
drivers/vhost/vsock.c | 9 ++++++---
7 files changed, 42 insertions(+), 24 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index 831d824..fd30b53 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -1316,10 +1316,11 @@ static int vhost_net_open(struct inode *inode, struct file *f)
n->vqs[i].rx_ring = NULL;
vhost_net_buf_init(&n->vqs[i].rxq);
}
- vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX,
- UIO_MAXIOV + VHOST_NET_BATCH,
- VHOST_NET_PKT_WEIGHT, VHOST_NET_WEIGHT, true,
- NULL);
+ if (vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX,
+ UIO_MAXIOV + VHOST_NET_BATCH,
+ VHOST_NET_PKT_WEIGHT, VHOST_NET_WEIGHT, true,
+ NULL))
+ goto err_dev_init;
vhost_poll_init(n->poll + VHOST_NET_VQ_TX, handle_tx_net, EPOLLOUT, dev);
vhost_poll_init(n->poll + VHOST_NET_VQ_RX, handle_rx_net, EPOLLIN, dev);
@@ -1330,6 +1331,8 @@ static int vhost_net_open(struct inode *inode, struct file *f)
return 0;
+err_dev_init:
+ kfree(xdp);
err_xdp:
kfree(queue);
err_queue:
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 86617bb..63ba363 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1632,14 +1632,17 @@ static int vhost_scsi_open(struct inode *inode, struct file *f)
vqs[i] = &vs->vqs[i].vq;
vs->vqs[i].vq.handle_kick = vhost_scsi_handle_kick;
}
- vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, UIO_MAXIOV,
- VHOST_SCSI_WEIGHT, 0, true, NULL);
+ if (vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, UIO_MAXIOV,
+ VHOST_SCSI_WEIGHT, 0, true, NULL))
+ goto err_dev_init;
vhost_scsi_init_inflight(vs, NULL);
f->private_data = vs;
return 0;
+err_dev_init:
+ kfree(vqs);
err_vqs:
kvfree(vs);
err_vs:
diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c
index a09dedc..c255ae5 100644
--- a/drivers/vhost/test.c
+++ b/drivers/vhost/test.c
@@ -119,12 +119,17 @@ static int vhost_test_open(struct inode *inode, struct file *f)
dev = &n->dev;
vqs[VHOST_TEST_VQ] = &n->vqs[VHOST_TEST_VQ];
n->vqs[VHOST_TEST_VQ].handle_kick = handle_vq_kick;
- vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV,
- VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT, true, NULL);
+ if (vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV,
+ VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT, true, NULL)
+ goto err_dev_init;
f->private_data = n;
return 0;
+
+err_dev_init:
+ kfree(vqs);
+ return -ENOMEM;
}
static void *vhost_test_stop_vq(struct vhost_test *n,
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index 62a9bb0..d413ceb 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -817,8 +817,9 @@ static int vhost_vdpa_open(struct inode *inode, struct file *filep)
vqs[i] = &v->vqs[i];
vqs[i]->handle_kick = handle_vq_kick;
}
- vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
- vhost_vdpa_process_iotlb_msg);
+ if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
+ vhost_vdpa_process_iotlb_msg))
+ goto err_dev_init;
dev->iotlb = vhost_iotlb_alloc(0, 0);
if (!dev->iotlb) {
@@ -836,6 +837,7 @@ static int vhost_vdpa_open(struct inode *inode, struct file *filep)
err_init_iotlb:
vhost_dev_cleanup(&v->vdev);
+err_dev_init:
kfree(vqs);
err:
atomic_dec(&v->opened);
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index fbb66f6..b05e690 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -459,12 +459,12 @@ static size_t vhost_get_desc_size(struct vhost_virtqueue *vq,
return sizeof(*vq->desc) * num;
}
-void vhost_dev_init(struct vhost_dev *dev,
- struct vhost_virtqueue **vqs, int nvqs,
- int iov_limit, int weight, int byte_weight,
- bool use_worker,
- int (*msg_handler)(struct vhost_dev *dev,
- struct vhost_iotlb_msg *msg))
+int vhost_dev_init(struct vhost_dev *dev,
+ struct vhost_virtqueue **vqs, int nvqs,
+ int iov_limit, int weight, int byte_weight,
+ bool use_worker,
+ int (*msg_handler)(struct vhost_dev *dev,
+ struct vhost_iotlb_msg *msg))
{
struct vhost_virtqueue *vq;
int i;
@@ -501,6 +501,8 @@ void vhost_dev_init(struct vhost_dev *dev,
vhost_poll_init(&vq->poll, vq->handle_kick,
EPOLLIN, dev);
}
+
+ return 0;
}
EXPORT_SYMBOL_GPL(vhost_dev_init);
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 11db183..a053318 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -167,11 +167,11 @@ struct vhost_dev {
};
bool vhost_exceeds_weight(struct vhost_virtqueue *vq, int pkts, int total_len);
-void vhost_dev_init(struct vhost_dev *, struct vhost_virtqueue **vqs,
- int nvqs, int iov_limit, int weight, int byte_weight,
- bool use_worker,
- int (*msg_handler)(struct vhost_dev *dev,
- struct vhost_iotlb_msg *msg));
+int vhost_dev_init(struct vhost_dev *dev, struct vhost_virtqueue **vqs,
+ int nvqs, int iov_limit, int weight, int byte_weight,
+ bool use_worker,
+ int (*msg_handler)(struct vhost_dev *dev,
+ struct vhost_iotlb_msg *msg));
long vhost_dev_set_owner(struct vhost_dev *dev);
bool vhost_dev_has_owner(struct vhost_dev *dev);
long vhost_dev_check_owner(struct vhost_dev *);
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index f40205f..a1a35e1 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -630,9 +630,10 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file)
vsock->vqs[VSOCK_VQ_TX].handle_kick = vhost_vsock_handle_tx_kick;
vsock->vqs[VSOCK_VQ_RX].handle_kick = vhost_vsock_handle_rx_kick;
- vhost_dev_init(&vsock->dev, vqs, ARRAY_SIZE(vsock->vqs),
- UIO_MAXIOV, VHOST_VSOCK_PKT_WEIGHT,
- VHOST_VSOCK_WEIGHT, true, NULL);
+ if (vhost_dev_init(&vsock->dev, vqs, ARRAY_SIZE(vsock->vqs),
+ UIO_MAXIOV, VHOST_VSOCK_PKT_WEIGHT,
+ VHOST_VSOCK_WEIGHT, true, NULL))
+ goto err_dev_init;
file->private_data = vsock;
spin_lock_init(&vsock->send_pkt_list_lock);
@@ -640,6 +641,8 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file)
vhost_work_init(&vsock->send_pkt_work, vhost_transport_send_pkt_work);
return 0;
+err_dev_init:
+ kfree(vqs);
out:
vhost_vsock_free(vsock);
return ret;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
2020-10-07 20:54 ` Mike Christie
(?)
(?)
@ 2020-10-08 0:58 ` kernel test robot
-1 siblings, 0 replies; 69+ messages in thread
From: kernel test robot @ 2020-10-08 0:58 UTC (permalink / raw)
To: Mike Christie, martin.petersen, linux-scsi, target-devel, mst,
jasowang, pbonzini, stefanha, virtualization
Cc: kbuild-all, clang-built-linux
[-- Attachment #1: Type: text/plain, Size: 3952 bytes --]
Hi Mike,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on vhost/linux-next]
[also build test WARNING on next-20201007]
[cannot apply to v5.9-rc8]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: x86_64-randconfig-a016-20201008 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project 4d1d8ae7100ec3c7e1709addb7b3ec6f9ad0b44f)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/7dc4d1082d406f391238a1897cb030f33c382bc3
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
git checkout 7dc4d1082d406f391238a1897cb030f33c382bc3
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> drivers/vhost/vdpa.c:820:6: warning: variable 'r' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/vhost/vdpa.c:844:9: note: uninitialized use occurs here
return r;
^
drivers/vhost/vdpa.c:820:2: note: remove the 'if' if its condition is always false
if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/vhost/vdpa.c:798:16: note: initialize the variable 'r' to silence this warning
int nvqs, i, r, opened;
^
= 0
1 warning generated.
vim +820 drivers/vhost/vdpa.c
792
793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
794 {
795 struct vhost_vdpa *v;
796 struct vhost_dev *dev;
797 struct vhost_virtqueue **vqs;
798 int nvqs, i, r, opened;
799
800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
801
802 opened = atomic_cmpxchg(&v->opened, 0, 1);
803 if (opened)
804 return -EBUSY;
805
806 nvqs = v->nvqs;
807 vhost_vdpa_reset(v);
808
809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
810 if (!vqs) {
811 r = -ENOMEM;
812 goto err;
813 }
814
815 dev = &v->vdev;
816 for (i = 0; i < nvqs; i++) {
817 vqs[i] = &v->vqs[i];
818 vqs[i]->handle_kick = handle_vq_kick;
819 }
> 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
821 vhost_vdpa_process_iotlb_msg))
822 goto err_dev_init;
823
824 dev->iotlb = vhost_iotlb_alloc(0, 0);
825 if (!dev->iotlb) {
826 r = -ENOMEM;
827 goto err_init_iotlb;
828 }
829
830 r = vhost_vdpa_alloc_domain(v);
831 if (r)
832 goto err_init_iotlb;
833
834 filep->private_data = v;
835
836 return 0;
837
838 err_init_iotlb:
839 vhost_dev_cleanup(&v->vdev);
840 err_dev_init:
841 kfree(vqs);
842 err:
843 atomic_dec(&v->opened);
844 return r;
845 }
846
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 32180 bytes --]
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-08 0:58 ` kernel test robot
0 siblings, 0 replies; 69+ messages in thread
From: kernel test robot @ 2020-10-08 0:58 UTC (permalink / raw)
To: Mike Christie, martin.petersen, linux-scsi, target-devel, mst,
jasowang, pbonzini, stefanha, virtualization
Cc: clang-built-linux, kbuild-all
[-- Attachment #1: Type: text/plain, Size: 3952 bytes --]
Hi Mike,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on vhost/linux-next]
[also build test WARNING on next-20201007]
[cannot apply to v5.9-rc8]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: x86_64-randconfig-a016-20201008 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project 4d1d8ae7100ec3c7e1709addb7b3ec6f9ad0b44f)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/7dc4d1082d406f391238a1897cb030f33c382bc3
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
git checkout 7dc4d1082d406f391238a1897cb030f33c382bc3
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> drivers/vhost/vdpa.c:820:6: warning: variable 'r' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/vhost/vdpa.c:844:9: note: uninitialized use occurs here
return r;
^
drivers/vhost/vdpa.c:820:2: note: remove the 'if' if its condition is always false
if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/vhost/vdpa.c:798:16: note: initialize the variable 'r' to silence this warning
int nvqs, i, r, opened;
^
= 0
1 warning generated.
vim +820 drivers/vhost/vdpa.c
792
793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
794 {
795 struct vhost_vdpa *v;
796 struct vhost_dev *dev;
797 struct vhost_virtqueue **vqs;
798 int nvqs, i, r, opened;
799
800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
801
802 opened = atomic_cmpxchg(&v->opened, 0, 1);
803 if (opened)
804 return -EBUSY;
805
806 nvqs = v->nvqs;
807 vhost_vdpa_reset(v);
808
809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
810 if (!vqs) {
811 r = -ENOMEM;
812 goto err;
813 }
814
815 dev = &v->vdev;
816 for (i = 0; i < nvqs; i++) {
817 vqs[i] = &v->vqs[i];
818 vqs[i]->handle_kick = handle_vq_kick;
819 }
> 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
821 vhost_vdpa_process_iotlb_msg))
822 goto err_dev_init;
823
824 dev->iotlb = vhost_iotlb_alloc(0, 0);
825 if (!dev->iotlb) {
826 r = -ENOMEM;
827 goto err_init_iotlb;
828 }
829
830 r = vhost_vdpa_alloc_domain(v);
831 if (r)
832 goto err_init_iotlb;
833
834 filep->private_data = v;
835
836 return 0;
837
838 err_init_iotlb:
839 vhost_dev_cleanup(&v->vdev);
840 err_dev_init:
841 kfree(vqs);
842 err:
843 atomic_dec(&v->opened);
844 return r;
845 }
846
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 32180 bytes --]
[-- Attachment #3: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-08 0:58 ` kernel test robot
0 siblings, 0 replies; 69+ messages in thread
From: kernel test robot @ 2020-10-08 0:58 UTC (permalink / raw)
To: Mike Christie, martin.petersen, linux-scsi, target-devel, mst,
jasowang, pbonzini, stefanha, virtualization
Cc: kbuild-all, clang-built-linux
[-- Attachment #1: Type: text/plain, Size: 3952 bytes --]
Hi Mike,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on vhost/linux-next]
[also build test WARNING on next-20201007]
[cannot apply to v5.9-rc8]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: x86_64-randconfig-a016-20201008 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project 4d1d8ae7100ec3c7e1709addb7b3ec6f9ad0b44f)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/7dc4d1082d406f391238a1897cb030f33c382bc3
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
git checkout 7dc4d1082d406f391238a1897cb030f33c382bc3
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> drivers/vhost/vdpa.c:820:6: warning: variable 'r' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/vhost/vdpa.c:844:9: note: uninitialized use occurs here
return r;
^
drivers/vhost/vdpa.c:820:2: note: remove the 'if' if its condition is always false
if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/vhost/vdpa.c:798:16: note: initialize the variable 'r' to silence this warning
int nvqs, i, r, opened;
^
= 0
1 warning generated.
vim +820 drivers/vhost/vdpa.c
792
793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
794 {
795 struct vhost_vdpa *v;
796 struct vhost_dev *dev;
797 struct vhost_virtqueue **vqs;
798 int nvqs, i, r, opened;
799
800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
801
802 opened = atomic_cmpxchg(&v->opened, 0, 1);
803 if (opened)
804 return -EBUSY;
805
806 nvqs = v->nvqs;
807 vhost_vdpa_reset(v);
808
809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
810 if (!vqs) {
811 r = -ENOMEM;
812 goto err;
813 }
814
815 dev = &v->vdev;
816 for (i = 0; i < nvqs; i++) {
817 vqs[i] = &v->vqs[i];
818 vqs[i]->handle_kick = handle_vq_kick;
819 }
> 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
821 vhost_vdpa_process_iotlb_msg))
822 goto err_dev_init;
823
824 dev->iotlb = vhost_iotlb_alloc(0, 0);
825 if (!dev->iotlb) {
826 r = -ENOMEM;
827 goto err_init_iotlb;
828 }
829
830 r = vhost_vdpa_alloc_domain(v);
831 if (r)
832 goto err_init_iotlb;
833
834 filep->private_data = v;
835
836 return 0;
837
838 err_init_iotlb:
839 vhost_dev_cleanup(&v->vdev);
840 err_dev_init:
841 kfree(vqs);
842 err:
843 atomic_dec(&v->opened);
844 return r;
845 }
846
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 32180 bytes --]
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-08 0:58 ` kernel test robot
0 siblings, 0 replies; 69+ messages in thread
From: kernel test robot @ 2020-10-08 0:58 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 4062 bytes --]
Hi Mike,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on vhost/linux-next]
[also build test WARNING on next-20201007]
[cannot apply to v5.9-rc8]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: x86_64-randconfig-a016-20201008 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project 4d1d8ae7100ec3c7e1709addb7b3ec6f9ad0b44f)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/7dc4d1082d406f391238a1897cb030f33c382bc3
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
git checkout 7dc4d1082d406f391238a1897cb030f33c382bc3
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> drivers/vhost/vdpa.c:820:6: warning: variable 'r' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/vhost/vdpa.c:844:9: note: uninitialized use occurs here
return r;
^
drivers/vhost/vdpa.c:820:2: note: remove the 'if' if its condition is always false
if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/vhost/vdpa.c:798:16: note: initialize the variable 'r' to silence this warning
int nvqs, i, r, opened;
^
= 0
1 warning generated.
vim +820 drivers/vhost/vdpa.c
792
793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
794 {
795 struct vhost_vdpa *v;
796 struct vhost_dev *dev;
797 struct vhost_virtqueue **vqs;
798 int nvqs, i, r, opened;
799
800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
801
802 opened = atomic_cmpxchg(&v->opened, 0, 1);
803 if (opened)
804 return -EBUSY;
805
806 nvqs = v->nvqs;
807 vhost_vdpa_reset(v);
808
809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
810 if (!vqs) {
811 r = -ENOMEM;
812 goto err;
813 }
814
815 dev = &v->vdev;
816 for (i = 0; i < nvqs; i++) {
817 vqs[i] = &v->vqs[i];
818 vqs[i]->handle_kick = handle_vq_kick;
819 }
> 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
821 vhost_vdpa_process_iotlb_msg))
822 goto err_dev_init;
823
824 dev->iotlb = vhost_iotlb_alloc(0, 0);
825 if (!dev->iotlb) {
826 r = -ENOMEM;
827 goto err_init_iotlb;
828 }
829
830 r = vhost_vdpa_alloc_domain(v);
831 if (r)
832 goto err_init_iotlb;
833
834 filep->private_data = v;
835
836 return 0;
837
838 err_init_iotlb:
839 vhost_dev_cleanup(&v->vdev);
840 err_dev_init:
841 kfree(vqs);
842 err:
843 atomic_dec(&v->opened);
844 return r;
845 }
846
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 32180 bytes --]
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
2020-10-07 20:54 ` Mike Christie
` (2 preceding siblings ...)
(?)
@ 2020-10-09 11:41 ` Dan Carpenter
-1 siblings, 0 replies; 69+ messages in thread
From: Dan Carpenter @ 2020-10-09 11:41 UTC (permalink / raw)
To: kbuild, Mike Christie, martin.petersen, linux-scsi, target-devel,
mst, jasowang, pbonzini, stefanha, virtualization
Cc: lkp, kbuild-all
[-- Attachment #1: Type: text/plain, Size: 4484 bytes --]
Hi Mike,
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: x86_64-randconfig-m001-20201008 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
New smatch warnings:
drivers/vhost/vdpa.c:844 vhost_vdpa_open() error: uninitialized symbol 'r'.
Old smatch warnings:
drivers/vhost/vdpa.c:436 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
drivers/vhost/vdpa.c:489 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
vim +/r +844 drivers/vhost/vdpa.c
4c8cf31885f69e8 Tiwei Bie 2020-03-26 793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 794 {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 795 struct vhost_vdpa *v;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 796 struct vhost_dev *dev;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 797 struct vhost_virtqueue **vqs;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 798 int nvqs, i, r, opened;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 799
4c8cf31885f69e8 Tiwei Bie 2020-03-26 800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 801
4c8cf31885f69e8 Tiwei Bie 2020-03-26 802 opened = atomic_cmpxchg(&v->opened, 0, 1);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 803 if (opened)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 804 return -EBUSY;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 805
4c8cf31885f69e8 Tiwei Bie 2020-03-26 806 nvqs = v->nvqs;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 807 vhost_vdpa_reset(v);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 808
4c8cf31885f69e8 Tiwei Bie 2020-03-26 809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 810 if (!vqs) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 811 r = -ENOMEM;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 812 goto err;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 813 }
4c8cf31885f69e8 Tiwei Bie 2020-03-26 814
4c8cf31885f69e8 Tiwei Bie 2020-03-26 815 dev = &v->vdev;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 816 for (i = 0; i < nvqs; i++) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 817 vqs[i] = &v->vqs[i];
4c8cf31885f69e8 Tiwei Bie 2020-03-26 818 vqs[i]->handle_kick = handle_vq_kick;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 819 }
7dc4d1082d406f3 Mike Christie 2020-10-07 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
7dc4d1082d406f3 Mike Christie 2020-10-07 821 vhost_vdpa_process_iotlb_msg))
7dc4d1082d406f3 Mike Christie 2020-10-07 822 goto err_dev_init;
"r" not set on this error path.
4c8cf31885f69e8 Tiwei Bie 2020-03-26 823
4c8cf31885f69e8 Tiwei Bie 2020-03-26 824 dev->iotlb = vhost_iotlb_alloc(0, 0);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 825 if (!dev->iotlb) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 826 r = -ENOMEM;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 827 goto err_init_iotlb;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 828 }
4c8cf31885f69e8 Tiwei Bie 2020-03-26 829
4c8cf31885f69e8 Tiwei Bie 2020-03-26 830 r = vhost_vdpa_alloc_domain(v);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 831 if (r)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 832 goto err_init_iotlb;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 833
4c8cf31885f69e8 Tiwei Bie 2020-03-26 834 filep->private_data = v;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 835
4c8cf31885f69e8 Tiwei Bie 2020-03-26 836 return 0;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 837
4c8cf31885f69e8 Tiwei Bie 2020-03-26 838 err_init_iotlb:
4c8cf31885f69e8 Tiwei Bie 2020-03-26 839 vhost_dev_cleanup(&v->vdev);
7dc4d1082d406f3 Mike Christie 2020-10-07 840 err_dev_init:
37787e9f81e2e58 Mike Christie 2020-09-21 841 kfree(vqs);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 842 err:
4c8cf31885f69e8 Tiwei Bie 2020-03-26 843 atomic_dec(&v->opened);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 @844 return r;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 845 }
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 41419 bytes --]
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-09 11:41 ` Dan Carpenter
0 siblings, 0 replies; 69+ messages in thread
From: Dan Carpenter @ 2020-10-09 11:41 UTC (permalink / raw)
To: kbuild, Mike Christie, martin.petersen, linux-scsi, target-devel,
mst, jasowang, pbonzini, stefanha, virtualization
Cc: kbuild-all, lkp
[-- Attachment #1: Type: text/plain, Size: 4484 bytes --]
Hi Mike,
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: x86_64-randconfig-m001-20201008 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
New smatch warnings:
drivers/vhost/vdpa.c:844 vhost_vdpa_open() error: uninitialized symbol 'r'.
Old smatch warnings:
drivers/vhost/vdpa.c:436 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
drivers/vhost/vdpa.c:489 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
vim +/r +844 drivers/vhost/vdpa.c
4c8cf31885f69e8 Tiwei Bie 2020-03-26 793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 794 {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 795 struct vhost_vdpa *v;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 796 struct vhost_dev *dev;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 797 struct vhost_virtqueue **vqs;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 798 int nvqs, i, r, opened;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 799
4c8cf31885f69e8 Tiwei Bie 2020-03-26 800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 801
4c8cf31885f69e8 Tiwei Bie 2020-03-26 802 opened = atomic_cmpxchg(&v->opened, 0, 1);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 803 if (opened)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 804 return -EBUSY;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 805
4c8cf31885f69e8 Tiwei Bie 2020-03-26 806 nvqs = v->nvqs;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 807 vhost_vdpa_reset(v);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 808
4c8cf31885f69e8 Tiwei Bie 2020-03-26 809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 810 if (!vqs) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 811 r = -ENOMEM;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 812 goto err;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 813 }
4c8cf31885f69e8 Tiwei Bie 2020-03-26 814
4c8cf31885f69e8 Tiwei Bie 2020-03-26 815 dev = &v->vdev;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 816 for (i = 0; i < nvqs; i++) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 817 vqs[i] = &v->vqs[i];
4c8cf31885f69e8 Tiwei Bie 2020-03-26 818 vqs[i]->handle_kick = handle_vq_kick;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 819 }
7dc4d1082d406f3 Mike Christie 2020-10-07 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
7dc4d1082d406f3 Mike Christie 2020-10-07 821 vhost_vdpa_process_iotlb_msg))
7dc4d1082d406f3 Mike Christie 2020-10-07 822 goto err_dev_init;
"r" not set on this error path.
4c8cf31885f69e8 Tiwei Bie 2020-03-26 823
4c8cf31885f69e8 Tiwei Bie 2020-03-26 824 dev->iotlb = vhost_iotlb_alloc(0, 0);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 825 if (!dev->iotlb) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 826 r = -ENOMEM;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 827 goto err_init_iotlb;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 828 }
4c8cf31885f69e8 Tiwei Bie 2020-03-26 829
4c8cf31885f69e8 Tiwei Bie 2020-03-26 830 r = vhost_vdpa_alloc_domain(v);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 831 if (r)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 832 goto err_init_iotlb;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 833
4c8cf31885f69e8 Tiwei Bie 2020-03-26 834 filep->private_data = v;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 835
4c8cf31885f69e8 Tiwei Bie 2020-03-26 836 return 0;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 837
4c8cf31885f69e8 Tiwei Bie 2020-03-26 838 err_init_iotlb:
4c8cf31885f69e8 Tiwei Bie 2020-03-26 839 vhost_dev_cleanup(&v->vdev);
7dc4d1082d406f3 Mike Christie 2020-10-07 840 err_dev_init:
37787e9f81e2e58 Mike Christie 2020-09-21 841 kfree(vqs);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 842 err:
4c8cf31885f69e8 Tiwei Bie 2020-03-26 843 atomic_dec(&v->opened);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 @844 return r;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 845 }
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 41419 bytes --]
[-- Attachment #3: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-09 11:41 ` Dan Carpenter
0 siblings, 0 replies; 69+ messages in thread
From: Dan Carpenter @ 2020-10-09 11:41 UTC (permalink / raw)
To: kbuild, Mike Christie, martin.petersen, linux-scsi, target-devel,
mst, jasowang, pbonzini, stefanha, virtualization
Cc: lkp, kbuild-all
[-- Attachment #1: Type: text/plain, Size: 4484 bytes --]
Hi Mike,
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: x86_64-randconfig-m001-20201008 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
New smatch warnings:
drivers/vhost/vdpa.c:844 vhost_vdpa_open() error: uninitialized symbol 'r'.
Old smatch warnings:
drivers/vhost/vdpa.c:436 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
drivers/vhost/vdpa.c:489 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
vim +/r +844 drivers/vhost/vdpa.c
4c8cf31885f69e8 Tiwei Bie 2020-03-26 793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 794 {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 795 struct vhost_vdpa *v;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 796 struct vhost_dev *dev;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 797 struct vhost_virtqueue **vqs;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 798 int nvqs, i, r, opened;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 799
4c8cf31885f69e8 Tiwei Bie 2020-03-26 800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 801
4c8cf31885f69e8 Tiwei Bie 2020-03-26 802 opened = atomic_cmpxchg(&v->opened, 0, 1);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 803 if (opened)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 804 return -EBUSY;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 805
4c8cf31885f69e8 Tiwei Bie 2020-03-26 806 nvqs = v->nvqs;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 807 vhost_vdpa_reset(v);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 808
4c8cf31885f69e8 Tiwei Bie 2020-03-26 809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 810 if (!vqs) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 811 r = -ENOMEM;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 812 goto err;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 813 }
4c8cf31885f69e8 Tiwei Bie 2020-03-26 814
4c8cf31885f69e8 Tiwei Bie 2020-03-26 815 dev = &v->vdev;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 816 for (i = 0; i < nvqs; i++) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 817 vqs[i] = &v->vqs[i];
4c8cf31885f69e8 Tiwei Bie 2020-03-26 818 vqs[i]->handle_kick = handle_vq_kick;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 819 }
7dc4d1082d406f3 Mike Christie 2020-10-07 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
7dc4d1082d406f3 Mike Christie 2020-10-07 821 vhost_vdpa_process_iotlb_msg))
7dc4d1082d406f3 Mike Christie 2020-10-07 822 goto err_dev_init;
"r" not set on this error path.
4c8cf31885f69e8 Tiwei Bie 2020-03-26 823
4c8cf31885f69e8 Tiwei Bie 2020-03-26 824 dev->iotlb = vhost_iotlb_alloc(0, 0);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 825 if (!dev->iotlb) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 826 r = -ENOMEM;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 827 goto err_init_iotlb;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 828 }
4c8cf31885f69e8 Tiwei Bie 2020-03-26 829
4c8cf31885f69e8 Tiwei Bie 2020-03-26 830 r = vhost_vdpa_alloc_domain(v);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 831 if (r)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 832 goto err_init_iotlb;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 833
4c8cf31885f69e8 Tiwei Bie 2020-03-26 834 filep->private_data = v;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 835
4c8cf31885f69e8 Tiwei Bie 2020-03-26 836 return 0;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 837
4c8cf31885f69e8 Tiwei Bie 2020-03-26 838 err_init_iotlb:
4c8cf31885f69e8 Tiwei Bie 2020-03-26 839 vhost_dev_cleanup(&v->vdev);
7dc4d1082d406f3 Mike Christie 2020-10-07 840 err_dev_init:
37787e9f81e2e58 Mike Christie 2020-09-21 841 kfree(vqs);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 842 err:
4c8cf31885f69e8 Tiwei Bie 2020-03-26 843 atomic_dec(&v->opened);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 @844 return r;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 845 }
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 41419 bytes --]
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-09 11:41 ` Dan Carpenter
0 siblings, 0 replies; 69+ messages in thread
From: Dan Carpenter @ 2020-10-09 11:41 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 4566 bytes --]
Hi Mike,
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: x86_64-randconfig-m001-20201008 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
New smatch warnings:
drivers/vhost/vdpa.c:844 vhost_vdpa_open() error: uninitialized symbol 'r'.
Old smatch warnings:
drivers/vhost/vdpa.c:436 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
drivers/vhost/vdpa.c:489 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
vim +/r +844 drivers/vhost/vdpa.c
4c8cf31885f69e8 Tiwei Bie 2020-03-26 793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 794 {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 795 struct vhost_vdpa *v;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 796 struct vhost_dev *dev;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 797 struct vhost_virtqueue **vqs;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 798 int nvqs, i, r, opened;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 799
4c8cf31885f69e8 Tiwei Bie 2020-03-26 800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 801
4c8cf31885f69e8 Tiwei Bie 2020-03-26 802 opened = atomic_cmpxchg(&v->opened, 0, 1);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 803 if (opened)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 804 return -EBUSY;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 805
4c8cf31885f69e8 Tiwei Bie 2020-03-26 806 nvqs = v->nvqs;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 807 vhost_vdpa_reset(v);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 808
4c8cf31885f69e8 Tiwei Bie 2020-03-26 809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 810 if (!vqs) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 811 r = -ENOMEM;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 812 goto err;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 813 }
4c8cf31885f69e8 Tiwei Bie 2020-03-26 814
4c8cf31885f69e8 Tiwei Bie 2020-03-26 815 dev = &v->vdev;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 816 for (i = 0; i < nvqs; i++) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 817 vqs[i] = &v->vqs[i];
4c8cf31885f69e8 Tiwei Bie 2020-03-26 818 vqs[i]->handle_kick = handle_vq_kick;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 819 }
7dc4d1082d406f3 Mike Christie 2020-10-07 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
7dc4d1082d406f3 Mike Christie 2020-10-07 821 vhost_vdpa_process_iotlb_msg))
7dc4d1082d406f3 Mike Christie 2020-10-07 822 goto err_dev_init;
"r" not set on this error path.
4c8cf31885f69e8 Tiwei Bie 2020-03-26 823
4c8cf31885f69e8 Tiwei Bie 2020-03-26 824 dev->iotlb = vhost_iotlb_alloc(0, 0);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 825 if (!dev->iotlb) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 826 r = -ENOMEM;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 827 goto err_init_iotlb;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 828 }
4c8cf31885f69e8 Tiwei Bie 2020-03-26 829
4c8cf31885f69e8 Tiwei Bie 2020-03-26 830 r = vhost_vdpa_alloc_domain(v);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 831 if (r)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 832 goto err_init_iotlb;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 833
4c8cf31885f69e8 Tiwei Bie 2020-03-26 834 filep->private_data = v;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 835
4c8cf31885f69e8 Tiwei Bie 2020-03-26 836 return 0;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 837
4c8cf31885f69e8 Tiwei Bie 2020-03-26 838 err_init_iotlb:
4c8cf31885f69e8 Tiwei Bie 2020-03-26 839 vhost_dev_cleanup(&v->vdev);
7dc4d1082d406f3 Mike Christie 2020-10-07 840 err_dev_init:
37787e9f81e2e58 Mike Christie 2020-09-21 841 kfree(vqs);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 842 err:
4c8cf31885f69e8 Tiwei Bie 2020-03-26 843 atomic_dec(&v->opened);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 @844 return r;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 845 }
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 41419 bytes --]
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-09 11:41 ` Dan Carpenter
0 siblings, 0 replies; 69+ messages in thread
From: Dan Carpenter @ 2020-10-09 11:41 UTC (permalink / raw)
To: kbuild
[-- Attachment #1: Type: text/plain, Size: 4566 bytes --]
Hi Mike,
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: x86_64-randconfig-m001-20201008 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
New smatch warnings:
drivers/vhost/vdpa.c:844 vhost_vdpa_open() error: uninitialized symbol 'r'.
Old smatch warnings:
drivers/vhost/vdpa.c:436 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
drivers/vhost/vdpa.c:489 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
vim +/r +844 drivers/vhost/vdpa.c
4c8cf31885f69e8 Tiwei Bie 2020-03-26 793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 794 {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 795 struct vhost_vdpa *v;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 796 struct vhost_dev *dev;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 797 struct vhost_virtqueue **vqs;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 798 int nvqs, i, r, opened;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 799
4c8cf31885f69e8 Tiwei Bie 2020-03-26 800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 801
4c8cf31885f69e8 Tiwei Bie 2020-03-26 802 opened = atomic_cmpxchg(&v->opened, 0, 1);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 803 if (opened)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 804 return -EBUSY;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 805
4c8cf31885f69e8 Tiwei Bie 2020-03-26 806 nvqs = v->nvqs;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 807 vhost_vdpa_reset(v);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 808
4c8cf31885f69e8 Tiwei Bie 2020-03-26 809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 810 if (!vqs) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 811 r = -ENOMEM;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 812 goto err;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 813 }
4c8cf31885f69e8 Tiwei Bie 2020-03-26 814
4c8cf31885f69e8 Tiwei Bie 2020-03-26 815 dev = &v->vdev;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 816 for (i = 0; i < nvqs; i++) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 817 vqs[i] = &v->vqs[i];
4c8cf31885f69e8 Tiwei Bie 2020-03-26 818 vqs[i]->handle_kick = handle_vq_kick;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 819 }
7dc4d1082d406f3 Mike Christie 2020-10-07 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
7dc4d1082d406f3 Mike Christie 2020-10-07 821 vhost_vdpa_process_iotlb_msg))
7dc4d1082d406f3 Mike Christie 2020-10-07 822 goto err_dev_init;
"r" not set on this error path.
4c8cf31885f69e8 Tiwei Bie 2020-03-26 823
4c8cf31885f69e8 Tiwei Bie 2020-03-26 824 dev->iotlb = vhost_iotlb_alloc(0, 0);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 825 if (!dev->iotlb) {
4c8cf31885f69e8 Tiwei Bie 2020-03-26 826 r = -ENOMEM;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 827 goto err_init_iotlb;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 828 }
4c8cf31885f69e8 Tiwei Bie 2020-03-26 829
4c8cf31885f69e8 Tiwei Bie 2020-03-26 830 r = vhost_vdpa_alloc_domain(v);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 831 if (r)
4c8cf31885f69e8 Tiwei Bie 2020-03-26 832 goto err_init_iotlb;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 833
4c8cf31885f69e8 Tiwei Bie 2020-03-26 834 filep->private_data = v;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 835
4c8cf31885f69e8 Tiwei Bie 2020-03-26 836 return 0;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 837
4c8cf31885f69e8 Tiwei Bie 2020-03-26 838 err_init_iotlb:
4c8cf31885f69e8 Tiwei Bie 2020-03-26 839 vhost_dev_cleanup(&v->vdev);
7dc4d1082d406f3 Mike Christie 2020-10-07 840 err_dev_init:
37787e9f81e2e58 Mike Christie 2020-09-21 841 kfree(vqs);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 842 err:
4c8cf31885f69e8 Tiwei Bie 2020-03-26 843 atomic_dec(&v->opened);
4c8cf31885f69e8 Tiwei Bie 2020-03-26 @844 return r;
4c8cf31885f69e8 Tiwei Bie 2020-03-26 845 }
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 41419 bytes --]
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
2020-10-09 11:41 ` Dan Carpenter
(?)
(?)
@ 2020-10-23 15:56 ` Michael S. Tsirkin
-1 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-23 15:56 UTC (permalink / raw)
To: Dan Carpenter
Cc: kbuild, Mike Christie, martin.petersen, linux-scsi, target-devel,
jasowang, pbonzini, stefanha, virtualization, lkp, kbuild-all
On Fri, Oct 09, 2020 at 02:41:26PM +0300, Dan Carpenter wrote:
> Hi Mike,
>
> url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
> base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
> config: x86_64-randconfig-m001-20201008 (attached as .config)
> compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
>
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
> Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
>
> New smatch warnings:
> drivers/vhost/vdpa.c:844 vhost_vdpa_open() error: uninitialized symbol 'r'.
>
> Old smatch warnings:
> drivers/vhost/vdpa.c:436 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
> drivers/vhost/vdpa.c:489 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
>
> vim +/r +844 drivers/vhost/vdpa.c
>
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 794 {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 795 struct vhost_vdpa *v;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 796 struct vhost_dev *dev;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 797 struct vhost_virtqueue **vqs;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 798 int nvqs, i, r, opened;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 799
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 801
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 802 opened = atomic_cmpxchg(&v->opened, 0, 1);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 803 if (opened)
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 804 return -EBUSY;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 805
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 806 nvqs = v->nvqs;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 807 vhost_vdpa_reset(v);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 808
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 810 if (!vqs) {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 811 r = -ENOMEM;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 812 goto err;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 813 }
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 814
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 815 dev = &v->vdev;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 816 for (i = 0; i < nvqs; i++) {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 817 vqs[i] = &v->vqs[i];
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 818 vqs[i]->handle_kick = handle_vq_kick;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 819 }
> 7dc4d1082d406f3 Mike Christie 2020-10-07 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
> 7dc4d1082d406f3 Mike Christie 2020-10-07 821 vhost_vdpa_process_iotlb_msg))
> 7dc4d1082d406f3 Mike Christie 2020-10-07 822 goto err_dev_init;
>
> "r" not set on this error path.
>
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 823
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 824 dev->iotlb = vhost_iotlb_alloc(0, 0);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 825 if (!dev->iotlb) {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 826 r = -ENOMEM;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 827 goto err_init_iotlb;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 828 }
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 829
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 830 r = vhost_vdpa_alloc_domain(v);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 831 if (r)
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 832 goto err_init_iotlb;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 833
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 834 filep->private_data = v;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 835
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 836 return 0;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 837
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 838 err_init_iotlb:
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 839 vhost_dev_cleanup(&v->vdev);
> 7dc4d1082d406f3 Mike Christie 2020-10-07 840 err_dev_init:
> 37787e9f81e2e58 Mike Christie 2020-09-21 841 kfree(vqs);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 842 err:
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 843 atomic_dec(&v->opened);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 @844 return r;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 845 }
Yes looks like it would use r uninitialized ...
Mike?
> ---
> 0-DAY CI Kernel Test Service, Intel Corporation
> https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-23 15:56 ` Michael S. Tsirkin
0 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-23 15:56 UTC (permalink / raw)
To: Dan Carpenter
Cc: kbuild-all, linux-scsi, martin.petersen, virtualization,
target-devel, stefanha, pbonzini, kbuild, Mike Christie, lkp
On Fri, Oct 09, 2020 at 02:41:26PM +0300, Dan Carpenter wrote:
> Hi Mike,
>
> url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
> base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
> config: x86_64-randconfig-m001-20201008 (attached as .config)
> compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
>
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
> Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
>
> New smatch warnings:
> drivers/vhost/vdpa.c:844 vhost_vdpa_open() error: uninitialized symbol 'r'.
>
> Old smatch warnings:
> drivers/vhost/vdpa.c:436 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
> drivers/vhost/vdpa.c:489 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
>
> vim +/r +844 drivers/vhost/vdpa.c
>
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 794 {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 795 struct vhost_vdpa *v;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 796 struct vhost_dev *dev;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 797 struct vhost_virtqueue **vqs;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 798 int nvqs, i, r, opened;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 799
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 801
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 802 opened = atomic_cmpxchg(&v->opened, 0, 1);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 803 if (opened)
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 804 return -EBUSY;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 805
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 806 nvqs = v->nvqs;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 807 vhost_vdpa_reset(v);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 808
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 810 if (!vqs) {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 811 r = -ENOMEM;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 812 goto err;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 813 }
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 814
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 815 dev = &v->vdev;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 816 for (i = 0; i < nvqs; i++) {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 817 vqs[i] = &v->vqs[i];
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 818 vqs[i]->handle_kick = handle_vq_kick;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 819 }
> 7dc4d1082d406f3 Mike Christie 2020-10-07 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
> 7dc4d1082d406f3 Mike Christie 2020-10-07 821 vhost_vdpa_process_iotlb_msg))
> 7dc4d1082d406f3 Mike Christie 2020-10-07 822 goto err_dev_init;
>
> "r" not set on this error path.
>
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 823
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 824 dev->iotlb = vhost_iotlb_alloc(0, 0);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 825 if (!dev->iotlb) {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 826 r = -ENOMEM;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 827 goto err_init_iotlb;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 828 }
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 829
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 830 r = vhost_vdpa_alloc_domain(v);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 831 if (r)
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 832 goto err_init_iotlb;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 833
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 834 filep->private_data = v;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 835
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 836 return 0;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 837
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 838 err_init_iotlb:
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 839 vhost_dev_cleanup(&v->vdev);
> 7dc4d1082d406f3 Mike Christie 2020-10-07 840 err_dev_init:
> 37787e9f81e2e58 Mike Christie 2020-09-21 841 kfree(vqs);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 842 err:
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 843 atomic_dec(&v->opened);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 @844 return r;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 845 }
Yes looks like it would use r uninitialized ...
Mike?
> ---
> 0-DAY CI Kernel Test Service, Intel Corporation
> https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-23 15:56 ` Michael S. Tsirkin
0 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-23 15:56 UTC (permalink / raw)
To: Dan Carpenter
Cc: kbuild, Mike Christie, martin.petersen, linux-scsi, target-devel,
jasowang, pbonzini, stefanha, virtualization, lkp, kbuild-all
On Fri, Oct 09, 2020 at 02:41:26PM +0300, Dan Carpenter wrote:
> Hi Mike,
>
> url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
> base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
> config: x86_64-randconfig-m001-20201008 (attached as .config)
> compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
>
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
> Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
>
> New smatch warnings:
> drivers/vhost/vdpa.c:844 vhost_vdpa_open() error: uninitialized symbol 'r'.
>
> Old smatch warnings:
> drivers/vhost/vdpa.c:436 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
> drivers/vhost/vdpa.c:489 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
>
> vim +/r +844 drivers/vhost/vdpa.c
>
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 794 {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 795 struct vhost_vdpa *v;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 796 struct vhost_dev *dev;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 797 struct vhost_virtqueue **vqs;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 798 int nvqs, i, r, opened;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 799
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 801
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 802 opened = atomic_cmpxchg(&v->opened, 0, 1);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 803 if (opened)
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 804 return -EBUSY;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 805
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 806 nvqs = v->nvqs;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 807 vhost_vdpa_reset(v);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 808
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 810 if (!vqs) {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 811 r = -ENOMEM;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 812 goto err;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 813 }
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 814
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 815 dev = &v->vdev;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 816 for (i = 0; i < nvqs; i++) {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 817 vqs[i] = &v->vqs[i];
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 818 vqs[i]->handle_kick = handle_vq_kick;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 819 }
> 7dc4d1082d406f3 Mike Christie 2020-10-07 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
> 7dc4d1082d406f3 Mike Christie 2020-10-07 821 vhost_vdpa_process_iotlb_msg))
> 7dc4d1082d406f3 Mike Christie 2020-10-07 822 goto err_dev_init;
>
> "r" not set on this error path.
>
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 823
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 824 dev->iotlb = vhost_iotlb_alloc(0, 0);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 825 if (!dev->iotlb) {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 826 r = -ENOMEM;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 827 goto err_init_iotlb;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 828 }
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 829
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 830 r = vhost_vdpa_alloc_domain(v);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 831 if (r)
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 832 goto err_init_iotlb;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 833
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 834 filep->private_data = v;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 835
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 836 return 0;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 837
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 838 err_init_iotlb:
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 839 vhost_dev_cleanup(&v->vdev);
> 7dc4d1082d406f3 Mike Christie 2020-10-07 840 err_dev_init:
> 37787e9f81e2e58 Mike Christie 2020-09-21 841 kfree(vqs);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 842 err:
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 843 atomic_dec(&v->opened);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 @844 return r;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 845 }
Yes looks like it would use r uninitialized ...
Mike?
> ---
> 0-DAY CI Kernel Test Service, Intel Corporation
> https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-23 15:56 ` Michael S. Tsirkin
0 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-23 15:56 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 4850 bytes --]
On Fri, Oct 09, 2020 at 02:41:26PM +0300, Dan Carpenter wrote:
> Hi Mike,
>
> url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
> base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
> config: x86_64-randconfig-m001-20201008 (attached as .config)
> compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
>
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
> Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
>
> New smatch warnings:
> drivers/vhost/vdpa.c:844 vhost_vdpa_open() error: uninitialized symbol 'r'.
>
> Old smatch warnings:
> drivers/vhost/vdpa.c:436 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
> drivers/vhost/vdpa.c:489 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
>
> vim +/r +844 drivers/vhost/vdpa.c
>
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 794 {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 795 struct vhost_vdpa *v;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 796 struct vhost_dev *dev;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 797 struct vhost_virtqueue **vqs;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 798 int nvqs, i, r, opened;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 799
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 801
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 802 opened = atomic_cmpxchg(&v->opened, 0, 1);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 803 if (opened)
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 804 return -EBUSY;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 805
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 806 nvqs = v->nvqs;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 807 vhost_vdpa_reset(v);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 808
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 810 if (!vqs) {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 811 r = -ENOMEM;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 812 goto err;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 813 }
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 814
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 815 dev = &v->vdev;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 816 for (i = 0; i < nvqs; i++) {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 817 vqs[i] = &v->vqs[i];
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 818 vqs[i]->handle_kick = handle_vq_kick;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 819 }
> 7dc4d1082d406f3 Mike Christie 2020-10-07 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
> 7dc4d1082d406f3 Mike Christie 2020-10-07 821 vhost_vdpa_process_iotlb_msg))
> 7dc4d1082d406f3 Mike Christie 2020-10-07 822 goto err_dev_init;
>
> "r" not set on this error path.
>
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 823
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 824 dev->iotlb = vhost_iotlb_alloc(0, 0);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 825 if (!dev->iotlb) {
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 826 r = -ENOMEM;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 827 goto err_init_iotlb;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 828 }
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 829
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 830 r = vhost_vdpa_alloc_domain(v);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 831 if (r)
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 832 goto err_init_iotlb;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 833
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 834 filep->private_data = v;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 835
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 836 return 0;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 837
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 838 err_init_iotlb:
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 839 vhost_dev_cleanup(&v->vdev);
> 7dc4d1082d406f3 Mike Christie 2020-10-07 840 err_dev_init:
> 37787e9f81e2e58 Mike Christie 2020-09-21 841 kfree(vqs);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 842 err:
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 843 atomic_dec(&v->opened);
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 @844 return r;
> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 845 }
Yes looks like it would use r uninitialized ...
Mike?
> ---
> 0-DAY CI Kernel Test Service, Intel Corporation
> https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
2020-10-23 15:56 ` Michael S. Tsirkin
(?)
@ 2020-10-23 16:21 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-23 16:21 UTC (permalink / raw)
To: Michael S. Tsirkin, Dan Carpenter
Cc: kbuild, martin.petersen, linux-scsi, target-devel, jasowang,
pbonzini, stefanha, virtualization, lkp, kbuild-all
On 10/23/20 10:56 AM, Michael S. Tsirkin wrote:
> On Fri, Oct 09, 2020 at 02:41:26PM +0300, Dan Carpenter wrote:
>> Hi Mike,
>>
>> url: https://urldefense.com/v3/__https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802__;!!GqivPVa7Brio!PSUeg8MO8B2TLNpewKuGd0oWY8N3pkO7w9hbCh3xgWK3TgFsPr78zcIUZ8Orgxgaqydf$
>> base: https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git__;!!GqivPVa7Brio!PSUeg8MO8B2TLNpewKuGd0oWY8N3pkO7w9hbCh3xgWK3TgFsPr78zcIUZ8Org7WbKd27$ linux-next
>> config: x86_64-randconfig-m001-20201008 (attached as .config)
>> compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
>>
>> If you fix the issue, kindly add following tag as appropriate
>> Reported-by: kernel test robot <lkp@intel.com>
>> Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
>>
>> New smatch warnings:
>> drivers/vhost/vdpa.c:844 vhost_vdpa_open() error: uninitialized symbol 'r'.
>>
>> Old smatch warnings:
>> drivers/vhost/vdpa.c:436 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
>> drivers/vhost/vdpa.c:489 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
>>
>> vim +/r +844 drivers/vhost/vdpa.c
>>
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 794 {
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 795 struct vhost_vdpa *v;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 796 struct vhost_dev *dev;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 797 struct vhost_virtqueue **vqs;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 798 int nvqs, i, r, opened;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 799
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 801
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 802 opened = atomic_cmpxchg(&v->opened, 0, 1);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 803 if (opened)
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 804 return -EBUSY;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 805
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 806 nvqs = v->nvqs;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 807 vhost_vdpa_reset(v);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 808
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 810 if (!vqs) {
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 811 r = -ENOMEM;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 812 goto err;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 813 }
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 814
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 815 dev = &v->vdev;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 816 for (i = 0; i < nvqs; i++) {
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 817 vqs[i] = &v->vqs[i];
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 818 vqs[i]->handle_kick = handle_vq_kick;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 819 }
>> 7dc4d1082d406f3 Mike Christie 2020-10-07 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
>> 7dc4d1082d406f3 Mike Christie 2020-10-07 821 vhost_vdpa_process_iotlb_msg))
>> 7dc4d1082d406f3 Mike Christie 2020-10-07 822 goto err_dev_init;
>>
>> "r" not set on this error path.
>>
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 823
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 824 dev->iotlb = vhost_iotlb_alloc(0, 0);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 825 if (!dev->iotlb) {
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 826 r = -ENOMEM;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 827 goto err_init_iotlb;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 828 }
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 829
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 830 r = vhost_vdpa_alloc_domain(v);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 831 if (r)
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 832 goto err_init_iotlb;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 833
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 834 filep->private_data = v;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 835
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 836 return 0;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 837
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 838 err_init_iotlb:
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 839 vhost_dev_cleanup(&v->vdev);
>> 7dc4d1082d406f3 Mike Christie 2020-10-07 840 err_dev_init:
>> 37787e9f81e2e58 Mike Christie 2020-09-21 841 kfree(vqs);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 842 err:
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 843 atomic_dec(&v->opened);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 @844 return r;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 845 }
>
>
> Yes looks like it would use r uninitialized ...
> Mike?
>
Ah sorry, I had posted a v3 of this patchset:
https://patchwork.kernel.org/project/target-devel/list/?series=368487
and I fixed that but there was another cases of uninitialized variable
that I missed. I fixed that up now, but have not posted an updated set.
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-23 16:21 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-23 16:21 UTC (permalink / raw)
To: Michael S. Tsirkin, Dan Carpenter
Cc: kbuild, martin.petersen, linux-scsi, target-devel, jasowang,
pbonzini, stefanha, virtualization, lkp, kbuild-all
On 10/23/20 10:56 AM, Michael S. Tsirkin wrote:
> On Fri, Oct 09, 2020 at 02:41:26PM +0300, Dan Carpenter wrote:
>> Hi Mike,
>>
>> url: https://urldefense.com/v3/__https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802__;!!GqivPVa7Brio!PSUeg8MO8B2TLNpewKuGd0oWY8N3pkO7w9hbCh3xgWK3TgFsPr78zcIUZ8Orgxgaqydf$
>> base: https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git__;!!GqivPVa7Brio!PSUeg8MO8B2TLNpewKuGd0oWY8N3pkO7w9hbCh3xgWK3TgFsPr78zcIUZ8Org7WbKd27$ linux-next
>> config: x86_64-randconfig-m001-20201008 (attached as .config)
>> compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
>>
>> If you fix the issue, kindly add following tag as appropriate
>> Reported-by: kernel test robot <lkp@intel.com>
>> Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
>>
>> New smatch warnings:
>> drivers/vhost/vdpa.c:844 vhost_vdpa_open() error: uninitialized symbol 'r'.
>>
>> Old smatch warnings:
>> drivers/vhost/vdpa.c:436 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
>> drivers/vhost/vdpa.c:489 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
>>
>> vim +/r +844 drivers/vhost/vdpa.c
>>
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 794 {
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 795 struct vhost_vdpa *v;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 796 struct vhost_dev *dev;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 797 struct vhost_virtqueue **vqs;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 798 int nvqs, i, r, opened;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 799
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 801
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 802 opened = atomic_cmpxchg(&v->opened, 0, 1);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 803 if (opened)
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 804 return -EBUSY;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 805
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 806 nvqs = v->nvqs;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 807 vhost_vdpa_reset(v);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 808
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 810 if (!vqs) {
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 811 r = -ENOMEM;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 812 goto err;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 813 }
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 814
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 815 dev = &v->vdev;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 816 for (i = 0; i < nvqs; i++) {
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 817 vqs[i] = &v->vqs[i];
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 818 vqs[i]->handle_kick = handle_vq_kick;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 819 }
>> 7dc4d1082d406f3 Mike Christie 2020-10-07 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
>> 7dc4d1082d406f3 Mike Christie 2020-10-07 821 vhost_vdpa_process_iotlb_msg))
>> 7dc4d1082d406f3 Mike Christie 2020-10-07 822 goto err_dev_init;
>>
>> "r" not set on this error path.
>>
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 823
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 824 dev->iotlb = vhost_iotlb_alloc(0, 0);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 825 if (!dev->iotlb) {
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 826 r = -ENOMEM;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 827 goto err_init_iotlb;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 828 }
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 829
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 830 r = vhost_vdpa_alloc_domain(v);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 831 if (r)
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 832 goto err_init_iotlb;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 833
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 834 filep->private_data = v;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 835
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 836 return 0;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 837
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 838 err_init_iotlb:
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 839 vhost_dev_cleanup(&v->vdev);
>> 7dc4d1082d406f3 Mike Christie 2020-10-07 840 err_dev_init:
>> 37787e9f81e2e58 Mike Christie 2020-09-21 841 kfree(vqs);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 842 err:
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 843 atomic_dec(&v->opened);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 @844 return r;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 845 }
>
>
> Yes looks like it would use r uninitialized ...
> Mike?
>
Ah sorry, I had posted a v3 of this patchset:
https://patchwork.kernel.org/project/target-devel/list/?series68487
and I fixed that but there was another cases of uninitialized variable
that I missed. I fixed that up now, but have not posted an updated set.
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures
@ 2020-10-23 16:21 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-23 16:21 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 5332 bytes --]
On 10/23/20 10:56 AM, Michael S. Tsirkin wrote:
> On Fri, Oct 09, 2020 at 02:41:26PM +0300, Dan Carpenter wrote:
>> Hi Mike,
>>
>> url: https://urldefense.com/v3/__https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802__;!!GqivPVa7Brio!PSUeg8MO8B2TLNpewKuGd0oWY8N3pkO7w9hbCh3xgWK3TgFsPr78zcIUZ8Orgxgaqydf$
>> base: https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git__;!!GqivPVa7Brio!PSUeg8MO8B2TLNpewKuGd0oWY8N3pkO7w9hbCh3xgWK3TgFsPr78zcIUZ8Org7WbKd27$ linux-next
>> config: x86_64-randconfig-m001-20201008 (attached as .config)
>> compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
>>
>> If you fix the issue, kindly add following tag as appropriate
>> Reported-by: kernel test robot <lkp@intel.com>
>> Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
>>
>> New smatch warnings:
>> drivers/vhost/vdpa.c:844 vhost_vdpa_open() error: uninitialized symbol 'r'.
>>
>> Old smatch warnings:
>> drivers/vhost/vdpa.c:436 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
>> drivers/vhost/vdpa.c:489 vhost_vdpa_unlocked_ioctl() warn: maybe return -EFAULT instead of the bytes remaining?
>>
>> vim +/r +844 drivers/vhost/vdpa.c
>>
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 793 static int vhost_vdpa_open(struct inode *inode, struct file *filep)
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 794 {
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 795 struct vhost_vdpa *v;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 796 struct vhost_dev *dev;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 797 struct vhost_virtqueue **vqs;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 798 int nvqs, i, r, opened;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 799
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 800 v = container_of(inode->i_cdev, struct vhost_vdpa, cdev);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 801
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 802 opened = atomic_cmpxchg(&v->opened, 0, 1);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 803 if (opened)
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 804 return -EBUSY;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 805
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 806 nvqs = v->nvqs;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 807 vhost_vdpa_reset(v);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 808
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 809 vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 810 if (!vqs) {
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 811 r = -ENOMEM;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 812 goto err;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 813 }
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 814
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 815 dev = &v->vdev;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 816 for (i = 0; i < nvqs; i++) {
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 817 vqs[i] = &v->vqs[i];
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 818 vqs[i]->handle_kick = handle_vq_kick;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 819 }
>> 7dc4d1082d406f3 Mike Christie 2020-10-07 820 if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
>> 7dc4d1082d406f3 Mike Christie 2020-10-07 821 vhost_vdpa_process_iotlb_msg))
>> 7dc4d1082d406f3 Mike Christie 2020-10-07 822 goto err_dev_init;
>>
>> "r" not set on this error path.
>>
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 823
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 824 dev->iotlb = vhost_iotlb_alloc(0, 0);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 825 if (!dev->iotlb) {
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 826 r = -ENOMEM;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 827 goto err_init_iotlb;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 828 }
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 829
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 830 r = vhost_vdpa_alloc_domain(v);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 831 if (r)
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 832 goto err_init_iotlb;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 833
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 834 filep->private_data = v;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 835
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 836 return 0;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 837
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 838 err_init_iotlb:
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 839 vhost_dev_cleanup(&v->vdev);
>> 7dc4d1082d406f3 Mike Christie 2020-10-07 840 err_dev_init:
>> 37787e9f81e2e58 Mike Christie 2020-09-21 841 kfree(vqs);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 842 err:
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 843 atomic_dec(&v->opened);
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 @844 return r;
>> 4c8cf31885f69e8 Tiwei Bie 2020-03-26 845 }
>
>
> Yes looks like it would use r uninitialized ...
> Mike?
>
Ah sorry, I had posted a v3 of this patchset:
https://patchwork.kernel.org/project/target-devel/list/?series=368487
and I fixed that but there was another cases of uninitialized variable
that I missed. I fixed that up now, but have not posted an updated set.
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH 05/16] vhost: move vq iovec allocation to dev init time
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
The next patches allow us to create vqs on demand after vhost_dev_init
and vhost_dev_set_owner have been called. For vhost-scsi we don't
know the number of vqs we really want until the vring/vq setup
operations have started up. For other devices we know the number of vqs
at vhost_dev_init time, so for those devs we init the vq and allocate
the needed iovecs. For vhost-scsi we will do it later when userspace has
instructed us to setup a specific vq.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/vhost.c | 71 +++++++++++++++++++++++++++------------------------
1 file changed, 38 insertions(+), 33 deletions(-)
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index b05e690..d6af5a4 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -384,29 +384,27 @@ static void vhost_vq_free_iovecs(struct vhost_virtqueue *vq)
vq->heads = NULL;
}
-/* Helper to allocate iovec buffers for all vqs. */
-static long vhost_dev_alloc_iovecs(struct vhost_dev *dev)
+static int vhost_vq_alloc_iovecs(struct vhost_dev *dev,
+ struct vhost_virtqueue *vq)
{
- struct vhost_virtqueue *vq;
- int i;
+ vq->indirect = kmalloc_array(UIO_MAXIOV, sizeof(*vq->indirect),
+ GFP_KERNEL);
+ if (!vq->indirect)
+ return -ENOMEM;
+
+ if (!dev->iov_limit)
+ return 0;
+
+ vq->log = kmalloc_array(dev->iov_limit, sizeof(*vq->log), GFP_KERNEL);
+ vq->heads = kmalloc_array(dev->iov_limit, sizeof(*vq->heads),
+ GFP_KERNEL);
+ if (!vq->log || !vq->heads)
+ goto err_nomem;
- for (i = 0; i < dev->nvqs; ++i) {
- vq = dev->vqs[i];
- vq->indirect = kmalloc_array(UIO_MAXIOV,
- sizeof(*vq->indirect),
- GFP_KERNEL);
- vq->log = kmalloc_array(dev->iov_limit, sizeof(*vq->log),
- GFP_KERNEL);
- vq->heads = kmalloc_array(dev->iov_limit, sizeof(*vq->heads),
- GFP_KERNEL);
- if (!vq->indirect || !vq->log || !vq->heads)
- goto err_nomem;
- }
return 0;
err_nomem:
- for (; i >= 0; --i)
- vhost_vq_free_iovecs(dev->vqs[i]);
+ vhost_vq_free_iovecs(vq);
return -ENOMEM;
}
@@ -459,6 +457,21 @@ static size_t vhost_get_desc_size(struct vhost_virtqueue *vq,
return sizeof(*vq->desc) * num;
}
+static int vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq)
+{
+ vq->log = NULL;
+ vq->indirect = NULL;
+ vq->heads = NULL;
+ vq->dev = dev;
+ mutex_init(&vq->mutex);
+ vhost_vq_reset(dev, vq);
+
+ if (vq->handle_kick)
+ vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev);
+
+ return vhost_vq_alloc_iovecs(dev, vq);
+}
+
int vhost_dev_init(struct vhost_dev *dev,
struct vhost_virtqueue **vqs, int nvqs,
int iov_limit, int weight, int byte_weight,
@@ -466,7 +479,6 @@ int vhost_dev_init(struct vhost_dev *dev,
int (*msg_handler)(struct vhost_dev *dev,
struct vhost_iotlb_msg *msg))
{
- struct vhost_virtqueue *vq;
int i;
dev->vqs = vqs;
@@ -490,19 +502,16 @@ int vhost_dev_init(struct vhost_dev *dev,
for (i = 0; i < dev->nvqs; ++i) {
- vq = dev->vqs[i];
- vq->log = NULL;
- vq->indirect = NULL;
- vq->heads = NULL;
- vq->dev = dev;
- mutex_init(&vq->mutex);
- vhost_vq_reset(dev, vq);
- if (vq->handle_kick)
- vhost_poll_init(&vq->poll, vq->handle_kick,
- EPOLLIN, dev);
+ if (vhost_vq_init(dev, dev->vqs[i]))
+ goto err_vq_init;
}
return 0;
+
+err_vq_init:
+ for (--i; i >= 0; --i)
+ vhost_vq_free_iovecs(dev->vqs[i]);
+ return -ENOMEM;
}
EXPORT_SYMBOL_GPL(vhost_dev_init);
@@ -607,10 +616,6 @@ long vhost_dev_set_owner(struct vhost_dev *dev)
goto err_cgroup;
}
- err = vhost_dev_alloc_iovecs(dev);
- if (err)
- goto err_cgroup;
-
return 0;
err_cgroup:
if (dev->worker) {
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 05/16] vhost: move vq iovec allocation to dev init time
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
The next patches allow us to create vqs on demand after vhost_dev_init
and vhost_dev_set_owner have been called. For vhost-scsi we don't
know the number of vqs we really want until the vring/vq setup
operations have started up. For other devices we know the number of vqs
at vhost_dev_init time, so for those devs we init the vq and allocate
the needed iovecs. For vhost-scsi we will do it later when userspace has
instructed us to setup a specific vq.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/vhost.c | 71 +++++++++++++++++++++++++++------------------------
1 file changed, 38 insertions(+), 33 deletions(-)
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index b05e690..d6af5a4 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -384,29 +384,27 @@ static void vhost_vq_free_iovecs(struct vhost_virtqueue *vq)
vq->heads = NULL;
}
-/* Helper to allocate iovec buffers for all vqs. */
-static long vhost_dev_alloc_iovecs(struct vhost_dev *dev)
+static int vhost_vq_alloc_iovecs(struct vhost_dev *dev,
+ struct vhost_virtqueue *vq)
{
- struct vhost_virtqueue *vq;
- int i;
+ vq->indirect = kmalloc_array(UIO_MAXIOV, sizeof(*vq->indirect),
+ GFP_KERNEL);
+ if (!vq->indirect)
+ return -ENOMEM;
+
+ if (!dev->iov_limit)
+ return 0;
+
+ vq->log = kmalloc_array(dev->iov_limit, sizeof(*vq->log), GFP_KERNEL);
+ vq->heads = kmalloc_array(dev->iov_limit, sizeof(*vq->heads),
+ GFP_KERNEL);
+ if (!vq->log || !vq->heads)
+ goto err_nomem;
- for (i = 0; i < dev->nvqs; ++i) {
- vq = dev->vqs[i];
- vq->indirect = kmalloc_array(UIO_MAXIOV,
- sizeof(*vq->indirect),
- GFP_KERNEL);
- vq->log = kmalloc_array(dev->iov_limit, sizeof(*vq->log),
- GFP_KERNEL);
- vq->heads = kmalloc_array(dev->iov_limit, sizeof(*vq->heads),
- GFP_KERNEL);
- if (!vq->indirect || !vq->log || !vq->heads)
- goto err_nomem;
- }
return 0;
err_nomem:
- for (; i >= 0; --i)
- vhost_vq_free_iovecs(dev->vqs[i]);
+ vhost_vq_free_iovecs(vq);
return -ENOMEM;
}
@@ -459,6 +457,21 @@ static size_t vhost_get_desc_size(struct vhost_virtqueue *vq,
return sizeof(*vq->desc) * num;
}
+static int vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq)
+{
+ vq->log = NULL;
+ vq->indirect = NULL;
+ vq->heads = NULL;
+ vq->dev = dev;
+ mutex_init(&vq->mutex);
+ vhost_vq_reset(dev, vq);
+
+ if (vq->handle_kick)
+ vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev);
+
+ return vhost_vq_alloc_iovecs(dev, vq);
+}
+
int vhost_dev_init(struct vhost_dev *dev,
struct vhost_virtqueue **vqs, int nvqs,
int iov_limit, int weight, int byte_weight,
@@ -466,7 +479,6 @@ int vhost_dev_init(struct vhost_dev *dev,
int (*msg_handler)(struct vhost_dev *dev,
struct vhost_iotlb_msg *msg))
{
- struct vhost_virtqueue *vq;
int i;
dev->vqs = vqs;
@@ -490,19 +502,16 @@ int vhost_dev_init(struct vhost_dev *dev,
for (i = 0; i < dev->nvqs; ++i) {
- vq = dev->vqs[i];
- vq->log = NULL;
- vq->indirect = NULL;
- vq->heads = NULL;
- vq->dev = dev;
- mutex_init(&vq->mutex);
- vhost_vq_reset(dev, vq);
- if (vq->handle_kick)
- vhost_poll_init(&vq->poll, vq->handle_kick,
- EPOLLIN, dev);
+ if (vhost_vq_init(dev, dev->vqs[i]))
+ goto err_vq_init;
}
return 0;
+
+err_vq_init:
+ for (--i; i >= 0; --i)
+ vhost_vq_free_iovecs(dev->vqs[i]);
+ return -ENOMEM;
}
EXPORT_SYMBOL_GPL(vhost_dev_init);
@@ -607,10 +616,6 @@ long vhost_dev_set_owner(struct vhost_dev *dev)
goto err_cgroup;
}
- err = vhost_dev_alloc_iovecs(dev);
- if (err)
- goto err_cgroup;
-
return 0;
err_cgroup:
if (dev->worker) {
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 06/16] vhost: support delayed vq creation
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
This allows vq creation to be done when it's first accessed by
userspace. vhost-scsi doesn't know how many queues the user requested
until they are first setup, and we don't want to allocate resources
like the iovecs for 128 vqs when we are only using 1 or 2 most of the
time. In the next pathces, vhost-scsi will also switch to preallocating
cmds per vq instead of per lio session and we don't want to allocate
them for 127 extra vqs if they are not in use.
With this patch when a driver calls vhost_dev_init they pass in the
number of vqs that they know they need and the max they can support.
This patch has all the drivers pass in the same value for both the
initial number of vqs and the max. The next patch will convert scsi.
The other drivers like net/vsock have their vqs hard coded in the
kernel or setup/discovered via other methods like with vdpa.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/net.c | 2 +-
drivers/vhost/scsi.c | 4 +--
drivers/vhost/test.c | 5 ++--
drivers/vhost/vdpa.c | 2 +-
drivers/vhost/vhost.c | 71 ++++++++++++++++++++++++++++++++++-----------------
drivers/vhost/vhost.h | 7 +++--
drivers/vhost/vsock.c | 11 ++++----
7 files changed, 66 insertions(+), 36 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index fd30b53..fce46f0 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -1316,7 +1316,7 @@ static int vhost_net_open(struct inode *inode, struct file *f)
n->vqs[i].rx_ring = NULL;
vhost_net_buf_init(&n->vqs[i].rxq);
}
- if (vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX,
+ if (vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX, VHOST_NET_VQ_MAX,
UIO_MAXIOV + VHOST_NET_BATCH,
VHOST_NET_PKT_WEIGHT, VHOST_NET_WEIGHT, true,
NULL))
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 63ba363..5d412f1 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1632,8 +1632,8 @@ static int vhost_scsi_open(struct inode *inode, struct file *f)
vqs[i] = &vs->vqs[i].vq;
vs->vqs[i].vq.handle_kick = vhost_scsi_handle_kick;
}
- if (vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, UIO_MAXIOV,
- VHOST_SCSI_WEIGHT, 0, true, NULL))
+ if (vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, VHOST_SCSI_MAX_VQ,
+ UIO_MAXIOV, VHOST_SCSI_WEIGHT, 0, true, NULL))
goto err_dev_init;
vhost_scsi_init_inflight(vs, NULL);
diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c
index c255ae5..9d2bfa3 100644
--- a/drivers/vhost/test.c
+++ b/drivers/vhost/test.c
@@ -119,8 +119,9 @@ static int vhost_test_open(struct inode *inode, struct file *f)
dev = &n->dev;
vqs[VHOST_TEST_VQ] = &n->vqs[VHOST_TEST_VQ];
n->vqs[VHOST_TEST_VQ].handle_kick = handle_vq_kick;
- if (vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV,
- VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT, true, NULL)
+ if (vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, VHOST_TEST_VQ_MAX,
+ UIO_MAXIOV, VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT,
+ true, NULL)
goto err_dev_init;
f->private_data = n;
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index d413ceb..9507ebb 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -817,7 +817,7 @@ static int vhost_vdpa_open(struct inode *inode, struct file *filep)
vqs[i] = &v->vqs[i];
vqs[i]->handle_kick = handle_vq_kick;
}
- if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
+ if (vhost_dev_init(dev, vqs, nvqs, nvqs, 0, 0, 0, false,
vhost_vdpa_process_iotlb_msg))
goto err_dev_init;
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index d6af5a4..1507400 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -294,7 +294,7 @@ static void vhost_vq_meta_reset(struct vhost_dev *d)
{
int i;
- for (i = 0; i < d->nvqs; ++i)
+ for (i = 0; i < d->max_nvqs; ++i)
__vhost_vq_meta_reset(d->vqs[i]);
}
@@ -332,6 +332,7 @@ static void vhost_vq_reset(struct vhost_dev *dev,
vq->busyloop_timeout = 0;
vq->umem = NULL;
vq->iotlb = NULL;
+ vq->initialized = false;
vhost_vring_call_reset(&vq->call_ctx);
__vhost_vq_meta_reset(vq);
}
@@ -412,7 +413,7 @@ static void vhost_dev_free_iovecs(struct vhost_dev *dev)
{
int i;
- for (i = 0; i < dev->nvqs; ++i)
+ for (i = 0; i < dev->max_nvqs; ++i)
vhost_vq_free_iovecs(dev->vqs[i]);
}
@@ -457,7 +458,7 @@ static size_t vhost_get_desc_size(struct vhost_virtqueue *vq,
return sizeof(*vq->desc) * num;
}
-static int vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq)
+static void __vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq)
{
vq->log = NULL;
vq->indirect = NULL;
@@ -468,12 +469,29 @@ static int vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq)
if (vq->handle_kick)
vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev);
+}
+
+static int vhost_vq_init(struct vhost_dev *dev, int vq_idx)
+{
+ struct vhost_virtqueue *vq;
+ int ret;
+
+ if (vq_idx >= dev->max_nvqs)
+ return -ENOBUFS;
+
+ vq = dev->vqs[vq_idx];
+ __vhost_vq_init(dev, vq);
+ ret = vhost_vq_alloc_iovecs(dev, vq);
+ if (ret)
+ return ret;
- return vhost_vq_alloc_iovecs(dev, vq);
+ vq->initialized = true;
+ dev->nvqs++;
+ return 0;
}
int vhost_dev_init(struct vhost_dev *dev,
- struct vhost_virtqueue **vqs, int nvqs,
+ struct vhost_virtqueue **vqs, int nvqs, int max_nvqs,
int iov_limit, int weight, int byte_weight,
bool use_worker,
int (*msg_handler)(struct vhost_dev *dev,
@@ -482,7 +500,8 @@ int vhost_dev_init(struct vhost_dev *dev,
int i;
dev->vqs = vqs;
- dev->nvqs = nvqs;
+ dev->nvqs = 0;
+ dev->max_nvqs = max_nvqs;
mutex_init(&dev->mutex);
dev->log_ctx = NULL;
dev->umem = NULL;
@@ -500,12 +519,15 @@ int vhost_dev_init(struct vhost_dev *dev,
INIT_LIST_HEAD(&dev->pending_list);
spin_lock_init(&dev->iotlb_lock);
-
- for (i = 0; i < dev->nvqs; ++i) {
- if (vhost_vq_init(dev, dev->vqs[i]))
+ for (i = 0; i < nvqs; ++i) {
+ if (vhost_vq_init(dev, i))
goto err_vq_init;
}
+ for (; i < dev->max_nvqs; ++i)
+ /* Just prep/clear the fields and set initialized=false */
+ __vhost_vq_init(dev, dev->vqs[i]);
+
return 0;
err_vq_init:
@@ -653,7 +675,7 @@ void vhost_dev_reset_owner(struct vhost_dev *dev, struct vhost_iotlb *umem)
/* We don't need VQ locks below since vhost_dev_cleanup makes sure
* VQs aren't running.
*/
- for (i = 0; i < dev->nvqs; ++i)
+ for (i = 0; i < dev->max_nvqs; ++i)
dev->vqs[i]->umem = umem;
}
EXPORT_SYMBOL_GPL(vhost_dev_reset_owner);
@@ -662,7 +684,7 @@ void vhost_dev_stop(struct vhost_dev *dev)
{
int i;
- for (i = 0; i < dev->nvqs; ++i) {
+ for (i = 0; i < dev->max_nvqs; ++i) {
if (dev->vqs[i]->kick && dev->vqs[i]->handle_kick) {
vhost_poll_stop(&dev->vqs[i]->poll);
vhost_poll_flush(&dev->vqs[i]->poll);
@@ -694,7 +716,7 @@ void vhost_dev_cleanup(struct vhost_dev *dev)
{
int i;
- for (i = 0; i < dev->nvqs; ++i) {
+ for (i = 0; i < dev->max_nvqs; ++i) {
if (dev->vqs[i]->error_ctx)
eventfd_ctx_put(dev->vqs[i]->error_ctx);
if (dev->vqs[i]->kick)
@@ -788,7 +810,7 @@ static bool memory_access_ok(struct vhost_dev *d, struct vhost_iotlb *umem,
{
int i;
- for (i = 0; i < d->nvqs; ++i) {
+ for (i = 0; i < d->max_nvqs; ++i) {
bool ok;
bool log;
@@ -1000,14 +1022,14 @@ static inline int vhost_put_used_idx(struct vhost_virtqueue *vq)
static void vhost_dev_lock_vqs(struct vhost_dev *d)
{
int i = 0;
- for (i = 0; i < d->nvqs; ++i)
+ for (i = 0; i < d->max_nvqs; ++i)
mutex_lock_nested(&d->vqs[i]->mutex, i);
}
static void vhost_dev_unlock_vqs(struct vhost_dev *d)
{
int i = 0;
- for (i = 0; i < d->nvqs; ++i)
+ for (i = 0; i < d->max_nvqs; ++i)
mutex_unlock(&d->vqs[i]->mutex);
}
@@ -1463,7 +1485,7 @@ static long vhost_set_memory(struct vhost_dev *d, struct vhost_memory __user *m)
d->umem = newumem;
/* All memory accesses are done under some VQ mutex. */
- for (i = 0; i < d->nvqs; ++i) {
+ for (i = 0; i < d->max_nvqs; ++i) {
mutex_lock(&d->vqs[i]->mutex);
d->vqs[i]->umem = newumem;
mutex_unlock(&d->vqs[i]->mutex);
@@ -1591,11 +1613,14 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg
r = get_user(idx, idxp);
if (r < 0)
return r;
- if (idx >= d->nvqs)
- return -ENOBUFS;
- idx = array_index_nospec(idx, d->nvqs);
+ idx = array_index_nospec(idx, d->max_nvqs);
vq = d->vqs[idx];
+ if (!vq->initialized) {
+ r = vhost_vq_init(d, idx);
+ if (r)
+ return r;
+ }
if (ioctl == VHOST_SET_VRING_NUM ||
ioctl == VHOST_SET_VRING_ADDR) {
@@ -1727,7 +1752,7 @@ int vhost_init_device_iotlb(struct vhost_dev *d, bool enabled)
oiotlb = d->iotlb;
d->iotlb = niotlb;
- for (i = 0; i < d->nvqs; ++i) {
+ for (i = 0; i < d->max_nvqs; ++i) {
struct vhost_virtqueue *vq = d->vqs[i];
mutex_lock(&vq->mutex);
@@ -1774,7 +1799,7 @@ long vhost_dev_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *argp)
r = -EFAULT;
break;
}
- for (i = 0; i < d->nvqs; ++i) {
+ for (i = 0; i < d->max_nvqs; ++i) {
struct vhost_virtqueue *vq;
void __user *base = (void __user *)(unsigned long)p;
vq = d->vqs[i];
@@ -1797,7 +1822,7 @@ long vhost_dev_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *argp)
break;
}
swap(ctx, d->log_ctx);
- for (i = 0; i < d->nvqs; ++i) {
+ for (i = 0; i < d->max_nvqs; ++i) {
mutex_lock(&d->vqs[i]->mutex);
d->vqs[i]->log_ctx = d->log_ctx;
mutex_unlock(&d->vqs[i]->mutex);
@@ -2612,7 +2637,7 @@ void vhost_set_backend_features(struct vhost_dev *dev, u64 features)
int i;
mutex_lock(&dev->mutex);
- for (i = 0; i < dev->nvqs; ++i) {
+ for (i = 0; i < dev->max_nvqs; ++i) {
vq = dev->vqs[i];
mutex_lock(&vq->mutex);
vq->acked_backend_features = features;
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index a053318..2c6473e 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -132,6 +132,8 @@ struct vhost_virtqueue {
bool user_be;
#endif
u32 busyloop_timeout;
+
+ bool initialized;
};
struct vhost_msg_node {
@@ -148,6 +150,7 @@ struct vhost_dev {
struct mutex mutex;
struct vhost_virtqueue **vqs;
int nvqs;
+ int max_nvqs;
struct eventfd_ctx *log_ctx;
struct llist_head work_list;
struct task_struct *worker;
@@ -168,8 +171,8 @@ struct vhost_dev {
bool vhost_exceeds_weight(struct vhost_virtqueue *vq, int pkts, int total_len);
int vhost_dev_init(struct vhost_dev *dev, struct vhost_virtqueue **vqs,
- int nvqs, int iov_limit, int weight, int byte_weight,
- bool use_worker,
+ int nvqs, int max_nvqs, int iov_limit, int weight,
+ int byte_weight, bool use_worker,
int (*msg_handler)(struct vhost_dev *dev,
struct vhost_iotlb_msg *msg));
long vhost_dev_set_owner(struct vhost_dev *dev);
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index a1a35e1..9200868 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -606,7 +606,7 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file)
{
struct vhost_virtqueue **vqs;
struct vhost_vsock *vsock;
- int ret;
+ int ret, nvqs;
/* This struct is large and allocation could fail, fall back to vmalloc
* if there is no other way.
@@ -615,7 +615,8 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file)
if (!vsock)
return -ENOMEM;
- vqs = kmalloc_array(ARRAY_SIZE(vsock->vqs), sizeof(*vqs), GFP_KERNEL);
+ nvqs = ARRAY_SIZE(vsock->vqs);
+ vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
if (!vqs) {
ret = -ENOMEM;
goto out;
@@ -630,9 +631,9 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file)
vsock->vqs[VSOCK_VQ_TX].handle_kick = vhost_vsock_handle_tx_kick;
vsock->vqs[VSOCK_VQ_RX].handle_kick = vhost_vsock_handle_rx_kick;
- if (vhost_dev_init(&vsock->dev, vqs, ARRAY_SIZE(vsock->vqs),
- UIO_MAXIOV, VHOST_VSOCK_PKT_WEIGHT,
- VHOST_VSOCK_WEIGHT, true, NULL))
+ if (vhost_dev_init(&vsock->dev, vqs, nvqs, nvqs, UIO_MAXIOV,
+ VHOST_VSOCK_PKT_WEIGHT, VHOST_VSOCK_WEIGHT, true,
+ NULL))
goto err_dev_init;
file->private_data = vsock;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 06/16] vhost: support delayed vq creation
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
This allows vq creation to be done when it's first accessed by
userspace. vhost-scsi doesn't know how many queues the user requested
until they are first setup, and we don't want to allocate resources
like the iovecs for 128 vqs when we are only using 1 or 2 most of the
time. In the next pathces, vhost-scsi will also switch to preallocating
cmds per vq instead of per lio session and we don't want to allocate
them for 127 extra vqs if they are not in use.
With this patch when a driver calls vhost_dev_init they pass in the
number of vqs that they know they need and the max they can support.
This patch has all the drivers pass in the same value for both the
initial number of vqs and the max. The next patch will convert scsi.
The other drivers like net/vsock have their vqs hard coded in the
kernel or setup/discovered via other methods like with vdpa.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/net.c | 2 +-
drivers/vhost/scsi.c | 4 +--
drivers/vhost/test.c | 5 ++--
drivers/vhost/vdpa.c | 2 +-
drivers/vhost/vhost.c | 71 ++++++++++++++++++++++++++++++++++-----------------
drivers/vhost/vhost.h | 7 +++--
drivers/vhost/vsock.c | 11 ++++----
7 files changed, 66 insertions(+), 36 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index fd30b53..fce46f0 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -1316,7 +1316,7 @@ static int vhost_net_open(struct inode *inode, struct file *f)
n->vqs[i].rx_ring = NULL;
vhost_net_buf_init(&n->vqs[i].rxq);
}
- if (vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX,
+ if (vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX, VHOST_NET_VQ_MAX,
UIO_MAXIOV + VHOST_NET_BATCH,
VHOST_NET_PKT_WEIGHT, VHOST_NET_WEIGHT, true,
NULL))
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 63ba363..5d412f1 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1632,8 +1632,8 @@ static int vhost_scsi_open(struct inode *inode, struct file *f)
vqs[i] = &vs->vqs[i].vq;
vs->vqs[i].vq.handle_kick = vhost_scsi_handle_kick;
}
- if (vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, UIO_MAXIOV,
- VHOST_SCSI_WEIGHT, 0, true, NULL))
+ if (vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, VHOST_SCSI_MAX_VQ,
+ UIO_MAXIOV, VHOST_SCSI_WEIGHT, 0, true, NULL))
goto err_dev_init;
vhost_scsi_init_inflight(vs, NULL);
diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c
index c255ae5..9d2bfa3 100644
--- a/drivers/vhost/test.c
+++ b/drivers/vhost/test.c
@@ -119,8 +119,9 @@ static int vhost_test_open(struct inode *inode, struct file *f)
dev = &n->dev;
vqs[VHOST_TEST_VQ] = &n->vqs[VHOST_TEST_VQ];
n->vqs[VHOST_TEST_VQ].handle_kick = handle_vq_kick;
- if (vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV,
- VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT, true, NULL)
+ if (vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, VHOST_TEST_VQ_MAX,
+ UIO_MAXIOV, VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT,
+ true, NULL)
goto err_dev_init;
f->private_data = n;
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index d413ceb..9507ebb 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -817,7 +817,7 @@ static int vhost_vdpa_open(struct inode *inode, struct file *filep)
vqs[i] = &v->vqs[i];
vqs[i]->handle_kick = handle_vq_kick;
}
- if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false,
+ if (vhost_dev_init(dev, vqs, nvqs, nvqs, 0, 0, 0, false,
vhost_vdpa_process_iotlb_msg))
goto err_dev_init;
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index d6af5a4..1507400 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -294,7 +294,7 @@ static void vhost_vq_meta_reset(struct vhost_dev *d)
{
int i;
- for (i = 0; i < d->nvqs; ++i)
+ for (i = 0; i < d->max_nvqs; ++i)
__vhost_vq_meta_reset(d->vqs[i]);
}
@@ -332,6 +332,7 @@ static void vhost_vq_reset(struct vhost_dev *dev,
vq->busyloop_timeout = 0;
vq->umem = NULL;
vq->iotlb = NULL;
+ vq->initialized = false;
vhost_vring_call_reset(&vq->call_ctx);
__vhost_vq_meta_reset(vq);
}
@@ -412,7 +413,7 @@ static void vhost_dev_free_iovecs(struct vhost_dev *dev)
{
int i;
- for (i = 0; i < dev->nvqs; ++i)
+ for (i = 0; i < dev->max_nvqs; ++i)
vhost_vq_free_iovecs(dev->vqs[i]);
}
@@ -457,7 +458,7 @@ static size_t vhost_get_desc_size(struct vhost_virtqueue *vq,
return sizeof(*vq->desc) * num;
}
-static int vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq)
+static void __vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq)
{
vq->log = NULL;
vq->indirect = NULL;
@@ -468,12 +469,29 @@ static int vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq)
if (vq->handle_kick)
vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev);
+}
+
+static int vhost_vq_init(struct vhost_dev *dev, int vq_idx)
+{
+ struct vhost_virtqueue *vq;
+ int ret;
+
+ if (vq_idx >= dev->max_nvqs)
+ return -ENOBUFS;
+
+ vq = dev->vqs[vq_idx];
+ __vhost_vq_init(dev, vq);
+ ret = vhost_vq_alloc_iovecs(dev, vq);
+ if (ret)
+ return ret;
- return vhost_vq_alloc_iovecs(dev, vq);
+ vq->initialized = true;
+ dev->nvqs++;
+ return 0;
}
int vhost_dev_init(struct vhost_dev *dev,
- struct vhost_virtqueue **vqs, int nvqs,
+ struct vhost_virtqueue **vqs, int nvqs, int max_nvqs,
int iov_limit, int weight, int byte_weight,
bool use_worker,
int (*msg_handler)(struct vhost_dev *dev,
@@ -482,7 +500,8 @@ int vhost_dev_init(struct vhost_dev *dev,
int i;
dev->vqs = vqs;
- dev->nvqs = nvqs;
+ dev->nvqs = 0;
+ dev->max_nvqs = max_nvqs;
mutex_init(&dev->mutex);
dev->log_ctx = NULL;
dev->umem = NULL;
@@ -500,12 +519,15 @@ int vhost_dev_init(struct vhost_dev *dev,
INIT_LIST_HEAD(&dev->pending_list);
spin_lock_init(&dev->iotlb_lock);
-
- for (i = 0; i < dev->nvqs; ++i) {
- if (vhost_vq_init(dev, dev->vqs[i]))
+ for (i = 0; i < nvqs; ++i) {
+ if (vhost_vq_init(dev, i))
goto err_vq_init;
}
+ for (; i < dev->max_nvqs; ++i)
+ /* Just prep/clear the fields and set initializedúlse */
+ __vhost_vq_init(dev, dev->vqs[i]);
+
return 0;
err_vq_init:
@@ -653,7 +675,7 @@ void vhost_dev_reset_owner(struct vhost_dev *dev, struct vhost_iotlb *umem)
/* We don't need VQ locks below since vhost_dev_cleanup makes sure
* VQs aren't running.
*/
- for (i = 0; i < dev->nvqs; ++i)
+ for (i = 0; i < dev->max_nvqs; ++i)
dev->vqs[i]->umem = umem;
}
EXPORT_SYMBOL_GPL(vhost_dev_reset_owner);
@@ -662,7 +684,7 @@ void vhost_dev_stop(struct vhost_dev *dev)
{
int i;
- for (i = 0; i < dev->nvqs; ++i) {
+ for (i = 0; i < dev->max_nvqs; ++i) {
if (dev->vqs[i]->kick && dev->vqs[i]->handle_kick) {
vhost_poll_stop(&dev->vqs[i]->poll);
vhost_poll_flush(&dev->vqs[i]->poll);
@@ -694,7 +716,7 @@ void vhost_dev_cleanup(struct vhost_dev *dev)
{
int i;
- for (i = 0; i < dev->nvqs; ++i) {
+ for (i = 0; i < dev->max_nvqs; ++i) {
if (dev->vqs[i]->error_ctx)
eventfd_ctx_put(dev->vqs[i]->error_ctx);
if (dev->vqs[i]->kick)
@@ -788,7 +810,7 @@ static bool memory_access_ok(struct vhost_dev *d, struct vhost_iotlb *umem,
{
int i;
- for (i = 0; i < d->nvqs; ++i) {
+ for (i = 0; i < d->max_nvqs; ++i) {
bool ok;
bool log;
@@ -1000,14 +1022,14 @@ static inline int vhost_put_used_idx(struct vhost_virtqueue *vq)
static void vhost_dev_lock_vqs(struct vhost_dev *d)
{
int i = 0;
- for (i = 0; i < d->nvqs; ++i)
+ for (i = 0; i < d->max_nvqs; ++i)
mutex_lock_nested(&d->vqs[i]->mutex, i);
}
static void vhost_dev_unlock_vqs(struct vhost_dev *d)
{
int i = 0;
- for (i = 0; i < d->nvqs; ++i)
+ for (i = 0; i < d->max_nvqs; ++i)
mutex_unlock(&d->vqs[i]->mutex);
}
@@ -1463,7 +1485,7 @@ static long vhost_set_memory(struct vhost_dev *d, struct vhost_memory __user *m)
d->umem = newumem;
/* All memory accesses are done under some VQ mutex. */
- for (i = 0; i < d->nvqs; ++i) {
+ for (i = 0; i < d->max_nvqs; ++i) {
mutex_lock(&d->vqs[i]->mutex);
d->vqs[i]->umem = newumem;
mutex_unlock(&d->vqs[i]->mutex);
@@ -1591,11 +1613,14 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg
r = get_user(idx, idxp);
if (r < 0)
return r;
- if (idx >= d->nvqs)
- return -ENOBUFS;
- idx = array_index_nospec(idx, d->nvqs);
+ idx = array_index_nospec(idx, d->max_nvqs);
vq = d->vqs[idx];
+ if (!vq->initialized) {
+ r = vhost_vq_init(d, idx);
+ if (r)
+ return r;
+ }
if (ioctl = VHOST_SET_VRING_NUM ||
ioctl = VHOST_SET_VRING_ADDR) {
@@ -1727,7 +1752,7 @@ int vhost_init_device_iotlb(struct vhost_dev *d, bool enabled)
oiotlb = d->iotlb;
d->iotlb = niotlb;
- for (i = 0; i < d->nvqs; ++i) {
+ for (i = 0; i < d->max_nvqs; ++i) {
struct vhost_virtqueue *vq = d->vqs[i];
mutex_lock(&vq->mutex);
@@ -1774,7 +1799,7 @@ long vhost_dev_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *argp)
r = -EFAULT;
break;
}
- for (i = 0; i < d->nvqs; ++i) {
+ for (i = 0; i < d->max_nvqs; ++i) {
struct vhost_virtqueue *vq;
void __user *base = (void __user *)(unsigned long)p;
vq = d->vqs[i];
@@ -1797,7 +1822,7 @@ long vhost_dev_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *argp)
break;
}
swap(ctx, d->log_ctx);
- for (i = 0; i < d->nvqs; ++i) {
+ for (i = 0; i < d->max_nvqs; ++i) {
mutex_lock(&d->vqs[i]->mutex);
d->vqs[i]->log_ctx = d->log_ctx;
mutex_unlock(&d->vqs[i]->mutex);
@@ -2612,7 +2637,7 @@ void vhost_set_backend_features(struct vhost_dev *dev, u64 features)
int i;
mutex_lock(&dev->mutex);
- for (i = 0; i < dev->nvqs; ++i) {
+ for (i = 0; i < dev->max_nvqs; ++i) {
vq = dev->vqs[i];
mutex_lock(&vq->mutex);
vq->acked_backend_features = features;
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index a053318..2c6473e 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -132,6 +132,8 @@ struct vhost_virtqueue {
bool user_be;
#endif
u32 busyloop_timeout;
+
+ bool initialized;
};
struct vhost_msg_node {
@@ -148,6 +150,7 @@ struct vhost_dev {
struct mutex mutex;
struct vhost_virtqueue **vqs;
int nvqs;
+ int max_nvqs;
struct eventfd_ctx *log_ctx;
struct llist_head work_list;
struct task_struct *worker;
@@ -168,8 +171,8 @@ struct vhost_dev {
bool vhost_exceeds_weight(struct vhost_virtqueue *vq, int pkts, int total_len);
int vhost_dev_init(struct vhost_dev *dev, struct vhost_virtqueue **vqs,
- int nvqs, int iov_limit, int weight, int byte_weight,
- bool use_worker,
+ int nvqs, int max_nvqs, int iov_limit, int weight,
+ int byte_weight, bool use_worker,
int (*msg_handler)(struct vhost_dev *dev,
struct vhost_iotlb_msg *msg));
long vhost_dev_set_owner(struct vhost_dev *dev);
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index a1a35e1..9200868 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -606,7 +606,7 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file)
{
struct vhost_virtqueue **vqs;
struct vhost_vsock *vsock;
- int ret;
+ int ret, nvqs;
/* This struct is large and allocation could fail, fall back to vmalloc
* if there is no other way.
@@ -615,7 +615,8 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file)
if (!vsock)
return -ENOMEM;
- vqs = kmalloc_array(ARRAY_SIZE(vsock->vqs), sizeof(*vqs), GFP_KERNEL);
+ nvqs = ARRAY_SIZE(vsock->vqs);
+ vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL);
if (!vqs) {
ret = -ENOMEM;
goto out;
@@ -630,9 +631,9 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file)
vsock->vqs[VSOCK_VQ_TX].handle_kick = vhost_vsock_handle_tx_kick;
vsock->vqs[VSOCK_VQ_RX].handle_kick = vhost_vsock_handle_rx_kick;
- if (vhost_dev_init(&vsock->dev, vqs, ARRAY_SIZE(vsock->vqs),
- UIO_MAXIOV, VHOST_VSOCK_PKT_WEIGHT,
- VHOST_VSOCK_WEIGHT, true, NULL))
+ if (vhost_dev_init(&vsock->dev, vqs, nvqs, nvqs, UIO_MAXIOV,
+ VHOST_VSOCK_PKT_WEIGHT, VHOST_VSOCK_WEIGHT, true,
+ NULL))
goto err_dev_init;
file->private_data = vsock;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 07/16] vhost scsi: support delayed IO vq creation
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
Each vhost-scsi device will need a evt and ctl queue, but the number
of IO queues depends on whatever the user has configured in userspace.
This patch has vhost-scsi create the evt, ctl and one IO vq at device
open time. We then create the other IO vqs when userspace starts to
set them up. We still waste some mem on the vq and scsi vq structs,
but we don't waste mem on iovec related arrays and for later patches
we know which queues are used by the dev->nvqs value.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/scsi.c | 19 +++++++++++++++----
1 file changed, 15 insertions(+), 4 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 5d412f1..ab1b656 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1401,7 +1401,7 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
mutex_lock(&vs->dev.mutex);
/* Verify that ring has been setup correctly. */
- for (index = 0; index < vs->dev.nvqs; ++index) {
+ for (index = 0; index < vs->dev.max_nvqs; ++index) {
/* Verify that ring has been setup correctly. */
if (!vhost_vq_access_ok(&vs->vqs[index].vq)) {
ret = -EFAULT;
@@ -1464,6 +1464,9 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
sizeof(vs->vs_vhost_wwpn));
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) {
vq = &vs->vqs[i].vq;
+ if (!vq->initialized)
+ continue;
+
mutex_lock(&vq->mutex);
vhost_vq_set_backend(vq, vs_tpg);
vhost_vq_init_access(vq);
@@ -1503,7 +1506,7 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
mutex_lock(&vhost_scsi_mutex);
mutex_lock(&vs->dev.mutex);
/* Verify that ring has been setup correctly. */
- for (index = 0; index < vs->dev.nvqs; ++index) {
+ for (index = 0; index < vs->dev.max_nvqs; ++index) {
if (!vhost_vq_access_ok(&vs->vqs[index].vq)) {
ret = -EFAULT;
goto err_dev;
@@ -1551,6 +1554,9 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
if (match) {
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) {
vq = &vs->vqs[i].vq;
+ if (!vq->initialized)
+ continue;
+
mutex_lock(&vq->mutex);
vhost_vq_set_backend(vq, NULL);
mutex_unlock(&vq->mutex);
@@ -1632,8 +1638,13 @@ static int vhost_scsi_open(struct inode *inode, struct file *f)
vqs[i] = &vs->vqs[i].vq;
vs->vqs[i].vq.handle_kick = vhost_scsi_handle_kick;
}
- if (vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, VHOST_SCSI_MAX_VQ,
- UIO_MAXIOV, VHOST_SCSI_WEIGHT, 0, true, NULL))
+
+ /*
+ * We will always need the ctl, evt and at least 1 IO vq. Create more
+ * IO vqs if userspace requests them.
+ */
+ if (vhost_dev_init(&vs->dev, vqs, 3, VHOST_SCSI_MAX_VQ, UIO_MAXIOV,
+ VHOST_SCSI_WEIGHT, 0, true, NULL))
goto err_dev_init;
vhost_scsi_init_inflight(vs, NULL);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 07/16] vhost scsi: support delayed IO vq creation
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
Each vhost-scsi device will need a evt and ctl queue, but the number
of IO queues depends on whatever the user has configured in userspace.
This patch has vhost-scsi create the evt, ctl and one IO vq at device
open time. We then create the other IO vqs when userspace starts to
set them up. We still waste some mem on the vq and scsi vq structs,
but we don't waste mem on iovec related arrays and for later patches
we know which queues are used by the dev->nvqs value.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/scsi.c | 19 +++++++++++++++----
1 file changed, 15 insertions(+), 4 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 5d412f1..ab1b656 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1401,7 +1401,7 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
mutex_lock(&vs->dev.mutex);
/* Verify that ring has been setup correctly. */
- for (index = 0; index < vs->dev.nvqs; ++index) {
+ for (index = 0; index < vs->dev.max_nvqs; ++index) {
/* Verify that ring has been setup correctly. */
if (!vhost_vq_access_ok(&vs->vqs[index].vq)) {
ret = -EFAULT;
@@ -1464,6 +1464,9 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
sizeof(vs->vs_vhost_wwpn));
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) {
vq = &vs->vqs[i].vq;
+ if (!vq->initialized)
+ continue;
+
mutex_lock(&vq->mutex);
vhost_vq_set_backend(vq, vs_tpg);
vhost_vq_init_access(vq);
@@ -1503,7 +1506,7 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
mutex_lock(&vhost_scsi_mutex);
mutex_lock(&vs->dev.mutex);
/* Verify that ring has been setup correctly. */
- for (index = 0; index < vs->dev.nvqs; ++index) {
+ for (index = 0; index < vs->dev.max_nvqs; ++index) {
if (!vhost_vq_access_ok(&vs->vqs[index].vq)) {
ret = -EFAULT;
goto err_dev;
@@ -1551,6 +1554,9 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
if (match) {
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) {
vq = &vs->vqs[i].vq;
+ if (!vq->initialized)
+ continue;
+
mutex_lock(&vq->mutex);
vhost_vq_set_backend(vq, NULL);
mutex_unlock(&vq->mutex);
@@ -1632,8 +1638,13 @@ static int vhost_scsi_open(struct inode *inode, struct file *f)
vqs[i] = &vs->vqs[i].vq;
vs->vqs[i].vq.handle_kick = vhost_scsi_handle_kick;
}
- if (vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, VHOST_SCSI_MAX_VQ,
- UIO_MAXIOV, VHOST_SCSI_WEIGHT, 0, true, NULL))
+
+ /*
+ * We will always need the ctl, evt and at least 1 IO vq. Create more
+ * IO vqs if userspace requests them.
+ */
+ if (vhost_dev_init(&vs->dev, vqs, 3, VHOST_SCSI_MAX_VQ, UIO_MAXIOV,
+ VHOST_SCSI_WEIGHT, 0, true, NULL))
goto err_dev_init;
vhost_scsi_init_inflight(vs, NULL);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 08/16] vhost scsi: alloc cmds per vq instead of session
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
We currently are limited to 256 cmds per session. This leads to problems
where if the user has increased virtqueue_size to more than 2 or
cmd_per_lun to more than 256 vhost_scsi_get_tag can fail and the guest
will get IO errors.
This patch moves the cmd allocation to per vq so we can easily match
whatever the user has specified for num_queues and
virtqueue_size/cmd_per_lun. It also makes it easier to control how much
memory we preallocate. For cases, where perf is not as important and
we can use the current defaults (1 vq and 128 cmds per vq) memory use
from preallocate cmds is cut in half. For cases, where we are willing
to use more memory for higher perf, cmd mem use will now increase as
the num queues and queue depth increases.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
V2
- fix use before set cpu var errors
- drop vhost_vq_is_setup
drivers/vhost/scsi.c | 207 +++++++++++++++++++++++++++++++--------------------
1 file changed, 128 insertions(+), 79 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index ab1b656..f6b9010 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -52,7 +52,6 @@
#define VHOST_SCSI_VERSION "v0.1"
#define VHOST_SCSI_NAMELEN 256
#define VHOST_SCSI_MAX_CDB_SIZE 32
-#define VHOST_SCSI_DEFAULT_TAGS 256
#define VHOST_SCSI_PREALLOC_SGLS 2048
#define VHOST_SCSI_PREALLOC_UPAGES 2048
#define VHOST_SCSI_PREALLOC_PROT_SGLS 2048
@@ -189,6 +188,9 @@ struct vhost_scsi_virtqueue {
* Writers must also take dev mutex and flush under it.
*/
int inflight_idx;
+ struct vhost_scsi_cmd *scsi_cmds;
+ struct sbitmap scsi_tags;
+ int max_cmds;
};
struct vhost_scsi {
@@ -324,7 +326,9 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
{
struct vhost_scsi_cmd *tv_cmd = container_of(se_cmd,
struct vhost_scsi_cmd, tvc_se_cmd);
- struct se_session *se_sess = tv_cmd->tvc_nexus->tvn_se_sess;
+ struct vhost_scsi_virtqueue *svq = container_of(tv_cmd->tvc_vq,
+ struct vhost_scsi_virtqueue, vq);
+ struct vhost_scsi_inflight *inflight = tv_cmd->inflight;
int i;
if (tv_cmd->tvc_sgl_count) {
@@ -336,8 +340,8 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
put_page(sg_page(&tv_cmd->tvc_prot_sgl[i]));
}
- vhost_scsi_put_inflight(tv_cmd->inflight);
- target_free_tag(se_sess, se_cmd);
+ sbitmap_clear_bit(&svq->scsi_tags, se_cmd->map_tag);
+ vhost_scsi_put_inflight(inflight);
}
static u32 vhost_scsi_sess_get_index(struct se_session *se_sess)
@@ -566,31 +570,31 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
}
static struct vhost_scsi_cmd *
-vhost_scsi_get_tag(struct vhost_virtqueue *vq, struct vhost_scsi_tpg *tpg,
+vhost_scsi_get_cmd(struct vhost_virtqueue *vq, struct vhost_scsi_tpg *tpg,
unsigned char *cdb, u64 scsi_tag, u16 lun, u8 task_attr,
u32 exp_data_len, int data_direction)
{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
struct vhost_scsi_cmd *cmd;
struct vhost_scsi_nexus *tv_nexus;
- struct se_session *se_sess;
struct scatterlist *sg, *prot_sg;
struct page **pages;
- int tag, cpu;
+ int tag;
tv_nexus = tpg->tpg_nexus;
if (!tv_nexus) {
pr_err("Unable to locate active struct vhost_scsi_nexus\n");
return ERR_PTR(-EIO);
}
- se_sess = tv_nexus->tvn_se_sess;
- tag = sbitmap_queue_get(&se_sess->sess_tag_pool, &cpu);
+ tag = sbitmap_get(&svq->scsi_tags, 0, false);
if (tag < 0) {
pr_err("Unable to obtain tag for vhost_scsi_cmd\n");
return ERR_PTR(-ENOMEM);
}
- cmd = &((struct vhost_scsi_cmd *)se_sess->sess_cmd_map)[tag];
+ cmd = &svq->scsi_cmds[tag];
sg = cmd->tvc_sgl;
prot_sg = cmd->tvc_prot_sgl;
pages = cmd->tvc_upages;
@@ -599,7 +603,6 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
cmd->tvc_prot_sgl = prot_sg;
cmd->tvc_upages = pages;
cmd->tvc_se_cmd.map_tag = tag;
- cmd->tvc_se_cmd.map_cpu = cpu;
cmd->tvc_tag = scsi_tag;
cmd->tvc_lun = lun;
cmd->tvc_task_attr = task_attr;
@@ -1070,11 +1073,11 @@ static u16 vhost_buf_to_lun(u8 *lun_buf)
scsi_command_size(cdb), VHOST_SCSI_MAX_CDB_SIZE);
goto err;
}
- cmd = vhost_scsi_get_tag(vq, tpg, cdb, tag, lun, task_attr,
+ cmd = vhost_scsi_get_cmd(vq, tpg, cdb, tag, lun, task_attr,
exp_data_len + prot_bytes,
data_direction);
if (IS_ERR(cmd)) {
- vq_err(vq, "vhost_scsi_get_tag failed %ld\n",
+ vq_err(vq, "vhost_scsi_get_cmd failed %ld\n",
PTR_ERR(cmd));
goto err;
}
@@ -1378,6 +1381,83 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
wait_for_completion(&old_inflight[i]->comp);
}
+static void vhost_scsi_destroy_vq_cmds(struct vhost_virtqueue *vq)
+{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
+ struct vhost_scsi_cmd *tv_cmd;
+ unsigned int i;
+
+ if (!svq->scsi_cmds)
+ return;
+
+ for (i = 0; i < svq->max_cmds; i++) {
+ tv_cmd = &svq->scsi_cmds[i];
+
+ kfree(tv_cmd->tvc_sgl);
+ kfree(tv_cmd->tvc_prot_sgl);
+ kfree(tv_cmd->tvc_upages);
+ }
+
+ sbitmap_free(&svq->scsi_tags);
+ kfree(svq->scsi_cmds);
+ svq->scsi_cmds = NULL;
+}
+
+static int vhost_scsi_setup_vq_cmds(struct vhost_virtqueue *vq, int max_cmds)
+{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
+ struct vhost_scsi_cmd *tv_cmd;
+ unsigned int i;
+
+ if (svq->scsi_cmds)
+ return 0;
+
+ if (sbitmap_init_node(&svq->scsi_tags, max_cmds, -1, GFP_KERNEL,
+ NUMA_NO_NODE))
+ return -ENOMEM;
+ svq->max_cmds = max_cmds;
+
+ svq->scsi_cmds = kcalloc(max_cmds, sizeof(*tv_cmd), GFP_KERNEL);
+ if (!svq->scsi_cmds) {
+ sbitmap_free(&svq->scsi_tags);
+ return -ENOMEM;
+ }
+
+ for (i = 0; i < max_cmds; i++) {
+ tv_cmd = &svq->scsi_cmds[i];
+
+ tv_cmd->tvc_sgl = kcalloc(VHOST_SCSI_PREALLOC_SGLS,
+ sizeof(struct scatterlist),
+ GFP_KERNEL);
+ if (!tv_cmd->tvc_sgl) {
+ pr_err("Unable to allocate tv_cmd->tvc_sgl\n");
+ goto out;
+ }
+
+ tv_cmd->tvc_upages = kcalloc(VHOST_SCSI_PREALLOC_UPAGES,
+ sizeof(struct page *),
+ GFP_KERNEL);
+ if (!tv_cmd->tvc_upages) {
+ pr_err("Unable to allocate tv_cmd->tvc_upages\n");
+ goto out;
+ }
+
+ tv_cmd->tvc_prot_sgl = kcalloc(VHOST_SCSI_PREALLOC_PROT_SGLS,
+ sizeof(struct scatterlist),
+ GFP_KERNEL);
+ if (!tv_cmd->tvc_prot_sgl) {
+ pr_err("Unable to allocate tv_cmd->tvc_prot_sgl\n");
+ goto out;
+ }
+ }
+ return 0;
+out:
+ vhost_scsi_destroy_vq_cmds(vq);
+ return -ENOMEM;
+}
+
/*
* Called from vhost_scsi_ioctl() context to walk the list of available
* vhost_scsi_tpg with an active struct vhost_scsi_nexus
@@ -1432,10 +1512,9 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
if (!strcmp(tv_tport->tport_name, t->vhost_wwpn)) {
if (vs->vs_tpg && vs->vs_tpg[tpg->tport_tpgt]) {
- kfree(vs_tpg);
mutex_unlock(&tpg->tv_tpg_mutex);
ret = -EEXIST;
- goto out;
+ goto undepend;
}
/*
* In order to ensure individual vhost-scsi configfs
@@ -1447,9 +1526,8 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
ret = target_depend_item(&se_tpg->tpg_group.cg_item);
if (ret) {
pr_warn("target_depend_item() failed: %d\n", ret);
- kfree(vs_tpg);
mutex_unlock(&tpg->tv_tpg_mutex);
- goto out;
+ goto undepend;
}
tpg->tv_tpg_vhost_count++;
tpg->vhost_scsi = vs;
@@ -1462,6 +1540,16 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
if (match) {
memcpy(vs->vs_vhost_wwpn, t->vhost_wwpn,
sizeof(vs->vs_vhost_wwpn));
+
+ for (i = VHOST_SCSI_VQ_IO; i < VHOST_SCSI_MAX_VQ; i++) {
+ vq = &vs->vqs[i].vq;
+ if (!vq->initialized)
+ continue;
+
+ if (vhost_scsi_setup_vq_cmds(vq, vq->num))
+ goto destroy_vq_cmds;
+ }
+
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) {
vq = &vs->vqs[i].vq;
if (!vq->initialized)
@@ -1484,7 +1572,22 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
vhost_scsi_flush(vs);
kfree(vs->vs_tpg);
vs->vs_tpg = vs_tpg;
+ goto out;
+destroy_vq_cmds:
+ for (i--; i >= VHOST_SCSI_VQ_IO; i--) {
+ if (!vhost_vq_get_backend(&vs->vqs[i].vq))
+ vhost_scsi_destroy_vq_cmds(&vs->vqs[i].vq);
+ }
+undepend:
+ for (i = 0; i < VHOST_SCSI_MAX_TARGET; i++) {
+ tpg = vs_tpg[i];
+ if (tpg) {
+ tpg->tv_tpg_vhost_count--;
+ target_undepend_item(&tpg->se_tpg.tpg_group.cg_item);
+ }
+ }
+ kfree(vs_tpg);
out:
mutex_unlock(&vs->dev.mutex);
mutex_unlock(&vhost_scsi_mutex);
@@ -1560,6 +1663,12 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
mutex_lock(&vq->mutex);
vhost_vq_set_backend(vq, NULL);
mutex_unlock(&vq->mutex);
+ /*
+ * Make sure cmds are not running before tearing them
+ * down.
+ */
+ vhost_scsi_flush(vs);
+ vhost_scsi_destroy_vq_cmds(vq);
}
}
/*
@@ -1861,23 +1970,6 @@ static void vhost_scsi_port_unlink(struct se_portal_group *se_tpg,
mutex_unlock(&vhost_scsi_mutex);
}
-static void vhost_scsi_free_cmd_map_res(struct se_session *se_sess)
-{
- struct vhost_scsi_cmd *tv_cmd;
- unsigned int i;
-
- if (!se_sess->sess_cmd_map)
- return;
-
- for (i = 0; i < VHOST_SCSI_DEFAULT_TAGS; i++) {
- tv_cmd = &((struct vhost_scsi_cmd *)se_sess->sess_cmd_map)[i];
-
- kfree(tv_cmd->tvc_sgl);
- kfree(tv_cmd->tvc_prot_sgl);
- kfree(tv_cmd->tvc_upages);
- }
-}
-
static ssize_t vhost_scsi_tpg_attrib_fabric_prot_type_store(
struct config_item *item, const char *page, size_t count)
{
@@ -1917,45 +2009,6 @@ static ssize_t vhost_scsi_tpg_attrib_fabric_prot_type_show(
NULL,
};
-static int vhost_scsi_nexus_cb(struct se_portal_group *se_tpg,
- struct se_session *se_sess, void *p)
-{
- struct vhost_scsi_cmd *tv_cmd;
- unsigned int i;
-
- for (i = 0; i < VHOST_SCSI_DEFAULT_TAGS; i++) {
- tv_cmd = &((struct vhost_scsi_cmd *)se_sess->sess_cmd_map)[i];
-
- tv_cmd->tvc_sgl = kcalloc(VHOST_SCSI_PREALLOC_SGLS,
- sizeof(struct scatterlist),
- GFP_KERNEL);
- if (!tv_cmd->tvc_sgl) {
- pr_err("Unable to allocate tv_cmd->tvc_sgl\n");
- goto out;
- }
-
- tv_cmd->tvc_upages = kcalloc(VHOST_SCSI_PREALLOC_UPAGES,
- sizeof(struct page *),
- GFP_KERNEL);
- if (!tv_cmd->tvc_upages) {
- pr_err("Unable to allocate tv_cmd->tvc_upages\n");
- goto out;
- }
-
- tv_cmd->tvc_prot_sgl = kcalloc(VHOST_SCSI_PREALLOC_PROT_SGLS,
- sizeof(struct scatterlist),
- GFP_KERNEL);
- if (!tv_cmd->tvc_prot_sgl) {
- pr_err("Unable to allocate tv_cmd->tvc_prot_sgl\n");
- goto out;
- }
- }
- return 0;
-out:
- vhost_scsi_free_cmd_map_res(se_sess);
- return -ENOMEM;
-}
-
static int vhost_scsi_make_nexus(struct vhost_scsi_tpg *tpg,
const char *name)
{
@@ -1979,12 +2032,9 @@ static int vhost_scsi_make_nexus(struct vhost_scsi_tpg *tpg,
* struct se_node_acl for the vhost_scsi struct se_portal_group with
* the SCSI Initiator port name of the passed configfs group 'name'.
*/
- tv_nexus->tvn_se_sess = target_setup_session(&tpg->se_tpg,
- VHOST_SCSI_DEFAULT_TAGS,
- sizeof(struct vhost_scsi_cmd),
+ tv_nexus->tvn_se_sess = target_setup_session(&tpg->se_tpg, 0, 0,
TARGET_PROT_DIN_PASS | TARGET_PROT_DOUT_PASS,
- (unsigned char *)name, tv_nexus,
- vhost_scsi_nexus_cb);
+ (unsigned char *)name, tv_nexus, NULL);
if (IS_ERR(tv_nexus->tvn_se_sess)) {
mutex_unlock(&tpg->tv_tpg_mutex);
kfree(tv_nexus);
@@ -2034,7 +2084,6 @@ static int vhost_scsi_drop_nexus(struct vhost_scsi_tpg *tpg)
" %s Initiator Port: %s\n", vhost_scsi_dump_proto_id(tpg->tport),
tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
- vhost_scsi_free_cmd_map_res(se_sess);
/*
* Release the SCSI I_T Nexus to the emulated vhost Target Port
*/
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 08/16] vhost scsi: alloc cmds per vq instead of session
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
We currently are limited to 256 cmds per session. This leads to problems
where if the user has increased virtqueue_size to more than 2 or
cmd_per_lun to more than 256 vhost_scsi_get_tag can fail and the guest
will get IO errors.
This patch moves the cmd allocation to per vq so we can easily match
whatever the user has specified for num_queues and
virtqueue_size/cmd_per_lun. It also makes it easier to control how much
memory we preallocate. For cases, where perf is not as important and
we can use the current defaults (1 vq and 128 cmds per vq) memory use
from preallocate cmds is cut in half. For cases, where we are willing
to use more memory for higher perf, cmd mem use will now increase as
the num queues and queue depth increases.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
V2
- fix use before set cpu var errors
- drop vhost_vq_is_setup
drivers/vhost/scsi.c | 207 +++++++++++++++++++++++++++++++--------------------
1 file changed, 128 insertions(+), 79 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index ab1b656..f6b9010 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -52,7 +52,6 @@
#define VHOST_SCSI_VERSION "v0.1"
#define VHOST_SCSI_NAMELEN 256
#define VHOST_SCSI_MAX_CDB_SIZE 32
-#define VHOST_SCSI_DEFAULT_TAGS 256
#define VHOST_SCSI_PREALLOC_SGLS 2048
#define VHOST_SCSI_PREALLOC_UPAGES 2048
#define VHOST_SCSI_PREALLOC_PROT_SGLS 2048
@@ -189,6 +188,9 @@ struct vhost_scsi_virtqueue {
* Writers must also take dev mutex and flush under it.
*/
int inflight_idx;
+ struct vhost_scsi_cmd *scsi_cmds;
+ struct sbitmap scsi_tags;
+ int max_cmds;
};
struct vhost_scsi {
@@ -324,7 +326,9 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
{
struct vhost_scsi_cmd *tv_cmd = container_of(se_cmd,
struct vhost_scsi_cmd, tvc_se_cmd);
- struct se_session *se_sess = tv_cmd->tvc_nexus->tvn_se_sess;
+ struct vhost_scsi_virtqueue *svq = container_of(tv_cmd->tvc_vq,
+ struct vhost_scsi_virtqueue, vq);
+ struct vhost_scsi_inflight *inflight = tv_cmd->inflight;
int i;
if (tv_cmd->tvc_sgl_count) {
@@ -336,8 +340,8 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
put_page(sg_page(&tv_cmd->tvc_prot_sgl[i]));
}
- vhost_scsi_put_inflight(tv_cmd->inflight);
- target_free_tag(se_sess, se_cmd);
+ sbitmap_clear_bit(&svq->scsi_tags, se_cmd->map_tag);
+ vhost_scsi_put_inflight(inflight);
}
static u32 vhost_scsi_sess_get_index(struct se_session *se_sess)
@@ -566,31 +570,31 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
}
static struct vhost_scsi_cmd *
-vhost_scsi_get_tag(struct vhost_virtqueue *vq, struct vhost_scsi_tpg *tpg,
+vhost_scsi_get_cmd(struct vhost_virtqueue *vq, struct vhost_scsi_tpg *tpg,
unsigned char *cdb, u64 scsi_tag, u16 lun, u8 task_attr,
u32 exp_data_len, int data_direction)
{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
struct vhost_scsi_cmd *cmd;
struct vhost_scsi_nexus *tv_nexus;
- struct se_session *se_sess;
struct scatterlist *sg, *prot_sg;
struct page **pages;
- int tag, cpu;
+ int tag;
tv_nexus = tpg->tpg_nexus;
if (!tv_nexus) {
pr_err("Unable to locate active struct vhost_scsi_nexus\n");
return ERR_PTR(-EIO);
}
- se_sess = tv_nexus->tvn_se_sess;
- tag = sbitmap_queue_get(&se_sess->sess_tag_pool, &cpu);
+ tag = sbitmap_get(&svq->scsi_tags, 0, false);
if (tag < 0) {
pr_err("Unable to obtain tag for vhost_scsi_cmd\n");
return ERR_PTR(-ENOMEM);
}
- cmd = &((struct vhost_scsi_cmd *)se_sess->sess_cmd_map)[tag];
+ cmd = &svq->scsi_cmds[tag];
sg = cmd->tvc_sgl;
prot_sg = cmd->tvc_prot_sgl;
pages = cmd->tvc_upages;
@@ -599,7 +603,6 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
cmd->tvc_prot_sgl = prot_sg;
cmd->tvc_upages = pages;
cmd->tvc_se_cmd.map_tag = tag;
- cmd->tvc_se_cmd.map_cpu = cpu;
cmd->tvc_tag = scsi_tag;
cmd->tvc_lun = lun;
cmd->tvc_task_attr = task_attr;
@@ -1070,11 +1073,11 @@ static u16 vhost_buf_to_lun(u8 *lun_buf)
scsi_command_size(cdb), VHOST_SCSI_MAX_CDB_SIZE);
goto err;
}
- cmd = vhost_scsi_get_tag(vq, tpg, cdb, tag, lun, task_attr,
+ cmd = vhost_scsi_get_cmd(vq, tpg, cdb, tag, lun, task_attr,
exp_data_len + prot_bytes,
data_direction);
if (IS_ERR(cmd)) {
- vq_err(vq, "vhost_scsi_get_tag failed %ld\n",
+ vq_err(vq, "vhost_scsi_get_cmd failed %ld\n",
PTR_ERR(cmd));
goto err;
}
@@ -1378,6 +1381,83 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
wait_for_completion(&old_inflight[i]->comp);
}
+static void vhost_scsi_destroy_vq_cmds(struct vhost_virtqueue *vq)
+{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
+ struct vhost_scsi_cmd *tv_cmd;
+ unsigned int i;
+
+ if (!svq->scsi_cmds)
+ return;
+
+ for (i = 0; i < svq->max_cmds; i++) {
+ tv_cmd = &svq->scsi_cmds[i];
+
+ kfree(tv_cmd->tvc_sgl);
+ kfree(tv_cmd->tvc_prot_sgl);
+ kfree(tv_cmd->tvc_upages);
+ }
+
+ sbitmap_free(&svq->scsi_tags);
+ kfree(svq->scsi_cmds);
+ svq->scsi_cmds = NULL;
+}
+
+static int vhost_scsi_setup_vq_cmds(struct vhost_virtqueue *vq, int max_cmds)
+{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
+ struct vhost_scsi_cmd *tv_cmd;
+ unsigned int i;
+
+ if (svq->scsi_cmds)
+ return 0;
+
+ if (sbitmap_init_node(&svq->scsi_tags, max_cmds, -1, GFP_KERNEL,
+ NUMA_NO_NODE))
+ return -ENOMEM;
+ svq->max_cmds = max_cmds;
+
+ svq->scsi_cmds = kcalloc(max_cmds, sizeof(*tv_cmd), GFP_KERNEL);
+ if (!svq->scsi_cmds) {
+ sbitmap_free(&svq->scsi_tags);
+ return -ENOMEM;
+ }
+
+ for (i = 0; i < max_cmds; i++) {
+ tv_cmd = &svq->scsi_cmds[i];
+
+ tv_cmd->tvc_sgl = kcalloc(VHOST_SCSI_PREALLOC_SGLS,
+ sizeof(struct scatterlist),
+ GFP_KERNEL);
+ if (!tv_cmd->tvc_sgl) {
+ pr_err("Unable to allocate tv_cmd->tvc_sgl\n");
+ goto out;
+ }
+
+ tv_cmd->tvc_upages = kcalloc(VHOST_SCSI_PREALLOC_UPAGES,
+ sizeof(struct page *),
+ GFP_KERNEL);
+ if (!tv_cmd->tvc_upages) {
+ pr_err("Unable to allocate tv_cmd->tvc_upages\n");
+ goto out;
+ }
+
+ tv_cmd->tvc_prot_sgl = kcalloc(VHOST_SCSI_PREALLOC_PROT_SGLS,
+ sizeof(struct scatterlist),
+ GFP_KERNEL);
+ if (!tv_cmd->tvc_prot_sgl) {
+ pr_err("Unable to allocate tv_cmd->tvc_prot_sgl\n");
+ goto out;
+ }
+ }
+ return 0;
+out:
+ vhost_scsi_destroy_vq_cmds(vq);
+ return -ENOMEM;
+}
+
/*
* Called from vhost_scsi_ioctl() context to walk the list of available
* vhost_scsi_tpg with an active struct vhost_scsi_nexus
@@ -1432,10 +1512,9 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
if (!strcmp(tv_tport->tport_name, t->vhost_wwpn)) {
if (vs->vs_tpg && vs->vs_tpg[tpg->tport_tpgt]) {
- kfree(vs_tpg);
mutex_unlock(&tpg->tv_tpg_mutex);
ret = -EEXIST;
- goto out;
+ goto undepend;
}
/*
* In order to ensure individual vhost-scsi configfs
@@ -1447,9 +1526,8 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
ret = target_depend_item(&se_tpg->tpg_group.cg_item);
if (ret) {
pr_warn("target_depend_item() failed: %d\n", ret);
- kfree(vs_tpg);
mutex_unlock(&tpg->tv_tpg_mutex);
- goto out;
+ goto undepend;
}
tpg->tv_tpg_vhost_count++;
tpg->vhost_scsi = vs;
@@ -1462,6 +1540,16 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
if (match) {
memcpy(vs->vs_vhost_wwpn, t->vhost_wwpn,
sizeof(vs->vs_vhost_wwpn));
+
+ for (i = VHOST_SCSI_VQ_IO; i < VHOST_SCSI_MAX_VQ; i++) {
+ vq = &vs->vqs[i].vq;
+ if (!vq->initialized)
+ continue;
+
+ if (vhost_scsi_setup_vq_cmds(vq, vq->num))
+ goto destroy_vq_cmds;
+ }
+
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) {
vq = &vs->vqs[i].vq;
if (!vq->initialized)
@@ -1484,7 +1572,22 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
vhost_scsi_flush(vs);
kfree(vs->vs_tpg);
vs->vs_tpg = vs_tpg;
+ goto out;
+destroy_vq_cmds:
+ for (i--; i >= VHOST_SCSI_VQ_IO; i--) {
+ if (!vhost_vq_get_backend(&vs->vqs[i].vq))
+ vhost_scsi_destroy_vq_cmds(&vs->vqs[i].vq);
+ }
+undepend:
+ for (i = 0; i < VHOST_SCSI_MAX_TARGET; i++) {
+ tpg = vs_tpg[i];
+ if (tpg) {
+ tpg->tv_tpg_vhost_count--;
+ target_undepend_item(&tpg->se_tpg.tpg_group.cg_item);
+ }
+ }
+ kfree(vs_tpg);
out:
mutex_unlock(&vs->dev.mutex);
mutex_unlock(&vhost_scsi_mutex);
@@ -1560,6 +1663,12 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
mutex_lock(&vq->mutex);
vhost_vq_set_backend(vq, NULL);
mutex_unlock(&vq->mutex);
+ /*
+ * Make sure cmds are not running before tearing them
+ * down.
+ */
+ vhost_scsi_flush(vs);
+ vhost_scsi_destroy_vq_cmds(vq);
}
}
/*
@@ -1861,23 +1970,6 @@ static void vhost_scsi_port_unlink(struct se_portal_group *se_tpg,
mutex_unlock(&vhost_scsi_mutex);
}
-static void vhost_scsi_free_cmd_map_res(struct se_session *se_sess)
-{
- struct vhost_scsi_cmd *tv_cmd;
- unsigned int i;
-
- if (!se_sess->sess_cmd_map)
- return;
-
- for (i = 0; i < VHOST_SCSI_DEFAULT_TAGS; i++) {
- tv_cmd = &((struct vhost_scsi_cmd *)se_sess->sess_cmd_map)[i];
-
- kfree(tv_cmd->tvc_sgl);
- kfree(tv_cmd->tvc_prot_sgl);
- kfree(tv_cmd->tvc_upages);
- }
-}
-
static ssize_t vhost_scsi_tpg_attrib_fabric_prot_type_store(
struct config_item *item, const char *page, size_t count)
{
@@ -1917,45 +2009,6 @@ static ssize_t vhost_scsi_tpg_attrib_fabric_prot_type_show(
NULL,
};
-static int vhost_scsi_nexus_cb(struct se_portal_group *se_tpg,
- struct se_session *se_sess, void *p)
-{
- struct vhost_scsi_cmd *tv_cmd;
- unsigned int i;
-
- for (i = 0; i < VHOST_SCSI_DEFAULT_TAGS; i++) {
- tv_cmd = &((struct vhost_scsi_cmd *)se_sess->sess_cmd_map)[i];
-
- tv_cmd->tvc_sgl = kcalloc(VHOST_SCSI_PREALLOC_SGLS,
- sizeof(struct scatterlist),
- GFP_KERNEL);
- if (!tv_cmd->tvc_sgl) {
- pr_err("Unable to allocate tv_cmd->tvc_sgl\n");
- goto out;
- }
-
- tv_cmd->tvc_upages = kcalloc(VHOST_SCSI_PREALLOC_UPAGES,
- sizeof(struct page *),
- GFP_KERNEL);
- if (!tv_cmd->tvc_upages) {
- pr_err("Unable to allocate tv_cmd->tvc_upages\n");
- goto out;
- }
-
- tv_cmd->tvc_prot_sgl = kcalloc(VHOST_SCSI_PREALLOC_PROT_SGLS,
- sizeof(struct scatterlist),
- GFP_KERNEL);
- if (!tv_cmd->tvc_prot_sgl) {
- pr_err("Unable to allocate tv_cmd->tvc_prot_sgl\n");
- goto out;
- }
- }
- return 0;
-out:
- vhost_scsi_free_cmd_map_res(se_sess);
- return -ENOMEM;
-}
-
static int vhost_scsi_make_nexus(struct vhost_scsi_tpg *tpg,
const char *name)
{
@@ -1979,12 +2032,9 @@ static int vhost_scsi_make_nexus(struct vhost_scsi_tpg *tpg,
* struct se_node_acl for the vhost_scsi struct se_portal_group with
* the SCSI Initiator port name of the passed configfs group 'name'.
*/
- tv_nexus->tvn_se_sess = target_setup_session(&tpg->se_tpg,
- VHOST_SCSI_DEFAULT_TAGS,
- sizeof(struct vhost_scsi_cmd),
+ tv_nexus->tvn_se_sess = target_setup_session(&tpg->se_tpg, 0, 0,
TARGET_PROT_DIN_PASS | TARGET_PROT_DOUT_PASS,
- (unsigned char *)name, tv_nexus,
- vhost_scsi_nexus_cb);
+ (unsigned char *)name, tv_nexus, NULL);
if (IS_ERR(tv_nexus->tvn_se_sess)) {
mutex_unlock(&tpg->tv_tpg_mutex);
kfree(tv_nexus);
@@ -2034,7 +2084,6 @@ static int vhost_scsi_drop_nexus(struct vhost_scsi_tpg *tpg)
" %s Initiator Port: %s\n", vhost_scsi_dump_proto_id(tpg->tport),
tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
- vhost_scsi_free_cmd_map_res(se_sess);
/*
* Release the SCSI I_T Nexus to the emulated vhost Target Port
*/
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 09/16] vhost scsi: fix cmd completion race
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
We might not do the final se_cmd put from vhost_scsi_complete_cmd_work.
When the last put happens a little later then we could race where
vhost_scsi_complete_cmd_work does vhost_signal, the guest runs and sends
more IO, and vhost_scsi_handle_vq runs but does not find any free cmds.
This patch has us delay completing the cmd until the last lio core ref
is dropped. We then know that once we signal to the guest that the cmd
is completed that if it queues a new command it will find a free cmd.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/scsi.c | 42 +++++++++++++++---------------------------
1 file changed, 15 insertions(+), 27 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index f6b9010..2fa48dd 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -322,7 +322,7 @@ static u32 vhost_scsi_tpg_get_inst_index(struct se_portal_group *se_tpg)
return 1;
}
-static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
+static void vhost_scsi_release_cmd_res(struct se_cmd *se_cmd)
{
struct vhost_scsi_cmd *tv_cmd = container_of(se_cmd,
struct vhost_scsi_cmd, tvc_se_cmd);
@@ -344,6 +344,16 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
vhost_scsi_put_inflight(inflight);
}
+static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
+{
+ struct vhost_scsi_cmd *cmd = container_of(se_cmd,
+ struct vhost_scsi_cmd, tvc_se_cmd);
+ struct vhost_scsi *vs = cmd->tvc_vhost;
+
+ llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list);
+ vhost_work_queue(&vs->dev, &vs->vs_completion_work);
+}
+
static u32 vhost_scsi_sess_get_index(struct se_session *se_sess)
{
return 0;
@@ -366,28 +376,15 @@ static int vhost_scsi_get_cmd_state(struct se_cmd *se_cmd)
return 0;
}
-static void vhost_scsi_complete_cmd(struct vhost_scsi_cmd *cmd)
-{
- struct vhost_scsi *vs = cmd->tvc_vhost;
-
- llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list);
-
- vhost_work_queue(&vs->dev, &vs->vs_completion_work);
-}
-
static int vhost_scsi_queue_data_in(struct se_cmd *se_cmd)
{
- struct vhost_scsi_cmd *cmd = container_of(se_cmd,
- struct vhost_scsi_cmd, tvc_se_cmd);
- vhost_scsi_complete_cmd(cmd);
+ transport_generic_free_cmd(se_cmd, 0);
return 0;
}
static int vhost_scsi_queue_status(struct se_cmd *se_cmd)
{
- struct vhost_scsi_cmd *cmd = container_of(se_cmd,
- struct vhost_scsi_cmd, tvc_se_cmd);
- vhost_scsi_complete_cmd(cmd);
+ transport_generic_free_cmd(se_cmd, 0);
return 0;
}
@@ -433,15 +430,6 @@ static void vhost_scsi_free_evt(struct vhost_scsi *vs, struct vhost_scsi_evt *ev
return evt;
}
-static void vhost_scsi_free_cmd(struct vhost_scsi_cmd *cmd)
-{
- struct se_cmd *se_cmd = &cmd->tvc_se_cmd;
-
- /* TODO locking against target/backend threads? */
- transport_generic_free_cmd(se_cmd, 0);
-
-}
-
static int vhost_scsi_check_stop_free(struct se_cmd *se_cmd)
{
return target_put_sess_cmd(se_cmd);
@@ -560,7 +548,7 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
} else
pr_err("Faulted on virtio_scsi_cmd_resp\n");
- vhost_scsi_free_cmd(cmd);
+ vhost_scsi_release_cmd_res(se_cmd);
}
vq = -1;
@@ -1096,7 +1084,7 @@ static u16 vhost_buf_to_lun(u8 *lun_buf)
&prot_iter, exp_data_len,
&data_iter))) {
vq_err(vq, "Failed to map iov to sgl\n");
- vhost_scsi_release_cmd(&cmd->tvc_se_cmd);
+ vhost_scsi_release_cmd_res(&cmd->tvc_se_cmd);
goto err;
}
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 09/16] vhost scsi: fix cmd completion race
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
We might not do the final se_cmd put from vhost_scsi_complete_cmd_work.
When the last put happens a little later then we could race where
vhost_scsi_complete_cmd_work does vhost_signal, the guest runs and sends
more IO, and vhost_scsi_handle_vq runs but does not find any free cmds.
This patch has us delay completing the cmd until the last lio core ref
is dropped. We then know that once we signal to the guest that the cmd
is completed that if it queues a new command it will find a free cmd.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/scsi.c | 42 +++++++++++++++---------------------------
1 file changed, 15 insertions(+), 27 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index f6b9010..2fa48dd 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -322,7 +322,7 @@ static u32 vhost_scsi_tpg_get_inst_index(struct se_portal_group *se_tpg)
return 1;
}
-static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
+static void vhost_scsi_release_cmd_res(struct se_cmd *se_cmd)
{
struct vhost_scsi_cmd *tv_cmd = container_of(se_cmd,
struct vhost_scsi_cmd, tvc_se_cmd);
@@ -344,6 +344,16 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
vhost_scsi_put_inflight(inflight);
}
+static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
+{
+ struct vhost_scsi_cmd *cmd = container_of(se_cmd,
+ struct vhost_scsi_cmd, tvc_se_cmd);
+ struct vhost_scsi *vs = cmd->tvc_vhost;
+
+ llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list);
+ vhost_work_queue(&vs->dev, &vs->vs_completion_work);
+}
+
static u32 vhost_scsi_sess_get_index(struct se_session *se_sess)
{
return 0;
@@ -366,28 +376,15 @@ static int vhost_scsi_get_cmd_state(struct se_cmd *se_cmd)
return 0;
}
-static void vhost_scsi_complete_cmd(struct vhost_scsi_cmd *cmd)
-{
- struct vhost_scsi *vs = cmd->tvc_vhost;
-
- llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list);
-
- vhost_work_queue(&vs->dev, &vs->vs_completion_work);
-}
-
static int vhost_scsi_queue_data_in(struct se_cmd *se_cmd)
{
- struct vhost_scsi_cmd *cmd = container_of(se_cmd,
- struct vhost_scsi_cmd, tvc_se_cmd);
- vhost_scsi_complete_cmd(cmd);
+ transport_generic_free_cmd(se_cmd, 0);
return 0;
}
static int vhost_scsi_queue_status(struct se_cmd *se_cmd)
{
- struct vhost_scsi_cmd *cmd = container_of(se_cmd,
- struct vhost_scsi_cmd, tvc_se_cmd);
- vhost_scsi_complete_cmd(cmd);
+ transport_generic_free_cmd(se_cmd, 0);
return 0;
}
@@ -433,15 +430,6 @@ static void vhost_scsi_free_evt(struct vhost_scsi *vs, struct vhost_scsi_evt *ev
return evt;
}
-static void vhost_scsi_free_cmd(struct vhost_scsi_cmd *cmd)
-{
- struct se_cmd *se_cmd = &cmd->tvc_se_cmd;
-
- /* TODO locking against target/backend threads? */
- transport_generic_free_cmd(se_cmd, 0);
-
-}
-
static int vhost_scsi_check_stop_free(struct se_cmd *se_cmd)
{
return target_put_sess_cmd(se_cmd);
@@ -560,7 +548,7 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
} else
pr_err("Faulted on virtio_scsi_cmd_resp\n");
- vhost_scsi_free_cmd(cmd);
+ vhost_scsi_release_cmd_res(se_cmd);
}
vq = -1;
@@ -1096,7 +1084,7 @@ static u16 vhost_buf_to_lun(u8 *lun_buf)
&prot_iter, exp_data_len,
&data_iter))) {
vq_err(vq, "Failed to map iov to sgl\n");
- vhost_scsi_release_cmd(&cmd->tvc_se_cmd);
+ vhost_scsi_release_cmd_res(&cmd->tvc_se_cmd);
goto err;
}
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 10/16] vhost scsi: Add support for LUN resets.
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
In newer versions of virtio-scsi we just reset the timer when an a
command times out, so TMFs are never sent for the cmd time out case.
However, in older kernels and for the TMF inject cases, we can still get
resets and we end up just failing immediately so the guest might see the
device get offlined and IO errors.
For the older kernel cases, we want the same end result as the
modern virtio-scsi driver where we let the lower levels fire their error
handling and handle the problem. And at the upper levels we want to
wait. This patch ties the LUN reset handling into the LIO TMF code which
will just wait for outstanding commands to complete like we are doing in
the modern virtio-scsi case.
Note: I did not handle the ABORT case to keep this simple. For ABORTs
LIO just waits on the cmd like how it does for the RESET case. If
an ABORT fails, the guest OS ends up escalating to LUN RESET, so in
the end we get the same behavior where we wait on the outstanding
cmds.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/scsi.c | 147 ++++++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 134 insertions(+), 13 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 2fa48dd..f543fa0 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -139,6 +139,7 @@ struct vhost_scsi_tpg {
struct se_portal_group se_tpg;
/* Pointer back to vhost_scsi, protected by tv_tpg_mutex */
struct vhost_scsi *vhost_scsi;
+ struct list_head tmf_queue;
};
struct vhost_scsi_tport {
@@ -211,6 +212,20 @@ struct vhost_scsi {
int vs_events_nr; /* num of pending events, protected by vq->mutex */
};
+struct vhost_scsi_tmf {
+ struct vhost_work vwork;
+ struct vhost_scsi_tpg *tpg;
+ struct vhost_scsi *vhost;
+ struct vhost_scsi_virtqueue *svq;
+ struct list_head queue_entry;
+
+ struct se_cmd se_cmd;
+ struct vhost_scsi_inflight *inflight;
+ struct iovec resp_iov;
+ int in_iovs;
+ int vq_desc;
+};
+
/*
* Context for processing request and control queue operations.
*/
@@ -344,14 +359,32 @@ static void vhost_scsi_release_cmd_res(struct se_cmd *se_cmd)
vhost_scsi_put_inflight(inflight);
}
+static void vhost_scsi_release_tmf_res(struct vhost_scsi_tmf *tmf)
+{
+ struct vhost_scsi_tpg *tpg = tmf->tpg;
+ struct vhost_scsi_inflight *inflight = tmf->inflight;
+
+ mutex_lock(&tpg->tv_tpg_mutex);
+ list_add_tail(&tpg->tmf_queue, &tmf->queue_entry);
+ mutex_unlock(&tpg->tv_tpg_mutex);
+ vhost_scsi_put_inflight(inflight);
+}
+
static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
{
- struct vhost_scsi_cmd *cmd = container_of(se_cmd,
+ if (se_cmd->se_cmd_flags & SCF_SCSI_TMR_CDB) {
+ struct vhost_scsi_tmf *tmf = container_of(se_cmd,
+ struct vhost_scsi_tmf, se_cmd);
+
+ vhost_work_queue(&tmf->vhost->dev, &tmf->vwork);
+ } else {
+ struct vhost_scsi_cmd *cmd = container_of(se_cmd,
struct vhost_scsi_cmd, tvc_se_cmd);
- struct vhost_scsi *vs = cmd->tvc_vhost;
+ struct vhost_scsi *vs = cmd->tvc_vhost;
- llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list);
- vhost_work_queue(&vs->dev, &vs->vs_completion_work);
+ llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list);
+ vhost_work_queue(&vs->dev, &vs->vs_completion_work);
+ }
}
static u32 vhost_scsi_sess_get_index(struct se_session *se_sess)
@@ -390,7 +423,10 @@ static int vhost_scsi_queue_status(struct se_cmd *se_cmd)
static void vhost_scsi_queue_tm_rsp(struct se_cmd *se_cmd)
{
- return;
+ struct vhost_scsi_tmf *tmf = container_of(se_cmd, struct vhost_scsi_tmf,
+ se_cmd);
+
+ transport_generic_free_cmd(&tmf->se_cmd, 0);
}
static void vhost_scsi_aborted_task(struct se_cmd *se_cmd)
@@ -1120,9 +1156,9 @@ static u16 vhost_buf_to_lun(u8 *lun_buf)
}
static void
-vhost_scsi_send_tmf_reject(struct vhost_scsi *vs,
- struct vhost_virtqueue *vq,
- struct vhost_scsi_ctx *vc)
+vhost_scsi_send_tmf_resp(struct vhost_scsi *vs, struct vhost_virtqueue *vq,
+ int in_iovs, int vq_desc, struct iovec *resp_iov,
+ int tmf_resp_code)
{
struct virtio_scsi_ctrl_tmf_resp rsp;
struct iov_iter iov_iter;
@@ -1130,17 +1166,87 @@ static u16 vhost_buf_to_lun(u8 *lun_buf)
pr_debug("%s\n", __func__);
memset(&rsp, 0, sizeof(rsp));
- rsp.response = VIRTIO_SCSI_S_FUNCTION_REJECTED;
+ rsp.response = tmf_resp_code;
- iov_iter_init(&iov_iter, READ, &vq->iov[vc->out], vc->in, sizeof(rsp));
+ iov_iter_init(&iov_iter, READ, resp_iov, in_iovs, sizeof(rsp));
ret = copy_to_iter(&rsp, sizeof(rsp), &iov_iter);
if (likely(ret == sizeof(rsp)))
- vhost_add_used_and_signal(&vs->dev, vq, vc->head, 0);
+ vhost_add_used_and_signal(&vs->dev, vq, vq_desc, 0);
else
pr_err("Faulted on virtio_scsi_ctrl_tmf_resp\n");
}
+static void vhost_scsi_tmf_resp_work(struct vhost_work *work)
+{
+ struct vhost_scsi_tmf *tmf = container_of(work, struct vhost_scsi_tmf,
+ vwork);
+ int resp_code;
+
+ if (tmf->se_cmd.se_tmr_req->response == TMR_FUNCTION_COMPLETE)
+ resp_code = VIRTIO_SCSI_S_FUNCTION_SUCCEEDED;
+ else
+ resp_code = VIRTIO_SCSI_S_FUNCTION_REJECTED;
+
+ vhost_scsi_send_tmf_resp(tmf->vhost, &tmf->svq->vq, tmf->in_iovs,
+ tmf->vq_desc, &tmf->resp_iov, resp_code);
+ vhost_scsi_release_tmf_res(tmf);
+}
+
+static void
+vhost_scsi_handle_tmf(struct vhost_scsi *vs, struct vhost_scsi_tpg *tpg,
+ struct vhost_virtqueue *vq,
+ struct virtio_scsi_ctrl_tmf_req *vtmf,
+ struct vhost_scsi_ctx *vc)
+{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
+ struct vhost_scsi_tmf *tmf;
+
+ if (vhost32_to_cpu(vq, vtmf->subtype) !=
+ VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET)
+ goto send_reject;
+
+ if (!tpg->tpg_nexus || !tpg->tpg_nexus->tvn_se_sess) {
+ pr_err("Unable to locate active struct vhost_scsi_nexus for LUN RESET.\n");
+ goto send_reject;
+ }
+
+ mutex_lock(&tpg->tv_tpg_mutex);
+ if (list_empty(&tpg->tmf_queue)) {
+ pr_err("Missing reserve TMF. Could not handle LUN RESET.\n");
+ mutex_unlock(&tpg->tv_tpg_mutex);
+ goto send_reject;
+ }
+
+ tmf = list_first_entry(&tpg->tmf_queue, struct vhost_scsi_tmf,
+ queue_entry);
+ list_del_init(&tmf->queue_entry);
+ mutex_unlock(&tpg->tv_tpg_mutex);
+
+ tmf->tpg = tpg;
+ tmf->vhost = vs;
+ tmf->svq = svq;
+ tmf->resp_iov = vq->iov[vc->out];
+ tmf->vq_desc = vc->head;
+ tmf->in_iovs = vc->in;
+ tmf->inflight = vhost_scsi_get_inflight(vq);
+
+ if (target_submit_tmr(&tmf->se_cmd, tpg->tpg_nexus->tvn_se_sess, NULL,
+ vhost_buf_to_lun(vtmf->lun), NULL,
+ TMR_LUN_RESET, GFP_KERNEL, 0,
+ TARGET_SCF_ACK_KREF) < 0) {
+ vhost_scsi_release_tmf_res(tmf);
+ goto send_reject;
+ }
+
+ return;
+
+send_reject:
+ vhost_scsi_send_tmf_resp(vs, vq, vc->in, vc->head, &vq->iov[vc->out],
+ VIRTIO_SCSI_S_FUNCTION_REJECTED);
+}
+
static void
vhost_scsi_send_an_resp(struct vhost_scsi *vs,
struct vhost_virtqueue *vq,
@@ -1166,6 +1272,7 @@ static u16 vhost_buf_to_lun(u8 *lun_buf)
static void
vhost_scsi_ctl_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq)
{
+ struct vhost_scsi_tpg *tpg;
union {
__virtio32 type;
struct virtio_scsi_ctrl_an_req an;
@@ -1247,12 +1354,12 @@ static u16 vhost_buf_to_lun(u8 *lun_buf)
vc.req += typ_size;
vc.req_size -= typ_size;
- ret = vhost_scsi_get_req(vq, &vc, NULL);
+ ret = vhost_scsi_get_req(vq, &vc, &tpg);
if (ret)
goto err;
if (v_req.type == VIRTIO_SCSI_T_TMF)
- vhost_scsi_send_tmf_reject(vs, vq, &vc);
+ vhost_scsi_handle_tmf(vs, tpg, vq, &v_req.tmf, &vc);
else
vhost_scsi_send_an_resp(vs, vq, &vc);
err:
@@ -1927,11 +2034,19 @@ static int vhost_scsi_port_link(struct se_portal_group *se_tpg,
{
struct vhost_scsi_tpg *tpg = container_of(se_tpg,
struct vhost_scsi_tpg, se_tpg);
+ struct vhost_scsi_tmf *tmf;
+
+ tmf = kzalloc(sizeof(*tmf), GFP_KERNEL);
+ if (!tmf)
+ return -ENOMEM;
+ INIT_LIST_HEAD(&tmf->queue_entry);
+ vhost_work_init(&tmf->vwork, vhost_scsi_tmf_resp_work);
mutex_lock(&vhost_scsi_mutex);
mutex_lock(&tpg->tv_tpg_mutex);
tpg->tv_tpg_port_count++;
+ list_add_tail(&tmf->queue_entry, &tpg->tmf_queue);
mutex_unlock(&tpg->tv_tpg_mutex);
vhost_scsi_hotplug(tpg, lun);
@@ -1946,11 +2061,16 @@ static void vhost_scsi_port_unlink(struct se_portal_group *se_tpg,
{
struct vhost_scsi_tpg *tpg = container_of(se_tpg,
struct vhost_scsi_tpg, se_tpg);
+ struct vhost_scsi_tmf *tmf;
mutex_lock(&vhost_scsi_mutex);
mutex_lock(&tpg->tv_tpg_mutex);
tpg->tv_tpg_port_count--;
+ tmf = list_first_entry(&tpg->tmf_queue, struct vhost_scsi_tmf,
+ queue_entry);
+ list_del(&tmf->queue_entry);
+ kfree(tmf);
mutex_unlock(&tpg->tv_tpg_mutex);
vhost_scsi_hotunplug(tpg, lun);
@@ -2211,6 +2331,7 @@ static ssize_t vhost_scsi_tpg_nexus_store(struct config_item *item,
}
mutex_init(&tpg->tv_tpg_mutex);
INIT_LIST_HEAD(&tpg->tv_tpg_list);
+ INIT_LIST_HEAD(&tpg->tmf_queue);
tpg->tport = tport;
tpg->tport_tpgt = tpgt;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 10/16] vhost scsi: Add support for LUN resets.
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
In newer versions of virtio-scsi we just reset the timer when an a
command times out, so TMFs are never sent for the cmd time out case.
However, in older kernels and for the TMF inject cases, we can still get
resets and we end up just failing immediately so the guest might see the
device get offlined and IO errors.
For the older kernel cases, we want the same end result as the
modern virtio-scsi driver where we let the lower levels fire their error
handling and handle the problem. And at the upper levels we want to
wait. This patch ties the LUN reset handling into the LIO TMF code which
will just wait for outstanding commands to complete like we are doing in
the modern virtio-scsi case.
Note: I did not handle the ABORT case to keep this simple. For ABORTs
LIO just waits on the cmd like how it does for the RESET case. If
an ABORT fails, the guest OS ends up escalating to LUN RESET, so in
the end we get the same behavior where we wait on the outstanding
cmds.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/scsi.c | 147 ++++++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 134 insertions(+), 13 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 2fa48dd..f543fa0 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -139,6 +139,7 @@ struct vhost_scsi_tpg {
struct se_portal_group se_tpg;
/* Pointer back to vhost_scsi, protected by tv_tpg_mutex */
struct vhost_scsi *vhost_scsi;
+ struct list_head tmf_queue;
};
struct vhost_scsi_tport {
@@ -211,6 +212,20 @@ struct vhost_scsi {
int vs_events_nr; /* num of pending events, protected by vq->mutex */
};
+struct vhost_scsi_tmf {
+ struct vhost_work vwork;
+ struct vhost_scsi_tpg *tpg;
+ struct vhost_scsi *vhost;
+ struct vhost_scsi_virtqueue *svq;
+ struct list_head queue_entry;
+
+ struct se_cmd se_cmd;
+ struct vhost_scsi_inflight *inflight;
+ struct iovec resp_iov;
+ int in_iovs;
+ int vq_desc;
+};
+
/*
* Context for processing request and control queue operations.
*/
@@ -344,14 +359,32 @@ static void vhost_scsi_release_cmd_res(struct se_cmd *se_cmd)
vhost_scsi_put_inflight(inflight);
}
+static void vhost_scsi_release_tmf_res(struct vhost_scsi_tmf *tmf)
+{
+ struct vhost_scsi_tpg *tpg = tmf->tpg;
+ struct vhost_scsi_inflight *inflight = tmf->inflight;
+
+ mutex_lock(&tpg->tv_tpg_mutex);
+ list_add_tail(&tpg->tmf_queue, &tmf->queue_entry);
+ mutex_unlock(&tpg->tv_tpg_mutex);
+ vhost_scsi_put_inflight(inflight);
+}
+
static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
{
- struct vhost_scsi_cmd *cmd = container_of(se_cmd,
+ if (se_cmd->se_cmd_flags & SCF_SCSI_TMR_CDB) {
+ struct vhost_scsi_tmf *tmf = container_of(se_cmd,
+ struct vhost_scsi_tmf, se_cmd);
+
+ vhost_work_queue(&tmf->vhost->dev, &tmf->vwork);
+ } else {
+ struct vhost_scsi_cmd *cmd = container_of(se_cmd,
struct vhost_scsi_cmd, tvc_se_cmd);
- struct vhost_scsi *vs = cmd->tvc_vhost;
+ struct vhost_scsi *vs = cmd->tvc_vhost;
- llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list);
- vhost_work_queue(&vs->dev, &vs->vs_completion_work);
+ llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list);
+ vhost_work_queue(&vs->dev, &vs->vs_completion_work);
+ }
}
static u32 vhost_scsi_sess_get_index(struct se_session *se_sess)
@@ -390,7 +423,10 @@ static int vhost_scsi_queue_status(struct se_cmd *se_cmd)
static void vhost_scsi_queue_tm_rsp(struct se_cmd *se_cmd)
{
- return;
+ struct vhost_scsi_tmf *tmf = container_of(se_cmd, struct vhost_scsi_tmf,
+ se_cmd);
+
+ transport_generic_free_cmd(&tmf->se_cmd, 0);
}
static void vhost_scsi_aborted_task(struct se_cmd *se_cmd)
@@ -1120,9 +1156,9 @@ static u16 vhost_buf_to_lun(u8 *lun_buf)
}
static void
-vhost_scsi_send_tmf_reject(struct vhost_scsi *vs,
- struct vhost_virtqueue *vq,
- struct vhost_scsi_ctx *vc)
+vhost_scsi_send_tmf_resp(struct vhost_scsi *vs, struct vhost_virtqueue *vq,
+ int in_iovs, int vq_desc, struct iovec *resp_iov,
+ int tmf_resp_code)
{
struct virtio_scsi_ctrl_tmf_resp rsp;
struct iov_iter iov_iter;
@@ -1130,17 +1166,87 @@ static u16 vhost_buf_to_lun(u8 *lun_buf)
pr_debug("%s\n", __func__);
memset(&rsp, 0, sizeof(rsp));
- rsp.response = VIRTIO_SCSI_S_FUNCTION_REJECTED;
+ rsp.response = tmf_resp_code;
- iov_iter_init(&iov_iter, READ, &vq->iov[vc->out], vc->in, sizeof(rsp));
+ iov_iter_init(&iov_iter, READ, resp_iov, in_iovs, sizeof(rsp));
ret = copy_to_iter(&rsp, sizeof(rsp), &iov_iter);
if (likely(ret = sizeof(rsp)))
- vhost_add_used_and_signal(&vs->dev, vq, vc->head, 0);
+ vhost_add_used_and_signal(&vs->dev, vq, vq_desc, 0);
else
pr_err("Faulted on virtio_scsi_ctrl_tmf_resp\n");
}
+static void vhost_scsi_tmf_resp_work(struct vhost_work *work)
+{
+ struct vhost_scsi_tmf *tmf = container_of(work, struct vhost_scsi_tmf,
+ vwork);
+ int resp_code;
+
+ if (tmf->se_cmd.se_tmr_req->response = TMR_FUNCTION_COMPLETE)
+ resp_code = VIRTIO_SCSI_S_FUNCTION_SUCCEEDED;
+ else
+ resp_code = VIRTIO_SCSI_S_FUNCTION_REJECTED;
+
+ vhost_scsi_send_tmf_resp(tmf->vhost, &tmf->svq->vq, tmf->in_iovs,
+ tmf->vq_desc, &tmf->resp_iov, resp_code);
+ vhost_scsi_release_tmf_res(tmf);
+}
+
+static void
+vhost_scsi_handle_tmf(struct vhost_scsi *vs, struct vhost_scsi_tpg *tpg,
+ struct vhost_virtqueue *vq,
+ struct virtio_scsi_ctrl_tmf_req *vtmf,
+ struct vhost_scsi_ctx *vc)
+{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
+ struct vhost_scsi_tmf *tmf;
+
+ if (vhost32_to_cpu(vq, vtmf->subtype) !+ VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET)
+ goto send_reject;
+
+ if (!tpg->tpg_nexus || !tpg->tpg_nexus->tvn_se_sess) {
+ pr_err("Unable to locate active struct vhost_scsi_nexus for LUN RESET.\n");
+ goto send_reject;
+ }
+
+ mutex_lock(&tpg->tv_tpg_mutex);
+ if (list_empty(&tpg->tmf_queue)) {
+ pr_err("Missing reserve TMF. Could not handle LUN RESET.\n");
+ mutex_unlock(&tpg->tv_tpg_mutex);
+ goto send_reject;
+ }
+
+ tmf = list_first_entry(&tpg->tmf_queue, struct vhost_scsi_tmf,
+ queue_entry);
+ list_del_init(&tmf->queue_entry);
+ mutex_unlock(&tpg->tv_tpg_mutex);
+
+ tmf->tpg = tpg;
+ tmf->vhost = vs;
+ tmf->svq = svq;
+ tmf->resp_iov = vq->iov[vc->out];
+ tmf->vq_desc = vc->head;
+ tmf->in_iovs = vc->in;
+ tmf->inflight = vhost_scsi_get_inflight(vq);
+
+ if (target_submit_tmr(&tmf->se_cmd, tpg->tpg_nexus->tvn_se_sess, NULL,
+ vhost_buf_to_lun(vtmf->lun), NULL,
+ TMR_LUN_RESET, GFP_KERNEL, 0,
+ TARGET_SCF_ACK_KREF) < 0) {
+ vhost_scsi_release_tmf_res(tmf);
+ goto send_reject;
+ }
+
+ return;
+
+send_reject:
+ vhost_scsi_send_tmf_resp(vs, vq, vc->in, vc->head, &vq->iov[vc->out],
+ VIRTIO_SCSI_S_FUNCTION_REJECTED);
+}
+
static void
vhost_scsi_send_an_resp(struct vhost_scsi *vs,
struct vhost_virtqueue *vq,
@@ -1166,6 +1272,7 @@ static u16 vhost_buf_to_lun(u8 *lun_buf)
static void
vhost_scsi_ctl_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq)
{
+ struct vhost_scsi_tpg *tpg;
union {
__virtio32 type;
struct virtio_scsi_ctrl_an_req an;
@@ -1247,12 +1354,12 @@ static u16 vhost_buf_to_lun(u8 *lun_buf)
vc.req += typ_size;
vc.req_size -= typ_size;
- ret = vhost_scsi_get_req(vq, &vc, NULL);
+ ret = vhost_scsi_get_req(vq, &vc, &tpg);
if (ret)
goto err;
if (v_req.type = VIRTIO_SCSI_T_TMF)
- vhost_scsi_send_tmf_reject(vs, vq, &vc);
+ vhost_scsi_handle_tmf(vs, tpg, vq, &v_req.tmf, &vc);
else
vhost_scsi_send_an_resp(vs, vq, &vc);
err:
@@ -1927,11 +2034,19 @@ static int vhost_scsi_port_link(struct se_portal_group *se_tpg,
{
struct vhost_scsi_tpg *tpg = container_of(se_tpg,
struct vhost_scsi_tpg, se_tpg);
+ struct vhost_scsi_tmf *tmf;
+
+ tmf = kzalloc(sizeof(*tmf), GFP_KERNEL);
+ if (!tmf)
+ return -ENOMEM;
+ INIT_LIST_HEAD(&tmf->queue_entry);
+ vhost_work_init(&tmf->vwork, vhost_scsi_tmf_resp_work);
mutex_lock(&vhost_scsi_mutex);
mutex_lock(&tpg->tv_tpg_mutex);
tpg->tv_tpg_port_count++;
+ list_add_tail(&tmf->queue_entry, &tpg->tmf_queue);
mutex_unlock(&tpg->tv_tpg_mutex);
vhost_scsi_hotplug(tpg, lun);
@@ -1946,11 +2061,16 @@ static void vhost_scsi_port_unlink(struct se_portal_group *se_tpg,
{
struct vhost_scsi_tpg *tpg = container_of(se_tpg,
struct vhost_scsi_tpg, se_tpg);
+ struct vhost_scsi_tmf *tmf;
mutex_lock(&vhost_scsi_mutex);
mutex_lock(&tpg->tv_tpg_mutex);
tpg->tv_tpg_port_count--;
+ tmf = list_first_entry(&tpg->tmf_queue, struct vhost_scsi_tmf,
+ queue_entry);
+ list_del(&tmf->queue_entry);
+ kfree(tmf);
mutex_unlock(&tpg->tv_tpg_mutex);
vhost_scsi_hotunplug(tpg, lun);
@@ -2211,6 +2331,7 @@ static ssize_t vhost_scsi_tpg_nexus_store(struct config_item *item,
}
mutex_init(&tpg->tv_tpg_mutex);
INIT_LIST_HEAD(&tpg->tv_tpg_list);
+ INIT_LIST_HEAD(&tpg->tmf_queue);
tpg->tport = tport;
tpg->tport_tpgt = tpgt;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 11/16] vhost scsi: remove extra flushes
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
The vhost work flush function was flushing the entire work queue, so
there is no need for the double vhost_work_dev_flush calls in
vhost_scsi_flush.
And we do not need to call vhost_poll_flush for each poller because
that call also ends up flushing the same work queue thread the
vhost_work_dev_flush call flushed.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/scsi.c | 8 --------
1 file changed, 8 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index f543fa0..b348e9c 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1443,11 +1443,6 @@ static void vhost_scsi_handle_kick(struct vhost_work *work)
vhost_scsi_handle_vq(vs, vq);
}
-static void vhost_scsi_flush_vq(struct vhost_scsi *vs, int index)
-{
- vhost_poll_flush(&vs->vqs[index].vq.poll);
-}
-
/* Callers must hold dev mutex */
static void vhost_scsi_flush(struct vhost_scsi *vs)
{
@@ -1466,9 +1461,6 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
kref_put(&old_inflight[i]->kref, vhost_scsi_done_inflight);
/* Flush both the vhost poll and vhost work */
- for (i = 0; i < VHOST_SCSI_MAX_VQ; i++)
- vhost_scsi_flush_vq(vs, i);
- vhost_work_dev_flush(&vs->dev);
vhost_work_dev_flush(&vs->dev);
/* Wait for all reqs issued before the flush to be finished */
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 11/16] vhost scsi: remove extra flushes
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
The vhost work flush function was flushing the entire work queue, so
there is no need for the double vhost_work_dev_flush calls in
vhost_scsi_flush.
And we do not need to call vhost_poll_flush for each poller because
that call also ends up flushing the same work queue thread the
vhost_work_dev_flush call flushed.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/scsi.c | 8 --------
1 file changed, 8 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index f543fa0..b348e9c 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1443,11 +1443,6 @@ static void vhost_scsi_handle_kick(struct vhost_work *work)
vhost_scsi_handle_vq(vs, vq);
}
-static void vhost_scsi_flush_vq(struct vhost_scsi *vs, int index)
-{
- vhost_poll_flush(&vs->vqs[index].vq.poll);
-}
-
/* Callers must hold dev mutex */
static void vhost_scsi_flush(struct vhost_scsi *vs)
{
@@ -1466,9 +1461,6 @@ static void vhost_scsi_flush(struct vhost_scsi *vs)
kref_put(&old_inflight[i]->kref, vhost_scsi_done_inflight);
/* Flush both the vhost poll and vhost work */
- for (i = 0; i < VHOST_SCSI_MAX_VQ; i++)
- vhost_scsi_flush_vq(vs, i);
- vhost_work_dev_flush(&vs->dev);
vhost_work_dev_flush(&vs->dev);
/* Wait for all reqs issued before the flush to be finished */
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 12/16] vhost: support multiple worker threads
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
This is a prep patch to support multiple vhost worker threads per vhost
dev. This patch converts the code that had assumed a single worker
thread by:
1. Moving worker related fields to a new struct vhost_worker.
2. Converting vhost.c code to use the new struct and assume we will
have an array of workers.
3. It also exports a helper function that will be used in the last
patch when vhost-scsi is converted to use this new functionality.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/vhost.c | 232 +++++++++++++++++++++++++++++++++++++++-----------
drivers/vhost/vhost.h | 12 ++-
2 files changed, 190 insertions(+), 54 deletions(-)
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index 1507400..e46b39e 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -231,18 +231,48 @@ void vhost_poll_stop(struct vhost_poll *poll)
}
EXPORT_SYMBOL_GPL(vhost_poll_stop);
-void vhost_work_dev_flush(struct vhost_dev *dev)
+static void vhost_work_queue_on(struct vhost_dev *dev, struct vhost_work *work,
+ int worker_id)
+{
+ if (!dev->workers)
+ return;
+
+ if (!test_and_set_bit(VHOST_WORK_QUEUED, &work->flags)) {
+ /* We can only add the work to the list after we're
+ * sure it was not in the list.
+ * test_and_set_bit() implies a memory barrier.
+ */
+ llist_add(&work->node, &dev->workers[worker_id]->work_list);
+ wake_up_process(dev->workers[worker_id]->task);
+ }
+}
+
+void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work)
+{
+ vhost_work_queue_on(dev, work, 0);
+}
+EXPORT_SYMBOL_GPL(vhost_work_queue);
+
+static void vhost_work_flush_on(struct vhost_dev *dev, int worker_id)
{
struct vhost_flush_struct flush;
- if (dev->worker) {
+ if (dev->workers) {
init_completion(&flush.wait_event);
vhost_work_init(&flush.work, vhost_flush_work);
- vhost_work_queue(dev, &flush.work);
+ vhost_work_queue_on(dev, &flush.work, worker_id);
wait_for_completion(&flush.wait_event);
}
}
+
+void vhost_work_dev_flush(struct vhost_dev *dev)
+{
+ int i;
+
+ for (i = 0; i < dev->num_workers; i++)
+ vhost_work_flush_on(dev, i);
+}
EXPORT_SYMBOL_GPL(vhost_work_dev_flush);
/* Flush any work that has been scheduled. When calling this, don't hold any
@@ -253,26 +283,20 @@ void vhost_poll_flush(struct vhost_poll *poll)
}
EXPORT_SYMBOL_GPL(vhost_poll_flush);
-void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work)
+/* A lockless hint for busy polling code to exit the loop */
+bool vhost_has_work(struct vhost_dev *dev)
{
- if (!dev->worker)
- return;
+ int i;
- if (!test_and_set_bit(VHOST_WORK_QUEUED, &work->flags)) {
- /* We can only add the work to the list after we're
- * sure it was not in the list.
- * test_and_set_bit() implies a memory barrier.
- */
- llist_add(&work->node, &dev->work_list);
- wake_up_process(dev->worker);
+ if (!dev->workers)
+ return false;
+
+ for (i = 0; i < dev->num_workers; i++) {
+ if (!llist_empty(&dev->workers[i]->work_list))
+ return true;
}
-}
-EXPORT_SYMBOL_GPL(vhost_work_queue);
-/* A lockless hint for busy polling code to exit the loop */
-bool vhost_has_work(struct vhost_dev *dev)
-{
- return !llist_empty(&dev->work_list);
+ return false;
}
EXPORT_SYMBOL_GPL(vhost_has_work);
@@ -339,7 +363,8 @@ static void vhost_vq_reset(struct vhost_dev *dev,
static int vhost_worker(void *data)
{
- struct vhost_dev *dev = data;
+ struct vhost_worker *worker = data;
+ struct vhost_dev *dev = worker->dev;
struct vhost_work *work, *work_next;
struct llist_node *node;
@@ -353,8 +378,7 @@ static int vhost_worker(void *data)
__set_current_state(TASK_RUNNING);
break;
}
-
- node = llist_del_all(&dev->work_list);
+ node = llist_del_all(&worker->work_list);
if (!node)
schedule();
@@ -507,13 +531,13 @@ int vhost_dev_init(struct vhost_dev *dev,
dev->umem = NULL;
dev->iotlb = NULL;
dev->mm = NULL;
- dev->worker = NULL;
+ dev->workers = NULL;
+ dev->num_workers = 0;
dev->iov_limit = iov_limit;
dev->weight = weight;
- dev->byte_weight = byte_weight;
dev->use_worker = use_worker;
+ dev->byte_weight = byte_weight;
dev->msg_handler = msg_handler;
- init_llist_head(&dev->work_list);
init_waitqueue_head(&dev->wait);
INIT_LIST_HEAD(&dev->read_list);
INIT_LIST_HEAD(&dev->pending_list);
@@ -559,16 +583,28 @@ static void vhost_attach_cgroups_work(struct vhost_work *work)
s->ret = cgroup_attach_task_all(s->owner, current);
}
-static int vhost_attach_cgroups(struct vhost_dev *dev)
+static int vhost_attach_cgroups_on(struct vhost_dev *dev, int worker_id)
{
struct vhost_attach_cgroups_struct attach;
attach.owner = current;
vhost_work_init(&attach.work, vhost_attach_cgroups_work);
- vhost_work_queue(dev, &attach.work);
- vhost_work_dev_flush(dev);
+ vhost_work_queue_on(dev, &attach.work, worker_id);
+ vhost_work_flush_on(dev, worker_id);
return attach.ret;
}
+static int vhost_attach_cgroups(struct vhost_dev *dev, int first_worker)
+{
+ int i, ret = 0;
+
+ for (i = first_worker; i < dev->num_workers; i++) {
+ ret = vhost_attach_cgroups_on(dev, i);
+ if (ret)
+ break;
+ }
+
+ return ret;
+}
/* Caller should have device mutex */
bool vhost_dev_has_owner(struct vhost_dev *dev)
@@ -607,10 +643,117 @@ static void vhost_detach_mm(struct vhost_dev *dev)
dev->mm = NULL;
}
+static void vhost_workers_free(struct vhost_dev *dev)
+{
+ struct vhost_worker *worker;
+ int i;
+
+ if (!dev->workers)
+ return;
+
+ for (i = 0; i < dev->num_workers; i++) {
+ worker = dev->workers[i];
+
+ WARN_ON(!llist_empty(&worker->work_list));
+ kthread_stop(worker->task);
+ kfree(worker);
+ }
+
+ kfree(dev->workers);
+ dev->workers = NULL;
+ dev->num_workers = 0;
+}
+
+static int vhost_worker_create(struct vhost_dev *dev, int worker_id)
+{
+ struct vhost_worker *worker;
+ struct task_struct *task;
+ int ret;
+
+ worker = kzalloc(sizeof(*worker), GFP_KERNEL);
+ if (!worker)
+ return -ENOMEM;
+
+ init_llist_head(&worker->work_list);
+ worker->dev = dev;
+
+ task = kthread_create(vhost_worker, worker, "vhost-%d", current->pid);
+ if (IS_ERR(task)) {
+ ret = PTR_ERR(task);
+ goto free_worker;
+ }
+
+ dev->workers[worker_id] = worker;
+ worker->task = task;
+ wake_up_process(task); /* avoid contributing to loadavg */
+ return 0;
+
+free_worker:
+ kfree(worker);
+ return ret;
+}
+
+/**
+ * vhost_workers_create - create vhost workers and attach to cgroup
+ * @dev: vhost device
+ * @new_num_workers: the total number of workers we want after this returns
+ *
+ * Caller must have the device mutex and have stopped operations that
+ * can access the workers array.
+ */
+int vhost_workers_create(struct vhost_dev *dev, int new_num_workers)
+{
+ struct vhost_worker **new_workers;
+ struct mm_struct *mm;
+ bool owner_match = true;
+ int i, err, start;
+
+ if (new_num_workers == dev->num_workers)
+ return 0;
+
+ if (new_num_workers < dev->num_workers)
+ return -EINVAL;
+
+ if (vhost_dev_has_owner(dev)) {
+ mm = get_task_mm(current);
+ if (mm != dev->mm)
+ owner_match = false;
+ mmput(mm);
+ if (!owner_match)
+ return -EBUSY;
+ }
+
+ new_workers = krealloc(dev->workers, new_num_workers * sizeof(*new_workers),
+ GFP_KERNEL);
+ if (!new_workers) {
+ err = -ENOMEM;
+ goto free_workers;
+ }
+ dev->workers = new_workers;
+
+ start = dev->num_workers;
+ for (i = start; i < new_num_workers; i++) {
+ err = vhost_worker_create(dev, i);
+ if (err)
+ goto free_workers;
+ dev->num_workers++;
+ }
+
+ err = vhost_attach_cgroups(dev, start);
+ if (err)
+ goto free_workers;
+
+ return 0;
+
+free_workers:
+ vhost_workers_free(dev);
+ return err;
+}
+EXPORT_SYMBOL_GPL(vhost_workers_create);
+
/* Caller should have device mutex */
long vhost_dev_set_owner(struct vhost_dev *dev)
{
- struct task_struct *worker;
int err;
/* Is there an owner already? */
@@ -623,27 +766,16 @@ long vhost_dev_set_owner(struct vhost_dev *dev)
dev->kcov_handle = kcov_common_handle();
if (dev->use_worker) {
- worker = kthread_create(vhost_worker, dev,
- "vhost-%d", current->pid);
- if (IS_ERR(worker)) {
- err = PTR_ERR(worker);
- goto err_worker;
- }
-
- dev->worker = worker;
- wake_up_process(worker); /* avoid contributing to loadavg */
-
- err = vhost_attach_cgroups(dev);
+ /*
+ * All drivers that set use_worker=true, use at least one
+ * worker. Drivers like vhost-scsi may override this later.
+ */
+ err = vhost_workers_create(dev, 1);
if (err)
- goto err_cgroup;
+ goto err_worker;
}
return 0;
-err_cgroup:
- if (dev->worker) {
- kthread_stop(dev->worker);
- dev->worker = NULL;
- }
err_worker:
vhost_detach_mm(dev);
dev->kcov_handle = 0;
@@ -736,12 +868,8 @@ void vhost_dev_cleanup(struct vhost_dev *dev)
dev->iotlb = NULL;
vhost_clear_msg(dev);
wake_up_interruptible_poll(&dev->wait, EPOLLIN | EPOLLRDNORM);
- WARN_ON(!llist_empty(&dev->work_list));
- if (dev->worker) {
- kthread_stop(dev->worker);
- dev->worker = NULL;
- dev->kcov_handle = 0;
- }
+ vhost_workers_free(dev);
+ dev->kcov_handle = 0;
vhost_detach_mm(dev);
}
EXPORT_SYMBOL_GPL(vhost_dev_cleanup);
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 2c6473e..a91be8a 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -25,6 +25,12 @@ struct vhost_work {
unsigned long flags;
};
+struct vhost_worker {
+ struct task_struct *task;
+ struct llist_head work_list;
+ struct vhost_dev *dev;
+};
+
/* Poll a file (eventfd or socket) */
/* Note: there's nothing vhost specific about this structure. */
struct vhost_poll {
@@ -39,6 +45,7 @@ struct vhost_poll {
void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn);
void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work);
bool vhost_has_work(struct vhost_dev *dev);
+int vhost_workers_create(struct vhost_dev *dev, int new_num_workers);
void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn,
__poll_t mask, struct vhost_dev *dev);
@@ -152,8 +159,8 @@ struct vhost_dev {
int nvqs;
int max_nvqs;
struct eventfd_ctx *log_ctx;
- struct llist_head work_list;
- struct task_struct *worker;
+ struct vhost_worker **workers;
+ int num_workers;
struct vhost_iotlb *umem;
struct vhost_iotlb *iotlb;
spinlock_t iotlb_lock;
@@ -175,6 +182,7 @@ int vhost_dev_init(struct vhost_dev *dev, struct vhost_virtqueue **vqs,
int byte_weight, bool use_worker,
int (*msg_handler)(struct vhost_dev *dev,
struct vhost_iotlb_msg *msg));
+int vhost_vq_set_worker(struct vhost_virtqueue *vq, int worker_id);
long vhost_dev_set_owner(struct vhost_dev *dev);
bool vhost_dev_has_owner(struct vhost_dev *dev);
long vhost_dev_check_owner(struct vhost_dev *);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 12/16] vhost: support multiple worker threads
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
This is a prep patch to support multiple vhost worker threads per vhost
dev. This patch converts the code that had assumed a single worker
thread by:
1. Moving worker related fields to a new struct vhost_worker.
2. Converting vhost.c code to use the new struct and assume we will
have an array of workers.
3. It also exports a helper function that will be used in the last
patch when vhost-scsi is converted to use this new functionality.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/vhost.c | 232 +++++++++++++++++++++++++++++++++++++++-----------
drivers/vhost/vhost.h | 12 ++-
2 files changed, 190 insertions(+), 54 deletions(-)
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index 1507400..e46b39e 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -231,18 +231,48 @@ void vhost_poll_stop(struct vhost_poll *poll)
}
EXPORT_SYMBOL_GPL(vhost_poll_stop);
-void vhost_work_dev_flush(struct vhost_dev *dev)
+static void vhost_work_queue_on(struct vhost_dev *dev, struct vhost_work *work,
+ int worker_id)
+{
+ if (!dev->workers)
+ return;
+
+ if (!test_and_set_bit(VHOST_WORK_QUEUED, &work->flags)) {
+ /* We can only add the work to the list after we're
+ * sure it was not in the list.
+ * test_and_set_bit() implies a memory barrier.
+ */
+ llist_add(&work->node, &dev->workers[worker_id]->work_list);
+ wake_up_process(dev->workers[worker_id]->task);
+ }
+}
+
+void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work)
+{
+ vhost_work_queue_on(dev, work, 0);
+}
+EXPORT_SYMBOL_GPL(vhost_work_queue);
+
+static void vhost_work_flush_on(struct vhost_dev *dev, int worker_id)
{
struct vhost_flush_struct flush;
- if (dev->worker) {
+ if (dev->workers) {
init_completion(&flush.wait_event);
vhost_work_init(&flush.work, vhost_flush_work);
- vhost_work_queue(dev, &flush.work);
+ vhost_work_queue_on(dev, &flush.work, worker_id);
wait_for_completion(&flush.wait_event);
}
}
+
+void vhost_work_dev_flush(struct vhost_dev *dev)
+{
+ int i;
+
+ for (i = 0; i < dev->num_workers; i++)
+ vhost_work_flush_on(dev, i);
+}
EXPORT_SYMBOL_GPL(vhost_work_dev_flush);
/* Flush any work that has been scheduled. When calling this, don't hold any
@@ -253,26 +283,20 @@ void vhost_poll_flush(struct vhost_poll *poll)
}
EXPORT_SYMBOL_GPL(vhost_poll_flush);
-void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work)
+/* A lockless hint for busy polling code to exit the loop */
+bool vhost_has_work(struct vhost_dev *dev)
{
- if (!dev->worker)
- return;
+ int i;
- if (!test_and_set_bit(VHOST_WORK_QUEUED, &work->flags)) {
- /* We can only add the work to the list after we're
- * sure it was not in the list.
- * test_and_set_bit() implies a memory barrier.
- */
- llist_add(&work->node, &dev->work_list);
- wake_up_process(dev->worker);
+ if (!dev->workers)
+ return false;
+
+ for (i = 0; i < dev->num_workers; i++) {
+ if (!llist_empty(&dev->workers[i]->work_list))
+ return true;
}
-}
-EXPORT_SYMBOL_GPL(vhost_work_queue);
-/* A lockless hint for busy polling code to exit the loop */
-bool vhost_has_work(struct vhost_dev *dev)
-{
- return !llist_empty(&dev->work_list);
+ return false;
}
EXPORT_SYMBOL_GPL(vhost_has_work);
@@ -339,7 +363,8 @@ static void vhost_vq_reset(struct vhost_dev *dev,
static int vhost_worker(void *data)
{
- struct vhost_dev *dev = data;
+ struct vhost_worker *worker = data;
+ struct vhost_dev *dev = worker->dev;
struct vhost_work *work, *work_next;
struct llist_node *node;
@@ -353,8 +378,7 @@ static int vhost_worker(void *data)
__set_current_state(TASK_RUNNING);
break;
}
-
- node = llist_del_all(&dev->work_list);
+ node = llist_del_all(&worker->work_list);
if (!node)
schedule();
@@ -507,13 +531,13 @@ int vhost_dev_init(struct vhost_dev *dev,
dev->umem = NULL;
dev->iotlb = NULL;
dev->mm = NULL;
- dev->worker = NULL;
+ dev->workers = NULL;
+ dev->num_workers = 0;
dev->iov_limit = iov_limit;
dev->weight = weight;
- dev->byte_weight = byte_weight;
dev->use_worker = use_worker;
+ dev->byte_weight = byte_weight;
dev->msg_handler = msg_handler;
- init_llist_head(&dev->work_list);
init_waitqueue_head(&dev->wait);
INIT_LIST_HEAD(&dev->read_list);
INIT_LIST_HEAD(&dev->pending_list);
@@ -559,16 +583,28 @@ static void vhost_attach_cgroups_work(struct vhost_work *work)
s->ret = cgroup_attach_task_all(s->owner, current);
}
-static int vhost_attach_cgroups(struct vhost_dev *dev)
+static int vhost_attach_cgroups_on(struct vhost_dev *dev, int worker_id)
{
struct vhost_attach_cgroups_struct attach;
attach.owner = current;
vhost_work_init(&attach.work, vhost_attach_cgroups_work);
- vhost_work_queue(dev, &attach.work);
- vhost_work_dev_flush(dev);
+ vhost_work_queue_on(dev, &attach.work, worker_id);
+ vhost_work_flush_on(dev, worker_id);
return attach.ret;
}
+static int vhost_attach_cgroups(struct vhost_dev *dev, int first_worker)
+{
+ int i, ret = 0;
+
+ for (i = first_worker; i < dev->num_workers; i++) {
+ ret = vhost_attach_cgroups_on(dev, i);
+ if (ret)
+ break;
+ }
+
+ return ret;
+}
/* Caller should have device mutex */
bool vhost_dev_has_owner(struct vhost_dev *dev)
@@ -607,10 +643,117 @@ static void vhost_detach_mm(struct vhost_dev *dev)
dev->mm = NULL;
}
+static void vhost_workers_free(struct vhost_dev *dev)
+{
+ struct vhost_worker *worker;
+ int i;
+
+ if (!dev->workers)
+ return;
+
+ for (i = 0; i < dev->num_workers; i++) {
+ worker = dev->workers[i];
+
+ WARN_ON(!llist_empty(&worker->work_list));
+ kthread_stop(worker->task);
+ kfree(worker);
+ }
+
+ kfree(dev->workers);
+ dev->workers = NULL;
+ dev->num_workers = 0;
+}
+
+static int vhost_worker_create(struct vhost_dev *dev, int worker_id)
+{
+ struct vhost_worker *worker;
+ struct task_struct *task;
+ int ret;
+
+ worker = kzalloc(sizeof(*worker), GFP_KERNEL);
+ if (!worker)
+ return -ENOMEM;
+
+ init_llist_head(&worker->work_list);
+ worker->dev = dev;
+
+ task = kthread_create(vhost_worker, worker, "vhost-%d", current->pid);
+ if (IS_ERR(task)) {
+ ret = PTR_ERR(task);
+ goto free_worker;
+ }
+
+ dev->workers[worker_id] = worker;
+ worker->task = task;
+ wake_up_process(task); /* avoid contributing to loadavg */
+ return 0;
+
+free_worker:
+ kfree(worker);
+ return ret;
+}
+
+/**
+ * vhost_workers_create - create vhost workers and attach to cgroup
+ * @dev: vhost device
+ * @new_num_workers: the total number of workers we want after this returns
+ *
+ * Caller must have the device mutex and have stopped operations that
+ * can access the workers array.
+ */
+int vhost_workers_create(struct vhost_dev *dev, int new_num_workers)
+{
+ struct vhost_worker **new_workers;
+ struct mm_struct *mm;
+ bool owner_match = true;
+ int i, err, start;
+
+ if (new_num_workers = dev->num_workers)
+ return 0;
+
+ if (new_num_workers < dev->num_workers)
+ return -EINVAL;
+
+ if (vhost_dev_has_owner(dev)) {
+ mm = get_task_mm(current);
+ if (mm != dev->mm)
+ owner_match = false;
+ mmput(mm);
+ if (!owner_match)
+ return -EBUSY;
+ }
+
+ new_workers = krealloc(dev->workers, new_num_workers * sizeof(*new_workers),
+ GFP_KERNEL);
+ if (!new_workers) {
+ err = -ENOMEM;
+ goto free_workers;
+ }
+ dev->workers = new_workers;
+
+ start = dev->num_workers;
+ for (i = start; i < new_num_workers; i++) {
+ err = vhost_worker_create(dev, i);
+ if (err)
+ goto free_workers;
+ dev->num_workers++;
+ }
+
+ err = vhost_attach_cgroups(dev, start);
+ if (err)
+ goto free_workers;
+
+ return 0;
+
+free_workers:
+ vhost_workers_free(dev);
+ return err;
+}
+EXPORT_SYMBOL_GPL(vhost_workers_create);
+
/* Caller should have device mutex */
long vhost_dev_set_owner(struct vhost_dev *dev)
{
- struct task_struct *worker;
int err;
/* Is there an owner already? */
@@ -623,27 +766,16 @@ long vhost_dev_set_owner(struct vhost_dev *dev)
dev->kcov_handle = kcov_common_handle();
if (dev->use_worker) {
- worker = kthread_create(vhost_worker, dev,
- "vhost-%d", current->pid);
- if (IS_ERR(worker)) {
- err = PTR_ERR(worker);
- goto err_worker;
- }
-
- dev->worker = worker;
- wake_up_process(worker); /* avoid contributing to loadavg */
-
- err = vhost_attach_cgroups(dev);
+ /*
+ * All drivers that set use_worker=true, use at least one
+ * worker. Drivers like vhost-scsi may override this later.
+ */
+ err = vhost_workers_create(dev, 1);
if (err)
- goto err_cgroup;
+ goto err_worker;
}
return 0;
-err_cgroup:
- if (dev->worker) {
- kthread_stop(dev->worker);
- dev->worker = NULL;
- }
err_worker:
vhost_detach_mm(dev);
dev->kcov_handle = 0;
@@ -736,12 +868,8 @@ void vhost_dev_cleanup(struct vhost_dev *dev)
dev->iotlb = NULL;
vhost_clear_msg(dev);
wake_up_interruptible_poll(&dev->wait, EPOLLIN | EPOLLRDNORM);
- WARN_ON(!llist_empty(&dev->work_list));
- if (dev->worker) {
- kthread_stop(dev->worker);
- dev->worker = NULL;
- dev->kcov_handle = 0;
- }
+ vhost_workers_free(dev);
+ dev->kcov_handle = 0;
vhost_detach_mm(dev);
}
EXPORT_SYMBOL_GPL(vhost_dev_cleanup);
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 2c6473e..a91be8a 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -25,6 +25,12 @@ struct vhost_work {
unsigned long flags;
};
+struct vhost_worker {
+ struct task_struct *task;
+ struct llist_head work_list;
+ struct vhost_dev *dev;
+};
+
/* Poll a file (eventfd or socket) */
/* Note: there's nothing vhost specific about this structure. */
struct vhost_poll {
@@ -39,6 +45,7 @@ struct vhost_poll {
void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn);
void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work);
bool vhost_has_work(struct vhost_dev *dev);
+int vhost_workers_create(struct vhost_dev *dev, int new_num_workers);
void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn,
__poll_t mask, struct vhost_dev *dev);
@@ -152,8 +159,8 @@ struct vhost_dev {
int nvqs;
int max_nvqs;
struct eventfd_ctx *log_ctx;
- struct llist_head work_list;
- struct task_struct *worker;
+ struct vhost_worker **workers;
+ int num_workers;
struct vhost_iotlb *umem;
struct vhost_iotlb *iotlb;
spinlock_t iotlb_lock;
@@ -175,6 +182,7 @@ int vhost_dev_init(struct vhost_dev *dev, struct vhost_virtqueue **vqs,
int byte_weight, bool use_worker,
int (*msg_handler)(struct vhost_dev *dev,
struct vhost_iotlb_msg *msg));
+int vhost_vq_set_worker(struct vhost_virtqueue *vq, int worker_id);
long vhost_dev_set_owner(struct vhost_dev *dev);
bool vhost_dev_has_owner(struct vhost_dev *dev);
long vhost_dev_check_owner(struct vhost_dev *);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH 12/16] vhost: support multiple worker threads
2020-10-07 20:54 ` Mike Christie
@ 2020-10-08 17:56 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-08 17:56 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
On 10/7/20 3:54 PM, Mike Christie wrote:
> This is a prep patch to support multiple vhost worker threads per vhost
> dev. This patch converts the code that had assumed a single worker
> thread by:
>
> 1. Moving worker related fields to a new struct vhost_worker.
> 2. Converting vhost.c code to use the new struct and assume we will
> have an array of workers.
> 3. It also exports a helper function that will be used in the last
> patch when vhost-scsi is converted to use this new functionality.
>
Oh yeah I also wanted to bring up this patch:
https://www.spinics.net/lists/netdev/msg192548.html
The problem with my multi-threading patches is that I was focused on
the cgroup support parts and that lead to some gross decisions.
1. I kept the cgroup support, but as a result I do not have control
over the threading affinity and making sure cmds are executed on a
optimal CPU like the above patches do.
When I drop the cgroup support and make sure threads are bound to
specific CPUs and then make sure IO is run on the CPU it came in on
then IOPs jumps from 600K to 800K for vhost-scsi.
2. I can possible create a lot of threads.
So a couple open issues are:
1. Can we do a thread per cpu that is shared across all vhost devices?
That would lead to dropping the cgroup vhost worker support.
2. Can we just use the kernel's workqueues then?
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 12/16] vhost: support multiple worker threads
@ 2020-10-08 17:56 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-08 17:56 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
On 10/7/20 3:54 PM, Mike Christie wrote:
> This is a prep patch to support multiple vhost worker threads per vhost
> dev. This patch converts the code that had assumed a single worker
> thread by:
>
> 1. Moving worker related fields to a new struct vhost_worker.
> 2. Converting vhost.c code to use the new struct and assume we will
> have an array of workers.
> 3. It also exports a helper function that will be used in the last
> patch when vhost-scsi is converted to use this new functionality.
>
Oh yeah I also wanted to bring up this patch:
https://www.spinics.net/lists/netdev/msg192548.html
The problem with my multi-threading patches is that I was focused on
the cgroup support parts and that lead to some gross decisions.
1. I kept the cgroup support, but as a result I do not have control
over the threading affinity and making sure cmds are executed on a
optimal CPU like the above patches do.
When I drop the cgroup support and make sure threads are bound to
specific CPUs and then make sure IO is run on the CPU it came in on
then IOPs jumps from 600K to 800K for vhost-scsi.
2. I can possible create a lot of threads.
So a couple open issues are:
1. Can we do a thread per cpu that is shared across all vhost devices?
That would lead to dropping the cgroup vhost worker support.
2. Can we just use the kernel's workqueues then?
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 12/16] vhost: support multiple worker threads
2020-10-08 17:56 ` Mike Christie
(?)
@ 2020-10-08 20:26 ` Michael S. Tsirkin
-1 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-08 20:26 UTC (permalink / raw)
To: Mike Christie
Cc: martin.petersen, linux-scsi, target-devel, jasowang, pbonzini,
stefanha, virtualization
On Thu, Oct 08, 2020 at 12:56:53PM -0500, Mike Christie wrote:
> On 10/7/20 3:54 PM, Mike Christie wrote:
> > This is a prep patch to support multiple vhost worker threads per vhost
> > dev. This patch converts the code that had assumed a single worker
> > thread by:
> >
> > 1. Moving worker related fields to a new struct vhost_worker.
> > 2. Converting vhost.c code to use the new struct and assume we will
> > have an array of workers.
> > 3. It also exports a helper function that will be used in the last
> > patch when vhost-scsi is converted to use this new functionality.
> >
>
> Oh yeah I also wanted to bring up this patch:
>
> https://www.spinics.net/lists/netdev/msg192548.html
>
> The problem with my multi-threading patches is that I was focused on
> the cgroup support parts and that lead to some gross decisions.
>
> 1. I kept the cgroup support, but as a result I do not have control
> over the threading affinity and making sure cmds are executed on a
> optimal CPU like the above patches do.
>
> When I drop the cgroup support and make sure threads are bound to
> specific CPUs and then make sure IO is run on the CPU it came in on
> then IOPs jumps from 600K to 800K for vhost-scsi.
>
> 2. I can possible create a lot of threads.
>
> So a couple open issues are:
>
> 1. Can we do a thread per cpu that is shared across all vhost devices?
> That would lead to dropping the cgroup vhost worker support.
>
> 2. Can we just use the kernel's workqueues then?
Problem is, we are talking about *lots* of CPU, IO etc and ATM cgroups
is how people expect to account for that overhead.
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 12/16] vhost: support multiple worker threads
@ 2020-10-08 20:26 ` Michael S. Tsirkin
0 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-08 20:26 UTC (permalink / raw)
To: Mike Christie
Cc: martin.petersen, linux-scsi, virtualization, target-devel,
stefanha, pbonzini
On Thu, Oct 08, 2020 at 12:56:53PM -0500, Mike Christie wrote:
> On 10/7/20 3:54 PM, Mike Christie wrote:
> > This is a prep patch to support multiple vhost worker threads per vhost
> > dev. This patch converts the code that had assumed a single worker
> > thread by:
> >
> > 1. Moving worker related fields to a new struct vhost_worker.
> > 2. Converting vhost.c code to use the new struct and assume we will
> > have an array of workers.
> > 3. It also exports a helper function that will be used in the last
> > patch when vhost-scsi is converted to use this new functionality.
> >
>
> Oh yeah I also wanted to bring up this patch:
>
> https://www.spinics.net/lists/netdev/msg192548.html
>
> The problem with my multi-threading patches is that I was focused on
> the cgroup support parts and that lead to some gross decisions.
>
> 1. I kept the cgroup support, but as a result I do not have control
> over the threading affinity and making sure cmds are executed on a
> optimal CPU like the above patches do.
>
> When I drop the cgroup support and make sure threads are bound to
> specific CPUs and then make sure IO is run on the CPU it came in on
> then IOPs jumps from 600K to 800K for vhost-scsi.
>
> 2. I can possible create a lot of threads.
>
> So a couple open issues are:
>
> 1. Can we do a thread per cpu that is shared across all vhost devices?
> That would lead to dropping the cgroup vhost worker support.
>
> 2. Can we just use the kernel's workqueues then?
Problem is, we are talking about *lots* of CPU, IO etc and ATM cgroups
is how people expect to account for that overhead.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 12/16] vhost: support multiple worker threads
@ 2020-10-08 20:26 ` Michael S. Tsirkin
0 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-08 20:26 UTC (permalink / raw)
To: Mike Christie
Cc: martin.petersen, linux-scsi, target-devel, jasowang, pbonzini,
stefanha, virtualization
On Thu, Oct 08, 2020 at 12:56:53PM -0500, Mike Christie wrote:
> On 10/7/20 3:54 PM, Mike Christie wrote:
> > This is a prep patch to support multiple vhost worker threads per vhost
> > dev. This patch converts the code that had assumed a single worker
> > thread by:
> >
> > 1. Moving worker related fields to a new struct vhost_worker.
> > 2. Converting vhost.c code to use the new struct and assume we will
> > have an array of workers.
> > 3. It also exports a helper function that will be used in the last
> > patch when vhost-scsi is converted to use this new functionality.
> >
>
> Oh yeah I also wanted to bring up this patch:
>
> https://www.spinics.net/lists/netdev/msg192548.html
>
> The problem with my multi-threading patches is that I was focused on
> the cgroup support parts and that lead to some gross decisions.
>
> 1. I kept the cgroup support, but as a result I do not have control
> over the threading affinity and making sure cmds are executed on a
> optimal CPU like the above patches do.
>
> When I drop the cgroup support and make sure threads are bound to
> specific CPUs and then make sure IO is run on the CPU it came in on
> then IOPs jumps from 600K to 800K for vhost-scsi.
>
> 2. I can possible create a lot of threads.
>
> So a couple open issues are:
>
> 1. Can we do a thread per cpu that is shared across all vhost devices?
> That would lead to dropping the cgroup vhost worker support.
>
> 2. Can we just use the kernel's workqueues then?
Problem is, we are talking about *lots* of CPU, IO etc and ATM cgroups
is how people expect to account for that overhead.
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH 13/16] vhost poll: fix coding style
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
We use like 3 coding styles in this struct. Switch to just tabs.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/vhost.h | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index a91be8a..2bbe85f 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -34,12 +34,12 @@ struct vhost_worker {
/* Poll a file (eventfd or socket) */
/* Note: there's nothing vhost specific about this structure. */
struct vhost_poll {
- poll_table table;
- wait_queue_head_t *wqh;
- wait_queue_entry_t wait;
- struct vhost_work work;
- __poll_t mask;
- struct vhost_dev *dev;
+ poll_table table;
+ wait_queue_head_t *wqh;
+ wait_queue_entry_t wait;
+ struct vhost_work work;
+ __poll_t mask;
+ struct vhost_dev *dev;
};
void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 13/16] vhost poll: fix coding style
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
We use like 3 coding styles in this struct. Switch to just tabs.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/vhost.h | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index a91be8a..2bbe85f 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -34,12 +34,12 @@ struct vhost_worker {
/* Poll a file (eventfd or socket) */
/* Note: there's nothing vhost specific about this structure. */
struct vhost_poll {
- poll_table table;
- wait_queue_head_t *wqh;
- wait_queue_entry_t wait;
- struct vhost_work work;
- __poll_t mask;
- struct vhost_dev *dev;
+ poll_table table;
+ wait_queue_head_t *wqh;
+ wait_queue_entry_t wait;
+ struct vhost_work work;
+ __poll_t mask;
+ struct vhost_dev *dev;
};
void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 14/16] vhost: poll support support multiple workers
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:54 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
The final patches are going to have vhost scsi create a vhost worker
per IO vq. This patch converts the poll code to poll and queue work on
the worker that is tied to the vq (in this patch we maintain the old
behavior where all vqs use a single worker).
For drivers that do not convert over to the multiple worker support
or for the case where the user just does not want to allocate the
resources then we maintain support for the single worker case.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/net.c | 6 ++++--
drivers/vhost/vhost.c | 25 ++++++++++++++++++++++---
drivers/vhost/vhost.h | 9 +++++++--
3 files changed, 33 insertions(+), 7 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index fce46f0..a316ed0 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -1322,8 +1322,10 @@ static int vhost_net_open(struct inode *inode, struct file *f)
NULL))
goto err_dev_init;
- vhost_poll_init(n->poll + VHOST_NET_VQ_TX, handle_tx_net, EPOLLOUT, dev);
- vhost_poll_init(n->poll + VHOST_NET_VQ_RX, handle_rx_net, EPOLLIN, dev);
+ vhost_poll_init(n->poll + VHOST_NET_VQ_TX, handle_tx_net, EPOLLOUT, dev,
+ vqs[VHOST_NET_VQ_TX]);
+ vhost_poll_init(n->poll + VHOST_NET_VQ_RX, handle_rx_net, EPOLLIN, dev,
+ vqs[VHOST_NET_VQ_RX]);
f->private_data = n;
n->page_frag.page = NULL;
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index e46b39e..f0038d6 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -187,13 +187,15 @@ void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn)
/* Init poll structure */
void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn,
- __poll_t mask, struct vhost_dev *dev)
+ __poll_t mask, struct vhost_dev *dev,
+ struct vhost_virtqueue *vq)
{
init_waitqueue_func_entry(&poll->wait, vhost_poll_wakeup);
init_poll_funcptr(&poll->table, vhost_poll_func);
poll->mask = mask;
poll->dev = dev;
poll->wqh = NULL;
+ poll->vq = vq;
vhost_work_init(&poll->work, fn);
}
@@ -283,6 +285,12 @@ void vhost_poll_flush(struct vhost_poll *poll)
}
EXPORT_SYMBOL_GPL(vhost_poll_flush);
+void vhost_vq_work_queue(struct vhost_virtqueue *vq, struct vhost_work *work)
+{
+ vhost_work_queue_on(vq->dev, work, vq->worker_id);
+}
+EXPORT_SYMBOL_GPL(vhost_vq_work_queue);
+
/* A lockless hint for busy polling code to exit the loop */
bool vhost_has_work(struct vhost_dev *dev)
{
@@ -302,7 +310,7 @@ bool vhost_has_work(struct vhost_dev *dev)
void vhost_poll_queue(struct vhost_poll *poll)
{
- vhost_work_queue(poll->dev, &poll->work);
+ vhost_vq_work_queue(poll->vq, &poll->work);
}
EXPORT_SYMBOL_GPL(vhost_poll_queue);
@@ -488,11 +496,12 @@ static void __vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq)
vq->indirect = NULL;
vq->heads = NULL;
vq->dev = dev;
+ vq->worker_id = 0;
mutex_init(&vq->mutex);
vhost_vq_reset(dev, vq);
if (vq->handle_kick)
- vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev);
+ vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev, vq);
}
static int vhost_vq_init(struct vhost_dev *dev, int vq_idx)
@@ -643,6 +652,16 @@ static void vhost_detach_mm(struct vhost_dev *dev)
dev->mm = NULL;
}
+int vhost_vq_set_worker(struct vhost_virtqueue *vq, int worker_id)
+{
+ if (vhost_vq_get_backend(vq))
+ return -EBUSY;
+
+ vq->worker_id = worker_id;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(vhost_vq_set_worker);
+
static void vhost_workers_free(struct vhost_dev *dev)
{
struct vhost_worker *worker;
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 2bbe85f..6e9643d 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -16,6 +16,7 @@
#include <linux/irqbypass.h>
struct vhost_work;
+struct vhost_virtqueue *vq;
typedef void (*vhost_work_fn_t)(struct vhost_work *work);
#define VHOST_WORK_QUEUED 1
@@ -32,7 +33,6 @@ struct vhost_worker {
};
/* Poll a file (eventfd or socket) */
-/* Note: there's nothing vhost specific about this structure. */
struct vhost_poll {
poll_table table;
wait_queue_head_t *wqh;
@@ -40,15 +40,19 @@ struct vhost_poll {
struct vhost_work work;
__poll_t mask;
struct vhost_dev *dev;
+ struct vhost_virtqueue *vq;
};
+int vhost_vq_set_worker(struct vhost_virtqueue *vq, int worker_id);
+void vhost_vq_work_queue(struct vhost_virtqueue *vq, struct vhost_work *work);
void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn);
void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work);
bool vhost_has_work(struct vhost_dev *dev);
int vhost_workers_create(struct vhost_dev *dev, int new_num_workers);
void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn,
- __poll_t mask, struct vhost_dev *dev);
+ __poll_t mask, struct vhost_dev *dev,
+ struct vhost_virtqueue *vq);
int vhost_poll_start(struct vhost_poll *poll, struct file *file);
void vhost_poll_stop(struct vhost_poll *poll);
void vhost_poll_flush(struct vhost_poll *poll);
@@ -92,6 +96,7 @@ struct vhost_virtqueue {
struct vhost_poll poll;
+ int worker_id;
/* The routine to call when the Guest pings us, or timeout. */
vhost_work_fn_t handle_kick;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 14/16] vhost: poll support support multiple workers
@ 2020-10-07 20:54 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:54 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
The final patches are going to have vhost scsi create a vhost worker
per IO vq. This patch converts the poll code to poll and queue work on
the worker that is tied to the vq (in this patch we maintain the old
behavior where all vqs use a single worker).
For drivers that do not convert over to the multiple worker support
or for the case where the user just does not want to allocate the
resources then we maintain support for the single worker case.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/net.c | 6 ++++--
drivers/vhost/vhost.c | 25 ++++++++++++++++++++++---
drivers/vhost/vhost.h | 9 +++++++--
3 files changed, 33 insertions(+), 7 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index fce46f0..a316ed0 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -1322,8 +1322,10 @@ static int vhost_net_open(struct inode *inode, struct file *f)
NULL))
goto err_dev_init;
- vhost_poll_init(n->poll + VHOST_NET_VQ_TX, handle_tx_net, EPOLLOUT, dev);
- vhost_poll_init(n->poll + VHOST_NET_VQ_RX, handle_rx_net, EPOLLIN, dev);
+ vhost_poll_init(n->poll + VHOST_NET_VQ_TX, handle_tx_net, EPOLLOUT, dev,
+ vqs[VHOST_NET_VQ_TX]);
+ vhost_poll_init(n->poll + VHOST_NET_VQ_RX, handle_rx_net, EPOLLIN, dev,
+ vqs[VHOST_NET_VQ_RX]);
f->private_data = n;
n->page_frag.page = NULL;
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index e46b39e..f0038d6 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -187,13 +187,15 @@ void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn)
/* Init poll structure */
void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn,
- __poll_t mask, struct vhost_dev *dev)
+ __poll_t mask, struct vhost_dev *dev,
+ struct vhost_virtqueue *vq)
{
init_waitqueue_func_entry(&poll->wait, vhost_poll_wakeup);
init_poll_funcptr(&poll->table, vhost_poll_func);
poll->mask = mask;
poll->dev = dev;
poll->wqh = NULL;
+ poll->vq = vq;
vhost_work_init(&poll->work, fn);
}
@@ -283,6 +285,12 @@ void vhost_poll_flush(struct vhost_poll *poll)
}
EXPORT_SYMBOL_GPL(vhost_poll_flush);
+void vhost_vq_work_queue(struct vhost_virtqueue *vq, struct vhost_work *work)
+{
+ vhost_work_queue_on(vq->dev, work, vq->worker_id);
+}
+EXPORT_SYMBOL_GPL(vhost_vq_work_queue);
+
/* A lockless hint for busy polling code to exit the loop */
bool vhost_has_work(struct vhost_dev *dev)
{
@@ -302,7 +310,7 @@ bool vhost_has_work(struct vhost_dev *dev)
void vhost_poll_queue(struct vhost_poll *poll)
{
- vhost_work_queue(poll->dev, &poll->work);
+ vhost_vq_work_queue(poll->vq, &poll->work);
}
EXPORT_SYMBOL_GPL(vhost_poll_queue);
@@ -488,11 +496,12 @@ static void __vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq)
vq->indirect = NULL;
vq->heads = NULL;
vq->dev = dev;
+ vq->worker_id = 0;
mutex_init(&vq->mutex);
vhost_vq_reset(dev, vq);
if (vq->handle_kick)
- vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev);
+ vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev, vq);
}
static int vhost_vq_init(struct vhost_dev *dev, int vq_idx)
@@ -643,6 +652,16 @@ static void vhost_detach_mm(struct vhost_dev *dev)
dev->mm = NULL;
}
+int vhost_vq_set_worker(struct vhost_virtqueue *vq, int worker_id)
+{
+ if (vhost_vq_get_backend(vq))
+ return -EBUSY;
+
+ vq->worker_id = worker_id;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(vhost_vq_set_worker);
+
static void vhost_workers_free(struct vhost_dev *dev)
{
struct vhost_worker *worker;
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 2bbe85f..6e9643d 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -16,6 +16,7 @@
#include <linux/irqbypass.h>
struct vhost_work;
+struct vhost_virtqueue *vq;
typedef void (*vhost_work_fn_t)(struct vhost_work *work);
#define VHOST_WORK_QUEUED 1
@@ -32,7 +33,6 @@ struct vhost_worker {
};
/* Poll a file (eventfd or socket) */
-/* Note: there's nothing vhost specific about this structure. */
struct vhost_poll {
poll_table table;
wait_queue_head_t *wqh;
@@ -40,15 +40,19 @@ struct vhost_poll {
struct vhost_work work;
__poll_t mask;
struct vhost_dev *dev;
+ struct vhost_virtqueue *vq;
};
+int vhost_vq_set_worker(struct vhost_virtqueue *vq, int worker_id);
+void vhost_vq_work_queue(struct vhost_virtqueue *vq, struct vhost_work *work);
void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn);
void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work);
bool vhost_has_work(struct vhost_dev *dev);
int vhost_workers_create(struct vhost_dev *dev, int new_num_workers);
void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn,
- __poll_t mask, struct vhost_dev *dev);
+ __poll_t mask, struct vhost_dev *dev,
+ struct vhost_virtqueue *vq);
int vhost_poll_start(struct vhost_poll *poll, struct file *file);
void vhost_poll_stop(struct vhost_poll *poll);
void vhost_poll_flush(struct vhost_poll *poll);
@@ -92,6 +96,7 @@ struct vhost_virtqueue {
struct vhost_poll poll;
+ int worker_id;
/* The routine to call when the Guest pings us, or timeout. */
vhost_work_fn_t handle_kick;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [PATCH 14/16] vhost: poll support support multiple workers
2020-10-07 20:54 ` Mike Christie
(?)
(?)
@ 2020-10-08 0:46 ` kernel test robot
-1 siblings, 0 replies; 69+ messages in thread
From: kernel test robot @ 2020-10-08 0:46 UTC (permalink / raw)
To: Mike Christie, martin.petersen, linux-scsi, target-devel, mst,
jasowang, pbonzini, stefanha, virtualization
Cc: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 2074 bytes --]
Hi Mike,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on vhost/linux-next]
[also build test ERROR on next-20201007]
[cannot apply to v5.9-rc8]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: m68k-allyesconfig (attached as .config)
compiler: m68k-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/c9bcafefcac3c32ade0ba533609d7b1ddc343c54
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
git checkout c9bcafefcac3c32ade0ba533609d7b1ddc343c54
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=m68k
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
>> m68k-linux-ld: drivers/vhost/scsi.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
m68k-linux-ld: drivers/vhost/vsock.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
m68k-linux-ld: drivers/vhost/vdpa.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
m68k-linux-ld: drivers/vhost/vhost.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 58118 bytes --]
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 14/16] vhost: poll support support multiple workers
@ 2020-10-08 0:46 ` kernel test robot
0 siblings, 0 replies; 69+ messages in thread
From: kernel test robot @ 2020-10-08 0:46 UTC (permalink / raw)
To: Mike Christie, martin.petersen, linux-scsi, target-devel, mst,
jasowang, pbonzini, stefanha, virtualization
Cc: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 2074 bytes --]
Hi Mike,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on vhost/linux-next]
[also build test ERROR on next-20201007]
[cannot apply to v5.9-rc8]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: m68k-allyesconfig (attached as .config)
compiler: m68k-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/c9bcafefcac3c32ade0ba533609d7b1ddc343c54
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
git checkout c9bcafefcac3c32ade0ba533609d7b1ddc343c54
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=m68k
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
>> m68k-linux-ld: drivers/vhost/scsi.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
m68k-linux-ld: drivers/vhost/vsock.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
m68k-linux-ld: drivers/vhost/vdpa.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
m68k-linux-ld: drivers/vhost/vhost.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 58118 bytes --]
[-- Attachment #3: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 14/16] vhost: poll support support multiple workers
@ 2020-10-08 0:46 ` kernel test robot
0 siblings, 0 replies; 69+ messages in thread
From: kernel test robot @ 2020-10-08 0:46 UTC (permalink / raw)
To: Mike Christie, martin.petersen, linux-scsi, target-devel, mst,
jasowang, pbonzini, stefanha, virtualization
Cc: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 2074 bytes --]
Hi Mike,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on vhost/linux-next]
[also build test ERROR on next-20201007]
[cannot apply to v5.9-rc8]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: m68k-allyesconfig (attached as .config)
compiler: m68k-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/c9bcafefcac3c32ade0ba533609d7b1ddc343c54
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
git checkout c9bcafefcac3c32ade0ba533609d7b1ddc343c54
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=m68k
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
>> m68k-linux-ld: drivers/vhost/scsi.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
m68k-linux-ld: drivers/vhost/vsock.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
m68k-linux-ld: drivers/vhost/vdpa.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
m68k-linux-ld: drivers/vhost/vhost.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 58118 bytes --]
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 14/16] vhost: poll support support multiple workers
@ 2020-10-08 0:46 ` kernel test robot
0 siblings, 0 replies; 69+ messages in thread
From: kernel test robot @ 2020-10-08 0:46 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 2114 bytes --]
Hi Mike,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on vhost/linux-next]
[also build test ERROR on next-20201007]
[cannot apply to v5.9-rc8]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
config: m68k-allyesconfig (attached as .config)
compiler: m68k-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/c9bcafefcac3c32ade0ba533609d7b1ddc343c54
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
git checkout c9bcafefcac3c32ade0ba533609d7b1ddc343c54
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=m68k
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
>> m68k-linux-ld: drivers/vhost/scsi.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
m68k-linux-ld: drivers/vhost/vsock.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
m68k-linux-ld: drivers/vhost/vdpa.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
m68k-linux-ld: drivers/vhost/vhost.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 58118 bytes --]
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 14/16] vhost: poll support support multiple workers
2020-10-08 0:46 ` kernel test robot
(?)
(?)
@ 2020-10-23 15:43 ` Michael S. Tsirkin
-1 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-23 15:43 UTC (permalink / raw)
To: kernel test robot
Cc: Mike Christie, martin.petersen, linux-scsi, target-devel,
jasowang, pbonzini, stefanha, virtualization, kbuild-all
On Thu, Oct 08, 2020 at 08:46:42AM +0800, kernel test robot wrote:
> Hi Mike,
>
> Thank you for the patch! Yet something to improve:
>
> [auto build test ERROR on vhost/linux-next]
> [also build test ERROR on next-20201007]
> [cannot apply to v5.9-rc8]
> [If your patch is applied to the wrong git tree, kindly drop us a note.
> And when submitting patch, we suggest to use '--base' as documented in
> https://git-scm.com/docs/git-format-patch]
>
> url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
> base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
> config: m68k-allyesconfig (attached as .config)
> compiler: m68k-linux-gcc (GCC) 9.3.0
> reproduce (this is a W=1 build):
> wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> chmod +x ~/bin/make.cross
> # https://github.com/0day-ci/linux/commit/c9bcafefcac3c32ade0ba533609d7b1ddc343c54
> git remote add linux-review https://github.com/0day-ci/linux
> git fetch --no-tags linux-review Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
> git checkout c9bcafefcac3c32ade0ba533609d7b1ddc343c54
> # save the attached .config to linux build tree
> COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=m68k
>
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
>
> All errors (new ones prefixed by >>):
>
> >> m68k-linux-ld: drivers/vhost/scsi.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
> m68k-linux-ld: drivers/vhost/vsock.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
> m68k-linux-ld: drivers/vhost/vdpa.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
> m68k-linux-ld: drivers/vhost/vhost.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
Mike, what's going on with these failures?
Can you figure it out pls?
> ---
> 0-DAY CI Kernel Test Service, Intel Corporation
> https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 14/16] vhost: poll support support multiple workers
@ 2020-10-23 15:43 ` Michael S. Tsirkin
0 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-23 15:43 UTC (permalink / raw)
To: kernel test robot
Cc: kbuild-all, linux-scsi, martin.petersen, virtualization,
target-devel, stefanha, pbonzini, Mike Christie
On Thu, Oct 08, 2020 at 08:46:42AM +0800, kernel test robot wrote:
> Hi Mike,
>
> Thank you for the patch! Yet something to improve:
>
> [auto build test ERROR on vhost/linux-next]
> [also build test ERROR on next-20201007]
> [cannot apply to v5.9-rc8]
> [If your patch is applied to the wrong git tree, kindly drop us a note.
> And when submitting patch, we suggest to use '--base' as documented in
> https://git-scm.com/docs/git-format-patch]
>
> url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
> base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
> config: m68k-allyesconfig (attached as .config)
> compiler: m68k-linux-gcc (GCC) 9.3.0
> reproduce (this is a W=1 build):
> wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> chmod +x ~/bin/make.cross
> # https://github.com/0day-ci/linux/commit/c9bcafefcac3c32ade0ba533609d7b1ddc343c54
> git remote add linux-review https://github.com/0day-ci/linux
> git fetch --no-tags linux-review Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
> git checkout c9bcafefcac3c32ade0ba533609d7b1ddc343c54
> # save the attached .config to linux build tree
> COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=m68k
>
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
>
> All errors (new ones prefixed by >>):
>
> >> m68k-linux-ld: drivers/vhost/scsi.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
> m68k-linux-ld: drivers/vhost/vsock.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
> m68k-linux-ld: drivers/vhost/vdpa.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
> m68k-linux-ld: drivers/vhost/vhost.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
Mike, what's going on with these failures?
Can you figure it out pls?
> ---
> 0-DAY CI Kernel Test Service, Intel Corporation
> https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 14/16] vhost: poll support support multiple workers
@ 2020-10-23 15:43 ` Michael S. Tsirkin
0 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-23 15:43 UTC (permalink / raw)
To: kernel test robot
Cc: Mike Christie, martin.petersen, linux-scsi, target-devel,
jasowang, pbonzini, stefanha, virtualization, kbuild-all
On Thu, Oct 08, 2020 at 08:46:42AM +0800, kernel test robot wrote:
> Hi Mike,
>
> Thank you for the patch! Yet something to improve:
>
> [auto build test ERROR on vhost/linux-next]
> [also build test ERROR on next-20201007]
> [cannot apply to v5.9-rc8]
> [If your patch is applied to the wrong git tree, kindly drop us a note.
> And when submitting patch, we suggest to use '--base' as documented in
> https://git-scm.com/docs/git-format-patch]
>
> url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
> base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
> config: m68k-allyesconfig (attached as .config)
> compiler: m68k-linux-gcc (GCC) 9.3.0
> reproduce (this is a W=1 build):
> wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> chmod +x ~/bin/make.cross
> # https://github.com/0day-ci/linux/commit/c9bcafefcac3c32ade0ba533609d7b1ddc343c54
> git remote add linux-review https://github.com/0day-ci/linux
> git fetch --no-tags linux-review Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
> git checkout c9bcafefcac3c32ade0ba533609d7b1ddc343c54
> # save the attached .config to linux build tree
> COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=m68k
>
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
>
> All errors (new ones prefixed by >>):
>
> >> m68k-linux-ld: drivers/vhost/scsi.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
> m68k-linux-ld: drivers/vhost/vsock.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
> m68k-linux-ld: drivers/vhost/vdpa.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
> m68k-linux-ld: drivers/vhost/vhost.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
Mike, what's going on with these failures?
Can you figure it out pls?
> ---
> 0-DAY CI Kernel Test Service, Intel Corporation
> https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 14/16] vhost: poll support support multiple workers
@ 2020-10-23 15:43 ` Michael S. Tsirkin
0 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-23 15:43 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 2332 bytes --]
On Thu, Oct 08, 2020 at 08:46:42AM +0800, kernel test robot wrote:
> Hi Mike,
>
> Thank you for the patch! Yet something to improve:
>
> [auto build test ERROR on vhost/linux-next]
> [also build test ERROR on next-20201007]
> [cannot apply to v5.9-rc8]
> [If your patch is applied to the wrong git tree, kindly drop us a note.
> And when submitting patch, we suggest to use '--base' as documented in
> https://git-scm.com/docs/git-format-patch]
>
> url: https://github.com/0day-ci/linux/commits/Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
> base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
> config: m68k-allyesconfig (attached as .config)
> compiler: m68k-linux-gcc (GCC) 9.3.0
> reproduce (this is a W=1 build):
> wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> chmod +x ~/bin/make.cross
> # https://github.com/0day-ci/linux/commit/c9bcafefcac3c32ade0ba533609d7b1ddc343c54
> git remote add linux-review https://github.com/0day-ci/linux
> git fetch --no-tags linux-review Mike-Christie/vhost-fix-scsi-cmd-handling-and-IOPs/20201008-045802
> git checkout c9bcafefcac3c32ade0ba533609d7b1ddc343c54
> # save the attached .config to linux build tree
> COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=m68k
>
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
>
> All errors (new ones prefixed by >>):
>
> >> m68k-linux-ld: drivers/vhost/scsi.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
> m68k-linux-ld: drivers/vhost/vsock.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
> m68k-linux-ld: drivers/vhost/vdpa.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
> m68k-linux-ld: drivers/vhost/vhost.o:(.bss+0x0): multiple definition of `vq'; drivers/vhost/net.o:(.bss+0x0): first defined here
Mike, what's going on with these failures?
Can you figure it out pls?
> ---
> 0-DAY CI Kernel Test Service, Intel Corporation
> https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH 15/16] vhost scsi: make completion per vq
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:55 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:55 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
In the last patches we are going to have a worker thread per IO vq.
This patch separates the scsi cmd completion code paths so we can
complete cmds based on their vq instead of having all cmds complete
on the same worker thread.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/scsi.c | 48 +++++++++++++++++++++++++-----------------------
1 file changed, 25 insertions(+), 23 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index b348e9c..4309f97 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -178,6 +178,7 @@ enum {
struct vhost_scsi_virtqueue {
struct vhost_virtqueue vq;
+ struct vhost_scsi *vs;
/*
* Reference counting for inflight reqs, used for flush operation. At
* each time, one reference tracks new commands submitted, while we
@@ -192,6 +193,9 @@ struct vhost_scsi_virtqueue {
struct vhost_scsi_cmd *scsi_cmds;
struct sbitmap scsi_tags;
int max_cmds;
+
+ struct vhost_work completion_work;
+ struct llist_head completion_list;
};
struct vhost_scsi {
@@ -202,9 +206,6 @@ struct vhost_scsi {
struct vhost_dev dev;
struct vhost_scsi_virtqueue vqs[VHOST_SCSI_MAX_VQ];
- struct vhost_work vs_completion_work; /* cmd completion work item */
- struct llist_head vs_completion_list; /* cmd completion queue */
-
struct vhost_work vs_event_work; /* evt injection work item */
struct llist_head vs_event_list; /* evt injection queue */
@@ -380,10 +381,11 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
} else {
struct vhost_scsi_cmd *cmd = container_of(se_cmd,
struct vhost_scsi_cmd, tvc_se_cmd);
- struct vhost_scsi *vs = cmd->tvc_vhost;
+ struct vhost_scsi_virtqueue *svq = container_of(cmd->tvc_vq,
+ struct vhost_scsi_virtqueue, vq);
- llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list);
- vhost_work_queue(&vs->dev, &vs->vs_completion_work);
+ llist_add(&cmd->tvc_completion_list, &svq->completion_list);
+ vhost_vq_work_queue(cmd->tvc_vq, &svq->completion_work);
}
}
@@ -545,18 +547,17 @@ static void vhost_scsi_evt_work(struct vhost_work *work)
*/
static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
{
- struct vhost_scsi *vs = container_of(work, struct vhost_scsi,
- vs_completion_work);
- DECLARE_BITMAP(signal, VHOST_SCSI_MAX_VQ);
+ struct vhost_scsi_virtqueue *svq = container_of(work,
+ struct vhost_scsi_virtqueue, completion_work);
struct virtio_scsi_cmd_resp v_rsp;
struct vhost_scsi_cmd *cmd, *t;
struct llist_node *llnode;
struct se_cmd *se_cmd;
struct iov_iter iov_iter;
- int ret, vq;
+ bool signal = false;
+ int ret;
- bitmap_zero(signal, VHOST_SCSI_MAX_VQ);
- llnode = llist_del_all(&vs->vs_completion_list);
+ llnode = llist_del_all(&svq->completion_list);
llist_for_each_entry_safe(cmd, t, llnode, tvc_completion_list) {
se_cmd = &cmd->tvc_se_cmd;
@@ -576,21 +577,16 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
cmd->tvc_in_iovs, sizeof(v_rsp));
ret = copy_to_iter(&v_rsp, sizeof(v_rsp), &iov_iter);
if (likely(ret == sizeof(v_rsp))) {
- struct vhost_scsi_virtqueue *q;
+ signal = true;
vhost_add_used(cmd->tvc_vq, cmd->tvc_vq_desc, 0);
- q = container_of(cmd->tvc_vq, struct vhost_scsi_virtqueue, vq);
- vq = q - vs->vqs;
- __set_bit(vq, signal);
} else
pr_err("Faulted on virtio_scsi_cmd_resp\n");
vhost_scsi_release_cmd_res(se_cmd);
}
- vq = -1;
- while ((vq = find_next_bit(signal, VHOST_SCSI_MAX_VQ, vq + 1))
- < VHOST_SCSI_MAX_VQ)
- vhost_signal(&vs->dev, &vs->vqs[vq].vq);
+ if (signal)
+ vhost_signal(&svq->vs->dev, &svq->vq);
}
static struct vhost_scsi_cmd *
@@ -1805,6 +1801,7 @@ static int vhost_scsi_set_features(struct vhost_scsi *vs, u64 features)
static int vhost_scsi_open(struct inode *inode, struct file *f)
{
+ struct vhost_scsi_virtqueue *svq;
struct vhost_scsi *vs;
struct vhost_virtqueue **vqs;
int r = -ENOMEM, i;
@@ -1820,7 +1817,6 @@ static int vhost_scsi_open(struct inode *inode, struct file *f)
if (!vqs)
goto err_vqs;
- vhost_work_init(&vs->vs_completion_work, vhost_scsi_complete_cmd_work);
vhost_work_init(&vs->vs_event_work, vhost_scsi_evt_work);
vs->vs_events_nr = 0;
@@ -1831,8 +1827,14 @@ static int vhost_scsi_open(struct inode *inode, struct file *f)
vs->vqs[VHOST_SCSI_VQ_CTL].vq.handle_kick = vhost_scsi_ctl_handle_kick;
vs->vqs[VHOST_SCSI_VQ_EVT].vq.handle_kick = vhost_scsi_evt_handle_kick;
for (i = VHOST_SCSI_VQ_IO; i < VHOST_SCSI_MAX_VQ; i++) {
- vqs[i] = &vs->vqs[i].vq;
- vs->vqs[i].vq.handle_kick = vhost_scsi_handle_kick;
+ svq = &vs->vqs[i];
+
+ vqs[i] = &svq->vq;
+ svq->vs = vs;
+ init_llist_head(&svq->completion_list);
+ vhost_work_init(&svq->completion_work,
+ vhost_scsi_complete_cmd_work);
+ svq->vq.handle_kick = vhost_scsi_handle_kick;
}
/*
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 15/16] vhost scsi: make completion per vq
@ 2020-10-07 20:55 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:55 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
In the last patches we are going to have a worker thread per IO vq.
This patch separates the scsi cmd completion code paths so we can
complete cmds based on their vq instead of having all cmds complete
on the same worker thread.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/scsi.c | 48 +++++++++++++++++++++++++-----------------------
1 file changed, 25 insertions(+), 23 deletions(-)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index b348e9c..4309f97 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -178,6 +178,7 @@ enum {
struct vhost_scsi_virtqueue {
struct vhost_virtqueue vq;
+ struct vhost_scsi *vs;
/*
* Reference counting for inflight reqs, used for flush operation. At
* each time, one reference tracks new commands submitted, while we
@@ -192,6 +193,9 @@ struct vhost_scsi_virtqueue {
struct vhost_scsi_cmd *scsi_cmds;
struct sbitmap scsi_tags;
int max_cmds;
+
+ struct vhost_work completion_work;
+ struct llist_head completion_list;
};
struct vhost_scsi {
@@ -202,9 +206,6 @@ struct vhost_scsi {
struct vhost_dev dev;
struct vhost_scsi_virtqueue vqs[VHOST_SCSI_MAX_VQ];
- struct vhost_work vs_completion_work; /* cmd completion work item */
- struct llist_head vs_completion_list; /* cmd completion queue */
-
struct vhost_work vs_event_work; /* evt injection work item */
struct llist_head vs_event_list; /* evt injection queue */
@@ -380,10 +381,11 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
} else {
struct vhost_scsi_cmd *cmd = container_of(se_cmd,
struct vhost_scsi_cmd, tvc_se_cmd);
- struct vhost_scsi *vs = cmd->tvc_vhost;
+ struct vhost_scsi_virtqueue *svq = container_of(cmd->tvc_vq,
+ struct vhost_scsi_virtqueue, vq);
- llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list);
- vhost_work_queue(&vs->dev, &vs->vs_completion_work);
+ llist_add(&cmd->tvc_completion_list, &svq->completion_list);
+ vhost_vq_work_queue(cmd->tvc_vq, &svq->completion_work);
}
}
@@ -545,18 +547,17 @@ static void vhost_scsi_evt_work(struct vhost_work *work)
*/
static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
{
- struct vhost_scsi *vs = container_of(work, struct vhost_scsi,
- vs_completion_work);
- DECLARE_BITMAP(signal, VHOST_SCSI_MAX_VQ);
+ struct vhost_scsi_virtqueue *svq = container_of(work,
+ struct vhost_scsi_virtqueue, completion_work);
struct virtio_scsi_cmd_resp v_rsp;
struct vhost_scsi_cmd *cmd, *t;
struct llist_node *llnode;
struct se_cmd *se_cmd;
struct iov_iter iov_iter;
- int ret, vq;
+ bool signal = false;
+ int ret;
- bitmap_zero(signal, VHOST_SCSI_MAX_VQ);
- llnode = llist_del_all(&vs->vs_completion_list);
+ llnode = llist_del_all(&svq->completion_list);
llist_for_each_entry_safe(cmd, t, llnode, tvc_completion_list) {
se_cmd = &cmd->tvc_se_cmd;
@@ -576,21 +577,16 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
cmd->tvc_in_iovs, sizeof(v_rsp));
ret = copy_to_iter(&v_rsp, sizeof(v_rsp), &iov_iter);
if (likely(ret = sizeof(v_rsp))) {
- struct vhost_scsi_virtqueue *q;
+ signal = true;
vhost_add_used(cmd->tvc_vq, cmd->tvc_vq_desc, 0);
- q = container_of(cmd->tvc_vq, struct vhost_scsi_virtqueue, vq);
- vq = q - vs->vqs;
- __set_bit(vq, signal);
} else
pr_err("Faulted on virtio_scsi_cmd_resp\n");
vhost_scsi_release_cmd_res(se_cmd);
}
- vq = -1;
- while ((vq = find_next_bit(signal, VHOST_SCSI_MAX_VQ, vq + 1))
- < VHOST_SCSI_MAX_VQ)
- vhost_signal(&vs->dev, &vs->vqs[vq].vq);
+ if (signal)
+ vhost_signal(&svq->vs->dev, &svq->vq);
}
static struct vhost_scsi_cmd *
@@ -1805,6 +1801,7 @@ static int vhost_scsi_set_features(struct vhost_scsi *vs, u64 features)
static int vhost_scsi_open(struct inode *inode, struct file *f)
{
+ struct vhost_scsi_virtqueue *svq;
struct vhost_scsi *vs;
struct vhost_virtqueue **vqs;
int r = -ENOMEM, i;
@@ -1820,7 +1817,6 @@ static int vhost_scsi_open(struct inode *inode, struct file *f)
if (!vqs)
goto err_vqs;
- vhost_work_init(&vs->vs_completion_work, vhost_scsi_complete_cmd_work);
vhost_work_init(&vs->vs_event_work, vhost_scsi_evt_work);
vs->vs_events_nr = 0;
@@ -1831,8 +1827,14 @@ static int vhost_scsi_open(struct inode *inode, struct file *f)
vs->vqs[VHOST_SCSI_VQ_CTL].vq.handle_kick = vhost_scsi_ctl_handle_kick;
vs->vqs[VHOST_SCSI_VQ_EVT].vq.handle_kick = vhost_scsi_evt_handle_kick;
for (i = VHOST_SCSI_VQ_IO; i < VHOST_SCSI_MAX_VQ; i++) {
- vqs[i] = &vs->vqs[i].vq;
- vs->vqs[i].vq.handle_kick = vhost_scsi_handle_kick;
+ svq = &vs->vqs[i];
+
+ vqs[i] = &svq->vq;
+ svq->vs = vs;
+ init_llist_head(&svq->completion_list);
+ vhost_work_init(&svq->completion_work,
+ vhost_scsi_complete_cmd_work);
+ svq->vq.handle_kick = vhost_scsi_handle_kick;
}
/*
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 16/16] vhost scsi: multiple worker support
2020-10-07 20:54 ` Mike Christie
@ 2020-10-07 20:55 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:55 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
Create a vhost_worker per IO vq. When using a more than 2 vqs and/or
multiple LUNs per vhost-scsi dev, we hit a bottleneck with the single
worker. The problem is that we want to start and complete all vqs and
all LUNs from the same thread. Combine with the previous patches that
allow us to add more than 2 vqs, we see a IOPs workloads (like 50/50
randrw 4K IOs) go from 150K to 400K where the native device is 500K.
For the lio rd_mcp backend, we see IOPs go for from 400K to 600K.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/scsi.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 4309f97..e5f73c1 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1624,6 +1624,22 @@ static int vhost_scsi_setup_vq_cmds(struct vhost_virtqueue *vq, int max_cmds)
memcpy(vs->vs_vhost_wwpn, t->vhost_wwpn,
sizeof(vs->vs_vhost_wwpn));
+ /*
+ * For compat, have the evt and ctl vqs share worker0 with
+ * the first IO vq like is setup as default already. Any
+ * addition vqs will get their own worker.
+ *
+ * Note: if we fail later, then the vhost_dev_cleanup call on
+ * release() will clean up all the workers.
+ */
+ ret = vhost_workers_create(&vs->dev,
+ vs->dev.nvqs - VHOST_SCSI_VQ_IO);
+ if (ret) {
+ pr_err("Could not create vhost-scsi workers. Error %d.",
+ ret);
+ goto undepend;
+ }
+
for (i = VHOST_SCSI_VQ_IO; i < VHOST_SCSI_MAX_VQ; i++) {
vq = &vs->vqs[i].vq;
if (!vq->initialized)
@@ -1631,6 +1647,7 @@ static int vhost_scsi_setup_vq_cmds(struct vhost_virtqueue *vq, int max_cmds)
if (vhost_scsi_setup_vq_cmds(vq, vq->num))
goto destroy_vq_cmds;
+ vhost_vq_set_worker(vq, i - VHOST_SCSI_VQ_IO);
}
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) {
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 16/16] vhost scsi: multiple worker support
@ 2020-10-07 20:55 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-07 20:55 UTC (permalink / raw)
To: martin.petersen, linux-scsi, target-devel, mst, jasowang,
pbonzini, stefanha, virtualization
Create a vhost_worker per IO vq. When using a more than 2 vqs and/or
multiple LUNs per vhost-scsi dev, we hit a bottleneck with the single
worker. The problem is that we want to start and complete all vqs and
all LUNs from the same thread. Combine with the previous patches that
allow us to add more than 2 vqs, we see a IOPs workloads (like 50/50
randrw 4K IOs) go from 150K to 400K where the native device is 500K.
For the lio rd_mcp backend, we see IOPs go for from 400K to 600K.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
---
drivers/vhost/scsi.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 4309f97..e5f73c1 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1624,6 +1624,22 @@ static int vhost_scsi_setup_vq_cmds(struct vhost_virtqueue *vq, int max_cmds)
memcpy(vs->vs_vhost_wwpn, t->vhost_wwpn,
sizeof(vs->vs_vhost_wwpn));
+ /*
+ * For compat, have the evt and ctl vqs share worker0 with
+ * the first IO vq like is setup as default already. Any
+ * addition vqs will get their own worker.
+ *
+ * Note: if we fail later, then the vhost_dev_cleanup call on
+ * release() will clean up all the workers.
+ */
+ ret = vhost_workers_create(&vs->dev,
+ vs->dev.nvqs - VHOST_SCSI_VQ_IO);
+ if (ret) {
+ pr_err("Could not create vhost-scsi workers. Error %d.",
+ ret);
+ goto undepend;
+ }
+
for (i = VHOST_SCSI_VQ_IO; i < VHOST_SCSI_MAX_VQ; i++) {
vq = &vs->vqs[i].vq;
if (!vq->initialized)
@@ -1631,6 +1647,7 @@ static int vhost_scsi_setup_vq_cmds(struct vhost_virtqueue *vq, int max_cmds)
if (vhost_scsi_setup_vq_cmds(vq, vq->num))
goto destroy_vq_cmds;
+ vhost_vq_set_worker(vq, i - VHOST_SCSI_VQ_IO);
}
for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) {
--
1.8.3.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* Re: [PATCH 00/16 V2] vhost: fix scsi cmd handling and IOPs
2020-10-07 20:54 ` Mike Christie
(?)
@ 2020-10-23 15:46 ` Michael S. Tsirkin
-1 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-23 15:46 UTC (permalink / raw)
To: Mike Christie
Cc: martin.petersen, linux-scsi, target-devel, jasowang, pbonzini,
stefanha, virtualization
On Wed, Oct 07, 2020 at 03:54:45PM -0500, Mike Christie wrote:
> The following patches were made over Michael's vhost branch here:
> https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git/log/?h=vhost
>
> The patches also apply to Linus's or Martin's trees if you apply
> https://patchwork.kernel.org/patch/11790681/
> which was merged into mst's tree already.
>
> The following patches are a follow up to this post:
> https://patchwork.kernel.org/cover/11790763/
> which originally was fixing how vhost-scsi handled cmds so we would
> not get IO errors when sending more than 256 cmds.
>
> In that patchset I needed to detect if a vq was in use and for this
> patch:
> https://patchwork.kernel.org/patch/11790685/
> it was suggested to add support for VHOST_RING_ENABLE. While doing
> that though I hit a couple problems:
>
> 1. The patches moved how vhost-scsi allocated cmds from per lio
> session to per vhost vq. To support both VHOST_RING_ENABLE and
> where userspace didn't support it, I would have to keep around the
> old per session/device cmd allocator/completion and then also maintain
> the new code. Or, I would still have to use this patch
> patchwork.kernel.org/cover/11790763/ for the compat case so there
> adding the new ioctl would not help much.
>
> 2. For vhost-scsi I also wanted to prevent where we allocate iovecs
> for 128 vqs even though we normally use a couple. To do this, I needed
> something similar to #1, but the problem is that the VHOST_RING_ENABLE
> call would come too late.
>
> To try and balance #1 and #2, these patches just allow vhost-scsi
> to setup a vq when userspace starts to config it. This allows the
> driver to only fully setup (we still waste some memory to support older
> setups but do not have to preallocate everything like before) what
> is used plus I do not need to maintain 2 code paths.
>
> Note that in this posting I am also including additional patches
> that create multiple vhost worker threads, because I wanted to see
> if people felt that maybe to support that and for this enablement
> issue we want a completely a new ioctl.
>
>
> V2:
> - fix use before set cpu var errors
> - drop vhost_vq_is_setup
> - include patches to do a worker thread per scsi IO vq
Stefan, Paolo, Jason any input?
--
MST
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 00/16 V2] vhost: fix scsi cmd handling and IOPs
@ 2020-10-23 15:46 ` Michael S. Tsirkin
0 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-23 15:46 UTC (permalink / raw)
To: Mike Christie
Cc: martin.petersen, linux-scsi, virtualization, target-devel,
stefanha, pbonzini
On Wed, Oct 07, 2020 at 03:54:45PM -0500, Mike Christie wrote:
> The following patches were made over Michael's vhost branch here:
> https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git/log/?h=vhost
>
> The patches also apply to Linus's or Martin's trees if you apply
> https://patchwork.kernel.org/patch/11790681/
> which was merged into mst's tree already.
>
> The following patches are a follow up to this post:
> https://patchwork.kernel.org/cover/11790763/
> which originally was fixing how vhost-scsi handled cmds so we would
> not get IO errors when sending more than 256 cmds.
>
> In that patchset I needed to detect if a vq was in use and for this
> patch:
> https://patchwork.kernel.org/patch/11790685/
> it was suggested to add support for VHOST_RING_ENABLE. While doing
> that though I hit a couple problems:
>
> 1. The patches moved how vhost-scsi allocated cmds from per lio
> session to per vhost vq. To support both VHOST_RING_ENABLE and
> where userspace didn't support it, I would have to keep around the
> old per session/device cmd allocator/completion and then also maintain
> the new code. Or, I would still have to use this patch
> patchwork.kernel.org/cover/11790763/ for the compat case so there
> adding the new ioctl would not help much.
>
> 2. For vhost-scsi I also wanted to prevent where we allocate iovecs
> for 128 vqs even though we normally use a couple. To do this, I needed
> something similar to #1, but the problem is that the VHOST_RING_ENABLE
> call would come too late.
>
> To try and balance #1 and #2, these patches just allow vhost-scsi
> to setup a vq when userspace starts to config it. This allows the
> driver to only fully setup (we still waste some memory to support older
> setups but do not have to preallocate everything like before) what
> is used plus I do not need to maintain 2 code paths.
>
> Note that in this posting I am also including additional patches
> that create multiple vhost worker threads, because I wanted to see
> if people felt that maybe to support that and for this enablement
> issue we want a completely a new ioctl.
>
>
> V2:
> - fix use before set cpu var errors
> - drop vhost_vq_is_setup
> - include patches to do a worker thread per scsi IO vq
Stefan, Paolo, Jason any input?
--
MST
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 00/16 V2] vhost: fix scsi cmd handling and IOPs
@ 2020-10-23 15:46 ` Michael S. Tsirkin
0 siblings, 0 replies; 69+ messages in thread
From: Michael S. Tsirkin @ 2020-10-23 15:46 UTC (permalink / raw)
To: Mike Christie
Cc: martin.petersen, linux-scsi, target-devel, jasowang, pbonzini,
stefanha, virtualization
On Wed, Oct 07, 2020 at 03:54:45PM -0500, Mike Christie wrote:
> The following patches were made over Michael's vhost branch here:
> https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git/log/?h=vhost
>
> The patches also apply to Linus's or Martin's trees if you apply
> https://patchwork.kernel.org/patch/11790681/
> which was merged into mst's tree already.
>
> The following patches are a follow up to this post:
> https://patchwork.kernel.org/cover/11790763/
> which originally was fixing how vhost-scsi handled cmds so we would
> not get IO errors when sending more than 256 cmds.
>
> In that patchset I needed to detect if a vq was in use and for this
> patch:
> https://patchwork.kernel.org/patch/11790685/
> it was suggested to add support for VHOST_RING_ENABLE. While doing
> that though I hit a couple problems:
>
> 1. The patches moved how vhost-scsi allocated cmds from per lio
> session to per vhost vq. To support both VHOST_RING_ENABLE and
> where userspace didn't support it, I would have to keep around the
> old per session/device cmd allocator/completion and then also maintain
> the new code. Or, I would still have to use this patch
> patchwork.kernel.org/cover/11790763/ for the compat case so there
> adding the new ioctl would not help much.
>
> 2. For vhost-scsi I also wanted to prevent where we allocate iovecs
> for 128 vqs even though we normally use a couple. To do this, I needed
> something similar to #1, but the problem is that the VHOST_RING_ENABLE
> call would come too late.
>
> To try and balance #1 and #2, these patches just allow vhost-scsi
> to setup a vq when userspace starts to config it. This allows the
> driver to only fully setup (we still waste some memory to support older
> setups but do not have to preallocate everything like before) what
> is used plus I do not need to maintain 2 code paths.
>
> Note that in this posting I am also including additional patches
> that create multiple vhost worker threads, because I wanted to see
> if people felt that maybe to support that and for this enablement
> issue we want a completely a new ioctl.
>
>
> V2:
> - fix use before set cpu var errors
> - drop vhost_vq_is_setup
> - include patches to do a worker thread per scsi IO vq
Stefan, Paolo, Jason any input?
--
MST
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 00/16 V2] vhost: fix scsi cmd handling and IOPs
2020-10-23 15:46 ` Michael S. Tsirkin
@ 2020-10-23 16:22 ` Mike Christie
-1 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-23 16:22 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: martin.petersen, linux-scsi, target-devel, jasowang, pbonzini,
stefanha, virtualization
On 10/23/20 10:46 AM, Michael S. Tsirkin wrote:
> On Wed, Oct 07, 2020 at 03:54:45PM -0500, Mike Christie wrote:
>> The following patches were made over Michael's vhost branch here:
>> https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git/log/?h=vhost__;!!GqivPVa7Brio!IVNw3V-uPEJyaYcHGpZrPo_0vnAuPXchguJJZG5qCapOYzR8bOwuFyTZf49rMcokFOMG$
>>
>> The patches also apply to Linus's or Martin's trees if you apply
>> https://urldefense.com/v3/__https://patchwork.kernel.org/patch/11790681/__;!!GqivPVa7Brio!IVNw3V-uPEJyaYcHGpZrPo_0vnAuPXchguJJZG5qCapOYzR8bOwuFyTZf49rMfl3id0D$
>> which was merged into mst's tree already.
>>
>> The following patches are a follow up to this post:
>> https://urldefense.com/v3/__https://patchwork.kernel.org/cover/11790763/__;!!GqivPVa7Brio!IVNw3V-uPEJyaYcHGpZrPo_0vnAuPXchguJJZG5qCapOYzR8bOwuFyTZf49rMWyfBwDA$
>> which originally was fixing how vhost-scsi handled cmds so we would
>> not get IO errors when sending more than 256 cmds.
>>
>> In that patchset I needed to detect if a vq was in use and for this
>> patch:
>> https://urldefense.com/v3/__https://patchwork.kernel.org/patch/11790685/__;!!GqivPVa7Brio!IVNw3V-uPEJyaYcHGpZrPo_0vnAuPXchguJJZG5qCapOYzR8bOwuFyTZf49rMWWcMjJi$
>> it was suggested to add support for VHOST_RING_ENABLE. While doing
>> that though I hit a couple problems:
>>
>> 1. The patches moved how vhost-scsi allocated cmds from per lio
>> session to per vhost vq. To support both VHOST_RING_ENABLE and
>> where userspace didn't support it, I would have to keep around the
>> old per session/device cmd allocator/completion and then also maintain
>> the new code. Or, I would still have to use this patch
>> patchwork.kernel.org/cover/11790763/ for the compat case so there
>> adding the new ioctl would not help much.
>>
>> 2. For vhost-scsi I also wanted to prevent where we allocate iovecs
>> for 128 vqs even though we normally use a couple. To do this, I needed
>> something similar to #1, but the problem is that the VHOST_RING_ENABLE
>> call would come too late.
>>
>> To try and balance #1 and #2, these patches just allow vhost-scsi
>> to setup a vq when userspace starts to config it. This allows the
>> driver to only fully setup (we still waste some memory to support older
>> setups but do not have to preallocate everything like before) what
>> is used plus I do not need to maintain 2 code paths.
>>
>> Note that in this posting I am also including additional patches
>> that create multiple vhost worker threads, because I wanted to see
>> if people felt that maybe to support that and for this enablement
>> issue we want a completely a new ioctl.
>>
>>
>> V2:
>> - fix use before set cpu var errors
>> - drop vhost_vq_is_setup
>> - include patches to do a worker thread per scsi IO vq
>
> Stefan, Paolo, Jason any input?
>
Just a FYI there is a updated version of this patchset here:
https://patchwork.kernel.org/project/target-devel/list/?series=368487
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 00/16 V2] vhost: fix scsi cmd handling and IOPs
@ 2020-10-23 16:22 ` Mike Christie
0 siblings, 0 replies; 69+ messages in thread
From: Mike Christie @ 2020-10-23 16:22 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: martin.petersen, linux-scsi, target-devel, jasowang, pbonzini,
stefanha, virtualization
On 10/23/20 10:46 AM, Michael S. Tsirkin wrote:
> On Wed, Oct 07, 2020 at 03:54:45PM -0500, Mike Christie wrote:
>> The following patches were made over Michael's vhost branch here:
>> https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git/log/?h=vhost__;!!GqivPVa7Brio!IVNw3V-uPEJyaYcHGpZrPo_0vnAuPXchguJJZG5qCapOYzR8bOwuFyTZf49rMcokFOMG$
>>
>> The patches also apply to Linus's or Martin's trees if you apply
>> https://urldefense.com/v3/__https://patchwork.kernel.org/patch/11790681/__;!!GqivPVa7Brio!IVNw3V-uPEJyaYcHGpZrPo_0vnAuPXchguJJZG5qCapOYzR8bOwuFyTZf49rMfl3id0D$
>> which was merged into mst's tree already.
>>
>> The following patches are a follow up to this post:
>> https://urldefense.com/v3/__https://patchwork.kernel.org/cover/11790763/__;!!GqivPVa7Brio!IVNw3V-uPEJyaYcHGpZrPo_0vnAuPXchguJJZG5qCapOYzR8bOwuFyTZf49rMWyfBwDA$
>> which originally was fixing how vhost-scsi handled cmds so we would
>> not get IO errors when sending more than 256 cmds.
>>
>> In that patchset I needed to detect if a vq was in use and for this
>> patch:
>> https://urldefense.com/v3/__https://patchwork.kernel.org/patch/11790685/__;!!GqivPVa7Brio!IVNw3V-uPEJyaYcHGpZrPo_0vnAuPXchguJJZG5qCapOYzR8bOwuFyTZf49rMWWcMjJi$
>> it was suggested to add support for VHOST_RING_ENABLE. While doing
>> that though I hit a couple problems:
>>
>> 1. The patches moved how vhost-scsi allocated cmds from per lio
>> session to per vhost vq. To support both VHOST_RING_ENABLE and
>> where userspace didn't support it, I would have to keep around the
>> old per session/device cmd allocator/completion and then also maintain
>> the new code. Or, I would still have to use this patch
>> patchwork.kernel.org/cover/11790763/ for the compat case so there
>> adding the new ioctl would not help much.
>>
>> 2. For vhost-scsi I also wanted to prevent where we allocate iovecs
>> for 128 vqs even though we normally use a couple. To do this, I needed
>> something similar to #1, but the problem is that the VHOST_RING_ENABLE
>> call would come too late.
>>
>> To try and balance #1 and #2, these patches just allow vhost-scsi
>> to setup a vq when userspace starts to config it. This allows the
>> driver to only fully setup (we still waste some memory to support older
>> setups but do not have to preallocate everything like before) what
>> is used plus I do not need to maintain 2 code paths.
>>
>> Note that in this posting I am also including additional patches
>> that create multiple vhost worker threads, because I wanted to see
>> if people felt that maybe to support that and for this enablement
>> issue we want a completely a new ioctl.
>>
>>
>> V2:
>> - fix use before set cpu var errors
>> - drop vhost_vq_is_setup
>> - include patches to do a worker thread per scsi IO vq
>
> Stefan, Paolo, Jason any input?
>
Just a FYI there is a updated version of this patchset here:
https://patchwork.kernel.org/project/target-devel/list/?series68487
^ permalink raw reply [flat|nested] 69+ messages in thread