* [PATCH v2 1/2] virtio: Reduce BUG if total_sg > virtqueue size to WARN.
2017-08-10 16:56 [PATCH v2 0/2] virtio_scsi: Set can_queue based on size of virtqueue Richard W.M. Jones
@ 2017-08-10 16:56 ` Richard W.M. Jones
2017-08-10 16:56 ` Richard W.M. Jones
` (4 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Richard W.M. Jones @ 2017-08-10 16:56 UTC (permalink / raw)
To: jejb
Cc: martin.petersen, mst, jasowang, linux-scsi, linux-kernel,
virtualization, hch, pbonzini
If using indirect descriptors, you can make the total_sg as large as
you want. If not, BUG is too serious because the function later
returns -ENOSPC.
Thanks Paolo Bonzini, Christoph Hellwig.
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
drivers/virtio/virtio_ring.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 5e1b548828e6..27cbc1eab868 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -296,7 +296,6 @@ static inline int virtqueue_add(struct virtqueue *_vq,
}
#endif
- BUG_ON(total_sg > vq->vring.num);
BUG_ON(total_sg == 0);
head = vq->free_head;
@@ -305,8 +304,10 @@ static inline int virtqueue_add(struct virtqueue *_vq,
* buffers, then go indirect. FIXME: tune this threshold */
if (vq->indirect && total_sg > 1 && vq->vq.num_free)
desc = alloc_indirect(_vq, total_sg, gfp);
- else
+ else {
desc = NULL;
+ WARN_ON_ONCE(total_sg > vq->vring.num && !vq->indirect);
+ }
if (desc) {
/* Use a single buffer which doesn't continue */
--
2.13.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v2 1/2] virtio: Reduce BUG if total_sg > virtqueue size to WARN.
2017-08-10 16:56 [PATCH v2 0/2] virtio_scsi: Set can_queue based on size of virtqueue Richard W.M. Jones
2017-08-10 16:56 ` [PATCH v2 1/2] virtio: Reduce BUG if total_sg > virtqueue size to WARN Richard W.M. Jones
@ 2017-08-10 16:56 ` Richard W.M. Jones
2017-08-10 16:56 ` [PATCH v2 2/2] virtio: virtio_scsi: Set can_queue to the length of the virtqueue Richard W.M. Jones
` (3 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Richard W.M. Jones @ 2017-08-10 16:56 UTC (permalink / raw)
To: jejb
Cc: martin.petersen, linux-scsi, mst, linux-kernel, virtualization,
pbonzini, hch
If using indirect descriptors, you can make the total_sg as large as
you want. If not, BUG is too serious because the function later
returns -ENOSPC.
Thanks Paolo Bonzini, Christoph Hellwig.
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
drivers/virtio/virtio_ring.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 5e1b548828e6..27cbc1eab868 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -296,7 +296,6 @@ static inline int virtqueue_add(struct virtqueue *_vq,
}
#endif
- BUG_ON(total_sg > vq->vring.num);
BUG_ON(total_sg == 0);
head = vq->free_head;
@@ -305,8 +304,10 @@ static inline int virtqueue_add(struct virtqueue *_vq,
* buffers, then go indirect. FIXME: tune this threshold */
if (vq->indirect && total_sg > 1 && vq->vq.num_free)
desc = alloc_indirect(_vq, total_sg, gfp);
- else
+ else {
desc = NULL;
+ WARN_ON_ONCE(total_sg > vq->vring.num && !vq->indirect);
+ }
if (desc) {
/* Use a single buffer which doesn't continue */
--
2.13.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v2 2/2] virtio: virtio_scsi: Set can_queue to the length of the virtqueue.
2017-08-10 16:56 [PATCH v2 0/2] virtio_scsi: Set can_queue based on size of virtqueue Richard W.M. Jones
2017-08-10 16:56 ` [PATCH v2 1/2] virtio: Reduce BUG if total_sg > virtqueue size to WARN Richard W.M. Jones
2017-08-10 16:56 ` Richard W.M. Jones
@ 2017-08-10 16:56 ` Richard W.M. Jones
2017-08-10 16:56 ` Richard W.M. Jones
` (2 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Richard W.M. Jones @ 2017-08-10 16:56 UTC (permalink / raw)
To: jejb
Cc: martin.petersen, linux-scsi, mst, linux-kernel, virtualization,
pbonzini, hch
Since switching to blk-mq as the default in commit 5c279bd9e406
("scsi: default to scsi-mq"), virtio-scsi LUNs consume about 10x as
much kernel memory.
qemu currently allocates a fixed 128 entry virtqueue. can_queue
currently is set to 1024. But with indirect descriptors, each command
in the queue takes 1 virtqueue entry, so the number of commands which
can be queued is equal to the length of the virtqueue.
Note I intend to send a patch to qemu to allow the virtqueue size to
be configured from the qemu command line.
Thanks Paolo Bonzini, Christoph Hellwig.
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
---
drivers/scsi/virtio_scsi.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
index 9be211d68b15..7c28e8d4955a 100644
--- a/drivers/scsi/virtio_scsi.c
+++ b/drivers/scsi/virtio_scsi.c
@@ -818,7 +818,6 @@ static struct scsi_host_template virtscsi_host_template_single = {
.eh_timed_out = virtscsi_eh_timed_out,
.slave_alloc = virtscsi_device_alloc,
- .can_queue = 1024,
.dma_boundary = UINT_MAX,
.use_clustering = ENABLE_CLUSTERING,
.target_alloc = virtscsi_target_alloc,
@@ -839,7 +838,6 @@ static struct scsi_host_template virtscsi_host_template_multi = {
.eh_timed_out = virtscsi_eh_timed_out,
.slave_alloc = virtscsi_device_alloc,
- .can_queue = 1024,
.dma_boundary = UINT_MAX,
.use_clustering = ENABLE_CLUSTERING,
.target_alloc = virtscsi_target_alloc,
@@ -972,6 +970,8 @@ static int virtscsi_probe(struct virtio_device *vdev)
if (err)
goto virtscsi_init_failed;
+ shost->can_queue = virtqueue_get_vring_size(vscsi->req_vqs[0].vq);
+
cmd_per_lun = virtscsi_config_get(vdev, cmd_per_lun) ?: 1;
shost->cmd_per_lun = min_t(u32, cmd_per_lun, shost->can_queue);
shost->max_sectors = virtscsi_config_get(vdev, max_sectors) ?: 0xFFFF;
--
2.13.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v2 2/2] virtio: virtio_scsi: Set can_queue to the length of the virtqueue.
2017-08-10 16:56 [PATCH v2 0/2] virtio_scsi: Set can_queue based on size of virtqueue Richard W.M. Jones
` (2 preceding siblings ...)
2017-08-10 16:56 ` [PATCH v2 2/2] virtio: virtio_scsi: Set can_queue to the length of the virtqueue Richard W.M. Jones
@ 2017-08-10 16:56 ` Richard W.M. Jones
2017-08-10 16:58 ` Paolo Bonzini
2017-08-11 0:24 ` [PATCH v2 0/2] virtio_scsi: Set can_queue based on size of virtqueue Martin K. Petersen
2017-08-11 0:24 ` Martin K. Petersen
5 siblings, 1 reply; 10+ messages in thread
From: Richard W.M. Jones @ 2017-08-10 16:56 UTC (permalink / raw)
To: jejb
Cc: martin.petersen, mst, jasowang, linux-scsi, linux-kernel,
virtualization, hch, pbonzini
Since switching to blk-mq as the default in commit 5c279bd9e406
("scsi: default to scsi-mq"), virtio-scsi LUNs consume about 10x as
much kernel memory.
qemu currently allocates a fixed 128 entry virtqueue. can_queue
currently is set to 1024. But with indirect descriptors, each command
in the queue takes 1 virtqueue entry, so the number of commands which
can be queued is equal to the length of the virtqueue.
Note I intend to send a patch to qemu to allow the virtqueue size to
be configured from the qemu command line.
Thanks Paolo Bonzini, Christoph Hellwig.
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
---
drivers/scsi/virtio_scsi.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
index 9be211d68b15..7c28e8d4955a 100644
--- a/drivers/scsi/virtio_scsi.c
+++ b/drivers/scsi/virtio_scsi.c
@@ -818,7 +818,6 @@ static struct scsi_host_template virtscsi_host_template_single = {
.eh_timed_out = virtscsi_eh_timed_out,
.slave_alloc = virtscsi_device_alloc,
- .can_queue = 1024,
.dma_boundary = UINT_MAX,
.use_clustering = ENABLE_CLUSTERING,
.target_alloc = virtscsi_target_alloc,
@@ -839,7 +838,6 @@ static struct scsi_host_template virtscsi_host_template_multi = {
.eh_timed_out = virtscsi_eh_timed_out,
.slave_alloc = virtscsi_device_alloc,
- .can_queue = 1024,
.dma_boundary = UINT_MAX,
.use_clustering = ENABLE_CLUSTERING,
.target_alloc = virtscsi_target_alloc,
@@ -972,6 +970,8 @@ static int virtscsi_probe(struct virtio_device *vdev)
if (err)
goto virtscsi_init_failed;
+ shost->can_queue = virtqueue_get_vring_size(vscsi->req_vqs[0].vq);
+
cmd_per_lun = virtscsi_config_get(vdev, cmd_per_lun) ?: 1;
shost->cmd_per_lun = min_t(u32, cmd_per_lun, shost->can_queue);
shost->max_sectors = virtscsi_config_get(vdev, max_sectors) ?: 0xFFFF;
--
2.13.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH v2 2/2] virtio: virtio_scsi: Set can_queue to the length of the virtqueue.
2017-08-10 16:56 ` Richard W.M. Jones
@ 2017-08-10 16:58 ` Paolo Bonzini
0 siblings, 0 replies; 10+ messages in thread
From: Paolo Bonzini @ 2017-08-10 16:58 UTC (permalink / raw)
To: Richard W.M. Jones, jejb
Cc: martin.petersen, linux-scsi, mst, linux-kernel, virtualization,
hch
On 10/08/2017 18:56, Richard W.M. Jones wrote:
> Since switching to blk-mq as the default in commit 5c279bd9e406
> ("scsi: default to scsi-mq"), virtio-scsi LUNs consume about 10x as
> much kernel memory.
>
> qemu currently allocates a fixed 128 entry virtqueue. can_queue
> currently is set to 1024. But with indirect descriptors, each command
> in the queue takes 1 virtqueue entry, so the number of commands which
> can be queued is equal to the length of the virtqueue.
>
> Note I intend to send a patch to qemu to allow the virtqueue size to
> be configured from the qemu command line.
>
> Thanks Paolo Bonzini, Christoph Hellwig.
>
> Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
> ---
> drivers/scsi/virtio_scsi.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
> index 9be211d68b15..7c28e8d4955a 100644
> --- a/drivers/scsi/virtio_scsi.c
> +++ b/drivers/scsi/virtio_scsi.c
> @@ -818,7 +818,6 @@ static struct scsi_host_template virtscsi_host_template_single = {
> .eh_timed_out = virtscsi_eh_timed_out,
> .slave_alloc = virtscsi_device_alloc,
>
> - .can_queue = 1024,
> .dma_boundary = UINT_MAX,
> .use_clustering = ENABLE_CLUSTERING,
> .target_alloc = virtscsi_target_alloc,
> @@ -839,7 +838,6 @@ static struct scsi_host_template virtscsi_host_template_multi = {
> .eh_timed_out = virtscsi_eh_timed_out,
> .slave_alloc = virtscsi_device_alloc,
>
> - .can_queue = 1024,
> .dma_boundary = UINT_MAX,
> .use_clustering = ENABLE_CLUSTERING,
> .target_alloc = virtscsi_target_alloc,
> @@ -972,6 +970,8 @@ static int virtscsi_probe(struct virtio_device *vdev)
> if (err)
> goto virtscsi_init_failed;
>
> + shost->can_queue = virtqueue_get_vring_size(vscsi->req_vqs[0].vq);
> +
> cmd_per_lun = virtscsi_config_get(vdev, cmd_per_lun) ?: 1;
> shost->cmd_per_lun = min_t(u32, cmd_per_lun, shost->can_queue);
> shost->max_sectors = virtscsi_config_get(vdev, max_sectors) ?: 0xFFFF;
>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCH v2 2/2] virtio: virtio_scsi: Set can_queue to the length of the virtqueue.
@ 2017-08-10 16:58 ` Paolo Bonzini
0 siblings, 0 replies; 10+ messages in thread
From: Paolo Bonzini @ 2017-08-10 16:58 UTC (permalink / raw)
To: Richard W.M. Jones, jejb
Cc: martin.petersen, mst, jasowang, linux-scsi, linux-kernel,
virtualization, hch
On 10/08/2017 18:56, Richard W.M. Jones wrote:
> Since switching to blk-mq as the default in commit 5c279bd9e406
> ("scsi: default to scsi-mq"), virtio-scsi LUNs consume about 10x as
> much kernel memory.
>
> qemu currently allocates a fixed 128 entry virtqueue. can_queue
> currently is set to 1024. But with indirect descriptors, each command
> in the queue takes 1 virtqueue entry, so the number of commands which
> can be queued is equal to the length of the virtqueue.
>
> Note I intend to send a patch to qemu to allow the virtqueue size to
> be configured from the qemu command line.
>
> Thanks Paolo Bonzini, Christoph Hellwig.
>
> Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
> ---
> drivers/scsi/virtio_scsi.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
> index 9be211d68b15..7c28e8d4955a 100644
> --- a/drivers/scsi/virtio_scsi.c
> +++ b/drivers/scsi/virtio_scsi.c
> @@ -818,7 +818,6 @@ static struct scsi_host_template virtscsi_host_template_single = {
> .eh_timed_out = virtscsi_eh_timed_out,
> .slave_alloc = virtscsi_device_alloc,
>
> - .can_queue = 1024,
> .dma_boundary = UINT_MAX,
> .use_clustering = ENABLE_CLUSTERING,
> .target_alloc = virtscsi_target_alloc,
> @@ -839,7 +838,6 @@ static struct scsi_host_template virtscsi_host_template_multi = {
> .eh_timed_out = virtscsi_eh_timed_out,
> .slave_alloc = virtscsi_device_alloc,
>
> - .can_queue = 1024,
> .dma_boundary = UINT_MAX,
> .use_clustering = ENABLE_CLUSTERING,
> .target_alloc = virtscsi_target_alloc,
> @@ -972,6 +970,8 @@ static int virtscsi_probe(struct virtio_device *vdev)
> if (err)
> goto virtscsi_init_failed;
>
> + shost->can_queue = virtqueue_get_vring_size(vscsi->req_vqs[0].vq);
> +
> cmd_per_lun = virtscsi_config_get(vdev, cmd_per_lun) ?: 1;
> shost->cmd_per_lun = min_t(u32, cmd_per_lun, shost->can_queue);
> shost->max_sectors = virtscsi_config_get(vdev, max_sectors) ?: 0xFFFF;
>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2 0/2] virtio_scsi: Set can_queue based on size of virtqueue.
2017-08-10 16:56 [PATCH v2 0/2] virtio_scsi: Set can_queue based on size of virtqueue Richard W.M. Jones
` (3 preceding siblings ...)
2017-08-10 16:56 ` Richard W.M. Jones
@ 2017-08-11 0:24 ` Martin K. Petersen
2017-08-11 0:24 ` Martin K. Petersen
5 siblings, 0 replies; 10+ messages in thread
From: Martin K. Petersen @ 2017-08-11 0:24 UTC (permalink / raw)
To: Richard W.M. Jones
Cc: jejb, martin.petersen, linux-scsi, mst, linux-kernel,
virtualization, pbonzini, hch
Richard,
> v1 was here:
>
> https://lkml.org/lkml/2017/8/10/689
>
> v1 -> v2:
>
> Remove .can_queue field from the templates.
Applied to 4.14/scsi-queue. Thank you!
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCH v2 0/2] virtio_scsi: Set can_queue based on size of virtqueue.
2017-08-10 16:56 [PATCH v2 0/2] virtio_scsi: Set can_queue based on size of virtqueue Richard W.M. Jones
` (4 preceding siblings ...)
2017-08-11 0:24 ` [PATCH v2 0/2] virtio_scsi: Set can_queue based on size of virtqueue Martin K. Petersen
@ 2017-08-11 0:24 ` Martin K. Petersen
5 siblings, 0 replies; 10+ messages in thread
From: Martin K. Petersen @ 2017-08-11 0:24 UTC (permalink / raw)
To: Richard W.M. Jones
Cc: jejb, martin.petersen, mst, jasowang, linux-scsi, linux-kernel,
virtualization, hch, pbonzini
Richard,
> v1 was here:
>
> https://lkml.org/lkml/2017/8/10/689
>
> v1 -> v2:
>
> Remove .can_queue field from the templates.
Applied to 4.14/scsi-queue. Thank you!
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 10+ messages in thread