linux-scsi.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Paolo Bonzini <pbonzini@redhat.com>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>,
	linux-scsi@vger.kernel.org, kvm@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	target-devel <target-devel@vger.kernel.org>,
	Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH 5/5] virtio-scsi: introduce multiqueue support
Date: Tue, 04 Sep 2012 12:25:03 +0200	[thread overview]
Message-ID: <5045D6FF.5020801@redhat.com> (raw)
In-Reply-To: <20120904084628.GA8437@redhat.com>

Il 04/09/2012 10:46, Michael S. Tsirkin ha scritto:
>>>> +static int virtscsi_queuecommand_multi(struct Scsi_Host *sh,
>>>> +				       struct scsi_cmnd *sc)
>>>> +{
>>>> +	struct virtio_scsi *vscsi = shost_priv(sh);
>>>> +	struct virtio_scsi_target_state *tgt = vscsi->tgt[sc->device->id];
>>>> +	unsigned long flags;
>>>> +	u32 queue_num;
>>>> +
>>>> +	/* Using an atomic_t for tgt->reqs lets the virtqueue handler
>>>> +	 * decrement it without taking the spinlock.
>>>> +	 */
> 
> Above comment is not really helpful - reader can be safely assumed to
> know what atomic_t is.

Sure, the comment explains that we use an atomic because _elsewhere_ the
tgt_lock is not held while modifying reqs.

> Please delete, and replace with the text from commit log
> that explains the heuristic used to select req_vq.

Ok.

> Also please add a comment near 'reqs' definition.
> Something like "number of outstanding requests - used to detect idle
> target".

Ok.

> 
>>>> +	spin_lock_irqsave(&tgt->tgt_lock, flags);
> 
> Looks like this lock can be removed - req_vq is only
> modified when target is idle and only used when it is
> not idle.

If you have two incoming requests at the same time, req_vq is also
modified when the target is not idle; that's the point of the lock.

Suppose tgt->reqs = 0 initially, and you have two processors/queues.
Initially tgt->req_vq is queue #1.  If you have this:

    queuecommand on CPU #0         queuecommand #2 on CPU #1
  --------------------------------------------------------------
    atomic_inc_return(...) == 1
                                   atomic_inc_return(...) == 2
                                   virtscsi_queuecommand to queue #1
    tgt->req_vq = queue #0
    virtscsi_queuecommand to queue #0

then two requests are issued to different queues without a quiescent
point in the middle.

>>>> +	if (atomic_inc_return(&tgt->reqs) == 1) {
>>>> +		queue_num = smp_processor_id();
>>>> +		while (unlikely(queue_num >= vscsi->num_queues))
>>>> +			queue_num -= vscsi->num_queues;
>>>> +		tgt->req_vq = &vscsi->req_vqs[queue_num];
>>>> +	}
>>>> +	spin_unlock_irqrestore(&tgt->tgt_lock, flags);
>>>> +	return virtscsi_queuecommand(vscsi, tgt, sc);
>>>> +}
>>>> +
>>>> +
> 
> .....
> 
>>>> +static int virtscsi_queuecommand_single(struct Scsi_Host *sh,
>>>> +                                       struct scsi_cmnd *sc)
>>>> +{
>>>> +       struct virtio_scsi *vscsi = shost_priv(sh);
>>>> +       struct virtio_scsi_target_state *tgt = vscsi->tgt[sc->device->id];
>>>> +
>>>> +       atomic_inc(&tgt->reqs);
>>>> +       return virtscsi_queuecommand(vscsi, tgt, sc);
>>>> +}
>>>> +
> 
> Here, reqs is unused - why bother incrementing it?
> A branch on completion would be cheaper IMHO.

Well, I could also let tgt->reqs go negative, but it would be a bit untidy.

Another alternative is to access the target's target_busy field with
ACCESS_ONCE, and drop reqs altogether.  Too tricky to do this kind of
micro-optimization so early, though.

>> virtio-scsi multiqueue has a performance benefit up to 20%
> 
> To be fair, you could be running in single queue mode.
> In that case extra atomics and indirection that this code
> brings will just add overhead without benefits.
> I don't know how significant would that be.

Not measurable in my experiments.

Paolo

  reply	other threads:[~2012-09-04 10:25 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-08-28 11:54 [PATCH 0/5] Multiqueue virtio-scsi Paolo Bonzini
2012-08-28 11:54 ` [PATCH 1/5] virtio-ring: move queue_index to vring_virtqueue Paolo Bonzini
2012-08-29  7:54   ` Jason Wang
2012-09-05 23:32   ` Rusty Russell
2012-08-28 11:54 ` [PATCH 2/5] virtio: introduce an API to set affinity for a virtqueue Paolo Bonzini
2012-09-05 23:32   ` Rusty Russell
2012-08-28 11:54 ` [PATCH 3/5] virtio-scsi: allocate target pointers in a separate memory block Paolo Bonzini
2012-08-28 14:07   ` Sasha Levin
2012-08-28 14:25     ` Paolo Bonzini
2012-08-28 11:54 ` [PATCH 4/5] virtio-scsi: pass struct virtio_scsi to virtqueue completion function Paolo Bonzini
2012-08-28 11:54 ` [PATCH 5/5] virtio-scsi: introduce multiqueue support Paolo Bonzini
2012-09-04  2:21   ` Nicholas A. Bellinger
2012-09-04  6:46     ` Paolo Bonzini
2012-09-04  8:46       ` Michael S. Tsirkin
2012-09-04 10:25         ` Paolo Bonzini [this message]
2012-09-04 11:09           ` Michael S. Tsirkin
2012-09-04 11:18             ` Paolo Bonzini
2012-09-04 13:35               ` Michael S. Tsirkin
2012-09-04 13:45                 ` Paolo Bonzini
2012-09-04 14:19                   ` Michael S. Tsirkin
2012-09-04 14:25                     ` Paolo Bonzini
2012-09-04 20:11       ` Nicholas A. Bellinger
2012-09-05  7:03         ` Paolo Bonzini
2012-09-04 12:48   ` Michael S. Tsirkin
2012-09-04 13:49     ` Paolo Bonzini
2012-09-04 14:21       ` Michael S. Tsirkin
2012-09-04 14:30         ` Paolo Bonzini
2012-09-04 14:41           ` Michael S. Tsirkin
2012-09-04 14:47   ` Michael S. Tsirkin
2012-09-04 14:55     ` Paolo Bonzini
2012-09-04 15:03       ` Michael S. Tsirkin
2012-08-30  7:13 ` [PATCH 0/5] Multiqueue virtio-scsi Stefan Hajnoczi
2012-08-30 14:53 ` Michael S. Tsirkin
2012-08-30 15:45   ` Paolo Bonzini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5045D6FF.5020801@redhat.com \
    --to=pbonzini@redhat.com \
    --cc=axboe@kernel.dk \
    --cc=hch@lst.de \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=mst@redhat.com \
    --cc=target-devel@vger.kernel.org \
    --cc=virtualization@lists.linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).