From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:44220) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QIKC7-0007MT-AI for qemu-devel@nongnu.org; Fri, 06 May 2011 08:31:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QIKC6-0000Y9-7f for qemu-devel@nongnu.org; Fri, 06 May 2011 08:31:43 -0400 Received: from mail-gw0-f45.google.com ([74.125.83.45]:63906) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QIKC6-0000Xt-51 for qemu-devel@nongnu.org; Fri, 06 May 2011 08:31:42 -0400 Received: by gwb19 with SMTP id 19so1342067gwb.4 for ; Fri, 06 May 2011 05:31:40 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <4DC2B946.80100@redhat.com> References: <4DC1D30F.4070408@redhat.com> <20110505094323.GC5298@stefanha-thinkpad.localdomain> <4DC29CCB.8050903@redhat.com> <4DC2B455.9090007@suse.de> <4DC2B946.80100@redhat.com> Date: Fri, 6 May 2011 13:31:40 +0100 Message-ID: From: Stefan Hajnoczi Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] virtio-scsi spec, first public draft List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: qemu-devel , Hannes Reinecke , Stefan Hajnoczi On Thu, May 5, 2011 at 3:50 PM, Paolo Bonzini wrote: > On 05/05/2011 04:29 PM, Hannes Reinecke wrote: >>> >>> I chose 1 requestq per target so that, with MSI-X support, each >>> target can be associated to one MSI-X vector. >>> >>> If you want a large number of units, you can subdivide targets into >>> logical units, or use multiple adapters if you prefer. We can have >>> 20-odd SCSI adapters, each with 65534 targets. I think we're way >>> beyond the practical limits even before LUN support is added to QEMU. >> >> But this will make queue full tracking harder. >> If we have one queue per LUN the SCSI stack is able to track QUEUE FULL >> states and will adjust the queue depth accordingly. >> When we have only one queue per target we cannot track QUEUE FULL >> anymore and have to rely on the static per-host 'can_queue' setting. >> Which doesn't work as well, especially in a virtualized environment >> where the queue full conditions might change at any time. > > So you want one virtqueue per LUN? =A0I had it in the first version, but = then > you had to associate a (target, 8-byte LUN) pair to each virtqueue manual= ly. > =A0That was very hairy, so I changed it to one target per queue. > >> But read on: >> >>> For comparison, Windows supports up to 1024 targets per adapter >>> (split across 8 channels); IBM vSCSI provides up to 128; VMware >>> supports a maximum of 15 SCSI targets per adapter and 4 adapters per >>> VM. >>> >> We don't have to impose any hard limits here. The virtio scsi transport >> would need to be able to detect the targets, and we would be using >> whatever targets have been found. > > Yes, that's what I wrote above. =A0Right now "detect the targets" means "= send > INQUIRY for LUN0 and/or REPORT LUNS to each virtqueue", thanks to the 1:1 > relationship. =A0In my first version it would mean: > > - associate each target's LUN0 to a virtqueue > - if needed, send INQUIRY for LUN0 and/or REPORT LUNS > - if needed, deassociate the LUN0 and the virtqueue > > Really, it was ugly. =A0It also brings a lot more the question, such as w= hat > to do if a virtqueue has pending requests at deassociation time. > >>> Yes, just add the first LUN to it (it will be LUN0 which must be >>> there anyway). The target's existence will be reported on the >>> control receiveq. >>> >> ?? How is this supposed to work? >> How can I detect the existence of a virtqueue ? > > Config space tells you how many virtqueue exist. =A0That gives how many > targets you can address at most. =A0If some of them are empty at the begi= nning > of the guest's life, their LUN0 will fail to answer INQUIRY and REPORT LU= NS. > > (It is the same for vmw_pvscsi by the way, except simpler: the maximum # = of > targets is not configurable, and there is just one queue + one interrupt)= . Okay, this explains how you plan to handle targets appearing - you want to set a maximum number of targets. I was wondering how we would add virtqueues dynamically (and why the control vqs are placed last at n,n+1 instead of 0,1). Like Hannes said, why introduce a limit here if we don't have to? I'm really not sure I understand the win of creating lots of virtqueues. I just want a pipe out onto the SCSI bus so I can talk to all devices in the SCSI domain. Creating separate virtqueues increases complexity in the driver and emulation IMO. What is the MSI-X win you mentioned? I guess if an application on vcpu0 is accessing target0 a lot then interrupt handling can be handled on vcpu0 while other vcpus handle interrupts for other SCSI targets? I remember VMware pv scsi has a trick here, each request can contain the vcpu number which influences interrupt routing somehow. Stefan