From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49270) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZALbw-00031D-UP for qemu-devel@nongnu.org; Wed, 01 Jul 2015 13:15:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZALbv-0005YX-UY for qemu-devel@nongnu.org; Wed, 01 Jul 2015 13:15:48 -0400 Received: from mail-ob0-x22f.google.com ([2607:f8b0:4003:c01::22f]:33220) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZALbv-0005YN-E4 for qemu-devel@nongnu.org; Wed, 01 Jul 2015 13:15:47 -0400 Received: by obpn3 with SMTP id n3so32663845obp.0 for ; Wed, 01 Jul 2015 10:15:46 -0700 (PDT) MIME-Version: 1.0 Sender: teja.foundation@gmail.com Date: Wed, 1 Jul 2015 22:45:45 +0530 Message-ID: From: Naredula Janardhana Reddy Content-Type: multipart/alternative; boundary=001a11c306ee35bb3b0519d379f6 Subject: [Qemu-devel] virtio-blk multiqueue support in qemu. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Stefan Hajnoczi --001a11c306ee35bb3b0519d379f6 Content-Type: text/plain; charset=UTF-8 HI, I am trying to improve the throughput of virtio-blk for my kernel ( https://github.com/naredula-jana/Jiny-Kernel ) to make use of the multi-core using multi-queue. But I found the latest qemu does not support multiqueue for virtio-blk, The feature flag does not return multi-queue support. Do we have any patches to the qemu for enabling multi-queue for virtio-blk similar to that of virtio-net. Can someone help some pointers related to this.. Thanks Jana --001a11c306ee35bb3b0519d379f6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
HI,
=C2=A0 =C2=A0=C2=A0 I am trying to improv= e the throughput of=C2=A0 virtio-blk for my kernel ( https://github.com/naredula-jana/Jiny-Ke= rnel=C2=A0 )=C2=A0 to make use of the multi-core using multi-queue. But= I found the latest qemu=C2=A0 does not support multiqueue for virtio-blk, = The feature flag does not return multi-queue support. Do we have any patche= s to the qemu for enabling multi-queue for virtio-blk similar to that of vi= rtio-net.=C2=A0 Can someone help some pointers=C2=A0 related to this..
<= br>

Thanks
Jana

--001a11c306ee35bb3b0519d379f6-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55545) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZATBw-0002bd-2U for qemu-devel@nongnu.org; Wed, 01 Jul 2015 21:21:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZATBs-0005A5-Mm for qemu-devel@nongnu.org; Wed, 01 Jul 2015 21:21:27 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47675) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZATBs-00059b-H7 for qemu-devel@nongnu.org; Wed, 01 Jul 2015 21:21:24 -0400 Date: Thu, 2 Jul 2015 09:21:21 +0800 From: Fam Zheng Message-ID: <20150702012121.GA4477@ad.nay.redhat.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] virtio-blk multiqueue support in qemu. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Naredula Janardhana Reddy Cc: qemu-devel@nongnu.org, Stefan Hajnoczi On Wed, 07/01 22:45, Naredula Janardhana Reddy wrote: > HI, > I am trying to improve the throughput of virtio-blk for my kernel ( > https://github.com/naredula-jana/Jiny-Kernel ) to make use of the > multi-core using multi-queue. But I found the latest qemu does not support > multiqueue for virtio-blk, The feature flag does not return multi-queue > support. Do we have any patches to the qemu for enabling multi-queue for > virtio-blk similar to that of virtio-net. Can someone help some pointers > related to this.. There was a series last year adding support for multi-queue but wan't merged to the code base: http://comments.gmane.org/gmane.comp.emulators.qemu/289126 Fam From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35087) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZCqTb-0000vy-Ma for qemu-devel@nongnu.org; Wed, 08 Jul 2015 10:37:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZCqTM-00085O-2B for qemu-devel@nongnu.org; Wed, 08 Jul 2015 10:37:31 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47181) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZCqTL-000855-Rz for qemu-devel@nongnu.org; Wed, 08 Jul 2015 10:37:15 -0400 Date: Wed, 8 Jul 2015 17:37:12 +0300 From: "Michael S. Tsirkin" Message-ID: <20150708173423-mutt-send-email-mst@redhat.com> References: <20150702012121.GA4477@ad.nay.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150702012121.GA4477@ad.nay.redhat.com> Subject: Re: [Qemu-devel] virtio-blk multiqueue support in qemu. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Fam Zheng Cc: Naredula Janardhana Reddy , Ming Lei , qemu-devel@nongnu.org, Stefan Hajnoczi On Thu, Jul 02, 2015 at 09:21:21AM +0800, Fam Zheng wrote: > On Wed, 07/01 22:45, Naredula Janardhana Reddy wrote: > > HI, > > I am trying to improve the throughput of virtio-blk for my kernel ( > > https://github.com/naredula-jana/Jiny-Kernel ) to make use of the > > multi-core using multi-queue. But I found the latest qemu does not support > > multiqueue for virtio-blk, The feature flag does not return multi-queue > > support. Do we have any patches to the qemu for enabling multi-queue for > > virtio-blk similar to that of virtio-net. Can someone help some pointers > > related to this.. > > There was a series last year adding support for multi-queue but wan't merged to > the code base: > > http://comments.gmane.org/gmane.comp.emulators.qemu/289126 > > Fam Looks like it was mostly about the missing GPL license on some files. Or did I miss some bigger issues? -- MST From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51738) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZD7tU-0005sn-On for qemu-devel@nongnu.org; Thu, 09 Jul 2015 05:13:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZD7tS-0002yn-3b for qemu-devel@nongnu.org; Thu, 09 Jul 2015 05:13:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47468) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZD7tR-0002yc-UL for qemu-devel@nongnu.org; Thu, 09 Jul 2015 05:13:22 -0400 Date: Thu, 9 Jul 2015 10:13:19 +0100 From: Stefan Hajnoczi Message-ID: <20150709091319.GC17340@stefanha-thinkpad.redhat.com> References: <20150702012121.GA4477@ad.nay.redhat.com> <20150708173423-mutt-send-email-mst@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IpbVkmxF4tDyP/Kb" Content-Disposition: inline In-Reply-To: <20150708173423-mutt-send-email-mst@redhat.com> Subject: Re: [Qemu-devel] virtio-blk multiqueue support in qemu. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: Naredula Janardhana Reddy , Fam Zheng , qemu-devel@nongnu.org, Ming Lei --IpbVkmxF4tDyP/Kb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 08, 2015 at 05:37:12PM +0300, Michael S. Tsirkin wrote: > On Thu, Jul 02, 2015 at 09:21:21AM +0800, Fam Zheng wrote: > > On Wed, 07/01 22:45, Naredula Janardhana Reddy wrote: > > > HI, > > > I am trying to improve the throughput of virtio-blk for my kern= el ( > > > https://github.com/naredula-jana/Jiny-Kernel ) to make use of the > > > multi-core using multi-queue. But I found the latest qemu does not s= upport > > > multiqueue for virtio-blk, The feature flag does not return multi-que= ue > > > support. Do we have any patches to the qemu for enabling multi-queue = for > > > virtio-blk similar to that of virtio-net. Can someone help some poin= ters > > > related to this.. > >=20 > > There was a series last year adding support for multi-queue but wan't m= erged to > > the code base: > >=20 > > http://comments.gmane.org/gmane.comp.emulators.qemu/289126 > >=20 > > Fam >=20 > Looks like it was mostly about the missing GPL license on some files. > Or did I miss some bigger issues? I think this led to a lot of optimization and some controversial design questions about bypassing parts of the QEMU block layer. So I doubt that this series can be merged without significant changes. Will take a look at it again though because I'm also interested in multiqueue block layer support. Stefan --IpbVkmxF4tDyP/Kb Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVnjsvAAoJEJykq7OBq3PIVK8IAKzVqPmVFpIZYtKIOWTCLj4d kcEWYB+CAu41L0bmx8keQPnEXop3lnL3HjnkTb7NBsH8iZEv7yYcKqmA0O9W4MWN yjw5V3n/nYJ1AxbqVEWMczqu6ggQaw2pdafAzCYA0Rlgc98AWlazyY3SGgrBXMkV DPvvv/WyTG2tSppRiEkbmk1BFZmu3ReLNpVozIufDvIDIis8WnbeBsdeRVV3wL6m sUPX8GFLZKnJcjyhrLKZKT0xPcdshUkUeF5pW5gW1EMFZ7fyQSbWGXEMOoxbRvOp Xf+3DDx3RLuoQ/3ftuGB46F20ulHS4DErMztzblJmuB5oCTQpim2AJjOwPLKORo= =Xiiw -----END PGP SIGNATURE----- --IpbVkmxF4tDyP/Kb-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46991) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZDBtT-0005Iu-MW for qemu-devel@nongnu.org; Thu, 09 Jul 2015 09:29:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZDBtO-0004e0-P7 for qemu-devel@nongnu.org; Thu, 09 Jul 2015 09:29:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41602) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZDBtO-0004dr-5r for qemu-devel@nongnu.org; Thu, 09 Jul 2015 09:29:34 -0400 Date: Thu, 9 Jul 2015 16:29:30 +0300 From: "Michael S. Tsirkin" Message-ID: <20150709162723-mutt-send-email-mst@redhat.com> References: <20150702012121.GA4477@ad.nay.redhat.com> <20150708173423-mutt-send-email-mst@redhat.com> <20150709132248.GA12399@stefanha-thinkpad.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150709132248.GA12399@stefanha-thinkpad.redhat.com> Subject: Re: [Qemu-devel] virtio-blk multiqueue support in qemu. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: Naredula Janardhana Reddy , Fam Zheng , Ming Lei , Stefan Hajnoczi , qemu-devel@nongnu.org On Thu, Jul 09, 2015 at 02:22:48PM +0100, Stefan Hajnoczi wrote: > On Wed, Jul 08, 2015 at 05:37:12PM +0300, Michael S. Tsirkin wrote: > > On Thu, Jul 02, 2015 at 09:21:21AM +0800, Fam Zheng wrote: > > > On Wed, 07/01 22:45, Naredula Janardhana Reddy wrote: > > > > HI, > > > > I am trying to improve the throughput of virtio-blk for my kernel ( > > > > https://github.com/naredula-jana/Jiny-Kernel ) to make use of the > > > > multi-core using multi-queue. But I found the latest qemu does not support > > > > multiqueue for virtio-blk, The feature flag does not return multi-queue > > > > support. Do we have any patches to the qemu for enabling multi-queue for > > > > virtio-blk similar to that of virtio-net. Can someone help some pointers > > > > related to this.. > > > > > > There was a series last year adding support for multi-queue but wan't merged to > > > the code base: > > > > > > http://comments.gmane.org/gmane.comp.emulators.qemu/289126 > > > > > > Fam > > > > Looks like it was mostly about the missing GPL license on some files. > > Or did I miss some bigger issues? > > I reviewed the series again. The most important part: > > It was not a true multiqueue implementation - all the virtqueues are > handled from a single thread in QEMU. The reason for this is that > QEMU's block layer isn't threadsafe so a lot of work is necessary to > support true multiqueue. > > Also, the performance numbers quoted cannot be attributed to multiqueue > due to the number of additional changes the series made. In the > meantime, some of these additional changes have been merged as separate > series. > > On the positive side, we do need something similar to the last 3 patches > to expose multiqueue to the guest. But it will require much more work > in QEMU to achieve true multiqueue. > > Stefan Might this be useful as an intermediate step, e.g. for testing guest code? -- MST From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51517) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZDcXH-0006Jm-GP for qemu-devel@nongnu.org; Fri, 10 Jul 2015 13:56:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZDcXG-0006Xk-1k for qemu-devel@nongnu.org; Fri, 10 Jul 2015 13:56:31 -0400 Received: from mail-ob0-x22c.google.com ([2607:f8b0:4003:c01::22c]:36401) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZDcXF-0006Xe-QU for qemu-devel@nongnu.org; Fri, 10 Jul 2015 13:56:29 -0400 Received: by obdbs4 with SMTP id bs4so195978491obd.3 for ; Fri, 10 Jul 2015 10:56:29 -0700 (PDT) MIME-Version: 1.0 Sender: teja.foundation@gmail.com In-Reply-To: <20150709162723-mutt-send-email-mst@redhat.com> References: <20150702012121.GA4477@ad.nay.redhat.com> <20150708173423-mutt-send-email-mst@redhat.com> <20150709132248.GA12399@stefanha-thinkpad.redhat.com> <20150709162723-mutt-send-email-mst@redhat.com> Date: Fri, 10 Jul 2015 23:26:29 +0530 Message-ID: From: Naredula Janardhana Reddy Content-Type: multipart/alternative; boundary=e89a8f923bea6a46d1051a8917b1 Subject: Re: [Qemu-devel] virtio-blk multiqueue support in qemu. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: Stefan Hajnoczi , Ming Lei , Fam Zheng , qemu-devel@nongnu.org, Stefan Hajnoczi --e89a8f923bea6a46d1051a8917b1 Content-Type: text/plain; charset=UTF-8 Yes, It will be usefull for testing the guest code. Currently virtio-blk does not returns the multi-queue in feature flag of virtio, so no way testing the feature in the guest driver. In the virtio-net, multi-queue feature can be enabled and tested. Thanks Jana On 9 July 2015 at 18:59, Michael S. Tsirkin wrote: > On Thu, Jul 09, 2015 at 02:22:48PM +0100, Stefan Hajnoczi wrote: > > On Wed, Jul 08, 2015 at 05:37:12PM +0300, Michael S. Tsirkin wrote: > > > On Thu, Jul 02, 2015 at 09:21:21AM +0800, Fam Zheng wrote: > > > > On Wed, 07/01 22:45, Naredula Janardhana Reddy wrote: > > > > > HI, > > > > > I am trying to improve the throughput of virtio-blk for my > kernel ( > > > > > https://github.com/naredula-jana/Jiny-Kernel ) to make use of > the > > > > > multi-core using multi-queue. But I found the latest qemu does > not support > > > > > multiqueue for virtio-blk, The feature flag does not return > multi-queue > > > > > support. Do we have any patches to the qemu for enabling > multi-queue for > > > > > virtio-blk similar to that of virtio-net. Can someone help some > pointers > > > > > related to this.. > > > > > > > > There was a series last year adding support for multi-queue but > wan't merged to > > > > the code base: > > > > > > > > http://comments.gmane.org/gmane.comp.emulators.qemu/289126 > > > > > > > > Fam > > > > > > Looks like it was mostly about the missing GPL license on some files. > > > Or did I miss some bigger issues? > > > > I reviewed the series again. The most important part: > > > > It was not a true multiqueue implementation - all the virtqueues are > > handled from a single thread in QEMU. The reason for this is that > > QEMU's block layer isn't threadsafe so a lot of work is necessary to > > support true multiqueue. > > > > Also, the performance numbers quoted cannot be attributed to multiqueue > > due to the number of additional changes the series made. In the > > meantime, some of these additional changes have been merged as separate > > series. > > > > On the positive side, we do need something similar to the last 3 patches > > to expose multiqueue to the guest. But it will require much more work > > in QEMU to achieve true multiqueue. > > > > Stefan > > Might this be useful as an intermediate step, e.g. for testing > guest code? > > -- > MST > --e89a8f923bea6a46d1051a8917b1 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Yes, It will be usefull for testing the gue= st code.=C2=A0 Currently virtio-blk does not returns the multi-queue in fea= ture flag of virtio, so no way testing the feature in the guest driver.
=
In the virtio-net,=C2=A0 multi-queue=C2=A0 feature can be enabled= =C2=A0 and tested.

Thanks
Jana

On 9 July 2015 at 18:59, Mi= chael S. Tsirkin <mst@redhat.com> wrote:
On Thu, Jul 09, 2015 = at 02:22:48PM +0100, Stefan Hajnoczi wrote:
> On Wed, Jul 08, 2015 at 05:37:12PM +0300, Michael S. Tsirkin wrote: > > On Thu, Jul 02, 2015 at 09:21:21AM +0800, Fam Zheng wrote:
> > > On Wed, 07/01 22:45, Naredula Janardhana Reddy wrote:
> > > > HI,
> > > >=C2=A0 =C2=A0 =C2=A0 I am trying to improve the throughp= ut of=C2=A0 virtio-blk for my kernel (
> > > > https://github.com/naredula-jana/Jin= y-Kernel=C2=A0 )=C2=A0 to make use of the
> > > > multi-core using multi-queue. But I found the latest qe= mu=C2=A0 does not support
> > > > multiqueue for virtio-blk, The feature flag does not re= turn multi-queue
> > > > support. Do we have any patches to the qemu for enablin= g multi-queue for
> > > > virtio-blk similar to that of virtio-net.=C2=A0 Can som= eone help some pointers
> > > > related to this..
> > >
> > > There was a series last year adding support for multi-queue = but wan't merged to
> > > the code base:
> > >
> > > http://comments.gmane.org/g= mane.comp.emulators.qemu/289126
> > >
> > > Fam
> >
> > Looks like it was mostly about the missing GPL license on some fi= les.
> > Or did I miss some bigger issues?
>
> I reviewed the series again.=C2=A0 The most important part:
>
> It was not a true multiqueue implementation - all the virtqueues are > handled from a single thread in QEMU.=C2=A0 The reason for this is tha= t
> QEMU's block layer isn't threadsafe so a lot of work is necess= ary to
> support true multiqueue.
>
> Also, the performance numbers quoted cannot be attributed to multiqueu= e
> due to the number of additional changes the series made.=C2=A0 In the<= br> > meantime, some of these additional changes have been merged as separat= e
> series.
>
> On the positive side, we do need something similar to the last 3 patch= es
> to expose multiqueue to the guest.=C2=A0 But it will require much more= work
> in QEMU to achieve true multiqueue.
>
> Stefan

Might this be useful as an intermediate step, e.g. for testing<= br> guest code?

--
MST

--e89a8f923bea6a46d1051a8917b1-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34196) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZEabL-0003hU-Ny for qemu-devel@nongnu.org; Mon, 13 Jul 2015 06:04:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZEabH-0007He-9Y for qemu-devel@nongnu.org; Mon, 13 Jul 2015 06:04:43 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60222) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZEabH-0007HU-1q for qemu-devel@nongnu.org; Mon, 13 Jul 2015 06:04:39 -0400 Date: Mon, 13 Jul 2015 11:04:37 +0100 From: Stefan Hajnoczi Message-ID: <20150713100437.GB17927@stefanha-thinkpad.redhat.com> References: <20150702012121.GA4477@ad.nay.redhat.com> <20150708173423-mutt-send-email-mst@redhat.com> <20150709132248.GA12399@stefanha-thinkpad.redhat.com> <20150709162723-mutt-send-email-mst@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Pd0ReVV5GZGQvF3a" Content-Disposition: inline In-Reply-To: <20150709162723-mutt-send-email-mst@redhat.com> Subject: Re: [Qemu-devel] virtio-blk multiqueue support in qemu. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: Stefan Hajnoczi , Naredula Janardhana Reddy , Fam Zheng , Ming Lei , qemu-devel@nongnu.org --Pd0ReVV5GZGQvF3a Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 09, 2015 at 04:29:30PM +0300, Michael S. Tsirkin wrote: > On Thu, Jul 09, 2015 at 02:22:48PM +0100, Stefan Hajnoczi wrote: > > On Wed, Jul 08, 2015 at 05:37:12PM +0300, Michael S. Tsirkin wrote: > > > On Thu, Jul 02, 2015 at 09:21:21AM +0800, Fam Zheng wrote: > > > > On Wed, 07/01 22:45, Naredula Janardhana Reddy wrote: > > > > > HI, > > > > > I am trying to improve the throughput of virtio-blk for my = kernel ( > > > > > https://github.com/naredula-jana/Jiny-Kernel ) to make use of t= he > > > > > multi-core using multi-queue. But I found the latest qemu does n= ot support > > > > > multiqueue for virtio-blk, The feature flag does not return multi= -queue > > > > > support. Do we have any patches to the qemu for enabling multi-qu= eue for > > > > > virtio-blk similar to that of virtio-net. Can someone help some = pointers > > > > > related to this.. > > > >=20 > > > > There was a series last year adding support for multi-queue but wan= 't merged to > > > > the code base: > > > >=20 > > > > http://comments.gmane.org/gmane.comp.emulators.qemu/289126 > > > >=20 > > > > Fam > > >=20 > > > Looks like it was mostly about the missing GPL license on some files. > > > Or did I miss some bigger issues? > >=20 > > I reviewed the series again. The most important part: > >=20 > > It was not a true multiqueue implementation - all the virtqueues are > > handled from a single thread in QEMU. The reason for this is that > > QEMU's block layer isn't threadsafe so a lot of work is necessary to > > support true multiqueue. > >=20 > > Also, the performance numbers quoted cannot be attributed to multiqueue > > due to the number of additional changes the series made. In the > > meantime, some of these additional changes have been merged as separate > > series. > >=20 > > On the positive side, we do need something similar to the last 3 patches > > to expose multiqueue to the guest. But it will require much more work > > in QEMU to achieve true multiqueue. > >=20 > > Stefan >=20 > Might this be useful as an intermediate step, e.g. for testing > guest code? Yes. It will have a performance impact since there are now more virtqueues and interrupts. It just won't scale the same way blk-mq does on the host. I'll put together patches based on qemu.git/master. Stefan --Pd0ReVV5GZGQvF3a Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVo401AAoJEJykq7OBq3PIXqAIAMIamidfMh8SGhN70tsnryE2 /8DoqkBSIuF1NEweYTKFyERJp8iLwnUcmspTv8SaN0g1hHnXs1drhGrM/JLizdV9 kswh9sro1dQ8NGO8bEBt1cCQ59TUtcT+8jX3sbrcB6CvRIHjHIocO4quiB981Drx GhPKoBr2RMuTvV7PbstIhupYRPlgp/Q0+uX0tUNiP3314fjQgUZADEXcpDsBArO7 ATu9jXOrdgkbZ01wXOjQm5blv0rbTWurglwpWdwITGQJOIwW5LjZoaAdU3aTkCqK H+DfZlzQwC+qR+fPZMn3g+GkPC/7mlfBCXsU8xg/g28rStFkPTLkgslCXZtAKN0= =QFFm -----END PGP SIGNATURE----- --Pd0ReVV5GZGQvF3a-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37048) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZEao3-0003FN-GW for qemu-devel@nongnu.org; Mon, 13 Jul 2015 06:17:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZEanz-0004Rm-BX for qemu-devel@nongnu.org; Mon, 13 Jul 2015 06:17:51 -0400 Received: from mail-wi0-x236.google.com ([2a00:1450:400c:c05::236]:34046) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZEanz-0004Rc-43 for qemu-devel@nongnu.org; Mon, 13 Jul 2015 06:17:47 -0400 Received: by wibud3 with SMTP id ud3so25392912wib.1 for ; Mon, 13 Jul 2015 03:17:46 -0700 (PDT) Sender: Paolo Bonzini References: <20150702012121.GA4477@ad.nay.redhat.com> <20150708173423-mutt-send-email-mst@redhat.com> <20150709132248.GA12399@stefanha-thinkpad.redhat.com> <20150709162723-mutt-send-email-mst@redhat.com> From: Paolo Bonzini Message-ID: <55A39047.5070806@redhat.com> Date: Mon, 13 Jul 2015 12:17:43 +0200 MIME-Version: 1.0 In-Reply-To: <20150709162723-mutt-send-email-mst@redhat.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] virtio-blk multiqueue support in qemu. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" , Stefan Hajnoczi Cc: Naredula Janardhana Reddy , Fam Zheng , Ming Lei , Stefan Hajnoczi , qemu-devel@nongnu.org On 09/07/2015 15:29, Michael S. Tsirkin wrote: > > On the positive side, we do need something similar to the last 3 patches > > to expose multiqueue to the guest. But it will require much more work > > in QEMU to achieve true multiqueue. > > Might this be useful as an intermediate step, e.g. for testing > guest code? Yes, this is already how virtio-scsi multiqueue is implemented. The speedup is noticeable. However, for virtio-blk there is a huge consumption of interrupt vectors in the guest, so it doesn't scale to multiple disks as well as virtio-scsi. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51876) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZEbo2-0006Vy-47 for qemu-devel@nongnu.org; Mon, 13 Jul 2015 07:21:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZEbnw-000596-Mh for qemu-devel@nongnu.org; Mon, 13 Jul 2015 07:21:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47080) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZEbnw-000592-G7 for qemu-devel@nongnu.org; Mon, 13 Jul 2015 07:21:48 -0400 Date: Mon, 13 Jul 2015 14:21:44 +0300 From: "Michael S. Tsirkin" Message-ID: <20150713141421-mutt-send-email-mst@redhat.com> References: <20150702012121.GA4477@ad.nay.redhat.com> <20150708173423-mutt-send-email-mst@redhat.com> <20150709132248.GA12399@stefanha-thinkpad.redhat.com> <20150709162723-mutt-send-email-mst@redhat.com> <55A39047.5070806@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <55A39047.5070806@redhat.com> Subject: Re: [Qemu-devel] virtio-blk multiqueue support in qemu. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: Naredula Janardhana Reddy , Fam Zheng , Stefan Hajnoczi , Ming Lei , qemu-devel@nongnu.org, Stefan Hajnoczi On Mon, Jul 13, 2015 at 12:17:43PM +0200, Paolo Bonzini wrote: > > > On 09/07/2015 15:29, Michael S. Tsirkin wrote: > > > On the positive side, we do need something similar to the last 3 patches > > > to expose multiqueue to the guest. But it will require much more work > > > in QEMU to achieve true multiqueue. > > > > Might this be useful as an intermediate step, e.g. for testing > > guest code? > > Yes, this is already how virtio-scsi multiqueue is implemented. The > speedup is noticeable. However, for virtio-blk there is a huge > consumption of interrupt vectors in the guest, so it doesn't scale to > multiple disks as well as virtio-scsi. Could you elaborate please? interrupt vector usage is generally up to the guest, so whatever the issue is might be fixable by driver changes alone. -- MST From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52284) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZEbqV-0000g5-Ea for qemu-devel@nongnu.org; Mon, 13 Jul 2015 07:24:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZEbqQ-0005P4-8W for qemu-devel@nongnu.org; Mon, 13 Jul 2015 07:24:27 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47607) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZEbqQ-0005Or-2E for qemu-devel@nongnu.org; Mon, 13 Jul 2015 07:24:22 -0400 References: <20150702012121.GA4477@ad.nay.redhat.com> <20150708173423-mutt-send-email-mst@redhat.com> <20150709132248.GA12399@stefanha-thinkpad.redhat.com> <20150709162723-mutt-send-email-mst@redhat.com> <55A39047.5070806@redhat.com> <20150713141421-mutt-send-email-mst@redhat.com> From: Paolo Bonzini Message-ID: <55A39FDF.1060800@redhat.com> Date: Mon, 13 Jul 2015 13:24:15 +0200 MIME-Version: 1.0 In-Reply-To: <20150713141421-mutt-send-email-mst@redhat.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] virtio-blk multiqueue support in qemu. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: Naredula Janardhana Reddy , Fam Zheng , Stefan Hajnoczi , Ming Lei , qemu-devel@nongnu.org, Stefan Hajnoczi On 13/07/2015 13:21, Michael S. Tsirkin wrote: > > Yes, this is already how virtio-scsi multiqueue is implemented. The > > speedup is noticeable. However, for virtio-blk there is a huge > > consumption of interrupt vectors in the guest, so it doesn't scale to > > multiple disks as well as virtio-scsi. > > Could you elaborate please? interrupt vector usage is generally > up to the guest, so whatever the issue is might be fixable by driver > changes alone. Yes, but in practice neither Linux nor Windows (AFAIK) are able to share the same vector across multiple MSIs. Paolo From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56564) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZEc7P-0004nS-Es for qemu-devel@nongnu.org; Mon, 13 Jul 2015 07:41:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZEc7L-0004wR-BY for qemu-devel@nongnu.org; Mon, 13 Jul 2015 07:41:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54728) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZEc7L-0004wF-5a for qemu-devel@nongnu.org; Mon, 13 Jul 2015 07:41:51 -0400 Date: Mon, 13 Jul 2015 14:41:47 +0300 From: "Michael S. Tsirkin" Message-ID: <20150713143402-mutt-send-email-mst@redhat.com> References: <20150702012121.GA4477@ad.nay.redhat.com> <20150708173423-mutt-send-email-mst@redhat.com> <20150709132248.GA12399@stefanha-thinkpad.redhat.com> <20150709162723-mutt-send-email-mst@redhat.com> <55A39047.5070806@redhat.com> <20150713141421-mutt-send-email-mst@redhat.com> <55A39FDF.1060800@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <55A39FDF.1060800@redhat.com> Subject: Re: [Qemu-devel] virtio-blk multiqueue support in qemu. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: Naredula Janardhana Reddy , Fam Zheng , Stefan Hajnoczi , Ming Lei , qemu-devel@nongnu.org, Stefan Hajnoczi On Mon, Jul 13, 2015 at 01:24:15PM +0200, Paolo Bonzini wrote: > > > On 13/07/2015 13:21, Michael S. Tsirkin wrote: > > > Yes, this is already how virtio-scsi multiqueue is implemented. The > > > speedup is noticeable. However, for virtio-blk there is a huge > > > consumption of interrupt vectors in the guest, so it doesn't scale to > > > multiple disks as well as virtio-scsi. > > > > Could you elaborate please? interrupt vector usage is generally > > up to the guest, so whatever the issue is might be fixable by driver > > changes alone. > > Yes, but in practice neither Linux nor Windows (AFAIK) are able to share > the same vector across multiple MSIs. > > Paolo AFAIK Linux can do this of you pass in IRQF_SHARED. We just never saw a need to do this before. Sharing between devices might also work, I think Amos Kong made it work under qemu, though this relied on reading ISR when in MSI mode, which is only allowed in virtio 1 mode. virtio spec also lets drivers control the mapping from VQs to MSI vectors. That's portable across OSes. That one is actually used, at least the Linux drivers support a mode where all VQs share a single vector. -- MST