From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52899) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZMAYD-0001DT-6U for qemu-devel@nongnu.org; Mon, 03 Aug 2015 03:52:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZMAY8-0000Vy-Ah for qemu-devel@nongnu.org; Mon, 03 Aug 2015 03:52:49 -0400 Received: from mx1.redhat.com ([209.132.183.28]:58361) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZMAY8-0000VX-35 for qemu-devel@nongnu.org; Mon, 03 Aug 2015 03:52:44 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id 2AFB991584 for ; Mon, 3 Aug 2015 07:52:43 +0000 (UTC) Date: Mon, 3 Aug 2015 09:52:38 +0200 From: Marc =?UTF-8?B?TWFyw60=?= Message-ID: <20150803095238.663a7bee@markmb_rh> In-Reply-To: <20150803030906.GA13938@ad.nay.redhat.com> References: <20150731174542.44862e3a@markmb_rh> <20150803030906.GA13938@ad.nay.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] Modularizing QEMU RFC List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Fam Zheng Cc: qemu-devel On Mon, 3 Aug 2015 11:09:06 +0800 Fam Zheng wrote: > On Fri, 07/31 17:45, Marc Mar=C3=AD wrote: > > Hi everyone > >=20 > > I propose improving the current modular driver system for QEMU so it > > can benefit everybody in speed and flexibility. I'm looking for > > other ideas, comments, critics, etc. > >=20 > > - Background - > > In order to speed up QEMU, I'm looking at the high number of > > libraries and dependencies that it loads. I've generated a QEMU > > image that needs 145 shared libraries to start, and there were > > still some options disabled. This is a lot, and this means that it > > is slow. > >=20 > > So I've been looking at actual module system. Yes, QEMU does have a > > module system, but disabled by default. The problem is, the modules > > get loaded always during startup. This means, booting with modules > > enabled is even slower, because loading at runtime is slower that > > letting the linker do all the work at the start. At this point, I > > doubt of the benefits of the actual modular system. > >=20 > > But, if disabling the actual block modules (iscsi, curl, rbd, > > gluster, ssh and dmg) gives 40 ms of speedup, I think is worth an > > effort of improving modules, and modularizing new parts > >=20 > > - Current module flow - > > The current module system is based on shared libraries. Each of > > these libraries has a constructor that registers the BlockDriver > > structure to the global bdrv_drivers list. This list is later > > searched by the type, the protocol, etc. > >=20 > > - Proposals - > > I have in mind two ideas, which are not mutually exclusive: > >=20 > > -- A -- > > Having a huge lookup table with the names of the drivers that are in > > modules, and the name of the modules where it can be found. When > > some type is not found in the driver lists, this table can be > > traversed to find the library and load it. > >=20 > > This requires an effort by the driver developer to fill the tables. > > But the refactoring work can stay localized in the drivers and the > > modules, it is (possibly) not necessary to touch any other part of > > the QEMU system. > >=20 > > I don't specially like this huge manual-edited table. >=20 > Yeah, the way to fill the tables is an (implementation) question, but > there are still more questions if we go this way... >=20 > bdrv_find_protocol is easy, the protocol name is extracted from image > uri or blockdev options, we can load the module by the name. >=20 > bdrv_probe_all is harder. If we modularize a format driver, > its .bdrv_probe code will be in the module. If we want to do the > format detection, we need to load all format drivers. This means if > the command line has an unspecified format, we'll still need to load > all drivers at starting phase. (I wish all formats are probed > according to magic bytes at offset 0, so we can simplify > the .bdrv_probe logic and do it with data matching in block.c like > the protocol case, but that's not true for VMDK :( ) >=20 > I believe other sub systems have this kind of paradox as well. I managed to overlook that... If the user doesn't specify the type of the block device, then, all block drivers will have to be tested. I see this is based on scores. If there is a score that means "this is my type for sure" (which I don't know), the probe could be stopped there. But the user can also specify the type for its block device (if I remember correctly). So, if he wants more speed, he could just specify the type of the block device. =20 > >=20 > > -- B -- > > The same --enable-X that is done at compile time, but directly on > > the command line when running QEMU or even in hot at the monitor. > >=20 > > This solution requires work on the monitor, the command line > > processing, the modules and the drivers system. It is also less > > transparent to the final user. >=20 > I'm afraid this breaks backward compatibility. :( :( Maybe my ideas are a bit too ideal without rewriting half QEMU. I should have left my ideas to cool down and rest before writting them down. So any other ideas to reduce the library overhead are appreciated. Thanks Marc