From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35874) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZM67l-00043g-SS for qemu-devel@nongnu.org; Sun, 02 Aug 2015 23:09:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZM67i-0004RI-L5 for qemu-devel@nongnu.org; Sun, 02 Aug 2015 23:09:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35874) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZM67i-0004RD-DV for qemu-devel@nongnu.org; Sun, 02 Aug 2015 23:09:10 -0400 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (Postfix) with ESMTPS id 70B8980B5A for ; Mon, 3 Aug 2015 03:09:09 +0000 (UTC) Date: Mon, 3 Aug 2015 11:09:06 +0800 From: Fam Zheng Message-ID: <20150803030906.GA13938@ad.nay.redhat.com> References: <20150731174542.44862e3a@markmb_rh> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <20150731174542.44862e3a@markmb_rh> Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] Modularizing QEMU RFC List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Marc =?iso-8859-1?Q?Mar=ED?= Cc: qemu-devel On Fri, 07/31 17:45, Marc Mar=ED wrote: > Hi everyone >=20 > I propose improving the current modular driver system for QEMU so it > can benefit everybody in speed and flexibility. I'm looking for other > ideas, comments, critics, etc. >=20 > - Background - > In order to speed up QEMU, I'm looking at the high number of libraries > and dependencies that it loads. I've generated a QEMU image that needs > 145 shared libraries to start, and there were still some options > disabled. This is a lot, and this means that it is slow. >=20 > So I've been looking at actual module system. Yes, QEMU does have a > module system, but disabled by default. The problem is, the modules get > loaded always during startup. This means, booting with modules enabled > is even slower, because loading at runtime is slower that letting the > linker do all the work at the start. At this point, I doubt of the > benefits of the actual modular system. >=20 > But, if disabling the actual block modules (iscsi, curl, rbd, gluster, > ssh and dmg) gives 40 ms of speedup, I think is worth an effort of > improving modules, and modularizing new parts >=20 > - Current module flow - > The current module system is based on shared libraries. Each of these > libraries has a constructor that registers the BlockDriver structure to > the global bdrv_drivers list. This list is later searched by the type, > the protocol, etc. >=20 > - Proposals - > I have in mind two ideas, which are not mutually exclusive: >=20 > -- A -- > Having a huge lookup table with the names of the drivers that are in > modules, and the name of the modules where it can be found. When > some type is not found in the driver lists, this table can be traversed > to find the library and load it. >=20 > This requires an effort by the driver developer to fill the tables. But > the refactoring work can stay localized in the drivers and the modules, > it is (possibly) not necessary to touch any other part of the QEMU > system. >=20 > I don't specially like this huge manual-edited table. Yeah, the way to fill the tables is an (implementation) question, but the= re are still more questions if we go this way... bdrv_find_protocol is easy, the protocol name is extracted from image uri= or blockdev options, we can load the module by the name. bdrv_probe_all is harder. If we modularize a format driver, its .bdrv_pro= be code will be in the module. If we want to do the format detection, we nee= d to load all format drivers. This means if the command line has an unspecifie= d format, we'll still need to load all drivers at starting phase. (I wish a= ll formats are probed according to magic bytes at offset 0, so we can simpli= fy the .bdrv_probe logic and do it with data matching in block.c like the protoc= ol case, but that's not true for VMDK :( ) I believe other sub systems have this kind of paradox as well. >=20 > -- B -- > The same --enable-X that is done at compile time, but directly on the > command line when running QEMU or even in hot at the monitor. >=20 > This solution requires work on the monitor, the command line > processing, the modules and the drivers system. It is also less > transparent to the final user. I'm afraid this breaks backward compatibility. :( >=20 > - My comments on my proposals - > Ideally, I'd like a mixed solution. The user can specify what wants to > load, but also, when something is not found, it is automatically > searched. >=20 > In both options, the current module system has to be partly rewritten, > and some of the current drivers with module capability might need to be > modified to adapt to the new specifications. >=20 > And, a part from improving the current modular interface, there are a > lot of other devices that might benefit from it, not just the block > devices. >=20 > I still haven't looked at the memory footprint of QEMU, but I'm sure > that the QEMU binary will lose a lot of weight with this addition. >=20 > - Closing - > This is just a brief draft. These proposals can be improved a lot, or > there might be some other solutions that I haven't thought of. >=20 > So, I'd like to ask for ideas, comments, critics, improvements, etc. > And also ask for contributors to this endeavour, because it will be a > huge amount of work. >=20 > Thanks for reading and have a nice weekend > Marc >=20