Re: [Qemu-devel] [Qemu-block] [PATCH v3 00/32] Dynamic module loading for block drivers

[John Snow <jsnow@redhat.com>]

On 07/05/2016 11:24 AM, Colin Lord wrote:
> This is the next version of this patch series. The first three patches
> in the series are mostly the same as they were last time, but with the
> issues mentioned in the reviews fixed. Most notably this means much less
> copy-paste happening in block.c.
> 
> The new changes, and the reason this series is so large, is a result of
> isolating the block driver probe functions from the rest of the drivers.
> This was mentioned in the reviews from the last version as something
> that should be added. This allows the probes to be accessed without
> needing to load all the modules. There are quite a few probes, which is
> why there are so many patches in this series. I tried to keep the
> patches where I was relocating the code separate from the patches where
> the probe function signatures were changed.
> 
> The probe functions now return both a score and a format/protocol name,
> which allows them to be completely separate from the drivers they
> correspond to. I didn't put all the probe functions in the same file as
> Fam suggested because it turns out there are some naming conflicts
> between some macros commonly used in the probe functions. For instance,
> QCowHeader is used in both qcow and qcow2 probes, but the meaning is
> different between them. Having separate files makes resolving these
> conflicts simple.
> 
> Also, just as a point of interest, I've done some basic profiling of my
> own to see what performance benefits there are to be made. I recorded
> the time to main of each option that you can enable/disable in the
> configuration by enabling each option by itself while explicitly
> disabling all other options. With each configuration I ran qemu 5 times
> while recording the time to main. All the results are in milliseconds.
> 
> minimal config      2.772
> vhdx                2.776
> attr                2.786
> coroutine-pool      2.836
> bzip                2.856
> linux-aio           2.867
> vnc-jpeg            2.872
> snappy              2.875
> fdt                 2.888
> brlapi              2.916
> vhost-net           2.922
> bluez               2.937
> cap-ng              2.940
> libiscsi            2.945
> nettle              2.945
> seccomp             2.973
> numa                2.983
> lzo                 2.994
> usb-redir           3.005
> cocoa               3.007
> libnfs              3.026
> vnc-sasl            3.071
> vnc-png             3.081
> virtfs              3.081
> uuid                3.086
> tpm                 3.136
> curses              3.145
> gcrypt              3.190
> smartcar            3.242
> rdma                3.570
> jemalloc            3.584
> glusterfs           3.597
> sdl                 3.677
> libusb              3.758
> libssh2             3.906
> tcmalloc            3.959
> xen                 4.151
> xen-pci-passthrough 4.167
> curl                5.245
> spice               5.289
> gnutls              5.400
> gtk                 8.782
> vte                 9.764
> rbd                 44.315
> 

Right around here is probably a pretty good place to include the
patchset changelog.

For larger series like this, you *may* also want to call on a script
written by Jeff Cody[1] to produce a patchset changelog. It works pretty
well in conjunction with git-publish[2].

(Not everybody does this. In fact, mostly just Max and I do. (You still
need at least a hand-written changelog, though.))

Using it is at your discretion, it can be useful at times.
Take a look at Max's series "[Qemu-devel] [PATCH v3 for-2.7 00/14] qapi:
Allow blockdev-add for NBD" for a good example [3].

the script git-backport-diff is usually used to check differences in a
series between an upstream and downstream version of a series, but it
can also be used to compare a v3 against a v2.

> Colin Lord (30):
>   blockdev: prepare iSCSI block driver for dynamic loading
>   blockdev: Move bochs probe into separate file
>   blockdev: Move cloop probe to its own file
>   blockdev: Move luks probe to its own file
>   blockdev: Move dmg probe to its own file
>   blockdev: Move parallels probe to its own file
>   blockdev: Move qcow probe to its own file
>   blockdev: Move qcow2 probe to its own file
>   blockdev: Move qed probe to its own file
>   blockdev: Move raw probe to its own file
>   blockdev: Move vdi probe to its own file
>   blockdev: Move vhdx probe to its own file
>   blockdev: Move vmdk probe to its own file
>   blockdev: Move vpc probe to its own file
>   blockdev: Separate bochs probe from its driver
>   blockdev: Separate cloop probe from its driver
>   blockdev: Separate luks probe from its driver
>   blockdev: Separate dmg probe from its driver
>   blockdev: Separate parallels probe from its driver
>   blockdev: Separate qcow probe from its driver
>   blockdev: Separate qcow2 probe from its driver
>   blockdev: Separate qed probe from its driver
>   blockdev: Separate raw probe from its driver
>   blockdev: Separate vdi probe from its driver
>   blockdev: Separate vhdx probe from its driver
>   blockdev: Separate vmdk probe from its driver
>   blockdev: Separate vpc probe from its driver
>   blockdev: Remove the .bdrv_probe field from BlockDrivers
>   blockdev: Separate out bdrv_probe_device functions
>   blockdev: Remove bdrv_probe_device field from BlockDriver
> 
> Marc Mari (2):
>   blockdev: Add dynamic generation of module_block.h
>   blockdev: Add dynamic module loading for block drivers
> 
>  Makefile                         |   7 ++
>  block.c                          | 181 +++++++++++++++++++++++++++++++--------
>  block/Makefile.objs              |   4 +
>  block/bochs.c                    |  56 +-----------
>  block/cloop.c                    |  17 +---
>  block/crypto.c                   |  22 +----
>  block/dmg.c                      |  17 +---
>  block/iscsi.c                    |  36 --------
>  block/parallels.c                |  44 +---------
>  block/probe/bochs.c              |  28 ++++++
>  block/probe/cloop.c              |  22 +++++
>  block/probe/dmg.c                |  22 +++++
>  block/probe/host_cdrom.c         |  47 ++++++++++
>  block/probe/host_device.c        |  42 +++++++++
>  block/probe/luks.c               |  26 ++++++
>  block/probe/parallels.c          |  26 ++++++
>  block/probe/qcow.c               |  22 +++++
>  block/probe/qcow2.c              |  22 +++++
>  block/probe/qed.c                |  24 ++++++
>  block/probe/raw.c                |  14 +++
>  block/probe/vdi.c                |  29 +++++++
>  block/probe/vhdx.c               |  27 ++++++
>  block/probe/vmdk.c               |  67 +++++++++++++++
>  block/probe/vpc.c                |  16 ++++
>  block/qcow.c                     |  32 +------
>  block/qcow2.c                    |  14 +--
>  block/qed.c                      |  16 +---
>  block/raw-posix.c                |  55 +-----------
>  block/raw-win32.c                |  11 +--
>  block/raw_bsd.c                  |  10 +--
>  block/vdi.c                      |  70 +--------------
>  block/vhdx.c                     |  21 +----
>  block/vmdk.c                     |  61 +------------
>  block/vpc.c                      |   9 +-
>  include/block/block_int.h        |   3 -
>  include/block/driver/bochs.h     |  40 +++++++++
>  include/block/driver/parallels.h |  26 ++++++
>  include/block/driver/qcow.h      |  21 +++++
>  include/block/driver/vdi.h       |  49 +++++++++++
>  include/block/driver/vmdk.h      |   7 ++
>  include/block/probe.h            |  33 +++++++
>  include/qemu/module.h            |   3 +
>  scripts/modules/module_block.py  | 108 +++++++++++++++++++++++
>  util/module.c                    |  38 +++-----
>  vl.c                             |  38 ++++++++
>  45 files changed, 948 insertions(+), 535 deletions(-)
>  create mode 100644 block/probe/bochs.c
>  create mode 100644 block/probe/cloop.c
>  create mode 100644 block/probe/dmg.c
>  create mode 100644 block/probe/host_cdrom.c
>  create mode 100644 block/probe/host_device.c
>  create mode 100644 block/probe/luks.c
>  create mode 100644 block/probe/parallels.c
>  create mode 100644 block/probe/qcow.c
>  create mode 100644 block/probe/qcow2.c
>  create mode 100644 block/probe/qed.c
>  create mode 100644 block/probe/raw.c
>  create mode 100644 block/probe/vdi.c
>  create mode 100644 block/probe/vhdx.c
>  create mode 100644 block/probe/vmdk.c
>  create mode 100644 block/probe/vpc.c
>  create mode 100644 include/block/driver/bochs.h
>  create mode 100644 include/block/driver/parallels.h
>  create mode 100644 include/block/driver/qcow.h
>  create mode 100644 include/block/driver/vdi.h
>  create mode 100644 include/block/driver/vmdk.h
>  create mode 100644 include/block/probe.h
>  create mode 100644 scripts/modules/module_block.py
> 

[1] https://github.com/codyprime/git-scripts
[2] https://github.com/stefanha/git-publish
[3] https://lists.gnu.org/archive/html/qemu-devel/2016-04/msg01048.html