Re: [Qemu-devel] [PATCH v2 00/12] Portable thread-pool/AIO, Win32 emulated AIO

[Stefan Weil <sw@weilnetz.de>]

Am 11.08.2012 21:27, schrieb Stefan Weil:
> Am 07.08.2012 13:17, schrieb Paolo Bonzini:
>> This patch series is part 2 in my EventNotifier/AIO improvements
>> for QEMU 1.2.  It extends use of EventNotifier to the main loop
>> and AIO subsystems.  A new API using EventNotifier is added to aio.c
>> and a new portable thread pool is introduced (based on code from
>> posix-aio-compat.c, mostly) that uses this API.  raw-posix.c is
>> converted to use the new thread pool, and support for asynchronous
>> I/O is finally added to Win32 as well.
>>
>> The network drivers (curl, libiscsi, nbd) have to be disabled
>> under Windows.  They are unlikely to have any users, since they
>> were broken until 1.0 and (unlike slirp) we never had any report.
>>
>> I tested this under Wine, with a RHEL virtual machine booting just as
>> glacially as before.  "info blockstats" does show a slightly higher
>> overhead, so I would like this to be tested on real Windows hosts.
>> Even if the result is negative, I would prefer to keep the early
>> parts (i.e. drop only the last patch) since they are a prerequisite for
>> improvements to block/raw-posix.c (such as asynchronous discard
>> support) scheduled for 1.3.  The platform independent APIs introduced
>> by patches 4 and 5 are also useful for native AIO on Win32.
>>
>>
>> Paolo Bonzini (12):
>>    event_notifier: enable it to use pipes
>>    event_notifier: add Win32 implementation
>>    main-loop: use event notifiers
>>    aio: provide platform-independent API
>>    aio: add Win32 implementation
>>    linux-aio: use event notifiers
>>    qemu-thread: add QemuSemaphore
>>    aio: add generic thread-pool facility
>>    block: switch posix-aio-compat to threadpool
>>    raw: merge posix-aio-compat.c into block/raw-posix.c
>>    raw-posix: rename raw-posix-aio.h, hide unavailable prototypes
>>    raw-win32: add emulated AIO support
>>
>>   Makefile.objs                        |  12 +-
>>   aio.c => aio-posix.c                 |   9 +
>>   aio-win32.c                          | 177 +++++++++
>>   block/Makefile.objs                  |   6 +-
>>   block/{raw-posix-aio.h => raw-aio.h} |  19 +-
>>   block/raw-posix.c                    | 297 ++++++++++++++-
>>   block/raw-win32.c                    | 187 +++++++---
>>   event_notifier-posix.c               | 118 ++++++
>>   event_notifier-win32.c               |  59 +++
>>   event_notifier.c                     |  67 ----
>>   event_notifier.h                     |  20 +-
>>   linux-aio.c                          |  51 ++-
>>   main-loop.c                          | 106 +-----
>>   oslib-posix.c                        |  31 --
>>   posix-aio-compat.c                   | 679 
>> -----------------------------------
>>   qemu-aio.h                           |  19 +-
>>   qemu-common.h                        |   1 -
>>   qemu-thread-posix.c                  |  74 ++++
>>   qemu-thread-posix.h                  |   5 +
>>   qemu-thread-win32.c                  |  35 ++
>>   qemu-thread-win32.h                  |   4 +
>>   qemu-thread.h                        |   7 +
>>   thread-pool.c                        | 279 ++++++++++++++
>>   thread-pool.h                        |  34 ++
>>   trace-events                         |   5 +
>>   25 file modificati, 1323 inserzioni(+), 978 rimozioni(-)
>>   rename aio.c => aio-posix.c (92%)
>>   create mode 100644 aio-win32.c
>>   rename block/{raw-posix-aio.h => raw-aio.h} (62%)
>>   create mode 100644 event_notifier-posix.c
>>   create mode 100644 event_notifier-win32.c
>>   delete mode 100644 event_notifier.c
>>   delete mode 100644 posix-aio-compat.c
>>   create mode 100644 thread-pool.c
>>   create mode 100644 thread-pool.h
>>
>
> Hi,
>
> I needed these changes for compilation with MinGW32:
>
> diff --git a/block/raw-posix.c b/block/raw-posix.c
> index ad65604..c948f97 100644
> --- a/block/raw-posix.c
> +++ b/block/raw-posix.c
> @@ -280,8 +280,10 @@ static int raw_open_common(BlockDriverState *bs, 
> const char *filename,
>
>      return 0;
>
> +#ifdef CONFIG_LINUX_AIO
>  out_free_buf:
>      qemu_vfree(s->aligned_buf);
> +#endif
>  out_close:
>      close(fd);
>      return -errno;
> diff --git a/block/raw-win32.c b/block/raw-win32.c
> index 7a5b86f..ef5e200 100644
> --- a/block/raw-win32.c
> +++ b/block/raw-win32.c
> @@ -42,7 +42,7 @@ typedef struct RawWin32AIOData {
>      struct iovec *aio_iov;
>      int aio_niov;
>      size_t aio_nbytes;
> -    off_t aio_offset;
> +    off64_t aio_offset;
>      int aio_type;
>  } RawWin32AIOData;
>
>
> Label out_free_buf is only used with CONFIG_LINUX_AIO.
>
> off_t is always 32 bit with MinGW header files, but we
> want support for large files, so off64_t is a better choice
> here. It also avoids a compiler warning when the value
> is right shifted by 32 bits.
>
> I noticed some messages from checkpatch.pl.
> Some (but not all) of them were caused by the renamed file.
> Maybe this file should be fixed before it is renamed.
>
> Regards,
> Stefan
>


More changes are needed for user mode emulation.
I hope they don't break linking of other executables:

diff --git a/Makefile.objs b/Makefile.objs
index afefa02..bbe5a56 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -20,8 +20,8 @@ universal-obj-y += $(qom-obj-y)
  #######################################################################
  # oslib-obj-y is code depending on the OS (win32 vs posix)
  oslib-obj-y = osdep.o
-oslib-obj-$(CONFIG_WIN32) += oslib-win32.o qemu-thread-win32.o 
event_notifier-win32.o
-oslib-obj-$(CONFIG_POSIX) += oslib-posix.o qemu-thread-posix.o 
event_notifier-posix.o
+oslib-obj-$(CONFIG_WIN32) += oslib-win32.o qemu-thread-win32.o
+oslib-obj-$(CONFIG_POSIX) += oslib-posix.o qemu-thread-posix.o

  #######################################################################
  # coroutines
@@ -223,8 +223,9 @@ universal-obj-y += $(qapi-obj-y)
  # guest agent

  qga-obj-y = qga/ qemu-ga.o module.o
-qga-obj-$(CONFIG_WIN32) += oslib-win32.o
-qga-obj-$(CONFIG_POSIX) += oslib-posix.o qemu-sockets.o qemu-option.o
+
+qga-obj-$(CONFIG_WIN32) += oslib-win32.o event_notifier-win32.o
+qga-obj-$(CONFIG_POSIX) += oslib-posix.o event_notifier-posix.o 
qemu-sockets.o qemu-option.o

  vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)