Re: [PULL 00/30] Next patches

[Stefan Hajnoczi <stefanha@gmail.com>]

On Tue, 15 Nov 2022 at 10:40, Juan Quintela <quintela@redhat.com> wrote:
>
> The following changes since commit 98f10f0e2613ba1ac2ad3f57a5174014f6dcb03d:
>
>   Merge tag 'pull-target-arm-20221114' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2022-11-14 13:31:17 -0500)
>
> are available in the Git repository at:
>
>   https://gitlab.com/juan.quintela/qemu.git tags/next-pull-request
>
> for you to fetch changes up to d896a7a40db13fc2d05828c94ddda2747530089c:
>
>   migration: Block migration comment or code is wrong (2022-11-15 10:31:06 +0100)
>
> ----------------------------------------------------------------
> Migration PULL request (take 2)
>
> Hi
>
> This time properly signed.
>
> [take 1]
> It includes:
> - Leonardo fix for zero_copy flush
> - Fiona fix for return value of readv/writev
> - Peter Xu cleanups
> - Peter Xu preempt patches
> - Patches ready from zero page (me)
> - AVX2 support (ling)
> - fix for slow networking and reordering of first packets (manish)
>
> Please, apply.
>
> ----------------------------------------------------------------
>
> Fiona Ebner (1):
>   migration/channel-block: fix return value for
>     qio_channel_block_{readv,writev}
>
> Juan Quintela (5):
>   multifd: Create page_size fields into both MultiFD{Recv,Send}Params
>   multifd: Create page_count fields into both MultiFD{Recv,Send}Params
>   migration: Export ram_transferred_ram()
>   migration: Export ram_release_page()
>   migration: Block migration comment or code is wrong
>
> Leonardo Bras (1):
>   migration/multifd/zero-copy: Create helper function for flushing
>
> Peter Xu (20):
>   migration: Fix possible infinite loop of ram save process
>   migration: Fix race on qemu_file_shutdown()
>   migration: Disallow postcopy preempt to be used with compress
>   migration: Use non-atomic ops for clear log bitmap
>   migration: Disable multifd explicitly with compression
>   migration: Take bitmap mutex when completing ram migration
>   migration: Add postcopy_preempt_active()
>   migration: Cleanup xbzrle zero page cache update logic
>   migration: Trivial cleanup save_page_header() on same block check
>   migration: Remove RAMState.f references in compression code
>   migration: Yield bitmap_mutex properly when sending/sleeping
>   migration: Use atomic ops properly for page accountings
>   migration: Teach PSS about host page
>   migration: Introduce pss_channel
>   migration: Add pss_init()
>   migration: Make PageSearchStatus part of RAMState
>   migration: Move last_sent_block into PageSearchStatus
>   migration: Send requested page directly in rp-return thread
>   migration: Remove old preempt code around state maintainance
>   migration: Drop rs->f
>
> ling xu (2):
>   Update AVX512 support for xbzrle_encode_buffer
>   Unit test code and benchmark code

This commit causes the following CI failure:

cc -m64 -mcx16 -Ilibauthz.fa.p -I. -I.. -Iqapi -Itrace -Iui/shader
-I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include
-fdiagnostics-color=auto -Wall -Winvalid-pch -Werror -std=gnu11 -O2 -g
-isystem /builds/qemu-project/qemu/linux-headers -isystem
linux-headers -iquote . -iquote /builds/qemu-project/qemu -iquote
/builds/qemu-project/qemu/include -iquote
/builds/qemu-project/qemu/tcg/i386 -pthread -U_FORTIFY_SOURCE
-D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64
-D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wundef
-Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common
-fwrapv -Wold-style-declaration -Wold-style-definition -Wtype-limits
-Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers
-Wempty-body -Wnested-externs -Wendif-labels -Wexpansion-to-defined
-Wimplicit-fallthrough=2 -Wno-missing-include-dirs
-Wno-shift-negative-value -Wno-psabi -fstack-protector-strong -fPIE
-MD -MQ libauthz.fa.p/authz_simple.c.o -MF
libauthz.fa.p/authz_simple.c.o.d -o libauthz.fa.p/authz_simple.c.o -c
../authz/simple.c
In file included from ../authz/simple.c:23:
../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file
or directory
1 | #include "trace/trace-authz.h"
| ^~~~~~~~~~~~~~~~~~~~~

https://gitlab.com/qemu-project/qemu/-/jobs/3326576115

I think the issue is that the test links against objects that aren't
present when qemu-user only build is performed. That's my first guess,
I might be wrong but it is definitely this commit that causes the
failure (I bisected it).

There is a second CI failure here:

clang -m64 -mcx16 -Itests/bench/xbzrle-bench.p -Itests/bench
-I../tests/bench -I. -Iqapi -Itrace -Iui -Iui/shader
-I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
-I/usr/include/sysprof-4 -flto -fcolor-diagnostics -Wall -Winvalid-pch
-Werror -std=gnu11 -O2 -g -isystem
/builds/qemu-project/qemu/linux-headers -isystem linux-headers -iquote
. -iquote /builds/qemu-project/qemu -iquote
/builds/qemu-project/qemu/include -iquote
/builds/qemu-project/qemu/tcg/i386 -pthread -D_GNU_SOURCE
-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes
-Wredundant-decls -Wundef -Wwrite-strings -Wmissing-prototypes
-fno-strict-aliasing -fno-common -fwrapv -Wold-style-definition
-Wtype-limits -Wformat-security -Wformat-y2k -Winit-self
-Wignored-qualifiers -Wempty-body -Wnested-externs -Wendif-labels
-Wexpansion-to-defined -Wno-initializer-overrides
-Wno-missing-include-dirs -Wno-shift-negative-value
-Wno-string-plus-int -Wno-typedef-redefinition
-Wno-tautological-type-limit-compare -Wno-psabi
-Wno-gnu-variable-sized-type-not-at-end -fstack-protector-strong
-fsanitize=safe-stack -fsanitize=cfi-icall
-fsanitize-cfi-icall-generalize-pointers -fno-sanitize-trap=cfi-icall
-fPIE -MD -MQ tests/bench/xbzrle-bench.p/xbzrle-bench.c.o -MF
tests/bench/xbzrle-bench.p/xbzrle-bench.c.o.d -o
tests/bench/xbzrle-bench.p/xbzrle-bench.c.o -c
../tests/bench/xbzrle-bench.c
../tests/bench/xbzrle-bench.c:84:15: error: implicit declaration of
function 'xbzrle_encode_buffer_avx512' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
dlen512 = xbzrle_encode_buffer_avx512(buffer512, buffer512, XBZRLE_PAGE_SIZE,
^
../tests/bench/xbzrle-bench.c:84:15: note: did you mean 'xbzrle_encode_buffer'?
./../migration/xbzrle.h:17:5: note: 'xbzrle_encode_buffer' declared here
int xbzrle_encode_buffer(uint8_t *old_buf, uint8_t *new_buf, int slen,
^
../tests/bench/xbzrle-bench.c:146:15: error: implicit declaration of
function 'xbzrle_encode_buffer_avx512' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
dlen512 = xbzrle_encode_buffer_avx512(test512, test512, XBZRLE_PAGE_SIZE,
^
../tests/bench/xbzrle-bench.c:205:15: error: implicit declaration of
function 'xbzrle_encode_buffer_avx512' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
dlen512 = xbzrle_encode_buffer_avx512(buffer512, test512, XBZRLE_PAGE_SIZE,
^
../tests/bench/xbzrle-bench.c:268:13: error: implicit declaration of
function 'xbzrle_encode_buffer_avx512' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
rc512 = xbzrle_encode_buffer_avx512(buffer512, test512, XBZRLE_PAGE_SIZE,
^
../tests/bench/xbzrle-bench.c:345:15: error: implicit declaration of
function 'xbzrle_encode_buffer_avx512' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
dlen512 = xbzrle_encode_buffer_avx512(test512, buffer512, XBZRLE_PAGE_SIZE,
^
../tests/bench/xbzrle-bench.c:414:15: error: implicit declaration of
function 'xbzrle_encode_buffer_avx512' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
dlen512 = xbzrle_encode_buffer_avx512(test512, buffer512, XBZRLE_PAGE_SIZE,
^

https://gitlab.com/qemu-project/qemu/-/jobs/3326576144

>
> manish.mishra (1):
>   migration: check magic value for deciding the mapping of channels
>
>  meson.build                   |  16 +
>  include/exec/ram_addr.h       |  11 +-
>  include/exec/ramblock.h       |   3 +
>  include/io/channel.h          |  25 ++
>  include/qemu/bitmap.h         |   1 +
>  migration/migration.h         |   7 -
>  migration/multifd.h           |  10 +-
>  migration/postcopy-ram.h      |   2 +-
>  migration/ram.h               |  23 +
>  migration/xbzrle.h            |   4 +
>  io/channel-socket.c           |  27 ++
>  io/channel.c                  |  39 ++
>  migration/block.c             |   4 +-
>  migration/channel-block.c     |   6 +-
>  migration/migration.c         | 109 +++--
>  migration/multifd-zlib.c      |  14 +-
>  migration/multifd-zstd.c      |  12 +-
>  migration/multifd.c           |  69 +--
>  migration/postcopy-ram.c      |   5 +-
>  migration/qemu-file.c         |  27 +-
>  migration/ram.c               | 794 +++++++++++++++++-----------------
>  migration/xbzrle.c            | 124 ++++++
>  tests/bench/xbzrle-bench.c    | 465 ++++++++++++++++++++
>  tests/unit/test-xbzrle.c      |  39 +-
>  util/bitmap.c                 |  45 ++
>  meson_options.txt             |   2 +
>  scripts/meson-buildoptions.sh |  14 +-
>  tests/bench/meson.build       |   4 +
>  28 files changed, 1379 insertions(+), 522 deletions(-)
>  create mode 100644 tests/bench/xbzrle-bench.c
>
> --
> 2.38.1
>
>