Re: [PATCH 00/23] Migration: Transmit and detect zero pages in the multifd threads

[Juan Quintela <quintela@redhat.com>]

Juan Quintela <quintela@redhat.com> wrote:
> Hi

Nack myself.

For some reasons my mail server decided to only sent part of the series.

Sorry, Juan.


>
> Since Friday version:
> - More cleanups on the code
> - Remove repeated calls to qemu_target_page_size()
> - Establish normal pages and zero pages
> - detect zero pages on the multifd threads
> - send zero pages through the multifd channels.
> - reviews by Richard addressed.
>
> It pases migration-test, so it should be perfect O:+)
>
> ToDo for next version:
> - check the version changes
>   I need that 6.2 is out to check for 7.0.
>   This code don't exist at all due to that reason.
> - Send measurements of the differences
>
> Please, review.
>
> [
>
> Friday version that just created a single writev instead of
> write+writev.
>
> ]
>
> Right now, multifd does a write() for the header and a writev() for
> each group of pages.  Simplify it so we send the header as another
> member of the IOV.
>
> Once there, I got several simplifications:
> * is_zero_range() was used only once, just use its body.
> * same with is_zero_page().
> * Be consintent and use offset insed the ramblock everywhere.
> * Now that we have the offsets of the ramblock, we can drop the iov.
> * Now that nothing uses iov's except NOCOMP method, move the iovs
>   from pages to methods.
> * Now we can use iov's with a single field for zlib/zstd.
> * send_write() method is the same in all the implementaitons, so use
>   it directly.
> * Now, we can use a single writev() to write everything.
>
> ToDo: Move zero page detection to the multifd thrteads.
>
> With RAM in the Terabytes size, the detection of the zero page takes
> too much time on the main thread.
>
> Last patch on the series removes the detection of zero pages in the
> main thread for multifd.  In the next series post, I will add how to
> detect the zero pages and send them on multifd channels.
>
> Please review.
>
> Later, Juan.
>
> Juan Quintela (23):
>   multifd: Delete useless operation
>   migration: Never call twice qemu_target_page_size()
>   multifd: Rename used field to num
>   multifd: Add missing documention
>   multifd: The variable is only used inside the loop
>   multifd: remove used parameter from send_prepare() method
>   multifd: remove used parameter from send_recv_pages() method
>   multifd: Fill offset and block for reception
>   multifd: Make zstd compression method not use iovs
>   multifd: Make zlib compression method not use iovs
>   multifd: Move iov from pages to params
>   multifd: Make zlib use iov's
>   multifd: Make zstd use iov's
>   multifd: Remove send_write() method
>   multifd: Use a single writev on the send side
>   multifd: Unfold "used" variable by its value
>   multifd: Use normal pages array on the send side
>   multifd: Use normal pages array on the recv side
>   multifd: recv side only needs the RAMBlock host address
>   multifd: Rename pages_used to normal_pages
>   multifd: Support for zero pages transmission
>   multifd: Zero pages transmission
>   migration: Use multifd before we check for the zero page
>
>  migration/multifd.h      |  52 +++++++---
>  migration/migration.c    |   7 +-
>  migration/multifd-zlib.c |  71 +++++--------
>  migration/multifd-zstd.c |  70 +++++--------
>  migration/multifd.c      | 214 +++++++++++++++++++++++----------------
>  migration/ram.c          |  22 ++--
>  migration/savevm.c       |   5 +-
>  migration/trace-events   |   4 +-
>  8 files changed, 231 insertions(+), 214 deletions(-)