Re: [PATCH v4 11/34] migration/ram: Introduce 'fixed-ram' migration capability

["Daniel P. Berrangé" <berrange@redhat.com>]

On Wed, Feb 21, 2024 at 10:24:05AM -0300, Fabiano Rosas wrote:
> Markus Armbruster <armbru@redhat.com> writes:
> 
> > Fabiano Rosas <farosas@suse.de> writes:
> >
> >> Add a new migration capability 'fixed-ram'.
> >>
> >> The core of the feature is to ensure that each RAM page has a specific
> >> offset in the resulting migration stream. The reasons why we'd want
> >> such behavior are:
> >>
> >>  - The resulting file will have a bounded size, since pages which are
> >>    dirtied multiple times will always go to a fixed location in the
> >>    file, rather than constantly being added to a sequential
> >>    stream. This eliminates cases where a VM with, say, 1G of RAM can
> >>    result in a migration file that's 10s of GBs, provided that the
> >>    workload constantly redirties memory.
> >>
> >>  - It paves the way to implement O_DIRECT-enabled save/restore of the
> >>    migration stream as the pages are ensured to be written at aligned
> >>    offsets.
> >>
> >>  - It allows the usage of multifd so we can write RAM pages to the
> >>    migration file in parallel.
> >>
> >> For now, enabling the capability has no effect. The next couple of
> >> patches implement the core functionality.
> >>
> >> Signed-off-by: Fabiano Rosas <farosas@suse.de>
> >
> > [...]
> >
> >> diff --git a/qapi/migration.json b/qapi/migration.json
> >> index 5a565d9b8d..3fce5fe53e 100644
> >> --- a/qapi/migration.json
> >> +++ b/qapi/migration.json
> >> @@ -531,6 +531,10 @@
> >>  #     and can result in more stable read performance.  Requires KVM
> >>  #     with accelerator property "dirty-ring-size" set.  (Since 8.1)
> >>  #
> >> +# @fixed-ram: Migrate using fixed offsets in the migration file for
> >> +#     each RAM page.  Requires a migration URI that supports seeking,
> >> +#     such as a file.  (since 9.0)
> >> +#
> >>  # Features:
> >>  #
> >>  # @deprecated: Member @block is deprecated.  Use blockdev-mirror with
> >> @@ -555,7 +559,7 @@
> >>             { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] },
> >>             'validate-uuid', 'background-snapshot',
> >>             'zero-copy-send', 'postcopy-preempt', 'switchover-ack',
> >> -           'dirty-limit'] }
> >> +           'dirty-limit', 'fixed-ram'] }
> >>  
> >>  ##
> >>  # @MigrationCapabilityStatus:
> >
> > Can we find a better name than @fixed-ram?  @fixed-ram-offsets?
> > @use-seek?
> 
> I have no idea how we came to fixed-ram. The archives don't provide any
> clarification. I find it confusing at first glance as well.
> 
> A little brainstorming on how fixed-ram is different from exiting
> migration:
> 
> Fixed-ram:
>   uses a file, like the 'file:' migration;
> 
>   needs a seeking medium, such as a file;
> 
>   migrates ram by placing a page always in the same offset in the
>   file, contrary to normal migration which streams the page changes
>   continuously;
> 
>   ensures a migration file of size bounded to VM RAM size, contrary to
>   normal 'file:' migration which creates a file with unbounded size;
> 
>   enables multi-threaded RAM migration, even though we only use it when
>   multifd is enabled;
> 
>   uses scatter-gatter APIs (pwritev, preadv);
> 
> So a few options:
> 
> (disconsidering use-seek, it might be even more generic/vague)
> 
> - fixed-ram-offsets
> - non-streaming (or streaming: false)
> - ram-scatter-gather (ram-sg)
> - parallel-ram (even with the slight inaccuracy that we sometimes do it single-threaded)

I could add 'mapped-ram', as an alternative to 'fixed-ram'.

The key distinguishing & motivating feature here is that
RAM regions are mapped directly to file regions, instead
of just being streamed at arbitrary points.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|