Re: [PATCH] migration: add FEATURE_SEEKABLE to QIOChannelBlock

[Fabiano Rosas <farosas@suse.de>]

"Marco Cavenati" <Marco.Cavenati@eurecom.fr> writes:

> On Friday, April 11, 2025 14:24 CEST, Fabiano Rosas <farosas@suse.de> wrote:
>
>> > If bitmap 0 implies zero page, we could call `ram_handle_zero`
>> > in `read_ramblock_mapped_ram` for the clear bits.
>> > Or do you fear this might be unnecessary expensive for migration?
>> 
>> Yes, unfortunately the peformance difference is noticeable. But we could
>> have a slightly different algorithm for savevm. At this point it might
>> be easier to just duplicate read_ramblock_mapped_ram(), check for savevm
>> in there and see what that the resulting code looks like.
>
> I tried to get some numbers in a "bad case" scenario restoring a
> clean, fully booted, idle Debian VM with 4GB of ram. The zero pages
> are ~90%. I'm using a nvme ssd to store the snapshot and I repeated
> the restore 10 times with and without zeroing (`ram_handle_zero`).
> The restore takes on average +25% of time.
> (It's not a broad nor deep investigation.)
>
> So, I see your point on performance, but I'm not fully comfortable
> with the difference in zero page handling between mapped-ram on
> and mapped-ram off. In the former case zero pages are skipped, while
> in the latter they are explicitly zeroed.
> Enabling mapped-ram has the implicit effect to also skip zero-pages.
> I think it is an optimization not really bound to mapped-ram and it
> could be better to have this feature external to mapped-ram, enabled
> when the destination ram is known to be already zeroed (also for
> mapped-ram off ideally).
>

From a design perspective that makes sense. The only use case currently
would be mapped-ram _migration_ (as opposed to snapshot) because
non-mapped-ram migration is usually done live. The stream of pages will
potentially have several versions of the same page, therefore we need to
clear it when it's a zero page.

We'd benefit from a separate "don't load zero pages" option only when
the VM is guaranteed to be stopped and more importantly, not allowed to
be unstopped. This is the tricky part. We have experimented with and
dropped the idea of detecting a stopped-vm-migration for mapped-ram (the
idea back then was not to do better zero page handling, but skip dirty
tracking altogether).

Since we're dealing with snapshots here, which are asynchronous, I'm
wondering wheter it would make sense to extend the zero-page-detection
option to the destination. Then we could bind the loadvm process to
zero-page-detection=none because I don't think we can safely ignore them
anyway.

>> By the way, what's your overall goal with enabling the feature? Do you
>> intent to enable further capabilities for snapshot? Specifically
>> multifd. I belive the zero page skip is responsible for most of the
>> performance gains for mapped-ram without direct-io and multifd. The
>> benefit of bounded stream size doesn't apply to snapshots because
>> they're not live.
>
> My overall goal is a hot-loadvm feature that currently lives in a fork
> downstream and has a long way before getting in a mergeable state :)

Cool. Don't hesitate to send stuff our way, the sooner and more often we
discuss this, the better the chances of getting it merged upstream.

Do you use libvirt at all? Mapped-ram support has been added to it in
the latest version. The original use-case for mapped-ram was quick
snapshot saving and loading after all. Libvirt does it in a way that
does not use savevm, which might be helpful.

> In a nutshell, I'm using dirty page tracking to load from the snapshot
> only the pages that have been dirtied between two loadvm;
> mapped-ram is required to seek and read only the dirtied pages.

But then you'd have a bitmap of pages you could correlate with the
file_bmap and force-load whatever pages you need. It doesn't seem like
zero page handling would affect you too much in that case.