Re: [PATCH v3] migration/rdma: add x-rdma-chunk-size parameter

[Markus Armbruster <armbru@redhat.com>]

"Zhang, GuoQing (Sam)" <guoqzhan@amd.com> writes:

> On 2026/3/27 17:45, Markus Armbruster wrote:
>> Samuel Zhang <guoqing.zhang@amd.com> writes:
>>
>>> The default 1MB RDMA chunk size causes slow live migration because
>>> each chunk triggers a write_flush (ibv_post_send). For 8GB RAM,
>>> 1MB chunk size produces ~15000 flushes vs ~3700 with 1024MB chunk size.
>>>
>>> Add x-rdma-chunk-size parameter to configure the RDMA chunk size for
>>> faster migration.
>>> Usage: `migrate_set_parameter x-rdma-chunk-size 1024M`
>>>
>>> Performance with RDMA live migration of 8GB RAM VM:
>>>
>>> | x-rdma-chunk-size (B) | time (s) | throughput (MB/s) |
>>> |-----------------------|----------|-------------------|
>>> | 1M (default)          | 37.915   |  1,007            |
>>> | 32M                   | 17.880   |  2,260            |
>>> | 1024M                 |  4.368   | 17,529            |
>>>
>>> Signed-off-by: Samuel Zhang <guoqing.zhang@amd.com>

[...]

>>> diff --git a/qapi/migration.json b/qapi/migration.json
>>> index 7134d4ce47..292d96c95a 100644
>>> --- a/qapi/migration.json
>>> +++ b/qapi/migration.json
>>> @@ -806,7 +806,7 @@
>>>  #
>>>  # Features:
>>>  #
>>> -# @unstable: Members @x-checkpoint-delay and
>>> +# @unstable: Members @x-checkpoint-delay, @x-rdma-chunk-size, and
>>>  #     @x-vcpu-dirty-limit-period are experimental.
>>>  #
>>>  # Since: 2.4
>>> @@ -831,6 +831,7 @@
>>>             'mode',
>>>             'zero-page-detection',
>>>             'direct-io',
>>> +           { 'name': 'x-rdma-chunk-size', 'features': [ 'unstable' ] },
>>>             'cpr-exec-command'] }
>>>  
>>>  ##
>>> @@ -1007,9 +1008,14 @@
>>>  #     is @cpr-exec.  The first list element is the program's filename,
>>>  #     the remainder its arguments.  (Since 10.2)
>>>  #
>>> +# @x-rdma-chunk-size: RDMA memory registration chunk size in bytes.
>>> +#     Default is 1MiB.  Must be a power of 2 in the range
>>> +#     [1MiB, 1024MiB].  Only takes effect for RDMA migration.
>>
>> I believe it applies to channels whose migration address type is is
>> "rdma".  In MigrationChannel syntax
>>
>>      {"channel-type": ..., "addr": {"transport": "rdma", ...}}
>>
>> Correct?
>
>
> Correct! Is it OK to update the doc to the following one? Thank you!
>
> `Only takes effect when migration address transport is "rdma".`

The phrasing in the patch feels okay as is.  Perhaps "Only applies when
migrating via RDMA" to more closely match the description of
MigrationAddressType @rdma: Migrate via RDMA.

Either way, QAPI schema
Acked-by: Markus Armbruster <armbru@redhat.com>

>>> +#     (Since 11.1)
>>> +#
>>>  # Features:
>>>  #
>>> -# @unstable: Members @x-checkpoint-delay and
>>> +# @unstable: Members @x-checkpoint-delay, @x-rdma-chunk-size, and
>>>  #     @x-vcpu-dirty-limit-period are experimental.
>>>  #
>>>  # Since: 2.4
>>> @@ -1046,6 +1052,8 @@
>>>              '*mode': 'MigMode',
>>>              '*zero-page-detection': 'ZeroPageDetection',
>>>              '*direct-io': 'bool',
>>> +            '*x-rdma-chunk-size': { 'type': 'uint64',
>>> +                                    'features': [ 'unstable' ] },
>>>              '*cpr-exec-command': [ 'str' ]} }
>>  
>>>  ##