Re: [PATCH 07/10] docs/migration: Split "dirty limit"

["Cédric Le Goater" <clg@redhat.com>]

On 1/9/24 07:46, peterx@redhat.com wrote:
> From: Peter Xu <peterx@redhat.com>
> 
> Split that into a separate file, put under "features".
> 
> Cc: Yong Huang <yong.huang@smartx.com>
> Signed-off-by: Peter Xu <peterx@redhat.com>



Reviewed-by: Cédric Le Goater <clg@redhat.com>

Thanks,

C.


> ---
>   docs/devel/migration/dirty-limit.rst | 71 ++++++++++++++++++++++++++++
>   docs/devel/migration/features.rst    |  1 +
>   docs/devel/migration/main.rst        | 71 ----------------------------
>   3 files changed, 72 insertions(+), 71 deletions(-)
>   create mode 100644 docs/devel/migration/dirty-limit.rst
> 
> diff --git a/docs/devel/migration/dirty-limit.rst b/docs/devel/migration/dirty-limit.rst
> new file mode 100644
> index 0000000000..8f32329d5f
> --- /dev/null
> +++ b/docs/devel/migration/dirty-limit.rst
> @@ -0,0 +1,71 @@
> +Dirty limit
> +===========
> +
> +The dirty limit, short for dirty page rate upper limit, is a new capability
> +introduced in the 8.1 QEMU release that uses a new algorithm based on the KVM
> +dirty ring to throttle down the guest during live migration.
> +
> +The algorithm framework is as follows:
> +
> +::
> +
> +  ------------------------------------------------------------------------------
> +  main   --------------> throttle thread ------------> PREPARE(1) <--------
> +  thread  \                                                |              |
> +           \                                               |              |
> +            \                                              V              |
> +             -\                                        CALCULATE(2)       |
> +               \                                           |              |
> +                \                                          |              |
> +                 \                                         V              |
> +                  \                                    SET PENALTY(3) -----
> +                   -\                                      |
> +                     \                                     |
> +                      \                                    V
> +                       -> virtual CPU thread -------> ACCEPT PENALTY(4)
> +  ------------------------------------------------------------------------------
> +
> +When the qmp command qmp_set_vcpu_dirty_limit is called for the first time,
> +the QEMU main thread starts the throttle thread. The throttle thread, once
> +launched, executes the loop, which consists of three steps:
> +
> +  - PREPARE (1)
> +
> +     The entire work of PREPARE (1) is preparation for the second stage,
> +     CALCULATE(2), as the name implies. It involves preparing the dirty
> +     page rate value and the corresponding upper limit of the VM:
> +     The dirty page rate is calculated via the KVM dirty ring mechanism,
> +     which tells QEMU how many dirty pages a virtual CPU has had since the
> +     last KVM_EXIT_DIRTY_RING_FULL exception; The dirty page rate upper
> +     limit is specified by caller, therefore fetch it directly.
> +
> +  - CALCULATE (2)
> +
> +     Calculate a suitable sleep period for each virtual CPU, which will be
> +     used to determine the penalty for the target virtual CPU. The
> +     computation must be done carefully in order to reduce the dirty page
> +     rate progressively down to the upper limit without oscillation. To
> +     achieve this, two strategies are provided: the first is to add or
> +     subtract sleep time based on the ratio of the current dirty page rate
> +     to the limit, which is used when the current dirty page rate is far
> +     from the limit; the second is to add or subtract a fixed time when
> +     the current dirty page rate is close to the limit.
> +
> +  - SET PENALTY (3)
> +
> +     Set the sleep time for each virtual CPU that should be penalized based
> +     on the results of the calculation supplied by step CALCULATE (2).
> +
> +After completing the three above stages, the throttle thread loops back
> +to step PREPARE (1) until the dirty limit is reached.
> +
> +On the other hand, each virtual CPU thread reads the sleep duration and
> +sleeps in the path of the KVM_EXIT_DIRTY_RING_FULL exception handler, that
> +is ACCEPT PENALTY (4). Virtual CPUs tied with writing processes will
> +obviously exit to the path and get penalized, whereas virtual CPUs involved
> +with read processes will not.
> +
> +In summary, thanks to the KVM dirty ring technology, the dirty limit
> +algorithm will restrict virtual CPUs as needed to keep their dirty page
> +rate inside the limit. This leads to more steady reading performance during
> +live migration and can aid in improving large guest responsiveness.
> diff --git a/docs/devel/migration/features.rst b/docs/devel/migration/features.rst
> index 0054e0c900..e257d0d100 100644
> --- a/docs/devel/migration/features.rst
> +++ b/docs/devel/migration/features.rst
> @@ -7,3 +7,4 @@ Migration has plenty of features to support different use cases.
>      :maxdepth: 2
>   
>      postcopy
> +   dirty-limit
> diff --git a/docs/devel/migration/main.rst b/docs/devel/migration/main.rst
> index 051ea43f0e..00b9c3d32f 100644
> --- a/docs/devel/migration/main.rst
> +++ b/docs/devel/migration/main.rst
> @@ -573,74 +573,3 @@ path.
>        Return path  - opened by main thread, written by main thread AND postcopy
>        thread (protected by rp_mutex)
>   
> -Dirty limit
> -=====================
> -The dirty limit, short for dirty page rate upper limit, is a new capability
> -introduced in the 8.1 QEMU release that uses a new algorithm based on the KVM
> -dirty ring to throttle down the guest during live migration.
> -
> -The algorithm framework is as follows:
> -
> -::
> -
> -  ------------------------------------------------------------------------------
> -  main   --------------> throttle thread ------------> PREPARE(1) <--------
> -  thread  \                                                |              |
> -           \                                               |              |
> -            \                                              V              |
> -             -\                                        CALCULATE(2)       |
> -               \                                           |              |
> -                \                                          |              |
> -                 \                                         V              |
> -                  \                                    SET PENALTY(3) -----
> -                   -\                                      |
> -                     \                                     |
> -                      \                                    V
> -                       -> virtual CPU thread -------> ACCEPT PENALTY(4)
> -  ------------------------------------------------------------------------------
> -
> -When the qmp command qmp_set_vcpu_dirty_limit is called for the first time,
> -the QEMU main thread starts the throttle thread. The throttle thread, once
> -launched, executes the loop, which consists of three steps:
> -
> -  - PREPARE (1)
> -
> -     The entire work of PREPARE (1) is preparation for the second stage,
> -     CALCULATE(2), as the name implies. It involves preparing the dirty
> -     page rate value and the corresponding upper limit of the VM:
> -     The dirty page rate is calculated via the KVM dirty ring mechanism,
> -     which tells QEMU how many dirty pages a virtual CPU has had since the
> -     last KVM_EXIT_DIRTY_RING_FULL exception; The dirty page rate upper
> -     limit is specified by caller, therefore fetch it directly.
> -
> -  - CALCULATE (2)
> -
> -     Calculate a suitable sleep period for each virtual CPU, which will be
> -     used to determine the penalty for the target virtual CPU. The
> -     computation must be done carefully in order to reduce the dirty page
> -     rate progressively down to the upper limit without oscillation. To
> -     achieve this, two strategies are provided: the first is to add or
> -     subtract sleep time based on the ratio of the current dirty page rate
> -     to the limit, which is used when the current dirty page rate is far
> -     from the limit; the second is to add or subtract a fixed time when
> -     the current dirty page rate is close to the limit.
> -
> -  - SET PENALTY (3)
> -
> -     Set the sleep time for each virtual CPU that should be penalized based
> -     on the results of the calculation supplied by step CALCULATE (2).
> -
> -After completing the three above stages, the throttle thread loops back
> -to step PREPARE (1) until the dirty limit is reached.
> -
> -On the other hand, each virtual CPU thread reads the sleep duration and
> -sleeps in the path of the KVM_EXIT_DIRTY_RING_FULL exception handler, that
> -is ACCEPT PENALTY (4). Virtual CPUs tied with writing processes will
> -obviously exit to the path and get penalized, whereas virtual CPUs involved
> -with read processes will not.
> -
> -In summary, thanks to the KVM dirty ring technology, the dirty limit
> -algorithm will restrict virtual CPUs as needed to keep their dirty page
> -rate inside the limit. This leads to more steady reading performance during
> -live migration and can aid in improving large guest responsiveness.
> -