From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4AF45C83F2C for ; Mon, 4 Sep 2023 13:29:58 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qd9eK-0007of-AT; Mon, 04 Sep 2023 09:29:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qd9eA-0007g5-V8 for qemu-devel@nongnu.org; Mon, 04 Sep 2023 09:29:43 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qd9e6-0000k1-9N for qemu-devel@nongnu.org; Mon, 04 Sep 2023 09:29:42 -0400 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-68c3b9f8333so897414b3a.1 for ; Mon, 04 Sep 2023 06:27:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1693834054; x=1694438854; darn=nongnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=IFhGCBAKg+gHQnVyRgyu1urOe/739zifFBUmXWMt6Pw=; b=dmGmVsGz/RLZuIJpDj+7XXdT+U1xcriLsOOgvgs6OIFFqADwaLjRS/TiTlHKsf4pd9 R6fmSdvNoIBz4eaUbf2H0z1MdVvXaFNxi3tRaC7A/NxvWdc0mUT50Q1jv0dpVDHK9JQR HKXh0YzT83sh+6zr8q0jjZ6cEMP2K50ve7fEqesLvyMQeoHsI+/WrgZ+jO1t6QSh5I/S sskb8930Yg4IPxhCPyr9Vh3cXEVfAn5NuDwKzzy7zNxPr8hA9WV8FOF93D78cW0MlYiI Rl7UPoD1fvI7JmIJio/vXavkk4UnKosgKfFTdg01tGAVVeffMHyHQ/0r5hpHkRxWMlIG vrqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693834054; x=1694438854; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=IFhGCBAKg+gHQnVyRgyu1urOe/739zifFBUmXWMt6Pw=; b=hfu7VEp2LmRy6gcfb098oH8Br4eUxZgA3rDbx5LuEaQBZyQRMdum6SMX7kby84mPo/ w6uKaVYG+a+u4hkRmb8O2QRNPCNvvGvq6N4LDRAM5C/C3taSzqPdi7FB60dZZTDRCJRF qlKCkOft4qBZOgCEWMMyr+SY7xY1zB33b+UAuX6oHuEBKUvONSE/3utnph/Wn6EBlXEp fxajxnDTwIYVzMKZCCX+C/Kjf47i1o7HKEs/Os1DhGSgdcYMimMNzjZ1xvE97jkOHrmi K61iADl5J0WOIYi2oAp0Yz/09bzeAZ0/UKJgWNvel5wvnR44+eolZgeA2jIoXf04TD61 WVbA== X-Gm-Message-State: AOJu0YxJT0iH98E4L42aGhR/RvW4kkRllv+RdKhvSbMa9UjwWifW7coX LyTJuooQmeMfak9iPgACSg4nxaHL52FVJC/t0TuKVQ== X-Google-Smtp-Source: AGHT+IELx6AVspEtU/nkTmyh/aax8A7afGSzoN300asYs1akLG/K1tnBioydAYtUEpnUv2/M5Rc/7k6/v5FgTrRNM6M= X-Received: by 2002:a05:6a00:22c1:b0:68b:a137:373d with SMTP id f1-20020a056a0022c100b0068ba137373dmr14258320pfj.17.1693834054162; Mon, 04 Sep 2023 06:27:34 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Yong Huang Date: Mon, 4 Sep 2023 21:27:17 +0800 Message-ID: Subject: Re: [PATCH] softmmu/dirtylimit: Fix usleep early return on signal To: alloc.young@outlook.com Cc: qemu-devel@nongnu.org Content-Type: multipart/alternative; boundary="0000000000001b72800604887901" Received-SPF: none client-ip=2607:f8b0:4864:20::434; envelope-from=yong.huang@smartx.com; helo=mail-pf1-x434.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org --0000000000001b72800604887901 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Sep 1, 2023 at 10:19=E2=80=AFAM wrote: > From: alloc > > Timeout functions like usleep can return early on signal, which reduces > more dirty pages than expected. In dirtylimit case, dirtyrate meter > thread needs to kick all vcpus out to sync. The callchain: > > vcpu_calculate_dirtyrate > global_dirty_log_sync > memory_global_dirty_log_sync > kvm_log_sync_global > kvm_dirty_ring_flush > kvm_cpu_synchronize_kick_all <---- send vcpu signal > > For long time sleep, use qemu_cond_timedwait_iothread to handle cpu stop > event. > > The Dirty Limit algorithm seeks to keep the vCPU dirty page rate within the set limit; since it focuses more emphasis on processing time and precision, I feel that improvement should strive for the same result. Could you please provide the final test results showing the impact of that improvement? > Signed-off-by: alloc > --- > softmmu/dirtylimit.c | 19 +++++++++++++++++-- > 1 file changed, 17 insertions(+), 2 deletions(-) > > diff --git a/softmmu/dirtylimit.c b/softmmu/dirtylimit.c > index fa959d7743..ee938c636d 100644 > --- a/softmmu/dirtylimit.c > +++ b/softmmu/dirtylimit.c > @@ -411,13 +411,28 @@ void dirtylimit_set_all(uint64_t quota, > > void dirtylimit_vcpu_execute(CPUState *cpu) > { > + int64_t sleep_us, endtime_us; > + > + dirtylimit_state_lock(); > if (dirtylimit_in_service() && > dirtylimit_vcpu_get_state(cpu->cpu_index)->enabled && > cpu->throttle_us_per_full) { > trace_dirtylimit_vcpu_execute(cpu->cpu_index, > cpu->throttle_us_per_full); > - usleep(cpu->throttle_us_per_full); > - } > + sleep_us =3D cpu->throttle_us_per_full; > + dirtylimit_state_unlock(); > + endtime_us =3D qemu_clock_get_us(QEMU_CLOCK_REALTIME) + sleep_us= ; > + while (sleep_us > 0 && !cpu->stop) { > + if (sleep_us > SCALE_US) { > + qemu_mutex_lock_iothread(); > + qemu_cond_timedwait_iothread(cpu->halt_cond, sleep_us / > SCALE_US); > + qemu_mutex_unlock_iothread(); > + } else > + g_usleep(sleep_us); > + sleep_us =3D endtime_us - > qemu_clock_get_us(QEMU_CLOCK_REALTIME); > + } > + } else > + dirtylimit_state_unlock(); > } > > static void dirtylimit_init(void) > -- > 2.39.3 > > --=20 Best regards --0000000000001b72800604887901 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Fri, Sep 1, 202= 3 at 10:19=E2=80=AFAM <alloc.= young@outlook.com> wrote:
From: alloc <= alloc.young@ou= tlook.com>

Timeout functions like usleep can return early on signal, which reduces
more dirty pages than expected. In dirtylimit case, dirtyrate meter
thread needs to kick all vcpus out to sync. The callchain:

vcpu_calculate_dirtyrate
=C2=A0 =C2=A0 global_dirty_log_sync
=C2=A0 =C2=A0 =C2=A0 =C2=A0 memory_global_dirty_log_sync
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 kvm_log_sync_global
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 kvm_dirty_ring_flus= h
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 kvm_c= pu_synchronize_kick_all <---- send vcpu signal

For long time sleep, use qemu_cond_timedwait_iothread to handle cpu stop event.


The Dirty L= imit algorithm seeks to keep the vCPU dirty page rate within
the set limit; since it focuses more emphasis on processing time and
precision, I feel that improvement should strive for the sa= me result.
Could you please provide the final test result= s showing the impact of
that improvement?
=C2= =A0
Signed-off-by: alloc <alloc.young@outlook.com>
---
=C2=A0softmmu/dirtylimit.c | 19 +++++++++++++++++--
=C2=A01 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/softmmu/dirtylimit.c b/softmmu/dirtylimit.c
index fa959d7743..ee938c636d 100644
--- a/softmmu/dirtylimit.c
+++ b/softmmu/dirtylimit.c
@@ -411,13 +411,28 @@ void dirtylimit_set_all(uint64_t quota,

=C2=A0void dirtylimit_vcpu_execute(CPUState *cpu)
=C2=A0{
+=C2=A0 =C2=A0 int64_t sleep_us, endtime_us;
+
+=C2=A0 =C2=A0 dirtylimit_state_lock();
=C2=A0 =C2=A0 =C2=A0if (dirtylimit_in_service() &&
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0dirtylimit_vcpu_get_state(cpu->cpu_ind= ex)->enabled &&
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cpu->throttle_us_per_full) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0trace_dirtylimit_vcpu_execute(cpu->cpu= _index,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cpu->throt= tle_us_per_full);
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 usleep(cpu->throttle_us_per_full);
-=C2=A0 =C2=A0 }
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 sleep_us =3D cpu->throttle_us_per_full;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 dirtylimit_state_unlock();
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 endtime_us =3D qemu_clock_get_us(QEMU_CLOCK_RE= ALTIME) + sleep_us;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 while (sleep_us > 0 && !cpu->sto= p) {
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (sleep_us > SCALE_US) { +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 qemu_mutex_lock_io= thread();
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 qemu_cond_timedwai= t_iothread(cpu->halt_cond, sleep_us / SCALE_US);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 qemu_mutex_unlock_= iothread();
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 } else
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 g_usleep(sleep_us)= ;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sleep_us =3D endtime_us - qemu_c= lock_get_us(QEMU_CLOCK_REALTIME);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
+=C2=A0 =C2=A0 } else
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 dirtylimit_state_unlock();
=C2=A0}

=C2=A0static void dirtylimit_init(void)
--
2.39.3



--
Best regards=
--0000000000001b72800604887901--