From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 799A1C433EF for ; Thu, 18 Nov 2021 18:20:12 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1887461B39 for ; Thu, 18 Nov 2021 18:20:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1887461B39 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=chinatelecom.cn Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:53608 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mnm15-0003uG-9d for qemu-devel@archiver.kernel.org; Thu, 18 Nov 2021 13:20:11 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55086) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mnlyS-0006uc-2G for qemu-devel@nongnu.org; Thu, 18 Nov 2021 13:17:28 -0500 Received: from prt-mail.chinatelecom.cn ([42.123.76.222]:34678 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mnlyP-0006If-9O for qemu-devel@nongnu.org; Thu, 18 Nov 2021 13:17:27 -0500 HMM_SOURCE_IP: 172.18.0.218:59574.644898278 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP Received: from clientip-171.223.99.79 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 99071280029; Fri, 19 Nov 2021 02:17:03 +0800 (CST) X-189-SAVE-TO-SEND: +huangy81@chinatelecom.cn Received: from ([172.18.0.218]) by app0025 with ESMTP id 32a1d44f1676476b8cd49bd05bb342bd for qemu-devel@nongnu.org; Fri, 19 Nov 2021 02:17:09 CST X-Transaction-ID: 32a1d44f1676476b8cd49bd05bb342bd X-Real-From: huangy81@chinatelecom.cn X-Receive-IP: 172.18.0.218 X-MEDUSA-Status: 0 From: huangy81@chinatelecom.cn To: qemu-devel Subject: [PATCH v2 0/3] support dirty restraint on vCPU Date: Fri, 19 Nov 2021 02:17:00 +0800 Message-Id: X-Mailer: git-send-email 1.8.3.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=42.123.76.222; envelope-from=huangy81@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , Hyman , David Hildenbrand , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , Paolo Bonzini , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Hyman Huang(黄勇) v2: - rebase on master - modify the following points according to the advices given by Juan 1. rename dirtyrestraint to dirtylimit 2. implement the full lifecyle function of dirtylimit_calc, include dirtylimit_calc and dirtylimit_calc_quit 3. introduce 'quit' field in dirtylimit_calc_state to implement the dirtylimit_calc_quit 4. remove the ready_cond and ready_mtx since it may not be suitable 5. put the 'record_dirtypage' function code at the beggining of the file 6. remove the unnecesary return; - other modifications has been made after code review 1. introduce 'bmap' and 'nr' field in dirtylimit_state to record the number of running thread forked by dirtylimit 2. stop the dirtyrate calculation thread if all the dirtylimit thread are stopped 3. do some renaming works dirtyrate calulation thread -> dirtylimit-calc dirtylimit thread -> dirtylimit-{cpu_index} function name do_dirtyrestraint -> dirtylimit_check qmp command dirty-restraint -> set-drity-limit qmp command dirty-restraint-cancel -> cancel-dirty-limit header file dirtyrestraint.h -> dirtylimit.h Please review, thanks ! thanks for the accurate and timely advices given by Juan. we really appreciate it if corrections and suggetions are proposed ! Best Regards ! Hyman v1: this patchset introduce a mechanism to impose dirty restraint on vCPU, aiming to keep the vCPU running in a certain dirtyrate given by user. dirty restraint on vCPU maybe an alternative method to implement convergence logic for live migration, which could improve guest memory performance during migration compared with traditional method in theory. For the current live migration implementation, the convergence logic throttles all vCPUs of the VM, which has some side effects. -'read processes' on vCPU will be unnecessarily penalized - throttle increase percentage step by step, which seems struggling to find the optimal throttle percentage when dirtyrate is high. - hard to predict the remaining time of migration if the throttling percentage reachs 99% to a certain extent, the dirty restraint machnism can fix these effects by throttling at vCPU granularity during migration. the implementation is rather straightforward, we calculate vCPU dirtyrate via the Dirty Ring mechanism periodically as the commit 0e21bf246 "implement dirty-ring dirtyrate calculation" does, for vCPU that be specified to impose dirty restraint, we throttle it periodically as the auto-converge does, once after throttling, we compare the quota dirtyrate with current dirtyrate, if current dirtyrate is not under the quota, increase the throttling percentage until current dirtyrate is under the quota. this patchset is the basis of implmenting a new auto-converge method for live migration, we introduce two qmp commands for impose/cancel the dirty restraint on specified vCPU, so it also can be an independent api to supply the upper app such as libvirt, which can use it to implement the convergence logic during live migration, supplemented with the qmp 'calc-dirty-rate' command or whatever. we post this patchset for RFC and any corrections and suggetions about the implementation, api, throttleing algorithm or whatever are very appreciated! Please review, thanks ! Best Regards ! Hyman Huang (3): migration/dirtyrate: implement vCPU dirtyrate calculation periodically cpu-throttle: implement vCPU throttle cpus-common: implement dirty limit on vCPU cpus-common.c | 41 ++++++ include/exec/memory.h | 5 +- include/hw/core/cpu.h | 7 + include/sysemu/cpu-throttle.h | 23 +++ include/sysemu/dirtylimit.h | 44 ++++++ migration/dirtyrate.c | 139 ++++++++++++++++-- migration/dirtyrate.h | 2 + qapi/misc.json | 44 ++++++ softmmu/cpu-throttle.c | 319 ++++++++++++++++++++++++++++++++++++++++++ softmmu/trace-events | 5 + softmmu/vl.c | 1 + 11 files changed, 619 insertions(+), 11 deletions(-) create mode 100644 include/sysemu/dirtylimit.h -- 1.8.3.1