From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60E50C4338F for ; Wed, 28 Jul 2021 05:42:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B88CD600EF for ; Wed, 28 Jul 2021 05:42:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B88CD600EF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:47438 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m8cKQ-0005yt-ON for qemu-devel@archiver.kernel.org; Wed, 28 Jul 2021 01:42:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45356) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m8cIx-0005Cw-Gj for qemu-devel@nongnu.org; Wed, 28 Jul 2021 01:40:32 -0400 Received: from mga11.intel.com ([192.55.52.93]:44108) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m8cIt-0007PW-MO for qemu-devel@nongnu.org; Wed, 28 Jul 2021 01:40:30 -0400 X-IronPort-AV: E=McAfee;i="6200,9189,10058"; a="209472569" X-IronPort-AV: E=Sophos;i="5.84,275,1620716400"; d="scan'208";a="209472569" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jul 2021 22:40:15 -0700 X-IronPort-AV: E=Sophos;i="5.84,275,1620716400"; d="scan'208";a="517283019" Received: from cqiang-mobl.ccr.corp.intel.com (HELO [10.238.0.162]) ([10.238.0.162]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jul 2021 22:40:13 -0700 From: Chenyi Qiang Subject: Re: [PATCH v4] i386: Add ratelimit for bus locks acquired in guest To: "Dr. David Alan Gilbert" References: <20210521043820.29678-1-chenyi.qiang@intel.com> Message-ID: Date: Wed, 28 Jul 2021 13:40:11 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Received-SPF: pass client-ip=192.55.52.93; envelope-from=chenyi.qiang@intel.com; helo=mga11.intel.com X-Spam_score_int: -72 X-Spam_score: -7.3 X-Spam_bar: ------- X-Spam_report: (-7.3 / 5.0 requ) BAYES_00=-1.9, NICE_REPLY_A=-0.438, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , Xiaoyao Li , Marcelo Tosatti , Richard Henderson , qemu-devel@nongnu.org, Paolo Bonzini Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On 7/27/2021 4:28 PM, Dr. David Alan Gilbert wrote: > * Chenyi Qiang (chenyi.qiang@intel.com) wrote: >> A bus lock is acquired through either split locked access to writeback >> (WB) memory or any locked access to non-WB memory. It is typically >1000 >> cycles slower than an atomic operation within a cache and can also >> disrupts performance on other cores. >> >> Virtual Machines can exploit bus locks to degrade the performance of >> system. To address this kind of performance DOS attack coming from the >> VMs, bus lock VM exit is introduced in KVM and it can report the bus >> locks detected in guest. If enabled in KVM, it would exit to the >> userspace to let the user enforce throttling policies once bus locks >> acquired in VMs. >> >> The availability of bus lock VM exit can be detected through the >> KVM_CAP_X86_BUS_LOCK_EXIT. The returned bitmap contains the potential >> policies supported by KVM. The field KVM_BUS_LOCK_DETECTION_EXIT in >> bitmap is the only supported strategy at present. It indicates that KVM >> will exit to userspace to handle the bus locks. >> >> This patch adds a ratelimit on the bus locks acquired in guest as a >> mitigation policy. >> >> Introduce a new field "bus_lock_ratelimit" to record the limited speed >> of bus locks in the target VM. The user can specify it through the >> "bus-lock-ratelimit" as a machine property. In current implementation, >> the default value of the speed is 0 per second, which means no >> restrictions on the bus locks. >> >> As for ratelimit on detected bus locks, simply set the ratelimit >> interval to 1s and restrict the quota of bus lock occurence to the value >> of "bus_lock_ratelimit". A potential alternative is to introduce the >> time slice as a property which can help the user achieve more precise >> control. >> >> The detail of bus lock VM exit can be found in spec: >> https://software.intel.com/content/www/us/en/develop/download/intel-architecture-instruction-set-extensions-programming-reference.html >> >> Signed-off-by: Chenyi Qiang > > Hi Chenyi, > > I noticed in this patch: > > >> +static void kvm_rate_limit_on_bus_lock(void) >> +{ >> + uint64_t delay_ns = ratelimit_calculate_delay(&bus_lock_ratelimit_ctrl, 1); >> + >> + if (delay_ns) { >> + g_usleep(delay_ns / SCALE_US); >> + } >> +} > > and wondered if this would block cpu kicks, and what would happen if > delay_ns got quite big - Eduardo thinks it might get upto 1s. > I did a rough test, force the delay_ns to 1s and see how long it will take to sleep 20s in guest. Actually, for 1-vcpu VM, the output of elapsed time is 20.4~20.6s, so I assume the applications in guest may lose some precision. Changing to a more refined time slice control is an solution. (But concerning that such ratelimit only happen in a malicious guest, maybe it is acceptable to lose some accuracy.) > Also, it feels similar to what migration does during 'auto converge'; > see softmuu/cpu-throttle.c - instead of doing your own g_usleep > you could call cpu_throttle_set with a given throttle rate. > Yes, looked at the cpu-throttle code, cpu_throttle_set works similarly, but need some refactor. Migration uses the static throttle_percentage to control the global throttling, so if bus lock throttling calls cpu_throttle_set, it needs to distinguish with migration. > Dave > >> + >> MemTxAttrs kvm_arch_post_run(CPUState *cpu, struct kvm_run *run) >> { >> X86CPU *x86_cpu = X86_CPU(cpu); >> @@ -4237,6 +4271,9 @@ MemTxAttrs kvm_arch_post_run(CPUState *cpu, struct kvm_run *run) >> } else { >> env->eflags &= ~IF_MASK; >> } >> + if (run->flags & KVM_RUN_X86_BUS_LOCK) { >> + kvm_rate_limit_on_bus_lock(); >> + } >> >> /* We need to protect the apic state against concurrent accesses from >> * different threads in case the userspace irqchip is used. */ >> @@ -4595,6 +4632,10 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) >> ioapic_eoi_broadcast(run->eoi.vector); >> ret = 0; >> break; >> + case KVM_EXIT_X86_BUS_LOCK: >> + /* already handled in kvm_arch_post_run */ >> + ret = 0; >> + break; >> default: >> fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason); >> ret = -1; >> -- >> 2.17.1 >> >>