From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63315C4361B for ; Mon, 14 Dec 2020 19:23:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0D5AF224B0 for ; Mon, 14 Dec 2020 19:23:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2440837AbgLNTUe (ORCPT ); Mon, 14 Dec 2020 14:20:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33836 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2440694AbgLNTUa (ORCPT ); Mon, 14 Dec 2020 14:20:30 -0500 Received: from mail-wm1-x342.google.com (mail-wm1-x342.google.com [IPv6:2a00:1450:4864:20::342]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66AFBC0613D3 for ; Mon, 14 Dec 2020 11:19:50 -0800 (PST) Received: by mail-wm1-x342.google.com with SMTP id v14so14790211wml.1 for ; Mon, 14 Dec 2020 11:19:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=asNcRPkAGuZAucIeskI3gXkj+rYqmbvG3dpJE/aYZvs=; b=XBkRd9CpZJMBYHCFtETi9mqmnUmQ7krsJqjSCNVbkD8h1+MyxeyXwqYpTmoEDqFjfV tWp9LkY87LBo2gSEis+6y3C6RPhUxZmAhFDZnIrYFSkt5G4BDtuTV2uVJHFRzmlmzQTD Ex2rr+j/BihDrEZE5+bok2huTbc9nVva6k5vUXHI0fB4mH+72RgvdajH2BsyL71W4ffm KWoHGALfJAbKwrBXKFG2eYunUPuXBxak5VVSgNIhtSRkr/OWtBaFSV7AqDEnE4m9L3me 7448u5xA+eVR2n/WSCCNtG1IxQA+Hd8HzcyKa2MQUSMQPHKJNL0YIK0EjjDIvuWQSWG4 +FCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=asNcRPkAGuZAucIeskI3gXkj+rYqmbvG3dpJE/aYZvs=; b=PbneySodATpSChhfHdXNb3qmtW2vV/xgJcxMPvNMi4b45/dxC5feyKwCKDqPURoEPF rruWmRq8to/SVDFH+RDhrhxn3EKMacyEVlesQBFnhlLeOuwQOwZXh6EWQhmMrbipG99i QHB0AGYRKVi7CKRmEo1N/P2FeJtdmkfpCUjVTMeE0+StzE6z/EUIccDmHCzHyKIcN2IN chq4lebmzq3kc0TXe8PLZxH8FFBPqjQwplSJzfW/yYRnHBknZVZaFs28lnOkMjXbdpB0 Ttp9SNUYIH50wFz/EDtscA+CNBlrXBFtep2TLoNVgsCf8UXaK8aVezlocFspftC7zReq ZKMQ== X-Gm-Message-State: AOAM5339IKfqCrUJVMhVs+NDQuzR3ULxUpre2mEfJfLJW0lxSEITp1Qk 81Mrro64Yh/Mm/RVZuDZLAA= X-Google-Smtp-Source: ABdhPJzss3UNfcSmv9gw+d/sCfzwlbu/kiX1ptaUuVfi4uiNTbdNPDF73/dpGnUV3HOnyQNHcxkOQA== X-Received: by 2002:a7b:c088:: with SMTP id r8mr29647174wmh.45.1607973589139; Mon, 14 Dec 2020 11:19:49 -0800 (PST) Received: from debian (host-92-5-241-147.as43234.net. [92.5.241.147]) by smtp.gmail.com with ESMTPSA id g192sm33482759wme.48.2020.12.14.11.19.48 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 14 Dec 2020 11:19:48 -0800 (PST) Date: Mon, 14 Dec 2020 19:19:46 +0000 From: Sudip Mukherjee To: gregkh@linuxfoundation.org Cc: xiaochen.shen@intel.com, bp@suse.de, stable@vger.kernel.org, tony.luck@intel.com Subject: Re: FAILED: patch "[PATCH] x86/resctrl: Fix incorrect local bandwidth when mba_sc is" failed to apply to 5.9-stable tree Message-ID: <20201214191946.4ui7hza2eqtirzad@debian> References: <160795595120091@kroah.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="tod2mk2dayn5nwa2" Content-Disposition: inline In-Reply-To: <160795595120091@kroah.com> User-Agent: NeoMutt/20170113 (1.7.2) Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org --tod2mk2dayn5nwa2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi Greg, On Mon, Dec 14, 2020 at 03:25:51PM +0100, gregkh@linuxfoundation.org wrote: > > The patch below does not apply to the 5.9-stable tree. > If someone wants it applied there, or to any other stable or longterm > tree, then please email the backport, including the original git commit > id to . Here is the backport with abe8f12b4425 ("x86/resctrl: Remove unused struct mbm_state::chunks_bw") which makes the backport easy. -- Regards Sudip --tod2mk2dayn5nwa2 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-x86-resctrl-Remove-unused-struct-mbm_state-chunks_bw.patch" >From 2cff132ee9a2aab95c2700f2957c5c138f367445 Mon Sep 17 00:00:00 2001 From: James Morse Date: Wed, 8 Jul 2020 16:39:20 +0000 Subject: [PATCH 1/2] x86/resctrl: Remove unused struct mbm_state::chunks_bw commit abe8f12b44250d02937665033a8b750c1bfeb26e upstream Nothing reads struct mbm_states's chunks_bw value, its a copy of chunks. Remove it. Signed-off-by: James Morse Signed-off-by: Borislav Petkov Reviewed-by: Reinette Chatre Link: https://lkml.kernel.org/r/20200708163929.2783-2-james.morse@arm.com Signed-off-by: Sudip Mukherjee --- arch/x86/kernel/cpu/resctrl/internal.h | 2 -- arch/x86/kernel/cpu/resctrl/monitor.c | 3 +-- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 5ffa32256b3b..72bb21068466 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -283,7 +283,6 @@ struct rftype { * struct mbm_state - status for each MBM counter in each domain * @chunks: Total data moved (multiply by rdt_group.mon_scale to get bytes) * @prev_msr Value of IA32_QM_CTR for this RMID last time we read it - * @chunks_bw Total local data moved. Used for bandwidth calculation * @prev_bw_msr:Value of previous IA32_QM_CTR for bandwidth counting * @prev_bw The most recent bandwidth in MBps * @delta_bw Difference between the current and previous bandwidth @@ -292,7 +291,6 @@ struct rftype { struct mbm_state { u64 chunks; u64 prev_msr; - u64 chunks_bw; u64 prev_bw_msr; u32 prev_bw; u32 delta_bw; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 837d7d012b7b..d6b92d7487a7 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -279,8 +279,7 @@ static void mbm_bw_count(u32 rmid, struct rmid_read *rr) return; chunks = mbm_overflow_count(m->prev_bw_msr, tval, rr->r->mbm_width); - m->chunks_bw += chunks; - m->chunks = m->chunks_bw; + m->chunks += chunks; cur_bw = (chunks * r->mon_scale) >> 20; if (m->delta_comp) -- 2.11.0 --tod2mk2dayn5nwa2 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-x86-resctrl-Fix-incorrect-local-bandwidth-when-mba_s.patch" >From 1277dd8aa14b279cf86c131f6817f673b6429000 Mon Sep 17 00:00:00 2001 From: Xiaochen Shen Date: Fri, 4 Dec 2020 14:27:59 +0800 Subject: [PATCH 2/2] x86/resctrl: Fix incorrect local bandwidth when mba_sc is enabled commit 06c5fe9b12dde1b62821f302f177c972bb1c81f9 upstream The MBA software controller (mba_sc) is a feedback loop which periodically reads MBM counters and tries to restrict the bandwidth below a user-specified value. It tags along the MBM counter overflow handler to do the updates with 1s interval in mbm_update() and update_mba_bw(). The purpose of mbm_update() is to periodically read the MBM counters to make sure that the hardware counter doesn't wrap around more than once between user samplings. mbm_update() calls __mon_event_count() for local bandwidth updating when mba_sc is not enabled, but calls mbm_bw_count() instead when mba_sc is enabled. __mon_event_count() will not be called for local bandwidth updating in MBM counter overflow handler, but it is still called when reading MBM local bandwidth counter file 'mbm_local_bytes', the call path is as below: rdtgroup_mondata_show() mon_event_read() mon_event_count() __mon_event_count() In __mon_event_count(), m->chunks is updated by delta chunks which is calculated from previous MSR value (m->prev_msr) and current MSR value. When mba_sc is enabled, m->chunks is also updated in mbm_update() by mistake by the delta chunks which is calculated from m->prev_bw_msr instead of m->prev_msr. But m->chunks is not used in update_mba_bw() in the mba_sc feedback loop. When reading MBM local bandwidth counter file, m->chunks was changed unexpectedly by mbm_bw_count(). As a result, the incorrect local bandwidth counter which calculated from incorrect m->chunks is shown to the user. Fix this by removing incorrect m->chunks updating in mbm_bw_count() in MBM counter overflow handler, and always calling __mon_event_count() in mbm_update() to make sure that the hardware local bandwidth counter doesn't wrap around. Test steps: # Run workload with aggressive memory bandwidth (e.g., 10 GB/s) git clone https://github.com/intel/intel-cmt-cat && cd intel-cmt-cat && make ./tools/membw/membw -c 0 -b 10000 --read # Enable MBA software controller mount -t resctrl resctrl -o mba_MBps /sys/fs/resctrl # Create control group c1 mkdir /sys/fs/resctrl/c1 # Set MB throttle to 6 GB/s echo "MB:0=6000;1=6000" > /sys/fs/resctrl/c1/schemata # Write PID of the workload to tasks file echo `pidof membw` > /sys/fs/resctrl/c1/tasks # Read local bytes counters twice with 1s interval, the calculated # local bandwidth is not as expected (approaching to 6 GB/s): local_1=`cat /sys/fs/resctrl/c1/mon_data/mon_L3_00/mbm_local_bytes` sleep 1 local_2=`cat /sys/fs/resctrl/c1/mon_data/mon_L3_00/mbm_local_bytes` echo "local b/w (bytes/s):" `expr $local_2 - $local_1` Before fix: local b/w (bytes/s): 11076796416 After fix: local b/w (bytes/s): 5465014272 Fixes: ba0f26d8529c (x86/intel_rdt/mba_sc: Prepare for feedback loop) Signed-off-by: Xiaochen Shen Signed-off-by: Borislav Petkov Reviewed-by: Tony Luck Cc: Link: https://lkml.kernel.org/r/1607063279-19437-1-git-send-email-xiaochen.shen@intel.com Signed-off-by: Sudip Mukherjee --- arch/x86/kernel/cpu/resctrl/monitor.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index d6b92d7487a7..ddd91344682c 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -279,7 +279,6 @@ static void mbm_bw_count(u32 rmid, struct rmid_read *rr) return; chunks = mbm_overflow_count(m->prev_bw_msr, tval, rr->r->mbm_width); - m->chunks += chunks; cur_bw = (chunks * r->mon_scale) >> 20; if (m->delta_comp) @@ -450,15 +449,14 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid) } if (is_mbm_local_enabled()) { rr.evtid = QOS_L3_MBM_LOCAL_EVENT_ID; + __mon_event_count(rmid, &rr); /* * Call the MBA software controller only for the * control groups and when user has enabled * the software controller explicitly. */ - if (!is_mba_sc(NULL)) - __mon_event_count(rmid, &rr); - else + if (is_mba_sc(NULL)) mbm_bw_count(rmid, &rr); } } -- 2.11.0 --tod2mk2dayn5nwa2--