From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B978C433B4 for ; Wed, 7 Apr 2021 08:27:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8E5C5613A7 for ; Wed, 7 Apr 2021 08:27:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E5C5613A7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1F0DC6B007D; Wed, 7 Apr 2021 04:27:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A0F66B007E; Wed, 7 Apr 2021 04:27:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0695B6B0080; Wed, 7 Apr 2021 04:27:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0202.hostedemail.com [216.40.44.202]) by kanga.kvack.org (Postfix) with ESMTP id DE61F6B007D for ; Wed, 7 Apr 2021 04:27:32 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A1A9C824999B for ; Wed, 7 Apr 2021 08:27:32 +0000 (UTC) X-FDA: 78004891944.13.DE1D554 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf09.hostedemail.com (Postfix) with ESMTP id 560836000109 for ; Wed, 7 Apr 2021 08:27:30 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id DFD5DB040; Wed, 7 Apr 2021 08:27:30 +0000 (UTC) Date: Wed, 7 Apr 2021 09:27:28 +0100 From: Mel Gorman To: Huang Ying Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Peter Zijlstra , Peter Xu , Johannes Weiner , Vlastimil Babka , Matthew Wilcox , Will Deacon , Michel Lespinasse , Arjun Roy , "Kirill A. Shutemov" Subject: Re: [PATCH -V2] NUMA balancing: reduce TLB flush via delaying mapping on hint page fault Message-ID: <20210407082728.GA15768@suse.de> References: <20210402082717.3525316-1-ying.huang@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: <20210402082717.3525316-1-ying.huang@intel.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-Stat-Signature: obi5hxw8dbfiwhuptsbqhzxrsmt9z5m6 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 560836000109 Received-SPF: none (suse.de>: No applicable sender policy available) receiver=imf09; identity=mailfrom; envelope-from=""; helo=mx2.suse.de; client-ip=195.135.220.15 X-HE-DKIM-Result: none/none X-HE-Tag: 1617784050-264492 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Apr 02, 2021 at 04:27:17PM +0800, Huang Ying wrote: > With NUMA balancing, in hint page fault handler, the faulting page > will be migrated to the accessing node if necessary. During the > migration, TLB will be shot down on all CPUs that the process has run > on recently. Because in the hint page fault handler, the PTE will be > made accessible before the migration is tried. The overhead of TLB > shooting down can be high, so it's better to be avoided if possible. > In fact, if we delay mapping the page until migration, that can be > avoided. This is what this patch doing. > > > Thanks, I think this is ok for Andrew to pick up to see if anything bisects to this commit but it's a low risk. Reviewed-by: Mel Gorman More notes; This is not a universal win given that not all workloads exhibit the pattern where accesses occur in parallel threads between when a page is marked accessible and when it is migrated. The impact of the patch appears to be neutral for those workloads. For workloads that do exhibit the pattern, there is a small gain with a reduction in interrupts as advertised unlike v1 of the patch. Further tests are running to confirm the reduction is in TLB shootdown interrupts but I'm reasonably confident that will be the case. Gains are typically small and the load described in the changelog appears to be a best case scenario but a 1-5% gain in some other workloads is still an improvement. There is still the possibility that some workloads will unnecessarily stall as a result of the patch for slightly longer periods of time but that is a relatively low risk and will be difficult to detect. If I'm wrong, a bisection will find it. Andrew? -- Mel Gorman SUSE Labs