From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29686CA9ED2 for ; Mon, 4 Nov 2019 06:11:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CE996222C6 for ; Mon, 4 Nov 2019 06:11:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CE996222C6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 34DDE6B0005; Mon, 4 Nov 2019 01:11:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2FEFE6B0006; Mon, 4 Nov 2019 01:11:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1ECF16B0007; Mon, 4 Nov 2019 01:11:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0098.hostedemail.com [216.40.44.98]) by kanga.kvack.org (Postfix) with ESMTP id 09C346B0005 for ; Mon, 4 Nov 2019 01:11:18 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id A82444DCB for ; Mon, 4 Nov 2019 06:11:17 +0000 (UTC) X-FDA: 76117572594.15.angle55_7f8c0e4e8785f X-HE-Tag: angle55_7f8c0e4e8785f X-Filterd-Recvd-Size: 4607 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Mon, 4 Nov 2019 06:11:16 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Nov 2019 22:11:14 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.68,266,1569308400"; d="scan'208";a="204511187" Received: from yhuang-dev.sh.intel.com (HELO yhuang-dev) ([10.239.159.29]) by orsmga003.jf.intel.com with ESMTP; 03 Nov 2019 22:11:11 -0800 From: "Huang\, Ying" To: Peter Zijlstra Cc: , , Andrew Morton , Michal Hocko , Rik van Riel , Mel Gorman , Ingo Molnar , Dave Hansen , Dan Williams , Fengguang Wu Subject: Re: [RFC 10/10] autonuma, memory tiering: Adjust hot threshold automatically References: <20191101075727.26683-1-ying.huang@intel.com> <20191101075727.26683-11-ying.huang@intel.com> <20191101093145.GT4131@hirez.programming.kicks-ass.net> Date: Mon, 04 Nov 2019 14:11:11 +0800 In-Reply-To: <20191101093145.GT4131@hirez.programming.kicks-ass.net> (Peter Zijlstra's message of "Fri, 1 Nov 2019 10:31:45 +0100") Message-ID: <877e4gcgsg.fsf@yhuang-dev.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Peter Zijlstra writes: > On Fri, Nov 01, 2019 at 03:57:27PM +0800, Huang, Ying wrote: > >> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c >> index 0a83e9cf6685..22bdbb7afac2 100644 >> --- a/kernel/sched/fair.c >> +++ b/kernel/sched/fair.c >> @@ -1486,6 +1486,41 @@ static bool numa_migration_check_rate_limit(struct pglist_data *pgdat, >> return true; >> } >> >> +#define NUMA_MIGRATION_ADJUST_STEPS 16 >> + >> +static void numa_migration_adjust_threshold(struct pglist_data *pgdat, >> + unsigned long rate_limit, >> + unsigned long ref_threshold) >> +{ >> + unsigned long now = jiffies, last_threshold_jiffies; >> + unsigned long unit_threshold, threshold; >> + unsigned long try_migrate, ref_try_migrate, mdiff; >> + >> + last_threshold_jiffies = pgdat->autonuma_threshold_jiffies; >> + if (now > last_threshold_jiffies + >> + msecs_to_jiffies(sysctl_numa_balancing_scan_period_max) && >> + cmpxchg(&pgdat->autonuma_threshold_jiffies, >> + last_threshold_jiffies, now) == last_threshold_jiffies) { > > That is seriously unreadable gunk. The basic idea here is to adjust hot threshold every sysctl_numa_balancing_scan_period_max. Because the application accessing pattern may have spatial locality, and autonuma scans address space linearly. In general, the statistics of NUMA hint page fault latency isn't stable in arbitrary internal (such as 1 s, or 200 ms, etc). But the scanning of the whole address space of the application is expected to be finished within sysctl_numa_balancing_scan_period_max. The statistics of NUMA hint page fault latency is expected to be much more stable for the whole application. So we choose to adjust hot threshold every sysctl_numa_balancing_scan_period_max. I will add comments for this in the next version. >> + >> + ref_try_migrate = rate_limit * >> + sysctl_numa_balancing_scan_period_max / 1000; >> + try_migrate = node_page_state(pgdat, NUMA_TRY_MIGRATE); >> + mdiff = try_migrate - pgdat->autonuma_threshold_try_migrate; >> + unit_threshold = ref_threshold / NUMA_MIGRATION_ADJUST_STEPS; >> + threshold = pgdat->autonuma_threshold; >> + if (!threshold) >> + threshold = ref_threshold; >> + if (mdiff > ref_try_migrate * 11 / 10) >> + threshold = max(threshold - unit_threshold, >> + unit_threshold); >> + else if (mdiff < ref_try_migrate * 9 / 10) >> + threshold = min(threshold + unit_threshold, >> + ref_threshold); > > And that is violating codingstyle. You mean I need to use braces here? >> + pgdat->autonuma_threshold_try_migrate = try_migrate; >> + pgdat->autonuma_threshold = threshold; >> + } >> +} > > > Maybe if you use variable names that are slightly shorter than half your > line length? Sure. I will use shorter variable name in the next version. Best Regards, Huang, Ying