From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35033C433EF for ; Mon, 7 Mar 2022 13:57:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F9218D0002; Mon, 7 Mar 2022 08:57:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A7808D0001; Mon, 7 Mar 2022 08:57:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56FBA8D0002; Mon, 7 Mar 2022 08:57:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0136.hostedemail.com [216.40.44.136]) by kanga.kvack.org (Postfix) with ESMTP id 478CB8D0001 for ; Mon, 7 Mar 2022 08:57:53 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id F2390A8339 for ; Mon, 7 Mar 2022 13:57:52 +0000 (UTC) X-FDA: 79217743626.22.CE43A5A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf01.hostedemail.com (Postfix) with ESMTP id 6BE564000B for ; Mon, 7 Mar 2022 13:57:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646661471; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wM6VOulZtYrylaSH6OVQrZB6SV0lfkKkm0v+oBQEvPk=; b=eJ5bmaaJhzay6jOhQFqjAFz+6SvdRhGCzgEiw+/hVRl3kKEO/ogyRddtjJwG2nBfwGkKnq DSUx5gUi4y0oJYo9PhwbCCJgep9ylo96zShf32bc4aDuH4aZTn+PQ/ELQrbrSfG6E9Hm3C svTZmz6DvqLOZE/lWzzqSNJgDjt8oxc= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-90-a1iUHJUNMoiXW5x6UiMunw-1; Mon, 07 Mar 2022 08:57:50 -0500 X-MC-Unique: a1iUHJUNMoiXW5x6UiMunw-1 Received: by mail-wr1-f69.google.com with SMTP id l10-20020a05600012ca00b001f1e4669c98so895470wrx.23 for ; Mon, 07 Mar 2022 05:57:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=wM6VOulZtYrylaSH6OVQrZB6SV0lfkKkm0v+oBQEvPk=; b=u3mesaJ6t0OcDx6p3YzUZnIZ29licfgoIjJZm9qBG+GJqbFKz6SdABQ0FNKIxbhUgf Qpn1Gjgqf0QXOMw2dy7oRz614CljCFwLWQ2HhBWS6QzFjKim44fcYTMdKu9fItJT+pPN TM2ZNLl9mu9/UsjRT9IBXSJ5Lq7yozYjGxf9VxAJCsUGriYSDyBLb7unPr2ye1w14kvx AIAWYlvaobRu7HODSNTvJJB4Lv6qKuQOZCEnBxUFNjqTDm0XJKQ9ck3DQz6fIPZfpN45 bgsu3sHnC/NrIJaPm/7mSDUhjj29j5IAzJOQ4EtRhfnY1Ib4iIUai2d5tUSiSmlX/4S1 SoUw== X-Gm-Message-State: AOAM530j+ldM+nNugMQCTJDkbW/xzfOXqwz+diuglBbxmtubo9x+cAis 1Tbv94cXNsmZMmnbIYEYrsImKJG5J8Svq0xnwBrKzBFULctFCdTOjJZATojxUq6MaPOwZrFhhS1 nh5LTLDa7aTo= X-Received: by 2002:adf:d1e4:0:b0:1fd:bcc4:66c8 with SMTP id g4-20020adfd1e4000000b001fdbcc466c8mr822255wrd.488.1646661469369; Mon, 07 Mar 2022 05:57:49 -0800 (PST) X-Google-Smtp-Source: ABdhPJzoPAJ77EFUcVlCEoryGdZ4Di7c8IlBBuTKzrsfodTyiISnPTIgOVliwHERXJfFacwYmqHQlQ== X-Received: by 2002:adf:d1e4:0:b0:1fd:bcc4:66c8 with SMTP id g4-20020adfd1e4000000b001fdbcc466c8mr822240wrd.488.1646661469127; Mon, 07 Mar 2022 05:57:49 -0800 (PST) Received: from ?IPv6:2a0c:5a80:1b14:b500:abb:f9d1:7bc2:3db8? ([2a0c:5a80:1b14:b500:abb:f9d1:7bc2:3db8]) by smtp.gmail.com with ESMTPSA id l5-20020a5d6745000000b001f1e4e40e42sm5589858wrw.77.2022.03.07.05.57.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Mar 2022 05:57:48 -0800 (PST) Message-ID: Subject: Re: [PATCH 0/2] mm/page_alloc: Remote per-cpu lists drain support From: Nicolas Saenz Julienne To: Mel Gorman Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, frederic@kernel.org, tglx@linutronix.de, mtosatti@redhat.com, linux-rt-users@vger.kernel.org, vbabka@suse.cz, cl@linux.com, paulmck@kernel.org, willy@infradead.org Date: Mon, 07 Mar 2022 14:57:47 +0100 In-Reply-To: <20220303114550.GE4363@suse.de> References: <20220208100750.1189808-1-nsaenzju@redhat.com> <20220303114550.GE4363@suse.de> User-Agent: Evolution 3.42.4 (3.42.4-1.fc35) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 6BE564000B X-Stat-Signature: 9zgw9xt56gtt1wiba5bx81jis1omiqfu X-Rspam-User: Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eJ5bmaaJ; spf=none (imf01.hostedemail.com: domain of nsaenzju@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=nsaenzju@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam07 X-HE-Tag: 1646661472-982323 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Mel, Thanks for having a look at this. On Thu, 2022-03-03 at 11:45 +0000, Mel Gorman wrote: > On Tue, Feb 08, 2022 at 11:07:48AM +0100, Nicolas Saenz Julienne wrote: > > This series replaces mm/page_alloc's per-cpu page lists drain mechani= sm with > > one that allows accessing the lists remotely. Currently, only a local= CPU is > > permitted to change its per-cpu lists, and it's expected to do so, on= -demand, > > whenever a process demands it by means of queueing a drain task on th= e local > > CPU. This causes problems for NOHZ_FULL CPUs and real-time systems th= at can't > > take any sort of interruption and to some lesser extent inconvenience= s idle and > > virtualised systems. > >=20 >=20 > I know this has been sitting here for a long while. Last few weeks have > not been fun. No problem. > > Note that this is not the first attempt at fixing this per-cpu page l= ists: > > - The first attempt[1] tried to conditionally change the pagesets lo= cking > > scheme based the NOHZ_FULL config. It was deemed hard to maintain = as the > > NOHZ_FULL code path would be rarely tested. Also, this only solves= the issue > > for NOHZ_FULL setups, which isn't ideal. > > - The second[2] unanimously switched the local_locks to per-cpu spin= locks. The > > performance degradation was too big. > >=20 >=20 > For unrelated reasons I looked at using llist to avoid locks entirely. = It > turns out it's not possible and needs a lock. We know "local_locks to > per-cpu spinlocks" took a large penalty so I considered alternatives on > how a lock could be used. I found it's possible to both remote drain > the lists and avoid the disable/enable of IRQs entirely as long as a > preempting IRQ is willing to take the zone lock instead (should be very > rare). The IRQ part is a bit hairy though as softirqs are also a proble= m > and preempt-rt needs different rules and the llist has to sort PCP > refills which might be a loss in total. However, the remote draining ma= y > still be interesting. The full series is at > https://git.kernel.org/pub/scm/linux/kernel/git/mel/linux.git/ mm-pcpll= ist-v1r2 I'll have a proper look at it soon. Regards, --=20 Nicol=C3=A1s S=C3=A1enz