From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F902C43334 for ; Mon, 4 Jul 2022 16:32:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B1D506B0074; Mon, 4 Jul 2022 12:32:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AA8406B0075; Mon, 4 Jul 2022 12:32:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91FA86B007E; Mon, 4 Jul 2022 12:32:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7CA916B0074 for ; Mon, 4 Jul 2022 12:32:12 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4F82D35684 for ; Mon, 4 Jul 2022 16:32:12 +0000 (UTC) X-FDA: 79649959704.18.2E6A78E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf02.hostedemail.com (Postfix) with ESMTP id 7AABC80009 for ; Mon, 4 Jul 2022 16:32:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1656952330; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CcX13tRiQ1CUkpKPTfZy53toNrkteRnr+JO8IbyM0ig=; b=QGdxRuzj5zv0C+02YH52LS+2PYOB72TMK9Mj3Ub1ZIKdobJPHyOEpEUvPbf1WpWKmIthTW 6ylt/ABocnPaVVnzMtGoRd6892odwEiJtWrlSOU8yxEwntkwyrj0yARN7ZDsgQDU5MsbZ2 S+u11KzAyFOx4S3MZbQ35+CDy5gzNIM= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-601-O4e6pQC6NZ6JH_Oc8PEFVw-1; Mon, 04 Jul 2022 12:32:09 -0400 X-MC-Unique: O4e6pQC6NZ6JH_Oc8PEFVw-1 Received: by mail-wm1-f70.google.com with SMTP id bg6-20020a05600c3c8600b003a03d5d19e4so5587467wmb.1 for ; Mon, 04 Jul 2022 09:32:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:content-transfer-encoding:user-agent:mime-version; bh=CcX13tRiQ1CUkpKPTfZy53toNrkteRnr+JO8IbyM0ig=; b=qGJ3LIOT2Pg0DO5VXarQQhHoQ/IGJME2jQ+yvgnkmT8SQXbpQGPX/nLe/tUnrivwZo GY1GANrc/IA70Je/RL8tK9s37gv32pDeWIBSkXuvOT06a8SQTWEK0c9K+4HGBxq3Dpu8 B0v8yfQ8z9uBwM0X4Ti6qZN51Juio9O0IjJliudwFbkFvmpqgo4VqUjRcU7uY8Nxv9jY /mz+OfLX/Dk6SoBhENOhQgngVmc45B6qgQK8Nbe89mR68Fz7YTAj3q4UJkBey0XmEdzq o6DvIBEJlcg9Ahan5Sb/0TnQsF0WbeXfAW7KjYTzcxSd8Bl2CVtFgFCT1Y51zYbtCrs5 sDAw== X-Gm-Message-State: AJIora8veow5xyG9cCkLQwWc9rr1YjGA7UA0qqO8dL+cKiG64YtfArOj XRTziLCNEzZN2ZmAaW0OrwL0SFtL0N7o/CZruzu3kx8S31rJetWiWilWcVwrgER+uo9wuTABAFQ 4RMecEZ9G5lQ= X-Received: by 2002:a05:600c:2207:b0:3a0:4659:a0b0 with SMTP id z7-20020a05600c220700b003a04659a0b0mr34770160wml.48.1656952328740; Mon, 04 Jul 2022 09:32:08 -0700 (PDT) X-Google-Smtp-Source: AGRyM1vnvKWgb7epV/6gnkjXMb5ckPO58u5LCltHJ6GzPrkl+ffwyyTHwI5y63zhOS8rkRw1W7EDBA== X-Received: by 2002:a05:600c:2207:b0:3a0:4659:a0b0 with SMTP id z7-20020a05600c220700b003a04659a0b0mr34770118wml.48.1656952328435; Mon, 04 Jul 2022 09:32:08 -0700 (PDT) Received: from ?IPv6:2a0c:5a80:1b12:b300:47b0:25ba:1a2e:72ac? ([2a0c:5a80:1b12:b300:47b0:25ba:1a2e:72ac]) by smtp.gmail.com with ESMTPSA id v192-20020a1cacc9000000b003975c7058bfsm20288132wme.12.2022.07.04.09.32.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Jul 2022 09:32:08 -0700 (PDT) Message-ID: <7c66ffb07a06f1c64985c3b6e3c212f1f247a652.camel@redhat.com> Subject: Re: [PATCH 5/7] mm/page_alloc: Protect PCP lists with a spinlock From: Nicolas Saenz Julienne To: Mel Gorman , Andrew Morton Cc: Marcelo Tosatti , Vlastimil Babka , Michal Hocko , Hugh Dickins , Yu Zhao , Marek Szyprowski , LKML , Linux-MM Date: Mon, 04 Jul 2022 18:32:06 +0200 In-Reply-To: <20220624125423.6126-6-mgorman@techsingularity.net> References: <20220624125423.6126-1-mgorman@techsingularity.net> <20220624125423.6126-6-mgorman@techsingularity.net> User-Agent: Evolution 3.44.2 (3.44.2-1.fc36) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656952331; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CcX13tRiQ1CUkpKPTfZy53toNrkteRnr+JO8IbyM0ig=; b=HbghijDUawqzLW9SYsKRIvttDxiI3x+al4MxfWT+yRzo2kTGjDL6X6cNKYY1jXPE8UqU4D HwtpHXhtREgaggW5FQ60FTn0Ig3btCu4UTVTHRlK6RYmsATsZQC41YyIc/WnXd7ZPhoc0w fbnQfhPbeiVlLabe7BaYcXgKgdooHDY= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QGdxRuzj; spf=none (imf02.hostedemail.com: domain of nsaenzju@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=nsaenzju@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656952331; a=rsa-sha256; cv=none; b=sCpbdu8siACgXvyWaEvCFyqkSsDE+/0AUnTQkfG1P6P/9r1nhxDksDE76zhPxGeNbbZily Nkz7sW8bU9IYsL+wFNDDRF2DB9BeV456noRN6PHVf1WWULt7AcEAZOI2KLuHcTR5k2RQCd gEeSXZX95LsXLXFwdtC9HWYi71Cnn58= X-Stat-Signature: bgwtwgfxgj9szrp8enj5sehbgdmkndwt X-Rspamd-Queue-Id: 7AABC80009 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QGdxRuzj; spf=none (imf02.hostedemail.com: domain of nsaenzju@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=nsaenzju@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspam-User: X-Rspamd-Server: rspam02 X-HE-Tag: 1656952331-701135 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, 2022-06-24 at 13:54 +0100, Mel Gorman wrote: > Currently the PCP lists are protected by using local_lock_irqsave to > prevent migration and IRQ reentrancy but this is inconvenient. Remote > draining of the lists is impossible and a workqueue is required and every > task allocation/free must disable then enable interrupts which is > expensive. >=20 > As preparation for dealing with both of those problems, protect the lists > with a spinlock. The IRQ-unsafe version of the lock is used because IRQs > are already disabled by local_lock_irqsave. spin_trylock is used in > preparation for a time when local_lock could be used instead of > lock_lock_irqsave. >=20 > The per_cpu_pages still fits within the same number of cache lines after > this patch relative to before the series. >=20 > struct per_cpu_pages { > spinlock_t lock; /* 0 4 *= / > int count; /* 4 4 *= / > int high; /* 8 4 *= / > int batch; /* 12 4 *= / > short int free_factor; /* 16 2 *= / > short int expire; /* 18 2 *= / >=20 > /* XXX 4 bytes hole, try to pack */ >=20 > struct list_head lists[13]; /* 24 208 *= / >=20 > /* size: 256, cachelines: 4, members: 7 */ > /* sum members: 228, holes: 1, sum holes: 4 */ > /* padding: 24 */ > } __attribute__((__aligned__(64))); >=20 > There is overhead in the fast path due to acquiring the spinlock even > though the spinlock is per-cpu and uncontended in the common case. Page > Fault Test (PFT) running on a 1-socket reported the following results on = a > 1 socket machine. >=20 > 5.19.0-rc3 5.19.0-rc3 > vanilla mm-pcpspinirq-v5r16 > Hmean faults/sec-1 869275.7381 ( 0.00%) 874597.5167 * 0.61%* > Hmean faults/sec-3 2370266.6681 ( 0.00%) 2379802.0362 * 0.40%* > Hmean faults/sec-5 2701099.7019 ( 0.00%) 2664889.7003 * -1.34%* > Hmean faults/sec-7 3517170.9157 ( 0.00%) 3491122.8242 * -0.74%* > Hmean faults/sec-8 3965729.6187 ( 0.00%) 3939727.0243 * -0.66%* >=20 > There is a small hit in the number of faults per second but given that th= e > results are more stable, it's borderline noise. >=20 > Signed-off-by: Mel Gorman > --- Reviewed-by: Nicolas Saenz Julienne Tested-by: Nicolas Saenz Julienne Thanks! --=20 Nicol=C3=A1s S=C3=A1enz