From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8D9BC369A2 for ; Tue, 8 Apr 2025 17:22:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C5EFA6B00AE; Tue, 8 Apr 2025 13:22:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE47B6B00AF; Tue, 8 Apr 2025 13:22:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A85D16B00B0; Tue, 8 Apr 2025 13:22:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 88B446B00AE for ; Tue, 8 Apr 2025 13:22:04 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id BD24D812DD for ; Tue, 8 Apr 2025 17:22:05 +0000 (UTC) X-FDA: 83311544610.04.A18C4E2 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by imf30.hostedemail.com (Postfix) with ESMTP id DF7528000C for ; Tue, 8 Apr 2025 17:22:03 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=JZngmq2u; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of 3Olv1ZwgKCA0wnpxzn0ot11tyr.p1zyv07A-zzx8npx.14t@flex--jackmanb.bounces.google.com designates 209.85.221.73 as permitted sender) smtp.mailfrom=3Olv1ZwgKCA0wnpxzn0ot11tyr.p1zyv07A-zzx8npx.14t@flex--jackmanb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744132924; a=rsa-sha256; cv=none; b=knYb4f8lgvXFGCyclD+7nxkudIPHjdmDe2l3ft+TxdcHityhmNl0o5dgtwonEtNxenmOrj 5AkobzZ5F3bZxXex//588vp0nZwN90B7E7X9mvmLoS8qeY3XB74Pr/V89UmyJneHQmc7iD IZCjZrSxtQvkaGUW3/tolU27Ir2r9OU= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=JZngmq2u; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of 3Olv1ZwgKCA0wnpxzn0ot11tyr.p1zyv07A-zzx8npx.14t@flex--jackmanb.bounces.google.com designates 209.85.221.73 as permitted sender) smtp.mailfrom=3Olv1ZwgKCA0wnpxzn0ot11tyr.p1zyv07A-zzx8npx.14t@flex--jackmanb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744132924; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=u+luVGLBxL0+rbze1Xlp+FWE6ZfU74Gm/3Cw7pY07Gk=; b=5h+I4G4Q4jF17RK+e2WeW6lCtHsp9Bg5TU7GPd9KOcINzEdtLbkF1gu7GSXIU2yrLTvdn3 NUtrBmV11mfaZD3zBq6J9V8OfLet/S6m2Ck1zt8hk3YCLZcs/98/TTST6VVaonxznXwxLC m7bmKrr411j33xYAf1MVyVYs5aNJv64= Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-39c2da64df9so3947700f8f.0 for ; Tue, 08 Apr 2025 10:22:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744132922; x=1744737722; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=u+luVGLBxL0+rbze1Xlp+FWE6ZfU74Gm/3Cw7pY07Gk=; b=JZngmq2uff7QLY5GsTALqOtVI+FJi7diey0YckBvNXG6EclvEoabZtX4jOFKGPfxuH tHHiPWRtfgPc/FBxEaNcBeSlVDylqGnWtvcXmh1cY3XWmy9tUSOZACgQ9DM8T43PT+nC Cnk+1lClbbb5LptfTdlkg505Hm99N3YMw2lhfz8LvYlDZ1u/Z95UF7RZIsMU39VvZYZx 8+oV++1xErv0DWlbO+dbACreBYiXRBC/ImOIwO0Ym656PIyy3v+qVtssZ32XxTB9CVK4 w0XTEccEGUb6ZUOzuV70xvuO3y7nYXsJTichbejrKb2sNpOi0CNR/wUWyiZoslfwZ1It cQ1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744132922; x=1744737722; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=u+luVGLBxL0+rbze1Xlp+FWE6ZfU74Gm/3Cw7pY07Gk=; b=WCPVgX05msVTdnqeiVTTpu0JeM7FykRASqL0zL0rI9l6kOTQ2xhOWklL48menzRp7i u+kWPL7ReMks1IiX2kpQbMfguzeovSV+QML84ktJWCvF3mEiTqv4NQEWrfBnhTesbzRV nP9/YVWV0BCtkrdHswDb92TCfmxKdoNZos420ApDBfOl2XSmG6VMCQW8Yn4wK0hzF4t9 YrM3sjT1ToMXt7ABgvupzAfQBtu0jftt8U3cGLbDGcLXJs58NZV5oVetka+iVAFlPyTt PbpbOSVxTaai0/kZgKXNCKafOcT5aAEZDrZICD6LNRLu250eF8xXv9oJcB5LOuk9v2jv yyQg== X-Forwarded-Encrypted: i=1; AJvYcCUTqrW5krSHaXjVQxYr/nQqPpo7+xJv8TUczllBRyazb9KCve6IirxiOkFF8vwhBDQRpWKkXkU3OQ==@kvack.org X-Gm-Message-State: AOJu0YzhLgAeDBoxZYRfueu85GnQjCpJZeppUHxZaxsapJNEnh2+d15B A7ycKBwgWyScwh87+1svOHXv3n/jmHvtyF9KNkDP/BVz9bPG8TOAdHCUphr7vjAT+leLv2Nwm2p rVuuIEnAFng== X-Google-Smtp-Source: AGHT+IHaAZMKhByVkpO7+93Ka+6E50kvY3a7Y4B/+srVfxOdRHrMhLAj+xwnnXkHAeq0Q/znDzgiMtXoGrlkKg== X-Received: from wmbbg30.prod.google.com ([2002:a05:600c:3c9e:b0:43d:44cf:11f8]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:1843:b0:391:2c0c:1247 with SMTP id ffacd0b85a97d-39d87aa8fd6mr53654f8f.1.1744132922425; Tue, 08 Apr 2025 10:22:02 -0700 (PDT) Date: Tue, 08 Apr 2025 17:22:00 +0000 In-Reply-To: <20250407180154.63348-1-hannes@cmpxchg.org> Mime-Version: 1.0 References: <20250407180154.63348-1-hannes@cmpxchg.org> X-Mailer: aerc 0.18.2 Message-ID: Subject: Re: [PATCH 1/2] mm: page_alloc: speed up fallbacks in rmqueue_bulk() From: Brendan Jackman To: Johannes Weiner , Andrew Morton Cc: Vlastimil Babka , Mel Gorman , Carlos Song , , , kernel test robot , Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: DF7528000C X-Stat-Signature: qx1jhyoepe8h3zeb5j883njnozk5dbc3 X-HE-Tag: 1744132923-943657 X-HE-Meta: U2FsdGVkX1+X2v7MxGif/2uXwU1//IQc6kx7rYHKWNXCiYxis7GU5E1OjVebLtcL6R9Qduaygg4K7djguMnva3RTUIrqu+jMaHlvPqzuseNYp+O+iYe+eTKJoUMUJ1XJmCuk+ejzoaj94QS2qA1tPysWpwU+cDreLRHfvkKkq2NlF3GhaXUayeM7p9iME/tTtwVnPjJzFyhi4HtGk8cuFpj+9wYEfUUSpmjUjg8staHFAR2rTgxBV1WLpmGIcSV/uMjJxGyh23HAsCLZ+0n8cIeKpSA4PvE7oAX2p4/hvOKG5BYquwOhun+TSP75pYvKZm1oBUbR8E1pFz9iTtsDCxy+sH2ou9oFjOEM4L90/vYFEmwi9uujdrq7sbbwfnToSlc1EJKA991CuUoTKUljMsV353zeMaYSrXQeWsi64HX9PkQtL+s1mksCcOLIQ2RThs+VSSBdnvsHtJWx8NOnRMcZhYrHq/Wo00ZjiIl5Va3JNOi1kSLuNDEZomPCHk1zCK7tLBUul5HVI/KxaZ4fc95fDaSZOPkSAyoveLE1AIzVKlSwYMuLY5F05GJx+F8p6bEaDnyPRHj9fEcHM4kL9/WjHQ/epaVnGIkiTWDvSDVTtZIsidI1Bczu2WpV5rPpq99RKZ4MhklY5RK1jVWQmL0IlnVQvKEeCN6OEGUBgt0336FAd/9AipK56qrvmintBpY13JTjILHwnQQBwPCbnM5HOaZVm+pYq1N2X2Pv668WALwMb9+tzUXu0fjnuBhhuue2B7Lq12aHAIjI6gEgsUl3r94K31DpsibkCeZFGm0o7exzNUwuIYk9pzG5YpMHz8nhTFG0da/7AvcwvrcurrGwPZ4F6X7R3WG5DV4veLrnjQOA6jwkZXI/CO+wSW16jOIG6Cknyb0chvpVqDsP2SGVBg6CVX7eBW3HU4gBxpQQecxWcu6ZqnFI5Oz6kjrpMS15wT2J31NrzgDryPm w2FcMkZx 5w1euaRyxQDXY8wDr7Gd1Hj+c2wKmSwo3lnwUusZj2aweoKmFoupLkSimKUqIEi9JrYJ7tIImN9BxRXejfQPheVuAPlektJaiOLe/QiqoKNayJ+FtEDEW8K7W+f8WVz/yKnF0+ragwvCQ8T5HcrXsoJ0gosj5xnU7j72Co71eSkILPOZlISqdYPrZi3p+v9nh6jXtAbollZ+XmhuzhK1OVnuP6T8Zqs+/3Aufkr6otdq2ZeO3aEGAt22qfpKmrvR9YodJYVnpsFAU/T6lH2oG/ZwU1ibVZFRna6NkQn6lqb9r5jnu4IevEySp9iWBq2iNjlFinEgYCQJwmF5wwF/1dy6mSqVXEBObQbG+VUFiT9KRmgJ9QmwvHZ97iMdLkSLNx5p7RaDQBVrq7IhdfaKSDRJReSnxKxNiuyzF/V6JUGyuG718FHrzWNTxaVZcmeQMbOmPFNF7yrmupTTgakhyTqaivPphqUv6m87sUabOTyOuYy1n3RuDsPgsHuvHArYVF2QrVVBrZZgvSnTrz7Z/1Ifil2l9MbYciDcblw11gQ4u4Vu7Vlu/Hci4uWYugYit8dhFmQPyI9zRh/JWBgsna2lmZPnkVVj4VqoThQUCB20iIcE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000005, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon Apr 7, 2025 at 6:01 PM UTC, Johannes Weiner wrote: > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -2194,11 +2194,11 @@ try_to_claim_block(struct zone *zone, struct page *page, > * The use of signed ints for order and current_order is a deliberate > * deviation from the rest of this file, to make the for loop > * condition simpler. > - * > - * Return the stolen page, or NULL if none can be found. > */ This commentary is pretty confusing now, there's a block of text that kinda vaguely applies to the aggregate of __rmqueue_steal(), __rmqueue_fallback() and half of __rmqueue(). I think this new code does a better job of speaking for itself so I think we should just delete this block comment and replace it with some more verbosity elsewhere. > + > +/* Try to claim a whole foreign block, take a page, expand the remainder */ Also on the commentary front, I am not a fan of "foreign" and "native": - "Foreign" is already used in this file to mean NUMA-nonlocal. - We already have "start" and "fallback" being used in identifiers as adjectives to describe the mitegratetype concept. I wouldn't say those are _better_, "native" and "foreign" might be clearer, but it's not worth introducing inconsistency IMO. > static __always_inline struct page * > -__rmqueue_fallback(struct zone *zone, int order, int start_migratetype, > +__rmqueue_claim(struct zone *zone, int order, int start_migratetype, > unsigned int alloc_flags) > { > struct free_area *area; [pasting in more context that wasn't in the original diff..] > /* > * Find the largest available free page in the other list. This roughly > * approximates finding the pageblock with the most free pages, which > * would be too costly to do exactly. > */ > for (current_order = MAX_PAGE_ORDER; current_order >= min_order; > --current_order) { IIUC we could go one step further here and also avoid repeating this iteration? Maybe something for a separate patch though? Anyway, the approach seems like a clear improvement, thanks. I will need to take a closer look at it tomorrow, I've run out of brain juice today. Here's what I got from redistributing the block comment and flipping the terminology: diff --git i/mm/page_alloc.c w/mm/page_alloc.c index dfb2b3f508af..b8142d605691 100644 --- i/mm/page_alloc.c +++ w/mm/page_alloc.c @@ -2183,21 +2183,13 @@ try_to_claim_block(struct zone *zone, struct page *page, } /* - * Try finding a free buddy page on the fallback list. - * - * This will attempt to claim a whole pageblock for the requested type - * to ensure grouping of such requests in the future. - * - * If a whole block cannot be claimed, steal an individual page, regressing to - * __rmqueue_smallest() logic to at least break up as little contiguity as - * possible. + * Try to allocate from some fallback migratetype by claiming the entire block, + * i.e. converting it to the allocation's start migratetype. * * The use of signed ints for order and current_order is a deliberate * deviation from the rest of this file, to make the for loop * condition simpler. */ - -/* Try to claim a whole foreign block, take a page, expand the remainder */ static __always_inline struct page * __rmqueue_claim(struct zone *zone, int order, int start_migratetype, unsigned int alloc_flags) @@ -2247,7 +2239,10 @@ __rmqueue_claim(struct zone *zone, int order, int start_migratetype, return NULL; } -/* Try to steal a single page from a foreign block */ +/* + * Try to steal a single page from some fallback migratetype. Leave the rest of + * the block as its current migratetype, potentially causing fragmentation. + */ static __always_inline struct page * __rmqueue_steal(struct zone *zone, int order, int start_migratetype) { @@ -2307,7 +2302,9 @@ __rmqueue(struct zone *zone, unsigned int order, int migratetype, } /* - * Try the different freelists, native then foreign. + * First try the freelists of the requested migratetype, then try + * fallbacks. Roughly, each fallback stage poses more of a fragmentation + * risk. * * The fallback logic is expensive and rmqueue_bulk() calls in * a loop with the zone->lock held, meaning the freelists are @@ -2332,7 +2329,7 @@ __rmqueue(struct zone *zone, unsigned int order, int migratetype, case RMQUEUE_CLAIM: page = __rmqueue_claim(zone, order, migratetype, alloc_flags); if (page) { - /* Replenished native freelist, back to normal mode */ + /* Replenished requested migratetype's freelist, back to normal mode */ *mode = RMQUEUE_NORMAL; return page; }