From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3315C54E66 for ; Wed, 13 Mar 2024 15:04:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 43B53940023; Wed, 13 Mar 2024 11:04:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3EC2E940010; Wed, 13 Mar 2024 11:04:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 28C0D940023; Wed, 13 Mar 2024 11:04:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 15DB5940010 for ; Wed, 13 Mar 2024 11:04:56 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E3449A1646 for ; Wed, 13 Mar 2024 15:04:55 +0000 (UTC) X-FDA: 81892338150.24.B7C0E27 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf23.hostedemail.com (Postfix) with ESMTP id 4F051140023 for ; Wed, 13 Mar 2024 15:04:52 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=u4TINEae; dmarc=none; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710342293; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BNxK5EuN2z1du2u9zHs8dUJAhvJjbhbwG8RbEiAeClc=; b=r6TpnJdZbPshpipdIKhdM1KZsApvFXsVu03tBBAFSEpiEDH9CHVtVItJTXPSVViuW97/He vR86utJSflJKi9RQv5t5iLme8Ctm7sTyYAOpWynUBKY9BAzhEbgan1C2vpZasAh/Xz6tkH vC9vJXGXibcXQ+4sDnk2FNmI9iaTClo= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=u4TINEae; dmarc=none; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710342293; a=rsa-sha256; cv=none; b=nhvUA/WjHQifRwTD+CZ/WJCYrC5gP+UMncrTTzPKC34vKzAae2WOUWx6tqIijwXB7klEgK uKk0GmqgaicXSm5wrW8zGQ1h+hhQD6Zrm3ZiUUhkIYxKTLNJ2XNMU73VRdgqyjsqAxe3q1 6f8mJ8yBDHRHtoM458HwM3nvECPDX8c= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=BNxK5EuN2z1du2u9zHs8dUJAhvJjbhbwG8RbEiAeClc=; b=u4TINEaeR8pBfu92hqQSLLqbw1 brpvAqC+UAYMwivkfgs4750GbK7A0qwui27QEc2QH9iUDu5HAbvxzTVnDcFLyemP/XUIx9MpoFbPi q4IVLXkwKfK0hyxYOhjsEbyDlhqHvslhIMnMIWKJYzFQxuE5veIEox3shpukVhJnJOi9hLp44r+EM DfO73+5LdareoKzomTSUfKrn8QRIKkQrbx5fl0+DUtdOpn8MXP/hU/JOuGLEbRDpiIk2OiOzBuZml hSNWxtuUjaxB+C0meEeMqttlRscGDUvIcxO795C5WNAIzRQjXMNkZ2PfpYOcF19WaReKmjCOY/R0S 0uWwi/1Q==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rkQ9R-00000005dNK-1tqa; Wed, 13 Mar 2024 15:04:17 +0000 Date: Wed, 13 Mar 2024 15:04:17 +0000 From: Matthew Wilcox To: Suren Baghdasaryan Cc: akpm@linux-foundation.org, kent.overstreet@linux.dev, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, roman.gushchin@linux.dev, mgorman@suse.de, dave@stgolabs.net, liam.howlett@oracle.com, penguin-kernel@i-love.sakura.ne.jp, corbet@lwn.net, void@manifault.com, peterz@infradead.org, juri.lelli@redhat.com, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, x86@kernel.org, peterx@redhat.com, david@redhat.com, axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, dennis@kernel.org, jhubbard@nvidia.com, tj@kernel.org, muchun.song@linux.dev, rppt@kernel.org, paulmck@kernel.org, pasha.tatashin@soleen.com, yosryahmed@google.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, andreyknvl@gmail.com, keescook@chromium.org, ndesaulniers@google.com, vvvvvv@google.com, gregkh@linuxfoundation.org, ebiggers@google.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, shakeelb@google.com, songmuchun@bytedance.com, jbaron@akamai.com, aliceryhl@google.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, kernel-team@android.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org Subject: Re: [PATCH v5 20/37] mm: fix non-compound multi-order memory accounting in __free_pages Message-ID: References: <20240306182440.2003814-1-surenb@google.com> <20240306182440.2003814-21-surenb@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240306182440.2003814-21-surenb@google.com> X-Rspamd-Queue-Id: 4F051140023 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: w6uzwjxy9heohdy4guod7tbioe1ks5j7 X-HE-Tag: 1710342292-72118 X-HE-Meta: U2FsdGVkX19vsg8vA1zD9TB8VhbtZhsLce6AUXgqhoyiC4kVNI4CesJ1eYEc3Mfdqz4l1fv+3Yu4W9cA2PujzM2+sZbycmsddA4iZ0xImzT6UrGc77Wlxbc0LiQVN00lo7J/fMERlauII1p9JctjS4FTpkUN0UmHOW8iUyhVbG4V68qrwyumvL5eWqfHwjFXMANw5JXg9oF/pAQlg1AuULVqhN13+vf4RkU4EVr1rUbYukIit+a1capGYGImBUCzXPEEaVSyjkNcFTNNXMyTO0Bv+cblwIvLfyZBeQqU9lbju412pdDSLWufIMLwjkZ5evul5AybZW77ghI3eJX8r7rXPLj3jhZ7Ur2p3eoUxXTCKyk7hV6lRShHTxOu6iIEdEtvgf40qqSxBf8fuG14XJ5jLCZpZo0hGYEX3UzlaUHV1C5zPm+fz+/8s7dyZSe7dEqdu090epoRxHAGPbmctWBR8RZW+4eoe7unZnDcaHBio5ammnbK5j9Ri4LAZP4pRakqy1xhxgrNUuhdMmzTcPyuBxQ8hL4Y0dngKLROsKXCm9tuClf0VKfva5VFMD3yFNrrXvJzu4kloPXcbG4Gg8+ND6JMwmvmIFf68HUDp1HEI5aIYgIQ1wWV/+1THpXKwo67YSz0BzbuFACoKj6MWdFxa6ItncSlcRaXVUGRbHw7pqrVSR/rJbBsbjGhLBIzaTUKQxqe8dlOwv3S6Pse+QhsmdyCQUONOWAR52uObePz/MTMFMjuAWLpeQe4j1SGhp6CvHH6eEq1lZx7XEJhIaUxJp+s+jAT2bqWFuJB8yHbg5ys19FWozo0xUbuBzGSjCGyUiKfjYVBm8em2KkyBe21NrhX+je1SijwzRnLxBUeg6jXzAE/Ch/xETW5CpAYRMqP8lvZvd8aodWQ+EJPX3KCYqFy1VQRc8fXWVO0f5R1BPcjvC7QHrkzvSuWbOicIg5Marq1wNg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Mar 06, 2024 at 10:24:18AM -0800, Suren Baghdasaryan wrote: > When a non-compound multi-order page is freed, it is possible that a > speculative reference keeps the page pinned. In this case we free all > pages except for the first page, which will be freed later by the last > put_page(). However put_page() ignores the order of the page being freed, > treating it as a 0-order page. This creates a memory accounting imbalance > because the pages freed in __free_pages() do not have their own alloc_tag > and their memory was accounted to the first page. To fix this the first > page should adjust its allocation size counter when "tail" pages are freed. It's not "ignored". It's not available! Better wording: However the page passed to put_page() is indisinguishable from an order-0 page, so it cannot do the accounting, just as it cannot free the subsequent pages. Do the accounting here, where we free the pages. (I'm sure further improvements are possible) > +static inline void pgalloc_tag_sub_bytes(struct alloc_tag *tag, unsigned int order) > +{ > + if (mem_alloc_profiling_enabled() && tag) > + this_cpu_sub(tag->counters->bytes, PAGE_SIZE << order); > +} This is a terribly named function. And it's not even good for what we want to use it for. static inline void pgalloc_tag_sub_pages(struct alloc_tag *tag, unsigned int nr) { if (mem_alloc_profiling_enabled() && tag) this_cpu_sub(tag->counters->bytes, PAGE_SIZE * nr); } > +++ b/mm/page_alloc.c > @@ -4697,12 +4697,21 @@ void __free_pages(struct page *page, unsigned int order) > { > /* get PageHead before we drop reference */ > int head = PageHead(page); > + struct alloc_tag *tag = pgalloc_tag_get(page); > > if (put_page_testzero(page)) > free_the_page(page, order); > else if (!head) > - while (order-- > 0) > + while (order-- > 0) { > free_the_page(page + (1 << order), order); > + /* > + * non-compound multi-order page accounts all allocations > + * to the first page (just like compound one), therefore > + * we need to adjust the allocation size of the first > + * page as its order is ignored when put_page() frees it. > + */ > + pgalloc_tag_sub_bytes(tag, order); - else if (!head + else if (!head) { + pgalloc_tag_sub_pages(1 << order - 1); while (order-- > 0) free_the_page(page + (1 << order), order); + } It doesn't need a comment, it's obvious what you're doing.