From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1124AC0015E for ; Tue, 15 Aug 2023 19:21:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E65D940021; Tue, 15 Aug 2023 15:21:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5957C8D0001; Tue, 15 Aug 2023 15:21:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 43640940021; Tue, 15 Aug 2023 15:21:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 318398D0001 for ; Tue, 15 Aug 2023 15:21:29 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 02F18A01AE for ; Tue, 15 Aug 2023 19:21:28 +0000 (UTC) X-FDA: 81127307898.02.87E166E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf09.hostedemail.com (Postfix) with ESMTP id D9A3F14001D for ; Tue, 15 Aug 2023 19:21:26 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=idFASFay; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf09.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692127286; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cCj7FTFwL6F6V9/SO6Bj9kXHRFGJMD24dS3CtJ8ffZE=; b=IU8EKOUf6x56CWpqxfH+9mAr+RuhGZypMALjZxJ5il++12q1/TMDVD0LQZSmkjljsdBQne bEvbRAoraS0e2zMjxidTFsnBzI39cYFYo8aJJTMPzHO40UbVa2sXhteYh9sjOHxJnEpDOZ ckdGfK0wpSgC6PsccVAYUo41RLMM+hg= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=idFASFay; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf09.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692127286; a=rsa-sha256; cv=none; b=fG9D074X7OUxnWUIJD99b0XrLhU7X1ZxSuZK8lOEOnqimKJHPem3u5xNPAKeXbSci1yPcA R1bz81Cn9fqbc/N52sH8gX4oYDTFiZhYzUtW6OaHvwzNYdVdCOvQYqQmr9/Lu/41IRpunf 4UDePuSeZgb3yEnfKhInfYpwvT2zZUk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692127286; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=cCj7FTFwL6F6V9/SO6Bj9kXHRFGJMD24dS3CtJ8ffZE=; b=idFASFayQxZvIXz3bSJJIDTEwoXMkjo2rzMKd1JKQvnb69Gh8A8dvt7+wGHLhHSiTAfaNm 0STjauDUtMtvOU7XduoKxexYi2liATFU6q0JE4E8h6sqo61wGrPtMx9TTSsEiRIzsYaZVT cx90beB4qOzNMxEI8empghJVVyVUvIA= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-144-qlH7q7zCO2OfI1ZbhoR8lA-1; Tue, 15 Aug 2023 15:21:24 -0400 X-MC-Unique: qlH7q7zCO2OfI1ZbhoR8lA-1 Received: by mail-qt1-f197.google.com with SMTP id d75a77b69052e-40ec97d5b60so16796301cf.1 for ; Tue, 15 Aug 2023 12:21:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692127284; x=1692732084; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=cCj7FTFwL6F6V9/SO6Bj9kXHRFGJMD24dS3CtJ8ffZE=; b=cvtQefjRmpbh/3Xr6QUGAQwMcXaiKpgFaCoosiZdVMJaOPawHGbpZ3ZrHV5ItyoNlA WfBWLJ4+u4y1CgdbU6Evcgzrvdkfcg844bNDygtpHWOZTkR23/zQ7Ywx+I1ieuidSGYD 7K/WdL3BVjRYApalZYDDRC6qz3QMcORE1jBV1mG3vk+6se+O8vdfphadKdfvKuoy7WLM dlEYJK+VFVAkZZ8tayDg1zPFQK3+QMb96uo9DZKny3jNZ94kVFzLvckCU8Km/+zYBjog vuAQaVNFf6Ia59E63OPIfDTurEekzFbB+HbHdRHmuU6li0rIOTzsY2CymdannhI3yuK3 O5LA== X-Gm-Message-State: AOJu0Yyzum7j2IZUH1+uO3bhN2gEheS4QKOQWNN0kPCkvH6O1Oy3t1wR j3K9oVUw12laU8VjLFapQ/JR5wxrDgDR5uByKsWQgKc8V1godrr1YfWWDWC67OMBbmPcRTfbKeg U6RmtBykvozc= X-Received: by 2002:a05:6214:dc6:b0:63c:7427:e7e9 with SMTP id 6-20020a0562140dc600b0063c7427e7e9mr7813154qvt.6.1692127283936; Tue, 15 Aug 2023 12:21:23 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEIwXaTyKTXy6svtJfFPnFrTAlBeXjdhHuUb89o0ka25yDqZPveTO2U3SkZ8a2bZNLetaPAoA== X-Received: by 2002:a05:6214:dc6:b0:63c:7427:e7e9 with SMTP id 6-20020a0562140dc600b0063c7427e7e9mr7813144qvt.6.1692127283632; Tue, 15 Aug 2023 12:21:23 -0700 (PDT) Received: from x1n (cpe5c7695f3aee0-cm5c7695f3aede.cpe.net.cable.rogers.com. [99.254.144.39]) by smtp.gmail.com with ESMTPSA id s14-20020a0cdc0e000000b006431027ac44sm1915699qvk.83.2023.08.15.12.21.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Aug 2023 12:21:23 -0700 (PDT) Date: Tue, 15 Aug 2023 15:21:14 -0400 From: Peter Xu To: "Matthew Wilcox (Oracle)" Cc: Andrew Morton , Jens Axboe , io-uring@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH 9/9] mm: Free up a word in the first tail page Message-ID: References: <20230815032645.1393700-1-willy@infradead.org> <20230815032645.1393700-10-willy@infradead.org> MIME-Version: 1.0 In-Reply-To: <20230815032645.1393700-10-willy@infradead.org> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Rspamd-Queue-Id: D9A3F14001D X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 1b5ks69pgyarwt6ocbmgtof4jgs4i8qk X-HE-Tag: 1692127286-888401 X-HE-Meta: U2FsdGVkX1/XX08BjoixvOGGjPzN0EmHm1RWs3HSysBRfBE2fvQRPpaU9th++RNmXVZKsX7Of1M29qMLsnVhjE8TkGpoeVIQ5S9Wf3H+fmHSi7ucjP83W0eTRrXJqbB3QZAWexHk9c6zRCZMMfAfRWEzbPMmnEU3uZvAsgF9tYg0u5IdmFuWhTbLgB31y2JDUeyIrfF2l59mhJ0SrUVRWQsNEqW7TvUfqLiE2OZKBcgK6NyUSbAKrF+AY5KyWxn2IJkz11TJD+Vbh6J2H+Y+OSYoN9TUdFKPtOcZ9Q4BCGfbu2L846SDJJaiRKoX+Ha/ymM9u6/O5igahoicP9Utop1ZRfRq7Hik3ZR1WtlLZ98vZ49AcIsuh+DBSmirlsyvGv7uLRLh9XJhazKl/40YiHG91DckvnFYF1orX5vQ8070QWY+1lG1h0m6TotK1W753PnxQGrkOzM5QTFUGto3cMjwAfMpJMWakmBx/QvLdKG0bJOQFKn9Zcn+MiNH6hVn5a3bUQOfn+skEQhTw5mboGyho94GHy5QwxmMYO7bH3O/DbGg+lSD6732YlAt9ia/w9a7R+wR5nRMsoITjWkycz0PuTzZlKbTXD4oaXVG9Fsduz7+QOh0g5qkmSIXH8Sivd4gLLwx2gN+zHWJVjYn/2ysGbcariyF+YceMEtsDaXr1yD43YkMCssHNnHTLf7AnWahw3kxUzTSzY0RTPV+hwM1coEBTbLwetNeJsBGptN5cJMWz6Wi1d/XvBpMQXXmJwft0WvJHvdXbStfrONuxLsyeyt1lpODNCdretvR5FkxRD+/KggPEl9GyovHqwr5sqqq7kvI8go7pAuNrWd89aUDV+OonrjTYMmtUrQJsHKVTZW6brZe1Z1r1C2InILh0LeGiLEqBNmqpaMVL+P0Kq9MK+B0oL3DghY4u9cG/cOdRftNDLLdqtTNFvzvxzPe4dTOYwVVyqdRhDi0vQC pAeE+VHY geTQHudsLeb2csMP1A3zzSA5S+Y24dYMi2TnSCT81DcsE2BWtWWJbds527m1wkAWNfNM2IuuIrLDat0evL3ZfkDxK/J6u5uDh3zJkl5fBUBpEYLa11aAVed8HHNEwupqvy7x1tLepIdt30lWkbHGOhsBPR5dx5/NV8rOdGsCnjAnASnJDp1lkdGQEWn1wzhMAYfrdk0dMkk+0FqfRT4WdGYHPnOu3jg5jpW1xtTObmTuPyw6zIyCVh4fXOB6WgWeBE1EXBRDK2R4Sct9EZX6PreH1SKrbI2Wchd8k8dmmg0K23MqSu+4IcCNWdpPTa/qTTEfCy0Bo0SmN9N+aowlLLM2Q6ZsWQ5OneeNKzYZI3ojNrCBdXB3Mm+omv5PXKAf7Ea3/XK2Aye5Lt4NUL33KGloJaCl8mErfdMjUYc7/jjtRyZkAYNppEDaz30ziI0bX+f3xPneoTjbeH3A2Q3LHbNxzlVZnNTjJd41DYI/9nAT8QhY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Aug 15, 2023 at 04:26:45AM +0100, Matthew Wilcox (Oracle) wrote: > Store the folio order in the low byte of the flags word in the first > tail page. This frees up the word that was being used to store the > order and dtor bytes previously. > > Signed-off-by: Matthew Wilcox (Oracle) > --- > include/linux/mm.h | 10 +++++----- > include/linux/mm_types.h | 3 +-- > kernel/crash_core.c | 1 - > mm/internal.h | 2 +- > mm/page_alloc.c | 4 +++- > 5 files changed, 10 insertions(+), 10 deletions(-) > > diff --git a/include/linux/mm.h b/include/linux/mm.h > index cf0ae8c51d7f..85568e2b2556 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -1028,7 +1028,7 @@ struct inode; > * compound_order() can be called without holding a reference, which means > * that niceties like page_folio() don't work. These callers should be > * prepared to handle wild return values. For example, PG_head may be > - * set before _folio_order is initialised, or this may be a tail page. > + * set before the order is initialised, or this may be a tail page. > * See compaction.c for some good examples. > */ > static inline unsigned int compound_order(struct page *page) > @@ -1037,7 +1037,7 @@ static inline unsigned int compound_order(struct page *page) > > if (!test_bit(PG_head, &folio->flags)) > return 0; > - return folio->_folio_order; > + return folio->_flags_1 & 0xff; > } > > /** > @@ -1053,7 +1053,7 @@ static inline unsigned int folio_order(struct folio *folio) > { > if (!folio_test_large(folio)) > return 0; > - return folio->_folio_order; > + return folio->_flags_1 & 0xff; > } > > #include > @@ -2025,7 +2025,7 @@ static inline long folio_nr_pages(struct folio *folio) > #ifdef CONFIG_64BIT > return folio->_folio_nr_pages; > #else > - return 1L << folio->_folio_order; > + return 1L << (folio->_flags_1 & 0xff); > #endif > } > > @@ -2043,7 +2043,7 @@ static inline unsigned long compound_nr(struct page *page) > #ifdef CONFIG_64BIT > return folio->_folio_nr_pages; > #else > - return 1L << folio->_folio_order; > + return 1L << (folio->_flags_1 & 0xff); > #endif > } > > diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h > index d45a2b8041e0..659c7b84726c 100644 > --- a/include/linux/mm_types.h > +++ b/include/linux/mm_types.h > @@ -282,7 +282,6 @@ static inline struct page *encoded_page_ptr(struct encoded_page *page) > * @_refcount: Do not access this member directly. Use folio_ref_count() > * to find how many references there are to this folio. > * @memcg_data: Memory Control Group data. > - * @_folio_order: Do not use directly, call folio_order(). > * @_entire_mapcount: Do not use directly, call folio_entire_mapcount(). > * @_nr_pages_mapped: Do not use directly, call folio_mapcount(). > * @_pincount: Do not use directly, call folio_maybe_dma_pinned(). > @@ -334,8 +333,8 @@ struct folio { > struct { > unsigned long _flags_1; > unsigned long _head_1; > + unsigned long _folio_avail; This can just be dropped? Having this single field as "avail" is weird, without mentioning the rest, IMHO. We can have a separate patch to resolve what's available, either you can leave that to my series, or if you dislike that you can propose what you've replied to my cover letter but add all the available bits. > /* public: */ > - unsigned char _folio_order; > atomic_t _entire_mapcount; > atomic_t _nr_pages_mapped; > atomic_t _pincount; > diff --git a/kernel/crash_core.c b/kernel/crash_core.c > index 934dd86e19f5..693445e1f7f6 100644 > --- a/kernel/crash_core.c > +++ b/kernel/crash_core.c > @@ -455,7 +455,6 @@ static int __init crash_save_vmcoreinfo_init(void) > VMCOREINFO_OFFSET(page, lru); > VMCOREINFO_OFFSET(page, _mapcount); > VMCOREINFO_OFFSET(page, private); > - VMCOREINFO_OFFSET(folio, _folio_order); > VMCOREINFO_OFFSET(page, compound_head); > VMCOREINFO_OFFSET(pglist_data, node_zones); > VMCOREINFO_OFFSET(pglist_data, nr_zones); > diff --git a/mm/internal.h b/mm/internal.h > index e3d11119b04e..c415260c1f06 100644 > --- a/mm/internal.h > +++ b/mm/internal.h > @@ -407,7 +407,7 @@ static inline void folio_set_order(struct folio *folio, unsigned int order) > if (WARN_ON_ONCE(!order || !folio_test_large(folio))) > return; > > - folio->_folio_order = order; > + folio->_flags_1 = (folio->_flags_1 & ~0xffUL) | order; > #ifdef CONFIG_64BIT > folio->_folio_nr_pages = 1U << order; > #endif > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > index 9fe9209605a5..0e0e0d18a81b 100644 > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -1115,8 +1115,10 @@ static __always_inline bool free_pages_prepare(struct page *page, > > VM_BUG_ON_PAGE(compound && compound_order(page) != order, page); > > - if (compound) > + if (compound) { > ClearPageHasHWPoisoned(page); > + page[1].flags &= ~0xffUL; Could we hide the hard-coded 0xff in some way? One easy way would be using a macro with a bunch of helpers, like folio_set|get|clear_order(). The other way is maybe we can also define _flags_1 an enum, where we can just move over the compound_order field at offset 0? But I'm not sure how that looks like at last. Thanks, > + } > for (i = 1; i < (1 << order); i++) { > if (compound) > bad += free_tail_page_prepare(page, page + i); > -- > 2.40.1 > > -- Peter Xu