From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 625A5C433F5 for ; Tue, 17 May 2022 00:34:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C15A16B0072; Mon, 16 May 2022 20:34:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B9D9E6B0073; Mon, 16 May 2022 20:34:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C9FA6B0074; Mon, 16 May 2022 20:34:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 899176B0072 for ; Mon, 16 May 2022 20:34:31 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 65E612069E for ; Tue, 17 May 2022 00:34:31 +0000 (UTC) X-FDA: 79473363942.14.5A49280 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf30.hostedemail.com (Postfix) with ESMTP id 6A0E2800B9 for ; Tue, 17 May 2022 00:34:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652747670; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bFzmceO8lMMO/+kL1LxPWjlpYtFkd84SSop/Z3TUE1w=; b=ZyVc3voblwpUHnC1r0QwFERrUyF1NqE2LGsYveHuWq1tC4mm/h4k3r2cIAl1BNilikDBIs RVw9kfGdOaIeZi0ZCAMfZHLRY5koTSHJsAhgcGzafjDgLLCZnryvLAbbvIEh5Q0PREX3B9 njxKPDffuHns+RJ6hjHXh16u2fDQbGs= Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-589-Poyax5OwO5mKlEn3aYJdCw-1; Mon, 16 May 2022 20:34:29 -0400 X-MC-Unique: Poyax5OwO5mKlEn3aYJdCw-1 Received: by mail-pl1-f198.google.com with SMTP id x4-20020a170902ec8400b0015f0c1ec803so8746457plg.15 for ; Mon, 16 May 2022 17:34:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=bFzmceO8lMMO/+kL1LxPWjlpYtFkd84SSop/Z3TUE1w=; b=WeeP+jt/7AoNCm50wDQE62mzak09tY6hOG+rtedRvGhd74YaJx7z2fKdxm74Ib+MRO ntaImwAVBfkh7a69gqSTjcNZi0AbuYBD9JhCXBshJH1c9wjA8hECEcSrRgWNVHi5A+aC 268sPN0CizimJ5bKhik0igdPXRkWlerQx08wAXq/59RAnd3y1w5cOZlTkA294Ry6Z3zD CZ/bnxlD3f/U0Qx8lXug6HmUd2citk0/9xPGkMp9cMiPPp6deakWihPT+jaCbRYwWKJz 3XVX7g9E4pfXLxD1mefN7kx5XdOZcWmdN7vde1ndIgCuJ6YSR/RgjPp4azRzI45FR1sk S5tg== X-Gm-Message-State: AOAM5322OxWuuR84WleJKlNY5WcrAXHnhbFofLOUSllt6tDEJhH++rX8 15xIUKHgUugchgrNGEwpG5b+AVfYeA1M3V7ogH5mQEzpUucRBcxd9F/TwsT90UCNGCRog3/Vih/ 21Vf3IHeXh8WCY274vTa7YhRKNnCOgRezxlbBfrhnBWx5xsFNyJz2J/HqSU9r X-Received: by 2002:a63:91c4:0:b0:3c6:2334:3ca1 with SMTP id l187-20020a6391c4000000b003c623343ca1mr17580713pge.53.1652747667748; Mon, 16 May 2022 17:34:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx+xhUzIhGcYO+zKBk4VrEJOB5fFKVxWWc4Sx5d0Gme43WJtY7QUsx+wllrZBb+F2NrxKnQsg== X-Received: by 2002:a63:91c4:0:b0:3c6:2334:3ca1 with SMTP id l187-20020a6391c4000000b003c623343ca1mr17580681pge.53.1652747667319; Mon, 16 May 2022 17:34:27 -0700 (PDT) Received: from [10.72.12.136] ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id d4-20020a170902e14400b001619cec6f95sm820122pla.257.2022.05.16.17.34.24 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 16 May 2022 17:34:26 -0700 (PDT) Subject: Re: Freeing page flags To: Matthew Wilcox , =?UTF-8?Q?Lu=c3=ads_Henriques?= Cc: Jeff Layton , Josef Bacik , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org References: <87sfpd22kq.fsf@brahms.olymp> From: Xiubo Li Message-ID: <1a86330d-d1a4-7c74-241e-11cc263254cf@redhat.com> Date: Tue, 17 May 2022 08:34:21 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Stat-Signature: iy3y1d4ryk5nwz4m7io4hbthiag186e9 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 6A0E2800B9 X-Rspam-User: Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ZyVc3vob; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf30.hostedemail.com: domain of xiubli@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=xiubli@redhat.com X-HE-Tag: 1652747648-256856 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 5/13/22 8:53 PM, Matthew Wilcox wrote: > On Fri, May 13, 2022 at 10:40:05AM +0100, Luís Henriques wrote: >> Matthew Wilcox writes: >> >>> On Thu, May 12, 2022 at 10:41:41PM -0400, Josef Bacik wrote: >>>> On Thu, May 12, 2022 at 09:54:59PM +0100, Matthew Wilcox wrote: >>>>> The LWN writeup [1] on merging the MGLRU reminded me that I need to send >>>>> out a plan for removing page flags that we can do without. >>>>> >>>>> 1. PG_error. It's basically useless. If the page was read successfully, >>>>> PG_uptodate is set. If not, PG_uptodate is clear. The page cache >>>>> doesn't use PG_error. Some filesystems do, and we need to transition >>>>> them away from using it. >>>>> >>>> What about writes? A cursory look shows we don't clear Uptodate if we fail to >>>> write, which is correct I think. The only way to indicate we had a write error >>>> to check later is the page error. >>> On encountering a write error, we're supposed to call mapping_set_error(), >>> not SetPageError(). >>> >>>>> 2. PG_private. This tells us whether we have anything stored at >>>>> page->private. We can just check if page->private is NULL or not. >>>>> No need to have this extra bit. Again, there may be some filesystems >>>>> that are a bit wonky here, but I'm sure they're fixable. >>>>> >>>> At least for Btrfs we serialize the page->private with the private_lock, so we >>>> could probably just drop PG_private, but it's kind of nice to check first before >>>> we have to take the spin lock. I suppose we can just do >>>> >>>> if (page->private) >>>> // do lock and check thingy >>> That's my hope! I think btrfs is already using folio_attach_private() / >>> attach_page_private(), which makes everything easier. Some filesystems >>> still manipulate page->private and PagePrivate by hand. >> In ceph we've recently [1] spent a bit of time debugging a bug related >> with ->private not being NULL even though we expected it to be. The >> solution found was to replace the check for NULL and use >> folio_test_private() instead, but we _may_ have not figured the whole >> thing out. >> >> We assumed that folios were being recycled and not cleaned-up. The values >> we were seeing in ->private looked like they were some sort of flags as >> only a few bits were set (e.g. 0x0200000): >> >> [ 1672.578313] page:00000000e23868c1 refcount:2 mapcount:0 mapping:0000000022e0d3b4 index:0xd8 pfn:0x74e83 >> [ 1672.581934] aops:ceph_aops [ceph] ino:10000016c9e dentry name:"faed" >> [ 1672.584457] flags: 0x4000000000000015(locked|uptodate|lru|zone=1) >> [ 1672.586878] raw: 4000000000000015 ffffea0001d3a108 ffffea0001d3a088 ffff888003491948 >> [ 1672.589894] raw: 00000000000000d8 0000000000200000 00000002ffffffff 0000000000000000 >> [ 1672.592935] page dumped because: VM_BUG_ON_FOLIO(1) >> >> [1] https://lore.kernel.org/all/20220508061543.318394-1-xiubli@redhat.com/ > I remember Jeff asking me about this problem a few days ago. A folio > passed to you in ->dirty_folio() or ->invalidate_folio() belongs to > your filesystem. Nobody else should be storing to the ->private field; > there's no race that could lead to it being freed while you see it. > There may, of course, be bugs that are overwriting folio->private, but > it's definitely not supposed to happen. I agree with you that it looks > like a bit has been set (is it possibly bad RAM?) I don't think so. Please see the values I saw from my tests locally below. > > We do use page->private in the buddy allocator, but that stores the order > of the page; it wouldn't be storing 1<<21. PG flag 21 is PG_mlocked, > which seems like a weird one to be setting in the wrong field, so probably > not that. > > Is it always bit 21 that gets set? > No, from my test I can reproduce it locally very easy and almost every time the values were different, which were random values, just like: 0000000000040000,0000000000070000,000000000002e0000... More detail please see https://tracker.ceph.com/issues/55421. I am sure that for all the none zero values the lower 16 bits were 0000. -- Xiubo