From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C5DAC54E58 for ; Mon, 25 Mar 2024 09:46:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8C0426B0087; Mon, 25 Mar 2024 05:46:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8707E6B0089; Mon, 25 Mar 2024 05:46:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 738356B008A; Mon, 25 Mar 2024 05:46:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6224F6B0087 for ; Mon, 25 Mar 2024 05:46:37 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 02494140AE3 for ; Mon, 25 Mar 2024 09:46:36 +0000 (UTC) X-FDA: 81935081634.27.F69F420 Received: from out-189.mta0.migadu.com (out-189.mta0.migadu.com [91.218.175.189]) by imf30.hostedemail.com (Postfix) with ESMTP id 04E1B80002 for ; Mon, 25 Mar 2024 09:46:34 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=TDnhqOlC; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf30.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.189 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711359995; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3kW6CpvNLcQN5MKehXI/yVTzkUIvW6nXEzppExY7bng=; b=zyyyiNn1165XufJK6RQRZr87+y61FadAEAPH3ayMFDXg3WEq/6aulnJmW8nHhSw1295oXO 5GmkCOBksBRG0PumbvWBuVaYVMuznPnorBPiVRawTkLIdrK52968Y+3YeNJlAH7wy6mur0 uiiJKENxewK0GFL6Iazqesvx2YfZ21Y= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=TDnhqOlC; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf30.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.189 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711359995; a=rsa-sha256; cv=none; b=jAUfpVA2k2WF+UI5uxnBS8TZs0inLeB1zMhkT7JKtSAmx8LiZXxMbogzZbXH1mct85PgfT yZSLfYUzU7pWF6090C6MhoKP+Xe6WdN0EIdp08BYrIy3dQKBzYuRZxQs9BMDG+hu9vLMNy wvdZTupjcH+0rax5WXuLLmLObAbyjbE= Message-ID: <339639f3-4334-44a8-a811-d20b3c578f74@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1711359990; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3kW6CpvNLcQN5MKehXI/yVTzkUIvW6nXEzppExY7bng=; b=TDnhqOlCJfkHT730fvX2kvpKxlweFV/uEXDJpP84aMp90lyehm7owaGbrJ+jekWYRCPeG4 28hTF/JVa/AjbHIbA+xixil1gG5rENt2vG0wlofv2lGI2y3QOt2UTGSi6NwFIkh8Rbv2Zz wY861xo6ZuEhsRS4oHE4k/lDMJwCCQM= Date: Mon, 25 Mar 2024 17:46:21 +0800 MIME-Version: 1.0 Subject: Re: [PATCH] mm: zswap: fix data loss on SWP_SYNCHRONOUS_IO devices Content-Language: en-US To: Yosry Ahmed Cc: Barry Song <21cnbao@gmail.com>, Johannes Weiner , Andrew Morton , Zhongkun He , Chengming Zhou , Chris Li , Nhat Pham , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kairui Song References: <20240324210447.956973-1-hannes@cmpxchg.org> <1e7ce417-b9dd-4d62-9f54-0adf1ccdae35@linux.dev> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 04E1B80002 X-Stat-Signature: j9m5m6kdpztbamu55gdp7n4hht5auja3 X-Rspam-User: X-HE-Tag: 1711359994-434568 X-HE-Meta: U2FsdGVkX19adkNpqdtqGneJe5xqO18gRCRvvl989mz2V1AG0fWEsk9ZXTMTPMWTrEdL1wFlH4lNb1JHMWxBHIswAOkXxfzUIYgbHd5Qsu4t5GddYefWptUVIwOWHqwyATOqej6W+MnbUsSWi0C0mpX8y09hBO5Tr7N7Ejsw7pb8nUF5Vm8V6Ls3SYFL8TxQGjtKWJiYOLqff3mu0R23G+cd0KVUuLp41cH4nyN/DfehS+eA8Fr3fQnuv35RFwjDAdUZgU9qhHPcYiJgspNcvbnleWsvxnQnuuReBGCVkAjP54XY1FuODYbrFrNmK2wgH3JRAJai4qjOxOWfvPxH8SUKpoWUdsMTqLpZxuhBOQITxQeH9GZ7/dozTEaaRXJOdxwxyZOia1TRh4QFTgWLHSwPaLEio1Vm1c4ME4+DdMwGzDqV7XwW8Gx0GVvVU/L53d9+kwuwPHztvaNfIoqLYBNnx8TkH6Gheea5rRs/spZLO1ymYRFcRI8XyEsyTIPwiLQPlXoY/IBBQFprwVsFHAmx92EWVV8XUBgxbOvCmD+xiBacyO3vcSBjKWguQeMpnftQGxX0q91GabhxrJyUOgOTO/DKyOmghYYaQ53BVVnB75ifTItjNlGLqm3e6wwvwiSD4A/z35U/yzUBVys7XNn46/nxBPyHYMzgVvwfG6oRdRewSvasWa2uWVkrRYiiCkfASGvQuh/C0dkFwyyAzTVLCWkZxvhn7Y/HitGGi3Z8ENXVCpYSKZHCVhEU12NzTBgYGw6SuKaQFRSAojk6/Q4MYk5851bX7bUmbygf/K0cTVrEvlNu6RBiqi6ClDncax0uHN/BT9l2KJ+NstQQrgo4ti36gUfb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/3/25 17:40, Yosry Ahmed wrote: > On Mon, Mar 25, 2024 at 2:22 AM Chengming Zhou wrote: >> >> On 2024/3/25 16:38, Yosry Ahmed wrote: >>> On Mon, Mar 25, 2024 at 12:33 AM Chengming Zhou >>> wrote: >>>> >>>> On 2024/3/25 15:06, Yosry Ahmed wrote: >>>>> On Sun, Mar 24, 2024 at 9:54 PM Barry Song <21cnbao@gmail.com> wrote: >>>>>> >>>>>> On Mon, Mar 25, 2024 at 10:23 AM Yosry Ahmed wrote: >>>>>>> >>>>>>> On Sun, Mar 24, 2024 at 2:04 PM Johannes Weiner wrote: >>>>>>>> >>>>>>>> Zhongkun He reports data corruption when combining zswap with zram. >>>>>>>> >>>>>>>> The issue is the exclusive loads we're doing in zswap. They assume >>>>>>>> that all reads are going into the swapcache, which can assume >>>>>>>> authoritative ownership of the data and so the zswap copy can go. >>>>>>>> >>>>>>>> However, zram files are marked SWP_SYNCHRONOUS_IO, and faults will try >>>>>>>> to bypass the swapcache. This results in an optimistic read of the >>>>>>>> swap data into a page that will be dismissed if the fault fails due to >>>>>>>> races. In this case, zswap mustn't drop its authoritative copy. >>>>>>>> >>>>>>>> Link: https://lore.kernel.org/all/CACSyD1N+dUvsu8=zV9P691B9bVq33erwOXNTmEaUbi9DrDeJzw@mail.gmail.com/ >>>>>>>> Reported-by: Zhongkun He >>>>>>>> Fixes: b9c91c43412f ("mm: zswap: support exclusive loads") >>>>>>>> Cc: stable@vger.kernel.org [6.5+] >>>>>>>> Signed-off-by: Johannes Weiner >>>>>>>> Tested-by: Zhongkun He >>>>>> >>>>>> Acked-by: Barry Song >>>>>> >>>>>>> >>>>>>> Do we also want to mention somewhere (commit log or comment) that >>>>>>> keeping the entry in the tree is fine because we are still protected >>>>>>> from concurrent loads/invalidations/writeback by swapcache_prepare() >>>>>>> setting SWAP_HAS_CACHE or so? >>>>>> >>>>>> It seems that Kairui's patch comprehensively addresses the issue at hand. >>>>>> Johannes's solution, on the other hand, appears to align zswap behavior >>>>>> more closely with that of a traditional swap device, only releasing an entry >>>>>> when the corresponding swap slot is freed, particularly in the sync-io case. >>>>> >>>>> It actually worked out quite well that Kairui's fix landed shortly >>>>> before this bug was reported, as this fix wouldn't have been possible >>>>> without it as far as I can tell. >>>>> >>>>>> >>>>>> Johannes' patch has inspired me to consider whether zRAM could achieve >>>>>> a comparable outcome by immediately releasing objects in swap cache >>>>>> scenarios. When I have the opportunity, I plan to experiment with zRAM. >>>>> >>>>> That would be interesting. I am curious if it would be as >>>>> straightforward in zram to just mark the folio as dirty in this case >>>>> like zswap does, given its implementation as a block device. >>>>> >>>> >>>> This makes me wonder who is responsible for marking folio dirty in this swapcache >>>> bypass case? Should we call folio_mark_dirty() after the swap_read_folio()? >>> >>> In shrink_folio_list(), we try to add anonymous folios to the >>> swapcache if they are not there before checking if they are dirty. >>> add_to_swap() calls folio_mark_dirty(), so this should take care of >> >> Right, thanks for your clarification, so should be no problem here. >> Although it was a fix just for MADV_FREE case. >> >>> it. There is an interesting comment there though. It says that PTE >>> should be dirty, so unmapping the folio should have already marked it >>> as dirty by the time we are adding it to the swapcache, except for the >>> MADV_FREE case. >> >> It seems to say the folio will be dirtied when unmap later, supposing the >> PTE is dirty. > > Oh yeah it could mean that the folio will be dirted later. > >> >>> >>> However, I think we actually unmap the folio after we add it to the >>> swapcache in shrink_folio_list(). Also, I don't immediately see why >>> the PTE would be dirty. In do_swap_page(), making the PTE dirty seems >> >> If all anon pages on LRU list are faulted by write, it should be true. >> We could just use the zero page if faulted by read, right? > > This applies for the initial fault that creates the folio, but this is > a swap fault. It could be a read fault and in that case we still need > to make the folio dirty because it's not in the swapcache and we need > to write it out if it's reclaimed, right? Yes, IMHO I think it should be marked as dirty here. But it should be no problem with that unconditional folio_mark_dirty() in add_to_swap(). Not sure if there are other issues. > >> >>> to be conditional on the fault being a write fault, but I didn't look >>> thoroughly, maybe I missed it. It is also possible that the comment is >>> just outdated. >> >> Yeah, dirty is only marked on write fault. >> >> Thanks.