From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA334D3ABD3 for ; Mon, 11 Nov 2024 17:40:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 448D96B0096; Mon, 11 Nov 2024 12:40:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3F89F6B0098; Mon, 11 Nov 2024 12:40:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2BFC26B0099; Mon, 11 Nov 2024 12:40:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 098E06B0096 for ; Mon, 11 Nov 2024 12:40:01 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7C26F1A1AD8 for ; Mon, 11 Nov 2024 17:40:00 +0000 (UTC) X-FDA: 82774525260.07.9FB8462 Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) by imf22.hostedemail.com (Postfix) with ESMTP id D5CB3C0023 for ; Mon, 11 Nov 2024 17:39:07 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=wLP45FV1; dmarc=none; spf=pass (imf22.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.45 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731346736; a=rsa-sha256; cv=none; b=2+vM7NSJ2POooT6mdeGy0YWo64Q6ssmrtfGy8kw9hmRj8bh16qgDjgC6ZQCb0BTerRcaVY as7GIcTvMEog1fIOZRz74wk47MHu2EG26V3c544Se6VKy2je7B0XF6qxjIiYVTcPaHh40u FkyY6coM4Wbsr2ZCnzdYxC8G1ZO0c44= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=wLP45FV1; dmarc=none; spf=pass (imf22.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.45 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731346736; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zornXYrQK1q5jDp72wLGbGUMhA7z4OfmXLBGicUzQUM=; b=OP9Ni4hw2cAsRu1GmI+XglSY8Ors/tdEpKdPDGwXgnPVMGoHtKggAIC+zds/XxWA2PQJOo rwH/BAMwLDIZbuaRdkKixH90b5kOhfzuG7tk6oqHPZ5hpLa4+HR3UMvbK2VN/W2cPTXM0S dNoiyR16VSxt/a7VBTNdixxrptRYtZI= Received: by mail-ot1-f45.google.com with SMTP id 46e09a7af769-718998df806so2336930a34.1 for ; Mon, 11 Nov 2024 09:39:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731346797; x=1731951597; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=zornXYrQK1q5jDp72wLGbGUMhA7z4OfmXLBGicUzQUM=; b=wLP45FV1CGDkg0L61JZ9HH3RHHV+xmfaNrQdkGSF4e3OSDGpVQM4/RXBokyR4bORVh Y0z7jzDJL6P4pdeTgDpcf3YOSu+ZzOt9qNfMr0eggQ31yANIwEjf/x6YG2sJcSdsTyDG 3VuaauTy0+S+O4yy0USxkto1CSn0JDBLzJhXeJIVxTgn+2I79G1DZRiRfb9DNN72CpC3 2FfMocIAEVT0aF1ZiuULQN4nPeVt9mau71EdG9AeRtR8Y/z5JsLj1N79iukYwEeVfBtw /R8AE4ZzVdbwdZI72yNbDo0ONxJ2ulpdE9dQvctNF4oxIWQKzq5uK/oxUCnOcvwZzxi+ inTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731346797; x=1731951597; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zornXYrQK1q5jDp72wLGbGUMhA7z4OfmXLBGicUzQUM=; b=TvF5D58WoWRZs65LoNOkWbkaOmJ3kHydJhnhUb7ShbzK8LzLnIC9Pxu8KRP4T06C2l 3LD5n2JCvcBjt0nEMRw6e1+oCnZ1aEAyOqGZfRqSeggQheUANV9U0GcUmQ8TDZPWmQz1 BW24qbnCSlvO4j5GzB5txtH9ncqIWM6GFSy3o1IO5Kk886vJSMFaeeQSL3N7c4IvMDEb rEOM5pqQGfUgXi0SfSZRp4DdZPR0syTqwSlQ8Ek7PCUw2Ux+qkVfWiWcFCNfNkE+kE3l VppQn8U6D2aMJI25IYOnficL1tUQhy7WznFfTBQTz2ryH+edU148xsg1LlgC0YFNXZ49 KJxg== X-Gm-Message-State: AOJu0YyRleIgDAh2CvITADCLAIznGjTb+gveS05EOKWGUBVBwo9Y7F4k RE/UpR6cXeDp8lnKuN+FyuTlIHHiwPJkqzn1rAtxTLADC+AuMozAHml2yT82HagaVA33cTcOeoO sj6g= X-Google-Smtp-Source: AGHT+IG4NBeRv1XZA/CiIhXb6FBtUBNiBbImRao8a/5WPPZ6TVlAlQmjjSke+YET1UaXJNr4aEjOnQ== X-Received: by 2002:a9d:7856:0:b0:718:c2e:a186 with SMTP id 46e09a7af769-71a1c2b11fdmr9515988a34.26.1731346797472; Mon, 11 Nov 2024 09:39:57 -0800 (PST) Received: from [192.168.1.116] ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-71a1092c14fsm2308234a34.72.2024.11.11.09.39.56 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 11 Nov 2024 09:39:56 -0800 (PST) Message-ID: <3c904d74-c685-4f8a-bc1d-edc24da59fa5@kernel.dk> Date: Mon, 11 Nov 2024 10:39:55 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCHSET v2 0/15] Uncached buffered IO To: Matthew Wilcox Cc: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org References: <20241110152906.1747545-1-axboe@kernel.dk> Content-Language: en-US From: Jens Axboe In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Queue-Id: D5CB3C0023 X-Rspamd-Server: rspam01 X-Stat-Signature: pjqwmruy7y3f16mxz9fnbsfrjzcrws1o X-HE-Tag: 1731346747-976298 X-HE-Meta: U2FsdGVkX198S6k0OcA3LPxFjt27WcpV7OGO3RRtiSYN+KV8FkGNPfBAWKPuKPIqJrSbx2xZYFt3DB1pV8QVH2zgbTfCyI9S3yaeJ9+eQQI8iYcoBiGvAg6/A9S8YmCpB33V/Y328NjOiJrFug0zS1ayf3ALOLC5xPes2q7wSbko7Jqr00na7yr3Io0PcPv8rFDFh4UizgXXPltsJitTBmD8yu79pbmC11eZjcrWihhvPIfyZLqlxLb+EdkXMcaDAT++hr7Z9VScbSbTJ0I+T0K24zqzrusPmFSvdoji2O8PvfAKseXTYIgM4eNxWh2GsRd7Rq7sMLObUHHU8NT2ahtgpCm0MbO4sz4HNCQ25XjydP712f3fl9jCAWO2uyGpwKmRlYe/hURlDk4Yk99fWnj7pPfErv8qCcOq492s6z8OJ1dDP+3zFpIqS6Ce1Gm6jDsFMGc5UvNdgcB83/5PSRoYF77xonID2Ca5CbN0+Ci/wN/jzVdVJtN37OKoHdvzbw9/dGlZR+AJqeX2lVqMoX5ti2+Lihp7IT2mrshWSQYMlxtMsdlgJwUw3TYa2Z5hgOXmOMhrydEUe+kPJpwFqZpH9IRPO1w3WMNoIMg5TDl8eKuY7bb+Q4oddjruXAQBIFgUBtIGtmpSobM3zehZwcGiMAmItzE0ktkLxmoRe88j1s00XVGRAlDTfABMhKKFjp3YogKGMdvJIngBc0rJYJdAByvFEnVCjQNEiHJ004IzUly9Qpf1jIiyox0jYgOGfKzRWokIr4CD9kGmNwWAODH8YKmeOqh09aDG64IrKVmwEBePwdJFKZu6lDDYznimRrvisIAh47KZ576NJfjShNx+sJb5U6/c+afhu7Dpaiv6wtiYwneKlmj3Xy7NZw53jMvbLknF60nMnlrt0oTdAH7/AQcgoUbc0/LnATq9pfGhUWfHdUjtKtLRKWw3EdNgH1UGKD0gzeTo69OD9/9 f15qht3n J0NiQjEtPJhIigJczGhSiEZUBmI/yNjtCKfRvuzrOOweAqxx1equfQSMg25YVyho7A2Al/K3uhy8DlPpNnEJQTau4+iyx3TODp3OQoiBuVnIwmNs0jk7OHynzNEWrrq+2ZZ8H7c+PjyTohv/uemy3A8MaQDVGH0cz7nBgnNOud/sEUbuXi+d7LqUOmxCwl7WYueAeH5ENTv1EBvC02ZixcqhCJzO3BOh8F6PntyE2u60vOaTyTs9WfoYIE4nkxl5VPTV9GqcX8AbeO1JPJEoKnOYxJoHMZDMKfSJ0dva0pftIfy75Wik0gUbR2xh1EkcouNWnaiB3H+xWWTYaRu69w9w76zz/uABbFmOJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 11/11/24 10:25 AM, Matthew Wilcox wrote: > On Sun, Nov 10, 2024 at 08:27:52AM -0700, Jens Axboe wrote: >> 5 years ago I posted patches adding support for RWF_UNCACHED, as a way >> to do buffered IO that isn't page cache persistent. The approach back >> then was to have private pages for IO, and then get rid of them once IO >> was done. But that then runs into all the issues that O_DIRECT has, in >> terms of synchronizing with the page cache. > > Today's a holiday, and I suspect you're going to do a v3 before I have > a chance to do a proper review of this version of the series. Probably, since I've done some fixes since v2 :-). So you can wait for v3, I'll post it later today anyway. > I think "uncached" isn't quite the right word. Perhaps 'RWF_STREAMING' > so that userspace is indicating that this is a streaming I/O and the > kernel gets to choose what to do with that information. Yeah not sure, it's the one I used back in the day, and I still haven't found a more descriptive word for it. That doesn't mean one doesn't exist, certainly taking suggestions. I don't think STREAMING is the right one however, you could most certainly be doing random uncached IO. > Also, do we want to fail I/Os to filesystems which don't support > it? I suppose really sophisticated userspace might fall back to > madvise(DONTNEED), but isn't most userspace going to just clear the flag > and retry the I/O? Also something that's a bit undecided, you can make arguments for both ways. For just ignoring the flag if not support, the argument would be that the application just wants to do IO, uncached if available. For the other argument, maybe you have an application that wants to fallback to O_DIRECT if uncached isn't available. That application certainly wants to know if it works or not. Which is why I defaulted to return -EOPNOTSUPP if it's not available. An applicaton may probe this upfront if it so desires, and just not set the flag for IO. That'd keep it out of the hot path. Seems to me that returning whether it's supported or not is the path of least surprises for applications, which is why I went that way. > Um. Now I've looked, we also have posix_fadvise(POSIX_FADV_NOREUSE), > which is currently a noop. But would we be better off honouring > POSIX_FADV_NOREUSE than introducing RWF_UNCACHED? I'll think about this > some more while I'm offline. That would certainly work too, for synchronous IO. But per-file hints are a bad idea for async IO, for obvious reasons. We really want per-IO hints for that, we have a long history of messing that up. That doesn't mean that FMODE_NOREUSE couldn't just set RWF_UNCACHED, if it's set. That'd be trivial. Then the next question is if setting POSIX_FADV_NOREUSE should fail of file->f_op->fop_flags & FOP_UNCACHED isn't true. Probably not, since it'd potentially break applications. So probably best to just set f_iocb_flags IFF FOP_UNCACHED is true for that file. And the bigger question is why on earth do we have this thing in the kernel that doesn't do anything... But yeah, now we could make it do something. -- Jens Axboe