From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AD478CA0EE4 for ; Thu, 14 Aug 2025 14:25:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:Cc:To:Subject:Message-ID:Date:From:In-Reply-To:References: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=kknE7rDUjEpODcHDSlY2nJGg21LqzdjsCFp42JS4Ico=; b=CoJUxThemVQ69VFTVFIYMcvvd7 cjDATl7qtjUyaW6QpRdsJbCAA3ZgEpakMBQbcs6mT8oJRIZLakYch4ksSuFjHwUXlhv+oUIw5yoSo xkGkR05JiiszQWwHDWaT9OEPSKPH143DZX4EhhPanEEDf+ad+lrbSInAo4KrT+OX3Ql/kstEcmMIk Xx8Qjtxc9pTKeQh0vmdnL0KIt2ghRciLRsSfXeCMyp5udIWDi1lcarypP1hsm0VjN38IFsfdtaUep CTadmg5D0FhXUihpTSrxCHv3LiY16iFYnY+w4uwbVf53MPIFenDwXUhQLNHr4rzKUOYXsJqUBO1kN tKwB6ZyA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1umYtY-0000000HFZh-0pRM; Thu, 14 Aug 2025 14:25:32 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1umYJP-0000000H6Gu-2qe7; Thu, 14 Aug 2025 13:48:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:Content-Type :Cc:To:Subject:Message-ID:Date:From:In-Reply-To:References:MIME-Version: Sender:Reply-To:Content-ID:Content-Description; bh=kknE7rDUjEpODcHDSlY2nJGg21LqzdjsCFp42JS4Ico=; b=gEqU/uofwjXkP2OrjPahe97mxp YCCAXvL9zCcAyjJ4slKHVrpUeVlLdjDsLkTZ4BRTR80IkhSHHX5vkN7Hwd7ZSIecnAwUXwtkeuzqt B8SoMjLomtwsLgG3CtJdSp770WoKX0dMRKjiiq1z5fNLoUJDaASCdjwxsx9/MEF57pFuUiiEIQMrG dOKq2gjxKSFKAXCN4hNZ2QD8mCpZ52zK1wHQsblLdgz6XeARCjoY8RxdvaRcaApr+YQIyv7MaUCmz luGu5+pYmFGwTRE3nqMnGg7yrP5OSMZNlkzwBRDCB/M6xXiv9U4s6D7uHGHsonOb/T8ieI8UgAARq 7q252cyw==; Received: from mail-ed1-x52c.google.com ([2a00:1450:4864:20::52c]) by desiato.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1umYJL-0000000GPzc-3pfR; Thu, 14 Aug 2025 13:48:10 +0000 Received: by mail-ed1-x52c.google.com with SMTP id 4fb4d7f45d1cf-6188b6f501cso1197034a12.2; Thu, 14 Aug 2025 06:48:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1755179285; x=1755784085; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=kknE7rDUjEpODcHDSlY2nJGg21LqzdjsCFp42JS4Ico=; b=dYSHMjrhNmBDxdIxJSzdNOYIGv0RdoKbkOxf/LBHFhRi+XlYyZllUeYOP0oqMJb1ti oFDlFi2QjFpTIYyO50YJgP0SRLTVULzaC06BSL3/gTavws8ZsSrIbrrx1yD0/vkTfry7 TPkOUsR2I6IRbh0s9+iU8s/N3VjtIcLgcVdkGeTBT6rBXq/oQ4zzM+rEj4N+eCWmKIkS dPsDWPzDZi9KwV2q89IHFcGwliHrR24lF0xQx6GUDLfhOoOnJi9lcJyHZpiMj3n9Vsgo lBd+o2anfOKzK5fPTX2FJh5VEc6WlPxSG/WBI4ThXtbXDfyPxwhiyqZP/QLlhQCTXaFb oP2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755179285; x=1755784085; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kknE7rDUjEpODcHDSlY2nJGg21LqzdjsCFp42JS4Ico=; b=IiYkJV3zNK6VhXukDtYhQbbfPBd57AX+oLTNpKKT7CniXGi5rOxUti5NpXLIwo6M9u nUbEv53TjB9abSiKmSax10i71+xT1rI5TvDTDhrc1hMoKO/WRl2lB3O0QRLI18+RTmkE 9s9bQJpx5pgh613ibVyCe4IqVXtfftAGzbqYDg+ygdoaT30vYXSlgP9un9K2nXQElks/ KjA4L8qSAoYb8GemTTNl7MJ0RshZATsNjP743YXfwCKlXd5ysJfNb0ZvT/MMVaRGcf0m C627uWCqQr4B+G1E8xrtHK0r/+ftuDSdpseKG8PSQ08nLNCTj8XJDPo738lmh/rld2/F VuIQ== X-Forwarded-Encrypted: i=1; AJvYcCV1DOZjIgxT209iYl5ua0ZKWu7WLmTpyt2j0yspdgPYN8aGGW57t6MKtkL4v263mViJio+Q2eKOgTM=@lists.infradead.org, AJvYcCX4jc9mSBRA2PrmwWcReayn7z0SRbu+Y9jgMOvZzlzvjXnS6NfDX/XM9Hx5AQ4UeFaF5D7ZWemUjN8=@lists.infradead.org X-Gm-Message-State: AOJu0YxeuTAUsdbhwS0NY/S+a8fdrbFnfKtIlZy8RyuVsBHCNhhomsow +gDGuX6mO9v8gio+/SyQAN1Eh30ZCKBFqh7CAAgzi8jemiVOsrMnVOUxf8kPlolv4dvJ7ehk1ns QC1wMlSdt70z7+T3TFETS10JNE6L/D+Y= X-Gm-Gg: ASbGncs+BGrv6HrApf0pbnYZAnWoHweaLUWJb+95XE6llZIoPs3alZRJA4+OXAsnnWs 0qDpqTw1phGEYSG/Q8A23Y0k3MjJVkEPA3Z5WaXmd9NSmva1W4/oN0wvajx47qm2ODENloCfN1k ccGMQc4hy8QdTLLl3ZqyVqyMJU6E+zbxREN2LmbMfPBa0B3pYo/Amygbb6X/1bDHplwpwbP6JyX kDLTo0= X-Google-Smtp-Source: AGHT+IEyNlWqeAhk/LpScYt9w4g0zV4KI7o0RDFgyEsYgfPOIWzZIEz6LWnNRfl0s84eTTaO2QcDUYXHT6IzkeoX6zs= X-Received: by 2002:a05:6402:348e:b0:617:b2ab:fba2 with SMTP id 4fb4d7f45d1cf-6188c1f81c9mr2881340a12.34.1755179284840; Thu, 14 Aug 2025 06:48:04 -0700 (PDT) MIME-Version: 1.0 References: <20250813065333.GG222315@ZenIV> <175513726277.2234665.5395852687971371437@noble.neil.brown.name> In-Reply-To: <175513726277.2234665.5395852687971371437@noble.neil.brown.name> From: Amir Goldstein Date: Thu, 14 Aug 2025 15:47:53 +0200 X-Gm-Features: Ac12FXwtQAorrE4W3bWM28WxLsn_dOcgt6iMm2QuEDqDEmfUynv58IGzMVEJihs Message-ID: Subject: Re: [PATCH 11/11] VFS: introduce d_alloc_noblock() and d_alloc_locked() To: NeilBrown Cc: Al Viro , Christian Brauner , Jan Kara , David Howells , Marc Dionne , Xiubo Li , Ilya Dryomov , Tyler Hicks , Miklos Szeredi , Richard Weinberger , Anton Ivanov , Johannes Berg , Trond Myklebust , Anna Schumaker , Chuck Lever , Jeff Layton , Steve French , Namjae Jeon , Carlos Maiolino , linux-fsdevel@vger.kernel.org, linux-afs@lists.infradead.org, netfs@lists.linux.dev, ceph-devel@vger.kernel.org, ecryptfs@vger.kernel.org, linux-um@lists.infradead.org, linux-nfs@vger.kernel.org, linux-unionfs@vger.kernel.org, linux-cifs@vger.kernel.org, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250814_144808_040802_C0491268 X-CRM114-Status: GOOD ( 36.32 ) X-BeenThere: linux-um@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-um" Errors-To: linux-um-bounces+linux-um=archiver.kernel.org@lists.infradead.org On Thu, Aug 14, 2025 at 4:08=E2=80=AFAM NeilBrown wrote: > > On Wed, 13 Aug 2025, Al Viro wrote: > > On Tue, Aug 12, 2025 at 12:25:14PM +1000, NeilBrown wrote: > > > Several filesystems use the results of readdir to prime the dcache. > > > These filesystems use d_alloc_parallel() which can block if there is = a > > > concurrent lookup. Blocking in that case is pointless as the lookup > > > will add info to the dcache and there is no value in the readdir wait= ing > > > to see if it should add the info too. > > > > > > Also these calls to d_alloc_parallel() are made while the parent > > > directory is locked. A proposed change to locking will lock the pare= nt > > > later, after d_alloc_parallel(). This means it won't be safe to wait= in > > > d_alloc_parallel() while holding the directory lock. > > > > > > So this patch introduces d_alloc_noblock() which doesn't block > > > but instead returns ERR_PTR(-EWOULDBLOCK). Filesystems that prime th= e > > > dcache now use that and ignore -EWOULDBLOCK errors as harmless. > > > > > > A few filesystems need more than -EWOULDBLOCK - they need to be able = to > > > create the missing dentry within the readdir. procfs is a good examp= le > > > as the inode number is not known until the lookup completes, so readd= ir > > > must perform a full lookup. > > > > > > For these filesystems d_alloc_locked() is provided. It will return a > > > dentry which is already d_in_lookup() but will also lock it against > > > concurrent lookup. The filesystem's ->lookup function must co-operat= e > > > by calling lock_lookup() before proceeding with the lookup. This way= we > > > can ensure exclusion between a lookup performed in ->iterate_shared a= nd > > > a lookup performed in ->lookup. Currently this exclusion is provided= by > > > waiting in d_wait_lookup(). The proposed changed to dir locking will > > > mean that calling d_wait_lookup() (in readdir) while already holding > > > i_rwsem could deadlock. > > > > The last one is playing fast and loose with one assertion that is used > > in quite a few places in correctness proofs - that the only thing other > > threads do to in-lookup dentries is waiting on them (and that - only > > in d_wait_lookup()). I can't tell whether it will be a problem without > > seeing what you do in the users of that thing, but that creates an > > unpleasant areas to watch out for in the future ;-/ > > Yeah, it's not my favourite part of the series. > > > > > Which filesystems are those, aside of procfs? > > > > afs in afs_lookup_atsys(). While looking up a name that ends "@sys" it > need to look up the prefix with various alternate suffixes appended. > So this isn't readdir related, but is a lookup-within-a-lookup. > > The use of d_add_ci() in xfs is the same basic pattern. > > overlayfs does something in ovl_lookup_real_one() that I don't > understand yet but it seems to need a lookup while the directory is > locked. We decoded a connected real directory path (from file handle) and we are trying to lookup in overlay a directory that is referencing the underlying real dir that we decoded. This is the context. Not sure what problem exactly this code gives you. > > ovl_cache_update is in the ovl iterate_shared code (which in fact holds > an exclusive lock). I think this is the same pattern as procfs in that > an inode number needs to be allocated at lookup time, but there might be > more too it. > It's kind of a hack I guess. ovl has those rules (see xino) to compose a consistent inode number from real inode number and layer number. lookup of children during readdir composes the child stack to realize the consistent xino. We could do this internally in ovl by doing lookups on the real layers and composing the xino, but calling lookup on ovl during readdir was so much easier :/ Thanks, Amir.