From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F896C0015E for ; Tue, 1 Aug 2023 16:05:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230050AbjHAQF6 (ORCPT ); Tue, 1 Aug 2023 12:05:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44150 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232112AbjHAQF4 (ORCPT ); Tue, 1 Aug 2023 12:05:56 -0400 Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A1D61996 for ; Tue, 1 Aug 2023 09:05:54 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id DB97E5C00D4; Tue, 1 Aug 2023 12:05:53 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Tue, 01 Aug 2023 12:05:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rath.org; h=cc :cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1690905953; x=1690992353; bh=nO 5DGjeIsutrs2uJ626SDXn1fcBjOv8s3a1cSCtAGic=; b=Fjv2EvfkRR7JD/t7YU S+a8JgBnKJ7tnBtAm9Bkm1ZNx8F7We0k+7n8Qxi5oTlVbS6THxePmjfkE8D+69sR E/mYUAZdY82whewnUx7A9QdGNY6CzPo0Qi1XIfsHnhDPYMUDUYCHdJNpeookoB57 Ot7a7s4e3NZhzCFTXAh3RVNuh6d4aj/kYLUR33ggalJvi9YkTLcoTdKWvgBJsDP2 H3pS87w2npB01JI80bU0Gu/vR/pi2vXNYDi+Vl9BEGB+3VRhi9ESQ3YPB2su6bcC KO9mI31ZyaFvU0Rdg5TMIgmVF21yimwBn0poUCICs2dWsnSLxLqmFEjmLVFFdEng uyXA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1690905953; x=1690992353; bh=nO5DGjeIsutrs 2uJ626SDXn1fcBjOv8s3a1cSCtAGic=; b=uvXADTDcS79SrOD/8jhN1gdM5//mj bfruIshaoSDEpEndihfd73NcIigud6+r6ftMLoZ3NOj0CZxbru+JcyIxajja5Ahe eU5tSrz7qagAQR9aYEZugySGoXG5UkNhijnUFLPFEt6gm+RNLe8M1dp3pBawanBY 2o+/9B0M/0T1wCKcEWGmlF9LuIPIozesHpkH3uGLDceWoHVjfsMWRwufxIn97cOD 4pqxqjxTRd9ogz+cmbu9n8K4EI4M7mAUGxktf+AtY48fC42RhcQi1FkpMIrBFTPq GaD+B/dyBo/jl7VfF8yLeWhpDVG3/U5zLMbIn6OBn7fS2Q11oEL63oMww== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrjeeigdelgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvfevufhfffgjkfgfgggtsehttddttddtredtnecuhfhrohhmpefpihhkohhl rghushcutfgrthhhuceopfhikhholhgruhhssehrrghthhdrohhrgheqnecuggftrfgrth htvghrnhepteekleehieffleeuvdelkeekvdettedtteehhfeiheektdeljeelffeuvdej udevnecuffhomhgrihhnpehgihhthhhusgdrtghomhenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepmhgrihhlfhhrohhmpefpihhkohhlrghushesrhgrthhhrdhorhhg X-ME-Proxy: Feedback-ID: i53a843ae:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 1 Aug 2023 12:05:52 -0400 (EDT) Received: from vostro.rath.org (vostro [192.168.12.4]) by ebox.rath.org (Postfix) with ESMTPS id 666E2C0F; Tue, 1 Aug 2023 16:05:51 +0000 (UTC) Received: by vostro.rath.org (Postfix, from userid 1000) id E701687274; Tue, 1 Aug 2023 17:05:50 +0100 (BST) From: Nikolaus Rath To: Miklos Szeredi Cc: Martin Kaspar via fuse-devel , Linux FS Devel , Miklos Szeredi Subject: Re: [fuse-devel] Semantics of fuse_notify_delete() References: <87wmymk0k9.fsf@vostro.rath.org> <87tttpk2kp.fsf@vostro.rath.org> <87r0osjufc.fsf@vostro.rath.org> <87o7jrjant.fsf@vostro.rath.org> <2e44acdd-b113-43c3-80cb-150f09478383@app.fastmail.com> Mail-Copies-To: never Mail-Followup-To: Miklos Szeredi , Martin Kaspar via fuse-devel , Linux FS Devel , Miklos Szeredi Date: Tue, 01 Aug 2023 17:05:50 +0100 In-Reply-To: (Miklos Szeredi's message of "Tue, 1 Aug 2023 16:48:03 +0200") Message-ID: <87mszarbmp.fsf@vostro.rath.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org On Aug 01 2023, Miklos Szeredi wrote: > On Tue, 1 Aug 2023 at 16:40, Nikolaus Rath wrote: >> >> On Tue, 1 Aug 2023, at 13:53, Miklos Szeredi via fuse-devel wrote: >> > Here's one with the virtual env and the correct head: >> > >> > root@kvm:~/s3ql# git log -1 --pretty="%h %s" >> > 3d35f18543d9 Reproducer for notify_delete issue. To confirm: >> > root@kvm:~/s3ql# ~/s3ql-python-env/bin/python bin/s3qlrm mnt/test >> > WARNING: Received unknown command via control inode >> > ERROR: Uncaught top-level exception: >> > Traceback (most recent call last): >> > File "/root/s3ql/bin/s3qlrm", line 21, in >> > s3ql.remove.main(sys.argv[1:]) >> > File "/root/s3ql/src/s3ql/remove.py", line 72, in main >> > pyfuse3.setxattr(ctrlfile, 'rmtree', cmd) >> > File "src/pyfuse3.pyx", line 629, in pyfuse3.setxattr >> > OSError: [Errno 22] Invalid argument: 'mnt/test/.__s3ql__ctrl__' >> >> This is odd. I have never heard of anyone having this problem before and it also works fine in the CI. >> >> I apologize that this is taking so much of your time. >> >> I have changed the code a bit to print out what exactly it is receiving: >> https://github.com/s3ql/s3ql/commit/eb31f7bff4bd985d68fa20c793c2f2edf5db61a5 >> >> Would you mind updating your branch and trying again? (You'll need to fetch and reset, >> since I rebased on top of current master just to be sure). >> >> I can still reproduce this every time (without any other error): >> >> $ mkdir bucket >> $ bin/mkfs.s3ql --plain local://bucket >> Before using S3QL, make sure to read the user's guide, especially >> the 'Important Rules to Avoid Losing Data' section. >> Creating metadata tables... >> Uploading metadata... >> Uploading metadata... >> Uploaded 1 out of ~1 dirty blocks (100%) >> Calculating metadata checksum... >> $ mkdir mnt >> $ bin/mount.s3ql --fg local://bucket mnt & >> Using 10 upload threads. >> Autodetected 1048514 file descriptors available for cache entries >> Using cached metadata. >> Setting cache size to 315297 MB >> Mounting local:///home/nikratio/in-progress/s3ql/bucket/ at /home/nikratio/in-progress/s3ql/mnt... >> >> $ md mnt/test; echo foo > mnt/test/bar >> $ bin/s3qlrm mnt/test >> fuse: writing device: Directory not empty >> ERROR: Failed to submit invalidate_entry request for parent inode 1, name b'test' >> Traceback (most recent call last): >> File "src/internal.pxi", line 125, in pyfuse3._notify_loop >> File "src/pyfuse3.pyx", line 915, in pyfuse3.invalidate_entry >> OSError: [Errno 39] fuse_lowlevel_notify_delete returned: Directory not empty >> >> nikratio@vostro ~/i/s3ql (notify_delete_bug)> > > WARNING: Received unknown command via control inode: b"1, b'test')" > ERROR: Uncaught top-level exception: > Traceback (most recent call last): > File "/root/s3ql/bin/s3qlrm", line 21, in > s3ql.remove.main(sys.argv[1:]) > File "/root/s3ql/src/s3ql/remove.py", line 74, in main > pyfuse3.setxattr(ctrlfile, 'rmtree', cmd) > File "src/pyfuse3.pyx", line 629, in pyfuse3.setxattr > OSError: [Errno 22] Invalid argument: 'mnt/test/.__s3ql__ctrl__' Thanks! It looks like the extended attribute name and value that S3QL receives from libfuse is corrupted. What reaches S3QL as the xattr name is actually the truncated xattr value (leading parenthesis is missing). Is it possible that you are running into a variant of https://github.com/libfuse/libfuse/issues/730? This was fixed in libfuse 3.14.1 and introduced in 3.13.0. Best, -Nikolaus