From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 304FEC433E2 for ; Wed, 2 Sep 2020 05:19:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 13D862078B for ; Wed, 2 Sep 2020 05:19:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726212AbgIBFTi (ORCPT ); Wed, 2 Sep 2020 01:19:38 -0400 Received: from mail105.syd.optusnet.com.au ([211.29.132.249]:52343 "EHLO mail105.syd.optusnet.com.au" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726177AbgIBFTh (ORCPT ); Wed, 2 Sep 2020 01:19:37 -0400 Received: from dread.disaster.area (pa49-195-191-192.pa.nsw.optusnet.com.au [49.195.191.192]) by mail105.syd.optusnet.com.au (Postfix) with ESMTPS id 328F23A705E; Wed, 2 Sep 2020 15:19:31 +1000 (AEST) Received: from dave by dread.disaster.area with local (Exim 4.92.3) (envelope-from ) id 1kDLBC-00087G-Uv; Wed, 02 Sep 2020 15:19:30 +1000 Date: Wed, 2 Sep 2020 15:19:30 +1000 From: Dave Chinner To: "Theodore Y. Ts'o" Cc: Matthew Wilcox , Miklos Szeredi , Al Viro , "Dr. David Alan Gilbert" , Greg Kurz , linux-fsdevel@vger.kernel.org, Stefan Hajnoczi , Miklos Szeredi , Vivek Goyal , Giuseppe Scrivano , Daniel J Walsh , Chirantan Ekbote Subject: Re: xattr names for unprivileged stacking? Message-ID: <20200902051930.GJ12096@dread.disaster.area> References: <20200830191016.GZ14765@casper.infradead.org> <20200831113705.GA14765@casper.infradead.org> <20200831132339.GD14765@casper.infradead.org> <20200831142532.GC4267@mit.edu> <20200901033405.GF12096@dread.disaster.area> <20200901145205.GA558530@mit.edu> <20200901151453.GC558530@mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200901151453.GC558530@mit.edu> X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.3 cv=KcmsTjQD c=1 sm=1 tr=0 cx=a_idp_d a=vvDRHhr1aDYKXl+H6jx2TA==:117 a=vvDRHhr1aDYKXl+H6jx2TA==:17 a=kj9zAlcOel0A:10 a=reM5J-MqmosA:10 a=7-415B0cAAAA:8 a=8nLuUakjdfBH0j0kmMAA:9 a=CjuIK1q_8ugA:10 a=biEYGPWJfzWAr4FL6Ov7:22 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org On Tue, Sep 01, 2020 at 11:14:53AM -0400, Theodore Y. Ts'o wrote: > On Tue, Sep 01, 2020 at 10:52:05AM -0400, Theodore Y. Ts'o wrote: > > On Tue, Sep 01, 2020 at 01:34:05PM +1000, Dave Chinner wrote: > > > > > > But, unlike your implication that this is -really complex and hard > > > to do-, it's actually relatively trivial to do with the XFS > > > implementation I mentioned as each ADS stream is a fully fledged > > > inode that can point to shared data extents. If you can do data > > > manipulation on a regular inode, you'll be able to do it on an ADS, > > > and that includes copying ADS streams via reflink. > > > > Is the reflink system call on a file with ADS's atomic, or not? We can implement as wholly atomic if we want to, yes. > > What > > if there are a million files is ADS hierarchy which is 100 > > subdirectories deep in some places, comprising several TB's worth of > > data? Is that all going to fit in a single XFS transaction? No, but we solved that "unbound transaction size" problem years ago for reflink and reverse mapping updates. We have constructs like intents and rolling atomic transactions to allow largely unbound modifications to run without being limited by the maximum size of a single transaction or even the size of the journal... > > What if > > you crash in the middle of it? Intents track modification progress and allow recovery to restart from exactly where the journal says the operation had got to.... > > Is a partially reflinked copy of an > > ADS file OK? Yes. > > Or a reflinked ADS file missing some portion of the > > alternate data streams? I don't know what this is refering to. > Oh, and if the answer is that the ADS inodes should be reflinked > individually in userspace, wonderful! You can do that too, if you want - FI_CLONE/clone_file_range work on open file descriptors, not file names, so if you can represent an ADS as a file descriptor, you can clone it, dedupe it, etc, just like you can with an other user data file. > An ADS inode could then just be > a directory, like it was in the NeXT operating system, and copying an > ADS file could *also* be done in userspace, as a cp -r. :-) And then you lose all the atomicity and recoverability of reflink copies. So, not they are not equivalent, and demonstrate where the limits of representing ADS as "directories in a file" reduce the ability to atomically manipulate ADS as group of objects attached to a file... Cheers, Dave. -- Dave Chinner david@fromorbit.com