From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from userp1050.oracle.com ([156.151.31.82]:45412 "EHLO userp1050.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756586AbcHXWps (ORCPT ); Wed, 24 Aug 2016 18:45:48 -0400 Received: from userp1040.oracle.com (userp1040.oracle.com [156.151.31.81]) by userp1050.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u7OMj5k1004208 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 24 Aug 2016 22:45:05 GMT Date: Wed, 24 Aug 2016 15:42:58 -0700 From: "Darrick J. Wong" To: Omar Sandoval Cc: Chris Murphy , Jeff Mahoney , Tomokhov Alexander , Btrfs BTRFS , Dave Chinner , Christoph Hellwig Subject: Re: Will Btrfs have an official command to "uncow" existing files? Message-ID: <20160824224258.GB20705@birch.djwong.org> References: <126611471805976@web2j.yandex.ru> <57BB5D1B.8040305@suse.com> <20160822230613.GA8256@birch.djwong.org> <20160824183427.GA12630@vader> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii In-Reply-To: <20160824183427.GA12630@vader> Sender: linux-btrfs-owner@vger.kernel.org List-ID: On Wed, Aug 24, 2016 at 11:34:27AM -0700, Omar Sandoval wrote: > On Mon, Aug 22, 2016 at 08:43:18PM -0600, Chris Murphy wrote: > > On Mon, Aug 22, 2016 at 5:06 PM, Darrick J. Wong > > wrote: > > > [add Dave and Christoph to cc] > > > > > > On Mon, Aug 22, 2016 at 04:14:19PM -0400, Jeff Mahoney wrote: > > >> On 8/21/16 2:59 PM, Tomokhov Alexander wrote: > > >> > Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py > > >> > > > >> > But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done. > > >> > > > >> > Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented? > > >> > > >> XFS is looking to do this via fallocate using a flag that all file > > >> systems can choose to honor. Once that lands, it would make sense for > > >> btrfs to use it as well. The idea is that when you pass the flag in, we > > >> examine the range and CoW anything that has a refcount != 1. > > > > > > There /was/ a flag to do that -- FALLOC_FL_UNSHARE_RANGE. However, > > > Christoph and Dave felt[1] that the fallocate call didn't need to have > > > an explicit 'unshare' mode because unsharing shared blocks is > > > necessary to guarantee that a subsequent write will not ENOSPC. I > > > felt that was sufficient justification to withdraw the unshare mode > > > flag. If you fallocate the entire length of a shared file on XFS, it > > > will turn off CoW for that file until you reflink/dedupe it again. > > > > > > At the time I wondered whether or not the btrfs developers (the list > > > was cc'd) would pipe up in support of the unshare flag, but nobody > > > did. Consequently it remains nonexistent. Christoph commented a few > > > months ago about unsharing fallocate over NFS atop XFS blocking for a > > > long time, though nobody asked for 'unshare' to be reinstated as a > > > separate fallocate mode, much less a 'don't unshare' flag for regular > > > fallocate mode. > > > > > > (FWIW I'm ok with not having to fight for more VFS changes. :)) > > > > > >> That code hasn't landed yet though. The last time I saw it posted was > > >> June. I don't speak with knowledge of the integration plan, but it > > >> might just be queued up for the next merge window now that the reverse > > >> mapping patches have landed in 4.8. > > > > > > I am going to try to land XFS reflink in 4.9; I hope to have an eighth > > > patchset out for review at the end of the week. > > > > > > So... if the btrfs folks really want an unshare flag I can trivially > > > re-add it to the VFS headers and re-enable it in the XFS > > > implementation but y'all better speak up now and hammer out an > > > acceptable definition. I don't think XFS needs a new flag. > > > > Use case wise I can't think of why I'd want to do unshare. There is a > > use case for wanting to set nocow after the fact. I have no idea what > > complexity is added on the Btrfs side for either operation, it seems > > like at the least to set it, data csum needs a way to be ignored or > > removed; and conversely to unset nocow it's a question whether that > > means the file should have csum's computed, strictly speaking I guess > > you could have cow without datacsum. > > One use case is for swapfile support on Btrfs -- I implemented it with > the requirement that the file was nocow with no shared extents. I think > there was some discussion about having the swapon operation do that > unshare, but I decided against that [1]. (I should take a look at > reviving that patch series.) > > Darrick, what's XFS doing for reflink + swap files? > > 1: http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg785536.html We neither allow swapon() for file with shared extents, nor reflinking/deduping files currently being used as swap. --D > > -- > Omar