From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dmitry Monakhov Subject: RE: [PATCH 3/3] ext4: Add support IOC_MOV_DATA ioctl Date: Fri, 18 Jul 2014 14:38:12 +0400 Message-ID: <87zjg7ar57.fsf@openvz.org> References: <004001cf9aa4$2670e280$7352a780$@samsung.com> <87tx6ktiay.fsf@openvz.org> <002901cfa265$636d2e50$2a478af0$@samsung.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Cc: 'Theodore Ts'o' , 'Brian Foster' , linux-kernel@vger.kernel.org, xfs@oss.sgi.com, 'Christoph Hellwig' , 'Ashish Sangwan' , linux-fsdevel@vger.kernel.org, =?utf-8?B?J0x1a8OhxaE=?= Czerner' , 'linux-ext4' To: Namjae Jeon Return-path: In-Reply-To: <002901cfa265$636d2e50$2a478af0$@samsung.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: xfs-bounces@oss.sgi.com Sender: xfs-bounces@oss.sgi.com List-Id: linux-ext4.vger.kernel.org T24gRnJpLCAxOCBKdWwgMjAxNCAxNzo1MDo1NSArMDkwMCwgTmFtamFlIEplb24gPG5hbWphZS5q ZW9uQHNhbXN1bmcuY29tPiB3cm90ZToKPiA+IE9uIFR1ZSwgOCBKdWwgMjAxNCAxNjowMjoyOCAr MDIwMCAoQ0VTVCksIEx1a8OhxaEgQ3plcm5lciA8bGN6ZXJuZXJAcmVkaGF0LmNvbT4gd3JvdGU6 Cj4gPiBOb24tdGV4dCBwYXJ0OiBNVUxUSVBBUlQvTUlYRUQKPiA+ID4gT24gVHVlLCA4IEp1bCAy MDE0LCBOYW1qYWUgSmVvbiB3cm90ZToKPiA+ID4KPiA+ID4gPiBEYXRlOiBUdWUsIDA4IEp1bCAy MDE0IDIxOjAwOjAyICswOTAwCj4gPiA+ID4gRnJvbTogTmFtamFlIEplb24gPG5hbWphZS5qZW9u QHNhbXN1bmcuY29tPgo+ID4gPiA+IFRvOiBEYXZlIENoaW5uZXIgPGRhdmlkQGZyb21vcmJpdC5j b20+LCBUaGVvZG9yZSBUcydvIDx0eXRzb0BtaXQuZWR1Pgo+ID4gPiA+IENjOiBsaW51eC1leHQ0 IDxsaW51eC1leHQ0QHZnZXIua2VybmVsLm9yZz4sIGxpbnV4LWZzZGV2ZWxAdmdlci5rZXJuZWwu b3JnLAo+ID4gPiA+ICAgICBsaW51eC1rZXJuZWxAdmdlci5rZXJuZWwub3JnLCBMdWvDocWhIEN6 ZXJuZXIgPGxjemVybmVyQHJlZGhhdC5jb20+LAo+ID4gPiA+ICAgICBCcmlhbiBGb3N0ZXIgPGJm b3N0ZXJAcmVkaGF0LmNvbT4sIENocmlzdG9waCBIZWxsd2lnIDxoY2hAaW5mcmFkZWFkLm9yZz4s Cj4gPiA+ID4gICAgIEFzaGlzaCBTYW5nd2FuIDxhLnNhbmd3YW5Ac2Ftc3VuZy5jb20+LCB4ZnNA b3NzLnNnaS5jb20KPiA+ID4gPiBTdWJqZWN0OiBbUEFUQ0ggMy8zXSBleHQ0OiBBZGQgc3VwcG9y dCBJT0NfTU9WX0RBVEEgaW9jdGwKPiA+ID4gPgo+ID4gPiA+IFRoaXMgcGF0Y2ggaW1wbGVtZW50 cyBmcyBpb2N0bCdzIElPQ19NT1ZfREFUQSBmb3IgRXh0NC4KPiA+ID4KPiA+ID4gSG1tIGlzbid0 IHRoaXMgYmFzaWNhbGx5IHdoYXQgZXh0NF9tb3ZlX2V4dGVudHMoKSBkb2VzID8gZWcuCj4gPiA+ IEVYVDRfSU9DX01PVkVfRVhUID8KPiA+ID4KPiA+ID4gSSBndWVzcyB0aGF0IHRoZSBpbnRlbnRp b24gaGVyZSBpcyB0byBkbyB0aGUgbW92ZSwgd2l0aG91dCBhY3R1YWxseQo+ID4gPiBtb3Zpbmcg dGhlIGRhdGEgcmlnaHQgPyBCdXQgbmV2ZXJ0aGVsZXNzIG1heWJlIHNvbWUgY29kZSBjYW4gYmUK PiA+ID4gc2hhcmVkIHdpdGggZXh0NF9tb3ZlX2V4dGVudHMoKSA/Cj4gPiBJdCBkZWZpbml0ZWx5 IGNhbiBiZSBzaGFyZWQsIGJlY2F1c2UgaXQgaGFzIHNwZWNpZmljIGNhc2UgZm9yIHVud3JpdHRl bgo+ID4gZGF0YSBzZWUgbW92ZV9leHRlbnRfcGVyX3BhZ2UoKS4KPiBJZiBJIHVuZGVyc3RhbmQg Y29ycmVjdGx5LCBtb3ZfZXh0ZW50X3Blcl9wYWdlIGNhbGxzIG1leHRfcmVwbGFjZV9icmFuY2hl cyB0bwo+IF9yZXBsYWNlXyBleHRlbnRzIGZyb20gMSBpbm9kZSB0byBvdGhlciBpbm9kZS4gUGxl YXNlIGNvcnJlY3QgbWUgaWYgSQo+ID4gPiBhbSB3cm9uZy4KWW91IGFyZSByaWdodC4KPiBpb2Nf bW92X2RhdGEgd2lsbCBub3QgcmVwbGFjZSBleHRlbnRzLCBidXQgaXQgd2lsbCBhY3R1YWxseSBf bW92ZV8gZXh0ZW50cyBpbnRvCj4gaG9sZSBmcm9tIGRvbm9yIHRvIHJlY2VpdmVyLCBsZWF2aW5n IGEgaG9sZSBhdCB0aGUgcGxhY2UgZnJvbSB3aGVyZSBleHRlbnRzIGFyZQo+IG1vdmVkLgpTbyB3 ZSBjYW4gcmVmZXIgZXh0NF9tb3ZlX2V4dGVudHMgYXMgU1dBUCwgYW5kIGlvY19tb3ZfZGF0YSBh cyBBU1NJTkcuCkFuZCBib3RoIHRhc2tzIGxvb2tzIHZlcnkgc2ltaWxhciwgZXhjZXB0IHRoZSB3 YXkgaG93IGhvbGVzIGFyZQppbnRlcnByZXRlZC4gSSB0aGluayBpdCBpcyByZWFzb25hYmxlIHRv IGFsbG93IGV4dDRfbW92ZV9leHRlbnRzKCkKdG8gaW50ZXJwcmV0IGhvbGVzIHNpbWlsYXIgdG8g aW9jX21vdl9kYXRhLiAKPiBDb3VsZCB5b3UgZWxhYm9yYXRlIG1vcmUgaG93IGl0IGNhbiBiZSBz aGFyZWQgZm9yIHVud3JpdHRlbiBkYXRhIGNhc2UgPwpJZiB3ZSBmb3VuZCB1bndyaXR0ZW4gZXh0 ZW50IHdlIGRvIG5vdCBoYXZlIHRvIGNvcHkgZGF0YSwganVzdCBzd2FwIHRvCmV4dGVudHMgYmV0 d2VlbiBpbm9kZXMuCj4gCj4gPiBCdXQgSSB0aGluayB3ZSBjYW4gb2JzZXJ2ZSBhbm90aGVyIHdh eSB0byB1bmlmeSB0aGlzIHR3byB0aGluZ3MuCj4gPiBBbiBpZGVhIGluc3BpcmVkIGJ5IHRoZSBm YWN0IHRoYXQgaW9jX21vdmVfZGF0YSB3b3JrcyBvbmx5IGZvcgo+ID4gcmVndWxhciBpbm9kZXMs IHdoZXJlIG9yaWdfb2Zmc2V0ID09IGRvbm9yX29mZnNldC4gCj4gQ291bGQgeW91IGVsYWJvcmF0 ZSB0aGlzIHBvaW50PwpBdCB0aGlzIG1vbWVudCBvZmZzZXQgZm9yIGJvdGggaW5vZGVzIHNob3Vs ZCBiZSBlcXVhbC4gVGhpcyBpcyBwdXJlCmFydGlmaWNpYWwgcmVzdHJpY3Rpb24uIEF0IHRoaXMg bW9tZW50IEknbSB3b3JraW5nIG9uIHBhdGNoIHdoaWNoCnJlbW92ZSB0aGlzIHJlc3RyaWN0aW9u LiAKPiAKPiA+IFRoaXMgaXMgc2hvd3N0b3BwZXIKPiA+IGZvciAgbXkgdXRpbGl0eSBlNGRlZnJh ZzIgKCBuZXcgdmVyc2lvbiBvZiBlNGRlZnJhZyB3aGljaCBpcyBhYmxlIGRlZnJhZ21lbnQKPiA+ IHBhY2sgc21hbGwgZmlsZXMgYXMgZGVzY3JpYmVkIGhlcmUgOgo+ID4gaHR0cDovL2xpc3RzLm9w ZW53YWxsLm5ldC9saW51eC1leHQ0LzIwMTQvMDQvMjgvMykKPiA+IAo+ID4gUHJvcG9zZWQgQVBJ IGlzIHZlcnkgc2ltaWxhciB0byBleHQ0X2V4dF9taWdyYXRlOgo+ID4gQXJnczoKPiA+ICAgb3Jp Z19maWxlOiBpbm9kZSB3aGljaCB3ZSB3YW50IHRvIGRlZnJhZ21lbnQKPiA+ICAgZG9ub3JfZmls ZTogYSBmaWxlIHdoaWNoIHdpbGwgYmUgdXNlZCBhcyBhIGRvbm9yIG9mIGJsb2Nrcwo+ID4gMSkg ZmFsbG9jYXRlIGJpZyBkb25vcl9maWxlCj4gPiAyKSBhKSBDcmVhdGUgdG1wIGlub2RlIHdpY2gg bmxpbmsgPSAwCj4gPiAgICBiKSBtb3ZlIGV4dGVudHMgcmVxdWlyZWQgZXh0ZW50cyBmcm9tICBk b25vciB0byB0bXBfZG9ub3JfaW5vZGUKPiA+ICAgIGMpIHJldHVybiBmaWxlIGRlc2NyaXB0b3Ig KHRtcF9mZCkgdG8gdGhhdCB0bXBfZG9ub3JfaW5vZGUKPiA+IDQpIE1hcmsgb3JpZ19maWxlJ3Mg aW5vZGUgd2l0aCBFWFQ0X1NUQVRFX0VYVF9NSUdSQVRFIHN0YXRlCj4gPiA1KSBDb3B5IGRhdGEg ZnJvbSBvcmlnX2ZpbGUgdG8gdG1wX2ZkCj4gPiA2KSBJT0NfU1dBUF9FWDogYXRvbWljYWxseSBz d2FwICBvcmlnX2ZpbGUtPmlfZGF0YSBhbmQgdG1wX2ZkLT5pX2RhdGEKPiA+ICAgIGlmIEVYVDRf U1RBVEVfRVhUX01JR1JBVEUgd2FzIG5vdCBjbGVhcmVkLgo+ID4gCj4gPiBUaGlzIGFwcHJvYWNo IGNhbiB3b3JrcyBub3Qgb25seSBmb3IgcmVndWxhciBmaWxlIHcvbyBqb3VybmFsaW5nCj4gPiBl bmFibGVkLCBidXQgYWxzbyBmb3Igam91cm5hbGVkIG9uZXMsIGFuZCBkaXJlY3Rvcmllcy4KPiA+ IAo+ID4gCj4gPiAKPiA+IAo+ID4gPgo+ID4gPiAtTHVrYXMKPiA+ID4KPiAKCl9fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCnhmcyBtYWlsaW5nIGxpc3QKeGZz QG9zcy5zZ2kuY29tCmh0dHA6Ly9vc3Muc2dpLmNvbS9tYWlsbWFuL2xpc3RpbmZvL3hmcwo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1761520AbaGRKiV (ORCPT ); Fri, 18 Jul 2014 06:38:21 -0400 Received: from mail-la0-f52.google.com ([209.85.215.52]:46484 "EHLO mail-la0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1761352AbaGRKiR convert rfc822-to-8bit (ORCPT ); Fri, 18 Jul 2014 06:38:17 -0400 From: Dmitry Monakhov To: Namjae Jeon Cc: "'Dave Chinner'" , "'Theodore Ts'o'" , "'linux-ext4'" , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, "'Brian Foster'" , "'Christoph Hellwig'" , "'Ashish Sangwan'" , xfs@oss.sgi.com, "=?utf-8?B?J0x1a8OhxaE=?= Czerner'" Subject: RE: [PATCH 3/3] ext4: Add support IOC_MOV_DATA ioctl In-Reply-To: <002901cfa265$636d2e50$2a478af0$@samsung.com> References: <004001cf9aa4$2670e280$7352a780$@samsung.com> <87tx6ktiay.fsf@openvz.org> <002901cfa265$636d2e50$2a478af0$@samsung.com> User-Agent: Notmuch/0.6.1 (http://notmuchmail.org) Emacs/23.3.1 (x86_64-redhat-linux-gnu) Date: Fri, 18 Jul 2014 14:38:12 +0400 Message-ID: <87zjg7ar57.fsf@openvz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8BIT Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 18 Jul 2014 17:50:55 +0900, Namjae Jeon wrote: > > On Tue, 8 Jul 2014 16:02:28 +0200 (CEST), Lukáš Czerner wrote: > > Non-text part: MULTIPART/MIXED > > > On Tue, 8 Jul 2014, Namjae Jeon wrote: > > > > > > > Date: Tue, 08 Jul 2014 21:00:02 +0900 > > > > From: Namjae Jeon > > > > To: Dave Chinner , Theodore Ts'o > > > > Cc: linux-ext4 , linux-fsdevel@vger.kernel.org, > > > > linux-kernel@vger.kernel.org, Lukáš Czerner , > > > > Brian Foster , Christoph Hellwig , > > > > Ashish Sangwan , xfs@oss.sgi.com > > > > Subject: [PATCH 3/3] ext4: Add support IOC_MOV_DATA ioctl > > > > > > > > This patch implements fs ioctl's IOC_MOV_DATA for Ext4. > > > > > > Hmm isn't this basically what ext4_move_extents() does ? eg. > > > EXT4_IOC_MOVE_EXT ? > > > > > > I guess that the intention here is to do the move, without actually > > > moving the data right ? But nevertheless maybe some code can be > > > shared with ext4_move_extents() ? > > It definitely can be shared, because it has specific case for unwritten > > data see move_extent_per_page(). > If I understand correctly, mov_extent_per_page calls mext_replace_branches to > _replace_ extents from 1 inode to other inode. Please correct me if I > > > am wrong. You are right. > ioc_mov_data will not replace extents, but it will actually _move_ extents into > hole from donor to receiver, leaving a hole at the place from where extents are > moved. So we can refer ext4_move_extents as SWAP, and ioc_mov_data as ASSING. And both tasks looks very similar, except the way how holes are interpreted. I think it is reasonable to allow ext4_move_extents() to interpret holes similar to ioc_mov_data. > Could you elaborate more how it can be shared for unwritten data case ? If we found unwritten extent we do not have to copy data, just swap to extents between inodes. > > > But I think we can observe another way to unify this two things. > > An idea inspired by the fact that ioc_move_data works only for > > regular inodes, where orig_offset == donor_offset. > Could you elaborate this point? At this moment offset for both inodes should be equal. This is pure artificial restriction. At this moment I'm working on patch which remove this restriction. > > > This is showstopper > > for my utility e4defrag2 ( new version of e4defrag which is able defragment > > pack small files as described here : > > http://lists.openwall.net/linux-ext4/2014/04/28/3) > > > > Proposed API is very similar to ext4_ext_migrate: > > Args: > > orig_file: inode which we want to defragment > > donor_file: a file which will be used as a donor of blocks > > 1) fallocate big donor_file > > 2) a) Create tmp inode wich nlink = 0 > > b) move extents required extents from donor to tmp_donor_inode > > c) return file descriptor (tmp_fd) to that tmp_donor_inode > > 4) Mark orig_file's inode with EXT4_STATE_EXT_MIGRATE state > > 5) Copy data from orig_file to tmp_fd > > 6) IOC_SWAP_EX: atomically swap orig_file->i_data and tmp_fd->i_data > > if EXT4_STATE_EXT_MIGRATE was not cleared. > > > > This approach can works not only for regular file w/o journaling > > enabled, but also for journaled ones, and directories. > > > > > > > > > > > > > > -Lukas > > > >