Re: can't get e4defrag to work

Re: can't get e4defrag to work

[Akira Fujita]

Hi Derkjan,

Thank you for using ext4 onlie defrag.

I tried to use the same kernel and command
(2.6.28 + http://www2.kernel.org/pub/linux/kernel/people/tytso/ext4-patches/LATEST/broken-out.tar.bz2 )
and I got a fine result as follows.

[root@bsd086 broken-out]# ./e4defrag  -v /mnt/mp1/file1
ext4 defragmentation for /mnt/mp1/file1
[1/1]/mnt/mp1/file1:    100%  extents: 3 -> 1    [ OK ]
  Success:            [1/1]

My environment was:
[root@bsd086 broken-out]# uname -a
Linux bsd086 2.6.28 #2 SMP Thu Jan 22 09:55:13 JST 2009 i686 i686 i386 GNU/Linux

[root@bsd086 broken-out]# mount
/dev/sda8 on /mnt/mp1 type ext4 (rw)

[root@bsd086 broken-out]# debugfs /dev/sda8
..
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype n
eeds_recovery extent sparse_super large_file huge_file uninit_bg dir_nlink extra
_isize
Filesystem flags:         signed_directory_hash
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              61824
Block count:              246991
Reserved block count:     12349
Free blocks:              223339
Free inodes:              61810
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      60
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         7728
..


Did you turn off the flex_bg feature in your case?
But the message "Inappropriate ioctl for device" you had is not
related to the flex_bg.
Something related to ioctl might be wrong in your case.

Were there any reject or hunks when you applied patches to kernel?
Did you boot with correct kernel?
I have no idea so can you tell me the detail of your test environment?

By the way, ext4 online defrag in the current ext4 patch queue (2.6.29-rc1)
does not work fine, because defrag does not handle some new ext4 functions correctly.
Therefore Ted commented them out recently.

I'm preparing to release new online defrag patches now.
Probably it will be the next week.
I appreciate it if you try to use new one as well.

Thanks,
Akira Fujita


Derkjan de Haan wrote:
> Dear Sirs,
> 
> I have tried to compile a usable e4defrag but to no avail. It doesn't
> defrag anything, and in verbose mode it reports: "Defrag
> fail:Inappropriate ioctl for device".
> 
> I have downloaded a stock 2.6.28 kernel, and applied all patches in
> http://www2.kernel.org/pub/linux/kernel/people/tytso/ext4-patches/LATEST/broken-out.tar.bz2
> I have uncommented the defrag patches (except -00 (the changelog) and
> -09 (the defrag tool) in the series file, and run "quilt push -a". It
> reports all patches applied successfully. An example e4defrag run
> looks like this:
> 
> [root@toshiba ~]# ./e4defrag -v .
> ext4 defragmentation for directory(.)
> [1/31] "/root"
>         File is not regular file                [ NG ]
> [2/31] "/root/.gnupg"
>         File is not regular file                [ NG ]
> [3/31]/root/.gnupg/gpg.conf:      0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [4/31] "/root/.gnupg/pubring.gpg"
>         File size is 0          [ NG ]
> [5/31]/root/e4defrag:     0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [6/31] "/root/Mail"
>         File is not regular file                [ NG ]
> [7/31]/root/defrag-09-online-defrag-command.c:    0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [8/31] "/root/.gconfd"
>         File is not regular file                [ NG ]
> [9/31]/root/.gconfd/saved_state:          0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [10/31]/root/.bashrc:     0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [11/31] "/root/.kde"
>         File is not regular file                [ NG ]
> [12/31] "/root/.kde/cache-toshiba.badmuts.org"
>         File is not regular file                [ NG ]
> [13/31] "/root/.kde/tmp-toshiba.badmuts.org"
>         File is not regular file                [ NG ]
> [14/31]/root/.lesshst:    0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [15/31] "/root/.ketchup"
>         File is not regular file                [ NG ]
> [16/31]/root/.ketchup/patch-2.6.29-rc2.bz2:       0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [17/31]/root/install.log.syslog:          0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [18/31]/root/.bash_logout:        0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [19/31]/root/.bash_history:       0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [20/31]/root/.bash_profile:       0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [21/31]/root/anaconda-ks.cfg:     0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [22/31] "/root/.lftp"
>         File is not regular file                [ NG ]
> [23/31]/root/.lftp/rl_history:    0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [24/31]/root/.lftp/transfer_log:          0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [25/31]/root/.lftp/cwd_history:   0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [26/31]/root/.cshrc:      0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [27/31]/root/install.log:         0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [28/31]/root/.tcshrc:     0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [29/31] "/root/.ssh"
>         File is not regular file                [ NG ]
> [30/31]/root/.ssh/known_hosts:    0%
>         Defrag fail:Inappropriate ioctl for device      [ NG ]
> [31/31] "/root/.gconf"
>         File is not regular file                [ NG ]
> 
>         Success:                        [ 0/31 ]
>         Failure:                        [ 31/31 ]
>         Total extents:                    36->36
>         Fragmented percentage:           20%->20%
> 
> I must be doing something wrong, but I can't figure out what. Do you
> have any hints?
> 
> Kind regards,
> 
> 
> Derkjan de Haan
> 

Re: can't get e4defrag to work

[Greg Freemyer]

On Fri, Jan 23, 2009 at 2:23 AM, Akira Fujita
<snip>
> I'm preparing to release new online defrag patches now.
> Probably it will be the next week.
> I appreciate it if you try to use new one as well.
>
> Thanks,
> Akira Fujita

Akira,

Will the new defrag patchset be based on the ioctl's that Ted
suggested last month?

Greg
-- 
Greg Freemyer
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com

Re: can't get e4defrag to work

[Akira Fujita]

Hi Greg,

Greg Freemyer wrote:
> Will the new defrag patchset be based on the ioctl's that Ted
> suggested last month?

Yes, but the new defrag that I will release corresponds to (3) of
Ted's suggestion.
(http://marc.info/?l=linux-ext4&m=122880166227883&w=3)

For (1) and (2), probably it will be necessary to change
the allocation method and add new ioctls, so I will address them later.

Have you already worked on these features?

Regards,
Akira Fujita

--Separator@a-fujita@rs.jp.nec.com:
greg.freemyer@gmail.com
haanjdj@gmail.com
sandeepksinha@gmail.com
linux-ext4@vger.kernel.org

Re: can't get e4defrag to work

[Sandeep K Sinha]

On Mon, Jan 26, 2009 at 1:07 PM, Akira Fujita <a-fujita@rs.jp.nec.com> wrote:
> Hi Greg,
>
> Greg Freemyer wrote:
>>
>> Will the new defrag patchset be based on the ioctl's that Ted
>> suggested last month?
>
> Yes, but the new defrag that I will release corresponds to (3) of
> Ted's suggestion.
> (http://marc.info/?l=linux-ext4&m=122880166227883&w=3)
>
> For (1) and (2), probably it will be necessary to change
> the allocation method and add new ioctls, so I will address them later.
>
> Have you already worked on these features?
>

Well,

for option (2), I did try it on ext2/3 and it works fine for me.
Same on ext4 was a bit problematic as a lot of interfaces are changing
to the mballoc from balloc.
And I have read lot of changes pipelined for block allocation in ext4.
So, just waiting to get some information on that so that I can do the
same for ext4 as well.

This is a sample code that I tried and it works fine. It you wish, I
can produce a formal version for the same.
Here are the changes:


#diff a/fs/ext2/ioctl.c b/fs/ext2/ioctl.c

19a20,35
> #define EXT2_BG_ALLOC 7
>
> struct ext2fs_bg_range {
> 	unsigned int bg_start;
> 	unsigned int bg_end;
> };	
>
> struct ext2fs_bg_ioc {
>
> 	struct ext2fs_bg_range bg_range[10];
> 	unsigned long fs_bg_range_count;
> 	unsigned long blk_req_count;
> };
>
> extern ext2_fsblk_t fs_bg_ext2_new_blocks(struct inode *inode, struct ext2fs_bg_range bg[],
> 		    unsigned long fs_bg_range_count, unsigned long *count, int *errp);
25a42
> 	struct ext2fs_bg_ioc bg_ioc;
27,28c44,45
< 	int ret;
>     int ret ,err;
<
> 	case EXT2_BG_ALLOC:
> 		copy_from_user((struct ext2fs_bg_ioc *)&bg_ioc, (const void __user *)arg, sizeof(struct ext2fs_bg_ioc));
> 		ret = fs_bg_ext2_new_blocks(inode, bg_ioc.bg_range, bg_ioc.fs_bg_range_count,
> 				&bg_ioc.blk_req_count, &err);
> 	
> 		break;	
162a198,200
> 	case EXT2_IOC32_BG_ALLOC_BLOCKS:
> 		cmd = EXT2_IOC_BG_ALLOC_BLOCKS;
> 		break;

diff a/fs/ext2/balloc.c b/fs/ext2/balloc.c

16a17
> #include <linux/module.h>
1543a1545,1732
> struct ext2fs_bg_range {
> 	unsigned int bg_start;
> 	unsigned int bg_end;
> };	
>
> ext2_fsblk_t fs_bg_ext2_new_blocks(struct inode *inode, struct ext2fs_bg_range bg_range[],
> 		    unsigned int fs_bg_range_count, unsigned long *count, int *errp)
> {
> 	struct buffer_head *bitmap_bh = NULL;
> 	struct buffer_head *gdp_bh;
> 	int group_no = 0;
> 	int goal_group = 0;
> 	int fs_bg;
> 	int bg_start = 0, bg_end = 0;
> 	ext2_grpblk_t grp_alloc_blk;	/* blockgroup-relative allocated block*/
> 	ext2_fsblk_t ret_block;		/* filesyetem-wide allocated block */
> 	ext2_fsblk_t goal;		/* goal block for allocation */
> 	int bgi;			/* blockgroup iteration index */
> 	int performed_allocation = 0;
> 	ext2_grpblk_t free_blocks;	/* number of free blocks in a group */
> 	struct super_block *sb;
> 	struct ext2_group_desc *gdp;
> 	struct ext2_super_block *es;
> 	struct ext2_sb_info *sbi;
> 	struct ext2_reserve_window_node *my_rsv = NULL;
> 	unsigned short windowsz = 0;
> 	unsigned long ngroups;
> 	unsigned long num = *count;
>
> 	sb = inode->i_sb;
> 	*errp = -ENOSPC;
> 	if (!sb) {
> 		printk("ext2_new_blocks: nonexistent device");
> 		return 0;
> 	}
>
> 	/*
> 	 * Check quota for allocation of this block.
> 	 */
> 	if (DQUOT_ALLOC_BLOCK(inode, num)) {
> 		*errp = -EDQUOT;
> 		return 0;
> 	}
>
> 	sbi = EXT2_SB(sb);
> 	es = EXT2_SB(sb)->s_es;
> 	ext2_debug("goal=%lu.\n", goal);
>
> 	if (!ext2_has_free_blocks(sbi)) {
> 		*errp = -ENOSPC;
> 		goto out;
> 	}
>
> 	goal = ext2_group_first_block_no(sb,EXT2_I(inode)->i_block_group);
> 	
> 	for(fs_bg = 0; fs_bg < fs_bg_range_count; fs_bg++)
>         {
>                 bg_start = bg_range[fs_bg].bg_start;
>                 bg_end = bg_range[fs_bg].bg_end;
>
> 		group_no = bg_start;
> 		goal_group = bg_start;
>
> retry_alloc:
> 	
> 		ngroups = (bg_end - bg_start) + 1;
> 		smp_rmb();
> 	
> 		/*
> 		 * Now search the rest of the groups.  We assume that
> 		 * group_no and gdp correctly point to the last group visited.
> 		 */
> 		
> 		for (bgi = 0; bgi < ngroups; bgi++, group_no++) {
> 			if (group_no > bg_end)  /* ngroups */
> 				group_no = bg_start; /* previous value 0 */
> 			gdp = ext2_get_group_desc(sb, group_no, &gdp_bh);
> 			if (!gdp)
> 				goto io_error;
> 	
> 			free_blocks = le16_to_cpu(gdp->bg_free_blocks_count);
> 			/*
> 			 * skip this group if the number of
> 			 * free blocks is less than half of the reservation
> 			 * window size.
> 			 */
> 			if (free_blocks <= (windowsz/2))
> 				continue;
>
> 			brelse(bitmap_bh);
> 			bitmap_bh = read_block_bitmap(sb, group_no);
> 			if (!bitmap_bh)
> 				goto io_error;
> 			/*
> 			 * try to allocate block(s) from this group, without a goal(-1).
> 			 */
> 			grp_alloc_blk = ext2_try_to_allocate_with_rsv(sb, group_no,
> 						bitmap_bh, -1, my_rsv, &num);
> 			if (grp_alloc_blk >= 0)
> 				goto allocated;
> 		}
> 	}
> 	/*
> 	 * We may end up a bogus ealier ENOSPC error due to
> 	 * filesystem is "full" of reservations, but
> 	 * there maybe indeed free blocks avaliable on disk
> 	 * In this case, we just forget about the reservations
> 	 * just do block allocation as without reservations.
> 	 */
> 	if (my_rsv) {
> 		my_rsv = NULL;
> 		windowsz = 0;
> 		group_no = goal_group;
> 		goto retry_alloc;
> 	}
> 	/* No space left on the device */
> 	*errp = -ENOSPC;
> 	goto out;
> 	
> allocated:
> 	
> 	ext2_debug("using block group %d(%d)\n",
> 				group_no, gdp->bg_free_blocks_count);
>
> 	ret_block = grp_alloc_blk + ext2_group_first_block_no(sb, group_no);
> 	
> 	if (in_range(le32_to_cpu(gdp->bg_block_bitmap), ret_block, num) ||
> 	    in_range(le32_to_cpu(gdp->bg_inode_bitmap), ret_block, num) ||
> 	    in_range(ret_block, le32_to_cpu(gdp->bg_inode_table),
> 		      EXT2_SB(sb)->s_itb_per_group) ||
> 	    in_range(ret_block + num - 1, le32_to_cpu(gdp->bg_inode_table),
> 		      EXT2_SB(sb)->s_itb_per_group)) {
> 		ext2_error(sb, "ext2_new_blocks",
> 			    "Allocating block in system zone - "
> 			    "blocks from "E2FSBLK", length %lu",
> 					    ret_block, num);
> 	/*
> 	 * ext2_try_to_allocate marked the blocks we allocated as in
> 	 * use.  So we may want to selectively mark some of the blocks
> 	 * as free
> 	 */
> 		goto retry_alloc;
> 	}
>
> 	performed_allocation = 1;
> 	
> 	if (ret_block + num - 1 >= le32_to_cpu(es->s_blocks_count)) {
> 		ext2_error(sb, "ext2_new_blocks",
> 		    "block("E2FSBLK") >= blocks count(%d) - "
> 		    "block_group = %d, es == %p ", ret_block,
> 		le32_to_cpu(es->s_blocks_count), group_no, es);
> 		goto out;
> 	}
>
> 	group_adjust_blocks(sb, group_no, gdp, gdp_bh, -num);
> 	percpu_counter_sub(&sbi->s_freeblocks_counter, num);
>
> 	mark_buffer_dirty(bitmap_bh);
> 	if (sb->s_flags & MS_SYNCHRONOUS)
> 		sync_dirty_buffer(bitmap_bh);
>
> 	*errp = 0;
> 	brelse(bitmap_bh);
> 	DQUOT_FREE_BLOCK(inode, *count-num);
> 	*count = num;
> 	return ret_block;
> io_error:
> 	*errp = -EIO;
> out:
> 	/*
> 	 * Undo the block allocation
> 	 */
> 	if (!performed_allocation)
> 		DQUOT_FREE_BLOCK(inode, *count);
> 	brelse(bitmap_bh);
> 	return 0;
> }
> EXPORT_SYMBOL(fs_bg_ext2_new_blocks);


This works fine.

I would surely like to see a similar ABI in the linux kernel soon.

> Regards,
> Akira Fujita
>
> --Separator@a-fujita@rs.jp.nec.com:
> greg.freemyer@gmail.com
> haanjdj@gmail.com
> sandeepksinha@gmail.com
> linux-ext4@vger.kernel.org
>



-- 
Regards,
Sandeep.





 	
"To learn is to change. Education is a process that changes the learner."

Re: can't get e4defrag to work

[Derkjan de Haan]

Akira,

I have downloaded the e4defrag patches of today, and I'm compiling a
kernel as we speak.
However, I can't compile e4defrag itself:

/tmp/ccHODC4D.o: In function `main':
3.c:(.text+0x393b): undefined reference to `powf'
collect2: ld returned 1 exit status

When I grep in /usr/include, powf only shows up in c++ include files.


regards,

Derkjan

Fwd: can't get e4defrag to work

[Derkjan de Haan]

I would like to report the following issues:

- the patch doesn't apply cleanly to 2.6.29-rc3:

Applying patch patches/1
patching file fs/ext4/Makefile
patching file fs/ext4/defrag.c
patching file fs/ext4/ext4.h
Hunk #1 succeeded at 303 (offset 10 lines).
Hunk #2 succeeded at 312 with fuzz 2 (offset -2 lines).
Hunk #3 succeeded at 1033 (offset 9 lines).
patching file fs/ext4/ext4_extents.h
patching file fs/ext4/extents.c
patching file fs/ext4/ioctl.c
Hunk #2 succeeded at 215 (offset -11 lines).

Applying patch patches/2
patching file fs/ext4/defrag.c

Now at patch patches/2

- I just found out e4defrag needs to be compiled with the -lm flag due
to the powf function being used.

- When compiling ext4 as a module, the kernel build fails on this error:

ERROR: "fget_light" [fs/ext4/ext4.ko] undefined!
make[1]: *** [__modpost] Error 1
make: *** [modules] Error 2

After that, I compiled a kernel with ext4 built in, and that one built
OK. I now have a working e4defrag; thanks for your work on this!


regards,

Derkjan

Re: Fwd: can't get e4defrag to work

[Akira Fujita]

Hi Derkjan,

Derkjan de Haan wrote:
> I would like to report the following issues:
> 
> - the patch doesn't apply cleanly to 2.6.29-rc3:
> 
> Applying patch patches/1
> patching file fs/ext4/Makefile
> patching file fs/ext4/defrag.c
> patching file fs/ext4/ext4.h
> Hunk #1 succeeded at 303 (offset 10 lines).
> Hunk #2 succeeded at 312 with fuzz 2 (offset -2 lines).
> Hunk #3 succeeded at 1033 (offset 9 lines).
> patching file fs/ext4/ext4_extents.h
> patching file fs/ext4/extents.c
> patching file fs/ext4/ioctl.c
> Hunk #2 succeeded at 215 (offset -11 lines).
> 
> Applying patch patches/2
> patching file fs/ext4/defrag.c
> 
> Now at patch patches/2

Defrag pathces applied cleanly to the ext4 patch queue
(7554f240866634a3b5394aced79e1768b4b1f3bf) without any hunks.

[root@bsd086 KERN]# patch -d linux-2.6.29-rc3 -p1 < 1.patch
patching file fs/ext4/Makefile
patching file fs/ext4/defrag.c
patching file fs/ext4/ext4.h
patching file fs/ext4/ext4_extents.h
patching file fs/ext4/extents.c
patching file fs/ext4/ioctl.c

Maybe your ext4 patch queue was not the latest version.

> 
> - I just found out e4defrag needs to be compiled with the -lm flag due
> to the powf function being used.

Yes.  I should have mentioned it in advance.

> - When compiling ext4 as a module, the kernel build fails on this error:
> 
> ERROR: "fget_light" [fs/ext4/ext4.ko] undefined!
> make[1]: *** [__modpost] Error 1
> make: *** [modules] Error 2
> 

Thank you for reporting.
I will use fget() and fput() in the next version instead.

Regards,
Akira Fujita

Re: can't get e4defrag to work

[Derkjan de Haan]

Hi Akira,

Just got e4defrag to work - to a certain degree. I'm using Fedora 10,
which has ext4 as a module. This produces the problem described here:

http://lkml.indiana.edu/hypermail/linux/kernel/0901.2/00701.html

(the patch 'ext4: Add a delayed allocation debugging ioctl' doesn't
export inode_lock -> I reverted this patch)

But now I'm bitten by the fact that Fedora 10 creates ext4 filesystems
with the flex_bg feature enabled, which seems to be incompatible with
e4defrag. I also can't seem to clear that flag. tune2fs refuses this,
and if try using debugfs the flag gets cleared, but when running
e2fsck it gets restored again, so I'm basically stuck here.


Regards,

Derkjan


2009/1/23 Akira Fujita <a-fujita@rs.jp.nec.com>:
> Hi Derkjan,
>
> Thank you for using ext4 onlie defrag.
>
> I tried to use the same kernel and command
> (2.6.28 +
> http://www2.kernel.org/pub/linux/kernel/people/tytso/ext4-patches/LATEST/broken-out.tar.bz2
> )
> and I got a fine result as follows.
>
> [root@bsd086 broken-out]# ./e4defrag  -v /mnt/mp1/file1
> ext4 defragmentation for /mnt/mp1/file1
> [1/1]/mnt/mp1/file1:    100%  extents: 3 -> 1    [ OK ]
>  Success:            [1/1]
>
> My environment was:
> [root@bsd086 broken-out]# uname -a
> Linux bsd086 2.6.28 #2 SMP Thu Jan 22 09:55:13 JST 2009 i686 i686 i386
> GNU/Linux
>
> [root@bsd086 broken-out]# mount
> /dev/sda8 on /mnt/mp1 type ext4 (rw)
>
> [root@bsd086 broken-out]# debugfs /dev/sda8
> ..
> Filesystem features:      has_journal ext_attr resize_inode dir_index
> filetype n
> eeds_recovery extent sparse_super large_file huge_file uninit_bg dir_nlink
> extra
> _isize
> Filesystem flags:         signed_directory_hash
> Default mount options:    (none)
> Filesystem state:         clean
> Errors behavior:          Continue
> Filesystem OS type:       Linux
> Inode count:              61824
> Block count:              246991
> Reserved block count:     12349
> Free blocks:              223339
> Free inodes:              61810
> First block:              0
> Block size:               4096
> Fragment size:            4096
> Reserved GDT blocks:      60
> Blocks per group:         32768
> Fragments per group:      32768
> Inodes per group:         7728
> ..
>
>
> Did you turn off the flex_bg feature in your case?
> But the message "Inappropriate ioctl for device" you had is not
> related to the flex_bg.
> Something related to ioctl might be wrong in your case.
>
> Were there any reject or hunks when you applied patches to kernel?
> Did you boot with correct kernel?
> I have no idea so can you tell me the detail of your test environment?
>
> By the way, ext4 online defrag in the current ext4 patch queue (2.6.29-rc1)
> does not work fine, because defrag does not handle some new ext4 functions
> correctly.
> Therefore Ted commented them out recently.
>
> I'm preparing to release new online defrag patches now.
> Probably it will be the next week.
> I appreciate it if you try to use new one as well.
>
> Thanks,
> Akira Fujita
>
>
> Derkjan de Haan wrote:
>>
>> Dear Sirs,
>>
>> I have tried to compile a usable e4defrag but to no avail. It doesn't
>> defrag anything, and in verbose mode it reports: "Defrag
>> fail:Inappropriate ioctl for device".
>>
>> I have downloaded a stock 2.6.28 kernel, and applied all patches in
>>
>> http://www2.kernel.org/pub/linux/kernel/people/tytso/ext4-patches/LATEST/broken-out.tar.bz2
>> I have uncommented the defrag patches (except -00 (the changelog) and
>> -09 (the defrag tool) in the series file, and run "quilt push -a". It
>> reports all patches applied successfully. An example e4defrag run
>> looks like this:
>>
>> [root@toshiba ~]# ./e4defrag -v .
>> ext4 defragmentation for directory(.)
>> [1/31] "/root"
>>        File is not regular file                [ NG ]
>> [2/31] "/root/.gnupg"
>>        File is not regular file                [ NG ]
>> [3/31]/root/.gnupg/gpg.conf:      0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [4/31] "/root/.gnupg/pubring.gpg"
>>        File size is 0          [ NG ]
>> [5/31]/root/e4defrag:     0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [6/31] "/root/Mail"
>>        File is not regular file                [ NG ]
>> [7/31]/root/defrag-09-online-defrag-command.c:    0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [8/31] "/root/.gconfd"
>>        File is not regular file                [ NG ]
>> [9/31]/root/.gconfd/saved_state:          0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [10/31]/root/.bashrc:     0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [11/31] "/root/.kde"
>>        File is not regular file                [ NG ]
>> [12/31] "/root/.kde/cache-toshiba.badmuts.org"
>>        File is not regular file                [ NG ]
>> [13/31] "/root/.kde/tmp-toshiba.badmuts.org"
>>        File is not regular file                [ NG ]
>> [14/31]/root/.lesshst:    0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [15/31] "/root/.ketchup"
>>        File is not regular file                [ NG ]
>> [16/31]/root/.ketchup/patch-2.6.29-rc2.bz2:       0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [17/31]/root/install.log.syslog:          0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [18/31]/root/.bash_logout:        0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [19/31]/root/.bash_history:       0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [20/31]/root/.bash_profile:       0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [21/31]/root/anaconda-ks.cfg:     0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [22/31] "/root/.lftp"
>>        File is not regular file                [ NG ]
>> [23/31]/root/.lftp/rl_history:    0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [24/31]/root/.lftp/transfer_log:          0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [25/31]/root/.lftp/cwd_history:   0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [26/31]/root/.cshrc:      0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [27/31]/root/install.log:         0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [28/31]/root/.tcshrc:     0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [29/31] "/root/.ssh"
>>        File is not regular file                [ NG ]
>> [30/31]/root/.ssh/known_hosts:    0%
>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>> [31/31] "/root/.gconf"
>>        File is not regular file                [ NG ]
>>
>>        Success:                        [ 0/31 ]
>>        Failure:                        [ 31/31 ]
>>        Total extents:                    36->36
>>        Fragmented percentage:           20%->20%
>>
>> I must be doing something wrong, but I can't figure out what. Do you
>> have any hints?
>>
>> Kind regards,
>>
>>
>> Derkjan de Haan
>>
>

Re: can't get e4defrag to work

[Akira Fujita]

Hi Derkjan,

Derkjan de Haan wrote:
> Hi Akira,
>
> Just got e4defrag to work - to a certain degree. I'm using Fedora 10,
> which has ext4 as a module. This produces the problem described here:
>
> http://lkml.indiana.edu/hypermail/linux/kernel/0901.2/00701.html
>
> (the patch 'ext4: Add a delayed allocation debugging ioctl' doesn't
> export inode_lock -> I reverted this patch)

I see.   :-)

> But now I'm bitten by the fact that Fedora 10 creates ext4 filesystems
> with the flex_bg feature enabled, which seems to be incompatible with
> e4defrag. I also can't seem to clear that flag. tune2fs refuses this,
> and if try using debugfs the flag gets cleared, but when running
> e2fsck it gets restored again, so I'm basically stuck here.

The new online defrag handles the flex_bg feature correctly,
so you will be able to run e4defrag on the Fedora 10 soon.

Regards,
Akira Fujita

> 
> 2009/1/23 Akira Fujita <a-fujita@rs.jp.nec.com>:
>> Hi Derkjan,
>>
>> Thank you for using ext4 onlie defrag.
>>
>> I tried to use the same kernel and command
>> (2.6.28 +
>> http://www2.kernel.org/pub/linux/kernel/people/tytso/ext4-patches/LATEST/broken-out.tar.bz2
>> )
>> and I got a fine result as follows.
>>
>> [root@bsd086 broken-out]# ./e4defrag  -v /mnt/mp1/file1
>> ext4 defragmentation for /mnt/mp1/file1
>> [1/1]/mnt/mp1/file1:    100%  extents: 3 -> 1    [ OK ]
>>  Success:            [1/1]
>>
>> My environment was:
>> [root@bsd086 broken-out]# uname -a
>> Linux bsd086 2.6.28 #2 SMP Thu Jan 22 09:55:13 JST 2009 i686 i686 i386
>> GNU/Linux
>>
>> [root@bsd086 broken-out]# mount
>> /dev/sda8 on /mnt/mp1 type ext4 (rw)
>>
>> [root@bsd086 broken-out]# debugfs /dev/sda8
>> ..
>> Filesystem features:      has_journal ext_attr resize_inode dir_index
>> filetype n
>> eeds_recovery extent sparse_super large_file huge_file uninit_bg dir_nlink
>> extra
>> _isize
>> Filesystem flags:         signed_directory_hash
>> Default mount options:    (none)
>> Filesystem state:         clean
>> Errors behavior:          Continue
>> Filesystem OS type:       Linux
>> Inode count:              61824
>> Block count:              246991
>> Reserved block count:     12349
>> Free blocks:              223339
>> Free inodes:              61810
>> First block:              0
>> Block size:               4096
>> Fragment size:            4096
>> Reserved GDT blocks:      60
>> Blocks per group:         32768
>> Fragments per group:      32768
>> Inodes per group:         7728
>> ..
>>
>>
>> Did you turn off the flex_bg feature in your case?
>> But the message "Inappropriate ioctl for device" you had is not
>> related to the flex_bg.
>> Something related to ioctl might be wrong in your case.
>>
>> Were there any reject or hunks when you applied patches to kernel?
>> Did you boot with correct kernel?
>> I have no idea so can you tell me the detail of your test environment?
>>
>> By the way, ext4 online defrag in the current ext4 patch queue (2.6.29-rc1)
>> does not work fine, because defrag does not handle some new ext4 functions
>> correctly.
>> Therefore Ted commented them out recently.
>>
>> I'm preparing to release new online defrag patches now.
>> Probably it will be the next week.
>> I appreciate it if you try to use new one as well.
>>
>> Thanks,
>> Akira Fujita
>>
>>
>> Derkjan de Haan wrote:
>>> Dear Sirs,
>>>
>>> I have tried to compile a usable e4defrag but to no avail. It doesn't
>>> defrag anything, and in verbose mode it reports: "Defrag
>>> fail:Inappropriate ioctl for device".
>>>
>>> I have downloaded a stock 2.6.28 kernel, and applied all patches in
>>>
>>> http://www2.kernel.org/pub/linux/kernel/people/tytso/ext4-patches/LATEST/broken-out.tar.bz2
>>> I have uncommented the defrag patches (except -00 (the changelog) and
>>> -09 (the defrag tool) in the series file, and run "quilt push -a". It
>>> reports all patches applied successfully. An example e4defrag run
>>> looks like this:
>>>
>>> [root@toshiba ~]# ./e4defrag -v .
>>> ext4 defragmentation for directory(.)
>>> [1/31] "/root"
>>>        File is not regular file                [ NG ]
>>> [2/31] "/root/.gnupg"
>>>        File is not regular file                [ NG ]
>>> [3/31]/root/.gnupg/gpg.conf:      0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [4/31] "/root/.gnupg/pubring.gpg"
>>>        File size is 0          [ NG ]
>>> [5/31]/root/e4defrag:     0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [6/31] "/root/Mail"
>>>        File is not regular file                [ NG ]
>>> [7/31]/root/defrag-09-online-defrag-command.c:    0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [8/31] "/root/.gconfd"
>>>        File is not regular file                [ NG ]
>>> [9/31]/root/.gconfd/saved_state:          0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [10/31]/root/.bashrc:     0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [11/31] "/root/.kde"
>>>        File is not regular file                [ NG ]
>>> [12/31] "/root/.kde/cache-toshiba.badmuts.org"
>>>        File is not regular file                [ NG ]
>>> [13/31] "/root/.kde/tmp-toshiba.badmuts.org"
>>>        File is not regular file                [ NG ]
>>> [14/31]/root/.lesshst:    0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [15/31] "/root/.ketchup"
>>>        File is not regular file                [ NG ]
>>> [16/31]/root/.ketchup/patch-2.6.29-rc2.bz2:       0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [17/31]/root/install.log.syslog:          0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [18/31]/root/.bash_logout:        0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [19/31]/root/.bash_history:       0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [20/31]/root/.bash_profile:       0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [21/31]/root/anaconda-ks.cfg:     0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [22/31] "/root/.lftp"
>>>        File is not regular file                [ NG ]
>>> [23/31]/root/.lftp/rl_history:    0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [24/31]/root/.lftp/transfer_log:          0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [25/31]/root/.lftp/cwd_history:   0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [26/31]/root/.cshrc:      0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [27/31]/root/install.log:         0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [28/31]/root/.tcshrc:     0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [29/31] "/root/.ssh"
>>>        File is not regular file                [ NG ]
>>> [30/31]/root/.ssh/known_hosts:    0%
>>>        Defrag fail:Inappropriate ioctl for device      [ NG ]
>>> [31/31] "/root/.gconf"
>>>        File is not regular file                [ NG ]
>>>
>>>        Success:                        [ 0/31 ]
>>>        Failure:                        [ 31/31 ]
>>>        Total extents:                    36->36
>>>        Fragmented percentage:           20%->20%
>>>
>>> I must be doing something wrong, but I can't figure out what. Do you
>>> have any hints?
>>>
>>> Kind regards,
>>>
>>>
>>> Derkjan de Haan
>>>
>