btrfs defrag problem

btrfs defrag problem

[ching]

Hi all,

I try to defrag my btrfs root partition (run by root privilege)

find / -type f -o -type d -print0 | xargs --null --no-run-if-empty btrfs filesystem defragment -t $((32*1024*1024))


1. This kind of error messages is prompted:

    failed to open /bin/bash
    open:: Text file busy
    total 1 failures
    failed to open /lib64/ld-2.15.so
    open:: Text file busy
    total 1 failures
    failed to open /sbin/agetty
    open:: Text file busy
    failed to open /sbin/btrfs
    open:: Text file busy
    failed to open /sbin/dhclient
    open:: Text file busy
    failed to open /sbin/init
    open:: Text file busy
    failed to open /sbin/udevd

    It seems that locked files cannot be defragged, is it expected behaviour?

2. Btrfs Wiki mentions that defrag directory will defrag metadata, is symlink/hardlink considered as metadata?

    P.S. inline data is already disabled by "max_inline=0"

3. Is any possible to online defrag a btrfs partition without hindered by mount point/polyinstantied directories?


Thank you.

ching

Re: btrfs defrag problem

[cwillu]

On Tue, Oct 30, 2012 at 5:47 AM, ching <lsching17@gmail.com> wrote:
> Hi all,
>
> I try to defrag my btrfs root partition (run by root privilege)
>
> find / -type f -o -type d -print0 | xargs --null --no-run-if-empty btrfs filesystem defragment -t $((32*1024*1024))
>
>
> 1. This kind of error messages is prompted:
>
>     failed to open /bin/bash
>     open:: Text file busy
>     total 1 failures
>     failed to open /lib64/ld-2.15.so
>     open:: Text file busy
>     total 1 failures
>     failed to open /sbin/agetty
>     open:: Text file busy
>     failed to open /sbin/btrfs
>     open:: Text file busy
>     failed to open /sbin/dhclient
>     open:: Text file busy
>     failed to open /sbin/init
>     open:: Text file busy
>     failed to open /sbin/udevd
>
>     It seems that locked files cannot be defragged, is it expected behaviour?

I can't reproduce that behaviour here, although maybe you're running
an older kernel with some bug that's since been fixed?

> 2. Btrfs Wiki mentions that defrag directory will defrag metadata, is symlink/hardlink considered as metadata?
>
>     P.S. inline data is already disabled by "max_inline=0"

Well, that's a silly thing to do, causing every small file to take up
a separate 4kb block rather than its size * 2, and requiring extra
seeks to read/write them (i.e., if you have a million 10 byte files,
they'll now take up 4GB instead of 20MB).

> 3. Is any possible to online defrag a btrfs partition without hindered by mount point/polyinstantied directories?

If you're asking if you can defrag an unmounted btrfs, not at this
time.  It's possible in principle, nobody has cared enough to
implement it yet.

Re: btrfs defrag problem

[ching]

On 10/30/2012 08:08 PM, cwillu wrote:
> On Tue, Oct 30, 2012 at 5:47 AM, ching <lsching17@gmail.com> wrote:
>> Hi all,
>>
>> I try to defrag my btrfs root partition (run by root privilege)
>>
>> find / -type f -o -type d -print0 | xargs --null --no-run-if-empty btrfs filesystem defragment -t $((32*1024*1024))
>>
>>
>> 1. This kind of error messages is prompted:
>>
>>     failed to open /bin/bash
>>     open:: Text file busy
>>     total 1 failures
>>     failed to open /lib64/ld-2.15.so
>>     open:: Text file busy
>>     total 1 failures
>>     failed to open /sbin/agetty
>>     open:: Text file busy
>>     failed to open /sbin/btrfs
>>     open:: Text file busy
>>     failed to open /sbin/dhclient
>>     open:: Text file busy
>>     failed to open /sbin/init
>>     open:: Text file busy
>>     failed to open /sbin/udevd
>>
>>     It seems that locked files cannot be defragged, is it expected behaviour?
> I can't reproduce that behaviour here, although maybe you're running
> an older kernel with some bug that's since been fixed?

I am running Gentoo with Kernel 3.6.3 x64, btrfs-progs git version

my root is mounted with noatime,space_cache,inode_cache,metadata_ratio=15,max_inline=0,ssd_spread


>> 2. Btrfs Wiki mentions that defrag directory will defrag metadata, is symlink/hardlink considered as metadata?
>>
>>     P.S. inline data is already disabled by "max_inline=0"
> Well, that's a silly thing to do, causing every small file to take up
> a separate 4kb block rather than its size * 2, and requiring extra
> seeks to read/write them (i.e., if you have a million 10 byte files,
> they'll now take up 4GB instead of 20MB).

this is off-topic to my question.

>> 3. Is any possible to online defrag a btrfs partition without hindered by mount point/polyinstantied directories?
> If you're asking if you can defrag an unmounted btrfs, not at this
> time.  It's possible in principle, nobody has cared enough to
> implement it yet.
>

Re: btrfs defrag problem

[David Sterba]

On Tue, Oct 30, 2012 at 07:47:28PM +0800, ching wrote:
>     failed to open /bin/bash
>     open:: Text file busy

That's not a btrfs problem, you can't directly modify an executable that
is being used.

>     failed to open /lib64/ld-2.15.so
>     failed to open /sbin/agetty
>     failed to open /sbin/btrfs
>     failed to open /sbin/dhclient
>     failed to open /sbin/init
>     failed to open /sbin/udevd

libraries, currently executed progs or daemons

>     It seems that locked files cannot be defragged, is it expected behaviour?

Yes.

> 2. Btrfs Wiki mentions that defrag directory will defrag metadata, is symlink/hardlink considered as metadata?
>     P.S. inline data is already disabled by "max_inline=0"

Both -links are metadata and also stored inside metadata blocks without
any chance to move to data blocks, so max_inline does not affect this.

> 3. Is any possible to online defrag a btrfs partition without hindered by mount point/polyinstantied directories?

Sorry, I do not understand the question.

david

Re: btrfs defrag problem

[ching]

>>     It seems that locked files cannot be defragged, is it expected behaviour?
> Yes.

but i am defragging a file, not modify it.

>> 2. Btrfs Wiki mentions that defrag directory will defrag metadata, is symlink/hardlink considered as metadata?
>>     P.S. inline data is already disabled by "max_inline=0"
> Both -links are metadata and also stored inside metadata blocks without
> any chance to move to data blocks, so max_inline does not affect this.

thank you for your explanation.

>> 3. Is any possible to online defrag a btrfs partition without hindered by mount point/polyinstantied directories?
> Sorry, I do not understand the question.
>

when a device is mounted under a directory, files in the directory is "hidden", and files in the device is "available", right?
when a directory is polyinstantied, files in the original directory is "hidden", and files in the polyinstantied directory is "available",

How to get past them and pass those "hidden" files to defrag command?


ching

Re: btrfs defrag problem

[David Sterba]

On Thu, Nov 01, 2012 at 05:17:04AM +0800, ching wrote:
> >> 3. Is any possible to online defrag a btrfs partition without hindered by mount point/polyinstantied directories?
> > Sorry, I do not understand the question.
> 
> when a device is mounted under a directory, files in the directory is "hidden", and files in the device is "available", right?
> when a directory is polyinstantied, files in the original directory is "hidden", and files in the polyinstantied directory is "available",
> 
> How to get past them and pass those "hidden" files to defrag command?

I hope I get it right, so unless you have a reference to the directory
with hidden files (using your term), there's no way to access them. And
this is a more generic question, not related to btrfs itself. The hidden
files may also belong to a different filesystem.

david

Re: btrfs defrag problem

[ching]

On 11/06/2012 06:57 AM, David Sterba wrote:
> On Thu, Nov 01, 2012 at 05:17:04AM +0800, ching wrote:
>>>> 3. Is any possible to online defrag a btrfs partition without hindered by mount point/polyinstantied directories?
>>> Sorry, I do not understand the question.
>> when a device is mounted under a directory, files in the directory is "hidden", and files in the device is "available", right?
>> when a directory is polyinstantied, files in the original directory is "hidden", and files in the polyinstantied directory is "available",
>>
>> How to get past them and pass those "hidden" files to defrag command?
> I hope I get it right, so unless you have a reference to the directory
> with hidden files (using your term), there's no way to access them. And
> this is a more generic question, not related to btrfs itself. The hidden
> files may also belong to a different filesystem.
>
> david
>

thank for your explanation

ching

Re: btrfs defrag problem

[Sander]

David Sterba wrote (ao):
> On Thu, Nov 01, 2012 at 05:17:04AM +0800, ching wrote:
> > when a device is mounted under a directory, files in the directory
> > is "hidden", and files in the device is "available", right?  when a
> > directory is polyinstantied, files in the original directory is
> > "hidden", and files in the polyinstantied directory is "available",
> > 
> > How to get past them and pass those "hidden" files to defrag
> > command?
> 
> I hope I get it right, so unless you have a reference to the directory
> with hidden files (using your term), there's no way to access them.
> And this is a more generic question, not related to btrfs itself. The
> hidden files may also belong to a different filesystem.

What Ching means (I think), is that if you have directories in /home,
and you mount a device onto /home, you cannot see the original
directories in /home anymore.

You can still access them though, with a 'mount -o bind':

# mount -o bind / /mnt
# ls /mnt/home

	Sander