enquiry about defrag

enquiry about defrag

[ching]

Hi all,

i am new on btrfs, i am testing KVM on btrfs (host: kernel x86-64 3.5.3), the performance is reasonable.

I have two question on defrag, can someone help me?

1. According to btrfs wiki, defragment a COW file will produce two unrelated files.

    Does it apply to the "autodefrag" mount option?

2. Is there any command for the fragmentation status of a file/dir ? e.g. fragment size, number of fragments.

Re: enquiry about defrag

[Jan Steffens]

On Sun, Sep 9, 2012 at 2:03 AM, ching <lsching17@gmail.com> wrote:
> 2. Is there any command for the fragmentation status of a file/dir ? e.g. fragment size, number of fragments.

Use the "filefrag" command, part of e2fsprogs.

Re: enquiry about defrag

[ching]

On 09/09/2012 08:30 AM, Jan Steffens wrote:
> On Sun, Sep 9, 2012 at 2:03 AM, ching <lsching17@gmail.com> wrote:
>> 2. Is there any command for the fragmentation status of a file/dir ? e.g. fragment size, number of fragments.
> Use the "filefrag" command, part of e2fsprogs.
>

my image is a 16G sparse file, after defragment, it still has 101387 extents, is it normal?

Re: enquiry about defrag

[Fajar A. Nugraha]

On Sun, Sep 9, 2012 at 2:49 PM, ching <lsching17@gmail.com> wrote:
> On 09/09/2012 08:30 AM, Jan Steffens wrote:
>> On Sun, Sep 9, 2012 at 2:03 AM, ching <lsching17@gmail.com> wrote:
>>> 2. Is there any command for the fragmentation status of a file/dir ? e.g. fragment size, number of fragments.
>> Use the "filefrag" command, part of e2fsprogs.
>>
>
> my image is a 16G sparse file, after defragment, it still has 101387 extents, is it normal?

Is compression enabled? If so, yes, it's normal.

-- 
Fajar

Re: enquiry about defrag

[ching]

On 09/09/2012 05:11 PM, Fajar A. Nugraha wrote:
> On Sun, Sep 9, 2012 at 2:49 PM, ching <lsching17@gmail.com> wrote:
>> On 09/09/2012 08:30 AM, Jan Steffens wrote:
>>> On Sun, Sep 9, 2012 at 2:03 AM, ching <lsching17@gmail.com> wrote:
>>>> 2. Is there any command for the fragmentation status of a file/dir ? e.g. fragment size, number of fragments.
>>> Use the "filefrag" command, part of e2fsprogs.
>>>
>> my image is a 16G sparse file, after defragment, it still has 101387 extents, is it normal?
> Is compression enabled? If so, yes, it's normal.
>

compression with lzo.

thanks. i will retry with compression disabled when 3.6 is stable.

Re: enquiry about defrag

[ching]

On 09/09/2012 08:05 PM, ching wrote:
> On 09/09/2012 05:11 PM, Fajar A. Nugraha wrote:
>> On Sun, Sep 9, 2012 at 2:49 PM, ching <lsching17@gmail.com> wrote:
>>> On 09/09/2012 08:30 AM, Jan Steffens wrote:
>>>> On Sun, Sep 9, 2012 at 2:03 AM, ching <lsching17@gmail.com> wrote:
>>>>> 2. Is there any command for the fragmentation status of a file/dir ? e.g. fragment size, number of fragments.
>>>> Use the "filefrag" command, part of e2fsprogs.
>>>>
>>> my image is a 16G sparse file, after defragment, it still has 101387 extents, is it normal?
>> Is compression enabled? If so, yes, it's normal.
>>
> compression with lzo.
>
> thanks. i will retry with compression disabled when 3.6 is stable.

i just realize that it is possible to control compression per file by file attribute

http://radudi.com/index.php/2011/08/31/using-btrfs-per-file-and-per-directory-compression/

when i try to follow example above, no file attribute is shown (my mount option is rw,noatime,compress-force=lzo,space_cache,inode_cache)

$lsattr -V Linux.raw_image
lsattr 1.42.5 (29-Jul-2012)
---------------- Linux.raw_image

is there any official resource/web page for control compression per file by file attribute?

Re: enquiry about defrag

[Liu Bo]

On 09/10/2012 08:19 PM, ching wrote:
> On 09/09/2012 08:05 PM, ching wrote:
>> On 09/09/2012 05:11 PM, Fajar A. Nugraha wrote:
>>> On Sun, Sep 9, 2012 at 2:49 PM, ching <lsching17@gmail.com> wrote:
>>>> On 09/09/2012 08:30 AM, Jan Steffens wrote:
>>>>> On Sun, Sep 9, 2012 at 2:03 AM, ching <lsching17@gmail.com> wrote:
>>>>>> 2. Is there any command for the fragmentation status of a file/dir ? e.g. fragment size, number of fragments.
>>>>> Use the "filefrag" command, part of e2fsprogs.
>>>>>
>>>> my image is a 16G sparse file, after defragment, it still has 101387 extents, is it normal?
>>> Is compression enabled? If so, yes, it's normal.
>>>
>> compression with lzo.
>>
>> thanks. i will retry with compression disabled when 3.6 is stable.
> 
> i just realize that it is possible to control compression per file by file attribute
> 
> http://radudi.com/index.php/2011/08/31/using-btrfs-per-file-and-per-directory-compression/
> 
> when i try to follow example above, no file attribute is shown (my mount option is rw,noatime,compress-force=lzo,space_cache,inode_cache)
> 
> $lsattr -V Linux.raw_image
> lsattr 1.42.5 (29-Jul-2012)
> ---------------- Linux.raw_image
> 
> is there any official resource/web page for control compression per file by file attribute?
> 

FYI, git://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git

> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: enquiry about defrag

[ching]

On 09/10/2012 09:10 PM, Liu Bo wrote:
> On 09/10/2012 08:19 PM, ching wrote:
>> On 09/09/2012 08:05 PM, ching wrote:
>>> On 09/09/2012 05:11 PM, Fajar A. Nugraha wrote:
>>>> On Sun, Sep 9, 2012 at 2:49 PM, ching <lsching17@gmail.com> wrote:
>>>>> On 09/09/2012 08:30 AM, Jan Steffens wrote:
>>>>>> On Sun, Sep 9, 2012 at 2:03 AM, ching <lsching17@gmail.com> wrote:
>>>>>>> 2. Is there any command for the fragmentation status of a file/dir ? e.g. fragment size, number of fragments.
>>>>>> Use the "filefrag" command, part of e2fsprogs.
>>>>>>
>>>>> my image is a 16G sparse file, after defragment, it still has 101387 extents, is it normal?
>>>> Is compression enabled? If so, yes, it's normal.
>>>>
>>> compression with lzo.
>>>
>>> thanks. i will retry with compression disabled when 3.6 is stable.
>> i just realize that it is possible to control compression per file by file attribute
>>
>> http://radudi.com/index.php/2011/08/31/using-btrfs-per-file-and-per-directory-compression/
>>
>> when i try to follow example above, no file attribute is shown (my mount option is rw,noatime,compress-force=lzo,space_cache,inode_cache)
>>
>> $lsattr -V Linux.raw_image
>> lsattr 1.42.5 (29-Jul-2012)
>> ---------------- Linux.raw_image
>>
>> is there any official resource/web page for control compression per file by file attribute?
>>
> FYI, git://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git
>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>

may i ask a stupid question, if i remove my "compress-force=lzo" option, will compression disabled for new written data?

Re: enquiry about defrag

[Li Zefan]

> may i ask a stupid question, if i remove my "compress-force=lzo" option, will compression disabled for new written data?

the answer is yes. :)

Re: enquiry about defrag

[ching]

On 09/11/2012 09:28 AM, Li Zefan wrote:
>> may i ask a stupid question, if i remove my "compress-force=lzo" option, will compression disabled for new written data?
> the answer is yes. :)
>
>

The problem seems fixed. The number of extent decrease to 14055 after defrag with compression off.

Re: enquiry about defrag

[Alex Lyakas]

Hi,

You need to compile the latest version of filefrag, it has some fixes
by Liu Bo wrt to that. However, even then, filefrag may not report the
correct the extent count for btrfs. According to filefrag_fiemap(), it
tries to check whether physical extents are consecutive on disk and
then considers them as a single extent. For btrfs this may not always
be true, because logically the extents might not be consecutive, even
though they are consecutive on disk. xfstest 281 fails for me because
of this. (filefrag -v, however, seems to show BTRFS extents
correctly).
For me the best way is to check the output of btrfs-debug-tree.

Alex.




On Sun, Sep 9, 2012 at 3:30 AM, Jan Steffens <jan.steffens@gmail.com> wrote:
> On Sun, Sep 9, 2012 at 2:03 AM, ching <lsching17@gmail.com> wrote:
>> 2. Is there any command for the fragmentation status of a file/dir ? e.g. fragment size, number of fragments.
>
> Use the "filefrag" command, part of e2fsprogs.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: enquiry about defrag

[ching]

On 09/09/2012 08:03 AM, ching wrote:
> Hi all,
>
> i am new on btrfs, i am testing KVM on btrfs (host: kernel x86-64 3.5.3), the performance is reasonable.
>
> I have two question on defrag, can someone help me?
>
> 1. According to btrfs wiki, defragment a COW file will produce two unrelated files.
>
>     Does it apply to the "autodefrag" mount option?
>
> 2. Is there any command for the fragmentation status of a file/dir ? e.g. fragment size, number of fragments.

can anybody helps on question 1?

Re: enquiry about defrag

[David Sterba]

On Tue, Sep 11, 2012 at 07:12:58PM +0800, ching wrote:
> > 1. According to btrfs wiki, defragment a COW file will produce two unrelated files.
> >
> >     Does it apply to the "autodefrag" mount option?
> 
> can anybody helps on question 1?

The data blocks associated with the files (that were originally created
by snapshotting and thus shared the same blocks) will become unshared,
so yes it would appear like two unlreated files.

Autodefrag option affects only newly written data. With current
implementation it could unshare some extents if the defragged range
picked by autodefrag logic overlaps with an existing one.

>From my experience, running 'btrfs fi defrag' with autodefrag on
produced better result. Beware that there are some unfixed deadlocks
that may happen with autodefrag enabled.

david

Re: enquiry about defrag

[ching]

On 09/11/2012 07:45 PM, David Sterba wrote:
> On Tue, Sep 11, 2012 at 07:12:58PM +0800, ching wrote:
>>> 1. According to btrfs wiki, defragment a COW file will produce two unrelated files.
>>>
>>>     Does it apply to the "autodefrag" mount option?
>> can anybody helps on question 1?
> The data blocks associated with the files (that were originally created
> by snapshotting and thus shared the same blocks) will become unshared,
> so yes it would appear like two unlreated files.
>
> Autodefrag option affects only newly written data. With current
> implementation it could unshare some extents if the defragged range
> picked by autodefrag logic overlaps with an existing one.
>
> >From my experience, running 'btrfs fi defrag' with autodefrag on
> produced better result. Beware that there are some unfixed deadlocks
> that may happen with autodefrag enabled.
>
> david
>

thanks for your help very much :)