filefrag and btrfs filesystem defragment and maybe snapshots

filefrag and btrfs filesystem defragment and maybe snapshots

[Sandy McArthur]

While exploring some btrfs maintenance with respect to defragmenting I
ran the following commands:

# filefrag /path/to/34G.file /path/to/5.7G.file
/path/to/34G.file: 2406 extents found
/path/to/5.7G.file: 572 extents found

Thinking those mostly static files could be less fragmented I ran:
# btrfs filesystem defragment -c /path/to/34G.file
# btrfs filesystem defragment -c /path/to/5.7G.file

and to my surprise the number of fragments/extends doubled:

# filefrag /path/to/34G.file /path/to/5.7G.file
/path/to/34G.file: 6324 extents found
/path/to/5.7G.file: 1079 extents found

Did I actually improve these files?

I do have a number rolling readonly snapshots on the subvolume these
files are on. I can imagine how that might be related but I'm not
sure. When the pre-defrag snapshots are purged will the filefrag
extents count drop.

-- 
Sandy McArthur

"He who dares not offend cannot be honest."
- Thomas Paine

Re: filefrag and btrfs filesystem defragment and maybe snapshots

[Liu Bo]

On Thu, Aug 01, 2013 at 05:18:50PM -0400, Sandy McArthur wrote:
> While exploring some btrfs maintenance with respect to defragmenting I
> ran the following commands:
> 
> # filefrag /path/to/34G.file /path/to/5.7G.file
> /path/to/34G.file: 2406 extents found
> /path/to/5.7G.file: 572 extents found
> 
> Thinking those mostly static files could be less fragmented I ran:
> # btrfs filesystem defragment -c /path/to/34G.file
> # btrfs filesystem defragment -c /path/to/5.7G.file
> 
> and to my surprise the number of fragments/extends doubled:
> 
> # filefrag /path/to/34G.file /path/to/5.7G.file
> /path/to/34G.file: 6324 extents found
> /path/to/5.7G.file: 1079 extents found
> 
> Did I actually improve these files?
> 
> I do have a number rolling readonly snapshots on the subvolume these
> files are on. I can imagine how that might be related but I'm not
> sure. When the pre-defrag snapshots are purged will the filefrag
> extents count drop.

I guess the reason is that you're doing the defragment with compression,
ZLIB in default case, and btrfs compression has a maximum length limit, 128k.
So if the original file's extents have a length larger than 128k, it's possible
to get the results of 'filefrag' doubled.

But I have no possible answer for
'why filefrag extents count drop after purging snapshots'.

-liubo

Re: filefrag and btrfs filesystem defragment and maybe snapshots

[Duncan]

Sandy McArthur posted on Thu, 01 Aug 2013 17:18:50 -0400 as excerpted:

> While exploring some btrfs maintenance with respect to defragmenting I
> ran the following commands:
> 
> # filefrag /path/to/34G.file /path/to/5.7G.file
> /path/to/34G.file: 2406 extents found
> /path/to/5.7G.file: 572 extents found
> 
> Thinking those mostly static files could be less fragmented I ran:
> # btrfs filesystem defragment -c /path/to/34G.file
> # btrfs filesystem defragment -c /path/to/5.7G.file
> 
> and to my surprise the number of fragments/extends doubled:
> 
> # filefrag /path/to/34G.file /path/to/5.7G.file
> /path/to/34G.file: 6324 extents found
> /path/to/5.7G.file: 1079 extents found
> 
> Did I actually improve these files?
> 
> I do have a number rolling readonly snapshots on the subvolume these
> files are on. I can imagine how that might be related but I'm not sure.
> When the pre-defrag snapshots are purged will the filefrag extents count
> drop.

I can't answer the snapshot angle, but do you have btrfs compression 
turned on?  I've read that filefrag always sees btrfs compressed files of 
sufficient size as fragmented, due to the way btrfs compression works.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: filefrag and btrfs filesystem defragment and maybe snapshots

[Sandy McArthur]

On Thu, Aug 1, 2013 at 10:27 PM, Duncan <1i5t5.duncan@cox.net> wrote:
> Sandy McArthur posted on Thu, 01 Aug 2013 17:18:50 -0400 as excerpted:
>
>> While exploring some btrfs maintenance with respect to defragmenting I
>> ran the following commands:
>>
>> # filefrag /path/to/34G.file /path/to/5.7G.file
>> /path/to/34G.file: 2406 extents found
>> /path/to/5.7G.file: 572 extents found
>>
>> Thinking those mostly static files could be less fragmented I ran:
>> # btrfs filesystem defragment -c /path/to/34G.file
>> # btrfs filesystem defragment -c /path/to/5.7G.file
>>
>> and to my surprise the number of fragments/extends doubled:
>>
>> # filefrag /path/to/34G.file /path/to/5.7G.file
>> /path/to/34G.file: 6324 extents found
>> /path/to/5.7G.file: 1079 extents found
>>
>> Did I actually improve these files?
>>
>> I do have a number rolling readonly snapshots on the subvolume these
>> files are on. I can imagine how that might be related but I'm not sure.
>> When the pre-defrag snapshots are purged will the filefrag extents count
>> drop.
>
> I can't answer the snapshot angle, but do you have btrfs compression
> turned on?  I've read that filefrag always sees btrfs compressed files of
> sufficient size as fragmented, due to the way btrfs compression works.

Compress at the mount options is not enabled. The '-c' in the defrag
command is to attempt compression during the defrag process.

I found that `btrfs filesystem defragment -c /path/to/5.7G.file` (with
compression) take a while every time you run it repeatedly. Presumably
to re-compress the file each time. But `btrfs filesystem defragment
/path/to/5.7G.file` (without compression) will return instantly after
the first defrag suggesting it knows when to avoid unnecessary work.


I also observed compression does increase filefrag extents count. I
think snapshots can keep the filefrag extents count high:

# filefrag /path/to/5.7G.file
/path/to/5.7G.file: 1173 extents found

# btrfs subvolume delete /path/to/.snapshots/*
Delete subvolume '/path/to/.snapshots/2013....'
[...]

# filefrag /path/to/5.7G.file
/path/to/5.7G.file: 556 extents found

Purging the old snapshots above reduced extents count.


# btrfs filesystem defragment -c /path/to/5.7G.file
# filefrag /path/to/5.7G.file
/path/to/5.7G.file: 1260 extents found

Compressing the file "increases" extents count.


# btrfs filesystem defragment /path/to/5.7G.file
# filefrag /path/to/5.7G.file
/path/to/5.7G.file: 597 extents found

Uncompressing file lowers extents count.

# /snapshop-script.sh
# filefrag /path/to/5.7G.file
/path/to/5.7G.file: 597 extents found

Taking a snapshot didn't directly increase the extents count.

I don't know how to intentionally fragment a file to test other
permutations of snapshots and defrag behavior.

-- 
Sandy McArthur

"He who dares not offend cannot be honest."
- Thomas Paine