Re: Effect of punching holes

["Austin S. Hemmelgarn" <ahferroin7@gmail.com>]

On 2019-10-22 06:01, Qu Wenruo wrote:
> 
> 
> On 2019/10/22 下午5:47, Tobias Reinhard wrote:
>> Hi,
>>
>>
>> I noticed that if you punch a hole in the middle of a file the available
>> filesystem space seems not to increase.
>>
>> Kernel is 5.2.11
>>
>> To reproduce:
>>
>> ->mkfs.btrfs /dev/loop1 -f
>>
>> btrfs-progs v4.15.1
>> See http://btrfs.wiki.kernel.org for more information.
>>
>> Detected a SSD, turning off metadata duplication.  Mkfs with -m dup if
>> you want to force metadata duplication.
>> Label:              (null)
>> UUID:               415e925a-588a-4b8f-bdc7-c30a4a0f5587
>> Node size:          16384
>> Sector size:        4096
>> Filesystem size:    1.00GiB
>> Block group profiles:
>>    Data:             single            8.00MiB
>>    Metadata:         single            8.00MiB
>>    System:           single            4.00MiB
>> SSD detected:       yes
>> Incompat features:  extref, skinny-metadata
>> Number of devices:  1
>> Devices:
>>     ID        SIZE  PATH
>>      1     1.00GiB  /dev/loop1
>>
>> ->mount /dev/loop1 /srv/btrtest2
>>
>> ->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M count=4 ;
>> sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done
>>
>> this failed from the 16th file on because of no space left
> 
> Btrfs doesn't free the space until all space of a data extent get freed.
> 
> In your case, your hole punch is [4k, 64M-4K), thus the 64M extent still
> has 4K being used.
> So the data extent won't be freed until you free the last 4K.
> 
>>
>> ->df -T .
>> Filesystem     Type  1K-blocks   Used Available Use% Mounted on
>> /dev/loop1     btrfs   1048576 935856      2272 100% /srv/btrtest2
>>
>> ->btrfs fi du .
>>       Total   Exclusive  Set shared  Filename
>>     8.00KiB     8.00KiB           -  ./test1
>>     8.00KiB     8.00KiB           -  ./test2
>>     8.00KiB     8.00KiB           -  ./test3
>>     8.00KiB     8.00KiB           -  ./test4
>>     8.00KiB     8.00KiB           -  ./test5
>>     8.00KiB     8.00KiB           -  ./test6
>>     8.00KiB     8.00KiB           -  ./test7
>>     8.00KiB     8.00KiB           -  ./test8
>>     8.00KiB     8.00KiB           -  ./test9
>>     8.00KiB     8.00KiB           -  ./test10
>>     8.00KiB     8.00KiB           -  ./test11
>>     8.00KiB     8.00KiB           -  ./test12
>>     8.00KiB     8.00KiB           -  ./test13
>>     8.00KiB     8.00KiB           -  ./test14
>>     8.00KiB     8.00KiB           -  ./test15
>>     4.00KiB     4.00KiB           -  ./test16
>>     4.00KiB     4.00KiB           -  ./test17
>>     4.00KiB     4.00KiB           -  ./test18
>>     4.00KiB     4.00KiB           -  ./test19
>>     4.00KiB     4.00KiB           -  ./test20
>>   140.00KiB   140.00KiB       0.00B  .
>>
>> When doing this on XFS or EXT4 it works as expected:
>>
>> Filesystem     Type 1K-blocks  Used Available Use% Mounted on
>> /dev/loop1     ext4    999320  2764    927744   1% /srv/btrtest
>> /dev/loop2     xfs    1038336 40456    997880   4% /srv/xfstest
>>
>> How to i reclaim the space on BTRFS? Defrag does not seem to help.
> 
> Rewrite the remaining 4K.
> 
> Then the new write 4K will be cowed into a new 4K extent, the old large
> 64M extent gets fully freed and free space.

Expanding on this a bit, defrag isn't working here because it doesn't, 
by default, touch extents larger than 32M in size.  You should be able 
to make it work by using the `-t` option with a size larger than 64M.

Alternatively, use `cp --reflink=never --sparse=always` to copy the file 
and then rename the copy over the original.  This will use more space, 
but is likely to be significantly faster than a defrag.