From: Tobias Reinhard <trtracer@gmail.com>
To: "Austin S. Hemmelgarn" <ahferroin7@gmail.com>
Cc: linux-btrfs@vger.kernel.org, Qu Wenruo <quwenruo.btrfs@gmx.com>
Subject: Re: Effect of punching holes
Date: Thu, 24 Oct 2019 20:52:41 +0200 [thread overview]
Message-ID: <f082e863-de9e-2dcc-0b9a-e4ff91cb3701@gmail.com> (raw)
In-Reply-To: <d3658060-83ee-c9c2-52b2-95d60d1ac0ca@gmail.com>
Am Di., 22. Okt. 2019 um 15:04 Uhr schrieb Austin S. Hemmelgarn
<ahferroin7@gmail.com <mailto:ahferroin7@gmail.com>>:
On 2019-10-22 06:01, Qu Wenruo wrote:
>
>
> On 2019/10/22 下午5:47, Tobias Reinhard wrote:
>> Hi,
>>
>>
>> I noticed that if you punch a hole in the middle of a file the
available
>> filesystem space seems not to increase.
>>
>> Kernel is 5.2.11
>>
>> To reproduce:
>>
>> ->mkfs.btrfs /dev/loop1 -f
>>
>> btrfs-progs v4.15.1
>> See http://btrfs.wiki.kernel.org for more information.
>>
>> Detected a SSD, turning off metadata duplication. Mkfs with -m
dup if
>> you want to force metadata duplication.
>> Label: (null)
>> UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587
>> Node size: 16384
>> Sector size: 4096
>> Filesystem size: 1.00GiB
>> Block group profiles:
>> Data: single 8.00MiB
>> Metadata: single 8.00MiB
>> System: single 4.00MiB
>> SSD detected: yes
>> Incompat features: extref, skinny-metadata
>> Number of devices: 1
>> Devices:
>> ID SIZE PATH
>> 1 1.00GiB /dev/loop1
>>
>> ->mount /dev/loop1 /srv/btrtest2
>>
>> ->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M
count=4 ;
>> sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done
>>
>> this failed from the 16th file on because of no space left
>
> Btrfs doesn't free the space until all space of a data extent get
freed.
>
> In your case, your hole punch is [4k, 64M-4K), thus the 64M
extent still
> has 4K being used.
> So the data extent won't be freed until you free the last 4K.
>
>>
>> ->df -T .
>> Filesystem Type 1K-blocks Used Available Use% Mounted on
>> /dev/loop1 btrfs 1048576 935856 2272 100% /srv/btrtest2
>>
>> ->btrfs fi du .
>> Total Exclusive Set shared Filename
>> 8.00KiB 8.00KiB - ./test1
>> 8.00KiB 8.00KiB - ./test2
>> 8.00KiB 8.00KiB - ./test3
>> 8.00KiB 8.00KiB - ./test4
>> 8.00KiB 8.00KiB - ./test5
>> 8.00KiB 8.00KiB - ./test6
>> 8.00KiB 8.00KiB - ./test7
>> 8.00KiB 8.00KiB - ./test8
>> 8.00KiB 8.00KiB - ./test9
>> 8.00KiB 8.00KiB - ./test10
>> 8.00KiB 8.00KiB - ./test11
>> 8.00KiB 8.00KiB - ./test12
>> 8.00KiB 8.00KiB - ./test13
>> 8.00KiB 8.00KiB - ./test14
>> 8.00KiB 8.00KiB - ./test15
>> 4.00KiB 4.00KiB - ./test16
>> 4.00KiB 4.00KiB - ./test17
>> 4.00KiB 4.00KiB - ./test18
>> 4.00KiB 4.00KiB - ./test19
>> 4.00KiB 4.00KiB - ./test20
>> 140.00KiB 140.00KiB 0.00B .
>>
>> When doing this on XFS or EXT4 it works as expected:
>>
>> Filesystem Type 1K-blocks Used Available Use% Mounted on
>> /dev/loop1 ext4 999320 2764 927744 1% /srv/btrtest
>> /dev/loop2 xfs 1038336 40456 997880 4% /srv/xfstest
>>
>> How to i reclaim the space on BTRFS? Defrag does not seem to help.
>
> Rewrite the remaining 4K.
>
> Then the new write 4K will be cowed into a new 4K extent, the old
large
> 64M extent gets fully freed and free space.
Expanding on this a bit, defrag isn't working here because it doesn't,
by default, touch extents larger than 32M in size. You should be able
to make it work by using the `-t` option with a size larger than 64M.
Alternatively, use `cp --reflink=never --sparse=always` to copy the
file
and then rename the copy over the original. This will use more space,
but is likely to be significantly faster than a defrag.
(sorry - for first bad formated post)
Hi,
I can't get the defrag way to work.
What is the right command to do it?
->df -hT .
Filesystem Type Size Used Avail Use% Mounted on
/dev/loop1 btrfs 1,0G 868M 49M 95% /srv/btrtest2
->btrfs fi du .
Total Exclusive Set shared Filename
0.00B 0.00B - ./runtest.sh
8.00KiB 8.00KiB - ./test1
8.00KiB 8.00KiB - ./test2
8.00KiB 8.00KiB - ./test3
8.00KiB 8.00KiB - ./test4
8.00KiB 8.00KiB - ./test5
8.00KiB 8.00KiB - ./test6
8.00KiB 8.00KiB - ./test7
8.00KiB 8.00KiB - ./test8
8.00KiB 8.00KiB - ./test9
8.00KiB 8.00KiB - ./test10
8.00KiB 8.00KiB - ./test11
8.00KiB 8.00KiB - ./test12
8.00KiB 8.00KiB - ./test13
8.00KiB 8.00KiB - ./test14
8.00KiB 8.00KiB - ./test15
120.00KiB 120.00KiB 0.00B .
-> btrfs fi de -t 128M *
-> sync
-> df -hT .
Filesystem Type Size Used Avail Use% Mounted on
/dev/loop1 btrfs 1,0G 868M 49M 95% /srv/btrtest2
Tobias
next prev parent reply other threads:[~2019-10-24 18:52 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-22 9:47 Effect of punching holes Tobias Reinhard
2019-10-22 10:01 ` Qu Wenruo
2019-10-22 13:04 ` Austin S. Hemmelgarn
2019-10-24 18:52 ` Tobias Reinhard [this message]
2019-10-24 19:04 ` Austin S. Hemmelgarn
2019-10-24 18:54 ` Tobias Reinhard
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f082e863-de9e-2dcc-0b9a-e4ff91cb3701@gmail.com \
--to=trtracer@gmail.com \
--cc=ahferroin7@gmail.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=quwenruo.btrfs@gmx.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox