From: "Austin S. Hemmelgarn" <ahferroin7@gmail.com>
To: Tobias Reinhard <trtracer@gmail.com>
Cc: linux-btrfs@vger.kernel.org, Qu Wenruo <quwenruo.btrfs@gmx.com>
Subject: Re: Effect of punching holes
Date: Thu, 24 Oct 2019 15:04:12 -0400 [thread overview]
Message-ID: <5996737b-ce7e-4e9c-a85e-ab611621a2e6@gmail.com> (raw)
In-Reply-To: <f082e863-de9e-2dcc-0b9a-e4ff91cb3701@gmail.com>
On 2019-10-24 14:52, Tobias Reinhard wrote:
> Am Di., 22. Okt. 2019 um 15:04 Uhr schrieb Austin S. Hemmelgarn
> <ahferroin7@gmail.com <mailto:ahferroin7@gmail.com>>:
>
> On 2019-10-22 06:01, Qu Wenruo wrote:
> >
> >
> > On 2019/10/22 下午5:47, Tobias Reinhard wrote:
> >> Hi,
> >>
> >>
> >> I noticed that if you punch a hole in the middle of a file the
> available
> >> filesystem space seems not to increase.
> >>
> >> Kernel is 5.2.11
> >>
> >> To reproduce:
> >>
> >> ->mkfs.btrfs /dev/loop1 -f
> >>
> >> btrfs-progs v4.15.1
> >> See http://btrfs.wiki.kernel.org for more information.
> >>
> >> Detected a SSD, turning off metadata duplication. Mkfs with -m
> dup if
> >> you want to force metadata duplication.
> >> Label: (null)
> >> UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587
> >> Node size: 16384
> >> Sector size: 4096
> >> Filesystem size: 1.00GiB
> >> Block group profiles:
> >> Data: single 8.00MiB
> >> Metadata: single 8.00MiB
> >> System: single 4.00MiB
> >> SSD detected: yes
> >> Incompat features: extref, skinny-metadata
> >> Number of devices: 1
> >> Devices:
> >> ID SIZE PATH
> >> 1 1.00GiB /dev/loop1
> >>
> >> ->mount /dev/loop1 /srv/btrtest2
> >>
> >> ->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M
> count=4 ;
> >> sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done
> >>
> >> this failed from the 16th file on because of no space left
> >
> > Btrfs doesn't free the space until all space of a data extent get
> freed.
> >
> > In your case, your hole punch is [4k, 64M-4K), thus the 64M
> extent still
> > has 4K being used.
> > So the data extent won't be freed until you free the last 4K.
> >
> >>
> >> ->df -T .
> >> Filesystem Type 1K-blocks Used Available Use% Mounted on
> >> /dev/loop1 btrfs 1048576 935856 2272 100% /srv/btrtest2
> >>
> >> ->btrfs fi du .
> >> Total Exclusive Set shared Filename
> >> 8.00KiB 8.00KiB - ./test1
> >> 8.00KiB 8.00KiB - ./test2
> >> 8.00KiB 8.00KiB - ./test3
> >> 8.00KiB 8.00KiB - ./test4
> >> 8.00KiB 8.00KiB - ./test5
> >> 8.00KiB 8.00KiB - ./test6
> >> 8.00KiB 8.00KiB - ./test7
> >> 8.00KiB 8.00KiB - ./test8
> >> 8.00KiB 8.00KiB - ./test9
> >> 8.00KiB 8.00KiB - ./test10
> >> 8.00KiB 8.00KiB - ./test11
> >> 8.00KiB 8.00KiB - ./test12
> >> 8.00KiB 8.00KiB - ./test13
> >> 8.00KiB 8.00KiB - ./test14
> >> 8.00KiB 8.00KiB - ./test15
> >> 4.00KiB 4.00KiB - ./test16
> >> 4.00KiB 4.00KiB - ./test17
> >> 4.00KiB 4.00KiB - ./test18
> >> 4.00KiB 4.00KiB - ./test19
> >> 4.00KiB 4.00KiB - ./test20
> >> 140.00KiB 140.00KiB 0.00B .
> >>
> >> When doing this on XFS or EXT4 it works as expected:
> >>
> >> Filesystem Type 1K-blocks Used Available Use% Mounted on
> >> /dev/loop1 ext4 999320 2764 927744 1% /srv/btrtest
> >> /dev/loop2 xfs 1038336 40456 997880 4% /srv/xfstest
> >>
> >> How to i reclaim the space on BTRFS? Defrag does not seem to help.
> >
> > Rewrite the remaining 4K.
> >
> > Then the new write 4K will be cowed into a new 4K extent, the old
> large
> > 64M extent gets fully freed and free space.
>
> Expanding on this a bit, defrag isn't working here because it doesn't,
> by default, touch extents larger than 32M in size. You should be able
> to make it work by using the `-t` option with a size larger than 64M.
>
> Alternatively, use `cp --reflink=never --sparse=always` to copy the
> file
> and then rename the copy over the original. This will use more space,
> but is likely to be significantly faster than a defrag.
>
> (sorry - for first bad formated post)
>
> Hi,
>
> I can't get the defrag way to work.
>
> What is the right command to do it?
>
> ->df -hT .
> Filesystem Type Size Used Avail Use% Mounted on
> /dev/loop1 btrfs 1,0G 868M 49M 95% /srv/btrtest2
>
> ->btrfs fi du .
> Total Exclusive Set shared Filename
> 0.00B 0.00B - ./runtest.sh
> 8.00KiB 8.00KiB - ./test1
> 8.00KiB 8.00KiB - ./test2
> 8.00KiB 8.00KiB - ./test3
> 8.00KiB 8.00KiB - ./test4
> 8.00KiB 8.00KiB - ./test5
> 8.00KiB 8.00KiB - ./test6
> 8.00KiB 8.00KiB - ./test7
> 8.00KiB 8.00KiB - ./test8
> 8.00KiB 8.00KiB - ./test9
> 8.00KiB 8.00KiB - ./test10
> 8.00KiB 8.00KiB - ./test11
> 8.00KiB 8.00KiB - ./test12
> 8.00KiB 8.00KiB - ./test13
> 8.00KiB 8.00KiB - ./test14
> 8.00KiB 8.00KiB - ./test15
> 120.00KiB 120.00KiB 0.00B .
> -> btrfs fi de -t 128M *
> -> sync
> -> df -hT .
> Filesystem Type Size Used Avail Use% Mounted on
> /dev/loop1 btrfs 1,0G 868M 49M 95% /srv/btrtest2
That's odd, what you used _should_ do it. For some reason, it's not
trying to rewrite things at all. In cases like this, you can force it
to rewrite the data by telling it to compress the file using a different
algorithm than whatever you have specified via mount options (or just
compress it at all if you don't have compression enabled for the mount)
and then re-defraging it with the original compression type.
next prev parent reply other threads:[~2019-10-24 19:04 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-22 9:47 Effect of punching holes Tobias Reinhard
2019-10-22 10:01 ` Qu Wenruo
2019-10-22 13:04 ` Austin S. Hemmelgarn
2019-10-24 18:52 ` Tobias Reinhard
2019-10-24 19:04 ` Austin S. Hemmelgarn [this message]
2019-10-24 18:54 ` Tobias Reinhard
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5996737b-ce7e-4e9c-a85e-ab611621a2e6@gmail.com \
--to=ahferroin7@gmail.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=quwenruo.btrfs@gmx.com \
--cc=trtracer@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox