* Effect of punching holes @ 2019-10-22 9:47 Tobias Reinhard 2019-10-22 10:01 ` Qu Wenruo 0 siblings, 1 reply; 6+ messages in thread From: Tobias Reinhard @ 2019-10-22 9:47 UTC (permalink / raw) To: linux-btrfs Hi, I noticed that if you punch a hole in the middle of a file the available filesystem space seems not to increase. Kernel is 5.2.11 To reproduce: ->mkfs.btrfs /dev/loop1 -f btrfs-progs v4.15.1 See http://btrfs.wiki.kernel.org for more information. Detected a SSD, turning off metadata duplication. Mkfs with -m dup if you want to force metadata duplication. Label: (null) UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587 Node size: 16384 Sector size: 4096 Filesystem size: 1.00GiB Block group profiles: Data: single 8.00MiB Metadata: single 8.00MiB System: single 4.00MiB SSD detected: yes Incompat features: extref, skinny-metadata Number of devices: 1 Devices: ID SIZE PATH 1 1.00GiB /dev/loop1 ->mount /dev/loop1 /srv/btrtest2 ->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M count=4 ; sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done this failed from the 16th file on because of no space left ->df -T . Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop1 btrfs 1048576 935856 2272 100% /srv/btrtest2 ->btrfs fi du . Total Exclusive Set shared Filename 8.00KiB 8.00KiB - ./test1 8.00KiB 8.00KiB - ./test2 8.00KiB 8.00KiB - ./test3 8.00KiB 8.00KiB - ./test4 8.00KiB 8.00KiB - ./test5 8.00KiB 8.00KiB - ./test6 8.00KiB 8.00KiB - ./test7 8.00KiB 8.00KiB - ./test8 8.00KiB 8.00KiB - ./test9 8.00KiB 8.00KiB - ./test10 8.00KiB 8.00KiB - ./test11 8.00KiB 8.00KiB - ./test12 8.00KiB 8.00KiB - ./test13 8.00KiB 8.00KiB - ./test14 8.00KiB 8.00KiB - ./test15 4.00KiB 4.00KiB - ./test16 4.00KiB 4.00KiB - ./test17 4.00KiB 4.00KiB - ./test18 4.00KiB 4.00KiB - ./test19 4.00KiB 4.00KiB - ./test20 140.00KiB 140.00KiB 0.00B . When doing this on XFS or EXT4 it works as expected: Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop1 ext4 999320 2764 927744 1% /srv/btrtest /dev/loop2 xfs 1038336 40456 997880 4% /srv/xfstest How to i reclaim the space on BTRFS? Defrag does not seem to help. Best regards Tobias ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Effect of punching holes 2019-10-22 9:47 Effect of punching holes Tobias Reinhard @ 2019-10-22 10:01 ` Qu Wenruo 2019-10-22 13:04 ` Austin S. Hemmelgarn 2019-10-24 18:54 ` Tobias Reinhard 0 siblings, 2 replies; 6+ messages in thread From: Qu Wenruo @ 2019-10-22 10:01 UTC (permalink / raw) To: Tobias Reinhard, linux-btrfs [-- Attachment #1.1: Type: text/plain, Size: 3791 bytes --] On 2019/10/22 下午5:47, Tobias Reinhard wrote: > Hi, > > > I noticed that if you punch a hole in the middle of a file the available > filesystem space seems not to increase. > > Kernel is 5.2.11 > > To reproduce: > > ->mkfs.btrfs /dev/loop1 -f > > btrfs-progs v4.15.1 > See http://btrfs.wiki.kernel.org for more information. > > Detected a SSD, turning off metadata duplication. Mkfs with -m dup if > you want to force metadata duplication. > Label: (null) > UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587 > Node size: 16384 > Sector size: 4096 > Filesystem size: 1.00GiB > Block group profiles: > Data: single 8.00MiB > Metadata: single 8.00MiB > System: single 4.00MiB > SSD detected: yes > Incompat features: extref, skinny-metadata > Number of devices: 1 > Devices: > ID SIZE PATH > 1 1.00GiB /dev/loop1 > > ->mount /dev/loop1 /srv/btrtest2 > > ->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M count=4 ; > sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done > > this failed from the 16th file on because of no space left Btrfs doesn't free the space until all space of a data extent get freed. In your case, your hole punch is [4k, 64M-4K), thus the 64M extent still has 4K being used. So the data extent won't be freed until you free the last 4K. > > ->df -T . > Filesystem Type 1K-blocks Used Available Use% Mounted on > /dev/loop1 btrfs 1048576 935856 2272 100% /srv/btrtest2 > > ->btrfs fi du . > Total Exclusive Set shared Filename > 8.00KiB 8.00KiB - ./test1 > 8.00KiB 8.00KiB - ./test2 > 8.00KiB 8.00KiB - ./test3 > 8.00KiB 8.00KiB - ./test4 > 8.00KiB 8.00KiB - ./test5 > 8.00KiB 8.00KiB - ./test6 > 8.00KiB 8.00KiB - ./test7 > 8.00KiB 8.00KiB - ./test8 > 8.00KiB 8.00KiB - ./test9 > 8.00KiB 8.00KiB - ./test10 > 8.00KiB 8.00KiB - ./test11 > 8.00KiB 8.00KiB - ./test12 > 8.00KiB 8.00KiB - ./test13 > 8.00KiB 8.00KiB - ./test14 > 8.00KiB 8.00KiB - ./test15 > 4.00KiB 4.00KiB - ./test16 > 4.00KiB 4.00KiB - ./test17 > 4.00KiB 4.00KiB - ./test18 > 4.00KiB 4.00KiB - ./test19 > 4.00KiB 4.00KiB - ./test20 > 140.00KiB 140.00KiB 0.00B . > > When doing this on XFS or EXT4 it works as expected: > > Filesystem Type 1K-blocks Used Available Use% Mounted on > /dev/loop1 ext4 999320 2764 927744 1% /srv/btrtest > /dev/loop2 xfs 1038336 40456 997880 4% /srv/xfstest > > How to i reclaim the space on BTRFS? Defrag does not seem to help. Rewrite the remaining 4K. Then the new write 4K will be cowed into a new 4K extent, the old large 64M extent gets fully freed and free space. Thanks, Qu > > Best regards > > > Tobias > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 520 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Effect of punching holes 2019-10-22 10:01 ` Qu Wenruo @ 2019-10-22 13:04 ` Austin S. Hemmelgarn 2019-10-24 18:52 ` Tobias Reinhard 2019-10-24 18:54 ` Tobias Reinhard 1 sibling, 1 reply; 6+ messages in thread From: Austin S. Hemmelgarn @ 2019-10-22 13:04 UTC (permalink / raw) To: Tobias Reinhard, linux-btrfs; +Cc: Qu Wenruo On 2019-10-22 06:01, Qu Wenruo wrote: > > > On 2019/10/22 下午5:47, Tobias Reinhard wrote: >> Hi, >> >> >> I noticed that if you punch a hole in the middle of a file the available >> filesystem space seems not to increase. >> >> Kernel is 5.2.11 >> >> To reproduce: >> >> ->mkfs.btrfs /dev/loop1 -f >> >> btrfs-progs v4.15.1 >> See http://btrfs.wiki.kernel.org for more information. >> >> Detected a SSD, turning off metadata duplication. Mkfs with -m dup if >> you want to force metadata duplication. >> Label: (null) >> UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587 >> Node size: 16384 >> Sector size: 4096 >> Filesystem size: 1.00GiB >> Block group profiles: >> Data: single 8.00MiB >> Metadata: single 8.00MiB >> System: single 4.00MiB >> SSD detected: yes >> Incompat features: extref, skinny-metadata >> Number of devices: 1 >> Devices: >> ID SIZE PATH >> 1 1.00GiB /dev/loop1 >> >> ->mount /dev/loop1 /srv/btrtest2 >> >> ->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M count=4 ; >> sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done >> >> this failed from the 16th file on because of no space left > > Btrfs doesn't free the space until all space of a data extent get freed. > > In your case, your hole punch is [4k, 64M-4K), thus the 64M extent still > has 4K being used. > So the data extent won't be freed until you free the last 4K. > >> >> ->df -T . >> Filesystem Type 1K-blocks Used Available Use% Mounted on >> /dev/loop1 btrfs 1048576 935856 2272 100% /srv/btrtest2 >> >> ->btrfs fi du . >> Total Exclusive Set shared Filename >> 8.00KiB 8.00KiB - ./test1 >> 8.00KiB 8.00KiB - ./test2 >> 8.00KiB 8.00KiB - ./test3 >> 8.00KiB 8.00KiB - ./test4 >> 8.00KiB 8.00KiB - ./test5 >> 8.00KiB 8.00KiB - ./test6 >> 8.00KiB 8.00KiB - ./test7 >> 8.00KiB 8.00KiB - ./test8 >> 8.00KiB 8.00KiB - ./test9 >> 8.00KiB 8.00KiB - ./test10 >> 8.00KiB 8.00KiB - ./test11 >> 8.00KiB 8.00KiB - ./test12 >> 8.00KiB 8.00KiB - ./test13 >> 8.00KiB 8.00KiB - ./test14 >> 8.00KiB 8.00KiB - ./test15 >> 4.00KiB 4.00KiB - ./test16 >> 4.00KiB 4.00KiB - ./test17 >> 4.00KiB 4.00KiB - ./test18 >> 4.00KiB 4.00KiB - ./test19 >> 4.00KiB 4.00KiB - ./test20 >> 140.00KiB 140.00KiB 0.00B . >> >> When doing this on XFS or EXT4 it works as expected: >> >> Filesystem Type 1K-blocks Used Available Use% Mounted on >> /dev/loop1 ext4 999320 2764 927744 1% /srv/btrtest >> /dev/loop2 xfs 1038336 40456 997880 4% /srv/xfstest >> >> How to i reclaim the space on BTRFS? Defrag does not seem to help. > > Rewrite the remaining 4K. > > Then the new write 4K will be cowed into a new 4K extent, the old large > 64M extent gets fully freed and free space. Expanding on this a bit, defrag isn't working here because it doesn't, by default, touch extents larger than 32M in size. You should be able to make it work by using the `-t` option with a size larger than 64M. Alternatively, use `cp --reflink=never --sparse=always` to copy the file and then rename the copy over the original. This will use more space, but is likely to be significantly faster than a defrag. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Effect of punching holes 2019-10-22 13:04 ` Austin S. Hemmelgarn @ 2019-10-24 18:52 ` Tobias Reinhard 2019-10-24 19:04 ` Austin S. Hemmelgarn 0 siblings, 1 reply; 6+ messages in thread From: Tobias Reinhard @ 2019-10-24 18:52 UTC (permalink / raw) To: Austin S. Hemmelgarn; +Cc: linux-btrfs, Qu Wenruo Am Di., 22. Okt. 2019 um 15:04 Uhr schrieb Austin S. Hemmelgarn <ahferroin7@gmail.com <mailto:ahferroin7@gmail.com>>: On 2019-10-22 06:01, Qu Wenruo wrote: > > > On 2019/10/22 下午5:47, Tobias Reinhard wrote: >> Hi, >> >> >> I noticed that if you punch a hole in the middle of a file the available >> filesystem space seems not to increase. >> >> Kernel is 5.2.11 >> >> To reproduce: >> >> ->mkfs.btrfs /dev/loop1 -f >> >> btrfs-progs v4.15.1 >> See http://btrfs.wiki.kernel.org for more information. >> >> Detected a SSD, turning off metadata duplication. Mkfs with -m dup if >> you want to force metadata duplication. >> Label: (null) >> UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587 >> Node size: 16384 >> Sector size: 4096 >> Filesystem size: 1.00GiB >> Block group profiles: >> Data: single 8.00MiB >> Metadata: single 8.00MiB >> System: single 4.00MiB >> SSD detected: yes >> Incompat features: extref, skinny-metadata >> Number of devices: 1 >> Devices: >> ID SIZE PATH >> 1 1.00GiB /dev/loop1 >> >> ->mount /dev/loop1 /srv/btrtest2 >> >> ->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M count=4 ; >> sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done >> >> this failed from the 16th file on because of no space left > > Btrfs doesn't free the space until all space of a data extent get freed. > > In your case, your hole punch is [4k, 64M-4K), thus the 64M extent still > has 4K being used. > So the data extent won't be freed until you free the last 4K. > >> >> ->df -T . >> Filesystem Type 1K-blocks Used Available Use% Mounted on >> /dev/loop1 btrfs 1048576 935856 2272 100% /srv/btrtest2 >> >> ->btrfs fi du . >> Total Exclusive Set shared Filename >> 8.00KiB 8.00KiB - ./test1 >> 8.00KiB 8.00KiB - ./test2 >> 8.00KiB 8.00KiB - ./test3 >> 8.00KiB 8.00KiB - ./test4 >> 8.00KiB 8.00KiB - ./test5 >> 8.00KiB 8.00KiB - ./test6 >> 8.00KiB 8.00KiB - ./test7 >> 8.00KiB 8.00KiB - ./test8 >> 8.00KiB 8.00KiB - ./test9 >> 8.00KiB 8.00KiB - ./test10 >> 8.00KiB 8.00KiB - ./test11 >> 8.00KiB 8.00KiB - ./test12 >> 8.00KiB 8.00KiB - ./test13 >> 8.00KiB 8.00KiB - ./test14 >> 8.00KiB 8.00KiB - ./test15 >> 4.00KiB 4.00KiB - ./test16 >> 4.00KiB 4.00KiB - ./test17 >> 4.00KiB 4.00KiB - ./test18 >> 4.00KiB 4.00KiB - ./test19 >> 4.00KiB 4.00KiB - ./test20 >> 140.00KiB 140.00KiB 0.00B . >> >> When doing this on XFS or EXT4 it works as expected: >> >> Filesystem Type 1K-blocks Used Available Use% Mounted on >> /dev/loop1 ext4 999320 2764 927744 1% /srv/btrtest >> /dev/loop2 xfs 1038336 40456 997880 4% /srv/xfstest >> >> How to i reclaim the space on BTRFS? Defrag does not seem to help. > > Rewrite the remaining 4K. > > Then the new write 4K will be cowed into a new 4K extent, the old large > 64M extent gets fully freed and free space. Expanding on this a bit, defrag isn't working here because it doesn't, by default, touch extents larger than 32M in size. You should be able to make it work by using the `-t` option with a size larger than 64M. Alternatively, use `cp --reflink=never --sparse=always` to copy the file and then rename the copy over the original. This will use more space, but is likely to be significantly faster than a defrag. (sorry - for first bad formated post) Hi, I can't get the defrag way to work. What is the right command to do it? ->df -hT . Filesystem Type Size Used Avail Use% Mounted on /dev/loop1 btrfs 1,0G 868M 49M 95% /srv/btrtest2 ->btrfs fi du . Total Exclusive Set shared Filename 0.00B 0.00B - ./runtest.sh 8.00KiB 8.00KiB - ./test1 8.00KiB 8.00KiB - ./test2 8.00KiB 8.00KiB - ./test3 8.00KiB 8.00KiB - ./test4 8.00KiB 8.00KiB - ./test5 8.00KiB 8.00KiB - ./test6 8.00KiB 8.00KiB - ./test7 8.00KiB 8.00KiB - ./test8 8.00KiB 8.00KiB - ./test9 8.00KiB 8.00KiB - ./test10 8.00KiB 8.00KiB - ./test11 8.00KiB 8.00KiB - ./test12 8.00KiB 8.00KiB - ./test13 8.00KiB 8.00KiB - ./test14 8.00KiB 8.00KiB - ./test15 120.00KiB 120.00KiB 0.00B . -> btrfs fi de -t 128M * -> sync -> df -hT . Filesystem Type Size Used Avail Use% Mounted on /dev/loop1 btrfs 1,0G 868M 49M 95% /srv/btrtest2 Tobias ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Effect of punching holes 2019-10-24 18:52 ` Tobias Reinhard @ 2019-10-24 19:04 ` Austin S. Hemmelgarn 0 siblings, 0 replies; 6+ messages in thread From: Austin S. Hemmelgarn @ 2019-10-24 19:04 UTC (permalink / raw) To: Tobias Reinhard; +Cc: linux-btrfs, Qu Wenruo On 2019-10-24 14:52, Tobias Reinhard wrote: > Am Di., 22. Okt. 2019 um 15:04 Uhr schrieb Austin S. Hemmelgarn > <ahferroin7@gmail.com <mailto:ahferroin7@gmail.com>>: > > On 2019-10-22 06:01, Qu Wenruo wrote: > > > > > > On 2019/10/22 下午5:47, Tobias Reinhard wrote: > >> Hi, > >> > >> > >> I noticed that if you punch a hole in the middle of a file the > available > >> filesystem space seems not to increase. > >> > >> Kernel is 5.2.11 > >> > >> To reproduce: > >> > >> ->mkfs.btrfs /dev/loop1 -f > >> > >> btrfs-progs v4.15.1 > >> See http://btrfs.wiki.kernel.org for more information. > >> > >> Detected a SSD, turning off metadata duplication. Mkfs with -m > dup if > >> you want to force metadata duplication. > >> Label: (null) > >> UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587 > >> Node size: 16384 > >> Sector size: 4096 > >> Filesystem size: 1.00GiB > >> Block group profiles: > >> Data: single 8.00MiB > >> Metadata: single 8.00MiB > >> System: single 4.00MiB > >> SSD detected: yes > >> Incompat features: extref, skinny-metadata > >> Number of devices: 1 > >> Devices: > >> ID SIZE PATH > >> 1 1.00GiB /dev/loop1 > >> > >> ->mount /dev/loop1 /srv/btrtest2 > >> > >> ->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M > count=4 ; > >> sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done > >> > >> this failed from the 16th file on because of no space left > > > > Btrfs doesn't free the space until all space of a data extent get > freed. > > > > In your case, your hole punch is [4k, 64M-4K), thus the 64M > extent still > > has 4K being used. > > So the data extent won't be freed until you free the last 4K. > > > >> > >> ->df -T . > >> Filesystem Type 1K-blocks Used Available Use% Mounted on > >> /dev/loop1 btrfs 1048576 935856 2272 100% /srv/btrtest2 > >> > >> ->btrfs fi du . > >> Total Exclusive Set shared Filename > >> 8.00KiB 8.00KiB - ./test1 > >> 8.00KiB 8.00KiB - ./test2 > >> 8.00KiB 8.00KiB - ./test3 > >> 8.00KiB 8.00KiB - ./test4 > >> 8.00KiB 8.00KiB - ./test5 > >> 8.00KiB 8.00KiB - ./test6 > >> 8.00KiB 8.00KiB - ./test7 > >> 8.00KiB 8.00KiB - ./test8 > >> 8.00KiB 8.00KiB - ./test9 > >> 8.00KiB 8.00KiB - ./test10 > >> 8.00KiB 8.00KiB - ./test11 > >> 8.00KiB 8.00KiB - ./test12 > >> 8.00KiB 8.00KiB - ./test13 > >> 8.00KiB 8.00KiB - ./test14 > >> 8.00KiB 8.00KiB - ./test15 > >> 4.00KiB 4.00KiB - ./test16 > >> 4.00KiB 4.00KiB - ./test17 > >> 4.00KiB 4.00KiB - ./test18 > >> 4.00KiB 4.00KiB - ./test19 > >> 4.00KiB 4.00KiB - ./test20 > >> 140.00KiB 140.00KiB 0.00B . > >> > >> When doing this on XFS or EXT4 it works as expected: > >> > >> Filesystem Type 1K-blocks Used Available Use% Mounted on > >> /dev/loop1 ext4 999320 2764 927744 1% /srv/btrtest > >> /dev/loop2 xfs 1038336 40456 997880 4% /srv/xfstest > >> > >> How to i reclaim the space on BTRFS? Defrag does not seem to help. > > > > Rewrite the remaining 4K. > > > > Then the new write 4K will be cowed into a new 4K extent, the old > large > > 64M extent gets fully freed and free space. > > Expanding on this a bit, defrag isn't working here because it doesn't, > by default, touch extents larger than 32M in size. You should be able > to make it work by using the `-t` option with a size larger than 64M. > > Alternatively, use `cp --reflink=never --sparse=always` to copy the > file > and then rename the copy over the original. This will use more space, > but is likely to be significantly faster than a defrag. > > (sorry - for first bad formated post) > > Hi, > > I can't get the defrag way to work. > > What is the right command to do it? > > ->df -hT . > Filesystem Type Size Used Avail Use% Mounted on > /dev/loop1 btrfs 1,0G 868M 49M 95% /srv/btrtest2 > > ->btrfs fi du . > Total Exclusive Set shared Filename > 0.00B 0.00B - ./runtest.sh > 8.00KiB 8.00KiB - ./test1 > 8.00KiB 8.00KiB - ./test2 > 8.00KiB 8.00KiB - ./test3 > 8.00KiB 8.00KiB - ./test4 > 8.00KiB 8.00KiB - ./test5 > 8.00KiB 8.00KiB - ./test6 > 8.00KiB 8.00KiB - ./test7 > 8.00KiB 8.00KiB - ./test8 > 8.00KiB 8.00KiB - ./test9 > 8.00KiB 8.00KiB - ./test10 > 8.00KiB 8.00KiB - ./test11 > 8.00KiB 8.00KiB - ./test12 > 8.00KiB 8.00KiB - ./test13 > 8.00KiB 8.00KiB - ./test14 > 8.00KiB 8.00KiB - ./test15 > 120.00KiB 120.00KiB 0.00B . > -> btrfs fi de -t 128M * > -> sync > -> df -hT . > Filesystem Type Size Used Avail Use% Mounted on > /dev/loop1 btrfs 1,0G 868M 49M 95% /srv/btrtest2 That's odd, what you used _should_ do it. For some reason, it's not trying to rewrite things at all. In cases like this, you can force it to rewrite the data by telling it to compress the file using a different algorithm than whatever you have specified via mount options (or just compress it at all if you don't have compression enabled for the mount) and then re-defraging it with the original compression type. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Effect of punching holes 2019-10-22 10:01 ` Qu Wenruo 2019-10-22 13:04 ` Austin S. Hemmelgarn @ 2019-10-24 18:54 ` Tobias Reinhard 1 sibling, 0 replies; 6+ messages in thread From: Tobias Reinhard @ 2019-10-24 18:54 UTC (permalink / raw) To: Qu Wenruo; +Cc: linux-btrfs Am Di., 22. Okt. 2019 um 12:01 Uhr schrieb Qu Wenruo <quwenruo.btrfs@gmx.com <mailto:quwenruo.btrfs@gmx.com>>: On 2019/10/22 下午5:47, Tobias Reinhard wrote: > Hi, > > > I noticed that if you punch a hole in the middle of a file the available > filesystem space seems not to increase. > > Kernel is 5.2.11 > > To reproduce: > > ->mkfs.btrfs /dev/loop1 -f > > btrfs-progs v4.15.1 > See http://btrfs.wiki.kernel.org for more information. > > Detected a SSD, turning off metadata duplication. Mkfs with -m dup if > you want to force metadata duplication. > Label: (null) > UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587 > Node size: 16384 > Sector size: 4096 > Filesystem size: 1.00GiB > Block group profiles: > Data: single 8.00MiB > Metadata: single 8.00MiB > System: single 4.00MiB > SSD detected: yes > Incompat features: extref, skinny-metadata > Number of devices: 1 > Devices: > ID SIZE PATH > 1 1.00GiB /dev/loop1 > > ->mount /dev/loop1 /srv/btrtest2 > > ->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M count=4 ; > sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done > > this failed from the 16th file on because of no space left Btrfs doesn't free the space until all space of a data extent get freed. In your case, your hole punch is [4k, 64M-4K), thus the 64M extent still has 4K being used. So the data extent won't be freed until you free the last 4K. > > ->df -T . > Filesystem Type 1K-blocks Used Available Use% Mounted on > /dev/loop1 btrfs 1048576 935856 2272 100% /srv/btrtest2 > > ->btrfs fi du . > Total Exclusive Set shared Filename > 8.00KiB 8.00KiB - ./test1 > 8.00KiB 8.00KiB - ./test2 > 8.00KiB 8.00KiB - ./test3 > 8.00KiB 8.00KiB - ./test4 > 8.00KiB 8.00KiB - ./test5 > 8.00KiB 8.00KiB - ./test6 > 8.00KiB 8.00KiB - ./test7 > 8.00KiB 8.00KiB - ./test8 > 8.00KiB 8.00KiB - ./test9 > 8.00KiB 8.00KiB - ./test10 > 8.00KiB 8.00KiB - ./test11 > 8.00KiB 8.00KiB - ./test12 > 8.00KiB 8.00KiB - ./test13 > 8.00KiB 8.00KiB - ./test14 > 8.00KiB 8.00KiB - ./test15 > 4.00KiB 4.00KiB - ./test16 > 4.00KiB 4.00KiB - ./test17 > 4.00KiB 4.00KiB - ./test18 > 4.00KiB 4.00KiB - ./test19 > 4.00KiB 4.00KiB - ./test20 > 140.00KiB 140.00KiB 0.00B . > > When doing this on XFS or EXT4 it works as expected: > > Filesystem Type 1K-blocks Used Available Use% Mounted on > /dev/loop1 ext4 999320 2764 927744 1% /srv/btrtest > /dev/loop2 xfs 1038336 40456 997880 4% /srv/xfstest > > How to i reclaim the space on BTRFS? Defrag does not seem to help. Rewrite the remaining 4K. Then the new write 4K will be cowed into a new 4K extent, the old large 64M extent gets fully freed and free space. Thanks, Qu (sorry - for previous bad formated post) Hi, okay - thanks for the explanation. How can I find out that rewriting parts of an extent would free up space? Tobias ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-10-24 19:04 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-10-22 9:47 Effect of punching holes Tobias Reinhard 2019-10-22 10:01 ` Qu Wenruo 2019-10-22 13:04 ` Austin S. Hemmelgarn 2019-10-24 18:52 ` Tobias Reinhard 2019-10-24 19:04 ` Austin S. Hemmelgarn 2019-10-24 18:54 ` Tobias Reinhard
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox