* Exclusive quota of snapshot exceeded despite no space used
@ 2015-10-22 20:38 Johannes Henninger
2015-10-22 23:47 ` Qu Wenruo
0 siblings, 1 reply; 11+ messages in thread
From: Johannes Henninger @ 2015-10-22 20:38 UTC (permalink / raw)
To: linux-btrfs
I'm having a weird problem with snapshots and exclusive quotas. After
creating a snapshot of a subvolume and setting an exclusive quota of
50MB for the snapshot, everything seems to work fine. I can write
approximately 50MB before the quota kicks in.
However, if I create a snapshot, set an exclusive quota and just wait
for some time, I suddenly cannot even create an empty file because I'm
getting a "quota exceeded" error. The time until the bug appears seems
to vary. During the waiting time, I'm changing neither the snapshot nor
the original subvolume. "qgroup show -e" reports an exclusive use of
only a few kilobytes for the snapshot, which is nowhere near the limit.
Steps to reproduce (/media/extern is a fresh and empty btrfs partition):
Enable quota and create an empty subvolume:
root@t420:/media/extern# btrfs quota enable .
root@t420:/media/extern# btrfs subvolume create sub
Create subvolume './sub'
Snapshot the subvolume and set a limit:
root@t420:/media/extern# btrfs subvolume snapshot sub snap
Create a snapshot of 'sub' in './snap'
root@t420:/media/extern# cd snap/
root@t420:/media/extern/snap# btrfs qgroup limit -e 50M .
Sometimes it takes "longer" for the quota to kick in, so I'm touching a
file every 5 minutes here:
root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
sleep 5m; done
touch: cannot touch ‘7’: Disk quota exceeded
^C
root@t420:/media/extern/snap# btrfs qgroup show -e .
qgroupid rfer excl max_excl
-------- ---- ---- --------
0/5 16.00KiB 16.00KiB none
0/257 16.00KiB 16.00KiB none
0/258 16.00KiB 16.00KiB 50.00MiB
Any idea why this happens?
Thanks,
Johannes
System info:
Linux t420 4.3.0-rc5 #1 SMP Tue Oct 13 13:21:02 CEST 2015 x86_64
GNU/Linux
Label: none uuid: 9551e3ca-1608-469c-9d8c-77b99ce0e8ec
Total devices 1 FS bytes used 816.00KiB
devid 1 size 931.51GiB used 2.04GiB path /dev/sdb1
btrfs-progs v4.1.2
Data, single: total=8.00MiB, used=256.00KiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=1.00GiB, used=544.00KiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=16.00MiB, used=0.00B
[249174.151820] sdb: sdb1
[249184.387377] sdb: sdb1
[249184.573096] sdb: sdb1
[249184.656274] BTRFS: device fsid
9551e3ca-1608-469c-9d8c-77b99ce0e8ec devid 1 transid 3 /dev/sdb1
[249186.323915] sdb: sdb1
[249186.534505] sdb: sdb1
[249186.538420] sdb: sdb1
[249196.781978] BTRFS info (device sdb1): disk space caching is enabled
[249196.781986] BTRFS: has skinny extents
[249196.781990] BTRFS: flagging fs with big metadata feature
[249196.818164] BTRFS: creating UUID tree
[249202.311983] BTRFS info (device sdb1): qgroup scan completed
(inconsistency flag cleared)
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: Exclusive quota of snapshot exceeded despite no space used 2015-10-22 20:38 Exclusive quota of snapshot exceeded despite no space used Johannes Henninger @ 2015-10-22 23:47 ` Qu Wenruo 2015-10-23 15:05 ` Johannes Henninger 0 siblings, 1 reply; 11+ messages in thread From: Qu Wenruo @ 2015-10-22 23:47 UTC (permalink / raw) To: Johannes Henninger, linux-btrfs 在 2015年10月23日 04:38, Johannes Henninger 写道: > I'm having a weird problem with snapshots and exclusive quotas. After > creating a snapshot of a subvolume and setting an exclusive quota of > 50MB for the snapshot, everything seems to work fine. I can write > approximately 50MB before the quota kicks in. > > However, if I create a snapshot, set an exclusive quota and just wait > for some time, I suddenly cannot even create an empty file because I'm > getting a "quota exceeded" error. The time until the bug appears seems > to vary. During the waiting time, I'm changing neither the snapshot nor > the original subvolume. "qgroup show -e" reports an exclusive use of > only a few kilobytes for the snapshot, which is nowhere near the limit. > > Steps to reproduce (/media/extern is a fresh and empty btrfs partition): > > Enable quota and create an empty subvolume: > root@t420:/media/extern# btrfs quota enable . > root@t420:/media/extern# btrfs subvolume create sub > Create subvolume './sub' > > Snapshot the subvolume and set a limit: > root@t420:/media/extern# btrfs subvolume snapshot sub snap > Create a snapshot of 'sub' in './snap' > root@t420:/media/extern# cd snap/ > root@t420:/media/extern/snap# btrfs qgroup limit -e 50M . > > Sometimes it takes "longer" for the quota to kick in, so I'm touching a > file every 5 minutes here: > > root@t420:/media/extern/snap# for file in {1..100}; do touch $file; > sleep 5m; done > touch: cannot touch ‘7’: Disk quota exceeded > ^C > root@t420:/media/extern/snap# btrfs qgroup show -e . > qgroupid rfer excl max_excl > -------- ---- ---- -------- > 0/5 16.00KiB 16.00KiB none > 0/257 16.00KiB 16.00KiB none > 0/258 16.00KiB 16.00KiB 50.00MiB > > Any idea why this happens? BTW, to make btrfs qgroup show work, it's better to call sync before qgroup show. It's a known bug that even after qgroup accounting rework, qgroup reserve still has bug and can cause reserved space to underflow, making such problem happen. For such case, btrfs qgroup show won't help as reserved space is not shown in the output. One workaround would be, umount the filesystem and mount again. Which will reset the underflow reserved space and work for sometime. If it's OK for you to recompile the kernel, you can try the following patchset: [PATCH v3 00/21] Rework btrfs qgroup reserved space framework Which should solve the problem. Thanks, Qu > > Thanks, > Johannes > > System info: > > Linux t420 4.3.0-rc5 #1 SMP Tue Oct 13 13:21:02 CEST 2015 x86_64 > GNU/Linux > > Label: none uuid: 9551e3ca-1608-469c-9d8c-77b99ce0e8ec > Total devices 1 FS bytes used 816.00KiB > devid 1 size 931.51GiB used 2.04GiB path /dev/sdb1 > > btrfs-progs v4.1.2 > > Data, single: total=8.00MiB, used=256.00KiB > System, DUP: total=8.00MiB, used=16.00KiB > System, single: total=4.00MiB, used=0.00B > Metadata, DUP: total=1.00GiB, used=544.00KiB > Metadata, single: total=8.00MiB, used=0.00B > GlobalReserve, single: total=16.00MiB, used=0.00B > > [249174.151820] sdb: sdb1 > [249184.387377] sdb: sdb1 > [249184.573096] sdb: sdb1 > [249184.656274] BTRFS: device fsid > 9551e3ca-1608-469c-9d8c-77b99ce0e8ec devid 1 transid 3 /dev/sdb1 > [249186.323915] sdb: sdb1 > [249186.534505] sdb: sdb1 > [249186.538420] sdb: sdb1 > [249196.781978] BTRFS info (device sdb1): disk space caching is enabled > [249196.781986] BTRFS: has skinny extents > [249196.781990] BTRFS: flagging fs with big metadata feature > [249196.818164] BTRFS: creating UUID tree > [249202.311983] BTRFS info (device sdb1): qgroup scan completed > (inconsistency flag cleared) > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Exclusive quota of snapshot exceeded despite no space used 2015-10-22 23:47 ` Qu Wenruo @ 2015-10-23 15:05 ` Johannes Henninger 2015-10-25 0:44 ` Qu Wenruo 0 siblings, 1 reply; 11+ messages in thread From: Johannes Henninger @ 2015-10-23 15:05 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs On 23.10.2015 01:47, Qu Wenruo wrote: > 在 2015年10月23日 04:38, Johannes Henninger 写道: >> I'm having a weird problem with snapshots and exclusive quotas. After >> creating a snapshot of a subvolume and setting an exclusive quota of >> 50MB for the snapshot, everything seems to work fine. I can write >> approximately 50MB before the quota kicks in. >> >> However, if I create a snapshot, set an exclusive quota and just wait >> for some time, I suddenly cannot even create an empty file because I'm >> getting a "quota exceeded" error. The time until the bug appears seems >> to vary. During the waiting time, I'm changing neither the snapshot nor >> the original subvolume. "qgroup show -e" reports an exclusive use of >> only a few kilobytes for the snapshot, which is nowhere near the limit. >> >> Steps to reproduce (/media/extern is a fresh and empty btrfs partition): >> >> Enable quota and create an empty subvolume: >> root@t420:/media/extern# btrfs quota enable . >> root@t420:/media/extern# btrfs subvolume create sub >> Create subvolume './sub' >> >> Snapshot the subvolume and set a limit: >> root@t420:/media/extern# btrfs subvolume snapshot sub snap >> Create a snapshot of 'sub' in './snap' >> root@t420:/media/extern# cd snap/ >> root@t420:/media/extern/snap# btrfs qgroup limit -e 50M . >> >> Sometimes it takes "longer" for the quota to kick in, so I'm touching a >> file every 5 minutes here: >> >> root@t420:/media/extern/snap# for file in {1..100}; do touch $file; >> sleep 5m; done >> touch: cannot touch ‘7’: Disk quota exceeded >> ^C >> root@t420:/media/extern/snap# btrfs qgroup show -e . >> qgroupid rfer excl max_excl >> -------- ---- ---- -------- >> 0/5 16.00KiB 16.00KiB none >> 0/257 16.00KiB 16.00KiB none >> 0/258 16.00KiB 16.00KiB 50.00MiB >> >> Any idea why this happens? > BTW, to make btrfs qgroup show work, it's better to call sync before > qgroup show. > > It's a known bug that even after qgroup accounting rework, qgroup > reserve still has bug and can cause reserved space to underflow, > making such problem happen. > > For such case, btrfs qgroup show won't help as reserved space is not > shown in the output. > > One workaround would be, umount the filesystem and mount again. > Which will reset the underflow reserved space and work for sometime. > > If it's OK for you to recompile the kernel, you can try the following > patchset: > [PATCH v3 00/21] Rework btrfs qgroup reserved space framework > > Which should solve the problem. > > Thanks, > Qu > Thanks a lot for your reply! While remounting the filesystem fixes the issue temporary, it doesn't take very long for the bug to happen again so it's not really a workaround I can work with. I did recompile the kernel using your patches, but unfortunately the problem still appears. Thanks, Johannes ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Exclusive quota of snapshot exceeded despite no space used 2015-10-23 15:05 ` Johannes Henninger @ 2015-10-25 0:44 ` Qu Wenruo 2015-10-25 11:10 ` Johannes Henninger 0 siblings, 1 reply; 11+ messages in thread From: Qu Wenruo @ 2015-10-25 0:44 UTC (permalink / raw) To: Johannes Henninger, linux-btrfs 在 2015年10月23日 23:05, Johannes Henninger 写道: > On 23.10.2015 01:47, Qu Wenruo wrote: >> 在 2015年10月23日 04:38, Johannes Henninger 写道: >>> I'm having a weird problem with snapshots and exclusive quotas. After >>> creating a snapshot of a subvolume and setting an exclusive quota of >>> 50MB for the snapshot, everything seems to work fine. I can write >>> approximately 50MB before the quota kicks in. >>> >>> However, if I create a snapshot, set an exclusive quota and just wait >>> for some time, I suddenly cannot even create an empty file because I'm >>> getting a "quota exceeded" error. The time until the bug appears seems >>> to vary. During the waiting time, I'm changing neither the snapshot nor >>> the original subvolume. "qgroup show -e" reports an exclusive use of >>> only a few kilobytes for the snapshot, which is nowhere near the limit. >>> >>> Steps to reproduce (/media/extern is a fresh and empty btrfs partition): >>> >>> Enable quota and create an empty subvolume: >>> root@t420:/media/extern# btrfs quota enable . >>> root@t420:/media/extern# btrfs subvolume create sub >>> Create subvolume './sub' >>> >>> Snapshot the subvolume and set a limit: >>> root@t420:/media/extern# btrfs subvolume snapshot sub snap >>> Create a snapshot of 'sub' in './snap' >>> root@t420:/media/extern# cd snap/ >>> root@t420:/media/extern/snap# btrfs qgroup limit -e 50M . >>> >>> Sometimes it takes "longer" for the quota to kick in, so I'm touching a >>> file every 5 minutes here: >>> >>> root@t420:/media/extern/snap# for file in {1..100}; do touch $file; >>> sleep 5m; done >>> touch: cannot touch ‘7’: Disk quota exceeded >>> ^C >>> root@t420:/media/extern/snap# btrfs qgroup show -e . >>> qgroupid rfer excl max_excl >>> -------- ---- ---- -------- >>> 0/5 16.00KiB 16.00KiB none >>> 0/257 16.00KiB 16.00KiB none >>> 0/258 16.00KiB 16.00KiB 50.00MiB >>> >>> Any idea why this happens? >> BTW, to make btrfs qgroup show work, it's better to call sync before >> qgroup show. >> >> It's a known bug that even after qgroup accounting rework, qgroup >> reserve still has bug and can cause reserved space to underflow, >> making such problem happen. >> >> For such case, btrfs qgroup show won't help as reserved space is not >> shown in the output. >> >> One workaround would be, umount the filesystem and mount again. >> Which will reset the underflow reserved space and work for sometime. >> >> If it's OK for you to recompile the kernel, you can try the following >> patchset: >> [PATCH v3 00/21] Rework btrfs qgroup reserved space framework >> >> Which should solve the problem. >> >> Thanks, >> Qu >> > > Thanks a lot for your reply! > > While remounting the filesystem fixes the issue temporary, it doesn't > take very long for the bug to happen again so it's not really a > workaround I can work with. > > I did recompile the kernel using your patches, but unfortunately the > problem still appears. > > Thanks, > Johannes > Interesting, just touching file will cause EQUOTA is quite a big problem. I'll try to reproduce it with my patchset and see what really caused the problem. The problem seems to do with snapshot qgroup hacking. But I'm not completely sure yet. BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? 16K is the correct number with only 6 empty files, just in case. Thanks, Qu ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Exclusive quota of snapshot exceeded despite no space used 2015-10-25 0:44 ` Qu Wenruo @ 2015-10-25 11:10 ` Johannes Henninger 2015-10-25 12:17 ` Qu Wenruo 2015-10-26 7:12 ` Qu Wenruo 0 siblings, 2 replies; 11+ messages in thread From: Johannes Henninger @ 2015-10-25 11:10 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs On 25.10.2015 02:44, Qu Wenruo wrote: > > > 在 2015年10月23日 23:05, Johannes Henninger 写道: >> On 23.10.2015 01:47, Qu Wenruo wrote: >>> 在 2015年10月23日 04:38, Johannes Henninger 写道: >>>> I'm having a weird problem with snapshots and exclusive quotas. After >>>> creating a snapshot of a subvolume and setting an exclusive quota of >>>> 50MB for the snapshot, everything seems to work fine. I can write >>>> approximately 50MB before the quota kicks in. >>>> >>>> However, if I create a snapshot, set an exclusive quota and just wait >>>> for some time, I suddenly cannot even create an empty file because I'm >>>> getting a "quota exceeded" error. The time until the bug appears seems >>>> to vary. During the waiting time, I'm changing neither the snapshot >>>> nor >>>> the original subvolume. "qgroup show -e" reports an exclusive use of >>>> only a few kilobytes for the snapshot, which is nowhere near the >>>> limit. >>>> >>>> Steps to reproduce (/media/extern is a fresh and empty btrfs >>>> partition): >>>> >>>> Enable quota and create an empty subvolume: >>>> root@t420:/media/extern# btrfs quota enable . >>>> root@t420:/media/extern# btrfs subvolume create sub >>>> Create subvolume './sub' >>>> >>>> Snapshot the subvolume and set a limit: >>>> root@t420:/media/extern# btrfs subvolume snapshot sub snap >>>> Create a snapshot of 'sub' in './snap' >>>> root@t420:/media/extern# cd snap/ >>>> root@t420:/media/extern/snap# btrfs qgroup limit -e 50M . >>>> >>>> Sometimes it takes "longer" for the quota to kick in, so I'm >>>> touching a >>>> file every 5 minutes here: >>>> >>>> root@t420:/media/extern/snap# for file in {1..100}; do touch >>>> $file; >>>> sleep 5m; done >>>> touch: cannot touch ‘7’: Disk quota exceeded >>>> ^C >>>> root@t420:/media/extern/snap# btrfs qgroup show -e . >>>> qgroupid rfer excl max_excl >>>> -------- ---- ---- -------- >>>> 0/5 16.00KiB 16.00KiB none >>>> 0/257 16.00KiB 16.00KiB none >>>> 0/258 16.00KiB 16.00KiB 50.00MiB >>>> >>>> Any idea why this happens? >>> BTW, to make btrfs qgroup show work, it's better to call sync before >>> qgroup show. >>> >>> It's a known bug that even after qgroup accounting rework, qgroup >>> reserve still has bug and can cause reserved space to underflow, >>> making such problem happen. >>> >>> For such case, btrfs qgroup show won't help as reserved space is not >>> shown in the output. >>> >>> One workaround would be, umount the filesystem and mount again. >>> Which will reset the underflow reserved space and work for sometime. >>> >>> If it's OK for you to recompile the kernel, you can try the following >>> patchset: >>> [PATCH v3 00/21] Rework btrfs qgroup reserved space framework >>> >>> Which should solve the problem. >>> >>> Thanks, >>> Qu >>> >> >> Thanks a lot for your reply! >> >> While remounting the filesystem fixes the issue temporary, it doesn't >> take very long for the bug to happen again so it's not really a >> workaround I can work with. >> >> I did recompile the kernel using your patches, but unfortunately the >> problem still appears. >> >> Thanks, >> Johannes >> > Interesting, just touching file will cause EQUOTA is quite a big problem. > > I'll try to reproduce it with my patchset and see what really caused > the problem. > The problem seems to do with snapshot qgroup hacking. > But I'm not completely sure yet. > > BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? > 16K is the correct number with only 6 empty files, just in case. > > Thanks, > Qu I ran my example from the first mail again and managed to write 7 files this time, "qgroup show" still shows 16kB after sync: root@t420:/media/extern/snap# btrfs qg limit -e 50M . root@t420:/media/extern/snap# for file in {1..100}; do touch $file; sleep 5m; done touch: cannot touch ‘8’: Disk quota exceeded ^C root@t420:/media/extern/snap# sync root@t420:/media/extern/snap# btrfs qgroup show -pcre . qgroupid rfer excl max_rfer max_excl parent child -------- ---- ---- -------- -------- ------ ----- 0/5 16.00KiB 16.00KiB none none --- --- 0/257 16.00KiB 16.00KiB none none --- --- 0/258 16.00KiB 16.00KiB none 50.00MiB --- --- root@t420:/media/extern/snap# btrfs fi sync . FSSync '.' root@t420:/media/extern/snap# btrfs qgroup show -pcre . qgroupid rfer excl max_rfer max_excl parent child -------- ---- ---- -------- -------- ------ ----- 0/5 16.00KiB 16.00KiB none none --- --- 0/257 16.00KiB 16.00KiB none none --- --- 0/258 16.00KiB 16.00KiB none 50.00MiB --- --- By the way, I don't if its relevant but the problem is not limited to exclusive quotas, but also happens when setting a "referenced" limit (qgroup limit without "-e"). Thanks, Johannes ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Exclusive quota of snapshot exceeded despite no space used 2015-10-25 11:10 ` Johannes Henninger @ 2015-10-25 12:17 ` Qu Wenruo 2015-10-26 7:12 ` Qu Wenruo 1 sibling, 0 replies; 11+ messages in thread From: Qu Wenruo @ 2015-10-25 12:17 UTC (permalink / raw) To: Johannes Henninger, linux-btrfs 在 2015年10月25日 19:10, Johannes Henninger 写道: > On 25.10.2015 02:44, Qu Wenruo wrote: >> >> >> 在 2015年10月23日 23:05, Johannes Henninger 写道: >>> On 23.10.2015 01:47, Qu Wenruo wrote: >>>> 在 2015年10月23日 04:38, Johannes Henninger 写道: >>>>> I'm having a weird problem with snapshots and exclusive quotas. After >>>>> creating a snapshot of a subvolume and setting an exclusive quota of >>>>> 50MB for the snapshot, everything seems to work fine. I can write >>>>> approximately 50MB before the quota kicks in. >>>>> >>>>> However, if I create a snapshot, set an exclusive quota and just wait >>>>> for some time, I suddenly cannot even create an empty file because I'm >>>>> getting a "quota exceeded" error. The time until the bug appears seems >>>>> to vary. During the waiting time, I'm changing neither the snapshot >>>>> nor >>>>> the original subvolume. "qgroup show -e" reports an exclusive use of >>>>> only a few kilobytes for the snapshot, which is nowhere near the >>>>> limit. >>>>> >>>>> Steps to reproduce (/media/extern is a fresh and empty btrfs >>>>> partition): >>>>> >>>>> Enable quota and create an empty subvolume: >>>>> root@t420:/media/extern# btrfs quota enable . >>>>> root@t420:/media/extern# btrfs subvolume create sub >>>>> Create subvolume './sub' >>>>> >>>>> Snapshot the subvolume and set a limit: >>>>> root@t420:/media/extern# btrfs subvolume snapshot sub snap >>>>> Create a snapshot of 'sub' in './snap' >>>>> root@t420:/media/extern# cd snap/ >>>>> root@t420:/media/extern/snap# btrfs qgroup limit -e 50M . >>>>> >>>>> Sometimes it takes "longer" for the quota to kick in, so I'm >>>>> touching a >>>>> file every 5 minutes here: >>>>> >>>>> root@t420:/media/extern/snap# for file in {1..100}; do touch >>>>> $file; >>>>> sleep 5m; done >>>>> touch: cannot touch ‘7’: Disk quota exceeded >>>>> ^C >>>>> root@t420:/media/extern/snap# btrfs qgroup show -e . >>>>> qgroupid rfer excl max_excl >>>>> -------- ---- ---- -------- >>>>> 0/5 16.00KiB 16.00KiB none >>>>> 0/257 16.00KiB 16.00KiB none >>>>> 0/258 16.00KiB 16.00KiB 50.00MiB >>>>> >>>>> Any idea why this happens? >>>> BTW, to make btrfs qgroup show work, it's better to call sync before >>>> qgroup show. >>>> >>>> It's a known bug that even after qgroup accounting rework, qgroup >>>> reserve still has bug and can cause reserved space to underflow, >>>> making such problem happen. >>>> >>>> For such case, btrfs qgroup show won't help as reserved space is not >>>> shown in the output. >>>> >>>> One workaround would be, umount the filesystem and mount again. >>>> Which will reset the underflow reserved space and work for sometime. >>>> >>>> If it's OK for you to recompile the kernel, you can try the following >>>> patchset: >>>> [PATCH v3 00/21] Rework btrfs qgroup reserved space framework >>>> >>>> Which should solve the problem. >>>> >>>> Thanks, >>>> Qu >>>> >>> >>> Thanks a lot for your reply! >>> >>> While remounting the filesystem fixes the issue temporary, it doesn't >>> take very long for the bug to happen again so it's not really a >>> workaround I can work with. >>> >>> I did recompile the kernel using your patches, but unfortunately the >>> problem still appears. >>> >>> Thanks, >>> Johannes >>> >> Interesting, just touching file will cause EQUOTA is quite a big problem. >> >> I'll try to reproduce it with my patchset and see what really caused >> the problem. >> The problem seems to do with snapshot qgroup hacking. >> But I'm not completely sure yet. >> >> BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? >> 16K is the correct number with only 6 empty files, just in case. >> >> Thanks, >> Qu > > I ran my example from the first mail again and managed to write 7 files > this time, "qgroup show" still shows 16kB after sync: > > root@t420:/media/extern/snap# btrfs qg limit -e 50M . > root@t420:/media/extern/snap# for file in {1..100}; do touch $file; > sleep 5m; done > touch: cannot touch ‘8’: Disk quota exceeded > ^C > root@t420:/media/extern/snap# sync > root@t420:/media/extern/snap# btrfs qgroup show -pcre . > qgroupid rfer excl max_rfer max_excl parent child > -------- ---- ---- -------- -------- ------ ----- > 0/5 16.00KiB 16.00KiB none none --- --- > 0/257 16.00KiB 16.00KiB none none --- --- > 0/258 16.00KiB 16.00KiB none 50.00MiB --- --- > root@t420:/media/extern/snap# btrfs fi sync . > FSSync '.' > root@t420:/media/extern/snap# btrfs qgroup show -pcre . > qgroupid rfer excl max_rfer max_excl parent child > -------- ---- ---- -------- -------- ------ ----- > 0/5 16.00KiB 16.00KiB none none --- --- > 0/257 16.00KiB 16.00KiB none none --- --- > 0/258 16.00KiB 16.00KiB none 50.00MiB --- --- > Thanks for the report. At least, the rfer/excl number is correct. So the problem is limited to qgroup reserved space. I'll debug it tomorrow to see what's going wrong inside the reserved space. > By the way, I don't if its relevant but the problem is not limited to > exclusive quotas, but also happens when setting a "referenced" limit > (qgroup limit without "-e"). This also indicates that the problem is in reserved space, not rfer/excl accounting, as reserved space is used for both rfer or excl, so if it's going wrong, no matter to which the limit is set, it will cause EDQUOT. Thanks, Qu > > Thanks, > Johannes > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Exclusive quota of snapshot exceeded despite no space used 2015-10-25 11:10 ` Johannes Henninger 2015-10-25 12:17 ` Qu Wenruo @ 2015-10-26 7:12 ` Qu Wenruo 2015-10-27 0:15 ` Johannes Henninger 1 sibling, 1 reply; 11+ messages in thread From: Qu Wenruo @ 2015-10-26 7:12 UTC (permalink / raw) To: Johannes Henninger, Qu Wenruo, linux-btrfs >>> Thanks a lot for your reply! >>> >>> While remounting the filesystem fixes the issue temporary, it doesn't >>> take very long for the bug to happen again so it's not really a >>> workaround I can work with. >>> >>> I did recompile the kernel using your patches, but unfortunately the >>> problem still appears. >>> >>> Thanks, >>> Johannes >>> >> Interesting, just touching file will cause EQUOTA is quite a big problem. >> >> I'll try to reproduce it with my patchset and see what really caused >> the problem. >> The problem seems to do with snapshot qgroup hacking. >> But I'm not completely sure yet. >> >> BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? >> 16K is the correct number with only 6 empty files, just in case. >> >> Thanks, >> Qu > > I ran my example from the first mail again and managed to write 7 files > this time, "qgroup show" still shows 16kB after sync: > > root@t420:/media/extern/snap# btrfs qg limit -e 50M . > root@t420:/media/extern/snap# for file in {1..100}; do touch $file; > sleep 5m; done > touch: cannot touch ‘8’: Disk quota exceeded > ^C > root@t420:/media/extern/snap# sync > root@t420:/media/extern/snap# btrfs qgroup show -pcre . > qgroupid rfer excl max_rfer max_excl parent child > -------- ---- ---- -------- -------- ------ ----- > 0/5 16.00KiB 16.00KiB none none --- --- > 0/257 16.00KiB 16.00KiB none none --- --- > 0/258 16.00KiB 16.00KiB none 50.00MiB --- --- > root@t420:/media/extern/snap# btrfs fi sync . > FSSync '.' > root@t420:/media/extern/snap# btrfs qgroup show -pcre . > qgroupid rfer excl max_rfer max_excl parent child > -------- ---- ---- -------- -------- ------ ----- > 0/5 16.00KiB 16.00KiB none none --- --- > 0/257 16.00KiB 16.00KiB none none --- --- > 0/258 16.00KiB 16.00KiB none 50.00MiB --- --- > > By the way, I don't if its relevant but the problem is not limited to > exclusive quotas, but also happens when setting a "referenced" limit > (qgroup limit without "-e"). > > Thanks, > Johannes > The bug is located, and turns out to be quite a stupid problem caused by myself. I just forgot to include a cleanup patch during rebase!!!! AGAIN!!! You can apply the following patch to resolve it: [PATCH 3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup double free Or just apply the whole patchset: [4.4][PATCH 0/3] btrfs: Qgroup hotfix At least, with the patchset based on Chris' integration-4.4 branch, it succeeded in touching all the 100 files in my test box. Thanks, Qu > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Exclusive quota of snapshot exceeded despite no space used 2015-10-26 7:12 ` Qu Wenruo @ 2015-10-27 0:15 ` Johannes Henninger 2015-10-27 1:06 ` Qu Wenruo 0 siblings, 1 reply; 11+ messages in thread From: Johannes Henninger @ 2015-10-27 0:15 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs On 26.10.2015 08:12, Qu Wenruo wrote: > >>>> Thanks a lot for your reply! >>>> >>>> While remounting the filesystem fixes the issue temporary, it doesn't >>>> take very long for the bug to happen again so it's not really a >>>> workaround I can work with. >>>> >>>> I did recompile the kernel using your patches, but unfortunately the >>>> problem still appears. >>>> >>>> Thanks, >>>> Johannes >>>> >>> Interesting, just touching file will cause EQUOTA is quite a big >>> problem. >>> >>> I'll try to reproduce it with my patchset and see what really caused >>> the problem. >>> The problem seems to do with snapshot qgroup hacking. >>> But I'm not completely sure yet. >>> >>> BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? >>> 16K is the correct number with only 6 empty files, just in case. >>> >>> Thanks, >>> Qu >> >> I ran my example from the first mail again and managed to write 7 files >> this time, "qgroup show" still shows 16kB after sync: >> >> root@t420:/media/extern/snap# btrfs qg limit -e 50M . >> root@t420:/media/extern/snap# for file in {1..100}; do touch $file; >> sleep 5m; done >> touch: cannot touch ‘8’: Disk quota exceeded >> ^C >> root@t420:/media/extern/snap# sync >> root@t420:/media/extern/snap# btrfs qgroup show -pcre . >> qgroupid rfer excl max_rfer max_excl parent >> child >> -------- ---- ---- -------- -------- ------ >> ----- >> 0/5 16.00KiB 16.00KiB none none --- --- >> 0/257 16.00KiB 16.00KiB none none --- --- >> 0/258 16.00KiB 16.00KiB none 50.00MiB --- --- >> root@t420:/media/extern/snap# btrfs fi sync . >> FSSync '.' >> root@t420:/media/extern/snap# btrfs qgroup show -pcre . >> qgroupid rfer excl max_rfer max_excl parent >> child >> -------- ---- ---- -------- -------- ------ >> ----- >> 0/5 16.00KiB 16.00KiB none none --- --- >> 0/257 16.00KiB 16.00KiB none none --- --- >> 0/258 16.00KiB 16.00KiB none 50.00MiB --- --- >> >> By the way, I don't if its relevant but the problem is not limited to >> exclusive quotas, but also happens when setting a "referenced" limit >> (qgroup limit without "-e"). >> >> Thanks, >> Johannes >> > > The bug is located, and turns out to be quite a stupid problem caused > by myself. > > I just forgot to include a cleanup patch during rebase!!!! AGAIN!!! > > You can apply the following patch to resolve it: > [PATCH 3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup > double free > > Or just apply the whole patchset: > [4.4][PATCH 0/3] btrfs: Qgroup hotfix > > At least, with the patchset based on Chris' integration-4.4 branch, it > succeeded in touching all the 100 files in my test box. > > Thanks, > Qu > It's working! Thank you so much for fixing this bug, you don't even know how much this has helped me! Thanks! Johannes ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Exclusive quota of snapshot exceeded despite no space used 2015-10-27 0:15 ` Johannes Henninger @ 2015-10-27 1:06 ` Qu Wenruo 2015-10-28 14:02 ` Johannes Henninger 0 siblings, 1 reply; 11+ messages in thread From: Qu Wenruo @ 2015-10-27 1:06 UTC (permalink / raw) To: Johannes Henninger, Qu Wenruo, linux-btrfs Johannes Henninger wrote on 2015/10/27 01:15 +0100: > On 26.10.2015 08:12, Qu Wenruo wrote: >> >>>>> Thanks a lot for your reply! >>>>> >>>>> While remounting the filesystem fixes the issue temporary, it doesn't >>>>> take very long for the bug to happen again so it's not really a >>>>> workaround I can work with. >>>>> >>>>> I did recompile the kernel using your patches, but unfortunately the >>>>> problem still appears. >>>>> >>>>> Thanks, >>>>> Johannes >>>>> >>>> Interesting, just touching file will cause EQUOTA is quite a big >>>> problem. >>>> >>>> I'll try to reproduce it with my patchset and see what really caused >>>> the problem. >>>> The problem seems to do with snapshot qgroup hacking. >>>> But I'm not completely sure yet. >>>> >>>> BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? >>>> 16K is the correct number with only 6 empty files, just in case. >>>> >>>> Thanks, >>>> Qu >>> >>> I ran my example from the first mail again and managed to write 7 files >>> this time, "qgroup show" still shows 16kB after sync: >>> >>> root@t420:/media/extern/snap# btrfs qg limit -e 50M . >>> root@t420:/media/extern/snap# for file in {1..100}; do touch $file; >>> sleep 5m; done >>> touch: cannot touch ‘8’: Disk quota exceeded >>> ^C >>> root@t420:/media/extern/snap# sync >>> root@t420:/media/extern/snap# btrfs qgroup show -pcre . >>> qgroupid rfer excl max_rfer max_excl parent >>> child >>> -------- ---- ---- -------- -------- ------ >>> ----- >>> 0/5 16.00KiB 16.00KiB none none --- --- >>> 0/257 16.00KiB 16.00KiB none none --- --- >>> 0/258 16.00KiB 16.00KiB none 50.00MiB --- --- >>> root@t420:/media/extern/snap# btrfs fi sync . >>> FSSync '.' >>> root@t420:/media/extern/snap# btrfs qgroup show -pcre . >>> qgroupid rfer excl max_rfer max_excl parent >>> child >>> -------- ---- ---- -------- -------- ------ >>> ----- >>> 0/5 16.00KiB 16.00KiB none none --- --- >>> 0/257 16.00KiB 16.00KiB none none --- --- >>> 0/258 16.00KiB 16.00KiB none 50.00MiB --- --- >>> >>> By the way, I don't if its relevant but the problem is not limited to >>> exclusive quotas, but also happens when setting a "referenced" limit >>> (qgroup limit without "-e"). >>> >>> Thanks, >>> Johannes >>> >> >> The bug is located, and turns out to be quite a stupid problem caused >> by myself. >> >> I just forgot to include a cleanup patch during rebase!!!! AGAIN!!! >> >> You can apply the following patch to resolve it: >> [PATCH 3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup >> double free >> >> Or just apply the whole patchset: >> [4.4][PATCH 0/3] btrfs: Qgroup hotfix >> >> At least, with the patchset based on Chris' integration-4.4 branch, it >> succeeded in touching all the 100 files in my test box. >> >> Thanks, >> Qu >> > > It's working! Thank you so much for fixing this bug, you don't even know > how much this has helped me! > > Thanks! > Johannes > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Glad to hear that. If it's working for you, it would be better to add a 'Tested-by' tag for the 3rd patch. Thanks, Qu ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Exclusive quota of snapshot exceeded despite no space used 2015-10-27 1:06 ` Qu Wenruo @ 2015-10-28 14:02 ` Johannes Henninger 2015-10-29 2:31 ` Qu Wenruo 0 siblings, 1 reply; 11+ messages in thread From: Johannes Henninger @ 2015-10-28 14:02 UTC (permalink / raw) To: Qu Wenruo, Qu Wenruo, linux-btrfs On 27.10.2015 02:06, Qu Wenruo wrote: > > > Johannes Henninger wrote on 2015/10/27 01:15 +0100: >> On 26.10.2015 08:12, Qu Wenruo wrote: >>> >>>>>> Thanks a lot for your reply! >>>>>> >>>>>> While remounting the filesystem fixes the issue temporary, it >>>>>> doesn't >>>>>> take very long for the bug to happen again so it's not really a >>>>>> workaround I can work with. >>>>>> >>>>>> I did recompile the kernel using your patches, but unfortunately the >>>>>> problem still appears. >>>>>> >>>>>> Thanks, >>>>>> Johannes >>>>>> >>>>> Interesting, just touching file will cause EQUOTA is quite a big >>>>> problem. >>>>> >>>>> I'll try to reproduce it with my patchset and see what really caused >>>>> the problem. >>>>> The problem seems to do with snapshot qgroup hacking. >>>>> But I'm not completely sure yet. >>>>> >>>>> BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? >>>>> 16K is the correct number with only 6 empty files, just in case. >>>>> >>>>> Thanks, >>>>> Qu >>>> >>>> I ran my example from the first mail again and managed to write 7 >>>> files >>>> this time, "qgroup show" still shows 16kB after sync: >>>> >>>> root@t420:/media/extern/snap# btrfs qg limit -e 50M . >>>> root@t420:/media/extern/snap# for file in {1..100}; do touch $file; >>>> sleep 5m; done >>>> touch: cannot touch ‘8’: Disk quota exceeded >>>> ^C >>>> root@t420:/media/extern/snap# sync >>>> root@t420:/media/extern/snap# btrfs qgroup show -pcre . >>>> qgroupid rfer excl max_rfer max_excl parent >>>> child >>>> -------- ---- ---- -------- -------- ------ >>>> ----- >>>> 0/5 16.00KiB 16.00KiB none none >>>> --- --- >>>> 0/257 16.00KiB 16.00KiB none none >>>> --- --- >>>> 0/258 16.00KiB 16.00KiB none 50.00MiB >>>> --- --- >>>> root@t420:/media/extern/snap# btrfs fi sync . >>>> FSSync '.' >>>> root@t420:/media/extern/snap# btrfs qgroup show -pcre . >>>> qgroupid rfer excl max_rfer max_excl parent >>>> child >>>> -------- ---- ---- -------- -------- ------ >>>> ----- >>>> 0/5 16.00KiB 16.00KiB none none >>>> --- --- >>>> 0/257 16.00KiB 16.00KiB none none >>>> --- --- >>>> 0/258 16.00KiB 16.00KiB none 50.00MiB >>>> --- --- >>>> >>>> By the way, I don't if its relevant but the problem is not limited to >>>> exclusive quotas, but also happens when setting a "referenced" limit >>>> (qgroup limit without "-e"). >>>> >>>> Thanks, >>>> Johannes >>>> >>> >>> The bug is located, and turns out to be quite a stupid problem caused >>> by myself. >>> >>> I just forgot to include a cleanup patch during rebase!!!! AGAIN!!! >>> >>> You can apply the following patch to resolve it: >>> [PATCH 3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup >>> double free >>> >>> Or just apply the whole patchset: >>> [4.4][PATCH 0/3] btrfs: Qgroup hotfix >>> >>> At least, with the patchset based on Chris' integration-4.4 branch, it >>> succeeded in touching all the 100 files in my test box. >>> >>> Thanks, >>> Qu >>> >> >> It's working! Thank you so much for fixing this bug, you don't even know >> how much this has helped me! >> >> Thanks! >> Johannes >> -- >> To unsubscribe from this list: send the line "unsubscribe >> linux-btrfs" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > Glad to hear that. > > If it's working for you, it would be better to add a 'Tested-by' tag > for the 3rd patch. > > Thanks, > Qu Sure! Is there anything I have to do? I'm a kernel and mailing list noob :) Thanks, Johannes ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Exclusive quota of snapshot exceeded despite no space used 2015-10-28 14:02 ` Johannes Henninger @ 2015-10-29 2:31 ` Qu Wenruo 0 siblings, 0 replies; 11+ messages in thread From: Qu Wenruo @ 2015-10-29 2:31 UTC (permalink / raw) To: Johannes Henninger, Qu Wenruo, linux-btrfs Johannes Henninger wrote on 2015/10/28 15:02 +0100: > On 27.10.2015 02:06, Qu Wenruo wrote: >> >> >> Johannes Henninger wrote on 2015/10/27 01:15 +0100: >>> On 26.10.2015 08:12, Qu Wenruo wrote: >>>> >>>>>>> Thanks a lot for your reply! >>>>>>> >>>>>>> While remounting the filesystem fixes the issue temporary, it >>>>>>> doesn't >>>>>>> take very long for the bug to happen again so it's not really a >>>>>>> workaround I can work with. >>>>>>> >>>>>>> I did recompile the kernel using your patches, but unfortunately the >>>>>>> problem still appears. >>>>>>> >>>>>>> Thanks, >>>>>>> Johannes >>>>>>> >>>>>> Interesting, just touching file will cause EQUOTA is quite a big >>>>>> problem. >>>>>> >>>>>> I'll try to reproduce it with my patchset and see what really caused >>>>>> the problem. >>>>>> The problem seems to do with snapshot qgroup hacking. >>>>>> But I'm not completely sure yet. >>>>>> >>>>>> BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? >>>>>> 16K is the correct number with only 6 empty files, just in case. >>>>>> >>>>>> Thanks, >>>>>> Qu >>>>> >>>>> I ran my example from the first mail again and managed to write 7 >>>>> files >>>>> this time, "qgroup show" still shows 16kB after sync: >>>>> >>>>> root@t420:/media/extern/snap# btrfs qg limit -e 50M . >>>>> root@t420:/media/extern/snap# for file in {1..100}; do touch $file; >>>>> sleep 5m; done >>>>> touch: cannot touch ‘8’: Disk quota exceeded >>>>> ^C >>>>> root@t420:/media/extern/snap# sync >>>>> root@t420:/media/extern/snap# btrfs qgroup show -pcre . >>>>> qgroupid rfer excl max_rfer max_excl parent >>>>> child >>>>> -------- ---- ---- -------- -------- ------ >>>>> ----- >>>>> 0/5 16.00KiB 16.00KiB none none >>>>> --- --- >>>>> 0/257 16.00KiB 16.00KiB none none >>>>> --- --- >>>>> 0/258 16.00KiB 16.00KiB none 50.00MiB >>>>> --- --- >>>>> root@t420:/media/extern/snap# btrfs fi sync . >>>>> FSSync '.' >>>>> root@t420:/media/extern/snap# btrfs qgroup show -pcre . >>>>> qgroupid rfer excl max_rfer max_excl parent >>>>> child >>>>> -------- ---- ---- -------- -------- ------ >>>>> ----- >>>>> 0/5 16.00KiB 16.00KiB none none >>>>> --- --- >>>>> 0/257 16.00KiB 16.00KiB none none >>>>> --- --- >>>>> 0/258 16.00KiB 16.00KiB none 50.00MiB >>>>> --- --- >>>>> >>>>> By the way, I don't if its relevant but the problem is not limited to >>>>> exclusive quotas, but also happens when setting a "referenced" limit >>>>> (qgroup limit without "-e"). >>>>> >>>>> Thanks, >>>>> Johannes >>>>> >>>> >>>> The bug is located, and turns out to be quite a stupid problem caused >>>> by myself. >>>> >>>> I just forgot to include a cleanup patch during rebase!!!! AGAIN!!! >>>> >>>> You can apply the following patch to resolve it: >>>> [PATCH 3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup >>>> double free >>>> >>>> Or just apply the whole patchset: >>>> [4.4][PATCH 0/3] btrfs: Qgroup hotfix >>>> >>>> At least, with the patchset based on Chris' integration-4.4 branch, it >>>> succeeded in touching all the 100 files in my test box. >>>> >>>> Thanks, >>>> Qu >>>> >>> >>> It's working! Thank you so much for fixing this bug, you don't even know >>> how much this has helped me! >>> >>> Thanks! >>> Johannes >>> -- >>> To unsubscribe from this list: send the line "unsubscribe >>> linux-btrfs" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >> Glad to hear that. >> >> If it's working for you, it would be better to add a 'Tested-by' tag >> for the 3rd patch. >> >> Thanks, >> Qu > > Sure! Is there anything I have to do? I'm a kernel and mailing list noob :) > > Thanks, > Johannes > Find the mail "[PATCH 3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup double free", and reply to it with the following contents if you tested the patch: Tested-by: Johannes Hennigner <johannes+btrfs@henniger.io> Also refer to kernel documentation/SubmittingPatches. Thanks, Qu ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2015-10-29 2:31 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-10-22 20:38 Exclusive quota of snapshot exceeded despite no space used Johannes Henninger 2015-10-22 23:47 ` Qu Wenruo 2015-10-23 15:05 ` Johannes Henninger 2015-10-25 0:44 ` Qu Wenruo 2015-10-25 11:10 ` Johannes Henninger 2015-10-25 12:17 ` Qu Wenruo 2015-10-26 7:12 ` Qu Wenruo 2015-10-27 0:15 ` Johannes Henninger 2015-10-27 1:06 ` Qu Wenruo 2015-10-28 14:02 ` Johannes Henninger 2015-10-29 2:31 ` Qu Wenruo
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).