Quota Ignored On write

Quota Ignored On write

[Kevin Brandstatter]

basing of the latest for-linus branch i found i can write way more than
the quota

btrfs quota enable
btrfs subvolume create test
btrfs qgruop limit 1G test
dd if=/dev/zero of=test/file bs=1024 count=1500000
output:
1500000+0 records in
1500000+0 records out
1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s

thats a full half gig over the quota limit. I noticed some changes to
the quota
accounting in the logs, what changed that could cause this?

-Kevin Brandstatter

Re: Quota Ignored On write

[Satoru Takeuchi]

Hi Kevin,

(2014/07/04 11:13), Kevin Brandstatter wrote:
> basing of the latest for-linus branch i found i can write way more than
> the quota
>
> btrfs quota enable
> btrfs subvolume create test
> btrfs qgruop limit 1G test
> dd if=/dev/zero of=test/file bs=1024 count=1500000
> output:
> 1500000+0 records in
> 1500000+0 records out
> 1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s
>
> thats a full half gig over the quota limit. I noticed some changes to
> the quota
> accounting in the logs, what changed that could cause this?

Do you remember what kernel version quota worked correctly?

Thanks,
Satoru

>
> -Kevin Brandstatter
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Quota Ignored On write

[Kevin Brandstatter]

3.15.3 via arch/ and from linux-git

-Kevin

On 07/03/2014 09:21 PM, Satoru Takeuchi wrote:
> Hi Kevin,
>
> (2014/07/04 11:13), Kevin Brandstatter wrote:
>> basing of the latest for-linus branch i found i can write way more than
>> the quota
>>
>> btrfs quota enable
>> btrfs subvolume create test
>> btrfs qgruop limit 1G test
>> dd if=/dev/zero of=test/file bs=1024 count=1500000
>> output:
>> 1500000+0 records in
>> 1500000+0 records out
>> 1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s
>>
>> thats a full half gig over the quota limit. I noticed some changes to
>> the quota
>> accounting in the logs, what changed that could cause this?
>
> Do you remember what kernel version quota worked correctly?
>
> Thanks,
> Satoru
>
>>
>> -Kevin Brandstatter
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe
>> linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>

Re: Quota Ignored On write

[Satoru Takeuchi]

(2014/07/04 11:25), Kevin Brandstatter wrote:
> 3.15.3 via arch/ and from linux-git

OK, I'll bisect it.

Satoru

>
> -Kevin
>
> On 07/03/2014 09:21 PM, Satoru Takeuchi wrote:
>> Hi Kevin,
>>
>> (2014/07/04 11:13), Kevin Brandstatter wrote:
>>> basing of the latest for-linus branch i found i can write way more than
>>> the quota
>>>
>>> btrfs quota enable
>>> btrfs subvolume create test
>>> btrfs qgruop limit 1G test
>>> dd if=/dev/zero of=test/file bs=1024 count=1500000
>>> output:
>>> 1500000+0 records in
>>> 1500000+0 records out
>>> 1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s
>>>
>>> thats a full half gig over the quota limit. I noticed some changes to
>>> the quota
>>> accounting in the logs, what changed that could cause this?
>>
>> Do you remember what kernel version quota worked correctly?
>>
>> Thanks,
>> Satoru
>>
>>>
>>> -Kevin Brandstatter
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe
>>> linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Quota Ignored On write

[Satoru Takeuchi]

Hi Chris and Kevin,

>> On 07/03/2014 09:21 PM, Satoru Takeuchi wrote:
>>> Hi Kevin,
>>>
>>> (2014/07/04 11:13), Kevin Brandstatter wrote:
>>>> basing of the latest for-linus branch i found i can write way more than
>>>> the quota
>>>>
>>>> btrfs quota enable
>>>> btrfs subvolume create test
>>>> btrfs qgruop limit 1G test
>>>> dd if=/dev/zero of=test/file bs=1024 count=1500000
>>>> output:
>>>> 1500000+0 records in
>>>> 1500000+0 records out
>>>> 1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s
>>>>
>>>> thats a full half gig over the quota limit. I noticed some changes to
>>>> the quota
>>>> accounting in the logs, what changed that could cause this?
>>>
>>> Do you remember what kernel version quota worked correctly?
(2014/07/04 11:32), Satoru Takeuchi wrote:
> (2014/07/04 11:25), Kevin Brandstatter wrote:
>> 3.15.3 via arch/ and from linux-git
>
> OK, I'll bisect it.

I made the following reproducer based on your operation.
It succeeded with 3.15 and failed with 3.16-rc3. So, the problematic
patch is not in mason/for-linux branch, but in somewhere between
3.15 and 3.16-rc3. Please wait for a while to finish my bisect...

===============================================================================
#!/bin/bash -x

TEST_DEV=/dev/vdb
TEST_MNT=/home/sat/mnt

umount $TEST_MNT
mkfs.btrfs -f $TEST_DEV
mount $TEST_DEV $TEST_MNT
btrfs quota enable $TEST_MNT

SUBVOLPATH=$TEST_MNT/quota_test
LIMIT=$((1024*1000000))
btrfs subvolume create $SUBVOLPATH
btrfs qgroup limit $LIMIT $SUBVOLPATH
TESTFILE=$SUBVOLPATH/test
dd if=/dev/zero of=$TESTFILE bs=1024 count=$(($LIMIT*3/2/1024))
SIZE=$(($(ls -s $TESTFILE | awk '{print $1}')*1024))

RET=0
if [ $SIZE -le $LIMIT ] ; then
   echo "[PASS] quota works correctly" >&2
else
   echo "[FAIL] quota doesn't work" >&2
   RET=1
fi

exit $RET
===============================================================================

Thanks,
Satoru

>>>
>>> Thanks,
>>> Satoru
>>>
>>>>
>>>> -Kevin Brandstatter
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe
>>>> linux-btrfs" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Quota Ignored On write

[Satoru Takeuchi]

Hi Josef, Chris, Kevin,

(2014/07/04 12:25), Satoru Takeuchi wrote:
> Hi Chris and Kevin,
>
>>> On 07/03/2014 09:21 PM, Satoru Takeuchi wrote:
>>>> Hi Kevin,
>>>>
>>>> (2014/07/04 11:13), Kevin Brandstatter wrote:
>>>>> basing of the latest for-linus branch i found i can write way more than
>>>>> the quota
>>>>>
>>>>> btrfs quota enable
>>>>> btrfs subvolume create test
>>>>> btrfs qgruop limit 1G test
>>>>> dd if=/dev/zero of=test/file bs=1024 count=1500000
>>>>> output:
>>>>> 1500000+0 records in
>>>>> 1500000+0 records out
>>>>> 1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s
>>>>>
>>>>> thats a full half gig over the quota limit. I noticed some changes to
>>>>> the quota
>>>>> accounting in the logs, what changed that could cause this?
>>>>
>>>> Do you remember what kernel version quota worked correctly?
> (2014/07/04 11:32), Satoru Takeuchi wrote:
>> (2014/07/04 11:25), Kevin Brandstatter wrote:
>>> 3.15.3 via arch/ and from linux-git
>>
>> OK, I'll bisect it.
>
> I made the following reproducer based on your operation.
> It succeeded with 3.15 and failed with 3.16-rc3. So, the problematic
> patch is not in mason/for-linux branch, but in somewhere between
> 3.15 and 3.16-rc3. Please wait for a while to finish my bisect...

I bisected and found the bad commit is the following patch.

===============================================================================
commit fcebe4562dec83b3f8d3088d77584727b09130b2
Author: Josef Bacik <jbacik@fb.com>
Date:   Tue May 13 17:30:47 2014 -0700

     Btrfs: rework qgroup accounting
===============================================================================

Josef, please take a look at this patch.

Thanks,
Satoru

>
> ===============================================================================
> #!/bin/bash -x
>
> TEST_DEV=/dev/vdb
> TEST_MNT=/home/sat/mnt
>
> umount $TEST_MNT
> mkfs.btrfs -f $TEST_DEV
> mount $TEST_DEV $TEST_MNT
> btrfs quota enable $TEST_MNT
>
> SUBVOLPATH=$TEST_MNT/quota_test
> LIMIT=$((1024*1000000))
> btrfs subvolume create $SUBVOLPATH
> btrfs qgroup limit $LIMIT $SUBVOLPATH
> TESTFILE=$SUBVOLPATH/test
> dd if=/dev/zero of=$TESTFILE bs=1024 count=$(($LIMIT*3/2/1024))
> SIZE=$(($(ls -s $TESTFILE | awk '{print $1}')*1024))
>
> RET=0
> if [ $SIZE -le $LIMIT ] ; then
>    echo "[PASS] quota works correctly" >&2
> else
>    echo "[FAIL] quota doesn't work" >&2
>    RET=1
> fi
>
> exit $RET
> ===============================================================================
>
> Thanks,
> Satoru
>
>>>>
>>>> Thanks,
>>>> Satoru
>>>>
>>>>>
>>>>> -Kevin Brandstatter
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>> linux-btrfs" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Quota Ignored On write

[Kevin Brandstatter]

hmm, is it possible that btrfs is doing some deduplication or compression?
The expected behavior works fine with small quotas like 10/20MB
but at 1GB i can overwrite quite a bit from /dev/zero
I also tried to dd from /dev/urandom (to get some variety other than zeros)
dd if=/dev/urandom of=meow bs=1024 count=1500000
output:
dd: error writing ‘meow’: Disk quota exceeded
1163330+0 records in
1163329+0 records out
1191248896 bytes (1.2 GB) copied, 110.25 s, 10.8 MB/s

So it looks like its stopping the write, but with a 1GB quota, thats a
20% over quota

-Kevin

On 07/03/2014 09:21 PM, Satoru Takeuchi wrote:
> Hi Kevin,
>
> (2014/07/04 11:13), Kevin Brandstatter wrote:
>> basing of the latest for-linus branch i found i can write way more than
>> the quota
>>
>> btrfs quota enable
>> btrfs subvolume create test
>> btrfs qgruop limit 1G test
>> dd if=/dev/zero of=test/file bs=1024 count=1500000
>> output:
>> 1500000+0 records in
>> 1500000+0 records out
>> 1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s
>>
>> thats a full half gig over the quota limit. I noticed some changes to
>> the quota
>> accounting in the logs, what changed that could cause this?
>
> Do you remember what kernel version quota worked correctly?
>
> Thanks,
> Satoru
>
>>
>> -Kevin Brandstatter
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe
>> linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>