Re: Non-existent qgroup in parent-child relation prevents makes qgroup commands fail

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

[-- Attachment #1.1: Type: text/plain, Size: 4788 bytes --]



On 2019/8/3 下午1:31, Andrei Borzenkov wrote:
> 03.08.2019 2:09, Qu Wenruo пишет:
>>
>>
>> On 2019/8/3 上午2:08, Andrei Borzenkov wrote:
>>> bor@tw:~> sudo btrfs qgroup show .
>>> ERROR: cannot find the qgroup 0/789
>>> bor@tw:~>
>>>
>>> Fine. This openSUSE with snapper which creates and automatically
>>> destroys snapshots and apparently either kernel or snapper now also
>>> remove corresponding qgroup. I played with snapshots and created several
>>> top level qgroups that included snapshot qgroups existing at this time.
>>> Now these snapshots are gone, their qgroups are gone ...
>>
>> Kernel version please.
>>
>> IIRC latest upstream kernel doesn't remove the level 0 qgroup.
> 
> Yes?
> 
>> It may be the userspace doing it improperly.
>>
> 
> Not sure what "improperly" means here. snapper removes qgroup after
> deleting snapshot. What is "improper" here?

Doing without using the qgroup ioctl, but some extra flag in snapshot
creation/deletion, which can also add relation at subv/snapshot creation
time.

> 
> Snapper obviously does not track every parent-child relationship beyond
> what it itself cares about (single summary qrgoup that includes all
> snapshots).
> 
>>> and what can I
>>> do? I have no way to even know what is wrong because the very command
>>> that shows it fails immediately.
>>>
>>> bor@tw:~/python-btrfs/examples> sudo ./show_tree_keys.py 8 . | grep 0/789
>>> (0/789 QGROUP_RELATION 2/792)
>>> (0/789 QGROUP_RELATION 2/793)
>>> (0/789 QGROUP_RELATION 2/795)
>>> (0/789 QGROUP_RELATION 2/799)
>>> (0/789 QGROUP_RELATION 2/800)
>>> (0/789 QGROUP_RELATION 2/803)
>>> (0/789 QGROUP_RELATION 2/804)
>>> (0/789 QGROUP_RELATION 2/805)
>>> (0/789 QGROUP_RELATION 2/806)
>>> (0/789 QGROUP_RELATION 2/807)
>>> (0/789 QGROUP_RELATION 2/808)
>>> (0/789 QGROUP_RELATION 2/809)
>>> (0/789 QGROUP_RELATION 2/814)
>>> (0/789 QGROUP_RELATION 2/818)
>>> (0/789 QGROUP_RELATION 2/819)
>>> (2/792 QGROUP_RELATION 0/789)
>>> (2/793 QGROUP_RELATION 0/789)
>>> (2/795 QGROUP_RELATION 0/789)
>>> (2/799 QGROUP_RELATION 0/789)
>>> (2/800 QGROUP_RELATION 0/789)
>>> (2/803 QGROUP_RELATION 0/789)
>>> (2/804 QGROUP_RELATION 0/789)
>>> (2/805 QGROUP_RELATION 0/789)
>>> (2/806 QGROUP_RELATION 0/789)
>>> (2/807 QGROUP_RELATION 0/789)
>>> (2/808 QGROUP_RELATION 0/789)
>>> (2/809 QGROUP_RELATION 0/789)
>>> (2/814 QGROUP_RELATION 0/789)
>>> (2/818 QGROUP_RELATION 0/789)
>>> (2/819 QGROUP_RELATION 0/789)
>>> bor@tw:~/python-btrfs/examples>
>>>
>>> And even if I find it out, I cannot fix it anyway
>>
>> Furthermore, latest kernel should automatically remove the relation when
>> deleting the qgroup.
>>
> 
> Yes, seems to be the case now with 5.2.3.
> 
>> Would you please provide the (minimal) script/reproducer causing the
>> situation and kernel version?
>>
> 
> I cannot reproduce it on purpose with kernel 5.2.3 and btrfsprogs 5.1. I
> do not remember when I created the configuration in question, it
> probably was late 4.x or early 5.x. System was periodically updated
> since then and I noticed it only now.
> 
> Actually kernel should be dropping parent-child relation when removing
> child since kernel 4.1 (commit
> f5a6b1c53bdd44f79e3904c0f5e59f956b49b2c8). May be there is (was) some
> other code path that skipped it.
> 
> OTOH this is not atomic. First qgroup item itself is removed, then
> relation. If removing relation fails for whatever reason item itself is
> already gone and we have situation above.

You're right about the removal, although the term is not "atomic".

For qgroup item deletion, we can return -ENOENT or other serious tree
corruption error from btrfs_search_slot()/btrfs_del_item().

But even for -ENOENT case, we still tries to delete the relation item.
So that part is OK.

The problem is the del_qgroup_relation part, where if we hit even
-ENOENT we abort deleting even if we may have extra items needs to delete.

We need a little more polish on the error handler.

So is the btrfs_del_qgroup_relation() where it doesn't delete all
relations, but only one.

I'll soon send out patches to address your bugs.

Thanks,
Qu

> 
>> Thanks,
>> Qu
>>
>>>
>>> bor@tw:~/python-btrfs/examples> sudo btrfs qgroup remove 0/789 2/792 .
>>> ERROR: unable to assign quota group: Invalid argument
>>> bor@tw:~/python-btrfs/examples>
>>>
>>> I can remove parent qgroup, but it does not clean up parent-child
>>> relationship
>>>
>>> bor@tw:~/python-btrfs/examples> sudo btrfs qgroup destroy 2/792 .
>>> bor@tw:~/python-btrfs/examples> sudo ./show_tree_keys.py 8 . | grep 2/792
>>> (0/789 QGROUP_RELATION 2/792)
>>> (2/792 QGROUP_RELATION 0/789)
>>> bor@tw:~/python-btrfs/examples>
>>>
>>
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]