Re: Retrieving CSUM-Tree

[Tobias Reinhard <trtracer@gmail.com>]

Am 25.01.2019 um 19:05 schrieb Hans van Kranenburg:
> On 1/25/19 5:59 PM, Tobias Reinhard wrote:
>> Am 13.01.2019 um 12:02 schrieb Qu Wenruo:
>>> On 2019/1/13 下午6:19, Tobias Reinhard wrote:
>>>> Hi,
>>>>
>>>> I want to read the complete CSUM-Tree from userspace. I tried it via the
>>>> ioctl. This is what the code looks like:
>>>>
>>>> struct btrfs_sv2_args sv2_args;
>>>> int fd = open(filename, O_RDONLY);
>>>> sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID;
>>>> sv2_args.key.min_objectid = 0;
>>>> sv2_args.key.max_objectid = -1;
>>>> sv2_args.key.min_offset = 0;
>>>> sv2_args.key.max_offset = -1;
>>>> sv2_args.key.min_transid = 0;
>>>> sv2_args.key.max_transid = -1;
>>>> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
>>>> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;
>>>> sv2_args.key.nr_items = -1;
>>>> sv2_args.buf_size = sizeof(sv2_args.buf);
>>>> ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args);
>>>>
>>>> But the device is not small and I hit the limit of the
>>>> btrfs_sv2_args.buf which seems to be 16 MB.
>>>>
>>>> How can I get the *complete* CSUM-Tree?
>>>>
>>>> Limiting to offset does not work (My first idea was to do it this way
>>>> and get it in chunks).
>>> That's strange.
>>>
>>> Are you still using 0~-1 objectid and 0~-1 type, just last_offset~-1?
>>>
>>> Have tried searching using the following parameters?
>>> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID
>>> min_type = max_type = BTRFS_CSUM_ITEM_KEY;
>>> min_offset = last_found_csum_offset
>>> max_offset = -1
>> Sorry for my late response.
>>
>> If I set
>>
>> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID
>>
>> I don't get anything. I have to set it to max=-1 (min doesn't matter).
>>
>> And in that I case, min_offset and max_offset doesn't matter - I always
>> get the same result. I can even use "wrong" filters like min=1000 max=500.
> First, it's important to understand how all these min/max values play
> together:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/btrfs.h#n441
>
> So, you define a single start key, and a single end key, and then you
> get everything that's in between (including the end value).
>
> So, this...
>
> sv2_args.key.min_objectid = 0;
> sv2_args.key.max_objectid = -1;
> sv2_args.key.min_offset = 0;
> sv2_args.key.max_offset = -1;
> sv2_args.key.min_transid = 0;
> sv2_args.key.max_transid = -1;
> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;
>
> ...translates to:
>
> min key: (0, CSUM_ITEM_KEY, 0)
> max key: (18446744073709551615, CSUM_ITEM_KEY, 18446744073709551615)
>
> Since the keys end up being just a single 136 bit number, it makes no
> sense to do anything with the middle field, if the first field, objectid
> is not the same in both start and end key. The search space is linear,
> not 3 dimensional. The invidual min/max values for objectid, type and
> offset cannot be used to filter the result, they only define the
> endpoints of an interval.
>
> Since all csum items have the same objectid number anyway, the second
> suggestion is fine, and gives you this start and end:
>
> min key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 0)
> max key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 18446744073709551615)
>
> Works for me (here in python, but using same ioctl):
>
> -$ cat show_csum_keys.py
> #!/usr/bin/python3
>
> import btrfs
> from btrfs.ctree import Key, CSUM_TREE_OBJECTID, \
>          EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY
>
> with btrfs.FileSystem('/mnt/tutorial') as fs:
>      min_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY, 0)
>      max_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY + 1, 0) - 1
>      print("Searching from {} to {}".format(min_key, max_key))
>      for header, data in btrfs.ioctl.search_v2(fs.fd, CSUM_TREE_OBJECTID,
>                                                min_key, max_key):
>          print(Key(header.objectid, header.type, header.offset))
>
> -# ./show_csum_keys.py
> Searching from (EXTENT_CSUM EXTENT_CSUM 0) to (EXTENT_CSUM EXTENT_CSUM -1)
> (EXTENT_CSUM EXTENT_CSUM 5700059136)
> (EXTENT_CSUM EXTENT_CSUM 5700321280)
> (EXTENT_CSUM EXTENT_CSUM 5700583424)
> (EXTENT_CSUM EXTENT_CSUM 5700845568)
> (EXTENT_CSUM EXTENT_CSUM 5701107712)
> (EXTENT_CSUM EXTENT_CSUM 5704646656)
> (EXTENT_CSUM EXTENT_CSUM 5705039872)
> (EXTENT_CSUM EXTENT_CSUM 5706350592)
> [...]
>
Reading your example, I noticed my mistake.

I took  BTRFS_CSUM_ITEM_KEY  instead of BTRFS_EXTENT_CSUM_KEY for the 
type, doh.

Now, it seems to works.

Thanks!

Tobias