btrfs fi du -s gives Inappropriate ioctl for device

btrfs fi du -s gives Inappropriate ioctl for device

[Piotr Szymaniak]

[-- Attachment #1: Type: text/plain, Size: 2957 bytes --]

Hi list,

I have some weird issue.

So, I have btrfs fs and some subvols, like that:
~ # btrfs sub list /mnt/red/
ID 260 gen 827956 top level 5 path @home
ID 261 gen 827926 top level 5 path @backup
ID 645 gen 827911 top level 5 path @svn
*snip*

and some snapshots on those subvols:
*snip*
ID 2501 gen 827894 top level 260 path @home/.snapshot/monthly_2017-05-01_05:30:01
ID 2603 gen 827894 top level 260 path @home/.snapshot/monthly_2017-06-01_05:30:01
ID 2604 gen 827927 top level 261 path @backup/.snapshot/monthly_2017-06-01_05:30:02
ID 2680 gen 827895 top level 260 path @home/.snapshot/monthly_2017-07-01_05:30:01
ID 2681 gen 827927 top level 261 path @backup/.snapshot/monthly_2017-07-01_05:30:01
ID 2734 gen 827895 top level 260 path @home/.snapshot/weekly_2017-07-22_04:20:01
ID 2735 gen 827905 top level 645 path @svn/.snapshot/weekly_2017-07-22_04:20:02
ID 2754 gen 827895 top level 260 path @home/.snapshot/weekly_2017-07-29_04:20:01
ID 2755 gen 827906 top level 645 path @svn/.snapshot/weekly_2017-07-29_04:20:01
ID 2763 gen 827896 top level 260 path @home/.snapshot/monthly_2017-08-01_05:30:01
*snip*

and... some issues:
~ # btrfs fi du -s /mnt/red/\@backup/
     Total   Exclusive  Set shared  Filename
ERROR: cannot check space of '/mnt/red/@backup/': Inappropriate ioctl for device

and it works for other subvols:
~ # btrfs fi du -s /mnt/red/\@svn/
     Total   Exclusive  Set shared  Filename
  52.22GiB    10.59MiB     4.13GiB  /mnt/red/@svn/

~ # btrfs fi du -s /mnt/red/\@home/
     Total   Exclusive  Set shared  Filename
   2.08TiB     4.54GiB   124.51GiB  /mnt/red/@home/

on the other hand, those values also look wrong (2.08TiB?):
~ # btrfs fi df /mnt/red/\@home/
Data, RAID1: total=448.00GiB, used=447.39GiB
System, RAID1: total=32.00MiB, used=80.00KiB
Metadata, RAID1: total=3.00GiB, used=2.02GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

~ # btrfs fi usage /mnt/red/\@home/
Overall:
    Device size:                   1.36TiB
    Device allocated:            902.06GiB
    Device unallocated:          493.21GiB
    Device missing:                  0.00B
    Used:                        898.81GiB
    Free (estimated):            247.22GiB      (min: 247.22GiB)
    Data ratio:                       2.00
    Metadata ratio:                   2.00
    Global reserve:              512.00MiB      (used: 0.00B)

Data,RAID1: Size:448.00GiB, Used:447.39GiB
   /dev/mapper/wd0       448.00GiB
   /dev/mapper/wd1       448.00GiB

Metadata,RAID1: Size:3.00GiB, Used:2.02GiB
   /dev/mapper/wd0         3.00GiB
   /dev/mapper/wd1         3.00GiB

System,RAID1: Size:32.00MiB, Used:80.00KiB
   /dev/mapper/wd0        32.00MiB
   /dev/mapper/wd1        32.00MiB

Unallocated:
   /dev/mapper/wd0       246.60GiB
   /dev/mapper/wd1       246.60GiB

~ # uname -sr
Linux 4.4.76

~ # btrfs --version
btrfs-progs v4.12

Whats wrong with one of my subvols? Whats wrong with space reported by fi du?


Best regards,
Piotr Szymaniak.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: btrfs fi du -s gives Inappropriate ioctl for device

[Chris Murphy]

On Mon, Aug 14, 2017 at 4:57 PM, Piotr Szymaniak <szarpaj@grubelek.pl> wrote:

>
> and... some issues:
> ~ # btrfs fi du -s /mnt/red/\@backup/
>      Total   Exclusive  Set shared  Filename
> ERROR: cannot check space of '/mnt/red/@backup/': Inappropriate ioctl for device


It's a bug, but I don't know if any devs are working on a fix yet.

The problem is that the subvolume being snapshot, contains subvolumes.
The resulting snapshot, contains an empty directory in place of the
nested subvolume(s), and that is the cause for the error.


Other messages on this:
https://www.spinics.net/lists/linux-btrfs/msg67330.html
https://www.spinics.net/lists/linux-btrfs/msg61883.html



-- 
Chris Murphy

Re: btrfs fi du -s gives Inappropriate ioctl for device

[Piotr Szymaniak]

[-- Attachment #1: Type: text/plain, Size: 1442 bytes --]

On Mon, Aug 14, 2017 at 05:40:30PM -0600, Chris Murphy wrote:
> On Mon, Aug 14, 2017 at 4:57 PM, Piotr Szymaniak <szarpaj@grubelek.pl> wrote:
> 
> >
> > and... some issues:
> > ~ # btrfs fi du -s /mnt/red/\@backup/
> >      Total   Exclusive  Set shared  Filename
> > ERROR: cannot check space of '/mnt/red/@backup/': Inappropriate ioctl for device
> 
> 
> It's a bug, but I don't know if any devs are working on a fix yet.
> 
> The problem is that the subvolume being snapshot, contains subvolumes.
> The resulting snapshot, contains an empty directory in place of the
> nested subvolume(s), and that is the cause for the error.

Ok, but why, on the same btrfs, it works on some subvols with subvols and does
not work on other subvols with subvols? If it does not work - OK, if it works -
OK, but that seems a bit... random?

~ # btrfs fi du -s /mnt/red/\@backup/ /mnt/red/\@backup/.snapshot/monthly_2017-08-01_05\:30\:01/ /mnt/red/\@svn/ /mnt/red/\@svn/.snapshot/weekly_2017-08-05_04\:20\:02/
     Total   Exclusive  Set shared  Filename
ERROR: cannot check space of '/mnt/red/@backup/': Inappropriate ioctl for device
ERROR: cannot check space of '/mnt/red/@backup/.snapshot/monthly_2017-08-01_05:30:01/': Inappropriate ioctl for device
  52.23GiB    10.57MiB     4.13GiB  /mnt/red/@svn/
   4.35GiB     1.03MiB     4.12GiB  /mnt/red/@svn/.snapshot/weekly_2017-08-05_04:20:02/


Best regards,
Piotr Szymaniak.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: btrfs fi du -s gives Inappropriate ioctl for device

[Chris Murphy]

On Wed, Aug 16, 2017 at 3:27 AM, Piotr Szymaniak <szarpaj@grubelek.pl> wrote:
> On Mon, Aug 14, 2017 at 05:40:30PM -0600, Chris Murphy wrote:
>> On Mon, Aug 14, 2017 at 4:57 PM, Piotr Szymaniak <szarpaj@grubelek.pl> wrote:
>>
>> >
>> > and... some issues:
>> > ~ # btrfs fi du -s /mnt/red/\@backup/
>> >      Total   Exclusive  Set shared  Filename
>> > ERROR: cannot check space of '/mnt/red/@backup/': Inappropriate ioctl for device
>>
>>
>> It's a bug, but I don't know if any devs are working on a fix yet.
>>
>> The problem is that the subvolume being snapshot, contains subvolumes.
>> The resulting snapshot, contains an empty directory in place of the
>> nested subvolume(s), and that is the cause for the error.
>
> Ok, but why, on the same btrfs, it works on some subvols with subvols and does
> not work on other subvols with subvols? If it does not work - OK, if it works -
> OK, but that seems a bit... random?
>
> ~ # btrfs fi du -s /mnt/red/\@backup/ /mnt/red/\@backup/.snapshot/monthly_2017-08-01_05\:30\:01/ /mnt/red/\@svn/ /mnt/red/\@svn/.snapshot/weekly_2017-08-05_04\:20\:02/
>      Total   Exclusive  Set shared  Filename
> ERROR: cannot check space of '/mnt/red/@backup/': Inappropriate ioctl for device
> ERROR: cannot check space of '/mnt/red/@backup/.snapshot/monthly_2017-08-01_05:30:01/': Inappropriate ioctl for device
>   52.23GiB    10.57MiB     4.13GiB  /mnt/red/@svn/
>    4.35GiB     1.03MiB     4.12GiB  /mnt/red/@svn/.snapshot/weekly_2017-08-05_04:20:02/

I don't know.

 It might be that there's something inconsistent about the inode for
the missing/ghost subvolume placeholder directory at snapshot creation
time?

-- 
Chris Murphy

Re: btrfs fi du -s gives Inappropriate ioctl for device

[Goffredo Baroncelli]

Hi,

On 08/15/2017 12:57 AM, Piotr Szymaniak wrote:
> and... some issues:
> ~ # btrfs fi du -s /mnt/red/\@backup/
>      Total   Exclusive  Set shared  Filename
> ERROR: cannot check space of '/mnt/red/@backup/': Inappropriate ioctl for device


could you share the output of 

strace btrfs fi du -s /mnt/red/\@backup/

this shouldn't  contain sensible information

BR
G.Baroncelli

-- 
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: btrfs fi du -s gives Inappropriate ioctl for device

[Chris Murphy]

# btrfs sub create test1
Create subvolume './test1'
# btrfs sub create test1/test2
Create subvolume 'test1/test2'
# btrfs sub snap test1 test1.snap
Create a snapshot of 'test1' in './test1.snap'
# btrfs fi du -s test1
     Total   Exclusive  Set shared  Filename
     0.00B       0.00B       0.00B  test1
# btrfs fi du -s test1.snap
     Total   Exclusive  Set shared  Filename
ERROR: cannot check space of 'test1.snap': Inappropriate ioctl for device
#


Pretty strace here:
https://pastebin.com/raw/ibnha9Sx


Ugly MUA strace here for the archive:


# strace btrfs fi du -s test1.snap
execve("/sbin/btrfs", ["btrfs", "fi", "du", "-s", "test1.snap"],
0x7fff8c0af970 /* 32 vars */) = 0
brk(NULL)                               = 0xa96a201000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7f84880a4000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=104181, ...}) = 0
mmap(NULL, 104181, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f848808a000
close(3)                                = 0
open("/lib64/libuuid.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\23\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=19568, ...}) = 0
mmap(NULL, 2113552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7f8487c7b000
mprotect(0x7f8487c7f000, 2093056, PROT_NONE) = 0
mmap(0x7f8487e7e000, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7f8487e7e000
mmap(0x7f8487e7f000, 16, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f8487e7f000
close(3)                                = 0
open("/lib64/libblkid.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\227\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=304792, ...}) = 0
mmap(NULL, 2397952, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7f8487a31000
mprotect(0x7f8487a75000, 2093056, PROT_NONE) = 0
mmap(0x7f8487c74000, 24576, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x43000) = 0x7f8487c74000
mmap(0x7f8487c7a000, 1792, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f8487c7a000
close(3)                                = 0
open("/lib64/libz.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\"\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=94104, ...}) = 0
mmap(NULL, 2187272, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7f848781a000
mprotect(0x7f8487830000, 2093056, PROT_NONE) = 0
mmap(0x7f8487a2f000, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x7f8487a2f000
mmap(0x7f8487a30000, 8, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f8487a30000
close(3)                                = 0
open("/lib64/liblzo2.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320$\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=138800, ...}) = 0
mmap(NULL, 2232328, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7f84875f8000
mprotect(0x7f8487619000, 2093056, PROT_NONE) = 0
mmap(0x7f8487818000, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x20000) = 0x7f8487818000
mmap(0x7f8487819000, 8, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f8487819000
close(3)                                = 0
open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340^\0\0\0\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=153896, ...}) = 0
mmap(NULL, 2220552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7f84873d9000
mprotect(0x7f84873f2000, 2097152, PROT_NONE) = 0
mmap(0x7f84875f2000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x19000) = 0x7f84875f2000
mmap(0x7f84875f4000, 12808, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f84875f4000
close(3)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\5\2\0\0\0\0\0"...,
832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=2163104, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7f8488088000
mmap(NULL, 4000096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x7f8487008000
mprotect(0x7f84871cf000, 2097152, PROT_NONE) = 0
mmap(0x7f84873cf000, 24576, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c7000) = 0x7f84873cf000
mmap(0x7f84873d5000, 14688, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f84873d5000
close(3)                                = 0
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7f8488085000
arch_prctl(ARCH_SET_FS, 0x7f84880858c0) = 0
mprotect(0x7f84873cf000, 16384, PROT_READ) = 0
mprotect(0x7f84875f2000, 4096, PROT_READ) = 0
mprotect(0x7f8487818000, 4096, PROT_READ) = 0
mprotect(0x7f8487a2f000, 4096, PROT_READ) = 0
mprotect(0x7f8487e7e000, 4096, PROT_READ) = 0
mprotect(0x7f8487c74000, 20480, PROT_READ) = 0
mprotect(0xa968435000, 20480, PROT_READ) = 0
mprotect(0x7f84880a6000, 4096, PROT_READ) = 0
munmap(0x7f848808a000, 104181)          = 0
set_tid_address(0x7f8488085b90)         = 13929
set_robust_list(0x7f8488085ba0, 24)     = 0
rt_sigaction(SIGRTMIN, {sa_handler=0x7f84873de960, sa_mask=[],
sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7f84873eb2c0}, NULL, 8)
= 0
rt_sigaction(SIGRT_1, {sa_handler=0x7f84873dea00, sa_mask=[],
sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO,
sa_restorer=0x7f84873eb2c0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024,
rlim_max=RLIM64_INFINITY}) = 0
uname({sysname="Linux", nodename="f26h.localdomain", ...}) = 0
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
brk(NULL)                               = 0xa96a201000
brk(0xa96a222000)                       = 0xa96a222000
brk(NULL)                               = 0xa96a222000
write(1, "     Total   Exclusive  Set shar"..., 45     Total
Exclusive  Set shared  Filename
) = 45
newfstatat(AT_FDCWD, "test1.snap", {st_mode=S_IFDIR|0755, st_size=10,
...}, 0) = 0
stat("test1.snap", {st_mode=S_IFDIR|0755, st_size=10, ...}) = 0
open("test1.snap", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=10, ...}) = 0
ioctl(3, BTRFS_IOC_INO_LOOKUP, {treeid=0, objectid=256 /*
BTRFS_FIRST_FREE_OBJECTID */} => {treeid=486, name=""}) = 0
getdents(3, /* 3 entries */, 32768)     = 80
newfstatat(3, "test2", {st_mode=S_IFDIR|0755, st_size=0, ...}, 0) = 0
stat("test1.snap/test2", {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
open("test1.snap/test2", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 4
fstat(4, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
ioctl(4, BTRFS_IOC_INO_LOOKUP, {treeid=0, objectid=256 /*
BTRFS_FIRST_FREE_OBJECTID */}) = -1 ENOTTY (Inappropriate ioctl for
device)
close(4)                                = 0
getdents(3, /* 0 entries */, 32768)     = 0
close(3)                                = 0
write(2, "ERROR: ", 7ERROR: )                  = 7
write(2, "cannot check space of 'test1.sna"..., 66cannot check space
of 'test1.snap': Inappropriate ioctl for device) = 66
write(2, "\n", 1
)                       = 1
exit_group(1)                           = ?
+++ exited with 1 +++
#


# uname -r
4.13.0-0.rc4.git1.1.fc27.x86_64
# rpm -q btrfs-progs
btrfs-progs-4.12-1.fc27.x86_64


----
Chris Murphy

Re: btrfs fi du -s gives Inappropriate ioctl for device

[Goffredo Baroncelli]

Hi,

On 08/17/2017 08:43 PM, Chris Murphy wrote:
> # btrfs sub create test1
> Create subvolume './test1'
> # btrfs sub create test1/test2
> Create subvolume 'test1/test2'
> # btrfs sub snap test1 test1.snap
> Create a snapshot of 'test1' in './test1.snap'
> # btrfs fi du -s test1
>      Total   Exclusive  Set shared  Filename
>      0.00B       0.00B       0.00B  test1
> # btrfs fi du -s test1.snap
>      Total   Exclusive  Set shared  Filename
> ERROR: cannot check space of 'test1.snap': Inappropriate ioctl for device
> #
> 

tanks for the test case. Now I was able to reproduce the problem. The bug(s) are two:

1) to get the treeid of the files/directory, the function lookup_path_rootid() is used, which behaves strangely when it is called on a directory that is BTRFS_EMPTY_SUBVOL_DIR_OBJECTID. In fact this function is commented as following:
[....]
/*
 * For a given:
 * - file or directory return the containing tree root id
 * - subvolume return its own tree id
 * - BTRFS_EMPTY_SUBVOL_DIR_OBJECTID (directory with ino == 2) the result is
 *   undefined and function returns -1
 */
int lookup_path_rootid(int fd, u64 *rootid)
{
[....]

The caller (du_add_file()) doesn't consider this case.

2) in the function du_walk_dir(), an error returned by du_add_file() is ignored, but the return value is not reset. So if the last entry has ino BTRFS_EMPTY_SUBVOL_DIR_OBJECTID (directory with ino == 2) an error is returned. But if there is another directory item no error is returned !

See the following tests cases:

# btrfs sub create test1
# btrfs sub create test1/test2
# btrfs sub snap test1 test1.snap
# btrfs fi du -s test1
     Total   Exclusive  Set shared  Filename
     0.00B       0.00B       0.00B  test1
# btrfs fi du -s test1.snap
     Total   Exclusive  Set shared  Filename
ERROR: cannot check space of 'test1.snap': Inappropriate ioctl for device

But if you add *another* file/dir in test1.snap you got:

# mkdir test1.snap/dir
# btrfs fi du -s test1.snap
     Total   Exclusive  Set shared  Filename
     0.00B       0.00B       0.00B  test1.snap

The error disappear !

Patches will follow shortly

> 
> # uname -r
> 4.13.0-0.rc4.git1.1.fc27.x86_64
> # rpm -q btrfs-progs
> btrfs-progs-4.12-1.fc27.x86_64
> 
> 
> ----
> Chris Murphy
> 


-- 
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: btrfs fi du -s gives Inappropriate ioctl for device

[Nikolay Borisov]

On 17.08.2017 23:59, Goffredo Baroncelli wrote:
> Hi,
> 
> On 08/17/2017 08:43 PM, Chris Murphy wrote:
>> # btrfs sub create test1
>> Create subvolume './test1'
>> # btrfs sub create test1/test2
>> Create subvolume 'test1/test2'
>> # btrfs sub snap test1 test1.snap
>> Create a snapshot of 'test1' in './test1.snap'
>> # btrfs fi du -s test1
>>      Total   Exclusive  Set shared  Filename
>>      0.00B       0.00B       0.00B  test1
>> # btrfs fi du -s test1.snap
>>      Total   Exclusive  Set shared  Filename
>> ERROR: cannot check space of 'test1.snap': Inappropriate ioctl for device
>> #
>>
> 
> tanks for the test case. Now I was able to reproduce the problem. The bug(s) are two:
> 
> 1) to get the treeid of the files/directory, the function lookup_path_rootid() is used, which behaves strangely when it is called on a directory that is BTRFS_EMPTY_SUBVOL_DIR_OBJECTID. In fact this function is commented as following:
> [....]
> /*
>  * For a given:
>  * - file or directory return the containing tree root id
>  * - subvolume return its own tree id
>  * - BTRFS_EMPTY_SUBVOL_DIR_OBJECTID (directory with ino == 2) the result is
>  *   undefined and function returns -1
>  */
> int lookup_path_rootid(int fd, u64 *rootid)
> {
> [....]
> 
> The caller (du_add_file()) doesn't consider this case.
> 
> 2) in the function du_walk_dir(), an error returned by du_add_file() is ignored, but the return value is not reset. So if the last entry has ino BTRFS_EMPTY_SUBVOL_DIR_OBJECTID (directory with ino == 2) an error is returned. But if there is another directory item no error is returned !
> 
> See the following tests cases:
> 
> # btrfs sub create test1
> # btrfs sub create test1/test2
> # btrfs sub snap test1 test1.snap
> # btrfs fi du -s test1
>      Total   Exclusive  Set shared  Filename
>      0.00B       0.00B       0.00B  test1
> # btrfs fi du -s test1.snap
>      Total   Exclusive  Set shared  Filename
> ERROR: cannot check space of 'test1.snap': Inappropriate ioctl for device
> 
> But if you add *another* file/dir in test1.snap you got:
> 
> # mkdir test1.snap/dir
> # btrfs fi du -s test1.snap
>      Total   Exclusive  Set shared  Filename
>      0.00B       0.00B       0.00B  test1.snap
> 
> The error disappea> Patches will follow shortly

It would be awesome if you manage to introduce xfstests for this case

> 
>>
>> # uname -r
>> 4.13.0-0.rc4.git1.1.fc27.x86_64
>> # rpm -q btrfs-progs
>> btrfs-progs-4.12-1.fc27.x86_64
>>
>>
>> ----
>> Chris Murphy
>>
> 
> 

Re: btrfs fi du -s gives Inappropriate ioctl for device

[Goffredo Baroncelli]

On 08/18/2017 08:34 AM, Nikolay Borisov wrote:
> 
> 
> On 17.08.2017 23:59, Goffredo Baroncelli wrote:
>> Hi,
>>
>> On 08/17/2017 08:43 PM, Chris Murphy wrote:
>>> # btrfs sub create test1
>>> Create subvolume './test1'
>>> # btrfs sub create test1/test2
>>> Create subvolume 'test1/test2'
>>> # btrfs sub snap test1 test1.snap
>>> Create a snapshot of 'test1' in './test1.snap'
>>> # btrfs fi du -s test1
>>>      Total   Exclusive  Set shared  Filename
>>>      0.00B       0.00B       0.00B  test1
>>> # btrfs fi du -s test1.snap
>>>      Total   Exclusive  Set shared  Filename
>>> ERROR: cannot check space of 'test1.snap': Inappropriate ioctl for device
>>> #
>>>
>>
>> tanks for the test case. Now I was able to reproduce the problem. The bug(s) are two:
>>
>> 1) to get the treeid of the files/directory, the function lookup_path_rootid() is used, which behaves strangely when it is called on a directory that is BTRFS_EMPTY_SUBVOL_DIR_OBJECTID. In fact this function is commented as following:
>> [....]
>> /*
>>  * For a given:
>>  * - file or directory return the containing tree root id
>>  * - subvolume return its own tree id
>>  * - BTRFS_EMPTY_SUBVOL_DIR_OBJECTID (directory with ino == 2) the result is
>>  *   undefined and function returns -1
>>  */
>> int lookup_path_rootid(int fd, u64 *rootid)
>> {
>> [....]
>>
>> The caller (du_add_file()) doesn't consider this case.
>>
>> 2) in the function du_walk_dir(), an error returned by du_add_file() is ignored, but the return value is not reset. So if the last entry has ino BTRFS_EMPTY_SUBVOL_DIR_OBJECTID (directory with ino == 2) an error is returned. But if there is another directory item no error is returned !
>>
>> See the following tests cases:
>>
>> # btrfs sub create test1
>> # btrfs sub create test1/test2
>> # btrfs sub snap test1 test1.snap
>> # btrfs fi du -s test1
>>      Total   Exclusive  Set shared  Filename
>>      0.00B       0.00B       0.00B  test1
>> # btrfs fi du -s test1.snap
>>      Total   Exclusive  Set shared  Filename
>> ERROR: cannot check space of 'test1.snap': Inappropriate ioctl for device
>>
>> But if you add *another* file/dir in test1.snap you got:
>>
>> # mkdir test1.snap/dir
>> # btrfs fi du -s test1.snap
>>      Total   Exclusive  Set shared  Filename
>>      0.00B       0.00B       0.00B  test1.snap
>>
>> The error disappea> Patches will follow shortly
> 
> It would be awesome if you manage to introduce xfstests for this case

I am not sure if this is the right thing to do: the bugs are related to 
a) an incorrect error handling of the function lookup_path_rootid()
b) the strange behavior of the BTRFS_EMPTY_SUBVOL_DIR_OBJECTID directory

For a) it is impossible to check for each lookup_path_rootid() call from a xfstest; for b) I am quite sure that there are a lot of corner case not properly handled in btrfs progs (and still btrfs fi-du handle it wrongly)

BR
G.Baroncelli

> 
>>
>>>
>>> # uname -r
>>> 4.13.0-0.rc4.git1.1.fc27.x86_64
>>> # rpm -q btrfs-progs
>>> btrfs-progs-4.12-1.fc27.x86_64
>>>
>>>
>>> ----
>>> Chris Murphy
>>>
>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


-- 
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: btrfs fi du -s gives Inappropriate ioctl for device

[Nikolay Borisov]

On 18.08.2017 10:09, Goffredo Baroncelli wrote:
> On 08/18/2017 08:34 AM, Nikolay Borisov wrote:
>>
>>
>> On 17.08.2017 23:59, Goffredo Baroncelli wrote:
>>> Hi,
>>>
>>> On 08/17/2017 08:43 PM, Chris Murphy wrote:
>>>> # btrfs sub create test1
>>>> Create subvolume './test1'
>>>> # btrfs sub create test1/test2
>>>> Create subvolume 'test1/test2'
>>>> # btrfs sub snap test1 test1.snap
>>>> Create a snapshot of 'test1' in './test1.snap'
>>>> # btrfs fi du -s test1
>>>>      Total   Exclusive  Set shared  Filename
>>>>      0.00B       0.00B       0.00B  test1
>>>> # btrfs fi du -s test1.snap
>>>>      Total   Exclusive  Set shared  Filename
>>>> ERROR: cannot check space of 'test1.snap': Inappropriate ioctl for device
>>>> #
>>>>
>>>
>>> tanks for the test case. Now I was able to reproduce the problem. The bug(s) are two:
>>>
>>> 1) to get the treeid of the files/directory, the function lookup_path_rootid() is used, which behaves strangely when it is called on a directory that is BTRFS_EMPTY_SUBVOL_DIR_OBJECTID. In fact this function is commented as following:
>>> [....]
>>> /*
>>>  * For a given:
>>>  * - file or directory return the containing tree root id
>>>  * - subvolume return its own tree id
>>>  * - BTRFS_EMPTY_SUBVOL_DIR_OBJECTID (directory with ino == 2) the result is
>>>  *   undefined and function returns -1
>>>  */
>>> int lookup_path_rootid(int fd, u64 *rootid)
>>> {
>>> [....]
>>>
>>> The caller (du_add_file()) doesn't consider this case.
>>>
>>> 2) in the function du_walk_dir(), an error returned by du_add_file() is ignored, but the return value is not reset. So if the last entry has ino BTRFS_EMPTY_SUBVOL_DIR_OBJECTID (directory with ino == 2) an error is returned. But if there is another directory item no error is returned !
>>>
>>> See the following tests cases:
>>>
>>> # btrfs sub create test1
>>> # btrfs sub create test1/test2
>>> # btrfs sub snap test1 test1.snap
>>> # btrfs fi du -s test1
>>>      Total   Exclusive  Set shared  Filename
>>>      0.00B       0.00B       0.00B  test1
>>> # btrfs fi du -s test1.snap
>>>      Total   Exclusive  Set shared  Filename
>>> ERROR: cannot check space of 'test1.snap': Inappropriate ioctl for device
>>>
>>> But if you add *another* file/dir in test1.snap you got:
>>>
>>> # mkdir test1.snap/dir
>>> # btrfs fi du -s test1.snap
>>>      Total   Exclusive  Set shared  Filename
>>>      0.00B       0.00B       0.00B  test1.snap
>>>
>>> The error disappea> Patches will follow shortly
>>
>> It would be awesome if you manage to introduce xfstests for this case
> 
> I am not sure if this is the right thing to do: the bugs are related to 
> a) an incorrect error handling of the function lookup_path_rootid()
> b) the strange behavior of the BTRFS_EMPTY_SUBVOL_DIR_OBJECTID directory
> 
> For a) it is impossible to check for each lookup_path_rootid() call from a xfstest; for b) I am quite sure that there are a lot of corner case not properly handled in btrfs progs (and still btrfs fi-du handle it wrongly)

So the bug is actually in the btrfsprogs not in the fs, am I correct?

> 
> BR
> G.Baroncelli
> 
>>
>>>
>>>>
>>>> # uname -r
>>>> 4.13.0-0.rc4.git1.1.fc27.x86_64
>>>> # rpm -q btrfs-progs
>>>> btrfs-progs-4.12-1.fc27.x86_64
>>>>
>>>>
>>>> ----
>>>> Chris Murphy
>>>>
>>>
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 
> 

Re: btrfs fi du -s gives Inappropriate ioctl for device

[Goffredo Baroncelli]

On 08/18/2017 09:12 AM, Nikolay Borisov wrote:
> 
> 
> On 18.08.2017 10:09, Goffredo Baroncelli wrote:
>> On 08/18/2017 08:34 AM, Nikolay Borisov wrote:
[...]
>>> It would be awesome if you manage to introduce xfstests for this case
>>
>> I am not sure if this is the right thing to do: the bugs are related to 
>> a) an incorrect error handling of the function lookup_path_rootid()
>> b) the strange behavior of the BTRFS_EMPTY_SUBVOL_DIR_OBJECTID directory
>>
>> For a) it is impossible to check for each lookup_path_rootid() call from a xfstest; for b) I am quite sure that there are a lot of corner case not properly handled in btrfs progs (and still btrfs fi-du handle it wrongly)
> 
> So the bug is actually in the btrfsprogs not in the fs, am I correct?
> 
Yes, definitively


BR
G.Baroncelli
-- 
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: btrfs fi du -s gives Inappropriate ioctl for device

[Piotr Szymaniak]

[-- Attachment #1: Type: text/plain, Size: 822 bytes --]

On Thu, Aug 17, 2017 at 10:59:33PM +0200, Goffredo Baroncelli wrote:
> Patches will follow shortly

With your patches applied:

~ # btrfs fi du -s /mnt/red/\@backup/ /mnt/red/\@backup/.snapshot/monthly_2017-08-01_05\:30\:01/ /mnt/red/\@svn/ /mnt/red/\@svn/.snapshot/weekly_2017-08-05_04\:20\:02/
     Total   Exclusive  Set shared  Filename
   1.86TiB    75.73MiB   312.77GiB  /mnt/red/@backup/
 320.29GiB       0.00B   312.19GiB  /mnt/red/@backup/.snapshot/monthly_2017-08-01_05:30:01/
  52.24GiB    10.19MiB     4.13GiB  /mnt/red/@svn/
   4.35GiB     1.05MiB     4.12GiB  /mnt/red/@svn/.snapshot/weekly_2017-08-05_04:20:02/


Best regards,
Piotr Szymaniak.
-- 
Mysle,  ze gdyby  diabel  nie istnial i mialby go stworzyc czlowiek, to
stworzylby go na swoj obraz i podobienstwo.
  -- Fiodor Dostojewski

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]