btrfs send receive ERROR: chown failed: No such file or directory

btrfs send receive ERROR: chown failed: No such file or directory

[Leonard Lausen]

Hello!

I observe the following issue with btrfs send | btrfs receive in a setup
with 2 machines and 3 btrfs file-systems. All machines run Linux 4.18.9.
Machine 1 runs btrfs-progs 4.17.1, machine 2 runs btrfs-progs 4.17 (via
https://packages.debian.org/stretch-backports/btrfs-progs).

1) Machine 1 takes regular snapshots and sends them to machine 2. btrfs 
   btrfs send ... | ssh user@machine2 "btrfs receive /path1"
2) Machine 2 backups all subvolumes stored at /path1 to a second
   independent btrfs filesystem. Let /path1/rootsnapshot be the first
   snapshot stored at /path1 (ie. it has no Parent UUID). Let
   /path1/incrementalsnapshot be a snapshot that has /path1/rootsnapshot
   as a parent. Then
   btrfs send -v /path1/rootsnapshot | btrfs receive /path2
   works without issues, but
   btrfs send -v -p /path1/rootsnapshot /path1/incrementalsnapshot | btrfs receive /path2
   fails as follows:
   ERROR: chown o257-4639416-0 failed: No such file or directory

No error is shown in dmesg. /path1 and /path2 denote two independent
btrfs filesystems.

Note that there was no issue with transferring incrementalsnapshot from
machine 1 to machine 2. No error is shown in dmesg.

Best regards
Leonard

Re: btrfs send receive ERROR: chown failed: No such file or directory

[Leonard Lausen]

Hello,

does anyone have an idea about below issue? It is a severe issue as it
renders btrfs send / receive dysfunctional and it is not clear if there
may be a data corruption issue hiding in the current send / receive
code.

Thank you.

Best regards
Leonard

Leonard Lausen <leonard@lausen.nl> writes:
> Hello!
>
> I observe the following issue with btrfs send | btrfs receive in a setup
> with 2 machines and 3 btrfs file-systems. All machines run Linux 4.18.9.
> Machine 1 runs btrfs-progs 4.17.1, machine 2 runs btrfs-progs 4.17 (via
> https://packages.debian.org/stretch-backports/btrfs-progs).
>
> 1) Machine 1 takes regular snapshots and sends them to machine 2. btrfs 
>    btrfs send ... | ssh user@machine2 "btrfs receive /path1"
> 2) Machine 2 backups all subvolumes stored at /path1 to a second
>    independent btrfs filesystem. Let /path1/rootsnapshot be the first
>    snapshot stored at /path1 (ie. it has no Parent UUID). Let
>    /path1/incrementalsnapshot be a snapshot that has /path1/rootsnapshot
>    as a parent. Then
>    btrfs send -v /path1/rootsnapshot | btrfs receive /path2
>    works without issues, but
>    btrfs send -v -p /path1/rootsnapshot /path1/incrementalsnapshot | btrfs receive /path2
>    fails as follows:
>    ERROR: chown o257-4639416-0 failed: No such file or directory
>
> No error is shown in dmesg. /path1 and /path2 denote two independent
> btrfs filesystems.
>
> Note that there was no issue with transferring incrementalsnapshot from
> machine 1 to machine 2. No error is shown in dmesg.
>
> Best regards
> Leonard

Re: btrfs send receive ERROR: chown failed: No such file or directory

[Filipe Manana]

On Tue, Oct 2, 2018 at 7:02 AM Leonard Lausen <leonard@lausen.nl> wrote:
>
> Hello,
>
> does anyone have an idea about below issue? It is a severe issue as it
> renders btrfs send / receive dysfunctional and it is not clear if there
> may be a data corruption issue hiding in the current send / receive
> code.
>
> Thank you.
>
> Best regards
> Leonard
>
> Leonard Lausen <leonard@lausen.nl> writes:
> > Hello!
> >
> > I observe the following issue with btrfs send | btrfs receive in a setup
> > with 2 machines and 3 btrfs file-systems. All machines run Linux 4.18.9.
> > Machine 1 runs btrfs-progs 4.17.1, machine 2 runs btrfs-progs 4.17 (via
> > https://packages.debian.org/stretch-backports/btrfs-progs).
> >
> > 1) Machine 1 takes regular snapshots and sends them to machine 2. btrfs
> >    btrfs send ... | ssh user@machine2 "btrfs receive /path1"
> > 2) Machine 2 backups all subvolumes stored at /path1 to a second
> >    independent btrfs filesystem. Let /path1/rootsnapshot be the first
> >    snapshot stored at /path1 (ie. it has no Parent UUID). Let
> >    /path1/incrementalsnapshot be a snapshot that has /path1/rootsnapshot
> >    as a parent. Then
> >    btrfs send -v /path1/rootsnapshot | btrfs receive /path2
> >    works without issues, but
> >    btrfs send -v -p /path1/rootsnapshot /path1/incrementalsnapshot | btrfs receive /path2

-v is useless. Use -vv, which will dump all commands.

> >    fails as follows:
> >    ERROR: chown o257-4639416-0 failed: No such file or directory
> >
> > No error is shown in dmesg. /path1 and /path2 denote two independent
> > btrfs filesystems.
> >
> > Note that there was no issue with transferring incrementalsnapshot from
> > machine 1 to machine 2. No error is shown in dmesg.
> >
> > Best regards
> > Leonard



-- 
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”

Re: btrfs send receive ERROR: chown failed: No such file or directory

[Leonard Lausen]

Hey Filipe,

thanks for the feedback. I ran the command again with -vv. Below are
the last commands logged by btrfs receive to stderr:

  mkfile o2138798-5016457-0
  rename leonard/mail/lists/emacs-orgmode/new/1530428589.M675528862P21583Q6R28ec1af3.leonard-xps13 -> o2138802-5207521-0
  rename o2138798-5016457-0 -> leonard/mail/lists/emacs-orgmode/new/1530428589.M675528862P21583Q6R28ec1af3.leonard-xps13
  utimes leonard/mail/lists/emacs-orgmode/new
  ERROR: cannot open /mnt/_btrbk_backups/leonard-xps13/@home.20180902T0045/o2138798-5016457-0: No such file or directory

The file which is mentioned above has in fact not changed between the
parent and current snapshot. I computed further a hash of the file on
source volume for parent and current snapshot and target on target
volume for parent snapshot. I get the same hash value in all 3 cases, so
the parent snapshot was apparently transferred correctly.

Note that the folder in which this file is placed contains 96558 files
whose sizes sum to 774M.

I further observed that the snapshot that previously failed to transfer
via send receive could now be sent / received without error. However,
the error then occured again for the next snapshot (which has the now
successfully transferred one as parent). There may be a race condition
that leads to non-deterministic failures.

Please let me know if I can help with any further information.

Best regards
Leonard

Filipe Manana <fdmanana@gmail.com> writes:

>> > 1) Machine 1 takes regular snapshots and sends them to machine 2. btrfs
>> >    btrfs send ... | ssh user@machine2 "btrfs receive /path1"
>> > 2) Machine 2 backups all subvolumes stored at /path1 to a second
>> >    independent btrfs filesystem. Let /path1/rootsnapshot be the first
>> >    snapshot stored at /path1 (ie. it has no Parent UUID). Let
>> >    /path1/incrementalsnapshot be a snapshot that has /path1/rootsnapshot
>> >    as a parent. Then
>> >    btrfs send -v /path1/rootsnapshot | btrfs receive /path2
>> >    works without issues, but
>> >    btrfs send -v -p /path1/rootsnapshot /path1/incrementalsnapshot | btrfs receive /path2
>
> -v is useless. Use -vv, which will dump all commands.
>
>> >    fails as follows:
>> >    ERROR: chown o257-4639416-0 failed: No such file or directory
>> >
>> > No error is shown in dmesg. /path1 and /path2 denote two independent
>> > btrfs filesystems.
>> >
>> > Note that there was no issue with transferring incrementalsnapshot from
>> > machine 1 to machine 2. No error is shown in dmesg.