ERROR: parent determination failed (btrfs send-receive)

ERROR: parent determination failed (btrfs send-receive)

[Dave]

Sometimes when using btrfs send-receive, I get errors like this:

ERROR: parent determination failed for <subvolume name>

When this happens, btrfs send-receive backups fail. And all subsequent
backups fail too.

The issue seems to stem from the fact that an automated cleanup
process removes certain earlier subvolumes. (I'm using Snapper.)

I'd like to understand exactly what is happening so that my backups do
not unexpectedly fail.

In my scenario, no parent subvolumes have been deleted from the
target. Some subvolumes have been deleted from the source, but why
does that matter? I am able to take a valid snapshot at this time and
every snapshot ever taken continues to reside at the target backup
destination (seemingly meaning that a parent subvolume can be found at
the target).

This issue seems to make btrfs send-receive a very fragile backup
solution. I hope, instead, there is some knowledge I'm missing, that
when learned, will make this a robust backup solution.

Thanks

Re: ERROR: parent determination failed (btrfs send-receive)

[Andrei Borzenkov]

18.09.2017 05:31, Dave пишет:
> Sometimes when using btrfs send-receive, I get errors like this:
> 
> ERROR: parent determination failed for <subvolume name>
> 
> When this happens, btrfs send-receive backups fail. And all subsequent
> backups fail too.
> 
> The issue seems to stem from the fact that an automated cleanup
> process removes certain earlier subvolumes. (I'm using Snapper.)
> 
> I'd like to understand exactly what is happening so that my backups do
> not unexpectedly fail.
> 

You try to send incremental changes but you deleted subvolume to compute
changes against. It is hard to tell more without seeing subvolume list
with uuid/parent uuid.

> In my scenario, no parent subvolumes have been deleted from the
> target. Some subvolumes have been deleted from the source, but why
> does that matter? I am able to take a valid snapshot at this time and
> every snapshot ever taken continues to reside at the target backup
> destination (seemingly meaning that a parent subvolume can be found at
> the target).
> 
> This issue seems to make btrfs send-receive a very fragile backup
> solution. 

btrfs send/receive is not backup solution - it is low level tool that
does exactly what it is told to do. You may create backup solution that
is using btrfs send/receive to transfer data stream, but then do not
blame tool for incorrect usage.

To give better advice how to fix your situation you need to describe
your backup solution - how exactly you select/create snapshots.

I hope, instead, there is some knowledge I'm missing, that
> when learned, will make this a robust backup solution.
> 
> Thanks
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: ERROR: parent determination failed (btrfs send-receive)

[Dave]

On Mon, Sep 18, 2017 at 12:23 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> 18.09.2017 05:31, Dave пишет:
>> Sometimes when using btrfs send-receive, I get errors like this:
>>
>> ERROR: parent determination failed for <subvolume name>
>>
>> When this happens, btrfs send-receive backups fail. And all subsequent
>> backups fail too.
>>
>> The issue seems to stem from the fact that an automated cleanup
>> process removes certain earlier subvolumes. (I'm using Snapper.)
>>
>> I'd like to understand exactly what is happening so that my backups do
>> not unexpectedly fail.
>>
>
> You try to send incremental changes but you deleted subvolume to compute
> changes against. It is hard to tell more without seeing subvolume list
> with uuid/parent uuid.

I do not have a current subvolume list to provide UUID's. To ensure
integrity of my backups, I was forced to delete all backup snapshots
and start over. (After this initial parent determination error, my
attempted solution created a new problem related to Received UUID's,
so removing all backups was the best solution in the end.)

For my understanding, what are the restrictions on deleting snapshots?

What scenarios can lead to "ERROR: parent determination failed"?

After all, it should not be necessary to retain every snapshot ever
made. We have to delete snapshots periodically.

In my case, I still retained every snapshot on the target volume. None
had ever been deleted (yet). And I also retained around 30 recent
snapshots on the source, according to the snapper timeline cleanup
config. Yet I still ran into "ERROR: parent determination failed".

>
>> In my scenario, no parent subvolumes have been deleted from the
>> target. Some subvolumes have been deleted from the source, but why
>> does that matter? I am able to take a valid snapshot at this time and
>> every snapshot ever taken continues to reside at the target backup
>> destination (seemingly meaning that a parent subvolume can be found at
>> the target).
>>
>> This issue seems to make btrfs send-receive a very fragile backup
>> solution.
>
> btrfs send/receive is not backup solution - it is low level tool that
> does exactly what it is told to do. You may create backup solution that
> is using btrfs send/receive to transfer data stream, but then do not
> blame tool for incorrect usage.
>
> To give better advice how to fix your situation you need to describe
> your backup solution - how exactly you select/create snapshots.

I use snap-sync to create and send snapshots.

GitHub - wesbarnett/snap-sync: Use snapper snapshots to backup to external drive
https://github.com/wesbarnett/snap-sync

>
> I hope, instead, there is some knowledge I'm missing, that
>> when learned, will make this a robust backup solution.
>>
>> Thanks
>> --

Re: ERROR: parent determination failed (btrfs send-receive)

[Graham Cobb]

On 18/09/17 07:10, Dave wrote:
> For my understanding, what are the restrictions on deleting snapshots?
> 
> What scenarios can lead to "ERROR: parent determination failed"?

The man page for btrfs-send is reasonably clear on the requirements
btrfs imposes. If you want to use incremental sends (i.e. the -c or -p
options) then the specified snapshots must exist on both the source and
destination. If you don't have a suitable existing snapshot then don't
use -c or -p and just do a full send.

> I use snap-sync to create and send snapshots.
> 
> GitHub - wesbarnett/snap-sync: Use snapper snapshots to backup to external drive
> https://github.com/wesbarnett/snap-sync

I am not familiar with this tool. Your question should be sent to the
author of the tool, if that is what is deciding what -p and -c options
are being used.

Personally I use and recommend btrbk. I have never had this issue and
the configuration options let me limit the snapshots it saves on both
the source and destination disks separately (so I keep fewer on the
source than on the backup disk).

Graham

Re: ERROR: parent determination failed (btrfs send-receive)

[Andrei Borzenkov]

18.09.2017 11:45, Graham Cobb пишет:
> On 18/09/17 07:10, Dave wrote:
>> For my understanding, what are the restrictions on deleting snapshots?
>>
>> What scenarios can lead to "ERROR: parent determination failed"?
> 
> The man page for btrfs-send is reasonably clear on the requirements
> btrfs imposes. If you want to use incremental sends (i.e. the -c or -p
> options) then the specified snapshots must exist on both the source and
> destination. If you don't have a suitable existing snapshot then don't
> use -c or -p and just do a full send.
> 

Well, I do not immediately see why -c must imply incremental send. We
want to reduce amount of data that is transferred, so reuse data from
existing snapshots, but it is really orthogonal to whether we send full
subvolume or just changes since another snapshot.

>> I use snap-sync to create and send snapshots.
>>
>> GitHub - wesbarnett/snap-sync: Use snapper snapshots to backup to external drive
>> https://github.com/wesbarnett/snap-sync
> 
> I am not familiar with this tool. Your question should be sent to the
> author of the tool, if that is what is deciding what -p and -c options
> are being used.
> 

I am not sure how it could come to this error. I looked on more or less
default installation of openSUSE here and all snapper snapshots have as
parent UUID the subvolume that is mounted as root (by default only one
configuration for root subvolume exists). So it is not possible to
remove this subvolume, unless some rollback to another snapshot was
performed.

Re: ERROR: parent determination failed (btrfs send-receive)

[Andrei Borzenkov]

18.09.2017 09:10, Dave пишет:
> I use snap-sync to create and send snapshots.
> 
> GitHub - wesbarnett/snap-sync: Use snapper snapshots to backup to external drive
> https://github.com/wesbarnett/snap-sync
> 

Are you trying to backup top-level subvolume? I just reproduced this
behavior with this tool. The problem is, snapshots of top-level
subvolume do not have parent UUID (I am not even sure if UUID exists at
all TBH). If you mount any other subvolume, it will work. On openSUSE
root is always mounted as subvolume (actually, the very first snapshot)
which explains why I did not see it before.

I.e.

mkfs -t btrfs /dev/sdb1
mount /dev/sdb1 /test
snapper -c test create-config /test

attempt to "snap-sync -c test" will fail second time. But

btrfs sub create /test/@
umount /test
mount -o subvol=@ /dev/sdb1 /test
snapper -c test create-config /test
...

will work.

As I told you in the first reply, showing output of "btrfs su li -qu
/path/to/src" would explain your problem much earlier.

Actually if snap-sync used "btrfs send -p" instead of "btrfs send -c" it
would work as well, as then no parent search would be needed (and as I
mentioned in another mail both commands are functionally equivalent).
But this becomes really off-topic on this list. As already suggested,
open issue for snap-sync.

Re: ERROR: parent determination failed (btrfs send-receive)

[Dave]

On Tue, Sep 19, 2017 at 3:37 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> 18.09.2017 09:10, Dave пишет:
>> I use snap-sync to create and send snapshots.
>>
>> GitHub - wesbarnett/snap-sync: Use snapper snapshots to backup to external drive
>> https://github.com/wesbarnett/snap-sync
>>
>
> Are you trying to backup top-level subvolume?

No. I never mount the top-level subvolume.

>  I just reproduced this > behavior with this tool. The problem is, snapshots of top-level
> subvolume do not have parent UUID (I am not even sure if UUID exists at
> all TBH). If you mount any other subvolume, it will work.

Well, that's not exactly my issue but it still helps me gain
additional understanding.  It may help me reproduce my original
problem too. Thank you.

>
>
> As I told you in the first reply, showing output of "btrfs su li -qu
> /path/to/src" would explain your problem much earlier.
>
> Actually if snap-sync used "btrfs send -p" instead of "btrfs send -c" it
> would work as well, as then no parent search would be needed (and as I
> mentioned in another mail both commands are functionally equivalent).
> But this becomes really off-topic on this list. As already suggested,
> open issue for snap-sync.

Thank you for this also. I will simply edit my current copy of the
snap-sync script to use "-p" now. That's a simple change I am
implement immediately.

Re: ERROR: parent determination failed (btrfs send-receive)

[Dave]

On Mon, Sep 18, 2017 at 12:23 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
>
> 18.09.2017 05:31, Dave пишет:
> > Sometimes when using btrfs send-receive, I get errors like this:
> >
> > ERROR: parent determination failed for <subvolume name>
> >
> > When this happens, btrfs send-receive backups fail. And all subsequent
> > backups fail too.
> >
> > The issue seems to stem from the fact that an automated cleanup
> > process removes certain earlier subvolumes. (I'm using Snapper.)
> >
> > I'd like to understand exactly what is happening so that my backups do
> > not unexpectedly fail.
> >
>
> You try to send incremental changes but you deleted subvolume to compute
> changes against. It is hard to tell more without seeing subvolume list
> with uuid/parent uuid.
>

Thanks to Duncan <1i5t5.duncan@cox.net> I have a bit better
understanding of -c and -p now.

My backup tool is using only the -c option. (The tool is GitHub -
wesbarnett/snap-sync https://github.com/wesbarnett/snap-sync .)

No subvolume at the destination had ever been deleted.

At the source, a number (about 30 in this case) preceding snapshots
(subvolumes) exist, but some others have been cleaned up with
Snapper's timeline algorithm.

I think I understand that with the -c option, it is **not** strictly
necessary that the specified snapshots exist on both the source and
destination. It seems I had sufficient subvolumes available for the
incremental send to work with the -c option.

Therefore, it still isn't clear why I got the error: ERROR: parent
determination failed.

Further general comments will be helpful.

I understand the limits in getting too specific in replies because I
cannot give subvolume lists with uuid's.

As mentioned, I cannot give that info because I tried to fix the above
error by sending a subvolume from the destination back to the target.
This resulted in my source volume having a "Received UUID" which
wrecked all subsequent backups.

I now understand that (for my use cases) a source subvolume for a
send-receive should **never** have a Received UUID. (If that is indeed
a general rule, it seems btrfs tools should check it. Or possibly
something about this the pitfalls of a "Received UUID" in a source
could be listed on the BTRFS incremental backup wiki page?)

I was previously referred to the FAQ here:
https://github.com/digint/btrbk/blob/master/doc/FAQ.md#im-getting-an-error-aborted-received-uuid-is-set

But in the end, I decided the safest strategy was to delete all prior
backup subvolumes so I could be sure my backups were valid going
forward.

I am asking these questions now to avoid getting into a situation like
this again (hopefully). Any general comments are appreciated.