Inconsistence between sender and receiver

Inconsistence between sender and receiver

[Liu Bo]

Hi,

In the following steps[1], if <parent> on receiver side has got
changed via 'btrfs property set', then after doing incremental
updates, receiver gets a different snapshot from what sender has sent.

The reason behind it is that there is no change about file 'foo' in
the send stream, such that receiver simply creates a snapshot of
<parent> on its side with nothing to apply from the send stream.

A possible way to avoid this is to check rtransid and ctranid of
<parent> on receiver side, but I'm not very sure whether the current
behavior is made deliberately, does anyone have an idea? 

Thanks,

-liubo

[1]:
$ btrfs sub create /mnt/send/sub
$ touch /mnt/send/sub/foo
$ btrfs sub snap -r /mnt/send/sub /mnt/send/parent

# send parent out
$ btrfs send /mnt/send/parent | btrfs receive /mnt/recv/

# change parent and file under it
$ btrfs property set -t subvol /mnt/recv/parent ro false
$ truncate -s 4096 /mnt/recv/parent/foo

$ btrfs sub snap -r /mnt/send/sub /mnt/send/update
$ btrfs send -p /mnt/send/parent /mnt/send/update | btrfs receive /mnt/recv

$ ls -l /mnt/send/update
total 0
-rw-r--r-- 1 root root 0 Mar  6 11:13 foo

$ ls -l /mnt/recv/update
total 0
-rw-r--r-- 1 root root 4096 Mar  6 11:14 foo

Re: Inconsistence between sender and receiver

[Andrei Borzenkov]

07.03.2018 21:49, Liu Bo пишет:
> Hi,
> 
> In the following steps[1], if <parent> on receiver side has got
> changed via 'btrfs property set', then after doing incremental
> updates, receiver gets a different snapshot from what sender has sent.
> 
> The reason behind it is that there is no change about file 'foo' in
> the send stream, such that receiver simply creates a snapshot of
> <parent> on its side with nothing to apply from the send stream.
> 
> A possible way to avoid this is to check rtransid and ctranid of
> <parent> on receiver side, but I'm not very sure whether the current
> behavior is made deliberately, does anyone have an idea? 
> 
> Thanks,
> 
> -liubo
> 
> [1]:
> $ btrfs sub create /mnt/send/sub
> $ touch /mnt/send/sub/foo
> $ btrfs sub snap -r /mnt/send/sub /mnt/send/parent
> 
> # send parent out
> $ btrfs send /mnt/send/parent | btrfs receive /mnt/recv/
> 
> # change parent and file under it
> $ btrfs property set -t subvol /mnt/recv/parent ro false

Is removing the ability to modify read-only property an option? What are
use cases for it? What can it do that "btrfs sub snap read-only
writable" cannot?

> $ truncate -s 4096 /mnt/recv/parent/foo
> 
> $ btrfs sub snap -r /mnt/send/sub /mnt/send/update
> $ btrfs send -p /mnt/send/parent /mnt/send/update | btrfs receive /mnt/recv
> 

This should fail right away because /mnt/send/parent is not read-only.
If it does not, this is really a bug.

Of course one may go one step further and set /mnt/send/parent read-only
again, then we get this issue.

> $ ls -l /mnt/send/update
> total 0
> -rw-r--r-- 1 root root 0 Mar  6 11:13 foo
> 
> $ ls -l /mnt/recv/update
> total 0
> -rw-r--r-- 1 root root 4096 Mar  6 11:14 foo
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Inconsistence between sender and receiver

[Filipe Manana]

On Wed, Mar 7, 2018 at 6:49 PM, Liu Bo <bo.li.liu@oracle.com> wrote:
> Hi,
>
> In the following steps[1], if <parent> on receiver side has got
> changed via 'btrfs property set', then after doing incremental
> updates, receiver gets a different snapshot from what sender has sent.
>
> The reason behind it is that there is no change about file 'foo' in
> the send stream, such that receiver simply creates a snapshot of
> <parent> on its side with nothing to apply from the send stream.
>
> A possible way to avoid this is to check rtransid and ctranid of
> <parent> on receiver side, but I'm not very sure whether the current
> behavior is made deliberately, does anyone have an idea?
>
> Thanks,
>
> -liubo
>
> [1]:
> $ btrfs sub create /mnt/send/sub
> $ touch /mnt/send/sub/foo
> $ btrfs sub snap -r /mnt/send/sub /mnt/send/parent
>
> # send parent out
> $ btrfs send /mnt/send/parent | btrfs receive /mnt/recv/
>
> # change parent and file under it
> $ btrfs property set -t subvol /mnt/recv/parent ro false
> $ truncate -s 4096 /mnt/recv/parent/foo
>
> $ btrfs sub snap -r /mnt/send/sub /mnt/send/update
> $ btrfs send -p /mnt/send/parent /mnt/send/update | btrfs receive /mnt/recv
>
> $ ls -l /mnt/send/update
> total 0
> -rw-r--r-- 1 root root 0 Mar  6 11:13 foo
>
> $ ls -l /mnt/recv/update
> total 0
> -rw-r--r-- 1 root root 4096 Mar  6 11:14 foo

Interesting. I wasn't aware of that and I'm surprised too, I wouldn't
expect that behaviour (and I must see the code to figure out why it
behaves like that).
No idea if it's intentional or not, but at a first glance it seems not.

>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”

Re: Inconsistence between sender and receiver

[Liu Bo]

On Thu, Mar 08, 2018 at 09:15:50AM +0300, Andrei Borzenkov wrote:
> 07.03.2018 21:49, Liu Bo пишет:
> > Hi,
> > 
> > In the following steps[1], if <parent> on receiver side has got
> > changed via 'btrfs property set', then after doing incremental
> > updates, receiver gets a different snapshot from what sender has sent.
> > 
> > The reason behind it is that there is no change about file 'foo' in
> > the send stream, such that receiver simply creates a snapshot of
> > <parent> on its side with nothing to apply from the send stream.
> > 
> > A possible way to avoid this is to check rtransid and ctranid of
> > <parent> on receiver side, but I'm not very sure whether the current
> > behavior is made deliberately, does anyone have an idea? 
> > 
> > Thanks,
> > 
> > -liubo
> > 
> > [1]:
> > $ btrfs sub create /mnt/send/sub
> > $ touch /mnt/send/sub/foo
> > $ btrfs sub snap -r /mnt/send/sub /mnt/send/parent
> > 
> > # send parent out
> > $ btrfs send /mnt/send/parent | btrfs receive /mnt/recv/
> > 
> > # change parent and file under it
> > $ btrfs property set -t subvol /mnt/recv/parent ro false
> 
> Is removing the ability to modify read-only property an option? What are
> use cases for it? What can it do that "btrfs sub snap read-only
> writable" cannot?
>

Tbh, I don't know any usecase for that, I just wanted to toggle off
the ro bit in order to change something inside <parent>.

> > $ truncate -s 4096 /mnt/recv/parent/foo
> > 
> > $ btrfs sub snap -r /mnt/send/sub /mnt/send/update
> > $ btrfs send -p /mnt/send/parent /mnt/send/update | btrfs receive /mnt/recv
> > 
> 
> This should fail right away because /mnt/send/parent is not read-only.
> If it does not, this is really a bug.
>

It's not '/mnt/send/parent', but '/mnt/recv/parent' got changed.

Thanks,

-liubo

> Of course one may go one step further and set /mnt/send/parent read-only
> again, then we get this issue.
> 
> > $ ls -l /mnt/send/update
> > total 0
> > -rw-r--r-- 1 root root 0 Mar  6 11:13 foo
> > 
> > $ ls -l /mnt/recv/update
> > total 0
> > -rw-r--r-- 1 root root 4096 Mar  6 11:14 foo
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> 

Re: Inconsistence between sender and receiver

[Liu Bo]

On Thu, Mar 08, 2018 at 03:29:48PM +0000, Filipe Manana wrote:
> On Wed, Mar 7, 2018 at 6:49 PM, Liu Bo <bo.li.liu@oracle.com> wrote:
> > Hi,
> >
> > In the following steps[1], if <parent> on receiver side has got
> > changed via 'btrfs property set', then after doing incremental
> > updates, receiver gets a different snapshot from what sender has sent.
> >
> > The reason behind it is that there is no change about file 'foo' in
> > the send stream, such that receiver simply creates a snapshot of
> > <parent> on its side with nothing to apply from the send stream.
> >
> > A possible way to avoid this is to check rtransid and ctranid of
> > <parent> on receiver side, but I'm not very sure whether the current
> > behavior is made deliberately, does anyone have an idea?
> >
> > Thanks,
> >
> > -liubo
> >
> > [1]:
> > $ btrfs sub create /mnt/send/sub
> > $ touch /mnt/send/sub/foo
> > $ btrfs sub snap -r /mnt/send/sub /mnt/send/parent
> >
> > # send parent out
> > $ btrfs send /mnt/send/parent | btrfs receive /mnt/recv/
> >
> > # change parent and file under it
> > $ btrfs property set -t subvol /mnt/recv/parent ro false
> > $ truncate -s 4096 /mnt/recv/parent/foo
> >
> > $ btrfs sub snap -r /mnt/send/sub /mnt/send/update
> > $ btrfs send -p /mnt/send/parent /mnt/send/update | btrfs receive /mnt/recv
> >
> > $ ls -l /mnt/send/update
> > total 0
> > -rw-r--r-- 1 root root 0 Mar  6 11:13 foo
> >
> > $ ls -l /mnt/recv/update
> > total 0
> > -rw-r--r-- 1 root root 4096 Mar  6 11:14 foo
> 
> Interesting. I wasn't aware of that and I'm surprised too, I wouldn't
> expect that behaviour (and I must see the code to figure out why it
> behaves like that).

The code about comparing rtransid and ctransid does exist in
cmds-receive.c, but had been commented.

Thanks,

-liubo

Re: Inconsistence between sender and receiver

[Andrei Borzenkov]

09.03.2018 08:38, Liu Bo пишет:
> On Thu, Mar 08, 2018 at 09:15:50AM +0300, Andrei Borzenkov wrote:
>> 07.03.2018 21:49, Liu Bo пишет:
>>> Hi,
>>>
>>> In the following steps[1], if <parent> on receiver side has got
>>> changed via 'btrfs property set', then after doing incremental
>>> updates, receiver gets a different snapshot from what sender has sent.
>>>
>>> The reason behind it is that there is no change about file 'foo' in
>>> the send stream, such that receiver simply creates a snapshot of
>>> <parent> on its side with nothing to apply from the send stream.
>>>
>>> A possible way to avoid this is to check rtransid and ctranid of
>>> <parent> on receiver side, but I'm not very sure whether the current
>>> behavior is made deliberately, does anyone have an idea? 
>>>
>>> Thanks,
>>>
>>> -liubo
>>>
>>> [1]:
>>> $ btrfs sub create /mnt/send/sub
>>> $ touch /mnt/send/sub/foo
>>> $ btrfs sub snap -r /mnt/send/sub /mnt/send/parent
>>>
>>> # send parent out
>>> $ btrfs send /mnt/send/parent | btrfs receive /mnt/recv/
>>>
>>> # change parent and file under it
>>> $ btrfs property set -t subvol /mnt/recv/parent ro false
>>
>> Is removing the ability to modify read-only property an option? What are
>> use cases for it? What can it do that "btrfs sub snap read-only
>> writable" cannot?
>>
> 
> Tbh, I don't know any usecase for that, I just wanted to toggle off
> the ro bit in order to change something inside <parent>.
> 
>>> $ truncate -s 4096 /mnt/recv/parent/foo
>>>
>>> $ btrfs sub snap -r /mnt/send/sub /mnt/send/update
>>> $ btrfs send -p /mnt/send/parent /mnt/send/update | btrfs receive /mnt/recv
>>>
>>
>> This should fail right away because /mnt/send/parent is not read-only.
>> If it does not, this is really a bug.
>>
> 
> It's not '/mnt/send/parent', but '/mnt/recv/parent' got changed.
> 

Ah, sorry.

What happened to this patch which clears received_uuid when ro is
flipped off?

https://patchwork.kernel.org/patch/9986521/

Re: Inconsistence between sender and receiver

[David Sterba]

On Fri, Mar 09, 2018 at 09:45:30AM +0300, Andrei Borzenkov wrote:
> 09.03.2018 08:38, Liu Bo пишет:
> > On Thu, Mar 08, 2018 at 09:15:50AM +0300, Andrei Borzenkov wrote:
> >> 07.03.2018 21:49, Liu Bo пишет:
> >>> Hi,
> >>>
> >>> In the following steps[1], if <parent> on receiver side has got
> >>> changed via 'btrfs property set', then after doing incremental
> >>> updates, receiver gets a different snapshot from what sender has sent.
> >>>
> >>> The reason behind it is that there is no change about file 'foo' in
> >>> the send stream, such that receiver simply creates a snapshot of
> >>> <parent> on its side with nothing to apply from the send stream.
> >>>
> >>> A possible way to avoid this is to check rtransid and ctranid of
> >>> <parent> on receiver side, but I'm not very sure whether the current
> >>> behavior is made deliberately, does anyone have an idea? 
> >>>
> >>> Thanks,
> >>>
> >>> -liubo
> >>>
> >>> [1]:
> >>> $ btrfs sub create /mnt/send/sub
> >>> $ touch /mnt/send/sub/foo
> >>> $ btrfs sub snap -r /mnt/send/sub /mnt/send/parent
> >>>
> >>> # send parent out
> >>> $ btrfs send /mnt/send/parent | btrfs receive /mnt/recv/
> >>>
> >>> # change parent and file under it
> >>> $ btrfs property set -t subvol /mnt/recv/parent ro false
> >>
> >> Is removing the ability to modify read-only property an option? What are
> >> use cases for it? What can it do that "btrfs sub snap read-only
> >> writable" cannot?
> >>
> > 
> > Tbh, I don't know any usecase for that, I just wanted to toggle off
> > the ro bit in order to change something inside <parent>.
> > 
> >>> $ truncate -s 4096 /mnt/recv/parent/foo
> >>>
> >>> $ btrfs sub snap -r /mnt/send/sub /mnt/send/update
> >>> $ btrfs send -p /mnt/send/parent /mnt/send/update | btrfs receive /mnt/recv
> >>>
> >>
> >> This should fail right away because /mnt/send/parent is not read-only.
> >> If it does not, this is really a bug.
> >>
> > 
> > It's not '/mnt/send/parent', but '/mnt/recv/parent' got changed.
> > 
> 
> Ah, sorry.
> 
> What happened to this patch which clears received_uuid when ro is
> flipped off?
> 
> https://patchwork.kernel.org/patch/9986521/

There are still things to fix:

"Shouldn't we also wipe the other related fields here, like stime, rtime,
stransid, rtransid?"