RE: [EXTERNAL] Re: Copy_up function fail with ubifs on upper layer

RE: [EXTERNAL] Re: Copy_up function fail with ubifs on upper layer

[Guerard, Vincent]

Here's the output when overlayfs try to do a copy up with the original function calls order and with debug enabled:

root@io2200:/etc# mv machine-id machine-id1
mv: cannot move 'machine-id' to 'machine-id1': No such file or directory
root@io2200:/etc# dmesg
[ 2999.275275] tmpfile(upper.work/work, 0100644) = 0
[ 2999.278789] setxattr(work/#232, "trusted.overlay.origin", "(null)", 0x0) = -2

And here's with inverted function calls:
root@io2200:/etc# mv securetty securetty1
root@io2200:/etc# dmesg
[  659.515200] tmpfile(upper.work/work, 0100400) = 0
[  659.515422] link(work/#164, etc/securetty) = 0
[  659.518577] setxattr(work/#164, "trusted.overlay.origin", "(null)", 0x0) = 0
[  659.522153] rename(etc/securetty, etc/securetty1, 0x4)

Thanks,
Vincent Guérard


-----Original Message-----
From: Miklos Szeredi [mailto:miklos@szeredi.hu] 
Sent: December 10, 2018 4:27 AM
To: Guerard, Vincent
Subject: [EXTERNAL] Re: Copy_up function fail with ubifs on upper layer

On Mon, Dec 3, 2018 at 8:51 PM Guerard, Vincent <VincentGuerard@eaton.com> wrote:
>
> Hello,
>
>
>
> I’m using the Linux kernel 4.14 of Texas Instruments, which have this 
> patch in the ubifs file system code: 
> https://lore.kernel.org/patchwork/patch/960502/
>
> I’m using a ubifs file system as the upper layer of an overlay, so when overlayfs try to do a copy up of a file, it fail with ENOENT.

Could you please enable debugging with:

  echo "file fs/overlayfs/* +p" > <debugfs>/dynamic_debug/control

?

Also would you mind adding <linux-unionfs@vger.kernel.org> to the discussion?

>
> The workaround I found was to invert the call  of copy_up_inode and ovl_install_temp in the function ovl_copy_up_locked.
>
>
>
> I’ve run the basic overlayfs tests which I found at : https://github.com/amir73il/overlayfs/wiki/Overlayfs-testing and all tests passed.
>
>
>
> So my questions are :
>
> Is it okay to invert these call in this case?

Inverting the calls make the copy-up operation non-atomic.   This
shouldn't normally be an issue, only when e.g. power is lost in the middle of the operation and after reboot the overlay will be corrupted.

Thanks,
Miklos

Re: [EXTERNAL] Re: Copy_up function fail with ubifs on upper layer

[Amir Goldstein]

On Mon, Dec 10, 2018 at 10:02 PM Guerard, Vincent
<VincentGuerard@eaton.com> wrote:
>
> Here's the output when overlayfs try to do a copy up with the original function calls order and with debug enabled:
>
> root@io2200:/etc# mv machine-id machine-id1
> mv: cannot move 'machine-id' to 'machine-id1': No such file or directory
> root@io2200:/etc# dmesg
> [ 2999.275275] tmpfile(upper.work/work, 0100644) = 0
> [ 2999.278789] setxattr(work/#232, "trusted.overlay.origin", "(null)", 0x0) = -2
>
> And here's with inverted function calls:
> root@io2200:/etc# mv securetty securetty1
> root@io2200:/etc# dmesg
> [  659.515200] tmpfile(upper.work/work, 0100400) = 0
> [  659.515422] link(work/#164, etc/securetty) = 0
> [  659.518577] setxattr(work/#164, "trusted.overlay.origin", "(null)", 0x0) = 0
> [  659.522153] rename(etc/securetty, etc/securetty1, 0x4)
>

Maybe this is the patch you need:
https://marc.info/?l=linux-unionfs&m=154070087430992&w=2

Not sure if it is upstream already.

Thanks,
Amir.

Re: [EXTERNAL] Re: Copy_up function fail with ubifs on upper layer

[Richard Weinberger]

Am Montag, 10. Dezember 2018, 21:37:46 CET schrieb Amir Goldstein:
> On Mon, Dec 10, 2018 at 10:02 PM Guerard, Vincent
> <VincentGuerard@eaton.com> wrote:
> >
> > Here's the output when overlayfs try to do a copy up with the original function calls order and with debug enabled:
> >
> > root@io2200:/etc# mv machine-id machine-id1
> > mv: cannot move 'machine-id' to 'machine-id1': No such file or directory
> > root@io2200:/etc# dmesg
> > [ 2999.275275] tmpfile(upper.work/work, 0100644) = 0
> > [ 2999.278789] setxattr(work/#232, "trusted.overlay.origin", "(null)", 0x0) = -2
> >
> > And here's with inverted function calls:
> > root@io2200:/etc# mv securetty securetty1
> > root@io2200:/etc# dmesg
> > [  659.515200] tmpfile(upper.work/work, 0100400) = 0
> > [  659.515422] link(work/#164, etc/securetty) = 0
> > [  659.518577] setxattr(work/#164, "trusted.overlay.origin", "(null)", 0x0) = 0
> > [  659.522153] rename(etc/securetty, etc/securetty1, 0x4)
> >
> 
> Maybe this is the patch you need:
> https://marc.info/?l=linux-unionfs&m=154070087430992&w=2

Hmm, this fixes something different.
Vincent, do you trigger this every time or just once?
Is a power-cut involved?

Long story short, I need more info.

Thanks,
//richard

RE: [EXTERNAL] Re: Copy_up function fail with ubifs on upper layer

[Guerard, Vincent]

Indeed the patch didn't solve the problem.

I trigger this every times.
No power cuts is involved.

Every time I try to do an operation on a file that hasn't already been copied on the upper layer, it fail when overlayfs try to set xattr. 
I tried removing the patch in ubifs which cause the fail (https://lore.kernel.org/patchwork/patch/960502/) and everything work now.
I also just found this patch: https://lore.kernel.org/patchwork/patch/986563/ , so I guess its safe to remove it.

Thanks,
Vincent

-----Original Message-----
From: Richard Weinberger [mailto:richard@nod.at] 
Sent: December 11, 2018 4:54 AM
To: Amir Goldstein
Cc: Guerard, Vincent; Miklos Szeredi; overlayfs
Subject: Re: [EXTERNAL] Re: Copy_up function fail with ubifs on upper layer

Am Montag, 10. Dezember 2018, 21:37:46 CET schrieb Amir Goldstein:
> On Mon, Dec 10, 2018 at 10:02 PM Guerard, Vincent 
> <VincentGuerard@eaton.com> wrote:
> >
> > Here's the output when overlayfs try to do a copy up with the original function calls order and with debug enabled:
> >
> > root@io2200:/etc# mv machine-id machine-id1
> > mv: cannot move 'machine-id' to 'machine-id1': No such file or 
> > directory root@io2200:/etc# dmesg [ 2999.275275] 
> > tmpfile(upper.work/work, 0100644) = 0 [ 2999.278789] 
> > setxattr(work/#232, "trusted.overlay.origin", "(null)", 0x0) = -2
> >
> > And here's with inverted function calls:
> > root@io2200:/etc# mv securetty securetty1 root@io2200:/etc# dmesg [  
> > 659.515200] tmpfile(upper.work/work, 0100400) = 0 [  659.515422] 
> > link(work/#164, etc/securetty) = 0 [  659.518577] 
> > setxattr(work/#164, "trusted.overlay.origin", "(null)", 0x0) = 0 [  
> > 659.522153] rename(etc/securetty, etc/securetty1, 0x4)
> >
> 
> Maybe this is the patch you need:
> https://marc.info/?l=linux-unionfs&m=154070087430992&w=2

Hmm, this fixes something different.
Vincent, do you trigger this every time or just once?
Is a power-cut involved?

Long story short, I need more info.

Thanks,
//richard

Re: [EXTERNAL] Re: Copy_up function fail with ubifs on upper layer

[Richard Weinberger]

Am Dienstag, 11. Dezember 2018, 16:32:25 CET schrieb Guerard, Vincent:
> Indeed the patch didn't solve the problem.
> 
> I trigger this every times.
> No power cuts is involved.
> 
> Every time I try to do an operation on a file that hasn't already been copied on the upper layer, it fail when overlayfs try to set xattr. 
> I tried removing the patch in ubifs which cause the fail (https://lore.kernel.org/patchwork/patch/960502/) and everything work now.
> I also just found this patch: https://lore.kernel.org/patchwork/patch/986563/ , so I guess its safe to remove it.

Good, the revert is already upstream.
So, case closed. :-)

Thanks,
//richard