Re: [PATCH v2] overlay: allow config override of metacopy/redirect defaults

[Vivek Goyal <vgoyal@redhat.com>]

On Tue, Jun 11, 2019 at 02:37:34PM +0200, Miklos Szeredi wrote:
> On Mon, Jun 10, 2019 at 8:45 PM Vivek Goyal <vgoyal@redhat.com> wrote:
> >
> > On Mon, Jun 10, 2019 at 02:40:43PM -0400, Vivek Goyal wrote:
> > > On Sun, Jun 09, 2019 at 09:14:38PM +0200, Miklos Szeredi wrote:
> > > > On Sat, Jun 8, 2019 at 8:47 PM Amir Goldstein <amir73il@gmail.com> wrote:
> > > >
> > > > > And then every time that a feature needs to be turned off for some reason
> > > > > that also needs to be taken into account.
> > > > > IOW, I advise against diving into this mess. You have been warned ;-)
> > > >
> > > > Also a much more productive direction would be to optimize building
> > > > the docker image based on the specific format used by overlayfs for
> > > > readirect_dir/metacopy.
> > > >
> > > > To me it seems like a no-brainer, but I don't know much about docker, so...
> > >
> > > [ cc Daniel Walsh]
> > >
> > > Hi Miklos,
> > >
> > > Can you elaborate a bit more on what docker/container-storoage and do
> > > here to expedite image generation with redirect_dir/metacopy enabled.
> > >
> > > They can't pack these xattrs in image because image will not be portable.
> > > It will be overlayfs specific and can't be made to work on target without
> > > overlayfs.
> >
> > Are you referring to apps being able to traverse lower layers and do
> > the redirect_dir and metacopy resoltion as kernel does. To me that process
> > is not trivial. Having a library might help with adoption though.
> 
> AFAICS what happens when generating a layer is to start with a clean
> upper layer, do some operations, then save the contents of the upper
> layer.  If redirect or metacopy is enabled, then the contents of the
> upper layer won't be portable.  So need to do something like this:
> 
> traverse(overlay_dir, upper_dir, target_dir)
> {
>     iterate name for entries in $upper_dir {
>         if ($name is non-directory) {
>             copy($overlay_dir/$name, $target_dir/$name)
>         } else if ($name is redirect) {
>             copy-recursive($overlay_dir/$name, $target_dir/$name)
>         } else {
>             copy($overlay_dir/$name, $target_dir/$name)
>             traverse($overlay_dir/$name, $upper_dir/$name, $target_dir/$name)
>         }
>     }
> }
> 
> Basically: traverse the *upper layer* but copy files and directories
> from the *overlay*.  Does that make sense?

[ cc nalin ]

Aha... This makes sense to me. This does away with need of separate
library or user space tool and hopefully its faster than naivediff
interface. 

Dan, Nalin, what do you think about above idea.

Thanks
Vivek