Optional switching off cow in overlayfs

Optional switching off cow in overlayfs

[Jan Olszak]

Hi there!
I was wondering about a small improvement to overlayfs - optional, per 
file copy of write.

1. By default overlayfs would work as usual.
2. If a file in 'lower' filesystem has an xattr set (e.g. 
trusted.overlay.cow=n) the file would not be copied.

What do you think?

Cheers,
Jan

Re: Optional switching off cow in overlayfs

[David Howells]

Jan Olszak <j.olszak@samsung.com> wrote:

> I was wondering about a small improvement to overlayfs - optional, per file
> copy of write.
> 
> 1. By default overlayfs would work as usual.
> 2. If a file in 'lower' filesystem has an xattr set
> (e.g. trusted.overlay.cow=n) the file would not be copied.
> 
> What do you think?

Ummm.  What's the use case?  How do you handle the file being part of multiple
overlays?

David

Re: Optional switching off cow in overlayfs

[Jan Olszak]

On 06/18/2015 03:39 PM, David Howells wrote:
> Jan Olszak <j.olszak@samsung.com> wrote:
>
>> I was wondering about a small improvement to overlayfs - optional, per file
>> copy of write.
>>
>> 1. By default overlayfs would work as usual.
>> 2. If a file in 'lower' filesystem has an xattr set
>> (e.g. trusted.overlay.cow=n) the file would not be copied.
>>
>> What do you think?
> Ummm.  What's the use case?  How do you handle the file being part of multiple
> overlays?
>
> David
>

I wan't to achieve 'files' granularity in overlayfs instead of 
'directory' granularity.
It would be useful in Linux containers configuration. Sometimes you 
don't have the
power to change the images that are run inside containers and in one 
folder you have both:
- files that have to be copied
- files that don't really need to be copied


trusted.overlay.cow==n indicates switching off copying the file to the 
upper layer.
So with multiple overlays you would always have only one file from the 
lower-most dir.

Jan

Re: Optional switching off cow in overlayfs

[Miklos Szeredi]

On Thu, Jun 18, 2015 at 4:43 PM, Jan Olszak <j.olszak@samsung.com> wrote:
>
>
> On 06/18/2015 03:39 PM, David Howells wrote:
>>
>> Jan Olszak <j.olszak@samsung.com> wrote:
>>
>>> I was wondering about a small improvement to overlayfs - optional, per
>>> file
>>> copy of write.
>>>
>>> 1. By default overlayfs would work as usual.
>>> 2. If a file in 'lower' filesystem has an xattr set
>>> (e.g. trusted.overlay.cow=n) the file would not be copied.
>>>
>>> What do you think?
>>
>> Ummm.  What's the use case?  How do you handle the file being part of
>> multiple
>> overlays?
>>
>> David
>>
>
> I wan't to achieve 'files' granularity in overlayfs instead of 'directory'
> granularity.
> It would be useful in Linux containers configuration. Sometimes you don't
> have the
> power to change the images that are run inside containers and in one folder
> you have both:
> - files that have to be copied
> - files that don't really need to be copied
>
>
> trusted.overlay.cow==n indicates switching off copying the file to the upper
> layer.
> So with multiple overlays you would always have only one file from the
> lower-most dir.

What type filesystem is the lowest layer?   Can you set immutable on
it (chattr +i)?  That would be the logical way to say "nobody can
modify this file, not even root".

Thanks,
Miklos

Re: Optional switching off cow in overlayfs

[Jan Olszak]

On 06/22/2015 01:32 PM, Miklos Szeredi wrote:
> On Thu, Jun 18, 2015 at 4:43 PM, Jan Olszak <j.olszak@samsung.com> wrote:
>>
>> On 06/18/2015 03:39 PM, David Howells wrote:
>>> Jan Olszak <j.olszak@samsung.com> wrote:
>>>
>>>> I was wondering about a small improvement to overlayfs - optional, per
>>>> file
>>>> copy of write.
>>>>
>>>> 1. By default overlayfs would work as usual.
>>>> 2. If a file in 'lower' filesystem has an xattr set
>>>> (e.g. trusted.overlay.cow=n) the file would not be copied.
>>>>
>>>> What do you think?
>>> Ummm.  What's the use case?  How do you handle the file being part of
>>> multiple
>>> overlays?
>>>
>>> David
>>>
>> I wan't to achieve 'files' granularity in overlayfs instead of 'directory'
>> granularity.
>> It would be useful in Linux containers configuration. Sometimes you don't
>> have the
>> power to change the images that are run inside containers and in one folder
>> you have both:
>> - files that have to be copied
>> - files that don't really need to be copied
>>
>>
>> trusted.overlay.cow==n indicates switching off copying the file to the upper
>> layer.
>> So with multiple overlays you would always have only one file from the
>> lower-most dir.
> What type filesystem is the lowest layer?   Can you set immutable on
> it (chattr +i)?  That would be the logical way to say "nobody can
> modify this file, not even root".
>
> Thanks,
> Miklos
> --
> To unsubscribe from this list: send the line "unsubscribe linux-unionfs" in
>

The lower fs is most likely ext4.
Unfortunately making a file immutable won't solve the problem. There's 
nothing wrong in modifying the file and it should stay possible.

I just want to switch off copy on write for some files. Can you see 
another way?

If you are OK with the optional xattr, then I'll prepare the patch.


Thanks,
Jan

Re: Optional switching off cow in overlayfs

[Miklos Szeredi]

On Mon, Jun 22, 2015 at 3:45 PM, Jan Olszak <j.olszak@samsung.com> wrote:

> The lower fs is most likely ext4.
> Unfortunately making a file immutable won't solve the problem. There's
> nothing wrong in modifying the file and it should stay possible.
>
> I just want to switch off copy on write for some files. Can you see another
> way?

Ah, you want to allow modification of file on the lower layer?

You can bind mount individual files from the lower layer to the
overlay.  That fixes the "allow modification" part.  But what should
happen on rename or unlink?

Thanks,
Miklos

Re: Optional switching off cow in overlayfs

[Jan Olszak]

On 06/22/2015 03:53 PM, Miklos Szeredi wrote:
> On Mon, Jun 22, 2015 at 3:45 PM, Jan Olszak <j.olszak@samsung.com> wrote:
>
>> The lower fs is most likely ext4.
>> Unfortunately making a file immutable won't solve the problem. There's
>> nothing wrong in modifying the file and it should stay possible.
>>
>> I just want to switch off copy on write for some files. Can you see another
>> way?
> Ah, you want to allow modification of file on the lower layer?
Exactly
> You can bind mount individual files from the lower layer to the
> overlay.  That fixes the "allow modification" part.
Well bind mounting every file that should have COW is unmaintainable - 
if new files appear admin has to mount each one.
Plus all the unnecessary mounts. It's much easier to do it the opposite 
way - by default have COW and only for some files switch it off.

>   But what should happen on rename or unlink?
Both would operate on the original inode.


Setting "trusted.overlay.cow" is a privileged operation so it's admin's 
decision to allow this.

Each layer could potentially make the a file COW again by 
setting/removing "trusted.overlay.cow" attr.


Thanks,
Jan

Re: Optional switching off cow in overlayfs

[Miklos Szeredi]

On Mon, Jun 22, 2015 at 4:26 PM, Jan Olszak <j.olszak@samsung.com> wrote:
>> You can bind mount individual files from the lower layer to the
>> overlay.  That fixes the "allow modification" part.
>
> Well bind mounting every file that should have COW is unmaintainable - if
> new files appear admin has to mount each one.

No.  I was proposing bind mounts for non-COW ones.

mount -t overlay -olowerdir=/lower,upperdir=/upper,workdir=/work  overlay /ovl
mount --bind /lower/this-file-is-non-cow /ovl/this-file-is-non-cow
...

>>   But what should happen on rename or unlink?
>
> Both would operate on the original inode.

And that would make this hugely complex.

Thanks,
Miklos

Re: Optional switching off cow in overlayfs

[Jan Olszak]

On 06/22/2015 04:37 PM, Miklos Szeredi wrote:
> On Mon, Jun 22, 2015 at 4:26 PM, Jan Olszak <j.olszak@samsung.com> wrote:
>>> You can bind mount individual files from the lower layer to the
>>> overlay.  That fixes the "allow modification" part.
>> Well bind mounting every file that should have COW is unmaintainable - if
>> new files appear admin has to mount each one.
> No.  I was proposing bind mounts for non-COW ones.
>
> mount -t overlay -olowerdir=/lower,upperdir=/upper,workdir=/work  overlay /ovl
> mount --bind /lower/this-file-is-non-cow /ovl/this-file-is-non-cow
> ...
Yes, you are right. This should work, but introduces an overhead of many 
mounts for each Linux container.

The proposed alternative is: Use setfattr only once for each file and 
then forget about it.

>>>    But what should happen on rename or unlink?
>> Both would operate on the original inode.
> And that would make this hugely complex.
Are you sure? I haven't looked into rename and unlink routines but this 
patch covers opening the file and is pretty simple (though it probably 
breaks fchmod/fchown...).

I can try to prepare a real solution if you like the idea.




 From 20e590c0e37f087955b25edfbc2ab4c9c034b5b1 Mon Sep 17 00:00:00 2001
From: Jan Olszak <j.olszak@samsung.com>
Date: Mon, 22 Jun 2015 17:07:31 +0200
Subject: [PATCH] Optional copy on write in overlayfs

---
  fs/overlayfs/inode.c     | 17 +++++++++++++++++
  fs/overlayfs/overlayfs.h |  1 +
  fs/overlayfs/super.c     |  5 -----
  3 files changed, 18 insertions(+), 5 deletions(-)

diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 04f1248..4f9f399 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -321,6 +321,20 @@ out:
      return err;
  }

+static bool ovl_dentry_need_copy_up(struct dentry *realdentry)
+{
+    int res;
+    char val;
+    struct inode *inode = realdentry->d_inode;
+
+    res = inode->i_op->getxattr(realdentry, OVL_XATTR_COW, &val, 1);
+    if (res == 1 && val == 'n'){
+        return false;
+    }
+
+    return true;
+}
+
  static bool ovl_open_need_copy_up(int flags, enum ovl_path_type type,
                    struct dentry *realdentry)
  {
@@ -330,6 +344,9 @@ static bool ovl_open_need_copy_up(int flags, enum 
ovl_path_type type,
      if (special_file(realdentry->d_inode->i_mode))
          return false;

+    if (!ovl_dentry_need_copy_up(realdentry))
+        return false;
+
      if (!(OPEN_FMODE(flags) & FMODE_WRITE) && !(flags & O_TRUNC))
          return false;

diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 17ac5af..b364888 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -26,6 +26,7 @@ enum ovl_path_type {
  #define OVL_XATTR_PRE_NAME "trusted.overlay."
  #define OVL_XATTR_PRE_LEN  16
  #define OVL_XATTR_OPAQUE   OVL_XATTR_PRE_NAME"opaque"
+#define OVL_XATTR_COW      OVL_XATTR_PRE_NAME"cow"

  static inline int ovl_do_rmdir(struct inode *dir, struct dentry *dentry)
  {
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index bf8537c..916a4b5 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -944,11 +944,6 @@ static int ovl_fill_super(struct super_block *sb, 
void *data, int silent)
              pr_err("overlayfs: failed to clone lowerpath\n");
              goto out_put_lower_mnt;
          }
-        /*
-         * Make lower_mnt R/O.  That way fchmod/fchown on lower file
-         * will fail instead of modifying lower fs.
-         */
-        mnt->mnt_flags |= MNT_READONLY;

          ufs->lower_mnt[ufs->numlower] = mnt;
          ufs->numlower++;
-- 
1.9.1





Thanks,
Jan

Re: Optional switching off cow in overlayfs

[David Howells]

Jan Olszak <j.olszak@samsung.com> wrote:

> >>>    But what should happen on rename or unlink?
> >> Both would operate on the original inode.
> > And that would make this hugely complex.
> Are you sure?

Unlink would be the same as now - you just whiteout over the lower layer.

Rename, though, would be complex - unless you go for the trivial option and
just error out.

The problem is that the only connection between the upper layer and lower
layer is coincidence governed by filename.  If you move the upper layer
object, the lower layer object is no longer coincident and is effectively
lost.  You cannot move the lower layer object because the lower layer may be
shared between multiple overlays.

At the moment, rename deals

You could, I suppose, store a "fall through" inode in the upper layer that has
the NFS fh of the lower layer object attached in some fashion (eg. xattr), but
it's messy.

Hard linking would be even more complex - you'd have to hard link the "fall
through" inodes on the upper layer.

David

Re: Optional switching off cow in overlayfs

[Jan Olszak]

On 06/23/2015 11:27 AM, David Howells wrote:
> Jan Olszak <j.olszak@samsung.com> wrote:
>
>>>>>     But what should happen on rename or unlink?
>>>> Both would operate on the original inode.
>>> And that would make this hugely complex.
>> Are you sure?
> Unlink would be the same as now - you just whiteout over the lower layer.
>
> Rename, though, would be complex - unless you go for the trivial option and
> just error out.
>
> The problem is that the only connection between the upper layer and lower
> layer is coincidence governed by filename.  If you move the upper layer
> object, the lower layer object is no longer coincident and is effectively
> lost.  You cannot move the lower layer object because the lower layer may be
> shared between multiple overlays.
>
> At the moment, rename deals
>
> You could, I suppose, store a "fall through" inode in the upper layer that has
> the NFS fh of the lower layer object attached in some fashion (eg. xattr), but
> it's messy.
>
> Hard linking would be even more complex - you'd have to hard link the "fall
> through" inodes on the upper layer.
>
> David
"Fall through" inodes seem a messy solution, agreed.

Why not just say: "rename causes copy" and leave everything as it is now.
Renaming a file shared by many mounts seems to be a corner case and I 
guess overlayfs handles this well right now.

Thanks,
Jan

Re: Optional switching off cow in overlayfs

[David Howells]

Jan Olszak <j.olszak@samsung.com> wrote:

> Why not just say: "rename causes copy" and leave everything as it is now.

That's another option - but your whole proposal was that we specifically don't
copy up these files IIRC.

David

Re: Optional switching off cow in overlayfs

[Jan Olszak]

[-- Attachment #1: Type: text/plain, Size: 828 bytes --]

On 06/23/2015 03:23 PM, David Howells wrote:
> Jan Olszak <j.olszak@samsung.com> wrote:
>
>> Why not just say: "rename causes copy" and leave everything as it is now.
> That's another option - but your whole proposal was that we specifically don't
> copy up these files IIRC.
Yes, so let's modify the proposal:

Overlayfs checks xattr "trusted.overlay.cow" to switch COW off whenever 
it's possible.

 1. This would be an optimization/convenience option, not a sercurity
    option.
 2. By default overlayfs uses COW like it does now
 3. trusted.overlay.cow=="n" means:
     1. Unlink, open, setattr etc. use the original inode from lowerdir
     2. Unmodifying operations also use inode from lowerdir
     3. Other operations (rename) may copy the file if it's required by
        the implementation of overlayfs


Thanks,
Jan

[-- Attachment #2: Type: text/html, Size: 1512 bytes --]

Re: Optional switching off cow in overlayfs

[Jan Olszak]

On 06/23/2015 03:23 PM, David Howells wrote:
> Jan Olszak <j.olszak@samsung.com> wrote:
>
>> Why not just say: "rename causes copy" and leave everything as it is now.
> That's another option - but your whole proposal was that we specifically don't
> copy up these files IIRC.
>

Yes, so let's modify the proposal:

Overlayfs checks xattr "trusted.overlay.cow" to switch COW off whenever 
it's possible.

1. This would be an optimization/convenience option, not a security option
2. By default overlayfs uses COW like it does now
3. trusted.overlay.cow=="n" means:
- Unlink, open, setattr etc. use the original inode from lowerdir
- Unmodifying operations also use inode from lowerdir
- Other operations (rename) may copy the file if it's required by the 
implementation of overlayfs

Thanks,
Jan

Re: Optional switching off cow in overlayfs

[Jan Olszak]

On 06/23/2015 06:07 PM, Jan Olszak wrote:
>
> On 06/23/2015 03:23 PM, David Howells wrote:
>> Jan Olszak <j.olszak@samsung.com> wrote:
>>
>>> Why not just say: "rename causes copy" and leave everything as it is 
>>> now.
>> That's another option - but your whole proposal was that we 
>> specifically don't
>> copy up these files IIRC.
>>
>
> Yes, so let's modify the proposal:
>
> Overlayfs checks xattr "trusted.overlay.cow" to switch COW off 
> whenever it's possible.
>
> 1. This would be an optimization/convenience option, not a security 
> option
> 2. By default overlayfs uses COW like it does now
> 3. trusted.overlay.cow=="n" means:
> - Unlink, open, setattr etc. use the original inode from lowerdir
> - Unmodifying operations also use inode from lowerdir
> - Other operations (rename) may copy the file if it's required by the 
> implementation of overlayfs
>
So does it sound upstreamable?

Jan