[PATCH] volatile-binds: use overlayfs if available

[PATCH] volatile-binds: use overlayfs if available

[Matt Hoosier]

From: Matt Hoosier <matt.hoosier@garmin.com>

Copying files from the read-only root filesystem to the tmpfs
providing the volatile directories can be slow and waste memory.
If the kernel supports the overlay filesystem, use it to mount
a writable tmpfs on top of the read-only directory from the
rootfs and avoid copies.

Analogous to the modification made to initscripts's
read-only-rootfs-hook in 370fda1b2e8d5dc011522131bba4106de26bfb19.

Signed-off-by: Matt Hoosier <matt.hoosier@gmail.com>
---
 .../volatile-binds/files/mount-copybind       | 30 ++++++++++++++++---
 1 file changed, 26 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-core/volatile-binds/files/mount-copybind b/meta/recipes-core/volatile-binds/files/mount-copybind
index 2aeaf84ddb..fddf520053 100755
--- a/meta/recipes-core/volatile-binds/files/mount-copybind
+++ b/meta/recipes-core/volatile-binds/files/mount-copybind
@@ -8,7 +8,10 @@ if [ $# -lt 2 ]; then
     exit 1
 fi
 
+# e.g. /var/volatile/lib
 spec=$1
+
+# e.g. /var/lib
 mountpoint=$2
 
 if [ $# -gt 2 ]; then
@@ -20,15 +23,34 @@ fi
 [ -n "$options" ] && options=",$options"
 
 mkdir -p "${spec%/*}"
+
 if [ -d "$mountpoint" ]; then
-    if [ ! -d "$spec" ]; then
+
+    if [ -d "$spec" ]; then
+        specdir_existed=yes
+    else
+        specdir_existed=no
         mkdir "$spec"
-        cp -pPR "$mountpoint"/. "$spec/"
+    fi
+
+    # Fast version of calculating `dirname ${spec}`/.`basename ${spec}`-work
+    overlay_workdir="${spec%/*}/.${spec##*/}-work"
+    mkdir "${overlay_workdir}"
+
+    # Try to mount using overlay, which is must faster than copying files.
+    # If that fails, fall back to slower copy.
+    if ! mount -t overlay overlay -olowerdir="$mountpoint",upperdir="$spec",workdir="$overlay_workdir" "$mountpoint" > /dev/null 2>&1; then
+
+        if [ "$specdir_existed" != "yes" ]; then
+            cp -pPR "$mountpoint"/. "$spec/"
+        fi
+
+        mount -o "bind$options" "$spec" "$mountpoint"
     fi
 elif [ -f "$mountpoint" ]; then
     if [ ! -f "$spec" ]; then
         cp -pP "$mountpoint" "$spec"
     fi
-fi
 
-mount -o "bind$options" "$spec" "$mountpoint"
+    mount -o "bind$options" "$spec" "$mountpoint"
+fi
-- 
2.17.2

Re: [PATCH] volatile-binds: use overlayfs if available

[Matt Hoosier]

[-- Attachment #1: Type: text/plain, Size: 2548 bytes --]

Hi; any interest in this one?

On Fri, Oct 19, 2018 at 11:04 AM Matt Hoosier <matt.hoosier@gmail.com>
wrote:

> From: Matt Hoosier <matt.hoosier@garmin.com>
>
> Copying files from the read-only root filesystem to the tmpfs
> providing the volatile directories can be slow and waste memory.
> If the kernel supports the overlay filesystem, use it to mount
> a writable tmpfs on top of the read-only directory from the
> rootfs and avoid copies.
>
> Analogous to the modification made to initscripts's
> read-only-rootfs-hook in 370fda1b2e8d5dc011522131bba4106de26bfb19.
>
> Signed-off-by: Matt Hoosier <matt.hoosier@gmail.com>
> ---
>  .../volatile-binds/files/mount-copybind       | 30 ++++++++++++++++---
>  1 file changed, 26 insertions(+), 4 deletions(-)
>
> diff --git a/meta/recipes-core/volatile-binds/files/mount-copybind
> b/meta/recipes-core/volatile-binds/files/mount-copybind
> index 2aeaf84ddb..fddf520053 100755
> --- a/meta/recipes-core/volatile-binds/files/mount-copybind
> +++ b/meta/recipes-core/volatile-binds/files/mount-copybind
> @@ -8,7 +8,10 @@ if [ $# -lt 2 ]; then
>      exit 1
>  fi
>
> +# e.g. /var/volatile/lib
>  spec=$1
> +
> +# e.g. /var/lib
>  mountpoint=$2
>
>  if [ $# -gt 2 ]; then
> @@ -20,15 +23,34 @@ fi
>  [ -n "$options" ] && options=",$options"
>
>  mkdir -p "${spec%/*}"
> +
>  if [ -d "$mountpoint" ]; then
> -    if [ ! -d "$spec" ]; then
> +
> +    if [ -d "$spec" ]; then
> +        specdir_existed=yes
> +    else
> +        specdir_existed=no
>          mkdir "$spec"
> -        cp -pPR "$mountpoint"/. "$spec/"
> +    fi
> +
> +    # Fast version of calculating `dirname ${spec}`/.`basename
> ${spec}`-work
> +    overlay_workdir="${spec%/*}/.${spec##*/}-work"
> +    mkdir "${overlay_workdir}"
> +
> +    # Try to mount using overlay, which is must faster than copying files.
> +    # If that fails, fall back to slower copy.
> +    if ! mount -t overlay overlay
> -olowerdir="$mountpoint",upperdir="$spec",workdir="$overlay_workdir"
> "$mountpoint" > /dev/null 2>&1; then
> +
> +        if [ "$specdir_existed" != "yes" ]; then
> +            cp -pPR "$mountpoint"/. "$spec/"
> +        fi
> +
> +        mount -o "bind$options" "$spec" "$mountpoint"
>      fi
>  elif [ -f "$mountpoint" ]; then
>      if [ ! -f "$spec" ]; then
>          cp -pP "$mountpoint" "$spec"
>      fi
> -fi
>
> -mount -o "bind$options" "$spec" "$mountpoint"
> +    mount -o "bind$options" "$spec" "$mountpoint"
> +fi
> --
> 2.17.2
>
>

[-- Attachment #2: Type: text/html, Size: 3523 bytes --]

Re: [PATCH] volatile-binds: use overlayfs if available

[Richard Purdie]

On Wed, 2018-10-24 at 09:40 -0500, Matt Hoosier wrote:
> Hi; any interest in this one?

Yes, it merged!

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=c4acf1b531843497297c492456daea1744400faa

Cheers,

Richard