[OE-core] [PATCH] image.bbclass: make sure do_rootfs run from a clean workspace

[OE-core] [PATCH] image.bbclass: make sure do_rootfs run from a clean workspace

[liu.ming50]

From: Ming Liu <liu.ming50@gmail.com>

Add ${IMAGE_ROOTFS} and ${IMGDEPLOYDIR} to do_rootfs[dirs] and
do_rootfs[cleandirs], this ensures do_rootfs run from a clean
workspace, with this change, we can now remove two bb.utils.mkdirhier
lines from meta/lib/oe/rootfs.py.

Also drop ${S} from do_rootfs[cleandirs], nothing being installed into
that directory.

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
---
 meta/classes/image.bbclass | 4 ++--
 meta/lib/oe/rootfs.py      | 4 ----
 2 files changed, 2 insertions(+), 6 deletions(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 2b0ce4a988..98a0555040 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -252,8 +252,8 @@ fakeroot python do_rootfs () {
 
     progress_reporter.finish()
 }
-do_rootfs[dirs] = "${TOPDIR}"
-do_rootfs[cleandirs] += "${S} ${IMGDEPLOYDIR}"
+do_rootfs[dirs] = "${IMAGE_ROOTFS} ${IMGDEPLOYDIR} ${TOPDIR}"
+do_rootfs[cleandirs] += "${IMAGE_ROOTFS} ${IMGDEPLOYDIR}"
 do_rootfs[file-checksums] += "${POSTINST_INTERCEPT_CHECKSUMS}"
 addtask rootfs after do_prepare_recipe_sysroot
 
diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index b0dd625539..98cf3f244d 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -190,10 +190,6 @@ class Rootfs(object, metaclass=ABCMeta):
         post_process_cmds = self.d.getVar("ROOTFS_POSTPROCESS_COMMAND")
         rootfs_post_install_cmds = self.d.getVar('ROOTFS_POSTINSTALL_COMMAND')
 
-        bb.utils.mkdirhier(self.image_rootfs)
-
-        bb.utils.mkdirhier(self.deploydir)
-
         execute_pre_post_process(self.d, pre_process_cmds)
 
         if self.progress_reporter:
-- 
2.25.1

Re: [OE-core] [PATCH] image.bbclass: make sure do_rootfs run from a clean workspace

[Richard Purdie]

On Sat, 2022-03-12 at 12:53 +0100, Ming Liu wrote:
> From: Ming Liu <liu.ming50@gmail.com>
> 
> Add ${IMAGE_ROOTFS} and ${IMGDEPLOYDIR} to do_rootfs[dirs] and
> do_rootfs[cleandirs], this ensures do_rootfs run from a clean
> workspace, with this change, we can now remove two bb.utils.mkdirhier
> lines from meta/lib/oe/rootfs.py.
> 
> Also drop ${S} from do_rootfs[cleandirs], nothing being installed into
> that directory.
> 
> Signed-off-by: Ming Liu <liu.ming50@gmail.com>
> ---
>  meta/classes/image.bbclass | 4 ++--
>  meta/lib/oe/rootfs.py      | 4 ----
>  2 files changed, 2 insertions(+), 6 deletions(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 2b0ce4a988..98a0555040 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -252,8 +252,8 @@ fakeroot python do_rootfs () {
>  
>      progress_reporter.finish()
>  }
> -do_rootfs[dirs] = "${TOPDIR}"
> -do_rootfs[cleandirs] += "${S} ${IMGDEPLOYDIR}"
> +do_rootfs[dirs] = "${IMAGE_ROOTFS} ${IMGDEPLOYDIR} ${TOPDIR}"
> +do_rootfs[cleandirs] += "${IMAGE_ROOTFS} ${IMGDEPLOYDIR}"
>  do_rootfs[file-checksums] += "${POSTINST_INTERCEPT_CHECKSUMS}"
>  addtask rootfs after do_prepare_recipe_sysroot

I think the removal of ${S} breaks things:

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3279/steps/14/logs/stdio

Cheers,

Richard

Re: [OE-core] [PATCH] image.bbclass: make sure do_rootfs run from a clean workspace

[Richard Purdie]

On Sat, 2022-03-12 at 16:22 +0000, Richard Purdie via lists.openembedded.org
wrote:
> On Sat, 2022-03-12 at 12:53 +0100, Ming Liu wrote:
> > From: Ming Liu <liu.ming50@gmail.com>
> > 
> > Add ${IMAGE_ROOTFS} and ${IMGDEPLOYDIR} to do_rootfs[dirs] and
> > do_rootfs[cleandirs], this ensures do_rootfs run from a clean
> > workspace, with this change, we can now remove two bb.utils.mkdirhier
> > lines from meta/lib/oe/rootfs.py.
> > 
> > Also drop ${S} from do_rootfs[cleandirs], nothing being installed into
> > that directory.
> > 
> > Signed-off-by: Ming Liu <liu.ming50@gmail.com>
> > ---
> >  meta/classes/image.bbclass | 4 ++--
> >  meta/lib/oe/rootfs.py      | 4 ----
> >  2 files changed, 2 insertions(+), 6 deletions(-)
> > 
> > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> > index 2b0ce4a988..98a0555040 100644
> > --- a/meta/classes/image.bbclass
> > +++ b/meta/classes/image.bbclass
> > @@ -252,8 +252,8 @@ fakeroot python do_rootfs () {
> >  
> >      progress_reporter.finish()
> >  }
> > -do_rootfs[dirs] = "${TOPDIR}"
> > -do_rootfs[cleandirs] += "${S} ${IMGDEPLOYDIR}"
> > +do_rootfs[dirs] = "${IMAGE_ROOTFS} ${IMGDEPLOYDIR} ${TOPDIR}"
> > +do_rootfs[cleandirs] += "${IMAGE_ROOTFS} ${IMGDEPLOYDIR}"
> >  do_rootfs[file-checksums] += "${POSTINST_INTERCEPT_CHECKSUMS}"
> >  addtask rootfs after do_prepare_recipe_sysroot
> 
> I think the removal of ${S} breaks things:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3279/steps/14/logs/stdio

FWIW I've dropped that piece of the patch and merged.

Cheers,

Richard

RE: [OE-core] [PATCH] image.bbclass: make sure do_rootfs run from a clean workspace

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-
> core@lists.openembedded.org> On Behalf Of Ming Liu
> Sent: den 12 mars 2022 12:53
> To: openembedded-core@lists.openembedded.org
> Cc: Ming Liu <liu.ming50@gmail.com>
> Subject: [OE-core] [PATCH] image.bbclass: make sure do_rootfs run from a
> clean workspace
> 
> From: Ming Liu <liu.ming50@gmail.com>
> 
> Add ${IMAGE_ROOTFS} and ${IMGDEPLOYDIR} to do_rootfs[dirs] and
> do_rootfs[cleandirs], this ensures do_rootfs run from a clean
> workspace, with this change, we can now remove two bb.utils.mkdirhier
> lines from meta/lib/oe/rootfs.py.
> 
> Also drop ${S} from do_rootfs[cleandirs], nothing being installed into
> that directory.
> 
> Signed-off-by: Ming Liu <liu.ming50@gmail.com>
> ---
>  meta/classes/image.bbclass | 4 ++--
>  meta/lib/oe/rootfs.py      | 4 ----
>  2 files changed, 2 insertions(+), 6 deletions(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 2b0ce4a988..98a0555040 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -252,8 +252,8 @@ fakeroot python do_rootfs () {
> 
>      progress_reporter.finish()
>  }
> -do_rootfs[dirs] = "${TOPDIR}"
> -do_rootfs[cleandirs] += "${S} ${IMGDEPLOYDIR}"
> +do_rootfs[dirs] = "${IMAGE_ROOTFS} ${IMGDEPLOYDIR} ${TOPDIR}"
> +do_rootfs[cleandirs] += "${IMAGE_ROOTFS} ${IMGDEPLOYDIR}"

There is no reason to include paths in do_rootfs[dirs] if they are 
also in do_rootfs[cleandirs] (except for the last path in 
do_rootfs[dirs]). It only leads to bb.utils.mkdirhier() being 
called for the same directory multiple times.

I have sent a patch series to correct this here and for all other 
cases I could find. For anyone interested, I used the following 
incantation to find the culprits:

grep -r 'cleandirs.*".*"' meta | grep -v '${@' | \
  perl -ne 'if (/^(.*):(.*)\[.*\].*"(.*)"/) { @vars = split(" ", $3); for $var (@vars) {  print("=" x 75, "\n$1 - $2 - $var\n"); system("grep", "--color=always", "$2\\[dirs\\].*\\$var ", $1); } }'

>  do_rootfs[file-checksums] += "${POSTINST_INTERCEPT_CHECKSUMS}"
>  addtask rootfs after do_prepare_recipe_sysroot
> 
> diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
> index b0dd625539..98cf3f244d 100644
> --- a/meta/lib/oe/rootfs.py
> +++ b/meta/lib/oe/rootfs.py
> @@ -190,10 +190,6 @@ class Rootfs(object, metaclass=ABCMeta):
>          post_process_cmds = self.d.getVar("ROOTFS_POSTPROCESS_COMMAND")
>          rootfs_post_install_cmds =
> self.d.getVar('ROOTFS_POSTINSTALL_COMMAND')
> 
> -        bb.utils.mkdirhier(self.image_rootfs)
> -
> -        bb.utils.mkdirhier(self.deploydir)
> -
>          execute_pre_post_process(self.d, pre_process_cmds)
> 
>          if self.progress_reporter:
> --
> 2.25.1

//Peter