[PATCH] Xen: fix do_deploy dependencies

[PATCH] Xen: fix do_deploy dependencies

[Bertrand Marquis]

do_deploy is using the files installed during do_install and it must be
executed after it but before do_build. This is fixing SSTATE behaviour
which had previously an empty scene in some cases.

Change-Id: I04b1f652c697ed0bc7e262571dd03e123ac096f6
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
 recipes-extended/xen/xen.inc | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/recipes-extended/xen/xen.inc b/recipes-extended/xen/xen.inc
index 36ab78e..41b1983 100644
--- a/recipes-extended/xen/xen.inc
+++ b/recipes-extended/xen/xen.inc
@@ -1099,7 +1099,9 @@ do_deploy() {
     fi
 }
 
-addtask deploy after do_populate_sysroot
+# do_deploy is using files created during do_install so force it to be called
+# after do_install but before do_build
+addtask deploy before do_build after do_install
 
 # Enable use of menuconfig directly from bitbake and also within the devshell
 OE_TERMINAL_EXPORTS += "HOST_EXTRACFLAGS HOSTLDFLAGS TERMINFO"
-- 
2.17.1

Re: [meta-virtualization] [PATCH] Xen: fix do_deploy dependencies

[Bertrand Marquis]

> On 14 Feb 2020, at 10:17, Bertrand Marquis via Lists.Yoctoproject.Org <bertrand.marquis=arm.com@lists.yoctoproject.org> wrote:
>
> do_deploy is using the files installed during do_install and it must be
> executed after it but before do_build. This is fixing SSTATE behaviour
> which had previously an empty scene in some cases.

Any chance somebody could check and ack on this ?

While there, it would make sense to also push this in zeus branch.

—
Bertrand Marquis

>
> Change-Id: I04b1f652c697ed0bc7e262571dd03e123ac096f6
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> recipes-extended/xen/xen.inc | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/recipes-extended/xen/xen.inc b/recipes-extended/xen/xen.inc
> index 36ab78e..41b1983 100644
> --- a/recipes-extended/xen/xen.inc
> +++ b/recipes-extended/xen/xen.inc
> @@ -1099,7 +1099,9 @@ do_deploy() {
>     fi
> }
>
> -addtask deploy after do_populate_sysroot
> +# do_deploy is using files created during do_install so force it to be called
> +# after do_install but before do_build
> +addtask deploy before do_build after do_install
>
> # Enable use of menuconfig directly from bitbake and also within the devshell
> OE_TERMINAL_EXPORTS += "HOST_EXTRACFLAGS HOSTLDFLAGS TERMINFO"
> --
> 2.17.1
>

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Re: [meta-virtualization] [PATCH] Xen: fix do_deploy dependencies

[Bruce Ashfield]

On Wed, Feb 19, 2020 at 8:09 AM Bertrand Marquis
<Bertrand.Marquis@arm.com> wrote:
>
>
> > On 14 Feb 2020, at 10:17, Bertrand Marquis via Lists.Yoctoproject.Org <bertrand.marquis=arm.com@lists.yoctoproject.org> wrote:
> >
> > do_deploy is using the files installed during do_install and it must be
> > executed after it but before do_build. This is fixing SSTATE behaviour
> > which had previously an empty scene in some cases.
>
> Any chance somebody could check and ack on this ?
>

There's an entire update / re-work of Xen on the mailing list. I won't
be merging any Xen patches until we have that sorted out.

Cheers,

Bruce

> While there, it would make sense to also push this in zeus branch.
>
> —
> Bertrand Marquis
>
> >
> > Change-Id: I04b1f652c697ed0bc7e262571dd03e123ac096f6
> > Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> > ---
> > recipes-extended/xen/xen.inc | 4 +++-
> > 1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/recipes-extended/xen/xen.inc b/recipes-extended/xen/xen.inc
> > index 36ab78e..41b1983 100644
> > --- a/recipes-extended/xen/xen.inc
> > +++ b/recipes-extended/xen/xen.inc
> > @@ -1099,7 +1099,9 @@ do_deploy() {
> >     fi
> > }
> >
> > -addtask deploy after do_populate_sysroot
> > +# do_deploy is using files created during do_install so force it to be called
> > +# after do_install but before do_build
> > +addtask deploy before do_build after do_install
> >
> > # Enable use of menuconfig directly from bitbake and also within the devshell
> > OE_TERMINAL_EXPORTS += "HOST_EXTRACFLAGS HOSTLDFLAGS TERMINFO"
> > --
> > 2.17.1
> >
>
> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] [PATCH] Xen: fix do_deploy dependencies

[Bertrand Marquis]

[-- Attachment #1: Type: text/plain, Size: 2369 bytes --]



On 19 Feb 2020, at 13:31, Bruce Ashfield <bruce.ashfield@gmail.com<mailto:bruce.ashfield@gmail.com>> wrote:

On Wed, Feb 19, 2020 at 8:09 AM Bertrand Marquis
<Bertrand.Marquis@arm.com<mailto:Bertrand.Marquis@arm.com>> wrote:


On 14 Feb 2020, at 10:17, Bertrand Marquis via Lists.Yoctoproject.Org<http://lists.yoctoproject.org/> <bertrand.marquis=arm.com@lists.yoctoproject.org<mailto:bertrand.marquis=arm.com@lists.yoctoproject.org>> wrote:

do_deploy is using the files installed during do_install and it must be
executed after it but before do_build. This is fixing SSTATE behaviour
which had previously an empty scene in some cases.

Any chance somebody could check and ack on this ?


There's an entire update / re-work of Xen on the mailing list. I won't
be merging any Xen patches until we have that sorted out.

Ok I will re-push my patch once the changes in Xen are merged then.

Cheers
Bertrand



Cheers,

Bruce

While there, it would make sense to also push this in zeus branch.

—
Bertrand Marquis


Change-Id: I04b1f652c697ed0bc7e262571dd03e123ac096f6
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com<mailto:bertrand.marquis@arm.com>>
---
recipes-extended/xen/xen.inc | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/recipes-extended/xen/xen.inc b/recipes-extended/xen/xen.inc
index 36ab78e..41b1983 100644
--- a/recipes-extended/xen/xen.inc
+++ b/recipes-extended/xen/xen.inc
@@ -1099,7 +1099,9 @@ do_deploy() {
   fi
}

-addtask deploy after do_populate_sysroot
+# do_deploy is using files created during do_install so force it to be called
+# after do_install but before do_build
+addtask deploy before do_build after do_install

# Enable use of menuconfig directly from bitbake and also within the devshell
OE_TERMINAL_EXPORTS += "HOST_EXTRACFLAGS HOSTLDFLAGS TERMINFO"
--
2.17.1


IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.



--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


[-- Attachment #2: Type: text/html, Size: 14724 bytes --]

Re: [meta-virtualization] [PATCH] Xen: fix do_deploy dependencies

[Bruce Ashfield]

On Wed, Feb 19, 2020 at 8:47 AM Bertrand Marquis
<Bertrand.Marquis@arm.com> wrote:
>
>
>
> On 19 Feb 2020, at 13:31, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Wed, Feb 19, 2020 at 8:09 AM Bertrand Marquis
> <Bertrand.Marquis@arm.com> wrote:
>
>
>
> On 14 Feb 2020, at 10:17, Bertrand Marquis via Lists.Yoctoproject.Org <bertrand.marquis=arm.com@lists.yoctoproject.org> wrote:
>
> do_deploy is using the files installed during do_install and it must be
> executed after it but before do_build. This is fixing SSTATE behaviour
> which had previously an empty scene in some cases.
>
>
> Any chance somebody could check and ack on this ?
>
>
> There's an entire update / re-work of Xen on the mailing list. I won't
> be merging any Xen patches until we have that sorted out.
>
>
> Ok I will re-push my patch once the changes in Xen are merged then.
>

the patch does look fine, and I've tagged it for merging. I just
wanted to get the big changes in before jiggling anything (even if it
shouldn't have an impact).

Bruce

> Cheers
> Bertrand
>
>
>
> Cheers,
>
> Bruce
>
> While there, it would make sense to also push this in zeus branch.
>
> —
> Bertrand Marquis
>
>
> Change-Id: I04b1f652c697ed0bc7e262571dd03e123ac096f6
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
> recipes-extended/xen/xen.inc | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/recipes-extended/xen/xen.inc b/recipes-extended/xen/xen.inc
> index 36ab78e..41b1983 100644
> --- a/recipes-extended/xen/xen.inc
> +++ b/recipes-extended/xen/xen.inc
> @@ -1099,7 +1099,9 @@ do_deploy() {
>    fi
> }
>
> -addtask deploy after do_populate_sysroot
> +# do_deploy is using files created during do_install so force it to be called
> +# after do_install but before do_build
> +addtask deploy before do_build after do_install
>
> # Enable use of menuconfig directly from bitbake and also within the devshell
> OE_TERMINAL_EXPORTS += "HOST_EXTRACFLAGS HOSTLDFLAGS TERMINFO"
> --
> 2.17.1
>
>
> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
>
>
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] [PATCH] Xen: fix do_deploy dependencies

[Christopher Clark]

On Wed, Feb 19, 2020 at 6:00 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Wed, Feb 19, 2020 at 8:47 AM Bertrand Marquis <Bertrand.Marquis@arm.com> wrote:
> >
> > On 19 Feb 2020, at 13:31, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >
> > On Wed, Feb 19, 2020 at 8:09 AM Bertrand Marquis  <Bertrand.Marquis@arm.com> wrote:
> >
> > On 14 Feb 2020, at 10:17, Bertrand Marquis via Lists.Yoctoproject.Org <bertrand.marquis=arm.com@lists.yoctoproject.org> wrote:
> >
> > > > > do_deploy is using the files installed during do_install and it must be
> > > > > executed after it but before do_build. This is fixing SSTATE behaviour
> > > > > which had previously an empty scene in some cases.
> > >
> > > > Any chance somebody could check and ack on this ?
> > >
> > > There's an entire update / re-work of Xen on the mailing list. I won't
> > > be merging any Xen patches until we have that sorted out.
> >
> > Ok I will re-push my patch once the changes in Xen are merged then.
>
> the patch does look fine, and I've tagged it for merging. I just
> wanted to get the big changes in before jiggling anything (even if it
> shouldn't have an impact).

Thanks for the report, Bertrand. I've taken a look at this. The
addition of "before do_build" looks OK to me, and should be sufficient
to address the behaviour reported about an empty scene observed in
some cases -- some more details about that observation that prompted
this patch, and some testing to confirm that just adding "before
do_build" addition addresses it would be helpful.

I'm not convinced about the change to move it to "after do_install" though.
do_populate_sysroot already runs after do_install, so no change is
needed to ensure that the binaries are available by the time the
deploy task runs, but more importantly I think that the deploy task
should run, as it currently does, after do_populate_sysroot to ensure
that the same binaries that are made available to other recipes via
sysroots are also those that are deployed. Scheduling it after the
do_populate_sysroot means that if the deploy task is run, the sysroot
copies are also up to date.

Christopher



> > While there, it would make sense to also push this in zeus branch.
> >
> > —
> > Bertrand Marquis
> >
> >
> > Change-Id: I04b1f652c697ed0bc7e262571dd03e123ac096f6
> > Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> > ---
> > recipes-extended/xen/xen.inc | 4 +++-
> > 1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/recipes-extended/xen/xen.inc b/recipes-extended/xen/xen.inc
> > index 36ab78e..41b1983 100644
> > --- a/recipes-extended/xen/xen.inc
> > +++ b/recipes-extended/xen/xen.inc
> > @@ -1099,7 +1099,9 @@ do_deploy() {
> >    fi
> > }
> >
> > -addtask deploy after do_populate_sysroot
> > +# do_deploy is using files created during do_install so force it to be called
> > +# after do_install but before do_build
> > +addtask deploy before do_build after do_install
> >
> > # Enable use of menuconfig directly from bitbake and also within the devshell
> > OE_TERMINAL_EXPORTS += "HOST_EXTRACFLAGS HOSTLDFLAGS TERMINFO"
> > --
> > 2.17.1

Re: [meta-virtualization] [PATCH] Xen: fix do_deploy dependencies

[Bertrand Marquis]

[-- Attachment #1: Type: text/plain, Size: 3678 bytes --]



On 20 Feb 2020, at 18:41, Christopher Clark <christopher.w.clark@gmail.com<mailto:christopher.w.clark@gmail.com>> wrote:

On Wed, Feb 19, 2020 at 6:00 AM Bruce Ashfield <bruce.ashfield@gmail.com<mailto:bruce.ashfield@gmail.com>> wrote:

On Wed, Feb 19, 2020 at 8:47 AM Bertrand Marquis <Bertrand.Marquis@arm.com<mailto:Bertrand.Marquis@arm.com>> wrote:

On 19 Feb 2020, at 13:31, Bruce Ashfield <bruce.ashfield@gmail.com<mailto:bruce.ashfield@gmail.com>> wrote:

On Wed, Feb 19, 2020 at 8:09 AM Bertrand Marquis  <Bertrand.Marquis@arm.com<mailto:Bertrand.Marquis@arm.com>> wrote:

On 14 Feb 2020, at 10:17, Bertrand Marquis via Lists.Yoctoproject.Org<http://Lists.Yoctoproject.Org> <bertrand.marquis=arm.com@lists.yoctoproject.org<mailto:bertrand.marquis=arm.com@lists.yoctoproject.org>> wrote:

do_deploy is using the files installed during do_install and it must be
executed after it but before do_build. This is fixing SSTATE behaviour
which had previously an empty scene in some cases.

Any chance somebody could check and ack on this ?

There's an entire update / re-work of Xen on the mailing list. I won't
be merging any Xen patches until we have that sorted out.

Ok I will re-push my patch once the changes in Xen are merged then.

the patch does look fine, and I've tagged it for merging. I just
wanted to get the big changes in before jiggling anything (even if it
shouldn't have an impact).

Thanks for the report, Bertrand. I've taken a look at this. The
addition of "before do_build" looks OK to me, and should be sufficient
to address the behaviour reported about an empty scene observed in
some cases -- some more details about that observation that prompted
this patch, and some testing to confirm that just adding "before
do_build" addition addresses it would be helpful.

A big grep in poky for “addtask deploy” is showing that most of the time this is added with:
addtask deploy before do_build after do_instal


I'm not convinced about the change to move it to "after do_install" though.
do_populate_sysroot already runs after do_install, so no change is
needed to ensure that the binaries are available by the time the
deploy task runs, but more importantly I think that the deploy task
should run, as it currently does, after do_populate_sysroot to ensure
that the same binaries that are made available to other recipes via
sysroots are also those that are deployed. Scheduling it after the
do_populate_sysroot means that if the deploy task is run, the sysroot
copies are also up to date.


The problem in doing this after do_populate_sysroot is that do_deploy is not using the populated sysroot but the result of do_install.

Bertrand


Christopher



While there, it would make sense to also push this in zeus branch.

—
Bertrand Marquis


Change-Id: I04b1f652c697ed0bc7e262571dd03e123ac096f6
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com<mailto:bertrand.marquis@arm.com>>
---
recipes-extended/xen/xen.inc | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/recipes-extended/xen/xen.inc b/recipes-extended/xen/xen.inc
index 36ab78e..41b1983 100644
--- a/recipes-extended/xen/xen.inc
+++ b/recipes-extended/xen/xen.inc
@@ -1099,7 +1099,9 @@ do_deploy() {
  fi
}

-addtask deploy after do_populate_sysroot
+# do_deploy is using files created during do_install so force it to be called
+# after do_install but before do_build
+addtask deploy before do_build after do_install

# Enable use of menuconfig directly from bitbake and also within the devshell
OE_TERMINAL_EXPORTS += "HOST_EXTRACFLAGS HOSTLDFLAGS TERMINFO"
--
2.17.1


[-- Attachment #2: Type: text/html, Size: 21485 bytes --]

Re: [meta-virtualization] [PATCH] Xen: fix do_deploy dependencies

[Christopher Clark]

On Fri, Feb 21, 2020 at 1:08 AM Bertrand Marquis
<Bertrand.Marquis@arm.com> wrote:
>
> On 20 Feb 2020, at 18:41, Christopher Clark <christopher.w.clark@gmail.com> wrote:
>
> > On Wed, Feb 19, 2020 at 6:00 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> > > On Wed, Feb 19, 2020 at 8:47 AM Bertrand Marquis <Bertrand.Marquis@arm.com> wrote:
>
> > > > On 19 Feb 2020, at 13:31, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> > > > > On Wed, Feb 19, 2020 at 8:09 AM Bertrand Marquis  <Bertrand.Marquis@arm.com> wrote:
>
> > > > > > On 14 Feb 2020, at 10:17, Bertrand Marquis via Lists.Yoctoproject.Org <bertrand.marquis=arm.com@lists.yoctoproject.org> wrote:
>
> > > > > > > do_deploy is using the files installed during do_install and it must be
> > > > > > > executed after it but before do_build. This is fixing SSTATE behaviour
> > > > > > > which had previously an empty scene in some cases.
>
> > > > > > Any chance somebody could check and ack on this ?
>
> > > > > There's an entire update / re-work of Xen on the mailing list. I won't
> > > > > be merging any Xen patches until we have that sorted out.
>
> > > > Ok I will re-push my patch once the changes in Xen are merged then.
>
> > > the patch does look fine, and I've tagged it for merging. I just
> > > wanted to get the big changes in before jiggling anything (even if it
> > > shouldn't have an impact).
>
> > Thanks for the report, Bertrand. I've taken a look at this. The
> > addition of "before do_build" looks OK to me, and should be sufficient
> > to address the behaviour reported about an empty scene observed in
> > some cases -- some more details about that observation that prompted
> > this patch, and some testing to confirm that just adding "before
> > do_build" addition addresses it would be helpful.
>
> A big grep in poky for “addtask deploy” is showing that most of the time this is added with:
> addtask deploy before do_build after do_instal

You're right that there are several recipes in poky that do that; but different
recipes have different requirements, based on what they build. I'll explain
below why the Xen recipe differs from those.

> > I'm not convinced about the change to move it to "after do_install" though.
> > do_populate_sysroot already runs after do_install, so no change is
> > needed to ensure that the binaries are available by the time the
> > deploy task runs, but more importantly I think that the deploy task
> > should run, as it currently does, after do_populate_sysroot to ensure
> > that the same binaries that are made available to other recipes via
> > sysroots are also those that are deployed. Scheduling it after the
> > do_populate_sysroot means that if the deploy task is run, the sysroot
> > copies are also up to date.
>
>
> The problem in doing this after do_populate_sysroot is that do_deploy is not using the populated sysroot but the result of do_install.

The current Xen recipe that is in the meta-virtualization layer builds both the
hypervisor and the tools. These tools-and-hypervisor have to match version in
order to function correctly. There are two separate paths for the build
products: the hypervisor binary is made available to 'deploy' and the tools are
staged for installation in the image rootfs.

To ensure that the correct version of the tools have been built and staged for
the rootfs to match the deployed hypervisor binary, the deploy task which makes
the hypervisor binary available depends on do_populate_sysroot task that stages
the built tools binaries. That's why the existing recipe has the
'addtask deploy'
statement 'after do_populate_sysroot'.

So: you are correct that do_deploy is indeed writing binaries that are
the result of
do_install, but those binaries also have a runtime dependency on the correct
version of the tools being staged for inclusion in the image rootfs.

For my patch series that updates the Xen recipes, it introduces separate
recipes for the tools and the hypervisor, and for the new version of the series
that I'm about to post here, I've updated the deploy tasks to ensure that
this deploy dependency for the hypervisor on the tools being staged correctly
is preserved.

thanks,

Christopher

Re: [meta-virtualization] [PATCH] Xen: fix do_deploy dependencies

[Bertrand Marquis]

[-- Attachment #1: Type: text/plain, Size: 4555 bytes --]



On 24 Feb 2020, at 04:55, Christopher Clark <christopher.w.clark@gmail.com<mailto:christopher.w.clark@gmail.com>> wrote:

On Fri, Feb 21, 2020 at 1:08 AM Bertrand Marquis
<Bertrand.Marquis@arm.com<mailto:Bertrand.Marquis@arm.com>> wrote:

On 20 Feb 2020, at 18:41, Christopher Clark <christopher.w.clark@gmail.com<mailto:christopher.w.clark@gmail.com>> wrote:

On Wed, Feb 19, 2020 at 6:00 AM Bruce Ashfield <bruce.ashfield@gmail.com<mailto:bruce.ashfield@gmail.com>> wrote:

On Wed, Feb 19, 2020 at 8:47 AM Bertrand Marquis <Bertrand.Marquis@arm.com<mailto:Bertrand.Marquis@arm.com>> wrote:

On 19 Feb 2020, at 13:31, Bruce Ashfield <bruce.ashfield@gmail.com<mailto:bruce.ashfield@gmail.com>> wrote:

On Wed, Feb 19, 2020 at 8:09 AM Bertrand Marquis  <Bertrand.Marquis@arm.com<mailto:Bertrand.Marquis@arm.com>> wrote:

On 14 Feb 2020, at 10:17, Bertrand Marquis via Lists.Yoctoproject.Org<http://Lists.Yoctoproject.Org> <bertrand.marquis=arm.com@lists.yoctoproject.org<mailto:bertrand.marquis=arm.com@lists.yoctoproject.org>> wrote:

do_deploy is using the files installed during do_install and it must be
executed after it but before do_build. This is fixing SSTATE behaviour
which had previously an empty scene in some cases.

Any chance somebody could check and ack on this ?

There's an entire update / re-work of Xen on the mailing list. I won't
be merging any Xen patches until we have that sorted out.

Ok I will re-push my patch once the changes in Xen are merged then.

the patch does look fine, and I've tagged it for merging. I just
wanted to get the big changes in before jiggling anything (even if it
shouldn't have an impact).

Thanks for the report, Bertrand. I've taken a look at this. The
addition of "before do_build" looks OK to me, and should be sufficient
to address the behaviour reported about an empty scene observed in
some cases -- some more details about that observation that prompted
this patch, and some testing to confirm that just adding "before
do_build" addition addresses it would be helpful.

A big grep in poky for “addtask deploy” is showing that most of the time this is added with:
addtask deploy before do_build after do_instal

You're right that there are several recipes in poky that do that; but different
recipes have different requirements, based on what they build. I'll explain
below why the Xen recipe differs from those.

I'm not convinced about the change to move it to "after do_install" though.
do_populate_sysroot already runs after do_install, so no change is
needed to ensure that the binaries are available by the time the
deploy task runs, but more importantly I think that the deploy task
should run, as it currently does, after do_populate_sysroot to ensure
that the same binaries that are made available to other recipes via
sysroots are also those that are deployed. Scheduling it after the
do_populate_sysroot means that if the deploy task is run, the sysroot
copies are also up to date.


The problem in doing this after do_populate_sysroot is that do_deploy is not using the populated sysroot but the result of do_install.

The current Xen recipe that is in the meta-virtualization layer builds both the
hypervisor and the tools. These tools-and-hypervisor have to match version in
order to function correctly. There are two separate paths for the build
products: the hypervisor binary is made available to 'deploy' and the tools are
staged for installation in the image rootfs.

To ensure that the correct version of the tools have been built and staged for
the rootfs to match the deployed hypervisor binary, the deploy task which makes
the hypervisor binary available depends on do_populate_sysroot task that stages
the built tools binaries. That's why the existing recipe has the
'addtask deploy'
statement 'after do_populate_sysroot'.

So: you are correct that do_deploy is indeed writing binaries that are
the result of
do_install, but those binaries also have a runtime dependency on the correct
version of the tools being staged for inclusion in the image rootfs.

For my patch series that updates the Xen recipes, it introduces separate
recipes for the tools and the hypervisor, and for the new version of the series
that I'm about to post here, I've updated the deploy tasks to ensure that
this deploy dependency for the hypervisor on the tools being staged correctly
is preserved.

Thanks for the explanations.
I will wait for your patch to be merged to test this again.

Bertrand


[-- Attachment #2: Type: text/html, Size: 29375 bytes --]

Re: [meta-virtualization] [PATCH] Xen: fix do_deploy dependencies

[Bruce Ashfield]

merged.

Bruce

In message: [meta-virtualization] [PATCH] Xen: fix do_deploy dependencies
on 14/02/2020 Bertrand Marquis wrote:

> do_deploy is using the files installed during do_install and it must be
> executed after it but before do_build. This is fixing SSTATE behaviour
> which had previously an empty scene in some cases.
> 
> Change-Id: I04b1f652c697ed0bc7e262571dd03e123ac096f6
> Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> ---
>  recipes-extended/xen/xen.inc | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/recipes-extended/xen/xen.inc b/recipes-extended/xen/xen.inc
> index 36ab78e..41b1983 100644
> --- a/recipes-extended/xen/xen.inc
> +++ b/recipes-extended/xen/xen.inc
> @@ -1099,7 +1099,9 @@ do_deploy() {
>      fi
>  }
>  
> -addtask deploy after do_populate_sysroot
> +# do_deploy is using files created during do_install so force it to be called
> +# after do_install but before do_build
> +addtask deploy before do_build after do_install
>  
>  # Enable use of menuconfig directly from bitbake and also within the devshell
>  OE_TERMINAL_EXPORTS += "HOST_EXTRACFLAGS HOSTLDFLAGS TERMINFO"
> -- 
> 2.17.1
> 

> 

Re: [meta-virtualization] [PATCH] Xen: fix do_deploy dependencies

[Bruce Ashfield]

ooops. Misfire on the email.

This isn't merged and can be revisited once I have the Xen updates in the tree.

Bruce

On Thu, Feb 27, 2020 at 4:57 PM Bruce Ashfield via
Lists.Yoctoproject.Org
<bruce.ashfield=gmail.com@lists.yoctoproject.org> wrote:
>
>
> merged.
>
> Bruce
>
> In message: [meta-virtualization] [PATCH] Xen: fix do_deploy dependencies
> on 14/02/2020 Bertrand Marquis wrote:
>
> > do_deploy is using the files installed during do_install and it must be
> > executed after it but before do_build. This is fixing SSTATE behaviour
> > which had previously an empty scene in some cases.
> >
> > Change-Id: I04b1f652c697ed0bc7e262571dd03e123ac096f6
> > Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
> > ---
> >  recipes-extended/xen/xen.inc | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/recipes-extended/xen/xen.inc b/recipes-extended/xen/xen.inc
> > index 36ab78e..41b1983 100644
> > --- a/recipes-extended/xen/xen.inc
> > +++ b/recipes-extended/xen/xen.inc
> > @@ -1099,7 +1099,9 @@ do_deploy() {
> >      fi
> >  }
> >
> > -addtask deploy after do_populate_sysroot
> > +# do_deploy is using files created during do_install so force it to be called
> > +# after do_install but before do_build
> > +addtask deploy before do_build after do_install
> >
> >  # Enable use of menuconfig directly from bitbake and also within the devshell
> >  OE_TERMINAL_EXPORTS += "HOST_EXTRACFLAGS HOSTLDFLAGS TERMINFO"
> > --
> > 2.17.1
> >
>
> >
>
> 



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II