Re: [PATCH] classes/externalsrc: Don't delete tasks

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Fri, 2018-01-12 at 10:24 -0600, Joshua Watt wrote:
> Set the noexec flag to prevent tasks from executing instead of
> deleting
> them. This allows inter-tasks dependencies on these tasks to still
> function. For example, perf has the line:
> 
>  do_populate_lic[depends] += "virtual/kernel:do_patch"
> 
> which will break if the kernel uses EXTERNALSRC and the do_patch task
> is deleted.
> 
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
>  meta/classes/externalsrc.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/externalsrc.bbclass
> b/meta/classes/externalsrc.bbclass
> index 65dd13ddc1f..12046d74e17 100644
> --- a/meta/classes/externalsrc.bbclass
> +++ b/meta/classes/externalsrc.bbclass
> @@ -102,7 +102,7 @@ python () {
>          for task in d.getVar("SRCTREECOVEREDTASKS").split():
>              if local_srcuri and task in fetch_tasks:
>                  continue
> -            bb.build.deltask(task, d)
> +            d.setVarFlag(task, 'noexec', '1')
>  
>          d.prependVarFlag('do_compile', 'prefuncs',
> "externalsrc_compile_prefunc ")
>          d.prependVarFlag('do_configure', 'prefuncs',
> "externalsrc_configure_prefunc ")

There are a few things I worry about with this:

a) Its confusing for users as it will now show a fetch/unpack/patch
task as part of the dependency tree but they won't actually run.

b) Dependencies for those tasks will be followed

c) There has always been a bit of ambiguity about what should happen
with task properties like cleandirs, dirs and others. Whilst these
don't run for noexec, I have worries that somehow we'll end up wiping
out the external source dir accidentally (e.g. with do_clean). I think
the noexec move makes this more likely, sadly.

externalsrc started out as a 30 line class back in 2012 and its now 235
lines and getting ridiculously complicated.

I'm starting to wonder if we should change the approach and have it
modify SRC_URI and take a copy of the source. Yes, it'd be fractionally
slower but its getting far too complex trying to handle all the corner
cases. It was meant so developers could quickly hack/test things and
its turning into a full development tool/workflow.

I think we do need to take a step back and try and come up with a way
to simplify it...

We should also think about whether there are changes to the core which
would make things like this easier. I do have a fairly strong belief
that we should make the system more robust and able to handle tasks
being missing, its now a lot better at handling that than it used to
be. Chris as talked about a "srcready" type task and reworking the
fetch/unpack/patch process  and I can see the reasoning for that, it
would help here too.

Not sure whether Paul has any thoughts on this as externalsrc will
significantly impact devtool too...

Cheers,

Richard