Re: [PATCH] classes/externalsrc: Don't delete tasks

[Joshua Watt <jpewhacker@gmail.com>]

On Sun, 2018-01-14 at 10:43 +0000, Richard Purdie wrote:
> On Fri, 2018-01-12 at 10:24 -0600, Joshua Watt wrote:
> > Set the noexec flag to prevent tasks from executing instead of
> > deleting
> > them. This allows inter-tasks dependencies on these tasks to still
> > function. For example, perf has the line:
> > 
> >  do_populate_lic[depends] += "virtual/kernel:do_patch"
> > 
> > which will break if the kernel uses EXTERNALSRC and the do_patch
> > task
> > is deleted.
> > 
> > Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> > ---
> >  meta/classes/externalsrc.bbclass | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/meta/classes/externalsrc.bbclass
> > b/meta/classes/externalsrc.bbclass
> > index 65dd13ddc1f..12046d74e17 100644
> > --- a/meta/classes/externalsrc.bbclass
> > +++ b/meta/classes/externalsrc.bbclass
> > @@ -102,7 +102,7 @@ python () {
> >          for task in d.getVar("SRCTREECOVEREDTASKS").split():
> >              if local_srcuri and task in fetch_tasks:
> >                  continue
> > -            bb.build.deltask(task, d)
> > +            d.setVarFlag(task, 'noexec', '1')
> >  
> >          d.prependVarFlag('do_compile', 'prefuncs',
> > "externalsrc_compile_prefunc ")
> >          d.prependVarFlag('do_configure', 'prefuncs',
> > "externalsrc_configure_prefunc ")
> 
> There are a few things I worry about with this:
> 
> a) Its confusing for users as it will now show a fetch/unpack/patch
> task as part of the dependency tree but they won't actually run.
> 
> b) Dependencies for those tasks will be followed
> 
> c) There has always been a bit of ambiguity about what should happen
> with task properties like cleandirs, dirs and others. Whilst these
> don't run for noexec, I have worries that somehow we'll end up wiping
> out the external source dir accidentally (e.g. with do_clean). I
> think
> the noexec move makes this more likely, sadly.
> 
> externalsrc started out as a 30 line class back in 2012 and its now
> 235
> lines and getting ridiculously complicated.
> 
> I'm starting to wonder if we should change the approach and have it
> modify SRC_URI and take a copy of the source. Yes, it'd be
> fractionally
> slower but its getting far too complex trying to handle all the
> corner
> cases. It was meant so developers could quickly hack/test things and
> its turning into a full development tool/workflow.

Ya, that seems reasonable. I think the tricky part is getting the data
copied over in such a way that incremental builds still work (which is
99% of the time the reason I use externalsrc), since some build systems
actually care about the timestamps. Hmmm.... maybe something with rsync
would work.... although, it would be a bit of a strange host tool
dependency. 

> 
> I think we do need to take a step back and try and come up with a way
> to simplify it...
> 
> We should also think about whether there are changes to the core
> which
> would make things like this easier. I do have a fairly strong belief
> that we should make the system more robust and able to handle tasks
> being missing, its now a lot better at handling that than it used to
> be. Chris as talked about a "srcready" type task and reworking the
> fetch/unpack/patch process  and I can see the reasoning for that, it
> would help here too.

Another idea I had (dunno if it is actually *good*): perhaps using
EXTERNALSRC should add some thing to OVERRIDES when it is enabled? That
way at least the corner cases can be codified close to the code they
modify instead of having to live generically in externalsrc.bbclass

> 
> Not sure whether Paul has any thoughts on this as externalsrc will
> significantly impact devtool too...

I don't use devtool much, but I do use externalsrc all the time. If I
get a bit of down time, I might try to take a look at it.

> 
> Cheers,
> 
> Richard
> 
> 
> 
>