[PATCH 0/1] command: Add command to invalidate parse cache

[PATCH 0/1] command: Add command to invalidate parse cache

[Paul Eggleton]

This is required by the OE-Core patch I'm about to send to fix
devtool's lack of handling dependencies during source extraction.


The following changes since commit ed3aab90dbf9b6efefd9eb66d81123cb94d3c7da:

  toaster: update pyro to rocko as stable release (2017-09-11 11:00:11 +0100)

are available in the git repository at:

  git://git.openembedded.org/bitbake-contrib paule/parsecache-invalidate
  http://cgit.openembedded.org/bitbake-contrib/log/?h=paule/parsecache-invalidate

Paul Eggleton (1):
  command: Add command to invalidate parse cache

 lib/bb/command.py | 3 +++
 1 file changed, 3 insertions(+)

-- 
2.9.5

[PATCH 1/1] command: Add command to invalidate parse cache

[Paul Eggleton]

If we make a change to the metadata within a tinfoil-using script
and then attempt to run tasks using build_targets(), the change won't
get noticed. Add an explicit command that will set the internal flag to
force reloading the cache and thus pick up the changes. This will be
used in OE by the devtool source extraction rework.

Part of the fix for [YOCTO #11198].

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 lib/bb/command.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/bb/command.py b/lib/bb/command.py
index 6c966e3..c3717ab 100644
--- a/lib/bb/command.py
+++ b/lib/bb/command.py
@@ -574,6 +574,9 @@ class CommandsSync:
         return DataStoreConnectionHandle(idx)
     parseRecipeFile.readonly = True
 
+    def invalidateParseCache(self, command, params):
+        command.cooker.parsecache_valid = False
+
 class CommandsAsync:
     """
     A class of asynchronous commands
-- 
2.9.5

Re: [PATCH 1/1] command: Add command to invalidate parse cache

[Richard Purdie]

On Tue, 2017-09-12 at 22:10 +1200, Paul Eggleton wrote:
> If we make a change to the metadata within a tinfoil-using script
> and then attempt to run tasks using build_targets(), the change won't
> get noticed. Add an explicit command that will set the internal flag
> to
> force reloading the cache and thus pick up the changes. This will be
> used in OE by the devtool source extraction rework.
> 
> Part of the fix for [YOCTO #11198].
> 
> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> ---
>  lib/bb/command.py | 3 +++
>  1 file changed, 3 insertions(+)

Just a quick sanity check. Why won't the change get noticed? Any file
writes should be seen by the inotify code which would invalidate the
cache? This could mean we're not watching some dependency we should be?
or is it that the commands aren't triggering a reparse due to ordering
somehow? I'd like to understand this more before we add this...

Cheers,

Richard

Re: [PATCH 1/1] command: Add command to invalidate parse cache

[Paul Eggleton]

On Tuesday, 12 September 2017 10:41:36 PM NZST Richard Purdie wrote:
> On Tue, 2017-09-12 at 22:10 +1200, Paul Eggleton wrote:
> > If we make a change to the metadata within a tinfoil-using script
> > and then attempt to run tasks using build_targets(), the change won't
> > get noticed. Add an explicit command that will set the internal flag
> > to
> > force reloading the cache and thus pick up the changes. This will be
> > used in OE by the devtool source extraction rework.
> > 
> > Part of the fix for [YOCTO #11198].
> > 
> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> > ---
> >  lib/bb/command.py | 3 +++
> >  1 file changed, 3 insertions(+)
> 
> Just a quick sanity check. Why won't the change get noticed? Any file
> writes should be seen by the inotify code which would invalidate the
> cache? This could mean we're not watching some dependency we should be?
> or is it that the commands aren't triggering a reparse due to ordering
> somehow? I'd like to understand this more before we add this...

That's a good question, and you may well be right. I can look into it more 
closely.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [PATCH 1/1] command: Add command to invalidate parse cache

[Paul Eggleton]

On Tuesday, 12 September 2017 11:05:50 PM NZST Paul Eggleton wrote:
> On Tuesday, 12 September 2017 10:41:36 PM NZST Richard Purdie wrote:
> > On Tue, 2017-09-12 at 22:10 +1200, Paul Eggleton wrote:
> > > If we make a change to the metadata within a tinfoil-using script
> > > and then attempt to run tasks using build_targets(), the change won't
> > > get noticed. Add an explicit command that will set the internal flag
> > > to
> > > force reloading the cache and thus pick up the changes. This will be
> > > used in OE by the devtool source extraction rework.
> > > 
> > > Part of the fix for [YOCTO #11198].
> > > 
> > > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> > > ---
> > >  lib/bb/command.py | 3 +++
> > >  1 file changed, 3 insertions(+)
> > 
> > Just a quick sanity check. Why won't the change get noticed? Any file
> > writes should be seen by the inotify code which would invalidate the
> > cache? This could mean we're not watching some dependency we should be?
> > or is it that the commands aren't triggering a reparse due to ordering
> > somehow? I'd like to understand this more before we add this...
> 
> That's a good question, and you may well be right. I can look into it more 
> closely.

You were right - disregard this patch, I'm about to send a new series with a 
proper fix.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre