Re: [PATCH] bitbake: invalidate mtime cache if file doesn't exist

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Tue, 2015-04-14 at 19:42 +0300, Ed Bartosh wrote:
> Mtime cache is desinged with assumption that files are not
> removed. Unfortunately it's not always the case for memory-resident
> bitbake as build/tmp can be removed to perform build from scratch.
> In this case bitbake crashes when trying to create timestamps if
> tmp/stamps/ hierarchy doesn't exist.
> 
> Simple check of file existance should solve this issue.
> 
> [YOCTO: #7562]
> 
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> ---
>  bitbake/lib/bb/parse/__init__.py | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/bitbake/lib/bb/parse/__init__.py b/bitbake/lib/bb/parse/__init__.py
> index 25effc2..e880732 100644
> --- a/bitbake/lib/bb/parse/__init__.py
> +++ b/bitbake/lib/bb/parse/__init__.py
> @@ -62,6 +62,8 @@ def cached_mtime(f):
>      return __mtime_cache[f]
>  
>  def cached_mtime_noerror(f):
> +    if not os.path.exists(f):
> +        return 0
>      if f not in __mtime_cache:
>          try:
>              __mtime_cache[f] = os.stat(f)[stat.ST_MTIME]

I don't think this is right. The whole idea of this cache is so we don't
repeatedly hit stat() syscalls and by adding in the exists() check, the
cache is made worthless and I suspect we'd see a bad performance change.

Most of the use of this cache is from cooker/cache.py and there, we use
inotify to refresh the cache if something in the filesystem changes. We
probably need to teach the other users of this cache about the need to
deal with inotify (or simply remove any stamp data from the cache
entirely at the start of each bitbake invocation).

Cheers,

Richard