sstate-cache contains many seemingly useless siginfo files

sstate-cache contains many seemingly useless siginfo files

[Mike Crowe]

Our sstate-cache cleanup script was written long before
80b3974081c4a8c604e23982a6db8fb32c616058, so it only pruned siginfo files
if the corresponding tgz file had not been accessed recently.

This worked well until siginfo files started being written for every task -
even those that didn't also generate a tgz file such as unpack, configure
and compile.

I've just cleared up over two million siginfo files from our sstate-cache!

This exercise has left me wondering why these siginfo files are being
written to the sstate-cache in the first place.
80b3974081c4a8c604e23982a6db8fb32c616058 suggests that they aren't being
used by anyone.

If writing the files is necessary, is there any reason not to just delete
these files periodically or could that confuse a build that is in progress?

Thanks.

Mike.

Re: sstate-cache contains many seemingly useless siginfo files

[Richard Purdie]

On Mon, 2016-02-01 at 16:59 +0000, Mike Crowe wrote:
> Our sstate-cache cleanup script was written long before
> 80b3974081c4a8c604e23982a6db8fb32c616058, so it only pruned siginfo
> files
> if the corresponding tgz file had not been accessed recently.
> 
> This worked well until siginfo files started being written for every
> task -
> even those that didn't also generate a tgz file such as unpack,
> configure
> and compile.
> 
> I've just cleared up over two million siginfo files from our sstate
> -cache!
> 
> This exercise has left me wondering why these siginfo files are being
> written to the sstate-cache in the first place.
> 80b3974081c4a8c604e23982a6db8fb32c616058 suggests that they aren't
> being
> used by anyone.

These are used by things like "bitbake -S printdiff" in order to debug
why things are being rebuilt. Without them, there are gaps in the
dependency chains and the tools can't figure out how things changed.

So they're not used by main builds but are useful for debug.

> If writing the files is necessary, is there any reason not to just
> delete
> these files periodically or could that confuse a build that is in
> progress?

I doubt it would cause a problem, other than meaning printdiff wouldn't
work well. Not that it works brilliantly right now, but I do hope to
fix that at some point.

Cheers,

Richard

Re: sstate-cache contains many seemingly useless siginfo files

[Mike Crowe]

On Monday 01 February 2016 at 17:52:31 +0000, Richard Purdie wrote:
> On Mon, 2016-02-01 at 16:59 +0000, Mike Crowe wrote:
> > Our sstate-cache cleanup script was written long before
> > 80b3974081c4a8c604e23982a6db8fb32c616058, so it only pruned siginfo
> > files
> > if the corresponding tgz file had not been accessed recently.
> > 
> > This worked well until siginfo files started being written for every
> > task -
> > even those that didn't also generate a tgz file such as unpack,
> > configure
> > and compile.
> > 
> > I've just cleared up over two million siginfo files from our sstate
> > -cache!
> > 
> > This exercise has left me wondering why these siginfo files are being
> > written to the sstate-cache in the first place.
> > 80b3974081c4a8c604e23982a6db8fb32c616058 suggests that they aren't
> > being
> > used by anyone.
> 
> These are used by things like "bitbake -S printdiff" in order to debug
> why things are being rebuilt. Without them, there are gaps in the
> dependency chains and the tools can't figure out how things changed.
> 
> So they're not used by main builds but are useful for debug.

It sounds like keeping them around would be useful, but since they aren't
touched during a normal build they would be expired from the sstate-cache
while they are still current.

But, perhaps running "bitbake -S printdiff" periodically would be
sufficient to ensure that they are accessed and won't expire early. I'll
try doing that.

Thanks.

Mike.