Re: [PATCH] Modify buildstats to be merged inside buildhistory

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Mon, 2013-11-04 at 09:24 +0000, Stoicescu, CorneliuX wrote:
> > -----Original Message-----
> > From: Richard Purdie [mailto:richard.purdie@linuxfoundation.org]
> > Sent: Monday, November 04, 2013 11:16 AM
> > To: Stoicescu, CorneliuX
> > Cc: openembedded-core@lists.openembedded.org
> > Subject: Re: [OE-core] [PATCH] Modify buildstats to be merged inside
> > buildhistory
> > 
> > On Sun, 2013-11-03 at 11:37 +0000, Stoicescu, CorneliuX wrote:
> > > > > +BUILDSTATS_BASE = "${BUILDHISTORY_DIR}/buildstats"
> > > > >  BNFILE = "${BUILDSTATS_BASE}/.buildname"
> > > > >  DEVFILE = "${BUILDSTATS_BASE}/.device"
> > > >
> > > > so when buildhistory is used, buildstats automatically becomes part of it?
> > > >
> > > > > @@ -33,7 +33,7 @@ def set_bn(e):
> > > > >
> > > > >  def get_bn(e):
> > > > >      with open(e.data.getVar('BNFILE', True)) as f:
> > > > > -        bn = f.readline()
> > > > > +        bn = str(f.readline()).split("/")[0]
> > > > >      return bn
> > > > >
> > > > >  def set_device(e):
> > > > > @@ -175,6 +175,10 @@ python run_buildstats () {
> > > > >          # set the buildname
> > > > >
> > > >
> > ##############################################################
> > > > ##########
> > > > >          try:
> > > > > +            bb.utils.remove(e.data.getVar('BUILDSTATS_BASE',
> > > > > + True),
> > > > > + recurse=True)
> > > >
> > > >
> > > > Do we need to remove this?
> > > >
> > >
> > > By reworking the buildstats directory structure to not use the build
> > > date to separate builds, if we don't remove buildstats each time we
> > > make a new build the information from multiple builds will merge and
> > > tools like bybootchartguy.py will not work anymore with buildstats.
> > 
> > I'd have expected that separate directories for each build are still
> > maintained. Testing locally here, I can use pybootchartgui in the form:
> > 
> > scripts/pybootchartgui/pybootchartgui.py build/tmp/buildstats/core-image-
> > sato-qemux86/201309302155/
> > 
> > and I'd have thought a similar command would still work in buildhistory since
> > we'd still have the timestamped directory there?
> > 
> 
> The timestamped directory was removed in order to reduce buildhistory git noise. 
> Creating a new directory for each build will also increase the git repo size dramatically
> In time.
> 
> If we choose to not use the timestamped directory, we need to delete the buildstats 
> data each build because, in contrast to buildhistory, buildstats data is relevant when
> compiled as a whole. This is also how pybootchart interprets it.

I hadn't realised you'd removed the timestamps. If you do that there is
a problem, yes. Paul, how do you think we should handle this? I'd have
thought it would be ok to save the historical data in there side by side
since the size of the git repo will be the same, the size of the
checkout will just be different. We do need to save something into the
timestamped directory saying which revision(s) a given build happened
against though so we have some kind of historical context.

Paul: Any thoughts on this?

Cheers,

Richard