orm_variable.changed

orm_variable.changed

[Barros Pena, Belen]

In the early days of the Toaster project, Yocto Project users we spoke
wanted a way to narrow down which variables could be the source of a build
failure. For example, if my latest build failed, but the previous one
completed successfully, it is reasonable to assume that the cause of the
failure is in some variable I've changed.

To cater for this need, we added a 'changed' field to our db variables
table, but we haven't worked out what 'changed' really means or how to
collect the data. 

Users brought up 2 possible meanings of 'changed':


1. Changed from the Yocto Project default value (the value the variable
has when you download / clone a stable release of the Yocto Project)
2. Changed since the last build

Is it possible to collect any of the 2?

Thanks,

Belén

Re: orm_variable.changed

[Richard Purdie]

On Mon, 2013-12-23 at 11:11 +0000, Barros Pena, Belen wrote:
> In the early days of the Toaster project, Yocto Project users we spoke
> wanted a way to narrow down which variables could be the source of a build
> failure. For example, if my latest build failed, but the previous one
> completed successfully, it is reasonable to assume that the cause of the
> failure is in some variable I've changed.
> 
> To cater for this need, we added a 'changed' field to our db variables
> table, but we haven't worked out what 'changed' really means or how to
> collect the data. 
> 
> Users brought up 2 possible meanings of 'changed':
> 
> 
> 1. Changed from the Yocto Project default value (the value the variable
> has when you download / clone a stable release of the Yocto Project)
> 2. Changed since the last build
> 
> Is it possible to collect any of the 2?

Changes in variables is effectively what we've been talking about when
we've talked about sstate signature differences and how to visualise
those, its effectively what diffsigs is about.

I'm not sure a simple column is going to do this justice.

In this case I think you're talking just about the base metadata and not
the task level variables though which I guess would simplify things a
bit...

Cheers,

Richard

Re: orm_variable.changed

[Damian, Alexandru]

[-- Attachment #1: Type: text/plain, Size: 2043 bytes --]

On Mon, Dec 23, 2013 at 11:36 AM, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Mon, 2013-12-23 at 11:11 +0000, Barros Pena, Belen wrote:
> > In the early days of the Toaster project, Yocto Project users we spoke
> > wanted a way to narrow down which variables could be the source of a
> build
> > failure. For example, if my latest build failed, but the previous one
> > completed successfully, it is reasonable to assume that the cause of the
> > failure is in some variable I've changed.
> >
> > To cater for this need, we added a 'changed' field to our db variables
> > table, but we haven't worked out what 'changed' really means or how to
> > collect the data.
> >
> > Users brought up 2 possible meanings of 'changed':
> >
> >
> > 1. Changed from the Yocto Project default value (the value the variable
> > has when you download / clone a stable release of the Yocto Project)
> > 2. Changed since the last build
> >
> > Is it possible to collect any of the 2?
>
> Changes in variables is effectively what we've been talking about when
> we've talked about sstate signature differences and how to visualise
> those, its effectively what diffsigs is about.
>
> I'm not sure a simple column is going to do this justice.
>

Actually, a single column doesn't help, especially when we have no idea
what it actually means.


>
> In this case I think you're talking just about the base metadata and not
> the task level variables though which I guess would simplify things a
> bit...
>

Yep, this is about base metadata. We can show the a single variable value
across a number of builds,
or do base metadata diff between two builds, if that helps.

We are not tracking task level variables, not sure if this is a problem or
not.


>
> Cheers,
>
> Richard
>
>
>
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
>



-- 
Alex Damian
Yocto Project
SSG / OTC

[-- Attachment #2: Type: text/html, Size: 3122 bytes --]

Re: orm_variable.changed

[Barros Pena, Belen]

On 23/12/2013 13:16, "Damian, Alexandru" <alexandru.damian@intel.com>
wrote:

>
>On Mon, Dec 23, 2013 at 11:36 AM, Richard Purdie
><richard.purdie@linuxfoundation.org> wrote:
>
>On Mon, 2013-12-23 at 11:11 +0000, Barros Pena, Belen wrote:
>> In the early days of the Toaster project, Yocto Project users we spoke
>> wanted a way to narrow down which variables could be the source of a
>>build
>> failure. For example, if my latest build failed, but the previous one
>> completed successfully, it is reasonable to assume that the cause of the
>> failure is in some variable I've changed.
>>
>> To cater for this need, we added a 'changed' field to our db variables
>> table, but we haven't worked out what 'changed' really means or how to
>> collect the data.
>>
>> Users brought up 2 possible meanings of 'changed':
>>
>>
>> 1. Changed from the Yocto Project default value (the value the variable
>> has when you download / clone a stable release of the Yocto Project)
>> 2. Changed since the last build
>>
>> Is it possible to collect any of the 2?
>
>
>Changes in variables is effectively what we've been talking about when
>we've talked about sstate signature differences and how to visualise
>those, its effectively what diffsigs is about.
>
>I'm not sure a simple column is going to do this justice.

Most likely this won¹t be just a single column.

>
>Actually, a single column doesn't help, especially when we have no idea
>what it actually means.

³What it actually means² is what are trying to work out here. And I
respectfully disagree: a single column does help. That humble column can
make a great entry point to the information I am interested in.

>
>In this case I think you're talking just about the base metadata and not
>the task level variables though which I guess would simplify things a
>bit...
>
>
>Yep, this is about base metadata. We can show the a single variable value
>across a number of builds,
>
>or do base metadata diff between two builds, if that helps.

So Œchanged¹ kind of means "Changed since the last build², kind of because
we don¹t need to limit ourselves to the last build, but potentially could
offer:

* the ability to choose which 2 builds, and also
* to see a full history of the variable across builds recorded by Toaster.

We have an issue in Bugzilla set to Œneed info¹ for this:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=5226

Any ideas / suggestions you might have about how to collect the
information would be very helpful.

Thanks

Belén

>
>We are not tracking task level variables, not sure if this is a problem
>or not.
>
> 
>
>
>Cheers,
>
>Richard
>
>
>
>_______________________________________________
>toaster mailing list
>toaster@yoctoproject.org
>https://lists.yoctoproject.org/listinfo/toaster
>
>
>
>
>
>
>
>
>-- 
>Alex Damian
>Yocto Project
>
>SSG / OTC 
>
>
>

Re: orm_variable.changed

[Damian, Alexandru]

[-- Attachment #1: Type: text/plain, Size: 3553 bytes --]

For diffs between builds and history of a variable for the base metadata,
we already have all the info we need.

I'm not sure what "previous build" means - if the user changes machines, or
targets, how do you find the "previous" build ?




On Mon, Dec 23, 2013 at 1:49 PM, Barros Pena, Belen <
belen.barros.pena@intel.com> wrote:

>
>
> On 23/12/2013 13:16, "Damian, Alexandru" <alexandru.damian@intel.com>
> wrote:
>
> >
> >On Mon, Dec 23, 2013 at 11:36 AM, Richard Purdie
> ><richard.purdie@linuxfoundation.org> wrote:
> >
> >On Mon, 2013-12-23 at 11:11 +0000, Barros Pena, Belen wrote:
> >> In the early days of the Toaster project, Yocto Project users we spoke
> >> wanted a way to narrow down which variables could be the source of a
> >>build
> >> failure. For example, if my latest build failed, but the previous one
> >> completed successfully, it is reasonable to assume that the cause of the
> >> failure is in some variable I've changed.
> >>
> >> To cater for this need, we added a 'changed' field to our db variables
> >> table, but we haven't worked out what 'changed' really means or how to
> >> collect the data.
> >>
> >> Users brought up 2 possible meanings of 'changed':
> >>
> >>
> >> 1. Changed from the Yocto Project default value (the value the variable
> >> has when you download / clone a stable release of the Yocto Project)
> >> 2. Changed since the last build
> >>
> >> Is it possible to collect any of the 2?
> >
> >
> >Changes in variables is effectively what we've been talking about when
> >we've talked about sstate signature differences and how to visualise
> >those, its effectively what diffsigs is about.
> >
> >I'm not sure a simple column is going to do this justice.
>
> Most likely this won¹t be just a single column.
>
> >
> >Actually, a single column doesn't help, especially when we have no idea
> >what it actually means.
>
> ³What it actually means² is what are trying to work out here. And I
> respectfully disagree: a single column does help. That humble column can
> make a great entry point to the information I am interested in.
>
> >
> >In this case I think you're talking just about the base metadata and not
> >the task level variables though which I guess would simplify things a
> >bit...
> >
> >
> >Yep, this is about base metadata. We can show the a single variable value
> >across a number of builds,
> >
> >or do base metadata diff between two builds, if that helps.
>
> So Œchanged¹ kind of means "Changed since the last build², kind of because
> we don¹t need to limit ourselves to the last build, but potentially could
> offer:
>
> * the ability to choose which 2 builds, and also
> * to see a full history of the variable across builds recorded by Toaster.
>
> We have an issue in Bugzilla set to Œneed info¹ for this:
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=5226
>
> Any ideas / suggestions you might have about how to collect the
> information would be very helpful.
>
> Thanks
>
> Belén
>
> >
> >We are not tracking task level variables, not sure if this is a problem
> >or not.
> >
> >
> >
> >
> >Cheers,
> >
> >Richard
> >
> >
> >
> >_______________________________________________
> >toaster mailing list
> >toaster@yoctoproject.org
> >https://lists.yoctoproject.org/listinfo/toaster
> >
> >
> >
> >
> >
> >
> >
> >
> >--
> >Alex Damian
> >Yocto Project
> >
> >SSG / OTC
> >
> >
> >
>
>


-- 
Alex Damian
Yocto Project
SSG / OTC

[-- Attachment #2: Type: text/html, Size: 4994 bytes --]

Re: orm_variable.changed

[Barros Pena, Belen]

On 23/12/2013 14:07, "Damian, Alexandru" <alexandru.damian@intel.com>
wrote:

>For diffs between builds and history of a variable for the base metadata,
>we already have all the info we need.
>
>I'm not sure what "previous build" means - if the user changes machines,
>or targets, how do you find the "previous" build ?

I can think of 2 answers (there might be more):

* Previous build is just the latest build you ran before the one you are
looking at (based on completed on time). This is simple, but might make
for nonsensical comparisons.

* Previous build is the latest build you ran before the one you are
looking at (based on completed on time) that has certain similarities with
the one you are looking at. I am not sure what those similarities should
be: Same architecture? Same machine? Same target? You tell me :)

Of course, the set of builds used to obtain the *previous build* can only
be builds recorded by Toaster.

>
>
>
>
>
>On Mon, Dec 23, 2013 at 1:49 PM, Barros Pena, Belen
><belen.barros.pena@intel.com> wrote:
>
>
>
>On 23/12/2013 13:16, "Damian, Alexandru" <alexandru.damian@intel.com>
>wrote:
>
>>
>>On Mon, Dec 23, 2013 at 11:36 AM, Richard Purdie
>><richard.purdie@linuxfoundation.org> wrote:
>>
>>On Mon, 2013-12-23 at 11:11 +0000, Barros Pena, Belen wrote:
>>> In the early days of the Toaster project, Yocto Project users we spoke
>>> wanted a way to narrow down which variables could be the source of a
>>>build
>>> failure. For example, if my latest build failed, but the previous one
>>> completed successfully, it is reasonable to assume that the cause of
>>>the
>>> failure is in some variable I've changed.
>>>
>>> To cater for this need, we added a 'changed' field to our db variables
>>> table, but we haven't worked out what 'changed' really means or how to
>>> collect the data.
>>>
>>> Users brought up 2 possible meanings of 'changed':
>>>
>>>
>>> 1. Changed from the Yocto Project default value (the value the variable
>>> has when you download / clone a stable release of the Yocto Project)
>>> 2. Changed since the last build
>>>
>>> Is it possible to collect any of the 2?
>>
>>
>>Changes in variables is effectively what we've been talking about when
>>we've talked about sstate signature differences and how to visualise
>>those, its effectively what diffsigs is about.
>>
>>I'm not sure a simple column is going to do this justice.
>
>
>Most likely this won¹t be just a single column.
>
>>
>>Actually, a single column doesn't help, especially when we have no idea
>>what it actually means.
>
>
>³What it actually means² is what are trying to work out here. And I
>respectfully disagree: a single column does help. That humble column can
>make a great entry point to the information I am interested in.
>
>>
>>In this case I think you're talking just about the base metadata and not
>>the task level variables though which I guess would simplify things a
>>bit...
>>
>>
>>Yep, this is about base metadata. We can show the a single variable value
>>across a number of builds,
>>
>>or do base metadata diff between two builds, if that helps.
>
>
>So Œchanged¹ kind of means "Changed since the last build², kind of because
>we don¹t need to limit ourselves to the last build, but potentially could
>offer:
>
>* the ability to choose which 2 builds, and also
>* to see a full history of the variable across builds recorded by Toaster.
>
>We have an issue in Bugzilla set to Œneed info¹ for this:
>
>https://bugzilla.yoctoproject.org/show_bug.cgi?id=5226
>
>Any ideas / suggestions you might have about how to collect the
>information would be very helpful.
>
>Thanks
>
>Belén
>
>>
>>We are not tracking task level variables, not sure if this is a problem
>>or not.
>>
>>
>>
>>
>>Cheers,
>>
>>Richard
>>
>>
>>
>>_______________________________________________
>>toaster mailing list
>>toaster@yoctoproject.org
>>https://lists.yoctoproject.org/listinfo/toaster
>>
>>
>>
>>
>>
>>
>>
>>
>>--
>>Alex Damian
>>Yocto Project
>>
>>SSG / OTC
>>
>>
>>
>
>
>
>
>
>
>
>
>
>-- 
>Alex Damian
>Yocto Project
>
>SSG / OTC 
>
>

Re: orm_variable.changed

[Barros Pena, Belen]

On 23/12/2013 13:16, "Damian, Alexandru" <alexandru.damian@intel.com>
wrote:

>We are not tracking task level variables, not sure if this is a problem
>or not.

Sorry for the noise, but just remembered this:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=5255#c5

Wouldn¹t "turning on variable history for all tasks and saving the
variable history for the executed task function² somehow track task level
variables?
 

Re: orm_variable.changed

[Damian, Alexandru]

[-- Attachment #1: Type: text/plain, Size: 921 bytes --]

Yep, it would. I am still evaluating the performance impact of turning on
history for all variables.
My latest data shows a consistent 5% time increase across various builds,
I've sent a mail about this. Not sure if this acceptable.

I have no other good solution to track task level variables.

Alex


On Mon, Dec 23, 2013 at 4:10 PM, Barros Pena, Belen <
belen.barros.pena@intel.com> wrote:

> On 23/12/2013 13:16, "Damian, Alexandru" <alexandru.damian@intel.com>
> wrote:
>
> >We are not tracking task level variables, not sure if this is a problem
> >or not.
>
> Sorry for the noise, but just remembered this:
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=5255#c5
>
> Wouldn¹t "turning on variable history for all tasks and saving the
> variable history for the executed task function² somehow track task level
> variables?
>
>
>


-- 
Alex Damian
Yocto Project
SSG / OTC

[-- Attachment #2: Type: text/html, Size: 1584 bytes --]