[Webhob] task types and outcomes

[Webhob] task types and outcomes

[Barros Pena, Belen]

I am looking into the BitBake task-related information we'll display in
Web Hob. Looking back to the agency work, it seems we identified 2 types
of tasks: sstate tasks and built tasks.

Sstate tasks can have 3 possible outcomes: missed, completed and failed.

Built tasks can have 3 possible outcomes: previously built, completed and
failed.

Looking at this now, I am not sure I understand the difference between a
missed sstate task and a built task, and between a previously built task
and an sstate task:

* Wouldn't a missed sstate task become a built task?
* Wouldn't a previously built task be an sstate task?

Sorry if this is a dumb question, but I am not sure if there is some
subtle difference I am missing, of if we are talking about different
'stages' of a single task's life cycle (to call it something).

Could anybody help me clarify this?

Thanks!!

Belén




---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [Webhob] task types and outcomes

[Paul Eggleton]

On Tuesday 28 May 2013 16:16:07 Barros Pena, Belen wrote:
> I am looking into the BitBake task-related information we'll display in
> Web Hob. Looking back to the agency work, it seems we identified 2 types
> of tasks: sstate tasks and built tasks.
> 
> Sstate tasks can have 3 possible outcomes: missed, completed and failed.

Correct (although I wonder if there is a further distinction between missed 
completely, and missed due to some change i.e. one or more sstate packages 
exist for the task but the signatures are different).
 
> Built tasks can have 3 possible outcomes: previously built, completed and
> failed.

Correct.
 
> Looking at this now, I am not sure I understand the difference between a
> missed sstate task and a built task, and between a previously built task
> and an sstate task:
> 
> * Wouldn't a missed sstate task become a built task?

Yes.

> * Wouldn't a previously built task be an sstate task?

If the definition of "previously built" is "previously built within the same 
build directory and not subsequently cleaned", then because there is a stamp 
present for the task, it doesn't even need to look at sstate for the task.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [Webhob] task types and outcomes

[Barros Pena, Belen]

Thanks, Paul. I guess we need to nail down what is that we are trying to
achieve by reporting sstate tasks vs. built tasks vs. previously built
tasks. From what I can remember, the main questions users need answered
are:

* Why is this a built task when I was expecting it to reuse sstate objects?
* Why is this an sstate task when I was expecting it to build from scratch?

If those are indeed the main questions we need to answer, reporting missed
sstate tasks and built tasks as one probably makes things easier. The
metaphor would be a single task with a certain life cycle, and not 2
separate tasks. So, "BitBake detected changes in the inputs for task x and
reran the task" would result in Web Hob reporting a single built task with
an sstate missed flag. If we report this case through 2 separate tasks
we'll need to link them to each other, so that users can reach one from
the other and get a full picture of what BitBake did during the build.
This is also a viable option, but it will result in a much longer table of
tasks, with 2 entries for a certain task + recipe combination.

Which way we report tasks impacts the way we structure the Web Hob tasks
table. From this classification of tasks:

* Sstate
** Missed
** Completed
** Failed

* Built
** Previously built
** Completed
** Failed

We would move to this one:

* Completed
** Previously built
** Sstate
** Built
*** Sstate missed? Y/N
*** Sstae failed? Y/N

* Failed
** Built

The first one reflects exactly what BitBake does; I would think the second
one gives me a better picture of how a certain outcome was reached.

Belen


On 28/05/2013 17:24, "Paul Eggleton" <paul.eggleton@linux.intel.com> wrote:

>On Tuesday 28 May 2013 16:16:07 Barros Pena, Belen wrote:
>> I am looking into the BitBake task-related information we'll display in
>> Web Hob. Looking back to the agency work, it seems we identified 2 types
>> of tasks: sstate tasks and built tasks.
>> 
>> Sstate tasks can have 3 possible outcomes: missed, completed and failed.
>
>Correct (although I wonder if there is a further distinction between
>missed 
>completely, and missed due to some change i.e. one or more sstate
>packages 
>exist for the task but the signatures are different).
> 
>> Built tasks can have 3 possible outcomes: previously built, completed
>>and
>> failed.
>
>Correct.
> 
>> Looking at this now, I am not sure I understand the difference between a
>> missed sstate task and a built task, and between a previously built task
>> and an sstate task:
>> 
>> * Wouldn't a missed sstate task become a built task?
>
>Yes.
>
>> * Wouldn't a previously built task be an sstate task?
>
>If the definition of "previously built" is "previously built within the
>same 
>build directory and not subsequently cleaned", then because there is a
>stamp 
>present for the task, it doesn't even need to look at sstate for the task.
>
>Cheers,
>Paul
>
>-- 
>
>Paul Eggleton
>Intel Open Source Technology Centre

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [Webhob] task types and outcomes

[Paul Eggleton]

On Tuesday 28 May 2013 17:17:35 Barros Pena, Belen wrote:
> Thanks, Paul. I guess we need to nail down what is that we are trying to
> achieve by reporting sstate tasks vs. built tasks vs. previously built
> tasks. From what I can remember, the main questions users need answered
> are:
> 
> * Why is this a built task when I was expecting it to reuse sstate objects?
> * Why is this an sstate task when I was expecting it to build from scratch?

These are common questions yes.

> If those are indeed the main questions we need to answer, reporting missed
> sstate tasks and built tasks as one probably makes things easier. The
> metaphor would be a single task with a certain life cycle, and not 2
> separate tasks. So, "BitBake detected changes in the inputs for task x and
> reran the task" would result in Web Hob reporting a single built task with
> an sstate missed flag. If we report this case through 2 separate tasks
> we'll need to link them to each other, so that users can reach one from
> the other and get a full picture of what BitBake did during the build.
> This is also a viable option, but it will result in a much longer table of
> tasks, with 2 entries for a certain task + recipe combination.
> 
> Which way we report tasks impacts the way we structure the Web Hob tasks
> table. From this classification of tasks:
> 
> * Sstate
> ** Missed
> ** Completed
> ** Failed
> 
> * Built
> ** Previously built
> ** Completed
> ** Failed
> 
> We would move to this one:
> 
> * Completed
> ** Previously built
> ** Sstate
> ** Built
> *** Sstate missed? Y/N
> *** Sstae failed? Y/N
> 
> * Failed
> ** Built

I think this is a reasonable approach. Although it does slightly obscure the 
details of the tasks bitbake is running it's going to be a bit easier for the 
user to follow with respect to the information they're most interested in.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [Webhob] task types and outcomes

[Richard Purdie]

On Wed, 2013-05-29 at 16:57 +0100, Paul Eggleton wrote:
> On Tuesday 28 May 2013 17:17:35 Barros Pena, Belen wrote:
> > Thanks, Paul. I guess we need to nail down what is that we are trying to
> > achieve by reporting sstate tasks vs. built tasks vs. previously built
> > tasks. From what I can remember, the main questions users need answered
> > are:
> > 
> > * Why is this a built task when I was expecting it to reuse sstate objects?
> > * Why is this an sstate task when I was expecting it to build from scratch?
> 
> These are common questions yes.
> 
> > If those are indeed the main questions we need to answer, reporting missed
> > sstate tasks and built tasks as one probably makes things easier. The
> > metaphor would be a single task with a certain life cycle, and not 2
> > separate tasks. So, "BitBake detected changes in the inputs for task x and
> > reran the task" would result in Web Hob reporting a single built task with
> > an sstate missed flag. If we report this case through 2 separate tasks
> > we'll need to link them to each other, so that users can reach one from
> > the other and get a full picture of what BitBake did during the build.
> > This is also a viable option, but it will result in a much longer table of
> > tasks, with 2 entries for a certain task + recipe combination.
> > 
> > Which way we report tasks impacts the way we structure the Web Hob tasks
> > table. From this classification of tasks:
> > 
> > * Sstate
> > ** Missed
> > ** Completed
> > ** Failed
> > 
> > * Built
> > ** Previously built
> > ** Completed
> > ** Failed
> > 
> > We would move to this one:
> > 
> > * Completed
> > ** Previously built
> > ** Sstate
> > ** Built
> > *** Sstate missed? Y/N
> > *** Sstae failed? Y/N
> > 
> > * Failed
> > ** Built
> 
> I think this is a reasonable approach. Although it does slightly obscure the 
> details of the tasks bitbake is running it's going to be a bit easier for the 
> user to follow with respect to the information they're most interested in.

Sorry about the delayed reply, as you can tell I have a bit of a backlog
today :(.

I hate to complicate things however there is a touch of added complexity
to consider. Imagine we have tasks A which depends upon B which depends
upon C.

In the no sstate available case, it builds C, then B then A, easy.

With sstate, it will try A first. If A is available, it will just
install A from sstate and skip B/C. If A is not available, it will try
B, install that and skip C. It will then build any missing pieces so it
might install B from sstate, then run C.

This means that giving the user meaningful numbers of tasks is hard for
example.

This can probably be presented in the above form fine, I just want to be
clear about the subtleties involved. Ideally we do want to try and
better convey to the user what happened too in this UI. It could be good
if for example we could show that we didn't run C as B was from sstate
and hence C wasn't needed.

I think the key concept we might also need to work into the UI is "dry
run". A mode where you run the build however rather than doing it, it
tells you what it would do (i.e. that it would need to run X tasks, Y
would be done from sstate). Combined with an indication of why something
didn't match, this would be extremely powerful/useful to most users.

I have no objection to the form you present above, the key detail is
more the dry run concept from the user perspective and start answering
questions like "what would bitbake do?".

Cheers,

Richard

Re: [Webhob] task types and outcomes

[Barros Pena, Belen]

On 29/05/2013 22:39, "Richard Purdie" <richard.purdie@linuxfoundation.org>
wrote:

>On Wed, 2013-05-29 at 16:57 +0100, Paul Eggleton wrote:
>> On Tuesday 28 May 2013 17:17:35 Barros Pena, Belen wrote:
>> > Thanks, Paul. I guess we need to nail down what is that we are trying
>>to
>> > achieve by reporting sstate tasks vs. built tasks vs. previously built
>> > tasks. From what I can remember, the main questions users need
>>answered
>> > are:
>> > 
>> > * Why is this a built task when I was expecting it to reuse sstate
>>objects?
>> > * Why is this an sstate task when I was expecting it to build from
>>scratch?
>> 
>> These are common questions yes.
>> 
>> > If those are indeed the main questions we need to answer, reporting
>>missed
>> > sstate tasks and built tasks as one probably makes things easier. The
>> > metaphor would be a single task with a certain life cycle, and not 2
>> > separate tasks. So, "BitBake detected changes in the inputs for task
>>x and
>> > reran the task" would result in Web Hob reporting a single built task
>>with
>> > an sstate missed flag. If we report this case through 2 separate tasks
>> > we'll need to link them to each other, so that users can reach one
>>from
>> > the other and get a full picture of what BitBake did during the build.
>> > This is also a viable option, but it will result in a much longer
>>table of
>> > tasks, with 2 entries for a certain task + recipe combination.
>> > 
>> > Which way we report tasks impacts the way we structure the Web Hob
>>tasks
>> > table. From this classification of tasks:
>> > 
>> > * Sstate
>> > ** Missed
>> > ** Completed
>> > ** Failed
>> > 
>> > * Built
>> > ** Previously built
>> > ** Completed
>> > ** Failed
>> > 
>> > We would move to this one:
>> > 
>> > * Completed
>> > ** Previously built
>> > ** Sstate
>> > ** Built
>> > *** Sstate missed? Y/N
>> > *** Sstae failed? Y/N
>> > 
>> > * Failed
>> > ** Built
>> 
>> I think this is a reasonable approach. Although it does slightly
>>obscure the 
>> details of the tasks bitbake is running it's going to be a bit easier
>>for the 
>> user to follow with respect to the information they're most interested
>>in.
>
>Sorry about the delayed reply, as you can tell I have a bit of a backlog
>today :(.
>
>I hate to complicate things however there is a touch of added complexity
>to consider.

>Imagine we have tasks A which depends upon B which depends
>upon C.
>
>In the no sstate available case, it builds C, then B then A, easy.
>
>With sstate, it will try A first. If A is available, it will just
>install A from sstate and skip B/C. If A is not available, it will try
>B, install that and skip C. It will then build any missing pieces so it
>might install B from sstate, then run C.

I see. So it seems we have also 'skipped' tasks.

>
>This means that giving the user meaningful numbers of tasks is hard for
>example.
>
>This can probably be presented in the above form fine, I just want to be
>clear about the subtleties involved. Ideally we do want to try and
>better convey to the user what happened too in this UI. It could be good
>if for example we could show that we didn't run C as B was from sstate
>and hence C wasn't needed.

I agree we should show those as well. So, it looks like we have a few
different kinds of tasks (previously built, skipped, sstate and built),
but those can be grouped into tasks that build something (the 'built'
kind, which we could call 'executed'), and those that don't build anything
(previously built, skipped and sstate, which we could call 'not
executed'). 

So for the 'all tasks' table, we could have a 'task type' column
classifying tasks as 'executed' and 'not executed'. Then, we could have an
'outcome' column that would classify tasks as 'succeeded', 'failed',
'previously built', 'sstate' and 'skipped'. That information is enough to
give me an overview of what BitBake has done, and if a task was handled in
an unexpected way (I thought it would build but it didn't, for example).

When I select a single task from that table, we could provide a 'task
history' section that would expose the BitBake process for that task (in
your example of sstate dependent tasks, for skipped task C the history
would tell me that task C was brought in by skipped task B, which in turn
was brought in by sstate task A. For sstate task A, the history would tell
me that the checksum did not change, that task A depended on skipped task
B, which path was searched to find the sstate object, and so on).

Would that work?

>
>I think the key concept we might also need to work into the UI is "dry
>run". A mode where you run the build however rather than doing it, it
>tells you what it would do (i.e. that it would need to run X tasks, Y
>would be done from sstate). Combined with an indication of why something
>didn't match, this would be extremely powerful/useful to most users.
>
>I have no objection to the form you present above, the key detail is
>more the dry run concept from the user perspective and start answering
>questions like "what would bitbake do?".

We should keep that 'dry run' mode in mind (even I can see it would be
incredibly useful), but it won't happen in 1.5 :(

>
>Cheers,
>
>Richard
>
>

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [Webhob] task types and outcomes

[Dragomir, CalinX L]

Hi everyone,

I have a couple of questions regarding the tasks if you could help me understand better what's going on in a build operation.

1. From what I've read in the e-mails so far, it seems that there are three types of tasks: built, sstate and skipped. Can you tell me what each of them mean and how are they different ?
2. It also seems that there can be multiple results for each of them. Can you detail the meaning of each final state? (failed, missed, etc.) ?
3. How can I tell what type of task is the current executed task (if it is sstate or built or anything else) ?

Thank you,
Calin

-----Original Message-----
From: Barros Pena, Belen 
Sent: Thursday, May 30, 2013 1:58 PM
To: Richard Purdie; Paul Eggleton
Cc: Damian, Alexandru; Dragomir, CalinX L; Zhang, Jessica; webhob@yoctoproject.org
Subject: Re: task types and outcomes



On 29/05/2013 22:39, "Richard Purdie" <richard.purdie@linuxfoundation.org>
wrote:

>On Wed, 2013-05-29 at 16:57 +0100, Paul Eggleton wrote:
>> On Tuesday 28 May 2013 17:17:35 Barros Pena, Belen wrote:
>> > Thanks, Paul. I guess we need to nail down what is that we are 
>> > trying
>>to
>> > achieve by reporting sstate tasks vs. built tasks vs. previously 
>> > built tasks. From what I can remember, the main questions users 
>> > need
>>answered
>> > are:
>> > 
>> > * Why is this a built task when I was expecting it to reuse sstate
>>objects?
>> > * Why is this an sstate task when I was expecting it to build from
>>scratch?
>> 
>> These are common questions yes.
>> 
>> > If those are indeed the main questions we need to answer, reporting
>>missed
>> > sstate tasks and built tasks as one probably makes things easier. 
>> > The metaphor would be a single task with a certain life cycle, and 
>> > not 2 separate tasks. So, "BitBake detected changes in the inputs 
>> > for task
>>x and
>> > reran the task" would result in Web Hob reporting a single built 
>> > task
>>with
>> > an sstate missed flag. If we report this case through 2 separate 
>> > tasks we'll need to link them to each other, so that users can 
>> > reach one
>>from
>> > the other and get a full picture of what BitBake did during the build.
>> > This is also a viable option, but it will result in a much longer
>>table of
>> > tasks, with 2 entries for a certain task + recipe combination.
>> > 
>> > Which way we report tasks impacts the way we structure the Web Hob
>>tasks
>> > table. From this classification of tasks:
>> > 
>> > * Sstate
>> > ** Missed
>> > ** Completed
>> > ** Failed
>> > 
>> > * Built
>> > ** Previously built
>> > ** Completed
>> > ** Failed
>> > 
>> > We would move to this one:
>> > 
>> > * Completed
>> > ** Previously built
>> > ** Sstate
>> > ** Built
>> > *** Sstate missed? Y/N
>> > *** Sstae failed? Y/N
>> > 
>> > * Failed
>> > ** Built
>> 
>> I think this is a reasonable approach. Although it does slightly 
>>obscure the  details of the tasks bitbake is running it's going to be 
>>a bit easier for the  user to follow with respect to the information 
>>they're most interested in.
>
>Sorry about the delayed reply, as you can tell I have a bit of a 
>backlog today :(.
>
>I hate to complicate things however there is a touch of added 
>complexity to consider.

>Imagine we have tasks A which depends upon B which depends upon C.
>
>In the no sstate available case, it builds C, then B then A, easy.
>
>With sstate, it will try A first. If A is available, it will just 
>install A from sstate and skip B/C. If A is not available, it will try 
>B, install that and skip C. It will then build any missing pieces so it 
>might install B from sstate, then run C.

I see. So it seems we have also 'skipped' tasks.

>
>This means that giving the user meaningful numbers of tasks is hard for 
>example.
>
>This can probably be presented in the above form fine, I just want to 
>be clear about the subtleties involved. Ideally we do want to try and 
>better convey to the user what happened too in this UI. It could be 
>good if for example we could show that we didn't run C as B was from 
>sstate and hence C wasn't needed.

I agree we should show those as well. So, it looks like we have a few different kinds of tasks (previously built, skipped, sstate and built), but those can be grouped into tasks that build something (the 'built'
kind, which we could call 'executed'), and those that don't build anything (previously built, skipped and sstate, which we could call 'not executed'). 

So for the 'all tasks' table, we could have a 'task type' column classifying tasks as 'executed' and 'not executed'. Then, we could have an 'outcome' column that would classify tasks as 'succeeded', 'failed', 'previously built', 'sstate' and 'skipped'. That information is enough to give me an overview of what BitBake has done, and if a task was handled in an unexpected way (I thought it would build but it didn't, for example).

When I select a single task from that table, we could provide a 'task history' section that would expose the BitBake process for that task (in your example of sstate dependent tasks, for skipped task C the history would tell me that task C was brought in by skipped task B, which in turn was brought in by sstate task A. For sstate task A, the history would tell me that the checksum did not change, that task A depended on skipped task B, which path was searched to find the sstate object, and so on).

Would that work?

>
>I think the key concept we might also need to work into the UI is "dry 
>run". A mode where you run the build however rather than doing it, it 
>tells you what it would do (i.e. that it would need to run X tasks, Y 
>would be done from sstate). Combined with an indication of why 
>something didn't match, this would be extremely powerful/useful to most users.
>
>I have no objection to the form you present above, the key detail is 
>more the dry run concept from the user perspective and start answering 
>questions like "what would bitbake do?".

We should keep that 'dry run' mode in mind (even I can see it would be incredibly useful), but it won't happen in 1.5 :(

>
>Cheers,
>
>Richard
>
>

Re: [Webhob] task types and outcomes

[Barros Pena, Belen]

Hi Calin,

I think I can answer 1 and 2 (see below). Hopefully Alex, Richard or Paul
can help with 3.

On 30/05/2013 14:10, "Dragomir, CalinX L" <calinx.l.dragomir@intel.com>
wrote:

>Hi everyone,
>
>I have a couple of questions regarding the tasks if you could help me
>understand better what's going on in a build operation.
>
>1. From what I've read in the e-mails so far, it seems that there are
>three types of tasks: built, sstate and skipped. Can you tell me what
>each of them mean and how are they different ?

In my last email I suggested we present 2 types of tasks:

1) Executed tasks: these are tasks that build something
2) Not executed tasks: these are tasks that do not build anything

>2. It also seems that there can be multiple results for each of them. Can
>you detail the meaning of each final state? (failed, missed, etc.) ?

I've suggested the following results for each task type:

1) Succeeded: these are executed tasks that complete
2) Failed: these are executed tasks that fail
3) Previously built: these are not executed tasks for which BitBake finds
a stamp file (I think: PaulE can correct me if I am wrong here)
4) Sstate: these are not executed tasks for which BitBake successfully
uses sstate objects. If BitBake tries to use sstate objects but fails,
that task will be reported as an executed task. A task history will show
the failed sstate attempt.
5) Skipped: these are not executed tasks brought in by sstate tasks.

I think this might cover everything. If it doesn't, let me know.

>3. How can I tell what type of task is the current executed task (if it
>is sstate or built or anything else) ?

>
>Thank you,
>Calin
>
>-----Original Message-----
>From: Barros Pena, Belen
>Sent: Thursday, May 30, 2013 1:58 PM
>To: Richard Purdie; Paul Eggleton
>Cc: Damian, Alexandru; Dragomir, CalinX L; Zhang, Jessica;
>webhob@yoctoproject.org
>Subject: Re: task types and outcomes
>
>
>
>On 29/05/2013 22:39, "Richard Purdie" <richard.purdie@linuxfoundation.org>
>wrote:
>
>>On Wed, 2013-05-29 at 16:57 +0100, Paul Eggleton wrote:
>>> On Tuesday 28 May 2013 17:17:35 Barros Pena, Belen wrote:
>>> > Thanks, Paul. I guess we need to nail down what is that we are
>>> > trying
>>>to
>>> > achieve by reporting sstate tasks vs. built tasks vs. previously
>>> > built tasks. From what I can remember, the main questions users
>>> > need
>>>answered
>>> > are:
>>> > 
>>> > * Why is this a built task when I was expecting it to reuse sstate
>>>objects?
>>> > * Why is this an sstate task when I was expecting it to build from
>>>scratch?
>>> 
>>> These are common questions yes.
>>> 
>>> > If those are indeed the main questions we need to answer, reporting
>>>missed
>>> > sstate tasks and built tasks as one probably makes things easier.
>>> > The metaphor would be a single task with a certain life cycle, and
>>> > not 2 separate tasks. So, "BitBake detected changes in the inputs
>>> > for task
>>>x and
>>> > reran the task" would result in Web Hob reporting a single built
>>> > task
>>>with
>>> > an sstate missed flag. If we report this case through 2 separate
>>> > tasks we'll need to link them to each other, so that users can
>>> > reach one
>>>from
>>> > the other and get a full picture of what BitBake did during the
>>>build.
>>> > This is also a viable option, but it will result in a much longer
>>>table of
>>> > tasks, with 2 entries for a certain task + recipe combination.
>>> > 
>>> > Which way we report tasks impacts the way we structure the Web Hob
>>>tasks
>>> > table. From this classification of tasks:
>>> > 
>>> > * Sstate
>>> > ** Missed
>>> > ** Completed
>>> > ** Failed
>>> > 
>>> > * Built
>>> > ** Previously built
>>> > ** Completed
>>> > ** Failed
>>> > 
>>> > We would move to this one:
>>> > 
>>> > * Completed
>>> > ** Previously built
>>> > ** Sstate
>>> > ** Built
>>> > *** Sstate missed? Y/N
>>> > *** Sstae failed? Y/N
>>> > 
>>> > * Failed
>>> > ** Built
>>> 
>>> I think this is a reasonable approach. Although it does slightly
>>>obscure the  details of the tasks bitbake is running it's going to be
>>>a bit easier for the  user to follow with respect to the information
>>>they're most interested in.
>>
>>Sorry about the delayed reply, as you can tell I have a bit of a
>>backlog today :(.
>>
>>I hate to complicate things however there is a touch of added
>>complexity to consider.
>
>>Imagine we have tasks A which depends upon B which depends upon C.
>>
>>In the no sstate available case, it builds C, then B then A, easy.
>>
>>With sstate, it will try A first. If A is available, it will just
>>install A from sstate and skip B/C. If A is not available, it will try
>>B, install that and skip C. It will then build any missing pieces so it
>>might install B from sstate, then run C.
>
>I see. So it seems we have also 'skipped' tasks.
>
>>
>>This means that giving the user meaningful numbers of tasks is hard for
>>example.
>>
>>This can probably be presented in the above form fine, I just want to
>>be clear about the subtleties involved. Ideally we do want to try and
>>better convey to the user what happened too in this UI. It could be
>>good if for example we could show that we didn't run C as B was from
>>sstate and hence C wasn't needed.
>
>I agree we should show those as well. So, it looks like we have a few
>different kinds of tasks (previously built, skipped, sstate and built),
>but those can be grouped into tasks that build something (the 'built'
>kind, which we could call 'executed'), and those that don't build
>anything (previously built, skipped and sstate, which we could call 'not
>executed'). 
>
>So for the 'all tasks' table, we could have a 'task type' column
>classifying tasks as 'executed' and 'not executed'. Then, we could have
>an 'outcome' column that would classify tasks as 'succeeded', 'failed',
>'previously built', 'sstate' and 'skipped'. That information is enough to
>give me an overview of what BitBake has done, and if a task was handled
>in an unexpected way (I thought it would build but it didn't, for
>example).
>
>When I select a single task from that table, we could provide a 'task
>history' section that would expose the BitBake process for that task (in
>your example of sstate dependent tasks, for skipped task C the history
>would tell me that task C was brought in by skipped task B, which in turn
>was brought in by sstate task A. For sstate task A, the history would
>tell me that the checksum did not change, that task A depended on skipped
>task B, which path was searched to find the sstate object, and so on).
>
>Would that work?
>
>>
>>I think the key concept we might also need to work into the UI is "dry
>>run". A mode where you run the build however rather than doing it, it
>>tells you what it would do (i.e. that it would need to run X tasks, Y
>>would be done from sstate). Combined with an indication of why
>>something didn't match, this would be extremely powerful/useful to most
>>users.
>>
>>I have no objection to the form you present above, the key detail is
>>more the dry run concept from the user perspective and start answering
>>questions like "what would bitbake do?".
>
>We should keep that 'dry run' mode in mind (even I can see it would be
>incredibly useful), but it won't happen in 1.5 :(
>
>>
>>Cheers,
>>
>>Richard
>>
>>
>

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [Webhob] task types and outcomes

[Richard Purdie]

On Thu, 2013-05-30 at 10:57 +0000, Barros Pena, Belen wrote:
> On 29/05/2013 22:39, "Richard Purdie" <richard.purdie@linuxfoundation.org>
> wrote:
> 
> >On Wed, 2013-05-29 at 16:57 +0100, Paul Eggleton wrote:
> >> On Tuesday 28 May 2013 17:17:35 Barros Pena, Belen wrote:
> >> > Thanks, Paul. I guess we need to nail down what is that we are trying
> >>to
> >> > achieve by reporting sstate tasks vs. built tasks vs. previously built
> >> > tasks. From what I can remember, the main questions users need
> >>answered
> >> > are:
> >> > 
> >> > * Why is this a built task when I was expecting it to reuse sstate
> >>objects?
> >> > * Why is this an sstate task when I was expecting it to build from
> >>scratch?
> >> 
> >> These are common questions yes.
> >> 
> >> > If those are indeed the main questions we need to answer, reporting
> >>missed
> >> > sstate tasks and built tasks as one probably makes things easier. The
> >> > metaphor would be a single task with a certain life cycle, and not 2
> >> > separate tasks. So, "BitBake detected changes in the inputs for task
> >>x and
> >> > reran the task" would result in Web Hob reporting a single built task
> >>with
> >> > an sstate missed flag. If we report this case through 2 separate tasks
> >> > we'll need to link them to each other, so that users can reach one
> >>from
> >> > the other and get a full picture of what BitBake did during the build.
> >> > This is also a viable option, but it will result in a much longer
> >>table of
> >> > tasks, with 2 entries for a certain task + recipe combination.
> >> > 
> >> > Which way we report tasks impacts the way we structure the Web Hob
> >>tasks
> >> > table. From this classification of tasks:
> >> > 
> >> > * Sstate
> >> > ** Missed
> >> > ** Completed
> >> > ** Failed
> >> > 
> >> > * Built
> >> > ** Previously built
> >> > ** Completed
> >> > ** Failed
> >> > 
> >> > We would move to this one:
> >> > 
> >> > * Completed
> >> > ** Previously built
> >> > ** Sstate
> >> > ** Built
> >> > *** Sstate missed? Y/N
> >> > *** Sstae failed? Y/N
> >> > 
> >> > * Failed
> >> > ** Built
> >> 
> >> I think this is a reasonable approach. Although it does slightly
> >>obscure the 
> >> details of the tasks bitbake is running it's going to be a bit easier
> >>for the 
> >> user to follow with respect to the information they're most interested
> >>in.
> >
> >Sorry about the delayed reply, as you can tell I have a bit of a backlog
> >today :(.
> >
> >I hate to complicate things however there is a touch of added complexity
> >to consider.
> 
> >Imagine we have tasks A which depends upon B which depends
> >upon C.
> >
> >In the no sstate available case, it builds C, then B then A, easy.
> >
> >With sstate, it will try A first. If A is available, it will just
> >install A from sstate and skip B/C. If A is not available, it will try
> >B, install that and skip C. It will then build any missing pieces so it
> >might install B from sstate, then run C.
> 
> I see. So it seems we have also 'skipped' tasks.

Yes, not to be confused with "skipped" recipes during parsing which is
something very different.

> >This means that giving the user meaningful numbers of tasks is hard for
> >example.
> >
> >This can probably be presented in the above form fine, I just want to be
> >clear about the subtleties involved. Ideally we do want to try and
> >better convey to the user what happened too in this UI. It could be good
> >if for example we could show that we didn't run C as B was from sstate
> >and hence C wasn't needed.
> 
> I agree we should show those as well. So, it looks like we have a few
> different kinds of tasks (previously built, skipped, sstate and built),
> but those can be grouped into tasks that build something (the 'built'
> kind, which we could call 'executed'), and those that don't build anything
> (previously built, skipped and sstate, which we could call 'not
> executed'). 

Yes.

> So for the 'all tasks' table, we could have a 'task type' column
> classifying tasks as 'executed' and 'not executed'.

I'd prefer "Prebuilt" than "not executed" but yes.

>  Then, we could have an
> 'outcome' column that would classify tasks as 'succeeded', 'failed',
> 'previously built', 'sstate' and 'skipped'. That information is enough to
> give me an overview of what BitBake has done, and if a task was handled in
> an unexpected way (I thought it would build but it didn't, for example).
> 
> When I select a single task from that table, we could provide a 'task
> history' section that would expose the BitBake process for that task (in
> your example of sstate dependent tasks, for skipped task C the history
> would tell me that task C was brought in by skipped task B, which in turn
> was brought in by sstate task A. For sstate task A, the history would tell
> me that the checksum did not change, that task A depended on skipped task
> B, which path was searched to find the sstate object, and so on).
> 
> Would that work?

I think so.

> >I think the key concept we might also need to work into the UI is "dry
> >run". A mode where you run the build however rather than doing it, it
> >tells you what it would do (i.e. that it would need to run X tasks, Y
> >would be done from sstate). Combined with an indication of why something
> >didn't match, this would be extremely powerful/useful to most users.
> >
> >I have no objection to the form you present above, the key detail is
> >more the dry run concept from the user perspective and start answering
> >questions like "what would bitbake do?".
> 
> We should keep that 'dry run' mode in mind (even I can see it would be
> incredibly useful), but it won't happen in 1.5 :(

FWIW, commandline bitbake does support this so its not hypothetical, its
not easy to use from there though.

I mention it just to ensure we keep it in mind in the design. It might
not actually work out that hard to do if we have the right analysis
display for a build since a dry run of a build and a real build would
look very similar.

Cheers,

Richard

Re: [Webhob] task types and outcomes

[Richard Purdie]

On Thu, 2013-05-30 at 13:10 +0000, Dragomir, CalinX L wrote:
> I have a couple of questions regarding the tasks if you could help me
> understand better what's going on in a build operation.
> 
> 1. From what I've read in the e-mails so far, it seems that there are
> three types of tasks: built, sstate and skipped. Can you tell me what
> each of them mean and how are they different ?
> 2. It also seems that there can be multiple results for each of them.
> Can you detail the meaning of each final state? (failed, missed,
> etc.) ?
> 3. How can I tell what type of task is the current executed task (if
> it is sstate or built or anything else) ?

Let me explain what bitbake actually does. How this is presented in the
UI is a different question :)

When you run "bitbake X", bitbake first goes through a "setscene" phase.

This attempts to get the pre-built objects from the sstate cache. If
does this in reverse dependency order so it will try X, then the things
X depends on and so on.

If it gets X, it would skip the dependencies in many cases.

Bitbake then moves to the runqueue phase. If everything was obtained
from sstate, great, we do nothing. If we couldn't get anything, we'd
build from scratch. We also cover all the states in between.

The tmp/stamps directory tells you the status of all the tasks. Each
task can have a stamp or a setscene version of the stamp.

Hope that helps a bit!

Cheers,

Richard

Re: [Webhob] task types and outcomes

[Barros Pena, Belen]

I have managed to gather answers to most questions regarding task
information for Web Hob (thanks Paul, Alex and Richard for you help!).
I've updated bug 4275 with a new task information set

https://bugzilla.yoctoproject.org/attachment.cgi?id=1232

And a list of columns that will be available in the tasks table:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=4275#c2

Any questions or comments, let me know.

Belen

On 30/05/2013 15:28, "Richard Purdie" <richard.purdie@linuxfoundation.org>
wrote:

>On Thu, 2013-05-30 at 13:10 +0000, Dragomir, CalinX L wrote:
>> I have a couple of questions regarding the tasks if you could help me
>> understand better what's going on in a build operation.
>> 
>> 1. From what I've read in the e-mails so far, it seems that there are
>> three types of tasks: built, sstate and skipped. Can you tell me what
>> each of them mean and how are they different ?
>> 2. It also seems that there can be multiple results for each of them.
>> Can you detail the meaning of each final state? (failed, missed,
>> etc.) ?
>> 3. How can I tell what type of task is the current executed task (if
>> it is sstate or built or anything else) ?
>
>Let me explain what bitbake actually does. How this is presented in the
>UI is a different question :)
>
>When you run "bitbake X", bitbake first goes through a "setscene" phase.
>
>This attempts to get the pre-built objects from the sstate cache. If
>does this in reverse dependency order so it will try X, then the things
>X depends on and so on.
>
>If it gets X, it would skip the dependencies in many cases.
>
>Bitbake then moves to the runqueue phase. If everything was obtained
>from sstate, great, we do nothing. If we couldn't get anything, we'd
>build from scratch. We also cover all the states in between.
>
>The tmp/stamps directory tells you the status of all the tasks. Each
>task can have a stamp or a setscene version of the stamp.
>
>Hope that helps a bit!
>
>Cheers,
>
>Richard
>
>
>

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [Webhob] task types and outcomes

[Damian, Alexandru]

[-- Attachment #1: Type: text/plain, Size: 2969 bytes --]

Thank you, Belen !

Recorded here, it this is part of our specification
https://wiki.yoctoproject.org/wiki/WebHob_general_architecture

Do we have a similar definition for build information, that I am not aware
of ?

Cheers,
Alex

On Fri, May 31, 2013 at 12:49 PM, Barros Pena, Belen <
belen.barros.pena@intel.com> wrote:

> I have managed to gather answers to most questions regarding task
> information for Web Hob (thanks Paul, Alex and Richard for you help!).
> I've updated bug 4275 with a new task information set
>
> https://bugzilla.yoctoproject.org/attachment.cgi?id=1232
>
> And a list of columns that will be available in the tasks table:
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=4275#c2
>
> Any questions or comments, let me know.
>
> Belen
>
> On 30/05/2013 15:28, "Richard Purdie" <richard.purdie@linuxfoundation.org>
> wrote:
>
> >On Thu, 2013-05-30 at 13:10 +0000, Dragomir, CalinX L wrote:
> >> I have a couple of questions regarding the tasks if you could help me
> >> understand better what's going on in a build operation.
> >>
> >> 1. From what I've read in the e-mails so far, it seems that there are
> >> three types of tasks: built, sstate and skipped. Can you tell me what
> >> each of them mean and how are they different ?
> >> 2. It also seems that there can be multiple results for each of them.
> >> Can you detail the meaning of each final state? (failed, missed,
> >> etc.) ?
> >> 3. How can I tell what type of task is the current executed task (if
> >> it is sstate or built or anything else) ?
> >
> >Let me explain what bitbake actually does. How this is presented in the
> >UI is a different question :)
> >
> >When you run "bitbake X", bitbake first goes through a "setscene" phase.
> >
> >This attempts to get the pre-built objects from the sstate cache. If
> >does this in reverse dependency order so it will try X, then the things
> >X depends on and so on.
> >
> >If it gets X, it would skip the dependencies in many cases.
> >
> >Bitbake then moves to the runqueue phase. If everything was obtained
> >from sstate, great, we do nothing. If we couldn't get anything, we'd
> >build from scratch. We also cover all the states in between.
> >
> >The tmp/stamps directory tells you the status of all the tasks. Each
> >task can have a stamp or a setscene version of the stamp.
> >
> >Hope that helps a bit!
> >
> >Cheers,
> >
> >Richard
> >
> >
> >
>
> ---------------------------------------------------------------------
> Intel Corporation (UK) Limited
> Registered No. 1134945 (England)
> Registered Office: Pipers Way, Swindon SN3 1RJ
> VAT No: 860 2173 47
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>

[-- Attachment #2: Type: text/html, Size: 4151 bytes --]

Re: [Webhob] task types and outcomes

[Barros Pena, Belen]

Not yet, no. Could you explain what you exactly mean by "build information"? There are 6 priority areas in the Web Hob analysis tool: build configuration, tasks, recipes, packages, image directory structure (if build target is a base image recipe) and time. When you say "build information", to which of these area(s) are you referring to? Once that's cleared, I can set to work in defining the data needed.

If you want to get a high level idea of what type of information is needed for each area, you can visit:

http://www.yoctoproject.org/webhob/phase3_final_web_prototype/project-build.html

And go through the pages in the left navigation.

Cheers

Belén



From: <Damian>, Alexandru <alexandru.damian@intel.com<mailto:alexandru.damian@intel.com>>
Date: Friday, 31 May 2013 13:48
To: "Belen Barros Pena (Intel)" <belen.barros.pena@intel.com<mailto:belen.barros.pena@intel.com>>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org<mailto:richard.purdie@linuxfoundation.org>>, "Dragomir, CalinX L" <calinx.l.dragomir@intel.com<mailto:calinx.l.dragomir@intel.com>>, Paul Eggleton <paul.eggleton@linux.intel.com<mailto:paul.eggleton@linux.intel.com>>, "Zhang, Jessica" <jessica.zhang@intel.com<mailto:jessica.zhang@intel.com>>, "webhob@yoctoproject.org<mailto:webhob@yoctoproject.org>" <webhob@yoctoproject.org<mailto:webhob@yoctoproject.org>>
Subject: Re: task types and outcomes

Thank you, Belen !

Recorded here, it this is part of our specification https://wiki.yoctoproject.org/wiki/WebHob_general_architecture

Do we have a similar definition for build information, that I am not aware of ?

Cheers,
Alex

On Fri, May 31, 2013 at 12:49 PM, Barros Pena, Belen <belen.barros.pena@intel.com<mailto:belen.barros.pena@intel.com>> wrote:
I have managed to gather answers to most questions regarding task
information for Web Hob (thanks Paul, Alex and Richard for you help!).
I've updated bug 4275 with a new task information set

https://bugzilla.yoctoproject.org/attachment.cgi?id=1232

And a list of columns that will be available in the tasks table:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=4275#c2

Any questions or comments, let me know.

Belen

On 30/05/2013 15:28, "Richard Purdie" <richard.purdie@linuxfoundation.org<mailto:richard.purdie@linuxfoundation.org>>
wrote:

>On Thu, 2013-05-30 at 13:10 +0000, Dragomir, CalinX L wrote:
>> I have a couple of questions regarding the tasks if you could help me
>> understand better what's going on in a build operation.
>>
>> 1. From what I've read in the e-mails so far, it seems that there are
>> three types of tasks: built, sstate and skipped. Can you tell me what
>> each of them mean and how are they different ?
>> 2. It also seems that there can be multiple results for each of them.
>> Can you detail the meaning of each final state? (failed, missed,
>> etc.) ?
>> 3. How can I tell what type of task is the current executed task (if
>> it is sstate or built or anything else) ?
>
>Let me explain what bitbake actually does. How this is presented in the
>UI is a different question :)
>
>When you run "bitbake X", bitbake first goes through a "setscene" phase.
>
>This attempts to get the pre-built objects from the sstate cache. If
>does this in reverse dependency order so it will try X, then the things
>X depends on and so on.
>
>If it gets X, it would skip the dependencies in many cases.
>
>Bitbake then moves to the runqueue phase. If everything was obtained
>from sstate, great, we do nothing. If we couldn't get anything, we'd
>build from scratch. We also cover all the states in between.
>
>The tmp/stamps directory tells you the status of all the tasks. Each
>task can have a stamp or a setscene version of the stamp.
>
>Hope that helps a bit!
>
>Cheers,
>
>Richard
>
>
>

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [Webhob] task types and outcomes

[Barros Pena, Belen]

On 30/05/2013 15:21, "Richard Purdie" <richard.purdie@linuxfoundation.org>
wrote:

>
>> So for the 'all tasks' table, we could have a 'task type' column
>> classifying tasks as 'executed' and 'not executed'.
>
>I'd prefer "Prebuilt" than "not executed" but yes.

Prebuilt sounds good to me, but it causes a bit of a naming issue. We have
identified 3 possible 'Outcomes' for prebuilt tasks: sstate, skipped and
previously built (tasks for which an stamp file exists). 'Prebuilt' and
'Previously built' are too similar, which is bound to cause confusion.

Does any of you have any ideas for a better name for the 'Previously
built' outcome?

Thanks!

Belén


>
>
>Cheers,
>
>Richard
>

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [Webhob] task types and outcomes

[Otavio Salvador]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

On Fri, May 31, 2013 at 10:50 AM, Barros Pena, Belen <
belen.barros.pena@intel.com> wrote:

> On 30/05/2013 15:21, "Richard Purdie" <richard.purdie@linuxfoundation.org>
> wrote:
>
> >
> >> So for the 'all tasks' table, we could have a 'task type' column
> >> classifying tasks as 'executed' and 'not executed'.
> >
> >I'd prefer "Prebuilt" than "not executed" but yes.
>
> Prebuilt sounds good to me, but it causes a bit of a naming issue. We have
> identified 3 possible 'Outcomes' for prebuilt tasks: sstate, skipped and
> previously built (tasks for which an stamp file exists). 'Prebuilt' and
> 'Previously built' are too similar, which is bound to cause confusion.
>
> Does any of you have any ideas for a better name for the 'Previously
> built' outcome?
>

What about 'Cached'?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://projetos.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

[-- Attachment #2: Type: text/html, Size: 1730 bytes --]

Re: [Webhob] task types and outcomes

[Paul Eggleton]

On Friday 31 May 2013 10:53:43 Otavio Salvador wrote:
> On Fri, May 31, 2013 at 10:50 AM, Barros Pena, Belen <
> 
> belen.barros.pena@intel.com> wrote:
> > On 30/05/2013 15:21, "Richard Purdie" <richard.purdie@linuxfoundation.org>
> > 
> > wrote:
> > >> So for the 'all tasks' table, we could have a 'task type' column
> > >> classifying tasks as 'executed' and 'not executed'.
> > >
> > >I'd prefer "Prebuilt" than "not executed" but yes.
> > 
> > Prebuilt sounds good to me, but it causes a bit of a naming issue. We have
> > identified 3 possible 'Outcomes' for prebuilt tasks: sstate, skipped and
> > previously built (tasks for which an stamp file exists). 'Prebuilt' and
> > 'Previously built' are too similar, which is bound to cause confusion.
> > 
> > Does any of you have any ideas for a better name for the 'Previously
> > built' outcome?
> 
> What about 'Cached'?

That clashes with the sstate "cache". Not that I have any better suggestion...

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [Webhob] task types and outcomes

[Dragomir, CalinX L]

What about "Checked" ?

Thanks,
Calin

-----Original Message-----
From: Paul Eggleton [mailto:paul.eggleton@linux.intel.com] 
Sent: Friday, May 31, 2013 5:03 PM
To: webhob@yoctoproject.org
Cc: Otavio Salvador; Barros Pena, Belen; Dragomir, CalinX L; Zhang, Jessica
Subject: Re: [Webhob] task types and outcomes

On Friday 31 May 2013 10:53:43 Otavio Salvador wrote:
> On Fri, May 31, 2013 at 10:50 AM, Barros Pena, Belen <
> 
> belen.barros.pena@intel.com> wrote:
> > On 30/05/2013 15:21, "Richard Purdie" 
> > <richard.purdie@linuxfoundation.org>
> > 
> > wrote:
> > >> So for the 'all tasks' table, we could have a 'task type' column 
> > >> classifying tasks as 'executed' and 'not executed'.
> > >
> > >I'd prefer "Prebuilt" than "not executed" but yes.
> > 
> > Prebuilt sounds good to me, but it causes a bit of a naming issue. 
> > We have identified 3 possible 'Outcomes' for prebuilt tasks: sstate, 
> > skipped and previously built (tasks for which an stamp file exists). 
> > 'Prebuilt' and 'Previously built' are too similar, which is bound to cause confusion.
> > 
> > Does any of you have any ideas for a better name for the 'Previously 
> > built' outcome?
> 
> What about 'Cached'?

That clashes with the sstate "cache". Not that I have any better suggestion...

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [Webhob] task types and outcomes

[Barros Pena, Belen]

So we have 'cached' and 'checked' so far. What about 'reused', 'recycled',
'restored' or something along those lines?

Belen

On 31/05/2013 15:04, "Dragomir, CalinX L" <calinx.l.dragomir@intel.com>
wrote:

>What about "Checked" ?
>
>Thanks,
>Calin
>
>-----Original Message-----
>From: Paul Eggleton [mailto:paul.eggleton@linux.intel.com]
>Sent: Friday, May 31, 2013 5:03 PM
>To: webhob@yoctoproject.org
>Cc: Otavio Salvador; Barros Pena, Belen; Dragomir, CalinX L; Zhang,
>Jessica
>Subject: Re: [Webhob] task types and outcomes
>
>On Friday 31 May 2013 10:53:43 Otavio Salvador wrote:
>> On Fri, May 31, 2013 at 10:50 AM, Barros Pena, Belen <
>> 
>> belen.barros.pena@intel.com> wrote:
>> > On 30/05/2013 15:21, "Richard Purdie"
>> > <richard.purdie@linuxfoundation.org>
>> > 
>> > wrote:
>> > >> So for the 'all tasks' table, we could have a 'task type' column
>> > >> classifying tasks as 'executed' and 'not executed'.
>> > >
>> > >I'd prefer "Prebuilt" than "not executed" but yes.
>> > 
>> > Prebuilt sounds good to me, but it causes a bit of a naming issue.
>> > We have identified 3 possible 'Outcomes' for prebuilt tasks: sstate,
>> > skipped and previously built (tasks for which an stamp file exists).
>> > 'Prebuilt' and 'Previously built' are too similar, which is bound to
>>cause confusion.
>> > 
>> > Does any of you have any ideas for a better name for the 'Previously
>> > built' outcome?
>> 
>> What about 'Cached'?
>
>That clashes with the sstate "cache". Not that I have any better
>suggestion...
>
>Cheers,
>Paul
>
>-- 
>
>Paul Eggleton
>Intel Open Source Technology Centre

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [Webhob] task types and outcomes

[Barros Pena, Belen]

On 03/06/2013 13:32, "Barros Pena, Belen" <belen.barros.pena@intel.com>
wrote:

>So we have 'cached' and 'checked' so far. What about 'reused', 'recycled',
>'restored' or something along those lines?

Paul Eggleton has suggested 'existing', so I think I'll use that for the
time being.

If anybody has a better term, let me know.

Belen


>
>Belen
>
>On 31/05/2013 15:04, "Dragomir, CalinX L" <calinx.l.dragomir@intel.com>
>wrote:
>
>>What about "Checked" ?
>>
>>Thanks,
>>Calin
>>
>>-----Original Message-----
>>From: Paul Eggleton [mailto:paul.eggleton@linux.intel.com]
>>Sent: Friday, May 31, 2013 5:03 PM
>>To: webhob@yoctoproject.org
>>Cc: Otavio Salvador; Barros Pena, Belen; Dragomir, CalinX L; Zhang,
>>Jessica
>>Subject: Re: [Webhob] task types and outcomes
>>
>>On Friday 31 May 2013 10:53:43 Otavio Salvador wrote:
>>> On Fri, May 31, 2013 at 10:50 AM, Barros Pena, Belen <
>>> 
>>> belen.barros.pena@intel.com> wrote:
>>> > On 30/05/2013 15:21, "Richard Purdie"
>>> > <richard.purdie@linuxfoundation.org>
>>> > 
>>> > wrote:
>>> > >> So for the 'all tasks' table, we could have a 'task type' column
>>> > >> classifying tasks as 'executed' and 'not executed'.
>>> > >
>>> > >I'd prefer "Prebuilt" than "not executed" but yes.
>>> > 
>>> > Prebuilt sounds good to me, but it causes a bit of a naming issue.
>>> > We have identified 3 possible 'Outcomes' for prebuilt tasks: sstate,
>>> > skipped and previously built (tasks for which an stamp file exists).
>>> > 'Prebuilt' and 'Previously built' are too similar, which is bound to
>>>cause confusion.
>>> > 
>>> > Does any of you have any ideas for a better name for the 'Previously
>>> > built' outcome?
>>> 
>>> What about 'Cached'?
>>
>>That clashes with the sstate "cache". Not that I have any better
>>suggestion...
>>
>>Cheers,
>>Paul
>>
>>-- 
>>
>>Paul Eggleton
>>Intel Open Source Technology Centre
>

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [Webhob] task types and outcomes

[Dragomir, CalinX L]

I've updated the Django models information based on this.
The outcome is now  limited only to these five options discussed.
Please find my changes here: https://wiki.yoctoproject.org/wiki/Django_models 

Thanks,
Calin

-----Original Message-----
From: Barros Pena, Belen 
Sent: Wednesday, June 05, 2013 1:40 PM
To: Dragomir, CalinX L; Paul Eggleton; webhob@yoctoproject.org
Cc: Zhang, Jessica; Damian, Alexandru
Subject: Re: [Webhob] task types and outcomes

On 03/06/2013 13:32, "Barros Pena, Belen" <belen.barros.pena@intel.com>
wrote:

>So we have 'cached' and 'checked' so far. What about 'reused', 
>'recycled', 'restored' or something along those lines?

Paul Eggleton has suggested 'existing', so I think I'll use that for the time being.

If anybody has a better term, let me know.

Belen


>
>Belen
>
>On 31/05/2013 15:04, "Dragomir, CalinX L" <calinx.l.dragomir@intel.com>
>wrote:
>
>>What about "Checked" ?
>>
>>Thanks,
>>Calin
>>
>>-----Original Message-----
>>From: Paul Eggleton [mailto:paul.eggleton@linux.intel.com]
>>Sent: Friday, May 31, 2013 5:03 PM
>>To: webhob@yoctoproject.org
>>Cc: Otavio Salvador; Barros Pena, Belen; Dragomir, CalinX L; Zhang, 
>>Jessica
>>Subject: Re: [Webhob] task types and outcomes
>>
>>On Friday 31 May 2013 10:53:43 Otavio Salvador wrote:
>>> On Fri, May 31, 2013 at 10:50 AM, Barros Pena, Belen <
>>> 
>>> belen.barros.pena@intel.com> wrote:
>>> > On 30/05/2013 15:21, "Richard Purdie"
>>> > <richard.purdie@linuxfoundation.org>
>>> > 
>>> > wrote:
>>> > >> So for the 'all tasks' table, we could have a 'task type' 
>>> > >> column classifying tasks as 'executed' and 'not executed'.
>>> > >
>>> > >I'd prefer "Prebuilt" than "not executed" but yes.
>>> > 
>>> > Prebuilt sounds good to me, but it causes a bit of a naming issue.
>>> > We have identified 3 possible 'Outcomes' for prebuilt tasks: 
>>> > sstate, skipped and previously built (tasks for which an stamp file exists).
>>> > 'Prebuilt' and 'Previously built' are too similar, which is bound 
>>> > to
>>>cause confusion.
>>> > 
>>> > Does any of you have any ideas for a better name for the 
>>> > 'Previously built' outcome?
>>> 
>>> What about 'Cached'?
>>
>>That clashes with the sstate "cache". Not that I have any better 
>>suggestion...
>>
>>Cheers,
>>Paul
>>
>>--
>>
>>Paul Eggleton
>>Intel Open Source Technology Centre
>

[Webhob] Django models

[Paul Eggleton]

Hi Calin,

On Wednesday 05 June 2013 11:23:57 Dragomir, CalinX L wrote:
> I've updated the Django models information based on this.
> The outcome is now  limited only to these five options discussed.
> Please find my changes here:
> https://wiki.yoctoproject.org/wiki/Django_models

Some feedback on the models:

* I understand task_history is meant to capture where BitBake could have 
accelerated the task using sstate but was unable to for some reason. Can we 
have a choice field which indicates this directly (e.g. sstate_result - not 
applicable, unavailable, failed, restored)?

* I agree we want to have a sequence number to allow us to easily show the 
order tasks ran in within the UI where needed (is that task_number in the 
current model? If so do we need the order field which is also listed?)

* I wonder if task_type should instead be an boolean value indicating if the 
task executed or not; I can't think of any other value we'd want here that 
isn't already covered by task_outcome.

* What is the "code" field?

* "py_sh" should probably be named "script_type" or similar

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [Webhob] Django models

[Calin Dragomir]

[-- Attachment #1: Type: text/plain, Size: 2535 bytes --]

On 05.06.2013 20:14, Paul Eggleton wrote:
> Hi Calin,
Hi Paul,
>
> On Wednesday 05 June 2013 11:23:57 Dragomir, CalinX L wrote:
>> I've updated the Django models information based on this.
>> The outcome is now  limited only to these five options discussed.
>> Please find my changes here:
>> https://wiki.yoctoproject.org/wiki/Django_models
> Some feedback on the models:
>
> * I understand task_history is meant to capture where BitBake could have
> accelerated the task using sstate but was unable to for some reason. Can we
> have a choice field which indicates this directly (e.g. sstate_result - not
> applicable, unavailable, failed, restored)?

I will change this task_history field into a CharField with only those 4 
possible choices and will name it sstate_result.
Belen, are you ok with this? Can you please update the info in the task 
table based on this?

>
> * I agree we want to have a sequence number to allow us to easily show the
> order tasks ran in within the UI where needed (is that task_number in the
> current model? If so do we need the order field which is also listed?)

This is actually a question I had earlier. Task_number isn't listed in 
the table from bug #4275. We should drop on one of the task_number and 
order fields.
I think order should be renamed as task_number in the table, and I will 
drop order in the Django model.
Belen, do you agree ?

>
> * I wonder if task_type should instead be an boolean value indicating if the
> task executed or not; I can't think of any other value we'd want here that
> isn't already covered by task_outcome.

Since this has only two options it acts like a boolean.
I think we can make it a BooleanField. Where True means Executed, and 
False means Prebuilt.

>
> * What is the "code" field?

On the agency page here ( 
http://www.yoctoproject.org/webhob/phase3_final_web_prototype/project-build-tasks-task.html) 
under the Code section I see a block of code, and
this is why I have put a TextField there.

On the other hand, going through the tasks tabel v4 , I see that 
depending on the script type, we have a path for shell and a path , 
function and line number for python code.
Belen, what is desired here exactly?

>
> * "py_sh" should probably be named "script_type" or similar

Yes, I agree, I'll change this accordingly.
>
> Cheers,
> Paul
>

Thank you for the feedback Paul, I will update the info on the wiki 
shortly if you don't have other comments on these.

Thanks,
Calin


[-- Attachment #2: Type: text/html, Size: 4174 bytes --]

Re: [Webhob] Django models

[Barros Pena, Belen]

From: Calin Dragomir <calinx.l.dragomir@linux.intel.com<mailto:calinx.l.dragomir@linux.intel.com>>
Date: Thursday, 6 June 2013 10:27
To: Paul Eggleton <paul.eggleton@linux.intel.com<mailto:paul.eggleton@linux.intel.com>>, "webhob@yoctoproject.org<mailto:webhob@yoctoproject.org>" <webhob@yoctoproject.org<mailto:webhob@yoctoproject.org>>, "Belen Barros Pena (Intel)" <belen.barros.pena@intel.com<mailto:belen.barros.pena@intel.com>>, "Damian, Alexandru" <alexandru.damian@intel.com<mailto:alexandru.damian@intel.com>>
Subject: Re: [Webhob] Django models

On 05.06.2013 20:14, Paul Eggleton wrote:

Hi Calin,

Hi Paul,

On Wednesday 05 June 2013 11:23:57 Dragomir, CalinX L wrote:


I've updated the Django models information based on this.
The outcome is now  limited only to these five options discussed.
Please find my changes here:
https://wiki.yoctoproject.org/wiki/Django_models

Some feedback on the models:

* I understand task_history is meant to capture where BitBake could have
accelerated the task using sstate but was unable to for some reason. Can we
have a choice field which indicates this directly (e.g. sstate_result - not
applicable, unavailable, failed, restored)?

I will change this task_history field into a CharField with only those 4 possible choices and will name it sstate_result.
Belen, are you ok with this? Can you please update the info in the task table based on this?

BELEN: Mmm, the question is that the history field will not only show sstate-related outcomes. It is meant to tell "the story of the task", i.e reveal the process BitBake followed to determine how to handle the task. So, for example, for an "existing" task, it will tell me that BitBake found a stamp file in location xyz, and that's why the task was not executed. For an "executed" task, it might tell me that the checksum indicates task inputs have changed and therefore the task needs to be executed, and it will also show me the output of the bitbake-diffsigs command so that I can check what changed in those inputs. For a "skipped" task, it will show me which task brings in the skipped task, and so on.  So I don't think the 4 choices identified by Paul cover all the cases. We might need to block some time with Paul to work out what exactly the task history should show for each task type and outcome.


* I agree we want to have a sequence number to allow us to easily show the
order tasks ran in within the UI where needed (is that task_number in the
current model? If so do we need the order field which is also listed?)

This is actually a question I had earlier. Task_number isn't listed in the table from bug #4275. We should drop on one of the task_number and order fields.
I think order should be renamed as task_number in the table, and I will drop order in the Django model.
Belen, do you agree ?

BELEN: Happy with that. The label used in the UI for task_number should still be 'order' though.

* I wonder if task_type should instead be an boolean value indicating if the
task executed or not; I can't think of any other value we'd want here that
isn't already covered by task_outcome.

Since this has only two options it acts like a boolean.
I think we can make it a BooleanField. Where True means Executed, and False means Prebuilt.


* What is the "code" field?

On the agency page here ( http://www.yoctoproject.org/webhob/phase3_final_web_prototype/project-build-tasks-task.html) under the Code section I see a block of code, and
this is why I have put a TextField there.

On the other hand, going through the tasks tabel v4 , I see that depending on the script type, we have a path for shell and a path , function and line number for python code.
Belen, what is desired here exactly?

BELEN: For shell tasks, the path to the shell script. For Python tasks, the path to the shell script, the function(s) called and the start line for each of those functions. The idea of showing code was specific to failed executed tasks. For those, the idea was showing the code lines that threw the error. Can we identify those? If the answer is yes, I need to add a new information item to the list: "error code" or something like that.


* "py_sh" should probably be named "script_type" or similar

Yes, I agree, I'll change this accordingly.

Cheers,
Paul



Thank you for the feedback Paul, I will update the info on the wiki shortly if you don't have other comments on these.

Thanks,
Calin

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [Webhob] Django models

[Paul Eggleton]

On Thursday 06 June 2013 11:06:39 Barros Pena, Belen wrote:
> On Thursday, 6 June 2013 10:27, Calin Dragomir wrote:
> > On 05.06.2013 20:14, Paul Eggleton wrote:
> > > * I understand task_history is meant to capture where BitBake could have
> > > accelerated the task using sstate but was unable to for some reason. Can
> > > we have a choice field which indicates this directly (e.g. sstate_result
> > > - not applicable, unavailable, failed, restored)?
> 
> > I will change this task_history field into a CharField with only those 4
> > possible choices and will name it sstate_result. Belen, are you ok with
> > this? Can you please update the info in the task table based on this?
> 
> BELEN: Mmm, the question is that the history field will not only show
> sstate-related outcomes. It is meant to tell "the story of the task", i.e
> reveal the process BitBake followed to determine how to handle the task.
> So, for example, for an "existing" task, it will tell me that BitBake found
> a stamp file in location xyz, and that's why the task was not executed.

Right, so we can infer that based upon the value of the task_type field.

> For an "executed" task, it might tell me that the checksum indicates task
> inputs have changed and therefore the task needs to be executed, and it
> will also show me the output of the bitbake-diffsigs command so that I can
> check what changed in those inputs. For a "skipped" task, it will show me
> which task brings in the skipped task, and so on.  So I don't think the 4
> choices identified by Paul cover all the cases.

I think that it does. The value of my proposed field should be filled in for 
both sstate and executed tasks, so we can know what happened during the sstate 
phase. I don't want to track this separately in a free-format text field 
because it's effectively another specialised event log that won't be easy to 
search/filter on and I don't think that's what we need.
 
> > > * I agree we want to have a sequence number to allow us to easily show
> > > the order tasks ran in within the UI where needed (is that task_number
> > > in the current model? If so do we need the order field which is also
> > > listed?)
> >
> > This is actually a question I had earlier. Task_number isn't listed in the
> > table from bug #4275. We should drop on one of the task_number and order
> > fields. I think order should be renamed as task_number in the table, and I
> > will drop order in the Django model. Belen, do you agree ?
> 
> BELEN: Happy with that. The label used in the UI for task_number should
> still be 'order' though.

Since the model definition is not set in stone perhaps we would be better 
advised to use the same terminology in the model definition as the UI; i.e. if 
"order" is preferred in the UI then let's use that in the model as well.

> * What is the "code" field?
> 
> On the agency page here (
> http://www.yoctoproject.org/webhob/phase3_final_web_prototype/project-build
> -tasks-task.html) under the Code section I see a block of code, and this is
> why I have put a TextField there.
> 
> On the other hand, going through the tasks tabel v4 , I see that depending
> on the script type, we have a path for shell and a path , function and line
> number for python code. Belen, what is desired here exactly?
> 
> BELEN: For shell tasks, the path to the shell script. For Python tasks, the
> path to the shell script, the function(s) called and the start line for
> each of those functions. The idea of showing code was specific to failed
> executed tasks. For those, the idea was showing the code lines that threw
> the error. Can we identify those? If the answer is yes, I need to add a new
> information item to the list: "error code" or something like that.

"error code" won't really work, since that would normally imply an error 
result code (number). In any case think we need to collect this information 
regardless of whether the task failed or not, so that the user can inspect it. 
Let's keep it simple and always collect the file and starting line number for 
both python and shell tasks (i.e. we need extra fields for this); for shell 
tasks we can also collect the run.do_xyz file and place its contents in this 
"code" field.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [Webhob] Django models

[Barros Pena, Belen]

So this is the outcome of our conversation this morning about the
task_history, order and code database fields.

* Richard and Paul have a problem with the label 'skipped' as a prebuilt
task outcome, because it is already used within the project in the context
of recipes. Paul has suggested the label 'covered' instead, but has asked
me to run it past Richard

* Order: change the task_number field name to 'order' (Paul also suggested
'sequence')

* Task history

     1. Existing tasks
          1.1 The stamp file itself has no interest. So I just need to
know that a stamp file was found. That is covered by the task_type field.
          1.2 I however might want to look at the task that generated that
stamp file, so the interface needs to provide a link to such task. To
identify it we'll use the state_checksum field

2. Covered tasks
2.1 The task history will show which task is covering the covered task and
link to it. To identify the covering task, we need a new field to store
the BitBake task ID (task_id).
2.2 This will require changes to BitBake to generate an event that
provides the information about covering / covered tasks

3. Sstate and executed tasks:
3.1 For executed tasks, the task history will show the output of the
bitbake-diffsigs command, but we have not discussed yet how we are going
to store this
3.2 For executed tasks, the task history will show if a reuse from sstate
was tried but generated a miss or a fail. For that we'll add a
sstate_result field that has 4 possible values: not applicable (the task
does not qualify for reuse, for example, do_fetch), missed (packages were
not found), failed (packages were found, but could not be restored) and
restored (this is a 'prebuilt' task with outcome 'sstate').

* Code

1. For both py and shell tasks, we'll be showing the path to the relevant
file and the initial line number. This means we'll need to replace the
code field in the database with two new fields:
          1.1 One for storing the file path
1.2 One for storing the line number
Or something along these lines. Calin will suggest something

2. We have decided not to store or show the content of the run file for
the time being. However, this feature was covered by the high level
design, so in order to make sure we don't forget about it I've opened a
feature in Bugzilla marked as 'future'
(https://bugzilla.yoctoproject.org/show_bug.cgi?id=4624)

* Delete: 

     1. It looks like we are going to need delete functionality to allow
users to delete information from the web hob database.
     2. The minimum unit of deletion is a build (you can not delete part
of the information for a build), and it was also suggested to provide
date-based deletion (delete data for builds started / completed between
this day and this day).
     3. The question was raised of what's deleted: the information in the
database only, or the build output as well? We don't have an answer yet.
     4. A feature request is open, with milestone 1.5M4
(https://bugzilla.yoctoproject.org/show_bug.cgi?id=4625). Feel free to
change it if needed.

Belen



On 06/06/2013 13:54, "Paul Eggleton" <paul.eggleton@linux.intel.com> wrote:

>On Thursday 06 June 2013 11:06:39 Barros Pena, Belen wrote:
>> On Thursday, 6 June 2013 10:27, Calin Dragomir wrote:
>> > On 05.06.2013 20:14, Paul Eggleton wrote:
>> > > * I understand task_history is meant to capture where BitBake could
>>have
>> > > accelerated the task using sstate but was unable to for some
>>reason. Can
>> > > we have a choice field which indicates this directly (e.g.
>>sstate_result
>> > > - not applicable, unavailable, failed, restored)?
>> 
>> > I will change this task_history field into a CharField with only
>>those 4
>> > possible choices and will name it sstate_result. Belen, are you ok
>>with
>> > this? Can you please update the info in the task table based on this?
>> 
>> BELEN: Mmm, the question is that the history field will not only show
>> sstate-related outcomes. It is meant to tell "the story of the task",
>>i.e
>> reveal the process BitBake followed to determine how to handle the task.
>> So, for example, for an "existing" task, it will tell me that BitBake
>>found
>> a stamp file in location xyz, and that's why the task was not executed.
>
>Right, so we can infer that based upon the value of the task_type field.
>
>> For an "executed" task, it might tell me that the checksum indicates
>>task
>> inputs have changed and therefore the task needs to be executed, and it
>> will also show me the output of the bitbake-diffsigs command so that I
>>can
>> check what changed in those inputs. For a "skipped" task, it will show
>>me
>> which task brings in the skipped task, and so on.  So I don't think the
>>4
>> choices identified by Paul cover all the cases.
>
>I think that it does. The value of my proposed field should be filled in
>for 
>both sstate and executed tasks, so we can know what happened during the
>sstate 
>phase. I don't want to track this separately in a free-format text field
>because it's effectively another specialised event log that won't be easy
>to 
>search/filter on and I don't think that's what we need.
> 
>> > > * I agree we want to have a sequence number to allow us to easily
>>show
>> > > the order tasks ran in within the UI where needed (is that
>>task_number
>> > > in the current model? If so do we need the order field which is also
>> > > listed?)
>> >
>> > This is actually a question I had earlier. Task_number isn't listed
>>in the
>> > table from bug #4275. We should drop on one of the task_number and
>>order
>> > fields. I think order should be renamed as task_number in the table,
>>and I
>> > will drop order in the Django model. Belen, do you agree ?
>> 
>> BELEN: Happy with that. The label used in the UI for task_number should
>> still be 'order' though.
>
>Since the model definition is not set in stone perhaps we would be better
>advised to use the same terminology in the model definition as the UI;
>i.e. if 
>"order" is preferred in the UI then let's use that in the model as well.
>
>> * What is the "code" field?
>> 
>> On the agency page here (
>> 
>>http://www.yoctoproject.org/webhob/phase3_final_web_prototype/project-bui
>>ld
>> -tasks-task.html) under the Code section I see a block of code, and
>>this is
>> why I have put a TextField there.
>> 
>> On the other hand, going through the tasks tabel v4 , I see that
>>depending
>> on the script type, we have a path for shell and a path , function and
>>line
>> number for python code. Belen, what is desired here exactly?
>> 
>> BELEN: For shell tasks, the path to the shell script. For Python tasks,
>>the
>> path to the shell script, the function(s) called and the start line for
>> each of those functions. The idea of showing code was specific to failed
>> executed tasks. For those, the idea was showing the code lines that
>>threw
>> the error. Can we identify those? If the answer is yes, I need to add a
>>new
>> information item to the list: "error code" or something like that.
>
>"error code" won't really work, since that would normally imply an error
>result code (number). In any case think we need to collect this
>information 
>regardless of whether the task failed or not, so that the user can
>inspect it. 
>Let's keep it simple and always collect the file and starting line number
>for 
>both python and shell tasks (i.e. we need extra fields for this); for
>shell 
>tasks we can also collect the run.do_xyz file and place its contents in
>this 
>"code" field.
>
>Cheers,
>Paul
>
>-- 
>
>Paul Eggleton
>Intel Open Source Technology Centre

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

[Webhob] Django models wiki updates

[Calin Dragomir]

[-- Attachment #1: Type: text/plain, Size: 1168 bytes --]

Hi everyone,

I've updated the info on the Wiki based on what we've discussed earlier.
The updates are:

* *task_type* has been replaced with a BooleanField called 
*task_executed* which will be True for Executed tasks and False for 
prebuilt. I know this is different from the UI, but having task_type in 
the DB with the value True / False was a bit confusing for the readers. 
If anyone has objections on this, I can change it back to its original 
state.
* *task_number* has been replaced by an Integer field called *order *.
* *py_sh* field has been renamed as *script_type* .
* *code* field has been replaced by two fields called *file_path* and 
*line_number* .
* *disk_io* has been made a decimal field.
* *task_id* has been added to the set.
* *sstate_result* field has replaced *task_history* and has the 4 
options we have considered. If the current key of the choices is to be 
changed to numbers please let me know and I'll modify it.

There are a couple of questions added at the end of the table, which 
should be answered.

Please review my changes here: 
https://wiki.yoctoproject.org/wiki/Django_models

Thank you,
Calin

[-- Attachment #2: Type: text/html, Size: 1761 bytes --]

Re: [Webhob] Django models wiki updates

[Damian, Alexandru]

[-- Attachment #1: Type: text/plain, Size: 1709 bytes --]

Hi,

My take on the questions:

- primary key should be uuid and task_id. This is what the task is
naturally referenced by when looked up on interrogations. We should not
have an Auto field
- sstate_result - yes, the value stored in the table should be a small
number. ditto for all choice fields
- dependent tasks - this should be another table mapping task -> task.
- text field

Alex


On Fri, Jun 7, 2013 at 12:52 PM, Calin Dragomir <
calinx.l.dragomir@linux.intel.com> wrote:

>  Hi everyone,
>
> I've updated the info on the Wiki based on what we've discussed earlier.
> The updates are:
>
> * *task_type* has been replaced with a BooleanField called *task_executed*which will be True for Executed tasks and False for prebuilt. I know this
> is different from the UI, but having task_type in the DB with the value
> True / False was a bit confusing for the readers. If anyone has objections
> on this, I can change it back to its original state.
> * *task_number* has been replaced by an Integer field called *order *.
> * *py_sh* field has been renamed as *script_type* .
> * *code* field has been replaced by two fields called *file_path* and *
> line_number* .
> * *disk_io* has been made a decimal field.
> * *task_id* has been added to the set.
> * *sstate_result* field has replaced *task_history* and has the 4 options
> we have considered. If the current key of the choices is to be changed to
> numbers please let me know and I'll modify it.
>
> There are a couple of questions added at the end of the table, which
> should be answered.
>
> Please review my changes here:
> https://wiki.yoctoproject.org/wiki/Django_models
>
> Thank you,
> Calin
>

[-- Attachment #2: Type: text/html, Size: 2540 bytes --]