[U-Boot] livetime of boards

[Heiko Schocher <hs@denx.de>]

Hello Andreas,

Am 07.11.2013 10:37, schrieb Andreas Bie?mann:
> Hello all together,
>
> On 11/07/2013 09:17 AM, Heiko Schocher wrote:
>> Am 06.11.2013 08:50, schrieb Wolfgang Denk:
>>> In message<20131105203736.GM5925@bill-the-cat>   you wrote:
>
> <snip problem description>
>
> Full ACK, we need some way to track which board is working with the
> current ToT or at least on a release basis.
>
>>>>> So, the question raises, should we introduce a column in boards.cfg,
>>>>> which shows the "livetime" of a board support in U-Boot?
>>>>
>>>> I sense a lot of conflicting patches.
>>>
>>> Again I agree.  Also, I fear that  boards.cfg  is becoming more and
>>> more unreadable by adding even more stuff.  If I see this correctly,
>>> the maximum line length in  boards.cfg  already exceeds 360 characters
>>> :-(
>>
>> Right, boards.cfg gets unhandy ... Hmm .. what with the column
>> "Staus" ... instead of "Active" it would be more informative to have
>> there the livetime counter, and a single digit saves some characters ;-)
>
> I can't understand the status field at all, just for the record ;)

Hmm.. good question ...

>> But you are right, that approach leads in a lot of conflicting
>> patches ... but I think, we just pooled board information in boards.cfg,
>> so this would be the right place in my eyes ...
>>
>> Maybe we get such Information "a Boards is tested with current mainline"
>> inform of an EMail with an Text "Board xy tested with commit mm. Please
>> update livetime" ... and we can add a script, which updates the
>> livetime for this board, so we can prevent conflicting patches ... ?
>
> I agree here with Tom. Beside the possibility of conflicting pahces I
> see another problem here.
> We will get a lot of patches just for increasing the tested counter for
> a single board. These patches needs to be handled in some way. If we
> shift to some integrated system (gerrit comes to mind) this could be
> easier than today, but it will bind resources anyways.
> Therefore I think it is a bad idea to save a such often changing
> information in the source code repository.

I see this info just changing once when releasing a new U-Boot version.

I not see anymore a patch for updating the livetime counter for every
board, instead we should have a script which a board maintainer can call,
after he did a board test, which sends an EMail to the U-Boot ML with
a special format, saying:
------------------------------------
subject: livetime: board name

Tested-by: ...

with commit ...
------------------------------------
On the mailserver a script scans all incoming EMails for his Subject,
(Is this possible?)
and collect the infos, for which boards, such EMails arrived. When
releasing a new U-Boot version, this collected info can be used to
update this livetime counter through another script saying
"collect_livetime_info" (also this script can automatically send
EMails to board maintainers, for boards which had reached end of
livetime, outputs a delete board lists ...)

So (in current case Tom) should, before releasing a new U-Boot
version, first call this script "collect_livetime_info" and he get:

-> one livetime counter patch for current release
-> one list for boards which reach end of life
-> one list for boards, which should be deleted

All Infos are "release info" I think, and fully fit in the commit
for the new release ...

... maybe "deleting boards" can be done automatically, but this is
not a trivial job ...

So, with such a solution, I see no big additional cost for adding
such a feature (except the task "deleting old boards", which is maybe
not trivial)

Do not understand me wrong, if we find another solution, I am
happy also ... just spinning around ...

>>> So nstead of adding this information to  boards.cfg  we could probably
>>> use separate files for such information.  We could provide tools to
>>> make test reports really easy, say something like
>>>
>>>      scripts/build_test
>>>      scripts/run_test
>>>
>>> which the user would just call with a "passed" or "failed" argument;
>>> the scripts could then auto-detect which configuration and which exact
>>> U-Boot version were in use, and send an email.  Whether that would be
>>> a patch against the source code or something that get's auto-added to
>>> a wiki page is just an implementation detail.  But if we had something
>>> like this, we could get a muchbetter understanding how actively boards
>>> are being tested.
>>
>> Yes, that sound also good. I want to see the test information in the
>> sourcecode, not somewhere on a wiki...
>
> I think another place than the source code repository would be best for
> gathering such frequently changing information. Why not use some wiki
> other other web service for that purpose?

See above explanation, I see this info not frequently changing, just
always with a new U-Boot release ... and can nearly (except the "delete
old boards" case) automatised ...

> I don't want to search a web page for the information 'board X is not
> tested since ...' either. But we could easily write some scripts and add
> them to the source code repository to provide it.

Ok, fine with me too. I just thinking about this problem, and how
we can fix it ;-)

>>> So when you're once again doing some change that requires touching
>>> files for some othe rboards, you could simply check that database.  If
>>> you see that 3 out of the last 5 releases have reported succesful
>>> run-time tests you will probably decide to accept the needed efforts,
>>
>> Hmm.. that works, if you have to touch some (some<  5) boards. But
>> If you have to touch>  5 boards, this gets unhandy...
>
> How about:
>
> MAKEALL --check-boards -s at91
>
> ;)

;-)

>>> but when you see the last test report is more than 5 years old, you
>>> will probably rather decide to initiate a code removal process.
>
> Why not save the SHA1 with the build-/runtime-tested information? Then
> we could easily build helper scripts to query that database when this
> board was last tested.

Ok, also good idea.

>> If we decide to delete older boards after n release cycles without
>> testreports, we must not decide nor look in a database. We are
>> sure, we have only "good and working" boards ... and we just
>> do the necessary work for new features ... and we are sure, that
>> we get back testreports within n release cycles ...
>>
>> So let us decide first, if we want to go this way ...
>
> Yes, we should introduce some mechanism to check when a specific board
> was last runtime tested. But I fear the overhead with patches that
> update a tested counter.

I thought with "decide": Do we want to delete "old boards"?
With this, we do not need a "MAKEALL --check-boards -s at91" when
we introduce new features, as all boards in mainline are in a well
tested shape ...

Ok, two decisions:

- Do we want to collect board testinginformation?
- Do we want to delete old boards automatically after we do not get
   some test reports after a time intervall?

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany