Re: Current Eclair analysis

[Nicola Vetrini <nicola.vetrini@bugseng.com>]

On 2025-12-11 11:50, Jan Beulich wrote:
> On 11.12.2025 11:08, Nicola Vetrini wrote:
>> On 2025-12-11 11:00, Jan Beulich wrote:
>>> On 10.12.2025 19:14, Andrew Cooper wrote:
>>>> The Eclair step is now the dominating aspect of wallclock time.  
>>>> While
>>>> the recent changes were a step in the right direction, we need some
>>>> adjustments.
>>> 
>>> One other question, related to the "dominating" aspect, but not to 
>>> any
>>> of the points raised so far. Can scan results possibly be recorded
>>> somehow, somewhere, such that upon re-scanning the same tree 
>>> (pre-push
>>> test followed by post-push test) the identical re-scan can be 
>>> avoided?
>>> And perhaps even incrementally - if only .c files change within a 
>>> (set
>>> of) commit(s), only re-scan those, rather than everything? Could be
>>> extended to .h files if dependencies were properly taken into 
>>> account.
>> 
>> We support incremental analysis, but it requires non-trivial changes 
>> to
>> the current Xen integration. If someone wants to invest time and/or
>> resources in it, I can support the effort, but it requires a fair deal
>> of rearrangements of scripting in order to avoid losing information
> 
> What about the initial part of the question, scanning the exact same 
> tree
> a 2nd time? Is that merely a special case of "incremental", and hence 
> would
> require the same amount of effort?
> 
> Jan

I originally misinterpreted your question, and in fact I had an internal 
discussion on this. What you are asking is not directly supported by 
eclair because ECLAIR acts based on compiler invocations triggered by 
the build system, so two different builds are entirely independent of 
each other, unless a correspondence between analyzed artifacts (analysis 
frames in ECLAIR jargon) is estabilished. Overall, in our opinion this 
is best done not internally by ECLAIR, but at a higher (or lower, 
depending on how you represent it) level, which is the build system 
level. The idea is basically as follows:

- set up ccache with a remote storage backend (one per machine where 
runners are hosted is probably the best approximation);
- let ccache be used transparently across multiple runs; it will trigger 
re-compilation only when the file is not already in its cache, and as a 
consequence trigger the analysis of that TU or program. Due to the 
dockerization of builds and how gitlab is set up this may require some 
fumbling with env vars to point to the right compilation cache, but 
should be doable without too much effort;
- For each run:
   - fetch the result from the base run (for some meaning of "base")
   - run an eclair incremental analysis using option +project. This runs 
the Xen build as usual, but using ccache (I think other projects such as 
Zephyr already do something similar in their CI);
   - load new results in the old database, substituting evidences related 
to the same TU (done with a specific eclair_report flag).

In essence this covers both usecases, but it requires a nontrivial 
amount of effort.

N.B. ECLAIR does not use ccache internally: its concept of incremental 
analysis is completely orthogonal to the one that ccache uses.


-- 
Nicola Vetrini, B.Sc.
Software Engineer
BUGSENG (https://bugseng.com)
LinkedIn: https://www.linkedin.com/in/nicola-vetrini-a42471253