Hi Randy,

On Sun, 2024-11-17 at 21:29 -0500, Randy MacLeod wrote:

On 2024-08-02 6:41 a.m., Alexandre Belloni wrote:

On 02/08/2024 12:24:06+0200, Martin Hundeb?ll wrote:

Hi Alexandre,

I see you keep carrying these patches. I'm still working with
ninja
upstream to add jobserver support there[1]. I was planning to
redo
these patches once ninja merges that. But I can do a respin with
my
patches added to ninja if you like?

[1] https://github.com/ninja-build/ninja/pull/2450

I carry them so they get some testing, I can drop until you submit
the new version.
  

Hi Martin, et. al.
 
I'm going to do some experimenting with the job server patches for a 
 talk that somehow got approved for the 2024.12 YP summit! 
   https://pretalx.com/yocto-project-summit-2024-12/talk/WGPUTX/

Thanks for picking this up. I got assigned to other projects, so this
was put on hold for now.

Hi Martin et. al.,

Errr, some things came up and I didn't feel that I had enough new to say so
I'm not presenting this week. 

Hopefully as you say, the ninja patch merges and I can get time to do more evaluation
and comparison and one of us can get a patch merged to master.

If all goes well, there will be a presentation at the next YP Summit!

Thanks,

../Randy

As expected, the patch doesn't apply with all that has changed since
April.

Do you have a a branch that you are keeping updated.
If so can you same me some time otherwise, I can rebase and fix
things up.

I have no branch, no. But I did try to get jobserver client support
into ninja. My PR got replaced by two other PRs, and I assume you are
following the progress on the latest one:
https://github.com/ninja-build/ninja/pull/2506

It looks like that PR will go into ninja eventually.

The qemu patches are no longer needed, so just the ninja patches, the
jobserver bbclass, and the python server script that are needed.

I'll of course give you all the credit for the real work.
I plan to mention the performance/behaviour improvements that you
mentioned in your cover letter:
 
---   
On build machines shared by multiple users, a single jobserver can be
shared between multiple builds (using the JOBSERVER_FIFO variable).
Running the above build in two different build directories at the
same time gives a ~12% improvement (43:17 -> 37:55).

Finally, the memory pressure from e.g. compiling multiple c++ based
projects is also reduced. In our case, a cloud based build machine
(with 32 cores and 32GB RAM) fails to compile llvm-rust-native (in
parallel to nodejs) without the jobserver due to a lack of memory.
---
 
Is there anything else you'd like people to know about?

Feel free to include my comment wherever you see fit. The credit is
fine, but not important to me. Again, do what you think is appropriate.

Does anyone else have jobserver, BB_PRESSURE_*, BB_LOADFACTOR_MAX use
cases or benchmarks that they'd like to share with the YP communtiy?
All input is welcome and appreciated.

There is a comment on the github PR that mentions yocto / bitbake:
https://github.com/ninja-build/ninja/pull/2506#issuecomment-2437267474

Good luck :)

// Martin

-- 
# Randy MacLeod
# Wind River Linux