Re: [OE-core] [PATCH v2 3/5] ninja: build modified version with GNU Make jobserver support

Alexandre Belloni
2024-08-02T10:41:42Z

On 02/08/2024 12:24:06+0200, Martin Hundeb?ll wrote:
> Hi Alexandre,
>
> I see you keep carrying these patches. I'm still working with ninja
> upstream to add jobserver support there[1]. I was planning to redo
> these patches once ninja merges that. But I can do a respin with my
> patches added to ninja if you like?
>
> [1] https://github.com/ninja-build/ninja/pull/2450

I carry them so they get some testing, I can drop until you submit the
new version.

>
> On Thu, 2024-04-25 at 13:30 +0200, Alexandre Belloni wrote:
> > Hello,
> >
> > This patch doesn't apply on master anymore because ninja has been
> > updated.
> >
> >
> > On 04/04/2024 13:16:11+0200, Martin Hundeb?ll wrote:
> > > Ninja doesn't (yet) support the GNU Make jobserver out of the box,
> > > but
> > > there is a pull request adding that support[1]. Since that pull
> > > request
> > > (and its derived three-part pull requests) seem to be ignored by
> > > upstream, kitware (creator/maintainer of cmake) has created a
> > > fork[2]
> > > only to carry the jobserver patches. Change the source uri to point
> > > at
> > > the kitware fork of ninja, and add two patches from the original
> > > pull
> > > request to also support the new-style fifo jobserver feature.
> > >
> > > Note that the kitware fork of ninja is also used by buildroot[3].
> > >
> > > [1] https://github.com/ninja-build/ninja/pull/2263
> > > [2] https://github.com/Kitware/ninja
> > > [3]
> > > https://gitlab.com/buildroot.org/buildroot/-/blob/master/package/ninja/ninja.mk
> > >
> > > Signed-off-by: Martin Hundebï¿½ll <martin@geanix.com>
> > > ---
> > > ï¿½...ename-TokenPool-Setup-to-SetupClient.patch | 113 ++++++++
> > > ï¿½...-jobserver-fifo-style-client-support.patch | 271
> > > ++++++++++++++++++
> > > ï¿½meta/recipes-devtools/ninja/ninja_1.11.1.bbï¿½ï¿½ |ï¿½ï¿½ 8 +-
> > > ï¿½3 files changed, 390 insertions(+), 2 deletions(-)
> > > ï¿½create mode 100644 meta/recipes-devtools/ninja/files/0001-Rename-
> > > TokenPool-Setup-to-SetupClient.patch
> > > ï¿½create mode 100644 meta/recipes-devtools/ninja/files/0002-Add-GNU-
> > > make-jobserver-fifo-style-client-support.patch
> > >
> > > diff --git a/meta/recipes-devtools/ninja/files/0001-Rename-
> > > TokenPool-Setup-to-SetupClient.patch b/meta/recipes-
> > > devtools/ninja/files/0001-Rename-TokenPool-Setup-to-
> > > SetupClient.patch
> > > new file mode 100644
> > > index 0000000000..a503f8c75f
> > > --- /dev/null
> > > +++ b/meta/recipes-devtools/ninja/files/0001-Rename-TokenPool-
> > > Setup-to-SetupClient.patch
> > > @@ -0,0 +1,113 @@
> > > +From f5642d8b49688dfc84679451b531d92f3b6e7cb0 Mon Sep 17 00:00:00
> > > 2001
> > > +From: Stefan Becker <chemobejk@gmail.com>
> > > +Date: Sat, 15 Dec 2018 19:29:42 +0200
> > > +Subject: [PATCH 1/2] Rename TokenPool::Setup() to SetupClient()
> > > +MIME-Version: 1.0
> > > +Content-Type: text/plain; charset=UTF-8
> > > +Content-Transfer-Encoding: 8bit
> > > +
> > > +Make space to add new API to set up token pool master.
> > > +
> > > +Signed-off-by: Martin Hundebï¿½ll <martin@geanix.com>
> > > +Upstream-Status: Submitted
> > > [https://github.com/Kitware/ninja/pull/2]
> > > +---
> > > + src/build.ccï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ | 6 +++---
> > > + src/subprocess_test.ccï¿½ï¿½ï¿½ | 3 ++-
> > > + src/tokenpool-gnu-make.cc | 6 +++---
> > > + src/tokenpool-gnu-make.hï¿½ | 3 ++-
> > > + src/tokenpool.hï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ | 3 ++-
> > > + src/tokenpool_test.ccï¿½ï¿½ï¿½ï¿½ | 2 +-
> > > + 6 files changed, 13 insertions(+), 10 deletions(-)
> > > +
> > > +diff --git a/src/build.cc b/src/build.cc
> > > +index 53e4405..d8a6dff 100644
> > > +--- a/src/build.cc
> > > ++++ b/src/build.cc
> > > +@@ -474,9 +474,9 @@ struct RealCommandRunner : public
> > > CommandRunner {
> > > + RealCommandRunner::RealCommandRunner(const BuildConfig& config) :
> > > config_(config) {
> > > +ï¿½ï¿½ max_load_average_ = config.max_load_average;
> > > +ï¿½ï¿½ if ((tokens_ = TokenPool::Get()) != NULL) {
> > > +-ï¿½ï¿½ï¿½ if (!tokens_->Setup(config_.parallelism_from_cmdline,
> > > +-ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ config_.verbosity ==
> > > BuildConfig::VERBOSE,
> > > +-ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ max_load_average_)) {
> > > ++ï¿½ï¿½ï¿½ if (!tokens_->SetupClient(config_.parallelism_from_cmdline,
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ config_.verbosity ==
> > > BuildConfig::VERBOSE,
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ max_load_average_)) {
> > > +ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ delete tokens_;
> > > +ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ tokens_ = NULL;
> > > +ï¿½ï¿½ï¿½ï¿½ }
> > > +diff --git a/src/subprocess_test.cc b/src/subprocess_test.cc
> > > +index eddc110..222b59b 100644
> > > +--- a/src/subprocess_test.cc
> > > ++++ b/src/subprocess_test.cc
> > > +@@ -40,7 +40,8 @@ struct TestTokenPool : public TokenPool {
> > > +ï¿½ï¿½ void Reserve()ï¿½ï¿½ï¿½ï¿½ {}
> > > +ï¿½ï¿½ void Release()ï¿½ï¿½ï¿½ï¿½ {}
> > > +ï¿½ï¿½ void Clear()ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ {}
> > > +-ï¿½ bool Setup(bool ignore_unused, bool verbose, double&
> > > max_load_average) { return false; }
> > > ++ï¿½ bool SetupClient(bool ignore_unused, bool verbose,
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ double& max_load_average) { return false; }
> > > +
> > > + #ifdef _WIN32
> > > +ï¿½ï¿½ bool _token_available;
> > > +diff --git a/src/tokenpool-gnu-make.cc b/src/tokenpool-gnu-make.cc
> > > +index 60e0552..6fb72a6 100644
> > > +--- a/src/tokenpool-gnu-make.cc
> > > ++++ b/src/tokenpool-gnu-make.cc
> > > +@@ -28,9 +28,9 @@ GNUmakeTokenPool::GNUmakeTokenPool() :
> > > available_(1), used_(0) {
> > > + GNUmakeTokenPool::~GNUmakeTokenPool() {
> > > + }
> > > +
> > > +-bool GNUmakeTokenPool::Setup(bool ignore,
> > > +-ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ bool verbose,
> > > +-ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ double& max_load_average) {
> > > ++bool GNUmakeTokenPool::SetupClient(bool ignore,
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ bool verbose,
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ double& max_load_average) {
> > > +ï¿½ï¿½ const char* value = GetEnv("MAKEFLAGS");
> > > +ï¿½ï¿½ if (!value)
> > > +ï¿½ï¿½ï¿½ï¿½ return false;
> > > +diff --git a/src/tokenpool-gnu-make.h b/src/tokenpool-gnu-make.h
> > > +index c94cca5..f4ab8d7 100644
> > > +--- a/src/tokenpool-gnu-make.h
> > > ++++ b/src/tokenpool-gnu-make.h
> > > +@@ -24,7 +24,8 @@ struct GNUmakeTokenPool : public TokenPool {
> > > +ï¿½ï¿½ virtual void Reserve();
> > > +ï¿½ï¿½ virtual void Release();
> > > +ï¿½ï¿½ virtual void Clear();
> > > +-ï¿½ virtual bool Setup(bool ignore, bool verbose, double&
> > > max_load_average);
> > > ++ï¿½ virtual bool SetupClient(bool ignore, bool verbose,
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ double& max_load_average);
> > > +
> > > +ï¿½ï¿½ // platform specific implementation
> > > +ï¿½ï¿½ virtual const char* GetEnv(const char* name) = 0;
> > > +diff --git a/src/tokenpool.h b/src/tokenpool.h
> > > +index 931c227..ce2bf48 100644
> > > +--- a/src/tokenpool.h
> > > ++++ b/src/tokenpool.h
> > > +@@ -26,7 +26,8 @@ struct TokenPool {
> > > +ï¿½ï¿½ virtual void Clear() = 0;
> > > +
> > > +ï¿½ï¿½ // returns false if token pool setup failed
> > > +-ï¿½ virtual bool Setup(bool ignore, bool verbose, double&
> > > max_load_average) = 0;
> > > ++ï¿½ virtual bool SetupClient(bool ignore, bool verbose,
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ double& max_load_average) = 0;
> > > +
> > > + #ifdef _WIN32
> > > +ï¿½ï¿½ virtual void WaitForTokenAvailability(HANDLE ioport) = 0;
> > > +diff --git a/src/tokenpool_test.cc b/src/tokenpool_test.cc
> > > +index 1bc1c84..9d07146 100644
> > > +--- a/src/tokenpool_test.cc
> > > ++++ b/src/tokenpool_test.cc
> > > +@@ -77,7 +77,7 @@ struct TokenPoolTest : public testing::Test {
> > > +ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ ENVIRONMENT_INIT(buf_);
> > > +ï¿½ï¿½ï¿½ï¿½ }
> > > +ï¿½ï¿½ï¿½ï¿½ if ((tokens_ = TokenPool::Get()) != NULL) {
> > > +-ï¿½ï¿½ï¿½ï¿½ï¿½ if (!tokens_->Setup(ignore_jobserver, false, load_avg_)) {
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ if (!tokens_->SetupClient(ignore_jobserver, false,
> > > load_avg_)) {
> > > +ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ delete tokens_;
> > > +ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ tokens_ = NULL;
> > > +ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ }
> > > +--
> > > +2.44.0
> > > +
> > > diff --git a/meta/recipes-devtools/ninja/files/0002-Add-GNU-make-
> > > jobserver-fifo-style-client-support.patch b/meta/recipes-
> > > devtools/ninja/files/0002-Add-GNU-make-jobserver-fifo-style-client-
> > > support.patch
> > > new file mode 100644
> > > index 0000000000..770a53fbc9
> > > --- /dev/null
> > > +++ b/meta/recipes-devtools/ninja/files/0002-Add-GNU-make-
> > > jobserver-fifo-style-client-support.patch
> > > @@ -0,0 +1,271 @@
> > > +From 830b7fb396383171c5fac2aae7de880ac83ce955 Mon Sep 17 00:00:00
> > > 2001
> > > +From: Stefan Becker <chemobejk@gmail.com>
> > > +Date: Mon, 7 Nov 2022 20:45:27 +0200
> > > +Subject: [PATCH 2/2] Add GNU make jobserver fifo style client
> > > support
> > > +MIME-Version: 1.0
> > > +Content-Type: text/plain; charset=UTF-8
> > > +Content-Transfer-Encoding: 8bit
> > > +
> > > +GNU make 4.4 introduced a new jobserver style "fifo" for POSIX
> > > systems
> > > +which passes a named pipe down to the clients.
> > > +
> > > +- update auth parser to recognize "fifo:<name>" format
> > > +- open named pipe for reading and writing
> > > +- make sure the file descriptors are closed in the destructor
> > > +- add 2 tests that aren't compiled for WIN32
> > > +
> > > +Signed-off-by: Martin Hundebï¿½ll <martin@geanix.com>
> > > +Upstream-Status: Submitted
> > > [https://github.com/Kitware/ninja/pull/2]
> > > +---
> > > + src/tokenpool-gnu-make-posix.cc | 46 ++++++++++++++++-
> > > + src/tokenpool_test.ccï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ | 88
> > > ++++++++++++++++++++++++++++++---
> > > + 2 files changed, 127 insertions(+), 7 deletions(-)
> > > +
> > > +diff --git a/src/tokenpool-gnu-make-posix.cc b/src/tokenpool-gnu-
> > > make-posix.cc
> > > +index 8447070..a82cce4 100644
> > > +--- a/src/tokenpool-gnu-make-posix.cc
> > > ++++ b/src/tokenpool-gnu-make-posix.cc
> > > +@@ -40,6 +40,7 @@ struct GNUmakeTokenPoolPosix : public
> > > GNUmakeTokenPool {
> > > +ï¿½ private:
> > > +ï¿½ï¿½ int rfd_;
> > > +ï¿½ï¿½ int wfd_;
> > > ++ï¿½ bool closeFds_;
> > > +
> > > +ï¿½ï¿½ struct sigaction old_act_;
> > > +ï¿½ï¿½ bool restore_;
> > > +@@ -48,14 +49,19 @@ struct GNUmakeTokenPoolPosix : public
> > > GNUmakeTokenPool {
> > > +ï¿½ï¿½ static void CloseDupRfd(int signum);
> > > +
> > > +ï¿½ï¿½ bool CheckFd(int fd);
> > > ++ï¿½ bool CheckFifo(const char* fifo);
> > > +ï¿½ï¿½ bool SetAlarmHandler();
> > > + };
> > > +
> > > +-GNUmakeTokenPoolPosix::GNUmakeTokenPoolPosix() : rfd_(-1), wfd_(-
> > > 1), restore_(false) {
> > > ++GNUmakeTokenPoolPosix::GNUmakeTokenPoolPosix() : rfd_(-1), wfd_(-
> > > 1), closeFds_(false), restore_(false) {
> > > + }
> > > +
> > > + GNUmakeTokenPoolPosix::~GNUmakeTokenPoolPosix() {
> > > +ï¿½ï¿½ Clear();
> > > ++ï¿½ if (closeFds_) {
> > > ++ï¿½ï¿½ï¿½ close(wfd_);
> > > ++ï¿½ï¿½ï¿½ close(rfd_);
> > > ++ï¿½ }
> > > +ï¿½ï¿½ if (restore_)
> > > +ï¿½ï¿½ï¿½ï¿½ sigaction(SIGALRM, &old_act_, NULL);
> > > + }
> > > +@@ -69,6 +75,36 @@ bool GNUmakeTokenPoolPosix::CheckFd(int fd) {
> > > +ï¿½ï¿½ return true;
> > > + }
> > > +
> > > ++bool GNUmakeTokenPoolPosix::CheckFifo(const char* fifo) {
> > > ++ï¿½ // remove possible junk from end of fifo name
> > > ++ï¿½ char *filename = strdup(fifo);
> > > ++ï¿½ char *end;
> > > ++ï¿½ if ((end = strchr(filename, ' ')) != NULL) {
> > > ++ï¿½ï¿½ï¿½ *end = '\0';
> > > ++ï¿½ }
> > > ++
> > > ++ï¿½ int rfd = open(filename, O_RDONLY);
> > > ++ï¿½ if (rfd < 0) {
> > > ++ï¿½ï¿½ï¿½ free(filename);
> > > ++ï¿½ï¿½ï¿½ return false;
> > > ++ï¿½ }
> > > ++
> > > ++ï¿½ int wfd = open(filename, O_WRONLY);
> > > ++ï¿½ if (wfd < 0) {
> > > ++ï¿½ï¿½ï¿½ close(rfd);
> > > ++ï¿½ï¿½ï¿½ free(filename);
> > > ++ï¿½ï¿½ï¿½ return false;
> > > ++ï¿½ }
> > > ++
> > > ++ï¿½ free(filename);
> > > ++
> > > ++ï¿½ rfd_ = rfd;
> > > ++ï¿½ wfd_ = wfd;
> > > ++ï¿½ closeFds_ = true;
> > > ++
> > > ++ï¿½ return true;
> > > ++}
> > > ++
> > > + int GNUmakeTokenPoolPosix::dup_rfd_ = -1;
> > > +
> > > + void GNUmakeTokenPoolPosix::CloseDupRfd(int signum) {
> > > +@@ -89,6 +125,13 @@ bool GNUmakeTokenPoolPosix::SetAlarmHandler()
> > > {
> > > + }
> > > +
> > > + bool GNUmakeTokenPoolPosix::ParseAuth(const char* jobserver) {
> > > ++ï¿½ // check for jobserver-fifo style
> > > ++ï¿½ const char* fifo;
> > > ++ï¿½ if (((fifo = strstr(jobserver, "=fifo:")) != NULL) &&
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ CheckFifo(fifo + 6))
> > > ++ï¿½ï¿½ï¿½ return SetAlarmHandler();
> > > ++
> > > ++ï¿½ // check for legacy simple pipe style
> > > +ï¿½ï¿½ int rfd = -1;
> > > +ï¿½ï¿½ int wfd = -1;
> > > +ï¿½ï¿½ if ((sscanf(jobserver, "%*[^=]=%d,%d", &rfd, &wfd) == 2) &&
> > > +@@ -100,6 +143,7 @@ bool GNUmakeTokenPoolPosix::ParseAuth(const
> > > char* jobserver) {
> > > +ï¿½ï¿½ï¿½ï¿½ return true;
> > > +ï¿½ï¿½ }
> > > +
> > > ++ï¿½ // some jobserver style we don't support
> > > +ï¿½ï¿½ return false;
> > > + }
> > > +
> > > +diff --git a/src/tokenpool_test.cc b/src/tokenpool_test.cc
> > > +index 9d07146..e15da6c 100644
> > > +--- a/src/tokenpool_test.cc
> > > ++++ b/src/tokenpool_test.cc
> > > +@@ -19,7 +19,10 @@
> > > + #ifdef _WIN32
> > > + #include <windows.h>
> > > + #else
> > > ++#include <fcntl.h>
> > > + #include <unistd.h>
> > > ++#include <sys/types.h>
> > > ++#include <sys/stat.h>
> > > + #endif
> > > +
> > > + #include <stdio.h>
> > > +@@ -35,6 +38,8 @@
> > > + #define AUTH_FORMAT(tmpl)ï¿½ï¿½ "foo " tmpl "=%d,%d bar"
> > > + #define ENVIRONMENT_CLEAR() unsetenv("MAKEFLAGS")
> > > + #define ENVIRONMENT_INIT(v) setenv("MAKEFLAGS", v, true)
> > > ++
> > > ++#define FIFO_NAME "ninja-test-tokenpool-fifo"
> > > + #endif
> > > +
> > > + namespace {
> > > +@@ -60,11 +65,24 @@ struct TokenPoolTest : public testing::Test {
> > > +ï¿½ï¿½ï¿½ï¿½ semaphore_name_ = SEMAPHORE_NAME;
> > > +ï¿½ï¿½ï¿½ï¿½ if ((semaphore_ = CreateSemaphore(0, 0, 2, SEMAPHORE_NAME))
> > > == NULL)
> > > + #else
> > > ++ï¿½ï¿½ï¿½ if (mkfifo(FIFO_NAME, 0600) < 0) {
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ ASSERT_TRUE(false);
> > > ++ï¿½ï¿½ï¿½ }
> > > ++
> > > +ï¿½ï¿½ï¿½ï¿½ if (pipe(fds_) < 0)
> > > + #endif
> > > +ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ ASSERT_TRUE(false);
> > > +ï¿½ï¿½ }
> > > +
> > > ++ï¿½ void GetPool(bool ignore_jobserver) {
> > > ++ï¿½ï¿½ï¿½ if ((tokens_ = TokenPool::Get()) != NULL) {
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ if (!tokens_->SetupClient(ignore_jobserver, false,
> > > load_avg_)) {
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ delete tokens_;
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ tokens_ = NULL;
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ }
> > > ++ï¿½ï¿½ï¿½ }
> > > ++ï¿½ }
> > > ++
> > > +ï¿½ï¿½ void CreatePool(const char* format, bool ignore_jobserver =
> > > false) {
> > > +ï¿½ï¿½ï¿½ï¿½ if (format) {
> > > +ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ sprintf(buf_, format,
> > > +@@ -76,18 +94,30 @@ struct TokenPoolTest : public testing::Test {
> > > +ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ );
> > > +ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ ENVIRONMENT_INIT(buf_);
> > > +ï¿½ï¿½ï¿½ï¿½ }
> > > +-ï¿½ï¿½ï¿½ if ((tokens_ = TokenPool::Get()) != NULL) {
> > > +-ï¿½ï¿½ï¿½ï¿½ï¿½ if (!tokens_->SetupClient(ignore_jobserver, false,
> > > load_avg_)) {
> > > +-ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ delete tokens_;
> > > +-ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ tokens_ = NULL;
> > > +-ï¿½ï¿½ï¿½ï¿½ï¿½ }
> > > +-ï¿½ï¿½ï¿½ }
> > > ++ï¿½ï¿½ï¿½ GetPool(ignore_jobserver);
> > > +ï¿½ï¿½ }
> > > +
> > > +ï¿½ï¿½ void CreateDefaultPool() {
> > > +ï¿½ï¿½ï¿½ï¿½ CreatePool(AUTH_FORMAT("--jobserver-auth"));
> > > +ï¿½ï¿½ }
> > > +
> > > ++#ifndef _WIN32
> > > ++ï¿½ void CreateFifoPool() {
> > > ++ï¿½ï¿½ï¿½ // close simple pipe fds...
> > > ++ï¿½ï¿½ï¿½ close(fds_[0]);
> > > ++ï¿½ï¿½ï¿½ close(fds_[1]);
> > > ++
> > > ++ï¿½ï¿½ï¿½ // ... and open named pipe instead
> > > ++ï¿½ï¿½ï¿½ if ((fds_[0] = open(FIFO_NAME, O_RDONLY|O_NONBLOCK)) < 0)
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ ASSERT_TRUE(false);
> > > ++ï¿½ï¿½ï¿½ if ((fds_[1] = open(FIFO_NAME, O_WRONLY)) < 0)
> > > ++ï¿½ï¿½ï¿½ï¿½ï¿½ ASSERT_TRUE(false);
> > > ++
> > > ++ï¿½ï¿½ï¿½ ENVIRONMENT_INIT("foo --jobserver-auth=fifo:" FIFO_NAME "
> > > bar");
> > > ++ï¿½ï¿½ï¿½ GetPool(false);
> > > ++ï¿½ }
> > > ++#endif
> > > ++
> > > +ï¿½ï¿½ virtual void TearDown() {
> > > +ï¿½ï¿½ï¿½ï¿½ if (tokens_)
> > > +ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ delete tokens_;
> > > +@@ -96,6 +126,7 @@ struct TokenPoolTest : public testing::Test {
> > > + #else
> > > +ï¿½ï¿½ï¿½ï¿½ close(fds_[0]);
> > > +ï¿½ï¿½ï¿½ï¿½ close(fds_[1]);
> > > ++ï¿½ï¿½ï¿½ unlink(FIFO_NAME);
> > > + #endif
> > > +ï¿½ï¿½ï¿½ï¿½ ENVIRONMENT_CLEAR();
> > > +ï¿½ï¿½ }
> > > +@@ -167,6 +198,15 @@ TEST_F(TokenPoolTest, MonitorFD) {
> > > +
> > > +ï¿½ï¿½ EXPECT_EQ(fds_[0], tokens_->GetMonitorFd());
> > > + }
> > > ++
> > > ++TEST_F(TokenPoolTest, MonitorFDFifo) {
> > > ++ï¿½ CreateFifoPool();
> > > ++
> > > ++ï¿½ ASSERT_NE(NULL, tokens_);
> > > ++ï¿½ EXPECT_EQ(kLoadAverageDefault, load_avg_);
> > > ++
> > > ++ï¿½ EXPECT_NE(-1, tokens_->GetMonitorFd());
> > > ++}
> > > + #endif
> > > +
> > > + TEST_F(TokenPoolTest, ImplicitToken) {
> > > +@@ -226,6 +266,42 @@ TEST_F(TokenPoolTest, TwoTokens) {
> > > +ï¿½ï¿½ EXPECT_TRUE(tokens_->Acquire());
> > > + }
> > > +
> > > ++#ifndef _WIN32
> > > ++TEST_F(TokenPoolTest, TwoTokensFifo) {
> > > ++ï¿½ CreateFifoPool();
> > > ++
> > > ++ï¿½ ASSERT_NE(NULL, tokens_);
> > > ++ï¿½ EXPECT_EQ(kLoadAverageDefault, load_avg_);
> > > ++
> > > ++ï¿½ // implicit token
> > > ++ï¿½ EXPECT_TRUE(tokens_->Acquire());
> > > ++ï¿½ tokens_->Reserve();
> > > ++ï¿½ EXPECT_FALSE(tokens_->Acquire());
> > > ++
> > > ++ï¿½ // jobserver offers 2nd token
> > > ++ï¿½ char test_tokens[1] = { '+' };
> > > ++ï¿½ ASSERT_EQ(1u, write(fds_[1], test_tokens,
> > > sizeof(test_tokens)));
> > > ++ï¿½ EXPECT_TRUE(tokens_->Acquire());
> > > ++ï¿½ tokens_->Reserve();
> > > ++ï¿½ EXPECT_FALSE(tokens_->Acquire());
> > > ++
> > > ++ï¿½ // release 2nd token
> > > ++ï¿½ tokens_->Release();
> > > ++ï¿½ EXPECT_TRUE(tokens_->Acquire());
> > > ++
> > > ++ï¿½ // release implicit token - must return 2nd token back to
> > > jobserver
> > > ++ï¿½ tokens_->Release();
> > > ++ï¿½ EXPECT_TRUE(tokens_->Acquire());
> > > ++
> > > ++ï¿½ // there must be one token available
> > > ++ï¿½ EXPECT_EQ(1u, read(fds_[0], buf_, sizeof(buf_)));
> > > ++ï¿½ EXPECT_EQ(test_tokens[0], buf_[0]);
> > > ++
> > > ++ï¿½ // implicit token
> > > ++ï¿½ EXPECT_TRUE(tokens_->Acquire());
> > > ++}
> > > ++#endif
> > > ++
> > > + TEST_F(TokenPoolTest, Clear) {
> > > +ï¿½ï¿½ CreateDefaultPool();
> > > +
> > > +--
> > > +2.44.0
> > > +
> > > diff --git a/meta/recipes-devtools/ninja/ninja_1.11.1.bb
> > > b/meta/recipes-devtools/ninja/ninja_1.11.1.bb
> > > index 8e297ec4d4..dbff2e191a 100644
> > > --- a/meta/recipes-devtools/ninja/ninja_1.11.1.bb
> > > +++ b/meta/recipes-devtools/ninja/ninja_1.11.1.bb
> > > @@ -6,9 +6,13 @@ LIC_FILES_CHKSUM =
> > > "file://COPYING;md5=a81586a64ad4e476c791cda7e2f2c52e"
> > > ï¿½
> > > ï¿½DEPENDS = "re2c-native ninja-native"
> > > ï¿½
> > > -SRCREV = "a524bf3f6bacd1b4ad85d719eed2737d8562f27a"
> > > +SRCREV = "95dee2a91d96c409d54f9fa0b70ea9aa2bdf8e63"
> > > ï¿½
> > > -SRC_URI = "git://github.com/ninja-
> > > build/ninja.git;branch=release;protocol=https"
> > > +SRC_URI = " \
> > > +ï¿½ï¿½ï¿½ git://github.com/kitware/ninja.git;branch=kitware-staged-
> > > features;protocol=https \
> > > +ï¿½ï¿½ï¿½ file://0001-Rename-TokenPool-Setup-to-SetupClient.patchï¿½\
> > > +ï¿½ï¿½ï¿½
> > > file://0002-Add-GNU-make-jobserver-fifo-style-client-support.patchï¿½
> > > \
> > > +"
> > > ï¿½UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>.*)"
> > > ï¿½
> > > ï¿½S = "${WORKDIR}/git"
> > > --
> > > 2.44.0
> > >
> >
> > >
> > >
> > >
> >
> >
>

>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#202897): https://lists.openembedded.org/g/openembedded-core/message/202897
> Mute This Topic: https://lists.openembedded.org/mt/105326967/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering