From: Kees Cook <keescook@chromium.org>
To: Jonathan Corbet <corbet@lwn.net>
Cc: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>,
linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH] docs: Use make invocation's -j argument for parallelism
Date: Thu, 19 Sep 2019 11:18:52 -0700 [thread overview]
Message-ID: <201909191116.192A096C68@keescook> (raw)
While sphinx 1.7 and later supports "-jauto" for parallelism, this
effectively ignores the "-j" flag used in the "make" invocation, which
may cause confusion for build systems. Instead, extract the available
parallelism from "make"'s job server (since it is not exposed in any
special variables) and use that for the "sphinx-build" run. Now things
work correctly for builds where -j is specified at the top-level:
make -j16 htmldocs
Signed-off-by: Kees Cook <keescook@chromium.org>
---
Documentation/Makefile | 3 +--
scripts/jobserver-count | 46 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 47 insertions(+), 2 deletions(-)
create mode 100755 scripts/jobserver-count
diff --git a/Documentation/Makefile b/Documentation/Makefile
index e145e4db508b..4408eeaf2891 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -33,8 +33,6 @@ ifeq ($(HAVE_SPHINX),0)
else # HAVE_SPHINX
-export SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version 2>&1 |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')
-
# User-friendly check for pdflatex and latexmk
HAVE_PDFLATEX := $(shell if which $(PDFLATEX) >/dev/null 2>&1; then echo 1; else echo 0; fi)
HAVE_LATEXMK := $(shell if which latexmk >/dev/null 2>&1; then echo 1; else echo 0; fi)
@@ -68,6 +66,7 @@ quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
PYTHONDONTWRITEBYTECODE=1 \
BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(srctree)/$(src)/$5/$(SPHINX_CONF)) \
$(SPHINXBUILD) \
+ -j $(shell $(srctree)/scripts/jobserver-count) \
-b $2 \
-c $(abspath $(srctree)/$(src)) \
-d $(abspath $(BUILDDIR)/.doctrees/$3) \
diff --git a/scripts/jobserver-count b/scripts/jobserver-count
new file mode 100755
index 000000000000..5fc9d2fd5254
--- /dev/null
+++ b/scripts/jobserver-count
@@ -0,0 +1,46 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# This determines how many parallel tasks "make" is expecting, as it is
+# not exposed via an special variables.
+# https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver
+import os, sys, fcntl
+
+# Set non-blocking for a given file descriptor.
+def nonblock(fd):
+ flags = fcntl.fcntl(fd, fcntl.F_GETFL)
+ fcntl.fcntl(fd, fcntl.F_SETFL, flags | os.O_NONBLOCK)
+ return fd
+
+# Fetch the make environment options.
+flags = os.environ.get('MAKEFLAGS', None)
+if flags == None:
+ print("1")
+ sys.exit(0)
+
+# Look for "--jobserver=R,W"
+opts = [x for x in flags.split(" ") if x.startswith("--jobserver")]
+if len(opts) != 1:
+ print("1")
+ sys.exit(0)
+
+# Parse out R,W file descriptor numbers and set them nonblocking.
+fds = opts[0].split("=", 1)[1]
+reader, writer = [nonblock(int(x)) for x in fds.split(",", 1)]
+
+# Read out as many jobserver slots as possible.
+jobs = b""
+while True:
+ try:
+ slot = os.read(reader, 1)
+ jobs += slot
+ except:
+ break
+# Return all the reserved slots.
+os.write(writer, jobs)
+
+# Report available slots (with a bump for our caller's reserveration).
+if len(jobs) < 1:
+ print("1")
+else:
+ print(len(jobs) + 1)
--
2.17.1
--
Kees Cook
next reply other threads:[~2019-09-19 18:19 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-19 18:18 Kees Cook [this message]
2019-09-19 19:25 ` [PATCH] docs: Use make invocation's -j argument for parallelism Mauro Carvalho Chehab
2019-09-19 21:19 ` Kees Cook
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=201909191116.192A096C68@keescook \
--to=keescook@chromium.org \
--cc=corbet@lwn.net \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mchehab+samsung@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox