From: Kees Cook <keescook@chromium.org>
To: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Randy Dunlap <rdunlap@infradead.org>,
Jonathan Corbet <corbet@lwn.net>,
"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>
Subject: Re: Sphinx parallel build errors
Date: Mon, 28 Oct 2019 12:07:18 -0700 [thread overview]
Message-ID: <201910281205.DD5B74F@keescook> (raw)
In-Reply-To: <20191027160507.6d7be266@coco.lan>
On Sun, Oct 27, 2019 at 04:05:07PM -0300, Mauro Carvalho Chehab wrote:
> Em Sun, 27 Oct 2019 11:30:22 -0700
> Randy Dunlap <rdunlap@infradead.org> escreveu:
>
> > On 10/27/19 11:17 AM, Jonathan Corbet wrote:
> > > On Sun, 27 Oct 2019 11:11:57 -0700
> > > Randy Dunlap <rdunlap@infradead.org> wrote:
> > >
> > >>> Yes, I see oom-kills when building docs for 5.4-rc2.
> > >>> But only after I ran 'gimp *.jpg' for 22 photos.
> > >>>
> > >>> I can see anywhere from 1 to 4 sphinx-build processes running.
> > >>
> > >> I continue to have oom-kills when building docs.
>
> My 2 cents here.
>
> I would keep the default to use more CPUs, as nowadays I suspect that
> most devs have a way more than 8GB installed. The worse machine I use
> here to build Kernel is a Gen-3 i5 that has has 16GB (and it is a
> personal laptop I bought 5 years ago).
>
> Yet, I agree that doc build should be saner on low-cap machines.
>
> Ideally, the fix should be, IMHO, at sphinx-build: -jauto should
> detect OOM possible conditions and reduce the number of used CPUs
> it the max available memory is too low.
How can we determine that dynamically, though? e.g. how do we determine
how much is needed per thread in a way that will move with the future
needs of Sphinx, etc?
> As there's now a script at Kernel, maybe the script could be modified
> to detect if the max available memory is too low, passing -j1 if,
> let's say, the machine has less than (let's say) 16 GB RAM.
Randy, are you able to just use -j2 or similar with your builds to avoid
this?
--
Kees Cook
next prev parent reply other threads:[~2019-10-28 19:07 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-09 21:37 Sphinx parallel build errors Randy Dunlap
2019-10-09 21:40 ` Jonathan Corbet
2019-10-09 22:18 ` Randy Dunlap
2019-10-10 2:30 ` Kees Cook
2019-10-10 3:00 ` Bhaskar Chowdhury
2019-10-10 18:48 ` Jonathan Corbet
2019-10-10 20:15 ` Randy Dunlap
2019-10-27 18:11 ` Randy Dunlap
2019-10-27 18:17 ` Jonathan Corbet
2019-10-27 18:30 ` Randy Dunlap
2019-10-27 19:05 ` Mauro Carvalho Chehab
2019-10-27 19:36 ` Randy Dunlap
2019-10-28 19:07 ` Kees Cook [this message]
2019-10-29 8:16 ` Markus Heiser
2019-10-29 16:09 ` Kees Cook
2019-10-29 19:41 ` Randy Dunlap
2019-10-29 19:57 ` Markus Heiser
2019-10-29 20:10 ` Randy Dunlap
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=201910281205.DD5B74F@keescook \
--to=keescook@chromium.org \
--cc=corbet@lwn.net \
--cc=linux-doc@vger.kernel.org \
--cc=mchehab@kernel.org \
--cc=rdunlap@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).