From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CD9C9ECAAD5 for ; Tue, 6 Sep 2022 15:02:13 +0000 (UTC) Received: from localhost ([::1]:51734 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oVa5c-0000eg-Tp for qemu-devel@archiver.kernel.org; Tue, 06 Sep 2022 11:02:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41770) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oVYpU-000540-0G for qemu-devel@nongnu.org; Tue, 06 Sep 2022 09:41:31 -0400 Received: from mail-yw1-x1132.google.com ([2607:f8b0:4864:20::1132]:39573) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oVYpS-0002Qt-8H for qemu-devel@nongnu.org; Tue, 06 Sep 2022 09:41:27 -0400 Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-3376851fe13so96757037b3.6 for ; Tue, 06 Sep 2022 06:41:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date; bh=vnAejslANxdZfpNpsgLqIMJJkSnk+410P40vhD4I7DA=; b=Z+WA0wuXxTk6mrwWe7Gvg1mJ5JdQPShaQ7k9fPx91wba9ZgFCprm+KBK5dy/5D8fQS LWe054QxnE1a6df1QQc9f7/ANnJsB2+KxhK5RWyYPpXldtPF+xstfXy/MxG415sEk1j8 Dz37PrnkOiwrldf4OKZhq8mYkvE4FB7qG7mGvpD/jTMZrzgSvEOu7lawiQ/ANYg7QHCV ju3gdIpXEXwqaRnZpvvBTqkG9+ymoaWAl/LrfLaKoPBbUsyqESeF6o8o234LmDK5jdWB XoIFm74PrJ2+1c+PrJSIkCj2wDQCVGJOVh0SsvE490lSktG145Mo/gwBd6sREdvSLeLM 1DgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date; bh=vnAejslANxdZfpNpsgLqIMJJkSnk+410P40vhD4I7DA=; b=KD+ZEl8IdX6znZJnUooCaGmloXxF/HNK1lhBrs4ZaO8vL20ttV1OjCJfxOscrpPlvl 9oHHxzjxibKTCc+fcR2nNI9puw5zjAK3jxXarJxhdjXfJdyZR/eJ93jjTxKYSFnPR3y3 ezPsny3FO3CgqjVdWqc60UnvtEtd+wgv84Zi+TGwJxpC3QJ2O/UBJLYLhlO2uVkvpFT+ N6QXP2EFXo1adbQ0j6joKfLMYOsOT4Cb/J9bNLiE9Bw4bCDfBztw3gnuj89IfcQoja4J In4fdQ91o1mN9LSVUK8fhDKNNzPDFDevRM3zQKBVZNlK7tuF1T84lTHkI9OZ0L3nun+o o+dA== X-Gm-Message-State: ACgBeo3KReRE1UNKB0CPWAHc4rKd/nQMrS9mMtAmJs4BkuyAyNrPryGW J2+X4WV180/6zy+c2saeX2eq+ovCUJHlUl618Y20mA== X-Google-Smtp-Source: AA6agR4WvwEy7VjfB5PVil0jsnkOTC2CZS/nZYSEv41o9+DGdZtWZIdu6IZ0MIcVBjQBiPy9OpEz/rsA/A4ywzYhCrA= X-Received: by 2002:a81:1615:0:b0:340:d343:7c11 with SMTP id 21-20020a811615000000b00340d3437c11mr39122250yww.257.1662471685047; Tue, 06 Sep 2022 06:41:25 -0700 (PDT) MIME-Version: 1.0 References: <53b94520-f02b-aef5-b372-0460531b566f@suse.de> In-Reply-To: From: Peter Maydell Date: Tue, 6 Sep 2022 14:41:13 +0100 Message-ID: Subject: Re: sphinx-build is really slow, any way to improve that? To: =?UTF-8?Q?Daniel_P=2E_Berrang=C3=A9?= Cc: Claudio Fontana , qemu-devel Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::1132; envelope-from=peter.maydell@linaro.org; helo=mail-yw1-x1132.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Tue, 6 Sept 2022 at 08:55, Daniel P. Berrang=C3=A9 = wrote: > > On Mon, Sep 05, 2022 at 10:21:55PM +0100, Peter Maydell wrote: > > On Mon, 5 Sept 2022 at 20:51, Claudio Fontana wrote: > > > when I build qemu, there is a lot of time spent at the end of the bui= ld where one cpu goes 100% on sphinx-build. > > > > > > Is there some way to parallelize that? It seems it is the current bot= tleneck for rebuilds for me.. > > > > It's a big fat python program, so I suspect not, but > > maybe I'm wrong. > > It annoys me too and I've had a look at what it is doing in the past and > failed to find an obvious way to improve it. I fear this could be an > inherant limitation of the way we use sphinx to build the docs as a > complete manual, as compared to say treating each docs source file as > a distinct standalone web page. IIRC sphinx really really wants to process the whole document tree in one go. You can see this in the way that for example the HTML build process creates HTML files for the top-level rst files that are supposed to be only for the manpage -- it will suck in and process everything, not just the files reachable via whatever top level file you point it at. -- PMM