Re: docs: requirements.txt has stopped working again

[Akira Yokosawa <akiyks@gmail.com>]

On Wed, 24 Jan 2024 08:25:57 -0700, Jonathan Corbet wrote:
> Akira Yokosawa <akiyks@gmail.com> writes:
> 
>> On Tue, 23 Jan 2024 14:21:32 +0100, Vegard Nossum wrote:
>>> On 23/01/2024 13:30, Jani Nikula wrote:
>>>> On the other hand, if you're using a virtual environment, what's the
>>>> point in holding back to a version as old as 2.4.4? You might just as
>>>> well go for latest, specifying only the top level dependencies,
>>>
>>> Performance... Specifying exact package requirements like 2.4.4 is
>>> useful since 2.4.4 is by far the fastest Sphinx version that builds our
>>> documentation correctly (AFAICT) and build speed matters a lot when the
>>> difference is 10 minutes vs 30 minutes.
>>>
>>
>> I've never observed such a huge difference, probably because I almost
>> always do clean build of the document, i.e., run "make cleandocs" before
>> running "make htmldocs".
>>
>> So I assumed the performance regression Vegard are emphasizing should
>> be in incremental builds.
>>
>> Here are some of the results comparing v2.4.5, v4.3.2 (of ubuntu jammy),
>> and v7.2.6.  (v2.4.5 needs "make -j2 htmldocs" to avoid OOM.)
>> Incremental builds are done after moving from v6.7 to v6.8-rc1.
>>
>> VM spec used: memory: 8GB, threads: 4, ubuntu jammy
>>
>> data in each cell: elapsed time, max resident memory
>>
>>                                     v2.4.5        v4.3.2        v7.2.6
>>   =============================  ============  ============  ============
>>   clean build at v6.7            10m08s 3.3GB  10m31s 1.1GB  10m14s 1.2GB
>>   incremental build at v6.8-rc1  11m22s 3.3GB  18m55s 1.2GB  19m50s 1.4GB
>>   clean build at v6.8-rc1        10m45s 3.2GB  10m32s 1.2GB  10m13s 1.3GB
>>
>>   empty make at v6.8-rc1         3.3s          6.6s          7.0s
>>   =============================  ============  ============  ============
> 
> So that is quite different from my experience.  For me, full builds got
> way slower starting with 3.x and haven't improved much since, though
> I've not played much with 7.x yet.

One of the reasons I can think of why 2.4.5 is not faster is
the "make -j2" I need to use.  2.4.x is way more eager to use
more parallel slots than >=3.1 in later stages of its processing.
I think you have a memory rich system that allows a lot of parallel
slots.  On a machine with 16GB memory, I can say -j4 (or -j5 if
I am lucky).

> 
> It's weird that incremental builds take longer than full for you.
> 

Incremental builds of small differences is faster than full for me
as well.

I used v6.7 --> v6.8-rc1 (full merge window) to emphasize the slowness.

But yes, it's strange to see incremental build becomes slower
than full build even if the diff is a lot.

        Thanks, Akira