Re: [PATCH] perf jevents: Limit the number of JSON reading workers

[sashiko-bot@kernel.org]

> The ProcessPoolExecutor is created without max_workers, so
> concurrent.futures defaults to one worker per CPU and forks all of them
> when the first task is submitted. Each worker costs the parent a few
> file descriptors. For systems with high core-count processors like the
> AMD EPYC 9996, the pool can easily exhaust RLIMIT_NOFILE as seen below.
> 
>   File ".../multiprocessing/popen_fork.py", line 65, in _launch
>     child_r, parent_w = os.pipe()
>   OSError: [Errno 24] Too many open files
> 
> The error is reported, but jevents.py does not exit. The workers that
> did start stay blocked waiting for work and are never joined, so the
> build eventually hangs. Other targets keep building in the meantime,
> leaving the traceback several lines above the point where the build
> stalls.
> 
> Limit the pool instead of sizing it against the descriptor limit, which
> would mean relying on how many descriptors concurrent.futures needs for
> each worker. At least for recent AMD EPYC processors, reading the JSON
> files does not need many workers to be fast and things typically stop
> scaling beyond 32 workers. Hence, a small upper bound keeps the speedup
> and stays well inside any usable limit.
> 
> Fixes: eaab2eb09dc2 ("perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py")
> Signed-off-by: Sandipan Das <sandipan.das@amd.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260917095921.2802275-1-sandipan.das@amd.com?part=1