Re: [bitbake-devel] Got "ValueError: filedescriptor out of range in select" on bitbake v1.46.0

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Fri, 2024-01-12 at 11:40 -0500, Roro Cui via lists.openembedded.org wrote:
> hi, Dear Sir or Madam
> 
> I hit this issue when building the target in yocto dunfell on our new server(512 cpu cores)
> The cooker daemon process quits abnormally. Here is the output
> 
> Command Completed
> Running command ['getVariable', 'BBINCLUDELOGS']
> Command Completed
> Running command ['getVariable', 'BBINCLUDELOGS_LINES']
> Command Completed
> Running command ['getSetVariable', 'BB_CONSOLELOG']
> Command Completed
> Running command ['getSetVariable', 'BB_LOGCONFIG']
> Command Completed
> Running command ['getUIHandlerNum']
> Command Completed
> Running command ['setEventMask', 1, 20, {'BitBake.RunQueue.HashEquiv': 19, 'BitBake.SigGen.HashEquiv': 19}, ['bb.runqueue.runQueueExitWait', 'bb.event.LogExecTTY', 'logging.LogRecord', 'bb.build.TaskFailed', 'bb.build.TaskBase', 'bb.event.ParseStarted', 'bb.event.ParseProgress', 'bb.event.ParseCompleted', 'bb.event.CacheLoadStarted', 'bb.event.CacheLoadProgress', 'bb.event.CacheLoadCompleted', 'bb.command.CommandFailed', 'bb.command.CommandExit', 'bb.command.CommandCompleted', 'bb.cooker.CookerExit', 'bb.event.MultipleProviders', 'bb.event.NoProvider', 'bb.runqueue.sceneQueueTaskStarted', 'bb.runqueue.runQueueTaskStarted', 'bb.runqueue.runQueueTaskFailed', 'bb.runqueue.sceneQueueTaskFailed', 'bb.event.BuildBase', 'bb.build.TaskStarted', 'bb.build.TaskSucceeded', 'bb.build.TaskFailedSilent', 'bb.build.TaskProgress', 'bb.event.ProcessStarted', 'bb.event.ProcessProgress', 'bb.event.ProcessFinished']]
> Command Completed
> Running command ['setConfig', 'cmd', 'fetch']
> Command Completed
> Running command ['buildTargets', ['quilt-native'], 'fetch']
> Command Completed
> Process ProcessServer-1:
> Traceback (most recent call last):
>   File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
>     self.run()
>   File "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py", line 115, in run
>     ret = self.main()
>   File "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py", line 232, in main
>     ready = self.idle_commands(.1, fds)
>   File "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py", line 334, in idle_commands
>     return select.select(fds,[],[],nextsleep)[0]
> ValueError: filedescriptor out of range in select()
> Var <PATH[:=]>:1: SyntaxWarning: "is not" with a literal. Did you mean "!="?
> 
> Is this a known issue? 

It isn't an issue we've been tracking, no. It sounds like your 512 core
machine has probably run more tasks that bitbake can manage correctly.
Try lower values of BB_NUMBER_THREADS and BB_NUMBER_PARSE_THREADS?

You could also open a bug in bugzilla. There are probably ways to
improve the code to avoid select() calls but those will be invasive.

Cheers,

Richard