Got "ValueError: filedescriptor out of range in select" on bitbake v1.46.0

Got "ValueError: filedescriptor out of range in select" on bitbake v1.46.0

[Roro Cui]

[-- Attachment #1.1: Type: text/plain, Size: 3211 bytes --]

hi, Dear Sir or Madam

I hit this issue when building the target in yocto dunfell on our new
server(512 cpu cores)
The cooker daemon process quits abnormally. Here is the output





























*Command CompletedRunning command ['getVariable', 'BBINCLUDELOGS']Command
CompletedRunning command ['getVariable', 'BBINCLUDELOGS_LINES']Command
CompletedRunning command ['getSetVariable', 'BB_CONSOLELOG']Command
CompletedRunning command ['getSetVariable', 'BB_LOGCONFIG']Command
CompletedRunning command ['getUIHandlerNum']Command CompletedRunning
command ['setEventMask', 1, 20, {'BitBake.RunQueue.HashEquiv': 19,
'BitBake.SigGen.HashEquiv': 19}, ['bb.runqueue.runQueueExitWait',
'bb.event.LogExecTTY', 'logging.LogRecord', 'bb.build.TaskFailed',
'bb.build.TaskBase', 'bb.event.ParseStarted', 'bb.event.ParseProgress',
'bb.event.ParseCompleted', 'bb.event.CacheLoadStarted',
'bb.event.CacheLoadProgress', 'bb.event.CacheLoadCompleted',
'bb.command.CommandFailed', 'bb.command.CommandExit',
'bb.command.CommandCompleted', 'bb.cooker.CookerExit',
'bb.event.MultipleProviders', 'bb.event.NoProvider',
'bb.runqueue.sceneQueueTaskStarted', 'bb.runqueue.runQueueTaskStarted',
'bb.runqueue.runQueueTaskFailed', 'bb.runqueue.sceneQueueTaskFailed',
'bb.event.BuildBase', 'bb.build.TaskStarted', 'bb.build.TaskSucceeded',
'bb.build.TaskFailedSilent', 'bb.build.TaskProgress',
'bb.event.ProcessStarted', 'bb.event.ProcessProgress',
'bb.event.ProcessFinished']]Command CompletedRunning command ['setConfig',
'cmd', 'fetch']Command CompletedRunning command ['buildTargets',
['quilt-native'], 'fetch']Command CompletedProcess
ProcessServer-1:Traceback (most recent call last):  File
"/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()  File
"/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py",
line 115, in run    ret = self.main()  File
"/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py",
line 232, in main    ready = self.idle_commands(.1, fds)  File
"/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py",
line 334, in idle_commands    return
select.select(fds,[],[],nextsleep)[0]ValueError: filedescriptor out of
range in select()Var <PATH[:=]>:1: SyntaxWarning: "is not" with a literal.
Did you mean "!="?*

Is this a known issue?

Thank you.
-- 

Regards

Roro Cui

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.

[-- Attachment #1.2: Type: text/html, Size: 4322 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4197 bytes --]

Re: [bitbake-devel] Got "ValueError: filedescriptor out of range in select" on bitbake v1.46.0

[Richard Purdie]

On Fri, 2024-01-12 at 11:40 -0500, Roro Cui via lists.openembedded.org wrote:
> hi, Dear Sir or Madam
> 
> I hit this issue when building the target in yocto dunfell on our new server(512 cpu cores)
> The cooker daemon process quits abnormally. Here is the output
> 
> Command Completed
> Running command ['getVariable', 'BBINCLUDELOGS']
> Command Completed
> Running command ['getVariable', 'BBINCLUDELOGS_LINES']
> Command Completed
> Running command ['getSetVariable', 'BB_CONSOLELOG']
> Command Completed
> Running command ['getSetVariable', 'BB_LOGCONFIG']
> Command Completed
> Running command ['getUIHandlerNum']
> Command Completed
> Running command ['setEventMask', 1, 20, {'BitBake.RunQueue.HashEquiv': 19, 'BitBake.SigGen.HashEquiv': 19}, ['bb.runqueue.runQueueExitWait', 'bb.event.LogExecTTY', 'logging.LogRecord', 'bb.build.TaskFailed', 'bb.build.TaskBase', 'bb.event.ParseStarted', 'bb.event.ParseProgress', 'bb.event.ParseCompleted', 'bb.event.CacheLoadStarted', 'bb.event.CacheLoadProgress', 'bb.event.CacheLoadCompleted', 'bb.command.CommandFailed', 'bb.command.CommandExit', 'bb.command.CommandCompleted', 'bb.cooker.CookerExit', 'bb.event.MultipleProviders', 'bb.event.NoProvider', 'bb.runqueue.sceneQueueTaskStarted', 'bb.runqueue.runQueueTaskStarted', 'bb.runqueue.runQueueTaskFailed', 'bb.runqueue.sceneQueueTaskFailed', 'bb.event.BuildBase', 'bb.build.TaskStarted', 'bb.build.TaskSucceeded', 'bb.build.TaskFailedSilent', 'bb.build.TaskProgress', 'bb.event.ProcessStarted', 'bb.event.ProcessProgress', 'bb.event.ProcessFinished']]
> Command Completed
> Running command ['setConfig', 'cmd', 'fetch']
> Command Completed
> Running command ['buildTargets', ['quilt-native'], 'fetch']
> Command Completed
> Process ProcessServer-1:
> Traceback (most recent call last):
>   File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
>     self.run()
>   File "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py", line 115, in run
>     ret = self.main()
>   File "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py", line 232, in main
>     ready = self.idle_commands(.1, fds)
>   File "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py", line 334, in idle_commands
>     return select.select(fds,[],[],nextsleep)[0]
> ValueError: filedescriptor out of range in select()
> Var <PATH[:=]>:1: SyntaxWarning: "is not" with a literal. Did you mean "!="?
> 
> Is this a known issue? 

It isn't an issue we've been tracking, no. It sounds like your 512 core
machine has probably run more tasks that bitbake can manage correctly.
Try lower values of BB_NUMBER_THREADS and BB_NUMBER_PARSE_THREADS?

You could also open a bug in bugzilla. There are probably ways to
improve the code to avoid select() calls but those will be invasive.

Cheers,

Richard

Re: [bitbake-devel] Got "ValueError: filedescriptor out of range in select" on bitbake v1.46.0

[Roro Cui]

[-- Attachment #1.1: Type: text/plain, Size: 4559 bytes --]

hi, Richard

Thank you for your reply.
I tried BB_NUMBER_THREADS before,  that does not work, since this issue
happens during 'parsing recipes' stage.
But I guess BB_NUMBER_PARSE_THREADS will work, let me try it.
Will update you soon.
roro

On Fri, Jan 12, 2024 at 12:23 PM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Fri, 2024-01-12 at 11:40 -0500, Roro Cui via lists.openembedded.org
> wrote:
> > hi, Dear Sir or Madam
> >
> > I hit this issue when building the target in yocto dunfell on our new
> server(512 cpu cores)
> > The cooker daemon process quits abnormally. Here is the output
> >
> > Command Completed
> > Running command ['getVariable', 'BBINCLUDELOGS']
> > Command Completed
> > Running command ['getVariable', 'BBINCLUDELOGS_LINES']
> > Command Completed
> > Running command ['getSetVariable', 'BB_CONSOLELOG']
> > Command Completed
> > Running command ['getSetVariable', 'BB_LOGCONFIG']
> > Command Completed
> > Running command ['getUIHandlerNum']
> > Command Completed
> > Running command ['setEventMask', 1, 20, {'BitBake.RunQueue.HashEquiv':
> 19, 'BitBake.SigGen.HashEquiv': 19}, ['bb.runqueue.runQueueExitWait',
> 'bb.event.LogExecTTY', 'logging.LogRecord', 'bb.build.TaskFailed',
> 'bb.build.TaskBase', 'bb.event.ParseStarted', 'bb.event.ParseProgress',
> 'bb.event.ParseCompleted', 'bb.event.CacheLoadStarted',
> 'bb.event.CacheLoadProgress', 'bb.event.CacheLoadCompleted',
> 'bb.command.CommandFailed', 'bb.command.CommandExit',
> 'bb.command.CommandCompleted', 'bb.cooker.CookerExit',
> 'bb.event.MultipleProviders', 'bb.event.NoProvider',
> 'bb.runqueue.sceneQueueTaskStarted', 'bb.runqueue.runQueueTaskStarted',
> 'bb.runqueue.runQueueTaskFailed', 'bb.runqueue.sceneQueueTaskFailed',
> 'bb.event.BuildBase', 'bb.build.TaskStarted', 'bb.build.TaskSucceeded',
> 'bb.build.TaskFailedSilent', 'bb.build.TaskProgress',
> 'bb.event.ProcessStarted', 'bb.event.ProcessProgress',
> 'bb.event.ProcessFinished']]
> > Command Completed
> > Running command ['setConfig', 'cmd', 'fetch']
> > Command Completed
> > Running command ['buildTargets', ['quilt-native'], 'fetch']
> > Command Completed
> > Process ProcessServer-1:
> > Traceback (most recent call last):
> >   File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in
> _bootstrap
> >     self.run()
> >   File
> "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py",
> line 115, in run
> >     ret = self.main()
> >   File
> "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py",
> line 232, in main
> >     ready = self.idle_commands(.1, fds)
> >   File
> "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py",
> line 334, in idle_commands
> >     return select.select(fds,[],[],nextsleep)[0]
> > ValueError: filedescriptor out of range in select()
> > Var <PATH[:=]>:1: SyntaxWarning: "is not" with a literal. Did you mean
> "!="?
> >
> > Is this a known issue?
>
> It isn't an issue we've been tracking, no. It sounds like your 512 core
> machine has probably run more tasks that bitbake can manage correctly.
> Try lower values of BB_NUMBER_THREADS and BB_NUMBER_PARSE_THREADS?
>
> You could also open a bug in bugzilla. There are probably ways to
> improve the code to avoid select() calls but those will be invasive.
>
> Cheers,
>
> Richard
>


-- 

Regards

Roro Cui

Principal Software Engineer, Software Engineering |  STB & CM

 *Broadcom*



*office:* 678.475.2897(X72897) |  *fax:* 678.475.2888

4385 River Green Parkway  |  Duluth, GA 30096

roro.cui@broadcom.com <danny.zhang@broadcom.com>   |   broadcom.com

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.

[-- Attachment #1.2: Type: text/html, Size: 7237 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4197 bytes --]

Re: [bitbake-devel] Got "ValueError: filedescriptor out of range in select" on bitbake v1.46.0

[Roro Cui]

[-- Attachment #1.1: Type: text/plain, Size: 6559 bytes --]

hi, Richard

Setting BB_NUMBER_PARSE_THREADS to 256 fixes my issue.
I am going to tune that value to see which max number triggers this kind of
problem.

Thank you very much for your help.
Roro


On Fri, Jan 12, 2024 at 12:29 PM Roro Cui via lists.openembedded.org
<roro.cui=broadcom.com@lists.openembedded.org> wrote:

> hi, Richard
>
> Thank you for your reply.
> I tried BB_NUMBER_THREADS before,  that does not work, since this issue
> happens during 'parsing recipes' stage.
> But I guess BB_NUMBER_PARSE_THREADS will work, let me try it.
> Will update you soon.
> roro
>
> On Fri, Jan 12, 2024 at 12:23 PM Richard Purdie <
> richard.purdie@linuxfoundation.org> wrote:
>
>> On Fri, 2024-01-12 at 11:40 -0500, Roro Cui via lists.openembedded.org
>> wrote:
>> > hi, Dear Sir or Madam
>> >
>> > I hit this issue when building the target in yocto dunfell on our new
>> server(512 cpu cores)
>> > The cooker daemon process quits abnormally. Here is the output
>> >
>> > Command Completed
>> > Running command ['getVariable', 'BBINCLUDELOGS']
>> > Command Completed
>> > Running command ['getVariable', 'BBINCLUDELOGS_LINES']
>> > Command Completed
>> > Running command ['getSetVariable', 'BB_CONSOLELOG']
>> > Command Completed
>> > Running command ['getSetVariable', 'BB_LOGCONFIG']
>> > Command Completed
>> > Running command ['getUIHandlerNum']
>> > Command Completed
>> > Running command ['setEventMask', 1, 20, {'BitBake.RunQueue.HashEquiv':
>> 19, 'BitBake.SigGen.HashEquiv': 19}, ['bb.runqueue.runQueueExitWait',
>> 'bb.event.LogExecTTY', 'logging.LogRecord', 'bb.build.TaskFailed',
>> 'bb.build.TaskBase', 'bb.event.ParseStarted', 'bb.event.ParseProgress',
>> 'bb.event.ParseCompleted', 'bb.event.CacheLoadStarted',
>> 'bb.event.CacheLoadProgress', 'bb.event.CacheLoadCompleted',
>> 'bb.command.CommandFailed', 'bb.command.CommandExit',
>> 'bb.command.CommandCompleted', 'bb.cooker.CookerExit',
>> 'bb.event.MultipleProviders', 'bb.event.NoProvider',
>> 'bb.runqueue.sceneQueueTaskStarted', 'bb.runqueue.runQueueTaskStarted',
>> 'bb.runqueue.runQueueTaskFailed', 'bb.runqueue.sceneQueueTaskFailed',
>> 'bb.event.BuildBase', 'bb.build.TaskStarted', 'bb.build.TaskSucceeded',
>> 'bb.build.TaskFailedSilent', 'bb.build.TaskProgress',
>> 'bb.event.ProcessStarted', 'bb.event.ProcessProgress',
>> 'bb.event.ProcessFinished']]
>> > Command Completed
>> > Running command ['setConfig', 'cmd', 'fetch']
>> > Command Completed
>> > Running command ['buildTargets', ['quilt-native'], 'fetch']
>> > Command Completed
>> > Process ProcessServer-1:
>> > Traceback (most recent call last):
>> >   File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in
>> _bootstrap
>> >     self.run()
>> >   File
>> "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py",
>> line 115, in run
>> >     ret = self.main()
>> >   File
>> "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py",
>> line 232, in main
>> >     ready = self.idle_commands(.1, fds)
>> >   File
>> "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py",
>> line 334, in idle_commands
>> >     return select.select(fds,[],[],nextsleep)[0]
>> > ValueError: filedescriptor out of range in select()
>> > Var <PATH[:=]>:1: SyntaxWarning: "is not" with a literal. Did you mean
>> "!="?
>> >
>> > Is this a known issue?
>>
>> It isn't an issue we've been tracking, no. It sounds like your 512 core
>> machine has probably run more tasks that bitbake can manage correctly.
>> Try lower values of BB_NUMBER_THREADS and BB_NUMBER_PARSE_THREADS?
>>
>> You could also open a bug in bugzilla. There are probably ways to
>> improve the code to avoid select() calls but those will be invasive.
>>
>> Cheers,
>>
>> Richard
>>
>
>
> --
>
> Regards
>
> Roro Cui
>
> Principal Software Engineer, Software Engineering |  STB & CM
>
>  *Broadcom*
>
>
>
> *office:* 678.475.2897(X72897) |  *fax:* 678.475.2888
>
> 4385 River Green Parkway  |  Duluth, GA 30096
>
> roro.cui@broadcom.com <danny.zhang@broadcom.com>   |   broadcom.com
>
>
>
>
> This electronic communication and the information and any files
> transmitted with it, or attached to it, are confidential and are intended
> solely for the use of the individual or entity to whom it is addressed and
> may contain information that is confidential, legally privileged, protected
> by privacy laws, or otherwise restricted from disclosure to anyone else. If
> you are not the intended recipient or the person responsible for delivering
> the e-mail to the intended recipient, you are hereby notified that any use,
> copying, distributing, dissemination, forwarding, printing, or copying of
> this e-mail is strictly prohibited. If you received this e-mail in error,
> please return the e-mail to the sender, delete it from your computer, and
> destroy any printed copy of it.
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#15791):
> https://lists.openembedded.org/g/bitbake-devel/message/15791
> Mute This Topic: https://lists.openembedded.org/mt/103686457/8130911
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> roro.cui@broadcom.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 

Regards

Roro Cui

Principal Software Engineer, Software Engineering |  STB & CM

 *Broadcom*



*office:* 678.475.2897(X72897) |  *fax:* 678.475.2888

4385 River Green Parkway  |  Duluth, GA 30096

roro.cui@broadcom.com <danny.zhang@broadcom.com>   |   broadcom.com

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.

[-- Attachment #1.2: Type: text/html, Size: 11816 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4197 bytes --]