Build hangs

Build hangs

[Kjeld Flarup]

[-- Attachment #1: Type: text/plain, Size: 1408 bytes --]

I sometimes encounters this situation, where a task is running forever. This example only one task, but I have seen 6 tasks which have been running for 10+ minutes

Currently  1 running tasks (680 of 2941)  23% |###########################                                                                                            |
0: openssl-native-1.1.1d-r0 do_install - 6m17s (pid 6007)

When I look into the docker container I see only these processes
user@yocto:~$ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
user         1     0  0 07:37 pts/0    00:00:00 /bin/bash scripts/ci.sh pdsystems prepare_artifacts
user        53     1  1 07:37 pts/0    00:00:03 python3 /home/user/meta-poky/bitbake/bin/bitbake machine
user        59     1  4 07:37 ?        00:00:14 python3 /home/user/meta-poky/bitbake/bin/bitbake machine
user       414     0  0 07:38 pts/1    00:00:00 bash
user       449    59  5 07:39 ?        00:00:11 python3 /home/user/meta-poky/bitbake/bin/bitbake-worker decafbad
user     15205   414  0 07:42 pts/1    00:00:00 ps -ef

Usually I can fix this by deleting all of by build folder, which of course is annoying as it takes a loong time. Just deleting caches does not help.

Anybody has an idea of why this happens.

Kjeld Flarup

[-- Attachment #2: Type: text/html, Size: 2706 bytes --]

Re: [yocto] Build hangs

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 1620 bytes --]

You can try to list the process tree with ps axjf, and the innermost under
do_install is probably the one that is stuck. Then you can attach gdb to it
to see what it is doing.

Alex

On Fri, 3 Jul 2020 at 10:07, Kjeld Flarup via lists.yoctoproject.org <kfa=
deif.com@lists.yoctoproject.org> wrote:

> I sometimes encounters this situation, where a task is running forever.
> This example only one task, but I have seen 6 tasks which have been running
> for 10+ minutes
>
> Currently  1 running tasks (680 of 2941)  23%
> |###########################
>                                             |
> 0: openssl-native-1.1.1d-r0 do_install - 6m17s (pid 6007)
>
> When I look into the docker container I see only these processes
> user@yocto:~$ ps -ef
> UID        PID  PPID  C STIME TTY          TIME CMD
> user         1     0  0 07:37 pts/0    00:00:00 /bin/bash scripts/ci.sh
> pdsystems prepare_artifacts
> user        53     1  1 07:37 pts/0    00:00:03 python3
> /home/user/meta-poky/bitbake/bin/bitbake machine
> user        59     1  4 07:37 ?        00:00:14 python3
> /home/user/meta-poky/bitbake/bin/bitbake machine
> user       414     0  0 07:38 pts/1    00:00:00 bash
> user       449    59  5 07:39 ?        00:00:11 python3
> /home/user/meta-poky/bitbake/bin/bitbake-worker decafbad
> user     15205   414  0 07:42 pts/1    00:00:00 ps -ef
>
> Usually I can fix this by deleting all of by build folder, which of course
> is annoying as it takes a loong time. Just deleting caches does not help.
>
> Anybody has an idea of why this happens.
>
> Kjeld Flarup
>
> 
>

[-- Attachment #2: Type: text/html, Size: 2852 bytes --]

Re: [yocto] Build hangs

[Kjeld Flarup]

Hi Alex

Thanks for Your suggestion, but there is actually only these three processes running.

user        53     1  1 07:37 pts/0    00:00:03 python3 /home/user/meta-poky/bitbake/bin/bitbake machine
user        59     1  4 07:37 ?        00:00:14 python3 /home/user/meta-poky/bitbake/bin/bitbake machine
user       449    59  5 07:39 ?        00:00:11 python3 /home/user/meta-poky/bitbake/bin/bitbake-worker decafbad

Regards
Kjeld Flarup



________________________________________
From: Alexander Kanavin <alex.kanavin@gmail.com>
Sent: Friday, July 3, 2020 10:25
To: Kjeld Flarup
Cc: yocto@lists.yoctoproject.org
Subject: Re: [yocto] Build hangs

You can try to list the process tree with ps axjf, and the innermost under do_install is probably the one that is stuck. Then you can attach gdb to it to see what it is doing.

Alex

On Fri, 3 Jul 2020 at 10:07, Kjeld Flarup via lists.yoctoproject.org<http://lists.yoctoproject.org> <kfa=deif.com@lists.yoctoproject.org<mailto:deif.com@lists.yoctoproject.org>> wrote:
I sometimes encounters this situation, where a task is running forever. This example only one task, but I have seen 6 tasks which have been running for 10+ minutes

Currently  1 running tasks (680 of 2941)  23% |###########################                                                                                            |
0: openssl-native-1.1.1d-r0 do_install - 6m17s (pid 6007)

When I look into the docker container I see only these processes
user@yocto:~$ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
user         1     0  0 07:37 pts/0    00:00:00 /bin/bash scripts/ci.sh pdsystems prepare_artifacts
user        53     1  1 07:37 pts/0    00:00:03 python3 /home/user/meta-poky/bitbake/bin/bitbake machine
user        59     1  4 07:37 ?        00:00:14 python3 /home/user/meta-poky/bitbake/bin/bitbake machine
user       414     0  0 07:38 pts/1    00:00:00 bash
user       449    59  5 07:39 ?        00:00:11 python3 /home/user/meta-poky/bitbake/bin/bitbake-worker decafbad
user     15205   414  0 07:42 pts/1    00:00:00 ps -ef

Usually I can fix this by deleting all of by build folder, which of course is annoying as it takes a loong time. Just deleting caches does not help.

Anybody has an idea of why this happens.

Kjeld Flarup

Re: [yocto] Build hangs

[Kjeld Flarup]

I found a cause for the error (this time)

In bitbake-cookerdaemon.log I found
recipes-codesys/codesysruntime/codesysruntime.bb: needs merge
recipes-codesys/codesysruntime/codesysruntime.bb: needs merge
recipes-codesys/codesysruntime/codesysruntime.bb: needs merge
recipes-codesys/codesysruntime/codesysruntime.bb: needs merge
recipes-codesys/codesysruntime/codesysruntime.bb: needs merge
recipes-codesys/codesysruntime/codesysruntime.bb: needs merge
recipes-codesys/codesysruntime/codesysruntime.bb: needs merge
recipes-codesys/codesysruntime/codesysruntime.bb: needs merge

After doing a git add on this file the build continued.

I'm however pretty sure that this was not the error in other cases where I experienced this.
In general bitbake does seem to ignore reporting some errors.

Regards
Kjeld Flarup



________________________________________
From: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> on behalf of Kjeld Flarup via lists.yoctoproject.org <kfa=deif.com@lists.yoctoproject.org>
Sent: Friday, July 3, 2020 10:07:07 AM
To: yocto@lists.yoctoproject.org
Subject: [yocto] Build hangs

I sometimes encounters this situation, where a task is running forever. This example only one task, but I have seen 6 tasks which have been running for 10+ minutes

Currently  1 running tasks (680 of 2941)  23% |###########################                                                                                            |
0: openssl-native-1.1.1d-r0 do_install - 6m17s (pid 6007)

When I look into the docker container I see only these processes
user@yocto:~$ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
user         1     0  0 07:37 pts/0    00:00:00 /bin/bash scripts/ci.sh pdsystems prepare_artifacts
user        53     1  1 07:37 pts/0    00:00:03 python3 /home/user/meta-poky/bitbake/bin/bitbake machine
user        59     1  4 07:37 ?        00:00:14 python3 /home/user/meta-poky/bitbake/bin/bitbake machine
user       414     0  0 07:38 pts/1    00:00:00 bash
user       449    59  5 07:39 ?        00:00:11 python3 /home/user/meta-poky/bitbake/bin/bitbake-worker decafbad
user     15205   414  0 07:42 pts/1    00:00:00 ps -ef

Usually I can fix this by deleting all of by build folder, which of course is annoying as it takes a loong time. Just deleting caches does not help.

Anybody has an idea of why this happens.

Kjeld Flarup