Openembedded Bitbake Development
 help / color / mirror / Atom feed
From: Mark Hatle <mark.hatle@amd.com>
To: Richard Purdie <richard.purdie@linuxfoundation.org>,
	bitbake-devel@lists.openembedded.org
Subject: Re: [bitbake-devel] [PATCH] runqueue: Don't defer setscene tasks behind already completed tasks
Date: Mon, 16 Nov 2026 08:16:57 -0600	[thread overview]
Message-ID: <dc348ddc-e7cd-4328-b764-6917b2c2030c@amd.com> (raw)
In-Reply-To: <c2030ad5623676d77adb9af2670235d134b6fede.camel@linuxfoundation.org>



On 11/16/26 7:32 AM, Richard Purdie wrote:
...

> 
> This doesn't look quite right to me.
> 
> If you do that, you need to also then follow through and handle
> anything that has a dependency on t.
> 
> Also, the code above this is the only codepath where
> self.runq_complete.add(task) is called. For something to be in
> runq_complete, this function has to have been called for t, so that
> deferral should have already been handled too.
> 
> I have a different suspicion. I think the loop ordering is subtly
> wrong, which is leading to the deadlocking.
> 
> We need to process deferred tasks in multilib order. The main code
> already does this but this section just has a sorted() call. I think it
> may need to be:
> 
> diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
> index 63615ac6c..b098ac170 100644
> --- a/lib/bb/runqueue.py
> +++ b/lib/bb/runqueue.py
> @@ -2109,7 +2109,10 @@ class RunQueueExecute:
>                   logger.debug("Marking task %s as buildable", revdep)
>   
>           found = None
> -        for t in sorted(self.sq_deferred.copy()):
> +        for mc in sorted(self.sqdata.multiconfigs):
> +          for t in sorted(self.sq_deferred.copy()):
> +            if mc_from_tid(t) != mc:
> +                continue
>               if self.sq_deferred[t] == task:
>                   # Allow the next deferred task to run. Any other deferred tasks should be deferred after that task.
>                   # We shouldn't allow all to run at once as it is prone to races.
> 
> which then matches the other code. We probably need a small helper
> function to give that sorting. (yes, I know the indentation needs
> fixing, I left it like this for clarity on what the change is).
> 
> Can you see if that helps please?

Assuming I applied this properly, I'm still getting the same results:

WARNING: Runqueue deadlocked on deferred tasks, forcing task mc:zynqmp-zcu102-sdt-full-microblaze-pmu:virtual:native:/scratch2/mhatle/git/external/experimental/release/meta/recipes-support/sqlite/sqlite3_3.54.1.bb:do_populate_lic blocked by mc:zynqmp-zcu102-sdt-full-cortexa53-fsbl:virtual:native:/scratch2/mhatle/git/external/experimental/release/meta/recipes-support/sqlite/sqlite3_3.54.1.bb:do_populate_lic
WARNING: Runqueue deadlocked on deferred tasks, forcing task mc:zynqmp-zcu102-sdt-full-microblaze-pmu:virtual:native:/scratch2/mhatle/git/external/experimental/release/meta/recipes-support/sqlite/sqlite3_3.54.1.bb:do_populate_sysroot blocked by mc:zynqmp-zcu102-sdt-full-cortexa53-fsbl:virtual:native:/scratch2/mhatle/git/external/experimental/release/meta/recipes-support/sqlite/sqlite3_3.54.1.bb:do_populate_sysroot
WARNING: Runqueue deadlocked on deferred tasks, forcing task mc:zynqmp-zcu102-sdt-full-microblaze-pmu:virtual:native:/scratch2/mhatle/git/external/experimental/release/meta/recipes-support/sqlite/sqlite3_3.54.1.bb:do_recipe_qa blocked by mc:zynqmp-zcu102-sdt-full-cortexa53-fsbl:virtual:native:/scratch2/mhatle/git/external/experimental/release/meta/recipes-support/sqlite/sqlite3_3.54.1.bb:do_recipe_qa

(and many more)

--Mark

  
> Thanks,
> 
> Richard
> 
> 


      reply	other threads:[~2026-11-16 14:17 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-15 22:09 [PATCH] runqueue: Don't defer setscene tasks behind already completed tasks Mark Hatle
2026-11-16 13:32 ` [bitbake-devel] " Richard Purdie
2026-11-16 14:16   ` Mark Hatle [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=dc348ddc-e7cd-4328-b764-6917b2c2030c@amd.com \
    --to=mark.hatle@amd.com \
    --cc=bitbake-devel@lists.openembedded.org \
    --cc=richard.purdie@linuxfoundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox