Re: [bitbake-devel] [PATCH] toaster: Add recipe variables view

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Wed, 2026-08-12 at 11:04 -0400, Paolo Wattebled via lists.openembedded.org wrote:
> Toaster records global build variables but does not expose the final
> values associated with individual recipes.
> 
> Collect variables defined or modified by recipes and their bbappends,
> resolve active overrides and expansions, and store one compressed JSON
> snapshot in the database for each build and recipe. Keep the variable
> count separately to avoid decompressing snapshots in recipe-list
> queries.
> 
> Expose the values through a searchable and paginated Variables tab and
> show their counts in the built-recipes table and recipe package view.
> Preserve compatibility with older dependency payloads and invalidate
> stale BitBake caches after extending the cached recipe information.
> 
> Tests cover variable collection, cache transport, database persistence,
> backward compatibility, searching, pagination, escaping, and counts.
> 
> AI-Generated: Uses GitHub Copilot and OpenCode with GPT-5.6 Sol
> Signed-off-by: Paolo Wattebled <paolo.wattebled@savoirfairelinux.com>
> ---
>  bin/bitbake-selftest                          |   1 +
>  lib/bb/cache.py                               |   2 +-
>  lib/bb/cache_extra.py                         |  66 +++++++-
>  lib/bb/cooker.py                              |   6 +-
>  lib/bb/tests/cache_extra.py                   | 141 ++++++++++++++++
>  lib/bb/ui/buildinfohelper.py                  |  21 ++-
>  .../orm/migrations/0022_recipevariable.py     |  27 +++
>  lib/toaster/orm/models.py                     |  10 ++
>  lib/toaster/tests/db/test_db.py               | 101 ++++++++++++
>  lib/toaster/tests/views/test_views.py         | 155 +++++++++++++++++-
>  lib/toaster/toastergui/buildtables.py         |  11 +-
>  .../templates/detail_pagination_bottom.html   |   4 +-
>  .../templates/detail_search_header.html       |   2 +-
>  lib/toaster/toastergui/templates/recipe.html  |  32 ++++
>  .../toastergui/templates/recipe_packages.html |   7 +
>  lib/toaster/toastergui/views.py               |  68 +++++++-
>  16 files changed, 642 insertions(+), 12 deletions(-)
>  create mode 100644 lib/bb/tests/cache_extra.py
>  create mode 100644 lib/toaster/orm/migrations/0022_recipevariable.py

At this point you're basically dumping out the entire datastore for
every possible recipe. Due to the size of the data, I suspect it is
just as expensive to reparse as it is to read the stored compressed
data and it isn't really a cache any more. The presence of that data
will slow the rest of the system down.

Have you looked into the performance (in speed, memory and disk space)
implications of this?

Also, do you know people actively using toaster?

Cheers,

Richard