Toaster tweaks

Toaster tweaks

[Richard Purdie]

Hi All,

We've noticed toaster was failing more frequently on the autobuilder. I
did a little digging as I wanted to understand why.

Problem 1:

I broke one of the scripts, fixed here:

https://git.openembedded.org/bitbake/commit/?id=75c85c54d9bcc5b732459bef183bab79db00c843

Problem 2:

The layer index has some data the code doesn't like after recent index
changes. I hacked around it:

https://git.openembedded.org/bitbake/commit/?id=74d8b3ad3fe2ce43860819fbb934bbc70cc14b56

We really could do with a better fix for that but this unblocks tests.

Problem 3:

Flaky tests, at least locally:

https://git.openembedded.org/bitbake/commit/?id=3e63c225d8e107526c4b4c8ccb5a85720c665ca1

There are probably more places we need this change. I've also realised
this doesn't fix all the failures I can see locally as the viewport may
end up in the wrong place. I have further local changes which I might
try and clean up next week. We need to continually improve this kind of
issue.

Problem 4:

Our version of django is out of support. I do have a patch from
"django-upgrade" in testing which I could send out for others to
test/review. I'm not 100% convinced by it yet.




For those new to toaster, you can run the tests locally with:

pip install -r ../layers/bitbake/toaster-requirements.txt -r ../layers/bitbake/lib/toaster/tests/toaster-tests-requirements.txt
mkdir toaster_logs
pytest -c ../layers/bitbake/lib/toaster/pytest.ini ../layers/bitbake/lib/toaster/tests/

or an individual test file as:

pytest -c ../layers/bitbake/lib/toaster/pytest.ini ../layers/bitbake/lib/toaster/tests/browser/test_layerdetails_page.py


Cheers,

Richard

RE: Toaster tweaks

[Reyna, David]

Hi Richard,

Thank you for these updates.

I received Bug 16410 Friday and was trying to testing it but was blocked by CloudFlare causing illegal requests with every Layer Index API request. 

I contacted Michael about this and received this reply Saturday:
	Michael: I've just adjusted the rules. Please let me know if the issue is resolved. If not can you help me track down the issue by generating some Forbidden responses and sending your IP address?

I will run the tests you recommend that let you know what I find.

David

-----Original Message-----
From: Richard Purdie <richard.purdie@linuxfoundation.org> 
Sent: Sunday, August 30, 2026 12:02 AM
To: Reyna, David <david.reyna@windriver.com>; toaster@lists.yoctoproject.org
Cc: MacLeod, Randy <Randy.MacLeod@windriver.com>
Subject: Toaster tweaks

Hi All,

We've noticed toaster was failing more frequently on the autobuilder. I
did a little digging as I wanted to understand why.

Problem 1:

I broke one of the scripts, fixed here:

https://git.openembedded.org/bitbake/commit/?id=75c85c54d9bcc5b732459bef183bab79db00c843

Problem 2:

The layer index has some data the code doesn't like after recent index
changes. I hacked around it:

https://git.openembedded.org/bitbake/commit/?id=74d8b3ad3fe2ce43860819fbb934bbc70cc14b56

We really could do with a better fix for that but this unblocks tests.

Problem 3:

Flaky tests, at least locally:

https://git.openembedded.org/bitbake/commit/?id=3e63c225d8e107526c4b4c8ccb5a85720c665ca1

There are probably more places we need this change. I've also realised
this doesn't fix all the failures I can see locally as the viewport may
end up in the wrong place. I have further local changes which I might
try and clean up next week. We need to continually improve this kind of
issue.

Problem 4:

Our version of django is out of support. I do have a patch from
"django-upgrade" in testing which I could send out for others to
test/review. I'm not 100% convinced by it yet.




For those new to toaster, you can run the tests locally with:

pip install -r ../layers/bitbake/toaster-requirements.txt -r ../layers/bitbake/lib/toaster/tests/toaster-tests-requirements.txt
mkdir toaster_logs
pytest -c ../layers/bitbake/lib/toaster/pytest.ini ../layers/bitbake/lib/toaster/tests/

or an individual test file as:

pytest -c ../layers/bitbake/lib/toaster/pytest.ini ../layers/bitbake/lib/toaster/tests/browser/test_layerdetails_page.py


Cheers,

Richard

Re: Toaster tweaks

[Richard Purdie]

Hi David,

On Mon, 2026-08-31 at 07:11 +0000, Reyna, David wrote:
> Thank you for these updates.
> 
> I received Bug 16410 Friday and was trying to testing it but was
> blocked by CloudFlare causing illegal requests with every Layer Index
> API request. 
> 
> I contacted Michael about this and received this reply Saturday:
> 	Michael: I've just adjusted the rules. Please let me know if
> the issue is resolved. If not can you help me track down the issue by
> generating some Forbidden responses and sending your IP address?
> 
> I will run the tests you recommend that let you know what I find.

Just to follow up, I've posted 5 toaster patches to the bitbake list.
One of them is the django upgrade but it I mainly used the automated
tests for testing. Wider review would be appreciated!

It seemed to expose some timing issues in the tests so most of the
fixes are around those.

Cheers,

Richard