Re: [PATCH v2 28/28] test: Add a test for booting Ubuntu 24.04

[Tom Rini <trini@konsulko.com>]

[-- Attachment #1: Type: text/plain, Size: 5181 bytes --]

On Tue, Feb 18, 2025 at 05:09:23AM -0700, Simon Glass wrote:
> Hi Tom,
> 
> On Mon, 17 Feb 2025 at 10:52, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Sun, Feb 16, 2025 at 01:44:13PM -0700, Simon Glass wrote:
> > > Now that U-Boot can boot this quickly, using kvm, add a test that the
> > > installer starts up correctly.
> > >
> > > Use the qemu-x86_64 board in the SJG lab.
> > >
> > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > ---
> > >
> > > Changes in v2:
> > > - Add more patches to support booting with kvm
> > > - Add new patch with a test for booting Ubuntu 24.04
> > >
> > >  .gitlab-ci.yml               |  5 ++++
> > >  test/py/tests/test_distro.py | 53 ++++++++++++++++++++++++++++++++++++
> > >  2 files changed, 58 insertions(+)
> > >  create mode 100644 test/py/tests/test_distro.py
> > >
> > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > > index 8c49d5b0a79..ec799e97c10 100644
> > > --- a/.gitlab-ci.yml
> > > +++ b/.gitlab-ci.yml
> > > @@ -745,3 +745,8 @@ zybo:
> > >    variables:
> > >      ROLE: zybo
> > >    <<: *lab_dfn
> > > +
> > > +qemu-x86_64:
> > > +  variables:
> > > +    ROLE: qemu-x86_64
> > > +  <<: *lab_dfn
> >
> > I'm not sure why this is in your lab stanza, rather than the normal
> > test.py QEMU stanza.
> 
> Are you wanting to add the Ubuntu image into CI? It is quite large.

If we're going to be able to run it on N platforms, yes, we need to
think of a good way to cache the download. There's not a particular
reason we can't run the stock Ubuntu RISC-V image on the two sifive
targets and also qemu-riscv64, is there?

> > > diff --git a/test/py/tests/test_distro.py b/test/py/tests/test_distro.py
> > > new file mode 100644
> > > index 00000000000..51eec45cecc
> > > --- /dev/null
> > > +++ b/test/py/tests/test_distro.py
> > > @@ -0,0 +1,53 @@
> > > +# SPDX-License-Identifier: GPL-2.0+
> > > +# Copyright 2025 Canonical Ltd.
> > > +# Written by Simon Glass <simon.glass@canonical.com>
> > > +
> > > +import pytest
> > > +
> > > +DOWN = '\x1b\x5b\x42\x0d'
> > > +
> > > +# Enable early console so that the test can see if something goes wrong
> > > +CONSOLE = 'earlycon=uart8250,io,0x3f8 console=uart8250,io,0x3f8'
> > > +
> > > +@pytest.mark.boardspec('qemu-x86_64')
> > > +@pytest.mark.role('qemu-x86_64')
> > > +def test_distro(ubman):
> > > +    """Test that of-platdata can be generated and used in sandbox"""
> > > +    with ubman.log.section('boot'):
> > > +        ubman.run_command('boot', wait_for_prompt=False)
> > > +
> > > +    with ubman.log.section('Grub'):
> > > +        # Wait for grub to come up and offset a menu
> > > +        ubman.p.expect(['Try or Install Ubuntu'])
> > > +
> > > +        # Press 'e' to edit the command line
> > > +        ubman.run_command('e', wait_for_prompt=False, send_nl=False)
> > > +
> > > +        # Wait until we see the editor appear
> > > +        ubman.p.expect(['/casper/initrd'])
> > > +
> > > +        # Go down to the 'linux' line
> > > +        ubman.send(DOWN * 3)
> > > +
> > > +        # Go to end of line
> > > +        ubman.ctrl('E')
> > > +
> > > +        # Backspace to remove 'quiet splash'
> > > +        ubman.send('\b' * len('quiet splash'))
> > > +
> > > +        # Send our noisy console
> > > +        ubman.send(CONSOLE)
> > > +
> > > +        # Tell grub to boot
> > > +        ubman.ctrl('X')
> > > +        ubman.p.expect(['Booting a command list'])
> > > +
> > > +    with ubman.log.section('Linux'):
> > > +        # Linux should start immediately
> > > +        ubman.p.expect(['Linux version'])
> > > +
> > > +    with ubman.log.section('Ubuntu'):
> > > +        # Shortly later, we should see this banner
> > > +        ubman.p.expect(['Welcome to .*Ubuntu 24.04.1 LTS.*!'])
> > > +
> > > +    ubman.restart_uboot()
> >
> > And this seems very inflexible. Please see
> > test/py/tests/test_net_boot.py for an example of how to have this be
> > configurable and work on arbitrary platforms. What I assume is tricky is
> > that the "role" part here is where you have a special disk image being
> > passed. That too could be dealt with in u-boot-test-hooks in a few ways,
> > and the images pre-fetched to the CI container. And if this was
> > configurable similar to the example I noted above, it could check real
> > hardware too.
> 
> That wasn't the reaction I expected.
> 
> Yes, it is inflexible, but it is a starting point. Isn't it better
> than what we have today?

Is your inflexible boot an OS test better than the flexible boot an OS
test that we have today? No, it's not.

> I would like to generalise this test to work on at least one real
> board, preferably one that doesn't use grub.

OK. The test we have today does that, if you check for the "Welcome to
..." string instead of the kernel has booted string. It also does
netboot rather than run default bootcmd. But that's an easy enough test
to write up. The only thing stopping me from doing that right now is I
need to find a board in the lab where we installed an OS to eMMC and not
SD card (some lab sd-mux issues).

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]