[PATCH 0/3] tpm: Start to tidy up TPM tests

[PATCH 0/3] tpm: Start to tidy up TPM tests

[Simon Glass]

This series is a starting point only. It tries to provide some direction
for how the TPM tests should be run on real hardware and on sandbox.

For sandbox, things are relatively easy since the TPM is reset before
each test. Tests should start up the TPM before doing anything. Tests
can be run in parallel, which is fine because tests are independent.

For real hardware, tests cannot be made independent, other than by
resetting the board, which if the hardware is correct, resets the TPM.
So there may be more work to do to figure that out. The approach taken
in this series for real hardware is to have a few tests which do init,
then have the rest of the tests assume that the init is done. Tests
that depend on the TPM already being inited can use 'tpm autostart'
which works OK on sandbox and real hardware.


Simon Glass (3):
  tpm: sandbox: Support self-test continue in emulator
  tpm: Convert sandbox-focussed tests to C
  tpm: Drop unwanted special cases for sandbox

 drivers/tpm/tpm_tis_sandbox.c |  1 +
 test/dm/tpm.c                 | 77 ++++++++++++++++++++++++++-
 test/py/tests/test_tpm2.py    | 99 ++++-------------------------------
 3 files changed, 87 insertions(+), 90 deletions(-)

-- 
2.42.0.820.g83a721a137-goog

[PATCH 1/3] tpm: sandbox: Support self-test continue in emulator

[Simon Glass]

Add support for the self-test continue command in the TPM v1.2 emulator,
to match the functionality in the TPM v2 emulator.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 drivers/tpm/tpm_tis_sandbox.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/tpm/tpm_tis_sandbox.c b/drivers/tpm/tpm_tis_sandbox.c
index 7350e1c4d525..ca992eeb90f9 100644
--- a/drivers/tpm/tpm_tis_sandbox.c
+++ b/drivers/tpm/tpm_tis_sandbox.c
@@ -222,6 +222,7 @@ static int sandbox_tpm_xfer(struct udevice *dev, const uint8_t *sendbuf,
 	case 0x72: /* physical set deactivated */
 	case 0x99: /* startup */
 	case 0x50: /* self test full */
+	case 0x53: /* self test continue */
 	case 0x4000000a:  /* assert physical presence */
 		*recv_len = 12;
 		memset(recvbuf, '\0', *recv_len);
-- 
2.42.0.820.g83a721a137-goog

Re: [PATCH 1/3] tpm: sandbox: Support self-test continue in emulator

[Ilias Apalodimas]

On Sun, Oct 29, 2023 at 05:28:12PM +1300, Simon Glass wrote:
> Add support for the self-test continue command in the TPM v1.2 emulator,
> to match the functionality in the TPM v2 emulator.
> 
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
> 
>  drivers/tpm/tpm_tis_sandbox.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/tpm/tpm_tis_sandbox.c b/drivers/tpm/tpm_tis_sandbox.c
> index 7350e1c4d525..ca992eeb90f9 100644
> --- a/drivers/tpm/tpm_tis_sandbox.c
> +++ b/drivers/tpm/tpm_tis_sandbox.c
> @@ -222,6 +222,7 @@ static int sandbox_tpm_xfer(struct udevice *dev, const uint8_t *sendbuf,
>  	case 0x72: /* physical set deactivated */
>  	case 0x99: /* startup */
>  	case 0x50: /* self test full */
> +	case 0x53: /* self test continue */
>  	case 0x4000000a:  /* assert physical presence */
>  		*recv_len = 12;
>  		memset(recvbuf, '\0', *recv_len);
> -- 
> 2.42.0.820.g83a721a137-goog
> 

Reviewed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>

[PATCH 2/3] tpm: Convert sandbox-focussed tests to C

[Simon Glass]

Some of the Python tests are a pain because they don't reset the TPM
state before each test. Driver model tests do this, so convert the
tests to C.

This means that these tests won't run on real hardware, but we have
tests which do TPM init, so there is still enough coverage.

Rename and update the Python tpm_init test to use 'tpm autostart',
since this deals with starting up ready for the tests below.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 test/dm/tpm.c              | 77 +++++++++++++++++++++++++++++++++++++-
 test/py/tests/test_tpm2.py | 57 +---------------------------
 2 files changed, 77 insertions(+), 57 deletions(-)

diff --git a/test/dm/tpm.c b/test/dm/tpm.c
index cde933ab2848..f8264af13789 100644
--- a/test/dm/tpm.c
+++ b/test/dm/tpm.c
@@ -50,14 +50,87 @@ static int test_tpm_init(struct unit_test_state *uts, enum tpm_version version)
 	return 0;
 }
 
-static int dm_test_tpm(struct unit_test_state *uts)
+static int dm_test_tpm_init(struct unit_test_state *uts)
 {
 	ut_assertok(test_tpm_init(uts, TPM_V1));
 	ut_assertok(test_tpm_init(uts, TPM_V2));
 
 	return 0;
 }
-DM_TEST(dm_test_tpm, UT_TESTF_SCAN_FDT);
+DM_TEST(dm_test_tpm_init, UT_TESTF_SCAN_FDT);
+
+/* Test TPM startup */
+static int test_tpm_startup(struct unit_test_state *uts,
+			    enum tpm_version version)
+{
+	struct udevice *dev;
+
+	/* check probe success */
+	ut_assertok(get_tpm_version(version, &dev));
+
+	ut_assertok(tpm_init(dev));
+	ut_assertok(tpm_startup(dev, TPM_ST_CLEAR));
+
+	return 0;
+}
+
+static int dm_test_tpm_startup(struct unit_test_state *uts)
+{
+	ut_assertok(test_tpm_startup(uts, TPM_V1));
+	ut_assertok(test_tpm_startup(uts, TPM_V2));
+
+	return 0;
+}
+DM_TEST(dm_test_tpm_startup, UT_TESTF_SCAN_FDT);
+
+/* Test TPM self-test full */
+static int test_tpm_self_test_full(struct unit_test_state *uts,
+				   enum tpm_version version)
+{
+	struct udevice *dev;
+
+	/* check probe success */
+	ut_assertok(get_tpm_version(version, &dev));
+
+	ut_assertok(tpm_init(dev));
+	ut_assertok(tpm_startup(dev, TPM_ST_CLEAR));
+
+	return 0;
+}
+
+static int dm_test_tpm_self_test_full(struct unit_test_state *uts)
+{
+	ut_assertok(test_tpm_self_test_full(uts, TPM_V1));
+	ut_assertok(test_tpm_self_test_full(uts, TPM_V2));
+
+	return 0;
+}
+DM_TEST(dm_test_tpm_self_test_full, UT_TESTF_SCAN_FDT);
+
+/* Test TPM self-test continue */
+static int test_tpm_self_test_cont(struct unit_test_state *uts,
+				   enum tpm_version version)
+{
+	struct udevice *dev;
+
+	/* check probe success */
+	ut_assertok(get_tpm_version(version, &dev));
+
+	ut_assertok(tpm_init(dev));
+	ut_assertok(tpm_startup(dev, TPM_ST_CLEAR));
+	ut_assertok(tpm_continue_self_test(dev));
+
+	return 0;
+}
+
+static int dm_test_tpm_self_test_cont(struct unit_test_state *uts)
+{
+	ut_assertok(test_tpm_self_test_cont(uts, TPM_V1));
+	ut_assertok(test_tpm_self_test_cont(uts, TPM_V2));
+
+	return 0;
+}
+DM_TEST(dm_test_tpm_self_test_cont, UT_TESTF_SCAN_FDT);
 
 /* Test report_state */
 static int dm_test_tpm_report_state(struct unit_test_state *uts)
diff --git a/test/py/tests/test_tpm2.py b/test/py/tests/test_tpm2.py
index c2579fa02c58..522ed41de633 100644
--- a/test/py/tests/test_tpm2.py
+++ b/test/py/tests/test_tpm2.py
@@ -56,25 +56,12 @@ def is_sandbox(cons):
     return sys_arch == 'sandbox'
 
 @pytest.mark.buildconfigspec('cmd_tpm_v2')
-def test_tpm2_init(u_boot_console):
+def test_tpm2_autostart(u_boot_console):
     """Init the software stack to use TPMv2 commands."""
     skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
     if skip_test:
         pytest.skip('skip TPM device test')
-    u_boot_console.run_command('tpm2 init')
-    output = u_boot_console.run_command('echo $?')
-    assert output.endswith('0')
-
-@pytest.mark.buildconfigspec('cmd_tpm_v2')
-def test_tpm2_startup(u_boot_console):
-    """Execute a TPM2_Startup command.
-
-    Initiate the TPM internal state machine.
-    """
-    skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
-    if skip_test:
-        pytest.skip('skip TPM device test')
-    u_boot_console.run_command('tpm2 startup TPM2_SU_CLEAR')
+    u_boot_console.run_command('tpm2 autostart')
     output = u_boot_console.run_command('echo $?')
     assert output.endswith('0')
 
@@ -92,46 +79,6 @@ def tpm2_sandbox_init(u_boot_console):
     if skip_test:
         pytest.skip('skip TPM device test')
 
-@pytest.mark.buildconfigspec('cmd_tpm_v2')
-def test_tpm2_sandbox_self_test_full(u_boot_console):
-    """Execute a TPM2_SelfTest (full) command.
-
-    Ask the TPM to perform all self tests to also enable full capabilities.
-    """
-    if is_sandbox(u_boot_console):
-        u_boot_console.restart_uboot()
-        u_boot_console.run_command('tpm2 init')
-        output = u_boot_console.run_command('echo $?')
-        assert output.endswith('0')
-
-        u_boot_console.run_command('tpm2 startup TPM2_SU_CLEAR')
-        output = u_boot_console.run_command('echo $?')
-        assert output.endswith('0')
-
-    skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
-    if skip_test:
-        pytest.skip('skip TPM device test')
-    u_boot_console.run_command('tpm2 self_test full')
-    output = u_boot_console.run_command('echo $?')
-    assert output.endswith('0')
-
-@pytest.mark.buildconfigspec('cmd_tpm_v2')
-def test_tpm2_continue_self_test(u_boot_console):
-    """Execute a TPM2_SelfTest (continued) command.
-
-    Ask the TPM to finish its self tests (alternative to the full test) in order
-    to enter a fully operational state.
-    """
-
-    skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
-    if skip_test:
-        pytest.skip('skip TPM device test')
-    if is_sandbox(u_boot_console):
-        tpm2_sandbox_init(u_boot_console)
-    u_boot_console.run_command('tpm2 self_test continue')
-    output = u_boot_console.run_command('echo $?')
-    assert output.endswith('0')
-
 @pytest.mark.buildconfigspec('cmd_tpm_v2')
 def test_tpm2_clear(u_boot_console):
     """Execute a TPM2_Clear command.
-- 
2.42.0.820.g83a721a137-goog

Re: [PATCH 2/3] tpm: Convert sandbox-focussed tests to C

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 2735 bytes --]

On Sun, Oct 29, 2023 at 05:28:13PM +1300, Simon Glass wrote:

> Some of the Python tests are a pain because they don't reset the TPM
> state before each test. Driver model tests do this, so convert the
> tests to C.
> 
> This means that these tests won't run on real hardware, but we have
> tests which do TPM init, so there is still enough coverage.
> 
> Rename and update the Python tpm_init test to use 'tpm autostart',
> since this deals with starting up ready for the tests below.
> 
> Signed-off-by: Simon Glass <sjg@chromium.org>

I worry that we're removing maybe a few too many of the tests we can be
used on real HW (and QEMU, another real use case) in favor of just
testing them on sandbox. We can certainly have parallel sandbox-only
test paths.

[snip]
> @@ -92,46 +79,6 @@ def tpm2_sandbox_init(u_boot_console):
>      if skip_test:
>          pytest.skip('skip TPM device test')
>  
> -@pytest.mark.buildconfigspec('cmd_tpm_v2')
> -def test_tpm2_sandbox_self_test_full(u_boot_console):
> -    """Execute a TPM2_SelfTest (full) command.
> -
> -    Ask the TPM to perform all self tests to also enable full capabilities.
> -    """
> -    if is_sandbox(u_boot_console):
> -        u_boot_console.restart_uboot()
> -        u_boot_console.run_command('tpm2 init')
> -        output = u_boot_console.run_command('echo $?')
> -        assert output.endswith('0')
> -
> -        u_boot_console.run_command('tpm2 startup TPM2_SU_CLEAR')
> -        output = u_boot_console.run_command('echo $?')
> -        assert output.endswith('0')
> -
> -    skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
> -    if skip_test:
> -        pytest.skip('skip TPM device test')
> -    u_boot_console.run_command('tpm2 self_test full')
> -    output = u_boot_console.run_command('echo $?')
> -    assert output.endswith('0')
> -
> -@pytest.mark.buildconfigspec('cmd_tpm_v2')
> -def test_tpm2_continue_self_test(u_boot_console):
> -    """Execute a TPM2_SelfTest (continued) command.
> -
> -    Ask the TPM to finish its self tests (alternative to the full test) in order
> -    to enter a fully operational state.
> -    """
> -
> -    skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
> -    if skip_test:
> -        pytest.skip('skip TPM device test')
> -    if is_sandbox(u_boot_console):
> -        tpm2_sandbox_init(u_boot_console)
> -    u_boot_console.run_command('tpm2 self_test continue')
> -    output = u_boot_console.run_command('echo $?')
> -    assert output.endswith('0')
> -

I would think these are useful cases to check outside of sandbox. But
I'll let Ilias chime in as I'm just assuming.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH 2/3] tpm: Convert sandbox-focussed tests to C

[Ilias Apalodimas]

Hi Tom

On Sun, 29 Oct 2023 at 16:33, Tom Rini <trini@konsulko.com> wrote:
>
> On Sun, Oct 29, 2023 at 05:28:13PM +1300, Simon Glass wrote:
>
> > Some of the Python tests are a pain because they don't reset the TPM
> > state before each test. Driver model tests do this, so convert the
> > tests to C.
> >
> > This means that these tests won't run on real hardware, but we have
> > tests which do TPM init, so there is still enough coverage.
> >
> > Rename and update the Python tpm_init test to use 'tpm autostart',
> > since this deals with starting up ready for the tests below.
> >
> > Signed-off-by: Simon Glass <sjg@chromium.org>
>
> I worry that we're removing maybe a few too many of the tests we can be
> used on real HW (and QEMU, another real use case) in favor of just
> testing them on sandbox. We can certainly have parallel sandbox-only
> test paths.

I think we are fine.  The auto start that is being called now
initializes the TPM and performs the sefltests.
So the removed tests are covered by just calling 'tpm autostart'.  The
only only opn that shouldn't be removed is
test_tpm2_continue_self_test

>
> [snip]
> > @@ -92,46 +79,6 @@ def tpm2_sandbox_init(u_boot_console):
> >      if skip_test:
> >          pytest.skip('skip TPM device test')
> >
> > -@pytest.mark.buildconfigspec('cmd_tpm_v2')
> > -def test_tpm2_sandbox_self_test_full(u_boot_console):
> > -    """Execute a TPM2_SelfTest (full) command.
> > -
> > -    Ask the TPM to perform all self tests to also enable full capabilities.
> > -    """
> > -    if is_sandbox(u_boot_console):
> > -        u_boot_console.restart_uboot()
> > -        u_boot_console.run_command('tpm2 init')
> > -        output = u_boot_console.run_command('echo $?')
> > -        assert output.endswith('0')
> > -
> > -        u_boot_console.run_command('tpm2 startup TPM2_SU_CLEAR')
> > -        output = u_boot_console.run_command('echo $?')
> > -        assert output.endswith('0')
> > -
> > -    skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
> > -    if skip_test:
> > -        pytest.skip('skip TPM device test')
> > -    u_boot_console.run_command('tpm2 self_test full')
> > -    output = u_boot_console.run_command('echo $?')
> > -    assert output.endswith('0')
> > -
> > -@pytest.mark.buildconfigspec('cmd_tpm_v2')
> > -def test_tpm2_continue_self_test(u_boot_console):
> > -    """Execute a TPM2_SelfTest (continued) command.
> > -
> > -    Ask the TPM to finish its self tests (alternative to the full test) in order
> > -    to enter a fully operational state.
> > -    """
> > -
> > -    skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
> > -    if skip_test:
> > -        pytest.skip('skip TPM device test')
> > -    if is_sandbox(u_boot_console):
> > -        tpm2_sandbox_init(u_boot_console)
> > -    u_boot_console.run_command('tpm2 self_test continue')
> > -    output = u_boot_console.run_command('echo $?')
> > -    assert output.endswith('0')
> > -
>
> I would think these are useful cases to check outside of sandbox. But
> I'll let Ilias chime in as I'm just assuming.

Yep that's the only one we should preserve

Thanks
/Ilias
>
> --
> Tom

Re: [PATCH 2/3] tpm: Convert sandbox-focussed tests to C

[Ilias Apalodimas]

Hi Simon,

Thanks for fixing this.

On Sun, 29 Oct 2023 at 06:28, Simon Glass <sjg@chromium.org> wrote:
>
> Some of the Python tests are a pain because they don't reset the TPM
> state before each test. Driver model tests do this, so convert the
> tests to C.
>
> This means that these tests won't run on real hardware, but we have
> tests which do TPM init, so there is still enough coverage.
>
> Rename and update the Python tpm_init test to use 'tpm autostart',
> since this deals with starting up ready for the tests below.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
>  test/dm/tpm.c              | 77 +++++++++++++++++++++++++++++++++++++-
>  test/py/tests/test_tpm2.py | 57 +---------------------------
>  2 files changed, 77 insertions(+), 57 deletions(-)
>
> diff --git a/test/dm/tpm.c b/test/dm/tpm.c
> index cde933ab2848..f8264af13789 100644
> --- a/test/dm/tpm.c
> +++ b/test/dm/tpm.c
> @@ -50,14 +50,87 @@ static int test_tpm_init(struct unit_test_state *uts, enum tpm_version version)
>         return 0;
>  }
>
> -static int dm_test_tpm(struct unit_test_state *uts)
> +static int dm_test_tpm_init(struct unit_test_state *uts)
>  {
>         ut_assertok(test_tpm_init(uts, TPM_V1));
>         ut_assertok(test_tpm_init(uts, TPM_V2));
>
>         return 0;
>  }
> -DM_TEST(dm_test_tpm, UT_TESTF_SCAN_FDT);
> +DM_TEST(dm_test_tpm_init, UT_TESTF_SCAN_FDT);
> +
> +/* Test TPM startup */
> +static int test_tpm_startup(struct unit_test_state *uts,
> +                           enum tpm_version version)
> +{
> +       struct udevice *dev;
> +
> +       /* check probe success */
> +       ut_assertok(get_tpm_version(version, &dev));
> +
> +       ut_assertok(tpm_init(dev));
> +       ut_assertok(tpm_startup(dev, TPM_ST_CLEAR));
> +
> +       return 0;
> +}
> +
> +static int dm_test_tpm_startup(struct unit_test_state *uts)
> +{
> +       ut_assertok(test_tpm_startup(uts, TPM_V1));
> +       ut_assertok(test_tpm_startup(uts, TPM_V2));
> +
> +       return 0;
> +}
> +DM_TEST(dm_test_tpm_startup, UT_TESTF_SCAN_FDT);
> +
> +/* Test TPM self-test full */
> +static int test_tpm_self_test_full(struct unit_test_state *uts,
> +                                  enum tpm_version version)
> +{
> +       struct udevice *dev;
> +
> +       /* check probe success */
> +       ut_assertok(get_tpm_version(version, &dev));
> +
> +       ut_assertok(tpm_init(dev));
> +       ut_assertok(tpm_startup(dev, TPM_ST_CLEAR));
> +
> +       return 0;
> +}
> +
> +static int dm_test_tpm_self_test_full(struct unit_test_state *uts)
> +{
> +       ut_assertok(test_tpm_self_test_full(uts, TPM_V1));
> +       ut_assertok(test_tpm_self_test_full(uts, TPM_V2));
> +
> +       return 0;
> +}
> +DM_TEST(dm_test_tpm_self_test_full, UT_TESTF_SCAN_FDT);
> +
> +/* Test TPM self-test continue */
> +static int test_tpm_self_test_cont(struct unit_test_state *uts,
> +                                  enum tpm_version version)
> +{
> +       struct udevice *dev;
> +
> +       /* check probe success */
> +       ut_assertok(get_tpm_version(version, &dev));
> +
> +       ut_assertok(tpm_init(dev));
> +       ut_assertok(tpm_startup(dev, TPM_ST_CLEAR));
> +       ut_assertok(tpm_continue_self_test(dev));
> +
> +       return 0;
> +}
> +
> +static int dm_test_tpm_self_test_cont(struct unit_test_state *uts)
> +{
> +       ut_assertok(test_tpm_self_test_cont(uts, TPM_V1));
> +       ut_assertok(test_tpm_self_test_cont(uts, TPM_V2));
> +
> +       return 0;
> +}
> +DM_TEST(dm_test_tpm_self_test_cont, UT_TESTF_SCAN_FDT);
>
>  /* Test report_state */
>  static int dm_test_tpm_report_state(struct unit_test_state *uts)
> diff --git a/test/py/tests/test_tpm2.py b/test/py/tests/test_tpm2.py
> index c2579fa02c58..522ed41de633 100644
> --- a/test/py/tests/test_tpm2.py
> +++ b/test/py/tests/test_tpm2.py
> @@ -56,25 +56,12 @@ def is_sandbox(cons):
>      return sys_arch == 'sandbox'
>
>  @pytest.mark.buildconfigspec('cmd_tpm_v2')
> -def test_tpm2_init(u_boot_console):
> +def test_tpm2_autostart(u_boot_console):
>      """Init the software stack to use TPMv2 commands."""
>      skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
>      if skip_test:
>          pytest.skip('skip TPM device test')
> -    u_boot_console.run_command('tpm2 init')
> -    output = u_boot_console.run_command('echo $?')
> -    assert output.endswith('0')
> -
> -@pytest.mark.buildconfigspec('cmd_tpm_v2')
> -def test_tpm2_startup(u_boot_console):
> -    """Execute a TPM2_Startup command.
> -
> -    Initiate the TPM internal state machine.
> -    """
> -    skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
> -    if skip_test:
> -        pytest.skip('skip TPM device test')
> -    u_boot_console.run_command('tpm2 startup TPM2_SU_CLEAR')
> +    u_boot_console.run_command('tpm2 autostart')
>      output = u_boot_console.run_command('echo $?')
>      assert output.endswith('0')
>
> @@ -92,46 +79,6 @@ def tpm2_sandbox_init(u_boot_console):
>      if skip_test:
>          pytest.skip('skip TPM device test')
>
> -@pytest.mark.buildconfigspec('cmd_tpm_v2')
> -def test_tpm2_sandbox_self_test_full(u_boot_console):
> -    """Execute a TPM2_SelfTest (full) command.
> -
> -    Ask the TPM to perform all self tests to also enable full capabilities.
> -    """
> -    if is_sandbox(u_boot_console):
> -        u_boot_console.restart_uboot()
> -        u_boot_console.run_command('tpm2 init')
> -        output = u_boot_console.run_command('echo $?')
> -        assert output.endswith('0')
> -
> -        u_boot_console.run_command('tpm2 startup TPM2_SU_CLEAR')
> -        output = u_boot_console.run_command('echo $?')
> -        assert output.endswith('0')
> -
> -    skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
> -    if skip_test:
> -        pytest.skip('skip TPM device test')
> -    u_boot_console.run_command('tpm2 self_test full')
> -    output = u_boot_console.run_command('echo $?')
> -    assert output.endswith('0')
> -
> -@pytest.mark.buildconfigspec('cmd_tpm_v2')
> -def test_tpm2_continue_self_test(u_boot_console):
> -    """Execute a TPM2_SelfTest (continued) command.
> -
> -    Ask the TPM to finish its self tests (alternative to the full test) in order
> -    to enter a fully operational state.
> -    """
> -
> -    skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
> -    if skip_test:
> -        pytest.skip('skip TPM device test')
> -    if is_sandbox(u_boot_console):
> -        tpm2_sandbox_init(u_boot_console)
> -    u_boot_console.run_command('tpm2 self_test continue')
> -    output = u_boot_console.run_command('echo $?')
> -    assert output.endswith('0')
> -

Can we please preserve this one only?

>  @pytest.mark.buildconfigspec('cmd_tpm_v2')
>  def test_tpm2_clear(u_boot_console):
>      """Execute a TPM2_Clear command.
> --
> 2.42.0.820.g83a721a137-goog
>

Thanks
/Ilias

[PATCH 3/3] tpm: Drop unwanted special cases for sandbox

[Simon Glass]

These don't seem to be needed.

Add a few notes about what to do next. Also mention parallel tests in
at the top of thefile.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 test/py/tests/test_tpm2.py | 44 +++++++++-----------------------------
 1 file changed, 10 insertions(+), 34 deletions(-)

diff --git a/test/py/tests/test_tpm2.py b/test/py/tests/test_tpm2.py
index 522ed41de633..e01a05143b63 100644
--- a/test/py/tests/test_tpm2.py
+++ b/test/py/tests/test_tpm2.py
@@ -27,6 +27,16 @@ behavior.
 * Setup env__tpm_device_test_skip to True if tests with TPM devices should be
 skipped.
 
+Parallel tests
+--------------
+
+These tests can be run in parallel on sandbox. In that case any action taken
+by one test may be independent of another. For sandbox, care should be taken to
+ensure that tests are independent.
+
+Unfortunately, tests cannot be made independent on real hardware, since there is
+no way to reset the TPM other than restarting the board. Perhaps that would be
+the best approach?
 """
 
 updates = 0
@@ -50,11 +60,6 @@ def force_init(u_boot_console, force=False):
             u_boot_console.run_command('tpm2 clear TPM2_RH_PLATFORM')
         u_boot_console.run_command('echo --- end of init ---')
 
-def is_sandbox(cons):
-    # Array slice removes leading/trailing quotes.
-    sys_arch = cons.config.buildconfig.get('config_sys_arch', '"sandbox"')[1:-1]
-    return sys_arch == 'sandbox'
-
 @pytest.mark.buildconfigspec('cmd_tpm_v2')
 def test_tpm2_autostart(u_boot_console):
     """Init the software stack to use TPMv2 commands."""
@@ -65,20 +70,6 @@ def test_tpm2_autostart(u_boot_console):
     output = u_boot_console.run_command('echo $?')
     assert output.endswith('0')
 
-def tpm2_sandbox_init(u_boot_console):
-    """Put sandbox back into a known state so we can run a test
-
-    This allows all tests to run in parallel, since no test depends on another.
-    """
-    u_boot_console.restart_uboot()
-    u_boot_console.run_command('tpm2 autostart')
-    output = u_boot_console.run_command('echo $?')
-    assert output.endswith('0')
-
-    skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
-    if skip_test:
-        pytest.skip('skip TPM device test')
-
 @pytest.mark.buildconfigspec('cmd_tpm_v2')
 def test_tpm2_clear(u_boot_console):
     """Execute a TPM2_Clear command.
@@ -91,9 +82,6 @@ def test_tpm2_clear(u_boot_console):
     not have a password set, otherwise this test will fail. ENDORSEMENT and
     PLATFORM hierarchies are also available.
     """
-    if is_sandbox(u_boot_console):
-        tpm2_sandbox_init(u_boot_console)
-
     skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
     if skip_test:
         pytest.skip('skip TPM device test')
@@ -114,8 +102,6 @@ def test_tpm2_change_auth(u_boot_console):
     Use the LOCKOUT hierarchy for this. ENDORSEMENT and PLATFORM hierarchies are
     also available.
     """
-    if is_sandbox(u_boot_console):
-        tpm2_sandbox_init(u_boot_console)
     force_init(u_boot_console)
 
     u_boot_console.run_command('tpm2 change_auth TPM2_RH_LOCKOUT unicorn')
@@ -140,9 +126,6 @@ def test_tpm2_get_capability(u_boot_console):
     There is no expected default values because it would depend on the chip
     used. We can still save them in order to check they have changed later.
     """
-    if is_sandbox(u_boot_console):
-        tpm2_sandbox_init(u_boot_console)
-
     force_init(u_boot_console)
     ram = u_boot_utils.find_ram_base(u_boot_console)
 
@@ -164,8 +147,6 @@ def test_tpm2_dam_parameters(u_boot_console):
     the authentication, otherwise the lockout will be engaged after the first
     failed authentication attempt.
     """
-    if is_sandbox(u_boot_console):
-        tpm2_sandbox_init(u_boot_console)
     force_init(u_boot_console)
     ram = u_boot_utils.find_ram_base(u_boot_console)
 
@@ -188,9 +169,6 @@ def test_tpm2_pcr_read(u_boot_console):
 
     Perform a PCR read of the 0th PCR. Must be zero.
     """
-    if is_sandbox(u_boot_console):
-        tpm2_sandbox_init(u_boot_console)
-
     force_init(u_boot_console)
     ram = u_boot_utils.find_ram_base(u_boot_console)
 
@@ -217,8 +195,6 @@ def test_tpm2_pcr_extend(u_boot_console):
     No authentication mechanism is used here, not protecting against packet
     replay, yet.
     """
-    if is_sandbox(u_boot_console):
-        tpm2_sandbox_init(u_boot_console)
     force_init(u_boot_console)
     ram = u_boot_utils.find_ram_base(u_boot_console)
 
-- 
2.42.0.820.g83a721a137-goog

Re: [PATCH 3/3] tpm: Drop unwanted special cases for sandbox

[Ilias Apalodimas]

On Sun, 29 Oct 2023 at 06:28, Simon Glass <sjg@chromium.org> wrote:
>
> These don't seem to be needed.

Please metnion something we can refer to in the future. e.g.
"A previous patch moves most of the sandbox tests to C code, those
arent needed anymore" etc

>
> Add a few notes about what to do next. Also mention parallel tests in
> at the top of thefile.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
>  test/py/tests/test_tpm2.py | 44 +++++++++-----------------------------
>  1 file changed, 10 insertions(+), 34 deletions(-)
>
> diff --git a/test/py/tests/test_tpm2.py b/test/py/tests/test_tpm2.py
> index 522ed41de633..e01a05143b63 100644
> --- a/test/py/tests/test_tpm2.py
> +++ b/test/py/tests/test_tpm2.py
> @@ -27,6 +27,16 @@ behavior.
>  * Setup env__tpm_device_test_skip to True if tests with TPM devices should be
>  skipped.
>
> +Parallel tests
> +--------------
> +
> +These tests can be run in parallel on sandbox. In that case any action taken
> +by one test may be independent of another. For sandbox, care should be taken to
> +ensure that tests are independent.
> +
> +Unfortunately, tests cannot be made independent on real hardware, since there is
> +no way to reset the TPM other than restarting the board. Perhaps that would be
> +the best approach?
>  """
>
>  updates = 0
> @@ -50,11 +60,6 @@ def force_init(u_boot_console, force=False):
>              u_boot_console.run_command('tpm2 clear TPM2_RH_PLATFORM')
>          u_boot_console.run_command('echo --- end of init ---')
>
> -def is_sandbox(cons):
> -    # Array slice removes leading/trailing quotes.
> -    sys_arch = cons.config.buildconfig.get('config_sys_arch', '"sandbox"')[1:-1]
> -    return sys_arch == 'sandbox'
> -
>  @pytest.mark.buildconfigspec('cmd_tpm_v2')
>  def test_tpm2_autostart(u_boot_console):
>      """Init the software stack to use TPMv2 commands."""
> @@ -65,20 +70,6 @@ def test_tpm2_autostart(u_boot_console):
>      output = u_boot_console.run_command('echo $?')
>      assert output.endswith('0')
>
> -def tpm2_sandbox_init(u_boot_console):
> -    """Put sandbox back into a known state so we can run a test
> -
> -    This allows all tests to run in parallel, since no test depends on another.
> -    """
> -    u_boot_console.restart_uboot()
> -    u_boot_console.run_command('tpm2 autostart')
> -    output = u_boot_console.run_command('echo $?')
> -    assert output.endswith('0')
> -
> -    skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
> -    if skip_test:
> -        pytest.skip('skip TPM device test')
> -
>  @pytest.mark.buildconfigspec('cmd_tpm_v2')
>  def test_tpm2_clear(u_boot_console):
>      """Execute a TPM2_Clear command.
> @@ -91,9 +82,6 @@ def test_tpm2_clear(u_boot_console):
>      not have a password set, otherwise this test will fail. ENDORSEMENT and
>      PLATFORM hierarchies are also available.
>      """
> -    if is_sandbox(u_boot_console):
> -        tpm2_sandbox_init(u_boot_console)
> -
>      skip_test = u_boot_console.config.env.get('env__tpm_device_test_skip', False)
>      if skip_test:
>          pytest.skip('skip TPM device test')
> @@ -114,8 +102,6 @@ def test_tpm2_change_auth(u_boot_console):
>      Use the LOCKOUT hierarchy for this. ENDORSEMENT and PLATFORM hierarchies are
>      also available.
>      """
> -    if is_sandbox(u_boot_console):
> -        tpm2_sandbox_init(u_boot_console)
>      force_init(u_boot_console)
>
>      u_boot_console.run_command('tpm2 change_auth TPM2_RH_LOCKOUT unicorn')
> @@ -140,9 +126,6 @@ def test_tpm2_get_capability(u_boot_console):
>      There is no expected default values because it would depend on the chip
>      used. We can still save them in order to check they have changed later.
>      """
> -    if is_sandbox(u_boot_console):
> -        tpm2_sandbox_init(u_boot_console)
> -
>      force_init(u_boot_console)
>      ram = u_boot_utils.find_ram_base(u_boot_console)
>
> @@ -164,8 +147,6 @@ def test_tpm2_dam_parameters(u_boot_console):
>      the authentication, otherwise the lockout will be engaged after the first
>      failed authentication attempt.
>      """
> -    if is_sandbox(u_boot_console):
> -        tpm2_sandbox_init(u_boot_console)
>      force_init(u_boot_console)
>      ram = u_boot_utils.find_ram_base(u_boot_console)
>
> @@ -188,9 +169,6 @@ def test_tpm2_pcr_read(u_boot_console):
>
>      Perform a PCR read of the 0th PCR. Must be zero.
>      """
> -    if is_sandbox(u_boot_console):
> -        tpm2_sandbox_init(u_boot_console)
> -
>      force_init(u_boot_console)
>      ram = u_boot_utils.find_ram_base(u_boot_console)
>
> @@ -217,8 +195,6 @@ def test_tpm2_pcr_extend(u_boot_console):
>      No authentication mechanism is used here, not protecting against packet
>      replay, yet.
>      """
> -    if is_sandbox(u_boot_console):
> -        tpm2_sandbox_init(u_boot_console)
>      force_init(u_boot_console)
>      ram = u_boot_utils.find_ram_base(u_boot_console)
>
> --
> 2.42.0.820.g83a721a137-goog
>

With this rebased on top of patch #2 & master
Reviewed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>

Re: [PATCH 0/3] tpm: Start to tidy up TPM tests

[Ilias Apalodimas]

Thanks Simon,

Apart from a nit in patch#2 this looks good.
Pleas note that this doesn't apply on -master so please rebase

On Sun, 29 Oct 2023 at 06:28, Simon Glass <sjg@chromium.org> wrote:
>
> This series is a starting point only. It tries to provide some direction
> for how the TPM tests should be run on real hardware and on sandbox.
>
> For sandbox, things are relatively easy since the TPM is reset before
> each test. Tests should start up the TPM before doing anything. Tests
> can be run in parallel, which is fine because tests are independent.
>
> For real hardware, tests cannot be made independent, other than by
> resetting the board, which if the hardware is correct, resets the TPM.
> So there may be more work to do to figure that out. The approach taken
> in this series for real hardware is to have a few tests which do init,
> then have the rest of the tests assume that the init is done. Tests
> that depend on the TPM already being inited can use 'tpm autostart'
> which works OK on sandbox and real hardware.
>
>
> Simon Glass (3):
>   tpm: sandbox: Support self-test continue in emulator
>   tpm: Convert sandbox-focussed tests to C
>   tpm: Drop unwanted special cases for sandbox
>
>  drivers/tpm/tpm_tis_sandbox.c |  1 +
>  test/dm/tpm.c                 | 77 ++++++++++++++++++++++++++-
>  test/py/tests/test_tpm2.py    | 99 ++++-------------------------------
>  3 files changed, 87 insertions(+), 90 deletions(-)
>
> --
> 2.42.0.820.g83a721a137-goog
>

Thanks
/Ilias