[PATCH 1/3] test/cmd: Make some "ut dm" tests only available on sandbox

[PATCH 1/3] test/cmd: Make some "ut dm" tests only available on sandbox

[Tom Rini]

Currently, the "dm" suite in unit tests (ut) is only available on
sandbox. Make sure that all cmd tests that are part of this suite are
only available on sandbox and not attempted to be run on hardware (where
it will fail to be able to be started).

Signed-off-by: Tom Rini <trini@konsulko.com>
---
 test/cmd/Makefile | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/test/cmd/Makefile b/test/cmd/Makefile
index 4b487c1d2cb4..fe7a2165af22 100644
--- a/test/cmd/Makefile
+++ b/test/cmd/Makefile
@@ -25,16 +25,16 @@ obj-$(CONFIG_CMD_MEM_SEARCH) += mem_search.o
 ifdef CONFIG_CMD_PCI
 obj-$(CONFIG_CMD_PCI_MPS) += pci_mps.o
 endif
-obj-$(CONFIG_CMD_PINMUX) += pinmux.o
-obj-$(CONFIG_CMD_PWM) += pwm.o
 obj-$(CONFIG_CMD_SEAMA) += seama.o
 ifdef CONFIG_SANDBOX
 obj-$(CONFIG_CMD_MBR) += mbr.o
+obj-$(CONFIG_CMD_PINMUX) += pinmux.o
+obj-$(CONFIG_CMD_PWM) += pwm.o
 obj-$(CONFIG_CMD_READ) += rw.o
 obj-$(CONFIG_CMD_SETEXPR) += setexpr.o
+obj-$(CONFIG_CMD_TEMPERATURE) += temperature.o
 ifdef CONFIG_NET
 obj-$(CONFIG_CMD_WGET) += wget.o
 endif
 obj-$(CONFIG_ARM_FFA_TRANSPORT) += armffa.o
 endif
-obj-$(CONFIG_CMD_TEMPERATURE) += temperature.o
-- 
2.43.0

[PATCH 2/3] test/hush: Add CONFIG_CONSOLE_RECORD where required

[Tom Rini]

The "dollar" tests require CONFIG_CONSOLE_RECORD to be enabled so guard
with that.

Signed-off-by: Tom Rini <trini@konsulko.com>
---
 test/hush/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/test/hush/Makefile b/test/hush/Makefile
index a2d98815e501..4c3a0be857ae 100644
--- a/test/hush/Makefile
+++ b/test/hush/Makefile
@@ -5,6 +5,8 @@
 
 obj-y += cmd_ut_hush.o
 obj-y += if.o
+ifdef CONFIG_CONSOLE_RECORD
 obj-y += dollar.o
+endif
 obj-y += list.o
 obj-y += loop.o
-- 
2.43.0

Re: [PATCH 2/3] test/hush: Add CONFIG_CONSOLE_RECORD where required

[Heinrich Schuchardt]

On 10/28/24 17:48, Tom Rini wrote:
> The "dollar" tests require CONFIG_CONSOLE_RECORD to be enabled so guard
> with that.
> 
> Signed-off-by: Tom Rini <trini@konsulko.com>
> ---
>   test/hush/Makefile | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/test/hush/Makefile b/test/hush/Makefile
> index a2d98815e501..4c3a0be857ae 100644
> --- a/test/hush/Makefile
> +++ b/test/hush/Makefile
> @@ -5,6 +5,8 @@
>   
>   obj-y += cmd_ut_hush.o
>   obj-y += if.o
> +ifdef CONFIG_CONSOLE_RECORD
>   obj-y += dollar.o

Or to keep it simpler:

obj-$(CONFIG_CONSOLE_RECORD) += dollar.o

Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>

> +endif
>   obj-y += list.o
>   obj-y += loop.o

[PATCH 3/3] test/lib/uuid: Make lib_test_dynamic_uuid depend on SANDBOX

[Tom Rini]

The dynamic UUID test checks for the sandbox specific capsule UUID to be
used, so we can only perform this test on sandbox currently.

Signed-off-by: Tom Rini <trini@konsulko.com>
---
 test/lib/uuid.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/test/lib/uuid.c b/test/lib/uuid.c
index d00e9563a472..55ef9e7454d7 100644
--- a/test/lib/uuid.c
+++ b/test/lib/uuid.c
@@ -84,6 +84,7 @@ static int lib_test_uuid_bits(struct unit_test_state *uts)
 LIB_TEST(lib_test_uuid_bits, 0);
 #endif
 
+#ifdef CONFIG_SANDBOX
 struct dynamic_uuid_test_data {
 	const char *compatible;
 	const u16 *images[4];
@@ -159,3 +160,4 @@ static int lib_test_dynamic_uuid(struct unit_test_state *uts)
 }
 
 LIB_TEST(lib_test_dynamic_uuid, 0);
+#endif
-- 
2.43.0

Re: [PATCH 3/3] test/lib/uuid: Make lib_test_dynamic_uuid depend on SANDBOX

[Heinrich Schuchardt]

On 10/28/24 17:48, Tom Rini wrote:
> The dynamic UUID test checks for the sandbox specific capsule UUID to be
> used, so we can only perform this test on sandbox currently.

The tested function is gen_v5_guid(). This function is used to generated 
capsule UUIDs. It receives the test data provided in 
dynamic_uuid_test_data test_data[]:

* compatible string
* image name

The generated UUID is compared to a UUID provided in the test data.

By chance the chosen test data contains the string 'sandbox'.

It is not obvious why this test should depend on running on the sandbox.

Where did it fail for you?

Best regards

Heinrich

> 
> Signed-off-by: Tom Rini <trini@konsulko.com>
> ---
>   test/lib/uuid.c | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/test/lib/uuid.c b/test/lib/uuid.c
> index d00e9563a472..55ef9e7454d7 100644
> --- a/test/lib/uuid.c
> +++ b/test/lib/uuid.c
> @@ -84,6 +84,7 @@ static int lib_test_uuid_bits(struct unit_test_state *uts)
>   LIB_TEST(lib_test_uuid_bits, 0);
>   #endif
>   
> +#ifdef CONFIG_SANDBOX
>   struct dynamic_uuid_test_data {
>   	const char *compatible;
>   	const u16 *images[4];
> @@ -159,3 +160,4 @@ static int lib_test_dynamic_uuid(struct unit_test_state *uts)
>   }
>   
>   LIB_TEST(lib_test_dynamic_uuid, 0);
> +#endif

Re: [PATCH 3/3] test/lib/uuid: Make lib_test_dynamic_uuid depend on SANDBOX

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 847 bytes --]

On Mon, Oct 28, 2024 at 10:24:58PM +0100, Heinrich Schuchardt wrote:
> On 10/28/24 17:48, Tom Rini wrote:
> > The dynamic UUID test checks for the sandbox specific capsule UUID to be
> > used, so we can only perform this test on sandbox currently.
> 
> The tested function is gen_v5_guid(). This function is used to generated
> capsule UUIDs. It receives the test data provided in dynamic_uuid_test_data
> test_data[]:
> 
> * compatible string
> * image name
> 
> The generated UUID is compared to a UUID provided in the test data.
> 
> By chance the chosen test data contains the string 'sandbox'.
> 
> It is not obvious why this test should depend on running on the sandbox.
> 
> Where did it fail for you?

On Pi 3, I forget if it was rpi_3 or rpi_arm64_defconfig (with tweaks
like enabling CONFIG_UNIT_TEST).

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH 3/3] test/lib/uuid: Make lib_test_dynamic_uuid depend on SANDBOX

[Heinrich Schuchardt]

On 10/28/24 22:34, Tom Rini wrote:
> On Mon, Oct 28, 2024 at 10:24:58PM +0100, Heinrich Schuchardt wrote:
>> On 10/28/24 17:48, Tom Rini wrote:
>>> The dynamic UUID test checks for the sandbox specific capsule UUID to be
>>> used, so we can only perform this test on sandbox currently.
>>
>> The tested function is gen_v5_guid(). This function is used to generated
>> capsule UUIDs. It receives the test data provided in dynamic_uuid_test_data
>> test_data[]:
>>
>> * compatible string
>> * image name
>>
>> The generated UUID is compared to a UUID provided in the test data.
>>
>> By chance the chosen test data contains the string 'sandbox'.
>>
>> It is not obvious why this test should depend on running on the sandbox.
>>
>> Where did it fail for you?
> 
> On Pi 3, I forget if it was rpi_3 or rpi_arm64_defconfig (with tweaks
> like enabling CONFIG_UNIT_TEST).
> 

On qemu_arm64_defconfig the tests runs fine:

=> ut lib lib_test_dynamic_uuid
Test: lib_test_dynamic_uuid: uuid.c
Failures: 0

Missing CONFIG_SANDBOX cannot be the cause of the issue that you observed.

Best regards

Heinrich

Re: [PATCH 3/3] test/lib/uuid: Make lib_test_dynamic_uuid depend on SANDBOX

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 1540 bytes --]

On Mon, Oct 28, 2024 at 11:01:05PM +0100, Heinrich Schuchardt wrote:
> On 10/28/24 22:34, Tom Rini wrote:
> > On Mon, Oct 28, 2024 at 10:24:58PM +0100, Heinrich Schuchardt wrote:
> > > On 10/28/24 17:48, Tom Rini wrote:
> > > > The dynamic UUID test checks for the sandbox specific capsule UUID to be
> > > > used, so we can only perform this test on sandbox currently.
> > > 
> > > The tested function is gen_v5_guid(). This function is used to generated
> > > capsule UUIDs. It receives the test data provided in dynamic_uuid_test_data
> > > test_data[]:
> > > 
> > > * compatible string
> > > * image name
> > > 
> > > The generated UUID is compared to a UUID provided in the test data.
> > > 
> > > By chance the chosen test data contains the string 'sandbox'.
> > > 
> > > It is not obvious why this test should depend on running on the sandbox.
> > > 
> > > Where did it fail for you?
> > 
> > On Pi 3, I forget if it was rpi_3 or rpi_arm64_defconfig (with tweaks
> > like enabling CONFIG_UNIT_TEST).
> > 
> 
> On qemu_arm64_defconfig the tests runs fine:
> 
> => ut lib lib_test_dynamic_uuid
> Test: lib_test_dynamic_uuid: uuid.c
> Failures: 0
> 
> Missing CONFIG_SANDBOX cannot be the cause of the issue that you observed.

Hunh, OK. Lets for now go with it being related to the 32bit UUID
problem:
https://lore.kernel.org/u-boot/4b60130d-030f-405c-a753-eeb52a5e7b1d@foss.st.com
and that it was on 32bit Pi only where I saw that failure, as indeed I
don't on 64bit Pi now. Thanks!

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH 1/3] test/cmd: Make some "ut dm" tests only available on sandbox

[Heinrich Schuchardt]

On 10/28/24 17:48, Tom Rini wrote:
> Currently, the "dm" suite in unit tests (ut) is only available on
> sandbox. Make sure that all cmd tests that are part of this suite are
> only available on sandbox and not attempted to be run on hardware (where
> it will fail to be able to be started).
>
> Signed-off-by: Tom Rini <trini@konsulko.com>


Adding a comment to the sources describing that the tests require
sandbox drivers would be helpful.

Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>


> ---
>   test/cmd/Makefile | 6 +++---
>   1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/test/cmd/Makefile b/test/cmd/Makefile
> index 4b487c1d2cb4..fe7a2165af22 100644
> --- a/test/cmd/Makefile
> +++ b/test/cmd/Makefile
> @@ -25,16 +25,16 @@ obj-$(CONFIG_CMD_MEM_SEARCH) += mem_search.o
>   ifdef CONFIG_CMD_PCI
>   obj-$(CONFIG_CMD_PCI_MPS) += pci_mps.o
>   endif
> -obj-$(CONFIG_CMD_PINMUX) += pinmux.o
> -obj-$(CONFIG_CMD_PWM) += pwm.o
>   obj-$(CONFIG_CMD_SEAMA) += seama.o
>   ifdef CONFIG_SANDBOX
>   obj-$(CONFIG_CMD_MBR) += mbr.o
> +obj-$(CONFIG_CMD_PINMUX) += pinmux.o
> +obj-$(CONFIG_CMD_PWM) += pwm.o
>   obj-$(CONFIG_CMD_READ) += rw.o
>   obj-$(CONFIG_CMD_SETEXPR) += setexpr.o
> +obj-$(CONFIG_CMD_TEMPERATURE) += temperature.o
>   ifdef CONFIG_NET
>   obj-$(CONFIG_CMD_WGET) += wget.o
>   endif
>   obj-$(CONFIG_ARM_FFA_TRANSPORT) += armffa.o
>   endif
> -obj-$(CONFIG_CMD_TEMPERATURE) += temperature.o

Re: (subset) [PATCH 1/3] test/cmd: Make some "ut dm" tests only available on sandbox

[Tom Rini]

On Mon, 28 Oct 2024 10:48:42 -0600, Tom Rini wrote:

> Currently, the "dm" suite in unit tests (ut) is only available on
> sandbox. Make sure that all cmd tests that are part of this suite are
> only available on sandbox and not attempted to be run on hardware (where
> it will fail to be able to be started).
> 
> 

Applied to u-boot/master, thanks!

-- 
Tom