Re: [RFC] make sandbox UT more generic

[AKASHI Takahiro <takahiro.akashi@linaro.org>]

Hi Simon,

On Wed, Aug 30, 2023 at 08:49:05PM -0600, Simon Glass wrote:
> Hi,
> 
> On Wed, 30 Aug 2023 at 18:38, AKASHI Takahiro
> <takahiro.akashi@linaro.org> wrote:
> >
> > Hi,
> >
> > I'm working on implementing SCMI-based pinctrl/gpio driver,
> > and want to re-use sandbox UT to test the code. However,
> > It is somehow sandbox-specific (with additional DT nodes).
> > How can/should we make it more generic for other targets/drivers
> > rather than just by copying the test code?
> > (I have already created a test for pinmux since there is only
> > one existing scenario, but gpio test has many.)
> >
> > Even if I say 'generic', my case may be special since real
> > hardware (device drivers) cannot always run all the test cases,
> > while SCMI-based drivers potentially can with a dummy SCMI server
> > for sandbox.
> > See:
> >     drivers/firmware/scmi/sandbox-scmi_agent.c
> 
> We don't have a good way to test drivers that talk to hardware, in general.
> 
> For I2C, SPI and some PCI devices you can sometimes write an emulator
> for the chip and then your driver can talk to the emulator as if it
> were talking to the hardware. Sandbox does actually support that with
> memory-mapped I/O too, although it is fairly rarely used.

Well, I don't want or need to emulate some *real* hardware.
Instead, I would like to emulate what the current sandbox drivers
(pinctrl-sandbox.c and gpio/sandbox.c) emulate so that we can re-use
(some portion of) test cases for sandbox (test/dm/pinmux.c and gpio.c).

As you might know, SCMI protocol with associated drivers on U-Boot is
so generic that it would be able to talk to any of real pinctrl/gpio
drivers/firmware (say, run on OPTEE or SCP).
By implementing/mimicking protocol messages in sandbox-scmi_agent.c,
SCMI drivers are expected to provide *virtual* pinctrl/gpio devices
similar to what sandbox does.

I have already implemented pinmux test with some tweaks by copying
test/dm/pinmux.c and duplicating almost the same DT nodes as "pinctrl-gpio"
in test.dts.
But I'm looking for any other means without test code duplication.

Did I clarify my question a bit?

-Takahiro Akashi


> We have done this a lot with Zephyr, as well[1] and achieved 90% code
> coverage on some boards.
> 
> But I'm not quite sure I am answering the right question, so I will stop here.
> 
> Regards,
> Simon
> 
> [1] https://www.youtube.com/watch?v=usXCAXR2G_c