Re: Question about TCG backend correctness

["Alex Bennée" <alex.bennee@linaro.org>]

LIU Zhiwei <zhiwei_liu@linux.alibaba.com> writes:

> On 2022/10/18 13:22, Richard Henderson wrote:
>
>  On 10/18/22 01:27, LIU Zhiwei wrote: 
>
>  Maybe I can run RISU on qemu-aarch64(x86) and qemu-aarch64(risc-v) to check the RISC-V backend. 
>
>  This is a good start for debugging a tcg backend. 
>
> After fixing some bugs, RISU can run  some instructions now. Thanks.
>
> IMHO,  the next 2 requests should be satisfied for TCG test.
>
> 1. Add a unit test case for any lower level and small function is easy. 
>
>  For example, the function in risc-v backend, 
>
>  static bool tcg_out_mov(TCGContext *s, TCGType type, TCGReg ret, TCGReg arg)
>   {
>       if (ret == arg) {
>           return true;
>       }
>       switch (type) {
>       case TCG_TYPE_I32:
>       case TCG_TYPE_I64:
>           tcg_out_opc_imm(s, OPC_ADDI, ret, arg, 0);
>           break;
>       default:
>           g_assert_not_reached();
>       }
>       return true;
>   }
>
>  Write a unit test case for it is not easy currently.  I don't know how to fill the TCGContext struct and other
>  parameters.
>  And there is no test framework where I can reuse.
>
>  The others may say that we can run a lot of benchmark to ensure we have a good coverage and don't
>  need such a low level test case. Some reasonable. But it will lead to a very high test burden and it is very hard to
>  get
>  a good coverage only through high level test.

You can at the very least use the gcov build to identify which bits of
the backend are not exercised by "check-tcg" and maybe write a few more
tests to fill it in. It's not a direct exercising of the code but it at
least ensures it has run and worked at least once.

>
> 2. Add a unit test case for any high level function is easy, such as tcg_gen_code. 
>
>  The check-tcg test belongs to the high level testing. 
>
> Best Regards，
> Zhiwei
>
>  It's not comprehensive, because RISU executes one instruction at a time then raises an exception to check the
>  results.  This means that the tcg optimizer doesn't have much to work with, which means that the tcg backend is not
>  as stressed as it could be. 
>
>  I've long wanted to have the ability to have TCG unit tests where a 
>  virtual processor could be defined for the purpose of directly 
>  exercising TCG. 
>
>  We already have many ISAs as the front end of TCG. Will the virtual processor here be some 
>  different? 
>
>  It wouldn't.  This is my argument against creating a new virtual processor. 
>
>  I do think we should be better about creating regression tests for bugs fixed, in the form of small focused assembly
>  test cases which get run via check-tcg. 
>
>  r~ 


-- 
Alex Bennée