Re: sparse-llvm test cases

[Dibyendu Majumdar <mobile@majumdar.org.uk>]

Hi Chris,

On 10 August 2017 at 19:38, Christopher Li <sparse@chrisli.org> wrote:
> On Sun, Mar 12, 2017 at 9:13 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>> https://github.com/dibyendumajumdar/dmr_c/tree/master/tests
>>
> I take a brief look. I think it has way to many duplicate copy of the
> testrun.sh. You might want to think coherently what exactly is need
> for your different test case. It is better to have one frame work to
> support all the test case.

Yes, the current arrangement is a stop gap. As I develop both on
Windows and Linux, I need a portable solution so I cannot really use
shell scripts.

>
> Another question is that, I don't know how many your test get impact
> by the llvm versions. If it get tight to a specific version of llvm, the
> test case has less value. People will get false positives due to different
> llvm version.
>

The tests are all designed to be runnable - i.e. you can generate the
code and run it either with lli or by generating an executable - so
they are completely independent of LLVM version. But they do need all
the LLVM fixes that are still pending as without those fixes most
tests won't even compile. Nevertheless we can start incorporating
these tests gradually so that when the LLVM fixes are all in then we
will have a good set of tests.

I would prefer to add validation of the Sparse IR as well - at present
I record these but do not try to validate mainly as I think Sparse IR
output is still changing so we need to get to a stable point first.

Regards
Dibyendu