Re: [tip:perf/core] tools lib api fs: Remove debugfs, tracefs and findfs objects

[Matt Fleming <matt@codeblueprint.co.uk>]

On Thu, 24 Sep, at 05:05:57PM, Michael Petlan wrote:
> 
> Hi!
> 
> Yes, we have some tests, but they really need some refactoring and then
> extending.
> 
> There are many "regression" tests that cover some extreme situations
> that failed with some kernel/perf version on some hardware. They are
> probably not very useful for the purpose mentioned here.
 
If they look anything like this,

  https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/commit/?h=perf/core&id=035827e9f2bd71a280f4eb58c65811d377ab2217

i.e. the tests trigger kernel bugs, then I think they would be useful.
If the tests are more along the lines of "you need a huge machine to
trigger the issue caught by the test", maybe not.

> Then there are some tests that should cover basic functionality and
> check for the correctness of perf's behaviour. Since it became being
> pretty messy, I have got an idea to rewrite that in a more structured
> and robust way and make it public.
 
These tests sounds incredibly useful. I would certainly feel better if
I could just hack on random pieces of tools/perf and have the safety
net of regression tests to catch mistakes.

> So I started with some skeleton and tests for perf stat builtin sub
> command [1]. My idea is to port there all the meaningful tests that
> we have at Red Hat. Then I will be happy if someone else is interested
> in contributing some more coverage, ideas or whatever...

My immediate reaction is: please put these tests into tools/perf, do
not create a separate repository.

Now, you've probably got a good reason for wanting to do that, but
definitely let's discuss it first before you go ahead and invest time
and energy in porting things.

You can see my current line of thinking for perf testing with the
perf arch tests series,

  https://lkml.kernel.org/r/1444056021-25721-1-git-send-email-matt@codeblueprint.co.uk  

I think tools/perf as a concenpt (include the userland tool in the
same repo as the kernel) has been very successful because you
frequently get the same developer writing both the userspace and
kernel code. Extending that so the same developer writes the
regression tests too (at the time they introduce their new code!) is
crucial.

-- 
Matt Fleming, Intel Open Source Technology Center