Re: FTRACE_WARN_ON((rec->flags & ~FTRACE_FL_MASK) == 0))

[Dave Jones <davej@redhat.com>]

On Wed, Aug 28, 2013 at 10:31:01AM -0400, Steven Rostedt wrote:
 > Dave,
 > 
 > BTW, is there a way to run trinity on a subset of syscalls. Basically,
 > I would like to run it on just the perf code, and nothing else. I have
 > a feeling that the bug you see is not caused by other operations
 > happening (although it could be), but from just out of order perf
 > calls. If I can reproduce it, I'll have a much better way of debugging
 > this.
 
indeed. -c perf_open_event for eg. (You can also specify multiple -c's
if you want to thow in some others)

There's a few other options in there for narrowing things down.
Say you think it's an interaction between perf and and unknown other syscall
for eg, you can do -c perf_open_event -r5. This will pick 5 random
syscalls, and fuzz those plus perf. See scripts/find.sh for an example of
how I've used this in the past.

 > Perhaps another thing you may think of adding to trinity (if it doesn't
 > already exist), is a log of what it is doing. That is, to log somewhere
 > the commands it writes, and that way, if something goes wrong, you have
 > a clue to how it got there. Because this is one of those bugs that
 > triggered before the code crashes, and the crash is just the symptom of
 > what went wrong and does not give you much clue to how it happened.
 
It does have logging already, though for a bug that takes hours, or days to
hit, they can grow to unmanagable sizes, and there's a problem if we have
a situation like..

syscall A
<24 hours of boring syscalls>
syscall B
oops as a result of B's interaction with A.

Quite often just rerunning that last syscall that caused the oops/warn
isn't sufficient to trigger an issue. (Though it may be for this specific
bug that may not be the case..)

Vince has a variant of trinity focussed just on perf which also has some
neat replay/bisecting capabilities to narrow down test cases.
I think I might need to add something like that at some point.

	Dave