Re: ftrace histogram sorting broken on BE architecures

Steven Rostedt
2019-12-11T15:36:07Z

On Wed, 11 Dec 2019 13:33:16 +0100
Sven Schnelle <svens@stackframe.org> wrote:

> Hi List,

Hi Sven,

>
> i was looking into a ftracetest failure on s390:
>
> # ./ftracetest test.d/trigger/trigger-hist.tc
> === Ftrace unit tests ===
> [1] event trigger - test histogram trigger [FAIL]
> [2] (instance) event trigger - test histogram trigger [FAIL]
>
> from the -vvv log: ++ fail 'sort param on sched_process_fork did not work'
>
> # cat events/sched/sched_process_fork/hist
>
> # event histogram
> #
> # trigger info: hist:keys=parent_pid,child_pid:vals=hitcount:sort=child_pid:size=2048 [active]
> #
>
> { parent_pid: 1406, child_pid: 1428 } hitcount: 1
> { parent_pid: 1406, child_pid: 1430 } hitcount: 1
> { parent_pid: 1406, child_pid: 1427 } hitcount: 1
> { parent_pid: 1406, child_pid: 1432 } hitcount: 1
> { parent_pid: 1406, child_pid: 1431 } hitcount: 1
> { parent_pid: 1406, child_pid: 1429 } hitcount: 1
>
> So the test is right, the entries are not sorted. After digging into the
> ftrace code i noticed that integer values always get extended to 64 bit
> in event_hist_trigger(), but cmp_entries_key() from tracing_map.c uses the
> type of the field (which is a pid_t, and therefore 4 bytes).
>
> On Little Endian this doesn't hurt, but on BE s390 this makes the compare
> function compare 4 zero bytes, which is the reason why sorting doesn't
> work. As a test i forced the compare function used in cmp_entries_key() to
> tracing_map_cmp_s64(), which made the ftrace tests pass.
>
> I also tested this on 64 bit parisc with the same results, so the architecture
> doesn't seem make a difference (besides LE vs. BE)
>
> Any thoughts on how to fix this? I'm not sure whether i fully understand the
> ftrace maps... ;-)

Your analysis makes sense. I'll take a deeper look at it.

Thanks for reporting it!

-- Steve