* [Xenomai] ipipe tracer question @ 2016-11-02 17:10 Stéphane ancelot 2016-11-03 8:36 ` Henning Schild 0 siblings, 1 reply; 6+ messages in thread From: Stéphane ancelot @ 2016-11-02 17:10 UTC (permalink / raw) To: xenomai@xenomai.org Hi, I have got an RT task that seems being disturbed by some system driver. So, I have setted up ipipe tracer in kernel. But I do not know where to go now ? I suppose being on the right way, but what to do next ? Regards, S.Ancelot ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Xenomai] ipipe tracer question 2016-11-02 17:10 [Xenomai] ipipe tracer question Stéphane ancelot @ 2016-11-03 8:36 ` Henning Schild 2016-11-03 15:29 ` Stéphane ancelot 0 siblings, 1 reply; 6+ messages in thread From: Henning Schild @ 2016-11-03 8:36 UTC (permalink / raw) To: Stéphane ancelot; +Cc: xenomai@xenomai.org Am Wed, 2 Nov 2016 18:10:48 +0100 schrieb Stéphane ancelot <sancelot@numalliance.com>: > Hi, > I have got an RT task that seems being disturbed by some system > driver. > > So, I have setted up ipipe tracer in kernel. The tracing uses the kernel tracing infrastructure. Use the sysfs-Interface or a tool like trace-cmd or perf to collect traces. https://www.kernel.org/doc/Documentation/trace/ftrace.txt > But I do not know where to go now ? > I suppose being on the right way, but what to do next ? > > Regards, > S.Ancelot > > > _______________________________________________ > Xenomai mailing list > Xenomai@xenomai.org > https://xenomai.org/mailman/listinfo/xenomai ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Xenomai] ipipe tracer question 2016-11-03 8:36 ` Henning Schild @ 2016-11-03 15:29 ` Stéphane ancelot 2016-11-03 16:38 ` Henning Schild 0 siblings, 1 reply; 6+ messages in thread From: Stéphane ancelot @ 2016-11-03 15:29 UTC (permalink / raw) To: xenomai@xenomai.org On 03/11/2016 09:36, Henning Schild wrote: > Am Wed, 2 Nov 2016 18:10:48 +0100 > schrieb Stéphane ancelot<sancelot@numalliance.com>: > > >> Hi, >> I have got an RT task that seems being disturbed by some system >> driver. >> >> So, I have setted up ipipe tracer in kernel. >> > The tracing uses the kernel tracing infrastructure. Use the > sysfs-Interface or a tool like trace-cmd or perf to collect traces. > https://www.kernel.org/doc/Documentation/trace/ftrace.txt > > I am now able to trace in the kernel. I made some function trace in the shell, but I have not managed to find where is my problem. I want to find where my tasks spend more time than usuallly. e.g. (end task - start task) > 500us I have some references to cobalt_head_sysentry , cobalt_head_sys I made a trace with trace-cmd and I am able to load it with kernelshark. a bit difficult to understand the graph. >> But I do not know where to go now ? >> I suppose being on the right way, but what to do next ? >> >> Regards, >> S.Ancelot >> >> >> _______________________________________________ >> Xenomai mailing list >> Xenomai@xenomai.org >> https://xenomai.org/mailman/listinfo/xenomai >> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Xenomai] ipipe tracer question 2016-11-03 15:29 ` Stéphane ancelot @ 2016-11-03 16:38 ` Henning Schild 2016-11-04 15:46 ` Stéphane ancelot 0 siblings, 1 reply; 6+ messages in thread From: Henning Schild @ 2016-11-03 16:38 UTC (permalink / raw) To: Stéphane ancelot; +Cc: xenomai@xenomai.org Am Thu, 3 Nov 2016 16:29:08 +0100 schrieb Stéphane ancelot <sancelot@numalliance.com>: > On 03/11/2016 09:36, Henning Schild wrote: > > Am Wed, 2 Nov 2016 18:10:48 +0100 > > schrieb Stéphane ancelot<sancelot@numalliance.com>: > > > > > >> Hi, > >> I have got an RT task that seems being disturbed by some system > >> driver. > >> > >> So, I have setted up ipipe tracer in kernel. > >> > > The tracing uses the kernel tracing infrastructure. Use the > > sysfs-Interface or a tool like trace-cmd or perf to collect traces. > > https://www.kernel.org/doc/Documentation/trace/ftrace.txt > > > > > I am now able to trace in the kernel. > I made some function trace in the shell, but I have not managed to > find where is my problem. > I want to find where my tasks spend more time than usuallly. e.g. > (end task - start task) > 500us > > I have some references to cobalt_head_sysentry , cobalt_head_sys > I made a trace with trace-cmd and I am able to load it with > kernelshark. a bit difficult to understand the graph. I have never worked with kernelshark, usually i read the traces in a pager or text-editor. To get a better correlation with the RT task activity and the kernel, you should leave traces from the application. Have a look at include/cobalt/trace.h to see functions that help you do that. Some of them are used in some of the examples, i did not find documentation, but the names sound kind of meaningful. It is best if you stop tracing as soon as you detected a violation. Filter the trace ie with grep after you collected them, the first thing to read it backwards from the spike filtering by CPU. If you are lucky you have left a "all good" mark somewhere close above the problem. If you still have trouble to read it, maybe kernelshark can process parts of a whole trace. Or you will at least have a better idea where to zoom in. > >> But I do not know where to go now ? > >> I suppose being on the right way, but what to do next ? > >> > >> Regards, > >> S.Ancelot > >> > >> > >> _______________________________________________ > >> Xenomai mailing list > >> Xenomai@xenomai.org > >> https://xenomai.org/mailman/listinfo/xenomai > >> > > _______________________________________________ > Xenomai mailing list > Xenomai@xenomai.org > https://xenomai.org/mailman/listinfo/xenomai ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Xenomai] ipipe tracer question 2016-11-03 16:38 ` Henning Schild @ 2016-11-04 15:46 ` Stéphane ancelot 2016-11-04 15:59 ` Henning Schild 0 siblings, 1 reply; 6+ messages in thread From: Stéphane ancelot @ 2016-11-04 15:46 UTC (permalink / raw) Cc: xenomai@xenomai.org On 03/11/2016 17:38, Henning Schild wrote: Am Thu, 3 Nov 2016 16:29:08 +0100 schrieb Stéphane ancelot [1]<sancelot@numalliance.com>: On 03/11/2016 09:36, Henning Schild wrote: Am Wed, 2 Nov 2016 18:10:48 +0100 schrieb Stéphane ancelot[2]<sancelot@numalliance.com>: Hi, I have got an RT task that seems being disturbed by some system driver. So, I have setted up ipipe tracer in kernel. The tracing uses the kernel tracing infrastructure. Use the sysfs-Interface or a tool like trace-cmd or perf to collect traces. [3]https://www.kernel.org/doc/Documentation/trace/ftrace.txt I am now able to trace in the kernel. I made some function trace in the shell, but I have not managed to find where is my problem. I want to find where my tasks spend more time than usuallly. e.g. (end task - start task) > 500us I have some references to cobalt_head_sysentry , cobalt_head_sys I made a trace with trace-cmd and I am able to load it with kernelshark. a bit difficult to understand the graph. I have never worked with kernelshark, usually i read the traces in a pager or text-editor. To get a better correlation with the RT task activity and the kernel, you should leave traces from the application. Have a look at include/cobalt/trace.h to see functions that help you do that. Some of them are used in some of the examples, i did not find documentation, but the names sound kind of meaningful. I found some informations in web archives regarding these functions: [4]https://mail.gna.org/public/xenomai-core/2006-06/msg00180.html I used xntrace_special() in my program, but unfortunately, altough it does not reply with ENOSYS error, I can't sea really how it is reported in trace log. how can I watch the numbered given in parameters.??? It is best if you stop tracing as soon as you detected a violation. Filter the trace ie with grep after you collected them, the first thing to read it backwards from the spike filtering by CPU. If you are lucky you have left a "all good" mark somewhere close above the problem. If you still have trouble to read it, maybe kernelshark can process parts of a whole trace. Or you will at least have a better idea where to zoom in. But I do not know where to go now ? I suppose being on the right way, but what to do next ? Regards, S.Ancelot _______________________________________________ Xenomai mailing list [5]Xenomai@xenomai.org [6]https://xenomai.org/mailman/listinfo/xenomai _______________________________________________ Xenomai mailing list [7]Xenomai@xenomai.org [8]https://xenomai.org/mailman/listinfo/xenomai References 1. mailto:sancelot@numalliance.com 2. mailto:sancelot@numalliance.com 3. https://www.kernel.org/doc/Documentation/trace/ftrace.txt 4. https://mail.gna.org/public/xenomai-core/2006-06/msg00180.html 5. mailto:Xenomai@xenomai.org 6. https://xenomai.org/mailman/listinfo/xenomai 7. mailto:Xenomai@xenomai.org 8. https://xenomai.org/mailman/listinfo/xenomai ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Xenomai] ipipe tracer question 2016-11-04 15:46 ` Stéphane ancelot @ 2016-11-04 15:59 ` Henning Schild 0 siblings, 0 replies; 6+ messages in thread From: Henning Schild @ 2016-11-04 15:59 UTC (permalink / raw) To: Stéphane ancelot; +Cc: xenomai@xenomai.org Am Fri, 4 Nov 2016 16:46:36 +0100 schrieb Stéphane ancelot <sancelot@numalliance.com>: > On 03/11/2016 17:38, Henning Schild wrote: > > Am Thu, 3 Nov 2016 16:29:08 +0100 > schrieb Stéphane ancelot [1]<sancelot@numalliance.com>: > > > On 03/11/2016 09:36, Henning Schild wrote: > > Am Wed, 2 Nov 2016 18:10:48 +0100 > schrieb Stéphane ancelot[2]<sancelot@numalliance.com>: > > > > Hi, > I have got an RT task that seems being disturbed by some system > driver. > > So, I have setted up ipipe tracer in kernel. > > > The tracing uses the kernel tracing infrastructure. Use the > sysfs-Interface or a tool like trace-cmd or perf to collect traces. > [3]https://www.kernel.org/doc/Documentation/trace/ftrace.txt > > > > I am now able to trace in the kernel. > I made some function trace in the shell, but I have not managed to > find where is my problem. > I want to find where my tasks spend more time than usuallly. e.g. > (end task - start task) > 500us > > I have some references to cobalt_head_sysentry , cobalt_head_sys > I made a trace with trace-cmd and I am able to load it with > kernelshark. a bit difficult to understand the graph. > > I have never worked with kernelshark, usually i read the traces in a > pager or text-editor. > To get a better correlation with the RT task activity and the kernel, > you should leave traces from the application. > Have a look at include/cobalt/trace.h to see functions that help you > do that. Some of them are used in some of the examples, i did not find > documentation, but the names sound kind of meaningful. > > > I found some informations in web archives regarding these > functions: > [4]https://mail.gna.org/public/xenomai-core/2006-06/msg00180.html I > used xntrace_special() in my program, but unfortunately, altough it > does not reply with ENOSYS error, I can't sea really how it is > reported in trace log. how can I watch the numbered given in > parameters.??? I have not used them before so i am not sure. Maybe worth reading the code to see how they eventually make their way to ftrace calls. Maybe they depend on some CONFIG_ I have once used invalid syscalls for somthing like that, not knowing about the official "trace api". That should create trace points, but be carefull not to run into valid syscall numbers. And doing that should be a last resort if for some reason you really can not use xntrace_* > XENOMAI_SYSCALL0(-1); > XENOMAI_SYSCALL0(-2); > ... > (skin max syscall number + 1) > It is best if you stop tracing as soon as you detected a violation. > > Filter the trace ie with grep after you collected them, the first > thing to read it backwards from the spike filtering by CPU. If you > are lucky you have left a "all good" mark somewhere close above the > problem. > > If you still have trouble to read it, maybe kernelshark can process > parts of a whole trace. Or you will at least have a better idea where > to zoom in. > > > > But I do not know where to go now ? > I suppose being on the right way, but what to do next ? > > Regards, > S.Ancelot > > > _______________________________________________ > Xenomai mailing list > [5]Xenomai@xenomai.org > [6]https://xenomai.org/mailman/listinfo/xenomai > > > _______________________________________________ > Xenomai mailing list > [7]Xenomai@xenomai.org > [8]https://xenomai.org/mailman/listinfo/xenomai > > References > > 1. mailto:sancelot@numalliance.com > 2. mailto:sancelot@numalliance.com > 3. https://www.kernel.org/doc/Documentation/trace/ftrace.txt > 4. https://mail.gna.org/public/xenomai-core/2006-06/msg00180.html > 5. mailto:Xenomai@xenomai.org > 6. https://xenomai.org/mailman/listinfo/xenomai > 7. mailto:Xenomai@xenomai.org > 8. https://xenomai.org/mailman/listinfo/xenomai > _______________________________________________ > Xenomai mailing list > Xenomai@xenomai.org > https://xenomai.org/mailman/listinfo/xenomai ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-11-04 15:59 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-11-02 17:10 [Xenomai] ipipe tracer question Stéphane ancelot 2016-11-03 8:36 ` Henning Schild 2016-11-03 15:29 ` Stéphane ancelot 2016-11-03 16:38 ` Henning Schild 2016-11-04 15:46 ` Stéphane ancelot 2016-11-04 15:59 ` Henning Schild
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.