From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758238AbZCXST2 (ORCPT ); Tue, 24 Mar 2009 14:19:28 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1754564AbZCXSTT (ORCPT ); Tue, 24 Mar 2009 14:19:19 -0400 Received: from mx2.redhat.com ([66.187.237.31]:42865 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754411AbZCXSTS (ORCPT ); Tue, 24 Mar 2009 14:19:18 -0400 Date: Tue, 24 Mar 2009 13:57:50 -0400 From: Jason Baron To: Ingo Molnar Cc: Mathieu Desnoyers , akpm@linux-foundation.org, linux-kernel@vger.kernel.org, ltt-dev@lists.casi.polymtl.ca, Frederic Weisbecker , Peter Zijlstra , Thomas Gleixner , Russell King , Masami Hiramatsu , "Frank Ch. Eigler" , Hideo AOKI , Takashi Nishiie , Steven Rostedt , Eduard - Gabriel Munteanu Subject: Re: [patch 2/9] LTTng instrumentation - irq Message-ID: <20090324175750.GE3129@redhat.com> References: <20090324155625.420966314@polymtl.ca> <20090324160148.080628193@polymtl.ca> <20090324173354.GC3129@redhat.com> <20090324175049.GC31117@elte.hu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090324175049.GC31117@elte.hu> User-Agent: Mutt/1.5.18 (2008-05-17) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 24, 2009 at 06:50:49PM +0100, Ingo Molnar wrote: > * Jason Baron wrote: > > > On Tue, Mar 24, 2009 at 11:56:27AM -0400, Mathieu Desnoyers wrote: > > > Instrumentation of IRQ related events : irq_entry, irq_exit and > > > irq_next_handler. > > > > > > It allows tracers to perform latency analysis on those various types of > > > interrupts and to detect interrupts with max/min/avg duration. It helps > > > detecting driver or hardware problems which cause an ISR to take ages to > > > execute. It has been shown to be the case with bogus hardware causing an mmio > > > read to take a few milliseconds. > > > > > > Those tracepoints are used by LTTng. > > > > > > About the performance impact of tracepoints (which is comparable to markers), > > > even without immediate values optimizations, tests done by Hideo Aoki on ia64 > > > show no regression. His test case was using hackbench on a kernel where > > > scheduler instrumentation (about 5 events in code scheduler code) was added. > > > See the "Tracepoints" patch header for performance result detail. > > > > > > irq_entry and irq_exit not declared static because they appear in x86 arch code. > > > > > > The idea behind logging irq/softirq/tasklet/(and eventually syscall) entry and > > > exit events is to be able to recreate the kernel execution state at a given > > > point in time. Knowing which execution context is responsible for a given trace > > > event is _very_ valuable in trace data analysis. > > > > > > The IRQ instrumentation instruments the IRQ handler entry and exit. Jason > > > instrumented the irq notifier chain calls (irq_handler_entry/exit). His approach > > > provides information about which handler is being called, but does not map > > > correctly to the fact that _multiple_ handlers are being called from within the > > > same interrupt handler. From an interrupt latency analysis POV, this is > > > incorrect. > > > > > > > Since we are passing back the irq number, and we can not be > > interrupted by the same irq, I think it should be pretty clear we > > are in the same handler. That said, the extra entry/exit > > tracepoints could make the sequence of events simpler to decipher, > > which is important. The code looks good, and provides at least as > > much information as the patch that I proposed. So i'll be happy > > either way :) > > We already have your patch merged up in the tracing tree and it > gives entry+exit tracepoints. > > Ingo maybe i wasn't clear. Entry and exit as I proposed and as in the tracing tree are for entry and exit into each handler per irq. Mathieu is proposing an entry/exit tracepoint per irq, and a 3rd tracepoint to tell us which handler is being called and its return code. hope this is clear. thanks, -Jason