Re: x86/smp: adding new trace points

Re: x86/smp: adding new trace points

[Wojciech Kudla]

Hi all,

I was trying to trace some IPIs (remote tlb shootdowns in this case) and noticed that:

1) irq_vectors:x86_platform_ipi_entry and irq_vectors:x86_platform_ipi_exit are not hit at all for my case. The backtrace on the receiving CPU:

0xffffffff81079535	flush_tlb_func_common.constprop.10+0x105/0x220 [kernel]
0xffffffff81079681	flush_tlb_func_remote+0x31/0x40 [kernel]
0xffffffff8111f76c	flush_smp_call_function_queue+0x4c/0xf0 [kernel]
0xffffffff81120253	generic_smp_call_function_single_interrupt+0x13/0x30 [kernel]
0xffffffff81a030c6	smp_call_function_single_interrupt+0x36/0xd0 [kernel]
0xffffffff81a02679	call_function_single_interrupt+0xa9/0xb0 [kernel]

I would expect that we would hit those trace point somewhere around call_function_single_interrupt()


2) there is no equivalent of ipi:ipi_raise for x86. For the following call stack:

0xffffffff81055d10	native_send_call_func_single_ipi+0x0/0x20 [kernel]
0xffffffff8111f86f	generic_exec_single+0x5f/0xc0 [kernel]
0xffffffff8111f9a2	smp_call_function_single+0xd2/0x100 [kernel]
0xffffffff8111fe3c	smp_call_function_many+0x1cc/0x250 [kernel]
0xffffffff8107982c	native_flush_tlb_others+0x3c/0xf0 [kernel]
(...)

I would expect to have a irq_vectors:x86_platform_ipi_raise (or similar) tracepoint.

Are there any reasons my expectations are wrong?
I'd love to submit a patch that addresses these issue but I'd rather get some more context (history maybe) before that.

Thanks,

Wojtek

Re: x86/smp: adding new trace points

[Thomas Gleixner]

Wojciech Kudla <wk.kernel@gmail.com> writes:
> I was trying to trace some IPIs (remote tlb shootdowns in this case) and noticed that:
>
> 1) irq_vectors:x86_platform_ipi_entry and irq_vectors:x86_platform_ipi_exit are not hit at all for my case. The backtrace on the receiving CPU:
>
> 0xffffffff81079535	flush_tlb_func_common.constprop.10+0x105/0x220 [kernel]
> 0xffffffff81079681	flush_tlb_func_remote+0x31/0x40 [kernel]
> 0xffffffff8111f76c	flush_smp_call_function_queue+0x4c/0xf0 [kernel]
> 0xffffffff81120253	generic_smp_call_function_single_interrupt+0x13/0x30 [kernel]
> 0xffffffff81a030c6	smp_call_function_single_interrupt+0x36/0xd0 [kernel]
> 0xffffffff81a02679	call_function_single_interrupt+0xa9/0xb0 [kernel]
>
> I would expect that we would hit those trace point somewhere around
> call_function_single_interrupt()

Why would the SMP call function single interrupt go through the
PLATFORM_IPI_VECTOR? It goes as the name says through the
CALL_FUNCTION_SINGLE_VECTOR.

Thanks,

        tglx

Re: x86/smp: adding new trace points

[Wojciech Kudla]

On 13/05/2020 13:24, Thomas Gleixner wrote:

> Why would the SMP call function single interrupt go through the
> PLATFORM_IPI_VECTOR? It goes as the name says through the
> CALL_FUNCTION_SINGLE_VECTOR.
> 

Wrong vector, my bad.

However 2) still stands in my opinion. We don't have "ipi raise" trace point for x86.
RESCHEDULE_VECTOR, CALL_FUNCTION_SINGLE_VECTOR, as well as TLB invalidation vectors are essentially inter-processor-interrupts if I'm not mistaken.
Would a patch adding such trace point be considered here?

Thanks,
Wojtek

Re: x86/smp: adding new trace points

[Thomas Gleixner]

Wojciech Kudla <wk.kernel@gmail.com> writes:
> On 13/05/2020 13:24, Thomas Gleixner wrote:
>
>> Why would the SMP call function single interrupt go through the
>> PLATFORM_IPI_VECTOR? It goes as the name says through the
>> CALL_FUNCTION_SINGLE_VECTOR.
>> 
>
> Wrong vector, my bad.
>
> However 2) still stands in my opinion. We don't have "ipi raise" trace
> point for x86.  RESCHEDULE_VECTOR, CALL_FUNCTION_SINGLE_VECTOR, as
> well as TLB invalidation vectors are essentially
> inter-processor-interrupts if I'm not mistaken.  Would a patch adding
> such trace point be considered here?

Maybe.

Though that IPI tracing is inconsistent across architectures. I'm not
really interested to have yet another x86 variant which is slightly
different than anything else.

ARM and ARM64 share generic tracepoints for that, though the actual
tracepoint invocation is in the architecture specific code.

If at all we really want to have the common IPIs which are required for
SMP support covered by generic tracepoints and have them in the generic
code and not sprinkled all over arch/*

Thanks,

        tglx

Re: x86/smp: adding new trace points

[Wojciech Kudla]

On 13/05/2020 17:43, Thomas Gleixner wrote:
> Wojciech Kudla <wk.kernel@gmail.com> writes:
>> On 13/05/2020 13:24, Thomas Gleixner wrote:
>>
>>> Why would the SMP call function single interrupt go through the
>>> PLATFORM_IPI_VECTOR? It goes as the name says through the
>>> CALL_FUNCTION_SINGLE_VECTOR.
>>>
>>
>> Wrong vector, my bad.
>>
>> However 2) still stands in my opinion. We don't have "ipi raise" trace
>> point for x86.  RESCHEDULE_VECTOR, CALL_FUNCTION_SINGLE_VECTOR, as
>> well as TLB invalidation vectors are essentially
>> inter-processor-interrupts if I'm not mistaken.  Would a patch adding
>> such trace point be considered here?
> 
> Maybe.
> 
> Though that IPI tracing is inconsistent across architectures. I'm not
> really interested to have yet another x86 variant which is slightly
> different than anything else.
> 
> ARM and ARM64 share generic tracepoints for that, though the actual
> tracepoint invocation is in the architecture specific code.
> 
> If at all we really want to have the common IPIs which are required for
> SMP support covered by generic tracepoints and have them in the generic
> code and not sprinkled all over arch/*

How about we add ipi:ipi_raise trace points before:
- arch_send_call_function_single_ipi(), and 
- arch_send_call_function_ipi_mask()

Would that be a good starting point to introduce more generic IPI tracing?

Thanks,
Wojtek