Re: [PATCH v13 01/17] preempt: Track NMI nesting to separate per-CPU counter

[Joel Fernandes <joelagnelf@nvidia.com>]

On 10/14/2025 3:43 PM, Peter Zijlstra wrote:
> On Tue, Oct 14, 2025 at 01:55:47PM -0400, Joel Fernandes wrote:
>>
>>
>> On 10/14/2025 6:48 AM, Peter Zijlstra wrote:
>>> On Mon, Oct 13, 2025 at 11:48:03AM -0400, Lyude Paul wrote:
>>>
>>>>  #define __nmi_enter()						\
>>>>  	do {							\
>>>>  		lockdep_off();					\
>>>>  		arch_nmi_enter();				\
>>>> -		BUG_ON(in_nmi() == NMI_MASK);			\
>>>> -		__preempt_count_add(NMI_OFFSET + HARDIRQ_OFFSET);	\
>>>> +		BUG_ON(__this_cpu_read(nmi_nesting) == UINT_MAX);	\
>>>> +		__this_cpu_inc(nmi_nesting);			\
>>>
>>> An NMI that nests from here..
>>>
>>>> +		__preempt_count_add(HARDIRQ_OFFSET);		\
>>>> +		if (__this_cpu_read(nmi_nesting) == 1)		\
>>>
>>> .. until here, will see nmi_nesting > 1 and not set NMI_OFFSET.
>>
>> This is true, I can cure it by setting NMI_OFFSET unconditionally when
>> nmi_nesting >= 1. Then the outer most NMI will then reset it. I think that will
>> work. Do you see any other issue with doing so?
> 
> unconditionally set NMI_FFSET, regardless of nmi_nesting
> and only clear on exit when nmi_nesting == 0.
> 
> Notably, when you use u64 __preempt_count, you can limit this to 32bit
> only. The NMI nesting can happen in the single instruction window
> between ADD and ADC. But on 64bit you don't have that gap and so don't
> need to fix it.

Awesome, I will give this a try, thanks a lot Peter!!

 - Joel