Re: [PATCH 3/4] nohz: add tick_nohz_full_clear_cpus() API

[Rik van Riel <riel@redhat.com>]

On 03/30/2015 12:20 PM, Chris Metcalf wrote:
> I wanted to ping the patch below again, since I haven't heard any
> feedback.
> 
> I note that Rik van Riel's change posted this weekend offers similar
> functionality for userspace.  My change offers a convenient API
> for, e.g., kernel drivers setting up default irq balancing.
> 
> https://lkml.org/lkml/2015/3/28/94

I submitted a patch to irqbalance to exclude nohz_full
cpus from having irqs assigned to them.  I could see
the same thing being useful for in-kernel irq assignment,
especially for multi-queue devices that set up irqs on
multiple CPUs.

> An alternate API would be one that just returned the full no_hz
> cpumask to kernel callers; I'd be happy with that as well, but my
> instinct was to make the API as narrow as possible to start with.
> 
> Comments?

What drivers and subsystems are you targeting?

I am just looking at blk-mq now, and it seems like the
API most appropriate for that would be an inline function
that tests whether or not a CPU is nohz_full.

for_each_possible_cpu(i) {

	...

	if (cpu_nohz_full(i))
		continue;

}

A lot of the other code in drivers and subsystems that
set up per-cpu queues and irqs seem to iterate over all
CPUs at init time, and could benefit from a function
allowing them to skip nohz_full CPUs.

Your tick_nohz_full_clear_cpus() function seems reasonable
too, for code that uses a cpumask to set up per cpu stuff.

> On 03/24/2015 03:21 PM, cmetcalf@ezchip.com wrote:
>> From: Chris Metcalf <cmetcalf@ezchip.com>
>>
>> This is useful, for example, to modify a cpumask to avoid the
>> nohz cores so that interrupts aren't sent to them.
>>
>> Signed-off-by: Chris Metcalf <cmetcalf@ezchip.com>
>> ---
>> Motivated by patch 4/4 in this series.
>>
>>   include/linux/tick.h | 7 +++++++
>>   1 file changed, 7 insertions(+)
>>
>> diff --git a/include/linux/tick.h b/include/linux/tick.h
>> index 9c085dc12ae9..d53ad4892a39 100644
>> --- a/include/linux/tick.h
>> +++ b/include/linux/tick.h
>> @@ -186,6 +186,12 @@ static inline bool tick_nohz_full_cpu(int cpu)
>>       return cpumask_test_cpu(cpu, tick_nohz_full_mask);
>>   }
>>   +static inline void tick_nohz_full_clear_cpus(struct cpumask *mask)
>> +{
>> +    if (tick_nohz_full_enabled())
>> +        cpumask_andnot(mask, mask, tick_nohz_full_mask);
>> +}
>> +
>>   extern void __tick_nohz_full_check(void);
>>   extern void tick_nohz_full_kick(void);
>>   extern void tick_nohz_full_kick_cpu(int cpu);
>> @@ -194,6 +200,7 @@ extern void __tick_nohz_task_switch(struct
>> task_struct *tsk);
>>   #else
>>   static inline bool tick_nohz_full_enabled(void) { return false; }
>>   static inline bool tick_nohz_full_cpu(int cpu) { return false; }
>> +static inline void tick_nohz_full_clear_cpus(struct cpumask *mask) { }
>>   static inline void __tick_nohz_full_check(void) { }
>>   static inline void tick_nohz_full_kick_cpu(int cpu) { }
>>   static inline void tick_nohz_full_kick(void) { }
> 


-- 
All rights reversed