Re: [Intel-wired-lan] [PATCH net-next v2 4/8] net: napi: add CPU affinity to napi->config

[Ahmed Zaki <ahmed.zaki@intel.com>]

On 2024-12-20 10:23 a.m., Jakub Kicinski wrote:
> On Fri, 20 Dec 2024 07:51:09 -0700 Ahmed Zaki wrote:
>> On 2024-12-19 8:42 p.m., Jakub Kicinski wrote:
>>> On Wed, 18 Dec 2024 09:58:39 -0700 Ahmed Zaki wrote:
>>>> +	if (!glue_created && flags & NAPIF_IRQ_AFFINITY) {
>>>> +		glue = kzalloc(sizeof(*glue), GFP_KERNEL);
>>>> +		if (!glue)
>>>> +			return;
>>>> +		glue->notify.notify = netif_irq_cpu_rmap_notify;
>>>> +		glue->notify.release = netif_napi_affinity_release;
>>>> +		glue->data = napi;
>>>> +		glue->rmap = NULL;
>>>> +		napi->irq_flags |= NAPIF_IRQ_NORMAP;
>>>
>>> Why allocate the glue? is it not possible to add the fields:
>>>
>>> 	struct irq_affinity_notify notify;
>>> 	u16 index;
>>>
>>> to struct napi_struct ?
>>
>> In the first branch of "if", the cb function netif_irq_cpu_rmap_notify()
>> is also passed to irq_cpu_rmap_add() where the irq notifier is embedded
>> in "struct irq_glue".
> 
> I don't understand what you're trying to say, could you rephrase?

Sure. After this patch, we have (simplified):

void netif_napi_set_irq(struct napi_struct *napi, int irq, unsigned long 
flags)
  {
	struct irq_glue *glue = NULL;
  	int  rc;

  	napi->irq = irq;

  #ifdef CONFIG_RFS_ACCEL
  	if (napi->dev->rx_cpu_rmap && flags & NAPIF_IRQ_ARFS_RMAP) {
		rc = irq_cpu_rmap_add(napi->dev->rx_cpu_rmap, irq, napi,
				      netif_irq_cpu_rmap_notify);
		.
		.
		.
  	}
  #endif

	if (flags & NAPIF_IRQ_AFFINITY) {
		glue = kzalloc(sizeof(*glue), GFP_KERNEL);
		if (!glue)
			return;
		glue->notify.notify = netif_irq_cpu_rmap_notify;
		glue->notify.release = netif_napi_affinity_release;
		.
		.
	}
  }


Both branches assign the new cb function "netif_irq_cpu_rmap_notify()" 
as the new IRQ notifier, but the first branch calls irq_cpu_rmap_add() 
where the notifier is embedded in "struct irq_glue". So the cb function 
needs to assume the notifier is inside irq_glue, so the second "if" 
branch needs to do the same.


> 
>> I think this cannot be changed as long as some drivers are directly
>> calling irq_cpu_rmap_add() instead of the proposed API.
> 
> Drivers which are not converted shouldn't matter if we have our own
> notifier and call cpu_rmap_update() directly, no?
> 

Only dependency is that irq_cpu_rmap_add() puts notifier inside irq_glue.

> Drivers which are converted should not call irq_cpu_rmap_add().

Correct, they don't.