Re: [RFD] Managed interrupt affinities [ Was: mlx5 broken affinity ]

[Thomas Gleixner <tglx@linutronix.de>]

On Mon, 13 Nov 2017, Sagi Grimberg wrote:
> > > Can you explain what do you mean by "subsystem"? I thought that the
> > > subsystem would be the irq subsystem (which means you are the one to
> > > provide
> > > the needed input :) ) and the driver would pass in something
> > > like msi_irq_ops to pci_alloc_irq_vectors() if it supports the driver
> > > requirements that you listed and NULL to tell the core to leave it alone
> > > and do what it sees fit (or pass msi_irq_ops with flag that means that).
> > > 
> > > ops structure is a very common way for drivers to communicate with a
> > > subsystem core.
> > 
> > So if you look at the above pseudo code then the subsys_*_move_callbacks
> > are probably subsystem specific, i.e. block or networking.
> > 
> > Those subsystem callbacks might either handle it at the subsystem level
> > directly or call into the particular driver.
> 
> Personally I do not think that integrating this to networking/block
> stacks in that level is going to work. drivers don't communicate any
> information on what they do with msi(x) vectors (and I'm not sure they
> should).

The callback does not need to transport any information about the interrupt
itself. The driver associates the interrupt to a queue and that queue _is_
known at the subsystem level. So a queue cookie can be handed in along with
the callback information.

I think that the subsystem, e.g. block should handle parts of this, at
least the generic driver independent management of the queue, i.e.

    - Marking the queue as closed so no new requests can be queued
      anymore. Having this at the driver level seems to be a layering
      violation.
    
    - Making sure that the outstanding requests are completed. That needs
      driver support for sure, but OTOH the subsystem should know about
      outstanding requests.

    - Marking the queue as open again. Again that's mostly subsystem level.

Thanks,

	tglx