From mboxrd@z Thu Jan 1 00:00:00 1970 From: Thomas Gleixner Subject: Re: [RFD] Managed interrupt affinities [ Was: mlx5 broken affinity ] Date: Tue, 14 Nov 2017 11:15:10 +0100 (CET) Message-ID: References: <2187e555-2c4e-ef55-1c3a-17f5af54d762@fb.com> <83d3944f-8a31-eb31-93db-294906630b0e@grimberg.me> <556f3ff5-c1d4-25c6-7bfc-9866c0d9b323@fb.com> <9384acdc-a5d8-872c-0034-9a3869f4fc8b@grimberg.me> <1d2e9304-089a-a769-9f38-a742dc066baf@grimberg.me> <67a1157e-06e7-479c-993e-bdf42fd613c6@fb.com> <69a46009-184f-d925-289c-6036f0bf2554@grimberg.me> <7f7aab77-b0b6-f8fa-0b57-3e3c1755eeaa@fb.com> <4888ac89-22d5-0700-daa1-d604e4c54970@fb.com> <4bae1d1d-d401-115d-91cc-4b7df88b02c5@grimberg.me> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Cc: Jens Axboe , Jes Sorensen , Tariq Toukan , Saeed Mahameed , Networking , Leon Romanovsky , Saeed Mahameed , Kernel Team , Christoph Hellwig To: Sagi Grimberg Return-path: Received: from Galois.linutronix.de ([146.0.238.70]:36629 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751887AbdKNKPO (ORCPT ); Tue, 14 Nov 2017 05:15:14 -0500 In-Reply-To: Sender: netdev-owner@vger.kernel.org List-ID: On Mon, 13 Nov 2017, Sagi Grimberg wrote: > > > Can you explain what do you mean by "subsystem"? I thought that the > > > subsystem would be the irq subsystem (which means you are the one to > > > provide > > > the needed input :) ) and the driver would pass in something > > > like msi_irq_ops to pci_alloc_irq_vectors() if it supports the driver > > > requirements that you listed and NULL to tell the core to leave it alone > > > and do what it sees fit (or pass msi_irq_ops with flag that means that). > > > > > > ops structure is a very common way for drivers to communicate with a > > > subsystem core. > > > > So if you look at the above pseudo code then the subsys_*_move_callbacks > > are probably subsystem specific, i.e. block or networking. > > > > Those subsystem callbacks might either handle it at the subsystem level > > directly or call into the particular driver. > > Personally I do not think that integrating this to networking/block > stacks in that level is going to work. drivers don't communicate any > information on what they do with msi(x) vectors (and I'm not sure they > should). The callback does not need to transport any information about the interrupt itself. The driver associates the interrupt to a queue and that queue _is_ known at the subsystem level. So a queue cookie can be handed in along with the callback information. I think that the subsystem, e.g. block should handle parts of this, at least the generic driver independent management of the queue, i.e. - Marking the queue as closed so no new requests can be queued anymore. Having this at the driver level seems to be a layering violation. - Making sure that the outstanding requests are completed. That needs driver support for sure, but OTOH the subsystem should know about outstanding requests. - Marking the queue as open again. Again that's mostly subsystem level. Thanks, tglx