From mboxrd@z Thu Jan 1 00:00:00 1970 From: Benjamin Herrenschmidt Subject: Re: [PATCH RFC 00/77] Re-design MSI/MSI-X interrupts enablement pattern Date: Tue, 08 Oct 2013 07:10:56 +1100 Message-ID: <1381176656.645.171.camel@pasglop> References: <1380840585.3419.50.camel@bwh-desktop.uk.level5networks.com> <20131004082920.GA4536@dhcp-26-207.brq.redhat.com> <1380922156.3214.49.camel@bwh-desktop.uk.level5networks.com> <20131005142054.GA11270@dhcp-26-207.brq.redhat.com> <1381009586.645.141.camel@pasglop> <20131006060243.GB28142@dhcp-26-207.brq.redhat.com> <1381040386.645.143.camel@pasglop> <20131006071027.GA29143@dhcp-26-207.brq.redhat.com> <20131007180111.GC2481@htj.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20131007180111.GC2481@htj.dyndns.org> Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org List-help: List-unsubscribe: List-software: Ecartis version 1.0.0 List-subscribe: List-owner: List-post: List-archive: To: Tejun Heo Cc: Alexander Gordeev , Ben Hutchings , linux-kernel@vger.kernel.org, Bjorn Helgaas , Ralf Baechle , Michael Ellerman , Martin Schwidefsky , Ingo Molnar , Dan Williams , Andy King , Jon Mason , Matt Porter , linux-pci@vger.kernel.org, linux-mips@linux-mips.org, linuxppc-dev@lists.ozlabs.org, linux390@de.ibm.com, linux-s390@vger.kernel.org, x86@kernel.org, linux-ide@vger.kernel.org, iss_storagedev@hp.com, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, e1000-devel@lists.sourceforge.net List-Id: linux-ide@vger.kernel.org On Mon, 2013-10-07 at 14:01 -0400, Tejun Heo wrote: > I don't think the same race condition would happen with the loop. The > problem case is where multiple msi(x) allocation fails completely > because the global limit went down before inquiry and allocation. In > the loop based interface, it'd retry with the lower number. > > As long as the number of drivers which need this sort of adaptive > allocation isn't too high and the common cases can be made simple, I > don't think the "complex" part of interface is all that important. > Maybe we can have reserve / cancel type interface or just keep the > loop with more explicit function names (ie. try_enable or something > like that). I'm thinking a better API overall might just have been to request individual MSI-X one by one :-) We want to be able to request an MSI-X at runtime anyway ... if I want to dynamically add a queue to my network interface, I want it to be able to pop a new arbitrary MSI-X. And we don't want to lock drivers into contiguous MSI-X sets either. And for the cleanup ... well that's what the "pcim" functions are for, we can just make MSI-X variants. Ben. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Mon, 07 Oct 2013 22:12:32 +0200 (CEST) Received: from gate.crashing.org ([63.228.1.57]:35344 "EHLO gate.crashing.org" rhost-flags-OK-OK-OK-OK) by eddie.linux-mips.org with ESMTP id S6868733Ab3JGUM2PNWUZ (ORCPT ); Mon, 7 Oct 2013 22:12:28 +0200 Received: from [127.0.0.1] (localhost.localdomain [127.0.0.1]) by gate.crashing.org (8.14.1/8.13.8) with ESMTP id r97KAxKe027459; Mon, 7 Oct 2013 15:11:00 -0500 Message-ID: <1381176656.645.171.camel@pasglop> Subject: Re: [PATCH RFC 00/77] Re-design MSI/MSI-X interrupts enablement pattern From: Benjamin Herrenschmidt To: Tejun Heo Cc: Alexander Gordeev , Ben Hutchings , linux-kernel@vger.kernel.org, Bjorn Helgaas , Ralf Baechle , Michael Ellerman , Martin Schwidefsky , Ingo Molnar , Dan Williams , Andy King , Jon Mason , Matt Porter , linux-pci@vger.kernel.org, linux-mips@linux-mips.org, linuxppc-dev@lists.ozlabs.org, linux390@de.ibm.com, linux-s390@vger.kernel.org, x86@kernel.org, linux-ide@vger.kernel.org, iss_storagedev@hp.com, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, e1000-devel@lists.sourceforge.net, linux-driver@qlogic.com, Solarflare linux maintainers , "VMware, Inc." , linux-scsi@vger.kernel.org Date: Tue, 08 Oct 2013 07:10:56 +1100 In-Reply-To: <20131007180111.GC2481@htj.dyndns.org> References: <1380840585.3419.50.camel@bwh-desktop.uk.level5networks.com> <20131004082920.GA4536@dhcp-26-207.brq.redhat.com> <1380922156.3214.49.camel@bwh-desktop.uk.level5networks.com> <20131005142054.GA11270@dhcp-26-207.brq.redhat.com> <1381009586.645.141.camel@pasglop> <20131006060243.GB28142@dhcp-26-207.brq.redhat.com> <1381040386.645.143.camel@pasglop> <20131006071027.GA29143@dhcp-26-207.brq.redhat.com> <20131007180111.GC2481@htj.dyndns.org> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.6.4-0ubuntu1 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 38241 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: benh@kernel.crashing.org Precedence: bulk List-help: List-unsubscribe: List-software: Ecartis version 1.0.0 List-Id: linux-mips X-List-ID: linux-mips List-subscribe: List-owner: List-post: List-archive: X-list: linux-mips On Mon, 2013-10-07 at 14:01 -0400, Tejun Heo wrote: > I don't think the same race condition would happen with the loop. The > problem case is where multiple msi(x) allocation fails completely > because the global limit went down before inquiry and allocation. In > the loop based interface, it'd retry with the lower number. > > As long as the number of drivers which need this sort of adaptive > allocation isn't too high and the common cases can be made simple, I > don't think the "complex" part of interface is all that important. > Maybe we can have reserve / cancel type interface or just keep the > loop with more explicit function names (ie. try_enable or something > like that). I'm thinking a better API overall might just have been to request individual MSI-X one by one :-) We want to be able to request an MSI-X at runtime anyway ... if I want to dynamically add a queue to my network interface, I want it to be able to pop a new arbitrary MSI-X. And we don't want to lock drivers into contiguous MSI-X sets either. And for the cleanup ... well that's what the "pcim" functions are for, we can just make MSI-X variants. Ben. From mboxrd@z Thu Jan 1 00:00:00 1970 From: benh@kernel.crashing.org (Benjamin Herrenschmidt) Date: Tue, 08 Oct 2013 07:10:56 +1100 Subject: [PATCH RFC 00/77] Re-design MSI/MSI-X interrupts enablement pattern In-Reply-To: <20131007180111.GC2481@htj.dyndns.org> References: <1380840585.3419.50.camel@bwh-desktop.uk.level5networks.com> <20131004082920.GA4536@dhcp-26-207.brq.redhat.com> <1380922156.3214.49.camel@bwh-desktop.uk.level5networks.com> <20131005142054.GA11270@dhcp-26-207.brq.redhat.com> <1381009586.645.141.camel@pasglop> <20131006060243.GB28142@dhcp-26-207.brq.redhat.com> <1381040386.645.143.camel@pasglop> <20131006071027.GA29143@dhcp-26-207.brq.redhat.com> <20131007180111.GC2481@htj.dyndns.org> Message-ID: <1381176656.645.171.camel@pasglop> On Mon, 2013-10-07@14:01 -0400, Tejun Heo wrote: > I don't think the same race condition would happen with the loop. The > problem case is where multiple msi(x) allocation fails completely > because the global limit went down before inquiry and allocation. In > the loop based interface, it'd retry with the lower number. > > As long as the number of drivers which need this sort of adaptive > allocation isn't too high and the common cases can be made simple, I > don't think the "complex" part of interface is all that important. > Maybe we can have reserve / cancel type interface or just keep the > loop with more explicit function names (ie. try_enable or something > like that). I'm thinking a better API overall might just have been to request individual MSI-X one by one :-) We want to be able to request an MSI-X at runtime anyway ... if I want to dynamically add a queue to my network interface, I want it to be able to pop a new arbitrary MSI-X. And we don't want to lock drivers into contiguous MSI-X sets either. And for the cleanup ... well that's what the "pcim" functions are for, we can just make MSI-X variants. Ben. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate.crashing.org (gate.crashing.org [63.228.1.57]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id BEDEF2C020A for ; Tue, 8 Oct 2013 07:12:56 +1100 (EST) Message-ID: <1381176656.645.171.camel@pasglop> Subject: Re: [PATCH RFC 00/77] Re-design MSI/MSI-X interrupts enablement pattern From: Benjamin Herrenschmidt To: Tejun Heo Date: Tue, 08 Oct 2013 07:10:56 +1100 In-Reply-To: <20131007180111.GC2481@htj.dyndns.org> References: <1380840585.3419.50.camel@bwh-desktop.uk.level5networks.com> <20131004082920.GA4536@dhcp-26-207.brq.redhat.com> <1380922156.3214.49.camel@bwh-desktop.uk.level5networks.com> <20131005142054.GA11270@dhcp-26-207.brq.redhat.com> <1381009586.645.141.camel@pasglop> <20131006060243.GB28142@dhcp-26-207.brq.redhat.com> <1381040386.645.143.camel@pasglop> <20131006071027.GA29143@dhcp-26-207.brq.redhat.com> <20131007180111.GC2481@htj.dyndns.org> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Cc: linux-mips@linux-mips.org, "VMware, Inc." , linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-ide@vger.kernel.org, linux-s390@vger.kernel.org, Andy King , linux-scsi@vger.kernel.org, linux-rdma@vger.kernel.org, x86@kernel.org, Alexander Gordeev , iss_storagedev@hp.com, linux-driver@qlogic.com, linux390@de.ibm.com, Bjorn Helgaas , Dan Williams , Jon Mason , Ingo Molnar , Solarflare linux maintainers , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Ralf Baechle , e1000-devel@lists.sourceforge.net, Martin Schwidefsky , Ben Hutchings , linuxppc-dev@lists.ozlabs.org List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Mon, 2013-10-07 at 14:01 -0400, Tejun Heo wrote: > I don't think the same race condition would happen with the loop. The > problem case is where multiple msi(x) allocation fails completely > because the global limit went down before inquiry and allocation. In > the loop based interface, it'd retry with the lower number. > > As long as the number of drivers which need this sort of adaptive > allocation isn't too high and the common cases can be made simple, I > don't think the "complex" part of interface is all that important. > Maybe we can have reserve / cancel type interface or just keep the > loop with more explicit function names (ie. try_enable or something > like that). I'm thinking a better API overall might just have been to request individual MSI-X one by one :-) We want to be able to request an MSI-X at runtime anyway ... if I want to dynamically add a queue to my network interface, I want it to be able to pop a new arbitrary MSI-X. And we don't want to lock drivers into contiguous MSI-X sets either. And for the cleanup ... well that's what the "pcim" functions are for, we can just make MSI-X variants. Ben. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Benjamin Herrenschmidt Subject: Re: [PATCH RFC 00/77] Re-design MSI/MSI-X interrupts enablement pattern Date: Tue, 08 Oct 2013 07:10:56 +1100 Message-ID: <1381176656.645.171.camel@pasglop> References: <1380840585.3419.50.camel@bwh-desktop.uk.level5networks.com> <20131004082920.GA4536@dhcp-26-207.brq.redhat.com> <1380922156.3214.49.camel@bwh-desktop.uk.level5networks.com> <20131005142054.GA11270@dhcp-26-207.brq.redhat.com> <1381009586.645.141.camel@pasglop> <20131006060243.GB28142@dhcp-26-207.brq.redhat.com> <1381040386.645.143.camel@pasglop> <20131006071027.GA29143@dhcp-26-207.brq.redhat.com> <20131007180111.GC2481@htj.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: Alexander Gordeev , Ben Hutchings , linux-kernel@vger.kernel.org, Bjorn Helgaas , Ralf Baechle , Michael Ellerman , Martin Schwidefsky , Ingo Molnar , Dan Williams , Andy King , Jon Mason , Matt Porter , linux-pci@vger.kernel.org, linux-mips@linux-mips.org, linuxppc-dev@lists.ozlabs.org, linux390@de.ibm.com, linux-s390@vger.kernel.org, x86@kernel.org, linux-ide@vger.kernel.org, iss_storagedev@hp.com, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, e1000-devel@lists.sourceforge.net, To: Tejun Heo Return-path: In-Reply-To: <20131007180111.GC2481@htj.dyndns.org> Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org List-help: List-unsubscribe: List-software: Ecartis version 1.0.0 List-subscribe: List-owner: List-post: List-archive: List-Id: netdev.vger.kernel.org On Mon, 2013-10-07 at 14:01 -0400, Tejun Heo wrote: > I don't think the same race condition would happen with the loop. The > problem case is where multiple msi(x) allocation fails completely > because the global limit went down before inquiry and allocation. In > the loop based interface, it'd retry with the lower number. > > As long as the number of drivers which need this sort of adaptive > allocation isn't too high and the common cases can be made simple, I > don't think the "complex" part of interface is all that important. > Maybe we can have reserve / cancel type interface or just keep the > loop with more explicit function names (ie. try_enable or something > like that). I'm thinking a better API overall might just have been to request individual MSI-X one by one :-) We want to be able to request an MSI-X at runtime anyway ... if I want to dynamically add a queue to my network interface, I want it to be able to pop a new arbitrary MSI-X. And we don't want to lock drivers into contiguous MSI-X sets either. And for the cleanup ... well that's what the "pcim" functions are for, we can just make MSI-X variants. Ben.