From mboxrd@z Thu Jan 1 00:00:00 1970 From: ebiederm@xmission.com (Eric W. Biederman) Subject: Re: [PATCH 8/25] msi: Simplify the msi irq limit policy. Date: Tue, 20 Jun 2006 21:48:41 -0600 Message-ID: References: <11508425183073-git-send-email-ebiederm@xmission.com> <11508425191381-git-send-email-ebiederm@xmission.com> <11508425192220-git-send-email-ebiederm@xmission.com> <11508425191063-git-send-email-ebiederm@xmission.com> <1150842520235-git-send-email-ebiederm@xmission.com> <11508425201406-git-send-email-ebiederm@xmission.com> <1150842520775-git-send-email-ebiederm@xmission.com> <11508425213394-git-send-email-ebiederm@xmission.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from ebiederm.dsl.xmission.com ([166.70.28.69]:46751 "EHLO ebiederm.dsl.xmission.com") by vger.kernel.org with ESMTP id S1750754AbWFUDtr (ORCPT ); Tue, 20 Jun 2006 23:49:47 -0400 In-Reply-To: (Roland Dreier's message of "Tue, 20 Jun 2006 19:46:22 -0700") Sender: linux-acpi-owner@vger.kernel.org List-Id: linux-acpi@vger.kernel.org To: Roland Dreier Cc: Andrew Morton , linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, linux-pci@atrey.karlin.mff.cuni.cz, discuss@x86-64.org, Ingo Molnar , Thomas Gleixner , Andi Kleen , Natalie Protasevich , Len Brown , Kimball Murray , Brice Goglin , Greg Lindahl , Dave Olson , Jeff Garzik , Greg KH , Grant Grundler , "bibo,mao" , Rajesh Shah , Jesper Juhl , Shaohua Li , Matthew Wilcox , "Michael S. Tsirkin" , Ashok Raj , Randy Dunlap , Tony Luck Roland Dreier writes: > > Only the s2io driver even takes advantage of this feature > > all other drivers have a fixed number of irqs they need and > > bail if they can't get them. > > My todo list for the mthca (InfiniBand HCA) driver includes adding > support for more event queues. When I do that, I'll likely want to > try to get something on the order of number_of_cpus plus two or three > MSI-X vectors, and fall back to a lower number of vectors if that > allocation fails. Allowing drivers that can take advantage of large numbers of irqs is part of this patch. To be clear the method still supports allocate a bunch of irqs and then falling back. We have to kinds of drivers. Those that allocate a lot of irqs and allocate fewer if they can't get them, and those just allocate a couple and fail if they can't get them. The policy I deleted tries to be fair, and give all drivers the same number of irqs. What I left us with is a simple first come first serve policy. To do better you need an accurate count and you need to separate out the various kinds of drivers, and you need a shortage of irqs. Currently x86_64 hardware supports up to 244*NR_CPUS irqs. I don't expect we will exceed that limit any time soon even with a first come first serve policy. The worst case I can think of with your proposed irq allocation policy in the mthca driver is a 128 cpu machine with 128 IB cards in it. That would just barely fail with every driver allocating 3 IRQs per cpu, and it could be trivially fixed by putting in dual core cpus :) So when we exceed the limit on the number of IRQs we actually have then it probably makes sense to see if a policy more aggressive than first come first serve makes sense. But until then it is a waste of time and we should be concentrating our efforts on making more IRQs usable. Eric