From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Morton Subject: Re: [patch 00/41] cpu alloc / cpu ops v3: Optimize per cpu access Date: Thu, 29 May 2008 22:49:59 -0700 Message-ID: <20080529224959.3297a4f1.akpm@linux-foundation.org> References: <20080530035620.587204923@sgi.com> <20080529215827.b659d032.akpm@linux-foundation.org> <20080529222143.5d7aa1e5.akpm@linux-foundation.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Return-path: Received: from smtp1.linux-foundation.org ([140.211.169.13]:54027 "EHLO smtp1.linux-foundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751160AbYE3FuP (ORCPT ); Fri, 30 May 2008 01:50:15 -0400 In-Reply-To: Sender: linux-arch-owner@vger.kernel.org List-ID: To: Christoph Lameter Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, David Miller , Eric Dumazet , Peter Zijlstra , Rusty Russell , Mike Travis On Thu, 29 May 2008 22:27:53 -0700 (PDT) Christoph Lameter wrote: > On Thu, 29 May 2008, Andrew Morton wrote: > > > > The per cpu memory use by subsystems is typically quite small. We already > > > have an 8k limitation for percpu space for modules. And that does not seem > > > to be a problem. > > > > eh? That's DEFINE_PERCPU memory, not alloc_pecpu() memory? > > No. The module subsystem has its own alloc_percpu subsystem that the > cpu_alloc replaces. That is to support DEFINE_PER_CPU, not alloc_percpu(). > > > We could do that yes. > > > > Phew. > > But its going to be even more complicated and I have a hard time managing > the complexity here. Could someone take pieces off my hand? It could be done later on. > > > But then per cpu data is not frequently allocated and freed. > > > > I think it is, in the TCP case. And that's the only one I looked at. > > Which tcp case? The one you just deleted from my reply :( > > Plus who knows what lies ahead of us? > > Well invariably we will end up with cpu area defragmentation.... Sigh. > > > I don't think there is presently any upper limit on alloc_percpu()? It > > uses kmalloc() and kmalloc_node()? > > > > Even if there is some limit, is it an unfixable one? > > No there is no limit. It just wastes lots of space (pointer arrays, > alignment etc) that we could use to configure sufficiently large per cpu > areas. Christoph, please. An allocator which is of fixed size and which is vulnerable to internal fragmentation is a huge problem! The kernel is subject to wildly varying workloads both between different users and in the hands of a single user. If we were to merge all this code and then run into the problems which I fear then we are tremendously screwed. We must examine this exhaustively, in the most paranoid fashion.