Re: [patch 00/41] cpu alloc / cpu ops v3: Optimize per cpu access

Andrew Morton
2008-05-30T05:50:28Z

On Thu, 29 May 2008 22:27:53 -0700 (PDT) Christoph Lameter <clameter@sgi.com> wrote:

> On Thu, 29 May 2008, Andrew Morton wrote:
>
> > > The per cpu memory use by subsystems is typically quite small. We already
> > > have an 8k limitation for percpu space for modules. And that does not seem
> > > to be a problem.
> >
> > eh? That's DEFINE_PERCPU memory, not alloc_pecpu() memory?
>
> No. The module subsystem has its own alloc_percpu subsystem that the
> cpu_alloc replaces.

That is to support DEFINE_PER_CPU, not alloc_percpu().

> > > We could do that yes.
> >
> > Phew.
>
> But its going to be even more complicated and I have a hard time managing
> the complexity here. Could someone take pieces off my hand?

It could be done later on.

> > > But then per cpu data is not frequently allocated and freed.
> >
> > I think it is, in the TCP case. And that's the only one I looked at.
>
> Which tcp case?

The one you just deleted from my reply :(

> > Plus who knows what lies ahead of us?
>
> Well invariably we will end up with cpu area defragmentation.... Sigh.
>
> > I don't think there is presently any upper limit on alloc_percpu()? It
> > uses kmalloc() and kmalloc_node()?
> >
> > Even if there is some limit, is it an unfixable one?
>
> No there is no limit. It just wastes lots of space (pointer arrays,
> alignment etc) that we could use to configure sufficiently large per cpu
> areas.

Christoph, please. An allocator which is of fixed size and which is
vulnerable to internal fragmentation is a huge problem! The kernel is
subject to wildly varying workloads both between different users and in
the hands of a single user.

If we were to merge all this code and then run into the problems which
I fear then we are tremendously screwed. We must examine this
exhaustively, in the most paranoid fashion.