Re: [PATCH] slub: reduce total stack usage of slab_err & object_err

Richard Kennedy
2008-10-01T09:50:30Z

On Tue, 2008-09-30 at 13:33 -0500, Matt Mackall wrote:
> On Tue, 2008-09-30 at 12:37 -0500, Matt Mackall wrote:
> > On Tue, 2008-09-30 at 17:20 +0100, Richard Kennedy wrote:
> > > Yes, using vprintk is better but you still have this path :
> > > ( with your patch applied)
> > >
> > > object_err -> slab_bug(208) -> printk(216)
> > > instead of
> > > object_err -> slab_bug_message(8) -> printk(216)
> > >
> > > unfortunately the overhead for having var_args is pretty big, at least
> > > on x86_64. I haven't measured it on 32 bit yet.
> >
> > That's fascinating. I tried a simple test case in userspace:
> >
> > #include <stdarg.h>
> > #include <stdio.h>
> >
> > void p(char *fmt, ...)
> > {
> > va_list args;
> >
> > va_start(args, fmt);
> > vprintf(fmt, args);
> > va_end(args);
> > }
> >
> > On 32-bit, I'm seeing 32 bytes of stack vs 216 on 64-bit. Disassembly
> > suggests it's connected to va_list fiddling with XMM registers, which
> > seems quite odd.
>
> Ok, on closer inspection, this is part of the x86_64 calling convention.
> When calling a varargs function, the caller passes the number of
> floating point SSE regs used in rax. The callee then has to save these
> away for va_list use. The GCC prologue apparently sets aside space for
> xmm0-xmm7 (16 bytes each) all the time (plus rdi, rsi, rdx, rcx, r8, and
> r9).
>
> Obviously, we're never passing floating point args in the kernel, so
> we're taking about a 40+ byte hit in code size and 128 byte hit in stack
> size for every varargs call.
>
> Looks like the gcc people have a patch in progress:
>
> http://gcc.gnu.org/ml/gcc-patches/2008-08/msg02165.html
>
> So I think we should assume that x86_64 will sort this out eventually.
>
thanks for the info.

so vprintk _is_ the best solution. I did think it a bit strange to have
to add code to remove varargs to save stack space, but I didn't even
consider that it might be a gcc issue.