SLAB vs. SLUB tbench regression (Was: [tbench regression fixes]: digging out smelly deadmen.)

SLAB vs. SLUB tbench regression (Was: [tbench regression fixes]: digging out smelly deadmen.)

[Pekka Enberg]

Hi Evgeniy,

On Fri, Oct 10, 2008 at 1:17 AM, Evgeniy Polyakov
<s0mbre@tservice.net.ru> wrote:
> It was reported recently that tbench has a long history of regressions,
> starting at least from 2.6.23 kernel. I verified that in my test
> environment tbench 'lost' more than 100 MB/s from 470 down to 355
> between at least 2.6.24 and 2.6.27. 2.6.26-2.6.27 performance regression
> in my machines is rougly corresponds to 375 down to 355 MB/s.

As you've already pointed out in private and in your blog, some part
of the tbench regression comes from SLUB vs. SLAB as well. Looks like
I can reproduce the regression locally as well:

  [ 8 clients and tbench_srv on the same machine on 2-way x86-64 ]

                  min      max      avg      sd
  2.6.28-rc2-slab 234.57   244.88   242.68   0.71
  2.6.28-rc2-slub 227.44   240.90   239.08   0.78

Oprofile seems to be busted for 2.6.28-rc2 so I'll follow up on this
as soon as that's settled.

Btw, just as one more data point, I accidentally tested with just 2
clients at first and SLUB actually beat SLAB.

                        Pekka

Re: SLAB vs. SLUB tbench regression (Was: [tbench regression fixes]: digging out smelly deadmen.)

[Pekka Enberg]

On Fri, Oct 31, 2008 at 4:28 PM, Pekka Enberg <penberg@cs.helsinki.fi> wrote:
> Hi Evgeniy,
>
> On Fri, Oct 10, 2008 at 1:17 AM, Evgeniy Polyakov
> <s0mbre@tservice.net.ru> wrote:
>> It was reported recently that tbench has a long history of regressions,
>> starting at least from 2.6.23 kernel. I verified that in my test
>> environment tbench 'lost' more than 100 MB/s from 470 down to 355
>> between at least 2.6.24 and 2.6.27. 2.6.26-2.6.27 performance regression
>> in my machines is rougly corresponds to 375 down to 355 MB/s.
>
> As you've already pointed out in private and in your blog, some part
> of the tbench regression comes from SLUB vs. SLAB as well. Looks like
> I can reproduce the regression locally as well:
>
>  [ 8 clients and tbench_srv on the same machine on 2-way x86-64 ]
>
>                  min      max      avg      sd
>  2.6.28-rc2-slab 234.57   244.88   242.68   0.71
>  2.6.28-rc2-slub 227.44   240.90   239.08   0.78

Christoph asked me to throw in the prefetch patch from Eric Dumazet.
Unfortunately, it doesn't seem to make much of a difference:

[ 2-way x86-64 with 8 clients and tbench_srv on the same machine ]

                           min      max      avg      sd
  2.6.28-rc2-slab          234.57   244.88   242.68   0.71
  2.6.28-rc2-slub          227.44   240.90   239.08   0.78
  2.6.28-rc2-slub+prefetch 237.42   244.32   239.78   0.92

Re: SLAB vs. SLUB tbench regression (Was: [tbench regression fixes]: digging out smelly deadmen.)

[Evgeniy Polyakov]

Hi Pekka.

On Fri, Oct 31, 2008 at 04:28:57PM +0200, Pekka Enberg (penberg@cs.helsinki.fi) wrote:
> As you've already pointed out in private and in your blog, some part
> of the tbench regression comes from SLUB vs. SLAB as well. Looks like
> I can reproduce the regression locally as well:
> 
>   [ 8 clients and tbench_srv on the same machine on 2-way x86-64 ]
> 
>                   min      max      avg      sd
>   2.6.28-rc2-slab 234.57   244.88   242.68   0.71
>   2.6.28-rc2-slub 227.44   240.90   239.08   0.78
> 
> Oprofile seems to be busted for 2.6.28-rc2 so I'll follow up on this
> as soon as that's settled.
> 
> Btw, just as one more data point, I accidentally tested with just 2
> clients at first and SLUB actually beat SLAB.

Looks like under the allocation/freeing pressure some tricky part of
slub starts slowing things down, while at usual rate amount of operations
is smaller, so slub gets its optimized results... Just a handwaving
though.

Tbench regression is not related to slab/slub fight though, since I
always used slab in tests (if not specially tested slub), but of course
fixing this issue in the more and more common allocator would be just
great. Thanks for working on this issue Pekka.

-- 
	Evgeniy Polyakov