Re: [PATCH] fio: increase max smalloc pools

[Jens Axboe <axboe@kernel.dk>]

On 03/04/2015 05:16 AM, Christian Ehrhardt wrote:
> On 03/03/15 23:10, Jens Axboe wrote:
>> On 03/03/2015 03:03 PM, Jens Axboe wrote:
>> > On 03/03/2015 04:44 AM, Christian Ehrhardt wrote:
>> >> From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
>> >>
>> >> For our tests with about 250k files we found the smalloc pool being
>> >> depleated.
>> >> Now for us values of 3-4 would be enough, but since it is a compile
>> >> time switch
>> >> I'd like to make it safe for everybody and set 8.
>> >>
>> >> Since it is a dynamic sizing anyway that should hopefully be ok for
>> >> everybody.
>> >
>> > The reason it was scaled down to 1 pool is because we could run into
>> > situations where one of the forked processes (or threads) would cause
>> > the expansion of pools, and smalloc() could then return memory that
>> > wasn't properly shared (or valid) between all jobs. This was recently
>> > found and fixed, and the smalloc code should probably just be
>> updated to
>> > reflect that. We can't runtime add pools safely.
>> >
>> > Right now it's 1 pool at 16MB - how about we just bump it to 64MB for
>> > that one pool? Or, alternatively, pre-add 4 pools initially when
>> smalloc
>> > is setup?
>>
>> Something like the attached, does that work for you? That's 4 pools of
>> 16MB added. I think that's more flexible than (the more ideal) 1 pool of
>> 64MB, since fio can survive if later pool additions fail. Or we can bump
>> it to 8x16 just to be on the safe side...
>>
> Hi,
> I saw you already checked it in with 8 max pools and your new code to
> initialize on sinit.
> We tested that and it works like a charm for our case now.

Yep, I ended up feeling fine about that patch. It also fixes a bug in 
the iteration of pools, where smalloc() would sometimes have failed too 
early.

-- 
Jens Axboe