Re: fio memory allocation problems

[Jiri Horky <jiri.horky@cesnet.cz>]

[-- Attachment #1: Type: text/plain, Size: 3383 bytes --]

Hi Jens,

the patch is attached. It resolves the issue but I would not bet it 
works in all scenarios.

Cheers
Jiri
------------

--- fio-head/smalloc.c  2011-08-15 10:40:24.000000000 +0200
+++ fio-head-patched/smalloc.c  2011-08-15 10:49:32.000000000 +0200
@@ -168,11 +168,11 @@
         blocks_iter(pool, pool_idx, idx, nr_blocks, mask_clear);
  }

-static int find_next_zero(int word, int start)
+static int find_zero(int word, int start)
  {
         assert(word != -1U);
-       word >>= (start + 1);
-       return ffz(word) + start + 1;
+       word >>= (start);
+       return ffz(word) + start;
  }

  static int add_pool(struct pool *pool, unsigned int alloc_size)
@@ -374,7 +374,7 @@
                         continue;
                 }

-               idx = find_next_zero(pool->bitmap[i], last_idx);
+               idx = find_zero(pool->bitmap[i], last_idx);
                 if (!blocks_free(pool, i, idx, nr_blocks)) {
                         idx += nr_blocks;
                         if (idx < SMALLOC_BPI)


On 08/15/2011 10:29 AM, Jens Axboe wrote:
> On 2011-08-12 16:42, Jiri Horky wrote:
>> Hi,
>>
>> I think I found a bug. If the device (file) used in fio has "wrong" size, and is used with "wrong" block size, the fio fails to allocate memory for randommap.
>> Following conditions must be true, considering just one thread for simplicity:
>>
>>   1) number of blocks on the device are too big to fill in the default's pool
>>   2) number of blocks on the device must be "right", so the size of newly allocated pool by smalloc.c:add_pool is exactly as big as requested size (there are some roundings in place, so not every size triggers this error)
>>
>> When this is true, function smalloc.c:__smalloc_pool fails to use the newly created (exactly big enough) pool, because of this line:
>>
>>                  idx = find_next_zero(pool->bitmap[i], last_idx);
>>
>> which always returns non-zero value (the minimum returned value is last_idx +1), and so in our case the idx is 1 even though, the block number 0 is free. As the direct consequence, the function doesn't find enough free space in the given pool and returns NULL pointer. This leads to another try to allocate enough memory in smalloc.c:smalloc_real function, and so on...
>>
>> The behavior could be reprocuded using both stable 1.57 and git version of the tool. The values that triggers the error are:
>> init_random_map - real_file_size: 21999995584512, o.rw_min_bs: 4096, blocks: 5371092672, num_maps: 83923323, alloc size: 671386584
>>
>> whereas this block device is fine:
>> init_random_map - real_file_size: 19999995985920, o.rw_min_bs: 4096, blocks: 4882811520, num_maps: 76293930, alloc size: 610351440
>>
>> the configuration file used:
>>
>> [global]
>> description=CESNET_test
>> [cesnet]
>> filename=/dev/sdd
>> rw=randread
>> size=10000g
>> ioengine=libaio
>> iodepth=4
>> bs=4k
>> runtime=10m
>> time_based
>> numjobs=1
>> #norandommap
>> group_reporting
>>
>>
>> I am not sure I fully understand all the logic in smalloc.c so I am afraid I would break something with a trivial patch.
>>
>> Let me know, if you need more information
> That does indeed look like a bug in smalloc, bummer. I'll try and take a
> look at it, if you do have a test patch you are using, please send it
> along.
>


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 6917 bytes --]