Re: Suspicious error for CMA stress test

Laura Abbott
2016-03-03T18:52:23Z

On 03/03/2016 04:49 AM, Hanjun Guo wrote:
> On 2016/3/3 15:42, Joonsoo Kim wrote:
>> 2016-03-03 10:25 GMT+09:00 Laura Abbott <labbott@redhat.com>:
>>> (cc -mm and Joonsoo Kim)
>>>
>>>
>>> On 03/02/2016 05:52 AM, Hanjun Guo wrote:
>>>> Hi,
>>>>
>>>> I came across a suspicious error for CMA stress test:
>>>>
>>>> Before the test, I got:
>>>> -bash-4.3# cat /proc/meminfo | grep Cma
>>>> CmaTotal: 204800 kB
>>>> CmaFree: 195044 kB
>>>>
>>>>
>>>> After running the test:
>>>> -bash-4.3# cat /proc/meminfo | grep Cma
>>>> CmaTotal: 204800 kB
>>>> CmaFree: 6602584 kB
>>>>
>>>> So the freed CMA memory is more than total..
>>>>
>>>> Also the the MemFree is more than mem total:
>>>>
>>>> -bash-4.3# cat /proc/meminfo
>>>> MemTotal: 16342016 kB
>>>> MemFree: 22367268 kB
>>>> MemAvailable: 22370528 kB
> [...]
>>>
>>> I played with this a bit and can see the same problem. The sanity
>>> check of CmaFree < CmaTotal generally triggers in
>>> __move_zone_freepage_state in unset_migratetype_isolate.
>>> This also seems to be present as far back as v4.0 which was the
>>> first version to have the updated accounting from Joonsoo.
>>> Were there known limitations with the new freepage accounting,
>>> Joonsoo?
>> I don't know. I also played with this and looks like there is
>> accounting problem, however, for my case, number of free page is slightly less
>> than total. I will take a look.
>>
>> Hanjun, could you tell me your malloc_size? I tested with 1 and it doesn't
>> look like your case.
>
> I tested with malloc_size with 2M, and it grows much bigger than 1M, also I
> did some other test:
>
> - run with single thread with 100000 times, everything is fine.
>
> - I hack the cam_alloc() and free as below [1] to see if it's lock issue, with
> the same test with 100 multi-thread, then I got:
>
> -bash-4.3# cat /proc/meminfo | grep Cma
> CmaTotal: 204800 kB
> CmaFree: 225112 kB
>
> It only increased about 30M for free, not 6G+ in previous test, although
> the problem is not solved, the problem is less serious, is it a synchronization
> problem?
>

'only' 30M is still an issue although I think you are right about something related
to synchronization. When I put the cma_mutex around free_contig_range I don't see
the issue. I wonder if free of the pages is racing with the undo_isolate_page_range
on overlapping ranges caused by outer_start?

Thanks,
Laura

> Thanks
> Hanjun
>
> [1]:
> index ea506eb..4447494 100644
> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -379,6 +379,7 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align)
> if (!count)
> return NULL;
>
> + mutex_lock(&cma_mutex);
> mask = cma_bitmap_aligned_mask(cma, align);
> offset = cma_bitmap_aligned_offset(cma, align);
> bitmap_maxno = cma_bitmap_maxno(cma);
> @@ -402,17 +403,16 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align)
> mutex_unlock(&cma->lock);
>
> pfn = cma->base_pfn + (bitmap_no << cma->order_per_bit);
> - mutex_lock(&cma_mutex);
> ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> - mutex_unlock(&cma_mutex);
> if (ret == 0) {
> page = pfn_to_page(pfn);
> break;
> }
>
> cma_clear_bitmap(cma, pfn, count);
> - if (ret != -EBUSY)
> + if (ret != -EBUSY) {
> break;
> + }
>
> pr_debug("%s(): memory range at %p is busy, retrying\n",
> __func__, pfn_to_page(pfn));
> @@ -420,6 +420,7 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align)
> start = bitmap_no + mask + 1;
> }
>
> + mutex_unlock(&cma_mutex);
> trace_cma_alloc(pfn, page, count, align);
>
> pr_debug("%s(): returned %p\n", __func__, page);
> @@ -445,15 +446,19 @@ bool cma_release(struct cma *cma, const struct page *pages, unsigned int count)
>
> pr_debug("%s(page %p)\n", __func__, (void *)pages);
>
> + mutex_lock(&cma_mutex);
> pfn = page_to_pfn(pages);
>
> - if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count)
> + if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count) {
> + mutex_unlock(&cma_mutex);
> return false;
> + }
>
> VM_BUG_ON(pfn + count > cma->base_pfn + cma->count);
>
> free_contig_range(pfn, count);
> cma_clear_bitmap(cma, pfn, count);
> + mutex_unlock(&cma_mutex);
> trace_cma_release(pfn, pages, count);
>
> return true;
>