All of lore.kernel.org
 help / color / mirror / Atom feed
From: maxime.coquelin-nonst@stericsson.com (Maxime Coquelin)
To: linux-arm-kernel@lists.infradead.org
Subject: [Linaro-mm-sig] [PATCHv16 0/9] Contiguous Memory Allocator
Date: Tue, 11 Oct 2011 09:30:16 +0200	[thread overview]
Message-ID: <4E93F088.60006@stericsson.com> (raw)
In-Reply-To: <00b001cc87e5$dc818cc0$9584a640$%szyprowski@samsung.com>

On 10/11/2011 09:17 AM, Marek Szyprowski wrote:
> On Monday, October 10, 2011 2:08 PM Maxime Coquelin wrote:
>
>       During our stress tests, we encountered some problems :
>
>       1) Contiguous allocation lockup:
>           When system RAM is full of Anon pages, if we try to allocate a
> contiguous buffer greater than the min_free value, we face a
> dma_alloc_from_contiguous lockup.
>           The expected result would be dma_alloc_from_contiguous() to fail.
>           The problem is reproduced systematically on our side.
> Thanks for the report. Do you use Android's lowmemorykiller? I haven't
> tested CMA on Android kernel yet. I have no idea how it will interfere
> with Android patches.
>

The software used for this test (v16) is a generic 3.0 Kernel and a 
minimal filesystem using Busybox.

With v15 patchset, I also tested it with Android.
IIRC, sometimes the lowmemorykiller succeed to get free space and the 
contiguous allocation succeed, sometimes we faced  the lockup.

>>       2) Contiguous allocation fail:
>>           We have developed a small driver and a shell script to
>> allocate/release contiguous buffers.
>>           Sometimes, dma_alloc_from_contiguous() fails to allocate the
>> contiguous buffer (about once every 30 runs).
>>           We have 270MB Memory passed to the kernel in our configuration,
>> and the CMA pool is 90MB large.
>>           In this setup, the overall memory is either free or full of
>> reclaimable pages.
> Yeah. We also did such stress tests recently and faced this issue. I've
> spent some time investigating it but I have no solution yet.
>
> The problem is caused by a page, which is put in the CMA area. This page
> is movable, but it's address space provides no 'migratepage' method. In
> such case mm subsystem uses fallback_migrate_page() function. Sadly this
> function only returns -EAGAIN. The migration loops a few times over it
> and fails causing the fail in the allocation procedure.
>
> We are investing now which kernel code created/allocated such problematic
> pages and how to add real migration support for them.
>

Ok, thanks for pointing this out.

Regards,
Maxime

WARNING: multiple messages have this Message-ID (diff)
From: Maxime Coquelin <maxime.coquelin-nonst@stericsson.com>
To: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>,
	"linux-media@vger.kernel.org" <linux-media@vger.kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	"linaro-mm-sig@lists.linaro.org" <linaro-mm-sig@lists.linaro.org>,
	'Daniel Walker' <dwalker@codeaurora.org>,
	'Russell King' <linux@arm.linux.org.uk>,
	'Arnd Bergmann' <arnd@arndb.de>,
	'Jonathan Corbet' <corbet@lwn.net>, 'Mel Gorman' <mel@csn.ul.ie>,
	'Chunsang Jeong' <chunsang.jeong@linaro.org>,
	'Michal Nazarewicz' <mina86@mina86.com>,
	'Dave Hansen' <dave@linux.vnet.ibm.com>,
	'Jesse Barker' <jesse.barker@linaro.org>,
	'Kyungmin Park' <kyungmin.park@samsung.com>,
	'Ankita Garg' <ankita@in.ibm.com>,
	'Andrew Morton' <akpm@linux-foundation.org>,
	'KAMEZAWA Hiroyuki' <kamezawa.hiroyu@jp.fujitsu.com>,
	"benjamin.gaignard@linaro.org" <benjamin.gaignard@linaro.org>,
	Ludovic BARRE <ludovic.barre@stericsson.com>,
	"vincent.guittot@linaro.org" <vincent.guittot@linaro.org>
Subject: Re: [Linaro-mm-sig] [PATCHv16 0/9] Contiguous Memory Allocator
Date: Tue, 11 Oct 2011 09:30:16 +0200	[thread overview]
Message-ID: <4E93F088.60006@stericsson.com> (raw)
In-Reply-To: <00b001cc87e5$dc818cc0$9584a640$%szyprowski@samsung.com>

On 10/11/2011 09:17 AM, Marek Szyprowski wrote:
> On Monday, October 10, 2011 2:08 PM Maxime Coquelin wrote:
>
>       During our stress tests, we encountered some problems :
>
>       1) Contiguous allocation lockup:
>           When system RAM is full of Anon pages, if we try to allocate a
> contiguous buffer greater than the min_free value, we face a
> dma_alloc_from_contiguous lockup.
>           The expected result would be dma_alloc_from_contiguous() to fail.
>           The problem is reproduced systematically on our side.
> Thanks for the report. Do you use Android's lowmemorykiller? I haven't
> tested CMA on Android kernel yet. I have no idea how it will interfere
> with Android patches.
>

The software used for this test (v16) is a generic 3.0 Kernel and a 
minimal filesystem using Busybox.

With v15 patchset, I also tested it with Android.
IIRC, sometimes the lowmemorykiller succeed to get free space and the 
contiguous allocation succeed, sometimes we faced  the lockup.

>>       2) Contiguous allocation fail:
>>           We have developed a small driver and a shell script to
>> allocate/release contiguous buffers.
>>           Sometimes, dma_alloc_from_contiguous() fails to allocate the
>> contiguous buffer (about once every 30 runs).
>>           We have 270MB Memory passed to the kernel in our configuration,
>> and the CMA pool is 90MB large.
>>           In this setup, the overall memory is either free or full of
>> reclaimable pages.
> Yeah. We also did such stress tests recently and faced this issue. I've
> spent some time investigating it but I have no solution yet.
>
> The problem is caused by a page, which is put in the CMA area. This page
> is movable, but it's address space provides no 'migratepage' method. In
> such case mm subsystem uses fallback_migrate_page() function. Sadly this
> function only returns -EAGAIN. The migration loops a few times over it
> and fails causing the fail in the allocation procedure.
>
> We are investing now which kernel code created/allocated such problematic
> pages and how to add real migration support for them.
>

Ok, thanks for pointing this out.

Regards,
Maxime


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2011-10-11  7:30 UTC|newest]

Thread overview: 179+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-10-06 13:54 [PATCHv16 0/9] Contiguous Memory Allocator Marek Szyprowski
2011-10-06 13:54 ` Marek Szyprowski
2011-10-06 13:54 ` Marek Szyprowski
2011-10-06 13:54 ` [PATCH 1/9] mm: move some functions from memory_hotplug.c to page_isolation.c Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-14 23:23   ` Andrew Morton
2011-10-14 23:23     ` Andrew Morton
2011-10-14 23:23     ` Andrew Morton
2011-10-18 12:05   ` Mel Gorman
2011-10-18 12:05     ` Mel Gorman
2011-10-18 12:05     ` Mel Gorman
2011-10-06 13:54 ` [PATCH 2/9] mm: alloc_contig_freed_pages() added Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-14 23:29   ` Andrew Morton
2011-10-14 23:29     ` Andrew Morton
2011-10-14 23:29     ` Andrew Morton
2011-10-16  8:01     ` Michal Nazarewicz
2011-10-16  8:01       ` Michal Nazarewicz
2011-10-16  8:01       ` Michal Nazarewicz
2011-10-16  8:31       ` Andrew Morton
2011-10-16  8:31         ` Andrew Morton
2011-10-16  8:31         ` Andrew Morton
2011-10-16  9:39         ` Michal Nazarewicz
2011-10-16  9:39           ` Michal Nazarewicz
2011-10-16  9:39           ` Michal Nazarewicz
2011-10-17 12:21     ` Marek Szyprowski
2011-10-17 12:21       ` Marek Szyprowski
2011-10-17 12:21       ` Marek Szyprowski
2011-10-17 18:39       ` Andrew Morton
2011-10-17 18:39         ` Andrew Morton
2011-10-17 18:39         ` Andrew Morton
2011-10-18 12:21   ` Mel Gorman
2011-10-18 12:21     ` Mel Gorman
2011-10-18 12:21     ` Mel Gorman
2011-10-18 17:26     ` Michal Nazarewicz
2011-10-18 17:26       ` Michal Nazarewicz
2011-10-18 17:26       ` Michal Nazarewicz
2011-10-18 17:48       ` Dave Hansen
2011-10-18 17:48         ` Dave Hansen
2011-10-18 17:48         ` Dave Hansen
2011-10-18 18:00         ` Michal Nazarewicz
2011-10-18 18:00           ` Michal Nazarewicz
2011-10-18 18:00           ` Michal Nazarewicz
2011-10-21 10:06       ` Mel Gorman
2011-10-21 10:06         ` Mel Gorman
2011-10-21 10:06         ` Mel Gorman
2011-10-24  1:00         ` Michal Nazarewicz
2011-10-24  1:00           ` Michal Nazarewicz
2011-10-24  1:00           ` Michal Nazarewicz
2011-10-24  4:05     ` Michal Nazarewicz
2011-10-24  4:05     ` Michal Nazarewicz
2011-10-24  4:05       ` Michal Nazarewicz
2011-10-24  4:05       ` Michal Nazarewicz
2011-11-01 15:04       ` Mel Gorman
2011-11-01 15:04         ` Mel Gorman
2011-11-01 15:04         ` Mel Gorman
2011-11-01 18:06         ` Michal Nazarewicz
2011-11-01 18:06           ` Michal Nazarewicz
2011-11-01 18:06           ` Michal Nazarewicz
2011-11-01 18:47           ` Mel Gorman
2011-11-01 18:47             ` Mel Gorman
2011-11-01 18:47             ` Mel Gorman
2011-10-24  4:05     ` Michal Nazarewicz
2011-10-24  4:05     ` Michal Nazarewicz
2011-10-06 13:54 ` [PATCH 3/9] mm: alloc_contig_range() added Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-14 23:35   ` Andrew Morton
2011-10-14 23:35     ` Andrew Morton
2011-10-14 23:35     ` Andrew Morton
2011-10-18 12:38   ` Mel Gorman
2011-10-18 12:38     ` Mel Gorman
2011-10-18 12:38     ` Mel Gorman
2011-10-06 13:54 ` [PATCH 4/9] mm: MIGRATE_CMA migration type added Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-14 23:38   ` Andrew Morton
2011-10-14 23:38     ` Andrew Morton
2011-10-14 23:38     ` Andrew Morton
2011-10-18 13:08   ` Mel Gorman
2011-10-18 13:08     ` Mel Gorman
2011-10-18 13:08     ` Mel Gorman
2011-10-24 19:32     ` Michal Nazarewicz
2011-10-24 19:32       ` Michal Nazarewicz
2011-10-24 19:32       ` Michal Nazarewicz
2011-10-27  9:10       ` Michal Nazarewicz
2011-10-27  9:10         ` Michal Nazarewicz
2011-10-27  9:10         ` Michal Nazarewicz
2011-10-06 13:54 ` [PATCH 5/9] mm: MIGRATE_CMA isolation functions added Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 13:54 ` [PATCH 6/9] drivers: add Contiguous Memory Allocator Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-14 23:57   ` Andrew Morton
2011-10-14 23:57     ` Andrew Morton
2011-10-14 23:57     ` Andrew Morton
2011-10-16 10:08     ` Russell King - ARM Linux
2011-10-16 10:08       ` Russell King - ARM Linux
2011-10-16 10:08       ` Russell King - ARM Linux
2011-10-18 13:43   ` Mel Gorman
2011-10-18 13:43     ` Mel Gorman
2011-10-18 13:43     ` Mel Gorman
2011-10-24 19:39     ` Michal Nazarewicz
2011-10-24 19:39       ` Michal Nazarewicz
2011-10-24 19:39       ` Michal Nazarewicz
2011-11-04 10:41     ` Marek Szyprowski
2011-11-04 10:41       ` Marek Szyprowski
2011-11-04 10:41       ` Marek Szyprowski
2011-10-06 13:54 ` [PATCH 7/7] ARM: integrate CMA with DMA-mapping subsystem Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 14:18   ` Marek Szyprowski
2011-10-06 14:18     ` Marek Szyprowski
2011-10-06 14:18     ` Marek Szyprowski
2011-10-15  0:03   ` Andrew Morton
2011-10-15  0:03     ` Andrew Morton
2011-10-15  0:03     ` Andrew Morton
2011-10-06 13:54 ` [PATCH 7/9] X86: " Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 13:54 ` [PATCH 8/9] ARM: " Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-14  4:33   ` [Linaro-mm-sig] " Subash Patel
2011-10-14  4:33     ` Subash Patel
2011-10-14  4:33     ` Subash Patel
2011-10-14  9:14     ` Marek Szyprowski
2011-10-14  9:14       ` Marek Szyprowski
2011-10-14  9:14       ` Marek Szyprowski
2011-10-06 13:54 ` [PATCH 9/9] ARM: Samsung: use CMA for 2 memory banks for s5p-mfc device Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-06 13:54   ` Marek Szyprowski
2011-10-07 16:27 ` [PATCHv16 0/9] Contiguous Memory Allocator Arnd Bergmann
2011-10-07 16:27   ` Arnd Bergmann
2011-10-07 16:27   ` Arnd Bergmann
2011-10-10  6:58   ` [Linaro-mm-sig] " Ohad Ben-Cohen
2011-10-10  6:58     ` Ohad Ben-Cohen
2011-10-10  6:58     ` Ohad Ben-Cohen
2011-10-10 12:02     ` Clark, Rob
2011-10-10 12:02       ` Clark, Rob
2011-10-10 12:02       ` Clark, Rob
2011-10-10 22:56   ` Andrew Morton
2011-10-10 22:56     ` Andrew Morton
2011-10-10 22:56     ` Andrew Morton
2011-10-11  6:57     ` Marek Szyprowski
2011-10-11  6:57       ` Marek Szyprowski
2011-10-11  6:57       ` Marek Szyprowski
2011-10-11 13:52     ` Arnd Bergmann
2011-10-11 13:52       ` Arnd Bergmann
2011-10-11 13:52       ` Arnd Bergmann
2011-10-14 23:19       ` Andrew Morton
2011-10-14 23:19         ` Andrew Morton
2011-10-14 23:19         ` Andrew Morton
2011-10-15 14:24         ` Arnd Bergmann
2011-10-15 14:24           ` Arnd Bergmann
2011-10-15 14:24           ` Arnd Bergmann
2011-10-10 12:07 ` [Linaro-mm-sig] " Maxime Coquelin
2011-10-10 12:07   ` Maxime Coquelin
2011-10-10 12:07   ` Maxime Coquelin
2011-10-11  7:17   ` Marek Szyprowski
2011-10-11  7:17     ` Marek Szyprowski
2011-10-11  7:17     ` Marek Szyprowski
2011-10-11  7:30     ` Maxime Coquelin [this message]
2011-10-11  7:30       ` Maxime Coquelin
2011-10-11 10:50       ` Marek Szyprowski
2011-10-11 10:50         ` Marek Szyprowski
2011-10-11 10:50         ` Marek Szyprowski
2011-10-11 11:25         ` Maxime Coquelin
2011-10-11 11:25           ` Maxime Coquelin
2011-10-11 13:05           ` Marek Szyprowski
2011-10-11 13:05             ` Marek Szyprowski
2011-10-11 13:05             ` Marek Szyprowski
2011-10-12 11:08       ` [PATCH] fixup: mm: alloc_contig_range: increase min_free_kbytes during allocation Marek Szyprowski
2011-10-12 11:08         ` Marek Szyprowski
2011-10-12 13:01         ` Maxime Coquelin
2011-10-12 13:01           ` Maxime Coquelin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4E93F088.60006@stericsson.com \
    --to=maxime.coquelin-nonst@stericsson.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.