* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-05-28 7:04 ` [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used Joonsoo Kim
@ 2014-05-29 7:24 ` Gioh Kim
2014-05-29 7:48 ` Joonsoo Kim
2014-05-30 7:53 ` Gioh Kim
` (3 subsequent siblings)
4 siblings, 1 reply; 24+ messages in thread
From: Gioh Kim @ 2014-05-29 7:24 UTC (permalink / raw)
To: Joonsoo Kim, Andrew Morton
Cc: Rik van Riel, Johannes Weiner, Mel Gorman, Laura Abbott,
Minchan Kim, Heesub Shin, Marek Szyprowski, Michal Nazarewicz,
Aneesh Kumar K.V, linux-mm, linux-kernel
I've not understand your code fully. Please let me ask some silly questions.
2014-05-28 ?AEA 4:04, Joonsoo Kim 3/4 ' +-U:
> CMA is introduced to provide physically contiguous pages at runtime.
> For this purpose, it reserves memory at boot time. Although it reserve
> memory, this reserved memory can be used for movable memory allocation
> request. This usecase is beneficial to the system that needs this CMA
> reserved memory infrequently and it is one of main purpose of
> introducing CMA.
>
> But, there is a problem in current implementation. The problem is that
> it works like as just reserved memory approach. The pages on cma reserved
> memory are hardly used for movable memory allocation. This is caused by
> combination of allocation and reclaim policy.
>
> The pages on cma reserved memory are allocated if there is no movable
> memory, that is, as fallback allocation. So the time this fallback
> allocation is started is under heavy memory pressure. Although it is under
> memory pressure, movable allocation easily succeed, since there would be
> many pages on cma reserved memory. But this is not the case for unmovable
> and reclaimable allocation, because they can't use the pages on cma
> reserved memory. These allocations regard system's free memory as
> (free pages - free cma pages) on watermark checking, that is, free
> unmovable pages + free reclaimable pages + free movable pages. Because
> we already exhausted movable pages, only free pages we have are unmovable
> and reclaimable types and this would be really small amount. So watermark
> checking would be failed. It will wake up kswapd to make enough free
> memory for unmovable and reclaimable allocation and kswapd will do.
> So before we fully utilize pages on cma reserved memory, kswapd start to
> reclaim memory and try to make free memory over the high watermark. This
> watermark checking by kswapd doesn't take care free cma pages so many
> movable pages would be reclaimed. After then, we have a lot of movable
> pages again, so fallback allocation doesn't happen again. To conclude,
> amount of free memory on meminfo which includes free CMA pages is moving
> around 512 MB if I reserve 512 MB memory for CMA.
>
> I found this problem on following experiment.
>
> 4 CPUs, 1024 MB, VIRTUAL MACHINE
> make -j16
>
> CMA reserve: 0 MB 512 MB
> Elapsed-time: 225.2 472.5
> Average-MemFree: 322490 KB 630839 KB
>
> To solve this problem, I can think following 2 possible solutions.
> 1. allocate the pages on cma reserved memory first, and if they are
> exhausted, allocate movable pages.
> 2. interleaved allocation: try to allocate specific amounts of memory
> from cma reserved memory and then allocate from free movable memory.
>
> I tested #1 approach and found the problem. Although free memory on
> meminfo can move around low watermark, there is large fluctuation on free
> memory, because too many pages are reclaimed when kswapd is invoked.
> Reason for this behaviour is that successive allocated CMA pages are
> on the LRU list in that order and kswapd reclaim them in same order.
> These memory doesn't help watermark checking from kwapd, so too many
> pages are reclaimed, I guess.
>
> So, I implement #2 approach.
> One thing I should note is that we should not change allocation target
> (movable list or cma) on each allocation attempt, since this prevent
> allocated pages to be in physically succession, so some I/O devices can
> be hurt their performance. To solve this, I keep allocation target
> in at least pageblock_nr_pages attempts and make this number reflect
> ratio, free pages without free cma pages to free cma pages. With this
> approach, system works very smoothly and fully utilize the pages on
> cma reserved memory.
>
> Following is the experimental result of this patch.
>
> 4 CPUs, 1024 MB, VIRTUAL MACHINE
> make -j16
>
> <Before>
> CMA reserve: 0 MB 512 MB
> Elapsed-time: 225.2 472.5
> Average-MemFree: 322490 KB 630839 KB
> nr_free_cma: 0 131068
> pswpin: 0 261666
> pswpout: 75 1241363
>
> <After>
> CMA reserve: 0 MB 512 MB
> Elapsed-time: 222.7 224
> Average-MemFree: 325595 KB 393033 KB
> nr_free_cma: 0 61001
> pswpin: 0 6
> pswpout: 44 502
>
> There is no difference if we don't have cma reserved memory (0 MB case).
> But, with cma reserved memory (512 MB case), we fully utilize these
> reserved memory through this patch and the system behaves like as
> it doesn't reserve any memory.
>
> With this patch, we aggressively allocate the pages on cma reserved memory
> so latency of CMA can arise. Below is the experimental result about
> latency.
>
> 4 CPUs, 1024 MB, VIRTUAL MACHINE
> CMA reserve: 512 MB
> Backgound Workload: make -jN
> Real Workload: 8 MB CMA allocation/free 20 times with 5 sec interval
>
> N: 1 4 8 16
> Elapsed-time(Before): 4309.75 9511.09 12276.1 77103.5
> Elapsed-time(After): 5391.69 16114.1 19380.3 34879.2
>
> So generally we can see latency increase. Ratio of this increase
> is rather big - up to 70%. But, under the heavy workload, it shows
> latency decrease - up to 55%. This may be worst-case scenario, but
> reducing it would be important for some system, so, I can say that
> this patch have advantages and disadvantages in terms of latency.
>
> Although I think that this patch is right direction for CMA, there is
> side-effect in following case. If there is small memory zone and CMA
> occupys most of them, LRU for this zone would have many CMA pages. When
> reclaim is started, these CMA pages would be reclaimed, but not counted
> for watermark checking, so too many CMA pages could be reclaimed
> unnecessarily. Until now, this can't happen because free CMA pages aren't
> used easily. But, with this patch, free CMA pages are used easily so
> this problem can be possible. I will handle it on another patchset
> after some investigating.
>
> v2: In fastpath, just replenish counters. Calculation is done whenver
> cma area is varied
>
> Acked-by: Michal Nazarewicz <mina86@mina86.com>
> Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
>
> diff --git a/arch/powerpc/kvm/book3s_hv_cma.c b/arch/powerpc/kvm/book3s_hv_cma.c
> index d9d3d85..84a7582 100644
> --- a/arch/powerpc/kvm/book3s_hv_cma.c
> +++ b/arch/powerpc/kvm/book3s_hv_cma.c
> @@ -132,6 +132,8 @@ struct page *kvm_alloc_cma(unsigned long nr_pages, unsigned long align_pages)
> if (ret == 0) {
> bitmap_set(cma->bitmap, pageno, nr_chunk);
> page = pfn_to_page(pfn);
> + adjust_managed_cma_page_count(page_zone(page),
> + nr_pages);
I think it should be -nr_pages to decrease the managed_cma_pages variable.
But it is not. I think there is a reason.
Why the managed_cma_pages is increased by allocation?
> memset(pfn_to_kaddr(pfn), 0, nr_pages << PAGE_SHIFT);
> break;
> } else if (ret != -EBUSY) {
> @@ -180,6 +182,7 @@ bool kvm_release_cma(struct page *pages, unsigned long nr_pages)
> (pfn - cma->base_pfn) >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT),
> nr_chunk);
> free_contig_range(pfn, nr_pages);
> + adjust_managed_cma_page_count(page_zone(pages), nr_pages);
> mutex_unlock(&kvm_cma_mutex);
>
> return true;
> @@ -210,6 +213,8 @@ static int __init kvm_cma_activate_area(unsigned long base_pfn,
> }
> init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> } while (--i);
> + adjust_managed_cma_page_count(zone, count);
> +
> return 0;
> }
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 165c2c2..c578d5a 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -160,6 +160,7 @@ static int __init cma_activate_area(struct cma *cma)
> }
> init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> } while (--i);
> + adjust_managed_cma_page_count(zone, cma->count);
>
> return 0;
> }
> @@ -307,6 +308,7 @@ struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> if (ret == 0) {
> bitmap_set(cma->bitmap, pageno, count);
> page = pfn_to_page(pfn);
> + adjust_managed_cma_page_count(page_zone(page), count);
I think this also should be -count.
> break;
> } else if (ret != -EBUSY) {
> break;
> @@ -353,6 +355,7 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> mutex_lock(&cma_mutex);
> bitmap_clear(cma->bitmap, pfn - cma->base_pfn, count);
> free_contig_range(pfn, count);
> + adjust_managed_cma_page_count(page_zone(pages), count);
> mutex_unlock(&cma_mutex);
>
> return true;
> diff --git a/include/linux/gfp.h b/include/linux/gfp.h
> index 39b81dc..51cffc1 100644
> --- a/include/linux/gfp.h
> +++ b/include/linux/gfp.h
> @@ -415,6 +415,7 @@ extern int alloc_contig_range(unsigned long start, unsigned long end,
> extern void free_contig_range(unsigned long pfn, unsigned nr_pages);
>
> /* CMA stuff */
> +extern void adjust_managed_cma_page_count(struct zone *zone, long count);
> extern void init_cma_reserved_pageblock(struct page *page);
>
> #endif
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index fac5509..f52cb96 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -389,6 +389,20 @@ struct zone {
> int compact_order_failed;
> #endif
>
> +#ifdef CONFIG_CMA
> + unsigned long managed_cma_pages;
> + /*
> + * Number of allocation attempt on each movable/cma type
> + * without switching type. max_try(movable/cma) maintain
> + * predefined calculated counter and replenish nr_try_(movable/cma)
> + * with each of them whenever both of them are 0.
> + */
> + int nr_try_movable;
> + int nr_try_cma;
> + int max_try_movable;
> + int max_try_cma;
> +#endif
> +
> ZONE_PADDING(_pad1_)
>
> /* Fields commonly accessed by the page reclaim scanner */
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 674ade7..ca678b6 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -788,6 +788,56 @@ void __init __free_pages_bootmem(struct page *page, unsigned int order)
> }
>
> #ifdef CONFIG_CMA
> +void adjust_managed_cma_page_count(struct zone *zone, long count)
> +{
> + unsigned long flags;
> + long total, cma, movable;
> +
> + spin_lock_irqsave(&zone->lock, flags);
> + zone->managed_cma_pages += count;
> +
> + total = zone->managed_pages;
> + cma = zone->managed_cma_pages;
> + movable = total - cma - high_wmark_pages(zone);
If cma can be negative value, above calcuation increase movable value becuase -cma becomes positive value.
Does it need a sign check?
> +
> + /* No cma pages, so do only movable allocation */
> + if (cma <= 0) {
> + zone->max_try_movable = pageblock_nr_pages;
> + zone->max_try_cma = 0;
> + goto out;
> + }
> +
> + /*
> + * We want to consume cma pages with well balanced ratio so that
> + * we have consumed enough cma pages before the reclaim. For this
> + * purpose, we can use the ratio, movable : cma. And we doesn't
> + * want to switch too frequently, because it prevent allocated pages
> + * from beging successive and it is bad for some sorts of devices.
> + * I choose pageblock_nr_pages for the minimum amount of successive
> + * allocation because it is the size of a huge page and fragmentation
> + * avoidance is implemented based on this size.
> + *
> + * To meet above criteria, I derive following equation.
> + *
> + * if (movable > cma) then; movable : cma = X : pageblock_nr_pages
> + * else (movable <= cma) then; movable : cma = pageblock_nr_pages : X
> + */
> + if (movable > cma) {
> + zone->max_try_movable =
> + (movable * pageblock_nr_pages) / cma;
I think you assume that cma value cannot be negative. If cma can be negative, the resule of dividing by cma becomes negative. Right?
> + zone->max_try_cma = pageblock_nr_pages;
> + } else {
> + zone->max_try_movable = pageblock_nr_pages;
> + zone->max_try_cma = cma * pageblock_nr_pages / movable;
> + }
> +
> +out:
> + zone->nr_try_movable = zone->max_try_movable;
> + zone->nr_try_cma = zone->max_try_cma;
> +
> + spin_unlock_irqrestore(&zone->lock, flags);
> +}
> +
> /* Free whole pageblock and set its migration type to MIGRATE_CMA. */
> void __init init_cma_reserved_pageblock(struct page *page)
> {
> @@ -1136,6 +1186,36 @@ __rmqueue_fallback(struct zone *zone, int order, int start_migratetype)
> return NULL;
> }
>
> +#ifdef CONFIG_CMA
> +static struct page *__rmqueue_cma(struct zone *zone, unsigned int order)
> +{
> + struct page *page;
> +
> + if (zone->nr_try_movable > 0)
> + goto alloc_movable;
> +
> + if (zone->nr_try_cma > 0) {
> + /* Okay. Now, we can try to allocate the page from cma region */
> + zone->nr_try_cma -= 1 << order;
> + page = __rmqueue_smallest(zone, order, MIGRATE_CMA);
> +
> + /* CMA pages can vanish through CMA allocation */
> + if (unlikely(!page && order == 0))
> + zone->nr_try_cma = 0;
> +
> + return page;
> + }
> +
> + /* Reset counter */
> + zone->nr_try_movable = zone->max_try_movable;
> + zone->nr_try_cma = zone->max_try_cma;
> +
> +alloc_movable:
> + zone->nr_try_movable -= 1 << order;
> + return NULL;
> +}
> +#endif
> +
> /*
> * Do the hard work of removing an element from the buddy allocator.
> * Call me with the zone->lock already held.
> @@ -1143,10 +1223,15 @@ __rmqueue_fallback(struct zone *zone, int order, int start_migratetype)
> static struct page *__rmqueue(struct zone *zone, unsigned int order,
> int migratetype)
> {
> - struct page *page;
> + struct page *page = NULL;
> +
> + if (IS_ENABLED(CONFIG_CMA) &&
You might know that CONFIG_CMA is enabled and there is no CMA memory, because CONFIG_CMA_SIZE_MBYTES can be zero.
Is IS_ENABLED(CONFIG_CMA) alright in that case?
> + migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages)
> + page = __rmqueue_cma(zone, order);
>
> retry_reserve:
> - page = __rmqueue_smallest(zone, order, migratetype);
> + if (!page)
> + page = __rmqueue_smallest(zone, order, migratetype);
>
> if (unlikely(!page) && migratetype != MIGRATE_RESERVE) {
> page = __rmqueue_fallback(zone, order, migratetype);
> @@ -4849,6 +4934,8 @@ static void __paginginit free_area_init_core(struct pglist_data *pgdat,
> zone_seqlock_init(zone);
> zone->zone_pgdat = pgdat;
> zone_pcp_init(zone);
> + if (IS_ENABLED(CONFIG_CMA))
> + zone->managed_cma_pages = 0;
>
> /* For bootup, initialized properly in watermark setup */
> mod_zone_page_state(zone, NR_ALLOC_BATCH, zone->managed_pages);
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-05-29 7:24 ` Gioh Kim
@ 2014-05-29 7:48 ` Joonsoo Kim
2014-05-29 8:09 ` Gioh Kim
0 siblings, 1 reply; 24+ messages in thread
From: Joonsoo Kim @ 2014-05-29 7:48 UTC (permalink / raw)
To: Gioh Kim
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, Mel Gorman,
Laura Abbott, Minchan Kim, Heesub Shin, Marek Szyprowski,
Michal Nazarewicz, Aneesh Kumar K.V, linux-mm, linux-kernel
On Thu, May 29, 2014 at 04:24:58PM +0900, Gioh Kim wrote:
> I've not understand your code fully. Please let me ask some silly questions.
>
> 2014-05-28 i??i?? 4:04, Joonsoo Kim i?' e,?:
> > CMA is introduced to provide physically contiguous pages at runtime.
> > For this purpose, it reserves memory at boot time. Although it reserve
> > memory, this reserved memory can be used for movable memory allocation
> > request. This usecase is beneficial to the system that needs this CMA
> > reserved memory infrequently and it is one of main purpose of
> > introducing CMA.
> >
> > But, there is a problem in current implementation. The problem is that
> > it works like as just reserved memory approach. The pages on cma reserved
> > memory are hardly used for movable memory allocation. This is caused by
> > combination of allocation and reclaim policy.
> >
> > The pages on cma reserved memory are allocated if there is no movable
> > memory, that is, as fallback allocation. So the time this fallback
> > allocation is started is under heavy memory pressure. Although it is under
> > memory pressure, movable allocation easily succeed, since there would be
> > many pages on cma reserved memory. But this is not the case for unmovable
> > and reclaimable allocation, because they can't use the pages on cma
> > reserved memory. These allocations regard system's free memory as
> > (free pages - free cma pages) on watermark checking, that is, free
> > unmovable pages + free reclaimable pages + free movable pages. Because
> > we already exhausted movable pages, only free pages we have are unmovable
> > and reclaimable types and this would be really small amount. So watermark
> > checking would be failed. It will wake up kswapd to make enough free
> > memory for unmovable and reclaimable allocation and kswapd will do.
> > So before we fully utilize pages on cma reserved memory, kswapd start to
> > reclaim memory and try to make free memory over the high watermark. This
> > watermark checking by kswapd doesn't take care free cma pages so many
> > movable pages would be reclaimed. After then, we have a lot of movable
> > pages again, so fallback allocation doesn't happen again. To conclude,
> > amount of free memory on meminfo which includes free CMA pages is moving
> > around 512 MB if I reserve 512 MB memory for CMA.
> >
> > I found this problem on following experiment.
> >
> > 4 CPUs, 1024 MB, VIRTUAL MACHINE
> > make -j16
> >
> > CMA reserve: 0 MB 512 MB
> > Elapsed-time: 225.2 472.5
> > Average-MemFree: 322490 KB 630839 KB
> >
> > To solve this problem, I can think following 2 possible solutions.
> > 1. allocate the pages on cma reserved memory first, and if they are
> > exhausted, allocate movable pages.
> > 2. interleaved allocation: try to allocate specific amounts of memory
> > from cma reserved memory and then allocate from free movable memory.
> >
> > I tested #1 approach and found the problem. Although free memory on
> > meminfo can move around low watermark, there is large fluctuation on free
> > memory, because too many pages are reclaimed when kswapd is invoked.
> > Reason for this behaviour is that successive allocated CMA pages are
> > on the LRU list in that order and kswapd reclaim them in same order.
> > These memory doesn't help watermark checking from kwapd, so too many
> > pages are reclaimed, I guess.
> >
> > So, I implement #2 approach.
> > One thing I should note is that we should not change allocation target
> > (movable list or cma) on each allocation attempt, since this prevent
> > allocated pages to be in physically succession, so some I/O devices can
> > be hurt their performance. To solve this, I keep allocation target
> > in at least pageblock_nr_pages attempts and make this number reflect
> > ratio, free pages without free cma pages to free cma pages. With this
> > approach, system works very smoothly and fully utilize the pages on
> > cma reserved memory.
> >
> > Following is the experimental result of this patch.
> >
> > 4 CPUs, 1024 MB, VIRTUAL MACHINE
> > make -j16
> >
> > <Before>
> > CMA reserve: 0 MB 512 MB
> > Elapsed-time: 225.2 472.5
> > Average-MemFree: 322490 KB 630839 KB
> > nr_free_cma: 0 131068
> > pswpin: 0 261666
> > pswpout: 75 1241363
> >
> > <After>
> > CMA reserve: 0 MB 512 MB
> > Elapsed-time: 222.7 224
> > Average-MemFree: 325595 KB 393033 KB
> > nr_free_cma: 0 61001
> > pswpin: 0 6
> > pswpout: 44 502
> >
> > There is no difference if we don't have cma reserved memory (0 MB case).
> > But, with cma reserved memory (512 MB case), we fully utilize these
> > reserved memory through this patch and the system behaves like as
> > it doesn't reserve any memory.
> >
> > With this patch, we aggressively allocate the pages on cma reserved memory
> > so latency of CMA can arise. Below is the experimental result about
> > latency.
> >
> > 4 CPUs, 1024 MB, VIRTUAL MACHINE
> > CMA reserve: 512 MB
> > Backgound Workload: make -jN
> > Real Workload: 8 MB CMA allocation/free 20 times with 5 sec interval
> >
> > N: 1 4 8 16
> > Elapsed-time(Before): 4309.75 9511.09 12276.1 77103.5
> > Elapsed-time(After): 5391.69 16114.1 19380.3 34879.2
> >
> > So generally we can see latency increase. Ratio of this increase
> > is rather big - up to 70%. But, under the heavy workload, it shows
> > latency decrease - up to 55%. This may be worst-case scenario, but
> > reducing it would be important for some system, so, I can say that
> > this patch have advantages and disadvantages in terms of latency.
> >
> > Although I think that this patch is right direction for CMA, there is
> > side-effect in following case. If there is small memory zone and CMA
> > occupys most of them, LRU for this zone would have many CMA pages. When
> > reclaim is started, these CMA pages would be reclaimed, but not counted
> > for watermark checking, so too many CMA pages could be reclaimed
> > unnecessarily. Until now, this can't happen because free CMA pages aren't
> > used easily. But, with this patch, free CMA pages are used easily so
> > this problem can be possible. I will handle it on another patchset
> > after some investigating.
> >
> > v2: In fastpath, just replenish counters. Calculation is done whenver
> > cma area is varied
> >
> > Acked-by: Michal Nazarewicz <mina86@mina86.com>
> > Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
> >
> > diff --git a/arch/powerpc/kvm/book3s_hv_cma.c b/arch/powerpc/kvm/book3s_hv_cma.c
> > index d9d3d85..84a7582 100644
> > --- a/arch/powerpc/kvm/book3s_hv_cma.c
> > +++ b/arch/powerpc/kvm/book3s_hv_cma.c
> > @@ -132,6 +132,8 @@ struct page *kvm_alloc_cma(unsigned long nr_pages, unsigned long align_pages)
> > if (ret == 0) {
> > bitmap_set(cma->bitmap, pageno, nr_chunk);
> > page = pfn_to_page(pfn);
> > + adjust_managed_cma_page_count(page_zone(page),
> > + nr_pages);
>
> I think it should be -nr_pages to decrease the managed_cma_pages variable.
> But it is not. I think there is a reason.
> Why the managed_cma_pages is increased by allocation?
Hello, Gioh.
It's my mistake. It should be -nr_pages.
Thanks for pointing out.
>
> > memset(pfn_to_kaddr(pfn), 0, nr_pages << PAGE_SHIFT);
> > break;
> > } else if (ret != -EBUSY) {
> > @@ -180,6 +182,7 @@ bool kvm_release_cma(struct page *pages, unsigned long nr_pages)
> > (pfn - cma->base_pfn) >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT),
> > nr_chunk);
> > free_contig_range(pfn, nr_pages);
> > + adjust_managed_cma_page_count(page_zone(pages), nr_pages);
> > mutex_unlock(&kvm_cma_mutex);
> >
> > return true;
> > @@ -210,6 +213,8 @@ static int __init kvm_cma_activate_area(unsigned long base_pfn,
> > }
> > init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> > } while (--i);
> > + adjust_managed_cma_page_count(zone, count);
> > +
> > return 0;
> > }
> >
> > diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> > index 165c2c2..c578d5a 100644
> > --- a/drivers/base/dma-contiguous.c
> > +++ b/drivers/base/dma-contiguous.c
> > @@ -160,6 +160,7 @@ static int __init cma_activate_area(struct cma *cma)
> > }
> > init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> > } while (--i);
> > + adjust_managed_cma_page_count(zone, cma->count);
> >
> > return 0;
> > }
> > @@ -307,6 +308,7 @@ struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> > if (ret == 0) {
> > bitmap_set(cma->bitmap, pageno, count);
> > page = pfn_to_page(pfn);
> > + adjust_managed_cma_page_count(page_zone(page), count);
>
> I think this also should be -count.
Ditto.
>
> > break;
> > } else if (ret != -EBUSY) {
> > break;
> > @@ -353,6 +355,7 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> > mutex_lock(&cma_mutex);
> > bitmap_clear(cma->bitmap, pfn - cma->base_pfn, count);
> > free_contig_range(pfn, count);
> > + adjust_managed_cma_page_count(page_zone(pages), count);
> > mutex_unlock(&cma_mutex);
> >
> > return true;
> > diff --git a/include/linux/gfp.h b/include/linux/gfp.h
> > index 39b81dc..51cffc1 100644
> > --- a/include/linux/gfp.h
> > +++ b/include/linux/gfp.h
> > @@ -415,6 +415,7 @@ extern int alloc_contig_range(unsigned long start, unsigned long end,
> > extern void free_contig_range(unsigned long pfn, unsigned nr_pages);
> >
> > /* CMA stuff */
> > +extern void adjust_managed_cma_page_count(struct zone *zone, long count);
> > extern void init_cma_reserved_pageblock(struct page *page);
> >
> > #endif
> > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> > index fac5509..f52cb96 100644
> > --- a/include/linux/mmzone.h
> > +++ b/include/linux/mmzone.h
> > @@ -389,6 +389,20 @@ struct zone {
> > int compact_order_failed;
> > #endif
> >
> > +#ifdef CONFIG_CMA
> > + unsigned long managed_cma_pages;
> > + /*
> > + * Number of allocation attempt on each movable/cma type
> > + * without switching type. max_try(movable/cma) maintain
> > + * predefined calculated counter and replenish nr_try_(movable/cma)
> > + * with each of them whenever both of them are 0.
> > + */
> > + int nr_try_movable;
> > + int nr_try_cma;
> > + int max_try_movable;
> > + int max_try_cma;
> > +#endif
> > +
> > ZONE_PADDING(_pad1_)
> >
> > /* Fields commonly accessed by the page reclaim scanner */
> > diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> > index 674ade7..ca678b6 100644
> > --- a/mm/page_alloc.c
> > +++ b/mm/page_alloc.c
> > @@ -788,6 +788,56 @@ void __init __free_pages_bootmem(struct page *page, unsigned int order)
> > }
> >
> > #ifdef CONFIG_CMA
> > +void adjust_managed_cma_page_count(struct zone *zone, long count)
> > +{
> > + unsigned long flags;
> > + long total, cma, movable;
> > +
> > + spin_lock_irqsave(&zone->lock, flags);
> > + zone->managed_cma_pages += count;
> > +
> > + total = zone->managed_pages;
> > + cma = zone->managed_cma_pages;
> > + movable = total - cma - high_wmark_pages(zone);
>
> If cma can be negative value, above calcuation increase movable value becuase -cma becomes positive value.
> Does it need a sign check?
This is leftover from version 1. They (totla, cma, movable)
can not be negative on this v2. I will fix it on v3.
>
>
>
> > +
> > + /* No cma pages, so do only movable allocation */
> > + if (cma <= 0) {
> > + zone->max_try_movable = pageblock_nr_pages;
> > + zone->max_try_cma = 0;
> > + goto out;
> > + }
> > +
> > + /*
> > + * We want to consume cma pages with well balanced ratio so that
> > + * we have consumed enough cma pages before the reclaim. For this
> > + * purpose, we can use the ratio, movable : cma. And we doesn't
> > + * want to switch too frequently, because it prevent allocated pages
> > + * from beging successive and it is bad for some sorts of devices.
> > + * I choose pageblock_nr_pages for the minimum amount of successive
> > + * allocation because it is the size of a huge page and fragmentation
> > + * avoidance is implemented based on this size.
> > + *
> > + * To meet above criteria, I derive following equation.
> > + *
> > + * if (movable > cma) then; movable : cma = X : pageblock_nr_pages
> > + * else (movable <= cma) then; movable : cma = pageblock_nr_pages : X
> > + */
> > + if (movable > cma) {
> > + zone->max_try_movable =
> > + (movable * pageblock_nr_pages) / cma;
>
> I think you assume that cma value cannot be negative. If cma can be negative, the resule of dividing by cma becomes negative. Right?
It cannot be negative.
>
> > + zone->max_try_cma = pageblock_nr_pages;
> > + } else {
> > + zone->max_try_movable = pageblock_nr_pages;
> > + zone->max_try_cma = cma * pageblock_nr_pages / movable;
> > + }
> > +
> > +out:
> > + zone->nr_try_movable = zone->max_try_movable;
> > + zone->nr_try_cma = zone->max_try_cma;
> > +
> > + spin_unlock_irqrestore(&zone->lock, flags);
> > +}
> > +
> > /* Free whole pageblock and set its migration type to MIGRATE_CMA. */
> > void __init init_cma_reserved_pageblock(struct page *page)
> > {
> > @@ -1136,6 +1186,36 @@ __rmqueue_fallback(struct zone *zone, int order, int start_migratetype)
> > return NULL;
> > }
> >
> > +#ifdef CONFIG_CMA
> > +static struct page *__rmqueue_cma(struct zone *zone, unsigned int order)
> > +{
> > + struct page *page;
> > +
> > + if (zone->nr_try_movable > 0)
> > + goto alloc_movable;
> > +
> > + if (zone->nr_try_cma > 0) {
> > + /* Okay. Now, we can try to allocate the page from cma region */
> > + zone->nr_try_cma -= 1 << order;
> > + page = __rmqueue_smallest(zone, order, MIGRATE_CMA);
> > +
> > + /* CMA pages can vanish through CMA allocation */
> > + if (unlikely(!page && order == 0))
> > + zone->nr_try_cma = 0;
> > +
> > + return page;
> > + }
> > +
> > + /* Reset counter */
> > + zone->nr_try_movable = zone->max_try_movable;
> > + zone->nr_try_cma = zone->max_try_cma;
> > +
> > +alloc_movable:
> > + zone->nr_try_movable -= 1 << order;
> > + return NULL;
> > +}
> > +#endif
> > +
> > /*
> > * Do the hard work of removing an element from the buddy allocator.
> > * Call me with the zone->lock already held.
> > @@ -1143,10 +1223,15 @@ __rmqueue_fallback(struct zone *zone, int order, int start_migratetype)
> > static struct page *__rmqueue(struct zone *zone, unsigned int order,
> > int migratetype)
> > {
> > - struct page *page;
> > + struct page *page = NULL;
> > +
> > + if (IS_ENABLED(CONFIG_CMA) &&
>
> You might know that CONFIG_CMA is enabled and there is no CMA memory, because CONFIG_CMA_SIZE_MBYTES can be zero.
> Is IS_ENABLED(CONFIG_CMA) alright in that case?
next line checks whether zone->managed_cma_pages is positive or not.
If there is no CMA memory, zone->managed_cma_pages will be zero and
we will skip to call __rmqueue_cma().
Thanks for review!!!
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-05-29 7:48 ` Joonsoo Kim
@ 2014-05-29 8:09 ` Gioh Kim
2014-05-30 0:45 ` Joonsoo Kim
0 siblings, 1 reply; 24+ messages in thread
From: Gioh Kim @ 2014-05-29 8:09 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, Mel Gorman,
Laura Abbott, Minchan Kim, Heesub Shin, Marek Szyprowski,
Michal Nazarewicz, Aneesh Kumar K.V, linux-mm, linux-kernel
>>> +
>>> /*
>>> * Do the hard work of removing an element from the buddy allocator.
>>> * Call me with the zone->lock already held.
>>> @@ -1143,10 +1223,15 @@ __rmqueue_fallback(struct zone *zone, int order, int start_migratetype)
>>> static struct page *__rmqueue(struct zone *zone, unsigned int order,
>>> int migratetype)
>>> {
>>> - struct page *page;
>>> + struct page *page = NULL;
>>> +
>>> + if (IS_ENABLED(CONFIG_CMA) &&
>>
>> You might know that CONFIG_CMA is enabled and there is no CMA memory, because CONFIG_CMA_SIZE_MBYTES can be zero.
>> Is IS_ENABLED(CONFIG_CMA) alright in that case?
>
> next line checks whether zone->managed_cma_pages is positive or not.
> If there is no CMA memory, zone->managed_cma_pages will be zero and
> we will skip to call __rmqueue_cma().
Is IS_ENABLED(CONFIG_CMA) necessary?
What about if (migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages) ?
>
> Thanks for review!!!
>
> Thanks.
>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-05-29 8:09 ` Gioh Kim
@ 2014-05-30 0:45 ` Joonsoo Kim
2014-05-31 0:02 ` Michal Nazarewicz
0 siblings, 1 reply; 24+ messages in thread
From: Joonsoo Kim @ 2014-05-30 0:45 UTC (permalink / raw)
To: Gioh Kim
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, Mel Gorman,
Laura Abbott, Minchan Kim, Heesub Shin, Marek Szyprowski,
Michal Nazarewicz, Aneesh Kumar K.V, linux-mm, linux-kernel
On Thu, May 29, 2014 at 05:09:34PM +0900, Gioh Kim wrote:
>
> >>>+
> >>> /*
> >>> * Do the hard work of removing an element from the buddy allocator.
> >>> * Call me with the zone->lock already held.
> >>>@@ -1143,10 +1223,15 @@ __rmqueue_fallback(struct zone *zone, int order, int start_migratetype)
> >>> static struct page *__rmqueue(struct zone *zone, unsigned int order,
> >>> int migratetype)
> >>> {
> >>>- struct page *page;
> >>>+ struct page *page = NULL;
> >>>+
> >>>+ if (IS_ENABLED(CONFIG_CMA) &&
> >>
> >>You might know that CONFIG_CMA is enabled and there is no CMA memory, because CONFIG_CMA_SIZE_MBYTES can be zero.
> >>Is IS_ENABLED(CONFIG_CMA) alright in that case?
> >
> >next line checks whether zone->managed_cma_pages is positive or not.
> >If there is no CMA memory, zone->managed_cma_pages will be zero and
> >we will skip to call __rmqueue_cma().
>
> Is IS_ENABLED(CONFIG_CMA) necessary?
> What about if (migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages) ?
Yes, field, managed_cma_pages exists only if CONFIG_CMA is enabled, so
removing IS_ENABLE(CONFIG_CMA) would break the build.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-05-30 0:45 ` Joonsoo Kim
@ 2014-05-31 0:02 ` Michal Nazarewicz
2014-06-02 6:17 ` Joonsoo Kim
0 siblings, 1 reply; 24+ messages in thread
From: Michal Nazarewicz @ 2014-05-31 0:02 UTC (permalink / raw)
To: Joonsoo Kim, Gioh Kim
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, Mel Gorman,
Laura Abbott, Minchan Kim, Heesub Shin, Marek Szyprowski,
Aneesh Kumar K.V, linux-mm, linux-kernel
> On Thu, May 29, 2014 at 05:09:34PM +0900, Gioh Kim wrote:
>> Is IS_ENABLED(CONFIG_CMA) necessary?
>> What about if (migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages) ?
On Fri, May 30 2014, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> Yes, field, managed_cma_pages exists only if CONFIG_CMA is enabled, so
> removing IS_ENABLE(CONFIG_CMA) would break the build.
That statement makes no sense. If zone->managed_cma_pages not being
defined is the problem, what you need is:
+#ifdef CONFIG_CMA
+ if (migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages)
+ page = __rmqueue_cma(zone, order);
+#endif
If you use IS_ENABLED, zone-managed_cma_pages has to be defined
regardless of result of state of CONFIG_CMA.
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz (o o)
ooo +--<mpn@google.com>--<xmpp:mina86@jabber.org>--ooO--(_)--Ooo--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-05-31 0:02 ` Michal Nazarewicz
@ 2014-06-02 6:17 ` Joonsoo Kim
0 siblings, 0 replies; 24+ messages in thread
From: Joonsoo Kim @ 2014-06-02 6:17 UTC (permalink / raw)
To: Michal Nazarewicz
Cc: Gioh Kim, Andrew Morton, Rik van Riel, Johannes Weiner,
Mel Gorman, Laura Abbott, Minchan Kim, Heesub Shin,
Marek Szyprowski, Aneesh Kumar K.V, linux-mm, linux-kernel
On Sat, May 31, 2014 at 09:02:51AM +0900, Michal Nazarewicz wrote:
> > On Thu, May 29, 2014 at 05:09:34PM +0900, Gioh Kim wrote:
> >> Is IS_ENABLED(CONFIG_CMA) necessary?
> >> What about if (migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages) ?
>
> On Fri, May 30 2014, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> > Yes, field, managed_cma_pages exists only if CONFIG_CMA is enabled, so
> > removing IS_ENABLE(CONFIG_CMA) would break the build.
>
> That statement makes no sense. If zone->managed_cma_pages not being
> defined is the problem, what you need is:
>
> +#ifdef CONFIG_CMA
> + if (migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages)
> + page = __rmqueue_cma(zone, order);
> +#endif
>
> If you use IS_ENABLED, zone-managed_cma_pages has to be defined
> regardless of result of state of CONFIG_CMA.
Hello,
Oops. I totally misunderstand how IS_ENABLED works.
Thanks for spotting this.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-05-28 7:04 ` [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used Joonsoo Kim
2014-05-29 7:24 ` Gioh Kim
@ 2014-05-30 7:53 ` Gioh Kim
2014-05-30 14:23 ` Joonsoo Kim
2014-05-31 0:11 ` Michal Nazarewicz
` (2 subsequent siblings)
4 siblings, 1 reply; 24+ messages in thread
From: Gioh Kim @ 2014-05-30 7:53 UTC (permalink / raw)
To: Joonsoo Kim, Andrew Morton
Cc: Rik van Riel, Johannes Weiner, Mel Gorman, Laura Abbott,
Minchan Kim, Heesub Shin, Marek Szyprowski, Michal Nazarewicz,
Aneesh Kumar K.V, linux-mm, linux-kernel
Joonsoo,
I'm attaching a patch for combination of __rmqueue and __rmqueue_cma.
I didn't test fully but my board is turned on and working well if no frequent memory allocations.
I'm sorry to send not-tested code.
I just want to report this during your working hour ;-)
I'm testing this this evening and reporting next week.
Have a nice weekend!
-------------------------------------- 8< -----------------------------------------
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 7f97767..9ced736 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -964,7 +964,7 @@ static int fallbacks[MIGRATE_TYPES][4] = {
[MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE, MIGRATE_R
#ifdef CONFIG_CMA
[MIGRATE_MOVABLE] = { MIGRATE_CMA, MIGRATE_RECLAIMABLE, MIGRATE_U
- [MIGRATE_CMA] = { MIGRATE_RESERVE }, /* Never used */
+ [MIGRATE_CMA] = { MIGRATE_MOVABLE, MIGRATE_RECLAIMABLE, MIGRATE_U
#else
[MIGRATE_MOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE, MIGRATE_R
#endif
@@ -1170,9 +1170,22 @@ static struct page *__rmqueue(struct zone *zone, unsigned int
int migratetype)
{
struct page *page;
+ long free, free_cma, free_wmark;
retry_reserve:
- page = __rmqueue_smallest(zone, order, migratetype);
+ if (IS_ENABLED(CONFIG_CMA) && migratetype == MIGRATE_MOVABLE) {
+ if (zone->nr_try_movable) {
+ zone->nr_try_movable -= 1 << order;
+ } else if (zone->nr_try_cma) {
+ zone->nr_try_cma -= 1 << order;
+ migratetype = MIGRATE_CMA;
+ } else {
+ zone->nr_try_movable = zone->max_try_movable;
+ zone->nr_try_movable -= 1 << order;
+ zone->nr_try_cma = zone->max_try_cma;
+ }
+ }
+ page = __rmqueue_smallest(zone, order, migratetype);
if (unlikely(!page) && migratetype != MIGRATE_RESERVE) {
page = __rmqueue_fallback(zone, order, migratetype);
2014-05-28 ?AEA 4:04, Joonsoo Kim 3/4 ' +-U:
> CMA is introduced to provide physically contiguous pages at runtime.
> For this purpose, it reserves memory at boot time. Although it reserve
> memory, this reserved memory can be used for movable memory allocation
> request. This usecase is beneficial to the system that needs this CMA
> reserved memory infrequently and it is one of main purpose of
> introducing CMA.
>
> But, there is a problem in current implementation. The problem is that
> it works like as just reserved memory approach. The pages on cma reserved
> memory are hardly used for movable memory allocation. This is caused by
> combination of allocation and reclaim policy.
>
> The pages on cma reserved memory are allocated if there is no movable
> memory, that is, as fallback allocation. So the time this fallback
> allocation is started is under heavy memory pressure. Although it is under
> memory pressure, movable allocation easily succeed, since there would be
> many pages on cma reserved memory. But this is not the case for unmovable
> and reclaimable allocation, because they can't use the pages on cma
> reserved memory. These allocations regard system's free memory as
> (free pages - free cma pages) on watermark checking, that is, free
> unmovable pages + free reclaimable pages + free movable pages. Because
> we already exhausted movable pages, only free pages we have are unmovable
> and reclaimable types and this would be really small amount. So watermark
> checking would be failed. It will wake up kswapd to make enough free
> memory for unmovable and reclaimable allocation and kswapd will do.
> So before we fully utilize pages on cma reserved memory, kswapd start to
> reclaim memory and try to make free memory over the high watermark. This
> watermark checking by kswapd doesn't take care free cma pages so many
> movable pages would be reclaimed. After then, we have a lot of movable
> pages again, so fallback allocation doesn't happen again. To conclude,
> amount of free memory on meminfo which includes free CMA pages is moving
> around 512 MB if I reserve 512 MB memory for CMA.
>
> I found this problem on following experiment.
>
> 4 CPUs, 1024 MB, VIRTUAL MACHINE
> make -j16
>
> CMA reserve: 0 MB 512 MB
> Elapsed-time: 225.2 472.5
> Average-MemFree: 322490 KB 630839 KB
>
> To solve this problem, I can think following 2 possible solutions.
> 1. allocate the pages on cma reserved memory first, and if they are
> exhausted, allocate movable pages.
> 2. interleaved allocation: try to allocate specific amounts of memory
> from cma reserved memory and then allocate from free movable memory.
>
> I tested #1 approach and found the problem. Although free memory on
> meminfo can move around low watermark, there is large fluctuation on free
> memory, because too many pages are reclaimed when kswapd is invoked.
> Reason for this behaviour is that successive allocated CMA pages are
> on the LRU list in that order and kswapd reclaim them in same order.
> These memory doesn't help watermark checking from kwapd, so too many
> pages are reclaimed, I guess.
>
> So, I implement #2 approach.
> One thing I should note is that we should not change allocation target
> (movable list or cma) on each allocation attempt, since this prevent
> allocated pages to be in physically succession, so some I/O devices can
> be hurt their performance. To solve this, I keep allocation target
> in at least pageblock_nr_pages attempts and make this number reflect
> ratio, free pages without free cma pages to free cma pages. With this
> approach, system works very smoothly and fully utilize the pages on
> cma reserved memory.
>
> Following is the experimental result of this patch.
>
> 4 CPUs, 1024 MB, VIRTUAL MACHINE
> make -j16
>
> <Before>
> CMA reserve: 0 MB 512 MB
> Elapsed-time: 225.2 472.5
> Average-MemFree: 322490 KB 630839 KB
> nr_free_cma: 0 131068
> pswpin: 0 261666
> pswpout: 75 1241363
>
> <After>
> CMA reserve: 0 MB 512 MB
> Elapsed-time: 222.7 224
> Average-MemFree: 325595 KB 393033 KB
> nr_free_cma: 0 61001
> pswpin: 0 6
> pswpout: 44 502
>
> There is no difference if we don't have cma reserved memory (0 MB case).
> But, with cma reserved memory (512 MB case), we fully utilize these
> reserved memory through this patch and the system behaves like as
> it doesn't reserve any memory.
>
> With this patch, we aggressively allocate the pages on cma reserved memory
> so latency of CMA can arise. Below is the experimental result about
> latency.
>
> 4 CPUs, 1024 MB, VIRTUAL MACHINE
> CMA reserve: 512 MB
> Backgound Workload: make -jN
> Real Workload: 8 MB CMA allocation/free 20 times with 5 sec interval
>
> N: 1 4 8 16
> Elapsed-time(Before): 4309.75 9511.09 12276.1 77103.5
> Elapsed-time(After): 5391.69 16114.1 19380.3 34879.2
>
> So generally we can see latency increase. Ratio of this increase
> is rather big - up to 70%. But, under the heavy workload, it shows
> latency decrease - up to 55%. This may be worst-case scenario, but
> reducing it would be important for some system, so, I can say that
> this patch have advantages and disadvantages in terms of latency.
>
> Although I think that this patch is right direction for CMA, there is
> side-effect in following case. If there is small memory zone and CMA
> occupys most of them, LRU for this zone would have many CMA pages. When
> reclaim is started, these CMA pages would be reclaimed, but not counted
> for watermark checking, so too many CMA pages could be reclaimed
> unnecessarily. Until now, this can't happen because free CMA pages aren't
> used easily. But, with this patch, free CMA pages are used easily so
> this problem can be possible. I will handle it on another patchset
> after some investigating.
>
> v2: In fastpath, just replenish counters. Calculation is done whenver
> cma area is varied
>
> Acked-by: Michal Nazarewicz <mina86@mina86.com>
> Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
>
> diff --git a/arch/powerpc/kvm/book3s_hv_cma.c b/arch/powerpc/kvm/book3s_hv_cma.c
> index d9d3d85..84a7582 100644
> --- a/arch/powerpc/kvm/book3s_hv_cma.c
> +++ b/arch/powerpc/kvm/book3s_hv_cma.c
> @@ -132,6 +132,8 @@ struct page *kvm_alloc_cma(unsigned long nr_pages, unsigned long align_pages)
> if (ret == 0) {
> bitmap_set(cma->bitmap, pageno, nr_chunk);
> page = pfn_to_page(pfn);
> + adjust_managed_cma_page_count(page_zone(page),
> + nr_pages);
> memset(pfn_to_kaddr(pfn), 0, nr_pages << PAGE_SHIFT);
> break;
> } else if (ret != -EBUSY) {
> @@ -180,6 +182,7 @@ bool kvm_release_cma(struct page *pages, unsigned long nr_pages)
> (pfn - cma->base_pfn) >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT),
> nr_chunk);
> free_contig_range(pfn, nr_pages);
> + adjust_managed_cma_page_count(page_zone(pages), nr_pages);
> mutex_unlock(&kvm_cma_mutex);
>
> return true;
> @@ -210,6 +213,8 @@ static int __init kvm_cma_activate_area(unsigned long base_pfn,
> }
> init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> } while (--i);
> + adjust_managed_cma_page_count(zone, count);
> +
> return 0;
> }
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 165c2c2..c578d5a 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -160,6 +160,7 @@ static int __init cma_activate_area(struct cma *cma)
> }
> init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> } while (--i);
> + adjust_managed_cma_page_count(zone, cma->count);
>
> return 0;
> }
> @@ -307,6 +308,7 @@ struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> if (ret == 0) {
> bitmap_set(cma->bitmap, pageno, count);
> page = pfn_to_page(pfn);
> + adjust_managed_cma_page_count(page_zone(page), count);
> break;
> } else if (ret != -EBUSY) {
> break;
> @@ -353,6 +355,7 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> mutex_lock(&cma_mutex);
> bitmap_clear(cma->bitmap, pfn - cma->base_pfn, count);
> free_contig_range(pfn, count);
> + adjust_managed_cma_page_count(page_zone(pages), count);
> mutex_unlock(&cma_mutex);
>
> return true;
> diff --git a/include/linux/gfp.h b/include/linux/gfp.h
> index 39b81dc..51cffc1 100644
> --- a/include/linux/gfp.h
> +++ b/include/linux/gfp.h
> @@ -415,6 +415,7 @@ extern int alloc_contig_range(unsigned long start, unsigned long end,
> extern void free_contig_range(unsigned long pfn, unsigned nr_pages);
>
> /* CMA stuff */
> +extern void adjust_managed_cma_page_count(struct zone *zone, long count);
> extern void init_cma_reserved_pageblock(struct page *page);
>
> #endif
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index fac5509..f52cb96 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -389,6 +389,20 @@ struct zone {
> int compact_order_failed;
> #endif
>
> +#ifdef CONFIG_CMA
> + unsigned long managed_cma_pages;
> + /*
> + * Number of allocation attempt on each movable/cma type
> + * without switching type. max_try(movable/cma) maintain
> + * predefined calculated counter and replenish nr_try_(movable/cma)
> + * with each of them whenever both of them are 0.
> + */
> + int nr_try_movable;
> + int nr_try_cma;
> + int max_try_movable;
> + int max_try_cma;
> +#endif
> +
> ZONE_PADDING(_pad1_)
>
> /* Fields commonly accessed by the page reclaim scanner */
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 674ade7..ca678b6 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -788,6 +788,56 @@ void __init __free_pages_bootmem(struct page *page, unsigned int order)
> }
>
> #ifdef CONFIG_CMA
> +void adjust_managed_cma_page_count(struct zone *zone, long count)
> +{
> + unsigned long flags;
> + long total, cma, movable;
> +
> + spin_lock_irqsave(&zone->lock, flags);
> + zone->managed_cma_pages += count;
> +
> + total = zone->managed_pages;
> + cma = zone->managed_cma_pages;
> + movable = total - cma - high_wmark_pages(zone);
> +
> + /* No cma pages, so do only movable allocation */
> + if (cma <= 0) {
> + zone->max_try_movable = pageblock_nr_pages;
> + zone->max_try_cma = 0;
> + goto out;
> + }
> +
> + /*
> + * We want to consume cma pages with well balanced ratio so that
> + * we have consumed enough cma pages before the reclaim. For this
> + * purpose, we can use the ratio, movable : cma. And we doesn't
> + * want to switch too frequently, because it prevent allocated pages
> + * from beging successive and it is bad for some sorts of devices.
> + * I choose pageblock_nr_pages for the minimum amount of successive
> + * allocation because it is the size of a huge page and fragmentation
> + * avoidance is implemented based on this size.
> + *
> + * To meet above criteria, I derive following equation.
> + *
> + * if (movable > cma) then; movable : cma = X : pageblock_nr_pages
> + * else (movable <= cma) then; movable : cma = pageblock_nr_pages : X
> + */
> + if (movable > cma) {
> + zone->max_try_movable =
> + (movable * pageblock_nr_pages) / cma;
> + zone->max_try_cma = pageblock_nr_pages;
> + } else {
> + zone->max_try_movable = pageblock_nr_pages;
> + zone->max_try_cma = cma * pageblock_nr_pages / movable;
> + }
> +
> +out:
> + zone->nr_try_movable = zone->max_try_movable;
> + zone->nr_try_cma = zone->max_try_cma;
> +
> + spin_unlock_irqrestore(&zone->lock, flags);
> +}
> +
> /* Free whole pageblock and set its migration type to MIGRATE_CMA. */
> void __init init_cma_reserved_pageblock(struct page *page)
> {
> @@ -1136,6 +1186,36 @@ __rmqueue_fallback(struct zone *zone, int order, int start_migratetype)
> return NULL;
> }
>
> +#ifdef CONFIG_CMA
> +static struct page *__rmqueue_cma(struct zone *zone, unsigned int order)
> +{
> + struct page *page;
> +
> + if (zone->nr_try_movable > 0)
> + goto alloc_movable;
> +
> + if (zone->nr_try_cma > 0) {
> + /* Okay. Now, we can try to allocate the page from cma region */
> + zone->nr_try_cma -= 1 << order;
> + page = __rmqueue_smallest(zone, order, MIGRATE_CMA);
> +
> + /* CMA pages can vanish through CMA allocation */
> + if (unlikely(!page && order == 0))
> + zone->nr_try_cma = 0;
> +
> + return page;
> + }
> +
> + /* Reset counter */
> + zone->nr_try_movable = zone->max_try_movable;
> + zone->nr_try_cma = zone->max_try_cma;
> +
> +alloc_movable:
> + zone->nr_try_movable -= 1 << order;
> + return NULL;
> +}
> +#endif
> +
> /*
> * Do the hard work of removing an element from the buddy allocator.
> * Call me with the zone->lock already held.
> @@ -1143,10 +1223,15 @@ __rmqueue_fallback(struct zone *zone, int order, int start_migratetype)
> static struct page *__rmqueue(struct zone *zone, unsigned int order,
> int migratetype)
> {
> - struct page *page;
> + struct page *page = NULL;
> +
> + if (IS_ENABLED(CONFIG_CMA) &&
> + migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages)
> + page = __rmqueue_cma(zone, order);
>
> retry_reserve:
> - page = __rmqueue_smallest(zone, order, migratetype);
> + if (!page)
> + page = __rmqueue_smallest(zone, order, migratetype);
>
> if (unlikely(!page) && migratetype != MIGRATE_RESERVE) {
> page = __rmqueue_fallback(zone, order, migratetype);
> @@ -4849,6 +4934,8 @@ static void __paginginit free_area_init_core(struct pglist_data *pgdat,
> zone_seqlock_init(zone);
> zone->zone_pgdat = pgdat;
> zone_pcp_init(zone);
> + if (IS_ENABLED(CONFIG_CMA))
> + zone->managed_cma_pages = 0;
>
> /* For bootup, initialized properly in watermark setup */
> mod_zone_page_state(zone, NR_ALLOC_BATCH, zone->managed_pages);
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 24+ messages in thread* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-05-30 7:53 ` Gioh Kim
@ 2014-05-30 14:23 ` Joonsoo Kim
2014-06-02 5:54 ` Gioh Kim
0 siblings, 1 reply; 24+ messages in thread
From: Joonsoo Kim @ 2014-05-30 14:23 UTC (permalink / raw)
To: Gioh Kim
Cc: Joonsoo Kim, Andrew Morton, Rik van Riel, Johannes Weiner,
Mel Gorman, Laura Abbott, Minchan Kim, Heesub Shin,
Marek Szyprowski, Michal Nazarewicz, Aneesh Kumar K.V,
Linux Memory Management List, LKML
2014-05-30 16:53 GMT+09:00 Gioh Kim <gioh.kim@lge.com>:
> Joonsoo,
>
> I'm attaching a patch for combination of __rmqueue and __rmqueue_cma.
> I didn't test fully but my board is turned on and working well if no frequent memory allocations.
>
> I'm sorry to send not-tested code.
> I just want to report this during your working hour ;-)
>
> I'm testing this this evening and reporting next week.
> Have a nice weekend!
Thanks Gioh. :)
> -------------------------------------- 8< -----------------------------------------
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 7f97767..9ced736 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -964,7 +964,7 @@ static int fallbacks[MIGRATE_TYPES][4] = {
> [MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE, MIGRATE_R
> #ifdef CONFIG_CMA
> [MIGRATE_MOVABLE] = { MIGRATE_CMA, MIGRATE_RECLAIMABLE, MIGRATE_U
> - [MIGRATE_CMA] = { MIGRATE_RESERVE }, /* Never used */
> + [MIGRATE_CMA] = { MIGRATE_MOVABLE, MIGRATE_RECLAIMABLE, MIGRATE_U
I don't want to use __rmqueue_fallback() for CMA.
__rmqueue_fallback() takes big order page rather than small order page
in order to steal large amount of pages and continue to use them in
next allocation attempts.
We can use CMA pages on limited cases, so stealing some pages from
other migrate type
to CMA type isn't good idea to me.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-05-30 14:23 ` Joonsoo Kim
@ 2014-06-02 5:54 ` Gioh Kim
2014-06-02 6:23 ` Joonsoo Kim
0 siblings, 1 reply; 24+ messages in thread
From: Gioh Kim @ 2014-06-02 5:54 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, Mel Gorman,
Laura Abbott, Minchan Kim, Heesub Shin, Marek Szyprowski,
Michal Nazarewicz, Aneesh Kumar K.V, Linux Memory Management List,
LKML, 이건호
I found 2 problems at my platform.
1st is occured when I set CMA size 528MB and total memory is 960MB.
I print some values in adjust_managed_cma_page_count(),
the total value becomes 105439 and cma value 131072.
Finally movable value becomes negative value.
The total value 105439 means 411MB.
Is the zone->managed_pages value pages amount except the CMA?
I think zone->managed_pages value is including CMA size but it's value is strange.
2nd is a kernel panic at __netdev_alloc_skb().
I'm not sure it is caused by the CMA.
I'm checking it again and going to send you another report with detail call-stacks.
2014-05-30 i??i?? 11:23, Joonsoo Kim i?' e,?:
> 2014-05-30 16:53 GMT+09:00 Gioh Kim <gioh.kim@lge.com>:
>> Joonsoo,
>>
>> I'm attaching a patch for combination of __rmqueue and __rmqueue_cma.
>> I didn't test fully but my board is turned on and working well if no frequent memory allocations.
>>
>> I'm sorry to send not-tested code.
>> I just want to report this during your working hour ;-)
>>
>> I'm testing this this evening and reporting next week.
>> Have a nice weekend!
>
> Thanks Gioh. :)
>
>> -------------------------------------- 8< -----------------------------------------
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index 7f97767..9ced736 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -964,7 +964,7 @@ static int fallbacks[MIGRATE_TYPES][4] = {
>> [MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE, MIGRATE_R
>> #ifdef CONFIG_CMA
>> [MIGRATE_MOVABLE] = { MIGRATE_CMA, MIGRATE_RECLAIMABLE, MIGRATE_U
>> - [MIGRATE_CMA] = { MIGRATE_RESERVE }, /* Never used */
>> + [MIGRATE_CMA] = { MIGRATE_MOVABLE, MIGRATE_RECLAIMABLE, MIGRATE_U
>
> I don't want to use __rmqueue_fallback() for CMA.
> __rmqueue_fallback() takes big order page rather than small order page
> in order to steal large amount of pages and continue to use them in
> next allocation attempts.
> We can use CMA pages on limited cases, so stealing some pages from
> other migrate type
> to CMA type isn't good idea to me.
>
> Thanks.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-06-02 5:54 ` Gioh Kim
@ 2014-06-02 6:23 ` Joonsoo Kim
2014-06-02 7:13 ` Gioh Kim
0 siblings, 1 reply; 24+ messages in thread
From: Joonsoo Kim @ 2014-06-02 6:23 UTC (permalink / raw)
To: Gioh Kim
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, Mel Gorman,
Laura Abbott, Minchan Kim, Heesub Shin, Marek Szyprowski,
Michal Nazarewicz, Aneesh Kumar K.V, Linux Memory Management List,
LKML, 이건호
On Mon, Jun 02, 2014 at 02:54:30PM +0900, Gioh Kim wrote:
> I found 2 problems at my platform.
>
> 1st is occured when I set CMA size 528MB and total memory is 960MB.
> I print some values in adjust_managed_cma_page_count(),
> the total value becomes 105439 and cma value 131072.
> Finally movable value becomes negative value.
>
> The total value 105439 means 411MB.
> Is the zone->managed_pages value pages amount except the CMA?
> I think zone->managed_pages value is including CMA size but it's value is strange.
Hmm...
zone->managed_pages includes nr of CMA pages.
Is there any mistake about your printk?
>
> 2nd is a kernel panic at __netdev_alloc_skb().
> I'm not sure it is caused by the CMA.
> I'm checking it again and going to send you another report with detail call-stacks.
Okay.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-06-02 6:23 ` Joonsoo Kim
@ 2014-06-02 7:13 ` Gioh Kim
0 siblings, 0 replies; 24+ messages in thread
From: Gioh Kim @ 2014-06-02 7:13 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, Mel Gorman,
Laura Abbott, Minchan Kim, Heesub Shin, Marek Szyprowski,
Michal Nazarewicz, Aneesh Kumar K.V, Linux Memory Management List,
LKML, 이건호
I'm not sure what I'm doing wrong.
These are my code.
770 #ifdef CONFIG_CMA
771 void adjust_managed_cma_page_count(struct zone *zone, long count)
772 {
773 unsigned long flags;
774 long total, cma, movable;
775
776 spin_lock_irqsave(&zone->lock, flags);
777
778 zone->managed_cma_pages += count;
779
780 total = zone->managed_pages;
781 cma = zone->managed_cma_pages;
782 movable = total - cma - high_wmark_pages(zone);
783
784 printk("count=%ld total=%ld cma=%ld movable=%ld\n",
785 count, total, cma, movable);
786
2014-06-02 i??i?? 3:23, Joonsoo Kim i?' e,?:
> On Mon, Jun 02, 2014 at 02:54:30PM +0900, Gioh Kim wrote:
>> I found 2 problems at my platform.
>>
>> 1st is occured when I set CMA size 528MB and total memory is 960MB.
>> I print some values in adjust_managed_cma_page_count(),
>> the total value becomes 105439 and cma value 131072.
>> Finally movable value becomes negative value.
>>
>> The total value 105439 means 411MB.
>> Is the zone->managed_pages value pages amount except the CMA?
>> I think zone->managed_pages value is including CMA size but it's value is strange.
>
> Hmm...
> zone->managed_pages includes nr of CMA pages.
> Is there any mistake about your printk?
>
>>
>> 2nd is a kernel panic at __netdev_alloc_skb().
>> I'm not sure it is caused by the CMA.
>> I'm checking it again and going to send you another report with detail call-stacks.
>
> Okay.
>
> Thanks.
>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-05-28 7:04 ` [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used Joonsoo Kim
2014-05-29 7:24 ` Gioh Kim
2014-05-30 7:53 ` Gioh Kim
@ 2014-05-31 0:11 ` Michal Nazarewicz
2014-10-30 10:37 ` Hui Zhu
[not found] ` <CADtm3G5Cb2vzVo61qDJ7-1ZNzQ2zOisfjb7GiFXvZR0ocKZy0A@mail.gmail.com>
4 siblings, 0 replies; 24+ messages in thread
From: Michal Nazarewicz @ 2014-05-31 0:11 UTC (permalink / raw)
To: Joonsoo Kim, Andrew Morton
Cc: Rik van Riel, Johannes Weiner, Mel Gorman, Laura Abbott,
Minchan Kim, Heesub Shin, Marek Szyprowski, Aneesh Kumar K.V,
linux-mm, linux-kernel
On Wed, May 28 2014, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> @@ -1143,10 +1223,15 @@ __rmqueue_fallback(struct zone *zone, int order, int start_migratetype)
> static struct page *__rmqueue(struct zone *zone, unsigned int order,
> int migratetype)
> {
> - struct page *page;
> + struct page *page = NULL;
> +
> + if (IS_ENABLED(CONFIG_CMA) &&
> + migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages)
> + page = __rmqueue_cma(zone, order);
Come to think of it, I would consider:
if (…) {
page = __rmqueue_cma(zone, order);
if (page)
goto done
}
…
done:
trace_mm_page_alloc_zone_locked(page, order, migratetype);
return page;
>
> retry_reserve:
> - page = __rmqueue_smallest(zone, order, migratetype);
> + if (!page)
> + page = __rmqueue_smallest(zone, order, migratetype);
>
The above would allow this if statement to go away.
> if (unlikely(!page) && migratetype != MIGRATE_RESERVE) {
> page = __rmqueue_fallback(zone, order, migratetype);
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz (o o)
ooo +--<mpn@google.com>--<xmpp:mina86@jabber.org>--ooO--(_)--Ooo--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2014-05-28 7:04 ` [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used Joonsoo Kim
` (2 preceding siblings ...)
2014-05-31 0:11 ` Michal Nazarewicz
@ 2014-10-30 10:37 ` Hui Zhu
[not found] ` <CADtm3G5Cb2vzVo61qDJ7-1ZNzQ2zOisfjb7GiFXvZR0ocKZy0A@mail.gmail.com>
4 siblings, 0 replies; 24+ messages in thread
From: Hui Zhu @ 2014-10-30 10:37 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, Mel Gorman,
Laura Abbott, Minchan Kim, Heesub Shin, Marek Szyprowski,
Michal Nazarewicz, Aneesh Kumar K.V, linux-mm,
linux-kernel@vger.kernel.org
On Wed, May 28, 2014 at 3:04 PM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> CMA is introduced to provide physically contiguous pages at runtime.
> For this purpose, it reserves memory at boot time. Although it reserve
> memory, this reserved memory can be used for movable memory allocation
> request. This usecase is beneficial to the system that needs this CMA
> reserved memory infrequently and it is one of main purpose of
> introducing CMA.
>
> But, there is a problem in current implementation. The problem is that
> it works like as just reserved memory approach. The pages on cma reserved
> memory are hardly used for movable memory allocation. This is caused by
> combination of allocation and reclaim policy.
>
> The pages on cma reserved memory are allocated if there is no movable
> memory, that is, as fallback allocation. So the time this fallback
> allocation is started is under heavy memory pressure. Although it is under
> memory pressure, movable allocation easily succeed, since there would be
> many pages on cma reserved memory. But this is not the case for unmovable
> and reclaimable allocation, because they can't use the pages on cma
> reserved memory. These allocations regard system's free memory as
> (free pages - free cma pages) on watermark checking, that is, free
> unmovable pages + free reclaimable pages + free movable pages. Because
> we already exhausted movable pages, only free pages we have are unmovable
> and reclaimable types and this would be really small amount. So watermark
> checking would be failed. It will wake up kswapd to make enough free
> memory for unmovable and reclaimable allocation and kswapd will do.
> So before we fully utilize pages on cma reserved memory, kswapd start to
> reclaim memory and try to make free memory over the high watermark. This
> watermark checking by kswapd doesn't take care free cma pages so many
> movable pages would be reclaimed. After then, we have a lot of movable
> pages again, so fallback allocation doesn't happen again. To conclude,
> amount of free memory on meminfo which includes free CMA pages is moving
> around 512 MB if I reserve 512 MB memory for CMA.
>
> I found this problem on following experiment.
>
> 4 CPUs, 1024 MB, VIRTUAL MACHINE
> make -j16
>
> CMA reserve: 0 MB 512 MB
> Elapsed-time: 225.2 472.5
> Average-MemFree: 322490 KB 630839 KB
>
> To solve this problem, I can think following 2 possible solutions.
> 1. allocate the pages on cma reserved memory first, and if they are
> exhausted, allocate movable pages.
> 2. interleaved allocation: try to allocate specific amounts of memory
> from cma reserved memory and then allocate from free movable memory.
>
> I tested #1 approach and found the problem. Although free memory on
> meminfo can move around low watermark, there is large fluctuation on free
> memory, because too many pages are reclaimed when kswapd is invoked.
> Reason for this behaviour is that successive allocated CMA pages are
> on the LRU list in that order and kswapd reclaim them in same order.
> These memory doesn't help watermark checking from kwapd, so too many
> pages are reclaimed, I guess.
Could you send more information about this part? I want to do some
test around it.
I use this way in my patch.
Thanks,
Hui
>
> So, I implement #2 approach.
> One thing I should note is that we should not change allocation target
> (movable list or cma) on each allocation attempt, since this prevent
> allocated pages to be in physically succession, so some I/O devices can
> be hurt their performance. To solve this, I keep allocation target
> in at least pageblock_nr_pages attempts and make this number reflect
> ratio, free pages without free cma pages to free cma pages. With this
> approach, system works very smoothly and fully utilize the pages on
> cma reserved memory.
>
> Following is the experimental result of this patch.
>
> 4 CPUs, 1024 MB, VIRTUAL MACHINE
> make -j16
>
> <Before>
> CMA reserve: 0 MB 512 MB
> Elapsed-time: 225.2 472.5
> Average-MemFree: 322490 KB 630839 KB
> nr_free_cma: 0 131068
> pswpin: 0 261666
> pswpout: 75 1241363
>
> <After>
> CMA reserve: 0 MB 512 MB
> Elapsed-time: 222.7 224
> Average-MemFree: 325595 KB 393033 KB
> nr_free_cma: 0 61001
> pswpin: 0 6
> pswpout: 44 502
>
> There is no difference if we don't have cma reserved memory (0 MB case).
> But, with cma reserved memory (512 MB case), we fully utilize these
> reserved memory through this patch and the system behaves like as
> it doesn't reserve any memory.
>
> With this patch, we aggressively allocate the pages on cma reserved memory
> so latency of CMA can arise. Below is the experimental result about
> latency.
>
> 4 CPUs, 1024 MB, VIRTUAL MACHINE
> CMA reserve: 512 MB
> Backgound Workload: make -jN
> Real Workload: 8 MB CMA allocation/free 20 times with 5 sec interval
>
> N: 1 4 8 16
> Elapsed-time(Before): 4309.75 9511.09 12276.1 77103.5
> Elapsed-time(After): 5391.69 16114.1 19380.3 34879.2
>
> So generally we can see latency increase. Ratio of this increase
> is rather big - up to 70%. But, under the heavy workload, it shows
> latency decrease - up to 55%. This may be worst-case scenario, but
> reducing it would be important for some system, so, I can say that
> this patch have advantages and disadvantages in terms of latency.
>
> Although I think that this patch is right direction for CMA, there is
> side-effect in following case. If there is small memory zone and CMA
> occupys most of them, LRU for this zone would have many CMA pages. When
> reclaim is started, these CMA pages would be reclaimed, but not counted
> for watermark checking, so too many CMA pages could be reclaimed
> unnecessarily. Until now, this can't happen because free CMA pages aren't
> used easily. But, with this patch, free CMA pages are used easily so
> this problem can be possible. I will handle it on another patchset
> after some investigating.
>
> v2: In fastpath, just replenish counters. Calculation is done whenver
> cma area is varied
>
> Acked-by: Michal Nazarewicz <mina86@mina86.com>
> Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
>
> diff --git a/arch/powerpc/kvm/book3s_hv_cma.c b/arch/powerpc/kvm/book3s_hv_cma.c
> index d9d3d85..84a7582 100644
> --- a/arch/powerpc/kvm/book3s_hv_cma.c
> +++ b/arch/powerpc/kvm/book3s_hv_cma.c
> @@ -132,6 +132,8 @@ struct page *kvm_alloc_cma(unsigned long nr_pages, unsigned long align_pages)
> if (ret == 0) {
> bitmap_set(cma->bitmap, pageno, nr_chunk);
> page = pfn_to_page(pfn);
> + adjust_managed_cma_page_count(page_zone(page),
> + nr_pages);
> memset(pfn_to_kaddr(pfn), 0, nr_pages << PAGE_SHIFT);
> break;
> } else if (ret != -EBUSY) {
> @@ -180,6 +182,7 @@ bool kvm_release_cma(struct page *pages, unsigned long nr_pages)
> (pfn - cma->base_pfn) >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT),
> nr_chunk);
> free_contig_range(pfn, nr_pages);
> + adjust_managed_cma_page_count(page_zone(pages), nr_pages);
> mutex_unlock(&kvm_cma_mutex);
>
> return true;
> @@ -210,6 +213,8 @@ static int __init kvm_cma_activate_area(unsigned long base_pfn,
> }
> init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> } while (--i);
> + adjust_managed_cma_page_count(zone, count);
> +
> return 0;
> }
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 165c2c2..c578d5a 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -160,6 +160,7 @@ static int __init cma_activate_area(struct cma *cma)
> }
> init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> } while (--i);
> + adjust_managed_cma_page_count(zone, cma->count);
>
> return 0;
> }
> @@ -307,6 +308,7 @@ struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> if (ret == 0) {
> bitmap_set(cma->bitmap, pageno, count);
> page = pfn_to_page(pfn);
> + adjust_managed_cma_page_count(page_zone(page), count);
> break;
> } else if (ret != -EBUSY) {
> break;
> @@ -353,6 +355,7 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> mutex_lock(&cma_mutex);
> bitmap_clear(cma->bitmap, pfn - cma->base_pfn, count);
> free_contig_range(pfn, count);
> + adjust_managed_cma_page_count(page_zone(pages), count);
> mutex_unlock(&cma_mutex);
>
> return true;
> diff --git a/include/linux/gfp.h b/include/linux/gfp.h
> index 39b81dc..51cffc1 100644
> --- a/include/linux/gfp.h
> +++ b/include/linux/gfp.h
> @@ -415,6 +415,7 @@ extern int alloc_contig_range(unsigned long start, unsigned long end,
> extern void free_contig_range(unsigned long pfn, unsigned nr_pages);
>
> /* CMA stuff */
> +extern void adjust_managed_cma_page_count(struct zone *zone, long count);
> extern void init_cma_reserved_pageblock(struct page *page);
>
> #endif
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index fac5509..f52cb96 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -389,6 +389,20 @@ struct zone {
> int compact_order_failed;
> #endif
>
> +#ifdef CONFIG_CMA
> + unsigned long managed_cma_pages;
> + /*
> + * Number of allocation attempt on each movable/cma type
> + * without switching type. max_try(movable/cma) maintain
> + * predefined calculated counter and replenish nr_try_(movable/cma)
> + * with each of them whenever both of them are 0.
> + */
> + int nr_try_movable;
> + int nr_try_cma;
> + int max_try_movable;
> + int max_try_cma;
> +#endif
> +
> ZONE_PADDING(_pad1_)
>
> /* Fields commonly accessed by the page reclaim scanner */
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 674ade7..ca678b6 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -788,6 +788,56 @@ void __init __free_pages_bootmem(struct page *page, unsigned int order)
> }
>
> #ifdef CONFIG_CMA
> +void adjust_managed_cma_page_count(struct zone *zone, long count)
> +{
> + unsigned long flags;
> + long total, cma, movable;
> +
> + spin_lock_irqsave(&zone->lock, flags);
> + zone->managed_cma_pages += count;
> +
> + total = zone->managed_pages;
> + cma = zone->managed_cma_pages;
> + movable = total - cma - high_wmark_pages(zone);
> +
> + /* No cma pages, so do only movable allocation */
> + if (cma <= 0) {
> + zone->max_try_movable = pageblock_nr_pages;
> + zone->max_try_cma = 0;
> + goto out;
> + }
> +
> + /*
> + * We want to consume cma pages with well balanced ratio so that
> + * we have consumed enough cma pages before the reclaim. For this
> + * purpose, we can use the ratio, movable : cma. And we doesn't
> + * want to switch too frequently, because it prevent allocated pages
> + * from beging successive and it is bad for some sorts of devices.
> + * I choose pageblock_nr_pages for the minimum amount of successive
> + * allocation because it is the size of a huge page and fragmentation
> + * avoidance is implemented based on this size.
> + *
> + * To meet above criteria, I derive following equation.
> + *
> + * if (movable > cma) then; movable : cma = X : pageblock_nr_pages
> + * else (movable <= cma) then; movable : cma = pageblock_nr_pages : X
> + */
> + if (movable > cma) {
> + zone->max_try_movable =
> + (movable * pageblock_nr_pages) / cma;
> + zone->max_try_cma = pageblock_nr_pages;
> + } else {
> + zone->max_try_movable = pageblock_nr_pages;
> + zone->max_try_cma = cma * pageblock_nr_pages / movable;
> + }
> +
> +out:
> + zone->nr_try_movable = zone->max_try_movable;
> + zone->nr_try_cma = zone->max_try_cma;
> +
> + spin_unlock_irqrestore(&zone->lock, flags);
> +}
> +
> /* Free whole pageblock and set its migration type to MIGRATE_CMA. */
> void __init init_cma_reserved_pageblock(struct page *page)
> {
> @@ -1136,6 +1186,36 @@ __rmqueue_fallback(struct zone *zone, int order, int start_migratetype)
> return NULL;
> }
>
> +#ifdef CONFIG_CMA
> +static struct page *__rmqueue_cma(struct zone *zone, unsigned int order)
> +{
> + struct page *page;
> +
> + if (zone->nr_try_movable > 0)
> + goto alloc_movable;
> +
> + if (zone->nr_try_cma > 0) {
> + /* Okay. Now, we can try to allocate the page from cma region */
> + zone->nr_try_cma -= 1 << order;
> + page = __rmqueue_smallest(zone, order, MIGRATE_CMA);
> +
> + /* CMA pages can vanish through CMA allocation */
> + if (unlikely(!page && order == 0))
> + zone->nr_try_cma = 0;
> +
> + return page;
> + }
> +
> + /* Reset counter */
> + zone->nr_try_movable = zone->max_try_movable;
> + zone->nr_try_cma = zone->max_try_cma;
> +
> +alloc_movable:
> + zone->nr_try_movable -= 1 << order;
> + return NULL;
> +}
> +#endif
> +
> /*
> * Do the hard work of removing an element from the buddy allocator.
> * Call me with the zone->lock already held.
> @@ -1143,10 +1223,15 @@ __rmqueue_fallback(struct zone *zone, int order, int start_migratetype)
> static struct page *__rmqueue(struct zone *zone, unsigned int order,
> int migratetype)
> {
> - struct page *page;
> + struct page *page = NULL;
> +
> + if (IS_ENABLED(CONFIG_CMA) &&
> + migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages)
> + page = __rmqueue_cma(zone, order);
>
> retry_reserve:
> - page = __rmqueue_smallest(zone, order, migratetype);
> + if (!page)
> + page = __rmqueue_smallest(zone, order, migratetype);
>
> if (unlikely(!page) && migratetype != MIGRATE_RESERVE) {
> page = __rmqueue_fallback(zone, order, migratetype);
> @@ -4849,6 +4934,8 @@ static void __paginginit free_area_init_core(struct pglist_data *pgdat,
> zone_seqlock_init(zone);
> zone->zone_pgdat = pgdat;
> zone_pcp_init(zone);
> + if (IS_ENABLED(CONFIG_CMA))
> + zone->managed_cma_pages = 0;
>
> /* For bootup, initialized properly in watermark setup */
> mod_zone_page_state(zone, NR_ALLOC_BATCH, zone->managed_pages);
> --
> 1.7.9.5
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread[parent not found: <CADtm3G5Cb2vzVo61qDJ7-1ZNzQ2zOisfjb7GiFXvZR0ocKZy0A@mail.gmail.com>]
* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
[not found] ` <CADtm3G5Cb2vzVo61qDJ7-1ZNzQ2zOisfjb7GiFXvZR0ocKZy0A@mail.gmail.com>
@ 2015-01-06 4:01 ` Gregory Fong
2015-01-06 8:23 ` Joonsoo Kim
0 siblings, 1 reply; 24+ messages in thread
From: Gregory Fong @ 2015-01-06 4:01 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, Mel Gorman,
Laura Abbott, Minchan Kim, Heesub Shin, Marek, linux-mm,
linux-kernel@vger.kernel.org
+linux-mm and linux-kernel (not sure how those got removed from cc,
sorry about that)
On Mon, Jan 5, 2015 at 7:58 PM, Gregory Fong <gregory.0xf0@gmail.com> wrote:
> Hi Joonsoo,
>
> On Wed, May 28, 2014 at 12:04 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index 674ade7..ca678b6 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -788,6 +788,56 @@ void __init __free_pages_bootmem(struct page *page, unsigned int order)
>> }
>>
>> #ifdef CONFIG_CMA
>> +void adjust_managed_cma_page_count(struct zone *zone, long count)
>> +{
>> + unsigned long flags;
>> + long total, cma, movable;
>> +
>> + spin_lock_irqsave(&zone->lock, flags);
>> + zone->managed_cma_pages += count;
>> +
>> + total = zone->managed_pages;
>> + cma = zone->managed_cma_pages;
>> + movable = total - cma - high_wmark_pages(zone);
>> +
>> + /* No cma pages, so do only movable allocation */
>> + if (cma <= 0) {
>> + zone->max_try_movable = pageblock_nr_pages;
>> + zone->max_try_cma = 0;
>> + goto out;
>> + }
>> +
>> + /*
>> + * We want to consume cma pages with well balanced ratio so that
>> + * we have consumed enough cma pages before the reclaim. For this
>> + * purpose, we can use the ratio, movable : cma. And we doesn't
>> + * want to switch too frequently, because it prevent allocated pages
>> + * from beging successive and it is bad for some sorts of devices.
>> + * I choose pageblock_nr_pages for the minimum amount of successive
>> + * allocation because it is the size of a huge page and fragmentation
>> + * avoidance is implemented based on this size.
>> + *
>> + * To meet above criteria, I derive following equation.
>> + *
>> + * if (movable > cma) then; movable : cma = X : pageblock_nr_pages
>> + * else (movable <= cma) then; movable : cma = pageblock_nr_pages : X
>> + */
>> + if (movable > cma) {
>> + zone->max_try_movable =
>> + (movable * pageblock_nr_pages) / cma;
>> + zone->max_try_cma = pageblock_nr_pages;
>> + } else {
>> + zone->max_try_movable = pageblock_nr_pages;
>> + zone->max_try_cma = cma * pageblock_nr_pages / movable;
>
> I don't know if anyone's already pointed this out (didn't see anything
> when searching lkml), but while testing this, I noticed this can
> result in a div by zero under memory pressure (movable becomes 0).
> This is not unlikely when the majority of pages are in CMA regions
> (this may seem pathological but we do actually do this right now).
>
> [ 0.249674] Division by zero in kernel.
> [ 0.249682] CPU: 2 PID: 1 Comm: swapper/0 Not tainted
> 3.14.13-1.3pre-00368-g4d90957-dirty #10
> [ 0.249710] [<c001619c>] (unwind_backtrace) from [<c0011fa4>]
> (show_stack+0x10/0x14)
> [ 0.249725] [<c0011fa4>] (show_stack) from [<c0538d6c>]
> (dump_stack+0x80/0x90)
> [ 0.249740] [<c0538d6c>] (dump_stack) from [<c025e9d0>] (Ldiv0+0x8/0x10)
> [ 0.249751] [<c025e9d0>] (Ldiv0) from [<c0094ba4>]
> (adjust_managed_cma_page_count+0x64/0xd8)
> [ 0.249762] [<c0094ba4>] (adjust_managed_cma_page_count) from
> [<c00cb2f4>] (cma_release+0xa8/0xe0)
> [ 0.249776] [<c00cb2f4>] (cma_release) from [<c0721698>]
> (cma_drvr_probe+0x378/0x470)
> [ 0.249787] [<c0721698>] (cma_drvr_probe) from [<c02ce9cc>]
> (platform_drv_probe+0x18/0x48)
> [ 0.249799] [<c02ce9cc>] (platform_drv_probe) from [<c02ccfb0>]
> (driver_probe_device+0xac/0x3a4)
> [ 0.249808] [<c02ccfb0>] (driver_probe_device) from [<c02cd378>]
> (__driver_attach+0x8c/0x90)
> [ 0.249817] [<c02cd378>] (__driver_attach) from [<c02cb390>]
> (bus_for_each_dev+0x60/0x94)
> [ 0.249825] [<c02cb390>] (bus_for_each_dev) from [<c02cc674>]
> (bus_add_driver+0x15c/0x218)
> [ 0.249834] [<c02cc674>] (bus_add_driver) from [<c02cd9a0>]
> (driver_register+0x78/0xf8)
> [ 0.249841] [<c02cd9a0>] (driver_register) from [<c02cea24>]
> (platform_driver_probe+0x20/0xa4)
> [ 0.249849] [<c02cea24>] (platform_driver_probe) from [<c0008958>]
> (do_one_initcall+0xd4/0x17c)
> [ 0.249857] [<c0008958>] (do_one_initcall) from [<c0719d00>]
> (kernel_init_freeable+0x13c/0x1dc)
> [ 0.249864] [<c0719d00>] (kernel_init_freeable) from [<c0534578>]
> (kernel_init+0x8/0xe8)
> [ 0.249873] [<c0534578>] (kernel_init) from [<c000ed78>]
> (ret_from_fork+0x14/0x3c)
>
> Could probably just add something above similar to the "no cma pages" case, like
>
> /* No movable pages, so only do CMA allocation */
> if (movable <= 0) {
> zone->max_try_cma = pageblock_nr_pages;
> zone->max_try_movable = 0;
> goto out;
> }
>
>> + }
>> +
>> +out:
>> + zone->nr_try_movable = zone->max_try_movable;
>> + zone->nr_try_cma = zone->max_try_cma;
>> +
>> + spin_unlock_irqrestore(&zone->lock, flags);
>> +}
>> +
>
> Best regards,
> Gregory
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 2/3] CMA: aggressively allocate the pages on cma reserved memory when not used
2015-01-06 4:01 ` Gregory Fong
@ 2015-01-06 8:23 ` Joonsoo Kim
0 siblings, 0 replies; 24+ messages in thread
From: Joonsoo Kim @ 2015-01-06 8:23 UTC (permalink / raw)
To: Gregory Fong
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, Mel Gorman,
Laura Abbott, Minchan Kim, Heesub Shin, Marek, linux-mm,
linux-kernel@vger.kernel.org
On Mon, Jan 05, 2015 at 08:01:45PM -0800, Gregory Fong wrote:
> +linux-mm and linux-kernel (not sure how those got removed from cc,
> sorry about that)
>
> On Mon, Jan 5, 2015 at 7:58 PM, Gregory Fong <gregory.0xf0@gmail.com> wrote:
> > Hi Joonsoo,
> >
> > On Wed, May 28, 2014 at 12:04 AM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> >> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> >> index 674ade7..ca678b6 100644
> >> --- a/mm/page_alloc.c
> >> +++ b/mm/page_alloc.c
> >> @@ -788,6 +788,56 @@ void __init __free_pages_bootmem(struct page *page, unsigned int order)
> >> }
> >>
> >> #ifdef CONFIG_CMA
> >> +void adjust_managed_cma_page_count(struct zone *zone, long count)
> >> +{
> >> + unsigned long flags;
> >> + long total, cma, movable;
> >> +
> >> + spin_lock_irqsave(&zone->lock, flags);
> >> + zone->managed_cma_pages += count;
> >> +
> >> + total = zone->managed_pages;
> >> + cma = zone->managed_cma_pages;
> >> + movable = total - cma - high_wmark_pages(zone);
> >> +
> >> + /* No cma pages, so do only movable allocation */
> >> + if (cma <= 0) {
> >> + zone->max_try_movable = pageblock_nr_pages;
> >> + zone->max_try_cma = 0;
> >> + goto out;
> >> + }
> >> +
> >> + /*
> >> + * We want to consume cma pages with well balanced ratio so that
> >> + * we have consumed enough cma pages before the reclaim. For this
> >> + * purpose, we can use the ratio, movable : cma. And we doesn't
> >> + * want to switch too frequently, because it prevent allocated pages
> >> + * from beging successive and it is bad for some sorts of devices.
> >> + * I choose pageblock_nr_pages for the minimum amount of successive
> >> + * allocation because it is the size of a huge page and fragmentation
> >> + * avoidance is implemented based on this size.
> >> + *
> >> + * To meet above criteria, I derive following equation.
> >> + *
> >> + * if (movable > cma) then; movable : cma = X : pageblock_nr_pages
> >> + * else (movable <= cma) then; movable : cma = pageblock_nr_pages : X
> >> + */
> >> + if (movable > cma) {
> >> + zone->max_try_movable =
> >> + (movable * pageblock_nr_pages) / cma;
> >> + zone->max_try_cma = pageblock_nr_pages;
> >> + } else {
> >> + zone->max_try_movable = pageblock_nr_pages;
> >> + zone->max_try_cma = cma * pageblock_nr_pages / movable;
> >
> > I don't know if anyone's already pointed this out (didn't see anything
> > when searching lkml), but while testing this, I noticed this can
> > result in a div by zero under memory pressure (movable becomes 0).
> > This is not unlikely when the majority of pages are in CMA regions
> > (this may seem pathological but we do actually do this right now).
Hello,
Yes, you are right. Thanks for pointing this out.
I will fix it on next version.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 24+ messages in thread