kswapd skips compaction if reclaim order drops to zero?

kswapd skips compaction if reclaim order drops to zero?

[Hillf Danton]

If the allocation order is not high, direct compaction does nothing.
Can we skip compaction here if order drops to zero?

--- a/mm/vmscan.c Thu Aug 15 17:47:26 2013
+++ b/mm/vmscan.c Thu Aug 15 17:48:58 2013
@@ -3034,7 +3034,7 @@ static unsigned long balance_pgdat(pg_da
  * Compact if necessary and kswapd is reclaiming at least the
  * high watermark number of pages as requsted
  */
- if (pgdat_needs_compaction && sc.nr_reclaimed > nr_attempted)
+ if (pgdat_needs_compaction && sc.nr_reclaimed > nr_attempted && order)
  compact_pgdat(pgdat, order);

  /*
--

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

kswapd skips compaction if reclaim order drops to zero?

[Hillf Danton]

If the allocation order is not high, direct compaction does nothing.
Can we skip compaction here if order drops to zero?

--- a/mm/vmscan.c Thu Aug 15 17:47:26 2013
+++ b/mm/vmscan.c Thu Aug 15 17:48:58 2013
@@ -3034,7 +3034,7 @@ static unsigned long balance_pgdat(pg_da
  * Compact if necessary and kswapd is reclaiming at least the
  * high watermark number of pages as requsted
  */
- if (pgdat_needs_compaction && sc.nr_reclaimed > nr_attempted)
+ if (pgdat_needs_compaction && sc.nr_reclaimed > nr_attempted && order)
  compact_pgdat(pgdat, order);

  /*
--

Re: kswapd skips compaction if reclaim order drops to zero?

[Mel Gorman]

On Thu, Aug 15, 2013 at 06:02:53PM +0800, Hillf Danton wrote:
> If the allocation order is not high, direct compaction does nothing.
> Can we skip compaction here if order drops to zero?
> 

If the allocation order is not high then

pgdat_needs_compaction == (order > 0) == false == no calling compact_pdatt

In the case where order is reset to 0 due to fragmentation then it does
call compact_pgdat but it does no work due to the cc->order check in
__compact_pgdat.

-- 
Mel Gorman
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: kswapd skips compaction if reclaim order drops to zero?

[Mel Gorman]

On Thu, Aug 15, 2013 at 06:02:53PM +0800, Hillf Danton wrote:
> If the allocation order is not high, direct compaction does nothing.
> Can we skip compaction here if order drops to zero?
> 

If the allocation order is not high then

pgdat_needs_compaction == (order > 0) == false == no calling compact_pdatt

In the case where order is reset to 0 due to fragmentation then it does
call compact_pgdat but it does no work due to the cc->order check in
__compact_pgdat.

-- 
Mel Gorman
SUSE Labs

Re: kswapd skips compaction if reclaim order drops to zero?

[Minchan Kim]

Hey Mel,

On Thu, Aug 15, 2013 at 11:47:27AM +0100, Mel Gorman wrote:
> On Thu, Aug 15, 2013 at 06:02:53PM +0800, Hillf Danton wrote:
> > If the allocation order is not high, direct compaction does nothing.
> > Can we skip compaction here if order drops to zero?
> > 
> 
> If the allocation order is not high then
> 
> pgdat_needs_compaction == (order > 0) == false == no calling compact_pdatt
> 
> In the case where order is reset to 0 due to fragmentation then it does
> call compact_pgdat but it does no work due to the cc->order check in
> __compact_pgdat.
> 

I am looking at mmotm-2013-08-07-16-55 but couldn't find cc->order
check right before compact_zone in __comact_pgdat.
Could you pinpoint code piece?

> -- 
> Mel Gorman
> SUSE Labs
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: kswapd skips compaction if reclaim order drops to zero?

[Minchan Kim]

Hey Mel,

On Thu, Aug 15, 2013 at 11:47:27AM +0100, Mel Gorman wrote:
> On Thu, Aug 15, 2013 at 06:02:53PM +0800, Hillf Danton wrote:
> > If the allocation order is not high, direct compaction does nothing.
> > Can we skip compaction here if order drops to zero?
> > 
> 
> If the allocation order is not high then
> 
> pgdat_needs_compaction == (order > 0) == false == no calling compact_pdatt
> 
> In the case where order is reset to 0 due to fragmentation then it does
> call compact_pgdat but it does no work due to the cc->order check in
> __compact_pgdat.
> 

I am looking at mmotm-2013-08-07-16-55 but couldn't find cc->order
check right before compact_zone in __comact_pgdat.
Could you pinpoint code piece?

> -- 
> Mel Gorman
> SUSE Labs
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Kind regards,
Minchan Kim

Re: kswapd skips compaction if reclaim order drops to zero?

[Mel Gorman]

On Thu, Aug 15, 2013 at 10:41:39PM +0900, Minchan Kim wrote:
> Hey Mel,
> 
> On Thu, Aug 15, 2013 at 11:47:27AM +0100, Mel Gorman wrote:
> > On Thu, Aug 15, 2013 at 06:02:53PM +0800, Hillf Danton wrote:
> > > If the allocation order is not high, direct compaction does nothing.
> > > Can we skip compaction here if order drops to zero?
> > > 
> > 
> > If the allocation order is not high then
> > 
> > pgdat_needs_compaction == (order > 0) == false == no calling compact_pdatt
> > 
> > In the case where order is reset to 0 due to fragmentation then it does
> > call compact_pgdat but it does no work due to the cc->order check in
> > __compact_pgdat.
> > 
> 
> I am looking at mmotm-2013-08-07-16-55 but couldn't find cc->order
> check right before compact_zone in __comact_pgdat.
> Could you pinpoint code piece?
> 

Thanks, I screwed up as that check happens too late. However, it still
ends up not mattering because it does this

compact_pgdat
  -> __compact_pgdat
    -> compact_zone
      -> compaction_suitable

For order == 0, compaction_suitable will return either COMPACT_SKIPPED
(if the watermarks are not met) and COMPACT_PARTIAL otherwise. Either
way, compaction doesn't run.

-- 
Mel Gorman
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: kswapd skips compaction if reclaim order drops to zero?

[Mel Gorman]

On Thu, Aug 15, 2013 at 10:41:39PM +0900, Minchan Kim wrote:
> Hey Mel,
> 
> On Thu, Aug 15, 2013 at 11:47:27AM +0100, Mel Gorman wrote:
> > On Thu, Aug 15, 2013 at 06:02:53PM +0800, Hillf Danton wrote:
> > > If the allocation order is not high, direct compaction does nothing.
> > > Can we skip compaction here if order drops to zero?
> > > 
> > 
> > If the allocation order is not high then
> > 
> > pgdat_needs_compaction == (order > 0) == false == no calling compact_pdatt
> > 
> > In the case where order is reset to 0 due to fragmentation then it does
> > call compact_pgdat but it does no work due to the cc->order check in
> > __compact_pgdat.
> > 
> 
> I am looking at mmotm-2013-08-07-16-55 but couldn't find cc->order
> check right before compact_zone in __comact_pgdat.
> Could you pinpoint code piece?
> 

Thanks, I screwed up as that check happens too late. However, it still
ends up not mattering because it does this

compact_pgdat
  -> __compact_pgdat
    -> compact_zone
      -> compaction_suitable

For order == 0, compaction_suitable will return either COMPACT_SKIPPED
(if the watermarks are not met) and COMPACT_PARTIAL otherwise. Either
way, compaction doesn't run.

-- 
Mel Gorman
SUSE Labs

Re: kswapd skips compaction if reclaim order drops to zero?

[Minchan Kim]

On Thu, Aug 15, 2013 at 02:56:27PM +0100, Mel Gorman wrote:
> On Thu, Aug 15, 2013 at 10:41:39PM +0900, Minchan Kim wrote:
> > Hey Mel,
> > 
> > On Thu, Aug 15, 2013 at 11:47:27AM +0100, Mel Gorman wrote:
> > > On Thu, Aug 15, 2013 at 06:02:53PM +0800, Hillf Danton wrote:
> > > > If the allocation order is not high, direct compaction does nothing.
> > > > Can we skip compaction here if order drops to zero?
> > > > 
> > > 
> > > If the allocation order is not high then
> > > 
> > > pgdat_needs_compaction == (order > 0) == false == no calling compact_pdatt
> > > 
> > > In the case where order is reset to 0 due to fragmentation then it does
> > > call compact_pgdat but it does no work due to the cc->order check in
> > > __compact_pgdat.
> > > 
> > 
> > I am looking at mmotm-2013-08-07-16-55 but couldn't find cc->order
> > check right before compact_zone in __comact_pgdat.
> > Could you pinpoint code piece?
> > 
> 
> Thanks, I screwed up as that check happens too late. However, it still
> ends up not mattering because it does this
> 
> compact_pgdat
>   -> __compact_pgdat
>     -> compact_zone
>       -> compaction_suitable
> 
> For order == 0, compaction_suitable will return either COMPACT_SKIPPED
> (if the watermarks are not met) and COMPACT_PARTIAL otherwise. Either
> way, compaction doesn't run.

In compaction_suitable, it could pass first zone_watermark_ok
but failed second zone_watermark_ok while fragindex is -1000
so compaction could run.

And we shouldn't depend on such coincidence.


> 
> -- 
> Mel Gorman
> SUSE Labs
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: kswapd skips compaction if reclaim order drops to zero?

[Minchan Kim]

On Thu, Aug 15, 2013 at 02:56:27PM +0100, Mel Gorman wrote:
> On Thu, Aug 15, 2013 at 10:41:39PM +0900, Minchan Kim wrote:
> > Hey Mel,
> > 
> > On Thu, Aug 15, 2013 at 11:47:27AM +0100, Mel Gorman wrote:
> > > On Thu, Aug 15, 2013 at 06:02:53PM +0800, Hillf Danton wrote:
> > > > If the allocation order is not high, direct compaction does nothing.
> > > > Can we skip compaction here if order drops to zero?
> > > > 
> > > 
> > > If the allocation order is not high then
> > > 
> > > pgdat_needs_compaction == (order > 0) == false == no calling compact_pdatt
> > > 
> > > In the case where order is reset to 0 due to fragmentation then it does
> > > call compact_pgdat but it does no work due to the cc->order check in
> > > __compact_pgdat.
> > > 
> > 
> > I am looking at mmotm-2013-08-07-16-55 but couldn't find cc->order
> > check right before compact_zone in __comact_pgdat.
> > Could you pinpoint code piece?
> > 
> 
> Thanks, I screwed up as that check happens too late. However, it still
> ends up not mattering because it does this
> 
> compact_pgdat
>   -> __compact_pgdat
>     -> compact_zone
>       -> compaction_suitable
> 
> For order == 0, compaction_suitable will return either COMPACT_SKIPPED
> (if the watermarks are not met) and COMPACT_PARTIAL otherwise. Either
> way, compaction doesn't run.

In compaction_suitable, it could pass first zone_watermark_ok
but failed second zone_watermark_ok while fragindex is -1000
so compaction could run.

And we shouldn't depend on such coincidence.


> 
> -- 
> Mel Gorman
> SUSE Labs
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Kind regards,
Minchan Kim

[PATCH] mm: compaction: Do not compact pgdat for order-0

[Mel Gorman]

If kswapd was reclaiming for a high order and resets it to 0 due to
fragmentation it will still call compact_pgdat. For the most part, this will
fail a compaction_suitable() test and not compact but it is unnecessarily
sloppy. It could be fixed in the caller but fix it in the API instead.

[dhillf@gmail.com: Pointed out that it was a potential problem]
Signed-off-by: Mel Gorman <mgorman@suse.de>
---
 mm/compaction.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/compaction.c b/mm/compaction.c
index 05ccb4c..c437893 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -1131,6 +1131,9 @@ void compact_pgdat(pg_data_t *pgdat, int order)
 		.sync = false,
 	};
 
+	if (!order)
+		return;
+
 	__compact_pgdat(pgdat, &cc);
 }
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH] mm: compaction: Do not compact pgdat for order-0

[Mel Gorman]

If kswapd was reclaiming for a high order and resets it to 0 due to
fragmentation it will still call compact_pgdat. For the most part, this will
fail a compaction_suitable() test and not compact but it is unnecessarily
sloppy. It could be fixed in the caller but fix it in the API instead.

[dhillf@gmail.com: Pointed out that it was a potential problem]
Signed-off-by: Mel Gorman <mgorman@suse.de>
---
 mm/compaction.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/compaction.c b/mm/compaction.c
index 05ccb4c..c437893 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -1131,6 +1131,9 @@ void compact_pgdat(pg_data_t *pgdat, int order)
 		.sync = false,
 	};
 
+	if (!order)
+		return;
+
 	__compact_pgdat(pgdat, &cc);
 }
 

Re: [PATCH] mm: compaction: Do not compact pgdat for order-0

[Minchan Kim]

On Thu, Aug 15, 2013 at 04:39:27PM +0100, Mel Gorman wrote:
> If kswapd was reclaiming for a high order and resets it to 0 due to
> fragmentation it will still call compact_pgdat. For the most part, this will
> fail a compaction_suitable() test and not compact but it is unnecessarily
> sloppy. It could be fixed in the caller but fix it in the API instead.
> 
> [dhillf@gmail.com: Pointed out that it was a potential problem]
> Signed-off-by: Mel Gorman <mgorman@suse.de>
Acked-by: Minchan Kim <minchan@kernel.org>

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: compaction: Do not compact pgdat for order-0

[Minchan Kim]

On Thu, Aug 15, 2013 at 04:39:27PM +0100, Mel Gorman wrote:
> If kswapd was reclaiming for a high order and resets it to 0 due to
> fragmentation it will still call compact_pgdat. For the most part, this will
> fail a compaction_suitable() test and not compact but it is unnecessarily
> sloppy. It could be fixed in the caller but fix it in the API instead.
> 
> [dhillf@gmail.com: Pointed out that it was a potential problem]
> Signed-off-by: Mel Gorman <mgorman@suse.de>
Acked-by: Minchan Kim <minchan@kernel.org>

-- 
Kind regards,
Minchan Kim

Re: kswapd skips compaction if reclaim order drops to zero?

[Wanpeng Li]

Hi Minchan,
On Thu, Aug 15, 2013 at 11:10:04PM +0900, Minchan Kim wrote:
>On Thu, Aug 15, 2013 at 02:56:27PM +0100, Mel Gorman wrote:
>> On Thu, Aug 15, 2013 at 10:41:39PM +0900, Minchan Kim wrote:
>> > Hey Mel,
>> > 
>> > On Thu, Aug 15, 2013 at 11:47:27AM +0100, Mel Gorman wrote:
>> > > On Thu, Aug 15, 2013 at 06:02:53PM +0800, Hillf Danton wrote:
>> > > > If the allocation order is not high, direct compaction does nothing.
>> > > > Can we skip compaction here if order drops to zero?
>> > > > 
>> > > 
>> > > If the allocation order is not high then
>> > > 
>> > > pgdat_needs_compaction == (order > 0) == false == no calling compact_pdatt
>> > > 
>> > > In the case where order is reset to 0 due to fragmentation then it does
>> > > call compact_pgdat but it does no work due to the cc->order check in
>> > > __compact_pgdat.
>> > > 
>> > 
>> > I am looking at mmotm-2013-08-07-16-55 but couldn't find cc->order
>> > check right before compact_zone in __comact_pgdat.
>> > Could you pinpoint code piece?
>> > 
>> 
>> Thanks, I screwed up as that check happens too late. However, it still
>> ends up not mattering because it does this
>> 
>> compact_pgdat
>>   -> __compact_pgdat
>>     -> compact_zone
>>       -> compaction_suitable
>> 
>> For order == 0, compaction_suitable will return either COMPACT_SKIPPED
>> (if the watermarks are not met) and COMPACT_PARTIAL otherwise. Either
>> way, compaction doesn't run.
>
>In compaction_suitable, it could pass first zone_watermark_ok
>but failed second zone_watermark_ok while fragindex is -1000
>so compaction could run.
>

I'm not sure why you said that second zone_watermark_ok failed while 
fragindex is -1000, actually they are the same check against order 0.

First:
watermark = low_wmark_pages(zone) + (2UL << order);
zone_watermark_ok(zone, 0, watermark, 0, 0);

Second:
fragindex == -1000 && zone_watermark_ok(zone, order, watermark, 0, 0)

Both COMPACT_SKIPPED and COMPACT_PARTIAL will fail compaction.

Regards,
Wanpeng Li 

>And we shouldn't depend on such coincidence.
>
>
>> 
>> -- 
>> Mel Gorman
>> SUSE Labs
>> 
>> --
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo@kvack.org.  For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>
>-- 
>Kind regards,
>Minchan Kim
>
>--
>To unsubscribe, send a message with 'unsubscribe linux-mm' in
>the body to majordomo@kvack.org.  For more info on Linux MM,
>see: http://www.linux-mm.org/ .
>Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: kswapd skips compaction if reclaim order drops to zero?

[Wanpeng Li]

Hi Minchan,
On Thu, Aug 15, 2013 at 11:10:04PM +0900, Minchan Kim wrote:
>On Thu, Aug 15, 2013 at 02:56:27PM +0100, Mel Gorman wrote:
>> On Thu, Aug 15, 2013 at 10:41:39PM +0900, Minchan Kim wrote:
>> > Hey Mel,
>> > 
>> > On Thu, Aug 15, 2013 at 11:47:27AM +0100, Mel Gorman wrote:
>> > > On Thu, Aug 15, 2013 at 06:02:53PM +0800, Hillf Danton wrote:
>> > > > If the allocation order is not high, direct compaction does nothing.
>> > > > Can we skip compaction here if order drops to zero?
>> > > > 
>> > > 
>> > > If the allocation order is not high then
>> > > 
>> > > pgdat_needs_compaction == (order > 0) == false == no calling compact_pdatt
>> > > 
>> > > In the case where order is reset to 0 due to fragmentation then it does
>> > > call compact_pgdat but it does no work due to the cc->order check in
>> > > __compact_pgdat.
>> > > 
>> > 
>> > I am looking at mmotm-2013-08-07-16-55 but couldn't find cc->order
>> > check right before compact_zone in __comact_pgdat.
>> > Could you pinpoint code piece?
>> > 
>> 
>> Thanks, I screwed up as that check happens too late. However, it still
>> ends up not mattering because it does this
>> 
>> compact_pgdat
>>   -> __compact_pgdat
>>     -> compact_zone
>>       -> compaction_suitable
>> 
>> For order == 0, compaction_suitable will return either COMPACT_SKIPPED
>> (if the watermarks are not met) and COMPACT_PARTIAL otherwise. Either
>> way, compaction doesn't run.
>
>In compaction_suitable, it could pass first zone_watermark_ok
>but failed second zone_watermark_ok while fragindex is -1000
>so compaction could run.
>

I'm not sure why you said that second zone_watermark_ok failed while 
fragindex is -1000, actually they are the same check against order 0.

First:
watermark = low_wmark_pages(zone) + (2UL << order);
zone_watermark_ok(zone, 0, watermark, 0, 0);

Second:
fragindex == -1000 && zone_watermark_ok(zone, order, watermark, 0, 0)

Both COMPACT_SKIPPED and COMPACT_PARTIAL will fail compaction.

Regards,
Wanpeng Li 

>And we shouldn't depend on such coincidence.
>
>
>> 
>> -- 
>> Mel Gorman
>> SUSE Labs
>> 
>> --
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo@kvack.org.  For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>
>-- 
>Kind regards,
>Minchan Kim
>
>--
>To unsubscribe, send a message with 'unsubscribe linux-mm' in
>the body to majordomo@kvack.org.  For more info on Linux MM,
>see: http://www.linux-mm.org/ .
>Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>