* Re: compaction: why depends on HUGETLB_PAGE
@ 2010-07-30 16:44 Round Robinjp
2010-07-30 16:49 ` Mel Gorman
0 siblings, 1 reply; 6+ messages in thread
From: Round Robinjp @ 2010-07-30 16:44 UTC (permalink / raw)
To: mel; +Cc: linux-mm, iram.shahzad
> > Please could you elaborate a little more why depending on
> > compaction to satisfy other high-order allocation is not good.
> >
>
> At the very least, it's not a situation that has been tested heavily and
> because other high-order allocations are typically not movable. In the
> worst case, if they are both frequent and long-lived they *may* eventually
> encounter fragmentation-related problems. This uncertainity is why it's
> not good. It gets worse if there is no swap as eventually all movable pages
> will be compacted as much as possible but there still might not be enough
> contiguous memory for a high-order page because other pages are pinned.
I am interested in this topic too.
How about using compaction for infrequent short-lived
high-order allocations? Is there any problem in that case?
(apart from the point that it is not tested for that purpose)
Also how about using compaction as a preparation
for partial refresh?
RR
--------------------------------------
Get the new Internet Explorer 8 optimized for Yahoo! JAPAN
http://pr.mail.yahoo.co.jp/ie8/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: compaction: why depends on HUGETLB_PAGE
2010-07-30 16:44 compaction: why depends on HUGETLB_PAGE Round Robinjp
@ 2010-07-30 16:49 ` Mel Gorman
0 siblings, 0 replies; 6+ messages in thread
From: Mel Gorman @ 2010-07-30 16:49 UTC (permalink / raw)
To: Round Robinjp; +Cc: linux-mm, iram.shahzad
On Sat, Jul 31, 2010 at 01:44:09AM +0900, Round Robinjp wrote:
> > > Please could you elaborate a little more why depending on
> > > compaction to satisfy other high-order allocation is not good.
> > >
> >
> > At the very least, it's not a situation that has been tested heavily and
> > because other high-order allocations are typically not movable. In the
> > worst case, if they are both frequent and long-lived they *may* eventually
> > encounter fragmentation-related problems. This uncertainity is why it's
> > not good. It gets worse if there is no swap as eventually all movable pages
> > will be compacted as much as possible but there still might not be enough
> > contiguous memory for a high-order page because other pages are pinned.
>
> I am interested in this topic too.
>
> How about using compaction for infrequent short-lived
> high-order allocations?
Depend on MIGRATE_RESERVE instead within fragmentation avoidance. It's
objective is to keep certain blocks of pages free unless there is no other
choice. How many blocks of MIGRATE_RESERVE there are depends on the value
of min_free_kbytes (which can be tuned to a recommended value with hugeadm)
MIGRATE_RESERVE is known to be important for short-lived high-order allocations
- particularly atomic ones.
> Is there any problem in that case?
> (apart from the point that it is not tested for that purpose)
>
It's racy, you are depending on compaction to happen at the right time
and with enough vigour to prevent allocation failures.
> Also how about using compaction as a preparation
> for partial refresh?
>
Hacky, but you could do it from userspace by periodically writing to
/proc/sys/vm/compact_memory. In the event allocation failures are
common, it would still be best to figure out how long-lived those
allocations are and why MIGRATE_RESERVE was insufficient.
I'm not saying pre-emptively compacting won't work, it probably will for
a large number of cases but there will be failure scenarios in the
field.
--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* compaction: why depends on HUGETLB_PAGE
@ 2010-07-29 1:53 Iram Shahzad
2010-07-29 12:57 ` Mel Gorman
0 siblings, 1 reply; 6+ messages in thread
From: Iram Shahzad @ 2010-07-29 1:53 UTC (permalink / raw)
To: linux-mm; +Cc: mel
Hi
I have found that memory compaction (CONFIG_COMPACTION)
is mainlined while looking at 2.6.35-rc5 source code.
I have a question regarding its dependency on HUGETLB_PAGE.
While trying to use CONFIG_COMPACTION on ARM architecture,
I found that I cannot enable CONFIG_COMPACTION because
it depends on CONFIG_HUGETLB_PAGE which is not available
on ARM.
I disabled the dependency and was able to build it.
And it looks like working!
My question is: why does it depend on CONFIG_HUGETLB_PAGE?
Is it wrong to use it on ARM by disabling CONFIG_HUGETLB_PAGE?
Iram
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: compaction: why depends on HUGETLB_PAGE
2010-07-29 1:53 Iram Shahzad
@ 2010-07-29 12:57 ` Mel Gorman
2010-07-30 2:56 ` Iram Shahzad
0 siblings, 1 reply; 6+ messages in thread
From: Mel Gorman @ 2010-07-29 12:57 UTC (permalink / raw)
To: Iram Shahzad; +Cc: linux-mm
On Thu, Jul 29, 2010 at 10:53:12AM +0900, Iram Shahzad wrote:
> Hi
>
> I have found that memory compaction (CONFIG_COMPACTION)
> is mainlined while looking at 2.6.35-rc5 source code.
> I have a question regarding its dependency on HUGETLB_PAGE.
>
> While trying to use CONFIG_COMPACTION on ARM architecture,
> I found that I cannot enable CONFIG_COMPACTION because
> it depends on CONFIG_HUGETLB_PAGE which is not available
> on ARM.
>
> I disabled the dependency and was able to build it.
> And it looks like working!
>
> My question is: why does it depend on CONFIG_HUGETLB_PAGE?
Because as the Kconfig says "Allows the compaction of memory for the
allocation of huge pages.". Depending on compaction to satisfy other
high-order allocation types is not likely to be a winning strategy.
> Is it wrong to use it on ARM by disabling CONFIG_HUGETLB_PAGE?
>
It depends on why you need compaction. If it's for some device that
requires high-order allocations (particularly if they are atomic), then
it's not likely to work very well in the long term.
--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: compaction: why depends on HUGETLB_PAGE
2010-07-29 12:57 ` Mel Gorman
@ 2010-07-30 2:56 ` Iram Shahzad
2010-07-30 9:57 ` Mel Gorman
0 siblings, 1 reply; 6+ messages in thread
From: Iram Shahzad @ 2010-07-30 2:56 UTC (permalink / raw)
To: Mel Gorman; +Cc: linux-mm
Mel Gorman wrote:
>> My question is: why does it depend on CONFIG_HUGETLB_PAGE?
>
> Because as the Kconfig says "Allows the compaction of memory for the
> allocation of huge pages.". Depending on compaction to satisfy other
> high-order allocation types is not likely to be a winning strategy.
Please could you elaborate a little more why depending on
compaction to satisfy other high-order allocation is not good.
>> Is it wrong to use it on ARM by disabling CONFIG_HUGETLB_PAGE?
>>
>
> It depends on why you need compaction. If it's for some device that
> requires high-order allocations (particularly if they are atomic), then
> it's not likely to work very well in the long term.
Would you please elaborate on this as well.
Many thanks for the reply
Iram
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: compaction: why depends on HUGETLB_PAGE
2010-07-30 2:56 ` Iram Shahzad
@ 2010-07-30 9:57 ` Mel Gorman
0 siblings, 0 replies; 6+ messages in thread
From: Mel Gorman @ 2010-07-30 9:57 UTC (permalink / raw)
To: Iram Shahzad; +Cc: linux-mm
On Fri, Jul 30, 2010 at 11:56:25AM +0900, Iram Shahzad wrote:
> Mel Gorman wrote:
>>> My question is: why does it depend on CONFIG_HUGETLB_PAGE?
>>
>> Because as the Kconfig says "Allows the compaction of memory for the
>> allocation of huge pages.". Depending on compaction to satisfy other
>> high-order allocation types is not likely to be a winning strategy.
>
> Please could you elaborate a little more why depending on
> compaction to satisfy other high-order allocation is not good.
>
At the very least, it's not a situation that has been tested heavily and
because other high-order allocations are typically not movable. In the
worst case, if they are both frequent and long-lived they *may* eventually
encounter fragmentation-related problems. This uncertainity is why it's
not good. It gets worse if there is no swap as eventually all movable pages
will be compacted as much as possible but there still might not be enough
contiguous memory for a high-order page because other pages are pinned.
>>> Is it wrong to use it on ARM by disabling CONFIG_HUGETLB_PAGE?
>>>
>>
>> It depends on why you need compaction. If it's for some device that
>> requires high-order allocations (particularly if they are atomic), then
>> it's not likely to work very well in the long term.
>
> Would you please elaborate on this as well.
>
In the case the allocation is atomic and there isn't a suitable page
available, compaction cannot occur either. Given enough uptime, there
will be failure reports as a result. Avoid high-order allocations where
possible.
--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2010-07-30 16:50 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-07-30 16:44 compaction: why depends on HUGETLB_PAGE Round Robinjp
2010-07-30 16:49 ` Mel Gorman
-- strict thread matches above, loose matches on Subject: below --
2010-07-29 1:53 Iram Shahzad
2010-07-29 12:57 ` Mel Gorman
2010-07-30 2:56 ` Iram Shahzad
2010-07-30 9:57 ` Mel Gorman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).