All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andrea Arcangeli <aarcange@redhat.com>
To: Dave Hansen <dave@linux.vnet.ibm.com>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	Michael J Wolf <mjwolf@us.ibm.com>,
	Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [RFC][PATCH 0/6] more detailed per-process transparent hugepage statistics
Date: Tue, 8 Feb 2011 19:17:09 +0100	[thread overview]
Message-ID: <20110208181709.GL3347@random.random> (raw)
In-Reply-To: <1297187674.6737.12145.camel@nimitz>

Hello,

On Tue, Feb 08, 2011 at 09:54:34AM -0800, Dave Hansen wrote:
> Just FYI, I did some profiling on a workload that constantly split and
> joined pages.  Very little of the overhead was in the scanning itself,
> so I think you're dead-on here.

Yep, my way to deduce it has been to set both to 100%, and check the
rate of increase of
/sys/kernel/mm/transparent_hugepage/khugepaged/full_scans vs
/sys/kernel/mm/ksm/full_scans and the differences is enormous. So a
100% CPU ksmd scan can probably be followed more than well with a 1%
CPU khugepaged scan and probably achieve the exact same hugepage ratio
of a 100% khugepaged scan. The default khugepaged scan is super
paranoid (it has to be, considering the default ksm scan is
zero). Maybe we can still increase the default pages_to_scan a bit. I
suspect most of the current cost should be in the scheduler and that
only accounts for 1 kthread schedule event every 10 sec.

WARNING: multiple messages have this Message-ID (diff)
From: Andrea Arcangeli <aarcange@redhat.com>
To: Dave Hansen <dave@linux.vnet.ibm.com>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	Michael J Wolf <mjwolf@us.ibm.com>,
	Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [RFC][PATCH 0/6] more detailed per-process transparent hugepage statistics
Date: Tue, 8 Feb 2011 19:17:09 +0100	[thread overview]
Message-ID: <20110208181709.GL3347@random.random> (raw)
In-Reply-To: <1297187674.6737.12145.camel@nimitz>

Hello,

On Tue, Feb 08, 2011 at 09:54:34AM -0800, Dave Hansen wrote:
> Just FYI, I did some profiling on a workload that constantly split and
> joined pages.  Very little of the overhead was in the scanning itself,
> so I think you're dead-on here.

Yep, my way to deduce it has been to set both to 100%, and check the
rate of increase of
/sys/kernel/mm/transparent_hugepage/khugepaged/full_scans vs
/sys/kernel/mm/ksm/full_scans and the differences is enormous. So a
100% CPU ksmd scan can probably be followed more than well with a 1%
CPU khugepaged scan and probably achieve the exact same hugepage ratio
of a 100% khugepaged scan. The default khugepaged scan is super
paranoid (it has to be, considering the default ksm scan is
zero). Maybe we can still increase the default pages_to_scan a bit. I
suspect most of the current cost should be in the scheduler and that
only accounts for 1 kthread schedule event every 10 sec.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2011-02-08 18:17 UTC|newest]

Thread overview: 76+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-02-01  0:33 [RFC][PATCH 0/6] more detailed per-process transparent hugepage statistics Dave Hansen
2011-02-01  0:33 ` Dave Hansen
2011-02-01  0:33 ` [RFC][PATCH 1/6] count transparent hugepage splits Dave Hansen
2011-02-01  0:33   ` Dave Hansen
2011-02-01  9:58   ` Johannes Weiner
2011-02-01  9:58     ` Johannes Weiner
2011-02-03 21:22   ` David Rientjes
2011-02-03 21:22     ` David Rientjes
2011-02-04 21:18     ` Andrea Arcangeli
2011-02-04 21:18       ` Andrea Arcangeli
2011-02-04 21:28       ` Dave Hansen
2011-02-04 21:28         ` Dave Hansen
2011-02-01  0:33 ` [RFC][PATCH 2/6] pagewalk: only split huge pages when necessary Dave Hansen
2011-02-01  0:33   ` Dave Hansen
2011-02-01 10:04   ` Johannes Weiner
2011-02-01 10:04     ` Johannes Weiner
2011-02-01 15:03     ` Dave Hansen
2011-02-01 15:03       ` Dave Hansen
2011-02-03 21:22   ` David Rientjes
2011-02-03 21:22     ` David Rientjes
2011-02-03 21:33     ` Dave Hansen
2011-02-03 21:33       ` Dave Hansen
2011-02-03 21:46       ` David Rientjes
2011-02-03 21:46         ` David Rientjes
2011-02-04 17:19         ` Dave Hansen
2011-02-04 17:19           ` Dave Hansen
2011-02-04 21:10           ` Andrea Arcangeli
2011-02-04 21:10             ` Andrea Arcangeli
2011-02-01  0:34 ` [RFC][PATCH 3/6] break out smaps_pte_entry() from smaps_pte_range() Dave Hansen
2011-02-01  0:34   ` Dave Hansen
2011-02-01 10:08   ` Johannes Weiner
2011-02-01 10:08     ` Johannes Weiner
2011-02-03 21:22   ` David Rientjes
2011-02-03 21:22     ` David Rientjes
2011-02-03 21:40     ` Dave Hansen
2011-02-03 21:40       ` Dave Hansen
2011-02-01  0:34 ` [RFC][PATCH 4/6] pass pte size argument in to smaps_pte_entry() Dave Hansen
2011-02-01  0:34   ` Dave Hansen
2011-02-01 10:09   ` Johannes Weiner
2011-02-01 10:09     ` Johannes Weiner
2011-02-03 21:22   ` David Rientjes
2011-02-03 21:22     ` David Rientjes
2011-02-01  0:34 ` [RFC][PATCH 5/6] teach smaps_pte_range() about THP pmds Dave Hansen
2011-02-01  0:34   ` Dave Hansen
2011-02-01 10:11   ` Johannes Weiner
2011-02-01 10:11     ` Johannes Weiner
2011-02-01 15:02     ` Dave Hansen
2011-02-01 15:02       ` Dave Hansen
2011-02-01 16:09       ` Andrea Arcangeli
2011-02-01 16:09         ` Andrea Arcangeli
2011-02-03 21:22   ` David Rientjes
2011-02-03 21:22     ` David Rientjes
2011-02-03 21:34     ` Dave Hansen
2011-02-03 21:34       ` Dave Hansen
2011-02-01  0:34 ` [RFC][PATCH 6/6] have smaps show transparent huge pages Dave Hansen
2011-02-01  0:34   ` Dave Hansen
2011-02-01 10:12   ` Johannes Weiner
2011-02-01 10:12     ` Johannes Weiner
2011-02-03 21:22   ` David Rientjes
2011-02-03 21:22     ` David Rientjes
2011-02-01 15:38 ` [RFC][PATCH 0/6] more detailed per-process transparent hugepage statistics Andrea Arcangeli
2011-02-01 15:38   ` Andrea Arcangeli
2011-02-01 17:15   ` Dave Hansen
2011-02-01 17:15     ` Dave Hansen
2011-02-01 20:39     ` Andrea Arcangeli
2011-02-01 20:39       ` Andrea Arcangeli
2011-02-01 20:56       ` Dave Hansen
2011-02-01 20:56         ` Dave Hansen
2011-02-02  0:07         ` Andrea Arcangeli
2011-02-02  0:07           ` Andrea Arcangeli
2011-02-08 17:54           ` Dave Hansen
2011-02-08 17:54             ` Dave Hansen
2011-02-08 18:17             ` Andrea Arcangeli [this message]
2011-02-08 18:17               ` Andrea Arcangeli
2011-02-03 21:54 ` David Rientjes
2011-02-03 21:54   ` David Rientjes

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110208181709.GL3347@random.random \
    --to=aarcange@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=dave@linux.vnet.ibm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mjwolf@us.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.