From: Borislav Petkov <bp@alien8.de>
To: Dave Hansen <dave@linux.vnet.ibm.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
Andrew Morton <akpm@linux-foundation.org>,
Michal Hocko <mhocko@suse.cz>,
linux-mm@kvack.org,
KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] add some drop_caches documentation and info messsge
Date: Thu, 25 Oct 2012 16:24:57 +0200 [thread overview]
Message-ID: <20121025142457.GB308@x1.osrc.amd.com> (raw)
In-Reply-To: <50892917.30201@linux.vnet.ibm.com>
On Thu, Oct 25, 2012 at 04:57:11AM -0700, Dave Hansen wrote:
> On 10/25/2012 02:24 AM, Borislav Petkov wrote:
> > But let's discuss this a bit further. So, for the benchmarking aspect,
> > you're either going to have to always require dmesg along with
> > benchmarking results or /proc/vmstat, depending on where the drop_caches
> > stats end up.
> >
> > Is this how you envision it?
> >
> > And then there are the VM bug cases, where you might not always get
> > full dmesg from a panicked system. In that case, you'd want the kernel
> > tainting thing too, so that it at least appears in the oops backtrace.
> >
> > Although the tainting thing might not be enough - a user could
> > drop_caches at some point in time and the oops happening much later
> > could be unrelated but that can't be expressed in taint flags.
>
> Here's the problem: Joe Kernel Developer gets a bug report, usually
> something like "the kernel is slow", or "the kernel is eating up all my
> memory". We then start going and digging in to the problem with the
> usual tools. We almost *ALWAYS* get dmesg, and it's reasonably common,
> but less likely, that we get things like vmstat along with such a bug
> report.
>
> Joe Kernel Developer digs in the statistics or the dmesg and tries to
> figure out what happened. I've run in to a couple of cases in practice
> (and I assume Michal has too) where the bug reporter was using
> drop_caches _heavily_ and did not realize the implications. It was
> quite hard to track down exactly how the page cache and dentries/inodes
> were getting purged.
>
> There are rarely oopses involved in these scenarios.
>
> The primary goal of this patch is to make debugging those scenarios
> easier so that we can quickly realize that drop_caches is the reason our
> caches went away, not some anomalous VM activity. A secondary goal is
> to tell the user: "Hey, maybe this isn't something you want to be doing
> all the time."
Ok, understood. So you will be requiring dmesg, ok, then it makes sense.
This way you're also getting timestamps of when exactly and how many
times drop_caches was used. For that, though, you'll need to add the
timestamp explicitly to the printk because CONFIG_PRINTK_TIME is not
always enabled.
Thanks.
--
Regards/Gruss,
Boris.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Borislav Petkov <bp@alien8.de>
To: Dave Hansen <dave@linux.vnet.ibm.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
Andrew Morton <akpm@linux-foundation.org>,
Michal Hocko <mhocko@suse.cz>,
linux-mm@kvack.org,
KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] add some drop_caches documentation and info messsge
Date: Thu, 25 Oct 2012 16:24:57 +0200 [thread overview]
Message-ID: <20121025142457.GB308@x1.osrc.amd.com> (raw)
In-Reply-To: <50892917.30201@linux.vnet.ibm.com>
On Thu, Oct 25, 2012 at 04:57:11AM -0700, Dave Hansen wrote:
> On 10/25/2012 02:24 AM, Borislav Petkov wrote:
> > But let's discuss this a bit further. So, for the benchmarking aspect,
> > you're either going to have to always require dmesg along with
> > benchmarking results or /proc/vmstat, depending on where the drop_caches
> > stats end up.
> >
> > Is this how you envision it?
> >
> > And then there are the VM bug cases, where you might not always get
> > full dmesg from a panicked system. In that case, you'd want the kernel
> > tainting thing too, so that it at least appears in the oops backtrace.
> >
> > Although the tainting thing might not be enough - a user could
> > drop_caches at some point in time and the oops happening much later
> > could be unrelated but that can't be expressed in taint flags.
>
> Here's the problem: Joe Kernel Developer gets a bug report, usually
> something like "the kernel is slow", or "the kernel is eating up all my
> memory". We then start going and digging in to the problem with the
> usual tools. We almost *ALWAYS* get dmesg, and it's reasonably common,
> but less likely, that we get things like vmstat along with such a bug
> report.
>
> Joe Kernel Developer digs in the statistics or the dmesg and tries to
> figure out what happened. I've run in to a couple of cases in practice
> (and I assume Michal has too) where the bug reporter was using
> drop_caches _heavily_ and did not realize the implications. It was
> quite hard to track down exactly how the page cache and dentries/inodes
> were getting purged.
>
> There are rarely oopses involved in these scenarios.
>
> The primary goal of this patch is to make debugging those scenarios
> easier so that we can quickly realize that drop_caches is the reason our
> caches went away, not some anomalous VM activity. A secondary goal is
> to tell the user: "Hey, maybe this isn't something you want to be doing
> all the time."
Ok, understood. So you will be requiring dmesg, ok, then it makes sense.
This way you're also getting timestamps of when exactly and how many
times drop_caches was used. For that, though, you'll need to add the
timestamp explicitly to the printk because CONFIG_PRINTK_TIME is not
always enabled.
Thanks.
--
Regards/Gruss,
Boris.
next prev parent reply other threads:[~2012-10-25 14:24 UTC|newest]
Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-12 12:57 [PATCH] add some drop_caches documentation and info messsge Michal Hocko
2012-10-12 12:57 ` Michal Hocko
2012-10-12 18:54 ` KOSAKI Motohiro
2012-10-12 18:54 ` KOSAKI Motohiro
2012-10-15 9:04 ` Kamezawa Hiroyuki
2012-10-15 9:04 ` Kamezawa Hiroyuki
2012-10-15 14:05 ` Dave Hansen
2012-10-15 14:05 ` Dave Hansen
2012-10-23 23:45 ` Andrew Morton
2012-10-23 23:45 ` Andrew Morton
2012-10-24 6:29 ` Michal Hocko
2012-10-24 6:29 ` Michal Hocko
2012-10-24 19:54 ` Andrew Morton
2012-10-24 19:54 ` Andrew Morton
2012-10-24 20:28 ` Dave Hansen
2012-10-24 20:28 ` Dave Hansen
2012-10-24 20:48 ` Andrew Morton
2012-10-24 20:48 ` Andrew Morton
2012-10-24 21:06 ` Borislav Petkov
2012-10-24 21:06 ` Borislav Petkov
2012-10-24 21:13 ` Andrew Morton
2012-10-24 21:13 ` Andrew Morton
2012-10-24 22:04 ` Rafael J. Wysocki
2012-10-24 22:04 ` Rafael J. Wysocki
2012-10-25 1:17 ` Andrew Morton
2012-10-25 1:17 ` Andrew Morton
2012-10-25 20:16 ` Rafael J. Wysocki
2012-10-25 20:16 ` Rafael J. Wysocki
2012-10-29 8:59 ` Jiri Kosina
2012-10-29 8:59 ` Jiri Kosina
2012-10-29 9:58 ` Borislav Petkov
2012-10-29 9:58 ` Borislav Petkov
2012-10-29 10:01 ` Jiri Kosina
2012-10-29 10:01 ` Jiri Kosina
2012-10-29 10:11 ` Borislav Petkov
2012-10-29 10:11 ` Borislav Petkov
2012-10-31 17:31 ` Pavel Machek
2012-10-31 17:31 ` Pavel Machek
2012-10-31 17:46 ` Borislav Petkov
2012-10-31 17:46 ` Borislav Petkov
2012-10-24 22:35 ` KOSAKI Motohiro
2012-10-24 22:35 ` KOSAKI Motohiro
2012-10-25 14:21 ` Michal Hocko
2012-10-25 14:21 ` Michal Hocko
2012-10-24 21:18 ` Dave Hansen
2012-10-24 21:18 ` Dave Hansen
2012-10-24 22:48 ` Borislav Petkov
2012-10-24 22:48 ` Borislav Petkov
2012-10-24 22:57 ` Dave Hansen
2012-10-24 22:57 ` Dave Hansen
2012-10-25 0:56 ` KOSAKI Motohiro
2012-10-25 0:56 ` KOSAKI Motohiro
2012-10-25 9:24 ` Borislav Petkov
2012-10-25 9:24 ` Borislav Petkov
2012-10-25 11:57 ` Dave Hansen
2012-10-25 11:57 ` Dave Hansen
2012-10-25 14:24 ` Borislav Petkov [this message]
2012-10-25 14:24 ` Borislav Petkov
2012-10-25 14:25 ` Michal Hocko
2012-10-25 14:25 ` Michal Hocko
2012-10-26 7:45 ` Mika Boström
2012-10-26 7:45 ` Mika Boström
2012-11-01 20:26 ` Pavel Machek
2012-11-01 20:26 ` Pavel Machek
2012-10-25 14:09 ` Michal Hocko
2012-10-25 14:09 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121025142457.GB308@x1.osrc.amd.com \
--to=bp@alien8.de \
--cc=akpm@linux-foundation.org \
--cc=dave@linux.vnet.ibm.com \
--cc=kamezawa.hiroyu@jp.fujitsu.com \
--cc=kosaki.motohiro@jp.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.