All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: Miklos Szeredi <miklos@szeredi.hu>
Cc: a.p.zijlstra@chello.nl, linux-mm@kvack.org,
	linux-kernel@vger.kernel.org, neilb@suse.de, dgc@sgi.com,
	tomoki.sekiyama.qu@hitachi.com, nikita@clusterfs.com,
	trond.myklebust@fys.uio.no, yingchao.zhou@gmail.com
Subject: Re: [PATCH 10/10] mm: per device dirty threshold
Date: Sun, 22 Apr 2007 23:39:36 -0700	[thread overview]
Message-ID: <20070422233936.97d78677.akpm@linux-foundation.org> (raw)
In-Reply-To: <E1Hfs3j-0005sN-00@dorka.pomaz.szeredi.hu>

On Mon, 23 Apr 2007 08:29:59 +0200 Miklos Szeredi <miklos@szeredi.hu> wrote:

> > > What about swapout?  That can increase the number of writeback pages,
> > > without decreasing the number of dirty pages, no?
> > 
> > Could we not solve that by enabling cap_account_writeback on
> > swapper_space, and thereby account swap writeback pages. Then the VM
> > knows it has outstanding IO and need not panic.
> 
> Hmm, I'm not sure that would be right, because then those writeback
> pages would be accounted twice: once for swapper_space, and once for
> the real device.
> 
> So there's a condition, when lots of anonymous pages are turned into
> swap-cache writeback pages, and we should somehow throttle this, because
> 
> >>>     This means that all memory is pinned and unreclaimable and the VM gets
> >>>     upset and goes oom.
> 
> although, it's not quite clear in my mind, how the VM gets upset about
> this.

I've been scratching my head on and off for a couple of days over this.

We've traditionally had reclaim problems when there's a huge amount of
dirty MAP_SHARED data, which the VM didn't know was dirty.  It's the old
"map a file which is the same size as physical memory and write to it all"
stresstest.

But we do not have such problems with anonymous memory, and I'm darned if I
can remember why :(


WARNING: multiple messages have this Message-ID (diff)
From: Andrew Morton <akpm@linux-foundation.org>
To: Miklos Szeredi <miklos@szeredi.hu>
Cc: a.p.zijlstra@chello.nl, linux-mm@kvack.org,
	linux-kernel@vger.kernel.org, neilb@suse.de, dgc@sgi.com,
	tomoki.sekiyama.qu@hitachi.com, nikita@clusterfs.com,
	trond.myklebust@fys.uio.no, yingchao.zhou@gmail.com
Subject: Re: [PATCH 10/10] mm: per device dirty threshold
Date: Sun, 22 Apr 2007 23:39:36 -0700	[thread overview]
Message-ID: <20070422233936.97d78677.akpm@linux-foundation.org> (raw)
In-Reply-To: <E1Hfs3j-0005sN-00@dorka.pomaz.szeredi.hu>

On Mon, 23 Apr 2007 08:29:59 +0200 Miklos Szeredi <miklos@szeredi.hu> wrote:

> > > What about swapout?  That can increase the number of writeback pages,
> > > without decreasing the number of dirty pages, no?
> > 
> > Could we not solve that by enabling cap_account_writeback on
> > swapper_space, and thereby account swap writeback pages. Then the VM
> > knows it has outstanding IO and need not panic.
> 
> Hmm, I'm not sure that would be right, because then those writeback
> pages would be accounted twice: once for swapper_space, and once for
> the real device.
> 
> So there's a condition, when lots of anonymous pages are turned into
> swap-cache writeback pages, and we should somehow throttle this, because
> 
> >>>     This means that all memory is pinned and unreclaimable and the VM gets
> >>>     upset and goes oom.
> 
> although, it's not quite clear in my mind, how the VM gets upset about
> this.

I've been scratching my head on and off for a couple of days over this.

We've traditionally had reclaim problems when there's a huge amount of
dirty MAP_SHARED data, which the VM didn't know was dirty.  It's the old
"map a file which is the same size as physical memory and write to it all"
stresstest.

But we do not have such problems with anonymous memory, and I'm darned if I
can remember why :(

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2007-04-23  6:41 UTC|newest]

Thread overview: 110+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-04-20 15:51 [PATCH 00/10] per device dirty throttling -v5 Peter Zijlstra
2007-04-20 15:51 ` Peter Zijlstra
2007-04-20 15:51 ` [PATCH 01/10] revert per-backing_dev-dirty-and-writeback-page-accounting Peter Zijlstra
2007-04-20 15:51   ` Peter Zijlstra
2007-04-20 15:51 ` [PATCH 02/10] nfs: remove congestion_end() Peter Zijlstra
2007-04-20 15:51   ` Peter Zijlstra
2007-04-20 15:51 ` [PATCH 03/10] lib: dampen the percpu_counter FBC_BATCH Peter Zijlstra
2007-04-20 15:51   ` Peter Zijlstra
2007-04-21  9:55   ` Andrew Morton
2007-04-21  9:55     ` Andrew Morton
2007-04-21 10:58     ` Peter Zijlstra
2007-04-21 10:58       ` Peter Zijlstra
2007-04-20 15:51 ` [PATCH 04/10] lib: percpu_counter_mod64 Peter Zijlstra
2007-04-20 15:51   ` Peter Zijlstra
2007-04-21  9:55   ` Andrew Morton
2007-04-21  9:55     ` Andrew Morton
2007-04-21 11:02     ` Peter Zijlstra
2007-04-21 11:02       ` Peter Zijlstra
2007-04-21 19:21       ` Andrew Morton
2007-04-21 19:21         ` Andrew Morton
2007-04-21 19:30         ` Peter Zijlstra
2007-04-21 19:30           ` Peter Zijlstra
2007-04-20 15:51 ` [PATCH 05/10] mm: bdi init hooks Peter Zijlstra
2007-04-20 15:51   ` Peter Zijlstra
2007-04-20 15:52 ` [PATCH 06/10] mm: scalable bdi statistics counters Peter Zijlstra
2007-04-20 15:52   ` Peter Zijlstra
2007-04-20 15:52 ` [PATCH 07/10] mm: count reclaimable pages per BDI Peter Zijlstra
2007-04-20 15:52   ` Peter Zijlstra
2007-04-21  9:55   ` Andrew Morton
2007-04-21  9:55     ` Andrew Morton
2007-04-21 11:04     ` Peter Zijlstra
2007-04-21 11:04       ` Peter Zijlstra
2007-04-20 15:52 ` [PATCH 08/10] mm: count writeback " Peter Zijlstra
2007-04-20 15:52   ` Peter Zijlstra
2007-04-21  9:55   ` Andrew Morton
2007-04-21  9:55     ` Andrew Morton
2007-04-21 11:07     ` Peter Zijlstra
2007-04-21 11:07       ` Peter Zijlstra
2007-04-22  7:19       ` Andrew Morton
2007-04-22  7:19         ` Andrew Morton
2007-04-22  9:08         ` Peter Zijlstra
2007-04-22  9:08           ` Peter Zijlstra
2007-04-20 15:52 ` [PATCH 09/10] mm: expose BDI statistics in sysfs Peter Zijlstra
2007-04-20 15:52   ` Peter Zijlstra
2007-04-21  9:55   ` Andrew Morton
2007-04-21  9:55     ` Andrew Morton
2007-04-21 11:08     ` Peter Zijlstra
2007-04-21 11:08       ` Peter Zijlstra
2007-04-20 15:52 ` [PATCH 10/10] mm: per device dirty threshold Peter Zijlstra
2007-04-20 15:52   ` Peter Zijlstra
2007-04-21  9:55   ` Andrew Morton
2007-04-21  9:55     ` Andrew Morton
2007-04-21 10:38     ` Miklos Szeredi
2007-04-21 10:38       ` Miklos Szeredi
2007-04-21 10:54       ` Andrew Morton
2007-04-21 10:54         ` Andrew Morton
2007-04-21 20:25         ` Miklos Szeredi
2007-04-21 20:25           ` Miklos Szeredi
2007-04-23  6:14           ` Peter Zijlstra
2007-04-23  6:14             ` Peter Zijlstra
2007-04-23  6:29             ` Miklos Szeredi
2007-04-23  6:29               ` Miklos Szeredi
2007-04-23  6:39               ` Andrew Morton [this message]
2007-04-23  6:39                 ` Andrew Morton
2007-04-21 12:01     ` Peter Zijlstra
2007-04-21 12:01       ` Peter Zijlstra
2007-04-21 12:15       ` Peter Zijlstra
2007-04-21 12:15         ` Peter Zijlstra
2007-04-21 19:50         ` Peter Zijlstra
2007-04-21 19:50           ` Peter Zijlstra
2007-04-23 15:48         ` Christoph Lameter
2007-04-23 15:48           ` Christoph Lameter
2007-04-23 15:58           ` Peter Zijlstra
2007-04-23 15:58             ` Peter Zijlstra
2007-04-23 16:08             ` Christoph Lameter
2007-04-23 16:08               ` Christoph Lameter
2007-04-22  7:26       ` Andrew Morton
2007-04-22  7:26         ` Andrew Morton
2007-04-24  2:58   ` Neil Brown
2007-04-24  2:58     ` Neil Brown
2007-04-24  7:09     ` Peter Zijlstra
2007-04-24  7:09       ` Peter Zijlstra
2007-04-24  8:19       ` Miklos Szeredi
2007-04-24  8:19         ` Miklos Szeredi
2007-04-24  8:31         ` Peter Zijlstra
2007-04-24  8:31           ` Peter Zijlstra
2007-04-24  9:14           ` Miklos Szeredi
2007-04-24  9:14             ` Miklos Szeredi
2007-04-24  9:26             ` Peter Zijlstra
2007-04-24  9:26               ` Peter Zijlstra
2007-04-24  9:47               ` Miklos Szeredi
2007-04-24  9:47                 ` Miklos Szeredi
2007-04-24 10:00                 ` Andrew Morton
2007-04-24 10:00                   ` Andrew Morton
2007-04-24 10:12                   ` Peter Zijlstra
2007-04-24 10:12                     ` Peter Zijlstra
2007-04-24 10:19                     ` Miklos Szeredi
2007-04-24 10:19                       ` Miklos Szeredi
2007-04-24 10:24                       ` Peter Zijlstra
2007-04-24 10:24                         ` Peter Zijlstra
2007-04-24 10:40                     ` Andrew Morton
2007-04-24 10:40                       ` Andrew Morton
2007-04-24 11:22                       ` Miklos Szeredi
2007-04-24 11:22                         ` Miklos Szeredi
2007-04-24 11:50                         ` Andrew Morton
2007-04-24 11:50                           ` Andrew Morton
2007-04-24 12:07                           ` Miklos Szeredi
2007-04-24 12:07                             ` Miklos Szeredi
2007-04-22  9:57 ` [PATCH 00/10] per device dirty throttling -v5 Andrew Morton
2007-04-22  9:57   ` Andrew Morton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20070422233936.97d78677.akpm@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=a.p.zijlstra@chello.nl \
    --cc=dgc@sgi.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=miklos@szeredi.hu \
    --cc=neilb@suse.de \
    --cc=nikita@clusterfs.com \
    --cc=tomoki.sekiyama.qu@hitachi.com \
    --cc=trond.myklebust@fys.uio.no \
    --cc=yingchao.zhou@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.