All of lore.kernel.org
 help / color / mirror / Atom feed
From: Bill Irwin <bill.irwin@oracle.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Bill Irwin <bill.irwin@oracle.com>, Ingo Molnar <mingo@elte.hu>,
	Nick Piggin <npiggin@suse.de>,
	Linux Memory Management <linux-mm@kvack.org>,
	Linux Kernel <linux-kernel@vger.kernel.org>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	"Paolo 'Blaisorblade' Giarrusso" <blaisorblade@yahoo.it>
Subject: Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)
Date: Wed, 7 Mar 2007 02:09:11 -0800	[thread overview]
Message-ID: <20070307100911.GO18774@holomorphy.com> (raw)
In-Reply-To: <20070307013942.5c0fadff.akpm@linux-foundation.org>

On Wed, 7 Mar 2007 01:29:03 -0800 Bill Irwin <bill.irwin@oracle.com> wrote:
>> Guess what major real-life application not only uses nonlinear daily
>> but would even be very happy to see it extended with non-vma-creating
>> protections and more?

On Wed, Mar 07, 2007 at 01:39:42AM -0800, Andrew Morton wrote:
> uh-oh.  SQL server?

Close enough. ;)


On Wed, 7 Mar 2007 01:29:03 -0800 Bill Irwin <bill.irwin@oracle.com> wrote:
>> It's not terribly typical for things to be
>> truncated while remap_file_pages() is doing its work, though it's been
>> proposed as a method of dynamism. It won't stress remap_file_pages() vs.
>> truncate() in any meaningful way, though, as userspace will be rather
>> diligent about clearing in-use data out of the file offset range to be
>> truncated away anyway, and all that via O_DIRECT.

On Wed, Mar 07, 2007 at 01:39:42AM -0800, Andrew Morton wrote:
> The problem here isn't related to truncate or direct-IO.  It's just
> plain-old MAP_SHARED.  nonlinear VMAs are now using the old-style
> dirty-memory management.  msync() is basically a no-op and the code is
> wildly tricky and pretty much untested.  The chances that we broke it are
> considerable.

This would be of concern for swapping out tmpfs-backed nonlinearly-
mapped files under extreme stress in Oracle's case, though it's rather
typical for it all to be mlock()'d in-core and cases where that's
necessary to be considered grossly underprovisioned. As far as I know,
msync() is not used to manage the nonlinearly-mapped objects, which are
most typically expected to be memory-backed, rendering writeback to
disk of questionable value. Also quite happily, I'm not aware of any
data integrity issues it would explain. Bug though it may be, it
requires a usage model very rarely used by Oracle to trigger, so we've
not run into it.


-- wli

WARNING: multiple messages have this Message-ID (diff)
From: Bill Irwin <bill.irwin@oracle.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Bill Irwin <bill.irwin@oracle.com>, Ingo Molnar <mingo@elte.hu>,
	Nick Piggin <npiggin@suse.de>,
	Linux Memory Management <linux-mm@kvack.org>,
	Linux Kernel <linux-kernel@vger.kernel.org>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Subject: Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)
Date: Wed, 7 Mar 2007 02:09:11 -0800	[thread overview]
Message-ID: <20070307100911.GO18774@holomorphy.com> (raw)
In-Reply-To: <20070307013942.5c0fadff.akpm@linux-foundation.org>

On Wed, 7 Mar 2007 01:29:03 -0800 Bill Irwin <bill.irwin@oracle.com> wrote:
>> Guess what major real-life application not only uses nonlinear daily
>> but would even be very happy to see it extended with non-vma-creating
>> protections and more?

On Wed, Mar 07, 2007 at 01:39:42AM -0800, Andrew Morton wrote:
> uh-oh.  SQL server?

Close enough. ;)


On Wed, 7 Mar 2007 01:29:03 -0800 Bill Irwin <bill.irwin@oracle.com> wrote:
>> It's not terribly typical for things to be
>> truncated while remap_file_pages() is doing its work, though it's been
>> proposed as a method of dynamism. It won't stress remap_file_pages() vs.
>> truncate() in any meaningful way, though, as userspace will be rather
>> diligent about clearing in-use data out of the file offset range to be
>> truncated away anyway, and all that via O_DIRECT.

On Wed, Mar 07, 2007 at 01:39:42AM -0800, Andrew Morton wrote:
> The problem here isn't related to truncate or direct-IO.  It's just
> plain-old MAP_SHARED.  nonlinear VMAs are now using the old-style
> dirty-memory management.  msync() is basically a no-op and the code is
> wildly tricky and pretty much untested.  The chances that we broke it are
> considerable.

This would be of concern for swapping out tmpfs-backed nonlinearly-
mapped files under extreme stress in Oracle's case, though it's rather
typical for it all to be mlock()'d in-core and cases where that's
necessary to be considered grossly underprovisioned. As far as I know,
msync() is not used to manage the nonlinearly-mapped objects, which are
most typically expected to be memory-backed, rendering writeback to
disk of questionable value. Also quite happily, I'm not aware of any
data integrity issues it would explain. Bug though it may be, it
requires a usage model very rarely used by Oracle to trigger, so we've
not run into it.


-- wli

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2007-03-07 10:09 UTC|newest]

Thread overview: 198+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-02-21  4:49 [patch 0/6] fault vs truncate/invalidate race fix Nick Piggin
2007-02-21  4:49 ` Nick Piggin
2007-02-21  4:49 ` [patch 1/6] mm: debug check for the fault vs invalidate race Nick Piggin
2007-02-21  4:49   ` Nick Piggin
2007-02-21  4:49 ` [patch 2/6] mm: simplify filemap_nopage Nick Piggin
2007-02-21  4:49   ` Nick Piggin
2007-02-21  4:50 ` [patch 3/6] mm: fix fault vs invalidate race for linear mappings Nick Piggin
2007-02-21  4:50   ` Nick Piggin
2007-03-07  6:36   ` Andrew Morton
2007-03-07  6:36     ` Andrew Morton
2007-03-07  6:57     ` Nick Piggin
2007-03-07  6:57       ` Nick Piggin
2007-03-07  7:08       ` Andrew Morton
2007-03-07  7:08         ` Andrew Morton
2007-03-07  7:25         ` Nick Piggin
2007-03-07  7:25           ` Nick Piggin
2007-02-21  4:50 ` [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear) Nick Piggin
2007-02-21  4:50   ` Nick Piggin
2007-03-07  6:51   ` Andrew Morton
2007-03-07  6:51     ` Andrew Morton
2007-03-07  7:08     ` Nick Piggin
2007-03-07  7:08       ` Nick Piggin
2007-03-07  8:19       ` Nick Piggin
2007-03-07  8:19         ` Nick Piggin
2007-03-07  8:27         ` Ingo Molnar
2007-03-07  8:27           ` Ingo Molnar
2007-03-07  8:35           ` Andrew Morton
2007-03-07  8:35             ` Andrew Morton
2007-03-07  8:53             ` Ingo Molnar
2007-03-07  8:53               ` Ingo Molnar
2007-03-07  9:28               ` Nick Piggin
2007-03-07  9:28                 ` Nick Piggin
2007-03-07  9:44                 ` Bill Irwin
2007-03-07  9:44                   ` Bill Irwin
2007-03-07  9:49                   ` Nick Piggin
2007-03-07  9:49                     ` Nick Piggin
2007-03-07 10:02                     ` Nick Piggin
2007-03-07 10:02                       ` Nick Piggin
2007-03-12 23:01                       ` Blaisorblade
2007-03-12 23:01                         ` Blaisorblade
2007-03-13  1:19                         ` Nick Piggin
2007-03-13  1:19                           ` Nick Piggin
2007-03-17 12:17                           ` Blaisorblade
2007-03-17 12:17                             ` Blaisorblade
2007-03-18  2:50                             ` Nick Piggin
2007-03-18  2:50                               ` Nick Piggin
2007-03-18 13:09                               ` Jeff Dike
2007-03-18 13:09                                 ` Jeff Dike
2007-03-19 12:04                               ` Bill Irwin
2007-03-19 12:04                                 ` Bill Irwin
2007-03-19 20:44                               ` Blaisorblade
2007-03-19 20:44                                 ` Blaisorblade
2007-03-20  6:00                                 ` Nick Piggin
2007-03-20  6:00                                   ` Nick Piggin
2007-03-21 19:45                                   ` Blaisorblade
2007-03-21 19:45                                     ` Blaisorblade
2007-03-08 12:39                   ` Blaisorblade
2007-03-08 12:39                     ` Blaisorblade
2007-03-07  9:29             ` Bill Irwin
2007-03-07  9:29               ` Bill Irwin
2007-03-07  9:39               ` Andrew Morton
2007-03-07  9:39                 ` Andrew Morton
2007-03-07 10:09                 ` Bill Irwin [this message]
2007-03-07 10:09                   ` Bill Irwin
2007-03-07  8:38           ` Miklos Szeredi
2007-03-07  8:38             ` Miklos Szeredi
2007-03-07  8:47             ` Andrew Morton
2007-03-07  8:47               ` Andrew Morton
2007-03-07  8:51               ` Miklos Szeredi
2007-03-07  8:51                 ` Miklos Szeredi
2007-03-07  9:07                 ` Andrew Morton
2007-03-07  9:07                   ` Andrew Morton
2007-03-07  9:18                   ` Nick Piggin
2007-03-07  9:18                     ` Nick Piggin
2007-03-07  9:26                     ` Andrew Morton
2007-03-07  9:26                       ` Andrew Morton
2007-03-07  9:28                       ` Miklos Szeredi
2007-03-07  9:28                         ` Miklos Szeredi
2007-03-07  9:38                       ` Nick Piggin
2007-03-07  9:38                         ` Nick Piggin
2007-03-07  9:25                   ` Miklos Szeredi
2007-03-07  9:25                     ` Miklos Szeredi
2007-03-07  9:32                   ` Peter Zijlstra
2007-03-07  9:32                     ` Peter Zijlstra
2007-03-07  9:45                     ` Nick Piggin
2007-03-07  9:45                       ` Nick Piggin
2007-03-07 10:04                       ` Nick Piggin
2007-03-07 10:04                         ` Nick Piggin
2007-03-07 10:06                         ` Peter Zijlstra
2007-03-07 10:06                           ` Peter Zijlstra
2007-03-07 10:13                           ` Miklos Szeredi
2007-03-07 10:13                             ` Miklos Szeredi
2007-03-07 10:21                             ` Nick Piggin
2007-03-07 10:21                               ` Nick Piggin
2007-03-07 10:24                               ` Peter Zijlstra
2007-03-07 10:24                                 ` Peter Zijlstra
2007-03-07 10:38                                 ` Nick Piggin
2007-03-07 10:38                                   ` Nick Piggin
2007-03-07 10:47                                   ` Peter Zijlstra
2007-03-07 10:47                                     ` Peter Zijlstra
2007-03-07 11:00                                     ` Nick Piggin
2007-03-07 11:00                                       ` Nick Piggin
2007-03-07 11:48                                       ` Peter Zijlstra
2007-03-07 11:48                                         ` Peter Zijlstra
2007-03-07 12:17                                         ` Nick Piggin
2007-03-07 12:17                                           ` Nick Piggin
2007-03-07 12:41                                           ` Peter Zijlstra
2007-03-07 12:41                                             ` Peter Zijlstra
2007-03-07 13:08                                             ` Nick Piggin
2007-03-07 13:08                                               ` Nick Piggin
2007-03-07 13:19                                               ` Peter Zijlstra
2007-03-07 13:19                                                 ` Peter Zijlstra
2007-03-07 13:36                                                 ` Nick Piggin
2007-03-07 13:36                                                   ` Nick Piggin
2007-03-07 13:52                                                   ` Peter Zijlstra
2007-03-07 13:52                                                     ` Peter Zijlstra
2007-03-07 13:56                                                     ` Miklos Szeredi
2007-03-07 13:56                                                       ` Miklos Szeredi
2007-03-07 14:34                                                     ` Peter Zijlstra
2007-03-07 14:34                                                       ` Peter Zijlstra
2007-03-07 15:01                                                       ` Nick Piggin
2007-03-07 15:01                                                         ` Nick Piggin
2007-03-07 16:58                                                         ` [RFC][PATCH] mm: fix page_mkclean() vs non-linear vmas Peter Zijlstra
2007-03-07 16:58                                                           ` Peter Zijlstra
2007-03-07 18:00                                                           ` Linus Torvalds
2007-03-07 18:00                                                             ` Linus Torvalds
2007-03-07 18:12                                                             ` Peter Zijlstra
2007-03-07 18:12                                                               ` Peter Zijlstra
2007-03-07 18:24                                                               ` Peter Zijlstra
2007-03-07 18:24                                                                 ` Peter Zijlstra
2007-03-08 11:21                                                           ` Miklos Szeredi
2007-03-08 11:21                                                             ` Miklos Szeredi
2007-03-08 11:37                                                             ` Peter Zijlstra
2007-03-08 11:37                                                               ` Peter Zijlstra
2007-03-08 11:48                                                               ` Miklos Szeredi
2007-03-08 11:48                                                                 ` Miklos Szeredi
2007-03-08 12:11                                                                 ` Peter Zijlstra
2007-03-08 12:11                                                                   ` Peter Zijlstra
2007-03-08 12:19                                                                   ` Nick Piggin
2007-03-08 12:19                                                                     ` Nick Piggin
2007-03-08 12:25                                                                     ` Miklos Szeredi
2007-03-08 12:25                                                                       ` Miklos Szeredi
2007-03-08 11:58                                                             ` Nick Piggin
2007-03-08 11:58                                                               ` Nick Piggin
2007-03-08 12:09                                                               ` Miklos Szeredi
2007-03-08 12:09                                                                 ` Miklos Szeredi
2007-03-07 15:10                                                     ` [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear) Jeff Dike
2007-03-07 15:10                                                       ` Jeff Dike
2007-03-07 13:53                                                   ` Miklos Szeredi
2007-03-07 13:53                                                     ` Miklos Szeredi
2007-03-07 14:50                                                     ` Nick Piggin
2007-03-07 14:50                                                       ` Nick Piggin
2007-03-07 12:22                                       ` Bill Irwin
2007-03-07 12:22                                         ` Bill Irwin
2007-03-07 12:36                                         ` Nick Piggin
2007-03-07 12:36                                           ` Nick Piggin
2007-03-07 10:30                             ` [rfc][patch 7/6] mm: merge page_mkwrite Nick Piggin
2007-03-07 10:30                               ` Nick Piggin
2007-03-07  8:59           ` [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear) Nick Piggin
2007-03-07  8:59             ` Nick Piggin
2007-03-07  9:11             ` Nick Piggin
2007-03-07  9:11               ` Nick Piggin
2007-03-07  9:22             ` Ingo Molnar
2007-03-07  9:22               ` Ingo Molnar
2007-03-07  9:32               ` Bill Irwin
2007-03-07  9:32                 ` Bill Irwin
2007-03-07  9:35                 ` Ingo Molnar
2007-03-07  9:35                   ` Ingo Molnar
2007-03-07  9:50                   ` Bill Irwin
2007-03-07  9:50                     ` Bill Irwin
2007-03-07  9:52               ` Nick Piggin
2007-03-07  9:52                 ` Nick Piggin
2007-03-07  7:19     ` Bill Irwin
2007-03-07  7:19       ` Bill Irwin
2007-03-07 10:05     ` Benjamin Herrenschmidt
2007-03-07 10:05       ` Benjamin Herrenschmidt
2007-03-07 10:17       ` Nick Piggin
2007-03-07 10:17         ` Nick Piggin
2007-03-07 10:46         ` Benjamin Herrenschmidt
2007-03-07 10:46           ` Benjamin Herrenschmidt
2007-02-21  4:50 ` [patch 5/6] mm: merge nopfn into fault Nick Piggin
2007-02-21  4:50   ` Nick Piggin
2007-02-21  5:13   ` Nick Piggin
2007-02-21  5:13     ` Nick Piggin
2007-02-21  4:50 ` [patch 6/6] mm: remove legacy cruft Nick Piggin
2007-02-21  4:50   ` Nick Piggin
2007-02-27  4:36 ` [patch 0/6] fault vs truncate/invalidate race fix Dave Airlie
2007-02-27  4:36   ` Dave Airlie
2007-02-27  5:32   ` Andrew Morton
2007-02-27  5:32     ` Andrew Morton
2007-02-27  6:26     ` Dave Airlie
2007-02-27  6:26       ` Dave Airlie
2007-02-27  6:54       ` Benjamin Herrenschmidt
2007-02-27  6:54         ` Benjamin Herrenschmidt
2007-03-18 23:13         ` Dave Airlie
2007-03-18 23:13           ` Dave Airlie
2007-02-27  8:50     ` Nick Piggin
2007-02-27  8:50       ` Nick Piggin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20070307100911.GO18774@holomorphy.com \
    --to=bill.irwin@oracle.com \
    --cc=akpm@linux-foundation.org \
    --cc=benh@kernel.crashing.org \
    --cc=blaisorblade@yahoo.it \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mingo@elte.hu \
    --cc=npiggin@suse.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.