All of lore.kernel.org
 help / color / mirror / Atom feed
From: Nick Piggin <nickpiggin@yahoo.com.au>
To: Jan Kara <jack@suse.cz>
Cc: Ying Han <yinghan@google.com>,
	"Martin J. Bligh" <mbligh@mbligh.org>,
	linux-ext4@vger.kernel.org,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	"linux-kernel" <linux-kernel@vger.kernel.org>,
	"linux-mm" <linux-mm@kvack.org>,
	guichaz@gmail.com, Alex Khesin <alexk@google.com>,
	Mike Waychison <mikew@google.com>,
	Rohit Seth <rohitseth@google.com>,
	Peter Zijlstra <a.p.zijlstra@chello.nl>
Subject: Re: ftruncate-mmap: pages are lost after writing to mmaped file.
Date: Fri, 3 Apr 2009 02:51:20 +1100	[thread overview]
Message-ID: <200904030251.22197.nickpiggin@yahoo.com.au> (raw)
In-Reply-To: <20090402113400.GC3010@duck.suse.cz>

On Thursday 02 April 2009 22:34:01 Jan Kara wrote:
> On Thu 02-04-09 22:24:29, Nick Piggin wrote:
> > On Thursday 02 April 2009 09:36:13 Ying Han wrote:
> > > Hi Jan:
> > >     I feel that the problem you saw is kind of differnt than mine. As
> > > you mentioned that you saw the PageError() message, which i don't see
> > > it on my system. I tried you patch(based on 2.6.21) on my system and
> > > it runs ok for 2 days, Still, since i don't see the same error message
> > > as you saw, i am not convineced this is the root cause at least for
> > > our problem. I am still looking into it.
> > >     So, are you seeing the PageError() every time the problem happened?
> > 
> > So I asked if you could test with my workaround of taking truncate_mutex
> > at the start of ext2_get_blocks, and report back. I never heard of any
> > response after that.
> > 
> > To reiterate: I was able to reproduce a problem with ext2 (I was testing
> > on brd to get IO rates high enough to reproduce it quite frequently).
> > I think I narrowed the problem down to block allocation or inode block
> > tree corruption because I was unable to reproduce it with that hack in
> > place.
>   Nick, what load did you use for reproduction? I'll try to reproduce it
> here so that I can debug ext2...

OK, I set up the filesystem like this:

modprobe rd rd_size=$[3*1024*1024]   #almost fill memory so we reclaim buffers
dd if=/dev/zero of=/dev/ram0 bs=4k   #prefill brd so we don't get alloc deadlock
mkfs.ext2 -b1024 /dev/ram0           #1K buffers

Test is basically unmodified except I use 64MB files, and start 8 of them
at once to (8 core system, so improve chances of hitting the bug). Although I
do see it with only 1 running it takes longer to trigger.

I also run a loop doing 'sync ; echo 3 > /proc/sys/vm/drop_caches' but I don't
know if that really helps speed up reproducing it. It is quite random to hit,
but I was able to hit it IIRC in under a minute with that setup.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Nick Piggin <nickpiggin@yahoo.com.au>
To: Jan Kara <jack@suse.cz>
Cc: Ying Han <yinghan@google.com>,
	"Martin J. Bligh" <mbligh@mbligh.org>,
	linux-ext4@vger.kernel.org,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	"linux-kernel" <linux-kernel@vger.kernel.org>,
	"linux-mm" <linux-mm@kvack.org>,
	guichaz@gmail.com, Alex Khesin <alexk@google.com>,
	Mike Waychison <mikew@google.com>,
	Rohit Seth <rohitseth@google.com>,
	Peter Zijlstra <a.p.zijlstra@chello.nl>
Subject: Re: ftruncate-mmap: pages are lost after writing to mmaped file.
Date: Fri, 3 Apr 2009 02:51:20 +1100	[thread overview]
Message-ID: <200904030251.22197.nickpiggin@yahoo.com.au> (raw)
In-Reply-To: <20090402113400.GC3010@duck.suse.cz>

On Thursday 02 April 2009 22:34:01 Jan Kara wrote:
> On Thu 02-04-09 22:24:29, Nick Piggin wrote:
> > On Thursday 02 April 2009 09:36:13 Ying Han wrote:
> > > Hi Jan:
> > >     I feel that the problem you saw is kind of differnt than mine. As
> > > you mentioned that you saw the PageError() message, which i don't see
> > > it on my system. I tried you patch(based on 2.6.21) on my system and
> > > it runs ok for 2 days, Still, since i don't see the same error message
> > > as you saw, i am not convineced this is the root cause at least for
> > > our problem. I am still looking into it.
> > >     So, are you seeing the PageError() every time the problem happened?
> > 
> > So I asked if you could test with my workaround of taking truncate_mutex
> > at the start of ext2_get_blocks, and report back. I never heard of any
> > response after that.
> > 
> > To reiterate: I was able to reproduce a problem with ext2 (I was testing
> > on brd to get IO rates high enough to reproduce it quite frequently).
> > I think I narrowed the problem down to block allocation or inode block
> > tree corruption because I was unable to reproduce it with that hack in
> > place.
>   Nick, what load did you use for reproduction? I'll try to reproduce it
> here so that I can debug ext2...

OK, I set up the filesystem like this:

modprobe rd rd_size=$[3*1024*1024]   #almost fill memory so we reclaim buffers
dd if=/dev/zero of=/dev/ram0 bs=4k   #prefill brd so we don't get alloc deadlock
mkfs.ext2 -b1024 /dev/ram0           #1K buffers

Test is basically unmodified except I use 64MB files, and start 8 of them
at once to (8 core system, so improve chances of hitting the bug). Although I
do see it with only 1 running it takes longer to trigger.

I also run a loop doing 'sync ; echo 3 > /proc/sys/vm/drop_caches' but I don't
know if that really helps speed up reproducing it. It is quite random to hit,
but I was able to hit it IIRC in under a minute with that setup.

  reply	other threads:[~2009-04-02 15:51 UTC|newest]

Thread overview: 121+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-03-18 19:44 ftruncate-mmap: pages are lost after writing to mmaped file Ying Han
2009-03-18 19:44 ` Ying Han
2009-03-18 22:11 ` Andrew Morton
2009-03-18 22:11   ` Andrew Morton
2009-03-18 22:40   ` Linus Torvalds
2009-03-18 22:40     ` Linus Torvalds
2009-03-18 23:18     ` Ying Han
2009-03-18 23:18       ` Ying Han
2009-03-18 23:36       ` Linus Torvalds
2009-03-18 23:36         ` Linus Torvalds
2009-03-18 23:54         ` Ying Han
2009-03-18 23:54           ` Ying Han
2009-03-19 15:48           ` Nick Piggin
2009-03-19 15:48             ` Nick Piggin
2009-03-19 16:16             ` Peter Zijlstra
2009-03-19 16:16               ` Peter Zijlstra
2009-03-19 16:36               ` Nick Piggin
2009-03-19 16:36                 ` Nick Piggin
2009-03-19 16:20             ` Linus Torvalds
2009-03-19 16:20               ` Linus Torvalds
2009-03-19 16:34               ` Nick Piggin
2009-03-19 16:34                 ` Nick Piggin
2009-03-19 16:51                 ` Linus Torvalds
2009-03-19 16:51                   ` Linus Torvalds
2009-03-19 17:03                   ` Jan Kara
2009-03-19 17:03                     ` Jan Kara
2009-03-19 17:06                     ` Jan Kara
2009-03-19 17:06                       ` Jan Kara
2009-03-19 20:05                     ` Linus Torvalds
2009-03-19 20:05                       ` Linus Torvalds
2009-03-19 20:21                   ` Linus Torvalds
2009-03-19 20:21                     ` Linus Torvalds
2009-03-19 21:17                     ` Ying Han
2009-03-19 21:17                       ` Ying Han
2009-03-19 22:16                     ` Jan Kara
2009-03-19 22:16                       ` Jan Kara
2009-03-19 16:46             ` Jan Kara
2009-03-19 16:46               ` Jan Kara
2009-03-24  7:44               ` Nick Piggin
2009-03-24  7:44                 ` Nick Piggin
2009-03-24 10:27                 ` Nick Piggin
2009-03-24 10:27                   ` Nick Piggin
2009-03-24 10:32                 ` Andrew Morton
2009-03-24 10:32                   ` Andrew Morton
2009-03-24 15:35                   ` Nick Piggin
2009-03-24 15:35                     ` Nick Piggin
2009-03-26 18:29                     ` Jan Kara
2009-03-26 18:29                       ` Jan Kara
2009-03-26  0:03                   ` Ying Han
2009-03-26  0:03                     ` Ying Han
2009-03-24 12:39                 ` Jan Kara
2009-03-24 12:39                   ` Jan Kara
2009-03-24 12:55                   ` Jan Kara
2009-03-24 12:55                     ` Jan Kara
2009-03-24 13:26                     ` Jan Kara
2009-03-24 13:26                       ` Jan Kara
2009-03-24 14:01                       ` Chris Mason
2009-03-24 14:01                         ` Chris Mason
2009-03-24 14:07                         ` Jan Kara
2009-03-24 14:07                           ` Jan Kara
2009-03-26  8:18                           ` Aneesh Kumar K.V
2009-03-26  8:18                             ` Aneesh Kumar K.V
2009-03-24 14:30                       ` Nick Piggin
2009-03-24 14:30                         ` Nick Piggin
2009-03-24 14:47                         ` Jan Kara
2009-03-24 14:47                           ` Jan Kara
2009-03-24 14:56                           ` Peter Zijlstra
2009-03-24 14:56                             ` Peter Zijlstra
2009-03-24 15:29                             ` Jan Kara
2009-03-24 15:29                               ` Jan Kara
2009-03-24 20:14                               ` OGAWA Hirofumi
2009-03-24 20:14                                 ` OGAWA Hirofumi
2009-03-26  8:47                               ` Aneesh Kumar K.V
2009-03-26  8:47                                 ` Aneesh Kumar K.V
2009-03-26 11:37                                 ` Jan Kara
2009-03-26 11:37                                   ` Jan Kara
2009-03-26 23:02                                 ` Linus Torvalds
2009-03-26 23:02                                   ` Linus Torvalds
2009-03-24 15:03                           ` Nick Piggin
2009-03-24 15:03                             ` Nick Piggin
2009-03-24 15:48                             ` Jan Kara
2009-03-24 15:48                               ` Jan Kara
2009-03-24 17:35                               ` Jan Kara
2009-03-24 17:35                                 ` Jan Kara
2009-03-24 17:35                                 ` Jan Kara
2009-04-01 22:36                                 ` Ying Han
2009-04-01 22:36                                   ` Ying Han
2009-04-02 10:11                                   ` Jan Kara
2009-04-02 10:11                                     ` Jan Kara
2009-04-02 11:24                                   ` Nick Piggin
2009-04-02 11:24                                     ` Nick Piggin
2009-04-02 11:34                                     ` Jan Kara
2009-04-02 11:34                                       ` Jan Kara
2009-04-02 15:51                                       ` Nick Piggin [this message]
2009-04-02 15:51                                         ` Nick Piggin
2009-04-02 17:44                                         ` Ying Han
2009-04-02 17:44                                           ` Ying Han
2009-04-02 22:52                                           ` Ying Han
2009-04-02 22:52                                             ` Ying Han
2009-04-02 23:39                                             ` Jan Kara
2009-04-02 23:39                                               ` Jan Kara
2009-04-03  0:25                                               ` Ying Han
2009-04-03  0:25                                                 ` Ying Han
2009-04-03  1:29                                               ` Ying Han
2009-04-03  1:29                                                 ` Ying Han
2009-04-03  9:41                                                 ` Jan Kara
2009-04-03  9:41                                                   ` Jan Kara
2009-04-03 21:34                                                   ` Ying Han
2009-04-03 21:34                                                     ` Ying Han
2009-04-03  0:13                                     ` Ying Han
2009-04-03  0:13                                       ` Ying Han
2009-03-27 20:35                 ` Ying Han
2009-03-27 20:35                   ` Ying Han
2009-03-20  0:34     ` Ying Han
2009-03-20  0:34       ` Ying Han
2009-03-20  0:49       ` Linus Torvalds
2009-03-20  0:49         ` Linus Torvalds
2009-03-20  7:00         ` Ying Han
2009-03-20  7:00           ` Ying Han
2009-03-25 23:15     ` Ying Han
2009-03-25 23:15       ` Ying Han

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=200904030251.22197.nickpiggin@yahoo.com.au \
    --to=nickpiggin@yahoo.com.au \
    --cc=a.p.zijlstra@chello.nl \
    --cc=akpm@linux-foundation.org \
    --cc=alexk@google.com \
    --cc=guichaz@gmail.com \
    --cc=jack@suse.cz \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mbligh@mbligh.org \
    --cc=mikew@google.com \
    --cc=rohitseth@google.com \
    --cc=torvalds@linux-foundation.org \
    --cc=yinghan@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.