[2.6.26-rc2-mm1] sync to speed up?

[2.6.26-rc2-mm1] sync to speed up?

[Dave Young]

Hi,

With 2.6.26-rc2-mm1 I need to sync frequently. It's not normal as my
understanding.

1. scp a big file from a lan server. Firstly the speed is about 8M/s,
but the speed will slow down to 100K/s at last. After I exec "sync",
the speed will restore.

Is this a known issue?

Regards
dave

Re: [2.6.26-rc2-mm1] sync to speed up?

[Andrew Morton]

On Mon, 19 May 2008 15:15:09 +0800 "Dave Young" <hidave.darkstar@gmail.com> wrote:

> Hi,
> 
> With 2.6.26-rc2-mm1 I need to sync frequently. It's not normal as my
> understanding.
> 
> 1. scp a big file from a lan server. Firstly the speed is about 8M/s,
> but the speed will slow down to 100K/s at last. After I exec "sync",
> the speed will restore.
> 
> Is this a known issue?
> 

Yes, probably related to the "mkfs takes forever" problem.  I haven't
looked into it yet.  Nor has anyone else afaik.

ext4-mm-delalloc-ext4-lock-reverse-fix

[Jiri Slaby]

Andrew Morton <akpm@linux-foundation.org> wrote:
> On Mon, 19 May 2008 15:15:09 +0800 "Dave Young" <hidave.darkstar@gmail.com> wrote:
> 
> > Hi,
> > 
> > With 2.6.26-rc2-mm1 I need to sync frequently. It's not normal as my
> > understanding.
> > 
> > 1. scp a big file from a lan server. Firstly the speed is about 8M/s,
> > but the speed will slow down to 100K/s at last. After I exec "sync",
> > the speed will restore.
> > 
> > Is this a known issue?
> > 
> 
> Yes, probably related to the "mkfs takes forever" problem.  I haven't
> looked into it yet.  Nor has anyone else afaik.

Absolutely not sure what this breaks in ext4, but fixes the problem for me:
---
 mm/page-writeback.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 040bc7c..789b6ad 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -881,7 +881,7 @@ int write_cache_pages(struct address_space *mapping,
 	pagevec_init(&pvec, 0);
 	if (wbc->range_cyclic) {
 		index = mapping->writeback_index; /* Start from prev offset */
-		end = wbc->range_end >> PAGE_CACHE_SHIFT;
+		end = -1;
 	} else {
 		index = wbc->range_start >> PAGE_CACHE_SHIFT;
 		end = wbc->range_end >> PAGE_CACHE_SHIFT;
-- 
1.5.4.5

Re: ext4-mm-delalloc-ext4-lock-reverse-fix

[Mingming Cao]

On Mon, 2008-05-19 at 17:24 +0200, Jiri Slaby wrote:
> Andrew Morton <akpm@linux-foundation.org> wrote:
> > On Mon, 19 May 2008 15:15:09 +0800 "Dave Young" <hidave.darkstar@gmail.com> wrote:
> > 
> > > Hi,
> > > 
> > > With 2.6.26-rc2-mm1 I need to sync frequently. It's not normal as my
> > > understanding.
> > > 
> > > 1. scp a big file from a lan server. Firstly the speed is about 8M/s,
> > > but the speed will slow down to 100K/s at last. After I exec "sync",
> > > the speed will restore.
> > > 
> > > Is this a known issue?
> > > 
> > 
> > Yes, probably related to the "mkfs takes forever" problem.  I haven't
> > looked into it yet.  Nor has anyone else afaik.
> 
> Absolutely not sure what this breaks in ext4, but fixes the problem for me:
> ---
>  mm/page-writeback.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
> 
> diff --git a/mm/page-writeback.c b/mm/page-writeback.c
> index 040bc7c..789b6ad 100644
> --- a/mm/page-writeback.c
> +++ b/mm/page-writeback.c
> @@ -881,7 +881,7 @@ int write_cache_pages(struct address_space *mapping,
>  	pagevec_init(&pvec, 0);
>  	if (wbc->range_cyclic) {
>  		index = mapping->writeback_index; /* Start from prev offset */
> -		end = wbc->range_end >> PAGE_CACHE_SHIFT;
> +		end = -1;
>  	} else {
>  		index = wbc->range_start >> PAGE_CACHE_SHIFT;
>  		end = wbc->range_end >> PAGE_CACHE_SHIFT;

Thanks, the patch does fixed the regression .

In the delalloc-ext4-lock-reverse.patch. It wants the
write_cache_pages() to do incremental writeout within the reserved
journal credits in case of delalyed allocation(turns on cyclic mode and
remembers starts from writeback_index). If the writeout caller sets the
range_end, it should not flushing to the end of the file in the case of
range_cyclic mode. The patch above fix the regression but makes possible
that we write too much. 


I'd say drop the delalloc-ext4-lock-reverse.patch for now, will re-think
of the delayed allocation support for page-journal-lock reserve method.

Mingming

Re: ext4-mm-delalloc-ext4-lock-reverse-fix

[Alexey Dobriyan]

On Mon, May 19, 2008 at 05:24:47PM +0200, Jiri Slaby wrote:
> Andrew Morton <akpm@linux-foundation.org> wrote:
> > On Mon, 19 May 2008 15:15:09 +0800 "Dave Young" <hidave.darkstar@gmail.com> wrote:
> > 
> > > With 2.6.26-rc2-mm1 I need to sync frequently. It's not normal as my
> > > understanding.
> > > 
> > > 1. scp a big file from a lan server. Firstly the speed is about 8M/s,
> > > but the speed will slow down to 100K/s at last. After I exec "sync",
> > > the speed will restore.
> > > 
> > > Is this a known issue?
> > > 
> > 
> > Yes, probably related to the "mkfs takes forever" problem.  I haven't
> > looked into it yet.  Nor has anyone else afaik.
> 
> Absolutely not sure what this breaks in ext4, but fixes the problem for me:

This helps, thank you!

> --- a/mm/page-writeback.c
> +++ b/mm/page-writeback.c
> @@ -881,7 +881,7 @@ int write_cache_pages(struct address_space *mapping,
>  	pagevec_init(&pvec, 0);
>  	if (wbc->range_cyclic) {
>  		index = mapping->writeback_index; /* Start from prev offset */
> -		end = wbc->range_end >> PAGE_CACHE_SHIFT;
> +		end = -1;
>  	} else {
>  		index = wbc->range_start >> PAGE_CACHE_SHIFT;
>  		end = wbc->range_end >> PAGE_CACHE_SHIFT;

Re: [2.6.26-rc2-mm1] sync to speed up?

[Theodore Tso]

On Mon, May 19, 2008 at 03:15:09PM +0800, Dave Young wrote:
> Hi,
> 
> With 2.6.26-rc2-mm1 I need to sync frequently. It's not normal as my
> understanding.
> 
> 1. scp a big file from a lan server. Firstly the speed is about 8M/s,
> but the speed will slow down to 100K/s at last. After I exec "sync",
> the speed will restore.
> 
> Is this a known issue?

Dave, what filesystem are you using?  Jiri's patch on this thread was
to fix a problem in ext4, which means that's fixing a problem which is
unrelated to the slow mkfs.ext2 problem (which is writing to a block
device, not a file in a ext4 filesystem), and so it's likely unrelated
to your problem (unless you're using ext4 and didn't tell us :-).

Andrew's probably right that it may be related to the slow mkfs
problem.  Can you confirm that you are seeing this on 2.6.26-rc2-mm1,
but not on stock 2.6.26.rc2?

       	       	       	      	      	    - Ted

Re: [2.6.26-rc2-mm1] sync to speed up?

[Dave Young]

On Tue, May 20, 2008 at 7:05 AM, Theodore Tso <tytso@mit.edu> wrote:
> On Mon, May 19, 2008 at 03:15:09PM +0800, Dave Young wrote:
>> Hi,
>>
>> With 2.6.26-rc2-mm1 I need to sync frequently. It's not normal as my
>> understanding.
>>
>> 1. scp a big file from a lan server. Firstly the speed is about 8M/s,
>> but the speed will slow down to 100K/s at last. After I exec "sync",
>> the speed will restore.
>>
>> Is this a known issue?
>
> Dave, what filesystem are you using?  Jiri's patch on this thread was
> to fix a problem in ext4, which means that's fixing a problem which is
> unrelated to the slow mkfs.ext2 problem (which is writing to a block
> device, not a file in a ext4 filesystem), and so it's likely unrelated
> to your problem (unless you're using ext4 and didn't tell us :-).

I'm using ext3. Jiri's patch fixes the my problem.  Seems the problem
is not related to filesystem.

>
> Andrew's probably right that it may be related to the slow mkfs
> problem.  Can you confirm that you are seeing this on 2.6.26-rc2-mm1,
> but not on stock 2.6.26.rc2?

I didn't notice same thing on 2.6.26-rc2, is it necessary for me to test on it?

Regards
dave

Re: ext4-mm-delalloc-ext4-lock-reverse-fix

[Dave Young]

On Tue, May 20, 2008 at 4:37 AM, Alexey Dobriyan <adobriyan@gmail.com> wrote:
> On Mon, May 19, 2008 at 05:24:47PM +0200, Jiri Slaby wrote:
>> Andrew Morton <akpm@linux-foundation.org> wrote:
>> > On Mon, 19 May 2008 15:15:09 +0800 "Dave Young" <hidave.darkstar@gmail.com> wrote:
>> >
>> > > With 2.6.26-rc2-mm1 I need to sync frequently. It's not normal as my
>> > > understanding.
>> > >
>> > > 1. scp a big file from a lan server. Firstly the speed is about 8M/s,
>> > > but the speed will slow down to 100K/s at last. After I exec "sync",
>> > > the speed will restore.
>> > >
>> > > Is this a known issue?
>> > >
>> >
>> > Yes, probably related to the "mkfs takes forever" problem.  I haven't
>> > looked into it yet.  Nor has anyone else afaik.
>>
>> Absolutely not sure what this breaks in ext4, but fixes the problem for me:

Jiri, it fixes my scp-slow-down problem as well. Thanks.

>
> This helps, thank you!
>
>> --- a/mm/page-writeback.c
>> +++ b/mm/page-writeback.c
>> @@ -881,7 +881,7 @@ int write_cache_pages(struct address_space *mapping,
>>       pagevec_init(&pvec, 0);
>>       if (wbc->range_cyclic) {
>>               index = mapping->writeback_index; /* Start from prev offset */
>> -             end = wbc->range_end >> PAGE_CACHE_SHIFT;
>> +             end = -1;
>>       } else {
>>               index = wbc->range_start >> PAGE_CACHE_SHIFT;
>>               end = wbc->range_end >> PAGE_CACHE_SHIFT;
>
>