[PATCH] data logging patches available for 2.4.21-preX

[PATCH] data logging patches available for 2.4.21-preX

[Chris Mason]

Hello all,

I've updated the data logging patches to 2.4.21-pre3, once the suse
mirror is done updating, you can download them from:

ftp.suse.com/pub/people/mason/patches/data-logging/2.4.21

Changes:

mount -o remount can switch between data modes.

transaction overflow bug fix (BUG in journal_mark_dirty)

fix for -ENOSPC hang when the block size < page size (ia64, alpha)

integration with akpm's b_journal_head code from -ac.  Should make it
possible for these patches to work both on vanilla kernels and Alan's
branch.  I haven't tried this yet though, so testers would be
appreciated.

commit_super is now sync_fs

I had to rework some of the data=ordered buffer handling code to merge
with the changes in 2.4.21-preX, so use some caution with this code.

-chris

Re: [PATCH] data logging patches available for 2.4.21-preX

[Dieter Nützel]

Chris Mason wrote:
> Hello all,
>
> I've updated the data logging patches to 2.4.21-pre3, once the suse
> mirror is done updating, you can download them from:
>
> ftp.suse.com/pub/people/mason/patches/data-logging/2.4.21
>
> Changes:
>
> mount -o remount can switch between data modes.
>
> transaction overflow bug fix (BUG in journal_mark_dirty)
>
> fix for -ENOSPC hang when the block size < page size (ia64, alpha)
>
> integration with akpm's b_journal_head code from -ac.  Should make it
> possible for these patches to work both on vanilla kernels and Alan's
> branch.  I haven't tried this yet though, so testers would be
> appreciated.
>
> commit_super is now sync_fs
>
> I had to rework some of the data=ordered buffer handling code to merge
> with the changes in 2.4.21-preX, so use some caution with this code.

Hello Chris,

as always very nice work!
I have it now running fine on top of 2.4.21-pre3-jam3 (2.4.21-pre3aa1).
Some fiddling was necessary but went smooth after all.
My /home partition is mounted with -o data=ordered and the performance is 
great. Sorry, no real benchmarks, yet.

But some question stay open:
Where is 01-akpm-sync_fs-fix-2.diff and 01-iput-deadlock-fix.diff?
Isn't it needed anylonger 'cause you merged them and SuSE's ftp isn't updated 
yet? All files are the "old" one's from 15. January.
ftp://ftp.suse.com/pub/people/mason/patches/data-logging/2.4.21

What about patch-2.4.20.rfs.06.05-transaction-overflow-fix-0.diff?
Should I put it on top, too?

Thanks,
	Dieter

-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
@home: Dieter.Nuetzel at hamburg.de (replace at with @)

Re: [PATCH] data logging patches available for 2.4.21-preX

[Chris Mason]

On Mon, 2003-01-27 at 13:14, Dieter Nützel wrote:

> Hello Chris,
> 
> as always very nice work!
> I have it now running fine on top of 2.4.21-pre3-jam3 (2.4.21-pre3aa1).
> Some fiddling was necessary but went smooth after all.
> My /home partition is mounted with -o data=ordered and the performance is 
> great. Sorry, no real benchmarks, yet.
> 

Thanks for trying them out.

> But some question stay open:
> Where is 01-akpm-sync_fs-fix-2.diff

01-akpm-sync_fs-fix-2.diff is in the 2.4.21 data logging directory you
linked to below.

>  and 01-iput-deadlock-fix.diff?

This is in the namesys pending directory, but doesn't apply cleanly
yet.  I'm rediffing both the quota code and the data logging code on top
of 01-iput-deadlock-fix (or I might just steal Manuel's rediff, which is
still in my linuxworld backlog).  

> Isn't it needed anylonger 'cause you merged them and SuSE's ftp isn't updated 
> yet? All files are the "old" one's from 15. January.
> ftp://ftp.suse.com/pub/people/mason/patches/data-logging/2.4.21
> 
> What about patch-2.4.20.rfs.06.05-transaction-overflow-fix-0.diff?
> Should I put it on top, too?

That is included in 05-data-logging-33.

-chris

Re: [PATCH] data logging patches available for 2.4.21-preX

[Dieter Nützel]

Am Montag, 27. Januar 2003 19:33 schrieb Chris Mason:
> On Mon, 2003-01-27 at 13:14, Dieter Nützel wrote:
> > Hello Chris,
> >
> > as always very nice work!
> > I have it now running fine on top of 2.4.21-pre3-jam3 (2.4.21-pre3aa1).
> > Some fiddling was necessary but went smooth after all.
> > My /home partition is mounted with -o data=ordered and the performance is
> > great. Sorry, no real benchmarks, yet.
>
> Thanks for trying them out.
>
> > But some question stay open:
> > Where is 01-akpm-sync_fs-fix-2.diff
>
> 01-akpm-sync_fs-fix-2.diff is in the 2.4.21 data logging directory you
> linked to below.

It was and I have it but it isn't any longer...
Both 01- are missing, now

> >  and 01-iput-deadlock-fix.diff?
>
> This is in the namesys pending directory, but doesn't apply cleanly
> yet.

OK, you moved it;-)

> I'm rediffing both the quota code and the data logging code on top
> of 01-iput-deadlock-fix (or I might just steal Manuel's rediff, which is
> still in my linuxworld backlog).

Manuel's version (33b) had some trouble with -pre3aa1 (-pre3-jam3), too.

> > Isn't it needed anylonger 'cause you merged them and SuSE's ftp isn't
> > updated yet? All files are the "old" one's from 15. January.
> > ftp://ftp.suse.com/pub/people/mason/patches/data-logging/2.4.21
> >
> > What about patch-2.4.20.rfs.06.05-transaction-overflow-fix-0.diff?
> > Should I put it on top, too?
>
> That is included in 05-data-logging-33.

Nice, no recompile needed then ;-)

I'll do some "normal" vs data=journal/ordered "time synctest -F -f -n 1 -t 100 
dir_name" tests, now.

Maybe I integrate preemption...

-Dieter

Re: [PATCH] data logging patches available for 2.4.21-preX

[Manuel Krause]

On 01/27/2003 07:33 PM, Chris Mason wrote:
> On Mon, 2003-01-27 at 13:14, Dieter Nützel wrote:
> 
>>Hello Chris,
>>
>>as always very nice work!
>>I have it now running fine on top of 2.4.21-pre3-jam3 (2.4.21-pre3aa1).
>>Some fiddling was necessary but went smooth after all.
>>My /home partition is mounted with -o data=ordered and the performance is 
>>great. Sorry, no real benchmarks, yet.
> 
> Thanks for trying them out.
> 
>>But some question stay open:
>>Where is 01-akpm-sync_fs-fix-2.diff
> 
> 01-akpm-sync_fs-fix-2.diff is in the 2.4.21 data logging directory you
> linked to below.
> 
>> and 01-iput-deadlock-fix.diff?
> 
> This is in the namesys pending directory, but doesn't apply cleanly
> yet.  I'm rediffing both the quota code and the data logging code on top
> of 01-iput-deadlock-fix (or I might just steal Manuel's rediff, which is
> still in my linuxworld backlog).  

If they're o.k. just feel free to do so... :-))


Dieter, you still aren't subscribed?
I (only) changed the rejected things so the patches would apply and put 
the files

for 2.4.20 @
http://manuel.krause.bei.t-online.de/downloads/2.4.20/03-data-logging-32b.diff.gz

for 2.4.21-pre(3) @
http://manuel.krause.bei.t-online.de/downloads/2.4.21-pre/05-data-logging-33b.diff.gz

on friday.
( First apply 01-iput-deadlock-fix, then Chris' patches with the one 
replaced from above. )

Bye,

Manuel

P.S.: The data=ordered mode (of which Chris wrote we should be careful 
with) runs fine here since the patches came out! :-)


>>Isn't it needed anylonger 'cause you merged them and SuSE's ftp isn't updated 
>>yet? All files are the "old" one's from 15. January.
>>ftp://ftp.suse.com/pub/people/mason/patches/data-logging/2.4.21
>>
>>What about patch-2.4.20.rfs.06.05-transaction-overflow-fix-0.diff?
>>Should I put it on top, too?
> 
> That is included in 05-data-logging-33.
> 
> -chris

Re: [PATCH] data logging patches available for 2.4.21-preX

[Dieter Nützel]

Am Montag, 27. Januar 2003 19:59 schrieb Manuel Krause:
> On 01/27/2003 07:33 PM, Chris Mason wrote:
> > On Mon, 2003-01-27 at 13:14, Dieter Nützel wrote:
> >>Hello Chris,
> >>
> >>as always very nice work!
> >>I have it now running fine on top of 2.4.21-pre3-jam3 (2.4.21-pre3aa1).
> >>Some fiddling was necessary but went smooth after all.
> >>My /home partition is mounted with -o data=ordered and the performance is
> >>great. Sorry, no real benchmarks, yet.
> >
> > Thanks for trying them out.
> >
> >>But some question stay open:
> >>Where is 01-akpm-sync_fs-fix-2.diff
> >
> > 01-akpm-sync_fs-fix-2.diff is in the 2.4.21 data logging directory you
> > linked to below.
> >
> >> and 01-iput-deadlock-fix.diff?
> >
> > This is in the namesys pending directory, but doesn't apply cleanly
> > yet.  I'm rediffing both the quota code and the data logging code on top
> > of 01-iput-deadlock-fix (or I might just steal Manuel's rediff, which is
> > still in my linuxworld backlog).
>
> If they're o.k. just feel free to do so... :-))
>
>
> Dieter, you still aren't subscribed?

I am ;-)

> I (only) changed the rejected things so the patches would apply and put
> the files
>
> for 2.4.20 @
> http://manuel.krause.bei.t-online.de/downloads/2.4.20/03-data-logging-32b.d
>iff.gz
>
> for 2.4.21-pre(3) @
> http://manuel.krause.bei.t-online.de/downloads/2.4.21-pre/05-data-logging-3
>3b.diff.gz
>
> on friday.
> ( First apply 01-iput-deadlock-fix, then Chris' patches with the one
> replaced from above. )

I've them running with some additional fiddling on top of 2.4.21-pre3-jam3 as 
I wrote.

> Bye,
>
> Manuel
>
> P.S.: The data=ordered mode (of which Chris wrote we should be careful
> with) runs fine here since the patches came out! :-)

Here, too.

-Dieter

Re: [PATCH] data logging patches available for 2.4.21-preX

[Manuel Krause]

On 01/27/2003 07:49 PM, Dieter Nützel wrote:
> Am Montag, 27. Januar 2003 19:33 schrieb Chris Mason:
> 
>>On Mon, 2003-01-27 at 13:14, Dieter Nützel wrote:
>>
>>>Hello Chris,
>>>
>>>as always very nice work!
>>>I have it now running fine on top of 2.4.21-pre3-jam3 (2.4.21-pre3aa1).
>>>Some fiddling was necessary but went smooth after all.
>>>My /home partition is mounted with -o data=ordered and the performance is
>>>great. Sorry, no real benchmarks, yet.
>>
>>Thanks for trying them out.
>>
>>>But some question stay open:
>>>Where is 01-akpm-sync_fs-fix-2.diff
>>
>>01-akpm-sync_fs-fix-2.diff is in the 2.4.21 data logging directory you
>>linked to below.
> 
> It was and I have it but it isn't any longer...
> Both 01- are missing, now

Must be an on-disk or ftp access problem known for SuSEs main server.
ftp://ftp.gwdg.de/pub/linux/suse/ftp.suse.com/people/mason/patches/data-logging/
still has all "old" files and _does_ transfer them ;-)
(Don't know how often they do their updates, though.)

Ooops. Getting web connection problems, too, in the moment I write.

>>> and 01-iput-deadlock-fix.diff?
>>
>>This is in the namesys pending directory, but doesn't apply cleanly
>>yet.
> 
> OK, you moved it;-)

:-)) Yess. Oleg posted first about.

>>I'm rediffing both the quota code and the data logging code on top
>>of 01-iput-deadlock-fix (or I might just steal Manuel's rediff, which is
>>still in my linuxworld backlog).
> 
> Manuel's version (33b) had some trouble with -pre3aa1 (-pre3-jam3), too.

It's only adjusted for namesys' 01-iput-deadlock-fix and tested to work 
with the rml.preempt-patch on here. Of course it's no developers work -- 
merely an editors one.

Where can I find the *jam* page with more information?

>>>Isn't it needed anylonger 'cause you merged them and SuSE's ftp isn't
>>>updated yet? All files are the "old" one's from 15. January.
>>>ftp://ftp.suse.com/pub/people/mason/patches/data-logging/2.4.21
>>>
>>>What about patch-2.4.20.rfs.06.05-transaction-overflow-fix-0.diff?
>>>Should I put it on top, too?
>>
>>That is included in 05-data-logging-33.
> 
> Nice, no recompile needed then ;-)
> 
> I'll do some "normal" vs data=journal/ordered "time synctest -F -f -n 1 -t 100 
> dir_name" tests, now.

BTW, does the on-disk FS structure matter with this comparison? (FS 
content created with data=journal/ordered <-> tested with 
data=journal/ordered) -- Or did this only have importance when Oleg 
introduced the new block allocator?

> Maybe I integrate preemption...

DO exactly this.

Bye,

Manuel

Re: [PATCH] data logging patches available for 2.4.21-preX

[Oleg Drokin]

Hello!

On Mon, Jan 27, 2003 at 09:42:12PM +0100, Manuel Krause wrote:

> BTW, does the on-disk FS structure matter with this comparison? (FS 

It should. Data layout determines how often (and how far) you do your seeks.

> content created with data=journal/ordered <-> tested with 
> data=journal/ordered) -- Or did this only have importance when Oleg 
> introduced the new block allocator?

Old allocator had different behaviour than new one. (in default config),
also they have different tail policies.

Bye,
    Oleg

Re: [PATCH] data logging patches available for 2.4.21-preX

[Dieter Nützel]

Am Montag, 27. Januar 2003 19:49 schrieb Dieter Nützel:
> Am Montag, 27. Januar 2003 19:33 schrieb Chris Mason:
> > On Mon, 2003-01-27 at 13:14, Dieter Nützel wrote:
> > > Hello Chris,
> > >
> > > as always very nice work!
> > > I have it now running fine on top of 2.4.21-pre3-jam3 (2.4.21-pre3aa1).
> > > Some fiddling was necessary but went smooth after all.
> > > My /home partition is mounted with -o data=ordered and the performance
> > > is great. Sorry, no real benchmarks, yet.
> >
> > Thanks for trying them out.

[-]

> I'll do some "normal" vs data=journal/ordered "time synctest -F -f -n 1 -t
> 100 dir_name" tests, now.

Good morning to Oleg ;-)

Puh, this took some time.

I've used one of my old IBM DDRS-34560 SCSI disks.
Second partition in the first half.

Raw transfer rate is:
SunWave1 /database# hdparm -tT /dev/sdb5

/dev/sdb5:
 Timing buffer-cache reads:   128 MB in  0.46 seconds =278.26 MB/sec
 Timing buffered disk reads:  64 MB in  6.00 seconds = 10.67 MB/sec

Latest aic7xx 6.2.28 SCSI driver seems to be a little bit slow.
I had ~12,8 MB/s before.

I've created a new file system before every run.

First round:
synctest -F -f -n 1 -t 100 /database/db1/test/
Two runs for all three mount options.

normal
........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................****************************************************************************************************
1.620u 67.130s 7:26.85 15.3%    0+0k 0+0io 607pf+0w
...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................*...***.**************.**********************************************************************************
2.260u 73.100s 7:01.34 17.8%    0+0k 0+0io 607pf+0w

data=ordered
....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................*..*****.************************************************.**********************************************
1.580u 75.950s 9:11.18 14.0%    0+0k 0+0io 607pf+0w
...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................*.******.**.*******************************.*******************.*****************************************
1.370u 76.460s 9:06.92 14.2%    0+0k 0+0io 607pf+0w

data=journal
........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................****************************************************************************************************
1.780u 67.840s 7:30.93 15.4%    0+0k 0+0io 607pf+0w
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................*****.********.********.*******************************************************************************
2.420u 71.410s 7:01.52 17.5%    0+0k 0+0io 607pf+0w


Second round:
synctest -n 1 -t 100 /database/db1/test/
This time only one trip for every mount option.

normal
..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................*********************************************************************************.*............................*****...*...........**.........*.......*......*....*.......**........*.*.**
0.240u 8.430s 0:28.94 29.9%     0+0k 0+0io 607pf+0w

data=ordered
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................**************************************************************************************....*..........*.*..*.........**.....*...*.........*..*...*......*.*...*
0.250u 9.140s 1:19.15 11.8%     0+0k 0+0io 607pf+0w

data=journal
.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................**********************************************************************************************..............*......*........*..*...**
0.210u 12.590s 3:14.74 6.5%     0+0k 0+0io 607pf+0w


> Maybe I integrate preemption...

Later.

Good night! ;-)

-Dieter