Comparison of notail and notail,iicache(14) 2.4.19-pre7

Comparison of notail and notail,iicache(14) 2.4.19-pre7

[Manuel Krause]

Hi all!

I tried the patch linux-2.4.19p6-iicache-new-14.patch on 2.4.19-pre7 in 
addition to Olegs first speedup patches, some AA and AKPM ones and made 
a comparison of my backup timings:

/dev/hda11             5550248   4804584    745664  87% /mnt/beta

noatime,notail:
---------------
to udma-33       real   9m46.648s   user 0m1.970s   sys 1m45.070s
                         7.800 MB/s      (used FS)
			
back to udma-66  real  11m 7.432s   user 0m1.940s   sys 1m43.530s
                         7.030 MB/s      (unused FS)
	
noatime,notail,iicache (new-14):
--------------------------------
to udma-33       real   9m38.230s    user 0m1.970s   sys 1m48.670s
                         8.114 MB/s      (unused FS)
                                  +1.45% rate (first copy)
				
back to udma-66  real  10m54.870s    user 0m2.230s   sys 1m46.280s
                         7.165 MB/s      (unused FS)
	                         +1.92% rate (second copy)
020424 23:55


File size distribution statistics are available upon requst.


Thanks
and best wishes,


Manuel

Re: Comparison of notail and notail,iicache(14) 2.4.19-pre7

[Manuel Krause]

On 04/25/2002 12:25 AM, Manuel Krause wrote:

> Hi all!
> 
> I tried the patch linux-2.4.19p6-iicache-new-14.patch on 2.4.19-pre7 in 
> addition to Olegs first speedup patches, some AA and AKPM ones and made 
> a comparison of my backup timings:
> 
> /dev/hda11             5550248   4804584    745664  87% /mnt/beta
> 
> noatime,notail:
> ---------------
> to udma-33       real   9m46.648s   user 0m1.970s   sys 1m45.070s
>                         7.800 MB/s      (used FS)
>            
> back to udma-66  real  11m 7.432s   user 0m1.940s   sys 1m43.530s
>                         7.030 MB/s      (unused FS)
>     
> noatime,notail,iicache (new-14):
> --------------------------------
> to udma-33       real   9m38.230s    user 0m1.970s   sys 1m48.670s
>                         8.114 MB/s      (unused FS)
>                                  +1.45% rate (first copy)
>                
> back to udma-66  real  10m54.870s    user 0m2.230s   sys 1m46.280s
>                         7.165 MB/s      (unused FS)
>                              +1.92% rate (second copy)
> 020424 23:55
> 
> 
> File size distribution statistics are available upon requst.
> 


Sorry, I just see I've always forgotten to mention I use RMLs preempt + 
lock-break patches, too. More info upon a stable configuration on this 
setup for dd backups and VMware sessions - just mail. (Just in case 
someone faced same problems I did.)

Thanks
and best wishes, again,



Manuel

Re: Comparison of notail and notail,iicache(14) 2.4.19-pre7

[Manuel Krause]

 >
 > Sorry, I just see I've always forgotten to mention I use RMLs preempt +
 > lock-break patches, too. More info upon a stable configuration on this
 > setup for dd backups and VMware sessions - just mail. (Just in case
 > someone faced same problems I did.)
 >

And I've hadone seldom file experience with iicache.

My NS6' localstore.rdf did not work after copying-to and copying-back.
Mmh. Could really be, I had this before with previous versions of
iicache applied. Nothing in the logs. No reiserfsck errors. Now I've
made a new one nearly automatically. Don't know how to catch this one.

Seldom.


Manuel

Re: Comparison of notail and notail,iicache(14) 2.4.19-pre7

[Manuel Krause]

On 04/25/2002 03:09 AM, Manuel Krause wrote:

>  >
>  > Sorry, I just see I've always forgotten to mention I use RMLs preempt +
>  > lock-break patches, too. More info upon a stable configuration on this
>  > setup for dd backups and VMware sessions - just mail. (Just in case
>  > someone faced same problems I did.)
>  >
> 
> And I've hadone seldom file experience with iicache.
> 
> My NS6' localstore.rdf did not work after copying-to and copying-back.
> Mmh. Could really be, I had this before with previous versions of
> iicache applied. Nothing in the logs. No reiserfsck errors. Now I've
> made a new one nearly automatically. Don't know how to catch this one.
> 
> Seldom.
> 


O.k., at the moment there have been many more files been truncated (my 
NS bookmarks, rabähh, seti@home status files, e.g.), and I don't want to 
dig further for now as it went fine without iicache it goes now to send 
this mail (mounted without ,iicache) and I need to go to bed.

Maybe my set of patches was a bit overloaded... or iicache has a bug 
with rdf & Co. files.

Thanks,

Manuel

Re: Comparison of notail and notail,iicache(14) 2.4.19-pre7

[Hans Reiser]

Manuel Krause wrote:

> On 04/25/2002 12:25 AM, Manuel Krause wrote:
>
>> Hi all!
>>
>> I tried the patch linux-2.4.19p6-iicache-new-14.patch on 2.4.19-pre7 
>> in addition to Olegs first speedup patches, some AA and AKPM ones and 
>> made a comparison of my backup timings:
>>
>> /dev/hda11             5550248   4804584    745664  87% /mnt/beta
>>
>> noatime,notail:
>> ---------------
>> to udma-33       real   9m46.648s   user 0m1.970s   sys 1m45.070s
>>                         7.800 MB/s      (used FS)
>>            back to udma-66  real  11m 7.432s   user 0m1.940s   sys 
>> 1m43.530s
>>                         7.030 MB/s      (unused FS)
>>     noatime,notail,iicache (new-14):
>> --------------------------------
>> to udma-33       real   9m38.230s    user 0m1.970s   sys 1m48.670s
>>                         8.114 MB/s      (unused FS)
>>                                  +1.45% rate (first copy)
>>                back to udma-66  real  10m54.870s    user 0m2.230s   
>> sys 1m46.280s
>>                         7.165 MB/s      (unused FS)
>>                              +1.92% rate (second copy)
>> 020424 23:55
>>
>>
>> File size distribution statistics are available upon requst.
>>
>
>
> Sorry, I just see I've always forgotten to mention I use RMLs preempt 
> + lock-break patches, too. More info upon a stable configuration on 
> this setup for dd backups and VMware sessions - just mail. (Just in 
> case someone faced same problems I did.)
>
> Thanks
> and best wishes, again,
>
>
>
> Manuel
>
>
>
>
Manuel, when you combine this many patches, and compare them all 
combined to no patches, it becomes hard to say exactly which patches are 
worth accepting into the main line even if all combined they are faster.

Hans

Re: Comparison of notail and notail,iicache(14) 2.4.19-pre7

[Manuel Krause]

On 04/25/2002 12:37 PM, Hans Reiser wrote:

> Manuel Krause wrote:
> 
>> On 04/25/2002 12:25 AM, Manuel Krause wrote:
>>
>>> Hi all!
>>>
>>> I tried the patch linux-2.4.19p6-iicache-new-14.patch on 2.4.19-pre7 
>>> in addition to Olegs first speedup patches, some AA and AKPM ones and 
>>> made a comparison of my backup timings:
>>>
>>> /dev/hda11             5550248   4804584    745664  87% /mnt/beta
>>>
>>> noatime,notail:
>>> ---------------
[...]
>>>     noatime,notail,iicache (new-14):
>>> --------------------------------
[...]
>>
>>
>> Sorry, I just see I've always forgotten to mention I use RMLs preempt 
>> + lock-break patches, too. More info upon a stable configuration on 
>> this setup for dd backups and VMware sessions - just mail. (Just in 
>> case someone faced same problems I did.)
>>
>> Thanks
>> and best wishes, again,
>>
>> Manuel
>>
> Manuel, when you combine this many patches, and compare them all 
> combined to no patches, it becomes hard to say exactly which patches are 
> worth accepting into the main line even if all combined they are faster.
> 
> Hans
> 


Hi!

I thought it was readable I compared no-iicache to iicache-applied-and- 
mounted.

Except for my mistake to get a cp timing of a used FS first when I timed 
the copies of unused FSs in the other cases -- I don't know in which 
statement I was unclear. What's really unclear is the problem with my 
failing Netscape files on the living iicache-mounted FS...

...and in this line, of course, you're right: The number of applied 
patches on-here exceeds my limit, too. It's ridiculous. I don't know how 
much time I'd save if I didn't like to test and configure this own setup 
to be stable. Mainline? Please, don't comment.
My most liked/useful patches are RML.preempt+lock-break, 
AKPM.read-latency, the recent AA.vm-patches, AA.low-latency and 
ReiserFS.speedup+pending!

Manuel

Re: Comparison of notail and notail,iicache(14) 2.4.19-pre7

[Dieter Nützel]

Manuel Krause wrote:

[-]
> What's really unclear is the problem with my 
> failing Netscape files on the living iicache-mounted FS...

I had one outlier with iicache-new-13 here, too.

/etc/cups/ppds.dat

couldn't be accessed. I reported back but got no response.

> ...and in this line, of course, you're right: The number of applied 
> patches on-here exceeds my limit, too. It's ridiculous. I don't know how 
> much time I'd save if I didn't like to test and configure this own setup 
> to be stable. Mainline? Please, don't comment.
> My most liked/useful patches are RML.preempt+lock-break, 
> AKPM.read-latency, the recent AA.vm-patches, AA.low-latency and 
> ReiserFS.speedup+pending!

No offenses, please. I truly running much the same, here ;-)))
Except of ReiserFS.speedup. But the new block allocation is of course 
interesting.

Next thing could be linux-2.4.19p6-compound-speedup.patch.
Any version for -pre7 ready?

Regards,
	Dieter

-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
@home: Dieter.Nuetzel@hamburg.de

Re: Comparison of notail and notail,iicache(14) 2.4.19-pre7

[Yury Yu. Rupasov]

Dieter Nützel wrote:

> Manuel Krause wrote:
>
> [-]
> > What's really unclear is the problem with my
> > failing Netscape files on the living iicache-mounted FS...
>
> I had one outlier with iicache-new-13 here, too.
>
> /etc/cups/ppds.dat
>
> couldn't be accessed. I reported back but got no response.
>
> > ...and in this line, of course, you're right: The number of applied
> > patches on-here exceeds my limit, too. It's ridiculous. I don't know how
> > much time I'd save if I didn't like to test and configure this own setup
> > to be stable. Mainline? Please, don't comment.
> > My most liked/useful patches are RML.preempt+lock-break,
> > AKPM.read-latency, the recent AA.vm-patches, AA.low-latency and
> > ReiserFS.speedup+pending!
>
> No offenses, please. I truly running much the same, here ;-)))
> Except of ReiserFS.speedup. But the new block allocation is of course
> interesting.
>
> Next thing could be linux-2.4.19p6-compound-speedup.patch.
> Any version for -pre7 ready?
>

This patch should work with linux-2.4.19-pre7 as well.

Thanks,
Yura.

Re: Comparison of notail and notail,iicache(14) 2.4.19-pre7

[Dieter Nützel]

On Friday 26 April 2002 13:41, Yury Yu. Rupasov wrote:
> Dieter Nützel wrote:
> > Manuel Krause wrote:
> >
> > [-]
> >
> > > What's really unclear is the problem with my
> > > failing Netscape files on the living iicache-mounted FS...
> >
> > I had one outlier with iicache-new-13 here, too.
> >
> > /etc/cups/ppds.dat
> >
> > couldn't be accessed. I reported back but got no response.

[-]

> > Next thing could be linux-2.4.19p6-compound-speedup.patch.
> > Any version for -pre7 ready?
>
> This patch should work with linux-2.4.19-pre7 as well.

Yes, Yura.
I tried it today and got one rejection with preempt+lock-break.
fs/reiserfs/journal.c.rej

***************
*** 576,592 ****
  /* lock the current transaction */
  inline static void lock_journal(struct super_block *p_s_sb) {
    PROC_INFO_INC( p_s_sb, journal.lock_journal );
-   while(atomic_read(&(SB_JOURNAL(p_s_sb)->j_wlock)) > 0) {
-     PROC_INFO_INC( p_s_sb, journal.lock_journal_wait );
-     sleep_on(&(SB_JOURNAL(p_s_sb)->j_wait)) ;
-   }
-   atomic_set(&(SB_JOURNAL(p_s_sb)->j_wlock), 1) ;
  }
  
  /* unlock the current transaction */
  inline static void unlock_journal(struct super_block *p_s_sb) {
-   atomic_dec(&(SB_JOURNAL(p_s_sb)->j_wlock)) ;
-   wake_up(&(SB_JOURNAL(p_s_sb)->j_wait)) ;
  }
  
  /*
--- 579,590 ----
  /* lock the current transaction */
  inline static void lock_journal(struct super_block *p_s_sb) {
    PROC_INFO_INC( p_s_sb, journal.lock_journal );
+   down(&SB_JOURNAL(p_s_sb)->j_lock);
  }
  
  /* unlock the current transaction */
  inline static void unlock_journal(struct super_block *p_s_sb) {
+   up(&SB_JOURNAL(p_s_sb)->j_lock);
  }
  
  /*

Had that before but Chris and Oleg gave me advice.
I tried it this way:

/* lock the current transaction */
inline static void lock_journal(struct super_block *p_s_sb) {
  PROC_INFO_INC( p_s_sb, journal.lock_journal );
  debug_lock_break(1);
  conditional_schedule();
  down(&SB_JOURNAL(p_s_sb)->j_lock);
}

/* unlock the current transaction */
inline static void unlock_journal(struct super_block *p_s_sb) {
  up(&SB_JOURNAL(p_s_sb)->j_lock);
}

But then it seems to be _NOT_ preempt (+lock-break) save.
System lock up (nothing in the logs, SysRq key didn't work) during 
latencytest0.42-png write test. Read test worked.

Thanks,
	Dieter

Re: Comparison of notail and notail,iicache(14) 2.4.19-pre7

[Manuel Krause]

On 04/26/2002 06:45 PM, Dieter Nützel wrote:

> On Friday 26 April 2002 13:41, Yury Yu. Rupasov wrote:
> 
>>Dieter Nützel wrote:
>>
>>>Manuel Krause wrote:
>>>
>>>[-]
>>>
>>>>What's really unclear is the problem with my
>>>>failing Netscape files on the living iicache-mounted FS...
>>>>
>>>I had one outlier with iicache-new-13 here, too.
>>>
>>>/etc/cups/ppds.dat
>>>
>>>couldn't be accessed. I reported back but got no response.
>>>
> 
> [-]
> 
>>>Next thing could be linux-2.4.19p6-compound-speedup.patch.
>>>Any version for -pre7 ready?
>>>
>>This patch should work with linux-2.4.19-pre7 as well.
>>
> 
> Yes, Yura.
> I tried it today and got one rejection with preempt+lock-break.
> fs/reiserfs/journal.c.rej
> 
[-]
> 
> Had that before but Chris and Oleg gave me advice.
> I tried it this way:
> 
> /* lock the current transaction */
> inline static void lock_journal(struct super_block *p_s_sb) {
>   PROC_INFO_INC( p_s_sb, journal.lock_journal );
>   debug_lock_break(1);
>   conditional_schedule();
>   down(&SB_JOURNAL(p_s_sb)->j_lock);
> }
> 
> /* unlock the current transaction */
> inline static void unlock_journal(struct super_block *p_s_sb) {
>   up(&SB_JOURNAL(p_s_sb)->j_lock);
> }
> 
> But then it seems to be _NOT_ preempt (+lock-break) save.
> System lock up (nothing in the logs, SysRq key didn't work) during 
> latencytest0.42-png write test. Read test worked.
> 
> Thanks,
> 	Dieter
> 
> 

Mmh, I made this adjustment in the patch, too, since the first speedup 
patches had been posted. I really don't know how serious this could be 
in this code context.

I had random hard locks,too, but blamed it to my degenerating hardware 
so far. Could anyone else advise on preempt+lock-break-awareness here?

The linux-2.4.19p6-compound-speedup.patch seems to be another measurable 
tick faster than Olegs first patchset. Maybe, I'll post some values on 
my setup over the weekend. I've had one faster run so far but it locked 
up without log hints... ;-))


Best regards,

Manuel

Re: Comparison of notail and notail,iicache(14) 2.4.19-pre7

[Dieter Nützel]

On Friday 26 April 2002 23:21, Manuel Krause wrote:
> On 04/26/2002 06:45 PM, Dieter Nützel wrote:
> > On Friday 26 April 2002 13:41, Yury Yu. Rupasov wrote:
> >>Dieter Nützel wrote:
> >>>Manuel Krause wrote:
> >>>
> >>>[-]

[-]

> > Had that before but Chris and Oleg gave me advice.
> > I tried it this way:
> >
> > /* lock the current transaction */
> > inline static void lock_journal(struct super_block *p_s_sb) {
> >   PROC_INFO_INC( p_s_sb, journal.lock_journal );
> >   debug_lock_break(1);
> >   conditional_schedule();
> >   down(&SB_JOURNAL(p_s_sb)->j_lock);
> > }
> >
> > /* unlock the current transaction */
> > inline static void unlock_journal(struct super_block *p_s_sb) {
> >   up(&SB_JOURNAL(p_s_sb)->j_lock);
> > }
> >
> > But then it seems to be _NOT_ preempt (+lock-break) save.
> > System lock up (nothing in the logs, SysRq key didn't work) during
> > latencytest0.42-png write test. Read test worked.
> >
> > Thanks,
> > 	Dieter
>
> Mmh, I made this adjustment in the patch, too, since the first speedup
> patches had been posted. I really don't know how serious this could be
> in this code context.
>
> I had random hard locks,too, but blamed it to my degenerating hardware
> so far.

I think it is hardly hardware related.
Try without lock-break (only disable it during make xconfig).

> Could anyone else advise on preempt+lock-break-awareness here?

Yura, Chris, Oleg or at least Robert Love :-)
But Robert hasn't updated the lock-break stuff for some time. He will do it 
soon.

> The linux-2.4.19p6-compound-speedup.patch seems to be another measurable
> tick faster than Olegs first patchset. Maybe, I'll post some values on
> my setup over the weekend. I've had one faster run so far but it locked
> up without log hints... ;-))

Ha, ha,...:-)

Then I have one more for you:

Page coloring patch.
It gave ~10% speedup on my 1 GHz Athlon II SlotA (0,18µm, L2 512K) for memory 
intensive apps. But be aware, it locks up randomly. The maintainer is looking 
for SMP testers 'cause it has something to-do with SMP -> preemption.
My system is stable but lockup from time to time with the page_color module 
loaded during "heavy" C and C++ compilations (~40 processes running in 
parallel).

Have a nice weekend.

-Dieter

BTW I'm working on Win VFAT disk recovery. Two damaged IBM IC35L060AVER07-0 
costomer disks. One with only a single partition and one with three 
partitions. Any advice if I should try with dd on the whole disk or every 
partition?

Re: Comparison of notail and notail,iicache(14) 2.4.19-pre7

[Manuel Krause]

On 04/27/2002 12:00 AM, Dieter Nützel wrote:

> On Friday 26 April 2002 23:21, Manuel Krause wrote:
> 
>>On 04/26/2002 06:45 PM, Dieter Nützel wrote:
>>
>>>On Friday 26 April 2002 13:41, Yury Yu. Rupasov wrote:
>>>
>>>>Dieter Nützel wrote:
>>>>
>>>>>Manuel Krause wrote:
>>>>>
>>>>>[-]
>>>>>
> 
> [-]
> 
> 
>>>Had that before but Chris and Oleg gave me advice.
>>>I tried it this way:
>>>
>>>/* lock the current transaction */
>>>inline static void lock_journal(struct super_block *p_s_sb) {
>>>  PROC_INFO_INC( p_s_sb, journal.lock_journal );
>>>  debug_lock_break(1);
>>>  conditional_schedule();
>>>  down(&SB_JOURNAL(p_s_sb)->j_lock);
>>>}
>>>
>>>/* unlock the current transaction */
>>>inline static void unlock_journal(struct super_block *p_s_sb) {
>>>  up(&SB_JOURNAL(p_s_sb)->j_lock);
>>>}
>>>
>>>But then it seems to be _NOT_ preempt (+lock-break) save.
>>>System lock up (nothing in the logs, SysRq key didn't work) during
>>>latencytest0.42-png write test. Read test worked.
>>>
>>>Thanks,
>>>	Dieter
>>>
>>Mmh, I made this adjustment in the patch, too, since the first speedup
>>patches had been posted. I really don't know how serious this could be
>>in this code context.
>>
>>I had random hard locks,too, but blamed it to my degenerating hardware
>>so far.
>>
> 
> I think it is hardly hardware related.
> Try without lock-break (only disable it during make xconfig).


I should, if it helped for you. I've lost too much work time inbetween 
for now.

> 
> 
>>Could anyone else advise on preempt+lock-break-awareness here?
>>
> 
> Yura, Chris, Oleg or at least Robert Love :-)
> But Robert hasn't updated the lock-break stuff for some time. He will do it 
> soon.
> 


Yepp. I observed this fact and hope Robert to do so.

> 
>>The linux-2.4.19p6-compound-speedup.patch seems to be another measurable
>>tick faster than Olegs first patchset. Maybe, I'll post some values on
>>my setup over the weekend. I've had one faster run so far but it locked
>>up without log hints... ;-))
>>
> 
> Ha, ha,...:-)
> 
> Then I have one more for you:
> 
> Page coloring patch.
> It gave ~10% speedup on my 1 GHz Athlon II SlotA (0,18µm, L2 512K) for memory 
> intensive apps. But be aware, it locks up randomly. The maintainer is looking 
> for SMP testers 'cause it has something to-do with SMP -> preemption.
> My system is stable but lockup from time to time with the page_color module 
> loaded during "heavy" C and C++ compilations (~40 processes running in 
> parallel).


If I read this correctly I should leave my fingers from it...
Did I make a general mistake? Preempt+ does bring valuable advantages to 
my PIII 933 _uniprocessor_ setup, too.

> 
> Have a nice weekend.
> 
> -Dieter
> 
> BTW I'm working on Win VFAT disk recovery. Two damaged IBM IC35L060AVER07-0 
> costomer disks. One with only a single partition and one with three 
> partitions. Any advice if I should try with dd on the whole disk or every 
> partition?
> 
> 

Maybe we should exchange our actual hdparm and powertweak settings in 
private via phone soon? ;-)

On my current setup I have higher disk troughput rates on small 
partitions (<0.5GB) but dropoffs in the middle of larger ones (~2GB) if 
I trust the ksysguard display. I simply do "dd if=/dev/hda3 bs=1M 
of=/dev/hdd3" after earlier timeless fiddling with bs=xyz and 
xyz~disk-cylinder-size(=16065 * 512 bytes)*2^(?)  did not give 
advantages in the past. I did not do complete disk dd-s so far but had 
many at the end of a dd with the current patch setup and partitions 
 >0.3GB with the conventional bdflush and disk read/write-latency 
parameters. Find your poison!

Have a nice weekend, too! And thanks for your comments, Dieter!

Manuel