Re: Linux 2.4.25-pre6

Re: Linux 2.4.25-pre6

[Xose Vazquez Perez]

Lukasz Trabinski wrote:

> SMP (2x2.66GHz Intel), with scsi aic79xx  with kernel -pre6 crashed after
> 3 days.

as usual:

- run memtest86+ [1] and cpuburn [2] to check the HW.
- update the firmware(MOBO BIOS and BackPlane/ESM, SCSI, hard disks, ...)
  to latest levels.


[1] http://www.memtest.org
[2] http://users.ev1.net/~redelm

--
Software is like sex, it's better when it's bug free.

Re: Linux 2.4.25-pre6

[Lukasz Trabinski]

On Tue, 20 Jan 2004, Xose Vazquez Perez wrote:
> as usual:
> 
> - run memtest86+ [1] and cpuburn [2] to check the HW.
> - update the firmware(MOBO BIOS and BackPlane/ESM, SCSI, hard disks, ...)
>   to latest levels.

Hardware is OK, with older kernel (2.4.22?) works fine.


-- 
*[ Łukasz Trąbiński ]*

Re: Linux 2.4.25-pre6

[Samium Gromoff]

Lukasz Trabinski wrote:
> On Tue, 20 Jan 2004, Xose Vazquez Perez wrote:                                         
> > as usual:                                                                            
> >                                                                                      
> > - run memtest86+ [1] and cpuburn [2] to check the HW.                                
> > - update the firmware(MOBO BIOS and BackPlane/ESM, SCSI, hard disks, ...)            
> >   to latest levels.                                                                  
>                                                                                        
> Hardware is OK, with older kernel (2.4.22?) works fine.                                
This could be misleading, still.

I`ve had a situation in the past when a n+1`th 2.4 kernel had sound crashes
where none of the ...n`th didn`t have any.

I`ve narrowed it down to cpu overclocking... ;-)

It was a very intricate situation for the kernel crashed more or less
after a certain fixed amount of time of sound playing, no matter how
long ago the bootup happened...

regards, Samium Gromoff

Linux 2.4.25-pre6

[Marcelo Tosatti]

Hi,

Here goes -pre6.

This release came out so quickly because -pre5 contains a deadly mistake
in one of the fs patches.

It contains SPARC/x86-64 updates, networking and crypto updates, amongst
others.

Summary of changes from v2.4.25-pre5 to v2.4.25-pre6
============================================

<jmorris:redhat.com>:
  o [CRYPTO]: Clean up tcrypt module and allow it to be unloaded

<kartik_me:hotmail.com>:
  o [CRYPTO]: Add CAST6 (CAST-256) algorithm

<marcelo:logos.cnet>:
  o Changed EXTRAVERSION to -pre6

<my:utfors.se>:
  o [CRYPTO]: Move ivsize from algorithm to tfm

Andi Kleen:
  o x86-64 update

Chas Williams:
  o [ATM]: br2684 incorrectly handles frames recvd with FCS (by Alex Zeffertt <ajz@cambridgebroadband.com>)
  o [ATM]: [nicstar] convert to new style pci module (by "Jorge Boncompte [DTI2]" <jorge@dti2.net>)
  o [ATM]: better behavior for sendmsg/recvmsg during async closes
  o [ATM]: refcount atm sockets

David S. Miller:
  o [SPARC64]: In early bootup, BUG() if cannot find TLB entry for remapping
  o [SPARC64]: Disable PCI ROM address OBP sanity check for now
  o [IPV4]: Print correct source addr in invalid ICMP msgs, from Dennis Jorgensen

David Stevens:
  o [IPV4/IPV6]: In MLD, add new filter first, then delete old one

David Woodhouse:
  o Do not leave inodes with stale waitqueue on slab cache

Harald Welte:
  o [NETFILTER]: Add config help texts for IP_NF_ARP{TABLES,FILTER}

Jean Tourrilhes:
  o NSC '39x support
  o VIA IrDA driver

Kurt Garloff:
  o [NETFILTER]: Align nulldevname properly in ip_tables

Marcel Holtmann:
  o [Bluetooth] Use R2 for default value of pscan_rep_mode
  o [Bluetooth] Set disconnect timer for incoming ACL links
  o [Bluetooth] Start inquiry if cache is empty
  o [Bluetooth] Change maintainer role of the Bluetooth subsystem

Re: Linux 2.4.25-pre6

[Maciej Soltysiak]

>   o [IPV4]: Print correct source addr in invalid ICMP msgs, from Dennis
Jorgensen
:-( That was my bad doing, when I was extending the information included in
that
message. Sorry.
The even worse thing is that I actually knew about the problem with it,
reported
by someone a few months ago, but never got around checking it.

Regards,
Maciej

Re: Linux 2.4.25-pre6

[Lukasz Trabinski]

In article <Pine.LNX.4.58L.0401161207000.28357@logos.cnet> you wrote:
> 
> Hi,
> 
> Here goes -pre6.
> 
> This release came out so quickly because -pre5 contains a deadly mistake
> in one of the fs patches.

SMP (2x2.66GHz Intel), with scsi aic79xx  with kernel -pre6 crashed after
3 days.

No ooops in logs files or console.
Output from console SysRq showTasks, showMem and showTasks you can see
here:

http://lukasz.eu.org/minicom.txt
or here, if first will not work:
http://www.pm.waw.pl/~lukasz/minicom.txt


-- 
*[ ŁT ]*

Re: Linux 2.4.25-pre6

[Marcelo Tosatti]

On Tue, 20 Jan 2004, Lukasz Trabinski wrote:

> In article <Pine.LNX.4.58L.0401161207000.28357@logos.cnet> you wrote:
> >
> > Hi,
> >
> > Here goes -pre6.
> >
> > This release came out so quickly because -pre5 contains a deadly mistake
> > in one of the fs patches.
>
> SMP (2x2.66GHz Intel), with scsi aic79xx  with kernel -pre6 crashed after
> 3 days.
>
> No ooops in logs files or console.
> Output from console SysRq showTasks, showMem and showTasks you can see
> here:
>
> http://lukasz.eu.org/minicom.txt
> or here, if first will not work:
> http://www.pm.waw.pl/~lukasz/minicom.txt

Hi Lukasz,

Can you please run the output through ksymoops?

Re: Linux 2.4.25-pre6

[Lukasz Trabinski]

On Tue, 20 Jan 2004, Marcelo Tosatti wrote:
> > Output from console SysRq showTasks, showMem and showTasks you can see
> > here:
> >
> > http://lukasz.eu.org/minicom.txt
> Hi Lukasz,
> 
> Can you please run the output through ksymoops?

Here is:
http://lukasz.eu.org/ksymoops.txt


-- 
*[ Łukasz Trąbiński ]*

Re: Linux 2.4.25-pre6

[Marcelo Tosatti]

On Wed, 21 Jan 2004, Lukasz Trabinski wrote:

> On Tue, 20 Jan 2004, Marcelo Tosatti wrote:
> > > Output from console SysRq showTasks, showMem and showTasks you can see
> > > here:
> > >
> > > http://lukasz.eu.org/minicom.txt
> > Hi Lukasz,
> >
> > Can you please run the output through ksymoops?
>
> Here is:
> http://lukasz.eu.org/ksymoops.txt

Thanks Lukasz,

Great, Sir Woodhouse is investingating this. The traces look pretty odd.

Can you put the vmlinux file for that kernel up on the net, too?

Re: Linux 2.4.25-pre6

[Lukasz Trabinski]

On Wed, 21 Jan 2004, Marcelo Tosatti wrote:
> Thanks Lukasz,
> 
> Great, Sir Woodhouse is investingating this. The traces look pretty odd.
> 
> Can you put the vmlinux file for that kernel up on the net, too?

http://lukasz.eu.org/vmlinux

oceanic:~$ gcc --version
gcc (GCC) 3.2.2 20030222 (Red Hat Linux 3.2.2-5)
Copyright (C) 2002 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR 
PURPOSE.

System is RedHat 9 + updates

-- 
*[ Łukasz Trąbiński ]*

Re: Linux 2.4.25-pre6

[David Woodhouse]

On Wed, 2004-01-21 at 12:28 +0100, Lukasz Trabinski wrote:
> On Wed, 21 Jan 2004, Marcelo Tosatti wrote:
> > Thanks Lukasz,
> > 
> > Great, Sir Woodhouse is investingating this. The traces look pretty odd.
> > 
> > Can you put the vmlinux file for that kernel up on the net, too?
> 
> http://lukasz.eu.org/vmlinux

Thanks. The fault is happening in the list_del() at line 301 of
fs/inode.c; the penultimate line of __refile_inode(). It seems 
that inode->i_list.prev is an invalid pointer, which caused it to oops
while holding the inode_lock.

At least, it _should_ have oopsed -- the call trace shows it's gone
through die() and do_exit(). I don't understand why you didn't get
anything on the console.

Neither do I understand why i_list.prev is corrupt. Not seeing the oops
and knowing what it actually was doesn't help. Rik?

-- 
dwmw2

Re: Linux 2.4.25-pre6

[Lukasz Trabinski]

On Wed, 21 Jan 2004, David Woodhouse wrote:

> Neither do I understand why i_list.prev is corrupt. Not seeing the oops
> and knowing what it actually was doesn't help. Rik?

I have logs from this host on different machine - no ooops.

Emergency Sync and Emergency Remount R/O didn't work.

SysRq : HELP : loglevel0-8 reBoot tErm kIll saK showMem showPc unRaw Sync 
showTa
sks Unmount 
SysRq : Emergency Sync
SysRq : Emergency Remount R/O



-- 
*[ Łukasz Trąbiński ]*

Re: Linux 2.4.25-pre6

[Marcelo Tosatti]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 715 bytes --]



On Wed, 21 Jan 2004, Lukasz Trabinski wrote:

> On Wed, 21 Jan 2004, David Woodhouse wrote:
>
> > Neither do I understand why i_list.prev is corrupt. Not seeing the oops
> > and knowing what it actually was doesn't help. Rik?
>
> I have logs from this host on different machine - no ooops.
>
> Emergency Sync and Emergency Remount R/O didn't work.
>
> SysRq : HELP : loglevel0-8 reBoot tErm kIll saK showMem showPc unRaw Sync
> showTa
> sks Unmount
> SysRq : Emergency Sync
> SysRq : Emergency Remount R/O

Lukasz,

Lets try the clueless approach and remove the inode reclaim highmem fixes
from Rik.

Please revert the attached patch (againts -pre6).

Thank you for yours efforts helping track the problem down :)

[-- Attachment #2: Type: TEXT/PLAIN, Size: 9688 bytes --]

diff -Naur -p -X /home/marcelo/lib/dontdiff linux-2.4.24/fs/inode.c linux-2.4.25-pre6/fs/inode.c
--- linux-2.4.24/fs/inode.c	2004-01-21 19:33:33.000000000 +0000
+++ linux-2.4.25-pre6/fs/inode.c	2004-01-21 19:26:26.000000000 +0000
@@ -49,8 +49,7 @@ static unsigned int i_hash_shift;
  * other linked list is the "type" list:
  *  "in_use" - valid inode, i_count > 0, i_nlink > 0
  *  "dirty"  - as "in_use" but also dirty
- *  "unused" - valid inode, i_count = 0, no pages in the pagecache
- *  "unused_pagecache" - valid inode, i_count = 0, data in the pagecache
+ *  "unused" - valid inode, i_count = 0
  *
  * A "dirty" list is maintained for each super block,
  * allowing for low-overhead inode sync() operations.
@@ -58,7 +57,6 @@ static unsigned int i_hash_shift;
 
 static LIST_HEAD(inode_in_use);
 static LIST_HEAD(inode_unused);
-static LIST_HEAD(inode_unused_pagecache);
 static struct list_head *inode_hashtable;
 static LIST_HEAD(anon_hash_chain); /* for inodes with NULL i_sb */
 
@@ -127,10 +125,10 @@ static void destroy_inode(struct inode *
 {
 	if (inode_has_buffers(inode))
 		BUG();
-	/* Reinitialise the waitqueue head because __wait_on_freeing_inode()
-	   may have left stale entries on it which it can't remove (since
-	   it knows we're freeing the inode right now */
-	init_waitqueue_head(&inode->i_wait);
+       /* Reinitialise the waitqueue head because __wait_on_freeing_inode()
+          may have left stale entries on it which it can't remove (since
+          it knows we're freeing the inode right now */
+        init_waitqueue_head(&inode->i_wait);
 	if (inode->i_sb->s_op->destroy_inode)
 		inode->i_sb->s_op->destroy_inode(inode);
 	else
@@ -254,10 +252,9 @@ static inline void wait_on_inode(struct 
  * ->read_inode, and we want to be sure that evidence of the deletion is found
  * by ->read_inode.
  *
- * Unlike the 2.6 version, this call call cannot return early, since inodes
- * do not share wait queue. Therefore, we don't call remove_wait_queue(); it
- * would be dangerous to do so since the inode may have already been freed, 
- * and it's unnecessary, since the inode is definitely going to get freed.
+ * This call might return early if an inode which shares the waitq is woken up.
+ * This is most easily handled by the caller which will loop around again
+ * looking for the inode.
  *
  * This is called with inode_lock held.
  */
@@ -293,36 +290,6 @@ static inline void __iget(struct inode *
 	inodes_stat.nr_unused--;
 }
 
-static inline void __refile_inode(struct inode *inode)
-{
-	struct list_head *to;
-
-	if (inode->i_state & I_FREEING)
-		return;
-	if (list_empty(&inode->i_hash))
-		return;
-
-	if (inode->i_state & I_DIRTY)
-		to = &inode->i_sb->s_dirty;
-	else if (atomic_read(&inode->i_count))
-		to = &inode_in_use;
-	else if (inode->i_data.nrpages)
-		to = &inode_unused_pagecache;
-	else
-		to = &inode_unused;
-	list_del(&inode->i_list);
-	list_add(&inode->i_list, to);
-}
-
-void refile_inode(struct inode *inode)
-{
-	if (!inode)
-		return;
-	spin_lock(&inode_lock);
-	__refile_inode(inode);
-	spin_unlock(&inode_lock);
-}
-
 static inline void __sync_one(struct inode *inode, int sync)
 {
 	unsigned dirty;
@@ -349,7 +316,17 @@ static inline void __sync_one(struct ino
 
 	spin_lock(&inode_lock);
 	inode->i_state &= ~I_LOCK;
-	__refile_inode(inode);
+	if (!(inode->i_state & I_FREEING)) {
+		struct list_head *to;
+		if (inode->i_state & I_DIRTY)
+			to = &inode->i_sb->s_dirty;
+		else if (atomic_read(&inode->i_count))
+			to = &inode_in_use;
+		else
+			to = &inode_unused;
+		list_del(&inode->i_list);
+		list_add(&inode->i_list, to);
+	}
 	wake_up(&inode->i_wait);
 }
 
@@ -726,7 +703,6 @@ int invalidate_inodes(struct super_block
 	spin_lock(&inode_lock);
 	busy = invalidate_list(&inode_in_use, sb, &throw_away);
 	busy |= invalidate_list(&inode_unused, sb, &throw_away);
-	busy |= invalidate_list(&inode_unused_pagecache, sb, &throw_away);
 	busy |= invalidate_list(&sb->s_dirty, sb, &throw_away);
 	busy |= invalidate_list(&sb->s_locked_inodes, sb, &throw_away);
 	spin_unlock(&inode_lock);
@@ -790,7 +766,7 @@ void prune_icache(int goal)
 {
 	LIST_HEAD(list);
 	struct list_head *entry, *freeable = &list;
-	int count, avg_pages;
+	int count;
 	struct inode * inode;
 
 	spin_lock(&inode_lock);
@@ -813,7 +789,7 @@ void prune_icache(int goal)
 		list_add(tmp, freeable);
 		inode->i_state |= I_FREEING;
 		count++;
-		if (--goal <= 0)
+		if (!--goal)
 			break;
 	}
 	inodes_stat.nr_unused -= count;
@@ -827,71 +803,8 @@ void prune_icache(int goal)
 	 * from here or we're either synchronously dogslow
 	 * or we deadlock with oom.
 	 */
-	if (goal > 0)
+	if (goal)
 		schedule_task(&unused_inodes_flush_task);
-
-#ifdef CONFIG_HIGHMEM
-	/*
-	 * On highmem machines it is possible to have low memory
-	 * filled with inodes that cannot be reclaimed because they
-	 * have page cache pages in highmem attached to them.
-	 * This could deadlock the system if the memory used by
-	 * inodes is significant compared to the amount of freeable
-	 * low memory.  In that case we forcefully remove the page
-	 * cache pages from the inodes we want to reclaim.
-	 *
-	 * Note that this loop doesn't actually reclaim the inodes;
-	 * once the last pagecache pages belonging to the inode is
-	 * gone it will be placed on the inode_unused list and the
-	 * loop above will prune it the next time prune_icache() is
-	 * called.
-	 */
-	if (goal <= 0)
-		return;
-	if (inodes_stat.nr_unused * sizeof(struct inode) * 10 <
-				freeable_lowmem() * PAGE_SIZE)
-		return;
-
-	wakeup_bdflush();
-
-	avg_pages = page_cache_size;
-	avg_pages -= atomic_read(&buffermem_pages) + swapper_space.nrpages;
-	avg_pages = avg_pages / (inodes_stat.nr_inodes + 1);
-	spin_lock(&inode_lock);
-	while (goal-- > 0) {
-		if (list_empty(&inode_unused_pagecache))
-			break;
-		entry = inode_unused_pagecache.prev;
-		list_del(entry);
-		list_add(entry, &inode_unused_pagecache);
-
-		inode = INODE(entry);
-		/* Don't nuke inodes with lots of page cache attached. */
-		if (inode->i_mapping->nrpages > 5 * avg_pages)
-			continue;
-		/* Because of locking we grab the inode and unlock the list .*/
-		if (inode->i_state & I_LOCK)
-			continue;
-		inode->i_state |= I_LOCK;
-		spin_unlock(&inode_lock);
-
-		/*
-		 * If the inode has clean pages only, we can free all its
-		 * pagecache memory; the inode will automagically be refiled
-		 * onto the unused_list.  The wakeup_bdflush above makes
-		 * sure that all inodes become clean eventually.
-		 */
-		if (list_empty(&inode->i_mapping->dirty_pages) &&
-				!inode_has_buffers(inode))
-			invalidate_inode_pages(inode);
-
-		/* Release the inode again. */
-		spin_lock(&inode_lock);
-		inode->i_state &= ~I_LOCK;
-		wake_up(&inode->i_wait);
-	}
-	spin_unlock(&inode_lock);
-#endif /* CONFIG_HIGHMEM */
 }
 
 int shrink_icache_memory(int priority, int gfp_mask)
diff -Naur -p -X /home/marcelo/lib/dontdiff linux-2.4.24/include/linux/fs.h linux-2.4.25-pre6/include/linux/fs.h
--- linux-2.4.24/include/linux/fs.h	2004-01-21 19:33:48.000000000 +0000
+++ linux-2.4.25-pre6/include/linux/fs.h	2004-01-21 18:57:13.000000000 +0000
@@ -1399,7 +1399,6 @@ extern struct dentry * lookup_hash(struc
 extern void inode_init_once(struct inode *);
 extern void __inode_init_once(struct inode *);
 extern void iput(struct inode *);
-extern void refile_inode(struct inode *inode);
 extern void force_delete(struct inode *);
 extern struct inode * igrab(struct inode *);
 extern struct inode * ilookup(struct super_block *, unsigned long);
diff -Naur -p -X /home/marcelo/lib/dontdiff linux-2.4.24/include/linux/swap.h linux-2.4.25-pre6/include/linux/swap.h
--- linux-2.4.24/include/linux/swap.h	2004-01-21 19:33:49.000000000 +0000
+++ linux-2.4.25-pre6/include/linux/swap.h	2004-01-21 18:57:13.000000000 +0000
@@ -85,7 +85,6 @@ extern int nr_swap_pages;
 
 extern unsigned int nr_free_pages(void);
 extern unsigned int nr_free_buffer_pages(void);
-extern unsigned int freeable_lowmem(void);
 extern int nr_active_pages;
 extern int nr_inactive_pages;
 extern unsigned long page_cache_size;
diff -Naur -p -X /home/marcelo/lib/dontdiff linux-2.4.24/mm/filemap.c linux-2.4.25-pre6/mm/filemap.c
--- linux-2.4.24/mm/filemap.c	2004-01-21 19:33:50.000000000 +0000
+++ linux-2.4.25-pre6/mm/filemap.c	2004-01-21 18:57:13.000000000 +0000
@@ -102,8 +102,6 @@ static inline void remove_page_from_inod
 	page->mapping = NULL;
 	wmb();
 	mapping->nrpages--;
-	if (!mapping->nrpages)
-		refile_inode(mapping->host);
 }
 
 static inline void remove_page_from_hash_queue(struct page * page)
diff -Naur -p -X /home/marcelo/lib/dontdiff linux-2.4.24/mm/page_alloc.c linux-2.4.25-pre6/mm/page_alloc.c
--- linux-2.4.24/mm/page_alloc.c	2004-01-21 19:33:50.000000000 +0000
+++ linux-2.4.25-pre6/mm/page_alloc.c	2004-01-21 18:57:13.000000000 +0000
@@ -534,23 +534,6 @@ unsigned int nr_free_highpages (void)
 
 	return pages;
 }
-
-unsigned int freeable_lowmem(void)
-{
-	unsigned int pages = 0;
-	pg_data_t *pgdat;
-
-	for_each_pgdat(pgdat) {
-		pages += pgdat->node_zones[ZONE_DMA].free_pages;
-		pages += pgdat->node_zones[ZONE_DMA].nr_active_pages;
-		pages += pgdat->node_zones[ZONE_DMA].nr_inactive_pages;
-		pages += pgdat->node_zones[ZONE_NORMAL].free_pages;
-		pages += pgdat->node_zones[ZONE_NORMAL].nr_active_pages;
-		pages += pgdat->node_zones[ZONE_NORMAL].nr_inactive_pages;
-	}
-
-	return pages;
-}
 #endif
 
 #define K(x) ((x) << (PAGE_SHIFT-10))

Re: Linux 2.4.25-pre6

[David Woodhouse]

On Wed, 2004-01-21 at 18:12 -0200, Marcelo Tosatti wrote:
> Lets try the clueless approach and remove the inode reclaim highmem fixes
> from Rik.
> 
> Please revert the attached patch (againts -pre6).

Did this make a difference?

-- 
dwmw2

Re: Linux 2.4.25-pre6

[Lukasz Trabinski]

On Wed, 28 Jan 2004, David Woodhouse wrote:

Hello.

> On Wed, 2004-01-21 at 18:12 -0200, Marcelo Tosatti wrote:
> > Lets try the clueless approach and remove the inode reclaim highmem fixes
> > from Rik.
> > 
> > Please revert the attached patch (againts -pre6).
> 
> Did this make a difference?

Machine works for 5 days without any crash or any oops.

oceanic:~# uptime
 17:58:56  up 5 days, 22:08, 28 users,  load average: 0.41, 0.53, 0.60

2.4.25-pre6 + nohighmem.patch


-- 
*[ Łukasz Trąbiński ]*
SysAdmin @wsisiz.edu.pl

Re: Linux 2.4.25-pre6

[Lukasz Trabinski]

On Wed, 28 Jan 2004, David Woodhouse wrote:

> On Wed, 2004-01-21 at 18:12 -0200, Marcelo Tosatti wrote:
> > Lets try the clueless approach and remove the inode reclaim highmem fixes
> > from Rik.
> > 
> > Please revert the attached patch (againts -pre6).
> 
> Did this make a difference?

I think so

oceanic:~$ uptime
 23:23:12  up 11 days,  3:32, 29 users,  load average: 0.21, 0.52, 1.10

2.4.25-pre6 +  nohighmem.patch


-- 
*[ Łukasz Trąbiński ]*
SysAdmin @wsisiz.edu.pl

Re: Linux 2.4.25-pre6

[Marcelo Tosatti]

Hi Lukasz,

I wonder if this SCSI errors might have anything to do with your problem.

I'm reluctant to revert riel's patch because it does not seem to be the
cause for such problems --- it is pretty straighforward and no one can see
why it would corrupt the inode list (as per dwmw2's investigation).

When/how often do this SCSI errors messages happen ? When you saw the
lockup, which driver were you using? (Justin's latest or 2.4.25 vanilla
aic7xxx).

Justin, can you take a look at this log and enlight me, please? Lukasz
says with your newer driver, this does not happen.

On Thu, 22 Jan 2004, Lukasz Trabinski wrote:

> On Thu, 22 Jan 2004, Marcelo Tosatti wrote:
>
> > > oceanic:~$ w |grep users
> > >  12:50:31  up 1 day,  6:09, 39 users,  load average: 0.70, 0.96, 1.15
> > > oceanic:~$ smbstatus -b |wc -l
> > >     138
> >
> > Are you running what kernel now? :)
>
> oceanic:~$ uname -r
> 2.4.25-pre6
>
> with newes driver scsi from http://people.freebsd.org/~gibbs/linux/SRC
> With orginal driver from 2.4.25-pre6 i had messages like this (maybe is
> not driver fault, i don't know):
>
> -------------------------------------------------------
> Jan 16 10:30:07 oceanic kernel: scsi0: FIFO0 Free, LONGJMP == 0x80ff, SCB
> 0x19
> Jan 16 10:30:07 oceanic kernel: scsi0: FIFO1 Free, LONGJMP == 0x8278, SCB
> 0x29
> Jan 16 10:30:07 oceanic kernel: scsi0: LQISTATE = 0x0, LQOSTATE = 0x0,
> OPTIONMODE = 0x42
> Jan 16 10:30:07 oceanic kernel: scsi0: OS_SPACE_CNT = 0x20 MAXCMDCNT = 0x1
> Jan 16 10:30:07 oceanic kernel: scsi0: REG0 == 0x29, SINDEX = 0x133,
> DINDEX = 0x106
> Jan 16 10:30:07 oceanic kernel: scsi0: SCBPTR == 0x29, SCB_NEXT == 0xffc0,
> SCB_NEXT2 == 0xff7e
> Jan 16 10:30:07 oceanic kernel: (scsi0:A:0:0): Device is disconnected,
> re-queuing SCB
> Jan 16 10:30:07 oceanic kernel: (scsi0:A:0:0): Task Management Func 0x1
> CompleteJan 16 10:30:07 oceanic kernel: scsi0:0:0:0: Attempting to abort
> cmd f76d0800: 0x0 0x0 0x0 0x0 0x0 0x0
> Jan 16 10:30:07 oceanic kernel: scsi0: At time of recovery, card was not
> paused
> Jan 16 10:30:07 oceanic kernel: scsi0: Dumping Card State at program
> address 0x27 Mode 0x22
> Jan 16 10:30:07 oceanic kernel: scsi0: FIFO0 Free, LONGJMP == 0x80ff, SCB
> 0x19
> Jan 16 10:30:07 oceanic kernel: scsi0: FIFO1 Free, LONGJMP == 0x8278, SCB
> 0x29
> Jan 16 10:30:07 oceanic kernel: scsi0: LQISTATE = 0x0, LQOSTATE = 0x0,
> OPTIONMODE = 0x42
> Jan 16 10:30:07 oceanic kernel: scsi0: OS_SPACE_CNT = 0x20 MAXCMDCNT = 0x1
> Jan 16 10:30:07 oceanic kernel: scsi0: REG0 == 0x3b, SINDEX = 0x122,
> DINDEX = 0x102
> Jan 16 10:30:07 oceanic kernel: scsi0: SCBPTR == 0xff3b, SCB_NEXT ==
> 0xff00, SCB_NEXT2 == 0x0
> Jan 16 10:30:07 oceanic kernel: scsi0:0:0:0: Unable to deliver message
> Jan 16 10:30:07 oceanic kernel: scsi0:0:0:0: Attempting to abort cmd
> f76d0a00: 0x2a 0x0 0x7 0xfa 0xa4 0x4e 0x0 0x0 0x88 0x0
> Jan 16 10:30:07 oceanic kernel: scsi0: At time of recovery, card was not
> paused
> Jan 16 10:30:07 oceanic kernel: scsi0: Dumping Card State at program
> address 0x94 Mode 0x33
> Jan 16 10:30:07 oceanic kernel: scsi0: FIFO0 Free, LONGJMP == 0x80ff, SCB
> 0x19
> Jan 16 10:30:07 oceanic kernel: scsi0: FIFO1 Free, LONGJMP == 0x8278, SCB
> 0x29
> Jan 16 10:30:07 oceanic kernel: scsi0: LQISTATE = 0x0, LQOSTATE = 0x0,
> OPTIONMODE = 0x42
> Jan 16 10:30:07 oceanic kernel: scsi0: OS_SPACE_CNT = 0x20 MAXCMDCNT = 0x1
> Jan 16 10:30:07 oceanic kernel: scsi0: REG0 == 0x29, SINDEX = 0x100,
> DINDEX = 0x102
> Jan 16 10:30:07 oceanic kernel: scsi0: SCBPTR == 0x3b, SCB_NEXT == 0xff00,
> SCB_NEXT2 == 0xff3c
> Jan 16 10:30:07 oceanic kernel: (scsi0:A:0:0): Device is disconnected,
> re-queuing SCB
> Jan 16 10:30:07 oceanic kernel: (scsi0:A:0:0): Task Management Func 0x1
> CompleteJan 16 10:30:07 oceanic kernel: scsi0:0:0:0: Attempting to abort
> cmd f76d0800: 0x0 0x0 0x0 0x0 0x0 0x0
> Jan 16 10:30:07 oceanic kernel: scsi0: At time of recovery, card was not
> paused
> Jan 16 10:30:07 oceanic kernel: scsi0: Dumping Card State at program
> address 0x27 Mode 0x22
> Jan 16 10:30:07 oceanic kernel: scsi0: FIFO0 Free, LONGJMP == 0x80ff, SCB
> 0x19
> Jan 16 10:30:07 oceanic kernel: scsi0: FIFO1 Free, LONGJMP == 0x8278, SCB
> 0x29
> Jan 16 10:30:07 oceanic kernel: scsi0: LQISTATE = 0x0, LQOSTATE = 0x0,
> OPTIONMODE = 0x42
> Jan 16 10:30:07 oceanic kernel: scsi0: OS_SPACE_CNT = 0x20 MAXCMDCNT = 0x1
> Jan 16 10:30:07 oceanic kernel: scsi0: REG0 == 0x3b, SINDEX = 0x122,
> DINDEX = 0x102

Re: Linux 2.4.25-pre6

[Lukasz Trabinski]

On Wed, 4 Feb 2004, Marcelo Tosatti wrote:

> I wonder if this SCSI errors might have anything to do with your problem.
> I'm reluctant to revert riel's patch because it does not seem to be the
> cause for such problems --- it is pretty straighforward and no one can see
> why it would corrupt the inode list (as per dwmw2's investigation).

> When/how often do this SCSI errors messages happen ? When you saw the
> lockup, which driver were you using? (Justin's latest or 2.4.25 vanilla
> aic7xxx).

I had tested new driver 2.4-20031222 with 2.4.25-pre6 and machine had 
permanent load about 1-1,5, commands like `ps aux` or `w` was take about 10
secunds. I had also problem with reboot, it stoped on "quota turn off", 
SysRq  didn't work. On 22 Jan 2004 I have sent dump from aic79xx 
controller to gibbs@scsiguy.com

With vanilla 2.4.25-pre6 +nohighmem.patch I have'n any problems
from 12 days. No ooops, no SCSI errors. :-)
UnfortunatelyI don't know that is it hardware problem, driver problem or
something else, that's why i have sent question to LKML.


Earlier SCSI errors (2.4.24 and 2.4.24-preX kernels):

oceanic:/var/log$ zcat messages.* |grep scsi0 |grep "Bus Device"
Nov 17 19:44:26 oceanic kernel: (scsi0:A:0:0): Bus Device Reset Message Sent
Nov 17 19:44:26 oceanic kernel: scsi0: Bus Device Reset on A:0. 1 SCBs aborted
Nov  3 08:48:24 oceanic kernel: (scsi0:A:0:0): Bus Device Reset Message Sent
Nov  3 08:48:24 oceanic kernel: scsi0: Bus Device Reset on A:0. 1 SCBs aborted
Oct 29 17:21:42 oceanic kernel: (scsi0:A:0:0): Bus Device Reset Message Sent
Oct 29 17:21:42 oceanic kernel: scsi0: Bus Device Reset on A:0. 1 SCBs aborted
aborted


Tech info:

Adaptec AIC7902 Ultra320 SCSI adapter
aic7902: Ultra320 Wide Channel A, SCSI Id=7, PCI-X 67-100Mhz, 512 SCBs
Allocated SCBs: 96, SG List Length: 85

Disks:
SEAGATE ST373307LW (on this disk was problem) X 1
SEAGATE ST373453LW X 4


-- 
*[ Łukasz Trąbiński ]*

Re: Linux 2.4.25-pre6

[Lukasz Trabinski]

On Wed, 4 Feb 2004, Lukasz Trabinski wrote:

> With vanilla 2.4.25-pre6 +nohighmem.patch I have'n any problems
> from 12 days. No ooops, no SCSI errors. :-)
> UnfortunatelyI don't know that is it hardware problem, driver problem or
> something else, that's why i have sent question to LKML.

I have got today messages like this:

Feb  4 16:18:21 oceanic kernel: EXT3-fs error (device sd(8,66)): 
ext3_free_blocks: Freeing blocks not in datazone - block = 1852787726, 
count = 1
Feb  4 16:18:21 oceanic kernel: EXT3-fs error (device sd(8,66)): 
ext3_free_blocks: Freeing blocks not in datazone - block = 1702112615, 
count = 1
Feb  4 16:18:21 oceanic kernel: EXT3-fs error (device sd(8,66)): 
ext3_free_blocks: Freeing blocks not in datazone - block = 1679846770, 
count = 1
Feb  4 16:18:21 oceanic kernel: EXT3-fs error (device sd(8,66)): 
ext3_free_blocks: Freeing blocks not in datazone - block = 242508389, 
count = 1

and more...


-- 
*[ Łukasz Trąbiński ]*
SysAdmin @wsisiz.edu.pl

Re: Linux 2.4.25-pre6

[Marcelo Tosatti]

On Wed, 21 Jan 2004, Lukasz Trabinski wrote:

> On Tue, 20 Jan 2004, Marcelo Tosatti wrote:
> > > Output from console SysRq showTasks, showMem and showTasks you can see
> > > here:
> > >
> > > http://lukasz.eu.org/minicom.txt
> > Hi Lukasz,
> >
> > Can you please run the output through ksymoops?
>
> Here is:
> http://lukasz.eu.org/ksymoops.txt

Hi again Lukasz,

I forgot to ask you: What compiler are you using?

Re: Linux 2.4.25-pre6

[David Woodhouse]

On Wed, 2004-01-21 at 07:47 +0100, Lukasz Trabinski wrote:
> Here is:
> http://lukasz.eu.org/ksymoops.txt

Some weirdness in there. This is init, for example. Apparently in R
state (as are 260-odd other processes), and:

>>EIP; c02d5d38 <contig_page_data+298/3b8>   <=====

Trace; c0106cfc <setup_frame+11c/210>
Trace; c0115ba8 <schedule_timeout+58/b0>
Trace; c013849c <__get_free_pages+1c/20>
Trace; c0115b30 <process_timeout+0/20>
Trace; c014be47 <pipe_poll+37/80>
Trace; c015315d <do_select+12d/250>

>From what I can tell, there are many people waiting on the inode_lock,
because kswapd has taken it and then taken a page fault....

Trace; c0130018 <filemap_nopage+f8/220>
Trace; c01148f0 <do_page_fault+0/5e4>
Trace; c01077f0 <error_code+34/3c>
Trace; c0158fb7 <refile_inode+47/a0>
Trace; c012d7a2 <__remove_inode_page+82/90>

I wonder if one of the lists is corrupt. I'd like to know _precisely_
what's at refile_inode+47 in your build. Please could you run 'gdb
vmlinux' and tell it 'disass refile_inode'. Then save the original
vmlinux somewhere, run 'make CFLAGS_inode.o=-g SUBDIRS=fs vmlinux', load
it in GDB again and repeat the disassembly, followed by 'list
*0xc0158fb7'? 

I assume this is a clean 2.4.25-pre6 build with no other patches or
modules?

-- 
dwmw2