ext3 journal/IDE problems ?

ext3 journal/IDE problems ?

[Bill Huey]

Hello,

What's going on with this ?

I get:

===========================================================

EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753210
Aborting journal on device ide0(3,5).
Remounting filesystem read-only
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753211
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753212
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753213
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753214
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753215
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753216
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753217
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753218
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753219
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753220
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753221
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753222
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753223
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753224
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753225
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753226
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753227
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753228
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753229
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753230
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753231
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753232
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753233
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753234
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753235
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753236
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753237
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753238
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753239
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753240
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753241
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753242
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753243
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753244
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753245
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753246
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753247
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753248
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753249
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753250
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753251
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753252
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753253
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753254
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753255
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753256
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753257
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753258
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753259
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753260
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753261
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753262
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753263
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753264
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753265
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753266
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753267
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753268
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753269
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753270
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753271
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753272
EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753273
ext3_free_blocks: aborting transaction: Journal has aborted in __ext3_journal_get_undo_access<2>EXT3-fs error (device ide0(3,5)) in ext3_free_blocks: Journal has aborted
ext3_reserve_inode_write: aborting transaction: Journal has aborted in __ext3_journal_get_write_access<2>EXT3-fs error (device ide0(3,5)) in ext3_reserve_inode_write: Journal has aborted
EXT3-fs error (device ide0(3,5)) in ext3_truncate: Journal has aborted
ext3_reserve_inode_write: aborting transaction: Journal has aborted in __ext3_journal_get_write_access<2>EXT3-fs error (device ide0(3,5)) in ext3_reserve_inode_write: Journal has aborted
EXT3-fs error (device ide0(3,5)) in ext3_orphan_del: Journal has aborted
ext3_reserve_inode_write: aborting transaction: Journal has aborted in __ext3_journal_get_write_access<2>EXT3-fs error (device ide0(3,5)) in ext3_reserve_inode_write: Journal has aborted
EXT3-fs error (device ide0(3,5)) in ext3_delete_inode: Journal has aborted

===========================================================

Re: ext3 journal/IDE problems ?

[Andreas Dilger]

On Aug 08, 2002  21:04 -0700, Bill Huey wrote:
> What's going on with this ?
> 
> I get:
> EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753210
> :
> :
> EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753273
> ext3_free_blocks: aborting transaction: Journal has aborted in __ext3_journal_get_undo_access<2>EXT3-fs error (device ide0(3,5)) in ext3_free_blocks: Journal has aborted

Looks like you got a block of zeros from disk when it should have been a
block bitmap, or your filesystem is otherwise corrupted.  You need to do
a full fsck on this filesystem.

As for cause, I have no idea.  IDE DMA, IDE cables, memory, kernel bug...

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: ext3 journal/IDE problems ? (softirq + sched assert ?)

[Bill Huey]

On Fri, Aug 09, 2002 at 12:37:25AM -0600, Andreas Dilger wrote:
> On Aug 08, 2002  21:04 -0700, Bill Huey wrote:
> > What's going on with this ?
> > 
> > I get:
> > EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753210
> > :
> > :
> > EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already cleared for block 753273
> > ext3_free_blocks: aborting transaction: Journal has aborted in __ext3_journal_get_undo_access<2>EXT3-fs error (device ide0(3,5)) in ext3_free_blocks: Journal has aborted
> 
> Looks like you got a block of zeros from disk when it should have been a
> block bitmap, or your filesystem is otherwise corrupted.  You need to do
> a full fsck on this filesystem.
> 
> As for cause, I have no idea.  IDE DMA, IDE cables, memory, kernel bug...

It's certainly a kernel bug. This machine has been reliable for years with
stable kernels and I started to run the 2.5 series over the last couple
of days just to test out the softirq stuff (+ preempt) and latency. I
applied Mingo's patch for a recent scheduler assertion bug and it looks
like some kind of race is possibly happening in the IDE layer.

And Mingo, yes, the patch you sent me did help, but it looks like there's
other problems.

Thanks

bill

Re: ext3 journal/IDE problems ?

[Greg Fitzgerald]

I have been having problems with ext3 hardlocking while in X and in
Console. Symptom one is while in console moving large ammounts of data
around between partions it will hardlock. Symptom two is sometimes I
will leave my computer for a few minutes (on a very rar occasion :P )
when i return x is hardlocked. Was running XFS and Resierfs before I
tried ext3 and never had these problems. Any information I can provide
let me know.

--Greg


On Thu, 8 Aug 2002 21:04:56 -0700
Bill Huey (Hui) <billh@gnuppy.monkey.org> wrote:

> 
> Hello,
> 
> What's going on with this ?
> 
> I get:
> 
> ===========================================================
> 
> EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753210 Aborting journal on device ide0(3,5).
> Remounting filesystem read-only
> EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753211 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753212 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753213 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753214 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753215 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753216 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753217 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753218 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753219 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753220 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753221 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753222 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753223 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753224 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753225 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753226 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753227 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753228 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753229 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753230 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753231 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753232 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753233 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753234 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753235 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753236 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753237 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753238 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753239 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753240 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753241 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753242 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753243 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753244 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753245 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753246 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753247 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753248 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753249 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753250 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753251 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753252 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753253 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753254 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753255 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753256 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753257 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753258 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753259 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753260 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753261 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753262 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753263 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753264 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753265 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753266 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753267 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753268 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753269 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753270 EXT3-fs error (device ide0(3,5)): ext3_free_blocks: bit already
> cleared for block 753271 EXT3-fs error (device ide0(3,5)):
> ext3_free_blocks: bit already cleared for block 753272 EXT3-fs error
> (device ide0(3,5)): ext3_free_blocks: bit already cleared for block
> 753273 ext3_free_blocks: aborting transaction: Journal has aborted in
> __ext3_journal_get_undo_access<2>EXT3-fs error (device ide0(3,5)) in
> ext3_free_blocks: Journal has aborted ext3_reserve_inode_write:
> aborting transaction: Journal has aborted in
> __ext3_journal_get_write_access<2>EXT3-fs error (device ide0(3,5)) in
> ext3_reserve_inode_write: Journal has aborted EXT3-fs error (device
> ide0(3,5)) in ext3_truncate: Journal has aborted
> ext3_reserve_inode_write: aborting transaction: Journal has aborted in
> __ext3_journal_get_write_access<2>EXT3-fs error (device ide0(3,5)) in
> ext3_reserve_inode_write: Journal has aborted EXT3-fs error (device
> ide0(3,5)) in ext3_orphan_del: Journal has aborted
> ext3_reserve_inode_write: aborting transaction: Journal has aborted in
> __ext3_journal_get_write_access<2>EXT3-fs error (device ide0(3,5)) in
> ext3_reserve_inode_write: Journal has aborted EXT3-fs error (device
> ide0(3,5)) in ext3_delete_inode: Journal has aborted
> 
> ===========================================================
> -
> To unsubscribe from this list: send the line "unsubscribe
> linux-kernel" in the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: ext3 journal/IDE problems ?

[Bill Huey]

On Fri, Aug 09, 2002 at 11:09:38AM -0400, Greg Fitzgerald wrote:
> I have been having problems with ext3 hardlocking while in X and in
> Console. Symptom one is while in console moving large ammounts of data
> around between partions it will hardlock. Symptom two is sometimes I
> will leave my computer for a few minutes (on a very rar occasion :P )
> when i return x is hardlocked. Was running XFS and Resierfs before I
> tried ext3 and never had these problems. Any information I can provide
> let me know.

I use to get those hard locks, but I wasn't sure if was my hardware or
the kernel using preempt. After using 2.5.30 for a while, those lock ups
went away, but the progression to this bug happen first with a scheduler
crash in the TCP/IP stack being triggered and then after that application of
Mingo's patch (from a previous email) that fixed the crashes, it started
to trigger ext3 assertions.

It's a very strange bug. Not sure what to say about it.

bill