ext3 seems to ignore ECC errors

ext3 seems to ignore ECC errors

[Samuel Thibault]

Hello,

I am having qemu report to its guest that a device is read-only (because
the backend format, file or device is) by returning ECC errors instead
of letting the guest believe that the writes went fine, see patch below.

However, to my surprise Linux (2.6.26) continued to mount the ext3
filesystem read/write, spitting out a lot of IDE errors.  Shouldn't
filesystems remount read-only and let the admin try to save data in such
case?

Samuel

Index: hw/ide.c
===================================================================
--- hw/ide.c	(révision 4938)
+++ hw/ide.c	(copie de travail)
@@ -891,7 +891,6 @@
     return 1;
 }
 
-/* XXX: handle errors */
 static void ide_read_dma_cb(void *opaque, int ret)
 {
     BMDMAState *bm = opaque;
@@ -899,6 +898,14 @@
     int n;
     int64_t sector_num;
 
+    if (ret) {
+        s->status = READY_STAT | ERR_STAT;
+        s->error = ABRT_ERR | ECC_ERR;
+        s->nsector = 0;
+        ide_set_irq(s);
+        goto eot;
+    }
+
     n = s->io_buffer_size >> 9;
     sector_num = ide_get_sector(s);
     if (n > 0) {
@@ -992,7 +999,6 @@
     }
 }
 
-/* XXX: handle errors */
 static void ide_write_dma_cb(void *opaque, int ret)
 {
     BMDMAState *bm = opaque;
@@ -1000,6 +1006,14 @@
     int n;
     int64_t sector_num;
 
+    if (ret) {
+        s->status = READY_STAT | ERR_STAT;
+        s->error = ABRT_ERR | ECC_ERR;
+        s->nsector = 0;
+        ide_set_irq(s);
+        goto eot;
+    }
+
     n = s->io_buffer_size >> 9;
     sector_num = ide_get_sector(s);
     if (n > 0) {

Re: ext3 seems to ignore ECC errors

[Samuel Thibault]

Samuel Thibault, le Mon 04 Aug 2008 16:03:53 +0100, a écrit :
> However, to my surprise Linux (2.6.26) continued to mount the ext3
> filesystem read/write, spitting out a lot of IDE errors.

Note: these look like

hdb: dma_intr: status=0x41 { DriveReady Error }
hdb: dma_intr: error=0x44 { DriveStatusError UncorrectableError }, LBAsect=1189689234687, sector=70911
ide: failed opcode was: unknown
end_request: I/O/error, dev hdb sector 70911
Buffer I/O error on device hdb1, logical block 8856
lost page write due to I/O error on hdb1
etc.

So it seems the error is properly reported up to the filesystem
((70911-63) * 512 / 4096 == 8856)

Samuel

Re: ext3 seems to ignore ECC errors

[Andreas Schwab]

Samuel Thibault <samuel.thibault@ens-lyon.org> writes:

> However, to my surprise Linux (2.6.26) continued to mount the ext3
> filesystem read/write, spitting out a lot of IDE errors.  Shouldn't
> filesystems remount read-only and let the admin try to save data in such
> case?

That's a tunable parameter of the filesystem, see tune2fs(8).

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: ext3 seems to ignore ECC errors

[Samuel Thibault]

Andreas Schwab, le Mon 04 Aug 2008 17:39:06 +0200, a écrit :
> Samuel Thibault <samuel.thibault@ens-lyon.org> writes:
> > However, to my surprise Linux (2.6.26) continued to mount the ext3
> > filesystem read/write, spitting out a lot of IDE errors.  Shouldn't
> > filesystems remount read-only and let the admin try to save data in such
> > case?
> 
> That's a tunable parameter of the filesystem, see tune2fs(8).

Oh, I had assumed it was on by default, hum.  But now it reads

Errors behavior:          Remount read-only

and I am still getting the same behavior.  Indeed after some VFS
modifications time it gets remounted read-only, but shouldn't that
happen as soon as possible?

Samuel

Re: ext3 seems to ignore ECC errors

[Andreas Schwab]

Samuel Thibault <samuel.thibault@ens-lyon.org> writes:

> Andreas Schwab, le Mon 04 Aug 2008 17:39:06 +0200, a écrit :
>> Samuel Thibault <samuel.thibault@ens-lyon.org> writes:
>> > However, to my surprise Linux (2.6.26) continued to mount the ext3
>> > filesystem read/write, spitting out a lot of IDE errors.  Shouldn't
>> > filesystems remount read-only and let the admin try to save data in such
>> > case?
>> 
>> That's a tunable parameter of the filesystem, see tune2fs(8).
>
> Oh, I had assumed it was on by default, hum.  But now it reads
>
> Errors behavior:          Remount read-only
>
> and I am still getting the same behavior.  Indeed after some VFS
> modifications time it gets remounted read-only, but shouldn't that
> happen as soon as possible?

The block device may do some retries before passing the error on.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: ext3 seems to ignore ECC errors

[Samuel Thibault]

Andreas Schwab, le Mon 04 Aug 2008 18:12:54 +0200, a écrit :
> Samuel Thibault <samuel.thibault@ens-lyon.org> writes:
> 
> > Andreas Schwab, le Mon 04 Aug 2008 17:39:06 +0200, a écrit :
> >> Samuel Thibault <samuel.thibault@ens-lyon.org> writes:
> >> > However, to my surprise Linux (2.6.26) continued to mount the ext3
> >> > filesystem read/write, spitting out a lot of IDE errors.  Shouldn't
> >> > filesystems remount read-only and let the admin try to save data in such
> >> > case?
> >> 
> >> That's a tunable parameter of the filesystem, see tune2fs(8).
> >
> > Oh, I had assumed it was on by default, hum.  But now it reads
> >
> > Errors behavior:          Remount read-only
> >
> > and I am still getting the same behavior.  Indeed after some VFS
> > modifications time it gets remounted read-only, but shouldn't that
> > happen as soon as possible?
> 
> The block device may do some retries before passing the error on.

Sure.  But I forgot to mention that after spitting all these write
errors, the filesystem remains mounted read-write.

Samuel

Re: ext3 seems to ignore ECC errors

[Theodore Tso]

On Mon, Aug 04, 2008 at 04:47:27PM +0100, Samuel Thibault wrote:
> Andreas Schwab, le Mon 04 Aug 2008 17:39:06 +0200, a écrit :
> > Samuel Thibault <samuel.thibault@ens-lyon.org> writes:
> > > However, to my surprise Linux (2.6.26) continued to mount the ext3
> > > filesystem read/write, spitting out a lot of IDE errors.  Shouldn't
> > > filesystems remount read-only and let the admin try to save data in such
> > > case?
> > 
> > That's a tunable parameter of the filesystem, see tune2fs(8).
> 
> Oh, I had assumed it was on by default, hum.  But now it reads
> 
> Errors behavior:          Remount read-only
> 
> and I am still getting the same behavior.  Indeed after some VFS
> modifications time it gets remounted read-only, but shouldn't that
> happen as soon as possible?

Errors writing to data blocks should get reflected up the application
(i.e., as EIO errors), but it won't force the filesystem read/only or
force a kernel panic (in the "tune2fs -e panic" case).  The rationale
being if it's a single isloated error writing to a single file, why
ruin everybody's day by taking more drastic action?

But if we can't write to core filesystem data structures, that's a
much more serious situation....

						- Ted