linux-lvm.redhat.com archive mirror
 help / color / mirror / Atom feed
* [linux-lvm] Oops unmounting snapshot of xfs filesystem
@ 2002-02-08 13:51 Stephenson, Dale
  2002-02-08 18:44 ` [linux-lvm] " Adrian Head
  2002-02-11  4:46 ` Klaus Strebel
  0 siblings, 2 replies; 5+ messages in thread
From: Stephenson, Dale @ 2002-02-08 13:51 UTC (permalink / raw)
  To: 'linux-xfs@oss.sgi.com', 'linux-lvm@sistina.com'

[-- Attachment #1: Type: text/plain, Size: 2364 bytes --]

I have been running into an oops when umounting the snapshot of an xfs
filesystem, or following the umount.  For instance, a umount followed by an
lvremove will oops in the lvremove, while a mount/umount/mount/umount
sequence will oops on the second or third umount.

What does seem consistent is the error message logged on each umount.  Here
are the messages from a mount/umount sequence:

<4> Feb  7 11:20:00 kernel: Mounting filesystem "lvm(58,1)" in no-recovery
mode.  Filesystem will be inconsistent. 
<2> Feb  7 11:20:02 kernel: lvm - lvm_map: ll_rw_blk write for readonly LV
/dev/volgr0/lvol1 
<2> Feb  7 11:20:02 kernel: lvm - lvm_map: ll_rw_blk write for readonly LV
/dev/volgr0/lvol1 
<4> Feb  7 11:20:02 kernel: I/O error in filesystem ("lvm(58,1)") meta-data
dev 0x3a01 block 0xa00020 
<4> Feb  7 11:20:02 kernel:        ("xlog_iodone") error 5 buf count 1024 
<4> Feb  7 11:20:02 kernel: xfs_force_shutdown(lvm(58,1),0x2) called from
line 939 of file xfs_log.c.  Return address = 0xc01b3cbc 
<4> Feb  7 11:20:02 kernel: Log I/O Error Detected.  Shutting down
filesystem: lvm(58,1) 
<4> Feb  7 11:20:02 kernel: Please umount the filesystem, and rectify the
problem(s) 

I've attached the oops (run through ksymoops) from this particular umount.
The snapshot is mounted ro,nouuid,norecovery.  The source of the snapshot
contains an unmounted xfs filesystem, which was unmounted at the time of
snapshot creation.  There has been no intentional I/O activity to the
snapshot, either.

It looks to me like xfs is trying to flush something to disk on the umount,
even though the filesystem is read only and no recovery.  I would not have
expected this.  Is it correct behavior?

I made the snapshot writeable, but kept the mount options the same.  I was
able to mount/umount many times without oops, I/O errors, or
xfs_force_shutdown.

I did notice similar behavior when a full snapshot returns an I/O error --
xfs_force_shutdown, with an oops following soon thereafter.

System details:
Celeron with 512 MB of RAM and WD 45GB harddrives.
128 MB swap, one vg consisting of a one-drive RAID 0.
Kernel 2.4.16 with 12/14/01 xfs CVS.
LVM CVS of 1/21/02 (functionally identical to 1.0.2, I believe).
LVM's linux-2.4.11-VFS-lock.patch.
xfs_fs_freeze() patch posted by Eric Sandeen.
Anselm Kruis' writable snapshot patch.

Dale J. Stephenson
steph@snapserver.com


[-- Attachment #2: oops.out --]
[-- Type: application/octet-stream, Size: 3919 bytes --]

ksymoops 2.4.0 on i686 2.4.9-2buildsmp.  Options used
     -V (default)
     -k ksyms (specified)
     -L (specified)
     -O (specified)
     -M (specified)

Error (expand_objects): cannot stat(/lib/modules/2.4.16-2snap/kernel/net/appletalk/appletalk.o) for appletalk
Error (expand_objects): cannot stat(/lib/modules/2.4.16-2snap/kernel/drivers/i2c/adm1024.o) for adm1024
Error (expand_objects): cannot stat(/lib/modules/2.4.16-2snap/kernel/drivers/i2c/i2c-proc.o) for i2c-proc
Error (expand_objects): cannot stat(/lib/modules/2.4.16-2snap/kernel/drivers/i2c/i2c-dev.o) for i2c-dev
Error (expand_objects): cannot stat(/lib/modules/2.4.16-2snap/kernel/drivers/i2c/i2c-core.o) for i2c-core
Error (expand_objects): cannot stat(/lib/modules/2.4.16-2snap/kernel/drivers/net/eepro100.o) for eepro100
Unable to handle kernel NULL pointer dereference at virtual address 00000020 
c012ffe3 
*pde = 00000000 
Oops: 0000 
CPU:    0 
EIP:    0010:[<c012ffe3>]    Not tainted 
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010206 
eax: 00003a01   ebx: 00000000   ecx: 00000160   edx: 00000000 
esi: 00000011   edi: 00000000   ebp: 00000000   esp: c2217d48 
ds: 0018   es: 0018   ss: 0018 
Process umount (pid: 2010, stackpage=c2217000) 
Stack: 00003a01 00000000 00000000 c0133dc4 00003a01 c033b120 00000160 c3e40360  
       00000001 c2216000 c132e000 c01300df c3e40360 00000001 c3722200 00000000  
       c016fb50 00003a01 00000001 00000000 c01cab1d c3722200 c132e000 c1240000  
Call Trace: [<c0133dc4>] [<c01300df>] [<c016fb50>] [<c01cab1d>] [<c01b3cbc>]  
   [<c016c628>] [<c016cfa5>] [<c01b3cf4>] [<c01b4407>] [<c01c93d2>] [<c01bfb31>]  
   [<c01b3b8b>] [<c01b5834>] [<c01b598c>] [<c01b35e9>] [<c01bb211>] [<c01c25ea>]  
   [<c01cc2e4>] [<c01d33cf>] [<c01335ac>] [<c0136f78>] [<c0142a71>] [<c012e573>]  
   [<c0121752>] [<c0142a8c>] [<c0106efb>]  
Code: 8b 7b 20 66 39 43 0c 0f 85 86 00 00 00 8b 53 30 85 d2 0f 84  

>>EIP; c012ffe3 <invalidate_bdev+53/130>   <=====
Trace; c0133dc4 <bdget+34/120>
Trace; c01300df <__invalidate_buffers+1f/a0>
Trace; c016fb50 <pagebuf_target_clear+10/30>
Trace; c01cab1d <pagebuf_unlock+5afad/64440>
Trace; c01b3cbc <pagebuf_unlock+4414c/64440>
Trace; c016c628 <pagebuf_iodone+38/80>
Trace; c016cfa5 <pagebuf_iorequest+115/150>
Trace; c01b3cf4 <pagebuf_unlock+44184/64440>
Trace; c01b4407 <pagebuf_unlock+44897/64440>
Trace; c01c93d2 <pagebuf_unlock+59862/64440>
Trace; c01bfb31 <pagebuf_unlock+4ffc1/64440>
Trace; c01b3b8b <pagebuf_unlock+4401b/64440>
Trace; c01b5834 <pagebuf_unlock+45cc4/64440>
Trace; c01b598c <pagebuf_unlock+45e1c/64440>
Trace; c01b35e9 <pagebuf_unlock+43a79/64440>
Trace; c01bb211 <pagebuf_unlock+4b6a1/64440>
Trace; c01c25ea <pagebuf_unlock+52a7a/64440>
Trace; c01cc2e4 <pagebuf_unlock+5c774/64440>
Trace; c01d33cf <pagebuf_unlock+6385f/64440>
Trace; c01335ac <get_super+9fc/ca0>
Trace; c0136f78 <path_release+28/140>
Trace; c0142a71 <may_umount+261/7d10>
Trace; c012e573 <filp_close+53/60>
Trace; c0121752 <do_munmap+292/2b0>
Trace; c0142a8c <may_umount+27c/7d10>
Trace; c0106efb <__up_wakeup+112f/2474>
Code;  c012ffe3 <invalidate_bdev+53/130>
00000000 <_EIP>:
Code;  c012ffe3 <invalidate_bdev+53/130>   <=====
   0:   8b 7b 20                  mov    0x20(%ebx),%edi   <=====
Code;  c012ffe6 <invalidate_bdev+56/130>
   3:   66 39 43 0c               cmp    %ax,0xc(%ebx)
Code;  c012ffea <invalidate_bdev+5a/130>
   7:   0f 85 86 00 00 00         jne    93 <_EIP+0x93> c0130076 <invalidate_bdev+e6/130>
Code;  c012fff0 <invalidate_bdev+60/130>
   d:   8b 53 30                  mov    0x30(%ebx),%edx
Code;  c012fff3 <invalidate_bdev+63/130>
  10:   85 d2                     test   %edx,%edx
Code;  c012fff5 <invalidate_bdev+65/130>
  12:   0f 84 00 00 00 00         je     18 <_EIP+0x18> c012fffb <invalidate_bdev+6b/130>


6 errors issued.  Results may not be reliable.

^ permalink raw reply	[flat|nested] 5+ messages in thread
* [linux-lvm] RE: Oops unmounting snapshot of xfs filesystem
@ 2002-02-18 13:34 Stephenson, Dale
  2002-02-23 14:35 ` [linux-lvm] " Stephen Lord
  0 siblings, 1 reply; 5+ messages in thread
From: Stephenson, Dale @ 2002-02-18 13:34 UTC (permalink / raw)
  To: 'linux-xfs@oss.sgi.com', 'linux-lvm@sistina.com'

[-- Attachment #1: Type: text/plain, Size: 6180 bytes --]

Klaus Strebel's suggestion to use 2.91.66 (kgcc) for the kernel did solve
the oops from snapshot umounting, in ordinary circumstances.  Thanks!

However, I still have similar behavior, from umounting an invalid
(filled-up) snapshot.  I took three snapshots of a single xfs logical
volume, mounted the snapshots, and ran I-O on the source until the snapshots
were invalidated.  I then umounted two of the snapshots.  I saw
xfs_force_shutdown messages in syslog for each umount, and the second umount
oopsed.

Here's the syslog entries:

<6> Feb 14 15:24:11 kernel: lvm -- giving up to snapshot /dev/volgr0/lvol0
on /dev/volgr0/lvol1: out of space 
<6> Feb 14 15:24:11 kernel: lvm -- giving up to snapshot /dev/volgr0/lvol0
on /dev/volgr0/lvol2: out of space 
<6> Feb 14 15:24:11 kernel: lvm -- giving up to snapshot /dev/volgr0/lvol0
on /dev/volgr0/lvol3: out of space 
<1> Feb 14 15:24:43 kernel: lvm - lvm_map: ll_rw_blk for inactive LV
/dev/volgr0/lvol1 
<1> Feb 14 15:24:43 kernel: lvm - lvm_map: ll_rw_blk for inactive LV
/dev/volgr0/lvol1 
<4> Feb 14 15:24:43 kernel: I/O error in filesystem ("lvm(58,1)") meta-data
dev 0x3a01 block 0xa00020 
<4> Feb 14 15:24:43 kernel:        ("xlog_iodone") error 5 buf count 1024 
<4> Feb 14 15:24:43 kernel: xfs_force_shutdown(lvm(58,1),0x2) called from
line 939 of file xfs_log.c.  Return address = 0xc01b934e 
<4> Feb 14 15:24:43 kernel: Log I/O Error Detected.  Shutting down
filesystem: lvm(58,1) 
<4> Feb 14 15:24:43 kernel: Please umount the filesystem, and rectify the
problem(s) 
<1> Feb 14 15:24:48 kernel: lvm - lvm_map: ll_rw_blk for inactive LV
/dev/volgr0/lvol2 
<1> Feb 14 15:24:48 kernel: lvm - lvm_map: ll_rw_blk for inactive LV
/dev/volgr0/lvol2 
<4> Feb 14 15:24:48 kernel: I/O error in filesystem ("lvm(58,2)") meta-data
dev 0x3a02 block 0xa00020 
<4> Feb 14 15:24:48 kernel:        ("xlog_iodone") error 5 buf count 1024 
<4> Feb 14 15:24:48 kernel: xfs_force_shutdown(lvm(58,2),0x2) called from
line 939 of file xfs_log.c.  Return address = 0xc01b934e 
<4> Feb 14 15:24:48 kernel: Log I/O Error Detected.  Shutting down
filesystem: lvm(58,2) 
<4> Feb 14 15:24:48 kernel: Please umount the filesystem, and rectify the
problem(s) 
<1> Feb 14 15:24:48 kernel: Unable to handle kernel NULL pointer dereference
at virtual address 00000020 
[oops snipped, but output of ksymoops attached to message]

System details:
Celeron with 512 MB of RAM and WD 45GB harddrives.
128 MB swap, one vg consisting of a one-drive RAID 0.
Kernel 2.4.16 with 12/14/01 xfs CVS.
LVM CVS of 1/21/02 (functionally identical to 1.0.2, I believe).
LVM's linux-2.4.11-VFS-lock.patch.
xfs_fs_freeze() patch posted by Eric Sandeen.
Anselm Kruis' writable snapshot patch.

Snapshots were mounted ro,nouuid,norecovery.  The only action taken to the
snapshots was mounting.

I hope to try it soon with 2.4.17, as Adrian Head suggested.

Has anyone else run this case with xfs snapshots?

Why does umount result in I/O activity to a read-only, norecovery file
system?

Thanks,
Dale Stephenson
steph@snapserver.com

> -----Original Message-----
> From: Klaus Strebel [mailto:klaus.strebel@eigner.com]
> Sent: Monday, February 11, 2002 2:46 AM
> To: Stephenson, Dale
> Cc: 'linux-xfs@oss.sgi.com'; 'linux-lvm@sistina.com'
> Subject: Re: Oops unmounting snapshot of xfs filesystem
> 
> 
> Stephenson, Dale wrote:
> > I have been running into an oops when umounting the 
> snapshot of an xfs
> > filesystem, or following the umount.  For instance, a 
> umount followed by an
> > lvremove will oops in the lvremove, while a 
> mount/umount/mount/umount
> > sequence will oops on the second or third umount.
> > 
> > What does seem consistent is the error message logged on 
> each umount.  Here
> > are the messages from a mount/umount sequence:
> > 
> > <4> Feb  7 11:20:00 kernel: Mounting filesystem "lvm(58,1)" 
> in no-recovery
> > mode.  Filesystem will be inconsistent. 
> > <2> Feb  7 11:20:02 kernel: lvm - lvm_map: ll_rw_blk write 
> for readonly LV
> > /dev/volgr0/lvol1 
> > <2> Feb  7 11:20:02 kernel: lvm - lvm_map: ll_rw_blk write 
> for readonly LV
> > /dev/volgr0/lvol1 
> > <4> Feb  7 11:20:02 kernel: I/O error in filesystem 
> ("lvm(58,1)") meta-data
> > dev 0x3a01 block 0xa00020 
> > <4> Feb  7 11:20:02 kernel:        ("xlog_iodone") error 5 
> buf count 1024 
> > <4> Feb  7 11:20:02 kernel: 
> xfs_force_shutdown(lvm(58,1),0x2) called from
> > line 939 of file xfs_log.c.  Return address = 0xc01b3cbc 
> > <4> Feb  7 11:20:02 kernel: Log I/O Error Detected.  Shutting down
> > filesystem: lvm(58,1) 
> > <4> Feb  7 11:20:02 kernel: Please umount the filesystem, 
> and rectify the
> > problem(s) 
> > 
> > I've attached the oops (run through ksymoops) from this 
> particular umount.
> > The snapshot is mounted ro,nouuid,norecovery.  The source 
> of the snapshot
> > contains an unmounted xfs filesystem, which was unmounted 
> at the time of
> > snapshot creation.  There has been no intentional I/O 
> activity to the
> > snapshot, either.
> > 
> > It looks to me like xfs is trying to flush something to 
> disk on the umount,
> > even though the filesystem is read only and no recovery.  I 
> would not have
> > expected this.  Is it correct behavior?
> > 
> > I made the snapshot writeable, but kept the mount options 
> the same.  I was
> > able to mount/umount many times without oops, I/O errors, or
> > xfs_force_shutdown.
> > 
> > I did notice similar behavior when a full snapshot returns 
> an I/O error --
> > xfs_force_shutdown, with an oops following soon thereafter.
> > 
> > System details:
> > Celeron with 512 MB of RAM and WD 45GB harddrives.
> > 128 MB swap, one vg consisting of a one-drive RAID 0.
> > Kernel 2.4.16 with 12/14/01 xfs CVS.
> > LVM CVS of 1/21/02 (functionally identical to 1.0.2, I believe).
> > LVM's linux-2.4.11-VFS-lock.patch.
> > xfs_fs_freeze() patch posted by Eric Sandeen.
> > Anselm Kruis' writable snapshot patch.
> 
> Had the same, disappeared when compiled the kernel with gcc-2.91.66 ! 
> Seems to be a compiler issue.
> 
> Ciao
> Klaus
> 
> -- 
> Klaus Strebel
> UNIX-Engineer
> klaus.strebel@eigner.com
> EIGNER - Precision Lifecycle Management -
> <http://www.eigner.com>
> 


[-- Attachment #2: oops.out --]
[-- Type: application/octet-stream, Size: 3648 bytes --]

ksymoops 2.4.0 on i686 2.4.9-2buildsmp.  Options used
     -V (default)
     -k ksyms (specified)
     -L (specified)
     -O (specified)
     -M (specified)

Error (expand_objects): cannot stat(/lib/modules/2.4.16-2snap/kernel/net/appletalk/appletalk.o) for appletalk
Error (expand_objects): cannot stat(/lib/modules/2.4.16-2snap/kernel/drivers/i2c/adm1024.o) for adm1024
Error (expand_objects): cannot stat(/lib/modules/2.4.16-2snap/kernel/drivers/i2c/i2c-proc.o) for i2c-proc
Error (expand_objects): cannot stat(/lib/modules/2.4.16-2snap/kernel/drivers/i2c/i2c-dev.o) for i2c-dev
Error (expand_objects): cannot stat(/lib/modules/2.4.16-2snap/kernel/drivers/i2c/i2c-core.o) for i2c-core
Error (expand_objects): cannot stat(/lib/modules/2.4.16-2snap/kernel/drivers/net/eepro100.o) for eepro100
Unable to handle kernel NULL pointer dereference at virtual address 00000020 
c012ea83 
Oops: 0000 
CPU:    0 
EIP:    0010:[<c012ea83>]    Not tainted 
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010206 
eax: 00000000   ebx: 00000000   ecx: 00003a02   edx: 00003a02 
esi: 00000006   edi: 00000000   ebp: 00000000   esp: c0719d60 
ds: 0018   es: 0018   ss: 0018 
Process umount (pid: 6416, stackpage=c0719000) 
Stack: c3e403a0 00000000 c0718000 c2bfe000 3a020168 00000000 c012eb68 c3e403a0  
       00000001 c3ace220 c016f038 00003a02 00000001 00000000 c01d18c2 c3ace220  
       00000000 c3ada680 c2f50000 00000400 c01b934e c2bfe000 00000002 c02b27c7  
Call Trace: [<c012eb68>] [<c016f038>] [<c01d18c2>] [<c01b934e>] [<c016bcd8>]  
   [<c016c62c>] [<c01b9388>] [<c01b9aa2>] [<c01baf6a>] [<c01bb0b0>] [<c01b8c94>]  
   [<c01c1032>] [<c01c84d1>] [<c01d2c7a>] [<c01da7cc>] [<c0131ef8>] [<c014098e>]  
   [<c013582f>] [<c0141201>] [<c0120665>] [<c014121c>] [<c0106d7b>]  
Code: 8b 53 20 89 54 24 14 0f b7 54 24 12 66 39 53 0c 75 7b 83 7b  

>>EIP; c012ea83 <invalidate_bdev+3f/104>   <=====
Trace; c012eb68 <__invalidate_buffers+20/7c>
Trace; c016f038 <pagebuf_target_clear+10/2c>
Trace; c01d18c2 <pagebuf_unlock+6286e/6c37c>
Trace; c01b934e <pagebuf_unlock+4a2fa/6c37c>
Trace; c016bcd8 <pagebuf_iodone+38/7c>
Trace; c016c62c <pagebuf_iorequest+108/144>
Trace; c01b9388 <pagebuf_unlock+4a334/6c37c>
Trace; c01b9aa2 <pagebuf_unlock+4aa4e/6c37c>
Trace; c01baf6a <pagebuf_unlock+4bf16/6c37c>
Trace; c01bb0b0 <pagebuf_unlock+4c05c/6c37c>
Trace; c01b8c94 <pagebuf_unlock+49c40/6c37c>
Trace; c01c1032 <pagebuf_unlock+51fde/6c37c>
Trace; c01c84d1 <pagebuf_unlock+5947d/6c37c>
Trace; c01d2c7a <pagebuf_unlock+63c26/6c37c>
Trace; c01da7cc <pagebuf_unlock+6b778/6c37c>
Trace; c0131ef8 <get_super+9f8/c98>
Trace; c014098e <__mntput+1e/624>
Trace; c013582f <path_release+27/144>
Trace; c0141201 <may_umount+26d/826c>
Trace; c0120665 <do_munmap+279/298>
Trace; c014121c <may_umount+288/826c>
Trace; c0106d7b <__up_wakeup+1057/2274>
Code;  c012ea83 <invalidate_bdev+3f/104>
00000000 <_EIP>:
Code;  c012ea83 <invalidate_bdev+3f/104>   <=====
   0:   8b 53 20                  mov    0x20(%ebx),%edx   <=====
Code;  c012ea86 <invalidate_bdev+42/104>
   3:   89 54 24 14               mov    %edx,0x14(%esp,1)
Code;  c012ea8a <invalidate_bdev+46/104>
   7:   0f b7 54 24 12            movzwl 0x12(%esp,1),%edx
Code;  c012ea8f <invalidate_bdev+4b/104>
   c:   66 39 53 0c               cmp    %dx,0xc(%ebx)
Code;  c012ea93 <invalidate_bdev+4f/104>
  10:   75 7b                     jne    8d <_EIP+0x8d> c012eb10 <invalidate_bdev+cc/104>
Code;  c012ea95 <invalidate_bdev+51/104>
  12:   83 7b 00 00               cmpl   $0x0,0x0(%ebx)


6 errors issued.  Results may not be reliable.

^ permalink raw reply	[flat|nested] 5+ messages in thread
end of thread, other threads:[~2002-02-23 14:35 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2002-02-08 13:51 [linux-lvm] Oops unmounting snapshot of xfs filesystem Stephenson, Dale
2002-02-08 18:44 ` [linux-lvm] " Adrian Head
2002-02-11  4:46 ` Klaus Strebel
  -- strict thread matches above, loose matches on Subject: below --
2002-02-18 13:34 [linux-lvm] " Stephenson, Dale
2002-02-23 14:35 ` [linux-lvm] " Stephen Lord

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).