[linux-lvm] kernel: zam-7001: io error in reiserfs_find_entry

[linux-lvm] kernel: zam-7001: io error in reiserfs_find_entry

[Jürgen Vollmer]

[-- Attachment #1: Type: text/plain, Size: 3250 bytes --]


Hi everybody,

I use kernel-source-2.4.16.SuSE-24.i386.rpm together with reiserfs and
made a snapshot which I then tar'ed.
Writing the tape seems to be ok. After that I did:

	tar --directory=/ --file=/dev/tape --compare

and got the message (while working i.e. modifying the  original filesystem)

Message from syslogd@joergli at Fri Jan 11 10:57:19 2002 ...
joergli kernel: zam-7001: io error in reiserfs_find_entry


The entire message in /var/log/message is:
--------------
Jan 11 10:57:19 joergli kernel: lvm -- giving up to snapshot /dev/LVMhome_vg/home on /dev/LVMhome_vg/lvm-snapshot: out of space
Jan 11 10:57:19 joergli kernel: lvm - lvm_map: ll_rw_blk for inactive LV /dev/LVMhome_vg/lvm-snapshot
Jan 11 10:57:19 joergli kernel: zam-7001: io error in reiserfs_find_entry
Jan 11 10:57:19 joergli kernel: invalid operand: 0000
Jan 11 10:57:19 joergli kernel: CPU:    1
Jan 11 10:57:19 joergli kernel: EIP:    0010:[ipv6:__insmod_ipv6_O/lib/modules/2.4.16-SMP/kernel/net/ipv6/ipv6+-21171255/96]    Tainted: PF
Jan 11 10:57:19 joergli kernel: EFLAGS: 00010286
Jan 11 10:57:19 joergli kernel: eax: 0000002d   ebx: c1fdea40   ecx: 00000001   edx: 00000001
Jan 11 10:57:19 joergli kernel: esi: c3170800   edi: cde2dee0   ebp: cde2de74   esp: cde2de2c
Jan 11 10:57:19 joergli kernel: ds: 0018   es: 0018   ss: 0018
Jan 11 10:57:19 joergli kernel: Process tar (pid: 19852, stackpage=cde2d000)
Jan 11 10:57:19 joergli kernel: Stack: c1fe031a c1fe3aa0 c1fdea40 cde2de50 cde2de5c c1fdeaac c1fc3f36 c3170800
Jan 11 10:57:19 joergli kernel:        c1fdea40 00000000 c1fdeaac cde2dea0 0001e7e7 0001e7e8 05c75bff 000001f4
Jan 11 10:57:19 joergli kernel:        00000000 00000003 d7bace60 c1fc4050 c77b8520 d7bacec0 00000008 cde2dea0
Jan 11 10:57:19 joergli kernel: Call Trace: [ipv6:__insmod_ipv6_O/lib/modules/2.4.16-SMP/kernel/net/ipv6/ipv6+-21101798/96] [ipv6:__insmod_ipv6_O/lib/modules/2.4.16-SMP/kernel/net/ipv6/ipv6+-21087584/96] [ipv6:__insmod_ipv6_O/lib/modules/2.4.16-SMP/kernel/net/ipv6/ipv6+-21108160/96] [ipv6:__insmod_ipv6_O/lib/modules/2.4.16-SMP/kernel/net/ipv6/ipv6+-21108052/96] [ipv6:__insmod_ipv6_O/lib/modules/2.4.16-SMP/kernel/net/ipv6/ipv6+-21217482/96]
Jan 11 10:57:19 joergli kernel:    [ipv6:__insmod_ipv6_O/lib/modules/2.4.16-SMP/kernel/net/ipv6/ipv6+-21108160/96] [ipv6:__insmod_ipv6_O/lib/modules/2.4.16-SMP/kernel/net/ipv6/ipv6+-21108052/96] [ipv6:__insmod_ipv6_O/lib/modules/2.4.16-SMP/kernel/net/ipv6/ipv6+-21217200/96] [d_alloc+27/376] [real_lookup+122/264] [link_path_walk+1479/2080]
Jan 11 10:57:19 joergli kernel:    [path_walk+26/28] [__user_walk+53/80] [sys_lstat64+25/112] [sys_close+95/116] [system_call+51/56]
Jan 11 10:57:19 joergli kernel:
Jan 11 10:57:19 joergli kernel: Code: 0f 0b 68 a0 3a fe c1 85 f6 74 0f 0f b7 46 08 50 e8 32 77 16
--------------

So it seems that the snapshot is full. Is that correct, or do I have a serious
problem? Note: I'm doing my backups, so a dead backup is not a very good
one :-)

With best regards

Jürgen


-- 
Dr.rer.nat. Juergen Vollmer, Viktoriastrasse 15, D-76133 Karlsruhe
Tel: +49(721) 9204871 Fax: +49(721) 24874
juergen@informatik-vollmer.de,vollmer@cocolab.de,Juergen.Vollmer@acm.org
www.informatik-vollmer.de



[-- Attachment #2: Type: application/pgp-signature, Size: 228 bytes --]

Re: [linux-lvm] kernel: zam-7001: io error in reiserfs_find_entry

[Patrick Caulfield]

On Fri, Jan 11, 2002 at 11:30:17AM +0100, J?rgen Vollmer wrote:
> 
> Hi everybody,
> 
> I use kernel-source-2.4.16.SuSE-24.i386.rpm together with reiserfs and
> made a snapshot which I then tar'ed.
> Writing the tape seems to be ok. After that I did:
> 
> 	tar --directory=/ --file=/dev/tape --compare
> 
> and got the message (while working i.e. modifying the  original filesystem)
> 
> Message from syslogd@joergli at Fri Jan 11 10:57:19 2002 ...
> joergli kernel: zam-7001: io error in reiserfs_find_entry
> 


> 
> So it seems that the snapshot is full. Is that correct, or do I have a serious
> problem? Note: I'm doing my backups, so a dead backup is not a very good
> one :-)

Yes, your snapshot is full. When this happens LVM fails all I/O to the snapshot.
Unfortunatetly reiserfs doesn't not currently cope very well with I/O errors on
a device hence the message you saw.

You will need to allocate more space to the snapshot LV to get around this.

It's not a serious problem but if you are bothered by the oops I posted a
patch to workaround to the problem here:

http://lists.sistina.com/pipermail/linux-lvm/2001-November/009913.html

It's not great and it doesn't stop the full snapshot being unusable, but it does
stop the oopses.

patrick

Re: [linux-lvm] kernel: zam-7001: io error in reiserfs_find_entry

[Thomas Ziegler]

On Fri, Jan 11, 2002 at 10:37:18AM +0000, Patrick Caulfield wrote:
> On Fri, Jan 11, 2002 at 11:30:17AM +0100, J?rgen Vollmer wrote:
> > 
> > Hi everybody,
> > 
> > I use kernel-source-2.4.16.SuSE-24.i386.rpm together with reiserfs and
> > made a snapshot which I then tar'ed.
> > Writing the tape seems to be ok. After that I did:
> > 
> > 	tar --directory=/ --file=/dev/tape --compare
> > 
> > and got the message (while working i.e. modifying the  original filesystem)
> > 
> > Message from syslogd@joergli at Fri Jan 11 10:57:19 2002 ...
> > joergli kernel: zam-7001: io error in reiserfs_find_entry
> > 
> 
> 
> > 
> > So it seems that the snapshot is full. Is that correct, or do I have a serious
> > problem? Note: I'm doing my backups, so a dead backup is not a very good
> > one :-)
> 
> Yes, your snapshot is full. When this happens LVM fails all I/O to the snapshot.
> Unfortunatetly reiserfs doesn't not currently cope very well with I/O errors on
> a device hence the message you saw.
> 
> You will need to allocate more space to the snapshot LV to get around this.
> 
> It's not a serious problem but if you are bothered by the oops I posted a
> patch to workaround to the problem here:
> 
> http://lists.sistina.com/pipermail/linux-lvm/2001-November/009913.html
> 
> It's not great and it doesn't stop the full snapshot being unusable, but it does
> stop the oopses.
> 

Some Questions to LVM and snapshots:

- Do other filesystems cope in a better way with a full snapshot than reiserfs? 

- What's the preferred fs in conjunction with lvm?

- Is there a way to get the snapshot not destroyed if it is getting full?

Thanks in advance,
Thomas


-- 
Dr.-Ing. Thomas Ziegler            |
Senior Research Engineer           | Coding Technologies
phone: +49 (0) 911 92891 -27       | Deutschherrnstr. 15-19
fax:   +49 (0) 911 92891 -99       | D-90429 N�rnberg, Germany
mailto:zie@CodingTechnologies.com  | http://www.CodingTechnologies.com

Re: [linux-lvm] kernel: zam-7001: io error in reiserfs_find_entry

[Patrick Caulfield]

On Fri, Jan 11, 2002 at 11:47:37AM +0100, Thomas Ziegler wrote:
> 
> Some Questions to LVM and snapshots:
> 
> - Do other filesystems cope in a better way with a full snapshot than reiserfs? 

Well ext2 just spits out errors without oopsing. I'm not sure about XFS or JFS.
 
> - What's the preferred fs in conjunction with lvm?

Whatever fs you prefer. That's why there's a choice !
 
> - Is there a way to get the snapshot not destroyed if it is getting full?

No. Once the snapshot is full then it can't be valid because there may be
blocks written to the original device that the snapshot can't backup. The only
way aoud this is to make your snapshots large enough to hold all the data that
is going to change on the original device during the backup.

patrick

Re: [linux-lvm] kernel: zam-7001: io error in reiserfs_find_entry

[Chris Mason]

On Friday, January 11, 2002 11:01:46 AM +0000 Patrick Caulfield
<caulfield@sistina.com> wrote:

> On Fri, Jan 11, 2002 at 11:47:37AM +0100, Thomas Ziegler wrote:
>> 
>> Some Questions to LVM and snapshots:
>> 
>> - Do other filesystems cope in a better way with a full snapshot than
>> reiserfs? 

I've started cleaning these and a few other io related errors up, this
won't be an open issue much longer.

-chris

RE: [linux-lvm] kernel: zam-7001: io error in reiserfs_find_entry

[Stephenson, Dale]

Patrick Caulfield wrote:
> On Fri, Jan 11, 2002 at 11:47:37AM +0100, Thomas Ziegler wrote:
[snip]
> > - Is there a way to get the snapshot not destroyed if it is 
> getting full?
> 
> No. Once the snapshot is full then it can't be valid because 
> there may be
> blocks written to the original device that the snapshot can't 
> backup. The only
> way aoud this is to make your snapshots large enough to hold 
> all the data that
> is going to change on the original device during the backup.
> 
If it isn't full yet, is there any reason you couldn't just do an lvextend
on the snapshot volume?  (This assumes there is space available in the
volume group for the lvextend.)

Dale Stephenson
steph@snapserver.com

Re: [linux-lvm] kernel: zam-7001: io error in reiserfs_find_entry

[Patrick Caulfield]

On Fri, Jan 11, 2002 at 09:32:47AM -0800, Stephenson, Dale wrote:
> Patrick Caulfield wrote:
> > On Fri, Jan 11, 2002 at 11:47:37AM +0100, Thomas Ziegler wrote:
> [snip]
> > > - Is there a way to get the snapshot not destroyed if it is 
> > getting full?
> > 
> > No. Once the snapshot is full then it can't be valid because 
> > there may be
> > blocks written to the original device that the snapshot can't 
> > backup. The only
> > way aoud this is to make your snapshots large enough to hold 
> > all the data that
> > is going to change on the original device during the backup.
> > 
> If it isn't full yet, is there any reason you couldn't just do an lvextend
> on the snapshot volume?  (This assumes there is space available in the
> volume group for the lvextend.)

Yes you can - you can also use the LV_SNAPSHOT_USE_RATE ioctl call to warn you
when this was happening.

patrick

Re: [linux-lvm] kernel: zam-7001: io error in reiserfs_find_entry

[Chris Mason]

[ reiserfs panic on io error when snapshots get full ]

BTW, 2.4.18-pre3 includes a patch from Oleg Drokin to return IO_ERROR instead
of panic in reiserfs_find_entry.  It is almost the same as Patrick's, which
means more work is needed to make it safe for all io errors.  But for lvm
usage, it is fine.

-chris