[linux-lvm] hangs when mounting snapshot

[linux-lvm] hangs when mounting snapshot

[Tracy R Reed]

Over the past few weeks I have been playing around with LVM with great success.
However, I have been experimenting with snapshots recently and run into a
problem. I have tried this with reiserfs and ext2 as the filesystem (not that
it should matter to LVM but I mention it because mount is involved) and get
identical results.

I am using kernel 2.4.0-test6 and lvm_0.8final.

I created a snapshot like so:

# lvcreate --size 16m --snapshot --name snap /dev/test_vg/test_lv 
lvcreate -- WARNING: all snapshots will be disabled if more than 16 MB are changed
lvcreate -- INFO: using default snapshot chunk size of 64 KB
lvcreate -- doing automatic backup of "test_vg"
lvcreate -- logical volume "/dev/test_vg/snap" successfully created

Everything looks cool:

# lvscan
lvscan -- ACTIVE   "/dev/another_test_vg/another_test_lv" [300 MB]
lvscan -- ACTIVE   "/dev/test_vg/test_lv" [63.91 GB] striped[2]
lvscan -- ACTIVE   "/dev/test_vg/snap" [63.91 GB] striped[2]
lvscan -- 3 logical volumes with 128.12 GB total in 2 volume groups
lvscan -- 3 active logical volumes

Then I try to mount it with:

# mount /dev/test_vg/snap /snap1
Segmentation fault

Then I try:

# mount /dev/test_vg/snap /snap1

which hangs the mount command. The rest of the system is ok but not even kill
-9 can get rid of that mount command. The system won't reboot properly because
it hangs when trying to unmount filesystems and I have to hit the switch.

Any ideas? Do I need to update my version of mount to support snapshots or
something?

-- 
Tracy Reed      http://www.ultraviolet.org

RE: [linux-lvm] hangs when mounting snapshot

[David Ayton]

Yes, I get the same.  kdb says:

Entering kdb (0xc1780000) Panic: divide error
due to panic @ 0xc0195f81
eax = 0x000013fc ebx = 0xc1b6e370 ecx = 0xc1b6e200 edx = 0x00000000
esi = 0x000013fe edi = 0x00000002 esp = 0xc1781d44 eip = 0xc0195f81
ebp = 0xc1781d68  ss = 0x00000018  cs = 0x00000010 eflags = 0x00010246
 ds = 0xc2aa0018  es = 0x00000018 origeax = 0xffffffff &regs = 0xc1781d10
kdb>

and the backtrace says:

kdb> bt
    EBP       EIP         Function(args)
0xc1781d68 0xc0195f81 lvm_snapshot_remap_block+0x11 (0xc1781de6, 0xc1781de8,
0x1
3fc, 0xc1b6e200)
                               kernel .text 0xc0100000 0xc0195f70 0xc019605c
0xc1781dec 0xc0198281 lvm_map+0x3dd (0xc17e7f80, 0x0)
                               kernel .text 0xc0100000 0xc0197ea4 0xc01982d8
0xc1781dfc 0xc0198314 lvm_make_request_fn+0x10 (0xc0391350, 0x0, 0xc17e7f80)
                               kernel .text 0xc0100000 0xc0198304 0xc0198320
0xc1781e24 0xc018d6d2 generic_make_request+0xb6 (0x0, 0xc17e7f80)
                               kernel .text 0xc0100000 0xc018d61c 0xc018d734
0xc1781e44 0xc018d81a ll_rw_block+0xe6 (0x0, 0x1, 0xc1781e58)
                               kernel .text 0xc0100000 0xc018d734 0xc018d884
0xc1781e5c 0xc012c239 bread+0x31 (0x3a01, 0x1, 0x400)
                               kernel .text 0xc0100000 0xc012c208 0xc012c270
0xc1781ec8 0xc014a96a ext2_read_super+0xb6 (0xc1b6ee00, 0xc16fd000, 0x0)
                               kernel .text 0xc0100000 0xc014a8b4 0xc014af64
0xc1781ee8 0xc012efb5 read_super+0x105 (0x3a01, 0xc1274420, 0xc02ef870, 0x1,
0xc
16fd000)
                               kernel .text 0xc0100000 0xc012eeb0 0xc012f00c
0xc1781f38 0xc012f19c get_sb_bdev+0x128 (0xc02ef870, 0xc16fe000, 0x1,
0xc16fd000
)
                               kernel .text 0xc0100000 0xc012f074 0xc012f1f0
0xc1781f8c 0xc012fcaa do_mount+0x19e (0xc16fe000, 0xc380c000, 0xc16ff000,
0xc0ed
0001, 0xc16fd000)
                               kernel .text 0xc0100000 0xc012fb0c 0xc012fdc0
0xc1781fbc 0xc012fe3c sys_mount+0x7c (0x8058408, 0x8058420, 0x8058438,
0xc0ed000
1, 0x8058468)
                               kernel .text 0xc0100000 0xc012fdc0 0xc012fe80
more>
           0xc0108e23 system_call+0x33
                               kernel .text 0xc0100000 0xc0108df0 0xc0108e28
kdb>

Hope this helps.

D



-----Original Message-----
From: owner-linux-lvm@msede.com [mailto:owner-linux-lvm@msede.com]On
Behalf Of Tracy R Reed
Sent: Thursday, September 21, 2000 6:50 AM
To: linux-lvm@msede.com
Subject: [linux-lvm] hangs when mounting snapshot


Over the past few weeks I have been playing around with LVM with great
success.
However, I have been experimenting with snapshots recently and run into a
problem. I have tried this with reiserfs and ext2 as the filesystem (not
that
it should matter to LVM but I mention it because mount is involved) and get
identical results.

I am using kernel 2.4.0-test6 and lvm_0.8final.

I created a snapshot like so:

# lvcreate --size 16m --snapshot --name snap /dev/test_vg/test_lv
lvcreate -- WARNING: all snapshots will be disabled if more than 16 MB are
changed
lvcreate -- INFO: using default snapshot chunk size of 64 KB
lvcreate -- doing automatic backup of "test_vg"
lvcreate -- logical volume "/dev/test_vg/snap" successfully created

Everything looks cool:

# lvscan
lvscan -- ACTIVE   "/dev/another_test_vg/another_test_lv" [300 MB]
lvscan -- ACTIVE   "/dev/test_vg/test_lv" [63.91 GB] striped[2]
lvscan -- ACTIVE   "/dev/test_vg/snap" [63.91 GB] striped[2]
lvscan -- 3 logical volumes with 128.12 GB total in 2 volume groups
lvscan -- 3 active logical volumes

Then I try to mount it with:

# mount /dev/test_vg/snap /snap1
Segmentation fault

Then I try:

# mount /dev/test_vg/snap /snap1

which hangs the mount command. The rest of the system is ok but not even
kill
-9 can get rid of that mount command. The system won't reboot properly
because
it hangs when trying to unmount filesystems and I have to hit the switch.

Any ideas? Do I need to update my version of mount to support snapshots or
something?

--
Tracy Reed      http://www.ultraviolet.org

Re: [linux-lvm] hangs when mounting snapshot

[Tracy R Reed]

Hello again. I upgraded to util-linux-2.10o which includes the latest version
of mount and it is still hanging when mounting snapshots. It segfaults first,
then hangs on the next attempt. Is there anything else I need to upgrade to get
this working? 

On Wed, Sep 20, 2000 at 10:50:21PM -0700, Tracy R Reed wrote:
> Over the past few weeks I have been playing around with LVM with great success.
> However, I have been experimenting with snapshots recently and run into a
> problem. I have tried this with reiserfs and ext2 as the filesystem (not that
> it should matter to LVM but I mention it because mount is involved) and get
> identical results.
> 
> I am using kernel 2.4.0-test6 and lvm_0.8final.
> 
> I created a snapshot like so:
> 
> # lvcreate --size 16m --snapshot --name snap /dev/test_vg/test_lv 
> lvcreate -- WARNING: all snapshots will be disabled if more than 16 MB are changed
> lvcreate -- INFO: using default snapshot chunk size of 64 KB
> lvcreate -- doing automatic backup of "test_vg"
> lvcreate -- logical volume "/dev/test_vg/snap" successfully created
> 
> Everything looks cool:
> 
> # lvscan
> lvscan -- ACTIVE   "/dev/another_test_vg/another_test_lv" [300 MB]
> lvscan -- ACTIVE   "/dev/test_vg/test_lv" [63.91 GB] striped[2]
> lvscan -- ACTIVE   "/dev/test_vg/snap" [63.91 GB] striped[2]
> lvscan -- 3 logical volumes with 128.12 GB total in 2 volume groups
> lvscan -- 3 active logical volumes
> 
> Then I try to mount it with:
> 
> # mount /dev/test_vg/snap /snap1
> Segmentation fault
> 
> Then I try:
> 
> # mount /dev/test_vg/snap /snap1
> 
> which hangs the mount command. The rest of the system is ok but not even kill
> -9 can get rid of that mount command. The system won't reboot properly because
> it hangs when trying to unmount filesystems and I have to hit the switch.
> 
> Any ideas? Do I need to update my version of mount to support snapshots or
> something?
> 
> -- 
> Tracy Reed      http://www.ultraviolet.org
> 

-- 
Tracy Reed      http://www.ultraviolet.org

Re: [linux-lvm] hangs when mounting snapshot

[Andreas Dilger]

Tracy writes:
> Hello again. I upgraded to util-linux-2.10o which includes the latest version
> of mount and it is still hanging when mounting snapshots. It segfaults first,
> then hangs on the next attempt. Is there anything else I need to upgrade to
> get this working? 
> 
> On Wed, Sep 20, 2000 at 10:50:21PM -0700, Tracy R Reed wrote:
> > [snip]
> > # mount /dev/test_vg/snap /snap1
> > Segmentation fault
> > 
> > Then I try:
> > 
> > # mount /dev/test_vg/snap /snap1
> > 
> > which hangs the mount command.

The segmentation fault is happening inside the kernel itself, so the problem
is not with the mount command, but with LVM.  It will have recorded an "OOPS"
record in /var/log/messages, and depending if you have syslogd set up
correctly, it would tell us where this is happening.  From what another
poster said, it is happening inside lvm_snapshot_remap_block(), but since
I've never really played with snapshots, I can't really comment...

Do you know if you have the latest patches for the user tools, which are
not included in the 0.8final tools you download from the LVM site?  These
fix several longstanding problems in the user tools which may (or may not)
be causing your problem.  The list of existing patches was posted by me
to this mailing list on Aug 4, as "Additions to LVM-BUGS-0.8final",
which would be a good starting point.  As I don't work with 2.4 lately,
I'm not sure if the in-kernel LVM files are up-to-date with all of the
fixes...  I know the 2.2 support is far behind 2.4, so I keep my own set
of kernel patches.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] hangs when mounting snapshot

[Tracy R Reed]

On Mon, Sep 25, 2000 at 07:46:50PM -0600, Andreas Dilger wrote:
> The segmentation fault is happening inside the kernel itself, so the problem
> is not with the mount command, but with LVM.  It will have recorded an "OOPS"
> record in /var/log/messages, and depending if you have syslogd set up
> correctly, it would tell us where this is happening.  From what another
> poster said, it is happening inside lvm_snapshot_remap_block(), but since
> I've never really played with snapshots, I can't really comment...

Right, I should have noticed it earlier but it did indeed leave an oops in the
messages file. I have run it through ksymoops:

No modules in ksyms, skipping objects
Warning (read_lsmod): no symbols in lsmod, is /proc/modules a valid lsmod file?
Sep 25 13:58:57 reiser kernel: CPU:    1 
Sep 25 13:58:57 reiser kernel: EIP:    0010:[lvm_snapshot_remap_block+17/244] 
Sep 25 13:58:57 reiser kernel: EFLAGS: 00010246 
Sep 25 13:58:57 reiser kernel: eax: 000007e0   ebx: dfacc770   ecx: dfacc600
edx: 00000000 
Sep 25 13:58:57 reiser kernel: esi: 00000000   edi: 00000000   ebp: 00000000
esp: df0d5c34 
Sep 25 13:58:57 reiser kernel: ds: 0018   es: 0018   ss: 0018 
Sep 25 13:58:57 reiser kernel: Process mount (pid: 555, stackpage=df0d5000) 
Sep 25 13:58:57 reiser kernel: Stack: dfacc770 00000000 00000000 00000000
00000000 0805734c 0000134c 00000000  
Sep 25 13:58:57 reiser kernel:        c0120018 c01aabb3 df0d5cce df0d5cd0
000007e0 dfacc600 df1ffe40 00000002  
Sep 25 13:58:57 reiser kernel:        000000e8 00000002 c0146a6c 0805734c
e0848000 c0147d5b 0805734c 0805734c  
Sep 25 13:58:57 reiser kernel: Call Trace: [sysctl_string+204/348]
[lvm_map+1095/1208] [padzero+28/32] [<e0848000>] [load_elf_binary+3023/3408]
[load_elf_binary+0/3408] [lvm_make_request_fn+15/24]  
Sep 25 13:58:57 reiser kernel: Code: f7 b1 64 01 00 00 8b 44 24 2c 89 d3 8b 30
89 f7 29 df 89 f8  
Using defaults from ksymoops -t elf32-i386 -a i386

Code;  00000000 Before first symbol
00000000 <_EIP>:
Code;  00000000 Before first symbol
   0:   f7 b1 64 01 00 00         divl   0x164(%ecx),%eax
Code;  00000006 Before first symbol
   6:   8b 44 24 2c               movl   0x2c(%esp,1),%eax
Code;  0000000a Before first symbol
   a:   89 d3                     movl   %edx,%ebx
Code;  0000000c Before first symbol
   c:   8b 30                     movl   (%eax),%esi
Code;  0000000e Before first symbol
   e:   89 f7                     movl   %esi,%edi
Code;  00000010 Before first symbol
  10:   29 df                     subl   %ebx,%edi
Code;  00000012 Before first symbol
  12:   89 f8                     movl   %edi,%eax


1 warning issued.  Results may not be reliable.

> Do you know if you have the latest patches for the user tools, which are
> not included in the 0.8final tools you download from the LVM site?  These
> fix several longstanding problems in the user tools which may (or may not)
> be causing your problem.  The list of existing patches was posted by me
> to this mailing list on Aug 4, as "Additions to LVM-BUGS-0.8final",
> which would be a good starting point.  As I don't work with 2.4 lately,
> I'm not sure if the in-kernel LVM files are up-to-date with all of the
> fixes...  I know the 2.2 support is far behind 2.4, so I keep my own set
> of kernel patches.

No, I do not have those patches. I am having these problems under 2.4 so my
problem may or may not be the same as the other posters problem. He was getting
the oops  in lvm_snapshot_remap_block but it looks like I am getting it in
lvm_make_request_fn.

-- 
Tracy Reed      http://www.ultraviolet.org

Re: [linux-lvm] hangs when mounting snapshot

[Tracy R Reed]

On Mon, Sep 25, 2000 at 07:46:50PM -0600, Andreas Dilger wrote:
> Do you know if you have the latest patches for the user tools, which are
> not included in the 0.8final tools you download from the LVM site?  These

I went back and looked over the 5 patches in the message you refer to. None of
them seem to have anything to do snapshots. I applied them anyway and they do
not seem to have helped.

Has anyone had success with snapshots and if so what kernel, user tools, etc?

-- 
Tracy Reed      http://www.ultraviolet.org

Re: [linux-lvm] hangs when mounting snapshot

[Andreas Dilger]

Tracy writes:
> No modules in ksyms, skipping objects
> Warning (read_lsmod): no symbols in lsmod, is /proc/modules a valid lsmod file?
> Sep 25 13:58:57 reiser kernel: CPU:    1 
> Sep 25 13:58:57 reiser kernel: EIP:    0010:[lvm_snapshot_remap_block+17/244] 
                                               ^^^^^^^^^^^^^^^^^^^^^^^^
 [snip]

> Sep 25 13:58:57 reiser kernel: Call Trace: [sysctl_string+204/348]
> [lvm_map+1095/1208] [padzero+28/32] [<e0848000>] [load_elf_binary+3023/3408]
> [load_elf_binary+0/3408] [lvm_make_request_fn+15/24]  
> 
> ... my problem may or may not be the same as the other posters problem.
> He was getting the oops in lvm_snapshot_remap_block but it looks like I
> am getting it in lvm_make_request_fn.

Actually, it looks like you are having the same problem.  From what I can
see, the only thing that is used as a divisor is "chunk_size" (inside
hash_fn in lvm_find_exception, which is inline).

Is it possible that your user tools are not up-to-date with the kernel
LVM headers?  It appears that the lv_chunk_size field is after the
silly __unused field in the lv_v2_t struct (as are all of the other
snapshot fields).  This may be throwing things off, because if you look
at tools/lib/lvm.h, this field isn't there...

I fixed this in my LVM tools RPM by using only the kernel lvm.h file
to ensure they always match.  Try adding the "uint8_t __unused" field to
lv_v2_t in tools/lib/lvm.h, recompile, and see if this fixes your problem...

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert