[linux-lvm] Debugging help.

[linux-lvm] Debugging help.

[Jay Weber]

Hi,

I'm focusing on trying to figure out why I get system hangs in LVM
when a snapshot is in use (or maybe it only triggers when it runs out of
space) while there is heavy IO (although I don't think 8 consecutive cp's
constitue heavy IO).

Any suggestions on what to turn on debugging wise?  I'm currently
utilizing SysRq and kdb v1.5 on stock 2.2.18 with LVM patches applied from
0.9.1beta2 using the PATCHES/README instructions.  All I manage to get
after the system hangs, using SysRq's PrintPC or dropping into kdb
(manually) and doing a backtrace, is reference to cpu_idle and
init_task_union.  This leads me to believe the system is just sitting in
an indefinate idle loop state?

I've also initially tried enabling the debug code in lvm-snap.c but it's
not much help since the box hangs and the info that may be output from it
is stuck in buffer and SysRq Sync don't manage to drop it to disk for me
after hanging.

Any clues, suggestions, hints, pointers?

Nobody else sees this problem?  I'm currently reproducing this on my Vaio
laptop using a loop device attached file as the simulated disk, but will
use another box with real disks here shortly and double check. Granted,
this problem surfaced on another box to start with. :)

Re: [linux-lvm] Debugging help.

[Joe Thornber]

On Wed, Jan 24, 2001 at 09:58:23AM -0800, Jay Weber wrote:
> Hi,
> 
> I'm focusing on trying to figure out why I get system hangs in LVM
> when a snapshot is in use (or maybe it only triggers when it runs out of
> space) while there is heavy IO (although I don't think 8 consecutive cp's
> constitue heavy IO).
> 
> Any suggestions on what to turn on debugging wise?  I'm currently
> utilizing SysRq and kdb v1.5 on stock 2.2.18 with LVM patches applied from
> 0.9.1beta2 using the PATCHES/README instructions.  All I manage to get
> after the system hangs, using SysRq's PrintPC or dropping into kdb
> (manually) and doing a backtrace, is reference to cpu_idle and
> init_task_union.  This leads me to believe the system is just sitting in
> an indefinate idle loop state?
> 
> I've also initially tried enabling the debug code in lvm-snap.c but it's
> not much help since the box hangs and the info that may be output from it
> is stuck in buffer and SysRq Sync don't manage to drop it to disk for me
> after hanging.
> 
> Any clues, suggestions, hints, pointers?

I use user mode linux, if I get a hang I just hit control-C and look
what the kernels doing in gdb.

- Joe

Re: [linux-lvm] Debugging help.

[Jay Weber]

Hmm, thats a thought.  Thanks.

> I use user mode linux, if I get a hang I just hit control-C and look
> what the kernels doing in gdb.

[linux-lvm] Debugging and testing

[Eric M. Hopper]

[-- Attachment #1: Type: text/plain, Size: 1097 bytes --]

	On a debugging and testing note...

	Has anybody tried building up sets of test data for the ioctl
calls that are designed to crash the kernel?  For example, random ioctls
on the character device, ones with invalid pointers, ones with strangely
inconsistent data, etc...

	Perhaps I'm being a bit too picky, but it seems to me there
should be as few ways as possible for a user space program (even one run
by root) to crash the kernel.  The LVM deice is especially this way
since whenever you're using it you're trying to perform some system
maintenance task with the express desire of not shutting down.  And,
since you're performing a system maintenance task, the system is likely
to be in an interesting state that may make it difficult to recover from
a crash.

	Anyway, I didn't know if anybody had done this or not, so I
thought I'd ask.

Have fun (if at all possible),
-- 
The best we can hope for concerning the people at large is that they
be properly armed.  -- Alexander Hamilton
-- Eric Hopper (hopper@omnifarious.mn.org http://www.omnifarious.org/~hopper) --

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [linux-lvm] Debugging help.

[Andreas Dilger]

Jay Webber writes:
> Any suggestions on what to turn on debugging wise?  I'm currently
> utilizing SysRq and kdb v1.5 on stock 2.2.18 with LVM patches applied from
> 0.9.1beta2 using the PATCHES/README instructions.  All I manage to get
> after the system hangs, using SysRq's PrintPC or dropping into kdb
> (manually) and doing a backtrace, is reference to cpu_idle and
> init_task_union.  This leads me to believe the system is just sitting in
> an indefinate idle loop state?
> 
> I've also initially tried enabling the debug code in lvm-snap.c but it's
> not much help since the box hangs and the info that may be output from it
> is stuck in buffer and SysRq Sync don't manage to drop it to disk for me
> after hanging.
> 
> Any clues, suggestions, hints, pointers?

One option is to use a serial console (preferrably to another PC) to capture
the debugging info.  Unfortunately, like me, you are using a VAIO which
doesn't have a serial port.  I've also used a patch "kmsgdump" which
allows you to save the syslog to floppy.  However, I haven't used it at
the same time as kdb, so there may be conflicts.  Both appear on the
console after an OOPS, so I'm not sure what would happen.

> Nobody else sees this problem?  I'm currently reproducing this on my Vaio
> laptop using a loop device attached file as the simulated disk, but will
> use another box with real disks here shortly and double check. Granted,
> this problem surfaced on another box to start with. :)

So far I haven't been using snapshots.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert