device-mapper oops (and more!)

device-mapper oops (and more!)

[Molle Bestefich]

[-- Attachment #1: Type: text/plain, Size: 3879 bytes --]

Hi

A status update on my experience with dmraid and the device-mapper.

Topics, ranging from "least serious" (1) to most (4):

  1. 'dmraid' and device-mapper target availability
  2. Error message propagation from device-mapper through dmraid to end user
  3. Assembly of mirrors using dmraid
  4. Oops while using dmraid / device-mapper


--__--__--

Topic 1: 'dmraid' and device-mapper target availability

Dmraid does not check for available targets before trying to assemble arrays.

Example:
If I run 'dmraid -ay', dmraid tries to create device-mapper tables
using the 'mirror' target,
even though dmsetup does not list "mirror" support in 'dmsetup targets'.
(The mirror target is not compiled in and there is no dm-mirror.ko available.)


--__--__--

Topic 2: Error message propagation from device-mapper through dmraid to end user

When the device-mapper fails to do something for dmraid, the error
does not get propagated to the end user.
Sometimes there is a follow-on error from dmraid, sometimes there is
no indication at all that something is wrong.
The device-mapper errors, however, is in the syslog, so it should be
possible to show them to the end user?

Examples of device mapper error messages in attached syslog_{1,2}.txt.
Example of follow-on error message from dmraid:

ERROR: dos: reading /dev/mapper/hpt45x_bbdfhdjicg[2]


--__--__--

Topic 3: Assembly of mirrors using dmraid

'dmraid' does not assembly the mirror on a HPT controller correctly.
An error message is provided in the syslog from device-mapper, saying:

device-mapper: device /dev/mapper/hpt45x_bbdfhdjicg too small for target

It seems that dmraid / the device-mapper (dunno which is responsible)
builds a 40 GB mirror, when in fact it should have been 80 GB.  Dmraid
then proceeds to try and add tables (or what's it called?) for the
first partition, which happens to be 60 GB in size, and device-mapper
reports that it has failed to do this, with good reason I might add.

The resulting /dev/mapper/hpt45x_bbdfhdjicg1 device is 0 bytes in
size, so 'mount' etc. of course fails.

How I found out (forgive me, but I think that it's rather clever :-)):

I have two test setups, one where Linux runs directly on the hardware
and one where it runs in a VMware which emulates physical access to
the disks in the system.  Using the direct approach, dmraid and
device-mapper does disk layout.  Using the emulated approach, access
goes through a BusLogic SCSI driver and the Windows drivers for the
HPT controller, so Linux sees the correct disk layout as physical
disks.

'fdisk -l' output from both test environments is attached; check out
the 40 GB vs. 80 GB difference:

device-mapper / dmraid:  Disk /dev/mapper/hpt45x_bbdfhdjicg: 40.0 GB,
40013178368 bytes
Windows HPT374 driver:   Disk /dev/sda: 80.0 GB, 80023749120 bytes


--__--__--

Topic 4: Oops while using dmraid / device-mapper

I got this oops in kmirrord after activating and deactivating randomly
a couple (3-4) times:

livecd root # dmraid -ay
Oops: 0000 [#1]
EIP:    0060:[<c02a1032>]    Not tainted VLI
EFLAGS: 00010246   (2.6.9)
EIP is at find_next_zero_bit+0x82/0xa4
eax: ffffffff   ebx: f8d35000   ecx: 07fffd10   edx: 00000000
esi: 00000000   edi: f8d35000   ebp: ffffa1f2   esp: f7ccbeb0
ds: 007b   es: 007b   ss: 0068
Process kmirrord/0 (pid: 6721, threadinfo=f7cca000 task=dfc01000)
Stack: 00000000 f8d0f000 0012a1f2 f7dc3480 f7ccbef0 f8ca9a21 f8d0f000 0012a1f2
       00130000 c19dfdb0 c19dfd8c c19dfd80 c19dfd8c f8caa507 f74644e0 f7ccbef0
       0012ffff 00000000 c19dfdb0 c19dfd8c c19dfd80 f74644e0 f8caa5c6 c19dfd8c
Call Trace:
 [<f8ca9a21>] core_get_resync_work+0x31/0x80 [dm_mirror]
 [...]

Above is only a snippet; rest of the oops message is attached separately.


--__--__--

I cross-posted this to two mailing lists, since topics 4 and perhaps 2
is probably relevant for device-mapper folks.  Hope it's ok?

[-- Attachment #2: syslog_1.txt --]
[-- Type: text/plain, Size: 155 bytes --]


Jan 18 19:28:36 livecd device-mapper: : unknown target type
Jan 18 19:28:36 livecd
Jan 18 19:28:36 livecd device-mapper: error adding target to table

[-- Attachment #3: syslog_2.txt --]
[-- Type: text/plain, Size: 359 bytes --]


Feb 12 11:28:08 livecd device-mapper: 4.1.0-ioctl (2003-12-10) initialised: dm@uk.sistina.com
Feb 12 11:29:25 livecd device-mapper: device /dev/mapper/hpt45x_bbdfhdjicg too small for target
Feb 12 11:29:25 livecd device-mapper: : dm-linear: Device lookup failed
Feb 12 11:29:25 livecd
Feb 12 11:29:25 livecd device-mapper: error adding target to table

[-- Attachment #4: fdisk_l__linux_dm.txt --]
[-- Type: text/plain, Size: 342 bytes --]


Disk /dev/mapper/hpt45x_bbdfhdjicg: 40.0 GB, 40013178368 bytes
255 heads, 63 sectors/track, 4864 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

                        Device Boot      Start         End      Blocks   Id  System
/dev/mapper/hpt45x_bbdfhdjicg1               1        7833    62918541    c  W95 FAT32 (LBA)

[-- Attachment #5: fdisk_l__windows_driver.txt --]
[-- Type: text/plain, Size: 279 bytes --]


Disk /dev/sda: 80.0 GB, 80023749120 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1        7833    62918541    c  W95 FAT32 (LBA)

[-- Attachment #6: oops.txt --]
[-- Type: text/plain, Size: 1983 bytes --]

livecd root # dmraid -ay
Oops: 0000 [#1]
Modules linked in: dm_mirror dm_mod vfat fat uhci_hcd snd_intel8x0 snd_ac97_codec snd_pcm snd_timer snd_page_alloc gameport snd_mpu401_uart snd_rawmidi snd_seq_device snd intel_agp agpgart tg3 parport_pc parport sbp2 ohci1394 ieee1394 ohci_hcd usb_storage
usbhid ehci_hcd usbcore
CPU:    0
EIP:    0060:[<c02a1032>]    Not tainted VLI
EFLAGS: 00010246   (2.6.9)
EIP is at find_next_zero_bit+0x82/0xa4
eax: ffffffff   ebx: f8d35000   ecx: 07fffd10   edx: 00000000
esi: 00000000   edi: f8d35000   ebp: ffffa1f2   esp: f7ccbeb0
ds: 007b   es: 007b   ss: 0068
Process kmirrord/0 (pid: 6721, threadinfo=f7cca000 task=dfc01000)
Stack: 00000000 f8d0f000 0012a1f2 f7dc3480 f7ccbef0 f8ca9a21 f8d0f000 0012a1f2
       00130000 c19dfdb0 c19dfd8c c19dfd80 c19dfd8c f8caa507 f74644e0 f7ccbef0
       0012ffff 00000000 c19dfdb0 c19dfd8c c19dfd80 f74644e0 f8caa5c6 c19dfd8c
Call Trace:
 [<f8ca9a21>] core_get_resync_work+0x31/0x80 [dm_mirror]
 [<f8caa507>] __rh_recovery_prepare+0x17/0xb0 [dm_mirror]
 [<f8caa5c6>] rh_recovery_prepare+0x26/0x50 [dm_mirror]
 [<f8caa914>] do_recovery+0x14/0x80 [dm_mirror]
 [<f8caae20>] do_work+0x0/0x70 [dm_mirror]
 [<f8caadfe>] do_mirror+0x4e/0x70 [dm_mirror]
 [<f8caae56>] do_work+0x36/0x70 [dm_mirror]
 [<c012c43b>] worker_thread+0x17b/0x220
 [<c011c200>] default_wake_function+0x0/0x20
 [<c03cddbc>] schedule+0x27c/0x480
 [<c011c200>] default_wake_function+0x0/0x20
 [<c012c2c0>] worker_thread+0x0/0x220
 [<c012fd53>] kthread+0xa3/0xb0
 [<c012fcb0>] kthread+0x0/0xb0
 [<c0104265>] kernel_thread_helper+0x5/0x10
Code: 76 00 8d bc 27 00 00 00 00 89 04 24 83 c7 04 89 f8 29 f0 c1 e0 03 31 f6 29 c5 74 25 8d 4d 1f c1 e9 05 89 fb b8 ff ff ff ff 31 d2 <f3> af 74 09 33 47 fc 83 ef 04 0f bc d0 29 df c1 e7 03 01 fa 89
 <4>device-mapper: device /dev/mapper/hpt45x_bbdfhdjicg too small for target
device-mapper: : dm-linear: Device lookup failed

device-mapper: error adding target to table


[-- Attachment #7: Type: text/plain, Size: 150 bytes --]

_______________________________________________
Ataraid-list mailing list
Ataraid-list@redhat.com
https://www.redhat.com/mailman/listinfo/ataraid-list

Re: device-mapper oops (and more!)

[Kevin Corry]

On Sat February 12 2005 5:23 am, Molle Bestefich wrote:
> Topic 1: 'dmraid' and device-mapper target availability
>
> Dmraid does not check for available targets before trying to assemble
> arrays.
>
> Example:
> If I run 'dmraid -ay', dmraid tries to create device-mapper tables
> using the 'mirror' target,
> even though dmsetup does not list "mirror" support in 'dmsetup targets'.
> (The mirror target is not compiled in and there is no dm-mirror.ko
> available.)

This is actually the way DM is intended to work. When the kernel gets a 
load-table command, it checks its target-module list for the specified 
target-type name. If it doesn't find it, it attempts to load that target 
using the kernel's module-loader routine. If the target module can't be 
loaded, DM can't load the table and returns an error.

You could obviously make the argument that dmraid could check from user-space 
that the target module is loaded and try to load it with modprobe if 
necessary. But if the module simply doesn't exist, then the end result is no 
different. You'll just get an error that the device can't be activated.

> Topic 2: Error message propagation from device-mapper through dmraid to end
> user
>
> When the device-mapper fails to do something for dmraid, the error
> does not get propagated to the end user.
> Sometimes there is a follow-on error from dmraid, sometimes there is
> no indication at all that something is wrong.
> The device-mapper errors, however, is in the syslog, so it should be
> possible to show them to the end user?
>
> Examples of device mapper error messages in attached syslog_{1,2}.txt.
> Example of follow-on error message from dmraid:
>
> ERROR: dos: reading /dev/mapper/hpt45x_bbdfhdjicg[2]

Yes, error reporting from DM to the user-space tools can be tricky. About the 
only meaningful thing that is returned in these types of error scenarios are 
a rather generic error code like ENOENT, EINVAL, ENOMEM, and such. These 
don't translate very well to end-user messages about the actual cause of the 
error. EVMS and I'd imagine LVM2 have similar problems when trying to report 
messages about errors returned from DM. Your idea about extracting error 
messages from syslog seems like a good one.

I don't work on dmraid, so I'll leave your dmraid-specific questions to those 
developers.

-- 
Kevin Corry
kevcorry@us.ibm.com
http://www.ibm.com/linux/ltc/
http://evms.sourceforge.net/

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel