[2.6.31-rc1] device-mapper: target device sda6 is misaligned

[2.6.31-rc1] device-mapper: target device sda6 is misaligned

[Frans Pop]

During boot of 2.6.31-rc1 on an arm EABI system I got the following errors:

device-mapper: ioctl: 4.15.0-ioctl (2009-04-01) initialised: dm-devel@redhat.com
device-mapper: table: 254:0: target device sda6 is misaligned
device-mapper: table: 254:0: target device sda6 is misaligned
device-mapper: table: 254:1: target device sda6 is misaligned
device-mapper: table: 254:1: target device sda6 is misaligned

The boot did complete successfully. A boot with 2.6.30 just previously was
clean, and when I reboot back into 2.6.30 the messages are gone again.

The system has a single volume group with two logical volumes.

Please advise and let me know what additional info is needed.

Cheers,
FJP

Re: [2.6.31-rc1] device-mapper: target device sda6 is misaligned

[Martin K. Petersen]

>>>>> "Frans" == Frans Pop <elendil@planet.nl> writes:

Frans> During boot of 2.6.31-rc1 on an arm EABI system I got the
Frans> following errors: device-mapper: ioctl: 4.15.0-ioctl (2009-04-01)
Frans> initialised: dm-devel@redhat.com device-mapper: table: 254:0:
Frans> target device sda6 is misaligned device-mapper: table: 254:0:
Frans> target device sda6 is misaligned device-mapper: table: 254:1:
Frans> target device sda6 is misaligned device-mapper: table: 254:1:
Frans> target device sda6 is misaligned

Frans> Please advise and let me know what additional info is needed.

What kind of disk is this?

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [2.6.31-rc1] device-mapper: target device sda6 is misaligned

[Mike Snitzer]

On Thu, Jun 25 2009 at  3:04am -0400,
Frans Pop <elendil@planet.nl> wrote:

> During boot of 2.6.31-rc1 on an arm EABI system I got the following errors:
> 
> device-mapper: ioctl: 4.15.0-ioctl (2009-04-01) initialised: dm-devel@redhat.com
> device-mapper: table: 254:0: target device sda6 is misaligned
> device-mapper: table: 254:0: target device sda6 is misaligned
> device-mapper: table: 254:1: target device sda6 is misaligned
> device-mapper: table: 254:1: target device sda6 is misaligned
> 
> The boot did complete successfully. A boot with 2.6.30 just previously was
> clean, and when I reboot back into 2.6.30 the messages are gone again.
> 
> The system has a single volume group with two logical volumes.
> 
> Please advise and let me know what additional info is needed.

Even though the messages don't contain "warning", and probably should,
they are printed via DMWARN().

These warnings are a function of device-mapper now being
topology-aware.  This means that your LVs are likely misaligned relative
to the underlying hardware.  Given that you're using partitions I'd
wager they are the source of the misalignment.  But there could be a bug
lurking somewhere.

Please provide the output from the following commands:

pvs -o +pe_start

cat /sys/block/sda/sda6/alignment_offset
cat /sys/block/sda/queue/physical_block_size
cat /sys/block/sda/queue/logical_block_size

If you could also provide the same info for all other PVs that are in
the VG that would be appreciated.

Mike

Re: [2.6.31-rc1] device-mapper: target device sda6 is misaligned

[Frans Pop]

On Thursday 25 June 2009, Martin K. Petersen wrote:
> >>>>> "Frans" == Frans Pop <elendil@planet.nl> writes:
>
> Frans> During boot of 2.6.31-rc1 on an arm EABI system I got the
> Frans> following errors: device-mapper: ioctl: 4.15.0-ioctl (2009-04-01)
> Frans> initialised: dm-devel@redhat.com device-mapper: table: 254:0:
> Frans> target device sda6 is misaligned device-mapper: table: 254:0:
> Frans> target device sda6 is misaligned device-mapper: table: 254:1:
> Frans> target device sda6 is misaligned device-mapper: table: 254:1:
> Frans> target device sda6 is misaligned 
>
> Frans> Please advise and let me know what additional info is needed.
>
> What kind of disk is this?

It's a standard SATA disk; the driver is sata_mv:

ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata1.00: ATA-7: HDT722516DLA380, V43OA91A, max UDMA/133
ata1.00: 321672960 sectors, multi 0: LBA48 NCQ (depth 31/32)
ata1.00: configured for UDMA/133
scsi 0:0:0:0: Direct-Access     ATA      HDT722516DLA380  V43O PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 321672960 512-byte logical blocks: (164 GB/153 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2 sda3 sda4 < sda5 sda6 >

I've just also tested rc1 in the Hercules s390 emulator, and I get exactly
the same there on a virtual dasd disk:

dasd-eckd 0.0.0121: New DASD 3390/02 (CU 3990/02) with 1113 cylinders, 15 heads, 224 sectors
dasd-eckd 0.0.0121: DASD with 4 KB/block, 801360 KB total size, 48 KB/track, compatible disk layout
 dasdb:VOL1/  0X0121: dasdb1 dasdb2
[...]
device-mapper: table: 254:0: target device dasdb1 is misaligned
device-mapper: table: 254:0: target device dasdb2 is misaligned
device-mapper: table: 254:0: target device dasdb1 is misaligned
device-mapper: table: 254:0: target device dasdb2 is misaligned

Re: [dm-devel] Re: [2.6.31-rc1] device-mapper: target device sda6 is misaligned

[Martin K. Petersen]

>>>>> "Frans" == Frans Pop <elendil@planet.nl> writes:

Frans> I've just also tested rc1 in the Hercules s390 emulator, and I
Frans> get exactly the same there on a virtual dasd disk:

Frans> dasd-eckd 0.0.0121: New DASD 3390/02 (CU 3990/02) with 1113
Frans> cylinders, 15 heads, 224 sectors dasd-eckd 0.0.0121: DASD with 4
Frans> KB/block, 801360 KB total size, 48 KB/track, compatible disk

DASD uses 4KB blocks so that makes more sense.

The SATA thing is weird.

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [2.6.31-rc1] device-mapper: target device sda6 is misaligned

[Mike Snitzer]

On Thu, Jun 25 2009 at 11:48am -0400,
Frans Pop <elendil@planet.nl> wrote:

> On Thursday 25 June 2009, Martin K. Petersen wrote:
> > >>>>> "Frans" == Frans Pop <elendil@planet.nl> writes:
> >
> > Frans> During boot of 2.6.31-rc1 on an arm EABI system I got the
> > Frans> following errors: device-mapper: ioctl: 4.15.0-ioctl (2009-04-01)
> > Frans> initialised: dm-devel@redhat.com device-mapper: table: 254:0:
> > Frans> target device sda6 is misaligned device-mapper: table: 254:0:
> > Frans> target device sda6 is misaligned device-mapper: table: 254:1:
> > Frans> target device sda6 is misaligned device-mapper: table: 254:1:
> > Frans> target device sda6 is misaligned 
> >
> > Frans> Please advise and let me know what additional info is needed.
> >
> > What kind of disk is this?
> 
> It's a standard SATA disk; the driver is sata_mv:
> 
> ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
> ata1.00: ATA-7: HDT722516DLA380, V43OA91A, max UDMA/133
> ata1.00: 321672960 sectors, multi 0: LBA48 NCQ (depth 31/32)
> ata1.00: configured for UDMA/133
> scsi 0:0:0:0: Direct-Access     ATA      HDT722516DLA380  V43O PQ: 0 ANSI: 5
> sd 0:0:0:0: [sda] 321672960 512-byte logical blocks: (164 GB/153 GiB)
> sd 0:0:0:0: [sda] Write Protect is off
> sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
> sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
>  sda: sda1 sda2 sda3 sda4 < sda5 sda6 >
> 
> I've just also tested rc1 in the Hercules s390 emulator, and I get exactly
> the same there on a virtual dasd disk:
> 
> dasd-eckd 0.0.0121: New DASD 3390/02 (CU 3990/02) with 1113 cylinders, 15 heads, 224 sectors
> dasd-eckd 0.0.0121: DASD with 4 KB/block, 801360 KB total size, 48 KB/track, compatible disk layout
>  dasdb:VOL1/  0X0121: dasdb1 dasdb2
> [...]
> device-mapper: table: 254:0: target device dasdb1 is misaligned
> device-mapper: table: 254:0: target device dasdb2 is misaligned
> device-mapper: table: 254:0: target device dasdb1 is misaligned
> device-mapper: table: 254:0: target device dasdb2 is misaligned

In addition to the other commands I asked for in my previous mail;
please provide the output from the following commands on both your real
and emulated configs:

dmsetup ls
dmsetup table

Mike

Re: [2.6.31-rc1] device-mapper: target device sda6 is misaligned

[Frans Pop]

On Thursday 25 June 2009, Mike Snitzer wrote:
> On Thu, Jun 25 2009 at  3:04am -0400,
> Frans Pop <elendil@planet.nl> wrote: 
> > During boot of 2.6.31-rc1 on an arm EABI system I got the following
> > errors:
> > device-mapper: table: 254:0: target device sda6 is misaligned
> > device-mapper: table: 254:0: target device sda6 is misaligned
> > device-mapper: table: 254:1: target device sda6 is misaligned
> > device-mapper: table: 254:1: target device sda6 is misaligned
> >
> > The boot did complete successfully. A boot with 2.6.30 just
> > previously was clean, and when I reboot back into 2.6.30 the messages
> > are gone again.
> >
> > The system has a single volume group with two logical volumes.
>
> Even though the messages don't contain "warning", and probably should,
> they are printed via DMWARN().

When I first saw the messages I almost got a heart attack, especially 
because I also got an fsck for the file systems on them. Luckily it 
turned out later that the reason for that was that I'd accidentally 
removed the rtc driver from my config, so the fsck was triggered by the 
system clock not being set :-/

> These warnings are a function of device-mapper now being
> topology-aware.  This means that your LVs are likely misaligned
> relative to the underlying hardware.  Given that you're using
> partitions I'd wager they are the source of the misalignment.  But
> there could be a bug lurking somewhere.

OK. If it does turn out to be benign, then I'd suggest marking the 
messages "info: " instead of "warning: ". With "warning" admins would 
still be given the idea that something might be desperately wrong and 
action should be taken NOW.
Personally, I'd even prefer just not to know, unless it for example has a 
major effect on performance _and_ is easily fixable using lvm2.

Wouldn't it be much more logical if the userspace tools warn about 
alignment problems when volumes are _created_?

The volumes were all created using Debian Installer's partitioning 
component, which uses libparted and lvm2.

> Please provide the output from the following commands:
>
> pvs -o +pe_start

  PV         VG   Fmt  Attr PSize   PFree  1st PE
  /dev/sda6  qnap lvm2 a-   124.50G 44.50G 192.00K

> cat /sys/block/sda/sda6/alignment_offset
> cat /sys/block/sda/queue/physical_block_size
> cat /sys/block/sda/queue/logical_block_size

I'm afraid those files don't exist on my systems (none of them: arm, s390 
and x86_64 notebook). Some kernel config option I'm missing maybe?

Here's the partition layout:
# fdisk -u -l /dev/sda

Disk /dev/sda: 164.7 GB, 164696555520 bytes
255 heads, 63 sectors/track, 20023 cylinders, total 321672960 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x00054f49

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63    19535039     9767488+  83  Linux
/dev/sda2        19535040    39070079     9767520   83  Linux
/dev/sda3        39070080    58605119     9767520   83  Linux
/dev/sda4        58605120   321669494   131532187+   5  Extended
/dev/sda5        58605183    60565049      979933+  82  Linux swap
/dev/sda6        60565113   321669494   130552191   8e  Linux LVM

> If you could also provide the same info for all other PVs that are in
> the VG that would be appreciated.

There's only the one PV.

On Thursday 25 June 2009, Mike Snitzer wrote:
> In addition to the other commands I asked for in my previous mail;
> please provide the output from the following commands on both your real
> and emulated configs:

# dmsetup ls
qnap-home       (254, 0)
qnap-music      (254, 1)

# dmsetup table
qnap-home: 0 83886080 linear 8:6 384
qnap-music: 0 83886080 linear 8:6 83886464

The above is all for the arm system.

Thanks,
FJP

Re: [2.6.31-rc1] device-mapper: target device sda6 is misaligned

[Mike Snitzer]

On Thu, Jun 25 2009 at 12:40pm -0400,
Frans Pop <elendil@planet.nl> wrote:

> On Thursday 25 June 2009, Mike Snitzer wrote:
> > On Thu, Jun 25 2009 at  3:04am -0400,
> > Frans Pop <elendil@planet.nl> wrote: 
> > > During boot of 2.6.31-rc1 on an arm EABI system I got the following
> > > errors:
> > > device-mapper: table: 254:0: target device sda6 is misaligned
> > > device-mapper: table: 254:0: target device sda6 is misaligned
> > > device-mapper: table: 254:1: target device sda6 is misaligned
> > > device-mapper: table: 254:1: target device sda6 is misaligned
> > >
> > > The boot did complete successfully. A boot with 2.6.30 just
> > > previously was clean, and when I reboot back into 2.6.30 the messages
> > > are gone again.
> > >
> > > The system has a single volume group with two logical volumes.
> >
> > Even though the messages don't contain "warning", and probably should,
> > they are printed via DMWARN().
> 
> When I first saw the messages I almost got a heart attack, especially 
> because I also got an fsck for the file systems on them. Luckily it 
> turned out later that the reason for that was that I'd accidentally 
> removed the rtc driver from my config, so the fsck was triggered by the 
> system clock not being set :-/
> 
> > These warnings are a function of device-mapper now being
> > topology-aware.  This means that your LVs are likely misaligned
> > relative to the underlying hardware.  Given that you're using
> > partitions I'd wager they are the source of the misalignment.  But
> > there could be a bug lurking somewhere.
> 
> OK. If it does turn out to be benign, then I'd suggest marking the 
> messages "info: " instead of "warning: ". With "warning" admins would 
> still be given the idea that something might be desperately wrong and 
> action should be taken NOW.
> Personally, I'd even prefer just not to know, unless it for example has a 
> major effect on performance _and_ is easily fixable using lvm2.
> 
> Wouldn't it be much more logical if the userspace tools warn about 
> alignment problems when volumes are _created_?
>
> The volumes were all created using Debian Installer's partitioning 
> component, which uses libparted and lvm2.

The device-mapper layer doesn't distinguish between the first creation
and every activation there after.  The blk_stack_limits() method is
called by DM when an LV's table is pushed down to the kernel.  As such,
you'll always see the warnings.  Think of it as motivation to fix
libparted and all the other storage tools (LVM2 included) to create
properly aligned devices :)
 

> > Please provide the output from the following commands:
> >
> > pvs -o +pe_start
> 
>   PV         VG   Fmt  Attr PSize   PFree  1st PE
>   /dev/sda6  qnap lvm2 a-   124.50G 44.50G 192.00K
> 
> > cat /sys/block/sda/sda6/alignment_offset
> > cat /sys/block/sda/queue/physical_block_size
> > cat /sys/block/sda/queue/logical_block_size
> 
> I'm afraid those files don't exist on my systems (none of them: arm, s390 
> and x86_64 notebook). Some kernel config option I'm missing maybe?

OK, that is not possible; if you're seeing messages like the following:
device-mapper: table: 254:0: target device sda6 is misaligned

You should definitely have the topology attributes in the associated
device's sysfs tree.  What does the following show on your 2.6.31-rc1
system that produced the above "target device sda6 is misaligned":

find /sys/block/sda/queue -type f

If you don't see attributes like the following you're collecting
information from the wrong machine:

/sys/block/sda/queue/logical_block_size
/sys/block/sda/queue/physical_block_size
/sys/block/sda/queue/minimum_io_size
/sys/block/sda/queue/optimal_io_size

Re: [2.6.31-rc1] device-mapper: target device sda6 is misaligned

[Frans Pop]

On Thursday 25 June 2009, Mike Snitzer wrote:
> > I'm afraid those files don't exist on my systems (none of them: arm,
> > s390 and x86_64 notebook). Some kernel config option I'm missing
> > maybe?
>
> You should definitely have the topology attributes in the associated
> device's sysfs tree.

Ah, sorry. I'd switched back to .30 for the arm system because of this.
(And on the s390 box I looked for sda instead of dasdb; duh!)

Here are the values for arm:

$ cat /sys/block/sda/sda6/alignment_offset
0
$ cat /sys/block/sda/queue/physical_block_size
512
$ cat /sys/block/sda/queue/logical_block_size
512

And here's the lot for the s390 emulator:

# grep . /sys/block/dasdb/dasdb*/alignment_offset
/sys/block/dasdb/dasdb1/alignment_offset:0
/sys/block/dasdb/dasdb2/alignment_offset:0
# grep . /sys/block/dasdb/queue/physical_block_size
4096
# grep . /sys/block/dasdb/queue/logical_block_size
4096

# pvs -o +pe_start
  PV          VG     Fmt  Attr PSize   PFree 1st PE
  /dev/dasdb1 mordor lvm2 a-   388.00M    0  192.00K
  /dev/dasdb2 mordor lvm2 a-   388.00M    0  192.00K
# dmsetup ls
mordor-home     (254, 0)
# dmsetup table
mordor-home: 0 794624 linear 94:5 384
mordor-home: 794624 794624 linear 94:6 384

# fdasd -p /dev/dasdb
reading volume label ..: VOL1
reading vtoc ..........: ok

Disk /dev/dasdb:
  cylinders ............: 1113
  tracks per cylinder ..: 15
  blocks per track .....: 12
  bytes per block ......: 4096
  volume label .........: VOL1
  volume serial ........: 0X0121
  max partitions .......: 3

 ------------------------------- tracks -------------------------------
               Device      start      end   length   Id  System
          /dev/dasdb1          2     8346     8345    1  Linux native
          /dev/dasdb2       8347    16694     8348    2  Linux native

Re: [2.6.31-rc1] device-mapper: target device sda6 is misaligned

[Mike Snitzer]

On Thu, Jun 25 2009 at  1:53pm -0400,
Frans Pop <elendil@planet.nl> wrote:

> On Thursday 25 June 2009, Mike Snitzer wrote:
> > > I'm afraid those files don't exist on my systems (none of them: arm,
> > > s390 and x86_64 notebook). Some kernel config option I'm missing
> > > maybe?
> >
> > You should definitely have the topology attributes in the associated
> > device's sysfs tree.
> 
> Ah, sorry. I'd switched back to .30 for the arm system because of this.
> (And on the s390 box I looked for sda instead of dasdb; duh!)
> 
> Here are the values for arm:
> 
> $ cat /sys/block/sda/sda6/alignment_offset
> 0
> $ cat /sys/block/sda/queue/physical_block_size
> 512
> $ cat /sys/block/sda/queue/logical_block_size
> 512
> 
> And here's the lot for the s390 emulator:
> 
> # grep . /sys/block/dasdb/dasdb*/alignment_offset
> /sys/block/dasdb/dasdb1/alignment_offset:0
> /sys/block/dasdb/dasdb2/alignment_offset:0
> # grep . /sys/block/dasdb/queue/physical_block_size
> 4096
> # grep . /sys/block/dasdb/queue/logical_block_size
> 4096
> 
> # pvs -o +pe_start
>   PV          VG     Fmt  Attr PSize   PFree 1st PE
>   /dev/dasdb1 mordor lvm2 a-   388.00M    0  192.00K
>   /dev/dasdb2 mordor lvm2 a-   388.00M    0  192.00K
> # dmsetup ls
> mordor-home     (254, 0)
> # dmsetup table
> mordor-home: 0 794624 linear 94:5 384
> mordor-home: 794624 794624 linear 94:6 384


OK, I found a really stupid bug.  I was passing the data start offset
(pe_start) through to blk_stack_limits() in terms of sectors rather than
bytes.  The following should silence your warnings:

diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c
index 4899ebe..2cba557 100644
--- a/drivers/md/dm-table.c
+++ b/drivers/md/dm-table.c
@@ -495,7 +495,7 @@ int dm_set_device_limits(struct dm_target *ti, struct dm_dev *dev,
 		return 0;
 	}
 
-	if (blk_stack_limits(limits, &q->limits, start) < 0)
+	if (blk_stack_limits(limits, &q->limits, start << 9) < 0)
 		DMWARN("%s: target device %s is misaligned",
 		       dm_device_name(ti->table->md), bdevname(bdev, b));
 

Re: [2.6.31-rc1] device-mapper: target device sda6 is misaligned

[Frans Pop]

On Thursday 25 June 2009, Mike Snitzer wrote:
> OK, I found a really stupid bug.  I was passing the data start offset
> (pe_start) through to blk_stack_limits() in terms of sectors rather
> than bytes.  The following should silence your warnings:

Yes, all clear again. So not broken userspace after all ;-)

Reported-by: Frans Pop <elendil@planet.nl>
Tested-by: Frans Pop <elendil@planet.nl>

Thanks,
FJP

> diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c
> index 4899ebe..2cba557 100644
> --- a/drivers/md/dm-table.c
> +++ b/drivers/md/dm-table.c
> @@ -495,7 +495,7 @@ int dm_set_device_limits(struct dm_target *ti,
> struct dm_dev *dev, return 0;
>  	}
>
> -	if (blk_stack_limits(limits, &q->limits, start) < 0)
> +	if (blk_stack_limits(limits, &q->limits, start << 9) < 0)
>  		DMWARN("%s: target device %s is misaligned",
>  		       dm_device_name(ti->table->md), bdevname(bdev, b));