strange partition table and slow speeds

strange partition table and slow speeds

[Alex Pientka]

Afternoon,

my 6x 2tb raid5 got moved into a different server. Before the move the
system was using an intel sata2 ahci on board controller. Nothing
special. Instead of creating a partition I pointed to the entire
device aka /dev/sdc instead of /dev/sdc1. Running an fdisk against the
drive doesn't come back with anything, but that is expected

The new system uses a promise 16300 sata pci-x controller. Slower, but
that system has more slots for drives. The drives got presented as a
jbod through the raid controller bios. Right away on the boot up the
linux system identified all the drives and put the /dev/md0 device
together.

Currently mdadm adjusted every partition table. Running fdisk against
any of the volumes comes back with this:

Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xffffffff

This doesn't look like a partition table
Probably you selected the wrong device.

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1   ?  4294967295  8589934589  2147483647+  ff  BBT
/dev/sdd2   ?  4294967295  8589934589  2147483647+  ff  BBT
/dev/sdd3   ?  4294967295  8589934589  2147483647+  ff  BBT
/dev/sdd4   ?  4294967295  8589934589  2147483647+  ff  BBT


Yesterday for testing purposes I failed a drive, cleaned the partition
table and then re-added it. The Disk identified got reset back to
0xffffffff and the partition table looks the same again.

/dev/md0:
        Version : 1.1
  Creation Time : Sat Apr 17 13:39:21 2010
     Raid Level : raid5
     Array Size : 9767567360 (9315.08 GiB 10001.99 GB)
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
   Raid Devices : 6
  Total Devices : 6
    Persistence : Superblock is persistent

    Update Time : Thu Dec 13 09:33:38 2012
          State : clean, degraded, recovering
 Active Devices : 5
Working Devices : 6
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 256K

 Rebuild Status : 35% complete

           Name : amy:0
           UUID : d64bd5fc:be602828:04c0c8c0:312502d9
         Events : 2919684

    Number   Major   Minor   RaidDevice State
       6       8       32        0      active sync   /dev/sdc
       7       8      112        1      active sync   /dev/sdh
       8       8       64        2      active sync   /dev/sde
       9       8       48        3      active sync   /dev/sdd
       5       8       80        4      active sync   /dev/sdf
      10       8       96        5      spare rebuilding   /dev/sdg

The reason why I stared this work was to identify why my raid is so
unbelievable slow at writing. After some adjustments dd can write a
file around 52mb/s. hdparm does reads at 378mb/s. I should have used
dd to do the read, but currently I am unable to do so.

Currently I am rebuilding my block size from 256k down to 4k. A
co-worker has the same setup as I do and his reads and writes are
twice almost three times as fast. The difference is he has 4k size,
uses 750gb and 500 gb drives (two different md devicse) and his
partition table doesn't look like my table.



Alex

Re: strange partition table and slow speeds

[Phil Turmel]

Hi Alex,

On 12/17/2012 09:17 AM, Alex Pientka wrote:

[trim /]
> 
> /dev/md0:
>         Version : 1.1

                   ^^^^^
You've deliberately chosen a metadata version that places the superblock
at sector 0 of the given device.  If that is a whole disk, it overwrites
the partition table.  The default metadata is v1.2 (which places the
superblock at offset 4k) for this very reason.

HTH,

Phil

Re: strange partition table and slow speeds

[Alex Pientka]

I assume upgrading to v1.2 is not possible. The only other way would
be to fail every raw device (one-by-one) and then create the fd
partition on it, correct?
Alex


On Mon, Dec 17, 2012 at 12:18 PM, Phil Turmel <philip@turmel.org> wrote:
> Hi Alex,
>
> On 12/17/2012 09:17 AM, Alex Pientka wrote:
>
> [trim /]
>>
>> /dev/md0:
>>         Version : 1.1
>
>                    ^^^^^
> You've deliberately chosen a metadata version that places the superblock
> at sector 0 of the given device.  If that is a whole disk, it overwrites
> the partition table.  The default metadata is v1.2 (which places the
> superblock at offset 4k) for this very reason.
>
> HTH,
>
> Phil

Re: strange partition table and slow speeds

[Phil Turmel]

[Top-posting repaired.  Please don't.]

On 12/17/2012 12:26 PM, Alex Pientka wrote:
> On Mon, Dec 17, 2012 at 12:18 PM, Phil Turmel <philip@turmel.org> wrote:
>> Hi Alex,
>>
>> On 12/17/2012 09:17 AM, Alex Pientka wrote:
>>
>> [trim /]
>>>
>>> /dev/md0:
>>>         Version : 1.1
>>
>>                    ^^^^^
>> You've deliberately chosen a metadata version that places the superblock
>> at sector 0 of the given device.  If that is a whole disk, it overwrites
>> the partition table.  The default metadata is v1.2 (which places the
>> superblock at offset 4k) for this very reason.

> I assume upgrading to v1.2 is not possible. The only other way would
> be to fail every raw device (one-by-one) and then create the fd
> partition on it, correct?

You could put the array back on partitions if you like.  I'd make a
complete backup, zero the superblocks, and use --create --assume-clean
to switch to v1.2 in place (with due care to maintain the device order
and data offsets).

(Save the output of "mdadm -E /dev/sdXX" for each member device before
you start.)

However, that fdisk can't understand the partition table shouldn't be
hurting anything, so I wouldn't make it a priority.

BTW, partition type 'fd' is deprecated along with v0.90 metadata, as it
only impacts kernel non-initramfs autoassembly, and that only works with
DOS partition tables and v0.90 metadata.

Phil

Re: strange partition table and slow speeds

[Alex Pientka]

That brings up the question: Any reason to get the partition table
fixed up? Is this just a cosmetic issue?
Alex


On Mon, Dec 17, 2012 at 12:41 PM, Phil Turmel <philip@turmel.org> wrote:
> [Top-posting repaired.  Please don't.]
>
> On 12/17/2012 12:26 PM, Alex Pientka wrote:
>> On Mon, Dec 17, 2012 at 12:18 PM, Phil Turmel <philip@turmel.org> wrote:
>>> Hi Alex,
>>>
>>> On 12/17/2012 09:17 AM, Alex Pientka wrote:
>>>
>>> [trim /]
>>>>
>>>> /dev/md0:
>>>>         Version : 1.1
>>>
>>>                    ^^^^^
>>> You've deliberately chosen a metadata version that places the superblock
>>> at sector 0 of the given device.  If that is a whole disk, it overwrites
>>> the partition table.  The default metadata is v1.2 (which places the
>>> superblock at offset 4k) for this very reason.
>
>> I assume upgrading to v1.2 is not possible. The only other way would
>> be to fail every raw device (one-by-one) and then create the fd
>> partition on it, correct?
>
> You could put the array back on partitions if you like.  I'd make a
> complete backup, zero the superblocks, and use --create --assume-clean
> to switch to v1.2 in place (with due care to maintain the device order
> and data offsets).
>
> (Save the output of "mdadm -E /dev/sdXX" for each member device before
> you start.)
>
> However, that fdisk can't understand the partition table shouldn't be
> hurting anything, so I wouldn't make it a priority.
>
> BTW, partition type 'fd' is deprecated along with v0.90 metadata, as it
> only impacts kernel non-initramfs autoassembly, and that only works with
> DOS partition tables and v0.90 metadata.
>
> Phil

Re: strange partition table and slow speeds

[Roy Sigurd Karlsbakk]

why would you want to use partitions for such a RAID in the first place?

----- Opprinnelig melding -----
> That brings up the question: Any reason to get the partition table
> fixed up? Is this just a cosmetic issue?
> Alex
> 
> 
> On Mon, Dec 17, 2012 at 12:41 PM, Phil Turmel <philip@turmel.org>
> wrote:
> > [Top-posting repaired. Please don't.]
> >
> > On 12/17/2012 12:26 PM, Alex Pientka wrote:
> >> On Mon, Dec 17, 2012 at 12:18 PM, Phil Turmel <philip@turmel.org>
> >> wrote:
> >>> Hi Alex,
> >>>
> >>> On 12/17/2012 09:17 AM, Alex Pientka wrote:
> >>>
> >>> [trim /]
> >>>>
> >>>> /dev/md0:
> >>>>         Version : 1.1
> >>>
> >>>                    ^^^^^
> >>> You've deliberately chosen a metadata version that places the
> >>> superblock
> >>> at sector 0 of the given device. If that is a whole disk, it
> >>> overwrites
> >>> the partition table. The default metadata is v1.2 (which places
> >>> the
> >>> superblock at offset 4k) for this very reason.
> >
> >> I assume upgrading to v1.2 is not possible. The only other way
> >> would
> >> be to fail every raw device (one-by-one) and then create the fd
> >> partition on it, correct?
> >
> > You could put the array back on partitions if you like. I'd make a
> > complete backup, zero the superblocks, and use --create
> > --assume-clean
> > to switch to v1.2 in place (with due care to maintain the device
> > order
> > and data offsets).
> >
> > (Save the output of "mdadm -E /dev/sdXX" for each member device
> > before
> > you start.)
> >
> > However, that fdisk can't understand the partition table shouldn't
> > be
> > hurting anything, so I wouldn't make it a priority.
> >
> > BTW, partition type 'fd' is deprecated along with v0.90 metadata, as
> > it
> > only impacts kernel non-initramfs autoassembly, and that only works
> > with
> > DOS partition tables and v0.90 metadata.
> >
> > Phil
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

-- 
Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
roy@karlsbakk.net
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: strange partition table and slow speeds

[Alex Pientka]

I want to understand why my partition table is suddenly different
moving it from one system to a different one. This happened after I
moved the 6 drives from one system to the new system.

Is this the reason why I suddenly have such bad performance?
Alex


On Mon, Dec 17, 2012 at 3:47 PM, Roy Sigurd Karlsbakk <roy@karlsbakk.net> wrote:
> why would you want to use partitions for such a RAID in the first place?
>
> ----- Opprinnelig melding -----
>> That brings up the question: Any reason to get the partition table
>> fixed up? Is this just a cosmetic issue?
>> Alex
>>
>>
>> On Mon, Dec 17, 2012 at 12:41 PM, Phil Turmel <philip@turmel.org>
>> wrote:
>> > [Top-posting repaired. Please don't.]
>> >
>> > On 12/17/2012 12:26 PM, Alex Pientka wrote:
>> >> On Mon, Dec 17, 2012 at 12:18 PM, Phil Turmel <philip@turmel.org>
>> >> wrote:
>> >>> Hi Alex,
>> >>>
>> >>> On 12/17/2012 09:17 AM, Alex Pientka wrote:
>> >>>
>> >>> [trim /]
>> >>>>
>> >>>> /dev/md0:
>> >>>>         Version : 1.1
>> >>>
>> >>>                    ^^^^^
>> >>> You've deliberately chosen a metadata version that places the
>> >>> superblock
>> >>> at sector 0 of the given device. If that is a whole disk, it
>> >>> overwrites
>> >>> the partition table. The default metadata is v1.2 (which places
>> >>> the
>> >>> superblock at offset 4k) for this very reason.
>> >
>> >> I assume upgrading to v1.2 is not possible. The only other way
>> >> would
>> >> be to fail every raw device (one-by-one) and then create the fd
>> >> partition on it, correct?
>> >
>> > You could put the array back on partitions if you like. I'd make a
>> > complete backup, zero the superblocks, and use --create
>> > --assume-clean
>> > to switch to v1.2 in place (with due care to maintain the device
>> > order
>> > and data offsets).
>> >
>> > (Save the output of "mdadm -E /dev/sdXX" for each member device
>> > before
>> > you start.)
>> >
>> > However, that fdisk can't understand the partition table shouldn't
>> > be
>> > hurting anything, so I wouldn't make it a priority.
>> >
>> > BTW, partition type 'fd' is deprecated along with v0.90 metadata, as
>> > it
>> > only impacts kernel non-initramfs autoassembly, and that only works
>> > with
>> > DOS partition tables and v0.90 metadata.
>> >
>> > Phil
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid"
>> in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> Vennlige hilsener / Best regards
>
> roy
> --
> Roy Sigurd Karlsbakk
> (+47) 98013356
> roy@karlsbakk.net
> http://blogg.karlsbakk.net/
> GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
> --
> I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: strange partition table and slow speeds

[Benjamin ESTRABAUD]

You simply don't have a partition table on those disks.

This is not a problem. Disks do not specifically need to be partitioned.
In a way, it's like if you did "mkfs.ext3 /dev/sdX" instead of 
"mkfs.ext3 /dev/sdXn". The entire drive (including the space for the 
partition table) can be expected to be used.

The fact that it was there "before" and not anymore could be a fluke, 
some of the superblock may have only been written later on.

The fact that it is not there will not affect performance in any way, 
and the array is likely the same as it always was.

Regards,
Ben.

On 17/12/12 21:16, Alex Pientka wrote:
> I want to understand why my partition table is suddenly different
> moving it from one system to a different one. This happened after I
> moved the 6 drives from one system to the new system.
>
> Is this the reason why I suddenly have such bad performance?
> Alex
>
>
> On Mon, Dec 17, 2012 at 3:47 PM, Roy Sigurd Karlsbakk <roy@karlsbakk.net> wrote:
>> why would you want to use partitions for such a RAID in the first place?
>>
>> ----- Opprinnelig melding -----
>>> That brings up the question: Any reason to get the partition table
>>> fixed up? Is this just a cosmetic issue?
>>> Alex
>>>
>>>
>>> On Mon, Dec 17, 2012 at 12:41 PM, Phil Turmel <philip@turmel.org>
>>> wrote:
>>>> [Top-posting repaired. Please don't.]
>>>>
>>>> On 12/17/2012 12:26 PM, Alex Pientka wrote:
>>>>> On Mon, Dec 17, 2012 at 12:18 PM, Phil Turmel <philip@turmel.org>
>>>>> wrote:
>>>>>> Hi Alex,
>>>>>>
>>>>>> On 12/17/2012 09:17 AM, Alex Pientka wrote:
>>>>>>
>>>>>> [trim /]
>>>>>>> /dev/md0:
>>>>>>>          Version : 1.1
>>>>>>                     ^^^^^
>>>>>> You've deliberately chosen a metadata version that places the
>>>>>> superblock
>>>>>> at sector 0 of the given device. If that is a whole disk, it
>>>>>> overwrites
>>>>>> the partition table. The default metadata is v1.2 (which places
>>>>>> the
>>>>>> superblock at offset 4k) for this very reason.
>>>>> I assume upgrading to v1.2 is not possible. The only other way
>>>>> would
>>>>> be to fail every raw device (one-by-one) and then create the fd
>>>>> partition on it, correct?
>>>> You could put the array back on partitions if you like. I'd make a
>>>> complete backup, zero the superblocks, and use --create
>>>> --assume-clean
>>>> to switch to v1.2 in place (with due care to maintain the device
>>>> order
>>>> and data offsets).
>>>>
>>>> (Save the output of "mdadm -E /dev/sdXX" for each member device
>>>> before
>>>> you start.)
>>>>
>>>> However, that fdisk can't understand the partition table shouldn't
>>>> be
>>>> hurting anything, so I wouldn't make it a priority.
>>>>
>>>> BTW, partition type 'fd' is deprecated along with v0.90 metadata, as
>>>> it
>>>> only impacts kernel non-initramfs autoassembly, and that only works
>>>> with
>>>> DOS partition tables and v0.90 metadata.
>>>>
>>>> Phil
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid"
>>> in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> --
>> Vennlige hilsener / Best regards
>>
>> roy
>> --
>> Roy Sigurd Karlsbakk
>> (+47) 98013356
>> roy@karlsbakk.net
>> http://blogg.karlsbakk.net/
>> GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
>> --
>> I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html