ceph-deploy disk partition alignment

ceph-deploy disk partition alignment

[Somnath Roy]

Hi,
I am seeing ceph-deploy is creating data partition from sector 1 ignoring what sgdisk is recommending (256 in my disk). Basically, it should be aligned with physical sector size (reported in /sys/block/<device>/queue/physical_block_size). In my case it is 16K physical and 4K logical...256 is perfectly fine as sgdisk/fdisk internally decides.
Disk performance will be severely impacted because of partitioning this way from ceph-deploy , any reason why we are doing like this ?

Thanks & Regards
Somnath
PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).

Re: ceph-disk disk partition alignment

[Loic Dachary]

Hi,

This not a ceph-deploy issue, rather a ceph-disk issue. Did you file an issue regarding this problem or would you like me to create one ? IIRC there are cases where sgdisk is not able to create a partition  aligned as recommended (but parted could). Do you know anything about that ?

Cheers

On 23/02/2016 01:27, Somnath Roy wrote:
> Hi,
> I am seeing ceph-deploy is creating data partition from sector 1 ignoring what sgdisk is recommending (256 in my disk). Basically, it should be aligned with physical sector size (reported in /sys/block/<device>/queue/physical_block_size). In my case it is 16K physical and 4K logical...256 is perfectly fine as sgdisk/fdisk internally decides.
> Disk performance will be severely impacted because of partitioning this way from ceph-deploy , any reason why we are doing like this ?



> Thanks & Regards
> Somnath
>

-- 
Loïc Dachary, Artisan Logiciel Libre

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: ceph-disk disk partition alignment

[Somnath Roy]

Yeah, my bad, ceph-disk issue, thanks...No, I didn't file any tracker, please go ahead..

<< IIRC there are cases where sgdisk is not able to create a partition  aligned as recommended (but parted could).

Nope, I don't know the scenarios where sgdisk/fdisk/gdisk won't be able to create properly aligned partitions..If you can share that knowledge would be helpful..

Thanks & Regards
Somnath

-----Original Message-----
From: Loic Dachary [mailto:loic@dachary.org]
Sent: Monday, February 22, 2016 9:12 PM
To: Somnath Roy; ceph-devel
Subject: Re: ceph-disk disk partition alignment

Hi,

This not a ceph-deploy issue, rather a ceph-disk issue. Did you file an issue regarding this problem or would you like me to create one ? IIRC there are cases where sgdisk is not able to create a partition  aligned as recommended (but parted could). Do you know anything about that ?

Cheers

On 23/02/2016 01:27, Somnath Roy wrote:
> Hi,
> I am seeing ceph-deploy is creating data partition from sector 1 ignoring what sgdisk is recommending (256 in my disk). Basically, it should be aligned with physical sector size (reported in /sys/block/<device>/queue/physical_block_size). In my case it is 16K physical and 4K logical...256 is perfectly fine as sgdisk/fdisk internally decides.
> Disk performance will be severely impacted because of partitioning this way from ceph-deploy , any reason why we are doing like this ?



> Thanks & Regards
> Somnath
>

--
Loïc Dachary, Artisan Logiciel Libre

PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ceph-disk disk partition alignment

[Loic Dachary]

Here it is http://tracker.ceph.com/issues/14842

On 23/02/2016 12:24, Somnath Roy wrote:
> eph-deploy is creating data partition from sector 1 ignoring what sgdisk is recommending (256 in my disk). Basically, it should be aligned with physical sector size (reported in /sys/block/<device>/queue/physical_block_size). In my case it is 16K physical and 4K logical...256 is perfectly fine as sgdisk/fdisk internally decides.
>> Disk performance will be severely impacted because of partitioning this way from ceph-deploy , 

-- 
Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ceph-disk disk partition alignment

[Василий Ангапов]

In my case (Hammer 0.94.5) I see actually the proper alignment made by
ceph-disk. Thing is that fdisk is not able to see GPT partitions:

root@slpeah001:[~]:# fdisk -l /dev/sdf

Disk /dev/sdf: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1  4294967295  2147483647+  ee  GPT
root@slpeah001:[~]:# gdisk -l /dev/sdf
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sdf: 7814037168 sectors, 3.6 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): E1063D97-3EFF-4F81-A08F-81D069625677
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 7814037134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048      7814037134   3.6 TiB     FFFF  ceph data

So I think ceph-disk does things right... But  before I use ceph-disk
I create an empty GPT with gdisk.

2016-02-23 13:49 GMT+08:00 Loic Dachary <loic@dachary.org>:
> Here it is http://tracker.ceph.com/issues/14842
>
> On 23/02/2016 12:24, Somnath Roy wrote:
>> eph-deploy is creating data partition from sector 1 ignoring what sgdisk is recommending (256 in my disk). Basically, it should be aligned with physical sector size (reported in /sys/block/<device>/queue/physical_block_size). In my case it is 16K physical and 4K logical...256 is perfectly fine as sgdisk/fdisk internally decides.
>>> Disk performance will be severely impacted because of partitioning this way from ceph-deploy ,
>
> --
> Loïc Dachary, Artisan Logiciel Libre
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: ceph-disk disk partition alignment

[Somnath Roy]

By default fdisk detects dos partitions , you need to change mode to see GPT partitions.
I didn't see ceph-disk output below , at what sector it is starting with ?


Thanks & Regards
Somnath

-----Original Message-----
From: Василий Ангапов [mailto:angapov@gmail.com]
Sent: Monday, February 22, 2016 11:09 PM
To: Loic Dachary
Cc: Somnath Roy; ceph-devel
Subject: Re: ceph-disk disk partition alignment

In my case (Hammer 0.94.5) I see actually the proper alignment made by ceph-disk. Thing is that fdisk is not able to see GPT partitions:

root@slpeah001:[~]:# fdisk -l /dev/sdf

Disk /dev/sdf: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1  4294967295  2147483647+  ee  GPT
root@slpeah001:[~]:# gdisk -l /dev/sdf
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sdf: 7814037168 sectors, 3.6 TiB Logical sector size: 512 bytes Disk identifier (GUID): E1063D97-3EFF-4F81-A08F-81D069625677
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 7814037134 Partitions will be aligned on 2048-sector boundaries Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048      7814037134   3.6 TiB     FFFF  ceph data

So I think ceph-disk does things right... But  before I use ceph-disk I create an empty GPT with gdisk.

2016-02-23 13:49 GMT+08:00 Loic Dachary <loic@dachary.org>:
> Here it is http://tracker.ceph.com/issues/14842
>
> On 23/02/2016 12:24, Somnath Roy wrote:
>> eph-deploy is creating data partition from sector 1 ignoring what sgdisk is recommending (256 in my disk). Basically, it should be aligned with physical sector size (reported in /sys/block/<device>/queue/physical_block_size). In my case it is 16K physical and 4K logical...256 is perfectly fine as sgdisk/fdisk internally decides.
>>> Disk performance will be severely impacted because of partitioning
>>> this way from ceph-deploy ,
>
> --
> Loïc Dachary, Artisan Logiciel Libre
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> in the body of a message to majordomo@vger.kernel.org More majordomo
> info at  http://vger.kernel.org/majordomo-info.html
PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).

Re: ceph-disk disk partition alignment

[Ric Wheeler]

On 02/23/2016 11:19 AM, Loic Dachary wrote:
> Here it is http://tracker.ceph.com/issues/14842
>
> On 23/02/2016 12:24, Somnath Roy wrote:
>> eph-deploy is creating data partition from sector 1 ignoring what sgdisk is recommending (256 in my disk). Basically, it should be aligned with physical sector size (reported in /sys/block/<device>/queue/physical_block_size). In my case it is 16K physical and 4K logical...256 is perfectly fine as sgdisk/fdisk internally decides.
>>> Disk performance will be severely impacted because of partitioning this way from ceph-deploy ,

I think that more modern partition tools handle alignment better (and handle the 
modern partition types) - sfdisk and parted both are better than fdisk I believe.

For RHEL & Fedora, we try hard to make sure that we use the advertised alignment 
when we create a partition and default to an alignment on sector 2048 (1MB) if 
it is not set by the device.

Ric

Re: ceph-disk disk partition alignment

[Loic Dachary]

Hi Ric,

On 07/03/2016 15:07, Ric Wheeler wrote:
> On 02/23/2016 11:19 AM, Loic Dachary wrote:
>> Here it is http://tracker.ceph.com/issues/14842
>>
>> On 23/02/2016 12:24, Somnath Roy wrote:
>>> eph-deploy is creating data partition from sector 1 ignoring what sgdisk is recommending (256 in my disk). Basically, it should be aligned with physical sector size (reported in /sys/block/<device>/queue/physical_block_size). In my case it is 16K physical and 4K logical...256 is perfectly fine as sgdisk/fdisk internally decides.
>>>> Disk performance will be severely impacted because of partitioning this way from ceph-deploy ,
> 
> I think that more modern partition tools handle alignment better (and handle the modern partition types) - sfdisk and parted both are better than fdisk I believe.
> 
> For RHEL & Fedora, we try hard to make sure that we use the advertised alignment when we create a partition and default to an alignment on sector 2048 (1MB) if it is not set by the device.

So far we've used https://sourceforge.net/projects/gptfdisk/, do you know if it has limitations that would be better addressed by other tools ?

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ceph-deploy disk partition alignment

[Alfredo Deza]

On Mon, Feb 22, 2016 at 1:27 PM, Somnath Roy <Somnath.Roy@sandisk.com> wrote:
> Hi,
> I am seeing ceph-deploy is creating data partition from sector 1 ignoring what sgdisk is recommending (256 in my disk). Basically, it should be aligned with physical sector size (reported in /sys/block/<device>/queue/physical_block_size). In my case it is 16K physical and 4K logical...256 is perfectly fine as sgdisk/fdisk internally decides.
> Disk performance will be severely impacted because of partitioning this way from ceph-deploy , any reason why we are doing like this ?

I don't think ceph-deploy is particularly opinionated on this. Have
you tried using ceph-disk directly to see if the behavior persists?

For OSDs/disks ceph-deploy mostly proxies to a remote node with ceph-disk.
>
> Thanks & Regards
> Somnath
> PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: ceph-deploy disk partition alignment

[Somnath Roy]

Yes, as Loic mentioned the title should be ceph-disk not ceph-deploy :-)
I saw ceph-disk is returning hardcoded sector 1 in the code on some condition (I forgot)
Anyways, will try with ceph-disk directly to see the behavior..

Thanks & Regards
Somnath

-----Original Message-----
From: Alfredo Deza [mailto:adeza@redhat.com] 
Sent: Tuesday, February 23, 2016 4:12 AM
To: Somnath Roy
Cc: ceph-devel
Subject: Re: ceph-deploy disk partition alignment

On Mon, Feb 22, 2016 at 1:27 PM, Somnath Roy <Somnath.Roy@sandisk.com> wrote:
> Hi,
> I am seeing ceph-deploy is creating data partition from sector 1 ignoring what sgdisk is recommending (256 in my disk). Basically, it should be aligned with physical sector size (reported in /sys/block/<device>/queue/physical_block_size). In my case it is 16K physical and 4K logical...256 is perfectly fine as sgdisk/fdisk internally decides.
> Disk performance will be severely impacted because of partitioning this way from ceph-deploy , any reason why we are doing like this ?

I don't think ceph-deploy is particularly opinionated on this. Have you tried using ceph-disk directly to see if the behavior persists?

For OSDs/disks ceph-deploy mostly proxies to a remote node with ceph-disk.
>
> Thanks & Regards
> Somnath
> PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" 
> in the body of a message to majordomo@vger.kernel.org More majordomo 
> info at  http://vger.kernel.org/majordomo-info.html

RE: ceph-disk disk partition alignment

[Somnath Roy]

Hi Alfredo,
As expected, wanted to confirm that this is happening with ceph-disk also..After running 'ceph-disk prepare' I am seeing the similar behavior..

Disk /dev/sdj: 7681.5 GB, 7681501126656 bytes
256 heads, 63 sectors/track, 116280 cylinders, total 1875366486 sectors
Units = sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 16384 bytes
I/O size (minimum/optimal): 16384 bytes / 16384 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdj1               1  1875366485  3206498644   ee  GPT
Partition 1 does not start on physical sector boundary.

Thanks & Regards
Somnath

-----Original Message-----
From: Somnath Roy 
Sent: Tuesday, February 23, 2016 8:58 AM
To: 'Alfredo Deza'
Cc: ceph-devel
Subject: RE: ceph-deploy disk partition alignment

Yes, as Loic mentioned the title should be ceph-disk not ceph-deploy :-) I saw ceph-disk is returning hardcoded sector 1 in the code on some condition (I forgot) Anyways, will try with ceph-disk directly to see the behavior..

Thanks & Regards
Somnath

-----Original Message-----
From: Alfredo Deza [mailto:adeza@redhat.com]
Sent: Tuesday, February 23, 2016 4:12 AM
To: Somnath Roy
Cc: ceph-devel
Subject: Re: ceph-deploy disk partition alignment

On Mon, Feb 22, 2016 at 1:27 PM, Somnath Roy <Somnath.Roy@sandisk.com> wrote:
> Hi,
> I am seeing ceph-deploy is creating data partition from sector 1 ignoring what sgdisk is recommending (256 in my disk). Basically, it should be aligned with physical sector size (reported in /sys/block/<device>/queue/physical_block_size). In my case it is 16K physical and 4K logical...256 is perfectly fine as sgdisk/fdisk internally decides.
> Disk performance will be severely impacted because of partitioning this way from ceph-deploy , any reason why we are doing like this ?

I don't think ceph-deploy is particularly opinionated on this. Have you tried using ceph-disk directly to see if the behavior persists?

For OSDs/disks ceph-deploy mostly proxies to a remote node with ceph-disk.
>
> Thanks & Regards
> Somnath
> PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" 
> in the body of a message to majordomo@vger.kernel.org More majordomo 
> info at  http://vger.kernel.org/majordomo-info.html