Can we have mutiple OSD in a single machine

Can we have mutiple OSD in a single machine

[Madhusudhana U]

Hi all,
I have a system with 2T SATA drive and I want to add it to my ceph 
cluster. I was thinking instead of creating one large OSD, can't
I have 44 osd's of 450G each ? Is this possible ? if possible, will 
this improve read/write performance ?

Thanks
__Madhusudhana

Re: Can we have mutiple OSD in a single machine

[Stefan Kleijkers]

Hello,

Yes that's no problem. I'm using that configuration for some time now. 
Just generate a config with multiple OSD clauses with the same node/host.

With the newer ceph version mkcephfs is smart enough to detect the osd's 
on the same node and will generate a crushmap whereby the objects get 
replicated to different nodes.

I didn't see any impact on the performance (if you have enough 
processing power, because you need more of that).

I wanted to use just a few OSD's per node with mdraid, so I could use 
RAID6. This way I could swap a faulty disk without bringing the node 
down. But I couldn't get it stable with mdraid.

Stefan

On 04/11/2012 09:42 AM, Madhusudhana U wrote:
> Hi all,
> I have a system with 2T SATA drive and I want to add it to my ceph
> cluster. I was thinking instead of creating one large OSD, can't
> I have 44 osd's of 450G each ? Is this possible ? if possible, will
> this improve read/write performance ?
>
> Thanks
> __Madhusudhana
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Can we have mutiple OSD in a single machine

[Madhusudhana U]

Stefan Kleijkers <stefan <at> unilogicnetworks.net> writes:

> 
> Hello,
> 
> Yes that's no problem. I'm using that configuration for some time now. 
> Just generate a config with multiple OSD clauses with the same node/host.
> 
> With the newer ceph version mkcephfs is smart enough to detect the osd's 
> on the same node and will generate a crushmap whereby the objects get 
> replicated to different nodes.
> 
> I didn't see any impact on the performance (if you have enough 
> processing power, because you need more of that).
> 
> I wanted to use just a few OSD's per node with mdraid, so I could use 
> RAID6. This way I could swap a faulty disk without bringing the node 
> down. But I couldn't get it stable with mdraid.
> 
This is how my OSD part in ceph.conf looks like

[osd.0]
        host = ceph-node-1
        btrfs devs = /dev/sda6

[osd.1]
        host = ceph-node-2
        btrfs devs = /dev/sda6

[osd.2]
        host = ceph-node-3
        btrfs devs = /dev/sda6

[osd.3]
        host = ceph-node-4
        btrfs devs = /dev/sda6



Can you please help me how I can add multiple OSD in the same machine 
considering that i have 4 partition created for OSD ?

I have powerful machines having 6 quad core Intel Xeon with 48G of RAM

Re: Can we have mutiple OSD in a single machine

[Tomasz Paszkowski]

If you're single machine change hostname in the cfg to be the same but
you need to change dev name to be different for each osd process on
single machine.

On Wed, Apr 11, 2012 at 2:42 PM, Madhusudhana U
<madhusudhana.u.acharya@gmail.com> wrote:
> Stefan Kleijkers <stefan <at> unilogicnetworks.net> writes:
>
>>
>> Hello,
>>
>> Yes that's no problem. I'm using that configuration for some time now.
>> Just generate a config with multiple OSD clauses with the same node/host.
>>
>> With the newer ceph version mkcephfs is smart enough to detect the osd's
>> on the same node and will generate a crushmap whereby the objects get
>> replicated to different nodes.
>>
>> I didn't see any impact on the performance (if you have enough
>> processing power, because you need more of that).
>>
>> I wanted to use just a few OSD's per node with mdraid, so I could use
>> RAID6. This way I could swap a faulty disk without bringing the node
>> down. But I couldn't get it stable with mdraid.
>>
> This is how my OSD part in ceph.conf looks like
>
> [osd.0]
>        host = ceph-node-1
>        btrfs devs = /dev/sda6
>
> [osd.1]
>        host = ceph-node-2
>        btrfs devs = /dev/sda6
>
> [osd.2]
>        host = ceph-node-3
>        btrfs devs = /dev/sda6
>
> [osd.3]
>        host = ceph-node-4
>        btrfs devs = /dev/sda6
>
>
>
> Can you please help me how I can add multiple OSD in the same machine
> considering that i have 4 partition created for OSD ?
>
> I have powerful machines having 6 quad core Intel Xeon with 48G of RAM
>
>
>
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Tomasz Paszkowski
SS7, Asterisk, SAN, Datacenter, Cloud Computing
+48500166299
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Can we have mutiple OSD in a single machine

[Stefan Kleijkers]

Hello,

You will get something like this:

[osd.0]
         host = ceph-node-1
         btrfs devs = /dev/sda6

[osd.1]
         host = ceph-node-1
         btrfs devs = /dev/sda7

[osd.2]
         host = ceph-node-1
         btrfs devs = /dev/sda8

[osd.3]
         host = ceph-node-1
         btrfs devs = /dev/sda9


[osd.4]
         host = ceph-node-2
         btrfs devs = /dev/sda6

[osd.5]
         host = ceph-node-2
         btrfs devs = /dev/sda7

etc...

But as Tomasz mentions, you get no extra performance, because in most cases the disk is the bottleneck.

Besides I recommend to not use btrfs devs anymore, they want to deprecate that option. So you only get the "osd data =<directory>" option.

If you really want to add performance use more disks or use a fast journal device (I use a SSD).

Stefan




On 04/11/2012 02:42 PM, Madhusudhana U wrote:
> Stefan Kleijkers<stefan<at>  unilogicnetworks.net>  writes:
>
>> Hello,
>>
>> Yes that's no problem. I'm using that configuration for some time now.
>> Just generate a config with multiple OSD clauses with the same node/host.
>>
>> With the newer ceph version mkcephfs is smart enough to detect the osd's
>> on the same node and will generate a crushmap whereby the objects get
>> replicated to different nodes.
>>
>> I didn't see any impact on the performance (if you have enough
>> processing power, because you need more of that).
>>
>> I wanted to use just a few OSD's per node with mdraid, so I could use
>> RAID6. This way I could swap a faulty disk without bringing the node
>> down. But I couldn't get it stable with mdraid.
>>
> This is how my OSD part in ceph.conf looks like
>
> [osd.0]
>          host = ceph-node-1
>          btrfs devs = /dev/sda6
>
> [osd.1]
>          host = ceph-node-2
>          btrfs devs = /dev/sda6
>
> [osd.2]
>          host = ceph-node-3
>          btrfs devs = /dev/sda6
>
> [osd.3]
>          host = ceph-node-4
>          btrfs devs = /dev/sda6
>
>
>
> Can you please help me how I can add multiple OSD in the same machine
> considering that i have 4 partition created for OSD ?
>
> I have powerful machines having 6 quad core Intel Xeon with 48G of RAM
>
>
>
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Can we have mutiple OSD in a single machine

[Tomasz Paszkowski]

Hi,

Please correct me if I'am wrong. You would like to partition single drive ?


Dnia 11-04-2012 o godz. 09:42 Madhusudhana U
<madhusudhana.u.acharya@gmail.com> napisał(a):

> Hi all,
> I have a system with 2T SATA drive and I want to add it to my ceph
> cluster. I was thinking instead of creating one large OSD, can't
> I have 44 osd's of 450G each ? Is this possible ? if possible, will
> this improve read/write performance ?
>
> Thanks
> __Madhusudhana
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Can we have mutiple OSD in a single machine

[Madhusudhana U]

Tomasz Paszkowski <ss7pro <at> gmail.com> writes:

> 
> Hi,
> 
> Please correct me if I'am wrong. You would like to partition single drive ?
> 
Yes,
I want to create 4 partitions in a  single drive. This will increase the
OSD number. Will this increase in OSD also increases performance ?

Thanks

Re: Can we have mutiple OSD in a single machine

[Tomasz Paszkowski]

Hi,

It'll not increase overall storage system performance. Partitioning of
single disk drive gives you no performance gains.





On Wed, Apr 11, 2012 at 2:38 PM, Madhusudhana U
<madhusudhana.u.acharya@gmail.com> wrote:
> Tomasz Paszkowski <ss7pro <at> gmail.com> writes:
>
>>
>> Hi,
>>
>> Please correct me if I'am wrong. You would like to partition single drive ?
>>
> Yes,
> I want to create 4 partitions in a  single drive. This will increase the
> OSD number. Will this increase in OSD also increases performance ?
>
> Thanks
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Tomasz Paszkowski
SS7, Asterisk, SAN, Datacenter, Cloud Computing
+48500166299
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Can we have mutiple OSD in a single machine

[Sage Weil]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1344 bytes --]

On Wed, 11 Apr 2012, Tomasz Paszkowski wrote:
> Hi,
> 
> It'll not increase overall storage system performance. Partitioning of
> single disk drive gives you no performance gains.

It will in fact slow these down, because each ceph-osd instance will by 
doing periodic syncfs(2) calls and they will interfere.

sage

> 
> 
> 
> 
> 
> On Wed, Apr 11, 2012 at 2:38 PM, Madhusudhana U
> <madhusudhana.u.acharya@gmail.com> wrote:
> > Tomasz Paszkowski <ss7pro <at> gmail.com> writes:
> >
> >>
> >> Hi,
> >>
> >> Please correct me if I'am wrong. You would like to partition single drive ?
> >>
> > Yes,
> > I want to create 4 partitions in a  single drive. This will increase the
> > OSD number. Will this increase in OSD also increases performance ?
> >
> > Thanks
> >
> >
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 
> -- 
> Tomasz Paszkowski
> SS7, Asterisk, SAN, Datacenter, Cloud Computing
> +48500166299
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
>