* Few questions about Ceph
@ 2012-12-10 19:17 Alexandre Maumené
2012-12-10 22:01 ` Joao Eduardo Luis
2012-12-10 22:22 ` Samuel Just
0 siblings, 2 replies; 4+ messages in thread
From: Alexandre Maumené @ 2012-12-10 19:17 UTC (permalink / raw)
To: ceph-devel
Hello all,
I have a few questions about Ceph:
1) Is it possible to run a cluster with "some" lantecy between monitor
nodes? Latency will be 30ms at worst.
2) When using RBD what are the best practices for a direct mount using
XFS filesystem? And for a qemu/kvm devices? I'm thinking about
writeback, rbd_cache, ...
3) About the CRUSH map, how can I separate 2 pools on different OSD?
I'd like to setup a cluster with different disks (like SATA/SAS) and I
want to be able to specify on which disks (or OSD) my data are going
to be write.
Thanks in advance for any answer.
Regards,
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Few questions about Ceph
2012-12-10 19:17 Few questions about Ceph Alexandre Maumené
@ 2012-12-10 22:01 ` Joao Eduardo Luis
2012-12-10 22:22 ` Samuel Just
1 sibling, 0 replies; 4+ messages in thread
From: Joao Eduardo Luis @ 2012-12-10 22:01 UTC (permalink / raw)
To: Alexandre Maumené; +Cc: ceph-devel
On 12/10/2012 07:17 PM, Alexandre Maumené wrote:
> Hello all,
>
> I have a few questions about Ceph:
>
> 1) Is it possible to run a cluster with "some" lantecy between monitor
> nodes? Latency will be 30ms at worst.
For the monitor, yes, that latency is acceptable. In between them, the
monitors are pretty permissive with the timeouts they establish, so 30ms
should be okay. If it goes much higher than that however, you may start
noticing some clock drifting related messages. Anyway, you should be fine.
-Joao
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Few questions about Ceph
2012-12-10 19:17 Few questions about Ceph Alexandre Maumené
2012-12-10 22:01 ` Joao Eduardo Luis
@ 2012-12-10 22:22 ` Samuel Just
2012-12-10 22:31 ` Samuel Just
1 sibling, 1 reply; 4+ messages in thread
From: Samuel Just @ 2012-12-10 22:22 UTC (permalink / raw)
To: Alexandre Maumené; +Cc: ceph-devel@vger.kernel.org
3) Forgive the style, it'll be going into the docs shortly :)
It's possible to have multiple independent crush heirarchies within the same
crush map. Suppose you want to have pools default to osds backed by large
spinning disks but have some pools mapped to osds backed by fast ssds:
device 0 osd.0
device 1 osd.1
device 2 osd.2
device 3 osd.3
device 4 osd.4
device 5 osd.5
device 6 osd.6
device 7 osd.7
host ceph-osd-ssd-server-1 {
id -1
alg straw
hash 0
item osd.0 weight 1.00
item osd.1 weight 1.00
}
host ceph-osd-ssd-server-2 {
id -2
alg straw
hash 0
item osd.2 weight 1.00
item osd.3 weight 1.00
}
host ceph-osd-platter-server-1 {
id -3
alg straw
hash 0
item osd.4 weight 1.00
item osd.5 weight 1.00
}
host ceph-osd-platter-server-2 {
id -4
alg straw
hash 0
item osd.6 weight 1.00
item osd.7 weight 1.00
}
root platter {
id -5
alg straw
hash 0
item ceph-osd-platter-server-1 weight 2.00
item ceph-osd-platter-server-2 weight 2.00
}
root ssd {
id -6
alg straw
hash 0
item ceph-osd-ssd-server-1 weight 2.00
item ceph-osd-ssd-server-2 weight 2.00
}
rule data {
ruleset 0
type replicated
min_size 2
max_size 2
step take platter
step chooseleaf 0 type host
step emit
}
rule metadata {
ruleset 1
type replicated
min_size 0
max_size 10
step take platter
step chooseleaf 0 type host
step emit
}
rule rbd {
ruleset 2
type replicated
min_size 0
max_size 10
step take platter
step chooseleaf 0 type host
step emit
}
rule platter {
ruleset 3
type replicated
min_size 0
max_size 10
step take platter
step chooseleaf 0 type host
step emit
}
rule ssd {
ruleset 4
type replicated
min_size 0
max_size 10
step take ssd
step chooseleaf 0 type host
step emit
}
rule ssd-primary {
ruleset 4
type replicated
min_size 0
max_size 10
step take ssd
step chooseleaf 1 type host
step emit
step take platter
step chooseleaf -1 type host
step emit
}
You can then set a pool to use the ssd rule by:
ceph osd pool set <poolname> crush_ruleset 4
Similarly, using the ssd-primary rule will cause
each pg in the pool to be placed with an ssd as
the primary and platters as the replicas.
-Sam
On Mon, Dec 10, 2012 at 11:17 AM, Alexandre Maumené
<alexandre@maumene.org> wrote:
> Hello all,
>
> I have a few questions about Ceph:
>
> 1) Is it possible to run a cluster with "some" lantecy between monitor
> nodes? Latency will be 30ms at worst.
>
> 2) When using RBD what are the best practices for a direct mount using
> XFS filesystem? And for a qemu/kvm devices? I'm thinking about
> writeback, rbd_cache, ...
>
> 3) About the CRUSH map, how can I separate 2 pools on different OSD?
> I'd like to setup a cluster with different disks (like SATA/SAS) and I
> want to be able to specify on which disks (or OSD) my data are going
> to be write.
>
> Thanks in advance for any answer.
>
> Regards,
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: Few questions about Ceph
2012-12-10 22:22 ` Samuel Just
@ 2012-12-10 22:31 ` Samuel Just
0 siblings, 0 replies; 4+ messages in thread
From: Samuel Just @ 2012-12-10 22:31 UTC (permalink / raw)
To: Alexandre Maumené; +Cc: ceph-devel@vger.kernel.org
oops, ssd-primary should be ruleset 5
-Sam
On Mon, Dec 10, 2012 at 2:22 PM, Samuel Just <sam.just@inktank.com> wrote:
> 3) Forgive the style, it'll be going into the docs shortly :)
>
> It's possible to have multiple independent crush heirarchies within the same
> crush map. Suppose you want to have pools default to osds backed by large
> spinning disks but have some pools mapped to osds backed by fast ssds:
>
> device 0 osd.0
> device 1 osd.1
> device 2 osd.2
> device 3 osd.3
> device 4 osd.4
> device 5 osd.5
> device 6 osd.6
> device 7 osd.7
>
> host ceph-osd-ssd-server-1 {
> id -1
> alg straw
> hash 0
> item osd.0 weight 1.00
> item osd.1 weight 1.00
> }
>
> host ceph-osd-ssd-server-2 {
> id -2
> alg straw
> hash 0
> item osd.2 weight 1.00
> item osd.3 weight 1.00
> }
>
> host ceph-osd-platter-server-1 {
> id -3
> alg straw
> hash 0
> item osd.4 weight 1.00
> item osd.5 weight 1.00
> }
>
> host ceph-osd-platter-server-2 {
> id -4
> alg straw
> hash 0
> item osd.6 weight 1.00
> item osd.7 weight 1.00
> }
>
> root platter {
> id -5
> alg straw
> hash 0
> item ceph-osd-platter-server-1 weight 2.00
> item ceph-osd-platter-server-2 weight 2.00
> }
>
> root ssd {
> id -6
> alg straw
> hash 0
> item ceph-osd-ssd-server-1 weight 2.00
> item ceph-osd-ssd-server-2 weight 2.00
> }
>
> rule data {
> ruleset 0
> type replicated
> min_size 2
> max_size 2
> step take platter
> step chooseleaf 0 type host
> step emit
> }
>
> rule metadata {
> ruleset 1
> type replicated
> min_size 0
> max_size 10
> step take platter
> step chooseleaf 0 type host
> step emit
> }
>
> rule rbd {
> ruleset 2
> type replicated
> min_size 0
> max_size 10
> step take platter
> step chooseleaf 0 type host
> step emit
> }
>
> rule platter {
> ruleset 3
> type replicated
> min_size 0
> max_size 10
> step take platter
> step chooseleaf 0 type host
> step emit
> }
>
> rule ssd {
> ruleset 4
> type replicated
> min_size 0
> max_size 10
> step take ssd
> step chooseleaf 0 type host
> step emit
> }
>
> rule ssd-primary {
> ruleset 4
> type replicated
> min_size 0
> max_size 10
> step take ssd
> step chooseleaf 1 type host
> step emit
> step take platter
> step chooseleaf -1 type host
> step emit
> }
>
> You can then set a pool to use the ssd rule by:
> ceph osd pool set <poolname> crush_ruleset 4
>
> Similarly, using the ssd-primary rule will cause
> each pg in the pool to be placed with an ssd as
> the primary and platters as the replicas.
>
> -Sam
>
> On Mon, Dec 10, 2012 at 11:17 AM, Alexandre Maumené
> <alexandre@maumene.org> wrote:
>> Hello all,
>>
>> I have a few questions about Ceph:
>>
>> 1) Is it possible to run a cluster with "some" lantecy between monitor
>> nodes? Latency will be 30ms at worst.
>>
>> 2) When using RBD what are the best practices for a direct mount using
>> XFS filesystem? And for a qemu/kvm devices? I'm thinking about
>> writeback, rbd_cache, ...
>>
>> 3) About the CRUSH map, how can I separate 2 pools on different OSD?
>> I'd like to setup a cluster with different disks (like SATA/SAS) and I
>> want to be able to specify on which disks (or OSD) my data are going
>> to be write.
>>
>> Thanks in advance for any answer.
>>
>> Regards,
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2012-12-10 22:31 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-12-10 19:17 Few questions about Ceph Alexandre Maumené
2012-12-10 22:01 ` Joao Eduardo Luis
2012-12-10 22:22 ` Samuel Just
2012-12-10 22:31 ` Samuel Just
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.