Fwd: Ceph Production Environment Setup and Configurations?

Fwd: Ceph Production Environment Setup and Configurations?

[femi anjorin]

Hi,

Please with regards to my questions on Ceph Production Environment ...
I like to give u these details.

i like to test a write, read and delete operation on ceph storage
cluster in a production environment.

i also like to check the self healing and managing functionalities.

i like to know in the production setup , are gateways required for any of the
three methods of accessing ceph cluster? or should the setup just be like
all the servers should be storage nodes with mon,mds and osd running
on each of them ...while i access these storage nodes through a single
computer which one can call a client just like you described in the 5
mins setup?


---------- Forwarded message ----------
Date: Tue, Jan 29, 2013 at 2:56 AM
Subject: Ceph Production Environment Setup and Configurations?
To: ceph-devel@vger.kernel.org


Please can anyone  an advise  on how exactly a CEPH production
environment should look like? and what the configuration files should
be. My hardwares include the following:



Server A, B, C configuration

CPU - Intel(R) Core(TM)2 Quad  CPU   Q9550  @ 2.83GHz

RAM - 16GB

Hard drive -  500GB

SSD - 120GB



Server D,E,F,G,H,J configuration

CPU - Intel(R) Atom(TM) CPU D525   @ 1.80GHz

RAM - 4 GB

Boot drive -  320gb

SSD - 120 GB

Storage drives - 16 X 2 TB



I am thinking of these configurations but i am not sure.

Server A - MDS and MON

Server B - MON

Server C - MON

Server D, E,F,G,H,J - OSD



Regards.

Re: Ceph Production Environment Setup and Configurations?

[Gandalf Corvotempesta]

2013/1/29 femi anjorin <femi.anjorin@gmail.com>:
> CPU - Intel(R) Atom(TM) CPU D525   @ 1.80GHz

Atom? For which kind of role ?

RE: Ceph Production Environment Setup and Configurations?

[Chen, Xiaoxi]

[The following views only behalf of myself, not relate with Intel..]
Looking forward for the performance data on Atom.
Atom perform badly in Swift, but since Ceph is slightly efficient than Swift, it must be better.
I have some concern about weather Atom can support such high throughput( you have 16 disks, assuming 50MB/s per disk, you would like to have 50*16 *2(include journal write)=1600MB/s ,together with corresponding network throughput, say 10GbE. The total IO throughput seems too high for Atom. Baidu (www.baidu.com) use Arm based storage node(not using ceph,but their own DFS), a node with a quad-core Arm together with 4 disks, and a 2U box can put up to 6 nodes, the 6 nodes share a 10Gb NIC. Although Atom is different with Arm, but you can take Baidu as a reference.


-----Original Message-----
From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Gandalf Corvotempesta
Sent: 2013年1月29日 17:25
To: femi anjorin
Cc: ceph-devel@vger.kernel.org; Ross Turk
Subject: Re: Ceph Production Environment Setup and Configurations?

2013/1/29 femi anjorin <femi.anjorin@gmail.com>:
> CPU - Intel(R) Atom(TM) CPU D525   @ 1.80GHz

Atom? For which kind of role ?
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Ceph Production Environment Setup and Configurations?

[Joao Eduardo Luis]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=GB2312, Size: 2166 bytes --]

On 01/29/2013 11:25 AM, Chen, Xiaoxi wrote:
> [The following views only behalf of myself, not relate with Intel..]
> Looking forward for the performance data on Atom.
> Atom perform badly in Swift, but since Ceph is slightly efficient than Swift, it must be better.
> I have some concern about weather Atom can support such high throughput( you have 16 disks, assuming 50MB/s per disk, you would like to have 50*16 *2(include journal write)=1600MB/s ,together with corresponding network throughput, say 10GbE. The total IO throughput seems too high for Atom. Baidu (www.baidu.com) use Arm based storage node(not using ceph,but their own DFS), a node with a quad-core Arm together with 4 disks, and a 2U box can put up to 6 nodes, the 6 nodes share a 10Gb NIC. Although Atom is different with Arm, but you can take Baidu as a reference.
> 
> 
> -----Original Message-----
> From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Gandalf Corvotempesta
> Sent: 2013Äê1ÔÂ29ÈÕ 17:25
> To: femi anjorin
> Cc: ceph-devel@vger.kernel.org; Ross Turk
> Subject: Re: Ceph Production Environment Setup and Configurations?
> 
> 2013/1/29 femi anjorin <femi.anjorin@gmail.com>:
>> CPU - Intel(R) Atom(TM) CPU D525   @ 1.80GHz
> 
> Atom? For which kind of role ?
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html
> N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^\x7f)Þº{.n\x7f+‰·œz˜]z÷¥Š{ay\x7f\x1dʇڙ\x7f,j\a­¢f£¢·hš‹àz\x7f\x1e®w¥¢\x7f\f¢·¦j:+v‰¨ŠwèjØm¶Ÿ\x7f\x7f\a«‘êçzZ+ƒùšŽŠÝ¢j"ú!tml=
> 


FWIW, Wido had been testing Ceph on Atoms a while back.  I don't know
what conclusions he reached, or even if he has already reached any
conclusions at all.  I, for one, would be really interested in knowing
how well (or how poorly) Atoms deal with Ceph.

Maybe Wido can share some thoughts on this one? :-)

  -Joao
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Ceph Production Environment Setup and Configurations?

[Mark Nelson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=GB2312, Size: 3024 bytes --]

On 01/29/2013 05:34 AM, Joao Eduardo Luis wrote:
> On 01/29/2013 11:25 AM, Chen, Xiaoxi wrote:
>> [The following views only behalf of myself, not relate with Intel..]
>> Looking forward for the performance data on Atom.
>> Atom perform badly in Swift, but since Ceph is slightly efficient than Swift, it must be better.
>> I have some concern about weather Atom can support such high throughput( you have 16 disks, assuming 50MB/s per disk, you would like to have 50*16 *2(include journal write)=1600MB/s ,together with corresponding network throughput, say 10GbE. The total IO throughput seems too high for Atom. Baidu (www.baidu.com) use Arm based storage node(not using ceph,but their own DFS), a node with a quad-core Arm together with 4 disks, and a 2U box can put up to 6 nodes, the 6 nodes share a 10Gb NIC. Although Atom is different with Arm, but you can take Baidu as a reference.
>>
>>
>> -----Original Message-----
>> From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Gandalf Corvotempesta
>> Sent: 2013Äê1ÔÂ29ÈÕ 17:25
>> To: femi anjorin
>> Cc: ceph-devel@vger.kernel.org; Ross Turk
>> Subject: Re: Ceph Production Environment Setup and Configurations?
>>
>> 2013/1/29 femi anjorin <femi.anjorin@gmail.com>:
>>> CPU - Intel(R) Atom(TM) CPU D525   @ 1.80GHz
>>
>> Atom? For which kind of role ?
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^\x7f\x7f)Þº{.n\x7f\x7f+‰·œz˜]z÷¥Š{ay\x7f\x7f\x1dʇڙ\x7f\x7f,j\a­¢f£¢·hš‹àz\x7f\x7f\x1e®w¥¢\x7f\x7f\f¢·¦j:+v‰¨ŠwèjØm¶Ÿ\x7f\x7f\x7f\x7f\a«‘êçzZ+ƒùšŽŠÝ¢j"ú!tml=
>>
> 
> 
> FWIW, Wido had been testing Ceph on Atoms a while back.  I don't know
> what conclusions he reached, or even if he has already reached any
> conclusions at all.  I, for one, would be really interested in knowing
> how well (or how poorly) Atoms deal with Ceph.
> 
> Maybe Wido can share some thoughts on this one? :-)
> 
>    -Joao
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Just FYI, I don't expect you'll be able to get anywhere close to 10GbE
performance on the Atom.  You'll almost certainly be CPU bound long
before that.  I actually would be curious how much throughput you could
push to those disks just doing straight fio tests and how much CPU
overhead you'd see.  I imagine you could max out the CPU without even
having Ceph involved.

I'm guessing you probably are going to be limited to about 0.75-1 OSD
per atom core, and that's assuming that your network and SAS controllers
aren't CPU hogs.

Mark
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html