Re: questions on networks and hardware

[Wido den Hollander <wido@widodh.nl>]

Hi,

On 01/19/2013 12:34 AM, John Nielsen wrote:
> I'm planning a Ceph deployment which will include:
> 	10Gbit/s public/client network
> 	10Gbit/s cluster network
> 	dedicated mon hosts (3 to start)
> 	dedicated storage hosts (multiple disks, one XFS and OSD per disk, 3-5 to start)
> 	dedicated RADOS gateway host (1 to start)
>
> I've done some initial testing and read through most of the docs but I still have a few questions. Please respond even if you just have a suggestion or response for one of them.
>
> If I have "cluster network" and "public network" entries under [global] or [osd], do I still need to specify "public addr" and "cluster addr" for each OSD individually?
>

Already answered, but no. You don't need to. The OSDs will bind to the 
available IP in that network.

> Which network(s) should the monitor hosts be on? If both, is it valid to have more than one "mon addr" entry per mon host or is there a different way to do it?
>

They should be on the "public" network since the clients also need to be 
able to reach the monitors.

> Is it worthwhile to have 10G NIC's on the monitor hosts? (The storage hosts will each have 2x 10Gbit/s NIC's.)
>

No, not really. 1Gbit should be more then enough for your monitors. 3 
monitors should also be good. No need to go for 5 or 7.

> I'd like to have 2x 10Gbit/s NIC's on the gateway host and maximize throughput. Any suggestions on how to best do that? I'm assuming it will talk to the OSD's on the Ceph public/client network, so does that imply a third even-more-public network for the gateway's clients?
>

No, there is no third network. You could trunk the 2 NICs with LACP or 
something. Since the client will open TCP connections to all the OSDs 
you will get a pretty good balancing over the available bandwith.

> I think this has come up before, but has anyone written up something with more details on setting up gateways? Hardware recommendations, strategies to improve caching and performance, multiple gateway setups with and without a load balancer, etc.

Not that I know. I'm still trying to play with RGW and Varnish in front 
of it, but haven't really taken the time yet. The goal is to offload a 
lot of the caching to Varnish and have Varnish 'ban' objects when they 
change.

You could also use Varnish as a loadbalancer. But in this case you could 
also use RR-DNS or LVS with Direct Routing, that way only incoming 
traffic goes through the loadbalancer and return traffic goes directly 
to your network's gateway.

Wido

>
> Thanks!
>
> JN
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>