Re: Hardware Requirements for RADOS Gateway Cluster

[Mark Nelson <mark.nelson@inktank.com>]

Hi Brice!

On 09/24/2012 05:58 PM, Brice Burgess wrote:
>I am planning the system architecture around a ceph cluster [my first!].
> While resources for setting up the basic cluster (MON+MDS+OSD) are
> readily available*, I haven't come across anything outlining production
> recommendations for a RADOS Gateway Cluster. Specifically;
>
> 1. Is it preferable to run the RADOS Gateway on a MDS machine [for
> latency issues], or should the Gateway run on seperate hosts/VMs than
> the ODS/MON/MDS servers?
>

It's entirely possible to run the gateway on an OSD/MON/MDS.  I don't 
think we've done any kind of extensive analysis of where the best place 
to run one is.  My guess is that it will only matter in very specific cases.

> 2. I've seen reference that multiple RADOS Gateway servers can be setup
> as a cluster "to scale". Is this possible? Is there an
> example/documentation for this? Is it just 2 standalone gateways with a
> load balancer in front?
>

Yes!  It's entirely possible to run multiple RGWs with a load balancer 
in front.  I'm not sure how much documentation is out there for this 
yet, but it's something that customers of ours have successfully 
implemented.

> My assumption is to provision a dedicated machine for the RADOS Gateway.
> I'd treat this machine as a "front end proxy/caching server" meaning it
> would have a lot of RAM for varnish/nginx and a low latency, high
> throughput network connection to the ODS/MDS machines as well as one to
> the public network.

Sounds good to me.  Other people might chime in with more suggestions.

>
> Any thoughts and suggestions are welcome. Eventually I'd like to release
> a whitepaper of this setup if we can hit the budget to actually
> implement ;)

Some things you'll want to keep in mind:

You'll want to make sure that your testing (especially of small IO) is 
done across many buckets.  Performance of a single bucket can be limited 
by the OSD where it's log is stored (or optionally you can turn the log 
off).  You'll also want to keep in mind that RGW will likely incur 
larger performance overhead for small objects than large ones.

Thanks,
Mark