firewall questions

firewall questions

[Ken Dreyer]

It would be really convenient to have human-readable firewalld service
definitions for Ceph, so that users could do things like:

  firewall-cmd --add-service=ceph-mon

or

  firewall-cmd --add-service=ceph

... instead of having to know specific port numbers to open.

In order to submit service definitions to firewalld upstream, I had a
couples questions:

1. In April there was a mailing list thread about the IANA #821110
ticket Sage filed for ceph-mon. Did anything come of that? I filed
http://tracker.ceph.com/issues/11689 to track this in Redmine.

2. I talked recently with Sam about the possible ports an OSD could use,
and our conversation made me think that our firewall docs for OSDs and
MDSs might need to be updated: http://tracker.ceph.com/issues/11688

Currently the docs say "calculate the number of OSDs or MDSs you're
running and that will tell you what ports to open". That makes it hard
to write a service definition for firewalld, since those are just a list
of static ports.

3. Lastly, on a scale of "yeah, sounds do-able" to "everything will come
to a grinding halt", how hard would it be to run with firewalls enabled
in our sepia and typica labs that are running Teuthology? :) Do our
Teuthology tests use TCP ports outside of 80, 6789, and 6800-7300 ?

- Ken

Re: firewall questions

[Sage Weil]

On Wed, 20 May 2015, Ken Dreyer wrote:
> It would be really convenient to have human-readable firewalld service
> definitions for Ceph, so that users could do things like:
> 
>   firewall-cmd --add-service=ceph-mon
> 
> or
> 
>   firewall-cmd --add-service=ceph
> 
> ... instead of having to know specific port numbers to open.
> 
> In order to submit service definitions to firewalld upstream, I had a
> couples questions:
> 
> 1. In April there was a mailing list thread about the IANA #821110
> ticket Sage filed for ceph-mon. Did anything come of that? I filed
> http://tracker.ceph.com/issues/11689 to track this in Redmine.

Not yet.  Was waiting for someone to suggest a port they liked off 
the free list and then got distracted.

Either way, though, we should make an interim rule on the current ports 
since the transition will take a while.

> 2. I talked recently with Sam about the possible ports an OSD could use,
> and our conversation made me think that our firewall docs for OSDs and
> MDSs might need to be updated: http://tracker.ceph.com/issues/11688
> 
> Currently the docs say "calculate the number of OSDs or MDSs you're
> running and that will tell you what ports to open". That makes it hard
> to write a service definition for firewalld, since those are just a list
> of static ports.

Yeah, I'm afraid it should just be teh full range we allow... I think 
6800-7100 by default?

> 3. Lastly, on a scale of "yeah, sounds do-able" to "everything will come
> to a grinding halt", how hard would it be to run with firewalls enabled
> in our sepia and typica labs that are running Teuthology? :) Do our
> Teuthology tests use TCP ports outside of 80, 6789, and 6800-7300 ?

Civetweb binds to 7280 (?) by default, but I don't think we use it on that 
port in teuthology.  Soo... I think it'll work!

s

Re: firewall questions

[Ken Dreyer]

On 05/20/2015 04:53 PM, Sage Weil wrote:
>> 2. I talked recently with Sam about the possible ports an OSD could use,
>> and our conversation made me think that our firewall docs for OSDs and
>> MDSs might need to be updated: http://tracker.ceph.com/issues/11688
>>
>> Currently the docs say "calculate the number of OSDs or MDSs you're
>> running and that will tell you what ports to open". That makes it hard
>> to write a service definition for firewalld, since those are just a list
>> of static ports.
> 
> Yeah, I'm afraid it should just be teh full range we allow... I think 
> 6800-7100 by default?

src/common/config_opts.h from master says:

  OPTION(ms_bind_port_min, OPT_INT, 6800)
  OPTION(ms_bind_port_max, OPT_INT, 7300)

I think that would mean we'd want to open 6800-7300 by default?

And for this firewalld service name, I was thinking of naming this
6800-7300 rule "ceph", since it encompasses both the OSD and MDS
services. Does that name sound ok? (And I'd name the 6789 rule "ceph-mon".)

- Ken

Re: firewall questions

[Sage Weil]

On Thu, 21 May 2015, Ken Dreyer wrote:
> On 05/20/2015 04:53 PM, Sage Weil wrote:
> >> 2. I talked recently with Sam about the possible ports an OSD could use,
> >> and our conversation made me think that our firewall docs for OSDs and
> >> MDSs might need to be updated: http://tracker.ceph.com/issues/11688
> >>
> >> Currently the docs say "calculate the number of OSDs or MDSs you're
> >> running and that will tell you what ports to open". That makes it hard
> >> to write a service definition for firewalld, since those are just a list
> >> of static ports.
> > 
> > Yeah, I'm afraid it should just be teh full range we allow... I think 
> > 6800-7100 by default?
> 
> src/common/config_opts.h from master says:
> 
>   OPTION(ms_bind_port_min, OPT_INT, 6800)
>   OPTION(ms_bind_port_max, OPT_INT, 7300)
> 
> I think that would mean we'd want to open 6800-7300 by default?
> 
> And for this firewalld service name, I was thinking of naming this
> 6800-7300 rule "ceph", since it encompasses both the OSD and MDS
> services. Does that name sound ok? (And I'd name the 6789 rule "ceph-mon".)

Yep, sounds good to me!

sage

Re: firewall questions

[Ken Dreyer]

On 05/21/2015 09:37 AM, Sage Weil wrote:
> On Thu, 21 May 2015, Ken Dreyer wrote:
>> I think that would mean we'd want to open 6800-7300 by default?
>>
>> And for this firewalld service name, I was thinking of naming this
>> 6800-7300 rule "ceph", since it encompasses both the OSD and MDS
>> services. Does that name sound ok? (And I'd name the 6789 rule "ceph-mon".)
> 
> Yep, sounds good to me!
> 

Cool. New Ceph configs submitted to firewalld at
https://github.com/t-woerner/firewalld/pull/22

I've also updated our own docs re: iptables, at
https://github.com/ceph/ceph/pull/4740

- Ken