rgw/civetweb privileged port bind

rgw/civetweb privileged port bind

[Karol Mroz]

[-- Attachment #1: Type: text/plain, Size: 938 bytes --]

Hello,

As I understand it, with the release of infernalis, ceph
daemons are no longer being run as root. Thus, rgw/civetweb
is unable to bind to privileged ports:

http://tracker.ceph.com/issues/13600

We encountered this problem as well in our downstream (hammer
based) product, where we run rgw/civetweb as "wwwuser". To allow
privileged port binding, we used file caps (setcap from the spec file).
Going forward, however, we were thinking of taking one of two
approaches:

1. Start rgw/civetweb as root and utilize an existing civetweb
config option (run_as_user) to drop permissions _after_
the port bind and after certificate files have been read.

2. Utilize systemd socket activation, and allow systemd to bind
to the necessary port. Once rgw/civetweb is started, civetweb
can pull the listening socket from systemd.

Is this something you folks upstream have given some thought to?

-- 
Regards,
Karol

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: rgw/civetweb privileged port bind

[Sage Weil]

On Thu, 26 Nov 2015, Karol Mroz wrote:
> Hello,
> 
> As I understand it, with the release of infernalis, ceph
> daemons are no longer being run as root. Thus, rgw/civetweb
> is unable to bind to privileged ports:
> 
> http://tracker.ceph.com/issues/13600
> 
> We encountered this problem as well in our downstream (hammer
> based) product, where we run rgw/civetweb as "wwwuser". To allow
> privileged port binding, we used file caps (setcap from the spec file).
> Going forward, however, we were thinking of taking one of two
> approaches:
> 
> 1. Start rgw/civetweb as root and utilize an existing civetweb
> config option (run_as_user) to drop permissions _after_
> the port bind and after certificate files have been read.
>
> 2. Utilize systemd socket activation, and allow systemd to bind
> to the necessary port. Once rgw/civetweb is started, civetweb
> can pull the listening socket from systemd.
> 
> Is this something you folks upstream have given some thought to?

I haven't. #2 sounds like it's harder, and I'm not sure it brings a lot fo 
benefit. Making #1 work is probably super simple (replace our set user 
option with the civetweb one?)...

What do you suggest?

sage

Re: rgw/civetweb privileged port bind

[Karol Mroz]

[-- Attachment #1: Type: text/plain, Size: 2385 bytes --]

On Thu, Nov 26, 2015 at 11:38:26AM -0800, Sage Weil wrote:
> On Thu, 26 Nov 2015, Karol Mroz wrote:
> > Hello,
> > 
> > As I understand it, with the release of infernalis, ceph
> > daemons are no longer being run as root. Thus, rgw/civetweb
> > is unable to bind to privileged ports:
> > 
> > http://tracker.ceph.com/issues/13600
> > 
> > We encountered this problem as well in our downstream (hammer
> > based) product, where we run rgw/civetweb as "wwwuser". To allow
> > privileged port binding, we used file caps (setcap from the spec file).
> > Going forward, however, we were thinking of taking one of two
> > approaches:
> > 
> > 1. Start rgw/civetweb as root and utilize an existing civetweb
> > config option (run_as_user) to drop permissions _after_
> > the port bind and after certificate files have been read.
> >
> > 2. Utilize systemd socket activation, and allow systemd to bind
> > to the necessary port. Once rgw/civetweb is started, civetweb
> > can pull the listening socket from systemd.
> > 
> > Is this something you folks upstream have given some thought to?
> 
> I haven't. #2 sounds like it's harder, and I'm not sure it brings a lot fo 
> benefit. Making #1 work is probably super simple (replace our set user 
> option with the civetweb one?)...
> 
> What do you suggest?

Hi Sage,

I agree with you that #2 would be more work. I expect there may be some gains
in startup time associated with socket activation along with the other
systemd benefits (cgroups, etc), which we may already be taking advantage of?

A couple months ago, I played around with option #1 in one of our downstream
branches. Basically something like this: https://github.com/SUSE/ceph/pull/22
Further, we'd best pull in a couple civetweb upstream commits into the
maintained civetweb submodule. Namely:

https://github.com/civetweb/civetweb/commit/5e753cc
  - civetweb.c: fix setgroups() -Wimplicit-function-declaration
https://github.com/civetweb/civetweb/commit/a3c460c
  - call setgroups() to avoid supplemental group leakage

So, option #1 is perhaps a decent solution for rgw/civetweb for the time being.
Later on, we might want to examine the prospect of leveraging socket activation
for all of ceph. I don't think the permissions drop in rgw/civetweb would negatively
impact such a plan in the future.

-- 
Regards,
Karol

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]