systemd unit files and multiple daemons

systemd unit files and multiple daemons

[Ken Dreyer]

I could really use some eyes on the systemd change proposed here:
http://tracker.ceph.com/issues/11344

Specifically, on bullet #4 there, should we have a single
"ceph-mon.service" (implying that users should only run one monitor
daemon per server) or if we should support multiple "ceph-mon@" services
(implying that users will need to specify additional information when
starting the service(s)). The version in our tree is "ceph-mon@". James'
work for Ubuntu Vivid is only "ceph-mon" [2]. Same thing for ceph-mds vs
ceph-mds@.

I'd prefer to keep Ubuntu downstream the same as Ceph upstream.

What do we want to do for this?

How common is it to run multiple monitor daemons or mds daemons on a
single host?

- Ken


[1] https://github.com/ceph/ceph/tree/master/systemd
[2]
http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/vivid/ceph/vivid/files/head:/debian/lib-systemd/

Re: systemd unit files and multiple daemons

[Gregory Farnum]

On Wed, Apr 22, 2015 at 2:57 PM, Ken Dreyer <kdreyer@redhat.com> wrote:
> I could really use some eyes on the systemd change proposed here:
> http://tracker.ceph.com/issues/11344
>
> Specifically, on bullet #4 there, should we have a single
> "ceph-mon.service" (implying that users should only run one monitor
> daemon per server) or if we should support multiple "ceph-mon@" services
> (implying that users will need to specify additional information when
> starting the service(s)). The version in our tree is "ceph-mon@". James'
> work for Ubuntu Vivid is only "ceph-mon" [2]. Same thing for ceph-mds vs
> ceph-mds@.
>
> I'd prefer to keep Ubuntu downstream the same as Ceph upstream.
>
> What do we want to do for this?
>
> How common is it to run multiple monitor daemons or mds daemons on a
> single host?

For a real deployment, you shouldn't be running multiple monitors on a
single node in the general case. I'm not sure if we want to prohibit
it by policy, but I'd be okay with the idea.
For testing purposes (in ceph-qa-suite or using vstart as a developer)
it's pretty common though, and we probably don't want to have to
rewrite all our tests to change that. I'm not sure that vstart ever
uses the regular init system, but teuthology/ceph-qa-suite obviously
do!

For MDSes, it's probably appropriate/correct to support multiple
daemons on the same host. This can be either a fault tolerance thing,
or just a way of better using multiple cores if you're living on the
(very dangerous) edge.
-Greg

Re: systemd unit files and multiple daemons

[HEWLETT, Paul (Paul)** CTR **]

What about running multiple clusters on the same host?

There is a separate mail thread about being able to run clusters with different conf files on the same host.
Will the new systemd service scripts cope with this?

Paul Hewlett
Senior Systems Engineer
Velocix, Cambridge
Alcatel-Lucent
t: +44 1223 435893



________________________________________
From: ceph-users [ceph-users-bounces-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org] on behalf of Gregory Farnum [greg-3KCAGdo1P2hBDgjK7y7TUQ@public.gmane.org]
Sent: 22 April 2015 23:26
To: Ken Dreyer
Cc: ceph-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
Subject: Re: [ceph-users] systemd unit files and multiple daemons

On Wed, Apr 22, 2015 at 2:57 PM, Ken Dreyer <kdreyer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> I could really use some eyes on the systemd change proposed here:
> http://tracker.ceph.com/issues/11344
>
> Specifically, on bullet #4 there, should we have a single
> "ceph-mon.service" (implying that users should only run one monitor
> daemon per server) or if we should support multiple "ceph-mon@" services
> (implying that users will need to specify additional information when
> starting the service(s)). The version in our tree is "ceph-mon@". James'
> work for Ubuntu Vivid is only "ceph-mon" [2]. Same thing for ceph-mds vs
> ceph-mds@.
>
> I'd prefer to keep Ubuntu downstream the same as Ceph upstream.
>
> What do we want to do for this?
>
> How common is it to run multiple monitor daemons or mds daemons on a
> single host?

For a real deployment, you shouldn't be running multiple monitors on a
single node in the general case. I'm not sure if we want to prohibit
it by policy, but I'd be okay with the idea.
For testing purposes (in ceph-qa-suite or using vstart as a developer)
it's pretty common though, and we probably don't want to have to
rewrite all our tests to change that. I'm not sure that vstart ever
uses the regular init system, but teuthology/ceph-qa-suite obviously
do!

For MDSes, it's probably appropriate/correct to support multiple
daemons on the same host. This can be either a fault tolerance thing,
or just a way of better using multiple cores if you're living on the
(very dangerous) edge.
-Greg
_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: systemd unit files and multiple daemons

[Sage Weil]

On Thu, 23 Apr 2015, HEWLETT, Paul (Paul)** CTR ** wrote:
> What about running multiple clusters on the same host?
> 
> There is a separate mail thread about being able to run clusters with different conf files on the same host.
> Will the new systemd service scripts cope with this?

As currently planned, no.  Unfortunately systemd only allows a single 
substitution/id for identifying a daemon instance.  If we try to use 
that for both cluster and (osd/mon) id (e.g., ceph-1, mycluster-232) 
it gets ugly because we can separate them into different fields.  
The current plan is for the cluster name to be specified in 
/etc/sysconfig/ceph or similar.

I'm hoping anyone who really needs multiple clusters on the same host can 
accomplish that using containers... would that cover your use case?

sage


> 
> Paul Hewlett
> Senior Systems Engineer
> Velocix, Cambridge
> Alcatel-Lucent
> t: +44 1223 435893
> 
> 
> 
> ________________________________________
> From: ceph-users [ceph-users-bounces-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org] on behalf of Gregory Farnum [greg-3KCAGdo1P2hBDgjK7y7TUQ@public.gmane.org]
> Sent: 22 April 2015 23:26
> To: Ken Dreyer
> Cc: ceph-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
> Subject: Re: [ceph-users] systemd unit files and multiple daemons
> 
> On Wed, Apr 22, 2015 at 2:57 PM, Ken Dreyer <kdreyer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> > I could really use some eyes on the systemd change proposed here:
> > http://tracker.ceph.com/issues/11344
> >
> > Specifically, on bullet #4 there, should we have a single
> > "ceph-mon.service" (implying that users should only run one monitor
> > daemon per server) or if we should support multiple "ceph-mon@" services
> > (implying that users will need to specify additional information when
> > starting the service(s)). The version in our tree is "ceph-mon@". James'
> > work for Ubuntu Vivid is only "ceph-mon" [2]. Same thing for ceph-mds vs
> > ceph-mds@.
> >
> > I'd prefer to keep Ubuntu downstream the same as Ceph upstream.
> >
> > What do we want to do for this?
> >
> > How common is it to run multiple monitor daemons or mds daemons on a
> > single host?
> 
> For a real deployment, you shouldn't be running multiple monitors on a
> single node in the general case. I'm not sure if we want to prohibit
> it by policy, but I'd be okay with the idea.
> For testing purposes (in ceph-qa-suite or using vstart as a developer)
> it's pretty common though, and we probably don't want to have to
> rewrite all our tests to change that. I'm not sure that vstart ever
> uses the regular init system, but teuthology/ceph-qa-suite obviously
> do!
> 
> For MDSes, it's probably appropriate/correct to support multiple
> daemons on the same host. This can be either a fault tolerance thing,
> or just a way of better using multiple cores if you're living on the
> (very dangerous) edge.
> -Greg
> _______________________________________________
> ceph-users mailing list
> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
>