RFC: tool for applying 'ceph daemon <osd>' command to all OSDs

RFC: tool for applying 'ceph daemon <osd>' command to all OSDs

[Dan Mick]

I needed something to fetch current config values from all OSDs (sorta
the opposite of 'injectargs --key value), so I hacked it, and then
spiffed it up a bit.  Does this seem like something that would be useful
in this form in the upstream Ceph, or does anyone have any thoughts on
its design or structure?

It requires a locally-installed ceph CLI and a ceph.conf that points to
the cluster and any required keyrings.  You can also provide it with
a YAML file mapping host to osds if you want to save time collecting
that info for a statically-defined cluster, or if you want just a subset
of OSDs.

https://github.com/dmick/tools/blob/master/osd_daemon_cmd.py

Excerpt from usage:

Execute a Ceph osd daemon command on every OSD in a cluster with
one connection to each OSD host.

Usage:
    osd_daemon_cmd [-c CONF] [-u USER] [-f FILE] (COMMAND | -k KEY)

Options:
   -c CONF   ceph.conf file to use [default: ./ceph.conf]
   -u USER   user to connect with ssh
   -f FILE   get names and osds from yaml
   COMMAND   command other than "config get" to execute
   -k KEY    config key to retrieve with config get <key>

-- 
Dan Mick
Red Hat, Inc.
Ceph docs: http://ceph.com/docs

Re: RFC: tool for applying 'ceph daemon <osd>' command to all OSDs

[Gregory Farnum]

On Mon, Dec 21, 2015 at 9:59 PM, Dan Mick <dmick@redhat.com> wrote:
> I needed something to fetch current config values from all OSDs (sorta
> the opposite of 'injectargs --key value), so I hacked it, and then
> spiffed it up a bit.  Does this seem like something that would be useful
> in this form in the upstream Ceph, or does anyone have any thoughts on
> its design or structure?
>
> It requires a locally-installed ceph CLI and a ceph.conf that points to
> the cluster and any required keyrings.  You can also provide it with
> a YAML file mapping host to osds if you want to save time collecting
> that info for a statically-defined cluster, or if you want just a subset
> of OSDs.
>
> https://github.com/dmick/tools/blob/master/osd_daemon_cmd.py
>
> Excerpt from usage:
>
> Execute a Ceph osd daemon command on every OSD in a cluster with
> one connection to each OSD host.
>
> Usage:
>     osd_daemon_cmd [-c CONF] [-u USER] [-f FILE] (COMMAND | -k KEY)
>
> Options:
>    -c CONF   ceph.conf file to use [default: ./ceph.conf]
>    -u USER   user to connect with ssh
>    -f FILE   get names and osds from yaml
>    COMMAND   command other than "config get" to execute
>    -k KEY    config key to retrieve with config get <key>

I naively like the functionality being available, but if I'm skimming
this correctly it looks like you're relying on the local node being
able to passwordless-ssh to all of the nodes, and for that account to
be able to access the ceph admin sockets. Granted we rely on the ssh
for ceph-deploy as well, so maybe that's okay, but I'm not sure in
this case since it implies a lot more network openness.

Relatedly (perhaps in an opposing direction), maybe we want anything
exposed over the network to have some sort of explicit permissions
model?

Maybe not and we should just ship the script for trusted users. I
would have liked it on the long-running cluster I'm sure you built it
for. ;)
Indecisively yours,
-Greg

Re: RFC: tool for applying 'ceph daemon <osd>' command to all OSDs

[Dan Mick]

On 12/21/2015 11:29 PM, Gregory Farnum wrote:
> On Mon, Dec 21, 2015 at 9:59 PM, Dan Mick <dmick@redhat.com> wrote:
>> I needed something to fetch current config values from all OSDs (sorta
>> the opposite of 'injectargs --key value), so I hacked it, and then
>> spiffed it up a bit.  Does this seem like something that would be useful
>> in this form in the upstream Ceph, or does anyone have any thoughts on
>> its design or structure?
>>
>> It requires a locally-installed ceph CLI and a ceph.conf that points to
>> the cluster and any required keyrings.  You can also provide it with
>> a YAML file mapping host to osds if you want to save time collecting
>> that info for a statically-defined cluster, or if you want just a subset
>> of OSDs.
>>
>> https://github.com/dmick/tools/blob/master/osd_daemon_cmd.py
>>
>> Excerpt from usage:
>>
>> Execute a Ceph osd daemon command on every OSD in a cluster with
>> one connection to each OSD host.
>>
>> Usage:
>>     osd_daemon_cmd [-c CONF] [-u USER] [-f FILE] (COMMAND | -k KEY)
>>
>> Options:
>>    -c CONF   ceph.conf file to use [default: ./ceph.conf]
>>    -u USER   user to connect with ssh
>>    -f FILE   get names and osds from yaml
>>    COMMAND   command other than "config get" to execute
>>    -k KEY    config key to retrieve with config get <key>
> 
> I naively like the functionality being available, but if I'm skimming
> this correctly it looks like you're relying on the local node being
> able to passwordless-ssh to all of the nodes, and for that account to
> be able to access the ceph admin sockets. Granted we rely on the ssh
> for ceph-deploy as well, so maybe that's okay, but I'm not sure in
> this case since it implies a lot more network openness.

Yep; it's basically the same model and role assumed as "cluster destroyer".

> Relatedly (perhaps in an opposing direction), maybe we want anything
> exposed over the network to have some sort of explicit permissions
> model?

Well, I've heard that idea floated about the admin socket for years, but
I don't think anyone's hot to add cephx to it :)

> Maybe not and we should just ship the script for trusted users. I
> would have liked it on the long-running cluster I'm sure you built it
> for. ;)

it's like you're clairvoyant.

RE: tool for applying 'ceph daemon <osd>' command to all OSDs

[Igor.Podoski]

> -----Original Message-----
> From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel-
> owner@vger.kernel.org] On Behalf Of Dan Mick
> Sent: Tuesday, December 22, 2015 7:00 AM
> To: ceph-devel
> Subject: RFC: tool for applying 'ceph daemon <osd>' command to all OSDs
> 
> I needed something to fetch current config values from all OSDs (sorta the
> opposite of 'injectargs --key value), so I hacked it, and then spiffed it up a bit.
> Does this seem like something that would be useful in this form in the
> upstream Ceph, or does anyone have any thoughts on its design or
> structure?
>

You could do it using socat too:

Node1 has osd.0

Node1:
cd /var/run/ceph
sudo socat TCP-LISTEN:60100,fork unix-connect:ceph-osd.0.asok

Node2:
cd /var/run/ceph
sudo  socat unix-listen:ceph-osd.0.asok,fork TCP:Node1:60100

Node2:
sudo ceph daemon osd.0 help | head
{
    "config diff": "dump diff of current config and default config",
    "config get": "config get <field>: get the config value",

This is more for development/test setup.

Regards,
Igor.

> It requires a locally-installed ceph CLI and a ceph.conf that points to the
> cluster and any required keyrings.  You can also provide it with a YAML file
> mapping host to osds if you want to save time collecting that info for a
> statically-defined cluster, or if you want just a subset of OSDs.
> 
> https://github.com/dmick/tools/blob/master/osd_daemon_cmd.py
> 
> Excerpt from usage:
> 
> Execute a Ceph osd daemon command on every OSD in a cluster with one
> connection to each OSD host.
> 
> Usage:
>     osd_daemon_cmd [-c CONF] [-u USER] [-f FILE] (COMMAND | -k KEY)
> 
> Options:
>    -c CONF   ceph.conf file to use [default: ./ceph.conf]
>    -u USER   user to connect with ssh
>    -f FILE   get names and osds from yaml
>    COMMAND   command other than "config get" to execute
>    -k KEY    config key to retrieve with config get <key>
> 
> --
> Dan Mick
> Red Hat, Inc.
> Ceph docs: http://ceph.com/docs
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the
> body of a message to majordomo@vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html

Re: tool for applying 'ceph daemon <osd>' command to all OSDs

[Dan Mick]

On 12/22/2015 12:21 AM, Igor.Podoski@ts.fujitsu.com wrote:
>> -----Original Message-----
>> From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel-
>> owner@vger.kernel.org] On Behalf Of Dan Mick
>> Sent: Tuesday, December 22, 2015 7:00 AM
>> To: ceph-devel
>> Subject: RFC: tool for applying 'ceph daemon <osd>' command to all OSDs
>>
>> I needed something to fetch current config values from all OSDs (sorta the
>> opposite of 'injectargs --key value), so I hacked it, and then spiffed it up a bit.
>> Does this seem like something that would be useful in this form in the
>> upstream Ceph, or does anyone have any thoughts on its design or
>> structure?
>>
> 
> You could do it using socat too:

Quite true.  That opens the permissions even more, but it works.
Maintaining a couple hundred x2 processes might be a little bit of a pain..