OpenStack and ceph integration with puppet

OpenStack and ceph integration with puppet

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1823 bytes --]

Hi Ceph,

Binding ceph to cinder and glance using puppet requires three steps:

* Deploy ceph ( with ceph-deploy, puppet, chef ... )

* Follow the ceph documentation instructions ( valid for both cinder and glance )

  http://ceph.com/docs/next/rbd/rbd-openstack/

* Part of the above instructions can be skipped if the following are used

  https://github.com/stackforge/puppet-glance/blob/stable/grizzly/manifests/backend/rbd.pp
  https://github.com/stackforge/puppet-cinder/blob/stable/grizzly/manifests/volume/rbd.pp

  They take care of installing a package on the glance and cinder node and writing the cinder and glance config files.

* Upgrading the librbd on the compute hosts to match the version of the cluster ( ubuntu precise has bobtail but you may want at least cuttlefish )

I would be delighted to know if there is a simpler way. If not, would it make sense to provide the puppet master with the IP of the monitors and admin rights so that it can automate http://ceph.com/docs/next/rbd/rbd-openstack/ ?

  * install ceph-common on cinder hosts and python-ceph on glance hosts
  * set the monitor addresses
  * copy the keyring to cinder / glance
  * create the client.volumes / client.images users ( support <= 0.53 ? )
  * upgrade the librbd package on the compute hosts to the version matching the cluster
  * virsh secret-set-value the volume key on each compute host
  * reload glance/nova/cinder where appropriate

The puppet master could even refresh the list of monitors from time to time and update the cinder/glance nodes accordingly. And it could do the right thing depending on the target openstack version and ceph version.

Thoughts ?

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: OpenStack and ceph integration with puppet

[Sébastien Han]

Hi Loïc,

Actually they are some steps that might be automated such as:

* set the virsh secret
* create both glance and cinder pools

Please take a look at:

* https://github.com/dontalton/puppet-cephdeploy/blob/master/manifests/init.pp#L121
* https://github.com/dontalton/puppet-cephdeploy/blob/master/manifests/osd.pp#L73

For the rest this might be already done but your puppet manifests.
Please also note that http://ceph.com/docs/next/rbd/rbd-openstack/ will need some updates for OpenStack Havana.

––––
Sébastien Han
Cloud Engineer

"Always give 100%. Unless you're giving blood.”

Phone: +33 (0)1 49 70 99 72
Mail: sebastien.han@enovance.com
Address : 10, rue de la Victoire - 75009 Paris
Web : www.enovance.com - Twitter : @enovance

On October 8, 2013 at 4:18:00 PM, Loic Dachary (loic@dachary.org) wrote:

Hi Ceph,  

Binding ceph to cinder and glance using puppet requires three steps:  

* Deploy ceph ( with ceph-deploy, puppet, chef ... )  

* Follow the ceph documentation instructions ( valid for both cinder and glance )  

http://ceph.com/docs/next/rbd/rbd-openstack/  

* Part of the above instructions can be skipped if the following are used  

https://github.com/stackforge/puppet-glance/blob/stable/grizzly/manifests/backend/rbd.pp  
https://github.com/stackforge/puppet-cinder/blob/stable/grizzly/manifests/volume/rbd.pp  

They take care of installing a package on the glance and cinder node and writing the cinder and glance config files.  

* Upgrading the librbd on the compute hosts to match the version of the cluster ( ubuntu precise has bobtail but you may want at least cuttlefish )  

I would be delighted to know if there is a simpler way. If not, would it make sense to provide the puppet master with the IP of the monitors and admin rights so that it can automate http://ceph.com/docs/next/rbd/rbd-openstack/ ?  

* install ceph-common on cinder hosts and python-ceph on glance hosts  
* set the monitor addresses  
* copy the keyring to cinder / glance  
* create the client.volumes / client.images users ( support <= 0.53 ? )  
* upgrade the librbd package on the compute hosts to the version matching the cluster  
* virsh secret-set-value the volume key on each compute host  
* reload glance/nova/cinder where appropriate  

The puppet master could even refresh the list of monitors from time to time and update the cinder/glance nodes accordingly. And it could do the right thing depending on the target openstack version and ceph version.  

Thoughts ?  

--  
Loïc Dachary, Artisan Logiciel Libre  
All that is necessary for the triumph of evil is that good people do nothing.
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: OpenStack and ceph integration with puppet

[Don Talton (dotalton)]

Hi Loic, 

As Sebastien mentioned already there is code that addresses this in my github repo. This was borne out of some needs we have at Cisco, the primary being complete automation of a rbd-backed cloud. We are still using eNovance's puppet-ceph (https://github.com/enovance/puppet-ceph) in our Grizzly orchestration, but will be moving to the cephdeploy module for Havana.

We utilize stackforge's puppet modules to do our heavy lifting, including p-openstack, p-cinder, p-glance. There are dependency chains so that services will be restarted after configuration changes are made. Since many of our customers don't allow their baremetal  nodes Internet access, we've added the packages to our APT repo to avoid the version issues with using either stock or public packages.

You can probably find some other useful code the https://github.com/CiscoSystems/ repo, including what is needed to cohabitate MON/OSD nodes with OpenStack service nodes (https://github.com/CiscoSystems/puppet-coe/tree/grizzly/manifests/ceph) and more. The primary orchestration is in grizzly-manifests. You can see HOWTOs for different deployment scenarios here: http://docwiki.cisco.com/wiki/OpenStack:Ceph-COI-Installation. 

Hope this helps some!

Don

Re: OpenStack and ceph integration with puppet

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 3152 bytes --]



On 08/10/2013 15:47, Sébastien Han wrote:
> Hi Loïc,
> 
> Actually they are some steps that might be automated such as:
> 
> * set the virsh secret
> * create both glance and cinder pools

Where do you think it might already be automated ?

> Please take a look at:
> 
> * https://github.com/dontalton/puppet-cephdeploy/blob/master/manifests/init.pp#L121
> * https://github.com/dontalton/puppet-cephdeploy/blob/master/manifests/osd.pp#L73

Right ! I overlooked this puppet module.

> For the rest this might be already done but your puppet manifests.

I plan to not write any manifest :-)

> Please also note that http://ceph.com/docs/next/rbd/rbd-openstack/ will need some updates for OpenStack Havana.

Cheers

> ––––
> Sébastien Han
> Cloud Engineer
> 
> "Always give 100%. Unless you're giving blood.”
> 
> Phone: +33 (0)1 49 70 99 72
> Mail: sebastien.han@enovance.com
> Address : 10, rue de la Victoire - 75009 Paris
> Web : www.enovance.com - Twitter : @enovance
> 
> On October 8, 2013 at 4:18:00 PM, Loic Dachary (loic@dachary.org) wrote:
> 
> Hi Ceph,  
> 
> Binding ceph to cinder and glance using puppet requires three steps:  
> 
> * Deploy ceph ( with ceph-deploy, puppet, chef ... )  
> 
> * Follow the ceph documentation instructions ( valid for both cinder and glance )  
> 
> http://ceph.com/docs/next/rbd/rbd-openstack/  
> 
> * Part of the above instructions can be skipped if the following are used  
> 
> https://github.com/stackforge/puppet-glance/blob/stable/grizzly/manifests/backend/rbd.pp  
> https://github.com/stackforge/puppet-cinder/blob/stable/grizzly/manifests/volume/rbd.pp  
> 
> They take care of installing a package on the glance and cinder node and writing the cinder and glance config files.  
> 
> * Upgrading the librbd on the compute hosts to match the version of the cluster ( ubuntu precise has bobtail but you may want at least cuttlefish )  
> 
> I would be delighted to know if there is a simpler way. If not, would it make sense to provide the puppet master with the IP of the monitors and admin rights so that it can automate http://ceph.com/docs/next/rbd/rbd-openstack/ ?  
> 
> * install ceph-common on cinder hosts and python-ceph on glance hosts  
> * set the monitor addresses  
> * copy the keyring to cinder / glance  
> * create the client.volumes / client.images users ( support <= 0.53 ? )  
> * upgrade the librbd package on the compute hosts to the version matching the cluster  
> * virsh secret-set-value the volume key on each compute host  
> * reload glance/nova/cinder where appropriate  
> 
> The puppet master could even refresh the list of monitors from time to time and update the cinder/glance nodes accordingly. And it could do the right thing depending on the target openstack version and ceph version.  
> 
> Thoughts ?  
> 
> --  
> Loïc Dachary, Artisan Logiciel Libre  
> All that is necessary for the triumph of evil is that good people do nothing.
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: OpenStack and ceph integration with puppet

[Dmitry Borodaenko]

Loic,

You might also find the Fuel manifests for Ceph useful:
https://github.com/Mirantis/fuel/tree/master/deployment/puppet/ceph
https://github.com/Mirantis/fuel/blob/master/deployment/puppet/cinder/manifests/volume/ceph.pp
https://github.com/Mirantis/fuel/blob/master/deployment/puppet/glance/manifests/backend/ceph.pp

We've been working on these for the last couple of months with the
goal of fully automating deployment of OpenStack environments with
Ceph as storage backend, and we're close to an official release of
Fuel that will support this configuration. It automates a lot of stuff
all the way from drive partitioning for OSDs up to generating and
configuring keys for Cinder and Glance pools. You'd need the rest of
Fuel to make the most out of it, but you can also provide the missing
bits by tweaking the provided site.pp.

Regards,
Dmitry Borodaenko

On Tue, Oct 8, 2013 at 11:03 AM, Loic Dachary <loic@dachary.org> wrote:
>
>
> On 08/10/2013 15:47, Sébastien Han wrote:
>> Hi Loïc,
>>
>> Actually they are some steps that might be automated such as:
>>
>> * set the virsh secret
>> * create both glance and cinder pools
>
> Where do you think it might already be automated ?
>
>> Please take a look at:
>>
>> * https://github.com/dontalton/puppet-cephdeploy/blob/master/manifests/init.pp#L121
>> * https://github.com/dontalton/puppet-cephdeploy/blob/master/manifests/osd.pp#L73
>
> Right ! I overlooked this puppet module.
>
>> For the rest this might be already done but your puppet manifests.
>
> I plan to not write any manifest :-)
>
>> Please also note that http://ceph.com/docs/next/rbd/rbd-openstack/ will need some updates for OpenStack Havana.
>
> Cheers
>
>> ––––
>> Sébastien Han
>> Cloud Engineer
>>
>> "Always give 100%. Unless you're giving blood.”
>>
>> Phone: +33 (0)1 49 70 99 72
>> Mail: sebastien.han@enovance.com
>> Address : 10, rue de la Victoire - 75009 Paris
>> Web : www.enovance.com - Twitter : @enovance
>>
>> On October 8, 2013 at 4:18:00 PM, Loic Dachary (loic@dachary.org) wrote:
>>
>> Hi Ceph,
>>
>> Binding ceph to cinder and glance using puppet requires three steps:
>>
>> * Deploy ceph ( with ceph-deploy, puppet, chef ... )
>>
>> * Follow the ceph documentation instructions ( valid for both cinder and glance )
>>
>> http://ceph.com/docs/next/rbd/rbd-openstack/
>>
>> * Part of the above instructions can be skipped if the following are used
>>
>> https://github.com/stackforge/puppet-glance/blob/stable/grizzly/manifests/backend/rbd.pp
>> https://github.com/stackforge/puppet-cinder/blob/stable/grizzly/manifests/volume/rbd.pp
>>
>> They take care of installing a package on the glance and cinder node and writing the cinder and glance config files.
>>
>> * Upgrading the librbd on the compute hosts to match the version of the cluster ( ubuntu precise has bobtail but you may want at least cuttlefish )
>>
>> I would be delighted to know if there is a simpler way. If not, would it make sense to provide the puppet master with the IP of the monitors and admin rights so that it can automate http://ceph.com/docs/next/rbd/rbd-openstack/ ?
>>
>> * install ceph-common on cinder hosts and python-ceph on glance hosts
>> * set the monitor addresses
>> * copy the keyring to cinder / glance
>> * create the client.volumes / client.images users ( support <= 0.53 ? )
>> * upgrade the librbd package on the compute hosts to the version matching the cluster
>> * virsh secret-set-value the volume key on each compute host
>> * reload glance/nova/cinder where appropriate
>>
>> The puppet master could even refresh the list of monitors from time to time and update the cinder/glance nodes accordingly. And it could do the right thing depending on the target openstack version and ceph version.
>>
>> Thoughts ?
>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
>> All that is necessary for the triumph of evil is that good people do nothing.
>>
>
> --
> Loïc Dachary, Artisan Logiciel Libre
> All that is necessary for the triumph of evil is that good people do nothing.
>



-- 
Dmitry Borodaenko
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: OpenStack and ceph integration with puppet

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 4715 bytes --]

Hi Dmitry,

On 09/10/2013 00:26, Dmitry Borodaenko wrote:
> Loic,
> 
> You might also find the Fuel manifests for Ceph useful:
> https://github.com/Mirantis/fuel/tree/master/deployment/puppet/ceph
> https://github.com/Mirantis/fuel/blob/master/deployment/puppet/cinder/manifests/volume/ceph.pp

The

"ceph auth get-or-create ..."

looks like a worthy contribution to

https://github.com/stackforge/puppet-cinder/blob/master/manifests/volume/rbd.pp

Is there a reason why it would not be relevant in this context ?

> https://github.com/Mirantis/fuel/blob/master/deployment/puppet/glance/manifests/backend/ceph.pp
> 
> We've been working on these for the last couple of months with the
> goal of fully automating deployment of OpenStack environments with
> Ceph as storage backend, and we're close to an official release of
> Fuel that will support this configuration. It automates a lot of stuff
> all the way from drive partitioning for OSDs up to generating and
> configuring keys for Cinder and Glance pools. You'd need the rest of
> Fuel to make the most out of it, but you can also provide the missing
> bits by tweaking the provided site.pp.

Thanks for the pointers !

Cheers

> 
> Regards,
> Dmitry Borodaenko
> 
> On Tue, Oct 8, 2013 at 11:03 AM, Loic Dachary <loic@dachary.org> wrote:
>>
>>
>> On 08/10/2013 15:47, Sébastien Han wrote:
>>> Hi Loïc,
>>>
>>> Actually they are some steps that might be automated such as:
>>>
>>> * set the virsh secret
>>> * create both glance and cinder pools
>>
>> Where do you think it might already be automated ?
>>
>>> Please take a look at:
>>>
>>> * https://github.com/dontalton/puppet-cephdeploy/blob/master/manifests/init.pp#L121
>>> * https://github.com/dontalton/puppet-cephdeploy/blob/master/manifests/osd.pp#L73
>>
>> Right ! I overlooked this puppet module.
>>
>>> For the rest this might be already done but your puppet manifests.
>>
>> I plan to not write any manifest :-)
>>
>>> Please also note that http://ceph.com/docs/next/rbd/rbd-openstack/ will need some updates for OpenStack Havana.
>>
>> Cheers
>>
>>> ––––
>>> Sébastien Han
>>> Cloud Engineer
>>>
>>> "Always give 100%. Unless you're giving blood.”
>>>
>>> Phone: +33 (0)1 49 70 99 72
>>> Mail: sebastien.han@enovance.com
>>> Address : 10, rue de la Victoire - 75009 Paris
>>> Web : www.enovance.com - Twitter : @enovance
>>>
>>> On October 8, 2013 at 4:18:00 PM, Loic Dachary (loic@dachary.org) wrote:
>>>
>>> Hi Ceph,
>>>
>>> Binding ceph to cinder and glance using puppet requires three steps:
>>>
>>> * Deploy ceph ( with ceph-deploy, puppet, chef ... )
>>>
>>> * Follow the ceph documentation instructions ( valid for both cinder and glance )
>>>
>>> http://ceph.com/docs/next/rbd/rbd-openstack/
>>>
>>> * Part of the above instructions can be skipped if the following are used
>>>
>>> https://github.com/stackforge/puppet-glance/blob/stable/grizzly/manifests/backend/rbd.pp
>>> https://github.com/stackforge/puppet-cinder/blob/stable/grizzly/manifests/volume/rbd.pp
>>>
>>> They take care of installing a package on the glance and cinder node and writing the cinder and glance config files.
>>>
>>> * Upgrading the librbd on the compute hosts to match the version of the cluster ( ubuntu precise has bobtail but you may want at least cuttlefish )
>>>
>>> I would be delighted to know if there is a simpler way. If not, would it make sense to provide the puppet master with the IP of the monitors and admin rights so that it can automate http://ceph.com/docs/next/rbd/rbd-openstack/ ?
>>>
>>> * install ceph-common on cinder hosts and python-ceph on glance hosts
>>> * set the monitor addresses
>>> * copy the keyring to cinder / glance
>>> * create the client.volumes / client.images users ( support <= 0.53 ? )
>>> * upgrade the librbd package on the compute hosts to the version matching the cluster
>>> * virsh secret-set-value the volume key on each compute host
>>> * reload glance/nova/cinder where appropriate
>>>
>>> The puppet master could even refresh the list of monitors from time to time and update the cinder/glance nodes accordingly. And it could do the right thing depending on the target openstack version and ceph version.
>>>
>>> Thoughts ?
>>>
>>> --
>>> Loïc Dachary, Artisan Logiciel Libre
>>> All that is necessary for the triumph of evil is that good people do nothing.
>>>
>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
>> All that is necessary for the triumph of evil is that good people do nothing.
>>
> 
> 
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: OpenStack and ceph integration with puppet

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1656 bytes --]



On 08/10/2013 16:20, Don Talton (dotalton) wrote:> Hi Loic, 
> 

> We utilize stackforge's puppet modules to do our heavy lifting, including p-openstack, p-cinder, p-glance. There are dependency chains so that services will be restarted after configuration changes are made. Since many of our customers don't allow their baremetal  nodes Internet access, we've added the packages to our APT repo to avoid the version issues with using either stock or public packages.
> 
> You can probably find some other useful code the https://github.com/CiscoSystems/ repo, including what is needed to cohabitate MON/OSD nodes with OpenStack service nodes (https://github.com/CiscoSystems/puppet-coe/tree/grizzly/manifests/ceph) and more. The primary orchestration is in grizzly-manifests. You can see HOWTOs for different deployment scenarios here: http://docwiki.cisco.com/wiki/OpenStack:Ceph-COI-Installation. 
> 
> Hope this helps some!

It does and it's great that all this is documented :-) Although there are a few modules around, re-using ceph-deploy seems to be the preferred method. I wonder what Alfredo would suggest. From a previous discussion we had I think he will suggest to use ceph-disk directly + cli / rest call instead. Looking at 

https://github.com/ceph/ceph-deploy/blob/master/ceph_deploy/new.py
https://github.com/ceph/ceph-deploy/blob/master/ceph_deploy/mon.py
etc.

the layer provided by ceph-deploy is indeed thin. But is it something that needs to be duplicated in a puppet module ?

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: OpenStack and ceph integration with puppet

[Dmitry Borodaenko]

On Wed, Oct 9, 2013 at 2:30 PM, Loic Dachary <loic@dachary.org> wrote:
>> https://github.com/Mirantis/fuel/blob/master/deployment/puppet/cinder/manifests/volume/ceph.pp
>
> The
>
> "ceph auth get-or-create ..."
>
> looks like a worthy contribution to
>
> https://github.com/stackforge/puppet-cinder/blob/master/manifests/volume/rbd.pp
>
> Is there a reason why it would not be relevant in this context ?

No, it's definitely relevant, take it if you like it :)

-- 
Dmitry Borodaenko

Re: OpenStack and ceph integration with puppet

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2947 bytes --]



On 09/10/2013 22:46, Loic Dachary wrote:
> 
> 
> On 08/10/2013 16:20, Don Talton (dotalton) wrote:> Hi Loic, 
>>
> 
>> We utilize stackforge's puppet modules to do our heavy lifting, including p-openstack, p-cinder, p-glance. There are dependency chains so that services will be restarted after configuration changes are made. Since many of our customers don't allow their baremetal  nodes Internet access, we've added the packages to our APT repo to avoid the version issues with using either stock or public packages.
>>
>> You can probably find some other useful code the https://github.com/CiscoSystems/ repo, including what is needed to cohabitate MON/OSD nodes with OpenStack service nodes (https://github.com/CiscoSystems/puppet-coe/tree/grizzly/manifests/ceph) and more. The primary orchestration is in grizzly-manifests. You can see HOWTOs for different deployment scenarios here: http://docwiki.cisco.com/wiki/OpenStack:Ceph-COI-Installation. 
>>
>> Hope this helps some!
> 
> It does and it's great that all this is documented :-) Although there are a few modules around, re-using ceph-deploy seems to be the preferred method. I wonder what Alfredo would suggest. From a previous discussion we had I think he will suggest to use ceph-disk directly + cli / rest call instead. Looking at 
> 
> https://github.com/ceph/ceph-deploy/blob/master/ceph_deploy/new.py
> https://github.com/ceph/ceph-deploy/blob/master/ceph_deploy/mon.py
> etc.
> 
> the layer provided by ceph-deploy is indeed thin. But is it something that needs to be duplicated in a puppet module ?
> 

I took a look at ceph-deploy and it won't rely on sudo if run from root

ceph_deploy/sudo_pushy.py
def needs_sudo():
    if getpass.getuser() == 'root':
        return False
    return True

and that it won't rely on ssh if the target host is localhost:

ceph_deploy/lib/remoto/connection.py
def needs_ssh(hostname, _socket=None):
    """
    Obtains remote hostname of the socket and cuts off the domain part
    of its FQDN.
    """
    _socket = _socket or socket
    local_hostname = _socket.gethostname()
    local_short_hostname = local_hostname.split('.')[0]
    if local_hostname == hostname or local_short_hostname == hostname:
        return False
    return True

Since puppet-cephdeploy runs on the target host as root, it means that

puppet-cephdeploy/manifests/init.pp
  file {"/home/$user/.ssh/authorized_keys":
...
etc.

could probably be avoided since puppet-cephdeploy/manifests/mon.pp runs

command => "/usr/local/bin/ceph-deploy mon create $::hostname",

runs as root, on the target host.

I'm not sure if the distribution of the keys would work though as it relies on files collected by "gatherkeys" which is still a little mysterious for me :-)

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.






[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: OpenStack and ceph integration with puppet

[Alfredo Deza]

On Thu, Oct 10, 2013 at 11:43 AM, Loic Dachary <loic@dachary.org> wrote:
>
>
> On 09/10/2013 22:46, Loic Dachary wrote:
>>
>>
>> On 08/10/2013 16:20, Don Talton (dotalton) wrote:> Hi Loic,
>>>
>>
>>> We utilize stackforge's puppet modules to do our heavy lifting, including p-openstack, p-cinder, p-glance. There are dependency chains so that services will be restarted after configuration changes are made. Since many of our customers don't allow their baremetal  nodes Internet access, we've added the packages to our APT repo to avoid the version issues with using either stock or public packages.
>>>
>>> You can probably find some other useful code the https://github.com/CiscoSystems/ repo, including what is needed to cohabitate MON/OSD nodes with OpenStack service nodes (https://github.com/CiscoSystems/puppet-coe/tree/grizzly/manifests/ceph) and more. The primary orchestration is in grizzly-manifests. You can see HOWTOs for different deployment scenarios here: http://docwiki.cisco.com/wiki/OpenStack:Ceph-COI-Installation.
>>>
>>> Hope this helps some!
>>
>> It does and it's great that all this is documented :-) Although there are a few modules around, re-using ceph-deploy seems to be the preferred method. I wonder what Alfredo would suggest. From a previous discussion we had I think he will suggest to use ceph-disk directly + cli / rest call instead. Looking at
>>
>> https://github.com/ceph/ceph-deploy/blob/master/ceph_deploy/new.py
>> https://github.com/ceph/ceph-deploy/blob/master/ceph_deploy/mon.py
>> etc.
>>
>> the layer provided by ceph-deploy is indeed thin. But is it something that needs to be duplicated in a puppet module ?
>>
>
> I took a look at ceph-deploy and it won't rely on sudo if run from root
>
> ceph_deploy/sudo_pushy.py
> def needs_sudo():
>     if getpass.getuser() == 'root':
>         return False
>     return True
>
> and that it won't rely on ssh if the target host is localhost:
>
> ceph_deploy/lib/remoto/connection.py
> def needs_ssh(hostname, _socket=None):
>     """
>     Obtains remote hostname of the socket and cuts off the domain part
>     of its FQDN.
>     """
>     _socket = _socket or socket
>     local_hostname = _socket.gethostname()
>     local_short_hostname = local_hostname.split('.')[0]
>     if local_hostname == hostname or local_short_hostname == hostname:
>         return False
>     return True
>
> Since puppet-cephdeploy runs on the target host as root, it means that
>
> puppet-cephdeploy/manifests/init.pp
>   file {"/home/$user/.ssh/authorized_keys":
> ...
> etc.
>
> could probably be avoided since puppet-cephdeploy/manifests/mon.pp runs
>
> command => "/usr/local/bin/ceph-deploy mon create $::hostname",
>
> runs as root, on the target host.

Loic is spot on here. Yes, ceph-deploy will avoid all of those things
described if they are not necessary. The one possible caveat is when
there is
an ~/.ssh/config that changes the login of a remote user to something
else which ceph-deploy would not be able to tell.

So say you have a `node1` defined in the ssh config with user `foo`
but you are executing ceph-deploy as `root`, then that would mean
that ceph-deploy would not run `sudo` commands in the remote host
because it assumes the ssh is happening with root.

If the manifest is doing all of this work locally, then this is not a
problem, but something to be aware of.
>
> I'm not sure if the distribution of the keys would work though as it relies on files collected by "gatherkeys" which is still a little mysterious for me :-)

gatherkeys will just go to standard locations and look for the
generated keys and copy them back to where ceph-deploy is executing
from. Really nothing
complex is  happening there.


>
> Cheers
>
> --
> Loïc Dachary, Artisan Logiciel Libre
> All that is necessary for the triumph of evil is that good people do nothing.
>
>
>
>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html