problems with ceph.init / etc/hosts and ceph.conf

problems with ceph.init / etc/hosts and ceph.conf

[Stefan Priebe - Profihost AG]

Hello list,

i've some problems to find the correct combination of ceph.conf, 
/etc/hosts for the init script to work correctly.

My systems are named:
ssdstor000i
ssdstor001i
ssdstor002i

These names point to the public external IP.

These systems also have an alias (ssdstor000,ssdstor001) pointing to the 
internal IP which should be used by ceph.

My ceph.conf looks like this:

...
[osd.10]
         host= ssdstor000
[osd.11]
         host= ssdstor000
...

The ceph.init Script looks now for an entry called ssdstor000i but can't 
find one and so it does not start or stop any services.

My /etc/hosts file:
192.168.0.100    ssdstor000.XYZ.de       ssdstor000i

10.0.255.100    ssdstor000

What is the correct way to archieve that the default hostname is still 
the public one but ceph uses the internal one?

Greets
Stefan

Re: problems with ceph.init / etc/hosts and ceph.conf

[Wido den Hollander]

On 06/20/2012 10:59 AM, Stefan Priebe - Profihost AG wrote:
> Hello list,
>
> i've some problems to find the correct combination of ceph.conf,
> /etc/hosts for the init script to work correctly.
>
> My systems are named:
> ssdstor000i
> ssdstor001i
> ssdstor002i
>

What does hostname | cut -d . -f 1 say?

This is looked up by the init script, see: /usr/lib/ceph/ceph_common.sh

> These names point to the public external IP.
>
> These systems also have an alias (ssdstor000,ssdstor001) pointing to the
> internal IP which should be used by ceph.
>
> My ceph.conf looks like this:
>
> ...
> [osd.10]
> host= ssdstor000
> [osd.11]
> host= ssdstor000
> ...
>
> The ceph.init Script looks now for an entry called ssdstor000i but can't
> find one and so it does not start or stop any services.
>
> My /etc/hosts file:
> 192.168.0.100 ssdstor000.XYZ.de ssdstor000i
>
> 10.0.255.100 ssdstor000
>
> What is the correct way to archieve that the default hostname is still
> the public one but ceph uses the internal one?

You can specify --hostname with the init script, but I recommend using 
the hostnames which is set in /etc/hostname in your ceph.conf

Wido

>
> Greets
> Stefan
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: problems with ceph.init / etc/hosts and ceph.conf

[Stefan Priebe - Profihost AG]

Am 20.06.2012 11:13, schrieb Wido den Hollander:
> On 06/20/2012 10:59 AM, Stefan Priebe - Profihost AG wrote:
>> Hello list,
>>
>> i've some problems to find the correct combination of ceph.conf,
>> /etc/hosts for the init script to work correctly.
>>
>> My systems are named:
>> ssdstor000i
>> ssdstor001i
>> ssdstor002i
>>
> What does hostname | cut -d . -f 1 say?
It says ssdstor000i as the machine defaults to it's public IP.

Whouldn't it make sense to go through all aliases set in /etc/hosts 
which point to a local IP?

Stefan

Re: problems with ceph.init / etc/hosts and ceph.conf

[Stefan Priebe - Profihost AG]

Am 20.06.2012 11:20, schrieb Stefan Priebe - Profihost AG:
> Am 20.06.2012 11:13, schrieb Wido den Hollander:
>> On 06/20/2012 10:59 AM, Stefan Priebe - Profihost AG wrote:
>>> Hello list,
>>>
>>> i've some problems to find the correct combination of ceph.conf,
>>> /etc/hosts for the init script to work correctly.
>>>
>>> My systems are named:
>>> ssdstor000i
>>> ssdstor001i
>>> ssdstor002i
>>>
>> What does hostname | cut -d . -f 1 say?
> It says ssdstor000i as the machine defaults to it's public IP.
>
> Whouldn't it make sense to go through all aliases set in /etc/hosts
> which point to a local IP?

I found mon addr to specify a different address than the hostname is set 
but i haven't found osd addr does this exist too?

So i'm able to set the hostname to the public IP but let ceph still use 
the internal ones.

Stefan

Re: problems with ceph.init / etc/hosts and ceph.conf

[Wido den Hollander]

On 06/20/2012 11:20 AM, Stefan Priebe - Profihost AG wrote:
> Am 20.06.2012 11:13, schrieb Wido den Hollander:
>> On 06/20/2012 10:59 AM, Stefan Priebe - Profihost AG wrote:
>>> Hello list,
>>>
>>> i've some problems to find the correct combination of ceph.conf,
>>> /etc/hosts for the init script to work correctly.
>>>
>>> My systems are named:
>>> ssdstor000i
>>> ssdstor001i
>>> ssdstor002i
>>>
>> What does hostname | cut -d . -f 1 say?
> It says ssdstor000i as the machine defaults to it's public IP.
>
> Whouldn't it make sense to go through all aliases set in /etc/hosts
> which point to a local IP?

Ceph is not really intended to work with public or private IP's.

The idea is that you have a Layer 2 or Layer 3 network over which all 
the daemons and clients connect over a routable network, peer-to-peer, 
no NAT in it's way.

You can however have a seperate "osd" network where the OSD's 
interconnect for their replication traffic, but that has nothing to do 
with the hostnames.

You can take a look at the configuration options "public_addr" and 
"cluster_addr" for these.

Besides that, Ceph would run on a "private" network. If you have KVM 
hosts these will have a WAN connection over which they (and their VM's) 
communicate with the outside world and over a dedicated NIC the host 
communicates with the Ceph cluster.

I'd recommend sticking with 1 IP-Address for each OSD, keep it simple.

Wido

>
> Stefan
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: problems with ceph.init / etc/hosts and ceph.conf

[Tommi Virtanen]

On Wed, Jun 20, 2012 at 2:20 AM, Stefan Priebe - Profihost AG
<s.priebe@profihost.ag> wrote:
>> What does hostname | cut -d . -f 1 say?
>
> It says ssdstor000i as the machine defaults to it's public IP.
>
> Whouldn't it make sense to go through all aliases set in /etc/hosts which
> point to a local IP?

Actually, the short hostname of the machine is completely unrelated to
what IP addresses etc it has. The hostname is a configurable string.
We treat it as such -- it's just expected to be a unique identifier
for the host.

There's no nice, reliable way to look up aliases that point to a local
IP. /etc/hosts is meant to be accessed through a library that also
uses DNS, and you really don't want to tie your daemon startup to DNS
availability and functioning.

Re: problems with ceph.init / etc/hosts and ceph.conf

[Stefan Priebe]

Am 20.06.2012 um 18:39 schrieb Tommi Virtanen <tv@inktank.com>:

> On Wed, Jun 20, 2012 at 2:20 AM, Stefan Priebe - Profihost AG
> <s.priebe@profihost.ag> wrote:
>>> What does hostname | cut -d . -f 1 say?
>> 
>> It says ssdstor000i as the machine defaults to it's public IP.
>> 
>> Whouldn't it make sense to go through all aliases set in /etc/hosts which
>> point to a local IP?
> 
> Actually, the short hostname of the machine is completely unrelated to
> what IP addresses etc it has. The hostname is a configurable string.
> We treat it as such -- it's just expected to be a unique identifier
> for the host.
> 
> There's no nice, reliable way to look up aliases that point to a local
> IP. /etc/hosts is meant to be accessed through a library that also
> uses DNS, and you really don't want to tie your daemon startup to DNS
> availability and functioning.

Correct. So i should just change the /etc/hostname to the name that points to the internal ip via hosts file?

Re: problems with ceph.init / etc/hosts and ceph.conf

[Tommi Virtanen]

On Wed, Jun 20, 2012 at 9:46 AM, Stefan Priebe <s.priebe@profihost.ag> wrote:
>> Actually, the short hostname of the machine is completely unrelated to
>> what IP addresses etc it has. The hostname is a configurable string.
>> We treat it as such -- it's just expected to be a unique identifier
>> for the host.
> Correct. So i should just change the /etc/hostname to the name that points to the internal ip via hosts file?

/etc/hostname is read into the kernel at bootup, using the
/bin/hostname command. So after editing, either reboot or use sudo
hostname --file /etc/hostname. Other than that, yes.