Redundancy with Ceph

Redundancy with Ceph

[Christian Baun]

Hi,

Is it possible to tell Ceph that the user data is stored in a redundant way across the osds?

Best Regards,
   Christian 

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Redundancy with Ceph

[Gregory Farnum]

On Sun, Jul 4, 2010 at 1:34 PM, Christian Baun <cray@unix-ag.uni-kl.de> wrote:
> Hi,
>
> Is it possible to tell Ceph that the user data is stored in a redundant way across the osds?
Could you clarify? What do you mean by "across the OSDs"?
Obviously Ceph is aware of how many copies it makes.

Re: Redundancy with Ceph

[Christian Baun]

Hi,

I created 2 servers and one client

Server 1 => mon, mds, osd
Server 2 => osd

I tried some tests with iozone and I don't think Server 2 is used.

Server 1:
# df | grep osd
/dev/sdc              10485760    600504   9885256   6% /data/osd1

Server 2:
# df | grep osd
/dev/sdc              10485760        32  10485728   1% /data/osd2

Client:
# df | grep osd
10.243.150.209:/      10485760    896000   9589760   9% /mnt/ceph

Theses stats don't look like there is any redundancy with the user data.

Server 1:
# ls /var/log/ceph/
ip-10-243-150-209.18773  ip-10-243-150-209.20598  mds.0   osd1.3
ip-10-243-150-209.18993  ip-10-243-150-209.20822  mon0    osd1.4
ip-10-243-150-209.19277  ip-10-243-150-209.20912  osd1    osd1.5
ip-10-243-150-209.19501  ip-10-243-150-209.20940  osd1.0  osd1.6
ip-10-243-150-209.19727  ip-10-243-150-209.20990  osd1.1  osd1.7
ip-10-243-150-209.19963  mds0                     osd1.2  stat

Server 2:
# ls /var/log/ceph/
ip-10-212-118-67.19342  osd2

How can I check if Server 2 is really recognized by Server 1? 

Best Regards,
   Christian 



Am Sonntag, 4. Juli 2010 schrieben Sie:
> On Sun, Jul 4, 2010 at 1:34 PM, Christian Baun <cray@unix-ag.uni-kl.de> wrote:
> > Hi,
> >
> > Is it possible to tell Ceph that the user data is stored in a redundant way across the osds?
> Could you clarify? What do you mean by "across the OSDs"?
> Obviously Ceph is aware of how many copies it makes.
> 

Re: Redundancy with Ceph

[Thomas Mueller]

Am Mon, 05 Jul 2010 00:33:50 +0200 schrieb Christian Baun:

> Hi,
> 
> I created 2 servers and one client
> 
> Server 1 => mon, mds, osd
> Server 2 => osd
> 
> I tried some tests with iozone and I don't think Server 2 is used.


did you read:

http://ceph.newdream.net/wiki/Adjusting_replication_level



- Thomas

Re: Redundancy with Ceph

[Christian Baun]

Hi Thomas,

Thanks a lot for the link and your help!
Now, the issue are clear.

The output of "ceph osd dump -o -" said that there shall be replication because max_osd is 3 and the size of all pools is 2. But osd2 was "out down".

# ceph osd dump -o -
...
max_osd 3
osd0 out down (up_from 0 up_thru 0 down_at 0 last_clean 0-0)
osd1 in weight 1 up   (up_from 2 up_thru 2 down_at 0 last_clean 0-0) 10.243.150.209:6801/20989 10.243.150.209:6802/20989
osd2 out down (up_from 0 up_thru 0 down_at 0 last_clean 0-0)

The reason: I forgot to start the ceph server at osd2.
Now, it looks better.

# ceph osd dump -o -
...
max_osd 3
osd0 out down (up_from 0 up_thru 0 down_at 0 last_clean 0-0)
osd1 in weight 1 up   (up_from 2 up_thru 6 down_at 0 last_clean 0-0) 10.243.150.209:6801/20989 10.243.150.209:6802/20989
osd2 in weight 1 up   (up_from 5 up_thru 5 down_at 0 last_clean 0-0) 10.212.118.67:6800/20983 10.212.118.67:6801/20983

Best Regards and thanks again,
   Christian


Am Montag, 5. Juli 2010 schrieb Thomas Mueller:
> Am Mon, 05 Jul 2010 00:33:50 +0200 schrieb Christian Baun:
> 
> > Hi,
> > 
> > I created 2 servers and one client
> > 
> > Server 1 => mon, mds, osd
> > Server 2 => osd
> > 
> > I tried some tests with iozone and I don't think Server 2 is used.
> 
> 
> did you read:
> 
> http://ceph.newdream.net/wiki/Adjusting_replication_level
> 
> 
> 
> - Thomas

Re: Redundancy with Ceph

[Thomas Mueller]

On 05.07.2010 09:55, Christian Baun wrote:
> Hi Thomas,
>
> Thanks a lot for the link and your help!
> Now, the issue are clear.
>
> The output of "ceph osd dump -o -" said that there shall be replication because max_osd is 3 and the size of all pools is 2. But osd2 was "out down".
> The reason: I forgot to start the ceph server at osd2.


you can also use the --allhosts with /etc/init.d/ceph

example:

/etc/init.d/ceph --allhosts start

this starts ceph on all configured hosts with ssh.

- Thomas