Re: [ceph-commit] HEALTH_WARN 192 pgs degraded

Re: [ceph-commit] HEALTH_WARN 192 pgs degraded

[Josh Durgin]

Moving to ceph-devel, ceph-commit is for notifications from the source 
code repository

On 2012-10-24 06:57, Roman Alekseev wrote:
> Hi there,
>
>
> I've made simple fresh installation of ceph on Debian server with the
> following configuration:
> ************************
> [global]
>     debug ms = 0
> [osd]
>     osd journal size = 1000
>     filestore xattr use omap = true
>
> [mon.a]
>
>     host = serv1
>     mon addr = 192.168.0.10:6789
>
> [osd.0]
>      host = serv1
>
> [mds.a]
>     host = serv1
> ************************
>
> Seems everything is working fine but when I perform "ceph health"
> command I receive the next message:
> HEALTH_WARN 192 pgs degraded; 192 pgs stuck unclean; recovery 21/42
> degraded (50.000%)

The default replication level is 2, but you only have 1 osd. Thus,
you only have 1 copy of everything, and ceph reports this condition
as 'degraded'. You can add a second osd and it will copy data
and report HEALTH_OK.

Josh

Re: [ceph-commit] HEALTH_WARN 192 pgs degraded

[Sage Weil]

[moved to ceph-devel]

On Wed, 24 Oct 2012, Roman Alekseev wrote:
> Hi there,
> 
> 
> I've made simple fresh installation of ceph on Debian server with the
> following configuration:
> ************************
> [global]
>     debug ms = 0
> [osd]
>     osd journal size = 1000
>     filestore xattr use omap = true
> 
> [mon.a]
> 
>     host = serv1
>     mon addr = 192.168.0.10:6789
> 
> [osd.0]
>      host = serv1
> 
> [mds.a]
>     host = serv1
> ************************
> 
> Seems everything is working fine but when I perform "ceph health" command I
> receive the next message:
> HEALTH_WARN 192 pgs degraded; 192 pgs stuck unclean; recovery 21/42 degraded
> (50.000%)

This is simply because you only have 1 osd but the default policy is 2x 
replication.  As such, all PGs are 'degraded' because they are only 
replicated once.

If you add another OSD to your cluster the warning will go away.

sage


> 
> Kindly help me to find the reason of this issue.
> Thanks.
> 
> -- 
> Kind regards,
> 
> R. Alekseev
> 
> _______________________________________________
> Ceph-commit mailing list
> Ceph-commit@lists.ceph.newdream.net
> http://lists.ceph.newdream.net/listinfo.cgi/ceph-commit-ceph.newdream.net
> 
> 

Re: [ceph-commit] HEALTH_WARN 192 pgs degraded

[Mark Kirkwood]

On 25/10/12 04:40, Sage Weil wrote:
> [moved to ceph-devel]
>
> On Wed, 24 Oct 2012, Roman Alekseev wrote:
>> Hi there,
>>
>>
>> I've made simple fresh installation of ceph on Debian server with the
>> following configuration:
>> ************************
>> [global]
>>      debug ms = 0
>> [osd]
>>      osd journal size = 1000
>>      filestore xattr use omap = true
>>
>> [mon.a]
>>
>>      host = serv1
>>      mon addr = 192.168.0.10:6789
>>
>> [osd.0]
>>       host = serv1
>>
>> [mds.a]
>>      host = serv1
>> ************************
>>
>> Seems everything is working fine but when I perform "ceph health" command I
>> receive the next message:
>> HEALTH_WARN 192 pgs degraded; 192 pgs stuck unclean; recovery 21/42 degraded
>> (50.000%)
> This is simply because you only have 1 osd but the default policy is 2x
> replication.  As such, all PGs are 'degraded' because they are only
> replicated once.
>
> If you add another OSD to your cluster the warning will go away.
>
> sage
>

The other alternative is to just set the pool(s) replication size to 1, 
if you are just wanting a single osd for (say) testing:

$ ceph osd pool set <your pool(s)> size 1

I find it I need to restart ceph after doing the above, it then sorts 
itself out to a nice healthy status!

Regards

Mark

Re: [ceph-commit] HEALTH_WARN 192 pgs degraded

[Dan Mick]

>>> HEALTH_WARN 192 pgs degraded; 192 pgs stuck unclean; recovery 21/42
>>> degraded
>>> (50.000%)
>> This is simply because you only have 1 osd but the default policy is 2x
>> replication.  As such, all PGs are 'degraded' because they are only
>> replicated once.
>>
>> If you add another OSD to your cluster the warning will go away.
>>
>> sage
>>
>
> The other alternative is to just set the pool(s) replication size to 1,
> if you are just wanting a single osd for (say) testing:
>
> $ ceph osd pool set <your pool(s)> size 1
>
> I find it I need to restart ceph after doing the above, it then sorts
> itself out to a nice healthy status!

OTOH it should be noted that I'm constantly running test clusters with
replication 2, #osd 1, and they're "unhealthy", and work just fine.
(These are test clusters, clearly, but...nothing wrong with them.)

Re: [ceph-commit] HEALTH_WARN 192 pgs degraded

[Mark Kirkwood]

On 25/10/12 17:55, Mark Nelson wrote:
> On Wed, Oct 24, 2012 at 10:58 PM, Dan Mick <dan.mick@inktank.com> wrote:
>
>>   HEALTH_WARN 192 pgs degraded; 192 pgs stuck unclean; recovery 21/42
>>> The other alternative is to just set the pool(s) replication size to 1,
>>> if you are just wanting a single osd for (say) testing:
>>>
>>> $ ceph osd pool set <your pool(s)> size 1
>>>
>>> I find it I need to restart ceph after doing the above, it then sorts
>>> itself out to a nice healthy status!
>>>
>>
>>
> I was actually just talking to Greg and Sam about this earlier today.  If
> you rely on ceph health as part of an automated process to determine
> whether or not tests should start running, having degraded PGs due to some
> of the pools expecting 2x replication (when there is 1 OSD) is annoying.
> It will go away if whatever default pools are created get manually set to
> 1x replication, but it's not something that is immediately obvious.  I
> don't know that changing the defaults is necessarily the right answer.
> Instead perhaps we just haven't done a good enough job of explaining what
> pools get created, how they are used, and when/if they should be modified
> in some way .  Maybe this belongs in a FAQ?
>



Ah yes - you are quite right, it is *not* required to restart ceph to 
make it sort out those stuck pages after changing the size. I believe at 
some point (maybe < 0.50) it was and I had gotten into the habit!

+1 for adding a FAQ about the defaults pools and replication levels etc.

Cheers

Mark