Scrubbing

Scrubbing

[Zenon Panoussis]

Hi

Two OSDs, 264 pgs on each. This is killing the OSD:

2011-04-21 17:49:52.673431    pg v102789: 528 pgs: 299 active+clean, 229 active+clean+scrubbing; 5675 MB data, 26696 MB used, 395 GB / 446 GB avail

Is there some way to tell it to favour serial scrubbing over
parallel?

Z

Re: Scrubbing

[Samuel Just]

  Hmm, it's supposed to throttle background scrubbing.  Could we get 
access to the osd logs?

Thanks,
-Sam

On 04/21/2011 10:40 AM, Zenon Panoussis wrote:
> On 04/21/2011 06:19 PM, Samuel Just wrote:
>
> Hi
>
>> Well, that is definitely not supposed to happen, sorry about
>> that!  Did you request the scrubs or did it start them automatically?
> I had my one monitor mounted by a client. I said "ceph osd scrub 0"
> and it behaved quite nicely, scrubbing two or three pgs at a time
> (I have default osd threads). Then I did "umount mon:/" on the
> client and as soon as umount returned (it took a few seconds) the
> scrubbing exploded.
>
> It's 0.26 tarball release BTW.
>
> Z
>

scrubbing

[Jens Rehpöhler]

[-- Attachment #1: Type: text/plain, Size: 1563 bytes --]

Happy new year  !!

another year another question:

I got the following from ceph -w :

2012-01-02 12:05:17.294350    pg v830384: 1254 pgs: 18 creating, 1235
active+clean, 1 active+clean+scrubbing; 586 GB data, 1378 GB used, 12763
GB / 14887 GB avail
2012-01-02 12:05:23.763982    pg v830385: 1254 pgs: 18 creating, 1235
active+clean, 1 active+clean+scrubbing; 586 GB data, 1378 GB used, 12763
GB / 14887 GB avail
2012-01-02 12:05:27.295568    pg v830386: 1254 pgs: 18 creating, 1235
active+clean, 1 active+clean+scrubbing; 586 GB data, 1378 GB used, 12763
GB / 14887 GB avail
2012-01-02 12:05:28.765457    pg v830387: 1254 pgs: 18 creating, 1235
active+clean, 1 active+clean+scrubbing; 586 GB data, 1378 GB used, 12763
GB / 14887 GB avail

ceph pg dump | grep scrub:
pg_stat    objects    mip    degr    unf    kb    bytes    log   
disklog    state    v    reported    up    acting    last_scrub
34.2    1583    0    0    0    6125554    6272545912    3319    3319   
active+clean+scrubbing    933'385350    926'63653    [3,4]    [3,4]   
927'337222    2011-12-27 13:20:42.436092

This happens once a week. If i restart ceph i takes a moment an the
"scrubbing" disappears.

-- 
mit freundlichen Grüssen

Jens Rehpöhler

----------------------------------------------------------------------
Filoo GmbH
Moltkestr. 25a
33330 Gütersloh
HRB4355 AG Gütersloh

Geschäftsführer: S.Grewing | J.Rehpöhler | C.Kunz
Telefon: +49 5241 8673012 | Mobil: +49 151 54645798
Hotline: 07000-3378658 (14 Ct/min) Fax: +49 5241 8673020



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: scrubbing

[Wido den Hollander]

Hi,

On 01/02/2012 12:10 PM, Jens Rehpöhler wrote:
> Happy new year  !!

Let this be a great year for Ceph :-)

>
> another year another question:
>
> I got the following from ceph -w :
>
> 2012-01-02 12:05:17.294350    pg v830384: 1254 pgs: 18 creating, 1235
> active+clean, 1 active+clean+scrubbing; 586 GB data, 1378 GB used, 12763
> GB / 14887 GB avail
> 2012-01-02 12:05:23.763982    pg v830385: 1254 pgs: 18 creating, 1235
> active+clean, 1 active+clean+scrubbing; 586 GB data, 1378 GB used, 12763
> GB / 14887 GB avail
> 2012-01-02 12:05:27.295568    pg v830386: 1254 pgs: 18 creating, 1235
> active+clean, 1 active+clean+scrubbing; 586 GB data, 1378 GB used, 12763
> GB / 14887 GB avail
> 2012-01-02 12:05:28.765457    pg v830387: 1254 pgs: 18 creating, 1235
> active+clean, 1 active+clean+scrubbing; 586 GB data, 1378 GB used, 12763
> GB / 14887 GB avail

Which version are you running?

This was a known bug but should be fixed about two weeks ago in the 
master branch.

I also encountered this one, but it's resolved for me right now.

Wido

>
> ceph pg dump | grep scrub:
> pg_stat    objects    mip    degr    unf    kb    bytes    log
> disklog    state    v    reported    up    acting    last_scrub
> 34.2    1583    0    0    0    6125554    6272545912    3319    3319
> active+clean+scrubbing    933'385350    926'63653    [3,4]    [3,4]
> 927'337222    2011-12-27 13:20:42.436092
>
> This happens once a week. If i restart ceph i takes a moment an the
> "scrubbing" disappears.
>

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: scrubbing

[Wido den Hollander]

Hi,

On 01/02/2012 12:34 PM, Jens Rehpöhler wrote:
> Hi Wido,
>
> Am 02.01.2012 12:13, schrieb Wido den Hollander:
>> Hi,
>>
>> On 01/02/2012 12:10 PM, Jens Rehpöhler wrote:
>>> Happy new year  !!
>>
>> Let this be a great year for Ceph :-)
>>
> i think it will be ;)
>>>
>>> another year another question:
>>>
>>> I got the following from ceph -w :
>>>
>>> 2012-01-02 12:05:17.294350    pg v830384: 1254 pgs: 18 creating, 1235
>>> active+clean, 1 active+clean+scrubbing; 586 GB data, 1378 GB used, 12763
>>> GB / 14887 GB avail
>>> 2012-01-02 12:05:23.763982    pg v830385: 1254 pgs: 18 creating, 1235
>>> active+clean, 1 active+clean+scrubbing; 586 GB data, 1378 GB used, 12763
>>> GB / 14887 GB avail
>>> 2012-01-02 12:05:27.295568    pg v830386: 1254 pgs: 18 creating, 1235
>>> active+clean, 1 active+clean+scrubbing; 586 GB data, 1378 GB used, 12763
>>> GB / 14887 GB avail
>>> 2012-01-02 12:05:28.765457    pg v830387: 1254 pgs: 18 creating, 1235
>>> active+clean, 1 active+clean+scrubbing; 586 GB data, 1378 GB used, 12763
>>> GB / 14887 GB avail
>>
>> Which version are you running?
>>
>> This was a known bug but should be fixed about two weeks ago in the
>> master branch.
>>
>> I also encountered this one, but it's resolved for me right now.
>>
> I use:
>
> ceph version 0.39-15-ge93961c
> (commit:e93961c11119942eae3a4cd14a79f779a5a4d277)
>
> from the "stable" branch.
>
> OS is Debian 6.0.3
>
> So i will wait until it is patched into the stable branch too.

I think it won't be to long until v0.40 comes out. But in the meantime 
you can always try to grab the latest code from the master branch.

Wido

>
> Thank you for the very quick answer.
>

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html