Is there any way to throttle recovery?

Is there any way to throttle recovery?

[Vladimir Bashkirtsev]

Hello devs,

I have noticed that while ceph recovering from OSD failure RBD 
performance becomes dismal. Judging by load on the net ceph attempts to 
shuffle around huge amount of data leaving virtually no bandwidth for 
normal operations. Is there any way to throttle recovery process? 
Network throttling is out of question as it will slow everything down. 
So I guess ceph should have (or already has) a setting which says do not 
recover faster than X.

Regards,
Vladimir

Re: Is there any way to throttle recovery?

[Wido den Hollander]

On 09/03/2012 08:42 AM, Vladimir Bashkirtsev wrote:
> Hello devs,
>
> I have noticed that while ceph recovering from OSD failure RBD
> performance becomes dismal. Judging by load on the net ceph attempts to
> shuffle around huge amount of data leaving virtually no bandwidth for
> normal operations. Is there any way to throttle recovery process?
> Network throttling is out of question as it will slow everything down.
> So I guess ceph should have (or already has) a setting which says do not
> recover faster than X.
>

I don't know if there is a bandwidth throttle, but you could lower the 
number of recovery ops with:

osd recovery max active = 1

By default it is set to 5, by tuning this down you'll put less stress on 
the OSDs and it will probably also result in less network traffic.

Wido

> Regards,
> Vladimir
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html