Re: RADOS + deep scrubbing performance issues in production environment

[Filippos Giannakos <philipgian@grnet.gr>]

On Mon, Jan 27, 2014 at 10:45:48AM -0800, Sage Weil wrote:
> There is also 
> 
>  ceph osd set noscrub
> 
> and then later
> 
>  ceph osd unset noscrub
> 
> I forget whether this pauses an in-progress PG scrub or just makes it stop 
> when it gets to the next PG boundary.
> 
> sage

I bumped into those settings but I couldn't find any documentation about them.
When I first tried them, they didn't do anything immediately, so I thought they
weren't the answer. After your mention, I tried them again, and after a while
the deep-scrubbing stopped. So I'm guessing they stop scrubbing on the next PG
boundary.

I see from this thread and others before, that some people think it is a spindle
issue. I'm not sure that it is just that. Replicating it to an idle cluster that
can do more than 250MiB/seconds and pausing for 4-5 seconds on a single request,
sounds like an issue by itself. Maybe there is too much locking or not enough
priority to the actual I/O ? Plus, that idea of throttling deep scrubbing based
on the iops sounds appealing.

Kind Regards,
-- 
Filippos
<philipgian@grnet.gr>