Fall back io scheduler for 2.6.15?

Fall back io scheduler for 2.6.15?

[Mingming Cao]

On Wed, 2006-01-11 at 11:43 -0800, Andrew Morton wrote:
> Mingming Cao <cmm@us.ibm.com> wrote:
> >
> > # time ./filetst  -b 1048576 -w -f /mnt/a
> >  	2.6.14		2.6.15
> >  real    0m21.710s	0m25.773s
> >  user    0m0.012s	0m0.004s
> >  sys     0m14.569s	0m15.065s
> 
> That's a big drop.
> 
> Was it doing I/O, or was it all from pagecache?
> 
> >  I also found tiobench(sequential write test) and dbench has similar
> >  regression between 2.6.14 and 2.6.15. Actually I found 2.6.15 rc2
> >  already has the regression.  Is this a known issue?
> 
> No, it is not known.
> 
> > Anyway I will continue looking at the issue...
> 
> Thanks.
Hi, Andrew,

I did some trace, it turns out there isn't regression between 2.6.14 and
2.6.15, and there is no problem in ext3 filesystem.  I am comparing
apple to orange: the tests were run on two different io schedulers. That
makes the bogus throughput difference that I reported to you earlier
this week.

I gave the same boot option "elevator=as" for both 2.6.14 and 2.6.15-rc2
(this has been working for me for a long time to get the anticipatory
scheduler on), but the results are, the io schedulers turned on on the
two kernels are different( see elevator_setup_default()). On 2.6.14, the
fall back io scheduler (if the chosen io scheduler is not found) is set
to the default io scheduler (anticipatory, in this case), but since
2.6.15-rc1, this semanistic is changed to fall back to noop.

Is there any reason to fall back to noop instead of as?  It seems
anticipatory is much better than noop for ext3 with large sequential
write tests (i.e, 1G dd test) ...

Thanks,

Mingming

Re: Fall back io scheduler for 2.6.15?

[Andrew Morton]

Mingming Cao <cmm@us.ibm.com> wrote:
>
> On 2.6.14, the
> fall back io scheduler (if the chosen io scheduler is not found) is set
> to the default io scheduler (anticipatory, in this case), but since
> 2.6.15-rc1, this semanistic is changed to fall back to noop.

OK.  And I assume that AS wasn't compiled, so that's why it fell back?

I actually thought that elevator= got removed, now we have
/sys/block/sda/queue/scheduler.  But I guess that's not very useful with
CONFIG_SYSFS=n.

> Is there any reason to fall back to noop instead of as?  It seems
> anticipatory is much better than noop for ext3 with large sequential
> write tests (i.e, 1G dd test) ...

I suspect that was an accident.  Jens?

Re: Fall back io scheduler for 2.6.15?

[Dave Jones]

On Fri, Jan 13, 2006 at 05:49:14PM -0800, Andrew Morton wrote:
 > Mingming Cao <cmm@us.ibm.com> wrote:
 > >
 > > On 2.6.14, the
 > > fall back io scheduler (if the chosen io scheduler is not found) is set
 > > to the default io scheduler (anticipatory, in this case), but since
 > > 2.6.15-rc1, this semanistic is changed to fall back to noop.
 > 
 > OK.  And I assume that AS wasn't compiled, so that's why it fell back?
 > 
 > I actually thought that elevator= got removed, now we have
 > /sys/block/sda/queue/scheduler.  But I guess that's not very useful with
 > CONFIG_SYSFS=n.

It's also a lifesaver if the default scheduler happens to trigger some breakage
preventing boot, and you can tell users to workaround it with a bootparam
until the real problem is fixed (which has bitten us twice now, as Fedora
like several other distros chooses CFQ by default).

		Dave

Re: Fall back io scheduler for 2.6.15?

[Chuck Ebbert]

In-Reply-To: <20060113174914.7907bf2c.akpm@osdl.org>

On Fri, 13 Jan 2006, Andrew Morton wrote:

> OK.  And I assume that AS wasn't compiled, so that's why it fell back?

As of 2.6.15 you need to use "anticipatory" instead of "as".

Maybe this patch would help?

Signed-off-by: Chuck Ebbert <76306.1226@compuserve.com>

--- 2.6.15a.orig/block/elevator.c
+++ 2.6.15a/block/elevator.c
@@ -150,6 +150,13 @@ static void elevator_setup_default(void)
 	if (!chosen_elevator[0])
 		strcpy(chosen_elevator, CONFIG_DEFAULT_IOSCHED);
 
+	/*
+	 * Be backwards-compatible with previous kernels, so users
+	 * won't get the wrong elevator.
+	 */
+	if (!strcmp(chosen_elevator, "as"))
+		strcpy(chosen_elevator, "anticipatory");
+
  	/*
  	 * If the given scheduler is not available, fall back to no-op.
  	 */
-- 
Chuck
Currently reading: _Olympos_ by Dan Simmons

Re: Fall back io scheduler for 2.6.15?

[Jens Axboe]

On Fri, Jan 13 2006, Andrew Morton wrote:
> Mingming Cao <cmm@us.ibm.com> wrote:
> >
> > On 2.6.14, the
> > fall back io scheduler (if the chosen io scheduler is not found) is set
> > to the default io scheduler (anticipatory, in this case), but since
> > 2.6.15-rc1, this semanistic is changed to fall back to noop.
> 
> OK.  And I assume that AS wasn't compiled, so that's why it fell back?
> 
> I actually thought that elevator= got removed, now we have
> /sys/block/sda/queue/scheduler.  But I guess that's not very useful with
> CONFIG_SYSFS=n.
> 
> > Is there any reason to fall back to noop instead of as?  It seems
> > anticipatory is much better than noop for ext3 with large sequential
> > write tests (i.e, 1G dd test) ...
> 
> I suspect that was an accident.  Jens?

It is, it makes more sense to fallback to the default of course.

-- 
Jens Axboe

Re: Fall back io scheduler for 2.6.15?

[Jens Axboe]

On Sat, Jan 14 2006, Chuck Ebbert wrote:
> In-Reply-To: <20060113174914.7907bf2c.akpm@osdl.org>
> 
> On Fri, 13 Jan 2006, Andrew Morton wrote:
> 
> > OK.  And I assume that AS wasn't compiled, so that's why it fell back?
> 
> As of 2.6.15 you need to use "anticipatory" instead of "as".
> 
> Maybe this patch would help?
> 
> Signed-off-by: Chuck Ebbert <76306.1226@compuserve.com>
> 
> --- 2.6.15a.orig/block/elevator.c
> +++ 2.6.15a/block/elevator.c
> @@ -150,6 +150,13 @@ static void elevator_setup_default(void)
>  	if (!chosen_elevator[0])
>  		strcpy(chosen_elevator, CONFIG_DEFAULT_IOSCHED);
>  
> +	/*
> +	 * Be backwards-compatible with previous kernels, so users
> +	 * won't get the wrong elevator.
> +	 */
> +	if (!strcmp(chosen_elevator, "as"))
> +		strcpy(chosen_elevator, "anticipatory");
> +
>   	/*
>   	 * If the given scheduler is not available, fall back to no-op.
>   	 */

We probably should apply this, since it used to be 'as'.

-- 
Jens Axboe

Re: Fall back io scheduler for 2.6.15?

[Nate Diller]

On 1/16/06, Jens Axboe <axboe@suse.de> wrote:
> On Fri, Jan 13 2006, Andrew Morton wrote:
> > Mingming Cao <cmm@us.ibm.com> wrote:
> > >
> > > On 2.6.14, the
> > > fall back io scheduler (if the chosen io scheduler is not found) is set
> > > to the default io scheduler (anticipatory, in this case), but since
> > > 2.6.15-rc1, this semanistic is changed to fall back to noop.
> >
> > OK.  And I assume that AS wasn't compiled, so that's why it fell back?
> >
> > I actually thought that elevator= got removed, now we have
> > /sys/block/sda/queue/scheduler.  But I guess that's not very useful with
> > CONFIG_SYSFS=n.
> >
> > > Is there any reason to fall back to noop instead of as?  It seems
> > > anticipatory is much better than noop for ext3 with large sequential
> > > write tests (i.e, 1G dd test) ...
> >
> > I suspect that was an accident.  Jens?
>
> It is, it makes more sense to fallback to the default of course.

Not an accident at all, actually, because the original patch i
submitted allowed you to select a scheduler as 'default' even if it
were compiled as a module in kconfig.  Since noop is guaranteed to be
present in any system, it is the obvious choice if the chosen or
default scheduler is not loaded.

If you change it to fall back to the default, it will oops if the
default is not available.

NATE

Re: Fall back io scheduler for 2.6.15?

[Nate Diller]

On 1/16/06, Jens Axboe <axboe@suse.de> wrote:
> On Sat, Jan 14 2006, Chuck Ebbert wrote:
> > In-Reply-To: <20060113174914.7907bf2c.akpm@osdl.org>
> >
> > On Fri, 13 Jan 2006, Andrew Morton wrote:
> >
> > > OK.  And I assume that AS wasn't compiled, so that's why it fell back?
> >
> > As of 2.6.15 you need to use "anticipatory" instead of "as".
> >
> > Maybe this patch would help?
> >
> > Signed-off-by: Chuck Ebbert <76306.1226@compuserve.com>
> >
> > --- 2.6.15a.orig/block/elevator.c
> > +++ 2.6.15a/block/elevator.c
> > @@ -150,6 +150,13 @@ static void elevator_setup_default(void)
> >       if (!chosen_elevator[0])
> >               strcpy(chosen_elevator, CONFIG_DEFAULT_IOSCHED);
> >
> > +     /*
> > +      * Be backwards-compatible with previous kernels, so users
> > +      * won't get the wrong elevator.
> > +      */
> > +     if (!strcmp(chosen_elevator, "as"))
> > +             strcpy(chosen_elevator, "anticipatory");
> > +
> >       /*
> >        * If the given scheduler is not available, fall back to no-op.
> >        */
>
> We probably should apply this, since it used to be 'as'.

i agree

NATE

Re: Fall back io scheduler for 2.6.15?

[Jens Axboe]

On Thu, Jan 19 2006, Nate Diller wrote:
> On 1/16/06, Jens Axboe <axboe@suse.de> wrote:
> > On Fri, Jan 13 2006, Andrew Morton wrote:
> > > Mingming Cao <cmm@us.ibm.com> wrote:
> > > >
> > > > On 2.6.14, the
> > > > fall back io scheduler (if the chosen io scheduler is not found) is set
> > > > to the default io scheduler (anticipatory, in this case), but since
> > > > 2.6.15-rc1, this semanistic is changed to fall back to noop.
> > >
> > > OK.  And I assume that AS wasn't compiled, so that's why it fell back?
> > >
> > > I actually thought that elevator= got removed, now we have
> > > /sys/block/sda/queue/scheduler.  But I guess that's not very useful with
> > > CONFIG_SYSFS=n.
> > >
> > > > Is there any reason to fall back to noop instead of as?  It seems
> > > > anticipatory is much better than noop for ext3 with large sequential
> > > > write tests (i.e, 1G dd test) ...
> > >
> > > I suspect that was an accident.  Jens?
> >
> > It is, it makes more sense to fallback to the default of course.
> 
> Not an accident at all, actually, because the original patch i
> submitted allowed you to select a scheduler as 'default' even if it
> were compiled as a module in kconfig.  Since noop is guaranteed to be
> present in any system, it is the obvious choice if the chosen or
> default scheduler is not loaded.

Yes and that was a bug in that patch. The default scheduler must be
builtin, that's a given. The Kconfig rules should make a default
selection as a module illegal. And they do, they have since been fixed.

> If you change it to fall back to the default, it will oops if the
> default is not available.

It must be.

-- 
Jens Axboe

Re: Fall back io scheduler for 2.6.15?

[Tejun Heo]

Jens Axboe wrote:
> On Sat, Jan 14 2006, Chuck Ebbert wrote:
> 
>>In-Reply-To: <20060113174914.7907bf2c.akpm@osdl.org>
>>
>>On Fri, 13 Jan 2006, Andrew Morton wrote:
>>
>>
>>>OK.  And I assume that AS wasn't compiled, so that's why it fell back?
>>
>>As of 2.6.15 you need to use "anticipatory" instead of "as".
>>
>>Maybe this patch would help?
>>
>>Signed-off-by: Chuck Ebbert <76306.1226@compuserve.com>
>>
>>--- 2.6.15a.orig/block/elevator.c
>>+++ 2.6.15a/block/elevator.c
>>@@ -150,6 +150,13 @@ static void elevator_setup_default(void)
>> 	if (!chosen_elevator[0])
>> 		strcpy(chosen_elevator, CONFIG_DEFAULT_IOSCHED);
>> 
>>+	/*
>>+	 * Be backwards-compatible with previous kernels, so users
>>+	 * won't get the wrong elevator.
>>+	 */
>>+	if (!strcmp(chosen_elevator, "as"))
>>+		strcpy(chosen_elevator, "anticipatory");
>>+
>>  	/*
>>  	 * If the given scheduler is not available, fall back to no-op.
>>  	 */
> 
> 
> We probably should apply this, since it used to be 'as'.
> 

Just out of curiousity, why did 'as' get renamed to 'anticipatory'?

-- 
tejun

Re: Fall back io scheduler for 2.6.15?

[Jens Axboe]

On Sat, Jan 21 2006, Tejun Heo wrote:
> Jens Axboe wrote:
> >On Sat, Jan 14 2006, Chuck Ebbert wrote:
> >
> >>In-Reply-To: <20060113174914.7907bf2c.akpm@osdl.org>
> >>
> >>On Fri, 13 Jan 2006, Andrew Morton wrote:
> >>
> >>
> >>>OK.  And I assume that AS wasn't compiled, so that's why it fell back?
> >>
> >>As of 2.6.15 you need to use "anticipatory" instead of "as".
> >>
> >>Maybe this patch would help?
> >>
> >>Signed-off-by: Chuck Ebbert <76306.1226@compuserve.com>
> >>
> >>--- 2.6.15a.orig/block/elevator.c
> >>+++ 2.6.15a/block/elevator.c
> >>@@ -150,6 +150,13 @@ static void elevator_setup_default(void)
> >>	if (!chosen_elevator[0])
> >>		strcpy(chosen_elevator, CONFIG_DEFAULT_IOSCHED);
> >>
> >>+	/*
> >>+	 * Be backwards-compatible with previous kernels, so users
> >>+	 * won't get the wrong elevator.
> >>+	 */
> >>+	if (!strcmp(chosen_elevator, "as"))
> >>+		strcpy(chosen_elevator, "anticipatory");
> >>+
> >> 	/*
> >> 	 * If the given scheduler is not available, fall back to no-op.
> >> 	 */
> >
> >
> >We probably should apply this, since it used to be 'as'.
> >
> 
> Just out of curiousity, why did 'as' get renamed to 'anticipatory'?

Side effect really, not intentional. 'as' always registered itself with
the elevator core as "anticipatory", the logic to match elevator=foo
string to scheduler used to be completely seperate prior to the addition
of online switchable elevators. So when those two were tied together,
the "as" disappeared. I guess the right thing to do at that time was to
rename "anticipatory", but that didn't happen...

-- 
Jens Axboe