Re: RSDL-mm 0.28

Con Kolivas
2007-03-11T02:28:26Z

On 11/03/07, Matt Mackall <mpm@selenic.com> wrote:
> I've tested -mm2 against -mm2+noyield and -mm2+rsdl+noyield. The
> noyield patch simply makes the sched_yield syscall return immediately.
> Xorg and all tests are run at nice 0.
>
> Loads:
> memload: constant memcpy of 16MB buffer
> execload: constant re-exec of a trivial shell script
> forkload: constant fork and exit of a trivial shell script
> make -j 5: hot-cache kernel build without ccache
> make -j 5 ccache: hot-cache kernel build with ccache
>
> Tests:
> beryl - 3D window manager, wiggle windows, spin desktop, etc.
> galeon - web browser, rapidly scrolling long web pages by grabbing
> the scroll bar
> mp3 - XMMS on a FUSE sshfs over wireless (during all tests)
> terminal - responsiveness of ssh and local terminal sessions
> mouse - responsiveness of mouse pointer
>
> Results:
> great = completely smooth
> good = fully responsive
> ok = visible latency
> bad = becomes difficult to use (or mp3 skips)
> awful = make it stop, please
>
> -mm2 -mm2+noyield rsdl+noyield
> no load
> beryl great great great
> galeon good good good
> mp3 good good good
> terminal good good good
> mouse good good good
> memload x10
> beryl awful/bad great good
> galeon good good ok/good
> mp3 good good good
> terminal good good good
> mouse good good good
> execload x10
> beryl awful/bad bad/good good
> galeon good bad/good ok/good
> mp3 good bad good
> terminal good bad/good good
> mouse good bad/good good
> forkload x10
> beryl good good great
> galeon good good ok/good
> mp3 good good good
> terminal good good ok/good
> mouse good good good
> make -j 5
> beryl ok good good/great
> galeon good good ok/good
> mp3 good good good
> terminal good good good
> mouse good good good
> make -j 5 ccache
> beryl ok good awful
> galeon good good bad
> mp3 good good bad
> terminal good good bad/ok
> mouse good good bad/ok
>
> make -j 5
> real 8m1.857s 8m50.659s 8m9.282s
> user 7m19.127s 8m3.494s 7m30.740s
> sys 0m30.910s 0m33.722s 0m29.542s
>
> make -j 5 ccache
> real 2m6.182s 2m19.032s 2m1.832s
> user 1m39.466s 1m48.787s 1m37.250s
> sys 0m19.741s 0m22.993s 0m20.109s

Thanks very much for that comprehensive summary and testing!

> There's a substantial performance hit for not yield, so we probably
> want to investigate alternate semantics for it. It seems reasonable
> for apps to say "let me not hog the CPU" without completely expiring
> them. Imagine you're in the front of the line (aka queue) and you
> spend a moment fumbling for your wallet. The polite thing to do is to
> let the next guy in front. But with the current sched_yield, you go
> all the way to the back of the line.

Well... are you advocating we change sched_yield semantics to a
gentler form? This is a cinch to implement but I know how Ingo feels
about this. It will only encourage more lax coding using sched_yield
instead of proper blocking (see huge arguments with the ldap people on
this one who insist it's impossible not to use yield).

> RSDL makes most of the noyield hit back in normal make and then some
> with ccache. Impressive. But ccache is still destroying interactivity
> somehow. The ccache effect is fairly visible even with non-parallel
> 'make'.

Ok I don't think there's any actual accounting problem here per se
(although I did just recently post a bugfix for rsdl however I think
that's unrelated). What I think is going on in the ccache testcase is
that all the work is being offloaded to kernel threads reading/writing
to/from the filesystem and the make is not getting any actual cpu
time. This is "worked around" in mainline thanks to the testing for
sleeping on uninterruptible sleep in the interactivity estimator. What
I suspect is happening is kernel threads that are running nice -5 are
doing all the work on make's behalf in the setting of ccache since it
is mostly i/o bound. The reason for -nice values on kernel threads is
questionable anyway. Can you try renicing your kernel threads all to
nice 0 and see what effect that has? Obviously this doesn't need a
recompile, but is simple enough to implement in kthread code as a new
default.

> Also note I could occassionally trigger nasty multi-second pauses with
> -mm2+noyield under exectest that didn't show up elsewhere. That's
> probably a bug in the mainline scheduler.

Ew. It's probably not a bug but a good example of some of the
starvation scenarios we're hitting on mainline (hence the need for a
rewrite ;))

Thanks!

---
ck