From: Mike Galbraith <efault@gmx.de>
To: Jason Garrett-Glaser <jason@x264.com>
Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: Scheduler still seems awful with x264, worse with patches
Date: Wed, 09 May 2012 18:24:29 +0200 [thread overview]
Message-ID: <1336580669.16079.68.camel@marge.simpson.net> (raw)
In-Reply-To: <CABS55+A_ZbJT4Tz-XfacbURpgfx2WuXqvPnn1_XkAfhxJMP6eA@mail.gmail.com>
On Wed, 2012-05-09 at 09:00 -0700, Jason Garrett-Glaser wrote:
> Many months ago, the topic of CFS's inefficiencies with x264 came up
> and some improvements were made, but BFS and Windows still stayed a
> little bit in the lead. This seemed to be because of a mix of two
> issues. Firstly, a combination of relatively short-lived jobs (x264
> uses a thread pool, so the actual threads are long-lived). Secondly,
> in frame threads, heavy dependencies between threads, benefiting
> greatly from a dumb scheduler. Thirdly, in sliced threads -- the
> focus of this post -- the best scheduling approach is to simply spread
> them throughout the cores and do nothing, so again, a dumb scheduler
> will do the right thing.
I took x264 for a quick test drive a short while ago, and it looks like
we slipped a bit. I didn't have time to futz with it much, but did find
that SCHED_IDLE kicked SCHED_OTHER's butt. x264 really really wants RR.
> Recently I tried multithreading x264's lookahead for a customer. The
> lookahead previously wasn't threaded, causing bottlenecks with many
> cores and threads. I do my development mainly on Windows, and the
> patch looked to be quite a success, with nice performance boosts for
> many target use-cases.
>
> And then I ran it on Linux and it choked horribly.
>
> The patch is here:
> https://github.com/DarkShikari/x264-devel/commit/99e830f1581eac3cf30f07b1d6c6c32bae1725c8
> . To replicate the test, simply test that version against the
> previous version. My guess is the reason it chokes is that it
> involves spawning even *shorter*-lived jobs than x264 typically does,
> something that CFS seems to simply collapse on.
>
> Here's some stats from a recent kernel:
>
> SD encoding (before -> after patch):
> CFS: 325.49 +/- 1.22 fps -> 251.68 +/- 2.32 fps
> BFS: 334.94 +/- 0.59 fps -> 344.47 +/- 0.68 fps
>
> HD encoding (before -> after patch):
> CFS: 39.05 +/- 0.22 fps -> 40.56 +/- 0.23 fps
> BFS: 40.15 +/- 0.05 fps -> 44.89 +/- 0.05 fps
>
> As can be seen, the longer the threads live (the lower the fps), the
> less horrific the penalty is. Furthermore, though I don't have
> numbers, using schedtool -R -p 1 does basically as well as BFS in
> eliminating the problem. Naturally, this is not really a solution as
> it requires root.
>
> To replicate this test, a commandline like this should work on any
> cached raw input file (a collection of free raw videos can be found
> here if you don't like making your own:
> http://media.xiph.org/video/derf/ ):
>
> ./x264 --preset superfast --tune zerolatency --threads X input -o /dev/null
>
> Most of my testing was done with X = 4 or 8 on quad-core or
> quad-core-with-HT machines. Though I don't have numbers, I should
> note that the absolute worst results are on a Core i7 machine where I
> set taskset 0xf and used threads 4; the patch literally cut the
> framerate in half, despite giving a performance boost when SCHED_RR
> was used.
>
> Jason
>
> P.S. Here is the structure of low-latency sliced threading with the
> patch, for anyone curious.
>
> To encode a frame with X threads:
> 1. The previous frame's X threads are still going -- they're
> finishing up hpel and deblock calculation, which can be done after the
> encoder returns, but before the next frame is encoded.
> 2. Serially, copy in the input frame and preprocess it, running
> various not-yet-parallelized tasks while those threads finish up.
> 3. In parallel, do lookahead analysis of the input frame -- split it
> into X slices and analyze them simultaneously. Then wait on the jobs
> and merge the results.
> 4. If any of the previous frame's threads are still going, wait on them now.
> 5. Split the main encode into X slices and do them simultaneously.
> Once they've all finished encoding -- after they've signaled the main
> thread with a condition variable -- serially finish up a few minor
> tasks and return to the caller. These threads will continue until
> step 4) of the next frame.
>
> When encoding at ~300fps, the encode jobs will typically run for about
> ~3ms each, and the lookahead threads likely for around 0.5ms, very
> roughly. For a low-latency encoder, there is no way to make these
> threads last longer while still getting parallelism, and it must be
> doable because Windows handles it just fine.
>
> Jason
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
next prev parent reply other threads:[~2012-05-09 16:24 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-05-09 16:00 Scheduler still seems awful with x264, worse with patches Jason Garrett-Glaser
2012-05-09 16:24 ` Mike Galbraith [this message]
2012-05-09 16:29 ` Jason Garrett-Glaser
2012-05-10 10:29 ` Mike Galbraith
2012-05-10 16:22 ` Jason Garrett-Glaser
2012-05-10 18:25 ` Mike Galbraith
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1336580669.16079.68.camel@marge.simpson.net \
--to=efault@gmx.de \
--cc=jason@x264.com \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.