[ANNOUNCE] RT for v2.6.34.8 now available.

[ANNOUNCE] RT for v2.6.34.8 now available.

[Paul Gortmaker]

As a value add to the 2.6.34 long term release, I'm happy to also
announce the availability of 2.6.34-RT.  

You can find it in the v2.6.34-rt branch at:

  git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git

as a repository of patches.  The v2.6.34-rt branch contains the latest
RT patches against the latest v2.6.34.8 kernel release. (The master
branch currently stops at v2.6.34 flat, i.e. 2.6.34.0 so to speak.)

I've also created over 400 known working RT enabled bisection points
between 33 and 34 that you can make use of for testing.  The details
on how/why those exist follows - read on if it is of interest to you.

Note that all credit of the thinking and engineering of the RT stuff
lies with the original patch authors -- to be clear, I'm just doing
the grunt work of a carry forward here.

Have fun with it, maybe drop me a note if you find it useful etc.

Paul.


The carry forward process:
---------------------------

This warrants some notes, since (a) it has strong consequence on how
you can diagnose/bisect problems that were not in 33-rt but may exist
in 34-rt and (b) we've recently seen people who wonder how to do a
similar task and I think they *really* underestimate the complexities
and sheer effort of trying to do any sort of RT carry-forward.

For those who track these things, you will know that the most recent
release of RT was based on v2.6.33, and in turn, it was created by
merging forward the mainline kernel.org v2.6.33 content into the older
v2.6.31 RT tree.  There was no rebase.

With that detail, some of you are probably now going "Aha, now I know
why he created that merge-free v2.6.33 based RT tree some weeks ago."
(see: http://www.spinics.net/lists/linux-rt-users/msg06310.html )

Step one was that I needed the RT commits rebased against 33 (and not 31)
as my starting point for rebasing RT onto v2.6.34 -- see the above
link for details on the significant work involved with that.

There are roughly 500 RT patches, and literally 10,000+ commits between
v2.6.33 and v2.6.34.  So if one was to move things ahead all in one go,
there can be roughly 5 million things that can go wrong.  Maybe some sharp
person can move those ahead all in one shot, and then figure out the
resulting inevitable runtime breakage, but that isn't me.  After all, a
man has to know his limitations, and I'm pretty much just a patch monkey. 

Knowing that the RT patches applied (and worked) on base X, but fail to
apply or fail to work on X+N (where N is small) can be very empowering
to a dumb person like me.  It reduces the problem space *immensely* and
it lets you focus on what changed and caused the problem/conflict.  With
that in mind, I saw no choice but to undertake what would have been the
equivalent of recreating history, had a continuous integration between
the two streams taken place during the 33 --> 34 dev cycle.

What this means in concrete terms, is incrementally applying, updating,
and testing the 500-odd RT patches at successive points along the 10000
commits found in the 33 --> 34 development cycle.  But assuming you
agree with that logic, what does a person choose/use as increments?

At a 1st guess, a person might suggest choosing the rcN tags (where we
have N=1,2,... 7) as places for these incremental tests.  The problem
is that the granularity is too coarse -- just for N=1 means you have
already jumped over 6000 commits ahead, and so you are still facing
being lost for the same reason 10000 commits left you lost before.

At the opposite end of the scale is a brute force approach, where a
person tries applying/updating/testing the RT patch on *each* of the
10,000 development commits. Sounds like it might work, but no.  The
problem here is that the kernel development history is non-linear
in time.  See the git FAQ on why bisects can drag you back in time, if
you've never personally encountered this yourself.  A lot of people
are surprised by this when they first encounter it.

My approach was to adopt a semi-brute force approach.  During the dev
cycle of any kernel, Linus merges the content from various subsystem
maintainers.  Each of these merge points represents a point where
you are guaranteed to not be "rewound in time", should you choose it
as a bisect point.  Using git, these points can easily be identified.

There happens to be ~400 of them, so the 10,000 development commits get
"digested" at an average rate of a humanly manageable number of about 25
commits each -- something that a stupid person like me has a chance to
be able to diagnose/debug without going insane.  You will find that there
is an un-annotated tag for each of these merges in the patch repository.
You really should use these for bisecting your own problems/issues.

On each of these, I've done a patch test, a compile test (x86, x86-64,
ppc, arm) and a boot test (x86, x86-64, ppc-SMP) to ensure that I've not
done a colossal screw-up.  I've probably still screwed *something* up,
but at least I've ensured some level of continuing sanity with these tests
being done across these integration points.
--

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Nathan Grennan]

On 03/04/2011 02:24 PM, Paul Gortmaker wrote:
> As a value add to the 2.6.34 long term release, I'm happy to also
> announce the availability of 2.6.34-RT.
>
> You can find it in the v2.6.34-rt branch at:
>
>    git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
>
> as a repository of patches.  The v2.6.34-rt branch contains the latest
> RT patches against the latest v2.6.34.8 kernel release. (The master
> branch currently stops at v2.6.34 flat, i.e. 2.6.34.0 so to speak.)
>
> I've also created over 400 known working RT enabled bisection points
> between 33 and 34 that you can make use of for testing.  The details
> on how/why those exist follows - read on if it is of interest to you.
>
> Note that all credit of the thinking and engineering of the RT stuff
> lies with the original patch authors -- to be clear, I'm just doing
> the grunt work of a carry forward here.
>
> Have fun with it, maybe drop me a note if you find it useful etc.

I tried compiling this, and it quickly failed. Ideas?

I did this on a Fedora 14 x86_64 system with gcc-4.5.1-4.fc14.x86_64.


Here is what I did:

1. Downloaded 2.6.34.8 tarball and extracted it
2. git clone -b v2.6.34-rt 
git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
3. cat ../rt-patches/series | xargs -i cat ../rt-patches/{} | patch -p1 
(Applied just fine)
4. Copied an 2.6.31-rt configuration to .config
5. Ran make oldconfig
6. make bzImage


CHK include/linux/version.h
CHK include/generated/utsrelease.h
HOSTCC scripts/basic/fixdep
HOSTCC scripts/basic/docproc
HOSTCC scripts/basic/hash
CC kernel/bounds.s
GEN include/generated/bounds.h
CC arch/x86/kernel/asm-offsets.s
GEN include/generated/asm-offsets.h
CALL scripts/checksyscalls.sh
HOSTCC scripts/genksyms/genksyms.o
HOSTCC scripts/genksyms/lex.o
HOSTCC scripts/genksyms/parse.o
HOSTLD scripts/genksyms/genksyms
CC scripts/mod/empty.o
HOSTCC scripts/mod/mk_elfconfig
MKELF scripts/mod/elfconfig.h
HOSTCC scripts/mod/file2alias.o
HOSTCC scripts/mod/modpost.o
HOSTCC scripts/mod/sumversion.o
HOSTLD scripts/mod/modpost
HOSTCC scripts/selinux/genheaders/genheaders
HOSTCC scripts/selinux/mdp/mdp
HOSTCC scripts/kallsyms
HOSTCC scripts/conmakehash
CC init/main.o
init/main.c: In function ‘do_one_initcall’:
init/main.c:740:10: warning: ‘calltime.tv64’ may be used uninitialized 
in this function
CHK include/generated/compile.h
CC init/version.o
CC init/do_mounts.o
In file included from include/linux/suspend.h:8:0,
from init/do_mounts.c:6:
include/linux/mm.h: In function ‘pte_lock_init’:
include/linux/mm.h:1125:2: error: implicit declaration of function ‘kmalloc’
include/linux/mm.h:1125:12: warning: assignment makes pointer from 
integer without a cast
include/linux/mm.h: In function ‘pte_lock_deinit’:
include/linux/mm.h:1137:2: error: implicit declaration of function ‘kfree’
In file included from include/linux/security.h:36:0,
from init/do_mounts.c:8:
include/linux/slab.h: At top level:
include/linux/slab.h:144:6: warning: conflicting types for ‘kfree’
include/linux/mm.h:1137:2: note: previous implicit declaration of 
‘kfree’ was here
In file included from include/linux/slab.h:172:0,
from include/linux/security.h:36,
from init/do_mounts.c:8:
include/linux/slab_def.h:128:52: error: conflicting types for ‘kmalloc’
include/linux/mm.h:1125:14: note: previous implicit declaration of 
‘kmalloc’ was here
make[1]: *** [init/do_mounts.o] Error 1
make: *** [init] Error 2
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Paul Gortmaker]

On Fri, Mar 4, 2011 at 10:05 PM, Nathan Grennan
<linux-rt-users@cygnusx-1.org> wrote:
> On 03/04/2011 02:24 PM, Paul Gortmaker wrote:
>>
>> As a value add to the 2.6.34 long term release, I'm happy to also
>> announce the availability of 2.6.34-RT.
>>
>> You can find it in the v2.6.34-rt branch at:
>>
>>   git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
>>
>> as a repository of patches.  The v2.6.34-rt branch contains the latest
>> RT patches against the latest v2.6.34.8 kernel release. (The master
>> branch currently stops at v2.6.34 flat, i.e. 2.6.34.0 so to speak.)
>>
>> I've also created over 400 known working RT enabled bisection points
>> between 33 and 34 that you can make use of for testing.  The details
>> on how/why those exist follows - read on if it is of interest to you.
>>
>> Note that all credit of the thinking and engineering of the RT stuff
>> lies with the original patch authors -- to be clear, I'm just doing
>> the grunt work of a carry forward here.
>>
>> Have fun with it, maybe drop me a note if you find it useful etc.
>
> I tried compiling this, and it quickly failed. Ideas?

Yes, it is the header cleanup done in the 33-->34 dev cycle, seen
between the rev list ^9e74e7c b66696e that changes slab.h use.

If you look at 8a2032dba45e24 *in the patch repo* you will see where
I fixed it for the baseline RT-enabled defconfig.  (Meaning I did a
make defconfig, then enabled the RT options).  Your config must be
setting something different/extra.

If you do a git show on the above rev list and on the fix in the patch
repo, you will probably be able to tell what is going on.  Also please
send me your config (off list) so I can ensure it compiles properly myself.

Thanks,
Paul.

>
> I did this on a Fedora 14 x86_64 system with gcc-4.5.1-4.fc14.x86_64.
>
>
> Here is what I did:
>
> 1. Downloaded 2.6.34.8 tarball and extracted it
> 2. git clone -b v2.6.34-rt
> git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
> 3. cat ../rt-patches/series | xargs -i cat ../rt-patches/{} | patch -p1
> (Applied just fine)
> 4. Copied an 2.6.31-rt configuration to .config
> 5. Ran make oldconfig
> 6. make bzImage
>
>
> CHK include/linux/version.h
> CHK include/generated/utsrelease.h
> HOSTCC scripts/basic/fixdep
> HOSTCC scripts/basic/docproc
> HOSTCC scripts/basic/hash
> CC kernel/bounds.s
> GEN include/generated/bounds.h
> CC arch/x86/kernel/asm-offsets.s
> GEN include/generated/asm-offsets.h
> CALL scripts/checksyscalls.sh
> HOSTCC scripts/genksyms/genksyms.o
> HOSTCC scripts/genksyms/lex.o
> HOSTCC scripts/genksyms/parse.o
> HOSTLD scripts/genksyms/genksyms
> CC scripts/mod/empty.o
> HOSTCC scripts/mod/mk_elfconfig
> MKELF scripts/mod/elfconfig.h
> HOSTCC scripts/mod/file2alias.o
> HOSTCC scripts/mod/modpost.o
> HOSTCC scripts/mod/sumversion.o
> HOSTLD scripts/mod/modpost
> HOSTCC scripts/selinux/genheaders/genheaders
> HOSTCC scripts/selinux/mdp/mdp
> HOSTCC scripts/kallsyms
> HOSTCC scripts/conmakehash
> CC init/main.o
> init/main.c: In function ‘do_one_initcall’:
> init/main.c:740:10: warning: ‘calltime.tv64’ may be used uninitialized in
> this function
> CHK include/generated/compile.h
> CC init/version.o
> CC init/do_mounts.o
> In file included from include/linux/suspend.h:8:0,
> from init/do_mounts.c:6:
> include/linux/mm.h: In function ‘pte_lock_init’:
> include/linux/mm.h:1125:2: error: implicit declaration of function ‘kmalloc’
> include/linux/mm.h:1125:12: warning: assignment makes pointer from integer
> without a cast
> include/linux/mm.h: In function ‘pte_lock_deinit’:
> include/linux/mm.h:1137:2: error: implicit declaration of function ‘kfree’
> In file included from include/linux/security.h:36:0,
> from init/do_mounts.c:8:
> include/linux/slab.h: At top level:
> include/linux/slab.h:144:6: warning: conflicting types for ‘kfree’
> include/linux/mm.h:1137:2: note: previous implicit declaration of ‘kfree’
> was here
> In file included from include/linux/slab.h:172:0,
> from include/linux/security.h:36,
> from init/do_mounts.c:8:
> include/linux/slab_def.h:128:52: error: conflicting types for ‘kmalloc’
> include/linux/mm.h:1125:14: note: previous implicit declaration of ‘kmalloc’
> was here
> make[1]: *** [init/do_mounts.o] Error 1
> make: *** [init] Error 2
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Nathan Grennan]

> Yes, it is the header cleanup done in the 33-->34 dev cycle, seen
> between the rev list ^9e74e7c b66696e that changes slab.h use.
>
> If you look at 8a2032dba45e24 *in the patch repo* you will see where
> I fixed it for the baseline RT-enabled defconfig.  (Meaning I did a
> make defconfig, then enabled the RT options).  Your config must be
> setting something different/extra.
>
> If you do a git show on the above rev list and on the fix in the patch
> repo, you will probably be able to tell what is going on.  Also please
> send me your config (off list) so I can ensure it compiles properly myself.
>

I found the cause. The patching was incomplete. The correct patch 
command is the one below, removing the comment lines.

cat ../rt-patches/series | grep -v '\#' | xargs -i cat ../rt-patches/{} 
| patch -p1

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Paul Gortmaker]

On 11-03-07 04:36 PM, Nathan Grennan wrote:
> 
>> Yes, it is the header cleanup done in the 33-->34 dev cycle, seen
>> between the rev list ^9e74e7c b66696e that changes slab.h use.
>>
>> If you look at 8a2032dba45e24 *in the patch repo* you will see where
>> I fixed it for the baseline RT-enabled defconfig.  (Meaning I did a
>> make defconfig, then enabled the RT options).  Your config must be
>> setting something different/extra.
>>
>> If you do a git show on the above rev list and on the fix in the patch
>> repo, you will probably be able to tell what is going on.  Also please
>> send me your config (off list) so I can ensure it compiles properly myself.
>>
> 
> I found the cause. The patching was incomplete. The correct patch 
> command is the one below, removing the comment lines.
> 
> cat ../rt-patches/series | grep -v '\#' | xargs -i cat ../rt-patches/{} 
> | patch -p1

Folks should really use git, or quilt, or guilt, or git quilt-import, or
for that matter, *anything* that at least checks exit status.

I like the fact that "git am" is quite strict about what it accepts
and applies, so I'll typically wrap things around that core operation. 

Here is a completely trivial example of using git am:

cat `grep ^[a-zA-Z0-9] series` > /tmp/foo.mbox
git am /tmp/foo.mbox

Paul.

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Madovsky]

Paul,

is there any link of how to patch the kernel ?
seems like it's not the same as 2.6.33-7..

thank you

Franck

----- Original Message ----- 
From: "Paul Gortmaker" <paul.gortmaker@windriver.com>
To: <linux-rt-users@vger.kernel.org>
Cc: <linux-kernel@vger.kernel.org>; <lwn@lwn.net>
Sent: Friday, March 04, 2011 5:24 PM
Subject: [ANNOUNCE] RT for v2.6.34.8 now available.


> As a value add to the 2.6.34 long term release, I'm happy to also
> announce the availability of 2.6.34-RT.
>
> You can find it in the v2.6.34-rt branch at:
>
>  git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
>
> as a repository of patches.  The v2.6.34-rt branch contains the latest
> RT patches against the latest v2.6.34.8 kernel release. (The master
> branch currently stops at v2.6.34 flat, i.e. 2.6.34.0 so to speak.)
>
> I've also created over 400 known working RT enabled bisection points
> between 33 and 34 that you can make use of for testing.  The details
> on how/why those exist follows - read on if it is of interest to you.
>
> Note that all credit of the thinking and engineering of the RT stuff
> lies with the original patch authors -- to be clear, I'm just doing
> the grunt work of a carry forward here.
>
> Have fun with it, maybe drop me a note if you find it useful etc.
>
> Paul.
>
>
> The carry forward process:
> ---------------------------
>
> This warrants some notes, since (a) it has strong consequence on how
> you can diagnose/bisect problems that were not in 33-rt but may exist
> in 34-rt and (b) we've recently seen people who wonder how to do a
> similar task and I think they *really* underestimate the complexities
> and sheer effort of trying to do any sort of RT carry-forward.
>
> For those who track these things, you will know that the most recent
> release of RT was based on v2.6.33, and in turn, it was created by
> merging forward the mainline kernel.org v2.6.33 content into the older
> v2.6.31 RT tree.  There was no rebase.
>
> With that detail, some of you are probably now going "Aha, now I know
> why he created that merge-free v2.6.33 based RT tree some weeks ago."
> (see: http://www.spinics.net/lists/linux-rt-users/msg06310.html )
>
> Step one was that I needed the RT commits rebased against 33 (and not 31)
> as my starting point for rebasing RT onto v2.6.34 -- see the above
> link for details on the significant work involved with that.
>
> There are roughly 500 RT patches, and literally 10,000+ commits between
> v2.6.33 and v2.6.34.  So if one was to move things ahead all in one go,
> there can be roughly 5 million things that can go wrong.  Maybe some sharp
> person can move those ahead all in one shot, and then figure out the
> resulting inevitable runtime breakage, but that isn't me.  After all, a
> man has to know his limitations, and I'm pretty much just a patch monkey.
>
> Knowing that the RT patches applied (and worked) on base X, but fail to
> apply or fail to work on X+N (where N is small) can be very empowering
> to a dumb person like me.  It reduces the problem space *immensely* and
> it lets you focus on what changed and caused the problem/conflict.  With
> that in mind, I saw no choice but to undertake what would have been the
> equivalent of recreating history, had a continuous integration between
> the two streams taken place during the 33 --> 34 dev cycle.
>
> What this means in concrete terms, is incrementally applying, updating,
> and testing the 500-odd RT patches at successive points along the 10000
> commits found in the 33 --> 34 development cycle.  But assuming you
> agree with that logic, what does a person choose/use as increments?
>
> At a 1st guess, a person might suggest choosing the rcN tags (where we
> have N=1,2,... 7) as places for these incremental tests.  The problem
> is that the granularity is too coarse -- just for N=1 means you have
> already jumped over 6000 commits ahead, and so you are still facing
> being lost for the same reason 10000 commits left you lost before.
>
> At the opposite end of the scale is a brute force approach, where a
> person tries applying/updating/testing the RT patch on *each* of the
> 10,000 development commits. Sounds like it might work, but no.  The
> problem here is that the kernel development history is non-linear
> in time.  See the git FAQ on why bisects can drag you back in time, if
> you've never personally encountered this yourself.  A lot of people
> are surprised by this when they first encounter it.
>
> My approach was to adopt a semi-brute force approach.  During the dev
> cycle of any kernel, Linus merges the content from various subsystem
> maintainers.  Each of these merge points represents a point where
> you are guaranteed to not be "rewound in time", should you choose it
> as a bisect point.  Using git, these points can easily be identified.
>
> There happens to be ~400 of them, so the 10,000 development commits get
> "digested" at an average rate of a humanly manageable number of about 25
> commits each -- something that a stupid person like me has a chance to
> be able to diagnose/debug without going insane.  You will find that there
> is an un-annotated tag for each of these merges in the patch repository.
> You really should use these for bisecting your own problems/issues.
>
> On each of these, I've done a patch test, a compile test (x86, x86-64,
> ppc, arm) and a boot test (x86, x86-64, ppc-SMP) to ensure that I've not
> done a colossal screw-up.  I've probably still screwed *something* up,
> but at least I've ensured some level of continuing sanity with these tests
> being done across these integration points.
> --
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" 
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html 

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Madovsky]

Hi Paul,

I get same(?) error on Fedora10 64bits

  CHK     include/linux/version.h
  CHK     include/generated/utsrelease.h
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h
  CC      init/do_mounts.o
In file included from include/linux/suspend.h:8,
                 from init/do_mounts.c:6:
include/linux/mm.h: In function âpte_lock_initâ:
include/linux/mm.h:1125: error: implicit declaration of function âkmallocâ
include/linux/mm.h:1125: warning: assignment makes pointer from integer 
without 
a cast
include/linux/mm.h: In function âpte_lock_deinitâ:
include/linux/mm.h:1137: error: implicit declaration of function âkfreeâ
In file included from include/linux/security.h:36,
                 from init/do_mounts.c:8:
include/linux/slab.h: At top level:
include/linux/slab.h:144: warning: conflicting types for âkfreeâ
include/linux/mm.h:1137: warning: previous implicit declaration of âkfreeâ 
was h 
ere
In file included from include/linux/slab.h:172,
                 from include/linux/security.h:36,
                 from init/do_mounts.c:8:
include/linux/slab_def.h:128: error: conflicting types for âkmallocâ
include/linux/mm.h:1125: error: previous implicit declaration of âkmallocâ 
was h 
ere
make[1]: *** [init/do_mounts.o] Error 1
make: *** [init] Error 2


thanks

Franck

----- Original Message ----- 
From: "Paul Gortmaker" <paul.gortmaker@windriver.com>
To: <linux-rt-users@vger.kernel.org>
Cc: <linux-kernel@vger.kernel.org>; <lwn@lwn.net>
Sent: Friday, March 04, 2011 5:24 PM
Subject: [ANNOUNCE] RT for v2.6.34.8 now available.


> As a value add to the 2.6.34 long term release, I'm happy to also
> announce the availability of 2.6.34-RT.
>
> You can find it in the v2.6.34-rt branch at:
>
>  git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
>
> as a repository of patches.  The v2.6.34-rt branch contains the latest
> RT patches against the latest v2.6.34.8 kernel release. (The master
> branch currently stops at v2.6.34 flat, i.e. 2.6.34.0 so to speak.)
>
> I've also created over 400 known working RT enabled bisection points
> between 33 and 34 that you can make use of for testing.  The details
> on how/why those exist follows - read on if it is of interest to you.
>
> Note that all credit of the thinking and engineering of the RT stuff
> lies with the original patch authors -- to be clear, I'm just doing
> the grunt work of a carry forward here.
>
> Have fun with it, maybe drop me a note if you find it useful etc.
>
> Paul.
>
>
> The carry forward process:
> ---------------------------
>
> This warrants some notes, since (a) it has strong consequence on how
> you can diagnose/bisect problems that were not in 33-rt but may exist
> in 34-rt and (b) we've recently seen people who wonder how to do a
> similar task and I think they *really* underestimate the complexities
> and sheer effort of trying to do any sort of RT carry-forward.
>
> For those who track these things, you will know that the most recent
> release of RT was based on v2.6.33, and in turn, it was created by
> merging forward the mainline kernel.org v2.6.33 content into the older
> v2.6.31 RT tree.  There was no rebase.
>
> With that detail, some of you are probably now going "Aha, now I know
> why he created that merge-free v2.6.33 based RT tree some weeks ago."
> (see: http://www.spinics.net/lists/linux-rt-users/msg06310.html )
>
> Step one was that I needed the RT commits rebased against 33 (and not 31)
> as my starting point for rebasing RT onto v2.6.34 -- see the above
> link for details on the significant work involved with that.
>
> There are roughly 500 RT patches, and literally 10,000+ commits between
> v2.6.33 and v2.6.34.  So if one was to move things ahead all in one go,
> there can be roughly 5 million things that can go wrong.  Maybe some sharp
> person can move those ahead all in one shot, and then figure out the
> resulting inevitable runtime breakage, but that isn't me.  After all, a
> man has to know his limitations, and I'm pretty much just a patch monkey.
>
> Knowing that the RT patches applied (and worked) on base X, but fail to
> apply or fail to work on X+N (where N is small) can be very empowering
> to a dumb person like me.  It reduces the problem space *immensely* and
> it lets you focus on what changed and caused the problem/conflict.  With
> that in mind, I saw no choice but to undertake what would have been the
> equivalent of recreating history, had a continuous integration between
> the two streams taken place during the 33 --> 34 dev cycle.
>
> What this means in concrete terms, is incrementally applying, updating,
> and testing the 500-odd RT patches at successive points along the 10000
> commits found in the 33 --> 34 development cycle.  But assuming you
> agree with that logic, what does a person choose/use as increments?
>
> At a 1st guess, a person might suggest choosing the rcN tags (where we
> have N=1,2,... 7) as places for these incremental tests.  The problem
> is that the granularity is too coarse -- just for N=1 means you have
> already jumped over 6000 commits ahead, and so you are still facing
> being lost for the same reason 10000 commits left you lost before.
>
> At the opposite end of the scale is a brute force approach, where a
> person tries applying/updating/testing the RT patch on *each* of the
> 10,000 development commits. Sounds like it might work, but no.  The
> problem here is that the kernel development history is non-linear
> in time.  See the git FAQ on why bisects can drag you back in time, if
> you've never personally encountered this yourself.  A lot of people
> are surprised by this when they first encounter it.
>
> My approach was to adopt a semi-brute force approach.  During the dev
> cycle of any kernel, Linus merges the content from various subsystem
> maintainers.  Each of these merge points represents a point where
> you are guaranteed to not be "rewound in time", should you choose it
> as a bisect point.  Using git, these points can easily be identified.
>
> There happens to be ~400 of them, so the 10,000 development commits get
> "digested" at an average rate of a humanly manageable number of about 25
> commits each -- something that a stupid person like me has a chance to
> be able to diagnose/debug without going insane.  You will find that there
> is an un-annotated tag for each of these merges in the patch repository.
> You really should use these for bisecting your own problems/issues.
>
> On each of these, I've done a patch test, a compile test (x86, x86-64,
> ppc, arm) and a boot test (x86, x86-64, ppc-SMP) to ensure that I've not
> done a colossal screw-up.  I've probably still screwed *something* up,
> but at least I've ensured some level of continuing sanity with these tests
> being done across these integration points.
> --
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" 
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html 

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Niccolò Belli]

What about 2.6.37-rt? Is there any news?

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Fernando Lopez-Lezcano]

On 03/04/2011 02:24 PM, Paul Gortmaker wrote:
> As a value add to the 2.6.34 long term release, I'm happy to also
> announce the availability of 2.6.34-RT.
>
> You can find it in the v2.6.34-rt branch at:
>
>    git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git

Great! Many thanks... Actually, impressive work.

I'm trying to integrate/build it into my Fedora based rt packages for 
Planet CCRMA. Warning: I'm doing this on top of the 2.6.34.8-68 fc13 
koji build, so rt is patched on top of the additional Fedora patches for 
that build (with a few trivial tweaks to patch cleanly). So I don't know 
if the following issues are due to that - I will investigate:

I found a couple of extra argument cases for __netif_tx_lock:

--- a/drivers/net/qlcnic/qlcnic_main.c	2010-05-16 17:17:36.000000000 -0400
+++ b/drivers/net/qlcnic/qlcnic_main.c	2011-03-06 20:08:16.000000000 -0500
@@ -1881,7 +1881,7 @@
  		smp_mb();

  		if (netif_queue_stopped(netdev) && netif_carrier_ok(netdev)) {
-			__netif_tx_lock(tx_ring->txq, smp_processor_id());
+			__netif_tx_lock(tx_ring->txq);
  			if (qlcnic_tx_avail(tx_ring) > TX_STOP_THRESH) {
  				netif_wake_queue(netdev);
  				adapter->tx_timeo_cnt = 0;
--- a/drivers/net/bnx2x_main.c	2010-05-16 17:17:36.000000000 -0400
+++ b/drivers/net/bnx2x_main.c	2011-03-06 19:58:45.000000000 -0500
@@ -976,7 +976,7 @@
  		 * stops the queue
  		 */

-		__netif_tx_lock(txq, smp_processor_id());
+		__netif_tx_lock(txq);

  		if ((netif_tx_queue_stopped(txq)) &&
  		    (bp->state == BNX2X_STATE_OPEN) &&
----

a missing include (incomplete type errors for semaphores):

--- a/drivers/net/mlx4/mlx4.h	2010-05-16 17:17:36.000000000 -0400
+++ b/drivers/net/mlx4/mlx4.h	2011-03-06 20:07:05.000000000 -0500
@@ -37,6 +37,7 @@
  #ifndef MLX4_H
  #define MLX4_H

+#include <linux/semaphore.h>
  #include <linux/mutex.h>
  #include <linux/radix-tree.h>
  #include <linux/timer.h>
----

and this (found compilation fix here:
http://groups.google.com/group/linux.kernel/browse_thread/thread/c4f0fd1ca9298c45/18316184209e6e84?show_docid=18316184209e6e84&fwc=1):

--- a/drivers/net/wireless/ath/ar9170/usb.c	2011-03-06 
19:13:12.000000000 -0500
+++ b/drivers/net/wireless/ath/ar9170/usb.c	2011-03-06 
20:13:06.000000000 -0500
@@ -741,10 +741,10 @@

  	/* unbind anything failed */
  	if (parent)
-		down(&parent->sem);
+		device_lock(parent);
  	device_release_driver(&aru->udev->dev);
  	if (parent)
-		up(&parent->sem);
+		device_unlock(parent);

  	usb_put_dev(aru->udev);
  }
----

I'll let you know how it goes (light testing probably tomorrow, still 
building).

-- Fernando "patiently waiting for 2.6.37-rtx"
:-)

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Paul Gortmaker]

On Mon, Mar 7, 2011 at 1:51 AM, Fernando Lopez-Lezcano
<nando@ccrma.stanford.edu> wrote:
> On 03/04/2011 02:24 PM, Paul Gortmaker wrote:
>>
>> As a value add to the 2.6.34 long term release, I'm happy to also
>> announce the availability of 2.6.34-RT.
>>
>> You can find it in the v2.6.34-rt branch at:
>>
>>   git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
>
> Great! Many thanks... Actually, impressive work.
>
> I'm trying to integrate/build it into my Fedora based rt packages for Planet
> CCRMA. Warning: I'm doing this on top of the 2.6.34.8-68 fc13 koji build, so
> rt is patched on top of the additional Fedora patches for that build (with a
> few trivial tweaks to patch cleanly). So I don't know if the following
> issues are due to that - I will investigate:

Thanks - I'm not surprised to see some fallout in new/updated drivers,
since they change so much per release and I didn't attempt to build every
driver under the sun - my core focus was on getting it stable with the main
defconfigs I could easily test.   I'll have a look and integrate accordingly.

P.

>
> I found a couple of extra argument cases for __netif_tx_lock:
>
> --- a/drivers/net/qlcnic/qlcnic_main.c  2010-05-16 17:17:36.000000000 -0400
> +++ b/drivers/net/qlcnic/qlcnic_main.c  2011-03-06 20:08:16.000000000 -0500
> @@ -1881,7 +1881,7 @@
>                smp_mb();
>
>                if (netif_queue_stopped(netdev) && netif_carrier_ok(netdev))
> {
> -                       __netif_tx_lock(tx_ring->txq, smp_processor_id());
> +                       __netif_tx_lock(tx_ring->txq);
>                        if (qlcnic_tx_avail(tx_ring) > TX_STOP_THRESH) {
>                                netif_wake_queue(netdev);
>                                adapter->tx_timeo_cnt = 0;
> --- a/drivers/net/bnx2x_main.c  2010-05-16 17:17:36.000000000 -0400
> +++ b/drivers/net/bnx2x_main.c  2011-03-06 19:58:45.000000000 -0500
> @@ -976,7 +976,7 @@
>                 * stops the queue
>                 */
>
> -               __netif_tx_lock(txq, smp_processor_id());
> +               __netif_tx_lock(txq);
>
>                if ((netif_tx_queue_stopped(txq)) &&
>                    (bp->state == BNX2X_STATE_OPEN) &&
> ----
>
> a missing include (incomplete type errors for semaphores):
>
> --- a/drivers/net/mlx4/mlx4.h   2010-05-16 17:17:36.000000000 -0400
> +++ b/drivers/net/mlx4/mlx4.h   2011-03-06 20:07:05.000000000 -0500
> @@ -37,6 +37,7 @@
>  #ifndef MLX4_H
>  #define MLX4_H
>
> +#include <linux/semaphore.h>
>  #include <linux/mutex.h>
>  #include <linux/radix-tree.h>
>  #include <linux/timer.h>
> ----
>
> and this (found compilation fix here:
> http://groups.google.com/group/linux.kernel/browse_thread/thread/c4f0fd1ca9298c45/18316184209e6e84?show_docid=18316184209e6e84&fwc=1):
>
> --- a/drivers/net/wireless/ath/ar9170/usb.c     2011-03-06
> 19:13:12.000000000 -0500
> +++ b/drivers/net/wireless/ath/ar9170/usb.c     2011-03-06
> 20:13:06.000000000 -0500
> @@ -741,10 +741,10 @@
>
>        /* unbind anything failed */
>        if (parent)
> -               down(&parent->sem);
> +               device_lock(parent);
>        device_release_driver(&aru->udev->dev);
>        if (parent)
> -               up(&parent->sem);
> +               device_unlock(parent);
>
>        usb_put_dev(aru->udev);
>  }
> ----
>
> I'll let you know how it goes (light testing probably tomorrow, still
> building).
>
> -- Fernando "patiently waiting for 2.6.37-rtx"
> :-)
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Nathan Grennan]

On 03/07/2011 06:44 AM, Paul Gortmaker wrote:
> On Mon, Mar 7, 2011 at 1:51 AM, Fernando Lopez-Lezcano
> <nando@ccrma.stanford.edu>  wrote:
>> On 03/04/2011 02:24 PM, Paul Gortmaker wrote:
>>> As a value add to the 2.6.34 long term release, I'm happy to also
>>> announce the availability of 2.6.34-RT.
>>>
>>> You can find it in the v2.6.34-rt branch at:
>>>
>>>    git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
>> Great! Many thanks... Actually, impressive work.
>>
>> I'm trying to integrate/build it into my Fedora based rt packages for Planet
>> CCRMA. Warning: I'm doing this on top of the 2.6.34.8-68 fc13 koji build, so
>> rt is patched on top of the additional Fedora patches for that build (with a
>> few trivial tweaks to patch cleanly). So I don't know if the following
>> issues are due to that - I will investigate:
> Thanks - I'm not surprised to see some fallout in new/updated drivers,
> since they change so much per release and I didn't attempt to build every
> driver under the sun - my core focus was on getting it stable with the main
> defconfigs I could easily test.   I'll have a look and integrate accordingly.
>
> P.

drivers/net/bnx2x_main.c
drivers/staging/comedi/drivers/quatech_daqp_cs.c

These drivers didn't compile for me. bnx2x_main was an easy fix, but I 
didn't bother with quatech_daqp_cs.

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Fernando Lopez-Lezcano]

On 03/06/2011 10:51 PM, Fernando Lopez-Lezcano wrote:
> On 03/04/2011 02:24 PM, Paul Gortmaker wrote:
>> As a value add to the 2.6.34 long term release, I'm happy to also
>> announce the availability of 2.6.34-RT.
>>
>> You can find it in the v2.6.34-rt branch at:
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
>
> Great! Many thanks... Actually, impressive work.
>
> I'm trying to integrate/build it into my Fedora based rt packages for
> Planet CCRMA. Warning: I'm doing this on top of the 2.6.34.8-68 fc13
> koji build, so rt is patched on top of the additional Fedora patches for
> that build (with a few trivial tweaks to patch cleanly). So I don't know
> if the following issues are due to that...

Caveat above still applies...
So far mixed success:

- Fedora 14 desktop w/radeon graphics: OK
- Lenovo laptop, Fedora 13, Nvidia binary driver (barf!): OK

- EeePC netbook, Fedora 13, Intel Mobile 945GME graphics: X does not start
- Lenovo laptop, fedora 14, Intel 9xx graphics (at home): X does not start
These last two share Intel graphics and the problem seems to be the 
same, something to do with dri. The standard Fedora kernel works fine so 
it must be some rt interaction (perhaps with the added Fedora patches).

But it is a good start!
-- Fernando

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Fernando Lopez-Lezcano]

On 03/07/2011 12:00 PM, Fernando Lopez-Lezcano wrote:
> On 03/06/2011 10:51 PM, Fernando Lopez-Lezcano wrote:
>> On 03/04/2011 02:24 PM, Paul Gortmaker wrote:
>>> As a value add to the 2.6.34 long term release, I'm happy to also
>>> announce the availability of 2.6.34-RT.
>>>
>>> You can find it in the v2.6.34-rt branch at:
>>>
>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
>>
>> Great! Many thanks... Actually, impressive work.
>>
>> I'm trying to integrate/build it into my Fedora based rt packages for
>> Planet CCRMA. Warning: I'm doing this on top of the 2.6.34.8-68 fc13
>> koji build, so rt is patched on top of the additional Fedora patches for
>> that build (with a few trivial tweaks to patch cleanly). So I don't know
>> if the following issues are due to that...
>
> Caveat above still applies...
...
> - EeePC netbook, Fedora 13, Intel Mobile 945GME graphics: X does not start
> - Lenovo laptop, fedora 14, Intel 9xx graphics (at home): X does not start
> These last two share Intel graphics and the problem seems to be the
> same, something to do with dri.

This is what I'm getting (dmesg|grep i915):

i915 0000:00:02.0 PCI INT A -> GSI 16 (level, low) -> IRQ 16
i915 0000:00:02.0 setting latency timer to 64
i915 0000:00:02.0 PCI INT A disabled
i915: probe of 0000:00:02.0 failed with error -5

Same kernel w/o rt...:

i915 0000:00:02.0 PCI INT A -> GSI 16 (level, low) -> IRQ 16
i915 0000:00:02.0 setting latency timer to 64
[drm] Initialized i915 1.6.0 20080730 for 0000:00:02.0 on minor 0

So, some problem with irq processing?
Where should I go look?
-- Fernando

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Paul Gortmaker]

On 11-03-07 03:41 PM, Fernando Lopez-Lezcano wrote:
> On 03/07/2011 12:00 PM, Fernando Lopez-Lezcano wrote:
>> On 03/06/2011 10:51 PM, Fernando Lopez-Lezcano wrote:
>>> On 03/04/2011 02:24 PM, Paul Gortmaker wrote:
>>>> As a value add to the 2.6.34 long term release, I'm happy to also
>>>> announce the availability of 2.6.34-RT.
>>>>
>>>> You can find it in the v2.6.34-rt branch at:
>>>>
>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
>>>
>>> Great! Many thanks... Actually, impressive work.
>>>
>>> I'm trying to integrate/build it into my Fedora based rt packages for
>>> Planet CCRMA. Warning: I'm doing this on top of the 2.6.34.8-68 fc13
>>> koji build, so rt is patched on top of the additional Fedora patches for
>>> that build (with a few trivial tweaks to patch cleanly). So I don't know
>>> if the following issues are due to that...
>>
>> Caveat above still applies...
> ...
>> - EeePC netbook, Fedora 13, Intel Mobile 945GME graphics: X does not start
>> - Lenovo laptop, fedora 14, Intel 9xx graphics (at home): X does not start
>> These last two share Intel graphics and the problem seems to be the
>> same, something to do with dri.
> 
> This is what I'm getting (dmesg|grep i915):
> 
> i915 0000:00:02.0 PCI INT A -> GSI 16 (level, low) -> IRQ 16
> i915 0000:00:02.0 setting latency timer to 64
> i915 0000:00:02.0 PCI INT A disabled
> i915: probe of 0000:00:02.0 failed with error -5
> 
> Same kernel w/o rt...:
> 
> i915 0000:00:02.0 PCI INT A -> GSI 16 (level, low) -> IRQ 16
> i915 0000:00:02.0 setting latency timer to 64
> [drm] Initialized i915 1.6.0 20080730 for 0000:00:02.0 on minor 0
> 
> So, some problem with irq processing?
> Where should I go look?

Try v2.6.33-rt -- see if it is a new regression, or was a problem that
already existed and has just been carried forward.

P.

> -- Fernando

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Fernando Lopez-Lezcano]

On 03/07/2011 12:52 PM, Paul Gortmaker wrote:
> On 11-03-07 03:41 PM, Fernando Lopez-Lezcano wrote:
>> On 03/07/2011 12:00 PM, Fernando Lopez-Lezcano wrote:
>>> On 03/06/2011 10:51 PM, Fernando Lopez-Lezcano wrote:
>>>> On 03/04/2011 02:24 PM, Paul Gortmaker wrote:
>>>>> As a value add to the 2.6.34 long term release, I'm happy to also
>>>>> announce the availability of 2.6.34-RT.
>>>>>
>>>>> You can find it in the v2.6.34-rt branch at:
>>>>>
>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
>>>>
>>>> Great! Many thanks... Actually, impressive work.
>>>>
>>>> I'm trying to integrate/build it into my Fedora based rt packages for
>>>> Planet CCRMA. Warning: I'm doing this on top of the 2.6.34.8-68 fc13
>>>> koji build, so rt is patched on top of the additional Fedora patches for
>>>> that build (with a few trivial tweaks to patch cleanly). So I don't know
>>>> if the following issues are due to that...
>>>
>>> Caveat above still applies...
>> ...
>>> - EeePC netbook, Fedora 13, Intel Mobile 945GME graphics: X does not start
>>> - Lenovo laptop, fedora 14, Intel 9xx graphics (at home): X does not start
>>> These last two share Intel graphics and the problem seems to be the
>>> same, something to do with dri.
>>
>> This is what I'm getting (dmesg|grep i915):
>>
>> i915 0000:00:02.0 PCI INT A ->  GSI 16 (level, low) ->  IRQ 16
>> i915 0000:00:02.0 setting latency timer to 64
>> i915 0000:00:02.0 PCI INT A disabled
>> i915: probe of 0000:00:02.0 failed with error -5
>>
>> Same kernel w/o rt...:
>>
>> i915 0000:00:02.0 PCI INT A ->  GSI 16 (level, low) ->  IRQ 16
>> i915 0000:00:02.0 setting latency timer to 64
>> [drm] Initialized i915 1.6.0 20080730 for 0000:00:02.0 on minor 0
>>
>> So, some problem with irq processing?
>> Where should I go look?
>
> Try v2.6.33-rt -- see if it is a new regression, or was a problem that
> already existed and has just been carried forward.

2.6.33.x-rt30 works on all of the above so it seems this is new... (yuck :-)
(just in case, this is not pristine 2.6.34.8 + rt git)

-- Fernando

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Paul Gortmaker]

On 11-03-07 03:54 PM, Fernando Lopez-Lezcano wrote:
> On 03/07/2011 12:52 PM, Paul Gortmaker wrote:
>> On 11-03-07 03:41 PM, Fernando Lopez-Lezcano wrote:

[...]

>>> ...
>>>> - EeePC netbook, Fedora 13, Intel Mobile 945GME graphics: X does not start
>>>> - Lenovo laptop, fedora 14, Intel 9xx graphics (at home): X does not start
>>>> These last two share Intel graphics and the problem seems to be the
>>>> same, something to do with dri.
>>>
>>> This is what I'm getting (dmesg|grep i915):
>>>
>>> i915 0000:00:02.0 PCI INT A ->  GSI 16 (level, low) ->  IRQ 16
>>> i915 0000:00:02.0 setting latency timer to 64
>>> i915 0000:00:02.0 PCI INT A disabled
>>> i915: probe of 0000:00:02.0 failed with error -5
>>>
>>> Same kernel w/o rt...:
>>>
>>> i915 0000:00:02.0 PCI INT A ->  GSI 16 (level, low) ->  IRQ 16
>>> i915 0000:00:02.0 setting latency timer to 64
>>> [drm] Initialized i915 1.6.0 20080730 for 0000:00:02.0 on minor 0
>>>
>>> So, some problem with irq processing?
>>> Where should I go look?
>>
>> Try v2.6.33-rt -- see if it is a new regression, or was a problem that
>> already existed and has just been carried forward.
> 
> 2.6.33.x-rt30 works on all of the above so it seems this is new... (yuck :-)
> (just in case, this is not pristine 2.6.34.8 + rt git)

I'm pretty sure I know what this is.  If you cherry pick the two
commits from tip, namely:

	e89fab2bf44afccaa48ffb8ae33c03f4f3dcd208 and
	cd4ac9b7a1c80cfcd9b8e76a1f6855a0329d982f

then I'll bet your i915 gets detected OK.  I was forced to
drop these because they allow ATOMIC_IOMAP to be selected
for RT & HIGHMEM, which causes sporadic bootup Oops (approx
one in 10) for intel ata_piix.  (This problem isn't new; it
is present in v2.6.33.7.2-rt30 by the way.)

But w/o ATOMIC_IOMAP, people on 32bit stuff end up trying to
grab a big chunk of address space that they can't afford, and
so the driver init fails.

The proper solution needs more thought, but in the meantime I'll
bet the above makes your i915 work again.

Related info to this can be found here:

http://lkml.indiana.edu/hypermail/linux/kernel/0811.0/00411.html

Tip repo, if you don't have it, is here:

git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git

Paul.


> 
> -- Fernando

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Fernando Lopez-Lezcano]

On 03/07/2011 04:52 PM, Paul Gortmaker wrote:
> On 11-03-07 03:54 PM, Fernando Lopez-Lezcano wrote:
>> On 03/07/2011 12:52 PM, Paul Gortmaker wrote:
>>> On 11-03-07 03:41 PM, Fernando Lopez-Lezcano wrote:
>
> [...]
>
>>>> ...
>>>>> - EeePC netbook, Fedora 13, Intel Mobile 945GME graphics: X does not start
>>>>> - Lenovo laptop, fedora 14, Intel 9xx graphics (at home): X does not start
>>>>> These last two share Intel graphics and the problem seems to be the
>>>>> same, something to do with dri.
>>>>
>>>> This is what I'm getting (dmesg|grep i915):
>>>>
>>>> i915 0000:00:02.0 PCI INT A ->   GSI 16 (level, low) ->   IRQ 16
>>>> i915 0000:00:02.0 setting latency timer to 64
>>>> i915 0000:00:02.0 PCI INT A disabled
>>>> i915: probe of 0000:00:02.0 failed with error -5
>>>>
>>>> Same kernel w/o rt...:
>>>>
>>>> i915 0000:00:02.0 PCI INT A ->   GSI 16 (level, low) ->   IRQ 16
>>>> i915 0000:00:02.0 setting latency timer to 64
>>>> [drm] Initialized i915 1.6.0 20080730 for 0000:00:02.0 on minor 0
>>>>
>>>> So, some problem with irq processing?
>>>> Where should I go look?
>>>
>>> Try v2.6.33-rt -- see if it is a new regression, or was a problem that
>>> already existed and has just been carried forward.
>>
>> 2.6.33.x-rt30 works on all of the above so it seems this is new... (yuck :-)
>> (just in case, this is not pristine 2.6.34.8 + rt git)
>
> I'm pretty sure I know what this is.  If you cherry pick the two
> commits from tip, namely:
>
> 	e89fab2bf44afccaa48ffb8ae33c03f4f3dcd208 and
> 	cd4ac9b7a1c80cfcd9b8e76a1f6855a0329d982f
>
> then I'll bet your i915 gets detected OK.

It does, thank you very much! I tested on the EeePc only but I suspect 
the Lenovo will be happy as well (it is at home right now). I have not 
hit any other problems so far with 2.6.34.8 + rtx (0?)...

-- Fernando

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Arnd Bergmann]

On Friday 04 March 2011 23:24:15 Paul Gortmaker wrote:
> As a value add to the 2.6.34 long term release, I'm happy to also
> announce the availability of 2.6.34-RT.  
> 
> You can find it in the v2.6.34-rt branch at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
> 
> as a repository of patches.  The v2.6.34-rt branch contains the latest
> RT patches against the latest v2.6.34.8 kernel release. (The master
> branch currently stops at v2.6.34 flat, i.e. 2.6.34.0 so to speak.)
> 
> I've also created over 400 known working RT enabled bisection points
> between 33 and 34 that you can make use of for testing.  The details
> on how/why those exist follows - read on if it is of interest to you.

A great achievement, but I think your approach can be improved in
case you want to do it again for another kernel version.

The problem with your method is that you have more work that depends
on the amount of changes in upstream than the work that depends on
the size of the patch set.

Note that I'm not using the -rt patch set myself, but I have some
experience with managing other patch sets.

> For those who track these things, you will know that the most recent
> release of RT was based on v2.6.33, and in turn, it was created by
> merging forward the mainline kernel.org v2.6.33 content into the older
> v2.6.31 RT tree.  There was no rebase.

I think 2.6.31 was done the right way, with a number of topic branches
that got merged into one release branch. This made it fairly easy to
come out with a quick-and-dirty forward port to 2.6.33, but from that
point on, it became much harder to port it again.

> There are roughly 500 RT patches, and literally 10,000+ commits between
> v2.6.33 and v2.6.34.  So if one was to move things ahead all in one go,
> there can be roughly 5 million things that can go wrong.  Maybe some sharp
> person can move those ahead all in one shot, and then figure out the
> resulting inevitable runtime breakage, but that isn't me.  After all, a
> man has to know his limitations, and I'm pretty much just a patch monkey. 

Doing it all in one shot is not really possible, I fully agree. However,
it's absolutely possible to port a small self-contained series of
interdependent patches over 10000 or more other changesets. The trick
is to split the 500 patches into e.g. 20 branches of less than 50 patches,
where each branch by itself adds one aspect of the full tree and can
be built and run independently of the other branches.

This is a lot of upfront work, but less work to keep it running if you
maintain the branches after the initial release (unlike the 2.6.33-rt
tree).

I've played around with your tree a bit and it seems fairly easy
to split into multiple series, e.g. one for each arch, one for
drivers, mm, rtmutex, mm using git-rebase and some heuristics.
Making sure that each of those branches actually makes sense
by itself may be a different story.

When you have these branches, you can merge them all into one release
branch, similar to how linux-next works, and then just merge updates
from the branches. To port the entire tree to a new upstream, you
look at each branch individually and do a merge, a rebase, a rewrite
or whatever is appropriate for that branch. Finally, you merge the
new branches into another release.

	Arnd

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Paul Gortmaker]

On 11-03-07 03:01 PM, Arnd Bergmann wrote:
> On Friday 04 March 2011 23:24:15 Paul Gortmaker wrote:
>> As a value add to the 2.6.34 long term release, I'm happy to also
>> announce the availability of 2.6.34-RT.  
>>
>> You can find it in the v2.6.34-rt branch at:
>>
>>   git://git.kernel.org/pub/scm/linux/kernel/git/paulg/rt-patches.git
>>
>> as a repository of patches.  The v2.6.34-rt branch contains the latest
>> RT patches against the latest v2.6.34.8 kernel release. (The master
>> branch currently stops at v2.6.34 flat, i.e. 2.6.34.0 so to speak.)
>>
>> I've also created over 400 known working RT enabled bisection points
>> between 33 and 34 that you can make use of for testing.  The details
>> on how/why those exist follows - read on if it is of interest to you.
> 
> A great achievement, but I think your approach can be improved in
> case you want to do it again for another kernel version.
> 
> The problem with your method is that you have more work that depends
> on the amount of changes in upstream than the work that depends on
> the size of the patch set.

You might think so, but that really isn't the case at all.  The
work involved is dictated only by the upstream subset that have
an impact on the RT commits.   If there was a stream of a thousand
contiguous upstream commits that did not break patch application,
or the run time boot test, I'd not have to lift a finger.  Scripts
applied the commits and ensured a continuous sanity.   If I was to
do it again, I'd employ the exact same tactic in a heartbeat.

> 
> Note that I'm not using the -rt patch set myself, but I have some
> experience with managing other patch sets.
> 
>> For those who track these things, you will know that the most recent
>> release of RT was based on v2.6.33, and in turn, it was created by
>> merging forward the mainline kernel.org v2.6.33 content into the older
>> v2.6.31 RT tree.  There was no rebase.
> 
> I think 2.6.31 was done the right way, with a number of topic branches
> that got merged into one release branch. This made it fairly easy to
> come out with a quick-and-dirty forward port to 2.6.33, but from that
> point on, it became much harder to port it again.
> 
>> There are roughly 500 RT patches, and literally 10,000+ commits between
>> v2.6.33 and v2.6.34.  So if one was to move things ahead all in one go,
>> there can be roughly 5 million things that can go wrong.  Maybe some sharp
>> person can move those ahead all in one shot, and then figure out the
>> resulting inevitable runtime breakage, but that isn't me.  After all, a
>> man has to know his limitations, and I'm pretty much just a patch monkey. 
> 
> Doing it all in one shot is not really possible, I fully agree. However,
> it's absolutely possible to port a small self-contained series of
> interdependent patches over 10000 or more other changesets. The trick
> is to split the 500 patches into e.g. 20 branches of less than 50 patches,
> where each branch by itself adds one aspect of the full tree and can
> be built and run independently of the other branches.

This could make sense if they were real independent functionality
boundaries that happened to give you this nice uniform division.
But it is unlikely in practice.   And adding an artificial division
may not work if there are implicit dependencies between the patches
that a patch monkey like me might not see.  Another added cost is that
I would now have 20 branches to individually compile and boot test.
Maybe I could get away with more sparse testing, but maybe not.

In any case, I really don't see what you are trying to propose here
as being really all that fundamentally different -- you want to move
a small set of patches over a bigger set of patches.  So you can say
I'm moving a set of 20 odd patches, which were merged by Linus (and hence
already known to be treatable as a whole self contained unit) over a larger
set of some 500 RT patches.  Does it matter which group you call the
big set and which one the small set?

> 
> This is a lot of upfront work, but less work to keep it running if you
> maintain the branches after the initial release (unlike the 2.6.33-rt
> tree).
> 
> I've played around with your tree a bit and it seems fairly easy
> to split into multiple series, e.g. one for each arch, one for
> drivers, mm, rtmutex, mm using git-rebase and some heuristics.
> Making sure that each of those branches actually makes sense
> by itself may be a different story.
> 
> When you have these branches, you can merge them all into one release
> branch, similar to how linux-next works, and then just merge updates
> from the branches. To port the entire tree to a new upstream, you
> look at each branch individually and do a merge, a rebase, a rewrite
> or whatever is appropriate for that branch. Finally, you merge the
> new branches into another release.

If any knowledge is embedded in a merge commit, it makes it difficult to
extract a commit and deal with it in isolation, should it be something
that has a chance of being pushed upstream.  A merge commit makes sense
when there is value in capturing that patches X-->Y were developed and
tested on baseline Z.  On the other hand, there is value in showing instead
X'--->Y' which are meant to apply to a newer baseline Z'.  For the RT
case, I think the latter is more valuable (at least to me) so I'll be
sticking to a linear merge-free patchset on anything RT related.

Thanks for the comments though -- at least I know someone was reading
the process I used and thinking about it.

Paul.

> 
> 	Arnd

Re: [ANNOUNCE] RT for v2.6.34.8 now available.

[Arnd Bergmann]

On Monday 07 March 2011, Paul Gortmaker wrote:
> On 11-03-07 03:01 PM, Arnd Bergmann wrote:
>
> > The problem with your method is that you have more work that depends
> > on the amount of changes in upstream than the work that depends on
> > the size of the patch set.
> 
> You might think so, but that really isn't the case at all.  The
> work involved is dictated only by the upstream subset that have
> an impact on the RT commits.   If there was a stream of a thousand
> contiguous upstream commits that did not break patch application,
> or the run time boot test, I'd not have to lift a finger.  Scripts
> applied the commits and ensured a continuous sanity.   If I was to
> do it again, I'd employ the exact same tactic in a heartbeat.

Fair enough, everyone has their own ways of dealing with this, I just
wanted to make sure that you are aware that there are other options
that people have used successfully.

Obviously, the number of conflicts is about the same, independent
of the way you partition the work. 
 
> This could make sense if they were real independent functionality
> boundaries that happened to give you this nice uniform division.
> But it is unlikely in practice. 

You have obviously worked a lot with these patches, so you have
a better understanding than me of how interrelated the patches
actually are. From a brief look at the contents and from the fact
that there already were independent branches in the 2.6.31-rt
kernel, my guess was that it is very possible to do a reasonable
split. Someone would have to seriously try it in order to know
for sure.

> And adding an artificial division
> may not work if there are implicit dependencies between the patches
> that a patch monkey like me might not see.  Another added cost is that
> I would now have 20 branches to individually compile and boot test.
> Maybe I could get away with more sparse testing, but maybe not.
>
> In any case, I really don't see what you are trying to propose here
> as being really all that fundamentally different -- you want to move
> a small set of patches over a bigger set of patches.  So you can say
> I'm moving a set of 20 odd patches, which were merged by Linus (and hence
> already known to be treatable as a whole self contained unit) over a larger
> set of some 500 RT patches.  Does it matter which group you call the
> big set and which one the small set?

The important difference IMHO is that when you have topic branches,
you can partition the work into hard and easy tasks at your own
choice, rather than having to go through all the merges that someone
else did in the same order:

* Half the branches will have little or no conflicts with the new
  upstream version, so you immediately reduce the amount of dull
  work by moving the branch up to the new upstream.
* Some particularly complex branches will have patches from only
  one or two people who know that code really well, so you can ask
  them to rebase the branch.
* A branch that has complex conflicts may turn out to be relevant
  only for a small group of users, e.g. a specific architecture,
  so you can defer rebasing it until someone else is interested
  in doing the work.
* Some feature from one branch (e.g. BKL or raw spinlocks) may
  have been solved upstream in a different way. When the branch
  is self-contained, you can immediately drop the branch instead
  of having to look through the 500 patches for possible
  interactions with the new upstream code.

Another very significant difference is that having topic branches
makes it much easier to merge code upstream. There may be some
branches that are just in need of a bit care to be acceptable to
upstream, and other branches contain the code that would need
a full redesign from scratch before anyone could seriously submit
them. In both cases, isolating the changes in a branch helps you
make forward progress towards upstream.

	Arnd

2.6.33-7-rt30 and 2.6.38 comparison

[Madovsky]

Just tried to compare real time app
at work with 2.6.33.7-rt30 and 2.6.38 and
the result is 2.6.38 is very impressive...

Re: 2.6.33-7-rt30 and 2.6.38 comparison

[Ilyes Gouta]

Hi,

Would you please elaborate a bit more?

Thanks,

-Ilyes

On Fri, Mar 18, 2011 at 9:15 PM, Madovsky <infos@madovsky.org> wrote:
> Just tried to compare real time app
> at work with 2.6.33.7-rt30 and 2.6.38 and
> the result is 2.6.38 is very impressive...
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 2.6.33-7-rt30 and 2.6.38 comparison

[Niccolò Belli]

:/ @#!*##@
I thought they released 2.6.38-rt :'(

Re: 2.6.33-7-rt30 and 2.6.38 comparison

[Madovsky]

not a lot, I'm only a rt user, not linux developer.
so my compare is only a view of for example
latency in audio/video....


> ----- Original Message ----- 
> From: "Ilyes Gouta" <ilyes.gouta@gmail.com>
> To: "Madovsky" <infos@madovsky.org>
> Cc: <linux-rt-users@vger.kernel.org>
> Sent: Friday, March 18, 2011 4:16 PM
> Subject: Re: 2.6.33-7-rt30 and 2.6.38 comparison
>
>
>> Hi,
>>
>> Would you please elaborate a bit more?
>>
>> Thanks,
>>
>> -Ilyes
>>
>> On Fri, Mar 18, 2011 at 9:15 PM, Madovsky <infos@madovsky.org> wrote:
>>> Just tried to compare real time app
>>> at work with 2.6.33.7-rt30 and 2.6.38 and
>>> the result is 2.6.38 is very impressive...
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe 
>>> linux-rt-users" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" 
>> in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: 2.6.33-7-rt30 and 2.6.38 comparison

[Madovsky]

I compiled 2.6.38 with exactly the same
config so I made different audio/video task
and very stable with short latency


----- Original Message ----- 
From: "Madovsky" <infos@madovsky.org>
To: "linux-rt-users" <linux-rt-users@vger.kernel.org>
Sent: Friday, March 18, 2011 4:28 PM
Subject: Re: 2.6.33-7-rt30 and 2.6.38 comparison


> not a lot, I'm only a rt user, not linux developer.
> so my compare is only a view of for example
> latency in audio/video....
>
>
>> ----- Original Message ----- 
>> From: "Ilyes Gouta" <ilyes.gouta@gmail.com>
>> To: "Madovsky" <infos@madovsky.org>
>> Cc: <linux-rt-users@vger.kernel.org>
>> Sent: Friday, March 18, 2011 4:16 PM
>> Subject: Re: 2.6.33-7-rt30 and 2.6.38 comparison
>>
>>
>>> Hi,
>>>
>>> Would you please elaborate a bit more?
>>>
>>> Thanks,
>>>
>>> -Ilyes
>>>
>>> On Fri, Mar 18, 2011 at 9:15 PM, Madovsky <infos@madovsky.org> wrote:
>>>> Just tried to compare real time app
>>>> at work with 2.6.33.7-rt30 and 2.6.38 and
>>>> the result is 2.6.38 is very impressive...
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe 
>>>> linux-rt-users" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe 
>>> linux-rt-users" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 

RE: 2.6.33-7-rt30 and 2.6.38 comparison

[Victor henri]

Hi

I have observed similar behavior with newer non-RT kernels but there were something strange to me ; 

Same results (very few X-runs, if any) with :

  
 -  on one hand, on an Asus laptop with a Core i3  : latency 2.3 msec 
with the 2.6.33.7.2-rt kernel and 23 msec with a 2.6.37 preempt kernel;

   
 -  on the other hand, on a Lenovo Thinkpad Core i5 :  latency 2.3 msec 
with the 2.6.33.7.2-rt kernel and 2.3 msec (yes the same) with a 2.6.37 
preempt kernel !!

Of course the Lenovo Thinkpad is a very good laptop, but how come that there is such a big difference? 

So, I have observed the same quality of performance as Madovsky but not 
on every machine I've used... I don't have a clue why....

Victor

----------------------------------------
> From: infos@madovsky.org
> To: linux-rt-users@vger.kernel.org
> Subject: Re: 2.6.33-7-rt30 and 2.6.38 comparison
> Date: Fri, 18 Mar 2011 16:34:51 -0400
>
> I compiled 2.6.38 with exactly the same
> config so I made different audio/video task
> and very stable with short latency
>
>
> ----- Original Message -----
> From: "Madovsky" 
> To: "linux-rt-users" 
> Sent: Friday, March 18, 2011 4:28 PM
> Subject: Re: 2.6.33-7-rt30 and 2.6.38 comparison
>
>
> > not a lot, I'm only a rt user, not linux developer.
> > so my compare is only a view of for example
> > latency in audio/video....
> >
> >
> >> ----- Original Message -----
> >> From: "Ilyes Gouta" 
> >> To: "Madovsky" 
> >> Cc: 
> >> Sent: Friday, March 18, 2011 4:16 PM
> >> Subject: Re: 2.6.33-7-rt30 and 2.6.38 comparison
> >>
> >>
> >>> Hi,
> >>>
> >>> Would you please elaborate a bit more?
> >>>
> >>> Thanks,
> >>>
> >>> -Ilyes
> >>>
> >>> On Fri, Mar 18, 2011 at 9:15 PM, Madovsky  wrote:
> >>>> Just tried to compare real time app
> >>>> at work with 2.6.33.7-rt30 and 2.6.38 and
> >>>> the result is 2.6.38 is very impressive...
> >>>> --
> >>>> To unsubscribe from this list: send the line "unsubscribe
> >>>> linux-rt-users" in
> >>>> the body of a message to majordomo@vger.kernel.org
> >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> >>>>
> >>> --
> >>> To unsubscribe from this list: send the line "unsubscribe
> >>> linux-rt-users" in
> >>> the body of a message to majordomo@vger.kernel.org
> >>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> >>
> >
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
 		 	   		  --
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 2.6.33-7-rt30 and 2.6.38 comparison

[jordan]

Hi Victor,

> Same results (very few X-runs, if any) with :
>
>  -  on one hand, on an Asus laptop with a Core i3  : latency 2.3 msec
> with the 2.6.33.7.2-rt kernel and 23 msec with a 2.6.37 preempt kernel;

23msec with 2.6.37 preemptive kernel is terrible. Not one of my
machines would perform that badly (and most use 2.6.37.3). Some are
also considerably older (like 5years older) than your asus laptop..

>  -  on the other hand, on a Lenovo Thinkpad Core i5 :  latency 2.3 msec
> with the 2.6.33.7.2-rt kernel and 2.3 msec (yes the same) with a 2.6.37
> preempt kernel !!

that's not surprising, that is what i would expect to see. 2.6.33-rt
is getting quite dated, and old. For industrial / embedded
applications it might be the only option but for Linux audio / video
editing, there are preemptive kernels that work very well.

> Of course the Lenovo Thinkpad is a very good laptop, but how come that there is such a big difference?

someone can probably answer this better, but i think it comes down to
hardware / software design / implementation - some PCs work better
because of the quality of parts, the software and how well they
integrate.

*** ....and how well-supported each component is under Linux. ***

> So, I have observed the same quality of performance as Madovsky but not
> on every machine I've used... I don't have a clue why....

I also have observed the same thing.  i actually only use RT on 1
computer right now. On the Rest of my machines, 2.6.37.3-zen (PREEMPT,
BFS and BFQ v2.0 + optimizations) works wonders.
In every use i have 2.6.37 is better ( i am a pro-audio person ).

I will say between 2.6.33 and 2.6.38 they have been many MANY changes,
so i am not surprised that 2.6.38 performs as well as 2.6.33-rt 9or
even just close). 2.6.38 (i believe) is the First kernel to have BKL
(Big kernel Lock) completely removed ....and also CFS (completely fair
scheduler) have changed quite a bit (since .33) and has been more
optimized. those 2 factors alone, should reduce latency. Coupled with
other fixes and changes in the kernel - should explain these kinds of
observations.

just my 2 cents

jordan
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: 2.6.33-7-rt30 and 2.6.38 comparison

[Victor henri]

Hello Jordan

> > - on one hand, on an Asus laptop with a Core i3 : latency 2.3 msec
> > with the 2.6.33.7.2-rt kernel and 23 msec with a 2.6.37 preempt kernel;
>
> 23msec with 2.6.37 preemptive kernel is terrible.

For sure 23 msec is terrible; I was so surprised to see that!

 Not one of my
> machines would perform that badly (and most use 2.6.37.3). Some are
> also considerably older (like 5years older) than your asus laptop..
>
> > - on the other hand, on a Lenovo Thinkpad Core i5 : latency 2.3 msec
> > with the 2.6.33.7.2-rt kernel and 2.3 msec (yes the same) with a 2.6.37
> > preempt kernel !!
>
> that's not surprising, that is what i would expect to see. 2.6.33-rt
> is getting quite dated, and old. For industrial / embedded
> applications it might be the only option but for Linux audio / video
> editing, there are preemptive kernels that work very well.
>
> > Of course the Lenovo Thinkpad is a very good laptop, but how come that there is such a big difference?
>
> someone can probably answer this better, but i think it comes down to
> hardware / software design / implementation - some PCs work better
> because of the quality of parts, the software and how well they
> integrate.
>
> *** ....and how well-supported each component is under Linux. ***

Yes Lenovo Thinkpad are famous for having good performance... and furthermore the T410s is "Ubuntu certified". However I experience hang on with the 2.6.33.7.2-rt kernel, but I have a touchscreen (input = 4 fingers multitouch working natively under Maverick... really great) and I suspect the mutitouch screen is responsible of this. Hopefully the performaces og the preempt are great!

>
> > So, I have observed the same quality of performance as Madovsky but not
> > on every machine I've used... I don't have a clue why....
>
> I also have observed the same thing. i actually only use RT on 1
> computer right now. On the Rest of my machines, 2.6.37.3-zen (PREEMPT,
> BFS and BFQ v2.0 + optimizations) works wonders.
> In every use i have 2.6.37 is better ( i am a pro-audio person ).
>
> I will say between 2.6.33 and 2.6.38 they have been many MANY changes,
> so i am not surprised that 2.6.38 performs as well as 2.6.33-rt 9or
> even just close). 2.6.38 (i believe) is the First kernel to have BKL
> (Big kernel Lock) completely removed ....and also CFS (completely fair
> scheduler) have changed quite a bit (since .33) and has been more
> optimized. those 2 factors alone, should reduce latency. Coupled with
> other fixes and changes in the kernel - should explain these kinds of
> observations.
>
> just my 2 cents
>
> jordan

Thank you for those explanations

Victor
 		 	   		  

Re: 2.6.33-7-rt30 and 2.6.38 comparison

[jordan]

Hey Victor,

> For sure 23 msec is terrible; I was so surprised to see that!

So was I man :)  On my old Dell inspiron 6400, coreduo with 2.6.37, i
am running quite stable at 2.3ms (as in no xruns, at all).  it's a
semi-slow machine, with 1 sketchy PCI bus, too. It has been recycled
into a rackmount, and has "on/off" operation. It's still good enough
though, to run several VSTi's at once, some effects chains,
sooperlooper and a sampler. I never have problems with it. i use it
live (with my band). RT runs like crap on it, so i used 2.6.37 with
BFS.

I'm very impressed with recent kernels, using a non-RT kernel seems to
not be a problem.
although, most of the time, they do require tweaking.

i won't be excited about RT-kernels, until i see 2.6.38-rt ~ that
should be a good one!

I recycle junk laptops - ie: busted display / keyboard / failed GPU /
etc ~ useless to most people, but they are easily converted into
dedicated music gear. (synth-modules, VSTi hosts, FX rack).
So i am often working with "sub-standard" equipment. surprisingly most
of the time you can tune
them quite well ~ and get very reasonable performance. RT can
misbehave on some gear so i use BFS scheduler in those corner cases. I
mostly will use Zen-kernel (www.zen-kernel.org) as that kernel can be
tuned up very well, and has all of the bellz and whistles.

> Yes Lenovo Thinkpad are famous for having good performance... and furthermore the T410s is "Ubuntu certified". However I experience hang on with the 2.6.33.7.2-rt kernel, but I have a touchscreen (input = 4 fingers multitouch working natively under Maverick... really great) and I suspect the mutitouch screen is responsible of this. Hopefully the performaces og the preempt are great!

I have heard lenovo laptops are quite nice for running linux. :)
...in this case it could be your multi-touch, but it also could be
your video card. . On my one machine if i use a compositor (compiz,
metacity, etc) it randomly will have soft lockups. All of the audio
stuff still will work but the display goes down the drain.  On another
machine, an older version of BFQ ( I/O scheduler ), would potentially
cause lockups. (although i had thought that it was my ATI card) both
of those machines are quite stable now though. :)

jordan

RE: 2.6.33-7-rt30 and 2.6.38 comparison

[Victor henri]

Hello Jordan

>
> Hi Victor,
>
> > Same results (very few X-runs, if any) with :
> >
> >  -  on one hand, on an Asus laptop with a Core i3  : latency 2.3 msec
> > with the 2.6.33.7.2-rt kernel and 23 msec with a 2.6.37 preempt kernel;
>
> 23msec with 2.6.37 preemptive kernel is terrible. Not one of my
> machines would perform that badly (and most use 2.6.37.3). Some are
> also considerably older (like 5years older) than your asus laptop..
>
> >  -  on the other hand, on a Lenovo Thinkpad Core i5 :  latency 2.3 msec
> > with the 2.6.33.7.2-rt kernel and 2.3 msec (yes the same) with a 2.6.37
> > preempt kernel !!
>
> that's not surprising, that is what i would expect to see. 2.6.33-rt
> is getting quite dated, and old. For industrial / embedded
> applications it might be the only option but for Linux audio / video
> editing, there are preemptive kernels that work very well.
>
> > Of course the Lenovo Thinkpad is a very good laptop, but how come that there is such a big difference?
>
> someone can probably answer this better, but i think it comes down to
> hardware / software design / implementation - some PCs work better
> because of the quality of parts, the software and how well they
> integrate.
>
> *** ....and how well-supported each component is under Linux. ***
>
> > So, I have observed the same quality of performance as Madovsky but not
> > on every machine I've used... I don't have a clue why....
>
> I also have observed the same thing. i actually only use RT on 1
> computer right now. On the Rest of my machines, 2.6.37.3-zen (PREEMPT,
> BFS and BFQ v2.0 + optimizations) works wonders.
> In every use i have 2.6.37 is better ( i am a pro-audio person ).

Could you briefly summarize why use zen-kernel instead of vanilla? As 
far as I understand, the zen-kernel brings features that may not (yet) 
be available on vanilla kernel, (such as those scheduler BFS and BFQ?) Could please say wich optimization is needed for the best performances? I'd like to give it a try actually. Or is there a place where I can find all these recommendations; I'm talking about the special features that you can find specifically in a zen-kernel, and that can help further for lower latency for music performance.

On the other hand, I've checked the www.zen-kernel.org website; the most recent kernel patch seem to be for 2.6.36 kernel; where did you find a 2.6.37 zen kernel? 

>
> I will say between 2.6.33 and 2.6.38 they have been many MANY changes,
> so i am not surprised that 2.6.38 performs as well as 2.6.33-rt 9or
> even just close). 2.6.38 (i believe) is the First kernel to have BKL
> (Big kernel Lock) completely removed ....and also CFS (completely fair
> scheduler) have changed quite a bit (since .33) and has been more
> optimized. those 2 factors alone, should reduce latency. Coupled with
> other fixes and changes in the kernel - should explain these kinds of
> observations.
>
> just my 2 cents
>
> jordan
> --

Victor
 		 	   		  

Re: 2.6.33-7-rt30 and 2.6.38 comparison

[jordan]

Hey Victor,

Sorry for the delayed response, i've been a little busy.

> Could you briefly summarize why use zen-kernel instead of vanilla? As
> far as I understand, the zen-kernel brings features that may not (yet)
> be available on vanilla kernel, (such as those scheduler BFS and BFQ?)

I use zen-kernel over vanilla, for the simple fact that it works, and
i don't have to apply multiple patches to a vanilla source. BFS will
likely never be upstream (which is old news), and BFQ i am not sure
about.

> Could please say wich optimization is needed for the best performances? I'd like to give it a try actually. Or is there a place where I can find all these recommendations; I'm talking about the special features that you can find specifically in a zen-kernel, and that can help further for lower latency for music performance.

BFS is the obvious patch set that helps with low-latency and cpu
scheduling. I do use BFQ io scheduler as well, and i have been
experimenting with the latest BFQ Cgroups stuff too. An example of
that can be found in the archlinux AUR ~ kernel26-ck.

I prefer to use non-vanilla, because often i am using more than a few
patches, on my kernel. Zen-kernel, -ck, etc already had other
tuning/fixes done to them.  then i can do the fine-tuning.

Other tunings you can do..? well, there are lots. When compiling a
kernel, use CFLAG optimizations.
Off hand, easy one's that are safe - CFLAGS='-march=native
-mtune=native' (those 2 will compile the code for your CPU). but there
are many that you can use. read through GCC documentation, Gentoo,
sourcemage and archlinux resources on compiling a kernel. (in fact
gentoo, and Arch have the best documentation on any Linux subject -
including guides on maximizing performance).

you could also only compile modules that you need (saves on compiling time too).

after getting a new kernel to run that is decent. You can muddle
through kernel subsystems;

disk scheduler tunables:

/sys/block/sda/queue
/sys/block/sda/queue/iosched

Cpu scheduler tunables:

/proc/sys/kernel

VM subsystem

/proc/sys/vm

I would have to go into way too much detail here, to explain these
and which things can be tuned.
But i think you will find that google is your friend in this case.
The idea is that you can adjust certain tunables to adjust the system
to better suit your needs. sometimes, reducing latency, or possibly
tuning scheduler performance, etc.

You then can store your changes in /etc/rc.local and /etc/sysctl.conf
- to be run on boot.

There are plenty of tools for benchmarking, latency-tests, etc.
...again, google.

There are also other daemons and tools that could be used...
Ulatencyd, verynice, schedtools, Cgroups(although, i wouldn't
recommend Cgroups just yet),etc. While not are meant for RT-related
Jack applications (actually some will hinder jackd performance).. Some
can be used to control/limit other processes that may be interferring,
introducing latency or causing xruns.  In most cases, you will require
testing/benchmarking, and also just as importantly - "real world
tests" (as in using your system).

tuning all hardware is also very important. using patched drivers, if
required. tuning your disk with tools like sdparm, hdparm. editing
your fstab, etc. I also tend to disable any hardware not being used.
(no kernel support/disable in bios)... I would run a stripped down
desktop too. Although, i do run gnome, i've optimized it.  Tuning your
IRQ manually, over using irqbalance, or rtirq...

> On the other hand, I've checked the www.zen-kernel.org website; the most recent kernel patch seem to be for 2.6.36 kernel; where did you find a 2.6.37 zen kernel?

master branch of Zen-stable (using Git). I have it locally, and update
the tree, when I am about to compile the kernel. I don't download
snapshots, or Zen-kernel patches, git only.

sorry that i don't have more specific info, but your asked a very
broad question.

cheerz

jordan