linux-rt-users.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [ANNOUNCE] 3.0-rc7-rt0
@ 2011-07-20  0:37 Thomas Gleixner
  2011-07-20  2:48 ` Frank Rowand
                   ` (7 more replies)
  0 siblings, 8 replies; 20+ messages in thread
From: Thomas Gleixner @ 2011-07-20  0:37 UTC (permalink / raw)
  To: LKML
  Cc: linux-rt-users, Peter Zijlstra, Ingo Molnar, Carsten Emde,
	Clark Williams, Paul E. McKenney, Kumar Gala, Ralf Baechle

Dear RT Folks,

I'm pleased to announce the first drop of the 3.0-rc7 based RT
patch.

It's been quite a while since 2.6.33-rt, but I went through a very
painful experience while trying to get a 2.6.38-rt stabilized. The
beast insisted on destroying filesystems with reproduction times
measured in days and the total refusal to reveal at least a
minimalistic hint to debug the root cause. Staring into completely
useless traces for months is not a very pleasant pastime.

That's the very first problem in the RT history which I gave up on.

[The truth: Linus avoiding the final 2.6.42 release made all my
 ultimate plans go down the drain ... ]

Though while trying to analyse the problem I had plenty of time to
twist my brain around the existing RT approach and its shortcomings.

The main issue which RT is fighting with is the ever growing per cpu
variable usage and the assumptions which are built around it. The
existing RT approach to work around this with PER_CPU_LOCKED
constructs and hand the CPU number around simply does not work anymore
because the number of sites which need to be patched is way too large
and the resulting mess in the code is neither acceptable nor
maintainable.

After lenghty and fruitful discussions with Peter Zijlstra - thanks a
lot Peter! - we finally agreed on trying a totally different approach
to tackle these issues: disabling migration over spinlock and get_cpu
sections. This had been discussed before, but nobody ever considered
to sit down and make it work.

This keeps the semantics which are expected by the per cpu users,
while keeping the regions preemptible. As a side effect, it allows us
to run softirq handlers directly from irq threads on local_bh_enable
which was a long desired feature to lower the performance impact of
RT.

Changing this required a major refactoring of the RT patch queue,
which took some time as I had to go through every single patch, fold
fixes back into the right places and sort them into various categories:

 - Mainline ready (raw lock annotations, infrastructure patches, code
   restructuring...)

 - Preparatory (_rt()/_nort() variants of preempt_*(), local_irq_*(),
   BUG*(), WARN*() and the annotations in various places)

 - Base patches (Reworking the slab/page_alloc code, bit_spinlock
   replacements, migrate disable infrastructure ...)

 - Full RT patches (sleeping spinlocks and the resulting fixups here
   and there)

In course of that exercise I weeded out a lot of historically grown
hackery and dropped stuff which was not essential for getting it up
and running. Thanks to Carsten for reintegrating the tracer addons
which he's using for the OSADL test farm:

  https://www.osadl.org/?id=1042

I probably have missed a few bits and pieces, but the overall outcome
is stable and survived testing on various systems. The latency
behaviour with cyclictest is on par with 33-rt at least on x86_64/32.

The overall patch size has shrunk significantly and the readability
(except for the missing changelogs in various patches) is at an
acceptable level.

If you download the quilt tarball, you'll find various sections:

- upstream fixes: Stuff broken upstream which we managed to trip
  over. This section contains real weird stuff from simple fixes, over
  mainline code which claims to contain (complete bogus) RT support up
  to an archaeologic bug in the floppy driver code.

  8 patches (size 8892)
  7 files changed, 59 insertions(+), 51 deletions(-)

- upstream submitted: Stuff which is on LKML already and needs some
  follow up.

  4 patches (size 9741)
  4 files changed, 81 insertions(+), 119 deletions(-)

- upstream ready: Stuff which needs a bit polishing and upstream
  submission
  
  79 patches (size 232566)
  192 files changed, 1204 insertions(+), 1097 deletions(-)

- upstream needs work: Stuff which should go upstream, but needs some
  or lots of care.

  7 patches (size 164120)
  49 files changed, 3292 insertions(+), 253 deletions(-)

- the real rt stuff:

  125 patches (size 280665)
  162 files changed, 4327 insertions(+), 592 deletions(-)

The overall patch is now:
  223 patches (size 680054)
  374 files changed, 8950 insertions(+), 2099 deletions(-)

Compared that to 2.6.33-rt:
  462 patches (size 1396505)
  690 files changed, 15994 insertions(+), 5123 deletions(-)

That's a significant reduction in size and impact. Some of it is due
to the new approach, but we also got quite a lot of the infrastructure
patches upstream in the last few kernel releases. Thanks to all folks
who have helped to get that done, especially to Peter Zijlstra for
getting the preemptible mmu gather problem and lots of the scheduler
issues which we discovered in RT over time sorted out!!!

What's new in 3.0-rt ?

 - No more split soft interrupt threads. We need to analyze whether
   this is a good decision.

 - softirq handling from the end of interrupt threads and on all
   thread sites where a nested local_bh disabled section ends

 - SPARSE interrupts and IOMMU interrupt remapping work now

 - Split config option CONFIG_PREEMPT_RT into CONFIG_PREEMPT_RT_BASE
   and CONFIG_PREEMPT_RT_FULL. RT_BASE covers some of the more complex
   changes (e.g. mm/* where we substitute interrupt disabled sections
   with per cpu locks and the bit_spinlock to spinlock conversion).
   RT_BASE allows us to test and verify these changes independently of
   the big RT_FULL modifications. That's mainly a debugability and
   maintainability issue.

What's the state:

   We've done quite some testing on x86 32/64 bit and basic tests on
   some ARM/MIPS/POWERPC platforms. Thank God, no file system eating so
   far :)

   Given the fact that it is a major rewrite it's amazinlgy stable and
   I consider it to be the best -rt1 release we ever had. That doesn't
   mean that there are no bugs, since it has not had the proper test
   coverage yet.

   Thanks to Carsten, Clark and Peter for all the help to get this far!

Want to help?

   Many people offered help in the past and I had to turn them down so
   far as refactoring that stuff really is not a task which can be
   shared easily. Though now is the point where I can use all the help
   you promised to provide.

   What's needed?

   - Testing, testing, testing ... you know the drill (good bug
     reports are 98% of the solution)

   - Compare and analyze the performance/troughput impact of the new
     approach with 33-rt

   - Help mainlining the "upstream ready section"

     That means reviewing the patches, cleaning them up, fixing the
     changelogs, submitting them through the proper channels ...

     Please do not blindly pick any of these patches and submit them
     to mailing lists w/o doing the above. Also please coordinate on
     the #linux-rt IRC channel on oftc.net so redundant and
     conflicting work can be avoided

   - Help getting the "upstream needs work" section into shape

     All of these patches need a close look and (especially the
     hwlatency detector) major cleanups. Please coordinate with the
     patch authors and lookout for previous discussions of some of
     those on LKML.

   - Tend to the FIXME annotations in the RT stuff section

     I have annotated some places with /* FIXME ... comments. These
     sections are not for the faint hearted and need some serious
     review and thought.

   - Help with the RCU modifications

     That's an easy one. We have a volunteer signed up for this
     involuntarily already. Thanks Paul!

   - Twist your brain around the schedulability impact of the
     migrate_disable() approach.

     A really interesting research topic for our friends from the
     academic universe. Relevant and conclusive (even short notice)
     papers and/or talks on that topic have a reserved slot in the
     Kernel developers track at the Realtime Linux Workshop in Prague
     in October this year.

Enough marketing, here comes the real stuff.

  Patch against 3.0-rc7 can be found here:

    http://www.kernel.org/pub/linux/kernel/projects/rt/patch-3.0-rc7-rt0.patch.bz2

  The split quilt queue is available at:

    http://www.kernel.org/pub/linux/kernel/projects/rt/patches-3.0-rc7-rt0.tar.gz

There is no git tree for now.

I'm not yet convinced that moving RT to git was a good idea as quilt
allows me to move stuff around in a way more flexible manner. So for
now no git version until someone comes up with a brilliant idea which
allows me to keep my workflow sane (do not even try to suggest stgit &
co!).

That said, have fun and make sure that you have the fire extinguisher
ready when you start using this!

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 20+ messages in thread
* Re: [ANNOUNCE] 3.0-rc7-rt0
@ 2011-07-21 16:22 hermann
  2011-07-21 17:35 ` Thomas Gleixner
  0 siblings, 1 reply; 20+ messages in thread
From: hermann @ 2011-07-21 16:22 UTC (permalink / raw)
  To: linux-rt-users


> Dear RT Folks,
> 
> I'm pleased to announce the first drop of the 3.0-rc7 based RT
> patch.

Hi 
Many thanks for all this work. 

I try to build the 3.0-rc7-rt0 but fail with the following message:
 CC [M]  fs/ext3/balloc.o
In file included from include/linux/jbd.h:247:0,
                 from fs/ext3/balloc.c:18:
include/linux/jbd_common.h: In function ‘jbd_lock_bh_state’:
include/linux/jbd_common.h:43:2: error: ‘struct buffer_head’ has no member named ‘b_state_lock’
include/linux/jbd_common.h: In function ‘jbd_trylock_bh_state’:
include/linux/jbd_common.h:52:9: error: ‘struct buffer_head’ has no member named ‘b_state_lock’
include/linux/jbd_common.h: In function ‘jbd_is_locked_bh_state’:
include/linux/jbd_common.h:61:27: error: ‘struct buffer_head’ has no member named ‘b_state_lock’
include/linux/jbd_common.h: In function ‘jbd_unlock_bh_state’:
include/linux/jbd_common.h:70:2: error: ‘struct buffer_head’ has no member named ‘b_state_lock’
include/linux/jbd_common.h: In function ‘jbd_lock_bh_journal_head’:
include/linux/jbd_common.h:79:2: error: ‘struct buffer_head’ has no member named ‘b_journal_head_lock’
include/linux/jbd_common.h: In function ‘jbd_unlock_bh_journal_head’:
include/linux/jbd_common.h:88:2: error: ‘struct buffer_head’ has no member named ‘b_journal_head_lock’
make[3]: *** [fs/ext3/balloc.o] Fehler 1
make[2]: *** [fs/ext3] Fehler 2
make[1]: *** [fs] Fehler 2
make[1]: Leaving directory `/home/brummer/Projekte/Kernel/linux-3.0-rc7'
make: *** [debian/stamp/build/kernel] Fehler 2


my system is:
System:    Host box Kernel 2.6.39-rc7 i686 (32 bit gcc 4.5.2) 
           Desktop Xfce 4.8.2 (Gtk 2.24.4) Distro Debian GNU/Linux wheezy/sid
Machine:   System Hewlett-Packard product hp workstation version x2100/2600
           Mobo Hewlett-Packard model HP WMTA System Board version A03 Bios Phoenix version JG.W1.04US date 07/26/2002
CPU:       Single core Intel Pentium 4 CPU (-UP-) cache 512 KB flags (sse sse2) bmips 5183.59 clocked at 2591.796 MHz 
Graphics:  Card: ATI Radeon R200 QH [Radeon 8500] bus-ID: 01:00.0 X.Org 1.10.1 drivers ati,radeon Resolution 1024x768@75.0hz 
           GLX Renderer Mesa X11 GLX Version 2.1 Mesa 7.10.2 Direct Rendering Yes
Audio:     Card Creative Labs SB Live! EMU10k1 driver EMU10K1_Audigy port 5040 bus-ID: 02:0a.0 Sound: ALSA v: 1.0.24
Network:   Card Intel 82801BA/BAM/CA/CAM Ethernet Controller driver e100 v: 3.5.24-k2-NAPI port 5000 bus-ID: 02:08.0
           IF: eth0 state: up speed: 100 Mbps duplex: full mac: 00:30:6e:26:c4:6d
Drives:    HDD Total Size: 58.1GB (57.4% used) 1: /dev/sda IC35L060AVV207 41.2GB 
           2: /dev/sdb IBM 16.9GB 
           Optical: /dev/sr0 model: TOSHIBA DVD-ROM SD-M1612 rev: 1004 dev-links: cdrom,dvd,scd0
           Features: speed: 48x multisession: yes audio: yes dvd: yes rw: none state: running
Partition: ID:/ size: 38G used: 27G (76%) fs: ext3 dev: /dev/sda2 label: N/A uuid: 1c8cbe6a-e15f-4a86-9b2b-2c75628ed183 
           ID:/media/sdb1 size: 15G used: 4.6G (33%) fs: ext4 dev: /dev/sdb1 label: N/A uuid: b6d0dca6-d88a-4096-bd54-c20dc98db436 
           ID:swap-1 size: 0.74GB used: 0.00GB (0%) fs: swap dev: /dev/sda1 label: N/A uuid: N/A 
Unmounted: ID: /dev/sdb5 size: 0.75G label: N/A uuid: 6b246475-8d0a-4414-9657-4586b3a57ea1 
Sensors:   Error: You do not have the sensors app installed.
Info:      Processes 113 Uptime 13:07 Memory 449.0/2026.4MB Runlevel 5 Client Shell

regards
hermann

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2011-07-21 18:40 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-07-20  0:37 [ANNOUNCE] 3.0-rc7-rt0 Thomas Gleixner
2011-07-20  2:48 ` Frank Rowand
2011-07-20  3:22   ` Fernando Lopez-Lezcano
2011-07-20  5:03 ` Paul E. McKenney
2011-07-20  9:02 ` David Kastrup
2011-07-20  9:48 ` Geunsik Lim
2011-07-20 11:08 ` Armin Steinhoff
2011-07-20 15:33   ` Madovsky
2011-07-20 15:56     ` Thomas Gleixner
2011-07-20 15:59       ` Madovsky
2011-07-20 17:02 ` Darren Hart
2011-07-20 18:35   ` [ANNOUNCE] 3.0-rc7-rt0 (hang then panic on dual socket xeon) Darren Hart
2011-07-20 19:10 ` [PATCH] Fix build failure for modular ext3/4 builds Uwe Kleine-König
2011-07-20 19:15 ` [ANNOUNCE] 3.0-rc7-rt0 Noah Watkins
  -- strict thread matches above, loose matches on Subject: below --
2011-07-21 16:22 hermann
2011-07-21 17:35 ` Thomas Gleixner
2011-07-21 17:43   ` Thomas Gleixner
2011-07-21 18:33     ` hermann
2011-07-21 18:28   ` andi
2011-07-21 18:40     ` hermann

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).