Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Mark_H_Johnson]

>i have released the -U5 Real-Time Preemption patch:
>
>
http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5

I am getting build problems - specifically with:

  CC [M]  drivers/char/ipmi/ipmi_watchdog.o
  CC [M]  fs/jfs/jfs_dmap.o
drivers/char/ipmi/ipmi_watchdog.c:389: warning: type defaults to `int' in
declaration of `DECLARE_MUTEX_LOCKED'
drivers/char/ipmi/ipmi_watchdog.c:389: warning: parameter names (without
types) in function declaration
drivers/char/ipmi/ipmi_watchdog.c: In function `heartbeat_free_smi':
drivers/char/ipmi/ipmi_watchdog.c:393: error: `heartbeat_wait_lock'
undeclared (first use in this function)
drivers/char/ipmi/ipmi_watchdog.c:393: error: (Each undeclared identifier
is reported only once
drivers/char/ipmi/ipmi_watchdog.c:393: error: for each function it appears
in.)
drivers/char/ipmi/ipmi_watchdog.c: In function `heartbeat_free_recv':
drivers/char/ipmi/ipmi_watchdog.c:398: error: `heartbeat_wait_lock'
undeclared (first use in this function)
drivers/char/ipmi/ipmi_watchdog.c: In function `ipmi_heartbeat':
drivers/char/ipmi/ipmi_watchdog.c:476: error: `heartbeat_wait_lock'
undeclared (first use in this function)
drivers/char/ipmi/ipmi_watchdog.c: At top level:
drivers/char/ipmi/ipmi_watchdog.c:389: warning: `DECLARE_MUTEX_LOCKED'
declared `static' but never defined

If I read the patch correctly, this should be recoded as
  DECLARE_MUTEX
instead, but a quick grep of the source code indicates we have about
20 more places where DECLARE_MUTEX_LOCKED is still used. Should I do
a global replace on that or is something else needed?

I also had a compile failure in XFS. The messages are:
  CC [M]  fs/xfs/quota/xfs_dquot_item.o
  CC [M]  fs/xfs/quota/xfs_trans_dquot.o
fs/xfs/quota/xfs_dquot_item.c: In function `xfs_qm_dquot_logitem_pushbuf':
fs/xfs/quota/xfs_dquot_item.c:266: error: structure has no member named
`count'
fs/xfs/quota/xfs_dquot_item.c:279: error: structure has no member named
`count'

This refers to a macro defined at
fs/xfs/linux-2.6/sema.h:51:#define valusema(sp)
(atomic_read(&(sp)->count))

Not quite sure if this is an error due to type changes or yet another
name collision.

Please advise how to proceed.

  --Mark

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* Mark_H_Johnson@raytheon.com <Mark_H_Johnson@raytheon.com> wrote:

> >i have released the -U5 Real-Time Preemption patch:
> >
> >
> http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5
> 
> I am getting build problems - specifically with:

>   CC [M]  drivers/char/ipmi/ipmi_watchdog.o

> If I read the patch correctly, this should be recoded as
>   DECLARE_MUTEX
> instead, but a quick grep of the source code indicates we have about
> 20 more places where DECLARE_MUTEX_LOCKED is still used. Should I do
> a global replace on that or is something else needed?

it's not normally used, and it's much simpler to rewrite those places
than to implement initialization. (which would be quite hairy)

> I also had a compile failure in XFS. The messages are:
>   CC [M]  fs/xfs/quota/xfs_dquot_item.o
>   CC [M]  fs/xfs/quota/xfs_trans_dquot.o

ok, i've re-uploaded a new version of -U5 that has this and the 
ipmi_watchdog compilation problems fixed.

please check whether it works, XFS does not seem to make use of count>1
semaphores but one never knows ...

	Ingo

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[K.R. Foley]

Ingo Molnar wrote:
> * Mark_H_Johnson@raytheon.com <Mark_H_Johnson@raytheon.com> wrote:
> 
> 
>>>i have released the -U5 Real-Time Preemption patch:
>>>
>>>
>>
>>http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5
>>
>>I am getting build problems - specifically with:
> 
> 
>>  CC [M]  drivers/char/ipmi/ipmi_watchdog.o
> 
> 
>>If I read the patch correctly, this should be recoded as
>>  DECLARE_MUTEX
>>instead, but a quick grep of the source code indicates we have about
>>20 more places where DECLARE_MUTEX_LOCKED is still used. Should I do
>>a global replace on that or is something else needed?
> 
> 
> it's not normally used, and it's much simpler to rewrite those places
> than to implement initialization. (which would be quite hairy)
> 
> 
>>I also had a compile failure in XFS. The messages are:
>>  CC [M]  fs/xfs/quota/xfs_dquot_item.o
>>  CC [M]  fs/xfs/quota/xfs_trans_dquot.o
> 
> 
> ok, i've re-uploaded a new version of -U5 that has this and the 
> ipmi_watchdog compilation problems fixed.
> 
> please check whether it works, XFS does not seem to make use of count>1
> semaphores but one never knows ...
> 
> 	Ingo
> 
> 

Well you just beat me with that one. :) And here is another for aha152x.

--- linux-2.6.9-rc4-mm1/drivers/scsi/aha152x.c.orig     2004-10-18 
12:05:02.891049751 -0500
+++ linux-2.6.9-rc4-mm1/drivers/scsi/aha152x.c  2004-10-18 
12:05:24.360353020 -0500
@@ -1160,7 +1160,7 @@
  static int aha152x_device_reset(Scsi_Cmnd * SCpnt)
  {
         struct Scsi_Host *shpnt = SCpnt->device->host;
-       DECLARE_MUTEX_LOCKED(sem);
+       DECLARE_MUTEX(sem);
         struct timer_list timer;
         int ret, issued, disconnected;
         unsigned long flags;

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* K.R. Foley <kr@cybsft.com> wrote:

> Well you just beat me with that one. :) And here is another for
> aha152x.

> -       DECLARE_MUTEX_LOCKED(sem);
> +       DECLARE_MUTEX(sem);

almost - the full patch is the one below. (DECLARE_MUTEX() initializes
the mutex as unlocked, so there's a difference.)

	Ingo

--- linux/drivers/scsi/aha152x.c.orig
+++ linux/drivers/scsi/aha152x.c
@@ -1160,11 +1160,12 @@ static void timer_expired(unsigned long 
 static int aha152x_device_reset(Scsi_Cmnd * SCpnt)
 {
 	struct Scsi_Host *shpnt = SCpnt->device->host;
-	DECLARE_MUTEX_LOCKED(sem);
+	DECLARE_MUTEX(sem);
 	struct timer_list timer;
 	int ret, issued, disconnected;
 	unsigned long flags;
 
+	init_MUTEX_LOCKED(&sem);
 #if defined(AHA152X_DEBUG)
 	if(HOSTDATA(shpnt)->debug & debug_eh) {
 		printk(INFO_LEAD "aha152x_device_reset(%p)", CMDINFO(SCpnt), SCpnt);

[patch] VP-2.6.9-rc4-mm1-T5

[Ingo Molnar]

* Mark_H_Johnson@Raytheon.com <Mark_H_Johnson@Raytheon.com> wrote:

> I would have to say this is "very rough" at this point. I had the
> following problems in the build:

i've uploaded -T5 which should fix most of the build issues:

 http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T5

CONFIG_PREEMPT_REALTIME is still an experimental feature and defaults to
'n'.

-T5 will likely not fix the exit.c warnings, which, unless they are
accompanied by real crashes, should be mostly harmless. (famous last
words.) (The zombie task and self-reaping thread handling is a really
hard nut to crack, and i have nobody but me to blame for that code ...)

	Ingo

[patch] VP-2.6.9-rc4-mm1-T6

[Ingo Molnar]

i've uploaded -T6:

  http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T6

this should fix the UP build issues reported by many. -T6 also brings
back the ->break_lock framework and converts a few more locks to raw.

SMP is still expected to be flaky due to the zombie-task problem(s). But
UP is not out of the 'extremely experimental' status either.

to create a -T6 tree from scratch the patching order is:

   http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
 + http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
 + http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
 + http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T6

	Ingo

Re: [patch] VP-2.6.9-rc4-mm1-T6

[Ingo Molnar]

i've uploaded -T7:

  http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T7

Changes since -T6:

- further stabilization of PREEMPT_REALTIME: fixed the task-reaping
  problem by moving TASK_ZOMBIE out of p->state and thus completely
  separating preemption from the child-exit mechanism. This got rid of 
  the 'Badness in exit.c' warnings on my SMP testbox (and related
  crashes).

- fixed the _mutex_trylock_bh missing symbol problem reported by K.R. 
  Foley and Florian Schmidt.

- turned the sysrq lock into a raw spinlock, to enable direct keyboard
  irqs.

PREEMPT_REALTIME is still experimental, but it's already looking much
better on my testboxes.

to create a -T7 tree from scratch the patching order is:

   http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
 + http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
 + http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
 + http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T7

	Ingo

[patch] VP-2.6.9-rc4-mm1-T8

[Ingo Molnar]

i've uploaded the -T8 VP patch:

  http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T8

lots of stabilization of CONFIG_PREEMPT_REALTIME. It's still in
experimental status but general stability is improving.

Changes since -T7:

 - fixed a nasty category of bugs that were introduced by the use of 
   rwsems for rwlocks. rwsems are not read-recursive, while rwlocks are. 
   Fortunately it was not too hard to identify & fix recursive users of
   tasklist_lock, in fact each of these also qualifies as a cleanup. The 
   symptom of this bug was a soft-deadlocking of the system. 

 - fixed profiler locks, i believe this could resolve the bootup crash
   reported by K.R. Foley.

 - fixed VP / XFS namespace collision reported by Mark H. Johnson

 - fix one more final detail of the new zombie-task handling code

 - fixed 3c59x.c, fasync-handling, ipv6, module-loader runtime
   warnings reported by K.R. Foley.

 - fixed the ali5451 locking

to create a -T8 tree from scratch the patching order is:

   http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
 + http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
 + http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
 + http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T8

	Ingo

[patch] VP-2.6.9-rc4-mm1-T9

[Ingo Molnar]

i've uploaded the -T9 VP patch:

  http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T9

this is a bugfixes-only release that should fix the highmem-related
issues reported by K.R. Foley and Mark H. Johnson: 3 more locks had to
be converted to raw.

to create a -T9 tree from scratch the patching order is:
 
   http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
 + http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
 + http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
 + http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T9

	Ingo

[patch] Real-Time Preemption, -VP-2.6.9-rc4-mm1-U0

[Ingo Molnar]

i'm pleased to announce a significantly improved version of the
Real-Time Preemption (PREEMPT_REALTIME) feature that i have been working
towards in the past couple of weeks.

the patch (against 2.6.9-rc4-mm1) can be downloaded from:

  http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U0

this is i believe the first correct conversion of the Linux kernel to a
fully preemptible (fully mutex-based) preemption model, while still
keeping all locking properties of Linux.

I also think that this feature can and should be integrated into the
upstream kernel sometime in the future. It will need improvements and
fixes and lots of testing, but i believe the basic concept is sound and
inclusion is manageable and desirable. So comments, testing and feedback
is more than welcome!

to recap the conceptual issues that needed solving: the previous patch
already converted a fair portion of spinlocks to mutexes, but that in a
fully preemptible kernel model the following locking primitives are
especially problematic:

	- RCU locking
	- per-cpu counters and per-cpu variables
	- tlb gather/release logic
	- seqlocks
	- atomic-kmaps
	- pte mapping

note that while we tend to think about these as SMP locking primitives,
in a fully preemptible model these locking rules are necessary for
correctness, even on a single-CPU embedded box.

Unfortunately none of the existing preemption patches solve these
problems: they concentrate on UP but these locking rules must not be
ignored on UP either!

(Bill Huey's mutex patch he just released is the one notable exception i
know about, which is also a correct implementation i believe, but it
doesnt attack these locking rules [yet]. Bill's locking hierarchy is i
believe quite similar to the -T9 patch - this i believe is roughly the
best one can get when only using spinlocks as a vehicle.)

In the previous (-T9) version of the Real-Time Preemption patch, the
above locking primitives kept large portions of kernel code
non-preemptable, causing a 'spreadout' of raw spinlocks and
non-preemptible sections to various kernel subsystems.

Another, even more problematic effect was that both network drivers and
block IO drivers got 'contaminated' by raw spinlocks, triggering lots of
per-driver changes and thus limiting testability. It is basically an
opt-in model to correctness which is bad from a maintainance and
upstream acceptance point of view. The -T9 patch was a prime example of
the problems the Linux locking primitives cause in a fully preemptible
kernel model.


To solve all these fundamental problems, i improved/fixed/changed all of
these locking methods to be preemption-friendly. Most of the time it was
necessary to introduce an additional API variant because e.g.
rcu_read_lock() is anonymous (it doesnt identify the data protected), so
i introduced a variant that takes the write-lock as an argument. In the
PREEMPT_REALTIME case we can thus properly serialize on that lock.

For per-cpu variables i introduced a new API variant that creates a
spinlock-array for the per-cpu-variable, and users must make sure the
cpu field doesnt change. Migration to another CPU can happen within the
critical section, but 'statistically' the variable is still per-CPU and
update correctness is fully preserved.

TLB shootdown was the source of another nasty type of critical section:
it keeps preemption disabled during much of the pagetable zapping and
also relies on a per-CPU variable to keep TLB state. The fix for
PREEMPT_REALTIME (on x86) was to implement a simpler but preemptible TLB
flushing method. This necessiated two minor API changes to the generic
TLB shootdown code - pretty straightforward ones.

Atomic kmaps were another source of non-preemptible sections:
pte_map/pte_unmap both used nontrivial functions within and ran for a
long time, creating a lock dependency and a latency problem as well. I
wrapped them via normal kmaps, which thus become preemptible. (The main
reason for atomic kmaps were non-preemptability constraints - but those
are not present in a fully preemptible model.)

seqlocks (used within the VFS and other places) were another problem:
the are now preemptible by default, the same auto-type-detection logic
applies to them as to preemptible/raw spinlocks: switching between a
preemptible and a non-preemptible type is done by changing the
prototype, the locking APIs stay the same.

The improvements to locking allowed the gradual 'pulling out' of all
raw-spinlocks from various kernel subsystems. In the -U0 patch i have
almost completely finished this 'pullout', and as a result the following
kernel subsystems are now completely spinlock-free and 100% mutex-based:

	- networking
	- IO subsystem
	- VFS and lowlevel filesystems
	- memory management (except the SLAB allocator lock)
	- signal code
	- all of drivers/* and sound/*

note: it is important that when PREEMPT_REALTIME is disabled, the old
locking rules apply and there is no performance impact whatsoever. So
what this implements is in essence a compile-time multi-tier locking
architecture enabling 4 types of preemption models:

  - stock                      (casual voluntary kernel preemption)
  - CONFIG_PREEMPT_VOLUNTARY   (lots of cond_resched() points)
  - CONFIG_PREEMPT             (involuntary preemption plus spinlocks)
  - CONFIG_PREEMPT_REALTIME    (everything is a mutex)

these models cover all the variants people are interested in: servers
with almost no latency worries, desktops with ~1msec needs and hard-RT
applications needing both microsecond-range latencies and provable
maximum latencies.

to quantitatively see the effects of these changes to the locking
landscape, here's the output of a script that disassembles the kernel
image and counts the number of actual spin lock/unlock function calls
done versus mutex lock/unlocks:

With PREEMPT_REALTIME disabled, all locks are spinlocks and old
semaphores:

        spinlock API calls:      5359  (71.6%)
        | old mutex API calls:   2120  (28.3%)
        | new mutex API calls:      2  (0%)
        all mutex API calls:     2122  (28.3%)
        --------------------------------------
        lock API calls:          7481 (100.0%)

with the -T9 kernel, which had the 'spread out' locking model, a
considerable portion of spinlocks were replaced by mutexes, but more
than 20% of usage was still spinlocks:

        spinlock API calls:      1835  (23.1%)
        | old mutex API calls:   2142  (26.9%)
        | new mutex API calls:   3961  (49.8%)
        all mutex API calls:     6103  (76.8%)
        --------------------------------------
        lock API calls:          7938 (100.0%)

here are some fresh numbers from Bill Huey's mmLinux kernel:

        spinlock API calls:      2452  (30.3%)
        all mutex API calls:     5614  (69.6%)
        --------------------------------------
        lock API calls:          8066 (100.0%)

(his mutex implementation directly falls back to up()/down() so the new
mutexes become part of the old mutexes.)

while i believe that the locking design is fundamentally incomplete in
the MontaVista kernel and thus is not directly comparable to
PREEMPT_REALTIME nor the mmLinux kernel, here are the stats from it
using a similar .config:

        spinlock API calls:      1444  (26.1%)
        | old mutex API calls:   2095  (37.9%)
        | new mutex API calls:   1981  (35.8%)
        all mutex API calls:     4076  (73.8%)
        --------------------------------------
        lock API calls:          5520 (100.0%)

(here it is visible that apparently a significant amount of [i believe
necessary] locking is missing from this kernel.)

finally, here are the stats from the new PREEMPT_REALTIME -U0 kernel:

        spinlock API calls:       491  (6.0%)
        | old mutex API calls:   2142  (26.2%)
        | new mutex API calls:   5536  (67.7%)
        all mutex API calls:     7678  (93.9%)
        --------------------------------------
        lock API calls:          8169 (100.0%)

note that almost all of the remaining spinlocks are held for a short
amount of time and have a finite maximum duration. They involve hardware
access, scheduling and interrupt handling and timers - almost all of
that code has O(1) characteristics.

what this means is that we are approaching hard-real-time domains ... 
using what is in essence the stock Linux kernel!

note that priority inheritance is still not part of this patch, but that
effort can now be centralized to the two basic Linux semaphore types,
solving the full spectrum of priority inheritance problems!

the code is still x86-only but only for practical reasons - other
architectures will be covered in the future as well.

to create a -U0 tree from scratch the patching order is:

   http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
 + http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
 + http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
 + http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U0

	Ingo

[patch] Real-Time Preemption, -VP-2.6.9-rc4-mm1-U1

[Ingo Molnar]

i have released the -U1 PREEMPT_REALTIME patch:
 
  http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U1

this is a strict bugfixes-only release. With -U1 i cannot reproduce any
of the bugs on my testsystems anymore, but take care nevertheless, this
is still experimental code.

Changes since -U0:

 - bugfix: fixed the highmem related crash reported by Adam Heath and i 
   think this could also fix the crash reported by Mark H Johnson.

 - bugfix: fixed a number of networking related soft-lockups, caused by
   a deadlock scenarios in the ipv4, netfilter and net-xmit locking
   code. This could fix the lockup reported by Lorenzo Allegrucci.

 - bugfix: enable interrupts in the int3 handler - gdb will otherwise
   trigger a kernel debug message.

 - cleanup: reworked the RCU API wrappers, we now have the following
   variants:
 
     rcu_read_[un]lock_spin(&spinlock)
     rcu_read_[un]lock_bh_spin(&spinlock)
     rcu_read_[un]lock_sem(&semaphore)

   this change was necessary for the network locking fixes.

 - debugging helper: SysRq-T will now print the stacktrace of currently
   running tasks too. (They might be a bit unreliable occasionally but
   very useful to debug deadlocks.)

 - configurability fix: disabled the /proc/kernel/softirq_preemption and
   hardirq_preemption runtime flags (and the softirq-preempt= and
   hardirq-preempt= boot flags) if PREEMPT_REALTIME is enabled - in the
   fully preemptible model these must always be on.

there are no known bugs at this moment, so please re-report any issues 
you might still encounter.

to create a -U1 tree from scratch the patching order is:

   http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
 + http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
 + http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
 + http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U1

	Ingo

[patch] Real-Time Preemption, -VP-2.6.9-rc4-mm1-U2

[Ingo Molnar]

i have released the -U2 PREEMPT_REALTIME patch:

  http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U2

this too is a bugfixes-only release, and it is still experimental code.

Changes since -U1:

 - bugfix: fix page_lock_anon_vma() crash reported by Adam Heath and 
   Lorenzo Allegrucci.

 - bugfix: fix selinux atomic-schedule warning messages, reported by
   Mark H Johnson.

 - bugfix: ip_tables atomic-schedule fix, fixes the messages reported by 
   Daniel Walker and K.R. Foley.

 - bugfix: fix warnings/deadlocks in inet_create(), reported by Mark H 
   Johnson.

 - bugfix: fixed a crash-in-shmfs-during-heavy-swapout bug

 - bugfix: enable preemption while doing mmdrop() in the scheduler - it 
   may schedule.

 - debugging feature: when PREEMPT_TIMING is enabled then the code also 
   keeps a trace/stack of preemption enabler EIPs. (if LATENCY_TRACE is 
   enabled as well then the parent EIP is recorded as well.) Whenever a 
   stack trace due to atomicity violations is printed, the preemption
   stack is printed as well. This makes it much easier to identify the 
   place that did the illegal preemption-disabling.

to create a -U2 tree from scratch the patching order is:

   http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
 + http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
 + http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
 + http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U2

	Ingo

[patch] Real-Time Preemption, -VP-2.6.9-rc4-mm1-U3

[Ingo Molnar]

i have released the -U3 PREEMPT_REALTIME patch:

  http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U3

this is a buildfixes-only release, and it is still experimental code.

Changes since -U2:

 - build fix: fixes the latency.c compilation error reported by Adam 
   Heath.

 - build fix: fixes !HIGHMEM compilation, patch from Andrew Rodland

to create a -U3 tree from scratch the patching order is:

   http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
 + http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
 + http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
 + http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U3

	Ingo

[patch] Real-Time Preemption, -VP-2.6.9-rc4-mm1-U4

[Ingo Molnar]

i have released the -U4 PREEMPT_REALTIME patch:

   http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U4

this is a fixes-only release, and it is still experimental code.

Changes since -U3:

 - crash fix: fix the rtc_lock related crash reported by Bill Huey, the 
   rtc_lock is now a raw spinlock again.

 - crash fix: avoid recursion into timer code when PREEMPT_TIMING is 
   enabled.

 - crash/printout fix: revert some of selinux's locks back to raw 
   spinlocks. This could fix the problems reported by Mark H. Johnson, 
   Adam Heath.

 - build fix: fix the compilation problems reported by Lee Revell

 - debug feature: implemented 'print backtrace on all CPUs' on SMP 
   systems, SysRq+L will trigger it.

 - build cleanup: restructure the debug config options. This should 
   resolve the build problems and incompatible debug-options
   problems reported.

 - cleanup: move definitions around, turn on generic rwlocks instead of
   the x86-specific version.

i think all bugs that were reported with logs are fixed in -U4. Please
re-report any issue that might remain.

to create a -U4 tree from scratch the patching order is:

   http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
 + http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
 + http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
 + http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U4

	Ingo

[patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

i have released the -U5 Real-Time Preemption patch:

  http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5

this is a release intended to increase stability, but since it also
includes new debug features and related cleanups it might introduce new
regressions. Be careful.

there are two big changes:

 - debug feature: automatic semaphore/rwsem deadlock detection, based on
   the code from Igor Manyilov and Bill Huey.

this is a very nice feature that should help in debugging the remaining
deadlocks. The deadlock detection feature has already helped me fix a
bug that was causing hangs in the VFS, so it's really useful.

 - generic semaphore implementation

the generic semaphore implementation (which uses rwsems) makes it
possible to use the deadlock detection mechanism for all the mutex types
we currently have: semaphores, rw-semaphores, spinlock-mutexes and
rwlock-mutexes. Another benefit is that PREEMPT_REALTIME becomes much
more portable this way. (although it's still x86-only at the moment.)

other changes since -U4:

 - crash fix: fixed a possible "unbound recursion upon IRQ entry" bug.
   introduced preempt_schedule_irq() which now schedules without
   enabling interrupts again, preventing new IRQs from hitting this
   task again and triggering preemption. This might fix the
   'infinite stackdumps' problem Rui Nuno Capela was seeing.

 - deadlock fix: is_subdir()'s PREEMPT_REALTIME locking was buggy. This 
   could perhaps fix the other problem reported by Rui Nuno Capela.

 - i8253_lock fixes: apm, hd.c, gameport.c and analog.c were all 
   improperly importing the variable while overriding the prototype. 
   This fixes the bug reported by Florian Schmidt.

 - possible crash fix: one particular lock in selinux has to be 
   mutex-based, because while held it calls other mutex-using code.

 - two more selinux locks converted to raw spinlocks, because they were
   called from within raw-critical sections.

 - debug feature: enforce interrupts-enabled upon schedule().
   (Note that this does not break sleep_on() because sleep_on() does not
   disable interrupts in the PREEMPT_REALTIME mode. It might break with 
   !PREEMPT_REALTIME though.)

 - locking cleanup: converted the IPC code from raw spinlocks & RCU to
   spinlock-mutexes.

 - code cleanup: cleaned up the generic rwsem code.

 - debug feature: implemented /proc/sys/kernel/trace_verbose runtime
   flag (default:0), which enables a much more verbose printout in
   /proc/latency_trace.  This trace format can be useful in e.g. 
   debugging timestamp weirdnesses.

 - irqs-off fix: there was one codepath where irqd would call schedule() 
   with interrupts disabled.

 - debug feature: the NMI entries in the latency trace now also include 
   the last-observed-EFLAGS value. Can be useful in figuring out what a
   certain CPU is doing and why.

 - cleanup: fixed preemption-off ordering: often the spinlock (and 
   scheduler) code would re-enable preemption and interrupts in the
   wrong order, opening up a small window for an interrupt handler to
   fit in and increase the latency of that almost-finished critical
   section.

 - cleanup: consolidated various bug-printouts. It should now be easy to
   find whether anything bad happens even amongst lots of preempt-timing
   printouts: 'dmesg | grep BUG'.

to create a -U5 tree from scratch, the patching order is:

   http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
 + http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
 + http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
 + http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5

	Ingo

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Jason Munro]

[-- Attachment #1: Type: text/plain, Size: 798 bytes --]

On 9:50:08 am 10/18/04 Ingo Molnar <mingo@elte.hu> wrote:
>
> i have released the -U5 Real-Time Preemption patch:
>
>   http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-r
> c4-mm1-U5

Im getting this on a make bzImage:

  CC      fs/xfs/xfs_inode_item.o
fs/xfs/xfs_inode_item.c: In function `xfs_inode_item_pushbuf':
fs/xfs/xfs_inode_item.c:803: error: structure has no member named `count'
fs/xfs/xfs_inode_item.c:825: error: structure has no member named `count'
make[2]: *** [fs/xfs/xfs_inode_item.o] Error 1

I'm currently running 2.6.9-rc4-mm1-U1 without error. I backed out the U1
patch, applied the new one, did a make oldconfig, make clean, make bzImage,
make modules. My config is attached. 

\__ Jason Munro
 \__ jason@stdbev.com
  \__ http://hastymail.sourceforge.net/

[-- Attachment #2: config --]
[-- Type: text/plain, Size: 35188 bytes --]

#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.9-rc4-mm1-RT-U5
# Mon Oct 18 10:23:45 2004
#
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y

#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
# CONFIG_CLEAN_COMPILE is not set
CONFIG_BROKEN=y
CONFIG_BROKEN_ON_SMP=y

#
# General setup
#
CONFIG_LOCALVERSION=""
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
# CONFIG_POSIX_MQUEUE is not set
# CONFIG_BSD_PROCESS_ACCT is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
CONFIG_LOG_BUF_SHIFT=14
CONFIG_HOTPLUG=y
CONFIG_KOBJECT_UEVENT=y
# CONFIG_IKCONFIG is not set
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
CONFIG_KALLSYMS_EXTRA_PASS=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SHMEM=y
# CONFIG_TINY_SHMEM is not set

#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
CONFIG_OBSOLETE_MODPARM=y
# CONFIG_MODVERSIONS is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y

#
# Processor type and features
#
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
CONFIG_MPENTIUM4=y
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=7
CONFIG_RWSEM_GENERIC=y
CONFIG_GENERIC_SEMAPHORES=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
# CONFIG_SMP is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_BKL=y
CONFIG_PREEMPT_VOLUNTARY=y
CONFIG_PREEMPT_SOFTIRQS=y
CONFIG_PREEMPT_HARDIRQS=y
CONFIG_PREEMPT_REALTIME=y
# CONFIG_X86_UP_APIC is not set
CONFIG_X86_TSC=y
CONFIG_X86_MCE=y
# CONFIG_X86_MCE_NONFATAL is not set
CONFIG_TOSHIBA=y
# CONFIG_I8K is not set
# CONFIG_MICROCODE is not set
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set

#
# Firmware Drivers
#
# CONFIG_EDD is not set
CONFIG_NOHIGHMEM=y
# CONFIG_HIGHMEM4G is not set
# CONFIG_HIGHMEM64G is not set
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
# CONFIG_EFI is not set
CONFIG_HAVE_DEC_LOCK=y
CONFIG_REGPARM=y

#
# Performance-monitoring counters support
#
# CONFIG_PERFCTR is not set
CONFIG_KEXEC=y
# CONFIG_CRASH_DUMP is not set

#
# Power management options (ACPI, APM)
#
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
# CONFIG_SOFTWARE_SUSPEND is not set

#
# ACPI (Advanced Configuration and Power Interface) Support
#
CONFIG_ACPI=y
CONFIG_ACPI_BOOT=y
CONFIG_ACPI_INTERPRETER=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_SLEEP_PROC_FS=y
CONFIG_ACPI_AC=y
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_FAN=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_ASUS is not set
CONFIG_ACPI_THINKPAD=m
CONFIG_ACPI_TOSHIBA=y
# CONFIG_ACPI_CUSTOM_DSDT is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_BUS=y
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_PCI=y
CONFIG_ACPI_SYSTEM=y
# CONFIG_X86_PM_TIMER is not set

#
# APM (Advanced Power Management) BIOS Support
#
# CONFIG_APM is not set

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
# CONFIG_CPU_FREQ_PROC_INTF is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_PERFORMANCE=m
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=y
# CONFIG_CPU_FREQ_24_API is not set
# CONFIG_CPU_FREQ_GOV_ONDEMAND is not set
CONFIG_CPU_FREQ_TABLE=y

#
# CPUFreq processor drivers
#
# CONFIG_X86_ACPI_CPUFREQ is not set
# CONFIG_X86_POWERNOW_K6 is not set
# CONFIG_X86_POWERNOW_K7 is not set
# CONFIG_X86_POWERNOW_K8 is not set
# CONFIG_X86_GX_SUSPMOD is not set
# CONFIG_X86_SPEEDSTEP_CENTRINO is not set
# CONFIG_X86_SPEEDSTEP_ICH is not set
# CONFIG_X86_SPEEDSTEP_SMI is not set
CONFIG_X86_P4_CLOCKMOD=y
# CONFIG_X86_CPUFREQ_NFORCE2 is not set
# CONFIG_X86_LONGRUN is not set
# CONFIG_X86_LONGHAUL is not set

#
# shared options
#
CONFIG_X86_SPEEDSTEP_LIB=y

#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
# CONFIG_PCI_LEGACY_PROC is not set
CONFIG_PCI_NAMES=y
CONFIG_ISA=y
# CONFIG_EISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set

#
# PCMCIA/CardBus support
#
CONFIG_PCMCIA=y
# CONFIG_PCMCIA_DEBUG is not set
CONFIG_YENTA=y
CONFIG_CARDBUS=y
# CONFIG_PD6729 is not set
# CONFIG_I82092 is not set
# CONFIG_I82365 is not set
# CONFIG_TCIC is not set
CONFIG_PCMCIA_PROBE=y

#
# PCI Hotplug Support
#
# CONFIG_HOTPLUG_PCI is not set

#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_AOUT=y
CONFIG_BINFMT_MISC=y

#
# Device Drivers
#

#
# Generic Driver Options
#
# CONFIG_STANDALONE is not set
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=m
# CONFIG_DEBUG_DRIVER is not set

#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set

#
# Parallel port support
#
CONFIG_PARPORT=m
CONFIG_PARPORT_PC=m
CONFIG_PARPORT_PC_CML1=m
# CONFIG_PARPORT_SERIAL is not set
# CONFIG_PARPORT_PC_FIFO is not set
# CONFIG_PARPORT_PC_SUPERIO is not set
# CONFIG_PARPORT_PC_PCMCIA is not set
# CONFIG_PARPORT_OTHER is not set
CONFIG_PARPORT_1284=y

#
# Plug and Play support
#
# CONFIG_PNP is not set

#
# Block devices
#
CONFIG_BLK_DEV_FD=m
# CONFIG_BLK_DEV_XD is not set
# CONFIG_PARIDE is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
CONFIG_BLK_DEV_LOOP=m
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
# CONFIG_LBD is not set
# CONFIG_CDROM_PKTCDVD is not set

#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y

#
# Please see Documentation/ide.txt for help/info on IDE drives
#
# CONFIG_BLK_DEV_IDE_SATA is not set
# CONFIG_BLK_DEV_HD_IDE is not set
CONFIG_BLK_DEV_IDEDISK=y
CONFIG_IDEDISK_MULTI_MODE=y
CONFIG_BLK_DEV_IDECS=m
CONFIG_BLK_DEV_IDECD=y
# CONFIG_BLK_DEV_IDETAPE is not set
CONFIG_BLK_DEV_IDEFLOPPY=m
# CONFIG_BLK_DEV_IDESCSI is not set
# CONFIG_IDE_TASK_IOCTL is not set

#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=y
# CONFIG_BLK_DEV_CMD640 is not set
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_SHARE_IRQ=y
# CONFIG_BLK_DEV_OFFBOARD is not set
CONFIG_BLK_DEV_GENERIC=y
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_RZ1000 is not set
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not set
# CONFIG_BLK_DEV_AMD74XX is not set
# CONFIG_BLK_DEV_ATIIXP is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5520 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_SC1200 is not set
CONFIG_BLK_DEV_PIIX=y
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_BLK_DEV_SVWKS is not set
# CONFIG_BLK_DEV_SIIMAGE is not set
# CONFIG_BLK_DEV_SIS5513 is not set
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
# CONFIG_BLK_DEV_VIA82CXXX is not set
# CONFIG_IDE_ARM is not set
# CONFIG_IDE_CHIPSETS is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set

#
# SCSI device support
#
CONFIG_SCSI=y
# CONFIG_SCSI_PROC_FS is not set

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=m
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
# CONFIG_BLK_DEV_SR is not set
CONFIG_CHR_DEV_SG=m

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
# CONFIG_SCSI_MULTI_LUN is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set

#
# SCSI Transport Attributes
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set

#
# SCSI low-level drivers
#
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_7000FASST is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AHA152X is not set
# CONFIG_SCSI_AHA1542 is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_SCSI_AIC7XXX is not set
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_SCSI_ADVANSYS is not set
# CONFIG_SCSI_IN2000 is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
# CONFIG_SCSI_SATA is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_SCSI_CPQFCTS is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_DTC3280 is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_EATA_PIO is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_GENERIC_NCR5380 is not set
# CONFIG_SCSI_GENERIC_NCR5380_MMIO is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_PPA is not set
# CONFIG_SCSI_IMM is not set
# CONFIG_SCSI_NCR53C406A is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_PAS16 is not set
# CONFIG_SCSI_PCI2000 is not set
# CONFIG_SCSI_PCI2220I is not set
# CONFIG_SCSI_PSI240I is not set
# CONFIG_SCSI_QLOGIC_FAS is not set
# CONFIG_SCSI_QLOGIC_ISP is not set
# CONFIG_SCSI_QLOGIC_FC is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
# CONFIG_SCSI_QLOGIC_1280_1040 is not set
CONFIG_SCSI_QLA2XXX=y
# CONFIG_SCSI_QLA21XX is not set
# CONFIG_SCSI_QLA22XX is not set
# CONFIG_SCSI_QLA2300 is not set
# CONFIG_SCSI_QLA2322 is not set
# CONFIG_SCSI_QLA6312 is not set
# CONFIG_SCSI_QLA6322 is not set
# CONFIG_SCSI_SEAGATE is not set
# CONFIG_SCSI_SYM53C416 is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_T128 is not set
# CONFIG_SCSI_U14_34F is not set
# CONFIG_SCSI_ULTRASTOR is not set
# CONFIG_SCSI_NSP32 is not set
# CONFIG_SCSI_DEBUG is not set

#
# PCMCIA SCSI adapter support
#
# CONFIG_PCMCIA_AHA152X is not set
# CONFIG_PCMCIA_FDOMAIN is not set
# CONFIG_PCMCIA_NINJA_SCSI is not set
# CONFIG_PCMCIA_QLOGIC is not set
# CONFIG_PCMCIA_SYM53C500 is not set

#
# Old CD-ROM drivers (not SCSI, not IDE)
#
# CONFIG_CD_NO_IDESCSI is not set

#
# Multi-device support (RAID and LVM)
#
# CONFIG_MD is not set

#
# Fusion MPT device support
#
# CONFIG_FUSION is not set

#
# IEEE 1394 (FireWire) support
#
# CONFIG_IEEE1394 is not set

#
# I2O device support
#
# CONFIG_I2O is not set

#
# Networking support
#
CONFIG_NET=y

#
# Networking options
#
CONFIG_PACKET=y
# CONFIG_PACKET_MMAP is not set
# CONFIG_NETLINK_DEV is not set
CONFIG_UNIX=y
CONFIG_NET_KEY=m
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
# CONFIG_IP_ADVANCED_ROUTER is not set
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_IP_MROUTE is not set
# CONFIG_ARPD is not set
# CONFIG_SYN_COOKIES is not set
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_IPCOMP=m
CONFIG_INET_TUNNEL=m

#
# IP: Virtual Server Configuration
#
# CONFIG_IP_VS is not set
# CONFIG_IPV6 is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set

#
# IP: Netfilter Configuration
#
# CONFIG_IP_NF_CONNTRACK is not set
# CONFIG_IP_NF_QUEUE is not set
# CONFIG_IP_NF_IPTABLES is not set
# CONFIG_IP_NF_ARPTABLES is not set
# CONFIG_IP_NF_COMPAT_IPCHAINS is not set
# CONFIG_IP_NF_COMPAT_IPFWADM is not set
CONFIG_XFRM=y
CONFIG_XFRM_USER=m

#
# SCTP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_SCTP is not set
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_HW_FLOWCONTROL is not set

#
# QoS and/or fair queueing
#
# CONFIG_NET_SCHED is not set
# CONFIG_NET_CLS_ROUTE is not set

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_KGDBOE is not set
# CONFIG_NETPOLL is not set
# CONFIG_NETPOLL_RX is not set
# CONFIG_NETPOLL_TRAP is not set
# CONFIG_NET_POLL_CONTROLLER is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set

#
# ARCnet devices
#
# CONFIG_ARCNET is not set

#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
CONFIG_MII=y
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
# CONFIG_NET_VENDOR_3COM is not set
# CONFIG_LANCE is not set
# CONFIG_NET_VENDOR_SMC is not set
# CONFIG_NET_VENDOR_RACAL is not set

#
# Tulip family network device support
#
# CONFIG_NET_TULIP is not set
# CONFIG_AT1700 is not set
# CONFIG_DEPCA is not set
# CONFIG_HP100 is not set
# CONFIG_NET_ISA is not set
CONFIG_NET_PCI=y
# CONFIG_PCNET32 is not set
# CONFIG_AMD8111_ETH is not set
# CONFIG_ADAPTEC_STARFIRE is not set
# CONFIG_AC3200 is not set
# CONFIG_APRICOT is not set
# CONFIG_B44 is not set
# CONFIG_FORCEDETH is not set
# CONFIG_CS89x0 is not set
# CONFIG_DGRS is not set
CONFIG_EEPRO100=y
# CONFIG_E100 is not set
# CONFIG_FEALNX is not set
# CONFIG_NATSEMI is not set
# CONFIG_NE2K_PCI is not set
# CONFIG_8139CP is not set
# CONFIG_8139TOO is not set
# CONFIG_SIS900 is not set
# CONFIG_EPIC100 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_TLAN is not set
# CONFIG_VIA_RHINE is not set
# CONFIG_NET_POCKET is not set

#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SK98LIN is not set
# CONFIG_VIA_VELOCITY is not set
# CONFIG_TIGON3 is not set

#
# Ethernet (10000 Mbit)
#
# CONFIG_IXGB is not set
# CONFIG_S2IO is not set

#
# Token Ring devices
#
# CONFIG_TR is not set

#
# Wireless LAN (non-hamradio)
#
CONFIG_NET_RADIO=y

#
# Obsolete Wireless cards support (pre-802.11)
#
# CONFIG_STRIP is not set
# CONFIG_ARLAN is not set
# CONFIG_WAVELAN is not set
# CONFIG_PCMCIA_WAVELAN is not set
# CONFIG_PCMCIA_NETWAVE is not set

#
# Wireless 802.11 Frequency Hopping cards support
#
# CONFIG_PCMCIA_RAYCS is not set

#
# Wireless 802.11b ISA/PCI cards support
#
# CONFIG_AIRO is not set
# CONFIG_HERMES is not set
# CONFIG_ATMEL is not set

#
# Wireless 802.11b Pcmcia/Cardbus cards support
#
# CONFIG_AIRO_CS is not set
# CONFIG_PCMCIA_WL3501 is not set

#
# Prism GT/Duette 802.11(a/b/g) PCI/Cardbus support
#
# CONFIG_PRISM54 is not set
CONFIG_NET_WIRELESS=y

#
# PCMCIA network device support
#
CONFIG_NET_PCMCIA=y
CONFIG_PCMCIA_3C589=m
CONFIG_PCMCIA_3C574=m
CONFIG_PCMCIA_FMVJ18X=m
CONFIG_PCMCIA_PCNET=m
CONFIG_PCMCIA_NMCLAN=m
CONFIG_PCMCIA_SMC91C92=m
CONFIG_PCMCIA_XIRC2PS=m
CONFIG_PCMCIA_AXNET=m

#
# Wan interfaces
#
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PLIP is not set
CONFIG_PPP=m
CONFIG_PPP_MULTILINK=y
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_BSDCOMP=m
# CONFIG_PPPOE is not set
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
# CONFIG_SHAPER is not set
# CONFIG_NETCONSOLE is not set

#
# ISDN subsystem
#
# CONFIG_ISDN is not set

#
# Telephony Support
#
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_TSDEV is not set
CONFIG_INPUT_EVDEV=m
# CONFIG_INPUT_EVBUG is not set

#
# Input I/O drivers
#
# CONFIG_GAMEPORT is not set
CONFIG_SOUND_GAMEPORT=y
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PARKBD is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=m
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_INPORT is not set
# CONFIG_MOUSE_LOGIBM is not set
# CONFIG_MOUSE_PC110PAD is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_SERIAL_NONSTANDARD is not set

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
# CONFIG_SERIAL_8250_CONSOLE is not set
CONFIG_SERIAL_8250_CS=m
# CONFIG_SERIAL_8250_ACPI is not set
CONFIG_SERIAL_8250_NR_UARTS=4
# CONFIG_SERIAL_8250_EXTENDED is not set

#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
CONFIG_PRINTER=m
# CONFIG_LP_CONSOLE is not set
# CONFIG_PPDEV is not set
# CONFIG_TIPAR is not set

#
# IPMI
#
# CONFIG_IPMI_HANDLER is not set

#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_NVRAM is not set
CONFIG_RTC=y
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set

#
# Ftape, the floppy tape device driver
#
# CONFIG_FTAPE is not set
CONFIG_AGP=y
# CONFIG_AGP_ALI is not set
# CONFIG_AGP_ATI is not set
# CONFIG_AGP_AMD is not set
# CONFIG_AGP_AMD64 is not set
CONFIG_AGP_INTEL=y
# CONFIG_AGP_INTEL_MCH is not set
# CONFIG_AGP_NVIDIA is not set
# CONFIG_AGP_SIS is not set
# CONFIG_AGP_SWORKS is not set
# CONFIG_AGP_VIA is not set
# CONFIG_AGP_EFFICEON is not set
# CONFIG_DRM is not set

#
# PCMCIA character devices
#
# CONFIG_SYNCLINK_CS is not set
# CONFIG_MWAVE is not set
# CONFIG_RAW_DRIVER is not set
# CONFIG_HPET is not set
# CONFIG_HANGCHECK_TIMER is not set

#
# I2C support
#
CONFIG_I2C=m
# CONFIG_I2C_CHARDEV is not set

#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=m
# CONFIG_I2C_ALGOPCF is not set
# CONFIG_I2C_ALGOPCA is not set

#
# I2C Hardware Bus support
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
# CONFIG_I2C_AMD756 is not set
# CONFIG_I2C_AMD8111 is not set
# CONFIG_I2C_ELEKTOR is not set
# CONFIG_I2C_I801 is not set
# CONFIG_I2C_I810 is not set
# CONFIG_I2C_ISA is not set
# CONFIG_I2C_NFORCE2 is not set
# CONFIG_I2C_PARPORT is not set
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_PIIX4 is not set
# CONFIG_I2C_PROSAVAGE is not set
# CONFIG_I2C_SAVAGE4 is not set
# CONFIG_SCx200_ACB is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_STUB is not set
# CONFIG_I2C_VIA is not set
# CONFIG_I2C_VIAPRO is not set
# CONFIG_I2C_VOODOO3 is not set
# CONFIG_I2C_PCA_ISA is not set

#
# Hardware Sensors Chip support
#
# CONFIG_I2C_SENSOR is not set
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ADM1025 is not set
# CONFIG_SENSORS_ADM1031 is not set
# CONFIG_SENSORS_ASB100 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_FSCHER is not set
# CONFIG_SENSORS_GL518SM is not set
# CONFIG_SENSORS_IT87 is not set
# CONFIG_SENSORS_LM75 is not set
# CONFIG_SENSORS_LM77 is not set
# CONFIG_SENSORS_LM78 is not set
# CONFIG_SENSORS_LM80 is not set
# CONFIG_SENSORS_LM83 is not set
# CONFIG_SENSORS_LM85 is not set
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_MAX1619 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_VIA686A is not set
# CONFIG_SENSORS_W83781D is not set
# CONFIG_SENSORS_W83L785TS is not set
# CONFIG_SENSORS_W83627HF is not set

#
# Other I2C Chip support
#
# CONFIG_SENSORS_EEPROM is not set
# CONFIG_SENSORS_PCF8574 is not set
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_SENSORS_RTC8564 is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set

#
# Dallas's 1-wire bus
#
# CONFIG_W1 is not set

#
# Misc devices
#
# CONFIG_IBM_ASM is not set

#
# Multimedia devices
#
CONFIG_VIDEO_DEV=m

#
# Video For Linux
#

#
# Video Adapters
#
# CONFIG_VIDEO_BT848 is not set
# CONFIG_VIDEO_PMS is not set
# CONFIG_VIDEO_BWQCAM is not set
# CONFIG_VIDEO_CQCAM is not set
# CONFIG_VIDEO_W9966 is not set
# CONFIG_VIDEO_CPIA is not set
# CONFIG_VIDEO_SAA5246A is not set
# CONFIG_VIDEO_SAA5249 is not set
# CONFIG_TUNER_3036 is not set
# CONFIG_VIDEO_STRADIS is not set
# CONFIG_VIDEO_ZORAN is not set
# CONFIG_VIDEO_ZR36120 is not set
# CONFIG_VIDEO_SAA7134 is not set
# CONFIG_VIDEO_MXB is not set
# CONFIG_VIDEO_DPC is not set
# CONFIG_VIDEO_HEXIUM_ORION is not set
# CONFIG_VIDEO_HEXIUM_GEMINI is not set
# CONFIG_VIDEO_CX88 is not set
# CONFIG_VIDEO_OVCAMCHIP is not set

#
# Radio Adapters
#
# CONFIG_RADIO_CADET is not set
# CONFIG_RADIO_RTRACK is not set
# CONFIG_RADIO_RTRACK2 is not set
# CONFIG_RADIO_AZTECH is not set
# CONFIG_RADIO_GEMTEK is not set
# CONFIG_RADIO_GEMTEK_PCI is not set
# CONFIG_RADIO_MAXIRADIO is not set
# CONFIG_RADIO_MAESTRO is not set
# CONFIG_RADIO_SF16FMI is not set
# CONFIG_RADIO_SF16FMR2 is not set
# CONFIG_RADIO_TERRATEC is not set
# CONFIG_RADIO_TRUST is not set
# CONFIG_RADIO_TYPHOON is not set
# CONFIG_RADIO_ZOLTRIX is not set

#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set

#
# Graphics support
#
CONFIG_FB=y
CONFIG_FB_MODE_HELPERS=y
# CONFIG_FB_TILEBLITTING is not set
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not set
CONFIG_FB_VESA=y
CONFIG_VIDEO_SELECT=y
# CONFIG_FB_HGA is not set
CONFIG_FB_RIVA=m
CONFIG_FB_RIVA_I2C=y
# CONFIG_FB_RIVA_DEBUG is not set
# CONFIG_FB_I810 is not set
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON_OLD is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_VIRTUAL is not set

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
# CONFIG_MDA_CONSOLE is not set
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
CONFIG_FONTS=y
# CONFIG_FONT_8x8 is not set
# CONFIG_FONT_8x16 is not set
CONFIG_FONT_6x11=y
# CONFIG_FONT_PEARL_8x8 is not set
# CONFIG_FONT_ACORN_8x8 is not set
# CONFIG_FONT_MINI_4x6 is not set
# CONFIG_FONT_SUN8x16 is not set
# CONFIG_FONT_SUN12x22 is not set

#
# Logo configuration
#
# CONFIG_LOGO is not set

#
# Sound
#
CONFIG_SOUND=y

#
# Advanced Linux Sound Architecture
#
CONFIG_SND=y
CONFIG_SND_TIMER=y
CONFIG_SND_PCM=y
CONFIG_SND_SEQUENCER=m
# CONFIG_SND_SEQ_DUMMY is not set
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=m
CONFIG_SND_PCM_OSS=m
CONFIG_SND_SEQUENCER_OSS=y
CONFIG_SND_RTCTIMER=y
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set

#
# Generic devices
#
# CONFIG_SND_DUMMY is not set
# CONFIG_SND_VIRMIDI is not set
# CONFIG_SND_MTPAV is not set
# CONFIG_SND_SERIAL_U16550 is not set
# CONFIG_SND_MPU401 is not set

#
# ISA devices
#
# CONFIG_SND_AD1848 is not set
# CONFIG_SND_CS4231 is not set
# CONFIG_SND_CS4232 is not set
# CONFIG_SND_CS4236 is not set
# CONFIG_SND_ES1688 is not set
# CONFIG_SND_ES18XX is not set
# CONFIG_SND_GUSCLASSIC is not set
# CONFIG_SND_GUSEXTREME is not set
# CONFIG_SND_GUSMAX is not set
# CONFIG_SND_INTERWAVE is not set
# CONFIG_SND_INTERWAVE_STB is not set
# CONFIG_SND_OPTI92X_AD1848 is not set
# CONFIG_SND_OPTI92X_CS4231 is not set
# CONFIG_SND_OPTI93X is not set
# CONFIG_SND_SB8 is not set
# CONFIG_SND_SB16 is not set
# CONFIG_SND_SBAWE is not set
# CONFIG_SND_WAVEFRONT is not set
# CONFIG_SND_CMI8330 is not set
# CONFIG_SND_OPL3SA2 is not set
# CONFIG_SND_SGALAXY is not set
# CONFIG_SND_SSCAPE is not set

#
# PCI devices
#
CONFIG_SND_AC97_CODEC=y
# CONFIG_SND_ALI5451 is not set
# CONFIG_SND_ATIIXP is not set
# CONFIG_SND_ATIIXP_MODEM is not set
# CONFIG_SND_AU8810 is not set
# CONFIG_SND_AU8820 is not set
# CONFIG_SND_AU8830 is not set
# CONFIG_SND_AZT3328 is not set
# CONFIG_SND_BT87X is not set
# CONFIG_SND_CS46XX is not set
# CONFIG_SND_CS4281 is not set
# CONFIG_SND_EMU10K1 is not set
# CONFIG_SND_KORG1212 is not set
# CONFIG_SND_MIXART is not set
# CONFIG_SND_NM256 is not set
# CONFIG_SND_RME32 is not set
# CONFIG_SND_RME96 is not set
# CONFIG_SND_RME9652 is not set
# CONFIG_SND_HDSP is not set
# CONFIG_SND_TRIDENT is not set
# CONFIG_SND_YMFPCI is not set
# CONFIG_SND_ALS4000 is not set
# CONFIG_SND_CMIPCI is not set
# CONFIG_SND_ENS1370 is not set
# CONFIG_SND_ENS1371 is not set
# CONFIG_SND_ES1938 is not set
# CONFIG_SND_ES1968 is not set
# CONFIG_SND_MAESTRO3 is not set
# CONFIG_SND_FM801 is not set
# CONFIG_SND_ICE1712 is not set
# CONFIG_SND_ICE1724 is not set
CONFIG_SND_INTEL8X0=y
# CONFIG_SND_INTEL8X0M is not set
# CONFIG_SND_SONICVIBES is not set
# CONFIG_SND_VIA82XX is not set
# CONFIG_SND_VX222 is not set

#
# USB devices
#
# CONFIG_SND_USB_AUDIO is not set
# CONFIG_SND_USB_USX2Y is not set

#
# PCMCIA devices
#
# CONFIG_SND_VXPOCKET is not set
# CONFIG_SND_VXP440 is not set
# CONFIG_SND_PDAUDIOCF is not set

#
# Open Sound System
#
# CONFIG_SOUND_PRIME is not set

#
# USB support
#
CONFIG_USB=y
# CONFIG_USB_DEBUG is not set

#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
CONFIG_USB_BANDWIDTH=y
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_SUSPEND is not set
# CONFIG_USB_OTG is not set

#
# USB Host Controller Drivers
#
CONFIG_USB_EHCI_HCD=y
# CONFIG_USB_EHCI_SPLIT_ISO is not set
# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
# CONFIG_USB_OHCI_HCD is not set
CONFIG_USB_UHCI_HCD=y

#
# USB Device Class drivers
#
# CONFIG_USB_AUDIO is not set
# CONFIG_USB_BLUETOOTH_TTY is not set
# CONFIG_USB_MIDI is not set
# CONFIG_USB_ACM is not set
# CONFIG_USB_PRINTER is not set
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_RW_DETECT is not set
# CONFIG_USB_STORAGE_DATAFAB is not set
# CONFIG_USB_STORAGE_FREECOM is not set
# CONFIG_USB_STORAGE_ISD200 is not set
# CONFIG_USB_STORAGE_DPCM is not set
# CONFIG_USB_STORAGE_HP8200e is not set
# CONFIG_USB_STORAGE_SDDR09 is not set
# CONFIG_USB_STORAGE_SDDR55 is not set
# CONFIG_USB_STORAGE_JUMPSHOT is not set

#
# USB Human Interface Devices (HID)
#
CONFIG_USB_HID=y
CONFIG_USB_HIDINPUT=y
# CONFIG_HID_FF is not set
# CONFIG_USB_HIDDEV is not set
# CONFIG_USB_AIPTEK is not set
# CONFIG_USB_WACOM is not set
# CONFIG_USB_KBTAB is not set
# CONFIG_USB_POWERMATE is not set
# CONFIG_USB_MTOUCH is not set
# CONFIG_USB_EGALAX is not set
# CONFIG_USB_XPAD is not set
# CONFIG_USB_ATI_REMOTE is not set

#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set
# CONFIG_USB_HPUSBSCSI is not set

#
# USB Multimedia devices
#
# CONFIG_USB_DABUSB is not set
# CONFIG_USB_VICAM is not set
# CONFIG_USB_DSBR is not set
# CONFIG_USB_IBMCAM is not set
# CONFIG_USB_KONICAWC is not set
# CONFIG_USB_OV511 is not set
# CONFIG_USB_SE401 is not set
# CONFIG_USB_SN9C102 is not set
# CONFIG_USB_STV680 is not set

#
# USB Network adaptors
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_USBNET is not set

#
# USB port drivers
#
# CONFIG_USB_USS720 is not set

#
# USB Serial Converter support
#
CONFIG_USB_SERIAL=m
CONFIG_USB_SERIAL_GENERIC=y
# CONFIG_USB_SERIAL_BELKIN is not set
# CONFIG_USB_SERIAL_WHITEHEAT is not set
# CONFIG_USB_SERIAL_DIGI_ACCELEPORT is not set
# CONFIG_USB_SERIAL_EMPEG is not set
# CONFIG_USB_SERIAL_FTDI_SIO is not set
CONFIG_USB_SERIAL_VISOR=m
# CONFIG_USB_SERIAL_IPAQ is not set
# CONFIG_USB_SERIAL_IR is not set
# CONFIG_USB_SERIAL_EDGEPORT is not set
# CONFIG_USB_SERIAL_EDGEPORT_TI is not set
# CONFIG_USB_SERIAL_KEYSPAN_PDA is not set
# CONFIG_USB_SERIAL_KEYSPAN is not set
# CONFIG_USB_SERIAL_KLSI is not set
# CONFIG_USB_SERIAL_KOBIL_SCT is not set
# CONFIG_USB_SERIAL_MCT_U232 is not set
CONFIG_USB_SERIAL_PL2303=m
# CONFIG_USB_SERIAL_SAFE is not set
# CONFIG_USB_SERIAL_CYBERJACK is not set
# CONFIG_USB_SERIAL_XIRCOM is not set
# CONFIG_USB_SERIAL_OMNINET is not set

#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_TIGL is not set
# CONFIG_USB_AUERSWALD is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_PHIDGETSERVO is not set
# CONFIG_USB_TEST is not set

#
# USB ATM/DSL drivers
#

#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set

#
# File systems
#
CONFIG_EXT2_FS=m
# CONFIG_EXT2_FS_XATTR is not set
CONFIG_EXT3_FS=m
CONFIG_EXT3_FS_XATTR=y
# CONFIG_EXT3_FS_POSIX_ACL is not set
# CONFIG_EXT3_FS_SECURITY is not set
CONFIG_JBD=m
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=m
CONFIG_REISERFS_FS=m
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
# CONFIG_REISERFS_FS_XATTR is not set
# CONFIG_JFS_FS is not set
CONFIG_XFS_FS=y
# CONFIG_XFS_RT is not set
# CONFIG_XFS_QUOTA is not set
# CONFIG_XFS_SECURITY is not set
# CONFIG_XFS_POSIX_ACL is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not set
# CONFIG_QUOTA is not set
# CONFIG_AUTOFS_FS is not set
CONFIG_AUTOFS4_FS=y

#
# Caches
#
# CONFIG_FSCACHE is not set

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_ZISOFS_FS=y
CONFIG_UDF_FS=y
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
CONFIG_NTFS_RW=y

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
CONFIG_DEVFS_FS=y
CONFIG_DEVFS_MOUNT=y
# CONFIG_DEVFS_DEBUG is not set
# CONFIG_DEVPTS_FS_XATTR is not set
CONFIG_TMPFS=y
# CONFIG_TMPFS_XATTR is not set
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_RAMFS=y

#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set

#
# Network File Systems
#
CONFIG_NFS_FS=m
# CONFIG_NFS_V3 is not set
# CONFIG_NFS_V4 is not set
# CONFIG_NFS_DIRECTIO is not set
CONFIG_NFSD=m
# CONFIG_NFSD_V3 is not set
# CONFIG_NFSD_TCP is not set
CONFIG_LOCKD=m
CONFIG_EXPORTFS=m
CONFIG_SUNRPC=m
# CONFIG_RPCSEC_GSS_KRB5 is not set
# CONFIG_RPCSEC_GSS_SPKM3 is not set
CONFIG_SMB_FS=m
# CONFIG_SMB_NLS_DEFAULT is not set
CONFIG_CIFS=m
# CONFIG_CIFS_STATS is not set
# CONFIG_CIFS_XATTR is not set
# CONFIG_CIFS_POSIX is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set

#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y

#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
# CONFIG_NLS_ASCII is not set
CONFIG_NLS_ISO8859_1=y
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
# CONFIG_NLS_ISO8859_15 is not set
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
CONFIG_NLS_UTF8=y

#
# Profiling support
#
# CONFIG_PROFILING is not set

#
# Kernel hacking
#
CONFIG_DEBUG_KERNEL=y
CONFIG_MAGIC_SYSRQ=y
# CONFIG_DEBUG_SLAB is not set
# CONFIG_DEBUG_PREEMPT is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_PREEMPT_TIMING is not set
# CONFIG_RWSEM_DEADLOCK_DETECT is not set
# CONFIG_DEBUG_INFO is not set
# CONFIG_FRAME_POINTER is not set
CONFIG_EARLY_PRINTK=y
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_KPROBES is not set
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_PAGEALLOC is not set
CONFIG_4KSTACKS=y
# CONFIG_SCHEDSTATS is not set
# CONFIG_KGDB is not set

#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY is not set

#
# Cryptographic options
#
CONFIG_CRYPTO=y
CONFIG_CRYPTO_HMAC=y
# CONFIG_CRYPTO_NULL is not set
# CONFIG_CRYPTO_MD4 is not set
CONFIG_CRYPTO_MD5=m
CONFIG_CRYPTO_SHA1=m
# CONFIG_CRYPTO_SHA256 is not set
# CONFIG_CRYPTO_SHA512 is not set
# CONFIG_CRYPTO_WP512 is not set
CONFIG_CRYPTO_DES=m
# CONFIG_CRYPTO_BLOWFISH is not set
# CONFIG_CRYPTO_TWOFISH is not set
# CONFIG_CRYPTO_SERPENT is not set
# CONFIG_CRYPTO_AES_586 is not set
# CONFIG_CRYPTO_CAST5 is not set
# CONFIG_CRYPTO_CAST6 is not set
# CONFIG_CRYPTO_TEA is not set
# CONFIG_CRYPTO_ARC4 is not set
# CONFIG_CRYPTO_KHAZAD is not set
CONFIG_CRYPTO_DEFLATE=m
# CONFIG_CRYPTO_MICHAEL_MIC is not set
# CONFIG_CRYPTO_CRC32C is not set
# CONFIG_CRYPTO_TEST is not set

#
# Library routines
#
CONFIG_CRC_CCITT=m
CONFIG_CRC32=m
# CONFIG_LIBCRC32C is not set
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=m
CONFIG_GENERIC_HARDIRQS=y
CONFIG_X86_BIOS_REBOOT=y
CONFIG_PC=y

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Adam Heath]

On Mon, 18 Oct 2004, Ingo Molnar wrote:

>
> i have released the -U5 Real-Time Preemption patch:
>
>   http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5
>
> this is a release intended to increase stability, but since it also
> includes new debug features and related cleanups it might introduce new
> regressions. Be careful.
>
> [snip]
>
>  - debug feature: implemented /proc/sys/kernel/trace_verbose runtime
>    flag (default:0), which enables a much more verbose printout in
>    /proc/latency_trace.  This trace format can be useful in e.g.
>    debugging timestamp weirdnesses.
>

With all these proc values, what do you recommend they should be set to?

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* Adam Heath <doogie@debian.org> wrote:

> >  - debug feature: implemented /proc/sys/kernel/trace_verbose runtime
> >    flag (default:0), which enables a much more verbose printout in
> >    /proc/latency_trace.  This trace format can be useful in e.g.
> >    debugging timestamp weirdnesses.
> 
> With all these proc values, what do you recommend they should be set
> to?

just the default values - same for the .config options. Once the feature
gets more stable the latency measurements can begin again - for them the
/proc values are important.

	Ingo

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Adam Heath]

On Mon, 18 Oct 2004, Ingo Molnar wrote:

>
> i have released the -U5 Real-Time Preemption patch:
>
>   http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5
>

EXT3 FS on hda5, internal journal
(mount/71/CPU#0): new 493 us maximum-latency critical section.
 => started at timestamp 29143938: <kernel_fpu_begin+0x21/0x60>
 =>   ended at timestamp 29144431: <_mmx_memcpy+0x131/0x180>
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01321ff>] check_preempt_timing+0x1af/0x250
 [<c01dcf51>] _mmx_memcpy+0x131/0x180
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01dcf51>] _mmx_memcpy+0x131/0x180
 [<c01dcf51>] _mmx_memcpy+0x131/0x180
 [<c01e3625>] vgacon_scroll+0x245/0x260
 [<c01f3bba>] scrup+0xda/0xf0
 [<c0113104>] mcount+0x14/0x18
 [<c01f5702>] lf+0x72/0x80
 [<c01f7e9f>] vt_console_print+0x13f/0x320
 [<c011c231>] __call_console_drivers+0x61/0x70
 [<c011c35a>] call_console_drivers+0x9a/0x140
 [<c011c721>] release_console_sem+0x71/0x100
 [<c011c5f6>] vprintk+0x116/0x180
 [<c011c4dd>] printk+0x1d/0x20
 [<c01a0957>] ext3_setup_super+0x127/0x1b0
 [<c01db07f>] up_write+0x4f/0x80
 [<c01a2552>] ext3_remount+0x132/0x190
 [<c01dae41>] down_write+0x71/0xa0
 [<c0162903>] do_remount_sb+0xb3/0xe0
 [<c0179712>] do_remount+0x72/0xc0
 [<c017a073>] do_mount+0x1a3/0x1b0
 [<c01dae41>] down_write+0x71/0xa0
 [<c017a3ec>] sys_mount+0x9c/0x100
 [<c0106013>] syscall_call+0x7/0xb
preempt count: 00000002
. 2-level deep critical section nesting:
.. entry 1: kernel_fpu_begin+0x21/0x60 / (_mmx_memcpy+0x36/0x180)
.. entry 2: print_traces+0x1d/0x70 / (dump_stack+0x23/0x30)

 =>   dump-end timestamp 29144924

The kernel is jsut getting ready to start init at this point(mounting root),
so I don't know if you are really interested in this high latency trace, but
I'm sending anyways.

However, after I reset the threshold to 50(and got a few small traces), I got
this whopper.

(XFree86/1129/CPU#0): new 4692 us maximum-latency critical section.
 => started at timestamp 358506933: <call_console_drivers+0x76/0x140>
 =>   ended at timestamp 358511625: <finish_task_switch+0x43/0xa0>
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01321ff>] check_preempt_timing+0x1af/0x250
 [<c0117ca3>] finish_task_switch+0x43/0xa0
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c0117ca3>] finish_task_switch+0x43/0xa0
 [<c0117ca3>] finish_task_switch+0x43/0xa0
 [<c02a2859>] __sched_text_start+0x2d9/0x570
 [<c02a3323>] schedule_timeout+0x63/0xc0
 [<c02a316e>] cond_resched+0xe/0x90
 [<c01253a0>] process_timeout+0x0/0x20
 [<c02a3185>] cond_resched+0x25/0x90
 [<c016f592>] do_select+0x172/0x280
 [<c016f280>] __pollwait+0x0/0xb0
 [<c016f984>] sys_select+0x294/0x570
 [<c0106013>] syscall_call+0x7/0xb
preempt count: 00000002
. 2-level deep critical section nesting:
.. entry 1: __sched_text_start+0x3b/0x570 / (schedule_timeout+0x63/0xc0)
.. entry 2: print_traces+0x1d/0x70 / (dump_stack+0x23/0x30)

 =>   dump-end timestamp 358512077

ps: I've never mentioned the hardware I am running.  Athlon XP 2000, 1G ram,
    460G(usable) software raid5(3*250g ide)(plus boot 120G), LVM, extra
    SiliconImage UDMA133 controller(mobo can only do 100).

    I'm not certain what kind of latencies to expect with this setup.  I'm
    tending to ignore <100us, at least for now.

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* Adam Heath <adam@doogie.org> wrote:

>  =>   dump-end timestamp 29144924
> 
> The kernel is jsut getting ready to start init at this point(mounting
> root), so I don't know if you are really interested in this high
> latency trace, but I'm sending anyways.

lets skip these for the time being, large runtime ones are the first 
ones to be squashed.

> However, after I reset the threshold to 50(and got a few small traces), I got
> this whopper.
> 
> (XFree86/1129/CPU#0): new 4692 us maximum-latency critical section.
>  => started at timestamp 358506933: <call_console_drivers+0x76/0x140>
>  =>   ended at timestamp 358511625: <finish_task_switch+0x43/0xa0>
>  [<c0132480>] sub_preempt_count+0x60/0x90

interesting - this could be a printk (trace) done in a critical section
though. What does /proc/latency_trace tell, is it full of console code
functions?

one of the best ways to avoid the console-printk-ing overhead is to do a
'dmesg -n 1' and reset the maximum back to 50. (i prefer to use the
preempt_max_latency option not the preempt_thresh option.)

> ps: I've never mentioned the hardware I am running.  Athlon XP 2000, 1G ram,
>     460G(usable) software raid5(3*250g ide)(plus boot 120G), LVM, extra
>     SiliconImage UDMA133 controller(mobo can only do 100).
> 
>     I'm not certain what kind of latencies to expect with this setup.  I'm
>     tending to ignore <100us, at least for now.

this setup shouldnt produce above-100 usec latencies with -U5 and
PREEMPT_REALTIME.

	Ingo

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Adam Heath]

On Mon, 18 Oct 2004, Ingo Molnar wrote:

> > However, after I reset the threshold to 50(and got a few small traces), I got
> > this whopper.
> >
> > (XFree86/1129/CPU#0): new 4692 us maximum-latency critical section.
> >  => started at timestamp 358506933: <call_console_drivers+0x76/0x140>
> >  =>   ended at timestamp 358511625: <finish_task_switch+0x43/0xa0>
> >  [<c0132480>] sub_preempt_count+0x60/0x90
>
> interesting - this could be a printk (trace) done in a critical section
> though. What does /proc/latency_trace tell, is it full of console code
> functions?

Too late, it's gone.  It'd be nice if there was some way to have history on
that file.

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* Adam Heath <doogie@debian.org> wrote:

> On Mon, 18 Oct 2004, Ingo Molnar wrote:
> 
> > > However, after I reset the threshold to 50(and got a few small traces), I got
> > > this whopper.
> > >
> > > (XFree86/1129/CPU#0): new 4692 us maximum-latency critical section.
> > >  => started at timestamp 358506933: <call_console_drivers+0x76/0x140>
> > >  =>   ended at timestamp 358511625: <finish_task_switch+0x43/0xa0>
> > >  [<c0132480>] sub_preempt_count+0x60/0x90
> >
> > interesting - this could be a printk (trace) done in a critical section
> > though. What does /proc/latency_trace tell, is it full of console code
> > functions?
> 
> Too late, it's gone.  It'd be nice if there was some way to have
> history on that file.

well - if it's gone it's always replaced by a larger latency (if you use
the preempt_max_latency method), which in most cases is more interesting
than the one you wanted to save.

	Ingo

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Adam Heath]

yOn Mon, 18 Oct 2004, Ingo Molnar wrote:

> > Too late, it's gone.  It'd be nice if there was some way to have
> > history on that file.
>
> well - if it's gone it's always replaced by a larger latency (if you use
> the preempt_max_latency method), which in most cases is more interesting
> than the one you wanted to save.

I reset the minimum to 50, and it's only gotten up to 83.

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[K.R. Foley]

Ingo Molnar wrote:
> i have released the -U5 Real-Time Preemption patch:
> 
>   http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5
> 

Ingo,

*** Warning: "__you_cannot_kmalloc_that_much" 
[drivers/scsi/aacraid/aacraid.ko] undefined!

This just appeared in U5. I was trying to track this one down just 
because I saw it, even though I don't need aacraid. I am having a hell 
of a time tracking down what changed that would cause this, but I figure 
you will know exactly what changed that would cause it. :)

kr

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* K.R. Foley <kr@cybsft.com> wrote:

> Ingo Molnar wrote:
> >i have released the -U5 Real-Time Preemption patch:
> >
> >  http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5
> >
> 
> Ingo,
> 
> *** Warning: "__you_cannot_kmalloc_that_much" 
> [drivers/scsi/aacraid/aacraid.ko] undefined!
> 
> This just appeared in U5. I was trying to track this one down just
> because I saw it, even though I don't need aacraid. I am having a hell
> of a time tracking down what changed that would cause this, but I
> figure you will know exactly what changed that would cause it. :)

i suspect this is due to the size increase of semaphores if
CONFIG_RWSEM_DEADLOCK_DETECT is enabled. Try lowering
CONFIG_RWSEM_MAX_OWNERS from the default 64 to 32, does that help?

	Ingo

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[K.R. Foley]

Ingo Molnar wrote:
> * K.R. Foley <kr@cybsft.com> wrote:
> 
> 
>>Ingo Molnar wrote:
>>
>>>i have released the -U5 Real-Time Preemption patch:
>>>
>>> http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5
>>>
>>
>>Ingo,
>>
>>*** Warning: "__you_cannot_kmalloc_that_much" 
>>[drivers/scsi/aacraid/aacraid.ko] undefined!
>>
>>This just appeared in U5. I was trying to track this one down just
>>because I saw it, even though I don't need aacraid. I am having a hell
>>of a time tracking down what changed that would cause this, but I
>>figure you will know exactly what changed that would cause it. :)
> 
> 
> i suspect this is due to the size increase of semaphores if
> CONFIG_RWSEM_DEADLOCK_DETECT is enabled. Try lowering
> CONFIG_RWSEM_MAX_OWNERS from the default 64 to 32, does that help?
> 
> 	Ingo
> 
Yes. That does help.

kr

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Bill Huey]

On Mon, Oct 18, 2004 at 04:50:08PM +0200, Ingo Molnar wrote:
> i have released the -U5 Real-Time Preemption patch:
> 
>   http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5

  CC      arch/i386/kernel/traps.o
arch/i386/kernel/traps.c: In function `do_debug':
arch/i386/kernel/traps.c:786: error: `sysenter_past_esp' undeclared (first use in this function)
arch/i386/kernel/traps.c:786: error: (Each undeclared identifier is reported only once
arch/i386/kernel/traps.c:786: error: for each function it appears in.)
make[1]: *** [arch/i386/kernel/traps.o] Error 1
make: *** [arch/i386/kernel] Error 2

bill

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Bill Huey]

On Mon, Oct 18, 2004 at 12:32:51PM -0700, Bill Huey wrote:
> On Mon, Oct 18, 2004 at 04:50:08PM +0200, Ingo Molnar wrote:
> > i have released the -U5 Real-Time Preemption patch:
> > 
> >   http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5
> 
>   CC      arch/i386/kernel/traps.o
> arch/i386/kernel/traps.c: In function `do_debug':
> arch/i386/kernel/traps.c:786: error: `sysenter_past_esp' undeclared (first use in this function)
> arch/i386/kernel/traps.c:786: error: (Each undeclared identifier is reported only once
> arch/i386/kernel/traps.c:786: error: for each function it appears in.)
> make[1]: *** [arch/i386/kernel/traps.o] Error 1
> make: *** [arch/i386/kernel] Error 2

bah, let me handle this, CONFIG_KGDB was turned on and I'm workin on
this now...

bill

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* Bill Huey <bhuey@lnxw.com> wrote:

> On Mon, Oct 18, 2004 at 04:50:08PM +0200, Ingo Molnar wrote:
> > i have released the -U5 Real-Time Preemption patch:
> > 
> >   http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U5
> 
>   CC      arch/i386/kernel/traps.o
> arch/i386/kernel/traps.c: In function `do_debug':
> arch/i386/kernel/traps.c:786: error: `sysenter_past_esp' undeclared (first use in this function)
> arch/i386/kernel/traps.c:786: error: (Each undeclared identifier is reported only once
> arch/i386/kernel/traps.c:786: error: for each function it appears in.)
> make[1]: *** [arch/i386/kernel/traps.o] Error 1
> make: *** [arch/i386/kernel] Error 2

i guess this might be an -mm1 breakage if CONFIG_KGDB enabled - does it
happen with vanilla -mm1 too?

	Ingo

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Bill Huey]

On Mon, Oct 18, 2004 at 09:36:03PM +0200, Ingo Molnar wrote:
> * Bill Huey <bhuey@lnxw.com> wrote:
> > 
> >   CC      arch/i386/kernel/traps.o
> > arch/i386/kernel/traps.c: In function `do_debug':
> > arch/i386/kernel/traps.c:786: error: `sysenter_past_esp' undeclared (first use in this function)
> > arch/i386/kernel/traps.c:786: error: (Each undeclared identifier is reported only once
> > arch/i386/kernel/traps.c:786: error: for each function it appears in.)
> > make[1]: *** [arch/i386/kernel/traps.o] Error 1
> > make: *** [arch/i386/kernel] Error 2
> 
> i guess this might be an -mm1 breakage if CONFIG_KGDB enabled - does it
> happen with vanilla -mm1 too?

yep, should I wait for -mm2 ?

bill

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* Bill Huey <bhuey@lnxw.com> wrote:

> On Mon, Oct 18, 2004 at 09:36:03PM +0200, Ingo Molnar wrote:
> > * Bill Huey <bhuey@lnxw.com> wrote:
> > > 
> > >   CC      arch/i386/kernel/traps.o
> > > arch/i386/kernel/traps.c: In function `do_debug':
> > > arch/i386/kernel/traps.c:786: error: `sysenter_past_esp' undeclared (first use in this function)
> > > arch/i386/kernel/traps.c:786: error: (Each undeclared identifier is reported only once
> > > arch/i386/kernel/traps.c:786: error: for each function it appears in.)
> > > make[1]: *** [arch/i386/kernel/traps.o] Error 1
> > > make: *** [arch/i386/kernel] Error 2
> > 
> > i guess this might be an -mm1 breakage if CONFIG_KGDB enabled - does it
> > happen with vanilla -mm1 too?
> 
> yep, should I wait for -mm2 ?

since 2.6.9's release is imminent there will unlikely be an -rc4-mm2,
2.6.9-mm1 should be the next one.

	Ingo

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Bill Huey]

On Mon, Oct 18, 2004 at 09:46:32PM +0200, Ingo Molnar wrote:
> since 2.6.9's release is imminent there will unlikely be an -rc4-mm2,
> 2.6.9-mm1 should be the next one.

The new kgdb logic is a bit foreign to me. I'll have to look at it a
bit more, but this kgdb build problem is criticial for a certain
part of the kernel community that needs it. I've commented out that
section and rebuilding it now.

To get kgdb work (referencing my tree), I've just demoted all of the
spinlocks in arch/i386/{kernel,lib}/kgdb*.c files. It's pretty straight
forward, nothing tricky at all.

bill

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Adam Heath]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 223 bytes --]

On Mon, 18 Oct 2004, Ingo Molnar wrote:

>
> i have released the -U5 Real-Time Preemption patch:

>From the first line of the attached oops file:

kernel BUG at lib/rwsem-generic.c:130!

Then, a BUG, sleeping while atomic.

[-- Attachment #2: Type: TEXT/PLAIN, Size: 5669 bytes --]

------------[ cut here ]------------
kernel BUG at lib/rwsem-generic.c:130!
invalid operand: 0000 [#1]
PREEMPT 
Modules linked in: nfsd exportfs snd_cmipci snd_opl3_lib snd_hwdep snd_mpu401_uart snd_rawmidi snd_seq_device 8250 serial_core nfs lockd sunrpc raid5 xor md dm_mirror dm_mod aic7xxx sg sr_mod cdrom sd_mod scsi_mod snd_intel8x0 snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd snd_page_alloc soundcore 3c59x mii sis900 crc32 siimage
CPU:    0
EIP:    0060:[<c01da56b>]    Not tainted VLI
EFLAGS: 00010002   (2.6.9-rc4-mm1-rt-u5) 
EIP is at rwsem_owner_del+0x6b/0x90
eax: 00000001   ebx: f7b31658   ecx: 00000000   edx: 00000001
esi: 00000000   edi: f7b31658   ebp: d3113f34   esp: d3113f28
ds: 007b   es: 007b   ss: 0068   preempt: 00000002
Process liquidwar (pid: 6352, threadinfo=d3113000 task=d7ec6e60)
Stack: f7b31658 00000202 d3113fa8 d3113f48 c01db057 f7b31658 f741ac00 d74036c0 
       d3113f64 f88c1c18 d3113f64 c0113104 00000000 f88c1be0 d74036c0 d3113f90 
       c015b61b d74036c0 080d77f8 00000800 d3113fa8 f0d94164 f0d94164 d74036c0 
Call Trace:
 [<c01db057>] up_write+0x27/0x80
 [<f88c1c18>] snd_pcm_oss_write+0x38/0x70 [snd_pcm_oss]
 [<c0113104>] mcount+0x14/0x18
 [<f88c1be0>] snd_pcm_oss_write+0x0/0x70 [snd_pcm_oss]
 [<c015b61b>] vfs_write+0xcb/0x140
 [<c015b760>] sys_write+0x50/0x80
 [<c0106013>] syscall_call+0x7/0xb
preempt count: 00000003
. 3-level deep critical section nesting:
.. entry 1: up_write+0x70/0x80 / (snd_pcm_oss_write+0x38/0x70 [snd_pcm_oss])
.. entry 2: die+0x3f/0x1a0 / (do_invalid_op+0x106/0x110)
.. entry 3: print_traces+0x1d/0x70 / (show_stack+0x83/0xa0)

Code: 16 bb 00 f0 ff ff 21 e3 8b 44 8f 10 3b 03 74 28 41 89 d6 39 d1 7c f1 8b 15 b8 fe 2e c0 85 d2 74 12 c7 05 b8 fe 2e c0 00 00 00 00 <0f> 0b 82 00 e1 05 2c c0 5b 5e 5f c9 c3 8d 56 ff 39 d1 74 08 8b 
 <6>note: liquidwar[6352] exited with preempt_count 1
BUG: sleeping function called from invalid context liquidwar(6352) at kernel/fork.c:421
in_atomic():1 [00000001], irqs_disabled():0
 [<c0119582>] __might_sleep+0xc2/0xe0
 [<c0119bce>] mm_release+0x6e/0xd0
 [<c011c4dd>] printk+0x1d/0x20
 [<c011e6d3>] do_exit+0x83/0x4a0
 [<c01075f0>] do_invalid_op+0x0/0x110
 [<c0107242>] die+0x192/0x1a0
 [<c0116c5c>] fixup_exception+0x1c/0x40
 [<c01076f6>] do_invalid_op+0x106/0x110
 [<c01da56b>] rwsem_owner_del+0x6b/0x90
 [<c0113104>] mcount+0x14/0x18
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c013193d>] __mcount+0x1d/0x20
 [<c01c890e>] selinux_file_permission+0xe/0x170
 [<c015b5f3>] vfs_write+0xa3/0x140
 [<c0113104>] mcount+0x14/0x18
 [<c0106a25>] error_code+0x2d/0x38
 [<c01da56b>] rwsem_owner_del+0x6b/0x90
 [<c01db057>] up_write+0x27/0x80
 [<f88c1c18>] snd_pcm_oss_write+0x38/0x70 [snd_pcm_oss]
 [<c0113104>] mcount+0x14/0x18
 [<f88c1be0>] snd_pcm_oss_write+0x0/0x70 [snd_pcm_oss]
 [<c015b61b>] vfs_write+0xcb/0x140
 [<c015b760>] sys_write+0x50/0x80
 [<c0106013>] syscall_call+0x7/0xb
preempt count: 00000002
. 2-level deep critical section nesting:
.. entry 1: task_rq_lock+0x1e/0x30 / (try_to_wake_up+0x2d/0xc0)
.. entry 2: print_traces+0x1d/0x70 / (dump_stack+0x23/0x30)

BUG: scheduling while atomic: liquidwar/0x04000001/6352
caller is cond_resched+0x62/0x90
 [<c02a2ab9>] __sched_text_start+0x539/0x570
 [<c02a31c2>] cond_resched+0x62/0x90
 [<c0113104>] mcount+0x14/0x18
 [<c02a31c2>] cond_resched+0x62/0x90
 [<c0119bd3>] mm_release+0x73/0xd0
 [<c011c4dd>] printk+0x1d/0x20
 [<c011e6d3>] do_exit+0x83/0x4a0
 [<c01075f0>] do_invalid_op+0x0/0x110
 [<c0107242>] die+0x192/0x1a0
 [<c0116c5c>] fixup_exception+0x1c/0x40
 [<c01076f6>] do_invalid_op+0x106/0x110
 [<c01da56b>] rwsem_owner_del+0x6b/0x90
 [<c0113104>] mcount+0x14/0x18
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c013193d>] __mcount+0x1d/0x20
 [<c01c890e>] selinux_file_permission+0xe/0x170
 [<c015b5f3>] vfs_write+0xa3/0x140
 [<c0113104>] mcount+0x14/0x18
 [<c0106a25>] error_code+0x2d/0x38
 [<c01da56b>] rwsem_owner_del+0x6b/0x90
 [<c01db057>] up_write+0x27/0x80
 [<f88c1c18>] snd_pcm_oss_write+0x38/0x70 [snd_pcm_oss]
 [<c0113104>] mcount+0x14/0x18
 [<f88c1be0>] snd_pcm_oss_write+0x0/0x70 [snd_pcm_oss]
 [<c015b61b>] vfs_write+0xcb/0x140
 [<c015b760>] sys_write+0x50/0x80
 [<c0106013>] syscall_call+0x7/0xb
preempt count: 04000002
. 2-level deep critical section nesting:
.. entry 1: __do_IRQ+0x111/0x190 / (do_IRQ+0x75/0xa0)
.. entry 2: print_traces+0x1d/0x70 / (dump_stack+0x23/0x30)

BUG: scheduling while atomic: liquidwar/0x00000001/6352
caller is do_exit+0x292/0x4a0
 [<c02a2ab9>] __sched_text_start+0x539/0x570
 [<c011e8e2>] do_exit+0x292/0x4a0
 [<c0113104>] mcount+0x14/0x18
 [<c011e8e2>] do_exit+0x292/0x4a0
 [<c01075f0>] do_invalid_op+0x0/0x110
 [<c0107242>] die+0x192/0x1a0
 [<c0116c5c>] fixup_exception+0x1c/0x40
 [<c01076f6>] do_invalid_op+0x106/0x110
 [<c01da56b>] rwsem_owner_del+0x6b/0x90
 [<c0113104>] mcount+0x14/0x18
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c013193d>] __mcount+0x1d/0x20
 [<c01c890e>] selinux_file_permission+0xe/0x170
 [<c015b5f3>] vfs_write+0xa3/0x140
 [<c0113104>] mcount+0x14/0x18
 [<c0106a25>] error_code+0x2d/0x38
 [<c01da56b>] rwsem_owner_del+0x6b/0x90
 [<c01db057>] up_write+0x27/0x80
 [<f88c1c18>] snd_pcm_oss_write+0x38/0x70 [snd_pcm_oss]
 [<c0113104>] mcount+0x14/0x18
 [<f88c1be0>] snd_pcm_oss_write+0x0/0x70 [snd_pcm_oss]
 [<c015b61b>] vfs_write+0xcb/0x140
 [<c015b760>] sys_write+0x50/0x80
 [<c0106013>] syscall_call+0x7/0xb
preempt count: 00000002
. 2-level deep critical section nesting:
.. entry 1: __do_IRQ+0x111/0x190 / (do_IRQ+0x75/0xa0)
.. entry 2: print_traces+0x1d/0x70 / (dump_stack+0x23/0x30)


[-- Attachment #3: Type: TEXT/PLAIN, Size: 15641 bytes --]

ksymoops 2.4.9 on i686 2.6.9-rc4-mm1-rt-u5.  Options used
     -V (default)
     -k /proc/ksyms (default)
     -l /proc/modules (default)
     -o /lib/modules/2.6.9-rc4-mm1-rt-u5/ (default)
     -m /boot/System.map-2.6.9-rc4-mm1-rt-u5 (default)

Warning: You did not tell me where to find symbol information.  I will
assume that the log matches the kernel and modules that are running
right now and I'll use the default options above for symbol resolution.
If the current kernel and/or modules do not match the log, you can get
more accurate output by telling me the kernel version and where to find
map, modules, ksyms etc.  ksymoops -h explains the options.

Error (regular_file): read_ksyms stat /proc/ksyms failed
No modules in ksyms, skipping objects
No ksyms, skipping lsmod
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01dae41>] down_write+0x71/0xa0
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01db07f>] up_write+0x4f/0x80
 [<c01320ac>] check_preempt_timing+0x5c/0x250
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01db07f>] up_write+0x4f/0x80
 [<c013193d>] __mcount+0x1d/0x20
 [<c01da9be>] __up_write+0xe/0x1b0
 [<c01db05f>] up_write+0x2f/0x80
 [<c0113104>] mcount+0x14/0x18
 [<c016ea97>] sys_ioctl+0x47/0x230
 [<c0106013>] syscall_call+0x7/0xb
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01321ff>] check_preempt_timing+0x1af/0x250
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c01c4b74>] avc_has_perm+0x14/0xa0
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c0113104>] mcount+0x14/0x18
 [<c01c4ba3>] avc_has_perm+0x43/0xa0
 [<c01c8a33>] selinux_file_permission+0x133/0x170
 [<c0113104>] mcount+0x14/0x18
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c01db07f>] up_write+0x4f/0x80
 [<c013193d>] __mcount+0x1d/0x20
 [<c01c890e>] selinux_file_permission+0xe/0x170
 [<c015b3d6>] vfs_read+0xa6/0x140
 [<c0113104>] mcount+0x14/0x18
 [<c01c8a33>] selinux_file_permission+0x133/0x170
 [<c0113104>] mcount+0x14/0x18
 [<c015b3d6>] vfs_read+0xa6/0x140
 [<c015c604>] fget_light+0x14/0xa0
 [<c0113104>] mcount+0x14/0x18
 [<c015b6e0>] sys_read+0x50/0x80
 [<c0106013>] syscall_call+0x7/0xb
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01321ff>] check_preempt_timing+0x1af/0x250
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c013193d>] __mcount+0x1d/0x20
 [<c01c499b>] avc_has_perm_noaudit+0x7b/0x240
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c01320ac>] check_preempt_timing+0x5c/0x250
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01183ee>] __wake_up+0x5e/0x90
 [<c01c4b74>] avc_has_perm+0x14/0xa0
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c0113104>] mcount+0x14/0x18
 [<c01c4ba3>] avc_has_perm+0x43/0xa0
 [<c01c8a33>] selinux_file_permission+0x133/0x170
 [<c0113104>] mcount+0x14/0x18
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01db07f>] up_write+0x4f/0x80
 [<c01320ac>] check_preempt_timing+0x5c/0x250
 [<c013193d>] __mcount+0x1d/0x20
 [<c01c890e>] selinux_file_permission+0xe/0x170
 [<c015b5f3>] vfs_write+0xa3/0x140
 [<c0113104>] mcount+0x14/0x18
 [<c01c8a33>] selinux_file_permission+0x133/0x170
 [<c0113104>] mcount+0x14/0x18
 [<c015b5f3>] vfs_write+0xa3/0x140
 [<c015c604>] fget_light+0x14/0xa0
 [<c0113104>] mcount+0x14/0x18
 [<c015b760>] sys_write+0x50/0x80
 [<c0106013>] syscall_call+0x7/0xb
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01321ff>] check_preempt_timing+0x1af/0x250
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c01c4b74>] avc_has_perm+0x14/0xa0
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c0113104>] mcount+0x14/0x18
 [<c01c4ba3>] avc_has_perm+0x43/0xa0
 [<c01c8a33>] selinux_file_permission+0x133/0x170
 [<c0113104>] mcount+0x14/0x18
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01db07f>] up_write+0x4f/0x80
 [<c013193d>] __mcount+0x1d/0x20
 [<c013193d>] __mcount+0x1d/0x20
 [<c01c890e>] selinux_file_permission+0xe/0x170
 [<c015b5f3>] vfs_write+0xa3/0x140
 [<c0113104>] mcount+0x14/0x18
 [<c01c8a33>] selinux_file_permission+0x133/0x170
 [<c01320ac>] check_preempt_timing+0x5c/0x250
 [<c0113104>] mcount+0x14/0x18
 [<c015b5f3>] vfs_write+0xa3/0x140
 [<c015c67f>] fget_light+0x8f/0xa0
 [<c015b760>] sys_write+0x50/0x80
 [<c0106013>] syscall_call+0x7/0xb
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01321ff>] check_preempt_timing+0x1af/0x250
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c01c4b74>] avc_has_perm+0x14/0xa0
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c0113104>] mcount+0x14/0x18
 [<c01c4ba3>] avc_has_perm+0x43/0xa0
 [<c01c8a33>] selinux_file_permission+0x133/0x170
 [<c0113104>] mcount+0x14/0x18
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c013193d>] __mcount+0x1d/0x20
 [<c0176db9>] inode_times_differ+0x9/0x50
 [<c0176f6f>] inode_update_time+0x9f/0xe0
 [<c0113104>] mcount+0x14/0x18
 [<c013193d>] __mcount+0x1d/0x20
 [<c01c890e>] selinux_file_permission+0xe/0x170
 [<c015b3d6>] vfs_read+0xa6/0x140
 [<c0113104>] mcount+0x14/0x18
 [<c01c8a33>] selinux_file_permission+0x133/0x170
 [<c01320ac>] check_preempt_timing+0x5c/0x250
 [<c0113104>] mcount+0x14/0x18
 [<c015b3d6>] vfs_read+0xa6/0x140
 [<c015c67f>] fget_light+0x8f/0xa0
 [<c015b6e0>] sys_read+0x50/0x80
 [<c0106013>] syscall_call+0x7/0xb
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01321ff>] check_preempt_timing+0x1af/0x250
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c0132480>] sub_preempt_count+0x60/0x90
 [<c01dafff>] up_read+0x4f/0x80
 [<c01c4b74>] avc_has_perm+0x14/0xa0
 [<c01cabcf>] selinux_ip_postroute_last+0x1bf/0x2a0
 [<c0113104>] mcount+0x14/0x18
 [<c01c4ba3>] avc_has_perm+0x43/0xa0
 [<c01cabcf>] selinux_ip_postroute_last+0x1bf/0x2a0
 [<c0113104>] mcount+0x14/0x18
 [<c01cacea>] selinux_ipv4_postroute_last+0x3a/0x40
 [<c02636d0>] ip_finish_output2+0x0/0x1f0
 [<c024fe65>] nf_iterate+0x75/0xb0
 [<c02636d0>] ip_finish_output2+0x0/0x1f0
 [<c0250280>] nf_hook_slow+0x90/0x140
 [<c02636d0>] ip_finish_output2+0x0/0x1f0
 [<c02610e3>] ip_finish_output+0x223/0x230
 [<c02636d0>] ip_finish_output2+0x0/0x1f0
 [<c0261716>] ip_queue_xmit+0x3d6/0x540
 [<c01c4bd1>] avc_has_perm+0x71/0xa0
 [<c0113104>] mcount+0x14/0x18
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c01c4a02>] avc_has_perm_noaudit+0xe2/0x240
 [<c01dafff>] up_read+0x4f/0x80
 [<c013193d>] __mcount+0x1d/0x20
 [<c027842e>] tcp_v4_send_check+0xe/0xf0
 [<c0272119>] tcp_transmit_skb+0x439/0x860
 [<c0113104>] mcount+0x14/0x18
 [<c027846f>] tcp_v4_send_check+0x4f/0xf0
 [<c02721c2>] tcp_transmit_skb+0x4e2/0x860
 [<c0113104>] mcount+0x14/0x18
 [<c0274d76>] tcp_send_ack+0xa6/0xf0
 [<c026fdf4>] __tcp_ack_snd_check+0x14/0xa0
 [<c0270427>] tcp_rcv_established+0x287/0x900
 [<c013193d>] __mcount+0x1d/0x20
 [<c027965d>] tcp_v4_do_rcv+0xd/0x120
 [<c0279e7b>] tcp_v4_rcv+0x70b/0x950
 [<c0113104>] mcount+0x14/0x18
 [<c0279765>] tcp_v4_do_rcv+0x115/0x120
 [<c0279e7b>] tcp_v4_rcv+0x70b/0x950
 [<c025db75>] ip_local_deliver+0xf5/0x250
 [<c025df31>] ip_rcv+0x261/0x4c0
 [<c01dae41>] down_write+0x71/0xa0
 [<c0245570>] netif_receive_skb+0x1b0/0x280
 [<c0130008>] nanosleep_wake_up+0x18/0x20
 [<c024564e>] process_backlog+0xe/0x150
 [<c024580f>] net_rx_action+0x7f/0x150
 [<c02456c8>] process_backlog+0x88/0x150
 [<c024580f>] net_rx_action+0x7f/0x150
 [<c0120f87>] ___do_softirq+0x87/0xd0
 [<c0121058>] _do_softirq+0x8/0x30
 [<c01213a4>] ksoftirqd+0x94/0xe0
 [<c0121070>] _do_softirq+0x20/0x30
 [<c01213a4>] ksoftirqd+0x94/0xe0
 [<c013072a>] kthread+0xaa/0xb0
 [<c0121310>] ksoftirqd+0x0/0xe0
 [<c0130680>] kthread+0x0/0xb0
 [<c0104099>] kernel_thread_helper+0x5/0xc
kernel BUG at lib/rwsem-generic.c:130!
invalid operand: 0000 [#1]
CPU:    0
EIP:    0060:[<c01da56b>]    Not tainted VLI
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010002   (2.6.9-rc4-mm1-rt-u5) 
eax: 00000001   ebx: f7b31658   ecx: 00000000   edx: 00000001
esi: 00000000   edi: f7b31658   ebp: d3113f34   esp: d3113f28
ds: 007b   es: 007b   ss: 0068   preempt: 00000002
Stack: f7b31658 00000202 d3113fa8 d3113f48 c01db057 f7b31658 f741ac00 d74036c0 
       d3113f64 f88c1c18 d3113f64 c0113104 00000000 f88c1be0 d74036c0 d3113f90 
       c015b61b d74036c0 080d77f8 00000800 d3113fa8 f0d94164 f0d94164 d74036c0 
Call Trace:
 [<c01db057>] up_write+0x27/0x80
 [<f88c1c18>] snd_pcm_oss_write+0x38/0x70 [snd_pcm_oss]
 [<c0113104>] mcount+0x14/0x18
 [<f88c1be0>] snd_pcm_oss_write+0x0/0x70 [snd_pcm_oss]
 [<c015b61b>] vfs_write+0xcb/0x140
 [<c015b760>] sys_write+0x50/0x80
 [<c0106013>] syscall_call+0x7/0xb
Warning (Oops_read): Code line not seen, dumping what data is available


>>EIP; c01da56b <rwsem_owner_del+6b/90>   <=====

>>ebx; f7b31658 <pg0+37735658/3fc02400>
>>edi; f7b31658 <pg0+37735658/3fc02400>
>>ebp; d3113f34 <pg0+12d17f34/3fc02400>
>>esp; d3113f28 <pg0+12d17f28/3fc02400>

Trace; c01db057 <up_write+27/80>
Trace; f88c1c18 <pg0+384c5c18/3fc02400>
Trace; c0113104 <mcount+14/18>
Trace; f88c1be0 <pg0+384c5be0/3fc02400>
Trace; c015b61b <vfs_write+cb/140>
Trace; c015b760 <sys_write+50/80>
Trace; c0106013 <syscall_call+7/b>

Code: 16 bb 00 f0 ff ff 21 e3 8b 44 8f 10 3b 03 74 28 41 89 d6 39 d1 7c f1 8b 15 b8 fe 2e c0 85 d2 74 12 c7 05 b8 fe 2e c0 00 00 00 00 <0f> 0b 82 00 e1 05 2c c0 5b 5e 5f c9 c3 8d 56 ff 39 d1 74 08 8b 


This architecture has variable length instructions, decoding before eip
is unreliable, take these instructions with a pinch of salt.

Code;  c01da540 <rwsem_owner_del+40/90>
00000000 <_EIP>:
Code;  c01da540 <rwsem_owner_del+40/90>
   0:   16                        push   %ss
Code;  c01da541 <rwsem_owner_del+41/90>
   1:   bb 00 f0 ff ff            mov    $0xfffff000,%ebx
Code;  c01da546 <rwsem_owner_del+46/90>
   6:   21 e3                     and    %esp,%ebx
Code;  c01da548 <rwsem_owner_del+48/90>
   8:   8b 44 8f 10               mov    0x10(%edi,%ecx,4),%eax
Code;  c01da54c <rwsem_owner_del+4c/90>
   c:   3b 03                     cmp    (%ebx),%eax
Code;  c01da54e <rwsem_owner_del+4e/90>
   e:   74 28                     je     38 <_EIP+0x38>
Code;  c01da550 <rwsem_owner_del+50/90>
  10:   41                        inc    %ecx
Code;  c01da551 <rwsem_owner_del+51/90>
  11:   89 d6                     mov    %edx,%esi
Code;  c01da553 <rwsem_owner_del+53/90>
  13:   39 d1                     cmp    %edx,%ecx
Code;  c01da555 <rwsem_owner_del+55/90>
  15:   7c f1                     jl     8 <_EIP+0x8>
Code;  c01da557 <rwsem_owner_del+57/90>
  17:   8b 15 b8 fe 2e c0         mov    0xc02efeb8,%edx
Code;  c01da55d <rwsem_owner_del+5d/90>
  1d:   85 d2                     test   %edx,%edx
Code;  c01da55f <rwsem_owner_del+5f/90>
  1f:   74 12                     je     33 <_EIP+0x33>
Code;  c01da561 <rwsem_owner_del+61/90>
  21:   c7 05 b8 fe 2e c0 00      movl   $0x0,0xc02efeb8
Code;  c01da568 <rwsem_owner_del+68/90>
  28:   00 00 00 

This decode from eip onwards should be reliable

Code;  c01da56b <rwsem_owner_del+6b/90>
00000000 <_EIP>:
Code;  c01da56b <rwsem_owner_del+6b/90>
   0:   0f 0b                     ud2a   
Code;  c01da56d <rwsem_owner_del+6d/90>
   2:   82                        (bad)  
Code;  c01da56e <rwsem_owner_del+6e/90>
   3:   00 e1                     add    %ah,%cl
Code;  c01da570 <rwsem_owner_del+70/90>
   5:   05 2c c0 5b 5e            add    $0x5e5bc02c,%eax
Code;  c01da575 <rwsem_owner_del+75/90>
   a:   5f                        pop    %edi
Code;  c01da576 <rwsem_owner_del+76/90>
   b:   c9                        leave  
Code;  c01da577 <rwsem_owner_del+77/90>
   c:   c3                        ret    
Code;  c01da578 <rwsem_owner_del+78/90>
   d:   8d 56 ff                  lea    0xffffffff(%esi),%edx
Code;  c01da57b <rwsem_owner_del+7b/90>
  10:   39 d1                     cmp    %edx,%ecx
Code;  c01da57d <rwsem_owner_del+7d/90>
  12:   74 08                     je     1c <_EIP+0x1c>
Code;  c01da57f <rwsem_owner_del+7f/90>
  14:   8b                        .byte 0x8b

 [<c0119582>] __might_sleep+0xc2/0xe0
 [<c0119bce>] mm_release+0x6e/0xd0
 [<c011c4dd>] printk+0x1d/0x20
 [<c011e6d3>] do_exit+0x83/0x4a0
 [<c01075f0>] do_invalid_op+0x0/0x110
 [<c0107242>] die+0x192/0x1a0
 [<c0116c5c>] fixup_exception+0x1c/0x40
 [<c01076f6>] do_invalid_op+0x106/0x110
 [<c01da56b>] rwsem_owner_del+0x6b/0x90
 [<c0113104>] mcount+0x14/0x18
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c013193d>] __mcount+0x1d/0x20
 [<c01c890e>] selinux_file_permission+0xe/0x170
 [<c015b5f3>] vfs_write+0xa3/0x140
 [<c0113104>] mcount+0x14/0x18
 [<c0106a25>] error_code+0x2d/0x38
 [<c01da56b>] rwsem_owner_del+0x6b/0x90
 [<c01db057>] up_write+0x27/0x80
 [<f88c1c18>] snd_pcm_oss_write+0x38/0x70 [snd_pcm_oss]
 [<c0113104>] mcount+0x14/0x18
 [<f88c1be0>] snd_pcm_oss_write+0x0/0x70 [snd_pcm_oss]
 [<c015b61b>] vfs_write+0xcb/0x140
 [<c015b760>] sys_write+0x50/0x80
 [<c0106013>] syscall_call+0x7/0xb
 [<c02a2ab9>] __sched_text_start+0x539/0x570
 [<c02a31c2>] cond_resched+0x62/0x90
 [<c0113104>] mcount+0x14/0x18
 [<c02a31c2>] cond_resched+0x62/0x90
 [<c0119bd3>] mm_release+0x73/0xd0
 [<c011c4dd>] printk+0x1d/0x20
 [<c011e6d3>] do_exit+0x83/0x4a0
 [<c01075f0>] do_invalid_op+0x0/0x110
 [<c0107242>] die+0x192/0x1a0
 [<c0116c5c>] fixup_exception+0x1c/0x40
 [<c01076f6>] do_invalid_op+0x106/0x110
 [<c01da56b>] rwsem_owner_del+0x6b/0x90
 [<c0113104>] mcount+0x14/0x18
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c013193d>] __mcount+0x1d/0x20
 [<c01c890e>] selinux_file_permission+0xe/0x170
 [<c015b5f3>] vfs_write+0xa3/0x140
 [<c0113104>] mcount+0x14/0x18
 [<c0106a25>] error_code+0x2d/0x38
 [<c01da56b>] rwsem_owner_del+0x6b/0x90
 [<c01db057>] up_write+0x27/0x80
 [<f88c1c18>] snd_pcm_oss_write+0x38/0x70 [snd_pcm_oss]
 [<c0113104>] mcount+0x14/0x18
 [<f88c1be0>] snd_pcm_oss_write+0x0/0x70 [snd_pcm_oss]
 [<c015b61b>] vfs_write+0xcb/0x140
 [<c015b760>] sys_write+0x50/0x80
 [<c0106013>] syscall_call+0x7/0xb
 [<c02a2ab9>] __sched_text_start+0x539/0x570
 [<c011e8e2>] do_exit+0x292/0x4a0
 [<c0113104>] mcount+0x14/0x18
 [<c011e8e2>] do_exit+0x292/0x4a0
 [<c01075f0>] do_invalid_op+0x0/0x110
 [<c0107242>] die+0x192/0x1a0
 [<c0116c5c>] fixup_exception+0x1c/0x40
 [<c01076f6>] do_invalid_op+0x106/0x110
 [<c01da56b>] rwsem_owner_del+0x6b/0x90
 [<c0113104>] mcount+0x14/0x18
 [<c01c6064>] inode_has_perm+0x64/0x90
 [<c013193d>] __mcount+0x1d/0x20
 [<c01c890e>] selinux_file_permission+0xe/0x170
 [<c015b5f3>] vfs_write+0xa3/0x140
 [<c0113104>] mcount+0x14/0x18
 [<c0106a25>] error_code+0x2d/0x38
 [<c01da56b>] rwsem_owner_del+0x6b/0x90
 [<c01db057>] up_write+0x27/0x80
 [<f88c1c18>] snd_pcm_oss_write+0x38/0x70 [snd_pcm_oss]
 [<c0113104>] mcount+0x14/0x18
 [<f88c1be0>] snd_pcm_oss_write+0x0/0x70 [snd_pcm_oss]
 [<c015b61b>] vfs_write+0xcb/0x140
 [<c015b760>] sys_write+0x50/0x80
 [<c0106013>] syscall_call+0x7/0xb

2 warnings and 1 error issued.  Results may not be reliable.

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Thomas Gleixner]

On Mon, 2004-10-18 at 16:50, Ingo Molnar wrote:
> i have released the -U5 Real-Time Preemption patch:
> 

starting nfsd triggers the deadlock detection, but it's not really a
deadlock.

The first nfsd thread acquires nlmsvc_sema. The other 7 nfsd threads
wait on it. The first thread creates lockd and tries to get lockd_start,
which is initialized locked and will be released when the lockd thread
starts. The deadlock complains about a deadlock on nlmsvc_sema inside of
down(&lockd_start). ?

Converting lockd_start to a waitqueue solves the problem and is more
sane, than the locked mutex.

tglx

--- 2.6.9-rc4-mm1-RT-U5/fs/lockd/svc.c.orig     2004-10-19
10:02:17.000000000 +0200
+++ 2.6.9-rc4-mm1-RT-U5/fs/lockd/svc.c  2004-10-19 09:54:22.000000000
+0200
@@ -46,7 +46,7 @@
 int                            nlmsvc_grace_period;
 unsigned long                  nlmsvc_timeout;

-static DECLARE_MUTEX(lockd_start);
+static DECLARE_WAIT_QUEUE_HEAD(lockd_start);
 static DECLARE_WAIT_QUEUE_HEAD(lockd_exit);

 /*
@@ -109,7 +109,7 @@
         * Let our maker know we're running.
         */
        nlmsvc_pid = current->pid;
-       up(&lockd_start);
+       wake_up(&lockd_start);

        daemonize("lockd");

@@ -230,6 +230,7 @@
                printk(KERN_WARNING
                        "lockd_up: no pid, %d users??\n", nlmsvc_users);

+
        error = -ENOMEM;
        serv = svc_create(&nlmsvc_program, LOCKD_BUFSIZE);
        if (!serv) {
@@ -258,7 +259,19 @@
                        "lockd_up: create thread failed, error=%d\n",
error);
                goto destroy_and_out;
        }
-       down(&lockd_start);
+       /*
+        * Wait for the lockd process to start, but since we're holding
+        * the lockd semaphore, we can't wait around forever ...
+        */
+       clear_thread_flag(TIF_SIGPENDING);
+       interruptible_sleep_on_timeout(&lockd_start, HZ);
+       if (!nlmsvc_pid) {
+               printk(KERN_WARNING
+                       "lockd_down: lockd failed to start\n");
+       }
+       spin_lock_irq(&current->sighand->siglock);
+       recalc_sigpending();
+       spin_unlock_irq(&current->sighand->siglock);

        /*
         * Note: svc_serv structures have an initial use count of 1,
@@ -423,7 +436,6 @@

 static int __init init_nlm(void)
 {
-       init_MUTEX_LOCKED(&lockd_start);
        nlm_sysctl_table = register_sysctl_table(nlm_sysctl_root, 0);
        return nlm_sysctl_table ? 0 : -ENOMEM;
 }


Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
 mountdBUG: semaphore deadlock detected!
.. task nfsd/1246 is holding c89f4560.
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
       00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000
       00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000
Call Trace:
 [<c0117fc7>] copy_process+0x5f7/0xc30
 [<c01c4e62>] __delay+0x12/0x20
 [<c01fe460>] serial8250_console_write+0x170/0x280
 [<c011930e>] __call_console_drivers+0x5e/0x60
 [<c011982a>] release_console_sem+0xda/0xe0
 [<c01196a7>] vprintk+0x107/0x160
 [<c01196a7>] vprintk+0x107/0x160
 [<c012a3ae>] __kernel_text_address+0x2e/0x40
 [<c0106bf5>] show_trace+0x35/0x90
 [<c0106bf5>] show_trace+0x35/0x90
 [<c0106cd0>] show_stack+0x80/0xa0
 [<c01c2976>] __rwsem_deadlock+0x136/0x180
 [<c01c28aa>] __rwsem_deadlock+0x6a/0x180
 [<c02c3c33>] __down_write+0x93/0x170
 [<c01c3033>] down_write+0x43/0x80
 [<c89e661d>] lockd_up+0x9d/0x120 [lockd]
 [<c89e62e0>] lockd+0x0/0x2a0 [lockd]
 [<c883d2e1>] nfsd+0x91/0x380 [nfsd]
 [<c0105e12>] ret_from_fork+0x6/0x14
 [<c883d250>] nfsd+0x0/0x380 [nfsd]
 [<c0103ffd>] kernel_thread_helper+0x5/0x18
...#0 task nfsd/1246 is holding c89f4560.
BUG: semaphore deadlock: nfsd/1252 is blocked on c89f4560, deadlocking
nfsd/1246
c733df60 00000046 c7adad10 c03d6060 00000000 00000000 00000000 00000000
       00000000 00000000 c7a306f0 00005b6e 39133ac7 0000001a c7adae6c
c7adad10
       c7adad10 c7584400 00000000 c02c3cad c89f4560 c6853f6c c6819f6c
c7adad10
Call Trace:
 [<c02c3cad>] __down_write+0x10d/0x170
 [<c01c3033>] down_write+0x43/0x80
 [<c89e6591>] lockd_up+0x11/0x120 [lockd]
 [<c883d250>] nfsd+0x0/0x380 [nfsd]
 [<c883d2e1>] nfsd+0x91/0x380 [nfsd]
 [<c0105e12>] ret_from_fork+0x6/0x14
 [<c883d250>] nfsd+0x0/0x380 [nfsd]
 [<c0103ffd>] kernel_thread_helper+0x5/0x18
------------[ cut here ]------------

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Thomas Gleixner]

On Mon, 2004-10-18 at 16:50, Ingo Molnar wrote:
> i have released the -U5 Real-Time Preemption patch:

All sleep_on variants trigger the irqs_disabled() check in schedule(). 
tglx

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* Thomas Gleixner <tglx@linutronix.de> wrote:

> On Mon, 2004-10-18 at 16:50, Ingo Molnar wrote:
> > i have released the -U5 Real-Time Preemption patch:
> 
> All sleep_on variants trigger the irqs_disabled() check in schedule(). 
> tglx

ah, forgot that the waitqueue lock is a raw lock. Is there _any_
scenario where sleep_on() is actually correct kernel code?

	Ingo

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Thomas Gleixner]

On Tue, 2004-10-19 at 11:04, Ingo Molnar wrote:
> > All sleep_on variants trigger the irqs_disabled() check in schedule(). 
> > tglx
> 
> ah, forgot that the waitqueue lock is a raw lock. Is there _any_
> scenario where sleep_on() is actually correct kernel code?

Hmm, the sleep_on() variants are used quite a lot over the kernel. Whats
wrong with them and to what should they be converted ?

tglx

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* Thomas Gleixner <tglx@linutronix.de> wrote:

> On Tue, 2004-10-19 at 11:04, Ingo Molnar wrote:
> > > All sleep_on variants trigger the irqs_disabled() check in schedule(). 
> > > tglx
> > 
> > ah, forgot that the waitqueue lock is a raw lock. Is there _any_
> > scenario where sleep_on() is actually correct kernel code?
> 
> Hmm, the sleep_on() variants are used quite a lot over the kernel.
> Whats wrong with them and to what should they be converted ?

they are racy on SMP. It does:

	current->state = TASK_INTERRUPTIBLE;

	schedule();

which is almost always a bug to go to sleep via sleep_on() _after_
checking for the condition, because the following could happen:

	CPU1				CPU2

	if (condition)
		goto done;

					wake_up(&waitqueue);

	current->state = TASK_INTERRUPTIBLE;

	schedule();

The proper interface is wait_event() (and variants).

your patch probably only works due to timing - the wakeup always happens
after sleep_on() has been called.

this particular NFS case is probably only correct due to userspace
behavior. The code is apparently relying on the wake_up() never
happening _before_ we do the sleep_on().

so, could you try the init_MUTEX_LOCKED() fix plus the patch below -
does that turn off the deadlock assert? (Plus also uncomment the
RWSEM_BUG() around line 130.)

	Ingo

--- linux/lib/rwsem-generic.c.orig
+++ linux/lib/rwsem-generic.c
@@ -750,6 +750,15 @@ void fastcall sema_init(struct semaphore
 	case 0:
 		init_rwsem(&sem->lock);
 		down(sem);
+#ifdef CONFIG_RWSEM_DEADLOCK_DETECT
+		{
+			unsigned long flags;
+
+			rwsem_lock_irqsave(&rwsem_lock, flags);
+			rwsem_owner_del(&sem->lock);
+			rwsem_unlock_irqrestore(&rwsem_lock, flags);
+		}
+#endif
 		break;
 	default:
 		RWSEM_BUG();

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* Ingo Molnar <mingo@elte.hu> wrote:

> so, could you try the init_MUTEX_LOCKED() fix plus the patch below -
> does that turn off the deadlock assert? (Plus also uncomment the
> RWSEM_BUG() around line 130.)

but i agree with you that this semaphore (ab-)use in the NFS code should
be fixed. Best would be to use a completion, but unfortunately there's
no wait_for_completion_timeout() API right now.

	Ingo

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Thomas Gleixner]

On Tue, 2004-10-19 at 11:34, Ingo Molnar wrote:
> > Hmm, the sleep_on() variants are used quite a lot over the kernel.
> > Whats wrong with them and to what should they be converted ?
> 
> they are racy on SMP. It does:
> The proper interface is wait_event() (and variants).

Sorry for beeing stupid. I remebered the wait_event stuff immidiately
after hitting send (:

> your patch probably only works due to timing - the wakeup always happens
> after sleep_on() has been called.
> 
> this particular NFS case is probably only correct due to userspace
> behavior. The code is apparently relying on the wake_up() never
> happening _before_ we do the sleep_on().

Correct fix appended. I think it's more sane than the locked mutex, as
we actually come back if lockd is not started for any reason.

> so, could you try the init_MUTEX_LOCKED() fix plus the patch below -
> does that turn off the deadlock assert? (Plus also uncomment the
> RWSEM_BUG() around line 130.)

Yep, that fixes the problem

tglx

--- 2.6.9-rc4-mm1-RT-U5/fs/lockd/svc.c.orig 2004-10-19
10:02:17.000000000 +0200
+++ 2.6.9-rc4-mm1-RT-U5/fs/lockd/svc.c     2004-10-19 11:34:12.000000000
+0200
@@ -46,7 +46,7 @@
 int                            nlmsvc_grace_period;
 unsigned long                  nlmsvc_timeout;

-static DECLARE_MUTEX(lockd_start);
+static DECLARE_WAIT_QUEUE_HEAD(lockd_start);
 static DECLARE_WAIT_QUEUE_HEAD(lockd_exit);

 /*
@@ -109,7 +109,7 @@
         * Let our maker know we're running.
         */
        nlmsvc_pid = current->pid;
-       up(&lockd_start);
+       wake_up(&lockd_start);

        daemonize("lockd");

@@ -230,6 +230,7 @@
                printk(KERN_WARNING
                        "lockd_up: no pid, %d users??\n", nlmsvc_users);

+
        error = -ENOMEM;
        serv = svc_create(&nlmsvc_program, LOCKD_BUFSIZE);
        if (!serv) {
@@ -258,8 +259,15 @@
                        "lockd_up: create thread failed, error=%d\n",
error);
                goto destroy_and_out;
        }
-       down(&lockd_start);
-
+       /*
+        * Wait for the lockd process to start, but since we're holding
+        * the lockd semaphore, we can't wait around forever ...
+        */
+       if (wait_event_interruptible_timeout(lockd_start,
+                                            nlmsvc_pid != 0, HZ)) {
+               printk(KERN_WARNING
+                       "lockd_down: lockd failed to start\n");
+       }
        /*
         * Note: svc_serv structures have an initial use count of 1,
         * so we exit through here on both success and failure.
@@ -298,16 +306,12 @@
         * Wait for the lockd process to exit, but since we're holding
         * the lockd semaphore, we can't wait around forever ...
         */
-       clear_thread_flag(TIF_SIGPENDING);
-       interruptible_sleep_on_timeout(&lockd_exit, HZ);
-       if (nlmsvc_pid) {
+       if (wait_event_interruptible_timeout(lockd_exit,
+                                            nlmsvc_pid == 0, HZ)) {
                printk(KERN_WARNING
                        "lockd_down: lockd failed to exit, clearing
pid\n");
                nlmsvc_pid = 0;
        }
-       spin_lock_irq(&current->sighand->siglock);
-       recalc_sigpending();
-       spin_unlock_irq(&current->sighand->siglock);
 out:
        up(&nlmsvc_sema);
 }
@@ -423,7 +427,6 @@

 static int __init init_nlm(void)
 {
-       init_MUTEX_LOCKED(&lockd_start);
        nlm_sysctl_table = register_sysctl_table(nlm_sysctl_root, 0);
        return nlm_sysctl_table ? 0 : -ENOMEM;
 }

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* Thomas Gleixner <tglx@linutronix.de> wrote:

> +        * Wait for the lockd process to start, but since we're holding
> +        * the lockd semaphore, we can't wait around forever ...
> +        */
> +       if (wait_event_interruptible_timeout(lockd_start,
> +                                            nlmsvc_pid != 0, HZ)) {
> +               printk(KERN_WARNING
> +                       "lockd_down: lockd failed to start\n");

yeah, this is much cleaner. I'd suggest to remove the init_sem() hack
from lib/rwsem-generic.c, it seems it is a nice facility to find
semaphore abuses.

	Ingo

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Thomas Gleixner]

[-- Attachment #1: Type: text/plain, Size: 786 bytes --]

On Tue, 2004-10-19 at 13:07, Ingo Molnar wrote:
> * Thomas Gleixner <tglx@linutronix.de> wrote:
> 
> > +        * Wait for the lockd process to start, but since we're holding
> > +        * the lockd semaphore, we can't wait around forever ...
> > +        */
> > +       if (wait_event_interruptible_timeout(lockd_start,
> > +                                            nlmsvc_pid != 0, HZ)) {
> > +               printk(KERN_WARNING
> > +                       "lockd_down: lockd failed to start\n");
> 
> yeah, this is much cleaner. 

Cleaner, but not perfect. The return value is > 0, if the timeout is not
reached. Grmbl.

> I'd suggest to remove the init_sem() hack
> from lib/rwsem-generic.c, it seems it is a nice facility to find
> semaphore abuses.

True. Will do so.

tglx



[-- Attachment #2: svc.c.diff --]
[-- Type: text/x-patch, Size: 2084 bytes --]

--- 2.6.9-rc4-mm1-RT-U5/fs/lockd/svc.c.orig	2004-10-19 10:02:17.000000000 +0200
+++ 2.6.9-rc4-mm1-VP-U5/fs/lockd/svc.c	2004-10-19 12:59:41.000000000 +0200
@@ -46,7 +46,7 @@
 int				nlmsvc_grace_period;
 unsigned long			nlmsvc_timeout;
 
-static DECLARE_MUTEX(lockd_start);
+static DECLARE_WAIT_QUEUE_HEAD(lockd_start);
 static DECLARE_WAIT_QUEUE_HEAD(lockd_exit);
 
 /*
@@ -109,7 +109,7 @@
 	 * Let our maker know we're running.
 	 */
 	nlmsvc_pid = current->pid;
-	up(&lockd_start);
+	wake_up(&lockd_start);
 
 	daemonize("lockd");
 
@@ -230,6 +230,7 @@
 		printk(KERN_WARNING
 			"lockd_up: no pid, %d users??\n", nlmsvc_users);
 
+
 	error = -ENOMEM;
 	serv = svc_create(&nlmsvc_program, LOCKD_BUFSIZE);
 	if (!serv) {
@@ -258,8 +259,15 @@
 			"lockd_up: create thread failed, error=%d\n", error);
 		goto destroy_and_out;
 	}
-	down(&lockd_start);
-
+	/*
+	 * Wait for the lockd process to start, but since we're holding
+	 * the lockd semaphore, we can't wait around forever ...
+	 */
+	if (wait_event_interruptible_timeout(lockd_start, 
+					     nlmsvc_pid != 0, HZ) <= 0) {
+		printk(KERN_WARNING 
+			"lockd_down: lockd failed to start\n");
+	}
 	/*
 	 * Note: svc_serv structures have an initial use count of 1,
 	 * so we exit through here on both success and failure.
@@ -298,16 +306,12 @@
 	 * Wait for the lockd process to exit, but since we're holding
 	 * the lockd semaphore, we can't wait around forever ...
 	 */
-	clear_thread_flag(TIF_SIGPENDING);
-	interruptible_sleep_on_timeout(&lockd_exit, HZ);
-	if (nlmsvc_pid) {
+	if (!wait_event_interruptible_timeout(lockd_exit, 
+					     nlmsvc_pid == 0, HZ) <= 0) {
 		printk(KERN_WARNING 
 			"lockd_down: lockd failed to exit, clearing pid\n");
 		nlmsvc_pid = 0;
 	}
-	spin_lock_irq(&current->sighand->siglock);
-	recalc_sigpending();
-	spin_unlock_irq(&current->sighand->siglock);
 out:
 	up(&nlmsvc_sema);
 }
@@ -423,7 +427,6 @@
 
 static int __init init_nlm(void)
 {
-	init_MUTEX_LOCKED(&lockd_start);
 	nlm_sysctl_table = register_sysctl_table(nlm_sysctl_root, 0);
 	return nlm_sysctl_table ? 0 : -ENOMEM;
 }

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Michal Schmidt]

[-- Attachment #1: Type: text/plain, Size: 507 bytes --]

Ingo Molnar wrote:
> there are two big changes:
> 
>  - debug feature: automatic semaphore/rwsem deadlock detection, based on
>    the code from Igor Manyilov and Bill Huey.
> 
> this is a very nice feature that should help in debugging the remaining
> deadlocks. The deadlock detection feature has already helped me fix a
> bug that was causing hangs in the VFS, so it's really useful.

I can trigger the deadlock detection when I try to ping my default 
gateway. I'm attaching the log from dmesg.

Michal

[-- Attachment #2: ping-deadlock.txt --]
[-- Type: text/plain, Size: 11700 bytes --]

BUG: semaphore deadlock detected!
.. task ksoftirqd/0/2 is holding c04019c4.
0000012c 00000001 c14f4000 00000000 c14f5f9c c011e713 c03a36b8 c011e7d9 
       c011eb18 0000000a c03a36b8 c14f4000 c14f4000 00000000 c14f5fa4 c011e7f1 
       c14f5fbc c011eb18 00000001 fffffff6 c14f4000 c14e3f70 c14f5fec c012d333 
Call Trace:
 [<c011e713>] ___do_softirq+0x83/0xc8
 [<c011e7d9>] _do_softirq+0x8/0x22
 [<c011eb18>] ksoftirqd+0x93/0xd1
 [<c011e7f1>] _do_softirq+0x20/0x22
 [<c011eb18>] ksoftirqd+0x93/0xd1
 [<c012d333>] kthread+0xaa/0xaf
 [<c011ea85>] ksoftirqd+0x0/0xd1
 [<c012d289>] kthread+0x0/0xaf
 [<c01032f1>] kernel_thread_helper+0x5/0xb
preempt count: 00000003
. 3-level deep critical section nesting:
.. entry 1: down_write+0x95/0x97 / (dev_queue_xmit_nit+0x41/0x127)
.. entry 2: __down_write+0x43/0x190 / (down_write+0x52/0x97)
.. entry 3: print_traces+0x1d/0x56 / (show_stack+0x85/0x9b)

...#0 task ksoftirqd/0/2 is holding c04019c4.
------------[ cut here ]------------
kernel BUG at lib/rwsem-generic.c:416!
invalid operand: 0000 [#1]
PREEMPT 
Modules linked in: ipt_REJECT ipt_pkttype ipt_LOG ipt_state ipt_multiport ipt_conntrack iptable_mangle ip_nat_irc ip_nat_tftp ip_nat_ftp iptable_nat ip_conntrack_irc ip_conntrack_tftp ip_conntrack_ftp ip_conntrack iptable_filter ip_tables af_packet 8139too 8139cp mii crc32 e1000 snd_intel8x0 snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc ehci_hcd usbhid uhci_hcd usbcore intel_mch_agp intel_agp evdev nls_iso8859_1 nls_cp437 vfat fat dm_mod floppy ide_cd cdrom psmouse unix
CPU:    0
EIP:    0060:[<c0289aca>]    Not tainted VLI
EFLAGS: 00010002   (2.6.9-rc4-mm1-RT-U5) 
EIP is at __down_write+0xad/0x190
eax: 00000001   ebx: c14e0d20   ecx: 00000000   edx: c14f4000
esi: c04019c4   edi: dff1b080   ebp: c14f5d64   esp: c14f5d48
ds: 007b   es: 007b   ss: 0068   preempt: 00000003
Process ksoftirqd/0 (pid: 2, threadinfo=c14f4000 task=c14e0d20)
Stack: c04019c4 c04019c8 c04019c8 c14e0d20 00000002 c14f4000 c04019c4 c14f5d7c 
       c01b33c8 c04019c4 c14e0d20 dec0f000 df59a000 c14f5da0 c02364c5 c04019c0 
       00000202 c14f5da0 df59a000 dec0f000 df59a000 df59a358 c14f5dc4 c02420cd 
Call Trace:
 [<c01b33c8>] down_write+0x52/0x97
 [<c02364c5>] dev_queue_xmit_nit+0x41/0x127
 [<c02420cd>] qdisc_restart+0x182/0x1ac
 [<c0236964>] dev_queue_xmit+0x194/0x230
 [<c023c3e3>] neigh_resolve_output+0xf1/0x1e5
 [<c023bf8d>] neigh_update+0x2c3/0x368
 [<c0272642>] arp_process+0x1f8/0x5ac
 [<c01132d0>] mcount+0x14/0x18
 [<c0272a0a>] arp_rcv+0x14/0x155
 [<c0236e14>] netif_receive_skb+0x10f/0x1ab
 [<c0241c7e>] eth_type_trans+0xe/0xc8
 [<e097582f>] rtl8139_rx+0x185/0x326 [8139too]
 [<e0975bbb>] rtl8139_poll+0x5d/0xfe [8139too]
 [<c023707b>] net_rx_action+0x7c/0x143
 [<c011e713>] ___do_softirq+0x83/0xc8
 [<c011e7d9>] _do_softirq+0x8/0x22
 [<c011eb18>] ksoftirqd+0x93/0xd1
 [<c011e7f1>] _do_softirq+0x20/0x22
 [<c011eb18>] ksoftirqd+0x93/0xd1
 [<c012d333>] kthread+0xaa/0xaf
 [<c011ea85>] ksoftirqd+0x0/0xd1
 [<c012d289>] kthread+0x0/0xaf
 [<c01032f1>] kernel_thread_helper+0x5/0xb
preempt count: 00000004
. 4-level deep critical section nesting:
.. entry 1: down_write+0x95/0x97 / (dev_queue_xmit_nit+0x41/0x127)
.. entry 2: __down_write+0x43/0x190 / (down_write+0x52/0x97)
.. entry 3: die+0x3f/0x196 / (do_invalid_op+0x106/0x108)
.. entry 4: print_traces+0x1d/0x56 / (show_stack+0x85/0x9b)

Code: 10 89 45 ec 89 34 24 e8 47 90 f2 ff 89 34 24 e8 04 91 f2 ff 85 c0 74 1b a1 58 36 2d c0 85 c0 74 12 c7 05 58 36 2d c0 00 00 00 00 <0f> 0b a0 01 2f 32 2a c0 89 b3 70 06 00 00 9c 58 c1 e8 09 83 f0 
 <3>BUG: sleeping function called from invalid context ksoftirqd/0(2) at lib/rwsem-generic.c:593
in_atomic():1 [00000002], irqs_disabled():0
 [<c0116e6c>] __might_sleep+0xc4/0xd6
 [<c01b3275>] down_read+0x27/0x96
 [<c011a353>] profile_task_exit+0x1b/0x43
 [<c01132d0>] mcount+0x14/0x18
 [<c011bfbf>] do_exit+0x1f/0x4a3
 [<c0105e9a>] do_invalid_op+0x0/0x108
 [<c0105af8>] do_divide_error+0x0/0x132
 [<c0114a1c>] fixup_exception+0x1c/0x38
 [<c0105fa0>] do_invalid_op+0x106/0x108
 [<c01132d0>] mcount+0x14/0x18
 [<c0289aca>] __down_write+0xad/0x190
 [<c0119e3b>] release_console_sem+0xa7/0xfe
 [<c0119ce2>] vprintk+0x116/0x179
 [<c0105309>] error_code+0x2d/0x38
 [<c0289aca>] __down_write+0xad/0x190
 [<c01b33c8>] down_write+0x52/0x97
 [<c02364c5>] dev_queue_xmit_nit+0x41/0x127
 [<c02420cd>] qdisc_restart+0x182/0x1ac
 [<c0236964>] dev_queue_xmit+0x194/0x230
 [<c023c3e3>] neigh_resolve_output+0xf1/0x1e5
 [<c023bf8d>] neigh_update+0x2c3/0x368
 [<c0272642>] arp_process+0x1f8/0x5ac
 [<c01132d0>] mcount+0x14/0x18
 [<c0272a0a>] arp_rcv+0x14/0x155
 [<c0236e14>] netif_receive_skb+0x10f/0x1ab
 [<c0241c7e>] eth_type_trans+0xe/0xc8
 [<e097582f>] rtl8139_rx+0x185/0x326 [8139too]
 [<e0975bbb>] rtl8139_poll+0x5d/0xfe [8139too]
 [<c023707b>] net_rx_action+0x7c/0x143
 [<c011e713>] ___do_softirq+0x83/0xc8
 [<c011e7d9>] _do_softirq+0x8/0x22
 [<c011eb18>] ksoftirqd+0x93/0xd1
 [<c011e7f1>] _do_softirq+0x20/0x22
 [<c011eb18>] ksoftirqd+0x93/0xd1
 [<c012d333>] kthread+0xaa/0xaf
 [<c011ea85>] ksoftirqd+0x0/0xd1
 [<c012d289>] kthread+0x0/0xaf
 [<c01032f1>] kernel_thread_helper+0x5/0xb
preempt count: 00000003
. 3-level deep critical section nesting:
.. entry 1: down_write+0x95/0x97 / (dev_queue_xmit_nit+0x41/0x127)
.. entry 2: __down_write+0x43/0x190 / (down_write+0x52/0x97)
.. entry 3: print_traces+0x1d/0x56 / (dump_stack+0x23/0x27)

BUG: scheduling while atomic: ksoftirqd/0/0x04000002/2
caller is cond_resched+0x62/0x82
 [<c0288df2>] __sched_text_start+0x53e/0x571
 [<c02894c4>] cond_resched+0x62/0x82
 [<c01132d0>] mcount+0x14/0x18
 [<c02894c4>] cond_resched+0x62/0x82
 [<c01b327a>] down_read+0x2c/0x96
 [<c011a353>] profile_task_exit+0x1b/0x43
 [<c01132d0>] mcount+0x14/0x18
 [<c011bfbf>] do_exit+0x1f/0x4a3
 [<c0105e9a>] do_invalid_op+0x0/0x108
 [<c0105af8>] do_divide_error+0x0/0x132
 [<c0114a1c>] fixup_exception+0x1c/0x38
 [<c0105fa0>] do_invalid_op+0x106/0x108
 [<c01132d0>] mcount+0x14/0x18
 [<c0289aca>] __down_write+0xad/0x190
(ksoftirqd/0/2/CPU#0): new 856 us maximum-latency critical section.
 => started at timestamp 144481672: <__call_console_drivers+0x19/0x65>
 =>   ended at timestamp 144482528: <call_console_drivers+0x9e/0x13f>
 [<c012ec10>] touch_preempt_timing+0x3d/0x41
 [<c012eb37>] check_preempt_timing+0x1b2/0x24e
 [<c0119a54>] call_console_drivers+0x9e/0x13f
 [<c012ec10>] touch_preempt_timing+0x3d/0x41
 [<c0119a54>] call_console_drivers+0x9e/0x13f
 [<c0119a54>] call_console_drivers+0x9e/0x13f
 [<c0288edb>] preempt_schedule+0x11/0x7a
 [<c0119e04>] release_console_sem+0x70/0xfe
 [<c0119ce2>] vprintk+0x116/0x179
 [<c0289aca>] __down_write+0xad/0x190
 [<c0119bc8>] printk+0x1d/0x21
 [<c01055f8>] show_trace+0x78/0xb2
 [<c0289aca>] __down_write+0xad/0x190
 [<c01056f0>] dump_stack+0x23/0x27
 [<c0288df2>] __sched_text_start+0x53e/0x571
 [<c02894c4>] cond_resched+0x62/0x82
 [<c01132d0>] mcount+0x14/0x18
 [<c02894c4>] cond_resched+0x62/0x82
 [<c01b327a>] down_read+0x2c/0x96
 [<c011a353>] profile_task_exit+0x1b/0x43
 [<c01132d0>] mcount+0x14/0x18
 [<c011bfbf>] do_exit+0x1f/0x4a3
 [<c0105e9a>] do_invalid_op+0x0/0x108
 [<c0105af8>] do_divide_error+0x0/0x132
 [<c0114a1c>] fixup_exception+0x1c/0x38
 [<c0105fa0>] do_invalid_op+0x106/0x108
 [<c01132d0>] mcount+0x14/0x18
 [<c0289aca>] __down_write+0xad/0x190
 [<c0119e3b>] release_console_sem+0xa7/0xfe
 [<c0119ce2>] vprintk+0x116/0x179
 [<c0105309>] error_code+0x2d/0x38
 [<c0289aca>] __down_write+0xad/0x190
 [<c01b33c8>] down_write+0x52/0x97
 [<c02364c5>] dev_queue_xmit_nit+0x41/0x127
 [<c02420cd>] qdisc_restart+0x182/0x1ac
 [<c0236964>] dev_queue_xmit+0x194/0x230
 [<c023c3e3>] neigh_resolve_output+0xf1/0x1e5
 [<c023bf8d>] neigh_update+0x2c3/0x368
 [<c0272642>] arp_process+0x1f8/0x5ac
 [<c01132d0>] mcount+0x14/0x18
 [<c0272a0a>] arp_rcv+0x14/0x155
 [<c0236e14>] netif_receive_skb+0x10f/0x1ab
 [<c0241c7e>] eth_type_trans+0xe/0xc8
 [<e097582f>] rtl8139_rx+0x185/0x326 [8139too]
 [<e0975bbb>] rtl8139_poll+0x5d/0xfe [8139too]
 [<c023707b>] net_rx_action+0x7c/0x143
 [<c011e713>] ___do_softirq+0x83/0xc8
 [<c011e7d9>] _do_softirq+0x8/0x22
 [<c011eb18>] ksoftirqd+0x93/0xd1
 [<c011e7f1>] _do_softirq+0x20/0x22
 [<c011eb18>] ksoftirqd+0x93/0xd1
 [<c012d333>] kthread+0xaa/0xaf
 [<c011ea85>] ksoftirqd+0x0/0xd1
 [<c012d289>] kthread+0x0/0xaf
 [<c01032f1>] kernel_thread_helper+0x5/0xb
preempt count: 04000003
. 3-level deep critical section nesting:
.. entry 1: down_write+0x95/0x97 / (dev_queue_xmit_nit+0x41/0x127)
.. entry 2: __down_write+0x43/0x190 / (down_write+0x52/0x97)
.. entry 3: print_traces+0x1d/0x56 / (dump_stack+0x23/0x27)

 =>   dump-end timestamp 144483248

 [<c0119e3b>] release_console_sem+0xa7/0xfe
 [<c0119ce2>] vprintk+0x116/0x179
 [<c0105309>] error_code+0x2d/0x38
 [<c0289aca>] __down_write+0xad/0x190
 [<c01b33c8>] down_write+0x52/0x97
 [<c02364c5>] dev_queue_xmit_nit+0x41/0x127
 [<c02420cd>] qdisc_restart+0x182/0x1ac
 [<c0236964>] dev_queue_xmit+0x194/0x230
 [<c023c3e3>] neigh_resolve_output+0xf1/0x1e5
 [<c023bf8d>] neigh_update+0x2c3/0x368
 [<c0272642>] arp_process+0x1f8/0x5ac
 [<c01132d0>] mcount+0x14/0x18
 [<c0272a0a>] arp_rcv+0x14/0x155
 [<c0236e14>] netif_receive_skb+0x10f/0x1ab
 [<c0241c7e>] eth_type_trans+0xe/0xc8
 [<e097582f>] rtl8139_rx+0x185/0x326 [8139too]
 [<e0975bbb>] rtl8139_poll+0x5d/0xfe [8139too]
 [<c023707b>] net_rx_action+0x7c/0x143
 [<c011e713>] ___do_softirq+0x83/0xc8
 [<c011e7d9>] _do_softirq+0x8/0x22
 [<c011eb18>] ksoftirqd+0x93/0xd1
 [<c011e7f1>] _do_softirq+0x20/0x22
 [<c011eb18>] ksoftirqd+0x93/0xd1
 [<c012d333>] kthread+0xaa/0xaf
 [<c011ea85>] ksoftirqd+0x0/0xd1
 [<c012d289>] kthread+0x0/0xaf
 [<c01032f1>] kernel_thread_helper+0x5/0xb
preempt count: 04000003
. 3-level deep critical section nesting:
.. entry 1: down_write+0x95/0x97 / (dev_queue_xmit_nit+0x41/0x127)
.. entry 2: __down_write+0x43/0x190 / (down_write+0x52/0x97)
.. entry 3: print_traces+0x1d/0x56 / (dump_stack+0x23/0x27)

note: ksoftirqd/0[2] exited with preempt_count 2
BUG: scheduling while atomic: ksoftirqd/0/0x00000002/2
caller is do_exit+0x2a7/0x4a3
 [<c0288df2>] __sched_text_start+0x53e/0x571
 [<c011c247>] do_exit+0x2a7/0x4a3
 [<c01132d0>] mcount+0x14/0x18
 [<c011c247>] do_exit+0x2a7/0x4a3
 [<c0105e9a>] do_invalid_op+0x0/0x108
 [<c0105af8>] do_divide_error+0x0/0x132
 [<c0114a1c>] fixup_exception+0x1c/0x38
 [<c0105fa0>] do_invalid_op+0x106/0x108
 [<c01132d0>] mcount+0x14/0x18
 [<c0289aca>] __down_write+0xad/0x190
 [<c0119e3b>] release_console_sem+0xa7/0xfe
 [<c0119ce2>] vprintk+0x116/0x179
 [<c0105309>] error_code+0x2d/0x38
 [<c0289aca>] __down_write+0xad/0x190
 [<c01b33c8>] down_write+0x52/0x97
 [<c02364c5>] dev_queue_xmit_nit+0x41/0x127
 [<c02420cd>] qdisc_restart+0x182/0x1ac
 [<c0236964>] dev_queue_xmit+0x194/0x230
 [<c023c3e3>] neigh_resolve_output+0xf1/0x1e5
 [<c023bf8d>] neigh_update+0x2c3/0x368
 [<c0272642>] arp_process+0x1f8/0x5ac
 [<c01132d0>] mcount+0x14/0x18
 [<c0272a0a>] arp_rcv+0x14/0x155
 [<c0236e14>] netif_receive_skb+0x10f/0x1ab
 [<c0241c7e>] eth_type_trans+0xe/0xc8
 [<e097582f>] rtl8139_rx+0x185/0x326 [8139too]
 [<e0975bbb>] rtl8139_poll+0x5d/0xfe [8139too]
 [<c023707b>] net_rx_action+0x7c/0x143
 [<c011e713>] ___do_softirq+0x83/0xc8
 [<c011e7d9>] _do_softirq+0x8/0x22
 [<c011eb18>] ksoftirqd+0x93/0xd1
 [<c011e7f1>] _do_softirq+0x20/0x22
 [<c011eb18>] ksoftirqd+0x93/0xd1
 [<c012d333>] kthread+0xaa/0xaf
 [<c011ea85>] ksoftirqd+0x0/0xd1
 [<c012d289>] kthread+0x0/0xaf
 [<c01032f1>] kernel_thread_helper+0x5/0xb
preempt count: 00000003
. 3-level deep critical section nesting:
.. entry 1: down_write+0x95/0x97 / (dev_queue_xmit_nit+0x41/0x127)
.. entry 2: __down_write+0x43/0x190 / (down_write+0x52/0x97)
.. entry 3: print_traces+0x1d/0x56 / (dump_stack+0x23/0x27)

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Kevin Hilman]

Ingo Molnar <mingo@elte.hu> writes:

> i have released the -U5 Real-Time Preemption patch:

[...] 

> the generic semaphore implementation (which uses rwsems) makes it
> possible to use the deadlock detection mechanism for all the mutex types
> we currently have: semaphores, rw-semaphores, spinlock-mutexes and
> rwlock-mutexes. Another benefit is that PREEMPT_REALTIME becomes much
> more portable this way. (although it's still x86-only at the moment.)

Speaking of portability, is anyone yet working on ports to other
platforms?  I'm particularily interested in ARM.

If anyone has started on ARM, I'd be glad to help out.

Kevin
http://hilman.org/kevin/ 

Re: [patch] Real-Time Preemption, -RT-2.6.9-rc4-mm1-U5

[Ingo Molnar]

* Kevin Hilman <kevin@hilman.org> wrote:

> > the generic semaphore implementation (which uses rwsems) makes it
> > possible to use the deadlock detection mechanism for all the mutex types
> > we currently have: semaphores, rw-semaphores, spinlock-mutexes and
> > rwlock-mutexes. Another benefit is that PREEMPT_REALTIME becomes much
> > more portable this way. (although it's still x86-only at the moment.)
> 
> Speaking of portability, is anyone yet working on ports to other
> platforms?  I'm particularily interested in ARM.

i'll do x64 a couple of days after stability has been reached. I dont
know of any ARM efforts though.

a good starting point would be to enhance the generic-hardirqs framework
for ARM. Generic-hardirqs is a portion of the PREEMPT_REALTIME patch
that i've split out and submitted upstream, and which is expected to be
merged into 2.6.10. The generic irq subsystem makes the irq-threading
feature really simple and maintainable. So for PREEMPT_REALTIME to work
on ARM the first step is to enable generic-hardirqs on ARM. (which is
far from simple though.)

	Ingo