Re: 3.2-rc1 and nvidia drivers

[Thomas Schauss <schauss@tum.de>]

[-- Attachment #1: Type: text/plain, Size: 5761 bytes --]

Hello,

actually I have wanted to write to this list about NVIDIA-drivers for 
some time and will use this opportunity.

Generally, it is obviously preferable to use the nouveau-driver if 
possible which is working just fine. At our institute however, we need 
to use the rt-patch in combination with the proprietary NVIDIA driver as 
we need CUDA-support combined with low-latency control (e.g. for visual 
servoing). And I am sure we are not the only ones requiring this 
combination.

We have been using the 2.6.33-rt29 for some time now without any issues 
and with very satisfactory latency. This required a small patch to the 
NVIDIA-driver (there is a check for PREEMPT_RT) as atomic_spin* was 
renamed to raw_spin* in one of the 2.6-rt-patches.

3.0-rt requires an additional slight modification as CONFIG_PREEMPT_RT 
is not defined anymore and we must check instead for 
CONFIG_PREEMPT_RT_FULL to decide whether to use raw spin_locks. Using 
this patch it is then not necessary to change any EXPORT_SYMBOL_GPL to 
EXPORT_SYMBOL in the kernel as was proposed in some other threads here.

You can find the patches for the nvidia-driver for 2.6.33-rt and 3.0-rt 
below.

Unfortunately, with 3.0-rt and the nvidia-driver we get complete system 
freezes when starting X on several different hardware setups (a few 
systems work fine). This is certainly caused by this combination. When 
using the nouveau-driver everything works fine.

I tested this with rt13 through rt20 and will check with the current 
version tomorrow. If it still fails I will put together a list of 
working vs. non-working hardware setups.

Unfortunately, apart from that I am not sure how to debug this issue, as 
the complete system freezes (including the serial console) and I can't 
find any suspicious messages in the logs. Any ideas?

Btw, changing EXPORT_SYMBOL_GPL to EXPORT_SYMBOL for 
migrate_enable/disable, ... and then using the non-raw spinlocks results 
in exactly the same behavior.

Best Regards,
Thomas Schauss

====================================================================

Patch for 3.0-rt:

--- a/nv-linux.h        2011-10-26 13:35:32.866579965 +0200
+++ b/nv-linux.h        2011-10-26 13:35:47.265117607 +0200
@@ -262,17 +262,17 @@
  #endif
  #endif

-#if defined(CONFIG_PREEMPT_RT)
-typedef atomic_spinlock_t         nv_spinlock_t;
-#define NV_SPIN_LOCK_INIT(lock)   atomic_spin_lock_init(lock)
-#define NV_SPIN_LOCK_IRQ(lock)    atomic_spin_lock_irq(lock)
-#define NV_SPIN_UNLOCK_IRQ(lock)  atomic_spin_unlock_irq(lock)
-#define NV_SPIN_LOCK_IRQSAVE(lock,flags) 
atomic_spin_lock_irqsave(lock,flags)
+#if defined(CONFIG_PREEMPT_RT_FULL)
+typedef raw_spinlock_t            nv_spinlock_t;
+#define NV_SPIN_LOCK_INIT(lock)   raw_spin_lock_init(lock)
+#define NV_SPIN_LOCK_IRQ(lock)    raw_spin_lock_irq(lock)
+#define NV_SPIN_UNLOCK_IRQ(lock)  raw_spin_unlock_irq(lock)
+#define NV_SPIN_LOCK_IRQSAVE(lock,flags) raw_spin_lock_irqsave(lock,flags)
  #define NV_SPIN_UNLOCK_IRQRESTORE(lock,flags) \
-  atomic_spin_unlock_irqrestore(lock,flags)
-#define NV_SPIN_LOCK(lock)        atomic_spin_lock(lock)
-#define NV_SPIN_UNLOCK(lock)      atomic_spin_unlock(lock)
-#define NV_SPIN_UNLOCK_WAIT(lock) atomic_spin_unlock_wait(lock)
+  raw_spin_unlock_irqrestore(lock,flags)
+#define NV_SPIN_LOCK(lock)        raw_spin_lock(lock)
+#define NV_SPIN_UNLOCK(lock)      raw_spin_unlock(lock)
+#define NV_SPIN_UNLOCK_WAIT(lock) raw_spin_unlock_wait(lock)
  #else
  typedef spinlock_t                nv_spinlock_t;
  #define NV_SPIN_LOCK_INIT(lock)   spin_lock_init(lock)
@@ -854,8 +854,8 @@
      return ret;
  }

-#if defined(CONFIG_PREEMPT_RT)
-#define NV_INIT_MUTEX(mutex) semaphore_init(mutex)
+#if defined(CONFIG_PREEMPT_RT_FULL)
+#define NV_INIT_MUTEX(mutex) sema_init(mutex,1)
  #else
  #if !defined(__SEMAPHORE_INITIALIZER) && 
defined(__COMPAT_SEMAPHORE_INITIALIZER)
  #define __SEMAPHORE_INITIALIZER __COMPAT_SEMAPHORE_INITIALIZER

====================================================================

Patch for 2.6.33-rt:

--- a/nv-linux.h        2011-10-28 10:31:47.416915958 +0200
+++ b/nv-linux.h        2011-10-28 10:32:48.592195509 +0200
@@ -263,16 +263,16 @@
  #endif

  #if defined(CONFIG_PREEMPT_RT)
-typedef atomic_spinlock_t         nv_spinlock_t;
-#define NV_SPIN_LOCK_INIT(lock)   atomic_spin_lock_init(lock)
-#define NV_SPIN_LOCK_IRQ(lock)    atomic_spin_lock_irq(lock)
-#define NV_SPIN_UNLOCK_IRQ(lock)  atomic_spin_unlock_irq(lock)
-#define NV_SPIN_LOCK_IRQSAVE(lock,flags) 
atomic_spin_lock_irqsave(lock,flags)
+typedef raw_spinlock_t            nv_spinlock_t;
+#define NV_SPIN_LOCK_INIT(lock)   raw_spin_lock_init(lock)
+#define NV_SPIN_LOCK_IRQ(lock)    raw_spin_lock_irq(lock)
+#define NV_SPIN_UNLOCK_IRQ(lock)  raw_spin_unlock_irq(lock)
+#define NV_SPIN_LOCK_IRQSAVE(lock,flags) raw_spin_lock_irqsave(lock,flags)
  #define NV_SPIN_UNLOCK_IRQRESTORE(lock,flags) \
-  atomic_spin_unlock_irqrestore(lock,flags)
-#define NV_SPIN_LOCK(lock)        atomic_spin_lock(lock)
-#define NV_SPIN_UNLOCK(lock)      atomic_spin_unlock(lock)
-#define NV_SPIN_UNLOCK_WAIT(lock) atomic_spin_unlock_wait(lock)
+  raw_spin_unlock_irqrestore(lock,flags)
+#define NV_SPIN_LOCK(lock)        raw_spin_lock(lock)
+#define NV_SPIN_UNLOCK(lock)      raw_spin_unlock(lock)
+#define NV_SPIN_UNLOCK_WAIT(lock) raw_spin_unlock_wait(lock)
  #else
  typedef spinlock_t                nv_spinlock_t;
  #define NV_SPIN_LOCK_INIT(lock)   spin_lock_init(lock)
@@ -855,7 +855,7 @@
  }

  #if defined(CONFIG_PREEMPT_RT)
-#define NV_INIT_MUTEX(mutex) semaphore_init(mutex)
+#define NV_INIT_MUTEX(mutex) sema_init(mutex,1)
  #else
  #if !defined(__SEMAPHORE_INITIALIZER) && 
defined(__COMPAT_SEMAPHORE_INITIALIZER)
  #define __SEMAPHORE_INITIALIZER __COMPAT_SEMAPHORE_INITIALIZER

[-- Attachment #2: schauss.vcf --]
[-- Type: text/x-vcard, Size: 342 bytes --]

begin:vcard
fn:Thomas Schauss
n:Schauss;Thomas
org:Technische Universitaet Muenchen (TUM);Institute of Automatic Control Engineering (LSR)
adr:;;Theresienstr. 90;Munich;;80333;Germany
email;internet:schauss@tum.de
title:Dipl.-Ing. (Univ.)
tel;work:+49 89 289 23406
tel;fax:+49 89 289 28340
url:http://www.lsr.ei.tum.de
version:2.1
end:vcard