[ANNOUNCE] The -rt git tree

Steven Rostedt
2008-12-09T16:13:55Z

The -rt branch has been static for some time. We are sorry about that,
but most of those that maintain it have been working on other parts of the
kernel. Mostly new mainline work.

People have also been asking about having an -rt git tree. Well, it is
time to create one.

We have started working on an -rt git tree. Note, it is currently broken
and not ready for use. An announcement will be made when it is ready. This
email is only to let people know that we are currently actively working on
the new -rt tree.

The git tree is located here:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-rt.git

The branches include:

linus - the mainline branch that the rest is based on
rt-master - all the rt/* branches merged into one
master - rt-master + converting of the locks (see below)
rt/rt - some rt git scripts
rt/convert-scripts - the scripts to convert spinlocks (see below)
rt/mainline - clean ups for mainline that we need to push forward
rt/not-for-mainline - rt only patches that we never want to push
to mainline
rt/fs - filesystem changes for rt
rt/ftrace - ftrace updates for rt
rt/java - IBM patches for java
rt/kmap
rt/lockdep
rt/pagecache
rt/percpu - this may turn into rt/mm
rt/rt-locks - the new lock API (see below)
rt/sched
rt/tasklet
rt/threadirqs - full soft and hard irq threading
rt/timer
rt/workqueue - priority queues

We may add more rt/* branches. Most of those are self explanatory.

The new lock API. The old -rt patch used macro magic to convert
spin_locks into mutexes depending on which type they were. A spinlock that
was declared as spinlock_t would convert into a mutex when PREEMPT_RT was
configured, where as a spinlock that was declared as raw_spinlock_t would
stay as a spinlock.

For example:

spinlock_t what_am_i;
raw_spinlock_t spinner;

spin_lock(&what_am_i);
spin_lock(&spinner);

The 'what_am_i' would turn into a mutex when PREEMPT_RT was configured and
would be a spinlock when it was not. 'spinner' would always stay as a
spinlock. The way this was done was with a builtin gcc option to compare
the type used to determine what function to call:

#define spin_lock(lock) \
if (__builtin_types_compatible(typeof(spinlock_t), lock)) \
rt_lock(lock); \
else if (__builtin_types_compatible(typeof(raw_spinlock_t), lock)) \
raw_spin_lock(lock); \
else \
__bad_func_type();

If you are looking at this and scratching your head saying "this is not
acceptable for mainline", we agree with you ;-) This is the old way of
doing things, which made porting -rt to mainline much easier than
converting all locks to a new API. But the time has come to do just that.

The new locks
-------------

The old way is not acceptable because a developer can not know when seeing
a 'spin_lock(x)' and knowing if it will stay a spinlock or not on
PREEMPT_RT. The developer would need to go search for x and see what type
it is. Then they still can not know if someone might change the type of
x. This is just ripe for bugs.

The new RT has added a new lock API that will be a mutex when
PREEMPT_RT is configured and a spinlock when it is not. The type is called
'lock_t' and introduces a new API acquire_lock() and release_lock(). This
also has all the irq variants that spin_locks have (acquire_lock_irqsave,
acquire_lock_irq, etc).

lock_t lock;

acquire_lock_irqsave(&lock, flags);

Although it has 'irqsave' when PREEMPT_RT is defined, it will not disable
interrupts. This is fine, because with PREEMPT_RT, interrupts are threads,
and we do not need to worry about being preempted by an interrupt and have
that interrupt taking the same lock cause a deadlock. But now, the
developer knows that this is a special lock, and that the lock is not a
spinlock all the time.

Note, you can not do:

spin_lock(&spinner);
acquire_lock(&lock);

This would be the same as doing:

spin_lock(&spinner);
mutex_lock(&mutex);

Converting the new locks
------------------------

In the git tree, there is a couple of scripts in rt/convert-scripts branch
that will convert all spinlocks into the lock_t. Then there are scripts to
convert the necessary spinlocks back to spinlock. All rt/* branches
should not need to define the type lock_t except for rt/rt-locks.

This git repo uses similar scripts that tip uses. The rt-master branch is
a merge of all rt/* branches (except rt/rt). This may define PREEMPT_RT
but no lock is made into 'lock_t' type yet.

The master branch is a copy of the rt-branch with the scripts to do the
lock conversion run on the tree. In other words, the master branch will
be the new -rt patch.

Note, the current state of this git tree is still very much broken. We are
rewriting a lot of code to pull this forward to 2.6.28. Unfortunately,
this means that we are skipping 2.6.27 altogether. I will be making
another announcement when this is ready for prime time. Some of the
branches have not been updated with the changes either.

-- Steve