[RFC][PATCH 2/4] change mnt_writers underflow protection logic

Dave Hansen
2008-01-10T19:08:51Z

The comment tells most of the story. I want to make the
spinlock in this case into a mutex, and the current
underflow protection mechanism uses preempt disabling from
put/get_cpu_Var(). I can't use that with a mutex.

Without the preempt disabling, there is no limit to the
number of cpus that might get to:

use_cpu_writer_for_mount(cpu_writer, mnt);
if (cpu_writer->count > 0) {
cpu_writer->count--;
} else {
atomic_dec(&mnt->__mnt_writers);
}
spin_unlock(&cpu_writer->lock);
---->HERE
if (must_check_underflow)
handle_write_count_underflow(mnt);

because they get preempted once the spinlock is unlocked.

So, there's no limit on how many times __mnt_writers may
be decremented. (I know the limit is still the number of
tasks on the system, but that's a heck of a lot higher
than the number of cpus.) Doing the simple check in this
patch before the decrement and under a lock removes the
possibility that this can happen.

Since there are only NR_CPUS mnt_writer[]s, we can only
have NR_CPUS lock holders in the critical section at a
time, __mnt_writers can only underflow by
MNT_WRITER_UNDERFLOW_LIMIT+NR_CPUS.

Signed-off-by: Dave Hansen <haveblue@us.ibm.com>
---

linux-2.6.git-dave/fs/namespace.c | 28 +++++++++++++++++-----------
1 file changed, 17 insertions(+), 11 deletions(-)

diff -puN fs/namespace.c~change-underflow-protection-logic fs/namespace.c
--- linux-2.6.git/fs/namespace.c~change-underflow-protection-logic 2008-01-10 10:36:37.000000000 -0800
+++ linux-2.6.git-dave/fs/namespace.c 2008-01-10 10:36:37.000000000 -0800
@@ -267,15 +267,30 @@ void mnt_drop_write(struct vfsmount *mnt
int must_check_underflow = 0;
struct mnt_writer *cpu_writer;

- cpu_writer = &get_cpu_var(mnt_writers);
+retry:
+ cpu_writer = &__get_cpu_var(mnt_writers);
spin_lock(&cpu_writer->lock);

use_cpu_writer_for_mount(cpu_writer, mnt);
if (cpu_writer->count > 0) {
cpu_writer->count--;
} else {
- must_check_underflow = 1;
+ /* Without this check, it is theoretically
+ * possible to underflow __mnt_writers.
+ * An unlimited number of processes could
+ * all do this decrement, unlock, and then
+ * stall before the underflow handling.
+ * Doing this check limits the underflow
+ * to the number of cpu_writer->lock
+ * holders (NR_CPUS).
+ */
+ if (atomic_read(&mnt->__mnt_writers) <
+ MNT_WRITER_UNDERFLOW_LIMIT) {
+ spin_unlock(&cpu_writer->lock);
+ goto retry;
+ }
atomic_dec(&mnt->__mnt_writers);
+ must_check_underflow = 1;
}

spin_unlock(&cpu_writer->lock);
@@ -286,15 +301,6 @@ void mnt_drop_write(struct vfsmount *mnt
*/
if (must_check_underflow)
handle_write_count_underflow(mnt);
- /*
- * This could be done right after the spinlock
- * is taken because the spinlock keeps us on
- * the cpu, and disables preemption. However,
- * putting it here bounds the amount that
- * __mnt_writers can underflow. Without it,
- * we could theoretically wrap __mnt_writers.
- */
- put_cpu_var(mnt_writers);
}
EXPORT_SYMBOL_GPL(mnt_drop_write);

_