* Re: [PATCH] mm, arch: Complete pagefault_disable abstraction
@ 2011-10-05 13:12 Peter Zijlstra
0 siblings, 0 replies; 4+ messages in thread
From: Peter Zijlstra @ 2011-10-05 13:12 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm@kvack.org, linux-arch, linux-kernel, Thomas Gleixner
For those with a strong stomach, here's what we do with kmap_atomic:
---
Subject: mm, rt: kmap_atomic scheduling
From: Peter Zijlstra <peterz@infradead.org>
Date: Thu, 28 Jul 2011 10:43:51 +0200
In fact, with migrate_disable() existing one could play games with
kmap_atomic. You could save/restore the kmap_atomic slots on context
switch (if there are any in use of course), this should be esp easy now
that we have a kmap_atomic stack.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
[dvhart@linux.intel.com: build fix]
Link: http://lkml.kernel.org/r/1311842631.5890.208.camel@twins
---
arch/x86/kernel/process_32.c | 36 ++++++++++++++++++++++++++++++++++++
include/linux/sched.h | 5 +++++
mm/memory.c | 2 ++
3 files changed, 43 insertions(+)
Index: linux-2.6/arch/x86/kernel/process_32.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/process_32.c
+++ linux-2.6/arch/x86/kernel/process_32.c
@@ -38,6 +38,7 @@
#include <linux/uaccess.h>
#include <linux/io.h>
#include <linux/kdebug.h>
+#include <linux/highmem.h>
#include <asm/pgtable.h>
#include <asm/system.h>
@@ -346,6 +347,41 @@ __switch_to(struct task_struct *prev_p,
task_thread_info(next_p)->flags & _TIF_WORK_CTXSW_NEXT))
__switch_to_xtra(prev_p, next_p, tss);
+#if defined CONFIG_PREEMPT_RT_FULL && defined CONFIG_HIGHMEM
+ /*
+ * Save @prev's kmap_atomic stack
+ */
+ prev_p->kmap_idx = __this_cpu_read(__kmap_atomic_idx);
+ if (unlikely(prev_p->kmap_idx)) {
+ int i;
+
+ for (i = 0; i < prev_p->kmap_idx; i++) {
+ int idx = i + KM_TYPE_NR * smp_processor_id();
+
+ pte_t *ptep = kmap_pte - idx;
+ prev_p->kmap_pte[i] = *ptep;
+ kpte_clear_flush(ptep, __fix_to_virt(FIX_KMAP_BEGIN + idx));
+ }
+
+ __this_cpu_write(__kmap_atomic_idx, 0);
+ }
+
+ /*
+ * Restore @next_p's kmap_atomic stack
+ */
+ if (unlikely(next_p->kmap_idx)) {
+ int i;
+
+ __this_cpu_write(__kmap_atomic_idx, next_p->kmap_idx);
+
+ for (i = 0; i < next_p->kmap_idx; i++) {
+ int idx = i + KM_TYPE_NR * smp_processor_id();
+
+ set_pte(kmap_pte - idx, next_p->kmap_pte[i]);
+ }
+ }
+#endif
+
/* If we're going to preload the fpu context, make sure clts
is run while we're batching the cpu state updates. */
if (preload_fpu)
Index: linux-2.6/include/linux/sched.h
===================================================================
--- linux-2.6.orig/include/linux/sched.h
+++ linux-2.6/include/linux/sched.h
@@ -63,6 +63,7 @@ struct sched_param {
#include <linux/nodemask.h>
#include <linux/mm_types.h>
+#include <asm/kmap_types.h>
#include <asm/system.h>
#include <asm/page.h>
#include <asm/ptrace.h>
@@ -1594,6 +1595,10 @@ struct task_struct {
struct rcu_head put_rcu;
int softirq_nestcnt;
#endif
+#if defined CONFIG_PREEMPT_RT_FULL && defined CONFIG_HIGHMEM
+ int kmap_idx;
+ pte_t kmap_pte[KM_TYPE_NR];
+#endif
};
#ifdef CONFIG_PREEMPT_RT_FULL
Index: linux-2.6/mm/memory.c
===================================================================
--- linux-2.6.orig/mm/memory.c
+++ linux-2.6/mm/memory.c
@@ -3431,6 +3431,7 @@ unlock:
#ifdef CONFIG_PREEMPT_RT_FULL
void pagefault_disable(void)
{
+ migrate_disable();
current->pagefault_disabled++;
/*
* make sure to have issued the store before a pagefault
@@ -3448,6 +3449,7 @@ void pagefault_enable(void)
*/
barrier();
current->pagefault_disabled--;
+ migrate_enable();
}
EXPORT_SYMBOL_GPL(pagefault_enable);
#endif
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH] mm, arch: Complete pagefault_disable abstraction
@ 2011-10-05 13:10 Peter Zijlstra
0 siblings, 0 replies; 4+ messages in thread
From: Peter Zijlstra @ 2011-10-05 13:10 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm@kvack.org, linux-arch, linux-kernel, Thomas Gleixner
For reference, here's the -rt patch that goes on top:
---
Subject: rt: Preemptable pagefault_disable()
From: Peter Zijlstra <a.p.zijlstra@chello.nl>
Date: Wed Oct 05 14:20:57 CEST 2011
Implement a preemptable pagefault_disable() by keeping a per-task
pagefault_disabled counter.
This allows disabling of the pagefault handler (and thus avoiding the
recursive fault/mmap_sem issues) without disabling preemption.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
include/linux/sched.h | 9 ++++++++-
include/linux/uaccess.h | 5 +++++
kernel/fork.c | 3 +++
mm/memory.c | 24 ++++++++++++++++++++++++
4 files changed, 40 insertions(+), 1 deletion(-)
Index: linux-2.6/include/linux/uaccess.h
===================================================================
--- linux-2.6.orig/include/linux/uaccess.h
+++ linux-2.6/include/linux/uaccess.h
@@ -4,6 +4,7 @@
#include <linux/preempt.h>
#include <asm/uaccess.h>
+#ifndef CONFIG_PREEMPT_RT_FULL
/*
* These routines enable/disable the pagefault handler in that
* it will not take any locks and go straight to the fixup table.
@@ -37,6 +38,10 @@ static inline void pagefault_enable(void
barrier();
preempt_check_resched();
}
+#else
+extern void pagefault_disable(void);
+extern void pagefault_enable(void);
+#endif
#ifndef ARCH_HAS_NOCACHE_UACCESS
Index: linux-2.6/kernel/fork.c
===================================================================
--- linux-2.6.orig/kernel/fork.c
+++ linux-2.6/kernel/fork.c
@@ -1200,6 +1200,9 @@ static struct task_struct *copy_process(
p->hardirq_context = 0;
p->softirq_context = 0;
#endif
+#ifdef CONFIG_PREEMPT_RT_FULL
+ p->pagefault_disabled = 0;
+#endif
#ifdef CONFIG_LOCKDEP
p->lockdep_depth = 0; /* no locks held yet */
p->curr_chain_key = 0;
Index: linux-2.6/mm/memory.c
===================================================================
--- linux-2.6.orig/mm/memory.c
+++ linux-2.6/mm/memory.c
@@ -3436,6 +3436,30 @@ int handle_pte_fault(struct mm_struct *m
return 0;
}
+#ifdef CONFIG_PREEMPT_RT_FULL
+void pagefault_disable(void)
+{
+ current->pagefault_disabled++;
+ /*
+ * make sure to have issued the store before a pagefault
+ * can hit.
+ */
+ barrier();
+}
+EXPORT_SYMBOL_GPL(pagefault_disable);
+
+void pagefault_enable(void)
+{
+ /*
+ * make sure to issue those last loads/stores before enabling
+ * the pagefault handler again.
+ */
+ barrier();
+ current->pagefault_disabled--;
+}
+EXPORT_SYMBOL_GPL(pagefault_enable);
+#endif
+
/*
* By the time we get here, we already hold the mm semaphore
*/
Index: linux-2.6/include/linux/sched.h
===================================================================
--- linux-2.6.orig/include/linux/sched.h
+++ linux-2.6/include/linux/sched.h
@@ -1434,6 +1434,9 @@ struct task_struct {
/* mutex deadlock detection */
struct mutex_waiter *blocked_on;
#endif
+#ifdef CONFIG_PREEMPT_RT_FULL
+ int pagefault_disabled;
+#endif
#ifdef CONFIG_TRACE_IRQFLAGS
unsigned int irq_events;
unsigned long hardirq_enable_ip;
@@ -1578,7 +1581,11 @@ struct task_struct {
static inline bool pagefault_disabled(void)
{
- return in_atomic();
+ return in_atomic()
+#ifdef CONFIG_PREEMPT_RT_FULL
+ || current->pagefault_disabled
+#endif
+ ;
}
/*
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 4+ messages in thread* [PATCH] mm, arch: Complete pagefault_disable abstraction
@ 2011-10-05 13:09 Peter Zijlstra
2011-10-05 21:39 ` Andrew Morton
0 siblings, 1 reply; 4+ messages in thread
From: Peter Zijlstra @ 2011-10-05 13:09 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm@kvack.org, linux-arch, linux-kernel, Thomas Gleixner
Subject: mm, arch: Complete pagefault_disable abstraction
From: Peter Zijlstra <a.p.zijlstra@chello.nl>
Date: Wed Oct 05 14:22:37 CEST 2011
Currently we already have pagefault_{disable,enable}() but they're
nothing more than a glorified preempt_{disable,enable}().
For -rt we want to make pagefault_disable() regions preemptible and
for that we need to complete the abstraction, replace the current
in_atomic() tests that are relevant to pagefault_disable() usage with
pagefault_disabled().
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
arch/alpha/mm/fault.c | 2 +-
arch/arm/mm/fault.c | 2 +-
arch/avr32/mm/fault.c | 2 +-
arch/cris/mm/fault.c | 2 +-
arch/frv/mm/fault.c | 2 +-
arch/ia64/mm/fault.c | 2 +-
arch/m32r/mm/fault.c | 2 +-
arch/m68k/mm/fault.c | 2 +-
arch/microblaze/mm/fault.c | 2 +-
arch/mips/mm/fault.c | 2 +-
arch/mn10300/mm/fault.c | 2 +-
arch/parisc/mm/fault.c | 2 +-
arch/powerpc/mm/fault.c | 2 +-
arch/s390/mm/fault.c | 6 ++++--
arch/score/mm/fault.c | 2 +-
arch/sh/mm/fault_32.c | 2 +-
arch/sparc/mm/fault_32.c | 4 ++--
arch/sparc/mm/fault_64.c | 2 +-
arch/tile/mm/fault.c | 2 +-
arch/um/kernel/trap.c | 2 +-
arch/unicore32/mm/fault.c | 2 +-
arch/x86/mm/fault.c | 2 +-
arch/xtensa/mm/fault.c | 2 +-
include/linux/sched.h | 6 ++++++
mm/filemap.c | 2 +-
25 files changed, 34 insertions(+), 26 deletions(-)
Index: linux-2.6/arch/alpha/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/alpha/mm/fault.c
+++ linux-2.6/arch/alpha/mm/fault.c
@@ -107,7 +107,7 @@ do_page_fault(unsigned long address, uns
/* If we're in an interrupt context, or have no user context,
we must not take the fault. */
- if (!mm || in_atomic())
+ if (!mm || pagefault_disabled())
goto no_context;
#ifdef CONFIG_ALPHA_LARGE_VMALLOC
Index: linux-2.6/arch/arm/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/arm/mm/fault.c
+++ linux-2.6/arch/arm/mm/fault.c
@@ -293,7 +293,7 @@ do_page_fault(unsigned long addr, unsign
* If we're in an interrupt or have no user
* context, we must not take the fault..
*/
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto no_context;
/*
Index: linux-2.6/arch/avr32/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/avr32/mm/fault.c
+++ linux-2.6/arch/avr32/mm/fault.c
@@ -81,7 +81,7 @@ asmlinkage void do_page_fault(unsigned l
* If we're in an interrupt or have no user context, we must
* not take the fault...
*/
- if (in_atomic() || !mm || regs->sr & SYSREG_BIT(GM))
+ if (!mm || regs->sr & SYSREG_BIT(GM) || pagefault_disabled())
goto no_context;
local_irq_enable();
Index: linux-2.6/arch/cris/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/cris/mm/fault.c
+++ linux-2.6/arch/cris/mm/fault.c
@@ -111,7 +111,7 @@ do_page_fault(unsigned long address, str
* user context, we must not take the fault.
*/
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto no_context;
down_read(&mm->mmap_sem);
Index: linux-2.6/arch/frv/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/frv/mm/fault.c
+++ linux-2.6/arch/frv/mm/fault.c
@@ -79,7 +79,7 @@ asmlinkage void do_page_fault(int datamm
* If we're in an interrupt or have no user
* context, we must not take the fault..
*/
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto no_context;
down_read(&mm->mmap_sem);
Index: linux-2.6/arch/ia64/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/ia64/mm/fault.c
+++ linux-2.6/arch/ia64/mm/fault.c
@@ -89,7 +89,7 @@ ia64_do_page_fault (unsigned long addres
/*
* If we're in an interrupt or have no user context, we must not take the fault..
*/
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto no_context;
#ifdef CONFIG_VIRTUAL_MEM_MAP
Index: linux-2.6/arch/m32r/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/m32r/mm/fault.c
+++ linux-2.6/arch/m32r/mm/fault.c
@@ -115,7 +115,7 @@ asmlinkage void do_page_fault(struct pt_
* If we're in an interrupt or have no user context or are running in an
* atomic region then we must not take the fault..
*/
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto bad_area_nosemaphore;
/* When running in the kernel we expect faults to occur only to
Index: linux-2.6/arch/m68k/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/m68k/mm/fault.c
+++ linux-2.6/arch/m68k/mm/fault.c
@@ -85,7 +85,7 @@ int do_page_fault(struct pt_regs *regs,
* If we're in an interrupt or have no user
* context, we must not take the fault..
*/
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto no_context;
down_read(&mm->mmap_sem);
Index: linux-2.6/arch/microblaze/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/microblaze/mm/fault.c
+++ linux-2.6/arch/microblaze/mm/fault.c
@@ -107,7 +107,7 @@ void do_page_fault(struct pt_regs *regs,
if ((error_code & 0x13) == 0x13 || (error_code & 0x11) == 0x11)
is_write = 0;
- if (unlikely(in_atomic() || !mm)) {
+ if (unlikely(!mm || pagefault_disabled())) {
if (kernel_mode(regs))
goto bad_area_nosemaphore;
Index: linux-2.6/arch/mips/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/mips/mm/fault.c
+++ linux-2.6/arch/mips/mm/fault.c
@@ -88,7 +88,7 @@ asmlinkage void __kprobes do_page_fault(
* If we're in an interrupt or have no user
* context, we must not take the fault..
*/
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto bad_area_nosemaphore;
down_read(&mm->mmap_sem);
Index: linux-2.6/arch/mn10300/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/mn10300/mm/fault.c
+++ linux-2.6/arch/mn10300/mm/fault.c
@@ -168,7 +168,7 @@ asmlinkage void do_page_fault(struct pt_
* If we're in an interrupt or have no user
* context, we must not take the fault..
*/
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto no_context;
down_read(&mm->mmap_sem);
Index: linux-2.6/arch/parisc/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/parisc/mm/fault.c
+++ linux-2.6/arch/parisc/mm/fault.c
@@ -176,7 +176,7 @@ void do_page_fault(struct pt_regs *regs,
unsigned long acc_type;
int fault;
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto no_context;
down_read(&mm->mmap_sem);
Index: linux-2.6/arch/powerpc/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/powerpc/mm/fault.c
+++ linux-2.6/arch/powerpc/mm/fault.c
@@ -162,7 +162,7 @@ int __kprobes do_page_fault(struct pt_re
}
#endif
- if (in_atomic() || mm == NULL) {
+ if (!mm || pagefault_disabled()) {
if (!user_mode(regs))
return SIGSEGV;
/* in_atomic() in user mode is really bad,
Index: linux-2.6/arch/s390/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/s390/mm/fault.c
+++ linux-2.6/arch/s390/mm/fault.c
@@ -295,7 +295,8 @@ static inline int do_exception(struct pt
* user context.
*/
fault = VM_FAULT_BADCONTEXT;
- if (unlikely(!user_space_fault(trans_exc_code) || in_atomic() || !mm))
+ if (unlikely(!user_space_fault(trans_exc_code) ||
+ !mm || pagefault_disabled()))
goto out;
address = trans_exc_code & __FAIL_ADDR_MASK;
@@ -426,7 +427,8 @@ void __kprobes do_asce_exception(struct
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma;
- if (unlikely(!user_space_fault(trans_exc_code) || in_atomic() || !mm))
+ if (unlikely(!user_space_fault(trans_exc_code) ||
+ !mm || pagefault_disabled()))
goto no_context;
down_read(&mm->mmap_sem);
Index: linux-2.6/arch/score/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/score/mm/fault.c
+++ linux-2.6/arch/score/mm/fault.c
@@ -72,7 +72,7 @@ asmlinkage void do_page_fault(struct pt_
* If we're in an interrupt or have no user
* context, we must not take the fault..
*/
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto bad_area_nosemaphore;
down_read(&mm->mmap_sem);
Index: linux-2.6/arch/sh/mm/fault_32.c
===================================================================
--- linux-2.6.orig/arch/sh/mm/fault_32.c
+++ linux-2.6/arch/sh/mm/fault_32.c
@@ -166,7 +166,7 @@ asmlinkage void __kprobes do_page_fault(
* If we're in an interrupt, have no user context or are running
* in an atomic region then we must not take the fault:
*/
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto no_context;
down_read(&mm->mmap_sem);
Index: linux-2.6/arch/sparc/mm/fault_32.c
===================================================================
--- linux-2.6.orig/arch/sparc/mm/fault_32.c
+++ linux-2.6/arch/sparc/mm/fault_32.c
@@ -248,8 +248,8 @@ asmlinkage void do_sparc_fault(struct pt
* If we're in an interrupt or have no user
* context, we must not take the fault..
*/
- if (in_atomic() || !mm)
- goto no_context;
+ if (!mm || pagefault_disabled())
+ goto no_context;
perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address);
Index: linux-2.6/arch/sparc/mm/fault_64.c
===================================================================
--- linux-2.6.orig/arch/sparc/mm/fault_64.c
+++ linux-2.6/arch/sparc/mm/fault_64.c
@@ -322,7 +322,7 @@ asmlinkage void __kprobes do_sparc64_fau
* If we're in an interrupt or have no user
* context, we must not take the fault..
*/
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto intr_or_no_mm;
perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address);
Index: linux-2.6/arch/tile/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/tile/mm/fault.c
+++ linux-2.6/arch/tile/mm/fault.c
@@ -346,7 +346,7 @@ static int handle_page_fault(struct pt_r
* If we're in an interrupt, have no user context or are running in an
* atomic region then we must not take the fault.
*/
- if (in_atomic() || !mm) {
+ if (!mm || pagefault_disabled()) {
vma = NULL; /* happy compiler */
goto bad_area_nosemaphore;
}
Index: linux-2.6/arch/um/kernel/trap.c
===================================================================
--- linux-2.6.orig/arch/um/kernel/trap.c
+++ linux-2.6/arch/um/kernel/trap.c
@@ -37,7 +37,7 @@ int handle_page_fault(unsigned long addr
* If the fault was during atomic operation, don't take the fault, just
* fail.
*/
- if (in_atomic())
+ if (!mm || pagefault_disabled())
goto out_nosemaphore;
down_read(&mm->mmap_sem);
Index: linux-2.6/arch/unicore32/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/unicore32/mm/fault.c
+++ linux-2.6/arch/unicore32/mm/fault.c
@@ -225,7 +225,7 @@ static int do_pf(unsigned long addr, uns
* If we're in an interrupt or have no user
* context, we must not take the fault..
*/
- if (in_atomic() || !mm)
+ if (!mm || pagefault_disabled())
goto no_context;
/*
Index: linux-2.6/arch/x86/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/fault.c
+++ linux-2.6/arch/x86/mm/fault.c
@@ -1084,7 +1084,7 @@ do_page_fault(struct pt_regs *regs, unsi
* If we're in an interrupt, have no user context or are running
* in an atomic region then we must not take the fault:
*/
- if (unlikely(in_atomic() || !mm)) {
+ if (unlikely(!mm || pagefault_disabled())) {
bad_area_nosemaphore(regs, error_code, address);
return;
}
Index: linux-2.6/arch/xtensa/mm/fault.c
===================================================================
--- linux-2.6.orig/arch/xtensa/mm/fault.c
+++ linux-2.6/arch/xtensa/mm/fault.c
@@ -57,7 +57,7 @@ void do_page_fault(struct pt_regs *regs)
/* If we're in an interrupt or have no user
* context, we must not take the fault..
*/
- if (in_atomic() || !mm) {
+ if (!mm || pagefault_disabled()) {
bad_page_fault(regs, address, SIGSEGV);
return;
}
Index: linux-2.6/include/linux/sched.h
===================================================================
--- linux-2.6.orig/include/linux/sched.h
+++ linux-2.6/include/linux/sched.h
@@ -91,6 +91,7 @@ struct sched_param {
#include <linux/latencytop.h>
#include <linux/cred.h>
#include <linux/llist.h>
+#include <linux/hardirq.h>
#include <asm/processor.h>
@@ -1575,6 +1576,11 @@ struct task_struct {
/* Future-safe accessor for struct task_struct's cpus_allowed. */
#define tsk_cpus_allowed(tsk) (&(tsk)->cpus_allowed)
+static inline bool pagefault_disabled(void)
+{
+ return in_atomic();
+}
+
/*
* Priority of a process goes from 0..MAX_PRIO-1, valid RT
* priority is 0..MAX_RT_PRIO-1, and SCHED_NORMAL/SCHED_BATCH
Index: linux-2.6/mm/filemap.c
===================================================================
--- linux-2.6.orig/mm/filemap.c
+++ linux-2.6/mm/filemap.c
@@ -2061,7 +2061,7 @@ size_t iov_iter_copy_from_user_atomic(st
char *kaddr;
size_t copied;
- BUG_ON(!in_atomic());
+ BUG_ON(!pagefault_disabled());
kaddr = kmap_atomic(page, KM_USER0);
if (likely(i->nr_segs == 1)) {
int left;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH] mm, arch: Complete pagefault_disable abstraction
2011-10-05 13:09 Peter Zijlstra
@ 2011-10-05 21:39 ` Andrew Morton
0 siblings, 0 replies; 4+ messages in thread
From: Andrew Morton @ 2011-10-05 21:39 UTC (permalink / raw)
To: Peter Zijlstra
Cc: linux-mm@kvack.org, linux-arch, linux-kernel, Thomas Gleixner
On Wed, 05 Oct 2011 15:09:29 +0200
Peter Zijlstra <peterz@infradead.org> wrote:
> Currently we already have pagefault_{disable,enable}() but they're
> nothing more than a glorified preempt_{disable,enable}().
That's not very accurate or useful. Unlike preempt_disable(),
pagefault_disable() will raise the preempt count when CONFIG_PREEMPT=n.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2011-10-05 21:39 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-10-05 13:12 [PATCH] mm, arch: Complete pagefault_disable abstraction Peter Zijlstra
-- strict thread matches above, loose matches on Subject: below --
2011-10-05 13:10 Peter Zijlstra
2011-10-05 13:09 Peter Zijlstra
2011-10-05 21:39 ` Andrew Morton
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).