* [PATCH 08/10] arm64: make cpu number a percpu variable
From: Mark Rutland @ 2016-10-19 19:10 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476904234-9511-1-git-send-email-mark.rutland@arm.com>
In the absence of CONFIG_THREAD_INFO_IN_TASK, core code maintains
thread_info::cpu, and low-level architecture code can access this to
build raw_smp_processor_id(). With CONFIG_THREAD_INFO_IN_TASK, core code
maintains task_struct::cpu, which for reasons of hte header soup is not
accessible to low-level arch code.
Instead, we can maintain a percpu variable containing the cpu number.
Non-preemptible code will receive the current CPU's number, while
preemptible code may race (as was previosuly the case), and see a stale
value. Thus, there shouldn't be a change in observable behaviour.
For both the old and new implementation of raw_smp_processor_id(), we
read a syreg into a GPR, add an offset, and load the result. As the
offset is now large, it probably won't be folded into the load, but
otherwise the assembly shouldn't change much.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/smp.h | 6 +++++-
arch/arm64/kernel/smp.c | 5 +++++
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h
index 0226447..f77ac0d 100644
--- a/arch/arm64/include/asm/smp.h
+++ b/arch/arm64/include/asm/smp.h
@@ -29,11 +29,15 @@
#ifndef __ASSEMBLY__
+#include <asm/percpu.h>
+
#include <linux/threads.h>
#include <linux/cpumask.h>
#include <linux/thread_info.h>
-#define raw_smp_processor_id() (current_thread_info()->cpu)
+DECLARE_PER_CPU_READ_MOSTLY(int, cpu_number);
+
+#define raw_smp_processor_id() (*this_cpu_ptr(&cpu_number))
struct seq_file;
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 7af46bf..2679722 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -58,6 +58,9 @@
#define CREATE_TRACE_POINTS
#include <trace/events/ipi.h>
+DEFINE_PER_CPU_READ_MOSTLY(int, cpu_number);
+EXPORT_PER_CPU_SYMBOL(cpu_number);
+
/*
* as from 2.5, kernels no longer have an init_tasks structure
* so we need some other way of telling a new secondary core
@@ -718,6 +721,8 @@ void __init smp_prepare_cpus(unsigned int max_cpus)
*/
for_each_possible_cpu(cpu) {
+ per_cpu(cpu_number, cpu) = cpu;
+
if (cpu == smp_processor_id())
continue;
--
1.9.1
^ permalink raw reply related
* [PATCH 07/10] arm64: smp: prepare for smp_processor_id() rework
From: Mark Rutland @ 2016-10-19 19:10 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476904234-9511-1-git-send-email-mark.rutland@arm.com>
Subsequent patches will make smp_processor_id() use a percpu variable.
This will make smp_processor_id() dependent on the percpu offset, and
thus we cannot use smp_processor_id() to figure out what to initialise
the offset to.
Prepare for this by initialising the percpu offset based on
current::cpu, which will work regardless of how smp_processor_id() is
implemented. Also, make this relationship obvious by placing this code
together at the start of secondary_start_kernel().
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/kernel/smp.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index d3f151c..7af46bf 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -208,7 +208,10 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle)
asmlinkage void secondary_start_kernel(void)
{
struct mm_struct *mm = &init_mm;
- unsigned int cpu = smp_processor_id();
+ unsigned int cpu;
+
+ cpu = task_cpu(current);
+ set_my_cpu_offset(per_cpu_offset(cpu));
/*
* All kernel threads share the same mm context; grab a
@@ -217,8 +220,6 @@ asmlinkage void secondary_start_kernel(void)
atomic_inc(&mm->mm_count);
current->active_mm = mm;
- set_my_cpu_offset(per_cpu_offset(smp_processor_id()));
-
/*
* TTBR0 is only used for the identity mapping at this stage. Make it
* point to zero page to avoid speculatively fetching new entries.
--
1.9.1
^ permalink raw reply related
* [PATCH 06/10] arm64: move sp_el0 and tpidr_el1 into cpu_suspend_ctx
From: Mark Rutland @ 2016-10-19 19:10 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476904234-9511-1-git-send-email-mark.rutland@arm.com>
When returning from idle, we rely on the fact that thread_info lives at
the end of the kernel stack, and restore this by masking the saved stack
pointer. Subsequent patches will sever the relationship between the
stack and thread_info, and to cater for this we must save/restore sp_el0
explicitly, storing it in cpu_suspend_ctx.
As cpu_suspend_ctx must be doubleword aligned, this leaves us with an
extra slot in cpu_suspend_ctx. We can use this to save/restore tpidr_el1
in the same way, which simplifies the code, avoiding pointer chasing on
the restore path (as we no longer need to load thread_info::cpu followed
by the relevant slot in __per_cpu_offset based on this).
This patch stashes both registers in cpu_suspend_ctx.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/suspend.h | 2 +-
arch/arm64/kernel/sleep.S | 3 ---
arch/arm64/kernel/suspend.c | 6 ------
arch/arm64/mm/proc.S | 6 ++++++
4 files changed, 7 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/include/asm/suspend.h b/arch/arm64/include/asm/suspend.h
index b8a313f..de5600f 100644
--- a/arch/arm64/include/asm/suspend.h
+++ b/arch/arm64/include/asm/suspend.h
@@ -1,7 +1,7 @@
#ifndef __ASM_SUSPEND_H
#define __ASM_SUSPEND_H
-#define NR_CTX_REGS 10
+#define NR_CTX_REGS 12
#define NR_CALLEE_SAVED_REGS 12
/*
diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S
index b8799e7..5062fd9 100644
--- a/arch/arm64/kernel/sleep.S
+++ b/arch/arm64/kernel/sleep.S
@@ -125,9 +125,6 @@ ENTRY(_cpu_resume)
/* load sp from context */
ldr x2, [x0, #CPU_CTX_SP]
mov sp, x2
- /* save thread_info */
- and x2, x2, #~(THREAD_SIZE - 1)
- msr sp_el0, x2
/*
* cpu_do_resume expects x0 to contain context address pointer
*/
diff --git a/arch/arm64/kernel/suspend.c b/arch/arm64/kernel/suspend.c
index ad73414..7db5620 100644
--- a/arch/arm64/kernel/suspend.c
+++ b/arch/arm64/kernel/suspend.c
@@ -44,12 +44,6 @@ void notrace __cpu_suspend_exit(void)
cpu_uninstall_idmap();
/*
- * Restore per-cpu offset before any kernel
- * subsystem relying on it has a chance to run.
- */
- set_my_cpu_offset(per_cpu_offset(cpu));
-
- /*
* Restore HW breakpoint registers to sane values
* before debug exceptions are possibly reenabled
* through local_dbg_restore.
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 352c73b..6a853a8 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -70,11 +70,14 @@ ENTRY(cpu_do_suspend)
mrs x8, mdscr_el1
mrs x9, oslsr_el1
mrs x10, sctlr_el1
+ mrs x11, tpidr_el1
+ mrs x12, sp_el0
stp x2, x3, [x0]
stp x4, xzr, [x0, #16]
stp x5, x6, [x0, #32]
stp x7, x8, [x0, #48]
stp x9, x10, [x0, #64]
+ stp x11, x12, [x0, #80]
ret
ENDPROC(cpu_do_suspend)
@@ -90,6 +93,7 @@ ENTRY(cpu_do_resume)
ldp x6, x8, [x0, #32]
ldp x9, x10, [x0, #48]
ldp x11, x12, [x0, #64]
+ ldp x13, x14, [x0, #80]
msr tpidr_el0, x2
msr tpidrro_el0, x3
msr contextidr_el1, x4
@@ -112,6 +116,8 @@ ENTRY(cpu_do_resume)
msr mdscr_el1, x10
msr sctlr_el1, x12
+ msr tpidr_el1, x13
+ msr sp_el0, x14
/*
* Restore oslsr_el1 by writing oslar_el1
*/
--
1.9.1
^ permalink raw reply related
* [PATCH 05/10] arm64: prep stack walkers for THREAD_INFO_IN_TASK
From: Mark Rutland @ 2016-10-19 19:10 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476904234-9511-1-git-send-email-mark.rutland@arm.com>
When CONFIG_THREAD_INFO_IN_TASK is selected, task stacks may be freed
before a task is destroyed. To account for this, the stacks are
refcounted, and when manipulating the stack of another task, it is
necessary to get/put the stack to ensure it isn't freed and/or re-used
while we do so.
This patch reworks the arm64 stack walking code to account for this.
When CONFIG_THREAD_INFO_IN_TASK is not selected these perform no
refcounting, and this should only be a structural change that does not
affect behaviour.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: AKASHI Takahiro <takahiro.akashi@linaro.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/kernel/process.c | 20 ++++++++++++++------
arch/arm64/kernel/stacktrace.c | 5 +++++
2 files changed, 19 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 27b2f13..2f39036 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -340,27 +340,35 @@ struct task_struct *__switch_to(struct task_struct *prev,
unsigned long get_wchan(struct task_struct *p)
{
struct stackframe frame;
- unsigned long stack_page;
+ unsigned long stack_page, ret = 0;
int count = 0;
if (!p || p == current || p->state == TASK_RUNNING)
return 0;
+ stack_page = (unsigned long)try_get_task_stack(p);
+ if (!stack_page)
+ return 0;
+
frame.fp = thread_saved_fp(p);
frame.sp = thread_saved_sp(p);
frame.pc = thread_saved_pc(p);
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
frame.graph = p->curr_ret_stack;
#endif
- stack_page = (unsigned long)task_stack_page(p);
do {
if (frame.sp < stack_page ||
frame.sp >= stack_page + THREAD_SIZE ||
unwind_frame(p, &frame))
- return 0;
- if (!in_sched_functions(frame.pc))
- return frame.pc;
+ goto out;
+ if (!in_sched_functions(frame.pc)) {
+ ret = frame.pc;
+ goto out;
+ }
} while (count ++ < 16);
- return 0;
+
+out:
+ put_task_stack(p);
+ return ret;
}
unsigned long arch_align_stack(unsigned long sp)
diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c
index 5b80068..c77e5b6 100644
--- a/arch/arm64/kernel/stacktrace.c
+++ b/arch/arm64/kernel/stacktrace.c
@@ -182,6 +182,9 @@ void save_stack_trace_tsk(struct task_struct *tsk, struct stack_trace *trace)
struct stack_trace_data data;
struct stackframe frame;
+ if (!try_get_task_stack(tsk))
+ return;
+
data.trace = trace;
data.skip = trace->skip;
@@ -203,6 +206,8 @@ void save_stack_trace_tsk(struct task_struct *tsk, struct stack_trace *trace)
walk_stackframe(tsk, &frame, save_trace, &data);
if (trace->nr_entries < trace->max_entries)
trace->entries[trace->nr_entries++] = ULONG_MAX;
+
+ put_task_stack(tsk);
}
void save_stack_trace(struct stack_trace *trace)
--
1.9.1
^ permalink raw reply related
* [PATCH 04/10] arm64: traps: simplify die() and __die()
From: Mark Rutland @ 2016-10-19 19:10 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476904234-9511-1-git-send-email-mark.rutland@arm.com>
In arm64's die and __die routines we pass around a thread_info, and
subsequently use this to determine the relevant task_struct, and the end
of the thread's stack. Subsequent patches will decouple thread_info from
the stack, and this approach will no longer work.
To figure out the end of the stack, we can use the new generic
end_of_stack() helper. As we only call __die() from die(), and die()
always deals with the current task, we can remove the parameter and have
both acquire current directly, which also makes it clear that __die
can't be called for arbitrary tasks.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/kernel/traps.c | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 32d4bdb..1cfbe10 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -228,10 +228,9 @@ void show_stack(struct task_struct *tsk, unsigned long *sp)
#endif
#define S_SMP " SMP"
-static int __die(const char *str, int err, struct thread_info *thread,
- struct pt_regs *regs)
+static int __die(const char *str, int err, struct pt_regs *regs)
{
- struct task_struct *tsk = thread->task;
+ struct task_struct *tsk = current;
static int die_counter;
int ret;
@@ -246,7 +245,8 @@ static int __die(const char *str, int err, struct thread_info *thread,
print_modules();
__show_regs(regs);
pr_emerg("Process %.*s (pid: %d, stack limit = 0x%p)\n",
- TASK_COMM_LEN, tsk->comm, task_pid_nr(tsk), thread + 1);
+ TASK_COMM_LEN, tsk->comm, task_pid_nr(tsk),
+ end_of_stack(tsk));
if (!user_mode(regs)) {
dump_mem(KERN_EMERG, "Stack: ", regs->sp,
@@ -265,7 +265,6 @@ static int __die(const char *str, int err, struct thread_info *thread,
*/
void die(const char *str, struct pt_regs *regs, int err)
{
- struct thread_info *thread = current_thread_info();
int ret;
oops_enter();
@@ -273,9 +272,9 @@ void die(const char *str, struct pt_regs *regs, int err)
raw_spin_lock_irq(&die_lock);
console_verbose();
bust_spinlocks(1);
- ret = __die(str, err, thread, regs);
+ ret = __die(str, err, regs);
- if (regs && kexec_should_crash(thread->task))
+ if (regs && kexec_should_crash(current))
crash_kexec(regs);
bust_spinlocks(0);
--
1.9.1
^ permalink raw reply related
* [PATCH 03/10] arm64: factor out current_stack_pointer
From: Mark Rutland @ 2016-10-19 19:10 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476904234-9511-1-git-send-email-mark.rutland@arm.com>
We define current_stack_pointer in <asm/thread_info.h>, though other
files and header relying upon it do not have this necessary include, and
are thus fragile to changes in the header soup.
Subsequent patches will affect the header soup such that directly
including <asm/thread_info.h> may result in a circular header include in
some of these cases, so we can't simply include <asm/thread_info.h>.
Instead, factor current_thread_info into its own header, and have all
existing users include this explicitly.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/percpu.h | 2 ++
arch/arm64/include/asm/perf_event.h | 2 ++
arch/arm64/include/asm/stack_pointer.h | 9 +++++++++
arch/arm64/include/asm/thread_info.h | 6 +-----
arch/arm64/kernel/return_address.c | 1 +
arch/arm64/kernel/stacktrace.c | 1 +
arch/arm64/kernel/traps.c | 1 +
7 files changed, 17 insertions(+), 5 deletions(-)
create mode 100644 arch/arm64/include/asm/stack_pointer.h
diff --git a/arch/arm64/include/asm/percpu.h b/arch/arm64/include/asm/percpu.h
index 2fee2f5..2127a41 100644
--- a/arch/arm64/include/asm/percpu.h
+++ b/arch/arm64/include/asm/percpu.h
@@ -16,6 +16,8 @@
#ifndef __ASM_PERCPU_H
#define __ASM_PERCPU_H
+#include <asm/stack_pointer.h>
+
static inline void set_my_cpu_offset(unsigned long off)
{
asm volatile("msr tpidr_el1, %0" :: "r" (off) : "memory");
diff --git a/arch/arm64/include/asm/perf_event.h b/arch/arm64/include/asm/perf_event.h
index 2065f46..9eee2be 100644
--- a/arch/arm64/include/asm/perf_event.h
+++ b/arch/arm64/include/asm/perf_event.h
@@ -17,6 +17,8 @@
#ifndef __ASM_PERF_EVENT_H
#define __ASM_PERF_EVENT_H
+#include <asm/stack_pointer.h>
+
#define ARMV8_PMU_MAX_COUNTERS 32
#define ARMV8_PMU_COUNTER_MASK (ARMV8_PMU_MAX_COUNTERS - 1)
diff --git a/arch/arm64/include/asm/stack_pointer.h b/arch/arm64/include/asm/stack_pointer.h
new file mode 100644
index 0000000..ffcdf74
--- /dev/null
+++ b/arch/arm64/include/asm/stack_pointer.h
@@ -0,0 +1,9 @@
+#ifndef __ASM_STACK_POINTER_H
+#define __ASM_STACK_POINTER_H
+
+/*
+ * how to get the current stack pointer from C
+ */
+register unsigned long current_stack_pointer asm ("sp");
+
+#endif /* __ASM_STACK_POINTER_H */
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index 76a9559..3a4f85d 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -36,6 +36,7 @@
struct task_struct;
+#include <asm/stack_pointer.h>
#include <asm/types.h>
typedef unsigned long mm_segment_t;
@@ -62,11 +63,6 @@ struct thread_info {
#define init_stack (init_thread_union.stack)
/*
- * how to get the current stack pointer from C
- */
-register unsigned long current_stack_pointer asm ("sp");
-
-/*
* how to get the thread information struct from C
*/
static inline struct thread_info *current_thread_info(void) __attribute_const__;
diff --git a/arch/arm64/kernel/return_address.c b/arch/arm64/kernel/return_address.c
index 1718706..12a87f2 100644
--- a/arch/arm64/kernel/return_address.c
+++ b/arch/arm64/kernel/return_address.c
@@ -12,6 +12,7 @@
#include <linux/export.h>
#include <linux/ftrace.h>
+#include <asm/stack_pointer.h>
#include <asm/stacktrace.h>
struct return_address_data {
diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c
index c2efddf..5b80068 100644
--- a/arch/arm64/kernel/stacktrace.c
+++ b/arch/arm64/kernel/stacktrace.c
@@ -22,6 +22,7 @@
#include <linux/stacktrace.h>
#include <asm/irq.h>
+#include <asm/stack_pointer.h>
#include <asm/stacktrace.h>
/*
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 5ff020f..32d4bdb 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -38,6 +38,7 @@
#include <asm/esr.h>
#include <asm/insn.h>
#include <asm/traps.h>
+#include <asm/stack_pointer.h>
#include <asm/stacktrace.h>
#include <asm/exception.h>
#include <asm/system_misc.h>
--
1.9.1
^ permalink raw reply related
* [PATCH 02/10] arm64: asm-offsets: remove unused definitions
From: Mark Rutland @ 2016-10-19 19:10 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476904234-9511-1-git-send-email-mark.rutland@arm.com>
Subsequent patches will move the thread_info::{task,cpu} fields, and the
current TI_{TASK,CPU} offset definitions are not used anywhere.
This patch removes the redundant definitions.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/kernel/asm-offsets.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 4a2f0f0..d30b232 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -39,8 +39,6 @@ int main(void)
DEFINE(TI_FLAGS, offsetof(struct thread_info, flags));
DEFINE(TI_PREEMPT, offsetof(struct thread_info, preempt_count));
DEFINE(TI_ADDR_LIMIT, offsetof(struct thread_info, addr_limit));
- DEFINE(TI_TASK, offsetof(struct thread_info, task));
- DEFINE(TI_CPU, offsetof(struct thread_info, cpu));
BLANK();
DEFINE(THREAD_CPU_CONTEXT, offsetof(struct task_struct, thread.cpu_context));
BLANK();
--
1.9.1
^ permalink raw reply related
* [PATCH 01/10] arm64: thread_info remove stale items
From: Mark Rutland @ 2016-10-19 19:10 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476904234-9511-1-git-send-email-mark.rutland@arm.com>
We have a comment claiming __switch_to() cares about where cpu_context
is located relative to cpu_domain in thread_info. However arm64 has
never had a thread_info::cpu_domain field, and neither __switch_to nor
cpu_switch_to care where the cpu_context field is relative to others.
Additionally, the init_thread_info alias is never used anywhere in the
kernel, and will shortly become problematic when thread_info is moved
into task_struct.
This patch removes both.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
---
arch/arm64/include/asm/thread_info.h | 2 --
1 file changed, 2 deletions(-)
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index e9ea5a6..76a9559 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -42,7 +42,6 @@
/*
* low level task data that entry.S needs immediate access to.
- * __switch_to() assumes cpu_context follows immediately after cpu_domain.
*/
struct thread_info {
unsigned long flags; /* low level flags */
@@ -60,7 +59,6 @@ struct thread_info {
.addr_limit = KERNEL_DS, \
}
-#define init_thread_info (init_thread_union.thread_info)
#define init_stack (init_thread_union.stack)
/*
--
1.9.1
^ permalink raw reply related
* [PATCH 00/10] arm64: move thread_info off of the task stack
From: Mark Rutland @ 2016-10-19 19:10 UTC (permalink / raw)
To: linux-arm-kernel
Hi all,
Building atop of Andy's work on x86 and generic code, these patches move
arm64's thread_info off of the stack and into task_struct. This protects
thread_info from corruption in the face of stack overflow, and serves as
a step towards fully robust stack overflow handling, which will be
addressed by subsequent patches.
These patches are based atop of a preparatory series [1] (itself based
on v4.9-rc1) that's also necessary for s390. I've placed those patches
in a branch [2] on my kernel.org repo, along with this series [3]. I'm
hoping that the prep work will be able to become a stable branch/tag
soon.
I've given the series some light testing on a couple of SMP arm64
platforms, but this has yet to see a thorough beating; please do try to
make this fall over!
Since RFC [4]:
* Rely on prior patches to make thread_info arch-specific
* Make smp_processor_id() use a per-cpu variable
* Split out current_stack_pointer
* Make SMP actually work
[1] http://lkml.kernel.org/r/1476901693-8492-1-git-send-email-mark.rutland at arm.com
[2] https://git.kernel.org/cgit/linux/kernel/git/mark/linux.git/log/?h=core/ti-stack-split
[3] https://git.kernel.org/cgit/linux/kernel/git/mark/linux.git/log/?h=arm64/ti-stack-split
[4] http://lkml.kernel.org/r/1473947349-14521-1-git-send-email-mark.rutland at arm.com
Thanks,
Mark.
Mark Rutland (10):
arm64: thread_info remove stale items
arm64: asm-offsets: remove unused definitions
arm64: factor out current_stack_pointer
arm64: traps: simplify die() and __die()
arm64: prep stack walkers for THREAD_INFO_IN_TASK
arm64: move sp_el0 and tpidr_el1 into cpu_suspend_ctx
arm64: smp: prepare for smp_processor_id() rework
arm64: make cpu number a percpu variable
arm64: assembler: introduce ldr_this_cpu
arm64: split thread_info from task stack
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/Kbuild | 1 -
arch/arm64/include/asm/assembler.h | 19 +++++++++++++++----
arch/arm64/include/asm/current.h | 22 ++++++++++++++++++++++
arch/arm64/include/asm/percpu.h | 2 ++
arch/arm64/include/asm/perf_event.h | 2 ++
arch/arm64/include/asm/smp.h | 7 ++++++-
arch/arm64/include/asm/stack_pointer.h | 9 +++++++++
arch/arm64/include/asm/suspend.h | 2 +-
arch/arm64/include/asm/thread_info.h | 32 +-------------------------------
arch/arm64/kernel/asm-offsets.c | 3 +--
arch/arm64/kernel/entry.S | 6 +++---
arch/arm64/kernel/head.S | 11 ++++++-----
arch/arm64/kernel/process.c | 33 +++++++++++++++++++++++++++------
arch/arm64/kernel/return_address.c | 1 +
arch/arm64/kernel/sleep.S | 3 ---
arch/arm64/kernel/smp.c | 14 +++++++++++---
arch/arm64/kernel/stacktrace.c | 6 ++++++
arch/arm64/kernel/suspend.c | 6 ------
arch/arm64/kernel/traps.c | 14 +++++++-------
arch/arm64/mm/proc.S | 6 ++++++
21 files changed, 127 insertions(+), 73 deletions(-)
create mode 100644 arch/arm64/include/asm/current.h
create mode 100644 arch/arm64/include/asm/stack_pointer.h
--
1.9.1
^ permalink raw reply
* [PATCH V2] dmaengine: qcom_hidma: cleanup sysfs entries during remove
From: Sinan Kaya @ 2016-10-19 18:42 UTC (permalink / raw)
To: linux-arm-kernel
The 4.8-rc8 kernel is printing duplicate file entry warnings while removing
the HIDMA object. This is caused by stale sysfs entries remaining from the
previous execution.
_sysfs_warn_dup+0x5c/0x78
sysfs_add_file_mode_ns+0x13c/0x1c0
sysfs_create_file_ns+0x2c/0x40
device_create_file+0x54/0xa0
hidma_probe+0x7c8/0x808
Create hidma_sysfs_init and hidma_sysfs_uninit functions and call them from
the probe and remove path. To do proper clean up, adding the attrs object
to the device data structure to keep it around until remove call is made.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/dma/qcom/hidma.c | 27 +++++++++++++++++++++------
drivers/dma/qcom/hidma.h | 3 +++
2 files changed, 24 insertions(+), 6 deletions(-)
diff --git a/drivers/dma/qcom/hidma.c b/drivers/dma/qcom/hidma.c
index e244e10..4adf249 100644
--- a/drivers/dma/qcom/hidma.c
+++ b/drivers/dma/qcom/hidma.c
@@ -567,8 +567,13 @@ static ssize_t hidma_show_values(struct device *dev,
return strlen(buf);
}
-static int hidma_create_sysfs_entry(struct hidma_dev *dev, char *name,
- int mode)
+static inline void hidma_sysfs_uninit(struct hidma_dev *dev)
+{
+ device_remove_file(dev->ddev.dev, dev->chid_attrs);
+}
+
+static struct device_attribute*
+hidma_create_sysfs_entry(struct hidma_dev *dev, char *name, int mode)
{
struct device_attribute *attrs;
char *name_copy;
@@ -576,18 +581,27 @@ static int hidma_create_sysfs_entry(struct hidma_dev *dev, char *name,
attrs = devm_kmalloc(dev->ddev.dev, sizeof(struct device_attribute),
GFP_KERNEL);
if (!attrs)
- return -ENOMEM;
+ return NULL;
name_copy = devm_kstrdup(dev->ddev.dev, name, GFP_KERNEL);
if (!name_copy)
- return -ENOMEM;
+ return NULL;
attrs->attr.name = name_copy;
attrs->attr.mode = mode;
attrs->show = hidma_show_values;
sysfs_attr_init(&attrs->attr);
- return device_create_file(dev->ddev.dev, attrs);
+ return attrs;
+}
+
+static int hidma_sysfs_init(struct hidma_dev *dev)
+{
+ dev->chid_attrs = hidma_create_sysfs_entry(dev, "chid", S_IRUGO);
+ if (!dev->chid_attrs)
+ return -ENOMEM;
+
+ return device_create_file(dev->ddev.dev, dev->chid_attrs);
}
static int hidma_probe(struct platform_device *pdev)
@@ -705,7 +719,7 @@ static int hidma_probe(struct platform_device *pdev)
dmadev->irq = chirq;
tasklet_init(&dmadev->task, hidma_issue_task, (unsigned long)dmadev);
hidma_debug_init(dmadev);
- hidma_create_sysfs_entry(dmadev, "chid", S_IRUGO);
+ hidma_sysfs_init(dmadev);
dev_info(&pdev->dev, "HI-DMA engine driver registration complete\n");
platform_set_drvdata(pdev, dmadev);
pm_runtime_mark_last_busy(dmadev->ddev.dev);
@@ -732,6 +746,7 @@ static int hidma_remove(struct platform_device *pdev)
dma_async_device_unregister(&dmadev->ddev);
devm_free_irq(dmadev->ddev.dev, dmadev->irq, dmadev->lldev);
tasklet_kill(&dmadev->task);
+ hidma_sysfs_uninit(dmadev);
hidma_debug_uninit(dmadev);
hidma_ll_uninit(dmadev->lldev);
hidma_free(dmadev);
diff --git a/drivers/dma/qcom/hidma.h b/drivers/dma/qcom/hidma.h
index e52e207..b4a512f 100644
--- a/drivers/dma/qcom/hidma.h
+++ b/drivers/dma/qcom/hidma.h
@@ -128,6 +128,9 @@ struct hidma_dev {
struct dentry *debugfs;
struct dentry *stats;
+ /* sysfs entry for the channel id */
+ struct device_attribute *chid_attrs;
+
/* Task delivering issue_pending */
struct tasklet_struct task;
};
--
1.9.1
^ permalink raw reply related
* [PATCH v6 2/2] Documentation: DT: MMC: meson-gx: new bindings doc
From: Kevin Hilman @ 2016-10-19 18:18 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20161019181825.2186-1-khilman@baylibre.com>
Signed-off-by: Kevin Hilman <khilman@baylibre.com>
---
.../devicetree/bindings/mmc/amlogic,meson-gx.txt | 33 ++++++++++++++++++++++
1 file changed, 33 insertions(+)
create mode 100644 Documentation/devicetree/bindings/mmc/amlogic,meson-gx.txt
diff --git a/Documentation/devicetree/bindings/mmc/amlogic,meson-gx.txt b/Documentation/devicetree/bindings/mmc/amlogic,meson-gx.txt
new file mode 100644
index 000000000000..a2fa9a1c26ae
--- /dev/null
+++ b/Documentation/devicetree/bindings/mmc/amlogic,meson-gx.txt
@@ -0,0 +1,33 @@
+Amlogic SD / eMMC controller for S905/GXBB family SoCs
+
+The MMC 5.1 compliant host controller on Amlogic provides the
+interface for SD, eMMC and SDIO devices.
+
+This file documents the properties in addition to those available in
+the MMC core bindings, documented by mmc.txt.
+
+Required properties:
+- compatible : contains one of:
+ - "amlogic,meson-gx-mmc"
+ - "amlogic,meson-gxbb-mmc"
+ - "amlogic,meson-gxl-mmc"
+ - "amlogic,meson-gxm-mmc"
+- clocks : A list of phandle + clock-specifier pairs for the clocks listed in clock-names.
+- clock-names: Should contain the following:
+ "core" - Main peripheral bus clock
+ "clkin0" - Parent clock of internal mux
+ "clkin1" - Other parent clock of internal mux
+ The driver has an interal mux clock which switches between clkin0 and clkin1 depending on the
+ clock rate requested by the MMC core.
+
+Example:
+
+ sd_emmc_a: mmc at 70000 {
+ compatible = "amlogic,meson-gxbb-mmc";
+ reg = <0x0 0x70000 0x0 0x2000>;
+ interrupts = < GIC_SPI 216 IRQ_TYPE_EDGE_RISING>;
+ clocks = <&clkc CLKID_SD_EMMC_A>, <&xtal>, <&clkc CLKID_FCLK_DIV2>;
+ clock-names = "core", "clkin0", "clkin1";
+ pinctrl-0 = <&emmc_pins>;
+ };
+
--
2.9.3
^ permalink raw reply related
* [PATCH v6 1/2] MMC: meson: initial support for GX platforms
From: Kevin Hilman @ 2016-10-19 18:18 UTC (permalink / raw)
To: linux-arm-kernel
Initial support for the SD/eMMC controller in the Amlogic S905/GX*
family of SoCs.
Signed-off-by: Kevin Hilman <khilman@baylibre.com>
---
Changes since v5:
- dropped MODULE_ALIAS
- renmaed file to meson-gx-mmc (for more useful module name)
- update DRIVER_NAME: s/gxbb/gx/
MAINTAINERS | 1 +
drivers/mmc/host/Kconfig | 10 +
drivers/mmc/host/Makefile | 1 +
drivers/mmc/host/meson-gx-mmc.c | 852 ++++++++++++++++++++++++++++++++++++++++
4 files changed, 864 insertions(+)
create mode 100644 drivers/mmc/host/meson-gx-mmc.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 1cd38a7e0064..73e8d64ec28c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1036,6 +1036,7 @@ F: arch/arm/mach-meson/
F: arch/arm/boot/dts/meson*
F: arch/arm64/boot/dts/amlogic/
F: drivers/pinctrl/meson/
+F: drivers/mmc/host/meson*
N: meson
ARM/Annapurna Labs ALPINE ARCHITECTURE
diff --git a/drivers/mmc/host/Kconfig b/drivers/mmc/host/Kconfig
index 5274f503a39a..5cf7ebaf1e8b 100644
--- a/drivers/mmc/host/Kconfig
+++ b/drivers/mmc/host/Kconfig
@@ -322,6 +322,16 @@ config MMC_SDHCI_IPROC
If unsure, say N.
+config MMC_MESON_GX
+ tristate "Amlogic S905/GX* SD/MMC Host Controller support"
+ depends on ARCH_MESON && MMC
+ help
+ This selects support for the Amlogic SD/MMC Host Controller
+ found on the S905/GX* family of SoCs. This controller is
+ MMC 5.1 compliant and supports SD, eMMC and SDIO interfaces.
+
+ If you have a controller with this interface, say Y here.
+
config MMC_MOXART
tristate "MOXART SD/MMC Host Controller support"
depends on ARCH_MOXART && MMC
diff --git a/drivers/mmc/host/Makefile b/drivers/mmc/host/Makefile
index e2bdaaf43184..e609bf04346b 100644
--- a/drivers/mmc/host/Makefile
+++ b/drivers/mmc/host/Makefile
@@ -53,6 +53,7 @@ obj-$(CONFIG_MMC_JZ4740) += jz4740_mmc.o
obj-$(CONFIG_MMC_VUB300) += vub300.o
obj-$(CONFIG_MMC_USHC) += ushc.o
obj-$(CONFIG_MMC_WMT) += wmt-sdmmc.o
+obj-$(CONFIG_MMC_MESON_GX) += meson-gx-mmc.o
obj-$(CONFIG_MMC_MOXART) += moxart-mmc.o
obj-$(CONFIG_MMC_SUNXI) += sunxi-mmc.o
obj-$(CONFIG_MMC_USDHI6ROL0) += usdhi6rol0.o
diff --git a/drivers/mmc/host/meson-gx-mmc.c b/drivers/mmc/host/meson-gx-mmc.c
new file mode 100644
index 000000000000..1fcab7116f26
--- /dev/null
+++ b/drivers/mmc/host/meson-gx-mmc.c
@@ -0,0 +1,852 @@
+/*
+ * Amlogic SD/eMMC driver for the GX/S905 family SoCs
+ *
+ * Copyright (c) 2016 BayLibre, SAS.
+ * Author: Kevin Hilman <khilman@baylibre.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ * The full GNU General Public License is included in this distribution
+ * in the file called COPYING.
+ */
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/device.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/ioport.h>
+#include <linux/spinlock.h>
+#include <linux/dma-mapping.h>
+#include <linux/mmc/host.h>
+#include <linux/mmc/mmc.h>
+#include <linux/mmc/sdio.h>
+#include <linux/mmc/slot-gpio.h>
+#include <linux/io.h>
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
+#include <linux/regulator/consumer.h>
+
+#define DRIVER_NAME "meson-gx-mmc"
+
+#define SD_EMMC_CLOCK 0x0
+#define CLK_DIV_SHIFT 0
+#define CLK_DIV_WIDTH 6
+#define CLK_DIV_MASK 0x3f
+#define CLK_DIV_MAX 63
+#define CLK_SRC_SHIFT 6
+#define CLK_SRC_WIDTH 2
+#define CLK_SRC_MASK 0x3
+#define CLK_SRC_XTAL 0 /* external crystal */
+#define CLK_SRC_XTAL_RATE 24000000
+#define CLK_SRC_PLL 1 /* FCLK_DIV2 */
+#define CLK_SRC_PLL_RATE 1000000000
+#define CLK_PHASE_SHIFT 8
+#define CLK_PHASE_MASK 0x3
+#define CLK_PHASE_0 0
+#define CLK_PHASE_90 1
+#define CLK_PHASE_180 2
+#define CLK_PHASE_270 3
+#define CLK_ALWAYS_ON BIT(24)
+
+#define SD_EMMC_DElAY 0x4
+#define SD_EMMC_ADJUST 0x8
+#define SD_EMMC_CALOUT 0x10
+#define SD_EMMC_START 0x40
+#define START_DESC_INIT BIT(0)
+#define START_DESC_BUSY BIT(1)
+#define START_DESC_ADDR_SHIFT 2
+#define START_DESC_ADDR_MASK (~0x3)
+
+#define SD_EMMC_CFG 0x44
+#define CFG_BUS_WIDTH_SHIFT 0
+#define CFG_BUS_WIDTH_MASK 0x3
+#define CFG_BUS_WIDTH_1 0x0
+#define CFG_BUS_WIDTH_4 0x1
+#define CFG_BUS_WIDTH_8 0x2
+#define CFG_DDR BIT(2)
+#define CFG_BLK_LEN_SHIFT 4
+#define CFG_BLK_LEN_MASK 0xf
+#define CFG_RESP_TIMEOUT_SHIFT 8
+#define CFG_RESP_TIMEOUT_MASK 0xf
+#define CFG_RC_CC_SHIFT 12
+#define CFG_RC_CC_MASK 0xf
+#define CFG_STOP_CLOCK BIT(22)
+#define CFG_CLK_ALWAYS_ON BIT(18)
+#define CFG_AUTO_CLK BIT(23)
+
+#define SD_EMMC_STATUS 0x48
+#define STATUS_BUSY BIT(31)
+
+#define SD_EMMC_IRQ_EN 0x4c
+#define IRQ_EN_MASK 0x3fff
+#define IRQ_RXD_ERR_SHIFT 0
+#define IRQ_RXD_ERR_MASK 0xff
+#define IRQ_TXD_ERR BIT(8)
+#define IRQ_DESC_ERR BIT(9)
+#define IRQ_RESP_ERR BIT(10)
+#define IRQ_RESP_TIMEOUT BIT(11)
+#define IRQ_DESC_TIMEOUT BIT(12)
+#define IRQ_END_OF_CHAIN BIT(13)
+#define IRQ_RESP_STATUS BIT(14)
+#define IRQ_SDIO BIT(15)
+
+#define SD_EMMC_CMD_CFG 0x50
+#define SD_EMMC_CMD_ARG 0x54
+#define SD_EMMC_CMD_DAT 0x58
+#define SD_EMMC_CMD_RSP 0x5c
+#define SD_EMMC_CMD_RSP1 0x60
+#define SD_EMMC_CMD_RSP2 0x64
+#define SD_EMMC_CMD_RSP3 0x68
+
+#define SD_EMMC_RXD 0x94
+#define SD_EMMC_TXD 0x94
+#define SD_EMMC_LAST_REG SD_EMMC_TXD
+
+#define SD_EMMC_CFG_BLK_SIZE 512 /* internal buffer max: 512 bytes */
+#define SD_EMMC_CFG_RESP_TIMEOUT 256 /* in clock cycles */
+#define SD_EMMC_CFG_CMD_GAP 16 /* in clock cycles */
+#define MUX_CLK_NUM_PARENTS 2
+
+struct meson_host {
+ struct device *dev;
+ struct mmc_host *mmc;
+ struct mmc_request *mrq;
+ struct mmc_command *cmd;
+
+ spinlock_t lock;
+ void __iomem *regs;
+ int irq;
+ u32 ocr_mask;
+ struct clk *core_clk;
+ struct clk_mux mux;
+ struct clk *mux_clk;
+ struct clk *mux_parent[MUX_CLK_NUM_PARENTS];
+ unsigned long mux_parent_rate[MUX_CLK_NUM_PARENTS];
+
+ struct clk_divider cfg_div;
+ struct clk *cfg_div_clk;
+
+ unsigned int bounce_buf_size;
+ void *bounce_buf;
+ dma_addr_t bounce_dma_addr;
+
+ bool vqmmc_enabled;
+};
+
+struct sd_emmc_desc {
+ u32 cmd_cfg;
+ u32 cmd_arg;
+ u32 cmd_data;
+ u32 cmd_resp;
+};
+#define CMD_CFG_LENGTH_SHIFT 0
+#define CMD_CFG_LENGTH_MASK 0x1ff
+#define CMD_CFG_BLOCK_MODE BIT(9)
+#define CMD_CFG_R1B BIT(10)
+#define CMD_CFG_END_OF_CHAIN BIT(11)
+#define CMD_CFG_TIMEOUT_SHIFT 12
+#define CMD_CFG_TIMEOUT_MASK 0xf
+#define CMD_CFG_NO_RESP BIT(16)
+#define CMD_CFG_NO_CMD BIT(17)
+#define CMD_CFG_DATA_IO BIT(18)
+#define CMD_CFG_DATA_WR BIT(19)
+#define CMD_CFG_RESP_NOCRC BIT(20)
+#define CMD_CFG_RESP_128 BIT(21)
+#define CMD_CFG_RESP_NUM BIT(22)
+#define CMD_CFG_DATA_NUM BIT(23)
+#define CMD_CFG_CMD_INDEX_SHIFT 24
+#define CMD_CFG_CMD_INDEX_MASK 0x3f
+#define CMD_CFG_ERROR BIT(30)
+#define CMD_CFG_OWNER BIT(31)
+
+#define CMD_DATA_MASK (~0x3)
+#define CMD_DATA_BIG_ENDIAN BIT(1)
+#define CMD_DATA_SRAM BIT(0)
+#define CMD_RESP_MASK (~0x1)
+#define CMD_RESP_SRAM BIT(0)
+
+static int meson_mmc_clk_set(struct meson_host *host, unsigned long clk_rate)
+{
+ struct mmc_host *mmc = host->mmc;
+ int ret = 0;
+ u32 cfg;
+
+ if (clk_rate) {
+ if (WARN_ON(clk_rate > mmc->f_max))
+ clk_rate = mmc->f_max;
+ else if (WARN_ON(clk_rate < mmc->f_min))
+ clk_rate = mmc->f_min;
+ }
+
+ if (clk_rate == mmc->actual_clock)
+ return 0;
+
+ /* stop clock */
+ cfg = readl(host->regs + SD_EMMC_CFG);
+ if (!(cfg & CFG_STOP_CLOCK)) {
+ cfg |= CFG_STOP_CLOCK;
+ writel(cfg, host->regs + SD_EMMC_CFG);
+ }
+
+ dev_dbg(host->dev, "change clock rate %u -> %lu\n",
+ mmc->actual_clock, clk_rate);
+
+ if (clk_rate == 0) {
+ mmc->actual_clock = 0;
+ return 0;
+ }
+
+ ret = clk_set_rate(host->cfg_div_clk, clk_rate);
+ if (ret)
+ dev_warn(host->dev, "Unable to set cfg_div_clk to %lu. ret=%d\n",
+ clk_rate, ret);
+ else if (clk_rate && clk_rate != clk_get_rate(host->cfg_div_clk))
+ dev_warn(host->dev, "divider requested rate %lu != actual rate %lu: ret=%d\n",
+ clk_rate, clk_get_rate(host->cfg_div_clk), ret);
+ else
+ mmc->actual_clock = clk_rate;
+
+ /* (re)start clock, if non-zero */
+ if (!ret && clk_rate) {
+ cfg = readl(host->regs + SD_EMMC_CFG);
+ cfg &= ~CFG_STOP_CLOCK;
+ writel(cfg, host->regs + SD_EMMC_CFG);
+ }
+
+ return ret;
+}
+
+/*
+ * The SD/eMMC IP block has an internal mux and divider used for
+ * generating the MMC clock. Use the clock framework to create and
+ * manage these clocks.
+ */
+static int meson_mmc_clk_init(struct meson_host *host)
+{
+ struct clk_init_data init;
+ char clk_name[32];
+ int i, ret = 0;
+ const char *mux_parent_names[MUX_CLK_NUM_PARENTS];
+ unsigned int mux_parent_count = 0;
+ const char *clk_div_parents[1];
+ unsigned int f_min = UINT_MAX;
+ u32 clk_reg, cfg;
+
+ /* get the mux parents */
+ for (i = 0; i < MUX_CLK_NUM_PARENTS; i++) {
+ char name[16];
+
+ snprintf(name, sizeof(name), "clkin%d", i);
+ host->mux_parent[i] = devm_clk_get(host->dev, name);
+ if (IS_ERR(host->mux_parent[i])) {
+ ret = PTR_ERR(host->mux_parent[i]);
+ if (PTR_ERR(host->mux_parent[i]) != -EPROBE_DEFER)
+ dev_err(host->dev, "Missing clock %s\n", name);
+ host->mux_parent[i] = NULL;
+ return ret;
+ }
+
+ host->mux_parent_rate[i] = clk_get_rate(host->mux_parent[i]);
+ mux_parent_names[i] = __clk_get_name(host->mux_parent[i]);
+ mux_parent_count++;
+ if (host->mux_parent_rate[i] < f_min)
+ f_min = host->mux_parent_rate[i];
+ }
+
+ /* cacluate f_min based on input clocks, and max divider value */
+ if (f_min != UINT_MAX)
+ f_min = DIV_ROUND_UP(CLK_SRC_XTAL_RATE, CLK_DIV_MAX);
+ else
+ f_min = 4000000; /* default min: 400 MHz */
+ host->mmc->f_min = f_min;
+
+ /* create the mux */
+ snprintf(clk_name, sizeof(clk_name), "%s#mux", dev_name(host->dev));
+ init.name = clk_name;
+ init.ops = &clk_mux_ops;
+ init.flags = 0;
+ init.parent_names = mux_parent_names;
+ init.num_parents = mux_parent_count;
+
+ host->mux.reg = host->regs + SD_EMMC_CLOCK;
+ host->mux.shift = CLK_SRC_SHIFT;
+ host->mux.mask = CLK_SRC_MASK;
+ host->mux.flags = 0;
+ host->mux.table = NULL;
+ host->mux.hw.init = &init;
+
+ host->mux_clk = devm_clk_register(host->dev, &host->mux.hw);
+ if (WARN_ON(IS_ERR(host->mux_clk)))
+ return PTR_ERR(host->mux_clk);
+
+ /* create the divider */
+ snprintf(clk_name, sizeof(clk_name), "%s#div", dev_name(host->dev));
+ init.name = devm_kstrdup(host->dev, clk_name, GFP_KERNEL);
+ init.ops = &clk_divider_ops;
+ init.flags = CLK_SET_RATE_PARENT;
+ clk_div_parents[0] = __clk_get_name(host->mux_clk);
+ init.parent_names = clk_div_parents;
+ init.num_parents = ARRAY_SIZE(clk_div_parents);
+
+ host->cfg_div.reg = host->regs + SD_EMMC_CLOCK;
+ host->cfg_div.shift = CLK_DIV_SHIFT;
+ host->cfg_div.width = CLK_DIV_WIDTH;
+ host->cfg_div.hw.init = &init;
+ host->cfg_div.flags = CLK_DIVIDER_ONE_BASED |
+ CLK_DIVIDER_ROUND_CLOSEST | CLK_DIVIDER_ALLOW_ZERO;
+
+ host->cfg_div_clk = devm_clk_register(host->dev, &host->cfg_div.hw);
+ if (WARN_ON(PTR_ERR_OR_ZERO(host->cfg_div_clk)))
+ return PTR_ERR(host->cfg_div_clk);
+
+ /* init SD_EMMC_CLOCK to sane defaults w/min clock rate */
+ clk_reg = 0;
+ clk_reg |= CLK_PHASE_180 << CLK_PHASE_SHIFT;
+ clk_reg |= CLK_SRC_XTAL << CLK_SRC_SHIFT;
+ clk_reg |= CLK_DIV_MAX << CLK_DIV_SHIFT;
+ clk_reg &= ~CLK_ALWAYS_ON;
+ writel(clk_reg, host->regs + SD_EMMC_CLOCK);
+
+ /* Ensure clock starts in "auto" mode, not "always on" */
+ cfg = readl(host->regs + SD_EMMC_CFG);
+ cfg &= ~CFG_CLK_ALWAYS_ON;
+ cfg |= CFG_AUTO_CLK;
+ writel(cfg, host->regs + SD_EMMC_CFG);
+
+ ret = clk_prepare_enable(host->cfg_div_clk);
+ if (!ret)
+ ret = meson_mmc_clk_set(host, f_min);
+
+ if (!ret)
+ clk_disable_unprepare(host->cfg_div_clk);
+
+ return ret;
+}
+
+static void meson_mmc_set_ios(struct mmc_host *mmc, struct mmc_ios *ios)
+{
+ struct meson_host *host = mmc_priv(mmc);
+ u32 bus_width;
+ u32 val, orig;
+
+ /*
+ * GPIO regulator, only controls switching between 1v8 and
+ * 3v3, doesn't support MMC_POWER_OFF, MMC_POWER_ON.
+ */
+ switch (ios->power_mode) {
+ case MMC_POWER_OFF:
+ if (!IS_ERR(mmc->supply.vmmc))
+ mmc_regulator_set_ocr(mmc, mmc->supply.vmmc, 0);
+
+ if (!IS_ERR(mmc->supply.vqmmc) && host->vqmmc_enabled) {
+ regulator_disable(mmc->supply.vqmmc);
+ host->vqmmc_enabled = false;
+ }
+
+ break;
+
+ case MMC_POWER_UP:
+ if (!IS_ERR(mmc->supply.vmmc))
+ mmc_regulator_set_ocr(mmc, mmc->supply.vmmc, ios->vdd);
+ break;
+
+ case MMC_POWER_ON:
+ if (!IS_ERR(mmc->supply.vqmmc) && !host->vqmmc_enabled) {
+ int ret = regulator_enable(mmc->supply.vqmmc);
+
+ if (ret < 0)
+ dev_err(mmc_dev(mmc),
+ "failed to enable vqmmc regulator\n");
+ else
+ host->vqmmc_enabled = true;
+ }
+
+ break;
+ }
+
+
+ meson_mmc_clk_set(host, ios->clock);
+
+ /* Bus width */
+ val = readl(host->regs + SD_EMMC_CFG);
+ switch (ios->bus_width) {
+ case MMC_BUS_WIDTH_1:
+ bus_width = CFG_BUS_WIDTH_1;
+ break;
+ case MMC_BUS_WIDTH_4:
+ bus_width = CFG_BUS_WIDTH_4;
+ break;
+ case MMC_BUS_WIDTH_8:
+ bus_width = CFG_BUS_WIDTH_8;
+ break;
+ default:
+ dev_err(host->dev, "Invalid ios->bus_width: %u. Setting to 4.\n",
+ ios->bus_width);
+ bus_width = CFG_BUS_WIDTH_4;
+ return;
+ }
+
+ val = readl(host->regs + SD_EMMC_CFG);
+ orig = val;
+
+ val &= ~(CFG_BUS_WIDTH_MASK << CFG_BUS_WIDTH_SHIFT);
+ val |= bus_width << CFG_BUS_WIDTH_SHIFT;
+
+ val &= ~(CFG_BLK_LEN_MASK << CFG_BLK_LEN_SHIFT);
+ val |= ilog2(SD_EMMC_CFG_BLK_SIZE) << CFG_BLK_LEN_SHIFT;
+
+ val &= ~(CFG_RESP_TIMEOUT_MASK << CFG_RESP_TIMEOUT_SHIFT);
+ val |= ilog2(SD_EMMC_CFG_RESP_TIMEOUT) << CFG_RESP_TIMEOUT_SHIFT;
+
+ val &= ~(CFG_RC_CC_MASK << CFG_RC_CC_SHIFT);
+ val |= ilog2(SD_EMMC_CFG_CMD_GAP) << CFG_RC_CC_SHIFT;
+
+ writel(val, host->regs + SD_EMMC_CFG);
+
+ if (val != orig)
+ dev_dbg(host->dev, "%s: SD_EMMC_CFG: 0x%08x -> 0x%08x\n",
+ __func__, orig, val);
+}
+
+static int meson_mmc_request_done(struct mmc_host *mmc, struct mmc_request *mrq)
+{
+ struct meson_host *host = mmc_priv(mmc);
+
+ WARN_ON(host->mrq != mrq);
+
+ host->mrq = NULL;
+ host->cmd = NULL;
+ mmc_request_done(host->mmc, mrq);
+
+ return 0;
+}
+
+static void meson_mmc_start_cmd(struct mmc_host *mmc, struct mmc_command *cmd)
+{
+ struct meson_host *host = mmc_priv(mmc);
+ struct sd_emmc_desc *desc, desc_tmp;
+ u32 cfg;
+ u8 blk_len, cmd_cfg_timeout;
+ unsigned int xfer_bytes = 0;
+
+ /* Setup descriptors */
+ dma_rmb();
+ desc = &desc_tmp;
+ memset(desc, 0, sizeof(struct sd_emmc_desc));
+
+ desc->cmd_cfg |= (cmd->opcode & CMD_CFG_CMD_INDEX_MASK) <<
+ CMD_CFG_CMD_INDEX_SHIFT;
+ desc->cmd_cfg |= CMD_CFG_OWNER; /* owned by CPU */
+ desc->cmd_arg = cmd->arg;
+
+ /* Response */
+ if (cmd->flags & MMC_RSP_PRESENT) {
+ desc->cmd_cfg &= ~CMD_CFG_NO_RESP;
+ if (cmd->flags & MMC_RSP_136)
+ desc->cmd_cfg |= CMD_CFG_RESP_128;
+ desc->cmd_cfg |= CMD_CFG_RESP_NUM;
+ desc->cmd_resp = 0;
+
+ if (!(cmd->flags & MMC_RSP_CRC))
+ desc->cmd_cfg |= CMD_CFG_RESP_NOCRC;
+
+ if (cmd->flags & MMC_RSP_BUSY)
+ desc->cmd_cfg |= CMD_CFG_R1B;
+ } else {
+ desc->cmd_cfg |= CMD_CFG_NO_RESP;
+ }
+
+ /* data? */
+ if (cmd->data) {
+ desc->cmd_cfg |= CMD_CFG_DATA_IO;
+ if (cmd->data->blocks > 1) {
+ desc->cmd_cfg |= CMD_CFG_BLOCK_MODE;
+ desc->cmd_cfg |=
+ (cmd->data->blocks & CMD_CFG_LENGTH_MASK) <<
+ CMD_CFG_LENGTH_SHIFT;
+
+ /* check if block-size matches, if not update */
+ cfg = readl(host->regs + SD_EMMC_CFG);
+ blk_len = cfg & (CFG_BLK_LEN_MASK << CFG_BLK_LEN_SHIFT);
+ blk_len >>= CFG_BLK_LEN_SHIFT;
+ if (blk_len != ilog2(cmd->data->blksz)) {
+ dev_warn(host->dev, "%s: update blk_len %d -> %d\n",
+ __func__, blk_len,
+ ilog2(cmd->data->blksz));
+ blk_len = ilog2(cmd->data->blksz);
+ cfg &= ~(CFG_BLK_LEN_MASK << CFG_BLK_LEN_SHIFT);
+ cfg |= blk_len << CFG_BLK_LEN_SHIFT;
+ writel(cfg, host->regs + SD_EMMC_CFG);
+ }
+ } else {
+ desc->cmd_cfg &= ~CMD_CFG_BLOCK_MODE;
+ desc->cmd_cfg |=
+ (cmd->data->blksz & CMD_CFG_LENGTH_MASK) <<
+ CMD_CFG_LENGTH_SHIFT;
+ }
+
+ cmd->data->bytes_xfered = 0;
+ xfer_bytes = cmd->data->blksz * cmd->data->blocks;
+ if (cmd->data->flags & MMC_DATA_WRITE) {
+ desc->cmd_cfg |= CMD_CFG_DATA_WR;
+ WARN_ON(xfer_bytes > host->bounce_buf_size);
+ sg_copy_to_buffer(cmd->data->sg, cmd->data->sg_len,
+ host->bounce_buf, xfer_bytes);
+ cmd->data->bytes_xfered = xfer_bytes;
+ dma_wmb();
+ } else {
+ desc->cmd_cfg &= ~CMD_CFG_DATA_WR;
+ }
+
+ if (xfer_bytes > 0) {
+ desc->cmd_cfg &= ~CMD_CFG_DATA_NUM;
+ desc->cmd_data = host->bounce_dma_addr & CMD_DATA_MASK;
+ } else {
+ /* write data to data_addr */
+ desc->cmd_cfg |= CMD_CFG_DATA_NUM;
+ desc->cmd_data = 0;
+ }
+
+ cmd_cfg_timeout = 12;
+ } else {
+ desc->cmd_cfg &= ~CMD_CFG_DATA_IO;
+ cmd_cfg_timeout = 10;
+ }
+ desc->cmd_cfg |= (cmd_cfg_timeout & CMD_CFG_TIMEOUT_MASK) <<
+ CMD_CFG_TIMEOUT_SHIFT;
+
+ host->cmd = cmd;
+
+ /* Last descriptor */
+ desc->cmd_cfg |= CMD_CFG_END_OF_CHAIN;
+ writel(desc->cmd_cfg, host->regs + SD_EMMC_CMD_CFG);
+ writel(desc->cmd_data, host->regs + SD_EMMC_CMD_DAT);
+ writel(desc->cmd_resp, host->regs + SD_EMMC_CMD_RSP);
+ wmb(); /* ensure descriptor is written before kicked */
+ writel(desc->cmd_arg, host->regs + SD_EMMC_CMD_ARG);
+}
+
+static void meson_mmc_request(struct mmc_host *mmc, struct mmc_request *mrq)
+{
+ struct meson_host *host = mmc_priv(mmc);
+
+ WARN_ON(host->mrq != NULL);
+
+ /* Stop execution */
+ writel(0, host->regs + SD_EMMC_START);
+
+ /* clear, ack, enable all interrupts */
+ writel(0, host->regs + SD_EMMC_IRQ_EN);
+ writel(IRQ_EN_MASK, host->regs + SD_EMMC_STATUS);
+ writel(IRQ_EN_MASK, host->regs + SD_EMMC_IRQ_EN);
+
+ host->mrq = mrq;
+
+ if (mrq->sbc)
+ meson_mmc_start_cmd(mmc, mrq->sbc);
+ else
+ meson_mmc_start_cmd(mmc, mrq->cmd);
+}
+
+static int meson_mmc_read_resp(struct mmc_host *mmc, struct mmc_command *cmd)
+{
+ struct meson_host *host = mmc_priv(mmc);
+
+ if (cmd->flags & MMC_RSP_136) {
+ cmd->resp[0] = readl(host->regs + SD_EMMC_CMD_RSP3);
+ cmd->resp[1] = readl(host->regs + SD_EMMC_CMD_RSP2);
+ cmd->resp[2] = readl(host->regs + SD_EMMC_CMD_RSP1);
+ cmd->resp[3] = readl(host->regs + SD_EMMC_CMD_RSP);
+ } else if (cmd->flags & MMC_RSP_PRESENT) {
+ cmd->resp[0] = readl(host->regs + SD_EMMC_CMD_RSP);
+ }
+
+ return 0;
+}
+
+static irqreturn_t meson_mmc_irq(int irq, void *dev_id)
+{
+ struct meson_host *host = dev_id;
+ struct mmc_request *mrq;
+ struct mmc_command *cmd = host->cmd;
+ u32 irq_en, status, raw_status;
+ irqreturn_t ret = IRQ_HANDLED;
+
+ if (WARN_ON(!host))
+ return IRQ_NONE;
+
+ mrq = host->mrq;
+
+ if (WARN_ON(!mrq))
+ return IRQ_NONE;
+
+ if (WARN_ON(!cmd))
+ return IRQ_NONE;
+
+ spin_lock(&host->lock);
+ irq_en = readl(host->regs + SD_EMMC_IRQ_EN);
+ raw_status = readl(host->regs + SD_EMMC_STATUS);
+ status = raw_status & irq_en;
+
+ if (!status) {
+ dev_warn(host->dev, "Spurious IRQ! status=0x%08x, irq_en=0x%08x\n",
+ raw_status, irq_en);
+ ret = IRQ_NONE;
+ goto out;
+ }
+
+ cmd->error = 0;
+ if (status & IRQ_RXD_ERR_MASK) {
+ dev_dbg(host->dev, "Unhandled IRQ: RXD error\n");
+ cmd->error = -EILSEQ;
+ }
+ if (status & IRQ_TXD_ERR) {
+ dev_dbg(host->dev, "Unhandled IRQ: TXD error\n");
+ cmd->error = -EILSEQ;
+ }
+ if (status & IRQ_DESC_ERR)
+ dev_dbg(host->dev, "Unhandled IRQ: Descriptor error\n");
+ if (status & IRQ_RESP_ERR) {
+ dev_dbg(host->dev, "Unhandled IRQ: Response error\n");
+ cmd->error = -EILSEQ;
+ }
+ if (status & IRQ_RESP_TIMEOUT) {
+ dev_dbg(host->dev, "Unhandled IRQ: Response timeout\n");
+ cmd->error = -ETIMEDOUT;
+ }
+ if (status & IRQ_DESC_TIMEOUT) {
+ dev_dbg(host->dev, "Unhandled IRQ: Descriptor timeout\n");
+ cmd->error = -ETIMEDOUT;
+ }
+ if (status & IRQ_SDIO)
+ dev_dbg(host->dev, "Unhandled IRQ: SDIO.\n");
+
+ if (status & (IRQ_END_OF_CHAIN | IRQ_RESP_STATUS))
+ ret = IRQ_WAKE_THREAD;
+ else {
+ dev_warn(host->dev, "Unknown IRQ! status=0x%04x: MMC CMD%u arg=0x%08x flags=0x%08x stop=%d\n",
+ status, cmd->opcode, cmd->arg,
+ cmd->flags, mrq->stop ? 1 : 0);
+ if (cmd->data) {
+ struct mmc_data *data = cmd->data;
+
+ dev_warn(host->dev, "\tblksz %u blocks %u flags 0x%08x (%s%s)",
+ data->blksz, data->blocks, data->flags,
+ data->flags & MMC_DATA_WRITE ? "write" : "",
+ data->flags & MMC_DATA_READ ? "read" : "");
+ }
+ }
+
+out:
+ /* ack all (enabled) interrupts */
+ writel(status, host->regs + SD_EMMC_STATUS);
+
+ if (ret == IRQ_HANDLED) {
+ meson_mmc_read_resp(host->mmc, cmd);
+ meson_mmc_request_done(host->mmc, cmd->mrq);
+ }
+
+ spin_unlock(&host->lock);
+ return ret;
+}
+
+static irqreturn_t meson_mmc_irq_thread(int irq, void *dev_id)
+{
+ struct meson_host *host = dev_id;
+ struct mmc_request *mrq = host->mrq;
+ struct mmc_command *cmd = host->cmd;
+ struct mmc_data *data;
+ unsigned int xfer_bytes;
+ int ret = IRQ_HANDLED;
+
+ if (WARN_ON(!mrq))
+ ret = IRQ_NONE;
+
+ if (WARN_ON(!cmd))
+ ret = IRQ_NONE;
+
+ data = cmd->data;
+ if (data) {
+ xfer_bytes = data->blksz * data->blocks;
+ if (data->flags & MMC_DATA_READ) {
+ WARN_ON(xfer_bytes > host->bounce_buf_size);
+ sg_copy_from_buffer(data->sg, data->sg_len,
+ host->bounce_buf, xfer_bytes);
+ data->bytes_xfered = xfer_bytes;
+ }
+ }
+
+ meson_mmc_read_resp(host->mmc, cmd);
+ if (!data || !data->stop || mrq->sbc)
+ meson_mmc_request_done(host->mmc, mrq);
+ else
+ meson_mmc_start_cmd(host->mmc, data->stop);
+
+ return ret;
+}
+
+/*
+ * NOTE: we only need this until the GPIO/pinctrl driver can handle
+ * interrupts. For now, the MMC core will use this for polling.
+ */
+static int meson_mmc_get_cd(struct mmc_host *mmc)
+{
+ int status = mmc_gpio_get_cd(mmc);
+
+ if (status == -ENOSYS)
+ return 1; /* assume present */
+
+ return status;
+}
+
+static const struct mmc_host_ops meson_mmc_ops = {
+ .request = meson_mmc_request,
+ .set_ios = meson_mmc_set_ios,
+ .get_cd = meson_mmc_get_cd,
+};
+
+static int meson_mmc_probe(struct platform_device *pdev)
+{
+ struct resource *res;
+ struct meson_host *host;
+ struct mmc_host *mmc;
+ int ret;
+
+ mmc = mmc_alloc_host(sizeof(struct meson_host), &pdev->dev);
+ if (!mmc)
+ return -ENOMEM;
+ host = mmc_priv(mmc);
+ host->mmc = mmc;
+ host->dev = &pdev->dev;
+ dev_set_drvdata(&pdev->dev, host);
+
+ spin_lock_init(&host->lock);
+
+ /* Get regulators and the supported OCR mask */
+ host->vqmmc_enabled = false;
+ ret = mmc_regulator_get_supply(mmc);
+ if (ret == -EPROBE_DEFER)
+ goto free_host;
+
+ ret = mmc_of_parse(mmc);
+ if (ret) {
+ dev_warn(&pdev->dev, "error parsing DT: %d\n", ret);
+ goto free_host;
+ }
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ host->regs = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(host->regs)) {
+ ret = PTR_ERR(host->regs);
+ goto free_host;
+ }
+
+ host->irq = platform_get_irq(pdev, 0);
+ if (host->irq == 0) {
+ dev_err(&pdev->dev, "failed to get interrupt resource.\n");
+ ret = -EINVAL;
+ goto free_host;
+ }
+
+ host->core_clk = devm_clk_get(&pdev->dev, "core");
+ if (IS_ERR(host->core_clk)) {
+ ret = PTR_ERR(host->core_clk);
+ goto free_host;
+ }
+
+ ret = clk_prepare_enable(host->core_clk);
+ if (ret)
+ goto free_host;
+
+ ret = meson_mmc_clk_init(host);
+ if (ret)
+ goto free_host;
+
+ /* Stop execution */
+ writel(0, host->regs + SD_EMMC_START);
+
+ /* clear, ack, enable all interrupts */
+ writel(0, host->regs + SD_EMMC_IRQ_EN);
+ writel(IRQ_EN_MASK, host->regs + SD_EMMC_STATUS);
+
+ ret = devm_request_threaded_irq(&pdev->dev, host->irq,
+ meson_mmc_irq, meson_mmc_irq_thread,
+ IRQF_SHARED, DRIVER_NAME, host);
+ if (ret)
+ goto free_host;
+
+ /* data bounce buffer */
+ host->bounce_buf_size = SZ_512K;
+ host->bounce_buf =
+ dma_alloc_coherent(host->dev, host->bounce_buf_size,
+ &host->bounce_dma_addr, GFP_KERNEL);
+ if (host->bounce_buf == NULL) {
+ dev_err(host->dev, "Unable to map allocate DMA bounce buffer.\n");
+ ret = -ENOMEM;
+ goto free_host;
+ }
+
+ mmc->ops = &meson_mmc_ops;
+ mmc_add_host(mmc);
+
+ return 0;
+
+free_host:
+ clk_disable_unprepare(host->cfg_div_clk);
+ clk_disable_unprepare(host->core_clk);
+ mmc_free_host(mmc);
+ return ret;
+}
+
+static int meson_mmc_remove(struct platform_device *pdev)
+{
+ struct meson_host *host = dev_get_drvdata(&pdev->dev);
+
+ if (WARN_ON(!host))
+ return 0;
+
+ if (host->bounce_buf)
+ dma_free_coherent(host->dev, host->bounce_buf_size,
+ host->bounce_buf, host->bounce_dma_addr);
+
+ clk_disable_unprepare(host->cfg_div_clk);
+ clk_disable_unprepare(host->core_clk);
+
+ mmc_free_host(host->mmc);
+ return 0;
+}
+
+static const struct of_device_id meson_mmc_of_match[] = {
+ { .compatible = "amlogic,meson-gx-mmc", },
+ { .compatible = "amlogic,meson-gxbb-mmc", },
+ { .compatible = "amlogic,meson-gxl-mmc", },
+ { .compatible = "amlogic,meson-gxm-mmc", },
+ {}
+};
+MODULE_DEVICE_TABLE(of, meson_mmc_of_match);
+
+static struct platform_driver meson_mmc_driver = {
+ .probe = meson_mmc_probe,
+ .remove = meson_mmc_remove,
+ .driver = {
+ .name = DRIVER_NAME,
+ .of_match_table = of_match_ptr(meson_mmc_of_match),
+ },
+};
+
+module_platform_driver(meson_mmc_driver);
+
+MODULE_DESCRIPTION("Amlogic S905*/GX* SD/eMMC driver");
+MODULE_AUTHOR("Kevin Hilman <khilman@baylibre.com>");
+MODULE_LICENSE("GPL v2");
+
--
2.9.3
^ permalink raw reply related
* [PATCH v4] MMC: meson: initial support for GX platforms
From: Kevin Hilman @ 2016-10-19 18:01 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <CABxcv=kQBAdXtks-tX1t-CkPnvFx0SWa3JzLRYK4KWk+EGZedw@mail.gmail.com>
Javier Martinez Canillas <javier@dowhile0.org> writes:
> Hello Kevin,
>
> On Tue, Oct 18, 2016 at 4:56 PM, Kevin Hilman <khilman@baylibre.com> wrote:
>> Initial support for the SD/eMMC controller in the Amlogic S905/GX*
>> family of SoCs.
>>
>> Signed-off-by: Kevin Hilman <khilman@baylibre.com>
>
> [snip]
>
>> +
>> +MODULE_ALIAS("platform:" DRIVER_NAME);
>
> Why is this module alias needed? I thought that the Amlogic was a
> DT-only platform and so devices will always be registered from OF (and
> the OF modalias used).
You're right, I probably copy/pasted it from another driver.
Testing without it also made me realize that the module filename (meson_gx)
isn't really useful when listed in lsmod, so I 'll respin to have mmc in
the filename so the module name also shows mmc.
Kevin
^ permalink raw reply
* [PATCH V6 10/10] dmaengine: qcom_hidma: add MSI support for interrupts
From: Sinan Kaya @ 2016-10-19 17:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476899512-20431-1-git-send-email-okaya@codeaurora.org>
The interrupts can now be delivered as platform MSI interrupts on newer
platforms. The code looks for a new OF and ACPI strings in order to enable
the functionality.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/dma/qcom/hidma.c | 143 ++++++++++++++++++++++++++++++++++++++++++--
drivers/dma/qcom/hidma.h | 2 +
drivers/dma/qcom/hidma_ll.c | 8 +++
3 files changed, 147 insertions(+), 6 deletions(-)
diff --git a/drivers/dma/qcom/hidma.c b/drivers/dma/qcom/hidma.c
index 4adf249..2ea6917 100644
--- a/drivers/dma/qcom/hidma.c
+++ b/drivers/dma/qcom/hidma.c
@@ -56,6 +56,7 @@
#include <linux/irq.h>
#include <linux/atomic.h>
#include <linux/pm_runtime.h>
+#include <linux/msi.h>
#include "../dmaengine.h"
#include "hidma.h"
@@ -70,6 +71,7 @@
#define HIDMA_ERR_INFO_SW 0xFF
#define HIDMA_ERR_CODE_UNEXPECTED_TERMINATE 0x0
#define HIDMA_NR_DEFAULT_DESC 10
+#define HIDMA_MSI_INTS 11
static inline struct hidma_dev *to_hidma_dev(struct dma_device *dmadev)
{
@@ -553,6 +555,15 @@ static irqreturn_t hidma_chirq_handler(int chirq, void *arg)
return hidma_ll_inthandler(chirq, lldev);
}
+static irqreturn_t hidma_chirq_handler_msi(int chirq, void *arg)
+{
+ struct hidma_lldev **lldevp = arg;
+ struct hidma_dev *dmadev = to_hidma_dev_from_lldev(lldevp);
+
+ return hidma_ll_inthandler_msi(chirq, *lldevp,
+ 1 << (chirq - dmadev->msi_virqbase));
+}
+
static ssize_t hidma_show_values(struct device *dev,
struct device_attribute *attr, char *buf)
{
@@ -604,6 +615,104 @@ static int hidma_sysfs_init(struct hidma_dev *dev)
return device_create_file(dev->ddev.dev, dev->chid_attrs);
}
+#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
+static void hidma_write_msi_msg(struct msi_desc *desc, struct msi_msg *msg)
+{
+ struct device *dev = msi_desc_to_dev(desc);
+ struct hidma_dev *dmadev = dev_get_drvdata(dev);
+
+ if (!desc->platform.msi_index) {
+ writel(msg->address_lo, dmadev->dev_evca + 0x118);
+ writel(msg->address_hi, dmadev->dev_evca + 0x11C);
+ writel(msg->data, dmadev->dev_evca + 0x120);
+ }
+}
+#endif
+
+static void hidma_free_msis(struct hidma_dev *dmadev)
+{
+#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
+ struct device *dev = dmadev->ddev.dev;
+ struct msi_desc *desc;
+
+ /* free allocated MSI interrupts above */
+ for_each_msi_entry(desc, dev)
+ devm_free_irq(dev, desc->irq, &dmadev->lldev);
+
+ platform_msi_domain_free_irqs(dev);
+#endif
+}
+
+static int hidma_request_msi(struct hidma_dev *dmadev,
+ struct platform_device *pdev)
+{
+#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
+ int rc;
+ struct msi_desc *desc;
+ struct msi_desc *failed_desc = NULL;
+
+ rc = platform_msi_domain_alloc_irqs(&pdev->dev, HIDMA_MSI_INTS,
+ hidma_write_msi_msg);
+ if (rc)
+ return rc;
+
+ for_each_msi_entry(desc, &pdev->dev) {
+ if (!desc->platform.msi_index)
+ dmadev->msi_virqbase = desc->irq;
+
+ rc = devm_request_irq(&pdev->dev, desc->irq,
+ hidma_chirq_handler_msi,
+ 0, "qcom-hidma-msi",
+ &dmadev->lldev);
+ if (rc) {
+ failed_desc = desc;
+ break;
+ }
+ }
+
+ if (rc) {
+ /* free allocated MSI interrupts above */
+ for_each_msi_entry(desc, &pdev->dev) {
+ if (desc == failed_desc)
+ break;
+ devm_free_irq(&pdev->dev, desc->irq,
+ &dmadev->lldev);
+ }
+ } else {
+ /* Add callback to free MSIs on teardown */
+ hidma_ll_setup_irq(dmadev->lldev, true);
+
+ }
+ if (rc)
+ dev_warn(&pdev->dev,
+ "failed to request MSI irq, falling back to wired IRQ\n");
+ return rc;
+#else
+ return -EINVAL;
+#endif
+}
+
+static bool hidma_msi_capable(struct device *dev)
+{
+ struct acpi_device *adev = ACPI_COMPANION(dev);
+ const char *of_compat;
+ int ret = -EINVAL;
+
+ if (!adev || acpi_disabled) {
+ ret = device_property_read_string(dev, "compatible",
+ &of_compat);
+ if (ret)
+ return false;
+
+ ret = strcmp(of_compat, "qcom,hidma-1.1");
+ } else {
+#ifdef CONFIG_ACPI
+ ret = strcmp(acpi_device_hid(adev), "QCOM8062");
+#endif
+ }
+ return ret == 0;
+}
+
static int hidma_probe(struct platform_device *pdev)
{
struct hidma_dev *dmadev;
@@ -613,6 +722,7 @@ static int hidma_probe(struct platform_device *pdev)
void __iomem *evca;
void __iomem *trca;
int rc;
+ bool msi;
pm_runtime_set_autosuspend_delay(&pdev->dev, HIDMA_AUTOSUSPEND_TIMEOUT);
pm_runtime_use_autosuspend(&pdev->dev);
@@ -674,6 +784,12 @@ static int hidma_probe(struct platform_device *pdev)
dmadev->ddev.device_terminate_all = hidma_terminate_all;
dmadev->ddev.copy_align = 8;
+ /*
+ * Determine the MSI capability of the platform. Old HW doesn't
+ * support MSI.
+ */
+ msi = hidma_msi_capable(&pdev->dev);
+
device_property_read_u32(&pdev->dev, "desc-count",
&dmadev->nr_descriptors);
@@ -702,10 +818,17 @@ static int hidma_probe(struct platform_device *pdev)
goto dmafree;
}
- rc = devm_request_irq(&pdev->dev, chirq, hidma_chirq_handler, 0,
- "qcom-hidma", dmadev->lldev);
- if (rc)
- goto uninit;
+ platform_set_drvdata(pdev, dmadev);
+ if (msi)
+ rc = hidma_request_msi(dmadev, pdev);
+
+ if (!msi || rc) {
+ hidma_ll_setup_irq(dmadev->lldev, false);
+ rc = devm_request_irq(&pdev->dev, chirq, hidma_chirq_handler,
+ 0, "qcom-hidma", dmadev->lldev);
+ if (rc)
+ goto uninit;
+ }
INIT_LIST_HEAD(&dmadev->ddev.channels);
rc = hidma_chan_init(dmadev, 0);
@@ -721,12 +844,14 @@ static int hidma_probe(struct platform_device *pdev)
hidma_debug_init(dmadev);
hidma_sysfs_init(dmadev);
dev_info(&pdev->dev, "HI-DMA engine driver registration complete\n");
- platform_set_drvdata(pdev, dmadev);
pm_runtime_mark_last_busy(dmadev->ddev.dev);
pm_runtime_put_autosuspend(dmadev->ddev.dev);
return 0;
uninit:
+ if (msi)
+ hidma_free_msis(dmadev);
+
hidma_debug_uninit(dmadev);
hidma_ll_uninit(dmadev->lldev);
dmafree:
@@ -744,7 +869,11 @@ static int hidma_remove(struct platform_device *pdev)
pm_runtime_get_sync(dmadev->ddev.dev);
dma_async_device_unregister(&dmadev->ddev);
- devm_free_irq(dmadev->ddev.dev, dmadev->irq, dmadev->lldev);
+ if (!dmadev->lldev->msi_support)
+ devm_free_irq(dmadev->ddev.dev, dmadev->irq, dmadev->lldev);
+ else
+ hidma_free_msis(dmadev);
+
tasklet_kill(&dmadev->task);
hidma_sysfs_uninit(dmadev);
hidma_debug_uninit(dmadev);
@@ -761,12 +890,14 @@ static int hidma_remove(struct platform_device *pdev)
#if IS_ENABLED(CONFIG_ACPI)
static const struct acpi_device_id hidma_acpi_ids[] = {
{"QCOM8061"},
+ {"QCOM8062"},
{},
};
#endif
static const struct of_device_id hidma_match[] = {
{.compatible = "qcom,hidma-1.0",},
+ {.compatible = "qcom,hidma-1.1",},
{},
};
MODULE_DEVICE_TABLE(of, hidma_match);
diff --git a/drivers/dma/qcom/hidma.h b/drivers/dma/qcom/hidma.h
index 7b40014..c7d0142 100644
--- a/drivers/dma/qcom/hidma.h
+++ b/drivers/dma/qcom/hidma.h
@@ -115,6 +115,7 @@ struct hidma_dev {
int irq;
int chidx;
u32 nr_descriptors;
+ int msi_virqbase;
struct hidma_lldev *lldev;
void __iomem *dev_trca;
@@ -156,6 +157,7 @@ struct hidma_lldev *hidma_ll_init(struct device *dev, u32 max_channels,
u8 chidx);
int hidma_ll_uninit(struct hidma_lldev *llhndl);
irqreturn_t hidma_ll_inthandler(int irq, void *arg);
+irqreturn_t hidma_ll_inthandler_msi(int irq, void *arg, int cause);
void hidma_cleanup_pending_tre(struct hidma_lldev *llhndl, u8 err_info,
u8 err_code);
int hidma_debug_init(struct hidma_dev *dmadev);
diff --git a/drivers/dma/qcom/hidma_ll.c b/drivers/dma/qcom/hidma_ll.c
index 999abda..d5c520f 100644
--- a/drivers/dma/qcom/hidma_ll.c
+++ b/drivers/dma/qcom/hidma_ll.c
@@ -457,6 +457,14 @@ irqreturn_t hidma_ll_inthandler(int chirq, void *arg)
return IRQ_HANDLED;
}
+irqreturn_t hidma_ll_inthandler_msi(int chirq, void *arg, int cause)
+{
+ struct hidma_lldev *lldev = arg;
+
+ hidma_ll_int_handler_internal(lldev, cause);
+ return IRQ_HANDLED;
+}
+
int hidma_ll_enable(struct hidma_lldev *lldev)
{
u32 val;
--
1.9.1
^ permalink raw reply related
* [PATCH V6 09/10] dmaengine: qcom_hidma: break completion processing on error
From: Sinan Kaya @ 2016-10-19 17:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476899512-20431-1-git-send-email-okaya@codeaurora.org>
We try to consume as much successful transfers as possible. Now that we
support MSI interrupts, an error interrupt might be observed by another
processor while we are finishing the successful ones.
Try to abort successful processing if this is the case.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/dma/qcom/hidma_ll.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/dma/qcom/hidma_ll.c b/drivers/dma/qcom/hidma_ll.c
index 62cdd42..999abda 100644
--- a/drivers/dma/qcom/hidma_ll.c
+++ b/drivers/dma/qcom/hidma_ll.c
@@ -291,6 +291,13 @@ static int hidma_handle_tre_completion(struct hidma_lldev *lldev)
evre_write_off =
readl_relaxed(lldev->evca + HIDMA_EVCA_WRITE_PTR_REG);
num_completed++;
+
+ /*
+ * An error interrupt might have arrived while we are processing
+ * the completed interrupt.
+ */
+ if (!hidma_ll_isenabled(lldev))
+ break;
}
if (num_completed) {
--
1.9.1
^ permalink raw reply related
* [PATCH V6 08/10] dmaengine: qcom_hidma: protect common data structures
From: Sinan Kaya @ 2016-10-19 17:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476899512-20431-1-git-send-email-okaya@codeaurora.org>
When MSI interrupts are supported, error and the transfer interrupt can
come from multiple processor contexts.
Each error interrupt is an MSI interrupt. If the channel is disabled by
the first error interrupt, the remaining error interrupts will gracefully
return in the interrupt handler.
If an error is observed while servicing the completions in success case,
the posting of the completions will be aborted as soon as channel disabled
state is observed. The error interrupt handler will take it from there and
finish the remaining completions. We don't want to create multiple success
and error messages to be delivered to the client in mixed order.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/dma/qcom/hidma_ll.c | 44 +++++++++++---------------------------------
1 file changed, 11 insertions(+), 33 deletions(-)
diff --git a/drivers/dma/qcom/hidma_ll.c b/drivers/dma/qcom/hidma_ll.c
index 3168687..62cdd42 100644
--- a/drivers/dma/qcom/hidma_ll.c
+++ b/drivers/dma/qcom/hidma_ll.c
@@ -198,13 +198,16 @@ static void hidma_ll_tre_complete(unsigned long arg)
}
}
-static int hidma_post_completed(struct hidma_lldev *lldev, int tre_iterator,
- u8 err_info, u8 err_code)
+static int hidma_post_completed(struct hidma_lldev *lldev, u8 err_info,
+ u8 err_code)
{
struct hidma_tre *tre;
unsigned long flags;
+ u32 tre_iterator;
spin_lock_irqsave(&lldev->lock, flags);
+
+ tre_iterator = lldev->tre_processed_off;
tre = lldev->pending_tre_list[tre_iterator / HIDMA_TRE_SIZE];
if (!tre) {
spin_unlock_irqrestore(&lldev->lock, flags);
@@ -223,6 +226,9 @@ static int hidma_post_completed(struct hidma_lldev *lldev, int tre_iterator,
atomic_set(&lldev->pending_tre_count, 0);
}
+ HIDMA_INCREMENT_ITERATOR(tre_iterator, HIDMA_TRE_SIZE,
+ lldev->tre_ring_size);
+ lldev->tre_processed_off = tre_iterator;
spin_unlock_irqrestore(&lldev->lock, flags);
tre->err_info = err_info;
@@ -244,13 +250,11 @@ static int hidma_post_completed(struct hidma_lldev *lldev, int tre_iterator,
static int hidma_handle_tre_completion(struct hidma_lldev *lldev)
{
u32 evre_ring_size = lldev->evre_ring_size;
- u32 tre_ring_size = lldev->tre_ring_size;
u32 err_info, err_code, evre_write_off;
- u32 tre_iterator, evre_iterator;
+ u32 evre_iterator;
u32 num_completed = 0;
evre_write_off = readl_relaxed(lldev->evca + HIDMA_EVCA_WRITE_PTR_REG);
- tre_iterator = lldev->tre_processed_off;
evre_iterator = lldev->evre_processed_off;
if ((evre_write_off > evre_ring_size) ||
@@ -273,12 +277,9 @@ static int hidma_handle_tre_completion(struct hidma_lldev *lldev)
err_code =
(cfg >> HIDMA_EVRE_CODE_BIT_POS) & HIDMA_EVRE_CODE_MASK;
- if (hidma_post_completed(lldev, tre_iterator, err_info,
- err_code))
+ if (hidma_post_completed(lldev, err_info, err_code))
break;
- HIDMA_INCREMENT_ITERATOR(tre_iterator, HIDMA_TRE_SIZE,
- tre_ring_size);
HIDMA_INCREMENT_ITERATOR(evre_iterator, HIDMA_EVRE_SIZE,
evre_ring_size);
@@ -295,16 +296,10 @@ static int hidma_handle_tre_completion(struct hidma_lldev *lldev)
if (num_completed) {
u32 evre_read_off = (lldev->evre_processed_off +
HIDMA_EVRE_SIZE * num_completed);
- u32 tre_read_off = (lldev->tre_processed_off +
- HIDMA_TRE_SIZE * num_completed);
-
evre_read_off = evre_read_off % evre_ring_size;
- tre_read_off = tre_read_off % tre_ring_size;
-
writel(evre_read_off, lldev->evca + HIDMA_EVCA_DOORBELL_REG);
/* record the last processed tre offset */
- lldev->tre_processed_off = tre_read_off;
lldev->evre_processed_off = evre_read_off;
}
@@ -314,27 +309,10 @@ static int hidma_handle_tre_completion(struct hidma_lldev *lldev)
void hidma_cleanup_pending_tre(struct hidma_lldev *lldev, u8 err_info,
u8 err_code)
{
- u32 tre_iterator;
- u32 tre_ring_size = lldev->tre_ring_size;
- int num_completed = 0;
- u32 tre_read_off;
-
- tre_iterator = lldev->tre_processed_off;
while (atomic_read(&lldev->pending_tre_count)) {
- if (hidma_post_completed(lldev, tre_iterator, err_info,
- err_code))
+ if (hidma_post_completed(lldev, err_info, err_code))
break;
- HIDMA_INCREMENT_ITERATOR(tre_iterator, HIDMA_TRE_SIZE,
- tre_ring_size);
- num_completed++;
}
- tre_read_off = (lldev->tre_processed_off +
- HIDMA_TRE_SIZE * num_completed);
-
- tre_read_off = tre_read_off % tre_ring_size;
-
- /* record the last processed tre offset */
- lldev->tre_processed_off = tre_read_off;
}
static int hidma_ll_reset(struct hidma_lldev *lldev)
--
1.9.1
^ permalink raw reply related
* [PATCH V6 07/10] dmaengine: qcom_hidma: add a common API to setup the interrupt
From: Sinan Kaya @ 2016-10-19 17:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476899512-20431-1-git-send-email-okaya@codeaurora.org>
Introducing the hidma_ll_setup_irq function to set up the interrupt
type externally from the OS interface.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/dma/qcom/hidma.h | 2 ++
drivers/dma/qcom/hidma_ll.c | 27 +++++++++++++++++++++++----
2 files changed, 25 insertions(+), 4 deletions(-)
diff --git a/drivers/dma/qcom/hidma.h b/drivers/dma/qcom/hidma.h
index 8318de7..7b40014 100644
--- a/drivers/dma/qcom/hidma.h
+++ b/drivers/dma/qcom/hidma.h
@@ -46,6 +46,7 @@ struct hidma_tre {
};
struct hidma_lldev {
+ bool msi_support; /* flag indicating MSI support */
bool initialized; /* initialized flag */
u8 trch_state; /* trch_state of the device */
u8 evch_state; /* evch_state of the device */
@@ -148,6 +149,7 @@ int hidma_ll_request(struct hidma_lldev *llhndl, u32 dev_id,
int hidma_ll_enable(struct hidma_lldev *llhndl);
void hidma_ll_set_transfer_params(struct hidma_lldev *llhndl, u32 tre_ch,
dma_addr_t src, dma_addr_t dest, u32 len, u32 flags);
+void hidma_ll_setup_irq(struct hidma_lldev *lldev, bool msi);
int hidma_ll_setup(struct hidma_lldev *lldev);
struct hidma_lldev *hidma_ll_init(struct device *dev, u32 max_channels,
void __iomem *trca, void __iomem *evca,
diff --git a/drivers/dma/qcom/hidma_ll.c b/drivers/dma/qcom/hidma_ll.c
index 6bdea90..3168687 100644
--- a/drivers/dma/qcom/hidma_ll.c
+++ b/drivers/dma/qcom/hidma_ll.c
@@ -694,17 +694,36 @@ int hidma_ll_setup(struct hidma_lldev *lldev)
writel(HIDMA_EVRE_SIZE * nr_tres,
lldev->evca + HIDMA_EVCA_RING_LEN_REG);
- /* support IRQ only for now */
+ /* configure interrupts */
+ hidma_ll_setup_irq(lldev, lldev->msi_support);
+
+ rc = hidma_ll_enable(lldev);
+ if (rc)
+ return rc;
+
+ return rc;
+}
+
+void hidma_ll_setup_irq(struct hidma_lldev *lldev, bool msi)
+{
+ u32 val;
+
+ lldev->msi_support = msi;
+
+ /* disable interrupts again after reset */
+ writel(0, lldev->evca + HIDMA_EVCA_IRQ_CLR_REG);
+ writel(0, lldev->evca + HIDMA_EVCA_IRQ_EN_REG);
+
+ /* support IRQ by default */
val = readl(lldev->evca + HIDMA_EVCA_INTCTRL_REG);
val &= ~0xF;
- val |= 0x1;
+ if (!lldev->msi_support)
+ val = val | 0x1;
writel(val, lldev->evca + HIDMA_EVCA_INTCTRL_REG);
/* clear all pending interrupts and enable them */
writel(ENABLE_IRQS, lldev->evca + HIDMA_EVCA_IRQ_CLR_REG);
writel(ENABLE_IRQS, lldev->evca + HIDMA_EVCA_IRQ_EN_REG);
-
- return hidma_ll_enable(lldev);
}
struct hidma_lldev *hidma_ll_init(struct device *dev, u32 nr_tres,
--
1.9.1
^ permalink raw reply related
* [PATCH V6 06/10] dmaengine: qcom_hidma: bring out interrupt cause
From: Sinan Kaya @ 2016-10-19 17:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476899512-20431-1-git-send-email-okaya@codeaurora.org>
Bring out the interrupt cause to the top level so that MSI interrupts
can be hooked at a later stage.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/dma/qcom/hidma_ll.c | 62 ++++++++++++++++++++++++---------------------
1 file changed, 33 insertions(+), 29 deletions(-)
diff --git a/drivers/dma/qcom/hidma_ll.c b/drivers/dma/qcom/hidma_ll.c
index 29fef4f..6bdea90 100644
--- a/drivers/dma/qcom/hidma_ll.c
+++ b/drivers/dma/qcom/hidma_ll.c
@@ -411,12 +411,24 @@ static int hidma_ll_reset(struct hidma_lldev *lldev)
* requests traditionally to the destination, this concept does not apply
* here for this HW.
*/
-irqreturn_t hidma_ll_inthandler(int chirq, void *arg)
+static void hidma_ll_int_handler_internal(struct hidma_lldev *lldev, int cause)
{
- struct hidma_lldev *lldev = arg;
- u32 status;
- u32 enable;
- u32 cause;
+ if (cause & HIDMA_ERR_INT_MASK) {
+ dev_err(lldev->dev, "error 0x%x, disabling...\n",
+ cause);
+
+ /* Clear out pending interrupts */
+ writel(cause, lldev->evca + HIDMA_EVCA_IRQ_CLR_REG);
+
+ /* No further submissions. */
+ hidma_ll_disable(lldev);
+
+ /* Driver completes the txn and intimates the client.*/
+ hidma_cleanup_pending_tre(lldev, 0xFF,
+ HIDMA_EVRE_STATUS_ERROR);
+
+ return;
+ }
/*
* Fine tuned for this HW...
@@ -425,35 +437,28 @@ irqreturn_t hidma_ll_inthandler(int chirq, void *arg)
* read and write accessors are used for performance reasons due to
* interrupt delivery guarantees. Do not copy this code blindly and
* expect that to work.
+ *
+ * Try to consume as many EVREs as possible.
*/
+ hidma_handle_tre_completion(lldev);
+
+ /* We consumed TREs or there are pending TREs or EVREs. */
+ writel_relaxed(cause, lldev->evca + HIDMA_EVCA_IRQ_CLR_REG);
+}
+
+irqreturn_t hidma_ll_inthandler(int chirq, void *arg)
+{
+ struct hidma_lldev *lldev = arg;
+ u32 status;
+ u32 enable;
+ u32 cause;
+
status = readl_relaxed(lldev->evca + HIDMA_EVCA_IRQ_STAT_REG);
enable = readl_relaxed(lldev->evca + HIDMA_EVCA_IRQ_EN_REG);
cause = status & enable;
while (cause) {
- if (cause & HIDMA_ERR_INT_MASK) {
- dev_err(lldev->dev, "error 0x%x, disabling...\n",
- cause);
-
- /* Clear out pending interrupts */
- writel(cause, lldev->evca + HIDMA_EVCA_IRQ_CLR_REG);
-
- /* No further submissions. */
- hidma_ll_disable(lldev);
-
- /* Driver completes the txn and intimates the client.*/
- hidma_cleanup_pending_tre(lldev, 0xFF,
- HIDMA_EVRE_STATUS_ERROR);
- goto out;
- }
-
- /*
- * Try to consume as many EVREs as possible.
- */
- hidma_handle_tre_completion(lldev);
-
- /* We consumed TREs or there are pending TREs or EVREs. */
- writel_relaxed(cause, lldev->evca + HIDMA_EVCA_IRQ_CLR_REG);
+ hidma_ll_int_handler_internal(lldev, cause);
/*
* Another interrupt might have arrived while we are
@@ -464,7 +469,6 @@ irqreturn_t hidma_ll_inthandler(int chirq, void *arg)
cause = status & enable;
}
-out:
return IRQ_HANDLED;
}
--
1.9.1
^ permalink raw reply related
* [PATCH V6 05/10] dmaengine: qcom_hidma: make pending_tre_count atomic
From: Sinan Kaya @ 2016-10-19 17:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476899512-20431-1-git-send-email-okaya@codeaurora.org>
Getting ready for the MSI interrupts. The pending_tre_count is used
in the interrupt handler to make sure all outstanding requests are
serviced.
The driver will allocate 11 MSI interrupts. Each MSI interrupt can be
assigned to a different CPU. Then, we have a race condition for common
variables as they share the same interrupt handler with a different
cause bit and they can potentially be executed in parallel. Making this
variable atomic so that it can be updated from multiple processor
contexts.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/dma/qcom/hidma.h | 2 +-
drivers/dma/qcom/hidma_dbg.c | 3 ++-
drivers/dma/qcom/hidma_ll.c | 13 ++++++-------
3 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/drivers/dma/qcom/hidma.h b/drivers/dma/qcom/hidma.h
index b4a512f..8318de7 100644
--- a/drivers/dma/qcom/hidma.h
+++ b/drivers/dma/qcom/hidma.h
@@ -58,7 +58,7 @@ struct hidma_lldev {
void __iomem *evca; /* Event Channel address */
struct hidma_tre
**pending_tre_list; /* Pointers to pending TREs */
- s32 pending_tre_count; /* Number of TREs pending */
+ atomic_t pending_tre_count; /* Number of TREs pending */
void *tre_ring; /* TRE ring */
dma_addr_t tre_dma; /* TRE ring to be shared with HW */
diff --git a/drivers/dma/qcom/hidma_dbg.c b/drivers/dma/qcom/hidma_dbg.c
index 3d83b99..3bdcb80 100644
--- a/drivers/dma/qcom/hidma_dbg.c
+++ b/drivers/dma/qcom/hidma_dbg.c
@@ -74,7 +74,8 @@ static void hidma_ll_devstats(struct seq_file *s, void *llhndl)
seq_printf(s, "tre_ring_handle=%pap\n", &lldev->tre_dma);
seq_printf(s, "tre_ring_size = 0x%x\n", lldev->tre_ring_size);
seq_printf(s, "tre_processed_off = 0x%x\n", lldev->tre_processed_off);
- seq_printf(s, "pending_tre_count=%d\n", lldev->pending_tre_count);
+ seq_printf(s, "pending_tre_count=%d\n",
+ atomic_read(&lldev->pending_tre_count));
seq_printf(s, "evca=%p\n", lldev->evca);
seq_printf(s, "evre_ring=%p\n", lldev->evre_ring);
seq_printf(s, "evre_ring_handle=%pap\n", &lldev->evre_dma);
diff --git a/drivers/dma/qcom/hidma_ll.c b/drivers/dma/qcom/hidma_ll.c
index 3224f24..29fef4f 100644
--- a/drivers/dma/qcom/hidma_ll.c
+++ b/drivers/dma/qcom/hidma_ll.c
@@ -218,10 +218,9 @@ static int hidma_post_completed(struct hidma_lldev *lldev, int tre_iterator,
* Keep track of pending TREs that SW is expecting to receive
* from HW. We got one now. Decrement our counter.
*/
- lldev->pending_tre_count--;
- if (lldev->pending_tre_count < 0) {
+ if (atomic_dec_return(&lldev->pending_tre_count) < 0) {
dev_warn(lldev->dev, "tre count mismatch on completion");
- lldev->pending_tre_count = 0;
+ atomic_set(&lldev->pending_tre_count, 0);
}
spin_unlock_irqrestore(&lldev->lock, flags);
@@ -321,7 +320,7 @@ void hidma_cleanup_pending_tre(struct hidma_lldev *lldev, u8 err_info,
u32 tre_read_off;
tre_iterator = lldev->tre_processed_off;
- while (lldev->pending_tre_count) {
+ while (atomic_read(&lldev->pending_tre_count)) {
if (hidma_post_completed(lldev, tre_iterator, err_info,
err_code))
break;
@@ -548,7 +547,7 @@ void hidma_ll_queue_request(struct hidma_lldev *lldev, u32 tre_ch)
tre->err_code = 0;
tre->err_info = 0;
tre->queued = 1;
- lldev->pending_tre_count++;
+ atomic_inc(&lldev->pending_tre_count);
lldev->tre_write_offset = (lldev->tre_write_offset + HIDMA_TRE_SIZE)
% lldev->tre_ring_size;
spin_unlock_irqrestore(&lldev->lock, flags);
@@ -654,7 +653,7 @@ int hidma_ll_setup(struct hidma_lldev *lldev)
u32 val;
u32 nr_tres = lldev->nr_tres;
- lldev->pending_tre_count = 0;
+ atomic_set(&lldev->pending_tre_count, 0);
lldev->tre_processed_off = 0;
lldev->evre_processed_off = 0;
lldev->tre_write_offset = 0;
@@ -816,7 +815,7 @@ int hidma_ll_uninit(struct hidma_lldev *lldev)
tasklet_kill(&lldev->task);
memset(lldev->trepool, 0, required_bytes);
lldev->trepool = NULL;
- lldev->pending_tre_count = 0;
+ atomic_set(&lldev->pending_tre_count, 0);
lldev->tre_write_offset = 0;
rc = hidma_ll_reset(lldev);
--
1.9.1
^ permalink raw reply related
* [PATCH V6 04/10] dmaengine: qcom_hidma: configure DMA and MSI for OF
From: Sinan Kaya @ 2016-10-19 17:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476899512-20431-1-git-send-email-okaya@codeaurora.org>
Configure the DMA bindings for the device tree based firmware.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/dma/qcom/hidma_mgmt.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/drivers/dma/qcom/hidma_mgmt.c b/drivers/dma/qcom/hidma_mgmt.c
index 82f36e4..185d29c 100644
--- a/drivers/dma/qcom/hidma_mgmt.c
+++ b/drivers/dma/qcom/hidma_mgmt.c
@@ -375,8 +375,15 @@ static int __init hidma_mgmt_of_populate_channels(struct device_node *np)
ret = PTR_ERR(new_pdev);
goto out;
}
+ of_node_get(child);
+ new_pdev->dev.of_node = child;
of_dma_configure(&new_pdev->dev, child);
-
+ /*
+ * It is assumed that calling of_msi_configure is safe on
+ * platforms with or without MSI support.
+ */
+ of_msi_configure(&new_pdev->dev, child);
+ of_node_put(child);
kfree(res);
res = NULL;
}
--
1.9.1
^ permalink raw reply related
* [PATCH V6 03/10] of: irq: make of_msi_configure accessible from modules
From: Sinan Kaya @ 2016-10-19 17:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476899512-20431-1-git-send-email-okaya@codeaurora.org>
The of_msi_configure routine is only accessible by the built-in
kernel drivers. Export this function so that modules can use it
too.
This function is useful for configuring MSI on child device tree
nodes on hierarchical objects.
Acked-by: Rob Herring <robh@kernel.org>
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
drivers/of/irq.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/of/irq.c b/drivers/of/irq.c
index 393fea8..3fda9a3 100644
--- a/drivers/of/irq.c
+++ b/drivers/of/irq.c
@@ -697,3 +697,4 @@ void of_msi_configure(struct device *dev, struct device_node *np)
dev_set_msi_domain(dev,
of_msi_get_domain(dev, np, DOMAIN_BUS_PLATFORM_MSI));
}
+EXPORT_SYMBOL_GPL(of_msi_configure);
--
1.9.1
^ permalink raw reply related
* [PATCH V6 02/10] Documentation: DT: qcom_hidma: correct spelling mistakes
From: Sinan Kaya @ 2016-10-19 17:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476899512-20431-1-git-send-email-okaya@codeaurora.org>
Fix the spelling mistakes and extra and statements in the sentences.
Acked-by: Rob Herring <robh@kernel.org>
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
Documentation/devicetree/bindings/dma/qcom_hidma_mgmt.txt | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Documentation/devicetree/bindings/dma/qcom_hidma_mgmt.txt b/Documentation/devicetree/bindings/dma/qcom_hidma_mgmt.txt
index 2c5e4b8..55492c2 100644
--- a/Documentation/devicetree/bindings/dma/qcom_hidma_mgmt.txt
+++ b/Documentation/devicetree/bindings/dma/qcom_hidma_mgmt.txt
@@ -5,13 +5,13 @@ memcpy and memset capabilities. It has been designed for virtualized
environments.
Each HIDMA HW instance consists of multiple DMA channels. These channels
-share the same bandwidth. The bandwidth utilization can be parititioned
+share the same bandwidth. The bandwidth utilization can be partitioned
among channels based on the priority and weight assignments.
There are only two priority levels and 15 weigh assignments possible.
Other parameters here determine how much of the system bus this HIDMA
-instance can use like maximum read/write request and and number of bytes to
+instance can use like maximum read/write request and number of bytes to
read/write in a single burst.
Main node required properties:
--
1.9.1
^ permalink raw reply related
* [PATCH V6 01/10] Documentation: DT: qcom_hidma: update binding for MSI
From: Sinan Kaya @ 2016-10-19 17:51 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1476899512-20431-1-git-send-email-okaya@codeaurora.org>
Adding a new binding for qcom,hidma-1.1 to distinguish HW supporting
MSI interrupts from the older revision.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
Documentation/devicetree/bindings/dma/qcom_hidma_mgmt.txt | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/Documentation/devicetree/bindings/dma/qcom_hidma_mgmt.txt b/Documentation/devicetree/bindings/dma/qcom_hidma_mgmt.txt
index fd5618b..2c5e4b8 100644
--- a/Documentation/devicetree/bindings/dma/qcom_hidma_mgmt.txt
+++ b/Documentation/devicetree/bindings/dma/qcom_hidma_mgmt.txt
@@ -47,12 +47,18 @@ When the OS is not in control of the management interface (i.e. it's a guest),
the channel nodes appear on their own, not under a management node.
Required properties:
-- compatible: must contain "qcom,hidma-1.0"
+- compatible: must contain "qcom,hidma-1.0" for initial HW or "qcom,hidma-1.1"
+for MSI capable HW.
- reg: Addresses for the transfer and event channel
- interrupts: Should contain the event interrupt
- desc-count: Number of asynchronous requests this channel can handle
- iommus: required a iommu node
+Optional properties for MSI:
+- msi-parent : See the generic MSI binding described in
+ devicetree/bindings/interrupt-controller/msi.txt for a description of the
+ msi-parent property.
+
Example:
Hypervisor OS configuration:
--
1.9.1
^ permalink raw reply related
* [PATCH V6 00/10] dmaengine: qcom_hidma: add MSI interrupt support
From: Sinan Kaya @ 2016-10-19 17:51 UTC (permalink / raw)
To: linux-arm-kernel
The new version of the HW supports MSI interrupts instead of wired
interrupts. The MSI interrupts are especially useful for the guest machine
execution. The wired interrupts usually trap to the hypervisor and then are
relayed to the actual interrupt.
The MSI interrupts can be directly fed into the interrupt controller.
Adding a new OF compat string (qcom,hidma-1.1) and ACPI string (QCOM8062)
to distinguish newer HW from the older ones.
v6:
* rebase 4.9 kernel
v5:
http://www.spinics.net/lists/arm-kernel/msg537014.html
* dmaengine: qcom_hidma: add MSI support for interrupts
** Return MSI interrupts before calling platform_msi_domain_free_irqs.
Also cleanup MSI interrupts on the error path.
** Free the legacy IRQ only if MSI is disabled
* add dmaengine: qcom_hidma: break completion processing on error
in order to break the completions if an error is observed while servicing
completed work.
* drop dmaengine: qcom_hidma: make error and success path common
as the success path assumes that we'll get the number of notifications for
the
jobs queued. This is not true under error conditions.
* simplify dmaengine: qcom_hidma: protect common data structures. We just
need to protect the TRE processed offset. It is the variable that keeps
track
of outstanding requests.
v4:
http://www.spinics.net/lists/devicetree/msg144563.html
* device tree binding update to refer to msi.txt
v3:
* day 0 fix for when ACPI is not compiled in
* https://www.spinics.net/lists/arm-kernel/msg532179.html
v2:
https://patchwork.kernel.org/patch/9326399/
* Documentation update for DT bindings
* Rebased to slave-next
* Dropped dmaengine: qcom_hidma: eliminate processed variables. Replaced it
with dmaengine: qcom_hidma: protect common data structures
v1:
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/444167.html
* initial implementation
Sinan Kaya (10):
Documentation: DT: qcom_hidma: update binding for MSI
Documentation: DT: qcom_hidma: correct spelling mistakes
of: irq: make of_msi_configure accessible from modules
dmaengine: qcom_hidma: configure DMA and MSI for OF
dmaengine: qcom_hidma: make pending_tre_count atomic
dmaengine: qcom_hidma: bring out interrupt cause
dmaengine: qcom_hidma: add a common API to setup the interrupt
dmaengine: qcom_hidma: protect common data structures
dmaengine: qcom_hidma: break completion processing on error
dmaengine: qcom_hidma: add MSI support for interrupts
.../devicetree/bindings/dma/qcom_hidma_mgmt.txt | 12 +-
drivers/dma/qcom/hidma.c | 143 +++++++++++++++++-
drivers/dma/qcom/hidma.h | 6 +-
drivers/dma/qcom/hidma_dbg.c | 3 +-
drivers/dma/qcom/hidma_ll.c | 161 +++++++++++----------
drivers/dma/qcom/hidma_mgmt.c | 9 +-
drivers/of/irq.c | 1 +
7 files changed, 250 insertions(+), 85 deletions(-)
--
1.9.1
^ permalink raw reply
* [PATCH V5 00/10] dmaengine: qcom_hidma: add MSI interrupt support
From: Sinan Kaya @ 2016-10-19 17:41 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20161019133406.GO2467@localhost>
Hi Vinod,
On 10/19/2016 6:34 AM, Vinod Koul wrote:
> On Fri, Oct 07, 2016 at 01:25:05AM -0400, Sinan Kaya wrote:
>> The new version of the HW supports MSI interrupts instead of wired
>> interrupts. The MSI interrupts are especially useful for the guest machine
>> execution. The wired interrupts usually trap to the hypervisor and then are
>> relayed to the actual interrupt.
>>
>> The MSI interrupts can be directly fed into the interrupt controller.
>>
>> Adding a new OF compat string (qcom,hidma-1.1) and ACPI string (QCOM8062)
>> to distinguish newer HW from the older ones.
>
> I was only able to apply 6 patches in this series. Which tree were these
> generated against?
>
> Please rebase rest..
>
I'll post V6 now with the remaining 4 patches. My tree is as follows.
I'll also repost V2 of the "dmaengine: qcom_hidma: cleanup sysfs entries during remove"
patch with your suggestion.
b8faa2a dmaengine: qcom_hidma: add MSI support for interrupts
54043eb dmaengine: qcom_hidma: break completion processing on error
31c9e2c dmaengine: qcom_hidma: protect common data structures
252ef1f dmaengine: qcom_hidma: add a common API to setup the interrupt
4e6c5ce dmaengine: qcom_hidma: bring out interrupt cause
784851a dmaengine: qcom_hidma: make pending_tre_count atomic
098685b dmaengine: qcom_hidma: configure DMA and MSI for OF
79f97f0 of: irq: make of_msi_configure accessible from modules
81e9b3a Documentation: DT: qcom_hidma: correct spelling mistakes
c55ceac Documentation: DT: qcom_hidma: update binding for MSI
7f3470c dmaengine: qcom_hidma: cleanup sysfs entries during remove
78e5299 dmaengine: qcom_hidma: remove useless debugfs file removal
96633c0 ACPI: platform: setup MSI domain for ACPI based platform device
851aadc irqchip: gicv3-its: platform-msi: scan MADT to create platform msi domain
5a8622e irqchip: gicv3-its: platform-msi: refactor its_pmsi_init() to prepare for ACPI
ca80550 ACPI: platform-msi: retrieve dev id from IORT
b631797 irqchip: gicv3-its: platform-msi: refactor its_pmsi_prepare()
1001354 Linux 4.9-rc1
Sinan
--
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox