* Re: [PATCH 21/22] arm64: replace ZONE_DMA with ZONE_DMA32
From: Christoph Hellwig @ 2018-01-10 15:55 UTC (permalink / raw)
To: Robin Murphy
Cc: Christoph Hellwig, iommu, linux-arch, linux-mips, Michal Simek,
linux-ia64, Christian König, x86, linux-kernel,
Konrad Rzeszutek Wilk, Guan Xuetao, linuxppc-dev,
linux-arm-kernel
In-Reply-To: <20180110155517.GA18774@lst.de>
On Wed, Jan 10, 2018 at 04:55:17PM +0100, Christoph Hellwig wrote:
> On Wed, Jan 10, 2018 at 12:58:14PM +0000, Robin Murphy wrote:
> > On 10/01/18 08:09, Christoph Hellwig wrote:
> >> arm64 uses ZONE_DMA for allocations below 32-bits. These days we
> >> name the zone for that ZONE_DMA32, which will allow to use the
> >> dma-direct and generic swiotlb code as-is, so rename it.
> >
> > I do wonder if we could also "upgrade" GFP_DMA to GFP_DMA32 somehow when
> > !ZONE_DMA - there are almost certainly arm64 drivers out there using a
> > combination of GFP_DMA and streaming mappings which will no longer get the
> > guaranteed 32-bit addresses they expect after this. I'm not sure quite how
> > feasible that is, though :/
>
> I can't find anything obvious in the tree. The alternative would be
> to keep ZONE_DMA and set ARCH_ZONE_DMA_BITS.
>
> > That said, I do agree that this is an appropriate change (the legacy of
> > GFP_DMA is obviously horrible), so, provided we get plenty of time to find
> > and fix the fallout when it lands:
> >
> > Reviewed-by: Robin Murphy <robin.murphy@arm.com>
>
> I was hoping to get this into 4.15. What would be proper time to
> fix the fallout?
Err, 4.16 of course.
^ permalink raw reply
* Re: [PATCH 21/22] arm64: replace ZONE_DMA with ZONE_DMA32
From: Christoph Hellwig @ 2018-01-10 15:55 UTC (permalink / raw)
To: Robin Murphy
Cc: Christoph Hellwig, iommu, linux-arch, linux-mips, Michal Simek,
linux-ia64, Christian König, x86, linux-kernel,
Konrad Rzeszutek Wilk, Guan Xuetao, linuxppc-dev,
linux-arm-kernel
In-Reply-To: <0371cef8-d980-96da-9cb5-3609c39be18a@arm.com>
On Wed, Jan 10, 2018 at 12:58:14PM +0000, Robin Murphy wrote:
> On 10/01/18 08:09, Christoph Hellwig wrote:
>> arm64 uses ZONE_DMA for allocations below 32-bits. These days we
>> name the zone for that ZONE_DMA32, which will allow to use the
>> dma-direct and generic swiotlb code as-is, so rename it.
>
> I do wonder if we could also "upgrade" GFP_DMA to GFP_DMA32 somehow when
> !ZONE_DMA - there are almost certainly arm64 drivers out there using a
> combination of GFP_DMA and streaming mappings which will no longer get the
> guaranteed 32-bit addresses they expect after this. I'm not sure quite how
> feasible that is, though :/
I can't find anything obvious in the tree. The alternative would be
to keep ZONE_DMA and set ARCH_ZONE_DMA_BITS.
> That said, I do agree that this is an appropriate change (the legacy of
> GFP_DMA is obviously horrible), so, provided we get plenty of time to find
> and fix the fallout when it lands:
>
> Reviewed-by: Robin Murphy <robin.murphy@arm.com>
I was hoping to get this into 4.15. What would be proper time to
fix the fallout?
^ permalink raw reply
* [PATCH v7 for 4.16 03/11] powerpc: membarrier: Skip memory barrier in switch_mm()
From: Mathieu Desnoyers @ 2018-01-10 15:46 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra
Cc: linux-kernel, linux-api, Andy Lutomirski, Paul E . McKenney,
Boqun Feng, Andrew Hunter, Maged Michael, Avi Kivity,
Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman,
Dave Watson, Thomas Gleixner, H . Peter Anvin, Andrea Parri,
Russell King, Greg Hackmann, Will Deacon, David Sehr,
Linus Torvalds, x86, Mathieu Desnoyers, Alan Stern,
Alexander Viro, Nicholas Piggin, linuxppc-dev, linux-arch
In-Reply-To: <20180110154625.4319-1-mathieu.desnoyers@efficios.com>
Allow PowerPC to skip the full memory barrier in switch_mm(), and
only issue the barrier when scheduling into a task belonging to a
process that has registered to use expedited private.
Threads targeting the same VM but which belong to different thread
groups is a tricky case. It has a few consequences:
It turns out that we cannot rely on get_nr_threads(p) to count the
number of threads using a VM. We can use
(atomic_read(&mm->mm_users) == 1 && get_nr_threads(p) == 1)
instead to skip the synchronize_sched() for cases where the VM only has
a single user, and that user only has a single thread.
It also turns out that we cannot use for_each_thread() to set
thread flags in all threads using a VM, as it only iterates on the
thread group.
Therefore, test the membarrier state variable directly rather than
relying on thread flags. This means
membarrier_register_private_expedited() needs to set the
MEMBARRIER_STATE_PRIVATE_EXPEDITED flag, issue synchronize_sched(), and
only then set MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY which allows
private expedited membarrier commands to succeed.
membarrier_arch_switch_mm() now tests for the
MEMBARRIER_STATE_PRIVATE_EXPEDITED flag.
Changes since v1:
- Use test_ti_thread_flag(next, ...) instead of test_thread_flag() in
powerpc membarrier_arch_sched_in(), given that we want to specifically
check the next thread state.
- Add missing ARCH_HAS_MEMBARRIER_HOOKS in Kconfig.
- Use task_thread_info() to pass thread_info from task to
*_ti_thread_flag().
Changes since v2:
- Move membarrier_arch_sched_in() call to finish_task_switch().
- Check for NULL t->mm in membarrier_arch_fork().
- Use membarrier_sched_in() in generic code, which invokes the
arch-specific membarrier_arch_sched_in(). This fixes allnoconfig
build on PowerPC.
- Move asm/membarrier.h include under CONFIG_MEMBARRIER, fixing
allnoconfig build on PowerPC.
- Build and runtime tested on PowerPC.
Changes since v3:
- Simply rely on copy_mm() to copy the membarrier_private_expedited mm
field on fork.
- powerpc: test thread flag instead of reading
membarrier_private_expedited in membarrier_arch_fork().
- powerpc: skip memory barrier in membarrier_arch_sched_in() if coming
from kernel thread, since mmdrop() implies a full barrier.
- Set membarrier_private_expedited to 1 only after arch registration
code, thus eliminating a race where concurrent commands could succeed
when they should fail if issued concurrently with process
registration.
- Use READ_ONCE() for membarrier_private_expedited field access in
membarrier_private_expedited. Matches WRITE_ONCE() performed in
process registration.
Changes since v4:
- Move powerpc hook from sched_in() to switch_mm(), based on feedback
from Nicholas Piggin.
Changes since v5:
- Rebase on v4.14-rc6.
- Fold "Fix: membarrier: Handle CLONE_VM + !CLONE_THREAD correctly on
powerpc (v2)"
Changes since v6:
- Rename MEMBARRIER_STATE_SWITCH_MM to MEMBARRIER_STATE_PRIVATE_EXPEDITED.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
CC: Boqun Feng <boqun.feng@gmail.com>
CC: Andrew Hunter <ahh@google.com>
CC: Maged Michael <maged.michael@gmail.com>
CC: Avi Kivity <avi@scylladb.com>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: Paul Mackerras <paulus@samba.org>
CC: Michael Ellerman <mpe@ellerman.id.au>
CC: Dave Watson <davejwatson@fb.com>
CC: Alan Stern <stern@rowland.harvard.edu>
CC: Will Deacon <will.deacon@arm.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Ingo Molnar <mingo@redhat.com>
CC: Alexander Viro <viro@zeniv.linux.org.uk>
CC: Nicholas Piggin <npiggin@gmail.com>
CC: linuxppc-dev@lists.ozlabs.org
CC: linux-arch@vger.kernel.org
---
MAINTAINERS | 1 +
arch/powerpc/Kconfig | 1 +
arch/powerpc/include/asm/membarrier.h | 26 ++++++++++++++++++++++++++
arch/powerpc/mm/mmu_context.c | 7 +++++++
include/linux/sched/mm.h | 13 ++++++++++++-
init/Kconfig | 3 +++
kernel/sched/core.c | 10 ----------
kernel/sched/membarrier.c | 8 ++++++++
8 files changed, 58 insertions(+), 11 deletions(-)
create mode 100644 arch/powerpc/include/asm/membarrier.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 95c3fa1f520f..a0ad9fe5735b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8931,6 +8931,7 @@ L: linux-kernel@vger.kernel.org
S: Supported
F: kernel/sched/membarrier.c
F: include/uapi/linux/membarrier.h
+F: arch/powerpc/include/asm/membarrier.h
MEMORY MANAGEMENT
L: linux-mm@kvack.org
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index c51e6ce42e7a..a63adb082c0a 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -140,6 +140,7 @@ config PPC
select ARCH_HAS_FORTIFY_SOURCE
select ARCH_HAS_GCOV_PROFILE_ALL
select ARCH_HAS_PMEM_API if PPC64
+ select ARCH_HAS_MEMBARRIER_HOOKS
select ARCH_HAS_SCALED_CPUTIME if VIRT_CPU_ACCOUNTING_NATIVE
select ARCH_HAS_SG_CHAIN
select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
diff --git a/arch/powerpc/include/asm/membarrier.h b/arch/powerpc/include/asm/membarrier.h
new file mode 100644
index 000000000000..98ff4f1fcf2b
--- /dev/null
+++ b/arch/powerpc/include/asm/membarrier.h
@@ -0,0 +1,26 @@
+#ifndef _ASM_POWERPC_MEMBARRIER_H
+#define _ASM_POWERPC_MEMBARRIER_H
+
+static inline void membarrier_arch_switch_mm(struct mm_struct *prev,
+ struct mm_struct *next,
+ struct task_struct *tsk)
+{
+ /*
+ * Only need the full barrier when switching between processes.
+ * Barrier when switching from kernel to userspace is not
+ * required here, given that it is implied by mmdrop(). Barrier
+ * when switching from userspace to kernel is not needed after
+ * store to rq->curr.
+ */
+ if (likely(!(atomic_read(&next->membarrier_state) &
+ MEMBARRIER_STATE_PRIVATE_EXPEDITED) || !prev))
+ return;
+
+ /*
+ * The membarrier system call requires a full memory barrier
+ * after storing to rq->curr, before going back to user-space.
+ */
+ smp_mb();
+}
+
+#endif /* _ASM_POWERPC_MEMBARRIER_H */
diff --git a/arch/powerpc/mm/mmu_context.c b/arch/powerpc/mm/mmu_context.c
index d60a62bf4fc7..0ab297c4cfad 100644
--- a/arch/powerpc/mm/mmu_context.c
+++ b/arch/powerpc/mm/mmu_context.c
@@ -12,6 +12,7 @@
#include <linux/mm.h>
#include <linux/cpu.h>
+#include <linux/sched/mm.h>
#include <asm/mmu_context.h>
@@ -58,6 +59,10 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
*
* On the read side the barrier is in pte_xchg(), which orders
* the store to the PTE vs the load of mm_cpumask.
+ *
+ * This full barrier is needed by membarrier when switching
+ * between processes after store to rq->curr, before user-space
+ * memory accesses.
*/
smp_mb();
@@ -80,6 +85,8 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
if (new_on_cpu)
radix_kvm_prefetch_workaround(next);
+ else
+ membarrier_arch_switch_mm(prev, next, tsk);
/*
* The actual HW switching method differs between the various
diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h
index 3d49b91b674d..1754396795f6 100644
--- a/include/linux/sched/mm.h
+++ b/include/linux/sched/mm.h
@@ -215,14 +215,25 @@ static inline void memalloc_noreclaim_restore(unsigned int flags)
#ifdef CONFIG_MEMBARRIER
enum {
MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY = (1U << 0),
- MEMBARRIER_STATE_SWITCH_MM = (1U << 1),
+ MEMBARRIER_STATE_PRIVATE_EXPEDITED = (1U << 1),
};
+#ifdef CONFIG_ARCH_HAS_MEMBARRIER_HOOKS
+#include <asm/membarrier.h>
+#endif
+
static inline void membarrier_execve(struct task_struct *t)
{
atomic_set(&t->mm->membarrier_state, 0);
}
#else
+#ifdef CONFIG_ARCH_HAS_MEMBARRIER_HOOKS
+static inline void membarrier_arch_switch_mm(struct mm_struct *prev,
+ struct mm_struct *next,
+ struct task_struct *tsk)
+{
+}
+#endif
static inline void membarrier_execve(struct task_struct *t)
{
}
diff --git a/init/Kconfig b/init/Kconfig
index 690a381adee0..e963c49e28fa 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1404,6 +1404,9 @@ config USERFAULTFD
Enable the userfaultfd() system call that allows to intercept and
handle page faults in userland.
+config ARCH_HAS_MEMBARRIER_HOOKS
+ bool
+
config EMBEDDED
bool "Embedded system"
option allnoconfig_y
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 644fa2e3d993..524b705892db 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2653,16 +2653,6 @@ static struct rq *finish_task_switch(struct task_struct *prev)
prev_state = prev->state;
vtime_task_switch(prev);
perf_event_task_sched_in(prev, current);
- /*
- * The membarrier system call requires a full memory barrier
- * after storing to rq->curr, before going back to user-space.
- *
- * TODO: This smp_mb__after_unlock_lock can go away if PPC end
- * up adding a full barrier to switch_mm(), or we should figure
- * out if a smp_mb__after_unlock_lock is really the proper API
- * to use.
- */
- smp_mb__after_unlock_lock();
finish_lock_switch(rq, prev);
finish_arch_post_lock_switch();
diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c
index 9bcbacba82a8..678577267a9a 100644
--- a/kernel/sched/membarrier.c
+++ b/kernel/sched/membarrier.c
@@ -118,6 +118,14 @@ static void membarrier_register_private_expedited(void)
if (atomic_read(&mm->membarrier_state)
& MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY)
return;
+ atomic_or(MEMBARRIER_STATE_PRIVATE_EXPEDITED, &mm->membarrier_state);
+ if (!(atomic_read(&mm->mm_users) == 1 && get_nr_threads(p) == 1)) {
+ /*
+ * Ensure all future scheduler executions will observe the
+ * new thread flag state for this process.
+ */
+ synchronize_sched();
+ }
atomic_or(MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY,
&mm->membarrier_state);
}
--
2.11.0
^ permalink raw reply related
* Re: [PATCH 10/22] swiotlb: refactor coherent buffer allocation
From: Christoph Hellwig @ 2018-01-10 15:46 UTC (permalink / raw)
To: Robin Murphy
Cc: Christoph Hellwig, iommu, linux-arch, linux-mips, Michal Simek,
linux-ia64, Christian König, x86, linux-kernel,
Konrad Rzeszutek Wilk, Guan Xuetao, linuxppc-dev,
linux-arm-kernel
In-Reply-To: <cecc98cf-2e6a-a7bc-7390-d6dcced038c4@arm.com>
On Wed, Jan 10, 2018 at 12:22:18PM +0000, Robin Murphy wrote:
>> + if (phys_addr == SWIOTLB_MAP_ERROR)
>> + goto out_warn;
>> - /* Confirm address can be DMA'd by device */
>> - if (dev_addr + size - 1 > dma_mask) {
>> - printk("hwdev DMA mask = 0x%016Lx, dev_addr = 0x%016Lx\n",
>> - (unsigned long long)dma_mask,
>> - (unsigned long long)dev_addr);
>> + *dma_handle = swiotlb_phys_to_dma(dev, phys_addr);
>
> nit: this should probably go after the dma_coherent_ok() check (as with the
> original logic).
But the originall logic also needs the dma_addr_t for the
dma_coherent_ok check:
dev_addr = swiotlb_phys_to_dma(hwdev, paddr);
/* Confirm address can be DMA'd by device */
if (dev_addr + size - 1 > dma_mask) {
...
goto err_warn;
}
or do you mean assining to *dma_handle? The dma_handle is not
valid for a failure return, so I don't think this should matter.
>> + if (ret) {
>> + *dma_handle = swiotlb_virt_to_bus(hwdev, ret);
>> + if (dma_coherent_ok(hwdev, *dma_handle, size)) {
>> + memset(ret, 0, size);
>> + return ret;
>> + }
>
> Aren't we leaking the pages here?
Yes, that free_pages got lost somewhere in the rebases, I've added
it back.
^ permalink raw reply
* Re: [PATCH 11/33] dma-mapping: move swiotlb arch helpers to a new header
From: Christoph Hellwig @ 2018-01-10 15:40 UTC (permalink / raw)
To: Robin Murphy
Cc: Christoph Hellwig, iommu, linux-mips, linux-ia64, linux-sh,
sparclinux, Guan Xuetao, linux-arch, linux-s390, linux-c6x-dev,
linux-hexagon, x86, Konrad Rzeszutek Wilk, linux-snps-arc,
linux-m68k, patches, linux-metag, linux-arm-kernel, Michal Simek,
linux-parisc, linux-cris-kernel, linux-kernel, linux-alpha,
linuxppc-dev
In-Reply-To: <b0f90ed0-989f-4dc2-6f86-de8b4e486b55@arm.com>
On Wed, Jan 10, 2018 at 03:31:08PM +0000, Robin Murphy wrote:
> Yes indeed, modulo Vladimir's comments - it does seem prudent to fix the
> obvious off-by-ones as we touch them. I've wanted to do something like this
> for ages, but never got around to it myself.
I've fixed these now in a separate patch. I'm taking this as an
Acked-by.
^ permalink raw reply
* Re: [PATCH 08/22] swiotlb: wire up ->dma_supported in swiotlb_dma_ops
From: Christoph Hellwig @ 2018-01-10 15:35 UTC (permalink / raw)
To: Robin Murphy
Cc: Christoph Hellwig, iommu, linux-arch, linux-mips, Michal Simek,
linux-ia64, Christian König, x86, linux-kernel,
Konrad Rzeszutek Wilk, Guan Xuetao, linuxppc-dev,
linux-arm-kernel
In-Reply-To: <7a058876-08fc-7323-7cb3-fe85116e2ea8@arm.com>
On Wed, Jan 10, 2018 at 12:16:15PM +0000, Robin Murphy wrote:
> On 10/01/18 08:09, Christoph Hellwig wrote:
>> To properly reject too small DMA masks based on the addressability of the
>> bounce buffer.
>
> I reckon this is self-evident enough that it should simply be squashed into
> the previous patch.
x86 didn't wire it up before, so I want a clear blaimpoint for this
change instead of mixing it up.
^ permalink raw reply
* Re: [PATCH 31/33] dma-direct: reject too small dma masks
From: Christoph Hellwig @ 2018-01-10 15:32 UTC (permalink / raw)
To: Robin Murphy
Cc: Christoph Hellwig, iommu, linux-mips, linux-ia64, linux-sh,
sparclinux, Guan Xuetao, linux-arch, linux-s390, linux-c6x-dev,
linux-hexagon, x86, Konrad Rzeszutek Wilk, linux-snps-arc,
linux-m68k, patches, linux-metag, linux-arm-kernel, Michal Simek,
linux-parisc, linux-cris-kernel, linux-kernel, linux-alpha,
linuxppc-dev
In-Reply-To: <0bcca030-a8da-c34a-a905-707986689f33@arm.com>
On Wed, Jan 10, 2018 at 11:49:34AM +0000, Robin Murphy wrote:
>> +#ifdef CONFIG_ZONE_DMA
>> + if (mask < DMA_BIT_MASK(ARCH_ZONE_DMA_BITS))
>> + return 0;
>> +#else
>> + /*
>> + * Because 32-bit DMA masks are so common we expect every architecture
>> + * to be able to satisfy them - either by not supporting more physical
>> + * memory, or by providing a ZONE_DMA32. If neither is the case, the
>> + * architecture needs to use an IOMMU instead of the direct mapping.
>> + */
>> + if (mask < DMA_BIT_MASK(32))
>> + return 0;
>
> Do you think it's worth the effort to be a little more accommodating here?
> i.e.:
>
> return dma_max_pfn(dev) >= max_pfn;
>
> We seem to have a fair few 28-31 bit masks for older hardware which
> probably associates with host systems packing equivalently small amounts of
> RAM.
And those devices don't have a ZONE_DMA? I think we could do something
like that, but I'd rather have it as a separate commit with a good
explanation. Maybe you can just send on on top of the series?
^ permalink raw reply
* Re: [PATCH 11/33] dma-mapping: move swiotlb arch helpers to a new header
From: Robin Murphy @ 2018-01-10 15:31 UTC (permalink / raw)
To: Christoph Hellwig
Cc: iommu, linux-mips, linux-ia64, linux-sh, sparclinux, Guan Xuetao,
linux-arch, linux-s390, linux-c6x-dev, linux-hexagon, x86,
Konrad Rzeszutek Wilk, linux-snps-arc, linux-m68k, patches,
linux-metag, linux-arm-kernel, Michal Simek, linux-parisc,
linux-cris-kernel, linux-kernel, linux-alpha, linuxppc-dev
In-Reply-To: <20180110152617.GB17790@lst.de>
On 10/01/18 15:26, Christoph Hellwig wrote:
> On Wed, Jan 10, 2018 at 02:56:01PM +0000, Robin Murphy wrote:
>> I took a look at these, and it seems their phys_to_dma() usage is doing the
>> thing which we subsequently formalised as dma_map_resource(). I've had a
>> crack at a quick patch to update the CESA driver; qcom_nandc looks slightly
>> more complex in that the changes probably need to span the BAM dmaengine
>> driver as well.
>
> Sounds great, although probably something for the next merge window.
>
> In the meantime does this patch looks good to you?
Yes indeed, modulo Vladimir's comments - it does seem prudent to fix the
obvious off-by-ones as we touch them. I've wanted to do something like
this for ages, but never got around to it myself.
Thanks,
Robin.
^ permalink raw reply
* Re: [PATCH 27/33] dma-direct: use node local allocations for coherent memory
From: Christoph Hellwig @ 2018-01-10 15:30 UTC (permalink / raw)
To: Robin Murphy
Cc: Christoph Hellwig, iommu, linux-mips, linux-ia64, linux-sh,
sparclinux, Guan Xuetao, linux-arch, linux-s390, linux-c6x-dev,
linux-hexagon, x86, Konrad Rzeszutek Wilk, linux-snps-arc,
linux-m68k, patches, linux-metag, linux-arm-kernel, Michal Simek,
linux-parisc, linux-cris-kernel, linux-kernel, linux-alpha,
linuxppc-dev
In-Reply-To: <3672aa56-b85c-5d2c-0c0e-709031b0c0a0@arm.com>
On Wed, Jan 10, 2018 at 12:06:22PM +0000, Robin Murphy wrote:
> On 10/01/18 08:00, Christoph Hellwig wrote:
>> To preserve the x86 behavior.
>
> And combined with patch 10/22 of the SWIOTLB refactoring, this means
> SWIOTLB allocations will also end up NUMA-aware, right? Great, that's what
> we want on arm64 too :)
Well, only for swiotlb allocations that can be satisfied by
dma_direct_alloc. If we actually have to fall back to the swiotlb
buffers there is not node affinity yet.
^ permalink raw reply
* Re: [PATCH 20/33] dma-mapping: clear harmful GFP_* flags in common code
From: Christoph Hellwig @ 2018-01-10 15:29 UTC (permalink / raw)
To: Robin Murphy
Cc: Christoph Hellwig, iommu, linux-mips, linux-ia64, linux-sh,
sparclinux, Guan Xuetao, linux-arch, linux-s390, linux-c6x-dev,
linux-hexagon, x86, Konrad Rzeszutek Wilk, linux-snps-arc,
linux-m68k, patches, linux-metag, linux-arm-kernel, Michal Simek,
linux-parisc, linux-cris-kernel, linux-kernel, linux-alpha,
linuxppc-dev
In-Reply-To: <27b90341-f9d0-356f-0194-1c7203a3f93e@arm.com>
On Wed, Jan 10, 2018 at 11:59:30AM +0000, Robin Murphy wrote:
> Just a note that if we're all happy to enshrine the "allocations are always
> zeroed" behaviour in the API (I am too, for the record), we should remember
> to follow up once the dust settles to update the docs and I guess just
> #define dma_zalloc_coherent dma_alloc_coherent.
And then just remove it. But yes, another item for the todo list..
^ permalink raw reply
* Re: [PATCH 11/33] dma-mapping: move swiotlb arch helpers to a new header
From: Christoph Hellwig @ 2018-01-10 15:26 UTC (permalink / raw)
To: Robin Murphy
Cc: Christoph Hellwig, iommu, linux-mips, linux-ia64, linux-sh,
sparclinux, Guan Xuetao, linux-arch, linux-s390, linux-c6x-dev,
linux-hexagon, x86, Konrad Rzeszutek Wilk, linux-snps-arc,
linux-m68k, patches, linux-metag, linux-arm-kernel, Michal Simek,
linux-parisc, linux-cris-kernel, linux-kernel, linux-alpha,
linuxppc-dev
In-Reply-To: <3721b4ba-0685-255e-06b9-6e60678a1a92@arm.com>
On Wed, Jan 10, 2018 at 02:56:01PM +0000, Robin Murphy wrote:
> I took a look at these, and it seems their phys_to_dma() usage is doing the
> thing which we subsequently formalised as dma_map_resource(). I've had a
> crack at a quick patch to update the CESA driver; qcom_nandc looks slightly
> more complex in that the changes probably need to span the BAM dmaengine
> driver as well.
Sounds great, although probably something for the next merge window.
In the meantime does this patch looks good to you?
^ permalink raw reply
* Re: [PATCH 11/33] dma-mapping: move swiotlb arch helpers to a new header
From: Christoph Hellwig @ 2018-01-10 15:22 UTC (permalink / raw)
To: Vladimir Murzin
Cc: Christoph Hellwig, iommu, linux-mips, linux-ia64, linux-sh,
sparclinux, Guan Xuetao, linux-arch, linux-s390, linux-c6x-dev,
linux-hexagon, x86, Konrad Rzeszutek Wilk, linux-snps-arc,
linux-m68k, patches, linux-metag, linux-arm-kernel, Michal Simek,
linux-parisc, linux-cris-kernel, linux-kernel, linux-alpha,
linuxppc-dev
In-Reply-To: <b2bd6f4b-a932-5251-517b-83bbccfe7c53@arm.com>
On Wed, Jan 10, 2018 at 09:31:45AM +0000, Vladimir Murzin wrote:
> I know it is copy&paste, but it seems it has off by one error and it should be
>
> return addr + size - 1 <= *dev->dma_mask;
I've added a new patch to fix the mips dma_capable() definition,
thanks.
^ permalink raw reply
* Re: [PATCH 11/33] dma-mapping: move swiotlb arch helpers to a new header
From: Robin Murphy @ 2018-01-10 14:56 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: linux-mips, linux-ia64, linux-sh, sparclinux, Guan Xuetao,
linux-arch, linux-s390, linux-c6x-dev, linux-hexagon, x86,
Konrad Rzeszutek Wilk, linux-snps-arc, linux-m68k, patches,
linux-metag, linux-arm-kernel, Michal Simek, linux-parisc,
linux-cris-kernel, linux-kernel, linux-alpha, linuxppc-dev
In-Reply-To: <20180110080027.13879-12-hch@lst.de>
On 10/01/18 08:00, Christoph Hellwig wrote:
> phys_to_dma, dma_to_phys and dma_capable are helpers published by
> architecture code for use of swiotlb and xen-swiotlb only. Drivers are
> not supposed to use these directly, but use the DMA API instead.
>
> Move these to a new asm/dma-direct.h helper, included by a
> linux/dma-direct.h wrapper that provides the default linear mapping
> unless the architecture wants to override it.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
[...]
> drivers/crypto/marvell/cesa.c | 1 +
> drivers/mtd/nand/qcom_nandc.c | 1 +
I took a look at these, and it seems their phys_to_dma() usage is doing
the thing which we subsequently formalised as dma_map_resource(). I've
had a crack at a quick patch to update the CESA driver; qcom_nandc looks
slightly more complex in that the changes probably need to span the BAM
dmaengine driver as well.
In the process, though, I stumbled across gen_pool_dma_alloc() - yuck,
something needs doing there, for sure...
Robin.
^ permalink raw reply
* [PATCH] powerpc/xmon: Don't print hashed pointers in paca dump
From: Michael Ellerman @ 2018-01-10 14:19 UTC (permalink / raw)
To: linuxppc-dev
Remember when the biggest problem we had to worry about was hashed
pointers, those were the days.
These were missed in my earlier patch because they don't match "%p",
but the macro is hiding a "%p", so these all end up being hashed,
which is not what we want in xmon. Convert them to "%px".
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/xmon/xmon.c | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
index b3bb5beec54a..0ddc7ac6c5f1 100644
--- a/arch/powerpc/xmon/xmon.c
+++ b/arch/powerpc/xmon/xmon.c
@@ -2344,10 +2344,10 @@ static void dump_one_paca(int cpu)
DUMP(p, kernel_toc, "lx");
DUMP(p, kernelbase, "lx");
DUMP(p, kernel_msr, "lx");
- DUMP(p, emergency_sp, "p");
+ DUMP(p, emergency_sp, "px");
#ifdef CONFIG_PPC_BOOK3S_64
- DUMP(p, nmi_emergency_sp, "p");
- DUMP(p, mc_emergency_sp, "p");
+ DUMP(p, nmi_emergency_sp, "px");
+ DUMP(p, mc_emergency_sp, "px");
DUMP(p, in_nmi, "x");
DUMP(p, in_mce, "x");
DUMP(p, hmi_event_available, "x");
@@ -2382,14 +2382,14 @@ static void dump_one_paca(int cpu)
#endif
DUMP(p, dscr_default, "llx");
#ifdef CONFIG_PPC_BOOK3E
- DUMP(p, pgd, "p");
- DUMP(p, kernel_pgd, "p");
- DUMP(p, tcd_ptr, "p");
- DUMP(p, mc_kstack, "p");
- DUMP(p, crit_kstack, "p");
- DUMP(p, dbg_kstack, "p");
+ DUMP(p, pgd, "px");
+ DUMP(p, kernel_pgd, "px");
+ DUMP(p, tcd_ptr, "px");
+ DUMP(p, mc_kstack, "px");
+ DUMP(p, crit_kstack, "px");
+ DUMP(p, dbg_kstack, "px");
#endif
- DUMP(p, __current, "p");
+ DUMP(p, __current, "px");
DUMP(p, kstack, "lx");
printf(" kstack_base = 0x%016lx\n", p->kstack & ~(THREAD_SIZE - 1));
DUMP(p, stab_rr, "lx");
@@ -2407,7 +2407,7 @@ static void dump_one_paca(int cpu)
#endif
#ifdef CONFIG_PPC_POWERNV
- DUMP(p, core_idle_state_ptr, "p");
+ DUMP(p, core_idle_state_ptr, "px");
DUMP(p, thread_idle_state, "x");
DUMP(p, thread_mask, "x");
DUMP(p, subcore_sibling_mask, "x");
--
2.14.3
^ permalink raw reply related
* Re: [PATCH V6] cxl: Add support for ASB_Notify on POWER9
From: Frederic Barrat @ 2018-01-10 13:44 UTC (permalink / raw)
To: Christophe Lombard, linuxppc-dev, vaibhav, andrew.donnellan
In-Reply-To: <1515513430-23959-1-git-send-email-clombard@linux.vnet.ibm.com>
Hi Christophe,
Couple of comments about file.c, the rest of the patch looks ok to me.
> @@ -193,6 +194,7 @@ static long afu_ioctl_start_work(struct cxl_context *ctx,
> amr = work.amr & mfspr(SPRN_UAMOR);
>
> ctx->mmio_err_ff = !!(work.flags & CXL_START_WORK_ERR_FF);
> + ctx->wflags = work.flags;
I would prefer to add a boolean "need to allocate a TID" to the context
instead of saving the full user land flags, that seems more in-line with
the existing code.
> /*
> * Increment the mapped context count for adapter. This also checks
> @@ -263,6 +265,12 @@ static long afu_ioctl_start_work(struct cxl_context *ctx,
> goto out;
> }
>
> + if (work.flags & CXL_START_WORK_TID) {
> + work.tid = ctx->tidr;
> + if (copy_to_user(uwork, &work, sizeof(work)))
> + return -EFAULT;
> + }
> +
In case of error writing the TID value to user space, we still need to
release the mutex and mark the context as STARTED.
Fred
> ctx->status = STARTED;
> rc = 0;
> out:
> diff --git a/drivers/misc/cxl/native.c b/drivers/misc/cxl/native.c
> index 02b6b45..0cccf27 100644
> --- a/drivers/misc/cxl/native.c
> +++ b/drivers/misc/cxl/native.c
> @@ -16,6 +16,7 @@
> #include <linux/uaccess.h>
> #include <linux/delay.h>
> #include <asm/synch.h>
> +#include <asm/switch_to.h>
> #include <misc/cxl-base.h>
>
> #include "cxl.h"
> @@ -655,6 +656,7 @@ static void update_ivtes_directed(struct cxl_context *ctx)
> static int process_element_entry_psl9(struct cxl_context *ctx, u64 wed, u64 amr)
> {
> u32 pid;
> + int rc;
>
> cxl_assign_psn_space(ctx);
>
> @@ -673,7 +675,16 @@ static int process_element_entry_psl9(struct cxl_context *ctx, u64 wed, u64 amr)
> pid = ctx->mm->context.id;
> }
>
> - ctx->elem->common.tid = 0;
> + /* Assign a unique TIDR (thread id) for the current thread */
> + if (!(ctx->tidr) && (ctx->wflags & CXL_START_WORK_TID)) {
> + rc = set_thread_tidr(current);
> + if (rc)
> + return -ENODEV;
> + ctx->tidr = current->thread.tidr;
> + pr_devel("%s: current tidr: %d\n", __func__, ctx->tidr);
> + }
> +
> + ctx->elem->common.tid = cpu_to_be32(ctx->tidr);
> ctx->elem->common.pid = cpu_to_be32(pid);
>
> ctx->elem->sr = cpu_to_be64(calculate_sr(ctx));
> diff --git a/include/uapi/misc/cxl.h b/include/uapi/misc/cxl.h
> index 49e8fd0..56376d3 100644
> --- a/include/uapi/misc/cxl.h
> +++ b/include/uapi/misc/cxl.h
> @@ -20,20 +20,22 @@ struct cxl_ioctl_start_work {
> __u64 work_element_descriptor;
> __u64 amr;
> __s16 num_interrupts;
> - __s16 reserved1;
> - __s32 reserved2;
> + __u16 tid;
> + __s32 reserved1;
> + __u64 reserved2;
> __u64 reserved3;
> __u64 reserved4;
> __u64 reserved5;
> - __u64 reserved6;
> };
>
> #define CXL_START_WORK_AMR 0x0000000000000001ULL
> #define CXL_START_WORK_NUM_IRQS 0x0000000000000002ULL
> #define CXL_START_WORK_ERR_FF 0x0000000000000004ULL
> +#define CXL_START_WORK_TID 0x0000000000000008ULL
> #define CXL_START_WORK_ALL (CXL_START_WORK_AMR |\
> CXL_START_WORK_NUM_IRQS |\
> - CXL_START_WORK_ERR_FF)
> + CXL_START_WORK_ERR_FF |\
> + CXL_START_WORK_TID)
>
>
> /* Possible modes that an afu can be in */
>
^ permalink raw reply
* [PATCH] powerpc/xmon: Add RFI flush related fields to paca dump
From: Michael Ellerman @ 2018-01-10 13:28 UTC (permalink / raw)
To: linuxppc-dev
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/xmon/xmon.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
index cab24f549e7c..b3bb5beec54a 100644
--- a/arch/powerpc/xmon/xmon.c
+++ b/arch/powerpc/xmon/xmon.c
@@ -2375,6 +2375,10 @@ static void dump_one_paca(int cpu)
DUMP(p, slb_cache_ptr, "x");
for (i = 0; i < SLB_CACHE_ENTRIES; i++)
printf(" slb_cache[%d]: = 0x%016lx\n", i, p->slb_cache[i]);
+
+ DUMP(p, rfi_flush_fallback_area, "px");
+ DUMP(p, l1d_flush_congruence, "llx");
+ DUMP(p, l1d_flush_sets, "llx");
#endif
DUMP(p, dscr_default, "llx");
#ifdef CONFIG_PPC_BOOK3E
--
2.14.3
^ permalink raw reply related
* Re: [PATCH 22/22] arm64: use swiotlb_alloc and swiotlb_free
From: Robin Murphy @ 2018-01-10 13:16 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: linux-arch, linux-mips, Michal Simek, linux-ia64,
Christian König, x86, linux-kernel, Konrad Rzeszutek Wilk,
Guan Xuetao, linuxppc-dev, linux-arm-kernel
In-Reply-To: <20180110080932.14157-23-hch@lst.de>
On 10/01/18 08:09, Christoph Hellwig wrote:
> The generic swiotlb_alloc and swiotlb_free routines already take care
> of CMA allocations and adding GFP_DMA32 where needed, so use them
> instead of the arm specific helpers.
It took a while to satisfy myself that the GFP_DMA(32) handling ends up
equivalent to the current behaviour, but I think it checks out. This
will certainly help with the long-overdue cleanup of this file that I've
had sat around half-finished for ages.
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm64/Kconfig | 1 +
> arch/arm64/mm/dma-mapping.c | 46 +++------------------------------------------
> 2 files changed, 4 insertions(+), 43 deletions(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 6b6985f15d02..53205c02b18a 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -59,6 +59,7 @@ config ARM64
> select COMMON_CLK
> select CPU_PM if (SUSPEND || CPU_IDLE)
> select DCACHE_WORD_ACCESS
> + select DMA_DIRECT_OPS
> select EDAC_SUPPORT
> select FRAME_POINTER
> select GENERIC_ALLOCATOR
> diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
> index 0d641875b20e..a96ec0181818 100644
> --- a/arch/arm64/mm/dma-mapping.c
> +++ b/arch/arm64/mm/dma-mapping.c
> @@ -91,46 +91,6 @@ static int __free_from_pool(void *start, size_t size)
> return 1;
> }
>
> -static void *__dma_alloc_coherent(struct device *dev, size_t size,
> - dma_addr_t *dma_handle, gfp_t flags,
> - unsigned long attrs)
> -{
> - if (IS_ENABLED(CONFIG_ZONE_DMA32) &&
> - dev->coherent_dma_mask <= DMA_BIT_MASK(32))
> - flags |= GFP_DMA32;
> - if (dev_get_cma_area(dev) && gfpflags_allow_blocking(flags)) {
> - struct page *page;
> - void *addr;
> -
> - page = dma_alloc_from_contiguous(dev, size >> PAGE_SHIFT,
> - get_order(size), flags);
> - if (!page)
> - return NULL;
> -
> - *dma_handle = phys_to_dma(dev, page_to_phys(page));
> - addr = page_address(page);
> - memset(addr, 0, size);
> - return addr;
> - } else {
> - return swiotlb_alloc_coherent(dev, size, dma_handle, flags);
> - }
> -}
> -
> -static void __dma_free_coherent(struct device *dev, size_t size,
> - void *vaddr, dma_addr_t dma_handle,
> - unsigned long attrs)
> -{
> - bool freed;
> - phys_addr_t paddr = dma_to_phys(dev, dma_handle);
> -
> -
> - freed = dma_release_from_contiguous(dev,
> - phys_to_page(paddr),
> - size >> PAGE_SHIFT);
> - if (!freed)
> - swiotlb_free_coherent(dev, size, vaddr, dma_handle);
> -}
> -
> static void *__dma_alloc(struct device *dev, size_t size,
> dma_addr_t *dma_handle, gfp_t flags,
> unsigned long attrs)
> @@ -152,7 +112,7 @@ static void *__dma_alloc(struct device *dev, size_t size,
> return addr;
> }
>
> - ptr = __dma_alloc_coherent(dev, size, dma_handle, flags, attrs);
> + ptr = swiotlb_alloc(dev, size, dma_handle, flags, attrs);
> if (!ptr)
> goto no_mem;
>
> @@ -173,7 +133,7 @@ static void *__dma_alloc(struct device *dev, size_t size,
> return coherent_ptr;
>
> no_map:
> - __dma_free_coherent(dev, size, ptr, *dma_handle, attrs);
> + swiotlb_free(dev, size, ptr, *dma_handle, attrs);
> no_mem:
> return NULL;
> }
> @@ -191,7 +151,7 @@ static void __dma_free(struct device *dev, size_t size,
> return;
> vunmap(vaddr);
> }
> - __dma_free_coherent(dev, size, swiotlb_addr, dma_handle, attrs);
> + swiotlb_free(dev, size, swiotlb_addr, dma_handle, attrs);
> }
>
> static dma_addr_t __swiotlb_map_page(struct device *dev, struct page *page,
>
^ permalink raw reply
* Re: [PATCH 21/22] arm64: replace ZONE_DMA with ZONE_DMA32
From: Robin Murphy @ 2018-01-10 12:58 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: linux-arch, linux-mips, Michal Simek, linux-ia64,
Christian König, x86, linux-kernel, Konrad Rzeszutek Wilk,
Guan Xuetao, linuxppc-dev, linux-arm-kernel
In-Reply-To: <20180110080932.14157-22-hch@lst.de>
On 10/01/18 08:09, Christoph Hellwig wrote:
> arm64 uses ZONE_DMA for allocations below 32-bits. These days we
> name the zone for that ZONE_DMA32, which will allow to use the
> dma-direct and generic swiotlb code as-is, so rename it.
I do wonder if we could also "upgrade" GFP_DMA to GFP_DMA32 somehow when
!ZONE_DMA - there are almost certainly arm64 drivers out there using a
combination of GFP_DMA and streaming mappings which will no longer get
the guaranteed 32-bit addresses they expect after this. I'm not sure
quite how feasible that is, though :/
That said, I do agree that this is an appropriate change (the legacy of
GFP_DMA is obviously horrible), so, provided we get plenty of time to
find and fix the fallout when it lands:
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Robin.
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm64/Kconfig | 2 +-
> arch/arm64/mm/dma-mapping.c | 6 +++---
> arch/arm64/mm/init.c | 16 ++++++++--------
> 3 files changed, 12 insertions(+), 12 deletions(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index c9a7e9e1414f..6b6985f15d02 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -227,7 +227,7 @@ config GENERIC_CSUM
> config GENERIC_CALIBRATE_DELAY
> def_bool y
>
> -config ZONE_DMA
> +config ZONE_DMA32
> def_bool y
>
> config HAVE_GENERIC_GUP
> diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
> index 6840426bbe77..0d641875b20e 100644
> --- a/arch/arm64/mm/dma-mapping.c
> +++ b/arch/arm64/mm/dma-mapping.c
> @@ -95,9 +95,9 @@ static void *__dma_alloc_coherent(struct device *dev, size_t size,
> dma_addr_t *dma_handle, gfp_t flags,
> unsigned long attrs)
> {
> - if (IS_ENABLED(CONFIG_ZONE_DMA) &&
> + if (IS_ENABLED(CONFIG_ZONE_DMA32) &&
> dev->coherent_dma_mask <= DMA_BIT_MASK(32))
> - flags |= GFP_DMA;
> + flags |= GFP_DMA32;
> if (dev_get_cma_area(dev) && gfpflags_allow_blocking(flags)) {
> struct page *page;
> void *addr;
> @@ -397,7 +397,7 @@ static int __init atomic_pool_init(void)
> page = dma_alloc_from_contiguous(NULL, nr_pages,
> pool_size_order, GFP_KERNEL);
> else
> - page = alloc_pages(GFP_DMA, pool_size_order);
> + page = alloc_pages(GFP_DMA32, pool_size_order);
>
> if (page) {
> int ret;
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index 00e7b900ca41..8f03276443c9 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -217,7 +217,7 @@ static void __init reserve_elfcorehdr(void)
> }
> #endif /* CONFIG_CRASH_DUMP */
> /*
> - * Return the maximum physical address for ZONE_DMA (DMA_BIT_MASK(32)). It
> + * Return the maximum physical address for ZONE_DMA32 (DMA_BIT_MASK(32)). It
> * currently assumes that for memory starting above 4G, 32-bit devices will
> * use a DMA offset.
> */
> @@ -233,8 +233,8 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
> {
> unsigned long max_zone_pfns[MAX_NR_ZONES] = {0};
>
> - if (IS_ENABLED(CONFIG_ZONE_DMA))
> - max_zone_pfns[ZONE_DMA] = PFN_DOWN(max_zone_dma_phys());
> + if (IS_ENABLED(CONFIG_ZONE_DMA32))
> + max_zone_pfns[ZONE_DMA32] = PFN_DOWN(max_zone_dma_phys());
> max_zone_pfns[ZONE_NORMAL] = max;
>
> free_area_init_nodes(max_zone_pfns);
> @@ -251,9 +251,9 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
> memset(zone_size, 0, sizeof(zone_size));
>
> /* 4GB maximum for 32-bit only capable devices */
> -#ifdef CONFIG_ZONE_DMA
> +#ifdef CONFIG_ZONE_DMA32
> max_dma = PFN_DOWN(arm64_dma_phys_limit);
> - zone_size[ZONE_DMA] = max_dma - min;
> + zone_size[ZONE_DMA32] = max_dma - min;
> #endif
> zone_size[ZONE_NORMAL] = max - max_dma;
>
> @@ -266,10 +266,10 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
> if (start >= max)
> continue;
>
> -#ifdef CONFIG_ZONE_DMA
> +#ifdef CONFIG_ZONE_DMA32
> if (start < max_dma) {
> unsigned long dma_end = min(end, max_dma);
> - zhole_size[ZONE_DMA] -= dma_end - start;
> + zhole_size[ZONE_DMA32] -= dma_end - start;
> }
> #endif
> if (end > max_dma) {
> @@ -467,7 +467,7 @@ void __init arm64_memblock_init(void)
> early_init_fdt_scan_reserved_mem();
>
> /* 4GB maximum for 32-bit only capable devices */
> - if (IS_ENABLED(CONFIG_ZONE_DMA))
> + if (IS_ENABLED(CONFIG_ZONE_DMA32))
> arm64_dma_phys_limit = max_zone_dma_phys();
> else
> arm64_dma_phys_limit = PHYS_MASK + 1;
>
^ permalink raw reply
* Re: [PATCH 10/22] swiotlb: refactor coherent buffer allocation
From: Robin Murphy @ 2018-01-10 12:22 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: linux-arch, linux-mips, Michal Simek, linux-ia64,
Christian König, x86, linux-kernel, Konrad Rzeszutek Wilk,
Guan Xuetao, linuxppc-dev, linux-arm-kernel
In-Reply-To: <20180110080932.14157-11-hch@lst.de>
On 10/01/18 08:09, Christoph Hellwig wrote:
> Factor out a new swiotlb_alloc_buffer helper that allocates DMA coherent
> memory from the swiotlb bounce buffer.
>
> This allows to simplify the swiotlb_alloc implemenation that uses
> dma_direct_alloc to try to allocate a reachable buffer first.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> lib/swiotlb.c | 122 +++++++++++++++++++++++++++++++---------------------------
> 1 file changed, 65 insertions(+), 57 deletions(-)
>
> diff --git a/lib/swiotlb.c b/lib/swiotlb.c
> index 1a147f1354a1..bf2d19ee91c1 100644
> --- a/lib/swiotlb.c
> +++ b/lib/swiotlb.c
> @@ -709,75 +709,79 @@ void swiotlb_tbl_sync_single(struct device *hwdev, phys_addr_t tlb_addr,
> }
> EXPORT_SYMBOL_GPL(swiotlb_tbl_sync_single);
>
> -void *
> -swiotlb_alloc_coherent(struct device *hwdev, size_t size,
> - dma_addr_t *dma_handle, gfp_t flags)
> +static inline bool dma_coherent_ok(struct device *dev, dma_addr_t addr,
> + size_t size)
> {
> - bool warn = !(flags & __GFP_NOWARN);
> - dma_addr_t dev_addr;
> - void *ret;
> - int order = get_order(size);
> - u64 dma_mask = DMA_BIT_MASK(32);
> + u64 mask = DMA_BIT_MASK(32);
>
> - if (hwdev && hwdev->coherent_dma_mask)
> - dma_mask = hwdev->coherent_dma_mask;
> + if (dev && dev->coherent_dma_mask)
> + mask = dev->coherent_dma_mask;
> + return addr + size - 1 <= mask;
> +}
>
> - ret = (void *)__get_free_pages(flags, order);
> - if (ret) {
> - dev_addr = swiotlb_virt_to_bus(hwdev, ret);
> - if (dev_addr + size - 1 > dma_mask) {
> - /*
> - * The allocated memory isn't reachable by the device.
> - */
> - free_pages((unsigned long) ret, order);
> - ret = NULL;
> - }
> - }
> - if (!ret) {
> - /*
> - * We are either out of memory or the device can't DMA to
> - * GFP_DMA memory; fall back on map_single(), which
> - * will grab memory from the lowest available address range.
> - */
> - phys_addr_t paddr = map_single(hwdev, 0, size, DMA_FROM_DEVICE,
> - warn ? 0 : DMA_ATTR_NO_WARN);
> - if (paddr == SWIOTLB_MAP_ERROR)
> - goto err_warn;
> +static void *
> +swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle,
> + unsigned long attrs)
> +{
> + phys_addr_t phys_addr;
> +
> + if (swiotlb_force == SWIOTLB_NO_FORCE)
> + goto out_warn;
>
> - ret = phys_to_virt(paddr);
> - dev_addr = swiotlb_phys_to_dma(hwdev, paddr);
> + phys_addr = swiotlb_tbl_map_single(dev,
> + swiotlb_phys_to_dma(dev, io_tlb_start),
> + 0, size, DMA_FROM_DEVICE, 0);
> + if (phys_addr == SWIOTLB_MAP_ERROR)
> + goto out_warn;
>
> - /* Confirm address can be DMA'd by device */
> - if (dev_addr + size - 1 > dma_mask) {
> - printk("hwdev DMA mask = 0x%016Lx, dev_addr = 0x%016Lx\n",
> - (unsigned long long)dma_mask,
> - (unsigned long long)dev_addr);
> + *dma_handle = swiotlb_phys_to_dma(dev, phys_addr);
nit: this should probably go after the dma_coherent_ok() check (as with
the original logic).
>
> - /*
> - * DMA_TO_DEVICE to avoid memcpy in unmap_single.
> - * The DMA_ATTR_SKIP_CPU_SYNC is optional.
> - */
> - swiotlb_tbl_unmap_single(hwdev, paddr,
> - size, DMA_TO_DEVICE,
> - DMA_ATTR_SKIP_CPU_SYNC);
> - goto err_warn;
> - }
> - }
> + if (dma_coherent_ok(dev, *dma_handle, size))
> + goto out_unmap;
>
> - *dma_handle = dev_addr;
> - memset(ret, 0, size);
> + memset(phys_to_virt(phys_addr), 0, size);
> + return phys_to_virt(phys_addr);
>
> - return ret;
> +out_unmap:
> + dev_warn(dev, "hwdev DMA mask = 0x%016Lx, dev_addr = 0x%016Lx\n",
> + (unsigned long long)(dev ? dev->coherent_dma_mask : 0),
> + (unsigned long long)*dma_handle);
>
> -err_warn:
> - if (warn && printk_ratelimit()) {
> - pr_warn("swiotlb: coherent allocation failed for device %s size=%zu\n",
> - dev_name(hwdev), size);
> + /*
> + * DMA_TO_DEVICE to avoid memcpy in unmap_single.
> + * DMA_ATTR_SKIP_CPU_SYNC is optional.
> + */
> + swiotlb_tbl_unmap_single(dev, phys_addr, size, DMA_TO_DEVICE,
> + DMA_ATTR_SKIP_CPU_SYNC);
> +out_warn:
> + if ((attrs & DMA_ATTR_NO_WARN) && printk_ratelimit()) {
> + dev_warn(dev,
> + "swiotlb: coherent allocation failed, size=%zu\n",
> + size);
> dump_stack();
> }
> -
> return NULL;
> }
> +
> +void *
> +swiotlb_alloc_coherent(struct device *hwdev, size_t size,
> + dma_addr_t *dma_handle, gfp_t flags)
> +{
> + int order = get_order(size);
> + unsigned long attrs = (flags & __GFP_NOWARN) ? DMA_ATTR_NO_WARN : 0;
> + void *ret;
> +
> + ret = (void *)__get_free_pages(flags, order);
> + if (ret) {
> + *dma_handle = swiotlb_virt_to_bus(hwdev, ret);
> + if (dma_coherent_ok(hwdev, *dma_handle, size)) {
> + memset(ret, 0, size);
> + return ret;
> + }
Aren't we leaking the pages here?
Robin.
> + }
> +
> + return swiotlb_alloc_buffer(hwdev, size, dma_handle, attrs);
> +}
> EXPORT_SYMBOL(swiotlb_alloc_coherent);
>
> static bool swiotlb_free_buffer(struct device *dev, size_t size,
> @@ -1103,6 +1107,10 @@ void *swiotlb_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
> {
> void *vaddr;
>
> + /* temporary workaround: */
> + if (gfp & __GFP_NOWARN)
> + attrs |= DMA_ATTR_NO_WARN;
> +
> /*
> * Don't print a warning when the first allocation attempt fails.
> * swiotlb_alloc_coherent() will print a warning when the DMA memory
> @@ -1112,7 +1120,7 @@ void *swiotlb_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
>
> vaddr = dma_direct_alloc(dev, size, dma_handle, gfp, attrs);
> if (!vaddr)
> - vaddr = swiotlb_alloc_coherent(dev, size, dma_handle, gfp);
> + vaddr = swiotlb_alloc_buffer(dev, size, dma_handle, attrs);
> return vaddr;
> }
>
>
^ permalink raw reply
* Re: [PATCH 08/22] swiotlb: wire up ->dma_supported in swiotlb_dma_ops
From: Robin Murphy @ 2018-01-10 12:16 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: linux-arch, linux-mips, Michal Simek, linux-ia64,
Christian König, x86, linux-kernel, Konrad Rzeszutek Wilk,
Guan Xuetao, linuxppc-dev, linux-arm-kernel
In-Reply-To: <20180110080932.14157-9-hch@lst.de>
On 10/01/18 08:09, Christoph Hellwig wrote:
> To properly reject too small DMA masks based on the addressability of the
> bounce buffer.
I reckon this is self-evident enough that it should simply be squashed
into the previous patch.
Robin.
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> lib/swiotlb.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/lib/swiotlb.c b/lib/swiotlb.c
> index 0fae2f45c3c0..539fd1099ba9 100644
> --- a/lib/swiotlb.c
> +++ b/lib/swiotlb.c
> @@ -1128,5 +1128,6 @@ const struct dma_map_ops swiotlb_dma_ops = {
> .unmap_sg = swiotlb_unmap_sg_attrs,
> .map_page = swiotlb_map_page,
> .unmap_page = swiotlb_unmap_page,
> + .dma_supported = swiotlb_dma_supported,
> };
> #endif /* CONFIG_DMA_DIRECT_OPS */
>
^ permalink raw reply
* Re: [PATCH 02/22] arm64: rename swiotlb_dma_ops
From: Robin Murphy @ 2018-01-10 12:13 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: linux-arch, linux-mips, Michal Simek, linux-ia64,
Christian König, x86, linux-kernel, Konrad Rzeszutek Wilk,
Guan Xuetao, linuxppc-dev, linux-arm-kernel
In-Reply-To: <20180110080932.14157-3-hch@lst.de>
On 10/01/18 08:09, Christoph Hellwig wrote:
> We'll need that name for a generic implementation soon.
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/arm64/mm/dma-mapping.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
> index f3a637b98487..6840426bbe77 100644
> --- a/arch/arm64/mm/dma-mapping.c
> +++ b/arch/arm64/mm/dma-mapping.c
> @@ -368,7 +368,7 @@ static int __swiotlb_dma_mapping_error(struct device *hwdev, dma_addr_t addr)
> return 0;
> }
>
> -static const struct dma_map_ops swiotlb_dma_ops = {
> +static const struct dma_map_ops arm64_swiotlb_dma_ops = {
> .alloc = __dma_alloc,
> .free = __dma_free,
> .mmap = __swiotlb_mmap,
> @@ -923,7 +923,7 @@ void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
> const struct iommu_ops *iommu, bool coherent)
> {
> if (!dev->dma_ops)
> - dev->dma_ops = &swiotlb_dma_ops;
> + dev->dma_ops = &arm64_swiotlb_dma_ops;
>
> dev->archdata.dma_coherent = coherent;
> __iommu_setup_dma_ops(dev, dma_base, size, iommu);
>
^ permalink raw reply
* Re: [PATCH 27/33] dma-direct: use node local allocations for coherent memory
From: Robin Murphy @ 2018-01-10 12:06 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: linux-mips, linux-ia64, linux-sh, sparclinux, Guan Xuetao,
linux-arch, linux-s390, linux-c6x-dev, linux-hexagon, x86,
Konrad Rzeszutek Wilk, linux-snps-arc, linux-m68k, patches,
linux-metag, linux-arm-kernel, Michal Simek, linux-parisc,
linux-cris-kernel, linux-kernel, linux-alpha, linuxppc-dev
In-Reply-To: <20180110080027.13879-28-hch@lst.de>
On 10/01/18 08:00, Christoph Hellwig wrote:
> To preserve the x86 behavior.
And combined with patch 10/22 of the SWIOTLB refactoring, this means
SWIOTLB allocations will also end up NUMA-aware, right? Great, that's
what we want on arm64 too :)
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> lib/dma-direct.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/dma-direct.c b/lib/dma-direct.c
> index a9ae98be7af3..f04a424f91fa 100644
> --- a/lib/dma-direct.c
> +++ b/lib/dma-direct.c
> @@ -38,7 +38,7 @@ static void *dma_direct_alloc(struct device *dev, size_t size,
> if (gfpflags_allow_blocking(gfp))
> page = dma_alloc_from_contiguous(dev, count, page_order, gfp);
> if (!page)
> - page = alloc_pages(gfp, page_order);
> + page = alloc_pages_node(dev_to_node(dev), gfp, page_order);
> if (!page)
> return NULL;
>
>
^ permalink raw reply
* Re: [v3 PATCH 2/3] powernv-cpufreq: Fix pstate_to_idx() to handle non-continguous pstates
From: Rafael J. Wysocki @ 2018-01-10 12:00 UTC (permalink / raw)
To: ego
Cc: Balbir Singh, Shilpasri G Bhat, Viresh Kumar, Abhishek,
Akshay Adiga, Michael Ellerman, Vaidyanathan Srinivasan, linux-pm,
linux-kernel@vger.kernel.org,
open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)
In-Reply-To: <20180110085545.GA13666@in.ibm.com>
On Wednesday, January 10, 2018 9:55:45 AM CET Gautham R Shenoy wrote:
> Hi Rafael,
>
> On Wed, Jan 03, 2018 at 11:47:58PM +1100, Balbir Singh wrote:
> > On Wed, Jan 3, 2018 at 11:07 PM, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> > > On Monday, December 18, 2017 9:38:20 AM CET Gautham R Shenoy wrote:
> > >> Hi Balbir,
> > >>
> > >> On Sun, Dec 17, 2017 at 02:15:25PM +1100, Balbir Singh wrote:
> > >> > On Wed, Dec 13, 2017 at 5:57 PM, Gautham R. Shenoy
> > >> > <ego@linux.vnet.ibm.com> wrote:
> > >> > > From: "Gautham R. Shenoy" <ego@linux.vnet.ibm.com>
> > >> > >
> > >> > > The code in powernv-cpufreq, makes the following two assumptions which
> > >> > > are not guaranteed by the device-tree bindings:
> > >> > >
> > >> > > 1) Pstate ids are continguous: This is used in pstate_to_idx() to
> > >> > > obtain the reverse map from a pstate to it's corresponding
> > >> > > entry into the cpufreq frequency table.
> > >> > >
> > >> > > 2) Every Pstate should always lie between the max and the min
> > >> > > pstates that are explicitly reported in the device tree: This
> > >> > > is used to determine whether a pstate reported by the PMSR is
> > >> > > out of bounds.
> > >> > >
> > >> > > Both these assumptions are unwarranted and can change on future
> > >> > > platforms.
> > >> >
> > >> > While this is a good thing, I wonder if it is worth the complexity. Pstates
> > >> > are contiguous because they define transitions in incremental value
> > >> > of change in frequency and I can't see how this can be broken in the
> > >> > future?
> > >>
> > >> In the future, we can have the OPAL firmware give us a smaller set of
> > >> pstates instead of expose every one of them. As it stands today, for
> > >> most of the workloads, we will need at best 20-30 pstates and not
> > >> beyond that.
> > >
> > > I'm not sure about the status here.
> > >
> > > Is this good to go as is or is it going to be updated?
> > >
> >
> > I have no major objections, except some of the added complexity, but
> > Gautham makes a point that this is refactoring for the future
>
> I have tested this across POWER8 and POWER9. The additional complexity
> introduced by the second patch is required for the future when we are
> going to reduce the number of pstates.
I have applied the series, thanks!
^ permalink raw reply
* Re: [PATCH 20/33] dma-mapping: clear harmful GFP_* flags in common code
From: Robin Murphy @ 2018-01-10 11:59 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: linux-mips, linux-ia64, linux-sh, sparclinux, Guan Xuetao,
linux-arch, linux-s390, linux-c6x-dev, linux-hexagon, x86,
Konrad Rzeszutek Wilk, linux-snps-arc, linux-m68k, patches,
linux-metag, linux-arm-kernel, Michal Simek, linux-parisc,
linux-cris-kernel, linux-kernel, linux-alpha, linuxppc-dev
In-Reply-To: <20180110080027.13879-21-hch@lst.de>
On 10/01/18 08:00, Christoph Hellwig wrote:
[...]
> diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
> index 9f28b2fa329e..88bcb1a8211d 100644
> --- a/include/linux/dma-mapping.h
> +++ b/include/linux/dma-mapping.h
> @@ -518,6 +518,13 @@ static inline void *dma_alloc_attrs(struct device *dev, size_t size,
> if (dma_alloc_from_dev_coherent(dev, size, dma_handle, &cpu_addr))
> return cpu_addr;
>
> + /*
> + * Let the implementation decide on the zone to allocate from, and
> + * decide on the way of zeroing the memory given that the memory
> + * returned should always be zeroed.
> + */
Just a note that if we're all happy to enshrine the "allocations are
always zeroed" behaviour in the API (I am too, for the record), we
should remember to follow up once the dust settles to update the docs
and I guess just #define dma_zalloc_coherent dma_alloc_coherent.
Robin.
> + flag &= ~(__GFP_DMA | __GFP_DMA32 | __GFP_HIGHMEM | __GFP_ZERO);
> +
> if (!arch_dma_alloc_attrs(&dev, &flag))
> return NULL;
> if (!ops->alloc)
>
^ permalink raw reply
* Re: [PATCH 31/33] dma-direct: reject too small dma masks
From: Robin Murphy @ 2018-01-10 11:49 UTC (permalink / raw)
To: Christoph Hellwig, iommu
Cc: linux-mips, linux-ia64, linux-sh, sparclinux, Guan Xuetao,
linux-arch, linux-s390, linux-c6x-dev, linux-hexagon, x86,
Konrad Rzeszutek Wilk, linux-snps-arc, linux-m68k, patches,
linux-metag, linux-arm-kernel, Michal Simek, linux-parisc,
linux-cris-kernel, linux-kernel, linux-alpha, linuxppc-dev
In-Reply-To: <20180110080027.13879-32-hch@lst.de>
On 10/01/18 08:00, Christoph Hellwig wrote:
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> include/linux/dma-direct.h | 1 +
> lib/dma-direct.c | 19 +++++++++++++++++++
> 2 files changed, 20 insertions(+)
>
> diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
> index 4788bf0bf683..bcdb1a3e4b1f 100644
> --- a/include/linux/dma-direct.h
> +++ b/include/linux/dma-direct.h
> @@ -42,5 +42,6 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
> gfp_t gfp, unsigned long attrs);
> void dma_direct_free(struct device *dev, size_t size, void *cpu_addr,
> dma_addr_t dma_addr, unsigned long attrs);
> +int dma_direct_supported(struct device *dev, u64 mask);
>
> #endif /* _LINUX_DMA_DIRECT_H */
> diff --git a/lib/dma-direct.c b/lib/dma-direct.c
> index 784a68dfdbe3..40b1f92f2214 100644
> --- a/lib/dma-direct.c
> +++ b/lib/dma-direct.c
> @@ -122,6 +122,24 @@ static int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl,
> return nents;
> }
>
> +int dma_direct_supported(struct device *dev, u64 mask)
> +{
> +#ifdef CONFIG_ZONE_DMA
> + if (mask < DMA_BIT_MASK(ARCH_ZONE_DMA_BITS))
> + return 0;
> +#else
> + /*
> + * Because 32-bit DMA masks are so common we expect every architecture
> + * to be able to satisfy them - either by not supporting more physical
> + * memory, or by providing a ZONE_DMA32. If neither is the case, the
> + * architecture needs to use an IOMMU instead of the direct mapping.
> + */
> + if (mask < DMA_BIT_MASK(32))
> + return 0;
Do you think it's worth the effort to be a little more accommodating
here? i.e.:
return dma_max_pfn(dev) >= max_pfn;
We seem to have a fair few 28-31 bit masks for older hardware which
probably associates with host systems packing equivalently small amounts
of RAM.
Otherwise though,
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Robin.
> +#endif
> + return 1;
> +}
> +
> static int dma_direct_mapping_error(struct device *dev, dma_addr_t dma_addr)
> {
> return dma_addr == DIRECT_MAPPING_ERROR;
> @@ -132,6 +150,7 @@ const struct dma_map_ops dma_direct_ops = {
> .free = dma_direct_free,
> .map_page = dma_direct_map_page,
> .map_sg = dma_direct_map_sg,
> + .dma_supported = dma_direct_supported,
> .mapping_error = dma_direct_mapping_error,
> };
> EXPORT_SYMBOL(dma_direct_ops);
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox