Re: [PATCH -tip 2/5] x86: use asm-generic/dma-mapping-common.h

Re: [PATCH -tip 2/5] x86: use asm-generic/dma-mapping-common.h

[Pekka Enberg]

On Mon, 2009-06-22 at 14:10 +0100, Catalin Marinas wrote:
> On Mon, 2009-06-22 at 14:49 +0200, Ingo Molnar wrote:
> > * Vegard Nossum <vegard.nossum@gmail.com> wrote:
> > > Seems to be CONFIG_DEBUG_SLAB=y that is the culprit in this case. Hm,
> > > is Kconfig busted?
> > > 
> > > lib/Kconfig.debug:301:config DEBUG_SLAB
> > > lib/Kconfig.debug-302-  bool "Debug slab memory allocations"
> > > lib/Kconfig.debug:303:  depends on DEBUG_KERNEL && SLAB && !KMEMCHECK
> > > 
> > > fujita-config:1475:CONFIG_DEBUG_SLAB=y
> > > fujita-config:1558:CONFIG_KMEMCHECK=y
> > > 
> > > ...what gives? Pekka?
> > 
> > Kmemleak introduced this piece of not so nice solution recently:
> > 
> > +config DEBUG_KMEMLEAK
> > +       bool "Kernel memory leak detector"
> > +       depends on DEBUG_KERNEL && EXPERIMENTAL && (X86 || ARM) && \
> > +               !MEMORY_HOTPLUG
> > +       select DEBUG_SLAB if SLAB
> > +       select SLUB_DEBUG if SLUB
> > +       select DEBUG_FS if SYSFS
> > +       select STACKTRACE if STACKTRACE_SUPPORT
> > +       select KALLSYMS
> > 
> > that should be a depends line, not a select line.
> 
> Kmemleak doesn't strictly need DEBUG_SLAB to make it a dependency. But
> enabling it may reduce (in theory) the false negatives by poisoning the
> allocated objects (and hence clearing any possible pointers to other
> objects). But I don't have any figures to show this is the case. I'll
> post a patch to drop those selects.
> 
> BTW, wouldn't it be feasible for kbuild to ignore the select statements
> if the selected config has unmet dependencies?

Hmm, no idea, lets cc some relevant people here. But can we remove the
select and add a config option help text to kmemleak as a short-term
solution?

			Pekka

Re: [PATCH -tip 2/5] x86: use asm-generic/dma-mapping-common.h

[Catalin Marinas]

On Mon, 2009-06-22 at 16:22 +0300, Pekka Enberg wrote:
> On Mon, 2009-06-22 at 14:10 +0100, Catalin Marinas wrote:
> > On Mon, 2009-06-22 at 14:49 +0200, Ingo Molnar wrote:
> > > * Vegard Nossum <vegard.nossum@gmail.com> wrote:
> > > > Seems to be CONFIG_DEBUG_SLAB=y that is the culprit in this case. Hm,
> > > > is Kconfig busted?
> > > > 
> > > > lib/Kconfig.debug:301:config DEBUG_SLAB
> > > > lib/Kconfig.debug-302-  bool "Debug slab memory allocations"
> > > > lib/Kconfig.debug:303:  depends on DEBUG_KERNEL && SLAB && !KMEMCHECK
> > > > 
> > > > fujita-config:1475:CONFIG_DEBUG_SLAB=y
> > > > fujita-config:1558:CONFIG_KMEMCHECK=y
> > > > 
> > > > ...what gives? Pekka?
> > > 
> > > Kmemleak introduced this piece of not so nice solution recently:
> > > 
> > > +config DEBUG_KMEMLEAK
> > > +       bool "Kernel memory leak detector"
> > > +       depends on DEBUG_KERNEL && EXPERIMENTAL && (X86 || ARM) && \
> > > +               !MEMORY_HOTPLUG
> > > +       select DEBUG_SLAB if SLAB
> > > +       select SLUB_DEBUG if SLUB
> > > +       select DEBUG_FS if SYSFS
> > > +       select STACKTRACE if STACKTRACE_SUPPORT
> > > +       select KALLSYMS
> > > 
> > > that should be a depends line, not a select line.
> > 
> > Kmemleak doesn't strictly need DEBUG_SLAB to make it a dependency. But
> > enabling it may reduce (in theory) the false negatives by poisoning the
> > allocated objects (and hence clearing any possible pointers to other
> > objects). But I don't have any figures to show this is the case. I'll
> > post a patch to drop those selects.
> > 
> > BTW, wouldn't it be feasible for kbuild to ignore the select statements
> > if the selected config has unmet dependencies?
> 
> Hmm, no idea, lets cc some relevant people here. But can we remove the
> select and add a config option help text to kmemleak as a short-term
> solution?

Here it is:


kmemleak: Do not force the slab debugging Kconfig options

From: Catalin Marinas <catalin.marinas@arm.com>

Selecting DEBUG_SLAB or SLUB_DEBUG by the KMEMLEAK menu entry may cause
issues with other dependencies (KMEMCHECK). These configuration options
aren't strictly needed by kmemleak but they may increase the chances of
finding leaks. This patch also updates the KMEMLEAK config entry help
text.

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
---
 lib/Kconfig.debug |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 23067ab..4c32b1a 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -340,8 +340,6 @@ config DEBUG_KMEMLEAK
 	bool "Kernel memory leak detector"
 	depends on DEBUG_KERNEL && EXPERIMENTAL && (X86 || ARM) && \
 		!MEMORY_HOTPLUG
-	select DEBUG_SLAB if SLAB
-	select SLUB_DEBUG if SLUB
 	select DEBUG_FS if SYSFS
 	select STACKTRACE if STACKTRACE_SUPPORT
 	select KALLSYMS
@@ -355,6 +353,9 @@ config DEBUG_KMEMLEAK
 	  allocations. See Documentation/kmemleak.txt for more
 	  details.
 
+	  Enabling DEBUG_SLAB or SLUB_DEBUG may increase the chances
+	  of finding leaks due to the slab objects poisoning.
+
 	  In order to access the kmemleak file, debugfs needs to be
 	  mounted (usually at /sys/kernel/debug).
 

-- 
Catalin

Re: [PATCH -tip 2/5] x86: use asm-generic/dma-mapping-common.h

[Pekka Enberg]

Catalin Marinas wrote:
> kmemleak: Do not force the slab debugging Kconfig options
> 
> From: Catalin Marinas <catalin.marinas@arm.com>
> 
> Selecting DEBUG_SLAB or SLUB_DEBUG by the KMEMLEAK menu entry may cause
> issues with other dependencies (KMEMCHECK). These configuration options
> aren't strictly needed by kmemleak but they may increase the chances of
> finding leaks. This patch also updates the KMEMLEAK config entry help
> text.
> 
> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>

Acked-by: Pekka Enberg <penberg@cs.helsinki.fi>

[PATCH -tip v2 0/5] add common functions for struct dma_map_ops

[FUJITA Tomonori]

We unified x86 and IA64's handling of multiple dma mapping operations
(struct dma_map_ops in linux/dma-mapping.h) so we can remove
duplication in their arch/include/asm/dma-mapping.h.

This patchset adds include/asm-generic/dma-mapping-common.h that
provides some generic dma mapping function definitions for the users
of struct dma_map_ops. This enables us to remove about 100 lines. This
also enables us to easily add CONFIG_DMA_API_DEBUG support, which only
x86 supports for now. The 4th patch adds CONFIG_DMA_API_DEBUG support
to IA64 by adding only 8 lines.

This is against tip/master since tip has some changes to
arch/x86/include/asm/dma-mapping.h.

The changes since the first version are

- fixed a bug that dma_attrs is not passed properly (thanks to Arnd
Bergmann).

- used a new name, dma-mapping-common.h, instead of dma-mapping.h
(suggested by Arnd Bergmann).

- added Joerg's Acked-by

=
 arch/ia64/Kconfig                        |    1 +
 arch/ia64/include/asm/dma-mapping.h      |  110 ++----------------
 arch/x86/Kconfig                         |    1 +
 arch/x86/include/asm/dma-mapping.h       |  174 +---------------------------
 include/asm-generic/dma-mapping-common.h |  190 ++++++++++++++++++++++++++++++
 lib/dma-debug.c                          |   82 ++++++++------
 6 files changed, 252 insertions(+), 306 deletions(-)

[PATCH -tip 2/5] x86: use asm-generic/dma-mapping-common.h

[FUJITA Tomonori]

Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: Joerg Roedel <joerg.roedel@amd.com>
---
 arch/x86/Kconfig                   |    1 +
 arch/x86/include/asm/dma-mapping.h |  174 +-----------------------------------
 2 files changed, 3 insertions(+), 172 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 5f50179..666cb29 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -47,6 +47,7 @@ config X86
 	select HAVE_KERNEL_BZIP2
 	select HAVE_KERNEL_LZMA
 	select HAVE_ARCH_KMEMCHECK
+	select HAVE_DMA_ATTRS
 
 config OUTPUT_FORMAT
 	string
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 916cbb6..1c3f943 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -33,6 +33,8 @@ static inline struct dma_map_ops *get_dma_ops(struct device *dev)
 #endif
 }
 
+#include <asm-generic/dma-mapping-common.h>
+
 /* Make sure we keep the same behaviour */
 static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
 {
@@ -53,178 +55,6 @@ extern int dma_set_mask(struct device *dev, u64 mask);
 extern void *dma_generic_alloc_coherent(struct device *dev, size_t size,
 					dma_addr_t *dma_addr, gfp_t flag);
 
-static inline dma_addr_t
-dma_map_single(struct device *hwdev, void *ptr, size_t size,
-	       enum dma_data_direction dir)
-{
-	struct dma_map_ops *ops = get_dma_ops(hwdev);
-	dma_addr_t addr;
-
-	kmemcheck_mark_initialized(ptr, size);
-	BUG_ON(!valid_dma_direction(dir));
-	addr = ops->map_page(hwdev, virt_to_page(ptr),
-			     (unsigned long)ptr & ~PAGE_MASK, size,
-			     dir, NULL);
-	debug_dma_map_page(hwdev, virt_to_page(ptr),
-			   (unsigned long)ptr & ~PAGE_MASK, size,
-			   dir, addr, true);
-	return addr;
-}
-
-static inline void
-dma_unmap_single(struct device *dev, dma_addr_t addr, size_t size,
-		 enum dma_data_direction dir)
-{
-	struct dma_map_ops *ops = get_dma_ops(dev);
-
-	BUG_ON(!valid_dma_direction(dir));
-	if (ops->unmap_page)
-		ops->unmap_page(dev, addr, size, dir, NULL);
-	debug_dma_unmap_page(dev, addr, size, dir, true);
-}
-
-static inline int
-dma_map_sg(struct device *hwdev, struct scatterlist *sg,
-	   int nents, enum dma_data_direction dir)
-{
-	struct dma_map_ops *ops = get_dma_ops(hwdev);
-	int ents;
-
-	struct scatterlist *s;
-	int i;
-
-	for_each_sg(sg, s, nents, i)
-		kmemcheck_mark_initialized(sg_virt(s), s->length);
-	BUG_ON(!valid_dma_direction(dir));
-	ents = ops->map_sg(hwdev, sg, nents, dir, NULL);
-	debug_dma_map_sg(hwdev, sg, nents, ents, dir);
-
-	return ents;
-}
-
-static inline void
-dma_unmap_sg(struct device *hwdev, struct scatterlist *sg, int nents,
-	     enum dma_data_direction dir)
-{
-	struct dma_map_ops *ops = get_dma_ops(hwdev);
-
-	BUG_ON(!valid_dma_direction(dir));
-	debug_dma_unmap_sg(hwdev, sg, nents, dir);
-	if (ops->unmap_sg)
-		ops->unmap_sg(hwdev, sg, nents, dir, NULL);
-}
-
-static inline void
-dma_sync_single_for_cpu(struct device *hwdev, dma_addr_t dma_handle,
-			size_t size, enum dma_data_direction dir)
-{
-	struct dma_map_ops *ops = get_dma_ops(hwdev);
-
-	BUG_ON(!valid_dma_direction(dir));
-	if (ops->sync_single_for_cpu)
-		ops->sync_single_for_cpu(hwdev, dma_handle, size, dir);
-	debug_dma_sync_single_for_cpu(hwdev, dma_handle, size, dir);
-	flush_write_buffers();
-}
-
-static inline void
-dma_sync_single_for_device(struct device *hwdev, dma_addr_t dma_handle,
-			   size_t size, enum dma_data_direction dir)
-{
-	struct dma_map_ops *ops = get_dma_ops(hwdev);
-
-	BUG_ON(!valid_dma_direction(dir));
-	if (ops->sync_single_for_device)
-		ops->sync_single_for_device(hwdev, dma_handle, size, dir);
-	debug_dma_sync_single_for_device(hwdev, dma_handle, size, dir);
-	flush_write_buffers();
-}
-
-static inline void
-dma_sync_single_range_for_cpu(struct device *hwdev, dma_addr_t dma_handle,
-			      unsigned long offset, size_t size,
-			      enum dma_data_direction dir)
-{
-	struct dma_map_ops *ops = get_dma_ops(hwdev);
-
-	BUG_ON(!valid_dma_direction(dir));
-	if (ops->sync_single_range_for_cpu)
-		ops->sync_single_range_for_cpu(hwdev, dma_handle, offset,
-					       size, dir);
-	debug_dma_sync_single_range_for_cpu(hwdev, dma_handle,
-					    offset, size, dir);
-	flush_write_buffers();
-}
-
-static inline void
-dma_sync_single_range_for_device(struct device *hwdev, dma_addr_t dma_handle,
-				 unsigned long offset, size_t size,
-				 enum dma_data_direction dir)
-{
-	struct dma_map_ops *ops = get_dma_ops(hwdev);
-
-	BUG_ON(!valid_dma_direction(dir));
-	if (ops->sync_single_range_for_device)
-		ops->sync_single_range_for_device(hwdev, dma_handle,
-						  offset, size, dir);
-	debug_dma_sync_single_range_for_device(hwdev, dma_handle,
-					       offset, size, dir);
-	flush_write_buffers();
-}
-
-static inline void
-dma_sync_sg_for_cpu(struct device *hwdev, struct scatterlist *sg,
-		    int nelems, enum dma_data_direction dir)
-{
-	struct dma_map_ops *ops = get_dma_ops(hwdev);
-
-	BUG_ON(!valid_dma_direction(dir));
-	if (ops->sync_sg_for_cpu)
-		ops->sync_sg_for_cpu(hwdev, sg, nelems, dir);
-	debug_dma_sync_sg_for_cpu(hwdev, sg, nelems, dir);
-	flush_write_buffers();
-}
-
-static inline void
-dma_sync_sg_for_device(struct device *hwdev, struct scatterlist *sg,
-		       int nelems, enum dma_data_direction dir)
-{
-	struct dma_map_ops *ops = get_dma_ops(hwdev);
-
-	BUG_ON(!valid_dma_direction(dir));
-	if (ops->sync_sg_for_device)
-		ops->sync_sg_for_device(hwdev, sg, nelems, dir);
-	debug_dma_sync_sg_for_device(hwdev, sg, nelems, dir);
-
-	flush_write_buffers();
-}
-
-static inline dma_addr_t dma_map_page(struct device *dev, struct page *page,
-				      size_t offset, size_t size,
-				      enum dma_data_direction dir)
-{
-	struct dma_map_ops *ops = get_dma_ops(dev);
-	dma_addr_t addr;
-
-	kmemcheck_mark_initialized(page_address(page) + offset, size);
-	BUG_ON(!valid_dma_direction(dir));
-	addr = ops->map_page(dev, page, offset, size, dir, NULL);
-	debug_dma_map_page(dev, page, offset, size, dir, addr, false);
-
-	return addr;
-}
-
-static inline void dma_unmap_page(struct device *dev, dma_addr_t addr,
-				  size_t size, enum dma_data_direction dir)
-{
-	struct dma_map_ops *ops = get_dma_ops(dev);
-
-	BUG_ON(!valid_dma_direction(dir));
-	if (ops->unmap_page)
-		ops->unmap_page(dev, addr, size, dir, NULL);
-	debug_dma_unmap_page(dev, addr, size, dir, false);
-}
-
 static inline void
 dma_cache_sync(struct device *dev, void *vaddr, size_t size,
 	enum dma_data_direction dir)
-- 
1.6.0.6

Re: [PATCH -tip 2/5] x86: use asm-generic/dma-mapping-common.h

[Andrew Morton]

On Fri, 15 May 2009 12:55:02 +0900 FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp> wrote:

> Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
> Acked-by: Joerg Roedel <joerg.roedel@amd.com>
>
> ...
>
>  /* Make sure we keep the same behaviour */
>  static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
>  {
> @@ -53,178 +55,6 @@ extern int dma_set_mask(struct device *dev, u64 mask);
>  extern void *dma_generic_alloc_coherent(struct device *dev, size_t size,
>  					dma_addr_t *dma_addr, gfp_t flag);
>  
> -static inline dma_addr_t
> -dma_map_single(struct device *hwdev, void *ptr, size_t size,
> -	       enum dma_data_direction dir)
> -{
> -	struct dma_map_ops *ops = get_dma_ops(hwdev);
> -	dma_addr_t addr;
> -
> -	kmemcheck_mark_initialized(ptr, size);

This patch attempts to delete kmemcheck stuff which isn't presently in
linux-next.

So I went ahead and ported the patch, but it'll break if/when kmemcheck
returns.

Re: [PATCH -tip 2/5] x86: use asm-generic/dma-mapping-common.h

[FUJITA Tomonori]

On Thu, 28 May 2009 00:20:32 -0700
Andrew Morton <akpm@linux-foundation.org> wrote:

> On Fri, 15 May 2009 12:55:02 +0900 FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp> wrote:
> 
> > Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
> > Acked-by: Joerg Roedel <joerg.roedel@amd.com>
> >
> > ...
> >
> >  /* Make sure we keep the same behaviour */
> >  static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
> >  {
> > @@ -53,178 +55,6 @@ extern int dma_set_mask(struct device *dev, u64 mask);
> >  extern void *dma_generic_alloc_coherent(struct device *dev, size_t size,
> >  					dma_addr_t *dma_addr, gfp_t flag);
> >  
> > -static inline dma_addr_t
> > -dma_map_single(struct device *hwdev, void *ptr, size_t size,
> > -	       enum dma_data_direction dir)
> > -{
> > -	struct dma_map_ops *ops = get_dma_ops(hwdev);
> > -	dma_addr_t addr;
> > -
> > -	kmemcheck_mark_initialized(ptr, size);
> 
> This patch attempts to delete kmemcheck stuff which isn't presently in
> linux-next.

kmemcheck_mark stuff is in the tip tree (this patchset is against the
tip tree).

I rework on this patchset against -mm if you merge all the patches.


> 
> So I went ahead and ported the patch, but it'll break if/when kmemcheck
> returns.