All of lore.kernel.org
 help / color / mirror / Atom feed
From: Konrad Rzeszutek Wilk <konrad.wilk-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
To: Dorian Gray <yourfavouritegod-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Cc: USB list <linux-usb-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	Kernel development list
	<linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	Alexander Duyck
	<alexander.duyck-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org,
	Alan Stern
	<stern-nwvwT67g6+6dFdvTe/nMLpVzexx5G7lz@public.gmane.org>
Subject: Re: Error: DMA: Out of SW-IOMMU space [was: External USB drives become unresponsive after few hours.]
Date: Thu, 16 Apr 2015 14:42:52 -0400	[thread overview]
Message-ID: <20150416184252.GE7388@x230.dumpdata.com> (raw)
In-Reply-To: <CAJ2095q3Gs=+e4ndKiD4EEX9LMh0Mjkv0nNOT=C-0aLD7tBDew-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

[-- Attachment #1: Type: text/plain, Size: 2181 bytes --]

On Thu, Apr 16, 2015 at 06:57:46PM +0200, Dorian Gray wrote:
> On 16 April 2015 at 16:15, Alan Stern <stern-nwvwT67g6+6dFdvTe/nMLpVzexx5G7lz@public.gmane.org> wrote:
> > This appears to be a problem with the IOMMU or SWIOTLB subsystems, not
> > the USB subsystem.  I have CC'ed the appropriate mailing lists.
> 
> Thanks, I'm far from being a kernel expert, so was expecting it could
> be wrong subsection.
> 
> 
> 
> On 16 April 2015 at 16:24, Suman Tripathi <stripathi-qTEPVZfXA3Y@public.gmane.org> wrote:
> > Try increasing the SWIOTLB size to 128MB .Default is 64MB.
> 
> Ok, so I'm back to k3.18.7 (default in the latest Fatdog), although
> I'm not sure what should be the exact value of swiotlb boot param?
> Got totally mixed results from uncle Google - some says the unit is in
> MiB, some that it's 4k pages and another that 128MiB = 65536, so I
> played it safe and used swiotlb=131072.
> Is this correct?
> It may take a few days, but I'll let you know if it worked (or for how
> long, if not).
> 
> 
> 
> On 16 April 2015 at 16:54, Alexander Duyck <alexander.duyck-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > More likely would be a device driver that is DMA mapping memory but not
> > unmapping it after it is done resulting in the bounce buffer pool being
> > depleted.
> > You might want dump the list of drivers loaded on the system with lsmod,
> > and then possibly look at doing a git bisect for something introduced
> > between 3.17 and 3.18 since that seems to be when you started seeing
> > this issue.
> 
> Ok, I'll (try to) look at this, but like I said - I'm not a kernel
> (nor git) expert.
> Anyway, I guess I'm gonna start with this:
> https://wiki.gentoo.org/wiki/Kernel_git-bisect
> Who knows...perhaps I'll find something...

And easier way is to compile the kernel with CONFIG_DMA_API_DEBUG
and then load the attached module.

That should tell you who and what else is holding on the buffers.


> 
> 
> 
> Thank you all for the replies.
> Jake
> _______________________________________________
> iommu mailing list
> iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu

[-- Attachment #2: dump_dma.c --]
[-- Type: text/plain, Size: 1616 bytes --]

/*
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License v2.0 as published by
 * the Free Software Foundation
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 */

#include <linux/module.h>
#include <linux/string.h>
#include <linux/types.h>
#include <linux/init.h>
#include <linux/stat.h>
#include <linux/err.h>
#include <linux/ctype.h>
#include <linux/slab.h>
#include <linux/limits.h>
#include <linux/device.h>
#include <linux/pci.h>
#include <linux/blkdev.h>
#include <linux/device.h>

#include <linux/init.h>
#include <linux/mm.h>
#include <linux/fcntl.h>
#include <linux/slab.h>
#include <linux/kmod.h>
#include <linux/major.h>
#include <linux/highmem.h>
#include <linux/blkdev.h>
#include <linux/module.h>
#include <linux/blkpg.h>
#include <linux/buffer_head.h>
#include <linux/mpage.h>
#include <linux/mount.h>
#include <linux/uio.h>
#include <linux/namei.h>
#include <asm/uaccess.h>

#include <linux/pagemap.h>
#include <linux/pagevec.h>

#include <linux/dma-debug.h>

#define DUMP_DMA_FUN  "0.1"

MODULE_AUTHOR("Konrad Rzeszutek Wilk <konrad@virtualiron>");
MODULE_DESCRIPTION("dump dma");
MODULE_LICENSE("GPL");
MODULE_VERSION(DUMP_DMA_FUN);

static int __init dump_dma_init(void)
{
	debug_dma_dump_mappings(NULL);
	return 0;
}

static void __exit dump_dma_exit(void)
{
}

module_init(dump_dma_init);
module_exit(dump_dma_exit);

[-- Attachment #3: Type: text/plain, Size: 0 bytes --]



WARNING: multiple messages have this Message-ID (diff)
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Dorian Gray <yourfavouritegod@gmail.com>
Cc: Alexander Duyck <alexander.duyck@gmail.com>,
	Alan Stern <stern@rowland.harvard.edu>,
	Suman Tripathi <stripathi@apm.com>,
	iommu@lists.linux-foundation.org,
	USB list <linux-usb@vger.kernel.org>,
	Kernel development list <linux-kernel@vger.kernel.org>
Subject: Re: Error: DMA: Out of SW-IOMMU space [was: External USB drives become unresponsive after few hours.]
Date: Thu, 16 Apr 2015 14:42:52 -0400	[thread overview]
Message-ID: <20150416184252.GE7388@x230.dumpdata.com> (raw)
In-Reply-To: <CAJ2095q3Gs=+e4ndKiD4EEX9LMh0Mjkv0nNOT=C-0aLD7tBDew@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 2066 bytes --]

On Thu, Apr 16, 2015 at 06:57:46PM +0200, Dorian Gray wrote:
> On 16 April 2015 at 16:15, Alan Stern <stern@rowland.harvard.edu> wrote:
> > This appears to be a problem with the IOMMU or SWIOTLB subsystems, not
> > the USB subsystem.  I have CC'ed the appropriate mailing lists.
> 
> Thanks, I'm far from being a kernel expert, so was expecting it could
> be wrong subsection.
> 
> 
> 
> On 16 April 2015 at 16:24, Suman Tripathi <stripathi@apm.com> wrote:
> > Try increasing the SWIOTLB size to 128MB .Default is 64MB.
> 
> Ok, so I'm back to k3.18.7 (default in the latest Fatdog), although
> I'm not sure what should be the exact value of swiotlb boot param?
> Got totally mixed results from uncle Google - some says the unit is in
> MiB, some that it's 4k pages and another that 128MiB = 65536, so I
> played it safe and used swiotlb=131072.
> Is this correct?
> It may take a few days, but I'll let you know if it worked (or for how
> long, if not).
> 
> 
> 
> On 16 April 2015 at 16:54, Alexander Duyck <alexander.duyck@gmail.com> wrote:
> > More likely would be a device driver that is DMA mapping memory but not
> > unmapping it after it is done resulting in the bounce buffer pool being
> > depleted.
> > You might want dump the list of drivers loaded on the system with lsmod,
> > and then possibly look at doing a git bisect for something introduced
> > between 3.17 and 3.18 since that seems to be when you started seeing
> > this issue.
> 
> Ok, I'll (try to) look at this, but like I said - I'm not a kernel
> (nor git) expert.
> Anyway, I guess I'm gonna start with this:
> https://wiki.gentoo.org/wiki/Kernel_git-bisect
> Who knows...perhaps I'll find something...

And easier way is to compile the kernel with CONFIG_DMA_API_DEBUG
and then load the attached module.

That should tell you who and what else is holding on the buffers.


> 
> 
> 
> Thank you all for the replies.
> Jake
> _______________________________________________
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu

[-- Attachment #2: dump_dma.c --]
[-- Type: text/plain, Size: 1616 bytes --]

/*
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License v2.0 as published by
 * the Free Software Foundation
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 */

#include <linux/module.h>
#include <linux/string.h>
#include <linux/types.h>
#include <linux/init.h>
#include <linux/stat.h>
#include <linux/err.h>
#include <linux/ctype.h>
#include <linux/slab.h>
#include <linux/limits.h>
#include <linux/device.h>
#include <linux/pci.h>
#include <linux/blkdev.h>
#include <linux/device.h>

#include <linux/init.h>
#include <linux/mm.h>
#include <linux/fcntl.h>
#include <linux/slab.h>
#include <linux/kmod.h>
#include <linux/major.h>
#include <linux/highmem.h>
#include <linux/blkdev.h>
#include <linux/module.h>
#include <linux/blkpg.h>
#include <linux/buffer_head.h>
#include <linux/mpage.h>
#include <linux/mount.h>
#include <linux/uio.h>
#include <linux/namei.h>
#include <asm/uaccess.h>

#include <linux/pagemap.h>
#include <linux/pagevec.h>

#include <linux/dma-debug.h>

#define DUMP_DMA_FUN  "0.1"

MODULE_AUTHOR("Konrad Rzeszutek Wilk <konrad@virtualiron>");
MODULE_DESCRIPTION("dump dma");
MODULE_LICENSE("GPL");
MODULE_VERSION(DUMP_DMA_FUN);

static int __init dump_dma_init(void)
{
	debug_dma_dump_mappings(NULL);
	return 0;
}

static void __exit dump_dma_exit(void)
{
}

module_init(dump_dma_init);
module_exit(dump_dma_exit);

  parent reply	other threads:[~2015-04-16 18:42 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CAJ2095qWQ7S2W9i1NiM6C-0cM=wnceyA9n-0UVLygUXHJ7yxzA@mail.gmail.com>
     [not found] ` <CAJ2095qWQ7S2W9i1NiM6C-0cM=wnceyA9n-0UVLygUXHJ7yxzA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-04-16 14:15   ` Error: DMA: Out of SW-IOMMU space [was: External USB drives become unresponsive after few hours.] Alan Stern
2015-04-16 14:15     ` Alan Stern
     [not found]     ` <Pine.LNX.4.44L0.1504161010370.1391-100000-IYeN2dnnYyZXsRXLowluHWD2FQJk+8+b@public.gmane.org>
2015-04-16 14:24       ` Suman Tripathi
2015-04-16 14:24         ` Suman Tripathi
2015-04-16 14:54       ` Alexander Duyck
2015-04-16 14:54         ` Alexander Duyck
     [not found]         ` <552FCD25.9060807-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2015-04-16 16:57           ` Dorian Gray
2015-04-16 16:57             ` Dorian Gray
     [not found]             ` <CAJ2095q3Gs=+e4ndKiD4EEX9LMh0Mjkv0nNOT=C-0aLD7tBDew-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-04-16 18:42               ` Konrad Rzeszutek Wilk [this message]
2015-04-16 18:42                 ` Konrad Rzeszutek Wilk
     [not found]                 ` <20150416184252.GE7388-sHAKZZqAc8NKMcnDSFYBzAC/G2K4zDHf@public.gmane.org>
2015-04-16 20:13                   ` Dorian Gray
2015-04-16 20:13                     ` Dorian Gray
2015-04-17 15:14                   ` Dorian Gray
2015-04-17 15:14                     ` Dorian Gray
     [not found]                     ` <CAJ2095qW5rQ4VfCDmZQtgUp1FgGQrB1Q3DBSSKrWvvLo+B8OSw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-04-17 20:06                       ` Konrad Rzeszutek Wilk
2015-04-17 20:06                         ` Konrad Rzeszutek Wilk
2015-04-18 10:10                         ` Dorian Gray
2015-04-18 10:10                           ` Dorian Gray
     [not found]                           ` <CAJ2095qSqTY3fDx5UCub9gkE_bPmX2jyHnuZrcsYtuaHOhSiQQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-04-18 19:59                             ` Dorian Gray
2015-04-18 19:59                               ` Dorian Gray
2015-04-19 15:43                               ` Dorian Gray
     [not found]                                 ` <CAJ2095rrd2ZR9mNJegBn1qO2-obod51GXL1bOJTtB+wH2MF+7g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-04-20 13:03                                   ` Konrad Rzeszutek Wilk
2015-04-20 13:03                                     ` Konrad Rzeszutek Wilk
2015-04-17 15:10             ` Dorian Gray

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150416184252.GE7388@x230.dumpdata.com \
    --to=konrad.wilk-qhclzuegtsvqt0dzr+alfa@public.gmane.org \
    --cc=alexander.duyck-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \
    --cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-usb-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=stern-nwvwT67g6+6dFdvTe/nMLpVzexx5G7lz@public.gmane.org \
    --cc=yourfavouritegod-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.