From: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
To: Jason Gunthorpe <jgunthorpe@obsidianresearch.com>
Cc: "Bjorn Helgaas" <bhelgaas@google.com>,
"Ezequiel Garcia" <ezequiel.garcia@free-electrons.com>,
"Gregory Clément" <gregory.clement@free-electrons.com>,
"Lior Amsalem" <alior@marvell.com>,
linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org
Subject: Issue with the emulated PCI bridge implementation
Date: Thu, 26 Dec 2013 16:05:34 +0100 [thread overview]
Message-ID: <20131226160534.36cc4203@skate> (raw)
Hello Jason,
I am contacting you regarding a design issue I have with the emulated
PCI bridge implementation that we have written to support the Marvell
PCI controllers.
I'm sure you remember that we emulate a bridge, and trap the writes to
this bridge to create/remove MBus windows as needed. But for I/O
regions, we also use these writes to do the appropriate
pci_ioremap_io() calls.
However, it turns how that the writes to the bridge registers are done
with IRQ disabled, and that pci_ioremap_io() can sleep (since it
essentially does an ioremap). Therefore, with the appropriate kernel
hacking options enabled, you get the following warning at boot time:
pci 0000:00:09.0: PCI bridge to [bus 02]
pci 0000:00:09.0: bridge window [io 0x10000-0x10fff]
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at /home/thomas/projets/linux-2.6/kernel/locking/lockdep.c:2740 lockdep_trace_alloc+0x11c/0x120()
DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags))
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.13.0-rc5-00002-gf197217 #450
[<c0014a9c>] (unwind_backtrace+0x0/0xf4) from [<c0011348>] (show_stack+0x10/0x14)
[<c0011348>] (show_stack+0x10/0x14) from [<c03b5318>] (dump_stack+0x98/0xb8)
[<c03b5318>] (dump_stack+0x98/0xb8) from [<c001cd94>] (warn_slowpath_common+0x6c/0x88)
[<c001cd94>] (warn_slowpath_common+0x6c/0x88) from [<c001cde0>] (warn_slowpath_fmt+0x30/0x40)
[<c001cde0>] (warn_slowpath_fmt+0x30/0x40) from [<c00533cc>] (lockdep_trace_alloc+0x11c/0x120)
[<c00533cc>] (lockdep_trace_alloc+0x11c/0x120) from [<c00885c8>] (__alloc_pages_nodemask+0x84/0x734)
[<c00885c8>] (__alloc_pages_nodemask+0x84/0x734) from [<c0088c88>] (__get_free_pages+0x10/0x24)
[<c0088c88>] (__get_free_pages+0x10/0x24) from [<c009dd14>] (__pte_alloc_kernel+0x18/0xb8)
[<c009dd14>] (__pte_alloc_kernel+0x18/0xb8) from [<c017b718>] (ioremap_page_range+0x194/0x1a8)
[<c017b718>] (ioremap_page_range+0x194/0x1a8) from [<c01abf68>] (mvebu_pcie_wr_conf+0x2c0/0x434)
[<c01abf68>] (mvebu_pcie_wr_conf+0x2c0/0x434) from [<c019b4dc>] (pci_bus_write_config_dword+0x5c/0x7c)
[<c019b4dc>] (pci_bus_write_config_dword+0x5c/0x7c) from [<c01a6b60>] (__pci_setup_bridge+0x1c0/0x250)
[<c01a6b60>] (__pci_setup_bridge+0x1c0/0x250) from [<c03b23c0>] (__pci_bus_assign_resources+0xfc/0x100)
[<c03b23c0>] (__pci_bus_assign_resources+0xfc/0x100) from [<c00131d0>] (pci_common_init_dev+0x23c/0x2d4)
[<c00131d0>] (pci_common_init_dev+0x23c/0x2d4) from [<c01ac4f4>] (mvebu_pcie_probe+0x3b4/0x61c)
[<c01ac4f4>] (mvebu_pcie_probe+0x3b4/0x61c) from [<c01dddc4>] (platform_drv_probe+0x18/0x48)
[<c01dddc4>] (platform_drv_probe+0x18/0x48) from [<c01dc564>] (really_probe+0x80/0x218)
[<c01dc564>] (really_probe+0x80/0x218) from [<c01dc7f0>] (__driver_attach+0xa0/0xa4)
[<c01dc7f0>] (__driver_attach+0xa0/0xa4) from [<c01dac34>] (bus_for_each_dev+0x60/0x94)
[<c01dac34>] (bus_for_each_dev+0x60/0x94) from [<c01dbdf8>] (bus_add_driver+0x148/0x1f0)
[<c01dbdf8>] (bus_add_driver+0x148/0x1f0) from [<c01dce28>] (driver_register+0x78/0xf8)
[<c01dce28>] (driver_register+0x78/0xf8) from [<c00088c8>] (do_one_initcall+0xf8/0x148)
[<c00088c8>] (do_one_initcall+0xf8/0x148) from [<c04fac48>] (kernel_init_freeable+0x13c/0x1dc)
[<c04fac48>] (kernel_init_freeable+0x13c/0x1dc) from [<c03b0924>] (kernel_init+0x8/0x120)
[<c03b0924>] (kernel_init+0x8/0x120) from [<c000e268>] (ret_from_fork+0x14/0x2c)
---[ end trace eebd66da7489756f ]---
pci 0000:00:09.0: bridge window [mem 0xe0000000-0xe00fffff]
Do you have any idea to solve this?
For now, my only idea would be to do the pci_ioremap_io()
unconditionally for all PCIe interfaces when the PCIe host controller
driver is initialized. We know the maximum size of the I/O region for
each PCIe interface, and this size is small (64 KB). We can keep the
creation of the corresponding MBus window as something dynamic done by
the bridge.
Do you think that's a reasonable solution? Do you see other
possibilities?
Thanks!
Thomas Petazzoni
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
next reply other threads:[~2013-12-26 15:05 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-12-26 15:05 Thomas Petazzoni [this message]
2013-12-26 15:52 ` Issue with the emulated PCI bridge implementation Thomas Petazzoni
2014-01-02 21:41 ` Bjorn Helgaas
2014-01-03 0:26 ` Jason Gunthorpe
2014-01-03 12:22 ` Arnd Bergmann
2014-01-03 18:44 ` Jason Gunthorpe
2014-01-03 19:03 ` Arnd Bergmann
2014-01-03 19:01 ` Thomas Petazzoni
2014-01-03 19:04 ` Arnd Bergmann
2014-01-03 19:10 ` Thomas Petazzoni
2014-01-03 19:13 ` Arnd Bergmann
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20131226160534.36cc4203@skate \
--to=thomas.petazzoni@free-electrons.com \
--cc=alior@marvell.com \
--cc=bhelgaas@google.com \
--cc=ezequiel.garcia@free-electrons.com \
--cc=gregory.clement@free-electrons.com \
--cc=jgunthorpe@obsidianresearch.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-pci@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).