From: Nishanth Aravamudan <nacc@linux.vnet.ibm.com>
To: Matthew Wilcox <willy@linux.intel.com>
Cc: Keith Busch <keith.busch@intel.com>,
Benjamin Herrenschmidt <benh@kernel.crashing.org>,
Paul Mackerras <paulus@samba.org>,
Michael Ellerman <mpe@ellerman.id.au>,
Alexey Kardashevskiy <aik@ozlabs.ru>,
David Gibson <david@gibson.dropbear.id.au>,
linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org,
linuxppc-dev@lists.ozlabs.org
Subject: [PATCH 0/2] Fix NVMe driver support on Power with 32-bit DMA
Date: Fri, 2 Oct 2015 10:16:06 -0700 [thread overview]
Message-ID: <20151002171606.GA41011@linux.vnet.ibm.com> (raw)
We received a bug report recently when DDW (64-bit direct DMA on Power)
is not enabled for NVMe devices. In that case, we fall back to 32-bit
DMA via the IOMMU, which is always done via 4K TCEs (Translation Control
Entries).
The NVMe device driver, though, assumes that the DMA alignment for the
PRP entries will match the device's page size, and that the DMA aligment
matches the kernel's page aligment. On Power, the the IOMMU page size,
as mentioned above, can be 4K, while the device can have a page size of
8K, while the kernel has a page size of 64K. This eventually trips the
BUG_ON in nvme_setup_prps(), as we have a 'dma_len' that is a multiple
of 4K but not 8K (e.g., 0xF000).
In this particular case, and generally, we want to use the IOMMU's page
size for the default device page size, rather than the kernel's page
size.
This series consists of two patches, one of which exposes the IOMMU's
page shift on Power (currently only the page size is exposed, and it
seems unnecessary to ilog2 that value in the driver). The second patch
leverages this value on Power in the NVMe driver.
With these patches, a NVMe device survives our internal hardware
exerciser; the kernel BUGs within a few seconds without the patch.
next reply other threads:[~2015-10-02 17:16 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-02 17:16 Nishanth Aravamudan [this message]
2015-10-02 17:18 ` [PATCH 1/2] powerpc/iommu: expose IOMMU page shift Nishanth Aravamudan
2015-10-02 17:23 ` [PATCH 2/2] drivers/nvme: default to the IOMMU page size on Power Nishanth Aravamudan
2015-10-02 17:25 ` Christoph Hellwig
2015-10-02 17:39 ` Nishanth Aravamudan
2015-10-02 17:41 ` Christoph Hellwig
2015-10-02 18:57 ` kbuild test robot
2015-10-06 3:19 ` [PATCH 1/2] powerpc/iommu: expose IOMMU page shift David Gibson
2015-10-12 16:03 ` Nishanth Aravamudan
2015-10-12 21:10 ` Nishanth Aravamudan
2015-10-02 20:09 ` [PATCH 0/5 v2] Fix NVMe driver support on Power with 32-bit DMA Nishanth Aravamudan
2015-10-02 20:11 ` [PATCH 1/5 v2] dma-mapping: add generic dma_get_page_shift API Nishanth Aravamudan
2015-10-02 20:16 ` [PATCH 2/5 v2] powerpc/dma-mapping: override dma_get_page_shift Nishanth Aravamudan
2015-10-02 20:19 ` [PATCH 3/5 v2] powerpc/dma: implement per-platform dma_get_page_shift Nishanth Aravamudan
2015-10-02 20:21 ` [PATCH 4/5 v2] pseries/iommu: implement DDW-aware dma_get_page_shift Nishanth Aravamudan
2015-10-02 20:30 ` [PATCH 5/5 v2] drivers/nvme: default to the IOMMU page size Nishanth Aravamudan
2015-10-02 20:43 ` kbuild test robot
2015-10-02 20:33 ` [PATCH 4/5 v2] pseries/iommu: implement DDW-aware dma_get_page_shift kbuild test robot
2015-10-06 3:39 ` Michael Ellerman
2015-10-07 13:56 ` [kbuild-all] " Fengguang Wu
2015-10-08 0:11 ` Michael Ellerman
2015-10-08 1:06 ` Fengguang Wu
2015-10-08 1:16 ` Fengguang Wu
2015-10-08 4:06 ` Michael Ellerman
2015-10-11 14:22 ` Fengguang Wu
2015-10-12 2:51 ` Michael Ellerman
2015-10-08 7:46 ` Christoph Hellwig
2015-10-11 14:19 ` testing email patches Fengguang Wu
2015-10-06 3:43 ` [PATCH 1/5 v2] dma-mapping: add generic dma_get_page_shift API Michael Ellerman
2015-10-06 9:51 ` Christoph Hellwig
2015-10-12 16:04 ` Nishanth Aravamudan
2015-10-12 21:06 ` Nishanth Aravamudan
2015-10-14 15:39 ` Nishanth Aravamudan
2015-10-14 15:42 ` Christoph Hellwig
2015-10-15 22:52 ` Nishanth Aravamudan
2015-10-19 17:56 ` Nishanth Aravamudan
2015-10-02 20:51 ` [PATCH 0/5 v2] Fix NVMe driver support on Power with 32-bit DMA Benjamin Herrenschmidt
2015-10-02 21:04 ` Nishanth Aravamudan
2015-10-02 21:35 ` Benjamin Herrenschmidt
2015-10-02 21:48 ` Nishanth Aravamudan
2015-10-03 8:19 ` Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20151002171606.GA41011@linux.vnet.ibm.com \
--to=nacc@linux.vnet.ibm.com \
--cc=aik@ozlabs.ru \
--cc=benh@kernel.crashing.org \
--cc=david@gibson.dropbear.id.au \
--cc=keith.busch@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mpe@ellerman.id.au \
--cc=paulus@samba.org \
--cc=willy@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).