Patch: Increase maximum bus memory window size (for Xeon Phi)

Patch: Increase maximum bus memory window size (for Xeon Phi)

[Bryce Lelbach]

[-- Attachment #1.1: Type: text/plain, Size: 971 bytes --]

The attached patch affects pbus_size_mem() in drivers/pci/setup-bus.c. The patch
increases the maximum memory window size for a PCI bus from 2 Gb to 2^63 Gb.

This change is necessary to support Intel Xeon Phi co-processors. These co-processors
are PCIe devices used for high-performance computing applications. The device requires
an 8Gb memory window to function (the co-processor has 8Gb of onboard memory).

This patch is a modified version of a patch from Intel's MPSS framework
(specifically, from the "KNC_gold_update_1-2.1.4982-15-rhel-6.3" package), which
will apply to a 3.7.8 kernel (I am about to try it on a 3.8 kernel). To the best
of my knowledge, newer RHEL kernels are shipped with this patch.
 
-- 
Bryce Adelstein-Lelbach aka wash
STE||AR Group, Center for Computation and Technology, LSU
--
225-317-3866 - iPhone
225-578-6182 - Work (no voicemail)
--
stellar.cct.lsu.edu
boost-spirit.com
llvm.linuxfoundation.org
cppnow.org
--

[-- Attachment #1.2: linux_3.7.8_pci_xeon_phi.patch --]
[-- Type: text/x-diff, Size: 997 bytes --]

diff -u -r -N linux-source-3.7/drivers/pci/setup-bus.c linux-source-3.7-xeon-phi/drivers/pci/setup-bus.c
--- linux-source-3.7/drivers/pci/setup-bus.c	2013-02-14 12:57:59.000000000 -0600
+++ linux-source-3.7-xeon-phi/drivers/pci/setup-bus.c	2013-02-20 18:08:16.000000000 -0600
@@ -852,7 +852,7 @@
 {
 	struct pci_dev *dev;
 	resource_size_t min_align, align, size, size0, size1;
-	resource_size_t aligns[12];	/* Alignments from 1Mb to 2Gb */
+	resource_size_t aligns[44];	/* Alignments from 1Mb to 2^63 */
 	int order, max_order;
 	struct resource *b_res = find_free_bus_resource(bus, type);
 	unsigned int mem64_mask = 0;
@@ -891,7 +891,8 @@
 			/* For bridges size != alignment */
 			align = pci_resource_alignment(dev, r);
 			order = __ffs(align) - 20;
-			if (order > 11) {
+			if ((sizeof(size_t) == 4 && order > 11) ||
+					(sizeof(size_t) == 8 && order > 43)) {
 				dev_warn(&dev->dev, "disabling BAR %d: %pR "
 					 "(bad alignment %#llx)\n", i, r,
 					 (unsigned long long) align);

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: Patch: Increase maximum bus memory window size (for Xeon Phi)

[Don Dutile]

On 05/24/2013 12:59 PM, Bryce Lelbach wrote:
> The attached patch affects pbus_size_mem() in drivers/pci/setup-bus.c. The patch
> increases the maximum memory window size for a PCI bus from 2 Gb to 2^63 Gb.
>
> This change is necessary to support Intel Xeon Phi co-processors. These co-processors
> are PCIe devices used for high-performance computing applications. The device requires
> an 8Gb memory window to function (the co-processor has 8Gb of onboard memory).
>
> This patch is a modified version of a patch from Intel's MPSS framework
> (specifically, from the "KNC_gold_update_1-2.1.4982-15-rhel-6.3" package), which
> will apply to a 3.7.8 kernel (I am about to try it on a 3.8 kernel). To the best
> of my knowledge, newer RHEL kernels are shipped with this patch.
>
This patch is not in (the latest) RHEL6(.4) or the internal staging tree for the next
RHEL6(.5) release.

Re: Patch: Increase maximum bus memory window size (for Xeon Phi)

[Bjorn Helgaas]

On Fri, May 24, 2013 at 10:59 AM, Bryce Lelbach <blelbach@cct.lsu.edu> wrote:
> The attached patch affects pbus_size_mem() in drivers/pci/setup-bus.c. The patch
> increases the maximum memory window size for a PCI bus from 2 Gb to 2^63 Gb.
>
> This change is necessary to support Intel Xeon Phi co-processors. These co-processors
> are PCIe devices used for high-performance computing applications. The device requires
> an 8Gb memory window to function (the co-processor has 8Gb of onboard memory).
>
> This patch is a modified version of a patch from Intel's MPSS framework
> (specifically, from the "KNC_gold_update_1-2.1.4982-15-rhel-6.3" package), which
> will apply to a 3.7.8 kernel (I am about to try it on a 3.8 kernel). To the best
> of my knowledge, newer RHEL kernels are shipped with this patch.

This needs a Signed-off-by before it can be applied (see
Documentation/SubmittingPatches for details about what this means).

Also, there's some confusion about this being in RHEL.  I take it that
it is not actually in RHEL, but perhaps in an Intel-supplied update
that is commonly applied on top of RHEL?

IIRC, a very similar patch was posted last year.  It'd be nice to dig
that up and credit that author as well.

It's more convenient for me to receive patches as plain-text email
(not attachments), also as described in SubmittingPatches.  I *can*
deal with attachments, but it's a hassle and I tend to keep putting
them off ;)

Bjorn

Re: Patch: Increase maximum bus memory window size (for Xeon Phi)

[Bryce Lelbach]

[-- Attachment #1: Type: text/plain, Size: 345 bytes --]

Sorry, my mistake (re: the RHEL thing).

I'll see if I can dig up the original author's name.

-- 
Bryce Adelstein-Lelbach aka wash
STE||AR Group, Center for Computation and Technology, LSU
--
225-317-3866 - iPhone
225-578-6182 - Work (no voicemail)
--
stellar.cct.lsu.edu
boost-spirit.com
llvm.linuxfoundation.org
cppnow.org
--

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]