* [patch] pcibus_to_node() addition to topology infrastructure
@ 2002-10-30 23:42 Matthew Dobson
2002-10-30 23:44 ` Matthew Dobson
2002-10-31 0:03 ` Jesse Barnes
0 siblings, 2 replies; 6+ messages in thread
From: Matthew Dobson @ 2002-10-30 23:42 UTC (permalink / raw)
To: linux-kernel, Linus Torvalds, Patrick Mansfield
Linus,
Here's a patch that adds PCI busses to the list of basic topology
elements (incl. CPUs, MemBlks, & Nodes).
pcibus_to_node-2.5.44.patch
This patch adds a new topology macro: pcibus_to_node(). This will be
useful to allow I/O bound processes to bind themselves to CPUs/Nodes
close to the PCI busses they are communicating over.
1) Adds pcibus_to_node() macro to asm-generic/topology.h
2) Makes small modifications to NUMA-Q PCI code, mostly modifying macros.
3) Uses the macros from #2 to implement pcibus_to_node() in
asm-i386/topology.h
[mcd@arrakis patches]$ diffstat api_patches/pcibus_to_node-2.5.44.patch
arch/i386/pci/numa.c | 33 +++++++++++++++------------------
include/asm-generic/topology.h | 3 +++
include/asm-i386/topology.h | 3 +++
3 files changed, 21 insertions(+), 18 deletions(-)
Cheers!
-Matt
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [patch] pcibus_to_node() addition to topology infrastructure
2002-10-30 23:42 [patch] pcibus_to_node() addition to topology infrastructure Matthew Dobson
@ 2002-10-30 23:44 ` Matthew Dobson
2002-10-31 0:03 ` Jesse Barnes
1 sibling, 0 replies; 6+ messages in thread
From: Matthew Dobson @ 2002-10-30 23:44 UTC (permalink / raw)
To: linux-kernel, Linus Torvalds, Patrick Mansfield
[-- Attachment #1: Type: text/plain, Size: 981 bytes --]
Whoops! A patch would be nice...
Cheers!
-Matt
Matthew Dobson wrote:
> Linus,
> Here's a patch that adds PCI busses to the list of basic topology
> elements (incl. CPUs, MemBlks, & Nodes).
>
> pcibus_to_node-2.5.44.patch
>
> This patch adds a new topology macro: pcibus_to_node(). This will be
> useful to allow I/O bound processes to bind themselves to CPUs/Nodes
> close to the PCI busses they are communicating over.
>
> 1) Adds pcibus_to_node() macro to asm-generic/topology.h
> 2) Makes small modifications to NUMA-Q PCI code, mostly modifying macros.
> 3) Uses the macros from #2 to implement pcibus_to_node() in
> asm-i386/topology.h
>
> [mcd@arrakis patches]$ diffstat api_patches/pcibus_to_node-2.5.44.patch
> arch/i386/pci/numa.c | 33 +++++++++++++++------------------
> include/asm-generic/topology.h | 3 +++
> include/asm-i386/topology.h | 3 +++
> 3 files changed, 21 insertions(+), 18 deletions(-)
>
> Cheers!
>
> -Matt
>
[-- Attachment #2: pcibus_to_node-2.5.44.patch --]
[-- Type: text/plain, Size: 4541 bytes --]
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.44-vanilla/arch/i386/pci/numa.c linux-2.5.44-pcibus_to_node/arch/i386/pci/numa.c
--- linux-2.5.44-vanilla/arch/i386/pci/numa.c Fri Oct 18 21:01:17 2002
+++ linux-2.5.44-pcibus_to_node/arch/i386/pci/numa.c Wed Oct 30 12:03:41 2002
@@ -5,13 +5,10 @@
#include <linux/pci.h>
#include <linux/init.h>
#include "pci.h"
-
-#define BUS2QUAD(global) (mp_bus_id_to_node[global])
-#define BUS2LOCAL(global) (mp_bus_id_to_local[global])
-#define QUADLOCAL2BUS(quad,local) (quad_local_to_mp_bus_id[quad][local])
+#include <asm/topology.h>
#define PCI_CONF1_MQ_ADDRESS(bus, dev, fn, reg) \
- (0x80000000 | (BUS2LOCAL(bus) << 16) | (dev << 11) | (fn << 8) | (reg & ~3))
+ (0x80000000 | mp_bus_id_to_local[bus] << 16) | (dev << 11) | (fn << 8) | (reg & ~3))
static int __pci_conf1_mq_read (int seg, int bus, int dev, int fn, int reg, int len, u32 *value)
{
@@ -22,17 +19,17 @@
spin_lock_irqsave(&pci_config_lock, flags);
- outl_quad(PCI_CONF1_MQ_ADDRESS(bus, dev, fn, reg), 0xCF8, BUS2QUAD(bus));
+ outl_quad(PCI_CONF1_MQ_ADDRESS(bus, dev, fn, reg), 0xCF8, __pcibus_to_node(bus));
switch (len) {
case 1:
- *value = inb_quad(0xCFC + (reg & 3), BUS2QUAD(bus));
+ *value = inb_quad(0xCFC + (reg & 3), __pcibus_to_node(bus));
break;
case 2:
- *value = inw_quad(0xCFC + (reg & 2), BUS2QUAD(bus));
+ *value = inw_quad(0xCFC + (reg & 2), __pcibus_to_node(bus));
break;
case 4:
- *value = inl_quad(0xCFC, BUS2QUAD(bus));
+ *value = inl_quad(0xCFC, __pcibus_to_node(bus));
break;
}
@@ -50,17 +47,17 @@
spin_lock_irqsave(&pci_config_lock, flags);
- outl_quad(PCI_CONF1_MQ_ADDRESS(bus, dev, fn, reg), 0xCF8, BUS2QUAD(bus));
+ outl_quad(PCI_CONF1_MQ_ADDRESS(bus, dev, fn, reg), 0xCF8, __pcibus_to_node(bus));
switch (len) {
case 1:
- outb_quad((u8)value, 0xCFC + (reg & 3), BUS2QUAD(bus));
+ outb_quad((u8)value, 0xCFC + (reg & 3), __pcibus_to_node(bus));
break;
case 2:
- outw_quad((u16)value, 0xCFC + (reg & 2), BUS2QUAD(bus));
+ outw_quad((u16)value, 0xCFC + (reg & 2), __pcibus_to_node(bus));
break;
case 4:
- outl_quad((u32)value, 0xCFC, BUS2QUAD(bus));
+ outl_quad((u32)value, 0xCFC, __pcibus_to_node(bus));
break;
}
@@ -96,7 +93,7 @@
*/
int pxb, reg;
u8 busno, suba, subb;
- int quad = BUS2QUAD(d->bus->number);
+ int quad = __pcibus_to_node(d->bus->number);
printk("PCI: Searching for i450NX host bridges on %s\n", d->slot_name);
reg = 0xd0;
@@ -106,9 +103,9 @@
pci_read_config_byte(d, reg++, &subb);
DBG("i450NX PXB %d: %02x/%02x/%02x\n", pxb, busno, suba, subb);
if (busno)
- pci_scan_bus(QUADLOCAL2BUS(quad,busno), pci_root_ops, NULL); /* Bus A */
+ pci_scan_bus(quad_local_to_mp_bus_id[quad][busno], pci_root_ops, NULL); /* Bus A */
if (suba < subb)
- pci_scan_bus(QUADLOCAL2BUS(quad,suba+1), pci_root_ops, NULL); /* Bus B */
+ pci_scan_bus(quad_local_to_mp_bus_id[quad][suba+1], pci_root_ops, NULL); /* Bus B */
}
pcibios_last_bus = -1;
}
@@ -130,8 +127,8 @@
if (clustered_apic_mode && (numnodes > 1)) {
for (quad = 1; quad < numnodes; ++quad) {
printk("Scanning PCI bus %d for quad %d\n",
- QUADLOCAL2BUS(quad,0), quad);
- pci_scan_bus(QUADLOCAL2BUS(quad,0),
+ quad_local_to_mp_bus_id[quad][0], quad);
+ pci_scan_bus(quad_local_to_mp_bus_id[quad][0],
pci_root_ops, NULL);
}
}
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.44-vanilla/include/asm-generic/topology.h linux-2.5.44-pcibus_to_node/include/asm-generic/topology.h
--- linux-2.5.44-vanilla/include/asm-generic/topology.h Fri Oct 18 21:01:11 2002
+++ linux-2.5.44-pcibus_to_node/include/asm-generic/topology.h Wed Oct 30 12:03:41 2002
@@ -47,5 +47,8 @@
#ifndef __node_to_memblk
#define __node_to_memblk(node) (0)
#endif
+#ifndef __pcibus_to_node
+#define __pcibus_to_node(bus) (0)
+#endif
#endif /* _ASM_GENERIC_TOPOLOGY_H */
diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.44-vanilla/include/asm-i386/topology.h linux-2.5.44-pcibus_to_node/include/asm-i386/topology.h
--- linux-2.5.44-vanilla/include/asm-i386/topology.h Fri Oct 18 21:01:16 2002
+++ linux-2.5.44-pcibus_to_node/include/asm-i386/topology.h Wed Oct 30 12:03:41 2002
@@ -83,6 +83,9 @@
/* Returns the number of the first MemBlk on Node 'node' */
#define __node_to_memblk(node) (node)
+/* Returns the number of the node containing PCI bus 'bus' */
+#define __pcibus_to_node(bus) (mp_bus_id_to_node[bus])
+
#else /* !CONFIG_X86_NUMAQ */
/*
* Other i386 platforms should define their own version of the
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [patch] pcibus_to_node() addition to topology infrastructure
2002-10-30 23:42 [patch] pcibus_to_node() addition to topology infrastructure Matthew Dobson
2002-10-30 23:44 ` Matthew Dobson
@ 2002-10-31 0:03 ` Jesse Barnes
2002-10-31 0:24 ` Matthew Dobson
1 sibling, 1 reply; 6+ messages in thread
From: Jesse Barnes @ 2002-10-31 0:03 UTC (permalink / raw)
To: Matthew Dobson; +Cc: linux-kernel
This is a nice addition, but just FYI there are SGI systems that have
PCI busses attached to more than one node. I guess for now we can just
round-robin through the attached nodes for the return value.
Thanks,
Jesse
On Wed, Oct 30, 2002 at 03:42:45PM -0800, Matthew Dobson wrote:
> Linus,
> Here's a patch that adds PCI busses to the list of basic topology
> elements (incl. CPUs, MemBlks, & Nodes).
>
> pcibus_to_node-2.5.44.patch
>
> This patch adds a new topology macro: pcibus_to_node(). This will be
> useful to allow I/O bound processes to bind themselves to CPUs/Nodes
> close to the PCI busses they are communicating over.
>
> 1) Adds pcibus_to_node() macro to asm-generic/topology.h
> 2) Makes small modifications to NUMA-Q PCI code, mostly modifying macros.
> 3) Uses the macros from #2 to implement pcibus_to_node() in
> asm-i386/topology.h
>
> [mcd@arrakis patches]$ diffstat api_patches/pcibus_to_node-2.5.44.patch
> arch/i386/pci/numa.c | 33 +++++++++++++++------------------
> include/asm-generic/topology.h | 3 +++
> include/asm-i386/topology.h | 3 +++
> 3 files changed, 21 insertions(+), 18 deletions(-)
>
> Cheers!
>
> -Matt
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [patch] pcibus_to_node() addition to topology infrastructure
2002-10-31 0:03 ` Jesse Barnes
@ 2002-10-31 0:24 ` Matthew Dobson
2002-10-31 0:59 ` Jesse Barnes
0 siblings, 1 reply; 6+ messages in thread
From: Matthew Dobson @ 2002-10-31 0:24 UTC (permalink / raw)
To: Jesse Barnes; +Cc: linux-kernel
Jesse Barnes wrote:
> This is a nice addition, but just FYI there are SGI systems that have
> PCI busses attached to more than one node. I guess for now we can just
> round-robin through the attached nodes for the return value.
>
> Thanks,
> Jesse
Ah, yes... The p-bricks, i-bricks, etc. right?
Yes, I suppose a round-robin return for the SGI version of the macro
would work... Certainly not ideal, but it would work. The problem is
if you assume that binding several processes to the node PCI bus X is on
will ensure that those processes are on the same node. In this case
(SGI), it would be a false assumption. I suppose that's what you get
for assuming, though? ;)
Cheers!
-Matt
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [patch] pcibus_to_node() addition to topology infrastructure
2002-10-31 0:24 ` Matthew Dobson
@ 2002-10-31 0:59 ` Jesse Barnes
2002-10-31 1:14 ` Matthew Dobson
0 siblings, 1 reply; 6+ messages in thread
From: Jesse Barnes @ 2002-10-31 0:59 UTC (permalink / raw)
To: Matthew Dobson; +Cc: linux-kernel
On Wed, Oct 30, 2002 at 04:24:13PM -0800, Matthew Dobson wrote:
> Ah, yes... The p-bricks, i-bricks, etc. right?
Yup.
> Yes, I suppose a round-robin return for the SGI version of the macro
> would work... Certainly not ideal, but it would work. The problem is
Can you think of any better way to do it? Perhaps make pcibus_to_node
return a list of nodes?
Thanks,
Jesse
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [patch] pcibus_to_node() addition to topology infrastructure
2002-10-31 0:59 ` Jesse Barnes
@ 2002-10-31 1:14 ` Matthew Dobson
0 siblings, 0 replies; 6+ messages in thread
From: Matthew Dobson @ 2002-10-31 1:14 UTC (permalink / raw)
To: Jesse Barnes; +Cc: linux-kernel
Jesse Barnes wrote:
> On Wed, Oct 30, 2002 at 04:24:13PM -0800, Matthew Dobson wrote:
>
>>Ah, yes... The p-bricks, i-bricks, etc. right?
>
>
> Yup.
>
>
>>Yes, I suppose a round-robin return for the SGI version of the macro
>>would work... Certainly not ideal, but it would work. The problem is
>
>
> Can you think of any better way to do it? Perhaps make pcibus_to_node
> return a list of nodes?
Yeah... I was thinking about having it return a bitmask... It makes it
a bit of a pain to iterate through the mask, only to find that in most
cases there is only one bit set in the mask. :(
It's a bit late in the day... I'll mull this over and see if I can come
up with something good.
Cheers!
-Matt
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2002-10-31 1:12 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2002-10-30 23:42 [patch] pcibus_to_node() addition to topology infrastructure Matthew Dobson
2002-10-30 23:44 ` Matthew Dobson
2002-10-31 0:03 ` Jesse Barnes
2002-10-31 0:24 ` Matthew Dobson
2002-10-31 0:59 ` Jesse Barnes
2002-10-31 1:14 ` Matthew Dobson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox