* [patch] pcibus_to_node() addition to topology infrastructure @ 2002-10-30 23:42 Matthew Dobson 2002-10-30 23:44 ` Matthew Dobson 2002-10-31 0:03 ` Jesse Barnes 0 siblings, 2 replies; 6+ messages in thread From: Matthew Dobson @ 2002-10-30 23:42 UTC (permalink / raw) To: linux-kernel, Linus Torvalds, Patrick Mansfield Linus, Here's a patch that adds PCI busses to the list of basic topology elements (incl. CPUs, MemBlks, & Nodes). pcibus_to_node-2.5.44.patch This patch adds a new topology macro: pcibus_to_node(). This will be useful to allow I/O bound processes to bind themselves to CPUs/Nodes close to the PCI busses they are communicating over. 1) Adds pcibus_to_node() macro to asm-generic/topology.h 2) Makes small modifications to NUMA-Q PCI code, mostly modifying macros. 3) Uses the macros from #2 to implement pcibus_to_node() in asm-i386/topology.h [mcd@arrakis patches]$ diffstat api_patches/pcibus_to_node-2.5.44.patch arch/i386/pci/numa.c | 33 +++++++++++++++------------------ include/asm-generic/topology.h | 3 +++ include/asm-i386/topology.h | 3 +++ 3 files changed, 21 insertions(+), 18 deletions(-) Cheers! -Matt ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [patch] pcibus_to_node() addition to topology infrastructure 2002-10-30 23:42 [patch] pcibus_to_node() addition to topology infrastructure Matthew Dobson @ 2002-10-30 23:44 ` Matthew Dobson 2002-10-31 0:03 ` Jesse Barnes 1 sibling, 0 replies; 6+ messages in thread From: Matthew Dobson @ 2002-10-30 23:44 UTC (permalink / raw) To: linux-kernel, Linus Torvalds, Patrick Mansfield [-- Attachment #1: Type: text/plain, Size: 981 bytes --] Whoops! A patch would be nice... Cheers! -Matt Matthew Dobson wrote: > Linus, > Here's a patch that adds PCI busses to the list of basic topology > elements (incl. CPUs, MemBlks, & Nodes). > > pcibus_to_node-2.5.44.patch > > This patch adds a new topology macro: pcibus_to_node(). This will be > useful to allow I/O bound processes to bind themselves to CPUs/Nodes > close to the PCI busses they are communicating over. > > 1) Adds pcibus_to_node() macro to asm-generic/topology.h > 2) Makes small modifications to NUMA-Q PCI code, mostly modifying macros. > 3) Uses the macros from #2 to implement pcibus_to_node() in > asm-i386/topology.h > > [mcd@arrakis patches]$ diffstat api_patches/pcibus_to_node-2.5.44.patch > arch/i386/pci/numa.c | 33 +++++++++++++++------------------ > include/asm-generic/topology.h | 3 +++ > include/asm-i386/topology.h | 3 +++ > 3 files changed, 21 insertions(+), 18 deletions(-) > > Cheers! > > -Matt > [-- Attachment #2: pcibus_to_node-2.5.44.patch --] [-- Type: text/plain, Size: 4541 bytes --] diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.44-vanilla/arch/i386/pci/numa.c linux-2.5.44-pcibus_to_node/arch/i386/pci/numa.c --- linux-2.5.44-vanilla/arch/i386/pci/numa.c Fri Oct 18 21:01:17 2002 +++ linux-2.5.44-pcibus_to_node/arch/i386/pci/numa.c Wed Oct 30 12:03:41 2002 @@ -5,13 +5,10 @@ #include <linux/pci.h> #include <linux/init.h> #include "pci.h" - -#define BUS2QUAD(global) (mp_bus_id_to_node[global]) -#define BUS2LOCAL(global) (mp_bus_id_to_local[global]) -#define QUADLOCAL2BUS(quad,local) (quad_local_to_mp_bus_id[quad][local]) +#include <asm/topology.h> #define PCI_CONF1_MQ_ADDRESS(bus, dev, fn, reg) \ - (0x80000000 | (BUS2LOCAL(bus) << 16) | (dev << 11) | (fn << 8) | (reg & ~3)) + (0x80000000 | mp_bus_id_to_local[bus] << 16) | (dev << 11) | (fn << 8) | (reg & ~3)) static int __pci_conf1_mq_read (int seg, int bus, int dev, int fn, int reg, int len, u32 *value) { @@ -22,17 +19,17 @@ spin_lock_irqsave(&pci_config_lock, flags); - outl_quad(PCI_CONF1_MQ_ADDRESS(bus, dev, fn, reg), 0xCF8, BUS2QUAD(bus)); + outl_quad(PCI_CONF1_MQ_ADDRESS(bus, dev, fn, reg), 0xCF8, __pcibus_to_node(bus)); switch (len) { case 1: - *value = inb_quad(0xCFC + (reg & 3), BUS2QUAD(bus)); + *value = inb_quad(0xCFC + (reg & 3), __pcibus_to_node(bus)); break; case 2: - *value = inw_quad(0xCFC + (reg & 2), BUS2QUAD(bus)); + *value = inw_quad(0xCFC + (reg & 2), __pcibus_to_node(bus)); break; case 4: - *value = inl_quad(0xCFC, BUS2QUAD(bus)); + *value = inl_quad(0xCFC, __pcibus_to_node(bus)); break; } @@ -50,17 +47,17 @@ spin_lock_irqsave(&pci_config_lock, flags); - outl_quad(PCI_CONF1_MQ_ADDRESS(bus, dev, fn, reg), 0xCF8, BUS2QUAD(bus)); + outl_quad(PCI_CONF1_MQ_ADDRESS(bus, dev, fn, reg), 0xCF8, __pcibus_to_node(bus)); switch (len) { case 1: - outb_quad((u8)value, 0xCFC + (reg & 3), BUS2QUAD(bus)); + outb_quad((u8)value, 0xCFC + (reg & 3), __pcibus_to_node(bus)); break; case 2: - outw_quad((u16)value, 0xCFC + (reg & 2), BUS2QUAD(bus)); + outw_quad((u16)value, 0xCFC + (reg & 2), __pcibus_to_node(bus)); break; case 4: - outl_quad((u32)value, 0xCFC, BUS2QUAD(bus)); + outl_quad((u32)value, 0xCFC, __pcibus_to_node(bus)); break; } @@ -96,7 +93,7 @@ */ int pxb, reg; u8 busno, suba, subb; - int quad = BUS2QUAD(d->bus->number); + int quad = __pcibus_to_node(d->bus->number); printk("PCI: Searching for i450NX host bridges on %s\n", d->slot_name); reg = 0xd0; @@ -106,9 +103,9 @@ pci_read_config_byte(d, reg++, &subb); DBG("i450NX PXB %d: %02x/%02x/%02x\n", pxb, busno, suba, subb); if (busno) - pci_scan_bus(QUADLOCAL2BUS(quad,busno), pci_root_ops, NULL); /* Bus A */ + pci_scan_bus(quad_local_to_mp_bus_id[quad][busno], pci_root_ops, NULL); /* Bus A */ if (suba < subb) - pci_scan_bus(QUADLOCAL2BUS(quad,suba+1), pci_root_ops, NULL); /* Bus B */ + pci_scan_bus(quad_local_to_mp_bus_id[quad][suba+1], pci_root_ops, NULL); /* Bus B */ } pcibios_last_bus = -1; } @@ -130,8 +127,8 @@ if (clustered_apic_mode && (numnodes > 1)) { for (quad = 1; quad < numnodes; ++quad) { printk("Scanning PCI bus %d for quad %d\n", - QUADLOCAL2BUS(quad,0), quad); - pci_scan_bus(QUADLOCAL2BUS(quad,0), + quad_local_to_mp_bus_id[quad][0], quad); + pci_scan_bus(quad_local_to_mp_bus_id[quad][0], pci_root_ops, NULL); } } diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.44-vanilla/include/asm-generic/topology.h linux-2.5.44-pcibus_to_node/include/asm-generic/topology.h --- linux-2.5.44-vanilla/include/asm-generic/topology.h Fri Oct 18 21:01:11 2002 +++ linux-2.5.44-pcibus_to_node/include/asm-generic/topology.h Wed Oct 30 12:03:41 2002 @@ -47,5 +47,8 @@ #ifndef __node_to_memblk #define __node_to_memblk(node) (0) #endif +#ifndef __pcibus_to_node +#define __pcibus_to_node(bus) (0) +#endif #endif /* _ASM_GENERIC_TOPOLOGY_H */ diff -Nur --exclude-from=/usr/src/.dontdiff linux-2.5.44-vanilla/include/asm-i386/topology.h linux-2.5.44-pcibus_to_node/include/asm-i386/topology.h --- linux-2.5.44-vanilla/include/asm-i386/topology.h Fri Oct 18 21:01:16 2002 +++ linux-2.5.44-pcibus_to_node/include/asm-i386/topology.h Wed Oct 30 12:03:41 2002 @@ -83,6 +83,9 @@ /* Returns the number of the first MemBlk on Node 'node' */ #define __node_to_memblk(node) (node) +/* Returns the number of the node containing PCI bus 'bus' */ +#define __pcibus_to_node(bus) (mp_bus_id_to_node[bus]) + #else /* !CONFIG_X86_NUMAQ */ /* * Other i386 platforms should define their own version of the ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [patch] pcibus_to_node() addition to topology infrastructure 2002-10-30 23:42 [patch] pcibus_to_node() addition to topology infrastructure Matthew Dobson 2002-10-30 23:44 ` Matthew Dobson @ 2002-10-31 0:03 ` Jesse Barnes 2002-10-31 0:24 ` Matthew Dobson 1 sibling, 1 reply; 6+ messages in thread From: Jesse Barnes @ 2002-10-31 0:03 UTC (permalink / raw) To: Matthew Dobson; +Cc: linux-kernel This is a nice addition, but just FYI there are SGI systems that have PCI busses attached to more than one node. I guess for now we can just round-robin through the attached nodes for the return value. Thanks, Jesse On Wed, Oct 30, 2002 at 03:42:45PM -0800, Matthew Dobson wrote: > Linus, > Here's a patch that adds PCI busses to the list of basic topology > elements (incl. CPUs, MemBlks, & Nodes). > > pcibus_to_node-2.5.44.patch > > This patch adds a new topology macro: pcibus_to_node(). This will be > useful to allow I/O bound processes to bind themselves to CPUs/Nodes > close to the PCI busses they are communicating over. > > 1) Adds pcibus_to_node() macro to asm-generic/topology.h > 2) Makes small modifications to NUMA-Q PCI code, mostly modifying macros. > 3) Uses the macros from #2 to implement pcibus_to_node() in > asm-i386/topology.h > > [mcd@arrakis patches]$ diffstat api_patches/pcibus_to_node-2.5.44.patch > arch/i386/pci/numa.c | 33 +++++++++++++++------------------ > include/asm-generic/topology.h | 3 +++ > include/asm-i386/topology.h | 3 +++ > 3 files changed, 21 insertions(+), 18 deletions(-) > > Cheers! > > -Matt > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [patch] pcibus_to_node() addition to topology infrastructure 2002-10-31 0:03 ` Jesse Barnes @ 2002-10-31 0:24 ` Matthew Dobson 2002-10-31 0:59 ` Jesse Barnes 0 siblings, 1 reply; 6+ messages in thread From: Matthew Dobson @ 2002-10-31 0:24 UTC (permalink / raw) To: Jesse Barnes; +Cc: linux-kernel Jesse Barnes wrote: > This is a nice addition, but just FYI there are SGI systems that have > PCI busses attached to more than one node. I guess for now we can just > round-robin through the attached nodes for the return value. > > Thanks, > Jesse Ah, yes... The p-bricks, i-bricks, etc. right? Yes, I suppose a round-robin return for the SGI version of the macro would work... Certainly not ideal, but it would work. The problem is if you assume that binding several processes to the node PCI bus X is on will ensure that those processes are on the same node. In this case (SGI), it would be a false assumption. I suppose that's what you get for assuming, though? ;) Cheers! -Matt ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [patch] pcibus_to_node() addition to topology infrastructure 2002-10-31 0:24 ` Matthew Dobson @ 2002-10-31 0:59 ` Jesse Barnes 2002-10-31 1:14 ` Matthew Dobson 0 siblings, 1 reply; 6+ messages in thread From: Jesse Barnes @ 2002-10-31 0:59 UTC (permalink / raw) To: Matthew Dobson; +Cc: linux-kernel On Wed, Oct 30, 2002 at 04:24:13PM -0800, Matthew Dobson wrote: > Ah, yes... The p-bricks, i-bricks, etc. right? Yup. > Yes, I suppose a round-robin return for the SGI version of the macro > would work... Certainly not ideal, but it would work. The problem is Can you think of any better way to do it? Perhaps make pcibus_to_node return a list of nodes? Thanks, Jesse ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [patch] pcibus_to_node() addition to topology infrastructure 2002-10-31 0:59 ` Jesse Barnes @ 2002-10-31 1:14 ` Matthew Dobson 0 siblings, 0 replies; 6+ messages in thread From: Matthew Dobson @ 2002-10-31 1:14 UTC (permalink / raw) To: Jesse Barnes; +Cc: linux-kernel Jesse Barnes wrote: > On Wed, Oct 30, 2002 at 04:24:13PM -0800, Matthew Dobson wrote: > >>Ah, yes... The p-bricks, i-bricks, etc. right? > > > Yup. > > >>Yes, I suppose a round-robin return for the SGI version of the macro >>would work... Certainly not ideal, but it would work. The problem is > > > Can you think of any better way to do it? Perhaps make pcibus_to_node > return a list of nodes? Yeah... I was thinking about having it return a bitmask... It makes it a bit of a pain to iterate through the mask, only to find that in most cases there is only one bit set in the mask. :( It's a bit late in the day... I'll mull this over and see if I can come up with something good. Cheers! -Matt ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2002-10-31 1:12 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2002-10-30 23:42 [patch] pcibus_to_node() addition to topology infrastructure Matthew Dobson 2002-10-30 23:44 ` Matthew Dobson 2002-10-31 0:03 ` Jesse Barnes 2002-10-31 0:24 ` Matthew Dobson 2002-10-31 0:59 ` Jesse Barnes 2002-10-31 1:14 ` Matthew Dobson
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox