[PATCH 1/4] try parent numa_node at first before using default

[PATCH 1/4] try parent numa_node at first before using default

[Yinghai Lu]

[PATCH 1/4] try parent numa_node at first before using default

Signed-off-by: Yinghai Lu <yinghai.lu@sun.com>

diff --git a/drivers/base/core.c b/drivers/base/core.c
index dd40d78..c344d82 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -547,6 +547,8 @@ static void klist_children_put(struct klist_node *n)
 
 void device_initialize(struct device *dev)
 {
+	int node;
+
 	kobj_set_kset_s(dev, devices_subsys);
 	kobject_init(&dev->kobj);
 	klist_init(&dev->klist_children, klist_children_get,
@@ -557,7 +559,9 @@ void device_initialize(struct device *dev)
 	spin_lock_init(&dev->devres_lock);
 	INIT_LIST_HEAD(&dev->devres_head);
 	device_init_wakeup(dev, 0);
-	set_dev_node(dev, -1);
+
+	node = dev->parent ? dev_to_node(dev->parent) : -1;
+	set_dev_node(dev, node);
 }
 
 #ifdef CONFIG_SYSFS_DEPRECATED

Re: [PATCH 1/4] try parent numa_node at first before using default

[Greg KH]

On Tue, Jul 03, 2007 at 05:31:02PM -0700, Yinghai Lu wrote:
> [PATCH 1/4] try parent numa_node at first before using default
> 
> Signed-off-by: Yinghai Lu <yinghai.lu@sun.com>
> 
> diff --git a/drivers/base/core.c b/drivers/base/core.c
> index dd40d78..c344d82 100644
> --- a/drivers/base/core.c
> +++ b/drivers/base/core.c
> @@ -547,6 +547,8 @@ static void klist_children_put(struct klist_node *n)
>  
>  void device_initialize(struct device *dev)
>  {
> +	int node;
> +
>  	kobj_set_kset_s(dev, devices_subsys);
>  	kobject_init(&dev->kobj);
>  	klist_init(&dev->klist_children, klist_children_get,
> @@ -557,7 +559,9 @@ void device_initialize(struct device *dev)
>  	spin_lock_init(&dev->devres_lock);
>  	INIT_LIST_HEAD(&dev->devres_head);
>  	device_init_wakeup(dev, 0);
> -	set_dev_node(dev, -1);
> +
> +	node = dev->parent ? dev_to_node(dev->parent) : -1;
> +	set_dev_node(dev, node);
>  }

What is this going to give us?

Doesn't it mean that all devices will end up on the same node?

thanks,

greg k-h

Re: [PATCH 1/4] try parent numa_node at first before using default

[Christoph Lameter]

On Mon, 9 Jul 2007, Greg KH wrote:

> > @@ -547,6 +547,8 @@ static void klist_children_put(struct klist_node *n)
> >  
> >  void device_initialize(struct device *dev)
> >  {
> > +	int node;
> > +
> >  	kobj_set_kset_s(dev, devices_subsys);
> >  	kobject_init(&dev->kobj);
> >  	klist_init(&dev->klist_children, klist_children_get,
> > @@ -557,7 +559,9 @@ void device_initialize(struct device *dev)
> >  	spin_lock_init(&dev->devres_lock);
> >  	INIT_LIST_HEAD(&dev->devres_head);
> >  	device_init_wakeup(dev, 0);
> > -	set_dev_node(dev, -1);
> > +
> > +	node = dev->parent ? dev_to_node(dev->parent) : -1;
> > +	set_dev_node(dev, node);
> >  }
> 
> What is this going to give us?
> 
> Doesn't it mean that all devices will end up on the same node?

It means that the node for the device is the same as the parent device.
F.e. if the parent device is a bus that is connected to node 4 then all 
the devices hooked up to the bus are allocated on that node.

Re: [PATCH 1/4] try parent numa_node at first before using default

[Greg KH]

On Mon, Jul 09, 2007 at 05:52:34PM -0700, Christoph Lameter wrote:
> On Mon, 9 Jul 2007, Greg KH wrote:
> 
> > > @@ -547,6 +547,8 @@ static void klist_children_put(struct klist_node *n)
> > >  
> > >  void device_initialize(struct device *dev)
> > >  {
> > > +	int node;
> > > +
> > >  	kobj_set_kset_s(dev, devices_subsys);
> > >  	kobject_init(&dev->kobj);
> > >  	klist_init(&dev->klist_children, klist_children_get,
> > > @@ -557,7 +559,9 @@ void device_initialize(struct device *dev)
> > >  	spin_lock_init(&dev->devres_lock);
> > >  	INIT_LIST_HEAD(&dev->devres_head);
> > >  	device_init_wakeup(dev, 0);
> > > -	set_dev_node(dev, -1);
> > > +
> > > +	node = dev->parent ? dev_to_node(dev->parent) : -1;
> > > +	set_dev_node(dev, node);
> > >  }
> > 
> > What is this going to give us?
> > 
> > Doesn't it mean that all devices will end up on the same node?
> 
> It means that the node for the device is the same as the parent device.
> F.e. if the parent device is a bus that is connected to node 4 then all 
> the devices hooked up to the bus are allocated on that node.

Yes, but is someone setting the parent device node information properly?

And this really needs some more changelog information please.  Why is
this needed from how things are done today?

thanks,

greg k-h

Re: [PATCH 1/4] try parent numa_node at first before using default

[Yinghai Lu]

On 7/9/07, Greg KH <greg@kroah.com> wrote:
> On Mon, Jul 09, 2007 at 05:52:34PM -0700, Christoph Lameter wrote:
> > It means that the node for the device is the same as the parent device.
> > F.e. if the parent device is a bus that is connected to node 4 then all
> > the devices hooked up to the bus are allocated on that node.
>
> Yes, but is someone setting the parent device node information properly?

> And this really needs some more changelog information please.  Why is
> this needed from how things are done today?

please check if you are happy with the changlog

----------------------------------------------------begin---------------------
For pci_device, pcibios_scan_root and pci_scan_root will call pci_device_add.
pci_device_add will call device_initialize and set_dev_node(&dev->dev,
pcibus_to_node(bus)).
other device such as netdev, and usb_device, set_dev_node is never be
used. So that field numa_node always is -1.

So for netdev, it will need to use dev->parent to get pci_device to
use it's numa_node. esp in netdev_alloc_skb()
not sure how other device such as infiniband do that.

Actually before patch
[PATCH 1/2] x86_64: get mp_bus_to_node as early
there is a bug about squence of bus->sysdata and using pcibus_to_node.
the numa_node of pci_dev->dev is never set correctly...always 0.

So some device have to use pcibus_to_node(to_pci_dev(dev)->bus) directly
such as dma_alloc_pages in arch/x86_64/kernel/pci-dma.c.
or hwif_to_node in include/linux/ide.h

with this patch, we could use device->numa_node direclty for all device.
-------------------------------------------------end-----------



YH

Re: [PATCH 1/4] try parent numa_node at first before using default

[Yinghai Lu]

Greg KH wrote:
> On Tue, Jul 03, 2007 at 05:31:02PM -0700, Yinghai Lu wrote:
>> [PATCH 1/4] try parent numa_node at first before using default
>>
> 
> What is this going to give us?
you can use dev_to_node(netdev->dev) or dev_to_node(usb_device->dev) directly, and don't need to go to find pci_device up to get numa_node.
> 
> Doesn't it mean that all devices will end up on the same node?
yes.
For x86_64, devices under node1 will have 1, and device under node0 will all have 0.

current only pci_dev->dev.numa_node is used.

YH