borked sysfs system devices in 2.5.72

borked sysfs system devices in 2.5.72

[Dave Hansen]

The per-node numa meminfo files in 2.5.72 are broken, the only display
node0's information.  The devices are being properly registered:
Registering sys device 'node0':c0423844 id:0 kobj:c042384c
Registering sys device 'node1':c0423888 id:1 kobj:c0423890
Registering sys device 'node2':c04238cc id:2 kobj:c04238d4
Registering sys device 'node3':c0423910 id:3 kobj:c0423918

When I look at the 4 nodes files with:
"cat /sys/devices/system/node/*/meminfo", I printed out some
information:
subsys_attr_show(kobj: c042384c, attr: c033ea30, page: e76ba000)
subsys_attr_show(kobj: c0423890, attr: c033ea30, page: e76ba000)
subsys_attr_show(kobj: c04238d4, attr: c033ea30, page: e76ba000)
subsys_attr_show(kobj: c0423918, attr: c033ea30, page: e76ba000)

As you can see, the kobj is the one which belongs to the sys device, yet
you do a to_subsys() on it.  Why?
struct subsystem * s = to_subsys(kobj);

I'm getting a 0 as the node ID out of pure dumb luck.  Is the NUMA code
broken or is sysfs?

-- 
Dave Hansen
haveblue@us.ibm.com

Re: borked sysfs system devices in 2.5.72

[Patrick Mochel]

On 17 Jun 2003, Dave Hansen wrote:

> The per-node numa meminfo files in 2.5.72 are broken, the only display
> node0's information.  The devices are being properly registered:
> Registering sys device 'node0':c0423844 id:0 kobj:c042384c
> Registering sys device 'node1':c0423888 id:1 kobj:c0423890
> Registering sys device 'node2':c04238cc id:2 kobj:c04238d4
> Registering sys device 'node3':c0423910 id:3 kobj:c0423918
> 
> When I look at the 4 nodes files with:
> "cat /sys/devices/system/node/*/meminfo", I printed out some
> information:
> subsys_attr_show(kobj: c042384c, attr: c033ea30, page: e76ba000)
> subsys_attr_show(kobj: c0423890, attr: c033ea30, page: e76ba000)
> subsys_attr_show(kobj: c04238d4, attr: c033ea30, page: e76ba000)
> subsys_attr_show(kobj: c0423918, attr: c033ea30, page: e76ba000)
> 
> As you can see, the kobj is the one which belongs to the sys device, yet
> you do a to_subsys() on it.  Why?
> struct subsystem * s = to_subsys(kobj);

Where exactly is that happening? 

> I'm getting a 0 as the node ID out of pure dumb luck.  Is the NUMA code
> broken or is sysfs?

It's taking the ID from the system device of the node that's passed to 
->show(). That is set in register_node(), so I'm not sure how they could 
get out of sync, unless I'm missing something obvious. 

BTW, I did request that the NUMA topology people take a look at these 
patches when I sent them a couple of weeks ago, so as to avoid this. :) 



	-pat

Re: borked sysfs system devices in 2.5.72

[Dave Hansen]

On Tue, 2003-06-17 at 15:26, Patrick Mochel wrote:
> On 17 Jun 2003, Dave Hansen wrote:
> 
> > The per-node numa meminfo files in 2.5.72 are broken, the only display
> > node0's information.  The devices are being properly registered:
> > Registering sys device 'node0':c0423844 id:0 kobj:c042384c
> > Registering sys device 'node1':c0423888 id:1 kobj:c0423890
> > Registering sys device 'node2':c04238cc id:2 kobj:c04238d4
> > Registering sys device 'node3':c0423910 id:3 kobj:c0423918
> > 
> > When I look at the 4 nodes files with:
> > "cat /sys/devices/system/node/*/meminfo", I printed out some
> > information:
> > subsys_attr_show(kobj: c042384c, attr: c033ea30, page: e76ba000)
> > subsys_attr_show(kobj: c0423890, attr: c033ea30, page: e76ba000)
> > subsys_attr_show(kobj: c04238d4, attr: c033ea30, page: e76ba000)
> > subsys_attr_show(kobj: c0423918, attr: c033ea30, page: e76ba000)
> > 
> > As you can see, the kobj is the one which belongs to the sys device, yet
> > you do a to_subsys() on it.  Why?
> > struct subsystem * s = to_subsys(kobj);
> 
> Where exactly is that happening? 

Look in subsys_attr_show().  It is being passed a kobject, which is a
member of a "struct sys_device".  We can tell this because I printed out
the address of the sys device in sys_device_register().  A to_subsys()
is being performed on that object, which is wrong, because the kobject
is not a member of a "struct subsystem".

> > I'm getting a 0 as the node ID out of pure dumb luck.  Is the NUMA code
> > broken or is sysfs?
> 
> It's taking the ID from the system device of the node that's passed to 
> ->show(). That is set in register_node(), so I'm not sure how they could 
> get out of sync, unless I'm missing something obvious. 

They're not out of sync, it's getting garbage because of the bogus
to_subsys().

> BTW, I did request that the NUMA topology people take a look at these 
> patches when I sent them a couple of weeks ago, so as to avoid this. :) 

I'll cc him to make him feel bad :)

-- 
Dave Hansen
haveblue@us.ibm.com

Re: borked sysfs system devices in 2.5.72

[Patrick Mochel]

> Look in subsys_attr_show().  It is being passed a kobject, which is a
> member of a "struct sys_device".  We can tell this because I printed out
> the address of the sys device in sys_device_register().  A to_subsys()
> is being performed on that object, which is wrong, because the kobject
> is not a member of a "struct subsystem".

My question was how the hell it was getting there in the first place, and 
I see that the type of the object isn't getting set properly, so it 
defaults to treat it as a struct subsystem. 

Could you please try the following patch, and let me know if it works? 

Thanks,


	-pat

===== drivers/base/sys.c 1.25 vs edited =====
--- 1.25/drivers/base/sys.c	Mon Jun 16 10:07:04 2003
+++ edited/drivers/base/sys.c	Tue Jun 17 15:50:48 2003
@@ -170,6 +172,9 @@
 
 	/* Make sure the kset is set */
 	sysdev->kobj.kset = &cls->kset;
+
+	/* But make sure we point to the right type for sysfs translation */
+	sysdev->kobj.ktype = &ktype_sysdev;
 
 	/* set the kobject name */
 	snprintf(sysdev->kobj.name,KOBJ_NAME_LEN,"%s%d",

Re: borked sysfs system devices in 2.5.72

[Dave Hansen]

On Tue, 2003-06-17 at 15:54, Patrick Mochel wrote:
> > Look in subsys_attr_show().  It is being passed a kobject, which is a
> > member of a "struct sys_device".  We can tell this because I printed out
> > the address of the sys device in sys_device_register().  A to_subsys()
> > is being performed on that object, which is wrong, because the kobject
> > is not a member of a "struct subsystem".
> 
> My question was how the hell it was getting there in the first place, and 
> I see that the type of the object isn't getting set properly, so it 
> defaults to treat it as a struct subsystem. 

Stack dump from si_meminfo_node():
Call Trace:
 [<c0133f39>] si_meminfo_node+0x4d/0x54
 [<c02029ac>] node_read_meminfo+0x1c/0x80
 [<c013385a>] __alloc_pages+0x82/0x2b4
 [<c011c8fb>] release_console_sem+0x9b/0xa4
 [<c0175ead>] subsys_attr_show+0x1d/0x28
 [<c0175f7a>] fill_read_buffer+0x96/0xb4
 [<c01ebf5e>] opost_block+0x18e/0x19c
 [<c01eed92>] pty_write+0x156/0x168
 [<c0154d8c>] do_lookup+0x18/0x8c
 [<c0151a2f>] cp_new_stat64+0xe7/0x100
 [<c017604b>] sysfs_read_file+0x1b/0x3c
 [<c014960c>] vfs_read+0x9c/0xcc
 [<c01497ed>] sys_read+0x31/0x4c
 [<c0108bd7>] syscall_call+0x7/0xb


> Could you please try the following patch, and let me know if it works? 

That fixed it, thanks.  

-- 
Dave Hansen
haveblue@us.ibm.com