kref refcounting breakage in mainline

kref refcounting breakage in mainline

[Andrew Morton]

-mm has a debugging patch which warns when atomic_dec_and_test() takes an
atomic_t negative
(ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.20/2.6.20-mm2/broken-out/detect-atomic-counter-underflows.patch).


When it is applied to current mainline, a simple `rmmod ipw2200' gives:

[   75.825072] BUG: atomic counter underflow at:
[   75.825180]  [<c01c6eb4>] kref_put+0x66/0x82
[   75.825278]  [<c022e4d4>] bus_remove_driver+0x66/0x75
[   75.825383]  [<c022ee2c>] driver_unregister+0x8/0x13
[   75.825484]  [<c01d7add>] pci_unregister_driver+0xc/0x45
[   75.825593]  [<c0132147>] sys_delete_module+0x157/0x17c
[   75.825703]  [<c013c663>] audit_syscall_entry+0x10d/0x137
[   75.825818]  [<c0103b14>] syscall_call+0x7/0xb
[   75.825913]  [<c02d0000>] xfrm4_dst_destroy+0xe/0xd5

This didn't happen in 2.6.20-mm2, so this bug was introduced by a patch
which was not in the -mm lineup twelve days ago.

Presumably the effect of this is a memory leak or a use-after-free.

Re: kref refcounting breakage in mainline

[Greg KH]

On Fri, Mar 02, 2007 at 12:58:33AM -0800, Andrew Morton wrote:
> 
> -mm has a debugging patch which warns when atomic_dec_and_test() takes an
> atomic_t negative
> (ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.20/2.6.20-mm2/broken-out/detect-atomic-counter-underflows.patch).
> 
> 
> When it is applied to current mainline, a simple `rmmod ipw2200' gives:
> 
> [   75.825072] BUG: atomic counter underflow at:
> [   75.825180]  [<c01c6eb4>] kref_put+0x66/0x82
> [   75.825278]  [<c022e4d4>] bus_remove_driver+0x66/0x75
> [   75.825383]  [<c022ee2c>] driver_unregister+0x8/0x13
> [   75.825484]  [<c01d7add>] pci_unregister_driver+0xc/0x45
> [   75.825593]  [<c0132147>] sys_delete_module+0x157/0x17c
> [   75.825703]  [<c013c663>] audit_syscall_entry+0x10d/0x137
> [   75.825818]  [<c0103b14>] syscall_call+0x7/0xb
> [   75.825913]  [<c02d0000>] xfrm4_dst_destroy+0xe/0xd5
> 
> This didn't happen in 2.6.20-mm2, so this bug was introduced by a patch
> which was not in the -mm lineup twelve days ago.
> 
> Presumably the effect of this is a memory leak or a use-after-free.

Ugh.

I'll add it to my local tree here and try to bisect to find the problem.

thanks for the pointer,

greg k-h

Re: kref refcounting breakage in mainline

[Greg KH]

On Fri, Mar 02, 2007 at 12:58:33AM -0800, Andrew Morton wrote:
> 
> -mm has a debugging patch which warns when atomic_dec_and_test() takes an
> atomic_t negative
> (ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.20/2.6.20-mm2/broken-out/detect-atomic-counter-underflows.patch).
> 
> 
> When it is applied to current mainline, a simple `rmmod ipw2200' gives:
> 
> [   75.825072] BUG: atomic counter underflow at:
> [   75.825180]  [<c01c6eb4>] kref_put+0x66/0x82
> [   75.825278]  [<c022e4d4>] bus_remove_driver+0x66/0x75
> [   75.825383]  [<c022ee2c>] driver_unregister+0x8/0x13
> [   75.825484]  [<c01d7add>] pci_unregister_driver+0xc/0x45
> [   75.825593]  [<c0132147>] sys_delete_module+0x157/0x17c
> [   75.825703]  [<c013c663>] audit_syscall_entry+0x10d/0x137
> [   75.825818]  [<c0103b14>] syscall_call+0x7/0xb
> [   75.825913]  [<c02d0000>] xfrm4_dst_destroy+0xe/0xd5
> 
> This didn't happen in 2.6.20-mm2, so this bug was introduced by a patch
> which was not in the -mm lineup twelve days ago.
> 
> Presumably the effect of this is a memory leak or a use-after-free.

Ok, after a zillion bisects, I've tracked this down to:
	commit 63ce18cfe685115ff8d341bae4c9204a79043cf0
	Author: Mike Galbraith <efault@gmx.de>
	Date:   Wed Feb 21 12:45:35 2007 -0800

	    driver core: refcounting fix
	    
	    Fix a reference counting bug exposed by commit
	    725522b5453dd680412f2b6463a988e4fd148757.  If driver.mod_name exists, we
	    take a reference in module_add_driver(), and never release it.  Undo that
	    reference in module_remove_driver().
	    
	    Signed-off-by: Mike Galbraith <efault@gmx.de>
	    Cc: Kay Sievers <kay.sievers@vrfy.org>
	    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
	    Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
	

Mike, I've reverted this patch, and I don't see any references leaking.
And, as your patch released the reference on the driver, and the
module_add_driver() call would not grab a reference to the driver, only
the module kobject, I don't see what you were trying to fix with this
patch.

Do you have a test case that this fixes?

Otherwise, I'll just revert it.

thanks,

greg k-h

Re: kref refcounting breakage in mainline

[Mike Galbraith]

On Mon, 2007-03-05 at 16:25 -0800, Greg KH wrote:

> Mike, I've reverted this patch, and I don't see any references leaking.
> And, as your patch released the reference on the driver, and the
> module_add_driver() call would not grab a reference to the driver, only
> the module kobject, I don't see what you were trying to fix with this
> patch.
> 
> Do you have a test case that this fixes?

What it fixed for me was the hard hang reported below.

http://lkml.org/lkml/2007/2/16/96


	-Mike

Re: kref refcounting breakage in mainline

[Mel Gorman]

On (05/03/07 16:25), Greg KH didst pronounce:
> On Fri, Mar 02, 2007 at 12:58:33AM -0800, Andrew Morton wrote:
> > 
> > -mm has a debugging patch which warns when atomic_dec_and_test() takes an
> > atomic_t negative
> > (ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.20/2.6.20-mm2/broken-out/detect-atomic-counter-underflows.patch).
> > 
> > 
> > When it is applied to current mainline, a simple `rmmod ipw2200' gives:
> > 
> > [   75.825072] BUG: atomic counter underflow at:
> > [   75.825180]  [<c01c6eb4>] kref_put+0x66/0x82
> > [   75.825278]  [<c022e4d4>] bus_remove_driver+0x66/0x75
> > [   75.825383]  [<c022ee2c>] driver_unregister+0x8/0x13
> > [   75.825484]  [<c01d7add>] pci_unregister_driver+0xc/0x45
> > [   75.825593]  [<c0132147>] sys_delete_module+0x157/0x17c
> > [   75.825703]  [<c013c663>] audit_syscall_entry+0x10d/0x137
> > [   75.825818]  [<c0103b14>] syscall_call+0x7/0xb
> > [   75.825913]  [<c02d0000>] xfrm4_dst_destroy+0xe/0xd5
> > 
> > This didn't happen in 2.6.20-mm2, so this bug was introduced by a patch
> > which was not in the -mm lineup twelve days ago.
> > 
> > Presumably the effect of this is a memory leak or a use-after-free.
> 
> Ok, after a zillion bisects, I've tracked this down to:
> 	commit 63ce18cfe685115ff8d341bae4c9204a79043cf0
> 	Author: Mike Galbraith <efault@gmx.de>
> 	Date:   Wed Feb 21 12:45:35 2007 -0800
> 
> 	    driver core: refcounting fix
> 	    
> 	    Fix a reference counting bug exposed by commit
> 	    725522b5453dd680412f2b6463a988e4fd148757.  If driver.mod_name exists, we
> 	    take a reference in module_add_driver(), and never release it.  Undo that
> 	    reference in module_remove_driver().
> 	    
> 	    Signed-off-by: Mike Galbraith <efault@gmx.de>
> 	    Cc: Kay Sievers <kay.sievers@vrfy.org>
> 	    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> 	    Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
> 	
> 

On http://test.kernel.org, elm3b132 is showing a similar error message for
the IPS driver on 2.6.21-rc2-mm1. There is no such device in the machine so
this is a normal error path for no devices found. The warning looks like

BUG: atomic counter underflow at:
 [<c027ccb4>] kref_put+0x7d/0x9d
 [<c02a4e38>] bus_remove_driver+0x36/0x41
 [<c02a5997>] driver_unregister+0xb/0x13
 [<c0286bdd>] pci_unregister_driver+0xb/0x13
 [<c04c653a>] ips_module_init+0x41/0x57
 [<c04b2bd2>] do_initcalls+0x58/0xf5
 [<c01353d5>] register_irq_proc+0x75/0x92
 [<c04b2c97>] init+0x0/0x8b
 [<c04b2ce0>] init+0x49/0x8b
 [<c01030ff>] kernel_thread_helper+0x7/0x10

This is essentially identical to the warning on ipw2200. It occurs whether
the driver is compiled into the kernel or as a module.  Reverting Mike's
patch fixes the problem - or at least the warning has disappeared.

However, I've cc'd the IPS maintainers so they can confirm they are
calling pci_unregister_driver() correctly. I note that many drivers in
drivers/scsi do not call pci_unregister_driver() in the module_init path.
ipw2200 also calls pci_unregister_driver() in the module_init path when
an error is encountered.

So, maybe this is an error in how the drivers use pci_[un]register_driver()
instead of a problem with Mike's patch?

> Mike, I've reverted this patch, and I don't see any references leaking.
> And, as your patch released the reference on the driver, and the
> module_add_driver() call would not grab a reference to the driver, only
> the module kobject, I don't see what you were trying to fix with this
> patch.
> 
> Do you have a test case that this fixes?
> 
> Otherwise, I'll just revert it.
> 
> thanks,
> 
> greg k-h
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

Re: kref refcounting breakage in mainline

[Greg KH]

On Tue, Mar 06, 2007 at 06:43:22AM +0100, Mike Galbraith wrote:
> On Mon, 2007-03-05 at 16:25 -0800, Greg KH wrote:
> 
> > Mike, I've reverted this patch, and I don't see any references leaking.
> > And, as your patch released the reference on the driver, and the
> > module_add_driver() call would not grab a reference to the driver, only
> > the module kobject, I don't see what you were trying to fix with this
> > patch.
> > 
> > Do you have a test case that this fixes?
> 
> What it fixed for me was the hard hang reported below.
> 
> http://lkml.org/lkml/2007/2/16/96

What specific module are you trying to unload that causes the hang?  I
think it might just be a problem with that module, and not with all
others.

So, I'm going to revert your patch and work to try to find the real
cause of this problem.

thanks,

greg k-h

Re: kref refcounting breakage in mainline

[Mike Galbraith]

On Tue, 2007-03-06 at 13:04 -0800, Greg KH wrote:
> On Tue, Mar 06, 2007 at 06:43:22AM +0100, Mike Galbraith wrote:
> > On Mon, 2007-03-05 at 16:25 -0800, Greg KH wrote:
> > 
> > > Mike, I've reverted this patch, and I don't see any references leaking.
> > > And, as your patch released the reference on the driver, and the
> > > module_add_driver() call would not grab a reference to the driver, only
> > > the module kobject, I don't see what you were trying to fix with this
> > > patch.
> > > 
> > > Do you have a test case that this fixes?
> > 
> > What it fixed for me was the hard hang reported below.
> > 
> > http://lkml.org/lkml/2007/2/16/96
> 
> What specific module are you trying to unload that causes the hang?  I
> think it might just be a problem with that module, and not with all
> others.

It's ipmi_si that's hanging, waits for completion that never comes.

> So, I'm going to revert your patch and work to try to find the real
> cause of this problem.

Yeah, my stab at it seems busted.  I'll take another poke at it to see
if I can find out why (post 725522b5453dd680412f2b6463a988e4fd148757)
I'm left with a reference.

	-Mike

Re: kref refcounting breakage in mainline

[Mike Galbraith]

On Wed, 2007-03-07 at 06:39 +0100, Mike Galbraith wrote:
> On Tue, 2007-03-06 at 13:04 -0800, Greg KH wrote:
> > On Tue, Mar 06, 2007 at 06:43:22AM +0100, Mike Galbraith wrote:
> > > On Mon, 2007-03-05 at 16:25 -0800, Greg KH wrote:
> > > 
> > > > Mike, I've reverted this patch, and I don't see any references leaking.
> > > > And, as your patch released the reference on the driver, and the
> > > > module_add_driver() call would not grab a reference to the driver, only
> > > > the module kobject, I don't see what you were trying to fix with this
> > > > patch.
> > > > 
> > > > Do you have a test case that this fixes?
> > > 
> > > What it fixed for me was the hard hang reported below.
> > > 
> > > http://lkml.org/lkml/2007/2/16/96
> > 
> > What specific module are you trying to unload that causes the hang?  I
> > think it might just be a problem with that module, and not with all
> > others.
> 
> It's ipmi_si that's hanging, waits for completion that never comes.
> 
> > So, I'm going to revert your patch and work to try to find the real
> > cause of this problem.
> 
> Yeah, my stab at it seems busted.  I'll take another poke at it to see
> if I can find out why (post 725522b5453dd680412f2b6463a988e4fd148757)
> I'm left with a reference.

Ok, stab #2.

My reference count woes stem from module_remove_driver() not removing
the link created in module_add_driver().  With the below, my box boots
fine.  Since I obviously know spit about driver layer glue, I'll just
call this one a diagnostic, and head for the hills :)

--- linux-2.6.20-rc3/kernel/module.c.org	2007-03-10 15:16:47.000000000 +0100
+++ linux-2.6.20-rc3/kernel/module.c	2007-03-10 15:43:09.000000000 +0100
@@ -2411,14 +2411,28 @@ void module_remove_driver(struct device_
 		return;
 
 	sysfs_remove_link(&drv->kobj, "module");
-	if (drv->owner && drv->owner->mkobj.drivers_dir) {
-		driver_name = make_driver_name(drv);
-		if (driver_name) {
-			sysfs_remove_link(drv->owner->mkobj.drivers_dir,
+	driver_name = make_driver_name(drv);
+	if (!driver_name)
+		return;
+	if (drv->owner && drv->owner->mkobj.drivers_dir)
+		sysfs_remove_link(drv->owner->mkobj.drivers_dir,
 					  driver_name);
-			kfree(driver_name);
-		}
+	else if (drv->mod_name) {
+		struct module_kobject *mk;
+		struct kobject *mkobj;
+
+		/* Lookup built-in module entry in /sys/modules */
+		mkobj = kset_find_obj(&module_subsys.kset, drv->mod_name);
+		if (!mkobj)
+			goto out_free;
+		mk = container_of(mkobj, struct module_kobject, kobj);
+		module_create_drivers_dir(mk);
+		sysfs_remove_link(mk->drivers_dir, driver_name);
+		/* Release reference taken via lookup */
+		kobject_put(mkobj);
 	}
+out_free:
+	kfree(driver_name);
 }
 EXPORT_SYMBOL(module_remove_driver);
 #endif

Re: kref refcounting breakage in mainline

[Mike Galbraith]

P.S.  forgot to include diagnostic log.  Kobject c0644890 is the source
of my woes.  Printk's come below WARN_ON(is_ipmi_si_kobj).  Post-tinker
log is huge, and probably not interesting.

[   30.397160] kobject ipmi_devintf: registering. parent: <NULL>, set: module
[   30.404033] kobject_uevent_env
[   30.407098] fill_kobj_path: path = '/module/ipmi_devintf'
[   30.412524] BUG: at lib/kobject.c:448 kobject_get()
[   30.417402]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   30.422564]  [<c01057b2>] show_trace+0x12/0x14
[   30.427031]  [<c0105856>] dump_stack+0x16/0x18
[   30.431501]  [<c02d3c2b>] kobject_get+0x66/0x87
[   30.436056]  [<c02d3f1e>] kobject_shadow_add+0x10/0x1e8
[   30.441312]  [<c02d4100>] kobject_add+0xa/0xc
[   30.445695]  [<c06804c0>] kernel_param_sysfs_setup+0x4d/0x7f
[   30.451376]  [<c068067a>] param_sysfs_init+0x188/0x1c3
[   30.456538]  [<c06725bc>] init+0x144/0x26c
[   30.460661]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   30.465907]  =======================
[   30.469486] get: c18f65c0 count after get is 2
[   30.473927] kobject ipmi_si: registering. parent: <NULL>, set: module
[   30.480372] kobject_uevent_env
[   30.483430] fill_kobj_path: path = '/module/ipmi_si'


......


[   73.266556] bus platform: add driver ipmi
[   73.278013] kobject ipmi: registering. parent: <NULL>, set: drivers
[   73.291847] kobject_uevent_env
[   73.302358] fill_kobj_path: path = '/bus/platform/drivers/ipmi'
[   73.315943] ipmi message handler version 39.1
[   73.327839] ipmi device interface
[   73.338524] device class 'ipmi': registering
[   73.350158] subsystem ipmi: registering
[   73.361309] kobject ipmi: registering. parent: <NULL>, set: class
[   73.374841] bus platform: add driver ipmi_si
[   73.386442] BUG: at lib/kobject.c:448 kobject_get()
[   73.398617]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   73.411079]  [<c01057b2>] show_trace+0x12/0x14
[   73.422780]  [<c0105856>] dump_stack+0x16/0x18
[   73.434324]  [<c02d3c2b>] kobject_get+0x66/0x87
[   73.445860]  [<c02d3f1e>] kobject_shadow_add+0x10/0x1e8
[   73.458088]  [<c02d4100>] kobject_add+0xa/0xc
[   73.469286]  [<c02d4248>] kobject_register+0x22/0xb3
[   73.480986]  [<c035a13e>] bus_add_driver+0x77/0x1ae
[   73.492592]  [<c035b109>] driver_register+0x54/0x84
[   73.504101]  [<c034c486>] init_ipmi_si+0x4d/0x829
[   73.515335]  [<c06725bc>] init+0x144/0x26c
[   73.525822]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   73.537452]  =======================
[   73.547290] get: c064475c count after get is 2
[   73.557969] kobject ipmi_si: registering. parent: <NULL>, set: drivers
[   73.570825] kobject_uevent_env
[   73.580011] fill_kobj_path: path = '/bus/platform/drivers/ipmi_si'
[   73.592622] BUG: at lib/kobject.c:242 kobject_register()
[   73.604312]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   73.615822]  [<c01057b2>] show_trace+0x12/0x14
[   73.626551]  [<c0105856>] dump_stack+0x16/0x18
[   73.637211]  [<c02d429f>] kobject_register+0x79/0xb3
[   73.648325]  [<c035a13e>] bus_add_driver+0x77/0x1ae
[   73.659297]  [<c035b109>] driver_register+0x54/0x84
[   73.670164]  [<c034c486>] init_ipmi_si+0x4d/0x829
[   73.680756]  [<c06725bc>] init+0x144/0x26c
[   73.690699]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   73.701854]  =======================
[   73.711295] register: c064475c count now is 2 error 0
[   73.722218] BUG: at lib/kobject.c:448 kobject_get()
[   73.732911]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   73.743927]  [<c01057b2>] show_trace+0x12/0x14
[   73.754112]  [<c0105856>] dump_stack+0x16/0x18
[   73.764253]  [<c02d3c2b>] kobject_get+0x66/0x87
[   73.774532]  [<c035b157>] get_driver+0x11/0x18
[   73.784725]  [<c035b194>] driver_create_file+0xf/0x32
[   73.795559]  [<c035a221>] bus_add_driver+0x15a/0x1ae
[   73.806343]  [<c035b109>] driver_register+0x54/0x84
[   73.817055]  [<c034c486>] init_ipmi_si+0x4d/0x829
[   73.827595]  [<c06725bc>] init+0x144/0x26c
[   73.837468]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   73.848544]  =======================
[   73.857932] get: c064475c count after get is 3
[   73.868223] BUG: at lib/kobject.c:494 kobject_put()
[   73.878943]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   73.890019]  [<c01057b2>] show_trace+0x12/0x14
[   73.900239]  [<c0105856>] dump_stack+0x16/0x18
[   73.910294]  [<c02d3ae7>] kobject_put+0x69/0x82
[   73.920304]  [<c035b144>] put_driver+0xb/0xd
[   73.929909]  [<c035b1b0>] driver_create_file+0x2b/0x32
[   73.940362]  [<c035a221>] bus_add_driver+0x15a/0x1ae
[   73.950686]  [<c035b109>] driver_register+0x54/0x84
[   73.960966]  [<c034c486>] init_ipmi_si+0x4d/0x829
[   73.971055]  [<c06725bc>] init+0x144/0x26c
[   73.980478]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   73.991119]  =======================
[   74.000050] put: c064475c count before put is 3
[   74.010026] BUG: at lib/kobject.c:448 kobject_get()
[   74.020367]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.031011]  [<c01057b2>] show_trace+0x12/0x14
[   74.040867]  [<c0105856>] dump_stack+0x16/0x18
[   74.050602]  [<c02d3c2b>] kobject_get+0x66/0x87
[   74.060353]  [<c035b157>] get_driver+0x11/0x18
[   74.070034]  [<c035b194>] driver_create_file+0xf/0x32
[   74.080418]  [<c035a233>] bus_add_driver+0x16c/0x1ae
[   74.090742]  [<c035b109>] driver_register+0x54/0x84
[   74.100979]  [<c034c486>] init_ipmi_si+0x4d/0x829
[   74.111041]  [<c06725bc>] init+0x144/0x26c
[   74.120431]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   74.131047]  =======================
[   74.139949] get: c064475c count after get is 3
[   74.149807] BUG: at lib/kobject.c:494 kobject_put()
[   74.160111]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.170739]  [<c01057b2>] show_trace+0x12/0x14
[   74.180569]  [<c0105856>] dump_stack+0x16/0x18
[   74.190286]  [<c02d3ae7>] kobject_put+0x69/0x82
[   74.200019]  [<c035b144>] put_driver+0xb/0xd
[   74.209503]  [<c035b1b0>] driver_create_file+0x2b/0x32
[   74.219982]  [<c035a233>] bus_add_driver+0x16c/0x1ae
[   74.230312]  [<c035b109>] driver_register+0x54/0x84
[   74.240550]  [<c034c486>] init_ipmi_si+0x4d/0x829
[   74.250614]  [<c06725bc>] init+0x144/0x26c
[   74.260001]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   74.270618]  =======================
[   74.279530] put: c064475c count before put is 3
[   74.289481] IPMI System Interface driver.
[   74.298864] bus pci: add driver ipmi_si
[   74.308007] BUG: at lib/kobject.c:448 kobject_get()
[   74.318260]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.328861]  [<c01057b2>] show_trace+0x12/0x14
[   74.338637]  [<c0105856>] dump_stack+0x16/0x18
[   74.348372]  [<c02d3c2b>] kobject_get+0x66/0x87
[   74.358245]  [<c02d3f1e>] kobject_shadow_add+0x10/0x1e8
[   74.368837]  [<c02d4100>] kobject_add+0xa/0xc
[   74.378528]  [<c02d4248>] kobject_register+0x22/0xb3
[   74.388817]  [<c035a13e>] bus_add_driver+0x77/0x1ae
[   74.399027]  [<c035b109>] driver_register+0x54/0x84
[   74.409229]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   74.420004]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   74.430196]  [<c06725bc>] init+0x144/0x26c
[   74.439611]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   74.450184]  =======================
[   74.459045] get: c0644890 count after get is 2
[   74.468796] kobject ipmi_si: registering. parent: <NULL>, set: drivers
[   74.480769] kobject_uevent_env
[   74.489210] fill_kobj_path: path = '/bus/pci/drivers/ipmi_si'
[   74.500675] BUG: at lib/kobject.c:242 kobject_register()
[   74.511779]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.522812]  [<c01057b2>] show_trace+0x12/0x14
[   74.533082]  [<c0105856>] dump_stack+0x16/0x18
[   74.543269]  [<c02d429f>] kobject_register+0x79/0xb3
[   74.554015]  [<c035a13e>] bus_add_driver+0x77/0x1ae
[   74.564685]  [<c035b109>] driver_register+0x54/0x84
[   74.575363]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   74.586595]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   74.597161]  [<c06725bc>] init+0x144/0x26c
[   74.606974]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   74.617989]  =======================
[   74.627292] register: c0644890 count now is 2 error 0
[   74.638150] BUG: at lib/kobject.c:448 kobject_get()
[   74.648812]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.659872]  [<c01057b2>] show_trace+0x12/0x14
[   74.670237]  [<c0105856>] dump_stack+0x16/0x18
[   74.680554]  [<c02d3c2b>] kobject_get+0x66/0x87
[   74.690962]  [<c02d434c>] kset_find_obj+0x73/0x77
[   74.701563]  [<c01442c5>] module_add_driver+0x64/0x7d
[   74.712578]  [<c035a170>] bus_add_driver+0xa9/0x1ae
[   74.723405]  [<c035b109>] driver_register+0x54/0x84
[   74.734247]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   74.745636]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   74.756393]  [<c06725bc>] init+0x144/0x26c
[   74.766429]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   74.777654]  =======================
[   74.787103] get: c18f65c0 count after get is 3
[   74.797461] BUG: at lib/kobject.c:448 kobject_get()
[   74.808209]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.819241]  [<c01057b2>] show_trace+0x12/0x14
[   74.829427]  [<c0105856>] dump_stack+0x16/0x18
[   74.839446]  [<c02d3c2b>] kobject_get+0x66/0x87
[   74.849465]  [<c01b4812>] sysfs_create_link+0xd3/0x151
[   74.860178]  [<c0144285>] module_add_driver+0x24/0x7d
[   74.870866]  [<c035a170>] bus_add_driver+0xa9/0x1ae
[   74.881370]  [<c035b109>] driver_register+0x54/0x84
[   74.891850]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   74.902918]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   74.913354]  [<c06725bc>] init+0x144/0x26c
[   74.923019]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   74.933904]  =======================
[   74.943042] get: c18f65c0 count after get is 4
[   74.953046] BUG: at lib/kobject.c:448 kobject_get()
[   74.963497]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.974262]  [<c01057b2>] show_trace+0x12/0x14
[   74.984220]  [<c0105856>] dump_stack+0x16/0x18
[   74.994120]  [<c02d3c2b>] kobject_get+0x66/0x87
[   75.004167]  [<c02d3f4a>] kobject_shadow_add+0x3c/0x1e8
[   75.014941]  [<c02d4100>] kobject_add+0xa/0xc
[   75.024787]  [<c02d4248>] kobject_register+0x22/0xb3
[   75.035216]  [<c02d4431>] kobject_add_dir+0x4f/0x6e
[   75.045511]  [<c01442d9>] module_add_driver+0x78/0x7d
[   75.055991]  [<c035a170>] bus_add_driver+0xa9/0x1ae
[   75.066262]  [<c035b109>] driver_register+0x54/0x84
[   75.076489]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   75.087297]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   75.097483]  [<c06725bc>] init+0x144/0x26c
[   75.106862]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   75.117454]  =======================
[   75.126306] get: c18f65c0 count after get is 5
[   75.136022] kobject drivers: registering. parent: ipmi_si, set: <NULL>
[   75.148068] kobject_uevent_env
[   75.156609] kobject filter function caused the event to drop!
[   75.168050] BUG: at lib/kobject.c:448 kobject_get()
[   75.178631]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   75.189595]  [<c01057b2>] show_trace+0x12/0x14
[   75.199789]  [<c0105856>] dump_stack+0x16/0x18
[   75.209905]  [<c02d3c2b>] kobject_get+0x66/0x87
[   75.220072]  [<c01b4812>] sysfs_create_link+0xd3/0x151
[   75.230853]  [<c01442a5>] module_add_driver+0x44/0x7d
[   75.241531]  [<c035a170>] bus_add_driver+0xa9/0x1ae
[   75.252030]  [<c035b109>] driver_register+0x54/0x84
[   75.262455]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   75.273444]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   75.283813]  [<c06725bc>] init+0x144/0x26c
[   75.293382]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   75.304189]  =======================
[   75.313370] get: c0644890 count after get is 3
[   75.323512] BUG: at lib/kobject.c:448 kobject_get()
[   75.334155]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   75.345215]  [<c01057b2>] show_trace+0x12/0x14
[   75.355547]  [<c0105856>] dump_stack+0x16/0x18
[   75.365827]  [<c02d3c2b>] kobject_get+0x66/0x87
[   75.376185]  [<c035b157>] get_driver+0x11/0x18
[   75.386420]  [<c035b194>] driver_create_file+0xf/0x32
[   75.397291]  [<c035a221>] bus_add_driver+0x15a/0x1ae
[   75.408056]  [<c035b109>] driver_register+0x54/0x84
[   75.418682]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   75.429904]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   75.440480]  [<c06725bc>] init+0x144/0x26c
[   75.450257]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   75.461238]  =======================
[   75.470402] get: c0644890 count after get is 4
[   75.480404] BUG: at lib/kobject.c:494 kobject_put()
[   75.490866]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   75.501613]  [<c01057b2>] show_trace+0x12/0x14
[   75.511530]  [<c0105856>] dump_stack+0x16/0x18
[   75.521420]  [<c02d3ae7>] kobject_put+0x69/0x82
[   75.531449]  [<c035b144>] put_driver+0xb/0xd
[   75.541209]  [<c035b1b0>] driver_create_file+0x2b/0x32
[   75.551845]  [<c035a221>] bus_add_driver+0x15a/0x1ae
[   75.562324]  [<c035b109>] driver_register+0x54/0x84
[   75.572682]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   75.583628]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   75.593942]  [<c06725bc>] init+0x144/0x26c
[   75.603469]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   75.614233]  =======================
[   75.623258] put: c0644890 count before put is 4
[   75.633227] BUG: at lib/kobject.c:448 kobject_get()
[   75.643575]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   75.654272]  [<c01057b2>] show_trace+0x12/0x14
[   75.664153]  [<c0105856>] dump_stack+0x16/0x18
[   75.673991]  [<c02d3c2b>] kobject_get+0x66/0x87
[   75.683977]  [<c035b157>] get_driver+0x11/0x18
[   75.693884]  [<c035b194>] driver_create_file+0xf/0x32
[   75.704423]  [<c035a233>] bus_add_driver+0x16c/0x1ae
[   75.714893]  [<c035b109>] driver_register+0x54/0x84
[   75.725245]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   75.736189]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   75.746496]  [<c06725bc>] init+0x144/0x26c
[   75.756023]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   75.766778]  =======================
[   75.775786] get: c0644890 count after get is 4
[   75.785652] BUG: at lib/kobject.c:494 kobject_put()
[   75.795990]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   75.806687]  [<c01057b2>] show_trace+0x12/0x14
[   75.816577]  [<c0105856>] dump_stack+0x16/0x18
[   75.826416]  [<c02d3ae7>] kobject_put+0x69/0x82
[   75.836390]  [<c035b144>] put_driver+0xb/0xd
[   75.846107]  [<c035b1b0>] driver_create_file+0x2b/0x32
[   75.856744]  [<c035a233>] bus_add_driver+0x16c/0x1ae
[   75.867222]  [<c035b109>] driver_register+0x54/0x84
[   75.877589]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   75.888545]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   75.898868]  [<c06725bc>] init+0x144/0x26c
[   75.908412]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   75.919193]  =======================
[   75.928228] put: c0644890 count before put is 4
[   75.949537] ipmi_si: pci_unregister_driver()
[   75.959258] bus pci: remove driver ipmi_si
[   75.968766] BUG: at lib/kobject.c:448 kobject_get()
[   75.979081]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   75.989802]  [<c01057b2>] show_trace+0x12/0x14
[   75.999832]  [<c0105856>] dump_stack+0x16/0x18
[   76.009799]  [<c02d3c2b>] kobject_get+0x66/0x87
[   76.019863]  [<c035b157>] get_driver+0x11/0x18
[   76.029822]  [<c035b16c>] driver_remove_file+0xe/0x27
[   76.040389]  [<c035a019>] bus_remove_driver+0x3a/0xa4
[   76.050989]  [<c035b0a8>] driver_unregister+0xb/0x18
[   76.061432]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   76.072312]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   76.082589]  [<c06725bc>] init+0x144/0x26c
[   76.092090]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   76.102742]  =======================
[   76.111690] get: c0644890 count after get is 4
[   76.121537] BUG: at lib/kobject.c:494 kobject_put()
[   76.131808]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   76.142477]  [<c01057b2>] show_trace+0x12/0x14
[   76.152463]  [<c0105856>] dump_stack+0x16/0x18
[   76.162397]  [<c02d3ae7>] kobject_put+0x69/0x82
[   76.172408]  [<c035b144>] put_driver+0xb/0xd
[   76.182134]  [<c035b181>] driver_remove_file+0x23/0x27
[   76.192759]  [<c035a019>] bus_remove_driver+0x3a/0xa4
[   76.203309]  [<c035b0a8>] driver_unregister+0xb/0x18
[   76.213726]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   76.224578]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   76.234840]  [<c06725bc>] init+0x144/0x26c
[   76.244316]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   76.254951]  =======================
[   76.263853] put: c0644890 count before put is 4
[   76.273754] BUG: at lib/kobject.c:448 kobject_get()
[   76.284057]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   76.294677]  [<c01057b2>] show_trace+0x12/0x14
[   76.304462]  [<c0105856>] dump_stack+0x16/0x18
[   76.314213]  [<c02d3c2b>] kobject_get+0x66/0x87
[   76.324122]  [<c035b157>] get_driver+0x11/0x18
[   76.333942]  [<c035b16c>] driver_remove_file+0xe/0x27
[   76.344405]  [<c035a025>] bus_remove_driver+0x46/0xa4
[   76.354882]  [<c035b0a8>] driver_unregister+0xb/0x18
[   76.365243]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   76.376110]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   76.386356]  [<c06725bc>] init+0x144/0x26c
[   76.395830]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   76.406544]  =======================
[   76.415524] get: c0644890 count after get is 4
[   76.425362] BUG: at lib/kobject.c:494 kobject_put()
[   76.435659]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   76.446287]  [<c01057b2>] show_trace+0x12/0x14
[   76.456107]  [<c0105856>] dump_stack+0x16/0x18
[   76.465876]  [<c02d3ae7>] kobject_put+0x69/0x82
[   76.475800]  [<c035b144>] put_driver+0xb/0xd
[   76.485457]  [<c035b181>] driver_remove_file+0x23/0x27
[   76.496030]  [<c035a025>] bus_remove_driver+0x46/0xa4
[   76.506536]  [<c035b0a8>] driver_unregister+0xb/0x18
[   76.516911]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   76.527772]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   76.538017]  [<c06725bc>] init+0x144/0x26c
[   76.547492]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   76.558194]  =======================
[   76.567160] put: c0644890 count before put is 4
[   76.577094] BUG: at lib/kobject.c:494 kobject_put()
[   76.587399]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   76.598051]  [<c01057b2>] show_trace+0x12/0x14
[   76.607880]  [<c0105856>] dump_stack+0x16/0x18
[   76.617676]  [<c02d3ae7>] kobject_put+0x69/0x82
[   76.627617]  [<c01b27a3>] sysfs_hash_and_remove+0x116/0x11f
[   76.638712]  [<c01b473d>] sysfs_remove_link+0xb/0xd
[   76.649096]  [<c0144151>] module_remove_driver+0x18/0x57
[   76.659972]  [<c035a06b>] bus_remove_driver+0x8c/0xa4
[   76.670600]  [<c035b0a8>] driver_unregister+0xb/0x18
[   76.681138]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   76.692128]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   76.702470]  [<c06725bc>] init+0x144/0x26c
[   76.712092]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   76.722899]  =======================
[   76.731916] put: c18f65c0 count before put is 5
[   76.741944] kobject ipmi_si: unregistering
[   76.751496] kobject_uevent_env
[   76.759873] fill_kobj_path: path = '/bus/pci/drivers/ipmi_si'
[   76.771235] BUG: at lib/kobject.c:494 kobject_put()
[   76.781811]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   76.792782]  [<c01057b2>] show_trace+0x12/0x14
[   76.803009]  [<c0105856>] dump_stack+0x16/0x18
[   76.813168]  [<c02d3ae7>] kobject_put+0x69/0x82
[   76.823379]  [<c02d3bc2>] unlink+0x3b/0x3e
[   76.833121]  [<c02d3caf>] kobject_del+0x16/0x19
[   76.843272]  [<c02d3ce6>] kobject_unregister+0x34/0xa1
[   76.854002]  [<c035a073>] bus_remove_driver+0x94/0xa4
[   76.864671]  [<c035b0a8>] driver_unregister+0xb/0x18
[   76.875185]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   76.886124]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   76.896439]  [<c06725bc>] init+0x144/0x26c
[   76.905974]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   76.916676]  =======================
[   76.925694] put: c0644890 count before put is 3
[   76.935799] BUG: at lib/kobject.c:430 kobject_unregister()
[   76.946990]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   76.957893]  [<c01057b2>] show_trace+0x12/0x14
[   76.968033]  [<c0105856>] dump_stack+0x16/0x18
[   76.978063]  [<c02d3d3a>] kobject_unregister+0x88/0xa1
[   76.988811]  [<c035a073>] bus_remove_driver+0x94/0xa4
[   76.999498]  [<c035b0a8>] driver_unregister+0xb/0x18
[   77.010064]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   77.021089]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   77.031524]  [<c06725bc>] init+0x144/0x26c
[   77.041179]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   77.051980]  =======================
[   77.061057] unregister: c0644890 count before put is 2
[   77.071811] BUG: at lib/kobject.c:494 kobject_put()
[   77.082360]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   77.093203]  [<c01057b2>] show_trace+0x12/0x14
[   77.103189]  [<c0105856>] dump_stack+0x16/0x18
[   77.113053]  [<c02d3ae7>] kobject_put+0x69/0x82
[   77.123022]  [<c02d3d09>] kobject_unregister+0x57/0xa1
[   77.133655]  [<c035a073>] bus_remove_driver+0x94/0xa4
[   77.144247]  [<c035b0a8>] driver_unregister+0xb/0x18
[   77.154753]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   77.165770]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   77.176205]  [<c06725bc>] init+0x144/0x26c
[   77.185880]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   77.196791]  =======================
[   77.205972] put: c0644890 count before put is 2

Re: kref refcounting breakage in mainline

[Greg KH]

On Sat, Mar 10, 2007 at 04:44:06PM +0100, Mike Galbraith wrote:
> On Wed, 2007-03-07 at 06:39 +0100, Mike Galbraith wrote:
> > On Tue, 2007-03-06 at 13:04 -0800, Greg KH wrote:
> > > On Tue, Mar 06, 2007 at 06:43:22AM +0100, Mike Galbraith wrote:
> > > > On Mon, 2007-03-05 at 16:25 -0800, Greg KH wrote:
> > > > 
> > > > > Mike, I've reverted this patch, and I don't see any references leaking.
> > > > > And, as your patch released the reference on the driver, and the
> > > > > module_add_driver() call would not grab a reference to the driver, only
> > > > > the module kobject, I don't see what you were trying to fix with this
> > > > > patch.
> > > > > 
> > > > > Do you have a test case that this fixes?
> > > > 
> > > > What it fixed for me was the hard hang reported below.
> > > > 
> > > > http://lkml.org/lkml/2007/2/16/96
> > > 
> > > What specific module are you trying to unload that causes the hang?  I
> > > think it might just be a problem with that module, and not with all
> > > others.
> > 
> > It's ipmi_si that's hanging, waits for completion that never comes.
> > 
> > > So, I'm going to revert your patch and work to try to find the real
> > > cause of this problem.
> > 
> > Yeah, my stab at it seems busted.  I'll take another poke at it to see
> > if I can find out why (post 725522b5453dd680412f2b6463a988e4fd148757)
> > I'm left with a reference.
> 
> Ok, stab #2.
> 
> My reference count woes stem from module_remove_driver() not removing
> the link created in module_add_driver().  With the below, my box boots
> fine.  Since I obviously know spit about driver layer glue, I'll just
> call this one a diagnostic, and head for the hills :)

Does ipmi_si not have a "owner"?  Ah, that makes sense, not all modules
do...

> --- linux-2.6.20-rc3/kernel/module.c.org	2007-03-10 15:16:47.000000000 +0100
> +++ linux-2.6.20-rc3/kernel/module.c	2007-03-10 15:43:09.000000000 +0100
> @@ -2411,14 +2411,28 @@ void module_remove_driver(struct device_
>  		return;
>  
>  	sysfs_remove_link(&drv->kobj, "module");
> -	if (drv->owner && drv->owner->mkobj.drivers_dir) {
> -		driver_name = make_driver_name(drv);
> -		if (driver_name) {
> -			sysfs_remove_link(drv->owner->mkobj.drivers_dir,
> +	driver_name = make_driver_name(drv);
> +	if (!driver_name)
> +		return;
> +	if (drv->owner && drv->owner->mkobj.drivers_dir)
> +		sysfs_remove_link(drv->owner->mkobj.drivers_dir,
>  					  driver_name);
> -			kfree(driver_name);
> -		}
> +	else if (drv->mod_name) {
> +		struct module_kobject *mk;
> +		struct kobject *mkobj;
> +
> +		/* Lookup built-in module entry in /sys/modules */
> +		mkobj = kset_find_obj(&module_subsys.kset, drv->mod_name);
> +		if (!mkobj)
> +			goto out_free;
> +		mk = container_of(mkobj, struct module_kobject, kobj);
> +		module_create_drivers_dir(mk);
> +		sysfs_remove_link(mk->drivers_dir, driver_name);
> +		/* Release reference taken via lookup */
> +		kobject_put(mkobj);
>  	}
> +out_free:
> +	kfree(driver_name);
>  }
>  EXPORT_SYMBOL(module_remove_driver);
>  #endif

That's pretty good for not knowing much about the subject matter here.
But can you try this version instead?  It should work a bit better than
yours.

thanks for your patience,

greg k-h

Subject: modules: fix reference counting logic for drivers without module pointers.

We weren't dropping the sysfs link for the module driver name if we
didn't happen to have the "owner" pointer in the driver.

Based on a patch from Mike Galbraith <efault@gmx.de>

Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

---
 kernel/module.c |   24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

--- a/kernel/module.c
+++ b/kernel/module.c
@@ -2405,20 +2405,30 @@ EXPORT_SYMBOL(module_add_driver);
 
 void module_remove_driver(struct device_driver *drv)
 {
+	struct module_kobject *mk = NULL;
+	struct kobject *mkobj = NULL;
 	char *driver_name;
 
 	if (!drv)
 		return;
 
 	sysfs_remove_link(&drv->kobj, "module");
-	if (drv->owner && drv->owner->mkobj.drivers_dir) {
-		driver_name = make_driver_name(drv);
-		if (driver_name) {
-			sysfs_remove_link(drv->owner->mkobj.drivers_dir,
-					  driver_name);
-			kfree(driver_name);
-		}
+	driver_name = make_driver_name(drv);
+	if (!driver_name)
+		return;
+
+	if (drv->owner && drv->owner->mkobj.drivers_dir)
+		mk = &drv->owner->mkobj;
+	else {
+		/* Lookup built-in module entry in /sys/modules */
+		mkobj = kset_find_obj(&module_subsys.kset, drv->mod_name);
+		if (!mkobj)
+			return;
+		mk = container_of(mkobj, struct module_kobject, kobj);
 	}
+	sysfs_remove_link(mk->drivers_dir, driver_name);
+	kobject_put(mkobj);
+	kfree(driver_name);
 }
 EXPORT_SYMBOL(module_remove_driver);
 #endif

Re: kref refcounting breakage in mainline

[Mike Galbraith]

On Wed, 2007-03-14 at 22:27 -0700, Greg KH wrote:

> But can you try this version instead?

It exploded in strcmp().  Unfortunately, the full oops didn't make it to
either console or serial console.

[   30.783048] ipmi message handler version 39.1
[   30.787632] ipmi device interface
[   30.791166] IPMI System Interface driver.
[   30.816961] BUG: unable to handle kernel NULL pointer dereference at virtual address 00000000
[   30.832700]  printing eip:
[   30.842383] c02d4098
[   30.851458] *pde = 00000000
[   30.861089] Oops: 0000 [#1]
[   30.870585] PREEMPT SMP 
[   30.879724] Modules linked in:
[   30.889288] CPU:    1
[   30.889290] EIP:    0060:[<c02d4098>]    Not tainted VLI
[   30.889292] EFLAGS: 00010282   (2.6.21-rc3-smp #81)


I did this...

--- kernel/module.c.org	2007-03-15 07:20:15.000000000 +0100
+++ kernel/module.c	2007-03-15 08:13:36.000000000 +0100
@@ -2419,15 +2419,19 @@ void module_remove_driver(struct device_
 
 	if (drv->owner && drv->owner->mkobj.drivers_dir)
 		mk = &drv->owner->mkobj;
-	else {
+	else if (drv->mod_name) {
 		/* Lookup built-in module entry in /sys/modules */
 		mkobj = kset_find_obj(&module_subsys.kset, drv->mod_name);
 		if (!mkobj)
-			return;
+			goto out_free;
 		mk = container_of(mkobj, struct module_kobject, kobj);
+	} else {
+		WARN_ON(1);
+		goto out_free;
 	}
 	sysfs_remove_link(mk->drivers_dir, driver_name);
 	kobject_put(mkobj);
+out_free:
 	kfree(driver_name);
 }
 EXPORT_SYMBOL(module_remove_driver);

...and it booted.

[   24.670410] ipmi message handler version 39.1
[   24.675000] ipmi device interface
[   24.678542] IPMI System Interface driver.
[   24.703956] BUG: at kernel/module.c:2429 module_remove_driver()
[   24.716837]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   24.728909]  [<c01057b2>] show_trace+0x12/0x14
[   24.740239]  [<c0105856>] dump_stack+0x16/0x18
[   24.751469]  [<c01441ce>] module_remove_driver+0xa5/0xa7
[   24.763584]  [<c035a014>] bus_remove_driver+0x6d/0x82
[   24.775390]  [<c035a978>] driver_unregister+0xb/0x18
[   24.787019]  [<c034c8c2>] init_ipmi_si+0x7a9/0x7c1
[   24.798450]  [<c06705bc>] init+0x144/0x26c
[   24.809129]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   24.820916]  =======================
[   24.830926] ipmi_si: Unable to find any System Interface(s)
[   24.842952] IPMI Watchdog: driver initialized
  24.853749] Copyright (C) 2004 MontaVista Software - IPMI Powerdown via sys_reboot.

	-Mike

Re: kref refcounting breakage in mainline

[Greg KH]

On Thu, Mar 15, 2007 at 08:53:07AM +0100, Mike Galbraith wrote:
> On Wed, 2007-03-14 at 22:27 -0700, Greg KH wrote:
> 
> > But can you try this version instead?
> 
> It exploded in strcmp().  Unfortunately, the full oops didn't make it to
> either console or serial console.
> 
> [   30.783048] ipmi message handler version 39.1
> [   30.787632] ipmi device interface
> [   30.791166] IPMI System Interface driver.
> [   30.816961] BUG: unable to handle kernel NULL pointer dereference at virtual address 00000000
> [   30.832700]  printing eip:
> [   30.842383] c02d4098
> [   30.851458] *pde = 00000000
> [   30.861089] Oops: 0000 [#1]
> [   30.870585] PREEMPT SMP 
> [   30.879724] Modules linked in:
> [   30.889288] CPU:    1
> [   30.889290] EIP:    0060:[<c02d4098>]    Not tainted VLI
> [   30.889292] EFLAGS: 00010282   (2.6.21-rc3-smp #81)

Gah, that just happened to me too, sorry for not booting with that
change before sending it to you.

> I did this...
> 
> --- kernel/module.c.org	2007-03-15 07:20:15.000000000 +0100
> +++ kernel/module.c	2007-03-15 08:13:36.000000000 +0100
> @@ -2419,15 +2419,19 @@ void module_remove_driver(struct device_
>  
>  	if (drv->owner && drv->owner->mkobj.drivers_dir)
>  		mk = &drv->owner->mkobj;
> -	else {
> +	else if (drv->mod_name) {
>  		/* Lookup built-in module entry in /sys/modules */
>  		mkobj = kset_find_obj(&module_subsys.kset, drv->mod_name);
>  		if (!mkobj)
> -			return;
> +			goto out_free;
>  		mk = container_of(mkobj, struct module_kobject, kobj);
> +	} else {
> +		WARN_ON(1);
> +		goto out_free;
>  	}
>  	sysfs_remove_link(mk->drivers_dir, driver_name);
>  	kobject_put(mkobj);
> +out_free:
>  	kfree(driver_name);
>  }
>  EXPORT_SYMBOL(module_remove_driver);
> 
> ...and it booted.

That's good.  But why don't we have a module name for this driver?

And if we don't have a module name, why would there be a symlink to
remove?  That's what is keeping your module from unloading, right?

> [   24.670410] ipmi message handler version 39.1
> [   24.675000] ipmi device interface
> [   24.678542] IPMI System Interface driver.
> [   24.703956] BUG: at kernel/module.c:2429 module_remove_driver()
> [   24.716837]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
> [   24.728909]  [<c01057b2>] show_trace+0x12/0x14
> [   24.740239]  [<c0105856>] dump_stack+0x16/0x18
> [   24.751469]  [<c01441ce>] module_remove_driver+0xa5/0xa7
> [   24.763584]  [<c035a014>] bus_remove_driver+0x6d/0x82
> [   24.775390]  [<c035a978>] driver_unregister+0xb/0x18
> [   24.787019]  [<c034c8c2>] init_ipmi_si+0x7a9/0x7c1
> [   24.798450]  [<c06705bc>] init+0x144/0x26c
> [   24.809129]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
> [   24.820916]  =======================
> [   24.830926] ipmi_si: Unable to find any System Interface(s)
> [   24.842952] IPMI Watchdog: driver initialized
>   24.853749] Copyright (C) 2004 MontaVista Software - IPMI Powerdown via sys_reboot.

With the above change, it all works correctly?

If the ipmi driver is loaded, what does the /sys/module/MODULE_NAME/
tree look like (replacing MODULE_NAME with whatever the module name
really is, sorry, I don't know)?

thanks,

greg k-h

Re: kref refcounting breakage in mainline

[Mike Galbraith]

On Thu, 2007-03-15 at 01:06 -0700, Greg KH wrote:

> That's good.  But why don't we have a module name for this driver?
> 
> And if we don't have a module name, why would there be a symlink to
> remove?  That's what is keeping your module from unloading, right?

Ya got me, but according to my debug logs, what's causing my lockup is
the reference we add while making the symlink when we hit...
 if (driver_name) in module_add_driver().  Maybe we go through there
twice, once with a name, and once without?  Dunno.
 
> > [   24.670410] ipmi message handler version 39.1
> > [   24.675000] ipmi device interface
> > [   24.678542] IPMI System Interface driver.
> > [   24.703956] BUG: at kernel/module.c:2429 module_remove_driver()
> > [   24.716837]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
> > [   24.728909]  [<c01057b2>] show_trace+0x12/0x14
> > [   24.740239]  [<c0105856>] dump_stack+0x16/0x18
> > [   24.751469]  [<c01441ce>] module_remove_driver+0xa5/0xa7
> > [   24.763584]  [<c035a014>] bus_remove_driver+0x6d/0x82
> > [   24.775390]  [<c035a978>] driver_unregister+0xb/0x18
> > [   24.787019]  [<c034c8c2>] init_ipmi_si+0x7a9/0x7c1
> > [   24.798450]  [<c06705bc>] init+0x144/0x26c
> > [   24.809129]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
> > [   24.820916]  =======================
> > [   24.830926] ipmi_si: Unable to find any System Interface(s)
> > [   24.842952] IPMI Watchdog: driver initialized
> >   24.853749] Copyright (C) 2004 MontaVista Software - IPMI Powerdown via sys_reboot.
> 
> With the above change, it all works correctly?

I don't know about _correctly_, but my diag patch _boots_, as does your
patch plus my addon diag bits.

> If the ipmi driver is loaded, what does the /sys/module/MODULE_NAME/
> tree look like (replacing MODULE_NAME with whatever the module name
> really is, sorry, I don't know)?

Well, I will never see that, because ipmi_si finds no interfaces, so
always backs out.  After backout, with my patch and yours + my addons, I
have there leftovers.

root@Homer: ls -R /sys/module/ipmi_si
/sys/module/ipmi_si:
drivers  parameters

/sys/module/ipmi_si/drivers:

/sys/module/ipmi_si/parameters:
bt_debug  hotmod  kcs_debug  smic_debug

Re: kref refcounting breakage in mainline

[Mike Galbraith]

On Thu, 2007-03-15 at 09:32 +0100, Mike Galbraith wrote:
> On Thu, 2007-03-15 at 01:06 -0700, Greg KH wrote:
> 
> > That's good.  But why don't we have a module name for this driver?
> > 
> > And if we don't have a module name, why would there be a symlink to
> > remove?  That's what is keeping your module from unloading, right?
> 
> Ya got me, but according to my debug logs, what's causing my lockup is
> the reference we add while making the symlink when we hit...
>  if (driver_name) in module_add_driver().  Maybe we go through there
> twice, once with a name, and once without?  Dunno.

I found the log (i think), and even with the patched kernel, gdb still
says...
(gdb) list *module_add_driver+0x44
0xc01442e5 is in module_add_driver (kernel/module.c:2401).
2396            driver_name = make_driver_name(drv);
2397            if (driver_name) {
2398                    module_create_drivers_dir(mk);
2399                    no_warn = sysfs_create_link(mk->drivers_dir, &drv->kobj,
2400                                                driver_name);
2401                    kfree(driver_name);
2402            }
2403    }
2404    EXPORT_SYMBOL(module_add_driver);
2405
(gdb)

See: kobject filter function caused the event to drop!

(erm, that spot caused some hmm action here. if that drop is bad, i have
another one as well.)

[   30.397160] kobject ipmi_devintf: registering. parent: <NULL>, set: module
[   30.404033] kobject_uevent_env
[   30.407098] fill_kobj_path: path = '/module/ipmi_devintf'
[   30.412524] BUG: at lib/kobject.c:448 kobject_get()
[   30.417402]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   30.422564]  [<c01057b2>] show_trace+0x12/0x14
[   30.427031]  [<c0105856>] dump_stack+0x16/0x18
[   30.431501]  [<c02d3c2b>] kobject_get+0x66/0x87
[   30.436056]  [<c02d3f1e>] kobject_shadow_add+0x10/0x1e8
[   30.441312]  [<c02d4100>] kobject_add+0xa/0xc
[   30.445695]  [<c06804c0>] kernel_param_sysfs_setup+0x4d/0x7f
[   30.451376]  [<c068067a>] param_sysfs_init+0x188/0x1c3
[   30.456538]  [<c06725bc>] init+0x144/0x26c
[   30.460661]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   30.465907]  =======================
[   30.469486] get: c18f65c0 count after get is 2
[   30.473927] kobject ipmi_si: registering. parent: <NULL>, set: module
[   30.480372] kobject_uevent_env
[   30.483430] fill_kobj_path: path = '/module/ipmi_si'

......

[   73.266556] bus platform: add driver ipmi
[   73.278013] kobject ipmi: registering. parent: <NULL>, set: drivers
[   73.291847] kobject_uevent_env
[   73.302358] fill_kobj_path: path = '/bus/platform/drivers/ipmi'
[   73.315943] ipmi message handler version 39.1
[   73.327839] ipmi device interface
[   73.338524] device class 'ipmi': registering
[   73.350158] subsystem ipmi: registering
[   73.361309] kobject ipmi: registering. parent: <NULL>, set: class
[   73.374841] bus platform: add driver ipmi_si
[   73.386442] BUG: at lib/kobject.c:448 kobject_get()
[   73.398617]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   73.411079]  [<c01057b2>] show_trace+0x12/0x14
[   73.422780]  [<c0105856>] dump_stack+0x16/0x18
[   73.434324]  [<c02d3c2b>] kobject_get+0x66/0x87
[   73.445860]  [<c02d3f1e>] kobject_shadow_add+0x10/0x1e8
[   73.458088]  [<c02d4100>] kobject_add+0xa/0xc
[   73.469286]  [<c02d4248>] kobject_register+0x22/0xb3
[   73.480986]  [<c035a13e>] bus_add_driver+0x77/0x1ae
[   73.492592]  [<c035b109>] driver_register+0x54/0x84
[   73.504101]  [<c034c486>] init_ipmi_si+0x4d/0x829
[   73.515335]  [<c06725bc>] init+0x144/0x26c
[   73.525822]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   73.537452]  =======================
[   73.547290] get: c064475c count after get is 2
[   73.557969] kobject ipmi_si: registering. parent: <NULL>, set: drivers
[   73.570825] kobject_uevent_env
[   73.580011] fill_kobj_path: path = '/bus/platform/drivers/ipmi_si'
[   73.592622] BUG: at lib/kobject.c:242 kobject_register()
[   73.604312]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   73.615822]  [<c01057b2>] show_trace+0x12/0x14
[   73.626551]  [<c0105856>] dump_stack+0x16/0x18
[   73.637211]  [<c02d429f>] kobject_register+0x79/0xb3
[   73.648325]  [<c035a13e>] bus_add_driver+0x77/0x1ae
[   73.659297]  [<c035b109>] driver_register+0x54/0x84
[   73.670164]  [<c034c486>] init_ipmi_si+0x4d/0x829
[   73.680756]  [<c06725bc>] init+0x144/0x26c
[   73.690699]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   73.701854]  =======================
[   73.711295] register: c064475c count now is 2 error 0
[   73.722218] BUG: at lib/kobject.c:448 kobject_get()
[   73.732911]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   73.743927]  [<c01057b2>] show_trace+0x12/0x14
[   73.754112]  [<c0105856>] dump_stack+0x16/0x18
[   73.764253]  [<c02d3c2b>] kobject_get+0x66/0x87
[   73.774532]  [<c035b157>] get_driver+0x11/0x18
[   73.784725]  [<c035b194>] driver_create_file+0xf/0x32
[   73.795559]  [<c035a221>] bus_add_driver+0x15a/0x1ae
[   73.806343]  [<c035b109>] driver_register+0x54/0x84
[   73.817055]  [<c034c486>] init_ipmi_si+0x4d/0x829
[   73.827595]  [<c06725bc>] init+0x144/0x26c
[   73.837468]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   73.848544]  =======================
[   73.857932] get: c064475c count after get is 3
[   73.868223] BUG: at lib/kobject.c:494 kobject_put()
[   73.878943]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   73.890019]  [<c01057b2>] show_trace+0x12/0x14
[   73.900239]  [<c0105856>] dump_stack+0x16/0x18
[   73.910294]  [<c02d3ae7>] kobject_put+0x69/0x82
[   73.920304]  [<c035b144>] put_driver+0xb/0xd
[   73.929909]  [<c035b1b0>] driver_create_file+0x2b/0x32
[   73.940362]  [<c035a221>] bus_add_driver+0x15a/0x1ae
[   73.950686]  [<c035b109>] driver_register+0x54/0x84
[   73.960966]  [<c034c486>] init_ipmi_si+0x4d/0x829
[   73.971055]  [<c06725bc>] init+0x144/0x26c
[   73.980478]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   73.991119]  =======================
[   74.000050] put: c064475c count before put is 3
[   74.010026] BUG: at lib/kobject.c:448 kobject_get()
[   74.020367]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.031011]  [<c01057b2>] show_trace+0x12/0x14
[   74.040867]  [<c0105856>] dump_stack+0x16/0x18
[   74.050602]  [<c02d3c2b>] kobject_get+0x66/0x87
[   74.060353]  [<c035b157>] get_driver+0x11/0x18
[   74.070034]  [<c035b194>] driver_create_file+0xf/0x32
[   74.080418]  [<c035a233>] bus_add_driver+0x16c/0x1ae
[   74.090742]  [<c035b109>] driver_register+0x54/0x84
[   74.100979]  [<c034c486>] init_ipmi_si+0x4d/0x829
[   74.111041]  [<c06725bc>] init+0x144/0x26c
[   74.120431]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   74.131047]  =======================
[   74.139949] get: c064475c count after get is 3
[   74.149807] BUG: at lib/kobject.c:494 kobject_put()
[   74.160111]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.170739]  [<c01057b2>] show_trace+0x12/0x14
[   74.180569]  [<c0105856>] dump_stack+0x16/0x18
[   74.190286]  [<c02d3ae7>] kobject_put+0x69/0x82
[   74.200019]  [<c035b144>] put_driver+0xb/0xd
[   74.209503]  [<c035b1b0>] driver_create_file+0x2b/0x32
[   74.219982]  [<c035a233>] bus_add_driver+0x16c/0x1ae
[   74.230312]  [<c035b109>] driver_register+0x54/0x84
[   74.240550]  [<c034c486>] init_ipmi_si+0x4d/0x829
[   74.250614]  [<c06725bc>] init+0x144/0x26c
[   74.260001]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   74.270618]  =======================
[   74.279530] put: c064475c count before put is 3
[   74.289481] IPMI System Interface driver.
[   74.298864] bus pci: add driver ipmi_si
[   74.308007] BUG: at lib/kobject.c:448 kobject_get()
[   74.318260]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.328861]  [<c01057b2>] show_trace+0x12/0x14
[   74.338637]  [<c0105856>] dump_stack+0x16/0x18
[   74.348372]  [<c02d3c2b>] kobject_get+0x66/0x87
[   74.358245]  [<c02d3f1e>] kobject_shadow_add+0x10/0x1e8
[   74.368837]  [<c02d4100>] kobject_add+0xa/0xc
[   74.378528]  [<c02d4248>] kobject_register+0x22/0xb3
[   74.388817]  [<c035a13e>] bus_add_driver+0x77/0x1ae
[   74.399027]  [<c035b109>] driver_register+0x54/0x84
[   74.409229]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   74.420004]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   74.430196]  [<c06725bc>] init+0x144/0x26c
[   74.439611]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   74.450184]  =======================
[   74.459045] get: c0644890 count after get is 2
[   74.468796] kobject ipmi_si: registering. parent: <NULL>, set: drivers
[   74.480769] kobject_uevent_env
[   74.489210] fill_kobj_path: path = '/bus/pci/drivers/ipmi_si'
[   74.500675] BUG: at lib/kobject.c:242 kobject_register()
[   74.511779]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.522812]  [<c01057b2>] show_trace+0x12/0x14
[   74.533082]  [<c0105856>] dump_stack+0x16/0x18
[   74.543269]  [<c02d429f>] kobject_register+0x79/0xb3
[   74.554015]  [<c035a13e>] bus_add_driver+0x77/0x1ae
[   74.564685]  [<c035b109>] driver_register+0x54/0x84
[   74.575363]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   74.586595]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   74.597161]  [<c06725bc>] init+0x144/0x26c
[   74.606974]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   74.617989]  =======================
[   74.627292] register: c0644890 count now is 2 error 0
[   74.638150] BUG: at lib/kobject.c:448 kobject_get()
[   74.648812]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.659872]  [<c01057b2>] show_trace+0x12/0x14
[   74.670237]  [<c0105856>] dump_stack+0x16/0x18
[   74.680554]  [<c02d3c2b>] kobject_get+0x66/0x87
[   74.690962]  [<c02d434c>] kset_find_obj+0x73/0x77
[   74.701563]  [<c01442c5>] module_add_driver+0x64/0x7d
[   74.712578]  [<c035a170>] bus_add_driver+0xa9/0x1ae
[   74.723405]  [<c035b109>] driver_register+0x54/0x84
[   74.734247]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   74.745636]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   74.756393]  [<c06725bc>] init+0x144/0x26c
[   74.766429]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   74.777654]  =======================
[   74.787103] get: c18f65c0 count after get is 3
[   74.797461] BUG: at lib/kobject.c:448 kobject_get()
[   74.808209]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.819241]  [<c01057b2>] show_trace+0x12/0x14
[   74.829427]  [<c0105856>] dump_stack+0x16/0x18
[   74.839446]  [<c02d3c2b>] kobject_get+0x66/0x87
[   74.849465]  [<c01b4812>] sysfs_create_link+0xd3/0x151
[   74.860178]  [<c0144285>] module_add_driver+0x24/0x7d
[   74.870866]  [<c035a170>] bus_add_driver+0xa9/0x1ae
[   74.881370]  [<c035b109>] driver_register+0x54/0x84
[   74.891850]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   74.902918]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   74.913354]  [<c06725bc>] init+0x144/0x26c
[   74.923019]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   74.933904]  =======================
[   74.943042] get: c18f65c0 count after get is 4
[   74.953046] BUG: at lib/kobject.c:448 kobject_get()
[   74.963497]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   74.974262]  [<c01057b2>] show_trace+0x12/0x14
[   74.984220]  [<c0105856>] dump_stack+0x16/0x18
[   74.994120]  [<c02d3c2b>] kobject_get+0x66/0x87
[   75.004167]  [<c02d3f4a>] kobject_shadow_add+0x3c/0x1e8
[   75.014941]  [<c02d4100>] kobject_add+0xa/0xc
[   75.024787]  [<c02d4248>] kobject_register+0x22/0xb3
[   75.035216]  [<c02d4431>] kobject_add_dir+0x4f/0x6e
[   75.045511]  [<c01442d9>] module_add_driver+0x78/0x7d
[   75.055991]  [<c035a170>] bus_add_driver+0xa9/0x1ae
[   75.066262]  [<c035b109>] driver_register+0x54/0x84
[   75.076489]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   75.087297]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   75.097483]  [<c06725bc>] init+0x144/0x26c
[   75.106862]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   75.117454]  =======================
[   75.126306] get: c18f65c0 count after get is 5
[   75.136022] kobject drivers: registering. parent: ipmi_si, set: <NULL>
[   75.148068] kobject_uevent_env
[   75.156609] kobject filter function caused the event to drop!
[   75.168050] BUG: at lib/kobject.c:448 kobject_get()
[   75.178631]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   75.189595]  [<c01057b2>] show_trace+0x12/0x14
[   75.199789]  [<c0105856>] dump_stack+0x16/0x18
[   75.209905]  [<c02d3c2b>] kobject_get+0x66/0x87
[   75.220072]  [<c01b4812>] sysfs_create_link+0xd3/0x151 <== here
[   75.230853]  [<c01442a5>] module_add_driver+0x44/0x7d
[   75.241531]  [<c035a170>] bus_add_driver+0xa9/0x1ae
[   75.252030]  [<c035b109>] driver_register+0x54/0x84
[   75.262455]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   75.273444]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   75.283813]  [<c06725bc>] init+0x144/0x26c
[   75.293382]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   75.304189]  =======================
[   75.313370] get: c0644890 count after get is 3
[   75.323512] BUG: at lib/kobject.c:448 kobject_get()
[   75.334155]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   75.345215]  [<c01057b2>] show_trace+0x12/0x14
[   75.355547]  [<c0105856>] dump_stack+0x16/0x18
[   75.365827]  [<c02d3c2b>] kobject_get+0x66/0x87
[   75.376185]  [<c035b157>] get_driver+0x11/0x18
[   75.386420]  [<c035b194>] driver_create_file+0xf/0x32
[   75.397291]  [<c035a221>] bus_add_driver+0x15a/0x1ae
[   75.408056]  [<c035b109>] driver_register+0x54/0x84
[   75.418682]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   75.429904]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   75.440480]  [<c06725bc>] init+0x144/0x26c
[   75.450257]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   75.461238]  =======================
[   75.470402] get: c0644890 count after get is 4
[   75.480404] BUG: at lib/kobject.c:494 kobject_put()
[   75.490866]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   75.501613]  [<c01057b2>] show_trace+0x12/0x14
[   75.511530]  [<c0105856>] dump_stack+0x16/0x18
[   75.521420]  [<c02d3ae7>] kobject_put+0x69/0x82
[   75.531449]  [<c035b144>] put_driver+0xb/0xd
[   75.541209]  [<c035b1b0>] driver_create_file+0x2b/0x32
[   75.551845]  [<c035a221>] bus_add_driver+0x15a/0x1ae
[   75.562324]  [<c035b109>] driver_register+0x54/0x84
[   75.572682]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   75.583628]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   75.593942]  [<c06725bc>] init+0x144/0x26c
[   75.603469]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   75.614233]  =======================
[   75.623258] put: c0644890 count before put is 4
[   75.633227] BUG: at lib/kobject.c:448 kobject_get()
[   75.643575]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   75.654272]  [<c01057b2>] show_trace+0x12/0x14
[   75.664153]  [<c0105856>] dump_stack+0x16/0x18
[   75.673991]  [<c02d3c2b>] kobject_get+0x66/0x87
[   75.683977]  [<c035b157>] get_driver+0x11/0x18
[   75.693884]  [<c035b194>] driver_create_file+0xf/0x32
[   75.704423]  [<c035a233>] bus_add_driver+0x16c/0x1ae
[   75.714893]  [<c035b109>] driver_register+0x54/0x84
[   75.725245]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   75.736189]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   75.746496]  [<c06725bc>] init+0x144/0x26c
[   75.756023]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   75.766778]  =======================
[   75.775786] get: c0644890 count after get is 4
[   75.785652] BUG: at lib/kobject.c:494 kobject_put()
[   75.795990]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   75.806687]  [<c01057b2>] show_trace+0x12/0x14
[   75.816577]  [<c0105856>] dump_stack+0x16/0x18
[   75.826416]  [<c02d3ae7>] kobject_put+0x69/0x82
[   75.836390]  [<c035b144>] put_driver+0xb/0xd
[   75.846107]  [<c035b1b0>] driver_create_file+0x2b/0x32
[   75.856744]  [<c035a233>] bus_add_driver+0x16c/0x1ae
[   75.867222]  [<c035b109>] driver_register+0x54/0x84
[   75.877589]  [<c02e2999>] __pci_register_driver+0x7e/0xb3
[   75.888545]  [<c034c830>] init_ipmi_si+0x3f7/0x829
[   75.898868]  [<c06725bc>] init+0x144/0x26c
[   75.908412]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   75.919193]  =======================
[   75.928228] put: c0644890 count before put is 4
[   75.949537] ipmi_si: pci_unregister_driver()
[   75.959258] bus pci: remove driver ipmi_si
[   75.968766] BUG: at lib/kobject.c:448 kobject_get()
[   75.979081]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   75.989802]  [<c01057b2>] show_trace+0x12/0x14
[   75.999832]  [<c0105856>] dump_stack+0x16/0x18
[   76.009799]  [<c02d3c2b>] kobject_get+0x66/0x87
[   76.019863]  [<c035b157>] get_driver+0x11/0x18
[   76.029822]  [<c035b16c>] driver_remove_file+0xe/0x27
[   76.040389]  [<c035a019>] bus_remove_driver+0x3a/0xa4
[   76.050989]  [<c035b0a8>] driver_unregister+0xb/0x18
[   76.061432]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   76.072312]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   76.082589]  [<c06725bc>] init+0x144/0x26c
[   76.092090]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   76.102742]  =======================
[   76.111690] get: c0644890 count after get is 4
[   76.121537] BUG: at lib/kobject.c:494 kobject_put()
[   76.131808]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   76.142477]  [<c01057b2>] show_trace+0x12/0x14
[   76.152463]  [<c0105856>] dump_stack+0x16/0x18
[   76.162397]  [<c02d3ae7>] kobject_put+0x69/0x82
[   76.172408]  [<c035b144>] put_driver+0xb/0xd
[   76.182134]  [<c035b181>] driver_remove_file+0x23/0x27
[   76.192759]  [<c035a019>] bus_remove_driver+0x3a/0xa4
[   76.203309]  [<c035b0a8>] driver_unregister+0xb/0x18
[   76.213726]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   76.224578]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   76.234840]  [<c06725bc>] init+0x144/0x26c
[   76.244316]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   76.254951]  =======================
[   76.263853] put: c0644890 count before put is 4
[   76.273754] BUG: at lib/kobject.c:448 kobject_get()
[   76.284057]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   76.294677]  [<c01057b2>] show_trace+0x12/0x14
[   76.304462]  [<c0105856>] dump_stack+0x16/0x18
[   76.314213]  [<c02d3c2b>] kobject_get+0x66/0x87
[   76.324122]  [<c035b157>] get_driver+0x11/0x18
[   76.333942]  [<c035b16c>] driver_remove_file+0xe/0x27
[   76.344405]  [<c035a025>] bus_remove_driver+0x46/0xa4
[   76.354882]  [<c035b0a8>] driver_unregister+0xb/0x18
[   76.365243]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   76.376110]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   76.386356]  [<c06725bc>] init+0x144/0x26c
[   76.395830]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   76.406544]  =======================
[   76.415524] get: c0644890 count after get is 4
[   76.425362] BUG: at lib/kobject.c:494 kobject_put()
[   76.435659]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   76.446287]  [<c01057b2>] show_trace+0x12/0x14
[   76.456107]  [<c0105856>] dump_stack+0x16/0x18
[   76.465876]  [<c02d3ae7>] kobject_put+0x69/0x82
[   76.475800]  [<c035b144>] put_driver+0xb/0xd
[   76.485457]  [<c035b181>] driver_remove_file+0x23/0x27
[   76.496030]  [<c035a025>] bus_remove_driver+0x46/0xa4
[   76.506536]  [<c035b0a8>] driver_unregister+0xb/0x18
[   76.516911]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   76.527772]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   76.538017]  [<c06725bc>] init+0x144/0x26c
[   76.547492]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   76.558194]  =======================
[   76.567160] put: c0644890 count before put is 4
[   76.577094] BUG: at lib/kobject.c:494 kobject_put()
[   76.587399]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   76.598051]  [<c01057b2>] show_trace+0x12/0x14
[   76.607880]  [<c0105856>] dump_stack+0x16/0x18
[   76.617676]  [<c02d3ae7>] kobject_put+0x69/0x82
[   76.627617]  [<c01b27a3>] sysfs_hash_and_remove+0x116/0x11f
[   76.638712]  [<c01b473d>] sysfs_remove_link+0xb/0xd
[   76.649096]  [<c0144151>] module_remove_driver+0x18/0x57
[   76.659972]  [<c035a06b>] bus_remove_driver+0x8c/0xa4
[   76.670600]  [<c035b0a8>] driver_unregister+0xb/0x18
[   76.681138]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   76.692128]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   76.702470]  [<c06725bc>] init+0x144/0x26c
[   76.712092]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   76.722899]  =======================
[   76.731916] put: c18f65c0 count before put is 5
[   76.741944] kobject ipmi_si: unregistering
[   76.751496] kobject_uevent_env
[   76.759873] fill_kobj_path: path = '/bus/pci/drivers/ipmi_si'
[   76.771235] BUG: at lib/kobject.c:494 kobject_put()
[   76.781811]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   76.792782]  [<c01057b2>] show_trace+0x12/0x14
[   76.803009]  [<c0105856>] dump_stack+0x16/0x18
[   76.813168]  [<c02d3ae7>] kobject_put+0x69/0x82
[   76.823379]  [<c02d3bc2>] unlink+0x3b/0x3e
[   76.833121]  [<c02d3caf>] kobject_del+0x16/0x19
[   76.843272]  [<c02d3ce6>] kobject_unregister+0x34/0xa1
[   76.854002]  [<c035a073>] bus_remove_driver+0x94/0xa4
[   76.864671]  [<c035b0a8>] driver_unregister+0xb/0x18
[   76.875185]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   76.886124]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   76.896439]  [<c06725bc>] init+0x144/0x26c
[   76.905974]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   76.916676]  =======================
[   76.925694] put: c0644890 count before put is 3
[   76.935799] BUG: at lib/kobject.c:430 kobject_unregister()
[   76.946990]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   76.957893]  [<c01057b2>] show_trace+0x12/0x14
[   76.968033]  [<c0105856>] dump_stack+0x16/0x18
[   76.978063]  [<c02d3d3a>] kobject_unregister+0x88/0xa1
[   76.988811]  [<c035a073>] bus_remove_driver+0x94/0xa4
[   76.999498]  [<c035b0a8>] driver_unregister+0xb/0x18
[   77.010064]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   77.021089]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   77.031524]  [<c06725bc>] init+0x144/0x26c
[   77.041179]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   77.051980]  =======================
[   77.061057] unregister: c0644890 count before put is 2
[   77.071811] BUG: at lib/kobject.c:494 kobject_put()
[   77.082360]  [<c0105086>] show_trace_log_lvl+0x1a/0x30
[   77.093203]  [<c01057b2>] show_trace+0x12/0x14
[   77.103189]  [<c0105856>] dump_stack+0x16/0x18
[   77.113053]  [<c02d3ae7>] kobject_put+0x69/0x82
[   77.123022]  [<c02d3d09>] kobject_unregister+0x57/0xa1
[   77.133655]  [<c035a073>] bus_remove_driver+0x94/0xa4
[   77.144247]  [<c035b0a8>] driver_unregister+0xb/0x18
[   77.154753]  [<c02e282c>] pci_unregister_driver+0x13/0x73
[   77.165770]  [<c034cc34>] init_ipmi_si+0x7fb/0x829
[   77.176205]  [<c06725bc>] init+0x144/0x26c
[   77.185880]  [<c0104cfb>] kernel_thread_helper+0x7/0x1c
[   77.196791]  =======================
[   77.205972] put: c0644890 count before put is 2

Re: kref refcounting breakage in mainline

[Greg KH]

On Thu, Mar 15, 2007 at 11:19:20AM +0100, Mike Galbraith wrote:
> On Thu, 2007-03-15 at 01:06 -0700, Greg KH wrote:
> 
> > That's good.  But why don't we have a module name for this driver?
> > 
> > And if we don't have a module name, why would there be a symlink to
> > remove?  That's what is keeping your module from unloading, right?
> 
> You keep saying "module", and that's making me a bit nervous ;-)
> 
> Just to be sure we're not talking past each other, when you say module,
> don't mean the modprobe kind... i hope.  This "module" as in driver is
> compiled in.  (said that before, but you may have missed it)

Ahh, that changes everything here, thanks for letting me know, I had
missed this.

The problem is that the module_init() is failing, yet this isn't really
a module, it's built into the kernel.  So some of the module teardown
logic is dieing when it thinks that we really have a full module
structure here (owner and such).

I'll look at this further tomorrow, as I'm travelling pretty much all
day today, sorry.

thanks,

greg k-h

Re: kref refcounting breakage in mainline

[Randy Dunlap]

On Thu, 15 Mar 2007 07:54:14 -0700 Greg KH wrote:

> On Thu, Mar 15, 2007 at 11:19:20AM +0100, Mike Galbraith wrote:
> > On Thu, 2007-03-15 at 01:06 -0700, Greg KH wrote:
> > 
> > > That's good.  But why don't we have a module name for this driver?
> > > 
> > > And if we don't have a module name, why would there be a symlink to
> > > remove?  That's what is keeping your module from unloading, right?
> > 
> > You keep saying "module", and that's making me a bit nervous ;-)
> > 
> > Just to be sure we're not talking past each other, when you say module,
> > don't mean the modprobe kind... i hope.  This "module" as in driver is
> > compiled in.  (said that before, but you may have missed it)
> 
> Ahh, that changes everything here, thanks for letting me know, I had
> missed this.
> 
> The problem is that the module_init() is failing, yet this isn't really
> a module, it's built into the kernel.  So some of the module teardown
> logic is dieing when it thinks that we really have a full module
> structure here (owner and such).

Urgh, it's not a "loadable" module, but it's still a logical module.

> I'll look at this further tomorrow, as I'm travelling pretty much all
> day today, sorry.


---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***