Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

Since this appears to be a bug in the SCSI layer, let's add some SCSI
people to the CC: list.

To summarize the problem: The SCSI core tries to unregister a host 
while its sysfs directory is still non-empty because the target hasn't 
been unregistered yet.

On Mon, 25 May 2009, Kay Sievers wrote:

> On Mon, 2009-05-25 at 13:45 +0200, Kay Sievers wrote:
> > On Mon, May 25, 2009 at 04:06, Alan Stern <stern@rowland.harvard.edu> wrote:
> 
> > > by the way -- so it's a little difficult to trigger.
> > 
> > I can trigger it pretty reliable now on plain -rc7 , but only with
> > more hubs in-between the storage device. It usually take less than
> > 10-15 connect/disconnect cycles.
> > 
> > It looks like a serious bug though, after the bug triggered, random,
> > likely unrelated, applications crash, and I can not cleanly shot down
> > anymore.
> > 
> > > I posted a patch, but the
> > > reporter never said whether or not the patch fixed the problem.  Hence
> > > the patch hasn't been submitted.
> > >
> > > Here it is for you to try out.
> > 
> > I'll give it try now.
> 
> It still shows the same issue. Here is the trace with the target
> directory left-over, when the host directory goes away: "host5/target5:0:0",
> and the devpath with the parents lost "path = '/host5/target5:0:0'":
> 
> Thanks,
> Kay
> 
> 
> [   58.399021] kobject: 'host5' (ffff88012c52b558): kobject_uevent_env
> [   58.399041] kobject: 'host5' (ffff88012c52b558): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host5/scsi_host/host5'
> [   58.399213] kobject: 'scsi_host' (ffff88012548cdd0): kobject_cleanup
> [   58.399217] kobject: 'scsi_host' (ffff88012548cdd0): auto cleanup kobject_del
> [   58.399236] kobject: 'scsi_host' (ffff88012548cdd0): calling ktype release
> [   58.399239] kobject: (ffff88012548cdd0): dynamic_kobj_release
> [   58.399243] kobject: 'scsi_host': free name
> [   58.399247] kobject: 'host5' (ffff88012c52b558): kobject_cleanup
> [   58.399250] kobject: 'host5' (ffff88012c52b558): calling ktype release
> [   58.399255] kobject: 'host5': free name
> [   58.399315] kobject: 'host5' (ffff88012c52b388): kobject_uevent_env
> [   58.399335] kobject: 'host5' (ffff88012c52b388): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host5'
> [   58.399484] ------------[ cut here ]------------
> [   58.399493] WARNING: at fs/sysfs/dir.c:794 sysfs_remove_dir+0xb2/0xd0()
> [   58.399497] Hardware name: 2776LEG
> [   58.399499] XXX dir: host5/target5:0:0
> ...
> [   58.399594] Pid: 226, comm: khubd Not tainted 2.6.30-rc7-dirty #40
> [   58.399598] Call Trace:
> [   58.399605]  [<ffffffff8023c6a8>] warn_slowpath_common+0x78/0xb0
> [   58.399610]  [<ffffffff8023c73c>] warn_slowpath_fmt+0x3c/0x40
> [   58.399614]  [<ffffffff803229f6>] ? sysfs_addrm_start+0x76/0xd0
> [   58.399619]  [<ffffffff80323012>] sysfs_remove_dir+0xb2/0xd0
> [   58.399626]  [<ffffffff803e5036>] kobject_del+0x16/0x40
> [   58.399632]  [<ffffffff80477645>] device_del+0x165/0x1a0
> [   58.399638]  [<ffffffff8048256f>] scsi_remove_host+0xcf/0x120
> [   58.399652]  [<ffffffffa02fd3cb>] quiesce_and_remove_host+0x6b/0xb0 [usb_storage]
> [   58.399662]  [<ffffffffa02fd4f8>] usb_stor_disconnect+0x18/0x30 [usb_storage]
> [   58.399686]  [<ffffffffa0062fae>] usb_unbind_interface+0x6e/0x140 [usbcore]
> [   58.399694]  [<ffffffff80479e29>] __device_release_driver+0x59/0xa0
> [   58.399699]  [<ffffffff80479f68>] device_release_driver+0x28/0x40
> [   58.399704]  [<ffffffff8047927c>] bus_remove_device+0xac/0xe0
> [   58.399709]  [<ffffffff80477607>] device_del+0x127/0x1a0
> [   58.399726]  [<ffffffffa005fb77>] usb_disable_device+0xa7/0x130 [usbcore]
> [   58.399744]  [<ffffffffa005a818>] usb_disconnect+0xc8/0x140 [usbcore]
> [   58.399761]  [<ffffffffa005a804>] usb_disconnect+0xb4/0x140 [usbcore]
> [   58.399778]  [<ffffffffa005b8db>] hub_thread+0x50b/0x1230 [usbcore]
> [   58.399784]  [<ffffffff80565a56>] ? _spin_unlock_irq+0x26/0x30
> [   58.399790]  [<ffffffff80237d1e>] ? finish_task_switch+0x7e/0x140
> [   58.399795]  [<ffffffff80237cdb>] ? finish_task_switch+0x3b/0x140
> [   58.399802]  [<ffffffff802549e0>] ? autoremove_wake_function+0x0/0x40
> [   58.399818]  [<ffffffffa005b3d0>] ? hub_thread+0x0/0x1230 [usbcore]
> [   58.399823]  [<ffffffff802545b5>] kthread+0x55/0xa0
> [   58.399829]  [<ffffffff8020cf3a>] child_rip+0xa/0x20
> [   58.399833]  [<ffffffff80254560>] ? kthread+0x0/0xa0
> [   58.399838]  [<ffffffff8020cf30>] ? child_rip+0x0/0x20
> [   58.399842] ---[ end trace a5fdfdfd6227b73e ]---
> ...
> [   58.853385] kobject: 'target5:0:0' (ffff880129980480): kobject_uevent_env
> [   58.853405] kobject: 'target5:0:0' (ffff880129980480): fill_kobj_path: path = '/host5/target5:0:0'
> [   58.853643] kobject: 'target5:0:0' (ffff880129980480): kobject_cleanup
> [   58.853647] kobject: 'target5:0:0' (ffff880129980480): calling ktype release
> [   58.853653] kobject: 'host5' (ffff88012c52b388): kobject_cleanup
> [   58.853657] kobject: 'host5' (ffff88012c52b388): calling ktype release
> [   58.853701] kobject: '2-2.4:1.0' (ffff8801255319f0): kobject_cleanup
> [   58.853705] kobject: '2-2.4:1.0' (ffff8801255319f0): calling ktype release
> [   58.853721] kobject: '2-2.4:1.0': free name
> [   58.853736] kobject: 'host5': free name
> [   58.853742] kobject: 'target5:0:0': free name
> [   58.853748] kobject: '5:0:0:0': free name

Can you provide more of the context?  I'd like to see the log starting 
from when these devices were first registered.

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Kay Sievers]

On Mon, 2009-05-25 at 11:49 -0400, Alan Stern wrote:
> Since this appears to be a bug in the SCSI layer, let's add some SCSI
> people to the CC: list.
> 
> To summarize the problem: The SCSI core tries to unregister a host 
> while its sysfs directory is still non-empty because the target hasn't 
> been unregistered yet.

> Can you provide more of the context?  I'd like to see the log starting 
> from when these devices were first registered.

I was able to trigger it with a USB storage device only, connected to a
hub, and I removed the hub from the host.

Kay


[21415.579166] usb 2-2: new high speed USB device using ehci_hcd and address 26
[21415.693822] kobject: '2-2' (ffff8801390c9128): kobject_add_internal: parent: 'usb2', set: 'devices'
[21415.694151] kobject: '2-2' (ffff8801390c9128): kobject_uevent_env
[21415.694186] kobject: '2-2' (ffff8801390c9128): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2'
[21415.694299] usb 2-2: configuration #1 chosen from 1 choice
[21415.695512] kobject: '2-2:1.0' (ffff8800b9716b48): kobject_add_internal: parent: '2-2', set: 'devices'
[21415.695612] kobject: '2-2:1.0' (ffff8800b9716b48): kobject_uevent_env
[21415.695645] kobject: '2-2:1.0' (ffff8800b9716b48): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2:1.0'
[21415.695750] hub 2-2:1.0: USB hub found
[21415.696415] hub 2-2:1.0: 4 ports detected
[21415.701428] kobject: 'usb_endpoint' (ffff8800b9648198): kobject_add_internal: parent: '2-2:1.0', set: '<NULL>'
[21415.701457] kobject: 'usbdev2.26_ep81' (ffff8800b96f56f0): kobject_add_internal: parent: 'usb_endpoint', set: 'devices'
[21415.701656] kobject: 'usbdev2.26_ep81' (ffff8800b96f56f0): kobject_uevent_env
[21415.701694] kobject: 'usbdev2.26_ep81' (ffff8800b96f56f0): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2:1.0/usb_endpoint/usbdev2.26_ep81'
[21415.701826] kobject: 'usb_endpoint' (ffff8800b96486e8): kobject_add_internal: parent: '2-2', set: '<NULL>'
[21415.701847] kobject: 'usbdev2.26_ep00' (ffff8800b96f5dc8): kobject_add_internal: parent: 'usb_endpoint', set: 'devices'
[21415.701972] kobject: 'usbdev2.26_ep00' (ffff8800b96f5dc8): kobject_uevent_env
[21415.702030] kobject: 'usbdev2.26_ep00' (ffff8800b96f5dc8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/usb_endpoint/usbdev2.26_ep00'
[21415.974325] usb 2-2.4: new high speed USB device using ehci_hcd and address 27
[21416.060815] kobject: '2-2.4' (ffff88013422cb20): kobject_add_internal: parent: '2-2', set: 'devices'
[21416.061184] kobject: '2-2.4' (ffff88013422cb20): kobject_uevent_env
[21416.061218] kobject: '2-2.4' (ffff88013422cb20): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4'
[21416.061330] usb 2-2.4: configuration #1 chosen from 1 choice
[21416.063752] kobject: '2-2.4:1.0' (ffff88013a9cc908): kobject_add_internal: parent: '2-2.4', set: 'devices'
[21416.063862] kobject: '2-2.4:1.0' (ffff88013a9cc908): kobject_uevent_env
[21416.063897] kobject: '2-2.4:1.0' (ffff88013a9cc908): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0'
[21416.097456] scsi10 : SCSI emulation for USB Mass Storage devices
[21416.097474] kobject: 'host10' (ffff8801001592f8): kobject_add_internal: parent: '2-2.4:1.0', set: 'devices'
[21416.097520] kobject: 'host10' (ffff8801001592f8): kobject_uevent_env
[21416.097541] kobject: 'host10' (ffff8801001592f8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10'
[21416.097797] kobject: 'scsi_host' (ffff8801340d3880): kobject_add_internal: parent: 'host10', set: '<NULL>'
[21416.097808] kobject: 'host10' (ffff8801001594c8): kobject_add_internal: parent: 'scsi_host', set: 'devices'
[21416.097888] kobject: 'host10' (ffff8801001594c8): kobject_uevent_env
[21416.097908] kobject: 'host10' (ffff8801001594c8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/scsi_host/host10'
[21416.109077] kobject: 'usb_endpoint' (ffff8801340d3f68): kobject_add_internal: parent: '2-2.4:1.0', set: '<NULL>'
[21416.109096] kobject: 'usbdev2.27_ep81' (ffff8800b9668b88): kobject_add_internal: parent: 'usb_endpoint', set: 'devices'
[21416.109238] kobject: 'usbdev2.27_ep81' (ffff8800b9668b88): kobject_uevent_env
[21416.109259] kobject: 'usbdev2.27_ep81' (ffff8800b9668b88): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/usb_endpoint/usbdev2.27_ep81'
[21416.110897] kobject: 'usbdev2.27_ep02' (ffff8800b9668268): kobject_add_internal: parent: 'usb_endpoint', set: 'devices'
[21416.111007] kobject: 'usbdev2.27_ep02' (ffff8800b9668268): kobject_uevent_env
[21416.111027] kobject: 'usbdev2.27_ep02' (ffff8800b9668268): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/usb_endpoint/usbdev2.27_ep02'
[21416.112326] kobject: 'usb_endpoint' (ffff880115b54990): kobject_add_internal: parent: '2-2.4', set: '<NULL>'
[21416.112340] kobject: 'usbdev2.27_ep00' (ffff8800b9668dd0): kobject_add_internal: parent: 'usb_endpoint', set: 'devices'
[21416.112439] kobject: 'usbdev2.27_ep00' (ffff8800b9668dd0): kobject_uevent_env
[21416.112459] kobject: 'usbdev2.27_ep00' (ffff8800b9668dd0): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/usb_endpoint/usbdev2.27_ep00'
[21416.112675] usb-storage: device found at 27
[21416.112678] usb-storage: waiting for device to settle before scanning
[21421.113280] scsi 10:0:0:0: Direct-Access     SanDisk  U3 Cruzer Micro  8.02 PQ: 0 ANSI: 0 CCS
[21421.113304] kobject: 'target10:0:0' (ffff88013a9c9158): kobject_add_internal: parent: 'host10', set: 'devices'
[21421.113378] kobject: 'target10:0:0' (ffff88013a9c9158): kobject_uevent_env
[21421.113414] kobject: 'target10:0:0' (ffff88013a9c9158): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0'
[21421.113836] kobject: '10:0:0:0' (ffff88012bb87588): kobject_add_internal: parent: 'target10:0:0', set: 'devices'
[21421.118125] kobject: '10:0:0:0' (ffff88012bb87588): kobject_uevent_env
[21421.118200] kobject: '10:0:0:0' (ffff88012bb87588): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0'
[21421.120456] kobject: 'scsi_device' (ffff8801078d8220): kobject_add_internal: parent: '10:0:0:0', set: '<NULL>'
[21421.120476] kobject: '10:0:0:0' (ffff88012bb87758): kobject_add_internal: parent: 'scsi_device', set: 'devices'
[21421.120538] kobject: '10:0:0:0' (ffff88012bb87758): kobject_uevent_env
[21421.120575] kobject: '10:0:0:0' (ffff88012bb87758): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/scsi_device/10:0:0:0'
[21421.121003] kobject: 'scsi_generic' (ffff8801388d55d8): kobject_add_internal: parent: '10:0:0:0', set: '<NULL>'
[21421.121022] kobject: 'sg3' (ffff8800b966a000): kobject_add_internal: parent: 'scsi_generic', set: 'devices'
[21421.121197] kobject: 'sg3' (ffff8800b966a000): kobject_uevent_env
[21421.121233] kobject: 'sg3' (ffff8800b966a000): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/scsi_generic/sg3'
[21421.127335] sd 10:0:0:0: Attached scsi generic sg3 type 0
[21421.127393] kobject: 'bsg' (ffff8801388d56e8): kobject_add_internal: parent: '10:0:0:0', set: '<NULL>'
[21421.127412] kobject: '10:0:0:0' (ffff8800b966a248): kobject_add_internal: parent: 'bsg', set: 'devices'
[21421.127541] kobject: '10:0:0:0' (ffff8800b966a248): kobject_uevent_env
[21421.127581] kobject: '10:0:0:0' (ffff8800b966a248): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/bsg/10:0:0:0'
[21421.128699] scsi 10:0:0:1: CD-ROM            SanDisk  U3 Cruzer Micro  8.02 PQ: 0 ANSI: 0
[21421.128720] kobject: '10:0:0:1' (ffff880139003b90): kobject_add_internal: parent: 'target10:0:0', set: 'devices'
[21421.128870] kobject: '10:0:0:1' (ffff880139003b90): kobject_uevent_env
[21421.128904] kobject: '10:0:0:1' (ffff880139003b90): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:1'
[21421.132180] sr0: scsi3-mmc drive: 48x/48x tray
[21421.132220] kobject: 'block' (ffff880139009d48): kobject_add_internal: parent: '10:0:0:1', set: '<NULL>'
[21421.132241] kobject: 'sr0' (ffff880138836fa8): kobject_add_internal: parent: 'block', set: 'devices'
[21421.132378] kobject: 'sr0' (ffff880138836fa8): kobject_uevent_env
[21421.132385] kobject: 'sr0' (ffff880138836fa8): kobject_uevent_env: uevent_suppress caused the event to drop!
[21421.132417] kobject: 'holders' (ffff880129f63440): kobject_add_internal: parent: 'sr0', set: '<NULL>'
[21421.132437] kobject: 'slaves' (ffff88013886c6e8): kobject_add_internal: parent: 'sr0', set: '<NULL>'
[21421.132450] kobject: 'sr0' (ffff880138836fa8): kobject_uevent_env
[21421.132485] kobject: 'sr0' (ffff880138836fa8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:1/block/sr0'
[21421.132837] kobject: 'queue' (ffff88013aea0ff0): kobject_add_internal: parent: 'sr0', set: '<NULL>'
[21421.132903] kobject: 'queue' (ffff88013aea0ff0): kobject_uevent_env
[21421.132910] kobject: 'queue' (ffff88013aea0ff0): kobject_uevent_env: filter function caused the event to drop!
[21421.132921] kobject: 'iosched' (ffff8801350a5c40): kobject_add_internal: parent: 'queue', set: '<NULL>'
[21421.136715] kobject: 'iosched' (ffff8801350a5c40): kobject_uevent_env
[21421.136721] kobject: 'iosched' (ffff8801350a5c40): kobject_uevent_env: filter function caused the event to drop!
[21421.136738] kobject: '11:0' (ffff8800b96696e0): kobject_add_internal: parent: 'bdi', set: 'devices'
[21421.136785] kobject: '11:0' (ffff8800b96696e0): kobject_uevent_env
[21421.136804] kobject: '11:0' (ffff8800b96696e0): fill_kobj_path: path = '/devices/virtual/bdi/11:0'
[21421.137067] sr 10:0:0:1: Attached scsi CD-ROM sr0
[21421.137086] kobject: 'scsi_device' (ffff880129c6a550): kobject_add_internal: parent: '10:0:0:1', set: '<NULL>'
[21421.137098] kobject: '10:0:0:1' (ffff880139003d60): kobject_add_internal: parent: 'scsi_device', set: 'devices'
[21421.137148] kobject: '10:0:0:1' (ffff880139003d60): kobject_uevent_env
[21421.137170] kobject: '10:0:0:1' (ffff880139003d60): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:1/scsi_device/10:0:0:1'
[21421.137369] kobject: 'scsi_generic' (ffff880129c6a6e8): kobject_add_internal: parent: '10:0:0:1', set: '<NULL>'
[21421.137380] kobject: 'sg4' (ffff8800b9669928): kobject_add_internal: parent: 'scsi_generic', set: 'devices'
[21421.137463] kobject: 'sg4' (ffff8800b9669928): kobject_uevent_env
[21421.137483] kobject: 'sg4' (ffff8800b9669928): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:1/scsi_generic/sg4'
[21421.137660] sr 10:0:0:1: Attached scsi generic sg4 type 5
[21421.137683] kobject: 'bsg' (ffff880129c6add0): kobject_add_internal: parent: '10:0:0:1', set: '<NULL>'
[21421.137692] kobject: '10:0:0:1' (ffff8800b96686e8): kobject_add_internal: parent: 'bsg', set: 'devices'
[21421.137746] kobject: '10:0:0:1' (ffff8800b96686e8): kobject_uevent_env
[21421.137766] kobject: '10:0:0:1' (ffff8800b96686e8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:1/bsg/10:0:0:1'
[21421.139671] kobject: '10:0:0:2' (ffff8801158df588): kobject_cleanup
[21421.139676] kobject: '10:0:0:2' (ffff8801158df588): calling ktype release
[21421.139696] kobject: '<NULL>' (ffff88013a9bbc40): kobject_cleanup
[21421.139700] kobject: '<NULL>' (ffff88013a9bbc40): calling ktype release
[21421.139708] kobject: '<NULL>' (ffff88013aea2730): kobject_cleanup
[21421.139712] kobject: '<NULL>' (ffff88013aea2730): calling ktype release
[21421.139737] kobject: '10:0:0:2': free name
[21421.139858] kobject: '10:0:1:0' (ffff8801158df588): kobject_cleanup
[21421.139862] kobject: '10:0:1:0' (ffff8801158df588): calling ktype release
[21421.139876] kobject: '<NULL>' (ffff880139385af8): kobject_cleanup
[21421.139880] kobject: '<NULL>' (ffff880139385af8): calling ktype release
[21421.139887] kobject: '<NULL>' (ffff88013aea2730): kobject_cleanup
[21421.139890] kobject: '<NULL>' (ffff88013aea2730): calling ktype release
[21421.139912] kobject: '10:0:1:0': free name
[21421.139918] kobject: 'target10:0:1' (ffff88013a9c99e8): kobject_cleanup
[21421.139922] kobject: 'target10:0:1' (ffff88013a9c99e8): calling ktype release
[21421.139928] kobject: 'target10:0:1': free name
[21421.140064] kobject: '10:0:2:0' (ffff8801158df588): kobject_cleanup
[21421.140068] kobject: '10:0:2:0' (ffff8801158df588): calling ktype release
[21421.140082] kobject: '<NULL>' (ffff880139385af8): kobject_cleanup
[21421.140086] kobject: '<NULL>' (ffff880139385af8): calling ktype release
[21421.140092] kobject: '<NULL>' (ffff88013aea2730): kobject_cleanup
[21421.140096] kobject: '<NULL>' (ffff88013aea2730): calling ktype release
[21421.140118] kobject: '10:0:2:0': free name
[21421.140124] kobject: 'target10:0:2' (ffff880124839158): kobject_cleanup
[21421.140142] kobject: 'target10:0:2' (ffff880124839158): calling ktype release
[21421.140149] kobject: 'target10:0:2': free name
[21421.140267] kobject: '10:0:3:0' (ffff8801158df588): kobject_cleanup
[21421.140271] kobject: '10:0:3:0' (ffff8801158df588): calling ktype release
[21421.140285] kobject: '<NULL>' (ffff880139385af8): kobject_cleanup
[21421.140289] kobject: '<NULL>' (ffff880139385af8): calling ktype release
[21421.140296] kobject: '<NULL>' (ffff88013aea2730): kobject_cleanup
[21421.140299] kobject: '<NULL>' (ffff88013aea2730): calling ktype release
[21421.140321] kobject: '10:0:3:0': free name
[21421.140329] kobject: 'target10:0:3' (ffff880121091158): kobject_cleanup
[21421.140332] kobject: 'target10:0:3' (ffff880121091158): calling ktype release
[21421.140338] kobject: 'target10:0:3': free name
[21421.140459] kobject: '10:0:4:0' (ffff8801158df588): kobject_cleanup
[21421.140463] kobject: '10:0:4:0' (ffff8801158df588): calling ktype release
[21421.140477] kobject: '<NULL>' (ffff880139385af8): kobject_cleanup
[21421.140481] kobject: '<NULL>' (ffff880139385af8): calling ktype release
[21421.140488] kobject: '<NULL>' (ffff88013aea2730): kobject_cleanup
[21421.140491] kobject: '<NULL>' (ffff88013aea2730): calling ktype release
[21421.140513] kobject: '10:0:4:0': free name
[21421.140520] kobject: 'target10:0:4' (ffff880121091158): kobject_cleanup
[21421.140524] kobject: 'target10:0:4' (ffff880121091158): calling ktype release
[21421.140529] kobject: 'target10:0:4': free name
[21421.140652] kobject: '10:0:5:0' (ffff8801158df588): kobject_cleanup
[21421.140655] kobject: '10:0:5:0' (ffff8801158df588): calling ktype release
[21421.140669] kobject: '<NULL>' (ffff880139385af8): kobject_cleanup
[21421.140673] kobject: '<NULL>' (ffff880139385af8): calling ktype release
[21421.140680] kobject: '<NULL>' (ffff88013aea2730): kobject_cleanup
[21421.140684] kobject: '<NULL>' (ffff88013aea2730): calling ktype release
[21421.140706] kobject: '10:0:5:0': free name
[21421.140713] kobject: 'target10:0:5' (ffff880121091158): kobject_cleanup
[21421.140716] kobject: 'target10:0:5' (ffff880121091158): calling ktype release
[21421.140722] kobject: 'target10:0:5': free name
[21421.140839] kobject: '10:0:6:0' (ffff8801158df588): kobject_cleanup
[21421.140843] kobject: '10:0:6:0' (ffff8801158df588): calling ktype release
[21421.140857] kobject: '<NULL>' (ffff880139385af8): kobject_cleanup
[21421.140861] kobject: '<NULL>' (ffff880139385af8): calling ktype release
[21421.140868] kobject: '<NULL>' (ffff88013aea2730): kobject_cleanup
[21421.140871] kobject: '<NULL>' (ffff88013aea2730): calling ktype release
[21421.140893] kobject: '10:0:6:0': free name
[21421.140900] kobject: 'target10:0:6' (ffff880121091158): kobject_cleanup
[21421.140904] kobject: 'target10:0:6' (ffff880121091158): calling ktype release
[21421.140909] kobject: 'target10:0:6': free name
[21421.141026] kobject: '10:0:7:0' (ffff8801158df588): kobject_cleanup
[21421.141030] kobject: '10:0:7:0' (ffff8801158df588): calling ktype release
[21421.141044] kobject: '<NULL>' (ffff880139385af8): kobject_cleanup
[21421.141048] kobject: '<NULL>' (ffff880139385af8): calling ktype release
[21421.141054] kobject: '<NULL>' (ffff88013aea2730): kobject_cleanup
[21421.141058] kobject: '<NULL>' (ffff88013aea2730): calling ktype release
[21421.141080] kobject: '10:0:7:0': free name
[21421.141086] kobject: 'target10:0:7' (ffff880121091158): kobject_cleanup
[21421.141090] kobject: 'target10:0:7' (ffff880121091158): calling ktype release
[21421.141095] kobject: 'target10:0:7': free name
[21421.141101] usb-storage: device scan complete
[21421.170592] kobject: 'scsi_disk' (ffff880134173770): kobject_add_internal: parent: '10:0:0:0', set: '<NULL>'
[21421.170611] kobject: '10:0:0:0' (ffff8800b9669dc8): kobject_add_internal: parent: 'scsi_disk', set: 'devices'
[21421.170684] kobject: '10:0:0:0' (ffff8800b9669dc8): kobject_uevent_env
[21421.170708] kobject: '10:0:0:0' (ffff8800b9669dc8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/scsi_disk/10:0:0:0'
[21421.171628] sd 10:0:0:0: [sdd] 31777279 512-byte hardware sectors: (16.2 GB/15.1 GiB)
[21421.172223] sd 10:0:0:0: [sdd] Write Protect is off
[21421.172228] sd 10:0:0:0: [sdd] Mode Sense: 45 00 00 08
[21421.172231] sd 10:0:0:0: [sdd] Assuming drive cache: write through
[21421.172265] kobject: 'block' (ffff8800b14ae000): kobject_add_internal: parent: '10:0:0:0', set: '<NULL>'
[21421.172285] kobject: 'sdd' (ffff88013af24920): kobject_add_internal: parent: 'block', set: 'devices'
[21421.188631] kobject: 'sdd' (ffff88013af24920): kobject_uevent_env
[21421.188637] kobject: 'sdd' (ffff88013af24920): kobject_uevent_env: uevent_suppress caused the event to drop!
[21421.188658] kobject: 'holders' (ffff8800b9648880): kobject_add_internal: parent: 'sdd', set: '<NULL>'
[21421.188669] kobject: 'slaves' (ffff8800b9648908): kobject_add_internal: parent: 'sdd', set: '<NULL>'
[21421.189054] kobject: '10:0:0:0' (ffff88012bb87588): kobject_uevent_env
[21421.189077] kobject: '10:0:0:0' (ffff88012bb87588): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0'
[21421.190637] sd 10:0:0:0: [sdd] Assuming drive cache: write through
[21421.190671]  sdd: sdd1
[21421.191270] kobject: 'sdd' (ffff88013af24920): kobject_uevent_env
[21421.191275] kobject: 'sdd' (ffff88013af24920): kobject_uevent_env: uevent_suppress caused the event to drop!
[21421.191282] sdd: p1 size 31793076 limited to end of disk
[21421.191305] kobject: 'sdd1' (ffff8800b9711588): kobject_add_internal: parent: 'sdd', set: 'devices'
[21421.191397] kobject: 'sdd1' (ffff8800b9711588): kobject_uevent_env
[21421.191401] kobject: 'sdd1' (ffff8800b9711588): kobject_uevent_env: uevent_suppress caused the event to drop!
[21421.191412] kobject: 'holders' (ffff8800b9648a18): kobject_add_internal: parent: 'sdd1', set: '<NULL>'
[21421.191420] kobject: 'sdd1' (ffff8800b9711588): kobject_uevent_env
[21421.191441] kobject: 'sdd1' (ffff8800b9711588): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/block/sdd/sdd1'
[21421.192001] kobject: 'sdd' (ffff88013af24920): kobject_uevent_env
[21421.192034] kobject: 'sdd' (ffff88013af24920): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/block/sdd'
[21421.192209] kobject: 'sdd1' (ffff8800b9711588): kobject_uevent_env
[21421.192230] kobject: 'sdd1' (ffff8800b9711588): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/block/sdd/sdd1'
[21421.192399] kobject: 'queue' (ffff88013aea1b90): kobject_add_internal: parent: 'sdd', set: '<NULL>'
[21421.192445] kobject: 'queue' (ffff88013aea1b90): kobject_uevent_env
[21421.192450] kobject: 'queue' (ffff88013aea1b90): kobject_uevent_env: filter function caused the event to drop!
[21421.192456] kobject: 'iosched' (ffff8801392a6158): kobject_add_internal: parent: 'queue', set: '<NULL>'
[21421.192488] kobject: 'iosched' (ffff8801392a6158): kobject_uevent_env
[21421.192492] kobject: 'iosched' (ffff8801392a6158): kobject_uevent_env: filter function caused the event to drop!
[21421.192505] kobject: '259:917504' (ffff8800b96f5498): kobject_add_internal: parent: 'bdi', set: 'devices'
[21421.192557] kobject: '259:917504' (ffff8800b96f5498): kobject_uevent_env
[21421.192575] kobject: '259:917504' (ffff8800b96f5498): fill_kobj_path: path = '/devices/virtual/bdi/259:917504'
[21421.192762] sd 10:0:0:0: [sdd] Attached SCSI removable disk
[21421.258331] kobject: 'sdd' (ffff88013af24920): kobject_uevent_env
[21421.258358] kobject: 'sdd' (ffff88013af24920): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/block/sdd'
[21421.276118] kobject: '65534' (ffff88013ab522e8): kobject_add_internal: parent: 'uids', set: 'uids'
[21421.281107] kobject: '65534' (ffff88013ab522e8): kobject_uevent_env
[21421.281162] kobject: '65534' (ffff88013ab522e8): fill_kobj_path: path = '/kernel/uids/65534'
[21421.333642] kobject: '10:0:0:1' (ffff880139003b90): kobject_uevent_env
[21421.333667] kobject: '10:0:0:1' (ffff880139003b90): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:1'
[21421.335399] kobject: '65534' (ffff88013ab522e8): kobject_uevent_env
[21421.335420] kobject: '65534' (ffff88013ab522e8): fill_kobj_path: path = '/kernel/uids/65534'
[21421.335554] kobject: '65534' (ffff88013ab522e8): kobject_cleanup
[21421.335558] kobject: '65534' (ffff88013ab522e8): calling ktype release
[21421.335562] kobject: '65534': free name
[21421.340587] kobject: '65534' (ffff88013ab522e8): kobject_add_internal: parent: 'uids', set: 'uids'
[21421.340610] kobject: '65534' (ffff88013ab522e8): kobject_uevent_env
[21421.340631] kobject: '65534' (ffff88013ab522e8): fill_kobj_path: path = '/kernel/uids/65534'
[21421.361164] kobject: '65534' (ffff88013ab522e8): kobject_uevent_env
[21421.361185] kobject: '65534' (ffff88013ab522e8): fill_kobj_path: path = '/kernel/uids/65534'
[21421.361344] kobject: '65534' (ffff88013ab522e8): kobject_cleanup
[21421.361348] kobject: '65534' (ffff88013ab522e8): calling ktype release
[21421.361351] kobject: '65534': free name
[21421.425012] kobject: '65534' (ffff88013ab522e8): kobject_add_internal: parent: 'uids', set: 'uids'
[21421.425033] kobject: '65534' (ffff88013ab522e8): kobject_uevent_env
[21421.425051] kobject: '65534' (ffff88013ab522e8): fill_kobj_path: path = '/kernel/uids/65534'
[21421.447972] kobject: 'sr0' (ffff880138836fa8): kobject_uevent_env
[21421.448000] kobject: 'sr0' (ffff880138836fa8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:1/block/sr0'
[21421.462036] kobject: '65534' (ffff88013ab522e8): kobject_uevent_env
[21421.462058] kobject: '65534' (ffff88013ab522e8): fill_kobj_path: path = '/kernel/uids/65534'
[21421.462129] kobject: '65534' (ffff88013ab522e8): kobject_cleanup
[21421.462133] kobject: '65534' (ffff88013ab522e8): calling ktype release
[21421.462136] kobject: '65534': free name
[21421.508994] kobject: '65534' (ffff88013ab522e8): kobject_add_internal: parent: 'uids', set: 'uids'
[21421.509015] kobject: '65534' (ffff88013ab522e8): kobject_uevent_env
[21421.509034] kobject: '65534' (ffff88013ab522e8): fill_kobj_path: path = '/kernel/uids/65534'
[21421.538933] kobject: '65534' (ffff88013ab522e8): kobject_uevent_env
[21421.538955] kobject: '65534' (ffff88013ab522e8): fill_kobj_path: path = '/kernel/uids/65534'
[21421.539110] kobject: '65534' (ffff88013ab522e8): kobject_cleanup
[21421.539114] kobject: '65534' (ffff88013ab522e8): calling ktype release
[21421.539118] kobject: '65534': free name
[21421.548145] kobject: '65534' (ffff88013ab522e8): kobject_add_internal: parent: 'uids', set: 'uids'
[21421.548165] kobject: '65534' (ffff88013ab522e8): kobject_uevent_env
[21421.548184] kobject: '65534' (ffff88013ab522e8): fill_kobj_path: path = '/kernel/uids/65534'
[21421.588200] kobject: '65534' (ffff88013ab522e8): kobject_uevent_env
[21421.588221] kobject: '65534' (ffff88013ab522e8): fill_kobj_path: path = '/kernel/uids/65534'
[21421.588365] kobject: '65534' (ffff88013ab522e8): kobject_cleanup
[21421.588370] kobject: '65534' (ffff88013ab522e8): calling ktype release
[21421.588373] kobject: '65534': free name
[21421.722647] kobject: '65534' (ffff8801390f02e8): kobject_add_internal: parent: 'uids', set: 'uids'
[21421.722668] kobject: '65534' (ffff8801390f02e8): kobject_uevent_env
[21421.722687] kobject: '65534' (ffff8801390f02e8): fill_kobj_path: path = '/kernel/uids/65534'
[21421.756036] kobject: '65534' (ffff8801390f02e8): kobject_uevent_env
[21421.756057] kobject: '65534' (ffff8801390f02e8): fill_kobj_path: path = '/kernel/uids/65534'
[21421.756195] kobject: '65534' (ffff8801390f02e8): kobject_cleanup
[21421.756199] kobject: '65534' (ffff8801390f02e8): calling ktype release
[21421.756202] kobject: '65534': free name
[21423.065910] usb 2-2: USB disconnect, address 26
[21423.065915] usb 2-2.4: USB disconnect, address 27
[21423.066144] kobject: 'usbdev2.27_ep81' (ffff8800b9668b88): kobject_uevent_env
[21423.066168] kobject: 'usbdev2.27_ep81' (ffff8800b9668b88): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/usb_endpoint/usbdev2.27_ep81'
[21423.066227] kobject: 'usbdev2.27_ep81' (ffff8800b9668b88): kobject_cleanup
[21423.066231] kobject: 'usbdev2.27_ep81' (ffff8800b9668b88): calling ktype release
[21423.066245] kobject: 'usbdev2.27_ep81': free name
[21423.066363] kobject: 'usbdev2.27_ep02' (ffff8800b9668268): kobject_uevent_env
[21423.066383] kobject: 'usbdev2.27_ep02' (ffff8800b9668268): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/usb_endpoint/usbdev2.27_ep02'
[21423.066430] kobject: 'usb_endpoint' (ffff8801340d3f68): kobject_cleanup
[21423.066434] kobject: 'usb_endpoint' (ffff8801340d3f68): auto cleanup kobject_del
[21423.066453] kobject: 'usb_endpoint' (ffff8801340d3f68): calling ktype release
[21423.066457] kobject: (ffff8801340d3f68): dynamic_kobj_release
[21423.066462] kobject: 'usb_endpoint': free name
[21423.066466] kobject: 'usbdev2.27_ep02' (ffff8800b9668268): kobject_cleanup
[21423.066470] kobject: 'usbdev2.27_ep02' (ffff8800b9668268): calling ktype release
[21423.066476] kobject: 'usbdev2.27_ep02': free name
[21423.066708] kobject: '10:0:0:0' (ffff8800b966a248): kobject_uevent_env
[21423.066727] kobject: '10:0:0:0' (ffff8800b966a248): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/bsg/10:0:0:0'
[21423.066773] kobject: 'bsg' (ffff8801388d56e8): kobject_cleanup
[21423.066776] kobject: 'bsg' (ffff8801388d56e8): auto cleanup kobject_del
[21423.066795] kobject: 'bsg' (ffff8801388d56e8): calling ktype release
[21423.066798] kobject: (ffff8801388d56e8): dynamic_kobj_release
[21423.066802] kobject: 'bsg': free name
[21423.066806] kobject: '10:0:0:0' (ffff8800b966a248): kobject_cleanup
[21423.066808] kobject: '10:0:0:0' (ffff8800b966a248): calling ktype release
[21423.066815] kobject: '10:0:0:0': free name
[21423.066928] kobject: 'sg3' (ffff8800b966a000): kobject_uevent_env
[21423.066945] kobject: 'sg3' (ffff8800b966a000): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/scsi_generic/sg3'
[21423.066988] kobject: 'scsi_generic' (ffff8801388d55d8): kobject_cleanup
[21423.066991] kobject: 'scsi_generic' (ffff8801388d55d8): auto cleanup kobject_del
[21423.067020] kobject: 'scsi_generic' (ffff8801388d55d8): calling ktype release
[21423.067023] kobject: (ffff8801388d55d8): dynamic_kobj_release
[21423.067026] kobject: 'scsi_generic': free name
[21423.067030] kobject: 'sg3' (ffff8800b966a000): kobject_cleanup
[21423.067033] kobject: 'sg3' (ffff8800b966a000): calling ktype release
[21423.067038] kobject: 'sg3': free name
[21423.067045] kobject: '<NULL>' (ffff8800b154c578): kobject_cleanup
[21423.067048] kobject: '<NULL>' (ffff8800b154c578): calling ktype release
[21423.067056] kobject: '<NULL>' (ffff88013af21178): kobject_cleanup
[21423.067059] kobject: '<NULL>' (ffff88013af21178): calling ktype release
[21423.067077] kobject: '10:0:0:0' (ffff88012bb87758): kobject_uevent_env
[21423.067094] kobject: '10:0:0:0' (ffff88012bb87758): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/scsi_device/10:0:0:0'
[21423.067144] kobject: 'scsi_device' (ffff8801078d8220): kobject_cleanup
[21423.067148] kobject: 'scsi_device' (ffff8801078d8220): auto cleanup kobject_del
[21423.067165] kobject: 'scsi_device' (ffff8801078d8220): calling ktype release
[21423.067169] kobject: (ffff8801078d8220): dynamic_kobj_release
[21423.067174] kobject: 'scsi_device': free name
[21423.067178] kobject: '10:0:0:0' (ffff88012bb87758): kobject_cleanup
[21423.067181] kobject: '10:0:0:0' (ffff88012bb87758): calling ktype release
[21423.067186] kobject: '10:0:0:0': free name
[21423.067469] kobject: '10:0:0:0' (ffff8800b9669dc8): kobject_uevent_env
[21423.067490] kobject: '10:0:0:0' (ffff8800b9669dc8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/scsi_disk/10:0:0:0'
[21423.067543] kobject: 'scsi_disk' (ffff880134173770): kobject_cleanup
[21423.067546] kobject: 'scsi_disk' (ffff880134173770): auto cleanup kobject_del
[21423.067564] kobject: 'scsi_disk' (ffff880134173770): calling ktype release
[21423.067568] kobject: (ffff880134173770): dynamic_kobj_release
[21423.067573] kobject: 'scsi_disk': free name
[21423.106153] kobject: 'holders' (ffff8800b9648a18): kobject_cleanup
[21423.106159] kobject: 'holders' (ffff8800b9648a18): auto cleanup kobject_del
[21423.106178] kobject: 'holders' (ffff8800b9648a18): calling ktype release
[21423.106182] kobject: (ffff8800b9648a18): dynamic_kobj_release
[21423.106187] kobject: 'holders': free name
[21423.106327] kobject: 'sdd1' (ffff8800b9711588): kobject_uevent_env
[21423.106352] kobject: 'sdd1' (ffff8800b9711588): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/block/sdd/sdd1'
[21423.106507] kobject: '259:917504' (ffff8800b96f5498): kobject_uevent_env
[21423.106526] kobject: '259:917504' (ffff8800b96f5498): fill_kobj_path: path = '/devices/virtual/bdi/259:917504'
[21423.106572] kobject: '259:917504' (ffff8800b96f5498): kobject_cleanup
[21423.106575] kobject: '259:917504' (ffff8800b96f5498): calling ktype release
[21423.106584] kobject: '259:917504': free name
[21423.106590] kobject: 'iosched' (ffff8801392a6158): kobject_uevent_env
[21423.106595] kobject: 'iosched' (ffff8801392a6158): kobject_uevent_env: filter function caused the event to drop!
[21423.106628] kobject: 'queue' (ffff88013aea1b90): kobject_uevent_env
[21423.106632] kobject: 'queue' (ffff88013aea1b90): kobject_uevent_env: filter function caused the event to drop!
[21423.106680] kobject: 'holders' (ffff8800b9648880): kobject_cleanup
[21423.106684] kobject: 'holders' (ffff8800b9648880): auto cleanup kobject_del
[21423.106694] kobject: 'holders' (ffff8800b9648880): calling ktype release
[21423.106698] kobject: (ffff8800b9648880): dynamic_kobj_release
[21423.106702] kobject: 'holders': free name
[21423.106706] kobject: 'slaves' (ffff8800b9648908): kobject_cleanup
[21423.106709] kobject: 'slaves' (ffff8800b9648908): auto cleanup kobject_del
[21423.106720] kobject: 'slaves' (ffff8800b9648908): calling ktype release
[21423.106724] kobject: (ffff8800b9648908): dynamic_kobj_release
[21423.106728] kobject: 'slaves': free name
[21423.106897] kobject: 'sdd' (ffff88013af24920): kobject_uevent_env
[21423.106918] kobject: 'sdd' (ffff88013af24920): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0/block/sdd'
[21423.106970] kobject: 'block' (ffff8800b14ae000): kobject_cleanup
[21423.106974] kobject: 'block' (ffff8800b14ae000): auto cleanup kobject_del
[21423.106995] kobject: 'block' (ffff8800b14ae000): calling ktype release
[21423.106999] kobject: (ffff8800b14ae000): dynamic_kobj_release
[21423.107023] kobject: 'block': free name
[21423.111501] kobject: '10:0:0:0' (ffff88012bb87588): kobject_uevent_env
[21423.111527] kobject: '10:0:0:0' (ffff88012bb87588): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:0'
[21423.111721] kobject: '10:0:0:1' (ffff8800b96686e8): kobject_uevent_env
[21423.111743] kobject: '10:0:0:1' (ffff8800b96686e8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:1/bsg/10:0:0:1'
[21423.111796] kobject: 'bsg' (ffff880129c6add0): kobject_cleanup
[21423.111800] kobject: 'bsg' (ffff880129c6add0): auto cleanup kobject_del
[21423.111821] kobject: 'bsg' (ffff880129c6add0): calling ktype release
[21423.111825] kobject: (ffff880129c6add0): dynamic_kobj_release
[21423.111831] kobject: 'bsg': free name
[21423.111835] kobject: '10:0:0:1' (ffff8800b96686e8): kobject_cleanup
[21423.111838] kobject: '10:0:0:1' (ffff8800b96686e8): calling ktype release
[21423.111847] kobject: '10:0:0:1': free name
[21423.112456] kobject: 'sg4' (ffff8800b9669928): kobject_uevent_env
[21423.112477] kobject: 'sg4' (ffff8800b9669928): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:1/scsi_generic/sg4'
[21423.112530] kobject: 'scsi_generic' (ffff880129c6a6e8): kobject_cleanup
[21423.112534] kobject: 'scsi_generic' (ffff880129c6a6e8): auto cleanup kobject_del
[21423.112553] kobject: 'scsi_generic' (ffff880129c6a6e8): calling ktype release
[21423.112556] kobject: (ffff880129c6a6e8): dynamic_kobj_release
[21423.112560] kobject: 'scsi_generic': free name
[21423.112565] kobject: 'sg4' (ffff8800b9669928): kobject_cleanup
[21423.112568] kobject: 'sg4' (ffff8800b9669928): calling ktype release
[21423.112574] kobject: 'sg4': free name
[21423.112581] kobject: '<NULL>' (ffff8800b154ced8): kobject_cleanup
[21423.112585] kobject: '<NULL>' (ffff8800b154ced8): calling ktype release
[21423.112594] kobject: '<NULL>' (ffff88013a9f2298): kobject_cleanup
[21423.112597] kobject: '<NULL>' (ffff88013a9f2298): calling ktype release
[21423.112618] kobject: '10:0:0:1' (ffff880139003d60): kobject_uevent_env
[21423.112638] kobject: '10:0:0:1' (ffff880139003d60): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:1/scsi_device/10:0:0:1'
[21423.112687] kobject: 'scsi_device' (ffff880129c6a550): kobject_cleanup
[21423.112690] kobject: 'scsi_device' (ffff880129c6a550): auto cleanup kobject_del
[21423.112709] kobject: 'scsi_device' (ffff880129c6a550): calling ktype release
[21423.112713] kobject: (ffff880129c6a550): dynamic_kobj_release
[21423.112717] kobject: 'scsi_device': free name
[21423.112722] kobject: '10:0:0:1' (ffff880139003d60): kobject_cleanup
[21423.112725] kobject: '10:0:0:1' (ffff880139003d60): calling ktype release
[21423.112730] kobject: '10:0:0:1': free name
[21423.113117] kobject: '11:0' (ffff8800b96696e0): kobject_uevent_env
[21423.113137] kobject: '11:0' (ffff8800b96696e0): fill_kobj_path: path = '/devices/virtual/bdi/11:0'
[21423.113187] kobject: '11:0' (ffff8800b96696e0): kobject_cleanup
[21423.113191] kobject: '11:0' (ffff8800b96696e0): calling ktype release
[21423.113197] kobject: '11:0': free name
[21423.113202] kobject: 'iosched' (ffff8801350a5c40): kobject_uevent_env
[21423.113206] kobject: 'iosched' (ffff8801350a5c40): kobject_uevent_env: filter function caused the event to drop!
[21423.113248] kobject: 'queue' (ffff88013aea0ff0): kobject_uevent_env
[21423.113252] kobject: 'queue' (ffff88013aea0ff0): kobject_uevent_env: filter function caused the event to drop!
[21423.113289] kobject: 'holders' (ffff880129f63440): kobject_cleanup
[21423.113292] kobject: 'holders' (ffff880129f63440): auto cleanup kobject_del
[21423.113303] kobject: 'holders' (ffff880129f63440): calling ktype release
[21423.113306] kobject: (ffff880129f63440): dynamic_kobj_release
[21423.113311] kobject: 'holders': free name
[21423.113315] kobject: 'slaves' (ffff88013886c6e8): kobject_cleanup
[21423.113319] kobject: 'slaves' (ffff88013886c6e8): auto cleanup kobject_del
[21423.113329] kobject: 'slaves' (ffff88013886c6e8): calling ktype release
[21423.113332] kobject: (ffff88013886c6e8): dynamic_kobj_release
[21423.113337] kobject: 'slaves': free name
[21423.113500] kobject: 'sr0' (ffff880138836fa8): kobject_uevent_env
[21423.113520] kobject: 'sr0' (ffff880138836fa8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:1/block/sr0'
[21423.113568] kobject: 'block' (ffff880139009d48): kobject_cleanup
[21423.113571] kobject: 'block' (ffff880139009d48): auto cleanup kobject_del
[21423.113606] kobject: 'block' (ffff880139009d48): calling ktype release
[21423.113610] kobject: (ffff880139009d48): dynamic_kobj_release
[21423.113615] kobject: 'block': free name
[21423.113755] kobject: 'sr0' (ffff880138836fa8): kobject_cleanup
[21423.113759] kobject: 'sr0' (ffff880138836fa8): calling ktype release
[21423.113770] kobject: 'sr0': free name
[21423.113780] kobject: '10:0:0:1' (ffff880139003b90): kobject_uevent_env
[21423.113800] kobject: '10:0:0:1' (ffff880139003b90): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/target10:0:0/10:0:0:1'
[21423.113878] kobject: '10:0:0:1' (ffff880139003b90): kobject_cleanup
[21423.113882] kobject: '10:0:0:1' (ffff880139003b90): calling ktype release
[21423.113905] kobject: 'iosched' (ffff8801350a5c40): kobject_cleanup
[21423.113908] kobject: 'iosched' (ffff8801350a5c40): calling ktype release
[21423.113916] kobject: 'iosched': free name
[21423.113921] kobject: 'queue' (ffff88013aea0ff0): kobject_cleanup
[21423.113925] kobject: 'queue' (ffff88013aea0ff0): calling ktype release
[21423.113949] kobject: 'queue': free name
[21423.113961] kobject: '10:0:0:1': free name
[21423.114098] kobject: 'host10' (ffff8801001594c8): kobject_uevent_env
[21423.114118] kobject: 'host10' (ffff8801001594c8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10/scsi_host/host10'
[21423.114164] kobject: 'scsi_host' (ffff8801340d3880): kobject_cleanup
[21423.114168] kobject: 'scsi_host' (ffff8801340d3880): auto cleanup kobject_del
[21423.114186] kobject: 'scsi_host' (ffff8801340d3880): calling ktype release
[21423.114190] kobject: (ffff8801340d3880): dynamic_kobj_release
[21423.114194] kobject: 'scsi_host': free name
[21423.114198] kobject: 'host10' (ffff8801001594c8): kobject_cleanup
[21423.114201] kobject: 'host10' (ffff8801001594c8): calling ktype release
[21423.114207] kobject: 'host10': free name
[21423.114258] kobject: 'host10' (ffff8801001592f8): kobject_uevent_env
[21423.114277] kobject: 'host10' (ffff8801001592f8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0/host10'
[21423.114310] ------------[ cut here ]------------
[21423.114319] WARNING: at fs/sysfs/dir.c:794 sysfs_remove_dir+0xb2/0xd0()
[21423.114322] Hardware name: 2776LEG
[21423.114325] XXX dir: host10/target10:0:0
[21423.114327] Modules linked in: sr_mod cdrom sit tunnel4 ipv6 tun aes_x86_64 aes_generic i915 drm i2c_algo_bit i2c_core acpi_cpufreq snd_pcm_oss snd_mixer_oss snd_seq snd_seq_device edd pl2303 usbserial usblp fuse loop dm_mod usb_storage usbhid hid cdc_ether usbnet cdc_wdm cdc_acm uvcvideo videodev v4l1_compat v4l2_compat_ioctl32 arc4 ecb snd_hda_codec_conexant snd_hda_intel kvm_intel snd_hda_codec snd_pcm iwlagn thinkpad_acpi kvm snd_timer iwlcore hwmon rfkill mac80211 backlight uhci_hcd joydev pcspkr evdev led_class snd sg cfg80211 battery nvram ehci_hcd ac soundcore e1000e usbcore snd_page_alloc thermal button processor intel_agp
[21423.114421] Pid: 213, comm: khubd Tainted: G        W  2.6.30-rc7-dirty #41
[21423.114425] Call Trace:
[21423.114433]  [<ffffffff8023c6a8>] warn_slowpath_common+0x78/0xb0
[21423.114438]  [<ffffffff8023c73c>] warn_slowpath_fmt+0x3c/0x40
[21423.114442]  [<ffffffff80322a16>] ? sysfs_addrm_start+0x76/0xd0
[21423.114447]  [<ffffffff80323032>] sysfs_remove_dir+0xb2/0xd0
[21423.114455]  [<ffffffff803e5056>] kobject_del+0x16/0x40
[21423.114461]  [<ffffffff80477665>] device_del+0x165/0x1a0
[21423.114467]  [<ffffffff8048258f>] scsi_remove_host+0xcf/0x120
[21423.114481]  [<ffffffffa02c93cb>] quiesce_and_remove_host+0x6b/0xb0 [usb_storage]
[21423.114492]  [<ffffffffa02c94f8>] usb_stor_disconnect+0x18/0x30 [usb_storage]
[21423.114514]  [<ffffffffa003ffae>] usb_unbind_interface+0x6e/0x140 [usbcore]
[21423.114523]  [<ffffffff80479e49>] __device_release_driver+0x59/0xa0
[21423.114528]  [<ffffffff80479f88>] device_release_driver+0x28/0x40
[21423.114533]  [<ffffffff8047929c>] bus_remove_device+0xac/0xe0
[21423.114538]  [<ffffffff80477627>] device_del+0x127/0x1a0
[21423.114557]  [<ffffffffa003cb77>] usb_disable_device+0xa7/0x130 [usbcore]
[21423.114575]  [<ffffffffa0037818>] usb_disconnect+0xc8/0x140 [usbcore]
[21423.114593]  [<ffffffffa0037804>] usb_disconnect+0xb4/0x140 [usbcore]
[21423.114611]  [<ffffffffa00388db>] hub_thread+0x50b/0x1230 [usbcore]
[21423.114617]  [<ffffffff80565a76>] ? _spin_unlock_irq+0x26/0x30
[21423.114623]  [<ffffffff80237d1e>] ? finish_task_switch+0x7e/0x140
[21423.114629]  [<ffffffff80237cdb>] ? finish_task_switch+0x3b/0x140
[21423.114635]  [<ffffffff802549e0>] ? autoremove_wake_function+0x0/0x40
[21423.114653]  [<ffffffffa00383d0>] ? hub_thread+0x0/0x1230 [usbcore]
[21423.114658]  [<ffffffff802545b5>] kthread+0x55/0xa0
[21423.114664]  [<ffffffff8020cf3a>] child_rip+0xa/0x20
[21423.114669]  [<ffffffff80254560>] ? kthread+0x0/0xa0
[21423.114674]  [<ffffffff8020cf30>] ? child_rip+0x0/0x20
[21423.114678] ---[ end trace 89cad9e6bbbcf701 ]---
[21423.114755] kobject: '2-2.4:1.0' (ffff88013a9cc908): kobject_uevent_env
[21423.114774] kobject: '2-2.4:1.0' (ffff88013a9cc908): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/2-2.4:1.0'
[21423.114976] kobject: 'usbdev2.27_ep00' (ffff8800b9668dd0): kobject_uevent_env
[21423.114995] kobject: 'usbdev2.27_ep00' (ffff8800b9668dd0): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4/usb_endpoint/usbdev2.27_ep00'
[21423.115059] kobject: 'usb_endpoint' (ffff880115b54990): kobject_cleanup
[21423.115063] kobject: 'usb_endpoint' (ffff880115b54990): auto cleanup kobject_del
[21423.115092] kobject: 'usb_endpoint' (ffff880115b54990): calling ktype release
[21423.115096] kobject: (ffff880115b54990): dynamic_kobj_release
[21423.115101] kobject: 'usb_endpoint': free name
[21423.115106] kobject: 'usbdev2.27_ep00' (ffff8800b9668dd0): kobject_cleanup
[21423.115109] kobject: 'usbdev2.27_ep00' (ffff8800b9668dd0): calling ktype release
[21423.115117] kobject: 'usbdev2.27_ep00': free name
[21423.115513] kobject: '2-2.4' (ffff88013422cb20): kobject_uevent_env
[21423.115532] kobject: '2-2.4' (ffff88013422cb20): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.4'
[21423.115583] kobject: '2-2.4' (ffff88013422cb20): kobject_cleanup
[21423.115586] kobject: '2-2.4' (ffff88013422cb20): calling ktype release
[21423.115607] kobject: '2-2.4': free name
[21423.115808] kobject: 'usbdev2.26_ep81' (ffff8800b96f56f0): kobject_uevent_env
[21423.115828] kobject: 'usbdev2.26_ep81' (ffff8800b96f56f0): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2:1.0/usb_endpoint/usbdev2.26_ep81'
[21423.115875] kobject: 'usb_endpoint' (ffff8800b9648198): kobject_cleanup
[21423.115879] kobject: 'usb_endpoint' (ffff8800b9648198): auto cleanup kobject_del
[21423.115898] kobject: 'usb_endpoint' (ffff8800b9648198): calling ktype release
[21423.115902] kobject: (ffff8800b9648198): dynamic_kobj_release
[21423.115907] kobject: 'usb_endpoint': free name
[21423.115912] kobject: 'usbdev2.26_ep81' (ffff8800b96f56f0): kobject_cleanup
[21423.115915] kobject: 'usbdev2.26_ep81' (ffff8800b96f56f0): calling ktype release
[21423.115924] kobject: 'usbdev2.26_ep81': free name
[21423.116154] kobject: '2-2:1.0' (ffff8800b9716b48): kobject_uevent_env
[21423.116173] kobject: '2-2:1.0' (ffff8800b9716b48): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2:1.0'
[21423.116224] kobject: '2-2:1.0' (ffff8800b9716b48): kobject_cleanup
[21423.116228] kobject: '2-2:1.0' (ffff8800b9716b48): calling ktype release
[21423.116235] kobject: '2-2:1.0': free name
[21423.116364] kobject: 'usbdev2.26_ep00' (ffff8800b96f5dc8): kobject_uevent_env
[21423.116383] kobject: 'usbdev2.26_ep00' (ffff8800b96f5dc8): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2/usb_endpoint/usbdev2.26_ep00'
[21423.116432] kobject: 'usb_endpoint' (ffff8800b96486e8): kobject_cleanup
[21423.116435] kobject: 'usb_endpoint' (ffff8800b96486e8): auto cleanup kobject_del
[21423.116455] kobject: 'usb_endpoint' (ffff8800b96486e8): calling ktype release
[21423.116459] kobject: (ffff8800b96486e8): dynamic_kobj_release
[21423.116463] kobject: 'usb_endpoint': free name
[21423.116468] kobject: 'usbdev2.26_ep00' (ffff8800b96f5dc8): kobject_cleanup
[21423.116472] kobject: 'usbdev2.26_ep00' (ffff8800b96f5dc8): calling ktype release
[21423.116478] kobject: 'usbdev2.26_ep00': free name
[21423.116830] kobject: '2-2' (ffff8801390c9128): kobject_uevent_env
[21423.116849] kobject: '2-2' (ffff8801390c9128): fill_kobj_path: path = '/devices/pci0000:00/0000:00:1d.7/usb2/2-2'
[21423.116897] kobject: '2-2' (ffff8801390c9128): kobject_cleanup
[21423.116901] kobject: '2-2' (ffff8801390c9128): calling ktype release
[21423.116917] kobject: '2-2': free name
[21423.269004] kobject: 'sdd1' (ffff8800b9711588): kobject_cleanup
[21423.269009] kobject: 'sdd1' (ffff8800b9711588): calling ktype release
[21423.269021] kobject: 'sdd1': free name
[21423.269040] kobject: '10:0:0:0' (ffff8800b9669dc8): kobject_cleanup
[21423.269044] kobject: '10:0:0:0' (ffff8800b9669dc8): calling ktype release
[21423.269055] kobject: '10:0:0:0': free name
[21423.269062] kobject: '10:0:0:0' (ffff88012bb87588): kobject_cleanup
[21423.269066] kobject: '10:0:0:0' (ffff88012bb87588): calling ktype release
[21423.269103] kobject: 'iosched' (ffff8801392a6158): kobject_cleanup
[21423.269106] kobject: 'iosched' (ffff8801392a6158): calling ktype release
[21423.269114] kobject: 'iosched': free name
[21423.269119] kobject: 'queue' (ffff88013aea1b90): kobject_cleanup
[21423.269123] kobject: 'queue' (ffff88013aea1b90): calling ktype release
[21423.272572] kobject: 'queue': free name
[21423.272654] kobject: 'target10:0:0' (ffff88013a9c9158): kobject_uevent_env
[21423.272674] kobject: 'target10:0:0' (ffff88013a9c9158): fill_kobj_path: path = '/host10/target10:0:0'
[21423.272864] kobject: 'target10:0:0' (ffff88013a9c9158): kobject_cleanup
[21423.272869] kobject: 'target10:0:0' (ffff88013a9c9158): calling ktype release
[21423.272876] kobject: 'host10' (ffff8801001592f8): kobject_cleanup
[21423.272879] kobject: 'host10' (ffff8801001592f8): calling ktype release
[21423.272936] kobject: '2-2.4:1.0' (ffff88013a9cc908): kobject_cleanup
[21423.272939] kobject: '2-2.4:1.0' (ffff88013a9cc908): calling ktype release
[21423.272954] kobject: '2-2.4:1.0': free name
[21423.272969] kobject: 'host10': free name
[21423.272984] kobject: 'target10:0:0': free name
[21423.272990] kobject: '10:0:0:0': free name
[21423.272995] kobject: 'sdd' (ffff88013af24920): kobject_cleanup
[21423.272998] kobject: 'sdd' (ffff88013af24920): calling ktype release
[21423.273011] kobject: 'sdd': free name

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Mon, 25 May 2009, Kay Sievers wrote:

> On Mon, 2009-05-25 at 11:49 -0400, Alan Stern wrote:
> > Since this appears to be a bug in the SCSI layer, let's add some SCSI
> > people to the CC: list.
> > 
> > To summarize the problem: The SCSI core tries to unregister a host 
> > while its sysfs directory is still non-empty because the target hasn't 
> > been unregistered yet.
> 
> > Can you provide more of the context?  I'd like to see the log starting 
> > from when these devices were first registered.
> 
> I was able to trigger it with a USB storage device only, connected to a
> hub, and I removed the hub from the host.

Okay, it's a long log dump, but useful.  Evidently the problem is
caused by the fact that scsi_target_reap() is called by
scsi_device_dev_release_usercontext() instead of by
__scsi_remove_device() (both functions in drivers/scsi/scsi_sysfs.c).

There's probably a reason for this, but I don't know what it is.  Maybe 
James can explain.

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Tue, 26 May 2009, Kay Sievers wrote:

> On Mon, May 25, 2009 at 13:45, Kay Sievers <kay.sievers@vrfy.org> wrote:
> > On Mon, May 25, 2009 at 04:06, Alan Stern <stern@rowland.harvard.edu> wrote:
> 
> >> by the way -- so it's a little difficult to trigger.
> >
> > I can trigger it pretty reliable now on plain -rc7 , but only with
> > more hubs in-between the storage device. It usually take less than
> > 10-15 connect/disconnect cycles.
> >
> > It looks like a serious bug though, after the bug triggered, random,
> > likely unrelated, applications crash, and I can not cleanly shot down
> > anymore.
> 
> Just a heads up if anybody is trying to reproduce this, it trashed my
> ext3 rootfs, which is not recoverable.
> 
> Not sure what exactly caused this, but I didn't have anything like
> this for a very long time.
> 
> I tried to reproduce the issue a few times more, and it crashed random
> processes after the bug triggered, like mentioned above, and the box
> never shut down cleanly.
> 
> It's entirely possible, that bug causes serious issues.

If you don't mind trashing some more ext3 root filesystems :-) you can
try this patch.  It's almost certainly not quite the right thing to do
and I have probably messed up the target's reference counting, but
maybe it's a step in the right direction.

This strange business of deferring unregistration into a workqueue 
means that the calls might not be executed in the same order that 
they're made.

Alan Stern


Index: usb-2.6/drivers/scsi/scsi_scan.c
===================================================================
--- usb-2.6.orig/drivers/scsi/scsi_scan.c
+++ usb-2.6/drivers/scsi/scsi_scan.c
@@ -956,6 +956,7 @@ static inline void scsi_destroy_sdev(str
 	if (sdev->host->hostt->slave_destroy)
 		sdev->host->hostt->slave_destroy(sdev);
 	transport_destroy_device(&sdev->sdev_gendev);
+	put_device(sdev->sdev_gendev.parent);
 	put_device(&sdev->sdev_gendev);
 }
 
Index: usb-2.6/drivers/scsi/scsi_sysfs.c
===================================================================
--- usb-2.6.orig/drivers/scsi/scsi_sysfs.c
+++ usb-2.6/drivers/scsi/scsi_sysfs.c
@@ -327,8 +327,6 @@ static void scsi_device_dev_release_user
 		sdev->request_queue = NULL;
 	}
 
-	scsi_target_reap(scsi_target(sdev));
-
 	kfree(sdev->inquiry);
 	kfree(sdev);
 
@@ -954,6 +952,7 @@ void __scsi_remove_device(struct scsi_de
 	if (sdev->host->hostt->slave_destroy)
 		sdev->host->hostt->slave_destroy(sdev);
 	transport_destroy_device(dev);
+	scsi_target_reap(scsi_target(sdev));
 	put_device(dev);
 }
 

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[James Bottomley]

On Tue, 2009-05-26 at 15:29 -0400, Alan Stern wrote:
> On Tue, 26 May 2009, Kay Sievers wrote:
> 
> > On Mon, May 25, 2009 at 13:45, Kay Sievers <kay.sievers@vrfy.org> wrote:
> > > On Mon, May 25, 2009 at 04:06, Alan Stern <stern@rowland.harvard.edu> wrote:
> > 
> > >> by the way -- so it's a little difficult to trigger.
> > >
> > > I can trigger it pretty reliable now on plain -rc7 , but only with
> > > more hubs in-between the storage device. It usually take less than
> > > 10-15 connect/disconnect cycles.
> > >
> > > It looks like a serious bug though, after the bug triggered, random,
> > > likely unrelated, applications crash, and I can not cleanly shot down
> > > anymore.
> > 
> > Just a heads up if anybody is trying to reproduce this, it trashed my
> > ext3 rootfs, which is not recoverable.
> > 
> > Not sure what exactly caused this, but I didn't have anything like
> > this for a very long time.
> > 
> > I tried to reproduce the issue a few times more, and it crashed random
> > processes after the bug triggered, like mentioned above, and the box
> > never shut down cleanly.
> > 
> > It's entirely possible, that bug causes serious issues.
> 
> If you don't mind trashing some more ext3 root filesystems :-) you can
> try this patch.  It's almost certainly not quite the right thing to do
> and I have probably messed up the target's reference counting, but
> maybe it's a step in the right direction.
> 
> This strange business of deferring unregistration into a workqueue 
> means that the calls might not be executed in the same order that 
> they're made.
> 
> Alan Stern
> 
> 
> Index: usb-2.6/drivers/scsi/scsi_scan.c
> ===================================================================
> --- usb-2.6.orig/drivers/scsi/scsi_scan.c
> +++ usb-2.6/drivers/scsi/scsi_scan.c
> @@ -956,6 +956,7 @@ static inline void scsi_destroy_sdev(str
>  	if (sdev->host->hostt->slave_destroy)
>  		sdev->host->hostt->slave_destroy(sdev);
>  	transport_destroy_device(&sdev->sdev_gendev);
> +	put_device(sdev->sdev_gendev.parent);
>  	put_device(&sdev->sdev_gendev);
>  }
>  
> Index: usb-2.6/drivers/scsi/scsi_sysfs.c
> ===================================================================
> --- usb-2.6.orig/drivers/scsi/scsi_sysfs.c
> +++ usb-2.6/drivers/scsi/scsi_sysfs.c
> @@ -327,8 +327,6 @@ static void scsi_device_dev_release_user
>  		sdev->request_queue = NULL;
>  	}
>  
> -	scsi_target_reap(scsi_target(sdev));
> -
>  	kfree(sdev->inquiry);
>  	kfree(sdev);
>  
> @@ -954,6 +952,7 @@ void __scsi_remove_device(struct scsi_de
>  	if (sdev->host->hostt->slave_destroy)
>  		sdev->host->hostt->slave_destroy(sdev);
>  	transport_destroy_device(dev);
> +	scsi_target_reap(scsi_target(sdev));
>  	put_device(dev);
>  }

Um, well, you're right, it's wrong.  The reap needs to be matched with
the reap_ref++

It's hard to follow the problem without full context, but if I
understand correctly the problem is you want all the target directories
removed before you call device_del() on the host and the thing that gets
in the way is the necessary user context removal of the host.  So a
simple solution, rather than mucking with the way it works, is to wait
for the workqueues to complete.  Does this fix it?

James

---

diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index c447838..5846c26 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -1877,6 +1877,12 @@ void scsi_forget_host(struct Scsi_Host *shost)
 		goto restart;
 	}
 	spin_unlock_irqrestore(shost->host_lock, flags);
+
+	/*
+	 * the sdev removal goes through a workqueue for user context, so
+	 * make sure it's all complete before we return
+	 */
+	flush_scheduled_work();
 }
 
 /*

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Kay Sievers]

On Tue, May 26, 2009 at 23:09, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Tue, 2009-05-26 at 15:29 -0400, Alan Stern wrote:

>> If you don't mind trashing some more ext3 root filesystems :-) you can
>> try this patch.  It's almost certainly not quite the right thing to do
>> and I have probably messed up the target's reference counting, but
>> maybe it's a step in the right direction.

> It's hard to follow the problem without full context, but if I
> understand correctly the problem is you want all the target directories
> removed before you call device_del() on the host and the thing that gets
> in the way is the necessary user context removal of the host.  So a
> simple solution, rather than mucking with the way it works, is to wait
> for the workqueues to complete.  Does this fix it?

Ok, I copied my newly installed system to another disks, to have a
root filesytem to trash again by this bug. :)

Which of your patches should I try?

Thanks,
Kay
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Tue, 26 May 2009, James Bottomley wrote:

> Um, well, you're right, it's wrong.  The reap needs to be matched with
> the reap_ref++

I was afraid of that.  Not understanding how the reap_ref counts are 
supposed to work makes it hard to get them right...

> It's hard to follow the problem without full context, but if I
> understand correctly the problem is you want all the target directories
> removed before you call device_del() on the host and the thing that gets
> in the way is the necessary user context removal of the host.

Removal of the _target_, not the host.  (Do targets ever get reaped
from non-process context?  I didn't notice any places where that would
happen.)

That's part of the problem.  The other part is that the target isn't
unregistered until all the sdevs have been released, which might not
happen for a long time if the sdevs are pinned for any reason.

That is, the target should be unregistered when all the sdevs are
deleted, in __scsi_remove_device(), not when scsi_device_dev_release()  
or scsi_device_dev_release_usercontext() runs.

>  So a
> simple solution, rather than mucking with the way it works, is to wait
> for the workqueues to complete.  Does this fix it?
> 
> James
> 
> ---
> 
> diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
> index c447838..5846c26 100644
> --- a/drivers/scsi/scsi_scan.c
> +++ b/drivers/scsi/scsi_scan.c
> @@ -1877,6 +1877,12 @@ void scsi_forget_host(struct Scsi_Host *shost)
>  		goto restart;
>  	}
>  	spin_unlock_irqrestore(shost->host_lock, flags);
> +
> +	/*
> +	 * the sdev removal goes through a workqueue for user context, so
> +	 * make sure it's all complete before we return
> +	 */
> +	flush_scheduled_work();
>  }
>  
>  /*

This may well fix Kay's problem, but I think the other point needs to 
be addressed also.

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Tue, 26 May 2009, Kay Sievers wrote:

> Ok, I copied my newly installed system to another disks, to have a
> root filesytem to trash again by this bug. :)
> 
> Which of your patches should I try?

James's patch, not mine.

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Kay Sievers]

On Tue, May 26, 2009 at 23:56, Alan Stern <stern@rowland.harvard.edu> wrote:
> On Tue, 26 May 2009, Kay Sievers wrote:
>
>> Ok, I copied my newly installed system to another disks, to have a
>> root filesytem to trash again by this bug. :)
>>
>> Which of your patches should I try?
>
> James's patch, not mine.

I tried both, both don't fix the issue. With Alan's patch it *seems*
the target device never gets removed, at least I didn't see anything
in the kobject debug logs.

Kay

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[James Bottomley]

On Wed, 2009-05-27 at 00:03 +0200, Kay Sievers wrote:
> On Tue, May 26, 2009 at 23:56, Alan Stern <stern@rowland.harvard.edu> wrote:
> > On Tue, 26 May 2009, Kay Sievers wrote:
> >
> >> Ok, I copied my newly installed system to another disks, to have a
> >> root filesytem to trash again by this bug. :)
> >>
> >> Which of your patches should I try?
> >
> > James's patch, not mine.
> 
> I tried both, both don't fix the issue. With Alan's patch it *seems*
> the target device never gets removed, at least I didn't see anything
> in the kobject debug logs.

OK ... perhaps we have to wait a little harder: try this; it waits until
all the targets have disappeared from visibility via an event.

James

---

diff --git a/drivers/scsi/hosts.c b/drivers/scsi/hosts.c
index 89d41a4..b2946bf 100644
--- a/drivers/scsi/hosts.c
+++ b/drivers/scsi/hosts.c
@@ -173,6 +173,8 @@ void scsi_remove_host(struct Scsi_Host *shost)
 		BUG_ON(scsi_host_set_state(shost, SHOST_DEL_RECOVERY));
 	spin_unlock_irqrestore(shost->host_lock, flags);
 
+	scsi_wait_for_targets_gone(shost);
+
 	transport_unregister_device(&shost->shost_gendev);
 	device_unregister(&shost->shost_dev);
 	device_del(&shost->shost_gendev);
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index c447838..367216c 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -32,6 +32,7 @@
 #include <linux/delay.h>
 #include <linux/kthread.h>
 #include <linux/spinlock.h>
+#include <linux/wait.h>
 #include <linux/async.h>
 
 #include <scsi/scsi.h>
@@ -324,6 +325,12 @@ out:
 	return NULL;
 }
 
+static  DECLARE_WAIT_QUEUE_HEAD(scsi_target_removed);
+void scsi_wait_for_targets_gone(struct Scsi_Host *shost)
+{
+	wait_event(scsi_target_removed, list_empty(&shost->__targets));
+}
+
 static void scsi_target_destroy(struct scsi_target *starget)
 {
 	struct device *dev = &starget->dev;
@@ -336,6 +343,7 @@ static void scsi_target_destroy(struct scsi_target *starget)
 		shost->hostt->target_destroy(starget);
 	list_del_init(&starget->siblings);
 	spin_unlock_irqrestore(shost->host_lock, flags);
+	wake_up(&scsi_target_removed);
 	put_device(dev);
 }
 
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index b62a097..b63a901 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -747,6 +747,7 @@ static inline int scsi_host_scan_allowed(struct Scsi_Host *shost)
 
 extern void scsi_unblock_requests(struct Scsi_Host *);
 extern void scsi_block_requests(struct Scsi_Host *);
+extern void scsi_wait_for_targets_gone(struct Scsi_Host *);
 
 struct class_container;
 

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Kay Sievers]

On Wed, May 27, 2009 at 01:49, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:

> OK ... perhaps we have to wait a little harder: try this; it waits until
> all the targets have disappeared from visibility via an event.

That seems to work fine here.

Thanks,
Kay

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Wed, 27 May 2009, Kay Sievers wrote:

> On Wed, May 27, 2009 at 01:49, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> 
> > OK ... perhaps we have to wait a little harder: try this; it waits until
> > all the targets have disappeared from visibility via an event.
> 
> That seems to work fine here.

It's good for a short-term fix.  For the longer term, I still think 
it's a mistake to wait for the sdevs to be released before deleting the 
target.  It gives user programs the ability to block the host-removal 
thread indefinitely.

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Hannes Reinecke]

Hi all,

Alan Stern wrote:
> On Wed, 27 May 2009, Kay Sievers wrote:
> 
>> On Wed, May 27, 2009 at 01:49, James Bottomley
>> <James.Bottomley@hansenpartnership.com> wrote:
>>
>>> OK ... perhaps we have to wait a little harder: try this; it waits until
>>> all the targets have disappeared from visibility via an event.
>> That seems to work fine here.
> 
> It's good for a short-term fix.  For the longer term, I still think 
> it's a mistake to wait for the sdevs to be released before deleting the 
> target.  It gives user programs the ability to block the host-removal 
> thread indefinitely.
> 
Quite so. We should rather see to have the reference counting fixed
properly, than this would go away automatically.

So I just have to look for someone to fix my iSCSI bugs, than I could
revamp my patchset ...

Sigh.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[James Bottomley]

On Wed, 2009-05-27 at 13:35 +0200, Hannes Reinecke wrote:
> Hi all,
> 
> Alan Stern wrote:
> > On Wed, 27 May 2009, Kay Sievers wrote:
> > 
> >> On Wed, May 27, 2009 at 01:49, James Bottomley
> >> <James.Bottomley@hansenpartnership.com> wrote:
> >>
> >>> OK ... perhaps we have to wait a little harder: try this; it waits until
> >>> all the targets have disappeared from visibility via an event.
> >> That seems to work fine here.
> > 
> > It's good for a short-term fix.  For the longer term, I still think 
> > it's a mistake to wait for the sdevs to be released before deleting the 
> > target.  It gives user programs the ability to block the host-removal 
> > thread indefinitely.
> > 
> Quite so. We should rather see to have the reference counting fixed
> properly, than this would go away automatically.

Hardly ... our current refcounting is on destruction (releases).  This
problem is an instance of visibility (the del calls) we need the
visibility teardown to work nicely.  We currently have no refcounting on
the visibility.  Even if we did (and we could add a ref on when the
underlying device del calls are done), what happens if the target needs
to become visible again.  Apparently the generic device infrastructure
can't accept doing an add on a previously del'd device.

The most obvious way of fixing this is to have a special case for
targets of dying hosts ... they could call del early on the
understanding that they're never getting new underlying devices.  That
would allow the wait to trigger on the last target del, which is what is
optimal.

James

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Wed, 27 May 2009, James Bottomley wrote:

> Hardly ... our current refcounting is on destruction (releases).  This
> problem is an instance of visibility (the del calls) we need the
> visibility teardown to work nicely.  We currently have no refcounting on
> the visibility.  Even if we did (and we could add a ref on when the
> underlying device del calls are done), what happens if the target needs
> to become visible again.  Apparently the generic device infrastructure
> can't accept doing an add on a previously del'd device.

Definitely not.

> The most obvious way of fixing this is to have a special case for
> targets of dying hosts ... they could call del early on the
> understanding that they're never getting new underlying devices.  That
> would allow the wait to trigger on the last target del, which is what is
> optimal.

I don't understand all the subtle issues here.  In other contexts, the 
solution would be to initialize a refcount to 1 when the target is 
allocated, increment it when a device is added, and decrement it when a 
device is removed or the host is removed.  When the refcount goes to 0, 
the target is deleted.  Why wouldn't this kind of approach work?

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[James Bottomley]

On Wed, 2009-05-27 at 12:16 -0400, Alan Stern wrote:
> On Wed, 27 May 2009, James Bottomley wrote:
> 
> > Hardly ... our current refcounting is on destruction (releases).  This
> > problem is an instance of visibility (the del calls) we need the
> > visibility teardown to work nicely.  We currently have no refcounting on
> > the visibility.  Even if we did (and we could add a ref on when the
> > underlying device del calls are done), what happens if the target needs
> > to become visible again.  Apparently the generic device infrastructure
> > can't accept doing an add on a previously del'd device.
> 
> Definitely not.
> 
> > The most obvious way of fixing this is to have a special case for
> > targets of dying hosts ... they could call del early on the
> > understanding that they're never getting new underlying devices.  That
> > would allow the wait to trigger on the last target del, which is what is
> > optimal.
> 
> I don't understand all the subtle issues here.  In other contexts, the 
> solution would be to initialize a refcount to 1 when the target is 
> allocated, increment it when a device is added, and decrement it when a 
> device is removed or the host is removed.  When the refcount goes to 0, 
> the target is deleted.  Why wouldn't this kind of approach work?

Um, well that's exactly how it works (modulo the fact that there are
parts of the lifecycle where the ref count is zero, like scanning).  The
problem you're complaining about is that the device ref on the target
may take a long time to release, so we can't key the del event on the
refcount going to zero, which is what we do today.

James

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Wed, 27 May 2009, James Bottomley wrote:

> > I don't understand all the subtle issues here.  In other contexts, the 
> > solution would be to initialize a refcount to 1 when the target is 
> > allocated, increment it when a device is added, and decrement it when a 
> > device is removed or the host is removed.  When the refcount goes to 0, 
> > the target is deleted.  Why wouldn't this kind of approach work?
> 
> Um, well that's exactly how it works (modulo the fact that there are
> parts of the lifecycle where the ref count is zero, like scanning).

Why does that happen?  It's reasonable that there should be times 
during scanning when the target doesn't have any children, but the 
refcount should still be positive.

>  The
> problem you're complaining about is that the device ref on the target
> may take a long time to release, so we can't key the del event on the
> refcount going to zero, which is what we do today.

Maybe we should be talking about two separate refcounts: a normal 
get_device/put_device kref counter for the target's lifetime, and a 
visibility counter (one for each child device and one overall) which 
keys the del event and must go to 0 before the host removal finishes.

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[James Bottomley]

On Wed, 2009-05-27 at 13:01 -0400, Alan Stern wrote:
> On Wed, 27 May 2009, James Bottomley wrote:
> 
> > > I don't understand all the subtle issues here.  In other contexts, the 
> > > solution would be to initialize a refcount to 1 when the target is 
> > > allocated, increment it when a device is added, and decrement it when a 
> > > device is removed or the host is removed.  When the refcount goes to 0, 
> > > the target is deleted.  Why wouldn't this kind of approach work?
> > 
> > Um, well that's exactly how it works (modulo the fact that there are
> > parts of the lifecycle where the ref count is zero, like scanning).
> 
> Why does that happen?  It's reasonable that there should be times 
> during scanning when the target doesn't have any children, but the 
> refcount should still be positive.

By refcount, I mean count of underlying devices.

> >  The
> > problem you're complaining about is that the device ref on the target
> > may take a long time to release, so we can't key the del event on the
> > refcount going to zero, which is what we do today.
> 
> Maybe we should be talking about two separate refcounts: a normal 
> get_device/put_device kref counter for the target's lifetime, and a 
> visibility counter (one for each child device and one overall) which 
> keys the del event and must go to 0 before the host removal finishes.

Um, well, that's roughly how I said we'd have to fix all of this in the
email to hannes ... it would be much easier if we could make a del'd
device visible, but now we have to have different behaviours depending
on whether the host is going away or not.

James

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Eric W. Biederman]

Alan Stern <stern@rowland.harvard.edu> writes:

> On Wed, 27 May 2009, Kay Sievers wrote:
>
>> On Wed, May 27, 2009 at 01:49, James Bottomley
>> <James.Bottomley@hansenpartnership.com> wrote:
>> 
>> > OK ... perhaps we have to wait a little harder: try this; it waits until
>> > all the targets have disappeared from visibility via an event.
>> 
>> That seems to work fine here.
>
> It's good for a short-term fix.  For the longer term, I still think 
> it's a mistake to wait for the sdevs to be released before deleting the 
> target.  It gives user programs the ability to block the host-removal 
> thread indefinitely.

How can user programs block removal indefinitely today?

Eric

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Wed, 27 May 2009, James Bottomley wrote:

> By refcount, I mean count of underlying devices.

Does that mean only registered devices, or does it include devices 
which are unregistered but not yet released?

> > >  The
> > > problem you're complaining about is that the device ref on the target
> > > may take a long time to release, so we can't key the del event on the
> > > refcount going to zero, which is what we do today.
> > 
> > Maybe we should be talking about two separate refcounts: a normal 
> > get_device/put_device kref counter for the target's lifetime, and a 
> > visibility counter (one for each child device and one overall) which 
> > keys the del event and must go to 0 before the host removal finishes.
> 
> Um, well, that's roughly how I said we'd have to fix all of this in the
> email to hannes ... it would be much easier if we could make a del'd
> device visible,

I don't follow.  Why would you want to delete a target before the host
is removed and then make it visible again later?  Because it doesn't
have any underlying devices at the moment but may gain some later on?

If that's the case, why not delete the target when there are no more
registered devices beneath it and then create a new target structure
when a new device appears?

> but now we have to have different behaviours depending
> on whether the host is going away or not.

Yes, one does get the feeling that we're going around in circles...

Okay.  So now I have made two proposals.  One is to delete targets and
create new ones as needed.  The other is to keep a target hanging
around, even if there are no underlying devices, until the host is
removed.  This can be implemented easily by making the counter
represent the number of registered devices plus one for the host.  

Alan Stern

P.S.: If the counter is made to refer to registered devices, as opposed
to un-released devices, would there ever be a situation where you want
to delete a target in a non-process context?

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Wed, 27 May 2009, Eric W. Biederman wrote:

> Alan Stern <stern@rowland.harvard.edu> writes:
> 
> > On Wed, 27 May 2009, Kay Sievers wrote:
> >
> >> On Wed, May 27, 2009 at 01:49, James Bottomley
> >> <James.Bottomley@hansenpartnership.com> wrote:
> >> 
> >> > OK ... perhaps we have to wait a little harder: try this; it waits until
> >> > all the targets have disappeared from visibility via an event.
> >> 
> >> That seems to work fine here.
> >
> > It's good for a short-term fix.  For the longer term, I still think 
> > it's a mistake to wait for the sdevs to be released before deleting the 
> > target.  It gives user programs the ability to block the host-removal 
> > thread indefinitely.
> 
> How can user programs block removal indefinitely today?

As fas as I know, they can't.  Instead, they can cause the SCSI layer 
to unregister a sysfs directory containing a child directory.  :-)

Basically, a user program can delay removal of the child (i.e., the
target) directory indefinitely, because currently the target isn't
unregistered when all its children are removed -- it's unregistered
when all its children are _released_.

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Eric W. Biederman]

Alan Stern <stern@rowland.harvard.edu> writes:

>
> As fas as I know, they can't.  Instead, they can cause the SCSI layer 
> to unregister a sysfs directory containing a child directory.  :-)
>
> Basically, a user program can delay removal of the child (i.e., the
> target) directory indefinitely, because currently the target isn't
> unregistered when all its children are removed -- it's unregistered
> when all its children are _released_.

Ok.  Is this opens of /dev/sda1 and the like that are being held open by
userspace that are potentially causing problems?

I think I have the fix to that...

Eric

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[James Bottomley]

On Wed, 2009-05-27 at 14:07 -0400, Alan Stern wrote:
> On Wed, 27 May 2009, James Bottomley wrote:
> 
> > By refcount, I mean count of underlying devices.
> 
> Does that mean only registered devices, or does it include devices 
> which are unregistered but not yet released?

All devices ... scsi_device has to has a target parent before its
usable.

> > > >  The
> > > > problem you're complaining about is that the device ref on the target
> > > > may take a long time to release, so we can't key the del event on the
> > > > refcount going to zero, which is what we do today.
> > > 
> > > Maybe we should be talking about two separate refcounts: a normal 
> > > get_device/put_device kref counter for the target's lifetime, and a 
> > > visibility counter (one for each child device and one overall) which 
> > > keys the del event and must go to 0 before the host removal finishes.
> > 
> > Um, well, that's roughly how I said we'd have to fix all of this in the
> > email to hannes ... it would be much easier if we could make a del'd
> > device visible,
> 
> I don't follow.  Why would you want to delete a target before the host
> is removed and then make it visible again later?  Because it doesn't
> have any underlying devices at the moment but may gain some later on?

Perhaps I haven't made the problem clear enough.  You only want early
del if the host is going away, otherwise the target might be reused and
it can't be if you've called del on it.  So there needs to be an
integration into the host lifecycle in some form.

James

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Wed, 27 May 2009, James Bottomley wrote:

> On Wed, 2009-05-27 at 14:07 -0400, Alan Stern wrote:
> > On Wed, 27 May 2009, James Bottomley wrote:
> > 
> > > By refcount, I mean count of underlying devices.
> > 
> > Does that mean only registered devices, or does it include devices 
> > which are unregistered but not yet released?
> 
> All devices ... scsi_device has to has a target parent before its
> usable.

I can't tell whether you understood my point.  After a scsi_device is
unregistered but before it is released -- i.e., when its state is
SDEV_DEL -- it _is_ essentially unusable.  So why wait until it is
released to decrement the target's device counter?  Why not do the
decrement in __scsi_remove_device()?


> > > Um, well, that's roughly how I said we'd have to fix all of this in the
> > > email to hannes ... it would be much easier if we could make a del'd
> > > device visible,
> > 
> > I don't follow.  Why would you want to delete a target before the host
> > is removed and then make it visible again later?  Because it doesn't
> > have any underlying devices at the moment but may gain some later on?
> 
> Perhaps I haven't made the problem clear enough.  You only want early
> del if the host is going away, otherwise the target might be reused and
> it can't be if you've called del on it.  So there needs to be an
> integration into the host lifecycle in some form.

Yes, granted.  That integration doesn't have to be complicated.  
Basically, you just decrement the counters in all the targets when
setting a host's state to SHOST_DEL or SHOST_DEL_RECOVERY.  At that 
point there's no reason to keep an unpopulated target around, right?

Up until that point, the counter's value should be one more than the
number of underlying sdevs.  So if the counter decrements to 0 then
there were no underlying sdevs and the target is deleted immediately;
otherwise it is deleted when the last remaining sdev is deleted.

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[James Bottomley]

On Wed, 2009-05-27 at 16:40 -0400, Alan Stern wrote:
> On Wed, 27 May 2009, James Bottomley wrote:
> 
> > On Wed, 2009-05-27 at 14:07 -0400, Alan Stern wrote:
> > > On Wed, 27 May 2009, James Bottomley wrote:
> > > 
> > > > By refcount, I mean count of underlying devices.
> > > 
> > > Does that mean only registered devices, or does it include devices 
> > > which are unregistered but not yet released?
> > 
> > All devices ... scsi_device has to has a target parent before its
> > usable.
> 
> I can't tell whether you understood my point.  After a scsi_device is
> unregistered but before it is released -- i.e., when its state is
> SDEV_DEL -- it _is_ essentially unusable.  So why wait until it is
> released to decrement the target's device counter?  Why not do the
> decrement in __scsi_remove_device()?

because the use model of the device still requires a valid target.  Even
though it gets gated in several places in SDEV_DEL, we still have use of
the target parent.  This is fixable, but only by a long audit of all the
sdev uses plus the enforcement of no use of target in DEL state rule,
which adds complexity.

> > > > Um, well, that's roughly how I said we'd have to fix all of this in the
> > > > email to hannes ... it would be much easier if we could make a del'd
> > > > device visible,
> > > 
> > > I don't follow.  Why would you want to delete a target before the host
> > > is removed and then make it visible again later?  Because it doesn't
> > > have any underlying devices at the moment but may gain some later on?
> > 
> > Perhaps I haven't made the problem clear enough.  You only want early
> > del if the host is going away, otherwise the target might be reused and
> > it can't be if you've called del on it.  So there needs to be an
> > integration into the host lifecycle in some form.
> 
> Yes, granted.  That integration doesn't have to be complicated.  
> Basically, you just decrement the counters in all the targets when
> setting a host's state to SHOST_DEL or SHOST_DEL_RECOVERY.  At that 

And SHOST_CANCEL and SHOST_CANCEL_RECOVERY.

> point there's no reason to keep an unpopulated target around, right?

If the child list were empty, sure.  However, it's likely not going to
be at this point.

> Up until that point, the counter's value should be one more than the
> number of underlying sdevs.  So if the counter decrements to 0 then
> there were no underlying sdevs and the target is deleted immediately;
> otherwise it is deleted when the last remaining sdev is deleted.

No, that's the problem.  It can be removed from visibility if it has no
visible sdevs, but it can't be deleted until the last sdev is released.

James

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Wed, 27 May 2009, James Bottomley wrote:

> > I can't tell whether you understood my point.  After a scsi_device is
> > unregistered but before it is released -- i.e., when its state is
> > SDEV_DEL -- it _is_ essentially unusable.  So why wait until it is
> > released to decrement the target's device counter?  Why not do the
> > decrement in __scsi_remove_device()?
> 
> because the use model of the device still requires a valid target.  Even
> though it gets gated in several places in SDEV_DEL, we still have use of
> the target parent.  This is fixable, but only by a long audit of all the
> sdev uses plus the enforcement of no use of target in DEL state rule,
> which adds complexity.

You're failing to distinguish properly between "delete" and "release".  
A target (or device in general) is deleted when it is removed from
visibility -- i.e., when device_del() is called.  It is released when
the final put_device() call occurs and the data structure is
deallocated.

So, all I'm saying is there's nothing wrong with deleting a target
when all its children are deleted, provided the target isn't released
until all the children are released.  Below you say the same thing.


> > > Perhaps I haven't made the problem clear enough.  You only want early
> > > del if the host is going away, otherwise the target might be reused and
> > > it can't be if you've called del on it.  So there needs to be an
> > > integration into the host lifecycle in some form.
> > 
> > Yes, granted.  That integration doesn't have to be complicated.  
> > Basically, you just decrement the counters in all the targets when
> > setting a host's state to SHOST_DEL or SHOST_DEL_RECOVERY.  At that 
> 
> And SHOST_CANCEL and SHOST_CANCEL_RECOVERY.

If you prefer.  I thought SHOST_DEL would be more appropriate because
it occurs after scsi_forget_host() is called.  All those transitions
occur in scsi_remove_host(), anyway.

> > point there's no reason to keep an unpopulated target around, right?
> 
> If the child list were empty, sure.  However, it's likely not going to
> be at this point.

Regardless, it will work either way.

> > Up until that point, the counter's value should be one more than the
> > number of underlying sdevs.  So if the counter decrements to 0 then
> > there were no underlying sdevs and the target is deleted immediately;
> > otherwise it is deleted when the last remaining sdev is deleted.
> 
> No, that's the problem.  It can be removed from visibility if it has no
> visible sdevs, but it can't be deleted until the last sdev is released.

Allow me to rephrase this: A target can be removed from visibility if 
it has no visible sdevs, but it can't be _released_ until the last sdev 
is released.

That's fine.  You remove a target from visibility when target->reap_ref
becomes 0.  The target isn't released until the target's embedded
struct device's refcount becomes 0.  To make this work, simply have
scsi_alloc_sdev() call

	get_device(&starget->dev);

and have scsi_device_dev_release_usercontext() call

	put_device(&starget->dev);

Doesn't that do exactly what you're asking for?

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Wed, 27 May 2009, Eric W. Biederman wrote:

> Alan Stern <stern@rowland.harvard.edu> writes:
> 
> >
> > As fas as I know, they can't.  Instead, they can cause the SCSI layer 
> > to unregister a sysfs directory containing a child directory.  :-)
> >
> > Basically, a user program can delay removal of the child (i.e., the
> > target) directory indefinitely, because currently the target isn't
> > unregistered when all its children are removed -- it's unregistered
> > when all its children are _released_.
> 
> Ok.  Is this opens of /dev/sda1 and the like that are being held open by
> userspace that are potentially causing problems?

Yes, plus any other mechanism for preventing a struct device's refcount 
from going to 0.

> I think I have the fix to that...

The fix is to delete the target when its children are deleted, and not
wait until the children are released.

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[James Bottomley]

On Wed, 2009-05-27 at 17:31 -0400, Alan Stern wrote:
> On Wed, 27 May 2009, James Bottomley wrote:
> 
> > > I can't tell whether you understood my point.  After a scsi_device is
> > > unregistered but before it is released -- i.e., when its state is
> > > SDEV_DEL -- it _is_ essentially unusable.  So why wait until it is
> > > released to decrement the target's device counter?  Why not do the
> > > decrement in __scsi_remove_device()?
> > 
> > because the use model of the device still requires a valid target.  Even
> > though it gets gated in several places in SDEV_DEL, we still have use of
> > the target parent.  This is fixable, but only by a long audit of all the
> > sdev uses plus the enforcement of no use of target in DEL state rule,
> > which adds complexity.
> 
> You're failing to distinguish properly between "delete" and "release".  
> A target (or device in general) is deleted when it is removed from
> visibility -- i.e., when device_del() is called.  It is released when
> the final put_device() call occurs and the data structure is
> deallocated.

I find the terms delete and release too close for comfort, which is why
I've always been careful to say remove from visibility.

> So, all I'm saying is there's nothing wrong with deleting a target
> when all its children are deleted, provided the target isn't released
> until all the children are released.  Below you say the same thing.
> 
> 
> > > > Perhaps I haven't made the problem clear enough.  You only want early
> > > > del if the host is going away, otherwise the target might be reused and
> > > > it can't be if you've called del on it.  So there needs to be an
> > > > integration into the host lifecycle in some form.
> > > 
> > > Yes, granted.  That integration doesn't have to be complicated.  
> > > Basically, you just decrement the counters in all the targets when
> > > setting a host's state to SHOST_DEL or SHOST_DEL_RECOVERY.  At that 
> > 
> > And SHOST_CANCEL and SHOST_CANCEL_RECOVERY.
> 
> If you prefer.  I thought SHOST_DEL would be more appropriate because
> it occurs after scsi_forget_host() is called.  All those transitions
> occur in scsi_remove_host(), anyway.

I mean in all four states.

> > > point there's no reason to keep an unpopulated target around, right?
> > 
> > If the child list were empty, sure.  However, it's likely not going to
> > be at this point.
> 
> Regardless, it will work either way.
> 
> > > Up until that point, the counter's value should be one more than the
> > > number of underlying sdevs.  So if the counter decrements to 0 then
> > > there were no underlying sdevs and the target is deleted immediately;
> > > otherwise it is deleted when the last remaining sdev is deleted.
> > 
> > No, that's the problem.  It can be removed from visibility if it has no
> > visible sdevs, but it can't be deleted until the last sdev is released.
> 
> Allow me to rephrase this: A target can be removed from visibility if 
> it has no visible sdevs, but it can't be _released_ until the last sdev 
> is released.
> 
> That's fine.  You remove a target from visibility when target->reap_ref
> becomes 0.  The target isn't released until the target's embedded
> struct device's refcount becomes 0.  To make this work, simply have
> scsi_alloc_sdev() call
> 
> 	get_device(&starget->dev);
> 
> and have scsi_device_dev_release_usercontext() call
> 
> 	put_device(&starget->dev);
> 
> Doesn't that do exactly what you're asking for?

That's um what we do to day ... the addition has to be to the visibility
management.

James

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Eric W. Biederman]

Alan Stern <stern@rowland.harvard.edu> writes:

> On Wed, 27 May 2009, Eric W. Biederman wrote:
>
>> Alan Stern <stern@rowland.harvard.edu> writes:
>> 
>> >
>> > As fas as I know, they can't.  Instead, they can cause the SCSI layer 
>> > to unregister a sysfs directory containing a child directory.  :-)
>> >
>> > Basically, a user program can delay removal of the child (i.e., the
>> > target) directory indefinitely, because currently the target isn't
>> > unregistered when all its children are removed -- it's unregistered
>> > when all its children are _released_.
>> 
>> Ok.  Is this opens of /dev/sda1 and the like that are being held open by
>> userspace that are potentially causing problems?
>
> Yes, plus any other mechanism for preventing a struct device's refcount 
> from going to 0.

Thanks.  The discussion makes sense now.

>> I think I have the fix to that...
>
> The fix is to delete the target when its children are deleted, and not
> wait until the children are released.

I think I can do both.

I am currently working on a patchset which at the VFS layer disconnects a
fd from an underlying device.  It does the necessary use count tracking
and when the disconnect is done it returns.  As part of the disconnect it
calls the release method.

We already do this in at least sysfs, proc, sysctl, and sound.  So I
figure it is time to move this into some generic code so we don't need
to duplicate the bugs and the insanities.

Once merged it would take just a few lines of code to use this functionality.

Eric

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Wed, 27 May 2009, James Bottomley wrote:

> I find the terms delete and release too close for comfort, which is why
> I've always been careful to say remove from visibility.

Okay; I'll use your terms when conversing with you.  :-)

> > That's fine.  You remove a target from visibility when target->reap_ref
> > becomes 0.  The target isn't released until the target's embedded
> > struct device's refcount becomes 0.  To make this work, simply have
> > scsi_alloc_sdev() call
> > 
> > 	get_device(&starget->dev);
> > 
> > and have scsi_device_dev_release_usercontext() call
> > 
> > 	put_device(&starget->dev);
> > 
> > Doesn't that do exactly what you're asking for?
> 
> That's um what we do to day ... the addition has to be to the visibility
> management.

That's what I was trying to accomplish in the patch you said was wrong.  
It moved the call to scsi_target_reap() from
scsi_device_dev_release_usercontext() into __scsi_remove_device().  
That is, the target's count of underlying sdevs was to be decremented
whenever an sdev was removed from visibility, not when the sdev was
released.

That's how the problem should be solved.  But the details need to be 
correct, and I don't understand how they all work (as you noticed when 
reading the patch).

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Wed, 27 May 2009, Eric W. Biederman wrote:

> >> I think I have the fix to that...
> >
> > The fix is to delete the target when its children are deleted, and not
> > wait until the children are released.
> 
> I think I can do both.
> 
> I am currently working on a patchset which at the VFS layer disconnects a
> fd from an underlying device.  It does the necessary use count tracking
> and when the disconnect is done it returns.  As part of the disconnect it
> calls the release method.
> 
> We already do this in at least sysfs, proc, sysctl, and sound.  So I
> figure it is time to move this into some generic code so we don't need
> to duplicate the bugs and the insanities.
> 
> Once merged it would take just a few lines of code to use this functionality.

That's good, but it isn't enough to solve this problem.  Even though
user programs might not be able to pin the device by holding the file
open any more, other mechanisms (internal to the kernel) could have the
same effect.  Only a short delay is needed.

The underlying cause should be fixed properly.

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[James Bottomley]

On Wed, 2009-05-27 at 18:15 -0400, Alan Stern wrote:
> On Wed, 27 May 2009, James Bottomley wrote:
> 
> > I find the terms delete and release too close for comfort, which is why
> > I've always been careful to say remove from visibility.
> 
> Okay; I'll use your terms when conversing with you.  :-)
> 
> > > That's fine.  You remove a target from visibility when target->reap_ref
> > > becomes 0.  The target isn't released until the target's embedded
> > > struct device's refcount becomes 0.  To make this work, simply have
> > > scsi_alloc_sdev() call
> > > 
> > > 	get_device(&starget->dev);
> > > 
> > > and have scsi_device_dev_release_usercontext() call
> > > 
> > > 	put_device(&starget->dev);
> > > 
> > > Doesn't that do exactly what you're asking for?
> > 
> > That's um what we do to day ... the addition has to be to the visibility
> > management.
> 
> That's what I was trying to accomplish in the patch you said was wrong.  
> It moved the call to scsi_target_reap() from
> scsi_device_dev_release_usercontext() into __scsi_remove_device().  
> That is, the target's count of underlying sdevs was to be decremented
> whenever an sdev was removed from visibility, not when the sdev was
> released.
> 
> That's how the problem should be solved.  But the details need to be 
> correct, and I don't understand how they all work (as you noticed when 
> reading the patch).

Right, and I think reap_ref can be seconded to count underlying device
visibility.  However, the piece that's missing, is the fact that all of
this has to be tied into the host state.  If the host is running, you
can't remove the target from visibility even if all its children are
invisible because it might get another visible child added. once it goes
into the cancel or del states, it can't acquire new children, so then
it's safe to make a target with no visible children invisible.

James

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Wed, 27 May 2009, James Bottomley wrote:

> Right, and I think reap_ref can be seconded to count underlying device
> visibility.

Exactly.  It should count the number of underlying devices that have 
not yet been removed from visibility (this may include some which still 
have to become visible), plus one if we want to keep the target hanging 
around for a while with no visible children (while scanning it, for 
example).

>  However, the piece that's missing, is the fact that all of
> this has to be tied into the host state.  If the host is running, you
> can't remove the target from visibility even if all its children are
> invisible because it might get another visible child added.

Are you sure about that?  It's not obvious at all to me.

For example, suppose during scanning it turns out there are no LUNs at
a particular target address.  Why should the empty target be retained?  
You'd end up with unusable targets at all possible bus addresses.

Besides, if a target is removed from visibility and then another child
is added, the answer is simply to create a new target structure.  
There's already code in scsi_alloc_target() to do this.

>  once it goes
> into the cancel or del states, it can't acquire new children, so then
> it's safe to make a target with no visible children invisible.

If you grant my point above, targets don't need to be tied into the
host state.  They can be removed from visibility whenever the reap_ref
counter goes to 0.  This will happen naturally while the host is in 
the CANCEL state, thanks to scsi_forget_host().

There's another point to consider.  If you do accept my argument that 
empty targets can be removed from visibility regardless of the host's 
state, then this removal races with addition of a new child.  Since 
removal involves calling device_del(), it can't be protected by the 
host lock.  Instead we'd have to use a mutex to protect both target 
addition and target removal.

Since the host's scan_mutex already protects target addition, extending 
its scope to encompass target removal (and perhaps sdev removal too) 
seems natural.  Do you agree?

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Eric W. Biederman]

Alan Stern <stern@rowland.harvard.edu> writes:

> There's another point to consider.  If you do accept my argument that 
> empty targets can be removed from visibility regardless of the host's 
> state, then this removal races with addition of a new child.  Since 
> removal involves calling device_del(), it can't be protected by the 
> host lock.  Instead we'd have to use a mutex to protect both target 
> addition and target removal.

Careful.  Holding a lock over device_del is an easy and hidden way
to trigger a rare deadlocks.

Eric

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Thu, 28 May 2009, Eric W. Biederman wrote:

> Alan Stern <stern@rowland.harvard.edu> writes:
> 
> > There's another point to consider.  If you do accept my argument that 
> > empty targets can be removed from visibility regardless of the host's 
> > state, then this removal races with addition of a new child.  Since 
> > removal involves calling device_del(), it can't be protected by the 
> > host lock.  Instead we'd have to use a mutex to protect both target 
> > addition and target removal.
> 
> Careful.  Holding a lock over device_del is an easy and hidden way
> to trigger a rare deadlocks.

Your point is well taken.  In addition, I don't really like the idea of 
forcing device removal to wait for some other device to be added.

I'll work around the problem somehow...  A short polling loop shoud do 
the job.

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[James Bottomley]

On Thu, 2009-05-28 at 11:24 -0400, Alan Stern wrote:
> On Wed, 27 May 2009, James Bottomley wrote:
> 
> > Right, and I think reap_ref can be seconded to count underlying device
> > visibility.
> 
> Exactly.  It should count the number of underlying devices that have 
> not yet been removed from visibility (this may include some which still 
> have to become visible), plus one if we want to keep the target hanging 
> around for a while with no visible children (while scanning it, for 
> example).
> 
> >  However, the piece that's missing, is the fact that all of
> > this has to be tied into the host state.  If the host is running, you
> > can't remove the target from visibility even if all its children are
> > invisible because it might get another visible child added.
> 
> Are you sure about that?  It's not obvious at all to me.

Yes ... otherwise you have to elongate the DEL interval from a few ms to
potentially anything.  That would allow locking a target in a dying
state and prevent any new LUNs being added.

> For example, suppose during scanning it turns out there are no LUNs at
> a particular target address.  Why should the empty target be retained?  
> You'd end up with unusable targets at all possible bus addresses.
> 
> Besides, if a target is removed from visibility and then another child
> is added, the answer is simply to create a new target structure.  
> There's already code in scsi_alloc_target() to do this.

As I've said several times, this could be done, but we'd have to audit
the code paths to make sure we allow for multiple same targets in the
list.

> >  once it goes
> > into the cancel or del states, it can't acquire new children, so then
> > it's safe to make a target with no visible children invisible.
> 
> If you grant my point above, targets don't need to be tied into the
> host state.  They can be removed from visibility whenever the reap_ref
> counter goes to 0.  This will happen naturally while the host is in 
> the CANCEL state, thanks to scsi_forget_host().
> 
> There's another point to consider.  If you do accept my argument that 
> empty targets can be removed from visibility regardless of the host's 
> state, then this removal races with addition of a new child.  Since 
> removal involves calling device_del(), it can't be protected by the 
> host lock.  Instead we'd have to use a mutex to protect both target 
> addition and target removal.

No, this is state model 101 ... you alter the state inside the lock and
call del outside of it.  Technically you're lying about the state for
the few us it takes to run out of the lock and del the target, but
there's a papal indulgence for that.

> Since the host's scan_mutex already protects target addition, extending 
> its scope to encompass target removal (and perhaps sdev removal too) 
> seems natural.  Do you agree?

James

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Thu, 28 May 2009, James Bottomley wrote:

> > >  However, the piece that's missing, is the fact that all of
> > > this has to be tied into the host state.  If the host is running, you
> > > can't remove the target from visibility even if all its children are
> > > invisible because it might get another visible child added.
> > 
> > Are you sure about that?  It's not obvious at all to me.
> 
> Yes ... otherwise you have to elongate the DEL interval from a few ms to
> potentially anything.  That would allow locking a target in a dying
> state and prevent any new LUNs being added.

How so?  Why not unlink the target from the host's list when the 
device_del() call returns?  A new target can be created any time after 
that, since the old one is now completely invisible.

> > For example, suppose during scanning it turns out there are no LUNs at
> > a particular target address.  Why should the empty target be retained?  
> > You'd end up with unusable targets at all possible bus addresses.
> > 
> > Besides, if a target is removed from visibility and then another child
> > is added, the answer is simply to create a new target structure.  
> > There's already code in scsi_alloc_target() to do this.
> 
> As I've said several times, this could be done, but we'd have to audit
> the code paths to make sure we allow for multiple same targets in the
> list.

No, not if the old target is removed from the host's list before the
new target is added.

Is there any reason the old target has to remain on the list?  If 
there is, we can introduce a new state: STARGET_CLEANUP.  The old 
target gets put into this state when device_del() returns.  List 
entries in that state are ignored by __scsi_find_target() or whatever 
else looks through the list.

Alan Stern

P.S.: Does scsi_target_reap() really ever get called in non-process
context?  I couldn't find any place where that might happen.

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[James Bottomley]

On Thu, 2009-05-28 at 16:02 -0400, Alan Stern wrote:
> On Thu, 28 May 2009, James Bottomley wrote:
> 
> > > >  However, the piece that's missing, is the fact that all of
> > > > this has to be tied into the host state.  If the host is running, you
> > > > can't remove the target from visibility even if all its children are
> > > > invisible because it might get another visible child added.
> > > 
> > > Are you sure about that?  It's not obvious at all to me.
> > 
> > Yes ... otherwise you have to elongate the DEL interval from a few ms to
> > potentially anything.  That would allow locking a target in a dying
> > state and prevent any new LUNs being added.
> 
> How so?  Why not unlink the target from the host's list when the 
> device_del() call returns?  A new target can be created any time after 
> that, since the old one is now completely invisible.

The answer to that one is several emails back: we need the target in the
host list for the lifetime of the devices ... it's alterable, but even
more auditing.

> > > For example, suppose during scanning it turns out there are no LUNs at
> > > a particular target address.  Why should the empty target be retained?  
> > > You'd end up with unusable targets at all possible bus addresses.
> > > 
> > > Besides, if a target is removed from visibility and then another child
> > > is added, the answer is simply to create a new target structure.  
> > > There's already code in scsi_alloc_target() to do this.
> > 
> > As I've said several times, this could be done, but we'd have to audit
> > the code paths to make sure we allow for multiple same targets in the
> > list.
> 
> No, not if the old target is removed from the host's list before the
> new target is added.
> 
> Is there any reason the old target has to remain on the list?  If 
> there is, we can introduce a new state: STARGET_CLEANUP.  The old 
> target gets put into this state when device_del() returns.  List 
> entries in that state are ignored by __scsi_find_target() or whatever 
> else looks through the list.
> 
> Alan Stern
> 
> P.S.: Does scsi_target_reap() really ever get called in non-process
> context?  I couldn't find any place where that might happen.

>From the device release, which is done by last put, which could be I/O
context.

James

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

On Thu, 28 May 2009, James Bottomley wrote:

> > How so?  Why not unlink the target from the host's list when the 
> > device_del() call returns?  A new target can be created any time after 
> > that, since the old one is now completely invisible.
> 
> The answer to that one is several emails back: we need the target in the
> host list for the lifetime of the devices ... it's alterable, but even
> more auditing.

I don't recall you mentioning that the target had to be linked into the 
host's list for the lifetime of the devices; I thought you said merely 
that the target had to _exist_ for the lifetime of the devices.

Does it really need to be linked, or is existence of the structure 
sufficient?

Likewise, after a device is removed from visibility, does it need to 
remain linked into the host's and target's lists?

> > P.S.: Does scsi_target_reap() really ever get called in non-process
> > context?  I couldn't find any place where that might happen.
> 
> From the device release, which is done by last put, which could be I/O
> context.

But scsi_target_reap() isn't called directly from the device release.  
It's called from scsi_device_dev_release_usercontext().

And besides, in the patch I'm working on it isn't called from either of 
those places -- it's called from __scsi_remove_device().  So I'll go 
ahead and get rid of scsi_target_reap_usercontext().

Alan Stern

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Hannes Reinecke]

[-- Attachment #1: Type: text/plain, Size: 844 bytes --]

Hi Alan,

> And besides, in the patch I'm working on it isn't called from either of 
> those places -- it's called from __scsi_remove_device().  So I'll go 
> ahead and get rid of scsi_target_reap_usercontext().
> 
And just for reference, here is my patchset I've created sometime ago
which streamlines the sdev and starget lifetime. I think I've tried
to send it upstream at one point but never got far with it.
Be aware that it's relative to a rather git tree (2.6.22?) so
it might not apply properly. But it's mainly to get you an idea
of what I've done so far.

And would have continued pushing it if real life hadn't interfered ...

HTH.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)

[-- Attachment #2: 0001-Fix-refcounting-for-attribute_container.patch --]
[-- Type: text/x-patch, Size: 1621 bytes --]

>From 404fc549f858cd0cf3a84865442fe85fedb920de Mon Sep 17 00:00:00 2001
From: Hannes Reinecke <hare@acerbis.suse.de>
Date: Sat, 8 Mar 2008 10:30:58 +0100
Subject: [PATCH] Fix refcounting for attribute_container

attribute_container_add_device() took an explicit reference on the
parent device, making it impossible to remove the parent by doing
a simple put. So we'd rather _not_ take a reference here as
attribute_container will be handled explicitly by calls to
attribute_container_remove_device()/_destroy_device() anyway.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/base/attribute_container.c |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/base/attribute_container.c b/drivers/base/attribute_container.c
index f57652d..6c7e633 100644
--- a/drivers/base/attribute_container.c
+++ b/drivers/base/attribute_container.c
@@ -114,10 +114,8 @@ static void attribute_container_release(struct device *classdev)
 {
 	struct internal_container *ic 
 		= container_of(classdev, struct internal_container, classdev);
-	struct device *dev = classdev->parent;
 
 	kfree(ic);
-	put_device(dev);
 }
 
 /**
@@ -164,7 +162,11 @@ attribute_container_add_device(struct device *dev,
 
 		ic->cont = cont;
 		device_initialize(&ic->classdev);
-		ic->classdev.parent = get_device(dev);
+		/*
+		 * Don't increase refcount here, device will be
+		 * removed explicitly by a call to _destroy().
+		 */
+		ic->classdev.parent = dev;
 		ic->classdev.class = cont->class;
 		cont->class->dev_release = attribute_container_release;
 		strcpy(ic->classdev.bus_id, dev->bus_id);
-- 
1.5.3.2


[-- Attachment #3: 0002-Remove-reap_ref.patch --]
[-- Type: text/x-patch, Size: 3927 bytes --]

>From 9e96c5dd094d3822093656e87b71cd433e818cd2 Mon Sep 17 00:00:00 2001
From: Hannes Reinecke <hare@acerbis.suse.de>
Date: Sat, 8 Mar 2008 12:28:17 +0100
Subject: [PATCH] Remove reap_ref

struct scsi_target contains a 'reap_ref' counter, which is
basically a reference counter for the target.
As we now have proper reference counting we can remove this
and clear out the calling sequence for scsi_target_reap().

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/scsi_scan.c   |   22 ++++++++++++++--------
 drivers/scsi/scsi_sysfs.c  |    3 ---
 include/scsi/scsi_device.h |    1 -
 3 files changed, 14 insertions(+), 12 deletions(-)

diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index d61a8e8..2feab2a 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -448,7 +448,6 @@ static struct scsi_target *scsi_alloc_target(struct device *parent,
 	return starget;
 
  found:
-	found_target->reap_ref++;
 	spin_unlock_irqrestore(shost->host_lock, flags);
 	if (found_target->state != STARGET_DEL) {
 		put_device(parent);
@@ -505,7 +504,7 @@ void scsi_target_reap(struct scsi_target *starget)
 
 	spin_lock_irqsave(shost->host_lock, flags);
 
-	if (--starget->reap_ref == 0 && list_empty(&starget->devices)) {
+	if (list_empty(&starget->devices)) {
 		if (starget->state == STARGET_CREATED) {
 			spin_unlock_irqrestore(shost->host_lock, flags);
 			starget->state = STARGET_DEL;
@@ -1516,8 +1515,13 @@ struct scsi_device *__scsi_add_device(struct Scsi_Host *shost, uint channel,
 	if (scsi_host_scan_allowed(shost))
 		scsi_probe_and_add_lun(starget, lun, NULL, &sdev, 1, hostdata);
 	mutex_unlock(&shost->scan_mutex);
-	transport_configure_device(&starget->dev);
-	scsi_target_reap(starget);
+	/*
+	 * scsi_target_reap is called from the release function
+	 * of each sdev.
+	 */
+	if (starget->state != STARGET_DEL)
+		transport_configure_device(&starget->dev);
+
 	put_device(&starget->dev);
 
 	return sdev;
@@ -1595,10 +1599,12 @@ static void __scsi_scan_target(struct device *parent, unsigned int channel,
 	}
 
  out_reap:
-	/* now determine if the target has any children at all
-	 * and if not, nuke it */
-	transport_configure_device(&starget->dev);
-	scsi_target_reap(starget);
+	/*
+	 * scsi_target_reap is called from the release function
+	 * of each sdev.
+	 */
+	if (starget->state != STARGET_DEL)
+		transport_configure_device(&starget->dev);
 
 	put_device(&starget->dev);
 }
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index bd49d4e..4db0fed 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -297,7 +297,6 @@ static void scsi_device_dev_release_usercontext(struct work_struct *work)
 	starget = to_scsi_target(parent);
 
 	spin_lock_irqsave(sdev->host->host_lock, flags);
-	starget->reap_ref++;
 	list_del(&sdev->siblings);
 	list_del(&sdev->same_target_siblings);
 	list_del(&sdev->starved_entry);
@@ -937,7 +936,6 @@ static void __scsi_remove_target(struct scsi_target *starget)
 	struct scsi_device *sdev;
 
 	spin_lock_irqsave(shost->host_lock, flags);
-	starget->reap_ref++;
  restart:
 	list_for_each_entry(sdev, &shost->__devices, siblings) {
 		if (sdev->channel != starget->channel ||
@@ -950,7 +948,6 @@ static void __scsi_remove_target(struct scsi_target *starget)
 		goto restart;
 	}
 	spin_unlock_irqrestore(shost->host_lock, flags);
-	scsi_target_reap(starget);
 }
 
 static int __remove_child (struct device * dev, void * data)
diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
index f6a9fe0..ccc437b 100644
--- a/include/scsi/scsi_device.h
+++ b/include/scsi/scsi_device.h
@@ -196,7 +196,6 @@ struct scsi_target {
 	struct list_head	siblings;
 	struct list_head	devices;
 	struct device		dev;
-	unsigned int		reap_ref; /* protected by the host lock */
 	unsigned int		channel;
 	unsigned int		id; /* target id ... replace
 				     * scsi_device.id eventually */
-- 
1.5.3.2


[-- Attachment #4: 0003-Improve-error-messages-in-scsi_sysfs_add_sdev.patch --]
[-- Type: text/x-patch, Size: 1185 bytes --]

>From b15634110e53a8418378c952bd1b6488f2746f86 Mon Sep 17 00:00:00 2001
From: Hannes Reinecke <hare@acerbis.suse.de>
Date: Mon, 25 Feb 2008 03:39:28 +0100
Subject: [PATCH] Improve error messages in scsi_sysfs_add_sdev()

When we fail to add a device to the driver core, only the very
helpful message 'error X' is displayed.
Print out some more meaningful messages.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/scsi_sysfs.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 4db0fed..8f674ac 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -830,12 +830,14 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
 	error = device_add(&sdev->sdev_gendev);
 	if (error) {
 		put_device(sdev->sdev_gendev.parent);
-		printk(KERN_INFO "error 1\n");
+		sdev_printk(KERN_INFO, sdev,
+			    "failed to add device: %d\n", error);
 		return error;
 	}
 	error = device_add(&sdev->sdev_dev);
 	if (error) {
-		printk(KERN_INFO "error 2\n");
+		sdev_printk(KERN_INFO, sdev,
+			    "failed to add class device: %d\n", error);
 		goto clean_device;
 	}
 
-- 
1.5.3.2


[-- Attachment #5: 0004-Remove-stale-put_device-from-scsi_sysfs_add_sdev.patch --]
[-- Type: text/x-patch, Size: 972 bytes --]

>From f1717cf66290b81f9b376ddeba65426c91fb7fe4 Mon Sep 17 00:00:00 2001
From: Hannes Reinecke <hare@acerbis.suse.de>
Date: Sat, 8 Mar 2008 18:01:40 +0100
Subject: [PATCH] Remove stale put_device() from scsi_sysfs_add_sdev()

In one obscure error path someone decided to do a put_device()
on the sdev parent.
This doesn't make much sense as we didn't take the reference
previously. So remove it.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/scsi_sysfs.c |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 8f674ac..7dc3015 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -829,7 +829,6 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
 
 	error = device_add(&sdev->sdev_gendev);
 	if (error) {
-		put_device(sdev->sdev_gendev.parent);
 		sdev_printk(KERN_INFO, sdev,
 			    "failed to add device: %d\n", error);
 		return error;
-- 
1.5.3.2


[-- Attachment #6: 0005-Implement-SDEV_NEW-state.patch --]
[-- Type: text/x-patch, Size: 3769 bytes --]

>From 57109790e4cb53561f36d73a8efc7b9abd2736a9 Mon Sep 17 00:00:00 2001
From: Hannes Reinecke <hare@acerbis.suse.de>
Date: Sat, 8 Mar 2008 18:04:13 +0100
Subject: [PATCH] Implement SDEV_NEW state

When a scsi_device is allocated it's state is set to SDEV_CREATED.
However, we don't have any chance to detect if slave_alloc() has
run successfully or not.
This patch introduces a state SDEV_NEW which is used instead of
SDEV_CREATED upon initial sdev creation. After slave_alloc() has
run successfully the state is changed to SDEV_CREATED.
This allows us to detect later on if we might call slave_destroy()
or not.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/scsi_lib.c    |   17 +++++++++++++----
 drivers/scsi/scsi_scan.c   |    9 ++++++++-
 drivers/scsi/scsi_sysfs.c  |    1 +
 include/scsi/scsi_device.h |    3 ++-
 4 files changed, 24 insertions(+), 6 deletions(-)

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index ba21d97..c398767 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -1999,12 +1999,21 @@ scsi_device_set_state(struct scsi_device *sdev, enum scsi_device_state state)
 		return 0;
 
 	switch (state) {
-	case SDEV_CREATED:
+	case SDEV_NEW:
 		/* There are no legal states that come back to
-		 * created.  This is the manually initialised start
+		 * new.  This is the manually initialised start
 		 * state */
 		goto illegal;
-			
+
+	case SDEV_CREATED:
+		switch (oldstate) {
+		case SDEV_NEW:
+			break;
+		default:
+			goto illegal;
+		}
+		break;
+
 	case SDEV_RUNNING:
 		switch (oldstate) {
 		case SDEV_CREATED:
@@ -2064,7 +2073,7 @@ scsi_device_set_state(struct scsi_device *sdev, enum scsi_device_state state)
 
 	case SDEV_DEL:
 		switch (oldstate) {
-		case SDEV_CREATED:
+		case SDEV_NEW:
 		case SDEV_RUNNING:
 		case SDEV_OFFLINE:
 		case SDEV_CANCEL:
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index 2feab2a..aa632f9 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -253,7 +253,7 @@ static struct scsi_device *scsi_alloc_sdev(struct scsi_target *starget,
 	sdev->id = starget->id;
 	sdev->lun = lun;
 	sdev->channel = starget->channel;
-	sdev->sdev_state = SDEV_CREATED;
+	sdev->sdev_state = SDEV_NEW;
 	INIT_LIST_HEAD(&sdev->siblings);
 	INIT_LIST_HEAD(&sdev->same_target_siblings);
 	INIT_LIST_HEAD(&sdev->cmd_list);
@@ -307,9 +307,16 @@ static struct scsi_device *scsi_alloc_sdev(struct scsi_target *starget,
 			 */
 			if (ret == -ENXIO)
 				display_failure_msg = 0;
+			/*
+			 * sdev remains in SDEV_NEW as the release
+			 * function has to know whether slave_alloc()
+			 * failed or not.
+			 */
 			goto out_device_destroy;
 		}
 	}
+	/* Device is created properly */
+	scsi_device_set_state(sdev, SDEV_CREATED);
 
 	return sdev;
 
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 7dc3015..3ec76dd 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -27,6 +27,7 @@ static const struct {
 	enum scsi_device_state	value;
 	char			*name;
 } sdev_states[] = {
+	{ SDEV_NEW, "new" },
 	{ SDEV_CREATED, "created" },
 	{ SDEV_RUNNING, "running" },
 	{ SDEV_CANCEL, "cancel" },
diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
index ccc437b..1616b26 100644
--- a/include/scsi/scsi_device.h
+++ b/include/scsi/scsi_device.h
@@ -28,7 +28,8 @@ struct scsi_mode_data {
  * scsi_lib:scsi_device_set_state().
  */
 enum scsi_device_state {
-	SDEV_CREATED = 1,	/* device created but not added to sysfs
+	SDEV_NEW = 1,		/* device created, slave_alloc has not run */
+	SDEV_CREATED,		/* device created but not added to sysfs
 				 * Only internal commands allowed (for inq) */
 	SDEV_RUNNING,		/* device properly configured
 				 * All commands allowed */
-- 
1.5.3.2


[-- Attachment #7: 0006-Rename-__scsi_remove_device-into-scsi_sysfs_remove.patch --]
[-- Type: text/x-patch, Size: 3669 bytes --]

>From a2d6ea3b183edc7a71f33e66fce63088c0f3e8a1 Mon Sep 17 00:00:00 2001
From: Hannes Reinecke <hare@acerbis.suse.de>
Date: Mon, 25 Feb 2008 04:17:51 +0100
Subject: [PATCH] Rename __scsi_remove_device() into scsi_sysfs_remove_sdev()

__scsi_remove_device() is actually the counterpart to
scsi_sysfs_add_sdev(). So we'd better rename it to avoid
confusion.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/scsi_priv.h  |    2 +-
 drivers/scsi/scsi_scan.c  |    4 ++--
 drivers/scsi/scsi_sysfs.c |   10 +++++-----
 3 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/scsi/scsi_priv.h b/drivers/scsi/scsi_priv.h
index b33e725..4ff548c 100644
--- a/drivers/scsi/scsi_priv.h
+++ b/drivers/scsi/scsi_priv.h
@@ -112,13 +112,13 @@ extern void scsi_exit_sysctl(void);
 
 /* scsi_sysfs.c */
 extern int scsi_sysfs_add_sdev(struct scsi_device *);
+extern void scsi_sysfs_remove_sdev(struct scsi_device *);
 extern int scsi_sysfs_add_host(struct Scsi_Host *);
 extern int scsi_sysfs_register(void);
 extern void scsi_sysfs_unregister(void);
 extern void scsi_sysfs_device_initialize(struct scsi_device *);
 extern int scsi_sysfs_target_initialize(struct scsi_device *);
 extern struct scsi_transport_template blank_transport_template;
-extern void __scsi_remove_device(struct scsi_device *);
 
 extern struct bus_type scsi_bus_type;
 extern struct attribute_group *scsi_sysfs_shost_attr_groups[];
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index aa632f9..971ac9e 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -1131,7 +1131,7 @@ static int scsi_probe_and_add_lun(struct scsi_target *starget,
 			if (scsi_device_get(sdev) == 0) {
 				*sdevp = sdev;
 			} else {
-				__scsi_remove_device(sdev);
+				scsi_sysfs_remove_sdev(sdev);
 				res = SCSI_SCAN_NO_RESPONSE;
 			}
 		}
@@ -1881,7 +1881,7 @@ void scsi_forget_host(struct Scsi_Host *shost)
 		if (sdev->sdev_state == SDEV_DEL)
 			continue;
 		spin_unlock_irqrestore(shost->host_lock, flags);
-		__scsi_remove_device(sdev);
+		scsi_sysfs_remove_sdev(sdev);
 		goto restart;
 	}
 	spin_unlock_irqrestore(shost->host_lock, flags);
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 3ec76dd..0f33b99 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -851,7 +851,7 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
 	else
 		error = device_create_file(&sdev->sdev_gendev, &dev_attr_queue_depth);
 	if (error) {
-		__scsi_remove_device(sdev);
+		scsi_sysfs_remove_sdev(sdev);
 		goto out;
 	}
 	if (sdev->host->hostt->change_queue_type)
@@ -859,7 +859,7 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
 	else
 		error = device_create_file(&sdev->sdev_gendev, &dev_attr_queue_type);
 	if (error) {
-		__scsi_remove_device(sdev);
+		scsi_sysfs_remove_sdev(sdev);
 		goto out;
 	}
 
@@ -879,7 +879,7 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
 			error = device_create_file(&sdev->sdev_gendev,
 					sdev->host->hostt->sdev_attrs[i]);
 			if (error) {
-				__scsi_remove_device(sdev);
+				scsi_sysfs_remove_sdev(sdev);
 				goto out;
 			}
 		}
@@ -899,7 +899,7 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
 	return error;
 }
 
-void __scsi_remove_device(struct scsi_device *sdev)
+void scsi_sysfs_remove_sdev(struct scsi_device *sdev)
 {
 	struct device *dev = &sdev->sdev_gendev;
 
@@ -926,7 +926,7 @@ void scsi_remove_device(struct scsi_device *sdev)
 	struct Scsi_Host *shost = sdev->host;
 
 	mutex_lock(&shost->scan_mutex);
-	__scsi_remove_device(sdev);
+	scsi_sysfs_remove_sdev(sdev);
 	mutex_unlock(&shost->scan_mutex);
 }
 EXPORT_SYMBOL(scsi_remove_device);
-- 
1.5.3.2


[-- Attachment #8: 0007-Remove-stale-reap_ref-reference.patch --]
[-- Type: text/x-patch, Size: 786 bytes --]

>From 96841a56d0127229a74d5555830759838ff06454 Mon Sep 17 00:00:00 2001
From: Hannes Reinecke <hare@acerbis.suse.de>
Date: Sat, 8 Mar 2008 18:37:22 +0100
Subject: [PATCH] Remove stale reap_ref reference

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/scsi_scan.c |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index 971ac9e..6130495 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -409,7 +409,6 @@ static struct scsi_target *scsi_alloc_target(struct device *parent,
 	}
 	dev = &starget->dev;
 	device_initialize(dev);
-	starget->reap_ref = 1;
 	dev->parent = get_device(parent);
 	sprintf(dev->bus_id, "target%d:%d:%d",
 		shost->host_no, channel, id);
-- 
1.5.3.2

Re: [PATCH 25/20] sysfs: Only support removing emtpy sysfs directories.

[Alan Stern]

To all interested parties:

I have just sent out a series of six patches addressing this problem.  
They are CC'ed to the linux-scsi list.

Alan Stern