PMP and Smart Enclosure ...

PMP and Smart Enclosure ...

[Herbert Poetzl]

following up on a thread about PMP and SEP from earlier this
year, I have now prepared a test system which should allow
me to control all involved parts (i.e. SATA controller, PMP
and the SEP) for testing and debugging ...

during the initial system tests, I found some strange
behaviour, which I am unable to explain, and it might as
well be a SATA/PMP bug in Linux ...

the problem basically arises when I pull a drive from the 
enclosure (without telling Linux first), wait some time
till the kernel becomes aware of the fact, and then put it 
back into the enclosure ...

at this point, the drive indicator (PMP and the drive one)
goes on, the drive spins up, but Linux refuses to detect
the device ... I tried the following things:

 echo "- - -" >/sys/class/scsi_host/host9/scan

 scsiadd -s 9

 resetting the PMP (via small front panel button)

note that the target is still listed in sysfs:
# ls /sys/class/scsi_host/host9/device/
power/  scsi_host/  subsystem@  target9:0:0/  target9:1:0/  target9:2:0/  target9:3:0/  target9:4:0/  uevent

only way to get the device back seems to be to remove the
driver (rmmod sata_sil24) and to reload it again, which is
probably a little tricky assuming that the other disks are
in use :)

many thanks in advance,
Herbert

Re: PMP and Smart Enclosure ...

[Herbert Poetzl]

On Tue, Mar 22, 2011 at 03:54:40PM +0100, Herbert Poetzl wrote:

> following up on a thread about PMP and SEP from earlier this
> year, I have now prepared a test system which should allow
> me to control all involved parts (i.e. SATA controller, PMP
> and the SEP) for testing and debugging ...

> during the initial system tests, I found some strange
> behaviour, which I am unable to explain, and it might as
> well be a SATA/PMP bug in Linux ...

> the problem basically arises when I pull a drive from the 
> enclosure (without telling Linux first), wait some time
> till the kernel becomes aware of the fact, and then put it 
> back into the enclosure ...

> at this point, the drive indicator (PMP and the drive one)
> goes on, the drive spins up, but Linux refuses to detect
> the device ... I tried the following things:

>  echo "- - -" >/sys/class/scsi_host/host9/scan

>  scsiadd -s 9

>  resetting the PMP (via small front panel button)

> note that the target is still listed in sysfs:
> # ls /sys/class/scsi_host/host9/device/
> power/  scsi_host/  subsystem@  target9:0:0/  target9:1:0/  target9:2:0/  target9:3:0/  target9:4:0/  uevent

> only way to get the device back seems to be to remove the
> driver (rmmod sata_sil24) and to reload it again, which is
> probably a little tricky assuming that the other disks are
> in use :)

just for the fun of doing it, I tried to tell Linux first that
the drive will be removed (via scsiadd -r) and discovered that
this causes an even stranger effect:

# scsiadd -r 13 4 0 0		; seems to remove the target
# scsiadd -a 13 4 0 0		; seems to add back the target

Host: scsi13 Channel: 04 Id: 00 Lun: 00
  Vendor: ATA      Model: ST31000340AS     Rev: SD15
  Type:   Direct-Access                    ANSI  SCSI revision: 05

and sometimes, the devices comes back with a different device
entry but the same scsi bus/chan/id/lun, in which case, it
cannot be removed via scsiadd -r anymore, as it just returns

 could not remove device 13 4 0 0 : No such device or address

also note that this doesn't change the fact that the device is
lost till the driver is reloaded once it is unplugged ...

> many thanks in advance,
> Herbert

> --
> To unsubscribe from this list: send the line "unsubscribe linux-ide" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: PMP and Smart Enclosure ...

[Tejun Heo]

On Tue, Mar 22, 2011 at 04:25:29PM +0100, Herbert Poetzl wrote:
> >  echo "- - -" >/sys/class/scsi_host/host9/scan
> 
> >  scsiadd -s 9
> 
> >  resetting the PMP (via small front panel button)

Can you please attach kernel log while this is happening?

Thanks.

-- 
tejun

Re: PMP and Smart Enclosure ...

[Herbert Poetzl]

On Mon, Mar 28, 2011 at 10:52:11AM +0200, Tejun Heo wrote:
> On Tue, Mar 22, 2011 at 04:25:29PM +0100, Herbert Poetzl wrote:
>>>  echo "- - -" >/sys/class/scsi_host/host9/scan

>>>  scsiadd -s 9

>>>  resetting the PMP (via small front panel button)

> Can you please attach kernel log while this is happening?

sure, any specific debug options and/or config settings
you want me to use for the test?

best,
Herbert

> Thanks.

> -- 
> tejun

Re: PMP and Smart Enclosure ...

[Tejun Heo]

On Mon, Mar 28, 2011 at 02:36:23PM +0200, Herbert Poetzl wrote:
> sure, any specific debug options and/or config settings
> you want me to use for the test?

CONFIG_PRINTK_TIME would be nice.

Thanks.

-- 
tejun

Re: PMP and Smart Enclosure ...

[Herbert Poetzl]

On Mon, Mar 28, 2011 at 06:01:36PM +0200, Tejun Heo wrote:
> On Mon, Mar 28, 2011 at 02:36:23PM +0200, Herbert Poetzl wrote:
>> sure, any specific debug options and/or config settings
>> you want me to use for the test?

> CONFIG_PRINTK_TIME would be nice.

http://vserver.13thfloor.at/Stuff/SATAPMP/dmesg_2.6.38.txt

note that it seems to be fixed in 2.6.38.2, I'm now
checking 2.6.38.1 as well to narrow down what fixed
it ...

thanks,
Herbert

> Thanks.

> -- 
> tejun

Re: PMP and Smart Enclosure ...

[Herbert Poetzl]

On Tue, Mar 29, 2011 at 12:46:42AM +0200, Herbert Poetzl wrote:
> On Mon, Mar 28, 2011 at 06:01:36PM +0200, Tejun Heo wrote:
>> On Mon, Mar 28, 2011 at 02:36:23PM +0200, Herbert Poetzl wrote:
>>> sure, any specific debug options and/or config settings
>>> you want me to use for the test?

>> CONFIG_PRINTK_TIME would be nice.

> http://vserver.13thfloor.at/Stuff/SATAPMP/dmesg_2.6.38.txt

> note that it seems to be fixed in 2.6.38.2, I'm now
> checking 2.6.38.1 as well to narrow down what fixed
> it ...

following up on that, already fixed in 2.6.38.1, so maybe

  libata: fix hotplug for drivers which don't implement LPM
  commit eb0e85e36b971ec31610eda7e3ff5c11c1c44785 upstream.

fixed it?

> thanks,
> Herbert

>> Thanks.

>> -- 
>> tejun

Re: PMP and Smart Enclosure ...

[Tejun Heo]

On Tue, Mar 29, 2011 at 01:47:29AM +0200, Herbert Poetzl wrote:
> On Tue, Mar 29, 2011 at 12:46:42AM +0200, Herbert Poetzl wrote:
> > On Mon, Mar 28, 2011 at 06:01:36PM +0200, Tejun Heo wrote:
> >> On Mon, Mar 28, 2011 at 02:36:23PM +0200, Herbert Poetzl wrote:
> >>> sure, any specific debug options and/or config settings
> >>> you want me to use for the test?
> 
> >> CONFIG_PRINTK_TIME would be nice.
> 
> > http://vserver.13thfloor.at/Stuff/SATAPMP/dmesg_2.6.38.txt
> 
> > note that it seems to be fixed in 2.6.38.2, I'm now
> > checking 2.6.38.1 as well to narrow down what fixed
> > it ...
> 
> following up on that, already fixed in 2.6.38.1, so maybe
> 
>   libata: fix hotplug for drivers which don't implement LPM
>   commit eb0e85e36b971ec31610eda7e3ff5c11c1c44785 upstream.
> 
> fixed it?

Ooh, right.  That makes sense.  I screwed up non-AHCI drivers while
reimplementing LPM.  :-)

Thanks.

-- 
tejun

Re: PMP and Smart Enclosure ...

[Herbert Poetzl]

On Tue, Mar 29, 2011 at 10:03:21AM +0200, Tejun Heo wrote:
> On Tue, Mar 29, 2011 at 01:47:29AM +0200, Herbert Poetzl wrote:
>> On Tue, Mar 29, 2011 at 12:46:42AM +0200, Herbert Poetzl wrote:
>>> On Mon, Mar 28, 2011 at 06:01:36PM +0200, Tejun Heo wrote:
>>>> On Mon, Mar 28, 2011 at 02:36:23PM +0200, Herbert Poetzl wrote:
>>>>> sure, any specific debug options and/or config settings
>>>>> you want me to use for the test?

>>>> CONFIG_PRINTK_TIME would be nice.

>>> http://vserver.13thfloor.at/Stuff/SATAPMP/dmesg_2.6.38.txt

>>> note that it seems to be fixed in 2.6.38.2, I'm now
>>> checking 2.6.38.1 as well to narrow down what fixed
>>> it ...

>> following up on that, already fixed in 2.6.38.1, so maybe

>>   libata: fix hotplug for drivers which don't implement LPM
>>   commit eb0e85e36b971ec31610eda7e3ff5c11c1c44785 upstream.

>> fixed it?

> Ooh, right.  That makes sense.  
> I screwed up non-AHCI drivers while reimplementing LPM.  :-)

no problem, thanks for fixing it :)

best,
Herbert

PS: any thoughts on how/where I should add the device 
    interface for the SATA PMP?

> Thanks.

> -- 
> tejun
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ide" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: PMP and Smart Enclosure ...

[Tejun Heo]

On Tue, Mar 29, 2011 at 04:49:06PM +0200, Herbert Poetzl wrote:
> PS: any thoughts on how/where I should add the device 
>     interface for the SATA PMP?

One way would be adding a device node for the PMP device but I don't
have much idea how it should look like.  The thing is that PMP isn't a
completely separate device.  It's more of a dummy hub and an extension
of the host SATA bus rather than a separate device.  Maybe it's best
to just put it together on the host itself.

-- 
tejun

Re: PMP and Smart Enclosure ...

[Herbert Poetzl]

On Tue, Mar 29, 2011 at 05:01:04PM +0200, Tejun Heo wrote:
> On Tue, Mar 29, 2011 at 04:49:06PM +0200, Herbert Poetzl wrote:
>> PS: any thoughts on how/where I should add the device 
>>     interface for the SATA PMP?

> One way would be adding a device node for the PMP device but I don't
> have much idea how it should look like.  The thing is that PMP isn't a
> completely separate device.  It's more of a dummy hub and an extension
> of the host SATA bus rather than a separate device.  Maybe it's best
> to just put it together on the host itself.

well, as I see it, there is a bunch of options here, with 
various advantages and of course certain limitations/drawbacks

 1) we could simply add an SCSI device for the PMP
    e.g. if there is 8:0:0:0 - 8:4:0:0 we could add 8:15:0:0

    pro: probably the most flexible interface as one can
         send whatever PMP messages they like, but I'm
         not sure how to handle the replies here
    con: might need special security etc, but OTOH, you
         can mess up a disk as well ...

 2) we could add some special IOCTL, based on the existing
    devices

    pro: could be limited to certain well defined commands
         and there is no need for new interfaces
    con: not really associated with the PMP, i.e. more a
         hack which needs special handling for non PMP
         attached disks

 3) we could have some specific sysfs interfaces doing
    certain parts of the communication

    pro: easy to add, well known interface and could be
         structured and associated with the host controller
    con: very limited and needs kernel code for each new
         PMP message, also request/reply type of commands
         get really ugly soon and finally we need to handle
         more than one PMP per host

 4) create an i2c module which 'attaches' to the PMP(s)

    pro: no problem with request/reply and basically the
         proper interface between SEMB and SEP
    con: only a subset of the PMP messages are related to
         SEMB/SEP stuff, so that would only solve this
         particular problem and not e.g. PMP GPIO

I'd lean towards 1, because it will give the most flexibility
and will allow fast prototyping in userspace, my second choice
would be 4) as I'm mainly interested in the SEMB/SEP stuff
right now, but I definitely want to avoid adding something
with almost zero chance to go upstream :)

thanks,
Herbert

> -- 
> tejun