RE: fibre channel sync cache question

RE: fibre channel sync cache question

[Moore, Eric]

-- Tuesday, July 25, 2006 4:22 PM, Michael Reed wrote: 

> Using fibre channel disks, I've noticed that when the system 
> shuts down
> that the sd_driver issues a sync cache command to the device. 
>  I've also
> noticed that when the lldd is removed via rmmod that this 
> sync cache is
> not executed.  I would think that the sync cache would be desirable
> under this circumstance.
> 

This is not handled from sg path as well.  Meaning if you
use sdparm, and enable the caching page WCE bit, then reboot,
there is no SYNC cache issued from above. 

We handle this in fusion drivers due to short coming from above.

Eric

RE: fibre channel sync cache question

[Arjan van de Ven]

On Tue, 2006-07-25 at 16:28 -0600, Moore, Eric wrote:
> -- Tuesday, July 25, 2006 4:22 PM, Michael Reed wrote: 
> 
> > Using fibre channel disks, I've noticed that when the system 
> > shuts down
> > that the sd_driver issues a sync cache command to the device. 
> >  I've also
> > noticed that when the lldd is removed via rmmod that this 
> > sync cache is
> > not executed.  I would think that the sync cache would be desirable
> > under this circumstance.
> > 
> 
> This is not handled from sg path as well.  Meaning if you
> use sdparm, and enable the caching page WCE bit, then reboot,
> there is no SYNC cache issued from above. 
> 
> We handle this in fusion drivers due to short coming from above.


Hi,

that sounds sooo like the wrong approach... wouldn't it be better to fix
sg instead?


Greetings,
    Arjan van de Ven

-- 
if you want to mail me at work (you don't), use arjan (at) linux.intel.com

Re: fibre channel sync cache question

[Douglas Gilbert]

Arjan van de Ven wrote:
> On Tue, 2006-07-25 at 16:28 -0600, Moore, Eric wrote:
>> -- Tuesday, July 25, 2006 4:22 PM, Michael Reed wrote: 
>>
>>> Using fibre channel disks, I've noticed that when the system 
>>> shuts down
>>> that the sd_driver issues a sync cache command to the device. 
>>>  I've also
>>> noticed that when the lldd is removed via rmmod that this 
>>> sync cache is
>>> not executed.  I would think that the sync cache would be desirable
>>> under this circumstance.
>>>
>> This is not handled from sg path as well.  Meaning if you
>> use sdparm, and enable the caching page WCE bit, then reboot,
>> there is no SYNC cache issued from above. 
>>
>> We handle this in fusion drivers due to short coming from above.
> 
> 
> Hi,
> 
> that sounds sooo like the wrong approach... wouldn't it be better to fix
> sg instead?

Uh? sg is just a pass through. As such it can subvert
policy decisions of the kernel. That isn't always a
bad thing :-)

The design flaw is any driver that tries to maintain a
state variable associated with a device (logical unit)
and can't cope with situations when it gets out of sync.
If you managed to neuter the pass through, how would you
cope with another initiator?

Doug Gilbert

Re: fibre channel sync cache question

[Michael Reed]

Douglas Gilbert wrote:
> Arjan van de Ven wrote:
>> On Tue, 2006-07-25 at 16:28 -0600, Moore, Eric wrote:
>>> -- Tuesday, July 25, 2006 4:22 PM, Michael Reed wrote:
>>>> Using fibre channel disks, I've noticed that when the system shuts down
>>>> that the sd_driver issues a sync cache command to the device.  I've
>>>> also
>>>> noticed that when the lldd is removed via rmmod that this sync cache is
>>>> not executed.  I would think that the sync cache would be desirable
>>>> under this circumstance.
>>>>
>>> This is not handled from sg path as well.  Meaning if you
>>> use sdparm, and enable the caching page WCE bit, then reboot,
>>> there is no SYNC cache issued from above.
>>> We handle this in fusion drivers due to short coming from above.
>>
>>
>> Hi,
>>
>> that sounds sooo like the wrong approach... wouldn't it be better to fix
>> sg instead?
> 
> Uh? sg is just a pass through. As such it can subvert
> policy decisions of the kernel. That isn't always a
> bad thing :-)
> 
> The design flaw is any driver that tries to maintain a
> state variable associated with a device (logical unit)
> and can't cope with situations when it gets out of sync.
> If you managed to neuter the pass through, how would you
> cope with another initiator?

Perhaps the best policy for sd is to assume that WCE is enabled
and just issue the sync cache.

--

I'm wondering about the policy of issuing a sync cache.  There
are target removal paths which result in it not being issued.

So, the real question is: when a scsi target is removed, is it
policy that sync cache will be issued?

In fibre channel, here are two code paths in which sync cache
is not issued.

	- removal of LLDD (rmmod)
	- removal of target via sysfs device/delete

In a closer look at the target removal via sysfs device/delete,
I observe that portions of the sysfs fc_remote_ports/rport-*
remain in place.

Do we need to tie the device/delete to the transport?

Can of worms?  Close eyes, run screaming?

Mike


> 
> Doug Gilbert
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: fibre channel sync cache question

[Michael Reed]

Michael Reed wrote:

> 
> In a closer look at the target removal via sysfs device/delete,
> I observe that portions of the sysfs fc_remote_ports/rport-*
> remain in place.
> 

Doh.  What a stupid observation I made.  It's working as intended.
It keeps the bindings around.

Mike

> Do we need to tie the device/delete to the transport?
> 
> Can of worms?  Close eyes, run screaming?
> 
> Mike
> 
> 

Re: fibre channel sync cache question

[Stefan Richter]

Michael Reed wrote:
>>>>-- Tuesday, July 25, 2006 4:22 PM, Michael Reed wrote:
>>>>>Using fibre channel disks, I've noticed that when the system shuts down
>>>>>that the sd_driver issues a sync cache command to the device.  I've
>>>>>also
>>>>>noticed that when the lldd is removed via rmmod that this sync cache is
>>>>>not executed.  I would think that the sync cache would be desirable
>>>>>under this circumstance.
[...]
> I'm wondering about the policy of issuing a sync cache.  There
> are target removal paths which result in it not being issued.
> 
> So, the real question is: when a scsi target is removed, is it
> policy that sync cache will be issued?
> 
> In fibre channel, here are two code paths in which sync cache
> is not issued.
> 
> 	- removal of LLDD (rmmod)
> 	- removal of target via sysfs device/delete
[...]

This problem exists with all transports which use scsi_remove_device or 
scsi_remove_host for soft device removal, as noted by Mike Christie in 
"question about sd_sync_cache and shutdown" on 2006-07-12. He pointed 
out that devices are brought into SDEV_CANCEL state before sd_shutdown 
is entered. Therefore sd_shutdown does not sync the cache.

This behavior changed a few Linux releases ago. I cannot pinpoint the 
responsible changeset. So, to answer your question about policy, there 
is no fixed policy in the specification and implementation of the 
mid-low SCSI API. Generally, neither "soft shutdown" nor "hot unplug" 
can be explicitly signaled from LLD to the SCSI core.

Unless somebody comes up with a general solution for SCSI core, I am 
planning to look for a particular solution in the sbp2 transport driver. 
(When an sbp2 LU is to be taken down, check if it is still physically 
present and somehow trigger SCSI high-level shutdown methods before 
scsi_remove_device.)
-- 
Stefan Richter
-=====-=-==- -=== ==-=-
http://arcgraph.de/sr/

RE: fibre channel sync cache question

[Moore, Eric]

-- Tuesday, July 25, 2006 4:40 PM, Arjan van de Ven 

> 
> Hi,
> 
> that sounds sooo like the wrong approach... wouldn't it be 
> better to fix
> sg instead?
> 
> 

Agreed.  We want to remove our sync cache code when its fully handled
from all cases above.

fibre channel sync cache question

[Michael Reed]

Hello All,

Using fibre channel disks, I've noticed that when the system shuts down
that the sd_driver issues a sync cache command to the device.  I've also
noticed that when the lldd is removed via rmmod that this sync cache is
not executed.  I would think that the sync cache would be desirable
under this circumstance.

Was it a conscious decision to not have sync cache executed during
module unload?

Is it desirable?

(There are various reasons why it doesn't work.  The rport port_state
is not ONLINE, the module is being unloaded so that a reference cannot
be acquired, the sdev state is SDEV_CANCEL.  Three strikes and you're
out?)


Mike