RE: libata, SCSI and storage drivers

RE: libata, SCSI and storage drivers

[James.Smart]

Folks,

This discussion brings up some latent questions...

The transport can be a subsystem on it's own and is perhaps independent
of SCSI altogether. In this case, SCSI just happens to be a personality
of something on the transport. This is at odds with the current design
in which the transport is something under SCSI and inherently bound to
the SCSI "host".

I understand how we got to where we are, but shouldn't we consider making
some transports independent subsystems ? If the only protocol that
can be run on the transport is SCSI (ex: SPI), then the transport can be
under SCSI. However, if the transport can support multiple protocols (FC
can support SCSI, IP, (or ATA)), shouldn't it be structured more like an io
bus like pci ? 

It does mess up the device tree heirarchy. In general, you want the
device tree to continue along the transport specific elements until it finds
remote endpoints (things your going to use), at which point the protocol
specific elements can kick in. For example (using FC):
 /sys/devices/<pci>/fcport5/rport-5:3/target10:0:0/10:0:0:0  - the SCSI lun

What this leaves out is : where is the scsi host device ? It doesn't make
sense to insert it in-between the transport elements. It likely just becomes
a leaf entity. Continuing the example:
 /sys/devices/<pci>/fcport5/host10  - scsi host interface
 /sys/devices/<pci>/fcport5/eth3    - network interface

Food for thought...   Is this out in left field ?

-- james s


> -----Original Message-----
> From: linux-scsi-owner@vger.kernel.org
> [mailto:linux-scsi-owner@vger.kernel.org]On Behalf Of Luben Tuikov
> Sent: Friday, May 27, 2005 10:42 AM
> To: dougg@torque.net
> Cc: Jeff Garzik; Jens Axboe; James Bottomley; SCSI Mailing List;
> linux-ide@vger.kernel.org
> Subject: Re: libata, SCSI and storage drivers
> 
> 
> On 05/27/05 02:45, Douglas Gilbert wrote:
> > In a) the interconnect is SATA. Still it is hard
> > to believe that the SAS HBA LLD would belong to
> > anything other than the SCSI subsystem (since
> > SAS HBAs come with 4 or 8 phys, others of which could
> > be in scenario b) and/or connected to SAS disks).
> > Hence the initiator_ports/phys/target_ports on and
> > seen by that SAS HBA should (also) have SAS transport
> > classes.
> 
> It should register with the SAS transport class, where
> the SAS discover "function" is.  The Discover function
> would find out what is out on the domain and "register"
> it with the proper "transport" (a more correct way to
> put it there is).
> 
> For SSP devices that would be a "scsi disk", for
> SATA devices that would be a "libata disk" or something
> like this.  The LLDD would either get a CDB for SSP,
> or taskfile for STP, and pass it along to the hardware.
>  
> > When a SAS HBA phy is not connected to anything is it
> > a member of a SAS transport class or a ATA transport
> > class (or both)?
> 
> Quite possibly both: SAS transport class, since it is
> a super class in meaning that the Discover function
> is in there.
>  
> > In scenario b) the left interconnect is SAS and the
> > right interconnect is SATA. The SAS_expander contains
> > the STP<->SATA bridging function (and, for sake of
> > argument, no SCSI-ATA Translation (SAT) layer).
> > Would scenario b) also have a ATA transport
> > class? I'll assume it does. To be discovered it also
> 
> Yes it would.  As far as the class is concerned it
> is a SATA device -- how you get to it (direct or through
> an expander) is the LLDD/firmware/Host Adapter concern.
> 
> > needs a SAS transport class. Larger enclosures are
> > likely to be amplifications of scenario b). The presence
> > of the SATA disk in scenario b) will be discovered via
> > the SAS SMP (i.e. talking to the SAS_expander) or via
> > the SES protocol (i.e. a SCSI Enclosure Services target
> > running near the SAS_expander). Either way if there are
> > a lot of SATA disks then they are likely to be held
> > in their initialization sequence to stop them spinning
> > up all at once. SAS transport intervention may be
> > required (staggered timers are another possibility).
> 
> Yes, spinup-hold states would be managed by the Discover
> function, or by the FW on the expander/ses device.
> 
> BTW, very nice to mention SES devices.  We'll have to
> bring that up more as they'd be very much en vogue
> with expanders.
> 
> > Now I may be wrong but I think that one of the SAS HBAs
> > that I have read about that looks (externally) like
> > scenario a) but is actually scenario b). In other words
> > the SAS_expander is silicon on the HBA and it is not
> > controlled via the PCI bus, but rather by SMP.
> 
> I know of such a SAS HA (Host Adapter), and from what it
> seems the role of the expander-on-chip is to simulate
> a larger number of phys than actually/physically supported
> by the HA.  By design, this should be transparent to the
> Discover function, and if it is not, I believe the Discover
> fn. would do the right thing (i.e. treat is as an expander).
> 
> > This suggests to me we would need an ordered sequence of
> > transport classes. I really wonder about trying to model
> > this level of complexity in sysfs, plus the nightmare of
> > keeping state data of (topologically) distant nodes up
> > to date. At some point one needs to supply a management
> 
> When a SSP talking device is found it is registered
> as a "scsi device".  When a STP talking device is
> discovered (directly attached or elsewhere) it would
> be registered as a "libata device".
> 
> > pass through and hand the problem over to the user
> > space. The question is, at what point.
> 
> I'm not 100% sure about user space.  The Discover function
> is pretty straightforward, and simple.  Plus the root/bootable
> device could be a SATA device on an expander or even deeper
> in the domain.
> 
> Thanks,
> 	Luben
> -
> To unsubscribe from this list: send the line "unsubscribe 
> linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: libata, SCSI and storage drivers

[Jeff Garzik]

James.Smart@Emulex.Com wrote:
> Folks,
> 
> This discussion brings up some latent questions...
> 
> The transport can be a subsystem on it's own and is perhaps independent
> of SCSI altogether. In this case, SCSI just happens to be a personality
> of something on the transport. This is at odds with the current design
> in which the transport is something under SCSI and inherently bound to
> the SCSI "host".

Yes.  This is something I really want to change, for libata.


> I understand how we got to where we are, but shouldn't we consider making
> some transports independent subsystems ? If the only protocol that
> can be run on the transport is SCSI (ex: SPI), then the transport can be
> under SCSI. However, if the transport can support multiple protocols (FC
> can support SCSI, IP, (or ATA)), shouldn't it be structured more like an io
> bus like pci ? 

Unfortunately this is an open-ended question that Linux is rather poor 
at answering, since the answer could range from "no" to "show me the 
code" to "you're an absolute visionary!"  :)

My own opinion:

I consciously avoid thinking too much in that direction.  Linux 
development is a lot like a biological process.  The evolution of the 
kernel code over time will give us the best answer.

Perhaps we will merge request_queue and network stack systems into a 
single "packet transport" system.  Perhaps net stack and request_queue 
systems will stay separate, and request_queue will evolve into a 
generalized system for RPC message transport.

With the device model, both IDE [as of yesterday] and SCSI -already- 
export bus topology in a standardized manner, just like PCI.

	Jeff

RE: libata, SCSI and storage drivers

[James Bottomley]

On Fri, 2005-05-27 at 13:45 -0400, James.Smart@Emulex.Com wrote:
> The transport can be a subsystem on it's own and is perhaps independent
> of SCSI altogether. In this case, SCSI just happens to be a personality
> of something on the transport. This is at odds with the current design
> in which the transport is something under SCSI and inherently bound to
> the SCSI "host".

Actually, no that's no longer true.  Initially I did it this way, but
now the SCSI transport classes are build on top of the generic transport
classes (and these are independent of SCSI).  I anticipate that soon
we'll get a PHY transport class that will attach both to SAS and SATA
devices (and won't care which subsystem they're under).

> I understand how we got to where we are, but shouldn't we consider making
> some transports independent subsystems ? If the only protocol that
> can be run on the transport is SCSI (ex: SPI), then the transport can be
> under SCSI. However, if the transport can support multiple protocols (FC
> can support SCSI, IP, (or ATA)), shouldn't it be structured more like an io
> bus like pci ? 
> 
> It does mess up the device tree heirarchy. In general, you want the
> device tree to continue along the transport specific elements until it finds
> remote endpoints (things your going to use), at which point the protocol
> specific elements can kick in. For example (using FC):
>  /sys/devices/<pci>/fcport5/rport-5:3/target10:0:0/10:0:0:0  - the SCSI lun
> 
> What this leaves out is : where is the scsi host device ? It doesn't make
> sense to insert it in-between the transport elements. It likely just becomes
> a leaf entity. Continuing the example:
>  /sys/devices/<pci>/fcport5/host10  - scsi host interface
>  /sys/devices/<pci>/fcport5/eth3    - network interface
> 
> Food for thought...   Is this out in left field ?

Well, that's why it's a class.  All the devices appear under 

/class/<transport class name>

and these devices are simply the names of the actual generic devices, so
there's no reason target0:3:0 can't co-exist happily with ata3:0 or
something here.  The idea being (I think) that the class infrastructure
actually flattens the tree.  So there's always a device link that points
into the true device tree, but all the class properties are available in
flattened form.

James

RE: libata, SCSI and storage drivers

[James.Smart]

Also, I should note that plain ATA (no SAT) is valid in this
configuration as well.

-- james s

> -----Original Message-----
> From: linux-scsi-owner@vger.kernel.org
> [mailto:linux-scsi-owner@vger.kernel.org]On Behalf Of Smart, James
> Sent: Friday, May 27, 2005 8:43 AM
> To: dougg@torque.net; jgarzik@pobox.com
> Cc: axboe@suse.de; James.Bottomley@SteelEye.com;
> linux-scsi@vger.kernel.org; linux-ide@vger.kernel.org
> Subject: RE: libata, SCSI and storage drivers
> 
> 
> FYI -  There is currently a standards proposal in T11 for the
> following scenario:
> 
>   c) FC HBA  <--------> FC fabric/loop <------>  SATA disk
> 
> In this case, the SAT layer could be in the FC HBA firmware or
> in the FC driver (or whatever linux transport exists).  A demo for
> this scenario was shown recently at SNW.
> 
> Also - I expect to see more and more implementations with the
> SAS Expander tightly integrated to the SAS HBA/phys.
> 
> -- james s
> 
> 
> > Jeff,
> > Yes, the SAS HBAs that I'm aware of can connect to
> > SATA I/II disks "directly" on their phys.
> > 
> > So here are two scenarios for connecting SATA disks:
> > 
> > a)
> >   SAS HBA < -------------------------> SATA disk
> > 
> > b)
> >   SAS HBA <------>SAS_expander<------> SATA disk
> > 
> > In a) the interconnect is SATA. Still it is hard
> > to believe that the SAS HBA LLD would belong to
> > anything other than the SCSI subsystem (since
> > SAS HBAs come with 4 or 8 phys, others of which could
> > be in scenario b) and/or connected to SAS disks).
> > Hence the initiator_ports/phys/target_ports on and
> > seen by that SAS HBA should (also) have SAS transport
> > classes.
> > 
> > When a SAS HBA phy is not connected to anything is it
> > a member of a SAS transport class or a ATA transport
> > class (or both)?
> > 
> > In scenario b) the left interconnect is SAS and the
> > right interconnect is SATA. The SAS_expander contains
> > the STP<->SATA bridging function (and, for sake of
> > argument, no SCSI-ATA Translation (SAT) layer).
> > Would scenario b) also have a ATA transport
> > class? I'll assume it does. To be discovered it also
> > needs a SAS transport class. Larger enclosures are
> > likely to be amplifications of scenario b). The presence
> > of the SATA disk in scenario b) will be discovered via
> > the SAS SMP (i.e. talking to the SAS_expander) or via
> > the SES protocol (i.e. a SCSI Enclosure Services target
> > running near the SAS_expander). Either way if there are
> > a lot of SATA disks then they are likely to be held
> > in their initialization sequence to stop them spinning
> > up all at once. SAS transport intervention may be
> > required (staggered timers are another possibility).
> > 
> > Now I may be wrong but I think that one of the SAS HBAs
> > that I have read about that looks (externally) like
> > scenario a) but is actually scenario b). In other words
> > the SAS_expander is silicon on the HBA and it is not
> > controlled via the PCI bus, but rather by SMP.
> > 
> > This suggests to me we would need an ordered sequence of
> > transport classes. I really wonder about trying to model
> > this level of complexity in sysfs, plus the nightmare of
> > keeping state data of (topologically) distant nodes up
> > to date. At some point one needs to supply a management
> > pass through and hand the problem over to the user
> > space. The question is, at what point.
> > 
> > 
> > There are already FCP enclosures filled with SATA disks
> > on the market so this problem in not unique to SAS.
> > However they have (I presume) a SAT layer in their
> > FC enclosures so the OS thinks it is taking to SCSI
> > disks.
> > 
> > Doug Gilbert
> > 
> > 
> > 
> > -
> > To unsubscribe from this list: send the line "unsubscribe 
> > linux-scsi" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> -
> To unsubscribe from this list: send the line "unsubscribe 
> linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

RE: libata, SCSI and storage drivers

[James.Smart]

FYI -  There is currently a standards proposal in T11 for the
following scenario:

  c) FC HBA  <--------> FC fabric/loop <------>  SATA disk

In this case, the SAT layer could be in the FC HBA firmware or
in the FC driver (or whatever linux transport exists).  A demo for
this scenario was shown recently at SNW.

Also - I expect to see more and more implementations with the
SAS Expander tightly integrated to the SAS HBA/phys.

-- james s


> Jeff,
> Yes, the SAS HBAs that I'm aware of can connect to
> SATA I/II disks "directly" on their phys.
> 
> So here are two scenarios for connecting SATA disks:
> 
> a)
>   SAS HBA < -------------------------> SATA disk
> 
> b)
>   SAS HBA <------>SAS_expander<------> SATA disk
> 
> In a) the interconnect is SATA. Still it is hard
> to believe that the SAS HBA LLD would belong to
> anything other than the SCSI subsystem (since
> SAS HBAs come with 4 or 8 phys, others of which could
> be in scenario b) and/or connected to SAS disks).
> Hence the initiator_ports/phys/target_ports on and
> seen by that SAS HBA should (also) have SAS transport
> classes.
> 
> When a SAS HBA phy is not connected to anything is it
> a member of a SAS transport class or a ATA transport
> class (or both)?
> 
> In scenario b) the left interconnect is SAS and the
> right interconnect is SATA. The SAS_expander contains
> the STP<->SATA bridging function (and, for sake of
> argument, no SCSI-ATA Translation (SAT) layer).
> Would scenario b) also have a ATA transport
> class? I'll assume it does. To be discovered it also
> needs a SAS transport class. Larger enclosures are
> likely to be amplifications of scenario b). The presence
> of the SATA disk in scenario b) will be discovered via
> the SAS SMP (i.e. talking to the SAS_expander) or via
> the SES protocol (i.e. a SCSI Enclosure Services target
> running near the SAS_expander). Either way if there are
> a lot of SATA disks then they are likely to be held
> in their initialization sequence to stop them spinning
> up all at once. SAS transport intervention may be
> required (staggered timers are another possibility).
> 
> Now I may be wrong but I think that one of the SAS HBAs
> that I have read about that looks (externally) like
> scenario a) but is actually scenario b). In other words
> the SAS_expander is silicon on the HBA and it is not
> controlled via the PCI bus, but rather by SMP.
> 
> This suggests to me we would need an ordered sequence of
> transport classes. I really wonder about trying to model
> this level of complexity in sysfs, plus the nightmare of
> keeping state data of (topologically) distant nodes up
> to date. At some point one needs to supply a management
> pass through and hand the problem over to the user
> space. The question is, at what point.
> 
> 
> There are already FCP enclosures filled with SATA disks
> on the market so this problem in not unique to SAS.
> However they have (I presume) a SAT layer in their
> FC enclosures so the OS thinks it is taking to SCSI
> disks.
> 
> Doug Gilbert
> 
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe 
> linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

[PATCH] libata: device suspend/resume

[Jeff Garzik]

Patch 2/2 for review by the linux-scsi crowd.

Please do not apply.



Patch is via Mark Lord, 

It's not ready for upstream, but I am posting it for two reasons:

1) People might find the feature useful (warning warning do not use in
production, just a test feature, might cook your hard drive and all your
data, standard disclaimer...)

2) Get comments from linux-scsi folks about the SCSI changes.




Index: drivers/scsi/ahci.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/ahci.c  (mode:100644 sha1:da5bd33d982d7e521b6b0603adcb0911395b184d)
+++ uncommitted/drivers/scsi/ahci.c  (mode:100644)
@@ -201,6 +201,8 @@
 	.slave_configure	= ata_scsi_slave_config,
 	.bios_param		= ata_std_bios_param,
 	.ordered_flush		= 1,
+	.resume			= ata_scsi_device_resume,
+	.suspend		= ata_scsi_device_suspend,
 };
 
 static struct ata_port_operations ahci_ops = {
@@ -272,6 +274,8 @@
 	.id_table		= ahci_pci_tbl,
 	.probe			= ahci_init_one,
 	.remove			= ata_pci_remove_one,
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
 };
 
 
Index: drivers/scsi/ata_piix.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/ata_piix.c  (mode:100644 sha1:3867f91ef8c7bebe3d0edf0bd2e3b326a1d4061b)
+++ uncommitted/drivers/scsi/ata_piix.c  (mode:100644)
@@ -104,6 +104,8 @@
 	.id_table		= piix_pci_tbl,
 	.probe			= piix_init_one,
 	.remove			= ata_pci_remove_one,
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
 };
 
 static Scsi_Host_Template piix_sht = {
@@ -124,6 +126,8 @@
 	.slave_configure	= ata_scsi_slave_config,
 	.bios_param		= ata_std_bios_param,
 	.ordered_flush		= 1,
+	.resume			= ata_scsi_device_resume,
+	.suspend		= ata_scsi_device_suspend,
 };
 
 static struct ata_port_operations piix_pata_ops = {
Index: drivers/scsi/libata-core.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/libata-core.c  (mode:100644 sha1:0b5d3a5b7edab5ce17889a0e698805ce346b4d1a)
+++ uncommitted/drivers/scsi/libata-core.c  (mode:100644)
@@ -3299,6 +3299,104 @@
 	ata_qc_complete(qc, ATA_ERR);
 }
 
+/*
+ * Execute a 'simple' command, that only consists of the opcode 'cmd' itself,
+ * without filling any other registers
+ */
+static int ata_do_simple_cmd(struct ata_port *ap, struct ata_device *dev,
+			     u8 cmd)
+{
+	DECLARE_COMPLETION(wait);
+	struct ata_queued_cmd *qc;
+	unsigned long flags;
+	int rc;
+
+	qc = ata_qc_new_init(ap, dev);
+	BUG_ON(qc == NULL);
+
+	qc->tf.command = cmd;
+	qc->tf.flags |= ATA_TFLAG_DEVICE;
+	qc->tf.protocol = ATA_PROT_NODATA;
+
+	qc->waiting = &wait;
+	qc->complete_fn = ata_qc_complete_noop;
+
+	spin_lock_irqsave(&ap->host_set->lock, flags);
+	rc = ata_qc_issue(qc);
+	spin_unlock_irqrestore(&ap->host_set->lock, flags);
+
+	if (!rc)
+		wait_for_completion(&wait);
+
+	return rc;
+}
+
+static int ata_flush_cache(struct ata_port *ap, struct ata_device *dev)
+{
+	u8 cmd;
+
+	if (!ata_try_flush_cache(dev))
+		return 0;
+
+	if (ata_id_has_flush_ext(dev->id))
+		cmd = ATA_CMD_FLUSH_EXT;
+	else
+		cmd = ATA_CMD_FLUSH;
+
+	return ata_do_simple_cmd(ap, dev, cmd);
+}
+
+static int ata_standby_drive(struct ata_port *ap, struct ata_device *dev)
+{
+	return ata_do_simple_cmd(ap, dev, ATA_CMD_STANDBYNOW1);
+}
+
+static int ata_start_drive(struct ata_port *ap, struct ata_device *dev)
+{
+	return ata_do_simple_cmd(ap, dev, ATA_CMD_IDLEIMMEDIATE);
+}
+
+/**
+ *	ata_device_resume - wakeup a previously suspended devices
+ *
+ *	Kick the drive back into action, by sending it an idle immediate
+ *	command and making sure its transfer mode matches between drive
+ *	and host.
+ *
+ */
+int ata_device_resume(struct ata_port *ap, struct ata_device *dev)
+{
+	if (ap->flags & ATA_FLAG_SUSPENDED) {
+		ap->flags &= ~ATA_FLAG_SUSPENDED;
+		ata_set_mode(ap);
+	}
+	if (!ata_dev_present(dev))
+		return 0;
+	if (dev->class == ATA_DEV_ATA)
+		ata_start_drive(ap, dev);
+
+	return 0;
+}
+
+/**
+ *	ata_device_suspend - prepare a device for suspend
+ *
+ *	Flush the cache on the drive, if appropriate, then issue a
+ *	standbynow command.
+ *
+ */
+int ata_device_suspend(struct ata_port *ap, struct ata_device *dev)
+{
+	if (!ata_dev_present(dev))
+		return 0;
+	if (dev->class == ATA_DEV_ATA)
+		ata_flush_cache(ap, dev);
+
+	ata_standby_drive(ap, dev);
+	ap->flags |= ATA_FLAG_SUSPENDED;
+	return 0;
+}
+
 int ata_port_start (struct ata_port *ap)
 {
 	struct device *dev = ap->host_set->dev;
@@ -3937,6 +4035,23 @@
 
 	return (tmp == bits->val) ? 1 : 0;
 }
+
+int ata_pci_device_suspend(struct pci_dev *pdev, pm_message_t state)
+{
+	pci_save_state(pdev);
+	pci_disable_device(pdev);
+	pci_set_power_state(pdev, PCI_D3hot);
+	return 0;
+}
+
+int ata_pci_device_resume(struct pci_dev *pdev)
+{
+	pci_set_power_state(pdev, PCI_D0);
+	pci_restore_state(pdev);
+	pci_enable_device(pdev);
+	pci_set_master(pdev);
+	return 0;
+}
 #endif /* CONFIG_PCI */
 
 
@@ -4021,4 +4136,11 @@
 EXPORT_SYMBOL_GPL(ata_pci_init_native_mode);
 EXPORT_SYMBOL_GPL(ata_pci_init_one);
 EXPORT_SYMBOL_GPL(ata_pci_remove_one);
+EXPORT_SYMBOL_GPL(ata_pci_device_suspend);
+EXPORT_SYMBOL_GPL(ata_pci_device_resume);
 #endif /* CONFIG_PCI */
+
+EXPORT_SYMBOL_GPL(ata_device_suspend);
+EXPORT_SYMBOL_GPL(ata_device_resume);
+EXPORT_SYMBOL_GPL(ata_scsi_device_suspend);
+EXPORT_SYMBOL_GPL(ata_scsi_device_resume);
Index: drivers/scsi/libata-scsi.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/libata-scsi.c  (mode:100644 sha1:4c96df060c3bad9af44ab4c4664d8cba38c63030)
+++ uncommitted/drivers/scsi/libata-scsi.c  (mode:100644)
@@ -387,6 +387,22 @@
 	return 0;
 }
 
+int ata_scsi_device_resume(struct scsi_device *sdev)
+{
+	struct ata_port *ap = (struct ata_port *) &sdev->host->hostdata[0];
+	struct ata_device *dev = &ap->device[sdev->id];
+
+	return ata_device_resume(ap, dev);
+}
+
+int ata_scsi_device_suspend(struct scsi_device *sdev)
+{
+	struct ata_port *ap = (struct ata_port *) &sdev->host->hostdata[0];
+	struct ata_device *dev = &ap->device[sdev->id];
+
+	return ata_device_suspend(ap, dev);
+}
+
 /**
  *	ata_scsi_flush_xlat - Translate SCSI SYNCHRONIZE CACHE command
  *	@qc: Storage for translated ATA taskfile
Index: drivers/scsi/sata_nv.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/sata_nv.c  (mode:100644 sha1:69009f853a4917f54a88aa0c2cf3e6fb0428c8f7)
+++ uncommitted/drivers/scsi/sata_nv.c  (mode:100644)
@@ -187,6 +187,8 @@
 	.id_table		= nv_pci_tbl,
 	.probe			= nv_init_one,
 	.remove			= ata_pci_remove_one,
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
 };
 
 static Scsi_Host_Template nv_sht = {
@@ -207,6 +209,8 @@
 	.slave_configure	= ata_scsi_slave_config,
 	.bios_param		= ata_std_bios_param,
 	.ordered_flush		= 1,
+	.resume			= ata_scsi_device_resume,
+	.suspend		= ata_scsi_device_suspend,
 };
 
 static struct ata_port_operations nv_ops = {
Index: drivers/scsi/sata_promise.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/sata_promise.c  (mode:100644 sha1:19a13e3590f4eca5fab24675ffec12aaa56dd57f)
+++ uncommitted/drivers/scsi/sata_promise.c  (mode:100644)
@@ -103,6 +103,8 @@
 	.slave_configure	= ata_scsi_slave_config,
 	.bios_param		= ata_std_bios_param,
 	.ordered_flush		= 1,
+	.resume			= ata_scsi_device_resume,
+	.suspend		= ata_scsi_device_suspend,
 };
 
 static struct ata_port_operations pdc_ata_ops = {
@@ -178,6 +180,8 @@
 	.id_table		= pdc_ata_pci_tbl,
 	.probe			= pdc_ata_init_one,
 	.remove			= ata_pci_remove_one,
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
 };
 
 
Index: drivers/scsi/sata_qstor.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/sata_qstor.c  (mode:100644 sha1:dfd3621047171dc748ba9a11c82a418f89faaed6)
+++ uncommitted/drivers/scsi/sata_qstor.c  (mode:100644)
@@ -140,6 +140,8 @@
 	.dma_boundary		= QS_DMA_BOUNDARY,
 	.slave_configure	= ata_scsi_slave_config,
 	.bios_param		= ata_std_bios_param,
+	.resume			= ata_scsi_device_resume,
+	.suspend		= ata_scsi_device_suspend,
 };
 
 static struct ata_port_operations qs_ata_ops = {
@@ -191,6 +193,8 @@
 	.id_table		= qs_ata_pci_tbl,
 	.probe			= qs_ata_init_one,
 	.remove			= ata_pci_remove_one,
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
 };
 
 static int qs_check_atapi_dma(struct ata_queued_cmd *qc)
Index: drivers/scsi/sata_sil.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/sata_sil.c  (mode:100644 sha1:f0489dc302a03054d6461ba54c9705f30190a8f4)
+++ uncommitted/drivers/scsi/sata_sil.c  (mode:100644)
@@ -115,6 +115,8 @@
 	.id_table		= sil_pci_tbl,
 	.probe			= sil_init_one,
 	.remove			= ata_pci_remove_one,
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
 };
 
 static Scsi_Host_Template sil_sht = {
@@ -135,6 +137,8 @@
 	.slave_configure	= ata_scsi_slave_config,
 	.bios_param		= ata_std_bios_param,
 	.ordered_flush		= 1,
+	.resume			= ata_scsi_device_resume,
+	.suspend		= ata_scsi_device_suspend,
 };
 
 static struct ata_port_operations sil_ops = {
Index: drivers/scsi/sata_sis.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/sata_sis.c  (mode:100644 sha1:5105ddd08447807df3710492f70c31bf6f0d8767)
+++ uncommitted/drivers/scsi/sata_sis.c  (mode:100644)
@@ -71,6 +71,8 @@
 	.id_table		= sis_pci_tbl,
 	.probe			= sis_init_one,
 	.remove			= ata_pci_remove_one,
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
 };
 
 static Scsi_Host_Template sis_sht = {
@@ -91,6 +93,8 @@
 	.slave_configure	= ata_scsi_slave_config,
 	.bios_param		= ata_std_bios_param,
 	.ordered_flush		= 1,
+	.resume			= ata_scsi_device_resume,
+	.suspend		= ata_scsi_device_suspend,
 };
 
 static struct ata_port_operations sis_ops = {
Index: drivers/scsi/sata_svw.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/sata_svw.c  (mode:100644 sha1:8d1a5d25c05364e41e7f98198b94335ed30d0046)
+++ uncommitted/drivers/scsi/sata_svw.c  (mode:100644)
@@ -289,6 +289,8 @@
 #endif
 	.bios_param		= ata_std_bios_param,
 	.ordered_flush		= 1,
+	.resume			= ata_scsi_device_resume,
+	.suspend		= ata_scsi_device_suspend,
 };
 
 
@@ -458,6 +460,8 @@
 	.id_table		= k2_sata_pci_tbl,
 	.probe			= k2_sata_init_one,
 	.remove			= ata_pci_remove_one,
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
 };
 
 
Index: drivers/scsi/sata_sx4.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/sata_sx4.c  (mode:100644 sha1:70118650c461e9d5d10ff0c3053ae5b49c6755a0)
+++ uncommitted/drivers/scsi/sata_sx4.c  (mode:100644)
@@ -189,6 +189,8 @@
 	.slave_configure	= ata_scsi_slave_config,
 	.bios_param		= ata_std_bios_param,
 	.ordered_flush		= 1,
+	.resume			= ata_scsi_device_resume,
+	.suspend		= ata_scsi_device_suspend,
 };
 
 static struct ata_port_operations pdc_20621_ops = {
@@ -235,6 +237,8 @@
 	.id_table		= pdc_sata_pci_tbl,
 	.probe			= pdc_sata_init_one,
 	.remove			= ata_pci_remove_one,
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
 };
 
 
Index: drivers/scsi/sata_uli.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/sata_uli.c  (mode:100644 sha1:0bff4f475f262c5c40254aecf85bd6a7137bd04e)
+++ uncommitted/drivers/scsi/sata_uli.c  (mode:100644)
@@ -63,6 +63,8 @@
 	.id_table		= uli_pci_tbl,
 	.probe			= uli_init_one,
 	.remove			= ata_pci_remove_one,
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
 };
 
 static Scsi_Host_Template uli_sht = {
@@ -83,6 +85,8 @@
 	.slave_configure	= ata_scsi_slave_config,
 	.bios_param		= ata_std_bios_param,
 	.ordered_flush		= 1,
+	.resume			= ata_scsi_device_resume,
+	.suspend		= ata_scsi_device_suspend,
 };
 
 static struct ata_port_operations uli_ops = {
Index: drivers/scsi/sata_via.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/sata_via.c  (mode:100644 sha1:3a7830667277f35f71a83db5705b8cb2d376bafc)
+++ uncommitted/drivers/scsi/sata_via.c  (mode:100644)
@@ -83,6 +83,8 @@
 	.id_table		= svia_pci_tbl,
 	.probe			= svia_init_one,
 	.remove			= ata_pci_remove_one,
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
 };
 
 static Scsi_Host_Template svia_sht = {
@@ -103,6 +105,8 @@
 	.slave_configure	= ata_scsi_slave_config,
 	.bios_param		= ata_std_bios_param,
 	.ordered_flush		= 1,
+	.resume			= ata_scsi_device_resume,
+	.suspend		= ata_scsi_device_suspend,
 };
 
 static struct ata_port_operations svia_sata_ops = {
Index: drivers/scsi/sata_vsc.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/sata_vsc.c  (mode:100644 sha1:2c28f0ad73c204420aa4b419a60341a8b801d7fa)
+++ uncommitted/drivers/scsi/sata_vsc.c  (mode:100644)
@@ -206,6 +206,8 @@
 	.slave_configure	= ata_scsi_slave_config,
 	.bios_param		= ata_std_bios_param,
 	.ordered_flush		= 1,
+	.resume			= ata_scsi_device_resume,
+	.suspend		= ata_scsi_device_suspend,
 };
 
 
@@ -382,6 +384,8 @@
 	.id_table		= vsc_sata_pci_tbl,
 	.probe			= vsc_sata_init_one,
 	.remove			= ata_pci_remove_one,
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
 };
 
 
Index: drivers/scsi/scsi_lib.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/scsi_lib.c  (mode:100644 sha1:d18da21c9c57e0ef590369ea815b7b1adba4042b)
+++ uncommitted/drivers/scsi/scsi_lib.c  (mode:100644)
@@ -1855,8 +1855,9 @@
 void
 scsi_device_resume(struct scsi_device *sdev)
 {
-	if(scsi_device_set_state(sdev, SDEV_RUNNING))
+	if (scsi_device_set_state(sdev, SDEV_RUNNING))
 		return;
+
 	scsi_run_queue(sdev->request_queue);
 }
 EXPORT_SYMBOL(scsi_device_resume);
Index: drivers/scsi/scsi_sysfs.c
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/drivers/scsi/scsi_sysfs.c  (mode:100644 sha1:e75ee4671ee3a0a6dcb608a73d6bd2f4007eff09)
+++ uncommitted/drivers/scsi/scsi_sysfs.c  (mode:100644)
@@ -199,9 +199,40 @@
 	return (sdp->inq_periph_qual == SCSI_INQ_PQ_CON)? 1: 0;
 }
 
+static int scsi_bus_suspend(struct device * dev, pm_message_t state)
+{
+	struct scsi_device *sdev = to_scsi_device(dev);
+	struct scsi_host_template *sht = sdev->host->hostt;
+	int err;
+
+	err = scsi_device_quiesce(sdev);
+	if (err)
+		return err;
+
+	if (sht->suspend)
+		err = sht->suspend(sdev);
+
+	return err;
+}
+
+static int scsi_bus_resume(struct device * dev)
+{
+	struct scsi_device *sdev = to_scsi_device(dev);
+	struct scsi_host_template *sht = sdev->host->hostt;
+	int err = 0;
+
+	if (sht->resume)
+		err = sht->resume(sdev);
+
+	scsi_device_resume(sdev);
+	return err;
+}
+
 struct bus_type scsi_bus_type = {
         .name		= "scsi",
         .match		= scsi_bus_match,
+	.suspend	= scsi_bus_suspend,
+	.resume		= scsi_bus_resume,
 };
 
 int scsi_sysfs_register(void)
Index: include/linux/ata.h
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/include/linux/ata.h  (mode:100644 sha1:f178894edd04f954a4f650661175b5fe009011c6)
+++ uncommitted/include/linux/ata.h  (mode:100644)
@@ -125,6 +125,8 @@
 	ATA_CMD_PACKET		= 0xA0,
 	ATA_CMD_VERIFY		= 0x40,
 	ATA_CMD_VERIFY_EXT	= 0x42,
+	ATA_CMD_STANDBYNOW1	= 0xE0,
+	ATA_CMD_IDLEIMMEDIATE	= 0xE1,
 
 	/* SETFEATURES stuff */
 	SETFEATURES_XFER	= 0x03,
Index: include/linux/libata.h
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/include/linux/libata.h  (mode:100644 sha1:505160ab472b0198ec5393380ce7c21b2fad85d1)
+++ uncommitted/include/linux/libata.h  (mode:100644)
@@ -113,6 +113,7 @@
 	ATA_FLAG_MMIO		= (1 << 6), /* use MMIO, not PIO */
 	ATA_FLAG_SATA_RESET	= (1 << 7), /* use COMRESET */
 	ATA_FLAG_PIO_DMA	= (1 << 8), /* PIO cmds via DMA */
+	ATA_FLAG_SUSPENDED	= (1 << 9), /* port is suspended */
 
 	ATA_QCFLAG_ACTIVE	= (1 << 1), /* cmd not yet ack'd to scsi lyer */
 	ATA_QCFLAG_SG		= (1 << 3), /* have s/g table? */
@@ -387,6 +388,8 @@
 extern int ata_pci_init_one (struct pci_dev *pdev, struct ata_port_info **port_info,
 			     unsigned int n_ports);
 extern void ata_pci_remove_one (struct pci_dev *pdev);
+extern int ata_pci_device_suspend(struct pci_dev *pdev, pm_message_t state);
+extern int ata_pci_device_resume(struct pci_dev *pdev);
 #endif /* CONFIG_PCI */
 extern int ata_device_add(struct ata_probe_ent *ent);
 extern int ata_scsi_detect(Scsi_Host_Template *sht);
@@ -395,6 +398,10 @@
 extern int ata_scsi_error(struct Scsi_Host *host);
 extern int ata_scsi_release(struct Scsi_Host *host);
 extern unsigned int ata_host_intr(struct ata_port *ap, struct ata_queued_cmd *qc);
+extern int ata_scsi_device_resume(struct scsi_device *);
+extern int ata_scsi_device_suspend(struct scsi_device *);
+extern int ata_device_resume(struct ata_port *, struct ata_device *);
+extern int ata_device_suspend(struct ata_port *, struct ata_device *);
 /*
  * Default driver ops implementations
  */
Index: include/scsi/scsi_host.h
===================================================================
--- b023d524fb0a3b71aa0b957ce7c5540611497370/include/scsi/scsi_host.h  (mode:100644 sha1:1cee1e100943dafe00c14d5329b73659062a4d1a)
+++ uncommitted/include/scsi/scsi_host.h  (mode:100644)
@@ -270,6 +270,12 @@
 	int (*proc_info)(struct Scsi_Host *, char *, char **, off_t, int, int);
 
 	/*
+	 * suspend support
+	 */
+	int (*resume)(struct scsi_device *);
+	int (*suspend)(struct scsi_device *);
+
+	/*
 	 * Name of proc directory
 	 */
 	char *proc_name;

Re: [PATCH] libata: device suspend/resume

[James Bottomley]

On Mon, 2005-05-23 at 16:15 -0400, Jeff Garzik wrote:
> Patch is via Mark Lord, 

Um, it is? it looks identical to this one from Jens (down to the
gratuitous style corrections)

http://marc.theaimsgroup.com/?t=111409327100001

However, before anything happens I'd like to get the SCSI pieces
correct.  The thread petered out before a resolution, however, I think
at least disk devices should be doing a synchronize cache on suspend.

James

Re: [PATCH] libata: device suspend/resume

[Jeff Garzik]

On Mon, May 23, 2005 at 03:41:15PM -0500, James Bottomley wrote:
> On Mon, 2005-05-23 at 16:15 -0400, Jeff Garzik wrote:
> > Patch is via Mark Lord, 
> 
> Um, it is? it looks identical to this one from Jens (down to the
> gratuitous style corrections)
> 
> http://marc.theaimsgroup.com/?t=111409327100001

Got it "via" Mark Lord; I didn't know the proper attribution.


> However, before anything happens I'd like to get the SCSI pieces
> correct.

The "anything" I want to happen here is getting the SCSI pieces correct ;-)



> The thread petered out before a resolution, however, I think
> at least disk devices should be doing a synchronize cache on suspend.

Agreed.

	Jeff

Re: [PATCH] libata: device suspend/resume

[James Bottomley]

On Mon, 2005-05-23 at 16:45 -0400, Jeff Garzik wrote:
> On Mon, May 23, 2005 at 03:41:15PM -0500, James Bottomley wrote:
> > The thread petered out before a resolution, however, I think
> > at least disk devices should be doing a synchronize cache on suspend.
> 
> Agreed.

OK, try this as a straw horse.  It plumbs our scsi bus model into
suspend and resume via the driver (which would be the ULD) function.
The only ULD which has an extra function is sd, and all it does is
synchronize the cache if it was writeback.  I believe you translate SYNC
CACHE in libata, so this patch should be sufficient (unless we still
have a problem with ATA devices that lie about their cache types?)

I think this is sufficient, because if the LLD also wants this
information, it can get it from the model of the bus it's attached to.

James

--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -199,10 +200,37 @@ static int scsi_bus_match(struct device 
 	return (sdp->inq_periph_qual == SCSI_INQ_PQ_CON)? 1: 0;
 }
 
+static int scsi_bus_suspend(struct device *dev, pm_message_t state)
+{
+	struct scsi_device *sdev = to_scsi_device(dev);
+	int err = scsi_device_quiesce(sdev);
+
+	if (err)
+		return err;
+
+	if(dev->driver && dev->driver->suspend)
+		return dev->driver->suspend(dev, state, 0);
+	return 0;
+}
+
+static int scsi_bus_resume(struct device *dev)
+{
+	struct scsi_device *sdev = to_scsi_device(dev);
+	int err = 0;
+
+	if(dev->driver && dev->driver->resume)
+		err = dev->driver->resume(dev, 0);
+
+	scsi_device_resume(sdev);
+	return err;
+}
+
 struct bus_type scsi_bus_type = {
         .name		= "scsi",
         .match		= scsi_bus_match,
+	.suspend	= scsi_bus_suspend,
+	.resume		= scsi_bus_resume
 };
 
 int scsi_sysfs_register(void)
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -119,6 +119,7 @@ static void sd_rw_intr(struct scsi_cmnd 
 static int sd_probe(struct device *);
 static int sd_remove(struct device *);
 static void sd_shutdown(struct device *dev);
+static int sd_suspend(struct device *dev, pm_message_t state, u32 level);
 static void sd_rescan(struct device *);
 static int sd_init_command(struct scsi_cmnd *);
 static int sd_issue_flush(struct device *, sector_t *);
@@ -134,6 +135,7 @@ static struct scsi_driver sd_template = 
 		.probe		= sd_probe,
 		.remove		= sd_remove,
 		.shutdown	= sd_shutdown,
+		.suspend	= sd_suspend,
 	},
 	.rescan			= sd_rescan,
 	.init_command		= sd_init_command,
@@ -1710,7 +1712,28 @@ static void sd_shutdown(struct device *d
 	printk(KERN_NOTICE "Synchronizing SCSI cache for disk %s: \n",
 			sdkp->disk->disk_name);
 	sd_sync_cache(sdp);
-}	
+}
+
+/* Just quietly quiesce the device and SYNCHRONIZE CACHE for suspend too */
+static int sd_suspend(struct device *dev, pm_message_t state, u32 level)
+{
+	struct scsi_device *sdp = to_scsi_device(dev);
+	struct scsi_disk *sdkp = dev_get_drvdata(dev);
+
+	if (!sdkp)
+		return 0;         /* this can happen */
+
+	if (!sdkp->WCE)
+		return 0;
+
+	/* don't try to sync an offline device ... it will only error */
+	if (!scsi_device_online(sdp))
+		return 0;
+
+	if (sd_sync_cache(sdp))
+		return -EIO;
+	return 0;
+}
 
 /**
  *	init_sd - entry point for this driver (both when built in or when

Re: [PATCH] libata: device suspend/resume

[Jens Axboe]

On Mon, May 23 2005, James Bottomley wrote:
> On Mon, 2005-05-23 at 16:45 -0400, Jeff Garzik wrote:
> > On Mon, May 23, 2005 at 03:41:15PM -0500, James Bottomley wrote:
> > > The thread petered out before a resolution, however, I think
> > > at least disk devices should be doing a synchronize cache on suspend.
> > 
> > Agreed.
> 
> OK, try this as a straw horse.  It plumbs our scsi bus model into
> suspend and resume via the driver (which would be the ULD) function.
> The only ULD which has an extra function is sd, and all it does is
> synchronize the cache if it was writeback.  I believe you translate SYNC
> CACHE in libata, so this patch should be sufficient (unless we still
> have a problem with ATA devices that lie about their cache types?)
> 
> I think this is sufficient, because if the LLD also wants this
> information, it can get it from the model of the bus it's attached to.

suspend/resume is a lot more complicated than just flushing a cache, the
below will probably get you safe asleep but you will never get devices
alive again after power-up on suspend-to-ram.

I also greatly prefer issuing a standby command to the drive after the
flush, so that we don't risk using the emergency parks of the drive. If
a drive happens to lie about wrt the flush command, it gets an extra
chance to flush the cache as it now knows that power will be gone very
soon. So I think the ->suspend/->resume hooks should belong to the LLD,
not the ULD as the ULD has no idea how to suspend all devices types.


-- 
Jens Axboe

Re: [PATCH] libata: device suspend/resume

[Jeff Garzik]

Jens Axboe wrote:
> suspend/resume is a lot more complicated than just flushing a cache, the
> below will probably get you safe asleep but you will never get devices
> alive again after power-up on suspend-to-ram.
> 
> I also greatly prefer issuing a standby command to the drive after the
> flush, so that we don't risk using the emergency parks of the drive. If
> a drive happens to lie about wrt the flush command, it gets an extra
> chance to flush the cache as it now knows that power will be gone very
> soon. So I think the ->suspend/->resume hooks should belong to the LLD,
> not the ULD as the ULD has no idea how to suspend all devices types.


You are right about issuing the standby command after flush.  I don't 
think an LLD hook is the proper way to accomplish it, however.

The SCSI layer needs to issue the START STOP UNIT command in response to 
a suspend event, and libata-scsi will (per SAT spec) translate that into 
the ATA standby command.  Merely following the relevant SCSI+SAT+ATA 
standards gets us there.

Longer term, SATA PM through SCSI will have three facets:

* Device PM.  This is best handled by the device class driver (sd/sr/st).

* Bus PM.  This is best handled by the transport class driver (need to 
write for SATA and SAS).

* Host PM.  This is handled in the obvious manner, using existing PM 
driver hooks.  PCI D0/D3, etc.

I can describe how this will look when libata is divorced from SCSI, if 
you would like, too...

	Jeff

Re: [PATCH] libata: device suspend/resume

[Jens Axboe]

On Tue, May 24 2005, Jeff Garzik wrote:
> Jens Axboe wrote:
> >suspend/resume is a lot more complicated than just flushing a cache, the
> >below will probably get you safe asleep but you will never get devices
> >alive again after power-up on suspend-to-ram.
> >
> >I also greatly prefer issuing a standby command to the drive after the
> >flush, so that we don't risk using the emergency parks of the drive. If
> >a drive happens to lie about wrt the flush command, it gets an extra
> >chance to flush the cache as it now knows that power will be gone very
> >soon. So I think the ->suspend/->resume hooks should belong to the LLD,
> >not the ULD as the ULD has no idea how to suspend all devices types.
> 
> 
> You are right about issuing the standby command after flush.  I don't 
> think an LLD hook is the proper way to accomplish it, however.
> 
> The SCSI layer needs to issue the START STOP UNIT command in response to 
> a suspend event, and libata-scsi will (per SAT spec) translate that into 
> the ATA standby command.  Merely following the relevant SCSI+SAT+ATA 
> standards gets us there.

That sounds fine!

> Longer term, SATA PM through SCSI will have three facets:
> 
> * Device PM.  This is best handled by the device class driver (sd/sr/st).
> 
> * Bus PM.  This is best handled by the transport class driver (need to 
> write for SATA and SAS).
> 
> * Host PM.  This is handled in the obvious manner, using existing PM 
> driver hooks.  PCI D0/D3, etc.
> 
> I can describe how this will look when libata is divorced from SCSI, if 
> you would like, too...

I was beginning to dispair you had given up that plan...

-- 
Jens Axboe

Re: [PATCH] libata: device suspend/resume

[Jeff Garzik]

Jens Axboe wrote:
> On Tue, May 24 2005, Jeff Garzik wrote:
>>I can describe how this will look when libata is divorced from SCSI, if 
>>you would like, too...
> 
> I was beginning to dispair you had given up that plan...

hehe, nope.  I promised Linus, and I plan to keep my promise :)

I know how to do it.  Internally things have been kept as separate as 
possible from the SCSI layer.

Bart even hinted at possibly using libata-without-SCSI for future 
/dev/hdX support.  It's certainly doable.

	Jeff

Re: [PATCH] libata: device suspend/resume

[Jens Axboe]

On Tue, May 24 2005, Jeff Garzik wrote:
> Jens Axboe wrote:
> >On Tue, May 24 2005, Jeff Garzik wrote:
> >>I can describe how this will look when libata is divorced from SCSI, if 
> >>you would like, too...
> >
> >I was beginning to dispair you had given up that plan...
> 
> hehe, nope.  I promised Linus, and I plan to keep my promise :)

You promised me, too :)

> I know how to do it.  Internally things have been kept as separate as 
> possible from the SCSI layer.

We should start a list of items that could potentially be moved to the
block layer that libata currently uses.
> 
> Bart even hinted at possibly using libata-without-SCSI for future 
> /dev/hdX support.  It's certainly doable.

>From my point of view, I think it would be a good way to kick off
/dev/disk device agnostic mappings.

-- 
Jens Axboe

libata, SCSI and storage drivers

[Jeff Garzik]

Jens Axboe wrote:
> On Tue, May 24 2005, Jeff Garzik wrote:
> 
>>Jens Axboe wrote:
>>
>>>On Tue, May 24 2005, Jeff Garzik wrote:
>>>
>>>>I can describe how this will look when libata is divorced from SCSI, if 
>>>>you would like, too...
>>>
>>>I was beginning to dispair you had given up that plan...
>>
>>hehe, nope.  I promised Linus, and I plan to keep my promise :)
> 
> 
> You promised me, too :)
> 
> 
>>I know how to do it.  Internally things have been kept as separate as 
>>possible from the SCSI layer.
> 
> 
> We should start a list of items that could potentially be moved to the
> block layer that libata currently uses.


Here are the two broad categories of things that immediately come to mind.


1.  Hardware in pre-production right now can do SAS or SATA on the same 
card.  So, real soon, a driver will need to do both SCSI and ATA 
depending on runtime conditions.

The SCSI transport class is a very nice way to connect low-level drivers 
and the class drivers (disk/cdrom/tape/...).  It works well with the 
device model, and is modular in just the right location.

I would like to develop ATA transport class(es).  In order to work well 
with SATA/SAS hardware, there will need to be at least one.

And as I hoped you have guessed.....  the ATA transport class should be 
a child of the block layer, not the SCSI layer.


2.  driver API.  Linux SCSI layer provides several services which are 
generalized to any "packet transport":
* mapping of devices to protocol buses (domain topology, etc.)
* command queueing
* error handling

For hardware like ATA or I2O, this "send-command" type of API is the 
most natural way to implement a low-level driver, particularly if the 
core code provides all the necessary queueing/mapping/EH services as well.

This infrastructure is -not- specific to SCSI at all.  And it is this 
infrastructure that allowed me to bring up libata so rapidly.

	Jeff

Re: libata, SCSI and storage drivers

[Douglas Gilbert]

Jeff Garzik wrote:
> Jens Axboe wrote:
> 
>> On Tue, May 24 2005, Jeff Garzik wrote:
>>
>>> Jens Axboe wrote:
>>>
>>>> On Tue, May 24 2005, Jeff Garzik wrote:
>>>>
>>>>> I can describe how this will look when libata is divorced from 
>>>>> SCSI, if you would like, too...
>>>>
>>>>
>>>> I was beginning to dispair you had given up that plan...
>>>
>>>
>>> hehe, nope.  I promised Linus, and I plan to keep my promise :)
>>
>>
>>
>> You promised me, too :)
>>
>>
>>> I know how to do it.  Internally things have been kept as separate as 
>>> possible from the SCSI layer.
>>
>>
>>
>> We should start a list of items that could potentially be moved to the
>> block layer that libata currently uses.
> 
> 
> 
> Here are the two broad categories of things that immediately come to mind.
> 
> 
> 1.  Hardware in pre-production right now can do SAS or SATA on the same 
> card.  So, real soon, a driver will need to do both SCSI and ATA 
> depending on runtime conditions.
> 
> The SCSI transport class is a very nice way to connect low-level drivers 
> and the class drivers (disk/cdrom/tape/...).  It works well with the 
> device model, and is modular in just the right location.
> 
> I would like to develop ATA transport class(es).  In order to work well 
> with SATA/SAS hardware, there will need to be at least one.
> 
> And as I hoped you have guessed.....  the ATA transport class should be 
> a child of the block layer, not the SCSI layer.

Jeff,
Yes, the SAS HBAs that I'm aware of can connect to
SATA I/II disks "directly" on their phys.

So here are two scenarios for connecting SATA disks:

a)
  SAS HBA < -------------------------> SATA disk

b)
  SAS HBA <------>SAS_expander<------> SATA disk

In a) the interconnect is SATA. Still it is hard
to believe that the SAS HBA LLD would belong to
anything other than the SCSI subsystem (since
SAS HBAs come with 4 or 8 phys, others of which could
be in scenario b) and/or connected to SAS disks).
Hence the initiator_ports/phys/target_ports on and
seen by that SAS HBA should (also) have SAS transport
classes.

When a SAS HBA phy is not connected to anything is it
a member of a SAS transport class or a ATA transport
class (or both)?

In scenario b) the left interconnect is SAS and the
right interconnect is SATA. The SAS_expander contains
the STP<->SATA bridging function (and, for sake of
argument, no SCSI-ATA Translation (SAT) layer).
Would scenario b) also have a ATA transport
class? I'll assume it does. To be discovered it also
needs a SAS transport class. Larger enclosures are
likely to be amplifications of scenario b). The presence
of the SATA disk in scenario b) will be discovered via
the SAS SMP (i.e. talking to the SAS_expander) or via
the SES protocol (i.e. a SCSI Enclosure Services target
running near the SAS_expander). Either way if there are
a lot of SATA disks then they are likely to be held
in their initialization sequence to stop them spinning
up all at once. SAS transport intervention may be
required (staggered timers are another possibility).

Now I may be wrong but I think that one of the SAS HBAs
that I have read about that looks (externally) like
scenario a) but is actually scenario b). In other words
the SAS_expander is silicon on the HBA and it is not
controlled via the PCI bus, but rather by SMP.

This suggests to me we would need an ordered sequence of
transport classes. I really wonder about trying to model
this level of complexity in sysfs, plus the nightmare of
keeping state data of (topologically) distant nodes up
to date. At some point one needs to supply a management
pass through and hand the problem over to the user
space. The question is, at what point.


There are already FCP enclosures filled with SATA disks
on the market so this problem in not unique to SAS.
However they have (I presume) a SAT layer in their
FC enclosures so the OS thinks it is taking to SCSI
disks.

Doug Gilbert

Re: libata, SCSI and storage drivers

[Luben Tuikov]

On 05/27/05 02:45, Douglas Gilbert wrote:
> In a) the interconnect is SATA. Still it is hard
> to believe that the SAS HBA LLD would belong to
> anything other than the SCSI subsystem (since
> SAS HBAs come with 4 or 8 phys, others of which could
> be in scenario b) and/or connected to SAS disks).
> Hence the initiator_ports/phys/target_ports on and
> seen by that SAS HBA should (also) have SAS transport
> classes.

It should register with the SAS transport class, where
the SAS discover "function" is.  The Discover function
would find out what is out on the domain and "register"
it with the proper "transport" (a more correct way to
put it there is).

For SSP devices that would be a "scsi disk", for
SATA devices that would be a "libata disk" or something
like this.  The LLDD would either get a CDB for SSP,
or taskfile for STP, and pass it along to the hardware.
 
> When a SAS HBA phy is not connected to anything is it
> a member of a SAS transport class or a ATA transport
> class (or both)?

Quite possibly both: SAS transport class, since it is
a super class in meaning that the Discover function
is in there.
 
> In scenario b) the left interconnect is SAS and the
> right interconnect is SATA. The SAS_expander contains
> the STP<->SATA bridging function (and, for sake of
> argument, no SCSI-ATA Translation (SAT) layer).
> Would scenario b) also have a ATA transport
> class? I'll assume it does. To be discovered it also

Yes it would.  As far as the class is concerned it
is a SATA device -- how you get to it (direct or through
an expander) is the LLDD/firmware/Host Adapter concern.

> needs a SAS transport class. Larger enclosures are
> likely to be amplifications of scenario b). The presence
> of the SATA disk in scenario b) will be discovered via
> the SAS SMP (i.e. talking to the SAS_expander) or via
> the SES protocol (i.e. a SCSI Enclosure Services target
> running near the SAS_expander). Either way if there are
> a lot of SATA disks then they are likely to be held
> in their initialization sequence to stop them spinning
> up all at once. SAS transport intervention may be
> required (staggered timers are another possibility).

Yes, spinup-hold states would be managed by the Discover
function, or by the FW on the expander/ses device.

BTW, very nice to mention SES devices.  We'll have to
bring that up more as they'd be very much en vogue
with expanders.

> Now I may be wrong but I think that one of the SAS HBAs
> that I have read about that looks (externally) like
> scenario a) but is actually scenario b). In other words
> the SAS_expander is silicon on the HBA and it is not
> controlled via the PCI bus, but rather by SMP.

I know of such a SAS HA (Host Adapter), and from what it
seems the role of the expander-on-chip is to simulate
a larger number of phys than actually/physically supported
by the HA.  By design, this should be transparent to the
Discover function, and if it is not, I believe the Discover
fn. would do the right thing (i.e. treat is as an expander).

> This suggests to me we would need an ordered sequence of
> transport classes. I really wonder about trying to model
> this level of complexity in sysfs, plus the nightmare of
> keeping state data of (topologically) distant nodes up
> to date. At some point one needs to supply a management

When a SSP talking device is found it is registered
as a "scsi device".  When a STP talking device is
discovered (directly attached or elsewhere) it would
be registered as a "libata device".

> pass through and hand the problem over to the user
> space. The question is, at what point.

I'm not 100% sure about user space.  The Discover function
is pretty straightforward, and simple.  Plus the root/bootable
device could be a SATA device on an expander or even deeper
in the domain.

Thanks,
	Luben