Supporting SATA Target mode (or any data transfer w/crossover cable) on AHCI

Supporting SATA Target mode (or any data transfer w/crossover cable) on AHCI

[Richard Aplin]

Dear wise people,

I'm interested in trying to get generic mobo SATA controllers to run
in Target mode; there are a few entertaining projects that become
possible if this can be done.

Two obvious uses are SATA drive emulation and ultra-high-speed
motherboard interconnect using bonded SATA3 channels.

I'm interested in getting a controller to fully emulate a real drive,
but any form of bidirectional data transfer over SATA would be good.
An obvious "hello world" is a working loopback onto another port.
Obviously you need a crossover cable for all this stuff, but I can
hack one together for now.

Target mode has come up on this list a few times over the years and
there are a smattering of comments in the libata Marvell driver about
it (see references)

Also I understand that the SCST project is the natural place for this
to live as they have extensive support for drive emulation - but their
hardware target support appears to be entirely SCSI oriented.

I know there is a Marvell driver with some code in (see footnotes),
but that's all I have so far.

Ok so, questions.

=1= Does anyone have experience of doing this with SATA that they'd
like to share? Seems like fertile ground.

Mark Lord said on this list back in 2007 "Everybody I've dealt with
thus far uses it as a high-speed local comms interface,which would
suggest that it might be done as a network interface (ethernet
emulation)."
http://www.mail-archive.com/linux-ide@vger.kernel.org/msg12342.html


=2= Suitability for AHCI devices -
I've been reading the AHCI and SATA specs but am not deep enough yet
to see if there is any hope of doing this with generic AHCI
controllers (which seems like best choice nowadays).  Jeff G said
"some ahci chips" support this (see refs). Would love to know which
ones, and how this is done.
 These would be vendor-specific registers or does AHCI 1.x allow for
it in any way?

=3= In the Libata Marvell driver appears
"[Experiment, Marvell value added] Is it possible to use target mode
to cross-connect two Linux boxes with Marvell cards? If so, creating
LibATA target mode support would be very interesting."
http://tomoyo.sourceforge.jp/cgi-bin/lxr/source/drivers/ata/sata_mv.c

Yes, it would! ;-)

Back In The Day of simpler hardware you'd have been almost certainly
able to do this sort of hackery by getting jiggy with the chip
registers, but it appears that AHCI is a (relatively) high-level
register spec where there is significant state-machine (or even CPU)
silicon on the chip-side to implement things. This is good for the
intended purpose (fast, generic interface) but I can imagine it is
less hackable.


I don't see this fitting into libata - I'd expect it to feed into SCST
and possibly a separate IP-over-SATA driver - possibly all that libata
would do is agree to leave one or more SATA ports alone so another
(new) kernel driver can play with them.

I believe there are a number of really neat-o things that could happen
if this can be made to work on commodity chips.

Thanks for any info or discussion on this.  It's great to be able to
ask the true subject experts.
Richard Aplin




==  Research so far ===

-  a kind soul contributed a patched Marvell chipset driver supporting
Target mode (in mvSata.c marked with MV_SATA_C2C_COMM)

-  Jeff G mentioned this elsewhere  ("Cool. I'm looking forward to
figuring out how to enable ATA target mode  in a similar manner...
sata_mv and some ahci chips support ATA target mode." -
http://www.gossamer-threads.com/lists/linux/kernel/943943 )


- There was a brief discussion of the subject back in 2007
http://www.spinics.net/lists/linux-ide/msg16348.html

- a recent question (may2010)
http://www.spinics.net/lists/linux-ide/msg37775.html didn't really go
anyhere

Re: Supporting SATA Target mode (or any data transfer w/crossover cable) on AHCI

[Mark Lord]

On 11-02-26 10:30 PM, Richard Aplin wrote:
> 
> I'm interested in trying to get generic mobo SATA controllers to run
> in Target mode; there are a few entertaining projects that become
> possible if this can be done.
> 
> Two obvious uses are SATA drive emulation and ultra-high-speed
> motherboard interconnect using bonded SATA3 channels.

Nobody here will argue against it -- it's something that could be
quite useful to all -- eg. homebrew SATA analyzers etc..

It does require specific support in the chipset, though.
And since there's no standard for that, this means the hardware
implementations are all vendor-specific, and somewhat "secret".

The Marvel 7042 PCIe chips implement target mode, and it probably
works on the older 60xx (non-AHCI) chips as well.  The "normal" SATA
driver for these is "sata_mv", but it does not implement target mode.

Both Jens and I have written code to attempt simple transfers
using "target mode", but neither effort had enough reverse-engineering
of the Marvell chips to actually make it work.  They don't exactly
spell out all of the details in their (NDA'd) documentation.

The standalone downloadable "proprietary" Marvell SATA driver,
which has "GPL" labelling, claims to to target mode.  But when I was
poking at it 2-3 years ago, the code therein was incomplete,
and appeared to be missing some secret jiggery to make it all work.

That's probably still the best starting point though.

Cheers

Re: Supporting SATA Target mode (or any data transfer w/crossover cable) on AHCI

[Richard Aplin]

>
> It does require specific support in the chipset, though.

..That was my guess from page 93 of the SATA 1 specs
http://www.sata-io.org/documents/serialata10a.zip
..the diagram shows the COMRESET protocol sequence which I notice is
not symmetrical (boo!!) hence if you cross-connect two SATA masters it
looks like you'd (at least) get an error when the slave device fails
to enter 'Device Calibrate' and 'Device COMWAKE' states.  It's a shame
that they didn't design it to sync symmetrically, but then the
protocol gets a bit more complicated and I expect everyone wanted to
be home by teatime.
Looks like we're in a similar situation to the wifi hackers, who have
to find specific chipsets that they can hack to support raw packets,
promiscuous mode, etc.   Doh. Well, anyway, let's see how far we get.
I'm new to this scene, but it appears there aren't _that_ many SATA
chipsets in mass-mass-production; most widespread I am assuming are
the big vendor integrated IO chips; the Intel ICH family, NVidia MCP,
etc.  Somewhere down the popularity list after that you've got the
dedicated silicon from SIL, Marvell, etc.


== Marvell target mode
I don't have one of these controllers but it's a useful discussion
point at least.
I also assume that Marvell reuse basically the same SATA macro block
in all their chips, which seems like a fair assumption nowadays.
Ok I'm looking at the (NDA'd? ;-)  Marvell  "88F6180, 88F6190,
88F6192, and 88F6281 Integrated Controller Functional Specifications"
which google found here
http://www.embeddedarm.com/documentation/third-party/MV88F5182-usermanual.pdf
on page 94 there is some talk about Target, and on page 397 there are
the relevant register bits
On paper it _looks_ pretty straightforwards; you set two bits on the
Target board and presumably it switches the chip to use an alternate
state machine for the handshake protocol so it connects properly with
a master.  However, I don't doubt that Things Are Not That Simple in
real life.
Mark, I assume that when you ran a Marvell board in target mode you
got at least successful connect (i.e. no COMRESET error)

== My experiments
I don't have a Marvell controller;  I have a few others in various
boxes; Intel IC7M, an IC8M, a Silicon Image Sil3531 in a windows box,
but the current victim in the test rig is an Nvidia MCP79 in a cheap
Atom/ION HTPC.
The MCP79 runs in ATA emulation or SATA AHCI mode depending on a BIOS
setting; I'm running it in AHCI mode of course.

-- Off the shelf performance of Nvidia MCP79 SATA (w/1.6Ghz Atom 330,
single channel DDR2-800 RAM)

Kingston 16GB SSD for testing, and using stock Debian 10.10 I
repeatedly get 218MBytes/sec with hdparm, so it's definitely running
at 3GBPS SATA II.

I built a crossover cable - very carefully peeled open a spare
SATA-eSATA mobo cable I had and swapped the tx/rx pairs (retaining
correct +/-)

--Test 1 - mobo loopback via crossover

Then we just plug in a regular eSATA cable to loopback to the mobo's
port, and dmesg spits out...
ata2: exception Emask 0x10 SAct 0x0 SErr 0x4000000 action 0xe frozen
ata2: irq_stat 0x00000040, connection status changed
ata2: SError: { DevExch }
ata2: hard resetting link
ata2: COMRESET failed (errno=-32)
ata2: reset failed (errno=-32), retrying in 8 secs
Basically, "yay" but "boo" at the same time - something happened, but
it's what we thought might happen -  the COMRESET fails. (thanks for
the nice clear error messages there btw)

-- Test 2 - between two machines via crossover

Plugging into two physically separate (windows) machines via eSATA
yields no dmesg output (silence from both ends), although I'm hopeful
that will resolve if I fix up the grounding on my homemade cable - I'd
expect it to yield same error on the linux box as I got with loopback,
if things are electrically adequate.

Ok so things I'll try...

a) See if the chips will _actually_ TX/RX FISs even if the COMRESET
failed. This is wildly optimistic. By the look of things (e.g. Marvell
chip) these controllers typically incorporate a full CPU to do stuff
like connection handshaking (rather than using a minimal state machine
or exposing the registers to the kernel driver).  I am guessing if
this CPU doesn't like the SATA link state, it's going to be a real wet
blanket about sending FISs. However, hope springs eternal.

b) I'm trying to get this to work on the major mobos, so ask Intel and
Nvidia  nicely if they have any ideas.  This "loopback connect" mode
is so useful and so zero-cost to do in the chip it's hard to believe
they didn't squirrel away a bit somewhere that lets you do this.

c) Try randomly banging unknown bits in the AHCI register space of my
MCP79 and see if it affects things

d) Your suggestion here, free as in beer

Cheers,
Rich Aplin

Re: Supporting SATA Target mode (or any data transfer w/crossover cable) on AHCI

[Richard Aplin]

BTW for those who missed it, there was another recent (may 2010)
thread on Marvell Target drivers
http://www.spinics.net/lists/linux-ide/msg37863.html

where Saeed Bishara posted mods to the GPL Marvell drivers (not
libata) for chips such as this...
http://www.marvell.com/products/processors/embedded/kirkwood/FS_88F6180_9x_6281_OpenSource.pdf
and added some form of C2C / Target support

I don't have such a controller but I'm going to see if I can find one
in an expresscard format, for testing purposes.

I'm still after AHCI support for popular mobo chips, but that may be a
pipe dream.

Frustratingly I have this suspicion that Target mode support on those
chips - if not already present - might be possible by updating these
chip's firmware. I am guessing this is in flash or bios-loaded SRAM
nowadays. I also guess this avenue will lead nowhere. ;-)

Cheers,
Rich.