Re: Scheduled Transfer Protocol on Linux

[Aman Singla <aman@cthulhu.engr.sgi.com>]

I think Larry's description (below) about how STP works is pretty
much accurate modulo 'various levels of details / mechanisms' but
hey, thats what protocol spec's are for :-)
http://www.hippi.org/cST.html

The STP Long Message Transfer stuff is realized to be useful for
network enabled storage - may it be through higher performance
solutions in the upper layer - NFS/BDS etc., or via the lower layers
like SCSI over STP.

What I'd like to add is, that STP also specifies persistent (longer
duration - spanning multiple messages) mappings and pinning down of
memory which is useful for low latency communications a'la clustering
applications. Again, what STP offers here is an open, media-independent
and scalable protocol.

Are there any other applications that people can forsee?

thanks,

:a

> What it does do is give you a high bandwidth, low CPU cycle way of moving
> the data.
> 
> The simplified view of STP is really just remote DMA.  How it works, again
> greatly simplified, is sort of like this:
> 
>         client                                  server
>         request to send 1GB ->
> 
>                         <- please wait while I set up some pages (optional)
> 
>                         <- clear to send first 100MB of data, here's a cookie
> 
>         data on wire, prefixed with incrementing cookie ->
>         data on wire, prefixed with incrementing cookie ->
>         data on wire, prefixed with incrementing cookie ->
>         data on wire, prefixed with incrementing cookie ->
>         data on wire, prefixed with incrementing cookie ->
>         data on wire, prefixed with incrementing cookie ->
>         data on wire, prefixed with incrementing cookie ->
>         data on wire, prefixed with incrementing cookie ->
>         data on wire, prefixed with incrementing cookie ->
> 
>                         -> data hits NIC, cockie is index into CAM, exactly
>                            like an ethernet switch indexes MAC addresses,
>                            except that the value here is a physical page
>                            address instead of an outgoing port
> 
>                         -> data hits NIC ....
>                         -> data hits NIC ....
>                         -> data hits NIC ....
>                         -> data hits NIC ....
>                         -> data hits NIC ....
>                         -> data hits NIC .... we're at 75MB or so
> 
>                            server goes off and locks down another 100MB
> 
>                         <- CTS another 100MB (this happens well before the
>                            first 100MB is drained, so there are no bubbles,
>                            the pipe is 100% full at all times)
> 
> I trust that the SGI guys, who know this stuff better than I, will correct
> and problems with this description.  I'm sure there are some, but I'm also
> sure this is pretty close to how it works modulo flow control, security,
> and other important bits.
> 
> What's important is that you notice two things:
> 
>     (a) this is what I call "polite networking".  Normal networking is
>         impolite, the packets show up on your doorstep uninvited.  Here,
>         the receiver is nicely asked if it is OK, and gets to set things
>         up first.  Just like in DMA.
> 
>     (b) There is some hardware on the NIC that translates from packets to
>         physical page addresses, this happens inline, so the packet gets
>         DMAed into memory as it is coming off the wire.  Very low buffering
>         requirements.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/