Re: TOE brain dump

[Werner Almesberger <werner@almesberger.net>]

David Lang wrote:
> exactly, Alan is saying that the hardware optimizations aren't nessasary.

Eventually, you'll want them, and if it's only to lower the
chip or pin count.

> putting an Opteron on a NIC card just to match the other processors in
> your system seems like a huge amount of overkill. you aren't going to have
> nearly the same access to memory so that processor will be crippled, but
> stil cost full price

You might be able to get them for free ;-) Just pick the
rejects where the FPU or such doesn't quite work. Call it
amd64sx :-)

But even if you get regular CPUs, they're not *that*
expensive. Particularly not for a first generation design.

> (and then some, remember you have to supply the thing
> with power and cool it)

Yes, this, chip count, and chip surface are what make me feel
queasy when thinking of somebody using something as powerful
as an amd64.

> as long as tools are written that have the same command line semantics the
> rest of the complexity can be hidden.

You want to be API and probably even ABI-compatible, so that
user-space demons (routing, management, etc.) work, too.

> and even this isn't strictly
> nessasary, these are special purpose cards and a special procedure for
> configuring them isn't unreasonable.

I'd think thrice before buying a card that requires me to
change my entire network management system - and change it
again, if I ever decide to switch brands, or if the next
generation of that special NIC gets a little more special.

> I'm saying treat the one machine with 10 of these specialty NIC's in it as
> a 11 machine cluster, one machne running your server software and 10
> others running your networking.

You can probably afford rather fancy TOE hardware for the
price of ten cluster nodes, a high-speed LAN to connect
your cluster, and a switch that connects the high-speed
link to the ten not-quite-so-high-speed links.

Likewise for power, cooling, and space.

And that's still assuming you can actually distribute all
this.

- Werner

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina     werner@almesberger.net /
/_http://www.almesberger.net/____________________________________________/