Re: POHMELFS high performance network filesystem. Transactions, failover, performance.

[Jamie Lokier <jamie@shareable.org>]

Evgeniy Polyakov wrote:
> > Look up Bittorrent, and bandwidth diffusion generally.  Also look up
> > multicast trees.
> > 
> > Sometimes it's faster for a client to send to many servers; sometimes
> > it's faster to send fewer and have them relayed by intermediaries -
> > because every packet takes time to transmit, and network topologies
> > aren't always homogenous or symmetric.
> > 
> > There is no simple answer which is optimal for all networks.
> 
> Yep, having multiple connections is worse for high-performance networks
> and is a great win for long latency links.

Not just long latency.  If you have a low latency link which is very
busy, perhaps a client doing lots of requests, or doing other things,
that pushes up the _effective_ latency.

> > If you have a single data forwarder elected per client, then if one
> > client generates a lot of traffic, you concentrate a lot of traffic to
> > one network link and one CPU.  Sometimes it's better to elect several
> > leaders per client, and hash requests onto them.  You diffuse CPU and
> > traffic, but reduce opportunities to aggregate transactions into fewer
> > message.  It's an interesting problem, again probably with different
> > optimal results for different networks.
> 
> Probably idea I described in other mail to Jeff, when client just
> connects to number of servers and can process command of adding/dropping
> server from that group, and balances reading between them and sends
> writes/metadata update to all of them, and all logic behind that group
> selection is hidded in the servers cloud, is the best choice...

I think that's a fine choice, but it doesn't solve difficult
problems.  You still have to implement the server cloud. :-)

It's possible that implementing server cloud protocol _and_ simple
client protocol may be more work than just server cloud protocol.  I'm
not sure.  Thoughts welcome.

-- Jamie