Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

[Stephen Frost <sfrost@snowman.net>]

[-- Attachment #1: Type: text/plain, Size: 2153 bytes --]

* Ingo Molnar (mingo@elte.hu) wrote:
> 
> On Tue, 9 Jan 2001, Stephen C. Tweedie wrote:
> 
> > but it just doesn't apply when you look at some other applications,
> > such as streaming out video data or performing fileserving in a
> > high-performance compute cluster where you are serving bulk data.
> > The multimedia and HPC worlds typically operate on datasets which are
> > far too large to cache, so you want to keep them in memory as little
> > as possible when you ship them over the wire.
> 
> i'd love to first see these kinds of applications (under Linux) before
> designing for them. Eg. if an IO operation (eg. streaming video webcast)
> does a DMA from a camera card to an outgoing networking card, would it be
> possible to access the packet data in case of a TCP retransmit? Basically
> these applications are limited enough in scope to justify even temporary
> 'hacks' that enable them - and once we *see* things in action, we could
> design for them. Not the other way around.

	Well, I know I for one use a system that you might have heard
of called 'MOSIX'.  It's a (kinda large) kernel patch with some user-space
tools but allows for migration of processes between machines without
modifying any code.  There are some limitations (threaded applications and
shared memory and whatnot) but it works very well for the rendering work
we use it for.  We use radiance which in general has pretty little inter-
process communication and what it has is done through the filesystem.

	Now, the interesting bit here is that the processes can grow to be
pretty large (200M+, up as high as 500M, higher if we let it ;) ) and what
happens with MOSIX is that entire processes get sent over the wire to 
other machines for work.  MOSIX will also attempt to rebalance the load on
all of the machines in the cluster and whatnot so it can often be moving
processes back and forth.

	So, anyhow, this is just an fyi if you weren't aware of it that I
believe more than a few people are using MOSIX these days for similar
appliactions and that it's availible at http://www.mosix.org if you're
curious.

		Stephen

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]