Fwd: load balancing

["Miguel Figueiredo Mascarenhas Sousa Filipe" <miguel.filipe@xxxxxxxxx>]

Hi,


On 10/31/06, Joerg Sonnenberger <joerg@xxxxxxxxxxxxxxxxx> wrote:
>  On Tue, Oct 31, 2006 at 12:24:05AM +0100, Simon 'corecode' Schubert wrote:
> > Joerg Sonnenberger wrote:
> > >The problem with proxy architectures is of course the second copying,
> > >esp. kernel -> userland -> kernel, which can put a bunch of additional
> > >load on frontend machines. Using approaches similar to state-full
> > >firewall handling e.g. of FTP can be used for this as well, e.g. make it
> > >a transparent proxy. This is a lot more work though. I'm not aware of
> > >any such Open Source solution though. It shouldn't be hard to do that
> > >e.g. in a small kernel module though.
> >
> > are you thinking of a way to bond two file descriptors together?  that
>  > would be nice.  first you do all header processing and whatnot, and then
> > you just say
> >
> > bond_fd(server_fd /* read side */, client_fd /* write side */);
>
> That and/or a generalised sendfile. That would be more in the line of
> sendfile(src, dst, size), whre src can be a socket as well. Both is
> useful for a number of situations to avoid unncessary round trips.


check out the new splice() and tee() syscalls on linux, they allow for
zero-copy data movement...

http://kerneltrap.org/node/6505

quoting from Linus Torvalds explanation:

"
Now, the advantage of splice()/tee() is that you can do zero-copy movement
of data, and unlike sendfile() you can do it on _arbitrary_ data (and, as
shown by "tee()", it's more than just sending the data to somebody else:
you can duplicate the data and choose to forward it to two or more
different users - for things like logging etc).

So while sendfile() can send files (surprise surprise), splice() really is
a general "read/write in user space" and then some, so you can forward
data from one socket to another, without ever copying it into user space.

Or, rather than just a boring socket->socket forwarding, you could, for
example, forward data that comes from a MPEG-4 hardware encoder, and tee()
it to duplicate the stream, and write one of the streams to disk, and the
other one to a socket for a real-time broadcast. Again, all without
actually physically copying it around in memory.
"



Best Regards,


--
Miguel Sousa Filipe