Re: SMP TCP userland access

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

:Do you think it would be beneficial to make the sockbuf access lockless a=
:s well?  Beneficial as in:  better throughput for the receiving applicati=
:on (maybe even overall?  imagine the tcp thread being blocked because of =
:readers on other CPUs)
:
:cheers
:  simon

    I think there is a benefit to not requiring the protocol stack to
    have to spinlock around every sockbuf access it makes, because the
    protocol stack is potentially having to process hundreds or thousands
    of connections in a loop and the cache footprint is pretty nasty if it
    has to spinlock each one.  I don't think it matters so much for the
    user process.

    This fits with the model.  We have a many-to-one and one-to-many 
    situation.  e.g. many user processes may be writing to the same socket,
    but there is only one reader (the network protocol stack).  And in
    the other direction, only the network protocol stack is dumping new
    data into a sockbuf but there may be many user processes trying to read
    it.

    Hence, the 'user' side is always the 'many' side and has to be locked
    no matter what, but the protocol thread side is always the 'one' side
    and it should be possible to build a FIFO algorithm for manipulating
    mbufs that does not require the protocol thread side to lock anything.

    One example, instead of using a linked list we could use a fixed array
    of mbuf slots for the sockbuf.  Then one just uses a separate read and
    write FIFO index.  The many side locks to access its index, and the 
    one-side doesn't have to lock to access its index.

					-Matt