Re: Network Slowdowns?

[Bill Hacker <wbh@xxxxxxxxxxxxx>]

Matthew Dillon wrote:

>     Packet overhead for bulk data transfers is only around 3.4%.  The
>     ethernet MAC is 12 bytes.  The TCP/IP header is 40 bytes (20 bytes
>     for UDP/IP), plus additional bytes for options, and the data payload
>     is usually around 1448 bytes.
>
>     Ethernet framing overhead is harder to calculate.  10BaseT is easy,
>     it's just manchester encoded at 20MHz -- so 8 bits per byte at 10MHz,
>     basically.  I think there is a short preamble and a CRC tacked on.
>
>     100BaseT encodes 4 bits into 5 over the wire (10 bits per byte), but
>     the line frequency is actually 125MHz, not 100MHz, so one byte of
>     data takes 10 bits @ 8ns/bit = 80ns == 12.5 MBytes/sec.  If you
>     subtract the 3.4% packet overhead you get around 12 MBytes/sec.  I
>     think there is also a premable and some other junk there that
>     reduces the effective bandwidth to 11 MBytes/sec with 1500 MTU packets.
>
>     GiGE also uses a line frequency of 125MHz, encoding 2 (or is it 3?)
>     bits into each 8ns period.  But (and now I am looking at the Wikipedia
>     entry for GigE) it transmits in both directions on all four cable pairs.
>     So you get 125 x 2 x 4 = 1 GBit.  Each 8 bit byte is converted to 10 bits.
>     It's all very confusing, actually.  GigE requires a lot of signal 
>     processing and uses echo cancellation to work.
>
>     CPU overhead is a different beast entirely.   SSH has a lot of encryption
>     and decryption overhead... my transfers over localhost top out at
>     21.8 MBytes/sec on my test box.  At 100BaseT speeds the packet rate
>     is only around 6000-8000 packets per second and cpu overhead for packet
>     processing is usually in the noise.  At GiGE rates, however, the packet
>     rate is over 66,000 packets per second and cpu overhead can be
>     significant even if you discount interrupt and device access overheads.
>
> 						-Matt

Stuff all well and true - as far as it goes, i.e in the lab, and under 
controlled conditions (no other daemons running, for starters).

Now .. put yourself in the shoes of a field-Engineer armed, in ascending order with:

- 'promiscuous' mode IP NIC settings:
 "Gee - there are IP packets eating up b/w that aren't mine at all..."

- protocol analyzer:
  "Hmm.. and ethernet traffic that isn't even IP protocol.."

- oscilloscope:
   Holy *shit* what fool routed the CAT5 next to the power cables for the 
elevator motors?

Used to laugh at the folks from our C&W 'sister company' on Ethernet frantically 
shouting across the office and phoning each other 'shut down your PC!' around 4 
in the afternoon as air-mass thunderstorms overtook Tyson's Corner, Virginia.

Our business unit kept right on working.

ARCnet/TCNS on coax basically ignore lightning strikes a mere 500 meters away. 
Ethernet doesn't have that luxury.

;-)

Bill