[vworld-tech] Some resources for ... TCP UDP

[Jeremy Noetzelman]

(No, I'm not picking on you, Adam, even if it may seem that way)

On Mon, 19 Jan 2004, ceo wrote:

> I'm a bit confused by the "does not support multichannel": that's what
> sockets were invented for, no? Are you saying that a single lost packet
> on any socket backs up all other socket comms on the same PC? If so, is
> this an OS-specific or NIC-specific problem?

Yes, you can emulate multichannel that way.  Additionally, you'll want to
differentiate 'socket' from 'connection'  ...  You can have multiple
connections to the same socket on a server as long as the source ports are
different on the client.  So, if you ask 'Does a single lost packet on a
connection back up all other connection comms on the same PC?' then the
answer is definitely no.

It's also a bit different than that, as a lost packet doesn't stall out
comms (ie it doesn't block while waiting for the lost packet) ... the
connection can continue to accept out of order packets until it's buffer
fills up with no impact on the connection.  As long as the lost packet
gets retransmitted before the buffer fills, you're good to go.


> 1. "Guaranteed delivery" is absolute hell to implement properly. I've
> seen dozens of "UDP with guaranteed delivery" schemes which didn't work
> properly (in fact, note: JDK 1.1.x RMI was at one point completely
> broken because it included a naive implementation of this - RMI couldn't
> be used in large production environments because it generated so many
> colliding packets that it brought down the LAN. Yuk. Unforgivably stupid
> by a company with such a strong heritage in networked computers).

Just because people get it wrong doesn't mean that it's 'hell to implement
properly'  .... guaranteed delivery is pretty trivial.  Adding order
guarantees, etc gets a bit more complicated, but it's really not that
tough.


> 3. TCP is normally as fast or faster than UDP. Normally, UDP is very
> inefficient and wastes bandwidth (no, seriously), mainly because of
> small packet-sizes. For VERY small packets, UDP is more efficient,
> because it has a much smaller constant-overhead-per-packet. From memory,
> the numbers are something like 15 bytes overhead per-packet for UDP,
> compared to about 30 for TCP. But, TCP packets can be MUCH larger, so
> that 30 bytes CAN get sent much less often.

Given a loss free network, UDP is faster than TCP because the TCP overhead
is higher than UDP overhead.  UDP is vastly more efficient and bandwidth
sensitive than TCP, actually.  Not sure where this quote came from, but
it's inaccurate.  TCP does not have distinct packets, for one ... as
stated above, it's a stream protocol, and the developer has no control
over the TCP packet size using standard methods, if you go down to raw
hardware obviously you can exert control.  This is a good abstraction and
is quite useful in many cases.

In practice, the TCP packet size will end up being right around the MTU
for the link, as larger packets will typically get dropped.  Oversize UDP
packets will usually get fragmented as opposed to dropped.

UDP headers are 8 bytes, IP header is 20 bytes, so total overhead for a
UDP/IP packet is 28 bytes.  As the maximum size for an IP packet is 65,535
bytes, your maximum UDP datagram size is 65,507.

TCP headers are 20 bytes at minimum (TCP headers can be larger).  This
results in a maximum payload of 65,495 bytes.


> Even games sending lots of messages per second may find TCP is
> fundamentally faster (but then they get bitten by the next problem,
> sooner or later). If you experiment on your LAN, you can often get TCP
> running faster; but don't bother - the next problem will screw you if
> you deploy your game like this.

As stated above, due to overhead, you will always see better UDP
performance than TCP, assuming a reliable, loss free network.


> 4. The vast majority of games developers only have one problem with TCP
> (but often mistakenly believe they need more!). They need to remove the
> "in-order arrival". Whenever you hear about "TCP is sloooow" or "TCP has
> high latencies", you are listening to someone who is 99% likely to have
> bitten by this problem but not understood it.
>
> The problem is that if you send 30 packets, and the fifth one gets
> dropped, but packets 6-10 arrive OK, your network stack will NOT allow
> your application to see those last 5 packets UNTIL it has received a
> re-transmitted packet 5.

You just can't do what is being proposed here.  The reason is that TCP is
a STREAM transport protocol.  UDP is a DATAGRAM transport protocol.  You
can certainly try to 'packetize' TCP, but there is zero guarantee on the
transport side that that one 'packet' you send to the stack won't be split
into a dozen TCP packets.

Using the above example (30 packet, 5th dropped, etc) the reason the stack
doesn't send 6-10 to your application is because TCP is a stream protocol.
To change that would require you to rewrite the TCP stack.  Not as big a
deal with your server, kinda hard to get a player to ruin their TCP stack
for a game.  (Note that if you do this, it's not TCP anymore...)

That's why people use UDP for this stuff.  It's alot easier to take the
'packet' paradigm and apply it to a transport that is already 'packet
aware' and add on whatever reliability measures you need.  Contrast that
with tearing TCP apart to remove the very things it is there for.


> 5. Lastly, there ARE alternatives to TCP and UDP. Not surprisingly,
> since almost every game finds that neither is really good enough (the
> games that just go TCP only suffer weird stuttering, the ones that are
> UDP only often get players freezing, or their guns not firing because of
> lost packets). The last time I looked, ENet seems to be the best
> widely/freely available implementation around, but people have suggested
> several others to me, including:
>     RAKnet (sp?)
>     RDP (covered by an official internet RFC)

eNet, etc are NOT alternatives to TCP and UDP.  They are stacks on TOP of
UDP which add things like reliability, etc.  RDP, on the other hand, is a
new transport protocol.  It is, of course, advisable to use something that
is done and proven instead of building it yourself ... but that's the
classic 'build or buy' problem which is beyond the scope of this email.


> 4. A depressingly large number of people who offer advice on these
> topics are naive or ignorant at best (i.e. they have gaps in their
> knowledge), or just plain wrong at worst. Be careful what advice you
> take! (and as endolf discovered, it's often not as simple as just
> experimenting with the alternatives people suggested and benchmarking
> them - there are nasty non-obvious subtleties in accidentally
> mis-implementing many of the protocols)."

Unfortunately, I'd have to say that the author of the email you're quoting
is 'naive or ignorant at best, or just plain wrong'.

J