The Transport Layer: TCP, UDP, and Congestion Control
The differences between TCP and UDP, the 3-way and 4-way handshakes, and flow and congestion control, from a backend operations perspective.
Contents
On the same IP layer, TCP chooses reliability and UDP chooses low latency. That choice is what separates them in handshakes and congestion control.
What the transport layer handles
The transport layer is responsible for end-to-end communication. IP moves a packet to the destination host; the transport layer connects the data to a specific process inside that host. The two pillars of this layer are TCP and UDP.
TCP (Transmission Control Protocol) provides a connection-oriented, reliable byte stream. The current specification is RFC 9293, which consolidates the earlier RFC 793. UDP (User Datagram Protocol), defined in RFC 768, provides only minimal datagram delivery and guarantees neither ordering nor retransmission.
The difference between the two protocols comes down to a trade between reliability and latency. What each one guarantees determines its header size, its connection procedure, and whether it does congestion control.
The trade-off between TCP and UDP
The table below compares the two protocols from an operations perspective. Header sizes are the values specified in the respective RFCs.
| Item | TCP | UDP |
|---|---|---|
| Connection model | Connection-oriented | Connectionless |
| Reliability | Ordering, retransmission, flow and congestion control guaranteed | Not guaranteed |
| Header size | 20-60 bytes | 8 bytes |
| Transmission unit | Byte stream | Message (datagram) |
| Typical uses | HTTP, HTTPS, SSH | DNS, streaming, gaming, QUIC |
Saying UDP is fast is only half right. UDP itself is not fast; it has no connection setup cost, and the application can choose which parts of ordering and retransmission to implement. That is why RFC 8085 recommends handling congestion control and message size directly when designing a protocol on top of UDP.
TCP's reliability is implemented with acknowledgements (ACK), retransmission timers, and ordering control. The calculation of the retransmission timer (RTO, Retransmission Timeout) is standardized in RFC 6298. To turn these guarantees on, TCP establishes a connection before sending any data.
The handshakes that open and close a connection
TCP establishes a connection with a 3-way handshake before sending data. Both sides exchange and confirm each other's sequence numbers, which guarantees that bidirectional communication is ready. Termination goes through a 4-way handshake.
There is a reason the step counts differ, three to establish and four to close. Even after the client requests termination, the server may still have data left to send. So the server's ACK and FIN are not bundled together but sent separately.
The side that sends the final ACK does not close immediately but enters the TIME_WAIT state. This is a normal state that prevents delayed packets from mixing into the next connection and allows a response to a retransmission if the final ACK is lost.
TIME_WAIT lasts 2 x MSL. MSL (Maximum Segment Lifetime) is the maximum time a segment can live in the network, which RFC 793 defines as 2 minutes. The theoretical value is therefore up to 240 seconds, but the Linux kernel fixes it at 60 seconds.
Flow control and congestion control
The two controls guard against different things. Flow control keeps the receiver's buffer from overflowing; congestion control regulates the sender so the network path does not get clogged. Treating them as the same thing leads to misdiagnosis.
Flow control centers on the sliding window. The sender may send continuously without an ACK only up to the RWND (Receive Window) advertised by the receiver. Congestion control maintains a separate CWND (Congestion Window), and the actual amount sent is the smaller of the two, min(RWND, CWND).
The standard set of congestion control mechanisms is the four defined in RFC 5681. The window unit is the MSS (Maximum Segment Size), the maximum number of bytes that fit in one segment.
| Algorithm | Behavior | Entry condition |
|---|---|---|
| Slow Start | Doubles every RTT starting from a 1 MSS window | Connection start or after a timeout |
| Congestion Avoidance | Increases linearly by 1 MSS per RTT | After reaching ssthresh |
| Fast Retransmit | Retransmits immediately without waiting for a timeout | 3 duplicate ACKs received |
| Fast Recovery | Halves the window instead of dropping it to 1 | Immediately after Fast Retransmit |
The transitions between these states look like the following. RTT (Round Trip Time) is the round-trip time, and ssthresh is the threshold at which exponential growth switches to linear growth.
The key point is that the reaction differs with the strength of the loss signal. Three duplicate ACKs are a weak loss signal indicating the path is still alive, so the window is only halved. A timeout is a strong signal, so the window returns to 1 and Slow Start begins again. The default congestion control algorithm on Linux is CUBIC.
When to choose UDP, and QUIC
UDP is used where latency hurts more than loss: short request-response exchanges, or cases where real-time behavior matters more than accuracy. The table below lists the typical reasons for choosing it.
| Use case | Reason for choosing it |
|---|---|
| DNS (53) | Short request/response, speed first |
| Real-time gaming | Low latency, tolerates some loss |
| Video and voice streaming | Real-time behavior over reliability |
| QUIC (443) | Implements reliability directly on top of UDP |
QUIC turns UDP's lack of reliability guarantees into design headroom. TCP is implemented in the OS kernel and therefore slow to change, while QUIC, running on top of UDP, can handle congestion control and retransmission in user space. HTTP/3 runs on QUIC, and one of its design goals is to isolate the impact of packet loss to individual streams.
Choosing UDP therefore does not mean giving up reliability. It means the application designs exactly as much guarantee as it needs. QUIC is the case where that choice was standardized.
Signals you meet in operations
Restarting a server often produces an 'Address already in use' error. The previous connection is still in TIME_WAIT and blocks binding to the same port. Enabling the SO_REUSEADDR option reuses a port in TIME_WAIT so the server can come back up immediately.
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server.bind(('0.0.0.0', 9000))
server.listen()TIME_WAIT itself is not something to eliminate. It is the trace of a normal shutdown, so if the count grows, first look at why short-lived connections increased. When connection problems recur, checking the proxy's keep-alive and connection pool settings before the application is usually faster.
Frequent retransmission should not be written off as a slow network but read as a signal of loss and congestion. Moving to QUIC or HTTP/3 leaves variables such as packet loss, certificates, and DNS exactly where they were.
Summary
The two pillars of the transport layer trade reliability against latency. TCP opens a connection with a 3-way handshake and closes it with a 4-way handshake, and its reliability rests on ACKs, retransmission, flow control, and congestion control. UDP leaves those guarantees empty and lets the application fill in as much as it needs.
Congestion control reacts differently depending on the strength of the loss signal. Halving the window on three duplicate ACKs and resetting it to 1 on a timeout is the core distinction in RFC 5681. Separating flow control's RWND from congestion control's CWND makes it easier to locate a bottleneck precisely.
In operations, TIME_WAIT, SO_REUSEADDR, and retransmission are signals in the same picture. Reading them as part of normal behavior avoids unnecessary tuning. QUIC and HTTP/3 are a redesign that pulls these transport characteristics into user space.