What is the cost of many TIME_WAIT on the server side?
network-programming, networking, tcp, time-wait
Solution
Each socket in `TIME_WAIT` consumes some memory in the kernel, usually somewhat less than an `ESTABLISHED` socket yet still significant. A sufficiently large number could exhaust kernel memory, or at least degrade performance because that memory could be used for other purposes. `TIME_WAIT` sockets do not hold open file descriptors (assuming they have been closed properly), so you should not need to worry about a "too many open files" error.
The socket also ties up that particular `src`/`dst` IP address and port so it cannot be reused for the duration of the `TIME_WAIT` interval. (This is the intended purpose of the `TIME_WAIT` state.) Tying up the port is not usually an issue unless you need to reconnect a with the same port pair. Most often one side will use an ephemeral port, with only one side anchored to a well known port. However, a very large number of `TIME_WAIT` sockets can exhaust the ephemeral port space if you are repeatedly and frequently connecting between the same two IP addresses. Note this only affects this particular IP address pair, and will not affect establishment of connections with other hosts.
Problem
Let's assume there is a client that makes a lot of short-living connections to a server. If the client closes the connection, there will be many ports in `TIME_WAIT` state on the client side. Since the client runs out of local ports, it becomes impossible to make a new connection attempt quickly. If the server closes the connection, I will see many `TIME_WAIT`s on the server side. However, does this do any harm? The client (or other clients) can keep making connection attempts since it never runs out of local ports, and the number of `TIME_WAIT` state will increase on the server side. What happens eventually? Does something bad happen? (slowdown, crash, dropped connections, etc.) Please note that my question is not "What is the purpose of `TIME_WAIT`?" but "What happens if there are so many `TIME_WAIT` states on the server?" I already know what happens when a connection is closed in TCP/IP and why `TIME_WAIT` state is required. I'm not trying to trouble-shoot it but just want to know what is the potential issue with it. To put simply, let's say `netstat -nat | grep :8080 | grep TIME_WAIT | wc -l` prints `100000`. What would happen? Does the OS's network stack slow down? "Too many open files" error? Or, just nothing to worry about?