ss, netstat & curl
tcpdump shows packets on the wire; ss/netstat show the connection
table your own machine is keeping, and curl shows exactly how long each
phase of a single HTTP request took. Together they answer "what is this host
doing right now" without needing a packet capture at all.
ss and netstat: connection state and listening ports
ss (socket statistics) is the modern replacement for netstat, and is
generally faster on a host with many connections since it reads directly from
kernel tables instead of /proc. The flags map almost one-to-one:
-t restricts to TCP, -a shows all sockets (not just established), -n
prints numeric addresses/ports instead of resolving them (much faster, and
avoids DNS lookups skewing the output).
The states you'll see most often while debugging (see TCP States in Practice for the full state machine and TIME-WAIT exhaustion in depth):
| State | Meaning |
|---|---|
| LISTEN | A process is bound to this port, waiting for incoming connections |
| SYN-SENT | We sent a SYN and are waiting for SYN-ACK - stuck here means the peer isn't responding |
| ESTABLISHED | Handshake complete, connection is active |
| TIME-WAIT | Connection closed locally, kept around briefly to catch any late/duplicate packets |
| CLOSE-WAIT | Peer closed their side; we haven't closed ours yet - many of these piling up usually means the application is leaking connections |
A large number of connections stuck in SYN-SENT almost always means
something between you and the destination is silently dropping SYNs (a
firewall rule, security group, or a dead route) rather than the destination
actively refusing - an active refusal comes back as a reset immediately, not a
hang. A pile of CLOSE-WAIT connections is an application bug: the local
process is not calling close() after the peer hangs up.
curl -v and curl -w: timing a single request
ss/netstat show the connection table; curl shows exactly how long one
request took, broken into phases. -v (verbose) prints the request/response
headers and the TLS handshake:
For the actual phase timing, -w with the built-in timing variables gives
exact numbers instead of eyeballing -v output:
Each variable is cumulative from the start of the request, so subtract consecutive ones to get that phase's individual cost:
| Variable | Cumulative time to | Isolate the phase by |
|---|---|---|
time_namelookup | DNS resolution completing | value itself (first phase) |
time_connect | TCP connection established | time_connect - time_namelookup |
time_appconnect | TLS handshake completing | time_appconnect - time_connect |
time_starttransfer | first byte of the response body (TTFB) | time_starttransfer - time_appconnect |
time_total | entire request/response cycle | time_total - time_starttransfer |
A high time_namelookup alone points at DNS (see
dig and DNS Debugging); a high time_connect - time_namelookup points at network path/TCP; a high time_appconnect - time_connect points at TLS/certificate issues; a high time_starttransfer - time_appconnect points at the server being slow to produce a response,
not the network at all.
curl --resolve: bypassing DNS entirely
--resolve lets you force a hostname to a specific IP for one request,
without touching /etc/hosts or DNS - the cleanest way to test "is it the new
server that's broken, or DNS pointing somewhere stale":
This also correctly sends the real hostname in the TLS SNI and the Host
header, so it works against servers that route by name (unlike just curling
the IP directly, which would fail SNI-based virtual hosting or certificate
validation).
--resolve is the tool for validating a new server before changing DNS to
point at it, and for confirming a "works on one machine" report is a stale-DNS
problem rather than something wrong with the new server itself.