Skip to main content

VPNs - IPsec, WireGuard and SSH Tunnels

TL;DR
  • A VPN puts each packet inside an encrypted outer packet addressed to a gateway; the gateway unwraps it and delivers the inner packet locally. The public path sees only gateway to gateway.
  • IPsec = IKE (UDP 500, or 4500 behind NAT) negotiates keys and Security Associations, then ESP (IP protocol 50) carries the data. Behind a NAT, ESP is wrapped in UDP 4500 (NAT-T).
  • WireGuard is the lean modern default: UDP only, one fixed crypto suite, a peer is a public key plus the IPs it may use, and the whole codebase is small enough to audit.
  • Every tunnel spends MTU: 54 to 81 bytes for ESP, 60 or 80 for WireGuard. Clamp the MSS, or large transfers hang.
  • A VPN hides contents from the path and moves trust to the VPN operator. It does not make you anonymous to the site you visit.

Mental model​

ANALOGYEveryday

You put a letter inside a sealed courier envelope addressed to your branch office. The couriers carrying it across town see only the outer address and the weight. The branch office opens the envelope and posts the inner letter locally, as if it had been written there.

Where it breaks: the couriers cannot read the letter, and a VPN likewise blinds your ISP, but it gives the VPN operator a full view of the inner letters. You did not remove the observer, you chose a different one.

ANALOGYFor engineers

A VPN is encapsulation with encryption (see layers and models): the inner IP packet becomes the payload of an outer IP packet, the same move as wrapping a request in an envelope struct before a network call. It is also a virtual network interface: the OS routes packets to wg0 or an IPsec policy exactly as it would to eth0, and the tunnel code does the wrapping.

Where it breaks: a function call has no size limit to worry about. The outer packet is bigger than the inner one, so the tunnel interface has a smaller MTU than the wire, and forgetting that is the most common VPN bug.

How it works​

A VPN has two ends, usually called gateways (or a client and a gateway). Each end has a private-side address space and a public address. The sending gateway takes an inner packet, encrypts and authenticates it, puts it behind a new outer IP header addressed to the other gateway, and sends it. The receiving gateway checks and decrypts, then forwards the inner packet onto its private network.

host AGW AGW Bhost Bpublic interneteavesdropper10.1.0.5203.0.113.10198.51.100.710.2.0.9encrypted tunnel (ESP)sees: both gateways, SPI, sizecannot read inner IPs or portsinner: 10.1.0.5 > 10.2.0.9outer: 203.0.113.10 > 198.51.100.7inner: 10.1.0.5 > 10.2.0.9ESP, IP protocol 50, payload encrypted
Host A sends to host B. Gateway A wraps the inner packet in an outer packet addressed to gateway B, so the public path sees only the two gateway addresses and the SPI. Addresses are documentation ranges.

Two things must be agreed before a single data packet moves: which keys and which traffic. IPsec and WireGuard answer those two questions differently, and that difference is most of this page.

Two deployment shapes​

  • Site-to-site: two gateways join two networks permanently. Hosts behind them know nothing about the tunnel. The cloud version of this is in private connectivity.
  • Remote access: a laptop is one end of the tunnel. It gets a virtual address on the corporate network and routes some or all of its traffic through the gateway (see split versus full tunnel below).

IPsec: the architecture​

IPsec is a suite defined by the architecture RFC (RFC 4301). Its pieces:

  • A Security Association (SA) is a one-way agreement: the algorithm, the keys, a 32-bit SPI (Security Parameter Index) that names it, and a sequence counter. A conversation needs two SAs, one per direction.
  • A policy database (SPD) says which traffic to protect, discard or bypass. A SA database (SAD) holds the live SAs. For each outbound packet the kernel consults the SPD, finds the SA and applies ESP; for each inbound ESP packet it uses the SPI in the header to find the SA.
  • ESP (Encapsulating Security Payload, IP protocol 50, RFC 4303) encrypts and authenticates. AH (protocol 51, RFC 4302) authenticates only. ESP is the one real deployments use.
  • IKE (RFC 7296, IKEv2) is the control protocol that authenticates the peers, runs a Diffie-Hellman exchange (see key exchange) and creates the SAs, so nobody types keys by hand.

Tunnel mode versus transport mode​

originaltransporttunnelIPTCP + dataIPESP hdrIVTCP + datatrailerICVnew IPESP hdrIVinner IPTCP + datatrailerICV20 B20 B8 B8 B2+ B16 B20 B8 B8 B20 B2+ B16 Bencryptedauthenticated (ESP header to trailer)encryptedauthenticated (ESP header to trailer)
Not to scale. Transport mode keeps the original IP header and encrypts the transport segment. Tunnel mode encrypts the whole original packet, including its IP header, behind a new outer header. Byte counts are for AES-GCM over IPv4: the 8-byte IV is the GCM nonce, and the 16-byte ICV is the integrity tag.

The ESP trailer is padding plus a 1-byte pad length plus a 1-byte next-header field, so it is 2 bytes or more (RFC 4303 section 2); the padding makes the encrypted part a multiple of 4 bytes.

Transport modeTunnel mode
What is encryptedThe transport segment (TCP or UDP and data)The whole original IP packet
Original IP headerKept, visible on the pathEncrypted inside; a new outer header is added
Outer addressesThe real endpointsThe two tunnel endpoints (gateways)
Typical useHost to host (and L2TP over IPsec)Site-to-site and remote-access VPNs

The cost: overhead and MTU​

The overhead is the sum of the added fields, so derive it. The inner MTU is the underlay MTU (1500 here) minus that overhead, and the largest TCP segment is the inner MTU minus 40 (20 inner IP + 20 TCP, no options). The general rule and the black hole it causes belong to MTU, MSS and fragmentation; this table is the VPN side of it.

TunnelDerivation (bytes)OverheadInner MTU on 1500Largest TCP MSS
ESP tunnel, AES-GCM, IPv420 outer IP + 8 ESP + 8 IV + 2 trailer + 16 ICV5414461406
Same, with NAT-T54 + 8 UDP6214381398
ESP tunnel, AES-CBC + HMAC-SHA1-96, NAT-T, worst case20 + 8 UDP + 8 ESP + 16 IV + 15 pad + 2 trailer + 12 ICV8114191379
WireGuard over IPv420 IP + 8 UDP + 32 WireGuard6014401400
WireGuard over IPv640 IP + 8 UDP + 32 WireGuard8014201380

The ESP figures use full-size packets (a smaller packet may carry up to 3 bytes of GCM padding). The WireGuard 32 bytes are the 16-byte data header (type, receiver index, counter) plus the 16-byte Poly1305 tag (whitepaper, section 5.4.6). The default wg-quick MTU of 1420 is 1500 minus the IPv6 worst case of 80, so it is safe on either IP version.

Worked example. A client sends a full 1500-byte packet into an AES-GCM ESP tunnel over a plain 1500-byte path. The tunnel adds 54 bytes, so the outer packet is 1554 bytes, which does not fit. Either the sender knows the inner MTU is 1446 (so it never builds a packet above 1446), or the gateway must fragment, or drop it and send ICMP. If that ICMP is filtered, the connection hangs as soon as a full-size packet appears: this is the PMTUD black hole of MTU, MSS and fragmentation, one layer deeper. The fix is to set the tunnel interface MTU and clamp the TCP MSS to 1406 (or lower for the worst case), so TCP never builds a packet that needs the ICMP.

How IKEv2 builds the tunnel​

InitiatorResponderIKE_SA_INIT req: HDR, SAi1, KEi, NiIKE_SA_INIT resp: HDR, SAr1, KEr, NrIKE_AUTH req: HDR, SK{IDi, AUTH, SAi2, TSi, TSr}IKE_AUTH resp: HDR, SK{IDr, AUTH, SAr2, TSi, TSr}ESP data, both directionsAlgorithms agreed, DH and nonces exchanged,keys derived (unencrypted so far)Peers authenticate, traffic selectors agreed, first Child SA created
Four messages, two round trips. The first pair agrees algorithms and runs Diffie-Hellman in the clear; the second pair is encrypted with keys derived from it, authenticates both peers and creates the first Child SA. SK means encrypted and integrity-protected.

Reading the ladder (RFC 7296, sections 1.2 and 2.14 to 2.17):

  1. IKE_SA_INIT. Each side proposes cipher suites (SA), sends its Diffie-Hellman public value (KE) and a nonce (N). Both compute the DH secret and derive a seed from it and the nonces; from that seed they derive the keys protecting the IKE conversation itself. The DH is the one in key exchange, and it gives forward secrecy.
  2. IKE_AUTH. Now encrypted (SK). Each side sends its identity (ID) and an AUTH payload proving it holds a pre-shared key or a private key matching a certificate (see certificates and PKI); remote-access setups can also use EAP. They also agree on a Child SA (the ESP SA) and its traffic selectors (TSi, TSr), the address ranges the SA will carry.
  3. Data. The Child SA keys are derived from the IKE keys, so ESP packets start flowing. Later CREATE_CHILD_SA exchanges rekey an SA (optionally with a fresh DH for perfect forward secrecy), and INFORMATIONAL exchanges carry deletes and liveness checks.

IKEv1 (1998, RFC 2409) used a "phase 1 then phase 2" structure that took around nine messages; the phase vocabulary survives in vendor configuration screens, but IKEv2 does the same job in four.

WireGuard in one page​

WireGuard (whitepaper, protocol) removes the negotiation. There is one cryptographic suite, no algorithm choice, no SPI zoo, and no IKE:

  • Each node has a Curve25519 keypair. A peer is identified by its public key, exchanged out of band like an SSH key.
  • A 1-RTT handshake based on the Noise IK pattern (Noise_IKpsk2_25519_ChaChaPoly_BLAKE2s) authenticates both sides and derives fresh symmetric keys; the DH side is described in key exchange, and ChaCha20-Poly1305 is the AEAD described in symmetric crypto and hashing.
  • Data then flows as small UDP datagrams. There is no separate "connection": WireGuard sends a handshake when it has data and no current session.
  • Cryptokey routing is the whole configuration model. Each peer has an AllowedIPs list. Outbound, the destination address of a packet selects the peer (longest-prefix match). Inbound, after decryption, the packet is accepted only if its source address is in that peer's AllowedIPs. So AllowedIPs is both the routing table and the access control list.

Deep dive​

Why ESP has no ports, and NAT-T​

ESP is its own IP protocol (50), not TCP or UDP, so it has no source or destination port. A NAT or PAT device (see NAT) maps flows by port, so it has nothing to key on; many simply drop protocol 50, and those that pass it can serve only one inside host per outside address. IKE has a milder problem: it uses UDP 500, and a NAT may rewrite the source port.

NAT traversal (NAT-T), RFC 3948 with detection in RFC 7296 section 2.23, fixes both:

  1. During IKE_SA_INIT each side sends hashes of its own and the peer's address and port (NAT_DETECTION_SOURCE_IP and NAT_DETECTION_DESTINATION_IP). If a hash does not match what the other side sees, a NAT sits between them.
  2. On detection, IKE switches to UDP port 4500, and ESP packets are wrapped in a UDP header on that same port. Now the NAT has a UDP flow to map. IKE messages on 4500 begin with four zero bytes so the receiver can tell them from ESP.
  3. Because the NAT mapping times out like any UDP mapping (see NAT), the tunnel sends keepalives: a single 0xFF byte in a UDP packet (RFC 3948 section 2.3).

The price is 8 more bytes per packet, which the overhead table above already counts.

AH and why it is rare​

AH authenticates the packet including the immutable fields of the IP header, so a NAT that rewrites the source address breaks the integrity check by design. It also adds no confidentiality. ESP with an integrity algorithm (or an AEAD such as AES-GCM) protects everything that matters inside the tunnel, which is why ESP is the protocol deployments use and AH is something you meet mainly in the RFCs.

Liveness, rekeying and lifetimes​

  • Dead peer detection is an empty INFORMATIONAL request; if no reply arrives after retransmissions, the SAs are torn down so traffic fails over instead of black-holing (RFC 7296 section 2.4).
  • Rekeying replaces SAs before they expire by time or byte count (the limits are local policy). ESP sequence numbers drive an anti-replay window (RFC 4303 section 3.4.3).
  • If one side reboots, the other still holds SAs for the old keys; stale SAs nobody deleted are a classic "tunnel is up but nothing passes" cause.

Policy-based versus route-based​

In a policy-based VPN the traffic selectors (for example 10.1.0.0/16 to 10.2.0.0/16) decide what is encrypted; there is no interface to route to, and every extra subnet is another policy pair that must match on both ends. In a route-based VPN the tunnel is a virtual interface (a VTI on Linux; vendors call it a tunnel interface); the selectors are wide open and ordinary routing, including a dynamic routing protocol, decides what goes in. Route-based is what you want when subnets change, and WireGuard is route-based by nature. AWS supports both static and BGP-learned routes over a Site-to-Site connection (AWS docs).

WireGuard internals worth knowing​

  • Handshake messages are fixed size: initiation 148 bytes, response 92, cookie reply 64 (whitepaper, section 5.4).
  • Silent to strangers. The initiation carries a MAC keyed with the responder's public key; a packet without a valid MAC is dropped with no reply. A port scan sees nothing, and an attacker cannot use the host as a reflector.
  • Roaming. The peer's endpoint (IP and port) is updated whenever an authenticated packet arrives from a new source address. A laptop that moves from Wi-Fi to cellular keeps the tunnel.
  • Rekeying. A session key is replaced after 120 seconds of use, and a key older than 180 seconds is rejected (whitepaper section 6.1). Optional PersistentKeepalive sends an empty packet at a fixed interval to hold NAT mappings open.
  • No cipher agility. If a primitive breaks, the protocol version changes. That removes downgrade attacks and a large amount of code, at the cost of a flag day. The whitepaper describes the Linux implementation as under 4,000 lines of code.
  • UDP only. On networks that block UDP it simply fails.

IPsec, WireGuard and OpenVPN side by side​

IPsec (IKEv2 + ESP)WireGuardOpenVPN (TLS)
TransportESP (IP proto 50), UDP 500 and 4500 for IKE and NAT-TUDP only, one portUDP or TCP, userspace
Crypto agilityMany suites negotiated by IKEOne fixed suite, no negotiationTLS suites, negotiated
AuthenticationPSK, certificates, EAPPublic keys, optional pre-shared keyTLS certificates, user and password plugins
NAT handlingNeeds NAT-T (UDP 4500)Native, it already is UDPNative, it already is UDP or TCP
InteroperabilityStandard, everywhere, many vendor quirksLinux, Windows, macOS, mobile; small config modelWide client support; not IPsec-compatible
Where it livesKernel (xfrm) plus an IKE daemon such as strongSwanKernel module or userspace (wireguard-go, BoringTun)Userspace process with a tun device

TLS-based VPNs (OpenVPN and several commercial clients; see the OpenVPN how-to) run the key exchange as a TLS session and then carry packets in that protected channel. They traverse almost any firewall, because TCP 443 or UDP is usually allowed, and reuse the certificate ecosystem in TLS. Running the tunnel over TCP is the cost: carrying TCP over TCP can make retransmissions pile up in both layers (Titz, Why TCP over TCP is a bad idea), so UDP mode is preferred where it works.

Split versus full tunnel​

Split tunnelFull tunnellaptopcorp netsinternetlaptopcorp netsinternetencrypted tunnelISP, directencrypted tunneltunnel, exits at gateway10.0.0.0/8 to tunnel; default to ISPdefault to tunnel
The only difference is the routing table. Split tunnel sends just the corporate prefixes through the tunnel and leaves the default route on the ISP. Full tunnel makes the tunnel the default route, so every byte crosses the gateway.
  • Split tunnel saves gateway bandwidth and keeps video calls off the VPN, but the device is on two networks at once and the security policy only covers the tunnelled part.
  • Full tunnel gives one inspection point and a single egress IP, but every byte, including personal streaming, crosses the gateway. Its bandwidth and any cloud egress bill scale with all user traffic. See host forwarding decisions for how the OS picks a route.
  • DNS leaks. Routes are not DNS settings. With a split tunnel, internal names must go to the corporate resolver, so the client must also override DNS for those names; if it does not, internal hostnames leak to the ISP's resolver or fail to resolve. With WireGuard AllowedIPs = 0.0.0.0/0, IPv6 traffic is not captured unless ::/0 is also listed.

SSH tunnels: a poor man's VPN for one port​

SSH can forward TCP (not UDP, and not whole IP packets) over its encrypted channel; the flags are in ssh(1):

  • -L [bind:]port:host:hostport (local forward). Listens on your machine; each connection travels through the SSH server and is opened from the server to host:hostport. host is resolved on the server's side.
  • -R [bind:]port:host:hostport (remote forward). The reverse: the SSH server listens, and connections come back through the tunnel to host:hostport as seen from your machine. The remote listener binds to loopback only unless the server's GatewayPorts setting allows otherwise.
  • -D [bind:]port (dynamic forward). Starts a local SOCKS proxy; each application connection becomes its own forward. Resolve names through the proxy (for example curl --socks5-hostname) or the DNS lookup happens outside the tunnel.
  • -N (no remote command) and -J (jump host) are the flags usually added to these. The VPN-like mode, -w, forwards whole IP packets through a tun device and needs the server's PermitTunnel enabled; it is rarely used.

SSH forwarding is TCP over TCP, authenticated by SSH keys, and needs no new infrastructure, which makes it right for one database or one web UI and wrong for a whole network.

Zero trust versus a VPN​

A VPN puts a device on the network, so once connected it can reach everything the routes and firewalls allow. Zero trust access inverts that: the user and the device authenticate to a proxy per application, and policy is decided per request, with no implicit network position. Google's BeyondCorp paper described this model in 2014 (BeyondCorp) and NIST later standardised the vocabulary in SP 800-207 (Zero Trust Architecture). The cost is that every application must sit behind a proxy that understands it; non-HTTP and legacy protocols are the hard part. The threat framing is in the threat model; for filtering in the path see firewalls.

Textbook vs real world​

The textbook saysProduction doesWhy
IPsec is the VPNIPsec is one of several; new deployments increasingly use WireGuard or a TLS-based VPN, and zero trust proxies replace the VPN for web appsIPsec's flexibility is also its complexity: many suites, vendor quirks, NAT workarounds
Use AH plus ESP for the most securityESP only; AH is rareESP with integrity or AEAD already authenticates what matters, and AH breaks across NAT
A VPN gives you privacyA VPN moves who can see your traffic from the ISP to the VPN operatorThe tunnel ends somewhere, and whoever runs that end sees the inner packets
Every employee uses the VPN for all accessPer-application access through an identity-aware proxy, with the VPN kept for the legacy remainderA flat network behind a VPN lets one compromised laptop reach everything
Set the MTU to 1500 everywhereSet the tunnel MTU to the underlay minus overhead and clamp MSSAn encrypted packet is bigger than the one inside it

In production​

  • AWS Site-to-Site VPN. Every connection comprises two IPsec tunnels to two AWS endpoints for redundancy, supports IKEv2 and NAT-T, and does not support Path MTU Discovery (AWS Site-to-Site VPN). The cloud-side design is in private connectivity.
  • Tailscale builds a mesh of WireGuard tunnels and does its own NAT traversal and relay fallback, so devices behind different NATs connect directly where possible (How NAT traversal works). The hole-punching mechanics are in NAT.
  • Cloudflare wrote BoringTun, a userspace WireGuard implementation in Rust, for its WARP product (BoringTun announcement), and WARP is the on-ramp for its zero-trust private-network offering. Current client protocol options are in Cloudflare's WARP documentation.
  • Linux carries both stacks in the kernel: XFRM for IPsec, with strongSwan or Libreswan running IKE, and WireGuard since version 5.6 (Linux 5.6 release notes).
  • Google moved its employees from a corporate-VPN model to per-application access with BeyondCorp, as described in the paper cited above.

History and evolution​

HISTORY
  • 1995 IPsec begins with RFC 1825: IP itself had no security and anyone on the path could read or forge packets, so security was added at the network layer.
  • 1998 the architecture is revised as RFC 2401 and IKE (RFC 2409) automates key exchange. IKEv1 is complex and takes roughly nine messages, with confusing "modes" and "phases".
  • 1999 PPTP (RFC 2637) and L2TP (RFC 2661) are the common dial-up VPN protocols. PPTP's MS-CHAP authentication was later broken, and L2TP is usually paired with IPsec for its encryption.
  • 2001 OpenVPN appears (around this year) as a userspace, TLS-based alternative that works through most firewalls.
  • 2005 RFC 4301 rewrites the architecture, IKEv2 is published as RFC 4306 (cleaned up as RFC 5996 in 2010, now RFC 7296 from 2014), and NAT-T becomes standard as RFC 3947 and 3948, because ESP had stopped working behind the NAT everyone now had.
  • 2014 Google's BeyondCorp paper argues network location should not grant trust, which starts the move from VPN to zero-trust access.
  • 2017 Jason Donenfeld's WireGuard paper (NDSS 2017) shows a VPN in a few thousand lines with one modern crypto suite.
  • 2020 WireGuard is merged in Linux 5.6, and RFC 8784 lets IKEv2 mix a pre-shared key for post-quantum protection.
  • Now IPsec is still the interop standard for site-to-site (cloud and on-premise) links, WireGuard is the default for new self-managed tunnels and mesh products, and ZTNA and SASE products replace remote-access VPNs for web and SaaS access.
  • Next post-quantum key exchange: RFC 9370 lets IKEv2 run several key exchanges for a hybrid, and WireGuard's optional pre-shared key already adds symmetric protection against a future quantum attacker.

Hands-on​

Every output below is illustrative (abbreviated, with documentation addresses and made-up key and SPI values). Yours will differ.

WireGuard status (Linux and macOS, wg from wireguard-tools):

sudo wg show
interface: wg0
public key: xTIBA5rboUvnH4htodjb6e697QjLERt1NAB4mZqp8Dg=
private key: (hidden)
listening port: 51820
 
peer: HIgo9xNzJMWLKASShiTqIybxZ0U3wGLiUeJ1PKf8ykw=
endpoint: 198.51.100.7:51820
allowed ips: 10.2.0.0/16
latest handshake: 41 seconds ago
transfer: 1.20 MiB received, 310.52 KiB sent

Look at latest handshake: a healthy tunnel with traffic shows a handshake within the last few minutes. No handshake line means no authenticated packet ever arrived, so check the endpoint, the firewall (UDP 51820) and that the public keys are not swapped.

IPsec state in the Linux kernel (needs root):

sudo ip xfrm state
sudo ip xfrm policy
src 203.0.113.10 dst 198.51.100.7
proto esp spi 0xc1a2b3d4 reqid 1 mode tunnel
replay-window 32 flag af-unspec
src 10.1.0.0/16 dst 10.2.0.0/16
dir out priority 375423
tmpl src 203.0.113.10 dst 198.51.100.7
proto esp spi 0xc1a2b3d4 reqid 1 mode tunnel

The first block is the SA (note proto esp, the SPI, mode tunnel), the second is the policy that sends 10.1/16 to 10.2/16 through it. ip xfrm state also prints the keys, so do not paste its full output into a ticket.

strongSwan's view (older ipsec command; newer installs use swanctl --list-sas):

sudo ipsec statusall
Security Associations (1 up, 0 connecting):
site-b[1]: ESTABLISHED 12 minutes ago, 203.0.113.10[gw-a]...198.51.100.7[gw-b]
site-b{1}: INSTALLED, TUNNEL, reqid 1, ESP SPIs: c1a2b3d4_i d4c3b2a1_o
site-b{1}: 10.1.0.0/16 === 10.2.0.0/16

ESTABLISHED is the IKE SA and INSTALLED is the Child SA. If you see the first and not the second, authentication worked but the traffic selectors did not match.

Capture the tunnel on the wire (see tcpdump and Wireshark):

sudo tcpdump -ni any 'esp or udp port 500 or udp port 4500'
IP 203.0.113.10.500 > 198.51.100.7.500: isakmp: parent_sa ikev2_init[I]
IP 203.0.113.10 > 198.51.100.7: ESP(spi=0xc1a2b3d4,seq=0x1), length 132
IP 203.0.113.10.4500 > 198.51.100.7.4500: UDP-encap: ESP(spi=0xc1a2b3d4,seq=0x2), length 132

ESP(...) with no ports is plain ESP; UDP-encap on port 4500 means NAT-T is in use. You can read the SPI and size but not the contents, which is the point.

Find the tunnel MTU by pinging with the Don't Fragment bit set and a growing size (the number is the ICMP payload; add 28 bytes for the ICMP and IP headers):

ping -M do -s 1400 10.2.0.9 # Linux
ping -D -s 1400 10.2.0.9 # macOS
ping: local error: message too long, mtu=1446

Linux prints the interface limit like this once you exceed it; on a remote hop you instead see Frag needed and DF set (mtu = ...) or nothing at all (the black hole). With an inner MTU of 1446, the largest working -s is 1446 - 28 = 1418.

SSH forwards:

ssh -N -L 8080:localhost:80 user@bastion
ssh -N -D 1080 user@bastion
curl --socks5-hostname localhost:1080 http://internal.example/
debug1: Local connections to LOCALHOST:8080 forwarded to remote address localhost:80

(That line is from ssh -v.) The first forward lets http://localhost:8080 reach port 80 on the bastion; the second makes a SOCKS proxy on 1080, and --socks5-hostname resolves the name on the far side.

Gotchas​

GOTCHAS6
A VPN that connects but hangs on large transfers is an MTU problem
ESP has no ports, so a NAT cannot map it
"A VPN hides me from the website"
Full tunnel does not automatically capture IPv6 or DNS
WireGuard AllowedIPs is routing and access control in one
SSH -R opens a hole on the server, not on you

Interview questions​

INTERVIEWWhat does a VPN do and what does it not do?junior
Show answer

It wraps your packets in an encrypted outer packet to a gateway, so anyone on the path between you and the gateway sees only that you are talking to the gateway and how much. It does not make you anonymous: the site still sees your accounts and cookies, and the VPN operator sees what your ISP no longer can. It also does not secure the traffic after the gateway unless that traffic is encrypted itself.

Follow-ups:

  • Who can see the traffic? The VPN operator sees the inner packets, and the destination sees them in whatever form that hop uses.
  • Does it protect against malware? No, it protects the path, not the endpoint.
INTERVIEWSite-to-site versus remote-access VPN: what is the difference?mid
Show answer

Site-to-site joins two networks permanently with a gateway at each end, and the hosts behind them are unaware of it. Remote access makes a single device one end of the tunnel, gives it a virtual address, and often runs a client with user authentication. The protocols can be the same; the difference is who the endpoints are and who authenticates.

Follow-ups:

  • Route-based versus policy-based? Policy-based encrypts what the selectors match, one policy pair per subnet pair. Route-based uses a tunnel interface and normal routing, so new subnets and BGP just work.
  • Which does AWS use? Both are supported, with static or BGP routes.
INTERVIEWHow does IPsec establish a tunnel?mid
Show answer

IKE does it in two exchanges (RFC 7296). IKE_SA_INIT agrees algorithms, runs Diffie-Hellman and exchanges nonces, giving both sides keys to protect the rest. IKE_AUTH, now encrypted, authenticates both peers and creates the first Child SA with its traffic selectors. ESP packets protected by that SA then flow, one SA per direction identified by an SPI.

Follow-ups:

  • ESP versus AH? ESP encrypts and authenticates; AH authenticates only and covers the IP header, so it breaks across NAT and is rarely used.
  • What were the "phases"? IKEv1 vocabulary: phase 1 made the IKE SA, phase 2 the IPsec SA. IKEv2 does it in four messages.
  • How do SAs get replaced? CREATE_CHILD_SA rekeys before the lifetime ends.
INTERVIEWWhy does my VPN hang on large transfers while small requests work?mid
Show answer

The encapsulation adds bytes (54 to 81 for ESP, 60 or 80 for WireGuard), so a full-size inner packet no longer fits the 1500-byte path. If the ICMP that would trigger path MTU discovery is blocked, the sender never learns the limit and large packets vanish silently, while small ones pass. Fix it by lowering the tunnel MTU and clamping the MSS.

Follow-ups:

  • What MSS would you clamp to? Inner MTU minus 40: 1406 for AES-GCM ESP and 1400 for WireGuard over IPv4, lower for the worst case with NAT-T.
  • Why is the failure hard to see? The handshake and small packets work, and the tunnel hides the ICMP from tools. See the black hole.
INTERVIEWHow does IPsec work through a NAT?mid
Show answer

ESP has no ports, so a NAT cannot map it. NAT-T (RFC 3948) detects a NAT during IKE by comparing address hashes, moves IKE to UDP 4500, and wraps ESP in UDP on port 4500 so the NAT sees an ordinary UDP flow. The peers send tiny keepalives to stop the mapping timing out.

Follow-ups:

  • Which ports must be open? UDP 500 for the first IKE messages and UDP 4500 once NAT-T is in use; ESP itself only when there is no NAT.
  • What does NAT-T cost? 8 bytes per packet for the UDP header.
  • Why does AH not survive? It authenticates the fields a NAT rewrites.
INTERVIEWWireGuard versus IPsec versus OpenVPN: how would you choose?mid
Show answer

WireGuard for new self-managed tunnels and meshes: small, fast, UDP only, one modern suite, simple config. IPsec when interoperating with cloud gateways and vendor appliances, where it is the standard. OpenVPN or another TLS VPN when the network blocks UDP or you need to ride TCP 443. WireGuard is smaller because it removes negotiation: one suite, no IKE, a fixed handshake.

Follow-ups:

  • What does WireGuard give up? Algorithm agility, a TCP mode and a standards body of interop partners.
  • How does it authenticate? Public keys, exchanged out of band, in a Noise IK handshake.
INTERVIEWHow is a VPN different from zero trust access, and what breaks with a full tunnel?senior
Show answer

A VPN grants network access: once in, you reach whatever routing and firewalls allow. Zero trust authenticates the user and the device per application through a proxy, with no implicit network trust, so a stolen laptop reaches only what policy lists. A full tunnel adds its own problems: all traffic, including video calls, hairpins through the gateway, its bandwidth and egress cost scale with every user, and DNS and IPv6 handling must be explicit.

Follow-ups:

  • What is hard about zero trust? Non-HTTP and legacy protocols, and every app needing a proxy in front.
  • When keep a VPN? Site-to-site links and legacy systems that cannot sit behind a proxy.
INTERVIEWTwo offices both use 10.0.0.0/16. How do you connect them?senior
Show answer

You cannot route between identical prefixes, so one side must be translated. Use NAT over the VPN: map one office's network to an unused range (for example 10.0.0.0/16 to 10.50.0.0/16) at its gateway, and have the other office talk to the translated range. Renumbering is the cleaner long-term fix. Policy-based VPNs make the translation easy to get wrong because the selectors must use the translated addresses.

Follow-ups:

  • What breaks? Protocols that embed IP addresses, and logs showing translated sources.
  • Where does it live in the cloud? The gateway or a NAT device in front of the VPN endpoint; see private connectivity.

Fun facts​

FACTSVPN trivia
  • A WireGuard server does not answer packets that fail the handshake MAC, so a port scan of its UDP port sees nothing at all.
  • ESP is IP protocol 50 and AH is 51; IKE is UDP 500, and 4500 once NAT-T is in play.
  • IPsec was originally mandatory for every IPv6 node; RFC 6434 later relaxed that requirement to a recommendation.
  • The default WireGuard MTU, 1420, is 1500 minus 80: the IPv6 worst case, chosen so one value works on both IP versions.

Cheat sheet​

NUMBERSVPN numbers to remember
  • UDP 500 / UDP 4500 / IP proto 50 / IP proto 51 - IKE, IKE and ESP behind NAT, ESP, AH
  • 54 / 62 / 81 - ESP tunnel overhead: AES-GCM, with NAT-T, CBC worst case with NAT-T
  • 60 / 80 - WireGuard overhead over IPv4 / IPv6; default wg-quick MTU 1420
  • 1446 / 1406 - inner MTU and MSS for an AES-GCM ESP tunnel on a 1500 path
  • 148 / 92 / 64 - WireGuard handshake initiation, response and cookie reply, in bytes
  • 120 s / 180 s - WireGuard session rekey time and rejection time
DECISIONWhich tunnel for the job?
Interoperate with a cloud gateway or vendor appliance→IPsec (IKEv2 + ESP), with NAT-T
New self-managed tunnel or mesh between hosts you control→WireGuard
Network blocks UDP, or only 443 gets out→A TLS-based VPN over TCP 443
One internal web UI or database for yourself→An SSH forward (-L), no VPN
Per-app access for employees on the web or SaaS→Zero trust access proxy, not a VPN
Pick this when: you can name the network that blocks one option and the interop partner that requires another