Reading packet loss in mtr and traceroute
Distinguish lost probe replies at one hop from loss that continues to the destination.
An isolated red hop in mtr can be misleading. Compare its loss with every later hop and with the destination before escalating.
How mtr probes a path
mtr sends probes with increasing TTL values. Routers can return ICMP Time Exceeded from their control plane when a probe expires. A router can rate-limit those replies while still forwarding traffic normally in its data plane.
Read the whole report
If hop 7 reports 42% loss while the destination and later hops report none, the router probably limits probe replies. Loss that persists through later hops to the destination calls for investigation.
$ mtr -rwbzc 100 198.51.100.10
HOST Loss% Snt Last Avg Best Wrst StDev
1. AS64512 10.0.0.1 0.0% 100 0.4 0.5 0.3 1.2 0.1
5. AS64500 203.0.113.17 0.0% 100 8.1 8.4 7.9 12.0 0.6
7. AS64501 192.0.2.33 42.0% 100 31.2 30.8 30.1 44.9 2.1 # 58 replies to 100 probes
9. AS64501 192.0.2.61 0.0% 100 31.0 31.1 30.6 33.0 0.4
12. AS64502 198.51.100.10 0.0% 100 32.4 32.2 31.8 35.1 0.5
Patterns worth investigating
- Loss begins at a hop and persists to the destination.
- Latency increases at one point and stays elevated farther along the path.
- Measurements vary considerably at the destination. This may show jitter that affects calls or video.
Useful options
-T -P 443switches to TCP probes on port 443.-b -zincludes addresses, names and AS numbers.-c 100 -r -wcreates a wide report after 100 probes.
When sending a report to the provider, capture both directions if possible. Return traffic can take a different route.
What to put in the ticket
Record the time and time zone, source, destination, service port and the symptom seen by the application. Include the final hop in the report. If the target normally ignores ICMP, compare the report with a connection to the service itself.
Each hop gets its own series of probes. Router ICMP rate limits are described in RFC 1812, § 4.3.2.8.