Hello,
Over the last week or so, I've been getting times of large packetloss - but only once into AT&T's network (i.e., not my immediate uplink), which is severely affecting DNS resolution, and also otherwise causing slow load times for pages. (Existing TCP flows seem less affected, interestingly).
I run my own local nameserver (bind/named), but I even have loss to the AT&T-delegated nameservers (99.99.99.53 and 99.99.99.153).
Loss has been pretty bad this morning, lots of crap in my nameserver logs about not getting a DNS response until advertised packet size was lowered to 512 bytes (EDNS stuff. Normally no issues. Is AT&T doing some weird blocking based on packet size?). e.g.:
named[18249]: success resolving 'google.com/A' (in 'google.com'?) after disabling EDNS
named[18249]: success resolving 'ns4.google.com/AAAA' (in 'google.com'?) after reducing the advertised EDNS UDP packet size to 512 octets
named[18249]: success resolving 'www.amazon.com/A' (in 'www.amazon.com'?) after reducing the advertised EDNS UDP packet size to 512 octets
named[18249]: success resolving 'client.fitbit.com/A' (in 'fitbit.com'?) after disabling EDNS
Nov 25 10:33:41 fs [syslog ] named[18249]: success resolving 'u1.amazonaws.com/A' (in 'com'?) after reducing the advertised EDNS UDP packet size to 512 octets
Nov 25 10:33:41 fs [syslog ] named[18249]: success resolving 'ns-931.amazon.com/A' (in 'amazon.com'?) after disabling EDNS
(etc...)
I'm also including a screenshot of my latency monitoring, which shows lots of packetloss (missing polls) across the board - these hosts are all over the place (one is international). [Ignore the IPv6 one that is blank ;)]
Here's a trace (mtr) to facebook. Hop 3's loss can be ignored, that host always drops packets directed at it. The real loss starts at/after hop 5:
HOST: (something local) Loss% Snt Drop Last Best Avg Wrst StDev 1.|-- [internal] 0.0% 50 0 0.2 0.2 0.2 0.3 0.0 2.|-- [another internal] 0.0% 50 0 0.7 0.5 0.6 1.7 0.2 3.|-- 108.251.64.2 72.0% 50 36 20.4 19.4 20.3 22.3 1.0 4.|-- 75.20.78.142 0.0% 50 0 20.6 19.2 21.8 33.9 3.3 5.|-- 12.83.70.145 20.0% 50 10 23.2 19.0 21.9 31.0 2.1 6.|-- 12.123.132.229 16.0% 50 8 23.3 22.8 38.5 200.0 30.2 7.|-- 12.252.12.6 12.0% 50 6 23.9 23.0 24.3 29.9 1.5 8.|-- be2.bb02.lax1.tfbnw.net 10.0% 50 5 53.9 52.5 53.5 54.7 0.4 9.|-- ae7.bb03.prn2.tfbnw.net 12.0% 50 6 44.3 43.6 47.7 86.6 9.3 10.|-- ae4.dr06.prn1.tfbnw.net 10.0% 50 5 51.4 50.9 51.9 53.7 0.6 11.|-- ??? 100.0 50 50 0.0 0.0 0.0 0.0 0.0 12.|-- ??? 100.0 50 50 0.0 0.0 0.0 0.0 0.0 13.|-- edge-star-shv-12-prn1.facebook.com 26.0% 50 13 51.6 51.0 52.2 57.2 1.2
This trace is similar regardless of where I trace to.
I tried emailing uversecare@, but that seems to be /dev/null these days...
Anyone else having issues? Is there some DDoS in another part of AT&T's network that is affecting me (and surely others)?
-Taner
↧