af.mil DNSSEC Outage: 2015-05-29
Updated: May 29, 2015
Overview
This page gives some details on the af.mil DNSSEC outage on May 29, 2015. The outage was fixed by removing DNSSEC support, by deleting the DS record.
Verisign's DNSSEC Debugger
Verisign doesn't archive test results, unlike DNSViz. So here's a screenshot I took on May 29, 2015:
Timeline / DNSViz
- 2015-05-29 10:04:43 UTC — Bogus RRSIGs
- 2015-05-29 14:09:32 UTC — Bogus RRSIGs
- 2015-05-29 15:32:40 UTC — Bogus RRSIGs
- 2015-05-29 16:10:11 UTC — Bogus RRSIGs
- 2015-05-29 17:06:40 UTC — Bogus RRSIGs
- 2015-05-29 18:05:09 UTC — Bogus RRSIGs
- 2015-05-29 18:39:45 UTC — Bogus RRSIGs
- 2015-05-29 21:39:20 UTC — af.mil unsigned; outage over
OpenDNS & Google Public DNS
OpenDNS does not support DNSSEC, instead supporting DNSCurve. Google Public DNS currently supports only DNSSEC, and thus, Google's users saw SERVFAIL for queries under af.mil during this outage.
With OpenDNS, queries succeed:
$ dig www.af.mil. @resolver2.opendns.com
; <<>> DiG 9.4.2-P2 <<>> www.af.mil. @resolver2.opendns.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20117
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.af.mil. IN A
;; ANSWER SECTION:
www.af.mil. 14 IN CNAME www.af.mil.edgesuite.net.
www.af.mil.edgesuite.net. 14 IN CNAME a499.dscb.akamai.net.
a499.dscb.akamai.net. 20 IN A 23.59.189.83
a499.dscb.akamai.net. 20 IN A 23.59.189.90
;; Query time: 36 msec
;; SERVER: 208.67.220.220#53(208.67.220.220)
;; WHEN: Fri May 29 14:24:47 2015
;; MSG SIZE rcvd: 129
With Google Public DNS, with DNSSEC, queries fail:
$ dig www.af.mil. @8.8.8.8
; <<>> DiG 9.4.2-P2 <<>> www.af.mil. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 51210
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.af.mil. IN A
;; Query time: 103 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri May 29 14:24:57 2015
;; MSG SIZE rcvd: 28
dnscheck
dnscheck.iis.se archived a DNSSEC outage at 2015-05-29 09:26:46 (requires javascript).
dnscheck.labs.nic.cz archived a DNSSEC outage at 2015-05-29 09:27:33 (requires javascript).
Zonemaster
Zonemaster archived this af.mil DNSSEC outage.
Logfile examples
- [1432916617] unbound[7918:0] info: validation failure <www.af.mil. A IN>: signature crypto failed from 132.3.65.10 for key af.mil. while building chain of trust
- [1432916629] unbound[7918:0] info: validation failure <af.mil. NS IN>: key for validation af.mil. is marked as invalid because of a previous validation failure <www.af.mil. A IN>: signature crypto failed from 132.3.65.10 for key af.mil. while building chain of trust
- [1432916803] unbound[7918:0] info: validation failure <www.eglin.af.mil. A IN>: signature crypto failed from 132.3.57.10 for key af.mil. while building chain of trust