af.mil DNSSEC Outage: 2015-06-24
Updated: June 24, 2015
Overview
This page gives some details on the af.mil DNSSEC outage on June 24, 2015. The outage was fixed by removing DNSSEC support (unsigning).
Verisign's DNSSEC Debugger
Verisign doesn't archive test results, unlike DNSViz. So here's a screenshot I took on June 24, 2015:
Timeline / DNSViz
- 2015-06-24 14:47:01 UTC — af.mil/SOA bogus RRSIG
- 2015-06-24 14:49:24 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 14:53:29 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 14:53:50 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 14:58:05 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 15:13:06 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 15:28:33 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 15:34:13 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 16:14:29 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 16:40:15 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 16:52:09 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 17:19:28 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 17:43:15 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 18:44:27 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 19:14:44 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 19:20:08 UTC — no valid RRSIGs, bogus DNSSEC delegation
- 2015-06-24 19:24:52 UTC — af.mil unsigned, DNSSEC outage over
OpenDNS & Google Public DNS
OpenDNS does not support DNSSEC, instead supporting DNSCurve. Google Public DNS currently supports only DNSSEC, and thus, Google's users saw SERVFAIL for queries under af.mil during this outage.
With OpenDNS, queries succeed:
$ dig www.af.mil. @resolver1.opendns.com.
; <<>> DiG 9.4.2-P2 <<>> www.af.mil. @resolver1.opendns.com.
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61894
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.af.mil. IN A
;; ANSWER SECTION:
www.af.mil. 52 IN CNAME www.af.mil.edgesuite.net.
www.af.mil.edgesuite.net. 52 IN CNAME a499.dscb.akamai.net.
a499.dscb.akamai.net. 20 IN A 23.212.53.196
a499.dscb.akamai.net. 20 IN A 23.212.53.206
;; Query time: 45 msec
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Wed Jun 24 14:58:58 2015
;; MSG SIZE rcvd: 129
With Google Public DNS, with DNSSEC, queries fail:
$ dig www.af.mil. @8.8.8.8
; <<>> DiG 9.4.2-P2 <<>> www.af.mil. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 42737
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.af.mil. IN A
;; Query time: 127 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Wed Jun 24 14:59:04 2015
;; MSG SIZE rcvd: 28
dnscheck
dnscheck.labs.nic.cz archived a DNSSEC outage at 2015-06-24 15:17:51 (requires javascript).
dnscheck.iis.se reported that it "could not connect to database." However later, the following af.mil DNSSEC outages were available:
- 2015-06-24 14:51:42 — "DS found at parent, but no DNSKEY found at child."
- 2015-06-24 15:17:16 — "DS found at parent, but no DNSKEY found at child."
- 2015-06-24 15:47:09 — "DS found at parent, but no DNSKEY found at child."
Zonemaster
Zonemaster (also run by iis.se) was also having trouble, but finally decided to archive this af.mil DNSSEC outage.
Logfile examples
- [1435157469] unbound[12254:0] info: validation failure <www.af.mil. A IN>: DS got unsigned CNAME answer from 88.221.81.194 and 96.7.50.66 and 132.3.25.10 for DS www.af.mil. while building chain of trust
- [1435159098] unbound[12254:0] info: validation failure <mcconnell.af.mil. A IN>: no signatures from 132.3.13.10 for DS mcconnell.af.mil. while building chain of trust
- [1435161434] unbound[13228:0] info: validation failure <www.patrick.af.mil. A IN>: No DNSKEY record from 132.3.65.10 for key af.mil. while building chain of trust
- [1435161521] unbound[13228:0] info: validation failure <www.wpafb.af.mil. A IN>: No DNSKEY record from 132.3.45.10 for key af.mil. while building chain of trust
- [1435161586] unbound[13228:0] info: validation failure <kirtland.af.mil. MX IN>: No DNSKEY record from 132.3.45.10 for key af.mil. while building chain of trust
- [1435162789] unbound[13228:0] info: validation failure <af.mil. NS IN>: No DNSKEY record from 132.3.13.10 for key af.mil. while building chain of trust
- [1435163392] unbound[6654:0] info: validation failure <www.af.mil. A IN>: No DNSKEY record from 132.3.13.10 for key af.mil. while building chain of trust
- [1435163668] unbound[6654:0] info: validation failure <www.offutt.af.mil. A IN>: No DNSKEY record from 132.3.29.10 for key af.mil. while building chain of trust
- [1435163952] unbound[6654:0] info: validation failure <www.af.mil. A IN>: No DNSKEY record from 132.3.13.10 for key af.mil. while building chain of trust
- [1435167052] unbound[6654:0] info: validation failure <www.af.mil. A IN>: No DNSKEY record from 132.3.65.10 for key af.mil. while building chain of trust
- [1435167147] unbound[6654:0] info: validation failure <af.mil. NS IN>: No DNSKEY record from 132.3.29.10 for key af.mil. while building chain of trust
- [1435170688] unbound[6654:0] info: validation failure <www.af.mil. A IN>: No DNSKEY record from 132.3.29.10 for key af.mil. while building chain of trust
- [1435174154] unbound[6654:0] info: validation failure <www.af.mil. A IN>: No DNSKEY record from 132.3.45.10 for key af.mil. while building chain of trust