usfj.mil DNSSEC Outage: 2016-04-08 to 2016-04-09
Updated: April 11, 2016
Overview
This page gives some details on the usfj.mil DNSSEC outage from April 8 to April 9, 2016. It was the 11th DNSSEC outage of the year for usfj.mil.
Note: usfj.mil had just had an outage from April 7 to April 8. It was fixed a few hours before this outage began. After a few hours of uptime, the old, expired DNSKEY RRSIG from that outage was re-deployed, causing a new distinct DNSSEC outage. Crazy but true.
Timeline / DNSViz
- (2016-04-07 21:48:24 UTC) — RRSIGs expire. This old DNSKEY was put back into service, causing the outage
- 2016-04-08 22:24:27 UTC — first personally observed DNSSEC failure
- 2016-04-08 23:00:06 UTC — expired RRSIGs
- 2016-04-09 06:46:03 UTC — some expired RRSIGs, but most "important" queries succeed
OpenDNS and Google Public DNS
OpenDNS does not support DNSSEC, instead supporting DNSCurve. Google Public DNS currently supports only DNSSEC, and thus, Google's users saw SERVFAIL for queries under usfj.mil during this outage.
With OpenDNS, queries succeed:
$ dig www.usfj.mil. @resolver1.opendns.com.
; <<>> DiG 9.4.2-P2 <<>> www.usfj.mil. @resolver1.opendns.com.
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44158
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.usfj.mil. IN A
;; ANSWER SECTION:
www.usfj.mil. 3956 IN CNAME www.dodpw.defense.gov.edgesuite.net.
www.dodpw.defense.gov.edgesuite.net. 300 IN CNAME a1603.dscb.akamai.net.
a1603.dscb.akamai.net. 20 IN A 23.219.160.89
a1603.dscb.akamai.net. 20 IN A 23.219.160.51
;; Query time: 276 msec
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Fri Apr 8 23:00:04 2016
;; MSG SIZE rcvd: 143
With Google Public DNS, because of DNSSEC, queries fail:
$ dig +dnssec www.usfj.mil. @8.8.8.8
; <<>> DiG 9.4.2-P2 <<>> +dnssec www.usfj.mil. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 53358
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;www.usfj.mil. IN A
;; Query time: 725 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Apr 8 23:00:05 2016
;; MSG SIZE rcvd: 41
dnscheck
dnscheck.labs.nic.cz archived a DNSSEC outage at 2016-04-08 23:01:42 (requires javascript).
dnscheck.iis.se archived a DNSSEC outage at 2016-04-08 23:00:56 (requires javascript).
zonemaster
zonemaster archived this usfj.mil DNSSEC outage.
Logfile examples
- [1460154267] unbound[19043:0] info: validation failure <usfj.mil. A IN>: signature expired from 215.1.46.29 for key usfj.mil. while building chain of trust