usfj.mil DNSSEC Outage: 2015-01-03 to 2015-01-07
Updated: January 7, 2015
Overview
This page gives some details on the usfj.mil DNSSEC outage from January 3 to January 7, 2015. The outage lasted approximately 94 hours.
Timeline / DNSViz
- 2015-01-03 05:34:48 UTC: RRSIGs expire
- 2015-01-03 07:04:09 UTC: expired RRSIGs
- 2015-01-03 16:49:52 UTC: expired RRSIGs
- 2015-01-04 02:33:18 UTC: expired RRSIGs
- 2015-01-04 05:15:39 UTC: expired RRSIGs
- 2015-01-04 12:20:10 UTC: expired RRSIGs
- 2015-01-04 17:11:18 UTC: expired RRSIGs
- 2015-01-04 20:20:14 UTC: expired RRSIGs
- 2015-01-05 05:02:50 UTC: expired RRSIGs
- 2015-01-05 18:52:28 UTC: expired RRSIGs
- 2015-01-05 22:16:55 UTC: expired RRSIGs
- 2015-01-06 04:26:39 UTC: expired RRSIGs
- 2015-01-06 07:48:47 UTC: expired RRSIGs
- 2015-01-06 18:01:18 UTC: expired RRSIGs
- 2015-01-06 23:07:04 UTC: expired RRSIGs
- 2015-01-07 03:16:44 UTC: Last observed outage logged by unbound; expired RRSIGs
- 2015-01-07 04:23:55 UTC: Outage debris, but not significant enough to cause failures
- 2015-01-07 20:03:10 UTC: Outage debris, but not significant enough to cause failures
Verisign's DNSSEC Debugger
Here's a screenshot I took on January 3, 2015, of the DNSSEC Debugger output:
OpenDNS vs. Google Public DNS
While Google Public DNS supports DNSSEC, OpenDNS supports the superior DNSCurve, which is (among other advantages) immune to DNSSEC failures. During this outage, Google failed to resolve names under usfj.mil while OpenDNS worked normally.
With OpenDNS, queries succeed:
$ dig www.usfj.mil. @resolver1.opendns.com
; <<>> DiG 9.4.2-P2 <<>> www.usfj.mil. @resolver1.opendns.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15882
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.usfj.mil. IN A
;; ANSWER SECTION:
www.usfj.mil. 3238 IN A 215.1.46.23
;; Query time: 250 msec
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Sun Jan 4 23:09:57 2015
;; MSG SIZE rcvd: 46
$ dig www.usfj.mil. @8.8.8.8
; <<>> DiG 9.4.2-P2 <<>> www.usfj.mil. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 41676
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.usfj.mil. IN A
;; Query time: 578 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Jan 4 23:10:18 2015
;; MSG SIZE rcvd: 30
Logfile examples
- [1420263805] unbound[3779:0] info: validation failure <usfj.mil. NS IN>: signature expired from 215.1.46.29 for key usfj.mil. while building chain of trust
- [1420434509] unbound[3779:0] info: validation failure <www.usfj.mil. A IN>: signature expired from 215.1.46.29 for key usfj.mil. while building chain of trust
- [1420438204] unbound[3779:0] info: validation failure <dns01.usfj.mil. A IN>: signature expired from 215.1.46.29 for key usfj.mil. while building chain of trust
- [1420496345] unbound[3779:0] info: validation failure <usfj.mil. AAAA IN>: signature expired from 199.211.150.66 for key usfj.mil. while building chain of trust
- [1420586029] unbound[3779:0] info: validation failure <usfj.mil. DNSKEY IN>: signature expired from 199.211.150.66 for key usfj.mil. while building chain of trust
- [1420591953] unbound[3779:0] info: validation failure <usfj.mil. A IN>: key for validation usfj.mil. is marked as invalid because of a previous validation failure <usfj.mil. NS IN>: signature expired from 215.1.46.29 for key usfj.mil. while building chain of trust
- [1420600604] unbound[3779:0] info: validation failure <usfj.mil. NS IN>: signature expired from 215.1.46.29 for key usfj.mil. while building chain of trust