www.nist.gov and time.nist.gov DNSSEC Outage: 2016-08-25 to 2016-08-26
Updated: August 29, 2016
Overview
This page gives some details on the www.nist.gov and time.nist.gov DNSSEC outages from August 25 to August 26, 2016. It was not the first DNSSEC outage within nist.gov. This particular outage was caused by a bogus DNSSEC delegation from nist.gov to glb.nist.gov. (CNAMEs for www.nist.gov and time.nist.gov point to www.glb.nist.gov and ntp1.glb.nist.gov, respectively.) All other hosts under glb.nist.gov were failing as well.
Timeline / DNSViz
- 2016-08-25 17:27:07 UTC — first personally observed DNSSEC failure
- 2016-08-25 17:27:37 UTC — bogus DNSSEC delegation
- 2016-08-25 18:59:11 UTC — bogus DNSSEC delegation
- 2016-08-25 19:23:03 UTC — bogus DNSSEC delegation
- 2016-08-25 20:53:43 UTC — bogus DNSSEC delegation
- 2016-08-25 22:14:29 UTC — bogus DNSSEC delegation
- 2016-08-26 01:02:49 UTC — bogus DNSSEC delegation
- 2016-08-26 13:52:16 UTC — bogus DNSSEC delegation
- 2016-08-26 14:52:22 UTC — last personally observed DNSSEC failure
- 2016-08-26 16:29:39 UTC — bogus DNSSEC delegation
- 2016-08-26 18:12:40 UTC — DNSSEC outage over
OpenDNS & Google Public DNS
OpenDNS does not support DNSSEC, instead supporting DNSCurve. Google Public DNS currently supports only DNSSEC, and thus, Google's users saw SERVFAIL for queries under www.nist.gov during this outage.
With OpenDNS, which doesn't support DNSSEC, queries succeed:
$ dig www.nist.gov @resolver1.opendns.com.
; <<>> DiG 9.4.2-P2 <<>> www.nist.gov @resolver1.opendns.com.
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61620
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.nist.gov. IN A
;; ANSWER SECTION:
www.nist.gov. 627 IN CNAME www.glb.nist.gov.
www.glb.nist.gov. 30 IN A 129.6.13.54
;; Query time: 32 msec
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Thu Aug 25 17:27:17 2016
;; MSG SIZE rcvd: 68
With Google Public DNS, because of DNSSEC, queries fail:
$ dig +dnssec www.nist.gov. @8.8.8.8
; <<>> DiG 9.4.2-P2 <<>> +dnssec www.nist.gov. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 49473
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;www.nist.gov. IN A
;; Query time: 170 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Thu Aug 25 19:40:06 2016
;; MSG SIZE rcvd: 41
dnscheck
- dnscheck.labs.nic.cz archived this glb.nist.gov DNSSEC outage, saying "The zone glb.nist.gov has published DS records, but none of them work."
- dnscheck.iis.se archived this glb.nist.gov DNSSEC outage, saying "The zone glb.nist.gov has published DS records, but none of them work."
Zonemaster
- zonemaster.net archived this glb.nist.gov DNSSEC outage, noting that the "Delegation from parent to child is not properly signed."
- zonemaster.fr archived this glb.nist.gov DNSSEC outage, noting that the "Delegation from parent to child is not properly signed."
Twitter mentions
- @fanf (Tony Finch) said "Why do US gov domains do DNSSEC so badly? time.nist.gov broken..."
- @bortzmeyer (Stéphane Bortzmeyer) wrote "@usnistgov You should fix the #DNSSEC error on glb\.nist\gov URGENTLY. People using DNSSEC resolvers cannot visit the Web site or get time."
- @bortzmeyer also wrote: "glb\.nist\.gov : four DNSKEY, four DS, and zero intersection between the two sets. #DNSSEC"
- @hdais (Daisuke HIGASHI) wrote: "time.nist. gov がDNSSEC bogusだってよ"
Outages mailing list
This outage was brought up in the thread [outages] NIST (time.nist.gov, etc) DNSSEC bogus.
Unbound mailing list
The outage was also discussed in DNSSEC and time.nist.gov on the Unbound mailing list.
Logfile examples
- [1472146027] unbound[5652:0] info: validation failure <www.nist.gov. A IN>: no keys have a DS with algorithm RSASHA1-NSEC3-SHA1 from 129.6.13.8 for key glb.nist.gov. while building chain of trust
- [1472151534] unbound[19059:0] info: validation failure <www.nist.gov. A IN>: no keys have a DS with algorithm RSASHA1-NSEC3-SHA1 from 129.6.13.8 for key glb.nist.gov. while building chain of trust
- [1472151584] unbound[19059:0] info: validation failure <www.nist.gov. A IN>: key for validation glb.nist.gov. is marked as invalid because of a previous validation failure <www.nist.gov. A IN>: no keys have a DS with algorithm RSASHA1-NSEC3-SHA1 from 129.6.13.8 for key glb.nist.gov. while building chain of trust
- [1472152994] unbound[5652:0] info: validation failure <www.nist.gov. A IN>: no keys have a DS with algorithm RSASHA1-NSEC3-SHA1 from 129.6.13.8 for key glb.nist.gov. while building chain of trust
- [1472223142] unbound[17390:0] info: validation failure <www.nist.gov. A IN>: key for validation glb.nist.gov. is marked as invalid because of a previous validation failure <www.nist.gov. A IN>: no keys have a DS with algorithm RSASHA1-NSEC3-SHA1 from 129.6.13.8 for key glb.nist.gov. while building chain of trust