nist.gov DNSSEC Outage: 2022-06-21
Date: June 21, 2022
Overview
This page gives some details on the nist.gov DNSSEC outage on June 21, 2022.
Timeline / DNSViz
- 2022-06-21 15:01:33 UTC — first personally observed nist.gov DNSSEC failure
- 2022-06-21 15:03:33 UTC — No DNSKEYs
Cloudflare DNS: with and without DNSSEC
DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.
With DNSSEC, DNS queries fail:
$ dig +dnssec mx nist.gov. @1.1.1.1.
; <<>> DiG 9.11.5-P4-5.1+deb10u7-Debian <<>> +dnssec mx nist.gov. @1.1.1.1.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 9818
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1232
; OPT=15: 00 16 74 69 6d 65 20 6c 69 6d 69 74 20 65 78 63 65 65 64 65 64 ("..time limit exceeded")
;; QUESTION SECTION:
;nist.gov. IN MX
;; Query time: 4005 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Tue Jun 21 11:01:59 EDT 2022
;; MSG SIZE rcvd: 62
You have to disable DNSSEC to make DNS queries work:
$ dig +cd mx nist.gov. @1.1.1.1.
; <<>> DiG 9.11.5-P4-5.1+deb10u7-Debian <<>> +cd mx nist.gov. @1.1.1.1.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56006
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;nist.gov. IN MX
;; ANSWER SECTION:
nist.gov. 1800 IN MX 0 nist-gov.mail.protection.outlook.com.
;; Query time: 914 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Tue Jun 21 11:01:59 EDT 2022
;; MSG SIZE rcvd: 89
drill trace
Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):
[T] nist.gov. 3600 IN DS 34014 7 2 f378cdaf7b41bea12ca2732a8e203dfc8458331b6fb9ea8a666abeac4de1def0
nist.gov. 3600 IN DS 35565 7 2 a051b24a47293959eee0d5e21d74d2bd60b814df03237dfd60ffdd1dd6aa25ae
nist.gov. 3600 IN DS 34014 7 1 ce7f1942de1c96f123fd32c7a46524cc75ac8b8c
nist.gov. 3600 IN DS 35565 7 1 db68a6356c64a8cc24357d3fe42e1b4194a96245
;; Domain: nist.gov.
[U] No data found for: nist.gov. type A
;;[S] self sig OK; [B] bogus; [T] trusted
Logfile examples
- [1655823693] unbound[10256:0] info: validation failure <nist.gov. A IN>: No DNSKEY record for key nist.gov. while building chain of trust
- [1655823693] unbound[10256:0] info: validation failure <www.nist.gov. A IN>: No DNSKEY record for key nist.gov. while building chain of trust
- [1655823707] unbound[10256:0] info: validation failure <time.nist.gov. A IN>: No DNSKEY record for key nist.gov. while building chain of trust