dotat.at DNSSEC Outage: 2017-11-12
Updated: November 15, 2017
Overview
This page gives some details on the dotat.at DNSSEC outage on November 12, 2017. It is notable for being the domain of Tony Finch, who is one of the most capable DNSSEC people there is.
Timeline / DNSViz
- 2017-11-12 10:47:33 UTC — expired RRSIGs
- 2017-11-12 10:47:42 UTC — expired RRSIGs
- 2017-11-12 10:51:38 UTC — expired RRSIGs
- 2017-11-12 10:52:57 UTC — expired RRSIGs
- 2017-11-12 10:53:04 UTC — expired RRSIGs
- 2017-11-12 15:09:33 UTC — expired RRSIGs
- 2017-11-12 20:57:33 UTC — last personally observed DNSSEC failure
DNSSEC Debugger
Unlike DNSViz, Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from November 12, 2017:
Google Public DNS: with and without DNSSEC
DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.
With DNSSEC, DNS queries fail:
$ dig +dnssec a dotat.at. @8.8.8.8
; <<>> DiG 9.4.2-P2 <<>> +dnssec a dotat.at. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 28101
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;dotat.at. IN A
;; Query time: 221 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Nov 12 10:46:41 2017
;; MSG SIZE rcvd: 37
You have to disable DNSSEC to make DNS queries work:
$ dig +cd a dotat.at. @8.8.8.8
; <<>> DiG 9.4.2-P2 <<>> +cd a dotat.at. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6765
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;dotat.at. IN A
;; ANSWER SECTION:
dotat.at. 3599 IN A 212.13.197.229
;; Query time: 146 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Nov 12 10:46:41 2017
;; MSG SIZE rcvd: 42
Zonemaster
- zonemaster.net archived "RRSIG with keytag 5677 and covering type(s) DNSKEY expires at : Tue Nov 14 10:32:06 2017."
- zonemaster.fr archived "RRSIG with keytag 5677 and covering type(s) DNSKEY expires at : Tue Nov 14 10:32:06 2017."
drill trace
Since DNSSEC contains so much garbage, I put the complete drill trace into its own file with the relevant portion below (emphasis added):
;; Domain: dotat.at.
[T] dotat.at. 3600 IN DNSKEY 256 3 5 ;{id = 56700 (zsk), size = 1024b}
dotat.at. 3600 IN DNSKEY 257 3 5 ;{id = 5677 (ksk), size = 2048b}
[B] dotat.at. 3600 IN A 212.13.197.229
;; Error: DNSSEC signature has expired
;;[S] self sig OK; [B] bogus; [T] trusted
The outage was acknowledged on Twitter:
Logfile examples
- [1510491082] unbound[97396:0] info: validation failure <dotat.at. A IN>: signature expired from 204.42.254.5
- [1510491454] unbound[97396:0] info: validation failure <www.dotat.at. A IN>: signature expired for CNAME from 185.43.209.139 and 131.111.57.57
- [1510499798] unbound[97396:0] info: validation failure <dotat.at. A IN>: signature expired from 131.111.57.57
- [1510500204] unbound[97396:0] info: validation failure <www.dotat.at. A IN>: signature expired for CNAME from 109.238.48.13 and 131.111.57.57
- [1510520253] unbound[14301:0] info: validation failure <dotat.at. A IN>: signature expired from 204.42.254.5