temple.edu DNSSEC Outage: 2018-09-23
Date: September 23, 2018
Overview
This page gives some details on the temple.edu (Temple University) DNSSEC outage on September 23, 2018. Temple has around 40,000 students.
Timeline / DNSViz
- 2018-09-23 05:35:24 UTC — bogus DNSSEC delegation
- 2018-09-23 05:36:36 UTC — bogus DNSSEC delegation
- 2018-09-23 13:30:43 UTC — bogus DNSSEC delegation
- 2018-09-23 13:45:09 UTC — bogus DNSSEC delegation
- 2018-09-23 13:51:28 UTC — bogus DNSSEC delegation
DNSSEC Debugger
Unlike DNSViz, Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from September 23, 2018.
Google Public DNS: with and without DNSSEC
DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC. With DNSSEC, DNS queries result in SERVFAIL:
$ dig +dnssec ns temple.edu. @8.8.8.8
; <<>> DiG 9.4.2-P2 <<>> +dnssec ns temple.edu. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 7318
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;temple.edu. IN NS
;; Query time: 147 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Sep 23 05:35:09 2018
;; MSG SIZE rcvd: 39
$ dig +cd ns temple.edu. @8.8.8.8
; <<>> DiG 9.4.2-P2 <<>> +cd ns temple.edu. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60924
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;temple.edu. IN NS
;; ANSWER SECTION:
temple.edu. 3599 IN NS ns1.temple.edu.
temple.edu. 3599 IN NS ns2.temple.edu.
;; Query time: 78 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Sep 23 05:35:09 2018
;; MSG SIZE rcvd: 64
drill trace
Since DNSSEC contains so much garbage, I put the complete drill trace into its own file with the relevant portion below (emphasis added):
;; Domain: temple.edu.
;; Signature ok but no chain to a trusted key or ds record
[S] temple.edu. 3600 IN DNSKEY 256 3 7 ;{id = 46170 (zsk), size = 1024b}
temple.edu. 3600 IN DNSKEY 257 3 7 ;{id = 37155 (ksk), size = 2048b}
temple.edu. 3600 IN DNSKEY 256 3 7 ;{id = 32274 (zsk), size = 1024b}
[S] temple.edu. 86400 IN A 155.247.166.60
;;[S] self sig OK; [B] bogus; [T] trusted
Logfile examples
- [1537680909] unbound[11465:0] info: validation failure <temple.edu. A IN>: no keys have a DS with algorithm RSASHA1-NSEC3-SHA1 from 155.247.166.2 for key temple.edu. while building chain of trust
- [1537710192] unbound[11465:0] info: validation failure <temple.edu. A IN>: no keys have a DS with algorithm RSASHA1-NSEC3-SHA1 from 155.247.19.2 for key temple.edu. while building chain of trust