berkeley.edu DNSSEC Outage: 2019-04-10
Date: April 10, 2019
Overview
This page gives some details on the berkeley.edu DNSSEC outage on April 10, 2019. The University of California at Berkeley has over 42,000 students, an endowment of 4.6 billion US dollars, and is ranked among the world's top 20 universities.
Timeline / DNSViz
- DNSViz is currently down again...
DNSSEC Debugger
Unlike DNSViz, Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from April 10, 2019:

DNS-OARC: Unbound
DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNS SEC.
With DNS-OARC's Unbound instance, because of DNSSEC, queries fail:
$ dig +dnssec a berkeley.edu. @184.105.193.74
; <<>> DiG 9.4.2-P2 <<>> +dnssec a berkeley.edu. @184.105.193.74
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 5008
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;berkeley.edu. IN A
;; Query time: 37 msec
;; SERVER: 184.105.193.74#53(184.105.193.74)
;; WHEN: Wed Apr 10 13:09:32 2019
;; MSG SIZE rcvd: 41
You have to disable DNSSEC to make DNS queries work:
$ dig +cd a berkeley.edu. @184.105.193.74
; <<>> DiG 9.4.2-P2 <<>> +cd a berkeley.edu. @184.105.193.74
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14274
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;berkeley.edu. IN A
;; ANSWER SECTION:
berkeley.edu. 300 IN A 35.163.72.93
;; Query time: 20 msec
;; SERVER: 184.105.193.74#53(184.105.193.74)
;; WHEN: Wed Apr 10 13:09:32 2019
;; MSG SIZE rcvd: 46
Zonemaster
- zonemaster.iis.se archived "No DS record had a DNSKEY with a matching keytag."
drill trace
Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):
;; Domain: berkeley.edu.
;; Signature ok but no chain to a trusted key or ds record
[S] berkeley.edu. 172800 IN DNSKEY 256 3 10 ;{id = 54122 (zsk), size = 1024b}
berkeley.edu. 172800 IN DNSKEY 256 3 10 ;{id = 5660 (zsk), size = 1024b}
berkeley.edu. 172800 IN DNSKEY 257 3 10 ;{id = 3456 (ksk), size = 2048b}
[S] berkeley.edu. 300 IN A 35.163.72.93
;;[S] self sig OK; [B] bogus; [T] trusted
Logfile examples
- [1554901769] unbound[92898:0] info: validation failure <berkeley.edu. A IN>: no keys have a DS with algorithm RSASHA512 from 128.32.136.14 for key berkeley.edu. while building chain of trust
- [1554902567] unbound[92898:0] info: validation failure <math.berkeley.edu. A IN>: no keys have a DS with algorithm RSASHA512 from 128.32.136.3 for key berkeley.edu. while building chain of trust
- [1554926153] unbound[92898:0] info: validation failure <eecs.berkeley.edu. A IN>: no keys have a DS with algorithm RSASHA512 from 128.32.136.14 for key berkeley.edu. while building chain of trust
- [1554927005] unbound[92898:0] info: validation failure <stat.berkeley.edu. A IN>: no keys have a DS with algorithm RSASHA512 from 192.107.102.142 for key berkeley.edu. while building chain of trust