uchicago.edu DNSSEC Outage: 2019-01-26

Date: January 26, 2019

Overview

This page gives some details on the uchicago.edu (University of Chicago) DNSSEC outage on January 26, 2019.

Timeline / DNSViz

DNSSEC Debugger

Unlike DNSViz, Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from January 26, 2019:

January 26, 2019 uchicago.edu DNSSEC outage

Zonemaster

Google Public DNS: with and without DNSSEC

DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.

With DNSSEC, DNS queries fail:

$ dig +dnssec www.uchicago.edu. @8.8.8.8

; <<>> DiG 9.4.2-P2 <<>> +dnssec www.uchicago.edu. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 56538
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;www.uchicago.edu. IN A

;; Query time: 120 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sat Jan 26 19:18:51 2019
;; MSG SIZE rcvd: 45

You have to disable DNSSEC to make DNS queries work:

$ dig +cd www.uchicago.edu. @8.8.8.8

; <<>> DiG 9.4.2-P2 <<>> +cd www.uchicago.edu. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51534
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.uchicago.edu. IN A

;; ANSWER SECTION:
www.uchicago.edu. 299 IN CNAME wsee.elb.uchicago.edu.
wsee.elb.uchicago.edu. 59 IN A 54.173.214.197
wsee.elb.uchicago.edu. 59 IN A 34.234.135.37
wsee.elb.uchicago.edu. 59 IN A 18.234.4.223

;; Query time: 122 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sat Jan 26 19:18:51 2019
;; MSG SIZE rcvd: 105

drill trace

Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):

;; Domain: uchicago.edu.
;; Signature ok but no chain to a trusted key or ds record
[S] uchicago.edu. 172800 IN DNSKEY 256 3 7 ;{id = 28700 (zsk), size = 1024b}
uchicago.edu. 172800 IN DNSKEY 256 3 7 ;{id = 56057 (zsk), size = 1024b}
uchicago.edu. 172800 IN DNSKEY 257 3 7 ;{id = 49530 (ksk), size = 2048b}
uchicago.edu. 172800 IN DNSKEY 256 3 7 ;{id = 36809 (zsk), size = 1024b}
[S] uchicago.edu. 3600 IN A 34.200.129.209
;;[S] self sig OK; [B] bogus; [T] trusted

Logfile examples