lsu.edu DNSSEC Outage: 2019-02-28

Date: February 28, 2019

Overview

This page gives some details on the lsu.edu (Louisiana State University) DNSSEC outage on February 28, 2019. LSU has over 30,000 students.

Timeline / DNSViz

DNSViz is at the time of this writing having a multi-week outage related to hardware being shipped to another location. The downtime was only supposed to take a week or so, but persists. I contend that DNSSEC proponents become numb to outages, which spreads to other areas of their work, so that multiple weeks of outage just seems normal and isn't urgent at all.

DNSSEC Debugger

Unlike DNSViz, Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from February 28, 2019:

February 28, 2019 lsu.edu DNSSEC outage

Zonemaster

Note: Zonemaster requires javascript.

Google Public DNS: with and without DNSSEC

DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.

With DNSSEC, DNS queries fail:

$ dig +dnssec mx lsu.edu. @8.8.8.8

; <<>> DiG 9.4.2-P2 <<>> +dnssec mx lsu.edu. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 39501
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;lsu.edu. IN MX

;; Query time: 155 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Thu Feb 28 06:20:36 2019
;; MSG SIZE rcvd: 36

You have to disable DNSSEC to make DNS queries work:

$ dig +cd mx lsu.edu. @8.8.8.8

; <<>> DiG 9.4.2-P2 <<>> +cd mx lsu.edu. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27474
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;lsu.edu. IN MX

;; ANSWER SECTION:
lsu.edu. 1131 IN MX 0 lsu-edu.mail.protection.outlook.com.

;; Query time: 8 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Thu Feb 28 06:20:36 2019
;; MSG SIZE rcvd: 76

drill trace

Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):

;; Domain: lsu.edu.
;; Signature ok but no chain to a trusted key or ds record
[S] lsu.edu. 7200 IN DNSKEY 256 3 5 ;{id = 36310 (zsk), size = 1024b}
lsu.edu. 7200 IN DNSKEY 257 3 5 ;{id = 23638 (ksk), size = 2048b}
[S] lsu.edu. 7200 IN A 130.39.6.220
;;[S] self sig OK; [B] bogus; [T] trusted

Logfile examples