tamu.edu DNSSEC Outage: 2023-01-05
Date: January 5, 2023
Overview
This page gives some details on the tamu.edu (Texas A&M University) DNSSEC outage on January 5, 2023. Texas A&M has around 72,000 students.
Timeline / DNSViz
- 2023-01-05 20:12:33 UTC — Expired RRSIGs
- 2023-01-05 20:09:30 UTC — Expired RRSIGs
CloudFlare Public DNS: with and without DNSSEC
DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC. With DNSSEC, DNS queries result in SERVFAIL:
$ dig +dnssec caa tamu.edu. @1.1.1.1.
; <<>> dig 9.10.8-P1 <<>> +dnssec caa tamu.edu. @1.1.1.1.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 35790
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1232
; EDE: 7 (Signature Expired): 66 6f 72 20 44 4e 53 4b 45 59 20 74 61 6d 75 2e 65 64 75 2e 2c 20 69 64 20 3d 20 33 30 32 37 34 3a 20 52 52 53 49 47 20 74 61 6d 75 2e 65 64 75 2e 2c 20 65 78 70 69 72 61 74 69 6f 6e 20 3d 20 31 36 37 32 34 30 39 31 30 33 ("for DNSKEY tamu.edu., id = 30274: RRSIG tamu.edu., expiration = 1672409103")
;; QUESTION SECTION:
;tamu.edu. IN CAA
;; Query time: 331 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Thu Jan 05 20:12:26 UTC 2023
;; MSG SIZE rcvd: 117
$ dig +cd caa tamu.edu. @1.1.1.1.
; <<>> dig 9.10.8-P1 <<>> +cd caa tamu.edu. @1.1.1.1.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30102
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; EDE: 7 (Signature Expired): 66 6f 72 20 44 4e 53 4b 45 59 20 74 61 6d 75 2e 65 64 75 2e 2c 20 69 64 20 3d 20 33 30 32 37 34 3a 20 52 52 53 49 47 20 74 61 6d 75 2e 65 64 75 2e 2c 20 65 78 70 69 72 61 74 69 6f 6e 20 3d 20 31 36 37 32 34 30 39 31 30 33 ("for DNSKEY tamu.edu., id = 30274: RRSIG tamu.edu., expiration = 1672409103")
;; QUESTION SECTION:
;tamu.edu. IN CAA
;; AUTHORITY SECTION:
tamu.edu. 900 IN SOA csce-info-grid.net.tamu.edu. infoblox.tamu.edu. 3183055 14400 1440 2419200 900
;; Query time: 273 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Thu Jan 05 20:12:26 UTC 2023
;; MSG SIZE rcvd: 181
drill trace
Since DNSSEC contains so much garbage, I put the complete drill trace into its own file with the relevant portion below (emphasis added):
;; Domain: tamu.edu.
;; Signature ok but no chain to a trusted key or ds record
[S] tamu.edu. 172800 IN DNSKEY 256 3 8 ;{id = 43116 (zsk), size = 2048b}
tamu.edu. 172800 IN DNSKEY 257 3 5 ;{id = 30274 (ksk), size = 2048b}
tamu.edu. 172800 IN DNSKEY 257 3 8 ;{id = 54682 (ksk), size = 2048b}
tamu.edu. 172800 IN DNSKEY 256 3 8 ;{id = 14109 (zsk), size = 2048b}
tamu.edu. 172800 IN DNSKEY 256 3 5 ;{id = 18378 (zsk), size = 1024b}
tamu.edu. 172800 IN DNSKEY 256 3 8 ;{id = 8370 (zsk), size = 2048b}
tamu.edu. 172800 IN DNSKEY 256 3 5 ;{id = 57843 (zsk), size = 1024b}
tamu.edu. 172800 IN DNSKEY 257 3 5 ;{id = 32956 (ksk), size = 2048b}
tamu.edu. 172800 IN DNSKEY 256 3 5 ;{id = 55857 (zsk), size = 1024b}
[S] tamu.edu. 300 IN A 165.91.22.70
;;[S] self sig OK; [B] bogus; [T] trusted; [U] unsigned
Logfile example
- [1672949535] unbound[28915:0] info: validation failure <tamu.edu. A IN>: signature expired from 192.195.94.166 for key tamu.edu. while building chain of trust