af.mil DNSSEC Outage: 2019-05-09

Date: May 9, 2019

Overview

This page gives some details on the af.mil DNSSEC outage on May 9, 2019. This is one of numerous af.mil DNSSEC outages, and this particular one began only 3 days after the last one ended. The US Air Force has over half a million employees, a budget of $161 Billion, 170 military satellites and enough nuclear weapons to end the human species.

Timeline / DNSViz

DNSViz has been down for about a month at the time of this writing. DNSSEC makes its users completely give up.

I've included a screenshot of DNSViz output since DNSSEC people don't care if things work or not.

May 9, 2019 DNSViz output for af.mil

DNSSEC Debugger

Unlike DNSViz, Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from May 9, 2019:

May 9, 2019 af.mil DNSSEC outage

Google DNS: with and without DNSSEC

DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.

With DNSSEC, DNS queries fail:

$ dig +dnssec a www.af.mil. @8.8.8.8

; <<>> DiG 9.4.2-P2 <<>> +dnssec a www.af.mil. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 46919
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;www.af.mil. IN A

;; Query time: 106 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Thu May 9 19:08:27 2019
;; MSG SIZE rcvd: 39

You have to disable DNSSEC to make DNS queries work:

$ dig +cd a www.af.mil. @8.8.8.8

; <<>> DiG 9.4.2-P2 <<>> +cd a www.af.mil. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23888
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.af.mil. IN A

;; ANSWER SECTION:
www.af.mil. 19 IN CNAME www.afpw.af.mil.edgekey.net.
www.afpw.af.mil.edgekey.net. 299 IN CNAME e11218.dscb.akamaiedge.net.
e11218.dscb.akamaiedge.net. 19 IN A 23.39.18.201

;; Query time: 186 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Thu May 9 19:08:27 2019
;; MSG SIZE rcvd: 122

dns.google.com

dns.google.com is related to but separate from Google Public DNS. During this DNSSEC outage, dns.google.com showed the following for www.af.mil:

May 9, 2019 dns.google.com output for www.af.mil

This data is also saved by archive.org.

drill trace

Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):

;; Domain: af.mil.
[T] af.mil. 212811 IN DNSKEY 257 3 8 ;{id = 40582 (ksk), size = 2048b}
af.mil. 212811 IN DNSKEY 256 3 8 ;{id = 10371 (zsk), size = 2048b}
[U] No data found for: af.mil. type A
;;[S] self sig OK; [B] bogus; [T] trusted

Logfile examples