e164.arpa DNSSEC Outage: 2017-06-21

Updated: June 22, 2017

Overview

This page gives some details on the e164.arpa DNSSEC outage on June 21, 2017. This is not the first DNSSEC outage for e164.arpa.

Timeline / DNSViz

DNSSEC Debugger

Unlike DNSViz, Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from June 21, 2017:

June 21, 2017 e164.arpa DNSSEC outage

OpenDNS & Google Public DNS

OpenDNS does not support DNSSEC, and instead supports DNSCurve. Google Public DNS currently supports only DNSSEC, and thus, Google's users saw SERVFAIL for queries under e164.arpa during this outage.

With OpenDNS, without DNSSEC, queries succeed:

$ dig ns e164.arpa. @resolver1.opendns.com.

; <<>> DiG 9.4.2-P2 <<>> ns e164.arpa. @resolver1.opendns.com.
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32214
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;e164.arpa. IN NS

;; ANSWER SECTION:
e164.arpa. 172800 IN NS tinnie.arin.net.
e164.arpa. 172800 IN NS sec3.apnic.net.
e164.arpa. 172800 IN NS ns3.afrinic.net.
e164.arpa. 172800 IN NS ns3.lacnic.net.
e164.arpa. 172800 IN NS sns-pb.isc.org.
e164.arpa. 172800 IN NS pri.authdns.ripe.net.

;; Query time: 1 msec
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Wed Jun 21 12:18:37 2017
;; MSG SIZE rcvd: 191

With Google Public DNS, because of DNSSEC, queries fail:

$ dig +dnssec ns e164.arpa. @8.8.8.8

; <<>> DiG 9.4.2-P2 <<>> +dnssec ns e164.arpa. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 59858
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;e164.arpa. IN NS

;; Query time: 86 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Wed Jun 21 12:18:38 2017
;; MSG SIZE rcvd: 38

drill trace

Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):

;; Domain: e164.arpa.
[B] e164.arpa. 3600 IN DNSKEY 256 3 8 ;{id = 3511 (zsk), size = 1024b}
e164.arpa. 3600 IN DNSKEY 257 3 8 ;{id = 44227 (ksk), size = 2048b}
e164.arpa. 3600 IN DNSKEY 256 3 8 ;{id = 53106 (zsk), size = 1024b}
[B] Error verifying denial of existence for e164.arpa. type A: No keys with the keytag and algorithm from the RRSIG found
;;[S] self sig OK; [B] bogus; [T] trusted

Logfile examples