xfinity.com DNSSEC Outage: 2017-09-28

Date: September 28, 2017

Overview

This page gives some details on the xfinity.com DNSSEC outage on September 28, 2017.

Timeline / DNSViz

DNSSEC Debugger

Unlike DNSViz, Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from September 28, 2017:

September 28, 2017 xfinity.com DNSSEC outage

drill trace

Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):

;; Domain: xfinity.com.
;; Signature ok but no chain to a trusted key or ds record
[S] xfinity.com. 3600 IN DNSKEY 256 3 5 ;{id = 55812 (zsk), size = 1024b}
xfinity.com. 3600 IN DNSKEY 257 3 5 ;{id = 60208 (ksk), size = 2048b}
[S] xfinity.com. 7200 IN A 68.87.41.40
xfinity.com. 7200 IN A 96.114.21.40
xfinity.com. 7200 IN A 96.114.14.140
;;[S] self sig OK; [B] bogus; [T] trusted

DNS-OARC: with and without DNSSEC

DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.

With DNSSEC, DNS queries fail:

$ dig +dnssec a xfinity.com. @184.105.193.73

; <<>> DiG 9.4.2-P2 <<>> +dnssec a xfinity.com. @184.105.193.73
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 60754
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;xfinity.com. IN A

;; Query time: 568 msec
;; SERVER: 184.105.193.73#53(184.105.193.73)
;; WHEN: Thu Sep 28 14:10:49 2017
;; MSG SIZE rcvd: 40

You have to disable DNSSEC to make DNS queries work:

$ dig +cd a xfinity.com. @184.105.193.73

; <<>> DiG 9.4.2-P2 <<>> +cd a xfinity.com. @184.105.193.73
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1797
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 3, AUTHORITY: 5, ADDITIONAL: 10

;; QUESTION SECTION:
;xfinity.com. IN A

;; ANSWER SECTION:
xfinity.com. 7120 IN A 96.114.14.140
xfinity.com. 7120 IN A 68.87.41.40
xfinity.com. 7120 IN A 96.114.21.40

;; AUTHORITY SECTION:
xfinity.com. 126422 IN NS dns103.comcast.net.
xfinity.com. 126422 IN NS dns104.comcast.net.
xfinity.com. 126422 IN NS dns101.comcast.net.
xfinity.com. 126422 IN NS dns102.comcast.net.
xfinity.com. 126422 IN NS dns105.comcast.net.

;; ADDITIONAL SECTION:
dns101.comcast.net. 1225 IN A 69.252.250.103
dns102.comcast.net. 1225 IN A 68.87.85.132
dns103.comcast.net. 1225 IN A 68.87.76.228
dns104.comcast.net. 1225 IN A 68.87.68.244
dns105.comcast.net. 1225 IN A 68.87.72.244
dns101.comcast.net. 1225 IN AAAA 2001:558:fe23:8:69:252:250:103
dns102.comcast.net. 1225 IN AAAA 2001:558:1004:7:68:87:85:132
dns103.comcast.net. 1225 IN AAAA 2001:558:1014:c:68:87:76:228
dns104.comcast.net. 1225 IN AAAA 2001:558:100a:5:68:87:68:244
dns105.comcast.net. 1225 IN AAAA 2001:558:100e:5:68:87:72:244

;; Query time: 476 msec
;; SERVER: 184.105.193.73#53(184.105.193.73)
;; WHEN: Thu Sep 28 14:10:50 2017
;; MSG SIZE rcvd: 413

Logfile examples