.fj (Fiji) DNSSEC Outage: 2022-03-07 to 2022-03-08

Date: March 8, 2022

Overview

This page gives some details on the .fj (Fiji) DNSSEC outage from March 7 to March 8, 2022.

Timeline / DNSViz

DNSSEC Debugger

Unlike DNSViz, Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from March 8, 2022.

March 8, 2022 Fiji DNSSEC outage

Google Public DNS: with and without DNSSEC

DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.

$ dig +dnssec aaaa fj. @8.8.8.8.

; <<>> dig 9.10.8-P1 <<>> +dnssec aaaa fj. @8.8.8.8.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 30425
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;fj. IN AAAA

;; Query time: 221 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Mon Mar 07 23:57:53 UTC 2022
;; MSG SIZE rcvd: 31

You have to disable DNSSEC to make DNS queries work:

$ dig +cd aaaa fj. @8.8.8.8.

; <<>> dig 9.10.8-P1 <<>> +cd aaaa fj. @8.8.8.8.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31002
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;fj. IN AAAA

;; AUTHORITY SECTION:
fj. 1800 IN SOA ns1.fj. domreg.usp.ac.fj. 2022030811 21600 3600 604800 3600

;; Query time: 16 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Mon Mar 07 23:57:53 UTC 2022
;; MSG SIZE rcvd: 85

Cloudflare

Cloudflare wrote about this DNSSEC outage in a blog post titled DNSSEC issues take Fiji domains offline.

Zonemaster

A couple zonemaster sites reported this DNSSEC outage:

Outages mailing list

This Fiji DNSSEC outage was discussed on the outages mailing list in the thread [outages] .fj TLD down.

There are copies courtesy of archive.org and archive.ph.

dns-operations mailing list

It was also discussed on the dns-operations mailing list hosted by DNS-OARC in the thread [dns-operations] TLD .fj broken (DNSSEC issue).

There are copies courtesy of archive.org and archive.ph.

dns.google.com

dns.google.com noticed the outage. And here's a screenshot:

.fj DNSSEC outage

drill trace

Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):

[T] fj. 86400 IN DS 18952 8 2 b22f5938ad822a76499a3ac295e061cc07fce36d7956e26a4f51aede1717f993
;; Domain: fj.
;; Signature ok but no chain to a trusted key or ds record
[S] fj. 3600 IN DNSKEY 257 3 8 ;{id = 12931 (ksk), size = 2048b}
fj. 3600 IN DNSKEY 256 3 8 ;{id = 24459 (zsk), size = 2048b}
[S] Existence denied: fj. A
;;[S] self sig OK; [B] bogus; [T] trusted

Logfile examples