.ke (Kenya) DNSSEC Outage: March 31, 2015

Updated: April 1, 2015

Overview

This page gives some details on the ke TLD DNSSEC outage on March 31, 2015. The outage lasted over 14 hours.

Timeline / DNSViz

• 2015-03-31 05:26:51 UTC — delegation to revoked/bogus key
• 2015-03-31 06:35:26 UTC — delegation to revoked/bogus key
• 2015-03-31 06:53:35 UTC — delegation to revoked/bogus key
• 2015-03-31 07:11:02 UTC — delegation to revoked/bogus key
• 2015-03-31 07:58:46 UTC — delegation to revoked/bogus key
• 2015-03-31 08:56:33 UTC — delegation to revoked/bogus key
• 2015-03-31 10:35:15 UTC — delegation to revoked/bogus key
• 2015-03-31 10:35:22 UTC — delegation to revoked/bogus key
• 2015-03-31 11:46:25 UTC — delegation to revoked/bogus key
• 2015-03-31 11:57:05 UTC — delegation to revoked/bogus key
• 2015-03-31 11:57:43 UTC — delegation to revoked/bogus key
• 2015-03-31 11:59:54 UTC — delegation to revoked/bogus key
• 2015-03-31 12:01:15 UTC — delegation to revoked/bogus key
• 2015-03-31 12:10:29 UTC — delegation to revoked/bogus key
• 2015-03-31 12:13:21 UTC — delegation to revoked/bogus key
• 2015-03-31 12:42:58 UTC — delegation to revoked/bogus key
• 2015-03-31 12:55:49 UTC — delegation to revoked/bogus key
• 2015-03-31 13:40:30 UTC — delegation to revoked/bogus key with a side of expired RRSIGs
• 2015-03-31 13:46:22 UTC — expired RRSIGs
• 2015-03-31 13:49:05 UTC — expired RRSIGs
• 2015-03-31 13:49:32 UTC — expired RRSIGs
• 2015-03-31 13:57:45 UTC — expired RRSIGs
• 2015-03-31 13:58:15 UTC — expired RRSIGs
• 2015-03-31 14:06:13 UTC — expired RRSIGs
• 2015-03-31 14:12:09 UTC — expired RRSIGs
• 2015-03-31 14:18:55 UTC — expired RRSIGs
• 2015-03-31 14:21:53 UTC — expired RRSIGs
• 2015-03-31 14:22:03 UTC — expired RRSIGs
• 2015-03-31 14:26:57 UTC — expired RRSIGs
• 2015-03-31 14:36:00 UTC — expired RRSIGs
• 2015-03-31 14:38:36 UTC — expired RRSIGs
• 2015-03-31 14:46:04 UTC — expired RRSIGs
• 2015-03-31 14:57:30 UTC — expired RRSIGs
• 2015-03-31 15:05:49 UTC — expired RRSIGs
• 2015-03-31 15:17:18 UTC — expired RRSIGs
• 2015-03-31 15:21:50 UTC — expired RRSIGs
• 2015-03-31 16:06:50 UTC — expired RRSIGs
• 2015-03-31 16:20:49 UTC — expired RRSIGs
• 2015-03-31 16:43:33 UTC — expired RRSIGs
• 2015-03-31 16:58:26 UTC — expired RRSIGs
• 2015-03-31 17:05:20 UTC — expired RRSIGs
• 2015-03-31 17:30:01 UTC — expired RRSIGs
• 2015-03-31 17:30:40 UTC — expired RRSIGs
• 2015-03-31 17:52:16 UTC — expired RRSIGs
• 2015-03-31 18:06:44 UTC — expired RRSIGs
• 2015-03-31 18:09:19 UTC — expired RRSIGs
• 2015-03-31 18:12:49 UTC — expired RRSIGs
• 2015-03-31 18:15:26 UTC — expired RRSIGs
• 2015-03-31 18:21:15 UTC — some expired RRSIGs
• 2015-03-31 18:21:33 UTC — some expired RRSIGs
• 2015-03-31 18:23:13 UTC — some expired RRSIGs
• 2015-03-31 18:23:21 UTC — some expired RRSIGs
• 2015-03-31 18:24:43 UTC — some expired RRSIGs
• 2015-03-31 18:28:59 UTC — some expired RRSIGs
• 2015-03-31 18:44:29 UTC — some expired RRSIGs
• 2015-03-31 18:51:53 UTC — some expired RRSIGs
• 2015-03-31 19:56:23 UTC — ke is unsigned; still some expired RRSIGs
• 2015-03-31 20:27:33 UTC — still unsigned, still some expired RRSIGs
• 2015-03-31 20:51:08 UTC — still unsigned, still some expired RRSIGs
• 2015-03-31 20:51:30 UTC — still unsigned, still some expired RRSIGs
• 2015-03-31 22:08:00 UTC — still unsigned, still some expired RRSIGs
• The saga continues...

KeNIC Statement

KeNIC tweeted "Dear All, We apologize for the interruption of .KE services experienced today. Our DNSSEC system is down and we are working to resolve it."

Mailing list discussions

• March 2015:
  • dns-operations: DNSSEC validation failures for .KE
  • dns-operations: .ke failing DNSSEC validation
  • afnog: .ke DNSSEC issues
• April 2015:
  • dnssec-deployment: DNSSEC validation failures for .KE
  • dns-operations: DNSSEC validation failures for .KE

Logfile examples

• [1427820895] unbound[68:0] info: validation failure <youtube.co.ke. A IN>: signature expired from 66.135.62.20 for key ke. while building chain of trust
• [1427820906] unbound[68:0] info: validation failure <www.google.co.ke. A IN>: key for validation ke. is marked as invalid because of a previous validation failure <youtube.co.ke. A IN>: signature expired from 66.135.62.20 for key ke. while building chain of trust