Lessons From an Internet Outage: Issues Caused by Let’s Encrypt DST Root CA X3 Expiration

As a monitoring and observability company, we have a lot of monitoring built into our systems as well. We have the standard monitoring to make sure that systems are performing properly, data is flowing through our infrastructure, etc. At the same time, we have monitoring for any sudden changes to tests that our customers are running.

On September 29, 2021, at 19:21:40 UTC, we started to see a tsunami of alerts at Catchpoint. They originated from some of our web tests from our synthetic nodes, occurring when our Let’s Encrypt “R3” certificate expired. These types of incidents are pretty rare.