Namecheap service outage update
On August 13, we experienced a major service outage that impacted Namecheap.com and a number of our services.
The cause was a cooling system failure at the RadiusDC: Phoenix data center, which hosts some essential Namecheap operations. Due to the cooling failure, RadiusDC instructed us to take services offline to protect customer infrastructure from overheating and possible damage.
Everything affected by the incident has now been restored, and our teams continue to monitor systems closely to ensure they remain stable.
We are deeply sorry for the impact on customers who depend on us for their websites, email, and other online services. The disruption this caused, and the circumstances behind it, are being taken extremely seriously.
What happened
The incident began when cooling systems failed at the Phoenix data center due to a major storm, causing temperatures around our infrastructure to reach critical levels.
RadiusDC: Phoenix is a Tier 3 datacenter, designed with a high level of redundancy. That made this an extremely unusual incident, and one we had not experienced before in our 25-year history.
Why we had to take services offline
Continuing to operate systems in such conditions risked equipment overheating and suffering longer-term and catastrophic damage. Taking services offline caused significant disruption, but it was necessary to protect our customers’ infrastructure.
Keeping equipment offline until the cooling systems were back online also gave temperatures time to return to safe levels before recovery began. This helped avoid creating further problems that could have further prolonged an already serious incident and caused even more serious damage.
How customers were affected
The outage affected Namecheap.com, along with services including Shared, VPS, and Dedicated Hosting, EasyWP, DNS management, and Private Email.
Here is a breakdown of how the situation affected key products and services:
DNS and URL Redirects
DNS zone resolution remained available, while DNS management was affected. URL Redirects using BasicDNS and PremiumDNS also continued to resolve, although management of these redirects was unavailable.
Hosting operations
Customer access to websites and hosting services was affected, with many customer websites unavailable, slow, or returning errors. Some billing operations were also unavailable after the system responsible for processing them was taken offline during the incident.
EasyWP
EasyWP’s Dashboard and customer websites were disrupted. During recovery, some customer websites remained unavailable after EasyWP.com and the Dashboard were restored, with some returning database connection errors. Our team has since resolved these issues for customers.
Private Email
Email services were also disrupted. Affected customers were unable to send email, and incoming email could not be delivered while the relevant servers were offline.
Importantly, this did not mean incoming messages were automatically lost. Mail providers normally make further delivery attempts when a receiving server is temporarily unavailable. So messages would arrive, but later than usual.
The outage also left our support helpdesk unable to assist customers through live chat or email.
How we restored services
RadiusDC began restoring its cooling capacity as soon as possible, while temporary chillers were installed and directed into our area of the data center to help reduce temperatures.
Our teams worked continuously on site alongside RadiusDC to restore cooling. We only began bringing services back online as soon as temperatures had returned to safe operating levels, and we were confident there was no risk of overheating or longer-term damage.
With safe operating conditions restored, we began a controlled recovery across our infrastructure. This needed to happen progressively, so customers saw different services return at different times.
Why services were restored in stages
The scale of the incident meant different parts of our platform were disrupted at the same time. While customers may experience these as separate services, the data center incident affected the infrastructure involved in delivering a number of them.
Services like hosting and email rely on multiple layers of technology, including servers, networks, storage, and databases. Restoring them meant bringing these different elements back online in the right order, rather than simply switching everything back on at once.
What happens next
We take this disruption and its impact on our customers very seriously.
Now that our systems are fully operational again, our teams are conducting a thorough review of the situation to strengthen our systems and do everything we can to ensure that similar incidents do not happen again.
We will implement even more redundancy across our different datacenters, which also include Europe and Asia.
We will also be completely transparent about what we learn and will share the results with you, along with the actions we are taking.
CORRECTION: Our data center operator is RadiusDC, not PhoenixNAP as referenced in several early posts on X on 13 August. PhoenixNAP is not involved in this incident.
