Coinbase says a routine infrastructure update disabled a critical network gateway, disrupting transfers, debit card purchases, and onchain services for about 50 minutes. Its new postmortem explains the error and why the company’s normal recovery tools also failed.
Key Takeaways
Coinbase traced the 50-minute outage to an undetected Kubernetes resource name collision during a routine infrastructure update.The incident disrupted transfers, Coinbase Card debit purchases, institutional settlements, and onchain services across multiple platforms.The outage also disabled Coinbase’s normal rollback process, prompting new safeguards and infrastructure changes.Crypto exchange Coinbase (Nasdaq: COIN) has identified the software error behind last week’s outage that left customers unable to complete transfers, make purchases with Coinbase Card, and access several onchain services for about 50 minutes.
Coinbase said:
“This was caused by an unintended misconfiguration of an important network component as a result of a routine configuration update. At no point were customer funds at risk.”
Kubernetes is software Coinbase uses to organize and run applications across its cloud infrastructure. The resource name collision caused the update to alter part of the company’s Istio ingress gateway instead of only the intended component. The gateway acts like a traffic controller, directing requests between Coinbase’s internal services.
Once the gateway went offline, systems responsible for processing trades, transfers, settlements and payment authorizations could no longer communicate.
Cards Declined and Transfers StalledThe disruption affected Coinbase’s retail, institutional and developer platforms from approximately 12:37 p.m. ET to 1:25 p.m. ET.
Some retail customers could not complete deposits, withdrawals or off-platform transactions. Transfers already underway appeared stuck but were processed after service returned. Coinbase Card debit purchases were declined, while credit card purchases continued.
The outage also interrupted onchain swaps through Coinbase DEX on Base and Solana. Coinbase Exchange and Prime customers experienced delayed or failed transfers and settlements, while developer platform customers could not onboard users, move funds, or use onramp services.
The gateway was restored at 1:20 p.m. ET, and Coinbase declared the incident mitigated three minutes later. Remaining transaction backlogs cleared over the following hours.
Recovery took longer because Coinbase’s deployment tools relied on the failed gateway. Engineers could not use the standard rollback process and instead manually launched an emergency deployment through the company’s cloud provider using temporary privileged access.
Coinbase is now adding checks to block naming conflicts, separating recovery tools from the infrastructure they manage and testing emergency access procedures more regularly.
Coinbase emphasized:
“Our goal is always zero downtime. We are actively redesigning our infrastructure to ensure that, in the highly unlikely event of another failure like this one, we can swiftly revert and recover.”
