The configuration change was successfully applied with minimal impact at 15:18 / 15:27 CEST. The root cause was an inconsistent Spanning Tree configuration within our maincubes FRA01 core network, which has now been resolved. Only a limited number of racks were affected, depending on the specific configuration of their Top-of-Rack switches. Over the coming weeks and months, we will review the configuration of all devices involved in the topology to identify potential improvements. Should further changes be necessary, we will implement them during announced maintenance windows at night, as the immediate issue has been resolved and no further disruptions are expected.
Posted Aug 05, 2025 - 15:58 CEST
Identified
We have identified the root cause and will carry out an emergency configuration change in the next 60 minutes. This may result in a brief period of packet loss but is necessary to ensure stable operation.
Posted Aug 05, 2025 - 14:24 CEST
Investigating
Our monitoring systems detected short periods of packet loss to some destinations within our maincubes FRA01 location at 11:47:42, 12:15:34, 12:18:02, and 12:24:23 CEST, each lasting around 15 seconds. While the situation has been stable since then, we're still looking into the root cause.
Initial findings point to log messages on one of the two redundant core switches involved, while the other didn't log anything unusual. This suggests that the impact for customers was likely minimal, but we're continuing our investigation and will keep the incident open until it's fully clarified and mitigated.
Posted Aug 05, 2025 - 12:59 CEST
This incident affected: [Datacenter] maincubes FRA01 (Datacenter Routing and Switching Infrastructure).