All Systems Operational
[dataforest Backbone] Interxion FRA8
?
Operational
Edge Network (affects all locations)
Operational
DDoS Protection (affects all locations)
Operational
Dedicated Servers (40G / 100G / 400G)
?
Operational
Facilites
?
Operational
[Datacenter] maincubes FRA01
?
Operational
Datacenter Routing and Switching Infrastructure
?
Operational
Ceph Cluster
?
Operational
Virtual Servers
Operational
Managed Servers
Operational
Dedicated Servers
Operational
Plesk Web Hosting
Operational
TeamSpeak3 Servers
Operational
Network Storage
Operational
Colocation Racks
Operational
Facilites
?
Operational
[Datacenter] firstcolo FRA4
?
Operational
Datacenter Routing and Switching Infrastructure
?
Operational
Dedicated Servers
Operational
Colocation Racks
Operational
Facilites
?
Operational
General Services
Operational
Avoro CP & Support
?
Operational
PHP-Friends CRM & Support
?
Operational
Cloud Control Panel
?
Operational
Dedicated Control Panel
?
Operational
IPMI VPN
?
Operational
DDoS Manager
?
Operational
Domain Robot
?
Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Scheduled Maintenance
[Rescheduled] Action Required: Disruptive Power Maintenance at maincubes FRA01 Jul 1, 2025 00:00 - Jul 31, 2025 23:59 CEST
Update -
We have received an update from the data center operator, stating that the new date for the power maintenance will be communicated with at least six weeks' notice. Consequently, we have adjusted the placeholder date on our status page, as the maintenance is not expected to take place before July.
May 17, 2025 - 13:36 CEST
May 17, 2025 - 13:36 CEST
Update -
The scheduled maintenance has been postponed by the datacenter operator. We deeply regret this change, especially given the short notice, as well as the extensive preparations that took several weeks of work. We understand that you, as our customers, had also prepared accordingly. Unfortunately, we have not yet been informed about the reason for the postponement. The date shown on our status page is currently a placeholder, as the new maintenance schedule is still to be determined.
May 08, 2025 - 22:38 CEST
May 08, 2025 - 22:38 CEST
Scheduled -
We would like to inform you about upcoming disruptive maintenance work on the power infrastructure at our maincubes FRA01 data center. Please read this announcement carefully, as it may affect your services hosted with us.
During the maintenance window, the low voltage main distribution board including its circuit breakers will be serviced according to the manufacturer’s specifications to ensure the long-term safe operation of the electrical infrastructure. For this purpose, shutting down the redundant power feeds (one after the other) is unavoidable.
The A-feed of our rack rows will be shut down between 10:00 and 11:00 CEST.
The B-feed will be shut down between 14:00 and 15:00 CEST.
Any systems without redundant power supply will be affected and must be shut down during the relevant window. Systems with redundant power will remain online; there is only an increased risk due to the temporary loss of redundancy. At no time will both power feeds be turned off simultaneously. If unexpected issues occur, we will update our status page with all affected components.
Which services are affected?
Virtual Servers, Web Hosting, Ceph Storage, NFS & Samba Shares, TeamSpeak:
These products are generally hosted on systems with redundant power (exceptions see below). The same level of redundancy applies to our own systems (support systems, websites, customer panels etc.). These systems are not expected to go offline, but there is an increased risk due to the temporary loss of redundancy.
One last (10 year old) legacy vServer host not yet consolidated to modern hardware with redundant power supplies is an exception: vnode16 is connected to the A-feed only and must be shut down for the first maintenance window.
Dedicated Servers:
We offer dedicated servers both with and without redundant power supplies. If your product description does not mention redundant power supplies, your server has no redundant power supply and will be affected. This also applies to Managed Dedicated Servers (see below for further instructions).
Top-of-Rack Switches & Network Redundancy:
All top-of-rack switches are powered redundantly and have redundant uplinks to the core network. This applies even if you have only ordered a single uplink as the switch itself is as redundant as possible. At dataforest, we do not implement “half redundancy” (e.g., Switch 1 on Feed A and Switch 2 on Feed B).
IPMI and Rescue Systems:
An exception to the above are our IPMI switches, which are intentionally not redundant as they serve only non-critical functions. During both maintenance windows, IPMI interfaces will be temporarily unavailable, meaning that IPMI interfaces as well as our installation and rescue systems - being dependent on an IPMI connection to the server - cannot be used. The IPMI switches are distributed almost evenly between Feed A and Feed B.
Colocation and Housing:
Colocation customers with full racks are responsible for ensuring redundancy within their own setups. Please check that redundant PSUs are present and functional.
If you have booked HE housing with us, we have verified whether your server is connected redundantly. If that is not the case, we have noted the respective feed in the Dedicated Server Control Panel (same as for Dedicated Servers, see next section). If you do not have access to this panel, you have already been contacted individually at the time of this announcement.
How to properly determine if and when a server (without redundant power supplies) is affected:
In the Dedicated Server Control Panel (https://dedicated.dataforest.cloud), you will find the hostname of your server. Previously, this followed the pattern root1234.mc-fra01.dataforest.net. To provide a reliable indicator of feed assignment, we have updated all hostnames to include their feed designation:
root1234.mc-fra01-A → connected to A-feed → affected from 10:00 to 11:00
root1234.mc-fra01-B → connected to B-feed → affected from 14:00 to 15:00
root1234.mc-fra01-AB → connected to both feeds, no disruption expected
Servers located at “ix-fra8” or “fc-fra4” (as seen in the hostname) are not affected by this maintenance.
Managed Dedicated Servers are subject to the same conditions. However, as they are managed by dataforest and therefore do not come with access to the control panel, we have added the A-feed or B-feed information to the respective product description. Managed Dedicated Server customers can check this at https://crm.php-friends.de.
Important instructions for affected systems:
Please shut down any affected servers at least 15 minutes prior to the start of the respective maintenance window. (For managed servers, we will handle this.) After this time, any remaining active servers will be manually powered down on-site via the power button. If your OS does not handle this gracefully, the server may be hard powered off, which can cause file system damage – please avoid this by shutting the system down cleanly beforehand. Once the maintenance is complete, power will be restored and systems will automatically restart if the BIOS is configured accordingly.
How to prepare (especially for non-redundant systems):
- Subscribe to our status page for real-time updates.
- Check which feed your server is connected to and when it will be affected. Plan for the shutdown and the expected downtime and inform your users and customers.
- Test your shutdown behavior – ensure your server reacts properly to power button presses and ACPI shutdown commands. You can test this via IPMI (accessible in the Dedicated Server Control Panel).
- During this test reboot, verify that all necessary software components start automatically and function correctly. We cannot offer support for software-related issues on the day of the maintenance.
- Use IPMI to verify that your BIOS is configured to automatically boot when power becomes available again (Restore on AC Power Loss -> Power On, https://www.google.com/search?q=Restore+on+AC+Power+Loss). This setting is our default, but we strongly recommend checking it. Our support can check this for you free of charge – please contact us by May 9, 2025 if you would like us to do so.
- Check your RAID for consistency and your backup strategy for functionality.
- If your server only has a single SSD or no RAID, make sure to create a complete backup before the maintenance window.
If your server does not come back online after the maintenance window:
Please do not contact our support immediately. First, check our status page to see if the maintenance has been successfully completed. If yes, use the Dedicated Server Control Panel to check the current status of your server via IPMI. If the server is still powered off, start it manually and ensure that the BIOS setting “Restore on AC Power Loss” is correctly set to “Power On” as described above. If the server did not start automatically, the setting is probably incorrect!
Only if this is unsuccessful and you suspect a hardware failure, contact our support via ticket.
Please also note that we cannot check or fix software issues on the day of the maintenance (even as a paid service), as we will only address hardware problems due to the number of systems affected. We therefore strongly urge you to perform the checks described above in good time. Everything beyond ensuring correct hardware functionality is your own responsibility (this does not apply to managed servers).
What dataforest is doing to prepare:
- Our spare parts inventory is being expanded so we can quickly respond to any defects.
- dataforest staff will be on-site from the start of the maintenance. One part of management will coordinate on-site, the other part will coordinate the support team. We are working in two shifts to avoid bottlenecks.
- Both the data center team and the support team will be specially trained and reinforced for this day.
- In the unlikely event of a complete power failure, we have an emergency plan in place that clearly defines procedures and responsibilities.
- We are equally prepared for partial power failures through predefined processes and prewritten status messages – enabling fast and transparent communication in worst-case scenarios and minimizing downtime.
- Redundant power supplies will be tested on-site and replaced proactively if defective. This applies to all systems and network components alike, so unexpected issues can be largely ruled out.
- We are reviewing all dedicated server contracts, product descriptions, and invoice texts to ensure power redundancy is clearly stated and understood. The information in the Dedicated Server Control Panel is already up to date.
- At the beginning of the maintenance, we will mute all expected monitoring alerts (e.g. missing redundancy or unreachable IPMIs) so our on-call staff can focus entirely on real issues instead of being overwhelmed by hundreds of expected alerts.
Thank you for your understanding and cooperation!
Your dataforest team
Apr 26, 2025 - 01:50 CEST
During the maintenance window, the low voltage main distribution board including its circuit breakers will be serviced according to the manufacturer’s specifications to ensure the long-term safe operation of the electrical infrastructure. For this purpose, shutting down the redundant power feeds (one after the other) is unavoidable.
The A-feed of our rack rows will be shut down between 10:00 and 11:00 CEST.
The B-feed will be shut down between 14:00 and 15:00 CEST.
Any systems without redundant power supply will be affected and must be shut down during the relevant window. Systems with redundant power will remain online; there is only an increased risk due to the temporary loss of redundancy. At no time will both power feeds be turned off simultaneously. If unexpected issues occur, we will update our status page with all affected components.
Which services are affected?
Virtual Servers, Web Hosting, Ceph Storage, NFS & Samba Shares, TeamSpeak:
These products are generally hosted on systems with redundant power (exceptions see below). The same level of redundancy applies to our own systems (support systems, websites, customer panels etc.). These systems are not expected to go offline, but there is an increased risk due to the temporary loss of redundancy.
One last (10 year old) legacy vServer host not yet consolidated to modern hardware with redundant power supplies is an exception: vnode16 is connected to the A-feed only and must be shut down for the first maintenance window.
Dedicated Servers:
We offer dedicated servers both with and without redundant power supplies. If your product description does not mention redundant power supplies, your server has no redundant power supply and will be affected. This also applies to Managed Dedicated Servers (see below for further instructions).
Top-of-Rack Switches & Network Redundancy:
All top-of-rack switches are powered redundantly and have redundant uplinks to the core network. This applies even if you have only ordered a single uplink as the switch itself is as redundant as possible. At dataforest, we do not implement “half redundancy” (e.g., Switch 1 on Feed A and Switch 2 on Feed B).
IPMI and Rescue Systems:
An exception to the above are our IPMI switches, which are intentionally not redundant as they serve only non-critical functions. During both maintenance windows, IPMI interfaces will be temporarily unavailable, meaning that IPMI interfaces as well as our installation and rescue systems - being dependent on an IPMI connection to the server - cannot be used. The IPMI switches are distributed almost evenly between Feed A and Feed B.
Colocation and Housing:
Colocation customers with full racks are responsible for ensuring redundancy within their own setups. Please check that redundant PSUs are present and functional.
If you have booked HE housing with us, we have verified whether your server is connected redundantly. If that is not the case, we have noted the respective feed in the Dedicated Server Control Panel (same as for Dedicated Servers, see next section). If you do not have access to this panel, you have already been contacted individually at the time of this announcement.
How to properly determine if and when a server (without redundant power supplies) is affected:
In the Dedicated Server Control Panel (https://dedicated.dataforest.cloud), you will find the hostname of your server. Previously, this followed the pattern root1234.mc-fra01.dataforest.net. To provide a reliable indicator of feed assignment, we have updated all hostnames to include their feed designation:
root1234.mc-fra01-A → connected to A-feed → affected from 10:00 to 11:00
root1234.mc-fra01-B → connected to B-feed → affected from 14:00 to 15:00
root1234.mc-fra01-AB → connected to both feeds, no disruption expected
Servers located at “ix-fra8” or “fc-fra4” (as seen in the hostname) are not affected by this maintenance.
Managed Dedicated Servers are subject to the same conditions. However, as they are managed by dataforest and therefore do not come with access to the control panel, we have added the A-feed or B-feed information to the respective product description. Managed Dedicated Server customers can check this at https://crm.php-friends.de.
Important instructions for affected systems:
Please shut down any affected servers at least 15 minutes prior to the start of the respective maintenance window. (For managed servers, we will handle this.) After this time, any remaining active servers will be manually powered down on-site via the power button. If your OS does not handle this gracefully, the server may be hard powered off, which can cause file system damage – please avoid this by shutting the system down cleanly beforehand. Once the maintenance is complete, power will be restored and systems will automatically restart if the BIOS is configured accordingly.
How to prepare (especially for non-redundant systems):
- Subscribe to our status page for real-time updates.
- Check which feed your server is connected to and when it will be affected. Plan for the shutdown and the expected downtime and inform your users and customers.
- Test your shutdown behavior – ensure your server reacts properly to power button presses and ACPI shutdown commands. You can test this via IPMI (accessible in the Dedicated Server Control Panel).
- During this test reboot, verify that all necessary software components start automatically and function correctly. We cannot offer support for software-related issues on the day of the maintenance.
- Use IPMI to verify that your BIOS is configured to automatically boot when power becomes available again (Restore on AC Power Loss -> Power On, https://www.google.com/search?q=Restore+on+AC+Power+Loss). This setting is our default, but we strongly recommend checking it. Our support can check this for you free of charge – please contact us by May 9, 2025 if you would like us to do so.
- Check your RAID for consistency and your backup strategy for functionality.
- If your server only has a single SSD or no RAID, make sure to create a complete backup before the maintenance window.
If your server does not come back online after the maintenance window:
Please do not contact our support immediately. First, check our status page to see if the maintenance has been successfully completed. If yes, use the Dedicated Server Control Panel to check the current status of your server via IPMI. If the server is still powered off, start it manually and ensure that the BIOS setting “Restore on AC Power Loss” is correctly set to “Power On” as described above. If the server did not start automatically, the setting is probably incorrect!
Only if this is unsuccessful and you suspect a hardware failure, contact our support via ticket.
Please also note that we cannot check or fix software issues on the day of the maintenance (even as a paid service), as we will only address hardware problems due to the number of systems affected. We therefore strongly urge you to perform the checks described above in good time. Everything beyond ensuring correct hardware functionality is your own responsibility (this does not apply to managed servers).
What dataforest is doing to prepare:
- Our spare parts inventory is being expanded so we can quickly respond to any defects.
- dataforest staff will be on-site from the start of the maintenance. One part of management will coordinate on-site, the other part will coordinate the support team. We are working in two shifts to avoid bottlenecks.
- Both the data center team and the support team will be specially trained and reinforced for this day.
- In the unlikely event of a complete power failure, we have an emergency plan in place that clearly defines procedures and responsibilities.
- We are equally prepared for partial power failures through predefined processes and prewritten status messages – enabling fast and transparent communication in worst-case scenarios and minimizing downtime.
- Redundant power supplies will be tested on-site and replaced proactively if defective. This applies to all systems and network components alike, so unexpected issues can be largely ruled out.
- We are reviewing all dedicated server contracts, product descriptions, and invoice texts to ensure power redundancy is clearly stated and understood. The information in the Dedicated Server Control Panel is already up to date.
- At the beginning of the maintenance, we will mute all expected monitoring alerts (e.g. missing redundancy or unreachable IPMIs) so our on-call staff can focus entirely on real issues instead of being overwhelmed by hundreds of expected alerts.
Thank you for your understanding and cooperation!
Your dataforest team
Apr 26, 2025 - 01:50 CEST