ungleich.ch (@[email protected])

Running a #zerocarbon data center powered by 100% renewable energy in the Alps
#OpenSource #IPv6 #Sustainability

[ungleich.ch (@[email protected]) @ungleich]

You can now find us on #mastodon, please follow us there: https://t.co/0qlGaxWMR5.

[ungleich.ch (@[email protected]) @ungleich]

We are currently experiencing multiple server outages @datacenterlight and are investigating the cause. Affected is the mainly the storage cluster in place10.

[Data Center Light @datacenterlight]

2024-11-26 power outage
* at the moment we experience a power outage
* the whole region of Glarus is having a power outage
* all systems are still up, running on battery power at the moment
* we are in touch with the grid and local power plant to get insight on what is going on

[Data Center Light @datacenterlight]

All VMs are back online.

[Data Center Light @datacenterlight]

Majority of storage cluster is back online.

[Data Center Light @datacenterlight]

All virtual machines are restarting.

[Data Center Light @datacenterlight]

Storage cluster is rebooting.

[Data Center Light @datacenterlight]

There was a power outage in place10. We are investigating the cause and now restarting all affected systems.

[Data Center Light @datacenterlight]

We are under #DDoS attack. One of our customers, a climate organisation is being targeted. Taking counter measurements. This has really become a war of fossil fuels.

[Data Center Light @datacenterlight]

DDOS attack has been mitigated. All system back to normal.

[Data Center Light @datacenterlight]

After taking counter measurements we are seeing that the attack traffic is decreasing.

[Data Center Light @datacenterlight]

We are experiencing a DDOS attach at the moment.

[Data Center Light @datacenterlight]

We are seeing network congestion issues in place6 and place10, we are investigating.

[Data Center Light @datacenterlight]

All VMs have been migrated and are back up.

[Data Center Light @datacenterlight]

We are having issues on a specific VM host. We are investigating and starting migration of all VMs from it to another host.

[ungleich.ch (@[email protected]) @ungleich]

Now #new on https://t.co/xL9MAaUpPQ: You can now get #fedora version 39 as a #VPS. Also available as #IPv6 only variant on https://t.co/bY58R7KzyC. Enjoy while it's fresh!

[ungleich.ch (@[email protected]) @ungleich]

Problem has been solved, a misconfiguration on upstream prevented a new announcement, we reverted to a previous BGP announcement.

[ungleich.ch (@[email protected]) @ungleich]

We are seeing traffic dropped via #IPv4, currently investigating with our upstreams.

[Data Center Light @datacenterlight]

All services are restored. If you experience any issues, please get in touch with us either via Twitter or [email protected]

[Data Center Light @datacenterlight]

Hardware replacement is in progress.

[Data Center Light @datacenterlight]

We have an hardware issue with our router. Our team is on the way to start the replacement process.