Hey folks!
Unfortunately, roughly 2 hours ago, lemm.ee went offline. The cause was our load balancer: it suddenly decided that all of our servers had become unhealthy, despite all health checks responding successfully when I requested them directly. In such cases, the load balancer stops serving all requests, effectively meaning that lemm.ee is unreachable for all users. I am still not sure what exactly caused the issue, but I will try to investigate more over the weekend.
For now, we have partially recovered, and I am continuing to work on remaining issues. Hopefully we will be back to 100% very soon. Sorry for the inconvenience!
I was wondering what was going on, status.lemm.ee said the server was ok but the federation was broken. Thank you for fixing it
Sorry for the delay in updating the status page - I actually had gone out for lunch just a few minutes before the downtime started, so I didn’t even realize anything was up until I was back at my computer about 45 minutes later 💀
no need to apologise. still a better response time, than some of the professionals I work with ;-)