At 8:30pm Friday night, the power supply in our main web application server failed and the server went down. (Yes, it had only one :) Hey, it was our first server purchase years ago and has had an uptime of more than 12 months! However, it was silly not to have any more power supplies on hand. Of course, no customer seedboxes went down or were affected by this, but it did result in downtime for the front end of seedboxhosting.com. We had to figure out another way to get the server online - and it would be well into Saturday before this happened.
This has resulted in the decision to escalate something we have been planning to do for some time. We have begun to move critical services (Dashboard, Accounting, Monitoring, DNS) off of single servers and into distributed cloud environments.
Being mostly a Ruby shop, we chose Engine Yard to help us with this - and they have been great so far. Their automation, simple deployment and ability to clone production environments into staging areas for testing is beautiful. We are very excited about the new possibilities of working with well tested and standardized Ruby on Rails stacks.
All this means our team can stop focusing on hardware for critical services and on providing a better seedbox experience for us all.
// Seedbox Hosting Team //