Database error - offline for 1 hour
Folks, sometimes bad things come all together and this morning we killed the database cluster on tradebit by accident (CDT is Houston time):
- New products entered since yesterday (from 6am CDT yesterday to 5.30am CDT today) were corrupted
- No sales were lost
- No files were lost, just the entry in the database
- Tradebit was unable to register any sales for 1 hour (5.30am CDT to 6.30 CDT this morning)
We had to restart the webservers and lock up the member area for around 1.5 hours and we are currently rebuilding the products entry from the backup. Yet we have managed to corrupt the products table that way that it will take some time to rebuild the entries from the last 24 hours.
We appologize for any inconveniences or extra workload this may cause to you. We do our best to make the rebuilding of the data quick and get the new products back online asap!
March 24th, 2009 at 12:53 pm
Everything back to normal and all data (besides one single file) was rebuilt. The servers are back to full performance!