Boosting out 1.5 Million products from 1 MySQL table
We are currently offer over 1.5 Million digital goods for your pleasure. One of the technical issues that is created by that fact is, that we have all products in one MySQL table. Including the descriptions, ownership, etc. That is putting a heavy load on the database, when 2-3 search engines spider this offer and 20-30 people look at the products.
Last week we finally got the caching improved for these products and the cluster is coming back to full speed.
Our major difference to the competition is, that we offer file hosting for our merchants as well. That means, to the 1.5M products there are another 2-3 million pictures, private files or previews… When the cluster died last friday, we needed over 1 hour to get the backup loaded into the database.
Is anybody out there, who has tried that amount of entries (over 3 Million now) in one table and still run it fast from one MySQL database? Without the cluster we would be simply overloaded and I believe that our knowledge with MySQL is too limited to really make it kick!
I know, that Flickr is using a simliar setup to ours and they seem to be a tick faster… could also be their hosting and bigger machines.
(0)