Distributing content with SQUID
This is one of my geek postings for the Linux guys out there.
Tradebit has now reached a size in terms of traffic where we need to scale quickly and without any hassles. I have written about the new hardware layout last year and how we basically approach this.
In the first days of the new year the whole platform got slower and slower by attracting more users and robots crawling through all the files we have online. We needed to add more resources to compensate for that and to provide a fast and good experience on tradebit.
I have done that by installing more local SQUID servers in Atlanta, Chicago, Los Angeles and Washington DC as “virtual private servers” (VPS) and offering a reverse transparent caching proxy on these little boxes.
The Open Source tool SQUID is pretty useful for that. If you need to deliver local content to certain metropolitan areas: we have learned how to do that right now and are glad to help, if you have questions! The gain of speed overall is pretty impressive and the average load on the main servers went down significantly!
