[Hiring] Help me speed up a web bot - $50 BTC minimum reward for advice that works.

Hi, sorry for the delay in replying.

I'm querying steam community market getting listings of certain item types. I'm currently changing randomizing a parameter which though I think works slows down the results being returned.

I find my whole issue really peculiar. I'm fairly sure I'm bypassing akamai at the moment as doing any kind of aggressive querying slows down the entire market regardless of which country/ip I'm searching from (checking the speeds using independent VPSs). On the flip side I can see when new people start searching the market using scripts as the latencies also take hits.

The main issue is there seems to be a massive propagation time from when one query gets results to when they all do. I can query using multiple threads using a pool of proxies and even proxies on the same subnet/location can have 8 seconds (4 or so queries) between receiving the change.

I'm still very much interested in an alternative way to try to bypass akamai if you think you know one.

Here's an example endpoint I'd query to get listings for items.

http://steamcommunity.com/market/listings/730/Five-SeveN | Case Hardened (Field-Tested)/render/?query=&start=0&count=100&country=GB&language=english&currency=2

/r/Jobs4Bitcoins Thread Parent