Page History: How to handle/avoid being IP blocked
Compare Page Revisions
Page Revision: 2009/06/05 17:26
Try setting your bandwidth throttling to 1 second between requests in:
- Package properties->
- Steps->
- Downloading step properties->
- Bandwidth Throttling tab->
- Change unit to one second and slide the slider to one
Some sites require more bandwidth throttling, but a vast majority do not. If you are limited to a small number of requests per day, using a proxy service like Tor may be the only way to go. Keep in mind that the limits are probably set to maintain a manageable level of traffic and not to stop people from scraping the site. So if you're hitting them more than their limit by using proxies, you may take their site down which would be bad for you and them.