Unclaimed project
Are you a maintainer of pyspider? Claim this project to take control of your public changelog and roadmap.
scheduler.PAUSE_TIME (default: 5min) when last scheduler.FAIL_PAUSE_NUM (default: 10) task failed, and dispatch scheduler.UNPAUSE_CHECK_NUM (default: 3) tasks after scheduler.PAUSE_TIME. Project will resume if any one of last scheduler.UNPAUSE_CHECK_NUM tasks success.--splash-endpoint=http://splash:8050/executefrom projects import project support.user_agent parameter in self.crawl, you can set user-agent by headers though.connection_timeout not working issue.need_auth option not applied on webdav issue.