python - pyspider爬取时fetcher报超时错误：fetcher/: 504 Gateway Time-out
有时爬虫会报如下超时错误：
Traceback (most recent call last):
File "/opt/pyspider/pyspider/run.py", line 351, in <lambda>
app.config['fetch'] = lambda x: umsgpack.unpackb(fetcher_rpc.fetch(x).data)
File "/usr/lib/python2.7/xmlrpclib.py", line 1233, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.7/xmlrpclib.py", line 1587, in __request
verbose=self.__verbose
File "/usr/lib/python2.7/xmlrpclib.py", line 1273, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib/python2.7/xmlrpclib.py", line 1321, in single_request
response.msg,
ProtocolError: <ProtocolError for fetcher/: 504 Gateway Time-out>请问有什么好的方法避免？
