There are no problems during debugging. If you click run, encoding problems will be reported. In the same two collections, one always reports an error, and the other one is completely fine
taskid
d7221a2be620c4ef60e874a1d93e79d1
lastcrawltime
1499144004.75187 (20 minutes ago)
updatetime
1499144004.7518892 (20 minutes ago)
exetime
1499144014.7518687 (20 minutes ago)
track.fetch 1.32ms
{
"content": "",
"encoding": null,
"error": "'ascii' codec can't encode character '\uff09' in position 94: ordinal not in range(128)",
"headers": {},
"ok": false,
"redirect_url": null,
"status_code": 599,
"time": 0.0013222694396972656
}
track.process 0.83ms
'ascii' codec can't encode character '\uff09' in position 94: ordinal not in range(128)
= self.gen.throw(*exc_info)
File "/root/workspaces/pyspider3/lib/python3.5/site-packages/pyspider/fetcher/tornado_fetcher.py", line 378, in http_fetch
response = yield gen.maybe_future(self.http_client.fetch(request))
File "/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/gen.py", line 1055, in run
value = future.result()
File "/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/concurrent.py", line 238, in result
raise_exc_info(self._exc_info)
File "<string>", line 4, in raise_exc_info
File "/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py", line 214, in _process_queue
curl.info["headers"])
File "/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py", line 306, in _curl_setup_request
for k, v in request.headers.get_all()])
Exception: 'ascii' codec can't encode character '\uff09' in position 94: ordinal not in range(128)
{
"exception": "'ascii' codec can't encode character '\uff09' in position 94: ordinal not in range(128)",
"follows": 0,
"logs": " = self.gen.throw(*exc_info)\n File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/pyspider/fetcher/tornado_fetcher.py\", line 378, in http_fetch\n response = yield gen.maybe_future(self.http_client.fetch(request))\n File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/gen.py\", line 1055, in run\n value = future.result()\n File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/concurrent.py\", line 238, in result\n raise_exc_info(self._exc_info)\n File \"<string>\", line 4, in raise_exc_info\n File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py\", line 214, in _process_queue\n curl.info[\"headers\"])\n File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py\", line 306, in _curl_setup_request\n for k, v in request.headers.get_all()])\n Exception: 'ascii' codec can't encode character '\uff09' in position 94: ordinal not in range(128)\n",
"ok": false,
"result": null,
"time": 0.0008292198181152344
}
schedule
{
"age": 10,
"exetime": 1499144014.7518687,
"retried": 3
}
process
{
"callback": "index_page"
}
fetch
{}
ringa_lee2017-07-05 10:29:10
There is something wrong with the headers setting, I will delete it and it will be fixed in seconds.
过去多啦不再A梦2017-07-05 10:29:10
#!/usr/bin/env python
# -*- encoding: utf-8 -*-
Look at the first two lines of your code, right?