How to install pyspider under Linux

First execute

pip install pyspider

At this time the system prompts

<span style="font-size: 16px;">Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-Lau0Qp/pycurl/
You are using pip version 9.0.1, however version 9.0.3 is available.
You should consider upgrading via the &#39;pip install --upgrade pip&#39; command.
</span>

This is a problem with the pip version, execute

<span style="font-size: 16px;">sudo python -m pip install --upgrade pip</span>

Upgrade pip

Continue execution

<span style="font-size: 16px;"> sudo pip install pyspider</span>

to install pyspider, an error is reported at this time:

<span style="font-size: 16px;">Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-szUHrj/pycurl/
</span>

The reason is that the dependent libraries of pyspider are not installed. You need to execute the

<span style="font-size: 16px;">sudo apt-get install python python-dev python-distribute python-pip libcurl4-openssl-dev libxml2-dev libxslt1-dev python-lxml</span>

command to install the following support libraries

I originally thought that pyspider could be installed normally, but I could continue to execute sudo pip install pyspider. When the system error message is:

compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
span style="color: rgb(255, 0, 0);">Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__=&#39;/tmp/pip-build-B8gZjb/pycurl/setup.py&#39;;f=getattr(tokenize, &#39;open&#39;, open)(__file__);code=f.read().replace(&#39;\r\n&#39;, &#39;\n&#39;);f.close();exec(compile(code, __file__, &#39;exec&#39;))" install --record /tmp/pip-aDBSCP-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-B8gZjb/pycurl/</span>

The reason is that when installing the tool package, it needs to crawl the web page and therefore needs to process https, and processing https depends on Encryption and decryption algorithm (i.e. cryptography package), and cryptography relies on the Fourier transform algorithm and the corresponding compilation environment. Ubuntu 16.04 does not install libffi-dev and libssl-dev by default, gcc is not necessarily installed, and the target installation package does not record the relevant software packages in the dependency list, so you need to install them manually first:

sudo apt-get install libssl-dev libffi-dev build-essential

At this time Then execute sudo pip install pyspider and it will be OK!

Therefore, you need to execute the following commands to install under Linux, (emphasis added!)

sudo python -m pip install --upgrade pip
sudo apt-get install python python-dev python-distribute python-pip libcurl4-openssl-dev libxml2-dev libxslt1-dev python-lxml
sudo apt-get install libssl-dev libffi-dev build-essential
sudo pip install pyspider

What is a Linux system

Linux is a kind of free use and free dissemination The UNIX operating system is a multi-user, multi-task, multi-thread and multi-CPU operating system based on POSIX. Linux can be used to run major Unix tool software, applications and network protocols.

The above is the detailed content of How to install pyspider under Linux. For more information, please follow other related articles on the PHP Chinese website!