Python网络爬虫之scrapy安装

参考：https://doc.scrapy.org/en/latest/index.html

1，安装pip

wget https://bootstrap.pypa.io/get-pip.py sudo python get-pip.py

2，安装virtualenv

sudo pip install virtualenv

3，创建ENV

virtualenv ENV

4，activate script

cd ENV
source bin/activate

This will change your $PATH so its first entry is the virtualenv’s bin/ directory. (You have to use source because it changes your shell environment in-place.) This is all it does; it’s purely a convenience. If you directly run a script or the python interpreter from the virtualenv’s bin/ directory (e.g. path/to/ENV/bin/pip or /path/to/ENV/bin/python-script.py) there’s no need for activation.

The activate script will also modify your shell prompt to indicate which environment is currently active. To disable this behaviour, see VIRTUAL_ENV_DISABLE_PROMPT.

To undo these changes to your path (and prompt), just run:

deactivate

5，To install scrapy on Ubuntu (or Ubuntu-based) systems, you need to install these dependencies:

(ENV) kylefan@kylefan-pc:~/ENV$ sudo apt-get install python-dev python-pip libxml2-dev libxslt1-dev zlib1g-dev libffi-dev libssl-dev

Inside a virtualenv, you can install Scrapy with pip after that:

(ENV) kylefan@kylefan-pc:~/ENV$ pip install scrapy