(pyasn1 0.1.6 (/usr/lib/python2.7/site-packages),
2016-04-12 00:00
597 查看
ubuntu14.04下安装爬虫工具scrapy
scrapy是目前准备要学习的爬虫框架,其在ubuntu14.04下的安装过程如下:ubuntu14.04下默认安装了2.7的python以及setuptools,若未安装,可通过下面指令安装:
sudo apt-get install pythonsudo apt-get install python-setuptools
然后安装Twisted:
sudo apt-get install python-twisted
然后是Scrapy:
sudo apt-get install python-scrapy
安装完成后,如果直接键入scrapy启动的话会报类似如下错误:
File "/usr/local/bin/scrapy", line 5, in <module> from pkg_resources import load_entry_point ...... pkg_resources.ContextualVersionConflict: (pyasn1 0.1.7 (/usr/lib/python2.7/dist-packages), Requirement.parse('pyasn1>=0.1.8'), set(['pyasn1-modules']))
按提示是个版本依赖的错误。
此时先安装pip:
sudo apt-get install python-pip
升级pip:
sudo pip install --upgrade pip
然后对提示的pyasn1升级:
sudo pip install --upgrade pyasn1
再启动scrapy就可以成功了:
Scrapy 1.0.3 - no active project Usage: scrapy <command> [options] [args] Available commands: bench Run quick benchmark test commands fetch Fetch a URL using the Scrapy downloader runspider Run a self-contained spider (without creating a project) settings Get settings values shell Interactive scraping console startproject Create new project version Print Scrapy version view Open URL in browser, as seen by Scrapy [ more ] More commands available when run from project directory Use "scrapy <command> -h" to see more info about a command
相关文章推荐
- python2-3 md5加密 方法
- Python Base64编码
- python浓缩(9)文件和输入输出
- python浓缩(10)
- Python定时任务框架APScheduler
- Python爬虫时 urllib2.urlopen中文乱码
- Python sort dict by value
- 8.python笔记之面向对象基础
- python登陆百度网盘并读取自己的所有文件,输出到文本文件中
- python下用dbfread操作DBF文件
- python pep8
- Python sort list customisation
- Python 生成器 generator之next和send运行流程
- Python金融应用编程(数据分析、定价与量化投资)
- Python金融应用编程(数据分析、定价与量化投资)
- 使用python抓取婚恋网用户数据并用决策树生成自己择偶观
- Python 如何安装各种模块(1)
- python模块
- python伪装浏览器爬虫
- Python+PyQt4+Eric6合并Excel的图形化界面设计