Scrapy安装步骤及错误汇总：scrapy OpenSSL error: command ‘gcc’ failed with exit status 1

安装easy_install：wget
-q http://peak.telecommunity.com/dist/ez_setup.py; python ez_setup.py

安装libxml和openssl（scrapy依赖需要用到）：yum -y install gcc python27-devel libxml2 libxml2-devel
libxslt libxslt-devel openssl libffi-devel libjpeg-turbo-devel
安装pyopenssl(el5和centos5需要打补丁，6可以忽略该步骤)：wget https://pypi.python.org/packages/source/p/pyOpenSSL/pyOpenSSL-0.13.1.tar.gz#md5=e27a3b76734c39ea03952ca94cc56715 https://bugs.launchpad.net/pyopenssl/+bug/845445/+attachment/2666639/+files/pyOpenSSL-0.13.centos5.patch --no-check-certificate; tar xzvf pyOpenSSL-0.13.1.tar.gz; mv pyOpenSSL-0.13.centos5.patch pyOpenSSL-0.13.1; cd pyOpenSSL-0.13.1; patch -p1 < pyOpenSSL-0.13.centos5.patch;
python setup.py install; cd ..; rm -fr pyOpenSSL-0.13.1*
安装PIL：wget http://effbot.org/media/downloads/Imaging-1.1.7.tar.gz; tar xzvf Imaging-1.1.7.tar.gz; python Imaging-1.1.7/setup.py install; rm -fr Imaging-1.1.7*
easy_install-2.7 scrapy pyquery scrapy-redis pymongo requests redis pil pillow
cp /usr/local/bin/scrapy /usr/bin/scrapy（要不crontab默认的sh是/usr/bin，找不到scrapy）

cat sohu.log | egrep 'Scrapy 0.18.4 started|INFO: Dumping Scrapy stats:' | tail -4

在安装scrapy出现了这个错误

OpenSSL/ssl/connection.c: In function ‘ssl_Connection_set_context’:

OpenSSL/ssl/connection.c:289: warning: implicit declaration of function ‘SSL_set_SSL_CTX’

OpenSSL/ssl/connection.c: In function ‘ssl_Connection_get_servername’:

OpenSSL/ssl/connection.c:313: error: ‘TLSEXT_NAMETYPE_host_name’ undeclared (first use in this function)

OpenSSL/ssl/connection.c:313: error: (Each undeclared identifier is reported only once

OpenSSL/ssl/connection.c:313: error: for each function it appears in.)

OpenSSL/ssl/connection.c:320: warning: implicit declaration of function ‘SSL_get_servername’

OpenSSL/ssl/connection.c:320: warning: assignment makes pointer from integer without a cast

OpenSSL/ssl/connection.c: In function ‘ssl_Connection_set_tlsext_host_name’:

OpenSSL/ssl/connection.c:346: warning: implicit declaration of function ‘SSL_set_tlsext_host_name’

error: command ‘gcc’ failed with exit status 1

原来是centos5下pyopenssl 0.13的版本和openssl不兼容

下载补丁

wget https://bugs.launchpad.net/pyopenssl/+bug/845445/+attachment/2666639/+files/pyOpenSSL-0.13.centos5.patch
移动补丁到对应的目录下

mv *.patch pyOpenSSL-0.13

进入目录

cd pyOpenSSL-0.13

patch -p1 < pyOpenSSL-0.13.centos5.patch

安装带补丁的pyOpenSSL完成后，再安装scrapy就成功了