安装ipython，使用scrapy shell来验证xpath选择的结果 | How to install iPython and how does it work with Scrapy Shell

1. scrapy shell

是scrapy包的一个很好的交互性工具，目前我使用它主要用于验证xpath选择的结果。安装好了scrapy之后，就能够直接在cmd上操作scrapy shell了。

具体的解释参考官网
https://docs.scrapy.org/en/latest/topics/shell.html
2. ipython

在官网推荐使用ipython来跑scrapy shell，于是我尝试安装。因为之前我的python环境都是通过conda配置的（参看前一篇），通过conda安装ipython很方便

conda install -c conda-forge ipython

然后整个ipython的包就会下载下来，因为都是编译好的，没有了烦人的编译失败的过程了。

3. 运行ipython和在ipython上运行scrapy shell

在当前的cmd运行框中，因为已经配置了系统环境，是可以直接运行python包的，所以直接在cmd运行框中敲入ipython就会进入到ipython的运行框，跟系统标准的cmd差不多，但是功能更丰富，色彩更丰富，布局也能好。

但是当我直接在这上面敲scrapy shell命令，但是一直说没有此命令，失败。在这里卡住了。

后来通过仔细阅读scrapy shell的说明

If you have IPython installed, the Scrapy shell will use it (instead of the standard Python console).

意思就是scrapy shell会自己去找ipython的运行框。

于是直接在cmd的标准运行框中输入scrapy shell <url>，返回的结果直接调用到ipython的运行框了。

例子

scrapy shell “http://scrapy.org” --nolog

注意这里的双引号，之前我遵从官网的例子，写了单引号，一直返回invalid hostname，改为双引号之后，才解决问题。

English Version

1. scrapy shell

is a good interactive tool that Scrapy provided. Currently I use it to validate xpath result before I write scripts. After installed Scrapy, the Scrapy shell can be used from CMD.

For detail specification, please refer to below
https://docs.scrapy.org/en/latest/topics/shell.html
2. ipython

Scrapy recommand iPython as user interface to run Scrapy Shell, so I try to install iPython, which is also a Python extention package. As my Python env is from Conda, it's very easy to install iPython (how to install package in Conda env). Just run below command.

conda install -c conda-forge ipython

3. Run iPython, and run Scrapy Shell on iPython

How to run iPython? just key in "iPython" in your CMD prompt that has included Python path in system env path. A python-like user interface will appear, but more colorful, more functional and the layout is clearer.

So when I was in iPython interface, I try to key in "scrapy shell", but failed to work. The prompt said the command is not exist.

I was confused, and stuck here. However I found below note when I read scrapy shell specification again,

If you have IPython installed, the Scrapy shell will use it (instead of the standard Python console).

So I just key in "scrapy shell <url>" in original windows CMD prompt, the scrapy shell automatically run at iPython interface

example

scrapy shell “http://scrapy.org” --nolog

Note, I use double quotation mark " " instead of single quotation mark ' ' which is used in scrapy shell specification example.

it returns "invalid hostname" when I use single quotation mark.