python网络爬虫采集联想词示例
2014-02-11 00:00
821 查看
python爬虫_采集联想词代码
#coding:utf-8 import urllib2 import urllib import re import time from random import choice #特别提示,下面这个list中的代理ip可能失效,请换上有效的代理ip iplist = ['27.24.158.153:81','46.209.70.74:8080','60.29.255.88:8888'] list1 = ["集团","科技"] for item in list1: ip= choice(iplist) gjc = urllib.quote(item) url = "http://sug.so.360.cn/suggest/word?callback=suggest_so&encodein=utf-8&encodeout=utf-8&word="+gjc headers = { "GET":url, "Host":"sug.so.360.cn", "Referer":"http://www.so.com/", "User-Agent":"sMozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56 Safari/537.17", } proxy_support = urllib2.ProxyHandler({'http':'http://'+ip}) opener = urllib2.build_opener(proxy_support) urllib2.install_opener( opener ) req = urllib2.Request(url) for key in headers: req.add_header(key,headers[key]) html = urllib2.urlopen(req).read() ss = re.findall("\"(.*?)\"",html) for item in ss: print item time.sleep(2)
相关文章推荐
- python重试装饰器示例
- python中的yield使用方法
- Python的print用法示例
- 优秀Python学习资源
- Python学习笔记(1)
- 优秀Python学习资源收集汇总(强烈推荐)
- [Python] The project structure in Django
- python - django - 初步认识
- 优秀Python学习资源收集汇总(强烈推荐)
- Python_C4_作出决策【上】
- Python菜鸟学习第三课
- Python字典排序的简单实现
- python中单引号,双引号,多引号区别
- Python_C3_变量【下】
- python:UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xef in position xxx: ordinal not in range(128)
- python 字符串转时间
- MySQL-python
- PHP / Web / Python
- 编程之美1.3 一摞烙饼的排序(Python)
- python的性能分析工具line_profiler