您的位置:首页 > 理论基础 > 计算机网络

Python网络爬虫实训:如何下载韩寒博客文章

2014-06-16 21:07 681 查看
根据智普培训视频,将抓取韩寒博客文章的Python代码记录如下:

#coding:utf-8

import urllib
import time

url = ['']*350
page = 1
link = 1
while page <= 7:
con = urllib.urlopen('http://blog.sina.com.cn/s/articlelist_1191258123_0_'+str(page)+'.html').read()
i = 0
title = con.find(r'<a title=')
href = con.find(r'href=',title)
html = con.find(r'.html',href)

while title != -1 and href != -1 and html != -1 and i < 50:
url[i] = con[href + 6 : html + 5]
print link, url[i]
content = urllib.urlopen(url[i]).read()
open(r'hanhan/'+url[i][-26:],'w+').write(content)
print 'downloading', url[i]
time.sleep(1)
title = con.find(r'<a title=', html)
href = con.find(r'href=', title)
html = con.find(r'.html', href)
i = i + 1
link = link + 1
else:
print page,'find end!'
page = page + 1
else:
print 'all find end'
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: