Windows Live Writer客户端在51CTO写博客

简单小爬虫

#!/usr/bin/env python
#coding:utf-8
import urllib2
import bs4
url = 'http://www.163.com'
content = urllib2.urlopen(url).read()
content = content.decode('gbk')

soup = bs4.BeautifulSoup(content)
links = soup.select('li a[href]')

result = []
for link in links:
href = link.attrs['href']
title = link.text
if '.html' in href and '163.com' in href and len(title) >3:
result.append(link)
for link in result:
print link.attrs['href'], link.text

print '共有新闻[%s]条', len(result)

本文出自 “Linux_Config” 博客，请务必保留此出处http://liang1026.blog.51cto.com/10119067/1681675