您的位置:首页 > 编程语言 > Python开发

第一个python小程序--解析网页

2012-08-19 21:22 417 查看
很久就想学python了,但一直找不到合适的项目来练习,python的语法很有意思,写起来很简洁,今天有空边找资料边写出来了这一个小项目。考虑到3.x的python库还不是很多,大部分资料也是python2.x的,所以我用的python2.7来进行

之前就听说python访问网络很easy,这次是真的体会到了。很简单几句话搞定,不像java那样,再简单的访问都要装饰几层才能使用。

这次是拿糗事百科的网站,从上面抓取新鲜事并整理打印出来

不想多说了,以下上代码:

import urllib2
import sgmllib

class Entry:
author=''
content=''
pic=''
up = 0
down = 0
tag = ''
comment = 0
def to_string(self):
return '[Entry: author=%s content=%s pic=%s tag=%s up=%d down=%d comment=%d]'\
%(self.author,self.content,self.pic,self.tag,self.up,self.down,self.comment)

class MyHTMLParser(sgmllib.SGMLParser):
#所有用到的声明
#note all the datas
datas = []
# all the entries
entries = []
#the entry now
entry = Entry()
#last Unclosed tag
div_tag_unclosed = ''

def start_div(self,attrs):
for name,value in attrs:
if name =='class' and value == 'content':
self.div_tag_unclosed = 'content'
elif name=='class' and value == 'tags' :
self.div_tag_unclosed = 'tags'
elif name=='class' and value=='up':
self.div_tag_unclosed = 'up'
elif name=='class' and value == 'down':
self.div_tag_unclosed = 'down'
elif name=='class' and value=='comment':
self.div_tag_unclosed = 'comment'
elif name=='class' and value=='author':
self.div_tag_unclosed = 'author'
self.entry = Entry()
elif name=='class' and value=='thumb':
self.div_tag_unclosed = 'thumb'

def end_div(self):
if self.div_tag_unclosed == 'content' :
self.div_tag_unclosed =''
self.entry.content =  self.datas.pop().strip()
def start_a(self,attrs):pass
def start_img(self,attrs):
if self.div_tag_unclosed == 'thumb':
for name,value in attrs:
if name=='src':
self.div_tag_unclosed =''
self.entry.img = value.strip()
def end_img(self):pass
def end_a(self):
if self.div_tag_unclosed == 'author':
self.div_tag_unclosed =''
self.entry.author = self.datas.pop().strip()
if self.div_tag_unclosed == 'tags':
self.div_tag_unclosed =''
self.entry.tag = self.datas.pop().strip()
elif self.div_tag_unclosed == 'up':
self.div_tag_unclosed =''
self.entry.up = int(self.datas.pop().strip())
elif self.div_tag_unclosed == 'down':
self.div_tag_unclosed =''
self.entry.down = int(self.datas.pop().strip())
elif self.div_tag_unclosed == 'comment':
self.div_tag_unclosed =''
self.entry.comment = int(self.datas.pop().strip())
self.entries.append(self.entry)
def handle_data(self, data):
#        print 'data',data
self.datas.append(data)

#request the url
response = urllib2.urlopen('http://www.qiushibaike.com/8hr')
all = response.read()

#parse HTML
parser = MyHTMLParser()
parser.feed(all)
#print all the entries
for entry in parser.entries:
print entry.to_string()


整个程序很简单,用到了urllib2来请求网络,sgmllib来解析Html,由于第一次写python程序,所以写的时候效率很低,尤其是一直想在if后面加上小括号=-=
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: