处理Python中urllib2/mechanize库进行socket通信超时的问题

最近实验室的网络状况不太稳定，信息采集程序经常阻塞在通信时的recv()处，Python的urllib2/mechanize库在做HTTP访问时，是用socket方式进行通信，那么我们可以设置一个timeout值，来检测是否超时，并作出相应的异常处理。

异常处理的几个片段代码如下：

import socket
import urllib2
try:
resp = urllib2.urlopen(req, timeout=32)
except urllib2.URLError, e:
print "Bad URL or timeout"
#处理异常
except socket.timeout, e:
print "socket timeout"
#处理异常

import urllib2
import socket
#In Python 2.7.3
class MyException(Exception):
pass

try:
urllib2.urlopen("http://example.com", timeout = 32)
except urllib2.URLError as e:
print type(e)    #not catch
except socket.timeout as e:
print type(e)    #catched
raise MyException("There was an error: %r" % e)

下面这部分代码是stackoverflow.com的dbr所写，感谢他的帮助！

import urllib2
import socket

class MyException(Exception):
pass

try:
urllib2.urlopen("http://example.com", timeout = 1)
except urllib2.URLError, e:
# For Python 2.6
if isinstance(e.reason, socket.timeout):
raise MyException("There was an error: %r" % e)
else:
# reraise the original error
raise
except socket.timeout, e:
# For Python 2.7
raise MyException("There was an error: %r" %

下面是关于使用mechanize库的open方法（函数）时，捕获超时异常的代码，感谢stackoverflow.com中RoadieRich的帮助

(If you're using Python 2.6 or better, and a correspondingly updated version of mechanize, mechanize.urlopen should accept a timeout=... optional argument which seems to be what you're looking for... 具体地址: http://stackoverflow.com/questions/3552928/how-do-i-set-a-timeout-value-for-pythons-mechanize)

片段代码如下：

tried=0
connected = False
while not Connected:
try:
r = b.open('http://www.google.com/foobar', timeout=32)
connected = true # if line above fails, this is never executed
except mechanize.HTTPError as e:
print e.code
tried += 1
if tried > 4:
exit()
sleep(30)

except mechanize.URLError as e:
print e.reason.args
tried += 1
if tried > 4:
exit()
sleep(30)