urllib通过Post请求爬去数据并解析JSON(Python)
2018-03-20 17:13
1026 查看
1. 封装的请求post函数:def downloadPostPage(url, dictdata, headers, charset='utf-8', reqnum=5):
data = bytes(parse.urlencode(dictdata), encoding=charset)
req = request.Request(url, data, headers=headers,method='POST')
info= None
try:
response = request.urlopen(req)
info = response.read().decode(charset)
except error.HTTPError as e:
# 服务器错误
print(e.code)
if reqnum > 0:
if hasattr(e, 'code') and 500 <= e.code <= 600:
time.sleep(random.randint(5, 11))
downloadPostPage(url,dictdata,headers, charset=charset, reqnum=reqnum-1)
except error.URLError:
print("url error")
return info2. 主函数的实现:if __name__ == '__main__':
url = 'http://fanyi.youdao.com/translate?smartresult=dict&smartresult=rule&smartresult=ugc&sessionFrom=null'
headers = {
"Accept": "application/json, text/javascript, */*; q=0.01",
"X-Requested-With": "XMLHttpRequest",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0",
"Content-Type": "application/x-www-form-urlencoded; charset=UTF-8",
}
while True:
key = input('输入搜索词:')
if not key:
break
dictdata = {
"i": key,
"from": "auto",
"to": "auto",
"smartresult": "dict",
"client": "fanyideskweb",
"salt": "1511219405946",
"sign": "f8965f67a1d3eee8a69ddf8ccc5f582b",
"doctype": "json",
"version": "2.1",
"keyfrom": "fanyi.web",
"action": "FY_BY_REALTIME",
"typoResult": "false"
}
info = downloadPostPage(url, dictdata, headers=headers,reqnum=1)
jsonLoads = json.loads(info)
print(jsonLoads)
print(jsonLoads['translateResult'][0][0]['tgt'])
data = bytes(parse.urlencode(dictdata), encoding=charset)
req = request.Request(url, data, headers=headers,method='POST')
info= None
try:
response = request.urlopen(req)
info = response.read().decode(charset)
except error.HTTPError as e:
# 服务器错误
print(e.code)
if reqnum > 0:
if hasattr(e, 'code') and 500 <= e.code <= 600:
time.sleep(random.randint(5, 11))
downloadPostPage(url,dictdata,headers, charset=charset, reqnum=reqnum-1)
except error.URLError:
print("url error")
return info2. 主函数的实现:if __name__ == '__main__':
url = 'http://fanyi.youdao.com/translate?smartresult=dict&smartresult=rule&smartresult=ugc&sessionFrom=null'
headers = {
"Accept": "application/json, text/javascript, */*; q=0.01",
"X-Requested-With": "XMLHttpRequest",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0",
"Content-Type": "application/x-www-form-urlencoded; charset=UTF-8",
}
while True:
key = input('输入搜索词:')
if not key:
break
dictdata = {
"i": key,
"from": "auto",
"to": "auto",
"smartresult": "dict",
"client": "fanyideskweb",
"salt": "1511219405946",
"sign": "f8965f67a1d3eee8a69ddf8ccc5f582b",
"doctype": "json",
"version": "2.1",
"keyfrom": "fanyi.web",
"action": "FY_BY_REALTIME",
"typoResult": "false"
}
info = downloadPostPage(url, dictdata, headers=headers,reqnum=1)
jsonLoads = json.loads(info)
print(jsonLoads)
print(jsonLoads['translateResult'][0][0]['tgt'])
相关文章推荐
- Struts2 Action接收POST请求JSON数据及其实现解析
- Struts2 Action接收POST请求JSON数据及其实现解析
- Java post请求发送json数据在filter中解析方法
- PHP如何通过Http Post请求发送Json对象数据?
- Python3.6通过自带的urllib通过get或post方法请求url的实例
- android通过httpClient请求获取JSON数据并且解析
- java 通过发送json,post请求,返回json数据
- python 解析http post传递的数据(FieldStorage类 如何 解析http body中json数据)
- python实现的json数据以HTTP GET,POST,PUT,DELETE方式页面请求
- android通过httpClient请求获取JSON数据并且解析
- python实现的json数据以HTTP GET,POST,PUT,DELETE方式页面请求
- C++通过HTTP请求Get或Post方式请求Json数据
- [SoapUI] 通过SoapUI发送POST请求,请求的body是JSON格式的数据
- Python Flask之处理客户端通过POST方法传送的数据(json文本,文件)
- java通过url请求服务器端json数据并解析成想要的数据
- Python3爬虫之urllib爬取异步Ajax数据,使用post请求!
- android通过httpClient请求获取JSON数据并且解析
- python http请求post参数,json解析
- python实现的json数据以HTTP GET,POST,PUT,DELETE方式页面请求
- 六、python urllib2请求json数据