Ajax数据爬取之微博m站(错误:TypeError: 'NoneType' object is not iterable)
2019-03-29 00:55
1071 查看
import requests from urllib.parse import urlencode from pyquery import PyQuery as pq base_url = 'https://m.weibo.cn/api/container/getIndex?' headers = { 'Referer': 'https://m.weibo.cn/u/2830678474', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36', 'x-requested-with': 'XMLHttpRequest', } def get_page(page): params = { 'type': 'uid', 'value': '2830678474', 'containerid': '1076032830678474', 'page': page } url = base_url + urlencode(params) try: response = requests.get(url, headers=headers) if response.status_code == 200: return response.json() except requests.ConnectionError as e: print('Error', e.args) def parse_page(json): if json: items = json.get('data').get('cards') for item in items: if item.get('mblog'): item = item.get('mblog') weibo = {} weibo['id'] = item.get('id') weibo['text'] = pq(item.get('text')).text() weibo['attitudes'] = item.get('attitudes_count') weibo['comments'] = item.get('comments_count') weibo['reposts'] = item.get('reposts_count') yield weibo if __name__ == '__main__': for page in range(1, 11): json = get_page(page) results = parse_page(json) for result in results: print(result) 代码参照了崔庆才老师的,但是在运行的时候出现了错误:TypeError: 'NoneType' object is not iterable 找了好久,才发现部分数据里面没有mblog这个key,所以在使用item.get('mblog')时,返回的数据为None。然后主动判断下item.get('mblog')是否为空,就可以搞定了。
相关文章推荐
- Python 出现错误TypeError: ‘NoneType’ object is not iterable解决办法
- 【Python】python "TypeError: 'NoneType' object is not iterable"错误分析
- Python错误代码分析:莫名的[TypeError: 'int' object is not iterable]
- TypeError: ‘NoneType’ object is not iterable
- python错误:TypeError: 'module' object is not callable
- python write()参数错误,导致报错TypeError: a bytes-like object is required, not 'str'
- Django: TypeError: 'MediaDefiningClass' object is not iterable
- SparkR去数据子集错误:object of type 'S4' is not subsettable
- Scrapy使用shell命令报错scrape shell TypeError: 'float' object is not iterable
- python错误: TypeError:'dict' object is not callable
- python错误:TypeError: 'module' object is not callable
- tensorflow代码运行的最后抛出异常:TypeError: 'NoneType' object is not callable
- Scrapy TypeError: 'float' object is not iterable
- Flask 中使用 JSON 响应页面的 Ajax 请求报错 TypeError: 'dict' object is not callable
- Anaconda3.6安装好Hyperopt包后,调用过程中出现错误TypeError”generator”object is not subscriptable。
- Anaconda下import caffe出现错误TypeError: 'module' object is not callable
- usecols = list(usecols) TypeError: 'int' object is not iterable
- Python中报“TypeError: 'int' object is not callable”错误的解决办法
- TypeError: 'WebElement' object is not iterable
- python TypeError: 'int' object is not iterable