python爬虫：抓取页面上的超链接

Beautiful Soup 是一个可以从HTML或XML文件中提取数据的Python库.它能够通过你喜欢的转换器实现惯用的文档导航,查找,修改文档的方式.Beautiful Soup会帮你节省数小时甚至数天的工作时间.

页面上的超链接

在HTML中，超链接用a表示，链接地址写作 href=。。。

<a href='http://www.baidu.com'>baidu</a>

发布到浏览器上就是：



点击这个超链接可以链接到相应的网页上去：、

用bs4抓取超链接

抓取豆瓣主页的所有超链接

from urllib.request import urlopen#用于获取网页
from bs4 import BeautifulSoup#用于解析网页

html = urlopen('https://www.douban.com/')
bsObj = BeautifulSoup(html, 'html.parser')
t1 = bsObj.find_all('a')
for t2 in t1:
t3 = t2.get('href')
print(t3)

控制台打印结果：

https://book.douban.com

https://movie.douban.com

https://music.douban.com

https://www.douban.com/group/

https://www.douban.com/location/

https://douban.fm

https://www.douban.com/time/?dt_time_source=douban-web_anonymous_index_top_nav

https://market.douban.com?

等等等等。。。。。（太多了，不粘贴了）

抓取图片连接

from urllib.request import urlopen#用于获取网页
from bs4 import BeautifulSoup#用于解析网页

html = urlopen('https://www.douban.com/')
bsObj = BeautifulSoup(html, 'html.parser')
t1 = bsObj.find_all('img')
for t2 in t1:
t3 = t2.get('src')
print(t3)

控制台打印结果：

https://www.douban.com/pics/blank.gif

https://img3.doubanio.com/f/sns/0c708de69ce692883c1310053c5748c538938cb0/pics/sns/anony_home/icon_qrcode_green.png

https://img3.doubanio.com/f/sns/1cad523e614ec4ecb6bf91b054436bb79098a958/pics/sns/anony_home/doubanapp_qrcode.png

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img1.doubanio.com/dae/niffler/niffler/images/513f432a-6545-11e7-a372-0242ac110018.png

https://img1.doubanio.com/dae/niffler/niffler/images/416add14-0254-11e7-8292-0242ac11002c.jpg

https://img1.doubanio.com/dae/niffler/niffler/images/7bd08eba-0496-11e7-a4c9-0242ac110038.png

https://img1.doubanio.com/dae/niffler/niffler/images/54d34448-1dea-11e7-afa9-0242ac11002c.jpg

https://img3.doubanio.com/dae/niffler/niffler/images/6db31128-39dd-11e7-895e-0242ac110016.png

https://img3.doubanio.com/dae/niffler/niffler/images/2621f522-50ad-11e7-a3d6-0242ac110041.png

https://img3.doubanio.com/dae/niffler/niffler/images/c3ab871e-2401-11e7-9fe8-0242ac110046.png

https://img1.doubanio.com/dae/niffler/niffler/images/7bea8578-3f69-11e7-8261-0242ac11001b.png

https://img3.doubanio.com/dae/niffler/niffler/images/5d4dd498-5676-11e7-bb3d-0242ac11002d.jpg

https://img3.doubanio.com/pics/new_menu.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif
https://img3.doubanio.com/f/shire/a1 d0ef
fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/pics/new_menu.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/img/songlist/large/476033-1.jpg

https://img3.doubanio.com/img/songlist/large/7610723-3.jpg

https://img3.doubanio.com/img/songlist/large/1321480-2.jpg

https://img3.doubanio.com/img/songlist/large/16075-2.jpg

https://img3.doubanio.com/img/songlist/large/449323-1.jpg

https://img3.doubanio.com/img/songlist/large/464547-1.jpg

https://img3.doubanio.com/view/dianpu_shop_icon/medium/public/2e35f89d2c7990d.jpg

https://img3.doubanio.com/view/dianpu_shop_icon/medium/public/9d03efbf705887d.jpg

https://img3.doubanio.com/view/dianpu_shop_icon/medium/public/169b1b76356f636.jpg

https://img3.doubanio.com/view/dianpu_shop_icon/medium/public/7f37444b0a6a0e2.jpg

https://img3.doubanio.com/view/dianpu_product_item/medium/public/p1971380.jpg

https://img3.doubanio.com/view/dianpu_product_item/medium/public/p458880.jpg

https://img3.doubanio.com/view/dianpu_product_item/medium/public/p270364.jpg

https://img3.doubanio.com/view/dianpu_product_item/medium/public/p521646.jpg

https://img1.doubanio.com/view/dianpu_product_item/medium/public/p509169.jpg

https://img3.doubanio.com/view/dianpu_product_item/medium/public/p377790.jpg

https://img3.doubanio.com/view/dianpu_product_item/medium/public/p1971380.jpg

https://img3.doubanio.com/view/dianpu_product_item/medium/public/p377790.jpg

https://img3.doubanio.com/view/dianpu_product_item/medium/public/p458880.jpg

https://img3.doubanio.com/view/dianpu_product_item/medium/public/p270364.jpg

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/f/shire/a1fdee122b95748d81cee426d717c05b5174fe96/pics/blank.gif

https://img3.doubanio.com/pics/biaoshi.gif

https://img3.doubanio.com/img/files/file-1423193113.png

https://img3.doubanio.com/pics/icon/jubao.png

https://img3.doubanio.com/img/files/file-1489464722.jpg