paip.uapi 获取网络url内容html 的方法java php ahk c++ python总结.

paip.uapi 获取网络url内容html 的方法java php ahk c++ python总结.

各种语言总结比较，脚本php.python果然是方便.简短,实用.

uapi : get_web_contents,file_get_contents

作者Attilax 艾龙， EMAIL:1466519819@qq.com

来源：attilax的专栏

地址：http://blog.csdn.net/attilax

ahk

UrlDownloadToFile, URL, Filename

---------php:

$param= $_SERVER['QUERY_STRING'];

$url="http://latu.3g.cn/RP/EntLatu/photoappnav.php?".$param;

//echo $param;

$content=file_get_contents($url);

简评file_get_contents与curl 效率及稳定性(refer)

function curl_file_get_contents($durl){

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, $durl);

curl_setopt($ch, CURLOPT_TIMEOUT, 5);

curl_setopt($ch, CURLOPT_USERAGENT, _USERAGENT_);

curl_setopt($ch, CURLOPT_REFERER,_REFERER_);

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

$r = curl_exec($ch);

curl_close($ch);

return $r;

}

--------python

def getPage():

url = 'http://topic.csdn.net/u/20101007/17/C17A944B-806B-4C78-9D20-EF4CF33F99AA.html'

page = urllib.urlopen(url)

html = page.readlines();

return html

///添加头字段

def getPage(url):

'''下载文件html代码，找出一楼的核心代码'''

opener = urllib2.build_opener()

#不加头信息则出现403错误和乱码

opener.addheaders = [('User-agent', 'Mozilla/5.0')];

htmlAll = opener.open( url ).read()

reg1Floor = '<div class="msgfont">(.*?)</div>'

html = re.search(reg1Floor,htmlAll)

html = html.group()

#文件保存编码和文件编辑编码都是utf-8，所以decode一次，不然会出现乱码，但是不影响结果。

return html.decode('utf-8')

---------c++

QString atiNet::getHtml(QString url)

{

QNetworkAccessManager *manager = new QNetworkAccessManager();

QNetworkReply *reply = manager->get(QNetworkRequest(QUrl(url)));

QByteArray responseData;

QEventLoop eventLoop;

connect(manager, SIGNAL(finished(QNetworkReply*)), &eventLoop, SLOT(quit()));

eventLoop.exec(); //block until finish

responseData = reply->readAll();

return QString(responseData);

}

参考

paip.c++ qt 网页爬虫 的 网络编程 总结 - c++语言程序开发技术文章_c++编程