您的位置:首页 > 理论基础 > 计算机网络

利用HttpWebRequest抓取网页数据

2015-07-16 14:29 696 查看
根据运单号来获取运单状态 以EMS快递为例:

第一步首先分析要抓取网站的入口及参数 http://www.ems.com.cn
第二部利用httpwebRequest 模拟浏览器请求来获取网页内容

HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(url);
Encoding encoding = Encoding.UTF8;
string param = "muMailNum=" + mailCode + "&checkCode=" + code;
byte[] bs = Encoding.ASCII.GetBytes(param);

req.Method = "POST";
req.Timeout = 30000;
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = bs.Length;
req.CookieContainer = _cookieCon;
using (Stream reqStream = req.GetRequestStream())
{
reqStream.Write(bs, 0, bs.Length);
reqStream.Close();
}
using (HttpWebResponse response = (HttpWebResponse)req.GetResponse())
{
using (StreamReader reader = new StreamReader(response.GetResponseStream(), encoding))
{
responseData = reader.ReadToEnd().ToString();
}
}


获取整个网页内容
然后利用正则过滤出想要获得的信息
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: