关于网络爬虫httpUrlConn.getResponseCode()取不到值得问题
2014-07-14 16:24
453 查看
今天有个人加我为好友,莫名其妙的就问我有没有整过网络爬虫,其实我网络学的不是很好,就说了不会,爱着面子又说我给他问问,我有个同事以前好像搞过,结果页没找到人,就硬着头皮看了下就几行代码:
try {
URL pageUrl = new URL("http://www.baidu.com/");
HttpURLConnection httpUrlConn = (HttpURLConnection) pageUrl
.openConnection();
int statusCode = httpUrlConn.getResponseCode();
System.out.println(statusCode);
} catch (Exception e) {
e.printStackTrace();
}
网上找了点资料看了看,上面说,这个不能再main这个主线程中跑,要另起一个线程来运行。
于是我就试了下:
import java.net.HttpURLConnection;
import java.net.URL;
public class TestUrl {
public static void main(String[] args) {
Test t = new Test();
Thread demo = new Thread(t);
demo.start();
}
}
class Test implements Runnable {
@Override
public void run() {
// TODO Auto-generated method stub
try {
URL pageUrl = new URL("http://www.baidu.com/");
HttpURLConnection httpUrlConn = (HttpURLConnection) pageUrl
.openConnection();
int statusCode = httpUrlConn.getResponseCode();
System.out.println(statusCode);
} catch (Exception e) {
e.printStackTrace();
}
}
}果然没问题了。这里记录下,下次有时间学习一下网络爬虫
try {
URL pageUrl = new URL("http://www.baidu.com/");
HttpURLConnection httpUrlConn = (HttpURLConnection) pageUrl
.openConnection();
int statusCode = httpUrlConn.getResponseCode();
System.out.println(statusCode);
} catch (Exception e) {
e.printStackTrace();
}
网上找了点资料看了看,上面说,这个不能再main这个主线程中跑,要另起一个线程来运行。
于是我就试了下:
import java.net.HttpURLConnection;
import java.net.URL;
public class TestUrl {
public static void main(String[] args) {
Test t = new Test();
Thread demo = new Thread(t);
demo.start();
}
}
class Test implements Runnable {
@Override
public void run() {
// TODO Auto-generated method stub
try {
URL pageUrl = new URL("http://www.baidu.com/");
HttpURLConnection httpUrlConn = (HttpURLConnection) pageUrl
.openConnection();
int statusCode = httpUrlConn.getResponseCode();
System.out.println(statusCode);
} catch (Exception e) {
e.printStackTrace();
}
}
}果然没问题了。这里记录下,下次有时间学习一下网络爬虫
相关文章推荐
- android请求http 关于connection.getResponseCode() 等于-1的问题
- Android 网络编程之HttpURLConnection(conn.getResponseCode() != 200,从url读取写入数据)
- HttpURLConnection调用getResponseCode()时抛出IOException异常
- 解析 HTTP(HttpURLConnection getResponseCode)
- 关于wp7中http的GetResponse抛异常的问题
- Web HttpServletRequest的getRequestURL方法获取不到https协议请求问题
- Android HttpURLConnection.getResponseCode()错误解决方法
- Android HttpURLConnection.getResponseCode() 505报错
- HttpURLConnection,getResponseCode,网络超时 相关总结。
- HttpURLConnection getResponseCode()方法阻塞
- 解析 HTTP(HttpURLConnection getResponseCode)
- 解析 HTTP(HttpURLConnection getResponseCode)
- 关于Servlet报错:405 HTTP method GET is not supported by this URL问题解决方法
- android中HttpURLConnection调用getResponseCode()时崩溃 解决方法
- getResponseCode获取URL响应状态码 HTTP状态返回码
- 解析 HTTP(HttpURLConnection getResponseCode)
- android 多线程环境下 HttpURLConnection getResponseCode 偶尔获取到的是 -1
- Android网络间接——HttpURLConnection.getResponseCode() 错误
- HttpURLConnection下载静态网页,getResponseCode的值是405
- 关于jetty和webx对于HttpServletResponse getWriter和getOutputStream的处理