模拟浏览器登录网站
2013-09-07 00:49
381 查看
本程序采用JAVA所定,举个例子是模拟浏览器登录小米官网,采用的类主要是HttpURLConnection,HttpsURLConnectionURL请求的类别: 分二类,GET与POST请求。二者的区别在于:
a:) get请求可以获取静态页面,也可以把参数放在URL字符串后面,传递给servlet,
b:) post与get的不同之处在于post的参数不是放在URL字串里面,而是放在http请求的正文内。 小米官网登录分析,浏览器首先打开此地址,进行第一次会话,https://account.xiaomi.com/pass/serviceLogin,会话的过程中会保存会话ID,以后每次请求都需要附上此ID才行,否则服务器会认为用户是重新登录,有了此ID,服务器会认为是同一个用户此次请求是GET请求,相应的HttpURLConnction的设置如下
url=new URL(urlStr); httpConn=(HttpURLConnection) url.openConnection(); httpConn.setRequestMethod("GET"); HttpsURLConnection.setFollowRedirects(false); httpConn.setRequestProperty("User-Agent","Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; qdesk 2.4.1263.203; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727"); httpConn.setRequestProperty("Accept-Language","zh-cn"); httpConn.setRequestProperty("Accept", "application/msword, application/xaml+xml, application/x-ms-xbap, application/x-ms-application, */*"); httpConn.setRequestProperty("Accept-Encoding","gzip, deflate"); httpConn.setUseCaches(false); httpConn.setRequestProperty("Host","account.xiaomi.com"); httpConn.setRequestProperty("Connection", "Keep-Alive"); httpConn.setConnectTimeout(15000); httpConn.setReadTimeout(20000); httpConn.setAllowUserInteraction(false);
设置完后,即发送请求:
httpConn.getInputStream(); String cookieVal = null; String key=null; for (int i = 1; (key = httpConn.getHeaderFieldKey(i)) != null; i++ ) { System.out.println(key+"-----"+httpConn.getHeaderField(i)); if (key.equalsIgnoreCase("set-cookie")){ cookieVal = httpConn.getHeaderField(i); cookieVal = cookieVal.substring(0, cookieVal.indexOf(";")); System.out.println(cookieVal); String []str=sessionId.split(";"); boolean flag=false; for(int t=0;t<str.length;t++) { if(str[t]==cookieVal) flag=true; } if(!flag) { sessionId=sessionId+cookieVal+";"; } } } BufferedReader in = null; sessionId=sessionId.substring(0,sessionId.length()-1); InputStream inz=httpConn.getInputStream(); in = new BufferedReader(new InputStreamReader(inz,"UTF-8")); String str=httpConn.getHeaderField("Content-Encoding"); if(str!=null&&str.trim().endsWith("gzip")) { GZIPInputStream gzin = new GZIPInputStream(inz); in = new BufferedReader(new InputStreamReader(gzin, "UTF-8")); } String line=""; while((line=in.readLine())!=null) { System.out.println(line); }
上述代码会获取会话ID,JSESSION,注意服务器可能会返回gzip格式的数据,要进行相应的处理
有了会话ID后,即可进行登录,登录采用的是POST请求,POST请求参数是通过PrintWriter写入,从返回的数据中获得相应的cookie值作下一次使用
注意请求参数的设置,参数的设置可以查看页面源代码可知,Post请求需要传的参数为passToken,user,pwd,callback,sid,hidden
相应的HttpURLConnction的设置如下
url=new URL(urlStr); httpConn=(HttpsURLConnection) url.openConnection(); httpConn.setRequestMethod("POST"); HttpsURLConnection.setFollowRedirects(false); System.out.println("second:"+sessionId); httpConn.setRequestProperty("Cookie", sessionId); httpConn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded"); httpConn.setRequestProperty("Accept", "application/msword, application/xaml+xml, application/x-ms-xbap, application/x-ms-application, */*"); httpConn.setRequestProperty("User-Agent","Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; qdesk 2.4.1263.203; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727"); httpConn.setRequestProperty("Accept-Language","zh-cn"); httpConn.setRequestProperty("Accept-Encoding", "gzip, deflate"); httpConn.setRequestProperty("Connection", "Keep-Alive"); httpConn.setRequestProperty("Host","account.xiaomi.com"); httpConn.setRequestProperty("Content-Length",String.valueOf(param.getBytes())); httpConn.setRequestProperty("Referer","https://account.xiaomi.com/pass/serviceLogin"); httpConn.setDoOutput(true); httpConn.setDoInput(true); httpConn.setConnectTimeout(15000); httpConn.setUseCaches(false); httpConn.setAllowUserInteraction(false); httpConn.setInstanceFollowRedirects(false);
获得数据并设置cookie为进一步访问做准备
PrintWriter out = new PrintWriter(httpConn.getOutputStream()); out.print(param); out.flush(); out.close(); InputStream inz=httpConn.getInputStream(); in = new BufferedReader(new InputStreamReader(inz,"UTF-8")); String key=null; String cookieVal = null; String str=httpConn.getHeaderField("Content-Encoding"); if(str!=null&&str.trim().endsWith("gzip")) { GZIPInputStream gzin = new GZIPInputStream(inz); in = new BufferedReader(new InputStreamReader(gzin, "UTF-8")); } for (int i = 1; (key = httpConn.getHeaderFieldKey(i)) != null; i++ ) { System.out.println(key+"-----"+httpConn.getHeaderField(i)); if (key.equalsIgnoreCase("set-cookie")) { cookieVal = httpConn.getHeaderField(i); cookieVal = cookieVal.substring(0, cookieVal.indexOf(";")); if(cookieVal.indexOf("passToken")>-1) { passToken=cookieVal.substring(cookieVal.indexOf("=")+1); } if(cookieVal.indexOf("userId")>-1) { userId=cookieVal.substring(cookieVal.indexOf("=")+1); } } } String line; while ((line = in.readLine())!= null) { System.out.println(line); } httpConn.getInputStream().close();
写到这了,你就可以看到登录成功后你自已的用户信息,随后的每一次访问都需要先前设置的cookie
相关文章推荐
- C# 利用 HttpWebRequest 和 HttpWebResponse 模拟登录有验证码的网站
- [python 爬虫学习]利用cookie模拟网站登录
- cookielib和urllib2模块相结合模拟网站登录
- Python模拟浏览器登录淘宝
- python+selenium模拟浏览器登录shibboleth登录的模拟
- python模拟登录 网站
- C#模拟网站登录介绍
- Opera浏览器测试移动端网站和模拟手机浏览器的方法
- python爬虫模拟登录网站(一)-----豆瓣
- 用Java模拟网站登录
- 操纵IE浏览器模拟用户登录网站
- HtmlUnit模拟登录网站时注意事项
- php教程 CURL实现带有验证码网站的模拟登录的方法
- 在PC上测试移动端网站和模拟手机浏览器的5大方法
- 写了一个Windows服务,通过C#模拟网站用户登录并爬取BUG列表查询有没有新的BUG,并提醒我
- 我网站用session做的登录,为什么清除浏览器数据后还是得重新登录?session是存在服务器上的。
- 爬虫三部曲之(三):Python模拟登录云笔记网站并爬取笔记内容
- java模拟浏览器抓取网站信息和下载附件
- [置顶] Java代码模拟网站登录
- C# 利用 HttpWebRequest 和 HttpWebResponse 模拟登录有验证码的网站