登录下载网络资源之Java代码

DevStore，一个源码的下载站。下载源码一个一个下载太麻烦了，所以就想着写个批量下载的。

首先看看单个下载的过程。

例如按关键词转发短信至指定手机号，当下载时需要登录，这里注册一个账号，进行下载。





这里会有两个Post

一个是检查花费的活力值，一个是扣除活力值

Post->Get

两个地址分别为：

http://www.devstore.cn/code/checkDownSourceCodeUserPointsAjax?sourceCodeId=715
响应：<pre role="list"><code class="wrappedText focusRow" role="listitem">{"price":0,"status":2}</code>

http://www.devstore.cn/code/costSourceCodeUserPointsAjax?sourceCodeId=715

响应：{

"status": 1,

"url": "http://ds.devstore.cn/dev_store/user/souce_code/file/20150227043055827e0vbr74d/按关键词转发短信至指定手机号.zip?87d45372b2babb3a34570fd107cac9747900e64e21876aa613ac0910e7d5d689b9a4cdf80e67f3bac16cad6fa4ca5b88d818e8a07f8f44e19604f2362a677492501b7eed6bbbc114e1687347f00ea624a069c08baf721201f502b82d3ccc3197"

}

真实的下载地址是

http://ds.devstore.cn/dev_store/user/souce_code/file/20150227043055827e0vbr74d/按关键词转发短信至指定手机号.zip

下载的代码

/**
*
* @param url  下载地址
* @param localPath 保存的路径
*/
public static void downLoad(String url,String localPath){

URL fileUrl = null;

HttpURLConnection httpUrl = null;

BufferedInputStream bis = null;

BufferedOutputStream bos = null;

File file = new File(localPath);

try {
fileUrl = new URL(url);
httpUrl = (HttpURLConnection) fileUrl.openConnection();
httpUrl.connect();
bis = new BufferedInputStream(httpUrl.getInputStream());
bos = new BufferedOutputStream(new FileOutputStream(file));
int len = 2048;
byte[] b = new byte[len];
while ((len = bis.read(b))!= -1) {
bos.write(b,0,len);

}
bos.flush();
bis.close();
httpUrl.disconnect();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally{
try {
bis.close();
bos.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

}
}

下载的代码好了，接下来就是登陆和获取下载地址的代码

直接把代码贴上吧

public class DownLoad {

/**
* @param 用户名
* @param 密码
* @return cookie
*/
public static Map<String, String> login(String userName,String passWord){
Map<String, String> cookies = null;

Map<String, String> map = new HashMap<String, String>();

map.put("pwd", passWord);
map.put("userName", userName);
Response response = null;
try {
response = Jsoup.connect("http://www.devstore.cn/user/login").
timeout(5000).data(map).ignoreContentType(true).execute();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
cookies = response.cookies();
return cookies;
}

/**
* 获取下载的链接
* @param url  下载的原地址
* @param cookies
* @return
*/
public static String getdownUrl(String url,Map<String, String> cookies){
String result = null;
Response response = null;
JSONObject json = null;
String id = (getInfoByUrl(url)).split("\\.")[0];
try {
response = Jsoup.connect("http://www.devstore.cn/code/checkDownSourceCodeUserPointsAjax").
timeout(5000).data("sourceCodeId",id).cookies(cookies).ignoreContentType(true).execute();
json = new JSONObject(response.body());
result = json.get("url").toString().split("\\?")[0];
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return result;
}

/**
* 截取
* @param url
* @return
*/
public static String getInfoByUrl(String url){
String result = null;
int length = url.split("\\/").length;
result = url.split("\\/")[length - 1];
return result;
}

/**
*
* @param url  下载地址
* @param localPath 保存的路径
*/
public static void downLoad(String url,String localPath){
String fileName = null;

URL fileUrl = null;

HttpURLConnection httpUrl = null;

BufferedInputStream bis = null;

BufferedOutputStream bos = null;

fileName = getInfoByUrl(url);

File file = new File(localPath + File.separator + fileName);

try {
fileUrl = new URL(url);
httpUrl = (HttpURLConnection) fileUrl.openConnection();
httpUrl.connect();
bis = new BufferedInputStream(httpUrl.getInputStream());
bos = new BufferedOutputStream(new FileOutputStream(file));
int len = 2048;
byte[] b = new byte[len];
while ((len = bis.read(b))!= -1) {
bos.write(b,0,len);

}
bos.flush();
bis.close();
httpUrl.disconnect();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally{
try {
bis.close();
bos.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

}
}
}

测试代码

Map<String, String> cookies = DownLoad.login("用户名", "密码");

String url = DownLoad.getdownUrl("http://www.devstore.cn/code/info/715.html", cookies);

DownLoad.downLoad(url, "G:\\src");

批量下载的话可以先获取页面中所有的下载链接

Document doc = Jsoup.connect("http://www.devstore.cn/code/list/pn1-or0.html").get();

Element element = doc.select("ul.source_list_ul").first();
//System.out.println(element);
Elements links = element.select("h4").select("a");
for (Element link : links) {
System.out.println(link.attr("href"));
}

代码如上。

返回结果类似

/code/info/715.html
/code/info/712.html
/code/info/713.html
/code/info/714.html
/code/info/708.html
....
....

链接有了，然后循环执行下载代码即可