Oracle海量数据导入MongoDB 采用JAVA反射-Penghao

首先人员表数据一共有100W条，前期开始是想先用PL/SQL导出为xml格式，然后在SAX解析xml，后来发现xml超过10W条的时候解析起来会有问题，特别占用CPU资源，后改用java程序直接连oracle和mongo，该表有50多个全大写字段,如果先获取到oracle表的列名和值,再写条件判断 if(列名==类属性){ 新实例赋值}, 要写50多个if代码块.觉得用反射更代码更简洁一点。

PatMaster实体类中的NAME字段

String NAME;
public String getNAME() {
return NAME;
}
public void setNAME(String nAME) {
NAME = nAME;
}

Class<PatMaster> c = PatMaster.class;

PatMaster patMaster = c.newInstance();

Method m1 = c.getDeclaredMethod("set" + column, String.class);

m1.setAccessible(true);

m1.invoke(patMaster, value);

column为oracle获取的字段列，String.class是指改set方法需要传入一个String，那么m1就获取到这个set方法了,如果不是public方法，需要setAccessible(true);然后invoke的时候传入实例和数据库中该字段值即可.

全部代码：直接拷过去就可以用，你改个IP定义个你的实体类就好了, 如果搞不定加我QQ468165108

用到的jar包 ：ojdbc14.jar，mongo-2.5.3.jar

JAR下载地址：http://pan.baidu.com/s/1kTGNVoj 密码：69fz ,

public void OdbcTOMongo() throws Exception {

Mongo mongo = new Mongo("192.168.1.3", 30000);
DB db = mongo.getDB("test");
db.authenticate("user", "user".toCharArray());
DBCollection dbc = db.getCollection("patMasterIndex");

//自定义数据字典
FieldfUtils fu = new FieldfUtils();
HashMap<String, String> map = fu.GetMinField();

Class.forName("oracle.jdbc.driver.OracleDriver");
String url = "jdbc:oracle:thin:@192.168.1.111:1521:dba";
String user = "user";
String password = "pwd";
Connection conn = DriverManager.getConnection(url, user, password);
PreparedStatement ps = conn.prepareStatement("select * from PAT_MASTER_INDEX");
ResultSet rs = ps.executeQuery();
int num = 0;
ResultSetMetaData rsmd = rs.getMetaData();
int colCount = rsmd.getColumnCount();

while (rs.next()) {

//计数器 每导入1W行记录输入
num++;
if(num%10000==0){
System.out.println(num);
}
Class<PatMaster> c = PatMaster.class;
PatMaster patMaster = c.newInstance();
BasicDBObject bdo = new BasicDBObject();

for (int i = 1; i <= colCount; i++) {
String column = rsmd.getColumnName(i);
Object object = rs.getObject(i);
String value = "";
if (object == null) {
} else {
value = (String) object.toString();
}
Method m1 = c.getDeclaredMethod("set" + column, String.class);
m1.setAccessible(true);
m1.invoke(patMaster, value);

String minName = map.get(column);
bdo.put(minName, value);
}
dbc.insert(bdo);
}

rs.close();
ps.close();
conn.close();
}

}

public class FieldfUtils {

public HashMap<String, String> GetPatMasterMinField() throws Exception{

Properties pro = new Properties();

InputStream in =getClass().getResourceAsStream("PAT_MASTER_Min.properties");

pro.load(in);

HashMap<String, String> mapMaxMin = new HashMap<String, String>();

HashMap<String, String> mapMinMax = new HashMap<String, String>();

Iterator it=pro.entrySet().iterator();

while(it.hasNext()){

Map.Entry entry=(Map.Entry)it.next();

String key = (String) entry.getKey();

String value = (String) entry.getValue();

mapMaxMin.put(key, value);

mapMinMax.put(value,key);

}

return mapMaxMin;

}

}

PAT_MASTER_Min.properties-----------------------------

PATIENT_ID=a1

INP_NO=a2

NAME=a3

NAME_PHONETIC=a4

SEX=a5

DATE_OF_BIRTH=a6

BIRTH_PLACE=a7

CITIZENSHIP=a8

#

PATIENT_ID=a1

左边是oralce字段 右边是Mongdb字段 自己定义就行了