Java后端爱上SpringBoot 第五节:Spring-Data-JPA批量提交

• Spring-Data-JPA批量提交
• EntityManager批量提交
• NamedParameterJdbcTemplate批量提交

Spring-Data-JPA批量提交

随便找一个实体的Repository来进行提交：
贴一下单元测试类：

@Test
public void test9() {

Optional<SysUser> optional = iSysUserRepository.findById("2c987c3167a23e990167a240e0c80008");

Long startTime=System.currentTimeMillis();
if (optional.isPresent()) {
List<SysUser> sysUsers = new ArrayList<>(10000);
// 提交1W個
for (int i = 0; i < 10000; i++) {
SysUser sysUser = new SysUser();
sysUser=optional.get();
sysUser.setId(String.valueOf(i));
sysUsers.add(sysUser);
}
iSysUserRepository.saveAll(sysUsers);
}
Long endTime=System.currentTimeMillis();

System.out.println("耗时："+(endTime-startTime)+"ms");

}

提交时间为：

又测了一次：

好慢啊！看一下saveAll()这个方法的源码。
于是我们找到了org.springframework.data.jpa.repository.support.SimpleJpaRepository<T, ID>的saveAll方法。

@Transactional
public <S extends T> List<S> saveAll(Iterable<S> entities) {

Assert.notNull(entities, "The given Iterable of entities not be null!");

List<S> result = new ArrayList<S>();

for (S entity : entities) {
result.add(save(entity));
}

return result;
}

再去看一下save方法：

@Transactional
public <S extends T> S save(S entity) {

if (entityInformation.isNew(entity)) {
em.persist(entity);
return entity;
} else {
return em.merge(entity);
}
}

发现每提交一个实体都会去 isNew 一下，去数据库查询这个实体是不是新的，如果是新的则进行提交，否则则进行update，于是乎我们自己实现一个EntityManager 提交的方法。

EntityManager批量提交

我们改造一下EntityManager的批量提交的方法，不用isNew来进行提交：

Optional<SysUser> optional = iSysUserRepository.findById("2c987c3167a23e990167a240e0c80008");
SysUser user = optional.get();

List<SysUser> sysUsers = new ArrayList<>();
Long startTime = System.currentTimeMillis();
if (optional.isPresent()) {
// 提交1W個
for (int i = 0; i < 10000; i++) {
SysUser sysUser = new SysUser();
sysUser.setEmail(user.getEmail());
sysUser.setLoginName(user.getLoginName());
sysUser.setPassword(user.getPassword());
sysUser.setPhone(user.getPhone());
sysUser.setStatus(user.getStatus());
sysUser.setUserName(user.getUserName());
sysUser.setModitime(new Date());
sysUser.setSysOrganization(user.getSysOrganization());
sysUser.setSysRoles(sysUser.getSysRoles());
sysUser.setSysStation(sysUser.getSysStation());
sysUsers.add(sysUser);
}
for (int i = 0; i < sysUsers.size(); i++) {
entityManager.persist(sysUsers.get(i));
if (i % 50 == 0) {
entityManager.flush();
entityManager.clear();
}
}
}
Long endTime = System.currentTimeMillis();
System.out.println("耗时：" + (endTime - startTime) + "ms");

效率略有提升。再测一下。

提交了10W数据，20秒。

NamedParameterJdbcTemplate批量提交

使用NamedParameterJdbcTemplate进行批量提交

@Autowired
private NamedParameterJdbcTemplate namedParameterJdbcTemplate;

//......

Optional<SysUser> optional = iSysUserRepository.findById("2c987c3167a23e990167a240e0c80008");
SysUser user = optional.get();

List<SysUser> sysUsers = new ArrayList<>();
Long startTime = System.currentTimeMillis();
if (optional.isPresent()) {
// 提交1W個
for (int i = 0; i < 10000; i++) {
SysUser sysUser = new SysUser();
sysUser.setId(String.valueOf(i));
sysUser.setEmail(user.getEmail());
sysUser.setLoginName(user.getLoginName());
sysUser.setPassword(user.getPassword());
sysUser.setPhone(user.getPhone());
sysUser.setStatus(user.getStatus());
sysUser.setUserName(user.getUserName());
sysUser.setModitime(new Date());
sysUser.setSysOrganization(user.getSysOrganization());
sysUser.setSysRoles(sysUser.getSysRoles());
sysUser.setSysStation(sysUser.getSysStation());
sysUsers.add(sysUser);
}
String sql = "insert into sys_user(id,email,login_name,moditime,password,phone,status,user_name) values (:id,:email,:loginName,:moditime,:password,:phone,:status,:userName)";
SqlParameterSource[] sqlParameterSource = SqlParameterSourceUtils.createBatch(sysUsers);
namedParameterJdbcTemplate.batchUpdate(sql, sqlParameterSource);
Long endTime = System.currentTimeMillis();
System.out.println("耗时：" + (endTime - startTime) + "ms");

}

再测一下

居然要50秒。