使用webflux提升数据导出效率

序

本文主要研究一下如何使用webflux提升数据导出效率

传统导出

@GetMapping("/download-old")
public ResponseEntity<Resource> downloadInOldWays(){
return ResponseEntity.ok()
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=demo.xls")
.header("Accept-Ranges", "bytes")
.body(new ByteArrayResource(exportBytes(1000)));
}

public byte[] exportBytes(int dataRow){
StringBuilder output = new StringBuilder();
output.append(ExcelUtil.startWorkbook());
output.append(ExcelUtil.startSheet());
output.append(ExcelUtil.startTable());
output.append(ExcelUtil.writeTitleRow(Sets.newHashSet("title","content")));
IntStream.rangeClosed(1,dataRow).forEach(i -> {
try {
TimeUnit.MILLISECONDS.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
output.append(ExcelUtil.writeDataRow(Lists.newArrayList("title"+i,"content"+i)));
});
output.append(ExcelUtil.endTable());
output.append(ExcelUtil.endSheet());
output.append(ExcelUtil.endWorkbook());
return output.toString().getBytes(StandardCharsets.UTF_8);
}

这里模拟的是等所有数据都准备好了再导出，这种速度肯定慢，差不多需要等待100秒浏览器才能弹出下载框，如果前面有网关，很容易在网关那里超时了

webflux导出

@GetMapping("/download")
public Mono<Void> downloadByWriteWith(ServerHttpResponse response) throws IOException {
response.getHeaders().set(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=demo.xls");
response.getHeaders().add("Accept-Ranges", "bytes");
Flux<DataBuffer> flux = excelService.export(1000);
return response.writeWith(flux);
}

public Flux<DataBuffer> export(int dataRow){
return Flux.create(sink -> {
sink.next(stringBuffer(ExcelUtil.startWorkbook()));
sink.next(stringBuffer(ExcelUtil.startSheet()));
sink.next(stringBuffer(ExcelUtil.startTable()));

//write title row
sink.next(stringBuffer(ExcelUtil.writeTitleRow(Sets.newHashSet("title","content"))));
//write data row
IntStream.rangeClosed(1,dataRow).forEach(i -> {
try {
TimeUnit.MILLISECONDS.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
sink.next(stringBuffer(ExcelUtil.writeDataRow(Lists.newArrayList("title"+i,"content"+i))));
});

sink.next(stringBuffer(ExcelUtil.endTable()));
sink.next(stringBuffer(ExcelUtil.endSheet()));
sink.next(stringBuffer(ExcelUtil.endWorkbook()));

sink.complete();
});
}

这里使用ReactiveHttpOutputMessage的writeWith(Publisher<? extends DataBuffer> body)方法，实现边准备数据边导出
等待十几秒就弹下载框，之后就server端一边输出，浏览器一边下载，100秒左右下载完毕

小结

两种方法目前看来用时差不多，不过后者可以避免超时。当然使用传统mvc也可以实现类似效果，就是拿到response的输出流不断地write和flush。不过webflux可以配合reactive的repository，实现端到端的reactive stream，同时也可以避免OOM。