FileInputStream RandomAccessFile BufferedReader MappedByteBuffer 大文件数据读取效率测试
2013-03-08 14:41
483 查看
文件大小为67M:
readGeneralFile:16981ms
readRandomFile:16868ms
readBufferInput:305ms
readMappedFile:11ms
MappedByteBuffer胜!
另外测试了6G大小的文件(只测试了BufferReader和MappedByteBuffer),BufferReader 可以顺利读下来,MappedByteBuffer 文件size()大小超出了Integer的最大值,需要分段读取。
做了按行读取测试,结果是 BufferReader 快于 MappedByteBuffer,应该是BufferReader.readLine() 优化过!但MappedByteBuffer 没有readLine()函数,自己判断10和13实现,水平有限效率不高。比较下来MappedByteBuffer 慢。
因为工作中用到读取行,只做了MappedByteBuffer和BufferReader的读取行的测试,没有测试比较读取字节的性能。
public class App { String generalFile = "/opt/A"; public void generalFileRead(String fileName) { try { System.out.print("readGeneralFile"); FileInputStream in = new FileInputStream(fileName); readIn(in); } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } } public void randomFileRead(String fileName) { try { System.out.print("readRandomFile"); RandomAccessFile file = new RandomAccessFile(fileName, "r"); int i = file.read(); while (i != -1) { i = file.read(); } } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } } public void bufferRead(String fileName) { try { System.out.print("readBufferInput"); InputStream in = new BufferedInputStream(new FileInputStream( fileName)); BufferedReader bf = new BufferedReader(new InputStreamReader(in)); while(bf.readLine()!=null){ bf.readLine(); } } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } } private void readIn(InputStream in) throws IOException { int i = in.read(); while (i != -1) { i = in.read(); } } public void mappedByteBufferRead(String fileName) { try { System.out.print("readMappedFile"); FileInputStream in = new FileInputStream(fileName); FileChannel channel = in.getChannel(); int length = (int) channel.size(); MappedByteBuffer buffer = channel.map( FileChannel.MapMode.READ_ONLY, 0, length); while(buffer.remaining()>0){ buffer.get(); } } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } } public void countTime(long start) { System.out.println(":" + (System.currentTimeMillis() - start) + "ms"); } public static void main(String[] args) { App app = new App(); long start = System.currentTimeMillis(); app.generalFileRead(app.generalFile); app.countTime(start); start = System.currentTimeMillis(); app.randomFileRead(app.generalFile); app.countTime(start); start = System.currentTimeMillis(); app.bufferRead(app.generalFile); app.countTime(start); start = System.currentTimeMillis(); app.mappedByteBufferRead(app.generalFile); app.countTime(start); } }
readGeneralFile:16981ms
readRandomFile:16868ms
readBufferInput:305ms
readMappedFile:11ms
MappedByteBuffer胜!
另外测试了6G大小的文件(只测试了BufferReader和MappedByteBuffer),BufferReader 可以顺利读下来,MappedByteBuffer 文件size()大小超出了Integer的最大值,需要分段读取。
做了按行读取测试,结果是 BufferReader 快于 MappedByteBuffer,应该是BufferReader.readLine() 优化过!但MappedByteBuffer 没有readLine()函数,自己判断10和13实现,水平有限效率不高。比较下来MappedByteBuffer 慢。
因为工作中用到读取行,只做了MappedByteBuffer和BufferReader的读取行的测试,没有测试比较读取字节的性能。
相关文章推荐
- RandomAccessFile、FileChannel、MappedByteBuffer读写文件
- MappedByteBuffer高速缓存文件、RandomAccessFile随机访问
- RandomAccessFile、FileChannel、MappedByteBuffer读写文件
- MappedByteBuffer+RandomAccessFile读写文件
- (转) Java RandomAccessFile与MappedByteBuffer
- Java RandomAccessFile与MappedByteBuffer
- RandomAccessFile和MappedByteBuffer
- Java的RandomAccessFile,FileInputStream,InputStreamReader,BufferedReader,FileOutputStream,OutputStreamWriter,BufferedWriter
- RandomAcessFile、MappedByteBuffer和缓冲读/写文件
- 内存映射文件:MappedByteBuffer
- RandomAcessFile、MappedByteBuffer和缓冲读/写文件
- RandomAcessFile、MappedByteBuffer和缓冲读/写文件
- Java NIO之内存映射文件——MappedByteBuffer
- MappedByteBuffer读写文件
- 为何要在Java中使用内存映射文件(Memory Mapped File)或者MappedByteBuffer
- MappedByteBuffer高速缓存文件、RandomAccessFile随机访问
- 高效率读取大文件进行数据处理-通过扩展RandomAccessFile类使之具备Buffer改善I/O性能
- java nio MappedByteBuffer 文件映射
- 使用MappedByteBuffer读取大文件(1G以上)和释放MappedByteBuffer的资源
- 高效的文件拷贝之MappedByteBuffer