mina高并发短连接报java.io.IOException: Too many open files
2015-08-27 16:44
519 查看
mina高并发短连接导致java.io.IOException: Too many open files解决方案
这几天在解决一个用mina开发的高并发通信过程中产生的一个bug。
模拟场景为:
通过定时触发启动线程模拟高并发短连接测试,测试的服务端有2个,一个是服务有起,一个没起,客户端和服务端均在同一服务器上。执行一段时间后linux主机上通过lsof命令查看,发现有递增的文件句柄,pipe和eventpoll。
抛出的异常如下:
Failed to create a new instance of org.apache.mina.transport.socket.nio.NioProcessor: null
java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedConstructorAccessor110.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27 )
at java.lang.reflect.Constructor.newInstance(Constructor.java:513 )
at org.apache.mina.core.service.SimpleIoProcessorPool.<init>(SimpleIoProcessorPool.java:180 )
at org.apache.mina.core.service.SimpleIoProcessorPool.<init>(SimpleIoProcessorPool.java:112 )
at org.apache.mina.core.polling.AbstractPollingIoConnector.<init>(AbstractPollingIoConnector.java:93 )
at org.apache.mina.transport.socket.nio.NioSocketConnector.<init>(NioSocketConnector.java:56 )
at com.develop.webplatform.funnel.client.JobClient.sendMessage(JobClient.java:39 )
at com.develop.webplatform.funnel.client.JobClient.sendJob(JobClient.java:126 )
at com.develop.webplatform.funnel.extend.JobExecRemotelyBySocket.execJobByTask(JobExecRemotelyBySocket.java:66 )
at com.develop.webplatform.funnel.JobManager.execJobByTask(JobManager.java:27 )
at com.develop.webplatform.quartz.job.TaskJob.executeInternal(TaskJob.java:38 )
at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86 )
at org.quartz.core.JobRunShell.run(JobRunShell.java:223 )
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549 )
Caused by: org.apache.mina.core.RuntimeIoException: Failed to open a selector.
at org.apache.mina.transport.socket.nio.NioProcessor.<init>(NioProcessor.java:61 )
... 15 more
Caused by: java.io.IOException: Too many open files
at sun.nio.ch.IOUtil.initPipe(Native Method)
at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:49 )
at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18 )
at java.nio.channels.Selector.open(Selector.java:209 )
at org.apache.mina.transport.socket.nio.NioProcessor.<init>(NioProcessor.java:59 )
... 15 more
原代码中,关于客户端连接的代码如下:
final NioSocketConnector connector = new NioSocketConnector();
final String[] result = new String[ 1 ];
connector.getFilterChain().addLast("codec" ,
new ProtocolCodecFilter( new ObjectSerializationCodecFactory()));
connector.setHandler(handler);
//设置超时
connector.setConnectTimeoutMillis(defaultConnectTimeOut);
ConnectFuture connectFuture = connector.connect(address);
connectFuture.awaitUninterruptibly(); //同步,等待,直到连接完成
if (connectFuture.isDone()) {
if (!connectFuture.isConnected()) { //若在指定时间内没连接成功,则抛出异常
logger.info("fail to connect " + logInfo);
throw new Exception();
}
}
经过分析,导致主机文件句柄泄露的原因为,客户端发起服务端连接时,会请求系统分配相关的文件句柄,在原代码中,仅仅判断是否连接成功,而未对连接失败进 行资源释放,从而造成文件句柄泄露。当总的文件句柄数超过系统设置值(ulimit -n 查看同一个进程允许的最大文件句柄数),则抛出异常“java.io.IOException:
Too many open files”,导致无法创建新的连接,服务器挂掉。
更改后的代码如下:
final NioSocketConnector connector = new NioSocketConnector();
final String[] result = new String[ 1 ];
connector.getFilterChain().addLast("codec" ,
new ProtocolCodecFilter( new ObjectSerializationCodecFactory()));
connector.setHandler(handler);
//设置超时
connector.setConnectTimeoutMillis(defaultConnectTimeOut);
ConnectFuture connectFuture = connector.connect(address);
connectFuture.awaitUninterruptibly(); //同步,等待,直到连接完成
if (connectFuture.isDone()) {
if (!connectFuture.isConnected()) { //若在指定时间内没连接成功,则抛出异常
logger.info("fail to connect " + logInfo);
connector.dispose(); //不关闭的话会运行一段时间后抛出,too many open files异常,导致无法连接
throw new Exception();
}
}
这几天在解决一个用mina开发的高并发通信过程中产生的一个bug。
模拟场景为:
通过定时触发启动线程模拟高并发短连接测试,测试的服务端有2个,一个是服务有起,一个没起,客户端和服务端均在同一服务器上。执行一段时间后linux主机上通过lsof命令查看,发现有递增的文件句柄,pipe和eventpoll。
抛出的异常如下:
Failed to create a new instance of org.apache.mina.transport.socket.nio.NioProcessor: null
java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedConstructorAccessor110.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27 )
at java.lang.reflect.Constructor.newInstance(Constructor.java:513 )
at org.apache.mina.core.service.SimpleIoProcessorPool.<init>(SimpleIoProcessorPool.java:180 )
at org.apache.mina.core.service.SimpleIoProcessorPool.<init>(SimpleIoProcessorPool.java:112 )
at org.apache.mina.core.polling.AbstractPollingIoConnector.<init>(AbstractPollingIoConnector.java:93 )
at org.apache.mina.transport.socket.nio.NioSocketConnector.<init>(NioSocketConnector.java:56 )
at com.develop.webplatform.funnel.client.JobClient.sendMessage(JobClient.java:39 )
at com.develop.webplatform.funnel.client.JobClient.sendJob(JobClient.java:126 )
at com.develop.webplatform.funnel.extend.JobExecRemotelyBySocket.execJobByTask(JobExecRemotelyBySocket.java:66 )
at com.develop.webplatform.funnel.JobManager.execJobByTask(JobManager.java:27 )
at com.develop.webplatform.quartz.job.TaskJob.executeInternal(TaskJob.java:38 )
at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86 )
at org.quartz.core.JobRunShell.run(JobRunShell.java:223 )
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549 )
Caused by: org.apache.mina.core.RuntimeIoException: Failed to open a selector.
at org.apache.mina.transport.socket.nio.NioProcessor.<init>(NioProcessor.java:61 )
... 15 more
Caused by: java.io.IOException: Too many open files
at sun.nio.ch.IOUtil.initPipe(Native Method)
at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:49 )
at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18 )
at java.nio.channels.Selector.open(Selector.java:209 )
at org.apache.mina.transport.socket.nio.NioProcessor.<init>(NioProcessor.java:59 )
... 15 more
原代码中,关于客户端连接的代码如下:
final NioSocketConnector connector = new NioSocketConnector();
final String[] result = new String[ 1 ];
connector.getFilterChain().addLast("codec" ,
new ProtocolCodecFilter( new ObjectSerializationCodecFactory()));
connector.setHandler(handler);
//设置超时
connector.setConnectTimeoutMillis(defaultConnectTimeOut);
ConnectFuture connectFuture = connector.connect(address);
connectFuture.awaitUninterruptibly(); //同步,等待,直到连接完成
if (connectFuture.isDone()) {
if (!connectFuture.isConnected()) { //若在指定时间内没连接成功,则抛出异常
logger.info("fail to connect " + logInfo);
throw new Exception();
}
}
经过分析,导致主机文件句柄泄露的原因为,客户端发起服务端连接时,会请求系统分配相关的文件句柄,在原代码中,仅仅判断是否连接成功,而未对连接失败进 行资源释放,从而造成文件句柄泄露。当总的文件句柄数超过系统设置值(ulimit -n 查看同一个进程允许的最大文件句柄数),则抛出异常“java.io.IOException:
Too many open files”,导致无法创建新的连接,服务器挂掉。
更改后的代码如下:
final NioSocketConnector connector = new NioSocketConnector();
final String[] result = new String[ 1 ];
connector.getFilterChain().addLast("codec" ,
new ProtocolCodecFilter( new ObjectSerializationCodecFactory()));
connector.setHandler(handler);
//设置超时
connector.setConnectTimeoutMillis(defaultConnectTimeOut);
ConnectFuture connectFuture = connector.connect(address);
connectFuture.awaitUninterruptibly(); //同步,等待,直到连接完成
if (connectFuture.isDone()) {
if (!connectFuture.isConnected()) { //若在指定时间内没连接成功,则抛出异常
logger.info("fail to connect " + logInfo);
connector.dispose(); //不关闭的话会运行一段时间后抛出,too many open files异常,导致无法连接
throw new Exception();
}
}
相关文章推荐
- java集合框架系列-TreeMap介绍
- Java项目导入外部的JAR包
- java 基础之枚举
- Java加密与解密的艺术-对称加密算法
- SpringBoot事务注解实现原理
- Java IO流详解
- Spring学习(六)spring整合注解形式的hibernate
- SpringBoot自定义参数注解
- Spring事务配置的五种方式
- java程序模拟QQ空间登录 - 并模拟刷说说的赞
- leetcode--LinkedListCycleII
- Spring和MyBatis环境整合
- ANT构建JAVA WEB工程
- [分布式java]基于JavaAPI实现消息方式的系统间通信:UDP/IP+BIO
- Spring4 多种定时器详解
- 【java】国际化
- 自定义Spring动态代理类型
- Mac 终端命令运行java程序
- Java用邻接矩阵实现图并进行深度优先搜索
- java动态代理--cglib