【异常】elasticsearch的RestClientAPI请求超时问题:java.lang.RuntimeException: error while performing request
2017-12-19 10:00
2316 查看
问题描述:
在Spark中使用elasticsearch的RestClient API在读取es中数据时,出现TimeoutException,在之前的优化的中做下面的超时设置,这个只能解决request retries exceeded max retry timeout [30000]这种超时:
RestClient build = RestClient.builder(httpHosts).setMaxRetryTimeoutMillis(5*60*1000).build();
但是在后面的作业启动中还是出现了:java.lang.RuntimeException: error while performing request
java.lang.RuntimeException: error while performing request
at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:681)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:219)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:191)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:396)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:382)
at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:323)
at com.bbd.finance.commons.utils.ESSparkUtils.readBySingleField(ESSparkUtils.java:410)
at com.bbd.finance.fc.functions.stream.StreamFuncUtils.lambda$null$4(StreamFuncUtils.java:366)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.TimeoutException
at org.apache.http.nio.pool.AbstractNIOConnPool.processPendingRequest(AbstractNIOConnPool.java:364)
at org.apache.http.nio.pool.AbstractNIOConnPool.processNextPendingRequest(AbstractNIOConnPool.java:344)
at org.apache.http.nio.pool.AbstractNIOConnPool.release(AbstractNIOConnPool.java:318)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.releaseConnection(PoolingNHttpClientConnectionManager.java:364)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.releaseConnection(AbstractClientExchangeHandler.java:245)
at org.apache.http.impl.nio.client.MainClientExec.responseCompleted(MainClientExec.java:387)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:172)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326)
at org.apache.http.impl.nio.client.InternalRequestExecutor.inputReady(InternalRequestExecutor.java:83)
at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
... 1 more并且这种异常是不定时出现,最后查看了RestClient的初始化(RestClientBuilder)源码发现有几个连接超时和Socket超时设置的非常小,如下:
public static final int DEFAULT_CONNECT_TIMEOUT_MILLIS = 1000;
public static final int DEFAULT_SOCKET_TIMEOUT_MILLIS = 30000;
public static final int DEFAULT_MAX_RETRY_TIMEOUT_MILLIS = DEFAULT_SOCKET_TIMEOUT_MILLIS;
public static final int DEFAULT_CONNECTION_REQUEST_TIMEOUT_MILLIS = 500;
所以我们将之前的初始化RestClient的代码做了如下的修改:
RestClientBuilder builder = RestClient.builder(httpHosts).setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
@Override
public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {
requestConfigBuilder.setConnectTimeout(5000);
requestConfigBuilder.setSocketTimeout(40000);
requestConfigBuilder.setConnectionRequestTimeout(1000);
return requestConfigBuilder;
}
}).setMaxRetryTimeoutMillis(5*60*1000);
RestHighLevelClient client = new RestHighLevelClient(builder.build());
任务工作正常!
在Spark中使用elasticsearch的RestClient API在读取es中数据时,出现TimeoutException,在之前的优化的中做下面的超时设置,这个只能解决request retries exceeded max retry timeout [30000]这种超时:
RestClient build = RestClient.builder(httpHosts).setMaxRetryTimeoutMillis(5*60*1000).build();
但是在后面的作业启动中还是出现了:java.lang.RuntimeException: error while performing request
java.lang.RuntimeException: error while performing request
at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:681)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:219)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:191)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:396)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:382)
at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:323)
at com.bbd.finance.commons.utils.ESSparkUtils.readBySingleField(ESSparkUtils.java:410)
at com.bbd.finance.fc.functions.stream.StreamFuncUtils.lambda$null$4(StreamFuncUtils.java:366)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.TimeoutException
at org.apache.http.nio.pool.AbstractNIOConnPool.processPendingRequest(AbstractNIOConnPool.java:364)
at org.apache.http.nio.pool.AbstractNIOConnPool.processNextPendingRequest(AbstractNIOConnPool.java:344)
at org.apache.http.nio.pool.AbstractNIOConnPool.release(AbstractNIOConnPool.java:318)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.releaseConnection(PoolingNHttpClientConnectionManager.java:364)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.releaseConnection(AbstractClientExchangeHandler.java:245)
at org.apache.http.impl.nio.client.MainClientExec.responseCompleted(MainClientExec.java:387)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:172)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326)
at org.apache.http.impl.nio.client.InternalRequestExecutor.inputReady(InternalRequestExecutor.java:83)
at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
... 1 more并且这种异常是不定时出现,最后查看了RestClient的初始化(RestClientBuilder)源码发现有几个连接超时和Socket超时设置的非常小,如下:
public static final int DEFAULT_CONNECT_TIMEOUT_MILLIS = 1000;
public static final int DEFAULT_SOCKET_TIMEOUT_MILLIS = 30000;
public static final int DEFAULT_MAX_RETRY_TIMEOUT_MILLIS = DEFAULT_SOCKET_TIMEOUT_MILLIS;
public static final int DEFAULT_CONNECTION_REQUEST_TIMEOUT_MILLIS = 500;
所以我们将之前的初始化RestClient的代码做了如下的修改:
RestClientBuilder builder = RestClient.builder(httpHosts).setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
@Override
public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {
requestConfigBuilder.setConnectTimeout(5000);
requestConfigBuilder.setSocketTimeout(40000);
requestConfigBuilder.setConnectionRequestTimeout(1000);
return requestConfigBuilder;
}
}).setMaxRetryTimeoutMillis(5*60*1000);
RestHighLevelClient client = new RestHighLevelClient(builder.build());
任务工作正常!
相关文章推荐
- FAILED: Hive Internal Error: java.lang.RuntimeException(Error while making MR scratch directory异常的解决
- RxVolley报错:Caused by: java.lang.RuntimeException: RequestQueue-> DiskBasedCache cache dir error
- hive 报错FAILED: Error in metadata: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient FAILED: Execu
- JMeter命令行执行及问题解决 Error in NonGUIDriver java.lang.RuntimeException: Could not find the TestPlan class!
- 解决Redisson出现Failed to instantiate [org.redisson.api.RedissonClient]: Factory method 'create' threw exception; nested exception is java.lang.ArrayIndexOutOfBoundsException: 0的问题
- 错误:java.lang.RuntimeException: An error occured while executing doInBackground();
- 异常-----springmvc + ajaxfileupload解决ajax不能异步上传图片的问题。java.lang.ClassCastException: org.apache.catalina.connector.RequestFacade cannot be cast to org.springframework.web.multipart.
- Storm 执行异常 java.lang.RuntimeException: java.nio.channels.UnresolvedAddressException 问题解决
- 解决Eclipse启动JBoss7.1异常的问题(Exception in thread "main" java.lang.ExceptionInInitializerError)
- RxVolley报错:Caused by: java.lang.RuntimeException: RequestQueue-> DiskBasedCache cache dir error
- resin WED服务器初用遇到的问题和解决方法 java.lang.RuntimeException: java.net.SocketException: Unrecognized Windows Socke ts error: 0: JVM_Bind
- 【解决问题】failed: java.lang.RuntimeException: org.openqa.selenium.WebDriverException: Unexpected error launching Internet Explorer.
- Android Studio 图片问题Error:java.lang.RuntimeException: Some file crunching failed, see logs for detail
- FAILED: Hive Internal Error: java.lang.RuntimeException(Error while making MR scratch directory
- Hive出现异常 FAILED: Error In Metadata: Java.Lang.RuntimeException: Unable To Instantiate Org.Apache.Had
- 第三方接入支付宝问题:nested exception is java.lang.NoClassDefFoundError: com/alipay/api/AlipayRequest
- Android 问题异常 java.lang.RuntimeException: Unable to instantiate activity ComponentInfo
- Exception -- java.lang.OutOfMemoryError: PermGen space” -- java内存溢出异常
- java.lang.RuntimeException异常的解决
- 项目启动异常:Exception: java.lang.RuntimeException: XPathFactory#newInstance()