Http连接池PoolingHttpClientConnectionManager的应用
2020-07-12 16:55
453 查看
前言
Http连接需要的三次握手开销很大, 传统的HttpURLConnection并不支持连接池, HTTP1.1以上默认开启keepalive, 对于现在springcloud横行的社会, feign可以配置好http连接池, 不过总会有某些个别的接口不在服务中, 还有一些非springboot的老旧项目也要加入cloud大家族中, 对于内部频繁访问的url地址, 这就需要一款量身定做的工具类了.
讲解
先来看看测试效果, 两种工具类的时间消耗对比.
这个是测试用的方法, 前后分别调用了某一接口100次, 进行时间统计, 系统页面按F12能看到平均握手速度大概是3ms
long start = System.currentTimeMillis(); for (int i = 0; i < 100; i++) { result = commonManager.getPolicyGoNew(inputStr); } long end = System.currentTimeMillis(); System.out.println(String.valueOf(end - start)); System.out.println("***********"); long start2 = System.currentTimeMillis(); for (int i = 0; i < 100; i++) { result = commonManager.getPolicyGo(inputStr); } long end2 = System.currentTimeMillis(); System.out.println(String.valueOf(end2 - start2));100次快了647ms, 平均一次6ms, 握三次手大概要这么久吧
9612 *********** 10259第二次我调换了顺序, 500.for, 500次快了2341ms, 平均一次4.6ms, 更接近3ms了
50906 *********** 48565 52327 *********** 47021错误的请求数据, 可以立马返回数据, 更能说明是否节省了握手时间
9625 *********** 8868 9251 *********** 8718
不过我之前都是调用的生产环境的接口, 效率还是比较高的, 下面我在uat环境测试一下
正常查询详情数据消耗时间500次快了4900ms, 平均一次10ms
72712 *********** 67839错误的请求数据, 可以立马返回数据, 更能说明是否节省了握手时间
500次快了2400ms, 平均一次4.8ms
6439 *********** 4063
代码
原httputil, 看实现部分就好
import org.apache.http.HttpStatus; import org.apache.http.client.config.RequestConfig; import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpPost; import org.apache.http.client.methods.HttpUriRequest; import org.apache.http.client.utils.HttpClientUtils; import org.apache.http.entity.ByteArrayEntity; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClientBuilder; import java.util.Map; public class HttpUtils { public interface ResponseCallback { void onResponse(CloseableHttpResponse response) throws Exception; void onError(int resultCode, String resultStr) throws Exception; } public static void headerPost(String urlStr, String data, ResponseCallback callback, Map<String, String> headerMap) throws Exception { doPost(urlStr, data, callback, headerMap, 300000); } private static void doPost(String urlStr, String data, ResponseCallback callback, Map<String, String> headerMap, int timeOut) throws Exception { HttpUriRequest request = null; CloseableHttpClient client = null; CloseableHttpResponse response = null; try { request = new HttpPost(urlStr); if (headerMap != null) { for(Map.Entry<String, String> entry : headerMap.entrySet()){ request.setHeader(entry.getKey(), entry.getValue()); } } if (null != data && !"".equals(data)) { ((HttpPost) request).setEntity(new ByteArrayEntity(data.getBytes("utf-8"))); } RequestConfig config = RequestConfig.custom().setConnectionRequestTimeout(timeOut).setConnectTimeout(timeOut).setSocketTimeout(timeOut).build(); client = HttpClientBuilder.create().setDefaultRequestConfig(config).build(); response = client.execute(request); int resultCode = response.getStatusLine().getStatusCode(); if (resultCode == HttpStatus.SC_OK) { callback.onResponse(response); } else { throw new Exception(resultCode + ":system error"); } } catch (Exception e) { callback.onError(HttpStatus.SC_INTERNAL_SERVER_ERROR, e.toString()); } finally { if (request != null && !request.isAborted()) { request.abort(); request = null; } if (client != null) { HttpClientUtils.closeQuietly(client); client = null; } if (response != null) { HttpClientUtils.closeQuietly(response); response = null; } } } }
带连接池的httputil, 初版支持一个url连接, 可更改为Map维护
import org.apache.http.*; import org.apache.http.client.HttpRequestRetryHandler; import org.apache.http.client.config.RequestConfig; import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpGet; import org.apache.http.client.methods.HttpPost; import org.apache.http.client.methods.HttpRequestBase; import org.apache.http.client.protocol.HttpClientContext; import org.apache.http.config.Registry; import org.apache.http.config.RegistryBuilder; import org.apache.http.conn.ConnectTimeoutException; import org.apache.http.conn.routing.HttpRoute; import org.apache.http.conn.socket.ConnectionSocketFactory; import org.apache.http.conn.socket.LayeredConnectionSocketFactory; import org.apache.http.conn.socket.PlainConnectionSocketFactory; import org.apache.http.conn.ssl.SSLConnectionSocketFactory; import org.apache.http.entity.ByteArrayEntity; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClients; import org.apache.http.impl.conn.PoolingHttpClientConnectionManager; import org.apache.http.protocol.HttpContext; import org.apache.http.util.EntityUtils; import javax.net.ssl.SSLException; import javax.net.ssl.SSLHandshakeException; import java.io.IOException; import java.io.InterruptedIOException; import java.net.UnknownHostException; import java.util.Map; public class HttpClientUtil { static final int timeOut = 10 * 1000; private static CloseableHttpClient httpClient = null; private final static Object syncLock = new Object(); private static void config(HttpRequestBase httpRequestBase) { // 配置请求的超时设置 RequestConfig requestConfig = RequestConfig.custom() .setConnectionRequestTimeout(timeOut) .setConnectTimeout(timeOut).setSocketTimeout(timeOut).build(); httpRequestBase.setConfig(requestConfig); } /** * 获取HttpClient对象 */ public static CloseableHttpClient getHttpClient(String url) { String hostname = url.split("/")[2]; int port = 80; if (hostname.contains(":")) { String[] arr = hostname.split(":"); hostname = arr[0]; port = Integer.parseInt(arr[1]); } if (httpClient == null) { synchronized (syncLock) { if (httpClient == null) { httpClient = createHttpClient(200, 40, 100, hostname, port); } } } return httpClient; } public static CloseableHttpClient createHttpClient(int maxTotal, int maxPerRoute, int maxRoute, String hostname, int port) { ConnectionSocketFactory plainsf = PlainConnectionSocketFactory.getSocketFactory(); LayeredConnectionSocketFactory sslsf = SSLConnectionSocketFactory.getSocketFactory(); Registry<ConnectionSocketFactory> registry = RegistryBuilder .<ConnectionSocketFactory> create().register("http", plainsf).register("https", sslsf).build(); PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(registry); // 将最大连接数增加 cm.setMaxTotal(maxTotal); // 将每个路由基础的连接增加 cm.setDefaultMaxPerRoute(maxPerRoute); HttpHost httpHost = new HttpHost(hostname, port); // 将目标主机的最大连接数增加 cm.setMaxPerRoute(new HttpRoute(httpHost), maxRoute); // 请求重试处理 HttpRequestRetryHandler httpRequestRetryHandler = new HttpRequestRetryHandler() { public boolean retryRequest(IOException exception, int executionCount, HttpContext context) { if (executionCount >= 5) {// 如果已经重试了5次,就放弃 return false; } if (exception instanceof NoHttpResponseException) {// 如果服务器丢掉了连接,那么就重试 return true; } if (exception instanceof SSLHandshakeException) {// 不要重试SSL握手异常 return false; } if (exception instanceof InterruptedIOException) {// 超时 return false; } if (exception instanceof UnknownHostException) {// 目标服务器不可达 return false; } if (exception instanceof ConnectTimeoutException) {// 连接被拒绝 return false; } if (exception instanceof SSLException) {// SSL握手异常 return false; } HttpClientContext clientContext = HttpClientContext.adapt(context); HttpRequest request = clientContext.getRequest(); // 如果请求是幂等的,就再次尝试 if (!(request instanceof HttpEntityEnclosingRequest)) { return true; } return false; } }; CloseableHttpClient httpClient = HttpClients.custom() .setConnectionManager(cm) .setRetryHandler(httpRequestRetryHandler).build(); return httpClient; } public static String post(String url, String data, Map<String, String> headerMap) throws IOException { HttpPost httppost = new HttpPost(url); config(httppost); if (headerMap != null) { for(Map.Entry<String, String> entry : headerMap.entrySet()){ httppost.setHeader(entry.getKey(), entry.getValue()); } } if (null != data && !"".equals(data)) { httppost.setEntity(new ByteArrayEntity(data.getBytes("utf-8"))); } CloseableHttpResponse response = null; try { response = getHttpClient(url).execute(httppost, HttpClientContext.create()); HttpEntity entity = response.getEntity(); String result = EntityUtils.toString(entity, "utf-8"); //EntityUtils.consume(entity); return result; } catch (Exception e) { throw e; } finally { try { if (response != null) response.close(); } catch (IOException e) { e.printStackTrace(); } } } public static String get(String url) { HttpGet httpget = new HttpGet(url); config(httpget); CloseableHttpResponse response = null; try { response = getHttpClient(url).execute(httpget, HttpClientContext.create()); HttpEntity entity = response.getEntity(); String result = EntityUtils.toString(entity, "utf-8"); EntityUtils.consume(entity); return result; } catch (IOException e) { e.printStackTrace(); } finally { try { if (response != null) response.close(); } catch (IOException e) { e.printStackTrace(); } } return null; } }
总结
不管怎么来回测试, 都有时间上的差别, 连接池明显要快一些, 而且对服务器的资源消耗也小, 对接口提供者也比较友好, 当然这一切有个前提, 就是双方都是http1.1以上, 默认开启keepAlive, 毕竟http2.0协议还没有太普及.
相关文章推荐
- Http请求连接池 - HttpClient 的 PoolingHttpClientConnectionManager
- PoolingHttpClientConnectionManager连接池探究
- Http请求连接池 - HttpClient 的 PoolingHttpClientConnectionManager
- Http请求连接池 - HttpClient 的 PoolingHttpClientConnectionManager
- Http请求连接池 - HttpClient 的 PoolingHttpClientConnectionManager
- Http请求连接池 - HttpClient 的 PoolingHttpClientConnectionManager
- Http请求连接池 - HttpClient 的 PoolingHttpClientConnectionManager
- Http请求连接池 - HttpClient 的 PoolingHttpClientConnectionManager
- Http请求连接池 - HttpClient 的 PoolingHttpClientConnectionManager
- 使用HttpClient的PoolingHttpClientConnectionManager实现Http请求连接池
- org.apache.http.impl.conn.PoolingClientConnectionManager
- httpclient源码分析之 PoolingHttpClientConnectionManager 获取连接
- 使用PoolingHttpClientConnectionManager解决友盟(umeng)推送在多线程环境推送失败的问题
- 使用HttpClient4.5 PoolingHttpClientConnectionManager 提交Http请求
- poolingHttpclientConnectionmanager 使用
- httpclient线程池,poolinghttpclientconnectionmanager
- HttpClient PoolingClientConnectionManager 参数含义
- HttpClient连接池抛出大量ConnectionPoolTimeoutException: Timeout waiting for connection异常排查
- HttpClient连接池抛出大量ConnectionPoolTimeoutException: Timeout waiting for connection异常排查...
- HttpClient异步,及连接池应用