Load Balance Tomcat with Nginx and Store Sessions in Redis
2015-04-24 16:32
417 查看
An awkward title, but that’s exactly what we’re going to do. For some time, I was looking for a way to push code to production systems with zero downtime and zero impact to any active users. Surprisingly, the solution took very little time to implement. At
a high level, we have Nginx load balancing two instances of Tomcat. Tomcat stores it’s sessions in Redis. Nginx is configured as non-sticky, since a request can go to any node in the cluster. When we need to push new code, simply take down any Tomcat instance.
All current users will now get routed to the active instance. Since session data is externalized in Redis, active users will not be impacted. Once the inactive instance has been updated, bring it up and repeat for the other node.
We’ll start with Nginx:
[raoul@raoul-wp ~]$ sudo rpm -ivh nginx-1.4.2-1.el6.ngx.x86_64.rpm
Edit /etc/nginx/nginx.conf and add the bolded text below
view
source
print?
Update /etc/nginx/conf.d/default.conf and replace the location section with this:
view
source
print?
Restart nginx:
[raoul@raoul-wp nginx]$ sudo service nginx restart
Next, install two instances of Tomcat. Change the server ports of the second instance, so that they do not conflict. At this point if you enter https://localhost in
your browser, you will be taken to the default tomcat page. However, since we have not setup sticky sessions, every request will get load balanced in round robin, which effectively means it will be creating a new session per request. You can easily see this
behavior using the built in tomcat examples. Navigate tohttp://localhost/examples/servlets/servlet/SessionExample and
refresh this page a few times and notice the Session ID changing each time. Let us fix this.
Download and install Redis. There is good documentation at http://redis.io/download so
I’m not going into the details. Start the server and use the client program to check that it’s working.
Finally, we need to configure Tomcat to store it’s sessions in Redis. For this we’ll be using tomcat-redis-session-manager (https://github.com/jcoleman/tomcat-redis-session-manager).
This did not work out-of-the-box and required some tweaking. You will need to download the source code of this project and re-build it after updating the dependent library versions. The versions I used are commons-pool2-2.2.jar and jedis-2.6.1.jar. Copy these
jars to the lib directory of both the tomcat instances.
Update the versions of commons-pool, jedis and the tomcat version that you are using in build.gradle of tomcat-redis-session-manager and build the project. Then copy the built tomcat-redis-session-manager-1.2.jar to tomcat lib directory of each instance. Add
the following to both the tomcat’s context.xml:
view
source
print?
Restart the tomcat instances and we’re done. You can now see tomcat’s session in Redis. Use the previous example and try various combinations by taking the tomcat instances up and down. The session data will remain unaffected. I even noticed that if you take
down both the instances and then bring them back up, the user’s existing session will be restored.
Thank you for your time.
a high level, we have Nginx load balancing two instances of Tomcat. Tomcat stores it’s sessions in Redis. Nginx is configured as non-sticky, since a request can go to any node in the cluster. When we need to push new code, simply take down any Tomcat instance.
All current users will now get routed to the active instance. Since session data is externalized in Redis, active users will not be impacted. Once the inactive instance has been updated, bring it up and repeat for the other node.
We’ll start with Nginx:
[raoul@raoul-wp ~]$ sudo rpm -ivh nginx-1.4.2-1.el6.ngx.x86_64.rpm
Edit /etc/nginx/nginx.conf and add the bolded text below
view
source
print?
1.
http {
2.
upstream tomcat {
3.
server localhost:8080;
4.
server localhost:8081;
5.
}
6.
include /etc/nginx/mime.types;
7.
default_type application/octet-stream;
Update /etc/nginx/conf.d/default.conf and replace the location section with this:
view
source
print?
1.
location / {
2.
proxy_pass http://tomcat;
3.
}
Restart nginx:
[raoul@raoul-wp nginx]$ sudo service nginx restart
Next, install two instances of Tomcat. Change the server ports of the second instance, so that they do not conflict. At this point if you enter https://localhost in
your browser, you will be taken to the default tomcat page. However, since we have not setup sticky sessions, every request will get load balanced in round robin, which effectively means it will be creating a new session per request. You can easily see this
behavior using the built in tomcat examples. Navigate tohttp://localhost/examples/servlets/servlet/SessionExample and
refresh this page a few times and notice the Session ID changing each time. Let us fix this.
Download and install Redis. There is good documentation at http://redis.io/download so
I’m not going into the details. Start the server and use the client program to check that it’s working.
Finally, we need to configure Tomcat to store it’s sessions in Redis. For this we’ll be using tomcat-redis-session-manager (https://github.com/jcoleman/tomcat-redis-session-manager).
This did not work out-of-the-box and required some tweaking. You will need to download the source code of this project and re-build it after updating the dependent library versions. The versions I used are commons-pool2-2.2.jar and jedis-2.6.1.jar. Copy these
jars to the lib directory of both the tomcat instances.
Update the versions of commons-pool, jedis and the tomcat version that you are using in build.gradle of tomcat-redis-session-manager and build the project. Then copy the built tomcat-redis-session-manager-1.2.jar to tomcat lib directory of each instance. Add
the following to both the tomcat’s context.xml:
view
source
print?
1.
<
Valve
className
=
"com.orangefunction.tomcat.redissessions.RedisSessionHandlerValve"
/>
2.
<
Manager
className
=
"com.orangefunction.tomcat.redissessions.RedisSessionManager"
3.
host
=
"localhost"
4.
port
=
"6379"
5.
database
=
"0"
6.
maxInactiveInterval
=
"60"
/>
Restart the tomcat instances and we’re done. You can now see tomcat’s session in Redis. Use the previous example and try various combinations by taking the tomcat instances up and down. The session data will remain unaffected. I even noticed that if you take
down both the instances and then bring them back up, the user’s existing session will be restored.
Thank you for your time.
相关文章推荐
- Load Balance Tomcat with Nginx and Store Sessions in Redis--reference
- Apache/Tomcat with Failover and Load Balancing in 20 minutes or less…
- load-balanceing tomcat with apache use by mod-proxy-ajp in SuSE Linux
- Tomcat load balance and clustering - tomcat负载均衡和集群
- Install Nginx with PHP5 and MySQL (LEMP) in Ubuntu 13.10 Server
- (转贴)How to use JDBC and transactions in Tomcat with JOTM
- A survey of task allocation and load balance in distributed system阅读笔记
- Dynamic nginx upstreams with Lua and Redis
- apache + tomcat load balance and cluster
- lab-bill-sys_NginxTomcatLoadbalance
- Building Applications with Force.com and VisualForce (DEV401) (二四):JavaScript in Visualforce
- Sams Teach Yourself Web Publishing with HTML and CSS in One Hour a Day (5th Edition)
- one command to delete all files with name "filename" in current dir and all its sub-dirs
- lvs之 lvs+nginx+tomcat_1、tomcat_2+redis(lvs dr 模式)
- ubuntu nginx+redis+tomcat集群配置
- [TypeScript] Using Lodash in TypeScript with Typings and SystemJS
- Upload files in ASP.NET MVC with JavaScript and C#
- Linux下搭建Nginx+Keepalived+Redis+Tomcat实现session共享 、负载均衡、高可用
- [Angular 2] Managing State in RxJS with StartWith and Scan
- how to add Javascript and CSS in page layout with sharepoint 2013