nginx + tomcat 集群实现负载均衡

1、安装nginx

所需的prce库

wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.10.tar.gz



tar zxvf pcre-8.10.tar.gz
cd pcre-8.10/

./configure

make && make install

cd ../

安装nginx

wget http://nginx.org/download/nginx-0.8.54.tar.gz
wget http://nginx.org/download/nginx-1.0.2.tar.gz

tar zxvf nginx-1.0.2.tar.gz

cd nginx-1.0.2/

./configure --user=www --group=www --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module

make && make install

cd ../

创建Nginx日志目录

mkdir -p /home/nginx/logs

chmod +w /home/nginx/logs

chown -R www:www /home/nginx/logs

创建Nginx配置文件

①、在/usr/local/nginx/conf/目录中创建nginx.conf文件：

rm -f /usr/local/nginx/conf/nginx.conf

vi /usr/local/nginx/conf/nginx.conf

输入以下内容：

user www www;
worker_processes 8;
error_log /home/nginx/logs/nginx_error.log crit;
pid /usr/local/nginx/nginx.pid;
worker_rlimit_nofile 65535;

events
{
use epoll;
worker_connections 65535;
}

http
{
include mime.types;
default_type application/octet-stream;
#charset gb2312;

server_names_hash_bucket_size 128;
client_header_buffer_size 32k;
large_client_header_buffers 4 32k;
client_max_body_size 8m;
sendfile on;
tcp_nopush on;

keepalive_timeout 60;

tcp_nodelay on;

fastcgi_connect_timeout 300;
fastcgi_send_timeout 300;
fastcgi_read_timeout 300;
fastcgi_buffer_size 64k;
fastcgi_buffers 4 64k;
fastcgi_busy_buffers_size 128k;
fastcgi_temp_file_write_size 128k;

gzip on;
gzip_min_length 1k;
gzip_buffers 4 16k;
gzip_http_version 1.0;
gzip_comp_level 2;
gzip_types text/plain application/x-javascript text/css application/xml;
gzip_vary on;

#limit_zone crawler$binary_remote_addr10m;

#测试
server
{
listen 80;
server_name www.test.com;
index index.htm index.html index.jsp;
root /home/htdocs/web/ROOT/;

location ~ .*$
{
index index.jsp;
proxy_pass http://www.test.com:8080;
}
access_log logs/sp.imichat.com.log combined;
error_page 404 = /404.html;
}

}

　②、在/usr/local/nginx/conf/目录中创建fcgi.conf文件：

vi /usr/local/nginx/conf/fcgi.conf

输入以下内容：

fastcgi_param GATEWAY_INTERFACE CGI/1.1;

fastcgi_param SERVER_SOFTWARE nginx;

fastcgi_param QUERY_STRING $query_string;

fastcgi_param REQUEST_METHOD $request_method;

fastcgi_param CONTENT_TYPE $content_type;

fastcgi_param CONTENT_LENGTH $content_length;

fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

fastcgi_param SCRIPT_NAME $fastcgi_script_name;

fastcgi_param REQUEST_URI $request_uri;

fastcgi_param DOCUMENT_URI $document_uri;

fastcgi_param DOCUMENT_ROOT $document_root;

fastcgi_param SERVER_PROTOCOL $server_protocol;

fastcgi_param REMOTE_ADDR $remote_addr;

fastcgi_param REMOTE_PORT $remote_port;

fastcgi_param SERVER_ADDR $server_addr;

fastcgi_param SERVER_PORT $server_port;

fastcgi_param SERVER_NAME $server_name;

# PHP only, required if PHP was built with --enable-force-cgi-redirect

fastcgi_param REDIRECT_STATUS 200;

启动Nginx

ulimit -SHn 65535

/usr/local/nginx/sbin/nginx

配置开机自动启动Nginx

vi /etc/rc.local

在末尾增加以下内容：

ulimit -SHn 65535

/usr/local/nginx/sbin/nginx

优化Linux内核参数

vi /etc/sysctl.conf

在末尾增加以下内容：

# Add

net.ipv4.tcp_max_syn_backlog = 65536

net.core.netdev_max_backlog = 32768

net.core.somaxconn = 32768

net.core.wmem_default = 8388608

net.core.rmem_default = 8388608

net.core.rmem_max = 16777216

net.core.wmem_max = 16777216

net.ipv4.tcp_timestamps = 0

net.ipv4.tcp_synack_retries = 2

net.ipv4.tcp_syn_retries = 2

net.ipv4.tcp_tw_recycle = 1

#net.ipv4.tcp_tw_len = 1

net.ipv4.tcp_tw_reuse = 1

net.ipv4.tcp_mem = 94500000 915000000 927000000

net.ipv4.tcp_max_orphans = 3276800

#net.ipv4.tcp_fin_timeout = 30

#net.ipv4.tcp_keepalive_time = 120

net.ipv4.ip_local_port_range = 1024 65535

使配置立即生效：

/sbin/sysctl -p

在不停止Nginx服务的情况下变更Nginx配置

修改/usr/local/nginx/conf/nginx.conf配置文件后，请执行以下命令检查配置文件是否正确：

/usr/local/nginx/sbin/nginx -t

如果屏幕显示以下两行信息，说明配置文件正确：

　　the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok

　　the configuration file /usr/local/nginx/conf/nginx.conf was tested successfully

重启nginx：

/usr/local/nginx/sbin/nginx -s reload

编写每天定时切割Nginx日志的脚本

创建脚本/usr/local/nginx/sbin/cut_nginx_log.sh

vi /usr/local/nginx/sbin/cut_nginx_log.sh

输入以下内容：

#!/bin/bash

# This script run at 00:00

# The Nginx logs path

logs_root_path="/home/nginx/logs/"

logs_path=${logs_root_path}$(date -d "yesterday" +"%Y")/$(date -d "yesterday" +"%m")/

mkdir -p ${logs_path}

#日志文件名 ====================================================

logs_name="weblogs"

logs_file=${logs_root_path}${logs_name}.log

cut_logs_file=${logs_path}${logs_name}_$(date -d "yesterday" +"%Y%m%d").log

mv ${logs_file} ${cut_logs_file}

tar czf ${cut_logs_file}.tar.gz ${cut_logs_file}

rm -f ${cut_logs_file}

#重启nginx

/usr/local/nginx/sbin/nginx -s reload

对这个脚本赋执行权限

# chmod +x /usr/local/nginx/sbin/cut_nginx_log.sh

设置crontab，每天凌晨00:00切割nginx访问日志

crontab -e

输入以下内容：

00 00 * * * /bin/bash /usr/local/nginx/sbin/cut_nginx_log.sh

2、配置nginx负载均衡

编辑nginx.conf

vim /usr/local/nginx/conf/nginx.conf

在server上面增加

upstream www.test.com {

server www.test.com:8888;

server www.test.com:8080;

#ip_hash; 注：如果加入了ip_hash， 这种只能适用于当某个用户连接上了一台服务器后，他登陆之后所做的一切操作都只会在那一台服务器不会跳转到另外的服务器，如果那台服务器over掉了，则会自动退出，连接到另一服务器

}

www.test.com 是自己的服务器，最好是用域名

upstream后面的名字要与proxy_pass 这个名字一样，建议upstream后面的名字，proxy_pass 后面的名字，server_name 后面的名字一致

示例：

upstream www.test.com {
server www.test.com:8888;
server www.test.com:8080;
}

#测试
server
{
listen 80;
server_name www.test.com;
index index.htm index.html index.jsp;
root /home/htdocs/web/ROOT/;

location ~ .*$
{
index index.jsp;
proxy_pass http://www.test.com; 注：做了负载均衡，这个后面就不能再加端口
}
access_log logs/sp.imichat.com.log combined;
error_page 404 = /404.html;
}

注：如果负载均衡不成功，请检查是否没有经过nginx，80端口直接被转发到了tomcat服务的端口上

3、配置tomcat集群

优化tomcat 最大并发数，编辑server.xml

<Connector
port="8080" maxHttpHeaderSize="8192"

maxThreads="2048" minSpareThreads="100" maxSpareThreads="200"

enableLookups="false" redirectPort="8443" acceptCount="500"

connectionTimeout="20000" disableUploadTimeout="true" />

<Engine name="Catalina" defaultHost="localhost"
jvmRoute="worker1">

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" tcpListenAddress="127.0.0.1" />

修改web.xml

增加标签：<distributable/>

直接加在</web-app> 之前就可以了，这个是加入tomcat 的session 复制的，做tomcat 集群必须需要这一步，否则用户的session 就无法正常使用.