varnish安装配置笔记
2010-09-12 18:46
211 查看
首发于http://www.linux-ch.com/post-12.html
在这个小小的vps上,从一个小小的emlog变成了emlog+ucenter+dz在上面奔跑的场面,是多么的壮观呀~
但是在实际中发现并不是那么的乐观:内存太小,后台的处理能力大打折扣;后台处理能力不高,却全是动态网站;由于是动态网站,所以更无法同时承受更多用户的访问........
在 硬件不会升级的情况下,我尝试了给php装上eaccelerator和memcache,效果并不明显,或者是我没有配置好的原因,nginx在后台依 旧很频繁是调用php-fpm.在这个时候我发现了一些曙光:让数据更少地在后台的进程间传输,然后直接传输给客户端.后在网友殘剑的介绍下尝试了 varnish这个代理软件.
软件下载:wget http://www.varnish-software.com/sites/default/files/varnish-2.1.3.tar.gz
为软件创建用户和存储数据的目录
useradd -M -r -s /sbin/nologin varnish
mkdir /var/vcache
解压,编译,安装
tar -xvf varnish-2.1.3.tar.gz
cd varnish-2.1.3
./configure --prefix=/usr/local/varnish
make
make install
修改配置文件
vim /usr/local/varnish/etc/varnish/default.vcl
backend linuxch {
.host = "127.0.0.1";
.port = "8000";
.connect_timeout = 1s;
.first_byte_timeout = 5s;
.between_bytes_timeout = 2s;
}
acl purge {
"127.0.0.1";
}
sub vcl_recv {
set req.grace = 30s;
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For ", " client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
if (req.request != "GET" &&
req.request != "HEAD" &&
req.request != "PUT" &&
req.request != "POST" &&
req.request != "TRACE" &&
req.request != "OPTIONS" &&
req.request != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}
if (req.request != "GET" && req.request != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}
if (req.http.Authorization || req.http.Cookie) {
return (pass);
}
if (req.http.host ~ "^(.*).linux-ch.com") {
set req.backend = linuxch;
}
else {
error 404 "Unknown HostName!";
}
if (req.request == "PURGE") {
if (!client.ip ~ purge) {
error 405 "Not Allowed.";
return (lookup);
}
}
if (req.url ~ "\.(php)($|\?)") {
return (pass);
}
if (req.http.Cache-Control ~ "(no-cache|max-age=0)") {
purge_url(req.url);
}
return (lookup);
}
sub vcl_pipe {
return (pipe);
}
sub vcl_pass {
return (pass);
}
sub vcl_hash {
set req.hash += req.url;
if (req.http.host) {
set req.hash += req.http.host;
} else {
set req.hash += server.ip;
}
return (hash);
}
sub vcl_hit {
if (!obj.cacheable) {
return (pass);
}
if (req.request == "PURGE") {
set obj.ttl = 0s;
error 200 "Purged.";
}
return (deliver);
}
sub vcl_miss {
if (req.request == "PURGE") {
error 404 "Not in cache.";
}
return (fetch);
}
sub vcl_fetch {
if (!beresp.cacheable) {
return (pass);
}
if (!(req.url ~ "(admin|login)")) {
remove beresp.http.Set-Cookie;
}
if (req.request == "GET" && req.url ~ "\.(png|swf|txt|png|gif|jpg|jpeg|ico)$") {
unset req.http.cookie;
set beresp.ttl = 10d;
}
elseif (req.request == "GET" && req.url ~ "\.(html|htm|js|css)$") {
set beresp.ttl = 3600s;
else {
set beresp.ttl = 120s;
}
return (deliver);
}
sub vcl_deliver {
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT";
} else {
set resp.http.X-Cache = "MISS";
}
return (deliver)
}
sub vcl_error {
set obj.http.Content-Type = "text/html; charset=utf-8";
synthetic {"
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>"} obj.status " " obj.response {"</title>
</head>
<body>
<h1>Error "} obj.status " " obj.response {"</h1>
<p>"} obj.response {"</p>
<h3>Guru Meditation:</h3>
<p>XID: "} req.xid {"</p>
<hr>
<p>Varnish cache server</p>
</body>
</html>
"};
return (deliver);
}
简单说明下这个配置文件,一开始是定义后端服务器,是在本机上的,端口是8000
然后是管理列表acl,再下面的vcl_recv和 vcl_fetch是用来处理相关请求的.我这里的设置是主机配置"^(.*).linux-ch.com"的交后端,所有php和php?的url也直 接交给后端.在vcl_fetch中定义,所有的png|swf|txt|png|gif|jpg|jpeg|ico文件将在varnish中保留10 天,同时向后端请求的时候会删除没有带有admin|login这些关键字符的以外URL的所有cookie头,这些分别是emlog和dz的登录认证相关的url. html|htm|js|css文件将保留1个小时,其它的保留两分钟.
在2.1后的版本里,原"obj.*"的变量全部变为"beresp.*"了,需要留意一下
然后修改nginx的监听端口至8000,伪静态输出
rewrite ^(.*)/archiver/((fid|tid)-[\w\-]+\.html)$ $1/archiver/index.php?$2 last;
rewrite ^(.*)/forum-([0-9]+)-([0-9]+)\.html$ $1/forumdisplay.php?fid=$2&page=$3 last;
rewrite ^(.*)/thread-([0-9]+)-([0-9]+)-([0-9]+)\.html$ $1/viewthread.php?tid=$2&extra=page%3D$4&page=$3 last;
rewrite ^(.*)/profile-(username|uid)-(.+)\.html$ $1/viewpro.php?$2=$3 last;
rewrite ^(.*)/space-(username|uid)-(.+)\.html$ $1/space.php?$2=$3 last;
rewrite ^(.*)/tag-(.+)\.html$ $1/tag.php?name=$2 last;
break;
启动varnishd
/usr/local/varnish/sbin/varnishd -n /var/vcache -f /usr/local/varnish/etc/varnish/default.vcl -a 74.82.172.143:80 -u varnish -g varnish -s file,/var/vcache/varnish_storage.bin,64M
为方便管理,利用官方的一个脚本,将varnish添加到系统服务里,并随机启动
vim /etc/rc.d/init.d/varnish
#! /bin/sh
#
# varnish Control the varnish HTTP accelerator
#
# chkconfig: - 90 10
# description: Varnish is a high-perfomance HTTP accelerator
# processname: varnishd
# config: /etc/sysconfig/varnish
# pidfile: /var/run/varnish/varnishd.pid
### BEGIN INIT INFO
# Provides: varnish
# Required-Start: $network $local_fs $remote_fs
# Required-Stop: $network $local_fs $remote_fs
# Should-Start: $syslog
# Short-Description: start and stop varnishd
# Description: Varnish is a high-perfomance HTTP accelerator
### END INIT INFO
# Source function library.
. /etc/init.d/functions
retval=0
pidfile=/var/run/varnish.pid
exec="/usr/local/varnish/sbin/varnishd"
prog="varnishd"
config="/usr/local/varnish/etc/varnish/varnish"
lockfile="/var/lock/subsys/varnish"
# Include varnish defaults
[ -e /usr/local/varnish/etc/varnish/varnish ] && . /usr/local/varnish/etc/varnish/varnish
start() {
if [ ! -x $exec ]
then
echo $exec not found
exit 5
fi
if [ ! -f $config ]
then
echo $config not found
exit 6
fi
echo -n "Starting varnish HTTP accelerator: "
# Open files (usually 1024, which is way too small for varnish)
ulimit -n ${NFILES:-131072}
# Varnish wants to lock shared memory log in memory.
ulimit -l ${MEMLOCK:-82000}
# $DAEMON_OPTS is set in /etc/sysconfig/varnish. At least, one
# has to set up a backend, or /tmp will be used, which is a bad idea.
if [ "$DAEMON_OPTS" = "" ]; then
echo "\$DAEMON_OPTS empty."
echo -n "Please put configuration options in $config"
return 6
else
# Varnish always gives output on STDOUT
daemon $exec -P $pidfile "$DAEMON_OPTS" > /dev/null 2>&1
retval=$?
if [ $retval -eq 0 ]
then
touch $lockfile
echo_success
echo
else
echo_failure
fi
return $retval
fi
}
stop() {
echo -n "Stopping varnish HTTP accelerator: "
killproc $prog
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}
restart() {
stop
start
}
reload() {
restart
}
force_reload() {
restart
}
rh_status() {
status $prog
}
rh_status_q() {
rh_status >/dev/null 2>&1
}
# See how we were called.
case "$1" in
start)
rh_status_q && exit 0
$1
;;
stop)
rh_status_q || exit 0
$1
;;
restart)
$1
;;
reload)
rh_status_q || exit 7
$1
;;
force-reload)
force_reload
;;
status)
rh_status
;;
condrestart|try-restart)
rh_status_q || exit 0
restart
;;
*)
echo "Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload}"
exit 2
esac
exit $?
给脚本执行权限,添加服务,修改启动等级
chmod +x /etc/rc.d/init.d/varnish
/sbin/chkconfig --add varnish
/sbin/chkconfig --level 345 varnish on
varnish的配置调用文件.是用来告诉程序从哪里读取配置文件,启动参数有哪些等
vim /usr/local/varnish/etc/varnish/varnish
# Configuration file for varnish
#
# /etc/init.d/varnish expects the variable $DAEMON_OPTS to be set from this
# shell script fragment.
#
# Maximum number of open files (for ulimit -n)
NFILES=131072
# Locked shared memory (for ulimit -l)
# Default log size is 82MB + header
MEMLOCK=82000
## Alternative 2, Configuration with VCL
DAEMON_OPTS="-a 74.82.172.143:80 \
-f /usr/local/varnish/etc/varnish/default.vcl \
-T localhost:6082 \
-u varnish -g varnish \
-n /var/vcache \
-s file,/var/vcache/varnish_storage.bin,64M"
到此,整个配置就已经弄好了,整个网站的数据拓扑如下:
client
↓↑ ↑
varnish
↓ ↑
nginx
↓ ↑
php-fpm
如果varnish上面有缓存,将直接传递给用户端,没有的再向后方要,再传递给用户,同时符合存储条件的将保留下来,直至过期.从测试的效果来看,还是比较理想的,实际情况,还得看观察一段时间~
本文出自 “山里的兔子” 博客，请务必保留此出处http://motus.blog.51cto.com/78324/390672
在这个小小的vps上,从一个小小的emlog变成了emlog+ucenter+dz在上面奔跑的场面,是多么的壮观呀~
但是在实际中发现并不是那么的乐观:内存太小,后台的处理能力大打折扣;后台处理能力不高,却全是动态网站;由于是动态网站,所以更无法同时承受更多用户的访问........
在 硬件不会升级的情况下,我尝试了给php装上eaccelerator和memcache,效果并不明显,或者是我没有配置好的原因,nginx在后台依 旧很频繁是调用php-fpm.在这个时候我发现了一些曙光:让数据更少地在后台的进程间传输,然后直接传输给客户端.后在网友殘剑的介绍下尝试了 varnish这个代理软件.
软件下载:wget http://www.varnish-software.com/sites/default/files/varnish-2.1.3.tar.gz
为软件创建用户和存储数据的目录
useradd -M -r -s /sbin/nologin varnish
mkdir /var/vcache
解压,编译,安装
tar -xvf varnish-2.1.3.tar.gz
cd varnish-2.1.3
./configure --prefix=/usr/local/varnish
make
make install
修改配置文件
vim /usr/local/varnish/etc/varnish/default.vcl
backend linuxch {
.host = "127.0.0.1";
.port = "8000";
.connect_timeout = 1s;
.first_byte_timeout = 5s;
.between_bytes_timeout = 2s;
}
acl purge {
"127.0.0.1";
}
sub vcl_recv {
set req.grace = 30s;
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For ", " client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
if (req.request != "GET" &&
req.request != "HEAD" &&
req.request != "PUT" &&
req.request != "POST" &&
req.request != "TRACE" &&
req.request != "OPTIONS" &&
req.request != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}
if (req.request != "GET" && req.request != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}
if (req.http.Authorization || req.http.Cookie) {
return (pass);
}
if (req.http.host ~ "^(.*).linux-ch.com") {
set req.backend = linuxch;
}
else {
error 404 "Unknown HostName!";
}
if (req.request == "PURGE") {
if (!client.ip ~ purge) {
error 405 "Not Allowed.";
return (lookup);
}
}
if (req.url ~ "\.(php)($|\?)") {
return (pass);
}
if (req.http.Cache-Control ~ "(no-cache|max-age=0)") {
purge_url(req.url);
}
return (lookup);
}
sub vcl_pipe {
return (pipe);
}
sub vcl_pass {
return (pass);
}
sub vcl_hash {
set req.hash += req.url;
if (req.http.host) {
set req.hash += req.http.host;
} else {
set req.hash += server.ip;
}
return (hash);
}
sub vcl_hit {
if (!obj.cacheable) {
return (pass);
}
if (req.request == "PURGE") {
set obj.ttl = 0s;
error 200 "Purged.";
}
return (deliver);
}
sub vcl_miss {
if (req.request == "PURGE") {
error 404 "Not in cache.";
}
return (fetch);
}
sub vcl_fetch {
if (!beresp.cacheable) {
return (pass);
}
if (!(req.url ~ "(admin|login)")) {
remove beresp.http.Set-Cookie;
}
if (req.request == "GET" && req.url ~ "\.(png|swf|txt|png|gif|jpg|jpeg|ico)$") {
unset req.http.cookie;
set beresp.ttl = 10d;
}
elseif (req.request == "GET" && req.url ~ "\.(html|htm|js|css)$") {
set beresp.ttl = 3600s;
else {
set beresp.ttl = 120s;
}
return (deliver);
}
sub vcl_deliver {
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT";
} else {
set resp.http.X-Cache = "MISS";
}
return (deliver)
}
sub vcl_error {
set obj.http.Content-Type = "text/html; charset=utf-8";
synthetic {"
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>"} obj.status " " obj.response {"</title>
</head>
<body>
<h1>Error "} obj.status " " obj.response {"</h1>
<p>"} obj.response {"</p>
<h3>Guru Meditation:</h3>
<p>XID: "} req.xid {"</p>
<hr>
<p>Varnish cache server</p>
</body>
</html>
"};
return (deliver);
}
简单说明下这个配置文件,一开始是定义后端服务器,是在本机上的,端口是8000
然后是管理列表acl,再下面的vcl_recv和 vcl_fetch是用来处理相关请求的.我这里的设置是主机配置"^(.*).linux-ch.com"的交后端,所有php和php?的url也直 接交给后端.在vcl_fetch中定义,所有的png|swf|txt|png|gif|jpg|jpeg|ico文件将在varnish中保留10 天,同时向后端请求的时候会删除没有带有admin|login这些关键字符的以外URL的所有cookie头,这些分别是emlog和dz的登录认证相关的url. html|htm|js|css文件将保留1个小时,其它的保留两分钟.
在2.1后的版本里,原"obj.*"的变量全部变为"beresp.*"了,需要留意一下
然后修改nginx的监听端口至8000,伪静态输出
rewrite ^(.*)/archiver/((fid|tid)-[\w\-]+\.html)$ $1/archiver/index.php?$2 last;
rewrite ^(.*)/forum-([0-9]+)-([0-9]+)\.html$ $1/forumdisplay.php?fid=$2&page=$3 last;
rewrite ^(.*)/thread-([0-9]+)-([0-9]+)-([0-9]+)\.html$ $1/viewthread.php?tid=$2&extra=page%3D$4&page=$3 last;
rewrite ^(.*)/profile-(username|uid)-(.+)\.html$ $1/viewpro.php?$2=$3 last;
rewrite ^(.*)/space-(username|uid)-(.+)\.html$ $1/space.php?$2=$3 last;
rewrite ^(.*)/tag-(.+)\.html$ $1/tag.php?name=$2 last;
break;
启动varnishd
/usr/local/varnish/sbin/varnishd -n /var/vcache -f /usr/local/varnish/etc/varnish/default.vcl -a 74.82.172.143:80 -u varnish -g varnish -s file,/var/vcache/varnish_storage.bin,64M
为方便管理,利用官方的一个脚本,将varnish添加到系统服务里,并随机启动
vim /etc/rc.d/init.d/varnish
#! /bin/sh
#
# varnish Control the varnish HTTP accelerator
#
# chkconfig: - 90 10
# description: Varnish is a high-perfomance HTTP accelerator
# processname: varnishd
# config: /etc/sysconfig/varnish
# pidfile: /var/run/varnish/varnishd.pid
### BEGIN INIT INFO
# Provides: varnish
# Required-Start: $network $local_fs $remote_fs
# Required-Stop: $network $local_fs $remote_fs
# Should-Start: $syslog
# Short-Description: start and stop varnishd
# Description: Varnish is a high-perfomance HTTP accelerator
### END INIT INFO
# Source function library.
. /etc/init.d/functions
retval=0
pidfile=/var/run/varnish.pid
exec="/usr/local/varnish/sbin/varnishd"
prog="varnishd"
config="/usr/local/varnish/etc/varnish/varnish"
lockfile="/var/lock/subsys/varnish"
# Include varnish defaults
[ -e /usr/local/varnish/etc/varnish/varnish ] && . /usr/local/varnish/etc/varnish/varnish
start() {
if [ ! -x $exec ]
then
echo $exec not found
exit 5
fi
if [ ! -f $config ]
then
echo $config not found
exit 6
fi
echo -n "Starting varnish HTTP accelerator: "
# Open files (usually 1024, which is way too small for varnish)
ulimit -n ${NFILES:-131072}
# Varnish wants to lock shared memory log in memory.
ulimit -l ${MEMLOCK:-82000}
# $DAEMON_OPTS is set in /etc/sysconfig/varnish. At least, one
# has to set up a backend, or /tmp will be used, which is a bad idea.
if [ "$DAEMON_OPTS" = "" ]; then
echo "\$DAEMON_OPTS empty."
echo -n "Please put configuration options in $config"
return 6
else
# Varnish always gives output on STDOUT
daemon $exec -P $pidfile "$DAEMON_OPTS" > /dev/null 2>&1
retval=$?
if [ $retval -eq 0 ]
then
touch $lockfile
echo_success
echo
else
echo_failure
fi
return $retval
fi
}
stop() {
echo -n "Stopping varnish HTTP accelerator: "
killproc $prog
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}
restart() {
stop
start
}
reload() {
restart
}
force_reload() {
restart
}
rh_status() {
status $prog
}
rh_status_q() {
rh_status >/dev/null 2>&1
}
# See how we were called.
case "$1" in
start)
rh_status_q && exit 0
$1
;;
stop)
rh_status_q || exit 0
$1
;;
restart)
$1
;;
reload)
rh_status_q || exit 7
$1
;;
force-reload)
force_reload
;;
status)
rh_status
;;
condrestart|try-restart)
rh_status_q || exit 0
restart
;;
*)
echo "Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload}"
exit 2
esac
exit $?
给脚本执行权限,添加服务,修改启动等级
chmod +x /etc/rc.d/init.d/varnish
/sbin/chkconfig --add varnish
/sbin/chkconfig --level 345 varnish on
varnish的配置调用文件.是用来告诉程序从哪里读取配置文件,启动参数有哪些等
vim /usr/local/varnish/etc/varnish/varnish
# Configuration file for varnish
#
# /etc/init.d/varnish expects the variable $DAEMON_OPTS to be set from this
# shell script fragment.
#
# Maximum number of open files (for ulimit -n)
NFILES=131072
# Locked shared memory (for ulimit -l)
# Default log size is 82MB + header
MEMLOCK=82000
## Alternative 2, Configuration with VCL
DAEMON_OPTS="-a 74.82.172.143:80 \
-f /usr/local/varnish/etc/varnish/default.vcl \
-T localhost:6082 \
-u varnish -g varnish \
-n /var/vcache \
-s file,/var/vcache/varnish_storage.bin,64M"
到此,整个配置就已经弄好了,整个网站的数据拓扑如下:
client
↓↑ ↑
varnish
↓ ↑
nginx
↓ ↑
php-fpm
如果varnish上面有缓存,将直接传递给用户端,没有的再向后方要,再传递给用户,同时符合存储条件的将保留下来,直至过期.从测试的效果来看,还是比较理想的,实际情况,还得看观察一段时间~
本文出自 “山里的兔子” 博客，请务必保留此出处http://motus.blog.51cto.com/78324/390672
内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理 
相关文章推荐
- [2013-07-22]varnish-cache 安装配置及体验笔记
- Hadoop学习笔记_Ubuntu下伪分布式安装及配置
- varnish安装配置
- ANT的安装/配置笔记
- alfresco安装配置笔记
- ubuntu笔记 - 安装和配置Sublime Text
- Supervisord安装配置笔记
- Net-SNMP V3协议 安装配置笔记（CentOS 6.3/5.6）
- Mysql Cluster集群安装、配置笔记（版本7.2.8）
- varnisha 安装配置
- 学习笔记1——Linux（CentOS）在虚拟机上最小化安装之后的网络配置及其与主机的连接
- 安装配置Varnish3.0手记
- Panabit 安装笔记之Panabit的安装和配置 推荐
- 第116讲：Hadoop集群之安装Java、创建Hadoop用户、配置SSH等实战学习笔记
- （开发笔记）转载：MySQL 5.6 for Windows 解压缩版配置安装
- (笔记) BT5 Nessus安装、配置、离线升级和插件调试方法
- 快速笔记01-MySQL多实例的安装与配置
- Hadoop学习笔记0001——Hadoop安装配置
- maven（一）学习笔记-安装，配置