CDN缓存系统构建(用varnish实现)

过程

client->dns->cdn->server->cdn->cache->client
cdn反向代理：可以缓解真实服务器的压力
VLC流程处理图：

处理过程大致分为如下几个步骤:
(1)Receive 状态,也就是请求处理的入口状态,根据 VCL 规则判断该请求应该是 Pas 4000 s 或Pipe,或者进入 Lookup(本地查询)。其中Pipe会让客户端直接和服务端通讯，传送完毕就释放掉，后续请求和varnish无关。
(2)Lookup 状态,进入此状态后,会在 hash 表中查找数据,若找到,则进入 Hit 状态,否则进入 miss 状态。
(3)Pass 状态,在此状态下,会进入后端请求,即进入 fetch 状态。
(4)Fetch 状态,在 Fetch 状态下,对请求进行后端的获取,发送请求,获得数据,并进行本地的存储。
(5)Deliver 状态, 将获取到的数据发送给客户端,然后完成本次请求。

实验环境

cdn：server1（172.25.69.1）
服务器1：server2（172.25.69.2)
服务器2：server3（172.25.69.3)

CDN:

[root@server1 ~]# ls
varnish-3.0.5-1.el6.x86_64.rpm  varnish-libs-3.0.5-1.el6.x86_64.rpm
[root@server1 ~]# yum install * -y   //下载varnish安装包
[root@server1 ~]# cd /etc/varnish
[root@server1 varnish]# vim /etc/sysconfig/varnish
//配置服务端口为80（http的端口为80，为了在浏览时方便查看）文件内容如下

[root@server1 varnish]# sysctl -a | grep file  //显示文件的系统内核参数
fs.file-nr = 416	0	98864
fs.file-max = 98864
[root@server1 varnish]# ulimit -a  //显示当前的各种用户进程限制
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 7820
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 7820
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[root@server1 varnish]# cat /etc/passwd
varnish:x:498:499:Varnish Cache:/var/lib/varnish:/sbin/nologin
[root@server1 varnish]# vim /etc/security/limits.conf //末尾添加下图内容
//系统限制值limits，根据文件/etc/sysconfig/varnish里面的限制值进行配置，对cpu不做限制

配置一个后端服务器

[root@server1 varnish]# /etc/init.d/varnish start   //开启varnish服务
[root@server1 varnish]# vim default.vcl   //配置后端服务器

[root@server1 varnish]# /etc/init.d/varnish reload

服务器1：

[root@server2 ~]# yum install -y httpd
[root@server2 ~]# vim /var/www/html/index.html
www.westos.org
[root@server2 ~]# /etc/init.d/httpd start

客户端测试：

[root@foundation69 Desktop]# vim /etc/hosts  //写解析
172.25.69.1 www.westos.org bbs.westos.org westos.org
[root@foundation69 Desktop]# ping www.westos.org
PING www.westos.org (172.25.69.1) 56(84) bytes of data.
64 bytes from www.westos.org (172.25.69.1): icmp_seq=1 ttl=64 time=0.114 ms
64 bytes from www.westos.org (172.25.69.1): icmp_seq=2 ttl=64 time=0.168 ms
^C
--- www.westos.org ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.114/0.141/0.168/0.027 ms

[root@foundation69 Desktop]# curl 172.25.69.1 -I
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 03:48:35 GMT
ETag: "a0043-f-572071a4cb6e9"
Content-Type: text/html; charset=UTF-8
Content-Length: 15
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 03:57:53 GMT
X-Varnish: 1184392887
Age: 0
Via: 1.1 varnish
Connection: keep-alive
X-Cache: MISS from ying cache

[root@foundation69 Desktop]# curl 172.25.69.1
www.westos.org

浏览器可以查看到服务器1的发布目录如下图：

排错：
查看配置文件是否有编辑错误
[root@server1 varnish]# iptables -L //查看有没有访问规则，查看iptables是否关闭
[root@server1 varnish]# netstat -antlp |grep :80 //查看varnish是否在80端口的服务
[root@server1 varnish]# ip addr //查看ip是否在同网段
[root@server1 varnish]# /etc/init.d/varnish stop //关闭服务
[root@server1 varnish]# /etc/init.d/varnish start //开启服务
[root@server1 varnish]# /etc/init.d/varnish restart

varnishadm ban.url .*$ 清除所有
varnishadm ban.url /index.html 清除 index.html 页面缓存，清除首页,在client查看时age是从0开始
varnishadm ban.url /admin/$ 清除 admin 目录缓存

测试

客户端：

[root@foundation69 Desktop]# curl www.westos.org/index.html -I
//查看时有缓存
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 03:48:35 GMT
ETag: "a0043-f-572071a4cb6e9"
Content-Type: text/html; charset=UTF-8
Content-Length: 15
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 05:43:50 GMT
X-Varnish: 1184392904 1184392903
Age: 16
Via: 1.1 varnish
Connection: keep-alive
X-Cache: HIT from ying cache

CDN：

[root@server1 varnish]# varnishadm ban.url /index.html
//对index.html的页面缓存清除

客户端：

[root@foundation69 Desktop]# curl www.westos.org/index.html -I
//再次查看时没有缓存
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 03:48:35 GMT
ETag: "a0043-f-572071a4cb6e9"
Content-Type: text/html; charset=UTF-8
Content-Length: 15
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 05:44:29 GMT
X-Varnish: 1184392905
Age: 0     //显示次数清零
Via: 1.1 varnish
Connection: keep-alive
X-Cache: MISS from ying cache

配置多个不同域名站点的后端服务器

服务器2：

[root@server3 ~]# yum install -y httpd
[root@server3 ~]# vim /var/www/html/index.html
bbs.westos.org
[root@server3 ~]# /etc/init.d/httpd start  //开启httpd

CDN：

root@server1 varnish]# vim default.vcl

[root@server1 varnish]# /etc/init.d/varnish reload

客户端：

[root@foundation69 Desktop]# curl 172.25.69.1/index.html -I
//以ip访问时会报错
HTTP/1.1 404 westos cache
Server: Varnish
Content-Type: text/html; charset=utf-8
Retry-After: 5
Content-Length: 398
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 05:52:47 GMT
X-Varnish: 1621918283
Age: 0
Via: 1.1 varnish
Connection: close
X-Cache: MISS from ying cache

[root@foundation69 Desktop]# curl www.westos.org/index.html -I
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 03:48:35 GMT
ETag: "a0043-f-572071a4cb6e9"
Content-Type: text/html; charset=UTF-8
Content-Length: 15
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 05:53:00 GMT
X-Varnish: 1621918284 1621918278
Age: 85
Via: 1.1 varnish
Connection: keep-alive
X-Cache: HIT from ying cache

[root@foundation69 Desktop]# curl westos.org/index.html -I
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 03:48:35 GMT
ETag: "a0043-f-572071a4cb6e9"
Content-Type: text/html; charset=UTF-8
Content-Length: 15
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 05:53:04 GMT
X-Varnish: 1621918285 1621918278
Age: 90
Via: 1.1 varnish
Connection: keep-alive
X-Cache: HIT from ying cache

[root@foundation69 Desktop]# curl bbs.westos.org/index.html -I
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 05:48:13 GMT
ETag: "a0043-f-57208c62be6c5"
Content-Type: text/html; charset=UTF-8
Content-Length: 15
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 05:53:12 GMT
X-Varnish: 1621918286
Age: 0
Via: 1.1 varnish
Connection: keep-alive
X-Cache: MISS from ying cache

负载均衡

服务器2：

[root@server3 ~]# vim /etc/httpd/conf/httpd.conf

[root@server3 ~]# mkdir /www1
[root@server3 ~]# vim /www1/index.html
[root@server3 ~]# cat /www1/index.html  //编写发布目录
www.westos.org -server3
[root@server3 ~]# cat /var/www/html/index.html
bbs.westos.org
[root@server3 ~]# /etc/init.d/httpd restart

服务器1：

[root@server2 ~]# cat /var/www/html/index.html
www.westos.org -server2
[root@server2 ~]# /etc/init.d/httpd restart

CDN：

[root@server1 varnish]# vim default.vcl

1、加上pass时相当于一个负载均衡器，不进行缓存，两个服务器交换使用
在客户端测试如下：

[root@foundation69 Desktop]# curl www.westos.org
www.westos.org -server2
[root@foundation69 Desktop]# curl www.westos.org
www.westos.org -server3
[root@foundation69 Desktop]# curl www.westos.org
www.westos.org -server2
[root@foundation69 Desktop]# curl www.westos.org
www.westos.org -server3
[root@foundation69 Desktop]# curl www.westos.org
www.westos.org -server2
[root@foundation69 Desktop]# curl www.westos.org/index.html -I
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 06:08:15 GMT
ETag: "a016a-18-572090dc815c9"
Content-Type: text/html; charset=UTF-8
Content-Length: 24
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 06:15:27 GMT
X-Varnish: 1674890205
Age: 0
Via: 1.1 varnish
Connection: keep-alive
X-Cache: MISS from ying cache

[root@foundation69 Desktop]# curl www.westos.org/index.html -I
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 06:08:15 GMT
ETag: "a016a-18-572090dc815c9"
Accept-Ranges: bytes
Content-Length: 24
Content-Type: text/html; charset=UTF-8
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 06:17:12 GMT
X-Varnish: 1850682020
Age: 0
Via: 1.1 varnish
Connection: keep-alive
X-Cache: MISS from ying cache

2、不加pass时，会访问第一个缓存的记录，在到达访问限制次数120时，再访问并缓存另外一个正常工作的服务器，当一个服务器坏的时候还会继续访问此服务器的缓存，次数到达120时，再访问正常的服务器进行缓存
在客户端测试如下：

[root@foundation69 Desktop]# curl www.westos.org
www.westos.org -server2
[root@foundation69 Desktop]# curl www.westos.org
www.westos.org -server2
[root@foundation69 Desktop]# curl www.westos.org
www.westos.org -server2
[root@foundation69 Desktop]# curl www.westos.org/index.html -I
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 06:06:25 GMT
ETag: "1fed8-18-572090748156b"
Content-Type: text/html; charset=UTF-8
Content-Length: 24
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 06:14:07 GMT
X-Varnish: 1674890204 1674890202
Age: 82
Via: 1.1 varnish
Connection: keep-alive
X-Cache: HIT from ying cache

[root@foundation69 Desktop]# curl www.westos.org/index.html -I
//访问缓存另外一个server
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 06:08:15 GMT
ETag: "a016a-18-572090dc815c9"
Content-Type: text/html; charset=UTF-8
Content-Length: 24
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 06:15:27 GMT
X-Varnish: 1674890205
Age: 0
Via: 1.1 varnish
Connection: keep-alive
X-Cache: MISS from ying cache

[root@foundation69 Desktop]# curl www.westos.org
www.westos.org -server3  //查看，确定访问的服务器换了

Varnish cdn 推送平台

CDN：

[root@server1 ~]# yum install httpd -y
[root@server1 ~]# vim /etc/httpd/conf/httpd.conf  //修改内容如下

[root@server1 ~]# /etc/init.d/httpd start
[root@server1 ~]# yum install -y unzip
[root@server1 ~]# unzip bansys.zip -d /var/www/html/
[root@server1 ~]# cd /var/www/html
[root@server1 html]# ls
bansys
[root@server1 html]# cd bansys/
[root@server1 html]# ls
[root@server1 html]# mv * ..
[root@server1 html]# ls
[root@server1 html]# cd ..
[root@server1 html]# ls
bansys  class_socket.php  config.php  index.php  purge_action.php  static
[root@server1 html]# vim config.php  //内容如下
<?php

//varnish主机列表
//可定义多个主机列表
$var_group1 = array(
'host' => array('172.25.69.1'),		#varnish主机ip
'port' => '80',   #varnish端口
);

//varnish群组定义
//对主机列表进行绑定
$VAR_CLUSTER = array(
'www.westos.org' => $var_group1,
);

//varnish版本
//2.x和3.x推送命令不一样
$VAR_VERSION = "3";
?>
[root@server1 html]# netstat -antlp
SHED 1910/sshd
tcp        0      0 :::8080                     :::*                        LISTEN
[root@server1 html]# vim /etc/varnish/default.vcl  //添加如下内容

[root@server1 html]# /etc/init.d/varnish reload

客户端测试：

[root@foundation69 Desktop]# curl www.westos.org/index.html
//查看内容已经变化
www.westos.org -server2
[root@foundation69 Desktop]# curl www.westos.org/index.html
www.westos.org -server2

浏览器：

172.25.69.1:8080

[root@foundation69 Desktop]# curl www.westos.org/index.html
//查看时有变化，
1a4bc
在浏览器没有变化是因为浏览器有自己的缓存
www.westos.org -server3