Redis数据持久化-----RDB和AOF详解

一、RDB（Redis DataBase）
1、官网介绍：
在指定的时间间隔内将内存中的数据集快照写入磁盘，也就是进行Snapshot快照，它恢复时是将快照文件直接写入内存中
2、什么是RDB（Redis DataBase）&AOF(Append Only File)?
Redis会单独创建（fork）一个子进程来进行持久化操作，会先将数据写入到一个临时文件中，主进程是不进行任何IO操作的，这就确保了极高的性能。如果需要进行大规模的数据的恢复，且对于数据恢复的完整性不是非常敏感，那么RDB方式要比AOF方式更加的高效。RDB的缺点就是最后一次持久化后的数据可能会丢失。
3、fork
Fork的作用是赋值一个与当前进程一样的进程，新进程的所有数据（变量，环境变量，程序计数器等）数值都和原进程一样，但是是一个全新的进程，并且作为原进程的子进程。
4、RDB保存的是dump.rdb文件
5、配置文件中的位置：
redis.conf

################### SNAPSHOTTING  ########################
#
# Save the DB on disk:
#
#   save <seconds> <changes>
#
#   Will save the DB if both the given number of seconds and the given
#   number of write operations against the DB occurred.
#
#   In the example below the behaviour will be to save:
#   after 900 sec (15 min) if at least 1 key changed
#   after 300 sec (5 min) if at least 10 keys changed
#   after 60 sec if at least 10000 keys changed
#
#   Note: you can disable saving completely by commenting out all "save" lines.
#
#   It is also possible to remove all the previously configured save
#   points by adding a save directive with a single empty string argument
#   like in the following example:
#
#   save ""
save 900 1
save 300 10
save 60 10000
# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes
# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes
# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
rdbchecksum yes
# The filename where to dump the DB
dbfilename dump.rdb
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir ./

配置文件详解：
5.1、SNAPSHOTTING快照
5.1.1、保存策略：
save
save 秒钟 写操作次数

数值可以修改

如果要使某个key-value要求即时生效写入dump.rdb文件中，可用如下操作

5.1.2、stop-writes-on-bgsave-error 默认是yes

5.1.3、rdbcompression 默认是yes

5.1.4、rdbchecksum 默认是yes

5.1.5、dbfilename 默认为 dump.rdb 存储RDB快照的默认文件名
5.1.6、dir 默认为./ rdb快照文件的存储路径
6、如何触发RDB快照
6.1、配置文件中默认的快照—>冷拷贝后重新使用—>cp dump.rdb dump_new.rdb
6.2、命令save或者是bgsave
(1)、save：save时只管保存，其他不管，全部阻塞
(2)、bgsave：redis会在后台异步进行快照操作，快照同时还可以响应客户端请求，可以通过lastsave命令获取最后一次成功执行快照的时间
6.3、执行flashall命令，也会产生dump.rdb文件，但是里面是空的，没有什么意义
7、如何恢复
将备份文件dump.rdb移动到redis安装目录并启动服务即可
CONFIG GET dir获取目录

8、优势
适合大规模的数据恢复，对数据的完整性和一致性要求不高
9、劣势
fork的时候，内存中的数据被克隆了一份，大致两倍的膨胀性需要考虑；最后save的数据可能会丢失
10、如何停止
动态所有停止RDB保存规则的方法：redis-cli config set save “”
11、RDB小总结

二、AOF(Append Only File)
1、什么是AOF？
AOF是以日志的形式记录每个写操作，将Redis执行过的所有写指令记录下来（读操作不记录），只许追加文件但不可以改写文件，redis启动之初会读取该文件重新构建数据，换言之，redis重启的话就根据日志文件的内容将写指令从前到后执行一次以完成数据的恢复工作
2、AOF保存的是appendonly.aof文件
3、配置位置：
redis.conf
################ APPEND ONLY MODE ####################

# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF,
1cca7
that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.

appendonly no

# The name of the append only file (default: "appendonly.aof")

appendfilename "appendonly.aof"

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".

# appendfsync always
appendfsync everysec
# appendfsync no

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.

no-appendfsync-on-rewrite no

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

# When rewriting the AOF file, Redis is able to use an RDB preamble in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
#   [RDB file][AOF tail]
#
# When loading Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, and continues loading the AOF
# tail.
aof-use-rdb-preamble yes

3.1、默认情况下是关闭AOF

开启AOF支持
3.2、当appendonly.aof和dum.rdb文件同时存在时，redis启动时优先加载.aof文件，如果加载.aof文件失败（如.aof文件内部出现乱码等，此时redis服务也无法启动），可使用命令redis-check-aof --fix appendonly.aof尝试修复
AOF and RDB persistence can be enable at the same time without peoblem.If the AOF is enable on start up Redis will load the AOF, that is the file with the better durability guarantees.

3.3、AOF同步策略

always: 同步持久化 每次发生数据变更时会被立即记录到磁盘 性能比较差但是数据完整性好；
everysec: 出厂默认推荐，异步操作，每秒记录 如果一秒宕机，就会有数据丢失；
no: 不使用同步持久化
3.4、No-appendfsync-on-rewrite：重写时是否可以运用Appendfsync，用默认的no就可以，保证数据安全性

3.5、auto-aof-rewrite-min-size：设置重写的基准值
3.6、auto-aof-rewrite-percentage：设置重写的基准值
4、AOF启动/修复/恢复
正常恢复：启动：修改默认的appenonly no,改为yes
将有数据的aof文件复制一份保存到对应目录（config get dir)
恢复：重启redis然后重新加载
异常恢复：启动：修改默认的appenonly no，改为yes
备份被谢怀的aof文件
修复： redis-check-aof --fix appendonly.aof 进行修复
恢复：重启redis然后重新加载
5、Rewrite
5.1、是什么？
AOF采用文件追加的方式，文件会越来越大为了避免出现此种情况，新增了重写机制，当AOF文件的大小超过所设定的阈值时，Redis就会启动AOF文件的内容压缩，只保留可以恢复数据的最小指令集，可以使用命令bgrewriteaof
5.2、重写原理
AOF文件持续增长而过大的时候，会fork出一条新进程来讲文件重写（也就是先写临时文件最后再rename），遍历新进程的内存中数据，每条记录有一条的Set语句，重写aof文件的操作，并没有读取旧的aof文件，而是将整个内存中的数据库内容用命令的方式重写了一个新的aof文件，这点和快照有点类似
5.3、触发机制
redis会记录上次重写时的AOF大小，默认配置是当AOF文件大小是上次rewrite后大小的已被且文件大于64M时触发

6、优势
a. 每修改同步：appendfsync always 同步持久化 每次发生数据变更时会被立即记录到磁盘，性能较差但是完整性较好
b. 每秒同步：appendfsync everysec 异步操作，每秒记录 如果一秒内宕机，有数据丢失
c. 不同步：appendfsync no 从不同步
7、劣势
相同数据集的数据而言aof文件要远大于rdb文件，恢复速度慢于rdb
Aof运行效率要慢于rdb，每秒同步策略效率较好。不同步效率与rdb相同
8、小总结

三、总结（AOF VS RDB） which one should I use?
a、RDB持久化方式能够在指定的时间间隔能对你的数据进行快照存储
b、AOF持久化方式记录每次对服务器写的操作，当服务器重启的时候会重新执行这些命令来恢复原始的数据，AOF命令以redis协议追加保存每次写的操作到文件末尾，Redis还能对AOF文件进行后台重写，使得AOF文件的体积不至于过大
c、只做缓存：如果你希望你的数据在服务器上运行的时候存在，你也可以不使用任何的持久化方式
d、同时开启两种持久化方式
在这种情况下，当redis重启的时候会优先载入aof文件来恢复原始的数据，因为在通常情况下AOF文件保存的数据集要比RDB文件保存的数据集要完整；
RDB数据实时，同时RDB更适合用于备份数据库（AOF在不断变化不好备份），快速重启，而且不会有AOF可能潜在的bug,留着作为一个万一的手段
e、性能建议
因为RDB文件只用作后备用途，建议只在Slave上持久化RDB文件，而且只要15分钟备份一次就够了，只保留save 900 1这条规则
如果Enable AOF，好处是在最恶劣的情况下也只会丢失不超过两秒的数据，启动脚本较为简单，只load自己的AOF文件就可以了，代价是带来了持续的IO，二是AOF rewrite的最后将rewrite过程中产生的新数据写到新文件造成的阻塞基乎是不可能避免的，只要硬盘许可，应该尽量减少AOF rewrite的频率，AOF重写的基础大小默认是64M太小了，可以设置到5G以上，默认超过原数据文件100%大小时重写，可以改到适当数值。
如果不Enable AOF，仅靠Master-Slave Replication实现高可用也可以。能省掉一大笔IO也减少了rewrite时带来的系统波动，代价是如果Master/slave同时倒掉，会丢失十几分钟的数据，启动脚本也要比较两个Master/Slave中的RDB文件，载入较新的那个。新浪微博就采用的这种架构。