关于archive引擎收集的海量数据表的备份
2015-05-08 09:26
274 查看
每天按ID区间备份表数据
#!/bin/bash
date -d yesterday +"DATE %Y-%m-%d %H:%M:%S" >> /Mysql_Data/t_collection/ID.log
/usr/bin/mysql -uroot --password="admin" -D userbehavior -e "select max(ID) from t_collection;"|grep -v 'max(ID)' >> /Mysql_Data/t_collection/ID.log
today=`tail -n 1 /Mysql_Data/t_collection/ID.log`
yesterday=`tail -n 3 /Mysql_Data/t_collection/ID.log|grep -v 'DATE'|grep -v $today`
sql="use userbehavior;\n
select * from t_collection where ID>$yesterday and ID<=$today into outfile '/Mysql_Data/t_collection/`date -d yesterday +"%Y-%m-%d"`.csv'\n
FIELDS TERMINATED BY ',' ENCLOSED BY '\"' ESCAPED BY '' LINES TERMINATED BY '\n'"
/usr/bin/mysql -uroot --password="admin" -D userbehavior -e "$sql";
cd /Mysql_Data/t_collection
tar czvf `date -d yesterday +"%Y-%m-%d"`.tgz `date -d yesterday +"%Y-%m-%d"`.csv
rm -f /Mysql_Data/t_collection/`date -d yesterday +"%Y-%m-%d"`.csv
备份某个具体区间的表数据
mysql -p'password' -D userbehavior -e "select * from t_collection where ID>3899999999 and ID<4000000000 into outfile '/mnt/backup/t_c/40WW.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '\"' ESCAPED BY '' LINES TERMINATED BY '\n';"
程序修改为分表后的表数据备份脚本如下:
#!/bin/bash
table=t_collection`date -d yesterday +"%Y%m%d"`
sql="use userbehavior;\n
select ID,REPLACE(REPLACE(Path,'\n','@'),'\r','@'),REPLACE(REPLACE(Content,'\n','@'),'\r','@') ,CreateTime from $table into outfile '/data/backup/Mysql_Data/new_collection/`date -d yesterday +"%Y-%m-%d"`.csv'\n
FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'"
/usr/bin/mysql -uroot --password="password" -D userbehavior -e "$sql";
cd /data/backup/Mysql_Data/new_collection/
tar czvf `date -d yesterday +"%Y-%m-%d"`.tgz `date -d yesterday +"%Y-%m-%d"`.csv
rm -f `date -d yesterday +"%Y-%m-%d"`.csv
echo "$(date +%Y%m%d_%H:%M:%S) $table is bak finish">>/data/backup/Mysql_Data/new_collection/t_collection_bak.log
cp `date -d yesterday +"%Y-%m-%d"`.tgz /mnt/GZ_GuangGao_29/Collection/
加入crontab
35 0 * * * root sh /data/backup/scriptshell/t_collectionnew.sh
#!/bin/bash
sql="use userbehavior;\n
select a.* from coocaaDaoHang a\n
where a.createtime like '$1%'\n
into outfile '/data/backup/Mysql_Data/DaoHang/$1.csv'\n
CHARACTER SET gbk \n
fields terminated by ',' \n
optionally enclosed by '\"' escaped by '\\\' \n
lines terminated by '\r\n';"
/usr/bin/mysql -uroot --password="password" -D userbehavior -e "${sql}";
加入crontab
01 0 * * * root /data/backup/scriptshell/coocaaDaoHang.sh `date -d yesterday +"\%Y-\%m-\%d"`
#!/bin/bash
date -d yesterday +"DATE %Y-%m-%d %H:%M:%S" >> /Mysql_Data/t_collection/ID.log
/usr/bin/mysql -uroot --password="admin" -D userbehavior -e "select max(ID) from t_collection;"|grep -v 'max(ID)' >> /Mysql_Data/t_collection/ID.log
today=`tail -n 1 /Mysql_Data/t_collection/ID.log`
yesterday=`tail -n 3 /Mysql_Data/t_collection/ID.log|grep -v 'DATE'|grep -v $today`
sql="use userbehavior;\n
select * from t_collection where ID>$yesterday and ID<=$today into outfile '/Mysql_Data/t_collection/`date -d yesterday +"%Y-%m-%d"`.csv'\n
FIELDS TERMINATED BY ',' ENCLOSED BY '\"' ESCAPED BY '' LINES TERMINATED BY '\n'"
/usr/bin/mysql -uroot --password="admin" -D userbehavior -e "$sql";
cd /Mysql_Data/t_collection
tar czvf `date -d yesterday +"%Y-%m-%d"`.tgz `date -d yesterday +"%Y-%m-%d"`.csv
rm -f /Mysql_Data/t_collection/`date -d yesterday +"%Y-%m-%d"`.csv
备份某个具体区间的表数据
mysql -p'password' -D userbehavior -e "select * from t_collection where ID>3899999999 and ID<4000000000 into outfile '/mnt/backup/t_c/40WW.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '\"' ESCAPED BY '' LINES TERMINATED BY '\n';"
程序修改为分表后的表数据备份脚本如下:
#!/bin/bash
table=t_collection`date -d yesterday +"%Y%m%d"`
sql="use userbehavior;\n
select ID,REPLACE(REPLACE(Path,'\n','@'),'\r','@'),REPLACE(REPLACE(Content,'\n','@'),'\r','@') ,CreateTime from $table into outfile '/data/backup/Mysql_Data/new_collection/`date -d yesterday +"%Y-%m-%d"`.csv'\n
FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'"
/usr/bin/mysql -uroot --password="password" -D userbehavior -e "$sql";
cd /data/backup/Mysql_Data/new_collection/
tar czvf `date -d yesterday +"%Y-%m-%d"`.tgz `date -d yesterday +"%Y-%m-%d"`.csv
rm -f `date -d yesterday +"%Y-%m-%d"`.csv
echo "$(date +%Y%m%d_%H:%M:%S) $table is bak finish">>/data/backup/Mysql_Data/new_collection/t_collection_bak.log
cp `date -d yesterday +"%Y-%m-%d"`.tgz /mnt/GZ_GuangGao_29/Collection/
加入crontab
35 0 * * * root sh /data/backup/scriptshell/t_collectionnew.sh
#!/bin/bash
sql="use userbehavior;\n
select a.* from coocaaDaoHang a\n
where a.createtime like '$1%'\n
into outfile '/data/backup/Mysql_Data/DaoHang/$1.csv'\n
CHARACTER SET gbk \n
fields terminated by ',' \n
optionally enclosed by '\"' escaped by '\\\' \n
lines terminated by '\r\n';"
/usr/bin/mysql -uroot --password="password" -D userbehavior -e "${sql}";
加入crontab
01 0 * * * root /data/backup/scriptshell/coocaaDaoHang.sh `date -d yesterday +"\%Y-\%m-\%d"`
相关文章推荐
- 收集关于scrollTop信息 http://www.cnblogs.com/FrameWork/archive/2007/04/13/712750.html
- MySQL20个经典面试题(关于引擎 日志 备份 比较深入)
- 关于oracle的backup database plus archivelog delete input备份命令
- 关于使用BE备份数据库的问题收集
- 转:收集两个关于xml和xsl的例子。
- 关于git(二)-详解不使用fork迁移备份git仓库
- 关于mysql数据库备份的一些选项和参数
- 关于Oracle备份内容
- IOS学习资源收集--关于动画的代码学习资源总汇(很棒的动画效果哦)
- 关于数据库建表失败——数据库引擎
- 关于Exchange 无法备份问题的解决方案
- 关于模板引擎--望批评指正
- 驰骋工作流程引擎设计-关于workid的设计.
- 关于引擎的设计
- 关于乱码的收集
- 关于JAVA垃圾收集器与类的finalize()方法的一些总结
- 在网上收集了一部分关于使用Google API进行手机定位的资料和大家分享
- 关于进程间的访问权限等进程间控制资料收集
- 关于在Mac上挂载移动硬盘实现数据备份的方法
- 关于索引的总结[收集中]