把数据库里的大表放到REDIS中（从Mysql将数据快速导入到redis中 ）

在把一个大表从 MySQL 迁移到 Redis 时，你可能会发现，每次提取、转换、导入一条数据是让人难以忍受的慢!这里有一个技巧，你可以通过使用管道把 MySQL 的输出直接输入到 redis-cli输入端，这可以使两个数据库都能以他们的最顶级速度来运行。

t_loan_risk_program (658.1461G),9999204295 条 MySQL 数据导入到 Redis 的时间从 12 小时缩短到了30分钟。

 

MySQL数据表结构：

CREATE TABLE `t_loan_risk_program` (

  `id` bigint(20) NOT NULL AUTO_INCREMENT,

  `call_histories` longtext,

  `contacts` longtext,

  `create_time` datetime DEFAULT NULL,

  `current_version` varchar(32) CHARACTER SET utf8 DEFAULT NULL,

  `customer_id` bigint(20) DEFAULT NULL,

  `device` varchar(10) CHARACTER SET utf8 DEFAULT NULL,

  `device_info` longtext CHARACTER SET utf8,

  `gps` text CHARACTER SET utf8,

  `id_card_correct` bit(1) DEFAULT NULL,

  `imei` varchar(64) CHARACTER SET utf8 DEFAULT NULL,

  `in_black_list` bit(1) DEFAULT NULL,

  `jailbreak_status` bit(1) NOT NULL,

  `like_screenshot_score` double DEFAULT NULL,

  `live_score` double DEFAULT NULL,

  `loan_id` bigint(20) DEFAULT NULL,

  `messages` longtext,

  `packages` longtext,

  `simulator_status` bit(1) DEFAULT NULL,

  `zhima_black_list` bit(1) DEFAULT NULL,

  `zhima_score` varchar(15) CHARACTER SET utf8 DEFAULT NULL,

  PRIMARY KEY (`id`),

  KEY `cusIdAndloanId` (`customer_id`,`loan_id`) USING BTREE,

  KEY `cusId` (`customer_id`) USING BTREE,

  KEY `loanId` (`loan_id`) USING BTREE

) ENGINE=InnoDB AUTO_INCREMENT=8289585 DEFAULT CHARSET=utf8mb4 ；

Redis存储结构：

    HSET events_all_time [action] [count]

下面是重点，能过下面SQL语句将MySQL输出直接变更成redis-cli可接收的格式：

root@storm-master-01:/home#more /home/t_loan_rask_programredis.sql

 SELECT CONCAT(

 "*4\r\n",

 '$', LENGTH(redis_cmd), '\r\n',

 redis_cmd, '\r\n',

 '$', LENGTH(redis_key), '\r\n',

 redis_key, '\r\n',

 '$', LENGTH(hkey), '\r\n',

 hkey, '\r\n',

 '$', LENGTH(hval), '\r\n',

 hval, '\r'

)

FROM (

 SELECT

 'HSET' as redis_cmd,

 't_loan_risk_program ' AS redis_key,

 id AS hkey,

 call_histories AS hval,

 contacts as co,

 create_time as crtim,

 current_version as cuver,

customer_id as cuid,

device as dev,

device_info as devinfo,

gps as gps,

id_card_correct as idcc,

imei as im,

in_black_list as inbli,

jailbreak_status as ja,

like_screenshot_score as lik,

live_score as li,

loan_id as loid,

messages as mess,

packages as pak,

simulator_status as sim,

zhima_black_list as zh,

zhima_score as zhsc

FROM t_loan_risk_program

) AS ttt

ok, 用下面的命令执行：

mysql -h192.168.1.247 -uloan -ploan  loan -P3309 --skip-column-names --raw  < /home/t_loan_rask_programredis.sql | redis-cli -h 192.168.1.247 --pipe

　很重要的mysql参数说明：

　　--raw: 使mysql不转换字段值中的换行符。

　　--skip-column-names: 使mysql输出的每行中不包含列名。