（二）Kafka动态增加Topic的副本(Replication)

（二）Kafka动态增加Topic的副本(Replication)

1. 查看topic的原来的副本分布

[hadoop@sdf-nimbus-perf ~]$ le-kafka-topics.sh --describe --topic http_zhixin_line1
Topic:http_zhixin_line1 PartitionCount:3    ReplicationFactor:1 Configs:
Topic: http_zhixin_line1    Partition: 0    Leader: 7   Replicas: 7 Isr: 7
Topic: http_zhixin_line1    Partition: 1    Leader: 8   Replicas: 8 Isr: 8
Topic: http_zhixin_line1    Partition: 2    Leader: 9   Replicas: 9 Isr: 9

2. 增加Topic的副本的json文件的编写

vi addReplicas.json

{
"version": 1,
"partitions": [
{
"topic": "http_zhixin_line1",
"partition": 0,
"replicas": [
7,
1,
2
]
},
{
"topic": "http_zhixin_line1",
"partition": 1,
"replicas": [
8,
2,
3
]
},
{
"topic": "http_zhixin_line1",
"partition": 2,
"replicas": [
9,
3,
4
]
}
]
}

3. 执行topic增加副本操作

kafka-reassign-partitions.sh --zookeeper $ZK_CONNECT --reassignment-json-file addReplicas.json --execute

[hadoop@sdf-nimbus-perf topic_reassgin]$ kafka-reassign-partitions.sh --zookeeper $ZK_CONNECT --reassignment-json-file addReplicas.json --execute
Current partition replica assignment

{"version":1,"partitions":[{"topic":"http_zhixin_line1","partition":2,"replicas":[9]},{"topic":"http_zhixin_line1","partition":1,"replicas":[8]},{"topic":"http_zhixin_line1","partition":0,"replicas":[7]}]}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions {"version":1,"partitions":[{"topic":"http_zhixin_line1","partition":0,"replicas":[7,1,2]},{"topic":"http_zhixin_line1","partition":1,"replicas":[8,2,3]},{"topic":"http_zhixin_line1","partition":2,"replicas":[9,3,4]}]}

4. 查看执行的状态

kafka-reassign-partitions.sh --zookeeper  $ZK_CONNECT --reassignment-json-file addReplicas.json --verify

[hadoop@sdf-nimbus-perf topic_reassgin]$ kafka-reassign-partitions.sh --zookeeper $ZK_CONNECT --reassignment-json-file addReplicas.json --verify
Status of partition reassignment:
Reassignment of partition [http_zhixin_line1,0] completed successfully
Reassignment of partition [http_zhixin_line1,1] completed successfully
Reassignment of partition [http_zhixin_line1,2] completed successfully

5. 观察日志目录的数据同步情况，生产者，消费者的影响。

观察partitions-0的数据同步情况，由于partitions-0 增加的两个副本为1,2；

登陆 broker 1 和 broker 2所在的服务器；

cd 副本数据所在的目录，观察日志增长情况：

ls /data/hadoop/data*/kafka/log

borker1 和 broker2 所在日志目录都有日志文件生成。borker1 和 borker2的副本数据是同时同步完成的。

观察生产者：没有影响。

观察消费者：没有影响。