HDFS Balancer（翻译）

官方文档：http://hadoop.apache.org/docs/r2.5.2/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html#Balancer

Balancer

HDFS data might not always be be placed uniformly across the DataNode. One common reason is addition of new DataNodes to an existing cluster. While placing new blocks (data for a file is stored as a series of blocks), NameNode considers various parameters before choosing the DataNodes to receive these blocks. Some of the considerations are:

HDFS存储在DataNode上的数据并不总是分布均匀的。一个常见的原因就是在集群中添加了新的DataNode节点。在存储新的数据块的时候，NameNode会考虑以下因素来选择具体使用哪些DataNode来存储新的数据块。

Policy to keep one of the replicas of a block on the same node as the node that is writing the block.

一份副本存放在正在写入的节点上。

Need to spread different replicas of a block across the racks so that cluster can survive loss of whole rack.

一份副本存放在不同机架上的节点上。

One of the replicas is usually placed on the same rack as the node writing to the file so that cross-rack network I/O is reduced.

一份副本存放在与正在写入的节点相同机架上的另一个节点上。

Spread HDFS data uniformly across the DataNodes in the cluster.

让HDFS数据均匀的分布在集群的DataNode节点中。

Due to multiple competing considerations, data might not be uniformly placed across the DataNodes. HDFS provides a tool for administrators that analyzes block placement and rebalanaces data across the DataNode. A brief administrator’s guide for balancer is available at HADOOP-1652.

由于一些因素，数据可能不会均匀的分布在DataNodes节点上，HDFS提供了一个管理员命令来重新分布和平衡DataNode上的数据块。下面会介绍HADOOP-1652中balance遵循的原则。

HADOOP-1652：https://issues.apache.org/jira/browse/HADOOP-1652

HADOOP-1652

When a new data node joins hdfs cluster, it does not hold much data. So any map task assigned to the machine most likely does not read local data, thus increasing the use of network bandwidth. On the other hand, when some data nodes become full, new data blocks are placed on only non-full data nodes, thus reducing their read parallelism.

当集群中添加了新的DataNode节点时，上面并没有存放数据，所以很多map任务执行的时候就不符合数据本地化，会增加很多网络传输数据的开销。另一方面，一些数据节点存储空间不足的时候，新的数据库就会存放到存储空间空闲的节点上，由此减少了并行读。

This jira aims to find an approach to redistribute data blocks when imbalance occurs in the cluster. An solution should meet the following requirements:

当数据分布不均衡时，Jira使用如下方式重新分布DataNode上的数据块，在重新分布的时候会遵循以下几点：

It maintains data availablility guranteens in the sense that rebalancing does not reduce the number of replicas that a block has or the number of racks that the block resides.

需要保证数据的可用性，不减少块的副本数和同一个机架中数据块数量（不会跨机架移动数据块）。

An adminstrator should be able to invoke and interrupt rebalancing from a command line.

管理员可以通过命令行的方式启用或者终止rebalancing。

Rebalancing should be throttled so that rebalancing does not cause a namenode to be too busy to serve any incoming request or saturate the network.

rebalacing的时候不会占用太多资源以免对namenode对外服务产生影响。

balancer使用方式

$ hdfs balancer //命令行

或

$ start-balancer.sh //脚本

可以使用的参数如下：

Usage: java Balancer
[-policy <policy>]      the balancing policy: datanode or blockpool
datanode (default): Cluster is balanced if each datanode is balanced.
blockpool: Cluster is balanced if each block pool in each datanode is balanced.
[-threshold <threshold>]        Percentage of disk capacity（默认10.0）
[-exclude [-f <hosts-file> | comma-sperated list of hosts]]     Excludes the specified datanodes.
[-include [-f <hosts-file> | comma-sperated list of hosts]]     Includes only the specified datanodes.

在使用balancer的时候默认带宽是有限制的（默认1MB），可以使用如下命令提高balancer可用带宽：

$ hdfs dfsadmin [-setBalancerBandwidth <bandwidth in bytes per second>]

该参数在hdfs-site.xml中默认配置如下：

<property>
<name>dfs.datanode.balance.bandwidthPerSec</name>
<value>1048576</value>
<description>
Specifies the maximum amount of bandwidth that each datanode
can utilize for the balancing purpose in term of
the number of bytes per second.
</description>
</property>