log4j + flume 1.6 集成

1. 安装flume 1.6；

安装前提：java 1.6 或者更高版本（推荐java 1.7）、建议linux环境（windows 环境中提供的bat缺少如zookeeper和logger级别配置的功能）

开始安装：

a. 下载flume 1.6

b. 解压flume

c. 在flume下的conf文件夹中添加文件：test-log4j.conf

## file name : test-log4j.conf
a1.sources = r1
a1.sinks = k1
a1.channels = c1

# Describe/configure the source
a1.sources.r1.type = avro
a1.sources.r1.bind = 192.168.3.152
a1.sources.r1.port = 4444

# Describe the sink
a1.sinks.k1.type = file_roll
a1.sinks.k1.sink.directory = /home/user/work-log/work-flume
a1.sinks.k1.sink.rollInterval=86400
a1.sinks.k1.sink.batchSize=100
a1.sinks.k1.sink.serializer=text
a1.sinks.k1.sink.serializer.appendNewline = false

# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 1000

# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1

d. 启动flume:

[user@localhost apache-flume-1.6.0-bin]$ bin/flume-ng agent -c conf/ -f conf/test-log4j.conf  -n a1 -Dflume.root.logger=INFO,console

2. pom.xml添加：

<dependency>
<groupId>org.apache.flume.flume-ng-clients</groupId>
<artifactId>flume-ng-log4jappender</artifactId>
<version>1.6.0</version>
</dependency>

3. log4j.properties内容：

log4j.rootLogger=INFO,stdout,flume

#### 输出到控制台 ###
log4j.appender.stdout = org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Threshold=INFO
log4j.appender.stdout.Target = System.out
log4j.appender.stdout.layout = org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern = %-d{yyyy-MM-dd HH:mm:ss} %5p %c{1}:%L - %m%n

#flume output
log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.Threshold=INFO
log4j.appender.flume.Hostname = 192.168.3.152
log4j.appender.flume.Port = 4444
log4j.appender.flume.UnsafeMode = true
log4j.appender.flume.layout=org.apache.log4j.PatternLayout
log4j.appender.flume.layout.ConversionPattern= %-d{yyyy-MM-dd HH:mm:ss} %5p %c{1}:%L - %m%n

4. 配置完成