apache flume 配置存储在Linux本地服务器
2016-06-07 17:16
537 查看
在Linux上flume的配置:
程序的log4j.properties的配置:
项目maven的pom.xml 添加配置:
参考资料:http://neoremind.com/2016/05/Flume+Kafka%E6%94%B6%E9%9B%86Docker%E5%AE%B9%E5%99%A8%E5%86%85%E5%88%86%E5%B8%83%E5%BC%8F%E6%97%A5%E5%BF%97%E5%BA%94%E7%94%A8%E5%AE%9E%E8%B7%B5/
agent.channels = ch-1 agent.sources = src-1 agent.sinks = sink-1 agent.sources.src-1.interceptors = i1 agent.sources.src-1.interceptors.i1.type = timestamp agent.channels.ch-1.type = memory agent.channels.ch-1.capacity = 10000000 agent.channels.ch-1.transactionCapacity = 1000 #agent.sinks.sink-1.type = file_roll #agent.sinks.sink-1.sink.directory=/opt/log/data #agent.sinks.sink-1.sink.rollInterval = 0 #agent.sinks.sink-1.sink.batchSize=100 #agent.sinks.sink-1.sink.serializer=text #agent.sinks.sink-1.sink.serializer.appendNewline = false agent.sinks.sink-1.type = com.baidu.unbiz.flume.sink.RollingByTypeAndDayFileSink agent.sinks.sink-1.sink.directory=/opt/log/data agent.sinks.sink-1.channel = ch-1 agent.sinks.sink-1.sink.serializer.appendNewline = false agent.sources.src-1.type = avro agent.sources.src-1.bind = 192.168.1.130 agent.sources.src-1.port = 41414 agent.sinks.sink-1.channel = ch-1 agent.sources.src-1.channels= ch-1
程序的log4j.properties的配置:
log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender log4j.appender.flume.Hostname = 192.168.1.130 log4j.appender.flume.Port = 41414 log4j.appender.flume.UnsafeMode = true log4j.appender.flume.layout=org.apache.log4j.PatternLayout log4j.appender.flume.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %p [%c:%L] - %m%n # configure a class's logger to output to the flume appender log4j.logger.com.cloudera.flume.flumelog4jtest.LogTestApp = INFO,flume
项目maven的pom.xml 添加配置:
<dependency> <artifactId>log4j</artifactId> <groupId>log4j</groupId> <version>1.2.16</version> </dependency> <dependency> <groupId>org.apache.flume.flume-ng-clients</groupId> <artifactId>flume-ng-log4jappender</artifactId> <version>1.6.0</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-jdk14</artifactId> <version>1.7.2</version> </dependency>
参考资料:http://neoremind.com/2016/05/Flume+Kafka%E6%94%B6%E9%9B%86Docker%E5%AE%B9%E5%99%A8%E5%86%85%E5%88%86%E5%B8%83%E5%BC%8F%E6%97%A5%E5%BF%97%E5%BA%94%E7%94%A8%E5%AE%9E%E8%B7%B5/
相关文章推荐
- Apache Log4j 2.0-rc1 发布
- Awstats处理多apache日志
- 典型入侵日志分析
- dedecms采集过滤常用代码集合
- MS SQL Server数据库清理错误日志的方法
- 对MySQL日志操作的一些基本命令总结
- 简介操作MySQL日志的一些方法
- MySQL的日志基础知识及基本操作学习教程
- MSSQL 2005/2008 日志压缩清理方法小结
- SQL Server误区30日谈 第19天 Truncate表的操作不会被记录到日志
- C#3.0使用EventLog类写Windows事件日志的方法
- Sqlserver2005日志文件太大如何减小
- ORA-00392ORA-00312日志正在清除故障
- MySQL中查询日志与慢查询日志的基本学习教程
- mysql binlog二进制日志详解
- C#实现线程安全的简易日志记录方法
- C#实现将程序运行信息写入日志的方法
- ORACLE DATAGUARD中手工处理日志v$archive_GAP的方法
- 某个aspx页面突然死了连日志也没有的解决方法
- php采集速度探究总结(原创)