[日志处理工作之四]从flume采集的event中提取能被kibana识别的时间戳 and 对比flume与logstash
2015-06-29 16:58
696 查看
1.extract timestamp field from events
use org.apache.flume.interceptor.RegexExtractorInterceptorMillisSerializer as the i1 interceptor's serializer
agent.sources.source1.interceptors.i1.type=regex_extractor
agent.sources.source1.interceptors.i1.regex =(\\d\\d\\d\\d-\\d\\d-\\d\\d-\\d\\d.\\d\\d.\\d\\d).\\d\\d\\d\\d\\d\\d[+]\\d\\d\\d\\s+
agent.sources.source1.interceptors.i1.serializers=s1
agent.sources.source1.interceptors.i1.serializers.s1.name=timestamp
agent.sources.source1.interceptors.i1.serializers.s1.type=org.apache.flume.interceptor.RegexExtractorInterceptorMillisSerializer
agent.sources.source1.interceptors.i1.serializers.s1.pattern=yyyy-MM-dd-HH.mm.ss
2. the comparation betwwen flume and logstash refer to Links document
贴上文档的截图吧
use org.apache.flume.interceptor.RegexExtractorInterceptorMillisSerializer as the i1 interceptor's serializer
agent.sources.source1.interceptors.i1.type=regex_extractor
agent.sources.source1.interceptors.i1.regex =(\\d\\d\\d\\d-\\d\\d-\\d\\d-\\d\\d.\\d\\d.\\d\\d).\\d\\d\\d\\d\\d\\d[+]\\d\\d\\d\\s+
agent.sources.source1.interceptors.i1.serializers=s1
agent.sources.source1.interceptors.i1.serializers.s1.name=timestamp
agent.sources.source1.interceptors.i1.serializers.s1.type=org.apache.flume.interceptor.RegexExtractorInterceptorMillisSerializer
agent.sources.source1.interceptors.i1.serializers.s1.pattern=yyyy-MM-dd-HH.mm.ss
2. the comparation betwwen flume and logstash refer to Links document
贴上文档的截图吧
相关文章推荐
- Flume环境部署和配置详解及案例大全
- Play! Akka Flume实现的完整数据收集
- flume自定义Interceptor
- #Note# Analyzing Twitter Data with Apache Hadoo...
- flume、kafka、storm常用命令
- logstash
- 用Kibana和logstash快速搭建实时日志查询、收集与分析系
- Flume向HDFS写数据实例
- flume+log4j整合到web项目
- 详细图解 Flume介绍、安装配置-1
- flume NG 中文 Welcome to Apache Flume 第一页 醉了
- flume 高可用性 高可靠性 agent source
- flume介绍及扩展开发心得
- Flume
- ElasticSearch的学习资源
- flume-两台机器上agent的串联运行
- Java调用WMI读取Windows EventLog (j-Interop库)
- log4j+flume+kafka管理日志,查询日志
- logstash开源日志管理系统-2-logstash配置语言介绍
- flume用户手册