Elasticsearch通过elasticsearch-curator 插件来定期删除Index
2020-06-15 14:48
86 查看
Elasticsearch管理中索引的管理非常重要。基于磁盘空间和性能的考量,索引的生命周期管理显得尤为重要。Curator允许对索引创建、删除等操作,下面是我们借助 elasticsearch-curator 插件来定期删除index.本文主要介绍elasticsearch-curator 插件的安装步骤。
Step 1下载
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/yum-repository.html
Step 2 上传安装包至服务器,并创建相应的文件
Step 3 安装
执行的安装命令如下:
yum localinstall elastics 1044 earch-curator-5.8.1-1.x86_64.rpm
Step 4 完善更新配置文件
config.yml文件
详细内容如下:
--- # Remember, leave a key empty if there is no value. None will be a string, # not a Python "NoneType" client: hosts: - 201.101.101.101 port: 端口 url_prefix: use_ssl: False certificate: client_cert: client_key: ssl_no_validate: False http_auth: 用户名:密码 timeout: 30 master_only: False logging: loglevel: INFO logfile: /data/elasticsearch-curator/log/curator.log logformat: default blacklist: ['elasticsearch', 'urllib3']
action.yml 文件的编辑
详细内容如下:
--- # Remember, leave a key empty if there is no value. None will be a string, # not a Python "NoneType" # # Also remember that all examples have 'disable_action' set to True. If you # want to use this action as a template, be sure to set this to False after # copying it. actions: 1: action: delete_indices description: >- Delete metric indices older than 3 days (based on index name), for .monitoring-es-6- .monitoring-kibana-6- .monitoring-logstash-6- .watcher-history-3- prefixed indices. Ignore the error if the filter does not result in an 1f2b actionable list of indices (ignore_empty_list) and exit cleanly. options: ignore_empty_list: True # disable_action: True filters: - filtertype: pattern kind: regex value: '^mysql-.*$' - filtertype: age source: name direction: older timestring: '%Y.%m.%d' unit: days unit_count: 3 2: action: close description: >- Close indices older than 30 days (based on index name), for syslog- prefixed indices. options: ignore_empty_list: True delete_aliases: False # disable_action: True filters: - filtertype: pattern kind: prefix value: syslog- - filtertype: age source: name direction: older timestring: '%Y.%m.%d' unit: days unit_count: 30 3: action: forcemerge description: >- forceMerge syslog- prefixed indices older than 2 days (based on index creation_date) to 2 segments per shard. Delay 120 seconds between each forceMerge operation to allow the cluster to quiesce. Skip indices that have already been forcemerged to the minimum number of segments to avoid reprocessing. options: ignore_empty_list: True max_num_segments: 2 delay: 120 timeout_override: continue_if_exception: False filters: - filtertype: pattern kind: prefix value: syslog- exclude: - filtertype: age source: name direction: older timestring: '%Y.%m.%d' unit: days unit_count: 2 - filtertype: forcemerged max_num_segments: 2 exclude:
Step 5 使用crontab定期执行curator
设置定时任务,例如以下的crontab
6 0 * * * curator --config /data/elasticsearch-curator/config.yml /data/elasticsearch-curator/action.yml
相关文章推荐
- 定期删除elasticsearch的过期index
- 定期删除elasticsearch 的index 索引
- 通过Rman定期删除归档脚本
- Elasticsearch的数据导出和导入操作(elasticdump工具),以及删除指定type的数据(delete-by-query插件)
- elasticsearch 学习博客系列<四> ES 中 index-doc 的 删除(java)
- Elasticsearch 5.6.3 通过script添加、删除数组元素
- 通过指定数组里面的val值来删除元素-------在无法通过index的情况下
- 定期删除elasticsearch集群45天以上的索引
- [通过制定SQL计划和系统计划实现]SQL Server 2005数据库定期备份和删除历史备份记录
- 关于wordpress插件通过ftp删除
- Elasticsearch:用Curator辅助Marvel,实现自动删除旧marvel索引
- elasticsearch删除index和数据
- 定时删除elasticsearch的index
- elasticsearch:用Curator辅助Marvel,实现自动删除marvel索引
- Script:创建一个job,通过存储过程定期删除数据的脚本
- 定期删除elasticsearch集群10天以上的索引
- elasticsearch 5.x删除index/type
- 通过制定SQL计划和系统计划实现]SQL Server 2005数据库定期备份和删除历史备份记录
- Elasticsearch:用Curator辅助Marvel,实现自动删除旧marvel索引
- 通过shell脚本定期删除linux用户下的日志