elasticsearch-curator: es索引生命周期（关闭、删除索引）

1，下载安装

• 参考：https://www.elastic.co/guide/en/elasticsearch/client/curator/current/installation.html

2, 配置使用

配置

• action.yaml 加上 ignore_empty_list: True, 防止条件不满足时报错，导致任务不能继续执行
• action.yaml 配置参考：
  action-close模板：https://www.elastic.co/guide/en/elasticsearch/client/curator/current/ex_close.html

索引名匹配-正则参考：https://www.elastic.co/guide/en/elasticsearch/client/curator/current/filtertype_pattern.html

TEST1:/root # cat curator-close.yml
---
actions:
  1:
    action: close
    description: >-
      关闭指定索引，5天前的：test_logstash_
    options:
      skip_flush: False
      delete_aliases: False
      ignore_sync_failures: True
      ignore_empty_list: True
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      #value: test_logstash_2023_11_12
      value: test_logstash_
    - filtertype: age
      source: name
      direction: older
      #timestring: '%Y.%m.%d'
      timestring: '%Y_%m_%d'
      unit: days
      unit_count: 5

  2:
    action: close
    description: >-
      关闭指定索引，10天前的：^(\w+)_logstash_.*$
    options:
      skip_flush: False
      delete_aliases: False
      ignore_sync_failures: True
      ignore_empty_list: True
      disable_action: False
    filters:
    - filtertype: pattern
      kind: regex
      #value: logstash-
      #value: test_logstash_2023_11_12
      value: '^(\w+)_logstash_.*$'
    - filtertype: age
      source: name
      direction: older
      #timestring: '%Y.%m.%d'
      timestring: '%Y_%m_%d'
      unit: days
      unit_count: 10


TEST1:/root # cat curator/curatorConfig.yaml 
client:
  hosts:
    - 185.86.0.106
    - 185.86.0.107
    - 185.86.0.108
  port: 9200
  url_prefix:
  use_ssl: False
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: False
  username: elastic
  password: elk123456
  timeout: 60
  master_only: False

logging:
  loglevel: INFO
  logfile: /applog/elk/curator/curator.log
  logformat: default
  blacklist: ['elasticsearch', 'urllib3']

使用

• curator --config 全局配置文件 action动作配置文件

TEST1:/root # curator --config curator/curatorConfig.yaml  curator-close.yml  


TEST1:/root #  tail  /applog/elk/curator/curator.log -f
2024-01-02 11:19:02,339 INFO      Preparing Action ID: 1, "close"
2024-01-02 11:19:02,339 INFO      Creating client object and testing connection
2024-01-02 11:19:02,341 INFO      Instantiating client object
2024-01-02 11:19:02,342 INFO      Testing client connectivity
2024-01-02 11:19:02,345 INFO      Successfully created Elasticsearch client object with provided settings
2024-01-02 11:19:02,348 INFO      Trying Action ID: 1, "close": 关闭指定索引，5天前的：test_logstash_
2024-01-02 11:19:04,088 INFO      Skipping action "close" due to empty list: <class 'curator.exceptions.NoIndices'>
2024-01-02 11:19:04,089 INFO      Action ID: 1, "close" completed.

2024-01-02 11:19:04,089 INFO      Preparing Action ID: 2, "close"
2024-01-02 11:19:04,089 INFO      Creating client object and testing connection
2024-01-02 11:19:04,089 INFO      Instantiating client object
2024-01-02 11:19:04,090 INFO      Testing client connectivity
2024-01-02 11:19:04,095 INFO      Successfully created Elasticsearch client object with provided settings
2024-01-02 11:19:04,097 INFO      Trying Action ID: 2, "close": 关闭指定索引，10天前的：^(\w+)_logstash_.*$
2024-01-02 11:19:04,610 INFO      Closing 30 selected indices: [ 'test_net_logstash_2023_12_15', 'metric_net_logstash_2023_12_16', 'mytest_logstash_2023_12_15',... ]
2024-01-02 11:19:06,240 INFO      Action ID: 2, "close" completed.

2024-01-02 11:19:06,240 INFO      Job completed.