安装elasticsearch-curator
黄啸
安装pip
[root 180-9 ~]$wget https://bootstrap.pypa.io/get-pip.py
--2020-12-26 15:39:41-- https://bootstrap.pypa.io/get-pip.py
Resolving bootstrap.pypa.io (bootstrap.pypa.io)... 151.101.228.175, 2a04:4e42:36::175
Connecting to bootstrap.pypa.io (bootstrap.pypa.io)|151.101.228.175|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1886796 (1.8M) [text/x-python]
Saving to: ‘get-pip.py’
100%[=======================================================================================================================================================================================================>] 1,886,796 7.85KB/s in 3m 19s
2020-12-26 15:43:05 (9.24 KB/s) - ‘get-pip.py’ saved [1886796/1886796]
[root 180-9 ~]$python get-pip.py
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Defaulting to user installation because normal site-packages is not writeable
Collecting pip
Downloading pip-20.3.3-py2.py3-none-any.whl (1.5 MB)
|████████████████████████████████| 1.5 MB 14 kB/s
Collecting wheel
Downloading wheel-0.36.2-py2.py3-none-any.whl (35 kB)
Installing collected packages: pip, wheel
Successfully installed pip-20.3.3 wheel-0.36.2
[root 180-9 ~]$pip --version
pip 20.3.3 from /home/service/.local/lib/python2.7/site-packages/pip (python 2.7)
安装elasticsearch-curator
-i https://mirrors.aliyun.com/pypi/simple/ 因为访问国外速度太慢，使用国内阿里仓库提高速度
[root 180-9 ~]$pip install elasticsearch-curator -i https://mirrors.aliyun.com/pypi/simple/
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Collecting elasticsearch-curator
Downloading https://mirrors.aliyun.com/pypi/packages/60/bd/c6fe71d50dca46e143d4f5ebb4f3cb048a0b8a73569d4388461cbe56446c/elasticsearch_curator-5.8.3-py2.py3-none-any.whl (106 kB)
|████████████████████████████████| 106 kB 6.0 MB/s
Collecting six>=1.11.0
Downloading https://mirrors.aliyun.com/pypi/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting boto3>=1.9.142
Downloading https://mirrors.aliyun.com/pypi/packages/55/c6/912cc2cfd1b4051621552fc5961c25e2f517a090d179a38f62d5cdaf5d37/boto3-1.16.43-py2.py3-none-any.whl (130 kB)
|████████████████████████████████| 130 kB 1.2 MB/s
Collecting certifi>=2019.9.11
Downloading https://mirrors.aliyun.com/pypi/packages/5e/a0/5f06e1e1d463903cf0c0eebeb751791119ed7a4b3737fdc9a77f1cdfb51f/certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
|████████████████████████████████| 147 kB 63.6 MB/s
Collecting click<7.0,>=6.7
Downloading https://mirrors.aliyun.com/pypi/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl (71 kB)
|████████████████████████████████| 71 kB 6.1 MB/s
Collecting elasticsearch==7.1.0
Downloading https://mirrors.aliyun.com/pypi/packages/86/3c/047e4985f81af98b71f19e318a6207187987bcd8af73b1edd4470cdee76b/elasticsearch-7.1.0-py2.py3-none-any.whl (83 kB)
|████████████████████████████████| 83 kB 2.2 MB/s
Collecting requests-aws4auth>=0.9
Downloading https://mirrors.aliyun.com/pypi/packages/4e/44/dec6791ef89cef395918f6e4268d0f30096ee798523fcf26edd53117210a/requests_aws4auth-1.0.1-py2.py3-none-any.whl (29 kB)
Collecting voluptuous>=0.9.3
Downloading https://mirrors.aliyun.com/pypi/packages/17/0d/6e1e3ab6653720903efd7f644cc5be839d59ceb396143c08c1bebeab080b/voluptuous-0.12.1.tar.gz (47 kB)
|████████████████████████████████| 47 kB 2.8 MB/s
Collecting pyyaml==3.13
Downloading https://mirrors.aliyun.com/pypi/packages/9e/a3/1d13970c3f36777c583f136c136f804d70f500168edc1edea6daa7200769/PyYAML-3.13.tar.gz (270 kB)
|████████████████████████████████| 270 kB 876 kB/s
Collecting urllib3<1.26,>=1.24.2
Downloading https://mirrors.aliyun.com/pypi/packages/56/aa/4ef5aa67a9a62505db124a5cb5262332d1d4153462eb8fd89c9fa41e5d92/urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
|████████████████████████████████| 127 kB 58.5 MB/s
Collecting requests>=2.20.0
Downloading https://mirrors.aliyun.com/pypi/packages/29/c1/24814557f1d22c56d50280771a17307e6bf87b70727d975fd6b2ce6b014a/requests-2.25.1-py2.py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 12.3 MB/s
Collecting botocore<1.20.0,>=1.19.43
Downloading https://mirrors.aliyun.com/pypi/packages/ef/6c/9f6e6a14f53b21b6f1670ccd789082015911458823914b7dabca333ae033/botocore-1.19.43-py2.py3-none-any.whl (7.2 MB)
|████████████████████████████████| 7.2 MB 1.6 MB/s
Collecting s3transfer<0.4.0,>=0.3.0
Downloading https://mirrors.aliyun.com/pypi/packages/69/79/e6afb3d8b0b4e96cefbdc690f741d7dd24547ff1f94240c997a26fa908d3/s3transfer-0.3.3-py2.py3-none-any.whl (69 kB)
|████████████████████████████████| 69 kB 5.6 MB/s
Collecting jmespath<1.0.0,>=0.7.1
Downloading https://mirrors.aliyun.com/pypi/packages/07/cb/5f001272b6faeb23c1c9e0acc04d48eaaf5c862c17709d20e3469c6e0139/jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting idna<3,>=2.5
Downloading https://mirrors.aliyun.com/pypi/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl (58 kB)
|████████████████████████████████| 58 kB 10.9 MB/s
Collecting chardet<5,>=3.0.2
Downloading https://mirrors.aliyun.com/pypi/packages/19/c7/fa589626997dd07bd87d9269342ccb74b1720384a4d739a1872bd84fbe68/chardet-4.0.0-py2.py3-none-any.whl (178 kB)
|████████████████████████████████| 178 kB 67.1 MB/s
Collecting python-dateutil<3.0.0,>=2.1
Downloading https://mirrors.aliyun.com/pypi/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
|████████████████████████████████| 227 kB 64.1 MB/s
Requirement already satisfied: futures<4.0.0,>=2.2.0; python_version == "2.7" in /usr/lib/python2.7/site-packages (from s3transfer<0.4.0,>=0.3.0->boto3>=1.9.142->elasticsearch-curator) (3.1.1)
Building wheels for collected packages: voluptuous, pyyaml
Building wheel for voluptuous (setup.py) ... done
Created wheel for voluptuous: filename=voluptuous-0.12.1-py2-none-any.whl size=29510 sha256=9422978bf7cd77f2ea2204a1669e2c3ae1356228b6e83f0072c093883790e568
Stored in directory: /home/service/.cache/pip/wheels/55/ab/4f/da1e50ee97eabd6f4bd80882d6a4cb706aba138e5d6dd5d669
Building wheel for pyyaml (setup.py) ... done
Created wheel for pyyaml: filename=PyYAML-3.13-cp27-cp27mu-linux_x86_64.whl size=43811 sha256=c43bfd1898dce455460d0fe20087cc43a3032b0bf1f2ae2ee64283c134f6a443
Stored in directory: /home/service/.cache/pip/wheels/25/11/db/544634e0157b7cfad08a6c5dd7d5fca3e27d92ebb542fc6797
Successfully built voluptuous pyyaml
Installing collected packages: six, urllib3, python-dateutil, jmespath, botocore, s3transfer, boto3, certifi, click, elasticsearch, idna, chardet, requests, requests-aws4auth, voluptuous, pyyaml, elasticsearch-curator
Successfully installed boto3-1.16.43 botocore-1.19.43 certifi-2020.12.5 chardet-4.0.0 click-6.7 elasticsearch-7.1.0 elasticsearch-curator-5.8.3 idna-2.10 jmespath-0.10.0 python-dateutil-2.8.1 pyyaml-3.13 requests-2.25.1 requests-aws4auth-1.0.1 s3transfer-0.3.3 six-1.15.0 urllib3-1.25.11 voluptuous-0.12.1
如果安装过程出现以下错误。
ERROR: No matching distribution found for botocore<1.18.0,>=1.17.57 (from boto3>=1.9.142->elasticsearch-curator)
在pip仓库里面没找到匹配的botocore版本。先独自安装botocore。然后再重新安装elasticsearch-curator 时跳过这个依赖
[root 180-9 ~]# pip install elasticsearch-curator --ignore-installed botocore -i https://mirrors.aliyun.com/pypi/simple/
查看是否成功及版本
[root 180-9 ~]# curator --version
curator, version 5.8.3
config.yml
# Remember,leave a key empty if there is no value. None will be a string,
# # not a Python "NoneType"
client:
hosts:
- 192.0.0.1
port: 9200
url_prefix:
use_ssl: False
certificate:
client_cert:
client_key:
ssl_no_validate: False
http_auth:
timeout: 30
master_only: False
logging:
loglevel: INFO
logfile: /opt/service/elasticsearch-curator/log/curator.log
logformat: default
blacklist: ['elasticsearch', 'urllib3']
action.yml
# Remember, leave a key empty if there is no value. None will be a string,
# not a Python "NoneType"
# #
# # Also remember that all examples have 'disable_action' set to True. If you
# # want to use this action as a template, be sure to set this to False after
# # copying it.
actions:
1:
action: delete_indices
# description: "删除过期索引"
description: "delete_expire_index"
# description: "删除过期索引"
description: "delete_expire_index"
options:
# action开关，为True表示不执行这个action，默认False
disable_action: False
# 如果为True，则在filters空列表时，继续下一个action处理而不是退出程序。
ignore_empty_list: True
# 发现错误后，继续执行下一个索引操作，默认False
continue_if_exception: True
filters:
# 是否排除隐藏索引，如.kibana
- filtertype: kibana
exclude: True
# 是否排除open状态的索引
- filtertype: opened
exclude: False
#前缀匹配logstash-es-
- filtertype: pattern
kind: prefix
value: logstash-es-
exclude: False
# 处理30天前的索引
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 30
exclude: False
运行curator,--dry-run该参数只是测试，并不实际操作
curator --config /opt/service/elasticsearch-curator/config/config.yml --dry-run /opt/service/elasticsearch-curator/config/action.yml
借助crontab周期性执行
借助crontab，每天零点10分执行
$ crontab -e
再添加如下命令
10 0 * * * /home/opt/.local/bin/curator --config /opt/service/elasticsearch-curator/config/config.yml /opt/service/elasticsearch-curator/config/action.yml
类似资料：