k8s+ECK+Fluentbit+prometheus-elasticsearch-exporter+elasticsearch-curator
童华池
在部署es集群收集日志，以及用prometheus实现监控的遇到很多的坑，大概花了两个礼拜的时间，走通了，希望可以对读者的有一定的帮助。
一、部署集群k8s, 这个需要自行部署，我选择的k8s 1.16
二、部署es集群
1、添加ECK自定义资源
kubectl apply -f all-in-one.yaml (说明，我设置的命名空间为logging-system)
kubectl apply -f https://download.elastic.co/downloads/eck/1.1.2/all-in-one.yaml
如果有需求的，可以把这个文件下载下来进行命名空间的设置，部署完可以查看如下。
kubectl get all -n logging-system
pod/elastic-operator-0 1/1 Running 0 18d
service/elastic-webhook-server ClusterIP 172.30.198.157 <none> 443/TCP 18d
statefulset.apps/elastic-operator 1/1 18d
2、部署elasticsearch
存储是通过节点本地存储的方式
(1) 创建存储类
kubectl apply -f es-data-storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: es-data
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Retain
(2) 创建PV
如果/var/lib/hawkeye/esdata文件夹不存在，需要先进行创建。
apiVersion: v1
kind: PersistentVolume
metadata:
name: es-data-0
spec:
capacity:
storage: 20Gi
accessModes:
- ReadWriteOnce
storageClassName: es-data
persistentVolumeReclaimPolicy: Retain
local:
path: /var/lib/hawkeye/esdata
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- nodeName(需要修改)
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: es-data-1
spec:
capacity:
storage: 20Gi
accessModes:
- ReadWriteOnce
storageClassName: es-data
persistentVolumeReclaimPolicy: Retain
local:
path: /var/lib/hawkeye/esdata
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- nodeName(需要修改)
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: es-data-2
spec:
capacity:
storage: 20Gi
accessModes:
- ReadWriteOnce
storageClassName: es-data
persistentVolumeReclaimPolicy: Retain
local:
path: /var/lib/hawkeye/esdata
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- nodeName(需要修改)
(3) 创建PVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: elasticsearch-data-elasticsearch-es-master-nodes-0
namespace: logging-system
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
storageClassName: es-data
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: elasticsearch-data-elasticsearch-es-data-nodes-0
namespace: logging-system
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
storageClassName: es-data
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: elasticsearch-data-elasticsearch-es-data-nodes-1
namespace: logging-system
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
storageClassName: es-data
(4)创建集群密码，如果不创建，那么集群就会随机生成
apiVersion: v1
data:
elastic: yourSecret #(需要base64加密)
kind: Secret
metadata:
labels:
common.k8s.elastic.co/type: elasticsearch
eck.k8s.elastic.co/credentials: "true"
elasticsearch.k8s.elastic.co/cluster-name: elasticsearch
name: elasticsearch-es-elastic-user #名字不能改
namespace: logging-system
type: Opaque
(5)创建es集群
这里创建三个节点，两个数据节点，一个管理节点，
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: elasticsearch
namespace: logging-system
spec:
version: 7.2.0
image: docker.elastic.co/elasticsearch/elasticsearch:7.2.0
nodeSets:
- name: master-nodes
count: 1
config:
node.master: true
node.data: false
podTemplate:
metadata:
namespace: logging-system
spec:
initContainers:
- name: sysctl
securityContext:
privileged: true
command: ['sh', '-c', 'sysctl -w vm.max_map_count=262144']
#volumes:
#- name: elasticsearch-data
# emptyDir: {}
containers:
- name: elasticsearch
env:
- name: ES_JAVA_OPTS
value: -Xms1g -Xmx1g
resources:
requests:
memory: 2Gi
limits:
memory: 10Gi
volumeClaimTemplates:
- metadata:
name: elasticsearch-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
storageClassName: es-data
- name: data-nodes
count: 2
config:
node.master: false
node.data: true
podTemplate:
metadata:
namespace: logging-system
spec:
initContainers:
- name: sysctl
securityContext:
privileged: true
command: ['sh', '-c', 'sysctl -w vm.max_map_count=262144']
#volumes:
#- name: elasticsearch-data
# emptyDir: {}
containers:
- name: elasticsearch
env:
- name: ES_JAVA_OPTS
value: -Xms1g -Xmx1g
resources:
requests:
memory: 2Gi
limits:
memory: 10Gi
volumeClaimTemplates:
- metadata:
name: elasticsearch-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
storageClassName: es-data
会生成如下
kubectl get all -n logging-system
pod/elasticsearch-es-data-nodes-0 1/1 Running 0 10d
pod/elasticsearch-es-data-nodes-1 1/1 Running 0 10d
pod/elasticsearch-es-master-nodes-0 1/1 Running 0 10d
service/elasticsearch-es-data-nodes ClusterIP None <none> <none> 10d
service/elasticsearch-es-http ClusterIP 172.30.229.248 <none> 9200/TCP 10d
service/elasticsearch-es-master-nodes ClusterIP None <none> <none> 10d
kubectl get elasticsearch -n logging-system
NAME HEALTH NODES VERSION PHASE AGE
elasticsearch green 3 7.2.0 Ready 10d
可以查看集群状态
curl -u "elastic:yourpassword" -k https://172.30.229.248(是你集群中生成的IP,为elasticsearch-es-http的ClusterIP):9200
{
"name" : "elasticsearch-es-data-nodes-0",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "caWatVXyQmSNDLQ7lEO7qg",
"version" : {
"number" : "7.2.0",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "508c38a",
"build_date" : "2019-06-20T15:54:18.811730Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
获取集群密码
kubectl -n logging-system get secret elasticsearch-es-elastic-user -o=jsonpath='{.data.elastic}' | base64 --decode; echo
2、部署kibana
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
name: kibana
namespace: logging-system
spec:
version: 7.2.0
count: 1
elasticsearchRef:
name: elasticsearch
http:
tls:
selfSignedCertificate:
disabled: true
这样就可以登录到kibana中，用户名密码和es集群中的相同。
3、部署fluentbit
分别包含文件fluentbit-clusterRoleBinding.yaml、fluentbit-clusterRole.yaml、fluentbit-configmap.yaml、fluentbit-daemonset.yaml、fluentbit-serviceAccount.yaml、fluentbit-service.yaml、kustomization.yaml
文件都放在同一个文件夹下，
执行 kubectl apply -k .
fluentbit-clusterRoleBinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: fluentbit-read
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: fluentbit-read
subjects:
- kind: ServiceAccount
name: fluentbit
namespace: logging-system
fluentbit-clusterRole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: fluentbit-read
rules:
- apiGroups: [""]
resources:
- namespaces
- pods
verbs: ["get", "list", "watch"]
fluentbit-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: fluentbit-config
namespace: logging-system
data:
filter-kubernetes.conf: |
[FILTER]
Name record_modifier
Match *
Record hostname ${HOSTNAME}
fluent-bit.conf: |
[SERVICE]
# Set an interval of seconds before to flush records to a destination
Flush 5
# Instruct Fluent Bit to run in foreground or background mode.
Daemon Off
# Set the verbosity level of the service, values can be:
Log_Level info
# Specify an optional 'Parsers' configuration file
Parsers_File parsers.conf
# Plugins_File plugins.conf
# Enable/Disable the built-in Server for metrics
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_Port 2020
@INCLUDE input-kubernetes.conf
@INCLUDE filter-kubernetes.conf
@INCLUDE output-elasticsearch.conf
input-kubernetes.conf: |
[INPUT]
Name systemd
Tag host.*
Path /var/log/journal
DB /var/log/fluentbit/td.sys.pos
output-elasticsearch.conf: |
[OUTPUT]
Name es
Match kube.*
Host ${FLUENT_ELASTICSEARCH_HOST}
Port ${FLUENT_ELASTICSEARCH_PORT}
tls ${TLS_ENABLE}
tls.verify ${TLS_VERIFY}
HTTP_User ${ELASTICSEARCH_USERNAME}
HTTP_Passwd ${ELASTICSEARCH_PASSWORD}
# Replace_Dots On
Retry_Limit False
Index kube
Type kube
Buffer_Size 2M
Include_Tag_Key On
Tag_Key component
Logstash_Format On
Logstash_prefix umstor-monitor
parsers.conf: |
[PARSER]
Name apache
Format regex
Regex ^(?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>.*)")?$
Time_Key time
Time_Format %d/%b/%Y:%H:%M:%S %z
[PARSER]
Name apache2
Format regex
Regex ^(?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$
Time_Key time
Time_Format %d/%b/%Y:%H:%M:%S %z
[PARSER]
Name apache_error
Format regex
Regex ^\[[^ ]* (?<time>[^\]]*)\] \[(?<level>[^\]]*)\](?: \[pid (?<pid>[^\]]*)\])?( \[client (?<client>[^\]]*)\])? (?<message>.*)$
[PARSER]
Name nginx
Format regex
Regex ^(?<remote>[^ ]*) (?<host>[^ ]*) (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$
Time_Key time
Time_Format %d/%b/%Y:%H:%M:%S %z
[PARSER]
Name json
Format json
Time_Key time
Time_Format %d/%b/%Y:%H:%M:%S %z
[PARSER]
Name docker
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
# Command | Decoder | Field | Optional Action
# =============|==================|=================
Decode_Field_As escaped log
[PARSER]
Name docker-daemon
Format regex
Regex time="(?<time>[^ ]*)" level=(?<level>[^ ]*) msg="(?<msg>[^ ].*)"
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
[PARSER]
Name syslog-rfc5424
Format regex
Regex ^\<(?<pri>[0-9]{1,5})\>1 (?<time>[^ ]+) (?<host>[^ ]+) (?<ident>[^ ]+) (?<pid>[-0-9]+) (?<msgid>[^ ]+) (?<extradata>(\[(.*)\]|-)) (?<message>.+)$
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
[PARSER]
Name syslog-rfc3164-local
Format regex
Regex ^\<(?<pri>[0-9]+)\>(?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<ident>[a-zA-Z0-9_\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$
Time_Key time
Time_Format %b %d %H:%M:%S
Time_Keep On
[PARSER]
Name syslog-rfc3164
Format regex
Regex /^\<(?<pri>[0-9]+)\>(?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<host>[^ ]*) (?<ident>[a-zA-Z0-9_\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$/
Time_Key time
Time_Format %b %d %H:%M:%S
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
[PARSER]
Name mongodb
Format regex
Regex ^(?<time>[^ ]*)\s+(?<severity>\w)\s+(?<component>[^ ]+)\s+\[(?<context>[^\]]+)]\s+(?<message>.*?) *(?<ms>(\d+))?(:?ms)?$
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
Time_Key time
[PARSER]
# http://rubular.com/r/izM6olvshn
Name crio
Format Regex
Regex /^(?<time>.+)\b(?<stream>stdout|stderr)\b(?<log>.*)$/
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%N%:z
Time_Keep On
[PARSER]
Name kube-custom
Format regex
Regex var\.log\.containers\.(?<pod_name>[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?<namespace_name>[^_]+)_(?<container_name>.+)-(?<docker_id>[a-z0-9]{64})\.log$
[PARSER]
Name filter-kube-test
Format regex
Regex .*kubernetes.(?<pod_name>[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?<namespace_name>[^_]+)_(?<container_name>.+)-(?<docker_id>[a-z0-9]{64})\.log$
[PARSER]
# umstor for all log files
# http://rubular.com/r/IvZVElTgNl
Name umstor
Format regex
Regex ^(?<log_time>[^ ][-.\d\+:T]+[ ]*[.:\d]*)\s+(?<thread_id>\w+)\s+(?<log_level>-*\d+)\s+(?<message>.*)$
Time_Format %Y-%m-%d %H:%M:%S.%L
Time_Keep Off
Time_Key log_time
[PARSER]
# scrub for osd
Name umstor-scrub
Format regex
Regex ^(?<log_time>[^ ][-.\d\+:T]+[ ]*[.:\d]*)\s+(?<m>\w+)\s+(?<ret>-*\d+)\s+(?<message>.*)\s+(?<scrub_pg>\d+.\w+)\s+(?<scrub_status>scrub\s\w+)$
Time_Format %Y-%m-%d %H:%M:%S.%L
Time_Keep Off
Time_Key log_time
[PARSER]
# deep-scrub for osd
Name umstor-deep-scrub
Format regex
Regex ^(?<log_time>[^ ][-.\d\+:T]+[ ]*[.:\d]*)\s+(?<m>\w+)\s+(?<ret>-*\d+)\s+(?<message>.*)\s+(?<scrub_pg>\d+.\w+)\s+(?<scrub_status>deep-scrub\s\w+)$
Time_Format %Y-%m-%d %H:%M:%S.%L
Time_Keep Off
Time_Key log_time
[PARSER]
# log warning for osd, mon
Name umstor-log-warn
Format regex
Regex ^(?<log_time>[^ ][-.\d\+:T]+[ ]*[.:\d]*)\s+(?<m>\w+)\s+(?<ret>-*\d+)\s+(?<log_channel>[^ ]+)\s+\w+\s+(?<log_level>[\[WRN\]]+)\s+(?<message>.*)$
Time_Format %Y-%m-%d %H:%M:%S.%L
Time_Keep Off
Time_Key log_time
[PARSER]
# log debug for osd, mon
Name umstor-log-debug
Format regex
Regex ^(?<log_time>[^ ][-.\d\+:T]+[ ]*[.:\d]*)\s+(?<m>\w+)\s+(?<ret>-*\d+)\s+(?<log_channel>[^ ]+)\s+\w+\s+(?<log_level>[\[DBG\]]+)\s+(?<message>.*)$
Time_Format %Y-%m-%d %H:%M:%S.%L
Time_Keep Off
Time_Key log_time
fluentbit-daemonset.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluentbit
namespace: logging-system
labels:
k8s-app: fluentbit-logging
kubernetes.io/cluster-service: "true"
spec:
selector:
matchLabels:
k8s-app: fluentbit-logging
kubernetes.io/cluster-service: "true"
template:
metadata:
labels:
k8s-app: fluentbit-logging
kubernetes.io/cluster-service: "true"
annotations:
prometheus.io/path: /api/v1/metrics/prometheus
spec:
containers:
- name: fluentbit
image: registry.umstor.io:5050/vendor/fluent-bit:1.3
imagePullPolicy: IfNotPresent
ports:
- containerPort: 2020
name: http-metrics
env:
- name: FLUENT_ELASTICSEARCH_HOST
value: "elasticsearch-es-http"
- name: FLUENT_ELASTICSEARCH_PORT
value: "9200"
- name: ELASTICSEARCH_USERNAME
value: "elastic"
- name: ELASTICSEARCH_PASSWORD
value: "r00tme"
- name: TLS_ENABLE
value: "On"
- name: TLS_VERIFY
value: "Off"
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: fluentbit-config
mountPath: /fluent-bit/etc/
terminationGracePeriodSeconds: 10
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: fluentbit-config
configMap:
name: fluentbit-config
serviceAccountName: fluentbit
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
- operator: "Exists"
effect: "NoExecute"
- operator: "Exists"
effect: "NoSchedule"
fluentbit-serviceAccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: fluentbit
namespace: logging-system
fluentbit-service.yaml
apiVersion: v1
kind: Service
metadata:
labels:
k8s-app: fluentbit-logging
name: fluentbit-logging
namespace: logging-system
spec:
clusterIP: None
ports:
- name: http-metrics
port: 2020
protocol: TCP
targetPort: http-metrics
type: ClusterIP
selector:
k8s-app: fluentbit-logging
kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: logging-system
resources:
- fluentbit-clusterRoleBinding.yaml
- fluentbit-clusterRole.yaml
- fluentbit-daemonset.yaml
- fluentbit-serviceAccount.yaml
- fluentbit-service.yaml
- fluentbit-configmap.yaml
三、prometheus-elasticsearch-exporter
1、创建deployment
这里面有个坑，就是设置容器的参数，先添加用户名和密码，以及要关闭SSL验证
在这里插入apiVersion: apps/v1
kind: Deployment
metadata:
name: elasticsearch-exporter
namespace: logging-system
labels:
app: elasticsearch-exporter
spec:
replicas: 1
selector:
matchLabels:
app: elasticsearch-exporter
template:
metadata:
labels:
app: elasticsearch-exporter
spec:
containers:
- name: elasticsearch-exporter
image: registry.umstor.io:5050/vendor/elasticsearch_exporter:1.1.0
resources:
limits:
cpu: 300m
requests:
cpu: 200m
ports:
- containerPort: 9114
name: https
command:
- /bin/elasticsearch_exporter
- --es.all
- --web.telemetry-path=/_prometheus/metrics
- --es.ssl-skip-verify
- --es.uri=https://elastic:r00tme@elasticsearch-es-http:9200
securityContext:
capabilities:
drop:
- SETPCAP
- MKNOD
- AUDIT_WRITE
- CHOWN
- NET_RAW
- DAC_OVERRIDE
- FOWNER
- FSETID
- KILL
- SETGID
- SETUID
- NET_BIND_SERVICE
- SYS_CHROOT
- SETFCAP
readOnlyRootFilesystem: true
livenessProbe:
httpGet:
path: /healthz
port: 9114
initialDelaySeconds: 30
timeoutSeconds: 10
readinessProbe:
httpGet:
path: /healthz
port: 9114
initialDelaySeconds: 10
timeoutSeconds: 10代码片
2、创建es对应prometheus的serviceMonitor
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
labels:
es-app: es-exporter
name: es-client-node
namespace: logging-system
spec:
endpoints:
- interval: 30s
honorLabels: true
port: https
path: /_prometheus/metrics
namespaceSelector:
matchNames:
- logging-system
jobLabel: es-app
selector:
matchLabels:
app: elasticsearch-exporter
3、创建elasticsearch-exporter 对应的service
这也是个坑，如果不创建service ,就不会生成endpoints,
apiVersion: v1
kind: Service
metadata:
labels:
app: elasticsearch-exporter
name: elasticsearch-exporter
namespace: logging-system
spec:
ports:
- name: https
port: 9114
protocol: TCP
targetPort: 9114
selector:
app: elasticsearch-exporter
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
四、es日志的自动清理
这可能是最大的坑，因为es7.2版本不兼容任何的elasticsearch-curator,所以作者自己要开发的一个版本的elasticsearch-curator,我放在github上。
对应的文件有个四个，actions.yaml, curator-cronjob.yaml, curator.yaml, kustmoization.yaml
actions.yaml
actions:
1:
action: delete_indices
description: >-
Delete metric indices older than 21 days (based on index name), for
.monitoring-es-6-
.monitoring-kibana-6-
umstor-os-
umstor-sys-
umstor-monitor-
umstor-internal-
security-auditlog-
prefixed indices. Ignore the error if the filter does not result in an
actionable list of indices (ignore_empty_list) and exit cleanly.
options:
continue_if_exception: False
disable_action: False
ignore_empty_list: True
filters:
- filtertype: pattern
kind: regex
value: '^(\.monitoring-(es|kibana)-6-|umstor-(os|sys|internal|kube|monitor)-|security-auditlog-).*$'
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 21
2:
action: close
description: >-
Close metric indices older than 14 days (based on index name), for
.monitoring-es-6-
.monitoring-kibana-6-
umstor-os-
umstor-sys-
umstor-monitor-
umstor-internal-
security-auditlog-
prefixed indices. Ignore the error if the filter does not result in an
actionable list of indices (ignore_empty_list) and exit cleanly.
options:
continue_if_exception: True
disable_action: False
ignore_empty_list: True
filters:
- filtertype: pattern
kind: regex
value: '^(\.monitoring-(es|kibana)-6-|umstor-(os|sys|internal|kube|monitor)-|security-auditlog-).*$'
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 14
curator-cronjob.yaml
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: curator
spec:
schedule: 59 23 * * *
jobTemplate:
spec:
template:
spec:
containers:
- name: curator
image: registry.umstor.io:5050/vendor/elasticsearch-curator:v1.0.0
volumeMounts:
- mountPath: /etc/curator/
name: curator-config
readOnly: true
- mountPath: /var/log/curator
name: curator-log
restartPolicy: OnFailure
volumes:
- configMap:
name: curator-config
name: curator-config
- hostPath:
path: /var/log/curator
name: curator-log
curator.yaml
client:
hosts:
- elasticsearch-es-http
port: 9200
url_prefix:
use_ssl: True
certificate:
client_cert:
client_key:
ssl_no_validate: True
http_auth: elastic:r00tme
timeout: 30
master_only: False
logging:
loglevel: INFO
logfile: /var/log/curator/curator.log
logformat: default
blacklist: []
kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: logging-system
resources:
- curator-cronjob.yaml
generatorOptions:
disableNameSuffixHash: true
configMapGenerator:
- files:
- curator.yaml
- actions.yaml
name: curator-config
images:
- name: registry.umstor.io:5050/vendor/elasticsearch-curator
newTag: "v1.0.0"
类似资料：