prometheus-operator项目给prometheus页面添加鉴权

一、环境准备

组件	版本
kube-prometheus-stack	kube-prometheus-stack-39.6.0
prometheus-operator	prometheus-operator:v0.58.0
prometheus	prometheus:v2.37.0
alertmanager	alertmanager:v0.24.0
grafana	grafana:9.0.5
node-explorter	node-exporter:v1.3.1
kube-state-metrics	kube-state-metrics:v2.5.0

二、安装部署

2.1 部署k8s集群

略

2.2 安装helm工具

[root@master1 helm]# wget https://get.helm.sh/helm-v3.5.4-linux-amd64.tar.gz
[root@master1 helm]# tar xvf helm-v3.5.4-linux-amd64.tar.gz
[root@master1 helm]# cp linux-amd64/helm /usr/bin/

检查是否安装成功

[root@k8s-master]-[~]-#helm version
version.BuildInfo{Version:"v3.5.4", GitCommit:"1b5edb69df3d3a08df77c9902dc17af864ff05d1", GitTreeState:"clean", GoVersion:"go1.15.11"}
[root@k8s-master]-[~]-#

2.3 部署kube-prometheus-stack

2.3.1 添加helm repo

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

2.3.2 下载chart包

helm pull prometheus-community/kube-prometheus-stack

2.4.3 创建一个新的ns

kubectl create ns monitoring

2.4.4 安装chart

helm install  kube-prometheus-stack -n monitoring ./kube-prometheus-stack

2.4.5 检查所有对象资源运行正常

[root@k8s-master]-[~]-#kubectl get all -n monitoring 
NAME                                                            READY   STATUS    RESTARTS   AGE
pod/alertmanager-kube-prometheus-stack-alertmanager-0           2/2     Running   0          137m
pod/kube-prometheus-stack-grafana-6ddfb54796-h4tqg              3/3     Running   0          139m
pod/kube-prometheus-stack-kube-state-metrics-677d866f69-t5frl   1/1     Running   0          139m
pod/kube-prometheus-stack-operator-748857655d-5ckqx             1/1     Running   0          139m
pod/kube-prometheus-stack-prometheus-node-exporter-9d7b6        1/1     Running   0          139m
pod/kube-prometheus-stack-prometheus-node-exporter-dz2qs        1/1     Running   0          139m
pod/kube-prometheus-stack-prometheus-node-exporter-k6nxw        1/1     Running   0          139m
pod/prometheus-kube-prometheus-stack-prometheus-0               2/2     Running   0          37m

NAME                                                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
service/alertmanager-operated                            ClusterIP   None            <none>        9093/TCP,9094/TCP,9094/UDP   137m
service/kube-prometheus-stack-alertmanager               NodePort    10.96.235.164   <none>        9093:30987/TCP               139m
service/kube-prometheus-stack-grafana                    ClusterIP   10.96.233.113   <none>        80/TCP                       139m
service/kube-prometheus-stack-kube-state-metrics         ClusterIP   10.96.76.27     <none>        8080/TCP                     139m
service/kube-prometheus-stack-operator                   ClusterIP   10.96.254.251   <none>        443/TCP                      139m
service/kube-prometheus-stack-prometheus                 NodePort    10.96.71.39     <none>        9090:30815/TCP               139m
service/kube-prometheus-stack-prometheus-node-exporter   ClusterIP   10.96.81.210    <none>        9100/TCP                     139m
service/prometheus-operated                              ClusterIP   None            <none>        9090/TCP                     137m

NAME                                                            DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/kube-prometheus-stack-prometheus-node-exporter   3         3         3       3            3           <none>          139m

NAME                                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kube-prometheus-stack-grafana              1/1     1            1           139m
deployment.apps/kube-prometheus-stack-kube-state-metrics   1/1     1            1           139m
deployment.apps/kube-prometheus-stack-operator             1/1     1            1           139m

NAME                                                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/kube-prometheus-stack-grafana-6ddfb54796              1         1         1       139m
replicaset.apps/kube-prometheus-stack-kube-state-metrics-677d866f69   1         1         1       139m
replicaset.apps/kube-prometheus-stack-operator-748857655d             1         1         1       139m

NAME                                                               READY   AGE
statefulset.apps/alertmanager-kube-prometheus-stack-alertmanager   1/1     137m
statefulset.apps/prometheus-kube-prometheus-stack-prometheus       1/1     137m

PS：有个别镜像pull不到，更换镜像地址即可

三、创建secret

3.1 使用bcrypt加密算法加密

参考：HTTPS and authentication | Prometheus

密码生成脚本如下：

import bcrypt

passwd = b'admin1234'

# start 加密
salt = bcrypt.gensalt()
hashed = bcrypt.hashpw(passwd, salt)

print(salt)
print(hashed)
# end 加密


# start 验证
print(bcrypt.checkpw(passwd, hashed))
# end 验证

生成密码配置文件：

[root@k8s-master]-[~]-#cat secret.txt 
basic_auth_users:
  admin: $2b$12$QkmXyjJlNsCI3HzMC.Srve6Dy0BClhWbeQirp7WGOrFXywd0Sr2Dm

base64加密：

cat secret.txt|base64 -w 0
YmFzaWNfYXV0aF91c2VyczoKICBhZG1pbjogJDJiJDEyJFFrbVh5akpsTnNDSTNIek1DLlNydmU2RHkwQkNsaFdiZVFpcnA3V0dPckZYeXdkMFNyMkRtCg==

3.2 创建secret对象

apiVersion: v1
data:
  web.yaml: YmFzaWNfYXV0aF91c2VyczoKICBhZG1pbjogJDJiJDEyJFFrbVh5akpsTnNDSTNIek1DLlNydmU2RHkwQkNsaFdiZVFpcnA3V0dPckZYeXdkMFNyMkRtCg==
kind: Secret
metadata:
  annotations:
    meta.helm.sh/release-name: kube-prometheus-stack
    meta.helm.sh/release-namespace: monitoring
  labels:
    app: kube-prometheus-stack-prometheus
    app.kubernetes.io/component: prometheus
    app.kubernetes.io/instance: kube-prometheus-stack
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/part-of: kube-prometheus-stack
    app.kubernetes.io/version: 39.6.0
    chart: kube-prometheus-stack-39.6.0
    heritage: Helm
    release: kube-prometheus-stack
  name: prometheus-basic-auth
  namespace: monitoring
type: Opaque

四、编辑prometheus对象资源

4.1 修改prometheus

[root@k8s-master]-[~]-#
[root@k8s-master]-[~]-#kubectl get prometheus -A
NAMESPACE    NAME                               VERSION   REPLICAS   AGE
monitoring   kube-prometheus-stack-prometheus   v2.37.0   1          17h

[root@k8s-master]-[~]-#kubectl edit prometheus -nmonitoring   kube-prometheus-stack-prometheus
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  annotations:
    meta.helm.sh/release-name: kube-prometheus-stack
    meta.helm.sh/release-namespace: monitoring
  creationTimestamp: "2022-08-15T07:31:20Z"
  generation: 13
  labels:
    app: kube-prometheus-stack-prometheus
    app.kubernetes.io/instance: kube-prometheus-stack
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/part-of: kube-prometheus-stack
    app.kubernetes.io/version: 39.6.0
    chart: kube-prometheus-stack-39.6.0
    heritage: Helm
    release: kube-prometheus-stack
  name: kube-prometheus-stack-prometheus
  namespace: monitoring
  resourceVersion: "208703"
  selfLink: /apis/monitoring.coreos.com/v1/namespaces/monitoring/prometheuses/kube-prometheus-stack-prometheus
  uid: 45d42fa1-b2a7-44a1-809d-f1e3ada94250
spec:
  alerting:
    alertmanagers:
    - apiVersion: v2
      name: kube-prometheus-stack-alertmanager
      namespace: monitoring
      pathPrefix: /
      port: http-web
  containers:
  - args:
    - --web.console.templates=/etc/prometheus/consoles
    - --web.console.libraries=/etc/prometheus/console_libraries
    - --storage.tsdb.retention.time=10d
    - --config.file=/etc/prometheus/config_out/prometheus.env.yaml
    - --storage.tsdb.path=/prometheus
    - --web.enable-lifecycle
    - --web.external-url=http://kube-prometheus-stack-prometheus.monitoring:9090
    - --web.route-prefix=/
    - --web.config.file=/etc/prometheus/secrets/prometheus-basic-auth/web.yaml  #修改Prometheus默认指定路径
    livenessProbe:
      failureThreshold: 6
      httpGet:
        httpHeaders:
        - name: Authorization
          value: Basic YWRtaW46YWRtaW4xMjM0   
        path: /-/healthy
        port: http-web
        scheme: HTTP
      periodSeconds: 5
      successThreshold: 1
      timeoutSeconds: 3
    name: prometheus
    readinessProbe:
      httpGet:
        httpHeaders:
        - name: Authorization
          value: Basic YWRtaW46YWRtaW4xMjM0
        path: /-/ready
        port: http-web
        scheme: HTTP
      periodSeconds: 5
      successThreshold: 1
      timeoutSeconds: 3
  enableAdminAPI: false
  evaluationInterval: 30s
  externalUrl: http://kube-prometheus-stack-prometheus.monitoring:9090
  image: quay.io/prometheus/prometheus:v2.37.0
  listenLocal: false
  logFormat: logfmt
  logLevel: info
  paused: false
  podMonitorNamespaceSelector: {}
  podMonitorSelector:
    matchLabels:
      release: kube-prometheus-stack
  portName: http-web
  probeNamespaceSelector: {}
  probeSelector:
    matchLabels:
      release: kube-prometheus-stack
  replicas: 1
  retention: 10d
  routePrefix: /
  ruleNamespaceSelector: {}
  ruleSelector:
    matchLabels:
      release: kube-prometheus-stack
  scrapeInterval: 30s
  secrets:
  - prometheus-basic-auth               #将配置好的secret挂载到prometheus容器中
  securityContext:
    fsGroup: 2000
    runAsGroup: 2000
    runAsNonRoot: true
    runAsUser: 1000
  serviceAccountName: kube-prometheus-stack-prometheus
  serviceMonitorNamespaceSelector: {}
  serviceMonitorSelector:
    matchLabels:
      release: kube-prometheus-stack
  shards: 1
  version: v2.37.0
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2022-08-15T09:13:15Z"
    status: "True"
    type: Available
  - lastTransitionTime: "2022-08-15T07:33:22Z"
    status: "True"
    type: Reconciled
  paused: false
  replicas: 1
  shardStatuses:
  - availableReplicas: 1
    replicas: 1
    shardID: "0"
    unavailableReplicas: 0
    updatedReplicas: 1
  unavailableReplicas: 0
  updatedReplicas: 1

就绪探针和存活探针一定要加认证，否则容器运行异常