如何为库伯内特斯中的Flink应用程序自定义资源提供保险库机密

Question

我想为在库伯内特斯集群中运行的Apache Flink作业提供来自Hashicorp Vault的机密。这些信用将用于访问检查点和保存点的状态后端。状态后端可能是例如Minio S3存储。在以下设置下，有人能为Flink Application操作员提供一个工作示例吗？

用户名和密码（或访问密钥）的Vault机密：

vault kv put vvp/storage/config username=user password=secret
vault kv put vvp/storage/config access-key=minio secret-key=minio123

Flink应用程序自定义资源的k8s清单：

apiVersion: flink.k8s.io/v1beta1
kind: FlinkApplication
metadata:
  name: processor
  namespace: default
spec:
  image: stream-processor:0.1.0
  deleteMode: None

  template:
    metadata:
      annotations:
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/role: vvp-flink-job
        vault.hashicorp.com/agent-inject-secret-storage-config.txt: vvp/data/storage/config

  flinkConfig:
    taskmanager.memory.flink.size: 1024mb
    taskmanager.heap.size: 200
    taskmanager.network.memory.fraction: 0.1
    taskmanager.network.memory.min: 10mb
    web.upload.dir: /opt/flink
  jobManagerConfig:
    resources:
      requests:
        memory: "1280Mi"
        cpu: "0.1"
    replicas: 1
  taskManagerConfig:
    taskSlots: 2
    resources:
      requests:
        memory: "1280Mi"
        cpu: "0.1"
  flinkVersion: "1.14.2"
  jarName: "stream-processor-1.0-SNAPSHOT.jar"
  parallelism: 3
  entryClass: "org.StreamingJob"
  programArgs: >
    --name value

flink应用程序的Docker文件：

FROM maven:3.8.4-jdk-11 AS build
ARG revision

WORKDIR /
COPY    src /src
COPY    pom.xml /

RUN mvn -B -Drevision=${revision} package

# runtime
FROM flink:1.14.2-scala_2.12-java11

ENV FLINK_HOME=/opt/flink

ENTRYPOINT ["/docker-entrypoint.sh"]
EXPOSE 6123 8081
CMD ["help"]

flink配置。yaml包含以下示例：

# state.backend: filesystem

# Directory for checkpoints filesystem, when using any of the default bundled
# state backends.
#
# state.checkpoints.dir: hdfs://namenode-host:port/flink-checkpoints

# Default target directory for savepoints, optional.
#
# state.savepoints.dir: hdfs://namenode-host:port/flink-savepoints

最终目标是替换硬编码的机密或从vault中以某种方式设置它们：

state.backend: filesystem
s3.endpoint: http://minio:9000
s3.path.style.access: true
s3.access-key: minio
s3.secret-key: minio123

非常感谢。

Answer 1

设置vault变量后

可以在展开中添加注释，以将vault中的变量导出到展开中

annotations:
        vault.hashicorp.com/agent-image: <Agent image>
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/agent-inject-secret-secrets: kv/<Path-of-secret>
        vault.hashicorp.com/agent-inject-template-secrets: |2

          {{- with secret "kv/<Path-of-secret>" -}}

          #!/bin/sh
          set -e

          {{- range $key, $value := .Data.data }}
          export {{ $key }}={{ $value }}
          {{- end }}

          exec "$@"
          {{- end }}
        vault.hashicorp.com/auth-path: auth/<K8s cluster for auth>
        vault.hashicorp.com/role: app

这将在POD中创建文件。

当应用程序运行时，它应该首先执行此文件，环境变量将被注入POD。

因此，vault annotation将创建一个与您获取的txt文件相同的文件，但我们将按照如下方式执行

{{- range $key, $value := .Data.data }}
       export {{ $key }}={{ $value }}
{{- end }}

它将在密钥之前保留并注入导出

将此文件保存在reop中，并将其添加到Docker中/bin/runapp

#!/bin/bash
if [ -f '/vault/secrets/secrets' ]; then
  source '/vault/secrets/secrets'
fi
node <path-insnide-docker>/index.js #Sorry dont know scala or Java

package.json

"start": "./bin/runapp",

文档文件

ADD ./bin/runapp ./
EXPOSE 4444
CMD ["npm", "start"]

您的vault注入文件将类似于位于vault/secrets/secrets的内部pod或您配置的路径。

#!/bin/sh
set -e
export development=false
export production=true
exec "$@"