k8s安全03--云安全工具 kube-bench & OPA

1 介绍

本文基于 k8s安全02–云安全工具与安全运行时 继续介绍几个
常用的安全工具，包括 kube-bench 和 OPA(Open Policy Agent)。

2 安全工具

2.1 kube-bench

kube-bench 努力像 Center for Internet Security, Inc. (CIS®), CIS 一样测试一些相同的问题, 能够帮助我们快速查找和发现集群的漏洞。
以下为其安装和使用方法：

1 安装：

# curl -L https://github.com/aquasecurity/kube-bench/releases/download/v0.3.1/kube-bench_0.3.1_linux_amd64.deb -o kube-bench_0.3.1_linux_amd64.deb
# curl -L https://bit.ly/32BQF8G -o kube-bench_0.3.1_linux_amd64.deb
# apt install ./kube-bench_0.3.1_linux_amd64.deb
# kube-bench version
0.3.1

2 使用：
直接执行 kube-bench即可输出集群的安全统计信息和建议，如下所示

#  kube-bench
[INFO] 1 Master Node Security Configuration
[INFO] 1.1 Master Node Configuration Files
[PASS] 1.1.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)
[PASS] 1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Scored)
[PASS] 1.1.3 Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Scored)
[PASS] 1.1.4 Ensure that the controller manager pod specification file ownership is set to root:root (Scored)
[PASS] 1.1.5 Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Scored)
[PASS] 1.1.6 Ensure that the scheduler pod specification file ownership is set to root:root (Scored)
[PASS] 1.1.7 Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Scored)
[PASS] 1.1.8 Ensure that the etcd pod specification file ownership is set to root:root (Scored)
[WARN] 1.1.9 Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Not Scored)
[WARN] 1.1.10 Ensure that the Container Network Interface file ownership is set to root:root (Not Scored)
[PASS] 1.1.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Scored)
[FAIL] 1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)
[PASS] 1.1.13 Ensure that the admin.conf file permissions are set to 644 or more restrictive (Scored)
[PASS] 1.1.14 Ensure that the admin.conf file ownership is set to root:root (Scored)
......
PASS] 2.7 Ensure that a unique Certificate Authority is used for etcd (Not Scored)

== Summary ==
7 checks PASS
0 checks FAIL
0 checks WARN
0 checks INFO
[INFO] 3 Control Plane Configuration
[INFO] 3.1 Authentication and Authorization
[WARN] 3.1.1 Client certificate authentication should not be used for users (Not Scored)
[INFO] 3.2 Logging
[WARN] 3.2.1 Ensure that a minimal audit policy is created (Scored)
[WARN] 3.2.2 Ensure that the audit policy covers key security concerns (Not Scored)

== Remediations ==
3.1.1 Alternative mechanisms provided by Kubernetes such as the use of OIDC should be
implemented in place of client certificates.

3.2.1 Create an audit policy file for your cluster.

3.2.2 Consider modification of the audit policy in use on the cluster to include these items, at a
minimum.

== Summary ==
0 checks PASS
0 checks FAIL
3 checks WARN
0 checks INFO
[INFO] 4 Worker Node Security Configuration
......

3 使用 job 运行kube-bench:
该方式将kubernetes 相关的目录挂载到pod中，然后通过kube-bench来检测集群部署是否安全

1. 下载 https://github.com/aquasecurity/kube-bench/blob/main/job.yaml
2.  kubec apply -f job.yaml
3. kubectl logs kube-bench-xxxxxx

2.2 OPA Gatekeeper

1 安裝 gatekeeper

1. 下載 https://github.com/open-policy-agent/gatekeeper/blob/master/deploy/gatekeeper.yaml
2. kubectl apply -f gatekeeper.yaml
3. 查看 gatekeeper
$ kubectl -n gatekeeper-system get deploy
NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
gatekeeper-audit                1/1     1            1           21s
gatekeeper-controller-manager   3/3     3            3           21s

2 限制ns

$ kubectl create -f gk-ns-constraintTemplate.yaml
constrainttemplate.templates.gatekeeper.sh/k8srequiredlabels created
$ kubectl create -f gk-ns-constraint.yaml 
k8srequiredlabels.constraints.gatekeeper.sh/ns-require-label created

$ kubectl create ns nslabel
Error from server ([ns-require-label] You must provide labels: {"gk-ns"}): admission webhook "validation.gatekeeper.sh" denied the request: [ns-require-label] You must provide labels: {"gk-ns"}

$ kubectl apply -f ns-label.yaml 
namespace/nslabel created

yaml

# vim gk-ns-constraintTemplate.yaml
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
      validation:
        openAPIV3Schema:
          properties:
            labels:
              type: array
              items: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels

        violation[{"msg": msg, "details": {"missing_labels": missing}}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_]}
          missing := required - provided
          count(missing) > 0
          msg := sprintf("You must provide labels: %v", [missing])
        }

---
# vim gk-ns-constraint.yaml 
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: ns-require-label
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Namespace"]
  parameters:
    labels: ["gk-ns"]

---
# vim ns-label
apiVersion: v1
kind: Namespace
metadata:
  creationTimestamp: null
  name: nslabel
  labels:
    gk-ns: "yes"
spec: {}

3 限制image

$ kubectl create -f gk-image-constraintTemplate.yaml 
constrainttemplate.templates.gatekeeper.sh/k8srequiredregistry created
$ kubectl create -f gk-image-constraint.yaml 
k8srequiredregistry.constraints.gatekeeper.sh/only-quay-images created

$ kubectl create deployment nginx --image=nginx:1.19.6
deployment.apps/nginx created
$ kubectl describe rs nginx-8548df5d4c 
Name:           nginx-8548df5d4c
Namespace:      default
...
Events:
  Type     Reason        Age                 From                   Message
  ----     ------        ----                ----                   -------
  Warning  FailedCreate  17s (x13 over 38s)  replicaset-controller  Error creating: admission webhook "validation.gatekeeper.sh" denied the request: [only-quay-images] Forbidden registry: nginx:1.19.6

$ kubectl create deployment busybox --image=busybox:1.31 -- sleep infinity
deployment.apps/busybox created

yaml

# vim gk-image-constraintTemplate.yaml 
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8srequiredregistry
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredRegistry
      validation:
        openAPIV3Schema:
          properties:
            image:
              type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredregistry
        violation[{"msg": msg, "details": {"Registry must be": required}}] {
          input.review.object.kind == "Pod"
          some i
          image := input.review.object.spec.containers[i].image
          required := input.parameters.registry
          not startswith(image,required)
          msg := sprintf("Forbidden registry: %v", [image])
        }

---
# vim gk-image-constraint.yaml 
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredRegistry
metadata:
  name: only-quay-images
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
  parameters:
    registry: "busybox"

3 注意事项

1. K8sRequiredRegistry 配置registry 的時候，配置後就會導致對應的命名空間只能使用該類型的鏡像。

4 说明

github.com/aquasecurity/kube-bench
github.com/open-policy-agent/gatekeeper