问题：
在prometheus中看不到nginx入口指标
戎洛城
我试图让prometheus监视我的入口nginx部署工作。我可以在prometheus用户界面中查看默认指标，但与nginx无关。
- 我已经通过集群监控部署了prometheus-运算符（同时更新serviceMonitorSelector和serviceMonitorNamespaceSelector值）：https://github.com/carlosedp/cluster-monitoring
- 我已经通过官方helm图表部署了ingres-nginx：https://github.com/kubernetes/ingress-nginx/tree/master/charts/ingress-nginx
kubectl获得普罗米修斯-A
...
runAsUser: 1000
serviceAccountName: prometheus-k8s
serviceMonitorNamespaceSelector:
matchLabels:
release: prometheus-operator
serviceMonitorSelector:
matchLabels:
release: prometheus-operator
version: v2.19.1
...
kubectl获取ns nginx
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: "2021-02-08T02:37:35Z"
labels:
release: prometheus-operator
managedFields:
- apiVersion: v1
...
kubectl get servicemonitor nginx ingres nginx controller-n monitoring-o yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
annotations:
meta.helm.sh/release-name: nginx
meta.helm.sh/release-namespace: nginx
creationTimestamp: "2021-02-14T19:09:01Z"
generation: 5
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/version: 0.44.0
helm.sh/chart: ingress-nginx-3.23.0
release: prometheus-operator
managedFields:
...
spec:
endpoints:
- interval: 30s
port: metrics
namespaceSelector:
any: true
selector:
matchLabels:
release: prometheus-operator
kubectl get svc nginx ingres nginx controller metrics-n nginx-o yaml
apiVersion: v1
kind: Service
metadata:
annotations:
meta.helm.sh/release-name: nginx
meta.helm.sh/release-namespace: nginx
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
creationTimestamp: "2021-02-11T03:58:38Z"
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/version: 0.44.0
helm.sh/chart: ingress-nginx-3.23.0
release: prometheus-operator
...
spec:
clusterIP: 10.43.90.125
clusterIPs:
- 10.43.90.125
ports:
- name: metrics
port: 9913
protocol: TCP
targetPort: metrics
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: nginx
app.kubernetes.io/name: ingress-nginx
sessionAffinity: None
type: ClusterIP
2017年2月更新：kubectl get prometheus-n monitoring-o yaml的完整输出
➜ ~ kubectl get -n monitoring prometheus -o yaml
apiVersion: v1
items:
- apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"monitoring.coreos.com/v1","kind":"Prometheus","metadata":{"annotations":{},"labels":{"prometheus":"k8s"},"name":"k8s","namespace":"monitoring"},"spec":{"affinity":{"podAntiAffinity":{"preferredDuringSchedulingIgnoredDuringExecution":[{"podAffinityTerm":{"labelSelector":{"matchExpressions":[{"key":"prometheus","operator":"In","values":["k8s"]}]},"namespaces":["monitoring"],"topologyKey":"kubernetes.io/hostname"},"weight":100}]}},"alerting":{"alertmanagers":[{"name":"alertmanager-main","namespace":"monitoring","port":"web"}]},"externalUrl":"http://prometheus.192.168.1.15.nip.io","image":"prom/prometheus:v2.19.1","nodeSelector":{"kubernetes.io/os":"linux"},"podMonitorNamespaceSelector":{},"podMonitorSelector":{},"replicas":1,"resources":{"requests":{"memory":"400Mi"}},"retention":"15d","ruleSelector":{"matchLabels":{"prometheus":"k8s","role":"alert-rules"}},"scrapeInterval":"30s","scrapeTimeout":"30s","securityContext":{"fsGroup":2000,"runAsNonRoot":true,"runAsUser":1000},"serviceAccountName":"prometheus-k8s","serviceMonitorNamespaceSelector":{},"serviceMonitorSelector":{"matchLabels":{"release":"prometheus-operator"}},"version":"v2.19.1"}}
creationTimestamp: "2021-02-13T04:13:49Z"
generation: 4
labels:
prometheus: k8s
managedFields:
- apiVersion: monitoring.coreos.com/v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.: {}
f:kubectl.kubernetes.io/last-applied-configuration: {}
f:labels:
.: {}
f:prometheus: {}
f:spec:
.: {}
f:affinity:
.: {}
f:podAntiAffinity:
.: {}
f:preferredDuringSchedulingIgnoredDuringExecution: {}
f:alerting:
.: {}
f:alertmanagers: {}
f:externalUrl: {}
f:image: {}
f:nodeSelector:
.: {}
f:kubernetes.io/os: {}
f:podMonitorNamespaceSelector: {}
f:podMonitorSelector: {}
f:replicas: {}
f:resources:
.: {}
f:requests:
.: {}
f:memory: {}
f:retention: {}
f:ruleSelector:
.: {}
f:matchLabels:
.: {}
f:prometheus: {}
f:role: {}
f:scrapeInterval: {}
f:scrapeTimeout: {}
f:securityContext:
.: {}
f:fsGroup: {}
f:runAsNonRoot: {}
f:runAsUser: {}
f:serviceAccountName: {}
f:serviceMonitorNamespaceSelector: {}
f:serviceMonitorSelector:
.: {}
f:matchLabels:
.: {}
f:release: {}
f:version: {}
manager: kubectl-client-side-apply
operation: Update
time: "2021-02-15T15:07:05Z"
name: k8s
namespace: monitoring
resourceVersion: "1321004"
uid: a9829d00-2765-416b-b068-2c1e57623832
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchExpressions:
- key: prometheus
operator: In
values:
- k8s
namespaces:
- monitoring
topologyKey: kubernetes.io/hostname
weight: 100
alerting:
alertmanagers:
- name: alertmanager-main
namespace: monitoring
port: web
externalUrl: http://prometheus.192.168.1.15.nip.io
image: prom/prometheus:v2.19.1
nodeSelector:
kubernetes.io/os: linux
podMonitorNamespaceSelector: {}
podMonitorSelector: {}
replicas: 1
resources:
requests:
memory: 400Mi
retention: 15d
ruleSelector:
matchLabels:
prometheus: k8s
role: alert-rules
scrapeInterval: 30s
scrapeTimeout: 30s
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 1000
serviceAccountName: prometheus-k8s
serviceMonitorNamespaceSelector: {}
serviceMonitorSelector:
matchLabels:
release: prometheus-operator
version: v2.19.1
kind: List
metadata:
resourceVersion: ""
selfLink: ""
我还尝试过让监控与其他部署一起工作，但没有用（所以我显然做错了什么），但这是我的prometheus日志：
➜ ~ kubectl logs prometheus-k8s-0 -n monitoring -c prometheus --tail 25
level=error ts=2021-02-17T15:29:18.588Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:361: Failed to list *v1.Endpoints: endpoints is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"endpoints\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:29:25.978Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:385: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" in the namespace \"mqtt\""
level=error ts=2021-02-17T15:29:43.562Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:363: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:29:46.748Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:362: Failed to list *v1.Service: services is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"services\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:29:56.841Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:385: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" in the namespace \"pihole\""
level=error ts=2021-02-17T15:30:05.943Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:361: Failed to list *v1.Endpoints: endpoints is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"endpoints\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:30:14.735Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:385: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" in the namespace \"mqtt\""
level=error ts=2021-02-17T15:30:18.489Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:362: Failed to list *v1.Service: services is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"services\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:30:28.171Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:385: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" in the namespace \"pihole\""
level=error ts=2021-02-17T15:30:30.412Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:363: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:30:57.878Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:361: Failed to list *v1.Endpoints: endpoints is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"endpoints\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:30:59.128Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:385: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" in the namespace \"mqtt\""
level=error ts=2021-02-17T15:31:15.689Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:362: Failed to list *v1.Service: services is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"services\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:31:21.136Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:363: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:31:27.498Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:385: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" in the namespace \"pihole\""
level=error ts=2021-02-17T15:31:36.739Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:385: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" in the namespace \"mqtt\""
level=error ts=2021-02-17T15:31:54.742Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:361: Failed to list *v1.Endpoints: endpoints is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"endpoints\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:32:03.270Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:362: Failed to list *v1.Service: services is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"services\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:32:13.738Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:363: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:32:20.258Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:385: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" in the namespace \"mqtt\""
level=error ts=2021-02-17T15:32:24.315Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:385: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" in the namespace \"pihole\""
level=error ts=2021-02-17T15:32:44.099Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:361: Failed to list *v1.Endpoints: endpoints is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"endpoints\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:32:58.299Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:362: Failed to list *v1.Service: services is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"services\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:33:03.925Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:363: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" at the cluster scope"
level=error ts=2021-02-17T15:33:05.050Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:385: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"pods\" in API group \"\" in the namespace \"mqtt\""
共有1个答案
武成和
经过大量的研究，我面临的问题涉及集群角色的定义。普罗米修斯日志表明我试图监视的许多名称空间存在权限问题，因此我不得不向我的集群角色添加权限，以包括pods、服务和endpoint等资源，并包括list和watch等动词：
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: prometheus-k8s
rules:
- apiGroups:
- ""
resources:
- nodes/metrics
- nodes
- services
- endpoints
- pods
verbs:
- get
- list
- watch
- nonResourceURLs:
- /metrics
verbs:
- get
类似资料：
-
来自服务器的错误（InternalError）：创建“stdin”时出错：发生内部错误：调用webhook“validate.nginx.ingress.kubernetes.io”失败：Post https：//ingress-nginx-controller-crission.ingress-nginx.svc：443/extensions/v1beta1/ingress？timeout=30
-
我用演示应用程序创建了简单的kubernetes集群。在创建集群时，我安装了prometheus stack和带有helm（默认值文件）的nginx ingress控制器。 集群设置好后，我创建Inete对象以公开prometheus、grafana和alertManager： 当我试图通过
-
我想通过Spring Boot应用程序中的Pushgateway将指标（例如HTTP请求的处理时间或当前堆大小）推送到Prometheus。 我很挣扎，因为有很多文档，要么来自Spring Metrics，要么来自Prometheus，但它们都没有解决我的具体问题。 最简单的方法是在RestController类上使用定时注释，如下所述：http://projects.spring.io/spri
-
我想监控Kafka指标，但不幸的是，在/exactor/prometheus endpoint下没有与Kafka相关的内容。我的设置有什么遗漏吗？ 我的接收器看起来像： 听众：
-
我创建了一个部署、一个服务和一个入口，以便能够从我的主机访问NGINX web服务器，但我一直没有找到404。经过长时间的故障排除，我已经到了一个非常欢迎帮助的地步。 步骤和相关yaml文件如下所示。 启用Minikube NGINX入口控制器 minikube插件支持入口 创建NGINX web服务器部署 创建ClusterIP服务以管理对POD的访问 创建入口以从集群外部访问服务 测验 连接到
-
问题内容： 我是第一次为jqGrid工作。我想在我的应用程序中使用它的所有本机功能，例如编辑，删除，填充，排序和所有其他功能。 我遇到了这个项目，它似乎是jqGrid的基本实现。这是一个很好的起点。但是，我在左寻呼机中显示“添加”，“删除”，“搜索”按钮时遇到问题。 我尝试将寻呼机设置为true，将其设置为div-id。尝试设置导航网格选项。尝试将.navGrid函数绑定到指令中的pager元素。