Prometheus和Openshift

  1. prometheus.yml配置中绑定alertmanager服务
1
2
3
4
5
6
7
......
alerting:
  alertmanagers:
  - scheme: http
    static_configs:
    - targets:
      - "localhost:9093"
  1. prometheus.rules设置prometheus告警规则
1
2
3
4
5
6
7
8
9
10
...
rules:
- alert: TooManyPods
  expr: kuberlet_running_pod_count > 10
  for: 2m
  labels:
    team: node
  annotations:
    summary: "{{$labels.instance}}: has {{$value}} pods"
    description: "{{$labels.instance}} be cateful"

规则：kuberlet_running_pod_count指标持续2分钟超过10，则发出告警。
3. AlertManager中设置告警接收器alertmanager.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
global:
  smtp_smarthost: 'mail.xx.com:25'
  smtp_from: 'service@xx.com'
  smtp_auth_username: 'service@xx.com'
  smtp_auth_password: 'password'
  smtp_require_tls: false
route:
  group_by: ['alertname']
  receiver: alert-email
receivers:
- name: alert-email
  email_configs:
  - to: 'pan@xx.com'
    send_resolved: true