ilert Documentation
WebsiteAPI ReferenceLoginStart for Free
  • Getting Started
    • Core concepts
    • FAQ
      • Renaming of Incidents to Alerts
    • Import from PagerDuty
    • Import from StatusPage.io
  • Alerting
    • Dashboard
    • Alert sources
    • Notification settings
      • Mute notifications
    • Support hours
    • Maintenance windows
    • Heartbeat monitoring
      • Prometheus Heartbeat Example
      • CLI Heartbeat Examples
    • Deployment events
    • 🏛️Understanding event flows
    • ilert sender IDs, domains and IPs
      • SMS and voice alerts in China
  • On-call management & Escalations
    • Escalation policies
    • On-call schedules
      • My on-call shifts
      • Recurring schedules
      • Static schedules
    • Coverage requests
  • ChatOps
    • Overview
    • Integration for Slack
      • Receive and respond to alerts in Slack
      • Create a dedicated Slack channel for an existing alert
      • Create alerts in Slack
      • Look up who is on-call
    • Microsoft Teams Integration
      • Microsoft Teams Chat Integration
        • Microsoft Teams Integration via Incoming Webhook
        • Microsoft Teams Integration via Workflows
      • Microsoft Teams Meeting Integration
      • Create a channel for an existing alert in Microsoft Teams
      • Create alerts in Microsoft Teams
      • Look up who is on-call in Microsoft Teams
  • Incident comms & status pages
    • Getting started
    • Services
    • Incidents
    • Status pages
      • Public vs private status pages
      • Audience-specific status page
    • Metrics
      • Import metrics from Datadog
      • Import metrics from Prometheus
  • 🪄ilert AI
    • Introduction
    • Using ilert AI for efficient incident communication
    • Using ilert AI for post-mortem creation
    • Using ilert AI for schedule generation
    • Using ilert AI for alert grouping
    • Global Search enhanced by ilert AI
      • Bulk resolving similar alerts
  • Call Routing
    • Getting started with call routing
    • Routing calls using call flows
    • Call routing (legacy)
      • Routing calls based on support hours
      • Voicemail only mode
      • Managing call routing alerts
      • Adding webhooks and outbound chat messages
      • Uploading custom audio responses
  • User Administration
    • User roles and permissions
    • Team-based organisation
    • Single sign on
      • Setting up SSO with GSuite
      • Setting up SSO with Microsoft Azure Active Directory
      • Setting up SSO with Okta
      • Setting up SSO with Auth0
      • Auto provisioning users & teams
    • 🔐Two-factor authentication / MFA
  • Reports
    • Overview
    • Alerts
  • Mobile App
    • Getting started with ilert mobile app
    • Mobile app notification settings
    • Critical push notifications and DND overrides
      • iOS critical alerts configuration
      • Android Push Notification DND Configuration
    • On-call widget
  • INTEGRATIONS
    • Types of integrations
    • Inbound integrations
      • 4me Integration
      • Ansible Automation Platform AWX Integration
      • Amazon CloudWatch Integration
      • Amazon SNS Integration
        • Amazon SNS Inbound Integration
        • Amazon SNS Outbound via AWS Lambda
      • Azure Alerts Integration
        • Azure Activity Logs
        • Azure Metric
        • Azure Logs
        • Azure Service Health
        • Azure Sentinel
        • Budget Alert
      • Apica Integration
      • AppDynamics Integration
      • AppSignal Integration
      • AWS Budgets Integration
      • AWS Cloudtrail Integration
      • AWS DevOps Guru Integration
      • AWS GuardDuty Integration
      • AWS Personal Health Dashboard Integration
      • AWS Security Hub Integration
      • Autotask Inbound Integration
      • Auvik Integration
      • Catchpoint Integration
      • Checkly Integration
      • Checkmk Integration
        • Checkmk Integration (v 1.x)
        • Checkmk Integration (v 2.0+ )
      • Cisco Meraki Integration
      • Cisco ThousandEyes Integration
      • Cisco Webex
      • Cloudflare Integration
      • ClusterControl Integration
      • Connectwise Manage Integration
      • Cortex Integration
      • Cortex XSOAR (formerly Demisto) Integration
      • CrowdStrike Integration
      • Dash0 Integration
      • Datadog Integration
      • Dynatrace Integration
      • Elastic Watcher Integration
      • Email Inbound Integration
        • Email Key Extraction and Resolve Examples
        • Automatically resolve Alerts with Emails
        • Migrating legacy email settings
      • FreshService Integration
      • Gatus Integration
      • GitHub Integration
        • GitHub Advanced Security Integration
        • GitHub Inbound Check Run (Actions) Integration
        • GitHub Inbound Issue Integration
        • GitHub advanced settings
      • GitLab Integration
      • Google Cloud Monitoring (formerly Stackdriver) Integration
      • Google Security Command Center
      • Grafana Integration Overview
        • Grafana Integration
        • Grafana Integration (v 9.x)
      • Graylog Integration
      • HaloITSM Integration
      • HaloPSA Integration
      • HashiCorp Consul
      • Healthchecks.io Integration
      • HetrixTools Integration
      • Honeybadger Integration
      • Honeycomb Integration
      • Hyperping Integration
      • CrowdStrike Falcon LogScale Integration
      • IBM Cloud Functions Integration
      • Icinga Integration
      • InfluxDB Integration
      • Instana Integration
      • IT-Conductor Integration
      • IXON Cloud Integration
      • Jira Inbound Integration
      • JumpCloud Integration
      • Kafka Integration
      • Kapacitor Integration
      • Kentix AlarmManager
      • Keep Integration
      • Kibana Integration
      • Kubernetes Integration
      • LibreNMS Integration
      • Lightstep Integration
      • Loki integration
      • Mezmo Integration
      • Microsoft SCOM
      • Mimir Integration
      • MongoDB Atlas Integration
      • MXToolBox Integration
      • MQTT Integration
      • Nagios Integration
      • N-central Integration
      • Netdata Integration
      • New Relic Integration
        • New Relic Integration (deprecated)
        • New Relic Workflow Integration
      • Oh Dear Integration
      • PandoraFMS Integration
      • Panther Integration
      • Particle Integration
      • Pingdom Integration
      • PostHog Integration
      • Postman Monitors Integration
      • Prometheus Integration
      • PRTG Network Monitor Integration
      • Prisma Cloud Integration
      • Push Notifications
      • RapidSpike Integration
      • Raygun Integration
      • Rollbar Integration
      • Salesforce Integration
      • Samsara Integration
      • Search Guard Integration
      • Sematext Integration
      • Sensu Integration
      • Sentry Integration
      • Server Density Integration
      • ServerGuard24 Integration
      • ServiceNow Inbound Integration
      • SignalFx Integration
      • Site24x7 Integration
      • SMS Integration
      • SolarWinds Integration
      • Splunk Integration
      • StatusCake Integration
      • StatusHub Integration
      • StatusPage Integration
      • Sumo Logic Integration
      • Sysdig Integration
      • TOPdesk Inbound Integration
      • TeamCity integration
      • Terraform Cloud / Terraform Enterprise
      • Tulip Integration
      • Twilio Alarms Integration
      • Twilio Errors Integration
      • Ubidots Integration
      • Uptime Kuma Integration
      • UptimeRobot Integration
      • VictoriaMetrics Integration
      • Zabbix Integration
        • Zabbix 4.4+ Integration
        • Zabbix 2.2 – 4.3 Integration
      • Zammad Inbound Integration
      • Zapier Inbound Integration
      • Zendesk Inbound Integration
    • Outbound integrations
      • Autotask Outbound Integration
      • DingTalk Integration
      • Discord Integration
      • Email Outbound Integration
      • Jira Outbound Integration
      • GitHub Outbound Issue Integration
      • Mattermost Integration
      • ServiceNow Outbound Integration
      • Telegram Integration
      • TOPdesk Outbound Integration
      • Webhook Integration
      • Zammad Outbound Integration
      • Zapier Outbound Integration
      • Zendesk Outbound Integration
      • Zoom Integration
        • Zoom Chat Integration
        • Zoom Meeting Integration
    • Deployment integrations
      • API deployment pipeline
      • Argo CD deployment pipeline
      • Github deployment pipeline
      • GitLab deployment pipeline
  • API
    • API Reference
    • API Version History
      • API user preference migration 2023
      • Discontinuation of Uptime Monitoring
    • Rate Limiting
    • Client Libraries
      • ilert Agent - ilagent
      • Go Client
      • Rust Client
      • Javascript / Node.js Client
    • Terraform
      • Importing ilert UI resources into Terraform state
    • 👩‍💻ICL - ilert condition language
    • ➿ITL - ilert template language
    • API endpoints / samples
      • Creating alerts through events
      • Importing public status page subscribers
    • 🔥Developing ilert Apps
      • Get started with ilert Apps
      • Understanding OAuth2
      • Developing a Backend App with OAuth2
      • Developing a web or native App with OAuth2 and PKCE
      • Token lifetimes, error codes, app verification, etc.
  • Contact us
  • ilert Release Notes
Powered by GitBook
LogoLogo

Product

  • Alerting & Notification
  • On-call Management & Escalations
  • Call Routing
  • Status Pages

Resources

  • Blog
  • Case Studies
  • Security
  • API Reference

Legal

  • Privacy policy
  • Imprint

Increase Your Uptime

  • Start for Free
  • Get a Demo

(c) 2011 - 2025 ilert GmbH

On this page
  • System Requirements
  • In ilert: Create a Prometheus alert source
  • In Prometheus Alertmanager: add a webhook receiver
  • Dynamic policy routing
  • Supported custom labels
  • FAQ
  • Will alerts in ilert be resolved automatically?
  • Can I link Prometheus to multiple alert sources in ilert?
  • What if my internet connection is interrupted? Are the alerts generated in Prometheus lost?
  • Not all Prometheus Alerts alerts are created in ilert. Why?
  • The integration does not work. How do I find the issue?
  • How to setup multiple alert sources in the same config?
  • Further References

Was this helpful?

Edit on GitHub
  1. INTEGRATIONS
  2. Inbound integrations

Prometheus Integration

Create alerts in ilert from Prometheus Alertmanager alerts

PreviousPostman Monitors IntegrationNextPRTG Network Monitor Integration

Last updated 6 months ago

Was this helpful?

is an open-source systems monitoring and alerting toolkit that uses a pull-based approach to collecting metrics in a timeseries database.

With ilert's Prometheus integration, you can automatically create alerts in ilert using the Prometheus' Alertmanager. That way, you will never miss a critical alert and always alert the right person using ilert's on-call schedules, automatic escalation, and multiple alerting channels. When the Alertmanager triggers an alert, ilert will alert the on-call person through their preferred channel, including SMS, phone calls, push notifications and Slack. ilert will automatically escalate to the next person, if the alert is not acknowledged. ilert also lets you define alerting rules based on support hours and delay alerts until your support hours start.

System Requirements

  • or higher. If you are using an older version of the Alertmanager, please contact the ilert support at .

In ilert: Create a Prometheus alert source

  1. Go to Alert sources --> Alert sources and click on Create new alert source

  2. Search for Prometheus in the search field, click on the Prometheus tile and click on Next.

  3. Give your alert source a name, optionally assign teams and click Next.

  4. Select an escalation policy by creating a new one or assigning an existing one.

  5. Select you preference and click Continue setup. You may click Do not group alerts for now and change it later.

  6. The next page show additional settings such as customer alert templates or notification prioritiy. Click on Finish setup for now.

  7. On the final page, an API key and / or webhook URL will be generated that you will need later in this guide.

In Prometheus Alertmanager: add a webhook receiver

receivers:
- name: 'ilert.web.hook'
  webhook_configs:
  - url: 'https://api.ilert.com/api/v1/events/prometheus/e6bcfcbf-a38f-462a-af9d-1687809b7594'
route:
group_by: ['alertname']
group_wait: 10s
group_interval: 10s
repeat_interval: 1h
receiver: 'ilert.web.hook'

3. Restart the alert manager

curl -d '[{"labels":{"Alertname":"iLert Test"},"annotations":{"summary":"iLert Test"}}]' http://localhost:9093/api/v1/alerts

Dynamic policy routing

ilert's Prometheus integration supports dynamic escalation policy routing with the help of routing keys.

In ilert navigate to the escalation policies that you want to route to and enter a unique routing key for for each policy.

In your Prometheus alert rule yml add a label called ilert_routingkey and set its value to the policy's routing key that you want to assign to the alert e.g. ilert_routingkey: policy1

When ilert receives Prometheus alert events it will look for the first alert with the specific label and decide upon the routing. If the label is not present the escalation policy that is assigned to the alert source is used instead.

Supported custom labels

  • gcp_project will be automatically added to the alert summary

  • url may be used to add a custom link to the alert

  • urlLabel may be used to set a defined label for the custom link (url)

Note: for custom labels to be accepted they must be part of the alert labels of an alert in status firing

FAQ


Will alerts in ilert be resolved automatically?

Can I link Prometheus to multiple alert sources in ilert?

Yes, create several Webhook receivers in Prometheus and enter the URL of the alert source from ilert in the Webhook URL.

What if my internet connection is interrupted? Are the alerts generated in Prometheus lost?

Not all Prometheus Alerts alerts are created in ilert. Why?

The alerts from Prometheus are grouped and sent to ilert and bundled in an alert. Grouping is affected by the group_by configuration in the Alert Manager route.

Example:

route:
  # The labels by which incoming alerts are grouped together. For example,
  # multiple alerts coming in for cluster=A and alertname=LatencyHigh would
  # be batched into a single group.
  #
  # To aggregate by all possible labels use '...' as the sole label name.
  # This effectively disables aggregation entirely, passing through all
  # alerts as-is. This is unlikely to be what you want, unless you have
  # a very low alert volume or your upstream notification system performs
  # its own grouping. Example: group_by: [...]
  group_by: ['alertname', 'cluster', 'service']

The integration does not work. How do I find the issue?

How to setup multiple alert sources in the same config?

You can configure multiple alert sources for different alerts based on routes and conditions aka matchers. Make sure your receivers use the proper integration urls generated by ilert.

global: {}
receivers:
  - name: "alert-source-1"
    webhook_configs:
      - url: "alert-source-1-URL"
        send_resolved: true
  - name: "alert-source-2"
    webhook_configs:
      - url: "alert-source-2-URL"
        send_resolved: true

route:
  group_by: ["alertname", "cluster", "region"]
  group_wait: 10s
  group_interval: 5m
  repeat_interval: 1h
  receiver: alert-source-1
  routes:
    - receiver: "alert-source-1"
      matchers:
        - severity="warning"
    - receiver: "alert-source-2"
      matchers:
        - severity="critical"

Sending the same alert to multiple alert sources in parallel, is also possible if you use continue.

global: {}
receivers:
  - name: "alert-source-1"
    webhook_configs:
      - url: "alert-source-1-URL"
        send_resolved: true
  - name: "alert-source-2"
    webhook_configs:
      - url: "alert-source-2-URL"
        send_resolved: true

route:
  group_by: ["alertname", "cluster", "region"]
  group_wait: 10s
  group_interval: 5m
  repeat_interval: 1h
  receiver: alert-source-1
  routes:
    - receiver: "alert-source-1"
      continue: true
    - receiver: "alert-source-2"

Further References

Here is the instruction on how to import metrics from Prometheus and display them on your ilert status page:

1. Add a from the alert manager in the configuration file. Use the URL generated in ilert as the Webhook URL:

2. You can now configure any route in the Alert Manager. In the following example, all alerts that do not match another are sent to ilert:

4. Optional: Send a test alert through the .

Yes, Prometheus also sends resolved events by default, as long as the send_resolved: false option is NOT set in the of the alert manager. Furthermore, resolved events - just like firing events - are not sent until the next group_interval configuration in the alert manager.

No, alerts are not lost. The alert manager has a retry mechanism. In addition, we recommend that you monitor your Internet connection with an external monitoring service (e.g. using or uptime monitoring). See here for a .

First, look in the log file of the alert manager. If you can not find the issue, please contact our support at .

Webhook configuration
route
Alert Manager API
Webhook configuration
iLert's heartbeat feature
Prometheus Heartbeat Example
support@ilert.com
Import metrics from Prometheus
Prometheus
Prometheus Alert Manager v0.6.0 / 2017-04-25
support@ilert.com
Alert grouping