# Welcome to ilert
Get started with ilert - the complete incident management platform for reliable alerting, on-call management, and status pages.
ilert is a complete **incident management platform** that provides **reliable alerting**, **on-call management**, and **status pages** to help DevOps teams, SREs, and IT administrators respond to incidents faster and more effectively.
### Quick start (5 minutes)
1. **Sign up** at [app.ilert.com](https://app.ilert.com/signup)
2. **Create your first alert source** - connect your monitoring tool
3. **Set up an escalation policy** - define who gets notified
4. **Configure notifications** - choose your preferred channels
5. **Test the integration** - send a test alert
### Key features
* **Reliable Alerting**: Never miss critical incidents with SMS, phone calls, push notifications, Slack, Teams, and more
* **On-Call Management**: Automated escalations, schedules, and coverage requests to ensure the right person is always notified
* **Mobile App**: Full incident response capabilities on iOS and Android for on-the-go incident management
* **Status Pages**: Communicate incidents to stakeholders and customers with professional status pages
* **100+ Integrations**: Connect with your existing monitoring, ticketing, and collaboration tools
* **AI-First**: Smart alert grouping, automated root cause analysis and post mortem creation to reduce noise and improve response times
### Documentation Sections
| Section | Description |
| --------------------------------------------------------------------------------- | -------------------------------------------------- |
| [**Getting Started**](/getting-started/quick-start-guide) | Quick setup guides and FAQs |
| [**Alerting**](/alerting/configure-alerting/alert-sources) | Configure alert sources and reliable notifications |
| [**On-Call Management**](/on-call-management-and-escalations/escalation-policies) | Set up schedules and escalation policies |
| [**Integrations**](/integrations/types-of-integrations) | Connect your tools and workflows |
| [**Incident Communication**](/incidents-and-status-pages/getting-started) | Communicate incidents to stakeholders |
| [**Mobile App**](/mobile-app/getting-started-with-ilert-mobile-app) | iOS and Android app documentation |
| [**Developer Documentation**](https://docs.ilert.com/developer-docs/) | REST API and client libraries |
### Popular Use Cases
* **Infrastructure Monitoring**: Connect Prometheus, CloudWatch, or Datadog for reliable infrastructure alerting
* **Application Monitoring**: Integrate with Sentry, New Relic, or AppDynamics for application incident management
* **Uptime Monitoring**: Use Pingdom, UptimeRobot, or StatusCake for website and service monitoring
* **Security Alerts**: Connect AWS GuardDuty, Azure Sentinel, or CrowdStrike for security incident response
* **CI/CD Pipelines**: Monitor deployments with GitHub, GitLab, or ArgoCD for deployment incident management
***
### Core Concepts
The most important concepts are explained in the following sections.
#### Alert
An alert is an event that requires immediate attention and needs to be resolved. Alerts are created by an alert source and immediately trigger the notification process using the alert source's escalation policy to ensure reliable incident response.
An alert can have the following states:
| `PENDING` | The alert hasn't been acknowledged yet and will escalate until the alert is either acknowledged or resolved. |
| ---------- | ----------------------------------------------------------------------------------------------------------------- |
| `ACCEPTED` | The user to whom the alert is assigned to is working on resolving the alert and the escalation process is halted. |
| `RESOLVED` | The alert is resolved and no more notifications are sent. A resolved alert cannot be opened again. |
#### Alert source (aka inbound integration)
An alert source represents the connection between your tools (usually a monitoring system, a ticketing tool, or an application) and ilert. We often refer to alert sources as **inbound integrations**.
{% content-ref url="/pages/J3y3tEP563wQvwa27nO1" %}
[Alert sources](/alerting/configure-alerting/alert-sources)
{% endcontent-ref %}
#### Deployment events
The deployment events view gives you a live overview of all deployments related to your account. If a deployment event can be correlated to an alert, ilert will enrich your alert's context with the most relevant deployment information to improve incident response.
{% content-ref url="/pages/iYc7bpmJRZ0vAyj44x6X" %}
[Deployment events](/alerting/deployment-events)
{% endcontent-ref %}
#### Connector and alert action (aka outbound integration)
Connectors and alert actions allow you to extend your incident response and communication to other tools. They allow you to either manually or automatically perform actions on alerts, such as
* Update your ilert status page
* Create a ticket in JIRA
* Post a message in Slack
* Post a webhook to a defined HTTP end point
A **connector** is created globally in ilert and contains all the information to connect with the target system (e.g. a URL, an API key or username and password, etc.). An **alert action** can be created at the team-level and assigned to alert sources. An alert action uses its connector to perform a concrete action. Example: Let's say we want to create an issue in JIRA for every alert in ilert. We need to create ...
* ... a JIRA **connector** that contains the URL of the JIRA server and the credentials to connect to it.
* ... an **alert action** that contains information such as whether to trigger the connection manually or automatically for every alert, the JIRA project ID and issue type, and any custom fields that we might want to set in the JIRA issue.
We often refer to connectors and alert actions as **outbound integrations**.
#### Escalation policy
An escalation policy connects an alert source with the users that are responsible for this alert source. It defines which users or on-call schedules should be notified when an alert is created to ensure reliable incident response.
{% content-ref url="/pages/mnYNEZKC2V9fk1po01Pv" %}
[Escalation policies](/on-call-management-and-escalations/escalation-policies)
{% endcontent-ref %}
#### On-call schedule
On-call schedules determine who will be notified when an alert is created based on the time of day. Only one user per schedule can be on-call at a time. You can reference an on-call schedule in an escalation policy.
{% content-ref url="/pages/-MUluo7xv\_kdaHEuns5d" %}
[On-call schedules](/on-call-management-and-escalations/on-call-schedules)
{% endcontent-ref %}
#### Incident, Status update, Service and Status page
An **incident** is the coordination record for a significant event. Declare an incident to link alerts, page responders, track a timeline, and run the response to resolution.
A **status update** is the public-facing message you post from an incident. Status updates appear on your status pages and are also delivered directly to subscribed ilert users—such as those with the VIEWER role—based on their service and status page subscriptions.
**Services** model the business capabilities your users care about. Incidents and status updates reference services, and users can subscribe to them to receive notifications.
**Status pages** help you inform users about the current status of your services and the history of recent status updates. Any ilert user can subscribe to status updates, services, and status pages.
{% content-ref url="/pages/MJo1tVBA6IBAozLBReYb" %}
[Getting started](/incidents-and-status-pages/getting-started)
{% endcontent-ref %}
#### Notification
A notification is a message that is sent to a user to inform them about alerts, incidents or on-call shifts. Users manage how they want to be notified in their notification settings to ensure reliable incident response.
{% content-ref url="/pages/vKzgTehGca500o6Nf9md" %}
[Notification settings](/alerting/configure-alerting/notification-settings)
{% endcontent-ref %}
# Quick start
Connect a tool to ilert, route its alerts to an on-call responder, and page yourself with a real event.
This guide takes you from an empty account to a real alert on your phone. It takes about 10 minutes.
By the end you will have:
* an **alert source** that receives events from your tools
* an **escalation policy** that decides who gets paged, and when
* verified **notification channels** on your own profile
* a **test alert** that reached you, which you then resolve
## Before you begin
* An ilert account. [Sign up](https://app.ilert.com/signup) if you do not have one.
* Your phone, to receive and verify notifications.
* A terminal with `curl`, if you want to send the test event yourself rather than through a monitoring tool.
## Create an alert source
An alert source is the connection between a tool and ilert. It receives events, decides whether each one becomes an alert, and hands that alert to an escalation policy.
{% stepper %}
{% step %}
## Open the alert source wizard
In the sidebar, go to **Alerting** → **Alert sources**, then click **Create new alert source**.
{% endstep %}
{% step %}
## Select an integration
Pick the tool that will send events. Use the search box or the category filters to narrow the list.
If you are only testing, choose **API** — it accepts any JSON payload you send it, so you do not need a monitoring tool wired up yet. **Email** is the other good starting point, because almost every tool can send email.
Click **Next**.
{% endstep %}
{% step %}
## Name it and assign a team
Give the alert source a name that identifies the system the alerts come from, not the tool that sends them — `Checkout API` tells an on-call responder more at 3am than `Prometheus` does.
Assigning a team is optional here, and controls who can manage the alert source and its settings. Click **Next**.
{% endstep %}
{% step %}
## Choose who gets notified
Under **Who should be notified about alerts?**, either select an existing policy or choose **Generate new escalation policy**, which creates a basic policy that notifies you first.
When you select an existing policy, ilert previews the escalation chain underneath it, so you can confirm who is paged before any alert exists.
The preview shows the full escalation chain: the on-call rotation first, then two named responders after 10 minutes.
Click **Next**.
{% endstep %}
{% step %}
## Choose how events are grouped
Grouping decides how many alerts a stream of events produces. Pick one:
| Option | Behavior |
| ------------------------------- | ------------------------------------------------------------------------------------- |
| **Smart grouping (AI-powered)** | Groups similar events by analyzing their content, within a grouping window you choose |
| **Time-based grouping** | Groups every event that arrives within a time window, or until the alert is accepted |
| **Default grouping** | Groups by the alert key that the integration sets |
**Auto-resolution** is separate, and optional: it resolves alerts that stay open longer than the period you set.
None of this is permanent — change the grouping mode on the alert source at any time.
Click **Continue setup**.
{% endstep %}
{% step %}
## Finish setup
The wizard hands you to the full settings page, where you can set alert templates, notification priority, and dynamic routing. None of it is required now.
Click **Finish setup** to save the alert source.
{% endstep %}
{% endstepper %}
{% hint style="info" %}
Until you click **Finish setup**, the alert source shows a **FINISH SETUP** badge in the list. It still receives events — the badge only marks that you have not been through the settings page.
{% endhint %}
## Get your integration key
The alert source detail page has everything you need to send events.
The integration key is masked by default. Use the eye icon to reveal it and the copy icon to copy it.
Copy the **integration key**. You will send it with every event.
{% hint style="warning" %}
The integration key identifies the alert source and lets anyone who holds it create alerts in your account. Treat it like a password: keep it out of source control, and rotate it with the regenerate icon if it leaks.
{% endhint %}
## Set up your notifications
An alert that pages nobody is not much use. Before you test, make sure ilert can reach you.
Open the user menu in the top right, then click **Notification settings**.
Add and verify the channels you want to be paged on, then set notification rules below.
1. Add and verify a phone number under **Phone number contacts** if you want calls or SMS.
2. Install the mobile app for push notifications — [iOS](https://apps.apple.com/app/ilert/id542915864) or [Android](https://play.google.com/store/apps/details?id=de.ilert.client.iphone) — then use **Send test notification** to confirm the device works.
3. Under **Notification rules**, set what happens when a high-priority alert is assigned to you, and how it differs from a low-priority one.
## Send a test event
Replace `YOUR_INTEGRATION_KEY` with the key you copied, and run:
```bash
curl -X POST https://api.ilert.com/api/events \
-H "Content-Type: application/json" \
-d '{
"integrationKey": "YOUR_INTEGRATION_KEY",
"eventType": "ALERT",
"summary": "Refund webhook backlog above threshold",
"details": "3,412 refund webhooks pending delivery. Oldest is 18 minutes behind."
}'
```
Only three fields are required: `integrationKey`, `eventType`, and `summary`. The Events API authenticates with the integration key in the body, so there is no `Authorization` header.
{% hint style="warning" %}
**A `202` does not mean an alert was created.**
The Events API answers `202 Accepted` with an empty body — your event was queued, not processed. You get no alert ID, and an event that the alert source later filters or rejects returns exactly the same `202`. The outcome is recorded in the alert source's **Alert source logs** tab, never in the response.
Keep the `correlation-id` response header if you need to ask support what happened to a specific event.
{% endhint %}
For JavaScript, Python, and the full field reference, see [Creating alerts through events](/developer-docs/api-samples/creating-alerts-through-events).
## Confirm it worked
Go to **Alerts** in the top navigation and open the alert you just created.
A newly created alert is PENDING and escalating. The timeline records every notification sent.
Check three things:
* **Status** is `PENDING` — the alert is live and escalating.
* **Responders** names the person the escalation policy picked. If your policy targets an on-call schedule, this is whoever is on call right now.
* The **Timeline** on the right lists each notification as it goes out, including the channel it used.
Then finish the lifecycle:
* **Accept** the alert to take ownership. This **halts escalation** — ilert treats an accepted alert as owned by a human and stops paging anyone else.
* **Resolve** it when the underlying problem is fixed. A resolved alert cannot be reopened; a later event creates a new one.
{% hint style="success" %}
If the alert reached your phone and you resolved it, your setup works end to end: a tool can now page the right person, and escalate if that person does not respond.
{% endhint %}
## What's next
Connect your real tools
Over 180 inbound integrations, from Prometheus and Datadog to Jira and ServiceNow.
# Troubleshooting
Diagnose events that never become alerts, alerts that never resolve, and notifications that never arrive.
Most setup problems come down to one of three questions: did the event arrive, did it become an alert, and did the alert reach a person. Work through them in that order — each section below tells you where to look for the answer rather than what to guess.
## Start with the alert source logs
The **Alert source logs** tab on an alert source is the record of what ilert actually received and what it did with it. Check it before changing any settings.
Each row is one received event, with the raw payload ilert parsed. Use the time range links to widen the window.
{% hint style="warning" %}
**`202 Accepted` does not mean an alert was created.**
The Events API queues your event and answers `202` with an empty body. An event that is later filtered, deduplicated into an existing alert, or rejected returns exactly the same `202`. The response never tells you the outcome — the alert source log does.
{% endhint %}
Events appear in the log with a short delay, so widen the range with **Last 1 hour** rather than concluding an event was lost.
## Events are not creating alerts
### Events appear in the log, but no alert exists
The alert source received the event and deliberately did not open a new alert. In order of likelihood:
* **The event was grouped into an existing alert.** If the event's `alertKey` matches an alert that is still open, ilert appends to that alert instead of creating another one — regardless of the grouping mode you selected. Alert keys are trimmed and compared **case-insensitively**, so `SRV-1` and `srv-1` are the same alert. Check **Grouped events** on the open alert.
* **A grouping window swallowed it.** With time-based or smart grouping, events arriving inside the window join the current alert.
* **An event filter rejected it.** Check the alert source's event filter conditions against the payload shown in the log.
### No events appear in the log at all
Nothing reached the alert source. The event never left your tool, or it went somewhere else:
* **Wrong integration key.** Each alert source has its own key. Copy it again from the source's **Settings** tab — a truncated or stale key silently routes nowhere.
* **Malformed JSON.** Confirm the payload parses. A missing comma is the most common cause, and it fails before ilert ever sees a valid event.
* **Missing required fields.** `integrationKey`, `eventType`, and `summary` are all required.
* **Network path.** Verify the sender can reach the API at all:
```bash
curl -I https://api.ilert.com/api
```
If that fails, check egress firewall rules, proxy configuration, and DNS from the host that sends the events.
### Alerts do not resolve automatically
* Confirm your tool actually sends resolve events. In Prometheus Alertmanager, for example, that means `send_resolved: true` on the receiver.
* The resolve event must carry the **same `alertKey`** as the alert it should close. Matching is case-insensitive, but a different key resolves nothing.
* Confirm the resolve event has `"eventType": "RESOLVE"`.
If you would rather close stale alerts on a timer, enable **Auto-resolution** on the alert source instead.
## Notifications are not arriving
Open the alert and read its **Timeline** first. It records every notification ilert sent, on which channel, and when. That single check splits the problem in two.
### The timeline shows notifications, but you received nothing
The alert reached the notification stage, so the problem is the channel or the contact:
* Confirm the phone number or email is **verified** in **Notification settings**.
* Check that you have not muted notifications — the user menu has a **Mute notifications** toggle.
* For push, use **Send test notification** next to the registered device. If the test fails, reinstall and re-register the device.
* On iOS, high-priority alerts need [critical alerts](/mobile-app/mobile-app-notification-settings/critical-push-notifications-and-dnd-overrides/ios-critical-alerts-configuration) enabled to break through Do Not Disturb. On Android, see [DND configuration](/mobile-app/mobile-app-notification-settings/critical-push-notifications-and-dnd-overrides/android-push-notification-dnd-configuration).
### The timeline shows no notifications at all
Nobody was selected to notify, or escalation stopped before reaching you:
* **The alert was accepted.** Accepting halts escalation by design — ilert treats the alert as owned and stops paging. This is the single most common cause of "escalation stopped working". If you want a safety net for alerts that are accepted but never resolved, use an alert action on the `v-alert-not-resolved` trigger rather than another escalation level.
* **Nobody is on call.** An escalation rule that targets a schedule with no active shift **falls through immediately**, without waiting for its escalation timeout. If no rule in the policy has a live target, nobody is notified at all. Check the schedule for the current time.
* **The alert is low priority.** A `LOW` alert only ever uses the **first** escalation rule and never advances past it.
* **Support hours downgraded it.** Support hours do not suppress alerts — they create `LOW` alerts outside the configured window, which then behave as above. If you expected a page at 2am and got none, this is usually why. Enable **Auto-raise alerts** to re-raise still-pending alerts when support hours begin.
* **Your notification rules have no matching rule.** Rules are per-priority; a profile with rules only for high-priority alerts stays silent on low-priority ones.
## API errors
| Status | Meaning | What to do |
| ------ | ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `400` | Invalid request | With `max-results`, the page cap is per endpoint and overshooting is a hard error, not a clamp. Lower the value. |
| `401` | No `Authorization` header | Add the header. Note the Events API does not use one — it authenticates with the integration key in the body. |
| `403` | Unknown, revoked, or malformed API key (`code: KEY_ERROR`), or a missing OAuth2 scope | Reissue the key. Retry logic keyed on `401` never fires here, so handle `403` explicitly. |
| `429` | Rate limited | Back off and retry. There are two independent buckets — REST calls per token, and events per integration key. |
{% hint style="info" %}
`429` responses carry no `Retry-After` header and no rate-limit headers. Derive your pacing from the `429`s you actually receive rather than from a fixed number, since limits vary per account and integration.
{% endhint %}
A `404` means the entity is not there **for you**. That covers both "it does not exist" and "your key cannot see it" — the API does not distinguish the two on purpose. Widen the key's permissions or team context and try again rather than reading it as proof of deletion.
## Contacting support
Include these so support can trace the exact request:
* The **`correlation-id`** response header from the failing API call. This is the fastest way to find your event in the logs.
* The **alert source ID** and a timestamp with time zone.
* The raw payload you sent, with the integration key removed.
* What you expected to happen, and what happened instead.
Reach support at or through the in-app chat.
# Overview
How ilert turns events from your tools into alerts, and how alerts reach the person who can do something about them.
Alerting is how a signal from one of your tools becomes a page to a human. An event arrives, an alert source decides what it means, and an escalation policy decides who hears about it.
## How an event becomes a page
{% stepper %}
{% step %}
## An event arrives
A monitoring, logging, or ticketing tool sends an event to an **alert source** — over the Events API, by email, or through one of the built-in integrations. The event carries a payload describing what changed.
{% endstep %}
{% step %}
## The alert source decides what it means
The alert source owns every parsing and grouping rule. It can filter the event out, group it into an alert that is already open, or treat it as something new. This is where noise is controlled.
{% endstep %}
{% step %}
## An alert is created or updated
A new **alert** appears in **Alerts**, or an existing one absorbs the event. An alert is the actionable record: it has a status, an owner, and a timeline.
{% endstep %}
{% step %}
## Responders are notified
The alert source's **escalation policy** decides who is notified and in what order. Each person is reached through the channels set in their own notification rules — push, SMS, voice, email, or chat.
{% endstep %}
{% endstepper %}
## Where incidents fit
Most alerts never become incidents. The two are different objects with different jobs:
* An **alert** is one actionable signal. It is created automatically and resolved when the underlying problem is fixed.
* An **incident** is a coordination record you declare deliberately when something has business impact — it gathers responders, links the alerts involved, and keeps a shared timeline.
Declaring an incident is a decision a person makes, not a stage every alert passes through. An incident also publishes nothing on its own; a **status update** posted from it is what reaches your status pages and subscribers.
{% hint style="info" %}
If you are looking for the customer-facing side of this — services, incidents, and status pages — see [Incidents & status pages](/incidents-and-status-pages/getting-started).
{% endhint %}
## Alert states
| State | Means |
| ---------- | --------------------------------------------------------------------------------------------------------------------- |
| `PENDING` | Not yet acknowledged. The alert is escalating through its policy. |
| `ACCEPTED` | Someone has taken ownership. **Escalation stops** — ilert will not page anyone else. |
| `RESOLVED` | The problem is fixed. No further notifications. A resolved alert cannot be reopened; a later event creates a new one. |
## Core objects
| Object | What it is |
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Event** | The raw payload a tool sends. Routed to an alert source by its integration key, and visible in that source's logs. |
| **Alert source** | Receives events. Owns the integration key, the escalation policy, and every filtering and grouping setting. Also called an *inbound integration*. |
| **Alert** | The actionable record created from one or more events. |
| **Escalation policy** | The ordered rules deciding who is notified, and how long each level has to respond. |
| **Notification** | One message to one person over one channel. |
| **Alert action** | Optional. Automation that fires when an alert is created, updated, or resolved — opening a Jira ticket, posting to Slack. Also called an *outbound integration*. |
| **Event flow** | Optional. A routing layer above alert sources, with its own ingest URL, for conditional routing and transformation. |
## In this section
How ilert processes events & alerts
The full lifecycle, including grouping, rate limits, and suppression.
# How ilert processes events & alerts
Learn how ilert processes incoming events into alerts and incidents, including grouping, deduplication, aggregation, and suppression mechanisms.
## Platform overview
This global platform overview is a sketch that tries to showcase all streams and moving parts across alerting (formed around **events** and **alerts**) and incident communications (formed around **services** and **incidents**). Different parts are drilled down upon and shown in more focus below.
## Lifecycle overview: Event **→** Alert **→** Incident
This illustration is an extraction of the core incoming event to outgoing notification stream from the top-level overview above.
## Event, alert, and notification aggregation
This illustration maps the core stream between receiving events and sending notifications and shows where rate limits, aggregations, and suppressions may apply.
If you want to know more about event (alert source) API rate limits, take a look at: REST API **→** Rate Limiting.
A note on "alertKey" grouping: as stated in the graphic, alert grouping based on alertKeys is usually automatically handled by the integration for the specific monitoring tool inside of ilert. However, not all tools support event payloads that contain specific information required to group and identify alerts. To know if an integration supports automatic grouping, keep an eye out for the "**Alert resolution**" or "**Alert acknowledgement**" feature either in the information modal of the **Alert source creation wizard** or in the **Alert source detail view** itself.
A note on "**Grouping in time window**" or "**Allowing a single open/pending alert**": this setting can be configured for every alert source and works in addition to grouping by "alertKey".
### Specific descriptions for the illustrated notification suppression\*
#### Voice calls:
* Only a single ongoing dial is supported per phone number across the platform at a single point in time, including a small buffer before and after the call. **Simply put,** **an outbound voice call to the same phone number is supported approximately every 10 seconds.**
* Each first call made starts a 180-second window, in which up to 2 additional calls can be made until every following call is aggregated until the end of the window. Finally, an aggregated information call is made informing about multiple alerts, if at least one of these alerts is not yet accepted. **Simply put, one call per minute per alert source per phone number**.
#### SMS or email:
* Each first SMS or email sent starts a 180-second window, in which up to 2 additional notifications can be sent until every following notification is aggregated until the end of the window. Finally, an aggregated information notification is sent informing about multiple alerts, if at least one of these alerts is not yet accepted. **Simply put:** **one SMS or email per minute per alert source per phone number or email address.**
#### Push notifications or messengers:
* Each first notification sent starts a 240-second window, in which every following notification is dropped if its content is identical to one of the others already sent in the window.
* Each first notification sent starts a 60-second window, in which up to 19 following notifications are allowed until the rest is dropped. S**imply put, 20 unique notifications per minute** (*Note: that this is a global limit on the target and not on a per alert source basis*)
{% hint style="success" %}
ilert never simply "drops" a notification due to rate limiting or suppression, you will always see an entry for the notification (no matter its state) in the related alert's timeline.
{% endhint %}
#### Suppression availability
Until 2020, notifications were not aggregated.
From 2020 to December 2023, all notifications were automatically aggregated in a 3-minute window, per target per alert source, that started with each first notification.
Specific suppressions and deduplications as listed in this doc have been in use since December 11, 2023.
# Data retention
How long ilert retains customer data, including alerts, logs, configuration data, and the timelines that apply when an account is deleted.
This page describes how long ilert retains customer data. It covers two categories: **master data** (configuration such as services, escalation policies, and users) and **moving data** (event-shaped records such as alerts and logs). Account deletion timelines are described at the end of the page.
## Master data
ilert keeps personal data as long as required to provide the online services you have requested or registered for and comply with applicable laws. This includes configuration data such as services, escalation policies, users, integrations, and schedules. Master data is not subject to the moving data retention periods below and is only removed when you delete it or when your account is deleted.
## Moving data retention
Retention is measured from the time each record is created. When a record's retention period ends, the record is removed from the live system and cannot be recovered through the product.
{% hint style="info" %}
If you need to retain data beyond the periods listed below, you can use the ilert API or webhook streams to export records into your own storage for long-term archival.
{% endhint %}
| Data type | Retention |
| --------------------------------------------- | --------- |
| Alerts & notifications | 18 months |
| Alert source logs (event data & correlations) | 18 months |
| Webhook logs | 18 months |
| Call flow logs | 6 years |
| Call flow log details | 18 months |
| Event flow logs | 2 years |
| Event flow log details | 18 months |
| Incidents & incident timeline | 6 years |
| Conversations & chat messages | 6 years |
| Maintenance windows | 6 years |
| Audit logs | 6 years |
| Reports | 10 years |
{% hint style="info" %}
The periods above are our target retention periods. We are still rolling out the corresponding rotation mechanisms across the platform, so some resources may currently still hold data for longer than listed.
{% endhint %}
## Account deletion
If you request deletion of your account, all your account data — both configuration and moving data — is removed from the live system within 30 days (this is unrelated to the retentions mentioned above).
## Backups
Backups have their own retention cycle, separate from the periods above. Data deleted from the live system continues to exist in backups until those backups rotate out, after which it is no longer retained.
# ilert sender IDs, domains and IPs
Add ilert phone numbers to white lists, ensuring voice alerts reach the responders at the right time. ilert's sender IDs used for outgoing notification (e.g., SMS phone numbers, caller IDs, etc.).
You can find information on our domains, notification email addresses, phone numbers below. We suggest whitelisting these in your tools and email clients.
## SMS phone numbers
The number that ilert uses to send SMS notifications depends on the target country:
| Country | Number |
| ----------------- | --------------------------------------------------------------------------------------------------- |
| USA and Canada |
+31 97010252693 +31 97010241585 (also used for DE)
|
| Poland | +48 732 230 768 |
| Rest of the world |
+1 256 391 4978
+1 240 258 2894
|
## Phone call numbers
The number that ilert uses to send voice notifications depends on the target country:
| Country | Number | Description |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- |
| USA and Canada |
+1 240 258 2894 +1 267 6696969
|
- starting March 1st, 2022
|
| Germany |
+49 157 3598 1212 +49 221 96303220
|
- starting March 1st, 2022
|
| United Kingdom | +44 7723 487389 | starting December 1st, 2020 |
| Australia | +61 480 020 343 | starting December 1st, 2020 |
| Belgium | +32 460 22 90 75 | starting December 1st, 2020 |
| Netherlands |
+31 97010252693 +31 97010241585 (also used for DE)
|
starting December 1st, 2020 starting March 1st, 2022
|
| China |
+1 2015026675
+1 2015026789
+1 2015026758
+1 2015026740
+1 2015026791
+1 2015026741
+1 2015026755
+1 2015026807
+1 2015026760
+1 2015026764
| rotated on 28.07.2023 |
| Rest of the world | +49 157 3598 1212 | |
## Email addresses
Name
Description
Email
Notifications
Email notifications sent by ilert
notifications@ilertnow.com
Incoming Emails
Emails sent by your tools to trigger alerts in ilert
{alert-source}@{your-domain}.ilertnow.com
{alert-source}@{your-domain}.ilertnotify.com
Help & Support
Our support email
support@ilert.com
## Domains
| Name | URL |
| ------------------------------------ | ------------------------------------------------------------- |
| Web application | and |
| API | [https://api.ilert.com/api](https://api.ilert.com/api/v1) |
| Incoming Heartbeat Pings | [https://beat.ilert.com](https://beat.ilert.com/) |
| Status Pages without a custom domain | [https://your-domain.ilert.io](https://your-domain.ilert.io/) |
| Website | |
| Documentation | |
| Service Status | [http://status.ilert.com](https://status.ilert.com/) |
## IP addresses
### Inbound
ilert's inbound architecture is highly distributed and spread across different data centers and regions. While it is possible to temporarily whitelist the IP blocks used, it is not recommended to do so. Inbound IPs are dynamic and, though mostly reserved, might change at any time.
### Outgoing webhooks (alert actions)
ilert uses flexible serverless infrastructures to provide webhooks in regions around the world. We offer static IP addresses to our[ Scale plan](https://www.ilert.com/pricing) customers. Please, reach out to our [support](mailto:support@ilert.com) if you are interested in using static IPs in case firewall whitelisting is required for your use case.
Address
Region
Description
18.159.7.25
Frankfurt
webhooks
3.123.64.23
Frankfurt
webhooks
54.73.206.214
Dublin
webhooks
52.215.200.26
Dublin
webhooks
Additional IPs of **other regions e.g. US** will be provided on request.
# SMS and voice alerts in China
You may find information regarding ilert voice and SMS alerts for Chinese landline and mobile devices below.
## Call Limitations to China
Due to strict spam monitoring and regulations on communications terminating to China, calls with short durations or repeated calls of the same number within a rolling hour have a high chance of being blocked. This is not something unique to ilert's platform and is a universal behaviour for all international traffic terminating to China.
## Receiving voice alerts in China
With the help of our platform solution and our partners ilert still tries to offer voice alerting to users and customers in China. Since China's monitoring happens downstream from ilert and ilert's partners, we have no control over the blocking process. Call delivery to China will be made on a best-effort basis.
## What you need to know
* By default China voice calls are not enabled, please contact our [support](mailto:support@ilert.com) if you need to alert Chinese devices from your account
* SMS alerting for China is currently not possible
* Call delivery to China will be made on a best-effort basis
* If a high number of unique alerts are triggering voice alerts for your Chinese number **in a single hour**, there is a chance that calls will be skipped to prevent blocking (this is logged in the alerts timeline), we suggest to always add a secondary notification preference a long voice alerts for Chinese users
* Currently DTMF feedback (e.g. pressing 1 to accept) is not supported in calls
* We suggest installing the ilert mobile app as it keeps all of our outbound numbers in sync with your devices contacts
* Chinese call notifications do not support direct alert responses e.g. pressing a key to accept or resolve, we suggest using the mobile app for this
## Which alternatives are available?
* Push notifications using the ilert mobile app for iOS or Android. Did you know, we offer iOS critical alerts and override DND mode for Android?
* Webhooks and outbound integrations can help you pipe alerts through third-party or custom solutions
{% hint style="info" %}
We are regularly validating additional options to move the Chinese calling experience up to the same level as the rest of the global ilert call features.
{% endhint %}
# Working with alerts
Filter the alert list down to what you care about, save those views, and act on alerts in bulk.
The **Alerts** page is where triage happens. On a busy account it lists far more than any one person needs, so most of the work is narrowing it — by label, by source, by status — and then acting on what is left.
## Filter alerts by labels
Labels come from the events your tools send. Filtering on them narrows the list to a service, an environment, or a cluster.
{% stepper %}
{% step %}
## Open the label filter
Go to **Alerts**, then click **Filter** under the **Labels** header.
{% endstep %}
{% step %}
## Choose a key, an operator, and a value
In **Add label filter**, pick the label key (for example `app`), choose an operator such as `is equal to`, then select or type the value (for example `database`).
{% endstep %}
{% step %}
## Add it
Click **Add filter**. Repeat to narrow further — multiple label filters apply together.
{% endstep %}
{% endstepper %}
## Save a view
A filter you rebuild every morning should be a saved view instead.
{% stepper %}
{% step %}
## Save the current filters
With your filters applied, click the **+** icon in the **Saved filters** bar.
{% endstep %}
{% step %}
## Name it and choose who sees it
Enter a name, then pick a visibility:
| Visibility | Who sees it |
| ------------- | ------------------------------- |
| **Personal** | Only you |
| **Team** | Members of the teams you select |
| **Public** | Everyone in the account |
| {% endstep %} | |
{% step %}
## Save
Click **Save**. The view appears as its own tab in the **Saved filters** bar.
{% endstep %}
{% endstepper %}
### Managing a saved view
Click the three-dot icon on a saved filter tab:
| Action | Does |
| -------------- | ---------------------------------------------------- |
| **Edit** | Change the name or visibility |
| **Copy link** | Share a direct link to this filtered view |
| **Duplicate** | Copy it as a starting point for another view |
| **Pin filter** | Make it the view you land on when opening **Alerts** |
| **Delete** | Remove it permanently |
## Act on alerts in bulk
During a noisy period, handling alerts one at a time is the bottleneck. Select several and act on them together.
Tick the checkboxes on the left of each row — or the checkbox in the header row to select everything in the current filtered view — and a bulk action bar appears above the list.
| Action | Does |
| ----------------- | ------------------------------------------------------------------------------------------------------- |
| **Accept** | Take ownership. This **stops escalation** on every selected alert. |
| **Reroute** | Move the selected alerts to a different escalation policy |
| **Resolve** | Close them, if the underlying problem is fixed |
| **Link incident** | Attach them to an existing [incident](/incidents-and-status-pages/incidents) |
| **Merge** | Combine them into one parent alert — see [Merging alerts](/alerting/working-with-alerts/merging-alerts) |
{% hint style="warning" %}
An alert that is already a parent — one that has other alerts merged into it — cannot be merged into another alert.
{% endhint %}
{% hint style="info" %}
Bulk **Accept** halts escalation on everything you selected. If you are clearing a backlog rather than taking ownership, **Resolve** is usually what you want.
{% endhint %}
# Alert view
All alert information and actions related to alerts – on one screen. See the latest deployments, chat with the team and ilert AI, and perform key actions in seconds to resolve issues faster.
The alert view brings everything you need to assess, collaborate on, and resolve an alert into one screen. It reduces context switching by surfacing escalation policies, responders, priorities, and other critical information. It provides you with a clear and exhaustive context of what is happening and helps you take fast actions toward resolution. This article aims to help you navigate this view and use helpful tools to respond to [incidents](/incidents-and-status-pages/incidents) more efficiently.
## How to open alert view
To switch to the Alert view, click one of the alerts you see in the alert list.
After click, you will see a similar screen. We will walk you through different elements of this view.
## Alert view elements
#### 1 - Escalation policy
Shows the [escalation policy](/getting-started/quick-start-guide#choose-who-gets-notified) currently handling the alert and the escalation status. Here, you can review the assigned policy to understand notification flow. Click the policy name to open details.
**2 - Responders**
Lists users currently engaged on the alert. You can hover over the names of [responders](/users-and-access-management/user-roles-and-permissions) and check the contact details. If a responder acknowledges the alert, his avatar is highlighted with a green circle.
**3 – Priority**
Highlights the business impact level used by automations and reporting. Priority can be raised from Low to High but not the other way around. To adjust priority for different alert sources, continue reading [*Notification priority and support hours".*](/alerting/configure-alerting/alert-sources#notification-priority-and-support-hours)
**4 - Duration**
Shows how long the alert has been active. After resolution, it displays total time. Hover over duration to see details and timestamps. Duration influences such metrics, as MTTA and MTTR, and will be reflected in your [Reports](/reports/overview).
**5 - Links**
One-click access to external systems such as Grafana dashboards. Links help you quickly switch between tools when gathering more information during investigation.
**6 - Quick access to the war room**
If you enable ChatOps, you can create dedicated incident channels in [Slack](/chatops/slack) and [Microsoft Teams](/chatops/microsoft-teams) to keep all communication related to the issue in your main communication tool. To connect your chat tool with ilert, proceed with the [*ChatOps overview*](/chatops/overview) article.
**7 - Grouping**
Shows the alert grouping status. There are five types of alert grouping available in ilert:
* Native integration based grouping
* Time-based grouping
* Grouping until accepted
* Grouping until resolved
* ilert AI similarity based grouping
If you are looking to enable grouping for your alert source, [read these instructions](/alerting/configure-alerting/alert-sources#event-grouping).
**8 - Similar open alerts**
ilert identifies similar active alerts that might be related to the same issue.
**9 - Labels**
Shows a structured list of key–value pairs. They help you scan the context faster and read high-signal labels like service, environment, region, and owner to understand impact.
**10 - Chat**
Use Chat to coordinate with other responders and to provide instructions to [ilert AI](/ai-and-agents/introduction). Use @-symbol to mention colleagues or to activate AI Responder.
**11 - Timeline**
Switch to Timeline to see a chronological log of actions, policy events, Slack notifications, posts to channels, and status changes.
**12 – Actions**
You can execute predefined alert actions, such as creating tickets in your ITSM system or automatically generating incidents. To explore how to utilize alert actions with third-party tools, please refer to the [O*utbound integrations"*](/integrations/outbound-integrations) section for more details. Also, actions recommended by ilert Responder are displayed here.
**13 - Alert details**
Displays a structured payload of the alert for quick diagnosis. Click the \ symbol to see raw event data. Read further about [Alert source logs (aka Event explorer)](/alerting/working-with-alerts/alert-source-logs).
**14 - Deployment events**
Shows recent code deploys that may correlate with the alert. Here, you can scan the deployment list for commits to the affected service or team, or use this context to decide on a rollback or owner escalation. If you haven't connected ilert to your CI/CD pipelines, find a step-by-step guide in "[*Deployment events*](/alerting/deployment-events)" or continue with the [Deployment integrations list](/integrations/deployment-integrations).
**15 – Incident communication**
Here, you can create incidents from alerts, link alerts to existing incidents and manage stakeholder updates. If you haven't created a status page yet, continue reading [the guide on incident comms in ilert](/incidents-and-status-pages/getting-started).
## FAQ
**Q: My view lacks some of the sections mentioned above, why?**
A: If you haven't enabled some of the ilert features, like Deployment events, you wont see them in this view. Also, ilert shows the most relevant sections for your environment to keep your attention focused.
# Merging alerts
The Alert Merge feature lets you combine multiple existing alerts into a single main alert. It's designed to help teams reduce noise and consolidate duplicate alerts.
### Overview
While [Event Grouping](/alerting/configure-alerting/alert-sources#event-grouping) automatically groups similar incoming events into one alert, **Alert Merge** is a manual action that merges **existing alerts** already created in ilert.
**Use cases:**
* Multiple monitoring tools trigger alerts for the same issue.
* Alerts were not automatically grouped.
* You want a single alert for reporting and resolution.
### How it works
* You can merge **two or more alerts** into one **main alert**.
* Merged alerts (called **child alerts**) retain their original state but become read-only for manual updates.
* Escalations and notifications for merged alerts stop immediately.
* Updates, accept, and resolve actions are managed through the **main alert**.
* Monitoring tool updates for merged alerts are **not redirected** to the main alert and will be processed on the child alerts.
### Merge states
Each alert has a `mergeState` in addition to its lifecycle `status` (`pending`, `accepted`, `resolved`).
| Merge state | Description |
| ----------- | ---------------------------------------------------------------------------------------- |
| **NONE** | Regular alert (not merged). |
| **MAIN** | The primary alert other alerts were merged into. |
| **MERGED** | A secondary alert merged into another. Escalations stop; updates are no longer accepted. |
### Merging alerts
#### From the alert list
1. Navigate to **Alerts**.
2. Select two or more alerts.
3. Click **Merge** in the bulk action toolbar.
4. In the dialog, choose which alert is the **main alert**.
5. Confirm with **Merge n alerts**.
#### From the alert detail
1. Open the alert detail view.
2. Click **More actions → Merge into another alert**.
3. Select the target main alert and confirm.
### What Happens After a Merge
| Area | Behavior |
| ------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| **Escalations & notifications** | Stop immediately on merged alerts. |
| **Timeline** | Timeline entries will be added to both the main and the child alert regarding the merge operation. |
| **War room channels** | Connected channels of child alerts will receive a message regarding the merge operation. |
| **Links, details, summaries** | Not merged. |
| **Responders** | Not merged; cannot add new responders on child. |
| **Manual alert actions** | Executable on child; not merged into parent. |
| **Priority** | Becomes read-only on child. |
| **Logs & comments** | Stay attached to the merged (child) alert. |
| **Chat** | Becomes read-only; discussions continue in main alert chat. |
| **Deployment events** | Not merged. |
| **Linked incidents & services** | Not merged. |
| **Grouping stats** | Parent alert shows dedicated **Merged alerts** metric. |
| **Accept/Resolve** | Disabled on child. Main alert's resolve dialog includes "Also resolve (n) merged alerts" (enabled by default). |
### Access & permissions
* Merge operation requires **Responder** role or higher.
* User must have access to **all alerts** involved in the merge.
### Reports and Dashboards
By default, merged alerts are **excluded** from dashboards and reports for accurate MTTR and SLA calculations.
You can customize this via report filters:
| Option | Description |
| ------------------------------------- | ----------------------------------------------------- |
| **Exclude merged alerts** *(default)* | Only main and unmerged alerts are counted. |
| **Include merged alerts** | Includes all alerts, including merged ones. |
| **Only merged alerts** | Shows merged alerts only (useful for noise analysis). |
Dashboards always default to **Exclude merged**, with an optional toggle to include them.
### Alert merge vs event grouping
| Feature | Purpose | When it happens |
| ------------------ | --------------------------------------------------- | ------------------------ |
| **Event grouping** | Groups multiple incoming events into one alert | During event ingestion |
| **Alert merge** | Merges multiple existing alerts into one main alert | After alerts are created |
{% hint style="info" %}
*Event Grouping* happens automatically based on rules or AI similarity.\
\&#xNAN;*Alert Merge* is a manual, user-initiated operation.
{% endhint %}
### Best Practices
* Use **Event Grouping** to prevent noise.
* Use **Alert Merge** for cleanup and consolidation once alerts exist.
* Merge only related alerts to maintain data integrity.
* Always merge into the alert with the **most context** (responders, timeline, and chat).
# Alert source logs
Inspect the raw events an alert source received, and what ilert did with each one.
The alert source log is the record of what ilert actually received. When an event does not produce the alert you expected, this is the only place that says why — the API response never does.
Open an alert source and select the **Alert source logs** tab.
Each row is one received event, with the raw payload as it arrived.
## What a log entry shows
| Column | Contains |
| ------------- | ---------------------------------------------------------------- |
| **Timestamp** | When ilert received the event, in your profile's time zone |
| **Body** | The raw payload as it arrived, before any template or filter ran |
Open an entry to see the converted event, the alert operations it triggered, and any outbound alert actions that fired as a result.
## Finding an event
Use **Search** to match on keywords in the payload — an alert key, a hostname, a summary fragment.
The range defaults to a narrow window. Widen it with **Last 24 hours** or **7 days** before concluding an event never arrived; entries take a short time to appear, so an event sent seconds ago may not be listed yet.
{% hint style="info" %}
Event history is available from **March 2022**, and alert correlations from **December 2023**. Logs are not available for every integration type.
{% endhint %}
## Reading the result
A `202 Accepted` from the Events API means your event was queued, not that an alert was created. An event that was filtered out, or grouped into an alert that was already open, returns exactly the same response. The log is where those outcomes are visible:
* **The event is listed and an alert exists** — working as intended.
* **The event is listed but no new alert appeared** — it was grouped into an open alert, or an event filter rejected it. Check **Grouped events** on the open alert, then the source's filter conditions.
* **The event is not listed at all** — it never reached this alert source. The integration key is wrong, the payload is malformed, or the request never left your network.
For the full diagnostic path, see [Troubleshooting](/getting-started/quick-start-guide/troubleshooting).
# Dashboard
Build custom dashboards to monitor your team's key metrics and gain valuable insights into various aspects of ilert.
The ilert dashboard is a customizable page that allows you to keep track of your favorite team-related metrics and gain insights into different aspects of ilert. You can create multiple dashboards, each tailored to specific needs or teams. With a variety of widgets at your disposal, you can add, remove, and rearrange widgets to build a dashboard that fits your needs.
Dashboard example
### Create your first Dashboard
To start with dashboards, click **Create new dashboard**, and give it a name.
Choose one of the three visibility types:
* **View only**: The dashboard will be available for everyone within your organization, but they cannot change anything in it.
* **View and edit**: The dashboard will be available for everyone within your organization, and they can make changes to it.
* **Private**: the dashboard will only be available to you.
Navigate to the **plus icon** and choose one of six widgets: Metrics, On-call now, Open alerts, Open incidents, Recent alert activity, or Services. Later, you can also add widgets by clicking the **Add** button at the top right corner of your screen.
You can adjust each widget's settings by clicking the **cogs icon** at the top right corner of the widget. Below are more details on each widget and its settings.
### Widgets
**Metrics.** If you are using the [Metrics](/incidents-and-status-pages/metrics) feature, you can display essential graphs in the Dashboard. In the settings under the cogs icon, you can choose a specific [team](/users-and-access-management/teams) or metric and adjust the time range.
**On-call now.** This widget helps to observe current [on-call duty](/on-call-management-and-escalations/on-call-schedules) shifts. Adjust a specific team schedule or specify the [escalation policy](/on-call-management-and-escalations/escalation-policies) you want to see in the dashboard.
**Open alerts.** This widget shows ongoing [alerts](/alerting/working-with-alerts) that have yet to be resolved. By clicking the cogs icon, you can specify a team or [alert source](/alerting/configure-alerting/alert-sources).
**Open incidents**. The widget displays current [incidents](/incidents-and-status-pages/incidents), their status, and the services they affected. In the widget settings, you can specify teams and services.
**Recent alert activity.** Alert sources, time stamps, and activity details are available in this widget. You can choose to display all types of timeline events to display or limit the list to only those events that are important to you. Additionally, you can limit teams and alert sources shown here.
**Services.** This widget shows the status of important [services](/incidents-and-status-pages/services). Again, you can view all the alert sources and teams or include only those that are important to you.
You can adjust the appearance of your dashboard by dragging widgets and changing their sizes.
If you want to change the sharing settings, such as making the private widget public, click **Edit** at the top right corner, and then the **Visibility** link under the widget's title.
# Configure alerting
Connect your tools, decide which events deserve a page, and control when and how responders are notified.
Configuring alerting is mostly a question of noise. Every tool you connect can page someone; the settings in this section decide which signals are worth waking a person for, and which should be grouped, downgraded, or dropped.
Start with an alert source. Everything else is optional refinement.
## Where each setting belongs
Most alerting behavior is configured on the **alert source** itself — filtering, grouping, templates, priority, and which escalation policy to use. Reach for the other objects when the alert source alone cannot express what you need.
| Reach for | When |
| ------------------------- | ---------------------------------------------------------------------------------------------------------- |
| **Alert sources** | Always. This is where events arrive and where most noise is controlled. |
| **Support hours** | Off-hours pages should be quieter, but not silent. |
| **Maintenance windows** | Planned work will generate alerts nobody needs to see. |
| **Event flows** | Routing depends on conditions, needs a delay, or needs the payload transformed before it reaches a source. |
| **Notification settings** | You are changing how *you* are reached, rather than what gets alerted. |
{% hint style="warning" %}
**Support hours downgrade, they do not suppress.**
Outside the hours you define, ilert creates `LOW` priority alerts rather than withholding them. A `LOW` alert uses only the **first** escalation rule and never advances past it. Real silence lives in each responder's notification rules, or in an event flow branch that drops the event.
{% endhint %}
## In this section
Alert sources
Connect a tool, then control filtering, grouping, templates, and priority.
# Alert sources
Connect your third party tools to ilert.
An alert source represents the connection between your tools (usually a monitoring system, a ticketing tool, or an application) and ilert. We often refer to alert sources as **inbound integrations**.
ilert provides the following inbound integration options:
These are pre-built integrations by ilert and work-out-of the box with your monitoring tools. If you're missing a tool, feel free to suggest an integration that you'd like to see in ilert.
A heartbeat alert source will automatically create an alert if it does not receive a heartbeat signal from your app at regular intervals.
## Create an alert source
1. In the sidebar, go to **Alerting** → **Alert sources**, then click **Create new alert source**.
2. Select your integration type in the search field and click **Next**.
3. Give your alert source a name, optionally assign teams, and click **Next**.
4. Select an **escalation policy** by creating a new one or assigning an existing one.
5. Choose an [alert grouping](#event-grouping) mode — **Smart grouping**, **Time-based grouping**, or **Default grouping** — and click **Continue setup**. You can change it later.
6. The next page shows additional settings, such as custom alert templates or notification priority. Click on **Finish setup** for now.
## Alert template
With alert templates, you can create your own template for the alert summary and alert details using preset fields from the integration. Moreover, our templating lets you extract links from the alert payload. Extracted links will be added to the links section of an alert.
### Custom alert summary and details template
1. Click on **Alert sources -> Alert sources** and choose an alert source to edit
2. Navigate to the section **Alert template** and check the boxes for **Alert summary** and/or **Alert details**
3. Create your custom template by selecting the **fields** you want use and entering any static text.\
The available fields are specific to the integration.
{% hint style="info" %}
**Field colors and accessing raw fields**
* Blue fields are preset fields provided by ilert.
* Orange fields are extracted from past alerts in your account that were sent from the specific integration
* Grey fields lets you extract any raw fields from the JSON payload by typing the name of the custom field, e.g. `custom_field`. You can also access nested fields and arrays, e.g. `custom_field.array_field[5].nested_field`
{% endhint %}
#### Testing your templates before saving
Using the preview button you may try out your current template. By default, ilert will try to find one of the latest event payload's that was received by your alert source. If there is none present, we will render a fallback JSON doc, which you might alter as you like.
#### Using the template text syntax
Your alert source template fields will start in text mode by default. In text mode you may use the **Insert data...** dropdown to help you add template variables quickly (see [ITL](/developer-docs/itl-ilert-template-language) for how ilert parses event data into variables) - the text syntax works like this:
Type
Sample
Description
Text
Some text
You may of course add generic text content to your liking
Variable
{{var}}
Extract content of the event and insert it. Note: there is no further sanitizing of the values
Accessing nested variables
{{ var.subfield.evenMore }}
Access sub fields
Accessing fields of an array
{{ var.arrayField[0].more }}
Access array contents
See ITL below to learn more about the templating language and its features for blocks, loops and functions:
For blocks, loops, and functions, see [ITL — the ilert template language](/developer-docs/itl-ilert-template-language).
{% embed url="" %}
### Alert links
ilert can extract alert links from the alert payload. Extracted links will be added to the alert's links section.
| Alert link template | Alert with extracted link |
| ------------------------------------------------------------------- | -------------------------------- |
| |  |
## Dynamic escalation policy routing
With dynamic escalation policy routing, the escalation policy to be used will be determined based on the incoming alert, instead of always using the same escalation policy that is configured on the alert source.
To extract the escalation policy [routing key](/on-call-management-and-escalations/escalation-policies#routing-key-optional) from the alert payload, add a routing key template in the section **Escalation -> Dynamic routing**.
In the above example, the field `Group key` from the alert payload will be used as the routing key.
## Notification priority and support hours
### Default notification priority
By using notification priority, you can easily customise your alert notification based on your notification rules.
1. Click on **Alert sources -> Alert sources** and choose an alert source to edit
2. Scroll down to the section **Notification priority** and set your desired **Notification priority**
ilert provides different priority settings to customize your alerts.
* **High (with escalation):** You will be notified based on your [high-priority notification rules](/alerting/configure-alerting/notification-settings#define-notification-rules) and an alert can be escalated based on escalation policy.
* **Low (no escalation):** You will be notified based on your low-priority notification rules and an alert cannot be escalated.
### Support hours based notification priority
ilert also lets you dynamically set the notification priority based on the alert source's [support hours](/alerting/configure-alerting/support-hours). This lets you, for instance, use more obtrusive notification methods like phone calls outside of business hours and use not so obtrusive ones during business hours.
* **High during support hours, low priority otherwise:** During your support hours, you are notified based on your high priority notification rules. At all other times, you are notified based on your low priority notification rules.
* **Low during support hours, high priority otherwise:** During your support hours, you are notified based on your low priority notification rules. At all other times, you are notified based on your high priority notification rules.
If you select **High during support hours, low priority otherwise,** you can choose to **Raise priority of all pending alerts** by ticking the checkbox located under the support hour selection. All your pending alerts for the current alert source will be raised to "high" when your support hours **begin**.
If you select **Low during support hours, high priority otherwise,** you can choose to **Raise priority of all pending alerts** by ticking the checkbox located under the support hour selection. All your pending alerts for the current alert source will be raised to "high" when your support hours **end**.
### Dynamic priority mapping
With dynamic priority mapping, you can use alert fields to extract and map notification priority. This will overwrite default priority, if enabled.
To enable dynamic priority mapping
1. Click on **Alert sources -> Alert sources** and choose an alert source to edit
2. Scroll down to the section **Notification priority** and check **Enable dynamic priority mapping**
3. Enter template to to extract the priority field from the alert payload
4. Add priority mappings. A priority mapping maps an extracted value from the alert payload to the ilert priority
{% hint style="info" %}
ilert will fallback to the alert source's default priority, if a priority could not be extracted.
{% endhint %}
## Event filter
Sometimes flagging alerts as low priority is not enough and it is necessary to drop events completely. e.g. `Grafana DatasourceNoData` This is why you can configure one or multiple event filter groups for your alert source to process only desired events into alerts.
You may choose between properties of the known **ilert event payload** such as priority or summary, some more **advanced dynamic fields** like **trigger counts** which allow you to define your own rate limits as well as **schedules** and **support hours** to fine tune accept windows - and of course **custom payload fields**.
If you wish for even more customization you can switch the UI into the code editor mode and get full access to the ICL, find out more about it here:
{% content-ref url="/spaces/684bnaGsqSFjESghpiiu/pages/PPDm0dAdQr995luzdAwH" %}
[ICL - ilert condition language](/developer-docs/icl-ilert-condition-language)
{% endcontent-ref %}
## Event grouping
Event grouping helps you reduce noise by clustering related events within a defined time window or by allowing only one open alert at a time per source.
Enable alert grouping during alert source creation or in the alert source's advanced settings
There are 5 types of alert grouping available:
* Native integration based grouping
* Time-based grouping
* Grouping until accepted
* Grouping until resolved
* ilert AI similarity based grouping
### Native integration based grouping
By default every alert source attempts to offer the best experience based on the features that the corresponding third party integration tool has available. Some tools offer more e.g. resolve events or proper alertKeys to group events, some tools offer a plain webhook without any additional context. ilert shows the integration features in the creation wizard:
Integrations such as Autotask, Jira, Grafana or Prometheus provide rich payloads which ilert automatically uses to identify alertKey and eventTypes, which are used to automatically group incoming events, if an unresolved alert with the same identifier is found.
### Time based grouping
An alert source with alert grouping enabled will group together alerts triggered within the defined time window and create only one alert. Grouped alerts will show up as events in the alert's timeline. You can select relative time windows e.g. 2 minutes, 5 minutes, etc.
### Grouping until accepted / resolved
Besides static relative windows you can also define action-based windows. By doing so alerts will be grouped until all alerts of the same alert source are accepted or resolved, before opening a new alert. (*You find these options at the end of the time-based selector*)
### ilert AI based similarity grouping
Check out the link below to see more about AI based grouping:
{% content-ref url="/pages/QPRTjVTK9BPys8bsuvWQ" %}
[Intelligent alert grouping with ilert AI](/alerting/configure-alerting/alert-sources/using-ilert-ai-for-alert-grouping)
{% endcontent-ref %}
# Intelligent alert grouping with ilert AI
Use ilert AI to group similar alerts automatically, reducing alert noise and enabling faster identification of root causes.
Besides window based grouping and action-window based grouping we have introduced event content similarity grouping using ilert AI for alert sources in 2024. With a few clicks users can set up and fine tune their similarity grouping for each alert source.
{% hint style="warning" %}
ilert AI for alert grouping **does not share data** with any third-parties in its AI processing, all models (and vector databases) that are used as part of the pipeline are (trained), maintained and hosted on ilert's infrastructure.
{% endhint %}
To get started head to your alert source's edit view and look in the **Advanced settings** section for the grouping options. Choose **Group by content similarity**.
Content similarity grouping comes with two options that can be adjusted to fine tune the grouping behaviour. You can come back and adjust these at any time.
## Grouping window
You may choose a larger value here if you are confident in your **Similarity threshold** setting or your monitoring tool events might occur over a larger timespan. In case you are still getting comfortable with your threshold it makes sense to leave the window small to prevent you from missing on alerts. If your main goal is to prevent flooding 5-10 minutes should suffice. If you want precisive stitching a 12-24 hour window might suit best.
## Similarity threshold
A value between **0.1 <-->0.995** where a value close to **1** describes almost identical alerts (**less grouping**) and a value closer to **0** describes less identical (*0 = completely different*) alerts (**more grouping**). You can fine tune to as low as **0.005** steps.
Clicking on the **set threshold** option will open a modal that runs an on demand clustering algorithm on up to 1000 alerts in the last 60 days that this alert source has received, every time you adjust the slider for threshold value.
A grouping preview will show you the sharpness of the selected threshold based on your live alerts that the alert source has processed in the described window.
{% hint style="info" %}
Remember you need to have sent a few alerts to the alert source to see any grouping preview results (the set threshold option is not available during the creation wizard)
{% endhint %}
## Fine-tuning the threshold value
As described, a higher threshold will result in more precise grouping based on the content of the alerts, shown in our sample above as no alert is grouped at all above. If we lower the threshold to 0.695 we see alerts being grouped.
Lowering the threshold further our group continues to form over all desired alert contents:
Seeing this it seems that for our sample use case might have a perfect threshold at around 0.53 <--> 0.63. If we continue to reduce our threshold we can see that our grouping becomes too generic:
As all alerts are suddenly grouped in a single group.
{% hint style="info" %}
Note that this is a sample case with demo content, in the wild monitoring tool specific payloads have large contents filled with technical language that will behave differently for each use case, so fine-tuning as shown above will be required to reach your desired behaviour.
{% endhint %}
## Alert grouping metrics
As soon as you enable alert grouping, new metrics will appear in the **Alert source settings**. You can adjust a timeframe to see how effective the feature is for the alert source for a specified period. The following metrics are available:
* **Reduced alert volume.** The percentage by which the total number of alerts decreases after grouping similar alerts.
* **Grouping precision.** The percentage of correctly grouped related alerts based on feedback from responders in the alert detail view.
* **Improved response time.** The reduction in time to resolve alerts after using alert grouping, assuming each grouped alert saves 30 seconds.
You can leave feedback for every group of alerts. To do so, click the alert and find 👍 and 👎 icons in the **Alert details** section.
## Video Tutorial: How to enable Intelligent alert grouping
{% embed url="" %}
# Support hours
Define support hours to downgrade the priority of alerts raised outside the hours your team covers.
Define support hours for alert sources and call routing numbers to manage the priority of alert notifications and to route calls to different targets.
Support hours **downgrade** alerts, they do not withhold them. Outside the window you define, ilert creates `LOW` priority alerts instead of `HIGH` ones, and a `LOW` alert uses only the first escalation rule and never advances past it. If you expected silence overnight and got a quiet page instead, this is why.
## Create support hours
1. In the sidebar, go to **Alerting** → **Support hours**
2. Click on **Create new support hours**
3. Select the team to which the support hours belong to (if any), give it a name, and enter your support hours.
4. Click **Save**. You can now use these support hours in [alert sources](/alerting/configure-alerting/alert-sources#notification-priority-and-support-hours) and call routing numbers to set notification priority of alerts and route incoming call routing calls to different targets based on support hours. You can also have a [recurring on-call schedule](/on-call-management-and-escalations/on-call-schedules/recurring-schedules#follow-support-hours-respects-holidays) follow these support hours, so your on-call coverage respects the same holidays.
{% embed url="" %}
## Holidays
The holiday feature is a part of Support hours in ilert. It provides a smart way to handle exceptions to your regular support schedule—like national holidays, company-wide days off, or any non-standard workday—without needing to adjust your on-call rotations or escalation policies manually.
To set up holidays, choose the Support hours you would like to adjust and click **Edit**.
At the bottom, you will see the "**Holiday exceptions**" section.
You can either import holidays from a calendar or manually create them. If you decide on the first option, you can adjust the location and the exact dates you want to include or exclude. Adjust the time and click **Import**.
If you want to add exceptions manually, you will need to assign the title, duration, and support status for this period.
## Use support hours in recurring on-call schedules
Support hours — including their holiday exceptions — can drive on-call coverage directly. In a [recurring schedule](/on-call-management-and-escalations/on-call-schedules/recurring-schedules), set a layer's coverage to **Follow support hours (respects holidays)** and pick a support hour. The layer then follows that support hour's weekly windows and holidays, so the rotation automatically pauses on non-working holidays and picks up on days with extra coverage — no manual restriction editing required.
Coverage is copied into the schedule, not linked live, so historical shifts never change. When you edit the support hour, every schedule that follows it is re-synced going forward.
### Where a support hour is used
The support hour's detail page lists everywhere it is referenced under **Used in** — both the alert sources and the recurring schedules that use it, each marked with an icon for its type. A support hour cannot be deleted while a schedule still follows it.
# Maintenance windows
Keep your alert sources quiet and service subscribers informed during maintenance.
ilert's Maintenance Windows feature allows users to schedule downtime for alert sources and services. This ensures that on-call responders won't receive alerts from alert sources during maintenance and service, and status page subscribers will be informed about planned and ongoing service maintenance.
## Overview
* **Alert Sources**: When placed in maintenance mode, they will not trigger any alerts.
* **Services**: If a service is put into maintenance mode, its status will be updated across all associated status pages.
* **Maintenance notifications**: Optionally inform service and status page subscribers about the maintenance in advance and / or at the start / end of the maintenance.
* **Templates**: Incorporate incident templates in your maintenance messages for consistent communication.
## Setting up a Maintenance Window
1. In the sidebar, go to **Alerting** → **Maintenance windows**. They are also reachable under **Incident comms** → **Status pages**.
2. Click on **Schedule maintenance**.
3. Provide a short summary and optionally message for your maintenance. You may also apply a [template](#using-incident-templates) to fill this information.
4. Define the **Start** and **End** time for the maintenance window.
5. Select the **Alert sources** and **Services** you wish to put into maintenance mode.
6. (For services only) Set your **notification preferences** on when to notify service and status pages subscribers
* **Notify Immediately**: Subscribers are informed as soon as the maintenance window is created.
* **Notify at Start**: Subscribers receive a notification at the beginning of the maintenance window.
* **Notify at End**: Subscribers are informed once the maintenance period concludes.
7. Click **Schedule maintenance window**.
{% hint style="info" %}
**"Message" field supports markdown**
The "Message" field in Maintenance Windows supports [markdown](https://www.markdownguide.org/), allowing you to format text with bold, italics, links, and more. Use it to make maintenance messages clearer and easier to read.
{% endhint %}
### Notifications
Keeping subscribers informed is pivotal for maintaining trust. With ilert's flexible notification settings, you can ensure that your subscribers are always in the loop about any scheduled downtimes:
* **Immediate notification**: This is beneficial for any urgent maintenance windows or for those that you want to give your subscribers ample time to prepare for.
* **24 hours before the maintenance starts**.
* **1 hour before this maintenance starts.**
* **Start notification**: A gentle reminder that a scheduled downtime is commencing.
* **End notification**: Inform subscribers once the service is back online.
### Using incident templates
For teams that have established incident communication templates, ilert allows you to use these templates when drafting your maintenance messages. This ensures that your messages maintain a consistent tone and format, further establishing trust and clarity with your subscribers.
## Maintenance mode and status pages
For services in maintenance mode, their status will be reflected on all associated status pages. This transparency ensures that stakeholders and customers are informed about the service's downtime, ensuring there are no surprises.
## Conclusion
The Maintenance Windows feature in ilert ensures that necessary downtimes don't interfere with your operations. By setting up maintenance windows, keeping subscribers informed, and using consistent messaging, you can maintain smooth operations and keep stakeholder trust intact.
# Notification settings
Configure how ilert should notify you. Choose the most convenient channels, add contacts, and when you want to be notified.
Each user defines in their profile how they will be notified about alerts, incidents and on-call shifts. To manage your notification settings, click on your avatar in the top right and go to **notification settings**.
{% hint style="info" %}
A user with admin rights can also change the notification settings for other users.
{% endhint %}
Before you can receive notifications from ilert, you need to:
1. Add notification contacts (such as an email address or phone number) and activate support channels (e.g. SMS, WhatsApp)
2. Define notification rules, e.g. "for every high priority alert, first send me an SMS and call me after 5 minutes, if I do not respond to the alert".
## Add notification contacts and activate supported channels
ilert lets you add email addresses, phone numbers and push notification devices that you can use to set up notification channels.
{% hint style="info" %}
ilert creates one email contact based on your login email by default. Push notification devices will appear automatically once you login to ilert via one of our mobile apps for Android or iOS.
{% endhint %}
To add a notification contact, click on the **Add email** or **Add phone number** link.
Once you have added a notification contact, a list of supported notification channels will appear. Before you can send notifications to a contact through a channel, you need to activate the respective notification channel.
When you activate a notification channel, ilert will ask you for a verification code that will be sent to the respective channel.
{% hint style="info" %}
Activating a phone number via SMS will also activate phone call notifications.
{% endhint %}
The process for activating a notification channel differs from channel to channel. Please follow the instructions provided during activation. ilert supports the following notification channels:
* Email
* SMS
* Phone calls
* iOS and Android push notification
* WhatsApp
* Telegram
Once activation was successful, you are able to test the notification channel by clicking on the **Send test notification** link.\
\
## Define notification rules
There are 5 different types of notification rules to configure:
Category
High priority alerts
notifies, when a high priority alert is assigned to the user (initially on alert creation, or when added as responder)
Low priority alerts
notifies, when a low priority alert is assigned to the user (initially on alert creation, or when added as responder)
Alert status updates
notifies, when user is already assigned to an alert and the alert status changes (e.g. another user accepts the alert)
Incident notifications
notifies, when user is added to an incident as a subscriber (initially on incident creation, or when added manually)
On-call notifications
notifies at the given point of time before the user is being on-call
To define notification rules
1. Choose a notification type (e.g. **High priority alerts**), expand the section and click on **Add notification rule**
2. Configure the notification rule by selecting a contact and channel and click on **Save**.
For alert notification rules, you will be able to set delays between notifications, like the following:
Above rules will first notify you via Push (on all push devices), then (after 3 minutes) ilert will send you an SMS, unless you respond to the alert within 3 minutes. In this case, it will not notify you via SMS.
Alert notifications in ilert are bi-directional, that is, you can respond to a notification using the same channel on which you were notified (without logging into ilert), e.g. by replying to an SMS.
You have the following response options:
1. Acknowledge the alert
2. Mark the alert as resolved
3. Escalation to the next user in the alert's escalation policy
# Mute notifications
Activate the Mute Notifications feature to silence all notifications from ilert channels during periods when you're unavailable, such as vacations.
To enable this feature:
1. Click on your avatar in the top-right corner of the ilert interface.
2. Toggle the 'Mute Notifications' switch to the on position.
**Setting the Quiet Period:**
You can specify the duration for which notifications should be muted:
* For predefined durations, select one of the available options.
* To define a custom period, click 'Custom…' and enter your start and end dates/times.
{% hint style="info" %}
It is important to know that incoming calls originating from call routing numbers are not muted. We advise to implement schedule overrides as a way to manage these cases.
{% endhint %}
**Visibility to Colleagues:**
When 'Mute Notifications' is active, your avatar will display a crossed-out bell icon. This signals to your team members that you are not receiving notifications during this time.
{% embed url="" %}
Check out our step-by-step tutorial on Youtube
{% endembed %}
# Event flows
Create automated workflows to streamline processing, routing, and escalating events in ilert.
Event flows in ilert enable teams to automate and streamline the processing of incoming events. With event flows, you can build visual workflows that evaluate, branch, and route events dynamically based on conditions, support hours, or alert source configurations. This helps reduce manual effort, improve response times, and ensure incidents are handled appropriately.
## Create a new event flow
1. Navigate to **Alert sources → Event flows** from the top navigation menu.
\
2\. Click **Create new event flow** on the top right or by clicking a blue button.
3. **Enter a name** for the event flow and assign it to one or more teams (*optional*).
## Add an incoming event
1. **Click the Incoming event** block to define how events will trigger the flow.
2. **Select one or more integrations** from the dropdown to associate them with the flow.
3. After selecting integrations, you'll see the generated **API endpoint**. Use this endpoint to send events. ilert allows you to use these **integrationKeys / integrationURLs** as a drop-in replacement of alert source versions right within monitoring tools.
## Add flow logic: Branches, alert sources, and support hours
Click the **+ icon** to add logic blocks after the incoming event. You can choose from the following nodes:
* **Branch**: Create various scenarios depending on the conditions.
* **Route to alert source**: Direct all alerts to a specified alert source.
* **Support hours**: Route events based on the predefined [Support hours](/alerting/configure-alerting/support-hours).
* **Wait**: Lets you temporarily pause an event flow before the next step runs.
{% hint style="info" %}
Many more blocks are coming soon, allowing for transformation, webhook calls, and more. Please reach out to us at if you have suggestions or feature requests.
{% endhint %}
### Branch: Route events based on conditions
1. Select **Branch** to define logic based on filters like `context.event.summary`, `context.event.customDetails.labels`, etc.
2. Add filter groups **using AND / OR logic** to create conditional branches.
3. Add more branches as needed. The first matching branch will be executed. If no branch matches, the event proceeds to the **Else** path.
### Route to alert source: Direct all alerts to a specified alert source
1. Select the **Route to alert source** to send the event to an existing alert source.
2. Optionally override the alert source defaults by selecting a different escalation policy or notification priority.
### Support hours: Route based on working time
1. Select **Support hours** to route events during or outside business hours.
2. Choose Support hours (configured separately under [Support hours](/alerting/configure-alerting/support-hours)).
3. Events are then routed based on their arrival time.
#### Wait: Pause for a specific duration
1. Enter a **title** to describe the pause, e.g., "Hold before escalate."
2. Under **Wait for**, keep **Duration** selected.
3. Enter the **duration value** (e.g., `5`) and choose the **time unit** from the dropdown (e.g., **minutes**).
4. Click **Close**, then **Save** your flow.
## Finalize and activate the flow
You can **copy or cut a node or a subtree** by clicking a three-dot icon.
Once all branches and logic have been added, click **Save** in the top-right corner of the screen. The flow is now active and will start processing events from the selected integrations.
## FAQ
**Q: Can I reuse the same integration in multiple event flows?**\
A: You can create multiple **integrationKey/integrationUrls** of the same integration type for multiple event flows.
**Q: What happens if no branch of a branch node matches in a flow?**\
A: If no branch conditions match, the event proceeds to the Else path by default.
**Q: Where do I configure Support hours?**\
A: Navigate to Alert sources → Support hours to define and edit team working hours.
# Heartbeat monitoring
Set up ilert heartbeat monitoring for the simplest way to monitor service, device, or workflow health.
ilert's heartbeat alert sources allow you to monitor services, devices, or workflows easily — depending only on an HTTPS outbound connection. It checks whether your systems or services are still running by expecting regular signals, and you get alerted automatically when those signals stop, indicating a failure or outage.
To start monitoring via ilert heartbeat, log into your ilert account, click Alert sources, and find **Heartbeat monitors** in the dropdown menu. Then, click **Create heartbeat monitor**.
Add new monitor to the [team](/users-and-access-management/teams), name it, and choose an [alert source](/alerting/configure-alerting/alert-sources) that will trigger an alert if the heartbeat ping is overdue.
Choose an interval after which the heartbeat will create an alert through the given alert source if it does not receive a response. You can choose between seconds, minutes, and hours.
Finally, provide a summary of the alert to help your team quickly identify the issue.
When ready, click **Create**. You will see the Monitor's details view, where the **integration URL** (API key) will be shown. You can use this key to call ping requests for your heartbeat alert source.
The heartbeat timer starts once the first ping is received. The heartbeat monitor will not do its job until it is activated by receiving the first ping.
If you need to update the monitor, click the **pencil** icon. You can also delete your monitor using the same menu by clicking the bin icon.
To adjust alerting settings for your newly created Heartbeat monitor, go to **Alert sources**, then, again, **Alert sources,** and choose **Heartbeat** from the list of integrations. You can have many heartbeats pointing to the same alert source and have unified alerting settings for all of them.
You can find more examples on how to implement heartbeats here:
{% content-ref url="/pages/-MJ3xwrMP9m9qyteB53i" %}
[Prometheus Heartbeat Example](/alerting/heartbeat-monitoring/prometheus-heartbeat-example)
{% endcontent-ref %}
{% content-ref url="/pages/-MJ3wu5-5cROcxAzcSAH" %}
[CLI Heartbeat Examples](/alerting/heartbeat-monitoring/cli-heartbeat-examples)
{% endcontent-ref %}
# Prometheus Heartbeat Example
Monitor the liveness of your Prometheus and Alertmanager stack with an always-firing alert that pings an ilert heartbeat monitor.
This recipe monitors the health of your Prometheus + Alertmanager stack itself. The idea is a **dead man's switch**: you define a single Prometheus alert that is *always* firing and route it through Alertmanager to an ilert heartbeat monitor. As long as the whole pipeline is healthy, Alertmanager keeps pinging the heartbeat endpoint at a regular interval. The moment Prometheus stops evaluating rules, Alertmanager goes down, or the network to ilert breaks, the pings stop — and ilert raises an alert because the heartbeat is overdue.
The flow looks like this:
```
Prometheus (always-firing rule) → Alertmanager (webhook receiver) → ilert heartbeat endpoint
```
## Prerequisite: Create a heartbeat monitor in ilert
Follow the steps in [Heartbeat monitoring](/alerting/heartbeat-monitoring) to create a heartbeat monitor. When choosing the interval, pick a value that is **longer** than the `repeat_interval` you will configure in Alertmanager below (for example, a 5-minute heartbeat interval with a 1-minute repeat interval). This gives Alertmanager several chances to ping before ilert considers the heartbeat overdue.
After creating the monitor, copy its **integration URL**. It has the following form:
```
https://beat.ilert.com/api/pings/${YOUR-APIKEY}
```
## Step 1: Define an always-firing alert in Prometheus
Prometheus does not send anything to Alertmanager unless an alert is firing. To generate a continuous signal, add an alerting rule whose expression is always true. Create a rule file, e.g. **ilert\_heartbeat\_rules.yml**:
```yaml
groups:
- name: ilert-heartbeat
rules:
- alert: iLert
expr: vector(1)
labels:
severity: none
annotations:
summary: "Prometheus/Alertmanager liveness heartbeat for ilert"
```
The expression `vector(1)` always evaluates to `1`, so this alert fires continuously and never resolves. The alert name `iLert` is what Alertmanager matches on in Step 3.
## Step 2: Load the rule and point Prometheus at Alertmanager
In your **prometheus.yml**, register the rule file and make sure Prometheus is configured to send alerts to your Alertmanager instance. These are the sections to **add to your existing `prometheus.yml`** — merge them in alongside your current `global`, `scrape_configs`, and other settings rather than replacing the whole file.
```yaml
rule_files:
- "ilert_heartbeat_rules.yml"
alerting:
alertmanagers:
- static_configs:
- targets:
- "alertmanager:9093"
```
{% hint style="warning" %}
The target `alertmanager:9093` is a **placeholder** — replace it with wherever your Alertmanager is actually reachable. `9093` is Alertmanager's default HTTP port, but the hostname depends on how you run it.
{% endhint %}
Reload Prometheus (`SIGHUP` or `POST /-/reload`) and confirm the `iLert` alert appears as **FIRING** under **Status → Alerts** in the Prometheus web UI.
## Step 3: Route the alert to ilert in Alertmanager
In your **AM.yml**, add a route that matches the `iLert` alert and forwards it to the heartbeat endpoint as a webhook receiver. If you already have routes and receivers, merge the `ilert` entries into your existing `route.routes` and `receivers` lists rather than replacing the file. The `repeat_interval` controls how often Alertmanager re-sends the notification — i.e. how often ilert is pinged — so keep it well below your heartbeat interval.
```yaml
route:
receiver: default
group_by:
- job
routes:
- receiver: ilert
match:
alertname: iLert
repeat_interval: 1m
receivers:
- name: ilert
webhook_configs:
- url: 'https://beat.ilert.com/api/pings/${YOUR-APIKEY}'
send_resolved: false
```
Reload Alertmanager to apply the configuration.
## How it works
* The `iLert` alert fires permanently, so Alertmanager receives a firing notification and re-sends it every `repeat_interval` (1 minute above).
* Each notification is an HTTP POST to `https://beat.ilert.com/api/pings/${YOUR-APIKEY}`, which counts as a ping to your ilert heartbeat monitor.
* As long as pings keep arriving within the monitor's interval, the heartbeat stays healthy and no alert is created.
* If Prometheus, Alertmanager, or the connection to ilert fails, the pings stop. Once the heartbeat interval elapses without a ping, ilert raises an alert through the alert source and escalation policy you assigned to the monitor.
## Related articles
{% content-ref url="/pages/-M9bd2vgpr1x\_yIPXlIE" %}
[Heartbeat monitoring](/alerting/heartbeat-monitoring)
{% endcontent-ref %}
{% content-ref url="/pages/-M9ScNiNyfedtovre9wO" %}
[Prometheus Integration](/integrations/inbound-integrations/prometheus)
{% endcontent-ref %}
# CLI Heartbeat Examples
Command line interface heartbeat examples
## Simple Heartbeat
Simple heartbeat call using curl
```
curl https://beat.ilert.com/api/pings/${YOUR-APIKEY}
```
## Cronjob Backup
Heartbeat after successful backup script cronjob execution
```
run_backup.sh && curl https://beat.ilert.com/api/pings/${YOUR-APIKEY}
```
# Email Heartbeat Example
Besides HTTP/CLI pings, ilert heartbeats can also be triggered by email. Each heartbeat monitor provides a unique Integration Email address.
If your system sends an email to this address within the configured interval, the heartbeat stays green. If no email arrives in time, the monitor becomes overdue and triggers an alert.
This option is intended for environments where outgoing HTTP(S) requests are not possible (for example due to firewall or network restrictions), but the monitored system can still send emails via SMTP.
To use email heartbeats, simply send an email periodically to the Integration Email address of the heartbeat monitor. The subject and body are optional — only successful delivery matters.
#### Example: send a heartbeat email with mail
```shellscript
echo "heartbeat ok" | mail -s "ilert heartbeat"
```
#### Example: send with sendmail
```shellscript
printf "Subject: ilert heartbeat\n\nheartbeat ok\n" | sendmail
```
#### Example: send with curl to SMTP server (if mail tools are not installed)
```shellscript
curl --url "smtp://your.smtp.server:25" \
--mail-from "heartbeat@yourdomain.com" \
--mail-rcpt "" \
--upload-file - <
If a deployment event can be correlated to an alert, ilert will enrich your alert's context with the most relevant deployment information. Allowing you to save precious time when trying to resolve the alert's origin and in the end reducing MTTR.
## How correlation grows in larger accounts
### Getting started fast: account-wide and time-based
To get you started fast, deployment events support a global account-wide alert mapping, which purely runs on a time-based correlation. If the alert is in a specific window of a deployment it will be mapped in its context.
{% hint style="info" %}
Let's take [GitHub](/integrations/deployment-integrations/github) as an example. The fastest way to kick off your integration would be to set up a global GitHub organization webhook sending Releases or Merge events of all repositories to a single GitHub deployment pipeline in ilert. This will immediately start collecting and mapping deployment relations.
{% endhint %}
For ilert customers that are compact enough not to use the [Teams feature](/users-and-access-management/teams), this might already do the trick, and no further fine-grained configuration might be needed.
### Scaling in larger organizations: team-based relation claiming
However, for larger or enterprise customers, the global approach might help adopt the feature quickly, but a time-based correlation at a large scale will no longer provide relevant mappings. **Here is where teams can start taking the matter into their own hands** if they wish to receive more relevant deployment mappings for their own resources.
All they need to do is set up their own deployment pipeline that hooks into their own repository events. While making sure that their deployment pipeline is assigned to their team, just like their alert sources and (even more important) escalation policies.
{% hint style="info" %}
If ilert finds a related alert resource, such as an escalation policy or alert source, with a team assignment while also finding at least one deployment pipeline with the same team assignment, it will only correlate deployment events from the specific team-owned deployment pipeline to the alert.
{% endhint %}
By assigning their team to policies and pipelines, a team can individually claim their specific deployment events for their alerts while seamlessly moving away from account-wide deployment events. This allows admins to introduce the feature quickly while still providing teams with the option to migrate to fine-grained correlations on their own—empowering admins and teams and finally reducing MTTR.
## Integrations
Jump into available native integrations or build your own using the generic API:
{% content-ref url="/pages/d03jP9Bn0M0ywWLe3Ad9" %}
[Deployment integrations](/integrations/deployment-integrations)
{% endcontent-ref %}
# Escalation policies
Define escalation policies in ilert to guarantee that alerts are delivered through various channels until acknowledged or resolved.
Escalation policies define whom to assign an alert to when an alert is triggered by an alert source. They consist of rules that specify a target (a user, a team, or a schedule) and an escalation timeout.
By defining multiple escalation rules, you can ensure that an alert doesn't get missed if the first responder fails to acknowledge the alert.
To create an escalation policy, navigate to **Escalation policies** and click on **Create new escalation policy**. Give it a name and optionally assign the policy to a team.
## Escalation rules
An escalation policy must have at least one escalation rule. You can add more rules by clicking on the **Add escalation rule** link. An escalation rule consists of:
* A **target**: an individual user, a team or a schedule. If the target is a schedule, the user on-call at the time of the alert assignment will be added as a responder to an alert. If an alert occurs during a time with no coverage in your schedule, then the alert will be escalated immediately to the next escalation level, without waiting for the escalation timeout. If no one is on-call on the entire escalation policy, no one will be notified.
* An **escalation timeout**: the amount of time that a responder has to accept an alert until it is automatically escalated to the next escalation level.
Each level names its target and the time the previous level had to respond. Used in lists every alert source relying on this policy.
{% hint style="info" %}
**Alerting multiple users at the same time**
A single escalation level can hold several targets, and they are all notified at once. How many depends on your plan; the limit is shown inline on the escalation rule when you edit the policy.
{% endhint %}
## Delayed escalation
With delayed escalation, you can delay escalation by a set period of time (up to 60 minutes). That way, you can avoid getting paged for alerts that quickly resolve themselves. When an alert is created and resolved before the defined delay has passed, ilert won't notify anyone. You can still have those alerts posted somewhere (e.g. Slack) by creating an alert action that gets triggered when an alert is created.
Set a delay at the top of an escalation policy
## Routing key (optional)
By default, an alert source is associated with a single escalation policy and always uses the same policy for every alert that it creates. An escalation policy's routing key can be used in the Events API to overwrite an alert source's escalation policy and route alerts dynamically based on the escalation policy's routing key. You can define multiple routing keys in an escalation policy by separating them with commas in the routing key field. The Events API also accepts multiple routing keys in the `routingKey` field when separated by commas and evaluates them from left to right. The first routing key that matches an escalation policy will be used. If no routing key matches, ilert uses the escalation policy assigned to the alert source. You can also extract the routing from the alert payload using a [routing key template](/alerting/configure-alerting/alert-sources#dynamic-escalation-policy-routing).
# On-call schedules
Plan on‑call schedules in ilert to orchestrate team rotation, ensure coverage for incidents, and adhere to SLA response windows.
A schedule decides who is on call at any given moment, so an alert can be assigned to a person rather than to a name someone hard-coded months ago.
A few things worth knowing before you build one:
* Only one user per schedule can be on-call at a time.
* A schedule notifies nobody on its own. Its members are only paged when the schedule is referenced by an escalation policy.
* To preserve the history of a schedule, any changes to schedules apply to current and future dates only. Shifts that are in the past cannot be deleted or modified.
* Schedules can be embedded into any calendar application that supports the iCal format.
Who is on call now, who is next, and which escalation policy the schedule feeds.
## Create or modify a schedule
{% hint style="info" %}
**User role permissions required**
* [User role permissions](/users-and-access-management/user-roles-and-permissions) are required to create or modify a schedule.
* Stakeholder users cannot be added to a schedule.
{% endhint %}
ilert offers two types of schedules - **recurring and static schedules**, which differ in the way a schedule is created and maintained. The schedule type cannot be changed after its creation.
| [Recurring schedule](/on-call-management-and-escalations/on-call-schedules/recurring-schedules) | [Static schedule](/on-call-management-and-escalations/on-call-schedules/static-schedules) |
| ------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
Use if shifts are recurring.
Example: every team member is on-call for a week.
Shifts are created automatically
|
Use if you cannot plan your schedule beyond a specific time frame and want to schedule them manually on a regular basis.
Shifts are created manually by dragging and dropping users onto the calendar
|
## Overrides
{% hint style="info" %}
**Responder role permissions required**
Overrides can be added by users with **Responder** role privileges. A responder can only add themselves as an override. Users with **User** privileges can add any user as an override.
{% endhint %}
Overrides are one-time changes to a schedule. Example uses of overrides include
* when a user becomes sick, goes on vacation, or would like to swap a shift with another user
* scheduling different shifts for holidays
An override leaves the underlying rotation untouched. Nothing about the recurring pattern changes; the override simply wins for the period it covers, then the normal schedule resumes.
### Add an override in the web app
Open the schedule and click **Schedule override**, or click directly on a shift in the timeline.
Start and End are prefilled from the current time. The links beneath each field set common durations without typing a date.
Pick the user taking the shift, then set the period. Rather than typing dates, use the shortcuts:
| Field | Shortcuts |
| --------- | -------------------------------- |
| **Start** | `now`, `tomorrow`, `next week` |
| **End** | `+1 hour`, `+3 hours`, `+1 week` |
Click **Save**.
Overrides can themselves be overridden by a later override. To remove one, click the override on the timeline and then the red **×**.
You can also start an override from the schedules list, using the **Override** button on any schedule row.
{% embed url="" %}
A step-by-step walkthrough on YouTube
{% endembed %}
### Adding overrides in the mobile app
Open the **Who is on-call?** screen, then tap **Override shift** on the shift you want to cover. Pick the user taking over, set the start and end, and tap **Done**.
This is the reason overrides only need the Responder role: someone can hand off a shift from their phone at 2am without an administrator.
### On-call widget in the mobile app
The on-call widget in the navigation menu of the mobile app displays your current on-call status and quickly lets you take someone else's on-call.
{% hint style="info" %}
The on-call status is based on the selected policy levels from the [My on-call shifts](/on-call-management-and-escalations/on-call-schedules/my-on-call-shifts) view. Therefore, the status **Not on-call** means that you are not on-call in any of the selected policies, but you could be on-call in other policies. Click on the **Select policies** link to modify your policy selection
{% endhint %}
## FAQ
### What happens if there is a gap in a schedule?
If an alert occurs during a time with no coverage in your schedule, then the alert will be escalated immediately to the next escalation level, without waiting for the escalation timeout. If no one is on-call on the entire escalation policy, no one will be notified.
### Can I choose my own shift color?
Yes. Open the user menu in the top right, go to **About me**, and set **Shift color**.
# My on-call shifts
View and manage your personal on‑call shifts in ilert to stay informed about upcoming duty, swap shifts, or request coverage when needed.
The **My on-call shifts** page allows users to see their shifts across all on-call schedules and escalation policies in a single view. Moreover, a user can easily create overrides across multiple schedules.
## Definition of "being on-call"
A user is considered to be on-call if they are part of an escalation policy (either directly or indirectly through an on-call schedule). This escalation policy must be used by at least one alert source.
{% hint style="info" %}
Escalation policies that aren't used by any alert source do not affect your on-call status and won't show up in this view.
{% endhint %}
If a user is on-call by being directly included in the escalation policy, then on-call entries will show up as "always on-call" starting from the date when the escalation policy was last updated until infinity. If a user is on-call by being included through the membership of an on-call schedule, then on-call entries will be shown as shifts from the on-call schedule.
## My on-call shifts page
* **My on-call calendar**: shows the shifts for the selected escalation policy levels. The date picker on the left lets you quickly navigate between months and jump to a specific date. Moreover, it highlights days for which you are on-call with an orange dot.
* **My escalation policies** selector lets you select the escalation levels that you wish to see on-call entries for. Note that escalation policies are only included in the list; if you are a member and they are used by at least one alert source. Your selection preference will be remembered.
* **Timezone** selector lets you change the time zone in which you view the on-call entries. By default, the timezone of your user profile is taken.
* **Take on-call** lets you take someone else's on-call
* **Override my shifts** lets you override your shifts from multiple schedules at once.
* **Export as calendar** lets you embed your on-call shifts in any calendar application.
## Scheduling overrides
There are two ways to schedule overrides:
* by taking someone else's on-call - adding yourself as an override to another shift
* by overriding your shifts - adding someone else as an override to your shift
### Take someone else's on-call
Taking someone else's on-call lets you add yourself as an override for someone else's on-call.
To take someone else's on-call
1. Click on the **Take on-call** button
2. Select the user whose shifts you want to take over
3. Select the date range for when you want to take over shifts
4. All shifts from the selected user between the selected date range will be shown grouped by schedule. You can optionally deselect individual shifts that you don't want to take over. Moreover, you can choose to partially override a particular shift. Click the :pencil2: icon at the right to partially override the shift.
5. Click on the **Take on-call** button
### Override your shifts
You can override your shifts
* by selection the dates in the calendar with your mouse
* by clicking on the **Override my shifts** button
A pop-up will appear showing you the shifts that will be affected by the override. You may change your end and start date and the affected shifts before finally scheduling any overrides.
{% hint style="info" %}
* Overrides that are created from the **My on-call shifts** page can only be deleted individually by going to each on-call schedule.
* Overrides only apply for on-call schedules and don't replace you in escalation policies.
{% endhint %}
## Exporting your on-call shifts as calendar events
You can subscribe to your on-call shifts by copying the calendar subscription link into your calendar app (Google Calendar, Apple Calendar, Outlook, etc.). The calendar subscription will include all your current and future on-call shifts.
# Recurring schedules
Configure recurring on‑call schedules in ilert for predictable team rotations, automating shift cycles over daily, weekly, or monthly intervals.
Recurring schedules are a flexible way to create complex recurring schedules with minimal effort.
## Create a recurring schedule
To create a recurring schedule, navigate to **On-call schedules,** click on **Create new on-call schedule,** and choose **Create recurring schedule** as the schedule type.
### Step 1: Add users
Add the users from the dropdown menu. Users will rotate in the order in which they are listed. You can change the order via drag and drop and remove a user by clicking the x icon.
### Step 2: Set on-call rotation
The **on-call rotation** determines when a shift should rotate from one user to the next user. Choose between daily and weekly rotations. For example, if you want your users to rotate every week, enter `1 week`, if you want them to rotate bi-weekly, enter `2 weeks`, and so on.
In the **Starts on** field, choose the start time of your schedule. Note that the start time will also determine the time when a user's on-call duty is handed over to the next user in the rotation.
### Step 3: Set on-call coverage (optional)
By default, the on-call coverage is 24 hours a day, 7 days a week, i.e., users are on-call all the time. If you want to restrict times on-call, e.g., to outside working hours only, you can do so in this step.
Restricting your on-call coverage will result in gaps in your schedule where no one is on call. If an alert occurs during a time with no coverage in your schedule, then the alert will be escalated immediately to the next escalation level without waiting for the escalation timeout. If no one is on-call on the entire escalation policy, no one will be notified.
#### Follow support hours (respects holidays)
Instead of drawing coverage by hand, a layer can **follow a support hour**. Set the layer's coverage to **Follow support hours (respects holidays)** and select one of your [support hours](/alerting/configure-alerting/support-hours). The layer then copies that support hour's **weekly windows** *and* its **holiday exceptions** as its on-call coverage — so the rotation automatically pauses on holidays that are marked as non-working and picks up on days with extra coverage.
{% hint style="info" %}
Coverage is **copied** from the support hour, not referenced live — so past shifts are never rewritten when the support hour changes. Whenever you edit the support hour, every schedule that follows it is automatically re-synced going forward.
{% endhint %}
The schedule timeline shows a **Holidays** row wherever a holiday exception changes coverage. Hover the row label to preview the linked support hour — its weekly hours, with a link to open it in a new tab.
{% hint style="warning" %}
A layer that follows a support hour is on call **only** during that support hour's weekly windows. If the support hour has no weekly hours defined, the layer covers nothing and nobody is on call — so make sure the support hour you select actually defines the hours you expect to be covered.
{% endhint %}
Following support hours requires a plan that includes the support hours feature. If it is not part of your plan, the coverage option appears locked with an upgrade prompt.
## Creating complex schedules with schedule layers
A schedule layer is the configuration element that defines an on-call schedule or a segment thereof. A schedule layer consists of the following parameters:
1. **Users**: an ordered list of users that will rotate in the on-call schedule. You can add a single user multiple times. E.g., if a user should take twice as many shifts as others, you can add them twice.
2. **Rotation:** The amount of time (in days or weeks) after which on-call duty is rotated from one to the next.
3. **Starts on:** the start date and time of the layer.
4. **Ends on (optional):** An optional end date and time of the layer. A layer with no end date will schedule shifts indefinitely. Once an end date is set, no more shifts will be scheduled past the end date.
5. **Restrictions (optional):** Optional times of day (e.g., between 9am – 6pm) or times of week (e.g., Fri 6pm – Mon 9am) to restrict times on-call.
6. **Name (optional):** an optional name to help you better organize layers \*\*\*\*
You can combine layers to create more complex schedules by having multiple active layers at the same time.
{% hint style="info" %}
**Combining layers with overlapping shifts**
Layers at the bottom take precedence over layers at the top. The example below illustrates the effect of this.
{% endhint %}
Let's look at a few examples to illustrate the power of schedule layers.
### Schedule examples
#### Follow-the-sun-schedule
A follow-the-sun schedule lets you have 24/7 coverage without putting the burden on one site and distributes on-call across multiple timezones.
Our final schedule configuration will look as follows:
Here are the steps to create this schedule:
1. Pick a **timezone** for the schedule. The timezone applies to all layers and cannot be edited after creation. In the above example, we select "America/Los\_Angeles"
2. Create a layer for the US team:
* Select the desired **users**, **rotation,** and **start time**. Note that the start time also denotes the handoff time between shifts. You can also pick a date in the past.
* Restrict on-call to **specific times of the day**. In the above example, the US team is on-call every day from 9:00 AM - 9:00 PM
* Optionally, enter a **name** for this layer by clicking on the pen icon in the header.
3. Click on the **Add schedule layer** link to create another schedule layer for the EU team:
* Select the desired **users**, **rotation,** and **start time**. Because shifts for the EU team start at 9:00 PM, we select 9:00 PM. Note that all times are local to the scheduled timezone "America/Los\_Angeles", i.e. 9:00 PM in "America/Los\_Angeles" is 6:00 AM in "Europe/Berlin" timezone.
* Restrict on-call to **specific times of the day**. In the above example, the EU team is on-call every day from 9:00 PM - 9:00 AM.
* Optionally enter a **name** for this layer.
Once you have created all the necessary layers, check the timeline at the bottom and verify that the final schedule is correct.
#### Schedule with different users on weekdays and weekends
In this example, we have one team that is on-call during weekdays and a separate team for the weekends. In our example, weekday shifts are between Monday 9:00 AM and Friday 5:00 PM, and weekend shifts start on Friday 5:00 PM until Monday 9:00 AM.
The final schedule configuration will look as follows:
The steps to create the schedule:
1. Pick a **timezone** for the schedule. The timezone applies to all layers and cannot be edited after creation. In the above example, we select "Europe/Berlin"
2. Create a layer for the users that are on-call on weekdays
* Select the desired **users**, **rotation,** and **start time**. Note that the start time also denotes the handoff time between shifts. We select Monday 9:00 AM as the start time / shift handover-
* Restrict on-call to **specific times of the week** and add "Mon 9:00 - Fri 17:00"
* Optionally, enter a **name** for this layer by clicking on the pen icon in the header.
3. Click on the **Add schedule layer** link to create another schedule layer for the weekend shifts:
* Select the desired **users**, **rotation,** and **start time**. Because shifts o the weekend start on Fridays 5:00 PM, we select Fri 5:00 PM for the start time.
* Restrict on-call to **specific times of the week** and add "Fri 17:00 - Mon 09:00."
* Optionally enter a **name** for this layer.
Once you have created all the necessary layers, check the timeline at the bottom and verify that the final schedule is correct.
### Create a recurring schedule with the help of AI
{% content-ref url="/pages/lJjudVgwPmO5s39W0S5T" %}
[Create schedules using ilert AI](/on-call-management-and-escalations/on-call-schedules/recurring-schedules/using-ilert-ai-for-schedule-generation)
{% endcontent-ref %}
You can also use a conversational interface and set up a recurring schedule. To do so, after choosing "**Create recurring schedules**," find the button "**Create schedule with ilert AI**" at the top right corner. Share your scheduling requirements, including your team members' names, rotation frequency, and times. When ready, click "**Generate**."
### Edit an existing recurring schedule
{% hint style="info" %}
**Are you looking for overrides?**
If you want to make a one-time change to a schedule, create an override instead. See [here](/on-call-management-and-escalations/on-call-schedules#overrides) for more info.
{% endhint %}
Whenever you need to make structural changes to the way you organize on-call, you will need to edit your schedule in ilert. Examples include when a new team member joins the on-call rotation, an existing team member leaves the rotation, you want to change the rotation (e.g. from weekly to bi-weekly), etc.
To edit an existing schedule ...
1. Click **On-call scheduling** -> **On-call schedules** in the navigation bar.
2. Select the desired schedule by clicking on the pen icon. Make sure it's a recurring schedule.
You can now make the changes to the layer directly (e.g. add another team member) and select the date for the changes to take effect:
Alternatively, instead of editing a layer directly, you can also end a layer and start a new layer. You might find this method clearer and easier to transition from one schedule configuration to another. In the example below, we will add a new member to an existing rotation.
1. To end a layer, click on the stop icon in the top right of the layer and select an end date and time.
2. To create a new layer, click on **Add schedule layer** and make the desired changes to your new schedule layer.
# Create schedules using ilert AI
Use a familiar chat interface for instructions and let ilert AI generate on‑call schedules intelligently based on your input.
Creating a recurring schedule can be a challenge for users that are new to on-call duty management or sometimes can be a complex challenge when the required schedule in itself has a complex structure to it e.g. follow the sun with multiple teams and besides that some people might just prefer to use natural language to express their schedule structure instead of timelines and restrictions- this is why we build a schedule assistant into ilert AI.
**Head to On-call -> On-call schedules -> Create new schedule -> Create recurring schedule** and you will find the **Generate schedule with ilert AI** button at the top right (*if you have enabled ilert AI for your account*).
A modal will appear on the right side of the screen asking you to describe your desired schedule:
While the assistant will automatically guide you, by asking questions for missing details for the schedule creation, keep in mind that you should provide some insights into
* who should be on call and at which time of the week or day?
* what is the rotation every day, every week or biweekly?
* when does the schedule start and does it end or is it constantly ongoing?
When the assistant has gathered all information required to set up the schedule for you, it will process for a little moment and provide you with a button to **Generate the schedule**
The assistant will preconfigure the schedule creation based on the generated input, you can now make final adjustments if you like - as soon as you are ready press the **Save** button. Congratulations, you have generated your schedule.
# Static schedules
Set fixed-date static on-call schedules in ilert for consistent incident coverage, ideal for special projects or long-term assignments.
Static schedules work like regular calendars and do not support recurring shifts. Use simple schedules if you have no recurring pattern in your on-call organization and want to regularly update your schedule.
## Create a static schedule
To create a static schedule, navigate to **On-call schedules,** click on **Create new on-call schedule** and chose **Create static schedule**.
Enter a name for the schedule, pick a timezone (by default the account-level timezone) and choose a default shift duration. The default shift duration is the duration of a shift that is set when you drag a user to the schedule. Follow the instructions below to create shifts.
### Creating shifts
Shifts can be created in two ways:
1. By dragging users from the left and dropping them to the calendar on right
2. By selecting dates on the calendar. To do so, click and drag your pointer over some dates (see below).
As in recurring schedules, only one user per schedule can be on-call at a time. Also note that you shifts cannot be created retroactively.
### Gap detection
If your on-call coverage is 24/7, we recommend to activate gap detection by selecting **Show gaps when saving**. In this case, you will get a warning when you are about to create a schedule with gaps.
# Coverage requests
Use ilert to request coverage swaps when you need to adjust your on‑call shift, with transparent review and approval workflows.
ilert's **Coverage requests** feature allows users to request coverage for their on-call shifts from colleagues. Rather than directly overriding or reassigning a shift, Coverage requests create a lightweight workflow where the requester asks a colleague to take over specific shifts. This is especially helpful if you need to step away from your on-call duties for a personal appointment, an unexpected emergency or personal time-off.
{% hint style="info" %}
**ilert mobile app required**
Currently, coverage requests are mobile-only, meaning requests can only be created, viewed, and managed in the ilert mobile app.
{% endhint %}
## Key benefits
* Collaboration: Allows on-call team members to coordinate coverage among themselves without admin intervention.
* Flexibility: Users can select exact shifts within a desired time period and send a request for coverage.
* Transparency: Both the requester and the recipient can track the status of each request - pending, accepted, or declined.
## How it differs from an override
* Coverage Request: A request-based workflow. The requester asks a teammate to cover one or more specific shifts. The teammate must accept or decline.
* Override (requires [User permission](/users-and-access-management/user-roles-and-permissions)): An admin can directly reassign a shift without requiring acceptance. Any responder can add themselves as an override for any shift, but cannot add another person.
## Creating a coverage request
1. Open the ilert mobile app. Ensure you have the latest version installed on your iOS or Android device.
2. Open the navigation by tapping on the menu icon on the top left.
3. Tap on **Request coverage**.
4. Select the User: choose the colleague you'd like to request coverage from.
5. Enter desired coverage period:
* Pick a start and end time to define the timeframe you need coverage.
* ilert will display your existing on-call shifts within this period.
6. Choose the individual shift(s) you'd like your colleague to cover.
7. Add an optional message.
8. Send the request.
Your colleague will receive a notification via the mobile app.
## Responding to coverage requests
When someone requests coverage from you:
1. You'll receive a push notification and a badge on the requests tab in the ilert mobile app.
2. Open **Coverage requests** from the main navigation. Review the shift dates/times and any message from the requester.
3. **Accept** or **Decline**
* Tap **Accept** if you can cover the shift.
* Tap **Decline** if you're unable to cover.
* Once you choose, both you and the requester will be notified of the decision.
## FAQs
### Can I send coverage requests from the ilert web app?
Not at this time. Coverage requests are available only via the ilert mobile app.
### Can an admin override my shift if I send a coverage request?
Yes. Admins retain the ability to directly override or reassign shifts, which bypasses the request/accept workflow.
### Can I cancel a request after sending it?
Yes. Navigate to **Coverage requests** in the mobile app, locate the request, and select **Cancel** if your colleague hasn't already accepted.
# Getting started
Coordinate your incident response and keep users informed with ilert's incidents, status updates, services, and status pages.
ilert helps you both **coordinate** your response to an incident and **communicate** it to the people who depend on your services—while keeping the two clearly separated.
## Why it matters
* Bring your response team together in one place instead of coordinating across noisy alerts.
* Keep customers and stakeholders informed, building trust during an incident.
* Reduce incoming support requests by communicating proactively.
* Decide deliberately what to share publicly, and when.
## The building blocks
* [**Incidents**](/incidents-and-status-pages/incidents) are the coordination record for a significant event. Declare an incident to link alerts, page responders, track a timeline, and run the response to resolution.
* [**Status updates**](/incidents-and-status-pages/status-updates) are the public-facing messages you post about an incident. They appear on your [status pages](/incidents-and-status-pages/status-pages) and are also delivered directly to ilert users—such as those with the VIEWER role—who are subscribed to an affected service or status page.
* [**Services**](/incidents-and-status-pages/services) model the business capabilities your users care about. Incidents and status updates reference services, and users can subscribe to them for notifications.
* [**Status pages**](/incidents-and-status-pages/status-pages) are public or private pages that show the current status of your services and the history of status updates.
* [**Metrics**](/incidents-and-status-pages/metrics) show how well your services are performing on a status page. You can submit your own metrics data or import it from tools such as Datadog.
## How it fits together
An [alert](/alerting/working-with-alerts) pages your on-call responder. When the issue needs coordinated response, you [declare an incident](/incidents-and-status-pages/incidents/declare-an-incident)—the internal hub where your team works the problem. If the incident affects customers, you [post a status update](/incidents-and-status-pages/status-updates) to your [status pages](/incidents-and-status-pages/status-pages), keeping internal detail private and public communication deliberate. Coordination hangs off the incident, not the alert: responders, the timeline, and the [chat channel](/incidents-and-status-pages/incidents/incident-channels) all belong to it.
# Services
Define and manage services in ilert to reflect availability status and impact across incidents and status pages.
Users can subscribe to services to receive notifications about status updates that affect specific business capabilities.
A service has any of the following **states**:
| Status | Description |
| ----------------- | ------------------------------------------------------------------------------------------------- |
| Operational | All aspects of the service are working as expected. |
| Under maintenance | The service is under maintenance. |
| Degraded | The service is working but is impacted in a minor way, e.g. its performance is slower than usual. |
| Partial outage | The service is not working for a subset of customers, e.g. a certain region. |
| Major outage | The service is not available. |
The status of a service can be updated when you post a [status update](/incidents-and-status-pages/status-updates), or independently on the service itself.
{% hint style="info" %}
**Service topology**
Services can depend on one another. Connect a [telemetry source](/incidents-and-status-pages/service-topology/telemetry-sources) and ilert reconstructs your live [service topology](/incidents-and-status-pages/service-topology) from OpenTelemetry traces — visualizing dependencies, blast radius and likely root cause on the Service Health map.
{% endhint %}
{% hint style="info" %}
**Service status changes and notifications**
Updating a service's status directly on the service does not notify subscribers about the change. Subscribers are notified only when a [status update](/incidents-and-status-pages/status-updates) is posted with affected services they are subscribed to.
{% endhint %}
## Create a service
To create a new service, navigate to the Services page, click on the **Create new service** button.
Each service carries its own 60-day uptime history, and users can subscribe per service.
Give the service a name that is descriptive to the users of the service and click on **Create new service**.
A service needs only a name to start with.
The service is now ready to be used in incidents or included in status pages.
## Automation with alert sources
You can automatically set the status of a service and post status updates using alert actions that are triggered by your alert sources.
To configure an automation
* in the sidebar, go to **Alerting** → **Alert sources**
Pick the alert source whose events should drive the service status.
* choose an alert source and navigate to the **Alert actions** tab
* click on **Create new alert action**
* select the **ilert incidents** action type and click **Next**
Selecting the ilert incidents action type
Once you have configured the automation, any alert created on the chosen alert source will run the action and—depending on how you configure it—set service status or post a status update. Make sure that the trigger mode is set to **Automatic** for the action to run on every alert:
Setting the trigger mode to Automatic
## Service uptime
### How to hide historical service uptime
By default, a service's historical uptime is shown and you can choose to hide service uptime on the status page. If you'd like to globally disable historical uptime, thus preventing any status page from displaying the historical uptime, you can do so in the service settings:
1. Go the services page and click on the service for which you want to hide the uptime
2. In the **Settings** tab, check the option **Never show historical uptime** 
3. Click on **Save**
### How is the uptime of a service calculated?
The uptime percentage of a service is calculated based on the status of the service over a given period. ilert displays the uptime over a period of up to 90 days. We consider the states **Operational**, **Degraded** and **Under maintenance** as uptime. The states **Major outage** and **Partial outage** are counted as downtime, where partial outage minutes only count 30% as much as major outages.
Over a period *t*, the uptime percentage is calculated according the following formula:
$$
u=1-{m+(p\*0.3) \above{1pt} t}
$$
where
* *u* is the uptime percentage
* *m* is the major outage in minutes
* *p* is the partial outage in minutes
* *t* is the period in minutes
**Example**: During a period of 24 hours, a service was 5 minutes in the status **Major outage** and 30 minutes in the status **Partial outage**. The uptime of the service for that 24 hours period is calculated as follows:
$$
u=1-{5+(30\*0.3) \above{1pt} 1440}=99.03 %
$$
{% hint style="info" %}
Outages that last less than a minute will be ignored and won't show up as downtime in the uptime graph.
{% endhint %}
### How is the color of each day in the uptime graph determined?
Each day in the uptime graph is color-coded based on the total outage minutes of that day. The outage minutes are calculated using the same weighted formula as above, where partial outage minutes count 30% as much as major outages.
The color ranges from **green** (no or minimal downtime) to **red** (significant downtime):
| Total outage minutes | Color |
| -------------------- | ------------ |
| 0 minutes | Green |
| > 0 – 6 minutes | Light green |
| > 6 – 9 minutes | Lime |
| 9 – 12 minutes | Light yellow |
| 12 – 25 minutes | Yellow |
| 25 – 30 minutes | Light orange |
| 30 – 40 minutes | Orange |
| 40 – 45 minutes | Dark orange |
| 45 – 50 minutes | Light red |
| 50 – 60 minutes | Red |
| 60+ minutes | Dark red |
Days in the future as well as days before the service was created (where no uptime data exists) are displayed in **grey**.
{% hint style="info" %}
The transition from green to red is gradual — even a few minutes of outage will shift the color slightly, giving you a quick visual indication of service health at a glance.
{% endhint %}
## Add / remove subscribers to a service
Subscribers will automatically receive notifications when the **Send notifications** option is selected on a status update. There are two ways to subscribe users to a service
1. Users can add / remove themselves as subscribers
2. A user with Responder privileges can manually subscribe users or entire teams to a service
To self-subscribe to a service, go the Services page and click on **Subscribe.** To unsubscribe, click on the **Unsubscribe** button.
{% hint style="info" %}
**Team subscriptions**
Note that an unsubscription takes precedence over a team subscription. That is, if you are subscribed to a service, because you are a member of a team that is subscribed to the service, and if you unsubscribe yourself from that service, you won't receive any further updates from that service.
{% endhint %}
To add or remove users and teams as subscribers from a service, go to the **Subscribers** tab from the service.
# Service topology
Visualize how your services depend on one another and understand the impact of an outage with the Service Health map, reconstructed automatically from your OpenTelemetry traces.
Service topology gives you a live, connected view of your [services](/incidents-and-status-pages/services) and the dependencies between them. Instead of maintaining a static architecture diagram, ilert reconstructs the dependency graph automatically from the OpenTelemetry traces your services emit, so the map always reflects how your system actually behaves in production.
The topology powers two things:
* the **Service Health map** — an interactive graph you can explore during an incident to find the blast radius and the likely root cause
* richer context for [AI SRE alert triage](broken://pages/gZKiwMVgY05ARI2wHmrW), which walks the dependency graph to reason about where a problem originates
{% hint style="info" %}
**Beta**
Service topology and [telemetry sources](/incidents-and-status-pages/service-topology/telemetry-sources) are currently in beta. Details of the UI and behavior may change.
{% endhint %}
## How the topology is built
There are two ways a dependency edge ends up in the graph:
1. **Discovered from traces** — when you connect a [telemetry source](/incidents-and-status-pages/service-topology/telemetry-sources), your services push OpenTelemetry traces to ilert. ilert inspects outbound (client and producer) spans to derive service-to-service edges and continuously updates the graph. A service is created automatically in your [service catalog](/incidents-and-status-pages/services) the first time it appears in a discovered edge — either as the caller (source) or as the target. A service that only emits server-side spans and never participates in a discovered edge is not created on its own.
2. **Added manually** — you can draw a dependency between two services yourself, for example to capture a relationship that isn't (yet) visible in traces.
Every edge and every auto-created service keeps its **provenance**, so you can always tell whether a dependency was discovered from telemetry, auto-discovered, or created by a human.
## The Service Health map
Open **Service Health** from the main navigation to explore your topology. The page offers two views:
* **Map** — the interactive dependency graph
* **List** — the same services in a sortable table, with **Status**, **Alerts** and **Edges** columns
The Service Health map, with a service selected and its side panel open
Switch to **List** for a scannable overview of the same services:
The Service Health list view
### Node status and legend
Each node is colored by the current status of the service. The map legend uses three colors:
| Color | Meaning |
| ----- | ----------- |
| Green | Operational |
| Amber | Degraded |
| Red | Outage |
The **List** view shows the precise [service state](/incidents-and-status-pages/services) instead — for example `Operational` or `Major outage`. Nodes you don't have permission to see are shown as **Restricted**.
### Spotlight, blast radius and root cause
When you select a node, the map pans and zooms to it, dims the unrelated nodes, and lights up the paths that matter:
* **Blast radius** — the services **upstream** of the selected service that are affected when it degrades (who depends on this).
* **Root-cause candidates** — the services **downstream** of the selected service that could be causing its problem (what this depends on). The panel hints where to *look next*.
A side panel opens alongside the selection with:
* **Open alerts** on the service, with a high/low breakdown and a link into Alerts
* **Recent incidents** affecting the service
* **Blast radius** and **Root-cause candidates**
* **Depends on** — the service's direct downstream dependencies
### Filtering the map
Use the controls above the map to focus on what matters:
* **Labels** — filter to the services carrying specific labels.
* **Has active alerts** — prune the view to only the services that are currently alerting.
* **Highlight SPOFs** — emphasize single points of failure in the graph.
* **Include orphans** — show services that have no dependencies. When off, only services connected to at least one other service are shown.
* **Search** — jump to a service by name.
{% hint style="info" %}
The map updates **live** as new traces arrive — the header shows when the topology was last updated. Very large graphs may exceed the display limit; filter by labels to render a sub-graph.
{% endhint %}
## Service dependencies
Every service has a **Dependencies** tab that lists its edges in both directions:
* **Depends on** — the services this service calls (downstream)
* **Depended on by** — the services that call this service (upstream)
Dependencies discovered from traces show when they were **last seen**, so stale edges are easy to spot. You can also add a dependency manually.
### Add a dependency manually
1. Open the service and go to the **Dependencies** tab.
2. Click **Add dependency**.
3. Choose the **target service** and, optionally, add a note describing why the dependency exists.
4. Save.
To remove an edge, use **Remove dependency** on the dependency you want to delete.
{% hint style="info" %}
Manually drawn dependencies are never overwritten by trace discovery, and edits to an existing service (its labels, links, or dependencies) are preserved — trace ingestion only *adds* what it discovers.
{% endhint %}
{% hint style="warning" %}
Removing a **discovered** dependency is only permanent if the underlying calls have stopped. If the two services are still communicating, the edge is observed again in your traces and re-created on the next sync. To remove such an edge for good, stop the traffic (or the instrumentation) that produces it.
{% endhint %}
## Provenance and auto-created services
When ilert first sees a service as part of a discovered dependency in your traces, it creates it in the service catalog and records how it was created. The **Provenance** of a service or dependency can be any of:
| Provenance | Meaning |
| ------------------------ | ------------------------------------------------------ |
| OpenTelemetry | Reported by traces from a telemetry source |
| Auto-discovered | Inferred as the target of a discovered dependency edge |
| Manual | Created or drawn by a user |
| Terraform / Pulumi / API | Created through infrastructure-as-code or the API |
Auto-created services are also enriched with **labels** derived from OpenTelemetry resource attributes (environment, namespace, version, language, Kubernetes and cloud metadata, and more). Labels are applied only when a service is first created — your manual edits are always safe. See [telemetry sources](/incidents-and-status-pages/service-topology/telemetry-sources#labels-on-discovered-services) for the full mapping.
## Next steps
* [Connect a telemetry source](/incidents-and-status-pages/service-topology/telemetry-sources) to start reconstructing your topology from traces.
* Learn how [AI SRE](broken://pages/gZKiwMVgY05ARI2wHmrW) uses the topology during alert triage.
# Telemetry sources
Connect your services to ilert by pushing OpenTelemetry traces to a telemetry source. ilert extracts the service dependencies from your traces and builds your live service topology.
A telemetry source is an OpenTelemetry (OTLP) ingestion endpoint that your services push traces to. ilert samples those traces, extracts the service-to-service dependencies, and continuously builds your [service topology](/incidents-and-status-pages/service-topology) — no manual wiring required.
{% hint style="info" %}
**Beta**
Telemetry sources and [service topology](/incidents-and-status-pages/service-topology) are currently in beta. Details of the UI and behavior may change.
{% endhint %}
## How it works
ilert uses a **push model** — you never share credentials with ilert. Your services (or an OpenTelemetry Collector) send OTLP traces to the endpoint, authenticated with an ingestion token:
1. Your services export **OTLP traces** to the telemetry source endpoint.
2. ilert inspects outbound **client** and **producer** spans to derive edges — the caller's `service.name` becomes the source, and the target is taken from span attributes such as `peer.service`, `server.address`, or `db.system`.
3. The discovered edges are merged into a single per-account **topology graph**, and any previously unseen service is created in your [service catalog](/incidents-and-status-pages/services).
4. The [Service Health map](/incidents-and-status-pages/service-topology#the-service-health-map) updates live as traces continue to arrive.
Multiple telemetry sources all merge into the same topology graph, so you can, for example, use one source per team or per environment.
## Create a telemetry source
The create wizard has two steps: **Configure** and **Connect & verify**.
1. Open **Telemetry sources** under **Catalog** in the sidebar and click **Create telemetry source**.
2. Give the source a **name**. This is a human label shown in the sources list.
3. Optionally add **labels** (free-form `key: value` pairs) and assign the source to one or more **teams**.
4. Click **Create source**.
Step 1 of the wizard — name, labels and teams
{% hint style="info" %}
Telemetry sources use a push model, so ilert collects no credentials from you. The ingest endpoint accepts OTLP over HTTP or gRPC, encoded as either **protobuf** (the exporter default) or **JSON**.
{% endhint %}
## Connect your services
In the **Connect & verify** step ilert provisions the endpoint and an ingestion token. Copy the token now — you configure your OpenTelemetry exporter to send traces to the endpoint with the token as a bearer credential.
Step 2 of the wizard — endpoints, ingestion token and ready-to-copy exporter configuration
### Endpoints
| Protocol | Base endpoint |
| ----------- | ----------------------------- |
| OTLP / HTTP | `https://otlp.ilert.com:4318` |
| OTLP / gRPC | `otlp.ilert.com:4317` |
These are **base** endpoints — configure them as your exporter's OTLP endpoint and the exporter appends the signal path (`/v1/traces`) automatically. If you are posting to the HTTP endpoint directly (not through an OTLP exporter), the full traces URL is `https://otlp.ilert.com:4318/v1/traces`.
Authenticate every request with the ingestion token:
```
Authorization: Bearer
```
The wizard provides ready-to-copy configuration for common setups:
{% tabs %}
{% tab title="OTEL Collector" %}
```yaml
# otel-collector.yaml
exporters:
otlphttp:
endpoint: https://otlp.ilert.com:4318 # base endpoint — /v1/traces is appended automatically
encoding: proto # proto (default) or json — both accepted
headers:
Authorization: "Bearer "
service:
pipelines:
traces:
exporters: [otlphttp]
```
{% endtab %}
{% tab title="SDK (env vars)" %}
```bash
export OTEL_EXPORTER_OTLP_ENDPOINT="https://otlp.ilert.com:4318"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer "
```
{% endtab %}
{% tab title="Kubernetes · OBI" %}
```yaml
# OBI DaemonSet env
env:
- name: OTEL_EXPORTER_OTLP_ENDPOINT
value: "https://otlp.ilert.com:4318"
- name: OTEL_EXPORTER_OTLP_HEADERS
value: "Authorization=Bearer "
```
{% endtab %}
{% endtabs %}
{% hint style="info" %}
The ingest endpoint accepts both **protobuf** and **JSON** OTLP payloads. Most exporters default to protobuf, which is recommended; JSON works too if your exporter is configured for it.
{% endhint %}
Once traces start flowing, the topology updates automatically — there is nothing else to configure.
## Endpoint and ingestion token
You can view the endpoints and the token — shown as **Integration Key** — again at any time from the telemetry source's detail page. The key is masked by default and visible only to users with write permission.
The telemetry source detail page, with endpoints, status and the masked integration key
If a key is ever exposed, rotate it: open the source and choose **Rotate integration key** to issue a new key, then roll it out to your exporters. Treat the exposed key as compromised and rotate as soon as possible.
## Ingestion status
The sources list shows each source with its labels, teams and ingestion status, so you can confirm data is arriving:
| Status | Meaning |
| --------- | ------------------------------------------------------------- |
| Receiving | Traces are being received. |
| Stopped | The source was receiving traces but none have arrived lately. |
| Error | Ingestion is failing. |
| Pending | The source is provisioned but no traces have been seen yet. |
The Telemetry sources list, scoped to the currently selected team
## Labels on discovered services
When ilert creates a service for the first time from your traces, it enriches it with labels derived from the OpenTelemetry **resource attributes** your services report. This makes it easy to filter the topology and service catalog by environment, namespace, cloud, and more.
Labels are applied **only when a service is first created** — an existing service's labels are never overwritten, so manual edits are safe. A service that only appears as a dependency target (for example a database that doesn't emit its own spans) carries no labels until it starts reporting.
A selection of the mapping:
| ilert label | OpenTelemetry resource attribute(s) |
| --------------------------------------------------- | ------------------------------------------------------------------------------- |
| `env` | `deployment.environment.name`, `deployment.environment` |
| `namespace` | `service.namespace` |
| `version` | `service.version` |
| `language` | `telemetry.sdk.language` |
| `runtime` | `process.runtime.name` |
| `k8s-namespace`, `k8s-cluster`, `k8s-deployment`, … | the matching `k8s.*.name` |
| `cloud`, `region`, `zone`, `cloud-account` | `cloud.provider`, `cloud.region`, `cloud.availability_zone`, `cloud.account.id` |
High-cardinality attributes (such as pod name, host name, and instance id) are deliberately excluded.
## Next steps
* Explore your [service topology](/incidents-and-status-pages/service-topology) on the Service Health map.
* Review the auto-created services in your [service catalog](/incidents-and-status-pages/services).
# Incidents
Declare and coordinate incidents in ilert to organize your response, page responders, track a timeline, and keep stakeholders informed.
## What is an incident?
An incident is the coordination record for a significant, business-impacting event. It is the place where your team comes together to respond: you link the alerts that contributed to it, page the people you need, track a timeline of everything that happened, and—when it is time—communicate with your customers.
Where an [alert](/alerting/working-with-alerts) is a machine-generated signal designed to page on-call responders, an incident is a deliberate, human-declared record designed for coordination. Declaring an incident does not depend on an alert firing—you can declare one whenever you need to mobilize a response.
## Alerts, incidents, and status updates
ilert separates the technical signal, the internal coordination, and the public communication into three distinct entities. Understanding how they relate is key to using incidents effectively.
| Entity | Created by | Audience | Purpose |
| ----------------- | ---------------------------------- | --------------------------------- | ---------------------------------------------------- |
| **Alert** | Monitoring tools (automatically) | On-call responders | A technical signal that pages the right people. |
| **Incident** | Declared manually or from an alert | Your response team (internal) | The coordination record that organizes the response. |
| **Status update** | Posted from an incident | Customers & stakeholders (public) | The public message shown on your status pages. |
An incident is **internal**. Nothing about an incident is visible to your customers until you deliberately [post a status update](/incidents-and-status-pages/status-updates). This lets you investigate and coordinate before deciding what—if anything—to communicate publicly.
{% hint style="info" %}
What ilert previously called an "incident" is now a [**status update**](/incidents-and-status-pages/status-updates). The name **Incident** now refers to the coordination record described on this page.
{% endhint %}
## The incident lifecycle
A typical incident follows this path:
{% stepper %}
{% step %}
### An alert pages the on-call responder
A monitoring tool fires an alert, and ilert pages whoever is on call. The responder begins investigating.
{% endstep %}
{% step %}
#### You declare an incident
When the issue needs coordinated response, you [declare an incident](/incidents-and-status-pages/incidents/declare-an-incident)—from the alert, or from scratch. The triggering alert is linked, and its responders join the incident.
{% endstep %}
{% step %}
#### You coordinate the response
You [page additional responders](/incidents-and-status-pages/incidents/responders-and-paging), open an [incident channel](/incidents-and-status-pages/incidents/incident-channels) for real-time discussion, and keep a [timeline](/incidents-and-status-pages/incidents/incident-timeline-and-comments) of decisions and findings.
{% endstep %}
{% step %}
#### You communicate publicly
If the incident affects customers, you [post status updates](/incidents-and-status-pages/status-updates) to your status pages, keeping severity and internal detail private.
{% endstep %}
{% step %}
#### You resolve the incident
Once the impact has ended, you set the incident to **Resolved**. Linked alerts resolve with it.
{% endstep %}
{% endstepper %}
## Severity
Every incident has a severity that communicates its business impact. ilert uses five fixed levels:
| Severity | Meaning |
| -------- | ----------------------------------------------------- |
| **SEV1** | Critical — major service disruption, highest priority |
| **SEV2** | High — significant impact, urgent response |
| **SEV3** | Medium — limited impact, partial degradation |
| **SEV4** | Low — minor degradation, no service impact |
| **SEV5** | Informational |
New incidents default to **SEV3**. You can change the severity at any time from the incident view.
## Status
The status reflects where an incident is in its lifecycle. It is internal to ilert and separate from the public status shown on a [status update](/incidents-and-status-pages/status-updates).
| Status | Meaning |
| ----------------- | ---------------------------------------------------------- |
| **Declared** | The incident has just been created. |
| **Investigating** | The team is actively investigating the root cause. |
| **Identified** | The root cause has been found and a fix is underway. |
| **Monitoring** | A fix is in place and the team is watching for recurrence. |
| **Resolved** | The impact has ended and the incident is closed. |
## The incidents list
Open **Incidents** in the main navigation to see every incident your teams own. Each row shows the severity, affected services, duration, how many responders have joined, the number of linked alerts, and the current status.
The incidents list
## The incident view
Opening an incident brings everything about the response onto one screen:
The incident view
* **Header** — the incident title, its `INC-` number, severity, and status. Severity and status are editable here.
* **Summary** — an internal description of what is known so far.
* **Affected services** — the [services](/incidents-and-status-pages/services) impacted by the incident, each with an impact level.
* **Responders and subscribers** — the people [working the incident and watching it](/incidents-and-status-pages/incidents/responders-and-paging).
* **Linked alerts** — the [alerts](/incidents-and-status-pages/incidents/declare-an-incident#linking-alerts-to-an-incident) that contributed to the incident.
* **Incident channel** — the [chat channel](/incidents-and-status-pages/incidents/incident-channels) connected to the incident, if one exists.
* **Timeline** — an [append-only log](/incidents-and-status-pages/incidents/incident-timeline-and-comments) of everything that happened, plus comments and status updates.
## Next steps
Declare an incident
Open an incident from scratch or from an alert, and link related alerts.
# Declare an incident
Declare an incident in ilert from scratch or from an alert, and link the alerts that contributed to it.
There are two ways to declare an [incident](/incidents-and-status-pages/incidents): from scratch, or from an existing alert. Either way, the incident starts in the **Declared** status and you can begin coordinating immediately.
## Declare an incident from scratch
Declare an incident from scratch when the issue is not tied to an alert—for example a customer-reported problem, a third-party outage, a security event, or a response exercise.
{% stepper %}
{% step %}
### Open the declare dialog
Go to **Incidents** in the main navigation and click **Declare incident**.
{% endstep %}
{% step %}
### Fill in the incident details
Declaring an incident from scratch
| Field | Required | Description |
| --------------------------- | -------- | ------------------------------------------------------------------------------------------------------------- |
| **Title** | Yes | A short, internal description of what is happening. It is never shown publicly. |
| **Severity** | Yes | The [business impact](/incidents-and-status-pages/incidents#severity) of the incident. Defaults to **SEV3**. |
| **Summary** | No | What is known so far and what is affected. You can draft it with the ilert AI button. |
| **Affected services** | No | The [services](/incidents-and-status-pages/services) impacted by the incident, each with an impact level. |
| **Create incident channel** | No | Connect a [chat channel](/incidents-and-status-pages/incidents/incident-channels) for real-time coordination. |
| {% endstep %} | | |
{% step %}
### Declare the incident
Click **Declare incident**. The incident is created and opens in the [incident view](/incidents-and-status-pages/incidents#the-incident-view), ready for you to page responders and track the response.
{% endstep %}
{% endstepper %}
## Declare an incident from an alert
When an alert reveals an issue that needs coordinated response, declare an incident directly from it. Open the alert and click **Declare incident**.
Declaring an incident from an alert
The dialog is pre-filled from the alert so you can declare in seconds:
* The **title** is taken from the alert title.
* The **summary** is generated from the alert details.
* **Affected services** are pre-selected from the alert's related services—you set the impact level for each.
* The alert is **linked** to the incident, and its responders are added to the incident.
If the alert has an escalation in progress, that escalation continues on the incident, so the people already being paged are not lost.
## Linking alerts to an incident
Linking tells ilert which alerts contributed to an incident. It keeps the noise of individual alerts out of the way while the incident becomes the single place to coordinate.
The alert you declare an incident from is linked automatically. To link more alerts, open the incident and click **Link alert** in the **Linked alerts** section.
When an alert is linked to an incident:
* the alert's status changes to **Linked**;
* its escalation stops—the incident now owns the response;
* its responders are added to the incident;
* it resolves automatically when the incident is resolved.
{% hint style="info" %}
An alert can be linked to only one incident at a time.
{% endhint %}
### Unlinking an alert
If an alert was linked by mistake, open the incident and click **Unlink** next to it. The alert returns to the **Accepted** status. Its escalation does **not** resume automatically—if the alert still needs attention, re-trigger escalation on the alert.
## Affected services and impact levels
Affected services record which business capabilities an incident is impacting, each with an [impact level](/incidents-and-status-pages/services) such as *Degraded* or *Major outage*.
Setting or changing affected services on an incident updates ilert **internally**—it is reflected on internal dashboards and on the incident itself. It does **not** change anything your customers see.
{% hint style="info" %}
The public status of a service changes only when you [post a status update](/incidents-and-status-pages/status-updates). This keeps you in control of what is communicated, and when.
{% endhint %}
# Responders and paging
Page the right people to an incident in ilert and manage who is responding and who is watching.
An [incident](/incidents-and-status-pages/incidents) has two kinds of members, shown in separate sections of the incident view:
| Member type | Role | Scale |
| -------------- | ----------------------------------------- | ---------------------------- |
| **Responder** | Actively working to resolve the incident. | Usually a handful of people. |
| **Subscriber** | Watching the incident for visibility. | Can be many stakeholders. |
## Paging responders
Paging notifies people that they are needed on an incident, using the same escalation infrastructure as alerts.
From the incident view, click **Page** to open the **Page responder** dialog. Search for and select one or more targets, optionally add a message for context, and click **Page**.
Paging responders to an incident
You can page several kinds of targets:
| Target | What happens |
| --------------------- | ---------------------------------------------------------------------------------- |
| **Escalation policy** | Pages step by step with timeouts, escalating automatically until someone responds. |
| **On-call schedule** | Pages the person currently on call for that schedule. |
| **Individual** | Pages one specific person directly. |
| **Entire team** | Pages every member of a team at once. |
You can add multiple targets in a single page—useful for incidents that span several teams.
When a responder joins, the escalation that paged them stops, so no one is paged unnecessarily. Other escalations on the incident continue until each is answered.
{% hint style="info" %}
When you page a team, every member is notified. As soon as one member joins, the others are told the page has been answered and no action is needed from them.
{% endhint %}
## Joining an incident
When someone is paged, they are notified just as they would be for an alert. They can **join** the incident to become an active responder. The incident view shows how many responders have joined.
Responders are also added automatically when you [declare an incident from an alert](/incidents-and-status-pages/incidents/declare-an-incident#declare-an-incident-from-an-alert) or [link an alert](/incidents-and-status-pages/incidents/declare-an-incident#linking-alerts-to-an-incident)—the alert's responders carry over to the incident.
## The Incident Commander
One responder on an incident can be the **Incident Commander**. The Incident Commander owns the incident, makes decisions, and coordinates the overall response.
To assign the role, open a responder in the incident view and click **Make Incident Commander**. There is one Incident Commander at a time—assigning the role to someone else moves it from the previous holder.
## Subscribers
Subscribers receive activity notifications for an incident without being expected to work on it—useful for managers, customer success, or other stakeholders who need visibility.
To add or remove subscribers, click **Manage** in the **Subscribers** section of the incident view.
{% hint style="info" %}
Incident subscribers are specific to a single incident. To receive updates across many incidents, subscribe to a [service](/incidents-and-status-pages/services) or a [status page](/incidents-and-status-pages/status-pages) instead.
{% endhint %}
# Incident timeline and comments
Follow the incident timeline in ilert, add comments to keep your team aligned, and communicate publicly with status updates.
The incident timeline is the running record of everything that happened during an [incident](/incidents-and-status-pages/incidents). It is internal to your response team and is the foundation for a post-mortem once the incident is over.
## The incident timeline
The timeline is an append-only log shown in the incident view. ilert records key events automatically, including:
* the incident being declared,
* changes to severity and status,
* responders joining the incident,
* alerts being linked or unlinked,
* status updates being posted,
* an incident channel being attached.
Entries appear in chronological order, so anyone joining the incident can catch up quickly.
## Comments
Comments are notes you write onto the timeline yourself. Use them to capture anything that helps coordinate the response or prepare the post-mortem:
* investigation findings ("Connection pool exhausted on db-primary-1"),
* decisions and their reasoning ("Rolling back instead of hotfixing because…"),
* handoff context when responsibility moves between people or regions.
To add a comment, type into the **Add comment to timeline** field in the incident view and send it. Comments are interleaved with automatic entries in chronological order.
{% hint style="info" %}
If the incident has an [incident channel](/incidents-and-status-pages/incidents/incident-channels), comments are kept in sync with that channel so the conversation and the timeline stay together.
{% endhint %}
## Communicating publicly with status updates
The incident timeline is **internal**. To communicate with customers, post a [status update](/incidents-and-status-pages/status-updates) from the incident by clicking **Post status update**.
A status update is a separate, public-facing message published to your [status pages](/incidents-and-status-pages/status-pages). Posting one is always a deliberate action—the incident's internal title, summary, severity, and timeline are never exposed. When you post a status update, the timeline records that it happened, but the public wording lives only on the status update itself.
See [Status updates](/incidents-and-status-pages/status-updates) for the full walkthrough of posting and resolving them.
# Incident channels
Connect a Slack, Microsoft Teams, or Google Chat channel to an incident in ilert for real-time coordination.
An incident channel is a dedicated chat channel connected to an [incident](/incidents-and-status-pages/incidents) for real-time coordination. It gives your responders one place to talk through the response while the incident keeps the structured record.
## Before you start
Incident channels rely on a connected chat integration. Make sure one of the following is set up for your account:
* [Slack](/chatops/slack)
* [Microsoft Teams](/chatops/microsoft-teams)
* [Google Chat](/chatops/google-chat)
See the [ChatOps overview](/chatops/overview) for how to connect a chat tool to ilert.
## Create an incident channel
You can connect a channel in two ways:
* **While declaring an incident** — select **Create incident channel** in the [Declare incident dialog](/incidents-and-status-pages/incidents/declare-an-incident).
* **From an existing incident** — in the incident view, use the **Incident channel** section to create one.
If more than one chat platform is connected to your account, you choose which one to use when the channel is created.
{% hint style="info" %}
An incident can have one incident channel. The platform is fixed once the channel is created—you can disconnect a channel, but you cannot switch it to a different platform.
{% endhint %}
## Keeping the channel and timeline in sync
Once a channel is connected, [comments](/incidents-and-status-pages/incidents/incident-timeline-and-comments#comments) are kept in sync between ilert and the channel, so the conversation and the incident [timeline](/incidents-and-status-pages/incidents/incident-timeline-and-comments) stay aligned. This also means the channel can be included when you [generate a post-mortem with ilert AI](/incidents-and-status-pages/status-updates/generating-incident-updates-through-ai).
# Status updates
Post status updates in ilert to communicate the state of an incident to your users and stakeholders on your status pages.
## What is a status update?
A status update is a public-facing message that communicates the state of an [incident](/incidents-and-status-pages/incidents) to your users. Status updates are how you keep customers and stakeholders informed when a service they rely on is degraded or unavailable.
A status update has one or more affected services and will be:
* posted on all status pages that contain at least one of the affected services,
* communicated to all subscribers of either the status page or an affected service,
* and additionally sent to subscribers that were manually added to the incident.
{% hint style="info" %}
**Status updates were previously called "incidents"**
What ilert previously called an "incident" is now a **status update**. The name **Incident** now refers to the [incident coordination record](/incidents-and-status-pages/incidents) used to organize your response. A status update is the public communication you post about an incident.
{% endhint %}
### Alerts, incidents, and status updates
ilert separates the technical signal, the internal coordination, and the public communication into three distinct entities:
| Entity | Created by | Audience | Purpose |
| ----------------- | ---------------------------------- | --------------------------------- | ---------------------------------------------------- |
| **Alert** | Monitoring tools (automatically) | On-call responders | A technical signal that pages the right people. |
| **Incident** | Declared manually or from an alert | Your response team (internal) | The coordination record that organizes the response. |
| **Status update** | Posted from an incident | Customers & stakeholders (public) | The public message shown on your status pages. |
## Post a status update
Status updates are posted from an [incident](/incidents-and-status-pages/incidents). This keeps your internal coordination and your public communication connected: the incident holds the full internal context, while each status update is a deliberate, customer-friendly message.
To post a status update, open an incident and click **Post status update**.
Posting a status update from an incident
A status update has the following fields:
| Field | Description |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Public title** | The customer-facing title shown on your status pages. It is set on the first update and shared across all pages the incident appears on. |
| **Status** | The public status of the incident: **Investigating**, **Identified**, **Monitoring**, or **Resolved**. |
| **Message** | The public-facing text of the update. You can draft it with [ilert AI](/incidents-and-status-pages/status-updates/generating-incidents-using-ai). |
| **Affected services** | The services impacted by the update, each with an impact level. The impact level determines the public status shown on your status pages. |
| **Notify subscribers** | Whether to notify subscribers of the affected services and status pages about this update. |
When you publish the update, ilert posts it to every status page that includes at least one of the affected services and—if enabled—notifies subscribers.
{% hint style="info" %}
The public status of a service on a status page changes only when a status update is posted. Declaring an incident or changing its affected services updates ilert internally, but nothing is shown publicly until you post a status update.
{% endhint %}
### Resolving an incident publicly
When the incident is over, post a status update with the status **Resolved**. The affected services return to **Operational** on every status page the update is published to, and subscribers are notified that the issue has been resolved.
## Incident templates
Create incident templates to:
* use as a starting point when posting status updates, and
* automatically create status updates using [alert source automation rules](/incidents-and-status-pages/services#automation-with-alert-sources).
When you post a status update, you can select one of your templates and all fields will be pre-filled using the values from the template.
### Create an incident template
1. Go to **Incident comms** or **Status pages** (depending on your pricing plan) and click on the **Incident templates** tab.
2. Click on **Create new template**.
3. Give the template a name and fill out the fields that you would like to have pre-filled.
4. Click **Create template**.
### Using alert placeholders in incident templates
When a status update is created from an alert (**using alert source automation rules**), placeholder variables are substituted with their corresponding values. To use a variable in your template, include it in double braces, like this: `New status update for service {{serviceName}}`.
The following variables are available for substitution:
| Variable | Description |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `serviceId` | The id of the service that is affected |
| `serviceName` | The name of the service that is affected |
| `serviceDescription` | The description of the service that is affected |
| `serviceStatus` | The **new** status of the service that is affected |
| `serviceStatusOld` | The **old**/current status of the service that is affected |
| `teamNames` | A comma separated list of team names. This corresponds to the same value that is shown in the teams field of an incident and is resolved by traversing the ownerships of the affected service and the alert source that created the alert. |
| `alertId` | The id of the alert that triggered this status update |
| `alertDetails` | The details of the alert that triggered this status update |
| `alertSummary` | The summary of the alert that triggered this status update |
| `alertStatus` | The status of the alert that triggered this status update |
| `alertPriority` | The priority of the alert that triggered this status update |
| `alertSourceId` | The id of the alert source of the alert that triggered this status update |
| `alertSourceName` | The name of the alert source of the alert that triggered this status update |
| `escalationPolicyId` | The id of the escalation policy of the alert that triggered this status update |
| `escalationPolicyName` | The name of the escalation policy of the alert that triggered this status update |
# AI status update generation
Leverage ilert AI to craft timely status updates, reducing manual effort and improving communication with stakeholders during incident response.
{% embed url="" %}
In the heat of incident response, it's essential to keep your focus on resolving the issue at hand. Sometimes, drafting a comprehensive and accurate [status update](/incidents-and-status-pages/status-updates) can be challenging, especially under stress. That's where ilert's AI-assisted status update generation comes into play.
## What does ilert's AI-assisted status update generation do?
ilert's AI-assisted status update generation is designed to help you write [status updates](/incidents-and-status-pages/status-updates) swiftly and accurately. It streamlines the process of documenting what is happening and what is affected, so you can concentrate your efforts on resolving the issue. ilert AI also intelligently guesses which services—along with their impact level—should be included in the update.
## How to use ilert AI for a first status update
When you post a status update, provide a very short summary of what you want to communicate in the summary or message field and click on the :magic\_wand: icon on the right of the input field.
Based on the summary (e.g. "spacex services unavailable"), ilert AI will generate the message and also predict and include relevant affected services.
## How to use ilert AI for follow-up status updates
In the midst of incident resolution, providing regular updates can be a complex task, especially when the pressure is high. That's where ilert's AI feature steps in to assist you.
Just as it helps with a first status update, ilert AI can also efficiently produce follow-up updates for an ongoing incident. Whether you're deeply engaged in the response or merely aiming for a swift closure, ilert AI is there to ensure proper communication about all impacted services.
## What kind of data is shared?
For a first status update:
* Service **names** for a small selection of services that ilert AI chooses based on the relation to the prompt or the context of the user entering the prompt, e.g. services owned by related teams. No other service data is shared.
* The provided **summary** is included in the prompt.
For follow-up status updates:
* Service names of the services included in the ongoing incident. Service names that are not part of the incident are not shared.
* The current status of the incident.
* The entered message content is used as the prompt.
# AI for post-mortem generation
Use ilert AI to draft a post-mortem from an incident's alerts, timeline, and status updates.
## From alert to incident, and what to do once the dust has settled
As a technical issue transitions from an alert into an incident because it has an effect on business services. Its disruption and impact might make it necessary to revisit what happened and which steps can be taken to prevent a similar issue from reoccurring in the future. Assume following sample incident that has reached its resolved state.
A simple example incident that has reached the end of its lifecycle in the RESOLVED state
## When the dust settles
As customers have been notified, the root cause identified and a potential fix in place, the dust and hectic has settled around the incident. Now it is time to sit back and think about what caused the issue in the first place and if/how such things can be prevented in the future.
Usually someone in the team responsible for the services that were impacted by the incident will now start working on gathering all information around the incident and set up a document with all details that is discussed afterwards. This can be quite cumbersome and ilert AI is here to help reduce the time it takes to get started with a proper doc significantly.
### Using ilert AI to generate a postmortem doc
If ilert AI is enabled, you will find the option to generate a postmortem in the detail view of the incident.
This will open a popup that offers multiple options to fine tune your postmortem document generation.
### By default (no selections)
By default the generation will take all available context of the incident into account, such as history updates, subscribers, services and involved users / responders.
### Include alerts
If an alert is linked to the incident, the alerts details will also be taken into account, as well as its timeline. This can help to significantly improve the generation of technical related action items in the final document. You can also link alerts afterwards, as long as they are not resolved yet.
### Including the incident channel
If the incident has an [incident channel](/incidents-and-status-pages/incidents/incident-channels) (Slack, Microsoft Teams, or Google Chat), the channel will also appear in the selection options of the popup. Channels that are included will be scanned and parsed by the connected ilert bot, and relevant messages will be included automatically in the content of the final document.
It is also possible to paste a chat transcript from any other source manually, this script may contain timestamps and usernames in any raw format, ilert AI will automatically parse and handle the content accordingly.
### Optional fine-tuning
You have three additional fields to provide optional content for fine-tuning of the postmortem document. The content here has a little higher weight in such that during the generation it can help shape action items or root cause. This is helpful if certain action items or root cause information are already known at the time of postmortem generation and you wish to include them.
## Postmortem creation
When you are satisfied with the postmortem input you can click on **Generate post mortem** to enqueue your postmortem creation. As certain sources are pulled, paginated, read and parsed during the creation process it has been decoupled and can take a little moment.
As soon as your document is ready, the status will transition to **created**.
You may now click on **View generated content** to see a simplified markdown rendered version of the generated content:
For our sample alert the generated markdown version looks like this:
At the bottom right you will find the **Open** option which will contain a link to the raw text file document of your generated postmortem. You will probably want to move this raw version of the document into a shared MS Word or Google Doc for further manual fine-tuning and adjustments before sharing it with colleagues.
## Publishing a postmortem
As soon as you are ready to share your document with internal or external stakeholders and subscribers you have the option to link your document to an incident.
{% hint style="success" %}
You may also choose to include your postmortem on all status pages that this incident appears on.
{% endhint %}
# Status pages
Build customizable status pages in ilert to provide transparent, real-time incident updates and uptime status to external stakeholders.
This article will walk you through the key steps to create your status page and make it accessible to your audience. You'll learn how to choose between different status page types, update statuses, and set up a custom domain and custom analytics.
{% hint style="info" %}
**Status page types**
ilert offers several types of status pages. If you're deciding between a public or private status page, please see the article [*Public vs. private status page*](/incidents-and-status-pages/status-pages/public-vs-private-status-pages). If you're considering an audience-specific status page, we recommend reading [the corresponding article](/incidents-and-status-pages/status-pages/audience-specific-status-page) for more details.
{% endhint %}
{% hint style="warning" %}
**Status pages add-on**
Please note that the ilert Free plan includes only a limited version of the public status page. If you intend to use a public status page for a wide audience, or a private or audience-specific status page, you will need to purchase an add-on. For more details, please refer to our [Pricing page](https://www.ilert.com/pricing).
{% endhint %}
## Create a status page
Creating a status page is a matter of a few clicks.
1\. Navigate to **Status pages** in the navigation bar and click on the **Create status page** button.
2\. Select your page type, ilert URL, and the teams that should manage the status page.
3\. Select the services that you would like to include in the status page. You can also create a new service and add it to the status page from the drop-down list.
4\. Optionally add your own logo and fav icon and click on **Save**.
5\. Click on the **Visit status page** button to preview your status page. Once you are satisfied with your status page, click on the **Activate** button to make your page available and viewable to your users.
## Update the status of your status page
The status of your status page is automatically updated whenever
* The status of a service that is included in the status page is updated (e.g. by setting a service to *Degraded*)
* A [status update is posted](/incidents-and-status-pages/status-updates) that affects any of the services from your status page
[-> How to post a status update](/incidents-and-status-pages/status-updates)
{% embed url="" %}
{% endembed %}
### Automation with alert sources
You can automatically update your status page in the following ways:
* Automatically update the status of a service (for example, set the status to *Degraded*)
* Automatically post a status update to your status page using an [incident template](/incidents-and-status-pages/status-updates#create-an-incident-template) and (optionally) notify subscribers
Both cases work by using the **ilert incidents** alert action on an alert source. See [-> Services: automation with alert sources](/incidents-and-status-pages/services#automation-with-alert-sources) for more information.
{% embed url="" %}
{% endembed %}
## Setting up your custom domain
Every status page (both private and public) comes with support for custom domains, so that you can make your status page accessible from your own subdomains (e.g., status.example.com).
Setting up a custom domain consists of the following steps:
1. Set the custom domain in ilert
2. Configure DNS
3. Wait for the changes to take effect
### 1. Set the custom domain in ilert
Go to your status page settings, enter a custom domain, and click **Save**.
### 2. Configure DNS
Configuring DNS happens outside of ilert, at the DNS provider you are using for your domain.
1. Create a new DNS record and select **CNAME** as the record type
2. Enter the name for the CNAME record. The **name** or **DNS entry** is where you enter your subdomain. You might need to enter it in full (e.g., **status.example.com**) or you might just need to enter the part before your apex domain (e.g., **status**). If you're not sure which to use, check with your DNS provider.
3. Enter the ilert URL of your status page as the value for the CNAME record. The **target,** **value,** or **destination** is where the subdomain should be pointed. Taking the screenshot from above as an example, you would need to enter `ilert-demo.ilert.io` .
You might also see a field named **TTL**, which stands for Time To Live. It's the number of seconds that the DNS record can be cached for. If you're not sure what to set, look at the TTL for your existing DNS records. You could set the same number. If you're still not sure, we suggest setting 43200 seconds (12 hours) or 86400 seconds (24 hours).
{% hint style="warning" %}
**Are you using Cloudflare?**
Whenever possible, please **turn off Cloudflare proxying** (the orange cloud, also called "Proxy status" in your domain settings) to ensure that your status page is served without issues and can be monitored by ilert.
{% endhint %}
### 3. Wait for the changes to take effect
You might need to wait 1-48 hours for the DNS changes to take effect. This depends on the TTL setting of your DNS record and the time it takes until the DNS change is propagated throughout the internet.
## Status page analytics
For pages with custom domains, it is possible to use external analytics tools. External tools can help you gain insights into your status page's most common traffic channels and user behavior.
You can choose from two options—Google Analytics and PostHog. You need a key for both solutions to connect an analytics platform and ilert.
{% hint style="info" %}
Please beware that using external analytics often requires user agreement with data collection and processing. Consult with your legal department regarding the best approach to collecting information.
{% endhint %}
## FAQ
### Q: We are a managed service provider and would like to use multiple status pages to communicate incidents to different customers with private pages without exposing data between customers. Is that possible?
A: Yes. All paid plans include unlimited Viewer licenses, so you can invite as many customers as you need without requiring a full license.
To isolate customers from each other, we recommend the following approach:
1. Create a team for every customer and make the team [private](/users-and-access-management/teams#private-teams)
2. Invite your customer to ilert choosing the **Stakeholder** role
3. Add the customer to the respective customer team.
4. Assign the status page to the respective customer team.
### Q: I have set my status page to Public, but it's still not accessible from the internet. What am I missing?
A: You probably forgot to activate your status page. Once you are satisfied with your status page, click on the **Activate** button to make your page available and viewable to your users.
### **Q: Can I export a list of status page subscribers?**
A: Yes. To export a CSV file of your status page subscribers, go to Subscribers tab in Status Page Settings and click "Download subscribers" button in the upper right corner.
### **Q: What if a subscriber hasn't confirmed a status page subscription?**
A: No worries, ilert will automatically send double opt-in confirmation reminders to your subscribers after 24 hours, three days, and one week.
# Status page layout
Customize your status page's appearance, organize services into groups, select embed options for your website, and use the announcement bar to share important updates.
This article will guide you in further customizing your status page. Now that you've completed the key configurations, you can explore various design options to make your page reflect your brand.
## Status page appearance
You can adjust the appearance of your status page to reflect your company brand. Choose between light and dark mode and upload your logotype and favicon.
{% hint style="info" %}
**Note:** If you are using a private (including audience-specific) status page, your logotype, name of the page, and favicon will also be visible on the login page of your status page.
{% endhint %}
## Status page layout options
### Service groups: How to group related services together
Service groups let you group related services together on a status page. This allows users to quickly see the overall health of a system and identify any issues that may be impacting multiple services.
Below is an example service group from our status page. When the service group is collapsed, it shows a combined status and uptime graph. When the service group is expanded, it shows the status and uptime graph for each service individually. The overall status of multiple services in a service group is calculated by taking the status of the service with the highest impact. Similarly, the uptime percentage and outage graph is calculated by combining the outages of all services.
 
To create a service group
1. Navigate to the status page for which you want to create a service group
2. Go to the **Services** tab.
3. Click on the **Create service group** button and give it a name.
4. You can add services to the service group during creation or later by dragging and dropping services into the group.
Use the following settings on a service group and service to further customize how they appear on your status page:
* **Expand group on page**: Activate this option if you want to expand the entire service group by default on your status page
* **Show uptime history:** Activate this option if you want to show the service's historical uptime and its SLA
### Services layout: single and responsive columns + responsive grid
You can choose between two layout options for displaying services on your status page.
* **A single column** presents items in a vertical list.
* **Responsive columns** arrange items in a grid with up to 3 columns that adapt to your screen size.
* **Responsive grid** introduces a high-density grid optimized for large service catalogs.
| Single column | Responsive columns | Responsive grid |
| :-----------------------------------------------------------------: | :-----------------------------------------------------------------: | ------------------------------------------------------------------- |
| | | |
To change the service on a status page
1. Go to **Status pages** and click on the status page you want to change its layout.
2. Click on the **Services** tab
3. Select your layout option and click save
## Embed the status of your status page on your website
Communicate incidents to your users right where they are using your service: on your website or web app.
To embed the status of your status page in your web app or website, you can use ilert's floating status widget or status badge. The status page widget will only be visible if there is an ongoing incident or scheduled maintenance and will be hidden if all services are operating normally. The status widget will always be visible and show the overall status of your status page.
{% hint style="info" %}
Your users don't have to update a page to see the changes in the widget. The **widget refresh is automatic**, so your clients will notice the updates as soon as they occur.
{% endhint %}
To configure your status page floating widget:
1. Go to **Status Pages**
2. Click on the Status Page where you would like to configure the widget
3. Click on the **Status embed** tab
4. Select **Floating widget**
5. Configure your widget by choosing your colors and widget position, copy the generated snippet code.
6. Paste the generated code snippet from step 4 into the `` section of your target page.
7. Test the widget by creating a test maintenance window or incident.
{% hint style="info" %}
**Making changes to your widget**
Changes to your status page will not propagate automatically to your pages. You need to update the code snippet in your target for any changes to take effect.
{% endhint %}
The localization of the widget will be automatically detected using the viewer's browser language. If you want to set the **language** to a fixed setting, you can optionally provide the query parameter `&lng=en` or `&lng=de` to the scripts already existing parameters.
## Announcement Bar
Share essential announcements via your status page. Enable an announcement bar in the status page settings. To make your message more engaging, use simple Markdown and emojis. The announcement bar will be shown at the top of your status page.
Announcement bar settings
Example of the announcement bar
# Public vs private status pages
Compare public vs private status pages in ilert to determine which audience gets visibility into incident updates and maintenance notices.
Status pages in ilert can either be private or public. You can change the page type at any time until the page was activated. Once activated, the page type cannot be changed. The table below compares both page types:
| | Public page | Private page |
| ------------------------------------------------- | ------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Use case** | For public incident communication with external users and customers | For private incident communication with employees, customers or partners that either have an ilert account, a whitelisted IP address or email address / email domain. |
| **Visibility** | Publicly visible on the internet | Visible to authenticated ilert users only or to users with whitelisted IP addresses or email addresses / domains |
| **Supports custom domains with SSL** | :white\_check\_mark: | :white\_check\_mark: |
| **Supports IP Whitelist** | :x: | :white\_check\_mark: |
| **Supports whitelisted emails and email domains** | :x: | :white\_check\_mark: |
{% hint style="info" %}
**Viewing a private page does not require a full user license**
Choose **Stakeholder** when the point is to restrict what someone can see. Otherwise **Viewer** is the cheaper and simpler option.
{% endhint %}
| License | Scope | Cost |
| ---------------------------------------------------------------------------------- | --------------------------------------------------------- | ---------------------------- |
| [Viewer](/users-and-access-management/user-roles-and-permissions#viewer) | Account-wide read-only access | Free and unlimited |
| [Stakeholder](/users-and-access-management/user-roles-and-permissions#stakeholder) | Only the status pages and services granted to their teams | Scale plan, sold in packages |
## Private status page authentication options
Only authenticated users can view ilert's private status pages. To adjust access, navigate to **Status Pages -> Authentication**, and select one or several authentication methods.
* **Accessible to all users of your account**: All users with any roles will have access to the chosen status page. Please note that the page will be visible even if it belongs to a private team.
* **IP whitelist:** Specify IP addresses, and users without an ilert account will be able to see the status page.
* **Passwordless email login**: Enter email addresses and/or domains that should have access to your status page.
# Audience-specific status page
Create audience‑specific status pages in ilert tailored to different stakeholder groups with custom branding and messaging.
## Overview
An **audience-specific status page** is a private page accessible only to authenticated ilert users or stakeholder accounts. It dynamically presents services and metrics tailored to each user's team assignments, ensuring that everyone sees only the information relevant to them.
{% hint style="info" %}
An audience-specific page doesn't support authentication via IP or email whitelists.
{% endhint %}
### Use cases
Audience-specific status pages are ideal for providing tailored content to different user groups. Below are key scenarios where such pages are beneficial:
* **Internal employee status page:** Allow internal employees to view the status of services they have permission to access. This ensures that sensitive information remains secure while keeping your team informed about the services that matter to them.
* **Customer-specific status page:** Show status updates to customers based on the services they subscribe to. This personalized approach enhances customer satisfaction by providing relevant information.
* **Single-Tenant Hosting:** If you offer dedicated instances to each of your customers, you can display status information specific to each customer. This allows for a customized experience that reflects the unique environment of each client.
### Benefits
Creating audience-specific status pages enhances both security and usability by allowing you to:
* **Customize visibility**: Select which services and metrics are visible to specific users or groups. This ensures that users only see information relevant to them.
* **Enhance security**: Restrict access to sensitive information by making the status page private and accessible only to authenticated users.
* **Improve user experience**: Provide users with a tailored view of services and metrics, improving the relevance and usefulness of the information they receive.
## Managing the audience of your status page
After creating an audience-specific status page, you can manage which services and metrics are visible to different audiences (teams) in your organization. Follow these steps to configure your audience settings:
1. **Add services and metrics**
Navigate to the **Services and Metrics** tab in your status page settings. Here, add all the services and metrics that you want to include on your status page, regardless of the audience. This will serve as the complete list of items that can be displayed to any audience.
2. **Configure audience visibility**
Switch to the **Audience** tab to manage the visibility of services and metrics for each audience. In ilert, each audience corresponds to a team. Any team that owns one or more of the services or metrics you selected in Step 1 will appear as an audience in this tab, with the relevant items listed.
* **Making services or metrics Visible to an Audience**
To control the visibility of a service or metric for a specific audience, use the toggle switch next to each item. Turning the switch on will make the service or metric visible to members of that audience (and also add it to the team); turning it off will hide it.
* **Adding users to an audience**
To add users to a team (audience), click on **Add users to this team** in the top-right corner of the audience section. This allows you to include specific users who should have access to the services and metrics designated for that audience. Note that this will also add the selected to the ilert team.
## FAQ
### Do I need to purchase full user licenses so that users can view an audience-specific status page?
No. A [Viewer](/users-and-access-management/user-roles-and-permissions#viewer) license is sufficient, and Viewer licenses are free and unlimited.
### How can I create a new audience?
Audiences are defined based on your team structure in ilert. You can create as many teams as you need, either by reusing your existing team structure or by creating new teams specifically for your audience-specific status pages.
To add a team as an audience:
1. **Navigate to the Audience Tab:** Go to your status page settings and switch to the **Audience** tab.
2. **Add a New Team:** Click on the **+ Add Teams** button.
3. **Select or Create a Team:**
* **Select an Existing Team:** Choose from the list of existing teams.
* **Create a New Team:** If you need a new audience, you can create a new team directly from this interface.
4. **Configure Visibility:** After adding the team, you can manage which services and metrics are visible to this audience by toggling the switches next to each item.
By organizing your audiences through teams, you can tailor the content of your status pages to specific groups within your organization or customer base, ensuring that each audience sees only the information relevant to them.
# Metrics
Provide basic information about the health of your services by including metrics in status pages
Metrics in ilert lets you provide additional information about the health of your services in status pages.
## What is a metric?
A metric is a numerical value that can track anything over time, such as API response time, API error rates, number of open tickets, etc. ilert stores metric data as pairs of floating point values and timestamps.
## Metric Display Options in ilert
ilert provides two ways to display a metric
1. Line graph (optionally with an aggregated value)
2. Single number
### Line Graph
Here's an example of a metric visualized as a line graph:
* **Display name**: The name of the metric that will be displayed on the status page
* **Period:** A metric can be visualized over a period of 24 hours (day), 7 days (week) or 28 days (month).
* **Aggregated value:** An single value that combines all values over the selected period by applying an aggregation function. The aggregated value is shown in the top right corner of the graph and can be hidden. ilert provides four aggregations that you can choose from:
* AVG: the arithmetic average of all values over the selected period
* SUM: the arithmetic sum of all values over the selected period
* MIN: the minimum value of all values over the selected period
* MAX: the maximum value of all values over the selected period
* **Display unit:** this unit will be shown as a suffix. Note that this unit is just for display purposes and can be left empty.
{% hint style="info" %}
**Interval aggregation in graphs (aka rollup aggregation)**
ilert stores a large number of data points per metric (up to one datapoint every 30 seconds). In most cases, there are more data points than what can be visualized on a graph. Therefore, ilert aggregates the values by combining them in time intervals. For example, when selecting Month as period, data points are aggregated into 2 hour intervals. To aggregate the values, ilert uses the same aggregation function that you select for the summary display in the right corner of the graph.
**Periods and Aggregation Intervals**
* Month - 2 hours
* Week - 30 minutes
* Day - 5 minutes
{% endhint %}
#### Linear interpolation
With linear interpolation, missing data points will be shown by drawing a straight line between two known data points. If you disable linear interpolation, missing data points will be shown as gaps in the line graph.
**Example**
The following metric has missing data points. Below you can see the effect of interpolation.
| Interpolation enabled | Interpolation disabled |
| :------------------------------: | :------------------------------: |
|  |  |
### Single number metric
Here's an example of a metric that is displayed as a single number:
Metrics that are displayed as a single number provide the same aggregation functions from above. Additionally, you can choose to display the last reported value (i.e. the value with the latest timestamp).
## Create a metric
1. In the main navigation bar, click on **Status pages** (or **Incident comms**) --> **Metrics**
2. Click on the **Create metric** button
3. Choose your metric data source. You can either submit metric data using our [Series API](https://api.ilert.com/api-docs/#tag/Series/paths/~1series~1{key}/post) or import metrics from a 3rd party metrics provider such as [Datadog](/incidents-and-status-pages/metrics/import-metrics-from-datadog) or [Prometheus](/incidents-and-status-pages/metrics/import-metrics-from-prometheus).
4. Configure your metric and click **Save**
## Add a metric to status page
Once you have created a metric, you can add the metric to any of your status pages.
1. Navigate to the status page to which you want to include the metric
2. Go to the **Metrics** tab, click on the **Add metric** button and select the metrics that you would like to add to the status page.
3. The metric will now be displayed on your status page. You can change the order of metrics by reordering the metrics in the metrics tab of the status using drag and drop.
## Submit data points to a metric
Submitting data points to your metric is as easy making an HTTP POST request using the API key of your metric.
{% hint style="info" %}
**Metrics API Key**
Every metric has its own API key, just like alert sources have their own API keys. You cannot use your personal API key so submit metrics data at the moment.
{% endhint %}
You will find the API key of the metric in the metric's settings page:
Here's an example curl command to update the above MTTA metric to 8.223 minutes:
```shell
curl -L -X POST 'https://api.ilert.com/api/series/ilm110229945419465b5494fxxxxxxxxxxxxxxxxf' \
-H 'Content-Type: application/json' \
--data-raw '{
"value": 8.223
}'
```
Note that the timestamp value is omitted in the JSON payload. In this case, the current time stamp will be used.
Our API also support submitting multiple data points at once. Refer to our [API documentation](https://api.ilert.com/api-docs/#tag/Series/paths/~1series~1{key}/post) for more information.
{% hint style="info" %}
**ilert's Metric Data Storage**
* A metric can store maximum 1 data point every 30 seconds. Each submitted data point is truncated to its nearest 30s interval.
* Submitting data points with a higher resolution will result in the last data point being the only one stored.
* A metric can store data up to 28 days in the past. When creating a new metric, we recommend to backfill data for the past 28 days.
{% endhint %}
### Generate sample data
ilert lets you generate one time sample data for your metric
1. Open your metrics settings page
2. Click on the `...` button in the top right and select **Generate demo data**
3. Select the range for the random data and click on **Generate demo data**
### Clear metrics data
To clear metrics data
1. Open your metrics settings page
2. Click on the `...` button in the top right and select **Delete metric data**
This will delete all data points for that metrics.
# Import metrics from Datadog
Seamlessly import Datadog metrics into ilert to visualize SLOs, automate incident alerts, and keep status pages up to date.
## Create a Datadog metric data source
### In Datadog
1. Go to **Organization Settings** → **API Keys** and click **New Key.**
2. Give the Key a name (e.g., ilert metrics import) and click **Copy Key,** and paste it somewhere. You will need this key later in ilert.
3. Go to **Organization Settings** → **Application Keys** and click **New Key.**
4. Give the Key a name and copy and paste somewhere. You will need the application key later in ilert.
### In ilert
1. Go to **Status pages** (or **Incident comms**) --> **Metric data sources,** and click on **Create metric data source.**
2. Give the metrics data source a name (e.g., Datadog).
3. Select **Datadog** in **Type** dropdown list.
4. Select your Datadog region. You can identify your region by matching your Datadog website URL to the URL from the list.
5. Enter the API and Application keys from above and click **Create metric data source**.
### Create a metric
### In Datadog
1. Find the metric you want to import to ilert, e.g., by opening a dashboard.
2. Click on **Edit this widget.**
3. In the **Edit** view, choose the graph you want to import and click on the `` icon to display the metric query.
4. Copy the metric query. You will need the query below in ilert.
{% hint style="info" %}
Note that metrics in ilert are not dimensional. Therefore, only Datadog metrics without grouping can be imported. See the Datadog documentation on [Anatomy of a metric query](https://docs.datadoghq.com/metrics/#anatomy-of-a-metric-query).
For example, the following query is not supported:
`max:system.disk.in_use{host:app_prod} by {device}`
While this query is supported:
`max:system.disk.in_use{host:app_prod}`
{% endhint %}
### In ilert
1. Go to **Status pages** (or **Incident comms**) --> **Metrics,** and click **Create metric.**
2. Select the Datadog metrics data source that you have created above and click **Next.**
3. On the next screen, enter a metric name and a display unit. Paste the metric query from step 4 above. Click **Save** to preview your metric data.
4. Preview your metric. You can make changes to your query and reload the data from Datadog. Note that once you have created your metric, you will no longer be able to change the metric query.
### Further References
Here are the articles on how to connect Datadog and ilert:\
\
[Datadog Inbound Integratio](/integrations/inbound-integrations/datadog)
# Import metrics from Prometheus
Import metrics from Prometheus into ilert to visualise data and make it available on a status page.
## Create a Prometheus metric data source
### In ilert
1. Go to **Incident comms** → **Metric data sources** and click on **Create metric data source.**
2. Give the metrics data source a name (e.g. Prometheus).
3. Select **Prometheus** in **Type** dropdown list.
4. Type in your Prometheus URL. For ex.: .
5. Choose your **Authentication** method.
{% hint style="info" %}
**Warning:** We recommend that you secure your Prometheus instance by using an authentication method.
{% endhint %}
**Basic auth:**
1. Enter your login credentials for Prometheus, e.g. username and password and click on **Create metric data source**.
**Custom header:**
1. Enter a custom header name, for ex. "Cookie", if you wish to authenticate via cookie header.
2. Paste your custom header value for the header name. In case of "Cookie" enter the cookie's value here and click on **Create metric data source**.
## Create a metric
### In Prometheus
1. Go to **Graph** --> Type in a **query** --> Test your result with **Execute.**
2. Select a query from which you want to import the data into ilert. Copy the query, you will need it later.
**In ilert**
1. Go to **Status pages** (or **Incident comms**) --> **Metrics** and click on **Create metric.**
2. Select the Prometheus metrics data source that you have created above and click on **Next.**
3. On the next screen, enter a metric name, a display unit and paste the metric query from step 2 "In Prometheus" above. Click **Save** to preview your metric data.
4. Preview your metric. You can make changes to your query and reload the data from Prometheus. Note that once you have created your metric, you will no longer be able to change the metric query.
# Overview
Receive and act on alerts, raise new ones, and look up who is on-call — from Slack, Microsoft Teams, or Google Chat.
ChatOps handles alerts where your team already talks. Alerts are published into channels you already have, responders accept, reroute, and resolve them there without opening ilert, and colleagues who have no ilert account can still raise one from a channel they already use.
## What you can do
ilert integrates with **Slack**, **Microsoft Teams**, and **Google Chat**. All three support the same four things:
* [Receive and respond to alerts](#receive-and-respond-to-alerts)
* [Raise an alert from chat](#raise-an-alert-from-chat)
* [Look up who is on-call](#look-up-who-is-on-call)
* [Open a dedicated channel](#open-a-dedicated-channel), on an incident
The platforms differ only in how you invoke them: Slack and Google Chat use slash commands, Microsoft Teams uses a mention of the ilert bot.
{% hint style="info" %}
**Existing channels carry alerts. A dedicated channel belongs to an incident.**
ChatOps sends alerts into channels your team already has, and lets responders act on them there. You cannot create a channel of its own for an alert. When a problem needs one, you [declare an incident](/incidents-and-status-pages/incidents/declare-an-incident) and create an [incident channel](/incidents-and-status-pages/incidents/incident-channels) with it.
{% endhint %}
## Receive and respond to alerts
Alerts arrive in the channel as cards showing status and priority, with buttons to accept, reroute, and resolve. The card updates in place as the alert changes, so the channel reflects the current state rather than a growing pile of notifications.
* [Slack](/chatops/slack/post-alerts-to-a-slack-channel)
* [Microsoft Teams](/chatops/microsoft-teams/chat)
* [Google Chat](/chatops/google-chat/google-chat-app)
## Raise an alert from chat
Anyone can report a problem from a channel. Each platform offers two modes:
| Mode | Who can raise an alert | Setup |
| -------------- | ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| **Restricted** | Only chat users who also have an ilert account | None beyond installing the app |
| **Open** | Any user in the workspace, ilert account or not | Requires a dedicated alert source, which fixes the channels, escalation policies, and priority available to them |
The open mode is the useful one for support and sales colleagues who need to escalate but will never log in to ilert.
* [Slack](/chatops/slack/create-alerts-in-slack)
* [Microsoft Teams](/chatops/microsoft-teams/create-alerts-in-microsoft-teams)
* [Google Chat](/chatops/google-chat/create-alerts-in-google-chat)
## Open a dedicated channel
A channel of its own belongs to an **incident**. Declare the incident — from scratch, or from the alert that surfaced the problem — and create the channel with it. Responders are invited automatically, and the incident keeps the structured record while the channel carries the conversation.
* [Incident channels](/incidents-and-status-pages/incidents/incident-channels) — how to open one, on any of the three platforms
## Look up who is on-call
Check who is currently on-call for an alert source or an escalation policy without leaving the channel. Like alert creation, this can be restricted to users with an ilert account or opened to the whole workspace.
* [Slack](/chatops/slack/lookup-who-is-on-call)
* [Microsoft Teams](/chatops/microsoft-teams/look-up-who-is-on-call-in-microsoft-teams)
* [Google Chat](/chatops/google-chat/look-up-who-is-on-call-in-google-chat)
## Set up your platform
# Integration for Slack
Connect a Slack workspace to ilert, then receive alerts, raise them, and look up who is on-call from your channels.
With the integration for Slack you can receive alerts in Slack channels and act on them there, raise new alerts from a channel, and check who is on-call — without opening ilert.
Connecting a workspace is a one-time step by an admin. After that, each capability below is configured on its own.
## Select your use case
Receive and respond to alerts in Slack
Receive and acknowledge alerts in a Slack channel of your choice.
## Authorize your Slack workspace
{% hint style="info" %}
**Admin permission required**
You need to have global **Admin** privileges to set up the integration in ilert.
{% endhint %}
{% stepper %}
{% step %}
## Open the Slack connector
In the sidebar, go to **Settings** → **Connectors**, then click **Slack Workspace Integration**.
{% endstep %}
{% step %}
## Authorize
Click **Authorize**. You are forwarded to Slack — log in if you are not already — and asked to allow ilert to access your workspace.
{% endstep %}
{% step %}
## Confirm
Slack returns you to ilert. A success message confirms the workspace is connected.
{% endstep %}
{% endstepper %}
Add the **ilert** bot to any channel where you want to receive alerts.
Re-authorizing your Slack Workspace
You **may** need to re-authorize the bot for your Slack workspace in case new features of the bot require additional permissions e.g. alert actions or automatic user mapping.
In the sidebar, go to **Settings** → **Connectors**, click **Slack Workspace Integration**, then click **Re-authorize Slack**.
## FAQ
**Can I link multiple Slack Workspaces to one ilert account?**
No, only one Slack Workspace can be linked to an ilert account.
**Can multiple ilert accounts be linked to the same Slack workspace?**
No, a Slack workspace can only be linked to one ilert account at a time.
# Receive and respond to alerts in Slack
Post ilert alerts into a Slack channel and act on them there, keeping the team informed as alerts are raised, accepted, and resolved.
{% hint style="info" %}
**Connect your Slack workspace with ilert first**
Before you proceed, please make sure that a global admin has connected your Slack workspace with your ilert account (as described in our [integration for Slack guide](/chatops/slack))
{% endhint %}
Alerts reach Slack through an **alert action** — a rule on an alert source that publishes to a channel. Create one per channel you want to post to.
## Create a Slack alert action
{% stepper %}
{% step %}
## Start a new alert action
In the sidebar, go to **Alerting** → **Alert actions** and click **Create new alert action**.
{% endstep %}
{% step %}
## Choose Slack
Select **Slack**. Leave **Use webhook** unticked — the webhook variant posts plain messages without the action buttons.
Pick Slack, and leave Use webhook unticked to get interactive alert messages.
{% endstep %}
{% step %}
## Pick the alert source and the events to publish
Click **Next**, then choose the alert source and which events should reach Slack. *Alert created, accepted, resolved and comment added* is a good default.
{% endstep %}
{% step %}
## Skip the conditions
Click **Next** past the conditional execution page unless you want payload filters now.
{% endstep %}
{% step %}
## Name it and choose the channel
Give the alert action a name and select the target channel.
The channel field is a typeahead. If it comes up empty, see the FAQ below.
{% endstep %}
{% step %}
## Save
Click **Save**, then **Save** again on the edit view to reach the alert action's detail page.
The detail page summarizes what will be published, and where.
{% endstep %}
{% step %}
## Test it
Click **Test** in the top right. A test alert appears in the channel.
The test message confirms the connection.
{% endstep %}
{% endstepper %}
{% hint style="info" %}
The test message has no action buttons. Real alerts arrive with them — accept, resolve, and more.
{% endhint %}
## FAQ
### Are updates to an alert published in Slack?
Yes. These events can be published:
| Event | Happens when |
| ----------------------- | --------------------------------------------------------- |
| **Alert escalated** | The alert moves to the next escalation level |
| **Alert re-routed** | Someone assigns the alert to a different policy or person |
| **Alert accepted** | A responder takes ownership |
| **Alert resolved** | The alert is closed |
| **Alert comment added** | Someone comments on the alert |
| **Merges** | The alert is merged into another |
### Can I choose which updates to an alert are published in Slack?
Yes. Edit the alert action and change its trigger events.
### Why am I not seeing my Slack channels in the ilert UI?
Three things have to be true before a channel appears in the dropdown.
**First**, your ilert account email must match your Slack account email. ilert enforces Slack's own channel permissions, and it uses the email to work out who you are in Slack. If they differ, the dropdown stays empty.
{% hint style="warning" %}
Where the two emails cannot be kept in sync, ilert detects this and lets administrators list every channel the Slack bot can access. Non-admin users still see an empty list and have to ask an administrator. Keeping the emails in sync is the better option — then Slack's channel permissions apply to every role, administrators included.
{% endhint %}
**Second**, you must be a member of the channel at the time you create the alert action. You can leave afterwards. In practice this means the channel has to appear under **Channels** in your Slack sidebar.
The channel has to be one you have joined.
**Third**, public channels appear immediately, but private ones do not. Invite the ilert bot to a private channel — after joining it yourself — by mentioning `@ilert` and sending the message.
Mention the bot and send the message.
Slack then offers to add the bot.
Click Add Them.
Confirmation that the bot has joined.
Back in the alert action, search again or reload the page. The channel now appears.
# Create alerts in Slack
Use Slack as an alert source. Trigger new alerts directly from Slack using slash commands, enabling teams to escalate issues without switching platforms.
{% hint style="info" %}
**Connect your Slack workspace with ilert first**
Before you proceed, please make sure that a global admin has connected your Slack workspace with your ilert account (as described in our [integration for Slack guide](/chatops/slack))
{% endhint %}
Our integration for Slack lets you create alerts directly within Slack and lets your team and colleagues without an ilert account report a problem from a channel they already use.
## Overview
There are two ways to configure the alert creation feature in Slack:
1. **Restrict to Slack users with an ilert account**: This mode doesn't require any additional configuration other than installing our Slack app (as described in the [integration for Slack guide](/chatops/slack)).
2. **Allow any Slack user to create an alert:** This method requires the creation of a dedicated Slack alert source in ilert and allows you to control the Slack channels where users will be able to create an alert, to which escalation policy the alert is routed to and what [notification priority](/alerting/configure-alerting/alert-sources#notification-priority-and-support-hours) is used.
## Option 1: Restrict to Slack users with an ilert account
Once you have our Slack app installed in your Slack workspace, any Slack user with an ilert account can create an alert from any channel by invoking the `/il-alert` slash command.
The user's permissions in ilert will be taken into account. Therefore, they will only see alert sources and escalation policies which they have access to.
## Option 2: A**llow any Slack user to create an alert**
You can let any Slack user (even if they don't have an ilert account) create an alert from Slack. In order to control which teams they will be able to alert and control notification priority, you need to create a dedicated Slack alert source in ilert.
1. In the sidebar, go to **Alerting** → **Alert sources** and click **Create new alert source**
2. Select **Slack** as the alert source type, give it a name, e.g. *Slack alerts from channel xyz*, select an escalation policy and click on **Create**.
3. In the next screen, click on the **Slack settings** tab to configure
* the Slack channels from where Slack users should be able to create alerts
* and the escalation policies that users can select from
4. Now any Slack user in your workspace will be able to create alerts from the configured channels. They will be limited to the pre-configured escalation policies and the alert creation will take into account whatever is configured in the alert source (e.g. notification priority and other alert actions). To create a new alert, use the following command in any of the configured channel
```
/il-alert
```
{% embed url="" %}
Check out our step-by-step tutorial on Youtube
{% endembed %}
## FAQ
### **I have received following error messages:** There is no Slack alert source configured for this channel in ilert, which is required to enable alert creation for Slack users without an ilert account.
If an unauthorized user tries to create a new alert from Slack, the ilert bot for Slack requires the channel to be set in a Slack alert source in ilert. To fix this issue, please follow the steps in [Option 2](#option-2-allow-any-slack-user-to-create-an-alert).
If your ilert bot for Slack has no permission, [re-authorize the workspace](/chatops/slack) — see **Re-authorizing your Slack Workspace** on the integration page.
# Look up who is on-call
Lookup who is on-call right from Slack using the /il-oncall slash command, eliminating confusion during escalations.
{% hint style="info" %}
**Connect your Slack workspace with ilert first**
Before you proceed, please make sure that a global admin has connected your Slack workspace with your ilert account (as described in our [integration for Slack guide](/chatops/slack))
{% endhint %}
## Overview
The `/il-oncall` Slash command lets you look up who is on-call from any Slack channel. There are two ways to configure the on-call lookup feature in Slack:
1. **Restrict lookup to Slack users with an ilert account**: This mode doesn't require any additional configuration other than installing our Slack app (as described in the [integration for Slack guide](/chatops/slack)).
2. **Allow any Slack user to lookup who is on-call:** This method requires the creation of a dedicated Slack alert source in ilert and allows you to control the Slack channels where users will be able to look up who is on-call. One way to use this feature is for example to configure the lookup for every team channel and letting any Slack user use the `/il-oncall` command to look up the on-call responder for that team, without having to know the names of the escalation policy for that team.
## Option 1: Restrict to Slack users with an ilert account
Once you have our Slack app installed in your Slack workspace, any Slack user with an ilert account can look up the on-call responder for any escalation policy or alert source by invoking the `/il-oncall` slash command.
The user's permissions in ilert will be taken into account. Therefore, they will only see alert sources and escalation policies which they have access to.
## Option 2: A**llow any Slack user to lookup who is on-call**
You can let any Slack user (even if they don't have an ilert account) lookup who is on-call from within a Slack channel. You need to create a dedicated Slack alert source in ilert in order to control where the lookup is available and which escalation policies should be used for the lookup.
1. In the sidebar, go to **Alerting** → **Alert sources** and click **Create new alert source**
2. Select **Slack** as the alert source type, give it a name, e.g. *Slack alerts from channel xyz*, select an escalation policy and click on **Create**.
3. In the next screen, click on the **Slack settings** tab to configure
* the Slack channels from where Slack users should be able lookup who is on-call
* and the escalation policies that will be used for the lookup
4. Save your changes.
5. Now any Slack user in your workspace will be able to look up who is on-call from the configured channels. The lookup will be limited to the pre-configured escalation policies. To lookup who is on-call, use the following the slash command in the configured Slack channel
```
/il-oncall
```
{% embed url="" %}
Check out our step-by-step tutorial on Youtube
{% endembed %}
# Microsoft Teams Integration
Install the certified ilert bot in Microsoft Teams to receive and act on alerts, raise them, and look up who is on-call.
Install the ilert bot in your Microsoft Teams tenant, then configure each capability you need. The bot is certified by Microsoft; the scopes it requests are listed under [Scope definitions](#scope-definitions).
## Select your use case
Receive and respond to alerts in Microsoft Teams
Post alerts into a channel and act on them there, via the chat app or an incoming webhook.
## Certification
Our bot has undergone a rigorous certification process by Microsoft, ensuring it meets the high standards for security, privacy, and compliance set forth by Microsoft. This attestation confirms that our app adheres to best practices in software development and data handling.
For a deeper understanding of what certification means and why it matters, please refer to the [Microsoft Certification Documentation](https://docs.microsoft.com/en-us/microsoft-365/compliance/offering-certifications?view=o365-worldwide) for third-party apps and services.
## Scope definitions
Below is a list and description of each scope used by our bot:
### 1. `Calendars.ReadWrite`
* **Description:** Allows the bot to create, read, update, and delete events in user calendars.
* **Use Case:** This permission gives bot the opportunity to manage meetings that related to an alert or directly from within Microsoft Teams
### 2. `Channel.Create`
* **Description:** Enables the bot to create new channels.
* **Use Case:** Create alert channels directly from alert details view or automatically via alert action.
### 3. `Channel.ReadBasic.All`
* **Description:** Grants the ability to read basic properties of channels, including names and descriptions.
* **Use Case:** This scope is commonly used to allow the bot to provide a summary of available channels or guide users to the right channel.
### 4. `ChannelMember.Read.All`
* **Description:** Allows reading the list of members in each channel.
* **Use Case:** Necessary for bot that actions based on channel membership e.g. alert acknowledgment.
### 5. `ChannelMember.ReadWrite.All`
* **Description:** Allows the bot to add or remove members from channels.
* **Use Case:** Necessary for bot to add users to the created channel that are working on an alert.
### 6. `Chat.Create`
* **Description:** Permits the bot to create one-on-one or group chats.
* **Use Case:** Enable bots to initiate conversations based on alert actions.
### 7. `Directory.Read.All`
* **Description:** Allows reading data in your organization's directory, such as users, groups, and apps.
* **Use Case:** Required for the bot to get basic information about a user who is active in ilert and Microsoft Teams, and then combine those activities together.
### 8. `Group.ReadWrite.All`
* **Description:** Enables the bot to read and modify groups, including creating new groups or updating group settings.
* **Use Case:** Enables the bot to show information about groups in ilert management UI
### 9. `Team.ReadBasic.All`
* **Description:** Provides access to read basic properties of teams, such as team name and description.
* **Use Case:** Enables the bot to show information about teams in ilert management UI
### 10. `User.Read.All`
* **Description:** Allows reading basic profiles of all users in your organization.
* **Use Case:** Required for the bot to get basic information about a user who is active in ilert and Microsoft Teams, and then combine those activities together.
The use of these scopes ensures that our bot can interact effectively with Microsoft Teams and its users, providing functionalities that enhance productivity and collaboration within the platform. The scopes are managed carefully to balance functionality with security and privacy concerns.
# Microsoft Teams Chat Integration
The ilert Microsoft Teams Chat Integration helps you to bring alerts into your channels, acknowledge or resolve alerts without leaving the chat.
[Microsoft Teams](https://www.microsoft.com/en-ww/microsoft-teams/group-chat-software) is the hub for team collaboration in Microsoft 365 that integrates the people, content, and tools your team needs to be more engaged and effective.
## In Microsoft Teams
### Install the ilert bot application
{% hint style="info" %}
**Admin permission required**
To install the bot application, you must have admin rights in Microsoft Teams.
{% endhint %}
1. Open the application in Microsoft Teams:
2. Add the ilert bot to a team
1. Choose a team and channel name and click on the **Set up a bot** button
1. You should have received a welcome message in the previously selected channel, if you do not see the message or you want to reconnect use the `@iLert connect` command to bring it up again. Click on the **Connect** button in the message, this will take you to ilert's login page to finish your connection.
{% hint style="info" %}
**Admin permission required**
To set up the integration, you must have admin rights in ilert.
{% endhint %}
1. Login to the ilert account which you want to connect to Microsoft Teams and ilert will automatically set up the connection for you - *depending on your login state in Microsoft 365 you will have to login to Microsoft again, afterwards you will be automatically taken back to ilert* and you should see a success message with your newly created connector.
## In ilert
Now that the initial connection between your Microsoft Teams and ilert accounts has been set up, you may choose alert sources which should send update messages to your Microsoft Teams channels - this is done by creating alert actions in ilert.
### Link the Microsoft Teams Chat Connector to the alert source
1. In the sidebar, go to **Alerting** → **Alert sources** and open the alert source whose alerts you want to publish to Microsoft Teams. On the **Alert actions** tab, click **Create new alert action**.
1. On the next page choose **Microsoft Teams** as the type, choose the connector created before, name your action\*\*,\*\* choose **Chat** as Teams Action, choose **Your team**, then choose **Your channel** and click on the **Save** button.
1. Finished! You can now test the connection by clicking on the button **Test this connection**. Thereafter, a test message will be posted in your Microsoft Teams channel.
## FAQ
**Can I link multiple Microsoft Teams Accounts to an ilert account?**
No.
**Are updates to an alert published on the Microsoft Teams Chat channel?**
Yes, the following updates to an alert are currently being published:
* **Escalations** : An alert is assigned to another user through an automatic escalation.
* **Manual Assignments** : An alert is manually assigned to someone.
* **Actions** : An alert is accepted or resolved.
**Can I choose which updates to an alert will be published in Microsoft Teams Chat?**
No.
**How can I uninstall the ilert App from my Microsoft Teams account?**
1. Login to your Microsoft Teams Account and navigate to your team
2. Click on the **More options** menu and then on the **Manage team** option
3. Click on the **Apps** tab
4. Find the **ilert** app
5. Click on the **Uninstall** button
**The ilert app does not see all team groups of my Microsoft Teams account, what can I do?**
The ilert app needs the `Group.ReadWrite.All` and `Team.ReadBasic.All` scopes to read the all team groups including legacy Skype groups. Please contact the Microsoft Teams admin who can give the permission for the ilert app. You can read more about the required scopes [here](https://docs.microsoft.com/en-us/graph/teams-list-all-teams).
**The ilert app does not see all channels of my team, what can I do?**
The ilert app needs the `Channel.ReadBasic.All` scope to read the team groups. Please contact the Microsoft Teams admin who can give the permission for the ilert app. You can read more about the required scope [here](https://docs.microsoft.com/en-us/graph/api/channel-list).
If you still don't see all the channels, then those are private channels that are protected, and you need to invite the bot to the channel.
**How can I add a comment to an alert from Microsoft Teams?**
Click on **Reply in thread** from the alert message and post your reply.
Inside channels with Threads layout
Inside channels with Posts layout
Your reply appears in the alert's **Comments** tab, if the user exists in ilert.
# Microsoft Teams Integration via Incoming Webhook
The ilert Microsoft Teams Integration helps you to bring alerts into your channels without installing the app.
{% hint style="danger" %}
**Office 365 connectors within Microsoft Teams are deprecated, they will no longer work after 15.08.24.** Please use [Workflow](/chatops/microsoft-teams/chat/workflows), if you want to migrate an existing alert action you will need to create a new one.
{% endhint %}
{% hint style="warning" %}
If possible we suggest to use [ilert's Microsoft Teams Bot](/chatops/microsoft-teams/chat) for the setup, this guide and integration is only suggested in case you do not want to grant any permissions to ilert
{% endhint %}
## In Microsoft Teams: Add an ilert Connector to a channel
{% hint style="info" %}
**Admin permission required**
To set up the integration, you must have admin rights in ilert.
{% endhint %}
1. Select the channel in which you want to publish ilert Alerts and click **...** -> **Connectors**
2. Type webhook in the search field and click **Add**
3. Again click on **Add**
4. Now in **Connectors** click on **Configure**.
5. Type the **connector Name** and click **Create**.
6. Copy the **connector URL** and click **Done**.
7. Your connector has now been set up. You will need the URL from step 4 in ilert.
## In Microsoft Teams: Manage existing Connectors
1. Select the channel in which you want to publish ilert Alerts and click **...** -> **Connectors**
2. In the **Manage** tab click on **Configured**.
3. Now open up the dropdown list by clicking on **Configured** and click on the **Manage** button of the desired connector.
## In ilert: Create the Microsoft Teams Connector and link it to the alert source
1. Click the gear icon → **Connectors**
2. Click **Create Connector**
3. Select **Microsoft Teams** as **Type**. Assign a name for the connector, enter the URL from [above](#add-to-channel) and save it.
4. In the sidebar, go to **Alerting** → **Alert sources** and open the alert source whose alerts you want to publish to Microsoft Teams. On the **Alert actions** tab, click **Create new alert action**.
5. Select **Microsoft Teams** as the **type**, select the connector created in step 3, and name it.
6. Select the alert filters and conditions for execution as needed and save the alert action.
7. Finished! You can now test the connection by clicking on the button **Test this alert action**. Thereafter, a test message will be posted on the Microsoft Teams channel.
## FAQ
**Can I link multiple Microsoft Teams Spaces to an ilert account?**
Yes.
**Are updates to an alert published on the Microsoft Teams channel?**
Yes, the following updates to an alert are currently being released:
* **Escalations** : An alert is assigned to another user through an automatic escalation.
* **Manual Assignments** : An alert is manually assigned to someone.
* **Actions** : An alert is accepted or resolved.
# Microsoft Teams Integration via Workflows
The ilert Microsoft Teams Integration helps you to bring alerts into your channels without installing the app.
{% hint style="warning" %}
If possible we suggest to use [iLert's Microsoft Teams Bot](/chatops/microsoft-teams/chat) for the setup, this guide and integration is only suggested in case you do not want to grant any permissions to ilert
{% endhint %}
## In Microsoft Teams: Add a workflow to a channel
{% hint style="info" %}
**Admin permission required**
To set up the integration, you must have admin rights in ilert.
{% endhint %}
1. Select the channel in which you want to publish ilert Alerts and click **...** -> **Connectors**
2. Type webhook in the search field and click on the **Webhook Template**
3. Name your workflow and click on **Next**
4. Review details and click on **Add workflow**.
5. On the next page copy the **workflow URL** and click **Done**.
7. Your connector has now been set up. You will need the URL from step 4 in ilert.
## In ilert: Create the Microsoft Teams Alert Action and link it to the alert source
1. In the sidebar, go to **Alerting** → **Alert actions** and click **Create new alert action**
2. Select **Microsoft Teams** as **Type** and choose **Use webhook** option. Then click on the **Next** button.
4. On the next page choose the alert source whose alerts you want to publish to Microsoft Teams, choose Trigger Events and click on the **Next** button
5. On the next page name the alert action, enter the **URL** from [above](#add-to-channel) and click on the **Save** button.
7. Finished! Your Microsoft Teams via Workflows integration is set up.
## FAQ
**Can I link multiple Microsoft Teams Spaces to an ilert account?**
Yes.
**Are updates to an alert published on the Microsoft Teams channel?**
Yes, the following updates to an alert are currently being released:
* **Escalations** : An alert is assigned to another user through an automatic escalation.
* **Manual Assignments** : An alert is manually assigned to someone.
* **Actions** : An alert is accepted or resolved.
# Create alerts in Microsoft Teams
Use Microsoft Teams as an alert source. Use chat commands in Microsoft Teams to create ilert alerts and notify your response team seamlessly where you already collaborate.
{% hint style="info" %}
**Connect your Microsoft Teams workspace with ilert first**
Before you proceed, please make sure that a global admin has connected your Microsoft Teams workspace with your ilert account (as described in our [integration for Microsoft Teams guide](/chatops/microsoft-teams/chat#install-the-ilert-bot-application))
{% endhint %}
Our integration for Microsoft Teams lets you create alerts directly within Microsoft Teams and lets your team and colleagues without an ilert account report a problem from a channel they already use.
## Overview
There are two ways to configure the alert creation feature in Microsoft Teams:
1. **Restrict to Microsoft Teams users with an ilert account**: This mode doesn't require any additional configuration other than installing our Microsoft Teams app (as described in the [integration for Microsoft Teams guide](/chatops/microsoft-teams/chat))
2. **Allow any Microsoft Teams user to create an alert:** This method requires the creation of a dedicated Microsoft Teams alert source in ilert and allows you to control the Microsoft Teams channels where users will be able to create an alert, to which escalation policy the alert is routed to and what [notification priority](/alerting/configure-alerting/alert-sources#notification-priority-and-support-hours) is used.
## Option 1: Restrict to Microsoft Teams users with an ilert account
Once you have our Microsoft Teams app installed in your Microsoft Teams workspace, any Microsoft Teams user with an ilert account can create an alert from any channel by mentioning the ilert bot with `@iLert alert`.
Enter an **Alert summary** and click on the **Continue** button.
The user's permissions in ilert will be taken into account. Therefore, they will only see alert sources and escalation policies which they have access to.
## Option 2: Allow any Microsoft Teams user to create an alert
You can let any Microsoft Teams user (even if they don't have an ilert account) create an alert from Microsoft Teams. In order to control which teams they will be able to alert and control notification priority, you need to create a dedicated Microsoft Teams alert source in ilert.
1. In the sidebar, go to **Alerting** → **Alert sources** and click **Create new alert source**
2. Select **Microsoft Teams** as the alert source type, give it a name, e.g. *Microsoft Teams alerts from channel xyz*, select an escalation policy and click on **Create**.
3. In the next screen, click on the **Microsoft Teams settings** tab to configure
* the Microsoft Teams channels from where Microsoft Teams users should be able to create alerts
* and the escalation policies that users can select from
4. Now any Microsoft Teams user in your workspace will be able to create alerts from the configured channels. They will be limited to the pre-configured escalation policies and the alert creation will take into account whatever is configured in the alert source (e.g. notification priority and other alert actions). To create a new alert, mention the ilert bot with the following command in any of the configured channel
```
@iLert alert
```
5. Enter an **Alert summary** and click on the **Continue** button.
{% embed url="" %}
Check out our step-by-step tutorial on YouTube
{% endembed %}
## FAQ
### **I have received following error messages:** There is no Microsoft Teams alert source configured for this channel in ilert, which is required to enable alert creation for Microsoft Teams users without an ilert account.
If an unauthorized user tries to create a new alert from Microsoft Teams, the ilert bot for Microsoft Teams requires the channel to be set in a Microsoft Teams alert source in ilert. To fix this issue, please follow the steps in [Option 2](#option-2-allow-any-microsoft-teams-user-to-create-an-alert).
If your ilert bot for Microsoft Teams bot has no permission. Please reauthorize your bot.
# Look up who is on-call in Microsoft Teams
Check who is on-call right from Microsoft Teams using the @ilert on-call command without leaving your team's channels.
{% hint style="info" %}
**Connect your Microsoft Teams with ilert first.**
Before you proceed, please make sure that a global admin has connected your Microsoft Teams workspace with your ilert account (as described in our [Integration for Microsoft Teams guide](/chatops/microsoft-teams/chat#install-the-ilert-bot-application)).
{% endhint %}
The `@ilert oncall` command lets you look up who is on-call from any Microsoft Teams channel. There are two ways to configure the on-call lookup feature:
1. **Restrict lookup to Microsoft Teams users with an ilert account:** This mode doesn't require any additional configuration other than installing the ilert bot application (it is described in this article: [Microsoft Teams Integration](/chatops/microsoft-teams/chat)).
2. **Allow any Microsoft Teams user to use the on-call lookup feature:** This method requires the creation of a dedicated Microsoft Teams alert source in ilert and allows you to control the Microsoft Teams channels where users can check who is on-call.
## Option 1: Restrict to Microsoft Teams users with an ilert account
Once you have ilert Microsoft Teams app installed in your Microsoft Teams workspace, any Microsoft Teams user with an ilert account can look up who is on-call from any channel by mentioning the ilert bot with `@ilert oncall`.
Push **Continue** button, then **Open Search Menu** in the popup menu, and choose between two options: on-call lookup for alert sources or for escalation policies. You will then see the on-call person's details.
## Option 2: Allow any Microsoft Teams user to use on-call lookup
You can let any Microsoft Teams user (even if they don't have an ilert account) check who is on-call from Microsoft Teams. To do so, you need to create a dedicated Microsoft Teams alert source in ilert.
1. In the sidebar, go to **Alerting** → **Alert sources** and click on **Create new alert source.**
2. Select **Microsoft Teams** as the alert source type, give it a name, select an escalation policy, and click on **Create** button.
3. In the next screen, click on the **Microsoft Teams settings** tab to configure Microsoft Teams team, where this alert source should be available, and where users should be able to look up who is on-call.
Now, any Microsoft Teams user in your workspace will be able to use the on-call lookup feature from the configured channels.
{% embed url="" %}
Check out our step-by-step tutorials on Youtube
{% endembed %}
# Google Workspace Integration
Connect ilert to Google Chat to receive and act on alerts, raise them, and look up who is on-call from your spaces.
Connect ilert to Google Chat to receive alerts in your spaces and act on them there, raise new alerts, and check who is on-call.
Start by choosing how alerts reach Google Chat, then configure the capabilities you need.
## Choose how alerts reach Google Chat
| | [Google Chat app](/chatops/google-chat/google-chat-app) | [Incoming webhook](/chatops/google-chat/incoming-webhook) |
| ---------------------------------- | ------------------------------------------------------- | --------------------------------------------------------- |
| Alert cards | Full, with status and priority | Basic message |
| Accept, reroute, resolve from chat | Yes | No |
| Slash commands | Yes | No |
| Status changes update in place | Yes | No |
| OAuth permissions required | Yes | No |
| Setup effort | Medium | Low |
The **Google Chat app** is the right choice unless your organization cannot grant app permissions — a webhook only posts notifications, so responders still have to open ilert to act.
## Select your use case
Receive and respond to alerts
Install the Google Chat app and act on alerts from the space.
# Google Chat App
Integrate ilert with Google Chat to receive alerts and respond to them directly in your spaces.
{% hint style="warning" %}
The Google Chat integration is currently in beta. Core functionality is stable, but some features may be added or refined based on user feedback.
{% endhint %}
The ilert Google Chat Integration helps you to bring alerts into your channels, acknowledge or resolve alerts without leaving the chat.
## Features
* **Receive alert notifications** directly in Google Chat spaces
* **Take action on alerts** - Accept, Reroute, or Resolve with one click
* **View alert details** - Priority, escalation policy, responders, and labels
* **Slash commands** - Quick access to help and connection management
## In Google Workspace
### Install the app from Google Marketplace
Follow the instructions to install the ilert app: [Install Marketplace apps for your organization - Google Workspace Admin Help](https://support.google.com/a/answer/172482)
{% hint style="info" %}
Administrator privileges are required to install the app for your organization.
{% endhint %}
### In Google Chat
#### Connect Google Chat to your ilert tenant
1. Add the ilert app to a space.
2. Click "Connect" in the message sent by the ilert app.
3. Sign in to your ilert tenant. Admin role is required to connect. You will land on ilert Connectors page and see "Connected" on Google Chat if the connection is successful.
{% hint style="info" %}
You will be informed of successful connection in the same space where the connection was initiated.
{% endhint %}
## In ilert
Now that the initial connection between your Google Workspace and ilert account has been set up, you may choose alert sources which should send update messages to your Google Chat spaces - this is done by creating alert actions in ilert.
### Create an alert action for Google Chat
1. In the sidebar, go to **Alerting** → **Alert actions** and click **Create new alert action**.
2. Choose **Google Chat** from the list. Make sure "Use webhook" is **unselected** to use the native app integration.
3. Click **Next** and configure the trigger settings:
* Select one or more **alert sources** whose alerts you want to publish to Google Chat
* Choose **Trigger mode**: Automatic (recommended) or Manual
* Select the **trigger events** you want to publish:
4. Click **Next**. Optionally configure **Conditional execution** to filter alerts based on specific conditions (e.g., only high-priority alerts).
5. Click **Next** and configure the action:
* Give the alert action a **name**
* Select the **Google Chat space** where you want to receive alerts
* Optionally assign the alert action to specific **teams**
6. Click **Save** to create the alert action.
7. Click the **Test** button in the top right to verify the connection. You should see a test message appear in your Google Chat space.
## Responding to alerts in Google Chat
When an alert is published to your Google Chat space, you'll see a rich notification card containing:
* **Status badge** - Shows current alert status (PENDING, ACCEPTED, RESOLVED)
* **Alert title** - Clickable link to view the full alert in ilert
* **Description** - Alert details (expandable with "Show more")
* **Escalation policy** - The assigned escalation policy
* **Priority** - Alert priority level
* **Responders** - Currently assigned responders
* **Links** - Any associated external links
* **Labels** - Custom labels attached to the alert
### Action buttons
Each alert card includes action buttons to respond without leaving Google Chat:
| Button | Description |
| ---------------- | ------------------------------------------------------- |
| **Accept** | Acknowledge the alert and become a responder |
| **Reroute** | Reassign the alert to another user or escalation policy |
| **Resolve** | Mark the alert as resolved |
| **More actions** | Access additional options |
## Slash commands
The ilert Google Chat app supports the following slash commands:
| Command | Description |
| ---------------- | ---------------------------------------------------- |
| `/il-help` | Display the help card with available commands |
| `/il-connect` | Connect your ilert account to the current space |
| `/il-disconnect` | Disconnect your ilert account from the current space |
To use a slash command, type it in the message compose field within a space where the ilert app is added.
## FAQ
### Can I connect multiple Google Workspace environments or ilert tenants?
No. Each Google Workspace can only be connected to one ilert tenant at a time.
### Why is the ilert app not appearing in the search results?
If you cannot find the app when trying to add it to a space, it likely hasn't been enabled for your domain. Please contact your Google Workspace administrator to ensure the ilert app is installed and permitted for your organizational unit.
### What permissions are required to set up the integration?
* **In ilert**: You need a user with **Admin** or **Account Owner** role to complete the connection.
* **In Google Workspace**: A super administrator account is required to install the app. See [Google's documentation](https://support.google.com/a/answer/172482?hl=en) for details.
### Are updates to an alert published in Google Chat?
Yes. The following alert events can be published to Google Chat (configurable when creating the alert action):
* Alert created
* Alert accepted
* Alert re-routed
* Alert escalated
* Alert priority raised
* Alert resolved
* Alert comment added
* Responder changes
* Channel attachment changes
* Escalation policy completion
* Alert timeout notifications
### Why am I not seeing my Google Chat spaces in the dropdown?
Make sure:
1. The ilert app has been added to the space you want to use
2. Your Google Workspace account has access to the space
3. The connection between ilert and Google Workspace is active (check Connectors page)
If spaces still don't appear, try reconnecting your Google Workspace integration from the Connectors page.
### How do I reconnect or disconnect the integration?
1. Go to **Settings** > **Connectors** in ilert
2. Find **Google Chat** in the ChatOps section
3. Click the three-dot menu and select:
* **Re-connect** - To refresh the connection (useful if spaces aren't appearing)
* **Disconnect** - To remove the integration entirely
### Can team members without ilert accounts respond to alerts?
No. To take actions on alerts (Accept, Reroute, Resolve), users must have an ilert account with appropriate permissions.
## Troubleshooting
### Connection failed during setup
1. Ensure you're logged into ilert with an Admin account
2. Verify your Google Workspace admin has installed the ilert app
3. Check that third-party apps are allowed in your Google Workspace settings
4. Try using an incognito/private browser window to avoid session conflicts
### Alerts not appearing in Google Chat
1. Verify the alert action is enabled and properly configured
2. Check that the selected alert source is generating alerts
3. Confirm the trigger events match the alert activity
4. Test the connection using the "Test" button on the alert action
### Space not showing in dropdown
1. Add the ilert app to the space first
2. Ensure you have access to the space in Google Chat
3. Wait a few moments and refresh the page
4. Try re-connecting the Google Workspace integration
## Related documentation
* [Google Chat Webhooks](/chatops/google-chat/incoming-webhook) - Alternative setup without app permissions
* [ChatOps Overview](/chatops/overview) - Compare with Slack and Microsoft Teams integrations
# Google Chat Webhooks
The ilert Google Chat webhook helps you to bring alerts into your channels without installing the app.
{% hint style="warning" %}
Prefer [ilert's Google Chat app](/chatops/google-chat/google-chat-app). A webhook only posts messages — responders cannot accept, reroute, or resolve from the space. Use this route only if you cannot grant the app permissions.
{% endhint %}
{% hint style="info" %}
To receive messages in a thread, see [Use threads in Chat](#use-threads-in-chat).
{% endhint %}
## In Google Chat
{% stepper %}
{% step %}
## Open the space's apps and integrations
In the space where you want alerts, open the space menu and choose **Apps and integrations**.
{% endstep %}
{% step %}
## Add a webhook
Select **Webhooks**, then click **Add webhooks**.
{% endstep %}
{% step %}
## Name it
Enter a name and, optionally, an avatar URL — use `https://cdn.ilert.com/favicon.ico` for the ilert icon. Click **Save**.
{% endstep %}
{% step %}
## Copy the webhook URL
Open the new webhook's menu and choose **Copy link**.
Treat this URL as a secret — anyone holding it can post to the space.
{% endstep %}
{% endstepper %}
## In ilert
{% stepper %}
{% step %}
## Start a new alert action
In the sidebar, go to **Alerting** → **Alert actions** and click **Create new alert action**.
{% endstep %}
{% step %}
## Choose Google Chat, with the webhook option
Select **Google Chat** and tick **Use webhook**.
{% endstep %}
{% step %}
## Pick the alert source and events
Click **Next**, choose the alert source, and select the events to publish.
Every event you tick becomes a message in the space.
{% endstep %}
{% step %}
## Skip the conditions
Set up conditional execution, or click past it for now.
{% endstep %}
{% step %}
## Paste the webhook URL
Name the alert action and paste the webhook link you copied.
{% endstep %}
{% step %}
## Save
Click **Save**, review the settings, then **Save** again to reach the alert action's detail page.
{% endstep %}
{% endstepper %}
## Use threads in Chat
To group messages into a thread, append `&messageReplyOption=REPLY_MESSAGE_FALLBACK_TO_NEW_THREAD` to the webhook URL.
Given a webhook link of:
```
https://chat.googleapis.com/v1/spaces/AAPQRs0mVWx/messages?key=KKKKKK&token=TTTTTT
```
use:
```
https://chat.googleapis.com/v1/spaces/AAPQRs0mVWx/messages?key=KKKKKK&token=TTTTTT&messageReplyOption=REPLY_MESSAGE_FALLBACK_TO_NEW_THREAD
```
# Create alerts in Google Chat
Use Google Chat as an alert source. Use chat commands in Google Chat to create ilert alerts and notify your response team seamlessly where you already collaborate.
{% hint style="info" %}
**Connect your Google Chat workspace with ilert first**
Before you proceed, please make sure that an administrator has connected your Google Chat workspace with your ilert account (as described in our [Google Chat App guide](/chatops/google-chat/google-chat-app))
{% endhint %}
Our integration for Google Chat lets you create alerts directly within Google Chat and lets your team and colleagues without an ilert account report a problem from a space they already use.
## Overview
There are two ways to configure the alert creation feature in Google Chat:
1. **Restrict to Google Chat users with an ilert account**: This mode doesn't require any additional configuration other than installing our Google Chat app (as described in the [Google Chat App guide](/chatops/google-chat/google-chat-app))
2. **Allow any Google Chat user to create an alert:** This method requires the creation of a dedicated Google Chat alert source in ilert and allows you to control the Google Chat spaces where users will be able to create an alert, to which escalation policy the alert is routed to and what [notification priority](/alerting/configure-alerting/alert-sources#notification-priority-and-support-hours) is used.
## Option 1: Restrict to Google Chat users with an ilert account
Once you have our Google Chat app installed in your Google Chat workspace, any Google Chat user with an ilert account can create an alert from any space by mentioning the ilert app with `/il-alert`.
{% hint style="info" %}
The text after `/il-alert` will be automatically filled in as the alert summary.
{% endhint %}
The user's permissions in ilert will be taken into account. Therefore, they will only see alert sources and escalation policies which they have access to.
## Option 2: Allow any Google Chat user to create an alert
You can let any Google Chat user (even if they don't have an ilert account) create an alert from Google Chat. In order to control which spaces they will be able to alert and control notification priority, you need to create a dedicated Google Chat alert source in ilert.
1. In the sidebar, go to **Alerting** → **Alert sources** and click **Create new alert source**
2. Select **Google Chat** as the alert source type, give it a name, e.g. *Google Chat alerts from space xyz*, select an escalation policy and click on **Create**.
3. In the next screen, click on the **Google Chat settings** tab to configure
* the Google Chat spaces from where Google Chat users should be able to create alerts, and
* the escalation policies that users can select from
4. Now any Google Chat user in your workspace will be able to create alerts from the configured spaces. They will be limited to the pre-configured escalation policies and the alert creation will take into account whatever is configured in the alert source (e.g. notification priority and other alert actions). To create a new alert, use the command `/il-alert` in any of the configured spaces.
## FAQ
### **I have received following error messages:** There is no Google Chat alert source configured for this space in ilert, which is required to enable alert creation for Google Chat users without an ilert account.
If an unauthorised user tries to create a new alert from Google Chat, the ilert app for Google Chat requires the space to be set in a Google Chat alert source in ilert. To fix this issue, please follow the steps in [Option 2](#option-2-allow-any-google-chat-user-to-create-an-alert).
If your ilert app for Google Chat has no permission. Please reauthorise your app.
# Look up who is on-call in Google Chat
Check who is on-call right from Google Chat using the /il-oncall command without leaving your team's spaces.
{% hint style="info" %}
**Connect your Google Chat with ilert first.**
Before you proceed, please make sure that an administrator has connected your Google Chat workspace with your ilert account (as described in our [Google Chat App guide](/chatops/google-chat/google-chat-app)).
{% endhint %}
The `/il-oncall` command lets you look up who is on-call from any Google Chat space. There are two ways to configure the on-call lookup feature:
1. **Restrict lookup to Google Chat users with an ilert account:** This mode doesn't require any additional configuration other than installing the ilert app (as described in the [Google Chat App guide](/chatops/google-chat/google-chat-app)).
2. **Allow any Google Chat user to use the on-call lookup feature:** This method requires the creation of a dedicated Google Chat alert source in ilert and allows you to control the Google Chat spaces where users can check who is on-call.
## Option 1: Restrict to Google Chat users with an ilert account
Once you have the ilert app installed in your Google Chat workspace, any Google Chat user with an ilert account can look up who is on-call from any space by mentioning the ilert app with `/il-oncall`.
Select one of the options: on-call lookup for alert sources or for escalation policies. Then click on the corresponding **Search** button. You will then see the on-call person's details.
## Option 2: Allow any Google Chat user to use on-call lookup
You can let any Google Chat user (even if they don't have an ilert account) check who is on-call from Google Chat. To do so, you need to create a dedicated Google Chat alert source in ilert.
1. In the sidebar, go to **Alerting** → **Alert sources** and click on **Create new alert source.**
2. Select **Google Chat** as the alert source type, give it a name, select an escalation policy, and click on **Create** button.
3. In the next screen, click on the **Google Chat settings** tab to configure the Google Chat space where this alert source should be available, and where users should be able to look up who is on-call.
Now, any Google Chat user in your workspace will be able to use the on-call lookup feature from the configured spaces.
# Getting started with call routing
Start with ilert call routing: configure incoming voice call paths, alert routing logic, and settings for incident response automation.
## **What is Call Routing?**
Call routing is an add-on in ilert, designed to ensure incoming phone calls and voicemails are efficiently directed to the appropriate on-call responder. Calls are routed via ilert's intricate on-call schedules and escalation policies. With call routing, you are assigned a static, dedicated phone number (either local or toll-free, subject to availability). This number then intelligently and dynamically routes calls or voicemails to the current on-call responder in ilert.
## Prerequisites
To access and utilize call routing, you must be on the Professional plan or a higher tier. Once you have purchased the call routing add-on, contact ilert support at to activate a phone number in your account.
## Phone number availability
We offer local phone numbers around the world. Please click on a continent below to see supported countries. For countries not listed below, please reach out to to ask for availability.
North America
* United States
* Canada
* Barbados
* Dominican Republic
* El Salvador
* Jamaica
* Mexico
* Panama
* Puerto Rico
South America
* Argentina
* Chile
* Colombia
* Ecuador
* Grenada
* Brazil
Europe
* Austria
* Belgium
* Bosnia and Herzegovina
* Bulgaria
* Croatia
* Cyprus
* Czech Republic
* Estonia
* France
* Germany
* Greece
* Hungary
* Ireland
* Iceland
* Italy
* Luxembourg
* Portugal
* Poland
* Romania
* Slovenia
* Spain
* Switzerland
* Sweden
* Turkey
* United Kingdom
* Denmark
* Finland
* Netherlands
* Norway
Asia
* Indonesia
* Israel
* Japan
* Macau
* Philippines
* Thailand
Oceania
* Australia
* New Zealand
Africa & Middle East
* Benin
* Ghana
* Kenya
* Mali
* Mauritius
* South Africa
* Sudan
* Tunisia
* Uganda
* Bahrain
* Egypt
* Qatar
* Saudi Arabia
* United Arab Emirates (UAE)
{% hint style="info" %}
**Phone number availability is subject to regulation**
Due to country-specific regulations, phone number provisioning may require identity documentation and location validation. Contact to check availability in your desired locality.
{% endhint %}
## Viewing provisioned phone numbers
Once a call routing number was provisioned from ilert support, you'll find your call routing numbers in the navigation bar at the top under **Call routing** -> **Phone numbers**.
Additionally, you can check if your number is in use and see which call flow it's in.
Next, create a call flow to configure your routing and assign a number to the call flow.
{% content-ref url="/pages/ghZbhBDy70TiAf9lgQhm" %}
[Routing calls using call flows](/call-routing/call-flows)
{% endcontent-ref %}
# Routing calls using call flows
Explore ilert Call flows for advanced voice routing, dynamic call workflows, and enhanced incident call management.
Call Routing 2.0 introduces a new visual call flow builder that allows you to visually construct custom call flows using pre-defined nodes, such as IVR Menus, PIN Code Verification, Audio Messages, Support Hours, and more to tailor the call routing logic to your specific needs.
{% embed url="" %}
{% endembed %}
## Create your first call flow with schedule-based routing
This section outlines the steps to create a call flow that connects incoming calls to the current on-call person via an ilert [on-call schedule](/on-call-management-and-escalations/on-call-schedules). If the on-call individual fails to respond, the call is then redirected to a designated fallback contact, in this example, Cathy Hugh.
1. In the navigation bar, go to **Call flows** and click on **Create call flow.**
2. Add a node by clicking on the ➕ icon and select **Route call**.
3. In the **Route call** configuration, select the desired schedule and user and save the node configuration.
4. Assign the call flow a phone number by clicking the **Incoming call** node at the top and clicking **Publish.**
## Which node types are available, and what do they do?
* [**Support hours**](/call-routing/call-flows/support-hours-node): branches on your defined [support hours](/alerting/configure-alerting/support-hours). Adds two branches: `During support hours` and `Outside support hours`.
* [**IVR menu**](/call-routing/call-flows/ivr-menu-node): adds up to 12 branches (`0-9`, `*`, `#`) and branches based on the callers' DTMF input ("Press 1 for ...").
* **Voicemail:** lets callers leave a voicemail. Voicemails will be appended to alerts created by subsequent **Create alert** nodes. The Voicemail node adds two branches: `Voicemail left` and `No voicemail left`.
* [**Create alert**](/call-routing/call-flows/create-alert-node): creates an alert in the selected alert source. By default, the alert is escalated according to the alert source's priority setting and escalation policy, but you can optionally [override the alert source defaults](/call-routing/call-flows/create-alert-node#override-the-alert-source-defaults) to escalate the alert using a specific escalation policy and/or notification priority instead. You can also accept an alert on an answer automatically.
* **Ask for PIN code:** Asks callers to enter a PIN code and adds two branches: `Code match` and `No code match` to the flow. You can enter multiple PIN codes and associate each PIN code with a label (e.g. if you would like to assign a PIN code to individual customers).
* **Audio message:** Plays an audio message to the caller
* [**Route call**](/call-routing/call-flows/route-call-node): connects the caller with a user, selected directly or via a schedule. You can control the [call options](/call-routing/call-flows/route-call-node#call-options) (order, repeats, ring timeout, hold music). Adds two branches: `Call completed` and `No one available`.
* **Block numbers:** Enables ilert users to create blacklists and reject calls from specific numbers.
* **AI Voice Agent (Beta)**: A human-like agent for natural conversations and intelligent routing. The Agent handles the first communication with callers, processes the information provided, and, depending on the input, creates an incident or calls an on-duty engineer.
{% hint style="info" %}
ilert hotlines can **forward calls directly to external support numbers, even if they use IVR menus—no confirmation needed from the other end**. This feature ensures smooth call handoffs without requiring manual workarounds or dedicated ilert users. It's a simple way to connect your callers with third-party support while keeping your workflows clean and efficient.
{% endhint %}
## **Use ilert AI to turn text into speech**
Make your automatic call routing responses sound human-like. Choose one of the available voice options from the dropdown menu at the top of the call flow editor. The chosen voice will be applied to all the nodes.
## Quick actions with nodes and subtrees
Each node has a menu of actions to simplify the editing process. Click a three-dot icon in front of the node; you will see the following actions:
* Copy node
* Copy subtree
* Cut subtree
These actions are especially useful for complex scenarios with many branches.
## FAQ
### *Question: What are the differences between the new and legacy Call Routing?*
Call Routing 2.0 introduces a radical transformation in how call flows are created and managed compared to legacy call routing. Here's an overview of the key differences:
*Call Flow Design*
* Legacy Call Routing implemented a static call flow design, typically following a linear sequence: Greeting -> IVR Menu -> Voicemail.
* Call Routing 2.0 offers a dynamic and visual interface for building call flows. This flexibility allows for the design of complex scenarios, including multi-level IVR menus and multi-language call flows.
*Nodes and Features*
* Legacy Call Routing lacked the versatility in customizing call flows beyond the basic configurations.
* Call Routing 2.0 introduces nodes, a set of new features that expand functionality. For example, the "Ask for PIN code" node adds an extra layer of security by requiring callers to enter a PIN code.
*Alert Creation*
* In Legacy Call Routing, incoming calls might automatically generate alerts based on predefined conditions.
* Call Routing 2.0 changes this approach by not creating alerts by default. Instead, it employs a Create Alert node within the call flow to generate alerts, offering more control over when and how alerts are created.
*Alert Acknowledgement*
* In Legacy Call Routing the related alert was automatically accepted by the first agent to answer and confirm the call.
* As described in "*Alert Creation*" above, Call Routing 2.0 does not create an alert by default and it also does not track the agent answer to this alert by default - however, due to many customers asking, we have introduced a feature that you can enable on the Create alert node to automatically accept the alert on any agent answer to replicate the legacy behaviour.
*Use of Escalation Policies*
* Legacy Call Routing was somewhat restrictive in including users or schedules into call flows and required you to use [escalation policies](/on-call-management-and-escalations/escalation-policies).
* Call Routing 2.0 simplifies this process, allowing for the direct inclusion of users or schedules, facilitating a more streamlined approach to call routing based on schedules or static user rules.
In summary, Call Routing 2.0 provides a more adaptable, feature-rich platform for call flow management, moving away from the limitations of our old call routing to embrace flexibility, enhanced security, and improved user experience.
### *Question: When do I need to migrate to Call Routing 2.0?*
Upon the general availability release of Call Routing 2.0, existing users must transition from their previous call routing setups to the new version. This migration must be completed within **12 months** of the release date.
### *Question: How to migrate an existing phone number to the new call routing?*
Follow the steps below to seamlessly transition your incoming call routing to the new Call Routing 2.0:
*Prepare your new call flow*
1. Rebuild your legacy configuration using the new Call Flow builder. Do not assign a phone number to it yet. Save your Call Flow by clicking the **Save** button in the top right.
2. (Optional) If you need assistance, our support team is here to help. Email with a link to your legacy routing number and the call flow you've created for a review.
*Publish your call flow*
1. Once your Call Flow is ready, assign a phone number to it. Click on the Incoming call node within your Call Flow and select the number you'd like to use:\
2. Publish your call flow by clicking on the **Publish** button.
3. A warning will appear if the phone number is already in use by the legacy routing system. Proceed by clicking **Publish anyway**.
4. Incoming calls will now use your new call flow.
# Support hours node
Route calls differently during and outside your defined support hours using the Support hours node in an ilert call flow.
The **Support hours** node branches a call flow based on your defined [support hours](/alerting/configure-alerting/support-hours), so callers reach the right destination depending on whether they call inside or outside of working hours — for example a live responder during the day and a voicemail at night.
## Select your support hours
1. *(Optional)* In the **Optional title** field, enter a name to label the node on the canvas *(e.g., "Business hours").*
2. From the **Choose support hours** dropdown, select an existing support-hours definition. The node will evaluate each call against this definition's schedule and time zone.
3. If you just created a new definition and don't see it yet, click the **refresh** icon next to the dropdown to reload the list.
{% hint style="info" %}
Support-hours definitions are created and managed under **Alerting → Support hours**. See [Support hours](/alerting/configure-alerting/support-hours) for details on defining schedules, holidays and time zones.
{% endhint %}
Support hours node configuration
## Branches
The Support hours node adds two branches to the flow. Click the **+** icon under each branch to define what happens next.
| Branch | Taken when |
| ----------------------- | --------------------------------------------------- |
| `During support hours` | The call arrives inside the selected support hours. |
| `Outside support hours` | The call arrives outside them. |
The node adds a "During support hours" and an "Outside support hours" branch
## Example
A typical business-hours flow:
* **During support hours** → [Route call](/call-routing/call-flows/route-call-node) to the on-call schedule, then [Create alert](/call-routing/call-flows/create-alert-node).
* **Outside support hours** → **Voicemail**, then [Create alert](/call-routing/call-flows/create-alert-node) so the recording reaches the team.
## FAQ
**Which time zone is used to decide the branch?**\
The time zone configured on the selected support-hours definition. Update it under **Alerting → Support hours**.
**Can I use the same support hours in more than one call flow?**\
Yes. A support-hours definition can be referenced by multiple Support hours nodes across different call flows.
**Can I nest support hours (e.g. business hours vs. extended hours)?**\
Yes. Add another Support hours node on one of the branches to create a second, more specific time-based split.
# IVR menu node
Build a phone menu that routes callers based on the key they press using the IVR menu node in an ilert call flow.
The **IVR menu** node plays a menu to the caller and routes them based on the key they press (DTMF input) — for example *"Press 1 for production incidents, press 2 for billing."* It adds one branch per key you enable — the digits `0`–`9`, `*`, and `#` (up to 12) — plus a `No input` branch.
## Select the menu keys
1. *(Optional)* In the **Optional title** field, enter a name to label the node on the canvas.
2. Under **Select menu items**, tap the keys the caller is allowed to press (`1`–`9`, `0`, `*`, `#`). Each key you select becomes its own branch in the flow.
## Record the menu prompt
1. Under **Message type**, choose how the prompt is delivered:
* **Text to speech** – type the prompt in the **IVR instructions** field and it is spoken using the call flow's selected [voice](/call-routing/call-flows#use-ilert-ai-to-turn-text-into-speech).
* **Sound file** – upload your own audio recording.
2. Use the **Preview** player to listen to the prompt before saving.
IVR menu node configuration — keypad, message type, instructions and retries
## Set retries
Use the **Retries** setting to control how many times the menu prompt is replayed if the caller provides no input or an invalid key before the flow follows the `No input` branch (default: `1`, maximum `3`).
## Branches
The IVR menu node adds one branch per selected key, plus a fallback branch. Click the **+** icon under each branch to define what happens next.
| Branch | Taken when |
| ---------------------------------------------- | ------------------------------------------------------------ |
| One branch per selected key (`1`, `2`, `*`, …) | The caller presses that key. |
| `No input` | The caller presses nothing valid within the allowed retries. |
Each selected key becomes a branch, plus a "No input" branch
## Example
A first-level menu that splits by team:
* **1** → [Route call](/call-routing/call-flows/route-call-node) to the infrastructure on-call schedule.
* **2** → [Route call](/call-routing/call-flows/route-call-node) to the billing schedule.
* **No input** → **Audio message** ("We didn't receive a selection") → **Voicemail**.
## FAQ
**How many options can one menu have?**\
Up to 12 — the digits `0`–`9`, plus `*` and `#`.
**Can I build multi-level menus?**\
Yes. Add another IVR menu node on any key's branch to create a sub-menu.
**What happens if the caller doesn't press anything?**\
The prompt is replayed according to the **Retries** setting; once retries are exhausted, the call follows the `No input` branch.
# Route call node
Connect callers to a user or on-call schedule and control how targets are dialed using the Route call node in an ilert call flow.
The **Route call** node connects the caller with a live person — a user selected directly, or the person currently on call in a [schedule](/on-call-management-and-escalations/on-call-schedules). It is the core "get a human on the phone" node, and you can fine-tune how multiple targets are dialed.
## Select who to call
1. *(Optional)* In the **Optional title** field, enter a name to label the node on the canvas.
2. Under **Select users or schedules**, add one or more targets via **+ Add user or schedule**:
* A **user** target rings that specific person.
* A **schedule** target rings whoever is currently on call in that schedule.
3. To ring an external number, use **Add phone number**. This requires a plan upgrade and is not available with the **Call all targets simultaneously** option.
Route call node configuration
## Choose how targets are dialed
Use **Call options** to control the order in which targets are tried:
* **Call in sequential order** – targets are called one by one until someone picks up (default).
* **Call in random order** – targets are called one by one in a random order.
* **Call all targets simultaneously** – every target rings at once. This option is available on the **Scale** plan.
Call options — sequential, random, or all targets simultaneously (Scale plan)
You can further tune the dialing behavior:
* **Repeats** – how many times to cycle through the targets if no one answers (default: `No repeats`).
* **Upload custom hold music** – replace the default hold music with your own audio file.
* **Ring timeout (seconds)** – how long each target rings before moving on to the next (default: `45`).
## Branches
The Route call node adds two branches to the flow. Click the **+** icon under each branch to define what happens next.
| Branch | Taken when |
| ------------------ | -------------------------------------------------- |
| `Call completed` | A target answered and the call connected. |
| `No one available` | No target answered within the configured attempts. |
The node adds a "Call completed" and a "No one available" branch
## Example
A schedule-based flow with a fallback:
* **Call completed** → end the call.
* **No one available** → **Voicemail**, then [Create alert](/call-routing/call-flows/create-alert-node) so the on-call team is notified with the recording attached.
## FAQ
**What's the difference between a user target and a schedule target?**\
A user target always rings the same person; a schedule target rings whoever is currently on call in that [schedule](/on-call-management-and-escalations/on-call-schedules), so it follows your rotation automatically.
**Can I ring several people at the same time?**\
Yes — choose **Call all targets simultaneously** under Call options. This option is available on the **Scale** plan.
**Can I route to an external phone number?**\
Yes, via **Add phone number**. This requires a plan upgrade, and external numbers can't be combined with the **Call all targets simultaneously** option.
**What happens if nobody answers?**\
Once the configured attempts and repeats are exhausted, the call follows the `No one available` branch — a good place for a Voicemail and Create alert node.
# Create alert node
Create an alert in an ilert alert source from a call flow, and optionally override the source's escalation policy and notification priority.
The **Create alert** node creates an alert in the selected call-routing alert source. This is how a call turns into an actionable alert for your on-call team. By default the alert is escalated according to the alert source's own notification priority and [escalation policy](/on-call-management-and-escalations/escalation-policies); you can override these per node.
## Select the alert source
1. *(Optional)* In the **Optional title** field, enter a name to label the node on the canvas.
2. From the **Alert source** dropdown, select the call-routing alert source the alert should be created in. If you just created a source and don't see it, click the **refresh** icon to reload the list.
## Accept the alert on answer
Enable **Accept alert on answer** to automatically mark the alert as accepted when a responder answers the call — the first responder to take the call changes the alert status to "Accepted." This is useful when the same node both notifies and connects the responder.
{% hint style="info" %}
A voicemail recorded by an upstream **Voicemail** node is automatically appended to the alert this node creates, so responders can listen to the caller's message from the alert.
{% endhint %}
## Override the alert source defaults
When a **Create alert** node fires, the resulting alert is escalated using the selected alert source's default notification priority and [escalation policy](/on-call-management-and-escalations/escalation-policies). On each **Create alert** node you can optionally override these defaults:
1. In the node configuration, enable **Override alert source defaults**.
2. Optionally select a different **escalation policy** and/or **notification priority** (`High` or `Low`). Each field shows the alert source's current default; any field you leave empty keeps that default.
Create alert node with "Override alert source defaults" enabled
## Example
Pair it with routing and voicemail so nothing is missed:
* **Route call → No one available** → **Voicemail** → **Create alert** (with *Accept alert on answer* enabled and a high-priority escalation policy override).
## FAQ
**Does a call create an alert automatically?**\
No. Call Routing 2.0 only creates alerts where you place a Create alert node, giving you full control over when and how alerts are raised.
**Can different branches create alerts in different sources or priorities?**\
Yes. Add a Create alert node per branch and configure each independently, using the override to set a specific escalation policy or notification priority.
**Are voicemails attached to the alert?**\
Yes — a voicemail from an upstream Voicemail node is appended to the alert created by this node.
# AI Voice Agent
The AI voice agent in ilert enables human-like phone interactions for faster, smarter incident handling. It understands caller intent, gathers key information, and provides real-time updates using connected service data. This helps reduce manual triage, streamline incident reporting, and improve MTTR.
## Add the AI voice agent to your call flow
1. Navigate to **Call routing** and open or create a call flow.
2. Click the **+** button to add a new node.
3. From the node list, select AI voice agent.
4. The AI voice agent configuration panel will open on the right.
## Configure the greeting and agent name
1. In the **Optional title** field, enter a name to label the node *(e.g., "AI Virtual Assistant").*
2. In the **Greeting** field, write a message to welcome the caller *(e.g., "Welcome to ACME, this is Alex, your virtual assistant. How may I assist you today?")*.
## Define caller intents
1. Under **Intent** **configuration**, click **+ Add intent**.
2. Select from existing intents like:
* Report critical incident
* Report system outage
* Report security breach
* Request technical support
* General inquire
3. Or click **Create** **new** to define a custom intent with a description.
4. Click **Add** to save.
Each intent becomes a branch in your call flow for further configuration.
## Configure information gathering
1. In the **Information** **gathering** section, select the data the agent should collect:
* Caller name
* Contact number
* Email address
* Incident description
* Affected services
2. To add a custom field:
* Click **+ Add custom field**
* Enter a label *(e.g., Customer ID)*
* Choose the field type *(e.g., Text)*
* Optionally, make it required and add a sample question
* Click **Add**\
## Enable knowledge enrichment (optional)
1. Toggle on **Enable** **knowledge** **enrichment**.
2. Choose which data types to include:
* Open incidents
* Maintenance windows (scheduled and ongoing)
* Service status (even if there are no incidents)
3. Select data sources *(e.g., API, UI, AI, Payment).*
This allows the AI to respond with dynamic updates based on real-time incident data.
## Route caller based on intent
1. After adding intents, each intent appears as a separate branch in the call flow.
2. Click the **+** icon under each intent to define follow-up actions, such as:
* Route call
* Create alert
* Play an audio message
* Ask for PIN
* Block numbers
3. Configure a **Fallback** action for unmatched inputs.
4. Click **Save** to apply changes to your call flow.
## Build the complete call flow
Once the AI voice agent is configured, you can create the full call handling flow:
1. **Incoming call node** – Entry point for all calls.
2. **AI voice agent node** – Greets caller, identifies intent, and collects requested information.
3. **Intent branches** – Each intent appears as its own branch. Example:
1. Report critical incident – Route to a live agent or create a high-priority alert.
2. Report system outage – Trigger a recorded status update and log the incident.
3. Fallback – Handle unknown inputs with a standard message or voicemail option.
4. **Actions under each branch** – Click the **+** icon under each intent to define what happens next, such as:
* Route call
* Create alert
* Play audio message
* Ask for PIN
* Block numbers
5. **End call node** – Close the call after actions are complete.
## This is how a Call Flow could look:
***
## FAQ
**Can I add multiple intents to one AI voice agent?**\
Yes, you can add and configure multiple intents per agent, each with its own branch and follow-up logic.
**Is knowledge enrichment required?**\
No, but enabling it helps the AI respond to callers with real-time service data.
**What languages does the AI voice agent support?**\
The AI voice agent supports English and German. Additional language support may be added in future releases.
**Can I test the AI voice agent before going live?**\
Yes, you can preview and simulate your call flow directly within ilert before assigning a real number.
***
# Overview
ilert AI brings intelligence and automation to your incident response process, from identifying related alerts and generating post-mortems to AI SRE agents that perform RCA autonomously
The ilert AI ecosystem combines **assistive AI features** that enhance user workflows with **autonomous agents** that can reason, investigate, and act on incidents using connected data sources through **MCP servers**.
ilert AI is built around two complementary layers:
1. **AI Features:** Assistive capabilities embedded directly into existing ilert workflows (e.g. alert grouping, post-mortem generation, incident message generation).
2. **Agents:** Autonomous, reasoning-driven agents that analyze signals, identify causes, and trigger actions to resolve or communicate incidents.
Together, they enable **AI-first incident management**, from detection to resolution, while keeping engineers fully in control.
### Model endpoints & data residency
ilert AI operates on region-specific model endpoints. Which models serve your organization depends on your region:
| Region | Model Hosting | Provider |
| ----------------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------- |
| **EU customers** | Models hosted **in the EU** | Anthropic models via **AWS Bedrock (EU region)**, xAI models via **xAI (EU)** for the Voice Agent |
| **Non-EU customers (including US)** | Models hosted **in the US** | OpenAI models via the **OpenAI platform**, Anthropic models via the **Anthropic platform** |
* You can view the **active region** used by your organization under\
**Account settings**.
* **AI features that rely on external LLMs can be globally disabled** in your account settings.
* When disabled, AI Features and Agents that depend on LLM reasoning will no longer process or transmit data externally.
### Control & auditability
ilert AI is designed for **auditable and controlled automation**:
* All AI and Agent actions are logged and traceable.
* You have full control to enable or disable AI features globally.
* MCP connections use HTTPS with strict authorization (API keys or OAuth).
For data processing, sub-processors, certifications and other compliance documentation, see the [ilert Trust Center](https://docs.ilert.com/trust-center).
### Learn more
{% content-ref url="/pages/tZyqyr1iIBMkE5ozfhhj" %}
[AI Features](/ai-and-agents/ai-features)
{% endcontent-ref %}
{% content-ref url="/pages/8mQaJXsxvcgaZcLVV4FA" %}
[Agents](/ai-and-agents/agents)
{% endcontent-ref %}
# AI Features
AI Features use contextual understanding, external or ilert-hosted large language models (LLMs) to enhance common workflows in ilert.
{% hint style="info" %}
AI features that use **external LLMs** can be turned off globally in the [account settings](/ai-and-agents/introduction#model-endpoints-and-data-residency).
{% endhint %}
# AI global search
Access all resources in your ilert account easily. Search alerts, incidents, users, and alert sources.
## Searching for resources
ilert global search is a superfast way to access any resource in your ilert account. No matter if master data like alert sources or dynamic data such as alerts or incidents.
Locate it at the top of your navigation bar in any view or press `CTRL/CMD + K` to open the search bar. Enter your desired search term and wait a moment for your results to be gathered.
The result window gives you a quick glance over all resource types in your account that match the search term. ilert uses a hybrid search approach that mixes full-text search and vector similarity with reranking to provide you with the best results to your search, even if your search term is not exactly matching.
## Drilling down deeper
You may drill down into more results for a specific category, this will widen the vectorized search part as well, trying to resolve even more results. Just **click on a category header** of your search results.
This will open the category specific results:
{% hint style="info" %}
Note that by hovering a result you can see the full text of the item
{% endhint %}
## Mouseless usage
You may use the search without interacting with your mouse.
1. Press `CTRL / CMD + K` to open the search bar
2. Enter your search term
3. Press Enter to open the detail view of the first top result
# Bulk resolving similar alerts
Bulk‑resolve similar alerts using ilert AI via global search to streamline cleanup and reduce alert handling time.
If you have not configured grouping for your alert sources you might end up with a view like this:
Let's say you wish to clean / resolve a bulk of some specific alerts that might be duplicates in this list.
Open the search by clicking on its icon at the top bar or press CTRL/CMD + K. Then enter your desired search term aka alert summary you wish to filter for.
**Click** on a matching alert in the search results.
{% hint style="info" %}
If your result hits an already resolved alert, try again, or widen your search in the category to find a PENDING alert. You may also of course just click on such an alert in the list view if you can identify it directly.
{% endhint %}
Click on **Resolve** to open the resolve context menu. Wait a second to let ilert AI fetch similar alerts, they will appear at the bottom of the menu.
Use the **Select all** option to select all similar alerts (if necessary, use the individual checkbox to remove single alerts that might not exactly match your use-case). Click on **Resolve x alerts** to bulk resolve all selected alerts.
Wait a short moment and you should see a confirmation message of your operation.
Looking at the alert list again, we can see that only the specific "blackbox probe failed" alerts have been resolved.
Confirming our intended operation.
{% hint style="success" %}
Note that we recommend configuring alert grouping for your alert sources, especially for monitoring tools that tend to be more noisy or miss grouping operations themselves. You can find a guide here: [../../../alerting/configure-alerting/alert-sources/#event-grouping](/alerting/configure-alerting/alert-sources#event-grouping) and for more complex or custom use cases we highly recommend checking out ilert AI alert grouping: [../../../alerting/configure-alerting/alert-sources/using-ilert-ai-for-alert-grouping.md](/alerting/configure-alerting/alert-sources/using-ilert-ai-for-alert-grouping)
{% endhint %}
# Agents
**Agents** are ilert's autonomous components that execute complex reasoning loops and tool calls to perform end-to-end tasks.
They are designed to **investigate, act, and collaborate** during incidents — securely and transparently.
Handles inbound calls via natural conversation, recognizes intent, gathers context, and routes call by intent.
# User roles and permissions
Define secure roles and permissions in ilert to manage user access, support granular team control, and ensure compliance.
## Available roles
A user in ilert can have one of the following roles:
* [Stakeholder](#stakeholder)
* [Viewer](#viewer)
* [Guest](#guest)
* [Responder](#responder)
* [User](#user)
* [Admin](#admin)
* [Account Owner](#account-owner)
### Stakeholder
Stakeholders will only be able to see the incidents to which they have been added as a subscriber and won't be able to see any other data, such as alerts, alert sources, escalation policies, etc. Additionally, they can see the status pages and services to which they have been granted access. In order to grant a stakeholder user access to resources, the stakeholder needs to be added to a team that contains said resources. This role is only available as part of our [Scale plan](https://www.ilert.com/pricing).
### Viewer
Viewers have account-wide read-only access to ilert. They can see all alerts, incidents, services, configurations (such as on-call schedules, escalation policies, and alert sources), and reports across the entire account — similar to a Responder, but without the ability to take actions on alerts or participate in on-call management. Another way to think of the Viewer role is "Responder without alert/on-call management".
This role is designed for engineering managers, executives, customer support leads, and other internal users who need comprehensive visibility into ilert without being active responders or on-call participants.
Unlike the Stakeholder role, which is team-scoped and designed for MSP use cases, the Viewer role provides full operational visibility across the entire account. Like Stakeholders, Viewers cannot have different team-roles.
Key characteristics of the Viewer role:
* **Read-only access**: Viewers can see all pages that a regular Responder can see, but action buttons are disabled with a tooltip explaining that Responder permissions are required.
* **Dashboard access**: Viewers can access the Dashboard in read-only mode with actions disabled.
* **Reports**: Viewers have full access to reports, just like Responders.
* **Alert chat**: Viewers can add comments in alert chat.
* **Saved filters**: Viewers can manage saved filters.
* **Incident notifications**: Viewers can subscribe to incidents, services, and status pages and manage their incident notification settings.
* **No alert management**: Viewers can see alerts but cannot take any action on them (e.g. acknowledge, resolve).
* **No on-call or escalation**: Viewers cannot be added to on-call schedules or escalation policies and will not receive alert notifications.
* **No call routing**: Viewers cannot be added as targets to route call nodes.
* **Downgrading to Viewer**: Existing users can only be downgraded to the Viewer role if they are not part of any escalation policy, not part of a schedule, and don't have alert notification settings configured.
### Guest
Guest users have access to the application. However, they cannot see any resources or users unless they are added as members of a team, which gives them (depending on their team role) permission to see or even edit the resources of the specific team.
### Responder
Responders can use the web UI and mobile app to manage alerts, just like Users. However, they have no permission to create or modify any objects, such as alert sources, schedules, or escalation policies. Besides taking actions on alerts, Responders can add themselves as overrides to schedules.
### User
Users can create or modify entities like alert sources or on-call schedules; however, they cannot create or modify (or invite) other users (of any role) or change account settings. Users are also not able to create or modify Teams. Although users may edit public resources globally, they may not change the ownership of a resource (team context).
### Team Admin
**Users** may not create or modify teams. But an Admin may grant a User, as a member of a team, the right to modify a certain team. A Team Admin is, therefore actually not a role in itself; it is an additional permission that may be granted to a **User** on a team basis.
### Admin
An Admin is a **User** with elevated privileges. He may not access or modify the account settings. However, he can create and modify **Users** as well as **Teams**, he may also change the role of **Users,** and can create and edit connectors. An **Admin** has the right to add and remove team ownerships to/from resources.
### Account Owner
An account owner has the same privileges as an **Admin**, with the addition of being able to access and modify the account settings as well as subscription and billing settings. Only the account owner himself is able to transfer the Account Owner role to another **User**. There can only be a single account owner per account.
## Role permissions
The table below gives an overview of the role permissions.
| **Operation** | **Stakeholder** | **Viewer** | **Guest** | **Responder** | **User** | **Admin** | **Account Owner** |
| ------------------------------------------------------ | :-------------: | :--------: | :-------: | :-----------: | :------: | :-------: | :---------------: |
| Modify profile settings | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Subscribe to incidents, services and status pages | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
| Manage alerts | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |
| View objects, e.g. schedules and escalation policies\* | ❌ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
| View reports\*\* | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Add him/herself as override to a schedule | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |
| Add (anyone) as overrides to schedules | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ |
| Modify objects, e.g. schedules and escalation policies | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ |
| Manage teams | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ |
| Manage users | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ |
| Add or remove team ownerships to/from resources | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ |
|
Manage account
settings and subscription
| ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
\*Stakeholders have view access to shared (team membership) or subscribed incidents, services and status pages. Viewers have account-wide read-only access to all objects.\
\*\**The actual data that is shown in reports will depend on the permissions of the current user.* *If the user has no access to a schedule, for example, he will not see the on-call-duty data of this schedule when browsing reports. (*[*Read more here*](/users-and-access-management/teams#report-visibility)*)*
## Change a user's role
{% hint style="warning" %}
This requires admin or account owner privileges
{% endhint %}
To change a user's role
1. Click on the cog-icon in the navigation bar and select **Users & teams**
2. Click the **Edit** link for the user you would like to change.
3. In the **Basic information** section, select the user's role from the dropdown menu:
# Team-based organisation
Create teams to manage access to resources and simplify the user interface to show only the alerts and resources relevant to a team.
ilert's team feature allows you to easily manage complex permission scenarios and keep the UI experience for your users and teams simple by only showing the resources owned by the selected team. It enables productivity while managing hundreds of alert sources and users, as well as hundreds or thousands of alerts.
## Team Filter
The team filter lets you switch between different team contexts. It gives you the following options depending on the corresponding teams existing in your account.
* **All teams** essentially removes any filter and shows you all objects that you have permission to view, including those that are not associated with any teams.
* **My teams** shows you all objects of the teams that you are a member of.
* **A specific team:** selecting a specific team will show you all objects that are associated with the selected team.
The team filter is located at the top right in the navigation bar:
In case your account requires a large number of teams, the team filter will automatically collapse and include a search field for quick navigation between different team views:
The team filter is also available in the mobile app:
The team filter selection is stored for each user and synced across devices—meaning users will always continue where they left off, even when changing from desktop to mobile app.
{% hint style="info" %}
**Visibility of the team filter**
The team filter will automatically disappear when the user has no selectable teams available. Please note, that the team filter has no effect on the team management UI, as all resources and users in the permission context of the current user may be assigned to the team.
{% endhint %}
## Creating Teams
Teams can be created by **Admin** users and the account owner. Existing team members may also be managed by users who have been elevated to **Admins** in the team itself (see [Team Roles](#team-roles)).
To create a new team, navigate to the list view using the **Teams** link in the settings navigation menu.
Click on **Add new team**, enter a name for your new team, and use the **Create team** button to create it.
You may now add new members or resources to the team.
You may also directly manage the team ownership of resources in their edit views, for example, alert sources:
In case no teams are available for the current user (with included write permission) the team selector may not appear in the edit views.
## Private Teams
You may choose to create a private team, which allows you to **restrict visibility** of the team and its associated objects (including users) **to the team members**. Note, that global admins and the account owner will be able to see the data of a private team, even if they are not a member of the team.
To make a team private, navigate to the team's settings page and choose **Private** under **Team visibility**.
{% hint style="warning" %}
**What happens to objects that are associated with both a private and a public team?**
The private visibility setting of a team takes precedence over its public visibility. If an object is assigned to two teams, one public and one private, the object will be visible to both teams **only** and won't be visible to members of other teams.
Such objects will be marked with the incognito icon in the list views.
{% endhint %}
## Team Roles
{% hint style="success" %}
When working with teams, we recommend keeping all global users that do not require elevated permissions on the **Responder** role and using team roles instead. For example, instead of assigning a user the **User** role as their base role, use the **Responder** role as their base and assign the user the team role **User.**
{% endhint %}
Team roles extend the permissions of base roles within the context of a team. For example, if you have been assigned the **Responder** role as your base role and have been assigned the **Team User** role within a team, you will be granted the permissions of the User role within that team.
{% hint style="info" %}
Note that you cannot be assigned a less permissive team role than your base role in a public team. For example, if your base role is **User**, you cannot be assigned the **Team Responder** role within a public team. However, this is not the case for private teams. The team roles in private take precedence over base roles. E.g., if your base role is **User**, you can be assigned **Team Responder** permissions in a private team.
{% endhint %}
The following team roles are available:
* Stakeholder
* Team Responder
* Team User
* Team Admin
The permissions of team roles match the permissions of[ global roles](/users-and-access-management/user-roles-and-permissions#role-permissions), except that they are limited to the team's context (meaning resources that this team has ownership of).
{% hint style="warning" %}
Global Stakeholder and Viewer users cannot be assigned team roles. Stakeholders are limited to their team-scoped access, while Viewers already have account-wide read-only access.
{% endhint %}
## Resource ownership adjustments
Besides plain resource read and write permissions which are based on [global roles](/users-and-access-management/user-roles-and-permissions) and may be overwritten in a team context by the equivalent team roles, as described above in [Team roles](#team-roles).\
The permission for the addition and removal of team ownership is validated under the following axiom:
{% hint style="danger" %}
Adding or removing ownership (*describing the assignment of a resource to a team*) is only allowed if the operating user is a member of the team (*referred to in the ownership*) with write level team permissions. This accounts for all global user roles, except for admins and account owners. A resource delete operation is, in this case, equal to a removal of all ownerships.
{% endhint %}
An example to put this axiom into action:
A user (with global **User** role) that is a member of *Team1* (with team role **User**) is not able to delete an alert source that is owned by *Team1* and Team2. Because he is neither an **Admin** nor a write-level team member of *Team2*. He may only remove the ownership of *Team1* of which he is a team member, and leave the resource ownership solemnly to *Team2*.
## Resource Visibility
An administrator with the All teams filter sees all alert sources in the account.
A user of the Mobility team with an active specific team filter only sees the alert sources of his interest.
{% hint style="info" %}
Resources and users of public teams, are shown to all users with read permissions.
{% endhint %}
{% hint style="warning" %}
Unassigned resources (without any owners / teams) are treated as public resources.
{% endhint %}
## User Visibility
User visibility is almost identical to the described behavior in Resource Visibility [above](#resource-visibility) except it has less restrictions. In general, all users are public and can be seen by all roles except **Guests**. However, when a user is added as a member of a private team he becomes a *private user*. From now on, he is only visible to either users with elevated permissions e.g. **Admins** / **Account Owners** or members of the teams he is a member of.
## Alert Visibility
An alert is visible to a user in a team context if
* its alert source is part of the team context or
* its escalation policy is part of the team context or
* the user is a responder of the alert
This way alerts may still be shared across team contexts or re-assigned by higher level permission users to ensure the most flexible workflow for your users in any scenario.
## Data Visibility in Report
The current team context will automatically be reflected in alert, on-call and notification reports. Meaning that the selected team filter will have a direct impact on the shown resources in the report, as well as the user permissions on the data accessible in any shared reports.
{% hint style="info" %}
Further restriction of resource permissions to a user will reflect onto shared report urls even after they are created e.g. when hiding an alert source from a user by placing it in a private team without his access, the shared alert report will also hide the alerts from this alert source automatically.
{% endhint %}
## FAQ
### When using ilert, am I required to configure teams?
No, you don't have to. Teams are meant to be an enhancement for larger organizations and enterprises to keep their users productive. When no teams are created in your account, the team-specific UI elements won't be shown, and you may use and share all resources globally across your account by default.
### Why is there no Guest team role?
The Guest user is meant to be without global read permission per default—meaning only the membership of a team will grant him rights to see resources within the context of these teams. Therefore there is no reason to offer a non-read permissioned role on the team level.
### Why is there no Viewer team role?
The Viewer role provides account-wide read-only access by design. Since Viewers can already see all resources across the entire account, assigning team-specific roles would not add value. Viewers are intended as read-only observers and cannot participate in alert management or on-call workflows, regardless of team membership.
### What happens if I turn a private team to a public with team roles giving lesser permissions?
The roles giving lesser permission e.g. a global User with a Responder team role will be automatically changed so that the team role reflects the global role in our example this will give the global User the User team role (a warning will be shown on the settings page beforehand).
### Is it possible to manage multiple teams that cannot see each other and their resources at all in the same account?
Absolutely. To achieve this, all users should have the global **Responder** role, while being placed in their specific teams with the **User** team role; this way users will always have to assign new resources to their team. Then all teams should be configured to be private, a user or resource should never be in more than one team; this way their contexts are completely separated. Do not forget to place the **Account owner** in its own private team to hide him from the others and to delete the Default escalation policy.
### I am a Team Admin trying to add a user to my team but he does not appear in the users dropdown. What am I doing wrong?
If you are certain that the user you are searching for exists in your ilert organization and he does not appear in the dropdown - there is a good chance that you have no permission to access him. Most likely this is related to him already being a member of a private team, take a look [here](#user-visibility).
# Single sign on
# Setting up SSO with GSuite
Set up SSO in ilert via G Suite to streamline secure login and simplify user access across your organization.
{% hint style="info" %}
**Account owner permissions required**
You need to be the account owner to setup SSO.
{% endhint %}
1\. Click on the cog-icon on the top right and go to **Account settings**
2\. Go to the **Single sign-on** tab.
3\. Chose **Google (OAuth 2.0)** as the authentication method and enter your Google domain.
4\. Optionally set **Allow login with username and password**, if you want to allow users in ilert that are not part of your G Suite organization, and therefore, will need to login via email/username and password.
5\. Optionally set **Provision new users on first SSO login**, if you don't want to manually create users in advance and have them automatically created on their first SSO login.
{% hint style="warning" %}
If you revoke a user's access at your SSO provider, the user won't be able to login to ilert via SSO, but this will not delete the user in ilert. The user needs to be manually deleted in ilert by an admin or by the account owner.
{% endhint %}
## FAQ
### Are multiple G Suite domains supported?
No, only a single domain is supported.
# Setting up SSO with Microsoft Azure Active Directory
The Azure Active Directory (Azure AD) enterprise identity service provides single sign-on and multi-factor authentication. You can configure ilert to use Azure AD as SAML provider for your users.
When starting with Azure AD Apps things can be a bit complicated and overwhelming. In this guide we take you from zero to your own Azure AD SAML App that integrates with ilert's SSO login.
## Creating an AD Application
Login to your Microsoft Azure Dashboard. Open the directory and create a new application.
From the selection choose a **non gallery application**.
Enter a name and create the application.
## Configure your AD App SAML Settings
Configure single sign on for your newly created application.
Choose SAML.
Configure the basic SAML Settings
## Setting up SSO in ilert
Log in to your ilert account as **account owner**, navigate to your **Account Settings** (cog right-side navigation) and click on the **Single sign-on** tab.
{% hint style="info" %}
SSO with SAML requires your account to be on a Pro or Enterprise Plan.
{% endhint %}
Copy your SAML Endpoint URL and Audience Restriction values into the Azure AD SAML App Basic Configuration.
Save and close the basic SAML settings. Scroll a bit down and copy the 3 values from your AD App, you will have to download the Certificate's Base64 representation and copy the value of its file into ilert's SSO settings certificate field.
Transfer the values to ilert's SSO settings.
Save the settings on both windows. SSO is now configured, however to make the login process work properly you will have to do 2 more things.
## Adjusting the SAML claim name in Azure AD
To ensure ilert gets passed the correct email of your users from Azure we have to adjust the SAML claim name.
Set the claim name source to `user.mail`.
Save and close the modal.
You have now properly adjusted the SAML claim name of your app.
## Adding Azure Users to your Azure AD SAML App
Right now both your ilert account and your Azure AD App are properly configured. However you have not yet added any users to your app, which means no one is able to login currently. Let's change that.
Go to your app's settings and click on **Users and Groups**.
Click on Users and select the users that should be able to login to your ilert account. Confirm the assignment afterwards.
Your users should now be able to login to ilert using their Azure AD accounts.
## Additional SSO Configurations
### Auto-provisioning Azure AD Users in ilert
You can auto-provision users on their first SSO login by enabling the checkbox for **Provision new users on first SSO login** in your ilert account's settings. This way user accounts will be automatically setup with the role **User** in ilert. Optionally, you can also pass in the user's role via custom SAML attributes. See below for more information.
Keep in mind that auto-provisioning new users will require your account to have enough seats booked.
### Disable login with username and password
You can optionally disable the login for username and password combinations on your ilert account and enforce users to use SSO by disabling the checkbox for **Allow login with username and password** in your ilert account's settings.
### Passing additional attributes during auto-provisioning
Besides the `NameID` you may pass additional parameters for the user or the team to be automatically setup on the first login, please check out our [auto provisioning section](/users-and-access-management/single-sign-on/auto-provisioning-users-and-teams).
# Setting up SSO with Okta
Okta identity management provides single sign-on and multi-factor authentication. You can configure ilert to use Okta as SAML provider for your users.
When starting with Okta Apps things can be a bit complicated and overwhelming. In this guide we take you from zero to your own Okta SAML App that integrates with ilert's SSO login.
## Creating an SAML Application
Login to your Okta Dashboard. Open the applications page and click on the **Add Application** button.
On the next page click the **Create New App** button
On the new modal view choose **SAML** in the Sign on method and click on the **Create** button
On the next page enter **the application name** (e.g. ilert), choose an application icon if you wish and click on the **Next** button
On the next page you need to fill in the information that you can find in your ilert account settings
## Setting up SSO in ilert
Log in to your ilert account as **account owner**, navigate to your **Account Settings** (cog right-side navigation) and click on the **Single sign-on** tab.
{% hint style="info" %}
SSO with SAML requires your account to be on a Pro or Enterprise Plan, please always feel free to reach out in case you have any questions.
{% endhint %}
Copy your **SAML Endpoint URL** and **Audience Restriction** values into the Okta SAML App settings, then choose **EmailAddress** in the **Name ID format** section. Scroll to bottom and click on the **Next** button.
On the next page choose **I'm an Okta customer adding an internal app** in the **Are you a customer or partner?** section and **This is an internal app that we have created** in the **App type** section, then click on the **Finish** button.
Before continuing with the setup, make sure that the application username update settings are adjusted accordingly. This prevents the loss of account bindings in case of username updates in Okta. *(You can read more about these bindings at the end of this page)*
After saving, click on the **View Setup Instructions** button
Here you can find all the values you need for ilert SSO
Transfer the values to ilert's SSO settings
Save the ilert SSO settings. SSO is now configured, however to make the login process work properly you will have to do one more thing.
## Adding Okta Users to your Okta SAML App
Right now both your ilert account and your Okta App are properly configured. However you have not yet added any users to your app, which means no one is able to login currently. Let's change that.
Head to your app's **Assignments** and click on the **Assign** button and then on the **Assign to People** button (or **Assign to Groups**)
Click on the **Assign** button beside your users that should be able to login to your ilert account. Confirm the assignment afterwards and click on the **Done** button.
Your users should now be able to login to ilert using their Okta accounts.
## Additional SSO Configurations
### Auto-provisioning Okta Users in ilert
You can easily auto-provision users on their first SSO login by enabling the checkbox for **Provision new users on first SSO login** in your ilert account's settings. This way user accounts will be automatically setup with the role **User** in ilert. Keep in mind that this will require your account to have enough seats booked.
### Disable login with username and password
You can optionally disable the login for username and password combinations on your ilert account and enforce users to use SSO by disabling the checkbox for Allow login with username and password in your ilert account's settings.
### Understanding Okta <-> ilert account bindings
Okta connects to your LDAP or OAuth2.0 or other identity provider and gives you quick access to import or export these users and groups for different kinds of applications, such as ilert.
When assigning a user in Okta to your ilert application, Okta creates a binding based on the configured application settings. If setup correctly, as described in this guide, this should be based on the username, which by default is the primary email address of the selected user. (These assignments are individual for each relationship of a user and an application).
If a user changes his primary email address the assignment (User Name) for the application will stay the same.
And even if a user or admin changes the username itself (if this guide has been followed correctly) the user name assignment of the application will not be updated.
In ilert itself the Okta assigned application user name will be read from the SAML 2.0 NameID field and mapped explicitly to an internal user id. Which means that even if your users email addresses or usernames are changed, they will still be able login to their existing ilert account.
In case of a username change, Okta might inform you in some cases that the changed user still exists and has to be deprovisioned manually from the application itself, however you can ignore these warnings.
{% hint style="info" %}
Please note that the email address can be changed without breaking the login or account binding. However the notifications of the ilert user will still be sent to the old email address. The ilert user can safely change the email address for his notifications under profile settings.
{% endhint %}
### Passing additional attributes during auto-provisioning
Besides the `NameID` you may pass additional parameters for the user or the team to be automatically setup on the first login, please check out our [auto provisioning section](/users-and-access-management/single-sign-on/auto-provisioning-users-and-teams).
# Setting up SSO with Auth0
Auth0 is an easy to implement, adaptable authentication and authorization platform. You can configure ilert to use Auth0 as SAML provider for your users.
When starting with Auth0 Apps things can be a bit complicated and overwhelming. In this guide we take you from zero to your own Auth0 SAML App that integrates with ilert's SSO login.
## Creating an SAML Application
1\. Login to your Auth0 Dashboard. Open the applications page and click on the **Create Application** button.
2\. On the modal window name the app e.g. ilert, choose **Regular Web Application** tile and lick on the **Create** button
3\. On the next page click on the **Addons** tab and enable the **SAML2** addon
On the next page you need to fill in the information that you can find in your ilert account settings
## Setting up SSO in ilert
1\. Log in to your ilert account as **account owner**, navigate to your **Account Settings** (cog right-side navigation) and click on the **Single sign-on** tab.
{% hint style="info" %}
SSO with SAML requires your account to be on a Pro or Enterprise Plan, please always feel free to reach out in case you have any questions.
{% endhint %}
2\. Go back to Auth0 and click on the **Settings** tab on the **SAML** modal window. Paste your **SAML Endpoint URL** value into the Auth0 **Application Callback URL** field, then paste the following settings by first replacing the **Audience Restriction** field:
```javascript
{
"audience": "
It works by checking the provided SAML attribute field right before the auto provisioning, if you fill it e.g. with "`role`": when a user logs in through your IdP (for the first time), ilert will check if the SAML attribute "`role`" is present in the SAML response, if it is not, the user is redirected to an error page displaying the information that he/she should reach out to an account admin, otherwise the user is auto-provisioned and logged in.
{% hint style="info" %}
Note that role here is just an example, you may use any kind of SAML response attribute that you prefer to set. The value does not matter as well.
{% endhint %}
As an admin or account manager, this gives you an additional option to control the auto provisioning flow and you can make sure users that want/need to be onboarded are properly setup with their requirements before e.g. making sure a certain user is auto-provisioned with the correct role and team before his first login.
# Two-factor authentication / MFA
Two improve your account's security, ilert suggests that you set up two-factor authentication for it. We especially encourage Account owners or Admins to use MFA.
ilert supports two-factor authentication setups for every user account, unrelated to the purchased subscription plan. You may activate multiple 2FA methods e.g. authenticator app as backup for your U2F device.
## Two-factor methods
### Code generation apps
Most used / common authenticator apps:
| App name | Apple Appstore | Google Playstore |
| ----------------------- | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| Google Authenticator | [Link](https://apps.apple.com/de/app/google-authenticator/id388497605) | [Link](https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2\&hl=de\&gl=US) |
| Duo Mobile | [Link](https://apps.apple.com/de/app/duo-mobile/id422663827) | [Link](https://play.google.com/store/apps/details?id=com.duosecurity.duomobile\&hl=de\&gl=US) |
| Authy | [Link](https://apps.apple.com/de/app/twilio-authy/id494168017) | [Link](https://play.google.com/store/apps/details?id=com.authy.authy\&hl=de\&gl=US) |
| Microsoft Authenticator | [Link](https://apps.apple.com/de/app/microsoft-authenticator/id983156458) | [Link](https://play.google.com/store/apps/details?id=com.azure.authenticator\&hl=de\&gl=US) |
### Face ID, Touch ID, Apple Watch or other device authentication
Even if you are not in possession of a hardware token like a Yubico Security Key, you may choose the option "Use security key" with supporting Apple or Microsoft devices that offer biometrical authentication sensors e.g. Touch ID or double-tap on Apple Watch. Do not forget that you have to register and validate again, meaning **you will be asked to verify twice during the setup** of this 2FA option.
### Hardware tokens / FIDO (U2F) Keys
If available you may select "**Use security key**" option with your U2F / [WebAuthn](https://www.w3.org/TR/webauthn-2/) supporting devices such as Yubico Security Keys. Do not forget that you have to register and validate again, meaning **you will be asked to verify twice during setup** of this 2FA option.
## FAQ
### I lost my authenticator app / TOTP generator / device how can I access my account?
If you have lost your app or device you may use your recovery-codes, handed out during initial setup of your 2FA to get access to the account. **Note**: you were handed out 5 recovery codes only and each becomes burnt when it is used. So it is highly advised to immediately use the recovery codes to access and remove 2FA from your account (requires at least 2 recovery codes). Once removed you may set up 2FA again using a new device / app.
In case you were using a FIDO device or you have lost / burnt your recovery codes you will need to contact [ilert support](broken://pages/-MA0haGzw_vocetU3uxE#support) providing additional details of verification e.g. passport to have 2FA reset from your account.
# Overview
Discover ilert reporting: monitor alerts, notifications, and on‑call assignments to analyze performance and improve SLAs.
Reports display historical data with charts and tables, giving insights into how your incident response team is performing. Fine-tune reports with advanced filters and download report data in CSV format for further analysis. There are three types of reports:
1. [Alerts](/reports/alerts)
2. [On-call duties](/reports/on-call-duties)
3. [Notifications](/reports/notifications)
## Alerts
View volume of alerts, Mean Time to Acknowledge (MTTA), and Mean Time to Resolution (MTTR). Group by alert sources or escalation policies.
## On-call duties
View time on-call, number of alerts, and time spent on alerts. View shifts by day.
## Notifications
View notification volume categorised by notification channel.
# Alerts
Generate alert reports in ilert to visualise alert volume, trends over time, and team response effectiveness.
View volume of alerts, Mean Time to Acknowledge (MTTA), and Mean Time to Resolve (MTTR). Group by alert sources or escalation policies.
## Page layout
Page layout of alert reports
1. Combined metrics
2. Metric filter
3. Chart
4. Detailed info of a data point
5. Metric data table
6. Report Parameters
7. Filters
### Combined metrics
The widgets display the three metrics for the selected time range, taking into consideration the applied filters.
### Metric filter
Select which metrics to be displayed in the chart and the table. When the **Group by** filter is **None**, multiple metrics can be selected.
### Chart
The chart displays the report data in a line chart. When multiple metrics are selected, the chart may display another scale on the Y-axis on the right for the metrics MTTA and MTTR.
{% hint style="info" %}
*The legend colours for alert sources and escalation policies are generated based on the hash consisting of type and identifier. Therefore, the colour stays the same even when the resource name is changed.*
{% endhint %}
### Detailed info of a data point
The chart tooltip appears when hovering over the chart. The tooltip displays the names and values of a specific data point. It is vertically scrollable when the list is longer. Click anywhere on the chart to see the alerts on that point.
{% hint style="info" %}
When **Group by** is **Alert Source** or **Escalation Policy**, the entries in the tooltip are sorted by descending value.
{% endhint %}
### Metric data table
View metrics grouped by day, week or month and optionally group by alert source or escalation policy by using the **Group by** filter. Click the CSV button to download the table in a CSV file.
{% hint style="warning" %}
**Unselected metrics are hidden in the table.**
*When the **Group by** filter is other than **None**, the table will show only the selected metric column. However, the downloaded CSV will include all three metrics.*
{% endhint %}
### Report parameters
#### Date range
The **Date range** parameter defines the period during which alerts are created. Select a date in the calendar view and then select either a different day or the same day. Or fill in the **From** and **Until** fields.
Date-range picker demo
There are four predefined date ranges: the last 12 months, the last 6 months, the last 3 months, and the last month. These date ranges are calculated relative to the current date.
**Example date ranges**
| Today | Preset | Range |
| ----------- | ------------- | ------------------ |
| 8 Jun 2025 | Last 3 months | 8 Mar–8 Jun 2025 |
| 1 Mar 2025 | Last 1 month | 1 Feb–1 Mar 2025 |
| 31 Mar 2025 | Last 1 month | 28 Feb–31 Mar 2025 |
{% hint style="info" %}
*The **From** date is translated to the beginning of that day, and the **Until** date is translated to the beginning of the following day. This ensures that all the alerts on the **From** date and the **Until** date, plus those between them, are included in the report.*
{% endhint %}
#### **Time granularity**
This parameter specifies the granularity at which the data is aggregated. Each data point in the charts and each row in tables will display a total or average value in the selected granularity.
**Week numbering**
With the **Week** granularity, the data is grouped by the ClickHouse [`toWeek`](https://clickhouse.com/docs/reference/functions/regular-functions/date-time-functions#toWeek) function. Its mode decides on which day a week starts and which week counts as week 1 of a year. Currently, ilert supports four of these modes:
| Mode | First day of week | Week 1 is the first week… |
| ---- | ----------------- | ----------------------------- |
| 3 | Monday | with 4 or more days this year |
| 6 | Sunday | with 4 or more days this year |
| 8 | Sunday | contains January 1 |
| 9 | Monday | contains January 1 |
*Source: ClickHouse* [*`toWeek`*](https://clickhouse.com/docs/reference/functions/regular-functions/date-time-functions#toWeek) *documentation.*
By default, the mode is derived from your tenant's locale. For example, **de‑DE** results in mode 3 (ISO 8601) and **en‑US** in mode 8. If the week numbering of a locale is unsupported, ISO 8601 is used.
The badge above the **Week** option shows which week numbering is in effect. Click the badge to select a different mode and click **Apply**, or click **Reset** to return to the default. The selection applies to all reports.
Tenant's week numbering supported
Custom week numbering selected
#### **Group by**
This parameter can be used to group the alerts by alert source or escalation policy.
### Filters
The filters can be used to fine-tune which data is included in the report. When the **Apply** button is clicked, all parameters and filters are saved in the page URL, making it easier to bookmark your filters.
#### Teams
The **Teams** filter is available for users with access to [Teams feature](/users-and-access-management/teams). This filter affects both the report data and the subsequent filters. When some teams are selected with **Includes**, the subsequent filters will show the resources and responders that belong to any of the selected teams. If **Excludes** is used, the subsequent filters will hide any resources or responders that belong to any of the selected teams. The same logic applies to the report data.
{% hint style="warning" %}
*Team selection is limited to a maximum of 10 teams.*
{% endhint %}
#### Alert sources
The **Alert sources** filter can be used to show or hide the alerts associated with the selected alert sources.
#### Escalation policies
The **Escalation policies** filter can be used to show or hide the alerts associated with the selected escalation policies.
#### Responders
The **Responders** filter can be used to show or hide the alerts that were accepted and/or resolved by the selected responders.
{% hint style="warning" %}
*When all three filters (**Alert sources**, **Escalation policies,** and **Responders**) are in **Includes** state, the report data includes any alerts that are either related to any of the selected alert sources and escalation policies or accepted and/or resolved by the selected responders. In other words, an alert is included if it is matched by at least one of the filters. **In the previous version of reports, an alert was included only if it matched all filters.***
{% endhint %}
{% hint style="info" %}
*When one or more filters are set to **Excludes**, these filters take precedence over the filters with **Includes**. In other words, all alerts matched by the excluded filters are not included.*
{% endhint %}
#### Alert priority
This filter can be used to select alerts based on their priority: High (with escalation) and Low (no escalation).
### Details
View the alerts, including their ID, summary, alert source, escalation policy, reported date, TTA, and TTR.
Page layout of alert details
1. Filter toolbar
2. Table of alerts matching the filters
3. Download CSV report
#### Filter toolbar
The **Search** field allows you to filter the rows. It searches text in every column and matches by words.
Use the **Date** filter to switch days, weeks, or months based on the selected **Time granularity**.
The last filter appears if **Group by** is not **None**. Use this filter to view details of a specific alert source or escalation policy.
#### Table
The table displays the alerts matching the filters in the toolbar. TTA and TTR columns may contain a hyphen `-` when nobody accepts an alert, or the alert is not resolved yet.
{% hint style="info" %}
The date format of **Reported on** column changes based on selected **Time granularity**. For example, when **Day** is selected, only time is shown. If **Week** or **Month** is selected, both date and time are shown.
{% endhint %}
#### Download CSV button
Click the button to download the table in CSV format.
# On-call duties
Review on‑call duties in ilert with reports that show coverage schedules, duty hours, and team workload patterns.
View time on-call, time spent on alert, and number of alerts. Narrow the metrics to only a part of the day by specifying start and end times of the day.
## Page layout
Page layout of on-call duty reports
1. Metric filter
2. Chart
3. Detailed info of a data point
4. On-call summary table
5. Report Parameters
6. Filters
### Metric filter
Select which metric to be displayed in the chart. The table always displays all metrics.
### Chart
The chart displays the report data in a bar chart.
{% hint style="info" %}
*The legend colours for users are generated based on the hash derived from user identifiers. Therefore, the colour stays the same even when the user data is changed.*
{% endhint %}
### Detailed info of a data point
The chart tooltip appears when hovering over the chart. The tooltip displays the names and values of a specific data point. It is vertically scrollable when the list is longer.
### On-call summary table
The table displays the summary of the metrics for each user. Switch to the tab **Shifts by day** to see the metrics for each day which can be revealed by expanding the user row. Click the CSV button to download the table in a CSV file.
On-call table with rows for each day
### Report parameters
#### Date range
The **Date range** parameter defines the period during which alerts are created. Select a date in the calendar view and select another day or the same day. Or fill in the **From** and **Until** fields.
Date-range picker demo
There are four predefined date ranges: the last 12 months, the last 6 months, the last 3 months, and the last month. These date ranges are calculated relative to the current date.
**Example date ranges**
| Today | Preset | Range |
| ----------- | ------------- | ------------------ |
| 8 Jun 2025 | Last 3 months | 8 Mar–8 Jun 2025 |
| 1 Mar 2025 | Last 1 month | 1 Feb–1 Mar 2025 |
| 31 Mar 2025 | Last 1 month | 28 Feb–31 Mar 2025 |
{% hint style="info" %}
*The **From** date is translated to the beginning of that day, and the **Until** date is translated to the beginning of the following day. This ensures that all the alerts on the **From** date and the **Until** date, plus those between them, are included in the report.*
{% endhint %}
#### **Time granularity**
This parameter specifies the granularity at which the data is aggregated. Each data point in the charts and each row in tables will display a total or average value in the selected granularity.
**Week numbering**
With the **Week** granularity, the data is grouped by the ClickHouse [`toWeek`](https://clickhouse.com/docs/reference/functions/regular-functions/date-time-functions#toWeek) function. Its mode decides on which day a week starts and which week counts as week 1 of a year. Currently, ilert supports four of these modes:
| Mode | First day of week | Week 1 is the first week… |
| ---- | ----------------- | ----------------------------- |
| 3 | Monday | with 4 or more days this year |
| 6 | Sunday | with 4 or more days this year |
| 8 | Sunday | contains January 1 |
| 9 | Monday | contains January 1 |
*Source: ClickHouse* [*`toWeek`*](https://clickhouse.com/docs/reference/functions/regular-functions/date-time-functions#toWeek) *documentation.*
By default, the mode is derived from your tenant's locale. For example, **de‑DE** results in mode 3 (ISO 8601) and **en‑US** in mode 8. If the week numbering of a locale is unsupported, ISO 8601 is used.
The badge above the **Week** option shows which week numbering is in effect. Click the badge to select a different mode and click **Apply**, or click **Reset** to return to the default. The selection applies to all reports.
Tenant's week numbering supported
Custom week numbering selected
### Filters
The filters can be used to fine-tune which data is included in the report. When the **Apply** button is clicked, all parameters and filters are saved in the page URL, making it easier to bookmark your filters.
#### Teams
The team filter is available for users with access to [Teams feature](/users-and-access-management/teams). This filter affects both the report data and the subsequent filters. When some teams are selected with **Includes**, the subsequent filters will show the resources and the users that belong to any of the selected teams. If **Excludes** is used, the subsequent filters will hide any resources or users that belong to any of the selected teams. The same logic applies to the report data.
{% hint style="warning" %}
*Team selection is limited to a maximum of 10 teams.*
{% endhint %}
#### Schedules
The **Schedules** filter can be used to show or hide the alerts associated with the selected schedules.
#### Users
The **Users** filter can be used to show or hide the metrics associated with particular users.
{% hint style="warning" %}
*When the two filters (**Schedules** and **Users**) are in the **Includes** state, the report data includes any metrics that are related to the selected schedules or users. In other words, an alert is included if it is matched by at least one of the filters. **In the previous version of reports, metrics were included only if they matched both filters.***
{% endhint %}
{% hint style="info" %}
*When one or more filters are set to Excludes, these filters take precedence over the filters with the **Includes**. In other words, all alerts matched by the excluded filters are not included.*
{% endhint %}
### Details
View the alerts, including their ID, summary, alert source, escalation policy, reported date, TTA, and TTR.
Page layout of alert details
1. Filter toolbar
2. Table of alerts matching the filters
3. Download CSV button
#### Filter toolbar
The **Search** field allows you to filter the rows. It searches text in every column and matches by words.
Use the **User** filter to see alerts responded by a certain user. Only users who responded in the selected **Date range** can be selectable.
The last filter appears when opening the **Alert details** view from the table under the **Shifts by day** tab. Use this filter to choose from the days on which the selected user responded on some alerts.
#### Table of alerts matching the filters
The table displays the alerts matching the filters in the toolbar.
#### Download CSV button
Click the button to download the table in CSV format.
# Notifications
Explore notification report analytics in ilert to evaluate delivery success, engagement rates, and channels.
View the number of notifications delivered and optionally group the data by user.
## Page layout
Page layout of notification reports
1. Chart
2. Detailed info of a data point
3. Notification data table
4. Report Parameters
5. Filters
### Chart
The chart displays the volumes of notifications sent categorised by notification channel in a bar chart.
When grouping by **User**, the legend is the combination of the user name and the channel. Only the user-channel pairs to which some notifications were sent during the selected date range are shown.
{% hint style="info" %}
*The legend colours for users are generated based on the hash derived from user identifiers. Therefore, the colour stays the same even when the user data is changed.*
{% endhint %}
Description of legend colours when grouping by User
### Detailed info of a data point
The chart tooltip appears when hovering over the chart. The tooltip displays the names and values of a specific data point. It is vertically scrollable when the list is longer.
### Notification data table
View notification volumes grouped by day, week or month and optionally group by user by using the **Group by** filter. Click the CSV button to download the table in a CSV file.
### Report parameters
#### Date range
The **Date range** parameter defines the period during which alerts are created. Select a date in the calendar view and then select either a different day or the same day. Or fill in the **From** and **Until** fields.
Date-range picker demo
There are four predefined date ranges: the last 12 months, the last 6 months, the last 3 months, and the last month. These date ranges are calculated relative to the current date.
**Example date ranges**
| Today | Preset | Range |
| ----------- | ------------- | ------------------ |
| 8 Jun 2025 | Last 3 months | 8 Mar–8 Jun 2025 |
| 1 Mar 2025 | Last 1 month | 1 Feb–1 Mar 2025 |
| 31 Mar 2025 | Last 1 month | 28 Feb–31 Mar 2025 |
{% hint style="info" %}
*The **From** date is translated to the beginning of that day, and the **Until** date is translated to the beginning of the following day. This ensures that all the alerts on the From date and the Until date, plus those between them, are included in the report.*
{% endhint %}
#### **Time granularity**
This parameter specifies the granularity at which the data is aggregated. Each data point in the charts and each row in tables will display a total or average value in the selected granularity.
**Week numbering**
With the **Week** granularity, the data is grouped by the ClickHouse [`toWeek`](https://clickhouse.com/docs/reference/functions/regular-functions/date-time-functions#toWeek) function. Its mode decides on which day a week starts and which week counts as week 1 of a year. Currently, ilert supports four of these modes:
| Mode | First day of week | Week 1 is the first week… |
| ---- | ----------------- | ----------------------------- |
| 3 | Monday | with 4 or more days this year |
| 6 | Sunday | with 4 or more days this year |
| 8 | Sunday | contains January 1 |
| 9 | Monday | contains January 1 |
*Source: ClickHouse* [*`toWeek`*](https://clickhouse.com/docs/reference/functions/regular-functions/date-time-functions#toWeek) *documentation.*
By default, the mode is derived from your tenant's locale. For example, **de‑DE** results in mode 3 (ISO 8601) and **en‑US** in mode 8. If the week numbering of a locale is unsupported, ISO 8601 is used.
The badge above the **Week** option shows which week numbering is in effect. Click the badge to select a different mode and click **Apply**, or click **Reset** to return to the default. The selection applies to all reports.
Tenant's week numbering supported
Custom week numbering selected
#### **Group by**
This parameter can be used to group notifications by user.
### Filters
The filters can be used to fine-tune which data is included in the report. When the **Apply** button is clicked, all parameters and filters are saved in the page URL, making it easier to bookmark your filters.
#### Teams
The **Teams** filter is available for users with access to [Teams feature](/users-and-access-management/teams). This filter affects both the report data and the **Users** filter. When some teams are selected with **Includes**, the **User** filter will show the users that belong to any of the selected teams. If **Excludes** is used, the subsequent filters will hide any users that belong to any of the selected teams. The same logic applies to the report data.
{% hint style="warning" %}
*Team selection is limited to a maximum of 10 teams.*
{% endhint %}
#### Users
The **Users** filter can be used to show or hide notifications that were sent to the selected users.
{% hint style="info" %}
*When one or more filters are set to **Excludes**, these filters take precedence over the filters with **Includes**. In other words, all notifications matched by the excluded filters are not included.*
{% endhint %}
# Getting started with ilert mobile app
Learn more about ilert app for iOS and Android
With the ilert mobile app, you can take full control of your incident response wherever you are: view and act on alerts (accept, resolve, escalate), manage incidents, check on-call schedules and escalation policies, review call flow logs, and oversee maintenance windows, heartbeat checks, service uptime, and status pages—all from a single, intuitive interface. The mobile app empowers you to take the first critical steps toward incident response and remediation on the go, ensuring you're always ready to act.
## Download the app for iOS
Download the **ilert** app from the App Store.
1. Open the [**App Store**](https://apps.apple.com/app/ilert/id542915864?l=en) on your device. You can find the app for iPhone and iPad.
2. Search for **ilert** and tap **Get**.
3. Wait for installation to finish.
## Download the app for Android
Install the **ilert** app from the Google Play Store.
1. Open the [**Play Store**](https://play.google.com/store/apps/details?id=de.ilert.client.iphone\&hl=en_US) on your device.
2. Search for **ilert** and tap **Install**.
3. Launch the app once the download completes.
## Login
Sign in to your ilert account:
* **Email & Password:** Enter your credentials and tap **Sign In**.
* **SSO:** Tap **Sign In with SSO** (if your organization has enabled single sign-on).
If you don't yet have an ilert account, you will need to [create it first via the desktop application](https://app.ilert.com/signup).
## Enable push notifications
You can turn on push notifications in two ways:
1. **Notification screens**\
Enable immediately after your first login using the in-app onboarding screens.
2. **Manual settings**\
If you skipped the cards, enable notifications via the app's Settings menu later.
## Notification s**creens**
After your first login, you'll see these screens. They aim to help you set up the push notifications. Swipe **left** or **right** to navigate between them, or tap **Not Now** to skip all.
### 1. Turn on motifications
Enable push notifications so you never miss an alert from ilert.
### 2. Enable Critical Alerts
Allow **high-priority** alerts to bypass Do Not Disturb and notify you immediately. [See here for more information](/mobile-app/mobile-app-notification-settings/critical-push-notifications-and-dnd-overrides).
### 3. Add ilert to your contacts
Install a **vCard** containing all of ilert's phone numbers—so calls from ilert are clearly identified on your device.
## Manual settings
If you skipped the onboarding screens, you can still enable push notifications at any time:
1. Click on the menu button in the top right corner.
2. Select the **Settings** (gear icon) in the ilert app.
3. Tap **Push notification settings**.
4. Toggle **Push notifications** on.
5. Tap **Go to settings**.
6. In your device's Settings, tap **Allow notifications** for ilert.
7. Return to the ilert app—push notifications are now enabled.
# Mobile app notification settings
Customize your notification preferences in the ilert mobile app for alerts, reminders, and on‑call updates.
## Accessing contact information & notification rules
This section walks you through managing your notification contacts (email, phone, push devices) and setting up notification rules in the ilert **mobile app**. If you're looking specifically to configure **critical alert behavior** (e.g., bypassing Do Not Disturb), please see our [**Critical Alerts**](/mobile-app/mobile-app-notification-settings/critical-push-notifications-and-dnd-overrides) article.
1. Open the navigation by tapping on the menu icon on the top left.
2. Select the **Gear Icon** → **Contact information | Notification rules**
## **Contact information**
ilert enables you to add and verify multiple contacts, then activate channels to deliver alerts.
### Email addresses
* Tap **Add email** and enter your address.
* Tap **Save**.
* Click the verification link sent to your inbox.
* Tap **Send test notification** to confirm delivery.
### Phone numbers
* Tap **Add phone number**.
* Select your country and enter your number.
* Tap **Save**.
* Enter the verification code sent to you.
> **Note:** Verifying via SMS also enables phone-call notifications.
### Push notification devices
* [Set up Push Notifications](/mobile-app/getting-started-with-ilert-mobile-app#enable-push-notifications)
* [Set up Critical Push Notifications](/mobile-app/mobile-app-notification-settings/critical-push-notifications-and-dnd-overrides)
* **View devices:** Listed under **Push devices**
* **Delete:** Swipe left on any device entry
## Supported Notification Channels
* **Email**
* **SMS**
* **Phone Call**
* **iOS & Android Push**
* **WhatsApp**
* **Telegram**
> You must activate each channel (via a verification code) before you can use it in any rule.
## Notification rules
Define when and how you want to be notified for each event type.
### Rule types
| Rule Type | Description |
| -------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| **High-priority alerts** | Notifies you when a high-priority alert is assigned (on creation or when added as a responder). |
| **Low-priority alerts** | Notifies you when a low-priority alert is assigned (on creation or when added as a responder). |
| **Alert status updates** | Notifies you when you're already assigned to an alert and its status changes (e.g., another user accepts the alert). |
| **Incident notifications** | Notifies you when you're added to an incident as a subscriber (on creation or when added manually). |
| **On-call notifications** | Notifies you at a specified time before your on-call shift begins (e.g., 15 min prior). |
### Creating a notification rule (Example: High-Priority Alerts)
Follow these steps to set up a rule that sends you an **email** **3 minutes** after a high-priority alert is created:
1. **Tap + Add notification rule**
2. **Configure the rule**
* **Contact & Channel:** Email
* **Timing:** Delay by 3 minutes
3. Save the Rule
Alert notifications in ilert are bi-directional; you can respond to a notification using the same channel on which you were notified (without logging into ilert), e.g., by replying to an SMS.
You have the following response options:
1. Acknowledge the alert
2. Mark the alert as resolved
3. Escalation to the next user in the alert's escalation policy
# Critical push notifications and DND overrides
This section contains information on critical push notifications or DND (Do Not Disturb or Sleep modes) overrides for mobile devices
In this article, we will guide you through the process of overriding Do Not Disturb and Sleep modes on mobile devices to ensure that critical notifications are always delivered. For applications handling urgent alerts—such as incident management—bypassing these settings is essential to guarantee timely responses. We will cover platform-specific approaches, including how to configure settings on both iOS and Android devices, ensuring that vital notifications can break through device silences when it matters most.
{% hint style="info" %}
**Download ilert app**
[For iOS](https://apps.apple.com/us/app/ilert/id542915864?l=en\&platform=iphone)
[For Android](https://play.google.com/store/apps/details?id=de.ilert.client.iphone\&hl=en_US\&pli=1)
{% endhint %}
## iOS (iPhone, iPad)
{% content-ref url="/pages/6QfUbIEzGDqqcsEKYlrv" %}
[iOS critical alerts configuration](/mobile-app/mobile-app-notification-settings/critical-push-notifications-and-dnd-overrides/ios-critical-alerts-configuration)
{% endcontent-ref %}
## Android
{% content-ref url="/pages/lNo00BkMJq0qzHVSjlKb" %}
[Android Push Notification DND Configuration](/mobile-app/mobile-app-notification-settings/critical-push-notifications-and-dnd-overrides/android-push-notification-dnd-configuration)
{% endcontent-ref %}
# iOS critical alerts configuration
This guide explains how to configure your push notification preferences for iOS devices.
{% hint style="info" %}
Note: this guide is meant for iOS users only. For Android, please refer to [this page](/mobile-app/mobile-app-notification-settings/critical-push-notifications-and-dnd-overrides/android-push-notification-dnd-configuration).
{% endhint %}
{% hint style="success" %}
[Download ilert app for iOS](https://apps.apple.com/us/app/ilert/id542915864?l=en\&platform=iphone).
{% endhint %}
On iOS devices, critical alerts in ilert ensure that you are immediately notified of high-priority issues, even if your phone is in Do Not Disturb (DND) mode or muted. Unlike standard push notifications, critical alerts bypass these restrictions, playing a distinct sound and appearing instantly on your screen. This capability is essential for on-call engineers who must be reachable at all times for urgent incidents, allowing ilert to deliver time-sensitive information without delay and helping teams maintain rapid incident response.
## Setting up critical alerts on iPhones and iPads
* Open up the ilert mobile app
* Open the main navigation (top left icon)
* Tap on **settings** (cog icon, top right)
* Tap on **High priority notifications** -> make sure **iPhone Push Notification** is selected in the list *and that no delay has been configured*
* Navigate **back to the Settings** menu and tap on **Push notification settings** -> **Enable Critical alerts for high priority**
* Choose an override volume for your critical alerts
{% hint style="warning" %}
Note that only high priority alerts will be delivered as critical alerts. Alerts with low priority will use low priority notification settings.
{% endhint %}
## Are you an Apple Watch User?
{% hint style="danger" %}
When wearing your Apple Watch, critical notifications will not appear on your phone, but on your watch and they will not use the custom sound as well as override volume settings. To prevent missing alerts you will have to make additional adjustments to your watch's settings.
{% endhint %}
* Open your **Watch** app on your iPhone
* Navigate to **Notifications**
* Scroll down in the list to find the **ilert** app and disable the watch notifications for the app.
This will enforce critical alerts to be sent to your iPhone even if you are wearing your Apple Watch.
# Android Push Notification DND Configuration
This guide explains how to configure your push notification preferences for Android devices.
{% hint style="info" %}
Note: this guide is meant for Android users only. For iOS, please refer to [this page](/mobile-app/mobile-app-notification-settings/critical-push-notifications-and-dnd-overrides/ios-critical-alerts-configuration).
{% endhint %}
{% hint style="success" %}
[Download ilert app for Android](https://play.google.com/store/apps/details?id=de.ilert.client.iphone\&hl=en_US\&pli=1).
{% endhint %}
Critical alerts are how ilert reaches you for a high priority alert when your phone is silenced. Once they are switched on, ilert asks Android to play the alert as an **alarm** rather than as a normal notification, so it can still be heard on a muted phone.
This page has five steps: set up the app, make sure the alert can be heard, stop Android from delaying alerts, set up calls and SMS, then check that it works.
{% hint style="warning" %}
Only **high priority** alerts are delivered as critical alerts, and only while **Critical alerts for high priority** is enabled in the app. Low priority alerts always use your low priority notification settings.
{% endhint %}
## Before you start: check your Android version
Android 17 stopped letting apps turn the volume up in the background, so ilert now plays critical alerts as an alarm instead. **Step 2 of this guide is different depending on your version**, so check yours first under **Settings -> About phone -> Android version**.
Your version
What this means
Android 17 and newer
Alerts play at your alarm volume. There is no volume slider in the app, and ilert never changes your phone's settings.
Android 16 and older
ilert turns your notification volume up itself, and you set how far in the app.
Everything else on this page is the same either way.
## Prerequisite
Install the ilert mobile app and sign in. This registers your device with your ilert account so it can receive push notifications.
Then open your **profile** settings, in either the web app or the mobile app, go to **Notification settings**, and use **Push** for the notification options you want. There is no separate Android notification method to pick — a push goes to every device you are signed in to the ilert app on.
## Step 1: Set up the app
* Open the **Navigation** in your app (top left icon)
* Tap the cog icon (top right in navigation) to open the **Settings** menu
* Tap on **Push notification settings**
* Make sure **Push notifications** at the top of the screen is on. While it is off, everything below it is greyed out and cannot be changed
* Pick a sound for **High** and **Low priority** notifications if you want one. Tap the row to select it — the selected sound is highlighted and shows a check mark on the right. The play icon only previews the sound and does not change your setting
* Turn on **Critical alerts for high priority**
Push notification settings on Android 17 and newer
The selected sound is check marked. The play icon only previews it
## Step 2: Make sure the alert can be heard
These settings live in Android, not in ilert. Without them, Android can silence an alert no matter how ilert is set up.
Two separate things decide whether you hear an alert. **Volume** decides how loud it is. **Do Not Disturb** decides whether it is allowed to make a sound at all. You need both.
**Allow alarms in Do Not Disturb.** This is usually on already — check under **Alarms & other interruptions**. If you use modes such as Sleep or Work, check each one, because every mode has its own rules. You can also add **ilert** to the apps allowed to interrupt in your Do Not Disturb settings.
Alarms must stay allowed in every mode you use
The volume to turn up, and whether ilert needs a permission from you, depend on your Android version. Pick your version below.
{% tabs %}
{% tab title="Android 17 and newer" %}
**Turn your alarm volume up.** Critical alerts play as an alarm, so your alarm volume is what decides how loud they are.
**You normally do not need to grant anything.** Do Not Disturb lets alarms through by default, so there is no permission prompt when you turn critical alerts on. Only if you have switched alarms off in your Do Not Disturb settings do you need the override. Grant it from the **Allow Do Not Disturb override** link on the **Push notification settings** screen, or from the **Never miss a critical alert** step when you first set up the app.
The Allow Do Not Disturb override link appears only while the override is missing
You can also grant it yourself at any time in **Android Settings -> Apps -> Special app access -> Modes access**, then pick **ilert** and turn on **Allow Modes access**.
Modes access lists every app, so find ilert in it
Turn on Allow Modes access for ilert
{% hint style="info" %}
The **Push notification settings** screen also checks your device and warns you when something would keep a critical alert silent — notifications turned off for ilert, the critical alerts notification category turned off or silenced, or your alarm volume all the way down. Each warning has a **Fix** link that takes you straight to the right setting. The check only runs while **Critical alerts for high priority** is on.
{% endhint %}
{% endtab %}
{% tab title="Android 16 and older" %}
**Turn your notification volume up.** ilert turns it up for you when an alert arrives, but only as far as you allow: set **Critical alerts volume** in **Push notification settings** to how loud you want alerts to be.
**Grant the Do Not Disturb override when asked.** Turning on **Critical alerts for high priority** prompts you for it. Tap **Go to settings**, search for **ilert**, and grant the permission.
{% hint style="warning" %}
On these versions, the first critical alert that arrives switches **Do Not Disturb off** and sets your sound mode to **Sound**. Your previous setting is not restored afterwards.
{% endhint %}
{% endtab %}
{% endtabs %}
{% hint style="info" %}
**One permission, three names.** Android 16 and newer call it **Modes access**, older versions call it **Do Not Disturb access**, and ilert calls it a **Do Not Disturb override**. They are the same permission.
{% endhint %}
## Step 3: Stop Android from delaying alerts
These two settings do not silence an alert, but they can make it arrive late.
**Turn off "Pause app activity if unused".** Android pauses apps you do not open often, which is easy to trigger with an on-call app. Open **Android Settings -> Apps -> ilert** and switch it off.
**Set battery usage to Unrestricted.** Android uses "Optimized" by default, which can delay alerts. Open **Android Settings -> Apps -> ilert -> App battery usage** and pick **Unrestricted**.
## Step 4: Set up calls and SMS
If your escalation policy also reaches you by phone call or SMS, whether those ring is decided by your phone's call settings rather than by ilert.
* Save the ilert contact card to your contacts. The app can do this for you: open **ilert Settings** and tap the **ilert vCard sync** tile
* Tap the star icon to mark the contact as a favorite
* In your Do Not Disturb settings, allow calls and messages from **favorite** or **starred** contacts
{% hint style="warning" %}
Allowing starred contacts through Do Not Disturb does not make calls ring while your phone is on **silent**. Those are two separate Android settings.
{% endhint %}
## Step 5: Check that it works
Send yourself a test notification to confirm your device is registered and receiving pushes. You can do this from either app.
**From the web app:** open **Notification settings**, find your device under **Contact information**, and click **Send test notification**.
**From the mobile app:** open **ilert Settings -> Contact information**, find your device, and tap **Send test notification**.
{% hint style="info" %}
A test notification confirms that pushes reach your device. It does not prove the critical alert sound is working, because only **high priority** alerts are delivered as critical alerts — so to check the alarm behaviour itself, you need a high priority alert.
{% endhint %}
## FAQ
**Where did the "Critical alerts volume" slider go?**
It is only hidden on **Android 17 and newer**. There, alerts play at your alarm volume, so the slider no longer had any effect. Turn your alarm volume up instead. On **Android 16 and older** the slider is still there and still works. It also stays on iOS, where Apple allows a volume per alert.
**Does ilert still turn off my Do Not Disturb?**
On **Android 17 and newer**, no. ilert no longer changes your phone's settings.
On **Android 16 and older**, yes. Those versions still switch Do Not Disturb off and set your sound mode to Sound when a critical alert arrives, and cannot put them back.
**Alerts arrive but make no sound. What should I check?**
On **Android 17 and newer**, open **Push notification settings** in the app first — it tells you if your notifications, the critical alerts category, or your alarm volume are the problem, and links straight to the fix.
The app warns you when something would keep a critical alert silent
Otherwise, check your volume — alarm volume on Android 17 and newer, notification volume on Android 16 and older. Then check that alarms are allowed in the Do Not Disturb mode you are actually in.
# On-call widget
The On-call widget lets you instantly see whether you're on-call, review your upcoming shifts, create schedule overrides and export your on-call calendar.
## Check your on-call status
1. Open the navigation by tapping the menu icon in the top-left corner.
2. Now, you can see the widget right below your name.
* **You are on-call**\
Shows how much time remains in your current shift.
* **Not on-call**\
Indicates you're off duty and displays how many policies are currently selected.
**Actions:**
* Tap the widget itself to open the My on-call shifts screen.
* Tap **Request coverage** to launch the [coverage request](/on-call-management-and-escalations/on-call-coverage-requests) dialog.
## My on-call shifts page
* **Policy selector (top-center):**\
Select the escalation levels for which you wish to see on-call entries.\
\&#xNAN;*Note: You can only select policies to which you're assigned.*
* **Actions menu (top-right):** Tap the **•••** icon to open available actions.
* **Shift list (bottom section):**\
Displays each shift's date, time range, and associated policy.\
Tap any entry to open the **Override shift** menu.
**Actions menu options:**
* **Take someone else's on-call**\
Opens the menu to temporarily assign yourself to someone's on-call shift.
* [**Request on-call coverage**](/on-call-management-and-escalations/on-call-coverage-requests)\
Request coverage for your on-call shifts from colleagues.
* **Export to my calendar**\
Add your shifts to your phone's calendar.
* **Change time zone**\
Adjust the calendar display to a different time zone.
# Types of integrations
Explore ilert integration types: inbound, outbound, and deployment‑based, to automate alerts and enhance system workflows.
ilert integrations connect your tools and workflows for efficient incident management and reliable alerting. They are organized into three categories:
1. **Inbound Integrations**\
Also called [alert sources](/alerting/configure-alerting/alert-sources), they enable you to connect monitoring tools, ticketing systems, or applications to ilert for reliable incident alerting. Use pre-built integrations, email, Event API, SMS, or heartbeat monitoring to create and manage incidents effectively.
2. **CI/CD integrations**\
[Deployment events](/alerting/deployment-events) enrich incidents by bringing deployment activity, such as releases or merges, into your ilert incident management platform.
3. **Outbound Integrations**\
Known as connectors and [alert actions](/#connector-and-alert-action-aka-outbound-integration), outbound integrations let you automate incident responses. Update status pages, create tickets in JIRA, or send notifications to Slack to ensure timely and coordinated incident response.
Learn how to configure each type of integration to enhance your incident management operations:
