Prometheus

Give an agent access to Prometheus, using whichever authentication method your server has.

Required permissions

Tool	Required Permission	Description
list_time_series_metadata	Read	Find series by label matchers. Returns a list of time series that match the provided label matchers. This endpoint returns the label sets of all series that match the provided matchers. The response includes the metric name and all label key-value pairs for each matching series. Useful for discovering available metrics, exploring label combinations, and building queries for the query_time_series_range tool.
query_time_series_range	Read	Evaluates a Prometheus expression query over a range of time to return timer series data of metrics. Use label filters in query to filter for specific metrics. Keep time ranges as short as possible, but as wide as needed. If time series metadata is unclear and the 'query' input cannot be build, the tool list_time_series_metadata may be used to filter for metric labels.

Create API token or configure authentication

Prometheus typically uses one of the following authentication methods:

Option 1: Basic Authentication

If your Prometheus instance is secured with basic authentication:

1. Ensure you have a username and password configured for Prometheus access
2. These credentials will be used when configuring the Prometheus MCP server in ilert

Option 2: Bearer Token Authentication

If your Prometheus instance uses bearer token authentication:

1. Generate a bearer token from your authentication provider (e.g., OAuth2 proxy, reverse proxy)
2. Copy the token and temporarily store it securely (e.g., a private note)
3. You will need this token when setting up the Prometheus MCP server in ilert

Option 3: No Authentication (Not Recommended)

If Prometheus is not secured (not recommended for production):

1. Ensure Prometheus is only accessible within your secure network
2. No credentials are required, but this is not recommended for production environments

Security Recommendation: We strongly recommend securing your Prometheus instance with authentication. Exposing Prometheus without authentication can lead to unauthorized access to your metrics and system information.

Verify Prometheus access

Before configuring the MCP server, verify that you can access Prometheus:

# Test Prometheus query API
curl -u username:password https://prometheus.example.com/api/v1/query?query=up

# Or with bearer token
curl -H "Authorization: Bearer YOUR_TOKEN" https://prometheus.example.com/api/v1/query?query=up

Replace prometheus.example.com with your Prometheus instance URL and use your authentication credentials.

Connect Prometheus with agent

To connect Prometheus with your AI SRE agent:

1

Navigate to Agents in the ilert navigation menu

2

Open the agent detail view for the agent you want to configure

3

Go to the Tools tab

4

Under the ilert hosted MCP servers section, locate the Prometheus card

5

Click Connect on the Prometheus card

6

In the connection dialog that appears:

• Select connector: Choose "Create new connector" (or select an existing connector if you have one)
• Type: Select Bearer Token or Basic Auth from the dropdown (depending on your Prometheus authentication method)
• Secret name: Enter a name for your connector (e.g., "Prometheus Production" or "Prometheus Main")
• Secret value: Paste the bearer token or enter username:password for basic auth
• Host URL: Enter your Prometheus instance URL (e.g., https://prometheus.example.com:9090)

7

Click Connect to complete the connection

8

After connecting, you can enable or disable specific tools needed for alert investigation, such as:

• list_time_series_metadata - Find series by label matchers to discover available metrics
• query_time_series_range - Query Prometheus metrics over a time range using PromQL

The Prometheus integration is now connected and ready to use for root cause analysis.