Ctrlk
ProductLoginStart for Free
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Grafana

Give an agent access to Grafana, Loki and Tempo with a service account token.

Required permissions

Tool
Required Permission
Description

list_loki_label_names

query api access

List all label names (keys) found in logs in a Loki datasource and time range. Defaults to the last hour.

list_loki_label_values

query api access

List the unique values of a given label in a Loki datasource. Useful for discovering filter options.

query_loki_logs

query api access

Query and retrieve log entries from a Loki datasource using label filters. Returns log lines with timestamps and labels.

search_dashboards

dashboards.permissions:read

Search Grafana dashboards to find the ones whose graphs might be relevant to the current alert.

list_dashboards

dashboards.permissions:read

List dashboards in a given namespace, for discovering which graphs exist.

get_dashboard

dashboards.permissions:read

Get a dashboard by name, including its panels with their queries and datasources.

list_tempo_services

query api access

Discover the available service.name values in a Tempo datasource.

search_tempo_traces

query api access

Search Tempo traces for a service and time range, optionally filtered by duration. Returns trace summaries including trace IDs.

get_tempo_trace

query api access

Fetch a single Tempo trace as a compacted span tree with service names, durations, and HTTP context.

The Loki and Tempo tools query datasources by UID, so the service account token must be able to read the Loki and Tempo datasources you want the agent to use. If you only use Loki, you can leave the Tempo tools disabled, and vice versa.

Create service account token

1

Create a Service account token with Viewer role

2

On the next screen, click on the Add service account token button

3

On the modal choose "No expiration" and click on the Generate token button

4

Copy your token and temporarily paste it somewhere secure (e.g. a private note). You will later need the token when setting up the Grafana MCP server in ilert.

Connect Grafana with agent

To connect Grafana with your AI SRE agent:

1

Navigate to Agents in the ilert navigation menu

2

Open the agent detail view for the agent you want to configure

3

Go to the Tools tab

4

Under the ilert hosted MCP servers section, locate the Grafana card

5

Click Connect on the Grafana card

6

In the connection dialog that appears:

  • Select connector: Choose "Create new connector" (or select an existing connector if you have one)

  • Type: Select Bearer Token from the dropdown

  • Secret name: Enter a name for your connector (e.g., "Grafana Production" or "Grafana Main")

  • Secret value: Paste the service account token you created earlier

  • Host URL: Enter your Grafana instance URL (e.g., https://grafana.example.com)

7

Click Connect to complete the connection

8

After connecting, you can enable or disable specific tools needed for alert investigation, such as:

  • list_loki_label_names - List available label names in a Loki datasource

  • list_loki_label_values - List the values of a given Loki label

  • query_loki_logs - Query log entries from a Loki datasource

  • search_dashboards - Search dashboards relevant to the alert

  • list_dashboards - List dashboards in a namespace

  • get_dashboard - Get a dashboard with its queries and datasources

  • list_tempo_services - Discover services in a Tempo datasource

  • search_tempo_traces - Search traces for a service and time range

  • get_tempo_trace - Fetch a single trace as a span tree

The Grafana integration is now connected and ready to use for root cause analysis.

PreviousGitHub
NextGraylog

Last updated

Was this helpful?