Kubernetes

Give an agent access to a Kubernetes cluster by deploying ilert-kube-agent with Helm or Terraform.

Required permissions

Tool	Required Permission	Description
list_pods	get, list	Gets name, status and namespace of pods (containers) in the Kubernetes cluster. Useful for finding pods that are not running or terminated/crashing containers.
describe_pod	get	Describe the configuration and status as well as other insightful information about a pod (container) in a Kubernetes cluster. Only use this tool when you have a valid, existing pod name. If the pod name is unknown, use list_pods first to find available pods.
fetch_pod_logs	get	Fetch tailing logs of a pod (container) in a Kubernetes cluster. Only use this tool when you have a valid, existing pod name. If the pod name is unknown, use list_pods first to find available pods.
fetch_previous_pod_logs	get	Fetch the previous tailing logs of a pod (container) in a Kubernetes cluster. This method is useful to investigate why a given pod crashed or has restarted. Only use this tool when you have a valid, existing pod name. If the pod name is unknown, use list_pods first to find available pods.
patch_cpu_workload_by_pod	patch	Update CPU request and limit of a workload related to a pod (container) in a Kubernetes cluster. Useful for fixing errors caused by low compute limits usually resulting in bad service performance or slow responses. Returns the name of a pod of the new rollout or null if a timeout occurs or no patch is applied. Only use this tool when you have a valid, existing pod name. If the pod name is unknown, use list_pods first to find available pods.
patch_memory_workload_by_pod	patch	Update the memory (ram) request and limit of a workload related to a pod (container) in a Kubernetes cluster. Useful for fixing errors caused by low memory limits usually resulting in bad service performance or OOM killed or crashes/unresponsiveness. Returns the name of a pod of the new rollout or null if a timeout occurs or no patch is applied. Only use this tool when you have a valid, existing pod name. If the pod name is unknown, use list_pods first to find available pods.
scale_workload_by_pod	patch	Set a new replica size for a workload related to a pod. Returns the name of a pod belonging to the workload or null if no pod was found after timeout. Only use this tool when you have a valid, existing pod name. If the pod name is unknown, use list_pods first to find available pods.
delete_pod	delete	Delete a pod. Only use this tool when you have a valid, existing pod name. If the pod name is unknown, use list_pods first to find available pods.
rollback_workload_by_pod	patch	Rollback a workload related to a pod (container) in a Kubernetes cluster. Useful for rolling back to a previous version of a workload. Returns the name of a pod of the new rollout or null if a timeout occurs or no rollback is applied. Only use this tool when you have a valid, existing pod name. If the pod name is unknown, use list_pods first to find available pods.

Deploy ilert-kube-agent

The Kubernetes MCP server requires deploying the ilert-kube-agent in your Kubernetes cluster. The ilert-kube-agent can be deployed using Helm or Terraform.

Option 1: Deploy using Helm

1

Add the ilert Helm repository:

helm repo add ilert https://ilert.github.io/charts/
helm repo update

2

Deploy ilert-kube-agent:

helm upgrade --install --namespace kube-system \
    ilert-kube-agent ilert/ilert-kube-agent \
    --set config.settings.apiKey="<YOUR ILERT API KEY>"

3

Generate an HTTP API secret (httpAuthorizationKey) to activate API mode:

# Generate a random secret
openssl rand -hex 32

4

Set the HTTP API secret in the Helm values or update the deployment:

helm upgrade --install --namespace kube-system \
    ilert-kube-agent ilert/ilert-kube-agent \
    --set config.settings.apiKey="<YOUR ILERT API KEY>" \
    --set config.settings.httpAuthorizationKey="<YOUR GENERATED SECRET>"

Option 2: Deploy using Terraform

1

Use the ilert-kube-agent Terraform module:

module "ilert_kube_agent" {
  source = "ilert/ilert-kube-agent/kubernetes"
  
  api_key = "<YOUR ILERT API KEY>"
  http_authorization_key = "<YOUR GENERATED SECRET>"
}

2

Generate an HTTP API secret (httpAuthorizationKey) to activate API mode:

# Generate a random secret
openssl rand -hex 32

3

Apply the Terraform configuration:

terraform init
terraform apply

For more details, see the ilert-kube-agent documentation.

Note: The HTTP API secret (httpAuthorizationKey) is required to activate API mode, which enables the MCP server to communicate with the ilert-kube-agent. Make sure to securely store this secret as it will be needed when connecting the Kubernetes integration in ilert.

Connect Kubernetes with agent

To connect Kubernetes with your AI SRE agent:

1

Navigate to Agents in the ilert navigation menu

2

Open the agent detail view for the agent you want to configure

3

Go to the Tools tab

4

Under the ilert hosted MCP servers section, locate the Kubernetes card

5

Click Connect on the Kubernetes card

6

In the connection dialog that appears:

• Select connector: Choose "Create new connector" (or select an existing connector if you have one)
• Type: Select Bearer Token from the dropdown
• Secret name: Enter a name for your connector (e.g., "Kubernetes Production" or "Kubernetes Main")
• Secret value: Enter the HTTP API secret (httpAuthorizationKey) you generated when deploying ilert-kube-agent
• Host URL: Enter the URL where your ilert-kube-agent is accessible (e.g., https://ilert-kube-agent.kube-system.svc.cluster.local:8080 or your external URL)

7

Click Connect to complete the connection

8

After connecting, you can enable or disable specific tools needed for alert investigation, such as:

• list_pods - Get name, status and namespace of pods in the Kubernetes cluster
• describe_pod - Describe configuration and status of a pod
• fetch_pod_logs - Fetch tailing logs of a pod
• fetch_previous_pod_logs - Fetch previous logs of a pod (useful for investigating crashes)
• patch_cpu_workload_by_pod - Update CPU request and limit of a workload
• patch_memory_workload_by_pod - Update memory request and limit of a workload
• scale_workload_by_pod - Set a new replica size for a workload
• delete_pod - Delete a pod
• rollback_workload_by_pod - Rollback a workload to a previous version

The Kubernetes integration is now connected and ready to use for root cause analysis.