Get started
You can programmatically control Grafana Incident via the Grafana Incident JSON/HTTP RPC API.
Grafana Incident provides a simple JSON/HTTP RPC request/reply API, that exposes a series of useful methods for you to call remotely from your own code.
The endpoint is available at the following URL:
https://your-stack.grafana.net/api/plugins/grafana-incident-app/resources/api/v1
Note: You will need to replace your-stack.grafana.net
with your own instance.
You can explore the entire API surface in the reference documentation or continue reading for an introduction.
Use cases
You can use the API along with the Incoming and Outgoing webhooks to solve a wide range of integration use cases.
Some examples include:
- Trigger a ServiceNow workflow whenever an incident is declared
- Automatically add some common tasks whenever a specific label is added
- Send the Incident summary in an email when it is resolved
- Send a Slack DM to the Security Team if somebody adds the
security
label - Trigger a custom tool whenever an incident is declared, and use the API to write an update to the activity feed
How it works
To make calls into the API, you can use one of the official client libraries or construct your own HTTP requests.
You will need to use a Service account token in the Authorization header in order to authenticate your request.
An RPC API is like calling remote methods. For every request you make, you get a response.
Examples
This section provides some real-world examples of how you will typically interact with the Grafana Incident JSON/HTTP RPC API.
Although the examples are in Go, your code will likely look very similar regardless of your language choice.
Declare an incident
Programmatically declare an incident in Go using the Grafana Incident Go client library:
import (
"fmt"
"os"
incident "github.com/grafana/incident-go"
)
func main() {
// create a client, and the services you need
serviceAccountToken := os.Getenv("SERVICE_ACCOUNT_TOKEN")
client := incident.NewClient(
"https://your-stack.grafana.net/api/plugins/grafana-incident-app/resources/api/v1",
serviceAccountToken,
)
incidentsService := incident.NewIncidentsService(client)
// declare an incident
createIncidentResp, err := incidentsService.CreateIncident(ctx, incident.CreateIncidentRequest{
Title: "short description explaining what's going wrong",
})
if err != nil {
// if something goes wrong, the error will help you
return fmt.Errorf("create incident: %w", err)
}
// success, get the details from the createIncidentResp object
incidentID := createIncidentResp.Incident.IncidentID
fmt.Println("declared Incident", createIncidentResp.Incident.IncidentID)
}
Automatically add a task
To add a task to the above Incident, we can use the *incident.TasksService
with the same client
:
tasksService := incident.NewTasksService(client)
addTaskResp, err := tasksService.AddTask(ctx, incident.AddTaskRequest{
IncidentID: incidentID,
Text: "Consider tweeting to let people know",
})
if err != nil {
// if something goes wrong, the error will help you
return fmt.Errorf("add task: %w", err)
}
// success, print the result
fmt.Println("added task %s to incident %s", addTaskResp.Task.TaskID, incidentID)
Post a Slack message if a specific label is added
By using webhooks when an incident is declared or updated,
getIncidentResp, err := incidentsService.GetIncident(ctx, incident.GetIncidentRequest{
IncidentID: incidentID,
})
if err != nil {
return fmt.Errorf("get incident %s: %w", incidentID, err)
}
for _, item := range getIncidentResp.Incident.Labels {
if item.Label == "specific-label" {
postToSlack(ctx, "An incident has been flagged with the security label.")
return
}
}
You would write your own postToSlack
method to interact with Slack’s API.
Note It is possible for webhooks to fire multiple times as the Incident changes. It is up to you to make sure you don’t spam people.
Make your own HTTP requests
You can interact with the Grafana Incident JSON/HTTP RPC API by making HTTP POST requests.
Figure out the correct endpoint by combining your host, API path, service and methods.
For example,
https://your-stack.grafana.net/api/plugins/grafana-incident-app/resources/api/v1/IncidentsService.CreateIncident
, replacingyour-stack.grafana.net
.Look up the relevant request and response objects from the reference documentation (they match the method name, for example,
CreateIncidentRequest
andCreateIncidentResponse
).Construct a JSON object representing the request object and set that as the body of the request.
Set the
Authorization
header toBearer your_service_account_token_here
.Learn more about how to get your Service account token.
Set the
Content-Type
andAccept
headers toapplication/json; charset=utf-8
.Make a
POST
request.Parse the appropriate response object.
Was this page helpful?
Related resources from Grafana Labs
![Getting started with the Grafana LGTM Stack](/static/assets/featured/getting-started-with-grafana-lgtm-stack-amer_featured.png?w=200 200w, /static/assets/featured/getting-started-with-grafana-lgtm-stack-amer_featured.png?w=400 400w)
![Intro to Kubernetes monitoring in Grafana Cloud](/media/webinar/Screenshot-2024-02-12-at-4.09.30-PM.png?w=200 200w, /media/webinar/Screenshot-2024-02-12-at-4.09.30-PM.png?w=400 400w)
![Building advanced Grafana dashboards](/media/webinar/AMER_buildadv_notxt.png?w=200 200w, /media/webinar/AMER_buildadv_notxt.png?w=400 400w)