Configure settings for Grafana Incident
Grafana Incident has several customizable settings that allow you to define all aspects of an incident to fit your response needs and team workflows. With these settings, you can:
Note: Grafana Cloud Admins can access incident settings.
About incident labels
Annotating incidents with labels helps you filter and better understand incident trends overtime. Use labels to group incidents and tag them with interesting metadata such as which teams were involved, what services were impacted, and if customers were impacted.
Use the + Add Label option in the incident card to apply labels. You can apply labels to both active and resolved incidents.
Add incident labels
- In the left-side menu, expand Alerts & IRM, then Incidents.
- Click Settings.
- Under Labels, click + Add new label.
- Provide a name and description for the label.
- Optional: Assign a color from the hue wheel or leave the default color.
- Click Add and repeat steps 2-4 as needed.
For example, suppose you want labels to track incidents by teams involved. Create team-based labels as shown in the following image:
Edit incident labels
- In the left-side menu, expand Alerts & IRM, then Incidents.
- Click Settings.
- On an existing label, click the pencil icon.
- Edit the name, description, or color as needed, and click Update.
About incident severities
Incident severities provide a defined measurement of the impact of an incident. Consistent and well-defined severities help others in your organization quickly understand the urgency of an incident.
Incident severities may influence your response process, such as how many people are engaged and who to inform. To get the most value out of incident severities, establish clear definitions of each severity level and the expected response within your organization.
The following recommended incident severities are pre-configured in Grafana Incident:
Severity | Description |
---|---|
Critical | Urgently requires immediate attention |
Major | Significant blocking problem that requires help |
Minor | May be affecting customers, but no one is blocked |
Pending | Severity to be decided and should be determined as soon as possible |
Edit incident severities
You can customize the severity label
and description
fields to match the severity levels your team uses.
- In the left-side menu, expand Alerts & IRM, then Incidents.
- Click Settings.
- On an existing severity, click the pencil icon.
- Edit the label and description fields as needed and click Save.
Note: To ensure your custom severities levels are mapped to the corresponding severity level in Grafana Incident, define your highest severity with the pre-configured critical severity.
For example, suppose your severity levels are defined as P1-P4:
Custom severity | Pre-configured severity |
---|---|
Priority 1 | Critical |
Priority 2 | Major |
Priority 3 | Minor |
Priority 4 | Pending |
Disable incident severities
If needed, you can disable severities that don’t align with the severity levels your team uses.
- In the left-side menu, expand Alerts & IRM, then Incidents.
- Click Settings.
- On the desired existing severity, click the toggle to switch from enabled to disabled.
About incident statuses
The status of an incident indicates whether the incident is ongoing or if the issue has been resolved. The incident status should immediately indicate whether or not incident response is still in progress.
The following incident statuses are pre-configured in Grafana Incident:
Status | Description |
---|---|
Active | The incident is happening now |
Resolved | The incident is resolved |
Edit incident statuses
Customize what you call active and resolved incidents:
- In the left-side menu, expand Alerts & IRM, then Incidents.
- Click Settings.
- Scroll down to Statuses, click the pencil icon for an existing status, and edit the label and description as needed.
- Click Save.
About channel prefixes
Having a central and dedicated place for incident response to take place can be helpful to keep communication organized during an incident and improve analysis for post-incident reviews.
Prefixes determine the name of automatically created incident channels. For example, if the channel prefix is security
then the channel name will be #security-date-title
Tip: To maintain searchable and easy to manage incident channels, consider customizing your channel prefixes for different teams or services.
Edit channel prefixes
To customize Slack channel prefixes:
- In the left-side menu, expand Alerts & IRM, then Incidents.
- Click Settings.
- Navigate to Prefixes at the bottom of the Settings page.
- Click + Add Prefix, provide a name and description, and click Add.
- Click the pencil icon to edit any existing prefixes and click Update.
Once your prefixes are defined, you can specify which prefix to use when you declare an incident in Grafana Incident.
About incident roles
Incident roles help to identify who’s involved and what they’re responsible for. Grafana Incident is pre-configured with two recommended key roles:
- Commander: Oversees the incident by managing communication, tasks, and necessary updates.
- Investigator: Responsible for diagnosing and resolving the incident.