Step to implement alerting on Grafana for Version V9.0 and above.

Introduction:

Grafana Alerting is a powerful feature that allows users to receive real-time alerts about potential issues in their systems. With Grafana Alerting, users can quickly identify problems as soon as they occur, enabling them to take immediate action to mitigate the impact on their systems.

Creating and managing alerts with Grafana is a simple and intuitive process. Users can define alert rules that specify the conditions under which an alert should be triggered, such as when a metric value exceeds a certain threshold or when a server goes offline. Once an alert rule is defined, users can configure how they want to be notified when the alert is triggered, such as by email, Slack, or other communication channels. 

Steps to follow for configuration.

Step 1: Define SMTP setting on gafana.ini file on location /etc/grafana/grafana.ini.

Step 2: Click on Alerting. Before Creating an alert rule first, we must create a Contact point, Mute Timing and Notification policy.

Step 3: Create Contact point by clicking on new contact point.

Step 4: Add Email ID for Notification and save contact point.

Step 5: Create a Notification policy but before we must create Mute time.

Mute Timing => Mute notification policies for specific times of the day .it allows you to specify a time interval when you don’t want new notifications to be generated

Create Mute time by clicking on New Mute Timing

Step 6: Create policy by clicking on new policy.

Step 7: Define the contact point, Labels, Mute time on Notification policy and save the policy. Labels allows you to route each alert instance to a specific contact point. Here I am creating two labels alert and contact with value cpu and manas. 

Step 8: Create Grafana Alert Rule by clicking on new alert Rule.

Step 9: Define Query and alert Condition.

(Grafana supports Flux when running Influx DB v1.8 and higher. If your data source is configured for Flux, you can use the Flux query and scripting language in the query editor.)

Step 10: That is the rule will get evaluated every 1 minute for 5 minutes. Once condition is breached, alert will go in to pending state. If it is pending for longer than 5min, it will become firing alert.

Step 11: Define the match label for notification policies so that alert will connect to contact points.

Step 12: Grafana dashboard on breached alert condition for 5min Become Firing Alert.

Step 13: Email Notification generated on firing Alert.