Skip to main content

NotificationRule

The Cloudders custom resource definition NotificationRule is used to create alerts for single or a list of specific HealthChecks.

Currently only supports Slack as destination receiver, but more receivers will be integrated with future releases.

Example NotificationRule receiver​

In the below example we will create a simple NotificationRule that sends alert for our cloudders-docs-configuration health check we created in the previous documentation page.

kubectl apply -f - <<EOF
apiVersion: check.cloudders.com/v1alpha1
kind: NotificationRule
metadata:
  name: cloudders-docs-configuration
spec:
  checks:
    - "cloudders-docs-configuration"
  slack:
    webhook:
      secret:
      - name: slack-webhook-secret
        key: org1
    username: "controller"
    channel: "#general"
    message: |
      "type": "section",
      "text": {
        "type": "mrkdwn",
      ...
  waitBeforeSend: 1
  repeatAfter: 1
EOF

After the NotificationRUle is applied with kubectl we can see the results with the following command:

kubectl get notificationrules

And the command will print the current running HealthCheck resources in the namespace:

NAME                           LAST NOTIFICATION      LAST RESOLVED NOTIFICATION   CURRENT STATUS
cloudders-docs-configuration                                                       {"cloudders-docs-configuration":true}

The NotificationRule supports simple text as payload to Slack, or Slack blocks.

The exported Status values of the HealthCheck operator NotificationRule object are:

  • %check% - name of the check for which we run the NotificationRule.
  • %response_time% - response time of the target (in case of full downtime this will be the default operator timeout)
  • %triggerTime% - time when the check has been triggered against the target.
  • %response_code% - the response code of the target.
  • %search_type% - what searchType is used for the HealthCheck
  • %search_used% - what search string or expression was used in the HealthCheck.

Here is example Slack block configuration:

{
"color": "#FF0000",
"blocks": [
{
  "type": "section",
  "text": {
    "type": "mrkdwn",
    "text": "Alert for check *%check%* :warning:"
  }
},
{
  "type": "section",
  "fields": [
    {
      "type": "mrkdwn",
      "text": "*Average response time: *\n%response_time%"
    },
    {
      "type": "mrkdwn",
      "text": "*Triggered at: *\n%triggerTime%"
    }
  ]
},
{
  "type": "section",
  "fields": [
    {
      "type": "mrkdwn",
      "text": "*Current response code is: *\n%response_code%"
    },
    {
      "type": "mrkdwn",
      "text": "*Search type used is: *\n%search_type%"
    },
    {
      "type": "mrkdwn",
      "text": "*Search match used is: *\n%search_used%"
    }

And here is example of simple text message:

Response time was: %response_time%
Triggered at: %triggeredAt%
Response code: %response_code%

Fields​

Below is a table of all fields supported by the NotificationRule CRD under spec:

Field nameTypeExampleDescription
checks:Listcloudders-docs-configurationSource HealthChecks for which to trigger this NotificationRule. Supported are single or multiple checks in form of a YAML List.
slack:SectionHere we host our Slack configuration
webhook:SectionHere we set our Slack webhook secrets config.
secret:ListSingle or multiple secrets are supported. This way you can have alert routed to different Slack workspaces
name:Stringslack-webhook-secretName of the Slack secret to use
key:Stringorg1Key inside the Slack secret
waitBeforeSend:Integer1Interval to wait before sending the alert. Perfect for reducing false-positives on external services that take some time to respond. Set in minutes.
repeatAfter:Integer1Interval to wait before sending the alert again. Perfect for reducing noise when for example a maintenance on a component is made. Set in minutes.