[Sensu Go Workshop] Lesson 6: Introduction to Filters

Sensu

Feb 27, 2023

💻 The Sensu Go Workshop is an instructor-led training series designed to empower developers, SREs, and DevOps teams begin their monitoring as code journeys. Why do I need an Observability Pipeline? What is Monitoring as Code? All these questions and more are answered in the workshop.

Taking the Workshop will help you to:

✅ Take control of your monitoring and observability data with the Sensu Go observability pipeline
✅ Schedule monitoring jobs across elastic multi-cloud infrastrucutre to execute custom scripts, including Nagios-style plugins
✅ Collect custom application metrics and integrate observability data with external data platforms for unified business intelligence
✅ Send alerts via Slack, Microsoft Teams, Pagerduty, ServiceNow, and more
✅ Fine tune observability data processing workflows and eliminate alert fatigue using Sensu Go filters
✅ Automate multi-cloud endpoint management – discovering new VMs, public cloud compute instances, containers, and more in real-time
✅ Automate remediation (self-healing) with Sensu Go

LESSON SIX OVERVIEW

In Lesson 6️⃣, we will discuss using filters in the observability pipeline. In the hands-on exercises you will use the built-in filters, then create and apply a custom filter.

This lesson is intended for operators of Sensu and assumes you have set up a local workshop environment.

0:00 - INTRO

00:50 - USING FILTERS IN THE OBSERVABILITY PIPELINE

02:27 - FILTERS AND HANDLERS

02:51 - SENSU QUERY EXPRESSIONS (SQEs)

03:25 - BUILT-IN FILTERS AND HELPER FUNCTIONS

05:10 - EXERCISE 1: USE A BUILT-IN FILTER TO ONLY ALERT ON PROBLEMS

10:16 - EXERCISE 2: CREATE A CUSTOM FILTER TO PREVENT REPEATED ALERTS

12:40 - EXERCISE 3: USING A CUSTOM FILTER IN A HANDLER

15:34 - DISCUSSION TIME

16:46 - WHAT'S NEXT

❓If you ever get stuck or have questions, feel free to reach out to us on Discourse: https://discourse.sensu.io/

NEXT STEPS

👀 Stay tuned for Lesson 7, where we will install and configure the Sensu agent, and discuss how events and other status are communicated to the backend.

RESOURCES:

📚 [Documentation] Event Filters Overview: https://docs.sensu.io/sensu-go/latest/observability-pipeline/observe-filter/
📚 [Documentation] Event Filters Reference: https://docs.sensu.io/sensu-go/latest/observability-pipeline/observe-filter/filters/
📚 [Documentation] Sensu Query Expressions Reference: https://docs.sensu.io/sensu-go/latest/observability-pipeline/observe-filter/sensu-query-expressions/
📚 [Documentation] Guide: Reduce alert fatigue with event filters: https://docs.sensu.io/sensu-go/latest/observability-pipeline/observe-filter/reduce-alert-fatigue/
📚 [Documentation] Guide: Route alerts with event filters: https://docs.sensu.io/sensu-go/latest/observability-pipeline/observe-filter/route-alerts/
📚 [Blog Post] Filters: valves for the Sensu Observability Pipeline: https://sensu.io/blog/filters-valves-for-the-sensu-monitoring-event-pipeline
📚 [Whitepaper] Alert fatigue: avoidance and course correction: https://sensu.io/resources/whitepaper/alert-fatigue-guide
📚 [Blog Post] Alert fatigue, part 1: avoidance and course correction: https://sensu.io/blog/alert-fatigue-part-1-avoidance-and-course-correction
📚 [Blog Post] Alert fatigue, part 2: alert reduction with Sensu filters & token substitution: https://sensu.io/blog/alert-fatigue-part-2-alert-reduction-with-sensu-filters-token-substitution
📚 [Blog Post] Alert fatigue, part 3: automating triage & remediation with check hooks & handlers: https://sensu.io/blog/alert-fatigue-part-3-automating-triage-remediation-with-checks-hooks-handlers
📚 [Blog Post] Alert fatigue, part 4: alert consolidation: https://sensu.io/blog/alert-fatigue-part-4-alert-consolidation
📚 [Blog Post] Alert fatigue, part 5: fine-tuning & silencing: https://sensu.io/blog/alert-fatigue-part-5-fine-tuning-silencing