Alerts & Incidents

Alert Rules

Define when and how alerts are triggered based on monitor results.

Alert Rules

Alert rules define the conditions that trigger notifications. Create rules to match your operational requirements and avoid alert fatigue.

Creating Alert Rules

  1. Go to Settings (Alert Rules section)
  2. Click New Rule
  3. Configure conditions and channels
  4. Apply to monitors

Rule Types

Consecutive Failures

Alert after N failures in a row. Most common rule type.

Name: 3 Consecutive Failures
Type: consecutive_failures
Threshold: 3
Channels:
  - Slack (#ops-alerts)
  - Email ([email protected])

Use when: You want to avoid alerting on transient issues.

ThresholdUse Case
1Critical systems, zero tolerance
2-3Standard production monitoring
5+Flaky checks, high-noise monitors

Error Rate

Alert when failure percentage exceeds threshold over a time window.

Name: High Error Rate
Type: error_rate
Threshold: 10%  # percentage
Window: 1 hour
Min Samples: 10  # minimum runs to evaluate
Channels:
  - Slack (#ops-alerts)

Use when: Occasional failures are acceptable, but sustained issues aren't.

Latency Threshold

Alert when response time exceeds limit.

Name: Slow Response Alert
Type: latency_threshold
Threshold: 2000  # milliseconds
Channels:
  - Slack (#performance)

Use when: Performance degradation impacts users.

Visual Diff Threshold

Alert when visual changes exceed threshold (journeys only).

Name: Visual Regression Alert
Type: visual_diff_threshold
Threshold: 5%  # pixel difference percentage
Channels:
  - Slack (#design-review)
  - Email ([email protected])

Use when: UI consistency is critical.

SSL Certificate Expiry

Alert before TLS certificate expires.

Name: SSL Expiry Warning
Type: ssl_expiry
Days Before: 30
Channels:
  - Email ([email protected])

Recommended thresholds:

  • 30 days: Warning
  • 14 days: Urgent
  • 7 days: Critical

Rule Configuration

Applying Rules to Monitors

Rules link to a specific monitor, API monitor, or journey when created. Each rule targets one resource and one alert channel.

Recovery Notifications

When a monitor recovers after a failure, izli.io automatically sends a recovery notification to the same channels configured on the rule.

Example Configurations

Checkout Journey — Zero Tolerance

Name: Checkout Flow Down
Type: consecutive_failures
Threshold: 1
Channel: Slack (#critical)

API — Sustained Failures

Name: API High Error Rate
Type: error_rate
Threshold: 10%
Channel: Slack (#ops-alerts)

Performance Degradation

Name: Slow Response
Type: latency_threshold
Threshold: 2000  # ms
Channel: Email ([email protected])

Best Practices

  1. Start conservative - Begin with higher thresholds, tune down
  2. Avoid single-failure alerts - Too noisy for most cases
  3. Set up multiple channels - Slack for speed, email for records
  4. Include recovery - Know when issues resolve
  5. Review regularly - Tune based on false positive rate

Alert fatigue is real. If your team ignores alerts because there are too many, adjust your thresholds. Better to miss a minor issue than ignore a critical one.

Troubleshooting

Too Many Alerts

  • Increase consecutive failure threshold
  • Use error rate instead of per-failure
  • Review monitor reliability

Missing Alerts

  • Check rule is applied to the correct monitor
  • Verify channels are configured and tested
  • Test with Send Test Alert on the channel

Alert Delays

  • Check channel delivery (Slack/email)
  • Verify webhook endpoints respond
  • Review alert processing logs