Docs / Guides

Monitor cron jobs and workers

A heartbeat monitor alerts you when something that should happen regularly doesn't: the nightly backup didn't run, the queue worker died, the scheduled sync silently stopped. Your job calls a URL when it finishes. If the call doesn't arrive on time, you're alerted.

#Set it up

1. Monitors → New → Heartbeat. Set Expected every to how often the job runs, and Grace time to how late it may be (jobs that take a variable time need more grace).
2. Copy its heartbeat URL: https://pulsefast.io/hb/…. The URL is the secret, so keep it out of public repositories.
3. Call it when the job succeeds. GET, POST and HEAD all work.

To report a failure right away, instead of waiting for the ping to go missing, call the URL with /fail at the end.

#Cron

# Ping on success, report a failure on error. -m 10: never let the ping hang the job.
0 3 * * * /usr/local/bin/backup.sh && curl -fsS -m 10 --retry 3 https://pulsefast.io/hb/YOUR_TOKEN || curl -fsS -m 10 https://pulsefast.io/hb/YOUR_TOKEN/fail

#systemd timers

Add to the job's .service unit:

[Service]
ExecStart=/usr/local/bin/backup.sh
ExecStartPost=/usr/bin/curl -fsS -m 10 --retry 3 https://pulsefast.io/hb/YOUR_TOKEN
ExecStopPost=/bin/sh -c '[ "$SERVICE_RESULT" = success ] || /usr/bin/curl -fsS -m 10 https://pulsefast.io/hb/YOUR_TOKEN/fail'

#Kubernetes CronJob

apiVersion: batch/v1
kind: CronJob
metadata:
  name: nightly-backup
spec:
  schedule: "0 3 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: backup
              image: your-backup-image
              env:
                - name: HEARTBEAT_URL
                  valueFrom: {secretKeyRef: {name: pulsefast, key: heartbeat-url}}
              command: ["/bin/sh", "-c"]
              args:
                - /backup.sh && wget -q -T 10 -O /dev/null "$HEARTBEAT_URL" || wget -q -T 10 -O /dev/null "$HEARTBEAT_URL/fail"

#GitHub Actions on a schedule

Scheduled workflows can be delayed or silently disabled (GitHub turns them off in inactive repositories), so they're worth watching:

on:
  schedule:
    - cron: "0 3 * * *"
jobs:
  sync:
    runs-on: ubuntu-latest
    steps:
      - run: ./sync.sh
      - if: success()
        run: curl -fsS -m 10 --retry 3 "${{ secrets.PULSEFAST_HEARTBEAT_URL }}"
      - if: failure()
        run: curl -fsS -m 10 "${{ secrets.PULSEFAST_HEARTBEAT_URL }}/fail"

#Long-running workers

For a queue worker or daemon, ping from its main loop, at most once a minute, and set Expected every to a few minutes:

import time, urllib.request

last = 0
while True:
    process_next_job()
    if time.time() - last > 60:
        try:
            urllib.request.urlopen("https://pulsefast.io/hb/YOUR_TOKEN", timeout=10)
        except OSError:
            pass  # never let monitoring break the worker
        last = time.time()

Pings are limited to 60 a minute per heartbeat.

#Good to know

• Maintenance windows pause heartbeats too, and each one gets a fresh period when the window ends.
• A new heartbeat expects its first ping within one period plus the grace time after you create it, so create it just before the job's next run, or with a generous grace time.
• Create heartbeats from code with the API ("type": "heartbeat"); the response includes heartbeat_url.

View as Markdown · Something unclear? Tell us