# Monitor cron jobs and workers

Heartbeat monitors for cron, systemd timers, Kubernetes CronJobs, GitHub Actions schedules and background workers.


A **heartbeat monitor** alerts you when something that should happen regularly *doesn't*: the nightly
backup didn't run, the queue worker died, the scheduled sync silently stopped. Your job calls a URL
when it finishes. If the call doesn't arrive on time, you're alerted.

## Set it up

1. **Monitors → New → Heartbeat.** Set **Expected every** to how often the job runs, and **Grace time**
   to how late it may be (jobs that take a variable time need more grace).
2. Copy its **heartbeat URL**: `https://pulsefast.io/hb/…`. The URL is the secret, so keep it out of
   public repositories.
3. Call it when the job succeeds. `GET`, `POST` and `HEAD` all work.

To report a failure **right away**, instead of waiting for the ping to go missing, call the URL with
`/fail` at the end.

## Cron

```sh
# Ping on success, report a failure on error. -m 10: never let the ping hang the job.
0 3 * * * /usr/local/bin/backup.sh && curl -fsS -m 10 --retry 3 https://pulsefast.io/hb/YOUR_TOKEN || curl -fsS -m 10 https://pulsefast.io/hb/YOUR_TOKEN/fail
```

## systemd timers

Add to the job's `.service` unit:

```ini
[Service]
ExecStart=/usr/local/bin/backup.sh
ExecStartPost=/usr/bin/curl -fsS -m 10 --retry 3 https://pulsefast.io/hb/YOUR_TOKEN
ExecStopPost=/bin/sh -c '[ "$SERVICE_RESULT" = success ] || /usr/bin/curl -fsS -m 10 https://pulsefast.io/hb/YOUR_TOKEN/fail'
```

## Kubernetes CronJob

```yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: nightly-backup
spec:
  schedule: "0 3 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: backup
              image: your-backup-image
              env:
                - name: HEARTBEAT_URL
                  valueFrom: {secretKeyRef: {name: pulsefast, key: heartbeat-url}}
              command: ["/bin/sh", "-c"]
              args:
                - /backup.sh && wget -q -T 10 -O /dev/null "$HEARTBEAT_URL" || wget -q -T 10 -O /dev/null "$HEARTBEAT_URL/fail"
```

## GitHub Actions on a schedule

Scheduled workflows can be delayed or silently disabled (GitHub turns them off in inactive repositories),
so they're worth watching:

```yaml
on:
  schedule:
    - cron: "0 3 * * *"
jobs:
  sync:
    runs-on: ubuntu-latest
    steps:
      - run: ./sync.sh
      - if: success()
        run: curl -fsS -m 10 --retry 3 "${{ secrets.PULSEFAST_HEARTBEAT_URL }}"
      - if: failure()
        run: curl -fsS -m 10 "${{ secrets.PULSEFAST_HEARTBEAT_URL }}/fail"
```

## Long-running workers

For a queue worker or daemon, ping from its main loop, at most once a minute, and set **Expected every**
to a few minutes:

```python
import time, urllib.request

last = 0
while True:
    process_next_job()
    if time.time() - last > 60:
        try:
            urllib.request.urlopen("https://pulsefast.io/hb/YOUR_TOKEN", timeout=10)
        except OSError:
            pass  # never let monitoring break the worker
        last = time.time()
```

Pings are limited to 60 a minute per heartbeat.

## Good to know

- **Maintenance windows** pause heartbeats too, and each one gets a fresh period when the window ends.
- A new heartbeat expects its first ping within one period plus the grace time after you create it, so
  create it just before the job's next run, or with a generous grace time.
- Create heartbeats from code with the [API](/docs/api) (`"type": "heartbeat"`); the response includes
  `heartbeat_url`.