Webhook Relay
Sign up free

Splunk Webhook Alerts: Payload, Limits and Fan-Out

Wire Splunk alert actions to webhooks: the exact JSON payload Splunk sends, the webhook action's limitations (no custom headers), URL allowlisting in Splunk Cloud, and how to fan alerts out to Slack, PagerDuty or anywhere else.

Splunk webhook alerts

Splunk's webhook alert action is the quickest way to get a saved-search alert out of Splunk and into your own systems — and also one of the most restrictive webhook senders you will meet: fixed payload, no custom headers, no retries to speak of. This guide shows exactly what it sends (from real captured traffic), the constraints, and the patterns that turn one Splunk alert into properly formatted notifications anywhere.

Setting it up

On a saved search: Save As → Alert → Trigger Actions → Add Actions → Webhook, then paste the target URL. That is the whole configuration surface — one URL field. Every time the alert triggers, Splunk POSTs JSON to it.

Splunk Cloud gotcha: deliveries only go to allowlisted destinations. An admin adds your endpoint under Server settings → Webhook allow list first; otherwise the action silently fails (check python.log/webhook logs).

The payload

This is what actually arrives (shape captured on our public webhook bin; values fictionalized):

{
  "sid": "scheduler__search__RMD5example_at_1768472000_100",
  "search_name": "High error rate on checkout service",
  "app": "search",
  "owner": "alerts",
  "results_link": "https://splunk.example.com:8000/app/search/search?q=%7Cloadjob%20…",
  "result": {
    "host": "web-03.example.internal",
    "source": "/var/log/app/error.log",
    "count": "147",
    "status_text": "error rate above threshold"
  }
}

Key facts:

  • result holds only the first row of the alert's search results, fields as strings. If you need all rows, have the receiver follow results_link back into Splunk's API, or restructure the search so row one carries what you need.
  • The request comes from user agent Splunk/<instance-uuid> with Content-Type: application/json.
  • There is no signature and no auth header. Anyone who learns the URL can POST fake alerts, so treat the URL as a secret (path token) and/or validate the source IP.
  • The bin's sample catalog includes this exact event shape — one click sends a realistic Splunk alert at your handler while you develop.

The limitations — and the pattern around them

The webhook action cannot: set headers, template the body, sign requests, or reshape JSON. Meanwhile real destinations want exactly those things — Slack wants {"text": …}, PagerDuty wants its Events API format, your internal gateway wants a bearer token.

The clean solution is a relay in the middle. Point Splunk's one URL field at a stable Webhook Relay endpoint, then:

  • Transform in flight — a small JavaScript transformation function reshapes Splunk's payload into the Slack/Teams/PagerDuty format and sets any headers your destination needs.
  • Fan out — deliver one alert to multiple destinations at once: the on-call channel, the incident tool, and a data sink.
  • Reach internal systems — if the real consumer sits in a private network, the relay agent forwards alerts inside without opening firewall ports, the same pattern as receiving webhooks on localhost.
  • Add reliability Splunk lacks — deliveries are stored and retried towards your destination for up to 30 days, so a receiver restart doesn't drop the alert. One allowlist entry in Splunk Cloud covers every downstream destination, present and future.

Testing without triggering real alerts

  1. Capture one real firing into a Webhook Bin to see your instance's exact fields (custom search fields land in result).
  2. Replay it against your handler while you iterate — or send the catalog's Splunk sample.
  3. Wire the real alert to the relay endpoint once your handler behaves.