Webhook Forwarding deep-dive

Webhook Relay provides public endpoints which can accept webhooks and then, based on user defined rules, forward them to either public or internal destinations.

Our service has a concept of bucket which is basically a grouping mechanism that enables you to accept webhooks on multiple endpoints and forward them to one or more destinations. Bucket inputs are configurable and can return custom responses, while outputs can have filtering rules and forward webhooks to both internal or public destinations:

Using configuration helpers

We have a step-by-step configuration wizards that will help you:

• https://my.webhookrelay.com/new-basic-forwarding - simple single input and single output to internal or public destination
• https://my.webhookrelay.com/new-forwarding - for more advanced use-cases when request transformation is required

Once configuration is created, you can always edit all details by visiting your buckets page.

Configuring manually

A newly created bucket will have a default input and you will need to create an output destination.

To do that, click on OUTPUT DESTINATIONS. If it’s a public destination, add the name, HTTP URL and click ‘create output’. All received webhooks will be forwarded to that destination.

If it’s an internal destination, check ‘internal network’ and start a relay agent:

$ relay forward --bucket new http://localhost:8080
Forwarding: 
https://vlndyzsibcil98gdte7l3r.hooks.webhookrelay.com -> http://localhost:8080

What can be forwarded?

Currently Webhook Relay forwards:

• Body (up to 3MB)
• Headers
• URL query, for example https://vlndyzsibcil98gdte7l3r.hooks.webhookrelay.com?foo=bar, will be forwarded to http://localhost:8080?foo=bar
• Extra path that’s after your public input endpoints: https://vlndyzsibcil98gdte7l3r.hooks.webhookrelay.com/directory/foo will be forwarded to http://localhost:8080/directory/foo

Ephemeral webhooks (no logs)

Buckets can be configure to be ephemeral so that request method, headers, query and body will not be saved to the database. To do this, go to bucket details page, click on SETTINGS tab and tick Ephemeral webhooks checkbox. All new webhooks will not have their details saved. When ephemeral mode is on, you will not be able to:

• view request details
• resend webhook

New webhook logs will appear like this:

Static or Dynamic response body settings

Sometimes applications that send webhooks accept certain headers, status code or body. To configure these settings, go to your bucket details and click on a CONFIGURE button:

Now, if you scroll down to Response configuration section, you will be able to:

• For dynamic responses select which (or any) output should respond to the caller
• For static responses, enter your preference in the “Static responses section”

Note: dynamic responses work for both public and internal endpoints as long as they respond within 10 seconds. Another option for responses are using functions.

Dynamic responses from Functions

Functions, when applied on Inputs can provide dynamic response. See Function examples.

Overriding webhook headers, timeouts, TLS verification

Output configuration allows overriding request headers so users can add any headers on top. To view output settings, go to outputs tab in bucket details and click CONFIGURE on any output:

You can also update output destination and headers through CLI:

relay output update OutputName --bucket Test --header secret-token=123456 --destination https://127.0.0.1:1880

Request matching rules

Rules can be applied on each output to filter incoming requests. There are a number of different configurations that allow:

• Match based on request body contents (such as contains, does not contain rule types)
• Match based on parsed JSON body
• Regex match based on URL query, headers or parsed JSON body
• Simple equality check based on URL query, headers or parsed JSON body

To start using rules, go to output details page. Below, there are several examples to do matching based on URL queries and request bodies.

Matching based on URL query

Value match

Let’s create a rule that checks whether URL query has a specific parameter that matches our given value:

With this configuration, if we send a webhook to https://my.webhookrelay.com/v1/webhooks/aa4d6bb5-87b6-4041-b5f4-a768338425c6?name=john, rule will extract and match parameters name=john.
While https://my.webhookrelay.com/v1/webhooks/...?name=john will be matching the rule, a webhook to https://my.webhookrelay.com/v1/webhooks/...?name=tom will not be relayed.

Regular expressions

To match based on regular expressions, select regex rule type:

With this configuration, a webhook request query will have to have a location parameter with value either london or paris: https://my.webhookrelay.com/v1/webhooks/...location=paris

You can view regex examples here: https://golang.org/pkg/regexp/syntax/

Contains or Does Not Contain

There are additional matching types such as contains and does not contain. These are general purpose, easy to use matchers.

Matching based on request body

Body matching can be achieved by:

• value based matching
• regex matching of parsed body into a JSON object
• contains matching that looks for a specified text in the body
• does not contain matching that checks whether body doesn’t have a specified word/phrase

value

Value type rule can match a parsed payload of a JSON webhook and check whether a specified parameter matches the rule value:

Only webhooks that have id = foo inside the nested ‘data’ key will be passed:

{
    "status": "xx",
    "data": {
        "id": "foo"
    }
}

Match ‘does not contain’

Regex rules can match based on specified parameters when JSON body is parsed:

With this configuration, a request with payload:

{
    "status": "xx",
    "data": {
        "id": "xxx"
    }
}

will be matched by a rule and passed through, while this one:

{
    "status": "ff",
    "data": {
        "id": "foo"
    }
}

will be rejected as it contains our unwanted value.

Match ‘contains’

Contains matching is one of the simplest examples:

Select a contains rule type and type a string that should be in the request body:

{
    "status": "ff",
    "data": {
        "id": "london"
    }
}

This payload will be relayed.

If body is not JSON, ‘contains’, ‘regex’ or ‘does not contain’ will be matching against plain text body.

Matching based on request headers

Similarly to URL query and request body matching, header rules can extract and compare values from headers:

Validating webhook SHA1 and SHA256

Rules can perform both SHA1 and SHA256 webhook payload signature validation. For that, you need to choose header as the source, set parameter to the name of the header that will have the hash, select either ‘payload SHA1’ or ‘payload SHA256’ from the menu and set your secret in the value field:

Grouping rules

Rules can be grouped using logical operators all and any which are equivalent to and and or if you are familiar with programming principles. You can have multiple levels of rules interacting with each other.

Custom domains

Webhook Relay allows using both your own domain names or registering subdomains under .hooks.webhookrelay.com.

Using your own DNS

Instead of having input endpoints such as https://my.webhookrelay.com/v1/webhooks/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX you can reserve your own subdomains either under *.hooks.webhookrelay.com domain or use your own such as hooks.example.com.

The first time you are using a custom domain for your inputs, you will need to go to domains reservation page and click on RESERVE DOMAIN. Enter your domain and click SUBMIT.

Now, go to your bucket details page and click CONFIGURE on an input. Here, select your reserved domain from a dropdown and it’s also a good practice to enter a path prefix for your input as you can have any number of inputs (even from different buckets) sharing the same domain:

Finally, go to your DNS provider and create a CNAME record for your chosen domain pointing at hooks.webhookrelay.com. In this case we are using Cloudflare to add the configuration:

When you send the first HTTP request to the newly configured subdomain, Webhook Relay will provision a certificate.

Custom subdomains under Webhook Relay domain

If you don’t have your own domain name but still want to have a custom subdomain that’s not randomly generated, you can reserve one under .hooks.webhookrelay.com. For example if I reserve one api.hooks.webhookrelay.com, I can use it in my Input settings:

and set path to /dogs so I can reuse it for multiple inputs. Now, in the response configuration let’s add some custom response such as:

[
 {
   "name": "Beethoven"
  }
]

And if you curl the endpoint (just replace the URL with your own reserved domain):

curl https://api.hooks.webhookrelay.com/dogs
[
 {
   "name": "Beethoven"
  }
]