Introducing WebSocket Server

For the past few months, our users have pointed out that they need an easier way to receive webhooks inside their applications. In short, they do not want to implement a web server just for one endpoint that does nothing but receives webhooks. We have evaluated several solutions on streaming events to clients but ended up using WebSockets. This is because WebSockets is a standard that is implemented in pretty much every programming language (another solution, which we use internally and was considered first, was NATS but it would have required too many modifications).

We originally started with webhookrelayd Docker image, then added relay CLI and Kubernetes ingress controller. While these tools did fulfill the requirements most of the time, in some cases a deeper integration was desired. Here’s a higher level diagram:

WebSocket clients can subscribe to multiple buckets (which are like topics or subjects, if you are familiar with messaging systems) and then it’s up to the user to decide what to do with the payload. Clients get Webhook Relay configuration, request headers, and body. Socket servers are designed to be efficient with long-running connections that are created by daemon processes.

1. Log into your account and go to buckets page
2. Create a bucket called my-bucket
3. Ensure that on the outputs side ‘WebSocket streaming’ is enabled

Now, let’s create our NodeJS application. The code is straightforward, WebSocket library is doing most of the heavy lifting here:

// client.js
const WebSocket = require('ws');

var server = 'wss://my.webhookrelay.com/v1/socket';
var reconnectInterval = 1000 * 3;
var ws;

var apiKey = process.env.RELAY_KEY;
var apiSecret = process.env.RELAY_SECRET;

var connect = function(){
    ws = new WebSocket(server);
    ws.on('open', function() {        
        console.log('connected, sending authentication request');
        ws.send(JSON.stringify({ action: 'auth', key: apiKey, secret: apiSecret }));
    });

    ws.on('message', function incoming(data) {
      console.log(data)

      // parse message and if we have authenticated, subscribe to our bucket
      var msg = JSON.parse(data);
      if (msg.type === 'status' && msg.status === 'authenticated') {
        ws.send(JSON.stringify({ action: 'subscribe', buckets: ['my-bucket'] }));
      }
    });

    ws.on('error', function() {
        console.log('socket error');       
    });

    ws.on('close', function() {
        console.log('socket closed, reconnecting');
        setTimeout(connect, reconnectInterval);
    });
};

connect();

Here, we see that:

• Once the connection is established, our agent sends an auth request.
• Once authenticated, the agent sends a subscribe event.
• If disconnected, just reconnect.

Clients can subscribe to multiple buckets. If a bucket list is empty ({'action': 'subscribe'}) then the client is subscribed to all buckets)

To run this application, let’s first install the websocket ws library:

npm i ws

Set token key and secret (from tokens page):

export RELAY_KEY=your-token-key
export RELAY_SECRET=your-token-secret

Start it:

node client.js

Once the authentication response is received, we send a subscribe event to start streaming. Send some webhooks to your bucket’s public input endpoint.

Client in action:

$ node client.js
connected, sending authentication request
{"type":"status","status":"authenticated","message":"connected successfully, subscribe to buckets"}
{"type":"status","status":"subscribed","message":"subscribed to buckets: my-bucket"}
{"type":"webhook","meta":{"bucked_id":"89e44c32-27ff-4832-8655-8a42d3851b6f","bucket_name":"my-bucket","input_id":"ee4ac550-12a4-41a7-837d-dd3356ed1771","input_name":"Default public endpoint","output_name":"22","output_destination":"https://bin.webhookrelay.com/v1/webhooks/e9274477-c8a7-4497-8f7a-a10d7fcb6ba9"},"headers":{"User-Agent":["insomnia/6.2.3"],"Cookie":["__cfduid=dc244a014f0b1e2965544ddb483c3fe1b1525866866"],"Content-Type":["application/json"],"Accept":["*/*"],"Content-Length":["15"]},"query":"foo=bar","body":"{\"hi\": \"there\"}","method":"POST"}
{"type":"webhook","meta":{"bucked_id":"89e44c32-27ff-4832-8655-8a42d3851b6f","bucket_name":"my-bucket","input_id":"ee4ac550-12a4-41a7-837d-dd3356ed1771","input_name":"Default public endpoint","output_name":"111","output_destination":"https://bin.webhookrelay.com/v1/webhooks/a0572aa4-bb90-4638-b97c-a37654798c73"},"headers":{"Content-Type":["application/json"],"Accept":["*/*"],"Content-Length":["15"],"User-Agent":["insomnia/6.2.3"],"Cookie":["__cfduid=dc244a014f0b1e2965544ddb483c3fe1b1525866866"]},"query":"foo=bar","body":"{\"hi\": \"there\"}","method":"POST"}

Socket Server documentation can be found here.

Our mission is to connect servers and services that are otherwise hard to reach. Webhook Relay provides the greatest value when you are operating in internal networks, your own computer or your servers can change IP addresses (or there are too many of them to give them unique addresses in the first place). Therefore, consider using WebSockets for:

• Any service that needs to receive webhook notifications but doesn’t have a web server or if web server cannot be used for this purpose.
• IoT devices - by subscribing multiple devices to the same webhooks bucket, all requests will be streamed to all subscriptions.
• Node-RED - start a workflow via a WebSocket event without exposing your Node-RED server to the internet.
• Integrating IFTTT and Zapier directly into your application. Just set up a webhook action on those systems to send a request to Webhook Relay and your application will be able to process it internally.
• Process Stripe webhooks directly inside your application without running a web server.
• Easily receive webhooks from Slack slash commands and process them.
• Custom processing of webhooks that are relayed to another service. Use Webhook Relay WebSockets to subscribe to an active bucket, for example webhooks from SendGrid could be relayed to multiple marketing systems and/or internal one as well.

A new streaming API:

• Uses WebSockets as a transport mechanism to enable direct integration into your applications
• Messages are encoded in JSON format
• Clients are expected to send 2 types of messages: auth & subscribe

Sounds interesting? Do you need help integrating Webhook Relay into your system? Send us an email