Introducing WebSocket Server

Dec 5, 2018, by Karolis Rusenas

automation javascript relay webhooks websocket

Socket Server picture

For the past few months, our users have pointed out that they need an easier way to receive webhooks inside their applications. In short, they do not want to implement a web server just for one endpoint that does nothing but receives webhooks. We have evaluated several solutions on streaming events to clients but ended up using WebSockets. This is because WebSockets is a standard that is implemented in pretty much every programming language (another solution, which we use internally and was considered first, was NATS but it would have required too many modifications).

How do WebSockets fit into Webhook Relay?

We originally started with webhookrelayd Docker image, then added relay CLI and Kubernetes ingress controller. While these tools did fulfill the requirements most of the time, in some cases a deeper integration was desired. Here’s a higher level diagram:

Webhook Relay high-level architecture

WebSocket clients can subscribe to multiple buckets (which are like topics or subjects, if you are familiar with messaging systems) and then it’s up to the user to decide what to do with the payload. Clients get Webhook Relay configuration, request headers, and body. Socket servers are designed to be efficient with long-running connections that are created by daemon processes.

JavaScript Example

  1. Log into your account and go to buckets page
  2. Create a bucket called my-bucket
  3. Ensure that on the outputs side ‘WebSocket streaming’ is enabled

Now, let’s create our NodeJS application. The code is straightforward, WebSocket library is doing most of the heavy lifting here:

// client.js
const WebSocket = require('ws');
var server = 'wss://';
var reconnectInterval = 1000 * 3;
var ws;
var apiKey = process.env.RELAY_KEY;
var apiSecret = process.env.RELAY_SECRET;
var connect = function(){
ws = new WebSocket(server);
ws.on('open', function() {
console.log('connected, sending authentication request');
ws.send(JSON.stringify({ action: 'auth', key: apiKey, secret: apiSecret }));
ws.on('message', function incoming(data) {
// parse message and if we have authenticated, subscribe to our bucket
var msg = JSON.parse(data);
if (msg.type === 'status' && msg.status === 'authenticated') {
ws.send(JSON.stringify({ action: 'subscribe', buckets: ['my-bucket'] }));
ws.on('error', function() {
console.log('socket error');
ws.on('close', function() {
console.log('socket closed, reconnecting');
setTimeout(connect, reconnectInterval);

Here, we see that:

Clients can subscribe to multiple buckets. If a bucket list is empty ({'action': 'subscribe'}) then the client is subscribed to all buckets)

To run this application, let’s first install the websocket ws library:

npm i ws

Set token key and secret (from tokens page):

export RELAY_KEY=your-token-key
export RELAY_SECRET=your-token-secret

Start it:

node client.js

Once the authentication response is received, we send a subscribe event to start streaming. Send some webhooks to your bucket’s public input endpoint.

Client in action:

$ node client.js
connected, sending authentication request
{"type":"status","status":"authenticated","message":"connected successfully, subscribe to buckets"}
{"type":"status","status":"subscribed","message":"subscribed to buckets: my-bucket"}
{"type":"webhook","meta":{"bucked_id":"89e44c32-27ff-4832-8655-8a42d3851b6f","bucket_name":"my-bucket","input_id":"ee4ac550-12a4-41a7-837d-dd3356ed1771","input_name":"Default public endpoint","output_name":"22","output_destination":""},"headers":{"User-Agent":["insomnia/6.2.3"],"Cookie":["__cfduid=dc244a014f0b1e2965544ddb483c3fe1b1525866866"],"Content-Type":["application/json"],"Accept":["*/*"],"Content-Length":["15"]},"query":"foo=bar","body":"{\"hi\": \"there\"}","method":"POST"}
{"type":"webhook","meta":{"bucked_id":"89e44c32-27ff-4832-8655-8a42d3851b6f","bucket_name":"my-bucket","input_id":"ee4ac550-12a4-41a7-837d-dd3356ed1771","input_name":"Default public endpoint","output_name":"111","output_destination":""},"headers":{"Content-Type":["application/json"],"Accept":["*/*"],"Content-Length":["15"],"User-Agent":["insomnia/6.2.3"],"Cookie":["__cfduid=dc244a014f0b1e2965544ddb483c3fe1b1525866866"]},"query":"foo=bar","body":"{\"hi\": \"there\"}","method":"POST"}

Socket Server documentation can be found here.

So, what can you do with Webhook Relay WebSockets?

Our mission is to connect servers and services that are otherwise hard to reach. Webhook Relay provides the greatest value when you are operating in internal networks, your own computer or your servers can change IP addresses (or there are too many of them to give them unique addresses in the first place). Therefore, consider using WebSockets for:

To Sum Up

A new streaming API:

Sounds interesting? Do you need help integrating Webhook Relay into your system? Send us an email