Webhooks

Introduction

Webhooks allow you set up an endpoint in your own application to receive programmatical notifications from the Foundations Platform about changes to our customers data as it happens.

Rather than requiring you to pull information via our APIs, webhooks will push information to your endpoint. When one of those events is triggered (eg. a new offer is added), our Platform will send this notification as an HTTP POST request to the endpoint(s) you configure.

Applications that use webhooks benefit from greater efficiency, reduced costs and the ability to respond to customer driven events in real-time.

Getting started

Registering your app

The first step to start receiving webhooks is to register your app within our AppMarket. You don't need to list the application at this stage, but note that some subscription topics require that your app is granted an associated scope. Please see our developer portal documentation for more information on submitting an app.

Provide an endpoint

You must provide an endpoint to receive the payload the Platform will send to your app. The endpoint should:

• Be a publicly available valid URI that:

  • Uses the https scheme

  • Accepts POST requests

  • Accepts a request body with an application-json content type

Securing your endpoint

Cryptographic Signing

Asymmetric request signing with a public/private key pair has now been introduced and should be used over the legacy method above, which will become obsolete at some point in the future.

When a new webhook is setup in the DeveloperPortal, a public/private key pair will be generated and stored for the app that the webhook is associated with. The keys are associated to the app, rather than the webhook itself, so if you have two webhooks for the same app, they will be signed with the same key.

As each event is prepared to be sent to your endpoint, it will be signed using the private key which is stored securely and only visible to the Platform Webhooks services. Once the message reaches you, you can verify it has come from the Reapit Platform by using the public key to check it's authenticity. To do this, complete the following steps

1. Read the X-Signature header from the request. This will be in the following format: s:keyId:timestamp:signature Full example: s:98da2881-0540-48db-8ffa-45d7003f1412:1650934596:LLD5mr9ynFEvjz8dVmwO4vNmEva32ZV6TjAcPzdIDO93Jhc82EhysiQPcw9ZdlbCcCUjDsaeHZUsFEMUVKbGBg

2. Retrieve the public key for your app by using one of the following methods: A) Use the Public Key option in the DeveloperPortal to retrieve the key for your app or B) Make a call to GET https://platform.reapit.cloud/webhooks/signing/{id} where {id} is the id obtained from the X-Signature header (segment 2) Data from this endpoint will be returned in the following format:

   Copy

   {
     "keys": [
       {
         "kid": "08da3720-99e7-42c3-8531-58bdc8f32ecf",
         "crv": "Ed25519",
         "x": "ijz3_2n1gmlfhqAa2XH_5uYmcL2L2VHb1IqDbRDLBnU"
       }
     ]
   }

   Calls to this endpoint must include the Authorization header containing a valid Bearer token. It is only possible to retrieve keys associated to the calling app when accessing the key programmatically.

3. Decode the x value from the results using the Base64 URL scheme (if using method A above, this is the value displayed to you in the DeveloperPortal)

4. Combine the timestamp (segment 3) from the X-Signature header and the webhook message body (do not include a separation character)

5. Verify the signature (segment 4 of the X-Signature header) using the Ed25519 curve, using the public key and a combination of the timestamp/message body as above.

There are various packages available for many popular programming languages that allow you to easily verify signatures generated using the Ed25519 curve. Whilst this algorithm is less widely used than RSA, it is considered more secure hence it's use in the Platform. The code examples below can be used as a reference point, but should not be considered production ready code:

const express = require('express');
const crypto = require('node-forge');
const bodyParser = require('body-parser');

// Configure Express
const app = express();
const port = 5000;

// Be sure to read the raw request body, rather than
// using a JSON pre-processor/JSON.stringify()
app.use(bodyParser.raw({ inflate: true, type: 'application/json' }));

// Setup POST handler
app.post('/', function (req, res) {
    const ED25519 = crypto.pki.ed25519;

    // Read the X-Signature header
    const sigHeader = req.header("x-signature");
    const sigParts = sigHeader.split(":");
    
    // Read the signature segments
    const keyId = sigParts[1];
    const timestamp = sigParts[2];
    const signature = sigParts[3];

    // The message to verify is a concatenation of the timestamp (from the X-Signature header)
    // and the raw request body, with no delimiting characters
    const msgToVerify = `${timestamp}${req.body.toString()}`;

    // Retrieve the public key from the respective endpoint
    // or store OUTSIDE your code. This is an example only
    const publicKeyBase64 = "ijz3_2n1gmlfhqAa2XH_5uYmcL2L2VHb1IqDbRDLBnU";

    // Verify the signature
    const verified = ED25519.verify({
        message: msgToVerify,
        encoding: "utf8",
        signature: Buffer.from(signature, "base64"),
        publicKey: Buffer.from(publicKeyBase64, "base64")
    });

    if (verified) {
        console.log("Signature is valid");

        // Continue to process webhook
    }
    else {
        console.log("Signature is invalid");
    }

    res.send("Webhook processing complete");
});

// Start the server
app.listen(port, () => {
    console.log(`Now listening on port ${port}`);
}); 

using System;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.WebUtilities;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Primitives;
using Org.BouncyCastle.Crypto.Parameters;
using Org.BouncyCastle.Crypto.Signers;

namespace WebhookHandlerExample.Controllers
{
    [ApiController]
    [Route("/")]
    public class WebhookHandlerController : ControllerBase
    {
        private readonly ILogger<WebhookHandlerController> logger;

        public WebhookHandlerController(ILogger<WebhookHandlerController> logger)
        {
            this.logger = logger ?? throw new ArgumentNullException(nameof(logger));
        }

        [HttpPost]
        public async Task<IActionResult> PostHandler([FromBody] dynamic body)
        {
            if (Request.Headers.TryGetValue("X-Signature", out StringValues signatureHeader))
            {
                string[] sigParts = signatureHeader.First().Split(":");

                // Read the signature segments
                var keyId = sigParts[1];
                var timestamp = sigParts[2];
                var signature = sigParts[3];

                // The message to verify is a concatenation of the timestamp (from the X-Signature header)
                // and the raw request body, with no delimiting characters
                var msgToVerify = $"{timestamp}{body}";

                // Retrieve the public key from the respective endpoint
                // or store OUTSIDE your code. This is an example only
                var publicKeyBase64 = "ijz3_2n1gmlfhqAa2XH_5uYmcL2L2VHb1IqDbRDLBnU";

                CryptographyService cryptoService = new CryptographyService();

                bool verified = cryptoService.VerifySignature(msgToVerify, publicKeyBase64, signature);

                if (verified)
                {
                    this.logger.LogInformation("Signature is valid");

                    // Continue to process webhook
                }
                else
                {
                    this.logger.LogError("Signature is invalid");
                }

                return Ok("Webhook processing complete");
            }

            return Ok("No signature was found in the message header collection");
          
        }
    }

    internal class CryptographyService
    {
        /// <summary>
        /// Verifies the specified signature against the public key supplied
        /// </summary>
        /// <param name="message">The message used to generate the isngautre</param>
        /// <param name="encodedPublicKey">The encoded public key</param>
        /// <param name="signature">The signature to verify</param>
        /// <returns>True if the signature is valid, else false</returns>
        public bool VerifySignature(string message, string encodedPublicKey, string signature)
        {
            byte[] decodedPublicKey = WebEncoders.Base64UrlDecode(encodedPublicKey);
            byte[] messageBytes = Encoding.UTF8.GetBytes(message);
            byte[] signatureBytes = WebEncoders.Base64UrlDecode(signature);

            // Setup validator using Ed25519 curve
            var keyParams = new Ed25519PublicKeyParameters(decodedPublicKey, 0);

            var validator = new Ed25519Signer();
            validator.Init(false, keyParams);
            validator.BlockUpdate(messageBytes, 0, messageBytes.Length);

            bool isValidSignature = validator.VerifySignature(signatureBytes);

            return isValidSignature;
        }
    }
}

Managing webhooks in the user interface

We offer a user interface to allow you to manage webhooks in a simple and straightforward way. You're able to create, update and remove webhooks for all of your applications in a single place.

Creating a webhook

First, select the application that you want to create a webhook for. You'll then be given the option to 'Add New Webhook', as above. The modal below allows you to input the endpoint where information should be pushed.

Our webhooks system is designed to flexibly work with how your application is built and deployed. If you wish, you can set up a single endpoint to catch all topics for all customers. Alternatively, you may wish to set up a different webhook subscription per topic or per customer.

Subscribe to topics

As part of creating a new webhook, you need to specify which of the available topics (type of event) that your application needs to respond to. If any application makes a change to a customers data that corresponds to a topic that your app is listening for, you'll receive a notification to describe the event.

Data changes from any application will result in a notification being sent to your endpoint(s), including:

• Reapit Agency Cloud

• Reapit Property Cloud

• Any other Reapit AppMarket application

For example, if a new offer is added in our AgencyCloud CRM and your application has a webhook set up containing the offers.created topic, your endpoint will be sent a specific, descriptive payload containing the full details of the new offer.

We currently support the following topics, but this will increase over time. Please note that you will only be presented topics if your application has been assigned the associated scope.

Available topics

Webhook payload types are available from @reapit/foundations-ts-definitions see here for more info

If you have a requirement to target very specific events in a customer's system, you may wish to look at building your own event filter to build on the default topic capability.

Subscribe to customers

You must configure the customer(s) that your webhook will respond to events for. Only customers who have installed your listed application will appear here.

• Specify one or more customers to receive only event originating from those customers

• Specify 'SBOX' to listen to events triggered from our sandbox (useful for testing)

• Leave this field blank to respond to events for all customers who have installed your application. You will immediately receive events for new customers who install your application without any configuration change required. This does not include sandbox events.

Building event filters

There are several scenarios where the list of available topics results in a large number of irrelevant events being submitted to your endpoint. In response to a high frequency of requests of additional topics, or more granular filtering capabilities, it is possible to attach one or more event schema filters to your webhooks. This is an advanced feature that allows you to build on top of the default topic availability and target specific events. Please note that this is still in beta testing and not generally available

Webhook event filters are based on the JSON Schema specification and, put simply, a schema is used to validate the content of an event from our customer's systems which will only be sent to your endpoint in the event that validation is successful. This mechanism gives you the ability to define specific rules, for example if you want to target specific appointment types when listening to appointments.created and/or appointments.modified topics. An event filter is tied to a single specific topic so it is not necessary to check the topic id as part of the filter, however if can be good practice to include this, as is shown in the Starter Template. The tabs below provide some examples of event filters that can be used as a starting point.

It is possible to stack event filters to keep each individual filter quite lightweight. For example, if you wanted to build a filter that ensured that an appointment was of valuation type AND associated to one of a defined list of negotiator ids, this could be achieved by creating two separate filters (one for each rule) or a single filter with both rules applied. All filters must pass validation for the event to be sent to your endpoint

{
    "$schema": "https://json-schema.org/draft/2019-09/schema",
    "description": "Filter Starter Template",
    "type": "object",
    "properties": {
        "new": {
        },
        "topicId": {
            "type": "string",
            "oneOf": [
                {
                    "pattern": "appointments.modified"
                },
                {
                    "pattern": "appointments.created"
                }
            ]
        }
    },
    "required": [
        "topicId",
        "new"
    ]
}

{
    "$schema": "https://json-schema.org/draft/2019-09/schema",
    "description": "Valuation Appointments",
    "type": "object",
    "properties": {
        "new": {
            "type": "object",
            "properties": {
                "typeId": {
                    "type": "string",
                    "oneOf": [
                        {
                            "pattern": "VL"
                        }
                    ]
                }
            },
            "required": [
                "typeId"
            ]
        },
        "topicId": {
            "type": "string",
            "oneOf": [
                {
                    "pattern": "appointments.modified"
                },
                {
                    "pattern": "appointments.created"
                }
            ]
        }
    },
    "required": [
        "topicId"
    ]
}

{
    "$schema": "https://json-schema.org/draft/2019-09/schema",
    "description": "Instructions above 500,000",
    "type": "object",
    "properties": {
        "new": {
            "type": "object",
            "properties": {
                "selling": {
                    "type": "object",
                    "properties": {
                        "price": {
                            "type": "number",
                            "minimum": 500000
                        },
                        "status": {
                            "type": "string",
                            "oneOf": [
                                {
                                    "pattern": "^forSale.*"
                                }
                            ]
                        }
                    },
                    "required": [
                        "price",
                        "status"
                    ]
                }
            },
            "required": [
                "selling"
            ]
        },
        "diff": {
            "type": "object",
            "properties": {
                "selling": {
                    "type": "object",
                    "properties": {
                        "price": {
                            "type": "array",
                            "items": [
                                {
                                    "type": "number"
                                },
                                {
                                    "type": "number",
                                    "minimum": 500000
                                }
                            ]
                        },
                        "status": {
                            "type": "array",
                            "items": [
                                {
                                    "type": "string"
                                },
                                {
                                    "type": "string",
                                    "oneOf": [
                                        {
                                            "pattern": "^forSale.*"
                                        }
                                    ]
                                }
                            ]
                        }
                    },
                    "required": [
                        "status"
                    ]
                }
            },
            "required": [
                "selling"
            ]
        }
    },
    "anyOf": [
        {
            "required": [
                "new"
            ]
        },
        {
            "required": [
                "new",
                "diff"
            ]
        }
    ]
}

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "description": "Changes to latitude or longitude",
  "type": "object",
  "properties": {
    "diff": {
      "type": "object",
      "properties": {
        "address": {
          "type": "object",
          "properties": {
            "geolocation": {
              "type": "object",
              "properties": {
                "latitude": { "type": "array" },
                "longitude": { "type": "array" }
              },
              "anyOf": [
                { "required": [ "latitude" ] },
                { "required": [ "longitude"] }
              ]
            }
          },
          "required": [ "geolocation" ]
        }
      },
      "required": [ "address" ]
    }
  },
  "required": [ "diff" ]
}

Once your event schema has been built, post it to the appropriate resthooks endpoint. If you wish to test your filter against a given event once it has been saved, the test endpoint can be used to validate your filter against the specified event payload, which you can take from the Ping webhooks function, or by using a real event already being received by your endpoint

Including semi-structured data in webhook events

The Foundations REST APIs allow semi-structured - known to customers as extra fields - data to be optionally included in responses and this data can also now be included in webhook payloads. This can be set up through the Webhooks management section of the DeveloperPortal.

When editing a webhook, semi structured fields can be optionally include as part of the configuration. If using this facility, it's likely you will already be aware that you can access these fields, and as with the APIs it's necessary for permission to be granted to these fields before you can use them, and an appropriate error message will be displayed if you attempt to configure a webhook with fields that are not permitted. To include semi-structured fields in webhook events, simply add each field you wish to include and have access to in the appropriate section of the configuration window, and click Update

Event payloads will now contain the selected fields, in the new, old, and diff objects where applicable. Note that when a field is not set in the database, it will not be returned at all which matches the behaviour of the APIs.

Optional webhook behaviour

By default, webhooks will not be emitted when only the entity's eTag and modified timestamp has changed. If you would prefer to receive notifications in this situation, please use the Ignore notifications where only the eTag has been modified toggle option when configuring your webhook

Managing webhooks using REST API

We also provide a REST API to allow webhooks to be programmatically created for the customer/application that your access token has been issued on behalf of. Please see the swagger documentation for technical details on how to integrate.

Receiving events

Example payload

We use a consistent schema to describe any event that we broadcast in a descriptive and self-contained way. The notifications we emit indicate that an event has happened - but additionally and where appropriate - we provide details about the actual data change that has occurred. This allows your application to ascertain granular details about the event without needing additional API calls.

Below is an example of what a contacts.modified webhook event might look like

{
  "SendAttempts": 1,
  "eventId": "9e7e4181-6210-49ea-abf5-d5ce16d23647",
  "entityId": "RPT20000029",
  "customerId": "webhook-test",
  "eventTime": "2020-05-13T09:33:16.8811358Z",
  "topicId": "contacts.modified",
  "new": {
    "id": "RPT20000029",
    "created": "2020-05-13T09:32:24Z",
    "modified": "2020-05-13T09:33:10Z",
    "title": "Mr",
    "forename": "John",
    "surname": "Smith",
    "dateOfBirth": null,
    "active": true,
    "marketingConsent": "notAsked",
    "identityCheck": "unchecked",
    "source": null,
    "homePhone": null,
    "workPhone": null,
    "mobilePhone": "07123 456789",
    "email": "[email protected]",
    "primaryAddress": {
      "type": "primary",
      "buildingName": "",
      "buildingNumber": "12",
      "line1": "High Street",
      "line2": "Clacton-On-Sea",
      "line3": "Essex",
      "line4": "",
      "postcode": "CO15 1AE",
      "countryId": "GB"
    },
    "secondaryAddress": null,
    "workAddress": null,
    "officeIds": ["RPT"],
    "negotiatorIds": ["RPT"],
    "_eTag": "\"4DF107A6EB05D792EEAFDF1432F6E275\""
  },
  "old": {
    "id": "RPT20000029",
    "created": "2020-05-13T09:32:24Z",
    "modified": null,
    "title": "Mr",
    "forename": "John",
    "surname": "Smith",
    "dateOfBirth": null,
    "active": true,
    "marketingConsent": "notAsked",
    "identityCheck": "unchecked",
    "source": null,
    "homePhone": null,
    "workPhone": null,
    "mobilePhone": null,
    "email": "[email protected]",
    "primaryAddress": {
      "type": "primary",
      "buildingName": "",
      "buildingNumber": "1",
      "line1": "High Street",
      "line2": "Clacton-On-Sea",
      "line3": "Essex",
      "line4": "",
      "postcode": "CO15 1AE",
      "countryId": "GB"
    },
    "secondaryAddress": null,
    "workAddress": null,
    "officeIds": ["RPT"],
    "negotiatorIds": ["RPT"],
    "_eTag": "\"65D680519E9762519D203891A694B85B\""
  },
  "diff": {
    "modified": [null, "2020-05-13T09:33:10Z"],
    "mobilePhone": [null, "07123 456789"],
    "primaryAddress": {
      "buildingNumber": ["1", "12"]
    },
    "_eTag": ["\"65D680519E9762519D203891A694B85B\"", "\"4DF107A6EB05D792EEAFDF1432F6E275\""]
  }
}

Payload schema

The following table outlines the purpose of each property in the payload

The content of the new, old, and diff properties in the webhook event payload use the same schema that the respective API endpoints use. For example, if the payload received has a topicId of contacts.modified then these payload properties will use the same schema as the GET /contacts API endpoints

Please note that metadata specific to your app/integration is not currently included in webhook event payloads

Testing

When you have an endpoint configured, you can test it by using the Ping function. This will send a test event with an example payload to the URL stored against the webhook for the selected topic. You will only be able to select topics that are applicable to the webhook being tested.

The user interface will show a success or failure based on the response back from the endpoint. In the event of a success response, you will have received the example payload to the configured endpoint.

We also recommend testing your webhooks using our sandbox environment before applying them to customers.

Failure handling and exponential backoff

As with any integrated system, there's always a possibility of the endpoint we try to send a webhook notification to being unavailable at certain periods. For this reason a retry policy with exponential backoff has been built so that small periods of downtime do not result in messages not being delivered to you.

Where we fail to deliver a webhook on the first attempt, we will retry up to 5 times at the following intervals (6 delivery attempts in total):

The number of attempts it took to deliver the message is available in the payload. See the Example payload for more information.

Please note that we will not retry to send messages that were not delivered on the first attempt in the following scenarios:

• Where the webhook event is associated to Sandbox data

• Where the response code we received from your configured endpoint was 4XX. This is indicative of a misconfigured webhook or authentication problem on the target system

It is considered good practice to process webhook events asynchronously, meaning your endpoint should respond quickly (typically with a 202 Accepted response) before invoking any processing logic. If a response from your endpoint is not received in a timely fashion (typically around 10 seconds), the event will go into the retry queue as described above. This can inflate webhook consumption which is included in monthly invoicing and so it is in your best interest to adhere to this practice. The 10 second limit is dynamic based on throughput from our system but will not drop below 5 seconds.

Additional information

• Events are generated in near real-time and though extremely unlikely, we do not guarantee that you will only get a single notification for an event.

• We do not guarantee that webhooks will be sent in the exact order that the events occurred. You can use the eventTime to determine when each event occurred.

• Webhooks originated requests contribute to developer analytics and billing in the same way as regular API requests do