Webhooks
Technical guidance for working with webhooks for real time event notifications
Introduction
Webhooks allow you set up an endpoint in your own application to receive programmatical notifications from the Foundations Platform about changes to our customers data as it happens.
Rather than requiring you to pull information via our APIs, webhooks will push information to your endpoint. When one of those events is triggered (eg. a new offer is added), our Platform will send this notification as an HTTP POST
request to the endpoint(s) you configure.
Applications that use webhooks benefit from greater efficiency, reduced costs and the ability to respond to customer driven events in real-time.
Getting started
Registering your app
The first step to start receiving webhooks is to register your app within our AppMarket. You don't need to list the application at this stage, but note that some subscription topics require that your app is granted an associated scope. Please see our developer portal documentation for more information on submitting an app.
Provide an endpoint
You must provide an endpoint to receive the payload the Platform will send to your app. The endpoint should:
Be a publicly available valid URI that:
Uses the
https
schemeAccepts
POST
requestsAccepts a request body with an
application-json
content type
You can use a third party service to quickly get to grips with webhooks. Services like RequestBin and Webhook.site instantly setup a secure URL to receive the POST
request on your behalf and provide a UI to review what data has been sent.
Securing your endpoint
Cryptographic Signing
Asymmetric request signing with a public/private key pair has now been introduced and should be used over the legacy method above, which will become obsolete at some point in the future.
When a new webhook is setup in the DeveloperPortal, a public/private key pair will be generated and stored for the app that the webhook is associated with. The keys are associated to the app, rather than the webhook itself, so if you have two webhooks for the same app, they will be signed with the same key.
As each event is prepared to be sent to your endpoint, it will be signed using the private key which is stored securely and only visible to the Platform Webhooks services. Once the message reaches you, you can verify it has come from the Reapit Platform by using the public key to check it's authenticity. To do this, complete the following steps
Read the
X-Signature
header from the request. This will be in the following format:s:keyId:timestamp:signature
Full example:s:98da2881-0540-48db-8ffa-45d7003f1412:1650934596:LLD5mr9ynFEvjz8dVmwO4vNmEva32ZV6TjAcPzdIDO93Jhc82EhysiQPcw9ZdlbCcCUjDsaeHZUsFEMUVKbGBg
Retrieve the public key for your app by using one of the following methods: A) Use the
Public Key
option in the DeveloperPortal to retrieve the key for your app or B) Make a call toGET https://platform.reapit.cloud/webhooks/signing/{id}
where {id} is the id obtained from the X-Signature header (segment 2) Data from this endpoint will be returned in the following format:Calls to this endpoint must include the Authorization header containing a valid Bearer token. It is only possible to retrieve keys associated to the calling app when accessing the key programmatically.
Decode the
x
value from the results using the Base64 URL scheme (if using method A above, this is the value displayed to you in the DeveloperPortal)Combine the timestamp (segment 3) from the
X-Signature
header and the webhook message body (do not include a separation character)Verify the signature (segment 4 of the
X-Signature
header) using the Ed25519 curve, using the public key and a combination of the timestamp/message body as above.
There are various packages available for many popular programming languages that allow you to easily verify signatures generated using the Ed25519 curve. Whilst this algorithm is less widely used than RSA, it is considered more secure hence it's use in the Platform. The code examples below can be used as a reference point, but should not be considered production ready code:
The following example uses Node JS with Express, Node Forge and Body Parser npm packages
Managing webhooks in the user interface
We offer a user interface to allow you to manage webhooks in a simple and straightforward way. You're able to create, update and remove webhooks for all of your applications in a single place.
Creating a webhook
First, select the application that you want to create a webhook for. You'll then be given the option to 'Add New Webhook', as above. The modal below allows you to input the endpoint where information should be pushed.
Our webhooks system is designed to flexibly work with how your application is built and deployed. If you wish, you can set up a single endpoint to catch all topics for all customers. Alternatively, you may wish to set up a different webhook subscription per topic or per customer.
If you are using AWS API Gateway as the entrypoint for your event handler, please note that you cannot use the AWS DNS namespace (in the format https://{api-id}.execute-api.{region}.{amazonaws}.com/) for your webhook URL due to the way some of the internal routing at Reapit is configured. To work around this, simply add a Custom Domain to your API Gateway, and use that in your webhook configuration instead
Subscribe to topics
As part of creating a new webhook, you need to specify which of the available topics (type of event) that your application needs to respond to. If any application makes a change to a customers data that corresponds to a topic that your app is listening for, you'll receive a notification to describe the event.
Data changes from any application will result in a notification being sent to your endpoint(s), including:
Any other Reapit AppMarket application
For example, if a new offer is added in our AgencyCloud CRM and your application has a webhook set up containing the offers.created topic, your endpoint will be sent a specific, descriptive payload containing the full details of the new offer.
We recommend registering your webhooks in an inactive state and using the Ping function. This allows you to test that your endpoint works as expected before opting to receive live customer updates.
We currently support the following topics, but this will increase over time. Please note that you will only be presented topics if your application has been assigned the associated scope.
Available topics
Webhook payload types are available from
@reapit/foundations-ts-definitions
see here for more info
If you have a requirement to target very specific events in a customer's system, you may wish to look at building your own event filter to build on the default topic capability.
application.install*
Occurs when a customer installs an application in the AppMarket
None
application.uninstall*
Occurs when a customer uninstalls an application in the AppMarket
None
applicants.created
applicants.read
applicants.modified
applicants.read
appointments.cancelled
appointments.read
appointments.confirmed
appointments.read
appointments.created
appointments.read
appointments.modified
appointments.read
companies.created
companies.read
companies.modified
companies.read
contacts.created
contacts.read
contacts.modified
contacts.read
contacts.optedout
contacts.read
conveyancing.modified
conveyancing.read
documents.created
documents.read
documents.modified
documents.read
enquiries.accepted
enquiries.read
enquiries.created
enquiries.read
enquiries.modified
enquiries.read
enquiries.rejected
enquiries.read
identitychecks.created
identitychecks.read
identitychecks.modified
identitychecks.read
landlords.created
landlords.read
landlords.modified
landlords.read
offers.accepted
offers.read
offers.created
offers.read
offers.modified
offers.read
offers.rejected
offers.read
offers.withdrawn
offers.read
offices.created
offices.read
offices.modified
offices.read
properties.created
properties.read
properties.modified
properties.read
properties.selling.askingpricechanged
properties.read
properties.selling.completed
properties.read
properties.selling.exchanged
properties.read
properties.selling.withdrawn
properties.read
properties.selling.instructed
properties.read
properties.selling.lostinstruction
properties.read
properties.selling.underoffer
properties.read
propertyimages.created
properties.read
propertyimages.modified
properties.read
referrals.created
referrals.read
referrals.modified
referrals.read
tenancies.created
tenancies.read
tenancies.modified
tenancies.write
vendors.created
vendors.read
vendors.modified
vendors.read
worksorders.cancelled
worksorders.read
worksorders.complete
worksorders.read
worksorders.modified
worksorders.read
worksorders.raised
worksorders.read
The application.install and application.uninstall webhook events include identifiers for your app in the payload. Please note that applicationId corresponds to the Authentication Client Id and internalAppId corresponds to the App Id displayed on your App Listing in the Developer Portal
Subscribe to customers
You must configure the customer(s) that your webhook will respond to events for. Only customers who have installed your listed application will appear here.
Specify one or more customers to receive only event originating from those customers
Specify 'SBOX' to listen to events triggered from our sandbox (useful for testing)
Leave this field blank to respond to events for all customers who have installed your application. You will immediately receive events for new customers who install your application without any configuration change required. This does not include sandbox events.
Building event filters
There are several scenarios where the list of available topics results in a large number of irrelevant events being submitted to your endpoint. In response to a high frequency of requests of additional topics, or more granular filtering capabilities, it is possible to attach one or more event schema filters to your webhooks. This is an advanced feature that allows you to build on top of the default topic availability and target specific events. Please note that this is still in beta testing and not generally available
Webhook event filters are based on the JSON Schema specification and, put simply, a schema is used to validate the content of an event from our customer's systems which will only be sent to your endpoint in the event that validation is successful. This mechanism gives you the ability to define specific rules, for example if you want to target specific appointment types when listening to appointments.created and/or appointments.modified topics. An event filter is tied to a single specific topic so it is not necessary to check the topic id as part of the filter, however if can be good practice to include this, as is shown in the Starter Template. The tabs below provide some examples of event filters that can be used as a starting point.
It is possible to stack event filters to keep each individual filter quite lightweight. For example, if you wanted to build a filter that ensured that an appointment was of valuation type AND associated to one of a defined list of negotiator ids, this could be achieved by creating two separate filters (one for each rule) or a single filter with both rules applied. All filters must pass validation for the event to be sent to your endpoint
Once your event schema has been built, post it to the appropriate resthooks endpoint. If you wish to test your filter against a given event once it has been saved, the test endpoint can be used to validate your filter against the specified event payload, which you can take from the Ping webhooks function, or by using a real event already being received by your endpoint
When using event filters, it is important to keep in mind that certain events will no longer be sent to you. As the diff is generated for each specific event (rather than being based on the last version you received as a consumer) you will likely encounter scenarios where it is necessary to use the new
object to understand all changes to a resource since the last time you received an update
Including semi-structured data in webhook events
The Foundations REST APIs allow semi-structured - known to customers as extra fields - data to be optionally included in responses and this data can also now be included in webhook payloads. This can be set up through the Webhooks management section of the DeveloperPortal.
When editing a webhook, semi structured fields can be optionally include as part of the configuration. If using this facility, it's likely you will already be aware that you can access these fields, and as with the APIs it's necessary for permission to be granted to these fields before you can use them, and an appropriate error message will be displayed if you attempt to configure a webhook with fields that are not permitted. To include semi-structured fields in webhook events, simply add each field you wish to include and have access to in the appropriate section of the configuration window, and click Update
Event payloads will now contain the selected fields, in the new
, old
, and diff
objects where applicable. Note that when a field is not set in the database, it will not be returned at all which matches the behaviour of the APIs.
When a webhook is configured for multiple customers, you must have been granted permission to the semi structured field(s) being configured for all customers. An error will be presented if this requirement is not met. In this scenario, either request access to the field for the additional customers, or consider splitting your webhook into multiple hook configurations
Optional webhook behaviour
By default, webhooks will not be emitted when only the entity's eTag and modified timestamp has changed. If you would prefer to receive notifications in this situation, please use the Ignore notifications where only the eTag has been modified toggle option when configuring your webhook
To test your webhook end to end with real data, use the sandbox database available to you in the Developer Portal. Set up a webhook to listen to 'SBOX' events and make changes to the sandbox using our APIs or Interactive API Explorer.
Managing webhooks using REST API
We also provide a REST API to allow webhooks to be programmatically created for the customer/application that your access token has been issued on behalf of. Please see the swagger documentation for technical details on how to integrate.
Receiving events
Example payload
We use a consistent schema to describe any event that we broadcast in a descriptive and self-contained way. The notifications we emit indicate that an event has happened - but additionally and where appropriate - we provide details about the actual data change that has occurred. This allows your application to ascertain granular details about the event without needing additional API calls.
Below is an example of what a contacts.modified webhook event might look like
Payload schema
The following table outlines the purpose of each property in the payload
SendAttempts
eventId
A unique identifier for the event. Some events will trigger multiple notifications (for example if the name of a contact changes, you'll also receive a notification for each of their associated roles. These will share the same event identifier)
entityId
If applicable, the unique identifier of the entity that the event is associated to. Please note that by itself this is not a globally unique identifier. Uniqueness can be achieved by also including the customerId
customerId
The unique identifier of the Reapit customer whom the data event is associated with
eventTime
The UTC date and time the event was emitted from our Platform
topicId
new
The new version of the entity, if applicable - this property will be null if a user has deleted or archived the entity.
old
The old version of the entity, if applicable - this property will be null in the event of a .created event.
diff
A diff outlining the exact changes between the new/old payloads. The diff is only populated for .modified events where both new
/old
representations of the entity are available.
The content of the new
, old
, and diff
properties in the webhook event payload use the same schema that the respective API endpoints use. For example, if the payload received has a topicId
of contacts.modified
then these payload properties will use the same schema as the GET /contacts
API endpoints
Please note that metadata specific to your app/integration is not currently included in webhook event payloads
Testing
When you have an endpoint configured, you can test it by using the Ping function. This will send a test event with an example payload to the URL stored against the webhook for the selected topic. You will only be able to select topics that are applicable to the webhook being tested.
The user interface will show a success or failure based on the response back from the endpoint. In the event of a success response, you will have received the example payload to the configured endpoint.
We also recommend testing your webhooks using our sandbox environment before applying them to customers.
All events submitted from the Ping function use example data and will have a customerId
of 'webhook-test' and an entityId
of '9e7e4181-6210-49ea-abf5-d5ce16d23647'. Receiving services need to handle this appropriately and not treat the payload as production event
Failure handling and exponential backoff
As with any integrated system, there's always a possibility of the endpoint we try to send a webhook notification to being unavailable at certain periods. For this reason a retry policy with exponential backoff has been built so that small periods of downtime do not result in messages not being delivered to you.
Where we fail to deliver a webhook on the first attempt, we will retry up to 5 times at the following intervals (6 delivery attempts in total):
2
At least 60s (1m) after the first delivery attempt
3
At least 120s (2m) after the second delivery attempt
4
At least 300s (5m) after the third delivery attempt
5
At least 600s (10m) after the fourth delivery attempt
6
At least 900s (15m) after the fifth delivery attempt. No more attempts to deliver the message will be made after this attempt
The number of attempts it took to deliver the message is available in the payload. See the Example payload for more information.
Please note that we will not retry to send messages that were not delivered on the first attempt in the following scenarios:
Where the webhook event is associated to Sandbox data
Where the response code we received from your configured endpoint was 4XX. This is indicative of a misconfigured webhook or authentication problem on the target system
It is considered good practice to process webhook events asynchronously, meaning your endpoint should respond quickly (typically with a 202 Accepted response) before invoking any processing logic. If a response from your endpoint is not received in a timely fashion (typically around 10 seconds), the event will go into the retry queue as described above. This can inflate webhook consumption which is included in monthly invoicing and so it is in your best interest to adhere to this practice. The 10 second limit is dynamic based on throughput from our system but will not drop below 5 seconds.
Additional information
Events are generated in near real-time and though extremely unlikely, we do not guarantee that you will only get a single notification for an event.
We do not guarantee that webhooks will be sent in the exact order that the events occurred. You can use the
eventTime
to determine when each event occurred.Webhooks originated requests contribute to developer analytics and billing in the same way as regular API requests do
Last updated