Issue

The events poller allows a customer to pro-actively retrieve events by way of polling an API for data. As the CDP is an event driven system, the events-api is reactive in getting it’s events from third-party system. After the events poller loads in it’s data, it forwards them to the events-api.

As such, the following requirements are part of it’s responsibilities:

• Being able to load in data from an API;
• Being able to add standard authorization to these API calls
• Having support for pagination;
• Being able to verify your dataset.

Product

• Mobile Marketing Cloud
• Customer Data Platform

Resolution

How to setup an event poller

In an account, start with adding an EventsPoller integration.

We will start by getting cat facts into the CDP. For that, we will use the following example API:

https://catfact.ninja/fact

While we support both HTTP and HTTPS, we do expect these HTTPS calls to have a valid and active SSL certificate behind it.

When using the “Skip SSL certification” option, you can skip this validation on the SSL certificate. This is generally discouraged and should be only be used for testing purposes. Skipping the SSL certificate by default could incur the risk of man-in-the-middle attacks on the customer data.

We currently have the following 3 options available for authentication:

• No authentication (generally used for public API’s);
• Token (generally used to access other CM.com API’s. Please refer to acquiring an API token for those services).
• Depending on the service, you will need to specify if the token needs to be part of the URL (default) or part of the Header.
• OAuth2
• We support authorization with OAuth2. The basis for OAuth2 is that one first needs to retrieve a valid token indicating access to a specific API resource. With that token, we are able to perform an authorized request to the API.

Note: We will attempt to authorize with the token where either the test of a previous run succeeded with. When the system cannot load the data, the authorization will be made invalid and a new token is attempted to retrieve. A maximum of 3 attempts to call the API is attempted per interval.

In our example, this would look as follows:

After setting up the API details, it’s time to give the event type a name. As our example is in loading catfacts, the event name would be “Catfact”.

As the system will be polling for data, we will need to give it a time between the interval it needs to do so. This heavily depends on the integration what is required.

We will not attempt to load data, while the previous attempt has not finished.

Some parties choose to rate limit the number of allowed requests. Please verify with a API owner if such restrictions apply.

There is an option for an advanced interval setup using CRON. This advanced setup is based on UTC, but will allow you to better finetune your setup. For example, it’s possible to setup a hourly setup that only executes on Monday, Tuesday and Saturday.

You will also need to setup the data container name. By default, the system will search for the result field in the result. Should the API result contain multiple result items, this will translate into multiple events sent to the events-api. Should the data not be specified in the field result field but in another field, please adjust accordingly.

In our cat fact example above, the data we need is not in the result field but in the fact field. This will result in our setup looking as follows:

Before completing the integration, you will need to test the intergration. If your API result returns an empty list and/or empty data, the issue could be in either the authentication or an incorrect data container name.

Result

The output of the intergration is that data is sent towards the events-api with the specified setting. By default, an empty event type is created where the data is sent to. Adding scripts and/or output event types and/or mappings are currently not generated. This will require more actions by our professional services department or your customer success manager to make sure that the event ends up in the profile.

Pagination

When working with restful API’s, it’s possible that loading in a specific resource may return slow and large payloads. To be able to better support such situations, an option is added to support pagination.

{  "item": [],  "total_pages": 10}