Batch Processing - Send Current Or Older Events in Batches

Sending events can be done on the spot or in batches, over time. Deciding when to send events depends on your implementation. Here are some use cases that may require batching.

Generally speaking, there are some situations where you cannot send events every time they occur. In this case, you’ll collect events over a period of time and send them in batches later. Other situations include sending historical events.

Insights events (click, conversion, view) used for analytics and/or personalization do not take immediate effect. The delay can range from 10 to 60 minutes depending on how long after the search they are sent. For precise times, see our page on when Insights events take effect.

Batch-Processing

There are two main use cases for this tutorial.

Use Case 1 - historical data

You’ve implemented event-sending. Unfortunately, on day one, you have no events, and so it’s going to take time before you can start personalizing your results.

If you have historical transaction data that can be useful in building your personalization data, this tutorial provide guidelines on how to upload that data into our personalization system.

Use Case 2 - collect and batch

You may want to simply collect events from your front end without immediately sending them to Algolia. This is often done to avoid too much traffic. You collect events during the day and send them all in batches later. Another reason this may be done is to run the events through some sort of algorithm or filtering process before sending the batch.

Other popular use cases for batch processing

• A sale outside of your web app which is still traceable to a certain user; for example, sales in your physical bookshop are still traceable to a customer linked to an account. You might want to collect all physical sales and send them to Algolia in one batch.
• A conversion for which no search has taken place. For example, when a user directly lands on a page through Google and directly buys the book. This can be done on the spot, or batched.
• Using your google and other analytics to populate personalization data. This can be done once with historical Google Analytics. You can also collect Google Analytics over the course of time and send them in batches.

How to send batched events

All external events can be sent using any one of our API Clients. All you need is the unique user ID and the objectID.

In the following snippets, we handle a specific scenario: an online bookshop that wants to implement personalization. To achieve that, we will send all historical data we have for each user. In this example, we will send two events - a click and a conversion event - for a user which has the id user-123456 in our system. This user bought the latest Harry Potter book some years ago, which has an objectID of 9780545139700 (the ISBN-13 code for this book).

Click Events

For additional information on sending click events, please refer to the ClickedObjectIds API Ref.

1
2
3
4
5
6
7
8
9
10
$insights = Algolia\AlgoliaSearch\InsightsClient::create(
  'YourApplicationID',
  'YourSearchOnlyAPIKey'
);

$insights->user("user-123456")->clickedObjectIDs(
  'book_click_on_search_page',
  'my_online_bookshop_index',
  ['9780545139700']
);

1
2
3
4
5
6
7
8
9
10
insights = Algolia::Insights.new(
  'YourApplicationID',
  'YourSearchOnlyAPIKey'
)

insights.user('user-123456').clicked_object_ids(
  'book_click_on_search_page',
  'my_online_bookshop_index',
  ['9780545139700']
)

1
2
3
4
5
6
7
8
9
10
// This requires installing the search-insights separate library:
// https://github.com/algolia/search-insights.js
// https://www.npmjs.com/package/search-insights

aa('clickedObjectIDs', {
    userToken: 'user-123456',
    index: 'my_online_bookshop_index',
    eventName: "book_click_on_search_page',
    objectIDs: ['9780545139700'],
});

1
2
3
4
5
6
7
insights = client.init_insights_client().user('user-123456')

insights.clicked_object_ids(
  'book_click_on_search_page',
  'my_online_bookshop_index',
  ['9780545139700']
)

1
2
3
4
5
6
7
8
9
10
11
Insights.register(
  appId: "YourApplicationID",
  apiKey: "YourSearchOnlyAPIKey",
  userToken: "user-123456"
)

Insights.shared?.clicked(
  eventName: "book_click_on_search_page",
  indexName: "my_online_bookshop_index",
  objectIDs: ["9780545139700"]
)

1
2
3
4
5
6
7
8
9
10
11
12
Insights.register(
  context,
  "YourApplicationID",
  "YourSearchOnlyAPIKey",
  "user-123456"
)

Insights.shared?.clicked(
  "book_click_on_search_page",
  "my_online_bookshop_index",
  "cba8245617aeace44"
)

1
2
3
4
5
6
7
8
9
10
var insights = new InsightsClient(
  "YourApplicationID",
  "YourSearchOnlyAPIKey"
).User("user-123456");

insights.ClickedObjectIDs(
  "book_click_on_search_page",
  "my_online_bookshop_index",
  new List<string> { "9780545139700" }
);

1
2
3
4
5
6
7
8
9
10
11
AsyncUserInsightsClient insights = new AsyncInsightsClient(
  "YourApplicationID",
  "YourSearchOnlyAPIKey",
  client
).user("user-123456");

insights.clickedObjectIDs(
  "book_click_on_search_page",
  "my_online_bookshop_index",
  Arrays.asList("9780545139700")
);

1
2
3
4
5
6
7
8
9
10
client := insights.NewClient(
	"YourApplicationID",
	"YourSearchOnlyAPIKey",
).User("user-123456")

res, err := client.ClickedObjectIDs(
	"book_click_on_search_page",
	"my_only_bookshop_index",
	[]string{"9780545139700"},
)

1
2
3
4
5
6
7
8
client.execute {
  send event ClickedObjectIDs(
    "user-123456",
    "book_click_on_search_page",
    "my_online_bookshop_index",
    Seq("9780545139700")
  )
}

Conversion Events

For additional information on sending conversion events, please refer to the ConvertedObjectIds API Ref.

1
2
3
4
5
6
7
8
9
10
$insights = Algolia\AlgoliaSearch\InsightsClient::create(
  'YourApplicationID',
  'YourSearchOnlyAPIKey'
);

$insights->user("user-123456")->convertedObjectIDs(
  'book_favorite_on_search_page',
  'my_online_bookshop_index',
  ['9780545139700']
);

1
2
3
4
5
6
7
8
9
10
insights = Algolia::Insights.new(
  'YourApplicationID',
  'YourSearchOnlyAPIKey'
)

insights.user('user-123456').converted_object_ids(
  'book_favorite_on_search_page',
  'my_online_bookshop_index',
  ['9780545139700']
)

1
2
3
4
5
6
7
8
9
10
// This requires installing the search-insights separate library:
// https://github.com/algolia/search-insights.js
// https://www.npmjs.com/package/search-insights

aa('convertedObjectIDs', {
    userToken: 'user-123456',
    index: 'my_online_bookshop_index',
    eventName: "book_click_on_search_page',
    objectIDs: ['9780545139700'],
});

1
2
3
4
5
6
7
insights = client.init_insights_client().user('user-123456')

insights.converted_object_ids(
  'book_favorite_on_search_page',
  'my_online_bookshop_index',
  ['9780545139700']
)

1
2
3
4
5
6
7
8
9
10
11
Insights.register(
  appId: "YourApplicationID",
  apiKey: "YourSearchOnlyAPIKey",
  userToken: "user-123456"
)

Insights.shared?.converted(
  eventName: "book_favorite_on_search_page",
  indexName: "my_online_bookshop_index",
  objectIDs: ["9780545139700"]
)

1
2
3
4
5
6
7
8
9
10
11
12
Insights.register(
  context,
  "YourApplicationID",
  "YourSearchOnlyAPIKey",
  "user-123456"
)

Insights.shared?.converted(
  "book_favorite_on_search_page",
  "my_online_bookshop_index",
  "cba8245617aeace44"
)

1
2
3
4
5
6
7
8
9
10
var insights = new InsightsClient(
  "YourApplicationID",
  "YourSearchOnlyAPIKey"
).User("user-123456");

insights.ConvertedObjectIDs(
  "book_favorite_on_search_page",
  "my_online_bookshop_index",
  new List<string> { "9780545139700" }
);

1
2
3
4
5
6
7
8
9
10
11
AsyncUserInsightsClient insights = new AsyncInsightsClient(
  "YourApplicationID",
  "YourSearchOnlyAPIKey",
  client
).user("user-123456");

insights.convertedObjectIDs(
  "book_favorite_on_search_page",
  "my_online_bookshop_index",
  Arrays.asList("9780545139700")
);

1
2
3
4
5
6
7
8
9
10
client := insights.NewClient(
	"YourApplicationID",
	"YourSearchOnlyAPIKey",
).User("user-123456")

res, err := client.ConvertedObjectIDs(
	"book_favorite_on_search_page",
	"my_only_bookshop_index",
	[]string{"9780545139700"},
)

1
2
3
4
5
6
7
8
client.execute {
  send event ConvertedObjectIDs(
    "user-123456",
    "book_favorite_on_search_page",
    "my_online_bookshop_index",
    Seq("9780545139700")
  )
}

Note: You can do the same with all other events - for example, ViewedObjectIds, ClickedFilters, ConvertedFilters, and ViewedFilters.