Using RethinkDB change feeds to make indexing Algolia a breeze

Robert Lancer
Algolia Stories
Published in
2 min readOct 27, 2016

--

Get the code https://github.com/rlancer/AlgoliaRethinkDB

At Collaborizm we use search all over our app.

Initially we used pure ReQL to power our search features, but as we grew, this became problematic as RethinkDB does not support proper indexing for search. After evaluating a few technologies, we went with Algolia. Luckily, implementing Algolia with RethinkDB was a breeze.

Search on Collaborizm delivered by Algolia

This guide should provide the basic structure needed to hook up Algolia with RethinkDB. Leveraging realtime change feeds enables you to forget about wiring up all the places where data is changed in your app. Instead, you simply let the change feed tell you when data is changed and you send it over to Algolia.

Running the code
Requires Node.js v4.x or higher

From the terminal

If that all worked visit http://localhost:3006 to launch the UI

You should see

Click to add 100 random names to the DB, have the node console open to see the changefeed in action.

Click a search result to inspect it / delete it, have the node console open to see the changefeed handle deletes.

Important parts of the code

Special considerations

Since it’s impossible to ensure that an update is properly indexed 100% of the time, it’s important to provide a mechanism to resync any data that might have failed to sync via the changefeed.

In our app we do this via a Cron, which syncs recently modified data. This does lead to multi Algolia operations per an update, but it’s the only reliable way to insure that data is properly synced even in edge case failure scenarios.

Be sure to not to reindex too much data too frequently or else it could eat into your Algolia operations.

--

--