Integrating with Meilisearch

Meilisearch is a popular search engine known for its speed and ease of use. It does not have built-in support for sponsored search listings. Thankfully sponsored product listings are Kleio's forte and Kleio also offers built-in support for Meilisearch.

How it works

The Kleio ad-server acts as a proxy between your application and Meilisearch. More concretely: if your search endpoint was previously configured as https://meili.example.org/indexes/movies/search, then you replace it with https://kleio.example.org/integrations/meilisearch/indexes/movies/search to integrate Meilisearch and Kleio.

Kleio forwards your search requests (including authentication parameters) to Meilisearch on your behalf, intercepting the results and running an auction among all the hits that have an ad associated with them. The results are then re-ordered such that winning ads are placed at the top of the search result. Subsequent impression and click tracking works just as you are used to with Kleio.

The search results that are turned into ads get additional Kleio specific metadata injected into them. For example: the search result for the movie Ariel which would normally look like this:

{
  "genres": ["Drama", "Crime", "Comedy"],
  "id": 2,
  "overview" : "Taisto Kasurinen is a Finnish coal miner whose father has just committed suicide and who is framed for a crime he did not commit. In jail, he starts to dream about leaving the country and starting a new life. He escapes from prison but things don't go as planned...",
  "poster" : "https://image.tmdb.org/t/p/w500/ojDg0PGvs6R9xYFodRct2kdI6wC.jpg",
  "release_date" : 593395200,
  "title" : "Ariel"
}

would, if it is turned into an ad, receive an extra kleio property with the all the relevant information one would also normally get through the regular auction endpoint. The result would then end up looking like this:

{
  "genres": ["Drama", "Crime", "Comedy"],
  "id": 2,
  "kleio": {
    "ad_id": "768d161b-ac4f-467e-b532-358b87ebf4ea",
    "bid" : 1,
    "metadata" : nil,
    "name" : "",
    "tracking_code" : "zk3Y7ZqHI9lFry8MH6uY_mt0ZWh0OlnEfT8K6eWwGJoAdo0WG6xPRn61MjWLh-v06nZYzayH1E9MiKweNxRPrVgAAAAAAAAAAQAAAABmYseH"
  },
  "overview" : "Taisto Kasurinen is a Finnish coal miner whose father has just committed suicide and who is framed for a crime he did not commit. In jail, he starts to dream about leaving the country and starting a new life. He escapes from prison but things don't go as planned...",
  "poster" : "https://image.tmdb.org/t/p/w500/ojDg0PGvs6R9xYFodRct2kdI6wC.jpg",
  "release_date" : 593395200,
  "title" : "Ariel"
}

The presence, or absence, of the kleio property can be used by your application to determine whether to render, and treat, a result as an ad or not.

Quickstart / Demo

If you want a quick example of how this might look in practice, please try our tech demo. Please note that this demo is not a production-ready application. Rather it's meant as a playground to see Meilisearch and Kleio working together in practice.

To follow along, you need to have Docker and Docker Compose installed on your machine.

Create a folder on your computer (you can call it meili_kleio_demo if you want). In it, copy and paste the following content into a file you call compose.yaml:

meili_kleio_demo/compose.yaml

services:
  kleio:
    image: getkleio/kleio:latest
    ports:
      - 8000:8000
    environment:
      SECRET_KEY_BASE: rlZ6+ueWM4n/cqbOHDjFTrCuL1dkzxpX1clY0ncyN2AW4bsTObj7tQv1O/Nx2n3J
      DATABASE_URL: ecto://kleio:password@postgres/kleio
      HOST: localhost
      PORT: 8000
      MEILISEARCH_ENDPOINT: http://meilisearch:7700
      MEILISEARCH_NUM_ADS: 2
      MEILISEARCH_LIMIT_INCREASE: 50
    depends_on:
      postgres:
        condition: service_healthy
    healthcheck:
      test: ["CMD-SHELL", "curl -f http://localhost:8000/up || exit 1"]
      start_period: 10s
      interval: 1s
      timeout: 1s
      retries: 10

  meilisearch_demo:
    image: getkleio/meilisearch_demo:latest
    ports:
      - 8080:8080
    environment:
      MEILISEARCH_API_KEY: aMasterKeyThatIsSufficientlyLongAndComplex
      SECRET_KEY_BASE: rlZ6+ueWM4n/cqbOHDjFTrCuL1dkzxpX1clY0ncyN2AW4bsTObj7tQv1O/Nx2n3J
      MEILISEARCH_ENDPOINT: http://meilisearch:7700
      KLEIO_ENDPOINT: http://kleio:8000
      HOST: localhost
      PORT: 8080
    depends_on:
      kleio:
        condition: service_healthy
      meilisearch:
        condition: service_healthy

  postgres:
    image: postgres:16
    ports:
      - 5432:5432
    volumes:
      - kleio-postgres-data:/var/lib/postgresql/data
    environment:
      POSTGRES_PASSWORD: password
      POSTGRES_USER: kleio
      POSTGRES_DB: kleio
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER}"]
      start_period: 10s
      interval: 1s
      timeout: 1s
      retries: 5
  
  meilisearch:
    image: getmeili/meilisearch:v1.8
    ports:
      - 7700:7700
    volumes:
      - kleio-meili-data:/meili_data
    environment:
      MEILI_MASTER_KEY: aMasterKeyThatIsSufficientlyLongAndComplex
    healthcheck:
      test: ["CMD-SHELL", "curl --fail http://localhost:7700/health || exit 1"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 3s

volumes:
  kleio-postgres-data:
  kleio-meili-data:

Running docker compose up will start all the services:

The Kleio ad server and the Postgres-database it depends on
An instance of Meilisearch
A demo application with a search interface that will populate the Meilisearch instance with the movies dataset frequently used by Meilisearch for demo purposes

Once all the services are running, you can access the demo application under http://localhost:8080. We would recommend also opening the Kleio interface in a separate window. You can access it under http://localhost:8000. In it, you can see the ads being created and impressions and clicks being tracked.

If you are interested, you can find the source code of the demo application on Github.

Configuration

Kleio requires some additional configuration for the Meilisearch integration to work. These configuration levers are exposed as environment variables.

Environment variables

Name
MEILISEARCH_ENDPOINT
Type
string
Description
This is the only required configuration. It should contain the full URL of your Meilisearch instance and must be accessible to Kleio. If your application has a search endpoint configured as https://meili.example.org/indexes/movies/search then this variable should be set to https://meili.example.org/.
Name
MEILISEARCH_NUM_ADS
Type
integer
Description
The maximum number of ads Kleio should include in the search result. If there are not enough ad candidates among the search results, the number of returned ads might be less.
Defaults to 2.
Name
MEILISEARCH_LIMIT_INCREASE
Type
integer
Description
When performing a search you typically provide a limit parameter to Meilisearch. This allows you to limit the results returned such that the user does not get overwhelmed and your application remains performant.
The lower the limit is, the lower the chance that one of the returned results also has an ad associated with it which in turn would result in unfilled ad spots and reduced ad revenues.
With this parameter, you can increase the number of results returned by Meilisearch to Kleio. If you typically only request the top 10 results but set this environmental variable to 90, then Kleio would request the 100 best results from Meilisearch (i.e. 10 + 90) and run an auction amongst all of these 100 candidates. Only the top 10 (including the ads) would be returned to your application, honoring your original desire to receive only 10 results.
Defaults to 0.
Name
MEILISEARCH_FILTER_TAG
Type
string
Description
If you also use Kleio for serving ads outside of search and want to be able to differentiate between which ads are available to search and which are not, then you can provide a tag by which the ads should be filtered using this environment variable.
For example, if you set this environment variable to search, then the auction request Kleio runs internally will include the following filter:
{ "product_ids": [the ids of the products returned by Meilisearch], "tags": ["search"] }
Defaults to an empty filter.
Name
MEILISEARCH_AUTO_TRACK_IMPRESSIONS
Type
boolean
Description
Specifies whether or not Kleio should automatically track impressions for all ads returned as part of a search.
If set to false you have to manually track impressions.
Defaults to true.