Controlling Precision of Custom Ranking Metrics

Modifying the data: an example

Before

Let’s say we’re adding search to a blog. Besides Algolia’s default ranking formula, we’ve set two custom ranking attributes: pageviews first, followed by comments. Here’s what the dataset would look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[
  {
    "title": "Algolia + SeaUrchin.IO",
    "author": "Nicolas Dessaigne",
    "pageviews": 1023,
    "comments": 23
  },
  {
    "title": "Search Analytics: Gain Insights from User Search Data",
    "author": "Nicolas Baissas",
    "pageviews": 508,
    "comments": 42
  },
  {
    "title": "Algolia Vault – Bringing Physical & Digital Data Security to Search",
    "author": "Liam Boogar",
    "pageviews": 1022,
    "comments": 54
    }
]

And here’s the setting:

1
2
3
4
5
6
$index->setSettings([
  'customRanking' => [
    'desc(pageviews)',
    'desc(comments)'
  ]
]);

1
2
3
4
5
6
index.set_settings({
  customRanking: [
    'desc(pageviews)',
    'desc(comments)'
  ]
})

1
2
3
4
5
6
index.setSettings({
  customRanking: [
    'desc(pageviews)',
    'desc(comments)'
  ]
});

1
2
3
4
5
6
index.set_settings({
    'customRanking': [
        'desc(pageviews)',
        'desc(comments)'
    ]
})

1
2
3
4
5
6
index.setSettings([
  "customRanking": [
    "desc(pageviews)",
    "desc(comments)"
  ]
])

1
2
3
4
5
6
7
8
index.setSettings(
  new JSONObject().put(
    "customRanking",
    new JSONArray()
      .put("desc(pageviews)")
      .put("desc(comments)")
  )
);

1
2
3
4
5
6
7
8
9
10
11
12
13
IndexSettings settings = new IndexSettings
{
    CustomRanking = new List<string>
    {
      "desc(pageviews)",
      "desc(comments)"
    }
};

index.SetSettings(settings);

// Asynchronous
await index.SetSettingsAsync(settings);

1
2
3
4
5
6
index.setSettings(
  new IndexSettings().setCustomRanking(Arrays.asList(
    "desc(pageviews)",
    "desc(comments)"
  ))
);

1
2
3
4
5
6
res, err := index.SetSettings(search.Settings{
  CustomRanking: opt.CustomRanking(
    "desc(pageviews)",
    "desc(comments)",
  ),
})

1
2
3
4
5
6
7
8
client.execute {
  changeSettings of "myIndex" `with` IndexSettings(
    customRanking = Some(Seq(
      CustomRanking.desc("pageviews"),
      CustomRanking.desc("comments")
    ))
  )
}

1
2
3
4
5
6
7
8
val settings = settings {
    customRanking {
        +Desc("pageviews")
        +Desc("comments")
    }
}

index.setSettings(settings)

Because custom ranking applies it’s criteria sequentially, pageviews has more weight than comments. When we search for “algolia”, the “Algolia + SeaUrchin.IO” article comes before “Algolia Vault” only because it has one more unique pageview. However, this additional view is not meaningful. It doesn’t make the article more relevant. Nonetheless, the engine ignores the next custom ranking attribute because of it.

For these two records, it would be better if the article with the most comments appeared first: namely, that “Algolia Vault” article should come first because it has significantly more comments.

But changing the order of the attributes in the customRanking parameter isn’t a good solution:

• it doesn’t necessarily make sense business-wise (in our case, we generally care more about unique page views than number of comments),
• we may face the same issue with number of comments, so we’re not solving anything.

For those reasons, we recommend that you reduce precision so you can effectively leverage tie-breaking.

After

We can solve the issue by adding a rounded_pageviews attribute where we reduce the precision of pageviews, and use it for custom ranking instead. In our case, one way of doing it could be to round page views to the nearest hundred.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[
  {
    "title": "Algolia + SeaUrchin.IO",
    "author": "Nicolas Dessaigne",
    "pageviews": 1023,
    "rounded_pageviews": 1000,
    "comments": 23
  },
  {
    "title": "Search Analytics: Gain Insights from User Search Data",
    "author": "Nicolas Baissas",
    "pageviews": 508,
    "rounded_pageviews": 500,
    "comments": 42
  },
  {
    "title": "Algolia Vault – Bringing Physical & Digital Data Security to Search",
    "author": "Liam Boogar",
    "pageviews": 1022,
    "rounded_pageviews": 1000,
    "comments": 54
    }
]

Then, you would use the rounded_pageviews attribute for custom ranking instead of pageviews. Now, searching for “algolia” would return the “Algolia Vault” article first, based on the comments attribute.

1
2
3
4
5
6
$index->setSettings([
  'customRanking' => [
    'desc(rounded_pageviews)',
    'desc(comments)'
  ]
]);

1
2
3
4
5
6
index.set_settings({
  customRanking: [
    'desc(rounded_pageviews)',
    'desc(comments)'
  ]
})

1
2
3
4
5
6
index.setSettings({
  customRanking: [
    'desc(rounded_pageviews)',
    'desc(comments)'
  ]
});

1
2
3
4
5
6
index.set_settings({
    'customRanking': [
        'desc(rounded_pageviews)',
        'desc(comments)'
    ]
})

1
2
3
4
5
6
index.setSettings([
  "customRanking": [
    "desc(rounded_pageviews)",
    "desc(comments)"
  ]
])

1
2
3
4
5
6
7
8
index.setSettings(
  new JSONObject().put(
    "customRanking",
    new JSONArray()
      .put("desc(rounded_pageviews)")
      .put("desc(comments)")
  )
);

1
2
3
4
5
6
7
8
9
10
11
12
13
IndexSettings settings = new IndexSettings
{
    CustomRanking = new List<string>
    {
      "desc(rounded_pageviews))",
      "desc(comments)",
    }
};

index.SetSettings(settings);

// Asynchronous
await index.SetSettingsAsync(settings);

1
2
3
4
5
6
index.setSettings(
  new IndexSettings().setCustomRanking(Arrays.asList(
    "desc(rounded_pageviews)",
    "desc(comments)"
  ))
);

1
2
3
4
5
6
res, err := index.SetSettings(search.Settings{
  CustomRanking: opt.CustomRanking(
    "desc(rounded_pageviews)",
    "desc(comments)",
  ),
})

1
2
3
4
5
6
7
8
client.execute {
  changeSettings of "myIndex" `with` IndexSettings(
    customRanking = Some(Seq(
      CustomRanking.desc("rounded_pageviews"),
      CustomRanking.desc("comments")
    ))
  )
}

1
2
3
4
5
6
7
8
val settings = settings {
    customRanking {
        +Desc("rounded_pageviews")
        +Desc("comments")
    }
}

index.setSettings(settings)

Note that we didn’t remove pageviews from the dataset; you might want to keep it for display purposes.

To know more on how to rank per custom attributes, check our guide.