Embeddings and Vector Search · Retrieval · lesson 8 of 8

Hybrid retrieval and reciprocal rank fusion

about 20 minutes · free · runs in your browser

Step 1 of 2

Adding the scores does not work

Two rankings, one answer. The obvious move — add the scores — fails immediately, because cosine similarity lives in −1..1 and BM25 is unbounded. Normalising them into the same range sounds like the fix and is not: the ranges shift with every query, so the weighting silently changes from one search to the next.

Reciprocal rank fusion sidesteps the problem by throwing the scores away and keeping only the positions:

              1
RRF(doc) = Σ ------
             k + rank

A document ranked first by either method scores 1/61; ranked tenth, 1/70. Appearing in both lists beats appearing at the top of one, which is exactly the behaviour wanted: agreement between two different methods is stronger evidence than enthusiasm from one.

k = 60 is the constant from the original paper and is used nearly everywhere. It damps the difference between rank 1 and rank 2, so a narrow win in one ranking does not dominate.

Your turn: write rrf(rankings, k=60) taking a list of rankings — each a list of document ids, best first — and returning ids sorted by fused score, best first.

You start from this, and edit it in the browser:

def rrf(rankings, k=60):
    """Fuse several rankings of ids into one, best first."""
    return []

Step 2 of 2

The whole retriever

Put it together: run both searches, fuse the rankings, return the top few. This is the retrieval half of a RAG system, and the next course builds the generation half on top of exactly this function.

Your turn: write hybrid_search(query, documents, k=2).

Semantic ranking: cosine similarity between the embedded query and each embedded document, best first.
Keyword ranking: BM25 over the lowercased words, best first, dropping documents that score zero — a non-match should not be ranked at all, because RRF rewards presence in a list.
Fuse with rrf and return the top k documents as text.

You start from this, and edit it in the browser:

import math

import fake_embeddings


def rrf(rankings, k=60):
    scores = {}
    for ranking in rankings:
        for position, doc_id in enumerate(ranking):
            scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + position + )
     (scores, key= doc_id: scores[doc_id], reverse=)


 ():
      documents:
         
    average = ((d)  d  documents) / (documents)
    score = 
     term  query_terms:
        tf = document.count(term)
         tf == :
            
        matching = (  d  documents  term  d)
        idf = math.log( + ((documents) - matching + ) / (matching + ))
        score += idf * (tf * (k1 + )) / (tf + k1 * ( - b + b * (document) / average))
     score


 ():
    
     []