Skip to main content

​
Rerank with MMR (Maximal Marginal Relevance)  

RerankMMR is a diversification technique that balances relevance with diversity to reduce redundancy in search results. It iteratively selects results that are both relevant to the query and dissimilar to already-selected results, ensuring users see varied content instead of near-duplicates. 
::RerankMMR(lambda: 0.7)                           // Cosine distance (default)
::RerankMMR(lambda: 0.5, distance: "euclidean")    // Euclidean distance
::RerankMMR(lambda: 0.6, distance: "dotproduct")   // Dot product distance
When using the SDKs or curling the endpoint, the query name must match what is defined in the queries.hx file exactly.

​
How It Works

MMR uses an iterative selection process with the following formula: 
MMR = λ × Sim1(d, q) - (1-λ) × max(Sim2(d, d_i))
Where:
  • d is a candidate document
  • q is the query
  • d_i are already-selected documents
  • λ (lambda) controls the relevance vs. diversity trade-off
  • Sim1 measures relevance to the query
  • Sim2 measures similarity to selected documents
The algorithm works by:
  1. Starting with the most relevant result
  2. For each subsequent position, calculating MMR scores for remaining candidates
  3. Selecting the candidate with the highest MMR score (balancing relevance and novelty)
  4. Repeating until all positions are filled

​
When to Use RerankMMR

Use RerankMMR when you want to:
  • Diversify search results: Eliminate near-duplicate content and show varied perspectives
  • Reduce redundancy: Avoid showing multiple similar articles, products, or documents
  • Improve user experience: Provide comprehensive coverage rather than repetitive results
  • Balance exploration and relevance: Give users both highly relevant and exploratory options

​
Parameters

​
lambda (required)

A value between 0.0 and 1.0 that controls the relevance vs. diversity trade-off:
  • Higher values (0.7-1.0): Favor relevance over diversity
    • Results will be more similar to original ranking
    • Use when relevance is paramount
    • Minimal diversification
  • Lower values (0.0-0.3): Favor diversity over relevance
    • Results will be maximally varied
    • Use when avoiding redundancy is critical
    • May include less relevant but diverse results
  • Balanced values (0.4-0.6): Balance relevance and diversity
    • Good compromise for most use cases
    • Maintains reasonable relevance while reducing duplicates
  • Typical recommendation: Start with 0.5-0.7 for most applications
The optimal lambda value depends on your specific use case. Test different values to find what works best for your users.

​
distance (optional, default: “cosine”)

The distance metric used for calculating similarity:
  • “cosine”: Cosine similarity (default)
    • Works well for normalized vectors
    • Common choice for text embeddings
    • Range: -1 to 1
  • “euclidean”: Euclidean distance
    • Works well for absolute distances
    • Sensitive to vector magnitude
    • Good for spatial data
  • “dotproduct”: Dot product similarity
    • Works well for unnormalized vectors
    • Computationally efficient
    • Considers vector magnitude

​
Example 1: Basic diversification with default cosine distance

QUERY DiverseSearch(query_vec: [F64]) =>
    results <- SearchV<Document>(query_vec, 100)
        ::RerankMMR(lambda: 0.7)
        ::RANGE(0, 10)
    RETURN results

QUERY InsertDocument(vector: [F64], title: String, category: String) =>
    document <- AddV<Document>(vector, {
        title: title,
        category: category
    })
    RETURN document
Here’s how to run the query using the SDKs or curl 
from helix.client import Client

client = Client(local=True, port=6969)

documents = [
    {"vector": [0.1, 0.2, 0.3, 0.4], "title": "Introduction to Python", "category": "Programming"},
    {"vector": [0.11, 0.21, 0.31, 0.41], "title": "Python Basics", "category": "Programming"},
    {"vector": [0.5, 0.6, 0.7, 0.8], "title": "Machine Learning Overview", "category": "AI"},
    {"vector": [0.51, 0.61, 0.71, 0.81], "title": "Deep Learning Intro", "category": "AI"},
    {"vector": [0.3, 0.4, 0.5, 0.6], "title": "Data Structures", "category": "Computer Science"},
]

for doc in documents:
    client.query("InsertDocument", doc)

query_vector = [0.12, 0.22, 0.32, 0.42]
results = client.query("DiverseSearch", {"query_vec": query_vector})
print("Diverse search results:", results)

​
Example 2: High diversity with euclidean distance

QUERY HighDiversitySearch(query_vec: [F64]) =>
    results <- SearchV<Document>(query_vec, 100)
        ::RerankMMR(lambda: 0.3, distance: "euclidean")
        ::RANGE(0, 15)
    RETURN results

QUERY InsertDocument(vector: [F64], title: String, category: String) =>
    document <- AddV<Document>(vector, {
        title: title,
        category: category
    })
    RETURN document
Here’s how to run the query using the SDKs or curl 
from helix.client import Client

client = Client(local=True, port=6969)

documents = [
    {"vector": [0.1, 0.2, 0.3, 0.4], "title": "React Tutorial", "category": "Frontend"},
    {"vector": [0.12, 0.22, 0.32, 0.42], "title": "React Hooks", "category": "Frontend"},
    {"vector": [0.5, 0.6, 0.7, 0.8], "title": "Node.js Guide", "category": "Backend"},
    {"vector": [0.2, 0.3, 0.4, 0.5], "title": "Vue.js Basics", "category": "Frontend"},
]

for doc in documents:
    client.query("InsertDocument", doc)

query_vector = [0.11, 0.21, 0.31, 0.41]
results = client.query("HighDiversitySearch", {"query_vec": query_vector})
print("High diversity results:", results)

​
Example 3: Balanced approach with dot product

QUERY BalancedSearch(query_vec: [F64]) =>
    results <- SearchV<Document>(query_vec, 100)
        ::RerankMMR(lambda: 0.5, distance: "dotproduct")
        ::RANGE(0, 10)
    RETURN results

QUERY InsertDocument(vector: [F64], title: String, category: String) =>
    document <- AddV<Document>(vector, {
        title: title,
        category: category
    })
    RETURN document
Here’s how to run the query using the SDKs or curl 
from helix.client import Client

client = Client(local=True, port=6969)

documents = [
    {"vector": [0.1, 0.2, 0.3, 0.4], "title": "Database Design", "category": "Architecture"},
    {"vector": [0.11, 0.21, 0.31, 0.41], "title": "SQL Optimization", "category": "Database"},
    {"vector": [0.5, 0.6, 0.7, 0.8], "title": "NoSQL Systems", "category": "Database"},
    {"vector": [0.3, 0.4, 0.5, 0.6], "title": "API Design", "category": "Architecture"},
]

for doc in documents:
    client.query("InsertDocument", doc)

query_vector = [0.12, 0.22, 0.32, 0.42]
results = client.query("BalancedSearch", {"query_vec": query_vector})
print("Balanced search results:", results)

​
Example 4: Dynamic lambda for user-controlled diversity

QUERY CustomDiversitySearch(query_vec: [F64], diversity: F64) =>
    results <- SearchV<Document>(query_vec, 100)
        ::RerankMMR(lambda: diversity)
        ::RANGE(0, 10)
    RETURN results

QUERY InsertDocument(vector: [F64], title: String, category: String) =>
    document <- AddV<Document>(vector, {
        title: title,
        category: category
    })
    RETURN document
Here’s how to run the query using the SDKs or curl 
from helix.client import Client

client = Client(local=True, port=6969)

documents = [
    {"vector": [0.1, 0.2, 0.3, 0.4], "title": "Cloud Computing", "category": "Infrastructure"},
    {"vector": [0.11, 0.21, 0.31, 0.41], "title": "AWS Services", "category": "Cloud"},
    {"vector": [0.5, 0.6, 0.7, 0.8], "title": "Docker Containers", "category": "DevOps"},
    {"vector": [0.3, 0.4, 0.5, 0.6], "title": "Kubernetes", "category": "Orchestration"},
]

for doc in documents:
    client.query("InsertDocument", doc)

query_vector = [0.12, 0.22, 0.32, 0.42]

# Try different diversity levels
for diversity_level in [0.3, 0.5, 0.7, 0.9]:
    results = client.query("CustomDiversitySearch", {
        "query_vec": query_vector,
        "diversity": diversity_level
    })
    print(f"Results with diversity={diversity_level}:", results)

​
Chaining with RerankRRF

Combine RRF and MMR for hybrid search with diversification: 
QUERY HybridDiverseSearch(query_vec: [F64]) =>
    results <- SearchV<Document>(query_vec, 150)
        ::RerankRRF(k: 60)           // First: combine rankings
        ::RerankMMR(lambda: 0.6)      // Then: diversify
        ::RANGE(0, 10)
    RETURN results

QUERY InsertDocument(vector: [F64], title: String, category: String) =>
    document <- AddV<Document>(vector, {
        title: title,
        category: category
    })
    RETURN document

​
Best Practices

​
Retrieve Sufficient Candidates

MMR works best with a larger pool of candidates: 
// Good: Fetch 100-200, return 10-20
SearchV<Document>(vec, 100)::RerankMMR(lambda: 0.7)::RANGE(0, 10)

// Not ideal: Fetch 10, return 10
SearchV<Document>(vec, 10)::RerankMMR(lambda: 0.7)::RANGE(0, 10)

​
Lambda Parameter Tuning

Start with these guidelines and adjust based on your results:
  • News/Articles: 0.5-0.6 (balance coverage and relevance)
  • E-commerce: 0.6-0.7 (favor relevance, some variety)
  • Content Discovery: 0.3-0.5 (favor diversity)
  • FAQ/Support: 0.7-0.9 (favor relevance)

​
Choosing Distance Metrics

  • Text embeddings: Use “cosine” (default)
  • Image embeddings: Use “cosine” or “euclidean”
  • Custom vectors: Test all three and compare results

​
Combining with Filtering

Apply filters before reranking for better performance: 
QUERY FilteredDiverseSearch(query_vec: [F64], category: String) =>
    results <- SearchV<Document>(query_vec, 200)
        ::WHERE(_::{category}::EQ(category))  // Filter first
        ::RerankMMR(lambda: 0.6)               // Then diversify
        ::RANGE(0, 15)
    RETURN results

​
Performance Considerations

MMR has O(n²) complexity due to pairwise comparisons. For optimal performance:
  • Limit the candidate set (e.g., 100-200 results)
  • Consider caching for frequently-accessed queries
  • Monitor query latency and adjust candidate size accordingly
Key performance factors:
  • Candidate set size: Larger sets increase computation time quadratically
  • Distance metric: Dot product is fastest, euclidean is slowest
  • Result count: More results require more iterations

​
Troubleshooting

​
Results seem too similar

Problem: Results still show near-duplicates Solutions:
  • Lower lambda value (try 0.3-0.5)
  • Increase candidate pool size
  • Verify your vectors capture meaningful differences

​
Results seem irrelevant

Problem: Diverse results but poor relevance Solutions:
  • Increase lambda value (try 0.7-0.9)
  • Ensure initial search retrieves quality candidates
  • Consider using RRF before MMR

​
Performance issues

Problem: Queries are too slow Solutions:
  • Reduce candidate pool size
  • Use dot product distance metric
  • Return fewer final results
  • Consider caching popular queries

​
Use Cases

​
News and Media

Diversify news articles to show varied perspectives: 
SearchV<Article>(query_vec, 100)::RerankMMR(lambda: 0.5)::RANGE(0, 10)

​
E-commerce

Show varied product categories while maintaining relevance: 
SearchV<Product>(query_vec, 150)::RerankMMR(lambda: 0.65)::RANGE(0, 20)

​
Content Discovery

Maximize exploration with diverse recommendations: 
SearchV<Content>(user_vec, 200)::RerankMMR(lambda: 0.4, distance: "euclidean")::RANGE(0, 15)

​
Document Retrieval

Balance comprehensive coverage with relevance: 
SearchV<Document>(query_vec, 100)::RerankMMR(lambda: 0.6)::RANGE(0, 10)