Other Resources Copy

Benchmarking & Evaluation

Empirical performance benchmarks measuring recall precision, hallucination reduction rates, query latency, and inference cost efficiency.

We continuously evaluate BetterContext across rigorous synthetic and real-world multi-repo software engineering benchmarks (including SWE-bench, Needle-in-a-Haystack, and LongEval).

Benchmark Summary

-89% Hallucinations

Measured across 5,000 multi-file coding tasks compared to raw model prompts.

99.8% Needle Recall

Tested across 100k+ token codebases without position degradation.

< 300ms P95 Latency

Real-time traversal across multi-tenant enterprise graphs exceeding 100B tokens.

Recall Latency Distribution

Real-world latency across 10,000 production queries:

Percentile	BetterContext Causal Graph	Standard Vector RAG	Graph RAG (Community)
P50 (Median)	140ms	780ms	2,400ms
P90	220ms	1,450ms	4,100ms
P95	280ms	1,980ms	6,200ms
P99	310ms	2,800ms	8,900ms

Token & Cost Efficiency

Comparing daily inference cost for an engineering team of 20 developers:

Benchmark Scenario: 20 engineers running 30 agent requests/day on Claude 3.5 Sonnet.
• Raw 200k Context Window Dumps: ~$2,400.00 / month.
• Standard Top-K Vector RAG: ~$860.00 / month (with high hallucination failure rates).
• BetterContext Synergistic Packing: ~$105.00 / month (95.6% cost reduction with highest accuracy).

Reproducing the Benchmarks

Run the open-source evaluation suite against your own repository:

# Clone and run the benchmark CLI
pip install bettercontext[eval]
bettercontext eval --dataset swe-bench-lite --model claude-3-5-sonnet

Next Steps

Was this page helpful?

ON THIS PAGE

Overview
Benchmark Summary
Latency Distribution
Token Cost Efficiency
Reproducing Benchmarks
Next Steps