How Cerebras Built a 15,000-Query/Day Internal Knowledge Base (and What You Can Learn From It)

⬅️ Back to Tutorials

Cerebras published a detailed engineering post on how they built “Cerebras Knowledge” – an internal RAG system handling 15,000+ employee queries per day across Slack, code repositories, wikis, and custom databases. The whole thing took three months and is now one of their most adopted internal tools.

What makes it worth studying is not the scale – 15,000 queries is impressive but not unheard of. It is the design philosophy. Cerebras rejected the “single source of truth” approach that most companies chase and instead built a retrieval layer that meets data where it already lives.

TLDR:

Index data in place instead of migrating it to a single platform. Slack stays Slack, GitHub stays GitHub, the retrieval layer sits on top.
Use a single Postgres embeddings table as the universal query interface. Every source writes into the same schema.
Use hybrid retrieval (full-text + embeddings + IDF + age decay) fused via reciprocal rank fusion before reranking. No single scoring method is trusted alone.
Distill unstructured conversations into structured records. Slack threads get LLM-processed into questions, summaries, resolutions, and system references before embedding.
Expose retrieval primitives as MCP tools so any agent can orchestrate them. The planner/synthesis loop lives in the client, not the server.

Reference: How We Built Our Knowledge Base on the Cerebras blog (July 2026), by Isaac, Daniel, and Zenghao.

Design philosophy: meet data where it lives

Most internal tooling projects start the same way: someone proposes moving everything into one platform. A single source of truth. It rarely works because information is generated where it is ergonomic. Slack for quick discussions, GitHub for code, Confluence for docs. Each platform is optimized for its domain.

Cerebras went the other way. They built a system that extracts data from each platform directly, indexes it into a shared embeddings table, and makes everything queryable through a single interface without asking anyone to change how they work.

The core: one Postgres table

At the center is a single Postgres table that holds embeddings, raw summaries, and metadata from every source. Any data source can write into this table. Anything in the table is immediately queryable through the same search pipeline. The design is deliberately simple: define a data source, connect to it, fetch on a schedule, embed, store.

Slack: the hard part

Slack was the most important source and the hardest to get right. Raw vector search over individual messages performs poorly because information density varies enormously. A one-word reply sits next to a detailed kernel debug session, and short filler messages rank higher in cosine similarity than long technical explanations.

Cerebras layered four techniques to solve it:

Thread-level indexing. Every message gets resolved into its full thread (parent, replies, everything). The entire thread is stored as one record.
LLM distillation. Each thread gets run through an LLM that extracts a structured record: the question being asked, a short summary, the resolution, and systems or code references mentioned. These structured fields are embedded, not the raw transcript. Accuracy went up notably in testing.
Bursting. Important individual messages inside long threads still get lost. Cerebras identifies “bursts” (consecutive messages from the same author) and embeds them separately if they clear a quality threshold (rare tokens, minimum length, or social reactions).
Hybrid retrieval. Every query runs against four signals simultaneously: full-text search (for exact token matches like error strings), embedding search (for paraphrase), inverse document frequency (to boost rare tokens over filler), and age decay (because Slack answers expire). These are fused with reciprocal rank fusion before a reranker scores them.

Code repositories with CocoIndex

Cerebras initially debated whether code embeddings were worth it. grep is powerful. After reading Cursor’s research on semantic search in codebases, they decided to try.

They use CocoIndex, an open-source framework for code embedding. Language-specific regex chunking splits files at class boundaries first, falling back to method boundaries. Each chunk gets embedded and stored. On every commit, only changed chunks are re-embedded. Some of their repos are over 40 GB.

Query pipeline: planner, executor, synthesis

Every query runs through three stages:

Planner: A lightweight LLM inspects the query and active project, then decides which tools to invoke. Available tools include search (unified vector), search_slack, search_code (ripgrep), who_knows, recent_prs, and subsystem_index.
Executor: Tool calls fan out in parallel. Results are normalized into a shared evidence schema with scores, recency, and source hints.
Synthesis: A final LLM pass produces the answer with citations and cross-source synthesis.

Two interfaces: MCP and Web UI

The same retrieval primitives are exposed as MCP tools for agent use. Each tool corresponds to one retrieval primitive: narrow, structured, and LLM-free. Claude Code or any MCP client becomes the orchestration engine.

The Web UI runs the same planner-executor-synthesis pipeline end to end. From the user’s perspective, it is simply “ask a question and get an answer.”

Projects for scoped search

Teams introduced projects as a way to bundle data sources. A compiler engineer does not want infrastructure runbooks in their results. Projects are lightweight bundles: the same Slack channel or repo can belong to multiple projects. New hires pick a default project during onboarding so they get relevant answers from day one.

Why read the full post

The Cerebras post covers the hard parts that most RAG write-ups skip: why raw vector search fails on Slack, how to set RRF parameters, when to use thread-level versus burst-level embeddings, and how to design MCP tools that stay simple on the server side. These are concrete decisions you will face building any internal knowledge base.