Enterprise RAG That Actually Works: What Cerebras's Internal Knowledge Base Gets Right

Most 'enterprise knowledge base' demos look the same: an impressive Obsidian graph, maybe a floating 3D brain on screen. None of it helps answer an actual business question. Cerebras, a chip company that builds extremely fast inference hardware and works with some of the largest companies in the world, recently published a technical writeup of their internal knowledge system - and unlike most examples online, there isn't a single flashy graph in it. There's just a reliable data pipeline that scans Slack, the internal wiki, code repositories, and custom databases, and turns all of it into something any employee can query and get an accurate answer from. The system currently handles roughly 15,000 questions a day. Here's what's actually happening underneath it.

What RAG actually does

Ask a language model "how tall is Nick?" and you'll get a vague, hedged answer, because there are hundreds of people named Nick and the model has to guess which one you mean. But add one short line right before the question - "Nick is 6'2" - and ask again, and the model answers instantly and confidently. That's essentially all retrieval augmented generation (RAG) does: it finds the relevant piece of information in a data store and injects it before the question, before the model even attempts to answer. For a business, that means instead of asking "what did Peter say in last week's talk?" and getting a vague non-answer, the system first finds the relevant transcript and feeds it in as context - then the model just reports what's actually there.

It's not just text - it's text with metadata, like a photo with EXIF data

The genuinely interesting part is that the system doesn't just store the raw text of every message or document. Every piece of information passes through a language model that enriches it with a metadata layer: who said it, when, to whom, why it matters, what it's about. It's exactly like a digital photo saved with EXIF data - not just the pixels, but the timestamp, location, and camera settings it was taken with. That layer is what lets the system not just retrieve similar-looking text, but weight it correctly: information from the founder 30 seconds ago gets weighted far higher than information from a junior employee three years ago. Ask about a recent seminar, and the system doesn't scan every seminar equally - it favors the most recent one. That's the actual difference between a system that feels like a real "brain" and a static pile of text.

Three layers of an enterprise knowledge system

Structurally, the system breaks into three main layers: a platform for collecting and storing internal data (Slack, wiki or Confluence, code repos, custom databases - everything the business already has, just scattered), a platform for querying that data (a search that returns exactly the relevant slice, not the whole store), and an authentication and audit layer that controls who can access each source. Permission checks matter for small businesses too: private customer or employee documents must not become accessible simply because they were added to a shared knowledge base. Apply document-level permissions before retrieved content reaches the model, and keep those permissions synchronized with the source. Reference: Microsoft document-level access control.

How it actually gets built

The build process looks more like rolling out a workflow than shipping a complex software project. You hand a coding agent (Claude Code, Codex, or any model that can take actions on your computer) the technical spec, and ask it to build an ingestion pipeline for each data source - Slack, email, GitHub, anything relevant to the business. The agent connects to existing accounts, and for any source it doesn't yet have access to, it explains how to set one up (open a cloud project, complete an OAuth consent flow, generate a refresh token) - a process that takes roughly 5 to 10 minutes per source. Once the connections are in place, the agent builds a distillation step: every conversation thread or document gets processed into a question, a summary, a resolution (where relevant), and a list of the systems involved - all stored in a database with a retrieval layer that combines exact text search with embedding search.

The result that actually proves it

What turns this from a nice idea into a proven case is a simple test: the same model, the same 20 business questions - once with access to the knowledge base, once without. Without it: zero correct answers. With it: 17 out of 20 correct, and the remaining three were honest partial answers or declines ("I don't have enough information to answer confidently"), not guesses. And that's with fewer than 650 documents total - a small fraction of what a large company like Cerebras actually holds. The real lesson: what determines a knowledge system's quality isn't document count, it's how precisely and relevantly it retrieves - and from there, weighting by recency and source is what makes the difference.

The takeaway

A good enterprise knowledge system doesn't need to look impressive. There's no spinning graph or floating brain - just a reliable data pipeline, a rich metadata layer, and correct weighting by time and source. The result is that any employee in the business can ask a question and get an answer grounded in the organization's actual accumulated knowledge - rather than relying only on a model's general knowledge. Retrieval does not guarantee a correct answer: check whether the cited source actually supports the response, and define when the system should decline to answer. Planning a knowledge-base project? See our AI implementation consulting process.

Continue the workflow

Before connecting an agent to data and tools, use the Zero Trust guide to AI agent security to plan permissions, secret isolation and human controls.