LLM wiki, second brain or AI OS: how to build one for your company

LLM wiki, second brain or AI OS: how to build one for your companyLLM wiki, second brain or AI OS: how to build one for your company

Oct 5, 2026 - 24 min

Updated Nov 4, 2026

AI Team

AI Team


Every new chat with an AI assistant starts from zero, and most of what your company knows sits where the assistant never looks: Slack threads, email, meeting notes, shared drives, and the heads of the people who were in the room. An LLM wiki is a fix a growing number of teams are trying: a knowledge base your AI writes and keeps current from your own documents and decisions, so every new chat starts from what the company already knows. You'll meet the same idea under two other names, second brain and AI OS.

Andrej Karpathy described the pattern in April 2026, in a short public note he kept abstract on purpose, and since then it has moved from personal experiments into company teams, while Claude, ChatGPT and Gemini have each shipped memory, project or skill features along the same lines. Six months of people running it for real have also shown where it holds up and where it breaks, and those lessons are the useful part for anyone deciding whether to build one.

Most teams need less of it than the tutorials suggest, and the part worth getting right is who owns the knowledge and how it gets checked, long before the question of which app it lives in. We run a version of it at Workspace for client reports, past decisions, contracts and estimates, described further down.

Why your AI needs a second brain

An assistant with no view of your company's knowledge produces work any company might have received, and your people then spend their time steering it back toward things the business already knew. Newer models haven't removed the problem, because what's missing is information the model has no way to see.

The re-explaining tax

A team without shared context pays a cost nobody invoices: each person keeps private prompts of varying quality, and the best explanation of a client or a process sits in one person's chat history, where a new colleague will never find it. Output quality ends up depending on who typed the request, and what the company collectively knows plays no part in it. We saw this in our own team before we did anything about it, when the first minutes of every task went into telling the assistant who the client was and how the project worked.

Context engineering vs prompt engineering

Prompt engineering deals with the wording of one request, and context engineering deals with what the model already knows when the request arrives: the instructions, documents, examples and history placed in front of it. The second term took off in mid-2025, when Shopify's CEO Tobi Lütke and Andrej Karpathy both argued it describes the real skill better, and Anthropic has since called it the natural progression of prompt engineering, which makes the prompt one part of a wider job.

The vendors have automated much of the mechanical side since then: assistants remember things between chats, they summarise long conversations on their own, and Anthropic's current advice is to keep instruction files shorter than you would have a year ago. What no product update decides for you is what your company's knowledge is, which version of the price list is current and what you agreed with a client last spring. An LLM wiki is the storage side of context engineering, the place where the right information lives so nobody has to assemble it by hand before every task.

Why a bigger context window isn't the fix

Models now read far more in one go than they did a year ago, enough to take in a whole project archive, so pasting everything into the chat looks like the obvious shortcut, and it falls short for two reasons. A pile of documents carries its old versions and its contradictions along, the model has no way of telling which one your team treats as current, and Anthropic's own documentation still says more context isn't automatically better, because accuracy drops as the input grows. Built-in memory closes part of the gap, though it's organised around one person or one project, while a company needs a shared record somebody has checked.

Six documents on the left, of which three relevant ones are selected and pass into a small context window

What is an LLM wiki?

An LLM wiki is a folder of plain text pages about your company, written and maintained by an AI model from the sources you give it. Andrej Karpathy published the pattern in April 2026 with a simple division of labour, "You never (or rarely) write the wiki yourself": you supply the sources and ask the questions, and the model writes the summaries, links related pages and updates them when something new arrives.

The original sources (documents, transcripts, contracts) form the first of three layers: nobody edits them, and they stay the source of truth. The wiki is the second, where the model writes a summary of each source plus a page per client, project or recurring topic and links them all together, and the third is a short instruction file such as CLAUDE.md, telling the model how the folder is organised and which rules apply. Two small files keep it readable for people and for the model, an index listing every page and a log recording every change.

Folder tree of a company wiki with raw, wiki, CLAUDE.md, index.md and log.md next to the rules written in the CLAUDE.md instruction file

Three operations keep the wiki alive, and each one is a plain-language request to the assistant. Ingest means the model reads a new source and updates every page it affects, query means it answers a question from the wiki and saves a good answer as a new page, and lint is the regular cleanup pass for contradictions, outdated claims and pages nothing links to.

Diagram of the wiki loop: a new source is ingested, a query returns an answer, good answers are filed back, and lint cleans up weekly

What separates this from search over a pile of files is what stays: search starts understanding from scratch with every question, while the wiki holds on to the synthesis, which Karpathy calls "a persistent, compounding artifact". It also removes the reason company wikis usually die, the upkeep nobody has time for, because the model does the cross-referencing and updating in one pass, and the people on the team move from writing pages to checking them.

LLM wiki, second brain, AI OS: are they the same thing?

They overlap, and they nest inside each other. Second brain is the oldest of the three names and comes from personal note-taking: one person's store of knowledge kept outside their head, traditionally organised by hand. LLM wiki is Karpathy's name for a way of building such a store where the model does the upkeep, and his note never uses the words second brain or operating system, those labels came from the people who picked the idea up.

An AI OS, short for AI operating system, is the knowledge base plus everything running on top of it: connectors to the tools you already use, skills for repeatable jobs, and scheduled runs. The same term has a second life outside this topic, where it covers anything from real operating systems to all-in-one software suites, so ask a vendor which meaning they have in mind. The newest label is company brain, or company second brain, for the same setup at team size, where permissions, ownership and review stop being optional.

Nested diagram: a second brain inside an LLM wiki, inside an AI OS that adds connectors, skills and scheduled runs

Do you need an LLM wiki? Three levels of context

Most teams don't need one on day one, and the sensible order is to get two simpler levels working first, since each level builds on the one before it.

Timeline of the three levels of context: a single chat, projects with memory and skills, and the LLM wiki

Level one is the single chat with the right material attached: who you are, who the result is for, an example of what good looks like, and the rules the result has to respect. Level two is the persistent workspace, which Claude, ChatGPT and Gemini all offer in some form by now. Projects hold your documents and standing instructions, memory carries corrections from one chat to the next (in Claude it's a list of entries you open and edit yourself), skills package a repeatable job such as a weekly report, and connectors pull live data from Slack, Gmail or your project tracker. For one person, or for one well-defined job, level two covers most of what a personal second brain used to promise.

Level three, the LLM wiki, starts paying off once several people depend on the same knowledge and it outlives any single project, which shows up when people re-explain the same client background in five different chats and project managers scroll through months of Slack to find one decision. A team of three with a tidy shared workspace doesn't need one yet, and no wiki rescues a team where nobody writes anything down. If you're still working out where AI fits in the business at all, start with the strategy before any of this.

How do you set up an LLM wiki?

The structure takes an afternoon to set up, and the habit around it takes a few months to settle:

  1. Create one folder of plain markdown files, keep its history with git or with the versioning your shared drive already has, and put it where every team member's assistant reads from. The files stay yours and stay readable by any model, which counts for more each time the tools change.
  2. Write a short instruction file, CLAUDE.md or AGENTS.md, describing the folder structure, where new knowledge goes, and two rules: every claim links to its source, and the model updates the index and the log with every change. Keep it under 200 lines, the length Anthropic recommends for instruction files, because current models follow short instruction files better than long ones.
  3. Seed a handful of pages with a named owner each: what the company does, who the customers are, how you sound, services and prices, active projects, and a decision log.
  4. Add sources and leave the originals untouched in their own folder: meeting transcripts, client emails, contracts, project documents. Drop them in by hand for the first weeks, and automate the feed with connectors once the structure has stopped changing.
  5. Run the three operations on a rhythm: ingest when a source arrives, query as part of daily work, and lint once a week.
  6. Name one owner for the whole wiki, decide who sees what before the second person joins, keep client-confidential material in its own restricted area, and have the model propose changes for a person to approve. Roll the wiki out in phases, a few weeks for yourself, then one more person, one department and only then the whole company, because a wiki everyone fills in their own way from day one turns unusable fast.

Checklist of six steps for setting up an LLM wiki, from the Markdown folder under git to naming an owner

The instruction file is a map, not a manual

An instruction file works best as a routing table: for each folder it says what's inside, when the model should open it and where new knowledge goes, and the company's knowledge itself stays out of it. Anthropic recommends keeping these files under 200 lines, since the whole file loads at the start of every session, so a long one eats context and dilutes the rules you care about.

AGENTS.md is a neutral name read by several AI agent tools, among them OpenAI Codex and Cursor, while Claude Code loads only CLAUDE.md on its own. Keep the wiki rules in AGENTS.md and put one line in CLAUDE.md, @AGENTS.md, which makes Claude Code import the same file, so every tool reads the same rules.

Once the wiki grows, the main map tells the model which folder to enter but not which file to open, so each folder gets its own index with one line per page, which the model reads before opening anything.

One wiki or several?

One wiki with a good map beats several separate ones, with clients, departments and projects as subfolders, because the answer to one question often needs knowledge from more than one of them. A separate wiki makes sense for a project you share with a client, or when part of the team mustn't see the rest, and separate repositories are the simplest form of access control in both cases.

A stable structure matters more than the perfect one, because every renamed or moved folder leaves the map out of date and sends new knowledge to the wrong place. Settle the structure before the first ingest, and if your shared drive already has a good folder layout, have the wiki mirror it with the same folder names, so a page's path in the wiki tells you where the original lives.

What six months of practice changed

Karpathy wrote the pattern for one person and kept it abstract on purpose: his note says it works well at moderate scale, around a hundred sources and a few hundred pages, and for teams it suggests keeping humans in the loop to review updates. People who've published their experience since April keep arriving at the same handful of rules, and all of them are about trust.

Every claim in the wiki points to where it came from and the original stays the authority, which matters most for contracts, prices and dates, where the wiki tells you which document holds the clause and you read the clause itself. Each fact with a habit of changing, a price list or a team roster, lives on one page with every other page linking to it, so an update happens in one place.

The wiki is a filtered layer on top of the raw data, not a copy of it: it stores conclusions, decisions, statuses, dates and amounts, while the original stays in Drive, email or the CRM, with the page pointing to it through a link or a path. The most valuable part of a wiki is what no tool holds, the reason a decision was made and what the team learned on a finished project.

A person approves what the model writes, with automatic checks backing them up, because a wrong summary nobody catches becomes the base for the pages built on top of it. Size brings the last rule: past a few hundred pages the index stops being enough and search joins it, a limit Karpathy names in the original note. Larger companies go further with a hybrid: they keep the knowledge their experts use every day in the wiki, serve rarely needed documents through ordinary search, and have an expert approve changes.

The upkeep doesn't disappear, it changes from writing pages to checking them, so plan a short review every week and treat a wiki nobody checks as a liability, since it will look tidy and still be wrong in places.

A review run on goodwill stops soon, so teams who keep their wiki current turn it into a schedule. A daily sync is a scheduled job going through email, the calendar, meeting notes and Slack, picking up only what's new or changed and updating the affected pages, including the ones the new information has made out of date, and meeting notes bring the most, since decisions get made in meetings. A weekly lint looks for broken links, orphan pages, duplicates, contradictions, stale pages and passwords pasted in by mistake, gives the wiki a health score from 0 to 100, fixes small issues itself and asks a person about bigger ones.

With the wiki in a private GitHub repository, a scheduled cloud job keeps running while your laptop is closed and opens every change as a pull request for the owner to review and approve. The rule of a person approving what the model writes becomes a step in the process, with a record of who approved what and when.

How we run our AI OS at Workspace

By the definitions above, what we run at Workspace is closer to an AI OS than to a pure wiki: a shared knowledge base with skills and connectors on top. The clearest example is the weekly client report: for MedAdapt, a Mediterranean climate adaptation project where we handle discovery and UX for PAP/RAC, a skill collects the week's Slack threads and emails, compiles the report in our fixed format and prepares the email as a draft, which the project manager reads, corrects and sends.

Past decisions are where project managers feel the difference most, because we've worked with Sailweek since 2022, and when a question comes up about what we agreed on their booking form months ago, nobody scrolls through Slack, the PM asks the assistant and gets the thread, the date and the decision. Contracts follow the rule from the previous section: drafts start from a clause library built from contracts we've signed, and when a change request arrives we look up what the signed contract says and read the clause in the original before we answer.

Wiki chat showing a question about the booking form decision and an answer with the thread, the date and the sources

Estimates lean on the same memory, since every new one starts from a catalogue of all the line items we've estimated since 2024, checked against the hours we tracked on the finished work. The comparison shows which kinds of tasks we tend to underestimate, a pattern nobody sees from inside a single project. The same setup runs outbound research for Serwizz, our CMMS for maintenance and service teams.

On top of our wiki, a dashboard refreshes every morning with active clients, open offers, contracts close to expiry and decisions waiting on someone, because markdown suits the model and is awkward for a person who needs to decide fast.

An AI assistant tends to agree with whoever is asking, so our instruction file holds one rule for every agent: act in the company's interest according to our goals and strategy, and if a request or a result contradicts them, say so plainly instead of agreeing. Practice added four trust rules:

  • The model never presents a calculated number as a figure from a source, it labels the calculation and lists the inputs. We added the rule after a project wiki showed hours multiplied by an hourly rate as if someone on the team had worked it out.
  • The model opens a cited source before repeating it, because a subagent's summary isn't evidence.
  • Contradictions stay in the wiki as content: both claims sit under their own heading, and a person decides.
  • Passwords and API keys never go into the wiki. When one arrives by email, it's replaced with [REDACTED] and the wiki records only where the original sits.

Frequently asked questions about LLM wikis

Do you need AGENTS.md or CLAUDE.md?

Claude Code needs CLAUDE.md, because Claude Code loads only CLAUDE.md on its own, while AGENTS.md is read by other tools such as OpenAI Codex and Cursor. If your team uses several tools, keep the rules in AGENTS.md and put one line in CLAUDE.md, @AGENTS.md.

How long should an instruction file be?

Anthropic recommends keeping an instruction file under 200 lines. The file holds the folder map and the rules, and the company's knowledge lives on the wiki pages.

Should a company run several second brains, per client or per department?

Most companies need one LLM wiki, with clients, departments and projects as subfolders. A separate wiki makes sense for a project you share with a client, or for knowledge part of the team mustn't see.

Should an LLM wiki store all the data from your CRM and Drive?

An LLM wiki doesn't copy your CRM or Drive, it stores conclusions, decisions, statuses, dates and amounts. The original stays in the CRM or on Drive, and the wiki page links to it.

How do you keep an LLM wiki current without manual work?

An LLM wiki stays current with a daily scheduled sync, which picks up only what's new in email, the calendar, meeting notes and Slack, and a weekly lint, which clears out contradictions, broken links and stale pages. A person reviews and approves the changes, most easily as pull requests.

How do you share an LLM wiki with your team and control access?

Roll an LLM wiki out in phases, from one person to one department to the whole company, with one owner approving changes to its structure. Separate repositories are the simplest way to control access, for example one for management and one for each project you share with a client.

Is an LLM wiki worth it for your team?

An LLM wiki is worth building once several people depend on the same knowledge and it outlives single projects, and it only stays useful while somebody owns it and checks it. Start with one folder, a handful of pages with named owners, a decision log and a weekly review, and add sources as the habit forms.

If you want help mapping where your team's knowledge sits today and what belongs in the first version, reach out and book a discovery session.

More from the blog

Looking for a long term digital partner?

Different skills, one team, focused on building reliable digital products, and becoming your go-to-partner.

28 reviews

Ivan
Roko
Ante
Luka
Toni

The team that turns your ideas into real world products.

LLM wiki, second brain or AI OS for your company | Workspace