How to keep a company brain from going stale

Most knowledge projects don’t fail at the start. The first month goes well: someone collects the documents, writes the important rules down, connects an AI to them. Six months later the return policy has changed twice, two people have left, and the AI is quoting a version nobody remembers approving.

A company brain has the same problem as a wiki, only worse, because an AI repeats a stale rule with full confidence and in your name. These are the practices that keep it right, collected from setting up our own and from what goes wrong in other teams’ setups.

Writing rules

1. One decision per rule, with a reason. “Returns are accepted within 30 days because our supplier takes them back within 45.” The reason does two jobs. It helps the AI handle the case the rule didn’t foresee, and it tells the next Owner whether the rule still makes sense when the supplier contract changes.

2. Rules, not facts. Today’s price, a customer’s address or which version is deployed belong in the systems that already hold them. The brain says what applies: who may grant a discount, how far, and when to ask. Mixing the two is the fastest way to a brain that’s out of date by Friday.

3. Write the exception next to the rule. The normal case is rarely where AI goes wrong. “Wholesale customers get 60 days” is the sentence your support AI needs, and it’s the one that’s usually missing. If you collect knowledge through interviews, ask about the last case that didn’t go the normal way.

Ownership

4. One Owner per Topic, and it’s a person. “The support team” can’t approve anything. Pick the person who would get the angry phone call if the rule were wrong. When they change jobs, handing over their Topics is part of the handover, like keys and passwords.

5. Changes go through the Owner, corrections included. When someone corrects the AI in a chat (“no, we never promise delivery dates”), that correction usually vanishes into a chat memory nobody else sees. Turn it into a proposal for the Owner instead. Once approved, every tool gets it, not just the one where the correction happened.

Keeping it consistent

6. Let conflicts surface, don’t smooth them. When two rules disagree, both Owners should see it, and neither should win by default. They can replace one rule with the other, limit one of them to a special case, merge them, or decide it wasn’t a real conflict. What they shouldn’t do is leave both in place and let the AI pick.

7. Replace rules, don’t delete them. Every rule gets a version and a date it applies from. When it changes, the old version is marked as replaced. Then you can still answer the question that comes up in every complaint: what did the AI tell this customer in March, and was that right at the time?

8. Mark what isn’t approved yet. Many rules are already in use before anyone formally approved them. Hiding that doesn’t help. Label those rules wherever they appear, including in the AI’s answer, and review them in batches. An honest “18 of 60 rules approved” beats a brain that only looks finished.

Checking that it works

9. Give important rules a test case. Write down a realistic question and what a good answer must contain. “Customer wants to return an opened item after 40 days” → the answer refers to the 30-day limit and doesn’t promise an exception. Run those cases whenever a rule changes, and every time you switch or upgrade an AI model. That’s how you find out the new model reads your rules differently before a customer does.

10. Make every answer traceable. When an answer looks wrong, you need to see which rule, in which version, the AI used. Without that trace, you’re guessing which document it found. With it, you fix one rule once and every tool gets the fix.

How to tell it’s working

You don’t need a dashboard for this. Three questions are enough:

• Do people still correct the AI on the same point more than once?
• Do two tools give the same answer to the same question?
• When a rule changed last time, how long until every AI followed it?

If the answers are “rarely”, “yes” and “the same day”, the brain is doing its job. If not, it usually comes down to one of the ten points above, most often a Topic without a real Owner.

Start small

None of this needs a big rollout. Pick one Topic, name its Owner, write the twenty rules that matter most with a reason each, and add three test cases. Connect every AI tool you use to that one source, not to copies of it. Then let it run for a month and count the corrections.

That’s the setup we build with CtxCore Brain: Owners approve, conflicts surface on their own, important rules carry test cases, and every answer shows which rules it followed. What we learned building it on ourselves is in What we learned running a company brain on ourselves.