Too Many CLAUDE.md and Skill Files? 4 Fixes for Agent Memory Bloat

Too Many CLAUDE.md and Skill Files? 4 Fixes for Agent Memory Bloat

You don't have a memory problem. You have 200 markdown files.

It starts as good hygiene. You write a CLAUDE.md so the agent stops reaching for the wrong package manager. Then a skill for the release process. Then one for the migration runbook, one for the API conventions, one for how your team writes commit messages. Add an AGENTS.md when a second tool joins, a .cursor/rules file when someone switches editors, a SKILL.md per capability. Six months later there are 187 markdown files across four folders, three of them contradict each other, and nobody on the team can tell you which one is authoritative.

The agent can't either. That's the part that hurts.

The symptoms are specific

A memory folder that has outgrown itself doesn't fail loudly. It degrades in ways that look like model problems:

• The agent ignores rules you know it read. Instructions in the middle of a large context get less attention than instructions at the edges. Your invariant is sitting at position 40,000, quietly not mattering.

• The wrong skill fires. When forty skill descriptions all start with "Use this when working on...", the selection step is a coin flip between plausible neighbors.

• Two files disagree and the agent picks one. Not the newer one. Not the better one. Whichever it saw last.

• Every session opens with a tax. Before the first token of real work, the preamble is already loaded.

That last one is measurable, and the number is usually worse than people expect. Sixty memory files averaging 800 words each is roughly 64,000 tokens — about a third of a 200k window spent before anyone has asked the agent to do anything. Two hundred skill descriptions at ~60 tokens of frontmatter apiece adds another 12,000, and that's the index, not the content.

You are paying that on every session, for every developer, whether or not the task touches any of it.

There are four ways out. The first three are cleanup. The fourth is an architecture change.

Fix 1: Consolidate the redundancy

Most bloated memory folders aren't 187 files of knowledge. They're 60 files of knowledge written down three times.

Cluster your files by topic and look for near-duplicates — the same rule restated in a project file, a global file, and a skill someone wrote later because they didn't know the first two existed. Merge each cluster into one canonical file. One rule, one home.

Then hunt specifically for contradiction pairs: two files that give conflicting guidance. These are worse than redundancy, because redundancy only costs tokens while contradiction costs correctness. When you find one, resolve it in the file and delete the loser. Do not archive it in a folder the agent can still read. Archived contradictions are still contradictions.

Expect to cut 30–50% on the first pass. It's the cheapest win available and it costs you an afternoon.

Fix 2: Go hierarchical, and make the top level thin

A flat folder forces a choice between loading everything and loading nothing. A hierarchy gives you a third option: load a map, then load the territory you actually entered.

The shape that works:

• A thin router at the top. Not content — pointers, plus the condition under which each one is worth reading. "Payments conventions → finance/payments.md, read before touching anything under services/billing."

• Domain folders in the middle, grouped the way the work is grouped, not the way the docs were written.

• Leaf procedures at the bottom, each one self-contained enough to be useful alone.

The point is that your standing context cost becomes the size of the index rather than the size of the corpus. Add fifty more procedures and the router grows by fifty lines, not fifty files.

Where your tooling supports it, push memory down next to the code it governs — a directory-scoped file only loads when the agent is working in that directory. That's free scoping you don't have to maintain.

Fix 3: Separate persistent rules from on-demand skills

This is the highest-leverage cleanup in the list, and almost nobody does it deliberately, because both things happen to be markdown files in the same folder. They are not the same kind of object and they should not have the same lifecycle.

Persistent global rules are invariants. Never force-push to main. Money is always integers in cents. Use pnpm. These need to be in context 100% of the time, which means they need a hard budget — a page, not a book. If your always-on file has grown past a screen or two, something in it is not actually an invariant.

On-demand skills are procedures. How to cut a release. How to run the backfill. How to add a new tenant. These are irrelevant to 99% of tasks and catastrophic to load into all of them.

The sorting test is one question: if the agent never read this file, would it do something wrong on every task, or only on one kind of task? Every task → global rule. One kind of task → on-demand skill.

Run every file through that question. Most teams discover their "global memory" is 80% procedures that snuck in.

Fix 4: Stop reading files. Start retrieving them.

Cleanup buys you a year at best. The corpus grows because the work grows, and eventually you're back where you started with better folder names.

The structural fix is to stop treating memory as text the agent reads and start treating it as an index the agent queries. Chunk every file, embed the chunks, store them in a vector index, and let the agent search at the moment of need instead of front-loading everything on the chance it's relevant.

The economics change completely:

• At rest, you pay for one tool definition. Not 64,000 tokens. Roughly a hundred.

• Per query, you pay for the k chunks you actually pulled. Three chunks of 400 tokens is ~1,200 tokens, on the calls that need it. That's roughly a 50× reduction against loading the corpus every session.

• It stays flat. Five hundred files cost the same at rest as five. Approximate nearest-neighbor search returns in single-digit milliseconds, so retrieval is effectively instant.

Retrieval also fixes something cleanup can't: the agent finds the right procedure by meaning rather than by matching a description you wrote six months ago and guessed the wording of.

Now the honest part, because this is where DIY implementations go sideways:

• Chunking strategy decides quality. Split on headings and keep procedures whole. A runbook cut in half mid-sequence retrieves as two useless fragments.

• Pure semantic search is bad at exact strings. Feature flag names, error codes, function names. You want hybrid retrieval — keyword and vector — or you'll miss the literal identifier the user just pasted.

• You need metadata filters. Scope by repo, service, language, environment. Without them, a query in one project happily retrieves another project's conventions.

• Staleness is now your problem. Edit a file, re-embed it. An index that silently drifts from the files is worse than no index, because it's confidently wrong.

• You need an eval set. Fifty query → expected-document pairs, run on every change. Retrieval regressions are invisible without one.

• And your memory is now sitting in a vector database that can read it. Procedural memory contains internal architecture, credentials patterns, business logic. Encryption at rest and tenant isolation are not optional extras here.

None of this is unsolvable. It's just two or three weeks of infrastructure work that has nothing to do with your product.

Four things we'd skip

Not every idea in this space is worth the week it costs. These are the ones we'd steer you away from.

Don't train your own skill classifier. This is the most common ambitious detour, and we've heard from several teams who went down it: fine-tune a small model to route each request to the right skill. It sounds clean and it doesn't work well in practice, for three reasons. You don't have the training data — you'd need thousands of realistic query-to-skill pairs, and the only honest source is production traffic you haven't collected yet. The label set is unstable — every new skill is a new class, so every skill anyone adds means another labeling and retraining cycle. And the accuracy the teams we spoke to actually achieved wasn't high enough to trust, which is the part that stings: a confidently wrong route is worse than no route, because the agent follows the wrong procedure to completion. Embeddings give you the same routing behavior with no training run, and the "label set" updates the instant someone writes a file.

Don't just buy a bigger context window. A larger window raises the ceiling; it doesn't fix attention. The instruction buried at position 40,000 is still the instruction the model weights least, and now you're paying for it on every call, on every session, at higher latency. More room to be ignored in is not a fix.

Don't compress your memory with an LLM. Summarizing 60 files into 10 feels like consolidation and isn't. Compression preferentially drops the specific — the exact flag name, the one environment where the order matters, the caveat someone added after an outage. That specificity was the entire reason the file existed. Summaries are great for prose and lossy for procedures.

Don't let every agent keep its own copy. The moment your coding agent, your support agent, and three teammates each maintain a private folder, you have four forks of the same knowledge drifting apart, and no one notices until two of them give opposite answers. Memory should be one shared store with many readers, not a file everyone copies.

Or skip the infrastructure detour

If your team's advantage is what your agents know, not how you host embeddings, that's exactly the work worth buying.

That's what XTrace is: a procedural memory cloud that does the indexing, hybrid retrieval, scoping, and encryption so your agents get instant semantic recall without a standing context tax — and without you evaluating six vector databases first.

No code: point your agent at the Memory Hub over MCP.

Building your own agent: the Procedural Memory API and SDK give you store and retrieve directly — npm install xtrace for JavaScript, or the Python SDK:

npm

npm

npm

import { MemoryClient } from '@xtraceai/memory';

const client = new MemoryClient({
  apiKey: process.env.XTRACE_API_KEY!,
  orgId:  process.env.XTRACE_ORG_ID!,
});

// write it once — extraction happens server-side
const job = await client.memories.ingest({
  messages: [
    { role: 'user', content: 'Releases go out Tuesdays. Tag the commit, then run deploy.sh --prod.' },
    { role: 'assistant', content: 'Noted.' },
  ],
  user_id: 'alice',
  conv_id: 'conv_2026_07_29',
});
await client.memories.jobs.pollUntilDone(job.id);

// retrieve at the moment of need, instead of reading the folder at startup
const results = await client.memories.search({
  query: 'how do we cut a release?',
  user_id: 'alice',
  limit: 10,
});

import { MemoryClient } from '@xtraceai/memory';

const client = new MemoryClient({
  apiKey: process.env.XTRACE_API_KEY!,
  orgId:  process.env.XTRACE_ORG_ID!,
});

// write it once — extraction happens server-side
const job = await client.memories.ingest({
  messages: [
    { role: 'user', content: 'Releases go out Tuesdays. Tag the commit, then run deploy.sh --prod.' },
    { role: 'assistant', content: 'Noted.' },
  ],
  user_id: 'alice',
  conv_id: 'conv_2026_07_29',
});
await client.memories.jobs.pollUntilDone(job.id);

// retrieve at the moment of need, instead of reading the folder at startup
const results = await client.memories.search({
  query: 'how do we cut a release?',
  user_id: 'alice',
  limit: 10,
});

import { MemoryClient } from '@xtraceai/memory';

const client = new MemoryClient({
  apiKey: process.env.XTRACE_API_KEY!,
  orgId:  process.env.XTRACE_ORG_ID!,
});

// write it once — extraction happens server-side
const job = await client.memories.ingest({
  messages: [
    { role: 'user', content: 'Releases go out Tuesdays. Tag the commit, then run deploy.sh --prod.' },
    { role: 'assistant', content: 'Noted.' },
  ],
  user_id: 'alice',
  conv_id: 'conv_2026_07_29',
});
await client.memories.jobs.pollUntilDone(job.id);

// retrieve at the moment of need, instead of reading the folder at startup
const results = await client.memories.search({
  query: 'how do we cut a release?',
  user_id: 'alice',
  limit: 10,
});

Two things worth noticing. Search defaults to compose mode, so what comes back isn't a pile of rows to post-process — it's an agent-selected subset already assembled into a markdown block you can inject. And scoping is a parameter, not a folder convention: user_id, group_ids, agent_id, app_id. The metadata-filtering problem from the previous section is just an argument here.

recall goes one step further and merges a user's own memories with their group's into a single deduplicated context block — which is the shared-memory point, made concrete.

It's framework-agnostic — Vercel AI SDK, Pydantic AI, Hermes Agents, or your own loop — and it's shared by default. What one agent learns, every agent and every teammate knows. The 187 markdown files stop being a liability and become the asset they were supposed to be.

Grab an API key at mem.xtrace.ai and read the docs.