---
name: kb-builder
description: "Use when the machine's multi-platform AI sessions (DSH / Codex / DeepSeek web export) need to be organised into a searchable knowledge base, or when an existing knowledge base needs an incremental rerun; also for requests like 'can these sessions be captured for reuse' and 'how do I top up a knowledge base that has gone stale'."
---

> Original skill · commissioned by the user · English edition, 2026-11-12

# Session knowledge-base construction

Turn the sessions scattered across the DSH session store, the Codex home and DeepSeek export packages into queryable cards + index + project pages + topic pages + lessons/decisions ledgers. **A one-command job is not retention; being rerunnable is.**

## When to use

- Organising historical sessions into a knowledge base (first build)
- Weeks after the knowledge base was built, adding new sessions
- Rebuilding an isomorphic library on another machine / for another user
- Auditing "what was actually decided in those past sessions"

Do not use for: just finding a piece of a conversation (use `memory-compounding` or plain grep), or just producing screenshots (use `*-conversation-capture`).

## Quick start

```powershell
$S = "<SKILL_DIR>/scripts"
$env:KB_ROOT  = "<KNOWLEDGE_BASE>"          # library root (changeable)
$env:KB_WORK  = "<KNOWLEDGE_BASE>/_work"    # process-files directory (must be writable while building)

python $S\extract.py            # DSH + Codex → sessions/*.json + text/*.md
python $S\extract_dsweb.py      # DeepSeek web export (reads the zip, keeps no raw copy)
python $S\extract_doubao.py     # Doubao office mode (agent_mode .sessions)
python $S\extract_qianwen.py    # Qianwen desktop (qwen-agent projects/*/sessions, JSONL event stream)
python $S\extract_grokbot.py    # Grok Bot export (index.json + md; the cloud agent reads and writes the disk itself)
python $S\make_batches.py       # batching (≤130k characters and ≤10 sessions per batch)
python $S\auto_cards.py         # minimal card for empty sessions
# —— card building and page writing go to subagents, see references/pipeline.md ——
python $S\render_cards.py       # card JSON → 10-session-cards/*.md
python $S\normalize.py          # work-line normalisation
python $S\index.py              # INDEX.md / index.csv
python $S\catalog.py            # distribution statistics
python $S\agg_all.py --assets-only
python $S\monthly.py
python $S\agg_auto.py
python $S\link_sources.py
python $S\check.py              # acceptance: PASS is the only finish
```

By default sources are read from local paths (`DSH_SESSIONS_ROOT` / `CODEX_HOME` / `DS_EXPORT_ZIP` / `DOUBAO_SESSIONS_ROOT` / `QIANWEN_SESSIONS_ROOT` / `GROKBOT_EXPORT_ROOT` can override).

## Three hard rules (all learned the hard way)

1. **Subagent concurrency ≤6.** 8 was measured as the ceiling; starting 65 at once fails the whole batch with `active child limit: 8`.
2. **A new batch gets a new prefix.** `make_batches.py --prefix w --tag dsweb`; otherwise the new batch's `b01` overwrites the old batch's card JSON.
3. **Wait for quiet before extracting.** Cards extracted while a source session is still being written come out truncated: wait for 15 consecutive minutes with no writes in DSH/Codex before extracting.
4. **The order of an incremental rerun may not be reversed: `extract` → `render_cards` → `make_batches`.** `extract` rewrites `_work/sessions/*.json` and clears the `card` field inside them; without `render_cards` backfilling first, `make_batches` treats every session that already has a card as missing (measured in a past run: <N> false pending items reported, only <N> genuinely new ones left after backfill).

## Common errors

| Symptom | Cause | Fix |
|---|---|---|
| `session.v4.jsonl.zstd` is all garbage | read as text with `open()` | decompress with `zstandard.stream_reader` |
| Real keys appear in cards | only the summary text was redacted, while `sessions/*.json` stores the raw arrays; every re-extraction brings the plaintext back | run a JSON rescan after every extract (`sk-*` / `ark-*` / `api_key: …` → `REDACTED`, see references/pipeline.md) |
| Every link to a Chinese-named file breaks | links were written as `%20` | run the fix after `link_sources.py`, or wrap in `<path with spaces>` |
| The lessons/decisions page has 700+ entries and is unreadable | full dump | curate (2–10 per group) + save the full set separately as `*_full.md` |
| A new work line gets classified elsewhere | the normalisation rules hit an old keyword first | insert the new rule at the top of the rule table |

## Acceptance

```powershell
python $S\check.py
# expected: formal sessions = card count, 0 missing fields, 0 plaintext keys, 0 dead links, verdict PASS
```

Details and the full card-building / page-writing prompts are in `references/pipeline.md`.