Content compilers

@repo/content is the heuristic brain of Sadie. Pure functions, no DB access, no network calls. Every exported function takes plain data and returns plain data; the caller (usually a server action or an API route handler) handles persistence.

Source: packages/content/src/. Top-level exports come through packages/content/src/index.ts; the SEPL pipeline is intentionally kept off the top-level namespace and imported from @repo/content/evolve-agent.

Wiki

compile-wiki.ts: compileWikiFromSources

First-pass wiki compile from user sources + calibration answers. Extracts themes, groups sources by theme, drafts one wiki entry per cluster, attaches source references, and emits a matching set of graph nodes and edges. Template-and-heuristic today; an LLM-backed replacement would implement the same signature.

incremental-wiki.ts

Incremental compile driver. Reads compile_cursors, picks up new sources + feed items since the last watermark, patches existing pages, and advances the cursor. Paired with compile-wiki on the full-compile path.

wiki-templates.ts

buildPageBody, defaultEmojiForKind, and the per-kind prose templates. All wiki entry bodies go through here so kind variations (concept vs. theme vs. question vs. narrative) stay consistent.

themes.ts: extractThemes, slugify

Theme extraction over mixed text corpus. Deterministic, TF-IDF-ish. Slug computation is unicode-aware.

wiki-lint.ts

Produces lint findings against the current wiki state. Covers orphan pages, stale pages, weak provenance, contradiction flags, missing pages (concepts mentioned in 3+ entries with no page), low-linkage, and Soul contradictions. Pure; the caller persists the results into lint_findings.

Briefs + discourse

briefs.ts: generateBriefs, generateBriefsWithLLM

Clusters feed items by theme, emits one brief per cluster above a minimum cluster size. Heuristic version is deterministic; *WithLLM variant takes a CompleteFn so the caller can wire any provider. Accepts briefTone from the agent config.

discourse.ts: generateDiscourseOpportunities, generateDiscourseOpportunitiesWithLLM

Ranks feed items into up to three Today cards. Scoring composes theme overlap with Soul, convergence across tracked voices, recency, and allergy penalties. Reads engagementWeights, discourseMinConfidence, todayRanker coefficients, and discourseOpener from the agent config.

engagement-features.ts, engagement-model.ts, engagement-calibration.ts

Feature extraction + scoring model for predicting reply / engagement likelihood on an angle. engagement-calibration.ts fits per-user weights from observed engagement on their own feed.

Soul

voice-portrait.ts: buildVoicePortrait

8-dimension voice model from the user’s representative sources, documents, messages, and calibration. Computes sentence rhythm, register range, characteristic moves, vocabulary fingerprint, avoidances, emotional texture, framing habits, and argument architecture. Emits a ready-to-use system prompt.

synthesize-narrative.ts: synthesizeSoulNarrative, synthesizeNarrativeWithLLM

Produces the six-section Soul narrative (work context, personal context, top of mind, three tiers of brief history). Pure heuristic or LLM-backed at the caller’s choice.

preference-engine.ts

Fingerprinting, scoring, and promotion gating for preference candidates. computeFingerprint collapses duplicates across phrasings; promotion score decides when a candidate crosses into Soul.

soul-decay.ts: computeDecayedConfidence

Exponential decay (90-day half-life default) of Soul items that stop receiving reinforcing evidence. Items below the confidence floor flip back to candidate for review.

rewrite-engine.ts

Captures rewrite events (before / after text), computes dimensionDeltas, and accumulates them into voice portrait adjustments.

Calibration + distillation (onboarding)

calibration.ts, distillation-probes.ts, distillation-weights.ts, distillation-ai.ts, rewrite-drill.ts, scenario-generator.ts

The onboarding distillation loop. Contrastive probes, rewrite drills, and scenario checks that infer the user’s working weights behaviorally rather than asking them to describe themselves. distillation-weights.ts fits the final writing weights; rewrite-drill.ts generates targeted drills to probe weak axes.

Policy

policy-engine.ts: runPolicyLint, runPolicyLintAsync

Deterministic text guardrails. Policies are regex | phrase | semantic patterns with block | rewrite | warn actions scoped across chat | studio | brief. runPolicyLintAsync takes an LLM-backed judge for semantic policies; runPolicyLint is sync (regex + phrase only).

Soul allergies are probabilistic nudges. Policies are rules. They coexist: if you need certainty, write a policy.

Agent evolution (@repo/content/evolve-agent)

Imported separately to avoid polluting the top-level namespace with generic names (reflect/select/improve/evaluate).

defaults.ts

BASE_AGENT_CONFIG: the v1 config every new user starts with. Prompt tone knobs, weight defaults, policy coefficients, empty catalog. DEFAULT_XP_WEIGHTS + tierThresholdFor gate when evolution is allowed to run.

xp.ts: computeXp, mergedXpWeights

Experience-point computation over a window of activity. Evolution only fires when XP crosses the tier threshold for the current agent version.

reflect.ts

Generates causal hypotheses (so-called “semantic gradients”) about how the current config is underperforming. Reads activity + brief signals + discourse signals + rewrite signals; emits ranked EvolutionHypothesis[].

select.ts

Enforces the ALLOWED_KEYS allow-list and per-numeric-key CLAMPS. Anything outside is not selectable. Emits concrete Proposal[] with before / after values.

improve.ts

Applies proposals to the current config, returns a candidate. Pure, non-mutating.

evaluate.ts

Safety + soft-delta eval. Structural invariants are the floor (always checked). With replayInputs, replays generateDiscourseOpportunities + generateBriefs under both configs and derives a delta score. Output AgentEvalReport carries delta, safe, replayScore, invariants, rationale.

labels.ts

Human labels for diff entries (“less hedging”, “shorter briefs”, etc.). Used in the level-up card copy.

Pipeline entry: runEvolutionCycle

Reflect → Select → Improve → Evaluate in one call. Accepts current config, activity, optional signals, and optional replay inputs. Returns a committed candidate or a rejected report. Never touches the database.

Shared types

types.ts defines the input/output shapes every compiler expects. Notable ones: SourceInput, SoulSnapshot, FeedItemInput, WikiSeed, CalibrationInput, and the CompleteFn signature every *WithLLM variant consumes.