Content compilers

@repo/content is the heuristic brain of Sadie. Pure functions, no DB access, no network calls. Every exported function takes plain data and returns plain data; the caller (usually a server action or an API route handler) handles persistence.

Source: packages/content/src/. Top-level exports come through packages/content/src/index.ts; the SEPL pipeline is intentionally kept off the top-level namespace and imported from @repo/content/evolve-agent.

Wiki

compile-wiki.ts: compileWikiFromSources

First-pass wiki compile from user sources + calibration answers. Extracts themes, groups sources by theme, drafts one wiki entry per cluster, attaches source references, and emits a matching set of graph nodes and edges. Template-and-heuristic today; an LLM-backed replacement would implement the same signature.

incremental-wiki.ts

Incremental compile driver. Reads compile_cursors, picks up new sources + feed items since the last watermark, patches existing pages, and advances the cursor. Paired with compile-wiki on the full-compile path.

wiki-templates.ts

buildPageBody, defaultEmojiForKind, and the per-kind prose templates. All wiki entry bodies go through here so kind variations (concept vs. theme vs. question vs. narrative) stay consistent.

compile-wiki-ai.ts: generateWikiEntryBody

LLM-backed wiki body synthesis. The post-generation applyWikiBodyQualityGate() verifies the required section plan, rejects placeholder or credibility-leak prose, checks that the “Evidence and sources” section only cites source labels actually passed into the generation call, and requires at least one provided source label to appear in substantive prose outside the final evidence list.

Incremental wiki compile passes recent wiki rewrite feedback into this prompt. User edits are rendered as correction guidance for style, density, and grounding, not as source evidence for the page being generated.

themes.ts: extractThemes, slugify

Theme extraction over mixed text corpus. Deterministic, TF-IDF-ish. Slug computation is unicode-aware.

wiki-lint.ts

Produces lint findings against the current wiki state. Covers orphan pages, stale pages, weak provenance, contradiction flags, missing pages (concepts mentioned in 3+ entries with no page), low-linkage, and Soul contradictions. Pure; the caller persists the results into lint_findings.

Briefs + discourse

briefs.ts: generateBriefs, generateBriefsWithLLM

Clusters feed items by theme, emits one brief per cluster above a minimum cluster size. Heuristic version is deterministic; *WithLLM variant takes a CompleteFn so the caller can wire any provider. Accepts briefTone from the agent config.

The LLM path applies applyBriefQualityGate() before returning generated briefs. The gate rejects duplicate or thin briefs, requires the canonical markdown sections in order, rejects known template-like candidate-angle phrasing, and requires valid feed-item citations from the prompt before a brief can be persisted. It also requires visible cited voices, user/Soul grounding, supported wiki anchors, Candidate angles that carry both cited-source language and user-memory language, at least one fresh cited feed item when a later brief overlaps an earlier accepted brief, and no credibility-leak phrasing. If real LLM generation fails parsing or quality gating, Sadie returns no briefs instead of falling back to heuristic filler; the deterministic generateBriefs path is only used when the caller explicitly chooses the stub/no-provider path.

Compile runs pass compact briefFeedback into the LLM prompt. Filed briefs are accepted taste evidence, user-edited briefs are rewritten correction signals, user-dismissed or policy-blocked brief activity is negative evidence, and recent titles are anti-repeat memory. The quality gate rejects exact prior-title repeats before persistence.

Compile runs record brief gate outcomes separately from discourse gate outcomes. briefQualityGateAccepted / briefQualityGateRejected appear in workflow metrics, compile outputs, and activity metadata so rejected briefs are visible as trust-preserving decisions rather than silent empty refreshes.

discourse.ts: generateDiscourseOpportunities, generateDiscourseOpportunitiesWithLLM

Ranks feed items into up to three Today cards. Scoring composes theme overlap with Soul, convergence across tracked voices, recency, and allergy penalties. Reads engagementWeights, discourseMinConfidence, todayRanker coefficients, and discourseOpener from the agent config.

The LLM path applies applyDiscourseQualityGate() before ranking/persistence: live cards need valid cited feed items, fresh cited evidence when an earlier accepted card already used the same feed items, title/summary framing grounded in concrete cited feed-body language, substantive whyNow / whyYou fields, whyNow language that overlaps with the cited feed body, non-template angles, visible angle text that connects back to the user’s wiki, Soul, source, or calibration language, and angles that are not near-duplicates of posts the user already made. Invalid wiki/feed IDs are stripped and missing whoTalking entries are rebuilt from cited items. If the model supplies whoTalking stance or quote text, that evidence must map to a cited feed item and reuse concrete source language.

The LLM prompt also carries closed-loop posted-angle outcomes when available. Recent observed posted angles are rendered as history rows and distilled into deterministic calibration lessons, so Today can learn from the user’s actual audience before the engagement-weight refit has enough rows. Direct Soul review activity is folded into the same feedback channels: confirmed/edited/raised-confidence Soul items become positive guidance, while rejected/lowered-confidence items become avoid guidance. The gate strips copied old-post text so the loop learns shape without recycling finished posts.

synthesis-evals.ts

Deterministic regression corpus for synthesis trust. runSynthesisEvalCorpus() evaluates accepted/rejected discourse and brief examples without a live model call. It layers on top of the runtime quality gates and preserves Sadie-specific rubric checks for cited live evidence, whyNow grounded in cited source-body language, concrete whyYou, a specific whatsMissing observation, visible Today angles that answer the gap and are grounded in live and user-memory evidence, source-backed whoTalking evidence, brief section order, multi-source convergence evidence, brief Candidate angles grounded in live/user evidence, user-memory fit, and supported anchor references.

engagement-features.ts, engagement-model.ts, engagement-calibration.ts

Feature extraction + scoring model for predicting reply / engagement likelihood on an angle. engagement-calibration.ts fits per-user weights from observed engagement on their own feed.

Soul

voice-portrait.ts: buildVoicePortrait

8-dimension voice model from the user’s representative sources, documents, messages, and calibration. Computes sentence rhythm, register range, characteristic moves, vocabulary fingerprint, avoidances, emotional texture, framing habits, and argument architecture. Emits a ready-to-use system prompt.

synthesize-narrative.ts: synthesizeSoulNarrative, synthesizeNarrativeWithLLM

Produces the six-section Soul narrative (work context, personal context, top of mind, three tiers of brief history). Pure heuristic or LLM-backed at the caller’s choice.

preference-engine.ts

Fingerprinting, scoring, and promotion gating for preference candidates. computeFingerprint collapses duplicates across phrasings; promotion score decides when a candidate crosses into Soul.

soul-decay.ts: computeDecayedConfidence

Exponential decay (90-day half-life default) of Soul items that stop receiving reinforcing evidence. Items below the confidence floor flip back to candidate for review.

rewrite-engine.ts

Captures rewrite events (before / after text), computes dimensionDeltas, and accumulates them into voice portrait adjustments. Signals come from direct edits to assistant chat messages and from accepted Studio replace-range suggestions. Insert-at-cursor suggestions are intentionally excluded because they have no before text to diff.

Calibration + distillation (onboarding)

calibration.ts, distillation-probes.ts, distillation-weights.ts, distillation-ai.ts, rewrite-drill.ts, scenario-generator.ts

The onboarding distillation loop. Contrastive probes, rewrite drills, and scenario checks that infer the user’s working weights behaviorally rather than asking them to describe themselves. distillation-weights.ts fits the final writing weights; rewrite-drill.ts generates targeted drills to probe weak axes.

Policy

policy-engine.ts: runPolicyLint, runPolicyLintAsync

Deterministic text guardrails. Policies are regex | phrase | semantic patterns with block | rewrite | warn actions scoped across chat | studio | brief. runPolicyLintAsync takes an LLM-backed judge for semantic policies; runPolicyLint is sync (regex + phrase only).

Soul allergies are probabilistic nudges. Policies are rules. They coexist: if you need certainty, write a policy.

Agent evolution (@repo/content/evolve-agent)

Imported separately to avoid polluting the top-level namespace with generic names (reflect/select/improve/evaluate).

defaults.ts

BASE_AGENT_CONFIG: the v1 config every new user starts with. Prompt tone knobs, weight defaults, policy coefficients, empty catalog. DEFAULT_XP_WEIGHTS + tierThresholdFor gate when evolution is allowed to run.

xp.ts: computeXp, mergedXpWeights

Experience-point computation over a window of activity. Evolution only fires when XP crosses the tier threshold for the current agent version.

reflect.ts

Generates causal hypotheses (so-called “semantic gradients”) about how the current config is underperforming. Reads activity + brief signals + discourse signals + wiki rewrite signals + rewrite signals; emits ranked EvolutionHypothesis[]. Repeated wiki rewrites can raise wikiPromotionThreshold.

select.ts

Enforces the ALLOWED_KEYS allow-list and per-numeric-key CLAMPS. Anything outside is not selectable. Emits concrete Proposal[] with before / after values.

improve.ts

Applies proposals to the current config, returns a candidate. Pure, non-mutating.

evaluate.ts

Safety + soft-delta eval. Structural invariants are the floor (always checked). With replayInputs, replays generateDiscourseOpportunities + generateBriefs under both configs and derives a delta score. Output AgentEvalReport carries delta, safe, replayScore, invariants, rationale.

labels.ts

Human labels for diff entries (“less hedging”, “shorter briefs”, etc.). Used in the level-up card copy.

Pipeline entry: runEvolutionCycle

Reflect → Select → Improve → Evaluate in one call. Accepts current config, activity, optional signals, and optional replay inputs. Returns a committed candidate or a rejected report. Never touches the database.

Shared types

types.ts defines the input/output shapes every compiler expects. Notable ones: SourceInput, SoulSnapshot, FeedItemInput, WikiSeed, CalibrationInput, and the CompleteFn signature every *WithLLM variant consumes.