Repo as System of Record for Research
A research repository can preserve the plans, source definitions, assumptions, transformations, artifacts, and verification records needed to reconstruct a result.
Evidence, retrieval, agents, and the paths that connect a question to a grounded answer.
Interactive model
Research state as connected artifacts
Plans, sources, assumptions, tasks, and outputs become connected nodes rather than scattered files.
Live HTML simulation · adjust the controls and watch the computed output respond.
Interactive
Agent systems are graphs of state, routing, and tool access
This is a simplified teaching model. Its displayed values are computed from the controls; the article explains where the model stops.
Site connection
RAIL makes the repository the contract for research plans, source definitions, ontology schemas, hydration pipelines, artifacts, and agent handoffs.
A repository is a useful system of record when a reviewer can start from a reported claim and trace backward through an artifact, executable transformation, pinned source definition, assumptions, and plan. Git supplies versioned snapshots and history; the research workflow must supply provenance, validation, and governance.
The Reported RAIL Architecture
Project fact: the RAIL portfolio source describes an open-source, repo-native platform for agent-assisted economic and policy research. It reports repository records for research plans, source definitions, ontology schemas, hydration pipelines, analysis artifacts, verification records, and agent handoffs.
The source also reports a Next.js command center, FastAPI control plane, Python engine, SDK and CLI, MCP server, ontology-backed project models, YAML configuration, and local and cloud execution modes. Those are architectural claims about RAIL; the file does not provide benchmark results or certify that every repository state is reproducible or secure.
| Statement | Classification |
|---|---|
| The repository is the contract for research state | Reported RAIL design |
| RAIL includes plans, sources, ontology, hydration, artifacts, and verification records | Reported RAIL surface |
| Every analysis should pin environment and input versions | Recommended reproducibility policy |
| Sensitive raw data should remain outside Git | Recommended governance policy, dependent on context |
What the Repository Should Record
The system of record is the minimum durable contract needed to reconstruct and review the work. Store the question and scope, permitted source definitions, data dictionaries, ontology terms, transformation code, environment lock, parameters, artifact manifests, checks, and named unresolved limitations. Git's official model records commits as steps in project history, but a commit message alone does not explain scientific intent.
Large, licensed, personal, or secret data may stay in governed storage. The repository can instead record a stable source identifier, retrieval query, time window, checksum or object version, schema, access instructions, and a non-sensitive fixture. This is not 'everything in Git'; it is enough metadata and code to identify the exact external input.
Analogy limit: a repository resembles a laboratory notebook because it preserves dated work. It does not automatically capture undocumented manual steps, external state, credentials, computational environments, or the validity of a research method.
| Artifact | Repository role | Typical verification |
|---|---|---|
| Research plan | Defines question, scope, and acceptance criteria | Reviewer confirms decision relevance |
| Source definition | Pins origin, query, version, and limitations | Checksum or provider version resolves |
| Ontology/schema | Defines concepts, units, and relationships | Schema and unit checks pass |
| Hydration pipeline | Recreates analysis-ready local inputs | Fixture and row-count tests pass |
| Artifact manifest | Connects output to code, inputs, and parameters | Rerun matches within tolerance |
| Verification record | Captures who checked what and when | Evidence links and test status resolve |
From Question to Verified Artifact
A durable workflow starts with an acceptance test: what result is needed, for whom, at what grain, and under which definitions? Source records then identify permitted evidence. Hydration converts external inputs into analysis tables; transformations produce intermediate and final artifacts; verification evaluates schema, units, coverage, reasonableness, and claim-to-artifact links.
Recommended policy: make every run produce a manifest with commit ID, environment or container digest, input versions, parameters, timestamps, output paths, checksums, and check results. An agent handoff should point to these records and name blocked decisions instead of summarizing the entire project from memory.
| Stage | Required question | Failure made visible |
|---|---|---|
| Plan | What exact claim or decision is in scope? | Scope drift |
| Source | Which version and definition are allowed? | Untraceable or stale evidence |
| Hydrate | Can the input be recreated? | Undocumented download or manual edit |
| Analyze | Which code and parameters produced the output? | Notebook hidden state |
| Verify | Which checks passed and what remains unknown? | False confidence |
| Handoff | Where should the next worker resume? | Chat-only context loss |
Worked Example
A policy team estimates county unemployment changes. The plan defines the date range, seasonal-adjustment choice, county identifier, and output tolerance. A source record pins the provider series, retrieval date, and vintage. The hydration script writes a local table plus a manifest; the analysis script computes changes and produces `county-change.csv` and a chart.
A verification run checks unique county keys, expected months, percentage-point units, null rates, and that every chart value comes from the CSV. The result links to commit `abc123`, environment lock `uv.lock`, source vintage `2026-07`, and output checksum `sha256:…`. These identifiers are illustrative, not reported RAIL results.
When the provider revises the prior month, a new source version changes the checksum. The repository keeps the old artifact, creates a new run, and records the difference rather than overwriting history. A reviewer can now say whether the public conclusion changed because of data revision, code, or assumptions.
| Record | Illustrative value | Review purpose |
|---|---|---|
| Plan | County change, Jan–Jun, percentage points | Fix scope and units |
| Source | Provider series, vintage 2026-07 | Identify external state |
| Run | commit abc123 + uv.lock | Identify code and environment |
| Checks | keys, months, units, nulls, lineage | Test structural and semantic validity |
| Artifact | CSV/chart checksum | Identify exact output |
Security, Scale, and Misconceptions
A repository is not an appropriate home for every payload. Use ignore rules and secret scanning, but remember that removing a file from the current tree does not necessarily remove it from Git history. Access controls, encrypted storage, data-use agreements, retention, and deletion procedures must cover external stores and generated artifacts as well as source code.
Git history supports change review; it does not guarantee reproducibility, statistical validity, provenance completeness, or safe agent execution. Large binary datasets can make clones expensive, and concurrent generated outputs can create noisy conflicts. Keep compact contracts and fixtures in Git, use content-addressed or versioned external storage for heavy inputs, and test restoration from a clean environment.
Common Pitfalls
- Committing credentials, personal data, licensed data, or sensitive raw exports.
- Assuming a Git commit alone captures external inputs, environment, and manual steps.
- Tracking final charts without their plan, source definitions, parameters, and transformation code.
- Letting notebooks depend on execution order or hidden kernel state.
- Overwriting revised artifacts instead of preserving run identity and differences.
- Treating repository history as proof of statistical or methodological validity.
- Describing recommended manifests and controls as verified RAIL implementation behavior.