# ARS v3.6.4 — Literature Corpus Input Port and Reference Adapters

- **Status**: Design approved, awaiting implementation plan
- **Author**: Cheng-I Wu (with AI collaborator)
- **Created**: 2026-04-23
- **ARS version target**: 3.6.4
- **Predecessor**: v3.6.3 (passport reset boundary)
- **Relates to**: [Material Passport data-layer boundary](../../academic-pipeline/references/passport_as_reset_boundary.md), Schema 9 in [`shared/handoff_schemas.md`](../../shared/handoff_schemas.md)

## 1. Scope and Non-Goals

### 1.1 In-scope

1. **Schema 9 extension**: add optional `literature_corpus: list[object]` field to the Material Passport.
2. **Entry contract**: `shared/contracts/passport/literature_corpus_entry.schema.json`. Entries use CSL-JSON name format for authors. Required set: `citation_key`, `title`, `authors`, `year`, `source_pointer`.
3. **Language-neutral adapter contract**: `academic-pipeline/references/adapters/overview.md` defines what any adapter must do (input assumptions, output files, error handling, rejection-log shape, provenance conventions, extension points).
4. **Three reference adapter implementations** in Python: `scripts/adapters/{folder_scan,zotero,obsidian}.py`, with tests and input/expected-output fixtures.
5. **Rejection log contract**: `shared/contracts/passport/rejection_log.schema.json`. Always emitted; empty when no rejections.
6. **CI lint**: schema validation (lint A), adapter-example validation (lint B), and schema→docs sync check (lint C) via `scripts/sync_adapter_docs.py --check`.
7. **pytest CI job** for `scripts/adapters/tests/`.
8. **Docs**: CHANGELOG, README EN + zh-TW, `.claude/CLAUDE.md`, version bump to 3.6.4, PERFORMANCE docs, and retirement of the "deferred to v3.6.4, PR-B" placeholders in `passport_as_reset_boundary.md`.

### 1.2 Out-of-scope (explicit)

- PDF parsing, OCR, text extraction. Data-layer responsibility.
- Zotero Web API client, Obsidian live vault daemon, or any service integration. Reference adapters read local files only.
- Paywall bypass, authenticated library crawling, cloud mirror fetching.
- Full-text ingestion. `full_text_ref` is reserved for a future release (tentatively v3.7.x), not v3.6.4.
- Privacy / copyright CI enforcement. `abstract` and `user_notes` carry text-only caveats in schema descriptions and `overview.md`. No lint blocks public passports that carry these fields.
- Non-Python reference implementations. `overview.md` is the language-neutral contract; Python is one reference implementation.
- Auto-ingestion from `literature_corpus[]` into `deep-research` or `synthesis_agent`. Consumer-side integration is deferred to v3.6.5+. This release only defines the input port and its producers.

### 1.3 Boundary rationale

ARS data-layer boundary is set by prior design decision: ARS defines `Material Passport` as the single input port; users own the corpus-to-passport adapter. v3.6.4 formalizes the `literature_corpus` portion of that contract and ships three reference adapters so the boundary is concrete, not aspirational. It does NOT ship production data-layer tooling.

## 2. Architecture and File Layout

### 2.1 Three layers

| Layer | Role | Artifact |
|-------|------|----------|
| **Contract** | Source of truth for passport and rejection shapes. | `shared/contracts/passport/literature_corpus_entry.schema.json`, `shared/contracts/passport/rejection_log.schema.json` |
| **Specification** | Language-neutral adapter behaviour. | `academic-pipeline/references/adapters/overview.md` |
| **Reference implementation** | Python adapters, tests, CI validation. | `scripts/adapters/**` |

Dependency direction: `overview.md` references the schemas; reference adapters read `overview.md` for the behavioural contract and emit output conforming to the schemas; `sync_adapter_docs.py` regenerates `overview.md`'s field tables from the schema so the two cannot drift.

### 2.2 New files

```
shared/contracts/passport/literature_corpus_entry.schema.json
shared/contracts/passport/rejection_log.schema.json
academic-pipeline/references/adapters/overview.md
scripts/adapters/README.md
scripts/adapters/_common.py
scripts/adapters/folder_scan.py
scripts/adapters/zotero.py
scripts/adapters/obsidian.py
scripts/adapters/examples/folder_scan/input_fixture/**
scripts/adapters/examples/folder_scan/expected_passport.yaml
scripts/adapters/examples/folder_scan/expected_rejection_log.yaml
scripts/adapters/examples/zotero/input_fixture/export.json
scripts/adapters/examples/zotero/expected_passport.yaml
scripts/adapters/examples/zotero/expected_rejection_log.yaml
scripts/adapters/examples/obsidian/input_fixture/vault/**
scripts/adapters/examples/obsidian/expected_passport.yaml
scripts/adapters/examples/obsidian/expected_rejection_log.yaml
scripts/adapters/tests/conftest.py
scripts/adapters/tests/test_folder_scan.py
scripts/adapters/tests/test_zotero.py
scripts/adapters/tests/test_obsidian.py
scripts/adapters/tests/test_common.py
scripts/check_literature_corpus_schema.py
scripts/sync_adapter_docs.py
.github/workflows/pytest.yml
docs/design/2026-04-23-ars-v3.6.4-literature-corpus-adapters-design.md   (this file)
```

### 2.3 Modified files

```
shared/handoff_schemas.md                                          Schema 9 optional-fields table gains literature_corpus row
academic-pipeline/references/passport_as_reset_boundary.md         Lines that say "deferred to v3.6.4, PR-B" get forward references
scripts/check_spec_consistency.py                                  Version pin 3.6.3 → 3.6.4
academic-pipeline/SKILL.md                                         Suite version footer → 3.6.4
academic-paper/SKILL.md                                            Suite version footer → 3.6.4
academic-paper-reviewer/SKILL.md                                   Suite version footer → 3.6.4
deep-research/SKILL.md                                             Suite version footer → 3.6.4
docs/PERFORMANCE.md                                                New section: "Literature corpus ingestion"
docs/PERFORMANCE.zh-TW.md                                          Same
CHANGELOG.md                                                       v3.6.4 entry
README.md                                                          Version / skill table
README.zh-TW.md                                                    Same
MODE_REGISTRY.md                                                   Sweep check (no-op expected)
.claude/CLAUDE.md                                                  Skills Overview version + v3.6.4 Key Additions
```

Existing CI workflow gains steps to run `check_literature_corpus_schema.py` and `sync_adapter_docs.py --check`; a new `pytest.yml` workflow (or additional job in existing workflow) runs adapter tests.

## 3. `literature_corpus_entry` schema

File: `shared/contracts/passport/literature_corpus_entry.schema.json`.

### 3.1 Shape

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://github.com/Imbad0202/academic-research-skills/shared/contracts/passport/literature_corpus_entry.schema.json",
  "title": "Material Passport Literature Corpus Entry",
  "description": "One entry in Material Passport Schema 9 literature_corpus[]. Represents a single piece of literature the user has included in their corpus. Produced by user-written adapters; see academic-pipeline/references/adapters/overview.md. ARS does NOT produce these entries itself.",

  "type": "object",
  "additionalProperties": false,

  "required": [
    "citation_key",
    "title",
    "authors",
    "year",
    "source_pointer"
  ],

  "properties": {
    "citation_key": {
      "type": "string",
      "minLength": 1,
      "pattern": "^[A-Za-z][A-Za-z0-9_:-]*$",
      "description": "Stable unique identifier for this entry within the passport. Typically BibTeX citekey (e.g., 'chen2024ai'). MUST be unique within literature_corpus[]. Uniqueness is enforced by the CI lint, not by JSON Schema."
    },
    "title": {
      "type": "string",
      "minLength": 1
    },
    "authors": {
      "type": "array",
      "minItems": 1,
      "items": { "$ref": "#/$defs/csl_name" },
      "description": "CSL-JSON name format. Each item is either a personal name ({family, given, ...}) or a corporate/institution name ({literal}). See CSL-JSON spec at https://docs.citationstyles.org/en/stable/specification.html#names."
    },
    "year": {
      "type": "integer",
      "minimum": 1000,
      "maximum": 2100,
      "description": "Publication year. If the adapter cannot determine a year, the entry MUST be rejected (go to rejection_log.yaml) rather than coerced to a placeholder."
    },
    "source_pointer": {
      "type": "string",
      "minLength": 1,
      "description": "Stable URI locating this work in the user's own KB. Examples: 'zotero://select/items/0_ABCD1234', 'obsidian://open?vault=kb&file=Chen2024', 'file:///path/to/refs/chen2024.pdf', 'https://doi.org/10.1234/xyz'. ARS does NOT dereference this; consumers that want the full text fetch it from the pointer themselves."
    },

    "venue":                  { "type": "string", "minLength": 1 },
    "doi": {
      "type": "string",
      "pattern": "^10\\.[0-9]{4,9}/[^\\s]+$",
      "description": "DOI without leading 'doi:' or URL prefix."
    },
    "tags": {
      "type": "array",
      "items": { "type": "string", "minLength": 1 },
      "description": "User-assigned tags from the source KB. Adapter-specific; treat as free-form."
    },

    "obtained_via": {
      "type": "string",
      "enum": ["zotero-api", "zotero-bbt-export", "obsidian-vault", "folder-scan", "manual", "other"],
      "description": "Strongly recommended. Which adapter produced this entry. 'other' is permitted for user-written custom adapters; those SHOULD also set adapter_name. Note: v3.6.4 reference implementations only use 'zotero-bbt-export', 'obsidian-vault', and 'folder-scan'. The remaining enum values ('zotero-api', 'manual') are reserved for user-written adapters and are permitted in the schema so such adapters remain valid without schema extension."
    },
    "obtained_at": {
      "type": "string",
      "format": "date-time",
      "description": "Strongly recommended. ISO 8601 timestamp when the adapter produced this entry."
    },
    "adapter_name": {
      "type": "string",
      "minLength": 1,
      "description": "Optional. Required when obtained_via='other'."
    },
    "adapter_version": {
      "type": "string",
      "minLength": 1
    },

    "abstract": {
      "type": "string",
      "description": "PRIVATE FIELD. May contain publisher-copyrighted material. Do NOT share passports containing this field publicly without confirming your right to do so. ARS consumers treat this as optional input; absence never causes failure."
    },
    "user_notes": {
      "type": "string",
      "description": "PRIVATE FIELD. User's own annotations from their KB. May contain copyrighted excerpts. Same sharing caveat as abstract."
    }
  },

  "$defs": {
    "csl_name": {
      "oneOf": [
        { "$ref": "#/$defs/csl_personal_name" },
        { "$ref": "#/$defs/csl_literal_name" }
      ]
    },
    "csl_personal_name": {
      "type": "object",
      "additionalProperties": false,
      "required": ["family"],
      "properties": {
        "family":                { "type": "string", "minLength": 1 },
        "given":                 { "type": "string" },
        "suffix":                { "type": "string" },
        "dropping-particle":     { "type": "string" },
        "non-dropping-particle": { "type": "string" },
        "comma-suffix":          { "type": ["string", "boolean"] },
        "static-ordering":       { "type": ["string", "boolean"] },
        "parse-names":           { "type": ["string", "boolean"] }
      }
    },
    "csl_literal_name": {
      "type": "object",
      "additionalProperties": false,
      "required": ["literal"],
      "properties": {
        "literal": { "type": "string", "minLength": 1 }
      }
    }
  }
}
```

### 3.2 Rationale

- **β required set** (citation_key / title / authors / year / source_pointer): downstream consumers can safely index `authors[0]` and format citations without null checks. Looser α would produce unusable entries for synthesis; stricter γ would make `folder_scan` unviable.
- **CSL-JSON full name schema** (not `list[string]` and not `{family, given}` only): matches Zotero's internal representation, handles institution authors via `literal`, supports `non-dropping-particle` for "van der Berg" and similar.
- **`year` is integer, not CSL date-parts**: keeps schema simple. In-press / "n.d." / forthcoming entries go to `rejection_log`, where the user can correct the source or decide to skip. A future release may widen this to the CSL date-parts form if need arises.
- **`obtained_via` splits `zotero-api` from `zotero-bbt-export`**: the two have different freshness and auth surfaces; downstream tooling may want to distinguish.
- **`additionalProperties: false`**: the schema is a hard contract. Custom fields go via `tags[]` or via user-maintained forks of the schema.
- **`abstract` and `user_notes` as private optional fields**: publishers typically retain rights to abstracts; user notes often contain copyrighted excerpts. The schema description is the warning; no CI block. Users who publish passports are responsible for pruning these fields themselves.

## 4. Rejection log schema

File: `shared/contracts/passport/rejection_log.schema.json`.

### 4.1 Shape

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://github.com/Imbad0202/academic-research-skills/shared/contracts/passport/rejection_log.schema.json",
  "title": "Literature Corpus Rejection Log",
  "description": "Output companion to passport.yaml. Records entries that the adapter could not include in literature_corpus[]. Always emitted; empty when no rejections. Consumers are not required to read this; it exists for user debugging and audit.",

  "type": "object",
  "additionalProperties": false,

  "required": ["adapter_name", "adapter_version", "generated_at", "rejected"],

  "properties": {
    "adapter_name":   { "type": "string", "minLength": 1 },
    "adapter_version":{ "type": "string", "minLength": 1 },
    "generated_at":   { "type": "string", "format": "date-time" },
    "input_source":   { "type": "string", "minLength": 1,
                        "description": "Optional. Human-readable description of the adapter input (path, URL, etc.)." },
    "rejected":       { "type": "array", "items": { "$ref": "#/$defs/rejection" } },
    "summary": {
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "total_input":    { "type": "integer", "minimum": 0 },
        "total_accepted": { "type": "integer", "minimum": 0 },
        "total_rejected": { "type": "integer", "minimum": 0 }
      }
    }
  },

  "$defs": {
    "rejection": {
      "type": "object",
      "additionalProperties": false,
      "required": ["source", "reason"],
      "properties": {
        "source": {
          "type": "string",
          "minLength": 1,
          "description": "Where the rejected item came from. Adapter-specific: filename for folder_scan, Zotero item key for zotero, .md path for obsidian."
        },
        "reason": {
          "type": "string",
          "enum": [
            "missing_required_field",
            "invalid_field_format",
            "duplicate_citation_key",
            "unresolvable_source_pointer",
            "year_unparseable",
            "authors_unparseable",
            "adapter_error",
            "other"
          ]
        },
        "detail": {
          "type": "string",
          "description": "Free-text explanation. Required when reason='other'. Recommended otherwise."
        },
        "raw": {
          "type": ["object", "string"],
          "description": "Optional. Adapter's best-effort reproduction of the original input item. Object for structured sources (Zotero export entry, Obsidian frontmatter dict); string for raw filenames or text lines."
        },
        "missing_fields": {
          "type": "array",
          "items": { "type": "string" },
          "description": "Optional but strongly recommended when reason='missing_required_field'. Enables batch fix in the user's KB."
        }
      }
    }
  }
}
```

### 4.2 Rationale

- **Always emit, even when empty**: removes "file-exists?" branching in downstream tooling.
- **Categorical `reason` enum**: users can `grep` / filter to batch-fix their KB.
- **`raw` typed as `object | string`**: lets structured adapters (Zotero) dump the whole input entry while letting unstructured adapters (`folder_scan`) dump just a filename. Rules out `null`, `array`, numeric types, so downstream viewers can assume printable shapes.
- **No `rejection_id`**: rejection is a snapshot; re-running the adapter produces a fresh log. No cross-run tracking needed.
- **`missing_fields` separate from `detail`**: batch-fix tooling can consume the structured list without parsing free text.

## 5. Adapter specifications

### 5.1 Common contract (applies to all adapters)

**Inputs**
- User-specified source: filesystem path, exported JSON file, or vault directory.
- Adapters do NOT make network requests. Zotero adapter reads a local Better BibTeX JSON export; it does not call the Zotero Web API.
- Adapters do NOT parse PDF content, run OCR, or extract full text.

**Outputs**
Each adapter always emits exactly two files, paths configurable via CLI:

- `passport.yaml`: contains a top-level `literature_corpus` key holding a list of entries conforming to `literature_corpus_entry.schema.json`.
- `rejection_log.yaml`: conforms to `rejection_log.schema.json`, always present, `rejected: []` when no rejections.

**Error-handling protocol (fail-soft)**
- Entry-level schema violation: push into `rejection_log.rejected[]`, continue processing.
- Adapter-level error (input file missing, JSON parse failure, vault dir not readable): fail-loud with `exit 1` and a stderr message. Do NOT produce a partial passport.
- All-rejections case: `passport.yaml` contains `literature_corpus: []`; `rejection_log.yaml` lists rejected entries normally.

**Provenance**
- Each entry SHOULD carry `obtained_via` and `obtained_at`. Schema marks these optional; the contract marks them strongly recommended.
- `rejection_log.yaml` MUST carry `adapter_name`, `adapter_version`, and `generated_at`.

**Determinism**
- Same input → same output (excluding `generated_at` in the rejection log and `obtained_at` on each entry).
- `literature_corpus[]` entries sorted ascending by `citation_key`.
- `rejection_log.rejected[]` entries sorted by `source`.

**CLI shape** (reference implementations; custom adapters may adopt their own)
```bash
python scripts/adapters/zotero.py \
  --input path/to/export.json \
  --passport out/passport.yaml \
  --rejection-log out/rejection_log.yaml
```

### 5.2 `folder_scan.py`

**Input**: a folder that contains PDF or other files. Scanned recursively.

**Filename convention** (best-effort parsing):
- `{AuthorLastname}{Year}{optional_title_slug}.{ext}` → author family=Lastname, year=Year.
- `{AuthorLastname}_{Year}_{title_slug}.{ext}` → same.
- Fallback: regex `(19|20)\d{2}` for year, first capitalized Latin word before the year for family.
- Single author only per filename. The adapter does NOT attempt to extract multiple authors from filenames like `Anderson_Chen_2024.pdf`; the first capitalized Latin word before the year is taken as the single family name, and additional names in the filename are dropped without warning. Users with multi-author corpora should use `zotero.py` or `obsidian.py` instead of relying on filename parsing for co-author metadata.
- If `authors` or `year` cannot be recovered: reject (`authors_unparseable` or `year_unparseable`).

**citation_key**: `{lastname_lowercase}{year}` plus first non-stopword title slug token when available; append `a`, `b`, `c`, … on collision.

**source_pointer**: `file://` URL using the absolute path.

**obtained_via**: `folder-scan`.

**Optional fields**: `abstract`, `user_notes`, `tags`, `venue`, `doi` are always omitted (no source for them).

**Known limitations** (documented in `overview.md`):
- Unicode filenames (e.g., Chinese names) are not parsed; such files are rejected. Unicode support is a future extension opportunity.
- No PDF metadata extraction; the adapter does not open PDF files.
- Empty or mis-named folders produce a high rejection rate — by design.

### 5.3 `zotero.py`

**Input**: Better BibTeX Extension's JSON export (not native CSL-JSON; BBT is a superset that includes `citationKey`).

**Field mapping**:

| BBT field | Entry field |
|-----------|-------------|
| `citationKey` | `citation_key` |
| `title` | `title` |
| `creators[type=author]` | `authors[]` (1:1 CSL-JSON mapping) |
| `date` / `issued` | `year` (parse 4-digit year) |
| `itemID` / `key` | `source_pointer: zotero://select/items/0_{key}` |
| `tags[].tag` | `tags[]` |
| `publicationTitle` / `proceedingsTitle` / `bookTitle` | `venue` (first non-empty wins) |
| `DOI` | `doi` (lowercased, leading URL/prefix stripped) |
| `abstractNote` | `abstract` (with copyright caveat) |
| `notes[].note` (HTML) | stripped to plain text, concatenated into `user_notes` |

**obtained_via**: `zotero-bbt-export`.

**Known limitations**:
- Only `creators[type=author]` are ingested; editors, translators, chairs are dropped (or, if the user has zero `type=author`, the entry is rejected as `authors_unparseable`).
- Date formats like "Spring 2024", "forthcoming", "n.d." are rejected (`year_unparseable`).
- This adapter does not call the Zotero Web API. Users who want live-sync behaviour are expected to write their own adapter based on `zotero.py`; see `overview.md` extension-point guidance.

### 5.4 `obsidian.py`

**Input**: an Obsidian vault directory. A vault is a folder full of `.md` files.

**Two supported conventions**. Detection runs per file and is deterministic: the adapter reads the frontmatter block first; if a `citekey` key is present, the file is processed as Convention A. If `citekey` is absent (including the case where there is no frontmatter at all), the file falls through to Convention B. Files that parse as neither A nor B are rejected.

Convention A — frontmatter-based (preferred):
```markdown
---
citekey: chen2024ai
title: "AI assessment in higher education"
authors:
  - family: Chen
    given: Cindy
year: 2024
venue: "Assessment & Evaluation"
doi: "10.1234/xyz"
tags: [assessment, ai]
---

Note body becomes user_notes (with copyright caveat).
```

Convention B — Karpathy-style (detected when `citekey` frontmatter absent):
- Filename: `{citekey}.md`
- Frontmatter contains a `source:` field holding the original URL/DOI/path.
- Body first H1 line (`# ...`) is the title.
- Second line like `**Authors**: Chen, C.; Wang, J.` parsed as semicolon-separated CSL names.
- Year parsed from either `**Year**: 2024` line or first 4-digit year in the `source:` field.
- Any parse failure → reject.

**source_pointer**: `obsidian://open?vault={vault_name}&file={url_encoded_relative_path}`.

**obtained_via**: `obsidian-vault`.

**user_notes**: the markdown body, preserved as-is (copyright caveat applied).

**Known limitations**:
- Obsidian wikilinks (`[[other note]]`) are NOT resolved; they appear as literal text in `user_notes`.
- Files under `_templates/`, `.obsidian/`, and hidden directories are skipped.
- Obsidian plugin config is not read; no Dataview queries are executed.
- Convention B is sensitive to formatting; files that match neither A nor B are rejected.

## 6. CI lint and pytest job

### 6.1 `scripts/check_literature_corpus_schema.py` (lint A + B)

Runs on every PR. Validates:

1. **A1 — schema self-consistency**: each JSON schema under `shared/contracts/passport/*.schema.json` validates as a valid JSON Schema Draft 2020-12 (`jsonschema.Draft202012Validator.check_schema`).
2. **A2 — passport-level invariants**: in every `expected_passport.yaml`, `literature_corpus[*].citation_key` values are unique. This is a global constraint JSON Schema cannot express.
3. **B — example output validation**: each `scripts/adapters/examples/*/expected_passport.yaml` and `expected_rejection_log.yaml` validates against its schema.

Exit 1 on any violation with a summary of which files failed and why.

### 6.2 `scripts/sync_adapter_docs.py` (lint C)

**Purpose**: generate the field reference tables in `overview.md` from the schemas, so the two cannot drift.

**Modes**:
- `python scripts/sync_adapter_docs.py`: rewrites the marked sections of `overview.md` in place.
- `python scripts/sync_adapter_docs.py --check`: diffs the generated tables against what is currently in `overview.md`; exit 1 on mismatch, showing the diff.

**Anchor format in `overview.md`**:
```markdown
<!-- GENERATED:LITERATURE_CORPUS_FIELDS:START -->
(generated table)
<!-- GENERATED:LITERATURE_CORPUS_FIELDS:END -->
```

Multiple anchors allowed (one per table). The script reads the schema `properties` block, grouping by `required` vs. optional, and emits a markdown table with field name, type, and description.

### 6.3 pytest CI job

New file `.github/workflows/pytest.yml` (or a new job in the existing CI workflow):

```yaml
name: pytest

on:
  pull_request:
    paths:
      - 'scripts/adapters/**'
      - 'shared/contracts/passport/**'
  push:
    branches: [main]
    paths:
      - 'scripts/adapters/**'
      - 'shared/contracts/passport/**'

jobs:
  pytest:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - name: Install deps
        run: |
          python -m pip install --upgrade pip
          pip install pytest pyyaml jsonschema
      - name: Run adapter tests
        run: pytest scripts/adapters/tests/ -v
```

The `paths:` filter prevents pytest from running on docs-only commits.

### 6.4 Test coverage per adapter

Each adapter test module exercises:

**Happy path**
- Read input fixture → produce passport and rejection log; output compared to `expected_passport.yaml` and `expected_rejection_log.yaml` byte-for-byte (after clearing non-deterministic timestamps).

**Rejection path**
- Input contains known-bad entries; rejection log carries them with the correct `reason` enum; passport excludes them.

**Edge cases (per adapter)**
- `folder_scan`: empty folder, non-parseable filename, duplicate citekey collision suffix.
- `zotero`: empty JSON array, malformed JSON (fail-loud path), corporate/institution author (`literal` variant), `creators` array with zero `type=author`.
- `obsidian`: vault with mixed A and B convention files, broken wikilink (handled as literal text), `_templates/` is skipped, `.obsidian/` hidden dir is skipped.

**Determinism**
- Running an adapter twice on the same input produces byte-identical output after removing `generated_at` and `obtained_at`.

**Common (`test_common.py`)**
- CSL name parsing helpers (e.g., `parse_semicolon_names("Chen, C.; van der Berg, J.")`).
- YAML emission (stable key order, proper multiline handling).
- citation_key sanitization (lowercase, strip punctuation, collision suffix `a`/`b`/`c`).

### 6.5 Existing CI workflow additions

In the existing CI workflow, add before the existing `check_passport_reset_contract.py` step:

```yaml
- name: Validate literature_corpus schemas and examples
  run: python scripts/check_literature_corpus_schema.py

- name: Check adapter docs up-to-date
  run: python scripts/sync_adapter_docs.py --check
```

## 7. Release plan

### 7.1 Version bump sweep

Files that require edits for the 3.6.3 → 3.6.4 bump:

- `CHANGELOG.md` (v3.6.4 entry)
- `README.md` and `README.zh-TW.md` (version / skill table)
- `.claude/CLAUDE.md` (Skills Overview version, v3.6.4 Key Additions)
- `scripts/check_spec_consistency.py` (version pin)
- `academic-pipeline/SKILL.md`, `academic-paper/SKILL.md`, `academic-paper-reviewer/SKILL.md`, `deep-research/SKILL.md` (suite version footer)
- `shared/handoff_schemas.md` (Schema 9 optional-fields table: add `literature_corpus` row)
- `academic-pipeline/references/passport_as_reset_boundary.md` (replace "deferred to v3.6.4, PR-B" placeholders with forward references)
- `docs/PERFORMANCE.md` and `docs/PERFORMANCE.zh-TW.md` (new "Literature corpus ingestion" section)
- `MODE_REGISTRY.md` (sweep check; no-op expected but verified by hand)

### 7.2 CHANGELOG entry

```markdown
## [3.6.4] — 2026-04-XX

### Added

- **Material Passport `literature_corpus[]` input port**. Schema 9 gains an optional
  `literature_corpus[]` field defined by
  `shared/contracts/passport/literature_corpus_entry.schema.json`. Each entry
  carries citation_key, CSL-JSON authors, year, title, and a source_pointer back
  to the user's own KB. `abstract` and `user_notes` are private optional fields
  with copyright caveats.

- **Adapter contract** (`academic-pipeline/references/adapters/overview.md`):
  language-neutral specification for producing literature_corpus entries from
  user-owned corpus sources. Covers fail-soft error handling, mandatory
  rejection-log output, deterministic ordering, and extension points.

- **Three reference adapter implementations** (`scripts/adapters/`):
  `folder_scan.py`, `zotero.py` (Better BibTeX export), `obsidian.py` (frontmatter
  or Karpathy-style convention). Each ships with tests and fixtures.

- **Rejection log contract** (`shared/contracts/passport/rejection_log.schema.json`).
  Always emitted; empty when no rejections.

- **CI lint and pytest job**: schema validation, adapter-example validation,
  schema→docs sync check, and pytest runs for adapter tests.

### Changed

- `academic-pipeline/references/passport_as_reset_boundary.md`: "deferred to v3.6.4"
  placeholders replaced with forward references.
- `shared/handoff_schemas.md`: Schema 9 optional fields table adds `literature_corpus`.

### Not changed (explicit non-goals)

- No ARS agent consumes `literature_corpus[]` yet. Consumer-side integration is
  deferred to a later release. v3.6.4 defines the input port only.
- No PDF parsing, no text extraction, no live API clients, no authenticated
  library crawling. ARS remains a writing/review-layer framework; data-layer
  integration belongs to user-owned adapters.
```

### 7.3 Release order

1. Feature branch: `feat/v3.6.4-literature-corpus-adapters`. Spec commit first, then schemas + adapters + tests + CI + docs, commit-by-commit.
2. Local test battery before push:
   ```
   pytest scripts/adapters/tests/
   python scripts/check_literature_corpus_schema.py
   python scripts/sync_adapter_docs.py --check
   python scripts/check_spec_consistency.py
   python scripts/check_passport_reset_contract.py
   ```
3. Push branch → open PR.
4. **Round 1 codex audit** (`/codex review --base main`). Apply P0/P1 fixes; commit.
5. **Round 2 codex audit**. Apply P0/P1 fixes; commit.
6. **Round 3+ codex audit until 0 P0/P1 remain**.
7. Run full local test battery again. Run `sync_adapter_docs.py --check` one more time.
8. **Fresh `codex review --base main`** in a new codex session (not resume). Apply any new P0/P1 findings; commit.
9. PR merge (via GitHub PR, not direct push to main).
10. Tag `v3.6.4`; publish GitHub release with notes from CHANGELOG entry.
11. Update memory: `project_academic_research_skills.md` → record v3.6.4 shipped, main SHA, tag.
12. Deferred: Substack / Michael follow-ups referencing the published spec and released adapters.

### 7.4 Rollback

v3.6.4 is additive and opt-in:
- Users who do not produce `literature_corpus[]` are unaffected. The field is optional.
- Adapters are reference examples under `scripts/`, not wired into any runtime.
- If a bug is found post-merge:
  - **Schema / adapter issue**: open v3.6.5 hotfix. Existing passports without `literature_corpus[]` continue to work.
  - **Sweep-level issue** (version pin drift, CLAUDE.md inconsistency): standard follow-up PR.

Worst-case revert: single `git revert` on the merge commit; no downstream system depends on the new field.

## 8. Open questions deferred to plan stage

- Exact Python version for `.github/workflows/pytest.yml`: confirm against existing CI (currently assumed 3.11). Align at plan stage.
- Whether `pytest.yml` is a new workflow file or a new job inside the existing CI workflow: either works; pick whichever reduces CI YAML duplication at implementation time.
- `_common.py` API surface: names of helper functions, whether to use `ruamel.yaml` vs `pyyaml` for YAML emission. Decided at plan stage; the spec only requires deterministic, round-tripped YAML.
- Exactly how to detect convention A vs. B in `obsidian.py`: the spec says "auto-detect"; the concrete rule (presence of `citekey` frontmatter key) is confirmed but the precise parse order can be tuned during TDD.

These are implementation-level details, not spec-level; they do not affect the contracts defined above.
