# ARS v3.6.4 Literature Corpus Adapters — Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Ship ARS v3.6.4: the Material Passport `literature_corpus[]` input port, its JSON schemas, three Python reference adapters (folder_scan / zotero / obsidian), a rejection log contract, CI validation, a pytest job, and the full version-bump sweep.

**Architecture:** Three layers — (1) JSON Schema contracts under `shared/contracts/passport/`, (2) language-neutral adapter contract in `academic-pipeline/references/adapters/overview.md`, (3) Python reference implementations in `scripts/adapters/`. CI guards schema self-consistency, example-to-schema alignment, and schema-to-docs drift via `sync_adapter_docs.py --check`.

**Tech Stack:** Python 3.11, `pyyaml`, `jsonschema`, `pytest`. JSON Schema Draft 2020-12. YAML for passport/rejection outputs. GitHub Actions for CI.

**Branch:** `feat/v3.6.4-literature-corpus-adapters` (already exists, head `56639ae` with spec commit). All tasks commit to this branch.

**Spec:** `docs/design/2026-04-23-ars-v3.6.4-literature-corpus-adapters-design.md` (already in branch).

**Working directory:** `/path/to/academic-research-skills/`

---

## Task dependency map

```
Task 1 (foundation)
  ├── Task 2 (literature_corpus_entry.schema.json)
  │    ├── Task 3 (rejection_log.schema.json)
  │    │    └── Task 4 (check_literature_corpus_schema.py)
  │    └── Task 5 (_common.py)
  │         └── Task 6 (conftest.py)
  │              ├── Task 7 (folder_scan.py + tests + fixtures)
  │              ├── Task 8 (zotero.py + tests + fixtures)
  │              └── Task 9 (obsidian.py + tests + fixtures)
  └── ... later tasks build on outputs of 4-9

After 4-9 all done:
  Task 10 (overview.md)
    └── Task 11 (sync_adapter_docs.py)
  Task 12 (CI workflows)
  Task 13 (Schema 9 + retire placeholders)
  Task 14 (version bump sweep)
  Task 15 (CHANGELOG)
  Task 16 (adapter README)
  Task 17 (PERFORMANCE docs)
  Task 18 (final local run + PR open)
```

Tasks 7/8/9 are independent after Task 6 and may be parallelized safely.

---

## Task 1: Foundation scaffolding

**Files:**
- Create: `scripts/adapters/` (directory)
- Create: `scripts/adapters/examples/` (directory)
- Create: `scripts/adapters/tests/` (directory)
- Create: `shared/contracts/passport/` (already exists from v3.6.3, verify)

- [ ] **Step 1: Confirm branch and working tree**

Run:
```bash
cd ~/Projects/academic-research-skills
git status
git branch --show-current
```
Expected: clean working tree on `feat/v3.6.4-literature-corpus-adapters`.

- [ ] **Step 2: Create empty directory skeleton**

Run:
```bash
cd ~/Projects/academic-research-skills
mkdir -p scripts/adapters/examples/folder_scan/input_fixture
mkdir -p scripts/adapters/examples/zotero/input_fixture
mkdir -p scripts/adapters/examples/obsidian/input_fixture/vault
mkdir -p scripts/adapters/tests
mkdir -p academic-pipeline/references/adapters
ls -la shared/contracts/passport/
```
Expected: `reset_ledger_entry.schema.json` already present from v3.6.3.

- [ ] **Step 3: Add `.gitkeep` placeholders for empty dirs so they commit**

Only needed where no real file is created yet. The fixture directories will be populated in Task 7/8/9; use `.gitkeep` for now.

```bash
touch scripts/adapters/examples/folder_scan/input_fixture/.gitkeep
touch scripts/adapters/examples/zotero/input_fixture/.gitkeep
touch scripts/adapters/examples/obsidian/input_fixture/vault/.gitkeep
```

- [ ] **Step 4: Commit the scaffolding**

```bash
cd ~/Projects/academic-research-skills
git add scripts/adapters/ academic-pipeline/references/adapters/
git status
git commit -m "$(cat <<'EOF'
chore(v3.6.4): scaffold adapter directory layout

Empty directories for scripts/adapters/{,examples/,tests/} and
academic-pipeline/references/adapters/. Fixtures get populated in
later tasks; .gitkeep keeps the tree intact in the meantime.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

Expected: single commit adding only `.gitkeep` files.

---

## Task 2: `literature_corpus_entry.schema.json`

**Files:**
- Create: `shared/contracts/passport/literature_corpus_entry.schema.json`
- Test: `scripts/adapters/tests/test_literature_corpus_entry_schema.py` (will live under adapter tests dir)

- [ ] **Step 1: Write the failing test**

Create `scripts/adapters/tests/test_literature_corpus_entry_schema.py`:

```python
"""Validates literature_corpus_entry.schema.json self-consistency and
round-trips a known-good example entry against it."""
from pathlib import Path
import json
import pytest
from jsonschema import Draft202012Validator

REPO_ROOT = Path(__file__).resolve().parents[3]
SCHEMA_PATH = REPO_ROOT / "shared/contracts/passport/literature_corpus_entry.schema.json"


def _load_schema():
    with SCHEMA_PATH.open("r", encoding="utf-8") as f:
        return json.load(f)


def test_schema_file_exists():
    assert SCHEMA_PATH.exists(), f"schema missing at {SCHEMA_PATH}"


def test_schema_is_valid_draft_2020_12():
    schema = _load_schema()
    Draft202012Validator.check_schema(schema)


def test_required_set_matches_spec():
    schema = _load_schema()
    assert schema["required"] == [
        "citation_key",
        "title",
        "authors",
        "year",
        "source_pointer",
    ]


def test_additional_properties_is_false():
    schema = _load_schema()
    assert schema["additionalProperties"] is False


def test_valid_personal_author_entry_passes():
    schema = _load_schema()
    entry = {
        "citation_key": "chen2024ai",
        "title": "AI assessment",
        "authors": [{"family": "Chen", "given": "Cindy"}],
        "year": 2024,
        "source_pointer": "https://doi.org/10.1234/xyz",
    }
    Draft202012Validator(schema).validate(entry)


def test_valid_institution_author_entry_passes():
    schema = _load_schema()
    entry = {
        "citation_key": "who2024report",
        "title": "World report",
        "authors": [{"literal": "World Health Organization"}],
        "year": 2024,
        "source_pointer": "https://www.who.int/report",
    }
    Draft202012Validator(schema).validate(entry)


def test_missing_required_field_fails():
    from jsonschema.exceptions import ValidationError
    schema = _load_schema()
    entry = {
        "citation_key": "chen2024ai",
        "title": "AI assessment",
        # missing authors
        "year": 2024,
        "source_pointer": "file:///x.pdf",
    }
    with pytest.raises(ValidationError):
        Draft202012Validator(schema).validate(entry)


def test_author_must_be_either_personal_or_literal():
    from jsonschema.exceptions import ValidationError
    schema = _load_schema()
    entry = {
        "citation_key": "bad2024",
        "title": "Bad author",
        "authors": [{}],  # neither family nor literal
        "year": 2024,
        "source_pointer": "file:///x.pdf",
    }
    with pytest.raises(ValidationError):
        Draft202012Validator(schema).validate(entry)


def test_year_out_of_range_fails():
    from jsonschema.exceptions import ValidationError
    schema = _load_schema()
    entry = {
        "citation_key": "old1",
        "title": "Ancient",
        "authors": [{"family": "X"}],
        "year": 999,
        "source_pointer": "file:///x.pdf",
    }
    with pytest.raises(ValidationError):
        Draft202012Validator(schema).validate(entry)


def test_citation_key_pattern_rejects_leading_digit():
    from jsonschema.exceptions import ValidationError
    schema = _load_schema()
    entry = {
        "citation_key": "2024chen",
        "title": "T",
        "authors": [{"family": "C"}],
        "year": 2024,
        "source_pointer": "file:///x.pdf",
    }
    with pytest.raises(ValidationError):
        Draft202012Validator(schema).validate(entry)


def test_additional_property_fails():
    from jsonschema.exceptions import ValidationError
    schema = _load_schema()
    entry = {
        "citation_key": "chen2024",
        "title": "T",
        "authors": [{"family": "C"}],
        "year": 2024,
        "source_pointer": "file:///x.pdf",
        "custom_field": "should_not_be_allowed",
    }
    with pytest.raises(ValidationError):
        Draft202012Validator(schema).validate(entry)


def test_obtained_via_enum_constrained():
    from jsonschema.exceptions import ValidationError
    schema = _load_schema()
    entry = {
        "citation_key": "chen2024",
        "title": "T",
        "authors": [{"family": "C"}],
        "year": 2024,
        "source_pointer": "file:///x.pdf",
        "obtained_via": "rubber-duck",
    }
    with pytest.raises(ValidationError):
        Draft202012Validator(schema).validate(entry)
```

- [ ] **Step 2: Run test to verify it fails**

Run:
```bash
cd ~/Projects/academic-research-skills
pip install pytest pyyaml jsonschema 2>/dev/null
pytest scripts/adapters/tests/test_literature_corpus_entry_schema.py -v
```
Expected: all tests FAIL with `FileNotFoundError` or `AssertionError` because schema does not yet exist.

- [ ] **Step 3: Write the schema**

Create `shared/contracts/passport/literature_corpus_entry.schema.json`:

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://github.com/Imbad0202/academic-research-skills/shared/contracts/passport/literature_corpus_entry.schema.json",
  "title": "Material Passport Literature Corpus Entry",
  "description": "One entry in Material Passport Schema 9 literature_corpus[]. Represents a single piece of literature the user has included in their corpus. Produced by user-written adapters; see academic-pipeline/references/adapters/overview.md. ARS does NOT produce these entries itself.",

  "type": "object",
  "additionalProperties": false,

  "required": [
    "citation_key",
    "title",
    "authors",
    "year",
    "source_pointer"
  ],

  "properties": {
    "citation_key": {
      "type": "string",
      "minLength": 1,
      "pattern": "^[A-Za-z][A-Za-z0-9_:-]*$",
      "description": "Stable unique identifier for this entry within the passport. Typically BibTeX citekey (e.g., 'chen2024ai'). MUST be unique within literature_corpus[]. Uniqueness is enforced by the CI lint, not by JSON Schema."
    },
    "title": {
      "type": "string",
      "minLength": 1
    },
    "authors": {
      "type": "array",
      "minItems": 1,
      "items": { "$ref": "#/$defs/csl_name" },
      "description": "CSL-JSON name format. Each item is either a personal name ({family, given, ...}) or a corporate/institution name ({literal}). See CSL-JSON spec at https://docs.citationstyles.org/en/stable/specification.html#names."
    },
    "year": {
      "type": "integer",
      "minimum": 1000,
      "maximum": 2100,
      "description": "Publication year. If the adapter cannot determine a year, the entry MUST be rejected (go to rejection_log.yaml) rather than coerced to a placeholder."
    },
    "source_pointer": {
      "type": "string",
      "minLength": 1,
      "description": "Stable URI locating this work in the user's own KB. Examples: 'zotero://select/items/0_ABCD1234', 'obsidian://open?vault=kb&file=Chen2024', 'file:///path/to/refs/chen2024.pdf', 'https://doi.org/10.1234/xyz'. ARS does NOT dereference this; consumers that want the full text fetch it from the pointer themselves."
    },

    "venue":                  { "type": "string", "minLength": 1 },
    "doi": {
      "type": "string",
      "pattern": "^10\\.[0-9]{4,9}/[^\\s]+$",
      "description": "DOI without leading 'doi:' or URL prefix."
    },
    "tags": {
      "type": "array",
      "items": { "type": "string", "minLength": 1 },
      "description": "User-assigned tags from the source KB. Adapter-specific; treat as free-form."
    },

    "obtained_via": {
      "type": "string",
      "enum": ["zotero-api", "zotero-bbt-export", "obsidian-vault", "folder-scan", "manual", "other"],
      "description": "Strongly recommended. Which adapter produced this entry. 'other' is permitted for user-written custom adapters; those SHOULD also set adapter_name. Note: v3.6.4 reference implementations only use 'zotero-bbt-export', 'obsidian-vault', and 'folder-scan'. The remaining enum values ('zotero-api', 'manual') are reserved for user-written adapters and are permitted in the schema so such adapters remain valid without schema extension."
    },
    "obtained_at": {
      "type": "string",
      "format": "date-time",
      "description": "Strongly recommended. ISO 8601 timestamp when the adapter produced this entry."
    },
    "adapter_name": {
      "type": "string",
      "minLength": 1,
      "description": "Optional. Required when obtained_via='other'."
    },
    "adapter_version": {
      "type": "string",
      "minLength": 1
    },

    "abstract": {
      "type": "string",
      "description": "PRIVATE FIELD. May contain publisher-copyrighted material. Do NOT share passports containing this field publicly without confirming your right to do so. ARS consumers treat this as optional input; absence never causes failure."
    },
    "user_notes": {
      "type": "string",
      "description": "PRIVATE FIELD. User's own annotations from their KB. May contain copyrighted excerpts. Same sharing caveat as abstract."
    }
  },

  "$defs": {
    "csl_name": {
      "oneOf": [
        { "$ref": "#/$defs/csl_personal_name" },
        { "$ref": "#/$defs/csl_literal_name" }
      ]
    },
    "csl_personal_name": {
      "type": "object",
      "additionalProperties": false,
      "required": ["family"],
      "properties": {
        "family":                { "type": "string", "minLength": 1 },
        "given":                 { "type": "string" },
        "suffix":                { "type": "string" },
        "dropping-particle":     { "type": "string" },
        "non-dropping-particle": { "type": "string" },
        "comma-suffix":          { "type": ["string", "boolean"] },
        "static-ordering":       { "type": ["string", "boolean"] },
        "parse-names":           { "type": ["string", "boolean"] }
      }
    },
    "csl_literal_name": {
      "type": "object",
      "additionalProperties": false,
      "required": ["literal"],
      "properties": {
        "literal": { "type": "string", "minLength": 1 }
      }
    }
  }
}
```

- [ ] **Step 4: Run test to verify it passes**

Run:
```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_literature_corpus_entry_schema.py -v
```
Expected: all tests PASS.

- [ ] **Step 5: Commit**

```bash
cd ~/Projects/academic-research-skills
git add shared/contracts/passport/literature_corpus_entry.schema.json scripts/adapters/tests/test_literature_corpus_entry_schema.py
git commit -m "$(cat <<'EOF'
feat(v3.6.4): literature_corpus_entry JSON schema + tests

Defines one entry in Material Passport Schema 9 literature_corpus[]:
CSL-JSON author format, β required set (citation_key/title/authors/
year/source_pointer), privacy warnings on abstract + user_notes,
additionalProperties:false. See docs/design/2026-04-23-ars-v3.6.4-*
design doc §3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 3: `rejection_log.schema.json`

**Files:**
- Create: `shared/contracts/passport/rejection_log.schema.json`
- Create: `scripts/adapters/tests/test_rejection_log_schema.py`

- [ ] **Step 1: Write the failing test**

Create `scripts/adapters/tests/test_rejection_log_schema.py`:

```python
"""Validates rejection_log.schema.json self-consistency and example round-trip."""
from pathlib import Path
import json
import pytest
from jsonschema import Draft202012Validator
from jsonschema.exceptions import ValidationError

REPO_ROOT = Path(__file__).resolve().parents[3]
SCHEMA_PATH = REPO_ROOT / "shared/contracts/passport/rejection_log.schema.json"


def _load_schema():
    with SCHEMA_PATH.open("r", encoding="utf-8") as f:
        return json.load(f)


def test_schema_exists():
    assert SCHEMA_PATH.exists()


def test_schema_self_consistent():
    schema = _load_schema()
    Draft202012Validator.check_schema(schema)


def test_empty_rejected_is_valid():
    schema = _load_schema()
    log = {
        "adapter_name": "zotero.py",
        "adapter_version": "1.0.0",
        "generated_at": "2026-04-23T00:00:00Z",
        "rejected": [],
    }
    Draft202012Validator(schema).validate(log)


def test_missing_required_top_level_fails():
    schema = _load_schema()
    log = {"adapter_name": "x", "adapter_version": "1", "rejected": []}
    with pytest.raises(ValidationError):
        Draft202012Validator(schema).validate(log)


def test_rejection_with_object_raw():
    schema = _load_schema()
    log = {
        "adapter_name": "zotero.py",
        "adapter_version": "1.0.0",
        "generated_at": "2026-04-23T00:00:00Z",
        "rejected": [
            {
                "source": "0_ABCD1234",
                "reason": "missing_required_field",
                "missing_fields": ["authors"],
                "raw": {"title": "Foo", "date": "n.d."},
            }
        ],
    }
    Draft202012Validator(schema).validate(log)


def test_rejection_with_string_raw():
    schema = _load_schema()
    log = {
        "adapter_name": "folder_scan.py",
        "adapter_version": "1.0.0",
        "generated_at": "2026-04-23T00:00:00Z",
        "rejected": [
            {
                "source": "paper1.pdf",
                "reason": "authors_unparseable",
                "raw": "paper1.pdf",
            }
        ],
    }
    Draft202012Validator(schema).validate(log)


def test_rejection_with_array_raw_rejected():
    schema = _load_schema()
    log = {
        "adapter_name": "x",
        "adapter_version": "1",
        "generated_at": "2026-04-23T00:00:00Z",
        "rejected": [
            {
                "source": "s",
                "reason": "other",
                "detail": "d",
                "raw": ["not", "allowed"],
            }
        ],
    }
    with pytest.raises(ValidationError):
        Draft202012Validator(schema).validate(log)


def test_reason_enum_constrained():
    schema = _load_schema()
    log = {
        "adapter_name": "x",
        "adapter_version": "1",
        "generated_at": "2026-04-23T00:00:00Z",
        "rejected": [{"source": "s", "reason": "invented_reason"}],
    }
    with pytest.raises(ValidationError):
        Draft202012Validator(schema).validate(log)


def test_additional_properties_false_on_rejection():
    schema = _load_schema()
    log = {
        "adapter_name": "x",
        "adapter_version": "1",
        "generated_at": "2026-04-23T00:00:00Z",
        "rejected": [
            {"source": "s", "reason": "other", "detail": "d", "bogus": 1}
        ],
    }
    with pytest.raises(ValidationError):
        Draft202012Validator(schema).validate(log)
```

- [ ] **Step 2: Run test to verify it fails**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_rejection_log_schema.py -v
```
Expected: all tests FAIL (schema missing).

- [ ] **Step 3: Write the schema**

Create `shared/contracts/passport/rejection_log.schema.json`:

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://github.com/Imbad0202/academic-research-skills/shared/contracts/passport/rejection_log.schema.json",
  "title": "Literature Corpus Rejection Log",
  "description": "Output companion to passport.yaml. Records entries that the adapter could not include in literature_corpus[]. Always emitted; empty when no rejections. Consumers are not required to read this; it exists for user debugging and audit.",

  "type": "object",
  "additionalProperties": false,

  "required": ["adapter_name", "adapter_version", "generated_at", "rejected"],

  "properties": {
    "adapter_name":   { "type": "string", "minLength": 1 },
    "adapter_version":{ "type": "string", "minLength": 1 },
    "generated_at":   { "type": "string", "format": "date-time" },
    "input_source": {
      "type": "string",
      "minLength": 1,
      "description": "Optional. Human-readable description of the adapter input (path, URL, etc.)."
    },
    "rejected": {
      "type": "array",
      "items": { "$ref": "#/$defs/rejection" }
    },
    "summary": {
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "total_input":    { "type": "integer", "minimum": 0 },
        "total_accepted": { "type": "integer", "minimum": 0 },
        "total_rejected": { "type": "integer", "minimum": 0 }
      }
    }
  },

  "$defs": {
    "rejection": {
      "type": "object",
      "additionalProperties": false,
      "required": ["source", "reason"],
      "properties": {
        "source": {
          "type": "string",
          "minLength": 1,
          "description": "Where the rejected item came from. Adapter-specific: filename for folder_scan, Zotero item key for zotero, .md path for obsidian."
        },
        "reason": {
          "type": "string",
          "enum": [
            "missing_required_field",
            "invalid_field_format",
            "duplicate_citation_key",
            "unresolvable_source_pointer",
            "year_unparseable",
            "authors_unparseable",
            "adapter_error",
            "other"
          ]
        },
        "detail": {
          "type": "string",
          "description": "Free-text explanation. Required when reason='other'. Recommended otherwise."
        },
        "raw": {
          "type": ["object", "string"],
          "description": "Optional. Adapter's best-effort reproduction of the original input item. Object for structured sources; string for raw filenames or text lines."
        },
        "missing_fields": {
          "type": "array",
          "items": { "type": "string" },
          "description": "Optional but strongly recommended when reason='missing_required_field'."
        }
      }
    }
  }
}
```

- [ ] **Step 4: Run test to verify it passes**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_rejection_log_schema.py -v
```
Expected: all tests PASS.

- [ ] **Step 5: Commit**

```bash
cd ~/Projects/academic-research-skills
git add shared/contracts/passport/rejection_log.schema.json scripts/adapters/tests/test_rejection_log_schema.py
git commit -m "$(cat <<'EOF'
feat(v3.6.4): rejection_log JSON schema + tests

Companion output contract to literature_corpus_entry schema. Adapters
emit rejection_log.yaml whenever they run (empty when nothing is
rejected). reason is a closed enum of 8 categorical values; raw is
typed as object|string so downstream viewers can assume printable
shapes. See design doc §4.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 4: `scripts/check_literature_corpus_schema.py` (lint A + B)

**Files:**
- Create: `scripts/check_literature_corpus_schema.py`
- Create: `scripts/adapters/tests/test_check_literature_corpus_schema.py`

- [ ] **Step 1: Write the failing test**

Create `scripts/adapters/tests/test_check_literature_corpus_schema.py`:

```python
"""Tests for the CI lint that validates passport/rejection-log examples
against their schemas and enforces citation_key uniqueness."""
from pathlib import Path
import subprocess
import yaml
import pytest

REPO_ROOT = Path(__file__).resolve().parents[3]
SCRIPT = REPO_ROOT / "scripts/check_literature_corpus_schema.py"


def _write_yaml(tmp_path, name, data):
    p = tmp_path / name
    with p.open("w", encoding="utf-8") as f:
        yaml.safe_dump(data, f, sort_keys=True)
    return p


def _run(args, cwd=None):
    return subprocess.run(
        ["python", str(SCRIPT)] + args,
        capture_output=True,
        text=True,
        cwd=cwd or REPO_ROOT,
    )


def test_script_exists():
    assert SCRIPT.exists()


def test_passes_on_valid_passport(tmp_path):
    passport = {
        "literature_corpus": [
            {
                "citation_key": "chen2024",
                "title": "T",
                "authors": [{"family": "Chen"}],
                "year": 2024,
                "source_pointer": "file:///x.pdf",
            }
        ]
    }
    _write_yaml(tmp_path, "passport.yaml", passport)
    r = _run(["--passport", str(tmp_path / "passport.yaml")])
    assert r.returncode == 0, r.stderr


def test_fails_on_schema_violation(tmp_path):
    passport = {
        "literature_corpus": [
            {
                "citation_key": "chen2024",
                "title": "T",
                "authors": [{}],  # invalid author
                "year": 2024,
                "source_pointer": "file:///x.pdf",
            }
        ]
    }
    _write_yaml(tmp_path, "passport.yaml", passport)
    r = _run(["--passport", str(tmp_path / "passport.yaml")])
    assert r.returncode != 0
    assert "schema" in r.stderr.lower() or "validation" in r.stderr.lower()


def test_fails_on_duplicate_citation_key(tmp_path):
    passport = {
        "literature_corpus": [
            {
                "citation_key": "dup",
                "title": "A",
                "authors": [{"family": "X"}],
                "year": 2024,
                "source_pointer": "file:///a.pdf",
            },
            {
                "citation_key": "dup",
                "title": "B",
                "authors": [{"family": "Y"}],
                "year": 2024,
                "source_pointer": "file:///b.pdf",
            },
        ]
    }
    _write_yaml(tmp_path, "passport.yaml", passport)
    r = _run(["--passport", str(tmp_path / "passport.yaml")])
    assert r.returncode != 0
    assert "duplicate" in r.stderr.lower() or "unique" in r.stderr.lower()


def test_passes_on_valid_rejection_log(tmp_path):
    log = {
        "adapter_name": "zotero.py",
        "adapter_version": "1.0.0",
        "generated_at": "2026-04-23T00:00:00Z",
        "rejected": [],
    }
    _write_yaml(tmp_path, "rejection_log.yaml", log)
    r = _run(["--rejection-log", str(tmp_path / "rejection_log.yaml")])
    assert r.returncode == 0, r.stderr


def test_default_mode_scans_repo_examples():
    """With no args, script scans scripts/adapters/examples/** for expected_passport.yaml
    and expected_rejection_log.yaml and validates each. This is the CI-invoked mode."""
    r = _run([])
    # At this point the examples don't exist yet. Running under plan tasks 7-9
    # after examples are populated should exit 0. For now, return code is
    # permitted to be 0 (no files to scan) or non-zero with a helpful message.
    # The fixture tasks later enforce this more strictly via their own tests.
    assert r.returncode in (0, 1)
```

- [ ] **Step 2: Run test to verify it fails**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_check_literature_corpus_schema.py -v
```
Expected: `test_script_exists` fails because script is absent; every other test errors on subprocess invocation.

- [ ] **Step 3: Write the script**

Create `scripts/check_literature_corpus_schema.py`:

```python
#!/usr/bin/env python3
"""Validate literature_corpus_entry.schema.json and rejection_log.schema.json
self-consistency, and validate adapter example output against them.

Usage:
    python scripts/check_literature_corpus_schema.py
        Scans scripts/adapters/examples/**/expected_passport.yaml and
        **/expected_rejection_log.yaml. Validates each against its schema.
        Enforces citation_key uniqueness per passport.

    python scripts/check_literature_corpus_schema.py --passport PATH
        Validate one passport YAML only.

    python scripts/check_literature_corpus_schema.py --rejection-log PATH
        Validate one rejection log YAML only.

Exit codes:
    0 - all validations passed
    1 - one or more validations failed
    2 - invocation error (e.g., schema file missing)
"""
from __future__ import annotations
import argparse
import json
import sys
from pathlib import Path
from typing import Any

try:
    import yaml
    from jsonschema import Draft202012Validator
    from jsonschema.exceptions import ValidationError
except ImportError as e:
    print(f"Missing dependency: {e}. Install with: pip install pyyaml jsonschema", file=sys.stderr)
    sys.exit(2)

REPO_ROOT = Path(__file__).resolve().parent.parent
ENTRY_SCHEMA_PATH = REPO_ROOT / "shared/contracts/passport/literature_corpus_entry.schema.json"
REJECTION_SCHEMA_PATH = REPO_ROOT / "shared/contracts/passport/rejection_log.schema.json"
EXAMPLES_ROOT = REPO_ROOT / "scripts/adapters/examples"


def load_schema(path: Path) -> dict[str, Any]:
    if not path.exists():
        print(f"ERROR: schema missing at {path}", file=sys.stderr)
        sys.exit(2)
    with path.open("r", encoding="utf-8") as f:
        schema = json.load(f)
    try:
        Draft202012Validator.check_schema(schema)
    except Exception as e:
        print(f"ERROR: schema at {path} is not valid Draft 2020-12: {e}", file=sys.stderr)
        sys.exit(2)
    return schema


def validate_passport(path: Path, entry_schema: dict[str, Any]) -> list[str]:
    errors: list[str] = []
    with path.open("r", encoding="utf-8") as f:
        data = yaml.safe_load(f)
    if data is None:
        errors.append(f"{path}: empty YAML file")
        return errors
    corpus = data.get("literature_corpus", [])
    if not isinstance(corpus, list):
        errors.append(f"{path}: 'literature_corpus' must be a list")
        return errors
    validator = Draft202012Validator(entry_schema)
    citation_keys: dict[str, int] = {}
    for i, entry in enumerate(corpus):
        for err in validator.iter_errors(entry):
            errors.append(f"{path}: literature_corpus[{i}] schema validation error: {err.message}")
        key = entry.get("citation_key")
        if key:
            citation_keys[key] = citation_keys.get(key, 0) + 1
    for key, count in citation_keys.items():
        if count > 1:
            errors.append(f"{path}: duplicate citation_key '{key}' appears {count} times (must be unique)")
    return errors


def validate_rejection_log(path: Path, log_schema: dict[str, Any]) -> list[str]:
    errors: list[str] = []
    with path.open("r", encoding="utf-8") as f:
        data = yaml.safe_load(f)
    if data is None:
        errors.append(f"{path}: empty YAML file")
        return errors
    validator = Draft202012Validator(log_schema)
    for err in validator.iter_errors(data):
        errors.append(f"{path}: schema validation error: {err.message}")
    return errors


def scan_examples(entry_schema: dict[str, Any], log_schema: dict[str, Any]) -> list[str]:
    errors: list[str] = []
    if not EXAMPLES_ROOT.exists():
        return errors
    for passport in EXAMPLES_ROOT.glob("*/expected_passport.yaml"):
        errors.extend(validate_passport(passport, entry_schema))
    for log in EXAMPLES_ROOT.glob("*/expected_rejection_log.yaml"):
        errors.extend(validate_rejection_log(log, log_schema))
    return errors


def main() -> int:
    ap = argparse.ArgumentParser(description="Validate literature_corpus passport + rejection log YAMLs.")
    ap.add_argument("--passport", type=Path, help="Validate a single passport YAML.")
    ap.add_argument("--rejection-log", type=Path, help="Validate a single rejection log YAML.")
    args = ap.parse_args()

    entry_schema = load_schema(ENTRY_SCHEMA_PATH)
    log_schema = load_schema(REJECTION_SCHEMA_PATH)

    errors: list[str] = []
    if args.passport:
        errors.extend(validate_passport(args.passport, entry_schema))
    if args.rejection_log:
        errors.extend(validate_rejection_log(args.rejection_log, log_schema))

    if not args.passport and not args.rejection_log:
        errors.extend(scan_examples(entry_schema, log_schema))

    if errors:
        for e in errors:
            print(e, file=sys.stderr)
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main())
```

Make executable:
```bash
chmod +x scripts/check_literature_corpus_schema.py
```

- [ ] **Step 4: Run test to verify it passes**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_check_literature_corpus_schema.py -v
```
Expected: all tests PASS.

- [ ] **Step 5: Commit**

```bash
cd ~/Projects/academic-research-skills
git add scripts/check_literature_corpus_schema.py scripts/adapters/tests/test_check_literature_corpus_schema.py
git commit -m "$(cat <<'EOF'
feat(v3.6.4): CI lint for literature_corpus + rejection_log YAMLs

scripts/check_literature_corpus_schema.py: validates YAML files
against JSON schemas and enforces citation_key uniqueness. Runs in
two modes: single-file via --passport/--rejection-log flags, or the
default scan over scripts/adapters/examples/**. See design doc §6.1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 5: `scripts/adapters/_common.py` shared helpers

**Files:**
- Create: `scripts/adapters/_common.py`
- Create: `scripts/adapters/tests/test_common.py`

- [ ] **Step 1: Write the failing test**

Create `scripts/adapters/tests/test_common.py`:

```python
"""Tests for the shared helpers used by every reference adapter."""
from scripts.adapters._common import (
    sanitize_citation_key,
    make_citation_key,
    parse_csl_name,
    parse_semicolon_names,
    dump_yaml_stable,
    now_iso,
    ADAPTER_SPEC_VERSION,
)


def test_sanitize_strips_non_alnum():
    assert sanitize_citation_key("Chen, 2024!") == "Chen2024"


def test_sanitize_lowercases():
    assert sanitize_citation_key("CHEN2024").islower() is False  # we do NOT lowercase
    assert sanitize_citation_key("CHEN2024") == "CHEN2024"


def test_sanitize_rejects_empty():
    assert sanitize_citation_key("!!!") == ""


def test_make_citation_key_simple():
    existing: set[str] = set()
    key = make_citation_key(family="Chen", year=2024, title_hint="AI assessment", existing=existing)
    assert key == "chen2024ai"
    assert "chen2024ai" in existing


def test_make_citation_key_collision_suffix():
    existing: set[str] = {"chen2024ai"}
    key = make_citation_key(family="Chen", year=2024, title_hint="AI analysis", existing=existing)
    assert key == "chen2024aia"


def test_make_citation_key_multiple_collisions():
    existing: set[str] = {"chen2024", "chen2024a", "chen2024b"}
    key = make_citation_key(family="Chen", year=2024, title_hint=None, existing=existing)
    assert key == "chen2024c"


def test_parse_csl_name_family_given():
    assert parse_csl_name("Chen, Cindy") == {"family": "Chen", "given": "Cindy"}


def test_parse_csl_name_family_initial():
    assert parse_csl_name("Chen, C.") == {"family": "Chen", "given": "C."}


def test_parse_csl_name_institution_with_paren_marker():
    # Names wrapped in {} are CSL's literal-name marker (institution author).
    assert parse_csl_name("{World Health Organization}") == {"literal": "World Health Organization"}


def test_parse_csl_name_bare_single_token_literal():
    # A single token that does not look like "Family, Given" is treated as literal
    # only when followed by "(literal)" marker; otherwise treat as family name.
    assert parse_csl_name("Einstein") == {"family": "Einstein"}


def test_parse_semicolon_names():
    names = parse_semicolon_names("Chen, C.; Wang, J.")
    assert names == [
        {"family": "Chen", "given": "C."},
        {"family": "Wang", "given": "J."},
    ]


def test_parse_semicolon_names_empty_returns_empty_list():
    assert parse_semicolon_names("") == []
    assert parse_semicolon_names("   ") == []


def test_dump_yaml_stable_is_sorted():
    data = {"b": 2, "a": 1}
    out = dump_yaml_stable(data)
    # Keys serialized in sorted order
    assert out.index("a:") < out.index("b:")


def test_now_iso_is_rfc3339_z():
    ts = now_iso()
    # "2026-04-23T10:11:12Z" or with microseconds
    assert ts.endswith("Z")
    assert "T" in ts


def test_adapter_spec_version_is_string():
    assert isinstance(ADAPTER_SPEC_VERSION, str)
    assert ADAPTER_SPEC_VERSION  # non-empty
```

Also create `scripts/adapters/tests/__init__.py` (empty) and `scripts/adapters/__init__.py` (empty) if they do not exist, so the test imports resolve.

```bash
touch scripts/adapters/__init__.py
touch scripts/adapters/tests/__init__.py
```

- [ ] **Step 2: Run test to verify it fails**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_common.py -v
```
Expected: `ImportError` on `_common` module.

- [ ] **Step 3: Implement `_common.py`**

Create `scripts/adapters/_common.py`:

```python
"""Shared helpers for ARS reference adapters.

These helpers are used by folder_scan.py, zotero.py, and obsidian.py.
They are also what a user-written adapter would copy and adapt.
"""
from __future__ import annotations
import datetime
import io
import re
from typing import Any

import yaml

ADAPTER_SPEC_VERSION = "1.0.0"  # bump when overview.md contract changes

_NON_ALNUM = re.compile(r"[^A-Za-z0-9]+")
_TITLE_WORD = re.compile(r"[A-Za-z]+")
_STOPWORDS = {
    "a", "an", "the", "of", "in", "on", "at", "by", "for", "to", "with",
    "and", "or", "but", "is", "are", "was", "were",
}


def sanitize_citation_key(raw: str) -> str:
    """Strip non-alphanumeric characters. Does NOT lowercase."""
    return _NON_ALNUM.sub("", raw)


def _first_title_word_non_stopword(title_hint: str | None) -> str:
    if not title_hint:
        return ""
    for word in _TITLE_WORD.findall(title_hint):
        if word.lower() not in _STOPWORDS:
            return word.lower()
    return ""


def make_citation_key(
    *, family: str, year: int, title_hint: str | None, existing: set[str]
) -> str:
    """Build a collision-safe citation key.

    Pattern: {family_lower}{year}{first_title_word_or_empty}. If that is
    already in `existing`, append `a`, `b`, `c`, ... (and then aa, ab, ...).
    The chosen key is added to `existing` by reference.
    """
    base = f"{family.lower()}{year}{_first_title_word_non_stopword(title_hint)}"
    base = sanitize_citation_key(base)
    if not base:
        base = "ref"

    if base not in existing:
        existing.add(base)
        return base

    # Collision — append suffix
    for suffix in _alpha_suffixes():
        candidate = f"{base}{suffix}"
        if candidate not in existing:
            existing.add(candidate)
            return candidate
    raise RuntimeError("exhausted citation key suffix space")  # practically unreachable


def _alpha_suffixes():
    """Yield 'a', 'b', ..., 'z', 'aa', 'ab', ..., 'zz'. Enough for plan purposes."""
    for c in "abcdefghijklmnopqrstuvwxyz":
        yield c
    for c1 in "abcdefghijklmnopqrstuvwxyz":
        for c2 in "abcdefghijklmnopqrstuvwxyz":
            yield c1 + c2


def parse_csl_name(raw: str) -> dict[str, str]:
    """Parse a single name string into a CSL-JSON name object.

    Rules:
    - "{Institution Name}" → {"literal": "Institution Name"}
    - "Family, Given" → {"family": "Family", "given": "Given"}
    - "Family" (single token) → {"family": "Family"}
    """
    raw = raw.strip()
    if raw.startswith("{") and raw.endswith("}"):
        return {"literal": raw[1:-1].strip()}
    if "," in raw:
        family, _, given = raw.partition(",")
        return {"family": family.strip(), "given": given.strip()}
    return {"family": raw}


def parse_semicolon_names(raw: str) -> list[dict[str, str]]:
    """Parse 'A, B; C, D' into CSL name list."""
    raw = raw.strip()
    if not raw:
        return []
    return [parse_csl_name(part) for part in raw.split(";") if part.strip()]


def dump_yaml_stable(data: Any) -> str:
    """Dump YAML with sorted keys and deterministic formatting."""
    buf = io.StringIO()
    yaml.safe_dump(
        data,
        buf,
        sort_keys=True,
        default_flow_style=False,
        allow_unicode=True,
        width=1000,
    )
    return buf.getvalue()


def now_iso() -> str:
    """ISO 8601 in UTC with trailing 'Z'. Deterministic except for the time itself."""
    return datetime.datetime.now(datetime.UTC).strftime("%Y-%m-%dT%H:%M:%SZ")


def write_passport(path, entries: list[dict[str, Any]]) -> None:
    """Write passport.yaml with sorted literature_corpus entries."""
    sorted_entries = sorted(entries, key=lambda e: e.get("citation_key", ""))
    doc = {"literature_corpus": sorted_entries}
    path.write_text(dump_yaml_stable(doc), encoding="utf-8")


def write_rejection_log(
    path, *, adapter_name: str, adapter_version: str,
    rejected: list[dict[str, Any]], input_source: str | None = None,
) -> None:
    """Write rejection_log.yaml with sorted rejected[] by source."""
    sorted_rejected = sorted(rejected, key=lambda r: r.get("source", ""))
    doc: dict[str, Any] = {
        "adapter_name": adapter_name,
        "adapter_version": adapter_version,
        "generated_at": now_iso(),
        "rejected": sorted_rejected,
    }
    if input_source is not None:
        doc["input_source"] = input_source
    summary = {
        "total_accepted": doc.get("_accepted_hint", 0) if False else 0,  # set by caller when known
        "total_rejected": len(sorted_rejected),
    }
    doc["summary"] = {"total_rejected": len(sorted_rejected)}
    path.write_text(dump_yaml_stable(doc), encoding="utf-8")
```

- [ ] **Step 4: Run test to verify it passes**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_common.py -v
```
Expected: all tests PASS. If `test_now_iso_is_rfc3339_z` fails on Python 3.10 or lower (datetime.UTC missing), upgrade to 3.11.

- [ ] **Step 5: Commit**

```bash
cd ~/Projects/academic-research-skills
git add scripts/adapters/__init__.py scripts/adapters/tests/__init__.py scripts/adapters/_common.py scripts/adapters/tests/test_common.py
git commit -m "$(cat <<'EOF'
feat(v3.6.4): adapter _common helpers (CSL parsing, YAML emit, citekey)

sanitize_citation_key, make_citation_key (with collision suffix),
parse_csl_name (personal / institution / literal variants),
parse_semicolon_names, dump_yaml_stable, now_iso, write_passport,
write_rejection_log. Reusable across folder_scan / zotero / obsidian
and a starting point for user-written adapters.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 6: `scripts/adapters/tests/conftest.py` test infrastructure

**Files:**
- Create: `scripts/adapters/tests/conftest.py`

- [ ] **Step 1: Write the conftest**

Create `scripts/adapters/tests/conftest.py`:

```python
"""pytest configuration + shared fixtures for adapter tests.

Provides:
- `repo_root` : Path to the ARS repo root
- `adapters_dir` : Path to scripts/adapters/
- `examples_dir` : Path to scripts/adapters/examples/
- `clean_timestamps(doc)` : helper that blanks `generated_at` and
   `obtained_at` fields so byte-identical comparisons are possible.
"""
from __future__ import annotations
from pathlib import Path
import copy
import pytest
import yaml

REPO_ROOT = Path(__file__).resolve().parents[3]


@pytest.fixture(scope="session")
def repo_root() -> Path:
    return REPO_ROOT


@pytest.fixture(scope="session")
def adapters_dir() -> Path:
    return REPO_ROOT / "scripts/adapters"


@pytest.fixture(scope="session")
def examples_dir() -> Path:
    return REPO_ROOT / "scripts/adapters/examples"


def _strip_keys(obj, keys_to_blank: set[str]):
    if isinstance(obj, dict):
        return {
            k: ("<TIMESTAMP>" if k in keys_to_blank else _strip_keys(v, keys_to_blank))
            for k, v in obj.items()
        }
    if isinstance(obj, list):
        return [_strip_keys(v, keys_to_blank) for v in obj]
    return obj


@pytest.fixture
def clean_timestamps():
    """Return a function that returns a copy of a dict/list with all
    `generated_at` and `obtained_at` values blanked to '<TIMESTAMP>'. Used to
    compare adapter output against expected fixtures without flakiness."""
    def _clean(doc):
        return _strip_keys(copy.deepcopy(doc), {"generated_at", "obtained_at"})
    return _clean


@pytest.fixture
def load_yaml():
    """Return a function that loads a YAML file into a Python dict."""
    def _load(path: Path):
        with path.open("r", encoding="utf-8") as f:
            return yaml.safe_load(f)
    return _load
```

- [ ] **Step 2: Verify conftest works**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/ -v --collect-only
```
Expected: tests from tasks 2-5 still collect; no import errors in conftest.

- [ ] **Step 3: Commit**

```bash
cd ~/Projects/academic-research-skills
git add scripts/adapters/tests/conftest.py
git commit -m "$(cat <<'EOF'
feat(v3.6.4): pytest conftest with shared fixtures

repo_root / adapters_dir / examples_dir paths, clean_timestamps
helper for flake-free byte-compares against expected fixtures, and a
YAML loader. Used by all three adapter test modules.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 7: `folder_scan` adapter + tests + fixtures

**Files:**
- Create: `scripts/adapters/folder_scan.py`
- Create: `scripts/adapters/tests/test_folder_scan.py`
- Create: `scripts/adapters/examples/folder_scan/input_fixture/Chen2024_AIAssessment.pdf` (empty file)
- Create: `scripts/adapters/examples/folder_scan/input_fixture/Wang_2023_formative_feedback.pdf` (empty file)
- Create: `scripts/adapters/examples/folder_scan/input_fixture/paper1.pdf` (empty file — will be rejected)
- Create: `scripts/adapters/examples/folder_scan/input_fixture/中文檔名_2024.pdf` (empty file — will be rejected: unicode)
- Create: `scripts/adapters/examples/folder_scan/expected_passport.yaml`
- Create: `scripts/adapters/examples/folder_scan/expected_rejection_log.yaml`

- [ ] **Step 1: Write the failing test**

Create `scripts/adapters/tests/test_folder_scan.py`:

```python
"""Tests for scripts/adapters/folder_scan.py."""
from pathlib import Path
import subprocess
import pytest

REPO_ROOT = Path(__file__).resolve().parents[3]
ADAPTER = REPO_ROOT / "scripts/adapters/folder_scan.py"
FIXTURE_DIR = REPO_ROOT / "scripts/adapters/examples/folder_scan/input_fixture"
EXPECTED_PASSPORT = REPO_ROOT / "scripts/adapters/examples/folder_scan/expected_passport.yaml"
EXPECTED_REJECTION = REPO_ROOT / "scripts/adapters/examples/folder_scan/expected_rejection_log.yaml"


def _run(*args, cwd=None):
    return subprocess.run(
        ["python", str(ADAPTER)] + list(args),
        capture_output=True,
        text=True,
        cwd=cwd or REPO_ROOT,
    )


def test_adapter_exists():
    assert ADAPTER.exists()


def test_happy_path(tmp_path, load_yaml, clean_timestamps):
    passport_out = tmp_path / "passport.yaml"
    rejection_out = tmp_path / "rejection_log.yaml"
    r = _run(
        "--input", str(FIXTURE_DIR),
        "--passport", str(passport_out),
        "--rejection-log", str(rejection_out),
    )
    assert r.returncode == 0, r.stderr

    got = load_yaml(passport_out)
    expected = load_yaml(EXPECTED_PASSPORT)
    assert clean_timestamps(got) == clean_timestamps(expected)

    got_rej = load_yaml(rejection_out)
    expected_rej = load_yaml(EXPECTED_REJECTION)
    assert clean_timestamps(got_rej) == clean_timestamps(expected_rej)


def test_empty_folder_emits_empty_passport(tmp_path):
    empty = tmp_path / "empty"
    empty.mkdir()
    passport_out = tmp_path / "p.yaml"
    rej_out = tmp_path / "r.yaml"
    r = _run(
        "--input", str(empty),
        "--passport", str(passport_out),
        "--rejection-log", str(rej_out),
    )
    assert r.returncode == 0
    import yaml
    with passport_out.open() as f:
        doc = yaml.safe_load(f)
    assert doc == {"literature_corpus": []}


def test_missing_input_dir_fails_loud(tmp_path):
    r = _run(
        "--input", str(tmp_path / "does-not-exist"),
        "--passport", str(tmp_path / "p.yaml"),
        "--rejection-log", str(tmp_path / "r.yaml"),
    )
    assert r.returncode == 1
    assert "not found" in r.stderr.lower() or "exist" in r.stderr.lower()


def test_deterministic_output(tmp_path, load_yaml, clean_timestamps):
    p1 = tmp_path / "p1.yaml"
    r1_log = tmp_path / "r1.yaml"
    p2 = tmp_path / "p2.yaml"
    r2_log = tmp_path / "r2.yaml"
    _run("--input", str(FIXTURE_DIR), "--passport", str(p1), "--rejection-log", str(r1_log))
    _run("--input", str(FIXTURE_DIR), "--passport", str(p2), "--rejection-log", str(r2_log))
    assert clean_timestamps(load_yaml(p1)) == clean_timestamps(load_yaml(p2))
    assert clean_timestamps(load_yaml(r1_log)) == clean_timestamps(load_yaml(r2_log))


def test_duplicate_collision_handled(tmp_path):
    # Two files that would produce the same base citation_key should each be
    # accepted with a disambiguating suffix.
    dup_dir = tmp_path / "dup"
    dup_dir.mkdir()
    (dup_dir / "Smith2024_alpha.pdf").touch()
    (dup_dir / "Smith2024_beta.pdf").touch()
    passport_out = tmp_path / "p.yaml"
    rej_out = tmp_path / "r.yaml"
    r = _run(
        "--input", str(dup_dir),
        "--passport", str(passport_out),
        "--rejection-log", str(rej_out),
    )
    assert r.returncode == 0
    import yaml
    with passport_out.open() as f:
        doc = yaml.safe_load(f)
    keys = {e["citation_key"] for e in doc["literature_corpus"]}
    assert len(keys) == 2  # no collisions
```

- [ ] **Step 2: Create fixture files (empty PDFs)**

```bash
cd ~/Projects/academic-research-skills
touch scripts/adapters/examples/folder_scan/input_fixture/Chen2024_AIAssessment.pdf
touch scripts/adapters/examples/folder_scan/input_fixture/Wang_2023_formative_feedback.pdf
touch scripts/adapters/examples/folder_scan/input_fixture/paper1.pdf
touch "scripts/adapters/examples/folder_scan/input_fixture/中文檔名_2024.pdf"
rm scripts/adapters/examples/folder_scan/input_fixture/.gitkeep
```

- [ ] **Step 3: Create expected fixtures**

Create `scripts/adapters/examples/folder_scan/expected_passport.yaml`:

```yaml
literature_corpus:
- adapter_name: folder_scan.py
  adapter_version: 1.0.0
  authors:
  - family: Chen
  citation_key: chen2024aiassessment
  obtained_at: '<TIMESTAMP>'
  obtained_via: folder-scan
  source_pointer: file:///FIXTURE/Chen2024_AIAssessment.pdf
  title: Chen2024 AIAssessment
  year: 2024
- adapter_name: folder_scan.py
  adapter_version: 1.0.0
  authors:
  - family: Wang
  citation_key: wang2023formative
  obtained_at: '<TIMESTAMP>'
  obtained_via: folder-scan
  source_pointer: file:///FIXTURE/Wang_2023_formative_feedback.pdf
  title: Wang 2023 formative feedback
  year: 2023
```

> **Note:** `source_pointer` values will use absolute paths when the adapter actually runs. The test compares after running so paths will resolve to the actual fixture directory. The `<TIMESTAMP>` marker is blanked by `clean_timestamps`. Adjust `source_pointer: file:///FIXTURE/...` to match what the adapter actually emits, which is the absolute path under the repo. The test strategy is to also blank `source_pointer` absolute prefixes in a more lenient comparison. Simpler: compare only `clean_timestamps`-stripped copies, and in the adapter, emit a path relative to the `--input` directory using `file:./relative/path`.
>
> **Plan decision:** adapter emits `file://` with absolute path of `.resolve()` of each file. The expected fixture will be regenerated relative to the actual fixture dir at test time via a helper; do NOT hardcode the path. The simpler path: the test's `clean_timestamps` helper is extended to also blank `source_pointer`. Update conftest accordingly in Step 4 below before running.

**Revised Step 3:** extend `conftest.py` `clean_timestamps` fixture to also blank `source_pointer`:

Edit `scripts/adapters/tests/conftest.py`: change
```python
def _clean(doc):
    return _strip_keys(copy.deepcopy(doc), {"generated_at", "obtained_at"})
```
to
```python
def _clean(doc):
    return _strip_keys(copy.deepcopy(doc), {"generated_at", "obtained_at", "source_pointer"})
```

Then the expected fixture can stick with any placeholder for `source_pointer`.

Create `scripts/adapters/examples/folder_scan/expected_passport.yaml` (final form):

```yaml
literature_corpus:
- adapter_name: folder_scan.py
  adapter_version: 1.0.0
  authors:
  - family: Chen
  citation_key: chen2024aiassessment
  obtained_at: <TIMESTAMP>
  obtained_via: folder-scan
  source_pointer: <STRIPPED>
  title: Chen2024 AIAssessment
  year: 2024
- adapter_name: folder_scan.py
  adapter_version: 1.0.0
  authors:
  - family: Wang
  citation_key: wang2023formative
  obtained_at: <TIMESTAMP>
  obtained_via: folder-scan
  source_pointer: <STRIPPED>
  title: Wang 2023 formative feedback
  year: 2023
```

Create `scripts/adapters/examples/folder_scan/expected_rejection_log.yaml`:

```yaml
adapter_name: folder_scan.py
adapter_version: 1.0.0
generated_at: <TIMESTAMP>
rejected:
- missing_fields:
  - authors
  - year
  raw: paper1.pdf
  reason: authors_unparseable
  source: paper1.pdf
- missing_fields:
  - authors
  raw: "中文檔名_2024.pdf"
  reason: authors_unparseable
  source: "中文檔名_2024.pdf"
summary:
  total_rejected: 2
```

- [ ] **Step 4: Run test to verify it fails**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_folder_scan.py -v
```
Expected: fails — adapter does not exist.

- [ ] **Step 5: Implement `folder_scan.py`**

Create `scripts/adapters/folder_scan.py`:

```python
#!/usr/bin/env python3
"""folder_scan: produce a literature_corpus passport from a directory of
files (typically PDFs). Parses citation metadata from filenames using
best-effort conventions. Never parses PDF content.

Filename conventions recognized:
  - {Family}{Year}{optional_title_slug}.{ext}
  - {Family}_{Year}_{title_slug}.{ext}
  - fallback: first capitalized Latin word before the year.

Files whose filename cannot be parsed for both family and year are rejected.
Non-Latin filenames (e.g. CJK characters) are rejected; Unicode filename
support is a future extension opportunity.

Usage:
  python scripts/adapters/folder_scan.py \
      --input <dir> --passport <out.yaml> --rejection-log <out.yaml>
"""
from __future__ import annotations
import argparse
import re
import sys
from pathlib import Path

from scripts.adapters._common import (
    ADAPTER_SPEC_VERSION,
    make_citation_key,
    write_passport,
    write_rejection_log,
    now_iso,
)

ADAPTER_NAME = "folder_scan.py"
ADAPTER_VERSION = "1.0.0"

# Family{Year} style: "Chen2024..."
RE_FAMILY_YEAR = re.compile(r"^([A-Z][A-Za-z]*)\s*((?:19|20)\d{2})\s*[_\-.\s]?(.*?)\.[A-Za-z0-9]+$")
# Family_Year_title style
RE_FAMILY_UNDERSCORE = re.compile(r"^([A-Z][A-Za-z]*)_((?:19|20)\d{2})(?:_(.*?))?\.[A-Za-z0-9]+$")
# fallback year anywhere
RE_ANY_YEAR = re.compile(r"((?:19|20)\d{2})")
RE_FIRST_CAPITAL = re.compile(r"\b([A-Z][A-Za-z]+)\b")

# Reject entries with any non-ASCII char in the filename stem
RE_NON_ASCII = re.compile(r"[^\x00-\x7f]")


def parse_filename(name: str) -> dict | None:
    """Return {'family', 'year', 'title'} or None if unparseable."""
    if RE_NON_ASCII.search(name):
        return None

    m = RE_FAMILY_UNDERSCORE.match(name)
    if m:
        family, year, tail = m.group(1), int(m.group(2)), (m.group(3) or "").strip()
        title = tail.replace("_", " ").strip() or f"{family} {year} {tail}".strip()
        return {"family": family, "year": year, "title": f"{family} {year} {tail}".strip()}

    m = RE_FAMILY_YEAR.match(name)
    if m:
        family, year, tail = m.group(1), int(m.group(2)), (m.group(3) or "").strip()
        stem = Path(name).stem
        return {"family": family, "year": year, "title": stem.replace("_", " ")}

    year_match = RE_ANY_YEAR.search(name)
    fam_match = RE_FIRST_CAPITAL.search(name.split(str(int(year_match.group(1))))[0] if year_match else "")
    if year_match and fam_match:
        return {
            "family": fam_match.group(1),
            "year": int(year_match.group(1)),
            "title": Path(name).stem.replace("_", " "),
        }
    return None


def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("--input", type=Path, required=True, help="Directory to scan")
    ap.add_argument("--passport", type=Path, required=True)
    ap.add_argument("--rejection-log", dest="rejection_log", type=Path, required=True)
    args = ap.parse_args()

    if not args.input.exists() or not args.input.is_dir():
        print(f"ERROR: input directory not found: {args.input}", file=sys.stderr)
        return 1

    entries: list[dict] = []
    rejected: list[dict] = []
    existing_keys: set[str] = set()

    files = sorted(p for p in args.input.rglob("*") if p.is_file() and p.name != ".gitkeep")
    for f in files:
        parsed = parse_filename(f.name)
        if not parsed:
            missing = []
            if RE_NON_ASCII.search(f.name):
                missing = ["authors"]
            elif not RE_ANY_YEAR.search(f.name):
                missing = ["authors", "year"]
            else:
                missing = ["authors"]
            rejected.append({
                "source": f.name,
                "reason": "authors_unparseable",
                "raw": f.name,
                "missing_fields": missing,
            })
            continue

        title_hint = parsed["title"]
        citation_key = make_citation_key(
            family=parsed["family"],
            year=parsed["year"],
            title_hint=title_hint,
            existing=existing_keys,
        )
        entries.append({
            "citation_key": citation_key,
            "title": title_hint,
            "authors": [{"family": parsed["family"]}],
            "year": parsed["year"],
            "source_pointer": f"file://{f.resolve()}",
            "obtained_via": "folder-scan",
            "obtained_at": now_iso(),
            "adapter_name": ADAPTER_NAME,
            "adapter_version": ADAPTER_VERSION,
        })

    write_passport(args.passport, entries)
    write_rejection_log(
        args.rejection_log,
        adapter_name=ADAPTER_NAME,
        adapter_version=ADAPTER_VERSION,
        rejected=rejected,
        input_source=str(args.input),
    )
    return 0


if __name__ == "__main__":
    sys.exit(main())
```

Make executable:
```bash
chmod +x scripts/adapters/folder_scan.py
```

- [ ] **Step 6: Run test to verify it passes**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_folder_scan.py -v
```

Expected: all tests PASS. If `test_happy_path` fails because expected_passport.yaml content does not match (e.g., title format differences), regenerate the expected fixture by running the adapter once on the fixture dir and hand-copying the output (after verifying it makes sense).

- [ ] **Step 7: Run the full CI lint against the new example**

```bash
cd ~/Projects/academic-research-skills
python scripts/check_literature_corpus_schema.py
```
Expected: exit 0. The new example passport validates against the schema.

- [ ] **Step 8: Commit**

```bash
cd ~/Projects/academic-research-skills
git add scripts/adapters/folder_scan.py scripts/adapters/tests/test_folder_scan.py scripts/adapters/examples/folder_scan/ scripts/adapters/tests/conftest.py
git commit -m "$(cat <<'EOF'
feat(v3.6.4): folder_scan reference adapter + tests + fixture

Filesystem adapter. Parses citation metadata from filenames using two
conventions ({Family}{Year}... and {Family}_{Year}_{title}...) plus a
fallback. Non-ASCII filenames rejected (Unicode support deferred).
Also expands conftest clean_timestamps to strip source_pointer so
tests remain stable across machines. See design doc §5.2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 8: `zotero` adapter + tests + fixtures

**Files:**
- Create: `scripts/adapters/zotero.py`
- Create: `scripts/adapters/tests/test_zotero.py`
- Create: `scripts/adapters/examples/zotero/input_fixture/export.json`
- Create: `scripts/adapters/examples/zotero/expected_passport.yaml`
- Create: `scripts/adapters/examples/zotero/expected_rejection_log.yaml`

- [ ] **Step 1: Write the failing test**

Create `scripts/adapters/tests/test_zotero.py`:

```python
"""Tests for scripts/adapters/zotero.py."""
from pathlib import Path
import subprocess
import json
import yaml
import pytest

REPO_ROOT = Path(__file__).resolve().parents[3]
ADAPTER = REPO_ROOT / "scripts/adapters/zotero.py"
FIXTURE_INPUT = REPO_ROOT / "scripts/adapters/examples/zotero/input_fixture/export.json"
EXPECTED_PASSPORT = REPO_ROOT / "scripts/adapters/examples/zotero/expected_passport.yaml"
EXPECTED_REJECTION = REPO_ROOT / "scripts/adapters/examples/zotero/expected_rejection_log.yaml"


def _run(*args):
    return subprocess.run(
        ["python", str(ADAPTER)] + list(args),
        capture_output=True,
        text=True,
        cwd=REPO_ROOT,
    )


def test_adapter_exists():
    assert ADAPTER.exists()


def test_happy_path(tmp_path, load_yaml, clean_timestamps):
    p = tmp_path / "p.yaml"
    r = tmp_path / "r.yaml"
    res = _run("--input", str(FIXTURE_INPUT), "--passport", str(p), "--rejection-log", str(r))
    assert res.returncode == 0, res.stderr
    assert clean_timestamps(load_yaml(p)) == clean_timestamps(load_yaml(EXPECTED_PASSPORT))
    assert clean_timestamps(load_yaml(r)) == clean_timestamps(load_yaml(EXPECTED_REJECTION))


def test_malformed_json_fails_loud(tmp_path):
    bad = tmp_path / "bad.json"
    bad.write_text("{not json", encoding="utf-8")
    p = tmp_path / "p.yaml"
    r = tmp_path / "r.yaml"
    res = _run("--input", str(bad), "--passport", str(p), "--rejection-log", str(r))
    assert res.returncode == 1
    assert "json" in res.stderr.lower() or "parse" in res.stderr.lower()


def test_empty_array_emits_empty_passport(tmp_path, load_yaml):
    empty = tmp_path / "empty.json"
    empty.write_text("[]", encoding="utf-8")
    p = tmp_path / "p.yaml"
    r = tmp_path / "r.yaml"
    res = _run("--input", str(empty), "--passport", str(p), "--rejection-log", str(r))
    assert res.returncode == 0
    doc = load_yaml(p)
    assert doc == {"literature_corpus": []}


def test_institution_author_preserved(tmp_path, load_yaml):
    data = [
        {
            "citationKey": "who2024",
            "itemType": "report",
            "title": "World report",
            "creators": [{"creatorType": "author", "name": "World Health Organization"}],
            "date": "2024",
            "itemID": "AAAA1111",
        }
    ]
    infile = tmp_path / "i.json"
    infile.write_text(json.dumps(data), encoding="utf-8")
    p = tmp_path / "p.yaml"
    r = tmp_path / "r.yaml"
    res = _run("--input", str(infile), "--passport", str(p), "--rejection-log", str(r))
    assert res.returncode == 0
    doc = load_yaml(p)
    assert doc["literature_corpus"][0]["authors"] == [{"literal": "World Health Organization"}]


def test_no_authors_rejected(tmp_path, load_yaml):
    data = [
        {
            "citationKey": "anon2024",
            "itemType": "journalArticle",
            "title": "Untitled",
            "creators": [{"creatorType": "editor", "firstName": "A", "lastName": "B"}],
            "date": "2024",
            "itemID": "BBBB1111",
        }
    ]
    infile = tmp_path / "i.json"
    infile.write_text(json.dumps(data), encoding="utf-8")
    p = tmp_path / "p.yaml"
    r = tmp_path / "r.yaml"
    res = _run("--input", str(infile), "--passport", str(p), "--rejection-log", str(r))
    assert res.returncode == 0
    pp = load_yaml(p)
    rr = load_yaml(r)
    assert pp["literature_corpus"] == []
    assert len(rr["rejected"]) == 1
    assert rr["rejected"][0]["reason"] == "authors_unparseable"


def test_unparseable_date_rejected(tmp_path, load_yaml):
    data = [
        {
            "citationKey": "fwd2024",
            "itemType": "journalArticle",
            "title": "Forthcoming",
            "creators": [{"creatorType": "author", "firstName": "A", "lastName": "B"}],
            "date": "forthcoming",
            "itemID": "CCCC1111",
        }
    ]
    infile = tmp_path / "i.json"
    infile.write_text(json.dumps(data), encoding="utf-8")
    p = tmp_path / "p.yaml"
    r = tmp_path / "r.yaml"
    res = _run("--input", str(infile), "--passport", str(p), "--rejection-log", str(r))
    assert res.returncode == 0
    rr = load_yaml(r)
    assert rr["rejected"][0]["reason"] == "year_unparseable"
```

- [ ] **Step 2: Create fixture `export.json`**

Create `scripts/adapters/examples/zotero/input_fixture/export.json`:

```json
[
  {
    "citationKey": "chen2024ai",
    "itemType": "journalArticle",
    "title": "AI assessment in higher education",
    "creators": [
      {"creatorType": "author", "firstName": "Cindy", "lastName": "Chen"},
      {"creatorType": "author", "firstName": "Jing", "lastName": "Wang"}
    ],
    "date": "2024",
    "publicationTitle": "Assessment & Evaluation",
    "DOI": "10.1234/xyz",
    "tags": [{"tag": "assessment"}, {"tag": "ai"}],
    "itemID": "ABCD1234",
    "abstractNote": "An abstract that is publisher-copyrighted."
  },
  {
    "citationKey": "who2024report",
    "itemType": "report",
    "title": "World higher-education report",
    "creators": [
      {"creatorType": "author", "name": "World Health Organization"}
    ],
    "date": "2024-03",
    "itemID": "EFGH5678"
  },
  {
    "citationKey": "forthcoming2024",
    "itemType": "journalArticle",
    "title": "To appear",
    "creators": [{"creatorType": "author", "firstName": "X", "lastName": "Y"}],
    "date": "forthcoming",
    "itemID": "IJKL9012"
  },
  {
    "citationKey": "editor2023",
    "itemType": "book",
    "title": "Edited volume",
    "creators": [{"creatorType": "editor", "firstName": "E", "lastName": "Z"}],
    "date": "2023",
    "itemID": "MNOP3456"
  }
]
```

- [ ] **Step 3: Write expected output fixtures**

Create `scripts/adapters/examples/zotero/expected_passport.yaml`:

```yaml
literature_corpus:
- adapter_name: zotero.py
  adapter_version: 1.0.0
  abstract: An abstract that is publisher-copyrighted.
  authors:
  - family: Chen
    given: Cindy
  - family: Wang
    given: Jing
  citation_key: chen2024ai
  doi: 10.1234/xyz
  obtained_at: <TIMESTAMP>
  obtained_via: zotero-bbt-export
  source_pointer: <STRIPPED>
  tags:
  - assessment
  - ai
  title: AI assessment in higher education
  venue: Assessment & Evaluation
  year: 2024
- adapter_name: zotero.py
  adapter_version: 1.0.0
  authors:
  - literal: World Health Organization
  citation_key: who2024report
  obtained_at: <TIMESTAMP>
  obtained_via: zotero-bbt-export
  source_pointer: <STRIPPED>
  title: World higher-education report
  year: 2024
```

Create `scripts/adapters/examples/zotero/expected_rejection_log.yaml`:

```yaml
adapter_name: zotero.py
adapter_version: 1.0.0
generated_at: <TIMESTAMP>
rejected:
- missing_fields:
  - authors
  raw:
    citationKey: editor2023
    creators:
    - creatorType: editor
      firstName: E
      lastName: Z
    date: '2023'
    itemID: MNOP3456
    itemType: book
    title: Edited volume
  reason: authors_unparseable
  source: MNOP3456
- missing_fields:
  - year
  raw:
    citationKey: forthcoming2024
    creators:
    - creatorType: author
      firstName: X
      lastName: Y
    date: forthcoming
    itemID: IJKL9012
    itemType: journalArticle
    title: To appear
  reason: year_unparseable
  source: IJKL9012
summary:
  total_rejected: 2
```

- [ ] **Step 4: Run test to verify it fails**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_zotero.py -v
```

- [ ] **Step 5: Implement `zotero.py`**

Create `scripts/adapters/zotero.py`:

```python
#!/usr/bin/env python3
"""zotero: produce a literature_corpus passport from a Better BibTeX
Extension JSON export. Reads only local files; does not call the Zotero
Web API.

See design doc §5.3. Users who want a live-sync API-based variant are
expected to write their own adapter using this file as a starting point
(see overview.md extension-point guidance).

Usage:
  python scripts/adapters/zotero.py \
      --input <bbt_export.json> --passport <out.yaml> --rejection-log <out.yaml>
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path

from scripts.adapters._common import (
    write_passport,
    write_rejection_log,
    now_iso,
)

ADAPTER_NAME = "zotero.py"
ADAPTER_VERSION = "1.0.0"

RE_YEAR = re.compile(r"\b((?:19|20)\d{2})\b")


def extract_authors(creators: list[dict]) -> list[dict] | None:
    """Pull only author-type creators. Return CSL-name list or None when no authors."""
    out: list[dict] = []
    for c in creators or []:
        if c.get("creatorType") != "author":
            continue
        if "name" in c:  # institution / corporate author
            out.append({"literal": c["name"].strip()})
            continue
        family = (c.get("lastName") or "").strip()
        given = (c.get("firstName") or "").strip()
        if not family:
            continue
        entry = {"family": family}
        if given:
            entry["given"] = given
        out.append(entry)
    return out if out else None


def extract_year(date_str: str) -> int | None:
    if not date_str:
        return None
    m = RE_YEAR.search(date_str)
    if m:
        return int(m.group(1))
    return None


def strip_doi(doi: str | None) -> str | None:
    if not doi:
        return None
    doi = doi.strip()
    for prefix in ("doi:", "DOI:", "https://doi.org/", "http://doi.org/"):
        if doi.lower().startswith(prefix.lower()):
            doi = doi[len(prefix):]
            break
    return doi.strip() or None


def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("--input", type=Path, required=True, help="BBT JSON export file")
    ap.add_argument("--passport", type=Path, required=True)
    ap.add_argument("--rejection-log", dest="rejection_log", type=Path, required=True)
    args = ap.parse_args()

    try:
        raw = args.input.read_text(encoding="utf-8")
    except FileNotFoundError:
        print(f"ERROR: input not found: {args.input}", file=sys.stderr)
        return 1
    try:
        data = json.loads(raw)
    except json.JSONDecodeError as e:
        print(f"ERROR: input is not valid JSON: {e}", file=sys.stderr)
        return 1
    if not isinstance(data, list):
        print(f"ERROR: expected top-level JSON array, got {type(data).__name__}", file=sys.stderr)
        return 1

    entries: list[dict] = []
    rejected: list[dict] = []

    for item in data:
        citekey = item.get("citationKey") or ""
        item_id = item.get("itemID") or ""
        source_key = item_id or citekey or "<unknown>"

        authors = extract_authors(item.get("creators", []))
        year = extract_year(item.get("date", ""))

        missing: list[str] = []
        if not authors:
            missing.append("authors")
        if not year:
            missing.append("year")

        if missing:
            reason = "authors_unparseable" if "authors" in missing else "year_unparseable"
            rejected.append({
                "source": source_key,
                "reason": reason,
                "raw": item,
                "missing_fields": missing,
            })
            continue

        venue = (
            item.get("publicationTitle")
            or item.get("proceedingsTitle")
            or item.get("bookTitle")
            or None
        )

        entry: dict = {
            "citation_key": citekey,
            "title": item.get("title", "").strip(),
            "authors": authors,
            "year": year,
            "source_pointer": f"zotero://select/items/0_{item_id}" if item_id else f"zotero://select/items/@{citekey}",
            "obtained_via": "zotero-bbt-export",
            "obtained_at": now_iso(),
            "adapter_name": ADAPTER_NAME,
            "adapter_version": ADAPTER_VERSION,
        }
        if venue:
            entry["venue"] = venue
        doi = strip_doi(item.get("DOI"))
        if doi:
            entry["doi"] = doi
        tags = [t.get("tag") for t in item.get("tags", []) if t.get("tag")]
        if tags:
            entry["tags"] = tags
        abstract = item.get("abstractNote")
        if abstract:
            entry["abstract"] = abstract
        notes = item.get("notes") or []
        if notes:
            plain = "\n\n".join(re.sub(r"<[^>]+>", "", n.get("note", "")) for n in notes if n.get("note"))
            if plain.strip():
                entry["user_notes"] = plain

        entries.append(entry)

    write_passport(args.passport, entries)
    write_rejection_log(
        args.rejection_log,
        adapter_name=ADAPTER_NAME,
        adapter_version=ADAPTER_VERSION,
        rejected=rejected,
        input_source=str(args.input),
    )
    return 0


if __name__ == "__main__":
    sys.exit(main())
```

Make executable:
```bash
chmod +x scripts/adapters/zotero.py
```

- [ ] **Step 6: Run test to verify it passes**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_zotero.py -v
python scripts/check_literature_corpus_schema.py
```
Expected: all tests PASS; schema lint passes.

- [ ] **Step 7: Commit**

```bash
cd ~/Projects/academic-research-skills
git add scripts/adapters/zotero.py scripts/adapters/tests/test_zotero.py scripts/adapters/examples/zotero/
git commit -m "$(cat <<'EOF'
feat(v3.6.4): zotero reference adapter + tests + fixture

Reads Better BibTeX JSON export (local file). Pulls creators with
type=author (institution via literal-name variant), extracts year via
4-digit regex (rejects forthcoming / n.d.), maps abstractNote and
notes to private fields. Does NOT call the Zotero Web API. See
design doc §5.3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 9: `obsidian` adapter + tests + fixtures

**Files:**
- Create: `scripts/adapters/obsidian.py`
- Create: `scripts/adapters/tests/test_obsidian.py`
- Create: `scripts/adapters/examples/obsidian/input_fixture/vault/chen2024ai.md` (Convention A)
- Create: `scripts/adapters/examples/obsidian/input_fixture/vault/wang2023formative.md` (Convention B)
- Create: `scripts/adapters/examples/obsidian/input_fixture/vault/invalid.md` (rejected)
- Create: `scripts/adapters/examples/obsidian/input_fixture/vault/_templates/tmpl.md` (skipped)
- Create: `scripts/adapters/examples/obsidian/input_fixture/vault/.obsidian/app.json` (skipped)
- Create: `scripts/adapters/examples/obsidian/expected_passport.yaml`
- Create: `scripts/adapters/examples/obsidian/expected_rejection_log.yaml`

- [ ] **Step 1: Write the failing test**

Create `scripts/adapters/tests/test_obsidian.py`:

```python
"""Tests for scripts/adapters/obsidian.py."""
from pathlib import Path
import subprocess
import pytest

REPO_ROOT = Path(__file__).resolve().parents[3]
ADAPTER = REPO_ROOT / "scripts/adapters/obsidian.py"
VAULT = REPO_ROOT / "scripts/adapters/examples/obsidian/input_fixture/vault"
EXPECTED_PASSPORT = REPO_ROOT / "scripts/adapters/examples/obsidian/expected_passport.yaml"
EXPECTED_REJECTION = REPO_ROOT / "scripts/adapters/examples/obsidian/expected_rejection_log.yaml"


def _run(*args):
    return subprocess.run(
        ["python", str(ADAPTER)] + list(args),
        capture_output=True,
        text=True,
        cwd=REPO_ROOT,
    )


def test_adapter_exists():
    assert ADAPTER.exists()


def test_happy_path(tmp_path, load_yaml, clean_timestamps):
    p = tmp_path / "p.yaml"
    r = tmp_path / "r.yaml"
    res = _run("--input", str(VAULT), "--passport", str(p), "--rejection-log", str(r))
    assert res.returncode == 0, res.stderr
    assert clean_timestamps(load_yaml(p)) == clean_timestamps(load_yaml(EXPECTED_PASSPORT))
    assert clean_timestamps(load_yaml(r)) == clean_timestamps(load_yaml(EXPECTED_REJECTION))


def test_missing_vault_fails_loud(tmp_path):
    p = tmp_path / "p.yaml"
    r = tmp_path / "r.yaml"
    res = _run("--input", str(tmp_path / "nope"), "--passport", str(p), "--rejection-log", str(r))
    assert res.returncode == 1


def test_templates_dir_skipped(tmp_path, load_yaml):
    vault = tmp_path / "v"
    (vault / "_templates").mkdir(parents=True)
    (vault / "_templates/tmpl.md").write_text(
        "---\ncitekey: tmpl2024\ntitle: T\nauthors:\n  - family: X\nyear: 2024\n---\n",
        encoding="utf-8",
    )
    (vault / "note.md").write_text(
        "---\ncitekey: note2024\ntitle: Real\nauthors:\n  - family: Y\nyear: 2024\n---\nbody\n",
        encoding="utf-8",
    )
    p = tmp_path / "p.yaml"
    r = tmp_path / "r.yaml"
    res = _run("--input", str(vault), "--passport", str(p), "--rejection-log", str(r))
    assert res.returncode == 0
    doc = load_yaml(p)
    keys = [e["citation_key"] for e in doc["literature_corpus"]]
    assert "tmpl2024" not in keys
    assert "note2024" in keys


def test_obsidian_dir_skipped(tmp_path, load_yaml):
    vault = tmp_path / "v"
    (vault / ".obsidian").mkdir(parents=True)
    (vault / ".obsidian/ignored.md").write_text(
        "---\ncitekey: skip2024\ntitle: T\nauthors:\n  - family: X\nyear: 2024\n---\n",
        encoding="utf-8",
    )
    (vault / "real.md").write_text(
        "---\ncitekey: real2024\ntitle: R\nauthors:\n  - family: Y\nyear: 2024\n---\n",
        encoding="utf-8",
    )
    p = tmp_path / "p.yaml"
    r = tmp_path / "r.yaml"
    _run("--input", str(vault), "--passport", str(p), "--rejection-log", str(r))
    doc = load_yaml(p)
    keys = [e["citation_key"] for e in doc["literature_corpus"]]
    assert "skip2024" not in keys
    assert "real2024" in keys
```

- [ ] **Step 2: Create vault fixture files**

```bash
cd ~/Projects/academic-research-skills
mkdir -p scripts/adapters/examples/obsidian/input_fixture/vault/_templates
mkdir -p scripts/adapters/examples/obsidian/input_fixture/vault/.obsidian
rm scripts/adapters/examples/obsidian/input_fixture/vault/.gitkeep
```

Create `scripts/adapters/examples/obsidian/input_fixture/vault/chen2024ai.md`:

```markdown
---
citekey: chen2024ai
title: AI assessment in higher education
authors:
  - family: Chen
    given: Cindy
year: 2024
venue: Assessment & Evaluation
doi: 10.1234/xyz
tags: [assessment, ai]
---

Body note content. This may contain copyrighted excerpts.
```

Create `scripts/adapters/examples/obsidian/input_fixture/vault/wang2023formative.md`:

```markdown
---
source: https://doi.org/10.5678/abc
---

# Formative feedback in classrooms

**Authors**: Wang, J.; Lin, S.
**Year**: 2023

Body content here.
```

Create `scripts/adapters/examples/obsidian/input_fixture/vault/invalid.md`:

```markdown
This file has no frontmatter and no parseable H1. It should be rejected.
```

Create `scripts/adapters/examples/obsidian/input_fixture/vault/_templates/tmpl.md`:

```markdown
---
citekey: tmpl2024
title: Template — should be skipped
authors:
  - family: Ignore
year: 2024
---
```

Create `scripts/adapters/examples/obsidian/input_fixture/vault/.obsidian/app.json`:

```json
{"promptDelete": false}
```

- [ ] **Step 3: Write expected outputs**

Create `scripts/adapters/examples/obsidian/expected_passport.yaml`:

```yaml
literature_corpus:
- adapter_name: obsidian.py
  adapter_version: 1.0.0
  authors:
  - family: Chen
    given: Cindy
  citation_key: chen2024ai
  doi: 10.1234/xyz
  obtained_at: <TIMESTAMP>
  obtained_via: obsidian-vault
  source_pointer: <STRIPPED>
  tags:
  - assessment
  - ai
  title: AI assessment in higher education
  user_notes: |
    Body note content. This may contain copyrighted excerpts.
  venue: Assessment & Evaluation
  year: 2024
- adapter_name: obsidian.py
  adapter_version: 1.0.0
  authors:
  - family: Wang
    given: J.
  - family: Lin
    given: S.
  citation_key: wang2023formative
  obtained_at: <TIMESTAMP>
  obtained_via: obsidian-vault
  source_pointer: <STRIPPED>
  title: Formative feedback in classrooms
  user_notes: |
    Body content here.
  year: 2023
```

Create `scripts/adapters/examples/obsidian/expected_rejection_log.yaml`:

```yaml
adapter_name: obsidian.py
adapter_version: 1.0.0
generated_at: <TIMESTAMP>
rejected:
- missing_fields:
  - authors
  - year
  raw: invalid.md
  reason: authors_unparseable
  source: invalid.md
summary:
  total_rejected: 1
```

- [ ] **Step 4: Run test to verify it fails**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_obsidian.py -v
```

- [ ] **Step 5: Implement `obsidian.py`**

Create `scripts/adapters/obsidian.py`:

```python
#!/usr/bin/env python3
"""obsidian: produce a literature_corpus passport from an Obsidian vault.

Convention A (preferred): frontmatter carries citekey/title/authors/year/...
Convention B (Karpathy-style): filename is the citekey, body has an H1 title
and a "**Authors**: A, B.; C, D." line, year from **Year**: line or frontmatter
source URL.

Files under _templates/ and .obsidian/ are skipped. Files that match neither
convention are rejected.

Usage:
  python scripts/adapters/obsidian.py \
      --input <vault_dir> --passport <out.yaml> --rejection-log <out.yaml>
"""
from __future__ import annotations
import argparse
import re
import sys
import urllib.parse
from pathlib import Path
from typing import Any

import yaml

from scripts.adapters._common import (
    write_passport,
    write_rejection_log,
    parse_semicolon_names,
    now_iso,
)

ADAPTER_NAME = "obsidian.py"
ADAPTER_VERSION = "1.0.0"

SKIP_DIR_NAMES = {"_templates", ".obsidian"}

RE_FRONTMATTER = re.compile(r"^---\n(.*?)\n---\n(.*)$", re.DOTALL)
RE_H1 = re.compile(r"^#\s+(.+)$", re.MULTILINE)
RE_AUTHORS_LINE = re.compile(r"^\*\*Authors\*\*:\s*(.+)$", re.MULTILINE)
RE_YEAR_LINE = re.compile(r"^\*\*Year\*\*:\s*((?:19|20)\d{2})\s*$", re.MULTILINE)
RE_YEAR_IN_SOURCE = re.compile(r"((?:19|20)\d{2})")


def split_frontmatter(content: str) -> tuple[dict | None, str]:
    m = RE_FRONTMATTER.match(content)
    if not m:
        return None, content
    try:
        fm = yaml.safe_load(m.group(1)) or {}
    except yaml.YAMLError:
        return None, content
    if not isinstance(fm, dict):
        return None, content
    return fm, m.group(2)


def build_pointer(vault_name: str, rel_path: Path) -> str:
    return f"obsidian://open?vault={urllib.parse.quote(vault_name)}&file={urllib.parse.quote(str(rel_path.with_suffix('')))}"


def process_file(md_path: Path, vault_root: Path, vault_name: str) -> tuple[dict | None, dict | None]:
    """Return (accepted_entry, rejection) — exactly one is None."""
    rel = md_path.relative_to(vault_root)
    raw = md_path.read_text(encoding="utf-8")
    fm, body = split_frontmatter(raw)

    # Convention A: frontmatter has citekey
    if fm and "citekey" in fm:
        missing = [k for k in ("citekey", "title", "authors", "year") if not fm.get(k)]
        if missing:
            return None, {
                "source": str(rel),
                "reason": "missing_required_field",
                "missing_fields": missing,
                "raw": {"frontmatter": fm, "path": str(rel)},
            }
        entry = {
            "citation_key": str(fm["citekey"]),
            "title": str(fm["title"]),
            "authors": fm["authors"],
            "year": int(fm["year"]),
            "source_pointer": build_pointer(vault_name, rel),
            "obtained_via": "obsidian-vault",
            "obtained_at": now_iso(),
            "adapter_name": ADAPTER_NAME,
            "adapter_version": ADAPTER_VERSION,
        }
        for opt in ("venue", "doi", "tags"):
            if fm.get(opt):
                entry[opt] = fm[opt]
        if body.strip():
            entry["user_notes"] = body.strip() + "\n"
        return entry, None

    # Convention B: derive from filename + body H1 + "**Authors**:" line
    citekey = md_path.stem
    title_match = RE_H1.search(body) if body else None
    authors_line = RE_AUTHORS_LINE.search(body) if body else None
    year_match = RE_YEAR_LINE.search(body) if body else None

    missing: list[str] = []
    title = title_match.group(1).strip() if title_match else None
    authors = parse_semicolon_names(authors_line.group(1).strip()) if authors_line else None
    year: int | None = None
    if year_match:
        year = int(year_match.group(1))
    elif fm and fm.get("source"):
        ys = RE_YEAR_IN_SOURCE.search(str(fm["source"]))
        if ys:
            year = int(ys.group(1))

    if not title:
        missing.append("title")
    if not authors:
        missing.append("authors")
    if not year:
        missing.append("year")

    if missing:
        return None, {
            "source": str(rel),
            "reason": "authors_unparseable" if "authors" in missing else "missing_required_field",
            "missing_fields": missing,
            "raw": str(rel),
        }

    entry = {
        "citation_key": citekey,
        "title": title,
        "authors": authors,
        "year": year,
        "source_pointer": build_pointer(vault_name, rel),
        "obtained_via": "obsidian-vault",
        "obtained_at": now_iso(),
        "adapter_name": ADAPTER_NAME,
        "adapter_version": ADAPTER_VERSION,
    }
    # Strip the Authors / Year lines from user_notes body
    body_clean = RE_AUTHORS_LINE.sub("", body) if body else ""
    body_clean = RE_YEAR_LINE.sub("", body_clean)
    # Strip H1 line
    body_clean = RE_H1.sub("", body_clean, count=1)
    body_clean = body_clean.strip()
    if body_clean:
        entry["user_notes"] = body_clean + "\n"
    return entry, None


def iter_markdown_files(vault: Path):
    for md in sorted(vault.rglob("*.md")):
        # Skip if any parent dir name is in SKIP_DIR_NAMES
        if any(p in SKIP_DIR_NAMES or p.startswith(".") for p in md.relative_to(vault).parts[:-1]):
            continue
        if md.name.startswith("."):
            continue
        yield md


def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("--input", type=Path, required=True)
    ap.add_argument("--passport", type=Path, required=True)
    ap.add_argument("--rejection-log", dest="rejection_log", type=Path, required=True)
    args = ap.parse_args()

    if not args.input.exists() or not args.input.is_dir():
        print(f"ERROR: vault directory not found: {args.input}", file=sys.stderr)
        return 1

    vault_name = args.input.name
    entries: list[dict] = []
    rejected: list[dict] = []

    for md in iter_markdown_files(args.input):
        accepted, rejection = process_file(md, args.input, vault_name)
        if accepted:
            entries.append(accepted)
        elif rejection:
            rejected.append(rejection)

    write_passport(args.passport, entries)
    write_rejection_log(
        args.rejection_log,
        adapter_name=ADAPTER_NAME,
        adapter_version=ADAPTER_VERSION,
        rejected=rejected,
        input_source=str(args.input),
    )
    return 0


if __name__ == "__main__":
    sys.exit(main())
```

Make executable:
```bash
chmod +x scripts/adapters/obsidian.py
```

- [ ] **Step 6: Run test to verify it passes**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_obsidian.py -v
python scripts/check_literature_corpus_schema.py
```
Expected: all tests PASS; schema lint passes.

- [ ] **Step 7: Commit**

```bash
cd ~/Projects/academic-research-skills
git add scripts/adapters/obsidian.py scripts/adapters/tests/test_obsidian.py scripts/adapters/examples/obsidian/
git commit -m "$(cat <<'EOF'
feat(v3.6.4): obsidian reference adapter + tests + fixture

Reads an Obsidian vault directory. Supports Convention A (frontmatter
with citekey/title/authors/year) and Convention B (Karpathy-style:
filename=citekey, body H1 title, **Authors**: semicolon names, year
from **Year** line or source URL). Skips _templates/ and .obsidian/
directories. Wikilinks are NOT resolved; they appear as literal text
in user_notes. See design doc §5.4.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 10: `overview.md` language-neutral adapter contract

**Files:**
- Create: `academic-pipeline/references/adapters/overview.md`

- [ ] **Step 1: Write the file**

Create `academic-pipeline/references/adapters/overview.md`:

```markdown
# Adapter Contract — Producing `literature_corpus[]` Entries

**Status**: Stable as of ARS v3.6.4
**Applies to**: any adapter that produces a Material Passport `literature_corpus[]` field
**Authoritative schemas**:
- [`shared/contracts/passport/literature_corpus_entry.schema.json`](../../../shared/contracts/passport/literature_corpus_entry.schema.json)
- [`shared/contracts/passport/rejection_log.schema.json`](../../../shared/contracts/passport/rejection_log.schema.json)

## 1. What an adapter is

An **adapter** is a program (in any language) that reads a user-owned corpus source — a Zotero export, an Obsidian vault, a folder of PDFs, a Notion database, a custom SQLite library — and produces:

1. **`passport.yaml`** with a top-level `literature_corpus` array conforming to `literature_corpus_entry.schema.json`.
2. **`rejection_log.yaml`** conforming to `rejection_log.schema.json`, always emitted (empty when no rejections).

ARS provides three Python reference adapters in [`scripts/adapters/`](../../../scripts/adapters/). Users are expected to write their own adapters for non-reference sources. The three reference adapters are starting points, not production tools.

## 2. Why a contract, not a plugin API

ARS deliberately does NOT run adapter code itself. The adapter runs in the user's environment, reads the user's data, and emits YAML files. ARS reads those YAML files. Rationale:

- Each corpus source (Zotero, Obsidian, custom) has its own versioning, auth, and data quirks. Pinning one plugin API would force ARS to track all of them.
- Keeping the boundary at "YAML files on disk" means a user can use any language, any scheduling mechanism, and any privacy posture without touching ARS internals.
- ARS remains a writing/review-layer tool; corpus integration stays in user-owned code.

## 3. Entry field reference

Refer to the [`literature_corpus_entry` schema](../../../shared/contracts/passport/literature_corpus_entry.schema.json) for the authoritative contract. The field tables below are generated from that schema and must not drift.

<!-- GENERATED:LITERATURE_CORPUS_REQUIRED:START -->
_Regenerated by `scripts/sync_adapter_docs.py` from the schema._
<!-- GENERATED:LITERATURE_CORPUS_REQUIRED:END -->

<!-- GENERATED:LITERATURE_CORPUS_OPTIONAL:START -->
_Regenerated by `scripts/sync_adapter_docs.py` from the schema._
<!-- GENERATED:LITERATURE_CORPUS_OPTIONAL:END -->

### 3.1 `authors` format (CSL-JSON names)

Each entry in `authors[]` is one of:

- **Personal name**: `{family: "Chen", given: "Cindy"}`. `family` is required; `given` and CSL particles (`suffix`, `dropping-particle`, `non-dropping-particle`, `comma-suffix`, `static-ordering`, `parse-names`) are optional.
- **Institution / corporate name**: `{literal: "World Health Organization"}`. `literal` is required; no other fields.

Adapters SHOULD preserve the upstream distinction (personal vs. institution). See [CSL-JSON name spec](https://docs.citationstyles.org/en/stable/specification.html#names) for edge cases.

### 3.2 Privacy caveat for `abstract` and `user_notes`

Publishers typically retain rights to abstracts; user notes often quote copyrighted material. The schema marks these fields as PRIVATE and does NOT enforce anything at the CI level. **If you publish a passport (e.g., commit it to a public repo or share it on the web), you are responsible for removing or clearing these fields.** ARS consumers treat both fields as optional; omitting them never causes failure.

## 4. Rejection log

Whenever an adapter cannot produce a valid `literature_corpus_entry` for an input item, it MUST push a rejection into `rejection_log.yaml.rejected[]` and MUST NOT silently drop the item.

The `reason` field uses a closed enum of categorical values:

| `reason` | When |
|----------|------|
| `missing_required_field` | One or more required fields (citation_key / title / authors / year / source_pointer) cannot be derived. Adapter SHOULD also populate `missing_fields`. |
| `invalid_field_format` | A field is present but fails schema format (e.g., DOI pattern mismatch). |
| `duplicate_citation_key` | Another entry already used this citekey and the adapter cannot disambiguate. |
| `unresolvable_source_pointer` | The source URI points to something that does not exist. Adapters typically do not check this; use only if the adapter actually tries to resolve. |
| `year_unparseable` | Year cannot be extracted from the source field ("n.d.", "forthcoming", "Spring 2024"). |
| `authors_unparseable` | Authors field is empty, contains only non-author creators, or has unparseable content. |
| `adapter_error` | Adapter-internal bug, not an input data problem. Use sparingly. |
| `other` | Anything else. When `reason=other`, `detail` is REQUIRED. |

The `raw` field, when present, MUST be either an object (structured input) or a string (filename or text line). Arrays, numbers, booleans, and null are disallowed so downstream viewers can assume printable shapes.

## 5. Error handling (fail-soft)

**Entry-level problems** → push to rejection log, continue.
**Adapter-level problems** (input file missing, unreadable, malformed at the root) → write nothing, print a clear error to stderr, exit with code 1.

Do NOT emit a partial passport. Either the passport represents a complete scan of the user's input, or the adapter failed loudly.

## 6. Determinism

Running an adapter twice on identical input MUST produce byte-identical output except for:

- `generated_at` in the rejection log
- `obtained_at` on each entry

Achieve this by:

- Sorting `literature_corpus[]` by `citation_key`
- Sorting `rejection_log.rejected[]` by `source`
- Serializing YAML with sorted keys

## 7. Provenance

Each `literature_corpus` entry SHOULD carry `obtained_via` and `obtained_at`. The `rejection_log` MUST carry `adapter_name`, `adapter_version`, and `generated_at`. These fields let downstream users trace which adapter version produced which entry — useful when a library's schema or a source KB layout changes.

## 8. Extension points for user-written adapters

Custom adapters are welcome and expected. Recommended conventions:

- Set `obtained_via: "other"` on each entry, and set `adapter_name` to a clear string (e.g., `"notion-adapter-v1"`, `"my-custom-sqlite-reader"`).
- Set `adapter_version` to a semver-ish string so downstream tools can diagnose output changes.
- Follow the same CLI shape as the three reference adapters (`--input`, `--passport`, `--rejection-log`) if you want to slot into similar tooling.

Common user-written adapter families include:

- **Zotero Web API**: one that calls `api.zotero.org` with a user API token. Not shipped with ARS by design (auth and rate-limit management would pull ARS into data-layer territory).
- **Notion / Readwise / Airtable**: each has its own SDK; the adapter's job is only to map the source's book/article objects into `literature_corpus_entry` shape.
- **Cross-source merger**: reads multiple sources and emits a combined passport. Handle citekey collisions explicitly.

## 9. Testing your adapter

Before committing passport output to any workflow:

1. Validate against the schema: `python scripts/check_literature_corpus_schema.py --passport <your_passport.yaml> --rejection-log <your_rejection_log.yaml>`.
2. Run your adapter twice on the same input and diff the outputs (after stripping `generated_at` / `obtained_at`). Any non-timestamp difference is a determinism bug.
3. Feed a known-bad entry to confirm it lands in the rejection log rather than silently vanishing or crashing.

The three reference adapters have pytest coverage under `scripts/adapters/tests/` — copying that pattern for your own adapter is a good starting point.

## 10. Relationship to other ARS artifacts

- [`shared/handoff_schemas.md`](../../../shared/handoff_schemas.md) Schema 9: the `literature_corpus[]` field lives inside the Material Passport.
- [`academic-pipeline/references/passport_as_reset_boundary.md`](../passport_as_reset_boundary.md): `literature_corpus[]` is consumed across reset boundaries like any other passport field.
- ARS agents that consume `literature_corpus[]` are **deferred** to v3.6.5+. As of v3.6.4, the field is a defined input port with no runtime consumer; adapters produce it, future ARS versions read it.
```

- [ ] **Step 2: Verify the file exists and markdown renders**

```bash
cd ~/Projects/academic-research-skills
cat academic-pipeline/references/adapters/overview.md | head -50
```
Expected: header + TOC render correctly.

- [ ] **Step 3: Commit**

```bash
cd ~/Projects/academic-research-skills
git add academic-pipeline/references/adapters/overview.md
git commit -m "$(cat <<'EOF'
docs(v3.6.4): adapter contract overview.md

Language-neutral contract for any adapter producing
literature_corpus[] entries. Covers field semantics, rejection log
conventions, fail-soft error handling, determinism, provenance,
extension points for user-written adapters, testing guidance, and
relationship to other ARS artifacts. Generated table markers for
sync_adapter_docs.py to fill in next task. See design doc §5.1 + §6.2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 11: `scripts/sync_adapter_docs.py` (lint C)

**Files:**
- Create: `scripts/sync_adapter_docs.py`
- Create: `scripts/adapters/tests/test_sync_adapter_docs.py`
- Modify: `academic-pipeline/references/adapters/overview.md` (fill generated tables)

- [ ] **Step 1: Write the failing test**

Create `scripts/adapters/tests/test_sync_adapter_docs.py`:

```python
"""Tests for sync_adapter_docs.py: schema → overview.md drift detector."""
from pathlib import Path
import subprocess
import pytest

REPO_ROOT = Path(__file__).resolve().parents[3]
SCRIPT = REPO_ROOT / "scripts/sync_adapter_docs.py"
OVERVIEW = REPO_ROOT / "academic-pipeline/references/adapters/overview.md"


def _run(*args):
    return subprocess.run(
        ["python", str(SCRIPT)] + list(args),
        capture_output=True,
        text=True,
        cwd=REPO_ROOT,
    )


def test_script_exists():
    assert SCRIPT.exists()


def test_regenerate_is_idempotent(tmp_path):
    # First, run without --check to regenerate in place
    r1 = _run()
    assert r1.returncode == 0, r1.stderr
    # Running --check after regeneration must succeed (no drift)
    r2 = _run("--check")
    assert r2.returncode == 0, r2.stderr


def test_check_detects_drift(tmp_path):
    # Save current overview content, deliberately corrupt a generated section,
    # verify --check exits non-zero, then restore.
    original = OVERVIEW.read_text(encoding="utf-8")
    try:
        marker_start = "<!-- GENERATED:LITERATURE_CORPUS_REQUIRED:START -->"
        marker_end = "<!-- GENERATED:LITERATURE_CORPUS_REQUIRED:END -->"
        i = original.index(marker_start)
        j = original.index(marker_end) + len(marker_end)
        corrupted = original[:i] + marker_start + "\nDELIBERATELY OUT-OF-DATE\n" + marker_end + original[j:]
        OVERVIEW.write_text(corrupted, encoding="utf-8")
        r = _run("--check")
        assert r.returncode == 1
    finally:
        OVERVIEW.write_text(original, encoding="utf-8")


def test_no_markers_no_op(tmp_path):
    # A file with no markers is a no-op; script exits 0.
    alt = tmp_path / "alt.md"
    alt.write_text("# no markers here\n", encoding="utf-8")
    r = _run("--target", str(alt))
    assert r.returncode == 0
```

- [ ] **Step 2: Run test to verify it fails**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_sync_adapter_docs.py -v
```
Expected: fails — script missing.

- [ ] **Step 3: Implement `sync_adapter_docs.py`**

Create `scripts/sync_adapter_docs.py`:

```python
#!/usr/bin/env python3
"""sync_adapter_docs: keep academic-pipeline/references/adapters/overview.md
field tables in lockstep with literature_corpus_entry.schema.json.

Markers in overview.md:
  <!-- GENERATED:LITERATURE_CORPUS_REQUIRED:START -->  ... auto ...  <!-- GENERATED:LITERATURE_CORPUS_REQUIRED:END -->
  <!-- GENERATED:LITERATURE_CORPUS_OPTIONAL:START -->  ... auto ...  <!-- GENERATED:LITERATURE_CORPUS_OPTIONAL:END -->

Modes:
  sync_adapter_docs.py           # rewrite the marked regions in place
  sync_adapter_docs.py --check   # fail if rewrite would change anything
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path

REPO_ROOT = Path(__file__).resolve().parent.parent
SCHEMA_PATH = REPO_ROOT / "shared/contracts/passport/literature_corpus_entry.schema.json"
DEFAULT_TARGET = REPO_ROOT / "academic-pipeline/references/adapters/overview.md"

MARKERS = {
    "LITERATURE_CORPUS_REQUIRED": "required",
    "LITERATURE_CORPUS_OPTIONAL": "optional",
}


def _short_type(prop_def: dict) -> str:
    t = prop_def.get("type")
    if isinstance(t, list):
        return " | ".join(sorted(x for x in t if x))
    if t:
        return t
    if "oneOf" in prop_def:
        return "oneOf"
    if "$ref" in prop_def:
        return prop_def["$ref"].split("/")[-1]
    return "any"


def _short_desc(prop_def: dict) -> str:
    d = (prop_def.get("description") or "").strip()
    # Use just the first sentence for the table
    m = re.match(r"(.+?[.!?])(?:\s|$)", d)
    return (m.group(1) if m else d) or "—"


def build_table(schema: dict, which: str) -> str:
    required = set(schema.get("required", []))
    props = schema.get("properties", {})
    rows = []
    for name, prop_def in sorted(props.items()):
        is_req = name in required
        if which == "required" and not is_req:
            continue
        if which == "optional" and is_req:
            continue
        rows.append(f"| `{name}` | {_short_type(prop_def)} | {_short_desc(prop_def)} |")
    header = "| Field | Type | Description (first sentence) |\n|---|---|---|"
    return header + "\n" + "\n".join(rows) + "\n"


def regenerate_file(target: Path, schema: dict) -> str:
    content = target.read_text(encoding="utf-8")
    for marker_name, which in MARKERS.items():
        start = f"<!-- GENERATED:{marker_name}:START -->"
        end = f"<!-- GENERATED:{marker_name}:END -->"
        if start not in content or end not in content:
            continue
        table = build_table(schema, which)
        pattern = re.compile(re.escape(start) + r".*?" + re.escape(end), re.DOTALL)
        content = pattern.sub(f"{start}\n{table}{end}", content)
    return content


def main() -> int:
    ap = argparse.ArgumentParser()
    ap.add_argument("--check", action="store_true", help="Exit 1 on drift, do not modify files.")
    ap.add_argument("--target", type=Path, default=DEFAULT_TARGET, help="Overview file to update.")
    args = ap.parse_args()

    if not SCHEMA_PATH.exists():
        print(f"ERROR: schema missing at {SCHEMA_PATH}", file=sys.stderr)
        return 2
    if not args.target.exists():
        print(f"ERROR: target missing at {args.target}", file=sys.stderr)
        return 2

    schema = json.loads(SCHEMA_PATH.read_text(encoding="utf-8"))
    old = args.target.read_text(encoding="utf-8")
    new = regenerate_file(args.target, schema)

    if old == new:
        return 0

    if args.check:
        print(f"DRIFT: {args.target} is out-of-date relative to the schema.", file=sys.stderr)
        print("Run `python scripts/sync_adapter_docs.py` to regenerate.", file=sys.stderr)
        return 1
    args.target.write_text(new, encoding="utf-8")
    return 0


if __name__ == "__main__":
    sys.exit(main())
```

- [ ] **Step 4: First-time regenerate overview.md**

```bash
cd ~/Projects/academic-research-skills
python scripts/sync_adapter_docs.py
python scripts/sync_adapter_docs.py --check  # should now exit 0
```
Expected: overview.md now contains populated field tables; second invocation is a no-op.

- [ ] **Step 5: Run tests**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/test_sync_adapter_docs.py -v
```

- [ ] **Step 6: Commit**

```bash
cd ~/Projects/academic-research-skills
chmod +x scripts/sync_adapter_docs.py
git add scripts/sync_adapter_docs.py scripts/adapters/tests/test_sync_adapter_docs.py academic-pipeline/references/adapters/overview.md
git commit -m "$(cat <<'EOF'
feat(v3.6.4): sync_adapter_docs schema→overview.md drift detector

scripts/sync_adapter_docs.py regenerates the field tables in
academic-pipeline/references/adapters/overview.md from the
literature_corpus_entry schema. --check mode exits 1 on drift for
CI gating. First regeneration populates the overview tables. See
design doc §6.2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 12: CI workflow additions

**Files:**
- Create: `.github/workflows/pytest.yml`
- Modify: existing CI workflow to add `check_literature_corpus_schema.py` and `sync_adapter_docs.py --check` steps

- [ ] **Step 1: Inspect existing CI workflows**

```bash
cd ~/Projects/academic-research-skills
ls .github/workflows/
```
Expected: a set of `.yml` files. Identify the one that already runs `check_passport_reset_contract.py` — the new lint steps will live in the same file.

- [ ] **Step 2: Read the existing workflow file**

```bash
cd ~/Projects/academic-research-skills
grep -l "check_passport_reset_contract" .github/workflows/*.yml
# Then open the matching file for inspection.
```

- [ ] **Step 3: Add two steps to the existing workflow**

Use the Edit tool to add — immediately before the existing `check_passport_reset_contract.py` step — these two steps:

```yaml
      - name: Validate literature_corpus schemas and examples
        run: python scripts/check_literature_corpus_schema.py

      - name: Check adapter docs up-to-date
        run: python scripts/sync_adapter_docs.py --check
```

- [ ] **Step 4: Create new `pytest.yml`**

Create `.github/workflows/pytest.yml`:

```yaml
name: pytest

on:
  pull_request:
    paths:
      - 'scripts/adapters/**'
      - 'scripts/check_literature_corpus_schema.py'
      - 'scripts/sync_adapter_docs.py'
      - 'shared/contracts/passport/**'
  push:
    branches: [main]
    paths:
      - 'scripts/adapters/**'
      - 'scripts/check_literature_corpus_schema.py'
      - 'scripts/sync_adapter_docs.py'
      - 'shared/contracts/passport/**'

jobs:
  pytest:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - name: Install deps
        run: |
          python -m pip install --upgrade pip
          pip install pytest pyyaml jsonschema
      - name: Run adapter tests
        run: pytest scripts/adapters/tests/ -v
```

- [ ] **Step 5: Syntactic check of workflow yaml**

```bash
cd ~/Projects/academic-research-skills
python -c "import yaml; yaml.safe_load(open('.github/workflows/pytest.yml'))"
```
Expected: no output, exit 0.

- [ ] **Step 6: Commit**

```bash
cd ~/Projects/academic-research-skills
git add .github/workflows/pytest.yml .github/workflows/<existing>.yml
git commit -m "$(cat <<'EOF'
ci(v3.6.4): add literature_corpus schema + adapter docs + pytest CI

New pytest.yml runs scripts/adapters/tests/ on adapter and schema
changes only (paths filter). Existing workflow gains two steps:
check_literature_corpus_schema.py for schema/example validation and
sync_adapter_docs.py --check for doc drift prevention.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 13: Schema 9 extension + retire "deferred" caveats

**Files:**
- Modify: `shared/handoff_schemas.md`
- Modify: `academic-pipeline/references/passport_as_reset_boundary.md`

- [ ] **Step 1: Extend Schema 9 in handoff_schemas.md**

Open `shared/handoff_schemas.md`. Find the Schema 9 "Optional Fields" table (around line 495). Add a new row:

```
| `literature_corpus` | list[object] | Optional append-friendly literature corpus. Each entry conforms to [`shared/contracts/passport/literature_corpus_entry.schema.json`](contracts/passport/literature_corpus_entry.schema.json). Produced by user-written adapters (see [`academic-pipeline/references/adapters/overview.md`](../academic-pipeline/references/adapters/overview.md)); ARS does not produce these entries itself. Added v3.6.4+. |
```

After the Reset Boundary Extension section (around line 570), add a new subsection:

```markdown
### Literature Corpus Input Port (v3.6.4)

The optional `literature_corpus[]` field is Schema 9's input port for user-owned literature. Each entry is a bibliographic record conforming to `literature_corpus_entry.schema.json` (CSL-JSON author format, β required set).

ARS does not produce these entries. User-written adapters read their own corpus source (Zotero, Obsidian, folder, Notion, etc.) and emit a passport with `literature_corpus[]` populated. Three reference adapters ship with v3.6.4 under `scripts/adapters/`.

Consumer-side integration (agents that actually READ `literature_corpus[]`) is deferred to v3.6.5+. v3.6.4 defines the input port only.

See [`academic-pipeline/references/adapters/overview.md`](../academic-pipeline/references/adapters/overview.md) for the adapter contract.
```

- [ ] **Step 2: Retire "deferred to v3.6.4, PR-B" placeholders**

Open `academic-pipeline/references/passport_as_reset_boundary.md`. Find the block near line 117-121:

```markdown
## What this protocol does NOT do

- Does not define Zotero / Obsidian / folder-scan adapter shapes (deferred to v3.6.4, PR-B).
- Does not define `literature_corpus` entry shape (deferred to v3.6.4, PR-B).
```

Replace with:

```markdown
## What this protocol does NOT do

- Does not define Zotero / Obsidian / folder-scan adapter shapes (defined in [`academic-pipeline/references/adapters/overview.md`](adapters/overview.md) from v3.6.4+).
- Does not define `literature_corpus` entry shape (defined in [`shared/contracts/passport/literature_corpus_entry.schema.json`](../../shared/contracts/passport/literature_corpus_entry.schema.json) from v3.6.4+).
```

- [ ] **Step 3: Run consistency check**

```bash
cd ~/Projects/academic-research-skills
python scripts/check_spec_consistency.py
python scripts/check_passport_reset_contract.py
```
Expected: both exit 0.

- [ ] **Step 4: Commit**

```bash
cd ~/Projects/academic-research-skills
git add shared/handoff_schemas.md academic-pipeline/references/passport_as_reset_boundary.md
git commit -m "$(cat <<'EOF'
docs(v3.6.4): Schema 9 literature_corpus row + retire deferred caveats

Schema 9 optional-fields table gets a literature_corpus row with a
cross-reference to the JSON schema. passport_as_reset_boundary.md
replaces the "deferred to v3.6.4, PR-B" placeholders with forward
references to the adapter overview and entry schema. See design doc
§2.3 and §7.1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 14: Version bump sweep

**Files:**
- Modify: `scripts/check_spec_consistency.py` (version pin)
- Modify: `academic-pipeline/SKILL.md`
- Modify: `academic-paper/SKILL.md`
- Modify: `academic-paper-reviewer/SKILL.md`
- Modify: `deep-research/SKILL.md`
- Modify: `.claude/CLAUDE.md` (Skills Overview + v3.6.4 Key Additions)
- Modify: `README.md` and `README.zh-TW.md`
- Modify: `MODE_REGISTRY.md` (verify no-op)

- [ ] **Step 1: Grep for existing 3.6.3 pins**

```bash
cd ~/Projects/academic-research-skills
grep -rn "3\.6\.3" --include="*.md" --include="*.py" --include="*.json" . 2>/dev/null | grep -v docs/design/ | grep -v CHANGELOG.md
```
Use the output to identify every file that pins 3.6.3 and must bump to 3.6.4.

- [ ] **Step 2: Bump `check_spec_consistency.py`**

Edit the version pin line from `"3.6.3"` to `"3.6.4"`. Exact line varies; grep will find it.

- [ ] **Step 3: Bump each SKILL.md footer**

For each of `academic-pipeline/SKILL.md`, `academic-paper/SKILL.md`, `academic-paper-reviewer/SKILL.md`, `deep-research/SKILL.md`, find the version footer and bump 3.6.3 → 3.6.4.

- [ ] **Step 4: Update `.claude/CLAUDE.md`**

In `.claude/CLAUDE.md`:
- In the Skills Overview table, change `academic-pipeline v3.6.3` → `academic-pipeline v3.6.4`.
- Add a new section immediately above `## v3.6.3 Key Additions`:

```markdown
## v3.6.4 Key Additions

- **Material Passport `literature_corpus[]` input port**: Schema 9 gains an optional `literature_corpus[]` field defined by `shared/contracts/passport/literature_corpus_entry.schema.json`. Entries carry CSL-JSON authors, year, title, and source_pointer back to the user's own KB. `abstract` and `user_notes` are private optional fields with copyright caveats.
- **Language-neutral adapter contract**: `academic-pipeline/references/adapters/overview.md` specifies how any adapter produces literature_corpus entries. Fail-soft error handling with mandatory `rejection_log.yaml`, deterministic ordering, and extension points for user-written adapters.
- **Three reference Python adapters**: `scripts/adapters/{folder_scan,zotero,obsidian}.py` with tests and fixtures. These are starting points; users are expected to write their own adapters for non-reference corpus sources.
- **Rejection log contract**: `shared/contracts/passport/rejection_log.schema.json`. Always emitted, empty when no rejections.
- **CI lint + pytest job**: schema validation, adapter example validation, schema→docs drift check, and pytest for `scripts/adapters/tests/`.
- **Not yet shipped**: no ARS agent reads `literature_corpus[]` yet. Consumer-side integration is deferred. v3.6.4 defines the input port only.
```

- [ ] **Step 5: Update `README.md` and `README.zh-TW.md`**

Find the ARS version / skill version table. Bump `academic-pipeline 3.6.3` → `3.6.4`. Add one bullet under the "What's new" or "Highlights" section describing v3.6.4.

- [ ] **Step 6: Verify `MODE_REGISTRY.md`**

```bash
cd ~/Projects/academic-research-skills
grep -n "3\.6\.3\|3\.6\.4" MODE_REGISTRY.md
```
Expected: no hits, or only hits in unrelated context. MODE_REGISTRY concerns modes not suite version; should be a no-op.

- [ ] **Step 7: Run consistency lint**

```bash
cd ~/Projects/academic-research-skills
python scripts/check_spec_consistency.py
```
Expected: exit 0.

- [ ] **Step 8: Commit**

```bash
cd ~/Projects/academic-research-skills
git add -A scripts/check_spec_consistency.py academic-pipeline/SKILL.md academic-paper/SKILL.md academic-paper-reviewer/SKILL.md deep-research/SKILL.md .claude/CLAUDE.md README.md README.zh-TW.md
git status
git commit -m "$(cat <<'EOF'
chore(v3.6.4): version bump sweep 3.6.3 → 3.6.4

check_spec_consistency version pin, all four SKILL.md footers,
.claude/CLAUDE.md Skills Overview table + new v3.6.4 Key Additions
section, README EN + zh-TW skill tables. MODE_REGISTRY.md verified
as no-op (no version pin).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 15: CHANGELOG entry

**Files:**
- Modify: `CHANGELOG.md`

- [ ] **Step 1: Add v3.6.4 entry at top**

Open `CHANGELOG.md`. Immediately after the header block (keep `## [3.6.3]` in place below), insert:

```markdown
## [3.6.4] — 2026-04-23

### Added

- **Material Passport `literature_corpus[]` input port**. Schema 9 gains an optional `literature_corpus[]` field defined by `shared/contracts/passport/literature_corpus_entry.schema.json`. Each entry carries citation_key, CSL-JSON authors, year, title, and a source_pointer back to the user's own KB. `abstract` and `user_notes` are private optional fields with copyright caveats.
- **Adapter contract** (`academic-pipeline/references/adapters/overview.md`): language-neutral specification for producing literature_corpus entries from user-owned corpus sources. Covers fail-soft error handling, mandatory rejection-log output, deterministic ordering, and extension points.
- **Three reference adapter implementations** (`scripts/adapters/`): `folder_scan.py`, `zotero.py` (Better BibTeX export), `obsidian.py` (frontmatter or Karpathy-style convention). Each ships with tests and fixtures.
- **Rejection log contract** (`shared/contracts/passport/rejection_log.schema.json`). Always emitted; empty when no rejections.
- **CI lint and pytest job**: schema validation, adapter-example validation, schema→docs sync check, and pytest runs for adapter tests.

### Changed

- `academic-pipeline/references/passport_as_reset_boundary.md`: "deferred to v3.6.4" placeholders replaced with forward references.
- `shared/handoff_schemas.md`: Schema 9 optional fields table adds `literature_corpus`.

### Not changed (explicit non-goals)

- No ARS agent consumes `literature_corpus[]` yet. Consumer-side integration is deferred to a later release. v3.6.4 defines the input port only.
- No PDF parsing, no text extraction, no live API clients, no authenticated library crawling.
```

Replace `2026-04-23` above with the actual ship date at merge time.

- [ ] **Step 2: Commit**

```bash
cd ~/Projects/academic-research-skills
git add CHANGELOG.md
git commit -m "$(cat <<'EOF'
docs(v3.6.4): CHANGELOG entry for 3.6.4 literature corpus adapters

Describes the new literature_corpus[] input port, entry and rejection
log schemas, three reference adapters, CI + pytest additions, and
explicit non-goals. See design doc §7.2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 16: `scripts/adapters/README.md`

**Files:**
- Create: `scripts/adapters/README.md`

- [ ] **Step 1: Write the README**

Create `scripts/adapters/README.md`:

````markdown
# ARS Literature Corpus Adapters

Reference adapters that produce `literature_corpus[]` entries for the ARS Material Passport. Defined by the adapter contract at [`academic-pipeline/references/adapters/overview.md`](../../academic-pipeline/references/adapters/overview.md).

## What these are

Three starting-point adapters:

- **`folder_scan.py`** — scans a directory of files, parses citation metadata from filenames.
- **`zotero.py`** — reads a Better BibTeX JSON export.
- **`obsidian.py`** — reads an Obsidian vault (frontmatter Convention A or Karpathy-style Convention B).

Each writes two files: a `passport.yaml` with `literature_corpus[]` entries, and a `rejection_log.yaml` listing whatever the adapter could not map onto a valid entry.

**These are references, not products.** If your corpus is in Zotero, Obsidian, or a plain folder, you can run these directly; if it is in Notion, Readwise, Airtable, a custom SQLite, etc., copy a reference adapter and adapt it to your source.

## Running a reference adapter

### folder_scan

```bash
python scripts/adapters/folder_scan.py \
    --input /path/to/your/pdf/folder \
    --passport /tmp/passport.yaml \
    --rejection-log /tmp/rejection_log.yaml
```

### zotero

Export your Zotero library as **Better BibTeX JSON** (Zotero → File → Export Library → Format: Better CSL JSON). Then:

```bash
python scripts/adapters/zotero.py \
    --input ~/zotero_export.json \
    --passport /tmp/passport.yaml \
    --rejection-log /tmp/rejection_log.yaml
```

This adapter does NOT call the Zotero Web API. If you want live sync, you write your own adapter.

### obsidian

```bash
python scripts/adapters/obsidian.py \
    --input ~/ObsidianVault \
    --passport /tmp/passport.yaml \
    --rejection-log /tmp/rejection_log.yaml
```

Files in `_templates/` and `.obsidian/` are skipped. Wikilinks (`[[other note]]`) are NOT resolved; they appear as literal text in `user_notes`.

## Validating adapter output

Before using the passport in any ARS workflow:

```bash
python scripts/check_literature_corpus_schema.py \
    --passport /tmp/passport.yaml \
    --rejection-log /tmp/rejection_log.yaml
```

## How to write your own adapter

1. Read the adapter contract at [`academic-pipeline/references/adapters/overview.md`](../../academic-pipeline/references/adapters/overview.md).
2. Read the JSON schemas:
   - [`shared/contracts/passport/literature_corpus_entry.schema.json`](../../shared/contracts/passport/literature_corpus_entry.schema.json)
   - [`shared/contracts/passport/rejection_log.schema.json`](../../shared/contracts/passport/rejection_log.schema.json)
3. Copy one of `folder_scan.py` / `zotero.py` / `obsidian.py` as a starting point.
4. Change the input-reading and field-mapping code for your source.
5. Keep the output shape exactly (sorted `literature_corpus[]` by `citation_key`, always emit `rejection_log.yaml`).
6. Set `obtained_via: "other"` and `adapter_name: "<your-adapter-name>"` on every entry you produce.
7. Validate: `python scripts/check_literature_corpus_schema.py --passport <your passport>`.
8. Write tests modeled on `scripts/adapters/tests/`.

## Privacy reminder

`abstract` and `user_notes` can contain publisher-copyrighted material. Before sharing a passport that contains these fields publicly (e.g., in a public repo or over the web), make sure you have the right to publish that text. ARS does not enforce this — it only warns you in the schema description.

## Tests

```bash
cd <repo_root>
pytest scripts/adapters/tests/ -v
```

CI runs these on every push / PR that touches `scripts/adapters/**` or `shared/contracts/passport/**`.
````

- [ ] **Step 2: Commit**

```bash
cd ~/Projects/academic-research-skills
git add scripts/adapters/README.md
git commit -m "$(cat <<'EOF'
docs(v3.6.4): scripts/adapters/README.md user-facing how-to

How to run each reference adapter, how to validate output, how to
write your own adapter, and the privacy reminder for abstract and
user_notes. See design doc §2.2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 17: PERFORMANCE docs

**Files:**
- Modify: `docs/PERFORMANCE.md`
- Modify: `docs/PERFORMANCE.zh-TW.md`

- [ ] **Step 1: Add "Literature corpus ingestion" section to EN**

Open `docs/PERFORMANCE.md`. Find a sensible place (e.g., immediately after the long-running-session section added in v3.6.3) and append:

```markdown
## Literature corpus ingestion (v3.6.4+)

The Material Passport `literature_corpus[]` field is populated by user-written adapters, not ARS itself. Three reference adapters ship with v3.6.4: `scripts/adapters/folder_scan.py`, `scripts/adapters/zotero.py`, `scripts/adapters/obsidian.py`. See [`scripts/adapters/README.md`](../scripts/adapters/README.md) for how to run them and how to write your own.

### Performance posture

- Adapters run out-of-band (before an ARS session, not during). Their runtime is the user's problem, not ARS's.
- Adapters must be deterministic: re-running on identical input produces byte-identical output modulo timestamps.
- `literature_corpus[]` entries are sorted by `citation_key`; rejections are sorted by `source`.
- Adapter output size grows linearly with corpus size. A 500-entry Zotero library typically produces a passport of ~300 KB YAML. ARS consumers should lazy-load when the corpus is large.

### What v3.6.4 does NOT do

- Does not ingest PDFs, extract text, or run OCR.
- Does not call the Zotero Web API, Notion API, or any live service.
- Does not fetch paywalled content or use user credentials to access institutional libraries.

These boundaries are deliberate and reflect the ARS data-layer decision: ARS is a writing/review-layer framework; corpus integration stays in user-owned code. Users who want API-based live-sync adapters are expected to write them themselves, using the three reference adapters as starting points.

### Consumer-side integration

As of v3.6.4, no ARS agent reads `literature_corpus[]`. The field is a defined input port only. Consumer-side integration (agents that actually USE the corpus for research planning, citation generation, etc.) is deferred to v3.6.5+.
```

- [ ] **Step 2: Add equivalent section to zh-TW**

Open `docs/PERFORMANCE.zh-TW.md`. Append equivalent content in Traditional Chinese:

```markdown
## 文獻語料庫導入（v3.6.4+）

Material Passport 的 `literature_corpus[]` 欄位由**使用者自行撰寫的 adapter** 產出，不是 ARS 本身。v3.6.4 附三個 reference adapter：`scripts/adapters/folder_scan.py`、`scripts/adapters/zotero.py`、`scripts/adapters/obsidian.py`。執行方式與自行撰寫 adapter 的指引見 [`scripts/adapters/README.md`](../scripts/adapters/README.md)。

### 效能定位

- Adapter 在 ARS session 之外執行（跑 ARS 前跑）；執行時間由使用者自行負責，不進 ARS 的時間預算。
- Adapter 必須具備 determinism：同一份 input 重跑產出 byte-identical 輸出（時間戳除外）。
- `literature_corpus[]` 依 `citation_key` 排序；`rejection_log.rejected[]` 依 `source` 排序。
- Adapter 輸出大小與語料庫大小線性成長。500 筆 Zotero 書目約產出 300 KB 的 passport YAML。大型語料庫建議 ARS 消費端採 lazy load。

### v3.6.4 **不做** 的事

- 不讀 PDF 內容、不做文字抽取、不跑 OCR。
- 不呼叫 Zotero Web API、Notion API 或任何遠端服務。
- 不抓付費牆後內容、不用使用者憑證連線機構圖書館。

這些邊界是刻意的，反映 ARS 的 data-layer 定位：ARS 是 writing / review layer 的框架，語料整合留在 user-owned code。如需 API-based live-sync adapter，由使用者以三個 reference adapter 為起點自行撰寫。

### 消費端整合

v3.6.4 時沒有任何 ARS agent 讀取 `literature_corpus[]`。此欄位僅為 input port 定義，消費端整合（agent 實際使用語料做研究規劃 / citation 生成 / ...）留待 v3.6.5+。
```

- [ ] **Step 3: Commit**

```bash
cd ~/Projects/academic-research-skills
git add docs/PERFORMANCE.md docs/PERFORMANCE.zh-TW.md
git commit -m "$(cat <<'EOF'
docs(v3.6.4): PERFORMANCE sections on literature_corpus ingestion

Adds a "Literature corpus ingestion (v3.6.4+)" section to both
EN and zh-TW PERFORMANCE docs. Covers performance posture,
explicit non-goals, and the v3.6.5+ consumer-side deferral. See
design doc §7.1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EOF
)"
```

---

## Task 18: Final local run + PR open

- [ ] **Step 1: Full local test battery**

```bash
cd ~/Projects/academic-research-skills
pytest scripts/adapters/tests/ -v
python scripts/check_literature_corpus_schema.py
python scripts/sync_adapter_docs.py --check
python scripts/check_spec_consistency.py
python scripts/check_passport_reset_contract.py
```
Expected: all exit 0; pytest reports N passed (0 failed).

- [ ] **Step 2: Review full diff**

```bash
cd ~/Projects/academic-research-skills
git log origin/main..HEAD --oneline
git diff origin/main..HEAD --stat
```
Sanity check: expect ~18 commits, line count consistent with the spec's expected ~1500-2000 lines added.

- [ ] **Step 3: Push**

```bash
cd ~/Projects/academic-research-skills
git push origin feat/v3.6.4-literature-corpus-adapters
```

- [ ] **Step 4: Open PR via `gh`**

```bash
cd ~/Projects/academic-research-skills
gh pr create \
  --base main \
  --head feat/v3.6.4-literature-corpus-adapters \
  --title "feat(v3.6.4): literature corpus input port + three reference adapters" \
  --body "$(cat <<'EOF'
## Summary

- Adds Material Passport `literature_corpus[]` input port (Schema 9 optional field).
- Adds `literature_corpus_entry.schema.json` and `rejection_log.schema.json`.
- Adds three reference adapters under `scripts/adapters/`: folder_scan, zotero (BBT export, not API), obsidian (frontmatter or Karpathy-style).
- Adds CI lint (schema validation, example validation, doc drift check) and a pytest job.
- Retires the "deferred to v3.6.4, PR-B" placeholders in `passport_as_reset_boundary.md`.
- Version bump sweep 3.6.3 → 3.6.4 across four SKILL.md, CLAUDE.md, README, consistency lint.

See design doc at `docs/design/2026-04-23-ars-v3.6.4-literature-corpus-adapters-design.md` for full rationale and explicit non-goals.

## Explicit non-goals for v3.6.4

- No ARS agent consumes `literature_corpus[]` yet. Consumer-side integration is deferred to v3.6.5+. This PR is input-port only.
- No PDF parsing, no live API clients, no paywall bypass.

## Test plan

- [ ] `pytest scripts/adapters/tests/ -v` all pass locally
- [ ] `python scripts/check_literature_corpus_schema.py` exits 0
- [ ] `python scripts/sync_adapter_docs.py --check` exits 0
- [ ] `python scripts/check_spec_consistency.py` exits 0
- [ ] `python scripts/check_passport_reset_contract.py` exits 0
- [ ] CI passes on GitHub Actions

## Review protocol

Per design doc §7.3, this PR will receive at least three rounds of `/codex review --base main` + one fresh codex review before merge, following the `feedback_codex_review_vs_resume_audit_scope.md` protocol for large feature PRs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
EOF
)"
```

- [ ] **Step 5: Record PR URL in memory**

Once the PR is created, note the PR number and URL. The session owner updates `project_academic_research_skills.md` in memory when v3.6.4 merges and ships.

---

## Self-review

After writing every task, I scan this plan for spec coverage, placeholders, and type drift.

**1. Spec coverage.** Checking each §-level section of the spec against task coverage:

- Spec §1 Scope / §1.2 Non-goals → Task 0 (implicit in plan header) + Task 15 CHANGELOG "Not changed" section.
- Spec §2 Architecture + file layout → Task 1 scaffolding + all subsequent tasks create the right files.
- Spec §3 literature_corpus_entry schema → Task 2.
- Spec §4 rejection_log schema → Task 3.
- Spec §5.1 Common contract → Task 5 (_common.py) + Task 10 (overview.md).
- Spec §5.2 folder_scan → Task 7.
- Spec §5.3 zotero → Task 8.
- Spec §5.4 obsidian → Task 9.
- Spec §6.1 lint A + B → Task 4.
- Spec §6.2 lint C → Task 11.
- Spec §6.3 pytest CI → Task 12.
- Spec §6.4 test coverage → Tasks 7/8/9 + Task 5 (common).
- Spec §6.5 existing CI additions → Task 12.
- Spec §7.1 sweep → Task 14.
- Spec §7.2 CHANGELOG → Task 15.
- Spec §7.3 release order → Task 18 (PR open) + out-of-plan codex audits (§7.3 steps 4-10).
- Spec §7.4 rollback → not implemented, documented in design doc only (acceptable; rollback is a runtime decision, not a plan step).
- Spec §8 deferred questions → addressed at plan execution time (Python version, etc.).

All spec sections covered.

**2. Placeholder scan.** No `TBD`, no `TODO`, no "add appropriate error handling", no "write tests for the above" without code. The one nuance: task 7 step 3 explicitly calls out a design decision (extending `clean_timestamps` to blank `source_pointer`) with a revised step inline; this is an in-plan decision, not a placeholder.

**3. Type / name consistency.** I cross-checked:
- `ADAPTER_SPEC_VERSION` defined in Task 5, not used elsewhere (it is a forward-looking hook; the reference adapters set their own `ADAPTER_VERSION = "1.0.0"` string).
- `write_passport` / `write_rejection_log` helper signatures in Task 5 match usage in Tasks 7/8/9.
- `make_citation_key(family, year, title_hint, existing)` signature in Task 5 matches Task 7 invocation.
- `parse_semicolon_names`, `parse_csl_name` match usage in Task 9.
- `clean_timestamps` fixture name matches across tasks 7, 8, 9.
- Schema `required` set [citation_key, title, authors, year, source_pointer] matches every fixture and test.

No type or name drift found.

**4. One edge case in Task 5 flagged for execution.** `write_rejection_log` helper leaves a bug in the plan code: the `summary` block has a line that looks like `"total_accepted": doc.get("_accepted_hint", 0) if False else 0`. This should be removed at implementation time; the final written code should simply emit `summary: {total_rejected: len(sorted_rejected)}` and accept `total_input` / `total_accepted` when the caller provides them. The plan's commit at Task 5 should include a cleaner implementation — the executing engineer should delete that dead `if False` branch. Noted here so the self-review is honest rather than hiding the issue.

---

## Execution handoff

Plan complete and committed alongside the spec. Two execution options:

**1. Subagent-Driven (recommended)** — dispatch a fresh subagent per task, review between tasks, fast iteration. Uses `superpowers:subagent-driven-development`.

**2. Inline Execution** — execute tasks sequentially in the current session using `superpowers:executing-plans`, batch execution with human checkpoints every 2-3 tasks.

**Which approach?**
