# Review Quality Thinking Framework

A cognitive framework for producing high-quality reviews. Teaches **how to think** about paper quality, not just what to score.

## The Three Lenses

Every paper should be evaluated through three lenses simultaneously:

### Lens 1: Internal Validity — "Does the evidence support the claims?"

Ask in order:
1. What is the central claim?
2. What evidence is presented?
3. Is there a logical chain from evidence to claim? (Warrant)
4. Are there alternative explanations the authors didn't consider?
5. Would removing any single piece of evidence collapse the argument?

**If #5 is yes**: The argument depends on a single linchpin. Flag it — the paper's contribution is only as strong as that one piece of evidence.

### Lens 2: External Validity — "Does this matter beyond this study?"

Ask in order:
1. Who is the population of interest?
2. Does the sample represent that population?
3. Are the conditions replicable?
4. Would the findings hold in a different context/culture/time?
5. What are the boundary conditions the authors don't mention?

**Judgment heuristic**: Most authors overstate generalizability. If the sample is from one university in one country, findings cannot claim to apply "generally" without qualification.

### Lens 3: Contribution — "So what?"

Ask in order:
1. What did we know before this paper?
2. What do we know after?
3. Is the delta meaningful (not just statistically significant)?
4. Who benefits from knowing this?
5. What new questions does this open?

**Judgment heuristic**: If you can't articulate the delta in one sentence, either the contribution is weak or the paper hasn't communicated it clearly. Both are review-worthy observations.

## Common Reviewer Traps

| Trap | Description | How to Avoid |
|------|-------------|-------------|
| **Methodological tunnel vision** | Only critiquing methods, ignoring whether the question matters | Start with Lens 3 (contribution) before Lens 1 |
| **Novelty bias** | Penalizing replication or incremental work | Replication IS valuable; evaluate on execution quality |
| **Expertise projection** | Expecting the paper to use your preferred method | Evaluate the chosen method on its own terms |
| **Positivity-severity oscillation** | Being too nice in comments, too harsh in scores | Write the score first, then justify with comments |
| **Missing forest for trees** | Listing 20 minor issues, missing the one fatal flaw | Always state the single most important issue first |

## Calibration Questions (ask after drafting your review)

1. If this paper were published as-is, would it mislead readers? (If yes → Major Revision or Reject)
2. Could the authors reasonably address my concerns in one revision cycle? (If no → Reject)
3. Am I being harder on this paper than I would be on my own work? (Calibration check)
4. Did I identify at least one genuine strength? (Balance check)
5. Would my review help the authors improve, even if the paper is rejected? (Constructiveness check)
