How to Use Project Lighthouse to Analyze Historical Scientific Manuscripts with AI

What Project Lighthouse Does

Project Lighthouse is an open research effort for examining scientific manuscripts, notebooks, letters, and neglected printed works at scale. It uses AI to transcribe digitized pages and reconstruct their mathematical meaning, while requiring human inspection, computational verification, literature research, and independent review before a candidate can be treated as a credible finding.

The project is designed to search for unfinished ideas, uncredited results, and overlooked connections. It also records false leads and known results in a shared ledger so contributors do not repeat investigations that have already reached a dead end.

According to the repository status dated November 9, 2026, approximately 12,000 pages from Gauss, Riemann, Hilbert, Dedekind, and Grothendieck had been deeply read. The leading candidate was L-019, concerning unpublished shock-wave sheets attributed to Riemann. Its algebra had been checked computationally, but the attribution was still awaiting confirmation from a human palaeographer.

A promising AI-assisted reading is not automatically a discovery. Project Lighthouse treats transcription, interpretation, verification, historical priority, and expert confirmation as separate questions.

Key Features

• Page-level transcription: Manuscript images are converted into verbatim text, with formulas represented in LaTeX and struck text explicitly marked.
• Contextual deep reading: Mathematical pages are analyzed with neighboring pages so arguments can be reconstructed rather than interpreted in isolation.
• Modern comparison: Deep reads identify a modern equivalent, the person normally credited with it, and a surprise score from 0 to 10.
• Four-stage candidate protocol: Every candidate must pass scan inspection, computational re-derivation, meaning-based literature research, and independent unanchored review.
• Shared ledger: ledger/LEADS.md records candidates and verdicts, including duplicates, mistakes, and the sources that resolved them.
• False-positive controls: docs/FALSE_POSITIVES.md documents twelve common ways an apparent discovery can fail and provides a required checklist.
• Contribution labels: Labels describe completed work rather than confidence or popularity.
• Open formats: Transcriptions and analyses use documented repository formats, allowing contributors to inspect and extend the data.

Understand the Repository Structure

Before processing pages, become familiar with the main paths:

queue/QUEUE.md
src/transcribe.py
src/deep.py
src/read_source.py
data/<author>/STATUS.tsv
data/<author>/transcripts/<bundle>/cNNN.json
data/<author>/deep/<bundle>_cNNN.json
data/FORMAT.md
verify/
ledger/LEADS.md
docs/HOT_PAGES.md
docs/FALSE_POSITIVES.md
CONTRIBUTING.md
CITATION.cff

The queue/ directory tracks sources waiting to be processed. The src/ scripts handle transcription and deep reading. Author-specific output is stored under data/, verification programs go in verify/, and investigated leads are recorded in the ledger.

Installation and Setup

1. Obtain the repository

Clone or download the Project Lighthouse repository from GitHub, then work from its root directory. Read CONTRIBUTING.md before adding data or reviewing a candidate because it defines the contribution process.

2. Inspect the documented data formats

Open data/FORMAT.md before creating transcript or deep-read files. Existing data follows these patterns:

data/<author>/transcripts/<bundle>/cNNN.json
data/<author>/deep/<bundle>_cNNN.json

Follow the documented schema rather than creating an alternative structure. Consistent filenames and fields allow contributors to determine which pages have already been processed.

3. Configure your API key

The repository expects contributors to bring their own Anthropic API key. Store it at the following local path:

~/.config/hunt/anthropic_key

Never commit the key to the repository. The README estimates a cost of approximately $0.25 per page for transcription and deep reading with Claude Opus 5.5. Actual usage should be monitored before processing a large source.

4. Choose an eligible source

A source may come from any author, field, language, or century, but it must be in the public domain, digitized, and hosted somewhere whose terms permit fetching. The project accepts manuscripts, notebooks, letters, and neglected printed works.

Add the source to queue/QUEUE.md and create its author directory under:

data/<author>/

Do not host library scans in the repository. Project Lighthouse links to scans while keeping locally produced transcriptions and analyses under the repository's licensing rules.

Basic Usage Workflow

Step 1: Transcribe each page

Use src/transcribe.py when the source is available through IIIF image URLs. For scan PDFs, use src/read_source.py, which supports the workflow from PDF pages to JPEG images and page records.

A transcription should preserve what appears on the page rather than silently correcting it. Record formulas in LaTeX and mark struck text. Save the result in the author and bundle structure defined by data/FORMAT.md.

data/example_author/transcripts/example_bundle/c001.json

This path is an illustration of the documented naming pattern. Use the actual author, bundle, and page identifiers for your source.

Step 2: Deep-read the mathematical pages

Use src/deep.py or the deep-reading functions in src/read_source.py. Include neighboring pages so notation, definitions, and intermediate steps are interpreted in context.

For each relevant page, reconstruct the argument, identify its modern equivalent, note who is generally credited, and assign the repository's 0–10 surprise score. Store the result under the corresponding deep-read path:

data/example_author/deep/example_bundle_c001.json

A high score only prioritizes attention. It does not establish historical priority or justify a stronger review label.

Step 3: Check processing status

Open data/<author>/STATUS.tsv to find bundles that have been transcribed but have not yet received deep reads. This is a practical way to contribute without introducing a new source.

Step 4: Investigate surprising pages

The fastest review route is docs/HOT_PAGES.md. It lists pages that the model rated as surprising but that no human has yet checked. Compare the transcription directly with the scan, inspect the surrounding pages, and decide whether the claim is already known, mistranscribed, misdated, or genuinely worth opening as a candidate.

Apply the Candidate Protocol

Before stating the significance of a possible finding, complete all four checks in order.

1. Read the page against the scan. Confirm handwriting, symbols, marginal notes, deletions, and page order. AI output alone is insufficient.
2. Re-derive the mathematics with code. Place reproducible calculations in verify/. Test the claimed identities or consequences instead of relying on a verbal resemblance.
3. Search the literature by meaning. Start with the author's published work, then inspect older sources. Do not trust an AI-generated “first credited to” date without verification because such dates may be too late.
4. Request independent, unanchored review. Give a second contributor or agent the relevant facts without revealing the desired conclusion. The reviewer must write a verdict and cite sources.

Every candidate file must also include the checklist from docs/FALSE_POSITIVES.md. This helps catch standard errors such as mistranscription, modernized interpretation, known publication under another name, or unsupported dating.

Record the Outcome in the Ledger

Add the candidate and its verdict to ledger/LEADS.md. Record negative results as carefully as positive ones. A candidate that turns out to be well-known history still prevents another contributor from repeating the same search.

The repository's prominent example is L-019. Its full record is in ledger/LEADS.md, while docs/L019_expert_request_draft.md contains the request prepared for specialists. The README describes computer verification and a blind AI handwriting comparison, but it explicitly states that a human palaeographer has not yet confirmed the attribution.

Use Labels Correctly

Project Lighthouse labels report completed actions, not belief in a discovery:

• transcribed: A page JSON file exists. Any contributor may set this.
• read: A deep read exists. Any contributor may set this.
• self-checked: The finder inspected the scan, ran the mathematics, and searched the literature.
• reviewed: Another contributor or an unanchored agent independently repeated all four checks and wrote a sourced verdict.
• expert: A named specialist signed off on the scan.

Votes can help determine which pages receive attention first, but votes never establish a label.

Advanced Tips for Reliable Research

Separate transcription from interpretation

Keep the transcript faithful even when you think you understand the mathematics. Put reconstructions, modern notation, and historical comparisons in the deep-read record instead of rewriting the source.

Read page neighborhoods

A formula can appear novel when detached from its definitions or earlier derivation. Always inspect preceding and following pages, bundle order, and related notebooks before opening a lead.

Search concepts, not only terminology

Historical authors may describe a modern result without using its present name. Search for the mathematical relationship, geometric construction, physical condition, or proof technique. Begin with the author's own publications before comparing later credited sources.

Keep reviews unanchored

Do not tell an independent reviewer that a page is expected to contain a famous later result. Supply the scan, transcription, date evidence, and mathematical facts while withholding the proposed conclusion. This reduces confirmation bias.

Make verification reproducible

Place scripts and calculations in verify/, document assumptions, and distinguish exact symbolic checks from numerical evidence. A useful verification should allow another contributor to rerun the derivation independently.

Prioritize unfinished work

Use STATUS.tsv to find transcribed bundles awaiting deep reading and docs/HOT_PAGES.md to find surprising pages awaiting human inspection. These paths help avoid duplicating completed processing.

Respect source-specific rights

Scans remain with their libraries and should be linked rather than hosted. Repository transcriptions and analyses are licensed under CC BY 4.0, while the code is MIT licensed. For Grothendieck's unpublished notes, the repository includes only analyses and short quotations, not full transcriptions, in accordance with the exception described in the README.

Preserve priority and citation information

Project Lighthouse treats the first commit or release containing a ledger entry as its public date. A GitHub release archived on Zenodo can add a DOI and independent timestamp. Use CITATION.cff when citing the repository, and credit the repository when reusing its CC BY transcriptions.

Ways to Contribute

• Share existing transcriptions or translations produced by hand or with any tool or model.
• Run the processing scripts on a new eligible source.
• Correct generated transcriptions by comparing them with scans.
• Deep-read bundles that are currently only transcribed.
• Review an existing candidate independently.
• Provide a named expert verdict when you have the relevant specialist knowledge.

The project emphasizes that expert reading and knowledge of the literature are scarcer than model tokens. Even a contribution that disproves a candidate can be valuable when it is documented with clear evidence and sources.

Conclusion

Project Lighthouse combines AI-scale manuscript processing with a deliberately skeptical research protocol. Start by selecting an eligible source or unfinished bundle, create faithful transcripts, deep-read pages in context, and investigate surprising claims through scan inspection, code, literature research, and independent review. Record every outcome in the shared ledger so both discoveries and dead ends become reusable scholarly evidence.