About This Project

The Corpus

This wiki is built on the Enron Email Dataset — 1.23 million messages from 158 mailboxes, spanning 1999 to 2002. The Federal Energy Regulatory Commission released the corpus during its investigation into Enron’s collapse. It remains the largest collection of authentic corporate email ever made public.

The email data was processed and released by ZL Technologies, Inc. under a Creative Commons Attribution 3.0 United States License.

The Infrastructure

The corpus is queried using duckdb-pst, an open-source DuckDB extension that reads Outlook PST archive files directly as queryable tables. No ETL pipeline. No intermediate database. One SQL query against a raw .pst file returns structured email data — sender, recipients, subject, body, timestamps — in seconds or less.

A foundation layer pre-computes TF-IDF indexes and message embeddings across all 158 mailboxes, enabling semantic search and volume-adjusted term analysis for every person in the corpus.

The Pipeline

Every article was produced by a pipeline of specialized AI agents.

The original pipeline runs entirely on open-source, non-proprietary components. LangGraph orchestrates the workflow as a directed state graph. LLMs run locally via LM Studio on consumer hardware — Qwen 3 4B for tool-calling agents that discover topics and research the corpus, Gemma 3 12B for article writing. Embeddings are computed by model2vec (potion-multilingual-128M).

flowchart TD
    PST["158 PST Archives — 1.23M Messages"] --> DISC["Topic Discoverer"]
    PST --> RES["Corpus Researcher"]
    FDB["Foundation DB — TF-IDF + Embeddings"] --> RES
    DISC -->|queue| RES
    RES -->|research/*.md| STR["Research Structurer"]
    RES -->|research/*.md| WRI["Article Writer"]
    STR -->|static/data/*.json| WRI
    WRI -->|content/*.md| REV["Article Reviewer"]
    REV -->|fail → rewrite| WRI

Topic Discovery. The discoverer receives 14 corpus tools (search_emails, term_owners, signature_terms, etc.) and runs autonomously for up to 10 tool-calling iterations, outputting a ranked JSON array of article candidates.

Corpus Research. The researcher deep-dives into primary sources — structured pass first (metadata, TF-IDF, correspondents, timeline), then free-form investigation pulling full email bodies and extracting verbatim quotes. Every claim is cited to a specific node_id and pst_path.

Research Structuring. Converts research markdown into JSON for the interactive research card UI, re-executing every SQL query to capture row data and EXPLAIN ANALYZE plans. This powers every “Find in Archive” button on the wiki.

Article Writing. The writer receives only the research file. No external knowledge, no hallucination vector.

Quality Review. The reviewer checks each article against its research file for unsupported claims, missing evidence, and editorial consistency. Failures return to the writer.

The architecture is model-agnostic. The same agent graph — system prompts, tool access, structured output, staged handoffs — runs with local open-source models or proprietary ones. As the wiki grew past 100 entries, production shifted to Claude via the Claude Agent SDK for output quality and speed, but the graph shape didn’t change.

The Numbers

Trademark Notice

The Enron name and logo are used for educational and historical documentation purposes. In December 2024, the dormant Enron trademark was purchased for $275 and briefly revived as a satirical venture. This wiki is not affiliated with that effort or any commercial use of the Enron brand.

Contact

This project was built by Intellekt.

[email protected]