This page is the public case study for that policy. It explains the research-infrastructure reasons (provenance, license, verification, contamination) and the project’s refusal to keep feeding a training loop that already exists at scale. It does not reproduce Reddit posts, comments, or dumps. The same essay lives in the repository as docs/why-not-reddit.md.
What Open Reason has to be able to claim
Every released row is supposed to answer three questions:
- Where did this come from? (
provenance, including source type, license, and generator or author). - May a downstream user use it under the stated terms? (
provenance.license_spdxis authoritative per row). - Was the answer actually checked? (
quality.verifiedis true only after a named check ran and passed).
A source that cannot support those claims cannot enter the corpus. Popularity, accessibility, and “everyone else trained on it” are not substitutes.
The loop we are refusing to feed
It is widely known that a large volume and variety of Reddit data has been used to train contemporary AI systems: comment dumps, archives, third-party dataset releases, and scraped threads. That material is attractive because it is abundant, conversational, and already sitting in public crawls. Abundance is not a license, and it is not a verification protocol.
Those models often absorb the personality of Reddit commenters. The failure mode is not only factual error. It is a voice: performative certainty in place of knowledge, pile-ons in place of argument, mockery in place of correction, and a style of “listening” that is not actually advice. A great deal of that text sounds like help and is not helpful.
That is dangerous. Medical-, legal-, and financial-sounding chatter travels in the same streams as ordinary conversation. Harassment norms and unverifiable claims travel with them. A system that has ingested that register will reproduce it under the appearance of competence. At some point the loop has to be stopped — not by hoping a later alignment pass will wash it out, but by refusing the source.
Open Reason is a concrete refusal to keep feeding that loop. I am contributing what I can personally: this dataset and this pipeline do not use Reddit, directly or indirectly. There is no dump, no archive, no “Reddit-derived” mix, and no search-result workaround. If a candidate’s provenance includes Reddit, it is rejected.
A very small trained model is part of this project so the exclusion is not only a policy file. It is not a frontier LLM. This repository trains a GPT-2-style causal LM from scratch (a few million parameters) on Open Reason JSONL, on CPU — Docker image open-reason-train:cpu when Docker is available, otherwise host CPU. Checkpoints write to training/work/open-reason-local/. The dataset is theworker02/open-reason. If a Hub upload succeeds, the companion id is theworker02/open-reason-small. We do not claim a 1B-parameter model (theworker02/open-reason-1b) until a CUDA 1B job has actually been trained, evaluated, and uploaded. If people find the small model useful, development will continue.
Provenance: Reddit posts cannot support the claims
Reddit items are not a stable, attributable research record in the sense Open Reason requires.
- Identity. Authors are typically pseudonymous. A username is not a reviewed publisher, an institution, or a pinned git identity. Open Reason cannot honestly fill
provenancewith a person or organization it cannot identify. - Stability. Posts and comments are edited, deleted, removed by moderators, or pulled when accounts disappear. A snapshot is not a version-pinned document.
- Authority. A subreddit is a community venue, not an official specification, a reviewed textbook, or a commit-pinned repository. Trust tiers in the source registry have no slot that turns forum consensus into an oracle.
- Traceability. Fabricating URLs, thread ids, or “source: Reddit” as if that were a complete citation is a pipeline bug. Omitting the origin is also a bug. The only honest action is rejection.
Unknown origin is allowed only with unknown_reason, is forbidden for tier S, and is not a path to launder Reddit text as “unspecified web.”
License: there is no clean redistributable grant
Open Reason ships pipeline and original dataset rows under Apache-2.0. That is the only project license. Per-row provenance.license_spdx must still be a real grant, not a guess.
User content on Reddit is posted under the platform’s terms. That arrangement is not a clean downstream license to Open Reason, and it is not a grant to relicense the text as Apache-2.0. We do not treat “it was publicly readable” as “it is redistributable in this corpus.”
The following remain Reddit-derived and are forbidden, even when they arrive with a different filename or host:
| Channel | Why it is still excluded |
|---|---|
| Official or unofficial API pulls | Access terms are not a dataset license for this project |
| Subreddit dumps and comment archives | Same user content, different packaging |
| Pushshift and successor archives | Archive is not permission; provenance is still Reddit |
| Third-party “Reddit datasets” | Secondary distribution does not create a new grant |
| Sites whose primary purpose is republishing Reddit | Mirrors do not change the source |
| Search results that reproduce Reddit threads | The snippet is still Reddit text |
There is no secondary-dataset workaround. Share-alike, non-commercial, unknown, or platform-encumbered text is never silently converted to Apache-2.0. This page is research-infrastructure policy, not legal advice to third parties. The operational rule is simpler: Reddit-derived material does not enter the tree or the release.
Quality: votes are not verification
quality.verified is set only after a check this repository can re-run: coding sandbox (unittest / SQLite / Node), sympy or exact integer arithmetic, independent numeric science checks, or a named constraint checker. If a check was not run, the flag stays false. There is no “probably correct” bit.
Karma, awards, upvote ratios, and “best comment” rankings measure engagement, not correctness. They cannot support the claims Open Reason makes about verified rows.
evidence.community_evidence may, for other reviewed community sources, inform a capped component of evidence_confidence. That score is not a claim of truth. community_evidence must never set quality.verified. Reddit is not admitted as community evidence at all. A record that looks Reddit-derived is dropped before scoring.
High-risk domains (medicine, law, finance, safety-critical engineering) cannot enter tier S without authoritative evidence and independent verification. Forum chatter that merely sounds like those domains is exactly the register described above. It is not a shortcut into the verified configuration.
Contamination and evaluation leakage
Reddit is heavily represented in web crawls and in instruction-tuning mixes. Contest problems, exam questions, benchmark snippets, and homework statements have been posted there for years. Ingesting Reddit would be an uncontrolled mixing channel between “training text” and “evaluation items.”
Open Reason already fingerprints known eval sets (prompt prefixes and distinctive needles — not answers). Hits are reported, not silently deleted; strict builds fail. Hold out the repository benchmarks from training. A Reddit dump would bypass that discipline: the same item can appear as a casual comment with no benchmark name attached.
Using Reddit would also contaminate style. Models trained on that register learn to imitate it. Open Reason’s generators and human-authored items are labeled as such; they are not a paraphrase farm for forum threads.
Indirect use is forbidden
The exclusion is on provenance, not on a single hostname. Rejected channels include Reddit hosts and short links, Pushshift and successor archives, subreddit and comment dumps, third-party datasets whose provenance includes Reddit, sites whose primary purpose is republishing Reddit, search results and caches that reproduce threads, paraphrases or “inspired by a thread” rows whose provenance still points at Reddit, and discovery candidates whose id or URL matches Reddit.
The record inspector is conservative on purpose. It flags obvious Reddit provenance so examples can be rejected, not so Reddit can be laundered through a secondary dataset. If a corpus is materially Reddit-derived, do not import it.
What Open Reason uses instead
| Kind | Role in this project |
|---|---|
| Official documentation | Version-pinned, license-reviewed docs inspire original tasks. Auto-approve is a license policy, not a scrape. |
| Permissive GitHub | Original tasks inspired by commit-pinned MIT/BSD/Apache-2.0 public snapshots (verbatim=false). Not Reddit-derived. |
| Stack Overflow | User-approved seed only: original rewrites and short attributed snippets, not copies of threads, and not Reddit mirrors. Votes never verify. |
| Original verified tasks | Human-authored items and deterministic synthetic generators. quality.verified only after sandbox, sympy, numeric, or named checkers. |
| Curriculum auto-approve | Public course and OER catalogs may inspire original tasks. Lectures are not copied. NC/SA never becomes Apache-2.0 verbatim rows. |
Quora is not a primary source of truth. Educational sites such as Khan Academy, MIT OCW, CS50, OpenStax, and MDN are registered for original curriculum tasks and are not scraped. Reddit cannot be approved, auto-approved, discovered, or ingested.
Enforcement
The ban is implemented, not merely documented: the source registry marks Reddit prohibited; source policy AUTO_REJECTS it; prohibited matchers list hosts and dataset-name substrings; inspect_record drops Reddit-derived rows; the pipeline counts reddit_rejected; scripts/check_no_reddit.py runs in CI; discovery cannot score Reddit; open-reason ingest --source reddit is rejected.
Do not add Reddit samples to the corpus. Rejection tests may use URL and source strings only to prove the inspector fails closed.
What this policy does not do
- It does not admit “old” dumps, research fair-use arguments, or anonymized comments as exceptions.
- It does not distinguish posts from comments, NSFW from SFW, or one subreddit from another.
- It does not treat web crawls, webtext corpora, or search snippets as clean if their provenance includes Reddit.
- It does not set
quality.verifiedfrom community scores anywhere in the pipeline.
The exclusion is not weakened by this case study. If a future contributor wants Reddit in the corpus, the answer is no.
How to cite
- Source tree: docs/why-not-reddit.md
- This page: theworker02.github.io/open-reason/why-not-reddit.html
- Dataset: huggingface.co/datasets/theworker02/open-reason
- Small companion model (if uploaded): huggingface.co/theworker02/open-reason-small