Generated examples need provenance and evidence beyond their judge's approval

Capture the generator identity and immutable revision when available, prompt template, source inputs, decoding configuration, seed where meaningful, tool responses, and generation time. An API name alone may not identify stable model behavior, so record the provider's available version information and acknowledge what cannot be reproduced. Keep the connection to seed instructions and retrieved documents: paraphrasing or generating a response does not automatically erase source conditions or sensitive information. The project's source-admission process still applies to inputs and derived use without inventing a universal legal status for generated text.

Self-Instruct is a primary example of generating instructions, inputs, and outputs, then filtering invalid or similar examples before instruction tuning. Its experiments demonstrate a specific augmentation approach, not that every generated answer is correct or that every model can improve through unlimited self-copying. Record why candidates were rejected or retained, including similarity rules and judge versions. Deduplicate against other generated examples and admitted human material under a stated representation. Preserve sufficient lineage to investigate an erroneous example without unnecessarily exposing sensitive raw prompts in routine public metadata.

Choose checks suited to the task: executable tests for some code, verifiable calculations for constrained mathematics, source-grounded evidence checks, and independent human review where appropriate. Passing a test suite proves only the tested behavior, and a model judge can reward persuasive explanations containing the wrong answer. If generator and judge share training data or reasoning patterns, their errors may correlate. Evaluate the retained set with a separate process and also inspect rejected examples, so the acceptance filter's precision and coverage can be estimated instead of inferred from its own confidence.

Track usable yield together with costs. Generating many low-quality candidates can be worthwhile when validation is reliable and cheap, but accepted volume alone hides both waste and residual defects. Estimate the retained set's rubric-valid fraction on an independent audit and report uncertainty. Multiplying candidate count, retention fraction, and measured valid fraction gives an expected useful count under a stated transfer assumption; it is not a guaranteed count or a proof that every useful example teaches a unique skill. Include generation, judging, and storage costs separately when comparing pipelines.

Recursive-training studies such as The Curse of Recursion analyze and demonstrate settings where learning repeatedly from generated material loses parts of the original distribution, especially its tails. That is a warning about feedback and replacement assumptions, not a theorem that all synthetic augmentation is harmful. Preserve appropriate real-data anchors, monitor rare tasks and languages, and test alternative synthetic proportions under controlled budgets. A judge that favors common formats can amplify selection even before training. Trace generations back through their parent models when comparing releases so recursive reuse is visible rather than mislabeled as independent fresh data.

The sampling lab examines an assumed retained-error audit. Reset, explicitly set BOTH error probability p=0.2 and n=20, predict sample-mean variance 0.008, then Play or Step. Reset and re-set BOTH p=0.2 and n=100; predict 0.0016. Edits retain t; playback highlights the exact iid distribution rather than generating examples. The displayed CDF bound concerns the normal approximation, not model-judge reliability or recursive collapse. There are no prompts, generators, benchmark searches, or correlated answers here, so actual audits still need task-specific design and protected external evaluations.