Your AI Built It. What Did It Actually Prove?

Why AI-generated outputs, verified discoveries, and scientific understanding are different achievements—and how to design workflows that keep them separate.

An AI system proposes a protein structure, designs a material, writes an algorithm, or generates a biological hypothesis.

The output looks impressive. It may even pass an initial test.

Then comes the leap:

“We generated it” becomes “we discovered it.”

“We built it” becomes “we understand it.”

“We evaluated it” becomes “we proved it.”

Each transition requires evidence. None should happen automatically.

In my paper, Constructing Possibility as an Epistemic Method: An Onturgic Audit of AI for Science, I develop a framework for examining these transitions. Its central distinction is between constructing candidates, verifying claims, and understanding mechanisms. These activities can reinforce one another, but they are not interchangeable.

For developers building AI agents, scientific tools, or automated experimentation pipelines, this is more than a philosophical distinction. It is a question about system architecture:

Where does candidate generation end, and where does warranted belief begin?

Construction is not proof

A successful build tells us something. The difficult question is what.

Suppose a system generates a candidate, and a laboratory produces an object matching parts of the prediction. Several questions remain:

• Was the object correctly identified?
• Does it have the intended property?
• Is it genuinely new?
• Does the result generalize beyond the tested conditions?
• Does the proposed explanation account for its behavior?

These are separate claims.

My framework begins with a deliberately restrictive principle: construction demonstrates compatibility with the world under particular conditions; it does not automatically establish the truth of the theory that guided the construction.

For an engineering audit, the implication is straightforward:

Do not give an entire pipeline a single “verified” badge when its stages support different conclusions.

Instead, identify exactly what each stage establishes.

First, ask what the AI actually constructed

“AI builds things” sounds clear until we examine what “things” means.

The paper distinguishes three modes of construction: objects, representations, and search processes. Each has a different relationship to evidence. ppl-ai-file-upload.s3.amazonaws

1. Constructing a representation

A predicted protein structure is a representation of a biological object. A weather-model output is a representation of an evolving physical system.

Here, we should separate at least three questions:

Does the representation match observations? Does it perform reliably within its intended scope? Does it explain the underlying mechanism? 

A positive answer to the first two does not automatically answer the third.

This is why the paper separates AlphaFold’s predictive achievement from an explanation of the physical folding process. The distinction does not diminish prediction; it prevents us from assigning it an additional achievement without additional evidence.

2. Constructing an object

Building a physical object creates opportunities for intervention.

We can change a component, alter a condition, or remove a feature and observe what happens.

That can support understanding—but the inferential work still matters.

If several explanations predict the same outcome, a successful build does not tell us which explanation is correct. If an opaque system selected the design, we may understand how to manufacture the object without understanding why it behaves as it does.

The framework therefore treats design transparency and informative intervention as important, rather than assuming that material construction automatically produces understanding.

3. Constructing a search process

Sometimes the central achievement is neither a model nor an object. It is a process for discovering candidates:

Generate → Evaluate → Select → Revise 

The generator explores possibilities. The evaluator determines which candidates satisfy its criteria.

FunSearch and AlphaEvolve occupy this category in the paper’s case set. Their significance lies in the combination of candidate generation and evaluation, not in generation alone. ppl-ai-file-upload.s3.amazonaws

This distinction changes how I would review an AI system.

Instead of asking only, “How capable is the model?”, I would also ask:

What is the evaluator capable of ruling out?

The evaluator deserves its own design review

For an audit, I propose treating the evaluator as a first-class subsystem.

A candidate can pass a check because it is correct. It can also pass because the check is incomplete, insensitive to the relevant error, or based on assumptions shared with the generator.

The paper calls attention to verification severity: whether an assessment has genuine power to expose a claim’s failure, rather than merely demonstrate compatibility with it.

For a practical review, I would ask:

1. What exact claim does this evaluation test?
2. What would failure look like?
3. Which plausible errors could still pass?
4. Does the evaluation depend on the same representation or assumptions used during generation?
5. Can someone outside the original pipeline inspect or challenge the result?

These questions are more informative than a generic statement that an output “passed validation.”

A useful evaluation report should describe not only what passed, but what the test was unable to establish.

Move from system-level confidence to claim-level evidence

Consider a hypothetical AI-generated material.

The announcement might contain several claims:

This table is an illustrative audit template, not a universal validation standard. The appropriate methods depend on the claim and domain.

The principle is to keep the chain of inference visible.

In the paper’s vault-particle discussion, evidence concerning assembly is separated from hypotheses about biological function. That separation matters: showing how something assembles does not automatically establish what it does in its original biological setting.

The same discipline should apply throughout an AI workflow.

Cheap generation is not the same as cheap knowledge

One of the paper’s central arguments is that AI changes the economy of conjecture and verification: it can make candidate generation substantially more accessible without removing the need to investigate the resulting claims. ppl-ai-file-upload.s3.amazonaws

For practical system design, I would separate two questions:

• How strong is the evidence?
• How expensive is it to obtain?

An expensive assessment may be informative. A cheap automated score may miss the central issue.

Cost determines how much validation we can perform. It should not be confused with the quality of the evidence itself.

This suggests maintaining two explicit budgets:

Generation budget: How many candidates can we produce and retain? Validation budget: How many claims can we investigate adequately? 

The second budget should influence the first.

If the pipeline produces candidates faster than they can be meaningfully assessed, the output is a growing queue of unresolved possibilities—not automatically a growing body of knowledge.

For an audit, I would therefore track more than generation throughput:

• Candidates receiving substantive evaluation.
• Claims independently supported.
• Claims rejected or narrowed.
• Unresolved claims awaiting investigation.
• Cost per adequately assessed claim.

These are proposed operational measures. Their purpose is to make the verification bottleneck visible.

Novelty needs its own investigation

A result can be useful without being new.

It can also be new in one sense and familiar in another: a different representation, a rediscovered relationship, or a previously known structure under another description.

The paper examines an A-Lab correction, a challenge to MatterGen, and an ART priority dispute. These cases motivate scrutiny of novelty claims, but they should not automatically be treated as evidence of the same underlying failure.

For a careful audit, I would distinguish:

Known result omitted from comparison ≠ Equivalent result represented differently ≠ Incorrect identification ≠ Unresolved priority dispute ≠ Demonstrated retrieval from training data 

“Not new” describes a problem with a novelty claim.

“Retrieved from training data” proposes a mechanism.

The mechanism requires its own evidence.

A useful provenance record should therefore document what was compared, which equivalences were considered, and what remains unknown about the candidate’s origin.

A deployment milestone is not an epistemic milestone

The paper separates two tracks.

The first concerns deployment:

Possible → Feasible → Constructible → Deployable → Adopted → Normalized 

The second concerns epistemic status:

Conjectured → Verified → Understood 

These tracks answer different questions. A tool can become widely used without providing mechanistic understanding; a result can be verified without becoming part of a deployed product. ppl-ai-file-upload.s3.amazonaws

For a product review, this means adoption should not substitute for scientific validation.

For a scientific review, it means lack of deployment should not automatically count against a well-supported result.

The audit must specify which achievement it is evaluating.

Diagnose the failure before choosing the fix

The framework also proposes distinguishing implementation, parameter, encoding, access, and conceptual failures.

I would translate these categories into a debugging conversation:

The point is not to invent more labels for “something went wrong.”

It is to avoid choosing a remedy before identifying the problem.

A bug may require a code fix.

Poor calibration may require a parameter change.

A representational limitation may require a different model of the problem.

An unsupported inference may require narrowing the claim.

More compute is not an answer to every category.

A lightweight claim record

Here is an illustrative record I would attach to important AI-generated claims:

claim: statement: "Candidate X satisfies property P" scope: environment: "Specified conditions" exclusions: [] generation: method: "Model or search pipeline" version: "Recorded version" assumptions: [] provenance: [] evaluation: procedure: "Named assessment" performed_by: "Identified evaluator" independence: data: "Documented relationship" method: "Documented relationship" organization: "Documented relationship" detectable_failures: [] blind_spots: [] novelty: comparison_sets: [] equivalence_checks: [] unresolved_questions: [] status: assessment: "proposed" independent_replication: false limitations: [] revision: correction_history: [] next_required_test: "Specified follow-up" 

This is a proposal, not an established standard.

Its value is accountability: a reader can inspect how the conclusion was reached, which assumptions it depends on, and what would change its status.

Audit who can challenge the result

Evidence does not exist independently of access.

The paper’s broader audit asks six questions concerning access, authority, memory, contestability, exit, and reversibility. Together, they examine not only what a system makes possible, but for whom.

For developers, I would turn those dimensions into review questions:

• Who can use the system?
• Who selected the representations and assumptions?
• Can relevant provenance be inspected?
• Can outsiders challenge the claims?
• Can users leave the system without prohibitive costs?
• Which decisions can be corrected or undone?

This is why “open” should not be treated as a single switch.

An audit should inspect code, weights, data, outputs, documentation, and permissions separately. The paper uses this layer-by-layer approach when discussing access conditions.

Reproducing the entire training process and checking a particular result are also different activities. A useful release should state which forms of scrutiny it enables.

Treat correction as an output of the workflow

A rejected candidate is not necessarily a wasted candidate.

If the test was informative, rejection can narrow a hypothesis or expose a bad assumption. The paper’s conclusion explicitly treats both surviving and correctly rejected claims as contributions to knowledge. ppl-ai-file-upload.s3.amazonaws

For an engineering workflow, I propose reporting correction as deliberately as success:

What failed? Which claim did the failure affect? What evidence changed its status? Was the candidate wrong, or was the original claim too broad? What should the next iteration preserve or change? 

A pipeline that produces fewer celebrated outputs but clearer corrections may be more useful than one that leaves every candidate in a permanent state of “promising.”

Before calling it a discovery

Here is the checklist I would use before announcing an AI-generated result:

1. State the claim precisely.
2. Separate existence, identification, novelty, performance, and explanation.
3. Map each claim to its supporting evidence.
4. Describe what the evaluation could have missed.
5. Record relevant provenance and prior comparisons.
6. Distinguish internal assessment from independent confirmation.
7. State the conditions under which the result holds.
8. Publish limitations and a correction path where possible.

This does not require every field to use the same test.

It requires the language of the announcement to match the evidence actually available.

Building matters. The inference matters too.

The argument is not that construction adds nothing to knowledge.

Building can make new experiments possible. A representation can support useful predictions. A search process can expose candidates that would otherwise remain unexplored.

The paper’s position is a middle one: construction is a meaningful method of search, while its contribution to knowledge depends on the verification and interpretation surrounding it.
For developers, that leads to a concrete responsibility:

Do not build only the generator.

Build the evidence trail, the evaluator, the provenance record, and the correction mechanism.

When an AI system produces something impressive, ask three separate questions:

What did it construct?

What has been established about the result?

What do we now understand?

The quality of the system depends partly on whether it helps us keep those answers distinct.

About the original paper

This article is adapted from:

Seyed Alireza Alhosseini Almodarresieh. Constructing Possibility as an Epistemic Method: An Onturgic Audit of AI for Science. November 12, 2026. The paper develops the construction typology, parallel deployment and epistemic tracks, failure taxonomy, possibility-horizon audit, and proposed tests discussed here. ppl-ai-file-upload.s3.amazonaws

Original paper on PhilPapers: Constructing Possibility as an Epistemic Method: An Onturgic Audit of AI for Science

The practical templates and developer checklists in this article are adaptations of the framework, not validated standards. The framework itself is presented as a proposal requiring further testing—not as an already confirmed theory.

my site:https://almodarresieh.github.io/