Eureka

Chapter 8 · Peirce 1903, Reichenbach 1938, Frankfurt 1958, Hanson 1958, Simon 1973

Chapter 7 ended at a failure. A goal can select which available variables matter, but the cause may require a variable the agent cannot yet express. No wider search over the same vocabulary repairs the omission. The hypothesis is not hidden in a large haystack. The language contains no needle.

This book reached the gap by subtraction. Deduction unfolds consequences. Induction judges them. The diff locates surprise. Framing selects available variables. Economy ranks candidates. Search enumerates a supplied space, and analogy transports structure already represented elsewhere. Remove each implemented operation from the loop and eureka remains.

Discovery is too broad a name for the residue. One may discover an island on a map or a hypothesis in an enumerated space. Eureka discovers a hypothesis that no space enumerable in the current representation contains. The answer is not a candidate with low prior; it is not a candidate. The inquiry has successor functions, but the missing concept lies outside their closure. Eureka extends the representation. Only then do new candidates, branches, and searches become available.

Peirce called abduction "the only logical operation which introduces any new idea" and included under it the operations by which theories and conceptions are engendered. The claim gave eureka a place in logic. It did not supply its operation.

We do not supply it either. We can mark the boundary, separate nearby operations often mistaken for generation, and state what a future theory would have to explain.

The target

An inquiry begins with an observation O represented in a vocabulary V₀. Every hypothesis expressible in that vocabulary fails. The missing operation produces a changed vocabulary V₁ and a hypothesis H that the old vocabulary could not state:

(O, V0) → (V1, H)    where H ¬in Express(V0)

The hard output is V₁, not H. Ordinary search assembles the hypothesis once the relevant variable, predicate, mechanism, or object exists. Before it exists, there is no candidate to rank and no proposition to test.

Call this conceptual generation: the construction of a representation that makes a previously inexpressible hypothesis available. It differs from finding a surprising value for a known variable. It changes what can count as a variable.

The closure is relative to the inquiry. That relativity is no claim that the target is mathematically uncomputable. A larger system may already contain the representation. That fact no more makes the answer available to this inquiry than an alphabet makes every unwritten book available to its author. The open problem concerns availability: how failure directs an extension beyond the operations currently licensed.

The interface without an implementation

Eureka is visible at its boundary. An inquiry can record what it knew before the leap and what it could express afterward. Each human instance of eureka has some physical history, but that does not prove a stable abstract operation exists. The behavior may arise from many contingent mechanisms rather than one encodable procedure. Attempts to decompose it have so far recovered its inputs, outputs, and neighboring operations without recovering the transformation itself.

The surrounding loop supplies inputs:

• an observation the current representation does not explain;
• the live purpose that makes the observation consequential;
• the frame that must remain invariant;
• the failed hypotheses and the exact modes of their failure.

The operation returns a changed representation and at least one candidate hypothesis. Its output must satisfy a contract:

• the new representation expresses a distinction unavailable before;
• the candidate makes the original surprise expected;
• the candidate preserves what the frame held fixed;
• the candidate entails a discriminating exposure.

eureka(surprise, purpose, frame, failures) -> (representation, hypothesis)

That is an interface, not an implementation. The caller can validate the return value without knowing how the function produced it. Every mechanism surveyed below either implements a neighboring operation or searches a meta-space of representations. The meta-space relocates the question: what made the needed representation available there?

The claim here is narrower than uncomputability: known mechanisms explain reachability once their operators are supplied. They do not yet explain how the structure of a failure supplies an operator outside the inquiry's current closure.

Three kinds of generation

The word generate hides three different achievements:

Enumeration can be enormous and still closed. A theorem prover searches more formulas than a person could inspect, but its grammar decides which formulas exist. Recombination can be astonishing and still inherited. A language model joins concepts no author joined before, but its output remains composed through a learned representational space. Novel output does not by itself demonstrate a new space.

Conceptual generation changes the basis on which enumeration and recombination run. Its novelty is not merely that nobody produced the output before. The prior system could not formulate the distinction the output requires.

Peirce's schema begins one step late

Peirce gave abduction its durable form:

The surprising fact C is observed.
If A were true, C would be expected.
Therefore there is reason to suspect A.

Harry Frankfurt identified the circularity in 1958. The schema cannot explain the invention of A because A already appears in its second premise. It explains why a conceived hypothesis deserves a place on the docket. It does not explain conception.

The distinction divided twentieth-century philosophy of science. Reichenbach separated the context of discovery from the context of justification. Ideas arise in the first; evidence evaluates them in the second. Popper made the separation methodological: the origin of a conjecture could belong to psychology while logic began with its exposure to falsification. Hanson resisted the exile and argued that discovery follows intelligible patterns around anomaly.

Herbert Simon refused the exile outright: discovery has a logic because discovery is ordinary problem solving, a claim he defended against Popper in print and then in code. His opening argument is one sentence long: if there is no logical method of having new ideas, "then there is no such thing as a logical method of having small new ideas." Popper's claim was universal, so a counterexample at any scale refutes it, and Simon chose his examples small enough to expose their machinery. The dispute moved discovery from philosophy into AI. It did not close the gap Frankfurt named.

Eight answers that presuppose the answer

Each mechanism below solves enumeration or recombination and borrows the representation it needs:

1. Heuristic search over supplied terms. From Simon and Langley's group, BACON rediscovered Kepler's third law by combining given variables until an invariant appeared. It demonstrates that closed-space discovery is mechanizable. It also inventories what must be supplied: the quantities, the operators, and data already carved to fit them.
2. Inference to the best explanation ranks explanations already available.
3. Analogy carries a represented relation from a represented source into a represented target. It does not explain the source, the mapping, or why that similarity became relevant.
4. Bayesian model selection updates over a supplied model class. A prior with no mass on the needed representation cannot discover it.
5. Program synthesis searches programs admitted by a supplied grammar.
6. Inductive logic programming invents predicates inside a supplied logical language and mode bias.
7. Evolutionary search varies a supplied genotype with supplied mutation operators and selection criteria.
8. Language-model generation samples and recombines a learned manifold. A surprising completion may reveal a representation in the weights, but surprise to the reader is not evidence that the model created its representational primitive during the inquiry.

The objection to these operations is bookkeeping. Each expands the reachable set relative to a smaller search procedure, and calling them insufficient is not calling them useless. Every procedure has a closure, and conceptual generation is the change to the closure that the procedure itself did not already license.

The bookkeeping objection is older than any system on the list. A reader of Simon's 1973 draft objected that his examples prejudice the case because "the range of alternatives can be delimited in advance," so they proved nothing about innovation that redefines the range. Simon answered twice. First, the objection rescues nothing for his opponents: their impossibility arguments drew no line between normal and revolutionary discovery, so his small counterexamples already refute them. Second, he refused to concede the line itself: recursive rules spawn infinities from finite primitives, so why deny that revolutionary hypotheses are "the products of this kind of generation of much from little"? The second reply is the meta-space move in its original form. An infinite generated space is still a closure; the primitives and the generation rule were supplied.

Laudan argued Simon had changed the subject from justifying new ideas to searching efficiently. Chalmers, French, and Hofstadter pressed the substantive version: BACON's representations arrived hand-carved by its authors. The carving, which Chalmers and his coauthors call high-level perception, is the part of cognition the program skips. Battleday and Gershman recently renamed the residue the hard problem of AI for science and proposed studying the scientists who solve it. Simon's empirical exhibit deserves a closer replay than any of these give it.

The Mendeleev test

Simon's exhibit against the boundary was Mendeleev. The periodic law, he argued, needs no pattern machinery beyond his letter sequences: order the elements by atomic weight, note the periodic recurrence of valence. He read the case as evidence that even revolutionary discovery may be effective search. Replayed microscopically, the case divides into three acts that land on opposite sides of the closure boundary.

The enumeration was real, and it was closed. Once atomic weight orders the elements, every grouping hypothesis (a period length, a family partition, a recurring valence) is a finite expression over a countable ordered set. Simon's own formalism certifies the closure: successor on an ordered set defines a cyclic group, and the space of such patterns is enumerable in the natural numbers. Independent multiples are the signature of a closed space, and the historical record supplies them. De Chancourtois, Newlands, Odling, and Meyer all reached periodicity within a decade. Once the vocabulary exists, enumeration is cheap and several searchers arrive at the same point.

The act that closed the space happened upstream, and Simon concedes it in a subordinate clause: Mendeleev discovered "just a few years after the notion of atomic weight had been clarified." The clarification was Cannizzaro's, circulated at the 1860 Karlsruhe Congress, which the young Mendeleev attended. Before Karlsruhe, competing weight conventions gave contradictory orderings, and "order the elements by atomic weight" was not a runnable operation. The conceptual generation happened there. The enumeration that followed was lawful because the space it swept had just been carved.

The closed space then produced its own inexpressible anomaly. Weight order is subtly wrong: tellurium outweighs iodine, and Mendeleev swapped them against his own key to preserve the chemistry. The anomaly has the exact shape Chapter 7 ended on, a persistent surprise that no rearrangement within the supplied vocabulary resolves. The repair required a variable outside the closure: nuclear charge, proposed by van den Broek and measured by Moseley in 1913. Under atomic number the elements embed in the natural numbers literally, and the inversions dissolve. The failure even localized the extension: the new key had to preserve the chemical sequence and break its tie to mass.

So the case Simon offered against the boundary is a miniature of the loop this book studies. A eureka carves the space; enumeration sweeps it; independent multiples confirm its closure; an inexpressible anomaly accumulates; a second eureka re-carves. His account covers the sweep. It is silent on the two carvings that bracket it.

Placeholders before meaning

If learning is hypothesis testing, the learner must represent each hypothesis before testing it. A hypothesis containing a genuinely new primitive therefore presupposes the primitive it is meant to explain. Fodor pressed this paradox into the gap's sharpest negative form, accepted the resulting continuity, and treated lexical concepts as unlearned primitives.

Susan Carey's Quinian bootstrapping addresses hypotheses that a learner cannot initially state. The learner acquires explicit symbols as placeholders and represents relations among them before fully understanding what they mean. Partial interpretations connect the placeholders to existing concepts. Analogy, modeling, limiting cases, thought experiments, and induction then constrain the new system until its symbols acquire content.

Carey's account is more than search over finished hypotheses. The placeholder structure lets syntax arrive before semantics and gives the learner something new to compute with. It therefore corrects any blanket claim that no mechanism has been proposed.

Bootstrapping leaves the directed step partly open. A teacher, notation, or scientific tradition supplies the placeholders and much of their relational structure. The account describes how an initially uninterpreted structure can become grounded. It does not provide a general operation that maps the shape of any failed inquiry to the placeholder structure that inquiry needs.

The shape of the open problem

Four boundaries recur across the literature:

1. Level. Boden calls a change to the rules of a conceptual space transformational creativity. Wiggins formalizes it as exploration in a meta-space of conceptual spaces. The reduction works only after a meta-language and its operators are fixed, so it moves the closure boundary outward.
2. Direction. Predicate invention, program synthesis, and evolutionary systems can extend an object language through meta-level search. Their supplied modes, grammars, and mutation operators determine which extensions are reachable. The unsolved part is how a particular failure constructs or selects an operator beyond that closure.
3. Grounding. Introducing a new symbol is cheap. The hard achievement is making it denote the distinction that resolves the surprise. Carey's placeholders explain how use can precede full meaning, but the transition from constrained symbol system to new conceptual content remains contested.
4. Locus. The operation may not live inside one mind. Language, diagrams, instruments, collaborators, and material models can hold intermediate representations that no individual could construct internally. An implementation of eureka may therefore be a distributed process rather than a function of one agent.

Espírito Santo, Wiggins, and Cardoso make the boundary measurable. Their definition calls an experience transformative when a learner outputs a different conceptual-space hypothesis after encountering it. That definition detects the interface transition. It deliberately treats the learner as a function and does not explain how the function constructs its next hypothesis.

The resulting open problem is narrower than creativity and stronger than search: explain how the structure of failed interaction directs the construction and grounding of an operator outside the inquiry's current closure. A theory may use a meta-space, external symbols, or social machinery. It must account for where those resources came from and why this failure recruited them.

Need constrains without constructing

Chapter 7 showed that purpose frames perception. The tempting next sentence is that need gives birth to the frame the act requires. That sentence states the desired result and skips its mechanism.

Need does work. A persistent prediction error says the current representation is inadequate. The shape of the failure localizes what the replacement must preserve and what it must separate. The tellurium inversion named both: preserve the chemical sequence, break its tie to mass. Motivation allocates attention and effort. None of this explains how constraints on an unknown representation construct that representation.

Powers called the response to persistent intrinsic error reorganization: the control hierarchy changes until error falls. Boyd argued that an orientation must be destroyed and rebuilt when it ceases to fit the world. Both explain why representation changes and how selection retains an improvement. If the variation is random or drawn from supplied operators, neither explains directed conceptual generation.

The research conjecture is stronger:

The structure of a failed inquiry can direct construction of a representation not enumerable from the inquiry's current vocabulary.

We do not yet know whether that sentence describes an operation, a family of historically contingent tricks, or an impossibility disguised by hindsight.

What surrounds the gap

The missing center does not make the surrounding machinery optional. Without that machinery, a generator emits possibilities with no disciplined relation to the inquiry. The preceding chapters construct a surprise, isolate its figure, preserve its ground, and frame the variables the current purpose makes relevant. The following chapters rank candidates, expose them to possible loss, read the evidence, and retain the path through failure.

That machinery gives conceptual generation four constraints:

• The new representation must explain the located surprise.
• It must preserve the frame that the surprise did not disturb.
• It must deduce an exposure that could distinguish it from live rivals.
• When killed, it must leave a failure precise enough to constrain its successor.

Constraints can make invention productive without making it mechanical. The machinery punts the scientist to the edge of the unknown, and its value lies in where on that edge they land and what they arrive holding. Michelson could not cross to relativity, but his interferometer delivered Einstein to a specific point on the boundary, and he arrived carrying the exact invariance the new representation would have to honor. Methodeutics formalizes the transport. It does not yet formalize the leap.

What a theory would owe

A theory of conceptual generation would have to meet four tests:

1. Novelty. The output cannot be expressible in the input vocabulary under its existing composition rules.
2. Direction. The observed failure must constrain the change more sharply than blind variation does.
3. Closure accounting. The theory must name every grammar, ontology, body, environment, and operator it presupposes.
4. Replayability. A stranger must be able to inspect why the new representation answered the surprise, even if the act that produced it cannot yet be replayed.

The third test prevents an infinite retreat into a hidden super-language. If a system searches a meta-grammar that already contains V₁, it has enlarged the closed space. It has not explained the creation of that meta-grammar. Simon granted the premise in passing: failure to detect a pattern may mean the pattern employs "relations that are not within the program's competence." The test asks every mechanism to declare that competence up front. A theory may still proceed by nested closures, but it must say where each closure came from and stop claiming generation where it only searched.

The fourth test separates genesis from justification. We may never replay the moment a representation arrived. We can still replay what the representation buys. The old anomaly becomes a deductive consequence, new consequences become visible, and a trial can make the claim lose. The hypothesis graph begins there: it records justification even where it cannot replay genesis.

The handoff

The remaining book starts after a candidate exists. Chapter 9 asks which candidate deserves the next experiment. Parts III and IV show how evidence updates, kills, composes, and converges. None of those later successes should be read backward as an account of where the candidate came from.

Simon put down a marker at this boundary. Before accepting that revolutionary science escapes laws of effective search, he wrote, "we would do well to await more microscopic studies than have generally been made to date of the histories of revolutionary discoveries." The Mendeleev test above is one such study, and it relocated the leap rather than dissolving it. The hypothesis graph is an instrument for producing more: it records what an inquiry could express before the leap, what it could express after, and which failure stood between them.

Methodeutics keeps the larger ambition. Its central unsolved problem is eureka in an open hypothesis space. This textbook contributes a method for active inquiry around that problem and a specification the missing operation must someday satisfy. The language still contains no needle. We have written down what would count as one.

Exercises

💻 marks exercises meant for a keyboard. ★ marks open-ended problems with no single right answer.

8.1 Classify each of the following as enumeration, recombination, or conceptual generation, then defend the hardest classification by naming what that operation was supplied: choosing a diagnosis from a medical differential; importing natural selection into an account of ideas; inventing temperature as a measurable quantity; searching programs in a domain-specific language.

8.2 Apply Frankfurt's objection to Peirce's schema. Write an abductive inference from smoke to fire, then underline the first place fire enters. What part of the inference remains valuable after you concede that it did not invent fire?

8.3 Choose an analogy that produced a useful hypothesis. List the source relation, target objects, mapping, and relevance criterion. Which of those were already represented before the analogy ran? State precisely what the analogy generated and what it borrowed.

8.4 💻 Define a tiny grammar for arithmetic expressions and enumerate everything it can express to depth four. Add a new primitive such as sin and enumerate again. Show one expression available only after the extension. Which program performed search, and which act changed the hypothesis space?

8.5 ★ Find a conceptual invention from your field and reconstruct the before and after vocabularies. Name the anomaly the old vocabulary could not express cleanly, the new distinction, and the first discriminating consequence the distinction made testable. Then identify the point in your reconstruction where the historical record stops explaining and starts narrating the invention after the fact.

Sources

• Peirce on Abduction — abduction in the context of discovery and Frankfurt's objection that the new idea already appears in the schema.
• Scientific Discovery — the discovery/justification distinction and attempts to recover rational patterns of discovery.
• Hans Reichenbach, Experience and Prediction (1938) — the canonical separation of discovery from justification.
• Norwood Russell Hanson, Patterns of Discovery (1958) — anomaly and theory-laden observation as a return to the rational structure of discovery.
• Harry Frankfurt, "Peirce's Notion of Abduction" (1958) — why Peirce's inference licenses adoption rather than explaining invention.
• Herbert Simon, "Does Scientific Discovery Have a Logic?" (1973) — discovery as ordinary problem solving, against Popper's exile of discovery from logic; section 6 records the closure objection and both of Simon's replies.
• Langley, Simon, Bradshaw, and Zytkow, Scientific Discovery: Computational Explorations of the Creative Processes (1987) — BACON and its siblings rediscover empirical laws by heuristic search over supplied variables.
• Larry Laudan, "Why Was the Logic of Discovery Abandoned?" in T. Nickles (ed.), Scientific Discovery, Logic, and Rationality (1980) — the charge that Simon's heuristics answer a different question than the classical logic of discovery asked.
• Chalmers, French, and Hofstadter, "High-Level Perception, Representation, and Analogy" (1992) — BACON runs on hand-coded representations; building the representation is the cognition the program skips.
• Battleday and Gershman, "Artificial Intelligence for Science: The Easy and Hard Problems" (2024) — current AI solves supplied optimization problems; formulating the problem itself requires continual conceptual revision.
• Geraint Wiggins, "A Preliminary Framework for Description, Analysis and Comparison of Creative Systems" (2006) — exploratory and transformational creativity formalized as object-level and meta-level search.
• Susan Carey, "The Origin of Concepts: A Précis" (2011) — Quinian bootstrapping through placeholders, partial interpretations, and modeling.
• Espírito Santo, Wiggins, and Cardoso, "Towards a Formal Creativity Theory" (2024) — an operational definition of transformation as a change in the learner's conceptual-space hypothesis.
• Dai and Muggleton, "Predicate Invention in Inductive Logic Programming" (2012) — language extension reduced to meta-level abductive search.
• Chandrasekharan and Nersessian, "Building Cognition" (2015) — scientific discovery through constructing and manipulating external computational representations.