A new synthesis for the field of open-endedness

Full list of released papers

The rapid advancement of large language models (LLMs) has produced an atmosphere of inevitability: If some skill is lacking this year, just wait until the next. A couple years ago, the complaint was that LLMs couldn’t reason; now they’re solving Millenium Prize problems! Scaling, that unstoppable force, seems to steamroll deficit after deficit. For what remains that scaling doesn’t fix, technical optimizations and architectural tweaks will probably get us there.

Or so the prevailing narrative goes.

However, creativity and open-endedness raise unique and subtle questions, explored in this release, that reach beyond this simple narrative. Creativity refers to the ability to generate genuinely interesting and novel ideas and hypotheses. Open-endedness describes a continual divergent creative process over time, whose artifacts gain in sophistication and complexity as the tree of exploration diverges ever outwards. At Lila Sciences, because our aim is to achieve scientific superintelligence—which includes the ability to generate the most interesting hypotheses and execute the most high-impact experiments—creativity and open-endedness are mission critical. Science as an ongoing enterprise is an open-ended process of discovery to its core.

Even in this era of massive scale, asking questions about how these elusive pillars of human achievement work algorithmically continues to hold the potential to inject fresh momentum and promising new research directions into the broader field. For example, is training for problem solving and objective performance the same as training for creativity? Do scientific (or artistic) taste and judgment rise to the level of conviction and courage through simply reinforcing being right? More broadly, what are the fundamental principles behind scientific institutions that allow revolutionary ideas to bubble up to the top and receive the investment and resources they deserve? Taking these questions seriously opens a doorway into a fantastic frontier of new possibilities with AI.

Benchmark performance, the north star of so much AI research today, is really only a weak proxy for the intelligence that matters in the world. After all, no one is remembered by future generations for their extraordinary test scores.  Rather, great intellects are remembered for their creative legacy. The most impactful ideas that we put into the world, whether they are scientific, technological, artistic, or social, are the real unit of intelligence that endures into the future. That is why interrogating the open-endedness of AI is so critical for progress.

Of course, open-endedness has been pursued as a research area for decades, first as a branch of artificial life, and eventually within AI (Stanley et al., 2017). But, as with many other areas, the rise of LLMs has profoundly shaped the modern field of open-endedness. And while we are not alone in calling out the essential role of open-endedness in AI (Hughes et al., 2024), open-endedness today largely manifests as a kind of outer wrapper or harness around frontier models, which must carry the burden of divergent creativity whether the models are ready for it or not.

While the LLM’s outer wrapper itself is worth revisiting, to do justice to the nuance and richness underlying how science produces genuine advances in understanding, scrutiny is needed beyond only that outer layer if we are to achieve the kind of open-ended discovery woven throughout the history of human science and civilization itself.

In this spirit of revisiting open-endedness with a more expansive and penetrating lens, here we introduce a novel synthesis of the field, spanning four key components, that reframes open-endedness in the era of capable LLMs. We offer this synthesis as a compass to future research in the field and get the ball rolling with the release of four papers with significant research results in two of the component areas. The time is ripe to kickstart a new era of open-endedness research.

Specifically, to effectively implement an artificial system reminiscent of the open-ended saga of discovery exhibited by humans over the course of civilization, four fundamental issues must be addressed. Some require peering inside the neural networks themselves, while others require rethinking a model's harness or how to manage a portfolio of diverse ideas:

1. The ability to generate genuinely interesting and novel ideas and hypotheses;

2. the ability to judge the interestingness and potential of a proposed idea or hypothesis;

3. the right neural network architecture and training procedure for the model to learn deep regularities that facilitate insights and connections that are otherwise easily missed;

4. and finally, the algorithmic basis to execute the entire scientific discovery loop, including an effective strategy to manage a population of candidate ideas and research directions (and perhaps in the future neural networks themselves), such as deciding where to invest further and what to remember for later.

A danger for the field of open-endedness is an inadvertent overemphasis of one of these issues (such as only thinking about harnessing LLMs in the outer loop) at the expense of others. Because these four fundamental issues are interdependent and synergistic with each other, they must all ultimately be tackled for the full potential of each to be realized.

With all four combined, powerful open-ended systems driving scientific discovery become possible. That’s not to make light of the challenge: Each piece is a major effort in its own right, with deep insights yet to be uncovered. In that spirit, here we present commentary (and in some cases new advances) in each of these four areas.

(1) Generating Ideas: The Creative Paradox and the Advantage of Fresh Eyes

The question of whether LLMs are or can be creative generates plenty of controversy. The skeptic asks why billions of dollars of training has not yet translated into a scientific and technological revolution. The optimist replies that the revolution is beginning, and anyway, just wait, because things are only getting better. Whichever side you take, perhaps we can all agree that it’s fruitful to examine the obstacles and see how they might be overcome.

One way to glimpse the kinds of obstacles LLMs face is to appreciate how difficult and counter-intuitive creativity is for humans as well. For example, becoming better at a subject like math does not necessarily make one increasingly creative in that subject (Sriraman, 2005). And trying to learn creativity itself is notoriously challenging, as creativity is different from the kinds of tasks one learns simply by absorbing data: K.H. Kim (2011) surprised many when she showed that while IQ in school-age children increased after 1990, performance in creative testing actually began to decrease. Furthermore, if more data and experience is always better (as scaling seems to suggest), then why does creative innovation trend towards the young and less experienced (Cui et al., 2026)?

These questions are relevant to LLMs because LLMs reflect a simple philosophical assumption that improvement across many skills and areas of knowledge naturally spills over into creative capacity. Scaling laws and the bitter lesson loom large over this assumption. But creativity doesn’t seem to parallel other areas of intelligence in humans, so why would it in LLMs?

To probe this question, we introduce two novel approaches to improving creative generation that take its idiosyncratic nature seriously. The first paper introduces a counterintuitive insight called the creative paradox: Training a model on examples of past creative ideas can actually inhibit its creative potential. The problem we identify is that anchoring on past examples of creative output can lead to converging towards similar ideas in the future, whereas creativity is about diverging away from existing ideas.

To overcome this problem, we introduce a method called creative priming that disentangles how to think creatively from the creative output itself, allowing reinforcement learning to feed directly into enhancing creative reasoning. That way, instead of anchoring on past examples of creative output, the model can focus on the lessons that can be abstracted from the process that led to its creative output. The result (as shown below) is that the learned primer can significantly enhance key components of creativity when paired with other models, including frontier LLMs.

Alon Albalak, Sebastian Gabriel, David B. D’Ambrosio, Joel Lehman, and Kenneth O. Stanley. "Creative Priming: Decoupling Process from Product to Escape the Creative Paradox." Preprint, arXiv, 2026. https://doi.org/10.48550/arXiv.2611.11740.

In our second paper on creative generation, we tackle the puzzle of youthful innovation by asking whether, contra-bitter-lesson, more data can sometimes be bad, particularly for creative thinking. One advantage of youth is the ability to see the world through fresh eyes, free from common dogma, and thereby organize an entirely new perspective on a field. In fact, the resulting trade-off of being naïve can be mitigated by a skilled mentor, yielding a powerful partnership, e.g. between PhD students and their advisors.

One example of this kind of partnership is Louis Leakey's decision to invite Jane Goodall to study wild chimpanzees at Gombe (Goodall, 1963). Leakey realized that, despite Goodall's lack of any formal scientific training, her patient observational style and complete independence from the conventions of the field yielded an advantage. Their collaboration led to immensely impactful research in primate behavior and human evolutionary biology.

Inspired by the creative advantage of youth, in this second paper we describe a novel method to encourage a network to forget much of a target area, thereby becoming naïve about that area while otherwise retaining its intelligence. Interestingly, such naïve networks tend to generate noticeably—and quantifiably—wilder and more radical ideas, which we then pass on to a “mentor” network to refine into a mature and workable artifact.

David B. D’Ambrosio, Alon Albalak, Joel Lehman, and Kenneth O. Stanley. "The Value of Not Knowing: How Fresh Eyes through Naive LLMs Fuel Creativity." Preprint, arXiv, 2026. https://doi.org/10.48550/arXiv.2611.11725.

Results show that knowing less can indeed confer significant creative advantages. For example, for most topics tested, naive models tend to generate seed ideas that fall far outside the distribution of a comparable baseline model. Furthermore, they are also far from the distribution of a baseline model that is merely prompted to be naive. In fact, as can be seen below, the prompted baseline struggles to meaningfully diverge from the distribution of the baseline itself, highlighting the substantive impact of a genuinely naive perspective. (The visualization shows response embeddings projected into one dimension along the most discriminatory axis. Full details are in the paper.)

To give an example of the kind of wild ideas naive networks produce, when a network trained to forget almost everything related to dental hygiene was asked for ideas for cleaning teeth, it suggested a “tongue stone” that you push around your mouth with your tongue to clean your teeth, almost like a teeth-cleaning fidget toy.

(2) Judging Ideas: The Delicate Matter of Genuine Interestingness

To be able to eventually surface revolutionary ideas, people (or LLMs) typically need to propose many ideas along the road that are not revolutionary. As a result, even given enhanced methods for creative generation, some sort of tool is needed to identify which ideas merit further investment. Interestingly, scientific experimentation is not the solution to this problem, because candidate ideas need to be eliminated before time and effort are wasted on trying them out.  The only plausible solution is a filter that can judge the interestingness of an idea before the idea is implemented.

Of course, the point that not every idea merits investment is not radical. A company or university may yield thousands of ideas every year, few of which are actually tried. While some ideas go through formal review processes, there are far too many even for formal review to evaluate. Accordingly, many are discarded by necessity by gatekeepers (like advisors or lab leaders) who conclude that the ideas are not interesting. The ability to judge interestingness is thereby essential for a creative institution to function. In this way, we need to take seriously in AI the subjective challenge of deciding what is interesting and what is not.

Open-ended systems that push the cutting edge of science will inevitably be forced to engage with this problem. As generating new ideas becomes increasingly cheap and efficient, research labs will be facing a flood of vastly more ideas than even an automated laboratory can reasonably attempt. While researchers have attempted to analyze and formalize interestingness in the past (Herrmann and Schmidhuber, 2026; Lehman and Stanley, 2012), the challenge today is how to engage with the temptation simply to take an LLM off the shelf and ask for its opinion.

As the judgment of LLMs improves, their value as idea filters also goes up, but human scientists seem to go beyond simply assessing untested ideas. Humans become genuinely excited about ideas; they exhibit conviction, often pursuing radical ideas to bitter ends even in the face of counterevidence. Which ideas are truly exciting and deserve a major investment despite forceful logical arguments against them? Should LLMs exhibit qualities like excitement and conviction, and if so, will they emerge naturally at the frontier or is something fundamental missing?

Our team is actively exploring the unanswered questions in this area. Given the coming flood of ideas and hypotheses, a filter, whatever form it takes, will be an integral part of any open-ended system of discovery, just as grant reviewers are integral to funding science and paper reviewers are integral to disseminating it.

‍(3) Representation Suitable for Creative Thought: Inducing Unified Factored Representation (UFR) and Avoiding Catastrophic Forgetting

Acing a test tells us only so much about what a person will do with their skills in the future. Take two people who both get every question right on a math Olympiad exam: One may go on to become a great mathematician while the other does nothing significant in the field of mathematics. If a test cannot differentiate outcomes, what can?

Generating correct answers is only part of the story of prodigious creativity. The other part is how you represent the knowledge you have. There are infinite ways to represent the same expertise, but, within that expertise, some representations are more amenable to insight and imagination than others. For example, representing numbers in base 10 provides many advantages over Roman numerals, though they may express the same underlying concepts.

An experimental result from 2025 illustrates this point visually. In that year, Lila Open-Endedness Team members Ken Stanley and Joel Lehman, with colleagues Akarsh Kumar (the lead author) and Jeff Clune from outside Lila, introduced the Fractured Entangled Representation Hypothesis. Together, they found that even when outputs from a neural network are perfect, some underlying neural representations are better than others, depending on how the network was trained.

In particular, the work revisited images, produced by neural networks, that were discovered through an open-ended process of discovery in an online crowd-sourced experiment called Picbreeder (Secretan et al., 2011). These Picbreeder images exhibit an elegant and well-organized internal representation called unified factored representation (UFR). In contrast, networks trained through stochastic gradient descent (SGD) produced identical outputs yet exhibited a piecemeal, disorganized representation called fractured entangled representation (FER).

Through weight-sweep experiments and examples from large models, the 2025 paper argues that the implications of UFR and FER for creativity, continual learning, and generalization are likely significant.

Motivated by this potentially critical role of representation in creativity, our team has been trying to devise methods to induce UFR in networks trained through SGD. Among likely multiple possible catalysts for UFR, we identified one key observation: Activation patterns of UFR networks appear compressible, in the sense that they exhibit clear regularities (like the symmetry in the activation patterns in blue and red on the left side of the above figure that lead to the “skull” output).

To understand UFR, picture a flock of starlings wheeling across the sky. You don't need each bird's exact position to describe the flock, because every bird is obeying the same simple rule relative to its neighbors, so a few parameters capture the essence of the flock’s behavior. UFR networks behave the same way: The activation levels of different hidden nodes aren't independent facts to be recorded one by one; instead, they're bound together by an underlying regularity, so the whole pattern compresses down to something far simpler than the sum of its parts.

To explicitly induce such regularity in activation patterns, our first paper on representation defines a class of methods called unified factored organizers (UFOs). UFOs regularize the representations of a model during training by sending loss back to them through an autoencoder that tries to compress their activation patterns, providing a surprisingly simple path to UFR on demand.

Thomas X. Sheppard, Alon Albalak, Joel Lehman, and Kenneth O. Stanley. "Are UFOs Real? Introducing Unified Factored Organizers." Preprint, arXiv, 2026. https://doi.org/10.48550/arXiv.2611.11862. Source code

Interestingly, this straightforward technique produces Picbreeder image networks much closer to UFR than conventional SGD, confirming that the autoencoder can induce an approximation of UFR. Here is what the internal FER-ridden representation looks like of a network trained conventionally to output the Picbreeder butterfly image (note how the features are piecemeal and without regularity):

And here is how different (more regular and visually elegant, i.e. more UFR-like) it looks when trained with a UFO:

Even beyond this striking difference, there is a deeper reason UFOs are interesting: We can now investigate the impact of UFR outside Picbreeder networks. That is, the autoencoder approach can be applied to any network. Building on the intuition that UFR should help with generalization and capturing the deeper structure of a problem, we chose to investigate its impact on “grokking” problems, i.e. problems where overtraining leads to a surge in understanding precipitated by a reorganization of underlying representation.

Specifically, we trained networks in modular division (which has been used to investigate grokking in the past; Powers et al., 2022) with the UFO approach. We found that it is possible to consistently reduce the number of transformer training steps required to grok by significant multiples (depending on the specific kind of UFO involved) compared to grokking through conventional weight decay alone. In fact, in one UFO setting, the median grok run assisted by weight decay becomes over 12x faster, as seen here:

These results show not only that it may be possible to induce representation approaching UFR, but that FER could be a critical factor in the sample inefficiency of deep learning. Significantly reducing the need for data could impact training pipelines while yielding networks better suited to creativity, continual learning, and generalization.

Regarding continual learning, strangely, despite all the significant advances in their performance, today’s LLMs cannot be continually trained without degrading what they learned earlier, a phenomenon called catastrophic forgetting. In our second paper on representation, we show that, as long as the pattern being learned is within the affordances of the network, networks that exhibit UFR are intrinsically resistant to catastrophic forgetting. In other words, networks have to be “ready” to learn a new topic or skill, like a student who has taken (or grokked) the necessary prerequisites. If they are prepared, UFR networks can absorb changes without the usual catastrophic forgetting. Remarkably, no other special technique is needed—the UFR network simply avoids catastrophic forgetting on its own.

Joel Lehman and Kenneth O. Stanley. "Unified Factored Representations Resist Catastrophic Forgetting." Preprint, arXiv, 2026. https://doi.org/10.48550/arXiv.2611.11020.

A fun way to see this capability is to watch a UFR network (left) from Picbreeder outputting the “skull” image side-by-side with an FER network from convention SGD (right) outputting the same skull pattern. Over and over again, both networks learn and unlearn a “smile” pattern that replaces the original expressionless mouth, and we can observe forgetting (or not) in all the pixels outside the mouth (which are not subject to learning). The resultant video gives an intuitive visualization of how catastrophic forgetting quickly corrupts the conventional SGD FER network while the UFR network sustains far less damage.

The implications for continual learning are stark: Good representation appears fundamental to the ability to absorb new knowledge without breaking old knowledge. Mechanistic interpretability research often focuses on how networks work rather than whether they actually work well even when they output the right answer. As results with FER and UFR show, there is much to learn if we stop assuming representation always works well.

(4) Implementing the Entire Scientific Discovery Loop

Science (or any open-ended creative enterprise) is not only about having a good idea. In aggregate, science is a continual churn of diverse ideas. At any given time, numerous diverse ideas of different quality are in various stages of development, investment, and implementation.

Ultimately, the entire scientific discovery loop, or wheel of science, encompasses a repeating succession of critical operations. It often begins with the generation of ideas about what might be interesting to investigate or even just to think about, which then leads to more refined hypotheses. Some of these hypotheses then become experimental designs that are executed to obtain results. Those results are analyzed and scrutinized for new insights, which leads to new ideas, and the loop repeats. At Lila Sciences, our hope is that this loop ultimately manifests as an open-ended process of discovery in collaboration with human participants, pushing ever forward the frontier of human knowledge.

Since the advent of LLMs, researchers have introduced a variety of systems to capture the open-ended nature of science. Examples include Sakana’s AI Scientist (Lu et al. 2026) and Google’s CoScientist (Gottweis et al., 2026). The lineage of LLM-driven evolutionary coding systems, beginning with Evolution through Large Models (ELM; Lehman et al., 2023)— which includes Deepmind’s AlphaEvolve (Novikov et al., 2025; reproduced publicly as OpenEvolve, from Sharma 2025)—and SkyDiscover (Lui et al., 2026), capture the spirit of the wheel of science by leaning on LLM-driven intelligent mutation, while general auto-research approaches crystallize the idea of a relentless push towards ever-improving performance (Karpathy, 2026). Many such systems draw ideas and inspiration from quality diversity (QD) algorithms, which provide a foundation for maintaining high-quality and diverse candidates in an ongoing search process (Mouret and Clune, 2015; Pugh et al., 2016).

While these approaches are diverse, they are often motivated either by the desire to produce an artifact worthy of publication or a solution of higher quality than what has come before to an outstanding problem. All these contributions have enhanced our ability to automate the scientific process and push the cutting edge of progress. Upon this considerable body of work we ask a further question: Can the automation of science succeed in further illuminating what science actually is? That is, could one expect a formalization of the wheel of science to offer not just a tool, but an interpretation (perhaps even like a work of art or depiction) of what science actually does?

An example of ideas we’re exploring is in the below panel. It demonstrates (in creative coding and circle-packing) a system that keeps humans in the loop of a living tree of open-ended discovery. Each node is a run by a swarm of AI agents that write code, run experiments, and evolve a world model on a particular project. Each project is itself a shareable, branchable unit: Anyone can take a run, point it at a related project, and carry its world model along. Rather than starting cold, a branched run builds on everything learned by its ancestors.

As seen above, in the circle-packing domain, it took only 22 evaluations to match the best-known result for 28 circles after branching from a run that already matched the record for packing 26 circles. And then it again took few evaluations to match the record for 32 circles a couple of branches later. This rapid succession of record-level discoveries highlights why the ability to branch from one problem to another is so potentially powerful. Together, these branches form a symbiotic network of humans and agents, each compounding what the others discover.

Our hope is that this kind of system will ultimately offer a meditation on how the scientific enterprise can be decomposed conceptually, and a functional realization of this conceptualization that pushes sciences forward.

Towards Scientific Superintelligence

Advances in the field of open-endedness are already feeding into scientific discovery at Lila. With an expanding array of AI Science Factories (AISFs), capable of autonomous physical experimentation in both life and material sciences, open-ended hypothesis formation and entire open-ended workflows are becoming possible. The benefits of such workflows compound because new learnings from a round of open-ended discovery make the next round even more insightful and innovative. Access to the physical world is an essential grounding to this open-ended flywheel.

The unprecedented potential for AI-driven discovery is why now is the time for a new synthesis around open-endedness and creativity. Open-endedness is one of the few areas of intelligence that does not follow the usual principles of machine learning. To push the frontier forward, the AI community needs the courage to recognize that there is more we could be doing than what currently feels inevitable.

And in the midst of widespread fascination with objective benchmarks and measures, the fear of subjectivity within the field is increasingly becoming a barrier to our imagination. We need to recognize that when the next frontier for the field is the land of new ideas themselves, subjective judgment will become unavoidable. The directions suggested here offer a glimpse of the opportunities that arise when we start questioning fundamental assumptions. Here we initiate those questions by asking whether it makes sense to optimize directly for creativity, whether more knowledge is always better for creative models, whether representation could be far better than what conventional gradient descent achieves, and whether catastrophic forgetting is actually a symptom of bad representation.

And we’re just getting started. Some may believe that the story of AI has reached a zenith of scaling and bitter lesson indigestion, but the time for disruptive thinking and fundamental insight is not over. Open-endedness is the next frontier for the field—and it doesn’t follow the usual rules.

Full list of released papers

Alon Albalak, Sebastian Gabriel, David B. D’Ambrosio, Joel Lehman, and Kenneth O. Stanley. "Creative Priming: Decoupling Process from Product to Escape the Creative Paradox." Preprint, arXiv, 2026. https://doi.org/10.48550/arXiv.2611.11740.

David B. D’Ambrosio, Alon Albalak, Joel Lehman, and Kenneth O. Stanley. "The Value of Not Knowing: How Fresh Eyes through Naive LLMs Fuel Creativity." Preprint, arXiv, 2026. https://doi.org/10.48550/arXiv.2611.11725.

Thomas X. Sheppard, Alon Albalak, Joel Lehman, and Kenneth O. Stanley. "Are UFOs Real? Introducing Unified Factored Organizers." Preprint, arXiv, 2026. https://doi.org/10.48550/arXiv.2611.11862. Source code

Joel Lehman and Kenneth O. Stanley. "Unified Factored Representations Resist Catastrophic Forgetting." Preprint, arXiv, 2026. https://doi.org/10.48550/arXiv.2611.11020.