Blogs · September 2026

The Second Scaling Axis

For a decade, AI got better primarily by training bigger models. In 2025, a second scaling axis became impossible to ignore: models were also getting better by thinking longer. That shift changes not only how much compute AI consumes, but what kind of compute matters, and what that compute can make possible.

More inference can now produce a better answer at the moment a model is used. It can also produce the attempts from which the next model learns. In science, the same pattern can search across proteins, genomes, medicines, materials, and experiments before anything is built in the physical world.

We believe the cost of inference will increasingly determine the pace of discovery.

A second axis

The first clear signal arrived when OpenAI published o1 in September 2024. The consequential claim was about scaling: performance improved along two axes, with more reinforcement learning during training and with more time spent reasoning over each problem at use. On the 2024 AIME, o1 solved 74% of problems with a single attempt and 93% when a scorer selected among a thousand.

By 2025, this was no longer a result from one lab. Compute-matched studies showed that inference, allocated well, could let a small model outperform one fourteen times its size. DeepSeek-R1 demonstrated the training side in the open, using reinforcement learning against verifiable answers to take a base model from 15.6% to 71.0% on the same exam.

This second scaling axis has a different computational shape from pretraining. It generates sequences one token at a time, carries forward everything produced so far, and often runs many attempts whose lengths cannot be known in advance. Arithmetic throughput still matters, but so do memory bandwidth, memory capacity, scheduling, and the cost of keeping state alive.

Inference now improves models twice: directly, by allowing more work on each task, and indirectly, by generating the rollouts used to train the next model.

Two years of scaling along this axis produced results that had recently seemed far away. In July 2026, two models scored a perfect 42 out of 42 at the International Mathematical Olympiad under official grading, receiving the problems only after the human contestants had finished. At the 2025 ICPC World Finals, an OpenAI entry solved all twelve problems under contest conditions, which no human team managed. ARC-AGI-2, a benchmark designed to resist memorization, went from 4% to a verified 95% in eighteen months.

The first large jump on its predecessor showed the price of this approach. Reaching 87.5% required 1,024 attempts per puzzle at a reported cost of about $4,560 each. Capability was bought with inference, and quite a lot of it.

The significance is larger than any benchmark. Once additional search and verification reliably improve an answer, the useful amount of inference is no longer bounded by the number of questions people ask. It is bounded by how much productive search can be performed economically.

Autonomous science

This is already becoming a field. Research groups, frontier labs, and a growing number of startups are connecting reasoning models to predictive models, simulators, robotic laboratories, and experimental instruments. What recently looked like a collection of isolated demonstrations is beginning to look like a new computational model for research: autonomous science.

The next step is to close the loop. An autonomous scientific system forms a hypothesis, designs candidates, selects an experiment, observes the result, and uses that evidence to decide what to try next. The laboratory performs the physical experiment, but inference determines how its limited time should be spent.

This is a different workload from serving a chatbot. It is persistent and iterative. It can generate large candidate populations, invoke multiple models and simulators, maintain state across long-running campaigns, and change direction as evidence arrives. Its value comes not from producing a token quickly, but from reaching a verified result with the least time, energy, and physical experimentation.

As these systems scale, inference becomes part of the scientific instrument itself.

Inference becomes infrastructure

The same operation is becoming part of the model factory. Reinforcement learning for reasoning works by having a model generate many complete attempts at a problem, score them, and only then update its weights. Generating those attempts is called rollout. Rollout is inference.

The same paper notes that producing each token is constrained by memory bandwidth. And rollout-heavy post-training is becoming a larger part of total model development: in February 2026, Cursor reported that post-training its coding model consumed more compute than pretraining the base model from which it started.

Every improvement in generation throughput makes reinforcement learning cheaper. Cheaper reinforcement learning permits more attempts, richer environments, and better models.

When reward comes from checking an answer, progress depends on having somewhere reliable to check it: a repository with failing tests, a replica of a website, a terminal with a task, a simulator, or a laboratory instrument.

These environments have become an industry. Epoch AI's survey of environment builders priced a basic website clone at about $20,000 and a complex product clone near $300,000, with individual tasks costing $200 to $2,000. It also found that leading labs were both building environments internally and buying them. One founder called maintaining quality while scaling “the number one bottleneck.”

Pretraining data still matters, but it is no longer the only scarce input separating frontier systems. The quality of environments and checkers increasingly determines what models can learn. That constraint is also a compute constraint: an environment must run the model inside it, often thousands of times, to observe what it does.

When OpenAI retired a widely used coding benchmark in February 2026 after finding that most audited failures came from broken tasks, the lesson was straightforward: the checker sets the ceiling.

Three quantities are rising together: the number of tasks run, the number of tokens used for each task, and the time for which each task remains active.

Google guided 2026 capital expenditure to $180–190 billion, roughly six times its 2022 level. Reasoning models emit about eight times as many tokens as their predecessors on the same questions, while reported output lengths have been growing about fivefold a year. Agents now operate for hours at a time while retaining accumulated context in memory. Deloitte estimated that inference represented about half of AI compute in 2025 and roughly two-thirds in 2026.

The first wave of inference infrastructure primarily served human requests. The next will increasingly serve machines conducting search: agents writing and testing software, models generating training rollouts, and autonomous systems exploring scientific possibility spaces. Their demand is not bounded by human attention. They can continue searching whenever the expected value of another attempt exceeds its computational cost.

Hardware for discovery

Reasoning models, reinforcement-learning rollouts, long-running agents, and computational design loops are not identical workloads. But they increasingly share a set of demands: repeated generation, large persistent state, variable-length parallelism, and continuous evaluation across many possible paths.

Autonomous science combines all of them. A system may reason about an objective, generate millions of candidates, score them with predictive models, call simulators and laboratory tools, incorporate the results, and begin the next search. It may run for hours, days, or longer.

General-purpose accelerators can run these workloads, and their vendors are adapting. But for important regimes, particularly low-batch decoding, long-context generation, and persistent agents, performance can be constrained as much by moving and retaining data as by arithmetic throughput.

We believe a workload this large, specific, and central to future progress deserves hardware designed around it from the beginning. The relevant measure is not peak operations per second, or even the cost of producing a token. It is the cost and energy required to reach a correct, experimentally useful result.

Why Grafen

We are not building Grafen only to make today's AI services less expensive. We are building it because cheaper, more efficient inference expands the frontier of what can be searched.

It determines whether a researcher can evaluate thousands of candidates or millions. It determines whether individualized design remains available only to the best-funded institutions or becomes an ordinary tool. And it will help determine whether autonomous science remains a collection of demonstrations or becomes infrastructure available to laboratories everywhere.

Our goal is to make computational search cheap and abundant enough that far more researchers can use it to discover medicines, materials, and experiments that are out of reach today.