Nowable's Reasoning Guide

Most modern language models today are the reasoning kind. Reasoning has done a lot of good for AI and deserves praise. It allows the models to create better and more elaborate answers and overall it is what makes today’s models “feel” more intelligent. It is also what makes it possible for AI agents to perform longer running and more complex tasks. At the same time reasoning is one of the primary sources of the explosion in token spend that we saw in 2026.

Reasoning models typically cost 5 to 20 times more per task than standard models because of internal thinking tokens. EY has quantified the combined effect for a single customer service interaction: a simple linear workflow cost $0.04 per interaction in 2023, while the same job done by an orchestrated system with tools, reasoning, and iterative loops costs $1.20 in 2026. That is a 30x increase per task.

While the blended cost of AI has fallen 67% year over year, total enterprise AI bills are rising because usage is growing faster. This stems from two simultaneous dynamics:

• The reasoning multiplier: reasoning tokens make up as much as 80% of a reasoning session and are billed as output tokens
• Agentic context accumulation: every API call sends the full accumulated context as input. A 50-step session can have 35,000 input tokens per call at step 50, compared with 5,000 at step 1. A loop running 10 cycles can consume 50 times the tokens of a single linear pass

While the agentic context accumulation is important, it is a topic for another day. What I will focus on here is the reasoning multiplier. 

In 2026 AI can do more, and that costs more, but what exactly are we paying for and are we getting a corresponding value realisation? In other words, how reasonable is reasoning and does reasoning quantity translate to reasoning quality? Is there such a thing as good or bad AI reasoning and if so, what does that look like?

How reasoning came about

Reasoning models originate from OpenAI, who released their groundbreaking o1 model in 2024. The underlying technique, Chain-of-Thought (CoT), had been known since 2022 as a prompting trick: ask a model to think step by step before answering, and its answers improve. OpenAI's contribution was to train that behaviour into the model itself.

During that work OpenAI reached an important insight: models can become markedly more intelligent simply by being given time to "think" before answering. This is called test-time compute (TTC) and is the key to reasoning as we know it today: simply turn up the volume of tokens allowed during the answering (inference) phase.

This is considered the second scaling paradigm in AI. Up until this discovery the AI scaling law was primarily considered something that applied to training: more parameters, more data, larger models lead to better performing models. However, recent efforts had only yielded marginal results. TTC just cemented what the industry already knew: that bigger models are not enough. What really moved the needle was scaling the effort. The underlying finding was inspired by how humans think. Give us an easy question and we will answer right away. Give us a hard question and we will have to think harder and longer. This is System 1 and System 2 thinking as popularised by Daniel Kahneman. Just as with humans OpenAI’s insight was that accuracy in the output improves predictably with the effort, in this case volume of tokens a model produces.

Before 2024 long outputs were considered a quality problem. A model that wrote 4,000 words instead of 400 was considered to be rambling. However, by training a model explicitly on verifiable results like coding, logic and math OpenAI demonstrated it was possible to shape long outputs into something productive. They didn’t reveal how, though.

There is also a deeper, structural reason it works. All modern AI models are based on the transformer architecture. Transformers process tokens in parallel through a fixed number of layers, which places a mathematical ceiling on what they can compute in a single pass. Writing intermediate steps lets the model use its own output as working memory, breaking through that ceiling. In other words reasoning tokens are more than just “more time”; they are a structural workaround for an architectural limit.

In essence OpenAI showed that if a model is allowed more thinking tokens, more compute can be shifted from training time to the moment of answering. And this is the origin of the massive growth in token spend we are witnessing today.

The second breakthrough came in January 2025, when DeepSeek's R1 showed that the reasoning recipe was far simpler than anyone assumed. OpenAI had kept its method closed, and the industry broadly assumed it rested on enormous amounts of human-annotated thinking data. DeepSeek demonstrated that reasoning can simply emerge from a model on its own, using a technique called Reinforcement Learning with Verifiable Rewards (RLVR): reward the model for correct answers on tasks that can be checked, and it discovers reasoning by itself.

The R1 team documented an “aha moment” during training in which the model spontaneously began writing “Wait” in its traces, pausing to re-check just before catching an error. Nobody taught it that. Self-verification appeared as a by-product of chasing correct answers.

After DeepSeek documented this to the world it was no longer a hidden recipe inside a closed lab.  Several smaller breakthroughs have followed since to improve the quality of the answer, like parallel-TTC, where models run several thinking traces in parallel and aggregate, which improves the answer precision, before more recent efforts have turned to the topic of reasoning efficiency and reasoning routing, which is where we find more recent concepts like adaptive thinking, introduced to the market by Anthropic in 2026.

Adaptive thinking is supposed to make reasoning efforts more granular and Claude users can today pick between a ladder of reasoning effort levels across their panel of models. However, this is basically routing work pushed onto the user like ChatGPT’s infamous model selector back in 2024. True adaptive thinking that dynamically routes reasoning without involving the user has been demonstrated in research in 2026 (SAT and AdapThink) but has yet to be implemented at scale.

Reasoning is one of the most active and heavily researched areas of improvement in AI today. There is a lot of incentive to improve because it is directly linked to model capability. At the same time token consumption which started with reasoning is directly linked to current chips and memory constraints. While reasoning made models much more capable it came at a steep cost of increased token usage, which today means that it claims 75-80% of token use on demanding tasks. For organisations and individuals who are trying to control their token spend, understanding the mechanics of reasoning is important in order to analyse and control this part of AI. Unnecessary or bloated reasoning is wasteful, but unfortunately still quite common across the models used today.

Distillation: how reasoning spreads

By now it would be reasonable to assume that a model needs enormous scale to think properly. The surprising truth is that reasoning has turned out to be a behaviour rather than a property of size. And behaviour can be taught.

The teaching method is called distillation. A large "teacher" model writes out thousands of complete reasoning traces: the decomposition, the intermediate steps, the self-corrections, the conclusion. A smaller "student" model is then trained on those traces. The student inherits the teacher's method rather than its vast knowledge. DeepSeek demonstrated this as well in January 2025, releasing distilled versions of R1 at a fraction of the size that suddenly beat far larger non-reasoning models on math and coding. This is why a small open model like Google's Gemma 4, 31 billion parameters and small enough to run on a single GPU, reasons today at a level that was reserved for the frontier two years ago.

It is also the key to understanding why the release of Moonshot AI’s Kimi K3 matters. K3 is a 2.8-trillion-parameter model, the largest open model ever released, and self-hosting it takes a cluster of around 64 high-end GPUs, so almost nobody will run it themselves. But that is not where its impact lies. From a closed model, the best a distiller can do is collect its answers, which the closed labs both forbid in their terms and actively frustrate by hiding their traces. Just like DeepSeek made their RLVR methodology available to the world, the open weights of K3 change the game entirely. With the weights in hand, a student model can be trained not just on what K3 writes, but on the full probability landscape behind every token it produces: how confident the teacher was, which alternatives it weighed. That is a far richer teaching signal than text alone, and K3's permissive license makes it explicitly legal. K3's biggest footprint will likely be the generation of small, cheap reasoning models trained in its image rather than the answers it gives itself.

For token economics, that carries a consequence worth holding on to for the rest of this guide: model size and price are no longer reliable signals of reasoning quality.

What reasoning looks like: a technical anatomy

What exactly is the difference between an ordinary language model and a reasoning model? In essence an ordinary language model writes the answer directly, one token at a time, left to right, committing to each word as it goes. It is fast, and it is impulsive: the model binds itself to its first inclination without weighing alternatives.

A reasoning model on the other hand inserts a phase before the answer: a thinking trace in which it decomposes the problem, tries a path, checks it, and only then writes the clean reply. Think of two students at an exam. The first model writes the answer straight down. The second works it out on scratch paper first, then writes the fair copy. The second uses more time and more paper and catches their own mistakes along the way.

Not all models let us observe their reasoning. The closed American models either hide or compact their thinking traces because this can be used for distillation. So, if we want to study reasoning qualitatively, we have to study the open models.

When you do that, you discover that reasoning is not only more words but actually relies on a certain set of reasoning tokens which direct the path of thinking. There are two levels of reasoning tokens: container tokens that delimit the thinking phase, and semantic thinking tokens that govern what the model does inside it.

A container token looks something like this:

The model uses the tokens <think> and </think> to delimit the thinking process. Within the thinking phase, the model uses specific words as control signals for what it is cognitively doing.

There are several semantic tokens each of which plays a role in the reasoning trace.

A common type is Reflection tokens typically expressed like

Wait is the most studied reasoning token. It functions as a trigger for self-correction and initiates a reflection step with a probability above 90%. This acts as a semantic signal that changes the model's next move.

Transition tokens are another common reasoning token type. Typically they are expressed with words like

The token Alternatively functions as a dedicated branch indicator, leading to a divergence step in over 95% of cases. The model will explore alternate paths, meaning the model splits its probability mass almost evenly between deductive statements and concluding summaries.

Other tokens have a similar effect and together they can be grouped into four distinct cognitive patterns in reasoning traces, which direct different behaviours and are observed across models:

These tokens act as control signals rather than filler. When they appear, they steer the model's next move. Their appearance is not random: it is shaped by training, which embeds the habit, and by the task, whose structure calls it forth. But it is hard to reproduce or reduce to exact rules, because language models are probabilistic by nature and every thought trace is unique. The same token can be a genuine turning point in one trace and a learned verbal tic in the next.

Three reasoning issues

If you have ever read a reasoning trace it should be clear to you the thinking done by these machines is by no means perfect. Watching it unfold can be equally awe-inspiring and amusing. You can argue that human thinking may not be pretty either, but then again you are not charged on a meter for the messy thoughts of your carpenter or dentist, only for the job they do using that thinking. Maybe that is where the AI economy will go someday. For now you pay for it all. It is a package deal and part of that package is a set of unresolved issues. Let’s consider some of the most prominent problems that reside in reasoning token architectures today. There are many more, but I selected three that directly relate to token usage.

Problem 1: Overthinking

The most widespread problem and most directly related to token spend. Models treat trivial questions and highly complex math problems with the same heavy, long-form token generation. This results in unnecessarily long, redundant, or overly complex reasoning sequences  even for simple tasks. This increases latency and cost, and can actually degrade answer quality.

Research has shown a "cascade effect": a single Wait-token increases the probability of further wait tokens by up to 100 times over the next 20 positions, an autoregressive amplification that shows systematic overconfidence in these tokens. The model puts itself in a reflection loop that produces redundant tokens without improving the answer. This remains one of the central challenges in building efficient reasoning models in 2026.

Problem 2: Reward hacking

The model finds shortcuts to maximize its reward signal without performing the intended task. Typical manifestations are reasoning without an actual solution, meaningless repetitions of valid steps and correct final answers via incorrect intermediate logic. This means that thinking essentially becomes a tool used by the model for the wrong reasons: to satisfy the model’s own inner reward system, at your expense.

While reward hacking might sound aggressive it is simply a consequence of how models are designed and trained today. Some researchers like Yoshua Bengio point to this coexistence of agency and goal seeking behavior as the main design flaw in current AI systems because it leads to this type of misaligned and uncontrolled behaviour. Today some frontier reasoning models actually explicitly state their intent to cheat within their internal thinking. When researchers tried to train this out, the models did not stop cheating. They learned to hide the intent while continuing to behave undesirably. Today the field is shifting towards rewarding process (PRM) rather than just grading the answer to mitigate this issue.

Problem 3: Token complexity and memory crisis

When an AI model faces a difficult problem, it cannot just guess the answer. It has to explore multiple different logical paths, almost like playing out several moves ahead in a game of chess.

As the model searches all these paths, the sheer volume of generated data skyrockets, a metric known as Token Complexity. While token complexity is usually discussed as a financial problem, because every single generated token costs money, it also triggers a severe physical hardware bottleneck.

When an AI model "thinks" for a long time, its high token complexity means it has to remember every single word and idea it has generated across those inflated search paths. To do this, the computer chip sets aside a dedicated section of its ultra-fast memory called the KV Cache, which acts as the AI's internal "scratchpad."

The higher the token complexity, the more pages the AI fills on its scratchpad. Holding that entire long conversation in its head at once takes up a massive amount of computer memory. The hardware simply runs out of room.

This is why companies running these models in practice cannot let everyone think deeply at the same time and the problem exists regardless of whether you are hosting your own model or using APIs. They have to process requests in batches rather than all at once whenever token complexity spikes. It is a hard physical limit of hardware memory, not just a token cost issue.

How reasonable is reasoning?

So what exactly are we paying for? The honest answer: a genuine capability and its overhead, bundled together and billed by the token. Reasoning earned its place. It works around a real architectural limit, and it carries the tasks that were out of reach two years ago. But the meter runs on everything: the disciplined deduction and the seventeenth recount of a word limit alike.

Three things are worth carrying out of this guide.

First, reasoning quality is a separate question from reasoning quantity. Models spend their tokens with wildly different discipline, and the difference lands directly on your bill. This is documented in my own research.

Second, capability has come loose from size and price. Distillation keeps moving frontier reasoning into small, cheap models, and every open release of teacher weights accelerates the shift.

Third, reasoning can be inspected, provided the model lets you see it. Models that show their thinking are the ones you can actually evaluate, which makes visible reasoning a requirement to write into the contract when you buy AI.

The industry is working on the waste: adaptive effort, better routing, process rewards. Until that work lands at scale, the burden of reasonable reasoning sits with the buyer.

Now that you know what reasoning is all about head over to my essay Penny for your thoughts, which is part of my tokenomics research, where I look 12 models over the shoulder and assess how they think and what that means for you, if you chose to hire them.