Large language models — they’re just like us. Or at least they’re trained to respond like us. Now, they’re even displaying some of the more inconvenient traits that come along with reasoning capabilities and “overthinking.” Reasoning models like OpenAI’s o1 or DeepSeek’s R1 have been trained to question their logic and check their own answers. However, if they dwell on these evaluations for too long, the quality of their responses begins to decline.
“The longer it thinks, the more likely it is to get the answer wrong because it’s getting stuck,” Jared Quincy Davis, the founder and CEO of Foundry, told Business Insider. Relatable, isn’t it?
“It’s like if a student is taking an exam and they’re spending three hours on the first question. It’s overthinking — it’s stuck in a loop,” Davis elaborated.
Davis, alongside researchers from Nvidia, Google, IBM, MIT, Stanford, DataBricks, and others, introduced an open-source framework called Ember, which may signal the next evolution of large language models.
Overthinking and Diminishing Returns
The notion of “overthinking” could appear to contradict another significant advancement in model enhancement: inference-time scaling. Just a few months ago, models that required additional time for more thoughtful responses were praised by AI pioneers like Jensen Huang as the future of model improvement.
Davis acknowledged that reasoning models and inference-time scaling represent significant strides forward but suggested that future developers may adopt different approaches to utilize them.
He and the Ember team are formalizing a structure around a concept that has been explored in AI research for some time. Nine months ago — an eternity in the machine learning realm — Davis described his technique of repeatedly asking ChatGPT 4 the same question to gather the best responses.
Now, researchers at Ember are enhancing that method, envisioning integrated systems where each question or task could draw from a combination of models, each with varying thinking times tailored to what works best for that particular query.
“Our system is a framework for building these networks of networks where you want to, for example, compose many, many calls into some broader system that possesses its own properties. It’s like a new discipline that I think has rapidly transitioned from research to practical application,” Davis commented.
In the Future, the Model Will Choose You
When humans overthink, therapists often advise breaking problems into smaller, manageable pieces and tackling them individually. Ember begins with that concept but quickly diverges from it.
Currently, when users log into platforms like Perplexity or ChatGPT, they select their model using a dropdown menu or a toggle switch. Davis predicts that this will change dramatically as AI companies pursue improved results through more intricate strategies that route queries through various models, each employing different numbers and durations of calls.
“You can imagine, instead of being a million calls, it might be a trillion or quadrillion calls. Sorting the calls becomes essential,” Davis explained. “You must choose models for each call. Should every call use GPT-4? Or would some calls benefit from GPT-3? Should some inquiries direct to Anthropic or Gemini, while others call DeepSeek? What should the prompts be for each query?”
This approach moves beyond the binary question-and-answer paradigm we have known, becoming increasingly significant as we transition into an era of AI agents that perform tasks autonomously.
Davis compared these sophisticated AI systems to chemical engineering, stating, “This is a new science.”
