As AI agents take on more responsibility across the enterprise, organizations need systems they can trust. By combining Quotient’s capabilities with Databricks AI research and platform, customers can deploy AI agents that not only run in production, but improve over time. Built by the engineers who led quality improvement for GitHub Copilot, Quotient analyzes full agent traces to detect issues and generate signals that drive continual monitoring and performance improvement.
Databricks announced the acquisition of Quotient AI, an innovator in evaluation and reinforcement learning for AI agents. Quotient helps enterprises monitor agent behavior in production, detect critical issues, and use those signals to continuously improve agent performance.
„As organizations deploy AI agents into business-critical workflows, they need reliable ways to ensure those systems perform as expected in the real world. By bringing Quotient into Databricks, we are strengthening Genie, Genie Code and Agent Bricks with continuous evaluation and learning, empowering both developers and organizations with AI agents that become more accurate, reliable and specialized over time.” – according to the press release.
Solving the Production Challenge for AI Agents
A new challenge emerges as enterprises move AI agents from pilots to production: how to reliably measure, debug and improve performance at scale. Since agents are complex compound systems of models, memory, tools, skills, and other components, teams need to not only evaluate quality, but also quickly gain insight into the root cause of failures. Without a robust evaluation system that can interpret system failures, fixing issues to improve performance becomes slow and product leaders fail to gain confidence to ship to production.
Introducing Quotient
„Quotient was built to close the gap in agent evaluation and continual learning. Created by the engineers who led quality improvement for GitHub Copilot, the team brings deep expertise measuring and improving large-scale AI systems.” – the company said.
„The Quotient platform analyzes full agent traces from production systems to detect issues such as hallucinations, reasoning failures and incorrect tool use. These signals are automatically clustered and transformed into structured evaluation datasets and reward signals that can be used to monitor and fine-tune agents. This approach enables organizations not only to observe agent behavior, but to systematically improve it, helping AI systems become domain experts that continuously learn from real-world usage.” – Databricks explained.
Quotient + Databricks
Databricks already provides powerful tools for evaluating and improving AI agents. With Quotient, we’re strengthening these capabilities across our platform by embedding a continuous evaluation and improvement layer that provides customers with more accurate and reliable AI systems. This strengthens three key product areas across Databricks:
. Genie is an AI agent that lets any employee chat with and get insights from their data
. Genie Code, also launching today, is an autonomous AI agent that helps data teams plan, build, and run data engineering, machine learning, and analytics workflows
. Agent Bricks makes it easy for organizations to build and scale high-quality agents on their data
„Freddie Vargus and myself started Quotient AI (acquired by Databricks) in 2023 with a dream and a hunch that the next era of AI will be unlocked by systems that can measure, evaluate, and improve AI in the real world.” said Iulia Neagu, co-founder and CEO of Quotient, a PhD in physics at Harvard.
She continued: „Over the past couple of years, we partnered with companies ranging from AI-first startups to Fortune 500 enterprises. They used Quotient to monitor AI systems in production, ensure their agents met strict enterprise policies, and reinforcement fine-tune agents to consistently achieve double-digit performance gains.
We built it all at the bleeding edge, with a small team that started in Boston, Massachusetts. Joining Databricks gives us the scale and resources to take this much, much further.„
Among the founders of Databricks are two computer scientists of Romanian origin, Ion Stoica and Matei Zaharia.
_________
Related content: Introducing Genie Code
Banking 4.0 – „how was the experience for you”
„To be honest I think that Sinaia, your conference, is much better then Davos.”
Many more interesting quotes in the video below:
