Study finds AI co-workers lifted team forecasting accuracy 18%

A new preprint from researchers at Unanimous AI and Carnegie Mellon University says proactive AI “scouting agents” improved real-time human team forecasting in MLB game meetings. The study found higher accuracy, lower error and strong participant support for AI co-workers in group deliberations.

Why it matters: - The study suggests AI can improve not just individual work, but live group decision-making. - The results point to a new role for AI in meetings: filling information gaps without replacing human participants. - The findings could influence how enterprise teams use AI for forecasting and collaborative planning.

What happened: - Researchers from Unanimous AI and Carnegie Mellon University tested proactive conversational AI agents in real-time human team meetings. - The work appears in a new preprint paper, "Evaluating the Participation of Proactive Scouting Agents in Real-time Human Team Meetings." - Small human teams of four or five people forecasted the winners of Major League Baseball games over multiple weeks. - Teams performed the task with and without AI assistance. - When a proactive scouting agent joined the discussion, forecasting performance improved.

The details: - Human teams improved raw predictive accuracy from 60% without AI assistance to 76% with the (Co)agents. - The proactive agents reduced mean-absolute forecasting error by 18%, with p=0.006. - The agents accounted for 18% of total conversation time and did not disrupt human interaction. - 86% of participants, or 116 people, said the scouting agents were genuinely helpful, with p<0.001. - 90% of respondents said enterprise business teams will want to use these agents in future forecasting processes, with p<0.001. - The agents monitored conversation in real time, identified knowledge gaps, searched the open internet, and added information to the discussion. - The system was designed to support human teams rather than replace them or present competing opinions. - Researchers said the key challenge was making the agents follow human group etiquette. - Hans Schumann said the team was encouraged that human teams made smarter predictions and that more than 80% of participants found the comments timely, relevant and helpful. - The study says proactive agentic assistants work best when human participants already have a baseline level of forecasting skill.

Between the lines: - The study frames AI less as a chatbot and more as a digital meeting participant. - That shift matters because forecasting often fails when teams miss outside information or overlook critical gaps. - The results also suggest adoption may depend on whether AI can contribute in a way that feels natural in live conversation. - The mention of Microsoft Teams, Google Meet, Slack and Zoom signals a possible path for broad workplace integration, though those platforms were not said to have launched the feature.

What's next: - Unanimous AI is positioning the technology for enterprise forecasting and team collaboration use cases. - The company says the approach could be used in real-time meetings on major collaboration platforms. - Further testing will likely focus on whether the performance gains hold across other tasks, industries and team sizes. - Unanimous AI says more information is available here.