Not all feedback is good feedback when partnering with AI

More than 500 early-career professionals scored nearly identical on critical thinking, domain knowledge, and AI literacy. A new field study from UT Austin's McCombs School of Business and KPMG then put them to work with AI agents on real tasks, and the results split sharply: Some pulled meaningfully ahead of what the AI produced alone. Others barely matched it, or fell behind, despite starting from the same baseline.

Same skills, different results

Workers who excelled "treated AI as a collaborator that needed direction, oversight, and judgment," rather than handing off a task and walking away. 

The underperforming group did something that looked like effort but rarely moved the work forward: They reviewed AI's output, offered feedback, and pushed back on what came back. The trouble was where they aimed that feedback. Their critiques chased details that didn't matter or nudged the model further from what the task actually needed, leaving the output about where it started.

Ashish Agarwal, professor at UT Austin and co-author of the study, framed the research goal beyond simple tool literacy. The team wanted to understand what lets some people consistently "create value beyond what AI can produce on its own." 

Feedback is a skill, and most people never learned it

The pattern will sound familiar to anyone who has managed people: Vague feedback rarely helps a junior employee improve, and it rarely helps an AI agent either. Telling a model to make something better, or telling a person to try harder, gives neither of them anything to act on. What actually moves output forward is specific: this reasoning breaks down because of X, this framing misses the real ask, redo the analysis with this constraint in place instead. That level of specificity already lives in how a good manager reviews a new hire's early drafts, long before AI entered the picture.

Rahsaan Shears, AI enterprise transformation leader at KPMG US, pointed to why this should worry leaders who assume fluency is the answer. Today's early-career workforce is the most AI-native generation to enter the job market, so comfort with the tools runs high across the board already. What separated the two groups in this study was how people applied what they knew, rather than how much access to AI they had. Shears called the difference coachable: "That gap is coachable."

Consider AI your intern

At Pinnacle, this research confirms something we've believed for a while. The skills of working with AI are the skills of managing. Feedback, standards, and teaching a system what good looks like through repetition, these are management fundamentals applied to a collaborator that happens to be a model instead of a person.

A good manager checks a new hire's work early, corrects the reasoning behind a bad decision alongside the decision itself, and builds judgment through repetition and specific feedback, well before two weeks pass without a look. Treat AI the same way. Review its first attempt at a task closely, name exactly what's off about the reasoning, and give it a second try with that correction built in. Do this consistently, and the model's output on similar tasks starts to track the standard you actually set, drifting further from its factory default with every round of feedback.

Making an invisible skill visible

Researchers offered a concrete way to build this into an organization, beyond individual habit. They recommended having employees document why they accepted, changed, or rejected each piece of AI output, then grading that process alongside the final deliverable. That single change turns a skill most companies have never measured into something a manager can coach directly.

‍