Managers at Verkada, HubSpot, and Marriott report gains from AI coaching tools. But the evidence has gaps: every metric comes from company spokespeople, measurement methods go undisclosed, and no independent audits verify the claims. Here's what we found—and what's missing.
AI coaching closes the gap between when managers need guidance and when they receive it. Traditional executive coaching costs $200–$500 per hour, limiting access to senior leaders. Mid-level managers receive minimal support despite driving most team engagement. Scheduled training happens weeks before or after managers need it.
AI coaching gives managers guidance during real work: preparing for difficult conversations, handling team conflicts, making performance decisions. Tools like Pascal by Pinnacle integrate into Slack, Teams, and Zoom.
The technology differs from chatbots in three ways: it joins meetings and identifies coaching opportunities proactively, it understands team dynamics and past interactions, and it applies structured coaching frameworks like RAPID or the Eisenhower Matrix.
Consider a typical scenario: A manager receives feedback that their team feels micromanaged. Traditional coaching might address this two weeks later, after the damage compounds. A chatbot might offer generic advice if the manager thinks to ask. AI coaching identifies the pattern during actual team interactions—noticing when the manager jumps in to solve problems team members could handle, or when they request unnecessary status updates. It surfaces this observation immediately with specific examples and suggests alternatives.
This real-time intervention creates a different learning experience. The manager receives guidance at the exact moment they can apply it, with context-specific recommendations based on their team's actual dynamics.
Jeff Diana, former CHRO at Calendly and Atlassian (now an advisor to Pinnacle), describes the value: "So much of the real learning comes from in-context coaching in the moment to drive performance and solve problems in the moment."
The timing matters because it determines whether coaching translates into behavior change. The feedback loop compresses from weeks to minutes.
HubSpot reached 98% employee AI tool adoption and 84% comfort level within months, but these metrics measure AI fluency training, not manager effectiveness. The approach focused on removing friction from adoption and providing ongoing support as employees encountered new use cases.
Helen Russell, Chief People Officer at HubSpot, reports that "AI isn't just making work faster; it's helping people perform better—supporting focus, clarity, and real growth."
The implementation strategy centered on three principles: integrate AI coaching into existing communication platforms, provide ongoing coaching on AI collaboration, and track both adoption rates and comfort levels to measure capability building.
HubSpot embedded AI capabilities directly into Slack and other tools employees already used dozens of times daily. This reduced the activation energy required to try AI assistance from several minutes to seconds.
The ongoing coaching component addressed a common failure pattern in technology adoption: the post-training performance cliff. Employees attend a workshop, feel confident for a few days, then encounter a situation not covered in training and revert to old habits. HubSpot's AI coaching provided continuous support, answering questions and offering suggestions as employees encountered new use cases.
The metrics HubSpot tracked reveal their focus on capability building. Adoption rate (98%) measures whether employees use the tools, while comfort level (84%) captures whether they feel confident applying AI in varied situations. The 14 percentage point gap represents employees who use AI tools but haven't yet developed full confidence—a realistic outcome that suggests honest measurement.
What's unclear: The HubSpot case focuses on teaching employees to use AI tools, not coaching managers on leadership decisions. The connection between AI fluency training and manager effectiveness isn't established. Did managers who became more comfortable with AI tools also improve their leadership capabilities? Did teams led by AI-fluent managers show different engagement or performance outcomes? The available information doesn't answer these questions.
Measurement gaps: HubSpot didn't disclose how they measured "comfort level" or what specific behaviors constitute "adoption." Without methodology details, other organizations can't replicate the measurement or verify the claims.
Verkada reports 83% improvement in direct report feedback for managers using Pascal, but the claim is meaningless without methodology details. The company didn't disclose what was measured, who observed the improvement, what instrument was used, over what timeframe, or compared to what baseline.
Pascal joined meetings, identified coaching opportunities in real-time, and provided guidance based on team dynamics. The reported improvements span three dimensions of manager effectiveness: communication clarity, decision-making speed, and team engagement.
Communication clarity addresses whether managers articulate expectations, context, and reasoning in ways their teams understand. Decision-making speed measures how quickly managers move from information gathering to action. Team engagement captures whether direct reports feel motivated, valued, and connected to their work.
These dimensions matter because they represent different failure modes. Some managers communicate clearly but decide slowly, creating frustration through analysis paralysis. Others decide quickly but communicate poorly, leaving teams confused about direction.
The real-time coaching mechanism works by analyzing meeting conversations for patterns that indicate coaching opportunities. When a manager dominates a brainstorming session, the AI might note that team members contributed few ideas and suggest techniques for drawing out quieter voices. When a manager makes a decision without explaining their reasoning, the AI might highlight the value of transparent decision-making for building trust.
Pascal's integration into existing workflows means managers receive coaching without leaving their normal communication channels. The tool joins Slack, Teams, and Zoom meetings as a participant, monitoring conversations and team dynamics. After meetings, it provides private feedback to managers with specific examples from the interaction and suggested approaches for improvement.
What's missing: Did 83% of managers show any improvement? Did managers improve by an average of 83% on some scale? Did 83% of direct reports observe improvement? Was this measured through surveys, 360-degree feedback, or some other mechanism? How long after implementation was improvement measured—one month, six months, a year?
Without this information, the number could represent a massive organizational transformation or statistical noise. CHROs making implementation decisions can't distinguish between genuine capability and marketing claims.
Marriott implemented AI coaching across 120,000+ associates in dozens of languages and hundreds of properties worldwide, but published no outcome data showing whether the implementation delivered value. Victor Arguelles, VP of Learning Design, implemented the program with a "kill zombies" philosophy: encouraging associates to identify outdated processes AI can replace.
The scale of Marriott's implementation presents challenges that smaller organizations don't face. With associates speaking dozens of languages across hundreds of properties worldwide, any learning solution must work across linguistic and cultural contexts. A coaching suggestion that resonates with a manager in Boston might confuse or offend a manager in Bangkok if cultural nuances aren't considered.
Marriott's threshold-based scaling approach addresses a common implementation mistake: rolling out technology before confirming it helps people. By defining specific employee satisfaction metrics and refusing to expand until those thresholds are met, Marriott created accountability for value delivery. The organization only scales AI tools after employee satisfaction reaches defined thresholds.
The "kill zombies" philosophy reflects a mature understanding of change management. Rather than positioning AI as an addition to existing workloads, Marriott encouraged associates to identify what they could stop doing. This framing transforms AI from a burden into a liberation—but only if the organization allows people to abandon the outdated processes AI replaces.
Integration with existing systems presents technical challenges at enterprise scale. AI coaching tools must connect with HRIS platforms, learning management systems, communication tools, and calendar applications. Each integration point creates potential failure modes: authentication issues, data synchronization delays, permission conflicts, and user experience inconsistencies.
Multilingual support extends beyond simple translation. Coaching frameworks developed in English may not translate directly to other languages and cultures. Concepts like "radical candor" or "servant leadership" carry cultural assumptions that don't universalize. Effective AI coaching at global scale requires cultural adaptation, not just linguistic translation.
What's missing: No outcome data. The case describes implementation philosophy without proving it worked. Did manager effectiveness increase? Did employee engagement scores change? Did retention improve? Did guest satisfaction metrics shift? Without outcome data, we can only confirm that Marriott implemented AI coaching at scale, not that the implementation delivered value.
Current AI coaching case studies lack information about failures, limitations, operational details, and situations where the technology doesn't work—preventing organizations from making informed implementation decisions.
Missing from all cases: Failures, limitations, or situations where AI coaching doesn't work. Every technology implementation faces challenges. Some managers resist AI guidance. Some organizational cultures clash with AI coaching approaches. Some use cases prove too complex or sensitive for AI intervention. Some teams experience technical integration problems. The absence of these stories suggests either dishonest reporting or implementations so recent that problems haven't yet surfaced.
Unanswered questions about user experience: What does the user experience look like? Does the AI interrupt meetings or send Slack messages afterward? How does it "join meetings"—as a bot participant or passive listener? How does the AI know when to escalate sensitive topics to human coaches?
These operational details determine whether AI coaching enhances or disrupts workflow. An AI that interrupts meetings with coaching suggestions might break conversational flow and undermine manager credibility. An AI that waits until after meetings preserves flow but reduces immediacy of feedback. The trade-offs matter, yet case studies don't address them.
Privacy and trust concerns remain unexplored in published cases. How do team members react to AI monitoring their meetings? What data does the AI collect and retain? Who has access to coaching conversations and performance data? How do organizations handle situations where AI coaching reveals manager performance issues?
The cost-benefit analysis remains opaque. What does AI coaching cost per manager? How does this compare to traditional coaching, training programs, or hiring better managers? What ROI should organizations expect, and over what timeframe? Without this information, CHROs can't make informed budget allocation decisions.
Measurement standards don't exist. Effective measurement frameworks would track multiple dimensions of impact: 360-degree feedback scores, direct report engagement surveys, decision-making speed (time from problem identification to resolution), and quality of decisions (outcomes achieved relative to objectives). Team performance metrics might include productivity measures, retention rates, internal mobility, and collaboration effectiveness.
The measurement challenge intensifies because AI coaching aims to improve soft skills that resist simple quantification. How do you measure whether a manager "communicates more clearly" or "builds stronger team engagement"? These outcomes require proxy metrics, multiple measurement methods, and longitudinal tracking to distinguish genuine improvement from measurement noise.
Data Breakdown:
• Company: HubSpot | AI Coaching Tool: Not specified | Reported Outcome: 98% AI tool adoption, 84% comfort level | Sample Size: 8,000+ employees | Measurement Period: Within months | Methodology Details: Not disclosed
• Company: Verkada | AI Coaching Tool: Pascal | Reported Outcome: 83% improvement in direct report feedback | Sample Size: Not disclosed | Measurement Period: Not disclosed | Methodology Details: Not disclosed
• Company: Marriott | AI Coaching Tool: Not specified | Reported Outcome: Scaled to 120,000+ associates | Sample Size: 120,000+ associates | Measurement Period: Not disclosed | Methodology Details: Not disclosed
Demand methodology details, start with a pilot, and define success criteria before implementation. The early evidence on AI coaching shows promise but lacks rigor.
If you're considering implementation:
Demand methodology details. Ask vendors for measurement instruments, sample sizes, timeframes, and baselines. If they can't provide this information, treat their claims as unverified.
Start with a pilot. Test AI coaching with a small group of managers (20-50) for 6 months. Measure baseline performance before implementation using 360-degree feedback, direct report engagement surveys, and decision-making speed. Measure again at 3 months and 6 months. Compare results to a control group of similar managers not using AI coaching.
Define success criteria before implementation. What specific behaviors do you want to change? How will you measure those behaviors? What magnitude of improvement would justify the cost?
Plan for privacy and trust. How will you communicate to teams that AI is monitoring meetings? What data will be collected and retained? Who will have access? How will you handle situations where AI coaching reveals performance issues?
Budget for integration complexity. AI coaching tools must connect with your HRIS, learning management system, communication tools, and calendar applications. Each integration point requires technical resources and creates potential failure modes.
The technology shows potential to democratize access to coaching and compress feedback loops from weeks to minutes. But the current evidence base consists of vendor-supplied metrics without independent verification. Treat early adoption as an experiment, not a proven solution.
Header photo by Vitaly Gariev on Unsplash
.png)
.webp)
“Thank you for setting the great foundation for my promotion; now I have a plan!"
Curious to see how AI Coaching can 10X the impact and scale of your development initiatives. Book a demo today for: