TLDR
We scored 75 AI coaching conversations on five dimensions of relational quality. The advice was detailed, structured, and confident. The weakest dimension was whether it was about anybody in particular.
I have spent eight years arguing that coaching should not be a privilege.
For most of that time the argument was about access. A coach cost hundreds of dollars an hour, one coach could only hold so many relationships, and coaching went to the people already closest to the top. I believed the day we solved that would be the day this got good.
We solved it. You can put coaching in front of every manager in your company this quarter.
And I no longer think access was the hard part.
Our first assumptions are almost always wrong, and mine was that once coaching reached everybody, the question left over would be adoption. It isn’t. It is what the coaching says when it gets there, and specifically what it says about the other person in the situation.
So Cloverleaf Labs built five common workplace conflicts and ran each one through five leading LLMs, three times each. Every conversation started professionally and escalated across three messages as the employee got more frustrated, which is how a real conflict tends to sound after someone has been sitting with it for a while. Three trained reviewers independently scored all 75 final responses on five dimensions: self-awareness, accountability, awareness of the other person, specificity of the advice, and whether the response moved the employee toward repairing the relationship or protecting themselves from it.
Every dimension came in below the neutral midpoint.
The weakest was specificity, meaning whether the advice was about the actual people involved rather than the shape of the situation. It scored 2.2 out of 5.
Not because the answers were thin. Because they were about nobody.
LLMs are telling your employees to quit.
There is a huge relational risk inside of AI rollouts. We ran the five most common workplace conflicts through five leading LLMs. See what your people are being told, and what it is costing the relationships your business runs on.
What is the best way to scale personalized coaching?
Assessment results, workflow integrations and AI have removed the constraint that made coaching a privilege. A thousand managers can get something useful on a Tuesday morning for a fraction of what one executive coach used to cost. Every guide to this subject explains how: mix your formats, train managers as coaches, add a digital layer, start with the populations where it matters most.
That advice is fine. It is also now everywhere, which tells you it has stopped being the hard part.
What almost none of it asks is what happens to the coaching itself when you stretch it across a thousand people who each have a different situation and a different colleague on the other side of it.
Is coaching still effective once it is automated?
On measured outcomes, the evidence has moved in the wrong direction, and I was wrong about this too.
I had assumed nobody was measuring whether coaching survives automation. Somebody was. In 2022, Terblanche and colleagues ran a randomised controlled trial of an AI coaching chatbot over six months, 75 participants against 94 controls. It produced a statistically significant gain in goal attainment, effect size 0.60. On wellbeing, resilience and perceived stress it produced nothing. The authors called that effectiveness “in a narrow application” and concluded AI could still make coaching available cost-effectively.
Then in 2026, de Haan, Terblanche and Nowack ran a randomised controlled comparison of accredited human coaches against an AI coach across 114 coachees, published in Human Resource Development International. The human arm showed substantial effectiveness at mid-to-high effect sizes. The AI arm showed none.
Same research program, just four years later, and worse result.
One finding in that second trial has stayed with me. What predicted whether someone benefited was their starting self-efficacy and hope. The people who arrived already confident got something out of it. The people who needed the development most got the least.
That is not our finding and it would be an overreach to say it is. Different method, different outcomes, different research group. But it is the same shape, and our own results found it from the opposite direction: the coaching got worse exactly where the employee had less power.
So the honest state of the evidence is this. Outcome measurement of coaching at scale exists, and right now it favours human coaching. What does not exist is any measurement of what the coaching says about the other person. That is the narrower claim, and it is the one I can defend.
What makes coaching that uses AI personalized versus generic?
Whether it knows anything about the other person in the situation. That is the difference, it is measurable, and neither of those trials looked at it.
Here is what that lowest-scoring dimension actually looked like. The responses were long. They were well organized. They were full of concrete steps. Schedule a one-on-one. Document the pattern. Prepare your talking points. Clarify ownership in writing. Follow up within the week.
Detailed advice about nobody in particular.
Detailed is not the same as personalized. A tool can produce a confident, structured, entirely actionable plan while knowing nothing about the two people who have to carry it out. It is the hardest failure to catch, because a generic answer delivered with enough detail reads exactly like a personal one. Nobody reviewing that answer thinks “this is generic.” They think “this is thorough.”
One caveat travels with that 2.2 everywhere I use it. Specificity is also the dimension where our three reviewers agreed with each other least, and we are rewriting how it is defined for the next round. The score will move. What holds is that the weakest of the five dimensions was the one about knowing who is involved.
Why were the longest answers from LLMs the worst responses?
Because length came from tactics, and tactics came from treating the other person as an obstacle.
This one surprised me. Reading through the responses, the pattern the three of us kept noticing was not that short answers were lazy. It was that the longest answers were the most adversarial. A brief response tended to be generic and harmless. A very long one had room to build a strategy, and the strategy was usually about how to work around a colleague rather than with them.
I want to be careful here. This is an observed pattern in 75 responses, not a measured correlation yet. We noticed that specificity tracked with response length, we are separating those two things out, and until we have, treat it as the thing we saw rather than the thing we proved.
But it changes what I would look at in a demo. If you evaluate a coaching tool by how substantial its answer feels, you are measuring the variable that ran in the wrong direction.
Why even the best ranking responses from LLMs were still not all that useful.
The responses that scored highest on relational quality, the ones that challenged the employee’s account and asked what they might be contributing, were often the least practically useful. They offered a point of self-reflection and very little on how to handle the actual situation on Thursday. Meanwhile the responses that gave the most usable tactics were the ones most likely to cast the colleague as an adversary.
Strong on reflection, light on help. Strong on help, hard on the relationship. That trade-off showed up across models, and I do not think it is an accident.
It matters commercially because of what people actually want. When someone brings a work conflict to an AI tool, they are not looking to be asked a series of thoughtful questions. They want to know what to do. Which means the market rewards the version that hands over tactics, and the tactical version is the one that scored worse on every relational dimension we measured.
That is a real problem for my own category, and I would rather name it than wait for someone else to.
Help employees approach their next conversation differently
Explore Cloverleaf’s approach to AI coaching for the relationships people navigate every day.
Who else is benchmarking how LLMs coach employees?
Sam Paech has been building public benchmarks for the relational behavior of language models since 2023. EQ-Bench puts models through 45 scenarios, mostly three-turn role-plays covering relationship conflict, parenting decisions and workplace situations, and scores how they handle them. Spiral-Bench runs 30 twenty-turn conversations against a simulated user and scores each reply against a named taxonomy: pushback, de-escalation, boundary setting and “validate feelings not thoughts” count as protective, while sycophancy, escalation and delusion reinforcement count against the model. An ensemble of three AI judges scores every conversation.
That was reassuring, and not only because someone independent arrived at a similar method. Look at the protective list again. Validate feelings, not thoughts. That is the distinction our whole rubric is built on, written down by someone with no commercial interest in coaching at all.
Here is what those benchmarks do not do, and it is not a criticism of them. They score how a model behaves inside a scenario. Nobody yet scores whether two real colleagues end up better able to work with each other after one of them asked for advice. The frontier labs benchmark coding, mathematics, reasoning and safety. If these systems are going to sit in the middle of how people give feedback and resolve disagreements, the relational layer deserves a benchmark too. Our full study, including the power dynamics we did not expect to find, is here: what leading LLMs tell employees in conflict. I would call it a starting point rather than a standard.
How can coaching remain personalized at scale?
Scale usually means one of three shortcuts. Personalize by role, so every new manager gets the new-manager track. Personalize by topic, so someone working on feedback gets the feedback module. Or personalize by conversation, so whatever the person types is all the system has to go on.
The first two are filters. They pick from a library based on a category someone belongs to, and employees work that out inside a week. The third looks like real personalization, and it is the one we measured.
Think about what the tool has in that moment. It knows how I take feedback, because I just told it. It does not know how she gives it. It knows what I find frustrating about her. It does not know what she is trying to accomplish. It knows the version of her I described while I was annoyed. It does not know her.
That asymmetry is the whole mechanism. It is why specificity scored lowest, and why the advice pointed away from the relationship far more often than toward it. The model is not being careless. It is doing the best possible job with one side of a two-sided problem.
Across the 75 conversations the models produced 638 distinct pieces of advice. We coded every one. Three of the 638 coached the employee to connect with, show vulnerability toward, or genuinely invest in the other person.
The fair objection is that plenty of responses did tell the employee to go and talk to their colleague. They did. But pointing someone toward a colleague to secure their cooperation is a different thing from investing in the relationship, and the coding separated the two before we scored anything. What the majority sounded like, verbatim:
“Since direct talks with her haven’t worked and she pivots away from reality, stop investing in those one-on-ones.”
“The goal shifts from earning trust to protecting your effectiveness and confidence while managing his behavior as a constraint.”
Both are precise. Both are actionable. Both are about a colleague the tool has met only through one frustrated account of her. Across all 75 responses on relational repair, 52% landed in self-protection, 36% stayed neutral, and 12% coached genuine investment in the relationship.
Nobody sets out to build a company of people who have become excellent at tolerating each other.
RESEARCH REPORT
Your people are already asking LLMs about their teammates.
Organizations are considering an LLM as their coaching layer. But there is a relational problem hiding inside the rush to adopt AI at work.
- LLMs are telling frustrated employees to leave
- The less power an employee has, the worse the advice
- LLMs give a lot of advice, and almost none of it helps repair relationships
Get the report to see what leading LLMs tell employees when they take conflict to it.
