Home » Managers » How good is your manager?

How good is your manager?

What makes a good manager? The answer depends in part on his job description. Some managers are in charge of recruitment, while others must leave it to a director of football or a transfer committee. Most managers take charge of tactics on the field, but a few give their coaches minute-to-minute control. All managers, however, try to make the whole greater than the sum of the parts, bringing players together in a lineup that works. And that’s what I’ve tried to measure with these new ratings.

As any student of the game knows, a club’s results can depend on both skill and something else that’s hard to influence or measure, often called luck. I wanted to distinguish luck from skill, and I also wanted to distinguish players’ skill from managers’ skill. Other analysts including Colin Trainor, Paul Riley, and Sarah Rudd (whose work is no longer available to the public) have looked at this question in the past, but without using player ratings as their point of comparison. For this experimental exercise, I decided to look at all the clubs in the last five English Premier League seasons.

I started by dividing skill into two parts: 1) putting a squad of stars on the field, and 2) getting the best performance possible out of them. Once I accounted for both types of skill, any unexpected improvement or erosion in results would be down to luck. My metric for the first kind of skill was the aggregate of one of my player ratings designed to pick up innate ability, which has been highly correlated for players from year to year. (We should never expect perfect correlations, since a player’s ability will rise and fall as he ages.) Naturally, this metric would depend in part on a club’s budget as well as a manager’s ability to identify good players, but I didn’t want to ignore the latter.

For the second kind of skill, I subtracted the measure of ability from the club’s expected goal difference for the season, after first standardizing both metrics. I used expected goal difference instead of actual goal difference because I wanted to eliminate luck in shooting and saving. It’s true that I might have discarded some forms of shooting and saving skill, too, but an adjustment that I developed recently appears to have limited these effects.

The gap between a club’s finish in the table and its rank for expected goal difference was my metric for luck. It’s worth noting that this gap implies two kinds of luck: the kind that helps to determine whether a given shot will score, and the kind that helps to determine the timing of each goal. A club that concedes a surprisingly small number of goals may be lucky, just like a club that always gets goals when it needs them. (On the latter point, I’ve tried to find evidence of a persistent impact from managers’ substitutions, and I haven’t come up with anything robust.)

So which managers were lucky, and which seemed to show skill? The overall distribution of club seasons shows that teams of all different levels of ability had managers with different levels of luck and skill at the helm:

coaches

The lack of teams in the bottom-left and top-right corners of the graph is just a result of how the league table works. There aren’t many teams that can have a manager who’s both very good and very lucky, since the highest anyone can finish is first. By the same token, it’s tough to have a manager who’s both very bad and very unlucky, since no team can finish below last place. The slight concentration of above-average teams on the left and below-average teams on the right is probably related, since teams expected to finish near the top of the table can’t outperform expectations by much.

Now let’s look at some specific clubs and managers. We’ll start with current champions Chelsea:

coacheschelsea

Chelsea didn’t necessarily have the league’s best squad this season (hence the blue marker), but José Mourinho got an unexpectedly great season out of his team without the benefit of any luck. He did almost as well in the previous season with a squad rated higher by my model, though that side was a little bit unlucky. Carlo Ancelotti turned in a similarly outstanding managerial performance in 2010-11, and the 2012-13 season shared by Roberto di Matteo and Rafael Benítez was also above average, with a touch of luck on its side. By contrast, André Villas-Boas likely did a below average job in 2011-12, when he was the manager until the beginning of March.

For comparison, let’s look at a squad that had the same manager for all five seasons – in fact, the only one that did, Arsenal:

coacheswenger

Arsène Wenger hasn’t been a lucky manager, but he did get his players to perform either close to par or better in all five seasons, with 2014-15 being his best of the five. He’s also managed to put a fantastic squad on the field for every season except 2013-14.

Next we can follow a single manager across several clubs. Tony Pulis had three seasons at Stoke in my sample, then most of a season at Crystal Palace and half a season at West Bromwich Albion. My guess is those last two clubs will wish they had gotten hold of him earlier:

coachespulis

Pulis consistently gets more out of his players than mere ability would predict, though he was a bit unlucky not to finish higher up the table at Stoke. He finally got some luck at Palace, but the overall pattern is clear: the more Pulis, the better. Sam Allardyce’s three seasons at West Ham draw a comparable picture.

That’s not the case with Mauricio Pochettino and Roberto Martínez. Both managers have led two clubs in the league, and they have always underperformed:

coachesmartinezpochettinoIt looks like luck – somehow Harry Kane finished 17% of his chances instead of the expected 9% – saved Spurs from finishing the past season close to the relegation zone. Only Swansea under Garry Monk and Newcastle under Alan Pardew in 2011-12 have been anywhere near as lucky.

Yet managers taking over new clubs, like Pochettino and Monk, often face a challenging first season as they impose their ideas and personnel. Moreover, though Pochettino’s squad at Tottenham has a ton of ability, his players may still be too young to gel into a cohesive unit. As a side note, Spurs defended surprisingly well on both corners and set pieces in 2014-15, which may suggest tactical innovation rather than luck.

Fortunately for Southampton, Ronald Koeman has turned things around as far as managerial performance. He’s got a positive rating for skill, though he was also unlucky last season. That other Dutchman, Louis van Gaal, didn’t do as well; a spot of good luck canceled out his stars’ failure to realize their full potential on the field.

Finally, let’s admire the personification of consistency, David Moyes. He showed measurable skill at both Everton and Manchester United, though he wasn’t particularly lucky with either. He may not have been the best manager in the league, but he was far from the worst:

coachesmoyes

Martínez may even have benefited from vestiges of Moyes’s system in his first season at Everton, as dismantling it doesn’t seem to have done the Toffees any favors. Moyes did a perfectly fine job with a less-than-exceptional squad at Manchester United, too. Of course, he also helped to assemble that squad using a big dose of new-manager cash. A manager’s job is multifaceted, and only a few do all of it consistently well.