After yesterday’s loss to FC Midtjylland, Louis van Gaal is again under pressure at Manchester United. Earlier this season, I wrote that he was doing fairly well given the players at his disposal, and he certainly didn’t have his first-choice team in Denmark. But after some embarrassing losses, it’s natural that fans – and maybe even executives at the club – are calling for the sack. In truth, it’s far from an easy decision.

The market for managers is full of informational asymmetries. In plain English, clubs can’t always tell how skilled a manager is or how good the match is likely to be. In the midst of this uncertainty, I think it helps to give a little structure to our thinking. Here are some common-sense starting points based on what I’ve seen in the data and the news:

1. Managers do seem to matter. There are some managers whose teams consistently over- or underachieve in a way that seems unlikely to be the result of noise. However, since we have limited seasons of data with which to evaluate this claim, we can’t be 100% certain. The residuals from the effect of wage bills on performance may have other causes.

2. A manager’s effect on a team has a mean and variance. Managers don’t add or subtract the same amount from a team’s performance all the time. Everyone has their good and bad patches; everyone focuses at some times and is distracted at others. Different conditions (owners, players, employment situations, league positions) will also bring out different aspects in managers. Each manager’s style may fit or clash with the club’s objectives, too. For example, a manager can help the club to create variance (Villas-Boas) or stability (Pulis) in results. And a manager’s effect can change over time, for example if players start to tune him out (Mourinho).

3. The decision to fire a manager relies on at least three assessments. (a) First, the club needs to form an opinion, which may still have some uncertainty attached, of its manager’s effect on performance. (b) Then the club needs to gauge how that effect (variable as it may be) fits the club’s objectives. (c) Finally, the club has to consider whether the alternatives to the current manager are better.

The time it takes to do 3a depends on a lot of factors. Brentford figured out pretty quickly this season that they had hired the wrong guy. Liverpool took three seasons to decide Rodgers wasn’t their man after all. But Rodgers’s effect on performance may have changed, and so might the conditions at the club.

3b depends a lot on the club’s ownership and whether it has a firm idea of the objective it’s trying to achieve. This can change, too, e.g., “He’ll keep us from being relegated this season, which is what we need, but for next season we want someone who can move us up the table.” Of course, this kind of thinking can create complications; players may not work as hard for a manager whose time is limited.

For me, clubs pay surprisingly little attention to 3c. Perhaps the relationship with the manager becomes so unworkable that a club has to let him go, even if no one has agreed to take the job. But interim coaches rarely get the best out of the team, in part for the reason above, so a club in need of points would want to have someone lined up to take the reins immediately. This isn’t always straightforward, though, since jockeying for a job that still has an incumbent may be taboo in the managerial community.

My own opinion is that it may take several seasons of watching a manager with different clubs to see whether he consistently over- or under-performs. Given the noise, we’re essentially asking how many times in a sequence a coin has to come up heads before we start thinking it’s not a fair coin: 4 out of 4, 4 out of 5, etc. With the risks inherent in football, a manager who underperforms in 3 out of 3 seasons might be worth avoiding, even if there’s still a chance he’s a good manager.

Indeed, because of the uncertainty inherent in the market, clubs are bound to make mistakes. But once a manager is in place, I think a club should have some idea of how well he fits after a full season and two transfer windows in charge – or at least an idea of whether it’s worth giving him a second season. The question is whether taking a new risk with a new manager offers better value in expectation.

One final comment: The labor market for managers is not unitary in the sense of all buyers interacting with all sellers; in fact, there are lots of small markets that may overlap to differing degrees. A lot of hires run through nepotism, connections, and agents. Because few clubs use objective criteria to evaluate managers, many can be swayed by factors that may be unrelated to effects on performance. So there is room for arbitrage by hiring good managers who are undervalued in the market. An important part of this can be to use analytics as a first cut across a broader pool of potential hires than what clubs might generally consider.