(Alternate title: “Do football analytics work?”)

This is a question that I like to ask – I don’t find it insulting, dismissive, or silly. To answer it, first we have to define analytics, and then we can talk about whether it might be any good. So let’s start with the basics.

There’s a hierarchy in the way we use numbers in soccer, or really any field:

• We can count things: “Joe Fullback made 32 tackles last season.”
• Or we can do some simple arithmetic: “Joe Fullback made 1.5 tackles per game last season.”
• And we can try to add some context: “Joe Fullback made 3.2 tackles per minute his team was out of possession last season.”

These uses of numbers are all descriptive and fairly easy to compute. They’re also just facts – to argue about them, you’d have to argue about the definition of a tackle or time out of possession. We could also say that any of them still qualifies as “analytics”, since they all involve a basic level of analysis of data. But clearly, none of this analytics means anything on its own. It only has a function if you find a way to add meaning.

What kind of meaning? Well, the numbers above aren’t trying to say anything about the process that generates Joe’s tackling opportunities, his likelihood of winning tackles, his future success at tackles, where he fits into the overall pool of fullbacks as a tackler, or his value to his club while on the pitch. To measure any of these, we’d need to go beyond facts and into the realms of modeling and inference; we’d have to use statistics.

This is where analytics can get into trouble. Statistics is a science of probability, and probability is by its nature inexact. Faced with a single question – say, “Will Joe Fullback make 40 tackles next season?” – no statistician can guarantee that her answer will be correct. Even if her model truly predicts tackles and nothing else, there’s always a chance that she’ll be wrong. And if it turns out that her model doesn’t actually predict tackles, then she has to go back to square one.

Being wrong in soccer can be costly, especially if you’re trying to answer the big questions: “How many points would Joe Fullback add to my club’s total in the league next season?” “How much should I pay for him in the transfer window?” “What should the incentives in his contract look like?” “How many minutes is he likely to play?” In response to these questions, analytics would optimally yield precise answers with very specific interpretations. But analytics answers always require a margin for error – and again, that’s assuming they’re answering the right questions.

So the analytics answer can’t be right every time. Of course, the human answer isn’t usually right every time, either. Soccer is filled with very smart and confident people who are sometimes wrong. The trick is to find analytics that’s wrong in a different way.

It’s this difference that creates value. If our analytics always says the same thing as the scouts, coaches, and video analysts, then it has no value at all. Naturally, there can be some uncomfortable moments when analytics suggests an unpopular opinion. But these potential conflicts offer an opportunity for everyone to explain, debate, and evaluate their ideas – and then come to a consensus that’s much more thoroughly tested than any of their own opinions.

Holding the raw materials equal, a club with a regular and routine process for creating this consensus will have an edge in making decisions. The word “regular” implies that the process goes through the same steps every time, so the consensus is always equally strong. The word “routine” is equally important; if you use a process some days and not others, it’s not really a process at all – and this is true whether analytics are part of it or not. Indeed, a club where scouts, coaches, and video analysts sit down once a week for an hour to talk about players will probably make better decisions than a club with a skilled data analyst but no process.

These days we’re still hearing the occasional, “The analytics is wrong, so analytics must be a crock.” It’s not an improvement to say, “The human is wrong, so humans must be a crock.” Rather, we have to acknowledge the strengths and weaknesses of both perspectives, then combine them in the most powerful way possible.