Coaching matters

Bad news armchair sports fans: Your favorite team that’s struggling through a bad season would not be better off playing without a coach, no matter how loudly you yell that opinion at the television.

A new research paper from two University of Chicago public policy professors, “How Much Do Coaches Matter,” finds that coaches “explain about 20-30 percent of the variation in a team’s success.” The belief that all sports coaches are created equal (and interchangeable) “is unwarranted,” the researchers write.

Christopher Berry and Anthony Fowler’s paper was one of eight finalists chosen for the MIT Sloan Sports Analytics Conference, held March 1 and 2 in Boston. 

“In most organizations, coaches are viewed as consumers of sports analytics,” the researchers write. “Relatively seldom, however, have coaches themselves been the subject of the sort of rigorous empirical analyses regularly applied to players.” 

The two men used their Randomization Inference for Leader Effects method for the study. They looked at statistics for hockey, baseball, and professional and college football and basketball. The data included games between teams, as well as home-field advantage, and quality of opponents.

While they aren’t the first researchers to study coaches, Berry and Fowler argue their new methodology factors in player quality and strength of opponents, along with the ability to separate team performance based on coaching versus performance by chance. 

The professors also point out in their paper that few studies look at the impact of individual coaches on a team or player performance, and instead emphasize general coaching success.

Here are some of their findings:

Baseball

When it comes to Major League Baseball, managers have a greater impact on runs allowed than runs scored.

The researchers write that one reason for this difference could be that baseball managers need to make more strategic defensive decisions than they do offensive ones.

“For the most part, the job of the manager on offense is to put the best hitters in the lineup in the best order, and most managers would probably make similar decisions with the same team,” the researchers write. “However, on defense, the manager must efficiently utilize their pitchers without wearing out their arms. Some managers may be better than others at determining when a starter has thrown too many pitches to be effective, when to use a reliever, and which reliever to use in a particular situation.” 

Football

The researchers looked at the NFL from 1922-2016, and found that coaches “significantly affect” a team’s fumbles and penalties. Though some coaches might hope they have an impact on penalties committed by the opposing team, the research found coaches had little effect in that area.

Football coaches also aren’t dramatically that different when it comes to decisions on passing and rushing.

“Perhaps most coaches are following the same rules of thumb and are getting their teams close to the optimal share of passing versus rushing,” Berry and Fowler write.

Basketball

The professors found that for college basketball, coaches had a greater impact on points scored and allowed, than on point margin between teams.

According to the paper, “one potential explanation is that coaches differ from each other in their preferences for fast- versus slow-paced games, with the fast-paced coaches both scoring and allowing more points.”

Bill Belichick

Using available data through 2016, the researchers found that New England Patriots head coach Bill Belichick was 18 percent more likely to win a game (based on the quality of his opponent and home field advantage) than an average coach. While that isn’t the highest percentage of his peers, it is “extremely unusual” given his 17 seasons with the team.

Could Belichick be the luckiest coach in the NFL? The professors ran a simulation based on 10,000 hypothetical NFL games, and in only five cases would a coach with a 17-year tenure have Belichick’s same record.

“In that sense,” Berry and Fowler write, “we can strongly reject the null hypothesis that Belichick is no better than an average coach.”