Winning the Super Data Bowl: Eagles Fly on 4th-Downs
For years, data analysts have decried NFL coaches for punting on 4th-down. After winning their first Super Bowl, the Philadelphia Eagles may have finally busted the myth that punting on 4th-down is the safe, conservative choice.
In Super Bowl LII, the Eagles shocked the NFL when they successfully converted two pivotal 4th-down conversions. And while Philadelphia finally won its first Super Bowl, the Eagles’ win may be remembered more for having finally busted one of the most frustrating myths in the NFL: punting on 4th-down is the safe, conservative choice.
Background on 4th-downs
In American football, teams have four chances (or “downs”) to either score points or advance the football ten yards down the field. If a team fails on its 4th-down, then the opposing teams takes possession of the ball.
Because coaches don’t want to set up their opponents for an easy score, NFL wisdom says a team should intentionally turn the football over on 4th-down by kicking (“punting”) it down the field. Similarly, if a team is in range of kicking a 3-point field goal on 4th-down, NFL wisdom says they should kick the field goal, rather than go for a 1st-down and risk losing possession.
For years, data analysts have decried NFL coaches for punting or kicking field goals on 4th-down. [4] Half of all NFL plays gain four or more yards, so “conversion is more likely than not.” [5]
Source: NY Times
The Eagles create value by embracing 4th-down conversion analytics
On the surface, the Eagles attempted more 4th-down conversions (29) than any other NFL team in the 2017 football season, and successfully converted 69% (20). [6] However, a closer look at the Eagles’ on-field operations shows that data analytics have become deeply woven into game prep and real-time decisions.
Even before Super Bowl LII, sports commentators acknowledged the Eagles’ analytics edge on the field. In January 2018, ESPN wrote how the Eagles’ approach to analytics is “so involved in the operation that two members of the department . . . communicate with Pederson in-game.” [7] Once a 3rd-down call is made, but before the 3rd-down ball is snapped, the data analytics team will run a model based on inputs like field position, number of yards needed for conversion, and game time remaining. Usually, before the third down snap, they’ll deliver a concise message to Coach Pederson: “If it’s anything less than 4th-and-N, the charts say go for it.” [8]
The Eagles also use data to implement real-time adaptive defense
The Eagles aren’t only using data on offense. Defensive coordinator Jim Schwartz has invented ways to share his love of analytics with the Eagles defense. “Each week there’s a PowerPoint presentation on how many possible plays an offense can run based on time, number of timeouts, and a kicker’s maximum field goal length,” and Schwartz gives briefings that “are detailed down to the yard, personnel, formation, and more.” [9] Schwartz frequently gives defensive player statistical pop quizzes, which inculcates situational propensities that can be quickly recalled on the field. Many Eagles credit Schwartz’s heat maps (color-coded graphics showing play tendencies) and relentless quizzing for the defense’s ability to narrow down the options available to the Atlanta Falcons on their final attempt of the game, which led the Eagles to their first playoff win. [10]
But the NFL is a “copycat league”
NFL coaches are constantly searching for a competitive edge, which is why the NFL is notorious for being a “copycat league”. Thus any edge the Eagles have from analytics is likely not defensible beyond 1-2 seasons. But embracing data requires buy-in from both team ownership and the head coach, which could delay the adoption of analytics by other NFL teams. [11]
So will 4th-down conversions become more frequent? Probably, so long as data keeps indicating that punting is the real “risky” move…
________________________________________________
[1] https://www.youtube.com/watch?v=Ye6dVwDb_u0 (at 11:07).
[2] https://www.youtube.com/watch?v=jZE1kufsrDo (at 2:16:52).
[3] https://www.youtube.com/watch?v=Ye6dVwDb_u0 (at 11:14). “You know, Mike, I hate to say this: I would feel better at this point taking the 3 points. If you make it, it’s great. But if you don’t make it, it’s a big momentum swing.” Id.
[4] See, e.g., David Romer, “Do Firms Maximize? Evidence from Professional Football,” Journal of Political Economy, vol. 114, no. 2 (2006), available at https://eml.berkeley.edu/~dromer/papers/JPE_April06.pdf.
[5] NY Times, “4th Down: When to Go for It and Why,” Sept. 4, 2014, https://www.nytimes.com/2014/09/05/upshot/4th-down-when-to-go-for-it-and-why.html (noting that as “a rule of thumb,” all teams should go for a 4th-down conversion on any 4th-and-1, so long as the team is beyond their own 9-yard-line).
[6] Compiled from http://www.espn.com/nfl/statistics/team/_/stat/downs, accessed April 3, 2018.
[7] Tim McManus, “Eagles’ secret weapon? An analytics-fueled attack,” ESPN.com, Jan. 18, 2018, http://www.espn.com/blog/nflnation/post/_/id/266982/eagles-secret-weapon-an-analytics-fueled-attack, accessed April 3, 2018 (note this was written before the two famous 4th-down attempts in Super Bowl LII).
[8] Id. (noting that “Philadelphia has gone for it on fourth down more than any other team in the NFL since Pederson became head coach last season”).
[9] Jeff McLane, “Jim Schwartz has his Eagles defense prepared for every possibility against Vikings,” Philly.com, January 18, 2018, http://www.philly.com/philly/sports/eagles/nfc-championship-philadelphia-eagles-minnesota-vikings-jim-schwartz-defense-coach-20180118.html.
[10] Id. “Based on the formation and the situation there were only one or two plays they would probably run. It’s one of their favorite concepts, [Eagles defensive safety Rodney] McLeod said. “But those are the things that [Schwartz] points out consistently week to week that give us an edge.” Id.
[11] Tim McManus, “Eagles’ secret weapon? An analytics-fueled attack,” ESPN.com, Jan. 18, 2018, http://www.espn.com/blog/nflnation/post/_/id/266982/eagles-secret-weapon-an-analytics-fueled-attack, accessed April 3, 2018 (“Eagles owner Jeffrey Lurie has long had an analytics team in place; the desire for an optimal fourth-down strategy is not new. The difference is that the Eagles now have a more willing dance partner on the sideline in Pederson.”)
Very interesting post, Hans. Looking at some of these sports analytics case studies, two of the big differentiators between the teams that are using analytics well (vs. just burning money on analysts) seem to be 1) clearly, effectively communicating insights to the coaching staff/ players (who often are not deep experts), and 2) delivering actionable insights in real time. Seems like the Eagles are nailing both of these, as well as creating a culture of data-based decision making/ statistical familiarity among the players. It would be interesting to know how often the coaches follow the algorithmic guidance vs. when they override it based on feel/ intuition (e.g., a sense that the QB is lagging) and what the outcomes are in each scenario; also interesting would be to know whether they have tried to codify some of that feel/ intuition and incorporate it into the algorithms.
Thanks, Hans. Watching the NFL for years, this has always really frustrated me. Coaches also make other analytically-unfriendly choices such as not going for two-point conversions often enough and clock management issues. However, I think a big issue is that going against the grain in the NFL is very tough because of the high turnover of coaches in the league. If a coach makes a “safe,” traditional decision, it’s unlikely to be bashed by the media or resulted in being fired. Most coaches in the league are too vulnerable to want to take the risk of an unconventional coaching call – only a couple of coaches have built up the goodwill to give them the courage to make these analytically correct decisions.
Thanks for posting about this, Hans! It’s an interesting point you bring up about the NFL being a copycat league, and is very reminiscent of the TSG Hoffenheim case we had. In some ways though, I think you could argue that the Eagles can always stay 1-2 years ahead of everyone else. When I worked with a couple of sports teams on a player care / development tech consulting project in 2013, the Eagles were generally viewed as leaders in player fitness development and training because of their use of technologies like on-body sensors, on-field motion cameras, etc. It seems that this embracing of analytics is not just in how they manage games, but also how they manage athlete development and down-time between games as well. I guess another interesting question here would be how this translates to “economic performance” if you think of the team as a business. To that end, I think you can make the argument that winning a championship is highly accretive to franchise value and income for the coming years, which made these types of investments highly worthwhile.
Really interesting, Hans, thanks for posting! I especially liked reading this after our moneyball European football case, (and I appreciated your explanations of the US version of the game so I could follow your post). I wonder if other teams will start to follow what the Eagles have done with data now that it has proven to be such a successful way to win a title. If other teams do start to copy, I wonder then how long this lead will last for the Eagles, as other teams might start seeing the same opportunities and doing the same things to try to win, converting 4th downs, etc. and the ‘edge’ that the Eagles have found now might not last. They will need to keep investing in data in order to be able to see gains over time.
Great post, Hans. This is a very interesting example of the data clearly going against the “traditional way” of doing things. I wonder how much of this boils down to basic human psychology… The concept of loss aversion in decision theory tells us that people tend to prefer avoiding losses to acquiring equivalent gains (most people prefer to not lost $5 than to find $5). In the business vocabulary, managers are generally more worried about protecting their downside than seeking an equivalent upside. That being the case, I think data-driven leaders in the NFL will have make an especially powerful and convincing case to convince others to stop punting so much.
Thanks for a great post, Hans. Great example of how data can empower decision-makers to make uncomfortable calls. Reminds me of OU’s long time / recently retired head coach Bob Stoops, who during his heyday was notorious for going for it on 4th. Many called him reckless – to Zach’s comment. Others admired his bravery. My friends would common do one or the other from game to game – depending on the outcome! Would have loved to have examined his choices in light of the NYT bot’s findings.
Thanks for this, Hans. Super insightful. Surprising that more sports teams don’t use data for in-game decisions. My thought here is that there are so many outside factors in a game (momentum, how the team is playing in the moment, injuries, etc.) that pure data plays are not necessarily as useful as in other scenarios. I wonder if there is a way to put these outside variables into the data set to make the decisions and probabilities even more informed? Without these non-tangible factors, it seems that some of the power of the “pure” data might be stunted.
Interesting topic, Hans, and congrats on the Super Bowl. It’s interesting that it’s taken this long for teams to decide to go for it more on 4th down (not to mention 2 point conversions). There is a high school coach that has been doing this since 2008 and has a bit of a cult following. According to MIT, “in the 2008 State Championship season, Kelley never punted and has done so just four times in the last three seasons. According to maxpreps.com, Kelley’s Bruins lead the nation in offense two of the past three seasons.” He was the USA Today Coach of the Year in 2016 and will be speaking at the MIT Sloan Sports Analytics Conference next year.
http://www.sloansportsconference.com/people/kevin-kelley/
With regards to the question on why teams forego the data and continue to punt, Malcolm Galdwell had an interesting podcast when he explored why more bad free throw shooters in the NBA don’t take granny shots (statistically it is a better free throw form). The answer came down to the threshold framework. Low threshold teams like the Eagles are less influenced by the crowd and are thus first to adopt technology that goes against conventional wisdom. The higher threshold teams will follow suit and the chances they do increases with each new adopter. I do think the league will eventually converge to the Eagles way, but the pattern of when teams do should tell us a lot about their organization philosophy.