JP Morgan COIN: A Bank’s Side Project Spells Disruption for the Legal Industry
JPMorgan has begun to automate the work of its law firms using machine learning. It'll save time and money, but the bank shouldn't write off human analysis just yet.
Ask a young corporate lawyer the most painful part of his job and you’ll probably hear “doc review.” Document review is the process of lawyers poring over thousands of documents to determine which are relevant for litigation. Rote and time-consuming, this work is mind-numbing for attorneys and expensive for clients. Largely because of this process, McKinsey reports that nearly a quarter of a lawyer’s job can be automated. [1] As a result, many law firms are looking to automate the document review process (top firms already outsource it).
But incentives are misaligned. Law firms generally bill by the hour, and clients already squeeze them to reduce hours charged. One academic study concluded that just adopting existing machine learning could reduce lawyers’ billable hours by about thirteen percent. [2] Lawyers would make less money in the immediate future as a consequence. A select few clients have instead begun to productize some of the work of their attorneys. JPMorgan has recently emerged as a leader in this trend.
Last year, JPMorgan announced it had developed and deployed new software called COIN—shorthand for Contract Intelligence—that automates document review for a certain class of contracts. The company first dispatched the program to review thousands of its own credit contracts. The software employs image recognition to identify patterns in these agreements. [3] While JPMorgan has been tight-lipped about the details of the proprietary technology, the bank has stated that the algorithm uses unsupervised learning: by digesting data on the bank’s numerous contracts, it can identify and categorize repeated clauses [4]. The bank reports that the algorithm classifies clauses into one of about one hundred and fifty different “attributes” of credit contracts. [5] For example, it may note certain patterns based on clause wording or location in the agreement.
The software reviews in seconds the number of contracts that previously took lawyers over 360,000 man-hours. JPMorgan’s economic incentive to develop the product is thus self-evident. But what’s more: the algorithm is more accurate than human lawyers. [6] So the bank’s investment in the technology is not just about costs, but also about quality since COIN improves the accuracy of the contract review process.
While automated “technology-assisted legal review” solutions are not new, JPMorgan benefits from the large scale and low variability it has in credit contracts. The bank processes over 12,000 credit agreements per year, which are far less complex than contracts that might better suit human review, such as custom M&A agreements. [7]
And the bank appears well suited to expand its use of machine learning in legal work. JPMorgan is already exploring other opportunities to tackle attorney costs. [8] In the short term, it intends to deploy COIN for more complex filings, such as credit-default swaps and custody agreements. [9] In the medium and long term, the bank also hopes to use machine learning to interpret altogether new regulations (questions of “first impression,” as lawyers often call them). [10] The idea is to move beyond data classification to data interpretation.
JPMorgan is solving an important problem for its own business while encouraging a more efficient legal industry. But it should be clear about the type of legal work a machine can and cannot automate—and prioritize its software development accordingly. First, although the use of machine learning for legal work has recently won favor in federal courts, judges and lawyers continue to create a human monopoly on much of their work by requiring that a licensed professional do it. [11] Until algorithms start earning JDs, much of attorneys’ work will be safe. Second, new legal questions do not always fit the mold of past legal doctrine; as technology and business change, so do judicial and regulatory decisions. Third, the law is dynamic: often jurists borrow from one area of the law to confront a new challenge in another. A machine might know everything about contracts law but it takes lateral thinking and reasoning by analogy to predict that a judge is about to borrow a tort law doctrine. In short, it would be a risk to permit a machine to anticipate new rules.
A better principle is to focus on the litigation and corporate work that is readily automatable—pattern identification and classification—and to delay software development on the more complicated tasks—analysis, interpretation and argumentation. But this author’s aware that a few questions might arise from this recommendation: is this proposed distinction between machine and human work collapsing as machine learning improves? And will companies look to displace law firms or their own in-house counsel with these new technologies?
(759)
[1] McKinsey & Company. (2018). Harnessing automation for a future that works. [online] Available at: https://www.mckinsey.com/featured-insights/digital-disruption/harnessing-automation-for-a-future-that-works [Accessed 14 Nov. 2018].
[2] Remus, D. and Levy, F. (2015). Can Robots Be Lawyers? Computers, Lawyers, and the Practice of Law. SSRN Electronic Journal.
[3] Megatrends by HP. (2018). Bank on AI to shape the future of finance – Megatrends by HP. [online] Available at: https://hpmegatrends.com/bank-on-ai-to-shape-the-future-of-finance-93520267e63a [Accessed 14 Nov. 2018].
[4] J. P. Morgan 2016 Annual Report. [online] Available at: https://www.jpmorganchase.com/corporate/investor-relations/document/2016-annualreport.pdf [Accessed 14 Nov. 2018].
[5] Bestpractice.ai. (2018). JPMorgan reduced lawyers’ hours by 36… – AI Case Study JPMorgan. [online] Available at: https://www.bestpractice.ai/studies/jpmorgan_reduced_lawyers_hours_by_360_000_annually_by_automating_loan_agreement_analysis_with_machine_learning_software_coin [Accessed 14 Nov. 2018].
[6] J. P. Morgan 2016 Annual Report. [online] Available at: https://www.jpmorganchase.com/corporate/investor-relations/document/2016-annualreport.pdf [Accessed 14 Nov. 2018].
[7]J. P. Morgan (@jpmorgan) “Showcasing Innovation: A quick look at how $JPM is using machine learning.” Feb 28, 2017, 1:41pm. Tweet. Available at: https://twitter.com/jpmorgan/status/836692784652435460
[8] J. P. Morgan 2016 Annual Report. [online] Available at: https://www.jpmorganchase.com/corporate/investor-relations/document/2016-annualreport.pdf [Accessed 14 Nov. 2018].
[9] Sennaar, Kumba. “AI in Banking – An Analysis of America’s 7 Top Banks.” TechEmergence, 29 Oct. 2018, www.techemergence.com/ai-in-banking-analysis/.
[10] Son, Hugh. “This Software Does in Seconds What Took Lawyers 360,000 Hours.” The Independent, Independent Digital News and Media, 28 Feb. 2017, www.independent.co.uk/news/business/news/jp-morgan-software-lawyers-coin-contract-intelligence-parsing-financial-deals-seconds-legal-working-a7603256.html.
[11] Michael Simon, Alvin F. Lindsay, Loly Sosa & Paige Comparato. Lola v. Skadden and the Automation of the Legal Profession. 20 Yale J.L. & Tech. 234 (2018).
I agree with your analysis of the benefits and risks, and I believe that this type of change is inevitable in the legal community. Historically, in any industry client needs have been the largest driver of innovation. Although there are indeed misaligned incentives as you mentioned above, the first few law firms that adopt this technology and are able to provide better basic services at a lower cost will enjoy enhanced competitive positioning. In the future, I believe that differentiation in the legal practice will rely more heavily on quality of thought and creativity in structuring or precedents, rather than ability to process documentation.
It is interesting that JPMorgan has created this technology, as it seems to be outside of their core competency. I would be interested to see if they license this technology to other firms and providers, or spin off these assets and sell to an investor.
This is a very interesting topic because it shows how machine learning is putting legal practices, one of the most highly-skilled and specialized industries traditionally, at risk of workforce replacement (i.e. human lawyers to machine algorithms). To answer the author’s question, I believe that the work that can be done by machines will keep increasing as the technology develops. Also, another intriguing point is that, as Jane Doe commented above, it was JP Morgan, not a law firm, that developed this algorithm. Given how law firms have been justifying the hefty legal fees because they are the only entities that are licensed to do the legal work, JP Morgan is apparently posing huge threats to law firm by signaling that it can develop algorithms that are smarter than human lawyers, and that it can potentially receive a legal practice license to have the algorithm compete against other traditional law firms. If the JP Morgan algorithm becomes superior than human lawyers in wider legal areas, I suggest that the traditional law firms buy or subscribe to this algorithm as soon as possible, so that they can protect themselves against the new powerful market entrants (a.k.a. machines).
Initially, I would think that machine learning will change the way human work entirely. Due to its sophisticated algorithm, machine learning seems to become more human. The way JP Morgan uses it is very smart. The document for credit evaluation is more repetitive and not so over complicated.
However, you raised a good point about types that machine learning should not touch. For example, in the sensitive area such as legal that could judge someone life or death, how should we consider replacing lawyers or judges with the machine? Or in a more closer to our life, would I put my child in a self-driving car alone for a short trip to another city? If there is anything happen, whom to blame?
I very much enjoyed reading this post. Intuitively, it makes sense that opportunities exist in the legal profession for machine learning to replace human lawyers, particularly in areas such as credit card contracts with a high degree of standardization and less judgment required. The benefits accruing to JP Morgan for utilizing this technology clearly exist, but I wonder if machine learning also provides an opportunity for law firms as well. Given their sophistication and expertise, presumably one of the larger law firms could partner with a technology company specialized in machine learning to develop their own in-house tool for assisting with contract review and other less sophisticated tasks. This would free up lawyers to spend more time on higher value services for clients.
Looking ahead, I do wonder to what extent the gap between human and machine work can be closed. I see a parallel to medicine, in that machines may be able to assist with the identification or diagnosis of issues as well as pattern recognition, but I am not convinced that the human element can entirely be removed. Also, as the author mentions, there are accreditation standards and other mechanisms in place that effectively serve as barriers to fully eliminating the human element. However, I would not be surprised to see more companies follow JP Morgan’s path, or for a technology company to develop and license a similar solution for use by larger corporations.
Great article and very interesting topic as well. A few years ago I worked with a legal firm that had automated systems to track files both in the justice and in government agencies. Through machine learning and advanced analytics it could take automated action and provide real time insight to lawyers to improve the efficiency of their procedures. This firm had a huge competitive advantage in some high-volume/low-cost procedures such as dealing with early retirement programs.
Regarding Mo’s point, I completely see what you say. However, I think that this use of machine learning is super low-risk and a great complement to highly-qualified attorney work. If developments like this are incremental and safe, sooner than later our environment will be really different.
Excerpt from the New York TOMs review:
Mr. Legal’s discussion of COIN’s prospects in the legal space highlight the importance of distinguishing what can be automated, such as pattern recognition and classification tasks, from what cannot, such as analysis and interpretation. In fact, one of the Bay Area’s fastest-growing startups, Atrium, makes use of precisely this distinction and maintains its own team of in-house lawyers who augment their work with the company’s proprietary machine learning tools. These tools allow its lawyers to replace many hours of costly searches on WestLaw and LexisNexis, passing the time savings and cost benefits on to clients. An interesting follow-up would be to address whether COIN and startups such as Atrium will ultimately become competitors, substitutes, or a mix of the two.
“A Better Law Firm for Startups.” Atrium, 15 Nov. 2018, https://www.atrium.co/.
Well written article and I enjoyed reading about the applications within the legal profession. I agree with the author that technology assistance document reviews can prove to be quite helpful, particularly for certain use cases. For instance, this method can be quite useful when reviewing standardized, easy to understand contracts such as credit contracts. However, as the author points out, there are limitations to the use of this technology today; would an executive truly let the review of a merger document be handled by a machine as opposed to a trusted lawyer?
Despite my hesitations with the broad application of this technology, I believe there are tremendous suicidal benefits particularly for lawyers. It is commonly known that the legal profession is particularly cumbersome, with paralegals and lawyers having to go through many hundreds of pages of documents in great detail. Technology assisted document reviews can tremendously reduce the amount of time time lawyers spend on low-value added work, focusing on high value tasks.