August 2024
Executive Summary
We highlight practical AI use cases in investing, showing that an “AI Financial Analyst” can effectively perform many financial analyst tasks, such as monitoring executive departures, analyzing patents, and generating research reports. However, while AI excels as a junior analyst, it still lacks the skills required for more senior roles. We discuss how firms can build hybrid teams that leverage the complementary strengths of AI and human talent.
Introduction
Disrupting Wall Street 📠
“While we do not know the full effect or the precise rate at which AI will change our business … the consequences will be extraordinary and possibly as transformational as some of the major technological inventions of the past several hundred years…”
🏦 Jamie Dimon, CEO JPMorgan (Apr 2024)
In Nov 2022, ChatGPT’s launch set off a frenzy of excitement about the transformative potential of AI, sparking a rush of AI-related investments that is expected to reach one trillion dollars. Tech luminaries boldly proclaim this will culminate in AI superintelligence and the end of human labor.
However, with ChatGPT’s two-year anniversary approaching, we are still waiting to enter the promised land. Despite the hype, we have yet to see much in the way of real adoption from the corporate sector, leading skeptics to declare AI a bubble with no real use cases.
In this paper, we aim to offer a more nuanced perspective as practitioners applying AI in finance. Although finance is just one industry, it is a bellwether for AI adoption. Finance jobs are among the most exposed to AI automation, and financial firms are some of the most heavily investing in AI talent.
In our opinion, AI does have powerful use cases in investing, three of which we highlight in this paper. At the same time, its capabilities are often overhyped. While AI is able to effectively perform many junior financial analyst tasks, it lacks the human-level skills required for more senior roles.
This underscores the fact that AI and human workers have very different strengths. AI has superhuman powers in some areas but lags even average human ability in others. We believe clearly understanding the complementary strengths of AI and human talent is the key to thriving in the AI age.
Exhibit 1
AI Exposed Industries
Source: O*NET, LinkUp, OpenAI, Eloundou et al (2023), Sparkline. Universe is the top 1000 U.S. stocks by market capitalization. Based on job postings from 7/1/2022 to 6/30/2023. Reproduced from Investing in AI: Navigating the Hype (Jul 2023).
Unstructured Data Revolution 🗽
So what is the “killer application” of AI in finance? While it may sound mundane, we believe it is the ability to rapidly process large amounts of unstructured data. To explain why this is so important, we’ll summarize our own AI journey.
Five years ago, in Investment Management in the Machine Learning Age (Jun 2019), we first called out the potential of unstructured data for investors. We argued that traditional structured data (e.g. accounting data, P/E ratios) constituted only a sliver of relevant corporate information, with much of its value already exhausted by other quantitative investors.
On the other hand, we contended that unstructured data (e.g., text, images), which makes up over 80% of the world’s data, represented the next frontier for investors. While we showed the utility of some basic text processing techniques (e.g., embeddings, topic modeling), we conceded that we had yet to fully realize the potential of unstructured data.
Exhibit 2
Unstructured Data Examples
Source: Sparkline. Reproduced from Investment Management in the Machine Learning Age (Jun 2019).
One year later, in Deep Learning in Investing: Opportunity in Unstructured Data (Jul 2020), we argued that large language models (LLMs) held the missing key to unlocking the value in unstructured data. Specifically, we showed that pre-trained language models could help with domain-specific financial applications, such as reading earnings call transcripts.
Since then, LLMs have become significantly more powerful and accessible. Today’s state-of-the-art models are over 200 times larger than the BERT model used in our 2020 paper. Even better, companies no longer need to train proprietary LLMs, as they can access pre-trained models from providers like OpenAI, Anthropic, and Meta.
Looking back on the history of technology in finance, the data-driven revolution initially focused on structured data, manipulated via spreadsheets, relational databases, and statistical models. With the advent of LLMs, the revolution can now proceed to its next stage – unlocking insights in vast, untapped troves of unstructured data.
AI Financial Analyst
AI Analyst: Episode Two 🤖
If AI’s superpower is its ability to process unstructured data, it should most impact finance jobs that involve repetitive natural language processing (NLP) tasks, such as extracting data from financial documents, generating reports, and analyzing company filings, news articles, call transcripts, and other documents.
A prime example of such a job is that of the financial analyst. In last year’s paper, Investing in AI: Navigating the Hype (Jul 2023), we trained an “AI Financial Analyst” to help automate the task of reading corporate filings. We gave our analyst access to a large library of documents, which it could then use to answer various free-form questions.
Exhibit 3
AI Analyst Schematic
Source: Sparkline. Reproduced from Investing in AI: Navigating the Hype (Jul 2023).
Our goal was to determine which firms were positioned to benefit from AI. Rather than spend days manually pouring over thousands of dull financial reports, we outsourced the task to our AI companion. In minutes, our AI analyst was able to provide assessments for thousands of stocks, along with clear explanations and citations to source documents.
Of course, identifying AI-linked stocks is just one of many possible applications of our AI analyst. In the remainder of this section, we’ll present three additional case studies to showcase its capabilities.
Case Study 1: Executive Departures 🪂
One of the most basic analyst tasks is to keep tabs on portfolio companies. Analysts must constantly be on the lookout for signs of impending trouble. In this case study, we'll focus on one such red flag, the unexpected departure of a company executive (e.g., CEO, CFO, COO).
Executive departures are concerning for many reasons. At a minimum, they are disruptive. Forced exits may signal cultural rot or other pervasive problems. Even voluntary departures can be a bad sign. Given executives’ access to privileged information, their decision to leave may reflect a troubling lack of confidence in their firms’ futures.
How can we efficiently monitor the comings and goings of management teams across hundreds of companies? In Searching for Superstars (Apr 2021), we used LinkedIn data to track the flow of human capital across firms. However, these profiles are often updated slowly, if at all.
Fortunately, U.S. public companies are required to disclose leadership changes in 8-K filings. These forms must be filed within four business days of certain material events, which include executive departures. Given the wide variety of such events, 8-Ks are among the most common regulatory filings. Over 700,000 8-Ks were filed in the past decade, more than the number of 10-Ks, 10-Qs, and earnings calls combined.
Exhibit 4
Piling Up Filings
Source: S&P, SEC, Sparkline. U.S. public companies only. From 1/1/2014 to 12/31/2023.
However, while 8-Ks are extremely common, their content tends to be quite simple. As the next exhibit shows, the key information is often contained in just a few sentences. This makes 8-Ks a perfect target for AI, which, as we’ll discuss later, excels at tasks requiring high throughput but not advanced reasoning.
Exhibit 5
8-K Example
Source: SEC, Sparkline. As of 5/15/2024.
As detailed in Investing in AI: Navigating the Hype (Jul 2023), for each stock, we utilize a Retrieval-Augmented Generation (RAG) pipeline to select the most relevant 8-K passages to answer the question: “Did the company experience the sudden or unplanned departure of any senior executives (e.g., CEO, CFO, VP)?” We then instruct our analyst to answer this question using only the selected passages. The next exhibit shows two examples of our AI analyst’s responses.
Exhibit 6
AI Sample Responses
Boeing (12/1/2019 to 12/31/2019) Yes. The company experienced the sudden departure of Dennis A. Muilenburg, who resigned as President and Chief Executive Officer and as a director on December 22, 2019. This change was reported alongside the election of David L. Calhoun as the new President and Chief Executive Officer effective January 13, 2020, and Gregory D. Smith was appointed as interim Chief Executive Officer until Mr. Calhoun's appointment takes effect. [8K 5.02, 12/23/19, Excerpt 1] and [8K 5.02, 12/23/19, Excerpt 2]. |
Beyond Meat (5/1/2021 to 5/28/2021) Yes. The company experienced the sudden or unplanned departures of two senior executives. 1. Mark J. Nelson, the Chief Financial Officer and Treasurer, announced his retirement effective May 5, 2021. [8-K 5.02, 5/6/21 Excerpt 1]. 2. Additionally, Charles Muth, the Chief Growth Officer, notified the company of his plans to retire effective June 4, 2021, which indicates another executive departure. [8-K 5.02, 5/26/21 Excerpt 1]. |
Source: SEC, OpenAI, Sparkline. Not a recommendation to buy or sell.
Our analyst successfully identifies the dismissal of Boeing’s CEO in Dec 2019 and the departures of two Beyond Meat executives in May 2021. Since these dates, the two firms’ stocks have fallen -44 and -95%, respectively. Beyond Meat has particularly struggled with its executive bench, firing its COO in Oct 2022 after he was arrested for allegedly “biting a man’s nose!”
Exhibit 7
Boeing and Beyond Meat
Source: S&P, SEC, OpenAI, Sparkline. Percentages are stock price declines as of the respective events. Not a recommendation to buy or sell. As of 6/30/2024.
Of course, these examples were chosen with the benefit of hindsight. While we expect executive departures to generally signal problems and negatively impact stock returns, this is not always the case. For example, if a CEO is the reason for a firm’s poor performance, firing him may clear the way to rebuild. Let’s see which effect dominates on average.
Each month, we build a portfolio of stocks from a universe of the largest 1,000 U.S. stocks that experienced unplanned executive departures in the prior month. In a typical month, the portfolio consists of 30 to 110 stocks, with sector composition similar to that of the broader market index.
The next exhibit shows examples of stocks held during 2024, along with the titles of each firm’s departing executives. Some executives left for opportunities at other firms while others were forced out due to poor performance, personal indiscretions, or as part of broader leadership shakeups.
Exhibit 8
Sample Portfolio Holdings (2024)
Source: S&P, SEC, OpenAI, Sparkline. For illustrative purposes. Not a recommendation to buy or sell. See important disclosure below. From 12/31/2023 to 6/30/2024.
Finally, let’s backtest the returns of the strategy. The next exhibit shows the stock returns of firms that experienced unplanned executive departures relative to the market.
Executive departures tend to be followed by stock underperformance, with the negative impact persisting for several months. One potential explanation is that the market is slow to digest information in unstructured data. Alternatively, investors may process but undervalue this information. This is consistent with Intangible Value (Jun 2021), which found that investors often undervalue intangible assets, such as human capital.
Exhibit 9
Executive Departure Backtest
Source: S&P, SEC, OpenAI, Sparkline. Executive departure stocks are those that experienced an unplanned departure of a senior executive over the previous month. The line shows the returns of these stocks relative to those of the investment universe, which consists of the top 1,000 U.S. stocks by market capitalization. The portfolios are rebalanced monthly and exclude transaction and financing costs. See important backtest disclosure below. From 12/31/2003 to 6/30/2024.
The beauty of our AI analyst is its versatility. While this case study focused on executive departures, it can help answer many other questions. For example, we could have it search for mentions of accounting changes, material litigation, supply chain disruptions, or rising order backlogs. LLMs give us a powerful new tool to systematically explore a wide range of novel research topics.
Case Study 2: Cybersecurity Patents 🛡️
For financial analysts, the ability to stay on top of regulatory filings, earnings calls, and news has become table stakes. Like many other investors, we believe the next frontier lies in so-called “alternative data” – non-traditional data ranging from job postings to social media. Alternative data is often large and unstructured, making it fertile ground for AI.
We’ll illustrate the value of AI-processed alternative data with a case study on cybersecurity, a theme that has recently dominated headlines. On Jul 19, CrowdStrike, a cybersecurity vendor, released a faulty update that crashed over 8 million corporate systems, with Delta Air Lines alone claiming a $500 million loss from the outage. Then, just a few days later, the cybersecurity startup Wiz made a splash by walking away from a historic $23 billion acquisition offer.
Let’s say we believe that cybersecurity will continue to grow in importance and want to identify companies investing in this critical area. Unfortunately, regulatory filings are not very helpful, as they mostly contain boilerplate risk disclosures. Moreover, managers rarely bring cybersecurity up proactively on calls, only discussing it after an incident has already occurred.
Instead, we can turn to alternative data, specifically patents. In Investing in Innovation (Apr 2022), we showed that patents could help identify firms investing in innovative technologies, such as AI, robotics, and virtual reality. In this section, we’ll take a similar approach but focus on firms patenting in the field of cybersecurity.
Patent data presents two challenges for manual analysis. First, it is extremely voluminous. Since 1790, the U.S. Patent and Trademark Office (USPTO) has granted over 11 million patents. Moreover, activity has increased over time, with over 3 million patents granted in the past decade alone.
Exhibit 10
Increasing Patent Activity
Source: USPTO, Sparkline. Originally from Investing in Innovation (Apr 2022). Updated as of 12/31/2023.
Second, patents are often filled with technical jargon. For example, the next exhibit contains the abstract of a 130-page cybersecurity patent granted earlier this year to Nvidia.
Exhibit 11
Cybersecurity Patent Example (Nvidia)
Title: [US11876782B2] Header-based packet filtering and inferencing to identify malicious network traffic using neural networks |
Abstract: In various examples, a first network interface duplicates received network traffic and forwards a first set of network traffic data to a central processing unit (CPU) and a second set of identical network traffic to one or more parallel processing units (PPUs). In an embodiment, the one or more PPUs analyze the second set of network traffic to identify whether the second set of network traffic is malicious. First, the one or more PPUs filter and classify the second set of network traffic into flows, or logical groupings or subsets of the second set of network traffic. Second, the one or more PPUs sort the network packets within each flow and extract features of interest specific to each flow. Using the extracted features of interest, one or more deep learning techniques infer a status indicating whether each flow is malicious (mal) or good. The one or more PPUs then forward the status for each flow to the CPU for use in determining which network traffic from the first set of network traffic is to be forwarded to a second network interface. |
Source: USPTO, Sparkline. As of 1/16/2024.
Fortunately, LLMs are trained on not only financial text but also a wide array of general and scientific documents. Thus, our AI has no trouble classifying this as a cybersecurity patent, calling it a “clear application in the field of cybersecurity aimed at monitoring and filtering harmful traffic.”
Next, we’ll provide our analyst with access to the full patent corpus. For each company, we ask our AI analyst: “Is the firm patenting in the field of cybersecurity?” The next exhibit shows an example of our analyst’s response for Nasdaq, based on patents granted over the past year.
Exhibit 12
AI Sample Response
Yes. The company is patenting in the field of cybersecurity, as evidenced by several excerpts that focus on data security and protection measures. 1. The first excerpt titled "Systems, methods, and computer-readable media for data security" discusses a server system designed to monitor requests from computing devices and determine if their activities indicate suspicious behavior, suggesting a focus on security monitoring. 2. Additionally, the excerpt "Systems and methods of secure provenance for distributed transaction databases" describes a system that provides electronic resource tracking and storage using blockchain, which is relevant to cybersecurity as it relates to securing digital transactions and maintaining the integrity of data. 3. The excerpt "Systems and methods for in-session refresh of entitlements associated with web applications" implies security by enabling real-time updates to a user's access privileges during an active session, which is crucial for maintaining secure access to applications. 4. Lastly, "Systems and methods for securing and disseminating time-sensitive information using a blockchain" outlines a system for securely releasing documents via blockchain, which inherently includes cybersecurity components. These excerpts collectively illustrate the company's engagement with cybersecurity through various innovative approaches to ensure data security, integrity, and access management. |
Source: USPTO, OpenAI, Sparkline. From 6/30/2023 to 6/30/2024.
Our AI analyst correctly determines that Nasdaq was granted multiple patents related to cybersecurity. As an interesting aside, the firm appears to be actively exploring blockchain applications for the purpose of cybersecurity.
In total, our AI is able to identify 138 U.S. stocks that received cybersecurity patents over the past year. Of these, 68 are technology firms, of which most are cybersecurity vendors, such as Check Point Software, Palo Alto Networks, and Fortinet, or tech giants, like Nvidia, Alphabet, and Apple.
Exhibit 13
Cybersecurity Stocks by Industry
Source: S&P, USPTO, OpenAI, Sparkline. As of 6/30/2024.
More interestingly, over half of these firms actually hail from non-tech sectors, such as finance, media, and capital goods. Since 2010, this non-tech share has grown from 29 to 51% as cybersecurity has become increasingly “mission critical” for all industries. The next exhibit shows examples of non-tech companies with cybersecurity patents, along with the titles of representative patents.
Exhibit 14
Non-Tech Cybersecurity Patent Holders
Source: S&P, USPTO, OpenAI, Sparkline. Not a recommendation to buy or sell. For illustrative purposes. See important disclosure below. From 6/30/2023 to 6/30/2024.
As expected, patents held by non-tech firms tend to relate to their industry. For example, Johnson Controls has a cybersecurity patent for building management, Medtronic for implantable medical devices, GE for industrial assets, and Morgan Stanley for detecting forged signatures.
Next, we’ll construct a backtest of stocks that have received cybersecurity patents over a rolling one-year window. We track the returns of tech and non-tech stocks separately, reporting performance relative to their respective industries.
Exhibit 15
Cybersecurity Stock Backtest
Source: S&P, USPTO, OpenAI, Sparkline. Strategy consists of stocks that the AI analyst identifies as having received cybersecurity patents over a rolling 1-year period. The blue line compares the returns of these stocks within tech relative to the broader tech sector. The red line does the same but for non-tech stocks. The portfolios are rebalanced monthly and exclude transaction and financing costs. See important backtest disclosure below. From 12/31/2009 to 6/30/2024.
Since 2010, companies receiving cybersecurity patents have outperformed their peers. Importantly, this success has not only been achieved by tech firms like CrowdStrike or Nvidia; non-tech companies with cybersecurity patents have also outperformed other industrial businesses.
To be clear, these are large public companies whose value is unlikely to be driven by just a handful of patents. Rather, we believe the value is more in the signaling. Firms investing in cybersecurity R&D tend to be more forward-thinking in their approach toward both cybersecurity and tech more broadly.
In Digitizing the Old Economy (Mar 2023), we used tech job postings to find “old economy” firms embracing digitization. Patents offer another way to determine which firms are not just paying lip service but truly investing in a digital future.
Case Study 3: Stock Research Reports 🪄
Financial analysts are responsible not only for gathering and synthesizing information but also presenting it to their bosses. We’ve already shown that AI can help with the first two steps. It can also help with the third.
The previous two examples showcased LLMs’ ability to convert large, messy unstructured data into more tractable structured data. While the AI also provided text-based responses to support its conclusions, these were only used for validation. Instead, the final output driving the backtests was a single boolean value (i.e., True or False).
Numerical outputs are especially useful for quant investors (like us) seeking to precisely define testable hypotheses, such as “Do executive departures tend to lead to poor future stock returns?” We outlined this quant use case in Text-Based Factor Investing (May 2021), proposing the addition of novel “text-based factors” to traditional quant factor libraries.
However, for discretionary investors, the text responses may themselves be the desired output. Rather than have the AI “compress” the supplied information all the way down to a single number, we can have it stop at this intermediate step. These text responses can then serve as fodder for further analysis by human investors.
Exhibit 16
Information Compression
Source: Sparkline.
In this case study, we’ll have our AI analyst generate a simple stock report. Intended as a “proof of concept,” the report will have five sections. We’ll provide our AI with a separate prompt and set of source documents for each section. For concision, we limit each section to less than 200 words.
Exhibit 17
Stock Report Template
Section | Prompt | Source Docs |
|---|---|---|
Business Model | Describe the firm’s business model in less than 200 words. | 10-K, Calls, Analyst Reports |
Recent Developments | Summarize key developments impacting business performance in the year ending in 6/30/2024 in less than 200 words. | 10-K, 10-Q, 8-K, Calls, News, Analyst Reports |
Financial Metrics | Create a table with the firm’s revenue and earnings over the past three fiscal years. Break out major segments separately. Group rows by year then segment and columns by metric. | 10-K, 10-Q, Calls, Analyst Reports |
Bullish Analyst Opinions | Summarize the arguments in favor of owning this stock in under 200 words. | Analyst Reports |
Bearish Analyst Opinions | Summarize the arguments against owning this stock in under 200 words. | Analyst Reports |
Source: Sparkline.
Let’s see what our AI analyst is able to spin up. The next exhibit shows an AI-generated stock report for Disney based on the template above.
Exhibit 18
Disney Example
Disney Stock Summary |
|---|
1. Business Model The company operates a diversified business model across three main segments: Entertainment, Sports, and Experiences. The Entertainment segment includes content production and licensing, generating revenue through box office sales, home entertainment, and television markets [10-K Item 1, 11/21/23]. The Sports segment encompasses global television and direct-to-consumer (DTC) video streaming, primarily through ESPN, generating revenue from affiliate fees, advertising, and subscription fees [10-K Item 1, 11/21/23]. The Experiences segment includes theme parks, hotels, and cruises, providing physical experiences with the company's intellectual property (IP) [Earnings Call, 8/9/23]. The company is transitioning from linear TV to streaming, focusing on DTC services to build stronger connections with users and gather more data for monetization [Analyst 2, 8/18/23]. This shift involves significant investment in technology and content to build scale and reduce churn [Earnings Call, 5/7/24]. The company also leverages its content production and distribution capabilities to enhance its theme parks and other experiences [Earnings Call, 8/9/23]. |
2. Recent Developments In the year ending 6/30/2024, several key developments impacted Disney's business performance. The company faced challenges in its entertainment segment due to lower theatrical results and production delays from strikes [Earnings Call, 2/7/24]. Despite these setbacks, Disney made significant progress in cost efficiency initiatives, achieving $7.5 billion in annualized savings and targeting over $8 billion in free cash flow for the fiscal year [Earnings Call, 5/7/24]. The Experiences segment saw robust growth, driven by strong performance at international parks and Disney Cruise Line, with plans to invest $60 billion over the next decade to expand capacity [Earnings Call, 2/7/24]. Additionally, Disney's restructuring efforts, including a reorganization of its media and entertainment businesses, led to improved operating results in streaming and a more unified approach to marketing, pricing, and programming [Earnings Call, 11/8/23]. |
3. Financial Metrics Revenue and Earnings by Segment (in millions) [10-K Item 7, 11/21/23]  |
4. Bullish Analyst Opinions Bullish opinions on Disney's stock highlight several key points. The company's parks are described as cash-generating machines, and its legacy media still produces significant inflows, allowing for investment in direct-to-consumer (DTC) initiatives [Analyst 2, 8/18/23]. One analyst even argues that Disney's Parks division, ESPN, and traditional media operations justify the company's current enterprise value, with streaming initiatives seen as a "free call option" [Analyst 3, 2/7/24]. Disney’s extensive content library, strong international position, and advertising technology are seen as underappreciated assets. Analysts also believe the company has yet to fully realize the potential of its streaming efforts [Analyst 3, 2/22/24]. Disney's management is actively addressing past errors and implementing corrective measures, with the return of Bob Iger and his focus on improving margins and providing a profitability timeline for streaming [Analyst 4, 8/15/23]. Given recent stock price declines, Disney's stock is arguably undervalued with significant long-term upside potential [Analyst 3, 2/7/24]. |
5. Bearish Analyst Opinions Bearish opinions regarding Disney's stock highlight several key concerns. Firstly, the company's streaming business has been under scrutiny, with investors remaining unconvinced about its profitability, with every major streaming service, except for Netflix, bleeding money [Analyst 3, 2/7/24]. Additionally, Disney's legacy media operations are in decline due to cord-cutting and the shift to streaming, which has not been fully offset by the growth in their direct-to-consumer (DTC) segment [Analyst 2, 8/18/23]. Moreover, Disney's studio arm is facing issues such as franchise fatigue and superhero overload, which could impact its content pipeline and profitability [Analyst 3, 2/7/24]. Lastly, the competitive landscape in streaming is intensifying, and Disney's late entry into the market has necessitated significant investment, which may strain financial resources [Analyst 2, 8/18/23]. These factors collectively paint a cautious picture for Disney's stock, emphasizing the challenges the company faces in achieving sustainable profitability and growth. |
Source: SEC, S&P, The Science of Hitting [Analyst 1], Games Companies Play [Analyst 2], Value Punks [Analyst 3], David Waldron [Analyst 4], OpenAI, Sparkline. For illustrative purposes. Not a recommendation to buy or sell. Reflects the opinions of third parties and subject to change. See important disclosure below. As of 6/30/2024.
The first section describes Disney’s three main segments, primarily drawing on the 10-K business section. The second mainly summarizes recent earnings calls. The third builds a data table, cobbling together values from disparate parts of the most recent 10-K MD&A section. The final two sections summarize the bull and bear cases for the stock based on the opinions of investment analysts.
We intentionally limited this example to a simple “proof of concept.” However, if desired, it could be made more robust in several ways. First, we could remove the 200-word limit to elicit more detailed responses. Second, we could include more sections, such as on competitors, insider ownership, or M&A. Third, we could give our analyst access to alternative data (e.g., Nielsen) and data visualizations (e.g., bar charts in analyst reports).
Fourth, we could expand our analyst research library. In this example, we restricted ourselves to publicly available research. However, in practice, investors may have access to additional research from sell-side brokers, independent analysts, industry experts, or internal firm employees. Investors could use research from either all analysts or a curated subset that they believe have the most alpha.
Finally, we could further “train” our analyst. If we had examples of reports that we particularly liked, we could instruct our AI to generate reports similar to these examples using techniques like “few-shot prompting” or “fine-tuning.” This would allow us to further tailor the AI’s work product to our specific preferences.
Obviously, AI-generated stock reports are no substitute for actual human due diligence. That said, they provide a useful starting point, rapidly pulling together information scattered across dozens of disparate documents. Given AI’s speed and scale advantage, these reports could be particularly useful for conducting initial screens of a broad stock universe and staying up-to-date as the investment landscape evolves.
The Silicon Ceiling
High School Reasoning 🧑🎓
In the previous section, we found that our AI analyst is able to effectively perform a diverse set of analyst tasks including monitoring executive departures, analyzing patents, and generating research reports.
While impressive, it’s important to note that none of these tasks required particularly advanced reasoning abilities. In fact, we would argue they are roughly equivalent in difficulty to the reading comprehension questions on the SAT college admissions exam, where high school students are given a passage and asked to draw simple inferences from that text.
Limiting the level of required reasoning to that of a smart high school student ensures that we do not push the model beyond its capabilities. For example, let’s say we instead asked a more challenging, open-ended question: “Do you expect the stock price to increase?” A sample affirmative response for Lululemon as of Dec 31, 2023 is below.
Exhibit 19
Lululemon Stock Prediction (Dec 31, 2023) ❌
Yes. Several signs suggest that the stock price is expected to increase: 1. Strong Financial Performance: The company has shown strong financial performance across different quarters. In Q2 2023, it exceeded expectations, with operating income of $479 million or 21.7% of net revenue compared to adjusted operating margin of 20.9% of net revenue in Q2 2022. The EPS also grew significantly to $2.68 per diluted share compared to $2.20 a year ago [Earnings Call 8/31/23, Excerpt 9]. 2. Guidance Indicates Continued Growth: The company raised its full-year 2023 revenue expectations to be in the range of $9.51 billion to $9.57 billion, representing growth of 17% to 18% relative to 2022. Adjusted EPS is forecasted to be between $12.34 and $12.42 for the full year 2023, a significant increase from $10.07 in 2022 [Earnings Call 12/7/23, Excerpt 12]. 3. Strategic Investments and Expansions: The company continues to invest in strategic initiatives and growth pillars, such as new store openings and digital infrastructure, which sets the stage for future growth. It plans to open approximately 55 net new company-operated stores in 2023 [Earnings Call 8/31/23, Excerpt 11]. 4. Share Repurchase Program: The company has an active share repurchase program, with $454 million remaining of $1 billion as of the end of Q2 2023. Share repurchase programs generally indicate that the company believes its stock is undervalued and is a good investment [Earnings Call 8/31/23, Excerpt 10]. 5. Decrease in Costs: For the full year 2023, the company expects a decrease in air freight costs, contributing to higher gross margins. Air freight expenses are expected to be down approximately 220 basis points versus 2022 [Earnings Call 12/7/23, Excerpt 12]. These consistent indicators of growth and strategic planning suggest a positive outlook for the company's stock price. |
Source: S&P, SEC, OpenAI, Sparkline. Not a recommendation to buy or sell. Subject to change. See important disclosure below. As of 12/31/2023.
While this prediction appears reasonable and well-organized, it was dead wrong. Over the next two quarters, Lululemon’s stock collapsed -42%. Although the five factors cited were accurately grounded in the source material, the analysis was too naive, merely extrapolating a strong 2023 into the future.
Unfortunately, the problem is not specific to Lululemon. The next exhibit shows the returns relative to the market of a strategy that systematically follows our AI analyst’s trade recommendations. The return stream is just random noise.
Exhibit 20
Nothing but Noise
Source: S&P, SEC, OpenAI, Sparkline. Strategy buys stocks expected to increase by the AI analyst. The line shows the returns of these stocks relative to those of the investment universe, which consists of the top 1,000 U.S. stocks by market cap. The portfolios are rebalanced monthly and exclude transaction and financing costs. See important backtest disclosure below. From 12/31/1997 to 6/30/2024.
Why isn’t this working? First, our AI analyst struggles with the initial information retrieval step. Given such an open-ended question, it has a hard time gathering all the necessary pieces of information, which vary across stocks and are scattered across many disparate texts. We likely need a more sophisticated retrieval approach than the embedding-based search used in our RAG system.
Second, even if given all the necessary information, security analysis requires high-level, domain-specific reasoning. One test of such ability is the Chartered Financial Analyst (CFA) exam, a challenging three-part test with pass rates ranging from 38 to 44%. The next exhibit summarizes the findings of Callanan et al (Oct 2023), who gave LLMs the first two levels of the exam (the third level would require human graders).
Exhibit 21
GPT Takes the CFA Exam
Source: Callanan et al (Oct 2023), Sparkline. Values show the average results across three prompting techniques (i.e., zero-shot, chain-of-thought, and multi-shot).
While the CFA Institute does not disclose minimum passing scores, the authors estimate that passing Level 1 (Level 2) requires a score of at least 60% (50%) in each topic and an overall score of 70% (60%). Based on these criteria, GPT-3.5 would fail both Levels 1 and 2, while GPT-4 would pass Level 1 but narrowly fail Level 2.
Furthermore, even if LLMs were able to consistently pass the CFA exam, excelling in an idealized test-taking situation does not necessarily translate into real-world success. Investing rarely has clean answers, as markets are fraught with ambiguity. Navigating the ever-shifting investing metagame requires not only analytical ability but also novel insight.
Small Data Problem 🐣
In the prior section, we used an off-the-shelf version of GPT-4. Could we achieve better results by fine-tuning our LLM on stock market data? In theory, we could provide the model with a corpus of documents along with subsequent stock returns, and it could learn the statistical relationships between the two “variables.”
In practice, we are skeptical about attempts to train LLMs to directly predict long-term stock returns. We first articulated this view several years ago in Deep Learning in Investing: Opportunity in Unstructured Data (Jul 2020), when many quant investors were pursuing this path. We argued that such an approach would be gated by inherent limitations in financial market training data (e.g. stock returns).
As the next exhibit shows, financial market data has three major limitations: it is small, noisy, and non-stationary.
Exhibit 22
Small Data Problem
Source: Sparkline.
First, financial market datasets used by long-term investors are relatively small. LLMs gain their power from massive amounts of training data. For example, Llama 3 was trained on a corpus of 15 trillion tokens. By comparison, fundamental stock investors have access to perhaps less than 100,000 distinct training observations (i.e., 3,000 stocks x 30 years).
Second, financial markets are extremely noisy. Even the best investors are unable to forecast stock returns with hit rates much better than 50%. This compares to 90%+ accuracy for LLMs on standard natural language and computer vision benchmarks. Noise makes it more difficult to train machine learning models, effectively diluting the informational content of each data point.
Exhibit 23
Financial Markets are Noisy
Source: Sparkline, Papers With Code. Reproduced from Deep Learning in Investing: Opportunity in Unstructured Data (Jul 2020).
Third, and perhaps most importantly, financial markets are non-stationary. Market structure is constantly evolving, and predictive features are “arbitraged” away as they become widely known. Models trained on historical market data may fail to generalize as markets change. In contrast, the natural language data used to train LLMs is very stable, with new slang or technical terms entering the lexicon infrequently.
At least for long-term investors, we believe the “brute force” approach of training AI on reams of historical stock returns is mostly a dead end. Instead, the path forward requires the more difficult task of improving AI’s ability to reason from first principles. As we saw in the prior section, current AI models are not there yet but are rapidly improving.
Analyst vs. Portfolio Manager 🤝
One helpful way to contextualize the current capabilities of AI is using the “analyst vs. portfolio manager” framework. In this highly stylized model, investment teams are made of two types of employees: analysts and portfolio managers.
Exhibit 24
Stylized Investment Team
Source: Sparkline.
In this framework, the job of a (junior) analyst is to gather, synthesize, and present information. This involves reading news, press releases, and third-party research, incorporating this information into financial models, and presenting the results to the portfolio manager. Not only is AI already adept at many of these more basic tasks, but it can also perform them with much greater speed and stamina.
At the other extreme, portfolio managers are responsible for reviewing the work of analysts and making high-level portfolio allocation decisions. They also direct analysts to conduct research on areas of elevated opportunity (e.g., trending themes, special situations). As we saw, AI still lacks the accuracy, judgment, and creativity required for this role.
Exhibit 25
AI-Powered Investment Team
Source: Sparkline.
Of course, AI will continue to improve over time. As we saw, GPT-4 did better than GPT-3.5 on the CFA exam. However, absent a huge breakthrough, we expect our analyst’s career trajectory to be incremental rather than discontinuous. It first needs to earn its promotion from first- to second-year analyst before moving up to portfolio manager!
The Hybrid Workforce
Reimagining the Investment Firm 🤔
Let’s build on this stylized model of an investment firm composed of two roles: analysts and portfolio managers.
Using the approach from Investing in AI: Navigating the Hype (Jul 2023), we first unbundle jobs into their underlying tasks and then assess the “AI exposure” of each task. We’ll use the profiles for “Investment Fund Manager” and “Financial and Investment Analyst” from the O*NET database.
The next exhibit shows the 29 analyst or portfolio manager tasks that collectively comprise the day-to-day activities of our stylized investment firm. In the final column, we denote tasks that GPT-4 determines can be completed more efficiently by a LLM (i.e., 50% faster with equivalent quality).
Exhibit 26
Reimagining the Investment Firm
Task | Job of | Can AI Do? |
Prepare reports on economic trends, corporations, and industries. | Analyst | 🤖 |
Perform securities valuation or pricing. | Analyst | 🤖 |
Monitor developments by analyzing financial publications. | Analyst | 🤖 |
Monitor developments in technology, business, finance, etc. | Analyst | 🤖 |
Interpret data on price, yield, investment-risk trends, etc. | Analyst | 🤖 |
Inform investment decisions by analyzing financial information. | Analyst | 🤖 |
Draw charts and graphs, using computer spreadsheets. | Analyst | 🤖 |
Create client presentations of plan details. | Analyst | 🤖 |
Present product risks, fees, or fund performance statistics. | PM | 🤖 |
Monitor financial or operational performance of investments. | PM | 🤖 |
Consult financial media to stay abreast of relevant markets. | PM | 🤖 |
Develop, implement, or monitor security valuation policies. | PM | 🤖 |
Evaluate the relative quality of securities in a given industry. | Analyst | 😊 |
Employ financial models to develop solutions to financial problems. | Analyst | 😊 |
Prepare plans of action for investment, using financial analyses. | Analyst | 😊 |
Recommend investments to investment firm staff. | Analyst | 😊 |
Develop and maintain client relationships. | Analyst | 😊 |
Supervise, train, or mentor junior team members. | Analyst | 😊 |
Perform or evaluate research to inform decision making. | PM | 😊 |
Ensure conformance with strategic goals or regulatory requirements. | PM | 😊 |
Evaluate the potential of new products. | PM | 😊 |
Identify group or individual target investors for a specific fund. | PM | 😊 |
Develop or implement fund investment policies or strategies. | PM | 😊 |
Select specific investments for purchase by an investment fund. | PM | 😊 |
Manage investment funds to maximize return on client investments. | PM | 😊 |
Select or direct the execution of trades. | PM | 😊 |
Meet with investors to determine goals or to discuss investments. | PM | 😊 |
Hire or evaluate staff. | PM | 😊 |
Source: O*NET, OpenAI, Sparkline. PM stands for portfolio manager. O*NET defines the analyst role broadly, so we remove tasks more typically associated with corporate finance jobs. We also remove compliance and operations tasks. As of 8/14/2024.
As expected, AI is more likely to encroach on analyst jobs. Around 57% of analyst tasks can be performed by AI, while the share is only 29% for portfolio managers. AI-advantaged tasks include creating presentations, drawing charts, and monitoring news, fund performance, and valuations. On the other hand, humans are still needed for tasks requiring interpersonal skills, security analysis, or strategic thinking.
Let’s imagine a future in which we have a third employee, an AI analyst, to whom we give tasks it can do more efficiently. While this leaves fewer tasks for human employees, this isn’t necessarily bad. In response, humans’ task mix will shift toward areas of our comparative advantage. At least in the case of investing, this shift is likely positive, as it offloads a lot of “grunt work,” freeing us to focus on higher-value tasks.
In addition, this assumes a static set of work. In reality, there will be two dynamic effects. First, AI will lead to the rise of new tasks and jobs. Autor et al (2022) found that most jobs held by people today did not even exist in 1940 and arose to complement new inventions (e.g., airplane pilots, web designers). Assuming AI is not fully autonomous, the ability to “manage” AI may itself become a valuable human skill (or even a full-time job).
Second, technology-driven productivity gains may impact overall labor demand. As illustration, let’s go back in the annals of finance history. In the 1970s, when the automated teller machine (ATM) was first introduced, many predicted that bank tellers would be automated away.
Exhibit 27
Bank Teller Employment and ATMs
Source: Bessen (2015), Bureau of Labor Statistics, Sparkline. As of May 2023.
Instead, the number of bank tellers actually increased over the next three decades. ATMs allowed banks to reduce the number of tellers per branch, lowering the cost of each branch. In response, banks opened many more branches, increasing net demand for tellers.
Moreover, bank tellers’ task mix also changed, arguably for the better. While ATMs did automate the task of dispensing cash, with this time-consuming task out of the way, tellers were free to spend more time talking to customers. Over time, the job morphed into a client service role, emphasizing the more “human” part of the job (Bessen (2015)).
However, this was only the first chapter of the story. Later, as banking moved online, the number of tellers did finally start shrinking. This case study neatly shows the contrasting labor market impact of two technologies – the ATM and internet. If history is any guide, AI is likely to have similarly complex and often counterintuitive effects on the workforce.
Complementary Strengths ☯️
Ultimately, AI’s entry into the finance workforce is inevitable. Moreover, as with prior technological innovations, such as the calculator, spreadsheet, and internet, AI will create winners and losers. Firms and employees able to take advantage of the new technology will thrive, while those unable to adapt will gradually fade into obsolescence.
In A Human View of Disruption (Feb 2021), we wrote: “Our overarching belief is that humans should seek to complement machines rather than compete with them. We should specialize in areas in which we have comparative advantage over machines. This includes interpersonal communication, creativity, problem solving, empathy, and cross-domain integration.”
In order to more rigorously demonstrate the above assertion regarding the “human comparative advantage,” let’s return to the O*NET database, which also contains a taxonomy of abilities, defined as “enduring attributes of the individual that influence performance.” As before, we ask GPT-4 to determine if humans or LLMs are stronger on each ability. Its response for the “written expression” ability is below:
Exhibit 28
Ability Assessment Example
Written Expression: The ability to communicate information and ideas in writing so others will understand. |
Tie. Both humans and AI have strengths in written expression, but in different contexts. Humans excel in creativity, emotional nuance, and cultural sensitivity, which are crucial for certain types of writing such as literature, persuasive essays, and nuanced communication. On the other hand, AI, particularly large language models (LLMs) like GPT-4, can generate coherent, grammatically correct, and contextually relevant text quickly and efficiently, making them strong in tasks that require speed, consistency, and the ability to process large amounts of information. Therefore, the effectiveness of written expression depends on the specific requirements of the task at hand. |
Source: O*NET, OpenAI, Sparkline. As of 8/14/2024.
On “written expression,” GPT-4 declares a tie between LLMs and humans. It notes that the two sides have different strengths, with humans excelling in creativity, emotional nuance, and cultural sensitivity and AI stronger on tasks requiring speed, consistency, and efficiency.
The next exhibit lists the winner for each of the 21 abilities in the “cognitive abilities” branch of the O*NET taxonomy (note: we ignore the psychomotor, physical, and sensory branches, where humans uniformly dominate LLMs).
Exhibit 29
O*NET Abilities: AI vs. Human
Source: O*NET, OpenAI, Sparkline. As of 8/14/2024.
Overall, humans beat LLMs with an 11-7-3 record. Humans possess stronger oral, reasoning, interpersonal, and spatial abilities; we are also more creative, nuanced, and flexible in our thinking. On the other hand, LLMs have stronger pattern recognition and fact recall; they also excel at tasks that require speed, precision, and the ability to process large data sets.
Importantly, the strengths of humans and AIs appear to be quite complementary. This makes us optimistic about the prospect of harmoniously integrating AI into the workforce. While AI may render some employees obsolete, we believe its more enduring impact will be to reshape and augment the human workforce in a mostly positive way.
Conclusion
Since ChatGPT’s release in Nov 2022, a huge amount of hype has developed around AI. This has led to mounting fears that AI will take all our jobs, while also sparking a backlash by skeptics who argue AI is a bubble with no real use cases.
This paper aimed to present a balanced and practical view on the debate. We provided three concrete AI use cases in investing to prove that AI is not pure hype. However, we also explored the limitations of LLMs. Ultimately, we concluded that AI is able to effectively perform many junior analyst tasks but still lacks the skills required for more senior roles.
This led us to articulate a framework of AI’s comparative strengths and weaknesses. If nothing else, we hope this helps you, our readers, develop a more nuanced view of AI’s capabilities and limitations. In our opinion, understanding clearly the complementary strengths of AI and human talent will be crucial for successfully navigating the age of AI.
Disclaimer
This paper is solely for informational purposes and is not an offer or solicitation for the purchase or sale of any security, nor is it to be construed as legal or tax advice. References to securities and strategies are for illustrative purposes only and do not constitute buy or sell recommendations. The information in this report should not be used as the basis for any investment decisions.
We make no representation or warranty as to the accuracy or completeness of the information contained in this report, including third-party data sources. This paper may contain forward-looking statements or projections based on our current beliefs and information believed to be reasonable at the time. However, such statements necessarily involve risk and uncertainty and should not be used as the basis for investment decisions. The views expressed are as of the publication date and subject to change at any time.
Backtest Disclosure
The performance shown reflects the simulated model performance an investor may have obtained had it invested in the manner shown but does not represent performance that any investor actually attained. This performance is not representative of any actual investment strategy or product and is provided solely for informational purposes.
Hypothetical performance has many significant limitations and may not reflect the impact of material economic and market factors if funds were actually managed in the manner shown. Actual performance may differ substantially from simulated model performance. Simulated performance may be prepared with the benefit of hindsight and changes in methodology may have a material impact on the simulated returns presented.
The simulated model performance is adjusted to reflect the reinvestment of dividends and other income. Simulations that include estimated transaction costs assume the payment of the historical bid-ask spread and $0.01 in commissions. Simulated fees, expenses, and transaction costs do not represent actual costs paid.
Index returns are shown for informational purposes only and/or as a basis of comparison. Indexes are unmanaged and do not reflect management or trading fees. One cannot invest directly in an index.
No representation or warranty is made as to the reasonableness of the methodology used or that all methodologies used in achieving the returns have been stated or fully considered. There can be no assurance that such hypothetical performance is achievable in the future. Past performance is no guarantee of future results.
Comentários