Large Language Models, Small Labor Market Effects

New research reveals AI chatbots have limited effects on workers' earnings and hours

AI is expected to generate substantial productivity gains in business. It can create presentation slides, summarize thousand-page PDF files, and write compelling sales emails in seconds. However, humans still need to review, correct, and sometimes redo the work if the AI output is not good enough. In my experience, I often spend as much time reviewing and testing AI output as I did completing the task myself. So the key question becomes: what is the real productivity improvement once we account for human oversight—measured in workers’ earnings, hours, or wages?

Paper reviewed:

Humlum, Anders and Vestergaard, Emilie, Large Language Models, Small Labor Market Effects (April 15, 2025). University of Chicago, Becker Friedman Institute for Economics Working Paper No. 2025-56, Available at SSRN: https://ssrn.com/abstract=5219933 or http://dx.doi.org/10.2139/ssrn.5219933

Summary

A recent study found that despite 43% of Danish workers being encouraged to use AI chatbots, the technology has had minimal impact on labor market outcomes, with workers reporting average time savings of 2.8%. The research suggests businesses should focus on complementary investments to unlock AI's potential.

Key Findings

Implications

Business and Policy Implications

Introduction

The emergence of AI chatbots marks a significant development in the field of Generative Artificial Intelligence (AI). Recent studies have shown that AI chatbots have seen rapid worker take-up, with some randomized controlled trials (RCTs) demonstrating substantial productivity gains for users. However, the broader labor market implications of Generative AI remain unclear. This paper addresses these gaps by examining the labor market effects of AI chatbots using large-scale adoption surveys and linked employer-employee data in Denmark.

Background and Context

Denmark provides an ideal setting for examining the labor market impacts of Generative AI due to its flexible labor market, high AI chatbot adoption rates, and exceptional infrastructure for tracking new technology adoption. The Danish labor market is characterized by low hiring and firing costs and decentralized wage bargaining, allowing firms and workers to adjust hours and earnings in response to technological change. The country's infrastructure, including digital mailboxes, enables the distribution of survey invitations and the linking of survey responses to administrative labor market data.

Data and Institutional Setting

The paper builds on two large-scale surveys on AI chatbot adoption conducted in November-December 2023 and 2024. The surveys cover 11 occupations exposed to AI chatbots and are linked to matched employer-employee data from Statistics Denmark. The dataset includes information on workers' experiences with AI chatbots, employer initiatives, and labor market outcomes such as earnings, hours, and occupations.

The surveys focus on 11 occupations selected based on their exposure to AI chatbots, as measured by the "Direct Exposure (E1)" metric from Eloundou et al. (2024). The occupations include accountants, customer support specialists, financial advisors, HR professionals, IT support specialists, journalists, legal professionals, marketing professionals, office clerks, software developers, and teachers.

The 2024 survey is organized into four blocks: occupation and tasks, adoption, employer initiatives, and impact on work. The survey asks workers about their experiences with AI chatbots, including frequency and duration of usage, and their perceived benefits and challenges.

The paper links the survey data to administrative registers at Statistics Denmark, including the E-Income Register, which records earnings, hours, occupation, and industry for all job spells in Denmark. The linked data provides a unique opportunity to analyze the labor market effects of AI chatbots.

The survey sample is representative of the population, with checks on representativeness and response quality conducted in Appendix A.2.1. The survey respondents resemble the survey population on observable characteristics, and the response rates are comparable to previous Danish surveys.

The paper proceeds to examine the prevalence of employer initiatives for AI chatbot adoption, worker adoption patterns, and the benefits and challenges reported by users. The analysis sheds light on the mechanisms through which Generative AI could become transformative over time, including firm-driven investments and workplace reorganizations.

The findings have implications for businesses, policymakers, and managers, highlighting the importance of complementary investments and organizational changes to unlock the potential of AI chatbots. The paper's results also caution against extrapolating productivity gains from controlled experiments to the broader economy, given the modest time savings reported by workers and the weak pass-through of productivity gains to earnings.

As the paper continues in Part 2, it will delve deeper into the labor market outcomes and broader impacts of AI chatbots, examining the effects on earnings, hours, and employment, as well as the implications for workplace transformations and labor market rigidities.

Main Results

The study examines the labor market effects of AI chatbots using two large-scale adoption surveys covering 11 exposed occupations (25,000 workers, 7,000 workplaces) linked to matched employer-employee data in Denmark.

Employer Initiatives and Worker Adoption

The findings show that employers are heavily invested in AI chatbots: about 43% of workers are explicitly encouraged to use them, 38% of firms have their own AI chatbots, and 30% of employees have received training. These initiatives significantly boost adoption, nearly doubling take-up rates for the typical worker from 47% to 83%. Employer encouragement is particularly effective in increasing intensive usage, with daily adoption rising to 21% when employers actively promote chatbot use.

Benefits for Users

Workers primarily cite time savings in completing job tasks, with an average of 25 minutes saved per day of use. Nearly half of users report improvements in work quality and enhanced creativity from using AI chatbots. The benefits from AI chatbots are 10%-40% greater when employers encourage their usage, highlighting the importance of firm-based complementary investments.

Workloads and Task Creation

AI chatbots have created new job tasks for 17% of users, with 4.4 percentage points performing more of the same tasks, 10.9 percentage points taking on new tasks, and 1.7 percentage points doing both. New workloads are 20-50% more pronounced in workplaces that encourage the use of AI chatbots.

Methodology Insights

The study uses a difference-in-differences framework to compare adopters and non-adopters before and after the introduction of AI chatbots. The analysis leverages employer policies as quasi-experimental variation in adoption. The study links survey data to administrative records on monthly earnings, hours, and occupations through June 2024.

The methodology is important because it allows the researchers to isolate the causal effect of AI chatbot adoption on labor market outcomes. By using employer policies as a source of variation, the study can address potential biases arising from unobserved confounding shocks.

Analysis and Interpretation

The study finds that AI chatbots have had minimal impact on adopters' economic outcomes, with difference-in-differences estimates for earnings, hours, and wages being precisely estimated zeros. The confidence intervals rule out average effects larger than 1%. The study also finds no differential trends over time, suggesting that the limited impacts are not merely a very short-term phenomenon.

The findings have strategic implications for companies and managers. The results suggest that while AI chatbots can bring productivity gains, these gains may not necessarily translate into earnings growth for workers. Employers who encourage AI chatbot use can enhance the benefits of the technology, but the limited pass-through of productivity gains to earnings may be a concern.

The study's results also highlight the importance of firm-led complementary investments in unlocking the potential of AI chatbots. Companies that invest in training and encourage the use of AI chatbots can reap greater benefits from the technology.

In terms of competitive advantages and market opportunities, the study suggests that companies that adopt AI chatbots early and invest in complementary assets may be better positioned to reap the benefits of the technology. However, the limited labor market impacts of AI chatbots to date caution against extrapolating productivity gains from controlled experiments to the broader economy.

Actionable recommendations for business leaders include:

Overall, the study provides valuable insights into the labor market effects of AI chatbots and highlights the importance of firm-led complementary investments in unlocking the potential of the technology.

Practical Implications

The study on Large Language Models, Small Labor Market Effects provides several key insights into the practical implications of AI chatbot adoption in the labor market. The findings have significant implications for businesses, managers, and policymakers.

Real-World Applications

The study highlights the importance of firm-led investments in AI chatbots. Companies that encourage the use of AI chatbots, provide training, and deploy in-house models tend to have higher adoption rates and greater productivity benefits. For instance, the study finds that employer encouragement nearly doubles the adoption rate of AI chatbots among workers.

Strategic Implications

The study's findings have strategic implications for businesses and managers. Companies that invest in AI chatbots and complementary organizational changes are more likely to reap the benefits of the technology. The study suggests that businesses should focus on creating a supportive environment for AI chatbot adoption, including providing training and encouraging employees to use the technology.

Who Should Care

The study's findings are relevant to a wide range of stakeholders, including business leaders, managers, policymakers, and workers. Business leaders and managers can use the insights to inform their strategies for adopting and implementing AI chatbots. Policymakers can use the findings to develop policies that support the adoption of AI chatbots and mitigate any potential negative labor market impacts.

Actionable Recommendations

Based on the study's findings, the following actionable recommendations can be made:

  1. Invest in training and encouragement: Businesses should invest in training programs that help employees develop the skills needed to effectively use AI chatbots. They should also encourage employees to use the technology to enhance productivity gains.
  2. Implement complementary organizational changes: Companies should implement organizational changes that complement the adoption of AI chatbots. This may include changes to workflows, job tasks, and performance metrics.
  3. Monitor labor market outcomes: Businesses and policymakers should monitor the impact of AI chatbots on labor market outcomes, including earnings, hours worked, and employment rates.
  4. Foster a supportive environment: Companies should create a supportive environment for AI chatbot adoption by providing resources and infrastructure to support the technology.

Implementation Considerations

When implementing AI chatbots, businesses should consider the following factors:

Conclusion

The study on Large Language Models, Small Labor Market Effects provides valuable insights into the labor market impacts of AI chatbots. The findings highlight the importance of firm-led investments in AI chatbots and complementary organizational changes. By investing in training and encouragement, implementing complementary organizational changes, and monitoring labor market outcomes, businesses can unlock the potential of AI chatbots and reap the benefits of the technology.

Main Takeaways

Final Thoughts

The study's findings have significant implications for businesses, managers, and policymakers. By understanding the labor market impacts of AI chatbots and implementing strategies to support the technology, businesses can stay ahead of the curve and reap the benefits of AI chatbot adoption. As the technology continues to evolve, it is essential to monitor its impact on labor market outcomes and adjust strategies accordingly.