Are LLMs making StackOverflow irrelevant?

Hi, this is Gergely with a free issue of the Pragmatic Engineer Newsletter. This article is one out of five sections from The Pulse #119. Full subscribers received this analysis a week and a half ago.

The volume of questions asked on StackOverflow started to fall quickly after ChatGPT was released in November 2022, and the drop continues into 2025 at alarming speed. Fresh data shows how bad things are, courtesy of software engineer, Theodore R. Smith, a top 1% StackOverflow contributor. He shared the number of questions posted by users in this Gist dump:

StackOverflow has not seen so few questions asked monthly since 2009! The graph shows the steep drop-off in usage accelerated with the launch of OpenAi’s chatbot, and It’s easy enough to figure out why: LLMs are the fastest and most efficient at helping developers to get “unstuck” with coding.

Before the rise of this technology, StackOverflow was the superior option to Googling in the hope of finding a blog post which answered a question. And if you couldn’t find an answer to a problem, you could post a question on StackOverflow and someone would probably answer it.

StackOverflow’s decline actually started before ChatGPT, even though it’s easy to blame for the fall in questions asked:

In April 2020 – a month into the Covid-19 pandemic – StackOverflow saw a short-lived surge in usage. However, from around June 2020, the site saw a slow, steady decline in questions asked. ChatGPT merely sped up the decline in the number of questions.

From 2018, StackOverflow drew more criticism for its moderation policies. On one hand: StackOverflow relies on moderators to de-duplicate questions, close off-topic posts, and to keep things civil. But moderation came to feel beginner-unfriendly, where newcomers struggled to post questions that were not immediately closed by a moderator. Asking a question that would stay open became an effort in itself, which was intentional. But it’s easy enough to see why a higher barrier to asking questions resulted in fewer questions being posted.

A year ago, I asked if reports of StackOverflow’s downfall were exaggerated. Back then, the data looked grim:

At the time, the company blamed some of the decline on search engine traffic. However, a year later, it’s safe to assume StackOverflow needs a miracle for developers to start asking questions again in the same numbers as before.

The drop in questions indicates that trouble is ahead. Most of StackOveflow’s traffic comes from search engines, so this decline is unlikely to have an equally dramatic immediate drop in visits. However, any fall can turn into a vicious cycle: with fewer questions asked, the content on the site becomes dated and less relevant, as fewer questions mean fewer up-to-date answers. In turn, the site gets less search engine traffic, and visitors who get to the site via search find answers woefully out of date.

StackOverflow’s decline is an example of how disruptive GenAI can be to previously stable businesses. StackOverflow was acquired for $1.8B in 2021 by private equity firm Prosus, and even with moderate traffic decline, the site had been one of the most trusted websites for software engineers, making it a valuable asset. But the new data indicates an irreversible decline, and it’s hard to see how StackOverflow will be relevant in future.

StackOverflow still sells a Teams product for internal Q&A. Still, the fall of the public-facing StackOverflow traffic suggests that former users prefer using internal LLMs at the companies to ask questions, rather than use an StackOverflow-like site.

Private equity often has a reputation for acquiring companies at lowest possible prices, then squeezing money out of them. In the case of StackOverflow, we might see the opposite: a private equity company taking a gamble with a large acquisition, and getting a sizable loss.

Another question: where will LLMs get coding Q&A training data in the future? In some ways, it feels to me that StackOverflow is the victim of LLMs ingesting data on its own Q&A site, and providing a much better interface for developers to solve programming problems with. But now the site gets far fewer questions and answers, where will training data come from?

This is a question with no clear answer, that’s similar to the one about where the next generation of entry-level software engineers will come from, when most businesses hire fewer than before because LLMs can do roughly the same job as a newly qualified human?

I expect the industry will adapt: perhaps LLMs in the future won’t be as good as today in answering StackOverflow-like questions, but will have other more advanced capabilities to make up for it; like trying various solutions and validating them, or coding agents might become more helpful.

The same applies to the question of entry-level engineers: the tech industry has always adapted, and I don’t see it being different this time, either.

The full The Pulse issue additionally covers:

1. Industry pulse. Fake GitHub stars on the rise, Anthropic to raise at $60B valuation, JP Morgan mandating 5-day RTO while Amazon struggles to find enough space for the same, Devin less productive than on first glance, and more.

2. Apples fires staff over fake charities scam. In order to get around $4,000 per year in additional tax cuts, six Apple employees tried to defraud Apple – and the IRS. They were caught, fired, and now face prosecution. A reminder that getting “clever” with corporate perks can wreck otherwise lucrative careers at Big Tech.

3. AI models just keep improving rapidly. Two months after wondering whether LLMs have hit a plateau, the answer seems to be a definite “no.” Google’s Gemini 2.0 LLM and Veo 2 video model is impressive, OpenAI previewed a capable o3 model, and Chinese startup DeepSeek unveiled a frontier model that cost less than $6M to train from scratch.

4. Middle manager burnout incoming? A Forbes article suggests a broader middle manager burnout to come across most professional sectors. This could simply be a consequence of higher interest rates, teams growing less, and more pressure on managers. It’s tougher to be an engineering manager, than it has been during the 2010-2022 period, that’s for sure.

Read the full The Pulse issue