The AI Reality Check: Why Competitive Intelligence is Crucial in Navigating the GenAI Hype
There are a few who think the LLM hype is over. Some tell us AI will make us all jobless multimillionaires—usually, the same type who told us EFTs are the future of investing. This article is all about the AI reality check.
But the LLM narrative is fascinating, capturing the attention of critics and advocates alike. If we dive beneath the surface of the disagreement, we’ll see it’s not a complete debunking of large language models (LLMs) or generative AI (GenAI).
Instead, we’re at the stage where expectations need to be recalibrated. The hype cycle has inflated expectations about AI’s reasoning and decision-making. But it’s too early and simplistic to call it a failed experiment.
The Fallacy of Overhype
First, it’s important to accept where the critics are correct. Like OpenAI’s GPT series, LLMs aren’t reasoning machines as we understand it in human cognition.
Papers from Apple and other researchers and tests like the ARC-AGI competition highlight limits in LLMs’ ability to generalise to out-of-distribution problems. These models often fail to make new, logical inferences on tasks outside their training data.
Critics argue that LLMs often just match patterns. They lack true understanding or reasoning. By design, LLMs are statistical engines, not cognitive entities.
But here’s where it gets tricky. The limitations cited aren’t failures of the technology. They stem from a mismatch between its current state and the sky-high expectations of an overzealous market.
LLMs were never designed to “think” like humans. Yet, some, especially outside the tech community, have confused advanced language processing with AGI. It’s a classic case of mis-marketing.
The 5% ROI Reality Check
Many industry reports now suggest that the real return on investment (ROI) from GenAI projects, outside of niche use cases, hovers around 5%. This figure should be a wake-up call for businesses. Boards and CxOs are under immense pressure to deliver on AI-driven transformation promises. The cold truth is that many AI projects have been used in unsuitable contexts, leading to limited results.
GenAI excels at automating tasks, generating content, and summarising information. However, it is not yet good at reasoning or making decisions.
It’s clear that some businesses are attempting to hammer square AI pegs into round business holes. Instead of seeing LLMs as cognitive systems, view them as they are. They are probabilistic tools for generating language-based outputs. They require significant human oversight. Using LLMs to optimise customer service. Help with code or drafting legal documents under a lawyer’s supervision is a valuable use. But using them to handle complex reasoning tasks or mission-critical decision-making? That’s where the jig is, indeed, up.
The Persistence of GenAI’s Utility
Despite the wave of critiques, writing off LLMs entirely would be a mistake. Unlike the crypto bubble, many saw no value in it. But, GenAI has proved useful. It’s just not the kind of utility some envisioned.
Consider the analogy of early search engines. They didn’t “understand” your queries in any human-like way, but they still transformed how we access information. In the same vein, while not reasoning machines, LLMs have reshaped industries that rely on language processing. Copywriting, customer support, marketing, legal research, and software development are now more efficient. These models can sift through vast text data. They can provide insightful summaries and assist in drafting documents. With human oversight, the drafts can meet professional standards. Great for automation tools.
This brings us to the real crux of the issue: the human in the loop. LLMs and GenAI technologies will augment, not replace, human expertise for the foreseeable future. The hype around fully autonomous systems is misplaced. But, GenAI as an augmentation tool is powerful. Companies that use LLMs as aids, not as agents, will likely see good returns.
The “Long Tail” of GenAI
While some voices call the LLM bubble burst, the reality is that the adoption of GenAI will be a long, slow burn rather than an immediate collapse. Like the early internet, the true value of GenAI will show as businesses learn to use these tools well and strategically. Currently, most GenAI deployments are still in the exploratory or proof-of-concept phase. We’ll see its real potential when companies stop trying to make AI do what it wasn’t designed for and start building systems around its strengths.
Moreover, advancements in AI architectures are continuing. We are already developing hybrid models, symbolic reasoning, and multimodal systems. These integrate different data types and reasoning methods. This doesn’t mean AGI is around the corner, but it does suggest that LLMs and their derivatives are far from a dead end.
The Right AI for the Job
What the business needs is a more nuanced approach to AI adoption. Boards should abandon grand visions of AI-powered decision-making. They should focus on using AI where it adds value. For example, in fields like customer service, LLMs paired with human operators have proven to be game-changers in summarising regulations in finance. But humans, butld make final decisions. The key is understanding AI’s limitations and using it to augment human capabilities, not replace them.
Where Competitive Intelligence Comes In
This shift in expectations for LLMs and AI makes competitive intelligence important. The role of competitive intelligence is not just about gathering lots and lots of data on competitors or market trends. It’s about interpreting the data, finding opportunities, and avoiding costly mistakes. As we wrestle with the hype around GenAI, competitive intelligence offers a view that separates hype from reality.
For example, some companies may be tempted to spend millions on AI initiatives that promise groundbreaking results. A competitive intelligence function would help them. It would analyse competitors, assess market adoption, and determine the value of AI investments. Competitive intelligence can help leaders. It shows where competitors overinvest in unproven tech. It also reveals where they gain practical benefits. Finally, it guides leaders in using AI tools in their firms. By analysing where AI fits into broader industry trends—and, more importantly, where it doesn’t. Competitive intelligence ensures that decisions are rooted in competitive reality, not wishful thinking.
Conclusion to The AI Reality Check: Why Competitive Intelligence is Crucial in Navigating the GenAI Hype
The hype may support the notion that LLMs are reasoning machines or a path to AGI, but it would be a grave mistake to abandon them entirely. It’s not going to happen anyway! GenAI has already proven its worth in specific applications. As long as businesses use these tools for what they do best—processing language, not reasoning—the returns can be significant.
At Octopus Intelligence, we understand the importance of applying the right tool for the job. If your business is dealing with AI adoption and competitive intelligence, we can help. We’ll find practical strategies to keep you ahead of your competition. Reach out to us, and let’s ensure your AI investments deliver real value, not just hype. Your own AI reality check.
We are Octopus. The Global People-Powered Competitive Intelligence Agency.
Outsmart your competition. Make the unknown known. Octopus helps you gain clarity in complex markets. With clients and tentacles around the world, we deliver sharp, actionable competitive intelligence through a blend of deep primary (HUMINT) and secondary research. If you’re looking to make smarter decisions, beat the competition, and reduce uncertainty, we’re the partner you want on your side.
