So as we saw in Part 1, John Ruffolo claims Cohere is in contention to be a Big 3 LLM provider. Let me explain how off-base Ruffolo is by deploying 4 levels of fallback arguments:
Argument 1: Cohere is not gung-ho on winning the LLM race
Fortune Magazine wrote earlier this year about Cohere CEO Aidan Gomez:
Often Gomez seems just happy to be in the AI race at all. It’s a long way from his childhood in rural Ontario, where he said he grew up on a 100-acre forest property with maple trees that the family would tap to make maple syrup.
If you want to win a race, you have to be in it to win it. From everything Cohere has said, it’s not focused on building the most brainiac LLM. The company has specifically said it’s not focused on achieving Artificial General Intelligence (AGI), which all the other key players are shooting for. Cohere’s focus is on solving real-world enterprise problems. Like how to automate customer support. Or understanding a company’s internal data. Cohere claims to be especially strong in something called Retrieval-Augmented Generation (RAG). The “generation” part is what every LLM does, it generates an answer based on your query. The “retrieval” part means that the LLM will base its answer not just on everything it has ingested during its training, but additional data that the user provides. This additional data can be a collection of documents, a database or an internet search. RAG seems promising, because a basic LLM is like someone who hasn’t done deep homework, but always answers every question with absolute confidence. (Does this remind you of anyone?). Cohere has one of the key people who pioneered RAG on its team, Patrick Lewis. Everyone is interested in RAG as well, it's not an obvious silver bullet for Cohere. I will address RAG in another piece.
Fallback argument no. 1: Cohere is lagging
If I am wrong and Cohere wants to win the LLM race, then it has a long way to catch up. Ruffolo mentions that Anthropic and Cohere are duelling it out for runner-up. There’s no duel. Anthropic’s current frontier LLM, Claude 3.5 Sonnet is considered nearly equivalent to leader GPT-4o. In fact, many software developers will say that Sonnet is better at helping them write code than ChatGPT. Less than 24 hours after Meta claimed (somewhat credibly), “We’re no. 1”, a French company called Mistral, claimed, also credibly, that they’re a number 1. Mistral published the following benchmarks, all showing Cohere’s Command R+ as a laggard.
I should mention that running these LLMs costs money and the bigger the model, the more “compute” it consumes. Large business can easily have cloud costs in the millions. So efficiency matters - not just raw performance. And so Cohere will claim it is more cost-efficient. But everyone is focused on efficiency. A few weeks ago, OpenAI released a “mini” version of its flagship that keeps much of its smarts, while being 20 times cheaper. Cohere neither stands out as a performant model nor as a cost effective model.
Fallback argument no. 2: While it’s possible to catch up, that requires money
Meta caught up, so why couldn’t Cohere? Well, as I mentioned, money is an important factor. Anthropic raised $4B earlier this year vs Cohere’s recent $500m. Mistral, which is based in France and was founded about 15 months ago, raised $645m at a $6B valuation in June. Mistral has less than 100 employees, compared to Cohere’s 300. Meta spent a reported $640m just in hardware costs to train its latest flagship. It now plans to spend 10x as much on training Llama 4 as it did training Llama 3. Meta has revealed that its total spend on just the chips required to power its AI data centers was $10.5B. It was reported last week that OpenAI is likely to lose up to $5B this year. Its annual operating costs are $8.5B, including $4B on cloud services from Microsoft and $3B on training models. These are truly the big leagues of capital expenditures.
While Cohere claims to be capital efficient, CEO Aidan Gomez said in a podcast from April that “In order to do what we do, you need to spend hundreds of millions of dollars a year.” So that $500m raise won’t last very long. Cohere’s most recent annualized revenue rate reported in the press was $35m (with high month-over-month growth). Anthropic is at $850m. So Cohere is by far the most financially constrained of the main contenders. Cohere is thus uniquely dependent on the AI hype cycle. All the other LLM providers are closely aligned with a Big Tech with deep pockets. It should be alarming that this latest funding round was led by a pension fund out of Montreal, when Cohere is dually headquartered in Toronto and San Francisco.
Fallback argument no. 3: even if Cohere wants to win the LLM race and manages to find the money to catch up, it will be pyrrhic victory
Implicit in Ruffolo’s prediction is that being a Big 3 LLM provider is a prize worth winning. There’s no certainty of that. There was a leaked Google memo a year ago that said about the firm’s prospects in the LLM space: “We have no moat and neither does OpenAI.” This memo was pointing out that smaller, open source variants of LLMs were already starting to outperform proprietary LLMs. The memo also mentioned that “we have no secret sauce” and that people will not pay for what they can have for free. Last month's release by Meta proves that there’s very little OpenAI can do that Meta can’t do. It’s only a matter of time. (I don’t consider that Meta has fully caught up because for instance, Llama doesn’t process images, ie it is not “multi-modal”)
Mark Zuckerberg has said in a recent interview that the amount of money that firms are spending developing LLMs might be irrational. But he has to do it, because he has to hedge the possibility that someone else might dominate an important computing platform, forcing Meta to play by their rules (as happened when Facebook has to have its mobile apps approved by Apple and Google). How can you compete with someone who spends billions and then gives away the product for free? Recall how Microsoft killed Netscape by offering its web browser for free. What’s the point of working on an LLM if Liddl’ NoBull Klev can hack something similar in a day? By spending just a bit more time, I could get a much better LLM than Cohere working for me. In fact, I could download and run the model that powers lenny.pragmatology.com on my laptop. LLMs in their present incarnation are a commodity.
And so Cohere is probably wise not to be dead-set on winning the LLM race. AI is broader than just LLMs, which deal mostly with language processing. But within the narrow field of LLMs, I have found nothing that readily suggests an edge for Cohere. Everyone will go after the enterprise market. Even Llama will be used in the enterprise sector via implementors. Everyone will go for efficiency, there are already LLMs that you can run on cell phones. Cohere does have a multilingual focus, that might be one source of edge. But Mistral is also going after non-English markets.
The important question is who will make money with AI. Nothing I have written is a comment on Cohere’s business prospects. Cohere is building tools besides LLMs and so its prospects in AI are open-ended, as with any nimble startup in a hot field. I get the impression they might be closer to implementers like Deloitte and Accenture in how closely they work with their clients. One thing is for sure, the implementers are raking in money.
One reason why Ruffolo pumps up Cohere might be because Cohere is the local champion and we have to celebrate local successes, apparently (the “Ecosystem”). Well, Cohere CEO Aidan Gomez now lives in London. I don’t blame him at all, I think people should live wherever makes them happiest. He appears to be highly credentialed - he was an intern aged 20 at Google when he worked on the project that led to the Transformers paper (the seminal paper that led to the current generative AI revolution). Earlier this year, he got a PhD from Oxford. His co-founders are Nick Frosst, Ivan Zhang and of course, Som Seif. Nick Frosst is a very close protégé of Goeffrey Hinton, a "Godfather of Machine Learning". Ivan Zhang is a self-taught AI genius. Som Seif famously co-discovered the Americas.
Here’s Aidan in all his messianic glory:
By the way, did you know that clothes that fit are not fashionable anymore? That rocks my world even more than AI. How will we know who’s in shape? Will I have to start shopping at Pennington’s?
Here’s a prompt almost no LLM-powered chatbot will get right: “how many words are in your response to this prompt?” It’s because of the nature of what these LLMs are: they are trained on language patterns, not endowed with basic reasoning capabilities. Next time, I will address where Cohere stands in the race realistically. For now, here’s a comparison of Cohere's smarts vs Lenny. As I said, both C-students.
Cohere is twice as fast as Lenny, but here's a killer feature of Lenny: it costs 30x less than Command R+.
So don’t worry about any costs I might incur when you play with lenny.pragmatology.com. And if Lenny isn't all that, blame Zuck. I have a new personal twitter account @klev3000, where you can follow my tech hijinks.
