The Great Myth of The "Token Spend" Problem

Over the last two to three months, the narrative around AI has drastically shifted. As costs per token and AI adoption metrics have gone up, we have somehow gone from a world of encouraging “token-maxxing” to now championing various different cost saving methods in an attempt to reduce spend.

Companies are not wrong. They DO have a problem. I've talked to companies in situations ranging from blowing there entire monthly budget in less than 12 hours to companies spending tens of thousands a month in tokens just to build systems to figure out where they are actually spending on tokens inside of their organization. The problem is NOT token spend, however.

The problem is that models are not being given valuable enough work by humans.

To elaborate on this, I'll talk about a few examples I've seen and / or heard about.

1. Backlogs for engineering teams are growing faster than ever. If this was due to rising product demand, there would be no problem worth discussing here. However, one of the few reasons this is happening across several companies I've interacted with is because it's easier than ever for PM's to make tickets, PRDs, or <insert your corporate acronym here> by leveraging LLMs to write the content for them. Unfortunately and ironically, this actually leads to hurting the company because the signal to noise ratio in backlogs has never been lower, the quality of these documents are extremely poor, and the company is spending real money on this output. You can blame the employee in this case, however the real problem is one of company culture; where either too much bureaucracy is required and / or redundant roles exist within the organization. Sadly, in many of these cases, the token spend to build the actual feature can be the same or less than the PRD to describe it.
2. Many companies have adopted policies that require or encourage every employee to “ship code.” While I do think it's valuable for everyone to know how to use tools like Codex and Claude Code, it's comical to assume that every employee in an organization will be able to build something that justifies the value of the cost being spent on it. I've talked to several people who have vibe coded some dashboard, as if they were doing busy work just trying to fulfill a quota of “shipped software” for a company metric. It doesn't take long to figure out where the actual problem is here.
3. For the first time in human history, anyone can harness intelligence capable of solving some of the world's hardest problems, yet it's being used to build minimally different version of feature A at competitor A and / or building things that don't actually a solve problems for humans. Unfortunately, most companies are not thinking big enough on what they can build. This is the biggest problem with “token spend” today.

The reality is that AI has changed our world forever, and there are new “truths” to the world that I believe to in fact, be true. These are:

1. Execution is cheap, (good) ideas are expensive. Pre-AI this was famously flipped. Now that intelligence is a commoditized, knowing what to use AI for and what work to avoid is where much of the value accrual happens.
2. Sales and marketing are most important, everything else is mostly an implementation detail. By no means here do I mean that every other position outside of these two functions are not valuable. For example, If you are in product / engineering, it is valuable to any company to ensure that a product can be iterated on quickly, can scale, and be analyzed by the organization. However, “good, clean code” is less valuable than it was pre-AI. If you think about it, it was never actually valuable. Rather, it was just a prerequisite to moving quickly, what is even more valuable today. Moving quickly and product are just subsets of marketing in today's world, not the other way around.
3. Never offload human-to-human communication to AI, ever. When a human can tell something was written by AI, they tend to respond extremely negatively towards the content. It's also embarrassing for the writer as it's a strong signal he or she can't come up with his or her own original thoughts, thus responding with your own doesn't warrant your own time. While there's good arguments on either side on whether “AI will take your job” that are out of scope for this article, no matter what is true, humans will always be in charge of where money is spent and allocated. Therefore there is no general skill more important than knowing how to communicate with humans, further reiterating #2 and building on #1.
4. The best, most expensive model should be the cheapest for the tasks you are working on. If it's not, you should start to question what you are spending your time on.

I can't predict the future, I don't know everything, and we are still in the very early days of AI. However, I remain completely unconvinced that token spend is a real issue that organizations face. Rather, I see it as an effect of other, deeply rooted issues that have existed for years that can now be quantified in dollar figures.