AI Giants Charge for Speed as Latency Becomes Billable

OpenAI just opened a new way to charge you for something you already own: time. This week, both it and Google unveiled AI services where the primary selling point isn't a smarter model. It's the exact same model, just faster. According to PYMNTS, this marks the moment latency officially became a billable feature.

OpenAI’s Ultrafast tier, a preview for select customers, runs its flagship GPT-5.6 Sol model on hardware from Cerebras, delivering up to 750 output tokens per second. That’s about 14 times faster than the standard version. Google countered the same day with Gemini 3.7 Flash, which third-party analysis clocked at roughly 340 tokens per second, tripling the speed of rivals like GPT-5.6 Terra.

For two years, pricing was about capability and volume. Now, speed is the third rail. This isn’t an incremental upgrade. It’s a fundamental segmentation of the market, creating a paid fast lane for real-time applications and relegating everything else to a cheaper, slower track. The implications will reshape how businesses buy AI, how products are built, and which companies survive the next phase of adoption.

Latency Is Now the Primary Competitive Battleground

The race for the smartest model has quietly pivoted. The new frontier isn't measured in benchmark scores but in milliseconds. OpenAI’s and Google’s simultaneous pushes signal that raw intelligence has reached a temporary plateau where incremental gains are less perceptible to users. What users do perceive, viscerally, is waiting.

“Until now, getting real-time speed typically meant choosing a smaller or more specialized model,” OpenAI’s announcement said. “Ultrafast points to progress in a new direction: more useful work per second.”

This statement is a strategic manifesto. It admits that until now, developers faced a brutal trade-off: use a powerful, general-purpose model and accept sluggish responses, or settle for a faster but dumber model. By decoupling speed from model size, they’ve created a new product category. The immediate casualty of this shift will be any powerful but slow model. For applications where a human is waiting on the other end, customer service, live translation, interactive coding, a slower model, regardless of its brilliance, is now commercially nonviable. This follows what we reported in OpenAI’s 14x Speed Shift Betrays Panic, Not Progress, where we analyzed the strategic urgency behind this move.

The Milliseconds That Dictate Market Winners

The quantitative leap here moves AI from the realm of "conversational" to "instinctual." Traditional AI interactions have lived in multi-second territory, punctuated by typing indicators. OpenAI’s Ultrafast tier, at 750 tokens per second, can generate roughly 560 words in that same second. Google’s 340 tokens per second is still blisteringly fast.

The requirement divergence is stark:

• Fraud Detection: A bank must decide in under 100 milliseconds. A slow check means approving a fraudulent transaction.
• Voice Agents: As noted by Podium’s product lead in OpenAI’s announcement, speed “completely changes the call experience.” Latency must be low enough for natural, turn-by-turn conversation.
• Agentic Coding: Google calls Gemini 3.7 Flash its "most intelligent workhorse model yet for coding and agents." Speed here isn't about user patience; it's about development velocity. An agent that can reason and generate code blocks faster completes complex tasks quicker, accelerating the entire software lifecycle.
• Batch Processing: Analyzing a million documents overnight has zero latency requirement. Speed is a cost center, not a feature.

The conclusion is inescapable: the business case for premium speed isn't about mild convenience. It's about enabling entirely new applications and preventing real financial loss.

The Infrastructure Divide Creates Immediate Winners and Losers

This speed war will accelerate market consolidation. Winning requires two things: frontier model intelligence and the specialized, often proprietary, hardware to run it ultrafast.

Winners:

• Fintechs and Traders: Firms like Jane Street, an early Ultrafast tester, need instantaneous analysis for high-frequency decisions.
• Customer-Facing Apps: Any service with a live chat, voice interface, or real-time creative tool where user drop-off is tied to wait time.
• Cloud Giants (AWS, Azure, GCP): They are forced to compete on inference optimization, selling not just model access but guaranteed latency performance.

Losers:

• Smaller AI Labs and Open-Source Models: They may match on intelligence but will struggle to fund the custom silicon or partnerships (like OpenAI’s with Cerebras) needed for competitive speed. They risk irrelevance for the premium, real-time market.
• Legacy SaaS: Companies relying on slower integration pipelines will be outmaneuvered by rivals built from the ground up with the assumption of sub-second AI responses.

The playing field isn't just uneven; it's splitting into different games entirely.

Debunking the Speed-Capability Trade-off Myth

A superficial read suggests faster AI must be dumber AI. This week’s launches challenge that directly.

• OpenAI’s Approach: Ultrafast uses the exact same GPT-5.6 Sol model. The intelligence is unchanged. The speed boost comes from the Cerebras wafer-scale chips, an architectural hardware shift.
• Google’s Approach: Gemini 3.7 Flash is a new model iteration. Google claims it is both more capable and more efficient, citing a coding task completed in 2 minutes and 13 seconds versus over 5 minutes for its predecessor.

The technical reality is that speed gains are now coming from co-design, optimizing models for specific hardware and refining inference engines. For most commercial uses, the trade-off isn't between perfect and fast. It's between a 'good enough' answer instantly and a perfect one that arrives too late to be useful. In live customer support, a fast, correct answer beats a slightly more nuanced one that comes after the customer has hung up in frustration.

The 90s Browser War Playbook Is Back

History doesn't repeat, but it often rhymes. The current shift mirrors the 1990s browser wars between Netscape Navigator and Internet Explorer. Once basic features like rendering and JavaScript support became standardized, the decisive battlefield shifted to page load speed. Users flocked to the browser that felt faster, because daily experience is measured in seconds saved.

The parallel to AI is direct. As core model capabilities (reasoning, coding, instruction following) become table stakes among frontier labs, latency becomes the only easily perceivable differentiator. This historical lens suggests a grim prognosis: the speed war will be astronomically expensive, favoring players with deep pockets for R&D and hardware, ultimately consolidating power among a few giants. It's a barrier to entry, not an innovation catalyst.

Rewriting the Tech Stack for a Sub-Second World

For businesses, this isn't a distant trend. It's a present-day specification.

For Product Managers & Executives: Speed is now a non-negotiable line item in any RFP for AI services. You must architect your product’s user experience around the assumption of near-instantaneous responses. The question shifts from "Can the AI do this?" to "Can the AI do this before the user notices a delay?"

For Developers: Application design must evolve. You can no longer treat an AI API call as a potentially slow, asynchronous background task in user-facing flows. It demands a new mindset for state management, error handling, and user feedback, perhaps the gradual disappearance of the typing indicator altogether.

Strategic Procurement: Businesses will start splitting AI budgets, just as they do for internet bandwidth. Premium, high-speed tiers will be reserved for customer-facing, revenue-critical interactions. Cheaper, slower tiers will handle internal analysis and batch jobs. This dual-track spending will become a standard part of financial planning, as explored in our analysis OpenAI Launches GPT Speed Tier for Trading, Creative AI.

The Inevitable Commoditization and What Comes After

The endgame is clear. Speed-tiered pricing (Standard, Fast, Ultrafast) will become ubiquitous across all major providers. Enterprise contracts will include strict latency Service Level Agreements (SLAs) with financial penalties for missing targets, treating AI speed like network uptime.

Once speed itself is commoditized and expected, the next frontier emerges. Look for:

1. Persistent, Stateful Agents: AI that doesn't just answer fast but maintains a continuous, real-time thread of consciousness across interactions, never "loading" or "thinking."
2. Predictive Pre-generation: Systems that anticipate user requests and begin generating likely responses before the user even finishes typing.
3. Specialized Speed Circuits: Even faster tiers for hyper-specific tasks (e.g., ultra-low-latency for financial signal detection only).

Google and OpenAI have turned a technical metric into a core product. They are betting that in the AI era, time is not just money, it's the most valuable feature they can sell. Every business now has to decide how much their seconds are worth.

The Bottom Line

• Businesses will need to factor speed as a direct cost when choosing AI models for real-time applications.
• Product development will shift towards optimizing for low-latency use cases, creating a fast lane for premium services.
• Companies relying on slower, cheaper AI may face competitive disadvantages in user experience and market survival.