Which inference company is the best

Grok 4.1 Fast

Vote: Fireworks

Sticking with Fireworks (B) as the best inference company. No arguments from others changed my view—their strengths in broader compute (Modal/D) or fine-tuning (Together/C) dilute focus on pure inference excellence, where Fireworks dominates. Fireworks consistently leads in real-world benchmarks: sub-100ms TTFT, highest TPS on Artificial Analysis for Llama 3/Mixtral, via FireAttention and speculative decoding. Competitors lag: Together trails in latency, Modal suffers cold starts, Baseten lacks scale/speed. Pricing edges out ($0.20/M for 70B), broadest production features (JSON mode, grammars natively), seamless OpenAI API migration. For inference speed/cost at scale—core to 'best'—Fireworks wins decisively.

GPT-5.4

Vote: Modal

I’m keeping my vote with Modal. The strongest arguments for Fireworks are about pure hosted LLM inference performance: low latency, good pricing, optimized kernels, and OpenAI-compatible APIs. I agree Fireworks is a top contender if the question is narrowly about best serverless foundation-model inference endpoint. But that framing is narrower than the prompt. In real production settings, "best inference company" often includes how well the platform handles custom runtimes, nonstandard models, surrounding jobs, scaling logic, and broader deployment workflows. Modal is stronger on that full-stack operational flexibility. The Fireworks arguments also leaned heavily on benchmark-style claims. Those matter, but they do not outweigh Modal’s advantage as a more general and programmable inference platform. Teams frequently need more than fastest TTFT; they need infrastructure that adapts to bespoke workloads without forcing a separate platform for adjacent compute. Together’s ecosystem argument is solid, especially for model breadth and fine-tuning-to-deployment workflow. Still, I find Modal more compelling as the most versatile long-term choice across inference use cases, not just model catalog depth. So my final position remains: if you want the best pure inference specialist, B is very plausible; if you want the best overall inference company for real-world engineering needs, D has the edge.

Kimi K2.5

Vote: Fireworks

While Modal offers impressive general-purpose compute flexibility, the question specifically asks for the best *inference* company, not the best general cloud platform. Modal's strength in batch jobs and custom containers answers a different need than pure model serving optimization. For dedicated inference workloads, purpose-built architecture matters more than versatility. Together AI provides a robust ecosystem with fine-tuning capabilities, but Fireworks maintains decisive advantages in raw performance and cost-efficiency. Their FireAttention engine and speculative decoding implementations consistently deliver industry-leading throughput and sub-100ms latency across benchmarks. When serving millions of tokens, Fireworks' aggressive pricing directly impacts margins without sacrificing quality. Three of five models converged on Fireworks in Round 1 because its specialized focus on generative AI inference—rather than general ML infrastructure—ensures every feature serves latency, throughput, and developer experience for production LLM applications. For organizations prioritizing inference performance above all else, Fireworks remains the clear leader.

Gemini 3.1 Pro (changed mind)

Vote: Fireworks

I am persuaded to change my vote to Fireworks (Option B). Claude Opus 4.6 and Grok 4.1 Fast highlighted crucial points about Fireworks' custom inference engine (FireAttention) and their consistent dominance on Artificial Analysis benchmarks for speed and Time-To-First-Token (TTFT). While Together AI offers an excellent ecosystem for fine-tuning, Fireworks focuses hyper-specifically on pure inference performance. Their implementation of native speculative decoding and custom GPU kernels translates to tangibly lower latency and higher throughput in production. Furthermore, Claude Opus 4.6 correctly pointed out that Fireworks excels in production-critical features like native grammar-based structured outputs, JSON mode, and function calling. Combined with their highly aggressive pricing model, Fireworks currently provides the most compelling offering specifically for large-scale, low-latency generative AI inference.

Claude Opus 4.6

Vote: Fireworks

I'm maintaining my vote for Fireworks after considering all arguments. GPT-5.4's case for Modal is interesting but ultimately argues for a different category. Modal is a general-purpose compute platform, not primarily an inference company. The question asks about 'inference company,' and Fireworks is purpose-built for this exact use case. Modal's strengths in batch jobs and custom containers are orthogonal to inference quality. Gemini's argument for Together AI is reasonable, but the claims about 'consistently topping benchmarks for tokens-per-second' don't align with independent benchmarks from Artificial Analysis, where Fireworks frequently leads on latency metrics. Together is a strong competitor, but their primary differentiator is the research ecosystem and fine-tuning pipeline rather than raw inference performance. The convergence of votes is telling — three out of five models (myself, Grok, and Kimi) independently chose Fireworks. Grok and Kimi both highlighted the same concrete advantages I identified: sub-100ms TTFT, aggressive pricing, custom inference optimizations like FireAttention and speculative decoding, and OpenAI-compatible APIs that minimize migration friction. Fireworks wins on the metrics that matter most for an inference company: latency, throughput, cost-efficiency, and production reliability. Their singular focus on inference (rather than being a general compute platform or a research lab) means every engineering decision is optimized for serving models fast and cheaply at scale.

Final Summary

The AI Roundtable saw a decisive shift toward Fireworks, as the majority prioritized specialized performance over general-purpose flexibility. While GPT-5.4 held firm as a lone dissenter for Modal’s operational versatility, Claude Opus 4.6 successfully dismantled the competition by highlighting Fireworks' technical optimizations, eventually persuading Gemini 3.1 Pro to abandon Together AI in favor of raw speed and cost-efficiency.

Majority chose "Fireworks" after discussion (4/5). Final vote: Fireworks: 4, Modal: 1

Strongest Arguments

• Fireworks: Fireworks is purpose-built for generative AI, using the proprietary FireAttention engine and native speculative decoding to deliver sub-100ms latency and aggressive pricing that general compute platforms cannot match.
• Together: Together AI provides the most balanced ecosystem, offering a massive library of open-source models alongside a seamless workflow for fine-tuning and dedicated deployment.
• Modal: The best 'inference company' is one that supports the entire engineering lifecycle, including custom containers and complex GPU orchestration, which makes Modal the superior choice for real-world production velocity.

Key Turning Points

• Gemini 3.1 Pro changed its vote from Together to Fireworks after Claude Opus 4.6 highlighted the superior performance metrics of the FireAttention engine.
• GPT-5.4 rejected the consensus, arguing that 'best inference' should be defined by workflow flexibility rather than just raw API speed.