Google's TPU 8tx and 8i Max Split the Eighth-Gen Line by Workload

Google's eighth-generation TPU lineup is no longer a simple training-versus-inference split. With TPU 8tx and TPU 8i Max, the company is building hardware around specific jobs: confidential inference on one side, long-context agent work on the other. That's a more interesting move than the April launch, and it hints at how quickly Google wants to produce specialized parts.

Two new parts, one shared silicon base

TPU 8tx uses the same silicon as the TPU 8t, which targets large-scale pre-training. The difference is in the surrounding system. 8tx swaps the Arm-based Axion host CPUs for 5th Gen AMD EPYC processors and adopts the Boardfly network topology that the TPU 8i uses, replacing the 8t's 3D torus. Google positions it for secure, confidential inference and small-scale training.

TPU 8i Max is the simpler story. It puts two TPU 8i chips on a single package. That doubles the theoretical compute, SRAM, and HBM bandwidth relative to one 8i. Google says the package fits twice the silicon into the existing rack footprint and delivers more compute per square foot and per watt. Nothing about the chip itself is new, but the packaging change matters for operators who want more throughput without redesigning their data halls.

Where 8i Max lands against Rubin and MI455X

Doubling the 8i gives rough derived figures of about 20.2 FP4 PFLOPS, 768 MB of on-chip SRAM, and about 17.2 TB/s of HBM bandwidth. Those are derived from Google's per-chip numbers, not separately published results. For comparison, Nvidia rates its Rubin R200 at 50 FP4 PFLOPS for inference with 22 TB/s of HBM4 per socket, and AMD's MI455X is rated at 40 FP4 PFLOPS. The compute gap is still wide, but it's narrower than the raw numbers suggest for a chip built around bandwidth-limited inference.

That bandwidth point is the real design argument. Google calls the memory wall a real concern "when dozens of AI agents work together across an entire software codebase." Longer context windows, with some models targeting 1 million tokens, produce large KV caches. Every new token has to read that cache back from memory, so bandwidth becomes the bottleneck. Doubling HBM bandwidth within one package is a different answer from Nvidia's LPX approach, which pairs a second chip type for decoding with a separate storage tier for context. Google is betting that more bandwidth on one package is simpler to deploy.

The software layer: TPU Sync and Agent Compute Engine

Google is pairing the hardware with software aimed at memory pressure. TPU Sync is an open-source "model-state layer for TPU clusters" that moves KV caches off HBM and down to host DRAM, and experimentally to shared storage. Google says this can cut KV transfer latency by "more than 90%," though time-to-first-token gains will vary by workload. Separately, the Agent Compute Engine on Google Cloud pauses idle agent sessions and moves their working memory aside until they're needed again. Both features address the same problem: agent sessions that hold a lot of state while doing very little.

Google also credits its vLLM-TorchTPU stack with "state-of-the-art price-performance marks" on InferenceX and AgentX, benchmarks that test realistic agentic coding and long-context work. The only third-party numbers published so far cover the previous generation. In SemiAnalysis's InferenceX tests from September, running Qwen3.5 397B in FP8 at 20 tokens per second per user, Ironwood delivered 50.4% more tokens per dollar than Nvidia's B200 and 96.0% more than the B300. Ironwood lacks native FP4, which complicates direct comparisons with FP4-focused rivals. November results showed its FP8 performance beating Nvidia's FP4 GB300 NVL72 on the same model, at 1.67 times the performance per dollar. None of that tells you how the eighth generation performs, and that's the gap to watch.

Confidential inference as a product

TPU 8tx is the more consequential launch for regulated customers. Google describes the security as built into the hardware, with isolation designed to prevent snooping on sensitive data, including model weights. The step being covered is the one where data has to be decrypted in memory during processing, which has been the hard part of confidential computing for accelerators.

Google's Private AI Compute already runs confidential serving on TPUs. It describes the platform as a "hardened TPU platform" with protections "comparable to a typical TEE," according to a September technical brief. The 8tx looks like that capability packaged as a product for customers. The target is organizations handling sensitive data with frontier or proprietary models, which want to fine-tune on private data, distill large models into smaller ones, and train agents without moving data between clusters. Google frames this as "secure agent execution." Earlier reporting described talks about running TPUs in classified environments, though the 8tx isn't named in that context.

Competitors take different routes. Nvidia's NVL72 offers one security domain spanning 72 Rubin GPUs, 36 Vera CPUs, and the interconnects. Intel's TDX is adding trusted I/O to cover data moving to accelerators, which addresses the same gap Google is targeting. Google hasn't said why it moved the 8tx host to AMD. Its Confidential VM configuration list includes both AMD and Intel series chips and no Arm-based ones, and Private AI Compute already uses an AMD-based hardware TEE for CPU workloads. That suggests Google swapped in a host it already trusts.

What's still missing

The 8tx's networking choice is worth a closer look. Google says Boardfly delivers "up to 2x lower latency compared to TPU 8t" with almost no performance penalty, though it doesn't say which measure it used. Torus networks suit training, where chips mostly exchange data with neighbors in predictable patterns. Inference on large mixture-of-experts and reasoning models sends smaller messages between distant chips, and each hop adds delay. Google puts Boardfly at a maximum of seven hops, versus 16 for a torus at 1,024 chips.

Still, the spec sheet has large holes. Neither 8tx nor 8i Max has published HBM capacity, pod sizes, pricing, or named customers. Attestation details, which would let a customer verify confidentiality themselves, are also absent for the 8tx. Both new systems, along with the 8i and 8t, are listed as "coming soon" on Google's Cloud TPU page.

My read: the more useful question is what this modular approach does to Google's roadmap cadence. Delivering two new systems about seven months after April's pair suggests Google wants to produce targeted hardware faster than it has in the past. Whether that holds up depends on the benchmarks and the availability dates still to come.