Google Cloud G4 beats AWS G7e in PT multi-GPU AI benchmark

Principled Technologies found that a Google Cloud G4 virtual machine outperformed an Amazon EC2 G7e instance in multi-GPU AI inference testing with eight NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs on both systems. The biggest gains came in multimodal and decode-heavy workloads, where G4 posted higher throughput and lower latency.

Why it matters: - Multi-GPU inference performance can affect response time, throughput, and infrastructure efficiency for production AI applications. - Principled Technologies found that Google Cloud G4 delivered better results than a comparable AWS EC2 G7e instance in workloads that matter for multimodal, agentic, and long-form generation use cases.

What happened: - Principled Technologies released hands-on benchmark results comparing a Google Cloud G4 virtual machine with an Amazon Web Services EC2 G7e instance for multi-GPU AI inference. - Both systems were configured with eight NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs. - PT tested Google Gemma 4 using the inference-perf benchmarking tool from the Kubernetes SIG community. - The tests ran across three scenarios: multimodal text-plus-image input, prefill-heavy text, and decode-heavy text.

The details: - In the multimodal scenario, Google Cloud G4 beat AWS EC2 G7e on every metric PT measured. - Google Cloud G4 posted 284.3 output tokens per second versus 241.8 for AWS, a 17.6% gain. - Google Cloud G4 reached 2,775.2 input tokens per second versus 2,324.7, a 19.3% gain. - Google Cloud G4 processed images 17.3% faster. - Google Cloud G4 cut time to first token to 1,006.5 ms from 1,267.1 ms, a 20.5% improvement. - Google Cloud G4 reduced time per output token to 62.4 ms from 90.4 ms, a 30.9% improvement. - Google Cloud G4 lowered end-to-end latency to 3,045.0 ms from 4,233.8 ms, a 28.0% improvement. - In the decode-heavy text scenario, Google Cloud G4 again led on throughput and latency. - Google Cloud G4 delivered 2,442.7 output tokens per second versus 2,222.3, a 9.9% gain. - Google Cloud G4 delivered 12.5% higher input token throughput. - Google Cloud G4 cut time to first token to 77.3 ms from 105.6 ms, a 26.8% improvement. - Google Cloud G4 reduced time per output token by 9.8% and end-to-end latency by 7.3%. - In the prefill-heavy text scenario, output token throughput was nearly identical at 823.5 tokens per second for Google Cloud G4 and 822.3 for AWS EC2 G7e. - Google Cloud G4 still held a 9.4% edge in input token throughput, at 39,076.0 versus 35,699.7 tokens per second. - PT said the performance gap likely stems from GPU interconnect design. - Google Cloud G4 includes dedicated peer-to-peer switching infrastructure that enables all eight GPUs to communicate directly in a full all-to-all configuration. - AWS G7e appears to support high-speed peer-to-peer communication only between adjacent GPU pairs, with cross-pair traffic taking a less direct path.

Between the lines: - The benchmark suggests that multi-GPU networking architecture can matter as much as raw GPU count in inference workloads. - The advantage was strongest when the workload required more GPU coordination, including image encoding, large-context prefill, and sustained token generation. - PT said cloud infrastructure choice can directly affect responsiveness, user experience, and infrastructure cost efficiency for large multimodal models. - PT also stated that a Google Cloud G4 VM delivered a significant performance advantage for these workloads over a comparable Amazon EC2 G7e instance.

What's next: - Principled Technologies pointed readers to the full report for methodology and complete results tables. - The findings may influence cloud selection decisions for teams running large multimodal or long-form AI inference workloads.

The bottom line: - In PT's testing, Google Cloud G4 outperformed AWS EC2 G7e most clearly in multimodal and decode-heavy AI inference, while prefill-heavy performance was closer but still favored Google on input throughput. - Principled Technologies is based in Durham, North Carolina, and says it provides fact-based marketing and competitive analysis backed by transparent methodologies. - More information is available on Principled Technologies.