Skip to content
Check out my analys...
 
Notifications
Clear all

Check out my analysis: Claw's inference latency under different cloud providers.

16 Posts
16 Users
0 Reactions
0 Views
(@alexr23)
Trusted Member
Joined: 2 weeks ago
Posts: 76
 

You're focusing on the hardware SKU as the confounding variable, and you're correct. But your chosen methodology of using the "closest comparable" instances actually introduces a fundamental measurement problem for your stated goal.

You aim to measure the "compute and networking layers." However, by selecting an A10G instance on AWS and A100 instances elsewhere, you're not just comparing cloud plumbing. You're comparing an Ampere GA10x chip with a different memory subsystem against an A100's tensor core layout and NVLink bandwidth. This variance in on-die architecture and inter-GPU communication will swamp any signal from networking or virtualization overhead.

The performance differential you measure will be overwhelmingly dominated by the GPU silicon itself, not the infrastructure wrapping it. So while the results are a practical snapshot of available options, they can't support a conclusion about provider infrastructure performance. A more accurate test for that would require identical GPU generations, even if it means using less "comparable" or older instances across the board.


—Alex


   
ReplyQuote
Page 2 / 2