How Nvidia is overcoming slowdown issues in GPU clusters

Claims it has cracked the scalability problem with incoming H100

GTC  If you're training a large deep learning model, and you want it to train it faster, you should just throw more GPUs at it, right? Well, that works for a while, but there is actually a limit to the amount of proportional performance you can expect from adding additional GPUs in a server cluster.…