Most cloud-based genAI performance stinks
I’ve been asked if generative AI systems are always slow. Of course, I reply, “Slow, as compared to what?” The response I always get is funny. “Slower than we thought it would be.” And the circle continues.
Performance is often an afterthought with generative AI development and deployment. Most deploying generative AI systems on the cloud, and even not the cloud, have yet to learn what the performance of their generative AI systems should be, take no steps to determine performance, and end up complaining about the performance after deployment. Or, more often, the users complain, and then generative AI designers and developers complain to me.