Descartes Labs builds one of the world’s TOP500 fastest supercomputers and the first system on the list to run on virtualized public cloud resources
This is an extremely interesting test and story. Building highly scalable systems in AWS is not anything new, but running HPC software on the cloud has always thought to be impossible, mostly because of the virtualization abstraction of the operating system, and because the network that connects the resources is not designed for High Performance Computing.
Finally, someone managed to disprove this, and they reached a peak performance of 1.926 petaflops, placing the virtual cluster at number 136 of the TOP500 list.
Another extremely interesting point of comparison was that no contact with AWS has been required, they just started the machines without any special AWS support, anyone could replicate this experiment for around $5000.
The potential for democratization of HPC was palpable since the cost to run custom hardware at that speed is probably closer to $20 to $30 million. Not to mention a 6–12 month wait time.
Now you can also imagine the huge scalability that AWS as a whole has. This experiment from the point of view of AWS was just a normal customer that needed some resources, nothing special.