Latest Trends in High Performance Computer Architectures

In 1956, the term Artificial Intelligence was coined. From then on, research began that was thought to yield tangible results in a very short time. This was not the case and AI entered what was called its polar winter. In recent years, the existence of a lot of data with which to train neural networks and high-speed computers that execute the enormous number of associated operations in a reasonable time, has made AI present in any human activity.

AI has changed the way we do research using supercomputers. And it is also revolutionizing the design of chips that efficiently execute AI programs in the supercomputer centers. 

In this talk, we are going to describe the characteristics of processors that already exist or that will be a reality in the coming years to design high performance computers, supercomputers. Although they are based on old ideas from the field of Computer Architecture, they use the incredible chip design technology to the maximum. Current LLMs with a high number of parameters need a huge number of operations to train the networks and then to make inferences. To reduce the execution time and the energy associated with these operations, formats for representing network parameters that use fewer bits have appeared. These short representations allow reducing the enormous memory needed to store the billions of parameters of current and future neural networks. In short, we need to do research, more than ever, in the software-hardware co-design of these architectures, as suggested by the TPC consortium. Finally, we will explain how the fact that the architectures use the RISC V philosophy makes the designs more tailored to the needs and more innovative.