What do an astrophysicist studying how black holes merge, a chemist predicting the biological characteristics of novel proteins, a psychologist investigating decision-making and a mechanical engineer studying how nanotubes transport heat have in common?
They all need tremendous amounts of computer power to conduct their research and they all will benefit directly from a huge boost to Vanderbilt’s supercomputing power.
The surge will be achieved by supplementing the central processing units (CPUs) in the Advanced Computing Center for Research & Education (ACCRE) with a cluster of graphic processing units. Adding GPUs – microprocessors found on graphics cards that manage video displays – to the existing banks of CPUs, like those that power personal computers, will boost the power of the campus supercomputing from trillions of computer operations per second (TeraFLOPS) to a level of thousands of trillions of computer operations per second (PetaFLOPS).
According to Greg Walker, the associate professor of mechanical engineering who is the project’s principal investigator, GPUs have a totally different architecture than CPUs. CPUs are designed to execute a large number of different tasks. Graphics processors, on the other hand, are designed to execute a more limited set of tasks but to do so very quickly and with a high level of concurrency. “This makes them ideal for a broad class of scientific problems, called n-body problems, characterized by large numbers of objects operated upon by a common set of physical rules,” he says.
The new cluster will be constructed with a $390,000 American Recovery and Reinvestment Act grant from the National Science Foundation and with $150,000 in matching support from the university administration.
ACCRE, which is located in the Hill Center on campus, currently consists of 3,000 processing cores that are linked together with a total capability of 17.5 TeraFLOPS. Using an approach called parallel processing, programmers break complex programs down into a number of small modules that are fed simultaneously to different processors. This approach can make short work of simulations that would take months or years if run on a single processor. For certain types of scientific simulations, GPU clusters provide processing power at a lower price and use less energy than equivalent CPU arrays. Although GPUs have been around for a long time, it is only recently that scientists have begun taking them seriously as a problem-solving tool.
“The GPU cluster represents a new technology for ACCRE,” says Alan Tackett, ACCRE’s technical director. “It has the potential to transform the kinds of scientific problems some of our best researchers can address. But what I like most about the cluster is that it was conceived and acquired through a grassroots effort by researchers who knew what the technology could do and understood that they could use it to address new questions at the forefront of their fields. To me, that is what ACCRE is all about.”
N-body problems involve a group of objects that are interacting through a force field like gravity. The problem is to calculate the motions of all the objects given a set of initial positions and velocities. Solving this problem for two objects is simple. But, despite the best efforts of a number of famous scientists and mathematicians including Newton, Euler, Lagrange and Poincaré, no one has come up with a general mathematical solution for the problem involving three bodies, let alone dozens, thousands, millions or more. So, scientists have turned to computer simulations to produce approximate solutions to n-body problems, like the motion of stars in a galaxy, the activity of neurons in the brain, the structure of proteins in cells or the movement of sound waves in carbon nanotubes.
When it is completed, the GPU cluster will benefit a number of different research projects. It will allow:
- Associate Professor of Mechanical Engineering Greg Walker, who is studying the way that carbon nanotubes conduct heat, to simulate this process in enough detail to predict the thermal properties of nanomaterials being developed for military, aerospace and energy applications with greater accuracy.
- Assistant Professor of Physics and Astronomy Kelly Holley-Bockelmann to produce more accurate simulations of the process by which star-sized black holes merge to form the supermassive black holes that sit at the center of many galaxies.
- Assistant Professor of Chemistry Jens Meiler, who has developed a “machine learning” method for predicting the biological activity of novel chemicals with the potential for speeding the drug discovery process, to predict the biological activity of novel chemicals 100 times faster.
- Associate Professor of Psychology Thomas Palmeri and Assistant Professor of Psychology Sean Polyn to develop and test simulations of how the brain learns, remembers and decides in unprecedented detail.
Before the researchers can realize these benefits, however, they must substantially restructure their basic programs. The Vanderbilt administration recognized the challenge that this represents and has awarded the participating researchers with a $138,000 grant to cover a portion of the salaries of several ACCRE staff programmers, freeing them to work on this reprogramming effort. The grant was awarded by the university’s Innovation and Discovery in Engineering and Science (IDEAS) program that invests institutional funds to “significantly advance a novel idea or eliminate a barrier to the solution of an important problem.”
The GPU cluster will also play an important role in ongoing efforts at Vanderbilt to revitalize undergraduate and graduate education in computational science for scientists and engineers by the addition of a new minor degree on the topic that will include GPU programming. In addition, Vanderbilt will become a regional hub for the Virtual School of Computational Science and Engineering offered by the Great Lakes Consortium for Petascale Computing. This will bring interested scientists and engineers to Vanderbilt from the Southeastern United States to take Great Lakes Consortium courses.