The keynote speech given by Dr. Jack Dongarra at HPC Asia 2009 examines the history of high performance computing (HPC) from its beginnings in the 1950s, through the present, and into the very near future. The talk also takes an in-depth look at the TOP500 supercomputing list which was begun by Dr. Dongarra and several of his colleagues in 1993.
This extremely educational and enlightening talk also takes a look at current trends in HPC such as “many-core” chips and GPUs as well as examines future obstacles in the ongoing development of HPC.
Part1 | Part2 | Part3 | Part4 | Part5 | Part6 | Part7 [PDF Download] [Video]
Part 2ˇGTOP500: Computational Growth 1993~2019
This is a snapshot of supercomputing that began in 1993 when we first began collecting the information. What we have here are three sets of data for each list. The first set of data is this red line here in the middle which charts the machine at the #1 position on the list. That’s the fastest computer in the world today and that’s at 1.1 PFlop/s. You can also see what happened over time. Back here in 1993, the fastest supercomputer was roughly 60 GFlop/s. This green line here at the bottom charts the machine at the bottom of the list. This is the machine that just barely made it on to the list—“machine #500” if you will, and today that’s at 12.6 TFlop/s. Back here in 1993, the machine lowest on the list was at 400 MFlop/s. Then there’s this blue line at the top of the chart which charts the sum total of the 500 machines. It’s an artificial number that is derived from an Excel spreadsheet and it shows the total sum of the 500 computers and it examines the overall performance that they have. Today that total sum performance for the 500 machines as its 17 PFlop/s. Back in 1993, the sum was at 1.1 TFlop/s.
So there are many interesting things to come out of this list. Notice that this line at the bottom of the list, the one that represents the lowest-ranked machine which made it onto the list, is pretty straight, as is the line that shows the sum, but notice here that the #1 machine has a lot of jumps in it. What this means is that, every so often, we get a machine that is a lot faster than the previous one, and it becomes #1 by a large margin. The other thing that’s interesting here is the slope of the line for the #1 machine. Moore’s Law says that something is going to double every 18 months so the slope should be doubling every 18 months. Well, it turns out the slope is doubling every 14 months instead. This means that it’s being accelerated and parallel computing is causing this acceleration. What this means is that we are using more and more processors every time. Moore’s Law talks about using only a single chip.
There are many interesting features that appear on this list. The machine at the bottom of the list is at about 12.6 TFlop/s. Notice that not too long ago, back in 1997, that was the total of all the machines on the list. Another interesting thing is that going from position #1 to falling off the list takes about 6 to 8 years. These high performance machines have a very short life span. One day you’re the fastest machine on the list and then six years later, you no longer exist on the list itself.
The other thing that I find rather striking is how rapidly things change. When change affects the things that I touch every day, I sometimes have a hard time understanding where we’re going in the future. For example, this laptop that I’m using right now has an Intel Duo core processor. If I ran the benchmark on it, it would run at over a GFlop which is a very impressive number! Now, if you had told me a few years ago that I would have a laptop today that would run at the speed of a GFlop, I wouldn’t have believed you. That would’ve been a joke. Well, as it turns out, not too long ago this computer that I’m using right here today would have been on the TOP500 list. Again, we have these tremendous changes that are taking place in a relatively short period of time!
This is looking at that same data we were looking at just a moment ago which we’re going to extrapolate from. If we look closely, you’ll notice a few things that are quite striking. First off, were at a PFlop today. In order to make that PFlop machine run, it requires on the order of 106 threads of execution. I’ll be more specific about what that machine looks like in a moment. So what’s interesting is that we reached the threshold of a TFlop 11 years ago. If we go back 22 years, we had machines that were at a GFlop. So there’s an 11 year cycle here where we are able to make machines that are three orders of a magnitude faster. Given that information, if we think about where things are going in the future, first off, we notice that all the computers on the TOP500 list in 2015 will be at a PFlop. And not too long after that, somewhere around 2019, all the computers on the TOP500 list that will be at an EFlop. We’ll have machines that are operating at 1018 floating points operations per second! Again, a very striking situation! Notice that the number of threads of execution in order for us to reach this point will be approaching a billion threads of execution!In order to use this machine and run “at scale,” or, in other words, for it to run at those rates of execution, we’ll have to have a billion threads of execution in process!