Center for the Physics of Information
The Impending Overthrow of the Silicon Monoply: Revolutionary Substrates Unite!
A Conversation with André DeHon, John Preskill, and David Rutledge
Silicon is a superb computational substrate...but sooner or later it will run out of room. The CPI is devoted to inventing the new computational substrates, architectures, and algorithms for the computing devices of the future.
PRESKILL: We're kind of an odd mix of people, you know. I'm a theoretical physicist, Dave's an electrical engineer, André is a computer scientist. But I think we have some things in common. In those areas of overlap there's a potential for some really exciting scientific and technological developments. We know that the advance of our information technology, which has been dazzling for so long, is confronting limitations that come from physics and, in particular, from the size of atoms. And we don't know beyond say, a decade, what we are going to do to continue the type of progress we've gotten accustomed to. It's going to require really new ideas. We don't know what. We don't know how we're going to get there. And that's what we're going to be thinking about in this Center. There are a lot of ideas about exotic ways of manipulating information, but there's a tremendous gulf between some of those concepts and practice. In particular, I'm interested in quantum computing. If it comes to fruition, we'll see an amazing advance in the speed of computation. It's really exciting. We have these beautiful theoretical ideas about quantum computing, but we really don't have any definite idea about how to progress along the road that will lead us to advanced quantum computing.
RUTLEDGE: One thing that I think is interesting about the Center is its ancestry, so to speak. Caltech has a very good history of making fundamental contributions to the physics of small things and information. Three people that come to mind are Richard Feynman, John Hopfield, and Carver Mead. There's a great tradition. But recently Caltech has hired many outstanding junior faculty in different departments across the campus who are connected to this area. That's really Caltech's advantage.
We have the opportunity here to take some of the ideas being developed on the scientific, physics side to see if they really work in engineering products. That would require, for example, getting some of the ideas to work on a silicon integrated circuit. This vertical integration—from the theoretical up through the practical—will mean strong collaboration between scientists and engineers to get really neat scientific ideas transformed into practical devices....the advance of our information technology, which has been dazzling for so long, is confronting limitations that come from physics...
DEHON: I think vertical integration on a higher level also means we'll be rethinking abstractions at many layers. Presently, we've got a very well-developed set of abstractions for designing computers and software on top of silicon. And we know "this is where we collapse into the gate level; this is where we build up some architectures on top of that; here is where we build the program; and then there are algorithms on top of that." There is a nice set of defined layers. On the other hand, when the rules change, the costs change, and really good engineers will be the ones saying, "Okay, these old abstractions are getting in my way." What's very clear here is that using some of the same interfaces and abstractions we have in the past will defeat the purpose. Silicon's been very reliable; things work because we're talking about a million atoms sitting in one place. But it's not clear whether we'll have that type of control with substrates where we will be working with individual (or very few) atoms. So that's going to force us to re-evaluate all of our models: what you use for computation, the programming language, and so on.
ENGENIOUS: So for instance, algorithms might not sit so high in the hierarchy any more? They might be more embedded in the fundamental substrate itself in some way?
DEHON: I would believe that.
RUTLEDGE: Also, with smaller numbers of atoms, you really have to deal with errors in a fundamental way...
DEHON: ...because there are some things that may be less hidden. One of the things you try to do in good engineering abstraction is hide unnecessary detail and bring up the dominant effects you need to optimize. I think the dominant effects are probably going to shift and change. There are different things we'll need to bring to the attention of the engineer....really good engineers will be the ones saying, "Okay, these old abstractions are getting in my way."
PRESKILL: Maybe the concept of a general-purpose device will be less central than it was in the past. Some physical systems may be better suited for certain applications than others. We should be willing to let blur those layers which had served us very well in the past—substrate, architecture, and algorithm—and to think things through from the start. Error correction is probably the best example. In quantum computing, this area is one of my major interests. For instance, we had to rethink what type of physical system would potentially be very resistant to errors. Some technologies with lots of good features may fail in that regard. So quantum computing just won't be a possibility for certain types of physical applications.
DEHON: The deeper I got into the VLSI work I started out with, the more I began to really understand that the underlying physics of the substrate was inseparable from the most efficient architecture possible, and the eventual implementation. And as VLSI got smaller, the landscape changed. Wires got more expensive, for instance. Ultimately, our computations do depend on the physics we use and the structure of the physical world. After looking down at VLSI for so long, it's good to just look up and realize scientists are working with som e amazing new phenomena: carbon nanotubes, experiments trapping a single atom. So you say to yourself, "How can we harness these things?"
For me, a central issue is understanding computational cost structure. When the cost structure changes things radically, the nature of the solutions changes as well. The general-purpose processor that made a lot of sense in VLSI just doesn't make sense for these new things. We are off in a completely new playground, which is very exciting for an architect. Caltech is a place that allows me to think sometimes at the circuit level, sometimes at the manufacturing level, sometimes at a mathematical/statistical yield level—all over the map. And for something new like this, where no established discipline exists, it's important to gather people from various areas who can think broadly about the issues. This is what the CPI will accomplish.
PRESKILL: We're searching for new paradigms, something that Caltech does especially well. Maybe we won't be the place that actually builds the next revolutionary generation of devices, but I think what we should aspire to is becoming the world's leading institution for laying the scientific foundations which will be the basis for information technologies of the future—we will be generating absolutely new ideas. And training students so they have the broad background that's necessary to get the big picture.
ENGENIOUS: How will the structure of the Center facilitate breakthroughs?
RUTLEDGE: We're interested in creating an environment conducive to professor and student interaction. And we're anticipating that there will be a new Information Science and Technology building as a result of the fundraising campaign. University professors are prone to being trapped in an area; this is a good way to force them out into new things....with smaller numbers of atoms, you really have to deal with errors in a fundamental way...
DEHON: People like Bill Dally [PhD '86; now Professor of Electrical Engineering and Computer Science at Stanford University] and others came to Caltech in the early '80s because it was the place for VLSI. And that's really what we want—for Caltech to be the place for the next revolution in novel computing. There's a great deal of uncertainty about what's going to happen in this area, and yet that's what makes it exciting. What's going to happen at the chemical level? At the biomolecular level? At the quantum level?
Look at this from a student's perspective. I maintain that our current and future students will go out into the world and have the same impact the Caltech VLSI students are having now—maybe even more so if we can get students from every area to interact with each other. For example, a student comes here to study molecular electronics, but this area doesn't exactly pan out. However, the real benefit will have come from interacting closely with other people doing perhaps biomolecular and quantum work, and from being taught how to think broadly about these areas. I think our students will certainly be in a position to found, transform, and lead the industry.
PRESKILL: The students are really the key. Caltech should be the place, the number one place, that a student thinks of if he or she is interested in the future of information technology in the long-term. Actually André and Erik [Winfree] did a great thing this summer—they were involved in the Computing Beyond Silicon Summer School, which attracted people from all over.
DEHON: We had 45 students for four weeks and 12 guest lecturers—the top people—coming from different institutions and intellectual areas. It was really something.
ENGENIOUS: How did the students deal with this new conceptual framework?
DEHON: It was interesting because it's not a "done thing," there is no orthodoxy. The students definitely went through a little mind expansion. There were EE students who thought [the EE framework] was the only way the world works...and in some cases biology students who didn't at the outset realize that maybe computational complexity meant something to them. All of them were challenged and out of their comfort zones. I think many of them had the experience of "Wow, the world is bigger than I thought it was." There is an opportunity to do interesting work at, for instance, the intersection of computer science and biology.It's so important to have the freedom to be daring...
PRESKILL: And in some ways, it's easier for students than it is for us, you know. For me, the work I do at the interface of physics and information science seems kind of "out there," novel and daring. But to my students, it seems very natural. Those are the things they're interested in. Combining computer science and physics is second nature for them.
ENGENIOUS: Caltech seems to have both a deep intellectual reservoir and a smallness of size that allows us to attack these problems much differently than anybody else. Are there other universities that can do what you anticipate doing?
RUTLEDGE: Smallness is a part of it. Caltech feels the same size as the entire EE department at Berkeley. There, someone "far away" from you intellectually meant someone that was making superconducting detectors in the electronics department. However, there are a lot of good places out there, and a lot of competition.
DEHON: Certainly MIT has the breadth. On the other hand, it's a big place—with something happening in the Media Lab, and then there are people over in the AI Lab, far from folks in Mechanical Engineering. So you know, maybe it's a little bit harder to get coherence between the groups.
ENGENIOUS: What is the one thing that excites you most about the Center?
PRESKILL: Well, from my own parochial point of view, I'm excited about making quantum computers a reality. It's just one of the emerging frontiers. If something like the Center for the Physics of Infor-mation can make that possible, I think that's very exciting.
DEHON: The Center will really allow us the opportunity to build critical intellectual mass. My students and I can sit there and ask each other questions, but having the ability to work with people from other areas thinking about the same problems will be powerful. The new solutions will create new abstraction hierarchies and new ways of decomposing problems. Things will not be the same as they were. Let's think out of that proverbial box and come up with some wild ideas.We are off in a completely new playground...
RUTLEDGE: I see two things. One is the opportunity to work with people across a wide range of disciplines in a serious way. And the second is consistent support. I've run government centers, and it's astonishing how much of your life gets taken up by requirements and crazy things that change right in the middle of established projects. Just to get out of the kind of environment where you're told what to do every two months is liberating.
PRESKILL: Absolutely. It's so important to have the freedom to be daring, not to have to defend the project on the basis of some short-term goal, some milestone event.
RUTLEDGE: I want to mention that junior faculty will be instrumental; they have already contributed in fundamental ways to getting things started. People like Erik Winfree [PhD '98], Ali Hajimiri, Hideo Mabuchi [PhD '98], André of course, and a handful of others.
PRESKILL: Yes, I think that's pretty good evidence that we're on the right track. Looking around campus and seeing so many young faculty involved in exciting projects at the interface of physical science and information science tells me that we are in a good position to live up to the legacy of Feynman, Mead, and Hopfield.
André DeHon is Assistant Professor of Computer Science. John P. Preskill is the John D. MacArthur Professor of Theoretical Physics. David B. Rutledge is the Kiyo and Eiko Tomiyasu Professor of Electrical Engineering.