Interview with Gregory Pfister

Reading Gregory Pfister's In Search of Clusters and then talking toWolfpack participants intrigued me with the person who brought together andexplained the ideas necessary to implement clustering. I wondered whether he wasaware of the impact of his book and what he thought about how vendors whereputting his concepts into practice. The following interview resulted from myinterest in this book and its author.

Q: Are you aware that the name Wolfpack isbased on your book?

Yes, Microsoft sent me a T-shirt and a letter letting me knowthat Clusters influenced the name.

Q: When you subtitled your book, TheComing Battle in Lowly Parallel Computing, what did you have in mind?

I was thinking of clusters of commodity PCs vs. large SMP systems(12-, 16-, 32-way). The battle lines are just now being drawn.

Q: Do you see a large market even forWolfpack Phase 1? Is it possible to see a tenfold increase in the quantity ofclusters shipped in the next year?

It's possible, but many vendors will have to participate beforeclusters reach critical mass. Can one large vendor accelerate the process?Absolutely.

Q: How does Intel's four-way Pentium Protake advantage of NT's SMP capabilities?

Going to the Pentium Pro, Intel made several changes thatsignificantly increase the bus capabilities, which is the key limiting factor inSMP. So SMPs made with the Intel bus and the Pentium Pro chipset will be a lotbetter than SMPs made from Pentium chip set. They will be formidable machines.

Q: Will four-way clusters provide a betterprice/performance point than large (8, 16, 32 CPUs) SMP boxes?

Large SMPs cost more to produce than clusters of small SMP boxesand will therefore be more expensive. However, until cluster vendors provide asingle-system image of the cluster, a large SMP will be easier to administerthan a cluster.

Q: As we move toward scaling clusters,will message passing between nodes become more important?

The Message Passing Interface (MPI) is an existing standard thatwould help this effort, but I imagine Microsoft will adopt its own proprietarysolution, Distributed Component Object Model (DCOM).

Q: What are the challenges that faceWolfpack as it moves toward scaling clusters?

The single-system image will be a huge problem to overcome. WillNT support a clustered file system to let a user view a set of files that arethe sum of all nodes in the cluster? Will communication from outside the clusterhave to know the address of an individual node, or will a message be able topoint to a cluster? Will network management systems see the cluster as a singleelement or as multiple separate computers? A whole lot of these services willhave to be available before an administrator will say that a cluster appears asa single computing resource in the same way a large parallel system appears. Thecompany that provides that stuff first and on the best price/performancehardware will be a winner.

Q: Is everything coming together now thatwill let clustering become commonplace?

Possibly. There's no new technology necessary to do this. Thetechnology exists. What's necessary is the will and the vision to do it.