(Transcript of "The Screen Savers" on Tech TV --- This episode was about FreeBSD clustering. Participants are host Leo LaPorte, Matt Olander, and Brooks Davis. Note that it's occasionally difficult to transcribe fast moving television "talk show" dialogue into good written English, but maybe I can have a good grade for trying?) LL: We get this question a lot --- people wanna do, clusters. Sometimes they say a "Beowulf" cluster. What is it, how do you use it? Our next guests are *experts* in clusters, they build them and they manage them, respectively and they're both part of the FreeBSD project ... First, Chief Technology Officer at "Off My Server", Matt Olander --- we've had you on the show, welcome back Matt, and government systems engineer Brook Davis it's great - Brooks, great to have you here. Now, Brooks is uh, you told me Matt, one of the foremost experts in clustering on FBSD, MO: Yeah, absolutely... LL: Maybe *the guy* ... MO: He is *the guy* for FreeBSD clustering. LL: And you build these for clients... MO: We sure do. Ah, like Sandia, Los Alamos, a lot of big government agencies. LL: Brooks said I shouldn't use the term "Beowulf"? BD: Well, you can use the term "Beowulf" it's certainly a popular way to describe it, but... LL: Yeah, it's how a lot of people *know* it, but... BD: It's very vague. LL: It's not really ... there's no such thing, uh, essentially, right? Or, there is... BD: It describes very little. It's a... LL: Very, all right. BD: So what is a cluster, then Matt, what is, I won't use the word Beowulf, what is a cluster? MO: Right, a cluster's just a group of computers, y'know running parallel processing so they can all be used together. All the CPU's can be used together ... Now there's... LL: It's just basically a massively parallel computer... MO: Yeah, a supercomputer, like those old Crays, right? LL: Right. MO: Now there's more than one type of cluster, you know, there's high availability, and, you know, other types of clusters, but we're talking about getting 'em all working together and throwing all that processing power on one task or job... LL: Typically engineering tasks, uh, movie rendering ... MO: Right, graphics rendering... LL: Graphics, that type of thing. How is that different from a render farm, is it basically the same thing? BD: A render farm would be a type of cluster... MO: Yeah. LL: It's just a kind of cluster. So this is a simple cluster you built here. Tell us a little bit about this Matt, what's in this cluster? MO: This is all off the shelf components, Pentium IV 1.8, 256 Megs... LL: There's four of those... MO: Yeah, there's four of those. They're in a 8 inch custom, y'know 1U chassis but these are very inexpensive. And you... LL: How much for each of these units? MO: Like $700.00 ... LL: It's inexpensive ... MO: Very cheap. LL: But by multiplying them together times four, we get a much faster computer. NIL Absolutely. LL: This is a big network box, they're connected via, by network cables? MO: Just a regular 10/100 switch... LL: So there's nothing special... MO: Nothing special; you can do this at home. I tell my friends, "Don't throw away those old computers, put 'em together and, you know, run SETI on 'em...." LL: Now, uh, we said that there's a, kind of, 3 cluster nodes and then there's a master node, is that, how it's kinda configured here? BD: Yeah, there a master node. It's also acting as a computing node so it's very much like the other cluster nodes... LL: So, so, all four of them are working, but one of them kind of runs the show... MO: The "front end", yeah... LL: It says, "OK, you do this, you do this" --- that kind of thing? BD: Some of that, it ah, also provides the home directories on all the machines; so they're shared --- LL: So it's the file server as well. BD: File server. LL: You have to have a central place that they put the data, obviously. Do they each have their own home directory, or is it all one home directory? BD: It's all shared from the one machine... LL: It's all shared.. BD: ... it's possible to run it the other way, but it's _much_ easier if your home directory is shared. The applications ... LL: A single home directory. BD: ... often assume that you're going to share some home, some space... LL: How 'bout page files, are they, uh, does each box has its own swap file? BD: Yeah. its own swapfile. LL: And, so, its own operating system, its own swapfile, its own RAM, its own CPU, it's as if it's a, it *is* a completely ... MO: Seperate computer. LL: Standalone computer... BD: Yeah. LL: What is the software that ties it together to make them act as one? BD: Um, the software for the particular application we're running today is, ah, software called the Message Passing Interface, uh, MPI... LL: MPI, okay. BD: ... we're running the LAM implementation of that; and then we're monitoring with a system called the Ganglia cluster tool kit. LL: Okay. Why would I use this instead of a four-way system, a four-way multiprocessor system? BD: Well, typically, ah, if you were, saying a *two-way* system, probably go with the two-way. But the four-way --- four-way systems tend to cost more than 4 processors would by themselves... LL: ** (So the real issue is it cost-effective?) (So it really saves, it's cost-effective...) BD: More importantly, though, is if you're building a hundred-way; there are ... LL: There's no way to do it. BD: There's only one hundred-way out there; you could buy a Sun Enterprise, LL: Yeah, I got it. BD: uh, 15K, but that's it.... LL: How, how many processors could I put on this thing? BD: Um, there are systems out there with over three thousand at the government labs ... MO: Lots and lots... LL: _Three_ *thousand* ... whoa, that's amazing.... Now, you're running some software here...this is (on?) a web browser though, this is not part of the cluster, this is just an interface to the cluster's software? MO: This is running on that node that's, that we consider the "front end" ... LL: Master..OK. BD: It's displaying load on the machines, how much memory we're using, network bandwidth usage... LL: OK. BD: and it also kinda some charts, different graphs for each machine in addition to the cluster as a whole...and the color tells you how loaded they are and gives you some monitoring information. LL: Now, we're running something on here, right --- what are you doing, what work are you doing? BD: We're running the high performance LinPack Benchmark... LL: So, it's a benchmark... BD: It's a benchmark; it's the benchmark used for the top 500 clusters list. It's not the greatest benchmark in the world, but it's a benchmark, and, um.... LL: Is it doing math? What is it doing? BD: It's doing linear algebra on a ten thousand by ten thousand matrix... LL: My brain'd be smoking. Wow, that's a big matrix. BD: Using (looks) most of a gigabyte of RAM on the cluster. LL: Is it doing them right now? BD: Yeah, it's running right now. We just finished a run, um, just under three Gigaflops... LL: So, that's about what you're getting, that's three billion floating point operations per second. BD: Yeah. LL: Where is that on the scale of computers? I mean, what is my Pentium IV doing, one gigaflop, maybe? MO: I don't even know if it would... BD: It ah, one ... it *might* be one. MO: *About* one. LL: *At most* one... MO: At most.... BD: A high-end Itanium might, might do more than one, but, um... LL: But nor is this a Cray or a supercomputer 'cause you only have four nodes on here... MO: Yeah, right, but you can _add_ to this --- y'know as ah, your Grandma gives you her computer, your dad... LL: So I, uh, they don't have to be all the same ... MO: No, they absolutely don't... LL: I could put a 386 or a 486? MO: Yeah, you bet. LL: And, so, then you're just kind of accreting power as you go along .... MO: And then you _grow_ the cluster... LL: Interesting. And the total cost of this four unit was, about --- under three grand, would you say? MO: Oh, yeah, easily. LL: What do these things do? MO: Those are the "computation antennae".... LL: Ha ha, they're the horns. That's the FreeBSD horns. You don't put them on every unit, though, do you? MO: No, no... LL: I like that, Matt, though; if I were your client, I'd order them ... MO: You'd want those, right .... LL: So, I would guess Pixar uses something like this, I would guess LucasFilm... MO: Well, the movie "the Matrix" was rendered on a FreeBSD cluster farm ... LL: Interesting. MO: by Manex, locally here, just like this... LL: Interesting, just like this... MO: with a lot more. LL: Yeah, 'cause you don't wanna --- you've got to get the movie *out* in a few years, you can't ... MO: Exactly. LL: ... you can't wait a decade... MO: Eventually... LL: ... and the government might use this for weather prediction, that kind of thing... BD: Weather prediction, simulating nuclear tests... LL: Anything really complex... BD: ... yeah, system design, and all sorts of things. LL: Cool. Well; we can't tell you what Brooks does; it's highly confidential, but we're glad he's here doing it, on the "Screen Savers"... thank you, Brooks! BD: (Sure)** LL: Matt, it's great to see you again ... MO: Thanks for having us! LL: We *finally* got a cluster on! If you wanna know more about this, you can take a look at the software used, we have links to the software used and an article about how to do it, and the management console as well ... go to "thescreensavers.com", all the specs, too, for this, inexpensive system giving you some pretty good power....