diff options
| author | Murray Stokely <murray@FreeBSD.org> | 2010-01-30 21:45:08 +0000 |
|---|---|---|
| committer | Murray Stokely <murray@FreeBSD.org> | 2010-01-30 21:45:08 +0000 |
| commit | cf5068ab40e09fed1688b8db7c4a66889360a440 (patch) | |
| tree | 27299e7c33d3af949a74ebaeb371725df9309712 | |
| parent | c33944681817e13ed9a6e993d306ff0c328cfee6 (diff) | |
| download | doc-cf5068ab40e09fed1688b8db7c4a66889360a440.tar.gz doc-cf5068ab40e09fed1688b8db7c4a66889360a440.zip | |
Add English language transcripts to 7 of the videos currently from the
'BSD Conferences' YouTube channel.
In addition to the obvious benefits to the hearing impaired, these
files are being checked in here to :
1. Enable additional human editing of the technical content to improve
the transcriptions.
2. Enable the translation teams to more easily track changes and make
translated closed captions of the videos.
3. Provide a centralized source for audio/video transcripts separate
from the particular YouTube hosting solution that is currently
used.
4. Improve the access to this content for search engines and web users.
Transcripts made with the help of YouTube Machine Translation and then
human editors hired through Amazon Mechanical Turk to improve the
transcriptions further.
Sponsored by: FreeBSD Foundation
Notes
Notes:
svn path=/head/; revision=35279
8 files changed, 18742 insertions, 0 deletions
diff --git a/en_US.ISO8859-1/captions/2006/mckusick-kernelinternals/mckusick-kernelinternals-1.sbv b/en_US.ISO8859-1/captions/2006/mckusick-kernelinternals/mckusick-kernelinternals-1.sbv new file mode 100644 index 0000000000..7f564d1d33 --- /dev/null +++ b/en_US.ISO8859-1/captions/2006/mckusick-kernelinternals/mckusick-kernelinternals-1.sbv @@ -0,0 +1,4302 @@ +0:00:09.469,0:00:11.309 +Hello my name is Marshall Kirk McKusick + +0:00:11.309,0:00:15.389 +and I've been around as long as dinosaurs +and mainframes have ruled the world + +0:00:15.389,0:00:18.429 +which is to say the sixties and seventies + +0:00:18.429,0:00:22.460 +however by 1970s a new breed of mammals had began to show up on +the scene, + +0:00:22.460,0:00:24.240 +known as mini computers + +0:00:24.240,0:00:28.230 +although they were just toys in the 1970s they would soon grow + +0:00:28.230,0:00:31.689 +and take over most of the computing market + +0:00:31.689,0:00:33.150 +In 1970 + +0:00:33.150,0:00:37.910 +at AT&T Bell laboratories two researchers Ken +Thompson and Dennis Ritchie began developing the + +0:00:37.910,0:00:39.900 +UNIX operating system + +0:00:39.900,0:00:42.040 +Ken Thompson who had been an alumnus at Berkeley + +0:00:42.040,0:00:46.100 +came back on a sabbotical in 1975 bringing UNIX +with him + +0:00:46.100,0:00:47.539 +In the he was there + +0:00:47.539,0:00:51.330 +he managed to get a number of graduate students interested +in UNIX + +0:00:51.330,0:00:53.940 +and by the time he left in 1976 + + +0:00:53.940,0:00:56.829 +Bill Joy has taken over in running the UNIX system + +0:00:56.829,0:01:00.470 +and in fact continuing to develop software for it. + +0:01:00.470,0:01:04.339 +Bill began packaging up the software that have +been developed under Berkeley UNIX and + +0:01:04.339,0:01:05.779 +and distributing it + +0:01:05.779,0:01:08.040 +as the Berkeley Software Distribution + +0:01:08.040,0:01:12.310 +whose name was quickly shortened to simply BSD + +0:01:12.310,0:01:16.330 +BSD continued to be distributed with +yearly distribution for almost fifteen + +0:01:16.330,0:01:17.490 +years + +0:01:17.490,0:01:21.920 +initially under Bill Joy and later under others including +yours truly. + +0:01:21.920,0:01:24.860 +By the late 1980s interest had began to grow + +0:01:24.860,0:01:27.400 +in freely redistributable software + +0:01:27.400,0:01:30.170 +so a number of us at Berkeley began separating +out + +0:01:30.170,0:01:32.649 +the AT&T proprietary bits of BSD + +0:01:32.649,0:01:35.710 +from those parts that were freely redistributable. + +0:01:35.710,0:01:40.590 +By the time of the final distribution at BSD +in 1992 + +0:01:40.590,0:01:43.620 +the entire distribution was freely redistributable. + +0:01:43.620,0:01:45.909 +I live in a capsule history here + +0:01:45.909,0:01:48.009 +but if you're interested in the entire story + +0:01:48.009,0:01:50.789 +I have this three-and-an-half hour epic + +0:01:50.789,0:01:54.590 +which is available from my website www.mckusick.com + +0:01:54.590,0:01:58.200 +that gives the entire history of Berkeley. + +0:01:58.200,0:02:00.239 +Following the final distribution from Berkeley + +0:02:00.239,0:02:01.450 +two groups sprung up + +0:02:01.450,0:02:03.600 +to continue supporting BSD + +0:02:03.600,0:02:08.080 +the first of this is the NetBSD whose primary +goal was to support + +0:02:08.080,0:02:10.459 +as many different architectures as possible + +0:02:10.459,0:02:14.769 +everything from your microwave oven all way +upto your cray XMP + +0:02:14.769,0:02:19.409 +In fact today NetBSD supports nearly +sixty architectures. + +0:02:19.409,0:02:22.419 +The other group that sprang up was FreeBSD. + +0:02:22.419,0:02:28.239 +Their goal was to bring up BSD and support +as wide set of devices as possible on the + +0:02:28.239,0:02:29.719 +PC architecture. + +0:02:29.719,0:02:36.549 +They also had a goal of trying to make the +the system as easy to install as possible to + +0:02:36.549,0:02:39.309 +attract by a wide group of developers + +0:02:39.309,0:02:42.319 +I chose to primarily work with the FreeBSD +group + +0:02:42.319,0:02:43.740 +both doing software + +0:02:43.740,0:02:46.140 +and also together with George Neville + +0:02:46.140,0:02:51.069 +writing this book ""The Design and Implementation +of the FreeBSD Operating System"". + +0:02:51.069,0:02:52.060 +Together with this book + +0:02:52.060,0:02:53.959 +I developed a course + +0:02:53.959,0:02:56.500 +which runs for twelve chapters + +0:02:56.500,0:02:58.179 +and thirty hours. + +0:02:58.179,0:02:59.749 +The purpose of this video + +0:02:59.749,0:03:01.089 +is to give you a taste + +0:03:01.089,0:03:02.819 +of that course. + +0:03:02.819,0:03:07.249 +What follows are excerpts from the first lecture +of the course + +0:03:07.249,0:03:11.139 +which of course you can also get from my website +www.mckusick.com. + +0:03:11.139,0:03:13.069 + + +0:03:13.069,0:03:17.739 +Enjoy. + +0:03:17.739,0:03:22.239 +This class is nominally about FreeBSD +because well + +0:03:22.239,0:03:26.379 +that's what I know best and that's what +the textbook is organized around + +0:03:26.379,0:03:29.979 +but the fact that the matter is back it's really + +0:03:29.979,0:03:32.339 +a class about your UNIX and that + +0:03:32.339,0:03:36.539 +really covers sort of the broad range of things +in the open source arena as its FreeBSD + +0:03:36.539,0:03:37.689 +in Linux + +0:03:37.689,0:03:38.899 +which of course + +0:03:38.899,0:03:41.159 +you use a lot out + +0:03:41.159,0:03:41.550 +and + +0:03:41.550,0:03:44.349 +it also covers a commercial systems + +0:03:44.349,0:03:46.950 +%uh Solaris, HP-UX, + +0:03:46.950,0:03:49.279 +AIX and so on. + +0:03:49.279,0:03:52.419 +I am going to tend more towards the open +side + +0:03:52.419,0:03:56.389 +open source side of things.So it's really +going to be more FreeBSD in Linux than it's + +0:03:56.389,0:03:57.579 +going to be + +0:03:57.579,0:04:00.849 +Solaris and HP-UX and so on. + +0:04:00.849,0:04:06.959 +For the most part at the level of this course +we're dealing with the interfaces to the system + +0:04:06.959,0:04:07.329 +and + +0:04:07.329,0:04:11.599 +the fact that the matter is a those interfaces are highly +standardized at this point + +0:04:11.599,0:04:12.060 +and + +0:04:12.060,0:04:15.280 +whether it's FreeBSD or Linux or Solaris +or whatever + +0:04:15.280,0:04:19.460 +the Socket system call has to do the same +thing, it has to have the same arguments + +0:04:19.460,0:04:20.150 +in that, + +0:04:20.150,0:04:23.909 +it has to have the same effect + +0:04:23.909,0:04:27.319 +and so until you get down to the really niddy +details + +0:04:27.319,0:04:29.600 +of how they actually go about implementing +that + +0:04:29.600,0:04:31.960 +the differences are relatively minor. + +0:04:31.960,0:04:35.830 +So I would say that sixty to seventy percent +of the material that I'm covering + +0:04:35.830,0:04:40.779 +is just as true for FreeBSD as it would +be for Linux + +0:04:40.779,0:04:42.580 +or for Solaris + +0:04:42.580,0:04:44.659 +%uh AIX is a little bit + +0:04:44.659,0:04:45.629 +sort of off in the weeds + +0:04:45.629,0:04:48.709 +%uh as is HP-UX + +0:04:48.709,0:04:51.099 +but luckily we don't have to worry too much about +that. + +0:04:51.099,0:04:54.569 +Okay so + +0:04:54.569,0:04:59.279 +the other thing is I'm going to assume that +all of you have used the system. I get + +0:04:59.279,0:05:00.910 +really sort of worried when people + +0:05:00.910,0:05:04.249 +you know raise the hands and ""Hey, what's a Shell?"" + +0:05:04.249,0:05:07.990 +or I don't +put a lot of code up but a small piece of code and someone said ""Why + +0:05:07.990,0:05:11.819 +are there two pipe symbols in the middle of +that that If statement?"". + +0:05:11.819,0:05:15.740 +No we're not programming the Shell we're programming +in C. + +0:05:15.740,0:05:19.970 +So hopefully you can tell the difference between +Shell scripts and C code. + +0:05:19.970,0:05:21.990 +okay but I am but am gonna assume + +0:05:21.990,0:05:24.610 +you haven't really looked inside the system. + +0:05:24.610,0:05:28.289 +So I gonna start everything to at a very +high level. + +0:05:28.289,0:05:32.969 +The problem is I have already discovered you come +from a lot of different sort of + +0:05:32.969,0:05:33.819 +backgrounds + +0:05:33.819,0:05:35.180 +and + +0:05:35.180,0:05:36.280 +levels of knowledge + +0:05:36.280,0:05:37.900 +and so + +0:05:37.900,0:05:42.620 +the way that I find works best the sort of +be useful to everybody is that three pass + +0:05:42.620,0:05:43.860 +algorithm + +0:05:43.860,0:05:49.060 +so what I will do is start the first pass a very +broad brush high level + +0:05:49.060,0:05:50.569 +description of what's going on + +0:05:50.569,0:05:54.719 +and then I will go back and i'll go through the +same material again but at a lower level of + +0:05:54.719,0:05:55.300 +detail + +0:05:55.300,0:05:59.939 +then i finally go back and go through a very neatly +low-level of detail + +0:05:59.939,0:06:04.649 +and the fact that is if you are learning new stuff +as I'm doing the high-level thing + +0:06:04.649,0:06:08.649 +you are gonna be utterly washed by the time I get to +low level nearly details + +0:06:08.649,0:06:10.699 +but since I'm going to do it topic by topic + +0:06:10.699,0:06:14.190 +when I get to the end of one of those nearly +low level details + +0:06:14.190,0:06:17.900 +i'll give you a clue as i will say ""Brain +reset, I'm starting a new topic"" so even if + +0:06:17.900,0:06:19.330 +you're completely lost + +0:06:19.330,0:06:23.530 +you can now start listening again plus am gonna get +the broad brush up again. + +0:06:23.530,0:06:27.059 +okay and for those of you that know a lot of +this stuff already + +0:06:27.059,0:06:31.770 +you'll probably find the broad brush rather boring + +0:06:31.770,0:06:35.759 +but by the time i get down nearly low level +details I think you'll actually + +0:06:35.759,0:06:37.860 +pick up some things that you will find + +0:06:37.860,0:06:39.710 +useful and interesting. + +0:06:39.710,0:06:43.759 +So in this way hopefully everybody will +get some + +0:06:43.759,0:06:47.699 +useful percentage of material out of the course. + +0:06:47.699,0:06:49.599 +I am gonna start out by just + +0:06:49.599,0:06:53.089 +walking through and giving you the + +0:06:53.089,0:06:56.919 +outline of what we're going to try and do +here + +0:06:56.919,0:07:01.169 +As i said we're going to go roughly + +0:07:01.169,0:07:03.270 +about two-and-an-half hours of lecture + +0:07:03.270,0:07:04.729 +about two hours forty minutes + +0:07:04.729,0:07:06.499 +per week + +0:07:06.499,0:07:07.619 +and + +0:07:07.619,0:07:11.770 +so we will start off this week with an introduction. + +0:07:11.770,0:07:13.860 +This is as I said we're going to start from the +top + +0:07:13.860,0:07:15.749 +and just start working our way down + +0:07:15.749,0:07:19.350 +so the general thing I'm going to do is +to talk about the interface + +0:07:19.350,0:07:21.439 +%uh which is something that you + +0:07:21.439,0:07:25.319 +are presumably fairly familiar with since +you've worked with that system + +0:07:25.319,0:07:27.249 +and then + +0:07:27.249,0:07:29.739 +sort of layout terminology + +0:07:29.739,0:07:32.080 +although we use normal english words + +0:07:32.080,0:07:34.419 +they have + +0:07:34.419,0:07:38.580 +sometimes rather bizarre meanings compared to their +common usage + +0:07:38.580,0:07:39.220 +and + +0:07:39.220,0:07:42.330 +I will just sort of lay out the terminology +lay out the + +0:07:42.330,0:07:45.750 +the way we talk about how the system is structured + +0:07:45.750,0:07:50.780 +and this week we will also talk about the +basic services ""What is it that the kernel is + +0:07:50.780,0:07:52.929 +providing for us?"" + +0:07:52.929,0:07:54.060 +and then of course + +0:07:54.060,0:07:58.499 +we'll proceed to dive down in and and see how +that is done + +0:07:58.499,0:07:59.970 +so here in + +0:07:59.970,0:08:01.400 +Week 2 + +0:08:01.400,0:08:05.450 +%uh we're gonna look at the system from the +perspective of + +0:08:05.450,0:08:07.039 +something that + +0:08:07.039,0:08:08.720 +manages processes. + +0:08:08.720,0:08:12.170 +One way of looking at the kernel it's really +just a + +0:08:12.170,0:08:16.440 +the resource manager and the resource that +is managing are things going to do with processes + +0:08:16.440,0:08:19.460 +So we'll look at a process, what the structure of +it is + +0:08:19.460,0:08:20.649 +%uh and + +0:08:20.649,0:08:23.559 +talk about the different ways that they can +be structured. + +0:08:23.559,0:08:28.379 +Process can for example be an address space +and can have one thread running in it can have + +0:08:28.379,0:08:29.749 +multiple threads running in it. + +0:08:29.749,0:08:34.620 +we'll talk about the different ways +that we think a process is. + +0:08:34.620,0:08:38.480 +We will look at the management of those processes + + +0:08:38.480,0:08:39.239 +we've got + +0:08:39.239,0:08:42.020 +to lay out the bits and pieces that +need to be managed + +0:08:42.020,0:08:44.660 +%uh and then talk about + +0:08:44.660,0:08:47.190 +how we do that. + +0:08:47.190,0:08:51.740 +we'll talk about jails.. this is something +that you currently find only in FreeBSD + +0:08:51.740,0:08:55.060 +hasn't made it into + +0:08:55.060,0:08:56.320 +Linux yet although + +0:08:56.320,0:09:01.630 +%uh the concept is %uh being actively worked +on so my guess is that you'll see that + +0:09:01.630,0:09:03.500 +%uh fairly soon. + +0:09:03.500,0:09:06.360 +%uh will also then talk about scheduling + +0:09:06.360,0:09:10.579 +which is in essence how we decide what gets +to run, when it gets to run, how long it takes + +0:09:10.579,0:09:13.500 +to run, etc. + +0:09:13.500,0:09:14.330 +okay + +0:09:14.330,0:09:19.020 +The week after that we will go into virtual +memory. + +0:09:19.020,0:09:23.800 +Signals aren't really part of virtual memory +but they didn't fit into next week's + +0:09:23.800,0:09:26.400 +material so I just would drop that at the +beginning + +0:09:26.400,0:09:29.850 +but the bulk of Week 3 is going to +be + +0:09:29.850,0:09:32.019 +the management of Virtual Memory. So we've got + +0:09:32.019,0:09:35.119 +a bunch of physical memory, a bunch of +processes that are + +0:09:35.119,0:09:37.940 +trying to use their address spaces + +0:09:37.940,0:09:39.590 +and we will talk about + +0:09:39.590,0:09:41.410 +essentially how you will make all that work + +0:09:41.410,0:09:43.510 +It's called a virtual memory because it's + +0:09:43.510,0:09:47.420 +sort of a cheat. We promise you the world and +then we deliver you + +0:09:47.420,0:09:51.480 +as small number of pages as we think we +can get away with. + +0:09:51.480,0:09:56.420 +Okay. So the first three weeks then essentially +get us through + +0:09:56.420,0:09:58.340 +looking at the world as if it was all + +0:09:58.340,0:10:00.560 +all about processes + +0:10:00.560,0:10:03.880 +and in Week 4 we change gears. we say +okay well you know + +0:10:03.880,0:10:07.570 +the kernel isn't just about processes. You can sort of look at it orthogonally +and you can + +0:10:07.570,0:10:10.000 +say it's really just a giant I/O switch + +0:10:10.000,0:10:12.910 +it's just like a traffic cop that's just managing +these + +0:10:12.910,0:10:14.860 +I/O streams + +0:10:14.860,0:10:15.450 +and + +0:10:15.450,0:10:18.610 +so let's look at it from that perspective. + +0:10:18.610,0:10:19.310 +And + +0:10:19.310,0:10:24.740 +%uh we'll start with special files, again this +where the interface when you talk about UNIX + +0:10:24.740,0:10:25.880 +systems, when you talk about + +0:10:25.880,0:10:27.950 +what's normally /dev + +0:10:27.950,0:10:34.170 +interface that gets you access +to the various I/O streams that are available + +0:10:34.170,0:10:37.220 +and we'll look at how that's organized and +the structure of it + +0:10:37.220,0:10:41.840 +%uh which is to be fairly simple but in the +last decade has gotten + +0:10:41.840,0:10:43.670 +incredibly complicated. + +0:10:43.670,0:10:48.540 +We will also talk about sudo terminals in +job control + +0:10:48.540,0:10:53.330 +this is about as interesting as watching the +grass grow but unfortunately it's + +0:10:53.330,0:10:55.490 +a major component of the system + +0:10:55.490,0:10:59.520 +and especially people that deal with system +administration have to know far more about + +0:10:59.520,0:11:06.520 +this than they probably ever thought they +wanted to. + +0:11:06.900,0:11:11.430 +Okay we will then continue in Week 5 with +the kernel I/O structure, + +0:11:11.430,0:11:16.090 +%uh We will start with multiplexing of I/O. The +kernel has done this + +0:11:16.090,0:11:17.360 +always + +0:11:17.360,0:11:22.110 +but we will really talking more about how do +we export I/O multiplexing to + +0:11:22.110,0:11:25.970 +user applications. + +0:11:25.970,0:11:29.250 +We will then move into auto configuration strategy + +0:11:29.250,0:11:31.370 +Auto configuration + +0:11:31.370,0:11:32.770 +is what happens + +0:11:32.770,0:11:36.619 +and typically or historically I guess you +could say as the system boots. + +0:11:36.619,0:11:39.500 +so all that stuff that comes out about + +0:11:39.500,0:11:40.810 +what + +0:11:40.810,0:11:43.550 +hardwares are on the machine and how it's all +interconnected + +0:11:43.550,0:11:47.350 +all of that is tied up in auto configuration + +0:11:47.350,0:11:50.040 +and that used to happen just once it boots + +0:11:50.040,0:11:52.000 +but in modern systems today + +0:11:52.000,0:11:55.839 +it's an ongoing process. It happens at boot +but it also happens + +0:11:55.839,0:12:00.550 +anytime you plug a new I/O device, a +PCMCIA card, + +0:12:00.550,0:12:03.680 +you remove a disk or you put in a new disk. + +0:12:03.680,0:12:07.010 +Any sort of activity that changes the the I/O + +0:12:07.010,0:12:08.360 +structure of the machine + +0:12:08.360,0:12:10.870 +auto configuration has to fired back up + +0:12:10.870,0:12:13.050 +and figure out what's disappeared + +0:12:13.050,0:12:18.330 +and cleanup and figure out what new has arrived +to conongiure it in. + +0:12:18.330,0:12:19.320 +and then we'll talk + +0:12:19.320,0:12:23.870 +a little bit about the configuration of the +device driver + +0:12:23.870,0:12:27.390 +this actually gets into an area of that + +0:12:27.390,0:12:28.660 +is + +0:12:28.660,0:12:33.440 +one well let me just give it as a bit +of advice to the class esspecially those of + +0:12:33.440,0:12:36.780 +you who work in system administration. + +0:12:36.780,0:12:42.010 +You'd really want to be careful that +you don't learn too much about device drivers + +0:12:42.010,0:12:44.670 +because there is really these three things that + +0:12:44.670,0:12:48.580 +it's not good to learn about and if you do +learn about it it's really good to keep it + +0:12:48.580,0:12:49.740 +to yourself + +0:12:49.740,0:12:51.949 +because if you become an expert or + +0:12:51.949,0:12:54.960 +viewed as an expert in any of these areas + +0:12:54.960,0:12:59.370 +you will become the designated stocky for +that and your psyche you'll never get to do + +0:12:59.370,0:13:01.760 +anything + +0:13:01.760,0:13:02.610 +but that + +0:13:02.610,0:13:07.360 +The three things that I highly +recommend not learning very much about are + +0:13:07.360,0:13:09.060 +device drivers, + +0:13:09.060,0:13:12.320 +send mail configuration files + +0:13:12.320,0:13:13.970 +or anything having to do + +0:13:13.970,0:13:19.350 +with LDAP or anything in +that general domain + +0:13:19.350,0:13:22.660 +%uh because as I say + +0:13:22.660,0:13:24.900 +that will become your life's work + +0:13:24.900,0:13:25.920 +and + +0:13:25.920,0:13:32.920 +there's other things that you might find more interesting. +""Do you have a question?"" + +0:13:33.870,0:13:36.659 +so one of my students empathized with my point + +0:13:36.659,0:13:39.640 +I believe you said you worked on that mail +system + +0:13:39.640,0:13:43.120 +%uh so you you might know something about +Sum mail configuration files but you don't + +0:13:43.120,0:13:47.850 +have to answer that + +0:13:47.850,0:13:52.100 +so we're going to talk about what a device +driver does and really just the entry + +0:13:52.100,0:13:53.170 +points to it + +0:13:53.170,0:13:57.180 +but we're not going to talk about how you +write such a thing, how you debug such a thing + +0:13:57.180,0:14:01.490 +or much of anything about it. I actually used +to teach an entire class believe it or not + +0:14:01.490,0:14:02.720 +about the device drivers + +0:14:02.720,0:14:05.849 +%uh but then I realized a year i waste +since + +0:14:05.849,0:14:12.580 +i gone through and made a point of forgetting +every slide in that talk. + +0:14:12.580,0:14:16.860 +so then we will move on to Filesystem + +0:14:16.860,0:14:21.540 +and as always will start at the high level +talk about the interface what is it that is + +0:14:21.540,0:14:23.020 +exported out of the system + +0:14:23.020,0:14:27.840 +%uh and then we will start diving down in sea and +how do we go about implementing that + +0:14:27.840,0:14:29.010 +%uh so + +0:14:29.010,0:14:31.010 +we'll start with that + +0:14:31.010,0:14:32.560 +so called + +0:14:32.560,0:14:33.680 +Block I/O system + +0:14:33.680,0:14:36.140 +it's historically been called buffer +cache + +0:14:36.140,0:14:38.590 +and it's still recalled that periodically + +0:14:38.590,0:14:42.720 +and the fact that the matter is that there isn't really +about buffer cache anymore,thers is just one big + +0:14:42.720,0:14:44.620 +cache in it.Its the VM cache + +0:14:44.620,0:14:47.810 +and the Filesystem has a view into it +and + +0:14:47.810,0:14:50.829 +the processes have a view into it but at +the end of the day + +0:14:50.829,0:14:54.660 +you really don't want the same information +on two different + +0:14:54.660,0:14:56.030 +pages of memory + +0:14:56.030,0:14:59.390 +because that just leads to trouble. + +0:14:59.390,0:15:03.390 +But Filesystems think that they have buffers and so +there's been a year where we make + +0:15:03.390,0:15:06.149 +these things that look like what historically +were buffers + +0:15:06.149,0:15:08.830 +that really just happened just really map into VM system + +0:15:08.830,0:15:11.720 +but they're still managed in the way that +they have been + +0:15:11.720,0:15:15.020 +managed historically + +0:15:15.020,0:15:20.670 +We will then get down into Filesystem implementation +the local file system if you will + +0:15:20.670,0:15:23.400 +and into also + +0:15:23.400,0:15:25.730 +soft updates and snapshots. + +0:15:25.730,0:15:26.440 +on this + +0:15:26.440,0:15:31.100 +for the time being is something that you see +only in FreeBSD + +0:15:31.100,0:15:35.310 +the alternative to soft updates is journalling +which is %uh more commonly used + +0:15:35.310,0:15:39.630 +for example what is used by ext3 + +0:15:39.630,0:15:41.179 +so i'll go through soft updates and + +0:15:41.179,0:15:45.260 +a lot of the issues in soft updates are the +same issues that you have to deal with journalling + +0:15:45.260,0:15:48.370 +what is it that we're protecting and how we +go about doing that + +0:15:48.370,0:15:51.150 +and the differences is in the detail. + +0:15:51.150,0:15:54.630 +There is actually a paper in the back to your +notes if this is something that interests + +0:15:54.630,0:15:55.240 +you + +0:15:55.240,0:15:59.930 +%uh it's a comparison of journalling versus +soft updates that was done + +0:15:59.930,0:16:02.120 +about five or eight years ago. + +0:16:02.120,0:16:08.460 +and not to spoil the punch line but the answers +they both work about are the same + +0:16:08.460,0:16:12.500 +Okay the snapshots again is something that +if + +0:16:12.500,0:16:15.920 +you've worked with things like the network +appilance box you're probably quite + +0:16:15.920,0:16:19.640 +aware of what snapshots are and how they do +or don't work for you + +0:16:19.640,0:16:21.959 +this is this the same functionality + +0:16:21.959,0:16:27.380 +%uh in the Filesystem implemented in a +somewhat different way + +0:16:27.380,0:16:28.449 +%uh so this + +0:16:28.449,0:16:31.940 +that Week 6 is really going to be the local +file system + +0:16:31.940,0:16:34.750 +the disk connected to the machine you work. + +0:16:34.750,0:16:39.140 +Week 7 then we get into multiple +Filesystem support so how do we abstract out that + +0:16:39.140,0:16:41.190 +Filesystem layer + +0:16:41.190,0:16:46.430 +and support Multiple Filesystems at the +same time so for example in FreeBSD + +0:16:46.430,0:16:50.199 +you can of course run with their traditional +fast Filesystem + +0:16:50.199,0:16:54.540 +but if you happen to like the Linux Filesystem +better or you have to share a disk + +0:16:54.540,0:16:55.690 +with a Linux machine + +0:16:55.690,0:16:58.310 +you can run the ext2 or ext3 + +0:16:58.310,0:17:01.020 +%uh and it will perfectly happily do that + +0:17:01.020,0:17:01.620 +so + +0:17:01.620,0:17:05.589 +we will have to look then at how do we provide +interface so that we can plug in all these different + +0:17:05.589,0:17:09.260 +Filesystems that we want to support + +0:17:09.260,0:17:12.250 +another area of which there's been a great + +0:17:12.250,0:17:15.309 +deal of growth at least in coding complexity + + +0:17:15.309,0:17:17.840 +is so-called Volume Management + +0:17:17.840,0:17:19.370 +%uh so in the + +0:17:19.370,0:17:24.480 +good old days a Filesystem lived on a disk or +piece of disk and that was that + +0:17:24.480,0:17:26.130 +but this day and age + +0:17:26.130,0:17:31.150 +that won't do any more so we aggregate disks +together by striping them or RAID + +0:17:31.150,0:17:31.980 +arraying them + +0:17:31.980,0:17:33.380 +and we need such really things + +0:17:33.380,0:17:39.210 +and we need a whole layer in the system just to +manage those disks + +0:17:39.210,0:17:44.280 +we'll then get to the as an example of an alternative +Filesystem we're going to talk about the + +0:17:44.280,0:17:46.530 +Network Filesystem or NFS + +0:17:46.530,0:17:48.500 +but that's not because this is + +0:17:48.500,0:17:51.090 +the world's best remote file system + +0:17:51.090,0:17:55.240 +or the cleanest design or any of the thing +properties you might hope that + +0:17:55.240,0:17:57.049 +such a class is this one would have + +0:17:57.049,0:17:58.600 +but it's ubiquitous + +0:17:58.600,0:18:00.210 +very widely used + +0:18:00.210,0:18:01.350 +and + +0:18:01.350,0:18:06.850 +we're going to talk about that one + +0:18:06.850,0:18:07.740 +we'll + +0:18:07.740,0:18:10.970 +then once again switch gears on Week 8 + +0:18:10.970,0:18:17.120 +and turn our attention to of Networking and +Interprocess communication + +0:18:17.120,0:18:18.200 +and + +0:18:18.200,0:18:23.210 +again we'll start from the very top so we'll +go through, we'll go with concepts, the terminology + +0:18:23.210,0:18:24.450 +that gets used + +0:18:24.450,0:18:30.230 +and what's difference between domain +based addressing and an address domain you know + +0:18:30.230,0:18:30.910 +we'll go through + +0:18:30.910,0:18:34.910 +well what the basic IPC services are, + +0:18:34.910,0:18:39.080 +essentially what all the system calls that +has anything to do with networking + +0:18:39.080,0:18:40.590 +and + +0:18:40.590,0:18:43.720 +just sort of describe what each of them are +and I'm going to go through + +0:18:43.720,0:18:45.830 +a somewhat contrived example + +0:18:45.830,0:18:49.840 +it makes use of every one of those interfaces + +0:18:49.840,0:18:52.860 +and just to show how they all conected +together + +0:18:52.860,0:18:54.169 +and for those of you that work + +0:18:54.169,0:18:57.400 +in networking or had done any kind of network +programming + +0:18:57.400,0:19:00.480 +if you're looking for a week to miss and this +Week 8 is the one to miss that's 'cause that's + +0:19:00.480,0:19:02.780 +the sort of most basic + +0:19:02.780,0:19:04.210 +lecture that I'm going to give + +0:19:04.210,0:19:07.910 +If you are not sure whether or not you need to +go through that, there is + +0:19:07.910,0:19:09.540 +one of the papers in the back + +0:19:09.540,0:19:12.620 +it is an introduction to Interprocess communication + +0:19:12.620,0:19:18.279 +%uh read that paper if you say yeah yeah yeah +yeah yeah you are done with Week 8. + +0:19:18.279,0:19:20.590 +on the other hand if you dont come to Week +8 + +0:19:20.590,0:19:22.790 +and then in Week 9 I say + +0:19:22.790,0:19:26.860 +I call on you and say alright what is it + +0:19:26.860,0:19:30.560 +that listen system call does and you +can't tell me + +0:19:30.560,0:19:32.610 +you're gonna get a demerit + +0:19:32.610,0:19:34.340 +okay + +0:19:34.340,0:19:37.770 +then in Week 9 we will get into the actual + +0:19:37.770,0:19:41.419 +networking implementation itself, we go +through system layers as we did + +0:19:41.419,0:19:43.310 +in all the other areas + +0:19:43.310,0:19:44.130 +and + +0:19:44.130,0:19:48.330 +we will spend a significant portion of that +class talking about routing + +0:19:48.330,0:19:50.230 +%uh routing + +0:19:50.230,0:19:53.610 +for those of you that haven't had the pleasure +of dealing with it + +0:19:53.610,0:19:55.540 +is a black heart + +0:19:55.540,0:19:58.050 +or at least a dark science + +0:19:58.050,0:19:59.170 +and + +0:19:59.170,0:19:59.930 +so + +0:19:59.930,0:20:02.490 +we'll talk about it + +0:20:02.490,0:20:06.270 +from the perspective first of all to what +do we do locally within the machine + +0:20:06.270,0:20:10.090 +and then what are some of the bigger strategies +that we can use for doing routing + +0:20:10.090,0:20:11.910 +%uh enterprise + +0:20:11.910,0:20:14.840 +wide routing or + +0:20:14.840,0:20:20.190 +area wide routing some like throughout the +state of California or throughout US whatever + +0:20:20.190,0:20:25.379 +this again like device drivers is really +just sort of a nickel + +0:20:25.379,0:20:26.480 +%uh throught the dark + +0:20:26.480,0:20:27.800 + +0:20:27.800,0:20:31.820 +what what the choices are what that the basic +strategies are that are used + +0:20:31.820,0:20:33.989 +If you're thinking you're going to walk out +of here + +0:20:33.989,0:20:36.110 +knowing how to set up a routing well sorry + +0:20:36.110,0:20:38.430 +we are not going to get that far + +0:20:38.430,0:20:41.559 +but you should at least have a pretty good idea +of what the issues are + +0:20:41.559,0:20:44.430 +and what the general solutions are + +0:20:44.430,0:20:48.950 +then finally in Week 10 well not finally +but next few weeks and + +0:20:48.950,0:20:52.380 +we will go through the Internet Protocols + +0:20:52.380,0:20:54.320 +primarily TCP/IP + +0:20:54.320,0:20:56.560 +and this is + +0:20:56.560,0:20:58.809 +what are the algorithms that are used + +0:20:58.809,0:21:01.030 +and I'm putting a particular emphasis + +0:21:01.030,0:21:03.050 +for this particular class + +0:21:03.050,0:21:05.080 +on + +0:21:05.080,0:21:07.730 +changes that have been made in the protocols + +0:21:07.730,0:21:14.310 +to deal with a lot of attacks that +we've been seeing SYN attacks and + +0:21:14.310,0:21:16.880 +that sort of thing + +0:21:16.880,0:21:19.440 +rather than just a straight + +0:21:19.440,0:21:22.440 +iteration of what the the actual protocols +are + +0:21:22.440,0:21:24.940 +i'll talk primarily about IPv4 + +0:21:24.940,0:21:31.940 +%uh but I will also try and talk a bit about +IPv6 as well + +0:21:33.510,0:21:35.850 +so the first ten weeks are + +0:21:35.850,0:21:38.100 +sort of kernel course + +0:21:38.100,0:21:40.800 +now back with two weeks on at the end + +0:21:40.800,0:21:42.010 +to talk about + +0:21:42.010,0:21:43.990 +through the bigger picture of + +0:21:43.990,0:21:48.240 +System Tuning,Crash dump analysis that level of +thing + +0:21:48.240,0:21:52.940 +The idea is really to consolidate what +we figured out or talked about in the first + +0:21:52.940,0:21:54.710 +ten weeks and + +0:21:54.710,0:21:58.760 +how that applies to tools that are available +to us to + +0:21:58.760,0:22:00.760 +look at what the system is doing, + +0:22:00.760,0:22:02.649 +to analyze what the system is doing + +0:22:02.649,0:22:03.650 +and hopefully + +0:22:03.650,0:22:04.720 +improves + +0:22:04.720,0:22:07.130 +the performance of what the system is doing + +0:22:07.130,0:22:07.750 +and + +0:22:07.750,0:22:12.169 +for the most part the kind of tuning I'm +talking about is not + +0:22:12.169,0:22:14.740 +going in and hack hack hacking your kernel + +0:22:14.740,0:22:16.510 +because the fact that the matter is + +0:22:16.510,0:22:18.600 +you can't do that anyway + +0:22:18.600,0:22:22.340 +%uh so it's more looking at it from the perspective +of saying + +0:22:22.340,0:22:26.390 +is this system running badly because it doesn't +have enough memory on it? + +0:22:26.390,0:22:29.470 +or is it running badly because there isn't enough +I/O capacity? + +0:22:29.470,0:22:33.549 +or is it running badly because the it's got +enough I/O capacity but + +0:22:33.549,0:22:35.940 +certain drives are being overloaded + +0:22:35.940,0:22:37.309 +or is it + +0:22:37.309,0:22:42.220 +being overrun because we're simply trying +to do too much on this machine?,etc. + +0:22:42.220,0:22:45.440 +so that's the sort of level of thing that we're +looking at it + +0:22:45.440,0:22:47.080 +but tied into + +0:22:47.080,0:22:52.130 +lot of concepts that we talked before so we can talk +about active virtual memory + +0:22:52.130,0:22:53.710 +and what that means + +0:22:53.710,0:22:55.120 +and + +0:22:55.120,0:22:58.750 +essentially measure what it is and hopefully +then you will understand the context of what + +0:22:58.750,0:23:00.690 +we talked about in the VM section + +0:23:00.690,0:23:03.990 +what that really means + +0:23:03.990,0:23:07.460 +the Crash dump analysis is is one of these +topics that + +0:23:07.460,0:23:08.730 +you are gonna love or hate + +0:23:08.730,0:23:12.530 +%uh you actually have to deal with crashed +dumps + +0:23:12.530,0:23:13.679 +its people find it invaluable + +0:23:13.679,0:23:15.580 +and if you don't have to deal with Crash dumps + +0:23:15.580,0:23:18.790 +it's an incredible mass of boring detail + +0:23:18.790,0:23:23.240 +the only good part of it is that that's the +whole session is only about an hour or long + +0:23:23.240,0:23:25.529 +If it interests you, listen closely + +0:23:25.529,0:23:28.950 +and if it bores you, well, its only an hour long + +0:23:28.950,0:23:32.880 +okay up lastly we'll talk a little bit about +security issues + +0:23:32.880,0:23:36.250 +again this is really more of the tools that +are available + +0:23:36.250,0:23:40.750 +to deal with security staff as opposed to a +complete tutorial on + +0:23:40.750,0:23:45.120 +how to implement security so those of you +that deal with security + +0:23:45.120,0:23:48.400 +it's just gonna to be sort of security one on +one + +0:23:48.400,0:23:50.029 +%uh for those of you + +0:23:50.029,0:23:51.500 +that have but + +0:23:51.500,0:23:54.399 +you'll have to deal with it but haven't really +thought about it + +0:23:54.399,0:23:58.549 +%uh it'll probably scare you to death and +you wonder have to keep the machines from + +0:23:58.549,0:24:02.840 +being hijacked everyday + +0:24:02.840,0:24:08.030 +Okay that's in essence what we're going +to try and do here + +0:24:08.030,0:24:15.030 +anybody have any comments, questions, thoughts. +Alright, no, well. + +0:24:16.130,0:24:17.840 +Let's get started + +0:24:17.840,0:24:22.180 +we will be begin on page fifteen with an +overview of the kernel. + +0:24:22.180,0:24:26.040 +Hopefully nobody's lost yet. + +0:24:26.040,0:24:29.310 +What's a kernel? Alright. + +0:24:29.310,0:24:31.370 +so starting at the very top + +0:24:31.370,0:24:33.070 +the big broad brush + +0:24:33.070,0:24:35.140 +what we have is + +0:24:35.140,0:24:38.330 +a UNIX virtual machine and + +0:24:38.330,0:24:41.660 +and virtual machines are actually something +that has been around + +0:24:41.660,0:24:44.539 +as a concept since the sixties + +0:24:44.539,0:24:48.919 +the diffrence is really just sort of the level +of the interface that people have dealt with + +0:24:48.919,0:24:51.360 +when they talk about Virtual Machines + +0:24:51.360,0:24:53.610 +in the 1960s + +0:24:53.610,0:24:56.770 +computers were these enormous things you would +have + +0:24:56.770,0:24:58.870 +your computer room would be something that be + +0:24:58.870,0:25:01.909 +three times the size of this conference +room if you had + +0:25:01.909,0:25:03.230 +a computer + +0:25:03.230,0:25:05.530 +the computer itself was this + +0:25:05.530,0:25:07.840 +tall as a refrigerator freezer + +0:25:07.840,0:25:08.950 +imagine + +0:25:08.950,0:25:13.909 +five or eight or ten of these units +side by side that itself made up the computer + +0:25:13.909,0:25:16.080 +that would be one + +0:25:16.080,0:25:20.030 +for the core processor and the one which +should be the floating point unit and several + +0:25:20.030,0:25:24.080 +of them that would be the memory the core +literally + +0:25:24.080,0:25:29.110 +%uh and then they'd be other rows of these +disk drives about the size of a washing + +0:25:29.110,0:25:29.660 +machine + +0:25:29.660,0:25:34.169 +%then behind that since you couldn't store +everything on disk so + +0:25:34.169,0:25:36.300 +that you had rows of tape drives + +0:25:36.300,0:25:37.880 +and then you had this little + +0:25:37.880,0:25:39.610 +set of sort of + +0:25:39.610,0:25:43.330 +munchkins that would run around and and tend +to the machine and they mount tapes and take + +0:25:43.330,0:25:46.710 +off tapes and mount disc packs and remove disc packs +because + +0:25:46.710,0:25:49.760 +the drives themselves were very expensive and +so + +0:25:49.760,0:25:53.110 +you wouldn't just as today we have a + + +0:25:53.110,0:25:56.090 +one spindle that was dedicated just to one set +of platters + +0:25:56.090,0:25:57.130 +you could take out a + +0:25:57.130,0:25:59.460 +set of platters and put in another + +0:25:59.460,0:26:02.540 +hundred megabytes of platters and these are +platters that are + +0:26:02.540,0:26:05.280 +this big around and it's like six or eight +of them and + +0:26:05.280,0:26:09.140 +a giant head assemblies they comes rumbling in and +out + +0:26:09.140,0:26:12.440 +anyway one of these giant giant machines + +0:26:12.440,0:26:17.380 +that costs many millions of dollars would run +at about ten + +0:26:17.380,0:26:21.120 +million instructions per second, 10 mips + +0:26:21.120,0:26:21.630 +and 10 mips + +0:26:21.630,0:26:28.330 +that's was more computing power than anybody +could possibly imagine using in a single application + +0:26:28.330,0:26:28.880 +just + +0:26:28.880,0:26:31.050 +by contrast this this + +0:26:31.050,0:26:34.070 +four-year-old laptop here is probably on +the order of + +0:26:34.070,0:26:36.440 +one or two hundred mips + +0:26:36.440,0:26:37.140 +but anyway + +0:26:37.140,0:26:40.760 +people couldn't really view what we would +do with a lot of computing power + +0:26:40.760,0:26:44.640 +and the other thing was that you didn't have +a notion of sort of operating system that had + +0:26:44.640,0:26:45.890 +applications running on it + +0:26:45.890,0:26:46.760 +because + +0:26:46.760,0:26:50.160 +everybody wanted to write straight to +the raw hardware + +0:26:50.160,0:26:51.750 +and so + +0:26:51.750,0:26:55.900 +IBM who was a big manufacturer +of machines in those days + +0:26:55.900,0:26:59.060 +did what they came up with this thing called +the VM + +0:26:59.060,0:27:00.770 +and this was a little + +0:27:00.770,0:27:02.549 +you'd call an operating system really + +0:27:02.549,0:27:05.130 +but what it did is a cloned + +0:27:05.130,0:27:09.270 +independent copies of the machine worked just +like the original machines you could boot + +0:27:09.270,0:27:11.769 +something that thought it was an operating +system + +0:27:11.769,0:27:13.380 +on top of VM + +0:27:13.380,0:27:16.750 +so you take one least ten that machines and +it would clone + +0:27:16.750,0:27:20.050 +six identical one mip copies + +0:27:20.050,0:27:22.030 +%uh and then you could boot + +0:27:22.030,0:27:24.700 +whatever you wanted on each one of those machines +so + +0:27:24.700,0:27:29.510 +if you were doing database that you do your +database because database cannot run on the raw hardware + +0:27:29.510,0:27:32.920 +or if you're doing payroll who would boot up the payroll +program + +0:27:32.920,0:27:37.950 +%uh or if you actually tried to service the +users you could boot a time sharing batch thing + +0:27:37.950,0:27:40.790 +that would read card images and print +stuff out + +0:27:40.790,0:27:44.460 +or they even had TSO the Time Sharing +Option where you could interactively sit + +0:27:44.460,0:27:45.559 +and type send + +0:27:45.559,0:27:47.560 +stuffs in and get answers back + +0:27:47.560,0:27:48.570 +%uh and + +0:27:48.570,0:27:51.429 +%uh also you could boot TSO so whatever set +of + +0:27:51.429,0:27:52.219 +%um + +0:27:52.219,0:27:55.339 +things you need you could boot them and they ran +independently as if they were running on their + +0:27:55.339,0:27:56.470 +own machine + +0:27:56.470,0:28:03.150 +but all the VM did was it give you an exact +raw copy of the hardware + +0:28:03.150,0:28:04.529 +so when UNIX came along + +0:28:04.529,0:28:07.350 +they sort of like the notion of + +0:28:07.350,0:28:11.509 +providing the concept of independent +things that you could operate in + +0:28:11.509,0:28:13.610 +but they wanted it at a higher level + +0:28:13.610,0:28:15.610 +so you're looking merely to do that + +0:28:15.610,0:28:17.480 +instead of at the raw hardware level + +0:28:17.480,0:28:19.679 +to do it at a process well + +0:28:19.679,0:28:23.799 +and the idea that then was that the interface you +would program to would be what we think of as + +0:28:23.799,0:28:26.090 +a System call interface today + +0:28:26.090,0:28:27.849 +and the idea then was that + +0:28:27.849,0:28:30.740 +you would be given a process or set of processes + +0:28:30.740,0:28:34.990 +and those were independent your process +couldn't affect + +0:28:34.990,0:28:38.830 +the address space of another process reach +over and mess around with the addresses, + +0:28:38.830,0:28:41.030 +you couldn't mess around with their I/O +channels + +0:28:41.030,0:28:43.179 +%uh you could slow them down by + +0:28:43.179,0:28:44.299 +being a pig but + +0:28:44.299,0:28:47.980 +that was about the only way that you could affect +other processes + +0:28:47.980,0:28:48.480 +and + +0:28:48.480,0:28:49.830 +%uh so + +0:28:49.830,0:28:52.669 +what the interfaces that they had there + +0:28:52.669,0:28:58.660 +with one that had these characteristics +he had a a paged virtual address space + +0:28:58.660,0:29:02.980 +so you din't have to know as in the old days how much physical +memory is on the machine and make your application + +0:29:02.980,0:29:04.740 +fit into that amount of memory + +0:29:04.740,0:29:07.950 +you just had what looked like a large + +0:29:07.950,0:29:11.710 +uniform address space even if the underlying +hardware had segments or some other + +0:29:11.710,0:29:13.580 +hardware brain damage + +0:29:13.580,0:29:17.390 +it look to you like he just had a big uniform +address space and + +0:29:17.390,0:29:21.070 +the size of the address space was independent +of the amount of memory that was on the machine + +0:29:21.070,0:29:23.900 +your address space could be bigger than amount of +physical memory + +0:29:23.900,0:29:26.499 +cause we got to move pages around underneath + +0:29:26.499,0:29:29.320 +whatever part address space was actually +active + +0:29:29.320,0:29:34.260 +that is obviously do this if you if +you are trying to run 1 gigabyte of + +0:29:34.260,0:29:35.630 +application on top of + +0:29:35.630,0:29:37.240 +ten megabytes of memory + +0:29:37.240,0:29:40.880 +it's probably going to bring new meaning to +same day service + +0:29:40.880,0:29:45.519 +but if you're willing to wait long enough it +will eventually move the pages around and you will + +0:29:45.519,0:29:49.740 +progress through getting your application run + +0:29:49.740,0:29:53.890 +anothe thing was dealing with software +interrupts + +0:29:53.890,0:29:55.789 +%uh in the old days + +0:29:55.789,0:29:58.749 +you had to understand how the hardware worked + +0:29:58.749,0:30:03.900 +in order to deal with exceptional conditions +so for example to did a divide by zero + +0:30:03.900,0:30:08.170 +the the hardware would have jumped to some +vector location or + +0:30:08.170,0:30:08.630 +something + +0:30:08.630,0:30:12.799 +and you had no khow how that worked and make +sure that you had your program + +0:30:12.799,0:30:16.510 +use just a little bit of assembly language which +set up to deal with that + +0:30:16.510,0:30:19.870 +and UNIX said let's let's get away +from the hardware here + +0:30:19.870,0:30:22.080 +and so they did this thing called signals + +0:30:22.080,0:30:25.700 +the answer is, the finest of the signals is that +if you do divide by zero + +0:30:25.700,0:30:29.529 +%uh you simply register a routine you +want to have called you don't have to know + +0:30:29.529,0:30:31.220 +how the hardware figured it out + +0:30:31.220,0:30:36.740 +you just know that that routine is going to get +called and you can deal with it at that point + +0:30:36.740,0:30:40.960 +well we got set of timers and counters to keep +track of what we're doing, this is really more + +0:30:40.960,0:30:43.490 +for counting than anything else but + +0:30:43.490,0:30:46.970 +applications may want to have access to that. + +0:30:46.970,0:30:51.720 +we have a set of identifiers that we're +going to use for things like accounting, + +0:30:51.720,0:30:54.830 +protection and scheduling and so on + +0:30:54.830,0:30:55.820 +and one of the + +0:30:55.820,0:31:00.320 +the early philosophies of UNIX was that to try +and keep it simple. + +0:31:00.320,0:31:02.630 +operating systems have gotten very baroque + +0:31:02.630,0:31:04.490 +in particular the thing that + +0:31:04.490,0:31:07.350 +pre dated UNIX was a thing called +Multix + +0:31:07.350,0:31:12.820 +%uh Multix was was a joint project between +Honeywell, a big computer manufacturer of the + +0:31:12.820,0:31:15.740 +time at + +0:31:15.740,0:31:17.129 +AT&T bell laboratories + +0:31:17.129,0:31:19.750 +the big industrial labratory at that time + +0:31:19.750,0:31:21.380 +and MIT + +0:31:21.380,0:31:23.430 +a big university then and + +0:31:23.430,0:31:24.690 +still today + +0:31:24.690,0:31:29.259 +%uh and that those three organizations got +together to try and build this + +0:31:29.259,0:31:31.400 +time sharing operating system + +0:31:31.400,0:31:32.280 +and it + +0:31:32.280,0:31:33.770 +it just got bigger and + +0:31:33.770,0:31:37.160 +more grandiose and more complex and never +finished + +0:31:37.160,0:31:38.979 +but as soon as that's what we see + +0:31:38.979,0:31:42.709 +although we know how to do that but also we could +do this other thing too and so then the teared it + +0:31:42.709,0:31:43.429 +apart and + +0:31:43.429,0:31:46.440 +they never really got to something that + +0:31:46.440,0:31:48.210 +could be put into production + +0:31:48.210,0:31:49.919 +and so the + +0:31:49.919,0:31:50.570 +AT&T + +0:31:50.570,0:31:54.340 +Bell laboratories decided to pull out of +that project + +0:31:54.340,0:31:55.940 +and + +0:31:55.940,0:32:00.000 +the two of the people that had been working on +that project, Ken Thompson and Dennis Richie + +0:32:00.000,0:32:04.390 +%uh were sort of bound because they were now +back to typing cards and putting them through + +0:32:04.390,0:32:05.259 +card reader and + +0:32:05.259,0:32:07.960 +they are just used to the idea that you could +actually + +0:32:07.960,0:32:11.559 +sit at an ASSR teletype and interact +with your computer + +0:32:11.559,0:32:13.440 +and so + +0:32:13.440,0:32:18.230 +they found an old %uh PDP-8 soon off in +the quarter that have been abandoned + +0:32:18.230,0:32:22.120 +and started working on this little tiny operating +system which they called UNIX + +0:32:22.120,0:32:26.549 +%uh which eventually moved to the PDP-11 and +became what we have today + +0:32:26.549,0:32:28.050 +but because it was + +0:32:28.050,0:32:32.120 +they were coming first of all from Multix +where everything had been done and + +0:32:32.120,0:32:34.110 +in great grandiose detail + +0:32:34.110,0:32:37.549 +and because they're fundamentally just two +the of them working on it and they wanted to get something + +0:32:37.549,0:32:38.370 +done and + +0:32:38.370,0:32:40.130 +within a year or so + +0:32:40.130,0:32:41.529 +one of their philosophies was + +0:32:41.529,0:32:44.099 +let's find out one way of doing things + +0:32:44.099,0:32:48.180 +let's not have eight ways from Sunday let's just +get the one way + +0:32:48.180,0:32:53.860 +and and that's what we will provide. So what is +the sort of core set of things that we need. + +0:32:53.860,0:32:58.620 +well first thing is when it comes to identifiers, +let's not have you know + +0:32:58.620,0:33:00.430 +eighty thousand different identifiers + +0:33:00.430,0:33:03.140 +so they came up with process identifiers, + +0:33:03.140,0:33:09.620 +user identifier and at that time a single group +identifier and later expanded + +0:33:09.620,0:33:14.200 +and they used that sort of identifies for everything +so its used for counting, used for main + +0:33:14.200,0:33:17.410 +protection decisions, used for scheduling +decisions + +0:33:17.410,0:33:19.470 +and + +0:33:19.470,0:33:24.279 +again it was the simplicity of thing which +was what was driving their decision + +0:33:24.279,0:33:28.840 +but they're really sort of true key idea +that they had + +0:33:28.840,0:33:30.880 +that really made a difference that + +0:33:30.880,0:33:32.539 +that's were set them up side + +0:33:32.539,0:33:34.749 +from what everybody else done before them + +0:33:34.749,0:33:35.450 +and which + +0:33:35.450,0:33:39.740 +in retrospect is something that has been pervasive +more or less ever since + +0:33:39.740,0:33:41.869 +the first of these was the notion + +0:33:41.869,0:33:44.840 +that we have a unique descriptive space + +0:33:44.840,0:33:46.289 +%uh that is + +0:33:46.289,0:33:51.250 +given a descriptor it can reference +any I/O device + +0:33:51.250,0:33:53.650 +or even any kind of I/O channel + +0:33:53.650,0:33:58.270 +so you can have a descriptor for terminal +or descriptor for file or descriptive for + +0:33:58.270,0:34:02.240 +a disk or descriptor for pipe or descriptor +for a socket + +0:34:02.240,0:34:03.500 +and + +0:34:03.500,0:34:04.790 +you don't need to know + +0:34:04.790,0:34:07.940 +what it references in order to be able to read +and write on that thing + +0:34:07.940,0:34:11.290 +so if i hand you a descriptor +can read from that the descriptor or you can write + +0:34:11.290,0:34:13.259 +to that descriptor + +0:34:13.259,0:34:15.189 +and + +0:34:15.189,0:34:17.359 +the correct thing will happen + +0:34:17.359,0:34:19.089 +and you'd say well + +0:34:19.089,0:34:23.629 +that's obvious I mean how else could you +possibly think about doing it + +0:34:23.629,0:34:25.179 +predating UNIX + +0:34:25.179,0:34:28.059 +everything was done with + +0:34:28.059,0:34:29.379 +a little subsystem + +0:34:29.379,0:34:33.419 +that would open a file, read a file, write a +file, close a file + +0:34:33.419,0:34:37.429 +and there was another set of system calls which +would open a terminal,read a terminal, write terminal + +0:34:37.429,0:34:38.089 +,close terminal + +0:34:38.089,0:34:39.210 +and yet another one + +0:34:39.210,0:34:42.409 +which was create a pipe,read a pipe, +write a pipe and so on. + +0:34:42.409,0:34:47.699 +so if you are just dropped at a stupid +program like CAD + +0:34:47.699,0:34:51.579 +you would have to have code in there and say was +my input a terminal which in case I need to + +0:34:51.579,0:34:53.159 +use the read terminal + +0:34:53.159,0:34:57.419 +or is it a file which in case i need +to use read file or is it a pipe in which in case + +0:34:57.419,0:34:59.189 +i need to use read pipe + +0:34:59.189,0:35:01.860 +and so the program itself had to have all +this + +0:35:01.860,0:35:02.859 +coding in it + +0:35:02.859,0:35:04.409 +whereas when they went to + +0:35:04.409,0:35:07.159 +the uniform descriptor space + +0:35:07.159,0:35:09.630 +CAD doesn't know and doesn't need to know +it just says + +0:35:09.630,0:35:10.819 +read my input, + +0:35:10.819,0:35:13.979 +write the output + +0:35:13.979,0:35:17.059 +it works and we add a new type of descriptor + +0:35:17.059,0:35:17.600 +and + +0:35:17.600,0:35:21.700 +CAD just continues to work just as it always +did. + +0:35:21.700,0:35:24.199 +So this proved to be a very powerful construct + +0:35:24.199,0:35:27.019 +and pretty much every operating system after +UNIX + +0:35:27.019,0:35:28.659 +did that and there's + +0:35:28.659,0:35:30.210 +one exception of %uh + +0:35:30.210,0:35:32.549 +large company in the Pacific North-West + +0:35:32.549,0:35:35.830 +that still has not quite uniform descriptor +space + +0:35:35.830,0:35:38.380 +but %uh that's part of their legacy that really + +0:35:38.380,0:35:39.900 +they're working on that. + +0:35:39.900,0:35:42.009 +Longhorn will be here. + +0:35:42.009,0:35:43.939 +and anyway + +0:35:43.939,0:35:46.190 +this set of facilities + +0:35:46.190,0:35:50.150 +makes up the UNIX virtual machine + +0:35:50.150,0:35:51.559 +and + +0:35:51.559,0:35:55.559 +in sometimes we still see virtual machines +being used today in fact we're seeing sort + +0:35:55.559,0:35:56.749 +of a reversion + +0:35:56.749,0:36:01.429 +%uh back to some of the ideas stop in things +like the VMware + +0:36:01.429,0:36:03.079 +which is + +0:36:03.079,0:36:07.029 +essentially allow you to go back to booting +native operating systems again so sort of + +0:36:07.029,0:36:08.280 +interesting to watch + +0:36:08.280,0:36:09.060 +that's the sort of + +0:36:09.060,0:36:12.919 +%uh %uh pendulum of back going back and forth +of what's the correct layer + +0:36:12.919,0:36:14.609 +for for doing + +0:36:14.609,0:36:18.890 +%uh virtual machines + +0:36:18.890,0:36:22.499 +Okay. so far so good. + +0:36:22.499,0:36:24.719 +all right i said there was + +0:36:24.719,0:36:27.160 +two key ideas that UNIX had + +0:36:27.160,0:36:30.279 +the first thing is uniform descriptor +space + +0:36:30.279,0:36:35.819 +the second one which was really critical was +this notion of processes as a commodity + +0:36:35.819,0:36:37.309 +item + +0:36:37.309,0:36:40.220 +so here on Page 17 I tried to lay +it out + +0:36:40.220,0:36:41.090 +the + +0:36:41.090,0:36:44.159 +that the components that make up a process + +0:36:44.159,0:36:45.759 +%uh and + +0:36:45.759,0:36:50.359 +what do I really mean when I say a process as +a commodity item + +0:36:50.359,0:36:53.650 +leading up to + +0:36:53.650,0:36:54.689 +UNIX + +0:36:54.689,0:36:56.800 +the systems that pre-dated it, + +0:36:56.800,0:36:59.200 +processes were these very large + +0:36:59.200,0:37:02.169 +heavyweight expensive thing + +0:37:02.169,0:37:02.779 +and + +0:37:02.779,0:37:04.539 +if you look at + +0:37:04.539,0:37:08.629 +MVS which was the operating system +that ran on IBM for doing multiple processing + +0:37:08.629,0:37:10.509 +and + +0:37:10.509,0:37:13.799 +the system administrator would decide at boot +time + +0:37:13.799,0:37:17.019 +what degree of multiprocessing they wish +to support + +0:37:17.019,0:37:18.140 +so they'd say well + +0:37:18.140,0:37:20.739 +well, let upto six things happen at once + +0:37:20.739,0:37:22.490 +and so as part of booting up + +0:37:22.490,0:37:24.419 +they would create six + +0:37:24.419,0:37:25.349 +processes + +0:37:25.349,0:37:30.059 +and now you as a user if you wanted to do +something let's say you wanted to + +0:37:30.059,0:37:32.009 +compile and ran a program + +0:37:32.009,0:37:34.960 +%uh you would be given a process + +0:37:34.960,0:37:36.019 +and it's up to you + +0:37:36.019,0:37:39.369 +to figure out how to stage what you needed +done + +0:37:39.369,0:37:39.819 +and + +0:37:39.819,0:37:43.930 +%uh that this was often fairly complex + +0:37:43.930,0:37:47.880 +and so you would have to write out all the +steps that you wanted + +0:37:47.880,0:37:50.300 +in this wonderful thing called JCL + +0:37:50.300,0:37:52.259 +Job Control Language and + +0:37:52.259,0:37:56.650 +Job Control Language was send mail configuration +file of the sixties + +0:37:56.650,0:38:00.679 +other where people who sold job that the company +was how to put this stuff together 'cause + +0:38:00.679,0:38:04.189 +like to do is get what one extra space or +missing comma + +0:38:04.189,0:38:05.000 +something in their + +0:38:05.000,0:38:08.630 +and I hope it would just blow up it would +just sort of spit the card deck back at + +0:38:08.630,0:38:09.799 +you and say well + +0:38:09.799,0:38:13.500 +somewhere in there is a mistake that's sort of +the general area of this card + +0:38:13.500,0:38:15.549 +and I can't deal with it. Fix it. + +0:38:15.549,0:38:16.489 +and of course + +0:38:16.489,0:38:20.550 +in those days it wasn't just a matter of hitting +carriage when you know make cariage return you have to + +0:38:20.550,0:38:25.239 +get your deck pull out the card, and type the +new one, put it back in and re-submit it + +0:38:25.239,0:38:28.729 +As heaven forbid you couldnt touch that +card reader you know, it had to be done by + +0:38:28.729,0:38:29.970 +an operator + +0:38:29.970,0:38:32.869 +so the card deck will read through it would +disappear and + +0:38:32.869,0:38:36.800 +you know if you're lucky a few minutes later +if you were not lucky a few hours later + +0:38:36.800,0:38:37.849 +you would get + +0:38:37.849,0:38:39.570 +a print print out + +0:38:39.570,0:38:43.419 +which was what it happened and then you could +look at it and you know + +0:38:43.419,0:38:47.209 +I put a comma in the wrong place I guess +I get to do it all again + +0:38:47.209,0:38:49.930 +so + +0:38:49.930,0:38:54.940 +if you would need to do something +like compiling and running a program + +0:38:54.940,0:38:59.579 +was you'd have to break into these steps. well +I need to run the the preprocessor + +0:38:59.579,0:39:04.670 +and so us clean out whatever gump that was left +over on that process from the previous user + +0:39:04.670,0:39:06.240 +put the preprocessor in there + +0:39:06.240,0:39:10.530 +%uh and then read from this file here let's +say gotta put it somewhere so creative + +0:39:10.530,0:39:12.510 +scratch file over on this disk and + +0:39:12.510,0:39:17.299 +it was excruciating detail like how many cylinders +and how many tracks and this and that + +0:39:17.299,0:39:19.139 +blocks blah blah blah + +0:39:19.139,0:39:23.119 +and don't forget any of those parameters 'cause +it'll spit it out if you do + +0:39:23.119,0:39:26.890 +and so then it would run the first step in that +if its successful then you'd have sitting + +0:39:26.890,0:39:28.899 +in this scratch file that you had created + +0:39:28.899,0:39:33.100 +the output of the preprocessor and then +you'd load the first pass of the compiler + +0:39:33.100,0:39:36.930 +and lets now read from that scratch file +and create this other scratch file over here and + +0:39:36.930,0:39:39.450 +when thats successful and we need to delete that +one + +0:39:39.450,0:39:43.830 +and then load the second pass, put that back +into another scratch file and then we run this + +0:39:43.830,0:39:45.950 +assembler, optimizer then the + +0:39:45.950,0:39:47.750 +loader this and that + +0:39:47.750,0:39:49.410 +finally run the program + +0:39:49.410,0:39:50.900 +and %uh if all goes well + +0:39:50.900,0:39:57.029 +you know at step sixteen out comes the answer + +0:39:57.029,0:39:58.129 +fourty two. so UNIX + +0:39:58.129,0:40:00.819 +said, look this is silly + +0:40:00.819,0:40:02.880 +a lot of this is just + +0:40:02.880,0:40:04.310 +bookkeeping + +0:40:04.310,0:40:07.249 +and computers do bookkeeping really well + +0:40:07.249,0:40:12.179 +and you'll recall yeah but it's going to take +all these cycles it's like + +0:40:12.179,0:40:16.309 +computers are supposed to be labor-saving +devices right so + +0:40:16.309,0:40:20.150 +they came up with this notion that they would +create processes on the fly as needed + +0:40:20.150,0:40:21.159 +you had + +0:40:21.159,0:40:25.549 +you've had a preprocessor in two +steps of the compiler and then + +0:40:25.549,0:40:27.109 +optimizer and then a loader + +0:40:27.109,0:40:29.410 +we just create Boom seven processes + +0:40:29.410,0:40:31.920 +and we connect them together with pipes + +0:40:31.920,0:40:35.180 +and so we take the input and you know run +through in + +0:40:35.180,0:40:38.270 +through the pipes and you know out the end +you get the the + +0:40:38.270,0:40:39.629 +executable + +0:40:39.629,0:40:40.030 +and + +0:40:40.030,0:40:42.880 +we simply create each of these processes + +0:40:42.880,0:40:44.650 +and + +0:40:44.650,0:40:46.549 +so you as a user just + +0:40:46.549,0:40:49.479 +type you know the C compiler just + +0:40:49.479,0:40:52.429 +fork these things pipe them together got the result + +0:40:52.429,0:40:53.640 +and + +0:40:53.640,0:40:57.509 +then once it was done with this processes is +just threw them away so any time you'd create a + +0:40:57.509,0:41:00.479 +new process and it came to you christine clean + +0:41:00.479,0:41:04.239 +and you needed a bunch of things it did +put everything in intermediate files + +0:41:04.239,0:41:07.549 +the fact that matter is in the early days + +0:41:07.549,0:41:08.129 +those computers + +0:41:08.129,0:41:11.910 +didn't really have enough memory to support +all that stuff at once so + +0:41:11.910,0:41:15.809 +behind you those pipes were actually implemented +as files + +0:41:15.809,0:41:19.319 +you didn't have atleast to remember to create +them and delete them + +0:41:19.319,0:41:20.200 +and deal with them + +0:41:20.200,0:41:24.020 +as far as you're concerned it just look stuff flowing through pipes +and of course today it + +0:41:24.020,0:41:24.490 +just + +0:41:24.490,0:41:27.989 +does flow through pipes in memory + +0:41:27.989,0:41:29.439 +okay so + +0:41:29.439,0:41:33.689 +this notion then that that we're just gonna +create processes on the fly is needed and + +0:41:33.689,0:41:35.559 +connect them together as needed + +0:41:35.559,0:41:38.039 +and that was a novel concept + +0:41:38.039,0:41:43.599 +and it wasn't that somehow mysteriously figured +out how to create processes cheaply + +0:41:43.599,0:41:44.839 +cause they hadn't + +0:41:44.839,0:41:46.180 +they were still + +0:41:46.180,0:41:49.959 +really expensive to create + +0:41:49.959,0:41:52.210 +but that extra effort + +0:41:52.210,0:41:53.029 +was + +0:41:53.029,0:41:56.089 +worth it because it was saving a lot of programming +time + +0:41:56.089,0:41:59.809 +so my favorite example is you run ls + +0:41:59.809,0:42:01.810 +so we have to create a process + +0:42:01.810,0:42:04.259 +load the ls binary into it + +0:42:04.259,0:42:06.180 +and it prints a line or two on your screen + +0:42:06.180,0:42:10.609 +and we tear the entire thing down and return +all its resources back to the system + +0:42:10.609,0:42:14.979 +more than ninety percent of the cost of running +ls is creating and destroying the process + +0:42:14.979,0:42:19.239 +a tiny fraction of it is actually running +ls + +0:42:19.239,0:42:24.259 +it goes so fast, who cares right + +0:42:24.259,0:42:25.749 +the point is that + +0:42:25.749,0:42:30.039 +%uh that concept of just creating things as +needed + +0:42:30.039,0:42:31.780 +again was very powerful + +0:42:31.780,0:42:35.709 +and is one that is just pervasive today + +0:42:35.709,0:42:38.639 +okay so what is a process actually made up +of + +0:42:38.639,0:42:43.179 +%uh it gets some amount of CPU time or at +least we do really hope that it gets some + +0:42:43.179,0:42:46.050 +amount of CPU time, the lack of getting +CPU time + +0:42:46.050,0:42:46.670 +that makes it + +0:42:46.670,0:42:47.979 +a pure so sluggish + +0:42:47.979,0:42:49.409 +of course + +0:42:49.409,0:42:51.920 +others really boils down to scheduling + +0:42:51.920,0:42:54.249 +and we're going to talk about scheduling + +0:42:54.249,0:42:56.279 +probably more than you care to + +0:42:56.279,0:42:59.219 +in a couple weeks time + +0:42:59.219,0:43:01.619 +we have the asynchronous events + +0:43:01.619,0:43:04.569 +%uh these are the external events that + +0:43:04.569,0:43:05.659 +are coming in + +0:43:05.659,0:43:07.679 +%uh so + +0:43:07.679,0:43:10.169 +they may be either things that + +0:43:10.169,0:43:14.339 +we're coming in from the outside world like +start, stop and quit + +0:43:14.339,0:43:15.279 +%uh oh + +0:43:15.279,0:43:18.170 +out-of-band arrival notification that kind +of thing + +0:43:18.170,0:43:22.339 +or it may in fact be things that the program +is bringing down upon itself + +0:43:22.339,0:43:25.590 +on such as a segment fault,a divide by zero + +0:43:25.590,0:43:26.910 +and some other + +0:43:26.910,0:43:31.959 +what would normally be viewed as incorrect +operation + +0:43:31.959,0:43:35.849 +and so will talk about that when we talk about +signals + +0:43:35.849,0:43:37.039 +every program + +0:43:37.039,0:43:38.899 +get some amount of memory + +0:43:38.899,0:43:42.659 +it gets an initial amount when it starts +up injured generally allocates more as it + +0:43:42.659,0:43:45.229 +goes along + +0:43:45.229,0:43:49.429 +of course we will deal with up her extensively +will spend an entire week on it + +0:43:49.429,0:43:54.249 +when we talk about how virtual memory is implemented + +0:43:54.249,0:43:54.609 +and + +0:43:54.609,0:43:57.429 +%uh then we get I/O descriptors + +0:43:57.429,0:44:02.259 +I used to say that every program had to have +at least one I/O descriptor since + +0:44:02.259,0:44:04.910 +it absolutely had no input + +0:44:04.910,0:44:06.329 +absolutely no output + +0:44:06.329,0:44:09.049 +then it was sort of pointless + +0:44:09.049,0:44:12.900 +of course I had to have one of my students +come up and point out to me there is a + +0:44:12.900,0:44:13.849 +class of programs + +0:44:13.849,0:44:16.469 +which don't need I/O descriptors + +0:44:16.469,0:44:17.440 +and that is + +0:44:17.440,0:44:19.549 +these things called benchmarks + +0:44:19.549,0:44:23.249 +the just compute something all we really care +about is how long it takes them to compute + +0:44:23.249,0:44:24.959 +we dont actually care what the answer is + +0:44:24.959,0:44:26.019 +In theory we dont + +0:44:26.019,0:44:29.779 +I personally felt like my benchmark stop with +something so I can see it there + +0:44:29.779,0:44:31.489 +doing computing the right thing + +0:44:31.489,0:44:33.169 +but in theory + +0:44:33.169,0:44:35.919 +that wouldn't be necessary + +0:44:35.919,0:44:38.650 +outside of that class of programs + +0:44:38.650,0:44:42.670 +everything needs some sort of descriptors and +of course we'll talk about descriptors + +0:44:42.670,0:44:43.659 +quite extensively + +0:44:43.659,0:44:47.349 +%uh as we go through the I/O subsystem + +0:44:47.349,0:44:50.969 +okay so the executive summary is that processes +are + +0:44:50.969,0:44:54.969 +the fundamental service that is provided by +UNIX + +0:44:54.969,0:44:58.430 +and + +0:44:58.430,0:45:02.849 +what we're going to spend essentially the +next two and a half weeks working on + +0:45:02.849,0:45:04.769 +is + +0:45:04.769,0:45:07.079 +what what makes up processes + +0:45:07.079,0:45:10.180 +we'll go into much more detail about each of these +four points + +0:45:10.180,0:45:11.769 +and + +0:45:11.769,0:45:13.630 +then how do we actually go about + +0:45:13.630,0:45:14.390 +providing that + +0:45:14.390,0:45:16.639 +bit of service + +0:45:16.639,0:45:17.900 +the next thing that I + +0:45:17.900,0:45:22.210 +going to do now is this go through and lay +out some of the terminology that + +0:45:22.210,0:45:23.239 +we have we when + +0:45:23.239,0:45:25.130 +we're talking about processes + +0:45:25.130,0:45:29.229 +so this is sort of the big picture here were +on page eighteen + +0:45:29.229,0:45:30.669 +and + +0:45:30.669,0:45:33.669 +you can see we have sort of three bits that +make up + +0:45:33.669,0:45:36.640 +the system + +0:45:36.640,0:45:39.029 +the currently running user process + +0:45:39.029,0:45:41.180 +and then what we call the top half of the kernel + +0:45:41.180,0:45:43.699 +and the bottom half of the kernel + +0:45:43.699,0:45:47.049 +now this would be a picture for a uniprocessor + +0:45:47.049,0:45:49.299 +so one CPU + +0:45:49.299,0:45:51.209 +if we had a multiprocessor + +0:45:51.209,0:45:54.009 +%uh then we would have + +0:45:54.009,0:45:57.130 +one instance of the kernel + +0:45:57.130,0:45:59.529 +but multiple instances of the user process + +0:45:59.529,0:46:02.879 +but for any given CPU on a multiprocessor + +0:46:02.879,0:46:05.709 +it is running exactly one process + +0:46:05.709,0:46:09.309 +so you may think they were running for four-five +processes all at once + +0:46:09.309,0:46:14.319 +but the fact of the matter is that any instant +in time there's only one process which is + +0:46:14.319,0:46:16.299 +actually running + +0:46:16.299,0:46:18.609 +and + +0:46:18.609,0:46:21.429 +that is the one that we have within the system + +0:46:21.429,0:46:25.199 +now we give the illusion that were running +lots of things because we switch between them + +0:46:25.199,0:46:26.100 +rather quickly + +0:46:26.100,0:46:29.269 +so it looks like things are happening in all +windows at once + +0:46:29.269,0:46:31.430 +but in reality + +0:46:31.430,0:46:33.619 +that's not really happening + +0:46:33.619,0:46:36.440 +okay there is a set of properties that I want to +look at + +0:46:36.440,0:46:40.899 +that had to do with each one of these parts here + +0:46:40.899,0:46:44.359 +but that just to look at it from the +big picture perspective + +0:46:44.359,0:46:45.970 +what you see here + +0:46:45.970,0:46:47.180 +is + +0:46:47.180,0:46:51.549 +there is boundary between the user process +and the top half of the kernel + +0:46:51.549,0:46:54.949 +which is really just like a glorified sovereignty +call + +0:46:54.949,0:46:59.539 +it's a lot like calling into a library routine +like calling strcat, strcpy or something + +0:46:59.539,0:47:00.319 +like that + +0:47:00.319,0:47:03.679 +%uh when you do a system call + +0:47:03.679,0:47:05.650 +we take that same set of parameters + +0:47:05.650,0:47:08.009 +now this is sort of + +0:47:08.009,0:47:09.780 +brick Wall here if you will + +0:47:09.780,0:47:11.380 +that is protecting + +0:47:11.380,0:47:13.680 +the the top half of the kernel + +0:47:13.680,0:47:15.299 +from the application + +0:47:15.299,0:47:18.899 +I'm more go into some detail about how that +actually gets implemented + +0:47:18.899,0:47:22.729 +but in a sense you can think of it +is is there sort of this whaling Wall with little + +0:47:22.729,0:47:24.990 +chinks there you can sort of push a request +through + +0:47:24.990,0:47:28.230 +and somebody other sites where polls that +looks at it and decide whether they're going + +0:47:28.230,0:47:28.690 +to + +0:47:28.690,0:47:30.769 +dained to provide service to you + +0:47:30.769,0:47:34.229 +and if they do then they will send it back + +0:47:34.229,0:47:37.649 +well like a library where you can just sort +of reach in and walk around if you want to + +0:47:37.649,0:47:38.290 +you + +0:47:38.290,0:47:40.950 +good programming practices you don't do that +but + +0:47:40.950,0:47:43.049 +you could + +0:47:43.049,0:47:44.579 +all right so + +0:47:44.579,0:47:49.089 +the the top half of the kernel is really looks +a lot like + +0:47:49.089,0:47:50.509 +a big library + +0:47:50.509,0:47:53.509 +%uh it just happens to be a library of +routines + +0:47:53.509,0:47:57.599 +that deal with things where processes need +to interact with each other + +0:47:57.599,0:48:01.399 +in fact for many people they don't understand +for what's the difference between the C + +0:48:01.399,0:48:03.259 +library and the top half of the kernel + +0:48:03.259,0:48:08.020 +%uh if it's something that you're doing that +no other process needs to know about + +0:48:08.020,0:48:09.799 +then it can be in the C library + +0:48:09.799,0:48:13.829 +so if you call strcat to concatenate two +strings together + +0:48:13.829,0:48:17.599 +nobody else needs to know you're doing that +you don't need to coordinate with anybody + +0:48:17.599,0:48:19.000 +else that you're doing that + +0:48:19.000,0:48:20.160 +it's just happening + +0:48:20.160,0:48:21.979 +so that goes in the C library. + +0:48:21.979,0:48:24.489 +on the other hand if you're reading or writing +the file + +0:48:24.489,0:48:28.029 +there may be other processes that are also +reading and writing that file + +0:48:28.029,0:48:29.910 +and therefore that + +0:48:29.910,0:48:31.579 +has to be done by the kernel + +0:48:31.579,0:48:33.120 +because they can coordinate + +0:48:33.120,0:48:37.189 +on the daemon processes that are trying to access +that file. + +0:48:37.189,0:48:40.529 +so the top of the kernel is pretty straightforward +code + +0:48:40.529,0:48:45.539 +it looks a lot like any other library that +you would write if you look at top half kernel + +0:48:45.539,0:48:49.640 +code you know you see all read,come in +it's got these parameters we Mark around we + +0:48:49.640,0:48:53.719 +get some data that we put it in the buffer and +we return back + +0:48:53.719,0:48:57.470 +in fact writing code for the top half of +the kernel is + +0:48:57.470,0:48:59.729 +not all that difficult to do + +0:48:59.729,0:49:00.989 +%uh it's + +0:49:00.989,0:49:01.959 +you have + +0:49:01.959,0:49:05.939 +for many the same properties that you would +when you're writing user level application + +0:49:05.939,0:49:07.529 +code + +0:49:07.529,0:49:11.779 +the bottom half of the kernel is where things +start to get nasty + +0:49:11.779,0:49:14.820 +thus the bottom half of the kernel is the part +of the system + +0:49:14.820,0:49:18.769 +that deals with all of the asynchronous events +in the system + +0:49:18.769,0:49:22.179 +%uh is things like device drivers, + +0:49:22.179,0:49:23.779 +timers + +0:49:23.779,0:49:25.010 +that level of thing + +0:49:25.010,0:49:28.029 +that are driven by hardware events + +0:49:28.029,0:49:28.659 +so + +0:49:28.659,0:49:31.459 +for example a packet arrives on the network + +0:49:31.459,0:49:33.670 +that causes and interrupt a command + +0:49:33.670,0:49:36.729 +that will be handled by the bottom half of +the kernel + +0:49:36.729,0:49:38.829 +and historically + +0:49:38.829,0:49:43.079 +when an interrupt came in it preempted whatever +else was going on + +0:49:43.079,0:49:45.400 +and it ran until it's finished and then return + +0:49:45.400,0:49:46.539 +and it could not + +0:49:46.539,0:49:49.439 +go to sleep to wait for resources or other +things + +0:49:49.439,0:49:51.339 +%uh in current systems + +0:49:51.339,0:49:54.549 +you can actually go to sleep in a interrupt driver +and wait for + +0:49:54.549,0:49:56.739 +some other activity to complete + +0:49:56.739,0:49:58.259 +it is however + +0:49:58.259,0:50:00.799 +not a good idea to do that + +0:50:00.799,0:50:01.909 +because + +0:50:01.909,0:50:06.739 +the usual case of most of device drivers is they +can finish whatever they're doing in an interrupt + +0:50:06.739,0:50:08.579 +without ever blocking + +0:50:08.579,0:50:09.580 +and so + +0:50:09.580,0:50:13.649 +when an interrupt comes in and we assume that you're +not going to sleep + +0:50:13.649,0:50:14.710 +and if you actually + +0:50:14.710,0:50:17.219 +then go to sleep.oh man + +0:50:17.219,0:50:20.469 +you didnt tell us you're going to do this we +have to go off into a whole lot of other work + +0:50:20.469,0:50:23.029 +that we had originally planned on doing + +0:50:23.029,0:50:25.460 +so if you go to sleep in a device driver + +0:50:25.460,0:50:28.209 +you are taking a very serious performance + +0:50:28.209,0:50:31.019 +so it's highly recommended that you don't +do that + +0:50:31.019,0:50:33.130 +but if you have to you can + +0:50:33.130,0:50:35.809 +on it's because of this historic behavior +or + +0:50:35.809,0:50:39.899 +of not being able to sleep in the bottom half +of the kernel + +0:50:39.899,0:50:42.119 +certain properties that have + +0:50:42.119,0:50:44.769 +%uh taken over in device drivers + +0:50:44.769,0:50:45.940 +and that is + +0:50:45.940,0:50:50.369 +that a device driver should be handed all +the resources it needs to get his job done + +0:50:50.369,0:50:54.490 +you don't give a disk device driver +Go read this + +0:50:54.490,0:50:56.549 +and put it somewhere + +0:50:56.549,0:50:57.580 +you have to say + +0:50:57.580,0:50:59.410 +Go read this particular block + +0:50:59.410,0:51:02.650 +here is a chunk of memory that I want that +my data to put in + +0:51:02.650,0:51:03.959 +and + +0:51:03.959,0:51:06.169 +no to find it when it's done + +0:51:06.169,0:51:06.970 +because + +0:51:06.970,0:51:10.660 +things like allocating memory are classic +places where you end up having to go to sleep + +0:51:10.660,0:51:12.939 +to wait for stuff to happen + +0:51:12.939,0:51:14.449 +and + +0:51:14.449,0:51:16.390 +historically you couldn't do that + +0:51:16.390,0:51:18.640 +even currently don't want to have to do that + +0:51:18.640,0:51:23.400 +so device drivers generally have all +resources pre allocated + +0:51:23.400,0:51:25.169 +and then they can just go + +0:51:25.169,0:51:27.279 +the one place where this doesn't work + +0:51:27.279,0:51:29.029 +is the network + +0:51:29.029,0:51:30.929 +and in particular + +0:51:30.929,0:51:34.630 +you don't know when somebody's going to send +packets to you + +0:51:34.630,0:51:37.040 +you say well you're looking you're open connections + +0:51:37.040,0:51:39.360 +but if you're doing something like IP forwarding + +0:51:39.360,0:51:40.969 +there's no + +0:51:40.969,0:51:45.039 +top half state it's dealing with this packets +they're just coming in on one interface being + +0:51:45.039,0:51:46.719 +sent out on another interface + +0:51:46.719,0:51:50.630 +they never pass through any part of the top +half of the kernel + +0:51:50.630,0:51:53.529 +and so in the case of network device drivers + +0:51:53.529,0:51:56.149 +they need to allocate memory + +0:51:56.149,0:51:56.640 +and + +0:51:56.640,0:51:58.829 +it memory gets into short supply + +0:51:58.829,0:52:01.689 +and they try to allocate memory and it's not +available + +0:52:01.689,0:52:05.049 +they historically coudnt wait for memory to be +available + +0:52:05.049,0:52:08.380 +and even in practice today don't wait + + +0:52:08.380,0:52:09.580 +for memory to become available + +0:52:09.580,0:52:12.469 +they simply drop the packet on the floor + +0:52:12.469,0:52:18.109 +it's like well I didn't have any place to +put it sorry oops + +0:52:18.109,0:52:20.940 +now that doesn't called incorrect behavior + +0:52:20.940,0:52:24.369 +because a higher level protocols what we transmit + +0:52:24.369,0:52:29.140 +but it does cause great performance problems +because we transmissions means that connections + +0:52:29.140,0:52:29.879 +stall + +0:52:29.879,0:52:31.110 +they have to back up + +0:52:31.110,0:52:33.010 +they have to resend data + +0:52:33.010,0:52:33.739 +and so on + +0:52:33.739,0:52:38.739 +so you really want to avoid dropping packets +if you can possibly help it + +0:52:38.739,0:52:42.029 +and consequently + +0:52:42.029,0:52:43.420 +we tend to + +0:52:43.420,0:52:46.499 +pre allocate a certain amount of memory for +the network drivers + +0:52:46.499,0:52:48.299 +%uh and + +0:52:48.299,0:52:52.169 +we try very hard to make sure that we're not +going to run out of memory but + +0:52:52.169,0:52:54.869 +if packets come fast enough and we can't deal +with them + +0:52:54.869,0:52:57.940 +as quickly as they are arriving over short period +of time + +0:52:57.940,0:53:03.489 +we get to the point where we simply have to start +dropping packets + +0:53:03.489,0:53:07.649 +this is a part of kernel that you do not wish to +write code for + +0:53:07.649,0:53:10.919 +because it is extremely difficult +debug + +0:53:10.919,0:53:12.759 +you get these bugs where + +0:53:12.759,0:53:18.779 +the only time it happens is on the third Tuesday +when there's a full moon + +0:53:18.779,0:53:19.300 +and + +0:53:19.300,0:53:24.199 +we have a disk interrupt followed by %uh a +terminal character coming in + +0:53:24.199,0:53:28.289 +and the network packet arriving of size fifteen +twenty two + +0:53:28.289,0:53:30.109 +and when all those things happened + +0:53:30.109,0:53:32.719 +the system panics + +0:53:32.719,0:53:37.380 +and of course there's like the panic +cause you're following some bad pointer + +0:53:37.380,0:53:40.969 +something that should have been there +but was freed some time in the distant past + +0:53:40.969,0:53:42.930 +we are not sure when + +0:53:42.930,0:53:44.049 +and + +0:53:44.049,0:53:47.400 +try to debug things like that is extremely +difficult + +0:53:47.400,0:53:48.509 +mean you can + +0:53:48.509,0:53:52.120 +think well I think I found the problem but +it's not reproduceable + +0:53:52.120,0:53:55.530 +you know you have to wait for the next third +Tuesday with a full moon and blah blah blah + +0:53:55.530,0:53:56.950 +to happen + +0:53:56.950,0:53:57.469 +and + +0:53:57.469,0:54:01.449 +you know so you sort of statistically +gets that you fix that you know I was getting + +0:54:01.449,0:54:03.510 +this bug once every three days + +0:54:03.510,0:54:06.099 +and now it's gone for two weeks without happening + +0:54:06.099,0:54:07.239 +did you fix that? + +0:54:07.239,0:54:08.969 +or if you've been lucky + +0:54:08.969,0:54:10.459 +and and it's + +0:54:10.459,0:54:14.349 +that that coupled with the fact that you're +dealing with hardware + +0:54:14.349,0:54:18.049 +and hardware really works the way it's documented +to work + +0:54:18.049,0:54:21.770 +and so you know they're doing everything that +it says you're supposed to do + +0:54:21.770,0:54:26.260 +it still doesn't work because you didn't set +the fiddle bit over on that other place over + +0:54:26.260,0:54:26.660 +there + +0:54:26.660,0:54:30.479 +that's not documented anywhere but if it's +not said it doesn't work + +0:54:30.479,0:54:33.769 +occasionally + +0:54:33.769,0:54:36.110 +this is another reason that you really want +of avoid + +0:54:36.110,0:54:40.459 +dealing with this part of the system if +you can possibly help + +0:54:40.459,0:54:44.369 +but lets go through and and look at some +of the properties here starting up with at + +0:54:44.369,0:54:45.789 +the user process + +0:54:45.789,0:54:47.980 +%uh we're running with + +0:54:47.980,0:54:51.449 +preemptive scheduling + +0:54:51.449,0:54:53.409 +now there's several caveats here + +0:54:53.409,0:54:55.239 +preemptive scheduling is the default + +0:54:55.239,0:54:56.970 +so called shared scheduler + +0:54:56.970,0:55:01.360 +that is what you normally use there are other +schedulers like the real time scheduler + +0:55:01.360,0:55:02.869 +where what I'm saying isnt that true + +0:55:02.869,0:55:05.709 +%uh will talk about some of the schedulers was +later + +0:55:05.709,0:55:09.930 +but the usual scheduler that you're running +on under UNIX is a shared scheduler + +0:55:09.930,0:55:13.229 +and under the shared scheduler the user applications + +0:55:13.229,0:55:15.159 +run with pre emptive scheduling + +0:55:15.159,0:55:17.449 +and pre emptive scheduling means that + +0:55:17.449,0:55:20.019 +you run at the whim of the system + +0:55:20.019,0:55:21.420 +if they want you to run + +0:55:21.420,0:55:22.140 +you run + +0:55:22.140,0:55:25.490 +once you start running you have no guarantee +of how long you're going to run + +0:55:25.490,0:55:29.370 +it might like to run for three instructions +and then decide it doesn't like you many more + +0:55:29.370,0:55:31.150 +it wants to run something else + +0:55:31.150,0:55:35.920 +while you might get to run for several seconds +and in a row with the with no intervening + +0:55:35.920,0:55:37.469 +things interrupting + +0:55:37.469,0:55:39.719 +you just don't know + +0:55:39.719,0:55:40.969 +and + +0:55:40.969,0:55:42.839 +really all you know is + +0:55:42.839,0:55:43.569 +that + +0:55:43.569,0:55:48.239 +%uh they claim that they're using statistics +and that and that the statistics are fair + +0:55:48.239,0:55:55.059 +and so on average you're going to get a reasonable +amount of time but thats + +0:55:55.059,0:55:57.129 +up to the system you don't control that + +0:55:57.129,0:55:58.439 +the real point here + +0:55:58.439,0:56:01.940 +is that you don't have any way of creating +a critical section + +0:56:01.940,0:56:04.950 +you can say okay I don't want to be interrupted + +0:56:04.950,0:56:07.429 +during this particular sequence of things + +0:56:07.429,0:56:09.809 +so you have to program + +0:56:09.809,0:56:13.469 +assuming that you may be interrupted at any +point + +0:56:13.469,0:56:14.979 +okay + +0:56:14.979,0:56:18.909 +the next thing is that when you're running +in user process + +0:56:18.909,0:56:20.719 +you are running in + +0:56:20.719,0:56:24.150 +with the processor in what's called unprivileged +mode + +0:56:24.150,0:56:28.109 +one of the requirements for running any kind +of a UNIX system + +0:56:28.109,0:56:31.759 +is that you have to have a processor that +support privileged and unprivileged + +0:56:31.759,0:56:33.709 +two different modes of operation + +0:56:33.709,0:56:37.049 +in privilege mode which is what the kernel +runs in + +0:56:37.049,0:56:38.950 +the entire repertoire + +0:56:38.950,0:56:40.869 +of the hardware is available + +0:56:40.869,0:56:45.339 +by this I mean you can set all the register +as you can fiddle with the memory management + +0:56:45.339,0:56:47.460 +unit you can initiate I/O + +0:56:47.460,0:56:50.519 +you can access any register anywhere + +0:56:50.519,0:56:51.919 +etc + +0:56:51.919,0:56:56.540 +when you're running in unprivileged +mode which is what user process run in and + +0:56:56.540,0:57:00.709 +this a large subset of the instructions which +you cannot execute + +0:57:00.709,0:57:03.480 +you cannot initiate I/O on + +0:57:03.480,0:57:04.209 +devices + +0:57:04.209,0:57:06.770 +you cannot change the memory mapping + +0:57:06.770,0:57:10.209 +you cannot access memory that's not part of +your address space + +0:57:10.209,0:57:13.299 +%uh you cannot execute certain instructions +like halt + +0:57:13.299,0:57:15.589 +%uh and + +0:57:15.589,0:57:19.039 +so in general you are protected + +0:57:19.039,0:57:21.789 +from manipulating anything that's outside of your +address space + +0:57:21.789,0:57:23.759 +this of course is desirable because + +0:57:23.759,0:57:27.059 +%uh when you're running in this unprevileged +mode + +0:57:27.059,0:57:28.300 +you're protected + +0:57:28.300,0:57:31.910 +from other processes manipulating you +and they're protected from you manipulating + +0:57:31.910,0:57:33.079 +them + +0:57:33.079,0:57:36.430 +for those of you that have had that misfortune +to have to use + +0:57:36.430,0:57:39.339 +early versions of windows up to about ninety +eight + +0:57:39.339,0:57:42.470 +they always ran with the processor +running in privileged mode + +0:57:42.470,0:57:44.009 +even in applications + +0:57:44.009,0:57:46.459 +and so either maliciously or accidentally + +0:57:46.459,0:57:50.000 +you could stop on other people address space +or you could stop on the kernel + +0:57:50.000,0:57:53.020 +and a lot of the blue screen of death was +people just + +0:57:53.020,0:57:56.319 +following wild pointers and trashing different +parts of the system + +0:57:56.319,0:57:58.819 +taking everything down + +0:57:58.819,0:58:00.020 +it also makes it + +0:58:00.020,0:58:02.320 +far easier to + +0:58:02.320,0:58:05.459 +implement things like viruses and worms and +other things because + +0:58:05.459,0:58:09.619 +user application can we rewrite the boot +block on the disk they can just the write down + +0:58:09.619,0:58:13.109 +and manipulate the registers that allow them +to do whatever they want + +0:58:13.109,0:58:16.730 +whereas when you're running in unprivileged +mode you cant write those kinds of + +0:58:16.730,0:58:20.179 +of things + +0:58:20.179,0:58:24.119 +modern versions of Windows from about +2000 on + +0:58:24.119,0:58:26.630 +now run with privileged and unprevileged mode + +0:58:26.630,0:58:28.649 +but UNIX has always required that + +0:58:28.649,0:58:30.219 +and so when you're running an + +0:58:30.219,0:58:31.319 +the user process + +0:58:31.319,0:58:33.389 +you cannot block i mean + +0:58:33.389,0:58:37.969 +you cannot execute the instructions which +cause a context switching to occur + +0:58:37.969,0:58:40.349 +you can't pick what's going to run next + +0:58:40.349,0:58:43.140 +you can't make that thing run next all you can +do + +0:58:43.140,0:58:45.189 +is go to the operating system and say + +0:58:45.189,0:58:49.269 +hey I've got nothing to do. pick somebody else +to run + +0:58:49.269,0:58:53.449 +the operating system is the think they can +then execute the instructions which cause + +0:58:53.449,0:58:57.609 +a different process to be loaded + +0:58:57.609,0:58:59.049 +and run + +0:58:59.049,0:59:03.400 +alright.finally while you're in a user application you're +running on a user stack + +0:59:03.400,0:59:06.410 +that's part of the user address space + +0:59:06.410,0:59:07.889 +so + +0:59:07.889,0:59:10.819 +part of creating a process gives you a runtime +stack + +0:59:10.819,0:59:14.369 +as part of a virtual address space and so it +can be + +0:59:14.369,0:59:18.199 +more or less up to the limits of the hardware +as big as you want it to be + +0:59:18.199,0:59:19.949 +so if you are running on thirty two-bit processor + +0:59:19.949,0:59:22.819 +you're stack can get 2 gigabytes + +0:59:22.819,0:59:23.319 +and + +0:59:23.319,0:59:26.839 +%uh the what this means is that anytime you +allocate local variables + +0:59:26.839,0:59:28.529 +you don't have to worry about Oh + +0:59:28.529,0:59:30.609 +is that gonna overrun my stack? + +0:59:30.609,0:59:31.610 +so if you need + +0:59:31.610,0:59:35.519 +a hundred thousand double precision floating +point numbers + +0:59:35.519,0:59:37.189 +you can just as a local variable allocate + +0:59:37.189,0:59:40.269 +an array of size a hundred-thousand type +double + +0:59:40.269,0:59:44.029 +and it just decrements your stack pointer by +hundred hundred thousand bytes + +0:59:44.029,0:59:45.009 +a way to go + +0:59:45.009,0:59:47.299 +it's just virtual address space + +0:59:47.299,0:59:49.020 +as you'll see when we get into the kernel + +0:59:49.020,0:59:50.210 +that ceases to be the case diff --git a/en_US.ISO8859-1/captions/2008/asiabsdcon/olander-pcbsd.sbv b/en_US.ISO8859-1/captions/2008/asiabsdcon/olander-pcbsd.sbv new file mode 100644 index 0000000000..95df19c09b --- /dev/null +++ b/en_US.ISO8859-1/captions/2008/asiabsdcon/olander-pcbsd.sbv @@ -0,0 +1,1732 @@ +0:00:05.120,0:00:08.599 +Our first talk is entitled + +0:00:08.599,0:00:12.079 +FreeBSD on the desktop by Matt Orlander + +0:00:12.079,0:00:16.409 +from iXSystems + +0:00:16.409,0:00:17.300 +>> MATT: Thank you. Good morning + +0:00:17.300,0:00:23.349 +I'd like to thank Sato San and George +and the rest of the Asia BSDCon organizers +for inviting us out + +0:00:23.349,0:00:27.099 +we have a little FreeBSD table + +0:00:27.099,0:00:29.189 +and have brought some goodies like + +0:00:29.189,0:00:30.380 +little beasties + +0:00:30.380,0:00:33.010 +you can put in your pocket and walk around Tokyo + +0:00:33.010,0:00:34.110 +and scare people with + +0:00:34.110,0:00:38.339 +%uh, any BSD committers, come on up, and we have +T-shirts for you + +0:00:38.339,0:00:41.490 +and it'll cost - for free - just to say thank +you + +0:00:41.490,0:00:42.729 +%uh this is + +0:00:42.729,0:00:44.640 +FreeBSD on the Desktop + +0:00:44.640,0:00:49.050 +I'm lucky enough to have - + +0:00:49.050,0:00:53.330 +Chris Moore, the founder of the PC-BSD +project here + +0:00:53.330,0:00:56.970 +so really, I don't know why I'm +giving this talk Chris + +0:00:56.970,0:00:59.060 +well, my name is Matt Orlander + +0:00:59.060,0:01:02.940 +I've been working with the BSD since +1998 + +0:01:02.940,0:01:04.970 +when I - uh - + +0:01:04.970,0:01:10.370 +I installed a BSD OS installation at a +little server company in San Jose California + +0:01:10.370,0:01:13.310 +called Telenet System Solutions + +0:01:13.310,0:01:14.280 +later on, I ended up + +0:01:14.280,0:01:16.090 +working full time there + +0:01:16.090,0:01:19.320 +and that they eventually got acquired by BSDi + +0:01:19.320,0:01:23.770 +and so I went on to continue working there + +0:01:23.770,0:01:27.549 +BSDi sold their software assets to WindRiver + +0:01:27.549,0:01:32.540 +and the hardware company lived on with a group +of employees that bought it out + +0:01:32.540,0:01:35.420 +and eventually changed our name back to iXsystems + +0:01:35.420,0:01:39.450 +and now we are very involved trying to support +the FreeBSD project + +0:01:39.450,0:01:41.950 +as well as the other BSDs + +0:01:41.950,0:01:44.229 +I'm on the BSD marketing team + +0:01:44.229,0:01:47.250 +some would say that I don't do enough for that + +0:01:47.250,0:01:50.700 +and of course I do some PC-BSD project +management as well + +0:01:50.700,0:01:56.120 +so people ask us a lot of times +why did we do PC-BSD +Chris can give you + +0:01:56.120,0:01:57.649 +a really good answer about + +0:01:57.649,0:01:59.909 +how he wanted to convert his Mom + +0:01:59.909,0:02:04.360 +and he tried actually to do this +on Linux at first and Linux is such + +0:02:04.360,0:02:10.439 +a fragmented mass, it was much easier for +him to turn to BSD and focus on one + +0:02:10.439,0:02:11.949 +core project + +0:02:11.949,0:02:15.549 +%uh the other reason I usually give is because I + +0:02:15.549,0:02:17.769 +chicks dig BSD + +0:02:17.769,0:02:20.369 +as you can see + +0:02:20.369,0:02:23.779 +that's PC-BSD babe - I don't know if you +can read it + +0:02:23.779,0:02:27.819 +but her site is pcbsdbabe.com +and she was a girl + +0:02:27.819,0:02:32.839 +that installed PC-BSD on one of the early +releases and loved it so much that she + +0:02:32.839,0:02:34.359 +she started a blog + +0:02:34.359,0:02:36.899 +it is pretty entertaining + +0:02:36.899,0:02:40.079 +I won't pull it up here + +0:02:40.079,0:02:44.560 +what is the PC-BSD - I meant to delete that, I swear + +0:02:44.560,0:02:48.159 +PC-BSD it is FreeBSD, but it's not a fork + +0:02:48.159,0:02:51.450 +all we do is take the default FreeBSD +operating system + +0:02:51.450,0:02:54.489 +and add a graphical installer + +0:02:54.489,0:02:59.079 +you know, it's integrated with KDE and +Fluxbox right out of the box + +0:02:59.079,0:03:03.040 +and of course we've got some tools that +Chris and his team have written + +0:03:03.040,0:03:07.159 +to make some of the typical system configuration + +0:03:07.159,0:03:09.729 +utilities a bit easier to use through a GUI + +0:03:09.729,0:03:12.979 +and I'll be showing you some of those + +0:03:12.979,0:03:15.570 +the big claim to fame for PC-BSD + +0:03:15.570,0:03:19.319 +is a different package management system +called PBI + +0:03:19.319,0:03:22.260 +and this is a graphical method of installing + +0:03:22.260,0:03:25.840 +say, like a FreeBSD port or more like a package + +0:03:25.840,0:03:28.759 +in a binary format using a front-end GUI + +0:03:28.759,0:03:31.949 +it contains all the dependencies of the package + +0:03:31.949,0:03:33.519 +in one directory + +0:03:33.519,0:03:37.699 +%uh so that we can save the package through +upgrades and + +0:03:37.699,0:03:40.139 +%uh make it very easy for other users + +0:03:40.139,0:03:41.099 +to install + +0:03:41.099,0:03:43.150 +we're working very hard + +0:03:43.150,0:03:48.059 +to integrate this more closely with FreeBSD's +methodology of ports and packages + +0:03:48.059,0:03:49.390 +and so far + +0:03:49.390,0:03:53.680 +Chris and his team have setup an auto-building PBI server + +0:03:53.680,0:03:56.759 +it's semi-auto, right? + +0:03:56.759,0:03:58.079 +>> CHRIS: it's pretty auto +>> MATT: pretty automatic now + +0:03:58.079,0:04:01.529 +it takes a little manual configuration of a port + +0:04:01.529,0:04:06.370 +and then it tears through the port and generates +the PC-BSD binary + +0:04:06.370,0:04:07.640 +and so now + +0:04:07.640,0:04:11.619 +whenever the port - how many are in their so far now Chris? + +0:04:11.619,0:04:13.219 +>> CHRIS: about a hundred +>> MATT: okay, so that's awesome + +0:04:13.219,0:04:16.989 +so I mean you know it's only hundred but +really for desktop use + +0:04:16.989,0:04:20.419 +%uh how many times you know how many applications +are going to be useful + +0:04:20.419,0:04:22.210 +for your average desktop user + +0:04:22.210,0:04:23.919 +%uh power users, you know, %uh + +0:04:23.919,0:04:30.050 +can just drop in and install a port +or a package so it's not a big deal + +0:04:30.050,0:04:31.860 +so Chris and his team have + +0:04:31.860,0:04:33.259 +setup scripts that tear through + +0:04:33.259,0:04:35.480 +%uh a hundred or so ports so far + +0:04:35.480,0:04:40.819 +and that whenever a port is updated +it will automatically generate the PC-BSD binary + +0:04:40.819,0:04:44.100 +and then it'll notify all of the PC-BSD users + +0:04:44.100,0:04:47.129 +with an annoying pop-up window + +0:04:47.129,0:04:48.219 +that updates are available + +0:04:48.219,0:04:52.889 +but of course you can turn that off + +0:04:52.889,0:04:55.150 +so the goal here is to make it + +0:04:55.150,0:04:57.290 +%uh easier for people to use + +0:04:57.290,0:04:59.870 +FreeBSD that are new to it especially + +0:04:59.870,0:05:03.280 +I think the jury that the PC-BSD community + +0:05:03.280,0:05:05.969 +as I look at forums and talk to people + +0:05:05.969,0:05:07.249 +is Windows users + +0:05:07.249,0:05:11.869 +so we're getting a lot of Windows users +coming on over and I think now we're +getting a lot of Linux users coming over + +0:05:11.869,0:05:14.040 +you know, they like Linux, but + +0:05:14.040,0:05:16.939 +they've tried twenty different flavors + +0:05:16.939,0:05:18.439 +they start to realize that + +0:05:18.439,0:05:22.849 +something's wrong + +0:05:22.849,0:05:25.739 +so we're now that at PC-BSD 1.5 + +0:05:25.739,0:05:28.289 +which includes Xorg 7.3 + +0:05:28.289,0:05:34.909 +3.5.8, actually, you're already bumped up to +like a 1.5.1, right? +[mumbles] + +0:05:34.909,0:05:39.249 +as soon as you get back, he's +been working on that at the hotel + +0:05:39.249,0:05:40.860 +we have a new system updater tool + +0:05:40.860,0:05:43.709 +that updates your PC-BSD system + +0:05:43.709,0:05:44.400 +on the fly + +0:05:44.400,0:05:47.610 +how does that work Chris? So you get +notified there's an update and + +0:05:47.610,0:05:52.229 +I got a call from my office in fact yesterday +that they had done that + +0:05:52.229,0:05:56.699 +the KMail call that I had to look into + +0:05:56.699,0:06:03.699 +but it just notifies you and you can just +download an ISO or download a patch +>> CHRIS: it notifies, downloads and applies the +patch to your system +>> MATT: brilliant +>> CHRIS: on the fly + +0:06:06.479,0:06:08.319 +>> MATT: and of course now we have a WiFi tool + +0:06:08.319,0:06:09.620 +which is really cool + +0:06:09.620,0:06:13.569 +it'll scan and you just double click on that +network + +0:06:13.569,0:06:16.080 +and enter your WPA or WEP key and + +0:06:16.080,0:06:19.489 +it'll remember these + +0:06:19.489,0:06:20.990 +%uh we've got + +0:06:20.990,0:06:22.900 +improvements to the PBI removal tool + +0:06:22.900,0:06:27.249 +who is similar to the add / remove programs +from Windows + +0:06:27.249,0:06:28.699 +so I'll show you that + +0:06:28.699,0:06:31.899 +%uh the new sound detection program + +0:06:31.899,0:06:36.339 +and we now finally have an amd64 build + +0:06:36.339,0:06:40.580 +as of PC-BSD 1.5 + +0:06:40.580,0:06:45.259 +so actually that you can follow along in the +your slides + +0:06:45.259,0:06:52.259 +I'm going to drop out of this and show +you the actual installation + +0:06:56.500,0:07:03.190 +it's like a cooking show + +0:07:03.190,0:07:06.349 +okay so here's our first screen after a very +typical looking + + +0:07:06.349,0:07:08.430 +FreeBSD bootstrap + +0:07:08.430,0:07:11.970 +and we have different languages and of +course keyboard layouts + +0:07:11.970,0:07:15.190 +and we do have Japanese and I believe that +Chris already installed + +0:07:15.190,0:07:16.730 +the Japanese version, is that right + +0:07:16.730,0:07:19.189 +so you have it running upstairs on the laptop? + +0:07:19.189,0:07:21.199 +oh right here? awesome + +0:07:21.199,0:07:25.249 +I would have done that but then I wouldn't +have been able to read anything + +0:07:25.249,0:07:28.009 +we default to allowing submitting + +0:07:28.009,0:07:31.370 +usage statistics to FreeBSD stats + +0:07:31.370,0:07:33.250 +and the reason we do that + +0:07:33.250,0:07:36.530 +is so that when I go to talk to companies +like Adobe + +0:07:36.530,0:07:41.589 +to persuade them to give us a native Flash 9 + +0:07:41.589,0:07:45.629 +which Kirk has already appeared out there + +0:07:45.629,0:07:51.019 +this gives me a place I can point them to +start to persuade for the business case + +0:07:51.019,0:07:53.060 +for them supporting + +0:07:53.060,0:07:54.370 +BSD natively + +0:07:54.370,0:07:59.419 +and I think really close to seeing that this +year + +0:07:59.419,0:08:01.710 +I think Desktop BSD are + +0:08:01.710,0:08:05.169 +I guess you can call them our competitor +although we don't really compete + +0:08:05.169,0:08:07.330 +we collaborate a bit + +0:08:07.330,0:08:12.179 +they just now started defaulting to allowing these +statistics and I saw a big jump in their + +0:08:12.179,0:08:15.659 +BSD stats + +0:08:15.659,0:08:19.860 +here is the license agreement, and all we have +here is the BSD license, some of the Intel +firmware licenses, + +0:08:19.860,0:08:26.860 +and then, that's about it + +0:08:28.249,0:08:33.390 +I'm going to go ahead and agree + +0:08:33.390,0:08:37.300 +here's where I can do a fresh install +or update my existing system + +0:08:37.300,0:08:40.250 +the update basically + +0:08:40.250,0:08:42.379 +%uh Chris, how does the update work? It TARs up + +0:08:42.379,0:08:44.780 +the user's home directory? + +0:08:44.780,0:08:49.000 +[mumbles] + +0:08:49.000,0:08:56.000 +and we've done it many times at work so far +and it works pretty darn well + +0:09:11.250,0:09:17.850 +so there you see, I didn't enter that correctly, +I was just testing that + +0:09:17.850,0:09:19.290 +so now the passwords match up + +0:09:19.290,0:09:21.020 +I go ahead and enter + +0:09:21.020,0:09:28.020 +a default user + +0:09:36.120,0:09:38.680 +you can choose your shells here + +0:09:38.680,0:09:42.180 +and Chris has added Bash as a default +here as well, because + +0:09:42.180,0:09:48.150 +well, the Linux users seem to prefer Bash + +0:09:48.150,0:09:50.980 +here's where I can toggle auto-login, +the Windows people + +0:09:50.980,0:09:56.740 +they're used to it + +0:09:56.740,0:09:57.819 +here's your partitioning + +0:09:57.819,0:10:02.950 +Chris has changed the partitioning a little bit +we used to have something where you actually could go in + +0:10:02.950,0:10:06.210 +and really tweak the partitioning and people were + +0:10:06.210,0:10:08.190 +things and so right now + +0:10:08.190,0:10:10.120 +it is kind of turned off for now + +0:10:10.120,0:10:12.580 +oh you can still do that, ok + +0:10:12.580,0:10:15.740 +what about what about adding + +0:10:15.740,0:10:17.610 +a partitioning tool something that + +0:10:17.610,0:10:19.080 +>> CHRIS: we've talked about it + +0:10:19.080,0:10:21.770 +>> MATT: it's dangerous, it's really dangerous + +0:10:21.770,0:10:25.220 +I'm going to go ahead, since I'm actually on a Mac, + +0:10:25.220,0:10:26.699 +I'm going to use the entire disk + +0:10:26.699,0:10:29.550 +I could customize and go setup my own + +0:10:29.550,0:10:32.060 +partitioning scheme + +0:10:32.060,0:10:35.820 +Chris, when are you going to bring back the server edition? + +0:10:35.820,0:10:40.180 +when we have enough tools to throw at it? +>> CHRIS: we don't have enough tools + +0:10:40.180,0:10:41.099 +>> MATT: what it was + +0:10:41.099,0:10:42.990 +we had a server edition + +0:10:42.990,0:10:44.900 +and we removed it as it wasn't really doing much yet + +0:10:44.900,0:10:49.150 +and basically it dropped you into a default +Fluxbox install + +0:10:49.150,0:10:50.830 +with minimal tools + +0:10:50.830,0:10:52.360 +so you could tweak it + +0:10:52.360,0:10:54.469 +once we bring that back, we'll have %uh + +0:10:54.469,0:10:55.230 +we'll have, based on iXsystems + +0:10:55.230,0:11:00.100 +based on the company's, the sponsors that +come out here + +0:11:00.100,0:11:04.300 +we'll throw together some default partitionings +that we see a lot of + +0:11:04.300,0:11:06.620 +as we %uh, as we ship out servers + +0:11:06.620,0:11:08.169 +and then have them in there as an option + +0:11:08.169,0:11:13.210 +like a LAMP server, a mail server, a DNS or what +have you + +0:11:13.210,0:11:14.390 +this is cool + +0:11:14.390,0:11:18.460 +so now I can go ahead and choose add-ons + +0:11:18.460,0:11:20.910 +and add them on to the system + +0:11:20.910,0:11:23.260 +it's a short list right now + +0:11:23.260,0:11:25.890 +but you've got some cool stuff + +0:11:25.890,0:11:27.299 +I grabbed the source in ports +we have %uh + +0:11:27.299,0:11:29.260 +Opera + +0:11:29.260,0:11:32.689 +they agreed within in forty eight hours +when I asked them + +0:11:32.689,0:11:34.180 +that we could ship with + +0:11:34.180,0:11:37.080 +their Opera binary +they said absolutely no problem + +0:11:37.080,0:11:39.890 +and Adobe agreed that we could ship with Flash + +0:11:39.890,0:11:40.889 +so we do + +0:11:40.889,0:11:43.040 +if you select any of these + +0:11:43.040,0:11:50.040 +%uh I'd have to insert the second disc +so I'll go ahead and skip it + +0:11:50.160,0:11:51.120 +and then, there you go + +0:11:51.120,0:11:57.120 +very secure + +0:11:57.120,0:12:04.120 +so we'll let this cook + +0:12:12.990,0:12:15.300 +so now like the cooking show + +0:12:15.300,0:12:19.020 +I put it in the oven +but I already have one finished + +0:12:19.020,0:12:22.060 +and so here's a finished PC-BSD installation + +0:12:22.060,0:12:26.530 +this is very close to what you would see by default +although I've already gone and installed a few things + +0:12:26.530,0:12:27.899 +noticeably + +0:12:27.899,0:12:32.040 +on the right this is a very %uh this is very +very cool when I show %uh + +0:12:32.040,0:12:35.860 +Windows web developers in particular love this + +0:12:35.860,0:12:39.220 +we can install what we call PAMP + +0:12:39.220,0:12:40.860 +I'm trying to come up with a better name + +0:12:40.860,0:12:43.180 +but for now it's PAMP + +0:12:43.180,0:12:46.880 +and that would of course by the +Apache, MySQL and PHP stack + +0:12:46.880,0:12:48.089 +what's interesting is + +0:12:48.089,0:12:52.590 +the guy, I don't know who wrote this for us +someone did this for us + +0:12:52.590,0:12:54.290 +you did it? + +0:12:54.290,0:12:56.140 +I do know the guy who did it + +0:12:56.140,0:12:58.750 +%uh I thought someone else did it originally? + +0:12:58.750,0:13:01.580 +did you update it? yeah? okay, great + +0:13:01.580,0:13:02.310 +this was great + +0:13:02.310,0:13:05.199 +I just took a class at UC Santa Cruz + +0:13:05.199,0:13:09.190 +I showed the teacher this +and she had me show the entire class + +0:13:09.190,0:13:11.130 +once I had showed her + +0:13:11.130,0:13:12.449 +literally two clicks + +0:13:12.449,0:13:15.980 +we install %uh Apache, PHP, MySQL + +0:13:15.980,0:13:18.060 +and then we get some nice little icons + +0:13:18.060,0:13:20.960 +%uh to start and to stop + +0:13:20.960,0:13:22.580 +we can %uh + +0:13:22.580,0:13:24.199 +just instantly switch + +0:13:24.199,0:13:26.440 +the PHP versions + +0:13:26.440,0:13:28.300 +with the %uh web devs really like + +0:13:28.300,0:13:30.660 +for testing + +0:13:30.660,0:13:32.790 +you get a little drop down to switch the version + +0:13:32.790,0:13:35.710 +especially, I noticed that even in this class + +0:13:35.710,0:13:40.449 +these guys are you know CSS wizards + +0:13:40.449,0:13:43.050 +and xHTML wizards and +they've never seen a command line + +0:13:43.050,0:13:43.959 +and so + +0:13:43.959,0:13:49.069 +everyone was looking at you know +this is my Mac that I use for testing +but I have a PC-BSD laptop + +0:13:49.069,0:13:50.280 +as well + +0:13:50.280,0:13:52.040 +for work +this is my personal Mac + +0:13:52.040,0:13:55.009 +but for work I use my PC-BSD laptop + +0:13:55.009,0:13:56.940 +and so I had that at the class + +0:13:56.940,0:14:00.110 +and I was able to share and +everyone was like what the +hell are you doing? + +0:14:00.110,0:14:02.190 +and so I gave them a quick demo + +0:14:02.190,0:14:02.950 +and %uh + +0:14:02.950,0:14:05.370 +you see here we can put a little icon + +0:14:05.370,0:14:06.299 +down in the tray + +0:14:06.299,0:14:09.770 +that lets us easily start and stop and monitor +the services + +0:14:09.770,0:14:13.420 +%uh for someone that's not familiar +with how these work + +0:14:13.420,0:14:14.989 +we can even go in and edit + +0:14:14.989,0:14:16.229 +the HTTPD.conf + +0:14:16.229,0:14:22.300 +or the PHP.ini + +0:14:22.300,0:14:26.970 +and so this is just one of the one hundred +packages that we have building + +0:14:26.970,0:14:29.050 +and this is a really cool one + +0:14:29.050,0:14:31.460 +and then they can open the web root + +0:14:31.460,0:14:35.040 +or open their personal web root +or toggle the web sites around + +0:14:35.040,0:14:37.549 +so a pretty clever tool for manipulating + +0:14:37.549,0:14:39.160 +Apache, MySQL or PHP + +0:14:39.160,0:14:41.220 +via the GUI + +0:14:41.220,0:14:44.060 +and it installs that %uh that + +0:14:44.060,0:14:48.230 +MySQL GUI admin tool that I've never used + +0:14:48.230,0:14:49.660 +yeah yeah they love that stuff + +0:14:49.660,0:14:52.110 +everyone in the class knew what that was + +0:14:52.110,0:14:54.410 +and I was like, what? + +0:14:54.410,0:14:56.940 +now let me show you the big deal + +0:14:56.940,0:14:58.279 +this is a PBI + +0:14:58.279,0:15:00.320 +that I downloaded + +0:15:00.320,0:15:04.690 +off of PBIDir.com + +0:15:04.690,0:15:06.960 +so this is the this is the GIM + +0:15:06.960,0:15:09.090 +and I'm going to install it + +0:15:09.090,0:15:11.330 +on a very fresh installation + +0:15:11.330,0:15:14.120 +the icon you get in the upper left hand corner + +0:15:14.120,0:15:16.090 +says Get PBIs and %uh + +0:15:16.090,0:15:19.880 +and it just opens up a Konqueror window +directly to the website + +0:15:19.880,0:15:22.210 +I've been talking to Chris about +maybe in the future + +0:15:22.210,0:15:23.089 +having the %uh + +0:15:23.089,0:15:26.820 +maybe grabbing the Desktop BSD's port tool + +0:15:26.820,0:15:28.170 +having that on a tab + +0:15:28.170,0:15:31.410 +and then having a PBI add / remove + +0:15:31.410,0:15:37.360 +on the other tab so that you can just browse +all the programs that you can install +via a port, package or PBI + +0:15:37.360,0:15:40.410 +on one single interface + +0:15:40.410,0:15:43.720 +so I'm going to install the GIMP + +0:15:43.720,0:15:50.720 +I'm going to agree to that nasty license + +0:15:59.090,0:16:01.530 +I love these graphics that Kinsalez does + +0:16:01.530,0:16:08.530 +ok and finished + +0:16:15.030,0:16:16.560 +and there you go + +0:16:16.560,0:16:18.460 +we got the GIMP on there + +0:16:18.460,0:16:21.310 +and it created of course a +menu icon + +0:16:21.310,0:16:23.480 +in here as well + +0:16:23.480,0:16:25.180 +under the GIMP + +0:16:25.180,0:16:31.910 +so now I could easily upgrade this +in fact when the PBI gets updated it will +notify me + +0:16:31.910,0:16:36.590 +that hey there's a new GIMP do you +want to download and install it right now? +and you just click yes and it + +0:16:36.590,0:16:38.540 +downloads and installs +and keeps %uh + +0:16:38.540,0:16:42.780 +your GIMP preferences or whatever +the application preferences that you +already have in there + +0:16:42.780,0:16:47.130 +and then since we're targeting mostly +you know, easy to use + +0:16:47.130,0:16:49.480 +desktop operating systems specifically + +0:16:49.480,0:16:52.869 +for Windows users so we have +a lot of familiar tools for them + +0:16:52.869,0:16:57.050 +if you go into the Start menu and look +at the settings, and here's my software + +0:16:57.050,0:17:00.880 +there's the add / remove that I was telling you about + +0:17:00.880,0:17:02.790 +and so I'll go in and + +0:17:02.790,0:17:09.790 +we will remove it + +0:17:10.190,0:17:12.959 +so here's all the software I have installed + +0:17:12.959,0:17:14.640 +you can see the GIMP there + +0:17:14.640,0:17:17.630 +and really since the GIMP + +0:17:17.630,0:17:21.090 +is really just installed in our + +0:17:21.090,0:17:26.370 +own little directory structure here + +0:17:26.370,0:17:27.610 +so you can see + +0:17:27.610,0:17:28.809 +the GIMP and + +0:17:28.809,0:17:30.009 +all its libs + +0:17:30.009,0:17:37.009 +are hiding in here + +0:17:37.820,0:17:39.530 +that's the %uh, I don't know if you can see that + +0:17:39.530,0:17:42.890 +but it's the automatic directory structure that's created + +0:17:42.890,0:17:49.890 +from the scripts that we use on the auto-building server + +0:17:50.270,0:17:52.570 +and so I go ahead and click remove + +0:17:52.570,0:17:56.970 +and of course all this is really doing +is an rm -rf on there + +0:17:56.970,0:17:58.940 +and bam - gone +and there's a little script + +0:17:58.940,0:18:05.940 +that pulls the icons out of there +and notifies you that it is gone + +0:18:10.330,0:18:15.470 +and we've got a services manager + +0:18:15.470,0:18:18.760 +this is actually PC-BSD 1.4 + +0:18:18.760,0:18:22.460 +but we've got 1.5 running upstairs + +0:18:22.460,0:18:27.850 +that didn't have any PBIs to show on 1.5 + +0:18:27.850,0:18:29.690 +and so here, of course, + +0:18:29.690,0:18:36.400 +anyone who has used Windows you can +see this looks very similar to whatever +that Windows tool is called + +0:18:36.400,0:18:38.540 +so we can start / stop services + +0:18:38.540,0:18:40.290 +that are enabled in the start-up + +0:18:40.290,0:18:41.350 +and so you know + +0:18:41.350,0:18:45.770 +it’s not like this can replace a system administrator +but it sure %uh + +0:18:45.770,0:18:49.580 +well I know that everyone at our office +runs this of course + +0:18:49.580,0:18:52.440 +and most of our family members now are running it + +0:18:52.440,0:18:54.980 +and I was down at + +0:18:54.980,0:18:56.580 +the Luxor in Las Vegas + +0:18:56.580,0:18:59.090 +and I was wearing my FreeBSD shirt + +0:18:59.090,0:19:00.870 +and I'm sitting at the pub + +0:19:00.870,0:19:01.760 +and this guy goes + +0:19:01.760,0:19:03.930 +you've heard of FreeBSD? + +0:19:03.930,0:19:05.120 +and I go 'yeah, dude' + +0:19:05.120,0:19:07.710 +he goes have you heard of PC-BSD? + +0:19:07.710,0:19:09.669 +and I thought it was really cool that + +0:19:09.669,0:19:15.010 +this random bartender down in Vegas is +telling me about PC-BSD. I go 'yeah, +I've heard of that'. + +0:19:15.010,0:19:19.020 +he goes 'we're running that at home, dude, it's cool' + +0:19:19.020,0:19:26.020 +so I went into the room and got some swag +for him and he was so happy + +0:19:29.350,0:19:33.530 +we've got our own users tool +we've got a WiFi tool + +0:19:33.530,0:19:35.170 +%uh + +0:19:35.170,0:19:39.540 +we did we do our own user management tool +over the KDE one? + +0:19:39.540,0:19:42.090 +>> CHRIS: it was the only sensible option + +0:19:42.090,0:19:42.910 +>> MATT: yeah + +0:19:42.910,0:19:47.410 +you know we're trying to get closer involved +with the KDE guys + +0:19:47.410,0:19:50.300 +so that they test a little more on FreeBSD + +0:19:50.300,0:19:53.010 +because we've discovered some of their stuff +just doesn't compile + +0:19:53.010,0:19:54.630 +or work properly + +0:19:54.630,0:19:57.760 +and we have to create BSD-specific tools for them + +0:19:57.760,0:20:02.080 +it would be nicer if we had a little +more collaboration there + +0:20:02.080,0:20:04.810 +there's your online update manager where I can +go + +0:20:04.810,0:20:06.620 +and manually check for updates + +0:20:06.620,0:20:10.090 +and install them + +0:20:10.090,0:20:13.910 +and I've got my %uh + +0:20:13.910,0:20:15.409 +my network settings + +0:20:15.409,0:20:19.799 +which pulls up the WiFi tool which is really +easy to use. I don't think it'll pull up + +0:20:19.799,0:20:21.039 +because my Mac + +0:20:21.039,0:20:23.170 +doesn't translate it + +0:20:23.170,0:20:26.750 +but it's pretty darn easy to use + +0:20:26.750,0:20:30.190 +actually, this is the old one huh? + +0:20:30.190,0:20:37.190 +we've got the new one upstairs +it's even better + +0:20:42.230,0:20:49.230 +almost done + +0:20:58.450,0:21:01.760 +ah yes, so we didn't see this yet +this is the X window configurator + +0:21:01.760,0:21:05.690 +and so this is really cool if you've +got a supported 3D graphics card + +0:21:05.690,0:21:06.970 +you can go ahead + +0:21:06.970,0:21:08.280 +and select it here + +0:21:08.280,0:21:11.220 +and be running Beryl right at startup + +0:21:11.220,0:21:13.160 +so when we do trade shows + +0:21:13.160,0:21:15.280 +which is part of what I do + +0:21:15.280,0:21:17.100 +to evangelize FreeBSD + +0:21:17.100,0:21:20.000 +we bring along a 32 inch LCD + +0:21:20.000,0:21:25.850 +and we throw PC-BSD up there with Beryl +running in it, and boy people just go +nuts over it + +0:21:25.850,0:21:27.160 +over those windows + +0:21:27.160,0:21:29.070 +you know, those squares, Beryl stuff + +0:21:29.070,0:21:32.380 +I'm over it already, but + +0:21:32.380,0:21:34.090 +and there's the boot screen + +0:21:34.090,0:21:36.940 +and the update manager +I've already covered the update + +0:21:36.940,0:21:39.600 +there's looking for a new PBI update + +0:21:39.600,0:21:43.530 +this was the test you were running for the FireFox +one before you rolled it out + +0:21:43.530,0:21:47.420 +so I was testing it and grabbing screenshots + +0:21:47.420,0:21:51.660 +and then it installed it and then +it told me it was finished + +0:21:51.660,0:21:53.500 +sure enough it fired right up +and worked like a charm + +0:21:53.500,0:21:55.040 +so, very cool + +0:21:55.040,0:21:57.460 +we just did this + +0:21:57.460,0:22:00.530 +in fact + +0:22:00.530,0:22:03.010 +the GIMP and we removed it + +0:22:03.010,0:22:06.370 +so, how can you help? + +0:22:06.370,0:22:07.580 +well you could grab a CD + +0:22:07.580,0:22:10.190 +today and install PC-BSD + +0:22:10.190,0:22:14.240 +and certainly one of the biggest contributions +any user can do + +0:22:14.240,0:22:15.980 +is reporting bugs back to us + +0:22:15.980,0:22:17.580 +I noticed, I don't know, last month + +0:22:17.580,0:22:23.810 +I think people had reported just a really small +list of bugs, but you troll the forums and +they're complaining in there + +0:22:23.810,0:22:26.900 +so you go on and say, please report the bug +where we will see it + +0:22:26.900,0:22:28.429 +because we can't be + +0:22:28.429,0:22:31.810 +trolling ten thousand posts every day + +0:22:31.810,0:22:35.540 +that would be one way you could help +documentation and translations + +0:22:35.540,0:22:37.370 +it would be nice if + +0:22:37.370,0:22:37.980 +%uh + +0:22:37.980,0:22:41.490 +I don't know, who did our Japanese translation? + +0:22:41.490,0:22:42.239 +we don't even know + +0:22:42.239,0:22:47.320 +it would be cool if a Japanese native speaker +could take a look at it and say +'hey, there are some errors here' + +0:22:47.320,0:22:48.010 +%uh + +0:22:48.010,0:22:49.050 +maybe it was a + +0:22:49.050,0:22:54.030 +%uh a British or an American guy who speaks Japanese +and maybe he made some mistakes + +0:22:54.030,0:22:57.700 +%uh some CDs out, download them, tell your friends + +0:22:57.700,0:22:58.999 +get your Mom on it + +0:22:58.999,0:23:01.890 +and prepare for all the phone calls + +0:23:01.890,0:23:03.320 +and of course evangelize + +0:23:03.320,0:23:08.299 +if you can do C, Qt, or Shell programming +we can always use help there + +0:23:08.299,0:23:10.340 +and you can just email the folk + +0:23:10.340,0:23:12.169 +or go to the forums + +0:23:12.169,0:23:15.059 +or the PC-BSD web site and see how to contact us + +0:23:15.059,0:23:19.530 +and there we have a core team list there very +similar to FreeBSD + +0:23:19.530,0:23:24.540 +so you can email the core team of PC-BSD +and everyone will see that + +0:23:24.540,0:23:27.460 +alright, so that's PC-BSD in a nutshell + +0:23:27.460,0:23:36.000 +does anyone have any questions 'cause we have Chris in the room + +0:23:36.000,0:23:35.450 +yes? + +0:23:35.450,0:23:39.750 +>> AUDIENCE MEMBER: like, so %uh +[unclear] + +0:23:39.750,0:23:43.890 +most of the computer is like a desktop +you know, it's an old computer + +0:23:43.890,0:23:48.890 +so, %uh, does this system have a +driver for the, like, the sound card + +0:23:48.890,0:23:54.090 +>> MATT: well this is FreeBSD 6.3 so +yeah, absolutely, FreeBSD + +0:23:54.090,0:23:56.070 +supports the older hardware really really well + +0:23:56.070,0:23:59.559 +from my experience + +0:23:59.559,0:24:02.190 +any other questions? + +0:24:02.190,0:24:06.299 +we have the FreeBSD table upstairs + +0:24:06.299,0:24:12.640 +so stop on by before I drink too much sake! + +0:24:12.640,0:24:17.770 +what's the latest on BSD stats? +I mean, there's always a certain +percentage of people + +0:24:17.770,0:24:19.250 +that turn off the reporting + +0:24:19.250,0:24:22.260 +for some weird reason + +0:24:22.260,0:24:24.650 +last month we had ten thousand installs + +0:24:24.650,0:24:26.859 +so you know - how many of those keep? + +0:24:26.859,0:24:30.859 +it's really hard to say, right +Yes? + +0:24:30.859,0:24:36.120 +>> AUDIENCE MEMBER: Matt, two questions +I'm from China + +0:24:36.120,0:24:40.220 +does FreeBSD or PC-BSD support Chinese? + +0:24:40.220,0:24:42.750 +>> CHRIS: we've had people translate into Chinese, yes + +0:24:42.750,0:24:55.360 +>> AUDIENCE MEMBER: we are interested in this project +because we've have seen that some consumers +we've tried to persuade them to use PC-BSD instead + +0:24:55.360,0:24:59.320 +>> MATT: this, this is great, we'd love to help you out +especially convincing Linux consumers + +0:24:59.320,0:25:02.130 +>> AUDIENCE MEMBER: can we, can we join this team? + +0:25:02.130,0:25:03.990 +>> MATT: yes absolutely + +0:25:03.990,0:25:05.199 +we would love to have you + +0:25:05.199,0:25:15.670 +>> AUDIENCE MEMBER: finally, can +PC-BSD have an alternative login like KDM? + +0:25:15.670,0:25:21.490 +>> MATT: yeah, it already does, it already does +so on the install, you just uncheck auto-login + +0:25:21.490,0:25:23.440 +and then it'll pull up with a customized KDM + +0:25:23.440,0:25:30.130 +>> AUDIENCE MEMBER: final question, +so does PC-BSD one disk? + +0:25:30.130,0:25:33.600 +>> MATT: yeah, I just install off +of this one disk + +0:25:33.600,0:25:36.820 +the second disk contains some optional + +0:25:36.820,0:25:41.400 +yes, some optional components as well as +some language translations + +0:25:41.400,0:25:43.860 +but yeah, if you want to change the language + +0:25:43.860,0:25:47.190 +during the install you would need the second disk + +0:25:47.190,0:25:49.160 +>> AUDIENCE MEMBER: can we contact +you after the conference? + +0:25:49.160,0:25:50.550 +>> MATT: absolutely and I would love it + +0:25:50.550,0:25:55.650 +we just had a school, an entire school district +in France switch over to PC-BSD so %uh + +0:25:55.650,0:25:58.900 +we're doing a case study + +0:25:58.900,0:25:59.539 +so we can throw that up on the FreeBSD site + +0:25:59.539,0:26:06.100 +they were ecstatic + +0:26:06.100,0:26:08.830 +yeah, they were absolutely ecstatic +and were all like, how can we help? + +0:26:08.830,0:26:13.160 +so all the servers in the city in France +were already on FreeBSD + +0:26:13.160,0:26:17.400 +so when the school district contacted +them to talk about + +0:26:17.400,0:26:20.980 +upgrading their Windows installs +they said why don't we do a test run + +0:26:20.980,0:26:25.960 +with the PC-BSD and see how you guys like it +and they said that the teachers and students loved it + +0:26:25.960,0:26:27.250 +so they rolled it out + +0:26:27.250,0:26:29.400 +and they rolled it out without having any issues + +0:26:29.400,0:26:34.179 +%uh we asked them, you know, I sent them a +bunch of questions and they responded +you know, what kind of issues did you have + +0:26:34.179,0:26:36.160 +how did you solve them? + +0:26:36.160,0:26:37.799 +how do you do upgrades + +0:26:37.799,0:26:39.980 +and so on and so on + +0:26:39.980,0:26:46.980 +the iXsystems marketing girl is +synthesing that into a case study that +we'll release shortly + +0:26:48.480,0:26:50.750 +absolutely + +0:26:50.750,0:26:51.809 +yes, Jason? + +0:26:51.809,0:26:56.760 +>> JASON: are there things like, %uh +remote home directories + +0:26:56.760,0:26:01.980 +to take your school example are +so students login to one PC and + +0:26:01.980,0:26:03.120 +they have the same home directory? + +0:26:03.120,0:26:05.080 +>> AUDIENCE MEMBER: like Terminal Server? + +0:26:05.080,0:26:10.150 +>> JASON: well, not necessarily a Terminal Server +but more, I don't know, let's go down the Windows path + +0:26:10.150,0:26:16.320 +what do they call it? they call it a Domain server +where you can bounce your home directories + +0:26:16.320,0:26:17.010 +>> AUDIENCE MEMBER: because it's all just FreeBSD + +0:26:17.10,0:27:22.290 +under the hood so you could +>> MATT: yeah, we don't have any +graphical tools right now for that + +0:27:22.290,0:27:23.860 +but any FreeBSD sys admin + +0:27:23.860,0:27:25.770 +could make it sit up and beg + +0:27:25.770,0:27:27.570 +what we are working on + +0:27:27.570,0:27:30.320 +right now some cool PXE stuff + +0:27:30.320,0:27:32.500 +so you'll be able to + +0:27:32.500,0:27:34.360 +boot off of a live CD + +0:27:34.360,0:27:36.630 +and then install tons of machines + +0:27:36.630,0:27:41.549 +from that one installation +so that's pretty cool + +0:27:41.549,0:27:43.950 +any other questions + +0:27:43.950,0:27:46.780 +>> AUDIENCE MEMBER: when are you +planning the next PC-BSD release? + +0:27:46.780,0:27:57.070 +>> CHRIS: 1.6 we're aiming for the end +of the summer to base the first one +off FreeBSD 7 + +0:27:57.070,0:28:03.680 +ideally, we'd like to use ZFS as well + +0:28:03.680,0:28:06.460 +>> MATT: KDE4 has a few bucks + +0:28:06.460,0:28:10.780 +so I think we should probably +wait for 4.1 + +0:28:10.780,0:28:17.780 +I've been reporting tons of them + +0:28:22.010,0:28:24.510 +[unclear question] + +0:28:24.510,0:28:27.950 +yes and PC-BSD 1.6 will %uh + +0:28:27.950,0:28:29.450 +include support for ZFS + +0:28:29.450,0:28:32.020 +with the front-end to it + +0:28:32.020,0:28:33.130 +and I don't want to put you on the spot + +0:28:33.130,0:28:35.530 +and %uh jails + +0:28:35.530,0:28:39.700 +I don't want to put you on the spot +that'd be cruel + +0:28:39.700,0:28:42.300 +okay well, come see us at the +table thank you very much" diff --git a/en_US.ISO8859-1/captions/2008/nycbsdcon/dixon-bsdvsgpl.sbv b/en_US.ISO8859-1/captions/2008/nycbsdcon/dixon-bsdvsgpl.sbv new file mode 100644 index 0000000000..4b35c0ab2e --- /dev/null +++ b/en_US.ISO8859-1/captions/2008/nycbsdcon/dixon-bsdvsgpl.sbv @@ -0,0 +1,932 @@ +0:00:02.199,0:00:08.939 +I + +0:00:08.939,0:00:11.849 +they said + +0:00:11.849,0:00:14.669 +welcome everybody welcome to BSD v. GPL + +0:00:14.669,0:00:19.259 +personaly i'd like to thank the organizers +and the volunteers for making the New York City BSDCon + +0:00:19.259,0:00:20.609 +possible + +0:00:20.609,0:00:23.579 +and my employer OmniTI + +0:00:23.579,0:00:24.830 +for sponsoring my talk + +0:00:24.830,0:00:26.789 +I had to get that in + +0:00:26.789,0:00:29.169 +a couple years ago I was invited to give a talk here + +0:00:29.169,0:00:33.990 +that was quite quite a lot of fun to give and seemed +be well received + +0:00:33.990,0:00:37.110 +fortunately they were fool hardy enough to invite me +back again + +0:00:37.110,0:00:42.460 +but as we know sequels really measure up so +the official title of this is + +0:00:42.460,0:00:44.890 +not the sequel to "BSD is Dying" + +0:00:44.890,0:00:51.790 +but don't worry it's just as short as the first one + +0:00:51.790,0:00:54.530 +BSD v. GSL the great debate + +0:00:54.530,0:00:57.990 +the ongoing conflict between users of the BSD + +0:00:57.990,0:01:00.880 +and GPL licenses + +0:01:00.880,0:01:02.290 +well what is the debate + +0:01:02.290,0:01:03.430 +do you mean like + +0:01:03.430,0:01:04.799 +gosh Jason + +0:01:04.799,0:01:07.000 +I didn't know there was a debate between + +0:01:07.000,0:01:10.990 +the BSD and GPL licenses + +0:01:10.990,0:01:13.130 +or like duh Jason + +0:01:13.130,0:01:14.220 +everyone knows that + +0:01:14.220,0:01:15.869 +BSD has already won + +0:01:20.220,0:01:22.360 +well for the sake of argument + +0:01:22.360,0:01:25.250 +let's assume that we're talking about the former + +0:01:25.250,0:01:27.480 +so back to the debate + +0:01:27.480,0:01:30.680 +which license is more free + +0:01:30.680,0:01:33.430 +it depends a lot on how you define freedom + +0:01:33.430,0:01:37.000 +is it a tool used to guarantee the freedom of the code + +0:01:37.000,0:01:38.560 +for everyone + +0:01:38.560,0:01:39.900 +or is it a mechanism + +0:01:39.900,0:01:42.620 +providing select freedoms + +0:01:42.620,0:01:49.620 +to a select group of users + +0:01:52.420,0:01:56.980 +I'll let you make that judgment for yourselfs + +0:01:56.980,0:01:59.540 +but first let's step back in time a moment + +0:01:59.540,0:02:01.570 +look at the origins of the BSD + +0:02:01.570,0:02:03.440 +and GPL licenses + +0:02:03.440,0:02:10.440 +which should help us better understand their +goals and philosophies + +0:02:12.420,0:02:15.079 +I know boo + +0:02:15.079,0:02:20.049 +even though work on the ancestors of UNIX +dates back to nineteen seventy four + +0:02:20.049,0:02:25.069 +it took roughly fifteen years to get to +a point where these licenses became a necessity + +0:02:25.069,0:02:26.959 +back in nineteen eighty five + +0:02:26.959,0:02:28.299 +Richard Stallman + +0:02:28.299,0:02:30.909 +a member of the staff at MIT + +0:02:30.909,0:02:31.840 +had a dispute + +0:02:31.840,0:02:34.769 +with the company named UniPress + +0:02:34.769,0:02:37.419 +another developer named James Gosling + +0:02:37.419,0:02:42.699 +developed the first version of Emacs written in C + +0:02:43.759,0:02:48.809 +after Stallman had incorporated parts Goslings Emacs +into Gnu Emacs + +0:02:48.809,0:02:53.079 +Gosling sold the rights to UniPress + +0:02:53.079,0:02:55.719 +UniPress theatened Stallman + +0:02:55.719,0:02:58.569 +against further redistribution of the Gosling bits + +0:02:58.569,0:03:01.169 +so he rewote the effected portions + +0:03:01.169,0:03:04.819 +and released a new version under a copyleft license + +0:03:04.819,0:03:08.449 +call Emacs General Public License + +0:03:08.449,0:03:10.220 +the June nineteen eighty eight issue + +0:03:10.220,0:03:12.159 +of the GNU bulletion + +0:03:12.159,0:03:15.669 +first mentioned that GNU general public +license + +0:03:15.669,0:03:18.449 +but even his earlest announcement foreshadowed + +0:03:18.449,0:03:25.449 +an underlying theme that forms the great debate + +0:03:27.009,0:03:29.599 +switching gears + +0:03:29.599,0:03:31.449 +switching gears we find ourselves + +0:03:31.449,0:03:33.609 +in the summer of 1989 + +0:03:33.609,0:03:36.010 +soaking up the summer rays in the computer +lab + +0:03:36.010,0:03:38.889 +of CSRG of Cal Berkeley + +0:03:38.889,0:03:41.019 +the Berkeley Software Distribution + +0:03:41.019,0:03:44.859 +already had thousands of copies distributed to +users + +0:03:44.859,0:03:46.769 +because the releases included the source code + +0:03:46.769,0:03:48.519 +AT&T Unix + +0:03:48.519,0:03:50.799 +as well as Berkeley's improvements + +0:03:50.799,0:03:52.480 +users were required to purchase + +0:03:52.480,0:03:55.619 +an AT&T source license + +0:03:55.619,0:03:57.079 +demand had built up + +0:03:57.079,0:03:59.139 +for a version of BSD Unix + +0:03:59.139,0:04:02.650 +that didn't require an AT&T source license + +0:04:02.650,0:04:05.269 +so BSD developers were able to put together a +release + +0:04:05.269,0:04:07.039 +free of AT&T code + +0:04:07.039,0:04:10.609 +that included the BSD originated TCP/IP networking + +0:04:10.609,0:04:13.979 +stack and it's supported utilities + +0:04:13.979,0:04:18.339 +this version was officially known as Networking Release 1 +or NET/1 + +0:04:18.339,0:04:20.359 +and was released under a liberal license + +0:04:20.359,0:04:22.839 +were the user could redistribute the code + +0:04:22.839,0:04:24.189 +modified or unmodified + +0:04:24.189,0:04:26.449 +source or binary form + +0:04:26.449,0:04:30.169 +with no accounting of roylaties to Berkeley + +0:04:30.169,0:04:32.069 +even with these liberal terms + +0:04:32.069,0:04:35.470 +hundreds of organizations step forward to paid +the one thousand dollar fee + +0:04:35.470,0:04:41.590 +for copies of the official NET/1 media + +0:04:41.590,0:04:45.989 +by 1991 the free software community had +considerable momentum + +0:04:45.989,0:04:48.239 +with the success of the GNU + +0:04:48.239,0:04:50.709 +compiler collection of userland tools + +0:04:50.709,0:04:53.059 +berkeley's own networking release 2 + +0:04:53.059,0:04:56.180 +GPLv2 and the LGPL + +0:04:56.180,0:04:58.419 +and the linux kernel + +0:04:59.639,0:05:02.130 +a fledgling knockoff of MINIX + +0:05:02.130,0:05:03.400 +created by a student + +0:05:03.400,0:05:08.219 +at the University of Helsinki + +0:05:08.219,0:05:12.879 +come 1992 the BSD community +was buckling up for a wild right + +0:05:12.879,0:05:15.080 +Unix Systems Laboratories + +0:05:15.080,0:05:17.709 +a subsidiary of AT&T + +0:05:17.709,0:05:20.050 +filed suit against BSDI + +0:05:20.050,0:05:22.840 +claiming that their derivative of Berkeley's NET/2 + +0:05:22.840,0:05:25.919 +infringed on the official the AT&T copyrights + +0:05:25.919,0:05:27.460 +by 1994 + +0:05:27.460,0:05:28.920 +the lawsuit was settled + +0:05:28.920,0:05:32.520 +largely in favor of BSDI and Cal Berkeley + +0:05:32.520,0:05:38.819 +but the impact of BSD adoption was +substantial + +0:05:38.819,0:05:41.149 +hello + +0:05:41.149,0:05:45.660 +new users gravitated to the Linux and +GNU alternative in droves + +0:05:45.660,0:05:48.469 +resulting in increased acceptance + +0:05:48.469,0:05:54.490 +of Stallman's GPL license and new manifesto + +0:05:54.490,0:05:57.910 +with a little bit of history under our belt +we can begin to take a deeper look at the + +0:05:57.910,0:05:59.180 +goals of each license + +0:05:59.180,0:06:02.479 +and the means it uses + +0:06:02.479,0:06:06.699 +and I means it uses to reach those ends + +0:06:06.699,0:06:09.909 +certainly these two licenses share +some common goals + +0:06:09.909,0:06:12.250 +they're both OSI approved licenses + +0:06:12.250,0:06:13.310 +suitable + +0:06:13.310,0:06:16.870 +for a variety of software projects + +0:06:16.870,0:06:20.459 +both licenses require attribution to remain intact + +0:06:20.459,0:06:23.770 +typically in the form of a copyright notice +within the license + +0:06:23.770,0:06:27.560 +every developer reserves the right to point +a completed works and take credit for their + +0:06:27.560,0:06:29.509 +efforts + +0:06:29.509,0:06:33.529 +admittedly some may choose not too + +0:06:33.529,0:06:37.439 +we don't have any MySQL developers in here do we + +0:06:37.439,0:06:41.729 +both licenses + +0:06:41.729,0:06:43.999 +but licenses convey freedoms to the user + +0:06:43.999,0:06:47.399 +to use the code as he or she sees fit + +0:06:47.399,0:06:50.059 +to read and learn from the code + +0:06:50.059,0:06:52.059 +use the resulting binaries + +0:06:52.059,0:06:54.470 +to develop your own applications + +0:06:54.470,0:06:59.960 +and stick it to your favorite three letter agencies + +0:06:59.960,0:07:02.619 +both licenses allow users + +0:07:02.619,0:07:04.500 +to redistribute + +0:07:04.500,0:07:06.979 +source code in binary forms + +0:07:06.979,0:07:08.930 +provided certain conditions are met + +0:07:08.930,0:07:11.949 +we'll come back to this in a few moments + +0:07:11.949,0:07:14.969 +both licenses provide some basic disclaimers + +0:07:14.969,0:07:17.699 +as a layer of protection for the developer + +0:07:17.699,0:07:21.159 +against claims of liability + +0:07:21.159,0:07:22.949 +for any direct + +0:07:22.949,0:07:24.539 +indirect + +0:07:24.539,0:07:26.550 +incidental + +0:07:26.550,0:07:27.999 +exemplary + +0:07:27.999,0:07:30.659 +thats a really hard one to get a picture of + +0:07:30.659,0:07:37.289 +or consequential damages + +0:07:37.289,0:07:39.070 +including but not limited to + +0:07:39.070,0:07:46.070 +procurement of substitute goods and services + +0:07:46.939,0:07:48.060 +loss of use, + +0:07:55.789,0:08:02.789 +loss of data, + +0:08:04.990,0:08:09.710 +lost profits, + +0:08:09.710,0:08:16.710 +or business interuption + +0:08:17.119,0:08:20.519 +I'd like to say that I did this slide before this +week + +0:08:20.519,0:08:24.449 +anyone who knows me knows that's not true + +0:08:24.449,0:08:28.599 +as we've seen both BSD and GPL licences share +some common freedoms + +0:08:28.599,0:08:31.839 +but as we know the devil's in the details + +0:08:31.839,0:08:35.600 +I'd actually like to ask for audience participation +in the following slides + +0:08:35.600,0:08:36.719 +this is my own experiment + +0:08:36.719,0:08:41.110 +to help gadge the communities familiarity with +the licenses + +0:08:41.110,0:08:42.870 +I'm going to read off a phrase + +0:08:42.870,0:08:45.990 +and I'd like you all to tell me which license +it is applicable too + +0:08:45.990,0:08:49.120 +k lets begin + +0:08:49.120,0:08:50.029 +freedom for everyone + +0:08:50.029,0:08:51.619 +even businesses + +0:08:51.619,0:08:54.810 +BSD + +0:08:54.810,0:08:57.060 +anti-capitalist socialism + +0:08:57.060,0:09:00.560 +GPL + +0:09:00.560,0:09:02.210 +thank you + +0:09:02.210,0:09:04.540 +that's correct + +0:09:04.540,0:09:08.190 +easy compliance + +0:09:08.190,0:09:09.529 +force code + +0:09:09.529,0:09:10.700 +to remain open + +0:09:10.700,0:09:13.190 +GDL + +0:09:13.190,0:09:15.230 +world domination + +0:09:15.230,0:09:18.180 +BSD + +0:09:18.180,0:09:19.690 +perhaps FreeBSD + +0:09:22.080,0:09:25.420 +I'll give you half credit + +0:09:25.420,0:09:29.960 +shut up and hack + +0:09:29.960,0:09:31.220 +brief and simple + +0:09:31.220,0:09:33.620 +BSD + +0:09:33.620,0:09:40.190 +complex and obtuse + +0:09:44.390,0:09:48.800 +this is not the ASB graph this is +something someone pointed to online + +0:09:48.800,0:09:55.800 +it represents the word count in each +version of the BSD and GPL licenses + +0:10:00.030,0:10:01.790 +yes and I think that that's true + +0:10:01.790,0:10:06.270 +the the for and third up losses + +0:10:06.270,0:10:11.340 +it was over the issue it before + +0:10:11.340,0:10:15.120 +take that as you will + +0:10:15.120,0:10:17.080 +and I mentioned previously that both include + +0:10:17.080,0:10:19.260 +some conditions regarding distribution + +0:10:19.260,0:10:23.370 +this in particular is where you see a real diversion +philosophies + +0:10:23.370,0:10:26.439 +the GPL license includes a number of +terms and conditions + +0:10:26.439,0:10:28.880 +intent on restricting the sale, use + +0:10:28.880,0:10:31.130 +and redistribution of the effect it works + +0:10:31.130,0:10:34.190 +let's look at the examples + +0:10:34.190,0:10:34.689 +first + +0:10:34.689,0:10:37.610 +your aloowed to copy and distribute the program source code + +0:10:37.610,0:10:39.090 +so long as you include + +0:10:39.090,0:10:40.920 +the copyright notice + +0:10:40.920,0:10:43.040 +the disclaimer of warrenty + +0:10:43.040,0:10:46.460 +and any notices that refer to the license + +0:10:46.460,0:10:52.230 +I wasn't sure that anybody would get this + +0:10:52.230,0:10:55.840 +you may also charge a fee for packaging or + +0:10:55.840,0:10:57.690 +transferal of the work + +0:10:57.690,0:11:02.750 +or offer warrenty protection for a fee + +0:11:02.750,0:11:07.330 +but you may not sell the software + +0:11:07.330,0:11:10.279 +you can copy or distribute any derivative works + +0:11:10.279,0:11:13.010 +providing you meet the following conditions + +0:11:13.010,0:11:17.860 +any modified files most prominently displayed +notice that the files were changed + +0:11:17.860,0:11:23.160 +it must be a license as a whole and no charge +to any thrid parties + +0:11:23.160,0:11:26.740 +and if the program reads commands interactivly when run + +0:11:26.740,0:11:28.800 +it must display an announcement + +0:11:28.800,0:11:35.800 +including a copyright notice and disclaimer +of warrenty + +0:11:37.570,0:11:40.860 +you may copy or distribute the code in binary form + +0:11:40.860,0:11:43.140 +provided you meet these conditions + +0:11:43.140,0:11:46.190 +you include the corresponding source code + +0:11:46.190,0:11:49.860 +a written offer + +0:11:49.860,0:11:52.380 +valid for at least three years + +0:11:52.380,0:11:57.720 +to provide any third party with a copy of the +source code + +0:11:57.720,0:12:04.720 +for a fee not to exceed the cost of distributing the source code + +0:12:06.120,0:12:09.690 +there are other restrictions as well we have covered +the most intrusive ones + +0:12:09.690,0:12:16.690 +the other is a mass of while the religious +right + +0:12:17.680,0:12:21.500 +to be fair the FCC licenses and without its +own restrictions + +0:12:21.500,0:12:26.620 +most of the issue mainstream the is the distributions +such as previously majesty + +0:12:26.620,0:12:31.950 +and ratified is the up for further to cosby +is the lessons for nafta + +0:12:31.950,0:12:34.420 +the only issue brought it goes one step further + +0:12:34.420,0:12:38.600 +using the issue like a modified highest the +license with no extra cost + +0:12:38.600,0:12:42.980 +but again even these have a number of his +first reactions use our freedoms + +0:12:42.980,0:12:44.500 +here's an exhaustive lives + +0:12:44.500,0:12:51.500 +of these restrictions + +0:12:52.270,0:12:58.040 +activists loyal there + +0:12:58.040,0:13:02.640 +no more free license is the easier it is to +comply + +0:13:02.640,0:13:05.190 +conversely the more complex a license becomes + +0:13:05.190,0:13:10.240 +the more likely it is that violations her + +0:13:10.240,0:13:13.250 +the saw for freedom law center + +0:13:13.250,0:13:20.150 +offers legal representation + +0:13:20.150,0:13:23.220 +the free and open source offer projects + +0:13:23.220,0:13:28.590 +the web site lead you to believe that they +defend the rights of all streets are process + +0:13:28.590,0:13:30.830 +but would review the press releases + +0:13:30.830,0:13:35.560 +shows that every single case it's a has been +released + +0:13:35.560,0:13:38.040 +in fact many of you may remember last year + +0:13:38.040,0:13:41.570 +when will it makes their story developers + +0:13:41.570,0:13:43.519 +realizing straight floaters + +0:13:43.519,0:13:46.840 +obviously if they're struggling the jeep you +know + +0:13:46.840,0:13:48.250 +what you saw + +0:13:48.250,0:13:52.860 +what they also removed rates copyright points +that the case + +0:13:52.860,0:13:55.020 +the software freedom also stepped in + +0:13:55.020,0:13:58.570 +in convince limits developers to correct the +rich states + +0:13:58.570,0:14:02.570 +but not until a storm of controversy was unleashed + +0:14:02.570,0:14:06.700 +causing hardship among the lyrics and the +it's the communities + +0:14:06.700,0:14:10.170 +the press releases carefully worded + +0:14:10.170,0:14:14.980 +to make it safe + +0:14:14.980,0:14:16.749 +to make it sound like they were betting + +0:14:16.749,0:14:20.360 +you can be as the license for compatibility + +0:14:20.360,0:14:25.420 +rather than explaining how the limits developers +infringed on race right + +0:14:25.420,0:14:31.830 +does anyone else in also + +0:14:31.830,0:14:35.030 +because copyright a softer also complex + +0:14:35.030,0:14:38.830 +the number of missus claus up we're going +to be as easy going as this is + +0:14:38.830,0:14:40.640 +i'd like to conclude my top + +0:14:40.640,0:14:43.740 +but they're not few days + +0:14:43.740,0:14:45.010 +this is number one + +0:14:45.010,0:14:47.390 +did you get a license is more free + +0:14:47.390,0:14:49.610 +although places restrictions on users + +0:14:49.610,0:14:51.540 +the integrity of freedom + +0:14:51.540,0:14:54.420 +possibly forcing uses to publish their changes + +0:14:54.420,0:14:59.390 +results in a greater level of freedom + +0:14:59.390,0:15:01.880 +the restrictions always means less free + +0:15:01.880,0:15:03.579 +what did the original developer + +0:15:03.579,0:15:04.620 +born in danger + +0:15:04.620,0:15:08.710 +did you kill always places additional restrictions +on the economy + +0:15:08.710,0:15:09.910 +modification + +0:15:09.910,0:15:12.950 +redistribution effect it works + +0:15:12.950,0:15:14.520 +that number two + +0:15:14.520,0:15:16.830 +the issue through to be still + +0:15:16.830,0:15:20.350 +he also claim that because the company can +take care + +0:15:20.350,0:15:22.110 +make the changes proprietary + +0:15:22.110,0:15:23.439 +that somehow lessons + +0:15:23.439,0:15:29.000 +the freemen's of the original up + +0:15:29.000,0:15:31.450 +this is completed on a ship + +0:15:31.450,0:15:35.060 +the result was always this tree is a ritual +developers she said + +0:15:35.060,0:15:41.130 +no evil corporation to subvert the rights +of other users to the the issue is + +0:15:41.130,0:15:42.600 +number three + +0:15:42.600,0:15:45.060 +so Richard Stallman is a socialist + +0:15:45.060,0:15:48.670 +intent on destroying commercial software companies + +0:15:48.670,0:15:49.830 +yes + +0:15:49.830,0:15:53.780 +I + +0:15:53.780,0:15:56.310 +Richard Stallman actually congress + +0:15:56.310,0:16:00.380 +intent on destroying commercial software companies + +0:16:00.380,0:16:03.920 +socialism actually from a state ownership +it means of production + +0:16:03.920,0:16:05.090 +rather + +0:16:05.090,0:16:07.870 +Stallman believes in the class of state fourteen +days of all the rules diff --git a/en_US.ISO8859-1/captions/2009/asiabsdcon/rao-kernellocking-1.sbv b/en_US.ISO8859-1/captions/2009/asiabsdcon/rao-kernellocking-1.sbv new file mode 100644 index 0000000000..3bc98ee9aa --- /dev/null +++ b/en_US.ISO8859-1/captions/2009/asiabsdcon/rao-kernellocking-1.sbv @@ -0,0 +1,2457 @@ +0:00:00.000,0:00:02.740 +My name is Attilio Rao and + +0:00:02.740,0:00:05.960 +I think that we are in time for the presentation + +0:00:05.960,0:00:10.870 +I want to ask sorry for my English because it's not really British English but I will + +0:00:10.870,0:00:12.480 +try to make this + + +0:00:12.480,0:00:16.359 +a little bit uncomfortable + +0:00:16.359,0:00:21.300 +Better? + +0:00:21.300,0:00:24.609 +Ok.Thank you.So we are going to speak about the + + +0:00:24.609,0:00:28.639 +the locking infrastructure in the FreeBSD kernel +which + +0:00:28.639,0:00:33.440 +is a bit interesting topic because + +0:00:33.440,0:00:38.890 +Its going to be with time very widely discussed on our mailing list not only + +0:00:38.890,0:00:43.100 +from developer's perspective but even from user's perspective. + +0:00:43.100,0:00:49.470 +and we will see why later + +0:00:49.470,0:00:52.990 +In this presentation we will specifically see what + +0:00:52.990,0:00:55.100 +was the situation + +0:00:55.100,0:00:57.010 +of the first + +0:00:57.010,0:00:59.150 +FreeBSD implementations + +0:00:59.150,0:01:01.120 +and what changed from that + +0:01:01.120,0:01:06.690 +what specifically what's called the SMPng era + +0:01:06.690,0:01:07.639 +and what + +0:01:07.639,0:01:10.500 +we had prior that + +0:01:10.500,0:01:12.780 + we are going to discuss + +0:01:12.780,0:01:13.579 +specifically + +0:01:13.579,0:01:19.160 +locking primitives that has been introduced with time until now + +0:01:19.160,0:01:20.910 + and + +0:01:20.910,0:01:24.730 +problems linked to + +0:01:24.730,0:01:27.620 +parellelism in general and how we solve that in + +0:01:27.620,0:01:30.950 +the FreeBSD kernel + +0:01:30.950,0:01:36.200 +You can see a table of content a little bit more detailed as + +0:01:36.200,0:01:39.850 +listing precisely what we + +0:01:39.850,0:01:43.210 +some problems like + +0:01:43.210,0:01:46.159 +Priority Inheritance + +0:01:46.159,0:01:53.159 +and Adaptive Spinning that we are going to discuss fruitfullly. + +0:01:53.370,0:01:58.890 +Mostly until FreeBSD 4.x + +0:01:58.890,0:02:00.830 +We had already moved to multitasking. + +0:02:00.830,0:02:05.210 +so the slide is a little bit confusing but + multitasking and preemptive system + +0:02:05.210,0:02:06.360 +since + +0:02:06.360,0:02:10.379 +that transition was not very + +0:02:10.379,0:02:14.180 +was not very difficult to implement in such systems + because + +0:02:14.180,0:02:17.479 +if you can see then our uniprocessor machine + +0:02:17.479,0:02:18.929 +you can get that + +0:02:18.929,0:02:20.019 +well + +0:02:20.019,0:02:24.029 +the sequential execution was +just + +0:02:24.029,0:02:25.699 +stopped by + +0:02:25.699,0:02:26.309 +preemption + +0:02:26.309,0:02:29.400 +and by arrival of interrupts + +0:02:29.400,0:02:33.969 +so you should adjustment in consistency of data structures + +0:02:33.969,0:02:36.289 +about these two issues + +0:02:36.289,0:02:37.079 +more precisely + +0:02:37.079,0:02:39.079 +we were handling + +0:02:39.079,0:02:41.779 +the interrupts and transitions through + +0:02:41.779,0:02:43.370 +a mechanism + +0:02:43.370,0:02:45.779 +called SPL + +0:02:45.779,0:02:50.769 +and for kernel threads, threads running in the kernel we were disabling + +0:02:50.769,0:02:51.379 +preemption + +0:02:51.379,0:02:53.019 +in order to avoid + +0:02:53.019,0:02:55.809 +the corruption of the data structure + +0:02:55.809,0:02:57.519 +This approach while was + +0:02:57.519,0:03:00.629 +pretty good on uniprocessor machines + +0:03:00.629,0:03:02.269 +was actually + +0:03:02.269,0:03:04.270 +impredictable for + +0:03:04.270,0:03:06.219 +the SMP environments + +0:03:06.219,0:03:10.199 +more precisely because we had more coures that + +0:03:10.199,0:03:12.959 +was running thread per time + +0:03:12.959,0:03:13.909 +and so + +0:03:13.909,0:03:14.980 +parallel + +0:03:14.980,0:03:19.309 +accesses to the data structures were possible + +0:03:19.309,0:03:21.290 +in order to + +0:03:21.290,0:03:22.469 +to avoid + +0:03:22.469,0:03:24.149 +big problems in the kernel + +0:03:24.149,0:03:25.799 +we have to just + +0:03:25.799,0:03:26.739 +allow + +0:03:26.739,0:03:28.989 +the entering of + +0:03:28.989,0:03:32.309 +one thread per time into kernel + +0:03:32.309,0:03:35.379 +while that was a pretty good approach + +0:03:35.379,0:03:39.049 +for workloads that were nearly user space + +0:03:39.049,0:03:40.969 +for work loads + +0:03:40.969,0:03:45.619 + requiring a lot of IO for example they were wateful because they wasn't + +0:03:45.619,0:03:47.839 +getting any advantage from the new + +0:03:47.839,0:03:49.819 +SMP architecture + +0:03:49.819,0:03:52.749 +like the parallelism was basically zero + +0:03:52.749,0:03:55.189 +at least in the kernel + +0:03:55.189,0:03:55.949 +in order + +0:03:55.949,0:04:00.650 +to fix that a new project was created +called SMP + +0:04:00.650,0:04:01.470 +New generation + + +0:04:01.470,0:04:05.169 +or NG + +0:04:05.169,0:04:07.309 +as you can see it from the slide + +0:04:07.309,0:04:10.329 +the entering in the kernel was preempted + +0:04:10.329,0:04:12.569 +by using Big Lock + +0:04:12.569,0:04:19.569 +called BKL basically + +0:04:23.199,0:04:28.109 +With FreeBSD 5.x we had the SMP new generation project + +0:04:28.109,0:04:30.110 +basically it was + +0:04:30.110,0:04:31.509 +a sanitization of the + +0:04:31.509,0:04:34.539 +of all + +0:04:34.539,0:04:40.039 +our kernel and the engineering over lot of mechanism inside our kernel. We could see it + +0:04:40.039,0:04:44.709 +FreeBSD 4.x and FreeBSD 5.x as mainly two different kernels + +0:04:44.709,0:04:45.550 +because of + +0:04:45.550,0:04:50.150 +substantial subsystem were rewritten and + +0:04:50.150,0:04:51.830 +were written with the + +0:04:51.830,0:04:56.949 +idea to use and implement a real parallelism in mind. + +0:04:56.949,0:05:02.610 + we can say that basically it was a major task a very big task + +0:05:02.610,0:05:04.029 +and that it required + +0:05:04.029,0:05:06.669 +a lot of years to be brought + +0:05:06.669,0:05:08.900 +in a good shape at least + +0:05:08.900,0:05:11.379 +In Italy, the people gave + +0:05:11.379,0:05:13.069 +a lot of + +0:05:13.069,0:05:16.350 +complaining about the + +0:05:16.350,0:05:20.430 +un-robustness of FreeBSD 5.x but + +0:05:20.430,0:05:22.249 +probably that's because they couldn't even + +0:05:22.249,0:05:28.929 +see that the changes were really really important and really huge + +0:05:28.929,0:05:34.490 +however for FreeBSD 5.x based this initial SMP system + +0:05:34.490,0:05:37.070 +inheriting from BSD/OS + +0:05:37.070,0:05:39.309 +that kindly + +0:05:39.309,0:05:42.699 +released this code above that + +0:05:42.699,0:05:44.009 +and the + + +0:05:44.009,0:05:46.579 +the process was break up in + +0:05:46.579,0:05:51.069 +some precise tasks at least in Italy + +0:05:51.069,0:05:55.429 +Mainly the first things was introducing in the kernel + +0:05:55.429,0:06:00.180 +new set of atomic instruction and locking primitives + +0:06:00.180,0:06:01.520 +Then introducing + +0:06:01.520,0:06:05.380 +an abstraction called interrupt threads that we are going to discuss + +0:06:05.380,0:06:06.929 +rather later but + +0:06:06.929,0:06:12.319 +it was basically restored completely the interrupt mechanism that was in the FreeBSD 4.x + +0:06:14.439,0:06:16.490 +the the BKL + +0:06:16.490,0:06:19.210 +lock was moved to a real + +0:06:19.210,0:06:20.679 +mutex called Giant + +0:06:20.679,0:06:23.180 +that still exists in our kernel + +0:06:23.180,0:06:26.660 +and they were introduced some threading primitives + +0:06:26.660,0:06:28.019 +the + +0:06:28.019,0:06:30.499 +like and and on and + +0:06:30.499,0:06:32.280 + threading primitives + +0:06:32.280,0:06:34.319 +called also KSE + +0:06:34.319,0:06:37.009 +which are actually never used in our kernel + +0:06:37.009,0:06:41.620 +and that being their being exit out in the past year + +0:06:41.620,0:06:43.409 +and the + +0:06:43.409,0:06:45.259 +slowly of the porting of + +0:06:45.259,0:06:50.459 +all the older subsystems to a finer locking was started + +0:06:50.459,0:06:55.919 +I have to say this task is not still completed, its still going on but + +0:06:55.919,0:06:58.889 +we are really good shape about that + +0:06:58.889,0:07:02.429 +just few subsystems remain which are still Giant protected + +0:07:02.429,0:07:05.939 +and with new release that we're going to ship this year, I think that we made + +0:07:05.939,0:07:10.220 +a very huge step forward in this direction + +0:07:12.319,0:07:18.599 +really the the SMPng has been considered closed around the end of + +0:07:18.599,0:07:20.600 +2007 + +0:07:20.600,0:07:22.579 +but the + +0:07:22.579,0:07:23.819 +the + +0:07:23.819,0:07:27.539 +the important parts where this initial moving + +0:07:27.539,0:07:32.669 +I rather thing that's not listed here but I can tell you is that + +0:07:32.669,0:07:38.279 +even that if Giant was preventing any parallelism initial parallelism + +0:07:38.279,0:07:43.219 +that were imported new kernel memory allocator that was + +0:07:43.219,0:07:45.009 +that I discovered + +0:07:45.009,0:07:48.439 +and the scheduler was move with a seperate lock + +0:07:48.439,0:07:50.449 +in order to + +0:07:50.449,0:07:52.080 +start getting some + +0:07:52.080,0:07:54.699 +a little bit of concurrency + +0:07:54.699,0:07:59.099 +a real concurrency + +0:07:59.099,0:08:01.520 +the + +0:08:01.520,0:08:06.280 +before to speak about FreeBSD specifics we can start digging in about + +0:08:06.280,0:08:08.219 +what kind of + +0:08:08.219,0:08:12.729 +of locking primitives you can find in our kernel. + +0:08:12.729,0:08:15.780 +from a more historical point of view + +0:08:15.780,0:08:19.710 +we have some versions of mutex which + +0:08:19.710,0:08:20.919 +I assume + +0:08:20.919,0:08:24.809 +people here knows about that but I'm going to give a little explanation + +0:08:24.809,0:08:26.449 +for people that doesn't know + +0:08:26.449,0:08:28.939 +a mutex is basically + +0:08:28.939,0:08:30.739 +a lock allowing to access + +0:08:30.739,0:08:36.700 +to some protected data's thread to just one thread per time + +0:08:36.700,0:08:38.150 +so if a thread + +0:08:38.150,0:08:39.690 +owns the lock, + +0:08:39.690,0:08:40.760 +owns the mutex + +0:08:40.760,0:08:42.539 +other threads + +0:08:42.539,0:08:44.039 +won't be able to + +0:08:44.039,0:08:46.090 +to access to this until + +0:08:46.090,0:08:48.730 +this lock is released + +0:08:48.730,0:08:50.430 +we offer even + +0:08:50.430,0:08:54.890 +some kind of locks called R/W lock Read/Write lock + +0:08:54.890,0:08:57.920 +which are basically a + +0:08:57.920,0:09:03.050 +locks that can be acquired in two different versions + +0:09:03.050,0:09:04.060 +one version + +0:09:04.060,0:09:07.980 +is the write lock which is the same as the mutex just one + +0:09:07.980,0:09:10.010 +in the protected part per time + +0:09:10.010,0:09:13.860 +and other one is the read mode which basically + +0:09:13.860,0:09:15.100 +allows + +0:09:15.100,0:09:18.410 +all the thread willing to acquire to read mode to + +0:09:18.410,0:09:23.699 +concurrently adjust to the structure but prevents the threads from + +0:09:23.699,0:09:25.390 +writing nto the protected path. + +0:09:25.390,0:09:28.890 +while the reader..while they are readers + +0:09:28.890,0:09:30.280 +then we have even + +0:09:30.280,0:09:33.030 +the locks called the Read Mostly + +0:09:33.030,0:09:37.570 +which are basically the same of Read/Write Locks but are + +0:09:37.570,0:09:42.500 +they have some optimization in order to make the Read + +0:09:42.500,0:09:44.180 +part be really fast + +0:09:44.180,0:09:46.930 +and to have like + +0:09:46.930,0:09:48.180 +zero overhead + +0:09:48.180,0:09:51.410 +zero overhead kind of lock + +0:09:51.410,0:09:53.350 +from the read path while + +0:09:53.350,0:09:55.590 +probably the write path is even + +0:09:55.590,0:09:59.210 +heavier than the other one but if you think about cases that + +0:09:59.210,0:10:01.710 +just + + +0:10:01.710,0:10:02.750 +where + +0:10:02.750,0:10:06.980 +there are a lot of reader chases and very few writer chases you can find that a + +0:10:06.980,0:10:08.220 +very useful + +0:10:08.220,0:10:11.070 +very useful primitive + +0:10:11.070,0:10:11.850 +then we have + +0:10:11.850,0:10:14.360 +some form of Wait channels + +0:10:14.360,0:10:16.030 +Wait channels + +0:10:16.030,0:10:17.140 +basically are what + +0:10:17.140,0:10:21.700 +generalizations of what other people con call like + +0:10:22.470,0:10:24.240 +condition variable and + +0:10:24.240,0:10:28.240 +they basically let that thread sleep + +0:10:28.240,0:10:30.870 +under some conditions that are + +0:10:30.870,0:10:35.200 +that are previously started with some + +0:10:35.200,0:10:36.610 +some variables + +0:10:36.610,0:10:37.150 +usually + +0:10:37.150,0:10:39.500 +having a Wait channel means that its + +0:10:39.500,0:10:45.080 +chases are controlled through another locking primitive like a mutex + +0:10:45.080,0:10:46.640 +or R/Wlock + +0:10:46.640,0:10:52.010 +and so often the Wait channel is associated to its + +0:10:52.010,0:10:53.620 +to its locking primitive + +0:10:53.620,0:11:00.140 +usually if you have no necessity to use a Wait channel without a primitive + +0:11:00.140,0:11:04.150 +a locking primitive you probably have bad code + +0:11:04.150,0:11:06.830 +but there are some edge cases + +0:11:06.830,0:11:09.660 +with that seem possible + +0:11:09.660,0:11:13.550 +As last thing FreeBSD sub primitive counting semaphore + +0:11:13.550,0:11:15.290 +even if thats considered not featured + +0:11:15.290,0:11:17.710 +as we are going to see I think they're going to see it and + +0:11:17.710,0:11:23.570 +its usage is pretty much discouraged + +0:11:23.570,0:11:28.320 +basically FreeBSD you can consider locking primative divided into three classes + +0:11:28.320,0:11:31.250 +three classes of + +0:11:31.250,0:11:32.450 +of locking + +0:11:32.450,0:11:34.090 +based mainly in + +0:11:34.090,0:11:35.600 +particular + +0:11:35.600,0:11:37.340 +from an outside perspective + +0:11:37.340,0:11:38.690 +based on the behavior + +0:11:38.690,0:11:42.680 +the contending threads as you regard of the lock + +0:11:42.680,0:11:48.100 +for example in case of a mutex you can can get that + +0:11:48.100,0:11:53.360 +spinning and blocking mutex do very different things about the contenders + +0:11:53.360,0:11:59.680 +as we are going to see more of this later + +0:11:59.680,0:12:03.410 +usually in the traditional literature, + +0:12:03.410,0:12:05.430 +there are just two + +0:12:05.430,0:12:07.280 +cases of the lock classes mainly + +0:12:07.280,0:12:08.620 +you will find the + +0:12:08.620,0:12:11.200 +spinning lock and the blocking lock + +0:12:11.200,0:12:14.370 +or what they called the sleeping lock + +0:12:14.370,0:12:16.670 +the I think that + +0:12:16.670,0:12:21.020 +as we're going to see why we have three types I think that things will be clear but + +0:12:21.020,0:12:27.100 +if you have any questions please ask us. Thats not a problem + +0:12:27.100,0:12:29.930 +Spinning primitives as I told you + +0:12:29.930,0:12:32.810 +allows the contesting thread to + +0:12:32.810,0:12:36.120 +to check the status of the lock periodically + +0:12:36.120,0:12:37.590 +and the + +0:12:37.590,0:12:40.420 +and they just do busy waiting around + +0:12:40.420,0:12:41.890 + the locking variable + +0:12:41.890,0:12:46.400 +as the spinning primitive FreeBSD just offers mutex + +0:12:46.400,0:12:50.689 +What are the problems linked with this kind of, with this class + +0:12:50.689,0:12:53.869 +of locks? Mainly its that CPU + +0:12:53.869,0:12:58.130 +remains busy without doing really nothing useful + +0:12:58.130,0:12:59.740 +it happens + +0:12:59.740,0:13:03.620 +that if several threads contest on the + +0:13:03.620,0:13:04.870 +on the locks + +0:13:04.870,0:13:08.210 +basically they share the same cache line where the lock is + +0:13:08.210,0:13:10.220 +where the lock is + +0:13:10.220,0:13:12.400 +that means that + +0:13:12.400,0:13:17.470 +contesting or sharing a cache line is a lot underlying activity + +0:13:17.470,0:13:20.150 +on a lot of architectures like for example + +0:13:20.150,0:13:23.660 +having a lot of snoop messages between CPUs + +0:13:23.660,0:13:26.450 +and some buses + +0:13:26.450,0:13:28.120 +some buses traffic + +0:13:28.120,0:13:31.980 +which means in a variety operations + +0:13:31.980,0:13:35.740 +and the last things even the most important you can note is that interrupts + +0:13:35.740,0:13:37.120 +are disabled + +0:13:37.120,0:13:39.330 +while spin locks are held + +0:13:39.330,0:13:40.810 +that was + +0:13:40.810,0:13:42.979 +that happens mainly because there are + +0:13:42.979,0:13:45.140 +there were identified in the past by some + +0:13:45.140,0:13:47.970 +kind of deadlocks possible + +0:13:47.970,0:13:50.180 +if you were going to lead + +0:13:50.180,0:13:51.710 +the spin locks + +0:13:51.710,0:13:55.900 +the interrupts enabled while holding a spin lock. In particular + +0:13:55.900,0:13:58.180 +you could find that there are + +0:13:58.180,0:14:02.530 +some problems with the interrupts angling good in the botom half that was + +0:14:02.530,0:14:05.040 +going to deadlock + +0:14:05.040,0:14:10.250 +Its not very simple to understand the thing so I've left out + +0:14:10.250,0:14:12.360 +but if you want to know + +0:14:12.360,0:14:15.990 +we could speak later probably + +0:14:17.820,0:14:21.320 +with spinning primitives we are even blocking primitives + +0:14:21.320,0:14:22.890 +blocking primitives + +0:14:25.260,0:14:26.860 +allows the + +0:14:26.860,0:14:28.440 +basically the contenders to be + +0:14:28.440,0:14:30.980 +descheduled from the runqueue + +0:14:30.980,0:14:35.790 +to be put on another kind of container + +0:14:35.790,0:14:38.000 +put on another kind of container + +0:14:38.000,0:14:40.489 +and basically + +0:14:40.489,0:14:41.399 +context switch immediately + +0:14:41.399,0:14:44.360 +immediately. + +0:14:44.360,0:14:49.440 +then we put again on runqueue of the scheduler just once the just when the owner + +0:14:49.440,0:14:51.570 + is going to release the lock + +0:14:51.570,0:14:53.260 +and it will be the owner + +0:14:53.260,0:14:56.930 +the owner that was going to + +0:14:56.930,0:15:00.310 +do all the operations about that + +0:15:00.310,0:15:05.550 +we have several primitives implemented as blocking primitives like mutexes + +0:15:05.550,0:15:10.470 +R/W locks and R-M locks + +0:15:11.430,0:15:13.140 +with + +0:15:13.140,0:15:16.890 +basically with + +0:15:16.890,0:15:21.780 +blocking primitives we have a lot of advantages over the spinning mutex + +0:15:21.780,0:15:24.650 +like having the contenders + +0:15:24.650,0:15:26.560 +that + +0:15:26.560,0:15:27.590 +that sleeps + +0:15:27.590,0:15:31.840 +or that blocks avoids CPU busyness + +0:15:31.840,0:15:34.660 +and mainly we can leave the + +0:15:34.660,0:15:37.150 +we can leave the + +0:15:37.150,0:15:42.040 +we can leave that basically the interrupts out + +0:15:42.040,0:15:45.760 +that happens mainly because the interrupts code is just allowed + +0:15:45.760,0:15:50.710 + at least the bottom of one is just allowed + +0:15:50.710,0:15:52.070 +to use spin locks + +0:15:52.070,0:15:56.049 + probably if it was going to use blocking primitives + +0:15:56.049,0:16:01.060 +we wouldnt have been able to disable interrupts here + +0:16:01.060,0:16:02.239 +There are however some + +0:16:02.239,0:16:04.790 +big drawbacks that as you will see + +0:16:04.790,0:16:07.210 +we handle in FreeBSD + +0:16:07.210,0:16:11.280 +in order to make the blobking primitives our + +0:16:11.280,0:16:13.540 +how could I tell + +0:16:13.540,0:16:16.440 +the first choice in terms of blocking + +0:16:16.440,0:16:19.690 +where the problem called Priority Inversion + +0:16:19.690,0:16:21.899 + and we have + +0:16:21.899,0:16:27.589 +the problem that context switches are very heavy in particular + +0:16:27.589,0:16:30.209 +on machines that FreeBSD uses as referral + +0:16:30.209,0:16:33.500 +like E38 and the MD64 + +0:16:33.500,0:16:37.940 +but as you're going to see we've used two techniques in order to + +0:16:37.940,0:16:40.020 +to cope with that + +0:16:42.020,0:16:45.830 +another thing is that while you cant + +0:16:45.830,0:16:47.920 +allow + +0:16:47.920,0:16:50.089 +context switches while having + +0:16:50.089,0:16:52.570 +while holding spin lock + +0:16:52.570,0:16:55.249 +it's obvious you cant + +0:16:55.249,0:16:59.580 +acquire a locking primitive while holding a spin lock + +0:16:59.580,0:17:02.110 +that's an important rule in FreeBSD + +0:17:02.110,0:17:06.089 +that sometimes its confused and often its not + +0:17:06.089,0:17:07.470 +observed + +0:17:07.470,0:17:09.929 +that leads to block refusal + +0:17:12.170,0:17:16.610 +usually you will always prefer a blocking primitive for a spin lock + +0:17:16.610,0:17:22.159 +if not in some very particular condition like what + +0:17:22.159,0:17:25.010 +Alrick said about the interrupt and even + +0:17:25.010,0:17:26.090 +about the + +0:17:28.160,0:17:30.570 +some parts that are very very short + +0:17:30.570,0:17:33.629 +we should have some example in the kernel even if I can + +0:17:33.629,0:17:35.390 +I can tell you one right now + +0:17:35.390,0:17:38.770 +I have no idea actually + +0:17:38.770,0:17:39.500 +so that + +0:17:39.500,0:17:43.740 +we're going to see the problemslinked with the blocking primitives the first one is + +0:17:43.740,0:17:45.679 +called Priority Inversion + +0:17:45.679,0:17:46.389 +basically + +0:17:46.389,0:17:49.130 +it could happen that like a thread A + +0:17:49.130,0:17:51.410 +which has a priority + +0:17:51.410,0:17:55.380 +owns a lock. call it L for example + +0:17:55.380,0:17:58.710 +then another thread with another priority than this one + +0:17:58.710,0:18:00.690 +locks on this lock + +0:18:00.690,0:18:03.299 +what happens is that the second thread + +0:18:03.299,0:18:04.120 +the thread B + +0:18:04.120,0:18:05.870 +for example + +0:18:05.870,0:18:08.920 +will need to wait for a lower priority thread + +0:18:08.920,0:18:13.070 +to finish its work load + +0:18:13.070,0:18:15.120 +we + +0:18:15.120,0:18:17.780 +solve this problem actually in the + +0:18:17.780,0:18:21.170 +kernel using a technique called priority propogation + +0:18:21.170,0:18:22.020 +basically + +0:18:22.020,0:18:24.620 +what happens is that priority of thread B + +0:18:25.760,0:18:27.880 +is lent to thread A + +0:18:27.880,0:18:31.460 +until it doesn't release the lock + +0:18:31.460,0:18:34.760 +of its directly implemented in the container + +0:18:34.760,0:18:36.180 +the turnstiles + +0:18:37.870,0:18:39.530 +while that could be done + +0:18:39.530,0:18:44.290 +even on the primitive it has been much convenient to use the container for + +0:18:44.290,0:18:45.190 +that + +0:18:45.190,0:18:45.990 +because + +0:18:45.990,0:18:52.990 +it was going to offer some advantage we are going to see right now + +0:18:53.030,0:18:54.240 +just note that + +0:18:54.240,0:18:56.090 +Read locks + +0:18:56.090,0:18:57.310 +cannot support + +0:18:57.310,0:19:03.430 +priority propogation fixes for read lock that happens because you'd like to + +0:19:03.430,0:19:07.290 +the turnstile should keep track of all the readers + +0:19:07.290,0:19:11.100 +and these would be very very expensive from + +0:19:11.100,0:19:12.880 +from a + +0:19:12.880,0:19:15.540 +from a point of view of the overhead + +0:19:15.540,0:19:19.800 +and even I think I've tried to do something in this regard and I + +0:19:19.800,0:19:24.050 +saw that there was some races that were trying to + +0:19:24.050,0:19:29.390 +acquire a spin lock as base even in fast path so it was a + +0:19:29.390,0:19:31.320 +an impredicable way + +0:19:31.320,0:19:32.380 +I will tell + +0:19:32.380,0:19:37.200 +at least for what we found so far + +0:19:37.200,0:19:37.630 +basically + +0:19:37.630,0:19:39.070 + what happens + +0:19:39.070,0:19:42.150 +about the priority propogation is that the + +0:19:42.150,0:19:44.830 +the threads and the turnstiles + +0:19:44.830,0:19:47.000 +are chained together + +0:19:47.000,0:19:48.350 +the thread + +0:19:48.350,0:19:50.970 +owns the a pointer + +0:19:50.970,0:19:53.710 +to wrench the turnstile is sleeping on + +0:19:53.710,0:19:58.540 +and the turnstile owns a pointer above + +0:19:58.540,0:20:00.549 +the owner of the lock + +0:20:00.549,0:20:04.620 +what happens is that for example in this case we have + +0:20:05.080,0:20:08.070 +a sleeper which is going to sleep on a turnstile + +0:20:08.070,0:20:08.990 +the first lock + +0:20:08.990,0:20:13.470 +which has a priority of one hundred and twenty eight + +0:20:14.120,0:20:15.520 +the turnstile + +0:20:15.520,0:20:18.370 +to the pointer + +0:20:18.370,0:20:20.570 +ts_owner knows which is its owner + +0:20:20.570,0:20:26.150 +and this owner has a priority of two hundred and fifty six + +0:20:26.150,0:20:31.120 +well as you know higher level, higher value means lower priority. so if this is +0:20:31.120,0:20:34.960 +a suitable pace for priority propogation + +0:20:34.960,0:20:40.820 +but what happens is that this owner is actually sleeping on another turnstile + +0:20:40.820,0:20:43.419 +and the other owner + +0:20:43.419,0:20:48.820 +of the second turnstile has always the same priority of its sleepers + +0:20:48.820,0:20:50.750 +so + +0:20:50.750,0:20:55.530 +just propogating priority to the first owner was just unuseful because the first + +0:20:55.530,0:20:56.340 +one + +0:20:56.340,0:20:57.320 +could + +0:20:57.320,0:20:58.760 +still + +0:20:58.760,0:21:00.580 + keep the chain to a + +0:21:00.580,0:21:04.820 +lower priority so it's was going to be propogated to the first one + +0:21:04.820,0:21:07.679 + actually running + +0:21:07.679,0:21:09.870 +owner of the chain + +0:21:09.870,0:21:14.670 +this is the situation after the propogation as you can see all of threads in the chain + +0:21:14.670,0:21:16.559 +has the same priority + +0:21:16.559,0:21:17.950 +either possible + +0:21:17.950,0:21:24.480 +in this case the one the last one arriving + +0:21:25.750,0:21:31.720 +there are question about that + +0:21:31.720,0:21:34.780 +no? + +0:21:34.780,0:21:36.760 +yeah when the + +0:21:36.760,0:21:39.720 +when the for example the third owner + +0:21:39.720,0:21:41.679 +the second owner there + +0:21:41.679,0:21:43.659 +when it goes to release the lock + +0:21:43.659,0:21:47.010 +it basically brings back the priority to the + +0:21:47.010,0:21:49.340 +to the + +0:21:49.340,0:21:52.490 +twenty hundred and sixty five to all the chains + +0:21:52.490,0:21:54.650 +he is responsible for + +0:21:54.650,0:22:01.179 +so it just happens at locking operation + +0:22:01.179,0:22:04.159 +and that is what we do about the Priority Inversion + +0:22:04.159,0:22:09.970 +inorder to fix instead the overhead given by the + +0:22:09.970,0:22:14.030 +big amount of context switch we use another technique called adaptive spinning + +0:22:14.030,0:22:16.030 +basically + +0:22:16.030,0:22:20.260 +as the context switch brings a lot of overhead + +0:22:22.310,0:22:26.090 +we prefer to not do + +0:22:26.090,0:22:27.770 +completely a context switch + +0:22:27.770,0:22:30.760 +in the case the lock owner is still running + +0:22:30.760,0:22:32.190 +on a runqueue + +0:22:32.190,0:22:38.340 +because there are very good chance that the owner is going to release the lock very early + +0:22:40.440,0:22:43.990 +that means that for example + +0:22:43.990,0:22:46.070 +we choose just to spin + +0:22:46.070,0:22:49.149 +in order to wait that the state of the + +0:22:49.149,0:22:52.240 +lock changed or the state of the owner + +0:22:52.240,0:22:57.660 +was going to change like the owner going to sleep on another turstile + +0:22:57.660,0:22:59.140 +and the + +0:22:59.140,0:23:03.270 +basically we, there have been very big measurement even in the + +0:23:03.270,0:23:07.510 +another operating system like solice that + +0:23:07.510,0:23:12.300 +where I think we brought in this approach the first time + +0:23:12.300,0:23:16.430 +that we're we're showing + +0:23:16.430,0:23:23.430 +a very big improvement in performance from this technique + +0:23:25.790,0:23:30.640 +apart from the two types of primitives, these are sleeping primitives + +0:23:30.640,0:23:36.120 +now there is a consideration we have to make about that + +0:23:36.120,0:23:38.110 +basically sleeping primitives + +0:23:38.110,0:23:42.320 +should be in theory just the + +0:23:42.320,0:23:44.340 +the wait channels + +0:23:44.340,0:23:49.170 +wait channels should have been the only one implemented using the + +0:23:49.170,0:23:50.630 +container called + +0:23:50.630,0:23:52.760 +sleepqueue + +0:23:52.760,0:23:53.910 +but + +0:23:53.910,0:23:56.170 +due to some legacy + +0:23:56.170,0:24:01.000 +the actually the sleepqueues were used to implement other kind of other + +0:24:01.000,0:24:03.290 +kinds of lock like the + +0:24:03.290,0:24:04.219 +lockmgr + +0:24:04.219,0:24:08.080 +and the sx locks and the + +0:24:08.080,0:24:11.100 +basically the + +0:24:11.100,0:24:13.679 +semaphore's condvars too + +0:24:13.679,0:24:16.010 +that has been this is + +0:24:16.010,0:24:18.809 +going to give some problems actually + +0:24:18.809,0:24:19.350 +because + +0:24:20.450,0:24:24.820 +as we're going to see + +0:24:24.820,0:24:26.889 +and as you can see on the line too + +0:24:26.889,0:24:27.929 +in the FreeBSD + +0:24:27.929,0:24:31.600 +while sleeping threads should not hold any kind of lock + +0:24:31.600,0:24:33.809 +neither blocking nor spinning + +0:24:33.809,0:24:36.770 +thats a simple thing to explain + +0:24:36.770,0:24:40.200 +we just want to enforce very + +0:24:40.200,0:24:43.490 +we just want to enforce + +0:24:43.490,0:24:46.060 +correct semantics of locking + +0:24:46.060,0:24:47.880 +so imagine to keep a lock + +0:24:47.880,0:24:50.190 +a blocking primitive while + +0:24:50.190,0:24:50.729 +sleeping + +0:24:50.729,0:24:53.010 +it's going to waste a lot of time + +0:24:53.010,0:24:56.530 +because all the contenders are going to + +0:24:56.530,0:24:58.760 +are going to start on the + +0:24:58.760,0:25:01.400 +lock owner which is sleeping + +0:25:01.400,0:25:03.120 +basically in fact what + +0:25:03.120,0:25:07.169 +as you should know condition variables do usually is to drop the lock + +0:25:07.169,0:25:11.070 +once it was passed to the primitives + +0:25:11.070,0:25:12.380 +in this case + +0:25:14.170,0:25:18.249 +basically we just dont allow that this means that's the + +0:25:18.249,0:25:23.160 +the same conditions happens even for other kinds of lock + +0:25:23.160,0:25:25.540 +lockmgr and the sx lock + +0:25:25.540,0:25:26.860 +so you cant hold + +0:25:26.860,0:25:29.410 +a mutex for example + +0:25:29.410,0:25:33.640 +of blocking mutex an R/W lock while trying to acquire + +0:25:33.640,0:25:38.559 +a lockmgr and sx + +0:25:38.559,0:25:41.850 +this is going to create some problems because + +0:25:41.850,0:25:46.830 +in some parts that is unavoidable so you have to drop the lock for example and try + +0:25:46.830,0:25:48.190 +to acquire + +0:25:48.190,0:25:49.770 +the other primitive + +0:25:49.770,0:25:51.320 +which is going to + +0:25:53.400,0:25:59.110 +and so can create some raisee problems + +0:26:00.130,0:26:04.779 +as the sleepqueues are born just to serve wait channels + +0:26:04.779,0:26:09.190 + they don't track owner too so they dont care about priority propogation and priority inversion problem + +0:26:09.190,0:26:14.430 +just because sleepqueues entirely should not have work + +0:26:14.430,0:26:20.150 +so for example lockmgr and sx have not priority propogation + +0:26:20.150,0:26:22.360 +systems and the + +0:26:22.360,0:26:29.360 +so they are discouraged to be used even for this thing mainly + +0:26:31.590,0:26:34.930 +sure + +0:26:36.780,0:26:39.000 +it's you mean why it's not + +0:26:39.000,0:26:41.790 +why doesnt blocking primitives exist yeah? + +0:26:41.790,0:26:44.250 +so imagine that for example the + +0:26:44.250,0:26:45.570 +you have a wait channel + +0:26:45.570,0:26:47.679 +condvar a condition variable + +0:26:47.679,0:26:50.950 +or M sleep + +0:26:50.950,0:26:52.090 +M sleep + +0:26:52.090,0:26:54.910 +the primitive that allows you to sleep on + +0:26:54.910,0:26:57.850 +a condition variable for example + +0:26:57.850,0:26:58.870 +however + +0:27:00.510,0:27:02.270 +the you are + +0:27:02.270,0:27:03.350 +using the blocking + +0:27:03.350,0:27:06.930 +the using the turnstile you will go to a + +0:27:06.930,0:27:12.110 +always the mechanism of priority propogation and priority inversion handling.Its + +0:27:12.110,0:27:13.760 +not very + +0:27:13.760,0:27:14.970 +it's pretty + +0:27:14.970,0:27:17.320 +it's not a simple operation + +0:27:17.320,0:27:20.219 +it acquires even some kind of spin locks + +0:27:20.219,0:27:22.650 + in order to avoid some raises + +0:27:22.650,0:27:23.340 +and so + +0:27:23.340,0:27:24.289 +it + +0:27:24.289,0:27:26.590 +so it has an overhead + +0:27:26.590,0:27:31.770 +if you do in this case it will be not to be useful it will be completely unuseful to have + +0:27:31.770,0:27:34.159 +a mechanism like that so + +0:27:34.159,0:27:37.410 +in theory if you just would have used + +0:27:37.410,0:27:41.320 +a sleeping the sleepqueue for wait channels + +0:27:41.320,0:27:42.990 +you are to add + +0:27:42.990,0:27:46.640 +bigperformance boost than just using the turnstile + +0:27:46.640,0:27:49.330 +for the same problem + +0:27:49.330,0:27:51.310 +in theory + +0:27:51.310,0:27:54.780 +but what happened is that other locks are implementedo + +0:27:54.780,0:27:55.839 +using this sleepqueue + +0:27:55.839,0:27:58.070 +that should have not be happened + +0:27:58.070,0:27:59.260 +on the principle + +0:27:59.260,0:28:02.960 +really I'm not sure who introduced the sx lock + +0:28:02.960,0:28:04.440 +I'm actually not sure + +0:28:04.440,0:28:06.280 +and even the lockmgr + +0:28:06.280,0:28:09.870 +but + +0:28:09.870,0:28:12.340 +however + +0:28:12.340,0:28:17.669 +as you could have seen before the three containers create a heirarchy that + +0:28:17.669,0:28:20.090 +should not be broken like + +0:28:20.090,0:28:21.639 +you have spinqueues + +0:28:21.639,0:28:26.900 +you have spin locks you have blocking primitives and sleeping primitives and + +0:28:26.900,0:28:31.470 +you cannot acquire you cannot mix them there are precise rules like + +0:28:31.470,0:28:33.710 +on the top the sleeping primitive + +0:28:33.710,0:28:37.690 +in the mid the blocking primitive and in the end the spinning primitive + +0:28:38.900,0:28:44.440 +the main choice will be to use blocking primitives always + +0:28:44.440,0:28:48.240 +because as you can see we handled a lot of problem that they have + +0:28:48.240,0:28:49.659 +and the practice + +0:28:49.659,0:28:52.229 +they have proven to be very + +0:28:52.229,0:28:53.799 +very helpful + +0:28:53.799,0:28:54.999 +but sometimes + +0:28:56.789,0:28:58.790 +some nasty conditions can happen + +0:28:58.790,0:29:02.900 +for example one of the most widespread is the + +0:29:02.900,0:29:06.350 +using a mallok with a flag M_WAITOK + +0:29:06.350,0:29:11.240 +in FreeBSD that means that if the allocator is pretty busy or going to + +0:29:11.240,0:29:12.680 + to sleep + +0:29:12.680,0:29:15.760 +in order to retreive your memory + +0:29:15.760,0:29:17.890 +and if you do with a lock hold + +0:29:17.890,0:29:22.080 +you're going to violate one of our rules and its not + +0:29:22.080,0:29:23.440 +possible + +0:29:23.440,0:29:25.320 +another one is just we just + +0:29:25.320,0:29:28.299 +said before like call a sleeping lock while + +0:29:28.299,0:29:32.090 +holding a blocking primitive + +0:29:33.390,0:29:37.530 +in the next example in the next I'm going to show you a way to + +0:29:37.530,0:29:41.140 +to handle for example the Mallock case + +0:29:41.140,0:29:42.520 +and similar + +0:29:42.520,0:29:45.000 +but the that usually + +0:29:46.830,0:29:47.620 +usually that + +0:29:47.620,0:29:49.980 +are not very common cases + +0:29:49.980,0:29:52.920 +at least for simple parts + +0:29:52.920,0:29:56.280 +you should even try to avoid the + +0:29:56.280,0:30:03.280 +the + +0:30:04.620,0:30:06.180 +yes + +0:30:06.180,0:30:07.050 +even in the + +0:30:07.050,0:30:09.120 +in the + +0:30:09.120,0:30:10.220 +wait channel + +0:30:10.220,0:30:14.530 +as in the FreeBSD you can differentiate between the condition variables and + +0:30:14.530,0:30:15.720 +Msleep + +0:30:15.720,0:30:17.510 + usually Msleep was + +0:30:17.510,0:30:22.210 +really Msleep was introduced as the first primitive + +0:30:22.210,0:30:26.190 +but it has an interface very very difficult to + +0:30:26.190,0:30:28.460 + to make saner and to understand + +0:30:28.460,0:30:30.470 +at least for + +0:30:30.470,0:30:31.220 +for people + +0:30:31.220,0:30:32.120 +which are + +0:30:32.120,0:30:34.960 +comfortable with + +0:30:34.960,0:30:39.260 +with interface of condition variable that we all saw but they are + +0:30:39.260,0:30:40.649 +newer primitive + +0:30:40.649,0:30:42.660 +mainly there is + +0:30:42.660,0:30:44.400 +so far the newer code + +0:30:44.400,0:30:46.960 +what you should do is just to + +0:30:46.960,0:30:49.000 +use condition variables + +0:30:49.000,0:30:50.659 +and not Msleep + +0:30:50.659,0:30:51.630 +basically + +0:30:51.630,0:30:56.220 +Msleep should be dropped off but they have avery nice feature which + +0:30:56.220,0:31:02.669 +is the the possibility to specify a wake up priority on the sleeping threads + +0:31:02.669,0:31:04.740 +once they are asleep + +0:31:04.740,0:31:07.470 +that condvar still doesnt + +0:31:07.470,0:31:12.430 +maybe if we could port these features to the condition variables we we will be able + +0:31:12.430,0:31:13.659 +to completely drop off Msleep + +0:31:13.659,0:31:18.529 +from the work arena + +0:31:18.529,0:31:20.450 +this is a + +0:31:20.450,0:31:25.580 +simple case that it's going to show a way to + +0:31:26.620,0:31:30.670 +a simple way to deal with the for example + +0:31:30.670,0:31:34.100 +condition I told before the Mallock willing to + +0:31:34.100,0:31:35.390 +to sleep + +0:31:35.390,0:31:38.260 +and the doing that while holding a lock + +0:31:38.260,0:31:45.070 +as you see we have some fake C as some members like flags + +0:31:45.070,0:31:47.659 +and an object called instructful + +0:31:47.659,0:31:49.940 +which needs to be allocated + +0:31:49.940,0:31:54.400 +and that they are protected by an internal lock + +0:31:54.400,0:31:58.810 +you imagine that for example the fake C create + +0:31:58.810,0:32:02.269 +holds lock of the object and does some things + +0:32:02.269,0:32:04.460 +which are not important + +0:32:04.460,0:32:07.650 +then in the end for example it's going to + +0:32:07.650,0:32:09.170 +to allocate + +0:32:09.170,0:32:14.110 +the FC object and that should be protected in + +0:32:14.110,0:32:16.470 +in anatomic part + +0:32:16.470,0:32:20.030 +something you can do is just to set the flag + +0:32:20.030,0:32:22.160 +for that + +0:32:22.160,0:32:22.730 +saying + +0:32:22.730,0:32:28.460 +the allocation is going to happen if you're adjust to this structure concurrently + +0:32:28.460,0:32:29.899 +just keep the allocation + +0:32:29.899,0:32:31.500 +and that's what we do + +0:32:31.500,0:32:32.919 +we check for this flag + +0:32:32.919,0:32:37.969 +and if its present it means that another thread is still + +0:32:37.969,0:32:40.149 +is already allocating and we just keep + +0:32:40.149,0:32:46.360 +so otherwise we set it and then we have locked the mutex + +0:32:46.360,0:32:49.100 +then we allocate the memory for the + +0:32:49.100,0:32:50.610 +for the object + +0:32:50.610,0:32:52.450 +acquire again the lock + +0:32:52.450,0:32:54.860 +and we simply have seen + +0:32:54.860,0:33:00.200 +please note that Ive used the temporary storage for that in order to make + +0:33:00.200,0:33:01.830 +some search on + +0:33:01.830,0:33:03.280 +like the MS + +0:33:03.280,0:33:04.180 +about the + +0:33:04.180,0:33:05.500 +the pointer + +0:33:05.500,0:33:10.700 +it was just a tricky note that you verify that really the structure was not + +0:33:10.700,0:33:14.330 +really allocated + +0:33:14.330,0:33:16.600 +and so that we can get some + +0:33:16.600,0:33:21.870 +kind of session about that + +0:33:22.640,0:33:26.340 +one of the biggest innovation that was brought to FreeBSD + +0:33:26.340,0:33:30.120 +about the locking primitive about the locking primitives + +0:33:30.120,0:33:33.770 +are the interrupts that + +0:33:34.640,0:33:36.850 +mainly + +0:33:36.850,0:33:40.820 +this is pretty simple to explain maybe + +0:33:40.820,0:33:44.070 +As the top half remains basically the same + +0:33:44.070,0:33:49.790 +and was going to handle the ISR for the interrupt line for example + +0:33:49.790,0:33:54.330 +the bottom half changed set and running the interrupts + +0:33:54.330,0:33:58.700 +handler is solid on that line as it was traditionally happened + +0:33:58.700,0:34:02.140 +it was going just to schedule a thread + +0:34:02.140,0:34:04.980 +that was going to run the + +0:34:04.980,0:34:06.940 +the interrupt handler in a + +0:34:06.940,0:34:12.389 +--- context and not the kind of --it was going to happen + +0:34:12.389,0:34:15.509 +traditionally in a lot of unique system + +0:34:16.699,0:34:23.179 +this has the big advantage that in using your own context you can + +0:34:23.179,0:34:24.429 +basically + +0:34:24.990,0:34:29.889 +you're not forced to use spin locks and you can do a lot of other fancy things + +0:34:29.889,0:34:32.209 +this necesity came over because + +0:34:32.209,0:34:33.149 +often + +0:34:33.149,0:34:38.529 +interrupts handlers needs to adjust to some + +0:34:38.529,0:34:42.589 +needs to adjust to some subsystem locks and the + +0:34:42.589,0:34:45.799 +as we were going to use blocking ---around + +0:34:45.799,0:34:50.379 +we had the necessity to support the + +0:34:50.379,0:34:52.589 +the locking of the + +0:34:52.589,0:34:57.119 +the possibilities of wide mutex actually + +0:34:57.559,0:35:01.759 +A similar thing was implemented using taskqueues + +0:35:01.759,0:35:02.879 +previously + +0:35:02.879,0:35:04.010 +and the sometimes it + +0:35:04.010,0:35:05.740 +I think I saw a lenux too + +0:35:05.740,0:35:08.439 +using taskqueues maybe + +0:35:08.439,0:35:10.029 +but the + +0:35:10.029,0:35:14.709 +it was basically something similar but not exactly in this way + +0:35:14.709,0:35:16.809 +a actually FreeBSD + +0:35:16.809,0:35:20.559 + from the release seven + +0:35:20.559,0:35:22.579 +the interrupt threads + +0:35:22.579,0:35:24.659 +are this model is a little bit changed + +0:35:24.659,0:35:26.499 +in order to include the + +0:35:26.499,0:35:29.739 +a new mechanism called the filtering + +0:35:29.739,0:35:36.249 +we have interrupt filters that basically if set then directly + +0:35:36.249,0:35:39.809 +directly + +0:35:39.809,0:35:40.879 +schedule the thread + +0:35:40.879,0:35:43.209 +linked to the parked line + +0:35:43.209,0:35:46.619 +they just check for + +0:35:46.619,0:35:50.939 +they just let run some new thing in the kernel or context + +0:35:50.939,0:35:52.449 +that will decide if + +0:35:52.449,0:35:56.709 + handle directly to requests or just schedule the kernel + +0:35:56.709,0:35:59.739 +it's like if you have the old bottom handler + +0:35:59.739,0:36:04.529 +that add the possibility to register a handler + +0:36:04.529,0:36:08.869 +still running in interrupt context and at the same time + +0:36:08.869,0:36:12.009 +decide if scheduled or not + +0:36:12.009,0:36:14.499 +so that it's no + +0:36:14.499,0:36:18.579 +no more madatory + +0:36:18.579,0:36:22.919 +So I think that the first part is going to finish so if you have some questions we can + +0:36:22.919,0:36:23.430 +handle + +0:36:23.430,0:36:28.699 + right now + +0:36:28.699,0:36:35.699 +this should be material for the second part actually + +0:36:45.279,0:36:48.529 +a new bus for example + +0:36:48.529,0:36:51.259 +some + +0:36:51.259,0:36:55.769 +some drivers that kind of a frequently used I'm not sure but which ones but all + +0:36:55.769,0:37:00.049 +the big ones are compared to finer locking + +0:37:00.049,0:37:03.109 +%um + +0:37:03.109,0:37:07.479 +actually the problem is not which parts are under Giant + +0:37:07.479,0:37:08.530 +well how we could + +0:37:08.530,0:37:12.380 +optimize the locking of some subsystems because + +0:37:12.380,0:37:15.079 +for example we have to virtual memory + +0:37:15.079,0:37:17.910 +which is not on the Giant but its + +0:37:17.910,0:37:19.719 +not locate + +0:37:19.719,0:37:24.400 +optimally and it's going to bring a lot of contention + +0:37:24.400,0:37:26.230 +so + +0:37:26.230,0:37:30.329 +it's not under Giant but it should be optimized + +0:37:30.329,0:37:37.329 + because the parts under Giant are very tiny.New bus for example + +0:37:37.599,0:37:44.599 +some parts relating to the VFS on the mounting +but yet a very short parts + +0:37:44.979,0:37:51.979 +I'm not sure about others + +0:37:57.479,0:37:59.170 +sorry + +0:38:02.069,0:38:08.549 +well usually it should be moved completely but + +0:38:08.549,0:38:11.019 +yes + +0:38:11.019,0:38:12.539 +it could + +0:38:32.909,0:38:34.809 +okay although + +0:38:34.809,0:38:38.289 +in the kernel we have a basically + +0:38:38.289,0:38:39.450 +%um + +0:38:39.450,0:38:43.019 +as you should know we already imported the trays for example + +0:38:43.019,0:38:47.839 +and I have wondered, I have submitted by developed + +0:38:47.839,0:38:48.669 +my country + +0:38:48.669,0:38:51.479 +called ---some patches that brings the + +0:38:51.479,0:38:54.689 +the ----- directly in our locking + +0:38:54.689,0:38:55.699 +in order to + +0:38:55.699,0:38:58.890 +allow it to be tracked with the trace. + +0:38:58.890,0:39:02.009 +which is very nice but it's still not completed + +0:39:02.009,0:39:03.310 +we are reviewing + +0:39:03.310,0:39:08.309 +above that we have a very the other useful tool called the lock profiling + +0:39:08.309,0:39:12.039 +that has been very helpful in the past in order to + +0:39:12.039,0:39:14.110 +find the most contended lock + +0:39:14.110,0:39:17.469 +and the to try to propose them to finer locking + +0:39:17.469,0:39:20.589 +so at least for the kernel we have such mechanism + +0:39:20.589,0:39:22.719 +I'm not sure what should + +0:39:22.719,0:39:26.640 +have been the user space.I'm sure we've not something similar + +0:39:26.640,0:39:28.310 +but maybe other systems + +0:39:28.310,0:39:29.469 +have + +0:39:29.469,0:39:30.749 +similar tools + +0:39:30.749,0:39:36.039 +I don't know I just know FreeBSD so + +0:39:58.479,0:39:59.220 +not sure + +0:39:59.220,0:39:59.919 +would you repeat + +0:39:59.919,0:40:03.879 + some voice please. No I cant hear + +0:40:03.879,0:40:05.509 +It seems to me that + +0:40:05.509,0:40:08.269 +you don't you have to do all the work that you do with locking + +0:40:08.269,0:40:11.469 +well if you're not on SMP right? + +0:40:11.469,0:40:13.029 +well no + +0:40:13.029,0:40:15.259 +it's not right because the + +0:40:15.259,0:40:20.210 +you have to protect even against some mechanism like preemption + +0:40:20.210,0:40:25.989 +which is going to be tricky.It is dfferent implemented than FreeBSD 4.x so + +0:40:25.989,0:40:28.909 +it's going to be with preemption its like + +0:40:28.909,0:40:30.099 +from + +0:40:30.099,0:40:34.479 +it's like if you have a real SMP system from our technical point of view + +0:40:34.479,0:40:35.809 +so you have to handle + +0:40:35.809,0:40:38.339 +problems typical of that + +0:40:38.339,0:40:43.249 +really in the kernel we have other kind of synchronization like atomics + +0:40:43.249,0:40:45.500 +I don't, I should have had + +0:40:45.500,0:40:50.609 +a slide about that but it disappeared so I can tell you by voice + +0:40:50.609,0:40:55.170 +its well like we have the possibility to use atomic instruction in the + +0:40:55.170,0:40:57.369 +in FreeBSD kernel directly + +0:40:57.369,0:40:59.249 +but the + +0:40:59.249,0:41:03.119 +to use even memory bytes linked with them + +0:41:03.119,0:41:08.869 +the only pitfall is that you cannot really trust about the + +0:41:08.869,0:41:10.469 +cash coherency + +0:41:10.469,0:41:14.339 +because as long as it's Im be specific you can just + +0:41:14.339,0:41:16.989 +you can just be trust about + +0:41:16.989,0:41:21.879 +what happens in your CPU where use the atomic and where to use the memory byte + +0:41:21.879,0:41:26.349 + you cannot make assumptions about the what happens about if other CPUs + +0:41:26.349,0:41:29.289 +can see your modifiers or not + +0:41:29.289,0:41:31.640 +and if the cache can handle that + +0:41:31.640,0:41:37.119 +we have a specific primitives in order to for example disable preemption + +0:41:37.119,0:41:39.379 +which are the critical sections + +0:41:39.379,0:41:42.179 +critical entry and critical exit + +0:41:42.179,0:41:45.309 +that what you call them you are not to + +0:41:45.309,0:41:48.219 +the preemption is simply allowed + +0:41:48.219,0:41:54.749 +it's that's a very fast primitive so there is not much overhead + +0:41:54.749,0:41:56.049 +so there's not much overhead + +0:41:56.049,0:42:00.679 +we also have a way to disable interrupt which is unofficial.I will tell + +0:42:00.679,0:42:03.079 +that + +0:42:03.079,0:42:07.720 +because you can do that in machine dependant way + +0:42:07.720,0:42:10.619 +with a spin lock entry and spin lock exit + +0:42:10.619,0:42:14.989 + and then + +0:42:14.989,0:42:16.049 +yeah that you can + +0:42:16.049,0:42:17.389 +even disable + +0:42:17.389,0:42:19.479 +some thread migration + +0:42:19.479,0:42:22.940 + using skid primitives + +0:42:22.940,0:42:25.319 +that are very useful + +0:42:25.319,0:42:29.779 +when you are going to adjust for example to per-CPU datas + +0:42:29.779,0:42:33.270 +and you have several chases and you don't want the CPU migrate + +0:42:33.270,0:42:34.200 +from that + +0:42:34.200,0:42:36.619 +thread migrate from that CPU + +0:42:36.619,0:42:38.729 +because you could read different + +0:42:38.729,0:42:45.369 +values from different CPU then + +0:42:45.369,0:42:46.479 +I'm not sure + +0:42:46.479,0:42:52.079 +if there is something else okay + +0:42:52.079,0:42:57.229 +questions? no? + +0:42:57.229,0:42:58.189 +so i'll see you later" diff --git a/en_US.ISO8859-1/captions/2009/dcbsdcon/davis-isolatingcluster.sbv b/en_US.ISO8859-1/captions/2009/dcbsdcon/davis-isolatingcluster.sbv new file mode 100644 index 0000000000..b05e61955b --- /dev/null +++ b/en_US.ISO8859-1/captions/2009/dcbsdcon/davis-isolatingcluster.sbv @@ -0,0 +1,3912 @@ +0:00:15.749,0:00:18.960 +I do apologize for the + +0:00:18.960,0:00:22.130 +for the Euro BSD Con slide. I've redone the + +0:00:22.130,0:00:23.890 +title page and redone the + +0:00:23.890,0:00:27.380 +and made some changes to the slides +and they didn't make it for approval + +0:00:27.380,0:00:33.130 +by this afternoon so + +0:00:33.130,0:00:34.640 + okay so + +0:00:34.640,0:00:36.390 +I'm gonna be talking about + +0:00:36.390,0:00:38.430 +doing + +0:00:38.430,0:00:42.889 +about isolating jobs for performance and predictability +in clusters + +0:00:42.889,0:00:43.970 +before I get into that + +0:00:43.970,0:00:46.010 +I want to talk a little bit about + +0:00:46.010,0:00:47.229 +who we are and + +0:00:47.229,0:00:49.520 +what our problem space is like because that + +0:00:49.520,0:00:54.760 +%uh a dictates that that %uh has an effect +are solutions base + +0:00:54.760,0:00:57.079 +I work for the aerospace corporation + +0:00:57.079,0:00:58.609 +%uh we we work + +0:00:58.609,0:01:02.480 + we operate a federally-funded +research and development center + +0:01:02.480,0:01:05.400 +%uh in the area national security space + +0:01:05.400,0:01:09.310 +and in particular we work with the air force +space and missile command + +0:01:09.310,0:01:13.090 +and with that the national reconnaissance +office + +0:01:13.090,0:01:16.670 + and our engineers support a wide variety + +0:01:16.670,0:01:20.550 +of activities within that area + +0:01:20.550,0:01:21.830 +so we have + +0:01:21.830,0:01:23.740 +a bit over fourteen hundred to correct + +0:01:23.740,0:01:25.860 +sorry twenty four hundred engineers + +0:01:25.860,0:01:28.820 + in virtually every discipline we have + +0:01:28.820,0:01:33.520 +as you would expect we have our rocket scientists + we have people who build satellites + +0:01:33.520,0:01:37.439 +we have people who build sensors that go on +satellites people who study this sort of things + +0:01:37.439,0:01:38.130 +that you + +0:01:38.130,0:01:39.590 +see when you + +0:01:39.590,0:01:40.819 +use those sensors + +0:01:40.819,0:01:42.040 +that sort of thing + +0:01:42.040,0:01:44.180 + we also have civil engineers and + +0:01:44.180,0:01:45.680 +electronic engineers + +0:01:45.680,0:01:46.649 +and process + +0:01:46.649,0:01:49.170 +computer process people + +0:01:49.170,0:01:53.120 +so we literally do everything related to space +and all sorts of things that you might not + +0:01:53.120,0:01:55.270 +expect to be related to space + +0:01:55.270,0:01:58.820 +because we also for instance help build ground +systems since satellites arent very useful if + +0:01:58.820,0:02:00.680 +there isn't anything to talk to them + +0:02:00.680,0:02:02.540 +%um + +0:02:02.540,0:02:04.090 +and these engineers + +0:02:04.090,0:02:07.420 +since they're solving all these different problems we have + +0:02:07.420,0:02:11.499 +engineering applications in you know +virtually every size you can think of + +0:02:11.499,0:02:15.539 +ranging from you know little spreadsheet things that +you might not think of as an engineering + +0:02:15.539,0:02:17.229 +application but they are + +0:02:17.229,0:02:22.249 +to Matlab programs or want to see code + +0:02:22.249,0:02:23.960 +or one of traditional parallel for us + +0:02:23.960,0:02:25.159 +serial code + +0:02:25.159,0:02:26.049 +and then + +0:02:26.049,0:02:30.949 +large parallel applications either in house +or genetic algorithms and that sort + +0:02:30.949,0:02:31.769 +of thing + +0:02:31.769,0:02:32.900 +or traditional + +0:02:32.900,0:02:34.749 +the classic parallel code + +0:02:34.749,0:02:37.599 +like you work around a crater or something material simulation + +0:02:37.599,0:02:40.119 +%uh or %uh + +0:02:40.119,0:02:41.459 +or that or food flow + +0:02:41.459,0:02:43.869 +or that sort of thing + +0:02:43.869,0:02:44.240 +so + +0:02:44.240,0:02:46.349 +so we have this big application space + +0:02:46.349,0:02:49.029 +just want to give a little introduction to that because +it + +0:02:49.029,0:02:51.529 +does come back and influence what we + +0:02:51.529,0:02:55.999 +the sort of solutions we work at + +0:02:55.999,0:03:00.499 +so the rest of the talk Im gonna talk about oops + +0:03:00.499,0:03:05.259 +we skipped a slide, There we are. Thats a little better + +0:03:05.259,0:03:08.940 +what I'm interested in is I do high +performance computing + +0:03:08.940,0:03:10.109 +at company + +0:03:10.109,0:03:13.949 +and I provide high performance computing resources +to our users + +0:03:13.949,0:03:19.949 +as part of my role in our technical +computing services organization + +0:03:19.949,0:03:20.370 +so + +0:03:20.370,0:03:23.120 +our primary resource at this point is + +0:03:23.120,0:03:25.429 + the fellowship cluster + +0:03:25.429,0:03:26.540 +it's a for the + +0:03:26.540,0:03:29.569 +named for the fellowship the ring + +0:03:29.569,0:03:30.449 +it's the + +0:03:30.449,0:03:32.520 +we're gonna wrap some nodes + +0:03:32.520,0:03:33.930 +wrap the core systems + +0:03:33.930,0:03:35.909 +%uh over here there's a + +0:03:35.909,0:03:39.659 +Cisco a large Cisco switch. Actually today +there are around two sixty five oh nines if + +0:03:39.659,0:03:40.899 +you assess them + +0:03:40.899,0:03:46.149 +and because we couldnt get the core density otherwise + +0:03:46.149,0:03:50.219 +and primarily the Gigabit Ethernet system runs +FreeBSD currently 6.0 because we havent upgraded + +0:03:50.219,0:03:51.089 +it yet + +0:03:51.089,0:03:55.639 +planning to move to probably to 7.1 +maybe slightly past 7.1 + +0:03:55.639,0:04:01.029 +%uh if we want to get the latest initial APM changes in + +0:04:01.029,0:04:05.900 +we use the Sun Grid Engine scheduler was one of +the two main options for open source + +0:04:05.900,0:04:08.949 +resource managers on cluster the other one being +that + +0:04:08.949,0:04:09.959 +the %uh + +0:04:09.959,0:04:11.499 +Torp + +0:04:11.499,0:04:15.939 +and now the combination from cluster resources + +0:04:15.939,0:04:17.389 +so we also have + +0:04:17.389,0:04:18.079 + that's actually + +0:04:18.079,0:04:22.090 +40 TB thats really the raw number on a sun thumper and + +0:04:22.090,0:04:23.219 +and + +0:04:23.219,0:04:26.290 +that thirty two usable once you start using ---- two + +0:04:26.290,0:04:30.939 +since you might actually like to have your data +should a disk fail + +0:04:30.939,0:04:32.969 +and with today's discs drade + +0:04:32.969,0:04:34.009 +grade five + +0:04:34.009,0:04:35.249 +doesn't really cut it + +0:04:35.249,0:04:37.379 +%um + +0:04:37.379,0:04:40.220 +we also have some other resources coming on but Im going to be + +0:04:40.220,0:04:43.530 +two smaller clusters unfortunately probably running Sun x and + +0:04:43.530,0:04:45.900 + some SMPs but + +0:04:45.900,0:04:49.990 +Im going to be concentrating here on the work we're +doing on our other + +0:04:49.990,0:04:54.259 +our FreeBSD based cluster + +0:04:54.259,0:04:55.060 +first of all + +0:04:55.060,0:04:59.410 +first of all I want to talk about why we want to +share resources. Should be fairly obvious + +0:04:59.410,0:05:00.610 +but I'll talk about it in a little bit + +0:05:00.610,0:05:04.900 +and then what goes wrong when you start sharing resources + +0:05:04.900,0:05:08.449 +after that I'll talk about some different solutions +to those problems + +0:05:08.449,0:05:09.759 +and + +0:05:09.759,0:05:13.399 +some fairly trivial experiments that we've done +so far in terms of the it's enghancing the schedule or + +0:05:13.399,0:05:15.860 +using operating system features + +0:05:15.860,0:05:17.730 +so you mitigate those problems + +0:05:17.730,0:05:19.349 +%um + +0:05:19.349,0:05:20.110 +and %uh + +0:05:20.110,0:05:25.110 +then conclude with some future work + +0:05:25.110,0:05:29.289 +obviously if you have a resource the size +of the size of our cluster fourteen hundred + +0:05:29.289,0:05:30.970 +cores roughly + +0:05:30.970,0:05:32.819 +you probably want to share it unless you + +0:05:32.819,0:05:35.080 +purpose built it for a single application + +0:05:35.080,0:05:37.340 + you're going to want to have your users + +0:05:37.340,0:05:39.440 +sharing it + +0:05:39.440,0:05:42.909 +and you don't want to just say you know you get on Monday + +0:05:42.909,0:05:45.330 +probably not going to be a very effective +option + +0:05:45.330,0:05:49.270 +especially not when we have as many uses we +do + +0:05:49.270,0:05:53.849 +we also can't just afford to buy another one +every time a user shows up + +0:05:53.849,0:05:54.959 +so one of our + +0:05:54.959,0:05:57.339 +senior VPs said a while back + +0:05:57.339,0:05:57.969 +you know + +0:05:57.969,0:06:02.349 +we could probably afford to buy just about +anything we could need once + +0:06:02.349,0:06:03.800 + we can't just + +0:06:03.800,0:06:06.359 +buy ten of them though + +0:06:06.359,0:06:08.939 +if you really really needed it + +0:06:08.939,0:06:09.680 +dropping + +0:06:09.680,0:06:11.460 +small numbers of millions of dollars on + +0:06:11.460,0:06:13.349 +computing resources wouldnt be + +0:06:13.349,0:06:15.039 +impossible + +0:06:15.039,0:06:20.829 +but we can't go to you know just have every engineer +who wants one just call Dell and say ship me ten racks + +0:06:20.829,0:06:24.030 +it's not going to work + +0:06:24.030,0:06:25.580 +and the other thing is that we cant + +0:06:25.580,0:06:28.360 +we need to also provide quick turnaround + +0:06:28.360,0:06:29.390 +for some users + +0:06:29.390,0:06:33.229 +so we can't have one user hogging the system and +hogging it until they are done + +0:06:33.229,0:06:34.720 +because we have some users + +0:06:34.720,0:06:37.099 +n then the next one can run + +0:06:37.099,0:06:40.949 +because we have some users who'll +come in and say well I need to run + +0:06:40.949,0:06:43.159 +for three months + +0:06:43.159,0:06:43.690 +and + +0:06:43.690,0:06:46.810 +we've had users come in and literally run + +0:06:46.810,0:06:49.740 +pretty much using the entire system for three months + +0:06:49.740,0:06:53.839 +well so we've had to provide some ability for other +users to still get their work done + +0:06:53.839,0:06:58.300 +so we can't just.. so we do have to have some share + +0:06:58.300,0:07:00.619 +however when you start to share any resource + +0:07:00.619,0:07:01.610 +like this + +0:07:01.610,0:07:03.509 +you start getting contention + +0:07:03.509,0:07:06.300 +users need the same thing at the same time + +0:07:06.300,0:07:09.700 +and so they fight back and forth for it and they +can't get what they want + +0:07:09.700,0:07:11.639 +so you have to balance them a bit + +0:07:11.639,0:07:12.999 +%um + +0:07:12.999,0:07:14.529 +you know also + +0:07:14.529,0:07:17.869 +%uh some jobs lie when they + +0:07:17.869,0:07:20.870 +request resources and they actually need +more than they ask for + +0:07:20.870,0:07:23.279 +which can cause problems + +0:07:23.279,0:07:27.229 +so we schedule them. We say you're going to fit +here fine and they run off and use + +0:07:27.229,0:07:28.580 +more than they said + +0:07:28.580,0:07:31.000 +and if we don't have a mechanism to constrain +them + +0:07:31.000,0:07:32.389 +we have problems + +0:07:32.389,0:07:34.270 +%uh likewise + +0:07:34.270,0:07:37.109 +once these users start to contend + +0:07:37.109,0:07:39.029 +that doesn't just result in + +0:07:39.029,0:07:40.439 +the jobs taking + +0:07:40.439,0:07:43.360 +taking longer in terms of wall clock time + +0:07:43.360,0:07:44.659 +because they are extremely slow + +0:07:44.659,0:07:48.430 +but there's overhead related to that contention +because they get swapped out due to to that pressure on + +0:07:48.430,0:07:49.219 +on + +0:07:49.219,0:07:51.509 +on various systems + +0:07:51.509,0:07:52.550 +if you really + +0:07:52.550,0:07:57.039 +for instance run put of memory then you go into +swap and you end up wasting all your cycles + +0:07:57.039,0:07:58.710 +pulling junk in and out of disc + +0:07:58.710,0:08:00.830 +wasting your bandwidth on that + +0:08:00.830,0:08:03.530 +so there are + +0:08:03.530,0:08:04.219 +resource + +0:08:04.219,0:08:08.139 +there are resource cost to the contention not merely + +0:08:08.139,0:08:11.979 +a delay in returning results + +0:08:11.979,0:08:16.590 +so now I'm going to switch gears and start talk so I'm +going to talk a little bit about different + +0:08:16.590,0:08:18.270 +solutions to these + + +0:08:18.270,0:08:20.610 +to the + +0:08:20.610,0:08:22.339 +these contention issues + +0:08:22.339,0:08:23.710 +and %uh + +0:08:23.710,0:08:27.840 +%uh and and look at different ways of solving the +problem.most of these are things that have + +0:08:27.840,0:08:29.440 +already been done + +0:08:29.440,0:08:30.620 +but I just want to talk about + +0:08:30.620,0:08:32.990 +the different ways and then + +0:08:32.990,0:08:35.710 +evaluate them in our context + +0:08:35.710,0:08:38.119 +so a classic solution to the problem is + +0:08:38.119,0:08:39.280 +Gang scheduling + +0:08:39.280,0:08:44.139 + it's basically conventional Unex process +context switching + +0:08:44.139,0:08:46.560 +written really big + +0:08:46.560,0:08:50.339 +you what you do is you have your parallel +job thats running + +0:08:50.339,0:08:51.390 +on a system + +0:08:51.390,0:08:52.839 +and it runs for a while + +0:08:52.839,0:08:57.920 +and then after a certain amount of time you basically +shove it all you kick it off of all the nodes + +0:08:57.920,0:08:59.940 +and let the next one come in + +0:08:59.940,0:09:04.030 +and typically when people do this they do it on +on the order of hours because the context switch + +0:09:04.030,0:09:09.270 +time is extremely large is extremely high + +0:09:09.270,0:09:10.639 +for example + +0:09:10.639,0:09:14.530 +because it's not just like swapping a process +internet. You suddenly have to co-ordinate + +0:09:14.530,0:09:17.470 +the this context which across to all your processes + +0:09:17.470,0:09:19.280 +if you're running say + +0:09:19.280,0:09:21.190 +MPI over TCP + +0:09:21.190,0:09:25.910 +you actually need to tear down the TCP sessions +because you can't just have TCP timers sitting + +0:09:25.910,0:09:26.570 +around + +0:09:26.570,0:09:28.260 +or that sort of thing so + +0:09:28.260,0:09:29.950 +there there's a there's a lot of overhead + +0:09:29.950,0:09:34.340 +associated with this.You take a long context switch + +0:09:34.340,0:09:36.820 +if all of your infrastructure supports this + +0:09:36.820,0:09:39.420 + it's fairly effective + +0:09:39.420,0:09:43.300 +and it does allow jobs to avoid interfering +with each other which is nice + +0:09:43.300,0:09:46.100 +so you can't you don't have issues + +0:09:46.100,0:09:47.689 +because you're typically allocating + +0:09:47.689,0:09:50.950 +whole swaps of the system + +0:09:50.950,0:09:53.390 +and for properly written applications + +0:09:55.000,0:09:59.690 +partial results can be returned which for some of +our users is really important where you're doing a + +0:09:59.690,0:10:00.490 +refinement + +0:10:00.490,0:10:04.350 +users would want to look at the results and +say okay + +0:10:04.350,0:10:06.130 +you know is this just going off into the weeds + +0:10:06.130,0:10:10.860 +or does it look like it's actually converging on +some sort of useful solution + +0:10:10.860,0:10:13.980 +as they don't want to just wait till the end + +0:10:13.980,0:10:19.270 +down side of course is that this context +switches costs are very high + +0:10:19.270,0:10:22.460 +and most importantly there's really a lack +of useful implementations + +0:10:22.460,0:10:25.340 +a number of platforms have implemented this in the past + +0:10:25.340,0:10:29.840 +but in practice on modern clusters which are +built on commodity hardware + +0:10:29.840,0:10:32.340 +with you know + +0:10:32.340,0:10:35.530 +communication libraries written on standard protocols + +0:10:35.530,0:10:37.050 +the tools just arent there + +0:10:37.050,0:10:39.100 +and so + +0:10:39.100,0:10:40.860 +it's not very practical + +0:10:40.860,0:10:44.010 +also it doesn't really make a lot of sense with small jobs + +0:10:44.010,0:10:47.789 +and one of the things that we found is we have users who have + +0:10:47.789,0:10:50.860 +embarrassingly parallel problems for they need to look at + +0:10:50.860,0:10:53.450 +you know twenty thousand studies + +0:10:53.450,0:10:57.400 +and they could write something that looked more like a +conventional parallel application where they + +0:10:57.400,0:11:01.930 +you know wrote a schedule and set up an MPI a Message Pasting Interface + +0:11:01.930,0:11:05.400 +and handed out tasks to pieces of their job and then you +could do this + +0:11:05.400,0:11:09.280 +but then they would be running a schedule and they would +probably do a bad job of it turns out it's actually + +0:11:09.280,0:11:10.820 +fairly difficult to do right + +0:11:10.820,0:11:13.740 +even a trivial case + +0:11:13.740,0:11:16.189 +and so what they do instead is they just select twenty + +0:11:16.189,0:11:18.730 +twenty thousand jobs to great and say okay + +0:11:18.730,0:11:21.330 +whatever I'll deal with it + +0:11:21.330,0:11:23.140 +earlier versions that might have been a problem + +0:11:23.140,0:11:24.730 +current versions of the code + +0:11:24.730,0:11:27.060 + handle easily a million jobs that + +0:11:27.060,0:11:29.370 +so not really a big deal + +0:11:29.370,0:11:31.610 +but those sort of users wouldn't fit well + +0:11:31.610,0:11:34.190 +into the gang scheduled environment + +0:11:34.190,0:11:35.690 +at least not in a + +0:11:35.690,0:11:39.149 +conventional gang scheduled environment where +you do gang scheduling on the regularity of + +0:11:39.149,0:11:40.940 +jobs + +0:11:40.940,0:11:44.140 +so from that perspective it wouldnt work very well + +0:11:44.140,0:11:48.380 +if you have all the pieces in place and you are +doing a big parallel applications it in fact + +0:11:48.380,0:11:53.770 +an extremely effective approach + +0:11:53.770,0:11:56.290 +another option which is sort of related + +0:11:56.290,0:11:57.420 +it's in fact + +0:11:57.420,0:12:00.079 +take taking an even course with regularity + +0:12:00.079,0:12:04.360 +is single application or single project +clusters or sub-clusters + +0:12:04.360,0:12:07.590 +%uh for instance this is used some national labs + +0:12:07.590,0:12:11.910 +where you're given a cycle allocation for a +year based on your grant proposals + +0:12:11.910,0:12:14.779 +and what your cycle allocation actually comes to you as is + +0:12:14.779,0:12:16.580 +here's your cluster + +0:12:16.580,0:12:17.489 +here's a fun-ed + +0:12:17.489,0:12:19.840 +here's this chunk of notes. they're yours. Go to it + +0:12:19.840,0:12:21.930 +Install your own OS. Whatever you want + +0:12:21.930,0:12:25.580 +it's yours + +0:12:25.580,0:12:30.310 +and then and at a sort of finer scale there's things such as + +0:12:30.310,0:12:31.800 +you could use Emulab + +0:12:31.800,0:12:36.300 +which is the network emulation system but also does a less install and configuration + +0:12:36.300,0:12:39.300 +so you could do dynamic allocation that way + +0:12:39.300,0:12:40.540 +Sun's + +0:12:40.540,0:12:44.040 +Project Hedeby now actually I think it's +called service domain manager + +0:12:44.040,0:12:46.500 + is the product size version + +0:12:46.500,0:12:50.010 +or some Clusters on Demand + +0:12:50.010,0:12:54.450 +they were actually talking about web hosting clusters but + +0:12:54.450,0:12:57.780 +things that allow rapid deployment unless you +do that a little + +0:12:57.780,0:12:59.510 +little + +0:12:59.510,0:13:02.810 +a more granular level than the + +0:13:02.810,0:13:05.580 +the allocate them once a year approach + +0:13:05.580,0:13:07.720 +none the less + +0:13:07.720,0:13:11.220 +lets you give people whole clusters to work with + +0:13:11.220,0:13:12.920 +nice one nice thing about it is + +0:13:12.920,0:13:15.450 +the at the isolation between the processes + +0:13:15.450,0:13:16.890 +is complete + + +0:13:16.890,0:13:20.800 +so you dont have to worry about users stomping on each other. +Its their own system. they can trash it all they + +0:13:20.800,0:13:22.230 +want + +0:13:22.230,0:13:24.709 +if they flood the network or they + +0:13:24.709,0:13:26.180 +run the nodes into swap + +0:13:26.180,0:13:28.480 +well that's their problem + +0:13:28.480,0:13:32.120 +but it also has the advantage that you can tailor the the images + +0:13:32.120,0:13:36.980 +on the nodes of the operative systems to +meet the exact needs of the application + +0:13:36.980,0:13:40.560 +down side of course is its course +theres a system whichgranularity. in our environment that doesn't work + +0:13:40.560,0:13:41.500 +very well + +0:13:41.500,0:13:46.800 + since we do have all of these all these different types of jobs + +0:13:46.800,0:13:51.710 +context switches are also pretty expensive. certainly on the order of minutes + +0:13:51.710,0:13:54.690 +Emulab typically claim something like ten minutes + +0:13:54.690,0:13:57.970 +there are some systems out there + +0:13:57.970,0:14:03.320 +for instance if you use I think its open boot that +they're calling it today. It used to be 1xBIOS + +0:14:03.320,0:14:06.790 +where you can actually deploy a system in + +0:14:06.790,0:14:08.700 +tens of seconds + +0:14:08.700,0:14:11.520 +mostly by getting rid of all that junk the BIOS writers wrote + +0:14:11.520,0:14:12.890 +and + +0:14:12.890,0:14:17.770 +the OS speed is pretty fast if you dont have all +that stuff to waylay you not + +0:14:17.770,0:14:19.940 +but in practice on sort of + +0:14:19.940,0:14:21.660 +on the shelf + +0:14:21.660,0:14:24.400 +the context switches time quite high + +0:14:24.400,0:14:26.930 +users of course can interfere with themselves + +0:14:26.930,0:14:29.200 +you can argue it's not a problem but + +0:14:29.200,0:14:31.660 +ideally you would like to prevent +that + +0:14:31.660,0:14:35.350 +one of the things that I have to deal with +is that my users are + +0:14:35.350,0:14:37.830 +almost universally + +0:14:37.830,0:14:40.410 +not trained as computer scientists are programmes + +0:14:40.410,0:14:42.550 +you know there's there's really no domain +area + +0:14:42.550,0:14:44.780 +they're really good in that area + +0:14:44.780,0:14:48.389 +their concepts of the way hardware works in the +way software works + +0:14:48.389,0:14:55.389 +dont match reality in many cases + +0:15:01.269,0:15:02.830 +its pretty rare in practice + +0:15:02.830,0:15:06.700 +well I've heard one one lab that does it significantly + +0:15:06.700,0:15:09.839 +but it's like they do it on sort of a yearly +allocation basis + +0:15:09.839,0:15:12.790 +and throw the hardware away after two or three years + +0:15:12.790,0:15:15.999 +and you do typically have some sort of the deployment + +0:15:15.999,0:15:18.340 +system in place + +0:15:18.340,0:15:20.680 +or in most types of cases actually + +0:15:20.680,0:15:22.359 +usually your application comes with + +0:15:22.359,0:15:26.500 +and here's what we're going to spend on this many people + +0:15:26.500,0:15:27.730 +on this project so this is + +0:15:27.730,0:15:34.730 +big resource allocation + +0:15:36.000,0:15:39.780 +and %uh yet and I guess one other issue with this is there's no real easy + +0:15:39.780,0:15:43.320 +way to capture on underutilized resources +for example + +0:15:43.320,0:15:44.389 +if you have + +0:15:44.389,0:15:49.190 +an application which you know say single-threaded +and uses a ton of memory + +0:15:49.190,0:15:51.210 +on and is running on a machine + +0:15:51.210,0:15:55.040 +the machines we're buying these days are eight core so + +0:15:55.040,0:16:00.040 +thats wasting a lot of CPU cycles you're just +generating a lot of heat doing nothing + +0:16:00.040,0:16:03.890 +so ideally you would like a scheduler that +said okay so you're using + +0:16:03.890,0:16:08.040 +using eight or seven of the eight Gigabytes of +RAM but we've got these jobs + +0:16:08.040,0:16:10.080 +sitting here that + +0:16:10.080,0:16:11.560 +need next to know need + +0:16:11.560,0:16:15.910 +a hundred megabytes so we swap seven of +those in along with the big job + +0:16:15.910,0:16:18.580 +and backfill and in this + +0:16:18.580,0:16:19.600 +mechanism there's no + +0:16:19.600,0:16:21.810 +there's no good way to do that + +0:16:21.810,0:16:26.820 +obviously if the users have that application +next they can do it themselves + +0:16:26.820,0:16:30.510 +it's not something where we can be easily +bring in + +0:16:30.510,0:16:35.090 +bring in more jobs and have a mix to +take advantage of the different + +0:16:35.090,0:16:37.300 +resources + +0:16:37.300,0:16:39.940 +a related approach is to + +0:16:39.940,0:16:43.950 +to install virtualization software on the +equipment and this is this is + +0:16:43.950,0:16:44.980 +a + +0:16:44.980,0:16:46.379 +this is the essence of + +0:16:46.379,0:16:49.800 +what Cloud computing is at the moment + +0:16:49.800,0:16:53.520 +it's Amazon providing Zen + +0:16:53.520,0:16:55.129 +Zen hosting for + +0:16:55.129,0:16:56.769 +relatively arbitrary yet + +0:16:56.769,0:16:59.710 +OS images + +0:16:59.710,0:17:02.720 +it does have advantage that allows rapid deployment + +0:17:02.720,0:17:06.510 +in theory if your application is scaleable provides for + +0:17:06.510,0:17:08.259 +extremely high scaleability + +0:17:08.259,0:17:10.110 +particularly if you + +0:17:10.110,0:17:14.470 +arent us and therefore can possibly somebody else's hardware + +0:17:14.470,0:17:16.520 +in in our application's face thats + +0:17:16.520,0:17:18.790 +not very practical so + +0:17:18.790,0:17:20.360 +we can't do that + +0:17:20.360,0:17:20.870 +and + +0:17:20.870,0:17:23.790 +it also has the advantage that you can run + +0:17:23.790,0:17:26.470 +you can have people with their own image in there + +0:17:26.470,0:17:30.000 +which is tightly resource constrained but you +can run more than one of them on it. but no for instance + +0:17:30.000,0:17:31.170 +you can give + +0:17:31.170,0:17:32.730 +one job + +0:17:32.730,0:17:35.489 +four cores and another job two cores another + +0:17:35.489,0:17:37.500 +you know and have a couple single core + +0:17:37.500,0:17:38.860 +jobs in theory + +0:17:38.860,0:17:43.340 +you can get fairly strong isolation there +obviously there are shared resources underneath + +0:17:43.340,0:17:44.710 +and you + +0:17:44.710,0:17:45.570 +probably can't + +0:17:45.570,0:17:48.370 +afford to completely isolate say network bandwidth + +0:17:48.370,0:17:49.520 +at the bottom layer + +0:17:49.520,0:17:51.580 +you can be some but + +0:17:51.580,0:17:56.170 +if you go overboard you can spend all your time on accounting + +0:17:56.170,0:17:58.830 +you also can again + +0:17:58.830,0:18:01.410 +tailor the images to the job + +0:18:01.410,0:18:05.030 +and in this environment actually you can +do that even more strongly than that + +0:18:05.030,0:18:07.030 +the sub-cluster approach + +0:18:07.030,0:18:09.860 +in that you can often do run + +0:18:09.860,0:18:16.360 +a five-year-old operating system or ten-year-old +operating system if you're using full virtualization + +0:18:16.360,0:18:19.030 +and that can allow + +0:18:19.030,0:18:23.820 +allow obsolete core with your baseline core which is +important in our space because + +0:18:23.820,0:18:27.390 +the average program runs ten years or more + +0:18:27.390,0:18:30.860 +our average project runs ten years or more + +0:18:30.860,0:18:32.530 +and as a result + +0:18:32.530,0:18:36.010 +you might have to go rerun this program that was written + +0:18:36.010,0:18:37.320 +way back on + +0:18:37.320,0:18:40.550 +some ancient version of windows or whatever + +0:18:40.550,0:18:41.890 +it also does provide + +0:18:41.890,0:18:43.840 +the ability to recover resources + +0:18:43.840,0:18:45.290 +as I was talking about before + +0:18:45.290,0:18:49.530 +%uh but you can't do easily with sub-clusters because you cant just slip + +0:18:49.530,0:18:50.360 +another image + +0:18:50.360,0:18:52.910 +on the on there and say are you can use anything and + +0:18:52.910,0:18:56.730 +you know get that image ideal priority essentially + +0:18:56.730,0:19:00.480 +down side of course is that it is in complete +isolation and that there is a shared + +0:19:00.480,0:19:02.340 +hardware + +0:19:02.340,0:19:06.490 +you're not likely to find I don't think +any the virtualization systems out there + +0:19:06.490,0:19:08.890 +right now + +0:19:08.890,0:19:09.890 +virtualize + +0:19:09.890,0:19:11.470 +your segment of + +0:19:11.470,0:19:13.540 +memory bandwidth + +0:19:13.540,0:19:15.159 +or your segment + +0:19:15.159,0:19:16.390 +of cache + +0:19:16.390,0:19:18.390 +of cache base + +0:19:18.390,0:19:24.809 +so users can in fact interfere with themselves and each other in this +environment + +0:19:24.809,0:19:25.589 +it's also + +0:19:25.589,0:19:30.479 +%uh not really efficient for small jobs from the cost of running an + entire arrest for every + +0:19:30.479,0:19:33.020 +job is fairly high + +0:19:33.020,0:19:34.020 +even with + +0:19:34.020,0:19:34.710 +relatively light + +0:19:34.710,0:19:38.250 +%uh you know it's like OS is you're still looking + +0:19:38.250,0:19:40.900 +couple hundred megabytes in practice + +0:19:40.900,0:19:46.240 +once you get everything up and running unless you get totally stripped +down + +0:19:46.240,0:19:47.230 +and %uh + +0:19:47.230,0:19:49.460 +theres significant overhead + +0:19:49.460,0:19:52.240 +theres CPU slowdown typically in the + +0:19:52.240,0:19:55.360 +you know typical estimates are in the twenty +percent range + +0:19:55.360,0:20:00.450 +numbers really range from fifty percent to +five percent depending on what exactly you're doing + +0:20:00.450,0:20:02.100 +possibly even lower + +0:20:02.100,0:20:04.830 +or higher + +0:20:04.830,0:20:05.870 +and and just + +0:20:05.870,0:20:09.920 +you know the overhead because you have the whole OS there's a lot of a lot + +0:20:09.920,0:20:11.420 +of duplicate + +0:20:11.420,0:20:13.320 +stuff + +0:20:13.320,0:20:15.010 +the various vendors + +0:20:15.010,0:20:17.090 +have their answers they claim you know we can + +0:20:17.090,0:20:21.430 +we can merge that and say oh you're running the same kernel so we'll keep your memory + +0:20:21.430,0:20:24.120 +we use the same memory but + +0:20:24.120,0:20:25.220 +at some level + +0:20:25.220,0:20:29.309 +it's all going to get duplicated + +0:20:29.309,0:20:30.590 +a related option + +0:20:30.590,0:20:34.820 +comes from sort of the the internet havesting +industry which is to use virtual private + +0:20:34.820,0:20:38.130 +which is the technology from virtual private servers + +0:20:38.130,0:20:42.110 +the example that everyone here is probably familiar with is jails where + +0:20:42.110,0:20:44.130 +you can provide + +0:20:44.130,0:20:46.720 +your own file system root + +0:20:46.720,0:20:49.060 +your network interface + +0:20:49.060,0:20:50.620 +and what not + +0:20:50.620,0:20:51.500 +and + +0:20:51.500,0:20:53.129 +the nice thing about this is + +0:20:53.129,0:20:56.210 +that unlike full virtualization + +0:20:56.210,0:20:58.680 +the overhead is very small + +0:20:58.680,0:21:01.030 +basically costs you + + +0:21:01.030,0:21:02.820 +an entry in your process table + +0:21:02.820,0:21:05.570 +or an entry in few structures + +0:21:05.570,0:21:08.760 +there's some extra tests in their kernel but otherwise + +0:21:10.220,0:21:14.900 +there's there's not a huge overhead for virtualization you don't need +an extra kernel for every + +0:21:14.900,0:21:15.460 +image + +0:21:15.460,0:21:18.390 +so you get you get the difference here +between + +0:21:18.390,0:21:21.620 +be able to run maybe + +0:21:21.620,0:21:25.250 +you might be able to squeeze two hundred VMR images onto a machine + +0:21:25.250,0:21:29.620 +VMR people say no no don't do that but we have machines that are running + +0:21:29.620,0:21:30.509 +nearly that many + +0:21:33.720,0:21:34.790 +they're what + +0:21:34.790,0:21:38.289 +on the other hand there are people out there on thousands of + +0:21:38.289,0:21:40.730 +virtual hosts + +0:21:40.730,0:21:43.170 +using this technique at a single machine so + +0:21:43.170,0:21:45.200 +big difference in resource use + +0:21:45.200,0:21:46.400 +on especially with light + +0:21:46.400,0:21:48.070 +in the lightly loaded use + +0:21:48.070,0:21:52.400 +in our environment we're looking more running a very small number of them but still + +0:21:52.400,0:21:55.880 +that overhead is significant + +0:21:55.880,0:21:59.440 +you still do have some ability to tailor the + +0:21:59.440,0:22:01.670 +images to jobs needs + +0:22:01.670,0:22:03.309 +you could have a + +0:22:03.309,0:22:05.400 +custom root that for instance you could be running + +0:22:05.400,0:22:07.380 +FreeBSD x6 in one + +0:22:07.380,0:22:08.650 +in one + +0:22:08.650,0:22:11.040 +virtual server and seven in another + +0:22:11.040,0:22:15.090 +you have to be running of course seven kernel or eight kernel to make +that work + +0:22:15.090,0:22:16.330 +but it allows you to do that + +0:22:16.330,0:22:18.500 +we also in principle can do + +0:22:18.500,0:22:23.080 +evil things like our sixty four-bit kernel and then thirty two bit +user spaces because + +0:22:23.080,0:22:26.400 +say you have applications that you can't find the source to do anymore + +0:22:26.400,0:22:31.830 +or wide worst wide worries you don't +have the source to any more + +0:22:31.830,0:22:32.990 +an answer + +0:22:32.990,0:22:34.150 +interesting things there + +0:22:34.150,0:22:36.680 +and the other nice thing is since you're + +0:22:36.680,0:22:39.629 +you're doing a very lightweight and incomplete +virtualization + +0:22:39.629,0:22:43.269 +you don't have to virtualize things you don't +care about so you dont have the overhead of + +0:22:43.269,0:22:45.520 +virtualizing everything + +0:22:45.520,0:22:48.070 +downsides of course are incomplete isolation + +0:22:48.070,0:22:50.690 +you are running processes that on the same kernel + +0:22:50.690,0:22:52.770 +and they can interfere with each other + +0:22:52.770,0:22:55.320 +and there's dubious flexibility obviously + +0:22:55.320,0:22:57.900 +I don't think anybody + +0:22:57.900,0:23:01.850 +should have the ability to run Windows in a jail + +0:23:01.850,0:23:02.860 +theres some + +0:23:02.860,0:23:04.960 +Net BSD peak of support but + +0:23:04.960,0:23:10.510 +and I dont think it's really gotten to that point + +0:23:10.510,0:23:12.420 +one one final area + +0:23:12.420,0:23:14.350 +that sort of diverges from this + +0:23:14.350,0:23:16.159 +is the classic + +0:23:16.159,0:23:18.400 +Unix solution to the problem + +0:23:18.400,0:23:20.580 +on this on single + +0:23:20.580,0:23:22.070 +in a single machines + +0:23:22.070,0:23:22.800 +which is + +0:23:22.800,0:23:28.950 +to use existing resource limits and resource partitioning techniques + +0:23:28.950,0:23:33.430 +you know for example all Unix like our Unix systems have to process +resource limits + +0:23:33.430,0:23:36.240 +a resource and typically + +0:23:36.240,0:23:36.999 +schedule a + + +0:23:38.340,0:23:41.510 +cluster schedulers support the common ones + +0:23:41.510,0:23:43.150 +so you can set a + +0:23:43.150,0:23:47.230 +memory limit on your process or a CPU time limit on your process + +0:23:47.230,0:23:49.830 +and the schedulers typically provide + +0:23:49.830,0:23:51.350 +at least + +0:23:51.350,0:23:54.740 +lot of support for + +0:23:54.740,0:23:56.850 +the limits on + +0:23:56.850,0:24:01.900 +a given set of process. thats part of the job + +0:24:01.900,0:24:02.850 +also the most + +0:24:02.850,0:24:05.640 +you know there are a number of forms of research +partitioning that + +0:24:05.640,0:24:07.170 +are available as + +0:24:07.170,0:24:08.100 +the in that + +0:24:08.100,0:24:09.700 +as the standard feature + +0:24:09.700,0:24:12.000 +on so memory discs are one of them so + +0:24:12.000,0:24:16.800 +if you want to create a file system space it's +limited in size. Create a memory disc + +0:24:16.800,0:24:17.969 +and back it + +0:24:17.969,0:24:21.130 +and back it with a --- file + +0:24:21.130,0:24:22.520 +Quotas another mechanism + +0:24:22.520,0:24:24.570 +of partitioning that + +0:24:24.570,0:24:26.330 +disc use + +0:24:26.330,0:24:30.330 +and then there are techniques like CPU affinities that you can walk +processes to it + +0:24:30.330,0:24:32.010 +a single process + +0:24:32.010,0:24:34.540 +processor a set of processors + +0:24:34.540,0:24:39.310 +and so they can't interfere with each other +with processes running on other processes + + +0:24:39.310,0:24:44.280 +the nice thing about this first is that you're using existing +facilities so you dont have to rewrite + +0:24:44.280,0:24:46.170 +also new features + +0:24:46.170,0:24:49.590 +for each application + +0:24:49.590,0:24:52.790 +and they tend to integrate well with existing schedulers +in many cases + +0:24:52.790,0:24:55.940 +parts of them are already implemented + +0:24:55.940,0:24:59.650 +and in fact the experiments that we'll talk about this later are using +this type of + +0:24:59.650,0:25:02.160 +technique + +0:25:02.160,0:25:02.830 +cons are of course + +0:25:02.830,0:25:04.850 +incomplete isolation again + +0:25:04.850,0:25:08.270 +and theres typically no unified framework + +0:25:08.270,0:25:12.310 +for the concept of a job when it comes to the center process + +0:25:12.310,0:25:16.710 +yeah there there are a number of data structures within the kernel for +instance the session + +0:25:16.710,0:25:18.120 +which + +0:25:18.120,0:25:19.499 +certain aggregate processes + +0:25:19.499,0:25:20.990 +but there isnt one + +0:25:20.990,0:25:22.230 +in + +0:25:22.230,0:25:24.800 +in BVSD at this point + +0:25:24.800,0:25:29.020 +allows you to place resource limits on this in a way that you can process + +0:25:29.020,0:25:32.520 +--- did have support like that + +0:25:32.520,0:25:34.160 +where they have a job ID + +0:25:34.160,0:25:36.210 +and there could be a job limit + +0:25:36.210,0:25:38.280 + and slurs projects + +0:25:38.280,0:25:41.320 +pursue similar not not quite the same + +0:25:41.320,0:25:43.149 +processes or part of a project but + +0:25:43.149,0:25:46.770 +it's not quite the same inherited relationship + +0:25:47.720,0:25:49.500 +and typically + +0:25:49.500,0:25:50.900 +there arent + +0:25:50.900,0:25:55.390 +limits on things like badwidth. there was + +0:25:55.390,0:25:56.430 +a sort of a + +0:25:56.430,0:25:58.350 +bandwidth limiting + +0:25:58.350,0:26:00.630 +nice type interface + +0:26:00.630,0:26:01.950 +on that I saw + +0:26:01.950,0:26:03.720 +first thing as a research project + +0:26:03.720,0:26:07.150 +many years ago I think it was that stage + +0:26:07.150,0:26:09.880 +where you could say this process can have + +0:26:09.880,0:26:11.580 +you know five megabits + +0:26:11.580,0:26:12.530 +or or whatever + +0:26:12.530,0:26:14.380 +but I haven't really seen anything take off + +0:26:14.380,0:26:16.940 +that would be a pretty neat thing to have + +0:26:16.940,0:26:19.309 +but one other exception there + +0:26:19.309,0:26:22.230 +is on Irex again + +0:26:22.230,0:26:28.210 +the XFS file system supported guaranteed data rates on file handles +you could say + +0:26:28.210,0:26:30.140 +you know if you would say I need + +0:26:30.140,0:26:32.940 +ten megabits read or ten megabits write + +0:26:32.940,0:26:34.029 +whatever you say + +0:26:34.029,0:26:35.529 +okay and go + +0:26:35.529,0:26:39.279 +and and then you could read and write and +it would do evil things file system + +0:26:39.279,0:26:40.600 +in some cases + +0:26:40.600,0:26:43.940 +all to making sure that you could get that terrific data rate + +0:26:43.940,0:26:44.900 +by + +0:26:44.900,0:26:49.710 +by keeping the file + + +0:26:49.710,0:26:53.620 +so now Im going to talk about what we've done + +0:26:53.620,0:26:59.510 + what we needed was the solution to handle +a wide range of job types + +0:26:59.510,0:27:01.570 +so all the options we looked at for instance + +0:27:01.570,0:27:04.990 +single application clusters of +project clusters + +0:27:04.990,0:27:11.990 +I think that the isolation they +provide is essentially unparalleled + +0:27:12.590,0:27:16.630 +and in our environment we probably have to +virtualize in order to be + +0:27:16.630,0:27:18.179 +efficient in terms of + +0:27:18.179,0:27:22.060 +being able to handle our job nix and what not to handle +the fact that our users + +0:27:22.060,0:27:23.740 +tend to have + +0:27:23.740,0:27:27.730 +spikes in their in their use + +0:27:27.730,0:27:32.799 +on a on a large scale so for instance we get those that show and say +that they need to run for a month + +0:27:32.799,0:27:33.780 +on and then + +0:27:33.780,0:27:38.460 +some indeterminate number of months later +they'll do it again + +0:27:38.460,0:27:40.840 +for that sort of quick + +0:27:40.840,0:27:41.480 +demands + +0:27:42.240,0:27:44.850 +we really need the virtuals something +virtualized + +0:27:44.850,0:27:47.120 +and then we got to pay the price of %uh + +0:27:47.120,0:27:48.380 +of the overhead + +0:27:48.380,0:27:51.590 +and again it doesn't handle small jobs and that is a + +0:27:51.590,0:27:54.050 +large portion of our job nix and + +0:27:54.050,0:27:55.180 +of that + +0:27:55.180,0:27:58.070 +quarter million or something jobs line + +0:27:58.070,0:27:59.700 +on our cluster + +0:27:59.700,0:28:02.490 +%uh I would guess that + +0:28:02.490,0:28:04.730 +more than half of those were submitted + +0:28:04.730,0:28:05.890 +in + +0:28:05.890,0:28:09.660 +batches of more than ten thousand + +0:28:09.660,0:28:11.400 +so they'll just pop up + +0:28:11.400,0:28:14.030 +the other method to have looked at + +0:28:14.030,0:28:14.800 +are up + +0:28:14.800,0:28:16.750 +were using resource limits + +0:28:16.750,0:28:19.060 +the nice thing of course is they're achievable +with + +0:28:19.060,0:28:21.429 +they acheive useful isolation + +0:28:21.429,0:28:26.289 +and the inflexible with under existing functionality or small +extension so that's what we think + +0:28:26.289,0:28:27.230 +concentrating on + +0:28:27.230,0:28:29.740 +also been doing some thinking about + +0:28:29.740,0:28:31.809 +could we use the techniques there + +0:28:31.809,0:28:33.940 +and combine them with jails + +0:28:33.940,0:28:36.170 +or related features + +0:28:36.170,0:28:40.019 +it may be bulking up jails to be more like ------ + +0:28:40.019,0:28:44.150 +or containers I think they're calling this +week + +0:28:44.150,0:28:44.840 +and + +0:28:44.840,0:28:46.770 +so we're looking that + +0:28:46.770,0:28:50.840 +to be able to provide + + +0:28:50.840,0:28:54.250 +to to to be able to provide pretty user operating environments + +0:28:54.250,0:28:59.200 +potentially isolating users from operating suffrance as we upgrade the kernel + +0:28:59.200,0:29:03.469 +and users can continue using it all the +images they don't have time to rebuild their + +0:29:03.469,0:29:04.330 +application in + +0:29:04.330,0:29:09.970 +and handle the updates in libraries and what not + +0:29:09.970,0:29:13.840 +they also have potential to provide strong isolation for security +purposes + +0:29:13.840,0:29:18.740 +%uh which could be useful in the future + +0:29:18.740,0:29:20.159 +we do think that that + +0:29:20.159,0:29:24.040 +of of its to of these of these mechanisms the nice thing is that +resource limit + +0:29:24.040,0:29:26.150 +the resource limits and partitioning scheme + +0:29:26.150,0:29:29.860 +as well as virtual private service are very +similar imitation requirements + +0:29:29.860,0:29:33.090 + set up a fair bit more expensive + +0:29:33.090,0:29:34.620 +in the VPS case + + +0:29:34.620,0:29:38.780 +while nonetheless they're fairly similar + +0:29:38.780,0:29:42.610 +what we've been doing is we've taken Sun Grid Engine we've taken Sun +Grid Engine + +0:29:42.610,0:29:46.880 +and we were originally intended to actually +extend Sun Grid Engine and modify its demands + +0:29:46.880,0:29:48.480 +to do work + +0:29:48.480,0:29:51.150 +on what we ended up doing instead is realize +that well + +0:29:51.150,0:29:54.910 +we can actually starts flying alternate program +to run instead of the shepherd + +0:29:54.910,0:29:57.990 +The shepherd is the process + +0:29:57.990,0:30:00.580 +that starts all + +0:30:00.580,0:30:02.250 +starts the the script that + +0:30:02.250,0:30:03.380 +can reach job + +0:30:03.380,0:30:04.920 +on a given node + +0:30:04.920,0:30:08.559 + it collects usage and forwards signals to the +children + +0:30:08.559,0:30:12.620 +and also is responsible for starting remote + components + +0:30:12.620,0:30:14.560 +so shepherd is started and then + +0:30:14.560,0:30:17.640 + traditionally in seperate engine it starts out + +0:30:17.640,0:30:19.910 +it's own --the event + +0:30:19.910,0:30:20.800 +and + +0:30:20.800,0:30:22.010 +jobs connect over + +0:30:22.010,0:30:23.670 +these days that for their own + +0:30:23.670,0:30:25.870 +you're a mechanism which is + +0:30:25.870,0:30:26.950 +secure + +0:30:26.950,0:30:28.000 +not using the + +0:30:28.840,0:30:30.530 +arch old code + +0:30:30.530,0:30:31.920 +on + +0:30:35.370,0:30:37.970 +so what we've done is we've implemented a rapid script + +0:30:37.970,0:30:40.139 +which allows a pre-command hook + +0:30:40.139,0:30:42.559 +to run before the shepherd starts + +0:30:42.559,0:30:47.170 +the command rapper send before we send shepherd before we can run like the N program + +0:30:47.170,0:30:49.150 +or the week and why not + +0:30:49.150,0:30:50.430 +troops or whatever + +0:30:50.430,0:30:54.040 + to set up the environment that it runs in or CPU + +0:30:54.040,0:30:56.600 +that will show later + +0:30:56.600,0:30:58.750 +on and I first met her for cleanup + +0:30:58.750,0:31:03.940 +simply move because I felt like it + +0:31:03.940,0:31:07.830 +the first thing we implemented is memory backs temporary directors. the motivation for + +0:31:07.830,0:31:08.700 +this + +0:31:08.700,0:31:09.640 +is that + +0:31:09.640,0:31:12.180 +we've had problems for users will you know + +0:31:12.180,0:31:15.510 +run slash ten on the nodes + +0:31:15.510,0:31:19.059 +where we have the figures is that they do have discs + +0:31:19.059,0:31:22.960 +and most of the disc is available as slash ten + +0:31:22.960,0:31:25.049 +we had some cases + +0:31:25.049,0:31:27.840 +particularly early on where users would fill the discs and not complete it + +0:31:27.840,0:31:32.300 +their job would crash and they wold forget to add clean up code or whatever + +0:31:32.300,0:31:35.100 +other jobs would fail strangely + +0:31:35.100,0:31:39.029 +you might expect that you just get a you would get a nice error message + +0:31:39.029,0:31:42.040 +being programmers + +0:31:42.040,0:31:42.909 +people would not + +0:31:42.909,0:31:44.630 +handle very correctly + +0:31:44.630,0:31:47.380 +now of course you have issues like for instance + +0:31:47.380,0:31:49.600 +the PDL library + +0:31:49.600,0:31:52.600 +unexpectedly fails and reports a completely strange error + +0:31:52.600,0:31:54.759 +if it can't create a file to have + +0:31:54.759,0:32:01.669 +because it needs to create an extra file in itself + +0:32:01.669,0:32:03.360 +so what we've done here + +0:32:03.360,0:32:08.059 +is it turns out that Sun Grid Engine actually creates a temporary +directory often the + +0:32:08.059,0:32:11.730 +typically but you can change +that + +0:32:11.730,0:32:14.490 +I think it's that's a + +0:32:14.490,0:32:15.370 +location + +0:32:15.370,0:32:17.499 +we educated most of all users now + +0:32:17.499,0:32:21.360 +to use that location correctly and values +that very cool + +0:32:21.360,0:32:23.279 +they treat their files understand her + +0:32:23.279,0:32:24.950 +and then when the job exits + +0:32:24.950,0:32:26.569 +the Grid Engine deletes the temp dir + +0:32:26.569,0:32:28.510 +and that all gets cleaned up + +0:32:28.510,0:32:32.720 +the problem of course being that of multiples +also warning on the same note same time + +0:32:32.720,0:32:35.290 +one of them could still fill temp + +0:32:35.290,0:32:38.759 +so the solution was pretty simple +we created a + +0:32:38.759,0:32:41.420 +rapper script at the beginning of the job + +0:32:41.420,0:32:42.760 +creates a + +0:32:42.760,0:32:43.940 +a + +0:32:43.940,0:32:47.260 +memory file to swap back to MB file system + +0:32:47.260,0:32:50.790 +of a user requestable size of the default + +0:32:50.790,0:32:53.310 +and + +0:32:53.310,0:32:56.520 +this has a number of advantages the biggest one of course is that + +0:32:56.520,0:32:58.320 +it's their fixed size so we get + +0:32:58.320,0:32:59.449 +you know + +0:32:59.449,0:33:01.000 + the user gets + +0:33:01.000,0:33:03.420 + what they asked for + +0:33:03.420,0:33:05.930 +and once they run of space, they run out of space well + +0:33:05.930,0:33:09.300 +and too bad they ran out of space + +0:33:09.300,0:33:12.760 +they should have asked for more + +0:33:12.760,0:33:16.350 +the other + +0:33:16.350,0:33:18.770 +the other advantage is the side-effect that + +0:33:18.770,0:33:21.619 +now that we're running swap back memory files systems for temp + +0:33:21.619,0:33:24.560 +the users who only use a fairly small amount of temp + +0:33:24.560,0:33:28.190 + should see vastly improved performance +because they're running in memory + +0:33:28.190,0:33:32.980 +rather than writing to disc + +0:33:32.980,0:33:34.690 +quick example + +0:33:34.690,0:33:38.270 +we've a little job script herel + +0:33:38.270,0:33:39.830 +prints chapter and + +0:33:39.830,0:33:41.950 +prints the + +0:33:41.950,0:33:43.080 +amount of space + +0:33:43.080,0:33:46.210 +we consider job request saying that we want + +0:33:46.210,0:33:51.539 +this is what we want hundred megabytes of +temp space + +0:33:51.539,0:33:53.580 +the same that's why if this + +0:33:53.580,0:33:55.230 +so the program doesn't + +0:33:55.230,0:33:57.620 +so the program ends at the end of it + +0:33:57.620,0:33:58.709 +%uh for doing it + +0:33:58.709,0:34:00.510 +heres a live demo + +0:34:00.510,0:34:01.840 +all and then + +0:34:01.840,0:34:03.389 +you look at the output + +0:34:03.389,0:34:04.280 +you can see it + +0:34:04.280,0:34:07.549 +does in fact it creates a memory file system + +0:34:07.549,0:34:10.449 +I attempted to do as great code + +0:34:10.449,0:34:13.409 +having a variable space that + +0:34:13.409,0:34:15.839 +having a variable space that is roughly what user asked for + +0:34:15.839,0:34:17.089 +the version I had + +0:34:17.089,0:34:20.739 +when I was attempting this was not entirely +accurate + +0:34:20.739,0:34:24.710 +trying to guess what all the +USFS overhead would be + +0:34:24.710,0:34:25.889 +as the result was + +0:34:25.889,0:34:28.399 +%uh not quite consistent + +0:34:30.790,0:34:33.899 +I couldn't figure out easy function so + +0:34:33.899,0:34:39.589 +it does a better job than it did to start with + +0:34:39.589,0:34:40.600 +sometimes however + +0:34:40.600,0:34:42.329 +today that that's a good case + +0:34:42.329,0:34:43.550 +we're coming to it + +0:34:43.550,0:34:45.359 +deployed pretty soon + +0:34:45.359,0:34:47.159 +it's pretty easily + +0:34:47.159,0:34:48.570 +well sometimes it's not enough + +0:34:48.570,0:34:51.390 +the biggest issue is that they were badly designed programs all + +0:34:51.390,0:34:52.720 +all over the world + +0:34:52.720,0:34:54.919 +don't you step to like they're supposed to + +0:34:54.919,0:34:59.319 +in fact + +0:35:10.099,0:35:12.759 +so there are all these applications + +0:35:12.759,0:35:17.979 +there are a lot about patience still that need +ten because they'll still need start up + +0:35:17.979,0:35:19.230 +that sort of thing + +0:35:19.230,0:35:20.809 +so + +0:35:20.809,0:35:22.599 +all + +0:35:22.599,0:35:25.829 +so we have problems with these + +0:35:25.829,0:35:26.290 +realistically + +0:35:26.290,0:35:27.799 +we cant change all of them + +0:35:27.799,0:35:30.019 +it's just not going to happen + +0:35:30.019,0:35:31.950 +so we still have a lot of people + +0:35:31.950,0:35:34.509 +running out resources + +0:35:34.509,0:35:35.819 +%uh so we probably + +0:35:35.819,0:35:37.489 +feel that + + +0:35:37.489,0:35:41.240 +was general solution is right a per job slash temp + +0:35:41.240,0:35:44.880 +the first was that her from the files system +at its best + +0:35:44.880,0:35:47.119 +we think there is some ways to me that + +0:35:47.119,0:35:52.539 +and so he said okay let's give it a shot + +0:35:52.539,0:35:56.969 +just to inter these concepts for people who are unfamiliar with him + +0:35:56.969,0:36:00.280 +offering someone services recent ones that +contained rules + +0:36:00.280,0:36:02.389 +Richard Senator long time + +0:36:02.389,0:36:05.549 +angeles half past be different for different +processes + +0:36:05.549,0:36:06.969 +for example + +0:36:06.969,0:36:08.689 +you create the files + +0:36:08.689,0:36:10.069 +on create + +0:36:10.069,0:36:12.459 +they ask someone whose contents are + +0:36:12.459,0:36:18.329 +this veritable which has a the fall not shells +fell the fall diet + +0:36:18.329,0:36:18.990 +and you + +0:36:18.990,0:36:24.949 +he didn't get different results with different +variables that + +0:36:24.949,0:36:27.170 +what about her the implementation we've got + +0:36:27.170,0:36:32.389 +it's drive from grateful and mission was to +the data structures are gonna call + +0:36:32.389,0:36:33.869 +authorities a number of changes + +0:36:33.869,0:36:39.649 +the biggest one is that the two the concept +of us troops and returned them entirely around + +0:36:39.649,0:36:40.409 +he added that + +0:36:40.409,0:36:45.329 +in trade as one of his assistants the which +is over overridden by the users scope and by a + +0:36:45.329,0:36:47.259 +progressive scope + +0:36:49.819,0:36:53.449 +problem with that is if you + +0:36:53.449,0:36:56.099 +only think about say the systems scope + +0:36:56.099,0:36:57.079 +and + +0:36:57.079,0:36:59.459 +you decide you want to do something clever like have + +0:36:59.459,0:37:02.219 +root file system which +%uh + +0:37:02.219,0:37:06.109 +were slashed with points the different things +for different %uh + +0:37:06.109,0:37:08.249 +different architectures + +0:37:08.249,0:37:11.849 +were seriously until the users come along +and + +0:37:11.849,0:37:14.189 +they're upset there are variable + +0:37:14.189,0:37:15.629 +up for you + +0:37:15.629,0:37:18.900 +if you have CSX like the program and you don't +defensively + +0:37:18.900,0:37:22.319 +and you don't implement correctly + +0:37:22.319,0:37:24.900 +the obvious that they sat obviously you would + +0:37:24.900,0:37:28.599 +Richard riordan often that I believe they +did + +0:37:28.599,0:37:31.700 +there's a whole class of problems where + +0:37:31.700,0:37:33.449 +it's easy to screw up + +0:37:33.449,0:37:36.219 +add and do something wrong there + +0:37:36.219,0:37:37.270 +so by + +0:37:37.270,0:37:38.509 +reversing the order + +0:37:38.509,0:37:41.849 +we can't we can reduce the risks + +0:37:41.849,0:37:43.329 +at the moment we don't + +0:37:43.329,0:37:44.309 +haven't you sir + +0:37:44.309,0:37:47.530 +I just don't like the idea of the users scope +to be honest + +0:37:47.530,0:37:50.900 +from being they see you dont have to have +poor user + +0:37:50.900,0:37:55.509 +that just sits around forever +you can never guard elected accepted the strain + +0:37:55.509,0:37:57.059 +of late + +0:37:57.059,0:37:59.489 +just doesn't seem like a great idea to me + +0:37:59.489,0:38:00.700 +on it + +0:38:00.700,0:38:04.609 +just hasn't been implemented + +0:38:04.609,0:38:09.809 +because it wasn't entirely clear as to what the semantics should be + +0:38:09.809,0:38:11.010 +i also + +0:38:11.010,0:38:14.719 +I also added default variable support variable +also shell style + +0:38:14.719,0:38:16.999 +for + +0:38:16.999,0:38:19.169 +to some extent undoes the scope + +0:38:19.169,0:38:20.870 +the scope change + +0:38:20.870,0:38:21.779 +in that + +0:38:21.779,0:38:24.749 +the default variable becomes a system scope + +0:38:24.749,0:38:26.540 +which is overridden by everything + +0:38:26.540,0:38:30.890 +but there are cases where we need to do that +in particular who wants implement their + +0:38:30.890,0:38:33.380 +slashed temp which varies + +0:38:33.380,0:38:36.240 +we have to do something like this because the temp needs to work + +0:38:36.240,0:38:37.209 +if + +0:38:37.209,0:38:42.059 +if we don't have the job values set + +0:38:42.059,0:38:45.829 +I also decided to use + +0:38:45.829,0:38:49.839 +percent instead of dollar signs to avoid +confusion over shell variables because these + +0:38:49.839,0:38:50.379 +are + +0:38:50.379,0:38:52.620 +a separate namespace in the kernel + +0:38:52.620,0:38:56.669 +can't do it in a nice to do all the evaluation in the +user space + +0:38:56.669,0:38:59.269 +it's classic vulnerability + +0:38:59.269,0:39:02.739 +the that in the database for instance + +0:39:02.739,0:39:08.109 +and when I was in the past and avoid confusion +with but yet that's and worthy and for the + +0:39:08.109,0:39:09.819 +now BST implementation + +0:39:09.819,0:39:11.019 +first is not allowed + +0:39:11.019,0:39:14.879 +he's a reduced rate of one set of core values + +0:39:14.879,0:39:17.019 +that will be enough ballots for + +0:39:17.019,0:39:20.359 +on I don't have any automated variables such +as the %uh + +0:39:20.359,0:39:25.789 +the process is not you which is universally +sat and that he is the implementation war + +0:39:25.789,0:39:26.750 +on + +0:39:26.750,0:39:28.039 +aids + +0:39:28.039,0:39:32.579 +hey I do i'd be very foolish they also have + +0:39:32.579,0:39:34.909 +and currently and it allows Senator + +0:39:34.909,0:39:40.880 +setting about using other processes yourself +in your own insurance + +0:39:40.880,0:39:42.699 +that may change but it's a + +0:39:42.699,0:39:47.339 +one of my goals here is because they were +subtle ways to make no mistakes and + +0:39:47.339,0:39:48.930 +cause securities undergo days + +0:39:48.930,0:39:52.479 +I I've attempted to storm the features that +down to the point where you + +0:39:52.479,0:39:54.909 +at some reasonable chance of not + +0:39:54.909,0:39:56.339 +doing that + +0:39:56.339,0:40:03.339 +if you start building systems on the for + +0:40:04.419,0:40:06.909 +the final area that we've worked on + +0:40:06.909,0:40:09.499 +he is moving away from the final system states + +0:40:09.499,0:40:12.559 +and the NCR see these sets of them + +0:40:12.559,0:40:16.379 +chuck roberts indictment alleges a out for +them out + +0:40:16.379,0:40:20.699 +people may I see he said functionality which +allows you to + +0:40:20.699,0:40:23.489 +but it also seems the issues that have been +set + +0:40:23.489,0:40:24.879 +the affinity of that + +0:40:24.879,0:40:26.269 +see he said + +0:40:26.269,0:40:29.189 +political observers also stopped in on this + +0:40:29.189,0:40:33.059 +and on a CD set was it was stuffed in the +one that was created by this + +0:40:33.059,0:40:37.269 +in an apparent + +0:40:37.269,0:40:38.619 +so what I hear + +0:40:38.619,0:40:40.740 +it's here unless you can figure issue + +0:40:40.740,0:40:42.769 +every day has won so what + +0:40:42.769,0:40:44.429 +first you here + +0:40:44.429,0:40:48.639 +there there are a number of other ways you +can configure basically us lost is something + +0:40:48.639,0:40:50.019 +in jobs and money + +0:40:50.019,0:40:56.719 +%uh federal jobs crosses what's going to happen +would be more than one swatch + +0:40:56.719,0:41:01.359 +pop quizzes making the applications where +persons who tends to spend a fair bit of time + +0:41:01.359,0:41:02.380 +waiting for Iran + +0:41:02.380,0:41:06.209 +you are looking at more than one slot per CPU so two slots per + +0:41:06.209,0:41:08.089 +CPU is not uncommon + +0:41:08.089,0:41:10.869 +but probably the most common configuration +and the one that + +0:41:10.869,0:41:13.719 +you get out of the box is you just install a Grid Engine + +0:41:13.719,0:41:16.739 + wants free CPU + +0:41:16.739,0:41:19.830 +and that's how that's how we run because we +want users to have + +0:41:19.830,0:41:23.699 +that whole CPU for whatever they want to do with +it + +0:41:23.699,0:41:26.130 +So drums are allocated one or more slots + +0:41:26.130,0:41:27.599 +if they're + +0:41:27.599,0:41:33.189 +depending on whether they're sequential or parallel jobs +and how many they ask for + +0:41:33.189,0:41:37.239 +but there is but this is just a convection +there's no actual connection between slots + +0:41:37.239,0:41:39.119 +and CPUs + +0:41:39.119,0:41:40.829 +it's quite possible to + +0:41:40.829,0:41:42.819 +submit a non-parallel job + +0:41:42.819,0:41:45.019 +that goes often spawns a zillion threads + +0:41:45.019,0:41:48.369 +and sucks up the whole system + +0:41:48.369,0:41:50.800 +in some early versions of grid engine + +0:41:50.800,0:41:53.569 +there actually was up + +0:41:53.569,0:41:55.729 +support for tying slots + +0:41:55.729,0:41:58.669 +for CPU to set it up that +way + +0:41:58.669,0:42:02.979 +there is a sensible implementation for Iraq's +and then things got weirder and weirder is + +0:42:02.979,0:42:06.010 +people try to implement it on other platforms +which had + +0:42:06.010,0:42:07.030 +vastly different + +0:42:07.030,0:42:09.839 +CPU binding semantics + +0:42:09.839,0:42:12.359 +and at this point in time we broke it + +0:42:12.359,0:42:14.959 +on every platform as far as I can tell + +0:42:14.959,0:42:18.759 +also the we decided okay we've got this rapper +let's see what we can do + +0:42:18.759,0:42:21.009 +on in terms of making things work +0:42:21.009,0:42:21.659 +certainly + +0:42:21.659,0:42:27.119 +we now have have the rapper store allocations in the final system + +0:42:27.119,0:42:31.239 +%uh three and nineteen ninety percent that +allocation I've read them + +0:42:31.239,0:42:33.369 +well we try to do years + +0:42:33.369,0:42:34.690 +find the best fit + +0:42:34.690,0:42:35.779 +fitting set of + +0:42:35.779,0:42:39.539 +adjacent cores + +0:42:39.539,0:42:42.329 +and then if that doesn't work we take orders +to repeat + +0:42:42.329,0:42:43.519 +%um + +0:42:43.519,0:42:45.180 +and until we fix + +0:42:45.180,0:42:47.300 +or until we've got enough slots + +0:42:47.300,0:42:50.800 +the goal is to minimize the fragments we haven't +done any analysis + +0:42:50.800,0:42:52.269 +to determine whether that's actually + +0:42:52.269,0:42:55.179 +an appropriate algorithm + +0:42:55.179,0:42:56.289 +off hand it seems + +0:42:56.289,0:43:00.519 +you'd find another part of a privilege + +0:43:00.519,0:43:02.810 +should forties lay down their arms the US +is + +0:43:02.810,0:43:09.649 +on turns out the FreeBSD, CPU API +and the last one + +0:43:09.649,0:43:12.519 +differ only in very small details + +0:43:12.519,0:43:13.599 +on that + +0:43:13.599,0:43:15.479 +essentially exactly + +0:43:15.479,0:43:17.569 +identical which is + +0:43:17.569,0:43:20.489 +correct in terms pretty semantically + +0:43:20.489,0:43:24.869 +so I think it is of interest to demonstrate +the effectiveness of cebu said they also happen + +0:43:24.869,0:43:27.019 +to demonstrate the %uh + +0:43:27.019,0:43:28.089 +the %uh + +0:43:28.089,0:43:29.359 +rather they probably have + +0:43:29.359,0:43:33.319 +the relevance + +0:43:33.319,0:43:35.229 +it's all of the young box + +0:43:35.229,0:43:36.629 +%um + +0:43:36.629,0:43:38.289 +the and %uh + +0:43:38.289,0:43:40.749 +%uh someone clearly is that + +0:43:40.749,0:43:43.239 +John dalton and backward it's not + +0:43:43.239,0:43:46.640 +our CD set it up for me + +0:43:46.640,0:43:49.039 +shortly before they released + +0:43:49.039,0:43:53.450 +well it's usually is supposed to be shortly +before + +0:43:53.450,0:43:55.579 +and that in essence six two + +0:43:55.579,0:43:59.739 +I will use the simple intervention or so and +greens + +0:43:59.739,0:44:02.519 +program were tested + +0:44:02.519,0:44:03.349 +for instance it + +0:44:03.349,0:44:05.360 +this any play for place + +0:44:05.360,0:44:08.069 +the queen so they can capture each other + +0:44:08.069,0:44:09.289 +on the board + +0:44:09.289,0:44:11.039 +%um + +0:44:11.039,0:44:13.680 +so it's a it's a simple symbol of benchmark + +0:44:13.680,0:44:18.800 +%uh that we ran a a small version of the problem +is our as a measure to man the man to generate + +0:44:18.800,0:44:19.599 +one of the + +0:44:19.599,0:44:24.439 +greta we're close and that we have much longer + +0:44:24.439,0:44:28.149 +some results + +0:44:28.149,0:44:30.129 +so for baseline do it for us + +0:44:30.129,0:44:33.170 +I think the most interesting thing is to do +a slot blot + +0:44:33.170,0:44:34.279 +you see this + +0:44:34.279,0:44:36.410 +some very it's not really very high + +0:44:36.410,0:44:38.979 +not surprising that doesn't really do anything + +0:44:38.979,0:44:40.979 +on accept socks see here + +0:44:40.979,0:44:41.729 +so on + +0:44:41.729,0:44:45.229 +going on what's going on + +0:44:45.229,0:44:50.029 +they don't think in this case for about seven +to one processes and a single + +0:44:50.029,0:44:52.789 +a single assassin process morning + +0:44:52.789,0:44:55.160 +this is the slogans wait wait + +0:44:55.160,0:44:55.890 +and %uh + +0:44:55.890,0:44:58.389 +the standard deviation goes up a bit + +0:44:58.389,0:45:00.829 +to live with a deviation from baseline + +0:45:00.829,0:45:03.659 + the obvious explanation is this + +0:45:03.659,0:45:07.339 +you know were just context switch is a bit +more + +0:45:07.339,0:45:08.840 +and %uh and %uh + +0:45:08.840,0:45:10.349 +because we don't have + +0:45:10.349,0:45:12.410 +CPUs that are doing nothing at all + +0:45:12.410,0:45:15.559 +on this is there some extra load from the system +as well + +0:45:15.559,0:45:20.049 +since the kernel have to run and not contests +have to run + +0:45:20.049,0:45:23.150 +you know if this is a story about maybe a +deposition story + +0:45:23.150,0:45:26.579 +we have people across this is what would some +couples the year + +0:45:26.579,0:45:28.879 +you know we try to run a marathon process + +0:45:28.879,0:45:30.639 +we see a + +0:45:30.639,0:45:32.739 +substantial performance the trees + +0:45:32.739,0:45:35.570 +you know abandon the interest rates that's +a + +0:45:35.570,0:45:37.289 +see if any + +0:45:37.289,0:45:40.140 +the trees + +0:45:40.140,0:45:43.220 +we fired up because the views that + +0:45:43.220,0:45:44.249 +when I saw it + +0:45:44.249,0:45:46.190 +the interesting thing here is to see it + +0:45:46.190,0:45:49.429 +we didn't know statistically significant difference + +0:45:49.429,0:45:52.819 +not between the baseline news with with a + +0:45:52.819,0:45:56.539 +southern cross is it for you see you sense +we don't see this very + +0:45:56.539,0:45:58.520 +which is nice to know that this is it + +0:45:58.520,0:45:59.509 +that's it + +0:45:59.509,0:46:02.869 +we have to we have to see a slight performance +improvement + +0:46:02.869,0:46:04.179 +and %uh + +0:46:04.179,0:46:05.579 +we %uh + +0:46:05.579,0:46:07.589 +we see a reduction in various + +0:46:07.589,0:46:11.569 +hans this issue he says action program performance +even lot of love it + +0:46:11.569,0:46:13.510 +and then you see a vote in the case + +0:46:13.510,0:46:15.589 +it's it's a it's the same + +0:46:15.589,0:46:20.319 +for the opposite the other on process is a +stop on others he years + +0:46:20.319,0:46:22.820 +one interesting side note actually is there + +0:46:22.820,0:46:26.719 +where is the what I was doing some tests early +on + +0:46:26.719,0:46:27.869 +we actually saw + +0:46:27.869,0:46:32.359 +the training base line in the base and a seat +in the senate he just fired off with the original + +0:46:32.359,0:46:33.869 +either of them which + +0:46:33.869,0:46:34.540 +greta + +0:46:34.540,0:46:36.489 +grass seed use your own + +0:46:36.489,0:46:39.339 +he's also if a performance decline + +0:46:39.339,0:46:42.319 +because there's a lot of stuff that ends up +running on see user + +0:46:42.319,0:46:43.819 +which %uh + +0:46:43.819,0:46:45.100 +what led to the year + +0:46:45.100,0:46:49.890 +the for a conservationist you want to allocate +from the large numbers down + +0:46:49.890,0:46:50.569 +so the issue + +0:46:50.569,0:46:55.069 +you see used to not learning the random process +is that it's gone forever we're getting all + +0:46:55.069,0:46:57.880 +the interruptions some architecture's + +0:46:57.880,0:47:02.199 +and it's a way to force the road project + +0:47:02.199,0:47:04.029 +so some conclusions + +0:47:04.029,0:47:07.530 +all I think we have useful prefer concept +of going to be deploying + +0:47:07.530,0:47:09.880 +I was certainly the man with the %uh + +0:47:09.880,0:47:11.000 +memories are seeing + +0:47:11.000,0:47:13.329 +once we have to bring it to you at seven what + +0:47:13.329,0:47:15.959 +definitely be going to see few sets up as +well + +0:47:15.959,0:47:16.849 +so it's a + +0:47:16.849,0:47:18.509 +both includes performance + +0:47:18.509,0:47:22.009 +in the contentious in the and contentious + +0:47:22.009,0:47:26.299 +we would like in the future to do some work +with a personal private superstar + +0:47:26.299,0:47:28.979 +in particular and the really interesting + +0:47:28.979,0:47:30.759 +you know when different + +0:47:30.759,0:47:32.540 +different previous the persons in jails + +0:47:32.540,0:47:37.660 +for to run up for instance up one several +sentences in jail since eleven cents lost + +0:47:37.660,0:47:40.649 +on our allies assistance + +0:47:40.649,0:47:43.240 +there could actually be some really interesting +things there + +0:47:43.240,0:47:45.759 +on in that process which one + +0:47:45.759,0:47:50.989 +we think is actually teach reason except occasions +it's never going to happen only if one takes + +0:47:50.989,0:47:53.069 +we are simply there's another example work + +0:47:53.069,0:47:56.269 +%uh also goes to Tsvangirai who recently + +0:47:56.269,0:48:01.039 +and what if the lights on the same farm workers +seen if we don't have time to implement + +0:48:01.039,0:48:04.900 +all in basic matrix multiplication + +0:48:04.900,0:48:07.230 +relative to current with with current because + +0:48:07.230,0:48:08.549 +%um + +0:48:08.549,0:48:11.849 +previously supervision functionality + +0:48:11.849,0:48:14.499 +that's been reduced the number of GOP entries + +0:48:14.499,0:48:16.150 +on anonymity a stable + +0:48:16.150,0:48:17.229 +and sir + +0:48:17.229,0:48:21.109 +that sort of thing can apply even to allow +me to sing population + +0:48:21.109,0:48:23.969 +a previous the summer winds there + +0:48:23.969,0:48:26.309 +on + +0:48:26.309,0:48:27.579 +Michael what did that work + +0:48:27.579,0:48:30.859 +on the whole on the planet who is leading +uses proliferates + +0:48:30.859,0:48:32.620 +one of the issues we've had is that + +0:48:32.620,0:48:34.019 +you will need to win in the fall + +0:48:34.019,0:48:38.399 +we have reasons to depend on all sorts of +a also to libraries immediate which + +0:48:38.399,0:48:41.380 +you know the vendors like to have them to +do + +0:48:41.380,0:48:44.640 +stupid eight API briefing change is fairly +regularly said + +0:48:44.640,0:48:48.380 +it be nice for users if we can get all the +benefits to cooperate + +0:48:48.380,0:48:51.699 +and you wouldn't have taken operate at their +leisure + +0:48:51.699,0:48:54.459 +so we're hoping to be that in future as well + +0:48:54.459,0:48:57.809 +all would like to see more women Sunday with +type resources + +0:48:57.809,0:48:59.219 +%um + +0:48:59.219,0:49:01.199 +for instance a limiting the amount of + +0:49:01.199,0:49:02.910 +%um + +0:49:02.910,0:49:05.649 +it's it's really what you want you know it's +like you know + +0:49:05.649,0:49:10.279 +all but it's her if you want a place of for +a limit on networking with by a particular + +0:49:10.279,0:49:11.819 +process + +0:49:11.819,0:49:16.979 +all our store almost forced to resign one +and ask how do you classify that traffic with + +0:49:16.979,0:49:17.649 +that + +0:49:17.649,0:49:21.259 +after going to change the current somehow +taking that + +0:49:21.259,0:49:23.799 +it's an interesting challenge + +0:49:23.799,0:49:28.309 +we'd also like to see it could be needed some +you implement something like blacks the irish + +0:49:28.309,0:49:30.089 +job when you're out there + +0:49:30.089,0:49:34.099 +so I was scheduled to just hang out at processes +as part of a job + +0:49:34.099,0:49:36.309 +%uh currently + +0:49:36.309,0:49:38.939 +I've heard it uses a clever but people past + +0:49:38.939,0:49:40.010 +were they out + +0:49:40.010,0:49:42.509 +an extra boost to the process + +0:49:42.509,0:49:44.819 +and they just outrageous troops there + +0:49:44.819,0:49:48.209 +available so they get inherited in the users +can drop them said + +0:49:48.209,0:49:51.889 +thousand track process that it's about what +happened with the correct limits on the number + +0:49:51.889,0:49:57.499 +of groups that can become a real problem + +0:49:57.499,0:49:59.529 +actually for raising questions + +0:49:59.529,0:49:59.980 +argumentative + +0:49:59.980,0:50:01.119 +one quick point + +0:50:01.119,0:50:05.100 +the think it's not interesting you live in +the area and if you're looking for + +0:50:05.100,0:50:06.430 +looking for jobs + +0:50:06.430,0:50:09.780 +we are trying to hire few people it's difficult +to hire good + +0:50:09.780,0:50:13.069 +we do have some some openings and you're looking +for + +0:50:13.069,0:50:17.409 +PSD people in general system ads +people + +0:50:17.409,0:50:24.409 +so questions + +0:50:38.419,0:50:40.989 +%um + +0:50:40.989,0:50:45.719 +I would would expect that to happen +but it's not something that attempted to test + +0:50:45.719,0:50:50.570 +what I would really like is a topology allocator + +0:50:50.570,0:50:53.179 +so that you can request that you know I want + +0:50:53.179,0:50:56.229 +I want to share cache or I don't want to share cache + +0:50:56.229,0:51:00.170 +I want to share memory band width or i want to not share memory band width + +0:51:00.170,0:51:02.459 +actually open MPI of three + +0:51:02.459,0:51:08.469 +on the 1x side have a topology where a rapper for that CPU + +0:51:08.469,0:51:10.159 +functionality + +0:51:10.159,0:51:12.249 +makes it something called %uh + +0:51:12.249,0:51:14.139 +the PLAP + +0:51:14.139,0:51:15.259 +portable lenux + +0:51:15.259,0:51:16.519 +%um + +0:51:16.519,0:51:19.599 +CPU allocator.Is that what +it's actually been + +0:51:19.599,0:51:21.959 +what it would act the act on his part + +0:51:21.959,0:51:25.400 +in essence they have to work around the fact +that there were three standard + +0:51:25.400,0:51:27.809 +there there were three different + +0:51:27.809,0:51:31.759 +currently the eyes for the same siskel + +0:51:31.759,0:51:38.759 +first the EU or takes to get all the letters +to get himself some have used the same number + +0:51:38.769,0:51:44.969 +they're completely about some of these guys +were for routine + +0:51:44.969,0:51:48.749 +when you first saw the application it calls +and this is called a test of figure out which + +0:51:48.749,0:51:50.579 +one it is + +0:51:50.579,0:51:52.719 +whitewater reserve returns depending on what + +0:51:52.719,0:51:56.139 +are you missing and completely evil + +0:51:56.139,0:52:00.859 +I think people support the heat very the eye +and we should have their lives were that + +0:52:00.859,0:52:05.650 +we don't need to do that job because they +didn't make that mistake + +0:52:05.650,0:52:12.650 +so I would like to see it of all its universal +particular + +0:52:30.710,0:52:32.529 +yes larry King + +0:52:32.529,0:52:37.180 +the trick is to do you want to be leaving +it's he's he's a limited application + +0:52:37.180,0:52:38.869 +then with + +0:52:38.869,0:52:39.500 +on + +0:52:39.500,0:52:42.269 +there's no easy limit application + +0:52:42.269,0:52:44.329 +the defense more difficult when you have to + +0:52:44.329,0:52:45.430 +the if you're + +0:52:45.430,0:52:49.759 +the new faces shared between application traffic + +0:52:49.759,0:52:50.880 +%um + +0:52:50.880,0:52:53.049 +they had a fast + +0:52:53.049,0:52:57.399 +getting classifying that is going to treat +you have to take you what the data for particular + +0:52:57.399,0:53:04.399 +to to trace that down through the current +interim I said certainly do + +0:53:12.069,0:53:15.499 +I I am + +0:53:15.499,0:53:18.389 +I I have talked contemplating doing just that + +0:53:18.389,0:53:22.059 +or in fact %uh the other thing we consider +doing + +0:53:22.059,0:53:24.829 +morris a research project that is not practical + +0:53:24.829,0:53:26.719 +jane would be actually help + +0:53:26.719,0:53:28.619 +with would be + +0:53:28.619,0:53:30.029 +independent he lands + +0:53:30.029,0:53:31.839 +because then we could do + +0:53:31.839,0:53:32.459 +things like + +0:53:32.459,0:53:35.489 +the peace process of the lambs' the couldn't +even + +0:53:35.489,0:53:37.979 +sheer at the internet where + +0:53:37.979,0:53:41.259 +once the images in place for instance we will +be able to do that + +0:53:41.259,0:53:45.049 +and that say you know you've got your cases +jurors whatever + +0:53:45.049,0:53:46.479 +on but then we can't win it + +0:53:46.479,0:53:49.959 +we cannot we could raise when did that occur +on can also have + +0:53:49.959,0:53:54.729 +we got of physically isolated we got a lot +of this would have been at work as well + +0:53:54.729,0:53:57.589 +with some analysts which is we can actually +raped women + +0:53:57.589,0:54:04.589 +in this without the switch as well + +0:54:19.939,0:54:22.369 +Bob so that's the first question + +0:54:22.369,0:54:26.190 +we get more and more visible sensitivity cannot +miss foster's + +0:54:26.190,0:54:27.639 +last night the oscars + +0:54:27.639,0:54:28.709 +I'm sorry + +0:54:28.709,0:54:30.460 +we've avoided that problem I think + +0:54:30.460,0:54:32.299 +not allowing it + +0:54:32.299,0:54:34.929 +but is it is a real issue + +0:54:34.929,0:54:36.939 +she's not one we've had to do it + +0:54:36.939,0:54:39.559 +%um + +0:54:39.559,0:54:42.109 +in practice was stuff that sensitive + +0:54:42.109,0:54:43.059 +%um + +0:54:43.059,0:54:47.579 +has handling requirements that you can't touch +the same program without a strong said + +0:54:47.579,0:54:49.859 +you need a mystery + +0:54:49.859,0:54:51.739 +ridiculously aggressive + +0:54:51.739,0:54:53.770 +you need a very close to a new letter to the +end + +0:54:53.770,0:54:57.240 +a ridiculous noted aging process that you +never moved over there + +0:54:57.240,0:55:00.959 +so if I were to do that until we get rid of +that test + +0:55:00.959,0:55:01.389 +just + +0:55:01.389,0:55:02.400 +the witness list + +0:55:02.400,0:55:04.910 +don't get rid of my number-one failure case +of + +0:55:04.910,0:55:07.839 +that would be pretty good but + +0:55:07.839,0:55:09.419 +but havent done that + +0:55:09.419,0:55:10.609 +on + +0:55:10.609,0:55:13.819 +so we we've had occasional problems of NFS overloading + + +0:55:13.819,0:55:15.679 +we haven't had real problem + +0:55:15.679,0:55:19.279 +we're all local network is fairly tightly +contained so we haven't had problems with + +0:55:19.279,0:55:20.539 +things + +0:55:20.539,0:55:21.819 +%uh with + +0:55:21.819,0:55:26.039 +you know the server going down for extended +periods and causing everything to hang + +0:55:26.039,0:55:27.819 +it's been more an issue of + +0:55:27.819,0:55:33.189 +I mean there there isn't there's a problem +that Panache is described as in cast + +0:55:33.189,0:55:36.109 +you can take out any NFS + +0:55:36.109,0:55:40.809 +I mean we have the BLueDrat guys come in and the +PGA this stuff multiple ten-gate I said + +0:55:40.809,0:55:42.049 +you know I've got + +0:55:42.049,0:55:46.779 +to do this and they said can we not try this with all your cluster + +0:55:46.779,0:55:47.950 +because if you got + +0:55:47.950,0:55:49.370 +three hundred and fifty + +0:55:49.370,0:55:52.599 +gigabit ethernet interface is going into +the system + +0:55:52.599,0:55:56.589 +even ten gig you can saturate pre-turbulate + +0:55:56.589,0:55:57.120 +so that level + +0:55:57.120,0:55:58.930 +there's an inherent problem + +0:55:58.930,0:56:01.969 +on we need to handle that kind of band width +we've + +0:56:01.969,0:56:04.459 +got to get it a parallel file system + +0:56:04.459,0:56:06.069 +get a cluster + +0:56:06.069,0:56:12.289 +before doing streaming stuff we could go file some loners + +0:56:12.289,0:56:14.949 +anyone else? + +0:56:14.949,0:56:15.429 +thank you diff --git a/en_US.ISO8859-1/captions/2009/dcbsdcon/dixon-bsdisstilldying.sbv b/en_US.ISO8859-1/captions/2009/dcbsdcon/dixon-bsdisstilldying.sbv new file mode 100644 index 0000000000..baf87e0de7 --- /dev/null +++ b/en_US.ISO8859-1/captions/2009/dcbsdcon/dixon-bsdisstilldying.sbv @@ -0,0 +1,1484 @@ +0:00:00.460,0:00:04.370 +BSD is Still Dying + +0:00:04.370,0:00:06.220 +Welcome to BSD is still Dying + +0:00:06.220,0:00:07.790 +It’s not quite dead yet + +0:00:07.790,0:00:09.409 +but we're getting there + +0:00:09.409,0:00:09.980 +and %uh + +0:00:09.980,0:00:16.980 +Well, welcome to the closing of the DCBSDCon +2009 + +0:00:17.240,0:00:19.330 +So, what is BSD? + +0:00:19.330,0:00:22.299 +Well, BSD is the derivative of UNIX + +0:00:22.299,0:00:23.879 +Okay, so what is UNIX? + +0:00:23.879,0:00:26.749 +UNIX is an operating system + +0:00:26.749,0:00:29.259 +What’s an operating system? + +0:00:29.259,0:00:32.560 +Operating system is the soul of the computer + +0:00:32.560,0:00:34.000 +But, what’s a computer? + +0:00:34.000,0:00:37.469 +A computer is a tool, it’s basically a glorified calculator + +0:00:37.469,0:00:42.160 +that enables users to accomplish tasks better + +0:00:42.160,0:00:44.060 +So, what is a user? + +0:00:44.060,0:00:46.190 +A user is someone who operates a computer + +0:00:46.190,0:00:49.580 +It tends to stand up right sort of like me + +0:00:49.580,0:00:51.940 +and Bob(?) + +0:00:51.940,0:00:52.840 +So, who am I? + +0:00:52.840,0:00:54.610 +My name is Jason Dixon + +0:00:54.610,0:00:57.080 +First and foremost, I’m a SysAdmin + +0:00:57.080,0:00:59.500 +I like to work on networks and firewalls + +0:00:59.500,0:01:02.870 +I like to tweak + +0:01:02.870,0:01:03.630 +no, yes + +0:01:03.630,0:01:05.650 +I'm a programmer, sort of + +0:01:05.650,0:01:07.190 +I enjoy programming with Perl + +0:01:08.450,0:01:10.340 +and Apache Web servers + +0:01:10.340,0:01:12.229 +I'm a consultant here + +0:01:12.229,0:01:15.159 +I'm an employee + +0:01:15.159,0:01:16.840 +there + +0:01:16.840,0:01:18.880 +.. + +0:01:18.880,0:01:19.819 +I'm sorry, [xx] meets the eye + +0:01:19.819,0:01:25.430 +I + +0:01:25.430,0:01:30.229 +[xx] + +0:01:30.229,0:01:32.909 +[xx] + +0:01:32.909,0:01:33.870 +And I'm a lover + +0:01:33.870,0:01:36.240 +of BSD + +0:01:36.240,0:01:38.360 +Okay, but why am I here? + +0:01:38.360,0:01:39.880 +To talk about why + +0:01:39.880,0:01:43.830 +BSD is dying + +0:01:43.830,0:01:45.270 +sex + +0:01:45.270,0:01:47.710 +and greed + +0:01:47.710,0:01:50.210 +It really has nothing to do with either of this + +0:01:50.210,0:01:53.170 +But if I told you licensing is a + +0:01:53.170,0:01:55.750 +[xx] + +0:01:55.750,0:01:58.160 +To quickly summarize, what is BSD? + +0:01:58.160,0:01:59.290 +What is UNIX? + +0:01:59.290,0:02:00.800 +What is an operating system? + +0:02:00.800,0:02:02.310 +What is a computer? + +0:02:02.310,0:02:04.929 +A computer is a device that computes + +0:02:04.929,0:02:08.429 +especially, a programmable electronic machine +that performs high speed mathematical + +0:02:08.429,0:02:10.180 +or logical operations + +0:02:10.180,0:02:13.869 +or that assembles, stores, correlates, or otherwise processes +information + +0:02:13.869,0:02:15.189 +This is a computer + +0:02:15.189,0:02:17.029 +This is a small computer + +0:02:17.029,0:02:18.899 +This is a big computer + +0:02:18.899,0:02:20.169 +This is a big + +0:02:20.169,0:02:23.479 +fake(?) computer + +0:02:23.479,0:02:26.889 +And this is a really old computer + +0:02:26.889,0:02:28.909 +Well, what does a computer really do? + +0:02:28.909,0:02:30.670 +It helps us write documents + +0:02:30.670,0:02:32.509 +For example, [xx] fun pages + +0:02:32.509,0:02:33.859 +I + +0:02:33.859,0:02:36.159 +[xx] shopping lists + +0:02:36.159,0:02:38.569 +Computers can even delete documents on the + +0:02:38.569,0:02:41.459 +fly + +0:02:41.459,0:02:43.340 +We can write emails + +0:02:43.340,0:02:44.370 +surf the Web + +0:02:44.370,0:02:46.539 +watch movies + +0:02:46.539,0:02:48.889 +movies + +0:02:48.889,0:02:50.489 +listen to our favorite music + +0:02:50.489,0:02:54.279 +and even play games + +0:02:54.279,0:02:56.839 +But how does the computer do all these things? + +0:02:56.839,0:02:59.609 +Let's start by taking text [xx] source code + +0:02:59.609,0:03:03.439 +and using the [xx] to translate it into binary +machine language + +0:03:03.439,0:03:05.119 +That’s the foundation + +0:03:05.119,0:03:06.310 +for the kernel + +0:03:06.310,0:03:06.910 +libraries + +0:03:06.910,0:03:09.100 +and userland application, otherwise known as + +0:03:09.100,0:03:14.609 +an operating system + +0:03:14.609,0:03:15.709 +like BSD. So you ask + +0:03:15.709,0:03:17.649 +What is a kernel? + +0:03:17.649,0:03:19.549 +It's a wonderful thing + +0:03:19.549,0:03:21.209 +[xx] for the management + +0:03:21.209,0:03:23.219 +or processes memory + +0:03:23.219,0:03:28.009 +and peripheral devices + +0:03:28.009,0:03:30.730 +and by extension, allows us to do [xx] stuff +like + +0:03:30.730,0:03:31.620 +networking + +0:03:31.620,0:03:33.319 +provide better security + +0:03:33.319,0:03:36.930 +work with disks and file systems, create user interfaces + +0:03:36.930,0:03:39.900 +interactive userland applications, allow us +to do things like + +0:03:39.900,0:03:43.249 +write documents, read emails, surf the Web, +watch movies + +0:03:43.249,0:03:44.639 +listen to music + +0:03:44.639,0:03:45.219 +play games + +0:03:45.219,0:03:51.879 +and much, much more + +0:03:51.879,0:03:54.619 +In summary + +0:03:54.619,0:03:58.359 +BSD is a UNIX-derived operating system that enables +users to harness the power of + +0:03:58.359,0:04:01.049 +the computer and process information better + +0:04:01.049,0:04:05.239 +It uses a combination of processes, memories, +and peripheral devices, and by extension + +0:04:05.239,0:04:09.379 +We can perform networking, enforce security, +read from and write to storage devices and interface visually to applications + +0:04:09.379,0:04:10.329 +like text editors + +0:04:10.329,0:04:17.329 +mail clients, Web browsers, multimedia players, and games + +0:04:18.239,0:04:19.560 +I'd like to look back + +0:04:19.560,0:04:22.350 +on the history of UNIX for a few minutes + +0:04:22.350,0:04:24.039 +Now, to be honest + +0:04:24.039,0:04:27.460 +Kirk McKusick just trumped everything I had +from my talk + +0:04:27.460,0:04:32.620 +But, this can be a repeat, but it's still pretty good(?) + +0:04:32.620,0:04:34.770 +So, in the beginning + +0:04:34.770,0:04:37.439 +And yes, there was life before UNIX + +0:04:37.439,0:04:39.559 +We got the Holy Trinity + +0:04:39.559,0:04:43.439 +MIT, Bell Labs and GE teamed to create +a system called Multics + +0:04:43.439,0:04:46.999 +the Multiplexed Information and Computing Service + +0:04:46.999,0:04:50.709 +We have a huge GE650 mainframe Multics + +0:04:50.709,0:04:56.550 +You can tell the engineers from the [xx] bosses + +0:04:56.550,0:04:58.800 +and so, it was a huge success + +0:04:58.800,0:05:01.399 +We don’t run Multics on a laptop, mainframes, of course + +0:05:01.399,0:05:04.959 +I should know, it was a commercial failure + +0:05:04.959,0:05:08.749 +[xx] is a computer scientist at Bell Labs +named Ken Thompson + +0:05:08.749,0:05:12.110 +He’d worked on the Multics project and was inspired +by interactive computing + +0:05:12.110,0:05:13.239 +It provided + +0:05:13.239,0:05:15.500 +Unfortunately, scrapping the Multics project + +0:05:15.500,0:05:19.240 +(but they no longer have a system that question +for serious work life) +[xx] known they had a system of which to perform +serious work like + +0:05:19.240,0:05:21.220 +Space Travel + +0:05:21.220,0:05:23.329 +a space simulation game + +0:05:23.329,0:05:25.319 +[xx] from Multics GE + +0:05:25.319,0:05:27.590 +650 mainframe + +0:05:27.590,0:05:29.260 +With Dennis Ritchie’s assistance + +0:05:29.260,0:05:30.960 +[xx] experience with the Multics project + +0:05:30.960,0:05:34.439 +they all craft(?) together an operating system +[xx] language + +0:05:34.439,0:05:36.569 +[xx] PDP-7 + +0:05:36.569,0:05:40.059 +This is capable of interactive +computing with the terminal + +0:05:40.059,0:05:43.979 +rather than just being punched +keycards + +0:05:43.979,0:05:45.909 +They termed that the UNIX systems + +0:05:45.909,0:05:49.630 +UNIX system, short for Uniplexed Information +and Computing System + +0:05:49.630,0:05:50.609 +a play + +0:05:50.609,0:05:53.409 +on the Multics name + +0:05:53.409,0:05:59.169 +it's [xx] a number of users + +0:05:59.169,0:06:06.050 +Did I miss something? + +0:06:06.050,0:06:13.050 +it's for a number of users + +0:06:15.020,0:06:19.879 +talk about anticlimactic files + +0:06:19.879,0:06:21.120 +By 1970 + +0:06:21.120,0:06:23.680 +it officially became known as U-N-I-X + +0:06:23.680,0:06:27.800 +probably [xx] + +0:06:27.800,0:06:31.020 +it would be really good [xx] + +0:06:31.020,0:06:34.409 +By 1971, the UNIX System was officially +put into production use + +0:06:34.409,0:06:36.330 +and imported to the PDP-11 + +0:06:36.330,0:06:37.989 +It was capable of text processing + +0:06:37.989,0:06:42.240 +for the purpose of filing patents + +0:06:42.240,0:06:44.789 +I was trying to visualize software patents + +0:06:44.789,0:06:51.789 +and tack it [xx] and probably the closest thing to [xx] + +0:06:52.729,0:06:56.669 +By 1973 + +0:06:56.669,0:06:59.929 +they rewrote UNIX + +0:06:59.929,0:07:01.950 +in a programming language created by Dennis Ritchie + +0:07:01.950,0:07:04.569 +The C programming language evolved from the B language + +0:07:04.569,0:07:07.759 +adding data type [xx] structures + +0:07:07.759,0:07:10.680 +Based in a 1958 antitrust +Case, AT&T had been + +0:07:10.680,0:07:12.520 +forbidden to enter the computer business + +0:07:12.520,0:07:14.799 +UNIX could not be turned into a product + +0:07:14.799,0:07:16.529 +so they shift [xx] + +0:07:16.529,0:07:20.489 +and just pass the source code to anyone who ask + +0:07:20.489,0:07:22.789 +Since the source code is readily available + +0:07:22.789,0:07:25.490 +and ran in a portable computer language + +0:07:25.490,0:07:29.349 +universities and research labs worldwide were able to run +UNIX on their own systems + +0:07:29.349,0:07:31.049 +By 1974 + +0:07:31.049,0:07:32.740 +Professor Bob Fabry + +0:07:32.740,0:07:34.750 +at the University of Cal Berkeley + +0:07:34.750,0:07:36.859 +purchased a copy of UNIX for $99 + +0:07:36.859,0:07:38.990 +for their own PDP-11 + +0:07:38.990,0:07:40.659 +By 1977 + +0:07:40.659,0:07:43.579 +Bill Jolitz, a graduate student of Cal Berkeley) + +0:07:43.579,0:07:46.330 +distributed(?) the Berkeley Software Distribution + +0:07:46.330,0:07:47.259 +otherwise known as + +0:07:47.259,0:07:49.029 +1BSD + +0:07:49.029,0:07:50.759 +It included a Pascal compiler + +0:07:50.759,0:07:52.460 +the ex Editor + +0:07:52.460,0:07:53.619 +vi ex Editor + +0:07:53.619,0:07:54.900 +By 1978 + +0:07:54.900,0:07:57.159 +2BSD had been released + +0:07:57.159,0:07:59.139 +which added the vi editor + +0:07:59.139,0:08:03.149 +and the C shell + +0:08:03.149,0:08:05.459 +I'm definitely not old school [xx] editor + +0:08:05.459,0:08:08.419 +[xx] for the win + +0:08:08.419,0:08:09.789 +1979 + +0:08:09.789,0:08:12.110 +3BSD was released + +0:08:12.110,0:08:13.879 +a support for the VAX platform + +0:08:13.879,0:08:16.599 +it was coined as Virtual VAX for VMUNIX + +0:08:16.599,0:08:20.349 +based on the new virtual memory implementation +specifically for the VAX,UNIX/32V + +0:08:20.349,0:08:23.059 +Computer Berkeley + +0:08:23.059,0:08:24.830 +Based on the success of 3BSD + +0:08:24.830,0:08:28.769 +the Defense Advanced Research Project Agency (DARPA) + +0:08:28.769,0:08:33.080 +[xx] Berkeley CSRG with the contract enhanced UNIX + +0:08:33.080,0:08:35.810 +for VLSI the project + +0:08:35.810,0:08:37.550 +these enhancements were + +0:08:37.550,0:08:38.570 +eventually released + +0:08:38.570,0:08:40.080 +as 4BSD + +0:08:40.080,0:08:42.300 +which [xx] job control for the C shell + +0:08:42.300,0:08:45.580 +delivermail, the predecessor to sendmail + +0:08:45.580,0:08:48.140 +the curses [xx] + +0:08:48.140,0:08:51.240 +and reliable signals + +0:08:51.240,0:08:55.960 +Unfortunately, 4BSD was criticized for bad +performance on the VAX platform + +0:08:55.960,0:08:59.950 +so [xx] went back to work [xx] kernel +to outperform [xx] VAX + +0:08:59.950,0:09:02.900 +eventually releasing 4.1BSD for these performance + +0:09:02.900,0:09:04.030 +fixes + +0:09:04.030,0:09:05.410 +Two years later + +0:09:05.410,0:09:07.940 +4.2BSD was released + +0:09:07.940,0:09:10.650 +incorporating a TCP/IP stack BBN Technologies + +0:09:10.650,0:09:13.360 +also, a contractor for DARPA + +0:09:13.360,0:09:17.090 +as well as the Berkeley Fast File System + +0:09:17.090,0:09:21.510 +written by a dapper young man by the name of Kirk McKusick + +0:09:21.510,0:09:24.500 +He's also kind enough to give us the original BSD mascot + +0:09:24.500,0:09:25.940 +In 1986 + +0:09:25.940,0:09:28.410 +4.3BSD was unleashed + +0:09:28.410,0:09:31.300 +incorporating numerous performance improvements + +0:09:31.300,0:09:34.870 +including a non-BBN version of the BSD TCP/IP stack, +which is found to be superior by + +0:09:34.870,0:09:36.370 +DARPA + +0:09:36.370,0:09:37.290 +1998 + +0:09:37.290,0:09:40.030 +the next version of BSD was released + +0:09:40.030,0:09:45.460 +4.3BSD-Tahoe, so named for the short-lived +Power 6/32 “Tahoe” platform + +0:09:45.460,0:09:49.960 +This was an initial attempt of moving away +from the VAX platform + +0:09:49.960,0:09:53.800 +Although the Tahoe was unsuccessful, it helped introduce machine +in a [xx] that will improve + +0:09:53.800,0:10:00.720 +[xx] BSD future credibility + +0:10:00.720,0:10:04.670 +Up to this point, all versions of BSD +included proprietary AT&T code + +0:10:04.670,0:10:07.550 +[xx] require license for AT&T for their use + +0:10:07.550,0:10:10.190 +These licenses have become prohibitively expensive + +0:10:10.190,0:10:13.260 +[xx] the demand for AT&T free version of UNIX + +0:10:13.260,0:10:15.130 +This led to the release of Net/1 + +0:10:15.130,0:10:17.740 +a BSD license distribution of the FreeBSD + +0:10:17.740,0:10:18.440 +The free + +0:10:18.440,0:10:20.230 +FreeBSD networking code + +0:10:20.230,0:10:23.340 +and uncovered by any AT&T [xx] licenses + +0:10:23.340,0:10:24.319 +The next year + +0:10:24.319,0:10:25.810 +4.3BSD-Reno + +0:10:25.810,0:10:26.930 +was released + +0:10:26.930,0:10:28.610 +The name suggesting + +0:10:28.610,0:10:32.360 +a gamble [xx] production use + +0:10:32.360,0:10:35.560 +regardless, it included the MACH virtual memory +system + +0:10:35.560,0:10:37.030 +Sun-compatible NFS + +0:10:37.030,0:10:40.280 +and continued the movement towards a POSIX compliance + +0:10:40.280,0:10:44.980 +It became apparent that the AT&T code was a +Hassle [xx] CSRG + +0:10:44.980,0:10:48.060 +virtually all of the utilities and code that was +stolen from AT&T + +0:10:48.060,0:10:49.720 +Then, in the middle of 1991 + +0:10:49.720,0:10:51.459 +Net/2 was released + +0:10:51.459,0:10:56.480 +A nearly complete BSD UNIX system that was freely +distributable under the BSD license + +0:10:56.480,0:11:00.320 +Net/2 was the basis for two separate parts +of BSD to the Intel + +0:11:00.320,0:11:02.100 +8386 architecture + +0:11:02.100,0:11:05.670 +Bill Jolitz started the 386BSD +distribution + +0:11:05.670,0:11:06.590 +[xx] + +0:11:06.590,0:11:11.000 +became the basis for 3BSD and NetBSD +projects + +0:11:11.000,0:11:13.430 +Berkeley Software Design BSDI + +0:11:13.430,0:11:15.199 +came out with BSD/386 + +0:11:15.199,0:11:22.199 +A proprietary version later renamed as BSD +OS + +0:11:22.560,0:11:23.520 +In 1992 + +0:11:23.520,0:11:25.970 +a wholly-owned subsidiary of AT&T + +0:11:25.970,0:11:27.800 +UNIX System Laboratories + +0:11:27.800,0:11:30.310 +also known as USL + +0:11:30.310,0:11:34.860 +filed a suit against BSDI claiming that AT&T’s +proprietary code which included + +0:11:34.860,0:11:37.420 +the BSDI UNIX products + +0:11:37.420,0:11:41.640 +The injunction was filed asking for BSDI +to discontinue their use of advertising + +0:11:41.640,0:11:43.380 +[xx] use of the UNIX name + +0:11:43.380,0:11:45.780 +own by AT&T and supposedly aimed + +0:11:45.780,0:11:47.510 +to confuse unsuspecting + +0:11:47.510,0:11:47.880 +users + +0:11:47.880,0:11:49.860 +for virus + +0:11:49.860,0:11:53.620 +Also for the fact that the BSD OS sold +for $995 + +0:11:53.620,0:11:57.090 +a 99% discount of AT&T UNIX + +0:11:57.090,0:12:00.400 +which [xx] out for $100 to +$2,000 + +0:12:00.400,0:12:07.140 +You can see why the average consumer might +get confused + +0:12:07.140,0:12:10.720 +After extensive analysis, it was +determined that the BSD OS + +0:12:10.720,0:12:14.960 +was little more than Net/2, probably six files from Bill +Jolitz’s 386BSD + +0:12:14.960,0:12:17.000 +The judge presiding over the case + +0:12:17.000,0:12:18.430 +denied the injunction + +0:12:18.430,0:12:20.750 +forcing USL [xx] complaint + +0:12:20.750,0:12:22.280 +to recent copyrights + +0:12:22.280,0:12:24.920 +and a possibility of the loss of trade secrets + +0:12:24.920,0:12:28.580 +The judge also recommended that the case be +heard at a state court before filing in a federal + +0:12:28.580,0:12:29.450 +court + +0:12:29.450,0:12:31.460 +So, taking this hint from the judge + +0:12:31.460,0:12:33.430 +they ran out to California + +0:12:33.430,0:12:34.110 +refiled + +0:12:34.110,0:12:38.260 +as quickly as possible the University of Cal Berkeley + +0:12:38.260,0:12:40.800 +take action to prevent + +0:12:40.800,0:12:44.260 +resulted if USL wanted to take any action against + +0:12:44.260,0:12:45.940 +[xx] will be forced to do so in California + +0:12:45.940,0:12:49.860 +rather than their home state of New Jersey + +0:12:49.860,0:12:51.720 +Soon after the filing in a state court + +0:12:51.720,0:12:55.380 +USL has bought(?) from AT&T by Novell + +0:12:55.380,0:12:58.760 +By 1994, a settlement had been reached in private + +0:12:58.760,0:13:01.100 +[xx] details of the settlement were not known + +0:13:01.100,0:13:03.920 +until 2004 when a California +public records law + +0:13:03.920,0:13:06.570 +allow the details to be released + +0:13:06.570,0:13:10.070 +[xx] published on the Groklaw site + +0:13:10.070,0:13:11.810 +Details of the settlement included + +0:13:11.810,0:13:13.870 +I think we've covered this on + +0:13:13.870,0:13:16.640 +Chri’s talk + +0:13:16.640,0:13:17.580 +there were those +[xx] + +0:13:17.580,0:13:23.740 +In hindsight, this confirmed our suspicions +Of USL’s fate + +0:13:23.740,0:13:25.800 +In June of 1994 + +0:13:25.800,0:13:27.930 +the 2BSD distributions were released + +0:13:27.930,0:13:29.509 +4.4BSD-Encumbered + +0:13:29.509,0:13:31.650 +was a version of BSD with AT&T code + +0:13:31.650,0:13:34.800 +It was only available to AT&T licensees + +0:13:34.800,0:13:38.510 +4.4BSD-Lite was also released which +now contained AT&T code + +0:13:38.510,0:13:42.120 +They become a new basis for the [xx] for the +previously [xx] + +0:13:42.120,0:13:46.890 +Trees +[xx] + +0:13:46.890,0:13:49.430 +As I mentioned. 4.4BSD-Lite + +0:13:49.430,0:13:53.530 +became the new baseline from which all the BSD +distributions will be based + +0:13:53.530,0:13:56.150 +FreeBSD, one of the first + +0:13:56.150,0:14:03.110 +and probably the most popular of the modern BSD +distributions + +0:14:03.110,0:14:07.130 +has a story [xx] focused on the Intel +8386 platform + +0:14:07.130,0:14:12.370 +and [xx] Windows markets + +0:14:12.370,0:14:13.570 +Nevertheless + +0:14:13.570,0:14:18.020 +FreeBSD has been [xx] platforms including +Intel x86, Itanium + + +0:14:18.020,0:14:18.680 +AMD64 + +0:14:18.680,0:14:20.360 +DEC Alpha + +0:14:20.360,0:14:23.130 +PowerPC, and Sun UltraSPARC + +0:14:23.130,0:14:27.420 +So, the more popular features include +application jails, access controls + +0:14:27.420,0:14:30.340 +excellent networking, SMP performance + +0:14:30.340,0:14:35.330 +and more recently, [xx] Solaris ZFS and dtrace + +0:14:35.330,0:14:36.190 +NetBSD + +0:14:36.190,0:14:40.200 +which was also richly derived from Jolitz’s [xx] +BSD + +0:14:40.200,0:14:44.710 +is typically known for portability through a wide range of +[xx] platforms + +0:14:44.710,0:14:51.060 +NetBSD has been [xx] to over 50 hardware platforms + +0:14:51.060,0:14:58.060 +and kitchen appliances + +0:15:00.110,0:15:04.020 +OpenBSD + +0:15:04.020,0:15:08.070 +no bias at all + +0:15:08.070,0:15:10.840 +OpenBSD is yet another modern BSD derivative + +0:15:10.840,0:15:13.140 +originally [xx] from NetBSD 1.0 + +0:15:13.140,0:15:14.769 +for being highly secure + +0:15:14.769,0:15:16.910 +with an emphasis on code correctness + +0:15:16.910,0:15:21.160 +proper documentation and truly open in free source + +0:15:21.160,0:15:23.600 +the mantra is secure by default + +0:15:23.600,0:15:28.220 +This philosophy is influenced in countless other free +proprietary operating systems which now + +0:15:28.220,0:15:29.950 +the example + +0:15:29.950,0:15:34.220 +Here are some of the more popular platforms +[xx] BSD has been imported to + +0:15:34.220,0:15:37.380 +And although you may see numerous security enhancements + +0:15:37.380,0:15:41.790 +to make efforts to integrate these changes into the [xx] system +[xx] transfer of technologies + +0:15:41.790,0:15:46.470 +The philosophy has been proven less effective +[xx] technology such as SELinux + +0:15:46.470,0:15:50.290 +[xx] is placed on the Systems Administrator + +0:15:50.290,0:15:53.590 +and is quite often disabled + +0:15:53.590,0:15:55.680 +Be generous + +0:15:55.680,0:15:57.710 +You see + +0:15:57.710,0:15:58.940 +You see + +0:15:58.940,0:16:05.940 +some of the features of OpenBSD + +0:16:08.460,0:16:09.710 +[xx] + +0:16:09.710,0:16:14.760 +[xx] + +0:16:14.760,0:16:20.720 +[xx] + +0:16:20.720,0:16:26.450 +They have consistent release schedule of new releases +available around the first of May and November + +0:16:26.450,0:16:27.510 +DragonFly + +0:16:27.510,0:16:31.850 +another BSD, started by Matt Dillon in 2003 + +0:16:31.850,0:16:34.500 +It's a logical continuation of FreeBSD +4.8 + +0:16:34.500,0:16:38.780 +He started the foot-dragging 5BSD project +when his vision for threading an SMP conflicted + +0:16:38.780,0:16:41.610 +(with the other developers working on FreeBSD +file + +0:16:41.610,0:16:45.540 +They continue work on SMP revamp +which, I think, is probably complete by now + +0:16:45.540,0:16:47.369 +as well as the other lightweight + +0:16:47.369,0:16:49.100 +kernel threads implementation + +0:16:49.100,0:16:53.550 +(well Martin balsam focused on spring jenner +jericho street supportive of the coral and) +[xx] focused on generating +[xx] support the kernel + +0:16:53.550,0:16:56.700 +and features like the [xx] + +0:16:56.700,0:16:58.599 +MAC OS X is an operating system + +0:16:58.599,0:16:59.459 +sold by Apple + +0:16:59.459,0:17:06.010 +which runs on both PowerPC and Intel platforms) + +0:17:06.010,0:17:08.210 +hecklers + +0:17:08.210,0:17:12.709 +BSD is the heart of MacOS X, +a full capable BSD UNIX derivative + +0:17:12.709,0:17:16.510 +the enhancement was [xx] to make OS X + +0:17:16.510,0:17:19.270 +MAC OS + +0:17:20.890,0:17:23.150 +a consumer-friendly operating system + +0:17:23.150,0:17:25.540 +Unfortunately, it's still somewhat of a hybrid + +0:17:25.540,0:17:30.320 +I + +0:17:30.320,0:17:32.630 +yeah + +0:17:32.630,0:17:34.530 +Merging the MAC kernel with NeXTSTEP + +0:17:34.530,0:17:41.350 +while it's very useful in +FreeBSD + +0:17:41.350,0:17:43.740 +It's about the BSD distributions including + +0:17:43.740,0:17:45.780 +364 units from DEC + +0:17:45.780,0:17:52.780 +then Compaq, then HP + +0:17:54.710,0:17:58.490 +Now, we've covered some of the history behind BSD. +I like to address the real shortcomings + +0:17:58.490,0:18:00.960 +of modern day BSD + +0:18:00.960,0:18:3.980 +So, why is BSD dying? + +0:18:03.980,0:18:05.890 +That's why we're here to uncover + +0:18:05.890,0:18:08.650 +First and foremost + +0:18:08.650,0:18:13.570 +because IDC says so + +0:18:13.570,0:18:15.290 +Market share is an all-time low + +0:18:15.290,0:18:17.830 +under 1% + +0:18:17.830,0:18:18.970 +And of course + +0:18:18.970,0:18:20.010 +Netcraft + +0:18:20.010,0:18:24.170 +confirms these findings + +0:18:24.170,0:18:26.309 +BSD came in last place + +0:18:26.309,0:18:28.240 +in a SysAdmin networking test + +0:18:28.240,0:18:35.240 +I don’t recall which of the BSDs were tested, +but it's probably safe to assume they all finished [xx] last + +0:18:37.570,0:18:41.190 +Market leaders and pundits have predicted that +open-source software can't make money + +0:18:41.190,0:18:45.320 +If we continue to give away free software, how +are we finance our developers? + +0:18:45.320,0:18:51.290 +[xx] the forecasting [xx] the spiraling into bankruptcy + +0:18:51.290,0:18:52.669 +I + +0:18:52.669,0:18:53.690 +yeah + +0:18:53.690,0:19:00.690 +It's a little dated, but man, you just got to love this guy + +0:19:00.700,0:19:05.470 +BSD + +0:19:05.470,0:19:06.520 +As a community + +0:19:06.520,0:19:10.500 +BSD has had a surprising inability to +adapt + +0:19:10.500,0:19:12.140 +As we can see by this graph + +0:19:12.140,0:19:14.930 +I + +0:19:14.930,0:19:19.630 +I + +0:19:19.630,0:19:24.940 +As we can see by this graph, the number of +pages served per hour + +0:19:24.940,0:19:27.030 +[xx] far outnumber those + +0:19:27.030,0:19:29.640 +on Linux and all the BSDs + +0:19:29.640,0:19:34.750 +Linux actually degenerates some [xx] we +have to presume because of [xx] + +0:19:34.750,0:19:40.520 +renaming their files ASP + +0:19:40.520,0:19:43.560 +Of course, a significant loss of talent + +0:19:43.560,0:19:46.310 +FreeBSD has lost 93% of their core +developers + +0:19:46.310,0:19:50.370 +This is certainly the [xx] to jump ship [xx] + +0:19:50.370,0:19:54.850 +Unfortunately, since none of the BSDs [xx] code, +they usually have to start from scratch all over again + +0:19:54.850,0:19:55.970 +Tracing the project’s roots + +0:19:55.970,0:19:59.880 +from 4.5BSD-lite + +0:19:59.880,0:20:04.890 +Fortunately, not all is lost + +0:20:04.890,0:20:11.890 +There's still a handful of very small companies still using +BSD today + +0:20:17.140,0:20:23.559 +And you probably haven’t heard most of these + +0:20:23.559,0:20:28.480 +This is my employer + +0:20:28.480,0:20:31.510 + +0:20:31.510,0:20:32.990 +Hopefully, the success + +0:20:32.990,0:20:37.230 +yes, the [xx] had told me +I had to put that on there + +0:20:37.230,0:20:44.230 +[xx] the success as seen [xx] seen in the industry’s +Will [xx] open in the mainstream + +0:20:45.050,0:20:47.230 +Looking forward, there's a number of challenges ahead + +0:20:47.230,0:20:50.960 +ahead of us to ensure the BSD survives +generations of future geeks + +0:20:50.960,0:20:53.540 +The challenges aren’t simply of the technological +nature + +0:20:53.540,0:20:57.750 +but included a number of political and legal +obstacles as well + +0:20:57.750,0:21:03.240 +First and foremost, virtualization is on everybody’s +radar + +0:21:03.240,0:21:04.400 +[xx] + +0:21:04.400,0:21:09.650 +[xx] to optimize our resource running BSD + +0:21:09.650,0:21:13.820 +All this means I can run over 100,000 [xx] of NetBSD +on a server four gigs of + +0:21:13.820,0:21:14.690 +memory + +0:21:14.690,0:21:18.520 +It also means I have to hire 1,000 +NetBSD systems administrators + +0:21:18.520,0:21:20.360 + +0:21:20.360,0:21:23.940 +Not only is this a miserable return of investment +I don’t think there are over a thousand NetBSD users out there + +0:21:23.940,0:21:30.280 + +0:21:30.280,0:21:33.350 +We're looking for more advanced file systems + +0:21:33.350,0:21:38.970 +to handle the current performance limits +associated with high capacity multi-tier + +0:21:38.970,0:21:43.850 +[xx] Solaris CFS have already been completed +[xx] previous BSD-7 and MAC) + +0:21:43.850,0:21:46.460 +OS X + +0:21:46.460,0:21:48.799 +They have blobs, NDA’s + +0:21:48.799,0:21:51.100 +Closed documentation can now go hand in hand + +0:21:51.100,0:21:55.550 +It's a clerical challenge that threatens +[xx] hardware availability to BSD developers + +0:21:55.550,0:21:56.809 +and endusers + +0:21:56.809,0:21:59.370 +Projects like Linux, and even FreeBSD + +0:21:59.370,0:22:00.830 +had casually signed + +0:22:00.830,0:22:04.770 +agreements and NDA’s to accept none free binary +drivers into the source stream + +0:22:04.770,0:22:07.320 +These arrangements worked against the spirit +of free software + +0:22:07.320,0:22:08.920 +and open-source + +0:22:08.920,0:22:10.940 +software, all for the purposes of short-term gains + +0:22:10.940,0:22:13.690 +by 3D [xx] and rotating desktops) + +0:22:13.690,0:22:18.510 +That’s coming the guys [xx] serving in the next room + +0:22:18.510,0:22:21.220 +Within the last year or so, legal challenges have surfaced + +0:22:21.220,0:22:24.720 +Linux developers have blatantly stolen BSD license code + +0:22:24.720,0:22:27.270 +replacing the license notification with GPL + +0:22:27.270,0:22:29.700 +and any of their copyrights when no work was done + +0:22:29.700,0:22:33.419 +Beyond the legal ramifications of these actions, +this demonstrate a lack of respect + +0:22:33.419,0:22:38.050 +in cooperation for their free +software peers + +0:22:38.050,0:22:39.649 +diversity in the BSD is a healthy trend + +0:22:39.649,0:22:41.620 +it promotes new features and competition + +0:22:41.620,0:22:43.220 +in the intellectual market + +0:22:43.220,0:22:47.169 +resulting in better systems for all of us + +0:22:47.169,0:22:49.740 +In the end, diversity can bring unity throughout our community + +0:22:49.740,0:22:50.519 +and with unity + +0:22:50.519,0:22:53.020 +a common goal + +0:22:53.020,0:22:54.290 +The end diff --git a/en_US.ISO8859-1/captions/2009/dcbsdcon/mckusick-historyofbsd.sbv b/en_US.ISO8859-1/captions/2009/dcbsdcon/mckusick-historyofbsd.sbv new file mode 100644 index 0000000000..8b93823c28 --- /dev/null +++ b/en_US.ISO8859-1/captions/2009/dcbsdcon/mckusick-historyofbsd.sbv @@ -0,0 +1,3908 @@ +0:00:00.570,0:00:03.510
+Anyways it is my great honour and privilege
+
+0:00:03.510,0:00:10.510
+to introduce Kirk McKusick uh! Which is
+give a talk our history of BSD
+
+0:00:13.570,0:00:17.170
+So this is actually a three-hour lecture
+
+0:00:17.170,0:00:19.949
+I but I don't have three hours so there is
+
+0:00:19.949,0:00:23.659
+uh is this where I allow audience participation
+
+0:00:23.659,0:00:27.099
+The lecture is really sort of three
+main parts
+
+0:00:27.099,0:00:30.460
+and what Ill do is
+
+0:00:30.460,0:00:34.210
+describe the three main parts. Then everyone
+will get to vote on which part they find the
+
+0:00:34.210,0:00:35.490
+most interesting
+
+0:00:35.490,0:00:36.510
+Then what I.. I wont..
+
+0:00:36.510,0:00:41.309
+I used to just do just that part and Ive got a
+lot of negative feedback about that's all
+
+0:00:41.309,0:00:43.819
+So Ill do those two parts don't get elected in
+
+0:00:43.819,0:00:47.210
+I'll just do quickly and then I'll concentrate
+on the the one
+
+0:00:47.210,0:00:48.010
+piece that
+
+0:00:48.010,0:00:49.720
+seems to be the most popular
+
+0:00:49.720,0:00:51.360
+so the history of BSD
+
+0:00:51.360,0:00:55.750
+This is BSD at Berkeley is supposed
+to one to shout into that
+
+0:00:55.750,0:00:57.050
+open source world
+
+0:00:57.050,0:01:01.810
+So the first period of time is what I call
+the Bill Joy era
+
+0:01:01.810,0:01:06.360
+And this is where Bill Joy is starting up BSD
+ and it's really getting a rolling and
+
+0:01:06.360,0:01:07.180
+so it ends when he
+
+0:01:07.180,0:01:08.370
+departs
+0:01:08.370,0:01:09.610
+and
+0:01:09.610,0:01:13.080
+Sponsers uh.. starts up this company
+called Sun Microsystems which you may have
+
+0:01:13.080,0:01:14.680
+heard of
+
+0:01:14.680,0:01:19.250
+Uh.. Then the second part is one of the heyday
+of BSD. So this is the period of time
+
+0:01:19.250,0:01:20.930
+where everybody is
+
+0:01:20.930,0:01:24.290
+running is forty two and for three and so on
+
+0:01:24.290,0:01:24.960
+and
+
+0:01:24.960,0:01:28.830
+going accurate time we were being funded
+by DARPA which
+
+0:01:28.830,0:01:31.020
+deploy a microphone somewhere in this room
+
+0:01:31.020,0:01:32.950
+Uh at any rate
+
+0:01:32.950,0:01:35.160
+there was some very interesting
+
+0:01:35.160,0:01:36.889
+Uh.. battles that went on
+
+0:01:36.889,0:01:38.630
+Uh.. about
+
+0:01:38.630,0:01:41.110
+exactly how to see or to be implemented
+
+0:01:41.110,0:01:46.180
+and a lot of this at that history all
+that all percolates out
+
+0:01:46.180,0:01:48.000
+and so that's sort of the middle period and
+
+0:01:48.000,0:01:54.000
+and then the end period is where we have
+gotten religion and it decided that
+
+0:01:54.000,0:01:58.130
+you know maybe BSD isnt going be at
+Berkeley forever and if we want there to be
+
+0:01:58.130,0:01:58.960
+any
+
+0:01:58.960,0:02:00.430
+possibility with the continue
+
+0:02:00.430,0:02:04.530
+Uh.. we better figure out how to get it outside
+the university to people that
+
+0:02:04.530,0:02:08.489
+other than to the people that are licensed
+by AT&T Bell Laboratories.
+
+0:02:08.489,0:02:09.970
+And so it's the whole story out
+
+0:02:09.970,0:02:13.859
+how we open-sourced it and fought a one lawsuit
+about that and
+
+0:02:13.859,0:02:15.189
+so on and so forth.
+
+0:02:15.189,0:02:19.049
+So here is the three main parts.. The Bill Joy .. that the middle
+PCP
+
+0:02:19.049,0:02:20.519
+building years
+
+0:02:20.519,0:02:20.940
+and
+
+0:02:20.940,0:02:24.099
+the lawsuit years at the end.
+
+0:02:24.099,0:02:28.879
+So how many people are most interested in hearing
+the Bill joy years
+
+0:02:28.879,0:02:33.999
+one two three four five six seven eight nine
+ten eleven twelve thirteen
+
+0:02:33.999,0:02:35.199
+fourteen fifteen
+
+0:02:35.199,0:02:39.559
+Sixteen. Ok.. how many people are interested
+in hearing through the middle era
+
+0:02:39.559,0:02:44.579
+one two three four five six seven eight nine
+ten eleven twelve thirteen fourteen fifteen
+
+0:02:44.579,0:02:45.320
+Sixteen.. All right..
+
+0:02:45.320,0:02:46.959
+Seventeen eighteen ninety twenty twenty one
+
+0:02:46.959,0:02:49.959
+twenty five OK and how many people want to hear
+the open
+
+0:02:49.959,0:02:51.079
+source
+
+0:02:51.079,0:02:53.159
+Lawsuits et cetera
+
+0:02:53.159,0:02:57.120
+Laugh.... Okkk
+
+0:02:57.120,0:02:58.969
+Do you see this is why I ask
+
+0:02:58.969,0:03:01.139
+This is usual it's interesting 'cause I get
+
+0:03:01.139,0:03:03.499
+different different groups votes for different
+things
+
+0:03:03.499,0:03:06.799
+When I was in Japan they all want to hear
+about the lawsuit.. you know.. thats all they
+
+0:03:06.799,0:03:09.229
+have those in Japan.
+
+0:03:09.229,0:03:11.669
+Yeah that story is quite well known here
+
+0:03:11.669,0:03:14.639
+Infact the so interesting tidbit is that
+
+0:03:14.639,0:03:16.459
+uh threw up
+
+0:03:16.459,0:03:20.289
+Freedom of Information Act California Freedom
+of Information Act
+
+0:03:20.289,0:03:23.489
+uh we were able to get the uh
+
+0:03:23.489,0:03:27.819
+decision on that lawsuit unsealed so that
+the actual text could get out which was
+
+0:03:27.819,0:03:30.319
+sort of interesting reading.
+
+0:03:30.319,0:03:34.349
+Okay, for those of you who are interested in
+a section that I'm not doing
+
+0:03:34.349,0:03:38.050
+I advice to buy them to make a shameless plug for
+my DVD.
+
+0:03:38.050,0:03:41.709
+This is the entire three-and-a-half hour version
+of this lecture
+
+0:03:41.709,0:03:45.949
+I wish I could conveniently have a box up your surf
+if it's worth twenty dollars to you could
+
+0:03:45.949,0:03:52.219
+got one of these and walk away with it.
+
+0:03:52.219,0:03:57.119
+Ok. So uh unlike most of my talks and I dont actually have a
+laptop with slides on it and I just have these
+
+0:03:57.119,0:03:59.529
+handwritten notes
+
+0:03:59.529,0:04:01.680
+and the top line of my handwritten note now it says
+
+0:04:01.680,0:04:06.190
+the first four pages of this were written
+on the Indian Pacific train on way to Greater Perth
+
+0:04:06.190,0:04:12.629
+in January 1986 because I had
+to give a lecture on the day I arrived.
+
+0:04:12.629,0:04:14.219
+So I have been uh told
+
+0:04:14.219,0:04:18.100
+ that this was what my talk was going
+to be and I was unprepared so
+
+0:04:18.100,0:04:23.259
+it's what a bouncy up and down the handwriting
+but it was the early history
+
+0:04:23.259,0:04:24.840
+so anyway
+
+0:04:24.840,0:04:25.880
+we're just gonna zip
+
+0:04:25.880,0:04:27.029
+quickly through
+
+0:04:27.029,0:04:29.750
+what BSD started at Berkeley
+
+0:04:29.750,0:04:31.150
+Uh by Bill Joy
+
+0:04:31.150,0:04:34.120
+uh Bill Joy was a graduate student I
+happen to
+
+0:04:34.120,0:04:38.050
+share the same adviser with him, we were both
+working on programming languages.
+
+0:04:38.050,0:04:41.610
+Uh you might find that hard to believe considering
+that he did the seashell but
+
+0:04:41.610,0:04:44.539
+Uh.. laugh
+0:04:44.539,0:04:48.000
+at any rate up that's what we're doing and
+
+0:04:48.000,0:04:50.740
+So I shared an office with him and along with
+
+0:04:50.740,0:04:53.720
+three other graduate students uh
+
+0:04:53.720,0:04:56.120
+and
+
+0:04:56.120,0:05:00.060
+So uh I sort of was sitting there watching a lot of
+this unfolding and in fact Bill was very good
+
+0:05:00.060,0:05:03.080
+at getting other people enthusiastic about
+doing work.
+
+0:05:03.080,0:05:06.849
+There was his typical comment was.. how hard
+can it be, you could just knock this off in
+
+0:05:06.849,0:05:08.869
+a couple hours
+
+0:05:08.869,0:05:12.709
+and usually you could knock off one thing
+but of course that with can you sucked
+
+0:05:12.709,0:05:16.060
+in and you know you end up writing the entire pascal
+interpreter of
+
+0:05:16.060,0:05:18.039
+the back and or something
+
+0:05:18.039,0:05:20.249
+At any rate
+
+0:05:20.249,0:05:24.749
+Bill started doing initially just utilities
+so was the seashell
+
+0:05:24.749,0:05:27.439
+uh that ex editor later
+
+0:05:27.439,0:05:29.750
+it also became the vi editor
+
+0:05:29.750,0:05:31.490
+the pascal system
+
+0:05:31.490,0:05:37.080
+and say start packaging these things out and
+sending out tapes to people and
+
+0:05:37.080,0:05:38.050
+then when
+
+0:05:38.050,0:05:39.930
+Ken Thompson who would
+
+0:05:39.930,0:05:42.099
+in an Alumnus of Berkeley
+
+0:05:42.099,0:05:45.610
+on without for one year's sabbatical and brought
+UNIX with him
+
+0:05:45.610,0:05:49.909
+when he left Bill sort of took over running
+the system what we would call assistant administrator
+
+0:05:49.909,0:05:51.729
+today
+
+0:05:51.729,0:05:55.660
+Uh at any rate this got him interested in this the system
+There were patches that were coming back
+
+0:05:55.660,0:05:58.550
+from AT&T he was putting those in and he was then
+
+0:05:58.550,0:06:01.590
+making some other sending them back to AT&T
+and he
+
+0:06:01.590,0:06:01.930
+and
+
+0:06:01.930,0:06:03.729
+ so
+
+0:06:03.729,0:06:07.770
+over time there were bigger and bigger bits of
+the system and finally got tired of trying
+
+0:06:07.770,0:06:08.370
+to
+
+0:06:08.370,0:06:10.909
+track what other people's UNIX is look
+like
+
+0:06:10.909,0:06:11.729
+and so
+
+0:06:11.729,0:06:16.569
+uh with the advance of 3 BSD the
+third distribution
+
+0:06:16.569,0:06:18.560
+He just put together an entire package
+
+0:06:18.560,0:06:22.310
+uh and this was actually to the VAX rather
+than the PDP-11 which is what we've started
+
+0:06:22.310,0:06:23.080
+on
+
+0:06:23.080,0:06:25.069
+all the two dot
+
+0:06:25.069,0:06:27.310
+versions of the of BSD
+
+0:06:27.310,0:06:30.889
+nicely worked on by the other people not by
+the Bill especially not the stuff where
+
+0:06:30.889,0:06:33.050
+the kernel begins to become involved
+
+0:06:33.050,0:06:35.300
+2.9, 10, 11 et cetera
+
+0:06:35.300,0:06:37.569
+uh was actually done by
+
+0:06:37.569,0:06:41.090
+other people many years uh later so 2.11
+
+0:06:41.090,0:06:43.279
+does not preceed 3.0
+
+0:06:43.279,0:06:48.550
+Uh 2.11 is you know contemporary
+with 4.4
+
+0:06:48.550,0:06:51.100
+Uh at any rate uh
+
+0:06:51.100,0:06:52.800
+once we got the
+
+0:06:52.800,0:06:54.790
+uh the BSD
+
+0:06:54.790,0:06:57.360
+distribution out into the world
+
+0:06:57.360,0:07:00.870
+Uh it started getting used by a lot of people
+because it
+
+0:07:00.870,0:07:01.589
+really
+
+0:07:01.589,0:07:05.000
+was a big step forward over what AT&T were
+shipping
+
+0:07:05.000,0:07:06.660
+in particular
+
+0:07:06.660,0:07:09.529
+uh as we start getting things like
+
+0:07:09.529,0:07:11.450
+the TCP/IP networking
+
+0:07:11.450,0:07:15.770
+Uh So you had a choice you can run a AT& T
+version which had UUCP as networking
+
+0:07:15.770,0:07:19.149
+or you could run BSD that had sockets et
+ cetera
+
+0:07:19.149,0:07:20.610
+you could run
+
+0:07:20.610,0:07:24.090
+AT&T's version which had a file system that
+could utilize 3 to 5 percent of the bandwidth
+
+0:07:24.090,0:07:24.979
+of the disk
+
+0:07:24.979,0:07:28.810
+or you could run BSD which could utilize 50 percent
+of the bandwidth of the disk
+
+0:07:28.810,0:07:30.519
+et cetera.
+
+0:07:30.519,0:07:34.939
+So many people would buy the AT&T license
+ and immediately drop Berkeley to add UNIX
+
+0:07:34.939,0:07:37.180
+on to it.
+
+0:07:37.180,0:07:41.879
+Uh consequently it was getting wide distribution
+atleast in the university environments
+
+0:07:41.879,0:07:44.139
+Uh
+
+0:07:44.139,0:07:46.979
+And uh starting this way he got into the commercial
+environments
+
+0:07:46.979,0:07:51.249
+The commercial environments were not large because
+commercial environments just werent using UNIX
+
+0:07:51.249,0:07:52.679
+at that time
+
+0:07:52.679,0:07:54.679
+for the most part
+
+0:07:54.679,0:07:58.090
+Uh at any rate
+
+0:07:58.090,0:07:59.290
+Bill got it
+
+0:07:59.290,0:08:00.349
+Uh well
+
+0:08:00.349,0:08:04.299
+with 3 BSD coming out uh
+
+0:08:04.299,0:08:06.340
+The DARPA which is
+
+0:08:06.340,0:08:07.540
+essentially
+
+0:08:07.540,0:08:10.160
+put out research contracts for
+
+0:08:10.160,0:08:12.629
+all the different branches of the military
+
+0:08:12.629,0:08:13.780
+Uh was
+
+0:08:13.780,0:08:15.960
+had a huge number of contracts
+
+0:08:15.960,0:08:19.850
+uh with lots of different people being stuff
+related computers all have different computers
+
+0:08:19.850,0:08:22.939
+and operating systems and maybe using some different
+languages
+
+0:08:22.939,0:08:24.969
+and so if one group would do something
+
+0:08:24.969,0:08:27.309
+they just couldn't really pass that
+
+0:08:27.309,0:08:30.739
+over take some other group that might want
+to utilize it because this was written in
+
+0:08:30.739,0:08:33.390
+FORTRAN and that was written in Pearl 1
+
+0:08:33.390,0:08:35.830
+You know this was running on
+
+0:08:35.830,0:08:40.830
+you know some deck machine that was running on some
+IBM's hardware et cetera.
+
+0:08:40.830,0:08:47.220
+and so DARPA decided that they
+really wanted to consolidate
+
+0:08:47.220,0:08:50.280
+on hardware and operating system
+
+0:08:50.280,0:08:55.000
+and then how people work there and so there's a
+better chance if they would be able to co-operate.
+
+0:08:55.000,0:08:57.640
+So they put out a a sort of
+
+0:08:57.640,0:08:59.829
+call sends a request for
+
+0:08:59.829,0:09:02.430
+proposals or whatever they called in those
+days
+
+0:09:02.430,0:09:04.290
+and
+
+0:09:04.290,0:09:05.520
+essentially there were two
+
+0:09:05.520,0:09:08.500
+uh main contending things that came in
+
+0:09:08.500,0:09:10.029
+one was the VAX
+
+0:09:10.029,0:09:11.320
+running VMX
+
+0:09:11.320,0:09:14.300
+and the other one was the VAX running
+
+0:09:14.300,0:09:16.800
+Berkeley UNIX
+
+0:09:16.800,0:09:18.150
+and of course
+
+0:09:18.150,0:09:21.650
+the uh the people that were promoting the
+VAX running VMS
+
+0:09:21.650,0:09:27.220
+were arguing like uh you know this is a real
+vendor supported operating system and
+
+0:09:27.220,0:09:32.059
+you know that this not just some flaky students
+at Berkeley that are doing it
+
+0:09:32.059,0:09:34.530
+then you know therefore there's sort of the operating
+system
+
+0:09:34.530,0:09:39.500
+And that really put the final nail
+on the coffin
+
+0:09:39.500,0:09:43.410
+Uh this time they encased and came up with a set of
+benchmarks
+
+0:09:43.410,0:09:47.370
+Uh what we would call micro benchmarks
+today so how fast can you do get a PID
+
+0:09:47.370,0:09:49.970
+and how fast you could pipe thing
+
+0:09:49.970,0:09:54.180
+When you pack it back and forth to see
+up that certain context which is so on.
+
+0:09:54.180,0:09:55.410
+and
+
+0:09:55.410,0:09:56.710
+uh
+
+0:09:56.710,0:10:00.910
+showed that we must have did these things much faster
+than BSD did.
+
+0:10:00.910,0:10:01.760
+and this of course
+
+0:10:01.760,0:10:03.680
+made Bill go ballistic
+
+0:10:03.680,0:10:04.360
+and
+
+0:10:04.360,0:10:07.780
+Uh besides saying that they were as stupid as benchmarks
+ he'd ever since have zipped that's the way
+
+0:10:07.780,0:10:11.070
+they're gonna decide then we'll just go
+make this thing run fast
+
+0:10:11.070,0:10:14.920
+and it was sort of an interesting story about
+other half and what
+
+0:10:14.920,0:10:19.980
+he got uh the BSD to run those benchmarks as fast
+as the VMS
+
+0:10:19.980,0:10:21.450
+and
+
+0:10:21.450,0:10:22.170
+so
+
+0:10:22.170,0:10:24.900
+the upshot of it was that DARPA
+
+0:10:24.900,0:10:28.740
+was still sort of sitting on the fence
+and Bills comment was well VMS will stock
+
+0:10:28.740,0:10:30.380
+with VMX forever
+
+0:10:30.380,0:10:34.490
+and you know may be some other computer might
+be interesting down the road
+
+0:10:34.490,0:10:40.700
+and BSD is set up so that it's much more portable
+uh you know it's tie. UNIX, in general is
+
+0:10:40.700,0:10:44.750
+the portable across different architectures we've
+already proved it by running it for any cost too
+
+0:10:44.750,0:10:45.910
+so therefore
+
+0:10:45.910,0:10:48.010
+it's proved point
+
+0:10:48.010,0:10:49.040
+so
+
+0:10:49.040,0:10:51.850
+at any rate DARPA agreed to that
+
+0:10:51.850,0:10:54.780
+and they put in some initial money to Berkeley
+
+0:10:54.780,0:10:57.230
+and the initial things that Berkley was
+really just
+
+0:10:57.230,0:11:00.720
+uh to come up with
+
+0:11:00.720,0:11:02.500
+Uh well we've put out 3 BSD
+
+0:11:02.500,0:11:05.950
+which had a lot of functionality but the
+performance wasn't so great
+
+0:11:05.950,0:11:09.330
+and so the that first point was to get
+4 BSD out
+
+0:11:09.330,0:11:10.790
+which was going to happen
+
+0:11:10.790,0:11:12.870
+sort of that
+
+0:11:12.870,0:11:14.960
+that additional functionality they really felt
+they needed
+
+0:11:14.960,0:11:16.459
+uh which would just is of the
+
+0:11:16.459,0:11:17.930
+really small tweeks
+
+0:11:17.930,0:11:22.510
+Uh things like the ability to automatically
+reboot after crash
+
+0:11:22.510,0:11:23.880
+Uh Job control
+
+0:11:23.880,0:11:29.540
+uh PRAMs list which was created by a bunch
+of their uh folks, deliver mail, smoke in air
+
+0:11:29.540,0:11:32.810
+ that was the uh the predecessor to
+send mail
+
+0:11:32.810,0:11:36.820
+except without a configuration file you just
+complied and check the way he needs things
+
+0:11:36.820,0:11:37.700
+that were
+
+0:11:37.700,0:11:39.720
+uh at any rate
+
+0:11:39.720,0:11:41.260
+Uh that came out
+
+0:11:41.260,0:11:42.800
+and that 4.1
+
+0:11:42.800,0:11:44.550
+so it nail that down cleaned it up
+
+0:11:44.550,0:11:46.590
+got the performance decent
+
+0:11:46.590,0:11:49.680
+and that fulfilled the first DARPA contract.
+
+0:11:49.680,0:11:52.450
+DARPA had put in something like
+
+0:11:52.450,0:11:56.390
+you know three-quarters of a million dollars or something
+and boomed topped this huge pay back from it
+
+0:11:56.390,0:11:56.960
+and
+
+0:11:56.960,0:11:58.380
+people seem to be happy with it
+
+0:11:58.380,0:12:02.880
+so they decided that maybe that the funky graduate students
+at Berkeley could do stuff
+
+0:12:02.880,0:12:05.640
+and so then they put in that of
+
+0:12:05.640,0:12:07.310
+considerably larger
+
+0:12:07.310,0:12:10.710
+contract it was I don't know what three or
+five million dollars
+
+0:12:10.710,0:12:12.260
+on this was a longer term
+
+0:12:12.260,0:12:13.520
+he knows he wouldve at
+
+0:12:13.520,0:12:15.870
+ those times he would get
+
+0:12:15.870,0:12:17.839
+for example a three-year contract
+
+0:12:17.839,0:12:20.320
+of which is sort of unheard of in lot of a
+
+0:12:20.320,0:12:22.440
+research stuff today.
+
+0:12:22.440,0:12:25.630
+So you could really sort of spent some time
+figuring out what you can do and do it
+
+0:12:25.630,0:12:28.789
+for eighteen months to two years before you had
+to start working about
+
+0:12:28.789,0:12:31.030
+the new assignment et cetera.
+
+0:12:31.030,0:12:33.180
+Uh in any way
+
+0:12:33.180,0:12:34.930
+the grand goals for this
+
+0:12:34.930,0:12:37.300
+Big hunt that DARPA gave
+
+0:12:37.300,0:12:38.800
+was
+
+0:12:38.800,0:12:41.470
+Uh there were four pieces. One was
+
+0:12:41.470,0:12:42.779
+to get a
+
+0:12:42.779,0:12:43.880
+real networking end
+
+0:12:43.880,0:12:46.220
+uh at that time
+
+0:12:46.220,0:12:47.980
+Uh there was just NCP
+
+0:12:47.980,0:12:49.680
+Uh which was that
+
+0:12:49.680,0:12:54.240
+Network Control Protocol predecessor to TCP/IP
+
+0:12:54.240,0:12:57.100
+Uh a good file system in their
+
+0:12:57.100,0:13:01.490
+Uh restart able signal so you could actually
+use them to drive work as opposed to just
+
+0:13:01.490,0:13:03.440
+terminating programs
+
+0:13:03.440,0:13:05.600
+and a new virtual memory system
+
+0:13:05.600,0:13:07.390
+Uh that wasn't
+
+0:13:07.390,0:13:09.810
+well essentially that allowed shared memory
+
+0:13:09.810,0:13:11.530
+uh that the one
+
+0:13:11.530,0:13:13.640
+the predecessor
+
+0:13:13.640,0:13:15.880
+that we were running BSD at that
+time
+
+0:13:15.880,0:13:19.810
+shared the text space that would be only the part
+of your program but all the rest
+
+0:13:19.810,0:13:21.580
+could not shared
+
+0:13:21.580,0:13:23.810
+So therere the four deliverables.
+
+0:13:23.810,0:13:25.980
+now for the kind of networking
+
+0:13:25.980,0:13:28.000
+the students really trust us.
+
+0:13:28.000,0:13:30.879
+So they decided that what they were going
+to do
+
+0:13:30.879,0:13:33.720
+was divide into two parts.
+
+0:13:33.720,0:13:35.300
+So they were going to get
+
+0:13:35.300,0:13:37.630
+ Bolt, Beranek and Newman BBN
+
+0:13:37.630,0:13:42.360
+we are going to actually write the TCP/IP protocols
+stuff
+
+0:13:42.360,0:13:44.709
+and What Berkley was supposed to do
+
+0:13:44.709,0:13:46.589
+was designed the
+
+0:13:46.589,0:13:47.550
+programming interface
+
+0:13:47.550,0:13:49.040
+uh so
+
+0:13:49.040,0:13:53.130
+the socket connect except so on
+
+0:13:53.130,0:13:55.639
+and then VB9 was to deliver
+
+0:13:55.639,0:14:00.159
+the thing plug-in underneath that interface
+and Berkley was to integrated into
+
+0:14:00.159,0:14:02.150
+BSD and uh
+
+0:14:02.150,0:14:04.570
+they would ship it.
+
+0:14:04.570,0:14:05.239
+and
+
+0:14:05.239,0:14:08.090
+Uh so
+
+0:14:08.090,0:14:11.240
+this all started on and VB9 gets to work
+
+0:14:11.240,0:14:13.090
+and
+
+0:14:13.090,0:14:15.750
+Joy gets to work on the interface and
+
+0:14:15.750,0:14:18.620
+uh so
+
+0:14:18.620,0:14:22.610
+eventually what happens is that Bill has sort
+of the framework of this stuff. And Bill being
+
+0:14:22.610,0:14:23.960
+Bill uh
+
+0:14:23.960,0:14:26.580
+he he work rather quickly
+
+0:14:26.580,0:14:29.880
+Uh someone wants to ask you to compare myself
+to build Joy and I said look
+
+0:14:29.880,0:14:33.980
+there's really nothing that he's done that
+I couldn't do but the problem is
+
+0:14:33.980,0:14:36.810
+what he gets done in a year would take me
+a decade
+
+0:14:36.810,0:14:39.430
+because he was the master of figuring out.
+
+0:14:39.430,0:14:40.730
+I'm here
+
+0:14:40.730,0:14:42.039
+that's where I want to get to
+
+0:14:42.039,0:14:44.740
+what is absolutely shortest half from here
+to there
+
+0:14:44.740,0:14:45.810
+and he would do it
+
+0:14:45.810,0:14:47.339
+and you end up with code
+
+0:14:47.339,0:14:48.260
+that worked but
+
+0:14:48.260,0:14:51.699
+that was not maintainable, expendable, changeable.
+
+0:14:51.699,0:14:56.130
+Anybody looked at the old version of the VI
+understand what Im talking about.
+
+0:14:56.130,0:14:56.860
+so
+
+0:14:56.860,0:14:58.430
+So some of the rest of us you know
+
+0:14:58.430,0:14:59.709
+like to have stuff that like
+
+0:14:59.709,0:15:02.910
+you could go back and do something with later
+but that takes longer
+
+0:15:02.910,0:15:03.689
+In any way
+
+0:15:03.689,0:15:06.300
+Bill figured out this socket interface the first
+
+0:15:06.300,0:15:07.930
+draft away it anyway
+
+0:15:07.930,0:15:12.140
+and he's got it and now he needs the newer call protocols
+to put underneath it
+
+0:15:12.140,0:15:15.480
+so he goes to to
+
+0:15:15.480,0:15:16.399
+Uh Rob Gurvitz who was
+
+0:15:16.399,0:15:18.259
+leading the program of
+
+0:15:18.259,0:15:20.429
+development of TCP/IP at PBN
+
+0:15:20.429,0:15:23.660
+and says hey I got a socket interface and I need something to put
+in here
+
+0:15:23.660,0:15:28.140
+and Rob says well we're not really done yet
+but we have we have a slight prototype code
+
+0:15:28.140,0:15:29.990
+and you know you could put that in it
+
+0:15:29.990,0:15:31.319
+see how that works in
+
+0:15:31.319,0:15:33.100
+you know then well iterate hopefully the race
+
+0:15:33.100,0:15:35.810
+up so that we can work in parallel of you.
+
+0:15:35.810,0:15:38.410
+So Bill gets the code and
+
+0:15:38.410,0:15:40.420
+he puts it in
+
+0:15:40.420,0:15:40.859
+and
+
+0:15:40.859,0:15:42.979
+cranks it up and gets a kind of working
+
+0:15:42.979,0:15:43.820
+and
+
+0:15:43.820,0:15:45.580
+then decides to run
+
+0:15:45.580,0:15:49.110
+Some full test between machines to see
+how well it works.
+
+0:15:49.110,0:15:55.150
+And were running at the at that time our development
+machines development inputs were
+
+0:15:55.150,0:15:55.820
+VAC 750s
+0:15:55.820,0:15:59.640
+so that was the big honk at VAC 780
+that ran entire NIP
+
+0:15:59.640,0:16:04.600
+and then there were these much smaller things
+that were merely the size of a large
+
+0:16:04.600,0:16:05.769
+uh dishwasher
+
+0:16:05.769,0:16:11.839
+uh its bigger than a dishwasher more like
+washing machine yes
+its over range at any rate and used about
+
+0:16:11.839,0:16:13.660
+as much power as the same.
+
+0:16:13.660,0:16:16.260
+Uh with all the burners turned ON assuming its electric
+
+0:16:16.260,0:16:23.080
+uh anyway and the VAC 750 only
+ran at 0.7 NIPs
+
+0:16:23.080,0:16:25.480
+Uh and so
+
+0:16:25.480,0:16:28.920
+Bill needs to be overrun sometimes back and forth
+between these machines
+
+0:16:28.920,0:16:31.080
+and of course we don't have
+
+0:16:31.080,0:16:34.900
+anything like FTP or TELNET
+anything that, which is not written yet
+
+0:16:34.900,0:16:36.890
+and so he just
+
+0:16:36.890,0:16:38.369
+being Bill as well
+
+0:16:38.369,0:16:39.699
+hack up something
+
+0:16:39.699,0:16:43.309
+uh which was all logged in RCP et cetera.
+
+0:16:43.309,0:16:47.820
+which was really just the design so that he
+could test running between these two machines
+
+0:16:47.820,0:16:48.710
+Uh
+
+0:16:48.710,0:16:52.130
+This is the legacy that we end up with and
+why the protocol number or anything like
+
+0:16:52.130,0:16:57.820
+that in the header you know its just a
+test program no one is gonna actually use this
+
+0:16:57.820,0:17:02.020
+so the uh upshot of all this that
+
+0:17:02.020,0:17:03.840
+Bill fires up
+
+0:17:03.840,0:17:05.530
+RCP and
+
+0:17:05.530,0:17:08.230
+to copy a file between machines
+
+0:17:08.230,0:17:08.730
+and
+
+0:17:08.730,0:17:15.300
+and it copies a file at about 50..6 or
+60 kilobits per second
+
+0:17:15.300,0:17:17.520
+and at that point the CPU is PEC
+
+0:17:17.520,0:17:19.799
+on the VAX 750
+
+0:17:19.799,0:17:20.810
+And it you know
+
+0:17:20.810,0:17:24.810
+there's plenty of bandwidth still available
+because we had 10 megabit Ethernet at that point
+
+0:17:24.810,0:17:28.519
+we had the new 10 megabit Ethernet because mostly
+ it was 3 megabit but
+
+0:17:28.519,0:17:31.059
+we had a 10 which was like really cool stuff.
+
+0:17:31.059,0:17:32.260
+uh and
+
+0:17:32.260,0:17:37.090
+uh so he talks to her rocker you know this thing runs
+and girl was his attitude
+
+0:17:37.090,0:17:39.280
+as well the backbone
+
+0:17:39.280,0:17:44.540
+of the only the 56 K so as long as
+we can saturate that what's the problem
+
+0:17:44.540,0:17:48.710
+and Bills attitude is well know I need to be
+ all you know get closer to the bandwidth of
+
+0:17:48.710,0:17:51.450
+the network between these local machines
+
+0:17:51.450,0:17:56.430
+And so of course Bill being Bill
+dives in and see why this is the case that it's not running very
+
+0:17:56.430,0:17:57.440
+fast.
+
+0:17:57.440,0:17:58.500
+Infact DRAPA
+
+0:17:58.500,0:17:59.880
+ wrote to
+
+0:17:59.880,0:18:01.680
+to some of this stuff to figure out
+
+0:18:01.680,0:18:06.320
+what was not running quickly. We already
+had it for usual of our programs thats what we
+
+0:18:06.320,0:18:09.170
+put it into the kernel
+
+0:18:09.170,0:18:12.160
+And said to say some of these tools are still like
+considered
+
+0:18:12.160,0:18:17.050
+like leading tools but are not leading tools but
+still used tools today
+
+0:18:17.050,0:18:22.800
+Uh which isn't to say bad that it's just that
+seems like in twenty years you could do better
+
+0:18:22.800,0:18:23.690
+at any rate
+
+0:18:23.690,0:18:27.490
+he quickly discovers that way the TCP/IP
+code is written
+
+0:18:27.490,0:18:30.000
+is its this elaborate state machine
+
+0:18:30.000,0:18:32.970
+and there's the state transitions that you
+go through and
+
+0:18:32.970,0:18:36.919
+each time we want to a state transition the
+you jump through an indirect
+
+0:18:36.919,0:18:39.090
+this and that and other things and
+
+0:18:39.090,0:18:42.520
+and he goes well you know this is just great
+and efficient and you know
+
+0:18:42.520,0:18:46.620
+we could just take this machine turned
+into a giant switch statement
+
+0:18:46.620,0:18:49.860
+And you know if you're going to see a red area
+know to get from here to there are going to
+
+0:18:49.860,0:18:53.070
+need to switch states buttoning ON and thats that
+
+0:18:53.070,0:18:54.380
+We'll just do that
+
+0:18:54.380,0:18:57.770
+And so he starts crank on his way
+and
+
+0:18:57.770,0:18:59.230
+pounding out all of that
+
+0:18:59.230,0:19:01.680
+the structure and getting a down
+
+0:19:01.680,0:19:04.110
+of what it looks like today
+
+0:19:04.110,0:19:06.200
+And uh
+
+0:19:06.200,0:19:09.390
+that's the spin cleaned up considerably over
+the years but
+
+0:19:09.390,0:19:12.280
+Uh so he cranks it down to this and
+
+0:19:12.280,0:19:14.770
+gets up to the point where
+
+0:19:14.770,0:19:19.420
+he can at least saturate that 3 megabit wire
+we cant saturate the
+
+0:19:19.420,0:19:21.960
+10 megabit wire but it's because the
+
+0:19:21.960,0:19:24.640
+actually Ethernet controllers are which themselves are
+
+0:19:24.640,0:19:26.740
+not able to uh
+
+0:19:26.740,0:19:30.310
+really run at 10 megabits uh
+
+0:19:30.310,0:19:33.330
+these were you know prototype early release
+stuff and so
+
+0:19:33.330,0:19:35.020
+we get it upto
+
+0:19:35.020,0:19:36.960
+somewhere around four
+
+0:19:36.960,0:19:39.370
+megabit uh
+
+0:19:39.370,0:19:43.370
+But we're only looking at 40% CPU time
+so in theory if we could have run at wires
+
+0:19:43.370,0:19:46.809
+speed we got up to 10 megabit
+and for other reasons it wouldn't have
+
+0:19:46.809,0:19:47.350
+but
+
+0:19:47.350,0:19:53.760
+the point is we were like uh a way way over of magnitude
+ahead of where we started from later of
+
+0:19:53.760,0:19:56.350
+and so
+
+0:19:56.350,0:19:59.480
+Uh there's a lot of pressure from a lot of the
+
+0:19:59.480,0:20:02.950
+uh the people that have been that have 4.1
+
+0:20:02.950,0:20:04.640
+that they want to
+
+0:20:04.640,0:20:07.320
+try out this networking stuff
+
+0:20:07.320,0:20:10.080
+and so uh
+
+0:20:10.080,0:20:11.870
+this code
+
+0:20:11.870,0:20:13.210
+as such as it was
+
+0:20:13.210,0:20:17.280
+uh got released as something that we call
+4.1a
+
+0:20:17.280,0:20:20.680
+Uh so 4.1a was sort of the
+
+0:20:20.680,0:20:22.559
+first intermediate release of
+
+0:20:22.559,0:20:26.020
+Uh between 4.1 and 4.2
+
+0:20:26.020,0:20:28.860
+and this got surprisingly large
+
+0:20:28.860,0:20:32.890
+larger distribution then we have really planned
+on because you know you give it to one person
+
+0:20:32.890,0:20:38.010
+and then someone else hears about it. May want it
+then someone else wants it nahnah.....
+0:20:38.010,0:20:42.030
+It's not like today and where you just put it up
+ just anonymous FTP. This was still
+
+0:20:42.030,0:20:45.340
+put it on a tape Mule tape somebody big
+
+0:20:45.340,0:20:46.640
+nine track tapes
+
+0:20:46.640,0:20:48.110
+uh
+
+0:20:48.110,0:20:52.900
+And uh if you'd want lucky enough to have a high
+tape drive 6250
+
+0:20:52.900,0:20:55.090
+Which for inchtape drive then you had to get
+
+0:20:55.090,0:20:56.430
+the 1604
+
+0:20:56.430,0:20:59.360
+ tapes together all out there
+
+0:20:59.360,0:21:01.660
+Uh at any rate
+
+0:21:01.660,0:21:03.820
+This this code goes out
+
+0:21:03.820,0:21:04.299
+Uh and uh
+
+0:21:04.299,0:21:10.039
+measured about this time Sam Leffler
+somebody's name probably know in contemporary
+
+0:21:10.039,0:21:11.800
+stuff having to do with wireless
+
+0:21:11.800,0:21:14.000
+Uh I actually joined
+
+0:21:14.000,0:21:15.710
+the Source Forge group there
+
+0:21:15.710,0:21:17.340
+uh and
+
+0:21:17.340,0:21:19.530
+he begins to help build with
+
+0:21:19.530,0:21:21.669
+a rewrite of the networking interface
+
+0:21:21.669,0:21:23.020
+on in particular
+
+0:21:23.020,0:21:27.210
+uh Bill hadn't quiet hard on to the
+fact that
+
+0:21:27.210,0:21:30.320
+uh When you may want to have multiple connections
+
+0:21:30.320,0:21:33.970
+concurrently on a particular port
+so if you're running some TP
+
+0:21:33.970,0:21:35.720
+server of HTTP
+
+0:21:35.720,0:21:38.640
+Uh the way he had written had a except
+
+0:21:38.640,0:21:42.780
+that the that the what we think of surround view of
+the socket today was actually connected
+
+0:21:42.780,0:21:46.350
+and so that nobody else to talk to you on
+that particular port to you. When youre
+
+0:21:46.350,0:21:48.170
+finished with that conversation and then
+
+0:21:48.170,0:21:51.520
+that the sock would become free again and you
+couldve the next connection.
+
+0:21:51.520,0:21:52.759
+he says like that no
+
+0:21:52.759,0:21:54.389
+no we got it like
+0:21:54.389,0:21:56.500
+to have more than one at a time
+
+0:21:56.500,0:22:01.510
+and so that's when the whole listen and the
+rendezvous socket works up returns to a new
+
+0:22:01.510,0:22:03.670
+connected socket and so on
+
+0:22:03.670,0:22:06.960
+uh and so at any rate
+
+0:22:06.960,0:22:08.900
+this this all it gets done
+
+0:22:08.900,0:22:10.970
+And uh
+
+0:22:10.970,0:22:15.930
+I've gotten ropped into doing the files system
+which is another part of an earlier story.
+
+0:22:15.930,0:22:20.130
+uh and so the file system is sort of the
+point where we're ready to inflicted I mean
+
+0:22:20.130,0:22:22.120
+let other users try it
+
+0:22:22.120,0:22:24.480
+and we have revise networking
+
+0:22:24.480,0:22:25.600
+and so this
+
+0:22:25.600,0:22:26.460
+Uh came up
+
+0:22:26.460,0:22:28.960
+as a release called 4.1b
+
+0:22:28.960,0:22:33.390
+and then fairly quickly thereafter the signal
+stuff got cleaned up
+
+0:22:33.390,0:22:36.670
+so 4.1b got replaced to 4.1c
+
+0:22:36.670,0:22:40.450
+Uh remember there were four things that we need
+to get done so A was the networking B was the file system
+
+0:22:40.450,0:22:42.309
+and C was the signalling and
+
+0:22:42.309,0:22:43.920
+D was supposed to be
+
+0:22:43.920,0:22:47.880
+the virtual memory which of course did happen
+for another six or seven years
+
+0:22:47.880,0:22:49.190
+Uh but
+
+0:22:49.190,0:22:52.390
+at any rate 4.1c was another one
+of these ones
+
+0:22:52.390,0:22:54.600
+which was an enormously picked up
+
+0:22:54.600,0:22:56.300
+very wide distribution
+
+0:22:56.300,0:22:57.860
+uh and in
+
+0:22:57.860,0:23:01.110
+in the meantime
+
+0:23:01.110,0:23:05.640
+Uh you know the folks at the PBNF continues to
+work and developing and
+
+0:23:05.640,0:23:06.570
+polishing their code
+
+0:23:06.570,0:23:09.720
+uh but Bill had asked another copy
+of it so
+
+0:23:09.720,0:23:12.570
+they hadnt delivered it because it
+wasnt done yet any way
+
+0:23:12.570,0:23:15.370
+uh and theyre just get it done
+
+0:23:15.370,0:23:18.450
+then give it to Berkley we could put it
+in this
+
+0:23:18.450,0:23:20.790
+preparation for shipping 4.2
+
+0:23:20.790,0:23:24.090
+Well 4.1c got out there
+
+0:23:24.090,0:23:25.610
+and uh
+
+0:23:25.610,0:23:29.760
+Bill then was setting about to work on
+the VM stuff
+
+0:23:29.760,0:23:34.260
+When one day going to his office to
+Chat with him. He says, Come into my office and you know I
+
+0:23:34.260,0:23:37.840
+ I am
+
+0:23:37.840,0:23:41.870
+involved with these other folks down at Stanford
+and they have some hardware
+
+0:23:41.870,0:23:42.860
+and they want
+
+0:23:42.860,0:23:45.020
+me to put some Berkley UNIX on it
+
+0:23:45.020,0:23:46.020
+and you know
+
+0:23:46.020,0:23:48.770
+starts selling these things as workstations
+
+0:23:48.770,0:23:49.889
+and uh
+
+0:23:49.889,0:23:52.500
+Im going I gonna go
+
+0:23:52.500,0:23:56.430
+I am going down there and you know that one of the
+founding members of this and you should come
+
+0:23:56.430,0:23:57.230
+along
+
+0:23:57.230,0:23:58.750
+uh because you know
+
+0:23:58.750,0:24:03.989
+you being a single digit
+employee at workstations.
+
+0:24:03.989,0:24:06.559
+Youll get huge stock option
+its gonna be really exciting and you know
+
+0:24:06.559,0:24:11.450
+I at that time I had already completed my degree
+in business from
+
+0:24:11.450,0:24:12.580
+Uh University California so
+
+0:24:12.580,0:24:14.710
+I knew something about business
+
+0:24:14.710,0:24:18.460
+in fact I've taken an entrepreneurial course
+so I knew a lot about starting up companies
+
+0:24:18.460,0:24:19.830
+and selling back.
+
+0:24:19.830,0:24:23.230
+And so I said, Well you know Bill
+
+0:24:23.230,0:24:27.750
+this all sounds good what you know what what
+is thats going to make you so successful.
+
+0:24:27.750,0:24:31.480
+He says, All those facts that you know we're
+not going to vendor lock in and you know it's going
+
+0:24:31.480,0:24:33.190
+to be running the UNIX and
+
+0:24:33.190,0:24:38.000
+if SUN doesnt do it right they
+can always go to another vendor and can get UNIX from now.
+
+0:24:38.000,0:24:40.730
+and uh thats that's going to be the story
+
+0:24:40.730,0:24:43.320
+And I said, Well you know this is well and good but
+
+0:24:43.320,0:24:46.120
+let me explain about workstations.
+
+0:24:46.120,0:24:50.860
+Uh workstations is about the install base of applications
+as people buy workstations because the applications
+
+0:24:50.860,0:24:52.700
+ run on it
+
+0:24:52.700,0:24:56.810
+And uh you know this is company APPOLO that had
+
+0:24:56.810,0:25:01.220
+a 4 years head start on SUN and they've
+got a lock on the applications
+
+0:25:01.220,0:25:04.860
+and people arent going to switch the SUN because
+they need the applications that run on APPOLO.
+
+0:25:04.860,0:25:06.690
+So you know
+
+0:25:06.690,0:25:09.400
+I'm sure it's going to be interesting but
+uh
+
+0:25:09.400,0:25:13.909
+you know I'm only a few months from finishing
+my PHD in I know I got a SUN with you that
+
+0:25:13.909,0:25:15.249
+won't happen
+
+0:25:15.249,0:25:18.390
+So Im gonna my PhD we'll see how SUN is doing in
+
+0:25:18.390,0:25:20.660
+over six or eight months and you know
+
+0:25:20.660,0:25:22.130
+Well evaluate
+
+0:25:22.130,0:25:26.640
+well of course it took me in and fourteen
+fifteen months to finish my PhD and of
+
+0:25:26.640,0:25:27.740
+course by that time
+
+0:25:27.740,0:25:31.590
+SUN had burst onto the scene and didn't
+have anything interesting stock options
+
+0:25:31.590,0:25:33.910
+left
+
+0:25:33.910,0:25:39.410
+So uh if you want it buy it's about business perhaps
+I'm not the one to ask
+
+0:25:39.410,0:25:41.410
+at any rate
+
+0:25:41.410,0:25:44.420
+Bill left without doing to the VM system
+
+0:25:44.420,0:25:49.500
+Uh and he took 4.1c with him to get it
+ported on to the the SUN hardware.
+
+0:25:49.500,0:25:51.300
+and
+
+0:25:51.300,0:25:54.240
+with this sort uh of hit
+
+0:25:54.240,0:25:55.910
+on the the main developer
+
+0:25:55.910,0:26:00.510
+uh we decided that rather than trying up to get
+someone else up to speed in getting to the
+
+0:26:00.510,0:26:02.710
+VM done and so on.
+
+0:26:02.710,0:26:05.520
+We would instead
+
+0:26:05.520,0:26:09.370
+just declare the first three pieces of this
+4 piece contract
+
+0:26:09.370,0:26:11.580
+uh work released
+
+0:26:11.580,0:26:14.020
+and so
+
+0:26:14.020,0:26:15.720
+uh around this time
+
+0:26:15.720,0:26:18.050
+this be back June of 1983
+
+0:26:18.050,0:26:20.620
+Uh Micheal Karl joins the group
+
+0:26:20.620,0:26:22.590
+essentially coming in uh
+
+0:26:22.590,0:26:25.500
+to replace in into the position that was
+
+0:26:25.500,0:26:27.119
+up formerly held by
+
+0:26:27.119,0:26:29.330
+Sam ..Sam uh out by
+
+0:26:29.330,0:26:32.000
+Bill as that sort of
+
+0:26:32.000,0:26:33.250
+Head of lead programmer
+
+0:26:33.250,0:26:36.510
+uh and
+
+0:26:36.510,0:26:40.490
+because Sam also Sam had also considered taking
+that position but ultimately decided that
+
+0:26:40.490,0:26:44.610
+uh he he didn't want to do that he was gonna
+to go off to do graphic and
+
+0:26:44.610,0:26:46.990
+ looks as well
+
+0:26:46.990,0:26:48.830
+and so
+
+0:26:48.830,0:26:51.030
+the uh
+
+0:26:51.030,0:26:52.980
+the upshot of this was that
+
+0:26:52.980,0:26:55.429
+4.2 got released
+
+0:26:55.429,0:26:57.920
+in about August of 83
+
+0:26:57.920,0:27:01.549
+And uh
+
+0:27:01.549,0:27:05.640
+therere uh about 1000 copies around remember for each
+copy that goes out thats like an institution gets
+
+0:27:05.640,0:27:07.429
+a copy and you know
+
+0:27:07.429,0:27:10.760
+spread across all those machines et cetera.
+
+0:27:10.760,0:27:14.470
+Uh there's another way of looking at it is you can
+look at the number of licenses that AT&T
+
+0:27:14.470,0:27:16.400
+ had outstanding
+
+0:27:16.400,0:27:21.420
+and how many licenses that Berkley had
+
+0:27:21.420,0:27:24.310
+at the time we had shipped a thousand, there were
+about twelve hundred
+
+0:27:24.310,0:27:27.450
+uh AT&T licenses. So it says that
+
+0:27:27.450,0:27:29.149
+fair number of people were running
+
+0:27:29.149,0:27:35.310
+Uh now at this point just for context AT&T is
+releasing system 5 release one
+
+0:27:35.310,0:27:37.380
+So that is short of where they are.
+
+0:27:37.380,0:27:39.290
+Uh They had taken their file system and
+
+0:27:39.290,0:27:44.010
+up the block size from 512 bytes
+ to 1 K
+
+0:27:44.010,0:27:46.750
+Uh at any rate
+
+0:27:46.750,0:27:47.870
+PBNN
+
+0:27:47.870,0:27:50.650
+was very upset that their TCP/IP
+
+0:27:50.650,0:27:53.340
+had not been incorporated.
+
+0:27:53.340,0:27:57.650
+and here we released it but it was this like
+prototype early version had been bastardize
+
+0:27:57.650,0:27:59.100
+by Bill
+
+0:27:59.100,0:28:04.290
+and like you know this is it that you know
+DARPA paid and now you know have the shiny wonderful
+
+0:28:04.290,0:28:05.990
+TCP/IP stack.
+
+0:28:05.990,0:28:09.150
+You should be putting this in
+
+0:28:09.150,0:28:10.369
+and so
+
+0:28:10.369,0:28:15.310
+they began agitating with DARPA and saying
+what you know Blah Blah got this happened
+
+0:28:15.310,0:28:16.280
+and so
+
+0:28:16.280,0:28:20.950
+uh Michael Karl being very deliberative
+what a person he is.
+
+0:28:20.950,0:28:23.330
+Says Ok well
+
+0:28:23.330,0:28:27.780
+I'll take the TCP/IP code from BBN and I'll
+take the TCP code from
+
+0:28:27.780,0:28:28.870
+that we have in 4.2
+
+0:28:28.870,0:28:31.550
+and will evaluate both of them
+
+0:28:31.550,0:28:38.430
+and the evaluation involves you know
+performance and maintenance abilities
+
+0:28:38.430,0:28:41.980
+and functionality and all these various
+necessary things
+
+0:28:41.980,0:28:42.779
+and
+
+0:28:42.779,0:28:47.370
+He says, Well you know the TCP/IP the idea comes
+from BBN and has some really good ideas
+
+0:28:47.370,0:28:49.039
+and we don't have so we'll just
+
+0:28:49.039,0:28:51.330
+Pick those up and drop it in our distribution
+
+0:28:51.330,0:28:55.510
+Uh but we don't really feel that their
+
+0:28:55.510,0:28:57.950
+code is uh you know
+
+0:28:57.950,0:29:01.090
+is the good place to start from. There are lot of people
+this point
+
+0:29:01.090,0:29:05.730
+that have worked with the prickly code
+and they know that code is based on huge amount of enhancements
+
+0:29:05.730,0:29:07.570
+on it.
+
+0:29:07.570,0:29:10.880
+Uh This is wonderful book that's been written
+all about it
+
+0:29:10.880,0:29:13.049
+So it is very well documented
+
+0:29:13.049,0:29:13.500
+and
+
+0:29:13.500,0:29:17.960
+it just doesn't seem to have really make sense
+to
+
+0:29:17.960,0:29:22.460
+reset back to here and then have to get all
+of those improvements changes et cetera.
+
+0:29:22.460,0:29:23.809
+Uh into this
+
+0:29:23.809,0:29:25.280
+particular new version.
+
+0:29:25.280,0:29:27.150
+uh it it
+
+0:29:27.150,0:29:30.440
+I finally graduated in 84 and during
+the project
+
+0:29:30.440,0:29:33.260
+uh foolishly that taking over as
+
+0:29:33.260,0:29:34.170
+it's technically but
+
+0:29:34.170,0:29:35.419
+is the lead
+
+0:29:35.419,0:29:38.960
+which meant that I got to deal with all of
+the political polls showed while
+
+0:29:38.960,0:29:41.870
+Mike got to do lots of programming
+
+0:29:41.870,0:29:45.140
+So let that be a lesson to you that if you think
+that they're trying to sweet talk
+
+0:29:45.140,0:29:50.409
+you and How great it is to be in-charge
+of things.
+
+0:29:50.409,0:29:50.960
+Uh
+
+0:29:50.960,0:29:53.080
+At any rate we
+
+0:29:53.080,0:29:56.680
+continue debating back and forth about this
+stuff with BBN.
+
+0:29:56.680,0:30:00.180
+We kept taking stuff putting it into
+And look you know that we've taken that we've weve taken that
+
+0:30:00.180,0:30:03.720
+No No. You have to take the code
+You got to take the code.
+
+0:30:03.720,0:30:05.440
+So at any rate 4.3
+
+0:30:05.440,0:30:07.750
+was really designed
+
+0:30:07.750,0:30:12.340
+much like 4.1 in the sense of consolidating
+what we already had. So throughout the eh
+
+0:30:12.340,0:30:17.350
+the even-numbered releases from Berkeley had
+a lot of functionality and odd releases
+
+0:30:17.350,0:30:19.930
+tried to really get the performance up
+
+0:30:19.930,0:30:24.170
+Uh we we're always striving to make sure that
+things worked when the released up, but we didn't
+
+0:30:24.170,0:30:24.830
+always
+
+0:30:24.830,0:30:27.169
+strive to make sure they work optimally.
+
+0:30:27.169,0:30:30.090
+Uh figuring that it was better to sort of
+tune things
+
+0:30:30.090,0:30:33.090
+after the fact that once you had a electrode working
+in right way.
+
+0:30:33.090,0:30:37.510
+So anyway we made an announcement
+of 4.3
+
+0:30:37.510,0:30:39.379
+Uh actually at the June 1985
+
+0:30:39.379,0:30:43.510
+it's the fact that two years basically have
+gone by here.
+
+0:30:43.510,0:30:44.510
+ and
+
+0:30:44.510,0:30:45.620
+uh
+
+0:30:45.620,0:30:46.470
+BBN
+
+0:30:46.470,0:30:50.620
+you know immediately objects that we have
+not taken their code
+
+0:30:50.620,0:30:51.110
+and
+
+0:30:51.110,0:30:52.970
+uh they go to DARPA
+
+0:30:52.970,0:30:57.940
+and they said, You paid BBN
+how many millions of dollars, more than you paid to
+
+0:30:57.940,0:30:59.719
+Berkley to do this code
+
+0:30:59.719,0:31:00.730
+you gonna uh you know
+0:31:00.730,0:31:02.900
+and now they're not taking it
+
+0:31:02.900,0:31:04.330
+and and you know
+
+0:31:04.330,0:31:06.360
+you got look like ninnies or something. I
+don't know
+
+0:31:06.360,0:31:08.390
+I dont know what theyve
+said I wasn't there
+
+0:31:08.390,0:31:10.179
+But at any rate
+
+0:31:10.179,0:31:13.280
+DARPA
+
+0:31:13.280,0:31:16.600
+representative shows up in my office
+
+0:31:16.600,0:31:18.220
+and
+
+0:31:18.220,0:31:19.240
+Uh very calmly
+
+0:31:19.240,0:31:21.090
+uh explains that.
+
+0:31:21.090,0:31:22.100
+You know
+
+0:31:22.100,0:31:27.940
+the agreement was that BBN&N was going to
+TCP/IP code and
+
+0:31:27.940,0:31:30.240
+Uh Berkley was to do the interface
+
+0:31:30.240,0:31:31.070
+and
+
+0:31:31.070,0:31:33.049
+This is this is the way the world and
+
+0:31:33.049,0:31:36.769
+you know we appreciate that you know youve
+done with others stuff and it was it was great work
+
+0:31:36.769,0:31:38.670
+but
+
+0:31:38.670,0:31:42.460
+they are funding us and this is the way it is.
+
+0:31:42.460,0:31:44.120
+And my quarrel is like
+
+0:31:44.120,0:31:48.370
+uh you know arguing with them and look, you
+know we've taken this and weve
+
+0:31:48.370,0:31:52.160
+put over here, all the reasons that I gave earlier
+
+0:31:52.160,0:31:55.410
+and you know the guy finally at the end just
+says,
+
+0:31:55.410,0:31:57.679
+ Not . You don't understand.
+
+0:31:57.679,0:31:58.549
+we're in charge
+
+0:31:58.549,0:32:02.500
+this is what you doing. Goodbye!
+
+0:32:02.500,0:32:04.960
+And you know
+
+0:32:04.960,0:32:08.000
+it just doesn't work very well at Berkeley
+just come in and say,
+
+0:32:08.000,0:32:10.730
+This is what you're going to do.
+
+0:32:10.730,0:32:13.000
+The free speech movement comes to mind.
+
+0:32:13.000,0:32:15.970
+Uh so
+
+0:32:15.970,0:32:17.970
+You know Mike just go particularly BANANAS.
+
+0:32:17.970,0:32:21.190
+This like , Aggarh
+
+0:32:21.190,0:32:23.370
+and then fires off
+
+0:32:23.370,0:32:26.570
+unannounced to me fires off the thing
+that the DARPA uh
+
+0:32:26.570,0:32:30.050
+which if I'd been would keep in shut
+because I would not let them blame SUN
+
+0:32:30.050,0:32:31.690
+At any rate
+
+0:32:31.690,0:32:35.350
+I mean I only see it because DARPA guy
+sent it back to me and says you need to keep
+
+0:32:35.350,0:32:38.720
+your staff under control.
+
+0:32:38.720,0:32:39.650
+Uh so
+
+0:32:39.650,0:32:44.130
+at any rate um you know
+0:32:44.130,0:32:46.090
+Mike just says, I'm not doing it.
+
+0:32:46.090,0:32:49.510
+They can't make me, I'll just leave.
+
+0:32:49.510,0:32:51.640
+ You know so you know
+
+0:32:51.640,0:32:56.450
+You tell them that they can either this way or
+they can have that way or we wont release it. Um
+
+0:32:56.450,0:32:57.619
+Agrh . um
+
+0:32:57.619,0:33:00.330
+Come on theres got to be some compromise
+we can do here.
+0:33:00.330,0:33:01.730
+So
+
+0:33:01.730,0:33:05.360
+Finally uh I say, Okay lets I
+
+0:33:05.360,0:33:07.820
+I propose that we have a break-off.
+
+0:33:07.820,0:33:13.150
+So we will uh we we had already incorporated
+into the paper there was big switch to compile one
+
+0:33:13.150,0:33:14.190
+ way or the other
+
+0:33:14.190,0:33:15.609
+So my first proposal was
+
+0:33:15.609,0:33:16.910
+we will just ship it
+
+0:33:16.910,0:33:20.820
+with a compile time options.
+So you can compile for VBN code
+or it may compile powerful Berkley code.
+
+0:33:20.820,0:33:21.640
+Even BBN
+
+0:33:21.640,0:33:24.799
+Even DARPA could see
+this was a no-win
+
+0:33:24.799,0:33:27.950
+because youll end up with some
+people running one some people running the
+
+0:33:27.950,0:33:30.549
+other, this wont work smoothly.
+
+0:33:30.549,0:33:31.539
+ This was just not
+
+0:33:31.539,0:33:33.550
+where they want it to go.
+
+0:33:33.550,0:33:35.409
+and so we said, Okay well
+
+0:33:35.409,0:33:38.480
+switch is there it's really easy to compile
+with two different systems.
+
+0:33:38.480,0:33:41.830
+Lets have a break-off
+
+0:33:41.830,0:33:43.649
+you know well have compiler
+
+0:33:43.649,0:33:46.549
+with one and compiler with other one and have bunch of tests
+
+0:33:46.549,0:33:49.059
+and you know will decide which one is better.
+
+0:33:49.059,0:33:50.610
+Well of course we're not going to let
+
+0:33:50.610,0:33:55.429
+Berkley do the testing because therell be biased
+for Berkley. Were not going to let BBN to do the testing
+
+0:33:55.429,0:33:57.640
+because theyll be biased for BB&N.
+
+0:33:57.640,0:33:59.720
+so weve to find a
+
+0:33:59.720,0:34:01.210
+uh
+
+0:34:01.210,0:34:03.480
+a third party and
+
+0:34:03.480,0:34:08.440
+uh the third party that DARPA actually proposes
+is Col Mike Moose at the Ballistics
+
+0:34:08.440,0:34:10.979
+Research Laboratory, just up the road from
+here and
+
+0:34:10.979,0:34:12.929
+Uh yeah
+
+0:34:12.929,0:34:14.509
+little known to
+
+0:34:14.509,0:34:18.659
+most of the people is that we actually knew Mike Moose
+very well because we worked a lot with him.
+
+0:34:18.659,0:34:19.910
+working on TCP/IP
+
+0:34:19.910,0:34:23.659
+so
+
+0:34:23.659,0:34:25.059
+were like uhm yah its ok with us.
+
+0:34:25.059,0:34:27.869
+And you know he was in military therere
+
+0:34:27.869,0:34:31.139
+chain of command and BBN mock
+new ones so
+
+0:34:31.139,0:34:33.589
+theyre fine with it
+
+0:34:33.589,0:34:34.370
+So ok great so
+
+0:34:34.370,0:34:36.469
+you know off we ship that tape and
+
+0:34:36.469,0:34:40.749
+He does not tell us what test he is gonna do
+so neither side gets to
+
+0:34:40.749,0:34:44.720
+do anything other than make some suggestions
+general suggestions so
+
+0:34:44.720,0:34:48.779
+Of course we suggest the 3 port performance
+test could be important
+
+0:34:48.779,0:34:51.659
+and the BBN suggests that
+
+0:34:51.659,0:34:55.320
+uh how well it works in that phase of large
+package
+
+0:34:55.320,0:34:57.049
+great losses is important
+
+0:34:57.049,0:35:01.430
+because of course the military you know will drop
+a nuclear bomb on Chicago we want to make sure
+
+0:35:01.430,0:35:03.589
+we make outstand through that
+
+0:35:03.589,0:35:07.320
+At any rate Mike Moose sets up these the systems
+and
+
+0:35:07.320,0:35:09.150
+runs a bunch of tests
+
+0:35:09.150,0:35:12.179
+and of course without talking to either side
+
+0:35:12.179,0:35:16.950
+and so we're sort of pens and needles
+because you know several weeks goes by
+
+0:35:16.950,0:35:18.550
+and finally duh
+
+0:35:18.550,0:35:19.589
+his report lands.
+
+0:35:19.589,0:35:23.029
+You know humh comes in the mail here's his report
+
+0:35:23.029,0:35:26.269
+And the first thing youd do is
+to flip to the last page to see what the conclusion
+
+0:35:26.269,0:35:29.479
+is but he knew that's what we're going to
+do so he hid the conclusion somewhere in the
+
+0:35:29.479,0:35:31.660
+middle
+
+0:35:31.660,0:35:33.369
+what's the need to read the entire thing
+
+0:35:33.369,0:35:36.229
+but it's a it's a very interesting week because
+
+0:35:36.229,0:35:39.910
+in terms of three protocols
+we expected we did better
+
+0:35:39.910,0:35:40.840
+Uh what
+
+0:35:40.840,0:35:44.929
+that he introduced a thing where it would
+randomly drop about every
+
+0:35:44.929,0:35:46.769
+fourth package
+
+0:35:46.769,0:35:47.799
+and
+
+0:35:47.799,0:35:49.390
+under this scenario
+
+0:35:49.390,0:35:50.919
+he says uh
+
+0:35:50.919,0:35:52.420
+the BBN code
+
+0:35:52.420,0:35:56.820
+Jumps out to an early start getting a lot
+more data through interface of hardware
+
+0:35:56.820,0:35:59.039
+packet loss uh
+
+0:35:59.039,0:36:03.099
+but but the Berkley part managers to catch up
+while the BB&N machine reboots.
+
+0:36:03.099,0:36:07.719
+Uh
+
+0:36:07.719,0:36:12.109
+Ultimately resulting in the conclusion
+that even in the phase of the packet
+
+0:36:12.109,0:36:14.739
+loss, Berkley code seems to
+
+0:36:14.739,0:36:16.349
+do a little better.
+
+0:36:16.349,0:36:17.739
+And uh so his
+
+0:36:17.739,0:36:22.119
+conclusion is that it should continue to ship
+with the BSD code.
+
+0:36:22.119,0:36:25.519
+Uh so with this result in hand
+
+0:36:25.519,0:36:28.749
+we are authorized by DARPA to do the release
+
+0:36:28.749,0:36:32.189
+and we finally managed to release
+
+0:36:32.189,0:36:35.829
+4.3 in June of
+
+0:36:35.829,0:36:38.420
+1986
+
+0:36:38.420,0:36:39.599
+so
+
+0:36:39.599,0:36:41.519
+at this point uh
+
+0:36:41.519,0:36:42.199
+At this point
+
+0:36:42.199,0:36:45.389
+the new TCP code is gone out
+
+0:36:45.389,0:36:47.189
+were you know that
+
+0:36:47.189,0:36:52.289
+The revised BSD TCP/IP code is gone out
+
+0:36:52.289,0:36:54.689
+[with or without subnets]
+With or without uh..subnets
+
+0:36:54.689,0:36:56.150
+uh that had
+
+0:36:56.150,0:36:59.809
+well the the class ABC type of subnets
+not
+
+0:36:59.809,0:37:01.319
+not the uh
+
+0:37:01.319,0:37:02.910
+the sider stuff
+
+0:37:02.910,0:37:09.910
+sider came along
+
+0:37:16.329,0:37:17.299
+okay
+
+0:37:17.299,0:37:19.409
+uh at any rate uh
+
+0:37:19.409,0:37:23.719
+now we start accelerating the story hope
+that again because I have to finish in
+
+0:37:23.719,0:37:25.559
+eight minutes
+
+0:37:25.559,0:37:27.540
+so at this point
+
+0:37:27.540,0:37:28.260
+Keith Bostic
+
+0:37:28.260,0:37:29.209
+comes to
+
+0:37:29.209,0:37:30.469
+comes to Berkeley
+
+0:37:30.469,0:37:34.059
+uh he had been at excise which some of you
+may know
+
+0:37:34.059,0:37:37.329
+and this is in the fall of 1986 so
+
+0:37:37.329,0:37:40.219
+we've released 4.3 were just
+getting cranked up
+
+0:37:40.219,0:37:43.049
+to work on 4.4 uh
+
+0:37:43.049,0:37:45.259
+remember that VM thing
+
+0:37:45.259,0:37:48.039
+and I think by this time this is other little
+
+0:37:48.039,0:37:50.329
+protocol MFS you mayve heard of it
+
+0:37:50.329,0:37:54.839
+and it was felt that we ought to have that
+as well
+
+0:37:54.839,0:37:57.150
+uh so bring Keith to pass the protocol
+
+0:37:57.150,0:37:59.560
+Uh one of his of requirements for coming
+
+0:37:59.560,0:38:03.420
+is that we have to agree to allow him to finish
+that port of
+
+0:38:03.420,0:38:06.539
+2.11 to the PDP 11 ah
+
+0:38:06.539,0:38:08.489
+and uh you know
+
+0:38:08.489,0:38:12.470
+it's it's a real challenge to get a kernel which
+adapt the port where it was about
+
+0:38:12.470,0:38:17.239
+just under 200 kilobyte of text
+to fit on to a machine that has a maximum
+
+0:38:17.239,0:38:20.299
+amount of program space of 64 kilobytes
+
+0:38:20.299,0:38:23.450
+ because what you have to do is to
+break and overlays
+
+0:38:23.450,0:38:27.789
+and then you map in the overly and
+ the granularity of mapping is 4k
+
+0:38:27.789,0:38:28.530
+okay
+
+0:38:28.530,0:38:33.079
+So there's only eight pieces that you
+can remap uh
+
+0:38:33.079,0:38:37.519
+Mike can I added to it is thats
+incredibly challenging problem and it's
+
+0:38:37.519,0:38:41.469
+probably you know intellectually very stimulating
+but there's no way how we would do that.
+
+0:38:41.469,0:38:43.139
+Well if you want to do it you know
+
+0:38:43.139,0:38:45.849
+by all means you can do that in your
+spare time
+
+0:38:45.849,0:38:48.309
+uh there is only three jobs you need to do
+
+0:38:48.309,0:38:51.540
+uh one is uh you need to be the answer the
+phone
+
+0:38:51.540,0:38:55.409
+uh first thing other than distribution like
+technical questions
+
+0:38:55.409,0:38:58.149
+and answer the incoming the mail stream for technical
+questions.
+
+0:38:58.149,0:39:03.900
+Uh do some programming in in in your spare
+time you can work on the PDP 11
+
+0:39:03.900,0:39:04.850
+and know he was
+
+0:39:04.850,0:39:08.669
+had no other support for the phones and the
+email
+
+0:39:08.669,0:39:09.460
+and we had
+
+0:39:09.460,0:39:12.209
+you know by this time about two thousand
+
+0:39:12.209,0:39:14.749
+uh distributions
+
+0:39:14.749,0:39:19.730
+So at any rate Keith did in fact finally
+finished the PDP 11.
+
+0:39:19.730,0:39:20.779
+Uh port uh
+
+0:39:20.779,0:39:23.029
+got 2.11 out of the door
+
+0:39:23.029,0:39:26.960
+and we have a party to celebrate the fact
+that he had completed this.
+
+0:39:26.960,0:39:29.850
+and part of the party was opening the window
+
+0:39:29.850,0:39:31.240
+on the fourth floor
+
+0:39:31.240,0:39:33.519
+of our uh building
+
+0:39:33.519,0:39:38.849
+and taking his PDP 11 and
+ceremoniously dropping out of the window to the cement
+
+0:39:38.849,0:39:40.979
+sidewalks four stories below
+
+0:39:40.979,0:39:43.100
+to insure that he would never
+
+0:39:43.100,0:39:44.749
+work on it again.
+
+0:39:44.749,0:39:50.409
+This was him doing this not us.
+
+0:39:50.409,0:39:51.849
+At any rate uh
+
+0:39:51.849,0:39:53.930
+Keith of course being in the front line
+
+0:39:53.930,0:39:55.919
+uh support
+
+0:39:55.919,0:39:59.199
+Uh had a lot of feedback from various users uh
+
+0:39:59.199,0:40:00.509
+in particular
+
+0:40:00.509,0:40:03.559
+there were a number of vendors that work
+
+0:40:03.559,0:40:05.159
+trying to build
+
+0:40:05.159,0:40:05.930
+uh
+0:40:05.930,0:40:08.560
+Network cards you could plug into a PC
+
+0:40:08.560,0:40:10.999
+that would provide TCP/IP network
+
+0:40:10.999,0:40:16.180
+and of course the PC it was way too feeble
+to actually run the code in the PC itself
+
+0:40:16.180,0:40:17.539
+so they had a outward
+
+
+0:40:17.539,0:40:18.799
+processor on the card
+
+0:40:18.799,0:40:22.039
+and theyd put the TCP/IP in there
+that would sort of look
+
+0:40:22.039,0:40:23.200
+uh like a
+
+0:40:23.200,0:40:26.779
+file or some kind of device on
+that PC and
+
+0:40:26.779,0:40:28.900
+so they would just open the uh
+
+0:40:28.900,0:40:30.709
+get connected to the network
+
+0:40:30.709,0:40:34.939
+and So of course they wanted TCP/IP
+code from Berkeley along with the important
+
+0:40:34.939,0:40:38.210
+utilities like O or log in and
+
+0:40:38.210,0:40:38.659
+RCP uh
+
+0:40:38.659,0:40:42.709
+by that time FTP and TELNET uh
+
+0:40:42.709,0:40:45.189
+and
+
+0:40:45.189,0:40:48.909
+they didn't our standard thing was
+to buy the distribution and take whatever
+
+0:40:48.909,0:40:50.100
+they need to get out of it.
+
+0:40:50.100,0:40:52.769
+But that meant they had to get an AT&T license.
+
+0:40:52.769,0:40:57.680
+By the late 80s the AT&T
+licenses or up to a quarter million dollars
+
+0:40:57.680,0:40:59.379
+and when you're in a little
+
+0:40:59.379,0:41:03.939
+commodity PC business a quarter million dollar
+cost just to get the surfer before you do
+
+0:41:03.939,0:41:04.619
+anything with it
+
+0:41:04.619,0:41:06.669
+was just a non-starter for them.
+
+0:41:06.669,0:41:09.850
+And so we're getting requests for just
+
+0:41:09.850,0:41:13.199
+that TCP/IP code and associated utilities.
+
+0:41:13.199,0:41:17.479
+clearly since it all been developed in Berkley
+was none of that in the original version 7 UNIX
+
+0:41:17.479,0:41:19.910
+which started from uh
+
+0:41:19.910,0:41:22.099
+AT&T couldnt place any claim to that.
+
+0:41:22.099,0:41:26.829
+And so we talked to the lawyers at Berkeley and
+theyre like this How can we make money on this? uh
+
+0:41:26.829,0:41:29.249
+because that's what they were paid to say and
+
+0:41:29.249,0:41:33.089
+so we pointed out to them that It had been developed
+under a government contract and we showed them
+
+0:41:33.089,0:41:36.630
+the government contract saying that you know
+this stuff done in this contract has to be
+
+0:41:36.630,0:41:37.229
+made
+
+0:41:37.229,0:41:40.769
+available to the public at reasonable cost or
+whatever it said
+
+0:41:40.769,0:41:41.380
+uh
+
+0:41:41.380,0:41:42.100
+so
+
+0:41:42.100,0:41:45.669
+it didnt really say that but be convinced the
+lawyers that it did
+
+0:41:45.669,0:41:46.359
+and so
+
+0:41:46.359,0:41:48.369
+They agreed to lead us to pull out
+
+0:41:48.369,0:41:50.889
+this chunk of Berkley code and we
+
+0:41:50.889,0:41:51.859
+released that
+
+0:41:51.859,0:41:52.579
+under
+
+0:41:52.579,0:41:54.980
+this new license that the lawyers coped up
+
+0:41:54.980,0:41:59.299
+uh which is the one that we today think of
+as the BSD license.
+
+0:41:59.299,0:42:03.569
+And of course our goal was to the absolutely
+as
+
+0:42:03.569,0:42:07.979
+free as you know free to give away as we could possibly could
+the really only thing you had to do was to give the credit to
+
+0:42:07.979,0:42:08.789
+Berkeley.
+
+0:42:08.789,0:42:12.140
+Uh so this was networking release one.
+
+0:42:12.140,0:42:14.019
+and uh it was a wild success
+
+0:42:14.019,0:42:17.519
+uh the amazing thing was that we sold at 100
+we could
+
+0:42:17.519,0:42:20.249
+Buy it from Berkeley for 1000 dollars
+
+0:42:20.249,0:42:24.169
+or you could just downloaded it off your net
+for free
+
+0:42:24.169,0:42:28.209
+and yet a thousand people chose to pay a 1000
+dollars at Berkeley to get it.
+
+0:42:28.209,0:42:30.220
+and the only difference was that you got
+
+0:42:30.220,0:42:33.660
+to take a 9 track tape which many people didn't
+have any way of reading
+
+0:42:33.660,0:42:34.100
+and
+
+0:42:34.100,0:42:35.719
+hey uh
+
+0:42:35.719,0:42:39.869
+that a piece of paper signed by Berkley
+lawyers
+
+0:42:39.869,0:42:42.289
+essentially saying Yes you can use this
+
+0:42:42.289,0:42:44.939
+and that's what they were paying a 1000 dollars for
+
+0:42:44.939,0:42:48.239
+and the lawyers of course got paid something
+like a hundred dollars for every one of
+
+0:42:48.239,0:42:50.790
+these that they signed because of the overhead
+
+0:42:50.790,0:42:51.650
+et cetera
+
+0:42:51.650,0:42:54.349
+they thought this was the cat's pyjamas
+
+0:42:54.349,0:42:57.499
+you know maybe the campus was making a lot of
+money but their office was confined to offer just to Thank
+
+0:42:57.499,0:42:59.009
+you very much!
+
+0:42:59.009,0:43:01.409
+so
+
+0:43:01.409,0:43:03.420
+Uh at any rate oh
+
+0:43:03.420,0:43:04.440
+Network one was
+
+0:43:04.440,0:43:06.059
+widely successful
+
+0:43:06.059,0:43:08.309
+and so there was pressure as Keith
+
+0:43:08.309,0:43:12.369
+would come inside uh someone today want to want to
+be I or somebody wants this is or somebody
+
+0:43:12.369,0:43:13.589
+wants that
+
+0:43:13.589,0:43:16.289
+and you know we really ought to do this and uh
+
+0:43:16.289,0:43:19.440
+you know how about figuring out other parts of the
+kernel.
+
+0:43:19.440,0:43:20.559
+Mike and I would
+
+0:43:20.559,0:43:24.929
+Michael and I would look at the each other we
+look at Keith its like yah yah all right.
+
+0:43:24.929,0:43:28.989
+Uh what you figured out you know I mean
+You know all entire of the C library and all
+
+0:43:28.989,0:43:29.819
+this and that
+0:43:29.819,0:43:31.760
+so we got to the conferences and
+
+0:43:31.760,0:43:32.859
+Keith is
+
+0:43:32.859,0:43:36.809
+putting up you know these huge list of utilities
+youll rewrite this utility and get their name
+
+0:43:36.809,0:43:37.729
+in lights.
+
+0:43:37.729,0:43:40.869
+and uh you know and uh
+
+0:43:40.869,0:43:43.939
+So he would get this stuff send too along
+
+0:43:43.939,0:43:48.529
+and he would almost always just take a
+bash and Keiths normal form which
+
+0:43:48.529,0:43:54.139
+usually get look a lot like what he got but
+he started from cleans so that
+
+0:43:54.139,0:43:55.119
+Was uh
+
+0:43:55.119,0:43:59.229
+you know good and you know its sort of easy
+ you know that first someone sensing cat in.
+
+0:43:59.229,0:44:01.450
+you know someone like really go ho hog and
+
+0:44:01.450,0:44:04.849
+does that all options to allow uh
+
+0:44:04.849,0:44:07.629
+this is in the days before the new software
+you know
+
+0:44:07.629,0:44:10.119
+lots of options for in vote
+
+0:44:10.119,0:44:13.079
+Uh and then
+
+0:44:13.079,0:44:15.239
+one day uh
+
+0:44:15.239,0:44:16.759
+Keith comes into to report
+
+0:44:16.759,0:44:18.470
+that somebody is redoing
+
+0:44:18.470,0:44:20.449
+the T graph family
+
+0:44:20.449,0:44:24.099
+and this is when Mike and I look at each
+sort of grope
+
+0:44:24.099,0:44:28.190
+because the next question out of
+Keiths mouth is Hows that kernel coming guys?
+
+0:44:28.190,0:44:32.299
+We can't go to the useless conferences and say rewrite
+the kernel to get your name in lights you know
+
+0:44:32.299,0:44:36.400
+so we actually have to like go through and
+figure out what's
+
+0:44:36.400,0:44:39.130
+Berkley and what's not Berkeley and
+
+0:44:39.130,0:44:40.499
+segregate stuff out uh
+
+0:44:40.499,0:44:43.459
+A long story but eventually we get
+
+0:44:43.459,0:44:45.599
+something we feel that was clean
+
+0:44:45.599,0:44:47.929
+and uh so
+
+0:44:47.929,0:44:51.699
+we go back to the lawyers and say well
+cut the update to the networking tape
+
+0:44:51.699,0:44:52.989
+that we'd like to do
+
+0:44:52.989,0:44:59.079
+uh that uh we're going to call it networking
+release 2 not 2 and
+
+0:44:59.079,0:45:00.470
+so actually
+
+0:45:00.470,0:45:04.599
+we do sort of come on and explain it is practically
+the entire system and
+
+0:45:04.599,0:45:07.759
+lots of stuff happens but they finally sign
+off on it
+
+0:45:07.759,0:45:09.469
+and we put it out there
+
+0:45:09.469,0:45:14.519
+and it's out there for good half a year before
+BSD i
+
+0:45:14.519,0:45:17.499
+was in the missing bits and start telling
+it
+
+0:45:17.499,0:45:19.650
+they you know
+
+0:45:19.650,0:45:21.439
+engineers are
+
+0:45:21.439,0:45:23.230
+not really that good at
+
+0:45:23.230,0:45:25.419
+marketing and so
+
+0:45:25.419,0:45:28.859
+in particular they think that the sort of
+cutesy see things like
+
+0:45:28.859,0:45:32.729
+having your phone number being 1800
+its UNIX
+
+0:45:32.729,0:45:38.759
+and adds to save a ninety nine percent of the
+price of a AT&T charges for the source code
+
+0:45:38.759,0:45:40.439
+Uh are cute
+
+0:45:40.439,0:45:41.670
+Got it really irritates
+
+0:45:41.670,0:45:44.759
+the piss out of AT&T
+
+0:45:44.759,0:45:46.270
+And so they sue us
+
+0:45:46.270,0:45:49.319
+imagine that why they sue BSD I
+
+0:45:49.319,0:45:53.509
+and BSD i is what added only the 6
+file and you know all the rest came from Berkley.
+
+0:45:53.509,0:45:57.129
+So we would be happy to talk about any of
+the six files youd like to
+
+0:45:57.129,0:45:59.530
+uh but otherwise you know forget it
+
+0:45:59.530,0:46:01.009
+and the judge agrees
+
+0:46:01.009,0:46:04.539
+because they don't have any problem about
+those six files and so
+
+0:46:04.539,0:46:07.769
+there's no choice but to sue the University
+
+0:46:07.769,0:46:09.259
+and uh
+
+0:46:09.259,0:46:11.160
+thats another long story for which I did
+
+0:46:11.160,0:46:12.629
+become an expert witness
+
+0:46:12.629,0:46:14.390
+which is actually proven
+
+0:46:14.390,0:46:18.809
+to be of relatively lucrative thing because
+being an expert witness
+
+0:46:18.809,0:46:20.039
+it's something that you
+
+0:46:20.039,0:46:24.669
+it's really hard to break into that is no
+one wants to hire someone hasn't done it before
+
+0:46:24.669,0:46:28.359
+top and it turns out that it really didn't
+matter how much you know, do
+
+0:46:28.359,0:46:33.829
+you have a PhD because if you have PhD by definition
+you must know you're talking about
+
+0:46:33.829,0:46:34.859
+himself
+
+0:46:34.859,0:46:36.219
+and number two is
+
+0:46:36.219,0:46:40.839
+How you speak in public then you know especially
+people harassing you look at ten thousand
+
+0:46:40.839,0:46:41.830
+students you know
+
+0:46:41.830,0:46:45.309
+there's not a question that a graduate student
+has never asked me but Id deal with it
+
+0:46:45.309,0:46:51.309
+and lawyer is not up to a graduate student.
+
+0:46:51.309,0:46:53.219
+So I did pretty well that score too
+
+0:46:53.219,0:46:55.129
+uh and so
+
+0:46:55.129,0:46:57.599
+because I that they were forced to use me
+ in this case.
+
+0:46:57.599,0:47:00.080
+And but I did ran after then
+
+0:47:00.080,0:47:03.879
+the lawyer the outside lawyer that the Berkeley
+got me to do some other stuff and
+
+0:47:03.879,0:47:08.800
+one thing leads to another so today I am uh
+an expert in the
+
+0:47:08.800,0:47:12.989
+network complaints vs SUN microsystem
+lawsuit vs ZFS lawsuit
+
+0:47:12.989,0:47:14.439
+which is uh
+
+0:47:14.439,0:47:17.739
+an interesting story that you can ask over
+here
+
+0:47:17.739,0:47:18.679
+At any rate
+
+0:47:18.679,0:47:21.629
+we eventually win the the lawsuit
+
+0:47:21.629,0:47:26.089
+And uh and we don't win it be subtle the
+lawsuit excuse me I have to use proper terminology
+
+0:47:26.089,0:47:27.029
+Here.
+
+0:47:27.029,0:47:28.750
+We have to agree
+
+0:47:28.750,0:47:32.999
+that networking released 2 did in fact have
+things in it that infringed
+
+0:47:32.999,0:47:34.490
+uh and
+
+0:47:34.490,0:47:35.999
+however
+
+0:47:35.999,0:47:40.329
+Uh the things that infringe were sufficiently
+minor that was some
+
+0:47:40.329,0:47:40.730
+changes
+
+0:47:40.730,0:47:42.039
+uh we can
+
+0:47:42.039,0:47:43.370
+get something else clean
+
+0:47:43.370,0:47:45.959
+ we agreed that we will re-release
+
+0:47:45.959,0:47:50.890
+we'd do new release that has all these
+changes made in it as 4.4 BSD lite.
+
+0:47:50.890,0:47:54.449
+So its 4.4 BSD which is the whole
+thing complete with AT&T stuff and release
+
+0:47:54.449,0:47:56.059
+AT&T licenses
+
+0:47:56.059,0:47:57.580
+and for 4.4 BSD lite
+
+0:47:57.580,0:48:00.979
+ which is got the stuff AT&T stuffs stripped out
+of it.
+
+0:48:00.979,0:48:04.869
+And AT&T really thought theyd won big-time
+because all these people had been using the
+
+0:48:04.869,0:48:05.509
+Net 2.
+
+0:48:05.509,0:48:08.909
+They're going to be forced to throw out
+because were not allowed to tell what the changes
+
+0:48:08.909,0:48:09.640
+were
+
+0:48:09.640,0:48:12.729
+and have to start all over again
+
+0:48:12.729,0:48:14.929
+from lite
+
+0:48:14.929,0:48:19.339
+And the Bill Joy in particular has never did that
+and so that's why this distribution got hammered
+
+0:48:19.339,0:48:22.150
+FreeBSD had to do it
+
+0:48:22.150,0:48:24.329
+not the BSD folks had to do it uh
+
+0:48:24.329,0:48:26.289
+and it was a huge amount of work but they
+did it
+
+0:48:26.289,0:48:30.489
+and it was great because we got two more years
+worth of our technology out
+
+0:48:30.489,0:48:32.959
+at any rate
+
+0:48:32.959,0:48:36.289
+we actually did one more incremental sort of the
+release to
+
+0:48:36.289,0:48:39.510
+I would just fix some of the most egregious
+bugs
+
+0:48:39.510,0:48:43.309
+thats what all happened was we got all these people
+who bought this BSD lite licenses
+
+0:48:43.309,0:48:46.519
+Because they wanted to pay the
+thousand dollars to get the piece of paper
+
+0:48:46.519,0:48:46.940
+saying
+
+0:48:46.940,0:48:48.899
+and this is fine too
+
+0:48:48.899,0:48:49.640
+and uh
+
+0:48:49.640,0:48:52.819
+so we had a chunk of money and so Keith
+and I started to do the work
+
+0:48:52.819,0:48:54.689
+through that money
+
+0:48:54.689,0:48:56.329
+and it's just spend it
+
+0:48:56.329,0:48:58.640
+essentially doing bug fixes and stuff on lite
+
+0:48:58.640,0:49:00.839
+And finally released when the money ran out then
+
+0:49:00.839,0:49:03.769
+we left the university and that was the last release
+
+0:49:03.769,0:49:05.679
+Uh from from Berkley
+
+0:49:05.679,0:49:12.679
+and that is the end of my story
+So any questions?
+Uh .. Yes
+
+
+0:49:20.069,0:49:21.929
+Sorry.. uh VM system
+
+
+0:49:21.929,0:49:25.359
+VM system yup.
+
+0:49:25.359,0:49:30.579
+So the VM system was actually sort of an interesting
+story uh..
+
+0:49:30.579,0:49:33.759
+Actually both VM and NFS were interesting
+Stories.
+
+0:49:33.759,0:49:36.890
+The VM system uh
+
+0:49:36.890,0:49:41.160
+one of our attitudes was that why always come
+up with a good idea when you can steal a better
+
+0:49:41.160,0:49:42.759
+one and
+
+0:49:42.759,0:49:46.369
+you know were just allowed to write
+a VM system this uh
+
+0:49:46.369,0:49:47.579
+you know that BSD folks
+
+0:49:47.579,0:49:48.599
+clearly know
+
+0:49:48.599,0:49:49.530
+and
+
+0:49:49.530,0:49:50.889
+uh so
+
+0:49:50.889,0:49:54.539
+we looked around to see if there's anything
+that we could use
+
+0:49:54.539,0:49:57.969
+rather than have to interpret it from the scratch
+ourselves. and
+
+0:49:57.969,0:50:01.689
+ evaluated two different things one
+was
+
+0:50:01.689,0:50:04.759
+uh the the stuff that SUN had done
+
+0:50:04.759,0:50:10.109
+and the other was the stuff that had been done at
+Carnegie Mellon under the Marks then and
+
+0:50:10.109,0:50:14.069
+Uh they've done a whole lot of very good work
+on the VM system we didn't really agree with
+
+0:50:14.069,0:50:18.179
+the microphone kernel principle but we liked
+the VM system
+
+0:50:18.179,0:50:21.589
+and so we talked to SUN about potentially
+
+0:50:21.589,0:50:23.479
+Contributing uh
+
+0:50:23.479,0:50:27.159
+chunk of their VM system we talked to them
+about contributing a chunk of their
+
+0:50:27.159,0:50:27.949
+VFS stuff.
+0:50:27.949,0:50:31.649
+We knew that we couldnt get the NFS from them but
+
+0:50:31.649,0:50:32.659
+they they you know
+
+0:50:32.659,0:50:36.809
+the folks at SUN is sorts of an engineering were too
+enthusiastic about it and percolated it all the way
+
+0:50:36.809,0:50:39.459
+up to the Scott McNealy, he was the CEO at that
+time
+
+0:50:39.459,0:50:42.979
+and even Scott McNealy was pretty enthusiastic about
+giving the stuff to Berkley.
+
+0:50:42.979,0:50:45.239
+you know as long as SUN would get credit for
+it
+
+0:50:45.239,0:50:47.140
+And uh
+
+0:50:47.140,0:50:51.549
+so we've really got to the point where it
+is needed to be signed off by the the SUN
+
+0:50:51.549,0:50:52.429
+lawyers
+
+0:50:52.429,0:50:58.919
+and sent off to them and after that SUN
+lawyers came back and said,
+
+0:50:58.919,0:51:03.199
+ Well you know your stock holders could accuse you of giving
+away company property
+
+0:51:03.199,0:51:07.309
+and and sue you in court and we really don't
+recommend that you do this.
+
+0:51:07.309,0:51:08.380
+and so
+
+0:51:08.380,0:51:09.999
+that the very last minute it was ..
+
+0:51:09.999,0:51:12.779
+We couldn't get any of that stuff from SUN.
+
+0:51:12.779,0:51:14.699
+So we ended up
+
+0:51:14.699,0:51:17.859
+adopting this stuff from Mark
+the Mark VM system
+
+0:51:17.859,0:51:18.699
+and
+
+0:51:18.699,0:51:22.369
+so it fell to myself to take that and
+
+0:51:22.369,0:51:24.669
+figure out essentially how to plug it in.
+
+0:51:24.669,0:51:27.140
+apart of all of this the old VM system
+
+0:51:27.140,0:51:32.259
+was very portable provided you are pouring
+into something that looked a lot like a VAX.
+
+0:51:32.259,0:51:37.429
+because it came down to uh you know the that the
+basic page data structure was VAX
+
+0:51:37.429,0:51:38.979
+page data structure.
+
+0:51:38.979,0:51:40.590
+and so when people will ported it.
+
+0:51:40.590,0:51:44.440
+If they have something with the page data structure
+to look different they wouldn't take it from
+
+0:51:44.440,0:51:48.360
+the VAX and would shuffle the bits around and put
+it in the right order and then dropped it into
+
+0:51:48.360,0:51:50.829
+the whatever that architecture was
+
+0:51:50.829,0:51:55.509
+a page table. So it worked pretty well for porting
+into the motorola uh because the
+
+0:51:55.509,0:51:57.959
+NMU network was a lot like the VAX
+uh
+
+0:51:57.959,0:52:01.899
+and some other machines but in other ways it
+just really didn't cut it at all
+
+0:52:01.899,0:52:06.130
+whereas the VM had been done by
+the the folks at the CMU.
+
+0:52:06.130,0:52:09.389
+In Mark was uh you know
+
+0:52:09.389,0:52:11.059
+very elegantly designed
+
+0:52:11.059,0:52:13.150
+uh and in fact
+
+0:52:13.150,0:52:14.740
+it had in some ways
+
+0:52:14.740,0:52:18.980
+too many bells and whistles and too many options
+and because a lot of sort of dead end so they
+
+0:52:18.980,0:52:21.189
+just sort of left hanging there
+
+0:52:21.189,0:52:22.589
+Uh but uh
+
+0:52:22.589,0:52:26.200
+the structure itself was nicely done about it was
+about 90% machine independent code
+
+0:52:26.200,0:52:28.459
+10 % machine dependent code so that
+
+0:52:28.459,0:52:30.119
+a well defined interface.
+
+0:52:30.119,0:52:32.239
+the PMAC interface.
+
+0:52:32.239,0:52:34.359
+got that put in and of course we
+
+0:52:34.359,0:52:40.019
+did onto the the Berkeley mmap Interface
+rather than what they did
+
+0:52:40.019,0:52:42.700
+with uh ports and so on
+
+0:52:42.700,0:52:43.160
+Uh and
+
+0:52:43.160,0:52:48.210
+So I got back together sort in the that
+the 4 3 phase I didnt talk about there were
+
+0:52:48.210,0:52:51.130
+two releases between 4.3 and 4.4
+4.3
+
+0:52:51.130,0:52:53.789
+Taha and 4.3 Reno and
+
+0:52:53.789,0:52:56.689
+the first of them having to the VM and the
+
+0:52:56.689,0:52:58.099
+second one having the NFS in it.
+
+0:52:58.099,0:52:59.759
+And uh
+
+0:52:59.759,0:53:01.499
+so
+
+0:53:01.499,0:53:05.529
+backup put out and then luckily people would feel
+sort of look at it and said well you know
+
+0:53:05.529,0:53:07.709
+this really needs like a lot of work
+
+0:53:07.709,0:53:11.699
+so Michael, University Utah did a tremendous
+amount of the clean up in it.
+
+0:53:11.699,0:53:16.489
+and then later actually after we release
+lite BSD in agreement
+
+0:53:16.489,0:53:18.719
+Uh did a whole lot more work on it.
+
+0:53:18.719,0:53:22.069
+really were just throwing tons of shit out
+that shouldnt have been at first place.
+
+0:53:22.069,0:53:26.559
+I had this whole notion of an external user
+driven page which was nice but the cost
+
+0:53:26.559,0:53:30.129
+of all the extra context switching other
+things that would happen on page was sort of
+
+0:53:30.129,0:53:32.549
+a killer uh
+
+0:53:32.549,0:53:39.479
+And so at any rate that's that's how the the
+VM system got in there.
+
+0:53:39.479,0:53:40.420
+Any more questions?
+
+0:53:40.420,0:53:45.349
+See I can take like five minutes to answer
+one question.
+
+0:53:45.349,0:53:45.599
+All right ! Well Thank You very much.
diff --git a/en_US.ISO8859-1/captions/README b/en_US.ISO8859-1/captions/README new file mode 100644 index 0000000000..b7bf9ce68c --- /dev/null +++ b/en_US.ISO8859-1/captions/README @@ -0,0 +1,15 @@ +$FreeBSD$ + +This directory holds closed captions for audio and video recordings of +technical presentations about FreeBSD. + +The expected hierarchy is : + +doc/${LANG}/captions/${YEAR}/${CONFERENCE}/${TALK} + +e.g. + +doc/en_US.ISO8859-1/captions/2009/asiabsdcon/mckusick-kernelinternals.sbv + +Individual captions are currently stored as YouTube 'sbv' files which +are simple ASCII text files with timing and transcription information. |