0:00:00.410,0:00:02.190 overruled 0:00:02.190,0:00:07.550 a couple of small little mistakes in there has been corrected as well 0:00:07.550,0:00:10.869 and that's at this location on the web 0:00:10.869,0:00:17.869 I also have this URL at the end of my talk 0:00:24.699,0:00:30.339 I'm going to start with a brief history of the MIPS platform 0:00:30.339,0:00:32.640 I go into this in a lot of detail in my paper 0:00:32.640,0:00:34.960 so I'm going to just hit 0:00:34.960,0:00:41.960 some of the highlights here. 0:00:43.620,0:00:46.300 interesting parts here are that there are 0:00:46.300,0:00:47.560 two implementations of MIPS 0:00:47.560,0:00:52.680 one's a thirty-two bit implementation one's a sixty-four bit implementation 0:00:52.680,0:00:55.920 that evolved over time 0:00:55.920,0:01:01.240 initially each of the implementations was cumulative 0:01:01.240,0:01:06.580 with a prior implementation so a MIPS VI or MIPS V processor 0:01:06.580,0:01:12.860 will implement anything MIPS IV implemented and 0:01:12.860,0:01:15.170 over time this was decided 0:01:15.170,0:01:22.170 that not to be such a good idea so the way this MIPS 0:01:22.490,0:01:27.060 ISA's that are available are MIPS 32 and MIPS 64 0:01:27.060,0:01:27.590 and those define 0:01:27.590,0:01:30.410 a base set in the number of options 0:01:30.410,0:01:34.640 that can be added 0:01:34.640,0:01:42.640 options for DSD processing, options for multiple instruction execution at the same time, SIMV sorts of things 0:01:44.580,0:01:51.580 and so that's kind of the history on MIPS in a nutshell 0:01:52.470,0:01:56.189 the other interesting piece was that happened with MIPS 0:01:56.189,0:01:58.490 is the original 0:01:58.490,0:02:00.180 MIPS ISA didn't define the TLBs 0:02:00.180,0:02:04.030 and all that and how that was interacting with the CPUs 0:02:04.030,0:02:05.060 there were a number of variations 0:02:05.060,0:02:10.569 over time and when the never embedded MIPS 0:02:10.569,0:02:14.539 ISA's were created 0:02:14.539,0:02:17.999 that information was specified as well 0:02:17.999,0:02:23.449 so that it's easier to write one kernel that will run on each of the processors 0:02:23.449,0:02:27.139 after they got some experience with the different processors 0:02:27.139,0:02:30.409 they updated the spec to come up with 0:02:30.409,0:02:34.480 MIPS32r2 and MIPS64r2 0:02:34.480,0:02:41.029 these are enhancements for viewing with primarily with multiple issue 0:02:41.029,0:02:44.109 highly parallel processors 0:02:44.109,0:02:47.189 one the things you would have to do prior to that changing 0:02:47.189,0:02:49.419 the CLB you would execute 0:02:49.419,0:02:53.249 a number of no ops so that 0:02:53.249,0:02:56.879 the processor pipeline would flush 0:02:56.879,0:03:03.879 and on MIPS R4000 you need there was six and on MIPS R10000 you need there was twelve 0:03:04.409,0:03:06.359 some of the newer processors you are going to know 0:03:06.359,0:03:07.060 how many you had 0:03:07.060,0:03:13.549 to execute because so many things were executing in parallel so they created some new instructions to 0:03:13.549,0:03:16.139 help you out with that 0:03:16.139,0:03:23.139 plus a couple of other things for making embedding easier and those are the two 0:03:25.469,0:03:26.809 core ISA's that FreeBSD 0:03:26.809,0:03:28.749 primarily targets 0:03:28.749,0:03:29.689 are the embedded 0:03:29.689,0:03:31.899 MIPS ABIs 0:03:31.899,0:03:34.069 the ABIs sorry the ISAs 0:03:34.069,0:03:36.589 for prior MIPS chips 0:03:36.589,0:03:39.969 probably would work with FreeBSD MIPS 0:03:39.969,0:03:43.459 nobody has taken the time to make them work 0:03:43.459,0:03:45.419 the other interesting thing 0:03:45.419,0:03:47.239 back in MIPS history is 0:03:47.239,0:03:48.799 there are a number of different 0:03:48.799,0:03:51.949 ABIs 0:03:51.949,0:03:57.259 the original 32 bit ABI which was later remained to o32 which was 0:03:57.259,0:03:58.750 for the R2000 and 3000 0:03:58.750,0:04:00.489 specified 32 bit registers 0:04:00.489,0:04:03.909 and a MIPS 2 0:04:03.909,0:04:10.909 ISA those replaced when SGI introduced 4000 based 0:04:12.879,0:04:18.109 machines with n32 and n64 0:04:18.109,0:04:19.049 n64 is 0:04:19.049,0:04:21.349 a fairly straight forward 0:04:21.349,0:04:25.629 64 bit ABI there's 0:04:25.629,0:04:27.929 nothing really all that weird about it 0:04:27.929,0:04:32.490 except the one weird thing for all MIPS ABIs is that 0:04:32.490,0:04:33.660 it's specified all the library code 0:04:33.660,0:04:35.889 has to be picked code even for static linking 0:04:35.889,0:04:38.999 0:04:38.999,0:04:42.520 n32 is a weird hybrid 0:04:42.520,0:04:47.669 and I'll get into that a little bit later but it's 0:04:47.669,0:04:48.650 an ABI that's designed 0:04:48.650,0:04:54.179 to allow transition from old code to new code 0:04:54.179,0:05:00.930 here in this context old code to new code was code written before 1995 or so 0:05:00.930,0:05:06.630 to new code which was being written since then with 64 bit 0:05:06.630,0:05:10.829 it has a number of interesting 0:05:10.829,0:05:16.639 idiosyncrasies 0:05:16.639,0:05:19.500 that happened when you tried to squeeze 0:05:19.500,0:05:20.719 when you tried to 0:05:20.719,0:05:24.410 fit 32 bit ABI to 64 bit registers and now 0:05:24.410,0:05:31.410 come later for some of the work that has done 0:05:35.090,0:05:37.529 So FreeBSD MIPS is been 0:05:37.529,0:05:40.680 around for a very long time it's not just been around in the FreeBSD tree 0:05:40.680,0:05:43.449 for along time initial ports 0:05:43.449,0:05:46.339 for FreeBSD MIPS 0:05:46.339,0:05:50.249 was in the 3.x time frame due to persistence 0:05:50.249,0:05:52.940 did a FreeBSD port to a 0:05:52.940,0:05:56.879 custom piece of 0:05:56.879,0:06:02.160 silicon that they had which had a MIPS core embedded in it and 0:06:02.160,0:06:09.160 and excuse me 0:06:11.009,0:06:16.749 and at the time they were looking for people in the FreeBSD community to commit this port back to the FreeBSD tree 0:06:16.749,0:06:18.539 there were a number of problems though 0:06:18.539,0:06:20.300 you couldn't just run this port 0:06:20.300,0:06:21.770 on a 0:06:21.770,0:06:24.280 particular on any given MIPS platform 0:06:24.280,0:06:29.289 you couldn't run it on a SGI MIPS you couldn't run it on an old deck station 0:06:29.289,0:06:30.960 it had a 0:06:30.960,0:06:32.999 high number of 0:06:32.999,0:06:38.689 tweaks in it for the custom silicon that they ran it 0:06:38.689,0:06:39.889 so interested the developer community was 0:06:39.889,0:06:42.289 ya we like a MIPS port but 0:06:42.289,0:06:49.289 we don't have any hardware to run it against and it would be a big effort to take the code that you have done and do it to 0:06:51.279,0:06:54.610 the platforms that we have access to 0:06:54.610,0:06:57.369 so 0:06:57.369,0:06:59.169 there were a lot of talks back and forth in this time frame 0:06:59.169,0:07:01.239 between members of the 0:07:01.239,0:07:03.629 FreeBSD community 0:07:03.629,0:07:05.899 and Juniper networks 0:07:05.899,0:07:08.729 and 0:07:08.729,0:07:11.669 they basically went nowhere 0:07:11.669,0:07:13.030 the fist internet bubble crashed 0:07:13.030,0:07:15.429 0:07:15.429,0:07:18.809 so it languished 0:07:18.809,0:07:19.829 everywhere except 0:07:19.829,0:07:20.930 inside of juniper 0:07:20.930,0:07:27.179 where they kept developing it and improving it over the years so 0:07:27.179,0:07:27.870 about the same time 0:07:27.870,0:07:30.500 I became interested in getting 0:07:30.500,0:07:32.300 FreeBSD up and running on the MIPS 0:07:32.300,0:07:33.979 so I got 0:07:33.979,0:07:34.539 the basic LibC support 0:07:34.539,0:07:37.429 going and the basic tool chains going in the tree 0:07:40.219,0:07:42.550 with some help from David O'Brien 0:07:42.550,0:07:44.859 and 0:07:44.859,0:07:47.159 kept trying to talk up different 0:07:47.159,0:07:53.499 people to try their for people who had more time than I had to get interested in it to pick up the banner 0:07:53.499,0:07:55.809 and this was 0:07:55.809,0:07:57.339 right after the internet 0:07:57.339,0:07:58.090 bubble burst 0:07:58.090,0:08:00.020 people didn't have a lot of spare time they were too busy 0:08:00.020,0:08:04.449 just surviving to do a MIPS port 0:08:04.449,0:08:06.520 so consequently the initial 0:08:06.520,0:08:08.819 work that I had done in 1999 was removed 0:08:08.819,0:08:15.819 three years later in 2002 when no progress had been made 0:08:18.910,0:08:21.229 a few years later 0:08:21.229,0:08:26.489 Julie Mallett started putting FreeBSD 0:08:26.489,0:08:32.990 to the SGI platform and he 0:08:32.990,0:08:34.520 got his SGI box booting into single user mode 0:08:34.520,0:08:37.550 0:08:37.550,0:08:39.380 but his situation 0:08:39.380,0:08:41.270 changed he didn't have 0:08:41.270,0:08:44.229 the time to 0:08:44.229,0:08:49.040 take it from single use mode to multi user mode to make it stable 0:08:49.040,0:08:51.720 he did this work in the Perforce repository 0:08:51.720,0:08:53.800 with one of the early examples of a 0:08:53.800,0:08:56.570 large body of work that was done 0:08:56.570,0:08:57.270 outside 0:08:57.270,0:08:58.620 FreeBSD 0:08:58.620,0:09:00.890 CVS tree 0:09:00.890,0:09:06.710 but inside of Perforce so that interested parties can look at it and work on it 0:09:06.710,0:09:07.920 since it was 0:09:07.920,0:09:13.890 SGI boxes and at the time 0:09:13.890,0:09:15.790 people call it the SGI boxes it was targeting 0:09:15.790,0:09:17.420 more obsolete 0:09:17.420,0:09:21.160 part of the developers in the community 0:09:21.160,0:09:26.310 really supported this also as 0:09:26.310,0:09:28.430 Julie would admit 0:09:28.430,0:09:30.700 she took a lot of shortcuts 0:09:30.700,0:09:34.670 to get the pot up and running 0:09:34.670,0:09:35.870 that were the basis of 0:09:35.870,0:09:42.870 a lot of the stability problems that she found out wound out having when she tried to go from single user to multi user 0:09:45.120,0:09:46.739 so the important thing here though 0:09:46.739,0:09:49.940 is that all of this code existed 0:09:49.940,0:09:51.320 Perforce repository 0:09:51.320,0:09:56.530 and just kept existing out in the Perforce repository 0:09:56.530,0:10:01.120 Julie took a lot of that code from that BSD and 0:10:01.120,0:10:04.020 in an effort to make it simpler I think wrote portions of it 0:10:04.020,0:10:06.970 0:10:06.970,0:10:11.480 but it also was simpler so there was 0:10:11.480,0:10:14.210 carefully selected pieces of code wound up in 0:10:14.210,0:10:17.010 later projects 0:10:17.010,0:10:24.010 and that was one of the value of this effort 0:10:24.440,0:10:28.940 once Julie lost interest in this she announced it publicly: I don't have time to do this 0:10:28.940,0:10:32.050 this is going nowhere 0:10:32.050,0:10:34.490 BSDcan 0:10:34.490,0:10:35.920 three years ago now 0:10:35.920,0:10:36.940 myself and 0:10:36.940,0:10:38.130 Peter Wemm 0:10:38.130,0:10:39.280 John Baldwin and 0:10:39.280,0:10:42.580 a few other people 0:10:42.580,0:10:44.400 Sat down and realized hey 0:10:44.400,0:10:46.600 there's a lot of people 0:10:46.600,0:10:47.910 who want FreeBSD 0:10:47.910,0:10:51.710 and FreeBSD's features to run 0:10:51.710,0:10:56.660 in embedded platform to run on 0:10:56.660,0:10:57.820 very cheap MIPS routers that 0:10:57.820,0:11:02.970 were becoming available at the time 0:11:02.970,0:11:04.680 and we decided hey 0:11:04.680,0:11:06.290 let's see if we can get 0:11:06.290,0:11:10.290 a jump start on this MIPS project 0:11:10.290,0:11:11.770 if the going generates 0:11:11.770,0:11:13.210 some excitement 0:11:13.210,0:11:16.310 may be we'll hit critical mass 0:11:16.310,0:11:17.060 and we did 0:11:17.060,0:11:19.340 a number of things 0:11:19.340,0:11:21.950 well we got the tool chain 0:11:21.950,0:11:25.280 up and running we got the kernel building 0:11:25.280,0:11:27.120 the MI portions 0:11:27.120,0:11:29.860 not the MD portions 0:11:29.860,0:11:32.000 unfortunately 0:11:32.000,0:11:37.000 we made a mistake on how we were doing this 0:11:37.000,0:11:41.950 we included a secret part of the Perforce repository 0:11:41.950,0:11:44.260 so there was no commit mails nobody saw what was going on 0:11:44.260,0:11:48.730 nobody knew what was happening 0:11:48.730,0:11:51.060 and the thinking at the time was well we'll 0:11:51.060,0:11:56.690 get it to a certain point and then announce it we don't want to have a public failure on our hands 0:11:56.690,0:11:59.310 so instead we had a private failure on our hands because 0:11:59.310,0:12:01.010 nobody knew it was there 0:12:01.010,0:12:07.640 nobody was excited about it we could access the code 0:12:07.640,0:12:08.540 so that was 0:12:08.540,0:12:10.020 not a good idea 0:12:10.020,0:12:12.260 you're an 0:12:12.260,0:12:13.770 open source project 0:12:13.770,0:12:20.770 you should learn from this mistake to make sure you do things in an open manner 0:12:22.140,0:12:23.480 so about six months after 0:12:23.480,0:12:25.170 we started this project 0:12:25.170,0:12:28.100 it was clear that it was going nowhere 0:12:28.100,0:12:31.110 about this time there were two individuals 0:12:31.110,0:12:33.690 Wojciech Koszek and 0:12:33.690,0:12:33.960 Oleksandr Tymoshenko 0:12:33.960,0:12:37.570 0:12:37.570,0:12:38.760 approached different members of the 0:12:38.760,0:12:43.050 had approached Julie and said hey I want to take your old port and make it better 0:12:43.050,0:12:44.100 0:12:44.100,0:12:46.500 0:12:46.500,0:12:49.560 and they talked to Julie about 0:12:49.560,0:12:50.550 our new port and 0:12:50.550,0:12:51.570 she says well 0:12:51.570,0:12:52.910 there's this new port 0:12:52.910,0:12:53.889 so these two took 0:12:53.889,0:12:56.100 the best pieces of 0:12:56.100,0:13:00.140 both of these efforts 0:13:00.140,0:13:02.910 and got FreeBSD MIPS booting on 0:13:02.910,0:13:04.250 emulated hardware 0:13:04.250,0:13:05.410 at the end of 2006 0:13:05.410,0:13:06.320 0:13:06.320,0:13:13.320 and then on real hardware in 2007 on a couple of different MIPS processors 0:13:15.920,0:13:18.290 and then 0:13:18.290,0:13:20.340 things started to 0:13:20.340,0:13:21.790 languish a little bit 0:13:21.790,0:13:24.190 except for one piece 0:13:24.190,0:13:25.990 Cavium Networks 0:13:25.990,0:13:28.380 saw this work and took one of the 0:13:28.380,0:13:30.360 MIPS 2 snapshots 0:13:30.360,0:13:31.639 and ported it to their 0:13:31.639,0:13:35.450 Octeon series of processors 0:13:35.450,0:13:39.860 The Octeon is a multicore MIPS processor 0:13:39.860,0:13:41.160 that has a 0:13:41.160,0:13:45.970 the number of additional units in it 0:13:45.970,0:13:49.370 for dealing with networking efficiently 0:13:49.370,0:13:50.970 dealing with moving packets 0:13:50.970,0:13:53.270 scheduling work 0:13:53.270,0:13:55.200 among the multiple CPUs 0:13:55.200,0:13:58.960 it's a fairly interesting platform 0:13:58.960,0:14:00.390 also 0:14:00.390,0:14:02.520 Bruce Sampson 0:14:02.520,0:14:03.600 got it running on the 0:14:03.600,0:14:05.960 the Sentry-5 0:14:05.960,0:14:06.730 series Broadcom 0:14:06.730,0:14:09.990 parts 0:14:09.990,0:14:11.860 the Sentry-5 series 0:14:11.860,0:14:15.490 which I'll get to in a little bit is different than 0:14:15.490,0:14:17.500 the Linksys WRTs 0:14:17.500,0:14:22.510 It's a higher quality implementation of MIPS so he was able to do that 0:14:22.510,0:14:25.850 so 0:14:25.850,0:14:28.460 we were seeing some small incremental steps 0:14:28.460,0:14:29.830 this little thing is added 0:14:29.830,0:14:32.930 that little thing is added 0:14:32.930,0:14:36.830 but it was still moving very slowly 0:14:36.830,0:14:42.550 it took a year and a half to get to this point 0:14:42.550,0:14:44.690 Juniper work up again 0:14:44.690,0:14:47.010 or at least decided 0:14:47.010,0:14:48.470 we need to upgrade from 0:14:48.470,0:14:51.490 our FreeBSD 4.x 0:14:51.490,0:14:57.340 to FreeBSD 6.1 to remain competitive there's a number of features 0:14:57.340,0:15:02.930 in FreeBSD as it happens we need to have those in our product 0:15:02.930,0:15:04.890 so they upgraded 0:15:04.890,0:15:05.810 their version of FreeBSD 0:15:05.810,0:15:07.400 called JunOS 0:15:07.400,0:15:09.220 to 6.1 0:15:09.220,0:15:10.680 as part 0:15:10.680,0:15:17.680 of this effort they upgraded all of their MIPS ports they had several different ones 0:15:18.060,0:15:20.410 unfortunately 0:15:20.410,0:15:22.180 they weren't able just to release these ports because they had been written with 0:15:22.180,0:15:24.630 material that came under NDA 0:15:24.630,0:15:29.580 they weren't allowed to disclose some of the 0:15:29.580,0:15:33.060 intellectual property for the specific MIPS chips 0:15:33.060,0:15:37.920 but they recognized the value of having a good FreeBSD MIPS support 0:15:37.920,0:15:38.929 so they 0:15:38.929,0:15:40.450 approached me with a 0:15:40.450,0:15:47.450 sanitized port that they had done they ported their code to an idealized MIPS machine one of the ones that conformed with the 0:15:52.300,0:15:56.770 MIPS 64 MIPS 32 ISA 0:15:56.770,0:16:01.020 they gave this code to me in September of 2007 0:16:01.020,0:16:03.360 and I started reviewing it 0:16:03.360,0:16:07.780 but the review was very slow it was a busy time for me 0:16:07.780,0:16:09.550 I was in the process 0:16:09.550,0:16:12.270 of changing jobs 0:16:12.270,0:16:15.520 so in 0:16:15.520,0:16:16.970 November of 2007 0:16:16.970,0:16:19.370 I started working for CISCO systems 0:16:19.370,0:16:22.310 CISCO systems had spearheaded an effort 0:16:22.310,0:16:24.450 to put FreeBSD as a part of the next 0:16:24.450,0:16:26.550 generation of routers they were producing 0:16:26.550,0:16:31.270 or at least for a couple of business units 0:16:31.270,0:16:34.130 and ultimately that project was abandoned 0:16:34.130,0:16:36.240 but before it was abandoned 0:16:36.240,0:16:40.330 we got FreeBSD MIPS up and going we merged 0:16:40.330,0:16:44.840 MIPS 2 code with the Juniper code the Juniper code was very good 0:16:44.840,0:16:48.580 for things like PMAP and VM system 0:16:48.580,0:16:55.580 the basic nuts and bolts of the system from a memory process point of view very stable 0:16:58.170,0:17:02.230 it had one driver in it SIO 0:17:02.230,0:17:03.259 one driver 0:17:03.259,0:17:04.960 wasn't really enough to produce a viable system 0:17:04.960,0:17:06.940 so we took 0:17:06.940,0:17:11.700 all the drivers and startup code from the MIPS 2 effort 0:17:11.700,0:17:13.920 that had been going on 0:17:13.920,0:17:20.010 so drivers for particular SOC mix drivers for 0:17:20.010,0:17:27.010 switches that were in these devices and so forth we ported those over and 0:17:27.130,0:17:33.400 got everything booting and just before 0:17:33.400,0:17:36.760 BSDcan 2008 0:17:36.760,0:17:38.130 we committed all this to the 0:17:38.130,0:17:38.590 tree except 0:17:38.590,0:17:44.360 for the Cavium port it had similar 0:17:44.360,0:17:50.510 IP issues that we needed to resolve before we committed it 0:17:50.510,0:17:54.410 so today we target MIPS 32 MIPS 64 we run 0:17:54.410,0:17:59.310 only o32 binaries we've not added the 0:17:59.310,0:17:59.730 n32 and n64 support 0:17:59.730,0:18:02.220 it runs on 0:18:02.220,0:18:04.380 about six different 0:18:04.380,0:18:11.380 SOCs different cores I'll get into those in a few minutes 0:18:13.620,0:18:14.840 there are some SMP support 0:18:14.840,0:18:17.240 for the different multicore chips 0:18:17.240,0:18:19.740 but its not 0:18:19.740,0:18:25.990 well tested we did not enable it because we could not test it to see if it works 0:18:25.990,0:18:31.809 beyond one early experiment we did 0:18:31.809,0:18:33.519 we've fixed a lot of stability 0:18:33.519,0:18:40.519 bugs since then we don't know if you turn it back on whether it'll work or not 0:18:42.020,0:18:42.529 so here's the different 0:18:42.529,0:18:43.890 SOCs that FreeBSD MIPS 0:18:43.890,0:18:49.880 runs on today it runs on the 0:18:49.880,0:18:51.420 ADM5120 in 0:18:51.420,0:18:56.720 united states anyway there are number of 0:18:56.720,0:19:02.670 small routers or tunnel servers 0:19:02.670,0:19:04.750 wireless devices as well 0:19:04.750,0:19:08.450 that have this chip in them it's 0:19:08.450,0:19:11.000 20 or 30 or 40 dollars to purchase 0:19:11.000,0:19:18.000 a high end development board is about 80 to 85 dollars that with a little more memory and a little more flash 0:19:19.860,0:19:25.320 we also do support one of the IDT network processors 0:19:25.320,0:19:28.620 as well as the 0:19:28.620,0:19:30.530 Broadcom 0:19:30.530,0:19:35.010 5000 6000 cores the Broadcom 4000 cores has a 0:19:35.010,0:19:37.930 number of bugs in its 0:19:37.930,0:19:40.710 pipelining so it requires 0:19:40.710,0:19:41.370 changes to 0:19:41.370,0:19:44.280 GCC and Binutils to schedule 0:19:44.280,0:19:51.280 instructions correctly and appropriately unfortunately these compilers changes whenever 0:19:53.250,0:19:59.440 donated back to the SFS or the FSF for inclusion 0:19:59.440,0:20:01.300 in these projects 0:20:01.300,0:20:03.890 in a timely fashion so they are not in 0:20:03.890,0:20:10.150 the base and the patches don't exist for the versions of the compilers that are in FreeBSD 0:20:10.150,0:20:14.600 so if we are to support this kernel 0:20:14.600,0:20:15.909 we would need some kind of 0:20:15.909,0:20:20.810 out of tree tools 0:20:20.810,0:20:25.620 and general the project tries not to do that although as we move in 0:20:25.620,0:20:32.620 to the embedded environment we may be forced to do that more and more 0:20:32.790,0:20:39.790 so here is a rundown of the different pieces of the ADM5120 that supports the basics 0:20:46.919,0:20:51.200 the only thing that these boards have is that is significant that 0:20:51.200,0:20:55.050 isn't supported right now is USB nobody who has done 0:20:55.050,0:20:56.390 development has had a board 0:20:56.390,0:21:03.390 with USB on it so we don't support USB 0:21:04.660,0:21:07.620 on the IDT the NIC 0:21:07.620,0:21:14.620 and the serial console are working there's 0:21:15.700,0:21:17.450 support for adding devices 0:21:17.450,0:21:22.100 on the PCI bus 0:21:22.100,0:21:23.860 the router board 0:21:23.860,0:21:25.090 the RB532 0:21:25.090,0:21:30.090 is one of the commercially available boards that has 0:21:30.090,0:21:31.470 this chip on it 0:21:31.470,0:21:34.130 it's not a particularly common chip 0:21:34.130,0:21:35.670 but Wojciech Koszek 0:21:35.670,0:21:37.590 had one of these laying around 0:21:37.590,0:21:39.160 and decided hey 0:21:39.160,0:21:40.960 I'll get FreeBSD working on that it's 0:21:40.960,0:21:46.340 well documented and he was able to do this 0:21:46.340,0:21:51.770 again some more details on the Broadcom platform 0:21:51.770,0:21:56.760 interesting things to note here is that 0:21:56.760,0:22:03.440 Broadcom MIPS have a relatively unique they have a way you could actually 0:22:03.440,0:22:07.160 enumerate all the devices that are into the system object 0:22:07.160,0:22:11.770 and most of the other systems that are available 0:22:11.770,0:22:13.120 you'll have 0:22:13.120,0:22:16.990 to know oh I'm on MPC8548 0:22:16.990,0:22:22.190 our PC I have this device at this address this device at this address and whatever 0:22:22.190,0:22:23.780 you have to compile in tables 0:22:23.780,0:22:25.370 but Broadcom products 0:22:25.370,0:22:28.289 you can go and ask 0:22:28.289,0:22:35.210 what devices are out there? and enumerate through them 0:22:35.210,0:22:37.950 and we support doing that 0:22:37.950,0:22:41.020 in a lot of ways it's like PCI where you can ask each individual device 0:22:41.020,0:22:44.620 what's your ID and it comes back with an ID 0:22:44.620,0:22:51.620 you can use that to select the proper driver for each of the devices 0:22:53.429,0:22:55.560 so work is currently underway 0:22:55.560,0:22:59.890 and this is in various degrees of completion 0:22:59.890,0:23:01.870 there's some work being done to get the Broadcom 0:23:01.870,0:23:04.309 6000 series working well 0:23:04.309,0:23:08.020 0:23:08.020,0:23:08.580 Cavium networks 0:23:08.580,0:23:09.249 has a full 0:23:09.249,0:23:11.380 Octeon port that supports 0:23:11.380,0:23:12.510 all the 0:23:12.510,0:23:14.710 members of the Octeon family 0:23:14.710,0:23:15.809 supports all the auxiliary 0:23:15.809,0:23:17.250 hardware that is on that 0:23:17.250,0:23:19.610 all the network packet 0:23:19.610,0:23:26.610 engine technology all the crypto technology that the MIPS 0:23:26.640,0:23:28.990 multi-core MIPS products have 0:23:28.990,0:23:31.820 one problem though is 0:23:31.820,0:23:34.320 that it was taken with the old MIPS 2 snapshot 0:23:34.320,0:23:40.770 and it is against FreeBSD that's about 22 months old at this point 0:23:40.770,0:23:44.590 so even though the port itself is fairly good but it's a fairly old version of 0:23:44.590,0:23:48.880 FreeBSD so 0:23:48.880,0:23:51.840 it's a lot of work to bring it forward I'm working on the Audigy 0:23:51.840,0:23:54.950 Au 1xxx port 0:23:54.950,0:24:00.830 which is languishing for a lack lo time 0:24:00.830,0:24:07.380 Oleksandr Tymoshenko has gone to work with a couple of other people 0:24:07.380,0:24:14.380 whose name are escaping me Dan White 0:24:18.880,0:24:19.410 um-hum 0:24:19.410,0:24:21.730 yeah just look Dan on the new Atheros 0:24:21.730,0:24:22.880 AR7000 and 0:24:22.880,0:24:28.530 AR9000 parts so this is just a 0:24:28.530,0:24:30.300 wondering list of what I'm working on 0:24:30.300,0:24:37.300 you can read it or I can read it so I'll just go to the next slide 0:24:44.260,0:24:46.660 we have a pseudo console working 0:24:46.660,0:24:49.040 we have the NIC working 0:24:49.040,0:24:51.920 there's some stability issues 0:24:51.920,0:24:52.769 that Oleksandr is 0:24:52.769,0:24:58.990 looking into he's trying to figure out 0:24:58.990,0:25:01.530 what's causing the underling 0:25:01.530,0:25:03.910 stability issues 0:25:03.910,0:25:07.299 this work is being done in the 0:25:07.299,0:25:10.890 FreeBSD SVN repository 0:25:10.890,0:25:14.460 although not in the naming tree one of the things that 0:25:14.460,0:25:21.460 the project has done is it's transition of most of the use of Perforce into subversion so there's a project MIPS tree that this work is being done in if you want to 0:25:26.770,0:25:31.490 check it out and look at it for yourself 0:25:31.490,0:25:34.470 the Cavium port I talked about a little bit 0:25:34.470,0:25:35.470 it supports all 3 ABIs 0:25:35.470,0:25:39.450 but its either or it doesn't support 0:25:39.450,0:25:41.429 all 3 at the same time 0:25:41.429,0:25:43.700 you can either have an o32 system 0:25:43.700,0:25:48.770 or a n32 system or a n64 system 0:25:48.770,0:25:50.410 so that portion of the 0:25:50.410,0:25:52.250 port needs some additional work 0:25:52.250,0:25:58.340 before we can roll it into the system 0:25:58.340,0:26:01.940 one of the other interesting things is 0:26:01.940,0:26:06.740 that this port you can build entirely on a Linux system you don't need a FreeBSD system to host it 0:26:06.740,0:26:08.830 normally with FreeBSD 0:26:08.830,0:26:10.230 you have to build FreeBSD on FreeBSD 0:26:10.230,0:26:12.620 people have either 0:26:12.620,0:26:14.520 FreeBSD boxes or virtual machines 0:26:14.520,0:26:19.670 around to build it 0:26:19.670,0:26:21.990 and ported in network tools 0:26:21.990,0:26:23.210 like NetBSD has done 0:26:23.210,0:26:30.210 so that you can build an environment that's more foreign than just FreeBSD 0:26:33.410,0:26:34.980 and the last item 0:26:34.980,0:26:38.650 Cavium has Cavium MIPS isn't like other MIPS 0:26:38.650,0:26:42.330 in some ways there are a number of extensions to the MIPS 0:26:42.330,0:26:46.350 ISAs that Cavium has done to improve performance 0:26:46.350,0:26:49.610 to get better 0:26:49.610,0:26:56.520 validation semantics than you can have from normal MIPS instructions so 0:26:56.520,0:26:58.690 this port also makes use of those 0:26:58.690,0:27:01.980 so that needs the special tools from Cavium 0:27:01.980,0:27:04.140 fortunately Cavium is giving 0:27:04.140,0:27:07.110 their technology pushed back into 0:27:07.110,0:27:10.550 the FSF releases 0:27:10.550,0:27:11.210 unfortunately it's being 0:27:11.210,0:27:12.160 pushed back to GPLP 3 versions 0:27:12.160,0:27:14.940 so 0:27:14.940,0:27:20.480 I'm not sure exactly how the project would deal with that the patches they have 0:27:20.480,0:27:27.480 will use GPLP 2 code, so those can potentially can come into the project 0:27:34.440,0:27:36.710 RMI has a 0:27:36.710,0:27:43.710 FreeBSD 6.1 port they are trying to work with the project to get into the tree 0:27:44.900,0:28:01.900 to put hardware into the project cluster so that they can be well supported by the project that code is not yet available even to the internal developers who are still talking and trying to make it all happen 0:28:08.350,0:28:15.350 I talked about that there's a number of items that needs to be done for 0:28:17.549,0:28:19.830 the next port as it exists in Perforce sorry 0:28:19.830,0:28:21.909 as it exists in the 0:28:21.909,0:28:28.909 FreeBSD subversion tree 0:28:29.460,0:28:30.680 the first one is that we need to 0:28:30.680,0:28:34.820 get a n32 and n64 support working 0:28:34.820,0:28:38.870 along with Multilib support in the tool chain so that 0:28:38.870,0:28:45.870 we can have the different ABIs coexist on the platform 0:28:46.890,0:28:53.890 we have a emulation working configuration with FreeBSD MIPS you can run FreeBSD MIPS emulation if you want to try it out or 0:28:57.530,0:29:02.870 your developing the code QEMU 0:29:02.870,0:29:05.130 is a little bit more widely deployed 0:29:05.130,0:29:07.850 and we'll like to have a configuration that works with that 0:29:07.850,0:29:10.850 right now there's a number of 0:29:10.850,0:29:15.030 problems relating to interrupts that need to be solved and looked at 0:29:15.030,0:29:19.030 we also need to have 64-bit PMAP support with some 0:29:19.030,0:29:25.540 rudiments of that in code right now but it's not enough to bring up 0:29:25.540,0:29:32.540 64-bit kernel 64-bit user space and 0:29:35.630,0:29:36.260 we need to take 0:29:36.260,0:29:43.260 the SMP code either from the Cavium port and make it work so that we can take advantage of the multicore processors 0:29:45.950,0:29:48.040 one of the nice things about the FreeBSD project 0:29:48.040,0:29:49.470 is that we've got 0:29:49.470,0:29:52.750 good scalability on Intel 0:29:52.750,0:29:59.750 and we would presume that scalability would translate to multicore systems in the embedded world and we would like to 0:30:00.010,0:30:02.279 take advantage of all the work that's being done 0:30:02.279,0:30:06.120 on Intel servers or the embedded space try to capture some of the 0:30:06.120,0:30:10.130 embedded market, finally 0:30:10.130,0:30:11.110 we need more 0:30:11.110,0:30:13.640 we need more support from system on chips 0:30:13.640,0:30:15.900 that's going to be a 0:30:15.900,0:30:19.950 a continuing battle 0:30:19.950,0:30:22.480 new parts are released all the time 0:30:22.480,0:30:28.610 FreeBSD needs to be ported to those 0:30:28.610,0:30:32.230 and in the embedded world it's a bit different than on Intel 0:30:32.230,0:30:36.010 where everything has a standard address you have standard devices 0:30:36.010,0:30:39.780 in the embedded world is not very convenient to the embedded designer 0:30:39.780,0:30:46.780 if they can save a little bit of money by putting something in a different location they will so each new 0:30:47.500,0:30:54.500 processors each new chip comes out we need to take the time to sit down and get 0:31:08.580,0:31:09.930 FreeBSD working on that so that wraps up some of the general MIPS history and that kind of occurred status support 0:31:09.930,0:31:10.850 we are going to talk a little bit about some 0:31:10.850,0:31:13.090 of the items that are relatively new to 0:31:13.090,0:31:20.090 FreeBSD that would make it attractive for embedding or running in an embedded environment 0:31:22.400,0:31:29.400 the PowerPC and ARM architecture have a number of things that have been added lately 0:31:30.350,0:31:37.350 Rahul was talking about some of the prescale improvements for multicore high-end powerful chips and 0:31:39.350,0:31:40.750 in an earlier talk 0:31:40.750,0:31:44.410 there's some other things I'll talk about here in a second 0:31:44.410,0:31:45.640 one of the things that I just 0:31:45.640,0:31:48.910 committed to the tree is the ability to build 0:31:48.910,0:31:51.970 the system compilers 0:31:51.970,0:31:56.090 for other architectures so you can use them as a cross compiler 0:31:56.090,0:31:58.060 natively rather than carry inside 0:31:58.060,0:31:59.230 the FreeBSD 0:31:59.230,0:32:03.820 build system again this is similar to 0:32:03.820,0:32:05.930 a lot of the build technology that 0:32:05.930,0:32:12.930 NetBSD has done for sometime 0:32:13.570,0:32:17.049 and that is the first step towards making 0:32:17.049,0:32:19.720 ports cross buildable 0:32:19.720,0:32:20.850 right now there are 0:32:20.850,0:32:27.850 three basic classes of ports there are the ports that have, are really stupid 0:32:27.990,0:32:31.230 that just compiled a lot of .C programs 0:32:31.230,0:32:32.260 those are very easy to point 0:32:32.260,0:32:33.460 and let the cross compiler 0:32:33.460,0:32:38.080 those just work 0:32:38.080,0:32:40.100 there's a class of ports that have been written 0:32:40.100,0:32:41.030 specifically 0:32:41.030,0:32:42.150 to understand 0:32:42.150,0:32:43.970 the new naming conventions 0:32:43.970,0:32:45.099 and you tell it 0:32:45.099,0:32:46.880 I want to compile 0:32:46.880,0:32:49.280 for an ARM 0:32:49.280,0:32:51.090 FreeBSD and it knows how to find 0:32:51.090,0:32:58.090 that compiler and build for it some of those ports work if you pass 0:33:00.020,0:33:07.020 the right configure arguments on the command-line and then there's a class of ports in the middle that they build tools to build the rest of the port and these tools need to run natively 0:33:15.760,0:33:18.160 but they don't have any support for cross building so 0:33:18.160,0:33:20.610 it uses the target compiler 0:33:20.610,0:33:23.010 the try to build the .h files 0:33:23.010,0:33:24.170 or 0:33:24.170,0:33:25.080 whatever for the program to build 0:33:25.080,0:33:28.440 the .h files or whatever they are using but when 0:33:28.440,0:33:29.330 that program goes to run you get an error 0:33:29.330,0:33:30.770 because you can't run 0:33:30.770,0:33:37.010 it on binary on an x86 machine also in the third class of ports are 0:33:37.010,0:33:39.070 there's a number of ports that try to do cross-compilation 0:33:39.070,0:33:41.050 and got it wrong 0:33:41.050,0:33:48.050 so that it just don't work 0:33:49.549,0:33:55.200 so some of the other things in FreeBSD that are interesting to the embedded world are 0:33:55.200,0:33:56.260 we're starting to get a number of support for 0:33:56.260,0:34:01.750 a number of different devices and technologies I went into some of these in my paper 0:34:01.750,0:34:05.310 I'll highlight a few of them here one of the 0:34:05.310,0:34:09.649 most important is the NOR flash support in 0:34:09.649,0:34:14.149 a lot of the low end routers switches that are 0:34:14.149,0:34:17.709 wireless access points that are available 0:34:17.709,0:34:21.219 they load off of NOR flash so having new drivers for that allows us 0:34:21.219,0:34:28.219 to get into that market and we can boot off of them we can manipulate the underlying flash 0:34:31.819,0:34:35.899 we do not yet have a flash file system, we really need one talk 0:34:35.899,0:34:38.159 about that a little bit more 0:34:38.159,0:34:40.119 in the embedded world pin count really really matters so 0:34:40.119,0:34:45.449 a lot of the devices are serial devices 0:34:45.449,0:34:48.329 and FreeBSD has got better 0:34:48.329,0:34:52.749 support for different serial protocols 0:34:52.749,0:34:54.229 that has recently had a new 0:34:54.229,0:35:00.650 USB stack integrated into the tree we've had improvements to the I2C 0:35:00.650,0:35:07.650 support we've got rudiment we've got new support 0:35:07.699,0:35:14.089 for I2S for the sound devices on both embedded systems and coincidentally on old 0:35:14.089,0:35:19.299 Macintosh laptops 0:35:19.299,0:35:20.650 we've got support for 0:35:20.650,0:35:27.149 the SPI bus which is a synchronous bus we've permanently 0:35:27.149,0:35:32.839 flashed a couple of other specialized devices 0:35:32.839,0:35:35.309 for years FreeBSD has also booted well 0:35:35.309,0:35:38.869 with a Compact Flash on a x86 machine 0:35:38.869,0:35:40.839 while in the embedded space 0:35:40.839,0:35:47.449 Compact Flash isn't very well favored because it's a 50 pin interface 0:35:47.449,0:35:52.069 but the SD cards that you find in your cameras are very well favored because it's a 0:35:52.069,0:35:59.069 very small footprint and also it has 9 pins you can implement it in I think 4 or 5 pins 0:36:01.439,0:36:05.449 which makes it fairly attractive and low cost high 0:36:05.449,0:36:07.569 storage medium and FreeBSD 0:36:07.569,0:36:09.829 has a good SD stack 0:36:09.829,0:36:12.819 we recently added support 0:36:12.819,0:36:14.269 for the high-capacity 0:36:14.269,0:36:16.569 SD cards and for MMC cards 0:36:16.569,0:36:19.889 finally a lot of the 0:36:19.889,0:36:20.999 embedded systems 0:36:20.999,0:36:21.989 are wireless 0:36:21.989,0:36:27.459 have some kind of wireless technology 0:36:27.459,0:36:29.039 and FreeBSD has a fairly good 0:36:29.039,0:36:30.730 wireless access stack 0:36:30.730,0:36:35.519 access point stack written by Sam Leffler 0:36:35.519,0:36:39.889 so I'm mentioning it here as well 0:36:39.889,0:36:46.779 there's a number of features that are private or in another stacks 0:36:46.779,0:36:53.779 on PowerPC there's a number of additional cores that are supported 0:36:54.329,0:37:01.329 in addition to the free scale parts which are the top portion the E500 is a it's called a core 0:37:05.219,0:37:12.219 presented this in a lot of detail so I'll just skim over it these are all the ones that are listed 0:37:13.989,0:37:16.019 also been instrumental in driving 0:37:16.019,0:37:18.749 some of the other 0:37:18.749,0:37:21.299 PowerPC chip-sets 0:37:21.299,0:37:24.559 the AMCC 440 0:37:24.559,0:37:30.489 support he's been working on has it booting single user or multiuser? 0:37:30.489,0:37:33.299 has it booting multiuser off of a USB 0:37:33.299,0:37:40.299 flash, last summer he sponsored a student on the E300 yeah it's the E300 and the MPC5200 that is 0:37:47.489,0:37:49.239 to bring up the FreeBSD on 0:37:49.239,0:37:54.889 the MPC5200 which is an E300 core it has an 0:37:54.889,0:37:58.669 number of differences between the 500 core 0:37:58.669,0:38:00.330 like explained there's a 0:38:00.330,0:38:07.330 number of things that are optional or different in the specification you need to code for 0:38:08.910,0:38:14.179 there's been some additional floating point support that's gone in and there's some work underway for the G5 Mac not embedded power platform 0:38:14.179,0:38:16.939 but some additional PowerPC 0:38:16.939,0:38:23.939 infrastructure that's going well 0:38:25.379,0:38:26.599 FreeBSD ARM 0:38:26.599,0:38:33.599 has recently gotten Marvel support for the 0:38:35.599,0:38:39.140 different members of the Orion family 0:38:39.140,0:38:46.140 there's three families of processors Orion, Kirkwood, and Discovery 0:38:46.400,0:38:53.400 that are supported 0:38:55.209,0:38:57.459 managed to get into the tree 0:38:57.459,0:38:59.679 0:38:59.679,0:39:00.539 so 0:39:00.539,0:39:07.539 this company does really good work 0:39:08.629,0:39:15.219 there's also support for Samsung devices that are in the Openmoko 0:39:15.219,0:39:17.029 and a couple of other boards 0:39:17.029,0:39:21.279 Sam Leffler recently added support 0:39:21.279,0:39:28.279 to the IXP 435 on Xscale boards chips 0:39:30.229,0:39:31.219 and there's some more 0:39:31.219,0:39:37.929 underway to get the Cirrus Logic family of ARM 9 parts to get into the tree 0:39:37.929,0:39:44.929 although that work is stalled the team working on it ran out of time 0:39:49.660,0:39:52.629 got interested in other things so there's a number of things 0:39:52.629,0:39:56.029 that the embedded world will be 0:39:56.029,0:40:00.299 very happy if FreeBSD did and 0:40:00.299,0:40:02.759 the biggest one is probably the 0:40:02.759,0:40:04.479 flash file system support 0:40:04.479,0:40:05.840 FreeBSD needs to have a flash 0:40:05.840,0:40:07.380 file system 0:40:07.380,0:40:10.049 with the number of 0:40:10.049,0:40:13.919 people talking about porting one from Linux or 0:40:13.919,0:40:20.919 using the same understructure as one of the Linux file systems no need to completely reinvent the wheel here 0:40:28.069,0:40:29.819 in addition to some of the technical challenges that we have we need 0:40:29.819,0:40:36.819 greater marketing and out reach and documentation to show people that this is possible and a reasonable thing to do 0:40:42.199,0:40:45.199 so that's the end of my talk 0:40:45.199,0:40:46.560 I guess it's now the time for 0:40:46.560,0:40:47.629 questions 0:40:47.629,0:40:51.169 and here are the links again 0:40:51.169,0:40:52.359 to the paper and the slides 0:40:52.359,0:40:59.359 0:40:59.779,0:41:01.520 because that was what I put on the slides 0:41:01.520,0:41:02.370 0:41:02.370,0:41:04.469 will also work equally well 0:41:04.469,0:41:07.249 there's no slide intended 0:41:07.249,0:41:09.700 0:41:09.700,0:41:16.700 there was talk for a while of using DesktopBSD to do the packeting for the embedded to add some nice attributes 0:41:18.219,0:41:19.530 so any 0:41:19.530,0:41:26.530 of those technologies that would enable that anything that works will be a reasonable thing are there any difficulties in common with 0:41:39.599,0:41:41.829 bringing up an embedded system 0:41:41.829,0:41:45.259 from one SSC to another to a third or is every effort 0:41:45.259,0:41:46.719 completely different from 0:41:46.719,0:41:52.579 in terms of implementation and the problems you run into 0:41:52.579,0:41:55.509 I can answer that question yes 0:41:55.509,0:41:59.569 there are a number of things unique to each individual 0:41:59.569,0:42:00.449 SSC there are 0:42:00.449,0:42:03.979 quarks in the silica there are quarks in the 0:42:03.979,0:42:05.089 cores implementation of the architecture you're trying 0:42:05.089,0:42:08.989 to do 0:42:08.989,0:42:11.190 those are almost never uncommon 0:42:11.190,0:42:14.299 writing crazy things 0:42:14.299,0:42:18.409 one of the problems that we have with FreeBSD 0:42:18.409,0:42:19.589 is adding additional 0:42:19.589,0:42:22.159 buses is a big 0:42:22.159,0:42:28.139 copy and paste operation now takes about 200 lines to implement a new bus which should take about 10 0:42:28.139,0:42:33.150 so that's one thing that's in common with those 0:42:33.150,0:42:36.399 between bringing up different processes 0:42:36.399,0:42:37.049 also you tend to use the same kind of tools 0:42:37.049,0:42:39.329 the Jtag Reader 0:42:39.329,0:42:40.400 the Jtag Debugger typically 0:42:40.400,0:42:43.809 on a well resourced operation 0:42:43.809,0:42:50.809 to bring up the different processes the different cores when you are doing the initial core 0:42:54.869,0:43:00.380 the answer varies and people have written entire books on how to do that 0:43:00.380,0:43:05.709 some in common some very unique and will like to make the things 0:43:05.709,0:43:12.709 that are incommon and hard easier to the extent that we can 0:43:13.259,0:43:20.019 one of the things is that it takes a lot of work to bring them up 0:43:20.019,0:43:23.079 are there other questions 0:43:23.079,0:43:26.130 how about the 0:43:26.130,0:43:27.169 multicore scalability 0:43:27.169,0:43:30.339 of FreeBSD 0:43:30.339,0:43:32.219 right now we don't know 0:43:32.219,0:43:36.239 how well FreeBSD MIPS will scale we don't know whether 0:43:36.239,0:43:43.239 there are bottle necks in 0:43:43.939,0:43:50.939 the PMAP for example we think it would be fairly well once we've carried on and get to the start that isn't working now debugged 0:43:55.599,0:43:59.929 It'll be comparable to x86 there may need to be a period of tuning 0:43:59.929,0:44:06.929 because the minute we implement one of the atomic primitives effecting performance of the other potential problem is cache coherency some 0:44:28.900,0:44:32.659 MIPS processors have really good cache coherency for 0:44:32.659,0:44:34.929 multi-processing others push 0:44:34.929,0:44:39.649 a lot of the burden of that on to the implement Cavium will probably scale really 0:44:39.649,0:44:46.649 well because it is fairly coherent the MIPS with a fairly coherent cache policy and architecture 0:44:52.439,0:44:59.439 it's really the only MIPS one multi-core MIPS that I'm really familiar with I don't know how the others will work 0:45:08.429,0:45:09.209 any other questions ? OK thank you very much