Intro: Some musings on the consistency and simplicity of BSD.
A brief interview with Adam Wright from No Starch Press, recorded by Micheal Dexter on behalf of BSDTalk. They talk about recent and future BSD books.
Interview with Dru Lavigne. We talk about her new book "The Best of FreeBSD Basics" and also get an update on some other projects including BSD Certification.
See the following links for more information:
Setting up a central syslog server.
Why I like the CLI:
Getting to know the X Window System.
Make sure you are in a text only mode. You might
need to change how the system boots, or boot into
single user mode.
What's your biggest Time Management problem?
Tom Limoncelli is a FreeBSD user and the author of the O'Reilly book,"Time Management for System Administrators". He'll be giving a brief presentation with highlights from his book then will take questions from the audience. Whether you are a system administrator, a developer (or even a Linux user) this presentation will help you with something more precious a quad-processor AMD box.
Money can buy you bandwidth, but latency is forever!
John Mashey, MIPS
Victor will cover an array of issues connected to Postfix performance tuning, including:
Speaker Bio
Victor Duchovni trained in mathematics, switched
tracks to CS in 1980s leaving Princeton with a
master's degree in mathematics and newly acquired
skills in Unix system administration and system
programming. In 1990 moved to Lehman Brothers,
worked on system management tooling, and network
engineering. Ported "Moira" from MIT to Lehman,
built efficient build systems that predated (and
partly inspired) Jumpstart. In 1994 joined ESM to
market "CMDB" tools to enterprise users, but this
did not pan out, in the mean time learned Tcl, and
contributed bunch of patches to the 7.x early 8.x
TCL releases. In 1997 returned to New York, working
in IT Security at Morgan Stanley since late 1999.
At Morgan Stanley, developed a hobby in perimeter
email security, becoming an active Postfix user and
very soon contributor in May of 2001. In addition
to many smaller feature improvements, contributed
initial implementation of SMTP connection caching,
overhauled and currently maintain LDAP and TLS
support. Made significant design contributions to
queue manager in collaboration with Wietse and
Patrik Raq. In 2.6 contributing support for TLS EC
ciphers and multi-instance management tooling,
ideally also TLS SNI if time permits.
What it is and how can it make system administration less painful
About the speaker:
Larry Ludwig - Principal Consultant/Founder of
Empowering Media. Empowering Media is a consulting
firm and managed hosting provider. Larry Ludwig
has been in the industry for over 15 years as a
system administration and system programmer. He's
had previous experience working for many Fortune
500 corporations and holds a BS in CS from Clemson
University. Larry, along with Eric E. Moore and
Brian Gupta are founding members of the NYC Puppet
usergroup.
Many modern CPUs provide on chip counters for performance events such as retiring instructions and cache misses. The hwpmc driver and libraries in FreeBSD give systems administrators and programmers access to APIs which make it possible to measure performance without modifying source code and with minimal intrusion into application execution. This talk will be a brief introduction to HWPMC, and how to use it.
Bio: George Neville-Neil is the co-author with Kirk McKusick of The Design and Implementation of the FreeBSD Operating System. He works on networking an operating systems for fun and profit.
BSD vs GPL is a sweeping epic, focused on the dichotomy between good and evil. It peers inside the hearts and minds of the creators of these movements and dissects their battle for world domination. No common documentary will dare to follow the path that BSD vs GPL blazes.
Two tools which have become the norm in Linux- and Unix-based environments are SSH for secure communications, and sudo for performing administrative tasks. These are independent programs with substantially different purposes, but they are often used in conjunction. In this talk, I describe a flaw in their interaction, and then present our solution called public-key sudo.
Public-key sudo is an extension to the sudo authentication mechanism which allows for public key authentication using the SSH public key framework. I describe our implementation of a generic SSH authentication module and the sudo modifications required to use this module.
Bio:
Matthew Burnside is a Ph.D. student in the Computer
Science department at Columbia University, in New
York. He works for Professor Angelos Keromytis in
the Network Security Lab. He received his B.A and
M.Eng from MIT in 2000, and 2002, respectively. His
research interests are in network anonymity, trust
management, and enterprise-scale policy enforcement.
Configuration Management with Cfengine
Cfengine is a policy-based configuration management system. Its primary function is to provide automated configuration and maintenance of computers, from a policy specification.
The cfengine project was started in 1993 as a reaction to the complexity and non-portability of shell scripting for Unix configuration management, and continues today. The aim was to absorb frequently used coding paradigms into a declarative, domain-specific language that would offer self-documenting configuration.
about the speaker:
Steven Kreuzer has been working with Open Source
technologies since as long as he can remember,
starting out with a 486 salvaged from a dumpster
behind his neighborhood computer store. In his spare
time he enjoys doing things with technology that
have absolutely no redeeming social value.
This talk is the result of an after-meeting discussion with a few folks, when it became apparent that there is some confusion as to how to deal with OpenBSD in small and large environments. The topic of installation and upgrading came up again. This talk is aimed to hopefully dispel many of the rumors, provide a thorough description and walk through of the various stages of running OpenBSD in any size environment, and some of the features and tools at the administrator's disposal.
Okan Demirmen has been working with UNIX-like systems for as long as he can remember and has found OpenBSD to match some of the same philosophies in which he believes, namely simplicity and correctness, and reap the benefits of such.
Special NYC*BUG meeting with FreeBSD developer Brooks Davis
Since late 2000 we have developed and maintained a
general purpose technical and scientific computing
cluster running the FreeBSD operating system. In
that time we have grown from a cluster of 8 dual
Intel Pentium III systems to our current mix of 64
dual, quad-core Intel Xeon and 289 dual AMD Opteron
systems.
In this talk we reflect on the system architecture as documented in our BSDCon 2003 paper "Building a High-performance Computing Cluster Using FreeBSD" and our changes since that time. After a brief overview of the current cluster we revisit the architectural decisions in that paper and reflect on their long term success. We then discuss lessons learned in the process. Finally, we conclude with thoughts on future cluster expansion and designs.
Bio
Brooks Davis is an Engineering Specialist in the
High Performance Computing Section of the Computer
Systems Research Department at The Aerospace
Corporation. He has been a FreeBSD user since 1994,
a FreeBSD committer since 2001, and a core team
member since 2006. He earned a Bachelors Degree in
Computer Science from Harvey Mudd College in 1998.
His computing interests include high performance computing, networking, security, mobility, and, of course, finding ways to use FreeBSD in all these areas. When not computing, he enjoys reading, cooking, brewing and pounding on red-hot iron in his garage blacksmith shop.
"User Interfaces and How People Think" will introduce concepts of designing software for different users by observing how they think about and do what they do. While much of design today focuses on the front-end of computer systems, there is opportunity to innovate in every area where a human interacts with software.
Bio: Jeffery Mau is a user experience designer with the leading business and technology consulting firm Sapient. He has helped clients create great customer experiences in the financial services, education, entertainment and telecommunications industries. With a passion for connecting people with technology, Jeff specializes in Information Architecture and Business Strategy. Jeff holds a Masters in Design from the IIT Institute of Design in Chicago, Illinois.
Open Meeting on OpenSSH
Febrary's NYCBUG meeting is a broad look at OpenSSH, the de facto method for remote administration and more. OpenSSH celebrated its 8th anniversary this past September, and we thought this would be a great opportunity to discuss OpenSSH, and for others to contribute their hacks and interesting applications.
SSARES: Secure Searchable Automated Remote Email Storage - A usable, secure email system on a remote untrusted server
The increasing centralization of networked services places user data at considerable risk. For example, many users store email on remote servers rather than on their local disk. Doing so allows users to gain the benefit of regular backups and remote access, but it also places a great deal of unwarranted trust in the server. Since most email is stored in plaintext, a compromise of the server implies the loss of confidentiality and integrity of the email stored therein. Although users could employ an end-to-end encryption scheme (e.g., PGP), such measures are not widely adopted, require action on the sender side, only provide partial protection (the email headers remain in the clear), and prevent the users from performing some common operations, such as server-side search.
To address this problem, we present Secure Searchable Automated Remote Email Storage (SSARES), a novel system that offers a practical approach to both securing remotely stored email and allowing privacy-preserving search of that email collection. Our solution encrypts email (the headers, body, and attachments) as it arrives on the server using public-key encryption. SSARES uses a combination of Identity Based Encryption and Bloom Filters to create a searchable index. This index reveals little information about search keywords and queries, even against adversaries that compromise the server. SSARES remains largely transparent to both the sender and recipient. However, the system also incurs significant costs, primarily in terms of expanded storage requirements. We view our work as a starting point toward creating privacy-friendly hosted services.
Angelos Keromytis is an Associate Professor with the Department of Computer Science at Columbia University, and director of the Network Security Laboratory. He received his B.Sc. in Computer Science from the University of Crete, Greece, and his M.Sc. and Ph.D. from the Computer and Information Science (CIS) Department, University of Pennsylvania. He is the author and co-author of more than 100 papers on refereed conferences and journals, and has served on over 40 conference program committees. He is an associate editor of the ACM Transactions on Information and Systems Security (TISSEC). He recently co-authored a book on using graphics cards for security, and is a co-founder of StackSafe Inc. His current research interests revolve around systems and network security, and cryptography.
This talk will be on some of the basics of IPv6 including addressing, subnetting, and tools to test connectivity. There will be a lab (network permitting), and setups for an as of yet undisclosed flavor of BSD as well as some of the well known daemons (Apache 2, SSHD) will be demonstrated. Setting up a BSD OS as an IPv6 router and tunneling system will also be covered.
Bio
Gene Cronk, CISSP-ISSAP, NSA-IAM is a freelance
network security consultant, specializing in *NIX
solutions. He has been working with computers for
well over 20 years, electronics for over 15, and
IPv6 specifically for 4 years. He has given talks
on IPv6 and a multitude of other topics at DefCon,
ShmooCon and other "underground" venues.
Gene is from Jacksonville, FL. When not involved in matters concerning IPv6, he can be found gaming (Anarchy Online), helping out with the Jacksonville Linux User's Group, being one of the benevolent dictators of the Hacker Pimps Security Think Tank, or fixing up his house.
Cryptography has a reputation of slowing down applications. However if done correctly, it can actually be used to improve performance by storing high-value/high-cost results "in public." In addition the same techniques can solve common security problems such as authorization, parameter scanning, and parameter rewriting.
All are welcome - no previous experience with cryptography is required, and the techniques will be presented in a programming-language neutral format.
Nick Galbreath have been working on high performance servers and web security at various high profile startups since 1994 (most recently Right Media). He holds a Master degree of Mathematics from Boston University, and published a book on cryptography. He currently lives in the Lower East Side.
Nagios is a platform for monitoring services and the hosts they reside on. It provides a reasonable tool for monitoring your network and you can not beat the price.
We plan on covering the following topics:
About the Speaker
Marc Spitzer started as a VAX/VMS operator who
taught himself some basic scripting in DCL to help
me remember how to do procedures that did not come
up enough to actually remember all the steps, this
was in 1990. Since then he has worked with HPUX,
Solaris, Windows, Linux, and the BSDs, FreeBSD being
his favorite. He has held a variety of positions,
admin and engineering, where he has been able to
introduce BSD into his work place. He currently
works for Columbia University as a Systems
Administrator.
He is a founding member of NYCBUG and LispNYC and on the board of UNIGroup.
Most of his career has been building tools to solve operational problems, with extra effort going to the ones that irritated him personally. He takes a great deal of pride in not needing a budget to solve most problems.
"The Real Unix Tradition"
UNIX hackers, all standing on the shoulders of giants.
"...the number of UNIX installations has grown to 10, with more expected..." - Dennis Ritchie and Ken Thompson, June 1972
"Well, it was all Open Source, before anybody really called it that". - Brian Redman, 2003
UNIX is the oldest active and growing computing culture alive today. From it's humble roots in the back room at Bell Laboratories, to today's global internet infrastructure- UNIX has consistently been at the core of major advances in computing. Today, the BSD legacy is the most direct continuation of the most successful principles in UNIX, and continues to lead major advances in computing.
Why? What's so great about UNIX?
This lecture aims to prove that UNIX history is surprisingly useful (and fun)- for developers, sysadmins, and anyone working with BSD systems.
About the speaker
Isaac Levy, (ike) is a freelance BSD hadker based
in NYC. He runs Diversaform Inc. as an engine to
make his hacking feed itself, (and ike). Diversaform
specializes in *BSD based solutions, providing 'IT
special weapons and tatics' for various sized
business clients, as well as running a small
high-availability datacenter operation from lower
Manhattan. With regard to FreeBSD jail(8), ike was
a partner in the first jail (8)-based web hosting
ISP in America, iMeme, and has been developing
internet applications in and out of jails since
1999. Isaac is a proud member of NYC*BUG (the New
York City *BSD Users Group), and a long time member
of LESMUUG, (the Lower East Side Mac Unix Users
Group).
Protecting your servers, workstations and networks can only go so far. Attacks which consume your available Internet-facing bandwidth, or overpower your CPU, can still take you offline. His presentation will discuss techniques for mitigating the effects of such attacks on servers designed to provide network intensive services such as HTTP or routing.
About the speaker
Steven Kreuzer is currently employed by Right Media
as a Systems Administrator focusing on building and
managing high transaction infrastructures around
the globe. He has been working with Open Source
technologies since as long as he can remember,
starting out with a 486 salvaged from a dumpster
behind his neighborhood computer store. In his spare
time he enjoys doing things with technology that
have absolutely no redeeming social value.
The fourth annual pkgsrcCon is April 27-29 in Barcelona. As might be expected when brains congregate, pkgsrcCon traditionally results in a flurry of activity toward new directions and initiatives. Mere hours after returning to New York, Amitai will give us a recap of the proceedings, including his presentation, "Packaging djbware."
Amitai Schlair is a pkgsrc developer who has worked in such diverse areas as Mac OS X platform support and packages of software by Dan Bernstein. His full-time undergraduate studies at Columbia are another contributing factor to his impending insanity. He consults in software and IT.
This presentation was inspired by the recent Subversion presentation. It will talk about the origins of OpenRCS and OpenCVS, its real-world usage in the OpenBSD project, and why OpenBSD will continue to use CVS.
Ray is an OpenBSD developer who uses Subversion by day, CVS by night. Taking the phrase "complexity is the enemy of security" to heart, he believes that the beauty of UNIX's security is in its simplicity.
Integrated Enterprise Security Management
Security policies are a key component in protecting enterprise networks. But, while there are many diverse defensive options available, current models and mechanisms for mechanically-enforced security policies are limited to traditional admission-based access control. Defensive capabilities include among others logging, firewalls, honeypots, rollback/recovery, and intrusion detection systems, while policy enforcement is essentially limited to one-off access control. Furthermore, access-control mechanisms operate independently on each service, which can (and often does) lead to inconsistent or incorrect application of the intended system-wide policy. We propose a new scheme for global security policies. Every policy decision is made with near-global knowledge, and re-evaluated as global knowledge changes. Using a variety of actuators, we make the full array of defensive capabilities available to the global policy. Our goal is a coherent, enterprise-wide response to any network threat.
Biography
Matthew Burnside is a Ph.D. student in the Computer
Science department at Columbia University, in New
York. He works for Professor Angelos Keromytis in
the Network Security Lab. He received his B.A and
M.Eng from MIT in 2000, and 2002, respectively. His
main research interests are in computer security,
trust management, and network anonymity.
The presentation will discuss Subversion from both client and server points of view. It will show how to create repositories and how to make them accessible over the network using different access schemes like http://, file:// or svn://. Pointers are given on securing the repositories and on authenticating and authorizing the clients. Next, the presentation shows how an user interacts with the repository and describes some of the important Subversion client commands. Finally, it deals with administrating the repository using "hook scripts".
Ivan Ivanov is generally interested in Version Control Systems since his student years in Sofia University, Bulgaria, where he set up and maintained a CVS server for an academic project. When Subversion became a fact and proved to be "a better CVS" he researched it and last year deployed it for his NYC-based employer Ariel Partners (http://www.arielpartners.com/). He integrated the Subversion repositories with Apache Web Server over https to enable a reliable and secure way to access them from any point.
BSD is Dying
A Cautionary Tale of Sex and Greed
Jason Dixon
October 28, 2006
First and foremost, I would like to thank the unique presentation styles of Dick Hardt and Lawrence Lessig for inspiring me to create this presentation.
The following videos were created by exporting the original Keynote presentation slides into QuickTime video, then manually synchronizing them using iMovie HD with the audio recordings captured by Nikolai Fetissov. They were then exported into QuickTime, mpeg4 (H.264/AAC), and iPod movie formats. If you are having difficulties with the MP4 copy, and are unable to view QuickTime movies, please contact me and I'll try to assist.
This week we talk about
COMPLETE Hard Disk Encryption with FreeBSD, by Marc Schiesser
Learn how to effectively protect not only your data but also your applications.
Most technologies and techniques intended for securing digital data focus on protection while the machine is turned on mostly by defending against remote attacks. An attacker with physical access to the machine, however, can easily circumvent these defenses by reading out the contents of the storage medium on a different, fully accessible system or even compromise program code on it in order to leak encrypted information. Especially for mobile users, that threat is real. And for those carrying around sensitive data, the risk is most likely high. This talk will introduce a method of mitigating that particular risk by protecting not only the data through encryption, but also the applications and the operating system from being compromised while the machine is turned off.
VG sponsored the creation of a web-accellerator called "Varnish" because Squid was too slow for them. Varnish is being developed by Poul-Henning Kamp and the Norwegian Linux consultancy Linpro. This is the releaseparty for version 1.0.
The first half of the talk will introduce Varnish and present some of the novel features it brings to the business of web-serving.
The second half of the talk, using Varnish as the example, will show ways to get the most performance out of modern hardware and operating systems.
(The English text starts at about 5 minutes in the stream)
[Commentary still being written]
For RSS readers: Please note that the download URL is an FTP site.
This is an extra track by the artist Ty Semaka (who really has "had Puffy on his mind") which we included on the audio CD.
This song details the process that Ty has to go through to make the art and music for each OpenBSD release. Ty and Theo really do go to a (very specific) bar and discuss what is going on in the project, and then try to find a theme that will work...
For RSS readers: Please note that the download URL is an FTP site.
Nearly 10 years ago Kirk McKusick wrote a history of the Berkeley Unix distributions for the O'Reilly book "Open Sources: Voices from the Open Source Revolution". We recommend you read his story, entitled "Twenty Years of Berkeley Unix From AT&T-Owned to Freely Redistributable" first, to see how Kirk remembers how we got here. Sadly, since it showed up in book form originally, this text has probably not been read by enough people.
The USL(AT&T) vs BSDI/UCB court case settlement documents were not public until recently; their disclosure has made the facts more clear. But the story of how three people decided to free the BSD codebase of corporate pollution -- and release it freely -- is more interesting than the lawsuit which followed. Sure, a stupid lawsuit happened which hindered the acceptance of the BSD code during a critical period. But how did a bunch of guys go through the effort of replacing so much AT&T code in the first place? After all, companies had lots of really evil lawyers back then too -- were they not afraid?
After a decade of development, most of the AT&T code had already been replaced by university researchers and their associates. So Keith Bostic, Mike Karels and Kirk McKusick (the main UCB CSRG group) started going through the 4.3BSD codebase to cleanse the rest. Keith, in particular, built a ragtag team (in those days, USENIX conferences were a gold mine for such team building) and led these rebels to rewrite and replace all the Imperial AT&T code, piece by piece, starting with the libraries and userland programs. Anyone who helped only got credit as a Contributor -- people like Chris Torek and a cast of .. hundreds more.
Then Mike and Kirk purified the kernel. After a bit more careful checking, this led to the release of a clean tree called Net/2 which was given to the world in June 1991 -- the largest dump of free source code the world had ever received (for those days -- not modern monsters like OpenOffice).
Some of these ragtags formed a company (BSDi) to sell a production system based on this free code base, and a year later Unix System Laboratories (basically AT&T) sued BSDi and UCB. Eventually AT&T lost and after a few trifling fixes (described in the lawsuit documents) the codebase was free. A few newer developments (and more free code) were added, and released in June 1994 as 4.4BSD-Lite. Just over 14 years later OpenBSD is releasing its own 4.4 release (and for a lot less than $1000 per copy).
The OpenBSD 4.4 release is dedicated to Keith Bostic, Mike Karels, Kirk McKusick, and all of those who contributed to making Net/2 and 4.4BSD-Lite free.
We are just plain tired of being lectured to by a man who is a lot like Naomi Campbell.
In 1998 when a United Airlines plane was waiting in the queue at Washington Dulles International Airport for take-off to New Orleans (where a Usenix conference was taking place), one man stood up from his seat, demanded that they stop waiting in the queue and be permitted to deplane. Even after orders from the crew and a pilot from the cockpit he refused to sit down. The plane exited the queue and returned to the airport gangway. Security personnel ran onto the plane and removed this man, Richard Stallman, from the plane. After Richard was removed from the plane, everyone else stayed onboard and continued their journey to New Orleans. A few OpenBSD developers were on that same plane, seated very closeby, so we have an accurate story of the events.
This is the man who presumes that he should preach to us about morality, freedom, and what is best for us. He believes it is his God-given role to tell us what is best for us, when he has shown that he takes actions which are not best for everyone. He prefers actions which he thinks are best for him -- and him alone -- and then lies to the public. Richard Stallman is no Spock.
We release our software in ways that are maximally free. We remove all restrictions on use and distribution, but leave a requirement to be known as the authors. We follow a pattern of free source code distribution that started in the mid-1980's in Berkeley, from before Richard Stallman had any powerful influence which he could use so falsely.
We have a development sub-tree called "ports". Our "ports" tree builds software that is 'found on the net' into packages that OpenBSD users can use more easily. A scaffold of Makefiles and scripts automatically fetch these pieces of software, apply patches as required by OpenBSD, and then build them into nice neat little tarballs. This is provided as a convenience for users. The ports tree is maintained by OpenBSD entirely separately from our main source tree. Some of the software which is fetched and compiled is not as free as we would like, but what can we do. All the other operating system projects make exactly the same decision, and provide these same conveniences to their users.
Richard felt that this "ports tree" of ours made OpenBSD non-free. He came to our mailing lists and lectured to us specifically, yet he said nothing to the many other vendors who do the same; many of them donate to the FSF and perhaps that has something to do with it. Meanwhile, Richard has personally made sure that all the official GNU software -- including Emacs -- compiles and runs on Windows.
That man is a false leader. He is a hypocrite. There may be some people who listen to him. But we don't listen to people who do not follow their own stupid rules.
Those of us who work on OpenBSD are often asked why we do what we do. This song's lyrics express the core motivations and goals which have remained unchanged over the years - secure, free, reliable software, that can be shared with anyone. Many other projects purport to share these same goals, and love to wrap themselves in a banner of "Open Source" and "Free Software". Given how many projects there are one would think it might be easy to stick to those goals, but it doesn't seem to work out that way. A variety of desires drag many projects away from the ideals very quickly.
Much of any operating system's usability depends on device support, and there are some very tempting alternative ways to support devices available to those who will surrender their moral code. A project could compromise by entering into NDA agreements with vendors, or including binary objects in the operating system for which no source code exists, or tying their users down with contract terms hidden inside copyright notices. All of these choices surrender some subset of the ideals, and we simply will not do this. Sure, we care about getting devices working, but not at the expense of our original goals.
Of course since "free to share with anyone" is part of our goals, we've been at the forefront of many licensing and NDA issues, resulting in a good number of successes. This success had led to much recognition for the advancement of Free Software causes, but has also led to other issues.
We fully admit that some BSD licensed software has been taken and used by many commercial entities, but contributions come back more often than people seem to know, and when they do, they're always still properly attributed to the original authors, and given back in the same spirit that they were given in the first place.
That's the best we can expect from companies. After all, we make our stuff so free so that everyone can benefit -- it remains a core goal; we really have not strayed at all in 10 years. But we can expect more from projects who talk about sharing -- such as the various Linux projects.
Now rather than seeing us as friends who can cooperatively improve all codebases, we are seen as foes who oppose the GPL. The participants of "the race" are being manipulated by the FSF and their legal arm, the SFLC, for the FSF's aims, rather than the goal of getting good source into Linux (and all other code bases). We don't want this to come off as some conspiracy theory, but we simply urge those developers caution -- they should ensure that the path they are being shown by those who have positioned themselves as leaders is still true. Run for yourself, not for their agenda.
The Race is there to be run, for ourselves, not for others. We do what we do to run our own race, and finish it the best we can. We don't rush off at every distraction, or worry how this will affect our image. We are here to have fun doing right.
As developers of a free operating system, one of our prime responsibilities is device support. No matter how nice an operating system is, it remains useless and unusable without solid support for a wide percentage of the hardware that is available on the market. It is therefore rather unsurprising that more than half of our efforts focus on various aspects relating to device support.
Most parts of the operating system (from low kernel, through to libraries, all the way up to X, and then even to applications) use fairly obvious interface layers, where the "communication protocols" or "argument passing" mechanisms (ie. APIs) can be understood by any developer who takes the time to read the free code. Device drivers pose an additional and significant challenge though: because many vendors refuse to document the exact behavior of their devices. The devices are black boxes. And often they are surprisingly weird, or even buggy.
When vendor documentation does not exist, the development process can become extremely hairy. Groups of developers have found themselves focused for months at a time, figuring out the most simple steps, simply because the hardware is a complete mystery. Access to documentation can ease these difficulties rapidly. However, getting access to the chip documentation from vendors is ... almost always a negotiation. If we had open access to documentation, anyone would be able to see how simple all these devices actually are, and device driver development would flourish (and not just in OpenBSD, either).
When we proceed into negotiations with vendors, asking for documentation, our position is often weak. One would assume that the modern market is fair, and that selling chips would be the primary focus of these vendors. But unfortunately a number of behemoth software vendors have spent the last 10 or 20 years building political hurdles against the smaller players.
A particularly nasty player in this regard has been the Linux vendors and some Linux developers, who have played along with an American corporate model of requiring NDAs for chip documentation. This has effectively put Linux into the club with Microsoft, but has left all the other operating system communities -- and their developers -- with much less available clout for requesting documentation. In a more fair world, the Linux vendors would work with us, and the device driver support in all free operating systems would be fantastic by now.
We only ask that users help us in changing the political landscape.
The last 10 years, every 6 month period has (without fail) resulted in an official OpenBSD release making it to the FTP servers. But CDs are also manufactured, which the project sells to continue our development goals.
While tests of the release binaries are done by developers around the world, Theo and some developers from Calgary or Edmonton (such as Peter Valchev or Bob Beck) test that the discs are full of (only) correct code. Ty Semaka works for approximately two months to design and draw artwork that will fit the designated theme, and coordinates with his music buddies to write and record a song that also matches the theme.
Then the discs and all the artwork gets delivered to the plant, so that they can be pressed in time for an official release date.
This release, instead of bemoaning vendors or organizations that try to make our task of writing free software more difficult, we instead celebrate the 10 years that we have been given (so far) to write free software, express our themes in art, and the 5 years that we have made music with a group of talented musicians.
OpenBSD developers have been torturing each other for years now with Humppa-style music, so this release our users get a taste of this too. Sometimes at hackathons you will hear the same songs being played on multiple laptops, out of sync. It is under such duress that much of our code gets written.
We feel like Pufferix and Bobilix delivering The Three Discs of Freedom to those who want them whenever the need arises, then returning to celebrate the (unlocked) source tree with all the other developers.
For RSS readers: Please note that the download URL is an FTP site.
Last month I attended a meeting of the Ottawa Amateur Radio Club (OARC) as a member of my local BUG was giving a presentation on Ham Radio on FreeBSD. Diane Bruce, call sign VA3DB, has had her operator license since 1969 and is well known in the BSD community and for the development of ircd-hybrid. In the past year she has assisted in the creation of the Hamradio category in the FreeBSD ports tree and has become the maintainer of over 20 of the hamradio ports. She also contributed to the FreeBSD entry at Hampedia, the Wikipedia for ham operators.
Her presentation slides are a great introduction to the various ham utilities which are available, including both descriptions and screenshots of the utilities in action.
pfSense: 2.0 and beyond
From firewall distribution to appliance building platform
pfSense is a BSD licensed customized distribution of FreeBSD tailored for use as a firewall and router. In addition to being a powerful, flexible firewalling and routing platform, it includes a long list of related features and a package system allowing further expandability without adding bloat and potential security vulnerabilities to the base distribution.
This session will start with an introduction to the project and its common uses, which have expanded considerably beyond firewalling. We will cover much of the new functionality coming in the 2.0 release, which contains significant enhancements to nearly every portion of the system as well as numerous new features.
While the primary function of the project is a firewalling and routing platform, with changes coming in pfSense 2.0, it has also become an appliance building framework enabling the creation of customized special purpose appliances. The m0n0wall code where pfSense originated has proved popular for this purpose, with AskoziaPBX and FreeNAS also based upon it, in addition to a number of commercial solutions. The goal of this appliance building framework is to enable creation of projects such as these without having to fork and maintain another code base. The existing appliances, including a DNS server using TinyDNS, VoIP with FreeSWITCH, and others will be discussed. For those interested in creating appliances, an overview of the process will be provided along with references for additional information.
GEOM based disk schedulers for FreeBSD
The high cost of seek operations makes the throughput of disk devices very sensitive to the offered workload. A disk scheduler can then help reorder requests to improve the overall throughput of the device, or improve the service guarantees for individual users, or both.
Research results in recent years have introduced, and proven the effectiveness of, a technique called "anticipatory scheduling". The basic idea behind this technique is that, in some cases, requests that cause a seek should not be served immediately; instead, the scheduler should wait for a short period of time in case other requests arrive that do not require a seek to be served. With many common workloads, dominated by sequential synchronous requests, the potential loss of throughput caused by the disk idling times is more than balanced by the overall reduction of seeks.
While a fair amount of research on disk scheduling has been conducted on FreeBSD, the results were never integrated in the OS, perhaps because the various prototype implementations were very device-specific and operated within the device drivers. Ironically, anticipatory schedulers are instead a standard part of Linux kernels.
This talk has two major contributions:
First, we will show how, thanks to the flexibility of the GEOM architecture, an anticipatory disk scheduling framework has been implemented in FreeBSD with little or no modification to a GENERIC kernel. While these schedulers operate slightly above the layer where one would naturally put a scheduler, they can still achieve substantial performance improvements over the standard disk scheduler; in particular, even the simplest anticipatory schedulers can prevent the complete trashing of the disk performance that often occurs in presence of multiple processes accessing the disk.
Secondly, we will discuss how the basic anticipatory scheduling technique can be used not only to improve the overall throughput of the disk, but also to give service guarantees to individual disk clients, a feature that is extremely important in practice e.g., when serving applications with pseudo-real-time constraints such as audio or video streaming ones.
A prototype implementation of the scheduler that will be covered in the presentation is available at http://info.iet.unipi.it/~luigi/FreeBSD/
Quiet Computing with BSD
Programming system hardware monitors for quiet computing
In this talk, we will present a detailed overview of the features and common problems of microprocessor system hardware monitors as they relate to the topic of silent computing. In a nutshell, the topic of programmable fan control will be explored.
Silent computing is an important subject as its practice reduces the amount of unnecessary stress and improves the motivation of the workforce, at home and in the office.
Attendees will gain knowledge on how to effectively programme the chips to minimise fan noise and avoid system failure or shutdown during temperature fluctuations, as well as some basic principles regarding quiet computing.
Shortly before the talk, a patch for programming the most popular chips (like those from Winbond) will be released for the OpenBSD operating system, although the talk itself will be more specific to the microprocessor system hardware monitors themselves, as opposed to the interfacing with thereof in modern operating systems like OpenBSD, NetBSD, DragonFly BSD and FreeBSD.
Results of a Security Assessment of the TCP and IP protocols and Common implementation Strategies
Fernando Gont will present the results of security assessment of the TCP and IP protocols carried out on behalf of the United Kingdom's Centre for the Protection of National Infrastructure (Centre for the Protection of National Infrastructure). His presentation will provide an overview of the aforementioned project, and will describe some of the new insights that were gained as a result of this project. Additionally, it will provide an overview of the state of affairs of the different TCP/IP implementations found in BSD operating systems with respect to the aforementioned issues.
During the last twenty years, many vulnerabilities have been identified in the TCP/IP stacks of a number of systems. The discovery of these vulnerabilities led in most cases to reports being published by a number of CSIRTs and vendors, which helped to raise awareness about the threats and the best possible mitigations known at the time the reports were published. For some reason, much of the effort of the security community on the Internet protocols did not result in official documents (RFCs) being issued by the organization in charge of the standardization of the communication protocols in use by the Internet: the Internet Engineering Task Force (IETF). This basically led to a situation in which "known" security problems have not always been addressed by all vendors. In addition, in many cases vendors have implemented quick "fixes" to the identified vulnerabilities without a careful analysis of their effectiveness and their impact on interoperability. As a result, producing a secure TCP/IP implementation nowadays is a very difficult task, in large part because of the hard task of identifying relevant documentation and differentiating between that which provides correct advisory, and that which provides misleading advisory based on inaccurate or wrong assumptions. During 2006, the United Kingdom's Centre for the Protection of National Infrastructure embarked itself in an ambitious and arduous project: performing a security assessment of the TCP and IP protocols. The project did not limit itself to an analysis of the relevant IETF specifications, but also included an analysis of common implementation strategies found in the most popular TCP and IP implementations. The result of the project was a set of documents which identifies possible threats for the TCP and IP protocols and, where possible, proposes counter-measures to mitigate the identified threats. This presentation will describe some of the new insights that were gained as a result of this project. Additionally, it will provide an overview of the state of affairs of the different TCP/IP implementations found in BSD operating systems.
Automating FreeBSD Installations
PXE Booting and install.cfg Demystified
This paper will provide an explanation of the tools involved in performing an automated FreeBSD install and a live demonstration of the process.
FreeBSD's sysinstall provides a powerful and flexible mechanism for automated installs but doesn't get used very often because of a lack of documentation.
Isolating Cluster Jobs for Performance and Predictability
At The Aerospace Corporation, we run a large FreeBSD based computing cluster to support engineering applications. These applications come in all shapes, sizes, and qualities of implementation. To support them and our diverse userbase we have been searching for ways to isolate jobs from one another in ways that are more effective than Unix time sharing and more fine grained than allocating whole nodes to jobs.
In this talk we discuss the problem space and our efforts so far. These efforts include implementation of partial file systems virtualization and CPU isolation using CPU sets.
Multiple Passes of the FreeBSD Device Tree
The existing device driver framework in FreeBSD works fairly well for many tasks. However, there are a few problems that are not easily solved with the current design. These problems include having "real" device drivers for low-level hardware such as clocks and interrupt controllers, proper resource discovery and management, and allowing most drivers to always probe and attach in an environment where interrupts are enabled. I propose extending the device driver framework to support multiple passes over the device tree during boot. This would allow certain classes of drivers to be attached earlier and perform boot-time setup before other drivers are probed and attached. This in turn can be used to develop solutions to the earlier list of problems.
scrypt: A new key derivation function
Doing our best to thwart TLAs armed with ASICs
Password-based key derivation functions are used for two primary purposes: First, to hash passwords so that an attacker who gains access to a password file does not immediately possess the passwords contained therewithin; and second, to generate cryptographic keys to be used for encrypting or authenticating data.
In both cases, if passwords do not have sufficient entropy, an attacker with the relevant data can perform a brute force attack, hashing potential passwords repeatedly until the correct key is found. While commonly used key derivation functions, such as Kamp's iterated MD5, Provos and Mazieres' bcrypt, and RSA Laboratories' PBKDF1 and PBKDF2 make an attempt to increase the difficulty of brute-force attacks, they all require very little memory, making them ideally suited to attack by custom hardware.
In this talk, I will introduce the concepts of memory-hard and sequential memory-hard functions, and argue that key derivation functions should be sequential memory-hard. I will present a key derivation function which, subject to common assumptions about cryptographic hash functions, is provably sequential memory-hard, and a variation which appears to be stronger (but not provably so). Finally, I will provide some estimates of the cost of performing brute force attacks on a variety of password strengths and key derivation functions.
Thinking about thinking in code
Proposed keynote talk
This is not a talk that's specific to any BSD but is a more general talk about how we think about coding and how our thinking changes the way we code.
I compare how we built systems to how other industries build their products and talk about what we can learn from how we work and from how others work as well.
Building products with NetBSD - thin-clients
NetBSD: delivering the goods
This talk will discuss what thin-clients are, why they are useful and why NetBSD is good choice to build such a device.
This talk will provide information on some alternatives and the strengths and weaknesses of NetBSD when used in such a device.
It will discuss problems that needed to be addressed such as how to get a device with rich functionality running from a small amount of flash storage, as well as recent developments in NetBSD that have helped improve the product.
Getting Started in Free and Open Source
Interested in getting involved? But don't really
know where or how to start?
The talk is called "Getting Started in Free and Open Source". It's a talk for beginners who are interested to getting involved but don't really know where or how to start.
We cover the basics of: -why you might want to get involved -what you can get out of participating -more than coding is needed -how to chose a project -how to get started -etiquette of lists and other communication -dos and don't of joining a community
Tracking FreeBSD in a commercial Environment
How to stay current while staying sane
The FreeBSD project publishes two lines of source code: current and stable. All changes must first be committed to current and then are merged into stable. Commercial organizations wishing to use FreeBSD in their products must be aware of this policy. Four different strategies have developed for tracking FreeBSD over time. A company can choose to run only unmodified release versions of FreeBSD. A company may choose to import FreeBSD's sources once and then never merge newer versions. A company can choose to import each new stable branch as it is created, adding its own changes to that branch, as well as integrating new versions from FreeBSD from time to time. A company can track FreeBSD's current branch, adding to it their changes as well as newer FreeBSD changes. Which method a company chooses depends on the needs of the company. These methods are explored in detail, and their advantages and disadvantages are discussed. Tracking FreeBSD's ports and packages is not discussed.
Companies building products based upon FreeBSD have many choices in how to use the projects sources and binaries. The choices range from using unmodified binaries from FreeBSD's releases, to tracking modify FreeBSD heavily and tracking FreeBSD's evolution in a merged tree. Some companies may only need to maintain a stable version of FreeBSD with more bug fixes or customizations than the FreeBSD project wishes to place in that branch. Some companies also wish to contribute some subset of their changes back to the FreeBSD project.
FreeBSD provides an excellent base technology with which to base products. It is a proven leader in performance, reliability and scalability. The technology also offers a very business friendly license that allows companies to pick and choose which changes they wish to contribute to the community rather than forcing all changes to be contributed back, or attaching other undesirable license conditions to the code.
However, the FreeBSD project does not focus on integration of its technology into customized commercial products. Instead, the project focuses on producing a good, reliable, fast and scalable operating system and associated packages. The project maintains two lines of development. A current branch, where the main development of the project takes place, and a stable branch which is managed for stability and reliability. While the project maintains documentation on the system, including its development model, relatively little guidance has been given to companies in how to integrate FreeBSD into their products with a minimum of trouble.
Developing a sensible strategy to deal with both these portions of FreeBSD requires careful planning and analysis. FreeBSD's lack of guidelines to companies leaves it up to them to develop a strategy. FreeBSD's development model differs from some of the other Free and Open Source projects. People familiar with those systems often discover that methods that were well suited to them may not work as well with FreeBSD's development model. These two issues cause many companies to make poor decisions without understanding the problems that lie in their future.
Very little formal guidance exists for companies wishing to integrate FreeBSD into their products. Some email threads can be located via a Google search that could help companies, but many of them are full of contradictory information, and it is very disorganized. While the information about the FreeBSD development process is in the FreeBSD handbook, the implications of that process for companies integrating FreeBSD into their products are not discussed.
PC-BSD - Making FreeBSD on the desktop a reality
FreeBSD on the Desktop
While FreeBSD is a all-around great operating system, it is greatly lagging behind in desktop appeal. Why is this? In this talk, we will take a look at some of the desktop drawbacks of FreeBSD, and how are are attempting to fix them through PC-BSD.
FreeBSD has a reputation for its rock-solid reliability, and top-notch performance in the server world, but is noticeably absent when it comes to the vast market of desktop computing. Why is this? FreeBSD offers many, if not almost all of the same open-source packages and software that can be found in the more popular Linux desktop distributions, yet even with the speed and reliability FreeBSD offers, a relative few number of users are deploying it on their desktops.
In this presentation we will take a look at some of the reasons why FreeBSD has not been as widely adopted in the desktop market as it has on the server side. Several of the desktop weaknesses of FreeBSD will be shown, along with how we are trying to fix these short-comings through a desktop-centric version of FreeBSD, known as PC-BSD. We will also take a look at the package management system employed by all open-source operating systems alike, and some of the pitfalls it brings, which may hinder widespread desktop adoption.
Implementation of TARGET_MODE applications
How we used TARGET_MODE in the kernel to create and
interesting product
This presentation will cover a real world implementation of the TARGET_MODE infrastructure in the kernel (stable/6). Topics to include: drivers used (isp, aic7xxx, firewire). scsi_target userland code vs kernel drivers missing drivers (4/8G isp support, iSCSI target)
Target Mode describes a feature within certain drivers that allows a FreeBSD system to emulate a Target in the SCSI sense of the word. By recompiling your kernel with this feature enabled, it permits one to turn a FreeBSD system into an external hard disk. This feature of the FreeBSD kernel provides many interesting implementations and is highly desirable to many organizations whom run FreeBSD as their platform.
I have been tasked with the maintenance of a proprietary target driver that interfaces with the FreeBSD kernel to do offsite data mirroring at the block level. This talk will discuss the implementation of that kernel mode driver and the process my employer went through to implement a robust and flexible appliance.
Since I took over the implementation, we have implemented U160 SCSI(via aic7xxx), 2G Fibre Channel(via isp) and Firewire 400 (via sbp_targ). Each driver has it's own subtleties and requirements. I personally enhanced the existing Firewire target driver and was able to get some interesting results.
I hope to demonstrate a functional Firewire 400/800 target and show how useful this application can be for the embedded space. Also, I wish to demonstrate the need for iSCSI. USB and 4/8G Fibre Channel target implementations that use the TARGET_MODE infrastructure that is currently in place to allow others to expand their various interface types.
The presentation should consist of a high level overview, followed by detailed implementation instructions with regards to the Firewire implementation and finish up with a hands-on demonstration with a FreeBSD PC flipped into TARGET_MODE and a Mac.
Understanding and Tuning SCHED_ULE
With the advent of widespread SMP and multicore CPU architectures it was necessary to implement a new scheduler in the FreeBSD operating system. The SCHEDULE scheduler was added for the 5 series of FreeBSD releases and has now matured to the point where it is the default scheduler in the 7.1 release. While scheduling processes was a difficult enough task in the uniprocessor world, moving to multiple processors, and multiple cores, has significantly increased the number of problems that await engineers who wish to squeeze every last ounce of performance out of their system. This talk will cover the basic design of SCHEDULE and focus a great deal of attention on how to tune the scheduler for different workloads, using the sysctl interfaces that have been provided for that purpose.
Understanding and tuning a scheduler used to be done only by operating systems designers and perhaps a small minority of engineers focusing on esoteric high performance systems. With the advent of widespread multi-processor and multi-core architectures it has become necessary for more users and administrators to decide how to tune their systems for the best performance. The SCHEDULE scheduler in FreeBSD provides a set of sysctl interfaces for tuning the scheduler at run time, but in order to use these interfaces effectively the scheduling process must first be understood. This presentation will give an overview of how SCHEDULE works and then will show several examples of tuning the system with the interfaces provided.
The goal of modifying the scheduler's parameters is to change the overall performance of programs on the system. One of the first problems presented to the person who wants to tune the scheduler is how to measure the effects of their changes. Simply tweaking the parameters and hoping that the tweaks will help is not going to lead to good results. In our recent experiments we have used the top(1) program to measure our results.
Improving the FreeBSD TCP Implementation.
An update on all things TCP in FreeBSD and how they
affect you.
My involvement in improving the FreeBSD TCP stack has continued this past year, with much of the work targeted at FreeBSD 8. This talk will cover what these changes entail, why they are of interest to the FreeBSD community and how they help to improve our TCP implementation.
It has been a busy year since attending my inaugural BSDCan in 2008, where I talked about some of my work with TCP in FreeBSD.
I have continued the work on TCP analysis/debugging tools and integrating modular congestion control into FreeBSD as part of the NewTCP research project. I will provide a progress update on this work.
Additionally, a grant win from the FreeBSD Foundation to undertake a project titled "Improving the FreeBSD TCP Implementation" at Swinburne University's Centre for Advanced Internet Architectures has been progressing well. The project focuses on bringing TCP Appropriate Byte Counting (RFC 3465), reassembly queue auto-tuning and integration of low-level analysis/debugging tools to the base system, all of which I will also discuss.
Journaling FFS with WAPBL
NetBSD 5 is the first NetBSD release with a journaling filesystem. This lecture introduces the structure of the Fast File System, the modifications for WAPBL and specific constraints of the implementation.
The Fast File System (FFS) has been used in the BSD land for more than two decades. The original implementation offered two operational modes:
Based on a donation from Wasabi Systems, Write Ahead Physical Block Logging (WAPBL) provides journaling for FFS with similar or better performance than soft dependencies during normal operation. Recovery time after crashes depends on the amount of outstanding IO operations and normally takes a few seconds.
This lecture gives a short overview of FFS and the consistency constraints for meta data updates. It introduces the WAPBL changes, both in terms of the on-disk format and the implementation in NetBSD. Finally the implementation is compared to the design of comparable file systems and specific issues of and plans for the current implementation are discussed.
Remote and mass management of systems with finstall
Automated management on a largish scale
An important part of the "finstall" project, created as a graphical installer for FreeBSD, is a configuration server that can be used to remotely administer and configure arbitrary systems. It allows for remote scripting of administration tasks and is flexible enough to support complete reconfiguration of running systems.
The finstall project has two major parts - the front-end and the back-end. The front-end is just a GUI allowing the users to install the system in a convenient way. The back-end is a network-enabled XML-RPC server that is used by the front-end to perform its tasks. It can be used as a stand-alone configuration daemon. This talk will describe a way to make use of this property of finstall to remotely manage large groups of systems.
Detecting TCP regressions with tcpdiff
Determining if a TCP stack is working correctly is hard. The tcpdiff project aims for a simpler goal: To automatically detect differences in TCP behavior between different versions of an operating system and display those differences in an easy to understand format. The value judgement of whether a certain change between version X and Y of a TCP stack is good or bad will be left to human eyes.
Determining if a TCP stack is working correctly is hard. The tcpdiff project aims for a simpler goal: To automatically detect differences in TCP behavior between different versions of an operating system and display those differences in an easy to understand format. The value judgement of whether a certain change between version X and Y of a TCP stack is good or bad will be left to human eyes.
The initial version of tcpdiff presented at NYCBSDCon 2008 demonstrated that it could be used to detect at least two major TCP bugs that were introduced into FreeBSD in the past few years. The work from that presentation can be viewed at http://www.silby.com/nycbsdcon08/.
For BSDCan 2009, I hope to fix a number of bugs in tcpdiff, make it easier to use, set up nightly tests of FreeBSD, and improve it so that additional known bugs can be detected. Additionally, I plan to run it on OSes other than FreeBSD.
Crypto Acceleration on FreeBSD
As more and more services on the internet become cryptographically secured, the load of cryptography on systems becomes heavier and heavier. Crypto acceleration hardware is available in different forms for different workloads. Embedded communications processors from VIA and AMD have limited acceleration facilities in silicon and various manufacturers build hardware for accelerating secure web traffic and IPSEC VPN tunnels.
This talk gives an overview of FreeBSD's crypto framework in the kernel and how it can be used together with OpenSSL to leverage acceleration hardware. Some numbers will be presented to demonstrate how acceleration can improve performance - and how it can curiously bring a system to a grinding halt.
Philip originally started playing with crypto acceleration when he saw the "crypto block" in one of his Soekris boards. As usual, addiction was instant and by the grace of the "you touch it, you own it" principle, he has been fiddling the crypto framework more than is good for him.
Firewire BoF Plugfest
Debugging and testing of Firewire products with FreeBSD
Come one come all to a Firewire plugfest. Let's debug and test together and see if we can't knock out some features and bugs.
A hands-on testing and debugging session of the Firewire stack in FreeBSD.
Everyone who wishes to attend should bring their Firewire devices, ext Drives and Cameras, and their Laptops. I will be debugging and capturing data points to enhance and improve features in the Firewire stack.
We should be able to knock out quite a bunch of bugs if folks can bring their various Firewire devices along with their various PCs.
Even if your Firewire device works perfectly, bring it by so it can be documented as supported by the Firewire team!
Building the Network You Need with PF, the OpenBSD packet filter.
Building the network you need is the central theme for any network admin. This tutorial is for aspiring or seasoned network professionals with at least a basic knowledge of networking in general and TCP/IP particular. The session aims at teaching tools and techniques to make sure you build your network to work the way it's supposed to, keeping you in charge. Central to the toolbox is the OpenBSD PF packet filter, supplemented with tools that interact with it. Whether you are a greybeard looking for ways to optimize your setups or a greenhorn just starting out, this session will give you valuable insight into the inner life of your network and provide pointers to how to use that knowledge to build the network you need. The session will also offer some fresh information on changes introduced in OpenBSD 4.5, the most recent version of PF and OpenBSD. The tutorial is loosely based on Hansteen's recent book, /The Book of PF/ (No Starch Press), with updates and adaptations based on developments since the book's publication date.
Networking from the Bottom Up: Device Drivers.
In this tutorial I will describe how to write and maintain network drivers in FreeBSD and use the example of the Intel Gigabit Ethernet driver (igb) throughout the course.
Students will learn the basic data structures and APIs necessary to implement a network driver in FreeBSD. The tutorial is general enough that it can be applied to other BSDs, and likely to other embedded and UNIX like systems while being specific enough that given a device and a manual the student should be able to develop a working driver on their own. This is the first of a series of lectures on network that I am developing over the next year or so.
iSCSI is not an Apple appliance.
The i in iSCSI stands for internet, some say for insecure, personally I like to think interesting. I'll try to share the road followed from RFC-3720 to the actual working driver, the challenges, the frustrations.
pfSense is a free, open source customized distribution of FreeBSD tailored for use as a firewall and router. In addition to being a powerful, flexible firewalling and routing platform, it includes a long list of related features and a package system allowing further expandability without adding bloat and potential security vulnerabilities to the base distribution. pfSense is a popular project with more than 1 million downloads since its inception, and proven in countless installations ranging from small home networks protecting a PC and an Xbox to large corporations, universities and other organizations protecting thousands of network devices.
This tutorial is being presented by the founders of the pfSense project, Chris Buechler and Scott Ullrich.
The session will start with an introduction to the project, hardware sizing and selection, installation, firewalling concepts and basic configuration, and continue to cover all the most popular features of the system. Common usage scenarios, deployment considerations, step by step configuration guidance, and best practices will be covered for each feature. Most configurations will be demonstrated in a live lab environment.
Attendees are assumed to have basic knowledge of TCP/IP and firewalling concepts, however no in-depth knowledge in these areas or prior knowledge of pfSense or FreeBSD is necessary.
The Bacula project started in January 2000 with several goals, one of which was the ability to backup any client from a Palm to a mainframe computer. Bacula is available under a GPL license.
Bacula uses several distinct components, each communicating via TCP/IP, to achieve a very scalable and robust solution to backups.
Kern is one of the original project founders and still one of the most productive Bacula developers.
FreeBSD now runs on the MIPS platform. FreeBSD/mips supports MIPS-32 and MIPS-64 targets, including SMP for multicore support.
FreeBSD/mips is targeted at the embedded MIPS marketplace. FreeBSD has run on the MIPS platform for many years. Juniper ported FreeBSD to the Mips platform in the late 1990's. However, concern about intellectual property issues kept Juniper from contributing the port back to FreeBSD until recently. The contributed port was a 64-bit mips port.
In the mean time, many efforts were made to bring FreeBSD to the mips platform. The first substantial effort to bring FreeBSD to the Mips platform was done by Juli Mallet. This effort made it to single user, but never further than that. This effort was abandoned due to a change in Juli's life. The port languished.
Two years ago at BSDcan, as my involvement with FreeBSD/arm was growing, I tried to rally the troops into doing a FreeBSD/mips port. My efforts resulted in what has been commonly called the "mips2" effort. The name comes from the choice of //depot/projects/mips2 to host the work in perforce. A number of people worked on the earliest versions of the port, but it too languished and seemed destined to suffer the same fate as earlier efforts. Then, two individuals stood up and started working on the port. Wojciech A. Koszek and Oleksandr Tymoshenko pulled in code from the prior efforts. Through their efforts of stabilizing this code, the port to the single user stage and ported it to three different platforms. Others ported it to a few more. Snapshots of this work were released from time to time.
Cavium Networks picked up one of these snapshots and ported it to their multicore mips64 network processor. Cavium has kindly donated much of their work to the comminuty.
In December, I started at Cisco systems. My first job was to merge all the divergent variants of FreeBSD/mips and get it into shape to push into the tree. With luck, this should be in the tree before I give my talk.
In parallel to this, other advances in the embedded support for FreeBSD have been happening as well. I'll talk about new device drivers, new subsystems, and new build tools that help to support the embedded developer.
PC-BSD provides a user-friendly desktop experience, for experts and casual users alike. PC-BSD is 100% FreeBSD under the hood, while providing desktop essentials, such as a graphical installation system, point-n-click package-management using the PBI system, and easy to use system management tools; All integrated into an easy to use K Desktop Environment (KDE).
The PBI (Push Button Installer) format is the cornerstone of the PC-BSD desktop, which allows users to install applications in a self-contained format, free from dependency problems, and compile issues that stop most casual users from desktop adoption. The PBI format also provides power and flexibility in user interaction, and scripting support, which allows applications to be fine-tuned to the best possible user experience.
This talk would go over in some detail our new PBI building system, which converts a FreeBSD port, such as FireFox, into a standalone self-contained PBI installer for PC-BSD desktops.
The presentation will be divided into two main sections:
The Push Button Installer (PBI) Format
Building PBIs from Ports "Auto-magically"
At Appalachian State University, we utilize an open source VPN to allow faculty, staff and vendors secure access to Appalachian State University's internal network from any location that has an Internet connection. To implement our virtual private network project, we needed a secure VPN that is flexible enough to work with our existing network registration and LDAP authentication systems, has simple client installation, is redundant, allows multiple VPN server instances for special site-to-site tunnels and unique configurations, and can run on multiple platforms. Using OpenVPN running on OpenBSD, we met those requirements and added a distributed administration system that allows select users to allow VPN access to specific computers for external users and vendors without requiring intervention from our network or security personnel. Our presentation will start with a quick overview of OpenVPN and OpenBSD and then detail the specifics of our VPN implementation.
Dissatisfied with IPSec for road warrior VPN usage we went looking for a better solution. We had hopped that we could find a solution that would run on multiple platforms, was flexible and worked well. We found OpenVPN and have been pleased. Initially we ran it on RHEL. We migrated to OpenBSD for pf functionality and general security concerns. ...and because we like OpenBSD.
Our presentation will focus on the specifics of our VPN implementation. We will quickly cover the basics of OpenVPN and the most used features of OpenBSD. Moving along we will cover multiple authentication methods, redundancy, running multiple instances, integration with our netreg system, how pf has extended functionality, embedding in appliances, and client configuration. The system has proven helpful with providing vendor access where needed and we'll cover this aspect as well. Time permitting we will cover current enhancement efforts and future plans.
OpenVPN has been called the "Swiss army knife" of VPN solutions. We hope our presentation leaves participants with that feeling.
The "finstall" project, sponsored by Google as a Summer of Code 2007 project, is an attempt to create a user-friendly graphical installer for FreeBSD, with enough strong technical features to appeal to the more professional users. A long term goal for it is to be a replacement for sysinstall, and as such should support almost all of the features present in sysinstall, as well as add support for new FreeBSD features such as GEOM, ZFS, etc. This talk will describe the architecture of "finstall" and focus on its lesser known features such as remote installation.
"finstall" is funded by Google SoC as a possible long-term replacement for sysinstall, as a "LiveCD" with the whole FreeBSD base system on the CD, with X11 and XFCE4 GUI. In the talk I intend to describe what I did so far, and what are the future plans for it. This includes the installer GUI, the backend (which has the potential to become a generic FreeBSD configuration backend) and the assorted tools developed for finstall ("LiveCD" creation scripts). More information on finstall can be found here: https://wiki.freebsd.org/finstall.
The new Danish Air Traffic Control system, CASIMO, prompted the development on a modular and general software platform for data collection, control and monitoring of "weird hardware" of all sorts.
The talk will present the "measured" daemon, and detail some of the uses it has been put to, as an, admittedly peripheral, component of the ATC system.
Many "SCADA" systems suffer from lack of usable interfaces for external access to the data. Measured takes the opposite point of view and makes real-time situation available, and accepts control instructions as ASCII text stream over TCP connections. Several examples of how this can be used will be demonstrated.
Measured will run on any FreeBSD system, but has not been ported to other UNIX variants yet, and it is perfect for that "intelligent house" project of yours.
I believe I gave a WIP presentation of this about two years ago.
LLVM is a suite of carefully designed open source libraries that implement compiler components (like language front-ends, code generators, aggressive optimizers, Just-In-Time compiler support, debug support, link-time optimization, etc.). The goal of the LLVM project is to build these components in a way that allows them to be combined together to create familiar tools (like a C compiler), interesting new tools (like an OpenGL JIT compiler), and many other things we haven't thought of yet. Because LLVM is under continuous development, clients of these components naturally benefit from improvements in the libraries.
This talk gives an overview of LLVM's design and approach to compiler construction, and gives several example applications. It describes applications of LLVM technology to llvm-gcc (a C/C++/Objective C compiler based on the GNU GCC front-end), the OpenGL stack in Mac OS/X Leopard, and Clang. Among other things, the Clang+LLVM Compiler provides a fully BSD-Licensed C and Objective-C compiler (with C++ in development) which compiles code several times faster than GCC, produces code that is faster than GCC in many cases, produces better warnings and error messages, and supports many other applications (e.g. static analysis and refactoring).
The traditional closing...
with some new and interesting twists. Sleep in if
you must, but don't miss this session.
In this talk, I will briefly discuss some general ways Google's Open Source Team contributes to the wider community. The rest of the talk will explore some highlights of the Google Summer of Code program, our initiative to get university students involved in Open Source development.
I will cover the program's inception, lessons learned over time and tips for success in the program for both mentors and students. In particular, the talk will detail some experiences of the *BSD mentoring organizations involved in the program as a case study in successfully managing the program from the Open Source project's perspective. Any Google Summer of Code participants in the audience are welcome and encouraged to chime in with their own insights.
SUN's ZFS file system became part of FreeBSD on 6th April 2007. ZFS is a new kind of file system that provides simple administration, transactional semantics, end-to-end data integrity, and immense scalability. ZFS is not an incremental improvement to existing technology; it is a fundamentally new approach to data management. We've blown away 20 years of obsolete assumptions, eliminated complexity at the source, and created a storage system that's actually a pleasure to use.
ZFS presents a pooled storage model that completely eliminates the concept of volumes and the associated problems of partitions, provisioning, wasted bandwidth and stranded storage. Thousands of file systems can draw from a common storage pool, each one consuming only as much space as it actually needs. The combined I/O bandwidth of all devices in the pool is available to all filesystems at all times.
All operations are copy-on-write transactions, so the on-disk state is always valid. There is no need to fsck(1M) a ZFS file system, ever. Every block is checksummed to prevent silent data corruption, and the data is self-healing in replicated (mirrored or RAID) configurations. If one copy is damaged, ZFS detects it and uses another copy to repair it.
In the embedded world U-Boot is a de facto standard for an initial level boot loader (firmware). It runs on a great number of platforms and architectures, and is open source.
This talk covers the development work on integrating FreeBSD with U-Boot-based systems. Starting with an overview of differences between booting an all-purpose desktop computer vs. embedded system, FreeBSD booting concepts are explained along with requirements for the underlying firmware.
Historical attempts to interface FreeBSD with this firmware are mentioned and explanation given on why they failed or proved incomplete. Finally, the recently developed approach to integrate FreeBSD and U-Boot is presented, with implementation details and particular attention on how it's been made architecture and platform independent, and how loader(8) has been bound to it.
Just like every other piece of software, the FreeBSD kernel has bugs. Debugging a kernel is a bit different from debugging a userland program as there is nothing underneath the kernel to provide debugging facilities such as ptrace() or procfs. This paper will give a brief overview of some of the tools available for investigating bugs in the FreeBSD kernel. It will cover the in-kernel debugger DDB and the external debugger kgdb which is used to perform post-mortem analysis on kernel crash dumps.
DTrace is a comprehensive dynamic tracing facility originally developed for Solaris that can be used by administrators and developers on live production systems to examine the behavior of both user programs and of the operating system itself. DTrace enables users to explore their system to understand how it works, track down performance problems across many layers of software, or locate the cause of aberrant behavior. DTrace lets users create their own custom programs to dynamically instrument the system and provide immediate, concise answers to arbitrary questions you can formulate using the DTrace D programming language.
This talk discusses the port of the DTrace facility to FreeBSD and demonstrates examples on a live FreeBSD system.
The X.Org project provides an open source implementation of the X Window System. The development work is being done in conjunction with the freedesktop.org community. The X.Org Foundation is the educational non-profit corporation whose Board serves this effort, and whose Members lead this work.
The X window system has been changing a lot in the recent years, and still changing. This talk will present this evolution, summarizing what has already been done and showing the current roadmap for future evolutions, with some focus on how *BSD kernels can be affected by the developments done with Linux as the primary target.
This talk will look at issues which face the modern network application developer, from the point of view of poorly-designed examples. This will cover internal code structure and dataflow, interaction with the TCP stack, IO scheduling in high and low latency environments and high-availability considerations. In essence, this presentation should be seen as a checklist of what not to do when writing network applications.
Plenty of examples of well designed network applications exist in the open and closed source world today. Unfortunately there are just as many examples of fast network applications as there are "fast but workload specific"; sometimes failing miserably in handling the general case. This may be due to explicit design (eg Varnish) but many are simply due to the designer not fully appreciating the wide variance in "networks" - and their network application degrades ungracefully when under duress. My aim in this presentation is to touch on a wide number of issues which face network application programmers - most of which seem not "application related" to the newcomer - such as including pipelining into network communication, managing a balance between accepting new requests and servicing existing requests, or providing back-pressure to a L4 loadbalancer in case of traffic bursts. Various schemes for working with these issues will be presented, and hopefully participants will walk away with more of an understanding about how the network, application and operating systems interact.
In this talk we present Aerosource, an initiative to bring Open Source Software development methods to internal software developers at The Aerospace Corporation.
Within Aerosource, FreeBSD is used in several key roles. First, we run most of our tools on top of FreeBSD. Second, the ports collection (both official ports and custom internal ones) eases our administrative burden. Third, the FreeBSD project serves as an example and role model for the results that can be achieved by an Open Source Software projects. We discuss the development infrastructure we have built for Aerosource based largely on BSD licensed software including FreeBSD, PostgreSQL, Apache, and Trac. We will also discuss our custom management tools including our system for managing our custom internal ports. Finally, we will cover our development successes and how we use projects like FreeBSD as exemplars of OSS development.
This talk will introduce the attendee into the interesting world of SCTP.
We will first discuss the new and different features that SCTP (a new transport in FreeBSD 7.0) provide to the user. Then we will shift gears and discuss the extended socket API that is available to SCTP users and will cover such items as:
This talk covers the development work on porting the FreeBSD/ARM to Marvell Orion family of highly integrated chips.
ARM architecture is widely adopted in the embedded devices, and since the architecture can be licensed, many implementation variations exist: Orion is a derivative compliant with the ARMv5TE definition, it provides a rich set of on-chip peripherals.
Present state of the FreeBSD support for ARM is explained, areas for improvement highlighted and its overall shape and condition presented.
The main discussion covers scope of the Orion port (what integrated peripherals required new development, what was adapted from existing code base); design decisions are explained for the most critical items, and implementation details revealed.
Summary notes are given on general porting methodology, debugging techniques and difficulties encountered during such undertaking.
Recently I have been attending Hostobzor 12th, the Russian conference of hosting providers, beeing held at Raivola hotel near St. Petersburg. The event was great as always thanks to organizers. There was a number of intersting talks given, a lot of interesting discussions held, and, what I appreciate better, a lot of new people with great ideas met.
I gave a talk on using the FreeBSD Ports system to mange a large-scale virtual hosting installations based on Hosting Telesystems experience. I tried to describe in detail how we use the ports collection to deploy a large number of servers diverced by architecture and OS versions, how we build packages and distribute them among servers, talked about how we use Mercurial VCS to incrementally merge upstream changes into our modified ports collection and FreeBSD src trees. Hopefully, I've not screwed it much... At least, some people was interested a lot and asked interesting questions.
In 2004 ben ik begonnen met het FreeBSD Dutch Documentation Project, een project dat inmiddels bijna het complete handboek vertaald heeft. Sinds die tijd zijn er vele wegen geweest die ik behandeld heb, van documentatie projectleider naar Security Team-lid tot aan FreeBSD Developer.
Remko Lodder is momenteel 25 jaar en werkt als Unix Engineer voor het bedrijf Snow B.V. waar hij zich momenteel met name bezig houd met security (firewalls etc). Hij is sinds 2004 lid van het FreeBSD Development team en is momenteel 1 van de meest actieve developers binnen het team.
Hans zal een historisch overzicht geven van het ontstaan van *BSD vanaf de oorsprong van UNIX tot aan de nu bekende *BSD varianten. Hij zal daarbij met name ingaan wat de oorsprong en het ontstaan van een aantal *BSD-projecten zijn. Hierbij zal hij zeer kort ingaan op de verschillende licentieproblemen die we in het verleden gezien hebben en worden een aantal bekende personen en data weer eens even op de kaart geplaatst.
Hans van de Looy is oprichter van Madison Gurkha. Een bedrijf dat gespecialiseerd is op het gebied van het uitvoeren van technische ICT-beveiligingsonderzoeken, in de media ook wel aangeduid met Etisch Hacken. Tijdens dergelijke onderzoeken maakt hij ook regelmatig gebruik van op BSD* gebaseerde systemen.
In April 2009 I was invited to speak on FreeBSD/PmcTools by the Bangalore chapter of the ACM.
This was an overview talk. The talk briefly touched upon: the motivations and goals of the project, the programming APIs, some aspects of the implementation and on possible future work.