path: root/sys/contrib/cloudabi
Commit message (Collapse)AuthorAgeFilesLines
* Drop cloudabiKonstantin Belousov2021-09-2111-4839/+0
| | | | | | | | | | | | | According to https://github.com/NuxiNL/cloudlibc: CloudABI is no longer being maintained. It was an awesome experiment, but it never got enough traction to be sustainable. There is no reason to keep it in FreeBSD. Approved by: ed (private mail) Reviewed by: emaste Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D31923
* Import the latest CloudABI definitions, v0.18.Ed Schouten2017-11-212-23/+755
| | | | | | | | | | | | | | | In addition to some small style fixes to the ARMv6 vDSO, this release includes a new vDSO that can be used for the execution of ARMv6/ARMv7 code on 64-bit platforms. Just like for i686 on x86-64, this new vDSO is responsible for padding arguments and return values to 64-bit values, so that the kernel can easily forward system calls to the native system calls. Obtained from: https://github.com/NuxiNL/cloudabi Notes: svn path=/head/; revision=326075
* Upgrade to CloudABI v0.17.Ed Schouten2017-11-085-84/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Compared to the previous version, v0.16, there are a couple of minor changes: - CLOUDABI_AT_PID: Process identifiers for CloudABI processes. Initially, BSD process identifiers weren't exposed inside the runtime, due to them being pretty much useless inside of a cluster computing environment. When jobs are scheduled across systems, the BSD process number doesn't act as an identifier. Even on individual systems they may recycle relatively quickly. With this change, the kernel will now generate a UUIDv4 when executing a process. These UUIDs can be obtained within the process using program_getpid(). Right now, FreeBSD will not attempt to store this value. This should of course happen at some point in time, so that it may be printed by administration tools. - Removal of some unused structure members for polling. With the polling framework being simplified/redesigned, it turns out some of the structure fields were not used by the C library. We can remove these to keep things nice and tidy. Obtained from: https://github.com/NuxiNL/cloudabi Notes: svn path=/head/; revision=325555
* Import the latest CloudABI definitions, version 0.16.Ed Schouten2017-10-1810-188/+81
| | | | | | | | | | | | | | | | | | | | | | | | | The most important change in this release is the removal of the poll_fd() system call; CloudABI's equivalent of kevent(). Though I think that kqueue is a lot saner than many of its alternatives, our experience is that emulating this system call on other systems accurately isn't easy. It has become a complex API, even though I'm not convinced this complexity is needed. This is why we've decided to take a different approach, by looking one layer up. We're currently adding an event loop to CloudABI's C library that is API compatible with libuv (except when incompatible with Capsicum). Initially, this event loop will be built on top of plain inefficient poll() calls. Only after this is finished, we'll work our way backwards and design a new set of system calls to optimize it. Interesting challenges will include integrating asynchronous I/O into such a system call API. libuv currently doesn't aio(4) on Linux/BSD, due to it being unreliable and having undesired semantics. Obtained from: https://github.com/NuxiNL/cloudabi Notes: svn path=/head/; revision=324727
* Merge pipes and socket pairs.Ed Schouten2017-09-051-2/+0
| | | | | | | | | | | | | | | Now that CloudABI's sockets API has been changed to be addressless and only connected socket instances are used (e.g., socket pairs), they have become fairly similar to pipes. The only differences on CloudABI is that socket pairs additionally support shutdown(), send() and recv(). To simplify the ABI, we've therefore decided to remove pipes as a separate file descriptor type and just let pipe() return a socket pair of type SOCK_STREAM. S_ISFIFO() and S_ISSOCK() are now defined identically. Notes: svn path=/head/; revision=323177
* Complete the CloudABI networking refactoring.Ed Schouten2017-08-308-188/+42
| | | | | | | | | | | | | | Now that all of the packaged software has been adjusted to either use Flower (https://github.com/NuxiNL/flower) for making incoming/outgoing network connections or can have connections injected, there is no longer need to keep accept() around. It is now a lot easier to write networked services that are address family independent, dual-stack, testable, etc. Remove all of the bits related to accept(), but also to getsockopt(SO_ACCEPTCONN). Notes: svn path=/head/; revision=323015
* Sync CloudABI compatibility against the latest upstream version (v0.13).Ed Schouten2017-08-258-253/+85
| | | | | | | | | | | | | | | | | | | With Flower (CloudABI's network connection daemon) becoming more complete, there is no longer any need for creating any unconnected sockets. Socket pairs in combination with file descriptor passing is all that is necessary, as that is what is used by Flower to pass network connections from the public internet to listening processes. Remove all of the kernel bits that were used to implement socket(), listen(), bindat() and connectat(). In principle, accept() and SO_ACCEPTCONN may also be removed, but there are still some consumers left. Obtained from: https://github.com/NuxiNL/cloudabi MFC after: 1 month Notes: svn path=/head/; revision=322885
* Upgrade to the latest sources generated from the CloudABI specification.Ed Schouten2017-07-2610-326/+197
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The CloudABI specification has had some minor changes over the last half year. No substantial features have been added, but some features that are deemed unnecessary in retrospect have been removed: - mlock()/munlock(): These calls tend to be used for two different purposes: real-time support and handling of sensitive (cryptographic) material that shouldn't end up in swap. The former use case is out of scope for CloudABI. The latter may also be handled by encrypting swap. Removing this has the advantage that we no longer need to worry about having resource limits put in place. - SOCK_SEQPACKET: Support for SOCK_SEQPACKET is rather inconsistent across various operating systems. Some operating systems supported by CloudABI (e.g., macOS) don't support it at all. Considering that they are rarely used, remove support for the time being. - getsockname(), getpeername(), etc.: A shortcoming of the sockets API is that it doesn't allow you to create socket(pair)s, having fake socket addresses associated with them. This makes it harder to test applications or transparently forward (proxy) connections to them. With CloudABI, we're slowly moving networking connectivity into a separate daemon called Flower. In addition to passing around socket file descriptors, this daemon provides address information in the form of arbitrary string labels. There is thus no longer any need for requesting socket address information from the kernel itself. This change also updates consumers of the generated code accordingly. Even though system calls end up getting renumbered, this won't cause any problems in practice. CloudABI programs always call into the kernel through a kernel-supplied vDSO that has the numbers updated as well. Obtained from: https://github.com/NuxiNL/cloudabi Notes: svn path=/head/; revision=321514
* Sync in the latest CloudABI generated source files.Ed Schouten2017-01-175-462/+401
| | | | | | | | | | | | | Languages like C++17 and Go provide direct support for slice types: pointer/length pairs. The CloudABI generator now has more complete for this, meaning that for the C binding, pointer/length pairs now use an automatic naming scheme of ${name} and ${name}_len. Apart from this change and some reformatting, the ABI definitions are identical. Binary compatibility is preserved entirely. Notes: svn path=/head/; revision=312353
* Sync in latest vDSOs from upstream.Ed Schouten2016-09-192-124/+86
| | | | | | | | | | | | | | | | | | - Use conditional instruction to simplify the ARMv6 vDSO. This means that we no longer perform any branching. In the failure case, we simply slide over the assignments of the return values. The vDSO could be improved even further by using stmia to do assignments in parallel. Unfortunately, the script used to generate these is not smart enough for that (yet). Spotted by: andrew@. - Make the style of the i686 vDSO more similar to the others by using decimal literals. Notes: svn path=/head/; revision=305987
* Fix badly computed register/stack offset of system call output arguments.Ed Schouten2016-09-181-5/+4
| | | | | | | | | | | | | | Bugs in the Python code used to generate this vDSO caused us to miscompute the register numbers/stack offsets at which addresses of the system call output arguments were stored. Together with some other patches, this vDSO allows us to make all of the cloudlibc unit tests pass. Obtained from: https://github.com/NuxiNL/cloudabi Notes: svn path=/head/; revision=305938
* Add very preliminary support for CloudABI for ARMv6.Ed Schouten2016-09-181-0/+490
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to make CloudABI work on ARMv6, start off by copying over the sysvec for ARM64 and adjust it to use 32-bit registers. Also add code for fetching arguments from the stack if needed, as there are fewer register than on ARM64. Also import the vDSO that is needed to invoke system calls. This vDSO uses the intra procedure call register (ip) to store the system call number. This is a bit simpler than what native FreeBSD does, as FreeBSD uses r7, while preserving the original r7 into ip. This sysvec seems to be complete enough to start CloudABI processes. These processes are capable of linking in the vDSO and are therefore capable of executing (most?) system calls successfully. Unfortunately, the biggest show stopper is still that TLS is completely broken: - The linker used by CloudABI, LLD, still has troubles with some of the relocations needed for TLS. See LLVM bug 30218 for more details. - Whereas FreeBSD uses the tpidruro register for TLS, for CloudABI I want to make use of tpidrurw, so that userspace can modify the base address directly. This is needed for efficient emulation. Unfortunately, this register doesn't seem to be preserved across context switches yet. Obtained from: https://github.com/NuxiNL/cloudabi (the vDSO) Notes: svn path=/head/; revision=305928
* Make execution of 32-bit CloudABI executables work on amd64.Ed Schouten2016-08-241-0/+1223
| | | | | | | | | | | | | | | | | | | | | | A nice thing about requiring a vDSO is that it makes it incredibly easy to provide full support for running 32-bit processes on 64-bit systems. Instead of letting the kernel be responsible for composing/decomposing 64-bit arguments across multiple registers/stack slots, all of this can now be done in the vDSO. This means that there is no need to provide duplicate copies of certain system calls, like the sys_lseek() and freebsd32_lseek() we have for COMPAT_FREEBSD32. This change imports a new vDSO from the CloudABI repository that has automatically generated code in it that copies system call arguments into a buffer, padding them to eight bytes and zero-extending any pointers/size_t arguments. After returning from the kernel, it does the inverse: extracting return values, in the process truncating pointers/size_t values to 32 bits. Obtained from: https://github.com/NuxiNL/cloudabi Notes: svn path=/head/; revision=304744
* Remove an unused header file.Ed Schouten2016-08-241-413/+0
| | | | | | | | | The native CloudABI data types header file used to be pulled in by the vDSOs when they were still written in C. Since they are now all rewritten in assembly, this can go away. Notes: svn path=/head/; revision=304743
* Make CloudABI work on i386.Ed Schouten2016-08-221-0/+477
| | | | | | | | | | | | | | | | | | Copy over amd64's cloudabi64_sysvec.c into i386 and tailor it to work. Again, we use a system call convention similar to FreeBSD, except that there is no support for indirect system calls (%eax == 0). Where i386 differs from amd64 is that we have to store thread/process entry arguments on the stack instead of using registers. We also have to put an extra pointer on the stack for TLS (for GSBASE). Place that pointer in the empty slot that is normally used to hold return addresses. That seems to keep the code simple. Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D7590 Notes: svn path=/head/; revision=304615
* Fix s/64/32/ conversion errors in the system call table.Ed Schouten2016-08-211-2/+2
| | | | | | | We should pull in the 32 bit headers when using this system call table. Notes: svn path=/head/; revision=304563
* Import the 32-bit system call table and data types into the tree.Ed Schouten2016-08-212-0/+547
| | | | | | | Obtained from: https://github.com/NuxiNL/cloudabi Notes: svn path=/head/; revision=304561
* Rewrite the vDSOs for CloudABI in assembly.Ed Schouten2016-08-214-2008/+1002
| | | | | | | | | | | | | | | | | | | | The reason why the old vDSOs were written in C using inline assembly was purely because they were embedded in the C library directly as static inline functions. This was practical during development, because it meant you could invoke system calls without any library dependencies. The vDSO was simply a copy of these functions. Now that we require the use of the vDSO, there is no longer any need for embedding them in C code directly. Rewriting them in assembly has the advantage that they are closer to ideal (less useless branching, less assumptions about registers remaining unclobbered by the kernel, etc). They are also easier to build, as they no longer depend on the C type information for CloudABI. Obtained from: https://github.com/NuxiNL/cloudabi Notes: svn path=/head/; revision=304554
* Import the new automatically generated system call table for CloudABI.Ed Schouten2016-08-196-3428/+2648
| | | | | | | | | | | | | | | | | | Now that we've switched over to using the vDSO on CloudABI, it becomes a lot easier for us to phase out old features. System call numbering is no longer something that's part of the ABI. It's fully based on names. As long as the numbering used by the kernel and the vDSO is consistent (which it always is), it's all right. Let's put this to the test by removing a system call (thread_tcb_set()) that's already unused for quite some time now, but was only left intact to serve as a placeholder. Sync in the new system call table that uses alphabetic sorting of system calls. Obtained from: https://github.com/NuxiNL/cloudabi Notes: svn path=/head/; revision=304478
* Import vDSO-related source files from the CloudABI repository.Ed Schouten2016-08-083-0/+3259
| | | | | | | | | | | | | | | | | | | | | CloudABI executables that are emulated on Mac OS X do not invoke system calls through "syscall". Instead, they make use of a vDSO that is provided by the emulator that provides symbols for all of the system call routines. The emulator can implement these any way it likes. At some point in time we want to do this for native execution as well, so that CloudABI executables are entirely oblivious of how system calls need to be performed. They will simply call into functions and let that deal with all of the details. These source files can be used to generate a simple vDSO that does nothing more than invoke "syscall". All we need to do now is map it into the processes. Obtained from: https://github.com/NuxiNL/cloudabi Notes: svn path=/head/; revision=303833
* Sync in the latest CloudABI constants and data types.Ed Schouten2016-08-071-12/+13
| | | | | | | | | | The only change is the addition of AT_SYSINFO_EHDR, which can be used for providing a vDSO. Obtained from: https://github.com/NuxiNL/cloudabi Notes: svn path=/head/; revision=303818
* Make CloudABI's way of doing TLS more friendly to userspace emulators.Ed Schouten2016-04-061-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We're currently seeing how hard it would be to run CloudABI binaries on operating systems cannot be modified easily (Windows, Mac OS X). The idea is that we want to just run them without any sandboxing. Now that CloudABI executables are PIE, this is already a bit easier, but TLS is still problematic: - CloudABI executables want to write to the %fs, which typically requires extra system calls by the emulator every time it needs to switch between CloudABI's and its own TLS. - If CloudABI executables overwrite the %fs base unconditionally, it also becomes harder for the emulator to store a backup of the old value of %fs. To solve this, let's no longer overwrite %fs, but just %fs:0. As CloudABI's C library does not use a TCB, this space can now be used by an emulator to keep track of its internal state. The executable can now safely overwrite %fs:0, as long as it makes sure that the TCB is copied over to the new TLS area. Ensure that there is an initial TLS area set up when the process starts, only containing a bogus TCB. We don't really care about its contents on FreeBSD. Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D5836 Notes: svn path=/head/; revision=297613
* Sync in the latest CloudABI system call definitions.Ed Schouten2016-03-313-12/+22
| | | | | | | | | | | | | | | | | Some time ago I made a change to merge together the memory scope definitions used by mmap (MAP_{PRIVATE,SHARED}) and lock objects (PTHREAD_PROCESS_{PRIVATE,SHARED}). Though that sounded pretty smart back then, it's backfiring. In the case of mmap it's used with other flags in a bitmask, but for locking it's an enumeration. As our plan is to automatically generate bindings for other languages, that looks a bit sloppy. Change all of the locking functions to use separate flags instead. Obtained from: https://github.com/NuxiNL/cloudabi Notes: svn path=/head/; revision=297468
* Replace the CloudABI system call table by a machine generated version.Ed Schouten2016-03-245-731/+996
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The type definitions and constants that were used by COMPAT_CLOUDABI64 are a literal copy of some headers stored inside of CloudABI's C library, cloudlibc. What is annoying is that we can't make use of cloudlibc's system call list, as the format is completely different and doesn't provide enough information. It had to be synced in manually. We recently decided to solve this (and some other problems) by moving the ABI definitions into a separate file: https://github.com/NuxiNL/cloudabi/blob/master/cloudabi.txt This file is processed by a pile of Python scripts to generate the header files like before, documentation (markdown), but in our case more importantly: a FreeBSD system call table. This change discards the old files in sys/contrib/cloudabi and replaces them by the latest copies, which requires some minor changes here and there. Because cloudabi.txt also enforces consistent names of the system call arguments, we have to patch up a small number of system call implementations to use the new argument names. The new header files can also be included directly in FreeBSD kernel space without needing any includes/defines, so we can now remove cloudabi_syscalldefs.h and cloudabi64_syscalldefs.h. Patch up the sources to include the definitions directly from sys/contrib/cloudabi instead. Notes: svn path=/head/; revision=297247
* Import the latest CloudABI system call definitions and table.Ed Schouten2015-08-052-4/+11
| | | | | | | | We're going to need these for next code I'm going to send out for review: support for poll() and kqueue() on CloudABI. Notes: svn path=/head/; revision=286318
* Sync in latest upstream system call definitions.Ed Schouten2015-07-272-8/+3
| | | | | | | | | Futex object scopes have been renamed from using their own constants to simply reusing the existing CLOUDABI_MAP_{PRIVATE,SHARED} flags, as they are more accurate in this context. Notes: svn path=/head/; revision=285906
* Import upstream changes to the system call definitions.Ed Schouten2015-07-222-0/+11
| | | | | | | | Support has been added for providing the scope of a futex operation, whether the futex is local to the process or shared between processes. Notes: svn path=/head/; revision=285790
* Implement CloudABI memory management system calls.Ed Schouten2015-07-171-0/+3
| | | | | | | | | | | | | Add support for the <sys/mman.h> functions by wrapping around our own implementations. There are no kern_*() variants of these system calls, but we also don't need them in this case. It is sufficient to just call into the sys_*() functions. Differential Revision: https://reviews.freebsd.org/D3033 Reviewed by: brooks Notes: svn path=/head/; revision=285652
* Add a sysentvec for CloudABI on x86-64.Ed Schouten2015-07-162-23/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: For CloudABI we need to put two things on the stack of new processes: the argument data (a binary blob; not strings) and a startup data structure. The startup data structure contains interesting things such as a pointer to the ELF program header, the thread ID of the initial thread, a stack smashing protection canary, and a pointer to the argument data. Fetching system call arguments and setting the return value is similar to FreeBSD. The only differences are that system call 0 does not exist and that we call into cloudabi_convert_errno() to convert the error code. We also need this function in a couple of other places, so we'd better reuse it here. Reviewers: dchagin, kib Reviewed By: kib Subscribers: imp Differential Revision: https://reviews.freebsd.org/D3098 Notes: svn path=/head/; revision=285641
* Import the CloudABI datatypes and create a system call table.Ed Schouten2015-07-092-0/+712
CloudABI is a pure capability-based runtime environment for UNIX. It works similar to Capsicum, except that processes already run in capabilities mode on startup. All functionality that conflicts with this model has been omitted, making it a compact binary interface that can be supported by other operating systems without too much effort. CloudABI is 'secure by default'; the idea is that it should be safe to run arbitrary third-party binaries without requiring any explicit hardware virtualization (Bhyve) or namespace virtualization (Jails). The rights of an application are purely determined by the set of file descriptors that you grant it on startup. The datatypes and constants used by CloudABI's C library (cloudlibc) are defined in separate files called syscalldefs_mi.h (pointer size independent) and syscalldefs_md.h (pointer size dependent). We import these files in sys/contrib/cloudabi and wrap around them in cloudabi*_syscalldefs.h. We then add stubs for all of the system calls in sys/compat/cloudabi or sys/compat/cloudabi64, depending on whether the system call depends on the pointer size. We only have nine system calls that depend on the pointer size. If we ever want to support 32-bit binaries, we can simply add sys/compat/cloudabi32 and implement these nine system calls again. The next step is to send in code reviews for the individual system call implementations, but also add a sysentvec, to allow CloudABI executabled to be started through execve(). More information about CloudABI: - GitHub: https://github.com/NuxiNL/cloudlibc - Talk at BSDCan: https://www.youtube.com/watch?v=SVdF84x1EdA Differential Revision: https://reviews.freebsd.org/D2848 Reviewed by: emaste, brooks Obtained from: https://github.com/NuxiNL/freebsd Notes: svn path=/head/; revision=285307