aboutsummaryrefslogtreecommitdiff
path: root/sys/cddl/dev
Commit message (Collapse)AuthorAgeFilesLines
* sdt: Fix the probe ID type in struct sdt_probeMark Johnston2026-01-131-0/+3
| | | | | | | | | | | | This is supposed to be a dtrace_id_t, which is a uint32_t, while id_t is a uint64_t. sdt.h avoids depending on dtrace.h so we can't use dtrace_id_t directly. Bump __FreeBSD_version since the layout of structures in the SDT probe linker set has changed. Sponsored by: NetApp, Inc. Sponsored by: Klara, Inc.
* zfs: fix build after openzfs/zfs@e63d026b9Martin Matuska2025-11-171-0/+3
| | | | | Fix Makefiles Update zfs_config.h and zfs_gitrev.h
* dtrace/arm64: properly traverse the symbol tableKonrad Witaszczyk2025-10-291-1/+1
| | | | | | | | | | | | | | LINKER_EACH_FUNCTION_NAMEVAL() stops processing the symbol table if a callback function returns a non-zero value. The fbt_provide_module_function() callback should not return 1 when ignoring symbols. Instead, always return 0, as in dtrace/x86. Fixes: 30b68ecda84e ("Changes that improve DTrace FBT reliability on freebsd/arm64:") Reviewed by: markj, oshogbo Approved by: oshogbo (mentor) Obtained from: CheriBSD Differential Revision: https://reviews.freebsd.org/D53399
* dtrace: Deduplicate dtrace_sync() and dtrace_xcall() implementationsMark Johnston2025-09-186-154/+0
| | | | | | | | | dtrace_xcall() is just a thin wrapper around smp_rendezvous_cpus(). There's no need for six identical implementations to live in MD layers. No functional change intended. MFC after: 2 weeks
* sdt: Initialize probes in two passesMark Johnston2025-07-281-6/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | Suppose a kernel module A defines an SDT provider and probes, and kernel linker file B, dependant on A, contains tracepoints for those probes. When sdt.ko is loaded, it iterates over all loaded KLDs to initialize probe structures and register them with dtrace. In particular it uses linker_file_foreach(), which is not sorted; in the above scenario, B may be visited before A. Thus, it's possible for sdt_kld_load_probes() to try to add tracepoints to an uninitialized SDT probe. An example of the above arises when pfsync, pf, and sdt are loaded in that exact order after commit 4bb3b36577645. Fix this by initializing probe structures in the first pass over loaded KLDs. Then, the second pass can safely add tracepoints to any probe structure. Note that the scenario where B and A are loaded after sdt.ko is already handled properly, as there, the kld_load eventhandler is responsible for registering probes with dtrace, and that eventhandler fires for dependencies before it does for the dependent KLD. This presumes, however, that there are no cycles in the dependency graph. Reported by: jenkins MFC after: 2 weeks
* machine/stdarg.h -> sys/stdarg.hBrooks Davis2025-06-113-4/+3
| | | | | | | | | | | | | Switch to using sys/stdarg.h for va_list type and va_* builtins. Make an attempt to insert the include in a sensible place. Where style(9) was followed this is easy, where it was ignored, aim for the first block of sys/*.h headers and don't get too fussy or try to fix other style bugs. Reviewed by: imp Exp-run by: antoine (PR 286274) Pull Request: https://github.com/freebsd/freebsd-src/pull/1595
* dtrace/profile: Stop storing the probe name in struct profile_probeMark Johnston2025-06-041-2/+0
| | | | | | | | | It's unused, and the naked strcpy() was susceptible to buffer overflow if one creates, say, a probe called "profile-2000000000ns". Reported by: CHERI MFC after: 1 week Sponsored by: Innovate UK
* dtrace/arm64: Fix dtrace_gethrtime()Mark Johnston2025-03-101-2/+31
| | | | | | | | | | | | | | | | | | | | | This routine returns a monotonic count of the number of nanoseconds elapsed since the previous call. On arm64 it uses the generic system timer. The implementation multiplies the counter value by 10**9 then divides by the counter frequency, but this multiplication can overflow. This can result in trace records with non-monotonic timestamps, which breaks libdtrace's temporal ordering algorithm. An easy fix is to reverse the order of operations, since the counter frequency will in general be smaller than 10**9. (In fact, it's mandated to be 1Ghz in ARMv9, which makes life simple.) However, this can give a fair bit of error. Adopt the calculation used on amd64, with tweaks to handle frequencies as low as 1MHz: the ARM generic timer documentation suggests that ARMv8 timers are typically in the 1MHz-50MHz range, which is true on arm64 systems that I have access to. MFC after: 2 weeks Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D49244
* Revert "dtrace/arm64: Simplify dtrace_getarg() slightly"Mark Johnston2025-01-291-1/+3
| | | | | | | | | The change is clearly wrong as it removes a dereference of the pointer into the stack. Just revert for now. This reverts commit 70c067062217a5295ed321427b91cfd857c18b77. Reported by: jrtc27
* dtrace/arm64: Fix enumeration of FBT return probesMark Johnston2025-01-291-2/+1
| | | | | | | | | | | On arm64, the FBT provider treats tail calls as return probes. Ignoring the question of whether this is really correct, the implementation is wrong: instr is a pointer to uint32_t, so the removed multiplication by the instruction size is wrong. As a result, FBT would create return probes for intra-function branches. MFC after: 2 weeks Sponsored by: Innovate UK
* dtrace/arm64: Simplify dtrace_getarg() slightlyMark Johnston2025-01-291-3/+1
| | | | | | | | | The use of memcpy here is redundant, and also incorrect since memcpy() might be instrumented by fbt or kinst. dtrace_bcopy() exists, but we don't need it. MFC after: 2 weeks Sponsored by: Innovate UK
* dtrace/arm64: Simplify variable declarations in the invop handlerMark Johnston2025-01-221-6/+4
| | | | | | | | | Remove some unused variables and reduce the scope of some others. No functional change intended. MFC after: 1 week Sponsored by: Innovate UK
* dtrace: Add some more annotations for KMSANMark Johnston2024-11-233-10/+59
| | | | | | | | | | | | - Don't allow FBT and kinst to instrument the KMSAN runtime. - When fetching data from the traced thread's stack, mark it as initialized. It may well be uninitialized, but as dtrace permits arbitrary inspection of kernel memory, it isn't very useful to raise KMSAN reports. - Mark data copied in from userspace as initialized, as we do for copyin() etc. using interceptors. MFC after: 2 weeks
* dtrace: Avoid excessive pcpu allocationsMark Johnston2024-11-222-3/+3
| | | | | | | | | | | | | | | | We were previously allocating MAXCPU structures for several purposes, but this is generally unnecessary and is quite excessive, especially after MAXCPU was bumped to 1024 on amd64 and arm64. We already are careful to allocate only as many per-CPU tracing buffers as are needed; extend this to other allocations. For example, in a 2-vCPU VM, the size of a consumer state structure drops from 64KB to 128B. The size of the per-consumer `dts_buffer` and `dts_aggbuffer` arrays shrink similarly. Ditto for pre-allocations of local and global D variable storage space. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D47667
* dtrace/amd64: Make invop providers usable with KMSAN enabledMark Johnston2024-11-222-0/+11
| | | | | | | | - Use a fresh context when entering dtrace_invop() via a breakpoint exception. - Mark the #BP trapframe as initialized. MFC after: 2 weeks
* dtrace: Address KMSAN warnings in dtrace_disx86Mark Johnston2024-11-221-2/+4
| | | | | | | wbit was not being initialized in a couple of cases. Reported by: asomers MFC after: 2 weeks
* Update mentions of makesyscalls.luaBrooks Davis2024-10-301-2/+2
| | | | It is obsolete and will be removed in a followup commit.
* sdt: Tear down probes in kernel modules during kldunloadJohn Baldwin2024-10-161-9/+102
| | | | | | | | | | | | Previously only providers in kernel modules were removed leaving dangling pointers to tracepoints, etc. in unloaded kernel modules. PR: 281825 Reported by: Sony Arpita Das <sonyarpitad@chelsio.com> Reviewed by: markj Fixes: ddf0ed09bd8f sdt: Implement SDT probes using hot-patching Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D46890
* dtrace/amd64: Remove the dtrace_invop_callsite symbolMark Johnston2024-09-191-4/+0
| | | | | | | | | It is not needed after commit 7e80fd5ef397. No functional change intended. Reviewed by: avg MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D46675
* dtrace tests: Add a test case which validates FBT probe argumentsMark Johnston2024-09-191-3/+25
| | | | | | Reviewed by: avg MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D46674
* dtrace_test: Remove the dependency on dtraceallMark Johnston2024-09-191-1/+0
| | | | | | | | | | | | | | FBT refuses to create probes in modules which depend on dtrace(all), but dtrace_test is a convenient place to add functions specifically for testing dtrace. The dependency on dtraceall is not needed, so just remove it. In fact, it can be useful to test SDT probe creation by loading dtrace_test with and without dtraceall loaded. Reviewed by: avg MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D46673
* dtrace/amd64: Fix probe argument fetchingMark Johnston2024-09-191-46/+31
| | | | | | | | | | | | | | | | | | dtrace_getarg() previously walked the call stack looking for a frame matching the dtrace_invop_callsite symbol, in order to look for a trapframe corresponding to an invop (i.e., FBT or kinst) probe. Commit 3ba8e9dc4a0e broke this in some cases by breaking the expected alignment of the dtrace_invop_callsite symbol. Rather than groveling around the stack to find invop probe arguments, simply use the trapframe reference saved by dtrace_invop(). This is simpler and less fragile. Reported by: avg Reviewed by: avg MFC after: 2 weeks Fixes: 3ba8e9dc4a0e ("dtrace/amd64: Implement emulation of call instructions") Differential Revision: https://reviews.freebsd.org/D46672
* dtrace: Avoid including dtrace_isa.c directly into dtrace.cMark Johnston2024-07-247-3/+12
| | | | | | | | | | | | | | This was done in the original DTrace import, presumably because that made it a bit easier to handle includes. However, this can cause dtrace_getpcstack() to be inlined into dtrace_probe(), resulting in a missing frame in stack traces since dtrace_getpcstack() takes care to bump "aframes" to account for its own stack frame. To avoid this, compile dtrace_isa.c separately on all platforms. Add requisite includes. MFC after: 2 weeks Sponsored by: Innovate UK
* sdt: Fix aframe handling after commit ddf0ed09bd8fMark Johnston2024-07-081-1/+12
| | | | | | | | | | | | | | | | | | | DTrace probes have an "aframes" attribute, used when unwinding the stack from dtrace_probe(). It counts the number of leading frames to skip when returning a stack trace, thus is used to hide internal functions. Commit ddf0ed09bd8f set the aframes value for SDT probes to 0, which was correct for an earlier iteration of the patch, but now doesn't take sdt_probe()/sdt_probe6() into account. Fix the aframes definition for SDT probes. Also try to improve lockstat(1) output by adding an additional aframe for lockstat probes, which otherwise show internal mtx(9), rwlock(9), etc. functions as the probe "caller". This is not quite correct as the number of frames to skip may differ depending on the lock type and kernel configuration (see e.g., the MUTEX_NOINLINE kernel option), but this is not a new problem. Reported by: mjg Fixes: ddf0ed09bd8f ("sdt: Implement SDT probes using hot-patching")
* dtrace: Add a partial implementation of dtrace_getarg() on arm64Mark Johnston2024-06-201-5/+28
| | | | | | | | | | | For invop providers (i.e., fbt and kinst) we can simply reach into the invop trapframe to fetch argument registers for arguments 0-7; for argument 8 and beyond we have to read the value off of the stack. Reviewed by: Domagoj Stolfa, avg MFC after: 2 weeks Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D45649
* sdt: Support fetching the probe sixth argument with MI machineryMark Johnston2024-06-202-5/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | SDT calls dtrace_probe() directly, and this can be used to pass up to five probe arguments directly. To pass the sixth argument (SDT currently doesn't support more than this), we use a hack: just add additional parameters to the call and cast dtrace_probe accordingly. This happens to work on amd64, but doesn't work in general. Modify SDT to call dtrace_probe() after storing arguments beyond the first five in thread-local storage. Implement sdt_getargval() to fetch extra argument values this way. An alternative would be to use invop handlers instead and make sdt_probe_func point to a breakpoint instruction, so that one can extract arguments using the breakpoint exception trapframe, but this makes the providers more expensive when enabled and doesn't seem justified. This approach works well unless we want to add more than one or two more parameters to SDT probes, which seems unlikely at present. In particular, this fixes fetching the last argument of most ip and tcp probes on arm64. Reported by: rwatson Reviewed by: Domagoj Stolfa MFC after: 1 month Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D45648
* sdt: Implement SDT probes using hot-patchingMark Johnston2024-06-192-12/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The idea here is to avoid a memory access and conditional branch per probe site. Instead, the probe is represented by an "unreachable" unconditional function call. asm goto is used to store the address of the probe site (represented by a no-op sled) and the address of the function call into a tracepoint record. Each SDT probe carries a list of tracepoints. When the probe is enabled, the no-op sled corresponding to each tracepoint is overwritten with a jmp to the corresponding label. The implementation uses smp_rendezvous() to park all other CPUs while the instruction is being overwritten, as this can't be done atomically in general. The compiler moves argument marshalling code and the sdt_probe() function call out-of-line, i.e., to the end of the function. Per gallatin@ in D43504, this approach has less overhead when probes are disabled. To make the implementation a bit simpler, I removed support for probes with 7 arguments; nothing makes use of this except a regression test case. It could be re-added later if need be. The approach taken in this patch enables some more improvements: 1. We can now automatically fill out the "function" field of SDT probe names. The SDT macros let the programmer specify the function and module names, but this is really a bug and shouldn't have been allowed. The intent was to be able to have the same probe in multiple functions and to let the user restrict which probes actually get enabled by specifying a function name or glob. 2. We can avoid branching on SDT_PROBES_ENABLED() by adding the ability to include blocks of code in the out-of-line path. For example: if (SDT_PROBES_ENABLED()) { int reason = CLD_EXITED; if (WCOREDUMP(signo)) reason = CLD_DUMPED; else if (WIFSIGNALED(signo)) reason = CLD_KILLED; SDT_PROBE1(proc, , , exit, reason); } could be written SDT_PROBE1_EXT(proc, , , exit, reason, int reason; reason = CLD_EXITED; if (WCOREDUMP(signo)) reason = CLD_DUMPED; else if (WIFSIGNALED(signo)) reason = CLD_KILLED; ); In the future I would like to use this mechanism more generally, e.g., to remove branches and marshalling code used by hwpmc, and generally to make it easier to add new tracepoint consumers without having to add more conditional branches to hot code paths. Reviewed by: Domagoj Stolfa, avg MFC after: 2 months Differential Revision: https://reviews.freebsd.org/D44483
* dtrace: Remove LOCK_LEVELMark Johnston2024-06-191-2/+0
| | | | | | It is unused on FreeBSD. No functional change intended. MFC after: 1 week
* arm64: Support BTI checking in most of the kernelAndrew Turner2024-06-051-0/+3
| | | | | | | | | | | | | | | | | LLD has the -zbti-report=error argument to check if the BTI note is present when linking. To allow for this to be used when linking the kernel and modules: - Add the BTI note to the remaining assembly files - Mark ptrauth.c as protected by BTI - Disable -zbti-report for vmm hypervisor switching code as it's not used there. The linux64 module doesn't build with the flag as it includes vdso code that doesn't include the note. Reviewed by: imp, kib, emaste Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D45466
* arm64: Use void pointers for arguments to arm64_get_writable_addrJohn Baldwin2024-03-152-4/+4
| | | | | | | | No functional change, but this reduces diffs with CheriBSD downstream. Reviewed by: andrew Sponsored by: University of Cambridge, Google, Inc. Differential Revision: https://reviews.freebsd.org/D44344
* arm64: Switch the address argument to cpu_*cache* to a pointerJohn Baldwin2024-03-152-5/+3
| | | | | | | | No functional change, but this reduces diffs with CheriBSD downstream. Reviewed by: andrew Sponsored by: University of Cambridge, Google, Inc. Differential Revision: https://reviews.freebsd.org/D44342
* dtrace/profile: Set t_dtrace_trapframe for profile probesMark Johnston2024-01-071-1/+6
| | | | | | | | | | profile provider probes fire in the context of a timer interrupt. Thus, the "regs" action can make use of the interrupt trap frame to get register values when the interrupt happened in kernel mode. Make that trap frame available when possible so that "regs" works more or less as it already does with the fbt and kinst providers. MFC after: 1 week
* sys: Automated cleanup of cdefs and other formattingWarner Losh2023-11-2713-16/+1
| | | | | | | | | | | | | | | | Apply the following automated changes to try to eliminate no-longer-needed sys/cdefs.h includes as well as now-empty blank lines in a row. Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/ Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/ Remove /\n+#if.*\n#endif.*\n+/ Remove /^#if.*\n#endif.*\n/ Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/ Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/ Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/ Sponsored by: Netflix
* sys: Remove ancient SCCS tags.Warner Losh2023-11-276-6/+0
| | | | | | | | Remove ancient SCCS tags from the tree, automated scripting, with two minor fixup to keep things compiling. All the common forms in the tree were removed with a perl script. Sponsored by: Netflix
* dtrace: remove x86 non-EARLY_AP_STARTUP supportEd Maste2023-10-102-56/+0
| | | | | | | | After 792655abd64c EARLY_AP_STARTUP is mandatory for x86. Reviewed by: markj Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D42139
* dtrace: fix fbt regression for aarch64Igor Ostapenko2023-10-031-9/+8
| | | | | | | | | | | | | | | fbt computes incorrect instruction position for AArch64 kernel module symbol. The issue is with the for loop, it does an extra increment of instr pointer after the required instruction is found. Hence, a wrong instruction is targeted for patching. Signed-off-by: Igor Ostapenko <pm@igoro.pro> Fixes: 980746e5cb26 ("fbt: simplify arm64 function-prologue parsing") Reviewed by: markj Pull Request: https://github.com/freebsd/freebsd-src/pull/855 MFC after: 1 week
* kinst: do not look for a function epilogueChristos Margiolis2023-09-152-14/+15
| | | | | | | | | | | | | kinst currently only traces functions that start and end with the usual function prologue and epilogue respectively. Ignoring functions that do not have an epilogue however, makes the filtering too strict, as this means that we can not trace functions that never return (e.g vnlru_proc()). This patch relaxes the filtering and only checks whether the function pushes the frame pointer. Reviewed by: markj Approved by: markj Differential Revision: https://reviews.freebsd.org/D41876
* sys: Remove $FreeBSD$: one-line .c patternWarner Losh2023-08-161-2/+0
| | | | Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/
* sys: Remove $FreeBSD$: one-line .c comment patternWarner Losh2023-08-161-1/+0
| | | | Remove /^/[*/]\s*\$FreeBSD\$.*\n/
* sys: Remove $FreeBSD$: one-line .h patternWarner Losh2023-08-167-7/+0
| | | | Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/
* sys: Remove $FreeBSD$: two-line .h patternWarner Losh2023-08-1645-90/+0
| | | | Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
* kinst.h: make pointer to probe in kinst_cpu_state constChristos Margiolis2023-07-191-1/+1
| | | | | | | Fixes: 5b701ed19c2e ("kinst: start moving towards per-probe trampolines") Sponsored by: The FreeBSD Foundation
* Revert "dtrace: cache current probe in kdtrace_thread_t"Christos Margiolis2023-07-191-2/+0
| | | | | | | | | | | This reverts commit 22508c8b6c284ad9ad1385c703e224e1ed3fbd7c. The t_kinst_curprobe field is no longer needed by kinst. Reviewed by: markj Approved by: markj (mentor) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D41031
* kinst: port to arm64Christos Margiolis2023-07-192-0/+481
| | | | | | | Reviewed by: markj Approved by: markj (mentor) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D40337
* kinst: use per-probe trampolines in riscvChristos Margiolis2023-07-191-71/+42
| | | | | | | Reviewed by: markj Approved by: markj (mentor) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D40963
* kinst: start moving towards per-probe trampolinesChristos Margiolis2023-07-193-5/+61
| | | | | | | | | | | | | | | | | | | | | | Using per-CPU and per-thread trampolines is expensive and error-prone, since we're rewriting the same memory blocks constantly. Per-probe trampolines solve this problem by giving each probe its own block of executable memory, which more or less remains the same after the initial write. What this patch does, is get rid of the initialization code which allocates a trampoline for each thread, and instead let each port of kinst allocate a trampoline for each new probe created. It also sets up the infrastructure needed to support the new trampoline scheme. This change is not currently supported on amd64, as the amd64 port needs further changes to work, so this is a temporary/gradual patch to fix the riscv and arm64 ports. Reviewed by: markj Approved by: markj (mentor) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D40962
* kinst: exclude cpu_switchChristos Margiolis2023-07-191-0/+7
| | | | | | | Reviewed by: markj Approved by: markj (mentor) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D40985
* kinst: check for 'push %rbp' anywhere in the functionChristos Margiolis2023-07-191-6/+17
| | | | | | | | | | | | | Currently kinst checks if only the first instruction is 'push %rbp', essentially excluding functions that do push RBP, but not in the first instruction. This patch modifies kinst to check for 'push %rbp', as well, as a following 'pop %rbp', anywhere in the function. This behavior also matches that of FBT. Reviewed by: markj Approved by: markj (mentor) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D40283
* kinst.h: reorder function declarations based on implementation fileChristos Margiolis2023-07-191-6/+9
| | | | | | | Reviewed by: markj Approved by: markj (mentor) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D41032
* kinst: update LICENSE headersChristos Margiolis2023-07-045-8/+28
| | | | | | | Reviewed by: markj Approved by: markj (mentor) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D40875
generated by cgit v1.2.3 (git 2.25.1) at 2026-02-04 16:18:31 +0000