aboutsummaryrefslogtreecommitdiff
path: root/sys/cddl/dev/dtrace/aarch64
Commit message (Collapse)AuthorAgeFilesLines
* dtrace: Deduplicate dtrace_sync() and dtrace_xcall() implementationsMark Johnston2025-09-181-27/+0
| | | | | | | | | dtrace_xcall() is just a thin wrapper around smp_rendezvous_cpus(). There's no need for six identical implementations to live in MD layers. No functional change intended. MFC after: 2 weeks
* dtrace/arm64: Fix dtrace_gethrtime()Mark Johnston2025-03-101-2/+31
| | | | | | | | | | | | | | | | | | | | | This routine returns a monotonic count of the number of nanoseconds elapsed since the previous call. On arm64 it uses the generic system timer. The implementation multiplies the counter value by 10**9 then divides by the counter frequency, but this multiplication can overflow. This can result in trace records with non-monotonic timestamps, which breaks libdtrace's temporal ordering algorithm. An easy fix is to reverse the order of operations, since the counter frequency will in general be smaller than 10**9. (In fact, it's mandated to be 1Ghz in ARMv9, which makes life simple.) However, this can give a fair bit of error. Adopt the calculation used on amd64, with tweaks to handle frequencies as low as 1MHz: the ARM generic timer documentation suggests that ARMv8 timers are typically in the 1MHz-50MHz range, which is true on arm64 systems that I have access to. MFC after: 2 weeks Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D49244
* Revert "dtrace/arm64: Simplify dtrace_getarg() slightly"Mark Johnston2025-01-291-1/+3
| | | | | | | | | The change is clearly wrong as it removes a dereference of the pointer into the stack. Just revert for now. This reverts commit 70c067062217a5295ed321427b91cfd857c18b77. Reported by: jrtc27
* dtrace/arm64: Simplify dtrace_getarg() slightlyMark Johnston2025-01-291-3/+1
| | | | | | | | | The use of memcpy here is redundant, and also incorrect since memcpy() might be instrumented by fbt or kinst. dtrace_bcopy() exists, but we don't need it. MFC after: 2 weeks Sponsored by: Innovate UK
* dtrace/arm64: Simplify variable declarations in the invop handlerMark Johnston2025-01-221-6/+4
| | | | | | | | | Remove some unused variables and reduce the scope of some others. No functional change intended. MFC after: 1 week Sponsored by: Innovate UK
* dtrace: Avoid including dtrace_isa.c directly into dtrace.cMark Johnston2024-07-241-0/+3
| | | | | | | | | | | | | | This was done in the original DTrace import, presumably because that made it a bit easier to handle includes. However, this can cause dtrace_getpcstack() to be inlined into dtrace_probe(), resulting in a missing frame in stack traces since dtrace_getpcstack() takes care to bump "aframes" to account for its own stack frame. To avoid this, compile dtrace_isa.c separately on all platforms. Add requisite includes. MFC after: 2 weeks Sponsored by: Innovate UK
* dtrace: Add a partial implementation of dtrace_getarg() on arm64Mark Johnston2024-06-201-5/+28
| | | | | | | | | | | For invop providers (i.e., fbt and kinst) we can simply reach into the invop trapframe to fetch argument registers for arguments 0-7; for argument 8 and beyond we have to read the value off of the stack. Reviewed by: Domagoj Stolfa, avg MFC after: 2 weeks Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D45649
* arm64: Support BTI checking in most of the kernelAndrew Turner2024-06-051-0/+3
| | | | | | | | | | | | | | | | | LLD has the -zbti-report=error argument to check if the BTI note is present when linking. To allow for this to be used when linking the kernel and modules: - Add the BTI note to the remaining assembly files - Mark ptrauth.c as protected by BTI - Disable -zbti-report for vmm hypervisor switching code as it's not used there. The linux64 module doesn't build with the flag as it includes vdso code that doesn't include the note. Reviewed by: imp, kib, emaste Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D45466
* sys: Remove ancient SCCS tags.Warner Losh2023-11-271-1/+0
| | | | | | | | Remove ancient SCCS tags from the tree, automated scripting, with two minor fixup to keep things compiling. All the common forms in the tree were removed with a perl script. Sponsored by: Netflix
* sys: Remove $FreeBSD$: one-line .h patternWarner Losh2023-08-161-1/+0
| | | | Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/
* sys: Remove $FreeBSD$: two-line .h patternWarner Losh2023-08-163-6/+0
| | | | Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
* dtrace/arm64: Fix user memory access routinesMark Johnston2023-06-151-7/+9
| | | | | | | | | | Use unprivileged loads to access user memory. Without this, the accesses trap and various dtrace actions such as ustack() fail. Reviewed by: andrew MFC after: 1 week Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D40540
* dtrace/arm64: Store the fault address when suppressing a page faultMark Johnston2023-06-151-1/+1
| | | | | | Reviewed by: andrew Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D40539
* arm64: Remove struct arm64_frameMark Johnston2023-06-151-1/+1
| | | | | | | | | | | It was used in one place and was added specifically to support dtrace stack unwinding code. Write an equivalent expression using struct unwind_state instead. No functional change intended. Reviewed by: andrew MFC after: 1 week Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D40538
* dtrace: rename rp to frame in dtrace_getreg()Christos Margiolis2023-05-231-5/+5
| | | | | | | Reviewed by: mhorne, markj Approved by: markj (mentor) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D40231
* dtrace: add register bindings for arm64Christos Margiolis2023-05-232-5/+48
| | | | | | | Reviewed by: mhorne, markj Approved by: markj (mentor) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D39956
* dtrace: implement dtrace_instr_size() for arm64Christos Margiolis2023-05-231-0/+14
| | | | | | | Reviewed by: markj Approved by; markj (mentor) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D39955
* dtrace: remove unused definesMitchell Horne2023-01-121-6/+0
| | | | | | | Reviewed by: markj, emaste MFC after: 3 days Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D37659
* dtrace: Add a "regs" variableMark Johnston2022-10-041-8/+9
| | | | | | | | | | | | | | | | | | | | | | | | This allows invop-based providers (i.e., fbt and kinst) to expose the register file of the CPU at the point where the probe fired. It does not work for SDT providers because their probes are implemented as plain function calls and so don't save registers. It's not clear what semantics "regs" should have for them anyway. This is akin to "uregs", which nominally provides access to the userspace registers. In fact, DIF already had a DIF_VAR_REGS variable defined, it was simply unimplemented. Usage example: print the contents of %rdi upon each call to amd64_syscall(): fbt::amd64_syscall:entry {printf("%x", regs[R_RDI]);} Note that the R_* constants are defined in /usr/lib/dtrace/regs_x86.d. Currently there are no similar definitions for non-x86 platforms. Reviewed by: christos MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D36799
* Fix dtrace fbt return probes on arm64Andrew Turner2021-12-141-1/+1
| | | | | | | | | | As with arm and riscv fix return fbt probes on arm64. arg0 should be the offset within the function of the return instruction and arg1 should be the return value. Reviewed by: kp, markj Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D33440
* Create sys/reg.h for the common code previously in machine/reg.hAndrew Turner2021-08-301-1/+0
| | | | | | | | | | Move the common kernel function signatures from machine/reg.h to a new sys/reg.h. This is in preperation for adding PT_GETREGSET to ptrace(2). Reviewed by: imp, markj Sponsored by: DARPA, AFRL (original work) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D19830
* Handle functions that use a nop in the arm64 fbtAndrew Turner2021-03-031-0/+5
| | | | | | | | | To trace leaf asm functions we can insert a single nop instruction as the first instruction in a function and trigger off this. Reviewed by: gnn Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D28132
* Reimplement the arm64 dtrace_gethrtime(), which provides theRobert Watson2021-02-191-10/+13
| | | | | | | | | | | | | high-resolution nanosecond timestamp used for the DTrace 'timestamp' built-in variable. The new implementation uses the EL0 cycle counter and frequency registers in ARMv8-A. This replaces a previous implementation that relied on an instrumentation-safe implementation of getnanotime(), which provided only timer resolution. MFC after: 3 days Reviewed by: andrew, bsdimp (older version) Useful comments appreciated: jrtc27, emaste
* arm64: Improve DDB backtrace supportJessica Clarke2021-02-011-10/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | The existing implementation relies on each trap handler saving a normal stack frame record, which is a waste of time and space when we're already saving a trapframe to the stack. It's also wrong as it currently saves LR not ELR. Instead of patching it up, rewrite it based on the RISC-V implementation with inspiration from the amd64 implementation for how to handle vectored traps to provide an improved implementation. This includes compressing the information down to one line like other architectures rather than the highly-verbose old form that repeats itself by printing LR and FP in one frame only to print them as PC and SP in the next. It also includes printing out actually useful information about the traps that occurred, though FAR is not saved in the trapframe so we cannot print it (in general it can be clobbered between when the trap happened and now), only ESR. The AAPCS also allows the stack frame record to be located anywhere in the frame, not just the top, so the caller's SP is not at a fixed offset from the callee's FP like on almost all other architectures in existence. This means there is no way to derive the caller's SP in the unwinder, and so we have to drop that bit of (unused) state everywhere. Reviewed by: jhb, markj Differential Revision: https://reviews.freebsd.org/D28026
* Handle using a sub instruction in the arm64 fbtAndrew Turner2021-01-121-0/+6
| | | | | | | | Some stack frames are too large for a store pair instruction we already detect in the arm64 fbt code. Add support for handling subtracting the stack pointer directly. Sponsored by: Innovate UK
* Changes that improve DTrace FBT reliability on freebsd/arm64:Robert Watson2021-01-111-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Implement a dtrace_getnanouptime(), matching the existing dtrace_getnanotime(), to avoid DTrace calling out to a potentially instrumentable function. (These should probably both be under KDTRACE_HOOKS. Also, it's not clear to me that they are correct implementations for the DTrace thread time functions they are used in .. fixes for another commit.) - Don't allow FBT to instrument functions involved in EL1 exception handling that are involved in FBT trap processing: handle_el1h_sync() and do_el1h_sync(). - Don't allow FBT to instrument DDB and KDB functions, as that makes it rather harder to debug FBT problems. Prior to these changes, use of FBT on FreeBSD/arm64 rapidly led to kernel panics due to recursion in DTrace. Reliable FBT on FreeBSD/arm64 is reliant on another change from @andrew to have the aarch64 instrumentor more carefully check that instructions it replaces are against the stack pointer, which can otherwise lead to memory corruption. That change remains under review. MFC after: 2 weeks Reviewed by: andrew, kp, markj (earlier version), jrtc27 (earlier version) Differential revision: https://reviews.freebsd.org/D27766
* Check that the frame pointer is within the current stack.John Baldwin2020-12-081-12/+7
| | | | | | | | | | | | | | This same check is used on other architectures. Previously this would permit a stack frame to unwind into any arbitrary kernel address (including unmapped addresses). Reviewed by: andrew, markj Obtained from: CheriBSD Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D27362 Notes: svn path=/head/; revision=368455
* Use uintptr_t instead of uint64_t for pointers in stack frames.John Baldwin2020-12-011-4/+4
| | | | | | | | | | Reviewed by: andrew Obtained from: CheriBSD Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D27361 Notes: svn path=/head/; revision=368245
* Don't overflow the trap frame when accessing lr or xzr.Andrew Turner2020-07-171-4/+29
| | | | | | | | | | | | | | | | When emulating a load pair or store pair in dtrace on arm64 we need to copy the data between the stack and trap frame. When the registers are either the link register or the zero register we will access memory past the end of the trap frame as these are encoded as registers 30 and 31 respectively while the array they access only has 30 entries. Fix this by creating 2 helper functions to perform the operation with special cases for these registers. Sponsored by: Innovate UK Notes: svn path=/head/; revision=363272
* DTrace aarch64: Avoid calling unwind_frame() in the probe context.Mark Johnston2018-05-121-4/+8
| | | | | | | | | | | | | | | unwind_frame() may be instrumented by FBT, leading to recursion into dtrace_probe(). Manually inline unwind_frame() as we do with stack unwinding code for other architectures. Submitted by: Domagoj Stolfa Reviewed by: manu MFC after: 1 week Sponsored by: DARPA / AFRL Differential Revision: https://reviews.freebsd.org/D15359 Notes: svn path=/head/; revision=333570
* Rename assym.s to assym.incEd Maste2018-03-201-1/+1
| | | | | | | | | | | | assym is only to be included by other .s files, and should never actually be assembled by itself. Reviewed by: imp, bdrewery (earlier) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D14180 Notes: svn path=/head/; revision=331254
* Corrected misspelled versions of rendezvous.Patrick Kelsey2017-04-091-2/+2
| | | | | | | | | | | | The MFC will include a compat definition of smp_no_rendevous_barrier() that calls smp_no_rendezvous_barrier(). Reviewed by: gnn, kib MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D10313 Notes: svn path=/head/; revision=316648
* Fix improper use of "its".Bryan Drewery2016-11-081-1/+1
| | | | | | | Sponsored by: Dell EMC Isilon Notes: svn path=/head/; revision=308457
* Set oldfp so the check for fp == oldfp works as expected.Andrew Turner2016-05-311-1/+3
| | | | | | | | Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation Notes: svn path=/head/; revision=301034
* Fix dtrace_interrupt_disable and dtrace_interrupt_enable by having theAndrew Turner2016-05-271-1/+2
| | | | | | | | | | | | | | | former return the current status for the latter to use. Without this we could enable interrupts when they shouldn't be. It's still not quite right as it should only update the bits we care about, bit should be good enough until the correct fix can be tested. PR: 204270 Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation Notes: svn path=/head/; revision=300850
* Try to unbreak the build after r300611 by including the headerBjoern A. Zeeb2016-05-241-0/+1
| | | | | | | | | defining VM_MIN_KERNEL_ADDRESS. Sponsored by: DARPA/AFRL Notes: svn path=/head/; revision=300619
* Mark all memory before the kernel as toxic to DTrace.Andrew Turner2016-05-241-1/+1
| | | | | | | | Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation Notes: svn path=/head/; revision=300611
* Make the second argument of dtrace_invop() a trapframe pointer.Mark Johnston2016-04-171-7/+7
| | | | | | | | | | | | | | | | Currently this argument is a pointer into the stack which is used by FBT to fetch the first five probe arguments. On all non-x86 architectures it's simply the trapframe address, so this change has no functional impact. On amd64 it's a pointer into the trapframe such that stack[1 .. 5] gives the first five argument registers, which are deliberately grouped together in the amd64 trapframe definition. A trapframe argument simplifies the invop handlers on !x86 and makes the x86 FBT invop handler easier to understand. Moreover, it allows for invop handlers that may want to modify the register set of the interrupted thread. Notes: svn path=/head/; revision=298171
* Implement dtrace_getupcstack in ARM64Wojciech Macek2016-04-061-1/+118
| | | | | | | | | | | | | | Allow using DTRACE for performance analysis of userspace applications - the function call stack can be captured. This is almost an exact copy of AMD64 solution. Obtained from: Semihalf Sponsored by: Cavium Reviewed by: emaste, gnn, jhibbits Differential Revision: https://reviews.freebsd.org/D5779 Notes: svn path=/head/; revision=297611
* Remove unused variables dtrace_in_probe and dtrace_in_probe_addr.Mark Johnston2016-03-171-2/+0
| | | | Notes: svn path=/head/; revision=296990
* As <machine/vmparam.h> is included from <vm/vm_param.h>, there is noSvatopluk Kraus2016-02-221-1/+0
| | | | | | | | | | | need to include it explicitly when <vm/vm_param.h> is already included. Suggested by: alc Reviewed by: alc Differential Revision: https://reviews.freebsd.org/D5379 Notes: svn path=/head/; revision=295882
* First cut of DTrace for AArch64.Ruslan Bukin2015-07-014-0/+822
Reviewed by: andrew, emaste Sponsored by: ARM Limited Differential Revision: https://reviews.freebsd.org/D2738 Notes: svn path=/head/; revision=285009
generated by cgit v1.2.3 (git 2.25.1) at 2026-02-09 19:05:30 +0000