aboutsummaryrefslogtreecommitdiff
path: root/sys/i386/include/cpufunc.h
Commit message (Collapse)AuthorAgeFilesLines
* x86: directly use clflushopt mnemonic in cpufunc.hKonstantin Belousov2025-09-211-1/+1
| | | | | | | | | We already use clflushopt in support.S, there is no reason to manually construct the encoding. Initially it was done because toolchains did not supported the (then) new instruction. Sponsored by: The FreeBSD Foundation MFC after: 1 week
* libkern: add ilog2 macroDoug Moore2024-06-031-9/+0
| | | | | | | | | | | | | | | The kernel source contains several definitions of an ilog2 function; some are slower than necessary, and one of them is incorrect. Elimininate them all and define an ilog2 macro in libkern to replace them, in a way that is fast, correct for all argument types, and, in a GENERIC kernel, includes a check for an invalid zero parameter. Folks at Microsoft have verified that having a correct ilog2 definition for their MANA driver doesn't break it. Reviewed by: alc, markj, mhorne (older version), jhibbits (older version) Differential Revision: https://reviews.freebsd.org/D45170 Differential Revision: https://reviews.freebsd.org/D45235
* sys: Remove $FreeBSD$: two-line .h patternWarner Losh2023-08-161-2/+0
| | | | Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
* Consistently provide ffs/fls using builtinsMitchell Horne2023-07-061-42/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use of compiler builtin ffs/ctz functions will result in optimized instruction sequences when possible, and fall back to calling a function provided by the compiler run-time library. We have slowly shifted our platforms to take advantage of these builtins in 60645781d613 (arm64), 1c76d3a9fbef (arm), 9e319462a03a (powerpc, partial). Some platforms still rely on the libkern implementations of these functions provided by libkern, namely riscv, powerpc (ffs*, flsll), and i386 (ffsll and flsll). These routines are slow, as they perform a linear search for the bit in question. Even on platforms lacking dedicated bit-search instructions, such as riscv, the compiler library will provide better-optimized routines, e.g. by using binary search. Consolidate all definitions of these functions (whether currently using builtins or not) to libkern.h. This should result in equivalent or better performing routines in all cases. One wart in all of this is the existing HAVE_INLINE_F*** macros, which we use in a few places to conditionally avoid the slow libkern routines. These aren't easily removed in one commit. For now, provide these defines unconditionally, but marked for removal after subsequent cleanup. Removal of the now unused libkern routines will follow in the next commit. Reviewed by: dougm, imp (previous version) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D40698
* x86: cleanup in machine/cpufunc.hBrooks Davis2022-06-131-7/+0
| | | | | | Reduce diffs between amd64 and i386 and improve whitespace Reviewed by: jhb, imp
* Remove checks for <sys/cdefs.h> being included.John Baldwin2022-04-121-4/+0
| | | | | | | | | These files no longer depend on the macros required when these checks were added. PR: 263102 (exp-run) Reviewed by: brooks, imp, emaste Differential Revision: https://reviews.freebsd.org/D34804
* Remove checks for __CC_SUPPORTS__INLINE assuming it is always true.John Baldwin2022-04-121-74/+0
| | | | | | | | | All supported compilers (modern versions of GCC and clang) support this. PR: 263102 (exp-run) Reviewed by: imp Differential Revision: https://reviews.freebsd.org/D34802
* Remove checks for __GNUCLIKE_ASM assuming it is always true.John Baldwin2022-04-121-3/+3
| | | | | | | | | | | | | All supported compilers (modern versions of GCC and clang) support this. Many places didn't have an #else so would just silently do the wrong thing. Ancient versions of icc (the original motivation for this) are no longer a compiler FreeBSD supports. PR: 263102 (exp-run) Reviewed by: brooks, imp Differential Revision: https://reviews.freebsd.org/D34797
* x86: cpufunc: Add rdtsc_ordered()Adam Fenn2021-08-141-0/+14
| | | | | | | | | | | | | Add a variant of 'rdtsc()' that performs the ordered version of 'rdtsc' appropriate for the invoking x86 variant. Also, expose the 'lfence'-ed and 'mfence'-ed 'rdtsc()' variants needed by 'rdtsc_ordered()' for general use. Sponsored by: Juniper Networks, Inc. Sponsored by: Klara, Inc. Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D31416
* x86: cpufunc: Add rdtscp_aux()Adam Fenn2021-08-141-0/+9
| | | | | | | | | | Add a variant of 'rdtscp()' that retains and returns the 'IA32_TSC_AUX' value read by 'rdtscp'. Sponsored By: Juniper Networks, Inc. Sponsored By: Klara, Inc. Reviewed by: markj, kib Differential Revision: https://reviews.freebsd.org/D31415
* x86: Add rdtscp32() into cpufunc.h.Konstantin Belousov2021-01-101-0/+9
| | | | | | | Suggested by: markj MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D27986
* Provide convenience C wrappers for RDPKRU and WRPKRU instructions.Konstantin Belousov2019-02-191-0/+16
| | | | | | | | | | | Reviewed by: markj Tested by: pho Sponsored by: The FreeBSD Foundation MFC after: 3 days Differential revision: https://reviews.freebsd.org/D18893 Notes: svn path=/head/; revision=344296
* Provide userspace versions of do_cpuid() and cpuid_count() on i386.Konstantin Belousov2019-02-141-4/+30
| | | | | | | | | | | | | Some older compilers, when generating PIC code, cannot handle inline asm that clobbers %ebx (because %ebx is used as the GOT offset register). Userspace versions avoid clobbering %ebx by saving it to stack before executing the CPUID instruction. Sponsored by: The FreeBSD Foundation MFC after: 1 week Notes: svn path=/head/; revision=344118
* Tell the compiler that rdtscp clobbers %ecx.Mark Johnston2018-06-091-1/+1
| | | | Notes: svn path=/head/; revision=334890
* cpufunc: add rdtscp for x86Matt Macy2018-06-071-0/+9
| | | | Notes: svn path=/head/; revision=334746
* sys: further adoption of SPDX licensing ID tags.Pedro F. Giffuni2017-11-201-0/+2
| | | | | | | | | | | | | | | | | Mainly focus on files that use BSD 3-Clause license. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. Special thanks to Wind River for providing access to "The Duke of Highlander" tool: an older (2014) run over FreeBSD tree was useful as a starting point. Notes: svn path=/head/; revision=326023
* x86: Tag some intrinsics with __pure2Conrad Meyer2017-08-031-6/+6
| | | | | | | | | | | | | Some C wrappers for x86 instructions do not touch global memory and only act on their arguments; they can be marked __pure2, aka __const__. Without this annotation, Clang 3.9.1 is not intelligent enough on its own to grok that these functions are __const__. Submitted by: Anton Rang <anton.rang AT isilon.com> Sponsored by: Dell EMC Isilon Notes: svn path=/head/; revision=322032
* Don't access the reserved registers %dr4 and %dr5 on i386.Bruce Evans2017-03-171-32/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On the original i386, %dr[4-5] were unimplemented but not very clearly reserved, so debuggers read them to print them. i386 was still doing this. On the original athlon64, %dr[4-5] are documented as reserved but are aliased to %dr[6-7] unless CR4_DE is set, when accessing them traps. On 2 of my systems, accessing %dr[4-5] trapped sometimes. On my Haswell system, the apparent randomness was because the boot CPU starts with CR4_DE set while all other CPUs start with CR4_DE clear. FreeBSD doesn't support the data breakpoints enabled by CR4_DE and it never changes this flag, so the flag remains different across CPUs and the behaviour seemed inconsistent except while booting when the CPU doesn't change. The invalid accesses broke: - read access for printing the registers in ddb "show watches" on CPUs with CR4_DE set - read accesses in fill_dbregs() on CPUs with CR4_DE set. This didn't implement panic(3) since the user case always skipped %dr[4-5]. - write accesses in set_dbregs(). This also didn't affect userland. When it didn't trap, the aliasing made it fragile. Don't print the dummy (zero) values of %dr[4-5] in "show watches" for i386 or amd64. Fix style bugs near this printing. amd64 also has space in the dbregs struct for the reserved %dr[8-15] and already didn't print the dummy values for these, and never accessed any of the 10 reserved debug registers. Remove cpufuncs for making the invalid accesses. Even amd64 had these. Notes: svn path=/head/; revision=315454
* Renumber copyright clause 4Warner Losh2017-02-281-1/+1
| | | | | | | | | | | | Renumber cluase 4 to 3, per what everybody else did when BSD granted them permission to remove clause 3. My insistance on keeping the same numbering for legal reasons is too pedantic, so give up on that point. Submitted by: Jan Schaumann <jschauma@stevens.edu> Pull Request: https://github.com/freebsd/freebsd/pull/96 Notes: svn path=/head/; revision=314436
* Use SFENCE for ordering CLFLUSHOPT.Konstantin Belousov2017-01-201-0/+7
| | | | | | | | | | | | SDM states that CLFLUSHOPT instructions can be ordered with other writes by SFENCE, heavier MFENCE is not required. Reviewed by: alc Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Notes: svn path=/head/; revision=312555
* The prefix for CLFLUSHOPT is 0x66. It was right on amd64.Konstantin Belousov2015-10-301-1/+1
| | | | | | | Sponsored by: The FreeBSD Foundation Notes: svn path=/head/; revision=290188
* Add CLFLUSHOPT instruction wrappers.Konstantin Belousov2015-10-231-0/+7
| | | | | | | | Sponsored by: The FreeBSD Foundation MFC after: 1 week Notes: svn path=/head/; revision=289824
* Remove support for Xen PV domU kernels. Support for HVM domU kernelsJohn Baldwin2015-04-301-42/+4
| | | | | | | | | | | | | | | | | | | | | | | | remains. Xen is planning to phase out support for PV upstream since it is harder to maintain and has more overhead. Modern x86 CPUs include virtualization extensions that support HVM guests instead of PV guests. In addition, the PV code was i386 only and not as well maintained recently as the HVM code. - Remove the i386-only NATIVE option that was used to disable certain components for PV kernels. These components are now standard as they are on amd64. - Remove !XENHVM bits from PV drivers. - Remove various shims required for XEN (e.g. PT_UPDATES_FLUSH, LOAD_CR3, etc.) - Remove duplicate copy of <xen/features.h>. - Remove unused, i386-only xenstored.h. Differential Revision: https://reviews.freebsd.org/D2362 Reviewed by: royger Tested by: royger (i386/amd64 HVM domU and amd64 PVH dom0) Relnotes: yes Notes: svn path=/head/; revision=282274
* Add x2APIC support. Enable it by default if CPU is capable. TheKonstantin Belousov2015-02-091-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | hw.x2apic_enable tunable allows disabling it from the loader prompt. To closely repeat effects of the uncached memory ops when accessing registers in the xAPIC mode, the x2APIC writes to MSRs are preceeded by mfence, except for the EOI notifications. This is probably too strict, only ICR writes to send IPI require serialization to ensure that other CPUs see the previous actions when IPI is delivered. This may be changed later. In vmm justreturn IPI handler, call doreti_iret instead of doing iretd inline, to handle corner conditions. Note that the patch only switches LAPICs into x2APIC mode. It does not enables FreeBSD to support > 255 CPUs, which requires parsing x2APIC MADT entries and doing interrupts remapping, but is the required step on the way. Reviewed by: neel Tested by: pho (real hardware), neel (on bhyve) Discussed with: jhb, grehan Sponsored by: The FreeBSD Foundation MFC after: 2 months Notes: svn path=/head/; revision=278473
* MFamd64: Add support for extended FPU states on i386. This includesJohn Baldwin2014-11-021-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | support for AVX on i386. - Similar to amd64, move the FPU save area out of the PCB and instead store saved FPU state in a variable-sized buffer after the PCB on the stack. - To support the variable PCB location, alter the locore code to only use the bottom-most page of proc0stack for init386(). init386() returns the correct stack pointer to locore which adjusts the stack for thread0 before calling mi_startup(). - Don't bother setting cr3 in thread0's pcb in locore before calling init386(). It wasn't used (init386() overwrote it at the end) and it doesn't work with the variable-sized FPU save area. - Remove the new-bus attachment from npx. This was only ever useful for external co-processors using IRQ13, but those have not been supported for several years. npxinit() is now called much earlier during boot (init386()) similar to amd64. - Implement PT_{GET,SET}XSTATE and I386_GET_XFPUSTATE. - npxsave() is now only called from context switch contexts so it can use XSAVEOPT. Differential Revision: https://reviews.freebsd.org/D1058 Reviewed by: kib Tested on: FreeBSD/i386 VM under bhyve on Intel i5-2520 Notes: svn path=/head/; revision=273995
* provide fast versions of ffsl and flsl for i386; ffsll and flsll for amd64Andriy Gapon2014-02-141-0/+16
| | | | | | | | | | Reviewed by: jhb MFC after: 10 days X-MFC note: consider thirdparty modules depending on these symbols Sponsored by: HybridCluster Notes: svn path=/head/; revision=261891
* Add lfence().Konstantin Belousov2012-08-011-0/+7
| | | | | | | MFC after: 1 week Notes: svn path=/head/; revision=238972
* Add a clts() wrapper around the 'clts' instruction to <machine/cpufunc.h>John Baldwin2012-07-091-0/+10
| | | | | | | | | | | | | | on x86 and use that to implement stop_emulating() in the fpu/npx code. Reimplement start_emulating() in the non-XEN case by using load_cr0() and rcr0() instead of the 'lmsw' and 'smsw' instructions. Intel explicitly discourages the use of 'lmsw' and 'smsw' on 80386 and later processors in the description of these instructions in Volume 2 of the ADM. Reviewed by: kib MFC after: 1 month Notes: svn path=/head/; revision=238311
* Correct cpu_monitor() and cpu_mwait() for amd64. These instructions takeJung-uk Kim2011-07-051-5/+7
| | | | | | | | | %rcx as "extensions" in long mode. If any unused bit is set in %rcx, these instructions cause general protection fault. Fix style nits and synchronize i386 with amd64. Notes: svn path=/head/; revision=223796
* Add a function rdtsc32() to read lower 32 bits from TSC and discard upperJung-uk Kim2011-04-141-0/+9
| | | | | | | | | 32 bits. Some times compiler inserts unnecessary instructions to preserve unused upper 32 bits even when it is casted to a 32-bit value. It reduces such compiler mistakes where every cycle counts. Notes: svn path=/head/; revision=220631
* Consistently use __volatile as the rest of this file.Jung-uk Kim2011-04-141-5/+5
| | | | Notes: svn path=/head/; revision=220629
* Consistently use C99 standard integers as the rest of this file.Jung-uk Kim2011-04-141-6/+6
| | | | Notes: svn path=/head/; revision=220627
* Change the parameter passed to the inline assembly to u_shortRoman Divacky2010-09-031-15/+15
| | | | | | | | | | | as we are dealing with 16bit segment registers. Change mov to movw. Approved by: rpaulo (mentor) Reviewed by: kib, rink Notes: svn path=/head/; revision=212177
* Quiet variable "shadows" warning:David E. O'Brien2010-01-011-18/+18
| | | | | | | | | | sys/vmmeter.h: warning: shadowed declaration is here machine/cpufunc.h: In function 'insw': machine/cpufunc.h: warning: declaration of 'cnt' shadows a global declaration ..snip.. Notes: svn path=/head/; revision=201369
* make read_eflags and write_eflags accomplish the same effect on PVM as native,Kip Macy2009-10-011-10/+10
| | | | | | | simplifying interrupt handling Notes: svn path=/head/; revision=197693
* cpufunc.h: unify/correct style of c extension namesAndriy Gapon2009-09-301-2/+2
| | | | | | | | | | | | | i386 and amd64 archs only. inline => __inline. [1] __asm__ => __asm. [2] Reviewed by: kib, jhb [1] Suggested by: kib [2] MFC after: 1 week Notes: svn path=/head/; revision=197647
* As was done in r195820 for amd64, use clflush for flushing cache linesKonstantin Belousov2009-07-291-0/+14
| | | | | | | | | | | | | | | when memory page caching attributes changed, and CPU does not support self-snoop, but implemented clflush, for i386. Take care of possible mappings of the page by sf buffer by utilizing the mapping for clflush, otherwise map the page transiently. Amd64 used direct map. Proposed and reviewed by: alc Approved by: re (kensmith) Notes: svn path=/head/; revision=195940
* Move (read|write)_cyrix_reg() inlines from specialreg.h to cpufunc.h.John Baldwin2009-06-161-0/+16
| | | | | | | specialreg.h now consists solely of register-related macros. Notes: svn path=/head/; revision=194295
* Clobber "cc" instead of using volatile.Ed Schouten2009-06-131-2/+2
| | | | | | | Submitted by: Christoph Mallon Notes: svn path=/head/; revision=194115
* Simplify in/out functions (for i386 and AMD64).Ed Schouten2009-04-111-79/+8
| | | | | | | | | | | Remove a hack to generate more efficient code for port numbers below 0x100, which has been obsolete for at least ten years, because GCC has an asm constraint to specify that. Submitted by: Christoph Mallon <christoph mallon gmx de> Notes: svn path=/head/; revision=190919
* Change some movl's to mov's. Newer GAS no longer accept 'movl' instructionsDavid E. O'Brien2009-01-311-5/+5
| | | | | | | | | for moving between a segment register and a 32-bit memory location. Looked at by: jhb Notes: svn path=/head/; revision=187948
* - move gdt, ldt allocation to before KPT allocationKip Macy2008-10-191-2/+6
| | | | | | | | | | | | - fix bugs where we would: - try to map the hypervisors address space - accidentally kick out an existing kernel mapping for some domain creation memory allocation sizes - accidentally skip a 2MB kernel mapping for some domain creation memory allocation sizes - don't rely on trapping in to xen to read rcr2, reference through vcpu - whitespace cleanups Notes: svn path=/head/; revision=184040
* - clean up interrupt handling for xen a tiny bitKip Macy2008-08-201-3/+3
| | | | | | | | | | - parse the command line in to kenv - defer shutdown watcher until later in boot MFC after: 1 month Notes: svn path=/head/; revision=181911
* Integrate support for xen in to i386 common code.Kip Macy2008-08-151-4/+40
| | | | | | | MFC after: 1 month Notes: svn path=/head/; revision=181775
* - Add cpuctl(4) pseudo-device driver to provide access to some low-levelStanislav Sedov2008-08-081-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | features of CPUs like reading/writing machine-specific registers, retrieving cpuid data, and updating microcode. - Add cpucontrol(8) utility, that provides userland access to the features of cpuctl(4). - Add subsequent manpages. The cpuctl(4) device operates as follows. The pseudo-device node cpuctlX is created for each cpu present in the systems. The pseudo-device minor number corresponds to the cpu number in the system. The cpuctl(4) pseudo- device allows a number of ioctl to be preformed, namely RDMSR/WRMSR/CPUID and UPDATE. The first pair alows the caller to read/write machine-specific registers from the correspondent CPU. cpuid data could be retrieved using the CPUID call, and microcode updates are applied via UPDATE. The permissions are inforced based on the pseudo-device file permissions. RDMSR/CPUID will be allowed when the caller has read access to the device node, while WRMSR/UPDATE will be granted only when the node is opened for writing. There're also a number of priv(9) checks. The cpucontrol(8) utility is intened to provide userland access to the cpuctl(4) device features. The utility also allows one to apply cpu microcode updates. Currently only Intel and AMD cpus are supported and were tested. Approved by: kib Reviewed by: rpaulo, cokane, Peter Jeremy MFC after: 1 month Notes: svn path=/head/; revision=181430
* - Add inlines for the monitor and mwait instructions.Jeff Roberson2008-04-181-0/+13
| | | | | | | Sponsored by: Nokia Notes: svn path=/head/; revision=178299
* Add "show sysregs" command to ddb. On i386, this gives gdt, idt, ldt,Nate Lawson2007-08-091-11/+47
| | | | | | | | | | | cr0-4, etc. Support should be added for other platforms that have a different set of registers for system use. Loosely based on: OpenBSD Approved by: re Notes: svn path=/head/; revision=171797
* Add a knob for disabling/enabling HTT, "machdep.hyperthreading_allowed".Jacques Vidrine2005-05-131-0/+8
| | | | | | | | | | Default off due to information disclosure on multi-user systems. Submitted by: cperciva Reviewed by: jhb Notes: svn path=/head/; revision=146170
* netchild's mega-patch to isolate compiler dependencies into a centralJoerg Wunsch2005-03-021-6/+10
| | | | | | | | | | | | | | | | | | | place. This moves the dependency on GCC's and other compiler's features into the central sys/cdefs.h file, while the individual source files can then refer to #ifdef __COMPILER_FEATURE_FOO where they by now used to refer to #if __GNUC__ > 3.1415 && __BARC__ <= 42. By now, GCC and ICC (the Intel compiler) have been actively tested on IA32 platforms by netchild. Extension to other compilers is supposed to be possible, of course. Submitted by: netchild Reviewed by: various developers on arch@, some time ago Notes: svn path=/head/; revision=143063
* Remove advertising clause from University of California Regent'sWarner Losh2004-04-071-4/+0
| | | | | | | | | | license, per letter dated July 22, 1999 and email from Peter Wemm, Alan Cox and Robert Watson. Approved by: core, peter, alc, rwatson Notes: svn path=/head/; revision=128019
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-20 03:10:25 +0000