| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Define _ALIGNBYTES using sizeof(void *) (no functional change on any
existing architecture) which will allow it to work with CHERI were we
must align things up to capability alignment.
In _ALIGN, replace integer manipulation which does not preserve pointer
provenance with a type and provenance preserving builtin. This requires
modest changes in code which assumes _ALIGN returns an integer, but
those are relatively rare.
Reviewed by: kib, markj
Effort: CHERI upstreaming
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D53947
|
| |
|
|
|
|
|
|
|
| |
The wrapper functions such as bus_alloc_resource_any() still support
passing the rid by value or pointer, but the underlying implementation
now passes by value.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D53402
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Due to an implementation bug, the GICv3 and ITS master ports may be connected
to a non-coherent bus. The new DT resolves this issue by marking these ports
with the "dma-nocoherent" attribute. The older DT does not have this attribute,
so we must match the affected SoC.
Additionally, the RK356x family has GIC master ports on a 32-bit bus.
Therefore, we must limit the address range for all tables and command buffers
allocated for it. In this case, the DT does not have an attribute for this
case, so the quirk should only be applied by SoC matching.
MFC after: 4 weeks
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D47488
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The platform-dependent macros are added to the various
_stdint.h headers, those that are always the same are
added directly to _stdint.h.
We may want to move the definitions for WCHAR_* and
WINT_* out of the platform header files as those are
always the same.
Approved by: markj (mentor)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D53830
|
| |
|
|
|
|
|
|
|
|
|
|
| |
For compliance with IOS/IEC 9899:2024 ("C23").
These macros define the width in bits of the basic integer types.
Another new macro, BITINT_MAXWIDTH, is not yet included as I do not
understand what it should be set to. Perhaps it is compiler-specific.
Approved by: markj (mentor)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D53825
|
| |
|
|
|
|
|
|
|
|
|
| |
Realtek changed how it styled its name 25 or so years ago, but the old
style persisted in many places. These products use the new styling in
their datasheets.
Signed-off-by: ykla yklaxds@gmail.com
Sponsored by: Chinese FreeBSD Community
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1901
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
It's not clear what the race described in the commit actually is, nor
how it could arise, but this commit is definitely wrong; curthread is no
longer set for intr_pic_init_secondary, and gic_v3's pic_init_secondary
uses mutex(9) in some places, which requires curthread, so it has led to
panics. Revert this change until the original issue this was intended to
be fixed can be more thorougly investigated and a better fix made.
Reported by: Herbert J. Skuhra <herbert@gojira.at>, jhb
This reverts commit a695ac2ce8bc8e8b989359002659063f2e056dcf.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This may have been called after intr_irq_shuffle. For most interrupt
controllers this appears to be safe, however for the GICv5 we need to
read a per-CPU ID register before we can assign interrupts to a given
CPU.
Fix the race by moving intr_pic_init_secondary earlier in the boot,
after devices have been enumerated and before the interrupts are moved
to their assigned CPUs.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D53685
|
| |
|
|
|
|
|
| |
It's no longer used after moving to gic_get_support_lpis.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D53665
|
| |
|
|
|
|
|
| |
The old gicv3_get_support_lpis will be removed.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D53664
|
| |
|
|
|
|
|
| |
GICv5 will need this too, so move to the GIC_IVAR namespace.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D53663
|
| |
|
|
|
|
|
| |
Add the arm64 parts to support for non-PCI MSI and MSI-X interrupts.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D53331
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add 64 bit support to vchiq:
* update fields to the appropriate fixed bit-size variants (everywhere [cf. e.g., ref:sizes and ref:sizes2])
* refer to event semaphores (that go into the very 32 bit VC) by offset instead of pointers [ref:sems]
* dsb() is dsb(sy) in arm64 (vchiq_{core.c,core.h,kmod.c}) [ref:dsb]
* comment out some unneeded code in parse_rx_slots around VCHIQ_MSG_BULK_RX (cf. [ref:deadcode])
* adapt remote_event_signal to arm64 caching behaviours (vchiq_kmod.c)
* refactor synchronization around remote_event_signal, forcing a wmb to be on the safe side; thereby make it look more like what linux does [ref:sync] (vchiq_{core,kmod}.c); and make a comment in vchiq_core.c true (wasn't before)
* add a few more syncs to be on the safe side (vchiq_2835_arm.c)
* use arm64 dcache invalidation mechanisms (vchiq_2835_arm.c)
* explicitly invalidate pages on arm64 post bulk-read (vchiq_2835_arm.c)
* support bulk transfers on rpi-4 (aka "long address space" transfers), by hard-coding their vc offset (0) and different bit-shift [ref:longbulk] (vchiq_2835_arm.c)
* refactor a loop-of-constant-test (vchiq_2835_arm.c)
* use the correct (hard-coded) cache-line size on arm64
* rework the handling of chipset "features" to account for the extra behaviours with 64 bit chipsets. (vchiq_kmod.c)
* add sysctl-s (log, arm_log) to control debug (vchiq_kmod.c)
* add example kernel config (GENERIC-VCHIQ)
Fixes:
* Rework error handling in create_pagelist, avoiding a potential panic when
freeing memory that had been dmamem_alloc, a potential null dereference,
and a leak when having problems pinning pages (vchiq_2835_arm.c)
* fix a confusion about the behaviour cv_wait_sig that lead to
uninterruptible looping (vchiq_bsd.c)
* implement detection of fatal signals (vchiq_bsd.c)
* fix a confusion with the name of a variable introduced by #a0b8746
that could lead to a panic when closing the cdev file (vchiq_arm.c)
* release user connection when destructing cdevpriv and avoid user processes
sharing connection data, which lead to stalls and data corruption. (vchiq_arm.c)
Update bcm2835_audio to work on 64bit systems:
* update VC audio fields (vc_vchi_audioserv_defs.h, bcm2835_audio.c)
* repurpose the hitherto unused callback field to help push a 64 bit pointer in (bcm2835_audio.c)
* increase (hopefully) the robustness of the code that shifts data to VC (bcm2835_audio.c)
* add a sysctl to control the amount of debugging info output by bcm2835_audio.c
Tested on zero, zero2 and 4+ with ping, functional, bulk and control vchiq_test-s, and omxplayer
[ref:dsb]: https://github.com/raspberrypi/linux/commit/35b7ebda57affcfd3616d39d5a727a4495b31123
[ref:sems]: https://github.com/raspberrypi/linux/commit/24a4262afb10907fce3cdbc3ae336fcf4cdaece5
[ref:sizes]: https://github.com/raspberrypi/linux/commit/e64568b8ea6c04e747e432c17ce2452652075216
[ref:sizes2]: https://github.com/raspberrypi/linux/commit/f9bee6dd24addfa00c2c8d50c25b73efbfbb28ba
[ref:deadcode]: https://github.com/raspberrypi/linux/commit/14f4d72fb799a9b3170a45ab80d4a3ddad541960
[ref:sync]: https://github.com/raspberrypi/linux/commit/51c071265079319583e4c6e8c61e09660300d0bf
[ref:longbulk]: https://github.com/raspberrypi/linux/commit/37f6f19a83722c9b866cecb5e455b2e16e5bbc6b
Differential Revision: https://reviews.freebsd.org/D37878
Submitted by: Marco Devesas Campos <devesas.campos@gmail.com>
|
| |
|
|
|
|
|
|
|
|
| |
Cast &end to vm_offset_t before subtracting VM_MIN_KERNEL_ADDRESS to
ensure the resulting size is correct for PRELOAD_PUSH_VALUE.
Previously the correct size was effectively divided by sizeof(void *).
Reviewed by: andrew
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D53699
|
| |
|
|
| |
Fixes: 68f185ccc9f8 ("arm64: Add Arm SPE support")
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add support for the Arm Statistical Profiling Extension (SPE). This
is an optional extension added in Armv8.1 to provide profiling of
software using randomised instruction sampling.
This adds the initial driver, and attached it to the hardware tracing
framework, hwt(4), in CPU mode to allow tracing of all threads on
a given CPU.
(commit message by andrew@)
Co-authored-by: Sarah Walker <sarah.walker2@arm.com>
Co-authored-by: Andrew Turner <andrew@FreeBSD.org>
Reviewed by: andrew
Sponsored by: Arm Ltd
Sponsored by: The FreeBSD Foundation (early driver)
Differential Revision: https://reviews.freebsd.org/D46241
|
| |
|
|
|
|
|
|
|
| |
This will be used by the SPE driver.
(commit message by andrew@)
Reviewed by: andrew
Sponsored by: Arm Ltd
|
| |
|
|
|
|
|
|
|
| |
This will be used by the SPE driver.
(commit message by andrew@)
Reviewed by: andrew
Sponsored by: Arm Ltd
|
| |
|
|
|
|
|
|
|
|
|
|
| |
TDE (make EL2 the target EL for debug exceptions) is set both when
setting guest breakpoints and when single-stepping the guest. In some
cases we may configure both capabilities, and when subsequently
disabling one of them we need to take care to avoid clearing TDE if the
other is still configured.
MFC after: 3 days
Fixes: 75cb949228bb ("arm64/vmm: Add breakpoint and single-stepping support")
Sponsored by: CHERI Research Centre (EPSRC grant UKRI3001)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When VHE mode is enabled, this results in a hang on the host. In
particular, when MDSCR_EL2.KDE is set to 1 and the CPU is executing at
EL_D, i.e., EL2, debug exceptions are enabled. In non-VHE mode, we call
into the guest by trapping to EL2, which implicitly masks debug
exceptions by setting PSTATE.D. However, in VHE mode, PSTATE.D remains
clear, so when the guest's MDSCR_EL1 value is loaded, we immediately
begin single-stepping.
In non-VHE mode there is no need to set KDE either, so just stop setting
it.
Reviewed by: andrew
MFC after: 3 days
Sponsored by: CHERI Research Centre (EPSRC grant UKRI3001)
Differential Revision: https://reviews.freebsd.org/D48965
|
| |
|
|
|
|
|
|
| |
No functional change intended.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D53528
|
| |
|
|
|
|
| |
Reviewed by: jmg
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D53390
|
| |
|
|
|
|
|
|
|
|
|
| |
* Enable RANDOM_ENABLE_TPM by default
* The commit of TPM_HARVEST failed to add it to NOTES
so that the LINT kernel would build the code.
Fixes: 4ee7d3b0118c82e651712bb65da53d08e78cd7b1
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D53460
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
vm_alloc_vcpu() is called quite frequently, and we don't need to apply
the vgic limit unless we're actually allocating a vcpu structure for the
first time.
No functional change intended.
Reviewed by: andrew
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53580
|
| |
|
|
|
|
|
|
|
|
| |
No functional change intended.
Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53477
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
vm_create() is only called from one place. Rather than having similar
checks everywhere, move them to vmmdev_create().
We can safely assume that the name is nul-terminated, the vmmctl ioctl
handler and the legacy sysctl handler ensure this. So, don't bother
with strnlen().
Finally, make sure that the name buffers are the same size on all
platforms. VM_MAX_NAMELEN is supposed to be the maximum, not including
the nul terminator.
Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53422
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Move the vmm_initialized check out of vm_create() and into the legacy
sysctl handler. If vmm_initialized is false, /dev/vmmctl will not be
available and so cannot be used to create VMs.
Introduce new MD vmm_modinit() and vmm_modcleanup() routines which
handle MD (de)initialization.
No functional change intended.
Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53421
|
| |
|
|
|
|
| |
- s/volatge/voltage/
MFC after: 5 days
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This works on older arm64 platforms, but may not work with arm64 devices
using GICv3, due to a quirk in the GICv3, where some registers are
write-once.
Most of the kexec reboot work on arm64 can be done entirely in C code,
by disabling the MMU, as the kernel is carved out of the vm_phys_segs
array, so cannot be overwritten.
Reviewed by: andrew
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D51621
|
| |
|
|
|
|
|
|
|
| |
We only needed armreg.h for the *_SPECIALREG macros. Use the new
_armreg.h directly.
Reviewed by: emaste
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D53325
|
| |
|
|
|
|
|
|
|
|
| |
We shouldn't need to include armreg.h just to access special registers
that are not defined in this file. Split out the parts that should be
common with arm64.h and hypervisor.h.
Reviewed by: emaste
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D53324
|
| |
|
|
|
|
|
|
| |
There is nothing in this file that needs armreg.h
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D53323
|
| |
|
|
|
|
|
|
|
| |
It is needed for INSN_SIZE. Include it directly rather than depending
on header pollution.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D53322
|
| |
|
|
|
|
| |
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D53321
|
| |
|
|
|
| |
Reviewed by: andrew
Sponsored by: Arm Ltd
|
| |
|
|
|
|
|
|
|
| |
fvp-base-revc.dtb works with the kernel now interrupt-maps are
supported in more cases.
Reviewed by: mhorne
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D51258
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It seems like _QUAD_LOWWORD was incorrectly expanded into 1,
which is correct for big endian but not little endian. This
means we always grab the padding word for the syscall number,
which is usually 0, causing SIGSYS to be delivered to the caller.
Reintroduce _QUAD_LOWWORD to fix the syscall.
PR: 290411
MFC after: 1 week
Discussed with: jrtc27
Reviewed by: cognet, emaste
Approved by: markj (mentor)
Fixes: 8c9c3144ccfa3061879b8cec015ee7d1010e4766
Differential Revision: https://reviews.freebsd.org/D53250
|
| |
|
|
|
|
|
|
|
|
|
| |
Make the ioctl handlers easy to read by moving local variables into
per-ioctl blocks. No functional change intended.
Reviewed by: corvink, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53145
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On non-amd64 platforms, check for negative register indices. This isn't
required today since we match against individual register indices, but
we might as well check it. On amd64, add a comment explaining why we
permit negative register indices.
Use mallocarray() for allocating register arrays in the ioctl layer.
No functional change intended.
Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53143
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
vm_smp_rendezvous() invokes a callback on all vCPUs, blocking the
initiator until all vCPUs have responded. vcpu_lock_all() blocks each
vCPU by waiting for it to go idle and setting the vCPU state to frozen.
These two operations can deadlock on each other, particularly when
booting a Windows guest, when vcpu_lock_all() blocks waiting for a
rendezvous initiator, and the initiator is blocked waiting for the vCPU
thread which called vcpu_lock_all() to invoke the rendezvous callback.
Implement vcpu_lock_all() in a way that avoids deadlocks with
vm_smp_rendezvous(). In particular, when traversing vCPUs, invoke the
rendezvous callback on the vCPU's behalf to help the initiator finish.
We can only safely do so when the vCPU is IDLE or we have already locked
it, otherwise we may be racing with the target vCPU thread. Thus:
- Use an exclusive lock to serialize vcpu_lock_all() callers, which lets
us lock vCPUs out of order without fear of deadlock with parallel
vcpu_lock_all() callers.
- If a rendezvous is pending, lock all idle vCPUs and invoke the
callback on their behalf. If the vcpu_lock_all() caller is itself a
vCPU thread, this will handle that thread.
- Block waiting for all non-idle vCPUs to idle, or until one of them
initiates a rendezvous, in which case we go back and invoke callbacks
on behalf of already-locked vCPUs.
Note that on !amd64 no changes are needed since there is no rendezvous
mechanism, so there is a separate vcpu_set_state_all() for them based on
the previous vcpu_lock_all(). These will be merged together once vcpu
state handling is consolidated into sys/dev/vmm.
Reviewed by: corvink (previous version)
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D52968
|
| |
|
|
|
|
|
|
|
| |
Port the BP hardening and SSBD workaround from cpu_quirks to the
CPU_FEAT framework.
Reviewed by: andrew
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D53112
|
| |
|
|
|
|
|
|
| |
No functional change intended.
Reviewed by: olce, kib, emaste
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D53062
|
| |
|
|
| |
MFC after: 1 week
|
| |
|
|
|
|
|
|
|
|
| |
No functional change intended.
Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53013
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This further consolidates handling of guest memory into MI code in
sys/dev/vmm.
No functional change intended.
Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53012
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- On amd64, make vmmops_* functions globally visible, as some will be
called from machine-independent code in the future.
- On arm64 and riscv, move declarations to vmm.h, since they're supposed
to be generic across different VMM backends (only amd64 has more than
one backend).
- Make the declaration macros consistent with each other.
- On amd64, make the function typedef names consistent with the
corresponding ifunc names.
No functional change intended.
Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53011
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Different vmm implementations were freeing the per-vCPU structure in
different places. Make the implementations consistent. No functional
change intended.
Reviewed by: corvink
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D53010
|
| |
|
|
|
|
|
|
|
| |
This is enough to boot FreeBSD again.
PR: 290044
Reported by: Siva Mahadevan <me@svmhdvn.name>
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D52931
|
| |
|
|
|
| |
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D52930
|
| |
|
|
|
|
|
|
| |
These are features we don't support or advertise to the guest so can
safely be trapped.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D52806
|