src/stand/lua, branch releng/12.2 FreeBSD source tree MFS12: r366422 r366588 2020-10-14T01:47:00+00:00 Warner Losh imp@FreeBSD.org 2020-10-14T01:47:00+00:00 8e83334bb67d06323f95546a9bad2fdb939659e5 r366588: fixes video display heuristic that prevented byhve and vmware from detecting dual consoles. r366422: Report the kernel console on the boot screen Report what console the boot loader is telling the kernel to use and allow toggling between them. Approved by: re@ (gjb) 
r366588: fixes video display heuristic that prevented byhve and vmware
from detecting dual consoles.

r366422: Report the kernel console on the boot screen
Report what console the boot loader is telling the kernel to use and allow
toggling between them.

Approved by:	re@ (gjb)
MFC r361709: lualoader: improve drawer error handling 2020-06-05T02:52:07+00:00 Kyle Evans kevans@FreeBSD.org 2020-06-05T02:52:07+00:00 89dbcf05a08d3939c37842437656aa266d1c2221 At least one user has landed in a scenario where logo files appear to be misnamed, and we failed to find them. Our fallback for missing logodefs is orb/orbbw, based on the color status. In a scenario where we can't locate the logos, though, this is not ideal. Add in one more layer of fallback to properly just don't draw any logo if the fan has been jam packed with foreign material. PR: 246046 
At least one user has landed in a scenario where logo files appear to be
misnamed, and we failed to find them. Our fallback for missing logodefs is
orb/orbbw, based on the color status. In a scenario where we can't locate
the logos, though, this is not ideal. Add in one more layer of fallback
to properly just don't draw any logo if the fan has been jam packed with
foreign material.

PR:		246046
MFC lualoader read-conf support: r360420-r360423, r360425, r360427, r360486, 2020-05-03T03:53:38+00:00 Kyle Evans kevans@FreeBSD.org 2020-05-03T03:53:38+00:00 22d90f6746aa6c320949069ebc5ea1ca5aff6bb9 \r360505-r360506 r360420: lualoader config: don't call loader.getenv() as much We don't actually need to fetch loader_conf_files as much as we do; we've already fetched it once at the beginning, we only really need to fetch it again after each file we've processed. If it changes, then we can stash that off into our local prefiles. While here, drop a note about the recursion so that I stop trying to change it. It may very well make redundant some of the work we're doing, but that's OK. r360421: lualoader: config: start exporting readConfFiles In the process, change it slightly: readConfFiles will take a string like loader_conf_files in addition to the loaded_files table that it normally takes. This is to facilitate the addition of a read-conf CLI command, which will just pass in the single file to read and an empty table. r360422: lualoader: cli: add read-conf This is a straightforward match to the command used by many in forthloader; it uses the newly-exported config.readConfFiles() to make sure that any loader_conf_files gets done as appropriate. r360423: lualoader: cli: clobber loader_conf_files before proceeding This makes sure that config.readConfFiles doesn't see a stale loader_conf_files from before, in case the newly loaded file doesn't set it. r360425: config.lua(8): "may should" is not proper grammar r360427: config.lua(8): catch up to recently added hooks While we're here, let's stylize these as functions instead of just raw text. A future change may allow arbitrary data arguments to be passed some of these, and the distinction is useful. r360486: loader.conf(5): document that loader_conf_files may be clobbered A future change in lualoader may take some liberties with the loader_conf_files in the name of efficiency; namely, it may start omitting it from the loader environment entirely so that it doesn't need to worry about maintaining any specific value. This variable has historically been incredibly volatile anyways, as it may get set to completely different values in any given configuration file to trigger a load of more files. Document now that we may not maintain it in the future, but perhaps we'll reserve the right to change our minds and eventually formally export all of the loader configuration files that were read using this variable. r360505: lualoader: config: add a table for restricted environment vars This new table should be used for transient values that don't need to end up in the loader environment. Generally, these will be things that are internal details that really aren't needed or interesting outside of the config module (e.g. if we changed how ${module}_* directives work, they might use this instead). To start, populate it with loader_conf_files. Any specific value of loader_conf_files isn't all that interesting; if we're going to export it, we should really instead export a loader_conf_files that indicates all of the configuration files we processed. This will be used to reduce bookkeeping overhead in a future commit that cleans up readConfFiles. r360506: lualoader: config: improve readConfFiles, rename to readConf The previous interface was pretty bad, and required the caller to get some implementation details correct that it really shouldn't need to (e.g. loader_conf_files handling) and pass in an empty table for it to use. The new and much improved interface, readConf, is much less of a hack; hiding these implementation details and just doing the right thing. config.lua will now use it to process /boot/defaults/loader.conf and the subsequent loader_conf_files from there, and read-conf will also use it. This improvement submitted by Olivier (cited below), loader_conf_files handling from the original patch was changed to just clobber it before processing and not bother restoring it after the fact following r360505 where it's now guaranteed to evade the loader environment. PR: 244640 
\r360505-r360506

r360420:
lualoader config: don't call loader.getenv() as much

We don't actually need to fetch loader_conf_files as much as we do; we've
already fetched it once at the beginning, we only really need to fetch it
again after each file we've processed. If it changes, then we can stash that
off into our local prefiles.

While here, drop a note about the recursion so that I stop trying to
change it. It may very well make redundant some of the work we're doing, but
that's OK.

r360421:
lualoader: config: start exporting readConfFiles

In the process, change it slightly: readConfFiles will take a string like
loader_conf_files in addition to the loaded_files table that it normally
takes. This is to facilitate the addition of a read-conf CLI command, which
will just pass in the single file to read and an empty table.

r360422:
lualoader: cli: add read-conf

This is a straightforward match to the command used by many in forthloader;
it uses the newly-exported config.readConfFiles() to make sure that any
loader_conf_files gets done as appropriate.

r360423:
lualoader: cli: clobber loader_conf_files before proceeding

This makes sure that config.readConfFiles doesn't see a stale
loader_conf_files from before, in case the newly loaded file doesn't set it.

r360425:
config.lua(8): "may should" is not proper grammar

r360427:
config.lua(8): catch up to recently added hooks

While we're here, let's stylize these as functions instead of just raw text.
A future change may allow arbitrary data arguments to be passed some of
these, and the distinction is useful.

r360486:
loader.conf(5): document that loader_conf_files may be clobbered

A future change in lualoader may take some liberties with the
loader_conf_files in the name of efficiency; namely, it may start omitting
it from the loader environment entirely so that it doesn't need to worry
about maintaining any specific value.

This variable has historically been incredibly volatile anyways, as it may
get set to completely different values in any given configuration file to
trigger a load of more files.

Document now that we may not maintain it in the future, but perhaps we'll
reserve the right to change our minds and eventually formally export all of
the loader configuration files that were read using this variable.

r360505:
lualoader: config: add a table for restricted environment vars

This new table should be used for transient values that don't need to end up
in the loader environment. Generally, these will be things that are internal
details that really aren't needed or interesting outside of the config
module (e.g. if we changed how ${module}_* directives work, they might use
this instead).

To start, populate it with loader_conf_files. Any specific value of
loader_conf_files isn't all that interesting; if we're going to export it,
we should really instead export a loader_conf_files that indicates all of
the configuration files we processed. This will be used to reduce
bookkeeping overhead in a future commit that cleans up readConfFiles.

r360506:
lualoader: config: improve readConfFiles, rename to readConf

The previous interface was pretty bad, and required the caller to get some
implementation details correct that it really shouldn't need to (e.g.
loader_conf_files handling) and pass in an empty table for it to use.

The new and much improved interface, readConf, is much less of a hack;
hiding these implementation details and just doing the right thing.
config.lua will now use it to process /boot/defaults/loader.conf and the
subsequent loader_conf_files from there, and read-conf will also use it.

This improvement submitted by Olivier (cited below), loader_conf_files
handling from the original patch was changed to just clobber it before
processing and not bother restoring it after the fact following r360505
where it's now guaranteed to evade the loader environment.

PR:		244640
MFC r360199 2020-04-29T05:11:17+00:00 Ryan Moeller freqlabs@FreeBSD.org 2020-04-29T05:11:17+00:00 73f266fa30111c3666aee70c9cb76c101df308a5 menu.lua: Give names to menu entries Make menu customizations easier by naming the entries and using the names to build the table entries. Reviewed by: kevans Approved by: mav (mentor) Sponsored by: iXsystems, Inc. Differential Revision: https://reviews.freebsd.org/D24527 
menu.lua: Give names to menu entries

Make menu customizations easier by naming the entries and using the
names to build the table entries.

Reviewed by:    kevans
Approved by:    mav (mentor)
Sponsored by:   iXsystems, Inc.
Differential Revision:  https://reviews.freebsd.org/D24527
MFC r359371 2020-04-03T01:31:48+00:00 Ryan Moeller freqlabs@FreeBSD.org 2020-04-03T01:31:48+00:00 9a42512e8a0262c9e226e06cd6924fa2f0394bbf loader: Fully reset terminal settings, not just colors Reviewed by: kevans, tsoome Approved by: mav (mentor) 
loader: Fully reset terminal settings, not just colors

Reviewed by:    kevans, tsoome
Approved by:    mav (mentor)
MFC r359072-r359073 2020-03-25T16:24:28+00:00 Ryan Moeller freqlabs@FreeBSD.org 2020-03-25T16:24:28+00:00 fbd05f81506d527ebaee8d8db6b58779d1773b29 loader: Add a "kernel.loaded" hook This hook can be useful, for example to run a local function to choose different modules to load when a user has picked a different kernel from the menu. Reviewed by: kevans Approved by: mav (mentor) Sponsored by: iXsystems, Inc. Differential Revision: https://reviews.freebsd.org/D24115 
loader: Add a "kernel.loaded" hook

This hook can be useful, for example to run a local function to choose
different modules to load when a user has picked a different kernel
from the menu.

Reviewed by:    kevans
Approved by:    mav (mentor)
Sponsored by:   iXsystems, Inc.
Differential Revision:  https://reviews.freebsd.org/D24115
MFC r357103-r357104: unbreak local.lua, add a modules.loaded hook 2020-01-28T02:42:33+00:00 Kyle Evans kevans@FreeBSD.org 2020-01-28T02:42:33+00:00 ff9e3f3b9667ec418c7ef27c27328d2f732fb2bb r357103: loader.lua: re-arrange to load local.lua *after* config loading The major problem with the current ordering is that loader.conf may contain all of the magic we need to actually setup the console, so loading local.lua prior to that can make it excessively difficult and annoying to debug (whoops, sorry Ravi & Warner). The new ordering has some implications, but I suspect they are a non-issue. The first is that it's no longer possible for the local module to inject any logic prior to loading config -- I suspect no one has relied on this. The second implication is that the config.loaded hook is now useless, as the local module will always be included after that hook would have fired. For config.loaded, I will opt to leave it in, just in case we add an early point for local lua to get injected or in case one wants to schedule some deferred logic in a custom loader.lua. The overhead of having it if no hooks will be invoked is relatively minimal. r357104: lua: add modules.loaded hook This may be used for the local module to hook in and load any additional modules that it wants, since it can't modify the modules table internal to config. We may consider adding API to do so at a later time, but I suspect it will be more complicated to use with little return. status is captured but ignored for the purpose of loading the hook. status will be false if *any* module failed to load, but we typically don't let that halt the boot so there's no reason to let it halt hooks. Some vendors or setups may have expected fails that would be actively thwarted by checking it. We may, at a later date, consider adding an API for letting non-config modules check which modules have successfully (or not) loaded in case an unexpected failure *should* halt whatever they are doing. 
r357103:
loader.lua: re-arrange to load local.lua *after* config loading

The major problem with the current ordering is that loader.conf may contain
all of the magic we need to actually setup the console, so loading local.lua
prior to that can make it excessively difficult and annoying to debug
(whoops, sorry Ravi & Warner).

The new ordering has some implications, but I suspect they are a non-issue.
The first is that it's no longer possible for the local module to inject any
logic prior to loading config -- I suspect no one has relied on this. The
second implication is that the config.loaded hook is now useless, as the
local module will always be included after that hook would have fired.

For config.loaded, I will opt to leave it in, just in case we add an early
point for local lua to get injected or in case one wants to schedule some
deferred logic in a custom loader.lua. The overhead of having it if no hooks
will be invoked is relatively minimal.

r357104:
lua: add modules.loaded hook

This may be used for the local module to hook in and load any additional
modules that it wants, since it can't modify the modules table internal to
config. We may consider adding API to do so at a later time, but I suspect
it will be more complicated to use with little return.

status is captured but ignored for the purpose of loading the hook. status
will be false if *any* module failed to load, but we typically don't let
that halt the boot so there's no reason to let it halt hooks. Some vendors
or setups may have expected fails that would be actively thwarted by
checking it.

We may, at a later date, consider adding an API for letting non-config
modules check which modules have successfully (or not) loaded in case an
unexpected failure *should* halt whatever they are doing.
MFC r354247, r355349: lualoader try_include improvement 2019-12-12T18:51:32+00:00 Kyle Evans kevans@FreeBSD.org 2019-12-12T18:51:32+00:00 b0857bd3eca7656962193fc093bb0f6537e107b1 r354247: lualoader: rewrite try_include using lfs + dofile Actual modules get require()'d in, rather than try_include(). All instances of try_include should be provided with proper hooks/API in the rest of loader to do the work they need to do, since we can't rely on them to exist. Convert this now to lfs + dofile since we won't really be treating them as modules. lfs is required because dofile will properly throw an error if the file doesn't exist, which is not in the spirit of 'optionally included'. Getting out of the pcall game allows us to provide a loader.exit() style call that backs out to the common bits of loader (autoboot sequence unless disabled with a loader.setenv("autoboot_delay", "NO")). The most ideal way identified so far to implement loader.exit() is to throw a special abort-style error that indicates to the caller in interp_lua that we've not actually errored out, just continue execution. Otherwise, we have to hack in logic to bubble up and return from loader.lua without continuing further, which gets kind of ugly depending on the context in which we're aborting. A compat shim is provided temporarily in case the executing loader doesn't yet have loader.lua_path, which was just added in r354246. r355349: lualoader: correct a typo from r354247 r354247 converted try_include to lfs + dofile with the loader.lua_path added just before. Fortunately, there was a hardcoded /boot/lua fallback in case loader.lua_path wasn't being set yet- I typo'd it as loader.lua_paths. Fix the typo. 
r354247: lualoader: rewrite try_include using lfs + dofile

Actual modules get require()'d in, rather than try_include(). All instances
of try_include should be provided with proper hooks/API in the rest of
loader to do the work they need to do, since we can't rely on them to exist.
Convert this now to lfs + dofile since we won't really be treating them as
modules.

lfs is required because dofile will properly throw an error if the file
doesn't exist, which is not in the spirit of 'optionally included'.

Getting out of the pcall game allows us to provide a loader.exit() style
call that backs out to the common bits of loader (autoboot sequence unless
disabled with a loader.setenv("autoboot_delay", "NO")). The most ideal way
identified so far to implement loader.exit() is to throw a special
abort-style error that indicates to the caller in interp_lua that we've not
actually errored out, just continue execution. Otherwise, we have to hack in
logic to bubble up and return from loader.lua without continuing further,
which gets kind of ugly depending on the context in which we're aborting.

A compat shim is provided temporarily in case the executing loader doesn't
yet have loader.lua_path, which was just added in r354246.

r355349: lualoader: correct a typo from r354247

r354247 converted try_include to lfs + dofile with the loader.lua_path added
just before. Fortunately, there was a hardcoded /boot/lua fallback in case
loader.lua_path wasn't being set yet- I typo'd it as loader.lua_paths.

Fix the typo.
MFC r354245, r354833, r354837: add flua to the base system 2019-12-03T18:25:16+00:00 Kyle Evans kevans@FreeBSD.org 2019-12-03T18:25:16+00:00 7825c2df5bf93ddc6d0aeb82825604b1a9cd59f2 r354245: stand: consolidate knowledge of lua path Multiple places coordinate to 'know' where lua scripts are installed. Knock this down to being formally defined (and overridable) in exactly one spot, defs.mk, and spread the knowledge to loaders and liblua alike. A future commit will expose this to lua as loader.lua_path, so it can build absolute paths to lua scripts as needed. r354833: Add flua to the base system, install to /usr/libexec FreeBSDlua ("flua") is a FreeBSD-private lua, flavored with whatever extensions we need for base system operations. We currently support a subset of lfs and lposix that are used in the rewrite of makesyscall.sh into lua, added in r354786. flua is intentionally written such that one can install standard lua and some set of lua modules from ports and achieve the same effect. linit_flua is a copy of linit.c from contrib/lua with lfs and lposix added in. This is similar to what we do in stand/. linit.c has been renamed to make it clear that this has flua-specific bits. luaconf has been slightly obfuscated to make extensions more difficult. Part of the problem is that flua is already hard enough to use as a bootstrap tool because it's not in PATH- attempting to do extension loading would require a special bootstrap version of flua with paths changed to protect the innocent. src.lua.mk has been added to make it easy for in-tree stuff to find flua, whether it's bootstrap-flua or relying on PATH frobbing by Makefile.inc1. r354837: flua: newer GCC complains about format-nonliteral at WARNS=2 Disable that one, too. 
r354245: stand: consolidate knowledge of lua path

Multiple places coordinate to 'know' where lua scripts are installed. Knock
this down to being formally defined (and overridable) in exactly one spot,
defs.mk, and spread the knowledge to loaders and liblua alike. A future
commit will expose this to lua as loader.lua_path, so it can build absolute
paths to lua scripts as needed.

r354833: Add flua to the base system, install to /usr/libexec

FreeBSDlua ("flua") is a FreeBSD-private lua, flavored with whatever
extensions we need for base system operations. We currently support a subset
of lfs and lposix that are used in the rewrite of makesyscall.sh into lua,
added in r354786.

flua is intentionally written such that one can install standard lua and
some set of lua modules from ports and achieve the same effect.

linit_flua is a copy of linit.c from contrib/lua with lfs and lposix added
in. This is similar to what we do in stand/. linit.c has been renamed to
make it clear that this has flua-specific bits.

luaconf has been slightly obfuscated to make extensions more difficult. Part
of the problem is that flua is already hard enough to use as a bootstrap
tool because it's not in PATH- attempting to do extension loading would
require a special bootstrap version of flua with paths changed to protect
the innocent.

src.lua.mk has been added to make it easy for in-tree stuff to find flua,
whether it's bootstrap-flua or relying on PATH frobbing by Makefile.inc1.

r354837: flua: newer GCC complains about format-nonliteral at WARNS=2

Disable that one, too.
MFC r353872-r353873: lualoader color handling fixes 2019-10-25T00:47:37+00:00 Kyle Evans kevans@FreeBSD.org 2019-10-25T00:47:37+00:00 7f647351455158790760129eb817342901352a37 r353872: lualoader: don't botch disabling of color When colors are disabled, color.escape{fg,bg} would return the passed in color rather than the proper ANSI sequence for the color. color.escape{fg,bg} would be wrong. Instead return '', as the associated reset* functions will also return ''. This should get rid of the funky '2' and '4' in the kernel selector if you're booting serial. r353873: lualoader: fix setting of loader_color=NO in loader.conf(5) Previously color.disabled would be calculated at color module load time, then never touched again. We can detect serial boots beyond just what we're told by loader.conf(5) so this works out in many cases, but we must re-evaluate the situation after the config is loaded to make sure we're not supposed to be forcing it enabled/disabled. Discovered while trying to test r353872. 
r353872:
lualoader: don't botch disabling of color

When colors are disabled, color.escape{fg,bg} would return the passed in
color rather than the proper ANSI sequence for the color.
color.escape{fg,bg} would be wrong.

Instead return '', as the associated reset* functions will also return ''.
This should get rid of the funky '2' and '4' in the kernel selector if
you're booting serial.

r353873:
lualoader: fix setting of loader_color=NO in loader.conf(5)

Previously color.disabled would be calculated at color module load time,
then never touched again. We can detect serial boots beyond just what we're
told by loader.conf(5) so this works out in many cases, but we must
re-evaluate the situation after the config is loaded to make sure we're not
supposed to be forcing it enabled/disabled.

Discovered while trying to test r353872.