path: root/documentation/content/en/books/fdp-primer/doc-build/_index.adoc

---
title: Chapter 5. The FreeBSD Documentation Build Process
prev: books/fdp-primer/structure
next: books/fdp-primer/asciidoctor-primer
description: Describes the FreeBSD Documentation Build Process
tags: ["build", "process", "make"]
showBookMenu: true
weight: 6
params:
  path: "/books/fdp-primer/doc-build/"
---

[[doc-build]]
= The FreeBSD Documentation Build Process
:doctype: book
:toc: macro
:toclevels: 1
:icons: font
:sectnums:
:sectnumlevels: 6
:sectnumoffset: 5
:partnums:
:source-highlighter: rouge
:experimental:
:images-path: books/fdp-primer/

ifdef::env-beastie[]
ifdef::backend-html5[]
:imagesdir: ../../../../images/{images-path}
endif::[]
ifndef::book[]
include::shared/authors.adoc[]
include::shared/mirrors.adoc[]
include::shared/releases.adoc[]
include::shared/attributes/attributes-{{% lang %}}.adoc[]
include::shared/{{% lang %}}/teams.adoc[]
include::shared/{{% lang %}}/mailing-lists.adoc[]
include::shared/{{% lang %}}/urls.adoc[]
toc::[]
endif::[]
ifdef::backend-pdf,backend-epub3[]
include::../../../../../shared/asciidoctor.adoc[]
endif::[]
endif::[]

ifndef::env-beastie[]
toc::[]
include::../../../../../shared/asciidoctor.adoc[]
endif::[]

This chapter covers organization of the documentation build process and how man:make[1] is used to control it.

[[doc-build-rendering]]
== Rendering AsciiDoc into Output

Different types of output can be produced from a single AsciiDoc source file.

[cols="20%,20%,60%", frame="none", options="header"]
|===
| Formats
| File Type
| Description

|`html`
|HTML
|An `article` or `book` chapter.

|`pdf`
|PDF
|Portable Document Format.

|`epub`
|EPUB
|Electronic Publication.
ePub file format.
|===

[[doc-build-rendering-html]]
=== Rendering to html

To render the documentation and the website to `html` use one of the following examples.

[[documentation-build-example]]
.Build the documentation
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make
....
====

[[website-build-example]]
.Build the website
[example]
====
[source,shell]
....
% cd ~/doc/website
% make
....
====

[[project-build-example]]
.Build the entire documentation project
[example]
====
[source,shell]
....
% cd ~/doc
% make -j2
....
====

Advanced build examples are given below:

[[documentation-build-example-verbose]]
.Build English and Spanish documentation with verbose and debug messages
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make DOC_LANG="en es" HUGO_ARGS="--verbose --debug"
....
====

[[documentation-build-example-server]]
.Build and serve the content with Hugo’s internal webserver
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make run
....
This webserver runs on `localhost`, port `1313` by default.

To serve the content with Hugo’s internal webserver binding a specific IP address and port:

[source,shell]
....
% make run BIND=192.168.15.10 HUGO_ARGS="-p 8080"
....

A `hostname` can also be set as base url to Hugo’s internal webserver:

[source,shell]
....
% make run BIND=192.168.15.10 HOSTNAME=example.com
....
====

[[documentation-build-example-offline]]
.Build documentation in html for offline usage
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make html
....

To compress the html output, add `DOC_HTML_ARCHIVE=1`:

[source,shell]
....
% cd ~/doc/documentation
% DOC_HTML_ARCHIVE=1 make html
....
====

[[doc-build-rendering-pdf]]
=== Rendering to pdf

To render the documentation to `pdf`, use one of the following examples.

[[document-pdf-example]]
.Build all documents in pdf
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make pdf
....
====

[[document-pdf-articles-example]]
.Build all articles in pdf
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make pdf-articles
....
====

[[document-pdf-books-example]]
.Build all books in pdf
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make pdf-books
....
====

[[document-pdf-language-example]]
.Build documents in pdf for specific languages
[example]
====
[source,shell]
....
% cd ~/doc/documentation
% make DOC_LANG="en" pdf
....
This will build all English documents in pdf.

[source,shell]
....
% cd ~/doc/documentation
% make DOC_LANG="en fr" pdf-books
....
This will build all English and French books in pdf.
====

[[doc-build-toolset]]
== The FreeBSD Documentation Build Toolset

These are the tools used to build and install the FDP documentation.

* The primary build tool is man:make[1], specifically Berkeley Make.
* Hugo
* AsciiDoctor
* Git

[[doc-build-makefile]]
== Understanding the Makefile in the Documentation Tree

There are three [.filename]#Makefile# files for building some or all of the documentation project.

* The [.filename]#Makefile# in the [.filename]#documentation# directory will build only the documentation.
* The [.filename]#Makefile# in the [.filename]#website# directory will build only the website.
* The [.filename]#Makefile# at the top of the tree will build both the documentation and the website.

The [.filename]#Makefile# appearing in subdirectories also support `make run` to serve built content with Hugo's internal webserver.
This webserver runs on port 1313 by default.

[[documentation-makefile]]
=== Documentation Makefile

This [.filename]#Makefile# takes the following form:

[source,shell]
....
# Generate the FreeBSD documentation
#
# Copyright (c) 2020-2025, The FreeBSD Documentation Project
# Copyright (c) 2020-2025, Sergio Carlavilla <carlavilla@FreeBSD.org>
#
# Targets intended for use on the command line
#
# all (default)	-	generate the books TOC and compile all the documentation
# clean		- 	removes generated files
# run		-	serves the built documentation site for local browsing
# pdf		-	build PDF versions of the articles and books.
# html		-	build HTML versions of the articles and books for
#			offline use.
#			If variable DOC_HTML_ARCHIVE is set, all documents will be
#			archived/compressed, and only these files will be kept in the public
#			directory.
# epub		-	build EPUB versions of the articles and books (Experimental).
#
# The run target uses hugo's built-in webserver to make the documentation site
# available for local browsing.  The documentation should have been built prior
# to attempting to use the `run` target.  By default, hugo will start its
# webserver on port 1313.

MAINTAINER=carlavilla@FreeBSD.org <.>

# List of languages without book translations
ARTICLEONLY_LANGS=	bn-bd da ko tr
# List of languages without article translations
BOOKONLY_LANGS=		mn

# List of all languages we have content for
ALL_LANGUAGES=	bn-bd da de el en es fr hu it ja ko mn nl pl pt-br ru tr zh-cn zh-tw <.>

LOCALBASE?=	/usr/local

RUBY_CMD =	${LOCALBASE}/bin/ruby <.>
HUGO_CMD =	${LOCALBASE}/bin/hugo <.>
HUGO_ARGS?=	--verbose --minify
HUGO_OFFLINE_ARGS?= 	--environment offline --verbose --minify
ASCIIDOCTOR_CMD=	${LOCALBASE}/bin/asciidoctor
ASCIIDOCTORPDF_CMD=	${LOCALBASE}/bin/asciidoctor-pdf

.if defined(DOC_LANG) && !empty(DOC_LANG)
LANGUAGES=	${DOC_LANG:S/,/ /g}
.if  ${LANGUAGES:Men} == "" && ${.TARGETS:Mpdf*} == "" && ${.TARGETS:Mhtml*} == ""
.warning "Warning: cannot skip 'en'; adding it back"
LANGUAGES+=	en
.endif
.else
LANGUAGES=	${ALL_LANGUAGES}
.endif

RUBYLIB =	../shared/lib
.export	RUBYLIB

RUN_DEPENDS=	${HUGO_CMD} \
		${LOCALBASE}/bin/asciidoctor \
		${LOCALBASE}/bin/rougify

.ifndef HOSTNAME
.  ifdef BIND
.HOST=$(BIND)
.  else
.HOST=localhost
.  endif
.else
.HOST=$(HOSTNAME)
.endif

# Strip the languages with only articles from the list of languages we
#  will use to build books.
BOOK_LANGS= ${LANGUAGES}
.for a in ${ARTICLEONLY_LANGS}
BOOK_LANGS:=	${BOOK_LANGS:N${a}}
.endfor

# Strip the languages with only books from the list of languages we
#  will use to build articles.
ARTICLE_LANGS= ${LANGUAGES}
.for a in ${BOOKONLY_LANGS}
ARTICLE_LANGS:=	${ARTICLE_LANGS:N${a}}
.endfor

# Take the list of all languages, and take out the ones we have been
#   asked for.  We'll feed this to hugo.
SKIP_LANGS=
.for a in ${ALL_LANGUAGES}
.if  ${LANGUAGES:M${a}} == ""
SKIP_LANGS+=    ${a}
.endif
.endfor

.ORDER: all run <.>

.ORDER: requirements <.>
.ORDER: starting-message
.ORDER: starting-message build
.ORDER: build

all: requirements starting-message generate-pgpkeys-txt build
run: requirements starting-message generate-pgpkeys-txt run-local

# clean does not call pdf-clean as that is a subset of hugo-clean
clean: hugo-clean pgp-clean

requirements:
.for dep in ${RUN_DEPENDS}
.if !exists(${dep})
	@(echo ${dep} not found, please run 'pkg install docproj'; exit 1)
.endif
.endfor

requirements-pdf:
.if !exists(${LOCALBASE}/bin/asciidoctor-pdf)
	@(echo ${LOCALBASE}/bin/asciidoctor-pdf not found, please run 'pkg install rubygem-asciidoctor-pdf'; exit 1)
.endif

requirements-epub:
.if !exists(${LOCALBASE}/bin/asciidoctor-epub3)
	@(echo ${LOCALBASE}/bin/asciidoctor-epub3 not found, please run 'pkg install rubygem-asciidoctor-epub3'; exit 1)
.endif

starting-message: .PHONY <.>
	@echo ---------------------------------------------------------------
	@echo                   Building the documentation
	@echo  included languages: ${LANGUAGES}
	@echo  excluded languages: ${SKIP_LANGS}
	@echo ---------------------------------------------------------------

generate-pgpkeys-txt: static/pgpkeys/pgpkeys.txt

static/pgpkeys/pgpkeys.txt: static/pgpkeys/*key
	${RUBY_CMD} ./tools/global-pgpkeys-creator.rb

run-local: .PHONY <.>
	HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} server \
		${HUGO_ARGS} -D $(BIND:D--bind=$(BIND)) --baseURL="http://$(.HOST):1313"

build: .PHONY <.>
	HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} ${HUGO_ARGS}

build-offline: .PHONY
	HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} ${HUGO_OFFLINE_ARGS}

pgp-clean: .PHONY
	rm -f static/pgpkeys/pgpkeys.txt

hugo-clean: .PHONY
	rm -rf resources public

#
# PDF targets
# Use DOC_LANG to choose the language, e.g., make DOC_LANG="en fr" pdf-books
#
pdf: pdf-articles pdf-books

pdf-books: requirements-pdf
.for _lang in ${BOOK_LANGS}
	./tools/asciidoctor.sh books ${_lang} pdf
.endfor

pdf-articles: requirements-pdf
.for _lang in ${ARTICLE_LANGS}
	./tools/asciidoctor.sh articles ${_lang} pdf
.endfor

pdf-clean: pdf-articles-clean pdf-books-clean

pdf-books-clean:
.for _lang in ${BOOK_LANGS}
	rm -fr ${.CURDIR}/public/${_lang}/books
	-rmdir ${.CURDIR}/public/${_lang}
.endfor
	-rmdir ${.CURDIR}/public/

pdf-articles-clean:
.for _lang in ${ARTICLE_LANGS}
	rm -fr ${.CURDIR}/public/${_lang}/articles
.if !exists(${.CURDIR}/public/${_lang}/books)
	rm -fr ${.CURDIR}/public/${_lang}
.endif
.endfor
	-rmdir ${.CURDIR}/public

#
# HTML targets
#
html: build-offline html-clean-global html-clean-articles html-clean-books html-archive html-archive-clean-files

html-clean: hugo-clean

html-clean-global:
	rm -fr ${.CURDIR}/public/index.html
	rm -rf pgpkeys js

html-clean-articles:
.for _lang in ${ARTICLE_LANGS}
	rm -fr ${.CURDIR}/public/${_lang}/index.html
	rm -fr ${.CURDIR}/public/${_lang}/articles/index.html
.endfor

html-clean-books:
.for _lang in ${BOOK_LANGS}
	rm -fr ${.CURDIR}/public/${_lang}/books/index.html
.endfor

html-archive:
.if defined(DOC_HTML_ARCHIVE)
.for _lang in ${ARTICLE_LANGS}
	./tools/asciidoctor.sh articles ${_lang} archive
.endfor
.for _lang in ${BOOK_LANGS}
	./tools/asciidoctor.sh books ${_lang} archive
.endfor
.endif

html-archive-clean-files:
.if defined(DOC_HTML_ARCHIVE)
	find ${.CURDIR}/public/ ! -name '*.pdf' ! -name '*.tar.gz' -type f -delete
	find ${.CURDIR}/public/ -type d -empty -delete
.endif

#
# EPUB targets
# Use DOC_LANG to choose the language, e.g., make DOC_LANG="en fr" epub-books
#
epub: epub-articles epub-books

epub-books: requirements-epub
	@echo ---------------------------------------------------------------
	@echo !!! EPUB output is experimental !!!
	@echo
	@echo Asciidoctor EPUB3 is currently alpha software. Use accordingly. Although the
	@echo bulk of AsciiDoc content is converted, there’s still work needed to fill in
	@echo gaps where conversion is incomplete or unstyled.
	@echo https://docs.asciidoctor.org/epub3-converter/latest/#project-status
	@echo ---------------------------------------------------------------
.for _lang in ${BOOK_LANGS}
	./tools/asciidoctor.sh books ${_lang} epub
.endfor

epub-articles: requirements-epub
	@echo ---------------------------------------------------------------
	@echo !!! EPUB output is experimental !!!
	@echo
	@echo Asciidoctor EPUB3 is currently alpha software. Use accordingly. Although the
	@echo bulk of AsciiDoc content is converted, there’s still work needed to fill in
	@echo gaps where conversion is incomplete or unstyled.
	@echo https://docs.asciidoctor.org/epub3-converter/latest/#project-status
	@echo ---------------------------------------------------------------
.for _lang in ${ARTICLE_LANGS}
	./tools/asciidoctor.sh articles ${_lang} epub
.endfor

epub-clean: epub-articles-clean epub-books-clean

epub-books-clean:
.for _lang in ${BOOK_LANGS}
	rm -fr ${.CURDIR}/public/${_lang}/books
	-rmdir ${.CURDIR}/public/${_lang}
.endfor
	-rmdir ${.CURDIR}/public/

epub-articles-clean:
.for _lang in ${ARTICLE_LANGS}
	rm -fr ${.CURDIR}/public/${_lang}/articles
.if !exists(${.CURDIR}/public/${_lang}/books)
	rm -fr ${.CURDIR}/public/${_lang}
.endif
.endfor
	-rmdir ${.CURDIR}/public
....

<.> The `MAINTAINER` flag specifies who is the maintainer of this Makefile.
<.> `ALL_LANGUAGES` flag specifies in which languages the table of contents has to be generated.
<.> `RUBY_CMD` flag specifies the location of the Ruby binary.
<.> `HUGO_CMD` flag specifies the location of the Hugo binary.
<.> `.ORDER` directives are used to ensure multiple make jobs may run without problem.
<.> `all` target builds the documentation and puts the result in *~/doc/documentation/public*.
<.> `starting-message` shows a message in the CLI to show the user that the process is running.
<.> `run-local` runs hugo webserver on port 1313, or a random free port if that is already in use.
<.> `build` builds the documentation and puts the result in the *~/doc/documentation/public*.

[[website-makefile]]
=== Website Makefile

This [.filename]#Makefile# takes the form of:

[source,shell]
....
# Generate the FreeBSD website
#
# Copyright (c) 2020-2025, The FreeBSD Documentation Project
# Copyright (c) 2020-2025, Sergio Carlavilla <carlavilla@FreeBSD.org>
#
# Targets intended for use on the command line
#
# all (default)	-	generate the releases.toml and compile all the website
# run	-			serves the built website for local browsing
#
# The run target uses hugo's built-in webserver to make the built website
# available for local browsing.  The website should have been built prior
# to attempting to use the `run` target.  By default, hugo will start its
# webserver on port 1313.

MAINTAINER=carlavilla@FreeBSD.org <.>

# List of all languages we have content for
ALL_LANGUAGES=	de el en es fr hu it ja nl ru tr zh-cn zh-tw

LOCALBASE?=	/usr/local

RUBY_CMD =	${LOCALBASE}/bin/ruby <.>
HUGO_CMD =	${LOCALBASE}/bin/hugo <.>
HUGO_ARGS?=	--verbose
RUBYLIB =	../shared/lib
.export	RUBYLIB

.ifndef HOSTNAME
.  ifdef BIND
.HOST=$(BIND)
.  else
.HOST=localhost
.  endif
.else
.HOST=$(HOSTNAME)
.endif

.if defined(DOC_LANG) && !empty(DOC_LANG)
LANGUAGES=      ${DOC_LANG:S/,/ /g}
.if  ${LANGUAGES:Men} == ""
.warning "Warning: cannot skip 'en'; adding it back"
LANGUAGES+=	en
.endif
.else
LANGUAGES=	${ALL_LANGUAGES}
.endif

# Take the list of all languages, and take out the ones we have been
#   asked for via DOC_LANG.  We'll feed this to hugo.
SKIP_LANGS=
.for a in ${ALL_LANGUAGES}
.if ${LANGUAGES:M${a}} == ""
SKIP_LANGS+=	${a}
.endif
.endfor

.ORDER: all run <.>

.ORDER: starting-message generate-releases
.ORDER: starting-message build
.ORDER: generate-releases build
.ORDER: build post-build
.ORDER: post-build end-message

all: starting-message generate-releases build post-build end-message <.>
run: starting-message generate-releases run-local
clean: hugo-clean releases-clean

starting-message: .PHONY <.>
	@echo "---------------------------------------------------------------"
	@echo "Building the website started on $$(date)"
	@echo " included languages: ${LANGUAGES}"
	@echo " excluded languages: ${SKIP_LANGS}"
	@echo "---------------------------------------------------------------"

end-message: .PHONY
	@echo "---------------------------------------------------------------"
	@echo "Building the website completed on $$(date)"
	@echo "---------------------------------------------------------------"

generate-releases: data/releases.toml <.>

data/releases.toml:
	${RUBY_CMD} ./tools/releases-toml.rb

run-local: .PHONY <.>
	HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} server \
	    ${HUGO_ARGS} -D $(BIND:D--bind=$(BIND)) --baseURL="http://$(.HOST):1313"

build: .PHONY <.>
	HUGO_DISABLELANGUAGES="${SKIP_LANGS}" ${HUGO_CMD} ${HUGO_ARGS}

post-build: cgi-permissions

cgi-permissions:
	@chmod 555 ./public/cgi/*.cgi

hugo-clean:
	rm -fr public resources

releases-clean:
	rm -f data/releases.toml
....

<.> The `MAINTAINER` flag specifies who is the maintainer of this Makefile.
<.> `RUBY_CMD` flag specifies the location of the Ruby binary.
<.> `HUGO_CMD` flag specifies the location of the Hugo binary.
<.> `.ORDER` directives are used to ensure multiple make jobs may run without problem.
<.> `all` target builds the website and puts the result in *~/doc/website/public*.
<.> `starting-message` shows a message in the CLI to show the user that the process is running.
<.> `generate-releases` calls the script used to convert from AsciiDoc variables to TOML variables.
With this conversion, the releases variables can be used in AsciiDoc and in the Hugo custom templates.
<.> `run-local` runs hugo webserver on port 1313, or a random free port if that is already in use.
<.> `build` builds the website and puts the result in the *~/doc/website/public*.