1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
|
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//FreeBSD//DTD XHTML 1.0 Transitional-Based Extension//EN"
"http://www.FreeBSD.org/XML/doc/share/xml/xhtml10-freebsd.dtd" [
<!ENTITY title "Quality Assurance Tasks for the Ports Management Team">
]>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>&title;</title>
<cvs:keyword xmlns:cvs="http://www.FreeBSD.org/XML/CVS">$FreeBSD$</cvs:keyword>
</head>
<body class="navinclude.about">
<p>There are a number of tasks that the Ports Management Team undertakes
to try to improve the quality of the Ports Collection. These fall into
two main categories:
<a href="#qa-before-release">activities during a release cycle</a> and
<a href="#qa-between-releases">activities between release cycles</a>.
</p>
<h3><a name="qa-before-release">Activities During a Release Cycle</a></h3>
<ul>
<li>
<p>Work with the
<a href="../releng/index.html">Release Engineering Team</a>
to coordinate the release schedule.</p>
</li>
<li>
<p>Work with the RE team to determine which pre-built packages
can be included on the default install ISOs.</p>
</li>
<li>
<p>Manage commits to the CVS tree for package builds via the
following steps:</p>
<ol>
<li>
<p>Institute a freeze and produce packages for all the
appropriate architectures. Often this process has to be
repeated because either bugs are identified in various ports,
or changes to the src tree create a risk that the packages
that have already been built would not work with those
changes.</p>
<p>To make sure that package builds are consistent and correct,
<i>all</i> commits must be approved by portmgr during a
freeze. Changes that are generally approved are:</p>
<ul>
<li><p>fixes to make a package build at all;</p></li>
<li><p>security fixes to critical packages;</p></li>
<li><p>problems that are noticed with licensing issues.</p></li>
</ul>
<p>Unfortunately, due to the sheer size of the Ports Collection
and the speed that applications are developed, it is
impossible to fix every single problem for a release.</p>
</li>
<li>
<p>The tree is then locked for all commits and a CVS tag is laid
down.</p>
</li>
<li>
<p>The tree is then unlocked and a <tt>slush</tt> is
announced. The intent of this state is to allow routine
changes to be made to the Ports Collection, but with the note
that these changes will not ship on the release ISOs. What
we particularly want to avoid is
<a href="implementation.html#sweeping_changes">
sweeping changes</a>.</p>
<p>The reason we want to avoid these commits is if some kind
of show-stopper problem is found (either security- or license-
related) such that we need to make a change that can go on
the release ISOs, we will need to slip the CVS tag on the
changed file(s). By allowing unlimited commits, the risk is
high that any such change would involve having to recreate all
the packages all over again, resulting in an endless release
cycle.</p>
</li>
</ol>
<p>Only once the RE team and portmgr are happy with the final
state of the release ISOs is the ports tree completely available
for commits again.</p>
</li>
</ul>
<h3><a name="qa-between-releases">Activities Between Release Cycles</a></h3>
<ul>
<li>
<p>Manage the <a href="http://pointyhat.FreeBSD.org">Ports Build
Cluster</a> machines. These machines continually build packages
on all possible combinations of OS release and CPU architecture
(in our terminology, <tt>build environments</tt>.)</p>
<p>These builds also produce error logs for packages that do not
build correctly (see the above URL). Periodically, the team
marks these ports as BROKEN so that maintainers may be notified.
(See below.)</p>
<p>Successfully built packages (at least, the ones that are freely
redistributable) are also copied to the master FTP server and thus
become the default "latest package" for installations
that use packages rather than ports.</p>
</li>
<li>
<p>Notify the FreeBSD community of problems within the Ports
Collection so that problems do not get overlooked. To do this,
there are a number of emailed reports. Ones marked
<tt>public</tt> are posted to freebsd-ports.</p>
<ul>
<li><p>a public list of all ports to be removed due to security
problems, build failures, or general obsolescence, unless they
are fixed first.</p></li>
<li><p>private email to all maintainers of the affected ports
(including ports dependent on the above).</p></li>
<li><p>private email to all maintainers of ports that are already
marked BROKEN and/or FORBIDDEN.</p></li>
<li><p>private email to maintainers who are not committers, who have
PRs filed against their ports (to flag PRs that might never have
been Cc:ed to them).</p></li>
<li><p>public email about port commits that break building of
INDEX.</p></li>
<li><p>public email about port commits that send the revision
metadata backwards (and thus confuse tools like portupgrade).</p></li>
<li><p>a public list of all ports that have at least one file
that fails to fetch from any non-FreeBSD mastersite. For the
complete list of results for all files versus all mastersites,
see <a href="http://people.freebsd.org/~ehaupt/distilator/">
Emanuel Haupt's port survey</a>.</p></li>
<li><p>private email to an affected port maintainer when a port
is about to be marked BROKEN, Cc:ed to the last committer to
the port. (This email is not automated but it should be sent
as a courtesy.)</p></li>
<li><p>a list of ports that do not set NO_LATEST_LINK. (Ports
that have a stable version, and a development version, will
generally have the development version set to a later revision.
If it is desirable that users should install the stable version
from packages, rather than the development version, this flag
should be set; otherwise, users will get the latest version by
default.)</p></li>
</ul>
</li>
<li>
<p>Remove expired ports. Ports that have been marked BROKEN for
some time are marked DEPRECATED (with an EXPIRATION_DATE) and then
are removed if no one has fixed them by that time. The intent of
this this process is to try to insure that if a user installs a port,
there is the best possible change that it can be made to work.</p>
<p>In other cases, ports are marked DEPRECATED when they have been
replaced by a newer version and the older version is no longer
maintained by the authors. The EXPIRATION_DATE should generally
be set at least two months in the future to allow everyone sufficient
time to upgrade.</p>
</li>
</ul>
</body>
</html>
|
