hgbook
view en/mq-collab.tex @ 105:ecacb6b4c9fd
Grouping patches in the series file.
| author | Bryan O'Sullivan <bos@serpentine.com> |
|---|---|
| date | Sat Oct 21 11:05:51 2006 -0700 (2006-10-21) |
| parents | 32bf9a5f22c0 |
| children | 9cbc5d0db542 |
line source
1 \chapter{Advanced uses of Mercurial Queues}
3 While it's easy to pick up straightforward uses of Mercurial Queues,
4 use of a little discipline and some of MQ's less frequently used
5 capabilities makes it possible to work in complicated development
6 environments.
8 In this chapter, I will use as an example a technique I have used to
9 manage the development of an Infiniband device driver for the Linux
10 kernel. The driver in question is large (at least as drivers go),
11 with 25,000 lines of code spread across 35 source files. It is
12 maintained by a small team of developers.
14 While much of the material in this chapter is specific to Linux, the
15 same principles apply to any code base for which you're not the
16 primary owner, and upon which you need to do a lot of development.
18 \section{The problem of many targets}
20 The Linux kernel changes rapidly, and has never been internally
21 stable; developers frequently make drastic changes between releases.
22 This means that a version of the driver that works well with a
23 particular released version of the kernel will not even \emph{compile}
24 correctly against, typically, any other version.
26 To maintain a driver, we have to keep a number of distinct versions of
27 Linux in mind.
28 \begin{itemize}
29 \item One target is the main Linux kernel development tree.
30 Maintenance of the code is in this case partly shared by other
31 developers in the kernel community, who make ``drive-by''
32 modifications to the driver as they develop and refine kernel
33 subsystems.
34 \item We also maintain a number of ``backports'' to older versions of
35 the Linux kernel, to support the needs of customers who are running
36 older Linux distributions that do not incorporate our drivers. (To
37 \emph{backport} a piece of code is to modify it to work in an older
38 version of its target environment than the version it was developed
39 for.)
40 \item Finally, we make software releases on a schedule that is
41 necessarily not aligned with those used by Linux distributors and
42 kernel developers, so that we can deliver new features to customers
43 without forcing them to upgrade their entire kernels or
44 distributions.
45 \end{itemize}
47 \subsection{Tempting approaches that don't work well}
49 There are two ``standard'' ways to maintain a piece of software that
50 has to target many different environments.
52 The first is to maintain a number of branches, each intended for a
53 single target. The trouble with this approach is that you must
54 maintain iron discipline in the flow of changes between repositories.
55 A new feature or bug fix must start life in a ``pristine'' repository,
56 then percolate out to every backport repository. Backport changes are
57 more limited in the branches they should propagate to; a backport
58 change that is applied to a branch where it doesn't belong will
59 probably stop the driver from compiling.
61 The second is to maintain a single source tree filled with conditional
62 statements that turn chunks of code on or off depending on the
63 intended target. Because these ``ifdefs'' are not allowed in the
64 Linux kernel tree, a manual or automatic process must be followed to
65 strip them out and yield a clean tree. A code base maintained in this
66 fashion rapidly becomes a rat's nest of conditional blocks that are
67 difficult to understand and maintain.
69 Neither of these approaches is well suited to a situation where you
70 don't ``own'' the canonical copy of a source tree. In the case of a
71 Linux driver that is distributed with the standard kernel, Linus's
72 tree contains the copy of the code that will be treated by the world
73 as canonical. The upstream version of ``my'' driver can be modified
74 by people I don't know, without me even finding out about it until
75 after the changes show up in Linus's tree.
77 These approaches have the added weakness of making it difficult to
78 generate well-formed patches to submit upstream.
80 In principle, Mercurial Queues seems like a good candidate to manage a
81 development scenario such as the above. While this is indeed the
82 case, MQ contains a few added features that make the job more
83 pleasant.
85 \section{Conditionally applying patches with
86 guards}
88 Perhaps the best way to maintain sanity with so many targets is to be
89 able to choose specific patches to apply for a given situation. MQ
90 provides a feature called ``guards'' (which originates with quilt's
91 \texttt{guards} command) that does just this. To start off, let's
92 create a simple repository for experimenting in.
93 \interaction{mq.guards.init}
94 This gives us a tiny repository that contains two patches that don't
95 have any dependencies on each other, because they touch different files.
97 The idea behind conditional application is that you can ``tag'' a
98 patch with a \emph{guard}, which is simply a text string of your
99 choosing, then tell MQ to select specific guards to use when applying
100 patches. MQ will then either apply, or skip over, a guarded patch,
101 depending on the guards that you have selected.
103 A patch can have an arbitrary number of guards;
104 each one is \emph{positive} (``apply this patch if this guard is
105 selected'') or \emph{negative} (``skip this patch if this guard is
106 selected''). A patch with no guards is always applied.
108 \section{Controlling the guards on a patch}
110 The \hgcmd{qguard} command lets you determine which guards should
111 apply to a patch, or display the guards that are already in effect.
112 Without any arguments, it displays the guards on the current topmost
113 patch.
114 \interaction{mq.guards.qguard}
115 To set a positive guard on a patch, prefix the name of the guard with
116 a ``\texttt{+}''.
117 \interaction{mq.guards.qguard.pos}
118 To set a negative guard on a patch, prefix the name of the guard with
119 a ``\texttt{-}''.
120 \interaction{mq.guards.qguard.neg}
122 \begin{note}
123 The \hgcmd{qguard} command \emph{sets} the guards on a patch; it
124 doesn't \emph{modify} them. What this means is that if you run
125 \hgcmdargs{qguard}{+a +b} on a patch, then \hgcmdargs{qguard}{+c} on
126 the same patch, the \emph{only} guard that will be set on it
127 afterwards is \texttt{+c}.
128 \end{note}
130 Mercurial stores guards in the \sfilename{series} file; the form in
131 which they are stored is easy both to understand and to edit by hand.
132 (In other words, you don't have to use the \hgcmd{qguard} command if
133 you don't want to; it's okay to simply edit the \sfilename{series}
134 file.)
135 \interaction{mq.guards.series}
137 \section{Selecting the guards to use}
139 The \hgcmd{qselect} command determines which guards are active at a
140 given time. The effect of this is to determine which patches MQ will
141 apply the next time you run \hgcmd{qpush}. It has no other effect; in
142 particular, it doesn't do anything to patches that are already
143 applied.
145 With no arguments, the \hgcmd{qselect} command lists the guards
146 currently in effect, one per line of output. Each argument is treated
147 as the name of a guard to apply.
148 \interaction{mq.guards.qselect.foo}
149 In case you're interested, the currently selected guards are stored in
150 the \sfilename{guards} file.
151 \interaction{mq.guards.qselect.cat}
152 We can see the effect the selected guards have when we run
153 \hgcmd{qpush}.
154 \interaction{mq.guards.qselect.qpush}
156 A guard cannot start with a ``\texttt{+}'' or ``\texttt{-}''
157 character. The name of a guard must start with an alphabetic
158 character (upper or lower case) or an underscore. The rest of the
159 guard's name can contain any of these characters, or a digit. These
160 rules are similar to those used for variable naming in most popular
161 programming languages. If you try to use a guard with an invalid
162 name, MQ will complain:
163 \interaction{mq.guards.qselect.error}
164 Changing the selected guards changes the patches that are applied.
165 \interaction{mq.guards.qselect.quux}
166 You can see in the example below that negative guards take precedence
167 over positive guards.
168 \interaction{mq.guards.qselect.foobar}
170 \section{MQ's rules for applying patches}
172 The rules that MQ uses when deciding whether to apply a patch
173 are as follows.
174 \begin{itemize}
175 \item A patch that has no guards is always applied.
176 \item If the patch has any negative guard that matches any currently
177 selected guard, the patch is skipped.
178 \item If the patch has any positive guard that matches any currently
179 selected guard, the patch is applied.
180 \item If the patch has positive or negative guards, but none matches
181 any currently selected guard, the patch is skipped.
182 \end{itemize}
184 \section{Trimming the work environment}
186 In working on the device driver I mentioned earlier, I don't apply the
187 patches to a normal Linux kernel tree. Instead, I use a repository
188 that contains only a snapshot of the source files and headers that are
189 relevant to Infiniband development. This repository is~1\% the size
190 of a kernel repository, so it's easier to work with.
192 I then choose a ``base'' version on top of which the patches are
193 applied. This is a snapshot of the Linux kernel tree as of a revision
194 of my choosing. When I take the snapshot, I record the changeset ID
195 from the kernel repository in the commit message. Since the snapshot
196 preserves the ``shape'' and content of the relevant parts of the
197 kernel tree, I can apply my patches on top of either my tiny
198 repository or a normal kernel tree.
200 Normally, the base tree atop which the patches apply should be a
201 snapshot of a very recent upstream tree. This best facilitates the
202 development of patches that can easily be submitted upstream with few
203 or no modifications.
205 \section{Dividing up the \sfilename{series} file}
207 I categorise the patches in the \sfilename{series} file into a number
208 of logical groups. Each section of like patches begins with a block
209 of comments that describes the purpose of the patches that follow.
211 The sequence of patch groups that I maintain follows. The ordering of
212 these groups is important; I'll describe why after I introduce the
213 groups.
214 \begin{itemize}
215 \item The ``accepted'' group. Patches that the development team has
216 submitted to the maintainer of the Infiniband subsystem, and which
217 he has accepted, but which are not present in the snapshot that the
218 tiny repository is based on. These are ``read only'' patches,
219 present only to transform the tree into a similar state as it is in
220 the upstream maintainer's repository.
221 \item The ``rework'' group. Patches that I have submitted, but that
222 the upstream maintainer has requested modifications to before he
223 will accept them.
224 \item The ``pending'' group. Patches that I have not yet submitted to
225 the upstream maintainer, but which we have finished working on.
226 These will be ``read only'' for a while. If the upstream maintainer
227 accepts them upon submission, I'll move them to the end of the
228 ``accepted'' group. If he requests that I modify any, I'll move
229 them to the beginning of the ``rework'' group.
230 \item The ``in progress'' group. Patches that are actively being
231 developed, and should not be submitted anywhere yet.
232 \item The ``backport'' group. Patches that adapt the source tree to
233 older versions of the kernel tree.
234 \item The ``do not ship'' group. Patches that for some reason should
235 never be submitted upstream. For example, one such patch might
236 change embedded driver identification strings to make it easier to
237 distinguish, in the field, between an out-of-tree version of the
238 driver and a version shipped by a distribution vendor.
239 \end{itemize}
241 Now to return to the reasons for ordering groups of patches in this
242 way. We would like the lowest patches in the stack to be as stable as
243 possible, so that we will not need to rework higher patches due to
244 changes in context. Putting patches that will never be changed first
245 in the \sfilename{series} file serves this purpose.
247 We would also like the patches that we know we'll need to modify to be
248 applied on top of a source tree that resembles the upstream tree as
249 closely as possible. This is why we keep accepted patches around for
250 a while.
252 The ``backport'' and ``do not ship'' patches float at the end of the
253 \sfilename{series} file in part because they'll never be shipped
254 upstream. Additionally, the backport patches must be applied on top
255 of all other patches.
257 %%% Local Variables:
258 %%% mode: latex
259 %%% TeX-master: "00book"
260 %%% End: