
bos@121: \chapter{Finding and fixing your mistakes}
bos@121: \label{chap:undo}
bos@121: 
bos@121: To err might be human, but to really handle the consequences well
bos@121: takes a top-notch revision control system.  In this chapter, we'll
bos@121: discuss some of the techniques you can use when you find that a
bos@121: problem has crept into your project.  Mercurial has some highly
bos@121: capable features that will help you to isolate the sources of
bos@121: problems, and to handle them appropriately.
bos@121: 
bos@122: \section{Erasing local history}
bos@121: 
bos@121: \subsection{The accidental commit}
bos@121: 
bos@121: I have the occasional but persistent problem of typing rather more
bos@121: quickly than I can think, which sometimes results in me committing a
bos@121: changeset that is either incomplete or plain wrong.  In my case, the
bos@121: usual kind of incomplete changeset is one in which I've created a new
bos@121: source file, but forgotten to \hgcmd{add} it.  A ``plain wrong''
bos@121: changeset is not as common, but no less annoying.
bos@121: 
bos@121: \subsection{Rolling back a transaction}
bos@126: \label{sec:undo:rollback}
bos@121: 
bos@121: In section~\ref{sec:concepts:txn}, I mentioned that Mercurial treats
bos@121: each modification of a repository as a \emph{transaction}.  Every time
bos@121: you commit a changeset or pull changes from another repository,
bos@121: Mercurial remembers what you did.  You can undo, or \emph{roll back},
bos@200: exactly one of these actions using the \hgcmd{rollback} command.  (See
bos@200: section~\ref{sec:undo:rollback-after-push} for an important caveat
bos@200: about the use of this command.)
bos@121: 
bos@121: Here's a mistake that I often find myself making: committing a change
bos@121: in which I've created a new file, but forgotten to \hgcmd{add} it.
bos@121: \interaction{rollback.commit}
bos@121: Looking at the output of \hgcmd{status} after the commit immediately
bos@121: confirms the error.
bos@121: \interaction{rollback.status}
bos@121: The commit captured the changes to the file \filename{a}, but not the
bos@121: new file \filename{b}.  If I were to push this changeset to a
bos@121: repository that I shared with a colleague, the chances are high that
bos@121: something in \filename{a} would refer to \filename{b}, which would not
bos@121: be present in their repository when they pulled my changes.  I would
bos@121: thus become the object of some indignation.
bos@121: 
bos@121: However, luck is with me---I've caught my error before I pushed the
bos@121: changeset.  I use the \hgcmd{rollback} command, and Mercurial makes
bos@121: that last changeset vanish.
bos@121: \interaction{rollback.rollback}
bos@121: Notice that the changeset is no longer present in the repository's
bos@121: history, and the working directory once again thinks that the file
bos@122: \filename{a} is modified.  The commit and rollback have left the
bos@122: working directory exactly as it was prior to the commit; the changeset
bos@122: has been completely erased.  I can now safely \hgcmd{add} the file
bos@122: \filename{b}, and rerun my commit.
bos@121: \interaction{rollback.add}
bos@121: 
bos@121: \subsection{The erroneous pull}
bos@121: 
bos@121: It's common practice with Mercurial to maintain separate development
bos@121: branches of a project in different repositories.  Your development
bos@121: team might have one shared repository for your project's ``0.9''
bos@121: release, and another, containing different changes, for the ``1.0''
bos@121: release.
bos@121: 
bos@121: Given this, you can imagine that the consequences could be messy if
bos@121: you had a local ``0.9'' repository, and accidentally pulled changes
bos@121: from the shared ``1.0'' repository into it.  At worst, you could be
bos@121: paying insufficient attention, and push those changes into the shared
bos@121: ``0.9'' tree, confusing your entire team (but don't worry, we'll
bos@121: return to this horror scenario later).  However, it's more likely that
bos@121: you'll notice immediately, because Mercurial will display the URL it's
bos@121: pulling from, or you will see it pull a suspiciously large number of
bos@121: changes into the repository.
bos@121: 
bos@121: The \hgcmd{rollback} command will work nicely to expunge all of the
bos@121: changesets that you just pulled.  Mercurial groups all changes from
bos@121: one \hgcmd{pull} into a single transaction, so one \hgcmd{rollback} is
bos@121: all you need to undo this mistake.
bos@121: 
bos@121: \subsection{Rolling back is useless once you've pushed}
bos@200: \label{sec:undo:rollback-after-push}
bos@121: 
bos@121: The value of the \hgcmd{rollback} command drops to zero once you've
bos@121: pushed your changes to another repository.  Rolling back a change
bos@121: makes it disappear entirely, but \emph{only} in the repository in
bos@121: which you perform the \hgcmd{rollback}.  Because a rollback eliminates
bos@121: history, there's no way for the disappearance of a change to propagate
bos@121: between repositories.
bos@121: 
bos@121: If you've pushed a change to another repository---particularly if it's
bos@121: a shared repository---it has essentially ``escaped into the wild,''
bos@121: and you'll have to recover from your mistake in a different way.  What
bos@121: will happen if you push a changeset somewhere, then roll it back, then
bos@121: pull from the repository you pushed to, is that the changeset will
bos@121: reappear in your repository.
bos@121: 
bos@121: (If you absolutely know for sure that the change you want to roll back
bos@121: is the most recent change in the repository that you pushed to,
bos@121: \emph{and} you know that nobody else could have pulled it from that
bos@121: repository, you can roll back the changeset there, too, but you really
bos@121: should really not rely on this working reliably.  If you do this,
bos@121: sooner or later a change really will make it into a repository that
bos@121: you don't directly control (or have forgotten about), and come back to
bos@121: bite you.)
bos@121: 
bos@121: \subsection{You can only roll back once}
bos@121: 
bos@121: Mercurial stores exactly one transaction in its transaction log; that
bos@121: transaction is the most recent one that occurred in the repository.
bos@121: This means that you can only roll back one transaction.  If you expect
bos@121: to be able to roll back one transaction, then its predecessor, this is
bos@121: not the behaviour you will get.
bos@121: \interaction{rollback.twice}
bos@121: Once you've rolled back one transaction in a repository, you can't
bos@121: roll back again in that repository until you perform another commit or
bos@121: pull.
bos@121: 
bos@122: \section{Reverting the mistaken change}
bos@122: 
bos@122: If you make a modification to a file, and decide that you really
bos@124: didn't want to change the file at all, and you haven't yet committed
bos@124: your changes, the \hgcmd{revert} command is the one you'll need.  It
bos@124: looks at the changeset that's the parent of the working directory, and
bos@124: restores the contents of the file to their state as of that changeset.
bos@124: (That's a long-winded way of saying that, in the normal case, it
bos@124: undoes your modifications.)
bos@122: 
bos@122: Let's illustrate how the \hgcmd{revert} command works with yet another
bos@122: small example.  We'll begin by modifying a file that Mercurial is
bos@122: already tracking.
bos@122: \interaction{daily.revert.modify}
bos@122: If we don't want that change, we can simply \hgcmd{revert} the file.
bos@122: \interaction{daily.revert.unmodify}
bos@122: The \hgcmd{revert} command provides us with an extra degree of safety
bos@122: by saving our modified file with a \filename{.orig} extension.
bos@122: \interaction{daily.revert.status}
bos@122: 
bos@124: Here is a summary of the cases that the \hgcmd{revert} command can
bos@124: deal with.  We will describe each of these in more detail in the
bos@124: section that follows.
bos@124: \begin{itemize}
bos@124: \item If you modify a file, it will restore the file to its unmodified
bos@124:   state.
bos@124: \item If you \hgcmd{add} a file, it will undo the ``added'' state of
bos@124:   the file, but leave the file itself untouched.
bos@124: \item If you delete a file without telling Mercurial, it will restore
bos@124:   the file to its unmodified contents.
bos@124: \item If you use the \hgcmd{remove} command to remove a file, it will
bos@124:   undo the ``removed'' state of the file, and restore the file to its
bos@124:   unmodified contents.
bos@124: \end{itemize}
bos@124: 
bos@122: \subsection{File management errors}
bos@122: \label{sec:undo:mgmt}
bos@122: 
bos@122: The \hgcmd{revert} command is useful for more than just modified
bos@122: files.  It lets you reverse the results of all of Mercurial's file
bos@122: management commands---\hgcmd{add}, \hgcmd{remove}, and so on.
bos@122: 
bos@122: If you \hgcmd{add} a file, then decide that in fact you don't want
bos@122: Mercurial to track it, use \hgcmd{revert} to undo the add.  Don't
bos@122: worry; Mercurial will not modify the file in any way.  It will just
bos@122: ``unmark'' the file.
bos@122: \interaction{daily.revert.add}
bos@122: 
bos@122: Similarly, if you ask Mercurial to \hgcmd{remove} a file, you can use
bos@122: \hgcmd{revert} to restore it to the contents it had as of the parent
bos@122: of the working directory.
bos@122: \interaction{daily.revert.remove}
bos@122: This works just as well for a file that you deleted by hand, without
bos@122: telling Mercurial (recall that in Mercurial terminology, this kind of
bos@122: file is called ``missing'').
bos@122: \interaction{daily.revert.missing}
bos@122: 
bos@122: If you revert a \hgcmd{copy}, the copied-to file remains in your
bos@123: working directory afterwards, untracked.  Since a copy doesn't affect
bos@123: the copied-from file in any way, Mercurial doesn't do anything with
bos@123: the copied-from file.
bos@122: \interaction{daily.revert.copy}
bos@122: 
bos@122: \subsubsection{A slightly special case: reverting a rename}
bos@122: 
bos@122: If you \hgcmd{rename} a file, there is one small detail that
bos@122: you should remember.  When you \hgcmd{revert} a rename, it's not
bos@122: enough to provide the name of the renamed-to file, as you can see
bos@122: here.
bos@122: \interaction{daily.revert.rename}
bos@122: As you can see from the output of \hgcmd{status}, the renamed-to file
bos@122: is no longer identified as added, but the renamed-\emph{from} file is
bos@122: still removed!  This is counter-intuitive (at least to me), but at
bos@122: least it's easy to deal with.
bos@122: \interaction{daily.revert.rename-orig}
bos@122: So remember, to revert a \hgcmd{rename}, you must provide \emph{both}
bos@122: the source and destination names.  
bos@122: 
bos@122: (By the way, if you rename a file, then modify the renamed-to file,
bos@122: then revert both components of the rename, when Mercurial restores the
bos@122: file that was removed as part of the rename, it will be unmodified.
bos@122: If you need the modifications in the renamed-to file to show up in the
bos@122: renamed-from file, don't forget to copy them over.)
bos@122: 
bos@123: These fiddly aspects of reverting a rename arguably constitute a small
bos@122: bug in Mercurial.
bos@122: 
bos@124: \section{Dealing with committed changes}
bos@124: 
bos@124: Consider a case where you have committed a change $a$, and another
bos@124: change $b$ on top of it; you then realise that change $a$ was
bos@124: incorrect.  Mercurial lets you ``back out'' an entire changeset
bos@124: automatically, and building blocks that let you reverse part of a
bos@124: changeset by hand.
bos@124: 
bos@126: Before you read this section, here's something to keep in mind: the
bos@126: \hgcmd{backout} command undoes changes by \emph{adding} history, not
bos@126: by modifying or erasing it.  It's the right tool to use if you're
bos@126: fixing bugs, but not if you're trying to undo some change that has
bos@126: catastrophic consequences.  To deal with those, see
bos@126: section~\ref{sec:undo:aaaiiieee}.
bos@126: 
bos@124: \subsection{Backing out a changeset}
bos@124: 
bos@124: The \hgcmd{backout} command lets you ``undo'' the effects of an entire
bos@124: changeset in an automated fashion.  Because Mercurial's history is
bos@124: immutable, this command \emph{does not} get rid of the changeset you
bos@124: want to undo.  Instead, it creates a new changeset that
bos@124: \emph{reverses} the effect of the to-be-undone changeset.
bos@124: 
bos@124: The operation of the \hgcmd{backout} command is a little intricate, so
bos@124: let's illustrate it with some examples.  First, we'll create a
bos@124: repository with some simple changes.
bos@124: \interaction{backout.init}
bos@124: 
bos@124: The \hgcmd{backout} command takes a single changeset ID as its
bos@124: argument; this is the changeset to back out.  Normally,
bos@124: \hgcmd{backout} will drop you into a text editor to write a commit
bos@124: message, so you can record why you're backing the change out.  In this
bos@124: example, we provide a commit message on the command line using the
bos@124: \hgopt{backout}{-m} option.
bos@124: 
bos@124: \subsection{Backing out the tip changeset}
bos@124: 
bos@124: We're going to start by backing out the last changeset we committed.
bos@124: \interaction{backout.simple}
bos@124: You can see that the second line from \filename{myfile} is no longer
bos@124: present.  Taking a look at the output of \hgcmd{log} gives us an idea
bos@124: of what the \hgcmd{backout} command has done.
bos@124: \interaction{backout.simple.log}
bos@124: Notice that the new changeset that \hgcmd{backout} has created is a
bos@124: child of the changeset we backed out.  It's easier to see this in
bos@124: figure~\ref{fig:undo:backout}, which presents a graphical view of the
bos@124: change history.  As you can see, the history is nice and linear.
bos@124: 
bos@124: \begin{figure}[htb]
bos@124:   \centering
bos@124:   \grafix{undo-simple}
bos@124:   \caption{Backing out a change using the \hgcmd{backout} command}
bos@124:   \label{fig:undo:backout}
bos@124: \end{figure}
bos@124: 
bos@124: \subsection{Backing out a non-tip change}
bos@124: 
bos@124: If you want to back out a change other than the last one you
bos@124: committed, pass the \hgopt{backout}{--merge} option to the
bos@124: \hgcmd{backout} command.
bos@124: \interaction{backout.non-tip.clone}
bos@124: This makes backing out any changeset a ``one-shot'' operation that's
bos@124: usually simple and fast.
bos@124: \interaction{backout.non-tip.backout}
bos@124: 
bos@124: If you take a look at the contents of \filename{myfile} after the
bos@124: backout finishes, you'll see that the first and third changes are
bos@124: present, but not the second.
bos@124: \interaction{backout.non-tip.cat}
bos@124: 
bos@124: As the graphical history in figure~\ref{fig:undo:backout-non-tip}
bos@124: illustrates, Mercurial actually commits \emph{two} changes in this
bos@124: kind of situation (the box-shaped nodes are the ones that Mercurial
bos@124: commits automatically).  Before Mercurial begins the backout process,
bos@124: it first remembers what the current parent of the working directory
bos@124: is.  It then backs out the target changeset, and commits that as a
bos@124: changeset.  Finally, it merges back to the previous parent of the
bos@124: working directory, and commits the result of the merge.
bos@124: 
bos@124: \begin{figure}[htb]
bos@124:   \centering
bos@124:   \grafix{undo-non-tip}
bos@124:   \caption{Automated backout of a non-tip change using the \hgcmd{backout} command}
bos@124:   \label{fig:undo:backout-non-tip}
bos@124: \end{figure}
bos@124: 
bos@124: The result is that you end up ``back where you were'', only with some
bos@124: extra history that undoes the effect of the changeset you wanted to
bos@124: back out.
bos@124: 
bos@124: \subsubsection{Always use the \hgopt{backout}{--merge} option}
bos@124: 
bos@124: In fact, since the \hgopt{backout}{--merge} option will do the ``right
bos@124: thing'' whether or not the changeset you're backing out is the tip
bos@124: (i.e.~it won't try to merge if it's backing out the tip, since there's
bos@124: no need), you should \emph{always} use this option when you run the
bos@124: \hgcmd{backout} command.
bos@124: 
bos@124: \subsection{Gaining more control of the backout process}
bos@124: 
bos@124: While I've recommended that you always use the
bos@124: \hgopt{backout}{--merge} option when backing out a change, the
bos@124: \hgcmd{backout} command lets you decide how to merge a backout
bos@124: changeset.  Taking control of the backout process by hand is something
bos@124: you will rarely need to do, but it can be useful to understand what
bos@124: the \hgcmd{backout} command is doing for you automatically.  To
bos@124: illustrate this, let's clone our first repository, but omit the
bos@124: backout change that it contains.
bos@124: 
bos@124: \interaction{backout.manual.clone}
bos@124: As with our earlier example, We'll commit a third changeset, then back
bos@124: out its parent, and see what happens.
bos@124: \interaction{backout.manual.backout} 
bos@124: Our new changeset is again a descendant of the changeset we backout
bos@124: out; it's thus a new head, \emph{not} a descendant of the changeset
bos@124: that was the tip.  The \hgcmd{backout} command was quite explicit in
bos@124: telling us this.
bos@124: \interaction{backout.manual.log}
bos@124: 
bos@124: Again, it's easier to see what has happened by looking at a graph of
bos@124: the revision history, in figure~\ref{fig:undo:backout-manual}.  This
bos@124: makes it clear that when we use \hgcmd{backout} to back out a change
bos@124: other than the tip, Mercurial adds a new head to the repository (the
bos@124: change it committed is box-shaped).
bos@124: 
bos@124: \begin{figure}[htb]
bos@124:   \centering
bos@124:   \grafix{undo-manual}
bos@124:   \caption{Backing out a change using the \hgcmd{backout} command}
bos@124:   \label{fig:undo:backout-manual}
bos@124: \end{figure}
bos@124: 
bos@124: After the \hgcmd{backout} command has completed, it leaves the new
bos@124: ``backout'' changeset as the parent of the working directory.
bos@124: \interaction{backout.manual.parents}
bos@124: Now we have two isolated sets of changes.
bos@124: \interaction{backout.manual.heads}
bos@124: 
bos@124: Let's think about what we expect to see as the contents of
bos@124: \filename{myfile} now.  The first change should be present, because
bos@124: we've never backed it out.  The second change should be missing, as
bos@124: that's the change we backed out.  Since the history graph shows the
bos@124: third change as a separate head, we \emph{don't} expect to see the
bos@124: third change present in \filename{myfile}.
bos@124: \interaction{backout.manual.cat}
bos@124: To get the third change back into the file, we just do a normal merge
bos@124: of our two heads.
bos@124: \interaction{backout.manual.merge}
bos@124: Afterwards, the graphical history of our repository looks like
bos@124: figure~\ref{fig:undo:backout-manual-merge}.
bos@124: 
bos@124: \begin{figure}[htb]
bos@124:   \centering
bos@124:   \grafix{undo-manual-merge}
bos@124:   \caption{Manually merging a backout change}
bos@124:   \label{fig:undo:backout-manual-merge}
bos@124: \end{figure}
bos@124: 
bos@126: \subsection{Why \hgcmd{backout} works as it does}
bos@124: 
bos@124: Here's a brief description of how the \hgcmd{backout} command works.
bos@124: \begin{enumerate}
bos@124: \item It ensures that the working directory is ``clean'', i.e.~that
bos@124:   the output of \hgcmd{status} would be empty.
bos@124: \item It remembers the current parent of the working directory.  Let's
bos@124:   call this changeset \texttt{orig}
bos@124: \item It does the equivalent of a \hgcmd{update} to sync the working
bos@124:   directory to the changeset you want to back out.  Let's call this
bos@124:   changeset \texttt{backout}
bos@124: \item It finds the parent of that changeset.  Let's call that
bos@124:   changeset \texttt{parent}.
bos@124: \item For each file that the \texttt{backout} changeset affected, it
bos@124:   does the equivalent of a \hgcmdargs{revert}{-r parent} on that file,
bos@124:   to restore it to the contents it had before that changeset was
bos@124:   committed.
bos@124: \item It commits the result as a new changeset.  This changeset has
bos@124:   \texttt{backout} as its parent.
bos@124: \item If you specify \hgopt{backout}{--merge} on the command line, it
bos@124:   merges with \texttt{orig}, and commits the result of the merge.
bos@124: \end{enumerate}
bos@124: 
bos@124: An alternative way to implement the \hgcmd{backout} command would be
bos@124: to \hgcmd{export} the to-be-backed-out changeset as a diff, then use
bos@124: the \cmdopt{patch}{--reverse} option to the \command{patch} command to
bos@124: reverse the effect of the change without fiddling with the working
bos@124: directory.  This sounds much simpler, but it would not work nearly as
bos@124: well.
bos@124: 
bos@124: The reason that \hgcmd{backout} does an update, a commit, a merge, and
bos@124: another commit is to give the merge machinery the best chance to do a
bos@124: good job when dealing with all the changes \emph{between} the change
bos@124: you're backing out and the current tip.  
bos@124: 
bos@124: If you're backing out a changeset that's~100 revisions back in your
bos@124: project's history, the chances that the \command{patch} command will
bos@124: be able to apply a reverse diff cleanly are not good, because
bos@124: intervening changes are likely to have ``broken the context'' that
bos@124: \command{patch} uses to determine whether it can apply a patch (if
bos@125: this sounds like gibberish, see \ref{sec:mq:patch} for a
bos@124: discussion of the \command{patch} command).  Also, Mercurial's merge
bos@124: machinery will handle files and directories being renamed, permission
bos@124: changes, and modifications to binary files, none of which
bos@124: \command{patch} can deal with.
bos@124: 
bos@126: \section{Changes that should never have been}
bos@126: \label{sec:undo:aaaiiieee}
bos@126: 
bos@126: Most of the time, the \hgcmd{backout} command is exactly what you need
bos@126: if you want to undo the effects of a change.  It leaves a permanent
bos@126: record of exactly what you did, both when committing the original
bos@126: changeset and when you cleaned up after it.
bos@126: 
bos@126: On rare occasions, though, you may find that you've committed a change
bos@126: that really should not be present in the repository at all.  For
bos@126: example, it would be very unusual, and usually considered a mistake,
bos@126: to commit a software project's object files as well as its source
bos@126: files.  Object files have almost no intrinsic value, and they're
bos@126: \emph{big}, so they increase the size of the repository and the amount
bos@126: of time it takes to clone or pull changes.
bos@126: 
bos@126: Before I discuss the options that you have if you commit a ``brown
bos@126: paper bag'' change (the kind that's so bad that you want to pull a
bos@126: brown paper bag over your head), let me first discuss some approaches
bos@126: that probably won't work.
bos@126: 
bos@126: Since Mercurial treats history as accumulative---every change builds
bos@126: on top of all changes that preceded it---you generally can't just make
bos@126: disastrous changes disappear.  The one exception is when you've just
bos@126: committed a change, and it hasn't been pushed or pulled into another
bos@126: repository.  That's when you can safely use the \hgcmd{rollback}
bos@126: command, as I detailed in section~\ref{sec:undo:rollback}.
bos@126: 
bos@126: After you've pushed a bad change to another repository, you
bos@126: \emph{could} still use \hgcmd{rollback} to make your local copy of the
bos@126: change disappear, but it won't have the consequences you want.  The
bos@126: change will still be present in the remote repository, so it will
bos@126: reappear in your local repository the next time you pull.
bos@126: 
bos@126: If a situation like this arises, and you know which repositories your
bos@126: bad change has propagated into, you can \emph{try} to get rid of the
bos@126: changeefrom \emph{every} one of those repositories.  This is, of
bos@126: course, not a satisfactory solution: if you miss even a single
bos@126: repository while you're expunging, the change is still ``in the
bos@126: wild'', and could propagate further.
bos@126: 
bos@126: If you've committed one or more changes \emph{after} the change that
bos@126: you'd like to see disappear, your options are further reduced.
bos@126: Mercurial doesn't provide a way to ``punch a hole'' in history,
bos@126: leaving changesets intact.
bos@126: 
bos@129: XXX This needs filling out.  The \texttt{hg-replay} script in the
bos@129: \texttt{examples} directory works, but doesn't handle merge
bos@129: changesets.  Kind of an important omission.
bos@129: 
bos@201: \subsection{Protect yourself from ``escaped'' changes}
bos@201: 
bos@201: If you've committed some changes to your local repository and they've
bos@201: been pushed or pulled somewhere else, this isn't necessarily a
bos@201: disaster.  You can protect yourself ahead of time against some classes
bos@201: of bad changeset.  This is particularly easy if your team usually
bos@201: pulls changes from a central repository.
bos@201: 
bos@201: By configuring some hooks on that repository to validate incoming
bos@201: changesets (see chapter~\ref{chap:hook}), you can automatically
bos@201: prevent some kinds of bad changeset from being pushed to the central
bos@201: repository at all.  With such a configuration in place, some kinds of
bos@201: bad changeset will naturally tend to ``die out'' because they can't
bos@201: propagate into the central repository.  Better yet, this happens
bos@201: without any need for explicit intervention.
bos@201: 
bos@201: For instance, an incoming change hook that verifies that a changeset
bos@201: will actually compile can prevent people from inadvertantly ``breaking
bos@201: the build''.
bos@201: 
bos@130: \section{Finding the source of a bug}
bos@200: \label{sec:undo:bisect}
bos@130: 
bos@130: While it's all very well to be able to back out a changeset that
bos@130: introduced a bug, this requires that you know which changeset to back
bos@130: out.  Mercurial provides an invaluable extension, called
bos@130: \hgext{bisect}, that helps you to automate this process and accomplish
bos@130: it very efficiently.
bos@130: 
bos@130: The idea behind the \hgext{bisect} extension is that a changeset has
bos@130: introduced some change of behaviour that you can identify with a
bos@130: simple binary test.  You don't know which piece of code introduced the
bos@130: change, but you know how to test for the presence of the bug.  The
bos@130: \hgext{bisect} extension uses your test to direct its search for the
bos@130: changeset that introduced the code that caused the bug.
bos@130: 
bos@130: Here are a few scenarios to help you understand how you might apply this
bos@130: extension.
bos@130: \begin{itemize}
bos@130: \item The most recent version of your software has a bug that you
bos@130:   remember wasn't present a few weeks ago, but you don't know when it
bos@130:   was introduced.  Here, your binary test checks for the presence of
bos@130:   that bug.
bos@130: \item You fixed a bug in a rush, and now it's time to close the entry
bos@130:   in your team's bug database.  The bug database requires a changeset
bos@130:   ID when you close an entry, but you don't remember which changeset
bos@130:   you fixed the bug in.  Once again, your binary test checks for the
bos@130:   presence of the bug.
bos@130: \item Your software works correctly, but runs~15\% slower than the
bos@130:   last time you measured it.  You want to know which changeset
bos@130:   introduced the performance regression.  In this case, your binary
bos@130:   test measures the performance of your software, to see whether it's
bos@130:   ``fast'' or ``slow''.
bos@130: \item The sizes of the components of your project that you ship
bos@130:   exploded recently, and you suspect that something changed in the way
bos@130:   you build your project.
bos@130: \end{itemize}
bos@130: 
bos@130: From these examples, it should be clear that the \hgext{bisect}
bos@130: extension is not useful only for finding the sources of bugs.  You can
bos@130: use it to find any ``emergent property'' of a repository (anything
bos@130: that you can't find from a simple text search of the files in the
bos@130: tree) for which you can write a binary test.
bos@130: 
bos@130: We'll introduce a little bit of terminology here, just to make it
bos@130: clear which parts of the search process are your responsibility, and
bos@130: which are Mercurial's.  A \emph{test} is something that \emph{you} run
bos@130: when \hgext{bisect} chooses a changeset.  A \emph{probe} is what
bos@130: \hgext{bisect} runs to tell whether a revision is good.  Finally,
bos@130: we'll use the word ``bisect'', as both a noun and a verb, to stand in
bos@130: for the phrase ``search using the \hgext{bisect} extension''.
bos@130: 
bos@130: One simple way to automate the searching process would be simply to
bos@130: probe every changeset.  However, this scales poorly.  If it took ten
bos@130: minutes to test a single changeset, and you had 10,000 changesets in
bos@130: your repository, the exhaustive approach would take on average~35
bos@130: \emph{days} to find the changeset that introduced a bug.  Even if you
bos@130: knew that the bug was introduced by one of the last 500 changesets,
bos@130: and limited your search to those, you'd still be looking at over 40
bos@130: hours to find the changeset that introduced your bug.
bos@130: 
bos@130: What the \emph{bisect} extension does is use its knowledge of the
bos@130: ``shape'' of your project's revision history to perform a search in
bos@130: time proportional to the \emph{logarithm} of the number of changesets
bos@130: to check (the kind of search it performs is called a dichotomic
bos@130: search).  With this approach, searching through 10,000 changesets will
bos@130: take less than two hours, even at ten minutes per test.  Limit your
bos@130: search to the last 500 changesets, and it will take less than an hour.
bos@130: 
bos@130: The \hgext{bisect} extension is aware of the ``branchy'' nature of a
bos@130: Mercurial project's revision history, so it has no problems dealing
bos@130: with branches, merges, or multiple heads in a repoository.  It can
bos@130: prune entire branches of history with a single probe, which is how it
bos@130: operates so efficiently.
bos@130: 
bos@130: \subsection{Using the \hgext{bisect} extension}
bos@130: 
bos@130: Here's an example of \hgext{bisect} in action.  To keep the core of
bos@130: Mercurial simple, \hgext{bisect} is packaged as an extension; this
bos@130: means that it won't be present unless you explicitly enable it.  To do
bos@130: this, edit your \hgrc\ and add the following section header (if it's
bos@130: not already present):
bos@130: \begin{codesample2}
bos@130:   [extensions]
bos@130: \end{codesample2}
bos@130: Then add a line to this section to enable the extension:
bos@130: \begin{codesample2}
bos@130:   hbisect =
bos@130: \end{codesample2}
bos@130: \begin{note}
bos@130:   That's right, there's a ``\texttt{h}'' at the front of the name of
bos@130:   the \hgext{bisect} extension.  The reason is that Mercurial is
bos@130:   written in Python, and uses a standard Python package called
bos@130:   \texttt{bisect}.  If you omit the ``\texttt{h}'' from the name
bos@130:   ``\texttt{hbisect}'', Mercurial will erroneously find the standard
bos@130:   Python \texttt{bisect} package, and try to use it as a Mercurial
bos@130:   extension.  This won't work, and Mercurial will crash repeatedly
bos@130:   until you fix the spelling in your \hgrc.  Ugh.
bos@130: \end{note}
bos@130: 
bos@130: Now let's create a repository, so that we can try out the
bos@130: \hgext{bisect} extension in isolation.
bos@130: \interaction{bisect.init}
bos@130: We'll simulate a project that has a bug in it in a simple-minded way:
bos@130: create trivial changes in a loop, and nominate one specific change
bos@130: that will have the ``bug''.  This loop creates 50 changesets, each
bos@130: adding a single file to the repository.  We'll represent our ``bug''
bos@130: with a file that contains the text ``i have a gub''.
bos@130: \interaction{bisect.commits}
bos@130: 
bos@130: The next thing that we'd like to do is figure out how to use the
bos@130: \hgext{bisect} extension.  We can use Mercurial's normal built-in help
bos@130: mechanism for this.
bos@130: \interaction{bisect.help}
bos@130: 
bos@130: The \hgext{bisect} extension works in steps.  Each step proceeds as follows.
bos@130: \begin{enumerate}
bos@130: \item You run your binary test.
bos@130:   \begin{itemize}
bos@130:   \item If the test succeeded, you tell \hgext{bisect} by running the
bos@130:     \hgcmdargs{bisect}{good} command.
bos@130:   \item If it failed, use the \hgcmdargs{bisect}{bad} command to let
bos@130:     the \hgext{bisect} extension know.
bos@130:   \end{itemize}
bos@130: \item The extension uses your information to decide which changeset to
bos@130:   test next.
bos@130: \item It updates the working directory to that changeset, and the
bos@130:   process begins again.
bos@130: \end{enumerate}
bos@130: The process ends when \hgext{bisect} identifies a unique changeset
bos@130: that marks the point where your test transitioned from ``succeeding''
bos@130: to ``failing''.
bos@130: 
bos@130: To start the search, we must run the \hgcmdargs{bisect}{init} command.
bos@130: \interaction{bisect.search.init}
bos@130: 
bos@130: In our case, the binary test we use is simple: we check to see if any
bos@130: file in the repository contains the string ``i have a gub''.  If it
bos@130: does, this changeset contains the change that ``caused the bug''.  By
bos@130: convention, a changeset that has the property we're searching for is
bos@130: ``bad'', while one that doesn't is ``good''.
bos@130: 
bos@130: Most of the time, the revision to which the working directory is
bos@130: synced (usually the tip) already exhibits the problem introduced by
bos@130: the buggy change, so we'll mark it as ``bad''.
bos@130: \interaction{bisect.search.bad-init}
bos@130: 
bos@130: Our next task is to nominate a changeset that we know \emph{doesn't}
bos@130: have the bug; the \hgext{bisect} extension will ``bracket'' its search
bos@130: between the first pair of good and bad changesets.  In our case, we
bos@130: know that revision~10 didn't have the bug.  (I'll have more words
bos@130: about choosing the first ``good'' changeset later.)
bos@130: \interaction{bisect.search.good-init}
bos@130: 
bos@130: Notice that this command printed some output.
bos@130: \begin{itemize}
bos@130: \item It told us how many changesets it must consider before it can
bos@130:   identify the one that introduced the bug, and how many tests that
bos@130:   will require.
bos@130: \item It updated the working directory to the next changeset to test,
bos@130:   and told us which changeset it's testing.
bos@130: \end{itemize}
bos@130: 
bos@130: We now run our test in the working directory.  We use the
bos@130: \command{grep} command to see if our ``bad'' file is present in the
bos@130: working directory.  If it is, this revision is bad; if not, this
bos@130: revision is good.
bos@131: \interaction{bisect.search.step1}
bos@130: 
bos@130: This test looks like a perfect candidate for automation, so let's turn
bos@130: it into a shell function.
bos@131: \interaction{bisect.search.mytest}
bos@130: We can now run an entire test step with a single command,
bos@130: \texttt{mytest}.
bos@131: \interaction{bisect.search.step2}
bos@130: A few more invocations of our canned test step command, and we're
bos@130: done.
bos@131: \interaction{bisect.search.rest}
bos@130: 
bos@130: Even though we had~40 changesets to search through, the \hgext{bisect}
bos@130: extension let us find the changeset that introduced our ``bug'' with
bos@130: only five tests.  Because the number of tests that the \hgext{bisect}
bos@130: extension grows logarithmically with the number of changesets to
bos@130: search, the advantage that it has over the ``brute force'' search
bos@130: approach increases with every changeset you add.
bos@130: 
bos@130: \subsection{Cleaning up after your search}
bos@130: 
bos@130: When you're finished using the \hgext{bisect} extension in a
bos@130: repository, you can use the \hgcmdargs{bisect}{reset} command to drop
bos@130: the information it was using to drive your search.  The extension
bos@130: doesn't use much space, so it doesn't matter if you forget to run this
bos@130: command.  However, \hgext{bisect} won't let you start a new search in
bos@130: that repository until you do a \hgcmdargs{bisect}{reset}.
bos@131: \interaction{bisect.search.reset}
bos@130: 
bos@130: \section{Tips for finding bugs effectively}
bos@130: 
bos@130: \subsection{Give consistent input}
bos@130: 
bos@130: The \hgext{bisect} extension requires that you correctly report the
bos@130: result of every test you perform.  If you tell it that a test failed
bos@130: when it really succeeded, it \emph{might} be able to detect the
bos@130: inconsistency.  If it can identify an inconsistency in your reports,
bos@130: it will tell you that a particular changeset is both good and bad.
bos@130: However, it can't do this perfectly; it's about as likely to report
bos@130: the wrong changeset as the source of the bug.
bos@130: 
bos@130: \subsection{Automate as much as possible}
bos@130: 
bos@130: When I started using the \hgext{bisect} extension, I tried a few times
bos@130: to run my tests by hand, on the command line.  This is an approach
bos@130: that I, at least, am not suited to.  After a few tries, I found that I
bos@130: was making enough mistakes that I was having to restart my searches
bos@130: several times before finally getting correct results.  
bos@130: 
bos@130: My initial problems with driving the \hgext{bisect} extension by hand
bos@130: occurred even with simple searches on small repositories; if the
bos@130: problem you're looking for is more subtle, or the number of tests that
bos@130: \hgext{bisect} must perform increases, the likelihood of operator
bos@130: error ruining the search is much higher.  Once I started automating my
bos@130: tests, I had much better results.
bos@130: 
bos@130: The key to automated testing is twofold:
bos@130: \begin{itemize}
bos@130: \item always test for the same symptom, and
bos@130: \item always feed consistent input to the \hgcmd{bisect} command.
bos@130: \end{itemize}
bos@130: In my tutorial example above, the \command{grep} command tests for the
bos@130: symptom, and the \texttt{if} statement takes the result of this check
bos@130: and ensures that we always feed the same input to the \hgcmd{bisect}
bos@130: command.  The \texttt{mytest} function marries these together in a
bos@130: reproducible way, so that every test is uniform and consistent.
bos@130: 
bos@130: \subsection{Check your results}
bos@130: 
bos@130: Because the output of a \hgext{bisect} search is only as good as the
bos@130: input you give it, don't take the changeset it reports as the
bos@130: absolute truth.  A simple way to cross-check its report is to manually
bos@130: run your test at each of the following changesets:
bos@130: \begin{itemize}
bos@130: \item The changeset that it reports as the first bad revision.  Your
bos@130:   test should still report this as bad.
bos@130: \item The parent of that changeset (either parent, if it's a merge).
bos@130:   Your test should report this changeset as good.
bos@130: \item A child of that changeset.  Your test should report this
bos@130:   changeset as bad.
bos@130: \end{itemize}
bos@130: 
bos@130: \subsection{Beware interference between bugs}
bos@130: 
bos@130: It's possible that your search for one bug could be disrupted by the
bos@130: presence of another.  For example, let's say your software crashes at
bos@130: revision 100, and worked correctly at revision 50.  Unknown to you,
bos@130: someone else introduced a different crashing bug at revision 60, and
bos@130: fixed it at revision 80.  This could distort your results in one of
bos@130: several ways.
bos@130: 
bos@130: It is possible that this other bug completely ``masks'' yours, which
bos@130: is to say that it occurs before your bug has a chance to manifest
bos@130: itself.  If you can't avoid that other bug (for example, it prevents
bos@130: your project from building), and so can't tell whether your bug is
bos@130: present in a particular changeset, the \hgext{bisect} extension cannot
bos@130: help you directly.  Instead, you'll need to manually avoid the
bos@130: changesets where that bug is present, and do separate searches
bos@130: ``around'' it.
bos@130: 
bos@130: A different problem could arise if your test for a bug's presence is
wbunaarfubss@248: not specific enough.  If you check for ``my program crashes'', then
bos@130: both your crashing bug and an unrelated crashing bug that masks it
bos@130: will look like the same thing, and mislead \hgext{bisect}.
bos@130: 
bos@130: \subsection{Bracket your search lazily}
bos@130: 
bos@130: Choosing the first ``good'' and ``bad'' changesets that will mark the
bos@130: end points of your search is often easy, but it bears a little
bos@130: discussion neverthheless.  From the perspective of \hgext{bisect}, the
bos@130: ``newest'' changeset is conventionally ``bad'', and the older
bos@130: changeset is ``good''.
bos@130: 
bos@130: If you're having trouble remembering when a suitable ``good'' change
bos@130: was, so that you can tell \hgext{bisect}, you could do worse than
bos@130: testing changesets at random.  Just remember to eliminate contenders
bos@130: that can't possibly exhibit the bug (perhaps because the feature with
bos@130: the bug isn't present yet) and those where another problem masks the
bos@130: bug (as I discussed above).
bos@130: 
bos@130: Even if you end up ``early'' by thousands of changesets or months of
bos@130: history, you will only add a handful of tests to the total number that
bos@130: \hgext{bisect} must perform, thanks to its logarithmic behaviour.
bos@130: 
bos@121: %%% Local Variables: 
bos@121: %%% mode: latex
bos@121: %%% TeX-master: "00book"
bos@121: %%% End: 




