@node Working with source code
@chapter Working with source code
+@warning{New contributors should read @ref{Quick start}, and in
+particular @ref{lily-git}, instead of this chapter.}
+
+Advanced contributors will find this material quite useful,
+particularly if they are working on major new features.
@menu
-* Using lily-git::
+* Manually installing lily-git.tcl::
* Starting with Git::
* Basic Git procedures::
* Advanced Git procedures::
@end menu
-@node Using lily-git
-@section Using lily-git
+@node Manually installing lily-git.tcl
+@section Manually installing lily-git.tcl
+
+We have created an easy-to-use GUI to simplify git for new
+contributors. If you are comfortable with the command-line, then
+skip ahead to @ref{Starting with Git}.
+
+@warning{These instructions are only for people who are @emph{not}
+using @ref{LilyDev}.}
+
+@c there's some duplication in this section with stuff covered in
+@c Quick Start, but moving it into a macro inside included/ would
+@c be getting a bit icky. -gp
+
+@enumerate
+@item
+If you haven't already, download and install Git.
+
+@itemize
+
+@item Windows users: download the @code{.exe} file labeled
+@qq{Full installer for official Git} from:
+
+@example
+@uref{https://git-for-windows.github.io/}
+@end example
+
+@item Other operating systems: either install @command{git} with
+your package manager, or download it from the @qq{Binaries}
+section of:
+
+@example
+@uref{http://git-scm.com/download}
+@end example
+
+@end itemize
+
+
+@item
+Download the @command{lily-git.tcl} script from:
+
+@c don't change the cgit link below to gitweb; gitweb uses
+@c long filenames like "scripts_auxiliar_lily-git.tcl"
+
+@smallexample
+@uref{http://git.sv.gnu.org/cgit/lilypond.git/plain/scripts/auxiliar/lily-git.tcl}
+@end smallexample
+
+@item
+To run the program from the command line, navigate to the
+directory containing @command{lily-git.tcl} and enter:
+
+@example
+wish lily-git.tcl
+@end example
+
+@item
+Click on the @qq{Get source} button.
+
+This will create a directory called @file{lilypond-git/} within
+your home directory, and will download the source code into that
+directory (around 150@tie{}Mb). When the process is finished, the
+@qq{Command output} window will display @qq{Done}, and the button
+label will change to say @qq{Update source}.
+
+@item
+Navigate to the @file{lilypond-git/} directory to view the source
+files.
+
+@end enumerate
+@warning{Throughout the rest of this manual, most command-line
+input should be entered from @file{$LILYPOND_GIT}. This is
+referred to as the @emph{top source directory}.}
-FIXME: Add instructions for using @command{lily-git} here.
+Further instructions are in @ref{How to use lily-git}.
@node Starting with Git
@section Starting with Git
-
Using the Git program directly (as opposed to using the
-@command{lily-git} GUI) allows you to have much greater control
+@command{lily-git.tcl} GUI) allows you to have much greater control
over the contributing process. You should consider using Git if
you want to work on complex projects, or if you want to work on
multiple projects concurrently.
@menu
* Setting up::
+* Git for the impatient::
+* Other repositories::
* Downloading remote branches::
@end menu
@node Setting up
@subsection Setting up
-
-FIXME: Remove this note if incorporating Windows instructions
-throughout this section:
-
@warning{These instructions assume that you are using the
command-line version of Git 1.5 or higher. Windows users should
skip to @ref{Git on Windows}.}
@node Installing Git
@unnumberedsubsubsec Installing Git
-
If you are using a Unix-based machine, the easiest way to download
and install Git is through a package manager such as @command{rpm}
-or @command{apt-get}---the installation is generally automatic.
+or @command{apt-get} -- the installation is generally automatic.
The only required package is (usually) called @command{git-core},
although some of the auxiliary @command{git@var{*}} packages are
also useful (such as @command{gitk}).
(@uref{http://git-scm.com/}) for downloadable binaries and
tarballs.
-FIXME: add Windows installation instructions (or @@ref@{Git on
-Windows@}).
-
@node Initializing a repository
@unnumberedsubsubsec Initializing a repository
-
-Once Git is installed, you'll need to create a new directory where
-your initial repository will be stored (the example below uses
-@file{~/lilypond-git/}, where @code{~} represents your home
-directory). Run @command{git@tie{}init} from within the new
-directory to initialize an empty repository:
+Once Git is installed, get a copy of the source code:
@example
-mkdir ~/lilypond-git/; cd ~/lilypond-git/
-git init
+git clone git://git.sv.gnu.org/lilypond.git ~/lilypond-git
@end example
+The above command will put the it in @file{~/lilypond-git}, where
+@code{~} represents your home directory.
+
@subsubheading Technical details
-This creates (within the @file{~/lilypond-git/} directory) a
+This creates (within the @file{$LILYPOND_GIT} directory) a
subdirectory called @file{.git/}, which Git uses to keep track of
changes to the repository, among other things. Normally you don't
need to access it, but it's good to know it's there.
@warning{Throughout the rest of this manual, all command-line
input should be entered from the top directory of the Git
-repository being discussed (eg. @file{~/lilypond-git/}). This is
-referred to as a @emph{top source directory}.}
+repository being discussed (eg. @file{$LILYPOND_GIT}). This is
+referred to as the @emph{top source directory}.}
-Before downloading a copy of the main LilyPond repository, you
+Before working with the copy of the main LilyPond repository, you
should configure some basic settings with the
@command{git@tie{}config} command. Git allows you to set both
global and repository-specific options.
To configure settings that affect all repositories, use the
-@command{--global} command line option. For example, the first
+@option{--global} command line option. For example, the first
two options that you should always set are your @var{name} and
@var{email}, since Git needs these to keep track of commit
authors:
git config --global core.editor @var{nano}
@end example
-FIXME: Add instructions for changing the editor on Windows, which
-is a little different, I think. -mp
+Finally, and in some ways most importantly, let's make sure that
+we can easily see the state of our working copy, without the need
+of typing @code{git status} repeatedly. If you're not using
+LilyDev, add the following lines to your @file{~/.bashrc}:
+
+@verbatim
+export PS1="\u@\h \w\$(__git_ps1)$ "
+export GIT_PS1_SHOWDIRTYSTATE=true
+export GIT_PS1_SHOWUNTRACKEDFILES=true
+export GIT_PS1_SHOWUPSTREAM=auto
+@end verbatim
+
+The first line will show the branch we're on. The other lines
+will use some symbols next to the branch name to indicate some
+kind of state. @qq{*} means that there are unstaged changes,
+@qq{+} indicates staged changes; if there are untracked files,
+a @qq{%} will appear. Finally, we can also see if our HEAD is
+behind (@qq{<}) or ahead (@qq{>}) of its upstream, and if they
+have diverged (@qq{<>}) or they are synced (@qq{=}).
+
+You may need to install the additional @code{bash-completion}
+package, but it is definitely worth it. After installation
+you must log out, and then log back in again to enable it.
+
@subsubheading Technical details
@end example
Using the @command{git@tie{}config} command @emph{without} the
-@command{--global} option configures repository-specific settings,
+@option{--global} option configures repository-specific settings,
which are stored in the file @file{.git/config}. This file is
created when a repository is initialized (using
@command{git@tie{}init}), and by default contains these lines:
in this manual.
+@node Git for the impatient
+@subsection Git for the impatient
+
+@advanced{The intent of this subsection is to get you working on lilypond as
+soon as possible. If you want to learn about git, go read
+@ref{Other Git documentation}.
+@*
+Also, these instructions are designed to eliminate the most common
+problems we have found in using git. If you already know git and
+have a different way of working, great! Feel free to ignore the
+advice in this subsection.}
+
+
+Ok, so you've been using @command{lily-git.tcl} for a while, but
+it's time to take the next step. Since our review process delays
+patches by 60-120 hours, and you want to be able to work on other
+stuff while your previous work is getting reviewed, you're going
+to use @strong{branches}.
+
+You can think of a branch as being a separate copy of the source
+code. But don't worry about it.
+
+@subsubheading Start work: make a new branch
+
+Let's pretend you want to add a section to the Contributor's Guide
+about using branches.
+
+Start by updating the repository, then making a new branch. Call
+the branch anything you want as long as the name starts with
+@code{dev/}. Branch names that don't begin with @code{dev/} are
+reserved for special things in lilypond.
+
+@example
+git checkout master
+git pull -r origin master
+git branch dev/cg
+@end example
+
+@subsubheading Switch to that branch
+
+Nothing has happened to the files yet. Let's change into the new
+branch. You can think of this as @qq{loading a file}, although in
+this case it's really @qq{loading a directory and subdirectories
+full of files}.
+
+@example
+git checkout dev/cg
+@end example
+
+Your prompt now shows you that you're on the other branch:
+
+@example
+gperciva@@LilyDev:~/lilypond-git (dev/cg)$
+@end example
+
+To be able to manage multiple lilypond issues at once, you'll need to switch
+branches. You should have each lilypond issue on a separate branch.
+Switching branches is easy:
+
+@example
+git checkout master
+git checkout origin/staging
+git checkout origin/release/unstable
+git checkout dev/cg
+@end example
+
+Branches that begin with @code{origin/} are part of the remote repository,
+rather than your local repository, so when you check them out you get a
+temporary local branch. You should never make changes directly on a
+branch beginning with @code{origin/}. You get changes into the remote
+repository by making them in local branches, and then pushing them to
+@code{origin/staging} as described below.
+
+@subsubheading Make your changes
+
+Edit files, then commit them.
+
+@example
+git commit -a
+@end example
+
+
+Remember how I said that switching to a branch was like
+@qq{loading a directory}? Well, you've just @qq{saved a
+directory}, so that you can @qq{load} it later.
+
+@advanced{If you have used @command{cvs} or @command{svn}, you may
+be very confused: those programs use @qq{commit} to mean
+@qq{upload my changes to the shared source repository}.
+Unfortunately, just to be different, @w{@command{git commit}}
+means @qq{save my changes to the files}.}
+
+When you create a new file, you need to add it to git, then commit it:
+
+@example
+git add input/regression/avoid-crash-on-condition.ly
+git commit -a
+@end example
+
+
+Edit more files. Commit them again. Edit yet more files, commit
+them again. Go eat dinner. Switch to @code{master} so you can
+play with the latest changes from other developers. Switch back
+to your branch and edit some more. Commit those changes.
+
+At this stage, don't worry about how many commits you have.
+
+
+@subsubheading Save commits to external files
+
+Branches are nerve-wracking until you get used to them. You can
+save your hard work as individual @file{.patch} files. Be sure to
+commit your changes first.
+
+@example
+git commit -a
+git format-patch master
+@end example
+
+I personally have between 4 and 20 of those files saved in a
+special folder at any point in time. Git experts might laugh as
+that behavior, but I feel a @emph{lot} better knowing that I've
+got those backups.
+
+
+@subsubheading Prepare your branch for review
+
+After committing, you can update your branch with the latest master:
+
+@example
+git commit -a
+git checkout master
+git pull -r origin master
+git checkout dev/cg
+git rebase master
+@end example
+
+
+Due to the speed of lilypond development, sometimes
+@code{master} has changed so much that your branch can no
+longer be applied to it. In that happens, you will have a merge
+conflict. Stop for a moment to either cry or have a stiff drink,
+then proceed to @ref{Merge conflicts}.
+
+
+@subsubheading Upload your branch
+
+Finally, you're finished your changes. Time to upload for review.
+Make sure that you're on your branch, then upload:
+
+@example
+git checkout dev/cg
+git-cl upload master
+@end example
+
+
+@subsubheading Wait for reviews
+
+While you're waiting for a countdown and reviews, go back to
+master, make a @code{dev/doc-beams} branch, and start adding doc
+suggestions from issue 12345 from the tracker. Or make a
+@code{dev/page-breaks} and fix bug in page breaking. Or whatever.
+Don't worry, your @code{dev/cg} is safe.
+
+
+@subsubheading Combining commits (optional unless you have broken commits)
+
+Does the history of your branch look good?
+
+@example
+gitk
+@end example
+
+If you have a lot of commits on your branch, you might want to
+combine some of them. Alternately, you may like your commits, but
+want to edit the commit messages.
+
+@example
+git rebase -i master
+@end example
+
+Follow instructions on the screen.
+
+@warning{This step gives you the power to completely lose your
+work. Make a backup of your commits by saving them to
+@file{.patch} files before playing with this. If you do lose
+your work, don't despair. You can get it back by using @code{git reflog}.
+The use of @code{git reflog} is not covered here.}
+
+@warning{If any of the commits on your branch represent partial work that will
+not pass @var{make && make doc}, you @strong{must} squash these
+commits into a working commit. Otherwise, your push will break staging
+and will not be able to be merged to master. In general, you will
+be safer to have one commit per push.}
+
+
+@subsubheading Push to staging
+
+When you've got the coveted @code{Patch-push} status, time to
+prepare your upload:
+
+@example
+git fetch
+git rebase origin/staging dev/cg~0
+gitk HEAD
+@end example
+
+@warning{Do not skip the @command{gitk} step; a quick 5-second
+check of the visual history can save a great deal of frustration
+later on. You should see a set of your commits that are ahead of
+@code{origin/staging}, with no label for the top commit -- only a
+SHA1 id.}
+
+@warning{If @code{origin/staging} and @code{origin/master} are the
+same commit, your branch (@code{dev/cg} in the example) will also
+be at the top of the @code{gitk} tree. This is normal.}
+
+If everything looks good, push it:
+
+@example
+git push origin HEAD:staging
+@end example
+
+Then change back to your working branch:
+
+@example
+git checkout dev/cg
+@end example
+
+@warning{It is a best practice to avoid rebasing any of your branches
+to @code{origin/staging}. If @code{origin/staging} is broken, it
+will be deleted and rebuilt. If you have rebased one of your branches
+to @code{origin/staging}, the broken commits can end up in your branch.
+The commands given above do the rebase on a temporary branch, and avoid
+changing your working branch.}
+
+
+@subsubheading Delete your branch (safe)
+
+After a few hours, if there's nothing wrong with your branch, it
+should be automatically moved to @code{origin/master}. Update,
+then try removing your branch:
+
+@example
+git checkout master
+git pull -r origin master
+git branch -d dev/cg
+@end example
+
+The last command will fail if the contents of @code{dev/cg} are
+not present in @code{origin/master}.
+
+
+@subsubheading Delete your branch (UNSAFE)
+
+@c don't give explicit commands here -- this is too dangerous to copy and paste
+Sometimes everything goes wrong. If you want to remove a branch even though
+it will cause your work to be lost (that is, if the contents of @code{dev/cg}
+are @strong{not} present in master), follow the instructions in @qq{Delete
+your branch (safe)}, but replace the @code{-d} on the final line with
+a @code{-D}.
+
+
+@node Other repositories
+@subsection Other repositories
+
+We have a few other code repositories.
+
+@menu
+* lilypond-extra::
+* Grand Unified Builder (GUB)::
+* LilyPad::
+* yet more repositories::
+@end menu
+
+@node lilypond-extra
+@unnumberedsubsubsec lilypond-extra
+
+There is a separate repository for general administrative scripts,
+as well as pictures and media files for the website. People
+interested in working on the website should download this
+repository, and set their @code{$LILYPOND_WEB_MEDIA_GIT}
+environment variable to point to that repository.
+
+@example
+@uref{https://github.com/gperciva/lilypond-extra}
+@end example
+
+To configure an environment variable in bash (the default for most
+GNU/Linux distributions),
+
+@example
+export LILYPOND_WEB_MEDIA_GIT=$HOME/dir/of/lilypond-extra/
+@end example
+
+Be aware that @code{lilypond-extra} is the definitive source for some binary
+files - in particular PDF versions of papers concerning LilyPond. To add
+further PDFs of this sort, all that is necessary is to add the PDF to
+@code{lilypond-extra} and then add a reference to it in the documentation. The
+file will then be copied to the website when @code{make website} is run.
+
+However, pictures that are also used in the documentation build are mastered in
+the main git repository. If any of these is changed, it should be updated in
+git, and then the updates copied to @code{lilypond-extra}.
+
+@node Grand Unified Builder (GUB)
+@unnumberedsubsubsec Grand Unified Builder (GUB)
+
+Another item of interest might be the Grand Unified Builder, our
+cross-platform building tool. Since it is used by other projects as
+well, it is not stored in our gub repository. For more info, see
+@uref{http://lilypond.org/gub}.
+
+There are two locations for this repository: the version being used to
+build lilypond, which is at
+
+@example
+@uref{http://github.com/gperciva/gub}
+@end example
+
+and the original version by Jan Nieuwenhuizen, kept at
+
+@example
+@uref{http://github.com/janneke/gub}
+@end example
+
+
+@node LilyPad
+@unnumberedsubsubsec LilyPad
+
+Our binary releases on MacOS X and Windows contain a lightweight
+text editor.
+
+To make any modifications the Windows editor, you will need to do the
+following:
+
+@enumerate
+@item
+Clone the git repository from @code{https://github.com/gperciva/lilypad}
+
+@item
+Make changes to the source, and check it compiles. In a Windows environment
+@code{MinGW} provides both a @code{Git} installation and a @code{gcc}
+compiler. This can be obtained from @code{http://www.mingw.org/}
+
+@item
+Update the version which is contained in the @file{rsrc.rc}. Check
+this compiles, too.
+
+@item
+Commit the changes with an informative commit message.
+
+@item
+Push the changes to github. You will need to use syntax similiar to this:
+
+@example
+git push https://UserName@@github.com/gperciva/lilypad.git
+@end example
+
+You will need to have push access to the git repository for this to be
+successful.
+
+@item
+Make a tarball of the source code to be used by GUB by pulling the updated
+repository from GitHub. Ensure that the tarball has the correct Version
+number.
+
+@item
+Copy the tarball to @code{http://lilypond.org/downloads/gub-sources/lilypad/}.
+You will need to have SSH access to @code{lilypond.org}. If you do not, contact
+the Release Manager via the lilypond-devel mailing list.
+
+@item
+Update GUB to make it use the new tarball by editing
+@file{gub/specs/lilypad.py} and changing the @code{source =} line to point to
+the new source.
+
+@item
+Push this updated @file{lilypad.py} version to the GUB repository on GitHub.
+
+@item
+Test the changes with a new GUB compile.
+
+@end enumerate
+
+@node yet more repositories
+@unnumberedsubsubsec yet more repositories
+
+There are a few other repositories floating around, which will
+hopefully be documented in the near future.
+
+
@node Downloading remote branches
@subsection Downloading remote branches
+@warning{contains obsolete + misleading info}
@menu
* Organization of remote branches::
and the website. Generally, the @code{master} branch is expected
to compile successfully.
-The @code{lilypond/translation} branch is a side branch that
+The @code{translation} branch is a side branch that
allows translators to work without needing to worry about
compilation problems. Periodically, the Translation Meister
(after verifying that it doesn't break compilation), will
-@emph{merge} this branch back into @code{master} to incorporate
+@emph{merge} this branch into @code{staging} to incorporate
recent translations. Similarly, the @code{master} branch is
-usually merged into the @code{lilypond/translation} branch after
+usually merged into the @code{translation} branch after
significant changes to the English documentation. See
@ref{Translating the documentation} for details.
@node Downloading individual branches
@unnumberedsubsubsec Downloading individual branches
+@warning{obsolete, should be deleted!}
+
Once you have initialized an empty Git repository on your system
(see @ref{Initializing a repository}), you can download a remote
origin git://git.sv.gnu.org/lilypond.git/
@end example
-To download the @code{lilypond/translation} branch, enter:
+To download the @code{translation} branch, enter:
@example
-git remote add -ft lilypond/translation -m \
- lilypond/translation origin git://git.sv.gnu.org/lilypond.git/
+git remote add -ft translation -m \
+ translation origin git://git.sv.gnu.org/lilypond.git/
@end example
The @command{git@tie{}remote@tie{}add} process could take up to
@noindent
where @code{@var{branch}} is the name of your tracking branch,
-either @code{master} or @code{lilypond/translation}.
+either @code{master} or @code{translation}.
Git will issue some warnings; this is normal:
@end example
By now the source files should be accessible---you should be able
-to edit any files in the @file{lilypond-git/} directory using a
+to edit any files in the @file{$LILYPOND_GIT} directory using a
text editor of your choice. But don't start just yet! Before
editing any source files, learn how to keep your changes organized
and prevent problems later---read @ref{Basic Git procedures}.
@item @code{stable/XYZ}:
The branches are kept for archival reasons.
-@end itemize
+@item @code{archive/XYZ}:
+The branches are kept for archival reasons.
-Another item of interest might be the Grand Unified Builder, our
-cross-platform building tool. Since it is used by projects as
-well, it is not stored in our gub repository. For more info, see
-@uref{http://lilypond.org/gub}. The git location is
-@uref{http://github.com/janneke/gub}.
+@end itemize
@node Basic Git procedures
* The Git contributor's cycle::
* Pulling and rebasing::
* Using local branches::
-* Commits and patches::
+* Commits::
+* Patches::
+* Uploading a patch for review::
+* The patch review cycle::
@end menu
@end example
If you're tracking the remote @code{master} branch, you should add
-the @code{-r} option (short for @code{--rebase}) to keep commits
+the @option{-r} option (short for @option{--rebase}) to keep commits
on your local branch current:
@example
@end example
If you don't edit translated documentation and don't want to type
-@code{-r} every time, configure the master branch to rebase by
+@option{-r} every time, configure the master branch to rebase by
default with this command:
@example
@noindent
it means that you have modified some files in you working tree
-without committing changes (see @ref{Commits and patches}); you
+without committing changes (see @ref{Commits}); you
can use the @command{git@tie{}stash} command to work around this:
@example
patch, and this may create a conflict. If this happens, see
@ref{Resolving conflicts}.
-FIXME: I think the next paragraph is confusing. Perhaps prepare
+TODO: I think the next paragraph is confusing. Perhaps prepare
the reader for new terms `committish' and `head'? -mp
@warning{translators and documentation editors, if you have
you have not yet pushed to @code{git.sv.gnu.org}, please do not
rebase. If you want to avoid wondering whether you should rebase
each time you pull, please always use committishes from master
-and/or lilypond/translation branch on @code{git.sv.gnu.org}, which
+and/or translation branch on @code{git.sv.gnu.org}, which
in particular implies that you must push your changes to
documentation except committishes updates (possibly after having
rebased), then update the committishes and push them.}
-FIXME: when committishes automatic conditional update have been
+TODO: when committishes automatic conditional update have been
tested and documented, append the following to the warning above:
Note that using update-committishes make target generally touches
committishes.
@end example
Git will ask you for confirmation if it sees that data would be
-lost by deleting the branch. Use @code{-D} instead of @code{-d}
+lost by deleting the branch. Use @option{-D} instead of @option{-d}
to bypass this. Note that you cannot delete a branch if it is
currently checked out.
If any conflict happens, see @ref{Resolving conflicts}.
-There are common usage cases for merging: as a translator, you
-will often want to merge @code{master} into
-@code{lilypond/translation}; on the other hand, the Translations
-meister wants to merge @code{lilypond/translation} into
-@code{master} whenever he has checked that
-@code{lilypond/translation} builds successfully.
+There are common usage cases for merging: as a translator, you will
+often want the Translations meister to merge @code{master} into
+@code{translation}; on the other hand, the Translations meister wants
+to merge @code{translation} into @code{staging} whenever he has
+checked that @code{translation} builds successfully.
-@node Commits and patches
-@subsection Commits and patches
-
+@node Commits
+@subsection Commits
@menu
* Understanding commits::
-* Making commits::
+* How to make a commit::
* Commit messages::
-* Making patches::
@end menu
-@node Understanding commits
-@unnumberedsubsubsec Understanding commits
+@node Understanding commits
+@unnumberedsubsubsec Understanding commits
+
+Technically, a @emph{commit} is a single point in the history of a
+branch, but most developers use the term to mean a @emph{commit
+object}, which stores information about a particular revision. A
+single commit can record changes to multiple source files, and
+typically represents one logical set of related changes (such as a
+bug-fix). You can list the ten most recent commits in your
+current branch with this command:
+
+@example
+git log -10 --oneline
+@end example
+
+If you're using an older version of Git and get an @q{unrecognized
+argument} error, use this instead:
+
+@example
+git log -10 --pretty=oneline --abbrev-commit
+@end example
+
+More interactive lists of the commits on the remote @code{master}
+branch are available at
+@uref{http://git.sv.gnu.org/gitweb/?p=lilypond.git;a=shortlog} and
+@uref{http://git.sv.gnu.org/cgit/lilypond.git/log/}.
+
+
+@node How to make a commit
+@unnumberedsubsubsec How to make a commit
+
+
+Once you have modified some source files in your working
+directory, you can make a commit with the following procedure:
+
+@enumerate
+@item
+Make sure you've configured Git properly (see @ref{Configuring
+Git}). Check that your changes meet the requirements described in
+@ref{Code style} and/or @ref{Documentation policy}. For advanced
+edits, you may also want to verify that the changes don't break
+the compilation process.
+
+@item
+Run the following command:
+
+@example
+git status
+@end example
+
+@noindent
+to make sure you're on the right branch, and to see which files
+have been modified, added or removed, etc. You may need to tell
+Git about any files you've added by running one of these:
+
+@example
+git add @var{file} # add untracked @var{file} individually
+git add . # add all untracked files in current directory
+@end example
+
+@noindent
+After @command{git@tie{}add}, run @command{git@tie{}status} again
+to make sure you got everything. You may also need to modify
+@file{GNUmakefile}.
+
+@item
+Preview the changes about to be committed (to make sure everything
+looks right) with:
+
+@example
+git diff HEAD
+@end example
+
+@noindent
+The @code{HEAD} argument refers to the most recent commit on the
+currently checked-out branch.
+
+@item
+Generate the commit with:
+
+@example
+git commit -a
+@end example
+
+@noindent
+The @option{-a} is short for @option{--all} which includes modified
+and deleted files, but only those newly created files that have
+previously been added.
+
+@end enumerate
+
+
+@node Commit messages
+@unnumberedsubsubsec Commit messages
+
+
+When you run the @command{git@tie{}commit@tie{}-a} command, Git
+automatically opens the default text editor so you can enter a
+@emph{commit message}. If you find yourself in a foreign editing
+environment, you're probably in @command{vi} or @command{vim}. If
+you want to switch to an editor you're more familiar with, quit by
+typing @code{:q!} and pressing @code{<Enter>}. See
+@ref{Configuring Git} for instructions on changing the default
+editor.
+
+In any case, Git will open a text file for your commit message
+that looks like this:
+
+@example
+
+# Please enter the commit message for your changes. Lines starting
+# with '#' will be ignored, and an empty message aborts the commit.
+# On branch master
+# Changes to be committed:
+# (use "git reset HEAD <file>..." to unstage)
+#
+# modified: working.itexi
+#
+@end example
+
+Your commit message should begin with a one-line summary
+describing the change (no more than 50 characters long), and if
+necessary a blank line followed by several lines giving the
+details:
+
+@c $ git log -1 --pretty=medium 4d6f1e5
+@example
+Doc: add Baerenreiter and Henle solo cello suites
+
+Added comparison of solo cello suite engravings to new essay with
+high-res images, fixed cropping on Finale example.
+@end example
+
+Commit messages often start with a short prefix describing the
+general location of the changes.
+
+@itemize
+@item
+Doc: and Doc-@var{**}: If a commit affects the
+documentation in English (or in several languages simultaneously)
+the commit message should be prefixed with @qq{Doc:@tie{}}. If the
+commit affects only one of the translations, the commit
+message should be prefixed with @qq{Doc-@var{**}:@tie{}}, where
+@var{**} is the two-letter language code.
+
+@item
+Web: and Web-@var{**}: Commits that affect the
+website should use @qq{Web:@tie{}} for English, and
+@qq{Web-@var{**}:@tie{}} for other languages.
+
+@item
+CSS: Commits that change CSS files should use @qq{Web:@tie{}CSS:@tie{}}
+or @qq{Doc:@tie{}CSS:@tie{}} depending on whether they affect the
+website or the documentation/manuals.
+
+@item
+Changes to a single file are often prefixed with the name of the file
+involved.
+@end itemize
+
+Visit the links listed in @ref{Understanding commits} for examples.
+
+
+
+@node Patches
+@subsection Patches
+
+@menu
+* How to make a patch::
+* Emailing patches::
+@end menu
+
+@node How to make a patch
+@unnumberedsubsubsec How to make a patch
+
+If you want to share your changes with other contributors and
+developers, you need to generate @emph{patches} from your commits.
+We prefer it if you follow the instructions in
+@ref{Uploading a patch for review}. However, we present an
+alternate method here.
+
+You should always run @command{git@tie{}pull@tie{}-r} (translators
+should leave off the @option{-r}) before doing this to ensure that
+your patches are as current as possible.
+
+Once you have made one or more commits in your local repository,
+and pulled the most recent commits from the remote branch, you can
+generate patches from your local commits with the command:
+
+@example
+git format-patch origin
+@end example
+
+The @code{origin} argument refers to the remote tracking branch at
+@code{git.sv.gnu.org}. This command generates a separate patch
+for each commit that's in the current branch but not in the remote
+branch. Patches are placed in the current working directory and
+will have names that look something like this:
+
+@example
+0001-Doc-Fix-typos.patch
+0002-Web-Remove-dead-links.patch
+⋮
+@end example
+
+Send an email (must be less than 64 KB) to
+@email{lilypond-devel@@gnu.org} briefly explaining your work, with
+the patch files attached. Translators should send patches to
+@email{translations@@lilynet.net}. After your patches are
+reviewed, the developers may push one or more of them to the main
+repository or discuss them with you.
+
+
+@node Emailing patches
+@unnumberedsubsubsec Emailing patches
+
+The default @code{x-diff} MIME type associated with patch files
+(i.e., files whose name ends in @code{.patch}) means that the
+encoding of line endings may be changed from UNIX to DOS format
+when they are sent as attachments. Attempting to apply such an
+inadvertently altered patch will cause git to fail with a message
+about @q{whitespace errors}.
+
+The solution to such problems is surprisingly simple---just change
+the default file extension of patches generated by git to end in
+@code{.txt}, for example:
+
+@example
+git config format.suffix '.patch.txt'
+@end example
+
+This should cause email programs to apply the correct base64
+encoding to attached patches.
+
+If you receive a patch with DOS instead of UNIX line-endings, it
+can be converted back using the @code{dos2unix} utility.
+
+Lots of useful information on email complications with patches is
+provided on the Wine wiki at
+@uref{http://wiki.winehq.org/GitWine}.
+
+
+@node Uploading a patch for review
+@subsection Uploading a patch for review
+
+Any non-trivial change should be uploaded to our @qq{Rietveld}
+code review website:
+
+@example
+@uref{http://codereview.appspot.com/}
+@end example
+
+You can upload a patch for review by using our custom @code{git-cl}
+@q{helper-script}. This section assumes you have already installed,
+updated, and configured @code{git-cl}. See @ref{git-cl}.
+
+@warning{Unless you are familiar with branches, only work on one
+set of changes at once.}
+
+There are two methods, depending on your git setup.
+
+@itemize
+@item
+@strong{Master branch}: (easy option)
+
+If you added your patch to @code{master}, then:
+
+@example
+git pull -r
+git-cl upload origin/master
+@end example
+
+@c Mention staging here?
+If you have git push ability, make sure that you @emph{remove}
+your patch (with @command{git rebase} or @command{git reset})
+before pushing other stuff.
+
+@c don't make this one an @example; we don't want to make it easy
+@c for people to use this accidently
+Notifications of patches are automatically added to our issue
+tracker to reduce the chance of patches getting lost. To suppress
+this (not recommended), add the @code{-n / --no-code-issue}
+option.
+
+@item
+@strong{Separate branch}: (complicated option)
+
+Ensure your changes are committed in a separate branch, which should
+differ from the reference branch to be used (usually
+@code{origin/master}) by just the changes to be uploaded. Checkout the
+branch with the changes:
+
+@example
+git checkout some-branch-with-changes
+@end example
+
+If the reference branch is to be @code{origin/master}, ensure that the
+branch containing the changes is up-to-date with it. Use
+@command{git rebase} or @command{git pull -r} to rebase the branch to
+the head of @code{origin/master}. For example:
+
+@example
+git pull -r origin master
+@end example
+
+Finally, start the upload by entering:
+
+@example
+git-cl upload <reference SHA1 ID>
+@end example
+
+@noindent
+where <reference SHA1 ID> is the SHA1 ID of the commit to be used
+as a reference source for the patch. Generally, this will be the
+SHA1 ID of origin/master, and in that case you can just use the command:
+
+@example
+git-cl upload origin/master
+@end example
+
+@end itemize
+
+First you will see a terminal editor where you can edit the
+message that will accompany your patch. @command{git-cl} will
+respect the @env{EDITOR} environment variable if defined,
+otherwise it will use @command{vi} as the default editor.
+
+After prompting for your Google email address and password, the
+patch set will be posted to Rietveld, and you will be given a URL
+for your patch.
+
+@warning{Some installations of git-cl fail when uploading a patch
+with certain filename extensions. If this happens, it can
+generally be fixed by editing the list of exceptions at top of
+@file{git-cl.py}.}
+
+@subsubheading Announcing your patch set
+
+You should then announce the patch by logging into the code review
+issue webpage and using @qq{Publish + Mail Comments} to add a
+(mostly bogus) comment to your issue. The text of your comment
+will be sent to our developer mailing list.
+
+@warning{There is no automatic notification of a new patch; you
+must add a comment yourself.}
+
+@subsubheading Revisions
+
+As revisions are made in response to comments, successive patch sets
+for the same issue can be uploaded by reissuing the git-cl command
+with the modified branch checked out.
+
+Sometimes in response to comments on revisions, the best way to
+work may require creation of a new branch in git. In order to
+associate the new branch with an existing Rietveld issue,
+the following command can be used:
+
+@example
+git-cl issue issue-number
+@end example
+
+@noindent
+where @code{issue-number} is the number of the existing Rietveld
+issue.
+
+@subsubheading Resetting git-cl
+
+If @command{git-cl} becomes confused, you can @qq{reset} it by
+running:
+
+@example
+git-cl issue 0
+@end example
+
+
+@node The patch review cycle
+@subsection The patch review cycle
+
+Your patch will be available for reviews for the next few hours or
+days. Three times a week, patches with no known problems are
+gathered into a @qq{patch countdown} and their status changed to
+@code{patch-countdown}. The countdown is a 48-hour waiting period
+in which any final reviews or complaints should be made.
+
+During the countdown, your patch may be set to
+@code{patch-needs_work}, indicating that you should fix something
+(or at least discuss why the patch needs no modification). If no
+problems are found, the patch will be set to @code{patch-push}.
+
+Once a patch has @code{patch-push}, it should be sent to your
+mentor for uploading. If you have git push ability, look at
+@ref{Pushing to staging}.
+
+@itemize
+
+@item
+Patches get added to the tracker and to Rietveld by the @qq{git-cl} tool, with
+a status of @qq{patch-new}.
+
+@item
+The automated tester, Patchy, verifies that the patch can be applied
+to current master. By default, it checks that the patch allows @code{make}
+and @code{make test} to complete successfully. It can also be configured to
+check that @code{make doc} is successful. If it passes, Patchy changes the
+status to @qq{patch-review} and emails the developer list. If the patch
+fails, Patchy sets it to @qq{patch-needs_work} and notifies the developer list.
+
+@item
+The Patch Meister reviews the tracker periodically, to list patches
+which have been on review for at least 24 hours. The list is found at
+
+@smallexample
+@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch%20patch=review&sort=modified+patch&colspec=ID%20Type%20Status%20Priority%20Owner%20Patch%20Summary%20Modified}
+@end smallexample
+
+@item
+For each patch, the Handler reviews any discussion on the tracker
+and on Rietveld, to determine whether the patch can go forward. If
+there is any indication that a developer thinks the patch is not
+ready, the Handler marks it @qq{patch-needs_work} and makes a comment
+regarding the reason, referring to the Rietveld item if needed.
+
+@item
+Patches with explicit approval, or at least no negative comment, can
+be updated to @qq{patch-countdown}. When saving the tracker item,
+clear the @qq{send email} box to prevent sending notification for
+each patch.
-Technically, a @emph{commit} is a single point in the history of a
-branch, but most developers use the term to mean a @emph{commit
-object}, which stores information about a particular revision. A
-single commit can record changes to multiple source files, and
-typically represents one logical set of related changes (such as a
-bug-fix). You can list the ten most recent commits in your
-current branch with this command:
+@item
+The Patch Meister sends an email to the developer list, with a fixed
+subject line, to enable filtering by email clients:
@example
-git log -10 --oneline
+PATCH: Countdown to 20130113
@end example
-If you're using an older version of Git and get an @q{unrecognized
-argument} error, use this instead:
+The text of the email sets the deadline for this countdown batch. At
+present, batches are done on Tuesday, Thursday and Sunday evenings.
+
+To create the countdown announcement, use the
+@code{make-countdown-announcement.sh} script, which takes the
+deadline date, and optionally your name. Follow the instructions
+provided:
@example
-git log -10 --pretty=oneline --abbrev-commit
+cd $LILYPOND_GIT
+scripts/auxiliar/make-countdown-announcement.sh "Jan 1, 2001" James
@end example
-More interactive lists of the commits on the remote @code{master}
-branch are available at
-@uref{http://git.sv.gnu.org/gitweb/?p=lilypond.git;a=shortlog} and
-@uref{http://git.sv.gnu.org/cgit/lilypond.git/log/}.
-
-
-@node Making commits
-@unnumberedsubsubsec Making commits
+The script produces an announcement that is easily readable in all
+email clients. Also, whenever a new contributor submits a patch,
+you will be prompted to add the new username and author name to
+the script itself, and then commit those changes to the main git
+repository.
-Once you have modified some source files in your working
-directory, you can make a commit with the following procedure:
+@item
+On the scheduled countdown day, the Patch Meister reviews the
+previous list of patches on countdown, with the same procedure and
+criteria as before. Patches with no controversy can be set to
+@qq{patch-push} with a courtesy message added to the comment block.
-@enumerate
@item
-Make sure you've configured Git properly (see @ref{Configuring
-Git}). Check that your changes meet the requirements described in
-@ref{Code style} and/or @ref{Documentation policy}. For advanced
-edits, you may also want to verify that the changes don't break
-the compilation process.
+Roughly at six month intervals, the Patch Meister can list the
+patches which have been set to @qq{patch-needs-work} and send the
+results to the developer list for review. In most cases, these
+patches should be marked @qq{patch-abandoned} but this should come
+from the developer if possible.
@item
-Run the following command:
+As in most organisations of unpaid volunteers, fixed procedures are
+useful in as much as they get the job done. In our community, there
+is room for senior developers to bypass normal patch handling flows,
+particularly now that the testing of patches is largely automated.
+Similarly, the minimum age of 24 hours can reasonably be waived if
+the patch is minor and from an experienced developer.
-@example
-git status
-@end example
-@noindent
-to make sure you're on the right branch, and to see which files
-have been modified, added or removed, etc. You may need to tell
-Git about any files you've added by running one of these:
+@end itemize
+
+@ignore
+There is a single Patch Meister, and a number of Patch Helpers
+(rename this?). The list of known patches awaiting review is:
@example
-git add @var{file} # add untracked @var{file} individually
-git add . # add all untracked files in current directory
+@uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch&sort=patch}
@end example
-@noindent
-After @command{git@tie{}add}, run @command{git@tie{}status} again
-to make sure you got everything. You may also need to modify
-@file{GNUmakefile}.
+@subheading Helpers: adding patches
+
+The primary duty is to add patches to the google tracker; we have
+a bad track record of losing patches in email. Patches generally
+come to the @code{lilypond-devel} mailing list, but are sometimes
+sent to @code{bug-lilypond}, @code{lilypond-users}, or
+@code{frogs} mailing list instead.
+
+@itemize
@item
-Preview the changes about to be committed (to make sure everything
-looks right) with:
+Unless a patch is clearly in response to an existing issue, add a
+new issue with the @code{Patch-new} label and a link to the patch
+(either on the mailing list archives or the codereview url).
-@example
-git diff HEAD
-@end example
+Issue numbers are cheap; losing developers because they got fed up
+with us losing their hard work is expensive.
-@noindent
-The @code{HEAD} argument refers to the most recent commit on the
-currently checked-out branch.
+
+@c if we enter patches immediately, I don't think this is relevant.
@item
-Generate the commit with:
+Before adding a patch-reminder issue, do a quick check to see if
+it was pushed without sending any email. This can be checked for
+searching for relevant terms (from the patch subject or commit
+message) on the webgit page:
@example
-git commit -a
+@uref{http://git.savannah.gnu.org/gitweb/?p=lilypond.git}
@end example
-@noindent
-The @code{-a} is short for @code{--all} which includes modified
-and deleted files, but only those newly created files that have
-previously been added.
-@end enumerate
+@item
+If the patch is clearly in response to an existing issue, then
+update that issue with the @code{Patch-new} label and a link to
+the patch (either on the mailing list archives or the codereview
+url).
+@item
+After adding the issue, please send a response email to the same
+group(s) that the initial patch was sent to.
-@node Commit messages
-@unnumberedsubsubsec Commit messages
+If the initial email was sent to multiple mailing lists (such as
+both @code{bugs} and @code{devel}), then reply to all those
+mailing lists as well. The email should contain a link to the
+issue you just added.
+@end itemize
-When you run the @command{git@tie{}commit@tie{}-a} command, Git
-automatically opens the default text editor so you can enter a
-@emph{commit message}. If you find yourself in a foreign editing
-environment, you're probably in @command{vi} or @command{vim}. If
-you want to switch to an editor you're more familiar with, quit by
-typing @code{:q!} and pressing @code{<Enter>}. See
-@ref{Configuring Git} for instructions on changing the default
-editor.
+@subheading Helpers: @code{Patch-review} label
-In any case, Git will open a text file for your commit message
-that looks like this:
+The secondary duty is to do make sure that every issue in the
+tracker with a @code{Patch-review} label has passed these
+@qq{obvious} tests:
-@example
+@itemize
+@item
+Applies automatically to git master.
-# Please enter the commit message for your changes. Lines starting
-# with '#' will be ignored, and an empty message aborts the commit.
-# On branch master
-# Changes to be committed:
-# (use "git reset HEAD <file>..." to unstage)
-#
-# modified: working.itexi
-#
-@end example
+It's ok to have offsets, but not conflicts.
-Your commit message should begin with a one-line summary
-describing the change (no more than 50 characters long), and if
-necessary a blank line followed by several lines giving the
-details:
+@item
+Regtest comparison looks ok; no unexpected changes.
-@c $ git log -1 --pretty=medium 4d6f1e5
-@example
-Doc: add Baerenreiter and Henle solo cello suites
+@item
+Descriptive subject line.
-Added comparison of solo cello suite engravings to new essay with
-high-res images, fixed cropping on Finale example.
-@end example
+Avoid subjects like @qq{fixes 123}; instead write @qq{Doc: discuss
+stacking-dir for BassFigureAlignment (fix 123)}.
-Commit messages often start with a short prefix describing the
-general location of the changes. If a commit affects the
-documentation in English (or in several languages simultaneously)
-the commit message should be prefixed with @qq{Doc:@tie{}}. If
-the commit affects only one of the translations, the commit
-message should be prefixed with @qq{Doc-@var{**}:@tie{}}, where
-@var{**} is the two-letter language code. Commits that affect the
-website should use @qq{Web:@tie{}} for English, and
-@qq{Web-@var{**}:@tie{}} for the other languages. Also, changes
-to a single file are often prefixed with the name of the file
-involved. Visit the links listed in @ref{Understanding commits}
-for examples.
+@item
+Compiles docs from scratch. Only check this if you have reason to
+suspect it might not work.
+@item
+(maybe)
-@node Making patches
-@unnumberedsubsubsec Making patches
+Check code indentation and style. This should be easier post-GOP
+when we have a better-defined code style.
+@end itemize
-If you want to share your changes with other contributors and
-developers, you need to generate @emph{patches} from your commits.
-You should always run @command{git@tie{}pull@tie{}-r} (translators
-should leave off the @code{-r}) before doing this to ensure that
-your patches are as current as possible.
-Once you have made one or more commits in your local repository,
-and pulled the most recent commits from the remote branch, you can
-generate patches from your local commits with the command:
+@subheading Patch Meister
-@example
-git format-patch origin
-@end example
+The Patch Meister will:
-The @code{origin} argument refers to the remote tracking branch at
-@code{git.sv.gnu.org}. This command generates a separate patch
-for each commit that's in the current branch but not in the remote
-branch. Patches are placed in the current working directory and
-will have names that look something like this:
+@itemize
-@example
-0001-Doc-Fix-typos.patch
-0002-Web-Remove-dead-links.patch
-⋮
-@end example
+@item
+send @qq{countdown} emails to
+@code{lilypond-devel} when patches appear to be ready.
+
+@item
+send general requests to review patches, or even nasty requests to
+review patches.
+
+@item
+downgrade patches from @code{Patch-review} to
+@code{Patch-needs_work} as appropriate.
-Send an email to @email{lilypond-devel@@gnu.org} briefly
-explaining your work, with the patch files attached. Translators
-should send patches to @email{translations@@lilynet.net}. After
-your patches are reviewed, the developers may push one or more of
-them to the main repository or discuss them with you.
+@item
+downgrade patches from @code{Patch-needs_work} to
+@code{Patch-abandoned} if no actions have been taken in four
+weeks.
-@seealso
+@end itemize
-If your patch includes a significant amount of code, you may want
-to see @ref{Adding or modifying features}, especially @emph{Post
-patch for comments}.
+@end ignore
@node Advanced Git procedures
It is possible to work with several branches on the same local Git
repository; this is especially useful for translators who may have
-to deal with both @code{lilypond/translation} and a stable branch,
+to deal with both @code{translation} and a stable branch,
e.g. @code{stable/2.12}.
Some Git commands are introduced first, then a workflow with
@menu
+* Merge conflicts::
* Advanced Git concepts::
* Resolving conflicts::
* Reverting all local changes::
* Working with remote branches::
* Git log::
* Applying remote patches::
-* Sending and receiving patches via email::
+* Cleaning up multiple patches::
* Commit access::
+* Pushing to staging::
@end menu
+@node Merge conflicts
+@subsection Merge conflicts
+
+To be filled in later, and/or moved to a different section. I
+just wanted to make sure that I had a stub ready somewhere.
+
+
@node Advanced Git concepts
@subsection Advanced Git concepts
@noindent
where @code{@var{branch}} is typically @code{master} or
-@code{lilypond/translation}; if you do not know or remember, see
+@code{translation}; if you do not know or remember, see
@ref{Downloading remote branches} to remember which commands you
issued or which source code you wanted to get.
A @emph{branch} is nothing more than a pointer to a particular
commit, which is called the @emph{head} of the branch; when
-referring to a branch, one often acutally thinks about its head
+referring to a branch, one often actually thinks about its head
and the ancestor commits of the head.
Now we will explain the two last commands you used to get the
Sometimes git will become hopelessly confused, and you just want
to get back to a known, stable state. This command destroys any
-local changes you have made, but at least you get back to the
-current online version:
+local changes you have made in the currently checked-out branch,
+but at least you get back to the current online version:
@example
git reset --hard origin/master
@subsubheading Local clones, or having several working trees
If you play with several Git branches, e.g. @code{master},
-@code{lilypond/translation}, @code{stable/2.12}), you may want to
+@code{translation}, @code{stable/2.12}), you may want to
have one source and build tree for each branch; this is possible
with subdirectories of your local Git repository, used as local
cloned subrepositories. To create a local clone for the branch
The commands above don't only bring you the latest version of the
-sources, but also the full history of revisions (revisons, also
+sources, but also the full history of revisions (revisions, also
called commits, are changes made to the sources), stored in the
@file{.git} directory. You can browse this history with
@subsection Applying remote patches
-FIXME: Explain how to determine if a patch was created with
+TODO: Explain how to determine if a patch was created with
@code{git@tie{}format-patch}.
Well-formed git patches created with @code{git@tie{}format-patch}
Patches created without @code{git@tie{}format-patch} can be
applied in two steps. The first step is to apply the patch to the
-working tree:
+working tree and the index:
@example
-git apply @var{patch}
+git apply --index @var{patch}
@end example
@noindent
author of the patch. This can be done with the following command:
@example
-git commit -a --author="@var{John Smith} <@var{john@@example.com}>"
+git commit --author="@var{John Smith} <@var{john@@example.com}>"
@end example
+Please note that using the @code{--index} option for patching is quite
+important here and @emph{cannot} reliably be replaced by using the
+@code{-a} option when committing: that would only commit files from the
+working tree that are already registered with git, so every file that
+the patch actually @emph{adds}, like a regtest for a fixed bug, would
+get lost. For the same reason, you should not use the git-independent
+@samp{patch} program for applying patches.
-@node Sending and receiving patches via email
-@subsection Sending and receiving patches via email
-The default @code{x-diff} MIME type associated with patch files
-(i.e., files whose name ends in @code{.patch}) means that the
-encoding of line endings may be changed from UNIX to DOS format
-when they are sent as attachments. Attempting to apply such an
-inadvertently altered patch will cause git to fail with a message
-about @q{whitespace errors}.
+@node Cleaning up multiple patches
+@subsection Cleaning up multiple patches
-The solution to such problems is surprisingly simple---just change
-the default file extension of patches generated by git to end in
-@code{.txt}, for example:
+If you have been developing on your own branch for a while, you
+may have more commmits than is really sensible. To revise your
+work and condense commits, use:
@example
-git config format.suffix '.patch.txt'
+git rebase origin/master
+git rebase -i origin/master
@end example
-This should cause email programs to apply the correct base64
-encoding to attached patches.
-
-If you receive a patch with DOS instead of UNIX line-endings, it
-can be converted back using the @code{dos2unix} utility.
-
-Lots of useful information on email complications with patches is
-provided on the Wine wiki at
-@uref{http://wiki.winehq.org/GitWine}.
+@warning{Be a bit cautious -- if you completely remove commits
+during the interactive session, you will... err... completely
+remove those commits.}
@node Commit access
@subsection Commit access
-
Most contributors are not able to commit patches directly to the
main repository---only members of the LilyPond development team
have @emph{commit access}. If you are a contributor and are
certificate in your browser, given at
@uref{http://savannah.gnu.org/tls/tutorial/}.
+@warning{Savannah will silently put your username in lower-case --
+do not try to use capital letters.}
+
+
@item
After registering, if you are not logged in automatically, login
at @uref{https://savannah.gnu.org/account/login.php}---this should
take you to your @qq{my} page
(@uref{https://savannah.gnu.org/my/}).
+
@item
Click on the @qq{My Groups} link to access the @qq{My Group
Membership} page. From there, find the @qq{Request for Inclusion}
activated, LilyPond should appear under the heading @qq{Groups I'm
Contributor of} on your @qq{My Group Membership} page.
+
+@item
+Generate an SSH @q{rsa} key pair. Enter the following at the
+command prompt:
+
+@example
+ssh-keygen -t rsa
+@end example
+
+When prompted for a location to save the key, press <ENTER> to
+accept the default location (@file{~/.ssh/id_rsa}).
+
+Next you are asked to enter an optional passphrase. On most
+systems, if you use a passphrase, you will likely be prompted for
+it every time you use @command{git@tie{}push} or
+@command{git@tie{}pull}. You may prefer this since it can protect
+you from your own mistakes (like pushing when you mean to pull),
+though you may find it tedious to keep re-entering it.
+
+You can change/enable/disable your passphrase at any time with:
+
+@example
+ssh-keygen -f ~/.ssh/id_rsa -p
+@end example
+
+Note that the GNOME desktop has a feature which stores your
+passphrase for you for an entire GNOME session. If you use a
+passphrase to @qq{protect you from yourself}, you will want to
+disable this feature, since you'll only be prompted once. Run the
+following command, then logout of GNOME and log back in:
+
+@example
+gconftool-2 --set -t bool \
+ /apps/gnome-keyring/daemon-components/ssh false
+@end example
+
+After setting up your passphrase, your private key is saved as
+@file{~/.ssh/id_rsa} and your public key is saved as
+@file{~/.ssh/id_rsa.pub}.
+
+
@item
-Go to the @qq{My Account Configuration} page. From there, click
-on @qq{Edit SSH Keys} and follow the instructions given.
+Register your public SSH @q{rsa} key with Savannah. From the
+@qq{My Account Configuration} page, click on @qq{Edit SSH Keys},
+then paste the contents of your @file{~/.ssh/id_rsa.pub} file into
+one of the @qq{Authorized keys} text fields, and click
+@qq{Update}.
+
+Savannah should respond with something like:
-FIXME: Explain the confusing warning I always get. -mp
+@example
+Success: Key #1 seen Keys registered
+@end example
-FIXME: Maybe add a note about enabling/disabling SSH passphrase?
@item
Configure Git to use the SSH protocol (instead of the GIT
@end example
@noindent
-where @var{user} is your username on Savannah.
+replacing @var{user} with your Savannah username.
+
@item
After your membership has been activated and you've configured Git
-to use SSH, try doing a @command{git@tie{}pull} or
-@command{git@tie{}pull@tie{}-r}. If that succeeds, this indicates
-that your SSH key stored at Savannah is working properly.
+to use SSH, test the connection with:
+
+@example
+git pull --verbose
+@end example
+
+SSH should issue the following warning:
+
+@example
+The authenticity of host 'git.sv.gnu.org (140.186.70.72)' can't
+be established.
+RSA key fingerprint is
+80:5a:b0:0c:ec:93:66:29:49:7e:04:2b:fd:ba:2c:d5.
+Are you sure you want to continue connecting (yes/no)?
+@end example
+
+Make sure the RSA key fingerprint displayed matches the one above.
+If it doesn't, respond @qq{no} and check that you configured Git
+properly in the previous step. If it does match, respond
+@qq{yes}. SSH should then issue another warning:
+
+@example
+Warning: Permanently added 'git.sv.gnu.org,140.186.70.72' (RSA) to
+the list of known hosts.
+@end example
+
+The list of known hosts is stored in the file
+@file{~/.ssh/known_hosts}.
+
+At this point, you are prompted for your passphrase if you have
+one, then Git will attempt a pull.
+
+If @command{git@tie{}pull@tie{}--verbose} fails, you should see
+error messages like these:
+
+@example
+Permission denied (publickey).
+fatal: The remote end hung up unexpectedly
+@end example
+
+If you get the above error, you may have made a mistake when
+registering your SSH key at Savannah. If the key is properly
+registered, you probably just need to wait for the Savannah server
+to activate it. It usually takes a few minutes for the key to be
+active after registering it, but if it still doesn't work after an
+hour, ask for help on the mailing list.
+
+If @command{git@tie{}pull@tie{}--verbose} succeeds, the output
+will include a @q{From} line that shows @q{ssh} as the protocol:
+
+@example
+From ssh://git.sv.gnu.org/srv/git/lilypond
+@end example
+
+If the protocol shown is not @q{ssh}, check that you configured
+Git properly in the previous step.
-FIXME: show what success/failure look like.
@item
Test your commit access with a dry run:
+@warning{Do not push directly to master; instead, push to staging.
+See @ref{Pushing to staging}.}
+
@example
git push --dry-run --verbose
@end example
@noindent
Then @code{git@tie{}push} should work as before. For more
details, consult the @code{git@tie{}push} man page.
-@end enumerate
+@item
+Repeat the steps from generating an RSA key through to testing
+your commit access, for each machine from which you will be
+making commits, or you may simply copy the files from your
+local @file{~/.ssh} folder to the same folder on the other
+machine.
+
+@end enumerate
+
@subsubheading Technical details
@itemize
@end example
@end itemize
+@knownissues
+Encryption protocols, including ssh, generally do not permit packet
+fragmentation to avoid introducing a point of insecurity. This
+means that the maximum packet size must not exceed the smallest
+MTU (Maximum Transmission Unit) set in the routers along the path.
+This smallest MTU is determined by a procedure during call set-up
+which relies on the transmission over the path of ICMP packets.
+If any of the routers in the path block ICMP packets this mechanism
+fails, resulting in the possibility of packets being transmitted
+which exceed the MTU of one of the routers. If this happens the
+packet is discarded, causing the ssh session to hang, timeout or
+terminate with the error message
+
+@example
+ssh: connect to host <host ip addr> port 22: Bad file number
+fatal: The remote end hung up unexpectedly
+@end example
+
+depending on precisely when in the proceedings the first large
+packet is transmitted. Most routers on the internet have MTU
+set to 1500, but routers installed in homes to connect via
+broadband may use a slightly smaller MTU for efficient transmission
+over ATM. If this problem is encountered a possible work-around is
+to set the MTU in the local router to 1500.
+
+
+@node Pushing to staging
+@subsection Pushing to staging
+
+Do not push directly to the git @code{master} branch. Instead,
+push to @code{staging}.
+
+You will not see your patch on @code{origin/master} until some
+automatic tests have been run. These tests are run every couple
+of hours; please wait at least 12 hours before wondering if your
+patch has been lost. Note that you can check the commits on
+@code{origin/staging} by looking at the git web interface on
+savannah.
+
+It may happen occasionally that the staging branch breaks automated
+testing. In this case the automatic move of staging material to
+master gets halted in order to avoid broken material entering master.
+This is a safety net. Please do not try breaking out from it by
+adding fixes on top of staging: in that case the whole sequence will
+end up in master after all, defeating the purpose of the system. The
+proper fix usually involves rewriting the staging branch and is best
+left to core developers after discussion on the developer list.
+
+Before pushing to staging it is a good practice to check whether
+staging is ahead of master, and if so, wait until master has caught up
+with staging before pushing. This simplifies things if changes to
+staging have to be backed out for some reason. To check whether
+master has caught up with staging you can look at the git web interface
+on savannah, or do:
+
+@example
+git fetch
+gitk
+@end example
+
+and check that @code{origin/master} is at the same commit as
+@code{origin/staging}. Another option is to see if any commits are
+listed when you do:
+
+@example
+git fetch
+git log origin/master..origin/staging
+@end example
+
+@subsubheading If your work is in a patch file
+
+Assuming that your patch is in a file called
+@file{0001-my-patch.patch} (see @ref{Patches}), and you are currently
+on git master, do:
+
+@example
+git checkout staging
+git pull -r
+git am 0001-my-patch.patch
+gitk
+git push origin staging
+git checkout master
+@end example
+
+@warning{Do not skip the @command{gitk} step; a quick 5-second
+check of the visual history can save a great deal of frustration
+later on. You should only see that @command{staging} is only 1
+commit ahead of @code{origin/staging}.}
+
+@subsubheading If your work is in a branch
+
+If you are working on branches and your work is in
+@code{my_branch_name}, then do:
+
+@example
+git checkout my_branch_name
+git pull -r origin staging
+@end example
+
+This will rebase your branch on @code{origin/staging}. At this point
+git will let you know if there are any conflicts. If so, resolve them
+before continuing:
+
+@example
+gitk
+git push origin HEAD:staging
+@end example
+
+@warning{Do not skip the @command{gitk} step; a quick 5-second
+check of the visual history can save a great deal of frustration
+later on. You should see that @code{my_branch_name} is only ahead of
+@code{origin/staging} by the commits from your branch.}
+
+
+
@node Git on Windows
@section Git on Windows
+@warning{We heavily recommend that development be done with our
+virtual machine @ref{LilyDev}.}
+
@c Some of this may duplicate stuff in other sections
@c But it is probably best for windows users to have it all together
@c If necessary, clear this up later -td
-FIXME: Decide what to do with this... Pare it down? Move
+TODO: Decide what to do with this... Pare it down? Move
paragraphs next to analogous Unix instructions? -mp
@subsection Background to nomenclature
Git is a system for tracking the changes made to source files by a
distributed set of editors. It is designed to work without a
-master repository, but we have chosen to have a master respository
+master repository, but we have chosen to have a master repository
for LilyPond files. Editors hold a local copy of the master
repository together with any changes they have made locally.
Local changes are held in a local @q{branch}, of which there may
@subsection Installing git
Obtain Git from
-@uref{http://code.google.com/p/msysgit/downloads/list} (note, not
-msysGit, which is for Git developers and not PortableGit, which is
-not a full git installation) and install it.
+@uref{https://git-for-windows.github.io/}.
Note that most users will not need to install SSH. That is not
required until you have been granted direct push permissions to
@subsection Git GUI
Almost all subsequent work will use the Git Graphical User
-Interface, which avoids having to type command line commands. To
+Interface, which avoids having to type command line commands. To
start Git GUI first start the Git bash shell by clicking on the
desktop icon, and type
@item
Book about git: @uref{http://progit.org/,Pro Git}
+
+@item
+Github help: @uref{http://help.github.com/}
+(very highly recommended by Graham)
+
@end itemize
projects / lilypond.git / blobdiff