@chapter Working with source code
@warning{New contributors should read @ref{Quick start}, and in
-particular @ref{Using lily-git}, instead of this chapter.}
+particular @ref{lily-git}, instead of this chapter.}
Advanced contributors will find this material quite useful,
particularly if they are working on major new features.
skip ahead to @ref{Starting with Git}.
@warning{These instructions are only for people who are @emph{not}
-using @ref{Lilydev}.}
+using @ref{LilyDev}.}
@c there's some duplication in this section with stuff covered in
@c Quick Start, but moving it into a macro inside included/ would
@end enumerate
@warning{Throughout the rest of this manual, most command-line
-input should be entered from @file{~/lilypond-git/}. This is
+input should be entered from @file{$LILYPOND_GIT}. This is
referred to as the @emph{top source directory}.}
-Further instructions are in @ref{Daily use of lily-git.tcl}.
+Further instructions are in @ref{How to use lily-git}.
@node Starting with Git
@menu
* Setting up::
* Git for the impatient::
+* Other repositories::
* Downloading remote branches::
@end menu
@subsection Setting up
@warning{These instructions assume that you are using the
-command-line version of Git 1.5 or higher. Windows users should
-skip to @ref{Git on Windows}.}
+command-line version of Git 1.7 or higher.}
@menu
* Installing Git::
@subsubheading Technical details
-This creates (within the @file{~/lilypond-git/} directory) a
+This creates (within the @file{$LILYPOND_GIT} directory) a
subdirectory called @file{.git/}, which Git uses to keep track of
changes to the repository, among other things. Normally you don't
need to access it, but it's good to know it's there.
@warning{Throughout the rest of this manual, all command-line
input should be entered from the top directory of the Git
-repository being discussed (eg. @file{~/lilypond-git/}). This is
+repository being discussed (eg. @file{$LILYPOND_GIT}). This is
referred to as the @emph{top source directory}.}
Before working with the copy of the main LilyPond repository, you
@end example
Finally, and in some ways most importantly, let's make sure that
-we know what branch we're on. If you're not using lilydev, add
+we know what branch we're on. If you're not using LilyDev, add
this to your @file{~/.bashrc}:
@verbatim
export PS1="\u@\h \w\$(__git_ps1)$ "
@end verbatim
-If you are not using lilydev, you may need to install the
+If you are not using LilyDev, you may need to install the
additional @code{git-completion} package, but it is definitely
worth it.
Your prompt now shows you that you're on the other branch:
@example
-gperciva@@lilydev:~/lilypond-git (dev/cg)$
+gperciva@@LilyDev:~/lilypond-git (dev/cg)$
@end example
To be able to manage multiple lilypond issues at once, you'll need to switch
Branches are nerve-wracking until you get used to them. You can
save your hard work as individual @file{.patch} files. Be sure to
-commit your chages first.
+commit your changes first.
@example
git commit -a
a @code{-D}.
-@node Downloading remote branches
-@subsection Downloading remote branches
+@node Other repositories
+@subsection Other repositories
+We have a few other code repositories.
@menu
-* Organization of remote branches::
-* LilyPond repository sources::
-* Downloading individual branches::
-* Downloading all remote branches::
-* Other branches::
+* lilypond-extra::
+* Grand Unified Builder (GUB)::
+* lilypad::
+* yet more repositories::
@end menu
+@node lilypond-extra
+@unnumberedsubsubsec lilypond-extra
-@node Organization of remote branches
-@unnumberedsubsubsec Organization of remote branches
-
-
-The main LilyPond repository is organized into @emph{branches} to
-facilitate development. These are often called @emph{remote}
-branches to distinguish them from @emph{local} branches you might
-create yourself (see @ref{Using local branches}).
-
-The @code{master} branch contains all the source files used to
-build LilyPond, which includes the program itself (both stable and
-development releases), the documentation (and its translations),
-and the website. Generally, the @code{master} branch is expected
-to compile successfully.
-
-The @code{lilypond/translation} branch is a side branch that
-allows translators to work without needing to worry about
-compilation problems. Periodically, the Translation Meister
-(after verifying that it doesn't break compilation), will
-@emph{merge} this branch back into @code{master} to incorporate
-recent translations. Similarly, the @code{master} branch is
-usually merged into the @code{lilypond/translation} branch after
-significant changes to the English documentation. See
-@ref{Translating the documentation} for details.
-
-
-@node LilyPond repository sources
-@unnumberedsubsubsec LilyPond repository sources
-
-
-The recommended source for downloading a copy of the main
-repository is:
+There is a separate repository for general administrative scripts,
+as well as pictures and media files for the website. People
+interested in working on the website should download this
+repository, and set their @code{$LILYPOND_WEB_MEDIA_GIT}
+environment variable to point to that repository.
@example
-git://git.sv.gnu.org/lilypond.git
+@uref{https://github.com/gperciva/lilypond-extra}
@end example
-However, if your internet router filters out connections using the
-GIT protocol, or if you experience difficulty connecting via GIT,
-you can try these other sources:
+To configure an environment variable in bash (the default for most
+GNU/Linux distributions),
@example
-ssh://git.sv.gnu.org/srv/git/lilypond.git
-http://git.sv.gnu.org/r/lilypond.git
+export LILYPOND_WEB_MEDIA_GIT=$HOME/dir/of/lilypond-extra/
@end example
-The SSH protocol can only be used if your system is properly set
-up to use it. Also, the HTTP protocol is slowest, so it should
-only be used as a last resort.
+Be aware that @code{lilypond-extra} is the definitive source for some binary
+files - in particular PDF versions of papers concerning LilyPond. To add
+further PDFs of this sort, all that is necessary is to add the PDF to
+@code{lilypond-extra} and then add a reference to it in the documentation. The
+file will then be copied to the website when @code{make website} is run.
+However, pictures that are also used in the documentation build are mastered in
+the main git repository. If any of these is changed, it should be updated in
+git, and then the updates copied to @code{lilypond-extra}.
-@node Downloading individual branches
-@unnumberedsubsubsec Downloading individual branches
+@node Grand Unified Builder (GUB)
+@unnumberedsubsubsec Grand Unified Builder (GUB)
+Another item of interest might be the Grand Unified Builder, our
+cross-platform building tool. Since it is used by other projects as
+well, it is not stored in our gub repository. For more info, see
+@uref{http://lilypond.org/gub}.
-Once you have initialized an empty Git repository on your system
-(see @ref{Initializing a repository}), you can download a remote
-branch into it. Make sure you know which branch you want to start
-with.
-
-To download the @code{master} branch, enter the following:
+There are two locations for this repository: the version being used to
+build lilypond, which is at
@example
-git remote add -ft master -m master \
- origin git://git.sv.gnu.org/lilypond.git/
+@uref{http://github.com/gperciva/gub}
@end example
-To download the @code{lilypond/translation} branch, enter:
+and the original version by Jan Nieuwenhuizen, kept at
@example
-git remote add -ft lilypond/translation -m \
- lilypond/translation origin git://git.sv.gnu.org/lilypond.git/
+@uref{http://github.com/janneke/gub}
@end example
-The @command{git@tie{}remote@tie{}add} process could take up to
-ten minutes, depending on the speed of your connection. The
-output will be something like this:
-@example
-Updating origin
-remote: Counting objects: 235967, done.
-remote: Compressing objects: 100% (42721/42721), done.
-remote: Total 235967 (delta 195098), reused 233311 (delta 192772)
-Receiving objects: 100% (235967/235967), 68.37 MiB | 479 KiB/s, done.
-Resolving deltas: 100% (195098/195098), done.
-From git://git.sv.gnu.org/lilypond
- * [new branch] master -> origin/master
-From git://git.sv.gnu.org/lilypond
- * [new tag] flower/1.0.1 -> flower/1.0.1
- * [new tag] flower/1.0.10 -> flower/1.0.10
-⋮
- * [new tag] release/2.9.6 -> release/2.9.6
- * [new tag] release/2.9.7 -> release/2.9.7
-@end example
+@node lilypad
+@unnumberedsubsubsec lilypad
-When @command{git@tie{}remote@tie{}add} is finished, the remote
-branch should be downloaded into your repository---though not yet
-in a form that you can use. In order to browse the source code
-files, you need to @emph{create} and @emph{checkout} your own
-local branch. In this case, however, it is easier to have Git
-create the branch automatically by using the @command{checkout}
-command on a non-existent branch. Enter the following:
+Our binary releases on MacOS X and Windows contain a lightweight
+text editor. This code is here:
@example
-git checkout -b @var{branch} origin/@var{branch}
+https://github.com/gperciva/lilypad
@end example
-@noindent
-where @code{@var{branch}} is the name of your tracking branch,
-either @code{master} or @code{lilypond/translation}.
-Git will issue some warnings; this is normal:
+@node yet more repositories
+@unnumberedsubsubsec yet more repositories
-@example
-warning: You appear to be on a branch yet to be born.
-warning: Forcing checkout of origin/master.
-Branch master set up to track remote branch master from origin.
-Already on 'master'
-@end example
+There are a few other repositories floating around, which will
+hopefully be documented in the near future.
-By now the source files should be accessible---you should be able
-to edit any files in the @file{lilypond-git/} directory using a
-text editor of your choice. But don't start just yet! Before
-editing any source files, learn how to keep your changes organized
-and prevent problems later---read @ref{Basic Git procedures}.
-@subsubheading Technical Details
+@node Downloading remote branches
+@subsection Downloading remote branches
-The @command{git@tie{}remote@tie{}add} command should add some
-lines to your local repository's @file{.git/config} file:
+@warning{contains obsolete + misleading info}
-@example
-[remote "origin"]
- url = git://git.sv.gnu.org/lilypond.git/
- fetch = +refs/heads/master:refs/remotes/origin/master
-@end example
+@menu
+* Organization of remote branches::
+* LilyPond repository sources::
+@end menu
-@node Downloading all remote branches
-@unnumberedsubsubsec Downloading all remote branches
+@node Organization of remote branches
+@unnumberedsubsubsec Organization of remote branches
-To download all remote branches at once, you can @command{clone}
-the entire repository:
+The main LilyPond repository is organized into @emph{branches} to
+facilitate development. These are often called @emph{remote}
+branches to distinguish them from @emph{local} branches you might
+create yourself (see @ref{Using local branches}).
-@example
-git clone git://git.sv.gnu.org/lilypond.git
-@end example
+The @code{master} branch contains all the source files used to
+build LilyPond, which includes the program itself (both stable and
+development releases), the documentation (and its translations),
+and the website. Generally, the @code{master} branch is expected
+to compile successfully.
+The @code{translation} branch is a side branch that
+allows translators to work without needing to worry about
+compilation problems. Periodically, the Translation Meister
+(after verifying that it doesn't break compilation), will
+@emph{merge} this branch into @code{staging} to incorporate
+recent translations. Similarly, the @code{master} branch is
+usually merged into the @code{translation} branch after
+significant changes to the English documentation. See
+@ref{Translating the documentation} for details.
-@node Other branches
-@unnumberedsubsubsec Other branches
Most contributors will never need to touch the other branches. If
you wish to do so, you will need more familiarity with Git; please
@item @code{stable/XYZ}:
The branches are kept for archival reasons.
+@item @code{archive/XYZ}:
+The branches are kept for archival reasons.
+
@end itemize
-Another item of interest might be the Grand Unified Builder, our
-cross-platform building tool. Since it is used by projects as
-well, it is not stored in our gub repository. For more info, see
-@uref{http://lilypond.org/gub}. The git location is
-@uref{http://github.com/janneke/gub}.
+
+@node LilyPond repository sources
+@unnumberedsubsubsec LilyPond repository sources
+
+
+The recommended source for downloading a copy of the main
+repository is:
+
+@example
+git://git.sv.gnu.org/lilypond.git
+@end example
+
+However, if your internet router filters out connections using the
+GIT protocol, or if you experience difficulty connecting via GIT,
+you can try these other sources:
+
+@example
+ssh://git.sv.gnu.org/srv/git/lilypond.git
+http://git.sv.gnu.org/r/lilypond.git
+@end example
+
+The SSH protocol can only be used if your system is properly set
+up to use it. Also, the HTTP protocol is slowest, so it should
+only be used as a last resort.
@node Basic Git procedures
can be used.
Occasionally you may need to rework some of your own modifications
-to match changes made to the remote branch (see @ref{Resolving
+to match changes made to the remote branch (see @ref{Merge
conflicts}), and it's considerably easier to rework things
incrementally. If you don't update your repository along the way,
you may have to spend a lot of time resolving branch conflicts and
Note that @command{git@tie{}stash@tie{}pop} will try to apply a
patch, and this may create a conflict. If this happens, see
-@ref{Resolving conflicts}.
+@ref{Merge conflicts}.
TODO: I think the next paragraph is confusing. Perhaps prepare
the reader for new terms `committish' and `head'? -mp
you have not yet pushed to @code{git.sv.gnu.org}, please do not
rebase. If you want to avoid wondering whether you should rebase
each time you pull, please always use committishes from master
-and/or lilypond/translation branch on @code{git.sv.gnu.org}, which
+and/or translation branch on @code{git.sv.gnu.org}, which
in particular implies that you must push your changes to
documentation except committishes updates (possibly after having
rebased), then update the committishes and push them.}
git merge @var{foo}
@end example
-If any conflict happens, see @ref{Resolving conflicts}.
+If any conflict happens, see @ref{Merge conflicts}.
-There are common usage cases for merging: as a translator, you
-will often want to merge @code{master} into
-@code{lilypond/translation}; on the other hand, the Translations
-meister wants to merge @code{lilypond/translation} into
-@code{master} whenever he has checked that
-@code{lilypond/translation} builds successfully.
+There are common usage cases for merging: as a translator, you will
+often want the Translations meister to merge @code{master} into
+@code{translation}; on the other hand, the Translations meister wants
+to merge @code{translation} into @code{staging} whenever he has
+checked that @code{translation} builds successfully.
@node Commits and patches
and @command{upload.py} scripts in one of your PATH
directories (such as @file{$HOME/bin}).
-In Ubuntu (and Lilydev), you can add directories to PATH
+In Ubuntu (and LilyDev), you can add directories to PATH
by adding this line to a hidden file @file{.bashrc},
located in your home directory:
@item
Move into the top source directory and then configure @command{git
-cl} with the following commands. If you do not understand any
-question, just answer with a newline (CR).
+cl} with the following commands:
@example
-cd $HOME/lilypond-git/
+cd $LILYPOND_GIT
git cl config
@end example
+For the @qq{Rietveld server} question, the default value
+(@qq{codereview.appspot.com}) should be accepted by
+answering with a newline (CR).
+
The @qq{CC list} question should be answered with:
@example
lilypond-devel@@gnu.org
@end example
+The @qq{Tree status URL} value should be left blank. So should
+the @qq{ViewVC URL} value, since it is used by @command{git cl
+dcommit} which is only for repositories which use @command{git
+svn} (LilyPond doesn't).
+
@end enumerate
@subsubheading Uploading patch set
git cl upload origin/master
@end example
+@c Mention staging here?
If you have git push ability, make sure that you @emph{remove}
your patch (with @command{git rebase} or @command{git reset})
before pushing other stuff.
@end itemize
+First you will see a terminal editor where you can edit the
+message that will accompany your patch. @command{git-cl} will
+respect the @env{EDITOR} environment variable if defined,
+otherwise it will use @command{vi} as the default editor.
+
After prompting for your Google email address and password, the
patch set will be posted to Rietveld, and you will be given a URL
for your patch.
It is possible to work with several branches on the same local Git
repository; this is especially useful for translators who may have
-to deal with both @code{lilypond/translation} and a stable branch,
+to deal with both @code{translation} and a stable branch,
e.g. @code{stable/2.12}.
Some Git commands are introduced first, then a workflow with
@menu
* Merge conflicts::
* Advanced Git concepts::
-* Resolving conflicts::
* Reverting all local changes::
-* Working with remote branches::
* Git log::
* Applying remote patches::
* Sending and receiving patches via email::
@node Merge conflicts
@subsection Merge conflicts
-To be filled in later, and/or moved to a different section. I
-just wanted to make sure that I had a stub ready somewhere.
+
+Occasionally an update may result in conflicts -- this happens
+when you and somebody else have modified the same part of the same
+file and git cannot figure out how to merge the two versions
+together. When this happens, you must manually merge the two
+versions.
+
+If you need some documentation to understand and resolve
+conflicts, see paragraphs @emph{How conflicts are presented} and
+@emph{How to resolve conflicts} in @command{git merge} man page.
+
+If all else fails, you can follow the instructions in
+@ref{Reverting all local changes}. Be aware that this eliminates
+any changes you have made!
@node Advanced Git concepts
@noindent
where @code{@var{branch}} is typically @code{master} or
-@code{lilypond/translation}; if you do not know or remember, see
+@code{translation}; if you do not know or remember, see
@ref{Downloading remote branches} to remember which commands you
issued or which source code you wanted to get.
referring to a branch, one often actually thinks about its head
and the ancestor commits of the head.
-Now we will explain the two last commands you used to get the
-source code from Git---see @ref{Downloading individual branches}.
-
-@example
-git remote add -ft @var{branch} -m @var{branch} \
- origin git://git.sv.gnu.org/lilypond.git/
-
-git checkout -b @var{branch} origin/@var{branch}
-@end example
-
-The @command{git@tie{}remote} has created a branch called
-@code{origin/@var{branch}} in your local Git repository. As this
-branch is a copy of the remote branch web from git.sv.gnu.org
-LilyPond repository, it is called a @emph{remote branch}, and is
-meant to track the changes on the branch from git.sv.gnu.org: it
-will be updated every time you run
-@command{git@tie{}pull@tie{}origin} or
-@command{git@tie{}fetch@tie{}origin}.
-
-The @command{git@tie{}checkout} command has created a branch named
-@code{@var{branch}}. At the beginning, this branch is identical
-to @code{origin/@var{branch}}, but it will differ as soon as you
-make changes, e.g. adding newly translated pages or editing some
-documentation or code source file. Whenever you pull, you merge
-the changes from @code{origin/@var{branch}} and
-@code{@var{branch}} since the last pulling. If you do not have
-push (i.e. @qq{write}) access on git.sv.gnu.org, your
-@code{@var{branch}} will always differ from
-@code{origin/@var{branch}}. In this case, remember that other
-people working like you with the remote branch @code{@var{branch}}
-of git://git.sv.gnu.org/lilypond.git/ (called
-@code{origin/@var{branch}} on your local repository) know nothing
-about your own @code{@var{branch}}: this means that whenever you
-use a committish or make a patch, others expect you to take the
-latest commit of @code{origin/@var{branch}} as a reference.
-
-Finally, please remember to read the man page of every Git command
-you will find in this manual in case you want to discover
-alternate methods or just understand how it works.
-
-
-@node Resolving conflicts
-@subsection Resolving conflicts
-
-
-Occasionally an update may result in conflicts -- this happens
-when you and somebody else have modified the same part of the same
-file and git cannot figure out how to merge the two versions
-together. When this happens, you must manually merge the two
-versions.
-
-If you need some documentation to understand and resolve
-conflicts, see paragraphs @emph{How conflicts are presented} and
-@emph{How to resolve conflicts} in @command{git merge} man page.
-
-If all else fails, you can follow the instructions in
-@ref{Reverting all local changes}. Be aware that this eliminates
-any changes you have made!
-
@node Reverting all local changes
@subsection Reverting all local changes
@end example
-@node Working with remote branches
-@subsection Working with remote branches
-
-
-@subsubheading Fetching new branches from git.sv.gnu.org
-
-To fetch and check out a new branch named @code{@var{branch}} on
-git.sv.gnu.org, run from top of the Git repository
-
-@example
-git config --add remote.origin.fetch \
- +refs/heads/@var{branch}:refs/remotes/origin/@var{branch}
-
-git checkout --track -b @var{branch} origin/@var{branch}
-@end example
-
-After this, you can pull @code{@var{branch}} from git.sv.gnu.org
-with:
-
-@example
-git pull
-@end example
-
-Note that this command generally fetches all branches you added
-with @command{git@tie{}remote@tie{}add} (when you initialized the
-repository) or @command{git@tie{}config@tie{}--add}, i.e. it
-updates all remote branches from remote @code{origin}, then it
-merges the remote branch tracked by the current branch into the
-current branch. For example, if your current branch is
-@code{master}, @code{origin/master} will be merged into
-@code{master}.
-
-
-@subsubheading Local clones, or having several working trees
-
-If you play with several Git branches, e.g. @code{master},
-@code{lilypond/translation}, @code{stable/2.12}), you may want to
-have one source and build tree for each branch; this is possible
-with subdirectories of your local Git repository, used as local
-cloned subrepositories. To create a local clone for the branch
-named @code{@var{branch}}, run
-
-@example
-git checkout @var{branch}
-git clone -lsn . @var{subdir}
-cd @var{subdir}
-git reset --hard
-@end example
-
-Note that @code{@var{subdir}} must be a directory name which does
-not already exist. In @code{@var{subdir}}, you can use all Git
-commands to browse revisions history, commit and uncommit changes;
-to update the cloned subrepository with changes made on the main
-repository, cd into @code{@var{subdir}} and run
-@command{git@tie{}pull}; to send changes made on the subrepository
-back to the main repository, run @command{git@tie{}push} from
-@code{@var{subdir}}. Note that only one branch (the currently
-checked out branch) is created in the subrepository by default; it
-is possible to have several branches in a subrepository and do
-usual operations (checkout, merge, create, delete...) on these
-branches, but this possibility is not detailed here.
-
-When you push @code{@var{branch}} from @code{@var{subdir}} to the
-main repository, and @code{@var{branch}} is checked out in the
-main repository, you must save uncommitted changes (see
-@command{git@tie{}stash}) and do
-@command{git@tie{}reset@tie{}--hard} in the main repository in
-order to apply pushed changes in the working tree of the main
-repository.
-
-
@node Git log
@subsection Git log
Patches created without @code{git@tie{}format-patch} can be
applied in two steps. The first step is to apply the patch to the
-working tree:
+working tree and the index:
@example
-git apply @var{patch}
+git apply --index @var{patch}
@end example
@noindent
author of the patch. This can be done with the following command:
@example
-git commit -a --author="@var{John Smith} <@var{john@@example.com}>"
+git commit --author="@var{John Smith} <@var{john@@example.com}>"
@end example
+Please note that using the @code{--index} option for patching is quite
+important here and @emph{cannot} reliably be replaced by using the
+@code{-a} option when committing: that would only commit files from the
+working tree that are already registered with git, so every file that
+the patch actually @emph{adds}, like a regtest for a fixed bug, would
+get lost. For the same reason, you should not use the git-independent
+@samp{patch} program for applying patches.
@node Sending and receiving patches via email
@subsection Sending and receiving patches via email
will include a @q{From} line that shows @q{ssh} as the protocol:
@example
-From ssh://@var{user}@@git.sv.gnu.org/srv/git/lilypond
+From ssh://git.sv.gnu.org/srv/git/lilypond
@end example
If the protocol shown is not @q{ssh}, check that you configured
@code{origin/staging} by looking at the git web interface on
savannah.
+It may happen occasionally that the staging branch breaks automated
+testing. In this case the automatic move of staging material to
+master gets halted in order to avoid broken material entering master.
+This is a safety net. Please do not try breaking out from it by
+adding fixes on top of staging: in that case the whole sequence will
+end up in master after all, defeating the purpose of the system. The
+proper fix usually involves rewriting the staging branch and is best
+left to core developers after discussion on the developer list.
+
@subsubheading If your work is in a patch file
Assuming that your patch is in a file called
@section Git on Windows
@warning{We heavily recommend that development be done with our
-virtual machine @ref{Lilydev}.}
+virtual machine @ref{LilyDev}.}
@c Some of this may duplicate stuff in other sections
@c But it is probably best for windows users to have it all together
projects / lilypond.git / blobdiff