@node Working with source code
@chapter Working with source code
+@warning{New contributors should read @ref{Quick start}, and in
+particular @ref{Using lily-git}, instead of this chapter.}
+
+Advanced contributors will find this material quite useful,
+particularly if they are working on major new features.
@menu
-* Using lily-git::
+* Manually installing lily-git.tcl::
* Starting with Git::
* Basic Git procedures::
* Advanced Git procedures::
@end menu
-@node Using lily-git
-@section Using lily-git
+@node Manually installing lily-git.tcl
+@section Manually installing lily-git.tcl
+We have created an easy-to-use GUI to simplify git for new
+contributors. If you are comfortable with the command-line, then
+skip ahead to @ref{Starting with Git}.
-If you haven't already, download and install Git. Go to
-@uref{http://git-scm.com/download}, and in the @qq{Binaries}
-section, select the appropriate package for your operating system.
-Windows users should visit
-@uref{http://code.google.com/p/msysgit/downloads/list} and
-download the @file{.exe} file labeled @qq{Full installer for
-official Git}.
+@warning{These instructions are only for people who are @emph{not}
+using @ref{Lilydev}.}
-Download the lily-git script from:
+@c there's some duplication in this section with stuff covered in
+@c Quick Start, but moving it into a macro inside included/ would
+@c be getting a bit icky. -gp
-@c don't change the cgit link below to gitweb; gitweb uses
-@c long filenames like "scripts_auxiliar_lily-git.tcl"
+@enumerate
+@item
+If you haven't already, download and install Git.
-@example
-@uref{http://git.sv.gnu.org/cgit/lilypond.git/plain/scripts/auxiliar/lily-git.tcl}
-@end example
+@itemize
-To run the program from the command line, navigate to the
-directory containing @file{lily-git.tcl} and enter:
+@item Windows users: download the @code{.exe} file labeled
+@qq{Full installer for official Git} from:
@example
-wish lily-git.tcl
+@uref{http://code.google.com/p/msysgit/downloads/list}
@end example
+@item Other operating systems: either install @command{git} with
+your package manager, or download it from the @qq{Binaries}
+section of:
-@subsubheading Get source / Update source
-
-When you click the @qq{Get source} button, @command{lily-git} will
-create a directory called @file{lilypond-git/} within your home
-directory, and will download the complete source code into that
-directory (around 55Mb). When the process is finished, the
-@qq{Command output} window will display @qq{Done}, and the button
-label will change to say @qq{Update source}.
-
-Navigate to the @file{lilypond-git/} directory to view the source
-files. You should now be able to modify the source files using
-your normal text editor.
-
-@knownissues
-
-The @qq{Get source} button does not fetch the entire history of
-the git repository, so utilities like @command{gitk} will only
-be able to display the most recent additions. As you continue to
-work with @command{lily-git}, the @qq{Update source} button will take
-any new additions and add it to whatever is currently in your repository's
-history.
-
-
-@subsubheading New local commit
-
-A single commit typically represents one logical set of related
-changes (such as a bug-fix), and may incorporate changes to
-multiple files at the same time.
-
-When you're finished making the changes for your first commit,
-click the @qq{New local commit} button. This will open the
-@qq{Git Commit Message} window. The message header is required,
-and the message body is optional. See @ref{Commits and patches}
-for more information regarding commits and commit messages.
-
-After entering a commit message, click @qq{OK} to finalize the
-commit.
-
-
-@subsubheading Amend previous commit
-
-You can go back and make changes to the most recent commit with
-the @qq{Amend previous commit} button. This is useful if a
-mistake is found after you have clicked the @qq{New local commit}
-button.
+@example
+@uref{http://git-scm.com/download}
+@end example
-To amend the most recent commit, re-edit the source files as
-needed and then click the @qq{Amend previous commit} button. The
-earlier version of the commit is not saved, but is replaced by the
-new one.
+@end itemize
-Note that this does not update the patch @strong{files}; if you
-have a patch file from an earlier version of the commit, you will
-need to make another patch set when using this feature. The old
-patch file will not be saved, but will be replaced by the new one
-after you click on @qq{Make patch set}.
+@item
+Download the @command{lily-git.tcl} script from:
-@subsubheading Make patch set
+@c don't change the cgit link below to gitweb; gitweb uses
+@c long filenames like "scripts_auxiliar_lily-git.tcl"
-Before making a patch set from any commits, you should click the
-@qq{Update source} button to make sure the commits are based on
-the most recent remote snapshot.
+@smallexample
+@uref{http://git.sv.gnu.org/cgit/lilypond.git/plain/scripts/auxiliar/lily-git.tcl}
+@end smallexample
-When you click the @qq{Make patch set} button, @command{lily-git}
-will produce patch files for any new commits, saving them to the
-current directory. The command output will display the name of
-the new patch files near the end of the output:
+@item
+To run the program from the command line, navigate to the
+directory containing @command{lily-git.tcl} and enter:
@example
-0001-CG-add-lily-git-instructions.patch
-Done.
+wish lily-git.tcl
@end example
-Send patch files to your mentor if you have one. Otherwise, write
-an email (must be less than 64 KB) to
-@email{lilypond-devel@@gnu.org} briefly explaining your work, with
-the patch files attached. Translators should send patches to
-@email{translations@@lilynet.net}.
+@item
+Click on the @qq{Get source} button.
+This will create a directory called @file{lilypond-git/} within
+your home directory, and will download the source code into that
+directory (around 150@tie{}Mb). When the process is finished, the
+@qq{Command output} window will display @qq{Done}, and the button
+label will change to say @qq{Update source}.
-@subsubheading The @qq{Abort changes -- Reset to origin} button
+@item
+Navigate to the @file{lilypond-git/} directory to view the source
+files.
-@warning{Only use this if your local commit history gets
-hopelessly confused!}
+@end enumerate
-The button labeled @qq{Abort changes -- Reset to origin} will copy
-all changed files to a subdirectory of @file{lilypond-git/} named
-@file{aborted_edits/}, and will reset the repository to the
-current state of the remote repository (at @code{git.sv.gnu.org}).
+@warning{Throughout the rest of this manual, most command-line
+input should be entered from @file{~/lilypond-git/}. This is
+referred to as the @emph{top source directory}.}
+
+Further instructions are in @ref{Daily use of lily-git.tcl}.
@node Starting with Git
@section Starting with Git
-
Using the Git program directly (as opposed to using the
-@command{lily-git} GUI) allows you to have much greater control
+@command{lily-git.tcl} GUI) allows you to have much greater control
over the contributing process. You should consider using Git if
you want to work on complex projects, or if you want to work on
multiple projects concurrently.
@node Setting up
@subsection Setting up
-
-TODO: Remove this note if incorporating Windows instructions
-throughout this section:
-
@warning{These instructions assume that you are using the
command-line version of Git 1.5 or higher. Windows users should
skip to @ref{Git on Windows}.}
@node Installing Git
@unnumberedsubsubsec Installing Git
-
If you are using a Unix-based machine, the easiest way to download
and install Git is through a package manager such as @command{rpm}
-or @command{apt-get}---the installation is generally automatic.
+or @command{apt-get} -- the installation is generally automatic.
The only required package is (usually) called @command{git-core},
although some of the auxiliary @command{git@var{*}} packages are
also useful (such as @command{gitk}).
(@uref{http://git-scm.com/}) for downloadable binaries and
tarballs.
-TODO: add Windows installation instructions (or @@ref@{Git on
-Windows@}).
-
@node Initializing a repository
@unnumberedsubsubsec Initializing a repository
-
-Once Git is installed, you'll need to create a new directory where
-your initial repository will be stored (the example below uses
-@file{~/lilypond-git/}, where @code{~} represents your home
-directory). Run @command{git@tie{}init} from within the new
-directory to initialize an empty repository:
+Once Git is installed, get a copy of the source code:
@example
-mkdir ~/lilypond-git/; cd ~/lilypond-git/
-git init
+git clone git://git.sv.gnu.org/lilypond.git ~/lilypond-git
@end example
+The above command will put the it in @file{~/lilypond-git}, where
+@code{~} represents your home directory.
+
@subsubheading Technical details
This creates (within the @file{~/lilypond-git/} directory) a
@warning{Throughout the rest of this manual, all command-line
input should be entered from the top directory of the Git
repository being discussed (eg. @file{~/lilypond-git/}). This is
-referred to as a @emph{top source directory}.}
+referred to as the @emph{top source directory}.}
-Before downloading a copy of the main LilyPond repository, you
+Before working with the copy of the main LilyPond repository, you
should configure some basic settings with the
@command{git@tie{}config} command. Git allows you to set both
global and repository-specific options.
To configure settings that affect all repositories, use the
-@command{--global} command line option. For example, the first
+@option{--global} command line option. For example, the first
two options that you should always set are your @var{name} and
@var{email}, since Git needs these to keep track of commit
authors:
git config --global core.editor @var{nano}
@end example
-TODO: Add instructions for changing the editor on Windows, which
-is a little different, I think. -mp
-
@subsubheading Technical details
Git stores the information entered with
@end example
Using the @command{git@tie{}config} command @emph{without} the
-@command{--global} option configures repository-specific settings,
+@option{--global} option configures repository-specific settings,
which are stored in the file @file{.git/config}. This file is
created when a repository is initialized (using
@command{git@tie{}init}), and by default contains these lines:
@end example
If you're tracking the remote @code{master} branch, you should add
-the @code{-r} option (short for @code{--rebase}) to keep commits
+the @option{-r} option (short for @option{--rebase}) to keep commits
on your local branch current:
@example
@end example
If you don't edit translated documentation and don't want to type
-@code{-r} every time, configure the master branch to rebase by
+@option{-r} every time, configure the master branch to rebase by
default with this command:
@example
@end example
Git will ask you for confirmation if it sees that data would be
-lost by deleting the branch. Use @code{-D} instead of @code{-d}
+lost by deleting the branch. Use @option{-D} instead of @option{-d}
to bypass this. Note that you cannot delete a branch if it is
currently checked out.
* Making commits::
* Commit messages::
* Making patches::
+* Uploading a patch for review::
@end menu
@end example
@noindent
-The @code{-a} is short for @code{--all} which includes modified
+The @option{-a} is short for @option{--all} which includes modified
and deleted files, but only those newly created files that have
previously been added.
@example
-# Please enter the commit message for your changes. Lines starting
+# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# On branch master
# Changes to be committed:
@node Making patches
@unnumberedsubsubsec Making patches
-
If you want to share your changes with other contributors and
developers, you need to generate @emph{patches} from your commits.
+We prefer it if you follow the instructions in
+@ref{Uploading a patch for review}. However, we present an
+alternate method here.
+
You should always run @command{git@tie{}pull@tie{}-r} (translators
-should leave off the @code{-r}) before doing this to ensure that
+should leave off the @option{-r}) before doing this to ensure that
your patches are as current as possible.
Once you have made one or more commits in your local repository,
reviewed, the developers may push one or more of them to the main
repository or discuss them with you.
-@seealso
-If your patch includes a significant amount of code, you may want
-to see @ref{Adding or modifying features}, especially @emph{Post
-patch for comments}.
+@node Uploading a patch for review
+@unnumberedsubsubsec Uploading a patch for review
+
+Any non-trivial change should be uploaded to our @qq{Rietveld}
+code review website:
+
+@example
+@uref{http://codereview.appspot.com/}
+@end example
+
+@subsubheading @command{git-cl} install
+
+LilyDev users should skip over these @q{install} instructions.
+
+@enumerate
+
+@item
+Install @command{git-cl} by entering:
+
+@example
+git clone https://github.com/gperciva/git-cl.git
+@end example
+
+If that command fails for some reason, try this instead:
+
+@example
+git clone git://github.com/gperciva/git-cl.git
+@end example
+
+@item
+Add the @file{git-cl/} directory to your PATH, or create a
+symbolic link to the @command{git-cl} and @command{upload.py}
+scripts in one of your PATH directories (such as
+@file{$HOME/bin}).
+
+@end enumerate
+
+@subsubheading @command{git-cl} configuration
+
+LilyDev users should perform these @q{configuration} instructions.
+
+@enumerate
+@item
+You must have a google account; please create one if you do not
+have one already.
+
+Note that a google account does not need to be a gmail account; you can
+use any email address for your google account when you sign up.
+
+@item
+Move into the top source directory and then configure @command{git
+cl} with the following commands. If you do not understand any
+question, just answer with a newline (CR).
+
+@example
+cd $HOME/lilypond-git/
+git cl config
+@end example
+
+The @qq{CC list} question should be answered with:
+
+@example
+lilypond-devel@@gnu.org
+@end example
+
+@end enumerate
+
+@subsubheading Uploading patch set
+
+@warning{Unless you are familiar with branches, only work on one
+set of changes at once.}
+
+There are two methods, depending on your git setup.
+
+@itemize
+@item
+@strong{Master branch}: (easy option, and used in @command{lily-git.tcl})
+
+If you added your patch to @code{master}, then:
+
+@example
+git pull -r
+git cl upload origin/master
+@end example
+
+If you have git push ability, make sure that you @emph{remove}
+your patch (with @command{git rebase} or @command{git reset})
+before pushing other stuff.
+
+@c don't make this one an @example; we don't want to make it easy
+@c for people to use this accidently
+Notifications of patches are automatically added to our issue
+tracker to reduce the chance of patches getting lost. To suppress
+this (not recommended), add the @code{-n / --no-code-issue}
+option.
+
+@item
+@strong{Separate branch}: (complicated option)
+
+Ensure your changes are committed in a separate branch, which
+should differ from the reference branch to be used by just the
+changes to be uploaded. If the reference branch is to be
+origin/master, ensure this is up-to-date. If necessary, use git
+rebase to rebase the branch containing the changes to the head of
+origin/master. Finally, check out branch with the changes and
+enter the command:
+
+@example
+git cl upload <reference SHA1 ID>
+@end example
+
+@noindent
+where <reference SHA1 ID> is the SHA1 ID of the commit to be used
+as a reference source for the patch. Generally, this will be the
+SHA1 ID of origin/master, and in that case the command:
+
+@example
+git cl upload origin/master
+@end example
+
+@noindent
+can be used.
+
+@end itemize
+
+After prompting for your Google email address and password, the
+patch set will be posted to Rietveld, and you will be given a URL
+for your patch.
+
+@warning{Some installations of git-cl fail when uploading a patch
+with certain filename extensions. If this happens, it can
+generally be fixed by editing the list of exceptions at top of
+@file{git-cl.py}.}
+
+@subsubheading Announcing your patch set
+
+You should then announce the patch by logging into the code review
+issue webpage and using @qq{Publish + Mail Comments} to add a
+(mostly bogus) comment to your issue. The text of your comment
+will be sent to our developer mailing list.
+
+@warning{There is no automatic notification of a new patch; you
+must add a comment yourself.}
+
+@subsubheading Revisions
+
+As revisions are made in response to comments, successive patch sets
+for the same issue can be uploaded by reissuing the git-cl command
+with the modified branch checked out.
+
+Sometimes in response to comments on revisions, the best way to
+work may require creation of a new branch in git. In order to
+associate the new branch with an existing Rietveld issue,
+the following command can be used:
+
+@example
+git cl issue issue-number
+@end example
+
+@noindent
+where @code{issue-number} is the number of the existing Rietveld
+issue.
+
+@subsubheading Resetting git cl
+
+If @command{git cl} becomes confused, you can @qq{reset} it by
+running:
+
+@example
+git cl issue 0
+@end example
+
+@subsubheading Wait for a countdown
+
+Your patch will be available for reviews for the next few hours or
+days. Three times a week, patches with no known problems are
+gathered into a @qq{patch countdown} and their status changed to
+@code{patch-countdown}. The countdown is a 48-hour waiting period
+in which any final reviews or complaints should be made.
+
+During the countdown, your patch may be set to
+@code{patch-needs_work}, indicating that you should fix something
+(or at least discuss why the patch needs no modification). If no
+problems are found, the patch will be set to @code{patch-push}.
+
+Once a patch has @code{patch-push}, it should be sent to your
+mentor for uploading. If you have git push ability, look at
+@ref{Pushing to staging}.
@node Advanced Git procedures
* Git log::
* Applying remote patches::
* Sending and receiving patches via email::
+* Cleaning up multiple patches::
* Commit access::
+* Pushing to staging::
@end menu
A @emph{branch} is nothing more than a pointer to a particular
commit, which is called the @emph{head} of the branch; when
-referring to a branch, one often acutally thinks about its head
+referring to a branch, one often actually thinks about its head
and the ancestor commits of the head.
Now we will explain the two last commands you used to get the
Sometimes git will become hopelessly confused, and you just want
to get back to a known, stable state. This command destroys any
-local changes you have made, but at least you get back to the
-current online version:
+local changes you have made in the currently checked-out branch,
+but at least you get back to the current online version:
@example
git reset --hard origin/master
The commands above don't only bring you the latest version of the
-sources, but also the full history of revisions (revisons, also
+sources, but also the full history of revisions (revisions, also
called commits, are changes made to the sources), stored in the
@file{.git} directory. You can browse this history with
@uref{http://wiki.winehq.org/GitWine}.
+@node Cleaning up multiple patches
+@subsection Cleaning up multiple patches
+
+If you have been developing on your own branch for a while, you
+may have more commmits than is really sensible. To revise your
+work and condense commits, use:
+
+@example
+git rebase origin/master
+git rebase -i origin/master
+@end example
+
+@warning{Be a bit cautious -- if you completely remove commits
+during the interactive session, you will... err... completely
+remove those commits.}
+
+
@node Commit access
@subsection Commit access
-
Most contributors are not able to commit patches directly to the
main repository---only members of the LilyPond development team
have @emph{commit access}. If you are a contributor and are
certificate in your browser, given at
@uref{http://savannah.gnu.org/tls/tutorial/}.
+@warning{Savannah will silently put your username in lower-case --
+do not try to use capital letters.}
+
@item
After registering, if you are not logged in automatically, login
@item
-Generate an SSH @q{dsa} key pair. Enter the following at the
+Generate an SSH @q{rsa} key pair. Enter the following at the
command prompt:
@example
-ssh-keygen -t dsa
+ssh-keygen -t rsa
@end example
When prompted for a location to save the key, press <ENTER> to
-accept the default location (@file{~/.ssh/id_dsa}).
+accept the default location (@file{~/.ssh/id_rsa}).
Next you are asked to enter an optional passphrase. On most
systems, if you use a passphrase, you will likely be prompted for
You can change/enable/disable your passphrase at any time with:
@example
-ssh-keygen -f ~/.ssh/id_dsa -p
+ssh-keygen -f ~/.ssh/id_rsa -p
@end example
Note that the GNOME desktop has a feature which stores your
@end example
After setting up your passphrase, your private key is saved as
-@file{~/.ssh/id_dsa} and your public key is saved as
-@file{~/.ssh/id_dsa.pub}.
+@file{~/.ssh/id_rsa} and your public key is saved as
+@file{~/.ssh/id_rsa.pub}.
@item
-Register your public SSH @q{dsa} key with Savannah. From the
+Register your public SSH @q{rsa} key with Savannah. From the
@qq{My Account Configuration} page, click on @qq{Edit SSH Keys},
-then paste the contents of your @file{~/.ssh/id_dsa.pub} file into
+then paste the contents of your @file{~/.ssh/id_rsa.pub} file into
one of the @qq{Authorized keys} text fields, and click
@qq{Update}.
@end example
@noindent
-where @var{user} is your username on Savannah.
+replacing @var{user} with your Savannah username.
@item
-After your membership has been activated and you’ve configured Git
+After your membership has been activated and you've configured Git
to use SSH, test the connection with:
@example
@item
Test your commit access with a dry run:
+@warning{Do not push directly to master; instead, push to staging.
+See @ref{Pushing to staging}.}
+
@example
git push --dry-run --verbose
@end example
@noindent
Then @code{git@tie{}push} should work as before. For more
details, consult the @code{git@tie{}push} man page.
-@end enumerate
+@item
+Repeat the steps from generating an RSA key through to testing
+your commit access, for each machine from which you will be
+making commits, or you may simply copy the files from your
+local @file{~/.ssh} folder to the same folder on the other
+machine.
+
+@end enumerate
+
@subsubheading Technical details
@itemize
over ATM. If this problem is encountered a possible work-around is
to set the MTU in the local router to 1500.
+
+@node Pushing to staging
+@subsection Pushing to staging
+
+Do not push directly to the git @code{master} branch. Instead,
+push to @code{staging}.
+
+You will not see your patch on @code{origin/master} until some
+automatic tests have been run. These tests are run every couple
+of hours; please wait at least 12 hours before wondering if your
+patch has been lost. Note that you can check the commits on
+@code{origin/staging} by looking at the git web interface on
+savannah.
+
+@subsubheading If your work is in a patch file
+
+Assuming that your patch is in a file called
+@file{0001-my-patch.patch}, and you are currently on git master,
+do:
+
+@example
+git checkout staging
+git pull -r
+git am 0001-my-patch.patch
+git push origin staging
+git checkout master
+@end example
+
+@subsubheading If your work is in a branch
+
+If you are working on branches and your work in is
+@code{my_branch_name}, then do:
+
+@example
+git checkout staging
+git pull -r
+git merge my_branch_name
+git push origin staging
+@end example
+
+
+
@node Git on Windows
@section Git on Windows
+@warning{We heavily recommend that development be done with our
+virtual machine @ref{Lilydev}.}
+
@c Some of this may duplicate stuff in other sections
@c But it is probably best for windows users to have it all together
@c If necessary, clear this up later -td
Git is a system for tracking the changes made to source files by a
distributed set of editors. It is designed to work without a
-master repository, but we have chosen to have a master respository
+master repository, but we have chosen to have a master repository
for LilyPond files. Editors hold a local copy of the master
repository together with any changes they have made locally.
Local changes are held in a local @q{branch}, of which there may
@subsection Git GUI
Almost all subsequent work will use the Git Graphical User
-Interface, which avoids having to type command line commands. To
+Interface, which avoids having to type command line commands. To
start Git GUI first start the Git bash shell by clicking on the
desktop icon, and type
projects / lilypond.git / blobdiff