1 @c -*- coding: utf-8; mode: texinfo; -*-
4 @node Working with source code
5 @chapter Working with source code
7 @warning{New contributors should read @ref{Quick start}, and in
8 particular @ref{lily-git}, instead of this chapter.}
10 Advanced contributors will find this material quite useful,
11 particularly if they are working on major new features.
14 * Manually installing lily-git.tcl::
16 * Basic Git procedures::
17 * Advanced Git procedures::
19 * Repository directory structure::
20 * Other Git documentation::
24 @node Manually installing lily-git.tcl
25 @section Manually installing lily-git.tcl
27 We have created an easy-to-use GUI to simplify git for new
28 contributors. If you are comfortable with the command-line, then
29 skip ahead to @ref{Starting with Git}.
31 @warning{These instructions are only for people who are @emph{not}
34 @c there's some duplication in this section with stuff covered in
35 @c Quick Start, but moving it into a macro inside included/ would
36 @c be getting a bit icky. -gp
40 If you haven't already, download and install Git.
44 @item Windows users: download the @code{.exe} file labeled
45 @qq{Full installer for official Git} from:
48 @uref{https://git-for-windows.github.io/}
51 @item Other operating systems: either install @command{git} with
52 your package manager, or download it from the @qq{Binaries}
56 @uref{http://git-scm.com/download}
63 Download the @command{lily-git.tcl} script from:
65 @c don't change the cgit link below to gitweb; gitweb uses
66 @c long filenames like "scripts_auxiliar_lily-git.tcl"
69 @uref{http://git.sv.gnu.org/cgit/lilypond.git/plain/scripts/auxiliar/lily-git.tcl}
73 To run the program from the command line, navigate to the
74 directory containing @command{lily-git.tcl} and enter:
81 Click on the @qq{Get source} button.
83 This will create a directory called @file{lilypond-git/} within
84 your home directory, and will download the source code into that
85 directory (around 150@tie{}Mb). When the process is finished, the
86 @qq{Command output} window will display @qq{Done}, and the button
87 label will change to say @qq{Update source}.
90 Navigate to the @file{lilypond-git/} directory to view the source
95 @warning{Throughout the rest of this manual, most command-line
96 input should be entered from @file{$LILYPOND_GIT}. This is
97 referred to as the @emph{top source directory}.}
99 Further instructions are in @ref{How to use lily-git}.
102 @node Starting with Git
103 @section Starting with Git
105 Using the Git program directly (as opposed to using the
106 @command{lily-git.tcl} GUI) allows you to have much greater control
107 over the contributing process. You should consider using Git if
108 you want to work on complex projects, or if you want to work on
109 multiple projects concurrently.
114 * Git for the impatient::
115 * Other repositories::
116 * Downloading remote branches::
121 @subsection Setting up
123 @warning{These instructions assume that you are using the
124 command-line version of Git 1.5 or higher. Windows users should
125 skip to @ref{Git on Windows}.}
129 * Initializing a repository::
135 @unnumberedsubsubsec Installing Git
137 If you are using a Unix-based machine, the easiest way to download
138 and install Git is through a package manager such as @command{rpm}
139 or @command{apt-get} -- the installation is generally automatic.
140 The only required package is (usually) called @command{git-core},
141 although some of the auxiliary @command{git@var{*}} packages are
142 also useful (such as @command{gitk}).
144 Alternatively, you can visit the Git website
145 (@uref{http://git-scm.com/}) for downloadable binaries and
149 @node Initializing a repository
150 @unnumberedsubsubsec Initializing a repository
152 Once Git is installed, get a copy of the source code:
155 git clone git://git.sv.gnu.org/lilypond.git ~/lilypond-git
158 The above command will put the it in @file{~/lilypond-git}, where
159 @code{~} represents your home directory.
161 @subsubheading Technical details
163 This creates (within the @file{$LILYPOND_GIT} directory) a
164 subdirectory called @file{.git/}, which Git uses to keep track of
165 changes to the repository, among other things. Normally you don't
166 need to access it, but it's good to know it's there.
169 @node Configuring Git
170 @unnumberedsubsubsec Configuring Git
172 @warning{Throughout the rest of this manual, all command-line
173 input should be entered from the top directory of the Git
174 repository being discussed (eg. @file{$LILYPOND_GIT}). This is
175 referred to as the @emph{top source directory}.}
177 Before working with the copy of the main LilyPond repository, you
178 should configure some basic settings with the
179 @command{git@tie{}config} command. Git allows you to set both
180 global and repository-specific options.
182 To configure settings that affect all repositories, use the
183 @option{--global} command line option. For example, the first
184 two options that you should always set are your @var{name} and
185 @var{email}, since Git needs these to keep track of commit
189 git config --global user.name "@var{John Smith}"
190 git config --global user.email @var{john@@example.com}
193 To configure Git to use colored output where possible, use:
196 git config --global color.ui auto
199 The text editor that opens when using @command{git@tie{}commit}
200 can also be changed. If none of your editor-related environment
201 variables are set ($GIT_EDITOR, $VISUAL, or $EDITOR), the default
202 editor is usually @command{vi} or @command{vim}. If you're not
203 familiar with either of these, you should probably change the
204 default to an editor that you know how to use. For example, to
205 change the default editor to @command{nano}, enter:
208 git config --global core.editor @var{nano}
211 Finally, and in some ways most importantly, let's make sure that
212 we know what branch we're on. If you're not using LilyDev, add
213 this to your @file{~/.bashrc}:
216 export PS1="\u@\h \w\$(__git_ps1)$ "
219 You may need to install the additional @code{bash-completion}
220 package, but it is definitely worth it. After installation
221 you must log out, and then log back in again to enable it.
224 @subsubheading Technical details
226 Git stores the information entered with
227 @command{git@tie{}config@tie{}--global} in the file
228 @file{.gitconfig}, located in your home directory. This file can
229 also be modified directly, without using
230 @command{git@tie{}config}. The @file{.gitconfig} file generated
231 by the above commands would look like this:
236 email = john@@example.com
243 Using the @command{git@tie{}config} command @emph{without} the
244 @option{--global} option configures repository-specific settings,
245 which are stored in the file @file{.git/config}. This file is
246 created when a repository is initialized (using
247 @command{git@tie{}init}), and by default contains these lines:
251 repositoryformatversion = 0
254 logallrefupdates = true
257 However, since different repository-specific options are
258 recommended for different development tasks, it is best to avoid
259 setting any now. Specific recommendations will be mentioned later
263 @node Git for the impatient
264 @subsection Git for the impatient
266 @advanced{The intent of this subsection is to get you working on lilypond as
267 soon as possible. If you want to learn about git, go read
268 @ref{Other Git documentation}.
270 Also, these instructions are designed to eliminate the most common
271 problems we have found in using git. If you already know git and
272 have a different way of working, great! Feel free to ignore the
273 advice in this subsection.}
276 Ok, so you've been using @command{lily-git.tcl} for a while, but
277 it's time to take the next step. Since our review process delays
278 patches by 60-120 hours, and you want to be able to work on other
279 stuff while your previous work is getting reviewed, you're going
280 to use @strong{branches}.
282 You can think of a branch as being a separate copy of the source
283 code. But don't worry about it.
285 @subsubheading Start work: make a new branch
287 Let's pretend you want to add a section to the Contributor's Guide
288 about using branches.
290 Start by updating the repository, then making a new branch. Call
291 the branch anything you want as long as the name starts with
292 @code{dev/}. Branch names that don't begin with @code{dev/} are
293 reserved for special things in lilypond.
297 git pull -r origin master
301 @subsubheading Switch to that branch
303 Nothing has happened to the files yet. Let's change into the new
304 branch. You can think of this as @qq{loading a file}, although in
305 this case it's really @qq{loading a directory and subdirectories
312 Your prompt now shows you that you're on the other branch:
315 gperciva@@LilyDev:~/lilypond-git (dev/cg)$
318 To be able to manage multiple lilypond issues at once, you'll need to switch
319 branches. You should have each lilypond issue on a separate branch.
320 Switching branches is easy:
324 git checkout origin/staging
325 git checkout origin/release/unstable
329 Branches that begin with @code{origin/} are part of the remote repository,
330 rather than your local repository, so when you check them out you get a
331 temporary local branch. You should never make changes directly on a
332 branch beginning with @code{origin/}. You get changes into the remote
333 repository by making them in local branches, and then pushing them to
334 @code{origin/staging} as described below.
336 @subsubheading Make your changes
338 Edit files, then commit them.
345 Remember how I said that switching to a branch was like
346 @qq{loading a directory}? Well, you've just @qq{saved a
347 directory}, so that you can @qq{load} it later.
349 @advanced{If you have used @command{cvs} or @command{svn}, you may
350 be very confused: those programs use @qq{commit} to mean
351 @qq{upload my changes to the shared source repository}.
352 Unfortunately, just to be different, @w{@command{git commit}}
353 means @qq{save my changes to the files}.}
355 When you create a new file, you need to add it to git, then commit it:
358 git add input/regression/avoid-crash-on-condition.ly
363 Edit more files. Commit them again. Edit yet more files, commit
364 them again. Go eat dinner. Switch to @code{master} so you can
365 play with the latest changes from other developers. Switch back
366 to your branch and edit some more. Commit those changes.
368 At this stage, don't worry about how many commits you have.
371 @subsubheading Save commits to external files
373 Branches are nerve-wracking until you get used to them. You can
374 save your hard work as individual @file{.patch} files. Be sure to
375 commit your changes first.
379 git format-patch master
382 I personally have between 4 and 20 of those files saved in a
383 special folder at any point in time. Git experts might laugh as
384 that behavior, but I feel a @emph{lot} better knowing that I've
388 @subsubheading Prepare your branch for review
390 After committing, you can update your branch with the latest master:
395 git pull -r origin master
401 Due to the speed of lilypond development, sometimes
402 @code{master} has changed so much that your branch can no
403 longer be applied to it. In that happens, you will have a merge
404 conflict. Stop for a moment to either cry or have a stiff drink,
405 then proceed to @ref{Merge conflicts}.
408 @subsubheading Upload your branch
410 Finally, you're finished your changes. Time to upload for review.
411 Make sure that you're on your branch, then upload:
419 @subsubheading Wait for reviews
421 While you're waiting for a countdown and reviews, go back to
422 master, make a @code{dev/doc-beams} branch, and start adding doc
423 suggestions from issue 12345 from the tracker. Or make a
424 @code{dev/page-breaks} and fix bug in page breaking. Or whatever.
425 Don't worry, your @code{dev/cg} is safe.
428 @subsubheading Combining commits (optional unless you have broken commits)
430 Does the history of your branch look good?
436 If you have a lot of commits on your branch, you might want to
437 combine some of them. Alternately, you may like your commits, but
438 want to edit the commit messages.
444 Follow instructions on the screen.
446 @warning{This step gives you the power to completely lose your
447 work. Make a backup of your commits by saving them to
448 @file{.patch} files before playing with this. If you do lose
449 your work, don't despair. You can get it back by using @code{git reflog}.
450 The use of @code{git reflog} is not covered here.}
452 @warning{If any of the commits on your branch represent partial work that will
453 not pass @var{make && make doc}, you @strong{must} squash these
454 commits into a working commit. Otherwise, your push will break staging
455 and will not be able to be merged to master. In general, you will
456 be safer to have one commit per push.}
459 @subsubheading Push to staging
461 When you've got the coveted @code{Patch-push} status, time to
466 git rebase origin/staging dev/cg~0
470 @warning{Do not skip the @command{gitk} step; a quick 5-second
471 check of the visual history can save a great deal of frustration
472 later on. You should see a set of your commits that are ahead of
473 @code{origin/staging}, with no label for the top commit -- only a
476 @warning{If @code{origin/staging} and @code{origin/master} are the
477 same commit, your branch (@code{dev/cg} in the example) will also
478 be at the top of the @code{gitk} tree. This is normal.}
480 If everything looks good, push it:
483 git push origin HEAD:staging
486 Then change back to your working branch:
492 @warning{It is a best practice to avoid rebasing any of your branches
493 to @code{origin/staging}. If @code{origin/staging} is broken, it
494 will be deleted and rebuilt. If you have rebased one of your branches
495 to @code{origin/staging}, the broken commits can end up in your branch.
496 The commands given above do the rebase on a temporary branch, and avoid
497 changing your working branch.}
500 @subsubheading Delete your branch (safe)
502 After a few hours, if there's nothing wrong with your branch, it
503 should be automatically moved to @code{origin/master}. Update,
504 then try removing your branch:
508 git pull -r origin master
512 The last command will fail if the contents of @code{dev/cg} are
513 not present in @code{origin/master}.
516 @subsubheading Delete your branch (UNSAFE)
518 @c don't give explicit commands here -- this is too dangerous to copy and paste
519 Sometimes everything goes wrong. If you want to remove a branch even though
520 it will cause your work to be lost (that is, if the contents of @code{dev/cg}
521 are @strong{not} present in master), follow the instructions in @qq{Delete
522 your branch (safe)}, but replace the @code{-d} on the final line with
526 @node Other repositories
527 @subsection Other repositories
529 We have a few other code repositories.
533 * Grand Unified Builder (GUB)::
535 * yet more repositories::
539 @unnumberedsubsubsec lilypond-extra
541 There is a separate repository for general administrative scripts,
542 as well as pictures and media files for the website. People
543 interested in working on the website should download this
544 repository, and set their @code{$LILYPOND_WEB_MEDIA_GIT}
545 environment variable to point to that repository.
548 @uref{https://github.com/gperciva/lilypond-extra}
551 To configure an environment variable in bash (the default for most
552 GNU/Linux distributions),
555 export LILYPOND_WEB_MEDIA_GIT=$HOME/dir/of/lilypond-extra/
558 Be aware that @code{lilypond-extra} is the definitive source for some binary
559 files - in particular PDF versions of papers concerning LilyPond. To add
560 further PDFs of this sort, all that is necessary is to add the PDF to
561 @code{lilypond-extra} and then add a reference to it in the documentation. The
562 file will then be copied to the website when @code{make website} is run.
564 However, pictures that are also used in the documentation build are mastered in
565 the main git repository. If any of these is changed, it should be updated in
566 git, and then the updates copied to @code{lilypond-extra}.
568 @node Grand Unified Builder (GUB)
569 @unnumberedsubsubsec Grand Unified Builder (GUB)
571 Another item of interest might be the Grand Unified Builder, our
572 cross-platform building tool. Since it is used by other projects as
573 well, it is not stored in our gub repository. For more info, see
574 @uref{http://lilypond.org/gub}.
576 There are two locations for this repository: the version being used to
577 build lilypond, which is at
580 @uref{http://github.com/gperciva/gub}
583 and the original version by Jan Nieuwenhuizen, kept at
586 @uref{http://github.com/janneke/gub}
591 @unnumberedsubsubsec LilyPad
593 Our binary releases on MacOS X and Windows contain a lightweight
596 To make any modifications the Windows editor, you will need to do the
601 Clone the git repository from @code{https://github.com/gperciva/lilypad}
604 Make changes to the source, and check it compiles. In a Windows environment
605 @code{MinGW} provides both a @code{Git} installation and a @code{gcc}
606 compiler. This can be obtained from @code{http://www.mingw.org/}
609 Update the version which is contained in the @file{rsrc.rc}. Check
613 Commit the changes with an informative commit message.
616 Push the changes to github. You will need to use syntax similiar to this:
619 git push https://UserName@@github.com/gperciva/lilypad.git
622 You will need to have push access to the git repository for this to be
626 Make a tarball of the source code to be used by GUB by pulling the updated
627 repository from GitHub. Ensure that the tarball has the correct Version
631 Copy the tarball to @code{http://lilypond.org/downloads/gub-sources/lilypad/}.
632 You will need to have SSH access to @code{lilypond.org}. If you do not, contact
633 the Release Manager via the lilypond-devel mailing list.
636 Update GUB to make it use the new tarball by editing
637 @file{gub/specs/lilypad.py} and changing the @code{source =} line to point to
641 Push this updated @file{lilypad.py} version to the GUB repository on GitHub.
644 Test the changes with a new GUB compile.
648 @node yet more repositories
649 @unnumberedsubsubsec yet more repositories
651 There are a few other repositories floating around, which will
652 hopefully be documented in the near future.
655 @node Downloading remote branches
656 @subsection Downloading remote branches
658 @warning{contains obsolete + misleading info}
661 * Organization of remote branches::
662 * LilyPond repository sources::
663 * Downloading individual branches::
664 * Downloading all remote branches::
669 @node Organization of remote branches
670 @unnumberedsubsubsec Organization of remote branches
673 The main LilyPond repository is organized into @emph{branches} to
674 facilitate development. These are often called @emph{remote}
675 branches to distinguish them from @emph{local} branches you might
676 create yourself (see @ref{Using local branches}).
678 The @code{master} branch contains all the source files used to
679 build LilyPond, which includes the program itself (both stable and
680 development releases), the documentation (and its translations),
681 and the website. Generally, the @code{master} branch is expected
682 to compile successfully.
684 The @code{translation} branch is a side branch that
685 allows translators to work without needing to worry about
686 compilation problems. Periodically, the Translation Meister
687 (after verifying that it doesn't break compilation), will
688 @emph{merge} this branch into @code{staging} to incorporate
689 recent translations. Similarly, the @code{master} branch is
690 usually merged into the @code{translation} branch after
691 significant changes to the English documentation. See
692 @ref{Translating the documentation} for details.
695 @node LilyPond repository sources
696 @unnumberedsubsubsec LilyPond repository sources
699 The recommended source for downloading a copy of the main
703 git://git.sv.gnu.org/lilypond.git
706 However, if your internet router filters out connections using the
707 GIT protocol, or if you experience difficulty connecting via GIT,
708 you can try these other sources:
711 ssh://git.sv.gnu.org/srv/git/lilypond.git
712 http://git.sv.gnu.org/r/lilypond.git
715 The SSH protocol can only be used if your system is properly set
716 up to use it. Also, the HTTP protocol is slowest, so it should
717 only be used as a last resort.
720 @node Downloading individual branches
721 @unnumberedsubsubsec Downloading individual branches
723 @warning{obsolete, should be deleted!}
726 Once you have initialized an empty Git repository on your system
727 (see @ref{Initializing a repository}), you can download a remote
728 branch into it. Make sure you know which branch you want to start
731 To download the @code{master} branch, enter the following:
734 git remote add -ft master -m master \
735 origin git://git.sv.gnu.org/lilypond.git/
738 To download the @code{translation} branch, enter:
741 git remote add -ft translation -m \
742 translation origin git://git.sv.gnu.org/lilypond.git/
745 The @command{git@tie{}remote@tie{}add} process could take up to
746 ten minutes, depending on the speed of your connection. The
747 output will be something like this:
751 remote: Counting objects: 235967, done.
752 remote: Compressing objects: 100% (42721/42721), done.
753 remote: Total 235967 (delta 195098), reused 233311 (delta 192772)
754 Receiving objects: 100% (235967/235967), 68.37 MiB | 479 KiB/s, done.
755 Resolving deltas: 100% (195098/195098), done.
756 From git://git.sv.gnu.org/lilypond
757 * [new branch] master -> origin/master
758 From git://git.sv.gnu.org/lilypond
759 * [new tag] flower/1.0.1 -> flower/1.0.1
760 * [new tag] flower/1.0.10 -> flower/1.0.10
762 * [new tag] release/2.9.6 -> release/2.9.6
763 * [new tag] release/2.9.7 -> release/2.9.7
766 When @command{git@tie{}remote@tie{}add} is finished, the remote
767 branch should be downloaded into your repository---though not yet
768 in a form that you can use. In order to browse the source code
769 files, you need to @emph{create} and @emph{checkout} your own
770 local branch. In this case, however, it is easier to have Git
771 create the branch automatically by using the @command{checkout}
772 command on a non-existent branch. Enter the following:
775 git checkout -b @var{branch} origin/@var{branch}
779 where @code{@var{branch}} is the name of your tracking branch,
780 either @code{master} or @code{translation}.
782 Git will issue some warnings; this is normal:
785 warning: You appear to be on a branch yet to be born.
786 warning: Forcing checkout of origin/master.
787 Branch master set up to track remote branch master from origin.
791 By now the source files should be accessible---you should be able
792 to edit any files in the @file{$LILYPOND_GIT} directory using a
793 text editor of your choice. But don't start just yet! Before
794 editing any source files, learn how to keep your changes organized
795 and prevent problems later---read @ref{Basic Git procedures}.
797 @subsubheading Technical Details
799 The @command{git@tie{}remote@tie{}add} command should add some
800 lines to your local repository's @file{.git/config} file:
804 url = git://git.sv.gnu.org/lilypond.git/
805 fetch = +refs/heads/master:refs/remotes/origin/master
809 @node Downloading all remote branches
810 @unnumberedsubsubsec Downloading all remote branches
813 To download all remote branches at once, you can @command{clone}
814 the entire repository:
817 git clone git://git.sv.gnu.org/lilypond.git
822 @unnumberedsubsubsec Other branches
824 Most contributors will never need to touch the other branches. If
825 you wish to do so, you will need more familiarity with Git; please
826 see @ref{Other Git documentation}.
829 @item @code{dev/XYZ}:
830 These branches are for individual developers. They store code
831 which is not yet stable enough to be added to the @code{master}
834 @item @code{stable/XYZ}:
835 The branches are kept for archival reasons.
837 @item @code{archive/XYZ}:
838 The branches are kept for archival reasons.
843 @node Basic Git procedures
844 @section Basic Git procedures
848 * The Git contributor's cycle::
849 * Pulling and rebasing::
850 * Using local branches::
853 * Uploading a patch for review::
854 * The patch review cycle::
858 @node The Git contributor's cycle
859 @subsection The Git contributor's cycle
862 Here is a simplified view of the contribution process on Git:
866 Update your local repository by @emph{pulling} the most recent
867 updates from the remote repository.
870 Edit source files within your local repository's @emph{working
874 @emph{Commit} the changes you've made to a local @emph{branch}.
877 Generate a @emph{patch} to share your changes with the developers.
881 @node Pulling and rebasing
882 @subsection Pulling and rebasing
885 When developers push new patches to the @code{git.sv.gnu.org}
886 repository, your local repository is @strong{not} automatically
887 updated. It is important to keep your repository up-to-date by
888 periodically @emph{pulling} the most recent @emph{commits} from
889 the remote branch. Developers expect patches to be as current as
890 possible, since outdated patches require extra work before they
893 Occasionally you may need to rework some of your own modifications
894 to match changes made to the remote branch (see @ref{Resolving
895 conflicts}), and it's considerably easier to rework things
896 incrementally. If you don't update your repository along the way,
897 you may have to spend a lot of time resolving branch conflicts and
898 reconfiguring much of the work you've already done.
900 Fortunately, Git is able to resolve certain types of branch
901 conflicts automatically with a process called @emph{rebasing}.
902 When rebasing, Git tries to modify your old commits so they appear
903 as new commits (based on the latest updates). For a more involved
904 explanation, see the @command{git-rebase} man page.
906 To pull without rebasing (recommended for translators), use the
910 git pull # recommended for translators
913 If you're tracking the remote @code{master} branch, you should add
914 the @option{-r} option (short for @option{--rebase}) to keep commits
915 on your local branch current:
918 git pull -r # use with caution when translating
921 If you don't edit translated documentation and don't want to type
922 @option{-r} every time, configure the master branch to rebase by
923 default with this command:
926 git config branch.master.rebase true
929 If pull fails because of a message like
932 error: Your local changes to 'Documentation/learning/tutorial.itely'
933 would be overwritten by merge. Aborting.
940 Documentation/learning/tutorial.itely: needs update
941 refusing to pull with rebase: your working tree is not up-to-date
945 it means that you have modified some files in you working tree
946 without committing changes (see @ref{Commits}); you
947 can use the @command{git@tie{}stash} command to work around this:
950 git stash # save uncommitted changes
951 git pull -r # pull using rebase (translators omit "-r")
952 git stash pop # reapply previously saved changes
955 Note that @command{git@tie{}stash@tie{}pop} will try to apply a
956 patch, and this may create a conflict. If this happens, see
957 @ref{Resolving conflicts}.
959 TODO: I think the next paragraph is confusing. Perhaps prepare
960 the reader for new terms `committish' and `head'? -mp
962 @warning{translators and documentation editors, if you have
963 changed committishes in the head of translated files using commits
964 you have not yet pushed to @code{git.sv.gnu.org}, please do not
965 rebase. If you want to avoid wondering whether you should rebase
966 each time you pull, please always use committishes from master
967 and/or translation branch on @code{git.sv.gnu.org}, which
968 in particular implies that you must push your changes to
969 documentation except committishes updates (possibly after having
970 rebased), then update the committishes and push them.}
972 TODO: when committishes automatic conditional update have been
973 tested and documented, append the following to the warning above:
974 Note that using update-committishes make target generally touches
977 @subsubheading Technical details
979 The @command{git@tie{}config} command mentioned above adds the
980 line @code{rebase = true} to the master branch in your local
981 repository's @file{.git/config} file:
986 merge = refs/heads/master
991 @node Using local branches
992 @subsection Using local branches
996 * Creating and removing branches::
997 * Listing branches and remotes::
998 * Checking out branches::
1003 @node Creating and removing branches
1004 @unnumberedsubsubsec Creating and removing branches
1007 Local branches are useful when you're working on several different
1008 projects concurrently. To create a new branch, enter:
1011 git branch @var{name}
1014 To delete a branch, enter:
1017 git branch -d @var{name}
1020 Git will ask you for confirmation if it sees that data would be
1021 lost by deleting the branch. Use @option{-D} instead of @option{-d}
1022 to bypass this. Note that you cannot delete a branch if it is
1023 currently checked out.
1026 @node Listing branches and remotes
1027 @unnumberedsubsubsec Listing branches and remotes
1029 You can get the exact path or URL of all remote branches by
1036 To list Git branches on your local repositories, run
1039 git branch # list local branches only
1040 git branch -r # list remote branches
1041 git branch -a # list all branches
1045 @node Checking out branches
1046 @unnumberedsubsubsec Checking out branches
1048 To know the currently checked out branch, i.e. the branch whose
1049 source files are present in your working tree, read the first line
1057 The currently checked out branch is also marked with an asterisk
1058 in the output of @command{git branch}.
1060 You can check out another branch @code{@var{other_branch}}, i.e.
1061 check out @code{@var{other_branch}} to the working tree, by
1065 git checkout @var{other_branch}
1068 Note that it is possible to check out another branch while having
1069 uncommitted changes, but it is not recommended unless you know
1070 what you are doing; it is recommended to run @command{git status}
1071 to check this kind of issue before checking out another branch.
1073 @node Merging branches
1074 @unnumberedsubsubsec Merging branches
1076 To merge branch @code{@var{foo}} into branch @code{@var{bar}},
1077 i.e. to @qq{add} all changes made in branch @code{@var{foo}} to
1078 branch @code{@var{bar}}, run
1081 git checkout @var{bar}
1085 If any conflict happens, see @ref{Resolving conflicts}.
1087 There are common usage cases for merging: as a translator, you will
1088 often want the Translations meister to merge @code{master} into
1089 @code{translation}; on the other hand, the Translations meister wants
1090 to merge @code{translation} into @code{staging} whenever he has
1091 checked that @code{translation} builds successfully.
1098 * Understanding commits::
1099 * How to make a commit::
1104 @node Understanding commits
1105 @unnumberedsubsubsec Understanding commits
1107 Technically, a @emph{commit} is a single point in the history of a
1108 branch, but most developers use the term to mean a @emph{commit
1109 object}, which stores information about a particular revision. A
1110 single commit can record changes to multiple source files, and
1111 typically represents one logical set of related changes (such as a
1112 bug-fix). You can list the ten most recent commits in your
1113 current branch with this command:
1116 git log -10 --oneline
1119 If you're using an older version of Git and get an @q{unrecognized
1120 argument} error, use this instead:
1123 git log -10 --pretty=oneline --abbrev-commit
1126 More interactive lists of the commits on the remote @code{master}
1127 branch are available at
1128 @uref{http://git.sv.gnu.org/gitweb/?p=lilypond.git;a=shortlog} and
1129 @uref{http://git.sv.gnu.org/cgit/lilypond.git/log/}.
1132 @node How to make a commit
1133 @unnumberedsubsubsec How to make a commit
1136 Once you have modified some source files in your working
1137 directory, you can make a commit with the following procedure:
1141 Make sure you've configured Git properly (see @ref{Configuring
1142 Git}). Check that your changes meet the requirements described in
1143 @ref{Code style} and/or @ref{Documentation policy}. For advanced
1144 edits, you may also want to verify that the changes don't break
1145 the compilation process.
1148 Run the following command:
1155 to make sure you're on the right branch, and to see which files
1156 have been modified, added or removed, etc. You may need to tell
1157 Git about any files you've added by running one of these:
1160 git add @var{file} # add untracked @var{file} individually
1161 git add . # add all untracked files in current directory
1165 After @command{git@tie{}add}, run @command{git@tie{}status} again
1166 to make sure you got everything. You may also need to modify
1170 Preview the changes about to be committed (to make sure everything
1178 The @code{HEAD} argument refers to the most recent commit on the
1179 currently checked-out branch.
1182 Generate the commit with:
1189 The @option{-a} is short for @option{--all} which includes modified
1190 and deleted files, but only those newly created files that have
1191 previously been added.
1196 @node Commit messages
1197 @unnumberedsubsubsec Commit messages
1200 When you run the @command{git@tie{}commit@tie{}-a} command, Git
1201 automatically opens the default text editor so you can enter a
1202 @emph{commit message}. If you find yourself in a foreign editing
1203 environment, you're probably in @command{vi} or @command{vim}. If
1204 you want to switch to an editor you're more familiar with, quit by
1205 typing @code{:q!} and pressing @code{<Enter>}. See
1206 @ref{Configuring Git} for instructions on changing the default
1209 In any case, Git will open a text file for your commit message
1210 that looks like this:
1214 # Please enter the commit message for your changes. Lines starting
1215 # with '#' will be ignored, and an empty message aborts the commit.
1217 # Changes to be committed:
1218 # (use "git reset HEAD <file>..." to unstage)
1220 # modified: working.itexi
1224 Your commit message should begin with a one-line summary
1225 describing the change (no more than 50 characters long), and if
1226 necessary a blank line followed by several lines giving the
1229 @c $ git log -1 --pretty=medium 4d6f1e5
1231 Doc: add Baerenreiter and Henle solo cello suites
1233 Added comparison of solo cello suite engravings to new essay with
1234 high-res images, fixed cropping on Finale example.
1237 Commit messages often start with a short prefix describing the
1238 general location of the changes.
1242 Doc: and Doc-@var{**}: If a commit affects the
1243 documentation in English (or in several languages simultaneously)
1244 the commit message should be prefixed with @qq{Doc:@tie{}}. If the
1245 commit affects only one of the translations, the commit
1246 message should be prefixed with @qq{Doc-@var{**}:@tie{}}, where
1247 @var{**} is the two-letter language code.
1250 Web: and Web-@var{**}: Commits that affect the
1251 website should use @qq{Web:@tie{}} for English, and
1252 @qq{Web-@var{**}:@tie{}} for other languages.
1255 Changes to a single file are often prefixed with the name of the file
1259 Visit the links listed in @ref{Understanding commits} for examples.
1267 * How to make a patch::
1268 * Emailing patches::
1271 @node How to make a patch
1272 @unnumberedsubsubsec How to make a patch
1274 If you want to share your changes with other contributors and
1275 developers, you need to generate @emph{patches} from your commits.
1276 We prefer it if you follow the instructions in
1277 @ref{Uploading a patch for review}. However, we present an
1278 alternate method here.
1280 You should always run @command{git@tie{}pull@tie{}-r} (translators
1281 should leave off the @option{-r}) before doing this to ensure that
1282 your patches are as current as possible.
1284 Once you have made one or more commits in your local repository,
1285 and pulled the most recent commits from the remote branch, you can
1286 generate patches from your local commits with the command:
1289 git format-patch origin
1292 The @code{origin} argument refers to the remote tracking branch at
1293 @code{git.sv.gnu.org}. This command generates a separate patch
1294 for each commit that's in the current branch but not in the remote
1295 branch. Patches are placed in the current working directory and
1296 will have names that look something like this:
1299 0001-Doc-Fix-typos.patch
1300 0002-Web-Remove-dead-links.patch
1304 Send an email (must be less than 64 KB) to
1305 @email{lilypond-devel@@gnu.org} briefly explaining your work, with
1306 the patch files attached. Translators should send patches to
1307 @email{translations@@lilynet.net}. After your patches are
1308 reviewed, the developers may push one or more of them to the main
1309 repository or discuss them with you.
1312 @node Emailing patches
1313 @unnumberedsubsubsec Emailing patches
1315 The default @code{x-diff} MIME type associated with patch files
1316 (i.e., files whose name ends in @code{.patch}) means that the
1317 encoding of line endings may be changed from UNIX to DOS format
1318 when they are sent as attachments. Attempting to apply such an
1319 inadvertently altered patch will cause git to fail with a message
1320 about @q{whitespace errors}.
1322 The solution to such problems is surprisingly simple---just change
1323 the default file extension of patches generated by git to end in
1324 @code{.txt}, for example:
1327 git config format.suffix '.patch.txt'
1330 This should cause email programs to apply the correct base64
1331 encoding to attached patches.
1333 If you receive a patch with DOS instead of UNIX line-endings, it
1334 can be converted back using the @code{dos2unix} utility.
1336 Lots of useful information on email complications with patches is
1337 provided on the Wine wiki at
1338 @uref{http://wiki.winehq.org/GitWine}.
1341 @node Uploading a patch for review
1342 @subsection Uploading a patch for review
1344 Any non-trivial change should be uploaded to our @qq{Rietveld}
1345 code review website:
1348 @uref{http://codereview.appspot.com/}
1351 You can upload a patch for review by using our custom @code{git-cl}
1352 @q{helper-script}. This section assumes you have already installed,
1353 updated, and configured @code{git-cl}. See @ref{git-cl}.
1355 @warning{Unless you are familiar with branches, only work on one
1356 set of changes at once.}
1358 There are two methods, depending on your git setup.
1362 @strong{Master branch}: (easy option)
1364 If you added your patch to @code{master}, then:
1368 git-cl upload origin/master
1371 @c Mention staging here?
1372 If you have git push ability, make sure that you @emph{remove}
1373 your patch (with @command{git rebase} or @command{git reset})
1374 before pushing other stuff.
1376 @c don't make this one an @example; we don't want to make it easy
1377 @c for people to use this accidently
1378 Notifications of patches are automatically added to our issue
1379 tracker to reduce the chance of patches getting lost. To suppress
1380 this (not recommended), add the @code{-n / --no-code-issue}
1384 @strong{Separate branch}: (complicated option)
1386 Ensure your changes are committed in a separate branch, which should
1387 differ from the reference branch to be used (usually
1388 @code{origin/master}) by just the changes to be uploaded. Checkout the
1389 branch with the changes:
1392 git checkout some-branch-with-changes
1395 If the reference branch is to be @code{origin/master}, ensure that the
1396 branch containing the changes is up-to-date with it. Use
1397 @command{git rebase} or @command{git pull -r} to rebase the branch to
1398 the head of @code{origin/master}. For example:
1401 git pull -r origin master
1404 Finally, start the upload by entering:
1407 git-cl upload <reference SHA1 ID>
1411 where <reference SHA1 ID> is the SHA1 ID of the commit to be used
1412 as a reference source for the patch. Generally, this will be the
1413 SHA1 ID of origin/master, and in that case you can just use the command:
1416 git-cl upload origin/master
1421 First you will see a terminal editor where you can edit the
1422 message that will accompany your patch. @command{git-cl} will
1423 respect the @env{EDITOR} environment variable if defined,
1424 otherwise it will use @command{vi} as the default editor.
1426 After prompting for your Google email address and password, the
1427 patch set will be posted to Rietveld, and you will be given a URL
1430 @warning{Some installations of git-cl fail when uploading a patch
1431 with certain filename extensions. If this happens, it can
1432 generally be fixed by editing the list of exceptions at top of
1435 @subsubheading Announcing your patch set
1437 You should then announce the patch by logging into the code review
1438 issue webpage and using @qq{Publish + Mail Comments} to add a
1439 (mostly bogus) comment to your issue. The text of your comment
1440 will be sent to our developer mailing list.
1442 @warning{There is no automatic notification of a new patch; you
1443 must add a comment yourself.}
1445 @subsubheading Revisions
1447 As revisions are made in response to comments, successive patch sets
1448 for the same issue can be uploaded by reissuing the git-cl command
1449 with the modified branch checked out.
1451 Sometimes in response to comments on revisions, the best way to
1452 work may require creation of a new branch in git. In order to
1453 associate the new branch with an existing Rietveld issue,
1454 the following command can be used:
1457 git-cl issue issue-number
1461 where @code{issue-number} is the number of the existing Rietveld
1464 @subsubheading Resetting git-cl
1466 If @command{git-cl} becomes confused, you can @qq{reset} it by
1474 @node The patch review cycle
1475 @subsection The patch review cycle
1477 Your patch will be available for reviews for the next few hours or
1478 days. Three times a week, patches with no known problems are
1479 gathered into a @qq{patch countdown} and their status changed to
1480 @code{patch-countdown}. The countdown is a 48-hour waiting period
1481 in which any final reviews or complaints should be made.
1483 During the countdown, your patch may be set to
1484 @code{patch-needs_work}, indicating that you should fix something
1485 (or at least discuss why the patch needs no modification). If no
1486 problems are found, the patch will be set to @code{patch-push}.
1488 Once a patch has @code{patch-push}, it should be sent to your
1489 mentor for uploading. If you have git push ability, look at
1490 @ref{Pushing to staging}.
1495 Patches get added to the tracker and to Rietveld by the @qq{git-cl} tool, with
1496 a status of @qq{patch-new}.
1499 The automated tester, Patchy, verifies that the patch can be applied
1500 to current master. By default, it checks that the patch allows @code{make}
1501 and @code{make test} to complete successfully. It can also be configured to
1502 check that @code{make doc} is successful. If it passes, Patchy changes the
1503 status to @qq{patch-review} and emails the developer list. If the patch
1504 fails, Patchy sets it to @qq{patch-needs_work} and notifies the developer list.
1507 The Patch Meister reviews the tracker periodically, to list patches
1508 which have been on review for at least 24 hours. The list is found at
1511 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch%20patch=review&sort=modified+patch&colspec=ID%20Type%20Status%20Priority%20Owner%20Patch%20Summary%20Modified}
1515 For each patch, the Handler reviews any discussion on the tracker
1516 and on Rietveld, to determine whether the patch can go forward. If
1517 there is any indication that a developer thinks the patch is not
1518 ready, the Handler marks it @qq{patch-needs_work} and makes a comment
1519 regarding the reason, referring to the Rietveld item if needed.
1522 Patches with explicit approval, or at least no negative comment, can
1523 be updated to @qq{patch-countdown}. When saving the tracker item,
1524 clear the @qq{send email} box to prevent sending notification for
1528 The Patch Meister sends an email to the developer list, with a fixed
1529 subject line, to enable filtering by email clients:
1532 PATCH: Countdown to 20130113
1535 The text of the email sets the deadline for this countdown batch. At
1536 present, batches are done on Tuesday, Thursday and Sunday evenings.
1538 To create the countdown announcement, use the
1539 @code{make-countdown-announcement.sh} script, which takes the
1540 deadline date, and optionally your name. Follow the instructions
1545 scripts/auxiliar/make-countdown-announcement.sh "Jan 1, 2001" James
1548 The script produces an announcement that is easily readable in all
1549 email clients. Also, whenever a new contributor submits a patch,
1550 you will be prompted to add the new username and author name to
1551 the script itself, and then commit those changes to the main git
1556 On the scheduled countdown day, the Patch Meister reviews the
1557 previous list of patches on countdown, with the same procedure and
1558 criteria as before. Patches with no controversy can be set to
1559 @qq{patch-push} with a courtesy message added to the comment block.
1562 Roughly at six month intervals, the Patch Meister can list the
1563 patches which have been set to @qq{patch-needs-work} and send the
1564 results to the developer list for review. In most cases, these
1565 patches should be marked @qq{patch-abandoned} but this should come
1566 from the developer if possible.
1569 As in most organisations of unpaid volunteers, fixed procedures are
1570 useful in as much as they get the job done. In our community, there
1571 is room for senior developers to bypass normal patch handling flows,
1572 particularly now that the testing of patches is largely automated.
1573 Similarly, the minimum age of 24 hours can reasonably be waived if
1574 the patch is minor and from an experienced developer.
1580 There is a single Patch Meister, and a number of Patch Helpers
1581 (rename this?). The list of known patches awaiting review is:
1584 @uref{http://code.google.com/p/lilypond/issues/list?can=2&q=label:patch&sort=patch}
1588 @subheading Helpers: adding patches
1590 The primary duty is to add patches to the google tracker; we have
1591 a bad track record of losing patches in email. Patches generally
1592 come to the @code{lilypond-devel} mailing list, but are sometimes
1593 sent to @code{bug-lilypond}, @code{lilypond-users}, or
1594 @code{frogs} mailing list instead.
1598 Unless a patch is clearly in response to an existing issue, add a
1599 new issue with the @code{Patch-new} label and a link to the patch
1600 (either on the mailing list archives or the codereview url).
1602 Issue numbers are cheap; losing developers because they got fed up
1603 with us losing their hard work is expensive.
1606 @c if we enter patches immediately, I don't think this is relevant.
1609 Before adding a patch-reminder issue, do a quick check to see if
1610 it was pushed without sending any email. This can be checked for
1611 searching for relevant terms (from the patch subject or commit
1612 message) on the webgit page:
1615 @uref{http://git.savannah.gnu.org/gitweb/?p=lilypond.git}
1620 If the patch is clearly in response to an existing issue, then
1621 update that issue with the @code{Patch-new} label and a link to
1622 the patch (either on the mailing list archives or the codereview
1626 After adding the issue, please send a response email to the same
1627 group(s) that the initial patch was sent to.
1629 If the initial email was sent to multiple mailing lists (such as
1630 both @code{bugs} and @code{devel}), then reply to all those
1631 mailing lists as well. The email should contain a link to the
1632 issue you just added.
1636 @subheading Helpers: @code{Patch-review} label
1638 The secondary duty is to do make sure that every issue in the
1639 tracker with a @code{Patch-review} label has passed these
1644 Applies automatically to git master.
1646 It's ok to have offsets, but not conflicts.
1649 Regtest comparison looks ok; no unexpected changes.
1652 Descriptive subject line.
1654 Avoid subjects like @qq{fixes 123}; instead write @qq{Doc: discuss
1655 stacking-dir for BassFigureAlignment (fix 123)}.
1658 Compiles docs from scratch. Only check this if you have reason to
1659 suspect it might not work.
1664 Check code indentation and style. This should be easier post-GOP
1665 when we have a better-defined code style.
1670 @subheading Patch Meister
1672 The Patch Meister will:
1677 send @qq{countdown} emails to
1678 @code{lilypond-devel} when patches appear to be ready.
1681 send general requests to review patches, or even nasty requests to
1685 downgrade patches from @code{Patch-review} to
1686 @code{Patch-needs_work} as appropriate.
1689 downgrade patches from @code{Patch-needs_work} to
1690 @code{Patch-abandoned} if no actions have been taken in four
1698 @node Advanced Git procedures
1699 @section Advanced Git procedures
1702 @warning{This section is not necessary for normal contributors;
1703 these commands are presented for information for people interested
1704 in learning more about git.}
1706 It is possible to work with several branches on the same local Git
1707 repository; this is especially useful for translators who may have
1708 to deal with both @code{translation} and a stable branch,
1709 e.g. @code{stable/2.12}.
1711 Some Git commands are introduced first, then a workflow with
1712 several Git branches of LilyPond source code is presented.
1717 * Advanced Git concepts::
1718 * Resolving conflicts::
1719 * Reverting all local changes::
1720 * Working with remote branches::
1722 * Applying remote patches::
1723 * Cleaning up multiple patches::
1725 * Pushing to staging::
1729 @node Merge conflicts
1730 @subsection Merge conflicts
1732 To be filled in later, and/or moved to a different section. I
1733 just wanted to make sure that I had a stub ready somewhere.
1736 @node Advanced Git concepts
1737 @subsection Advanced Git concepts
1740 A bit of Git vocabulary will be explained below. The following is
1741 only introductory; for a better understanding of Git concepts, you
1742 may wish to read @ref{Other Git documentation}.
1744 The @code{git@tie{}pull@tie{}origin} command above is just a
1745 shortcut for this command:
1748 git pull git://git.sv.gnu.org/lilypond.git/ @var{branch}:origin/@var{branch}
1752 where @code{@var{branch}} is typically @code{master} or
1753 @code{translation}; if you do not know or remember, see
1754 @ref{Downloading remote branches} to remember which commands you
1755 issued or which source code you wanted to get.
1757 A @emph{commit} is a set of changes made to the sources; it also
1758 includes the committish of the parent commit, the name and e-mail
1759 of the @emph{author} (the person who wrote the changes), the name
1760 and e-mail of the @emph{committer} (the person who brings these
1761 changes into the Git repository), and a commit message.
1763 A @emph{committish} is the SHA1 checksum of a commit, a number
1764 made of 40 hexadecimal digits, which acts as the internal unique
1765 identifier for this commit. To refer to a particular revision,
1766 don't use vague references like the (approximative) date, simply
1767 copy and paste the committish.
1769 A @emph{branch} is nothing more than a pointer to a particular
1770 commit, which is called the @emph{head} of the branch; when
1771 referring to a branch, one often actually thinks about its head
1772 and the ancestor commits of the head.
1774 Now we will explain the two last commands you used to get the
1775 source code from Git---see @ref{Downloading individual branches}.
1778 git remote add -ft @var{branch} -m @var{branch} \
1779 origin git://git.sv.gnu.org/lilypond.git/
1781 git checkout -b @var{branch} origin/@var{branch}
1784 The @command{git@tie{}remote} has created a branch called
1785 @code{origin/@var{branch}} in your local Git repository. As this
1786 branch is a copy of the remote branch web from git.sv.gnu.org
1787 LilyPond repository, it is called a @emph{remote branch}, and is
1788 meant to track the changes on the branch from git.sv.gnu.org: it
1789 will be updated every time you run
1790 @command{git@tie{}pull@tie{}origin} or
1791 @command{git@tie{}fetch@tie{}origin}.
1793 The @command{git@tie{}checkout} command has created a branch named
1794 @code{@var{branch}}. At the beginning, this branch is identical
1795 to @code{origin/@var{branch}}, but it will differ as soon as you
1796 make changes, e.g. adding newly translated pages or editing some
1797 documentation or code source file. Whenever you pull, you merge
1798 the changes from @code{origin/@var{branch}} and
1799 @code{@var{branch}} since the last pulling. If you do not have
1800 push (i.e. @qq{write}) access on git.sv.gnu.org, your
1801 @code{@var{branch}} will always differ from
1802 @code{origin/@var{branch}}. In this case, remember that other
1803 people working like you with the remote branch @code{@var{branch}}
1804 of git://git.sv.gnu.org/lilypond.git/ (called
1805 @code{origin/@var{branch}} on your local repository) know nothing
1806 about your own @code{@var{branch}}: this means that whenever you
1807 use a committish or make a patch, others expect you to take the
1808 latest commit of @code{origin/@var{branch}} as a reference.
1810 Finally, please remember to read the man page of every Git command
1811 you will find in this manual in case you want to discover
1812 alternate methods or just understand how it works.
1815 @node Resolving conflicts
1816 @subsection Resolving conflicts
1819 Occasionally an update may result in conflicts -- this happens
1820 when you and somebody else have modified the same part of the same
1821 file and git cannot figure out how to merge the two versions
1822 together. When this happens, you must manually merge the two
1825 If you need some documentation to understand and resolve
1826 conflicts, see paragraphs @emph{How conflicts are presented} and
1827 @emph{How to resolve conflicts} in @command{git merge} man page.
1829 If all else fails, you can follow the instructions in
1830 @ref{Reverting all local changes}. Be aware that this eliminates
1831 any changes you have made!
1834 @node Reverting all local changes
1835 @subsection Reverting all local changes
1837 Sometimes git will become hopelessly confused, and you just want
1838 to get back to a known, stable state. This command destroys any
1839 local changes you have made in the currently checked-out branch,
1840 but at least you get back to the current online version:
1843 git reset --hard origin/master
1847 @node Working with remote branches
1848 @subsection Working with remote branches
1851 @subsubheading Fetching new branches from git.sv.gnu.org
1853 To fetch and check out a new branch named @code{@var{branch}} on
1854 git.sv.gnu.org, run from top of the Git repository
1857 git config --add remote.origin.fetch \
1858 +refs/heads/@var{branch}:refs/remotes/origin/@var{branch}
1860 git checkout --track -b @var{branch} origin/@var{branch}
1863 After this, you can pull @code{@var{branch}} from git.sv.gnu.org
1870 Note that this command generally fetches all branches you added
1871 with @command{git@tie{}remote@tie{}add} (when you initialized the
1872 repository) or @command{git@tie{}config@tie{}--add}, i.e. it
1873 updates all remote branches from remote @code{origin}, then it
1874 merges the remote branch tracked by the current branch into the
1875 current branch. For example, if your current branch is
1876 @code{master}, @code{origin/master} will be merged into
1880 @subsubheading Local clones, or having several working trees
1882 If you play with several Git branches, e.g. @code{master},
1883 @code{translation}, @code{stable/2.12}), you may want to
1884 have one source and build tree for each branch; this is possible
1885 with subdirectories of your local Git repository, used as local
1886 cloned subrepositories. To create a local clone for the branch
1887 named @code{@var{branch}}, run
1890 git checkout @var{branch}
1891 git clone -lsn . @var{subdir}
1896 Note that @code{@var{subdir}} must be a directory name which does
1897 not already exist. In @code{@var{subdir}}, you can use all Git
1898 commands to browse revisions history, commit and uncommit changes;
1899 to update the cloned subrepository with changes made on the main
1900 repository, cd into @code{@var{subdir}} and run
1901 @command{git@tie{}pull}; to send changes made on the subrepository
1902 back to the main repository, run @command{git@tie{}push} from
1903 @code{@var{subdir}}. Note that only one branch (the currently
1904 checked out branch) is created in the subrepository by default; it
1905 is possible to have several branches in a subrepository and do
1906 usual operations (checkout, merge, create, delete...) on these
1907 branches, but this possibility is not detailed here.
1909 When you push @code{@var{branch}} from @code{@var{subdir}} to the
1910 main repository, and @code{@var{branch}} is checked out in the
1911 main repository, you must save uncommitted changes (see
1912 @command{git@tie{}stash}) and do
1913 @command{git@tie{}reset@tie{}--hard} in the main repository in
1914 order to apply pushed changes in the working tree of the main
1922 The commands above don't only bring you the latest version of the
1923 sources, but also the full history of revisions (revisions, also
1924 called commits, are changes made to the sources), stored in the
1925 @file{.git} directory. You can browse this history with
1928 git log # only shows the logs (author, committish and commit message)
1929 git log -p # also shows diffs
1930 gitk # shows history graphically
1933 @warning{The @code{gitk} command may require a separate
1934 @code{gitk} package, available in the appropriate distribution's
1938 @node Applying remote patches
1939 @subsection Applying remote patches
1942 TODO: Explain how to determine if a patch was created with
1943 @code{git@tie{}format-patch}.
1945 Well-formed git patches created with @code{git@tie{}format-patch}
1946 should be committed with the following command:
1952 Patches created without @code{git@tie{}format-patch} can be
1953 applied in two steps. The first step is to apply the patch to the
1954 working tree and the index:
1957 git apply --index @var{patch}
1961 The second step is to commit the changes and give credit to the
1962 author of the patch. This can be done with the following command:
1965 git commit --author="@var{John Smith} <@var{john@@example.com}>"
1968 Please note that using the @code{--index} option for patching is quite
1969 important here and @emph{cannot} reliably be replaced by using the
1970 @code{-a} option when committing: that would only commit files from the
1971 working tree that are already registered with git, so every file that
1972 the patch actually @emph{adds}, like a regtest for a fixed bug, would
1973 get lost. For the same reason, you should not use the git-independent
1974 @samp{patch} program for applying patches.
1978 @node Cleaning up multiple patches
1979 @subsection Cleaning up multiple patches
1981 If you have been developing on your own branch for a while, you
1982 may have more commmits than is really sensible. To revise your
1983 work and condense commits, use:
1986 git rebase origin/master
1987 git rebase -i origin/master
1990 @warning{Be a bit cautious -- if you completely remove commits
1991 during the interactive session, you will... err... completely
1992 remove those commits.}
1996 @subsection Commit access
1998 Most contributors are not able to commit patches directly to the
1999 main repository---only members of the LilyPond development team
2000 have @emph{commit access}. If you are a contributor and are
2001 interested in joining the development team, contact the Project
2002 Manager through the mailing list
2003 (@email{lilypond-devel@@gnu.org}). Generally, only contributors
2004 who have already provided a number of patches which have been
2005 pushed to the main repository will be considered for membership.
2007 If you have been approved by the Project Manager, use the
2008 following procedure to obtain commit access:
2012 If you don't already have one, set up a Savannah user account at
2013 @uref{https://savannah.gnu.org/account/register.php}. If your web
2014 browser responds with an @qq{untrusted connection} message when
2015 you visit the link, follow the steps for including the CAcert root
2016 certificate in your browser, given at
2017 @uref{http://savannah.gnu.org/tls/tutorial/}.
2019 @warning{Savannah will silently put your username in lower-case --
2020 do not try to use capital letters.}
2024 After registering, if you are not logged in automatically, login
2025 at @uref{https://savannah.gnu.org/account/login.php}---this should
2026 take you to your @qq{my} page
2027 (@uref{https://savannah.gnu.org/my/}).
2031 Click on the @qq{My Groups} link to access the @qq{My Group
2032 Membership} page. From there, find the @qq{Request for Inclusion}
2033 box and search for @qq{LilyPond}. Among the search results, check
2034 the box labeled @qq{GNU LilyPond Music Typesetter} and write a
2035 brief (required) message for the Project Manager (@qq{Hey it's
2036 me!} should be fine).
2038 Note that you will not have commit access until the Project
2039 Manager activates your membership. Once your membership is
2040 activated, LilyPond should appear under the heading @qq{Groups I'm
2041 Contributor of} on your @qq{My Group Membership} page.
2045 Generate an SSH @q{rsa} key pair. Enter the following at the
2052 When prompted for a location to save the key, press <ENTER> to
2053 accept the default location (@file{~/.ssh/id_rsa}).
2055 Next you are asked to enter an optional passphrase. On most
2056 systems, if you use a passphrase, you will likely be prompted for
2057 it every time you use @command{git@tie{}push} or
2058 @command{git@tie{}pull}. You may prefer this since it can protect
2059 you from your own mistakes (like pushing when you mean to pull),
2060 though you may find it tedious to keep re-entering it.
2062 You can change/enable/disable your passphrase at any time with:
2065 ssh-keygen -f ~/.ssh/id_rsa -p
2068 Note that the GNOME desktop has a feature which stores your
2069 passphrase for you for an entire GNOME session. If you use a
2070 passphrase to @qq{protect you from yourself}, you will want to
2071 disable this feature, since you'll only be prompted once. Run the
2072 following command, then logout of GNOME and log back in:
2075 gconftool-2 --set -t bool \
2076 /apps/gnome-keyring/daemon-components/ssh false
2079 After setting up your passphrase, your private key is saved as
2080 @file{~/.ssh/id_rsa} and your public key is saved as
2081 @file{~/.ssh/id_rsa.pub}.
2085 Register your public SSH @q{rsa} key with Savannah. From the
2086 @qq{My Account Configuration} page, click on @qq{Edit SSH Keys},
2087 then paste the contents of your @file{~/.ssh/id_rsa.pub} file into
2088 one of the @qq{Authorized keys} text fields, and click
2091 Savannah should respond with something like:
2094 Success: Key #1 seen Keys registered
2099 Configure Git to use the SSH protocol (instead of the GIT
2100 protocol). From your local Git repository, enter:
2103 git config remote.origin.url \
2104 ssh://@var{user}@@git.sv.gnu.org/srv/git/lilypond.git
2108 replacing @var{user} with your Savannah username.
2112 After your membership has been activated and you've configured Git
2113 to use SSH, test the connection with:
2119 SSH should issue the following warning:
2122 The authenticity of host 'git.sv.gnu.org (140.186.70.72)' can't
2124 RSA key fingerprint is
2125 80:5a:b0:0c:ec:93:66:29:49:7e:04:2b:fd:ba:2c:d5.
2126 Are you sure you want to continue connecting (yes/no)?
2129 Make sure the RSA key fingerprint displayed matches the one above.
2130 If it doesn't, respond @qq{no} and check that you configured Git
2131 properly in the previous step. If it does match, respond
2132 @qq{yes}. SSH should then issue another warning:
2135 Warning: Permanently added 'git.sv.gnu.org,140.186.70.72' (RSA) to
2136 the list of known hosts.
2139 The list of known hosts is stored in the file
2140 @file{~/.ssh/known_hosts}.
2142 At this point, you are prompted for your passphrase if you have
2143 one, then Git will attempt a pull.
2145 If @command{git@tie{}pull@tie{}--verbose} fails, you should see
2146 error messages like these:
2149 Permission denied (publickey).
2150 fatal: The remote end hung up unexpectedly
2153 If you get the above error, you may have made a mistake when
2154 registering your SSH key at Savannah. If the key is properly
2155 registered, you probably just need to wait for the Savannah server
2156 to activate it. It usually takes a few minutes for the key to be
2157 active after registering it, but if it still doesn't work after an
2158 hour, ask for help on the mailing list.
2160 If @command{git@tie{}pull@tie{}--verbose} succeeds, the output
2161 will include a @q{From} line that shows @q{ssh} as the protocol:
2164 From ssh://git.sv.gnu.org/srv/git/lilypond
2167 If the protocol shown is not @q{ssh}, check that you configured
2168 Git properly in the previous step.
2172 Test your commit access with a dry run:
2174 @warning{Do not push directly to master; instead, push to staging.
2175 See @ref{Pushing to staging}.}
2178 git push --dry-run --verbose
2181 Note that recent versions of Git (Git 1.6.3 or later) will issue a
2182 big warning if the above command is used. The simplest solution
2183 is to tell Git to push all matching branches by default:
2186 git config push.default matching
2190 Then @code{git@tie{}push} should work as before. For more
2191 details, consult the @code{git@tie{}push} man page.
2195 Repeat the steps from generating an RSA key through to testing
2196 your commit access, for each machine from which you will be
2197 making commits, or you may simply copy the files from your
2198 local @file{~/.ssh} folder to the same folder on the other
2203 @subsubheading Technical details
2207 On Firefox, to view or remove the CAcert root certificate, go to:
2208 Edit > Preferences > Advanced > Encryption > View Certificates >
2209 Authorities > Certificate Name > Root CA > CA Cert Signing
2213 The @command{git@tie{}config} commands above should modify your
2214 local repository's @file{.git/config} file. These lines:
2218 url = git://git.sv.gnu.org/lilypond.git/
2222 should now be changed to:
2226 url = ssh://@var{user}@@git.sv.gnu.org/srv/git/lilypond.git
2230 where @var{user} is your login name on Savannah.
2234 @command{git@tie{}config@tie{}push.default@tie{}matching} command
2235 should add these lines to @file{.git/config}:
2244 Encryption protocols, including ssh, generally do not permit packet
2245 fragmentation to avoid introducing a point of insecurity. This
2246 means that the maximum packet size must not exceed the smallest
2247 MTU (Maximum Transmission Unit) set in the routers along the path.
2248 This smallest MTU is determined by a procedure during call set-up
2249 which relies on the transmission over the path of ICMP packets.
2250 If any of the routers in the path block ICMP packets this mechanism
2251 fails, resulting in the possibility of packets being transmitted
2252 which exceed the MTU of one of the routers. If this happens the
2253 packet is discarded, causing the ssh session to hang, timeout or
2254 terminate with the error message
2257 ssh: connect to host <host ip addr> port 22: Bad file number
2258 fatal: The remote end hung up unexpectedly
2261 depending on precisely when in the proceedings the first large
2262 packet is transmitted. Most routers on the internet have MTU
2263 set to 1500, but routers installed in homes to connect via
2264 broadband may use a slightly smaller MTU for efficient transmission
2265 over ATM. If this problem is encountered a possible work-around is
2266 to set the MTU in the local router to 1500.
2269 @node Pushing to staging
2270 @subsection Pushing to staging
2272 Do not push directly to the git @code{master} branch. Instead,
2273 push to @code{staging}.
2275 You will not see your patch on @code{origin/master} until some
2276 automatic tests have been run. These tests are run every couple
2277 of hours; please wait at least 12 hours before wondering if your
2278 patch has been lost. Note that you can check the commits on
2279 @code{origin/staging} by looking at the git web interface on
2282 It may happen occasionally that the staging branch breaks automated
2283 testing. In this case the automatic move of staging material to
2284 master gets halted in order to avoid broken material entering master.
2285 This is a safety net. Please do not try breaking out from it by
2286 adding fixes on top of staging: in that case the whole sequence will
2287 end up in master after all, defeating the purpose of the system. The
2288 proper fix usually involves rewriting the staging branch and is best
2289 left to core developers after discussion on the developer list.
2291 Before pushing to staging it is a good practice to check whether
2292 staging is ahead of master, and if so, wait until master has caught up
2293 with staging before pushing. This simplifies things if changes to
2294 staging have to be backed out for some reason. To check whether
2295 master has caught up with staging you can look at the git web interface
2303 and check that @code{origin/master} is at the same commit as
2304 @code{origin/staging}. Another option is to see if any commits are
2309 git log origin/master..origin/staging
2312 @subsubheading If your work is in a patch file
2314 Assuming that your patch is in a file called
2315 @file{0001-my-patch.patch} (see @ref{Patches}), and you are currently
2319 git checkout staging
2321 git am 0001-my-patch.patch
2323 git push origin staging
2327 @warning{Do not skip the @command{gitk} step; a quick 5-second
2328 check of the visual history can save a great deal of frustration
2329 later on. You should only see that @command{staging} is only 1
2330 commit ahead of @code{origin/staging}.}
2332 @subsubheading If your work is in a branch
2334 If you are working on branches and your work is in
2335 @code{my_branch_name}, then do:
2338 git checkout my_branch_name
2339 git pull -r origin staging
2342 This will rebase your branch on @code{origin/staging}. At this point
2343 git will let you know if there are any conflicts. If so, resolve them
2348 git push origin HEAD:staging
2351 @warning{Do not skip the @command{gitk} step; a quick 5-second
2352 check of the visual history can save a great deal of frustration
2353 later on. You should see that @code{my_branch_name} is only ahead of
2354 @code{origin/staging} by the commits from your branch.}
2358 @node Git on Windows
2359 @section Git on Windows
2361 @warning{We heavily recommend that development be done with our
2362 virtual machine @ref{LilyDev}.}
2364 @c Some of this may duplicate stuff in other sections
2365 @c But it is probably best for windows users to have it all together
2366 @c If necessary, clear this up later -td
2368 TODO: Decide what to do with this... Pare it down? Move
2369 paragraphs next to analogous Unix instructions? -mp
2371 @subsection Background to nomenclature
2373 Git is a system for tracking the changes made to source files by a
2374 distributed set of editors. It is designed to work without a
2375 master repository, but we have chosen to have a master repository
2376 for LilyPond files. Editors hold a local copy of the master
2377 repository together with any changes they have made locally.
2378 Local changes are held in a local @q{branch}, of which there may
2379 be several, but these instructions assume you are using just one.
2380 The files visible in the local repository always correspond to
2381 those on the currently @q{checked out} local branch.
2383 Files are edited on a local branch, and in that state the changes
2384 are said to be @q{unstaged}. When editing is complete, the
2385 changes are moved to being @q{staged for commit}, and finally the
2386 changes are @q{committed} to the local branch. Once committed,
2387 the changes (called a @q{commit}) are given a unique 40-digit
2388 hexadecimal reference number called the @q{Committish} or @q{SHA1
2389 ID} which identifies the commit to Git. Such committed changes
2390 can be sent to the master repository by @q{pushing} them (if you
2391 have write permission) or by sending them by email to someone who
2392 has, either as a complete file or as a @q{diff} or @q{patch}
2393 (which send just the differences from the master repository).
2395 @subsection Installing git
2398 @uref{https://git-for-windows.github.io/}.
2400 Note that most users will not need to install SSH. That is not
2401 required until you have been granted direct push permissions to
2402 the master git repository.
2404 Start Git by clicking on the desktop icon. This will bring up a
2405 command line bash shell. This may be unfamiliar to Windows users.
2406 If so, follow these instructions carefully. Commands are entered
2407 at a $ prompt and are terminated by keying a newline.
2409 @subsection Initialising Git
2411 Decide where you wish to place your local Git repository, creating
2412 the folders in Windows as necessary. Here we call the folder to
2413 contain the repository @code{[path]/Git}, but if you intend using
2414 Git for other projects a directory name like @code{lilypond-git}
2415 might be better. You will need to have space for around
2418 Start the Git bash shell by clicking on the desk-top icon
2419 installed with Git and type
2425 to position the shell at your new Git repository.
2427 Note: if [path] contains folders with names containing spaces use
2439 to initialize your Git repository.
2441 Then type (all on one line; the shell will wrap automatically)
2444 git remote add -ft master origin git://git.sv.gnu.org/lilypond.git
2447 to download the lilypond master files.
2449 @warning{Be patient! Even on a broadband connection this can take
2450 10 minutes or more. Wait for lots of [new tag] messages and the $
2453 We now need to generate a local copy of the downloaded files in a
2454 new local branch. Your local branch needs to have a name. It is
2455 usual to call it @q{master} and we shall do that here.
2460 git checkout -b master origin/master
2463 This creates a second branch called @q{master}. You will see two
2464 warnings (ignore these), and a message advising you that your
2465 local branch @q{master} has been set up to track the remote
2466 branch. You now have two branches, a local branch called
2467 @q{master}, and a tracking branch called @q{origin/master}, which
2468 is a shortened form of @q{remotes/origin/master}.
2470 Return to Windows Explorer and look in your Git repository. You
2471 should see lots of folders. For example, the LilyPond
2472 documentation can be found in [path]/Git/Documentation/.
2474 The Git bash shell is terminated by typing @code{exit} or by
2475 clicking on the usual Windows close-window widget.
2479 Almost all subsequent work will use the Git Graphical User
2480 Interface, which avoids having to type command line commands. To
2481 start Git GUI first start the Git bash shell by clicking on the
2482 desktop icon, and type
2489 The Git GUI will open in a new window. It contains four panels
2490 and 7 pull-down menus. At this stage do not use any of the
2491 commands under Branch, Commit, Merge or Remote. These will be
2494 The top panel on the left contains the names of files which you
2495 are in the process of editing (Unstaged Changes), and the lower
2496 panel on the left contains the names of files you have finished
2497 editing and have staged ready for committing (Staged Changes). At
2498 present, these panels will be empty as you have not yet made any
2499 changes to any file. After a file has been edited and saved the
2500 top panel on the right will display the differences between the
2501 edited file selected in one of the panels on the left and the last
2502 version committed on the current branch.
2504 The panel at bottom right is used to enter a descriptive message
2505 about the change before committing it.
2507 The Git GUI is terminated by entering CNTL-Q while it is the
2508 active window or by clicking on the usual Windows close-window
2511 @subsection Personalising your local git repository
2513 Open the Git GUI, click on
2519 and enter your name and email address in the left-hand (Git
2520 Repository) panel. Leave everything else unchanged and save it.
2522 Note that Windows users must leave the default setting for line
2523 endings unchanged. All files in a git repository must have lines
2524 terminated by just a LF, as this is required for Merge to work,
2525 but Windows files are terminated by CRLF by default. The git
2526 default setting causes the line endings of files in a Windows git
2527 repository to be flipped automatically between LF and CRLF as
2528 required. This enables files to be edited by any Windows editor
2529 without causing problems in the git repository.
2531 @subsection Checking out a branch
2533 At this stage you have two branches in your local repository,
2534 both identical. To see them click on
2540 You should have one local branch called @q{master} and one
2541 tracking branch called @q{origin/master}. The latter is your
2542 local copy of the @q{remotes/origin/master} branch in the master
2543 LilyPond repository. The local @q{master} branch is where you
2544 will make your local changes.
2546 When a particular branch is selected, i.e., checked out, the files
2547 visible in your repository are changed to reflect the state of the
2548 files on that branch.
2550 @subsection Updating files from @q{remote/origin/master}
2552 Before starting the editing of a file, ensure your local
2553 repository contains the latest version of the files in the remote
2554 repository by first clicking
2557 Remote -> Fetch from -> origin
2563 This will place the latest version of every file, including all
2564 the changes made by others, into the @q{origin/master} branch of
2565 the tracking branches in your git repository. You can see these
2566 files by checking out this branch, but you must @emph{never} edit
2567 any files while this branch is checked out. Check out your local
2568 @q{master} branch again.
2570 You then need to merge these fetched files into your local
2571 @q{master} branch by clicking on
2574 Merge -> Local Merge
2578 and if necessary select the local @q{master} branch.
2580 Note that a merge cannot be completed if you have made any local
2581 changes which have not yet been committed.
2583 This merge will update all the files in the @q{master} branch to
2584 reflect the current state of the @q{origin/master} branch. If any
2585 of the changes conflict with changes you have made yourself
2586 recently you will be notified of the conflict (see below).
2588 @subsection Editing files
2590 First ensure your @q{master} branch is checked out, then simply
2591 edit the files in your local Git repository with your favourite
2592 editor and save them back there. If any file contains non-ASCII
2593 characters ensure you save it in UTF-8 format. Git will detect
2594 any changes whenever you restart Git GUI and the file names will
2595 then be listed in the Unstaged Changes panel. Or you can click
2596 the Rescan button to refresh the panel contents at any time. You
2597 may break off and resume editing any time.
2599 The changes you have made may be displayed in diff form in the top
2600 right-hand panel of Git GUI by clicking on the file name shown in
2601 one of the left panels.
2603 When your editing is complete, move the files from being Unstaged
2604 to Staged by clicking the document symbol to the left of each
2605 name. If you change your mind it can be moved back by clicking on
2606 the ticked box to the left of the name.
2608 Finally the changes you have made may be committed to your
2609 @q{master} branch by entering a brief message in the Commit
2610 Message box and clicking the Commit button.
2612 If you wish to amend your changes after a commit has been made,
2613 the original version and the changes you made in that commit may
2614 be recovered by selecting
2617 Commit -> Amend Last Commit
2621 or by checking the Amend Last Commit radio button at bottom right.
2622 This will return the changes to the Staged state, so further
2623 editing made be carried out within that commit. This must only be
2624 done @emph{before} the changes have been Pushed or sent to your
2625 mentor for Pushing - after that it is too late and corrections
2626 have to be made as a separate commit.
2629 @subsection Sending changes to @q{remotes/origin/master}
2631 If you do not have write access to @q{remotes/origin/master} you
2632 will need to send your changes by email to someone who does.
2634 First you need to create a diff or patch file containing your
2635 changes. To create this, the file must first be committed. Then
2636 terminate the Git GUI. In the git bash shell first cd to your Git
2643 if necessary, then produce the patch with
2646 git format-patch origin
2649 This will create a patch file for all the locally committed files
2650 which differ from @q{origin/master}. The patch file can be found
2651 in [path]/Git and will have a name formed from the commit message.
2653 @subsection Resolving merge conflicts
2655 As soon as you have committed a changed file your local
2656 @code{master} branch has diverged from @code{origin/master}, and
2657 will remain diverged until your changes have been committed in
2658 @code{remotes/origin/master} and Fetched back into your
2659 @code{origin/master} branch. Similarly, if a new commit has been
2660 made to @code{remotes/origin/master} by someone else and Fetched,
2661 your local @code{master} branch is divergent. You can detect a
2662 divergent branch by clicking on
2665 Repository -> Visualise all branch history
2668 This opens up a very useful new window called @q{gitk}. Use this
2669 to browse all the commits made by yourself and others.
2671 If the diagram at top left of the resulting window does not show
2672 your @code{master} tag on the same node as the
2673 @code{remotes/origin/master} tag your branch has diverged from
2674 @code{origin/master}. This is quite normal if files you have
2675 modified yourself have not yet been Pushed to
2676 @code{remotes/origin/master} and Fetched, or if files modified and
2677 committed by others have been Fetched since you last Merged
2678 @code{origin/master} into your local @code{master} branch.
2680 If a file being merged from @code{origin/master} differs from one
2681 you have modified in a way that cannot be resolved automatically
2682 by git, Merge will report a Conflict which you must resolve by
2683 editing the file to create the version you wish to keep.
2685 This could happen if the person updating
2686 @code{remotes/origin/master} for you has added some changes of his
2687 own before committing your changes to
2688 @code{remotes/origin/master}, or if someone else has changed the
2689 same file since you last fetched the file from
2690 @code{remotes/origin/master}.
2692 Open the file in your editor and look for sections which are
2695 [to be completed when I next have a merge conflict to be sure I
2696 give the right instructions -td]
2699 @subsection Other actions
2701 The instructions above describe the simplest way of using git on
2702 Windows. Other git facilities which may usefully supplement these
2706 @item Using multiple local branches (Create, Rename, Delete)
2707 @item Resetting branches
2708 @item Cherry-picking commits
2709 @item Pushing commits to @w{remote/origin/master}
2710 @item Using gitk to review history
2713 Once familiarity with using git on Windows has been gained the
2714 standard git manuals can be used to learn about these.
2717 @node Repository directory structure
2718 @section Repository directory structure
2721 @c TODO: integrate the roadmap better
2722 @verbatiminclude ROADMAP
2725 @node Other Git documentation
2726 @section Other Git documentation
2730 Official git man pages:
2731 @uref{http://www.kernel.org/pub/software/scm/git/docs/}
2734 More in-depth tutorials: @uref{http://git-scm.com/documentation}
2737 Book about git: @uref{http://progit.org/,Pro Git}
2740 Github help: @uref{http://help.github.com/}
2741 (very highly recommended by Graham)