@c -*- coding: us-ascii; mode: texinfo; -*- @node Starting with git @chapter Starting with git To complete or present in another form the introduction to Git usage in this chapter, it may be a good idea to look for Git documentation at @uref{http://git-scm.com/documentation}, @menu * Getting the source code:: * Updating the source code:: * Sharing your changes:: * Advanced git stuff:: * Git on Windows:: * Development inside VirtualBox (compiling on Windows):: @end menu @node Getting the source code @section Getting the source code @menu * Git introduction:: * Git user configuration:: * Main source code:: * Documentation translations source code:: * Other branches:: * Other locations for git:: @end menu @node Git introduction @subsection Git introduction The source code is kept in a Git respository. This allows us to track changes to files, and for multiple people to work on the same set of files efficiently. After downloading the source code, it is important to update the repository by @ref{Updating the source code}. @warning{These instructions assume that you are using the command-line version of Git 1.5 or higher. Windows users should skip to @ref{Git on Windows}.} @node Git user configuration @subsection Git user configuration Some useful cofiguration can be performed automatically; cut and paste the following section: @example git config --global color.ui auto git config --global branch.autosetuprebase always @end example To configure git to automatically use your name and email address for commits and patches, edit the below lines by changing all the @code{MY...} entries: @example git config --global user.name "MYNAME" git config --global user.email MYEMAIL@@EXAMPLE.NET @end example @node Main source code @subsection Main source code To get the main source code and documentation, @c WARNING: when updating the commands below, please @c update the other flavors in the two next nodes @c and in Introduction to Git concepts @smallexample mkdir lilypond; cd lilypond git init-db git remote add -f -t master -m master origin git://git.sv.gnu.org/lilypond.git/ git checkout -b master origin/master @end smallexample @node Documentation translations source code @subsection Documentation translations source code To translate the documentation (@emph{not} the website), FIXME: when website is integrated, modify this. @smallexample mkdir lilypond-translation; cd lilypond-translation git init-db git remote add -f -t lilypond/translation -m lilypond/translation origin git://git.sv.gnu.org/lilypond.git/ git checkout -b lilypond/translation origin/lilypond/translation @end smallexample @node Other branches @subsection Other branches Most contributors will never need to touch the other branches. If you wish to do so, you will need more familiarity with git. @itemize @item @code{dev/XYZ}: These branches are for individual developers. They store code which is not yet stable enough to be added to the @code{master} branch. @item @code{stable/XYZ}: The branches are kept for archival reasons. @end itemize Another item of interest might be the Grand Unified Builder, our cross-platform building tool. Since it is used by projects as well, it is not stored in our gub repository. For more info, see @uref{http://lilypond.org/gub}. The git location is: @example http://github.com/janneke/gub @end example @node Other locations for git @subsection Other locations for git If you have difficulty connecting to most of the repositories listed in earlier sections, try: @example http://git.sv.gnu.org/r/lilypond.git git://git.sv.gnu.org/lilypond.git ssh://git.sv.gnu.org/srv/git/lilypond.git @end example Using HTTP protocol is slowest, so it is not recommended unless both SSH and Git protocols fail, which happens e.g. if you connect to internet through a router that filters out Git and/or SSH connections. @node Updating the source code @section Updating the source code @menu * Importance of updating:: * Update command:: * Resolving conflicts:: @end menu @node Importance of updating @subsection Importance of updating In a large project like LilyPond, contributors sometimes edit the same file at the same time. As long as everybody updates their version of the file with the most recent changes (@emph{pulling}), there are generally no problems with this multiple-person editing. However, big problems can arise if you do not pull before attempting to commit. If this occurs, see @ref{Resolving conflicts}. @node Update command @subsection Updating command Whenever you are asked to pull, it means you should update your local copy of the repository with the changes made by others on the remote @code{git.sv.gnu.org} repository: @example git pull -r @end example @noindent The @code{-r} option is short for @code{--rebase}. If you don't want to type @code{-r} every time, add @code{rebase = true} to the master branch in your @file{.git/config}, like so: @example [branch "master"] remote = origin merge = refs/heads/master rebase = true @end example @node Resolving conflicts @subsection Resolving conflicts Occasionally an update may result in conflicts -- this happens when you and somebody else have modified the same part of the same file and git cannot figure out how to merge the two versions together. When this happens, you must manually merge the two versions. If you need some documentation to understand and resolve conflicts, see paragraphs @emph{How conflicts are presented} and @emph{How to resolve conflicts} in @command{git merge} man page. @node Sharing your changes @section Sharing your changes @menu * Producing a patch:: * Committing directly:: @end menu @node Producing a patch @subsection Producing a patch Once you have finished editing your files, checked that your changes meet the @ref{Code style}, and/or @ref{Documentation policy}, properly set up your name and email in @ref{Git user configuration}, and checked that the entire thing compiles, you may: @example git commit -a git format-patch origin @end example The commit should include a brief message describing the change. This consists of a one-line summary describing the change, and if necessary a blank line followed by several lines giving the details: @example Did household chores. I hung up the wet laundry and then washed the car. I also vacuumed the floors, rinsed the dirty dishes, fed the cat, and recalibrated the temporal flux machine. @end example If the change is to the documentation only then the one-line summary should be prefixed with @qq{Docs: }. If you added a file to the source code, you must add it to git with: @example git add FILENAME @end example @noindent (and possibly modify the @file{GNUmakefile}) These commands will produce one or more files named @file{0001-xyz}, @file{0002-abc}, etc. in the top directory of the git tree. Send an email to @email{lilypond-devel@@gnu.org} with these files attached, and a developer will review and apply the patches to the main repository. @node Committing directly @subsection Committing directly Most contributors do not have permission to commit directly. If you do, make sure you have set up your name and email in @ref{Git user configuration}, then edit @file{.git/config}: change the line @example url = git://git.sv.gnu.org/lilypond.git/ @end example @noindent into @example url = ssh://@var{user}@@git.sv.gnu.org/srv/git/lilypond.git @end example @noindent where @var{user} is your login name on Savannah. If you have not already done so, you should generate and upload a SSH key: open @uref{https://savannah.gnu.org/my/} in your browser, go to @q{Account Configuration}, then to something like @q{Edit SSH Keys}, and follow the instructions on that page. You may then: @example git push origin @end example Note that recent versions of Git (Git 1.6.3 or later) will issue a big warning if the above command is used. The simplest solution is to add a new section to @file{.git/config} that looks like this: @example [push] default = matching @end example @noindent Then @code{git push origin} will work as before. For more details, consult the @code{git push} man page. @node Advanced git stuff @section Advanced git stuff @warning{This section is not necessary for normal contributors; these commands are presented for information for people interested in learning more about git.} It is possible to work with several branches on the same local Git repository; this is especially useful for translators who may have to deal with both @code{lilypond/translation} and a stable branch, e.g. @code{stable/2.12}. Some Git commands are introduced first, then a workflow with several Git branches of LilyPond source code is presented. @menu * Introduction to Git concepts:: * Git commands for managing several branches:: * Working on LilyPond sources with several branches:: * Git log:: * Applying git patches:: * Reverting all local changes:: @end menu @node Introduction to Git concepts @subsection Introduction to Git concepts A bit of Git vocabulary will be explained below. The following is just introduction material; for better understanding of Git concepts, you are invited to read further documentation, especially Git Community Book at @uref{http://book.git-scm.com/}. The @code{git pull origin} command above is just a shortcut for this command: @example git pull git://git.sv.gnu.org/lilypond.git/ @var{branch}:origin/@var{branch} @end example @noindent where @code{@var{branch}} is typically @code{master}, @code{web} or @code{lilypond/translation}; if you do not know or remember, see @ref{Getting the source code} to remember which commands you issued or which source code you wanted to get. A @emph{commit} is a set of changes made to the sources; it also includes the committish of the parent commit, the name and e-mail of the @emph{author} (the person who wrote the changes), the name and e-mail of the @emph{committer} (the person who brings these changes into the Git repository), and a commit message. A @emph{committish} is the SHA1 checksum of a commit, a number made of 40 hexadecimal digits, which acts as the internal unique identifier for this commit. To refer to a particular revision, don't use vague references like the (approximative) date, simply copy and paste the committish. A @emph{branch} is nothing more than a pointer to a particular commit, which is called the @emph{head} of the branch; when referring to a branch, one often acutally thinks about its head and the ancestor commits of the head. Now we will explain the two last commands you used to get the source code from Git -- see @ref{Getting the source code}. @example git remote add -f -t @var{branch} -m @var{branch} origin git://git.sv.gnu.org/lilypond.git/ git checkout -b @var{branch} origin/@var{branch} @end example The @command{git remote} has created a branch called @code{origin/@var{branch}} in your local Git repository. As this branch is a copy of the remote branch web from git.sv.gnu.org LilyPond repository, it is called a @emph{remote branch}, and is meant to track the changes on the branch from git.sv.gnu.org: it will be updated every time you run @command{git pull origin} or @command{git fetch origin}. The @command{git checkout} command has created a branch named @code{@var{branch}}. At the beginning, this branch is identical to @code{origin/@var{branch}}, but it will differ as soon as you make changes, e.g. adding newly translated pages or editing some documentation or code source file. Whenever you pull, you merge the changes from @code{origin/@var{branch}} and @code{@var{branch}} since the last pulling. If you do not have push (i.e. @qq{write}) access on git.sv.gnu.org, your @code{@var{branch}} will always differ from @code{origin/@var{branch}}. In this case, remember that other people working like you with the remote branch @code{@var{branch}} of git://git.sv.gnu.org/lilypond.git/ (called @code{origin/@var{branch}} on your local repository) know nothing about your own @code{@var{branch}}: this means that whenever you use a committish or make a patch, others expect you to take the latest commit of @code{origin/@var{branch}} as a reference. Finally, please remember to read the man page of every Git command you will find in this manual in case you want to discover alternate methods or just understand how it works. @node Git commands for managing several branches @subsection Git commands for managing several branches @subsubheading Listing branches and remotes You can get the exact path or URL of all remotes with running @example git remote -v @end example To list Git branches on your local repositories, run @example git branch # list local branches only git branch -r # list remote branches git branch -a # list all branches @end example @subsubheading Checking out branches To know the currently checked out branch, i.e. the branch whose source files are present in your working tree, read the first line of the output of @example git status @end example @noindent The currently checked out branch is also marked with an asterisk in the output of @command{git branch}. You can check out another branch @code{@var{other_branch}}, i.e. check out @code{@var{other_branch}} to the working tree, by running @example git checkout @var{other_branch} @end example Note that it is possible to check out another branch while having uncommitted changes, but it is not recommended unless you know what you are doing; it is recommended to run @command{git status} to check this kind of issue before checking out another branch. @subsubheading Merging branches To merge branch @code{@var{foo}} into branch @code{@var{bar}}, i.e. to @qq{add} all changes made in branch @code{@var{foo}} to branch @code{@var{bar}}, run @example git checkout @var{bar} git merge @var{foo} @end example If any conflict happens, see @ref{Resolving conflicts}. There are common usage cases for merging: as a translator, you will often want to merge @code{master} into @code{lilypond/translation}; on the other hand, the Translations meister wants to merge @code{lilypond/translation} into @code{master} whenever he has checked that @code{lilypond/translation} builds successfully. @node Working on LilyPond sources with several branches @subsection Working on LilyPond sources with several branches @subsubheading Fetching new branches from git.sv.gnu.org To fetch and check out a new branch named @code{@var{branch}} on git.sv.gnu.org, run from top of the Git repository @example git config --add remote.origin.fetch +refs/heads/@var{branch}:refs/remotes/origin/@var{branch} git checkout --track -b @var{branch} origin/@var{branch} @end example After this, you can pull @code{@var{branch}} from git.sv.gnu.org with @example git pull origin @end example Note that this command generally fetches all branches you added with @command{git remote add} (when you initialized the repository) or @command{git config --add}, i.e. it updates all remote branches from remote @code{origin}, then it merges the remote branch tracked by current branch into current branch. For example, if your current branch is @code{master} --- which is the case if you got the sources with the commands described in @ref{Main source code} and did not issue any @command{git checkout} command --- @code{origin/master} will be merged into @code{master}. @subsubheading Local clones, or having several working trees If you play with several Git branches, e.g. @code{master}, @code{lilypond/translation}, @code{stable/2.12}), you may want to have one source and build tree for each branch; this is possible with subdirectories of your local Git repository, used as local cloned subrepositories. To create a local clone for the branch named @code{@var{branch}}, run @example git checkout @var{branch} git clone -l -s -n . @var{subdir} cd @var{subdir} git reset --hard @end example Note that @code{@var{subdir}} must be a directory name which does not already exist. In @code{@var{subdir}}, you can use all Git commands to browse revisions history, commit and uncommit changes; to update the cloned subrepository with changes made on the main repository, cd into @code{@var{subdir}} and run @command{git pull}; to send changes made on the subrepository back to the main repository, run @command{git push} from @code{@var{subdir}}. Note that only one branch (the currently checked out branch) is created in the subrepository by default; it is possible to have several branches in a subrepository and do usual operations (checkout, merge, create, delete...) on these branches, but this possibility is not detailed here. When you push @code{@var{branch}} from @code{@var{subdir}} to the main repository, and @code{@var{branch}} is checked out in the main repository, you must save uncommitted changes (see @command{git stash}) and do @command{git reset --hard} in the main repository in order to apply pushed changes in the working tree of the main repository. @node Git log @subsection Git log The commands above don't only bring you the latest version of the sources, but also the full history of revisions (revisons, also called commits, are changes made to the sources), stored in the .git directory. You can browse this history with @example git log # only shows the logs (author, committish and commit message) git log -p # also shows diffs gitk # shows history graphically @end example @warning{The @code{gitk} command may require a separate @code{gitk} package, available in the appropriate distribution's repositories.} @node Applying git patches @subsection Applying git patches Well-formed git patches created with @code{git format-patch} should be committed with the following command: @example git am @var{patch} @end example Patches created without @code{git format-patch} can be applied in two steps. The first step is to apply the patch to the working tree: @example git apply @var{patch} @end example @noindent The second step is to commit the changes and give credit to the author of the patch. This can be done with the following command: @example git commit -a --author="First Last " @end example @node Reverting all local changes @subsection Reverting all local changes Sometimes git will become hopelessly confused, and you just want to get back to a known, stable state. This command destroys any local changes you have made, but at least you get back to the current online version: @example git reset --hard origin/master @end example @node Git on Windows @section Git on Windows @c Some of this may duplicate stuff in other sections @c Clear this up later -td @subsection Background to nomenclature Git is a system for tracking the changes made to source files by a distributed set of editors. It is designed to work without a master repository, but we have chosen to have a master respository for LilyPond files. Editors hold local copies of the master repository together with any changes they have made locally. Local changes are held in a local @q{branch}, of which there may be several, but these instructions assume you are using just one. The files visible in the local repository always correspond to those on the currently @q{checked out} local branch. Files are edited on a local branch, and in that state the changes are said to be @q{unstaged}. When editing is complete, the changes are moved to being @q{staged for commit}, and finally the changes are @q{committed} to the local branch. Once committed, the changes are given a unique reference number called the @q{Committish} which identifies them to Git. Such committed changes can be sent to the master repository by @q{pushing} them (if you have write permission) or by sending them by email to someone who has, either complete or as a @q{diff} or @q{patch} (which send just the differences from master). @subsection Installing git Obtain Git from @uref{http://code.google.com/p/msysgit/downloads/list} (note, not msysGit, which is for Git developers and not PortableGit, which is not a full git installation) and install it. Note that most users will not need to install SSH. That is not required until you have been granted direct push permissions to the master git repository. Start Git by clicking on the desktop icon. This will bring up a command line bash shell. This may be unfamiliar to Windows users. If so, follow these instructions carefully. Commands are entered at a $ prompt and are terminated by keying a newline. @subsection Initialising Git Decide where you wish to place your local Git repository, creating the folders in Windows as necessary. Here we call the folder to contain the repository [path]/Git. You will need to have space for around 150Mbytes. Start the Git bash shell by clicking on the desk-top icon installed with Git and type @example cd [path]/Git @end example to position the shell at your new Git repository. Note: if [path] contains folders with names containing spaces use @example cd "[path]/Git" @end example Then type @example git init @end example to initialize your Git repository. Then type (all on one line; the shell will wrap automatically) @example git remote add -f -t master origin git://git.sv.gnu.org/lilypond.git @end example to download the lilypond master files. @warning{Be patient! Even on a broadband connection this can take 10 minutes or more. Wait for lots of [new tag] messages and the $ prompt.} We now need to generate a local copy of the downloaded files in a new local branch. Your local branch needs to have a name, here we call it @q{lily-local} - you may wish to make up your own. Then, finally, type @example git checkout -b lily-local origin/master @end example to create the lily-local branch containing the local copies of the master files. You will be advised your local branch has been set up to track the remote branch. Return to Windows Explorer and look in your Git repository. You should see lots of folders. For example, the LilyPond documentation can be found in Git/Documentation/user. Terminate the Git bash shell by typing @code{exit}. @subsection Git GUI Almost all subsequent work will use the Git Graphical User Interface, which avoids having to type command line commands. To start Git GUI first start the Git bash shell by clicking on the desktop icon, and type @example cd [path]/Git git gui @end example The Git GUI will open in a new window. It contains four panels and 7 pull-down menus. At this stage do not use any of the commands under Branch, Commit, Merge or Remote. These will be explained later. The two panels on the left contain the names of files which you are in the process of editing (Unstaged Changes), and files you have finished editing and have staged ready for committing (Staged Changes). At this stage these panels will be empty as you have not yet made any changes to any file. After a file has been edited and saved the top panel on the right will display the differences between the edited file selected in one of the panels on the left and the last version committed. The final panel at bottom right is used to enter a descriptive message about the change before committing it. The Git GUI is terminated by entering CNTL-Q while it is the active window or by clicking on the usual Windows close-window widget. @subsection Personalising your local git repository Open the Git GUI, click on @example Edit -> Options @end example and enter your name and email address in the left-hand (Git Repository) panel. Leave everything else unchanged and save it. Note that Windows users must leave the default setting for line endings unchanged. All files in a git repository must have lines terminated by just a LF, as this is required for Merge to work, but Windows files are terminated by CRLF by default. The git default setting causes the line endings of files in a Windows git repository to be flipped automatically between LF and CRLF as required. This enables files to be edited by any Windows editor without causing problems in the git repository. @subsection Checking out a branch At this stage you have two branches in your local repository, both identical. To see them click on @example Branch -> Checkout @end example You should have one local branch called @w{lily-local} and one tracking branch called @w{origin/master}. The latter is your local copy of the @w{remote/origin/master} branch in the master LilyPond repository. The @w{lily-local} branch is where you will make your local changes. When a particular branch is selected, i.e., checked out, the files visible in your repository are changed to reflect the state of the files on that branch. @subsection Updating files from @w{remote/origin/master} Before starting the editing of a file, ensure your local branches contain the latest version in @w{remote/origin/master} by first clicking @example Remote -> Fetch from -> origin @end example @noindent in the Git GUI. This will place the latest version of every file, including all the changes made by others, into the @q{origin/master} branch of the tracking branches in your git repository. You can see these files by checking out this branch. This will not affect any files you have modified in your local branch. You then need to merge these fetched files into your local branch by clicking on @example Merge -> Local Merge @end example @noindent and if necessary select the local branch into which the merge is to be made. Note that a merge cannot be completed if there are any local uncommitted changes on the lily-local branch. This will update all the files in that branch to reflect the current state of the @w{origin/master} branch. If any of the changes conflict with changes you have made yourself recently you will be notified of the conflict (see below). @subsection Editing files First ensure your lily-local branch is checked out, then simply edit the files in your local Git repository with your favourite editor and save them back there. If any file contains non-ASCII characters ensure you save it in UTF-8 format. Git will detect any changes whenever you restart Git GUI and the file names will then be listed in the Unstaged Changes panel. Or you can click the Rescan button to refresh the panel contents at any time. You may break off and resume at editing any time. The changes you have made may be displayed in diff form in the top right-hand panel by clicking on the name in Git GUI. When your editing is complete, move the files from being Unstaged to Staged by clicking the document symbol to the left of each name. If you change your mind it can be moved back by clicking on the ticked box to the left of the name. Finally the changes you have made may be committed to your lily-local branch by entering a brief message in the Commit Message box and clicking the Commit button. If you wish to amend your changes after a commit has been made, the original version and the changes you made in that commit may be recovered by selecting @example Commit -> Amend Last Commit @end example @noindent or by checking the Amend Last Commit radio button at bottom left. This will return the changes to the Staged state, so further editing made be carried out within that commit. This must only be done @emph{before} the changes have been Pushed or sent to your mentor for Pushing - after that it is too late and corrections have to be made as a separate commit. @subsection Sending changes to remote/origin/master If you do not have write access to @w{remote/origin/master} you will need to send your changes by email to someone who does. First you need to create a diff or patch file containing your changes. To create this, the file must first be committed. Then terminate the Git GUI. In the git bash shell first cd to your Git repository with @example cd [path]/Git @end example if necessary, then produce the patch with @example git format-patch origin @end example This will create a patch file for all the locally committed files which differ from @w{origin/master}. The patch file can be found in [path]/Git and will have a name formed from n and the commit message. @subsection Resolving merge conflicts As soon as you have committed a changed file your local branch has diverged from @w{origin/master}, and will remain diverged until your changes have been committed in @w{remote/origin/master} and Fetched back into your @w{origin/master}. Similarly, if a new commit has been made to @w{remote/origin/master} by someone else and Fetched, your lily-local branch is divergent. You can detect a divergent branch by clicking on @example Repository -> Visualise all branch history @end example This opens up a very useful new window called @q{gitk}. Use this to browse all the commits made by others. If the diagram at top left of the resulting window does not show your branch's tag on the same node as the @w{remote/origins/master} tag your branch has diverged from @w{origin/master}. This is quite normal if files you have modified yourself have not yet been Pushed to @w{remote/origin/master} and Fetched, or if files modified and committed by others have been Fetched since you last Merged @w{origin/master} into your lily-local branch. If a file being merged from @w{origin/master} differs from one you have modified in a way that cannot be resolved automatically by git, Merge will report a Conflict which you must resolve by editing the file to create the version you wish to keep. This could happen if the person updating @w{remote/origin/master} for you has added some changes of his own before committing your changes to @w{remote/origin/master}, or if someone else has changed the same file since you last fetched the file from @w{remote/origin/master}. Open the file in your editor and look for sections which are delimited with ... [to be completed when I next have a merge conflict to be sure I give the right instructions -td] @subsection Other actions The instructions above describe the simplest way of using git on Windows. Other git facilities which may usefully supplement these include @itemize @item Using multiple local branches (Create, Rename, Delete) @item Resetting branches @item Cherry-picking commits @item Pushing commits to @w{remote/origin/master} @item Using gitk to review history @end itemize Once familiarity with using git on Windows has been gained the standard git manuals can be used to learn about these. @node Development inside VirtualBox (compiling on Windows) @section Development inside VirtualBox (compiling on Windows)