X-Git-Url: https://git.donarmstrong.com/?a=blobdiff_plain;f=Documentation%2Fcontributor%2Frelease-work.itexi;h=8227763e40d08f819ede3f958451a49aba6c6736;hb=730d4eb3e0328e8ba0b376f8191a5bb6ae7f904c;hp=10bf481b362020139b447833825b020483885e80;hpb=08666d43e1dd61b654c40a77d5785544065af423;p=lilypond.git diff --git a/Documentation/contributor/release-work.itexi b/Documentation/contributor/release-work.itexi index 10bf481b36..8227763e40 100644 --- a/Documentation/contributor/release-work.itexi +++ b/Documentation/contributor/release-work.itexi @@ -7,24 +7,45 @@ * Minor release checklist:: * Major release checklist:: * Release extra notes:: +* Notes on builds with GUB:: @end menu @node Development phases @section Development phases -There are 2.5 states of development for LilyPond. +There are 2 states of development on @code{master}: -@itemize +@enumerate + +@item @strong{Normal development}: +Any commits are fine. + +@item @strong{Build-frozen}: +Do not require any additional or updated libraries or make +non-trivial changes to the build process. Any such patch (or +branch) may not be merged with master during this period. + +This should occur approximately 1 month before any alpha version +of the next stable release, and ends when the next unstable branch +begins. + +@end enumerate + + +After announcing a beta release, branch @code{stable/2.x}. There +are 2 states of development for this branch: -@item @strong{Stable phase}: -Starting from the release of a new major version @code{2.x.0}, the -following patches @strong{MAY NOT} be merged with master: +@enumerate +@item @strong{Normal maintenance}: +The following patches @strong{MAY NOT} be merged with this branch: @itemize @item Any change to the input syntax. If a file compiled with a -previous @code{2.x} version, then it must compile in the new -version. +previous @code{2.x} (beta) version, then it must compile in the +new version. + +Exception: any bugfix to a Critical issue. @item New features with new syntax @emph{may be committed}, although once committed that syntax cannot change during the @@ -38,74 +59,118 @@ version. @end itemize -@item @strong{Development phase}: -Any commits are fine. Readers may be familiar with the term -@qq{merge window} from following Linux kernel news. +@item @strong{Release prep}: +Only translation updates and important bugfixes are allowed. +@end enumerate -@item @strong{Release prep phase}: -TODO: I don't like that name. - -A new git branch @code{stable/2.x} is created, and a major release -is made in two weeks. -@itemize +@node Minor release checklist +@section Minor release checklist -@item @code{stable/2.x branch}: -Only translation updates and important bugfixes are allows. +A @qq{minor release} means an update of @code{y} in @code{2.x.y}. -@item @code{master}: -Normal @qq{stable phase} development occurs. +@subheading Pre-release -@end itemize +@enumerate -If we discover the need to change the syntax or build system, we -will apply it and re-start the release prep phase. +@item +Using any system with git pull access (not necessarily the GUB +build machine), use the commands below to do the following: -@end itemize +@itemize +@item +switch to the release branch -This marks a radical change from previous practice in LilyPond. -However, this setup is not intended to slow development -- as a -rule of thumb, the next development phase will start within a -month of somebody wanting to commit something which is not -permitted during the stable phase. +@item +update the release branch from origin/master +@item +update the translation files +@item +create the release announcement -@node Minor release checklist -@section Minor release checklist +@item +update the build versions. -A @qq{minor release} means an update of @code{y} in @code{2.x.y}. +@itemize +@item +VERSION_DEVEL = the current development version (previous VERSION_DEVEL + 0.01) -@subheading Pre-release +@item +VERSION_STABLE = the current stable version (probably no change here) -@enumerate +@end itemize @item -Switch to the release branch, get changes, prep release -announcement: +update the @qq{Welcome to Lilypond} version numbers to the version about to be +released + +@end itemize + +This requires a system which has the release/unstable +branch. If you get a warning saying you are in @code{detached HEAD} +state, then you should create a release/unstable branch with +@code{git checkout release/unstable}. + +Check the environment variables are set as in +@ref{Environment variables}. + +You need to ensure you have a clean index and work tree. If the +checkout displays modified files, you might want to run +@code{git reset --hard} before continuing. @example +git fetch git checkout release/unstable -git merge origin -vi Documentation/web/news-front.itexi Documentation/web/news.itexi +git merge origin/master +make -C $LILYPOND_BUILD_DIR po-replace +mv $LILYPOND_BUILD_DIR/po/lilypond.pot po/ +gedit Documentation/web/news-front.itexi Documentation/web/news.itexi +gedit VERSION +gedit ly/Wel*.ly @end example @item -Commit, push, switch back to master: +Commit, push, switch back to master (or wherever else): @example +git commit -m "Release: bump VERSION_DEVEL." VERSION +git commit -m "Release: bump VERSION_DEVEL." ly/Welcome_to_LilyPond.ly +git commit -m "Release: bump VERSION_DEVEL." ly/Welcome-to-LilyPond-MacOS.ly +git commit -m "PO: update template." po/lilypond.pot git commit -m "Release: update news." Documentation/web/ -git push origin +git commit -m "Release: bump Welcome versions." ly/Wel*.ly +git push origin HEAD:release/unstable +git checkout master @end example -@item (optional) Check that lilypond builds from scratch in an -out-of-tree build. - @item If you do not have the previous release test-output tarball, download it and put it in @code{regtests/} +@item Prepare GUB environment by running: + +@example +### my-gub.sh +# special terminal, and default PATH environment. +# import these special environment vars: +# HOME, HTTP_PROXY, TERM +env -i \ + HOME=$HOME \ + HTTP_PROXY=$HTTP_PROXY \ + bash --rcfile my-bashrc +@end example + +@example +### my-bashrc +export PS1="\[\e[1;33mGUB-ENV \w\]$ \[\e[0m\]" +export PATH=$PATH +export TERM=xterm +@end example + + @item Build release on GUB by running: @example @@ -116,18 +181,20 @@ make LILYPOND_BRANCH=release/unstable lilypond or something like: @example -make LILYPOND_BRANCH=stable/2.12 lilypond +make LILYPOND_BRANCH=stable/2.16 lilypond @end example -@item Check the regtest comparison in @file{uploads/@/webtest/} for +@item +Check the regtest comparison in @file{uploads/webtest/} for any unintentional breakage. More info in -@ref{Precompiled regression tests} +@ref{Precompiled regression tests}. -@item If any work was done on GUB since the last release, upload +@item +If any work was done on GUB since the last release, upload binaries to a temporary location, ask for feedback, and wait a day -or two in case there's any major problems. Or live dangerously -and just add a sentence to the release notes. Or live even more -dangerously, and don't tell anybody anything. +or two in case there's any major problems. + +@warning{Always do this for a stable release.} @end enumerate @@ -136,23 +203,29 @@ dangerously, and don't tell anybody anything. @enumerate -@item If you're not right user on the webserver, remove the "t" -from the rsync command in @file{test@/-lily/@/rsync@/-lily@/-doc@/.py} and -@file{test@/-lily/@/rsync@/-test@/.py} +@item If you're not the right user on the webserver, remove the +@code{t} from the rsync command in: -@code{graham} owns v2.13; @code{han-wen} owns v2.12. +@example +test-lily/rsync-lily-doc.py +test-lily/rsync-test.py +@end example @item Upload GUB by running: @example -make lilypond-upload LILYPOND_BRANCH=release/unstable LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git +make lilypond-upload \ + LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git \ + LILYPOND_BRANCH=release/unstable @end example @noindent or something like: @example -make lilypond-upload LILYPOND_BRANCH=stable/2.12 LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git +make lilypond-upload \ + LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git \ + LILYPOND_BRANCH=stable/2.12 @end example @end enumerate @@ -162,52 +235,43 @@ make lilypond-upload LILYPOND_BRANCH=stable/2.12 LILYPOND_REPO_URL=git://git.sv. @enumerate -@item Switch back to master and get the updated news: +@item +Update the current staging branch with the current news: @example -git checkout master -git merge release/unstable +git fetch +git checkout origin/staging +git merge origin/release/unstable @end example -@item Update @file{VERSION} in lilypond git and upload changes: +@item +Update @file{VERSION} in lilypond git and upload changes: @example -vi VERSION +gedit VERSION @end example @itemize @item VERSION = what you just did +0.0.1 - -@item -DEVEL_VERSION = what you just did (i.e. is now online) - -@item -STABLE_VERSION = what's online (probably no change here) - @end itemize @example -git commit -m "Release: bump version." VERSION -git push origin +git commit -m "Release: bump VERSION." VERSION +git push origin HEAD:staging @end example - -@item (for now) do a @code{make doc} and manually upload: +If the push fails with a message like @example -### upload-lily-web-media.sh -#!/bin/sh -BUILD_DIR=$HOME/src/build-lilypond - -PICS=$BUILD_DIR/Documentation/pictures/out-www/ -EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/out-www/ - -cd $BUILD_DIR -rsync -a $PICS graham@@lilypond.org:media/pictures -rsync -a $EXAMPLES graham@@lilypond.org:media/ly-examples + ! [rejected] HEAD -> staging (non-fast-forward) @end example +@noindent +it means that somebody else updated the staging branch while you were +preparing your change. In that case, you need to restart the Post +Release process. Otherwise, proceed: + @item Wait a few hours for the website to update. @item Email release notice to @code{info-lilypond} @@ -221,112 +285,192 @@ rsync -a $EXAMPLES graham@@lilypond.org:media/ly-examples A @qq{major release} means an update of @code{x} in @code{2.x.0}. -- happens when we have 0 Critical issues for two weeks (14 days). +@subheading Main requirements +These are the current official guidelines. -Before release: +@itemize +@item +0 Critical issues for two weeks (14 days) after the latest release +candidate. -* write release notes. note: stringent size requirements for - various websites, so be brief. +@end itemize -* write preface section for manual. -* submit pots for translation : send url of tarball to -translation@@iro.umontreal.ca, mentioning lilypond-VERSION.pot +@subheading Potential requirements -* Check reg test +These might become official guidelines in the future. -* Check all 2ly scripts. +@itemize +@item +Check reg test -* Run convert-ly on all files, bump parser minimum version. +@item +Check all 2ly scripts -* update links to distros providing lilypond packages? link in -Documentation/web/download.itexi . This has nothing to do with -the release, but I'm dumping this here so I'll find it when I -reorganize this list later. -gp +@item +Check for emergencies the docs: -* Make FTP directories on lilypond.org +@example +grep FIXME --exclude "misc/*" --exclude "*GNUmakefile" \ + --exclude "snippets/*" ????*/* +@end example -* website: - - Make new table in download.html +@item +Check for altered regtests, and document as necessary: - - add to documentation list +@example +git diff -u -r release/2.@var{FIRST-CURRENT-STABLE} \ + -r release/2.@var{LAST-CURRENT-DEVELOPMENT} input/regression/ +@end example - - revise examples tour.html/howto.html +@end itemize - - add to front-page quick links - - change all links to the stable documentation +@subheading Housekeeping requirements - - make a link from the old unstable to the next stable in - lilypond.org's /doc/ dir. Keep all previous unstable->stable - doc symlinks. +Before the release: - - doc auto redirects to v2.LATEST-STABLE +@itemize +@item +write release notes. note: stringent size requirements for +various websites, so be brief. - - add these two lines to http://www.lilypond.org/robots.txt: +@item +Run convert-ly on all files, bump parser minimum version. + +@item +Update lilypond.pot: @example -Disallow: /doc/v2.PREVIOUS-STABLE/ -Disallow: /doc/v2.CURRENT-DEVELOPMENT/ +make -C $LILYPOND_BUILD_DIR po-replace +mv $LILYPOND_BUILD_DIR/po/lilypond.pot po/ @end example -- check for emergencies the docs: +@item +Make directories on lilypond.org: @example -grep FIXME --exclude "misc/*" --exclude "*GNUmakefile" \ - --exclude "snippets/*" ????*/* +~/download/sources/v2.@var{NEW-STABLE} +~/download/sources/v2.@var{NEW-DEVELOPMENT} @end example -- check for altered regtests, and document as necessary. (update - tags as appropriate) +@item +Shortly after the release, move all current contributors to +previous contributors in +@file{Documentation/included/authors.itexi}. + +@item +Delete old material in @file{Documentation/changes.tely}, but +don't forget to check it still compiles! Also update the version +numbers: @example -git diff -u -r release/2.12.0-1 -r release/2.13.13-1 input/regression/ +@@node Top +@@top New features in 2.@var{NEW-STABLE} since 2.@var{OLD-STABLE} @end example +@item +Website: + +@itemize +@item +make a link from the old unstable to the next stable in +lilypond.org's @file{/doc/} dir. Keep all previous unstable->stable +doc symlinks. + +Also, make the old docs self-contained -- if there's a redirect in +@file{/doc/v2.@var{OLD-STABLE}/Documentation/index.html} , replace it with the +@file{index.html.old-2.@var{OLD-STABLE}} files. + +The post-2.13 docs will need another way of handling the +self-containment. It won't be hard to whip up a python script +that changes the link to @file{../../../../manuals.html} to +@file{../website/manuals.html}, but it's still a 30-minute task that +needs to be done before 2.16. + +@item +doc auto redirects to @code{v2.@var{NEW-STABLE}} + +@item +add these two lines to @file{Documentation/web/server/robots.txt}: + +@example +Disallow: /doc/v2.@var{OLD-STABLE}/ +Disallow: /doc/v2.@var{NEW-DEVELOPMENT}/ +@end example + +@end itemize + +@end itemize + +@subheading Unsorted + +@itemize +@item +submit po template for translation: send url of tarball to +@email{coordinator@@translationproject.org}, mentioning +lilypond-VERSION.pot + +@item +update links to distros providing lilypond packages? link in: +@file{Documentation/web/download.itexi} + +This has nothing to do with the release, but it's a @qq{periodic +maintenance} task that might make sense to include with releases. + +@item +Send announcements to... News: - comp.music.research - comp.os.linux.announce +@example +comp.music.research +comp.os.linux.announce - comp.text.tex - rec.music.compose +comp.text.tex +rec.music.compose +@end example Mail: - info-lilypond@@gnu.org +@example +info-lilypond@@gnu.org linux-audio-announce@@lists.linuxaudio.org linux-audio-user@@lists.linuxaudio.org linux-audio-dev@@lists.linuxaudio.org - tex-music@@icking-music-archive.org +tex-music@@icking-music-archive.org - --- non-existant? - abcusers@@blackmill.net +--- non-existant? +abcusers@@blackmill.net - rosegarden-user@@lists.sourceforge.net - info-gnu@@gnu.org - noteedit-user@@berlios.de +rosegarden-user@@lists.sourceforge.net +info-gnu@@gnu.org +noteedit-user@@berlios.de - gmane.comp.audio.fomus.devel - gmane.linux.audio.users - gmane.linux.audio.announce - gmane.comp.audio.rosegarden.devel +gmane.comp.audio.fomus.devel +gmane.linux.audio.users +gmane.linux.audio.announce +gmane.comp.audio.rosegarden.devel +@end example Web: - lilypond.org - freshmeat.net - linuxfr.com - http://www.apple.com/downloads - harmony-central.com (news@@harmony-central.com) - versiontracker.com [auto] - hitsquad.com [auto] - http://www.svgx.org - https://savannah.gnu.org/news/submit.php?group_id=1673 @c => planet.gnu.org +@example +lilypond.org +freshmeat.net +linuxfr.com +http://www.apple.com/downloads +harmony-central.com (news@@harmony-central.com) +versiontracker.com [auto] +hitsquad.com [auto] +http://www.svgx.org +https://savannah.gnu.org/news/submit.php?group_id=1673 @c => planet.gnu.org +@end example + +@end itemize @node Release extra notes @@ -346,7 +490,7 @@ take lilypond-book and any related makefile updates from the latest git. @item -- configure; make; make test +configure; make; make test @item tar -cjf lilypond-X.Y.Z-A.test-output.tar.bz2 input/regression/out-test/ @@ -400,7 +544,8 @@ build the specific package(s) you want, i.e. @example bin/gub mingw::lilypond-installer make LILYPOND_BRANCH=stable/2.12 -f lilypond.make doc -bin/gub --platform=darwin-x86 'git://git.sv.gnu.org/lilypond-doc.git?branch=stable/2.12' +bin/gub --platform=darwin-x86 \ + 'git://git.sv.gnu.org/lilypond-doc.git?branch=stable/2.12' @end example or @@ -419,5 +564,159 @@ delete stuff you didn't want to upload from the server. @end enumerate +@node Notes on builds with GUB +@section Notes on builds with GUB + +@subsubheading Building GUB + +GUB - the Grand Unified Builder - is used to build the release +versions of LilyPond. For background information, see +@ref{Grand Unified Builder (GUB)}. The simplest way to set up a +GUB build environment is to use a virtual machine with LilyDev +(@ref{LilyDev}). Follow the instructions on that page to set this +up. Make sure that your virtual machine has enough disk space - +a GUB installation takes over 30 GBytes of disk space, and if you +allocate too little, it will fail during the setting up stage and +you will have to start again. 64 GBytes should be sufficient. + +While GUB is being built, any interruptions are likely to make it +almost impossible to restart. If at all possible, leave the build +to continue uninterrupted. + +Download GUB and start the set up: +@example +git clone git://github.com/gperciva/gub/gub.git +cd gub +make bootstrap +@end example + +This will take a very long time, even on a very fast computer. +You will need to be patient. It's also liable to fail - it +downloads a number of tools, and some will have moved and others +won't respond to the network. For example, the perl archive. +If this happens, download it from +@uref{http://www.cpan.org/src/5.0/perl-5.10.0.tar.gz}, saving the +archive to @file{gub/downloads/perl/}. Continue the set up with: + +@example +make bootstrap +@end example + +Once this has completed successfully, you can build the LilyPond +release package. However, this uses an archived version of the +regression tests, so it is better to download this first. +Download the test output from lilypond.org (you will need to +replace @code{2.15.33-1} with the latest build): + +@smallexample +@uref{http://lilypond.org/download/binaries/test-output/lilypond-2.15.33-1.test-output.tar.bz2} +@end smallexample + +Copy the tarball into @file{gub/regtests/}, and tell the build +system that you have done this: + +@example +touch regtests/ignore +@end example + +Now start the GUB build: + +@example +make lilypond +@end example + +That's it. This will build LilyPond from current master. To build +the current unstable release, run: + +@example +make LILYPOND_BRANCH=release/unstable lilypond +@end example + +The first time you do this, it will take a very long time. + +Assuming the build has gone well, it can be uploaded using: + +@example +make lilypond-upload + LILYPOND_BRANCH=release/unstable + LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git +@end example + +@subsubheading Output files + +GUB builds the files it needs into the directory +@code{gub/target/}. As a general rule, these don't need to be +touched unless there is a problem building GUB (see below). +The files to be uploaded are in @code{gub/uploads/}. Once the +build has completed successfully, there should be 8 +installation files and 3 archives, totalling about 600MB. +There are also 4 directories: + +@example +gub/signatures +gub/localdoc +gub/webdoc +gub/webtest +@end example + +@code{signatures} contains files that are used to track whether +some of the archives have already been built. Don't touch +these. + +@code{localdoc} probably contains local copies of the +documentation. + +@code{webdoc} contains the documentation to be uploaded. + +@code{webtest} contains the regtest comparison, which should +be checked before upload, and is also uploaded for subsequent +checking. + +The total upload is about 700 MB in total, and on an ADSL +connection will take about 4 hours to upload. + +@subsubheading Subsequent builds + +In principle, building the next release of LilyPond requires +no action other then following the instructions in +@ref{Minor release checklist}. Because much of the +infrastructure has already been built, it will take much less +time - about an hour on a fast computer. + +Continuing to build LilyPond without any other +archiving/deletion of previous builds is likely to be successful, +but will take up a fair amount of disk space (around 2GB per +build) which may be a problem with a Virtual Machine. It's +therefore recommended to move (not copy) @code{gub/uploads} to +another machine/disk after each build, if space is at a premium. + +However, if a significant change has been made to the LilyPond +source (e.g. added source files) the build may fail if tried on +top of a previous build. If this happens, be sure to +move/delete @code{gub/uploads} and all mentions of LilyPond +in @code{gub/target}. The latter can be achieved with this +command: + +@example +rm -rf target/*/*/*lilypond* +@end example +Be @emph{very} careful with this command. Typing it wrongly +could wipe your disk completely. + +@subsubheading Updating the web site + +The @code{make lilypond-upload} command updates the documentation +on the LilyPond web site. However, it does @emph{not} update +any part of the site that is not part of the documentation - for +example, the front page (@code{index.html}). The website is +updated by 2 cron jobs running on the web server. One of these +pulls git master to the web server, and the other makes the +website with the standard @code{make website} command. They run +hourly, 30 minutes apart. So - to update the front page of the +website, it's necessary to update @code{VERSION} and +@code{news-front.itexi} in master and then wait for the cron +jobs to run. (N.B. - this is done by pushing the changes to +staging and letting patchy do its checks before it pushes to +master).