X-Git-Url: https://git.donarmstrong.com/?a=blobdiff_plain;f=Documentation%2Fcontributor%2Frelease-work.itexi;h=9120e6529ff1c225513026d21df7bd1a4e83cfd6;hb=4c027ac238950c80e80f2a18632ca1c3a795eb3a;hp=929a09988699cc7545a8bb32c6273e2996ba4f2c;hpb=11a8f13d5f9ca574cccc4bd5ecb0a159411be022;p=lilypond.git diff --git a/Documentation/contributor/release-work.itexi b/Documentation/contributor/release-work.itexi index 929a099886..9120e6529f 100644 --- a/Documentation/contributor/release-work.itexi +++ b/Documentation/contributor/release-work.itexi @@ -7,25 +7,45 @@ * Minor release checklist:: * Major release checklist:: * Release extra notes:: -* Administrative policies:: +* Notes on builds with GUB:: @end menu @node Development phases @section Development phases -There are 2.5 states of development for LilyPond. +There are 2 states of development on @code{master}: -@itemize +@enumerate + +@item @strong{Normal development}: +Any commits are fine. -@item @strong{Stable phase}: -Starting from the release of a new major version @code{2.x.0}, the -following patches @strong{MAY NOT} be merged with master: +@item @strong{Build-frozen}: +Do not require any additional or updated libraries or make +non-trivial changes to the build process. Any such patch (or +branch) may not be merged with master during this period. + +This should occur approximately 1 month before any alpha version +of the next stable release, and ends when the next unstable branch +begins. + +@end enumerate + + +After announcing a beta release, branch @code{stable/2.x}. There +are 2 states of development for this branch: + +@enumerate +@item @strong{Normal maintenance}: +The following patches @strong{MAY NOT} be merged with this branch: @itemize @item Any change to the input syntax. If a file compiled with a -previous @code{2.x} version, then it must compile in the new -version. +previous @code{2.x} (beta) version, then it must compile in the +new version. + +Exception: any bugfix to a Critical issue. @item New features with new syntax @emph{may be committed}, although once committed that syntax cannot change during the @@ -39,38 +59,10 @@ version. @end itemize -@item @strong{Development phase}: -Any commits are fine. Readers may be familiar with the term -@qq{merge window} from following Linux kernel news. - - -@item @strong{Release prep phase}: -TODO: I don't like that name. - -A new git branch @code{stable/2.x} is created, and a major release -is made in two weeks. - -@itemize - -@item @code{stable/2.x branch}: -Only translation updates and important bugfixes are allows. - -@item @code{master}: -Normal @qq{stable phase} development occurs. - -@end itemize - -If we discover the need to change the syntax or build system, we -will apply it and re-start the release prep phase. - -@end itemize - -This marks a radical change from previous practice in LilyPond. -However, this setup is not intended to slow development -- as a -rule of thumb, the next development phase will start within a -month of somebody wanting to commit something which is not -permitted during the stable phase. +@item @strong{Release prep}: +Only translation updates and important bugfixes are allowed. +@end enumerate @node Minor release checklist @@ -83,32 +75,65 @@ A @qq{minor release} means an update of @code{y} in @code{2.x.y}. @enumerate @item -Switch to the release branch, get changes, prep release -announcement: +Using any system with git pull access (not necessarily the GUB +build machine), use the commands below to switch to the release +branch, get changes and prepare the release +announcement. This requires a system which has the release/unstable +branch. If you get a warning saying you are in @code{detached HEAD} +state, then you should create a release/unstable branch with +@code{git checkout release/unstable}. + +Check the environment variables are set as in +@ref{Environment variables}. + +You need to ensure you have a clean index and work tree. If the +checkout displays modified files, you might want to run +@code{git reset --hard} before continuing. @example +git fetch git checkout release/unstable -git merge origin -vi Documentation/web/news-front.itexi Documentation/web/news.itexi +git merge origin/master +make -C $LILYPOND_BUILD_DIR po-replace +mv $LILYPOND_BUILD_DIR/po/lilypond.pot po/ +gedit Documentation/web/news-front.itexi Documentation/web/news.itexi @end example @item -Commit, push, switch back to master: +Commit, push, switch back to master (or wherever else): @example +git commit -m "PO: update template." po/lilypond.pot git commit -m "Release: update news." Documentation/web/ -git push origin +git push origin HEAD:release/unstable git checkout master -git merge release/unstable @end example -@item (optional) Check that lilypond builds from scratch in an -out-of-tree build. - @item If you do not have the previous release test-output tarball, download it and put it in @code{regtests/} +@item Prepare GUB environment by running: + +@example +### my-gub.sh +# special terminal, and default PATH environment. +# import these special environment vars: +# HOME, HTTP_PROXY, TERM +env -i \ + HOME=$HOME \ + HTTP_PROXY=$HTTP_PROXY \ + bash --rcfile my-bashrc +@end example + +@example +### my-bashrc +export PS1="\[\e[1;33mGUB-ENV \w\]$ \[\e[0m\]" +export PATH=$PATH +export TERM=xterm +@end example + + @item Build release on GUB by running: @example @@ -123,16 +148,15 @@ make LILYPOND_BRANCH=stable/2.12 lilypond @end example @item Check the regtest comparison in @file{uploads/webtest/} for -any unintentional breakage. - -Note that this test uses the bounding boxes inside lilypond. -Errors in ghostscript don't generate differences inside lilypond, -so they are not registered in the regtest comparison. +any unintentional breakage. More info in +@ref{Precompiled regression tests}. @item If any work was done on GUB since the last release, upload binaries to a temporary location, ask for feedback, and wait a day or two in case there's any major problems. +@warning{Always do this for a stable release.} + @end enumerate @@ -140,23 +164,31 @@ or two in case there's any major problems. @enumerate -@item If you're not right user on the webserver, remove the "t" -from the rsync command in @file{test-lily/rsync-lily-doc.py} and -@file{test-lily/rsync-test.py} +@item If you're not the right user on the webserver, remove the +@code{t} from the rsync command in: + +@example +test-lily/rsync-lily-doc.py +test-lily/rsync-test.py +@end example @code{graham} owns v2.13; @code{han-wen} owns v2.12. @item Upload GUB by running: @example -make lilypond-upload LILYPOND_BRANCH=master LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git +make lilypond-upload \ + LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git \ + LILYPOND_BRANCH=release/unstable @end example @noindent or something like: @example -make lilypond-upload LILYPOND_BRANCH=stable/2.12 LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git +make lilypond-upload \ + LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git \ + LILYPOND_BRANCH=stable/2.12 @end example @end enumerate @@ -166,22 +198,47 @@ make lilypond-upload LILYPOND_BRANCH=stable/2.12 LILYPOND_REPO_URL=git://git.sv. @enumerate -@item Switch back to master and get the updated news: +@item Update the current staging branch with the current news: @example -git checkout master -git merge release/unstable +git fetch +git checkout origin/staging +git merge origin/release/unstable @end example -@item Update @file{VERSION} in lilypond git: +@item Update @file{VERSION} in lilypond git and upload changes: @example +vi VERSION +@end example + +@itemize +@item VERSION = what you just did +0.0.1 + +@item DEVEL_VERSION = what you just did (i.e. is now online) -STABLE_VERSION = what's online + +@item +STABLE_VERSION = what's online (probably no change here) + +@end itemize + +@example +git commit -m "Release: bump version." VERSION +git push origin HEAD:staging +@end example + +If the push fails with a message like + +@example + ! [rejected] HEAD -> staging (non-fast-forward) @end example -@item Push changes. +@noindent +it means that somebody else updated the staging branch while you were +preparing your change. In that case, you need to restart the Post +Release process. Otherwise, proceed: @item (for now) do a @code{make doc} and manually upload: @@ -191,7 +248,7 @@ STABLE_VERSION = what's online BUILD_DIR=$HOME/src/build-lilypond PICS=$BUILD_DIR/Documentation/pictures/out-www/ -EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/out-www/ +EXAMPLES=$BUILD_DIR/Documentation/ly-examples/out-www/ cd $BUILD_DIR rsync -a $PICS graham@@lilypond.org:media/pictures @@ -211,108 +268,193 @@ rsync -a $EXAMPLES graham@@lilypond.org:media/ly-examples A @qq{major release} means an update of @code{x} in @code{2.x.0}. -- happens when we have 0 Critical issues for two weeks (14 days). +@subheading Main requirements + +This is the current official guidelines. + +@itemize +@item +0 Critical issues for two weeks (14 days) after the latest release +candidate. +@end itemize -Before release: -* write release notes. note: stringent size requirements for - various websites, so be brief. +@subheading Potential requirements -* write preface section for manual. +These might become official guidelines in the future. -* submit pots for translation : send url of tarball to -translation@@iro.umontreal.ca, mentioning lilypond-VERSION.pot +@itemize +@item +Check reg test -* Check reg test +@item +Check all 2ly scripts -* Check all 2ly scripts. +@item +Check for emergencies the docs: -* Run convert-ly on all files, bump parser minimum version. +@example +grep FIXME --exclude "misc/*" --exclude "*GNUmakefile" \ + --exclude "snippets/*" ????*/* +@end example -* update links to distros providing lilypond packages? link in -Documentation/web/download.itexi . This has nothing to do with -the release, but I'm dumping this here so I'll find it when I -reorganize this list later. -gp +@item +Check for altered regtests, and document as necessary. (update +numbers in the following command as appropriate) -* Make FTP directories on lilypond.org +@example +git diff -u -r release/2.12.0-1 -r release/2.13.13-1 input/regression/ +@end example -* website: - - Make new table in download.html +@end itemize - - add to documentation list - - revise examples tour.html/howto.html +@subheading Housekeeping requirements - - add to front-page quick links +Before the release: - - change all links to the stable documentation +@itemize +@item +write release notes. note: stringent size requirements for +various websites, so be brief. - - doc auto redirects to v2.LATEST-STABLE +@item +Run convert-ly on all files, bump parser minimum version. - - add these two lines to http://www.lilypond.org/robots.txt: +@item +Update lilypond.pot: @example -Disallow: /doc/v2.PREVIOUS-STABLE/ -Disallow: /doc/v2.CURRENT-DEVELOPMENT/ +make -C $LILYPOND_BUILD_DIR po-replace +mv $LILYPOND_BUILD_DIR/po/lilypond.pot po/ @end example -- check for emergencies the docs: +@item +Make directories on lilypond.org: @example -grep FIXME --exclude "misc/*" --exclude "*GNUmakefile" \ - --exclude "snippets/*" ????*/* +~/web/download/sources/v2.14 +~/web/download/sources/v2.15 @end example -- check for altered regtests, and document as necessary. (update - tags as appropriate) +@item +Shortly after the release, move all current contributors to +previous contributors in: @example -git diff -u -r release/2.12.0-1 -r release/2.13.13-1 input/regression/ +Documentation/included/authors.itexi +@end example + +Also, delete old material in: + +@example +Documentation/changes.tely +@end example + +but don't forget to check it still compiles! also update the +version numbers. + +@item +Website: + +@itemize +@item +make a link from the old unstable to the next stable in +lilypond.org's /doc/ dir. Keep all previous unstable->stable doc +symlinks. + +Also, make the old docs self-contained -- if there's a redirect in +/doc/v2.12/Documentation/index.html , replace it with the +index.html.old-2.12 files. + +The post-2.13 docs will need another way of handling the +self-containment. It won't be hard to whip up a python script +that changes the link to ../../../../manuals.html to +../website/manuals.html , but it's still a 30-minute task that +needs to be done before 2.16. + +@item +doc auto redirects to v2.LATEST-STABLE + +@item +add these two lines to http://www.lilypond.org/robots.txt: + +@example +Disallow: /doc/v2.PREVIOUS-STABLE/ +Disallow: /doc/v2.CURRENT-DEVELOPMENT/ @end example +@end itemize + +@end itemize + +@subheading Unsorted + +@itemize +@item +submit po template for translation: send url of tarball to +coordinator@@translationproject.org, mentioning lilypond-VERSION.pot + +@item +update links to distros providing lilypond packages? link in: +@file{Documentation/web/download.itexi} + +This has nothing to do with the release, but it's a "periodic +maintenance" task that might make sense to include with releases. + +@item +Send announcements to... News: - comp.music.research - comp.os.linux.announce +@example +comp.music.research +comp.os.linux.announce - comp.text.tex - rec.music.compose +comp.text.tex +rec.music.compose +@end example Mail: - info-lilypond@@gnu.org +@example +info-lilypond@@gnu.org linux-audio-announce@@lists.linuxaudio.org linux-audio-user@@lists.linuxaudio.org linux-audio-dev@@lists.linuxaudio.org - tex-music@@icking-music-archive.org +tex-music@@icking-music-archive.org - --- non-existant? - abcusers@@blackmill.net +--- non-existant? +abcusers@@blackmill.net - rosegarden-user@@lists.sourceforge.net - info-gnu@@gnu.org - noteedit-user@@berlios.de +rosegarden-user@@lists.sourceforge.net +info-gnu@@gnu.org +noteedit-user@@berlios.de - gmane.comp.audio.fomus.devel - gmane.linux.audio.users - gmane.linux.audio.announce - gmane.comp.audio.rosegarden.devel +gmane.comp.audio.fomus.devel +gmane.linux.audio.users +gmane.linux.audio.announce +gmane.comp.audio.rosegarden.devel +@end example Web: - lilypond.org - freshmeat.net - linuxfr.com - http://www.apple.com/downloads - harmony-central.com (news@@harmony-central.com) - versiontracker.com [auto] - hitsquad.com [auto] - http://www.svgx.org - https://savannah.gnu.org/news/submit.php?group_id=1673 @c => planet.gnu.org +@example +lilypond.org +freshmeat.net +linuxfr.com +http://www.apple.com/downloads +harmony-central.com (news@@harmony-central.com) +versiontracker.com [auto] +hitsquad.com [auto] +http://www.svgx.org +https://savannah.gnu.org/news/submit.php?group_id=1673 @c => planet.gnu.org +@end example + +@end itemize @node Release extra notes @@ -332,7 +474,7 @@ take lilypond-book and any related makefile updates from the latest git. @item -- configure; make; make test +configure; make; make test @item tar -cjf lilypond-X.Y.Z-A.test-output.tar.bz2 input/regression/out-test/ @@ -386,7 +528,8 @@ build the specific package(s) you want, i.e. @example bin/gub mingw::lilypond-installer make LILYPOND_BRANCH=stable/2.12 -f lilypond.make doc -bin/gub --platform=darwin-x86 'git://git.sv.gnu.org/lilypond-doc.git?branch=stable/2.12' +bin/gub --platform=darwin-x86 \ + 'git://git.sv.gnu.org/lilypond-doc.git?branch=stable/2.12' @end example or @@ -405,33 +548,159 @@ delete stuff you didn't want to upload from the server. @end enumerate +@node Notes on builds with GUB +@section Notes on builds with GUB + +@subsubheading Building GUB + +GUB - the Grand Unified Builder - is used to build the release +versions of LilyPond. For background information, see +@ref{Grand Unified Builder (GUB)}. The simplest way to set up a +GUB build environment is to use a virtual machine with LilyDev +(@ref{LilyDev}). Follow the instructions on that page to set this +up. Make sure that your virtual machine has enough disk space - +a GUB installation takes over 30 GBytes of disk space, and if you +allocate too little, it will fail during the setting up stage and +you will have to start again. 64 GBytes should be sufficient. + +While GUB is being built, any interruptions are likely to make it +almost impossible to restart. If at all possible, leave the build +to continue uninterrupted. + +Download GUB and start the set up: + +@example +git clone git://github.com/gperciva/gub/gub.git +cd gub +make bootstrap +@end example + +This will take a very long time, even on a very fast computer. +You will need to be patient. It's also liable to fail - it +downloads a number of tools, and some will have moved and others +won't respond to the network. For example, the perl archive. +If this happens, download it from +@uref{http://www.cpan.org/src/5.0/perl-5.10.0.tar.gz}, saving the +archive to @file{gub/downloads/perl/}. Continue the set up with: + +@example +make bootstrap +@end example + +Once this has completed successfully, you can build the LilyPond +release package. However, this uses an archived version of the +regression tests, so it is better to download this first. +Download the test output from lilypond.org (you will need to +replace @code{2.15.33-1} with the latest build): + +@smallexample +@uref{http://lilypond.org/download/binaries/test-output/lilypond-2.15.33-1.test-output.tar.bz2} +@end smallexample + +Copy the tarball into @file{gub/regtests/}, and tell the build +system that you have done this: + +@example +touch regtests/ignore +@end example + +Now start the GUB build: -@node Administrative policies -@section Administrative policies +@example +make lilypond +@end example -Not really release-specific notes, but I don't see enough material -here to make it a separate chapter, and the release person will -probably be the one taking care of this anyway. +That's it. This will build LilyPond from current master. To build +the current unstable release, run: -@subsubheading Language-specific mailing lists +@example +make LILYPOND_BRANCH=release/unstable lilypond +@end example -A translator can ask for an official lilypond-xy mailing list once -they've finished all @qq{priority 1} translation items. +The first time you do this, it will take a very long time. -@subsubheading Performing yearly copyright update (@qq{grand-replace}) +Assuming the build has gone well, it can be uploaded using: -At the start of each year, copyright notices for all source files -should be refreshed by running the following command from the top of -the source tree: +@example +make lilypond-upload + LILYPOND_BRANCH=release/unstable + LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git +@end example + +@subsubheading Output files + +GUB builds the files it needs into the directory +@code{gub/target/}. As a general rule, these don't need to be +touched unless there is a problem building GUB (see below). +The files to be uploaded are in @code{gub/uploads/}. Once the +build has completed successfully, there should be 8 +installation files and 3 archives, totalling about 600MB. +There are also 4 directories: @example -make grand-replace +gub/signatures +gub/localdoc +gub/webdoc +gub/webtest @end example -Internally, this invokes the script @file{scripts/build/grand-replace.py}, -which performs a regular expression substitution for old-year -> new-year -wherever it finds a valid copyright notice. +@code{signatures} contains files that are used to track whether +some of the archives have already been built. Don't touch +these. + +@code{localdoc} probably contains local copies of the +documentation. + +@code{webdoc} contains the documentation to be uploaded. + +@code{webtest} contains the regtest comparison, which should +be checked before upload, and is also uploaded for subsequent +checking. + +The total upload is about 700 MB in total, and on an ADSL +connection will take about 4 hours to upload. + +@subsubheading Subsequent builds + +In principle, building the next release of LilyPond requires +no action other then following the instructions in +@ref{Minor release checklist}. Because much of the +infrastructure has already been built, it will take much less +time - about an hour on a fast computer. + +Continuing to build LilyPond without any other +archiving/deletion of previous builds is likely to be successful, +but will take up a fair amount of disk space (around 2GB per +build) which may be a problem with a Virtual Machine. It's +therefore recommended to move (not copy) @code{gub/uploads} to +another machine/disk after each build, if space is at a premium. + +However, if a significant change has been made to the LilyPond +source (e.g. added source files) the build may fail if tried on +top of a previous build. If this happens, be sure to +move/delete @code{gub/uploads} and all mentions of LilyPond +in @code{gub/target}. The latter can be achieved with this +command: + +@example +rm -rf target/*/*/*lilypond* +@end example -Note that snapshots of third party files such as @file{texinfo.tex} should -not be included in the automatic update; @file{grand-replace.py} ignores these -files if they are listed in the variable @code{copied_files}. +Be @emph{very} careful with this command. Typing it wrongly +could wipe your disk completely. + +@subsubheading Updating the web site + +The @code{make lilypond-upload} command updates the documentation +on the LilyPond web site. However, it does @emph{not} update +any part of the site that is not part of the documentation - for +example, the front page (@code{index.html}). The website is +updated by 2 cron jobs running on the web server. One of these +pulls git master to the web server, and the other makes the +website with the standard @code{make website} command. They run +hourly, 30 minutes apart. So - to update the front page of the +website, it's necessary to update @code{VERSION} and +@code{news-front.itexi} in master and then wait for the cron +jobs to run. (N.B. - this is done by pushing the changes to +staging and letting patchy do its checks before it pushes to +master).