-@c -*- coding: us-ascii; mode: texinfo; -*-
+@c -*- coding: utf-8; mode: texinfo; -*-
@node Release work
@chapter Release work
@menu
-* Development phases::
-* Minor release checklist::
-* Major release checklist::
-* Making a release::
+* Development phases::
+* Minor release checklist::
+* Major release checklist::
+* Release extra notes::
+* Notes on builds with GUB::
@end menu
@node Development phases
@section Development phases
-There are 2.5 states of development for LilyPond.
+There are 2 states of development on @code{master}:
-@itemize
+@enumerate
+
+@item @strong{Normal development}:
+Any commits are fine.
+
+@item @strong{Build-frozen}:
+Do not require any additional or updated libraries or make
+non-trivial changes to the build process. Any such patch (or
+branch) may not be merged with master during this period.
+
+This should occur approximately 1 month before any alpha version
+of the next stable release, and ends when the next unstable branch
+begins.
-@item @strong{Stable phase}:
-Starting from the release of a new major version @code{2.x.0}, the
-following patches @strong{MAY NOT} be merged with master:
+@end enumerate
+
+
+After announcing a beta release, branch @code{stable/2.x}. There
+are 2 states of development for this branch:
+
+@enumerate
+@item @strong{Normal maintenance}:
+The following patches @strong{MAY NOT} be merged with this branch:
@itemize
@item Any change to the input syntax. If a file compiled with a
-previous @code{2.x} version, then it must compile in the new
-version.
+previous @code{2.x} (beta) version, then it must compile in the
+new version.
+
+Exception: any bugfix to a Critical issue.
@item New features with new syntax @emph{may be committed},
although once committed that syntax cannot change during the
@end itemize
-@item @strong{Development phase}:
-Any commits are fine. Readers may be familiar with the term
-@qq{merge window} from following Linux kernel news.
+@item @strong{Release prep}:
+Only translation updates and important bugfixes are allowed.
+
+@end enumerate
+
+
+@node Minor release checklist
+@section Minor release checklist
+
+A @qq{minor release} means an update of @code{y} in @code{2.x.y}.
+
+@subheading Pre-release
+@enumerate
-@item @strong{Release prep phase}:
-FIXME: I don't like that name.
+@item
+Don't forget to prepare the GUB build machine by deleting and moving unneeded
+files: see @qq{Subsequent builds} in @ref{Notes on builds with GUB}.
-A new git branch @code{stable/2.x} is created, and a major release
-is made in two weeks.
+@item
+Using any system with git pull access (not necessarily the GUB
+build machine), use the commands below to do the following:
@itemize
+@item
+switch to the release branch
+
+@item
+update the release branch from origin/master
+
+@item
+update the translation files
-@item @code{stable/2.x branch}:
-Only translation updates and important bugfixes are allows.
+@item
+create the release announcement
+
+@item
+update the build versions.
+
+@itemize
+@item
+VERSION_DEVEL = the current development version (previous VERSION_DEVEL + 0.01)
-@item @code{master}:
-Normal @qq{stable phase} development occurs.
+@item
+VERSION_STABLE = the current stable version (probably no change here)
@end itemize
-If we discover the need to change the syntax or build system, we
-will apply it and re-start the release prep phase.
+@item
+update the @qq{Welcome to LilyPond} version numbers to the version about to be
+released
@end itemize
-This marks a radical change from previous practice in LilyPond.
-However, this setup is not intended to slow development -- as a
-rule of thumb, the next development phase will start within a
-month of somebody wanting to commit something which is not
-permitted during the stable phase.
+This requires a system which has the release/unstable
+branch. If you get a warning saying you are in @code{detached HEAD}
+state, then you should create a release/unstable branch with
+@code{git checkout release/unstable}.
+Check the environment variables are set as in
+@ref{Environment variables}.
+You need to ensure you have a clean index and work tree. If the
+checkout displays modified files, you might want to run
+@code{git reset --hard} before continuing.
-@node Minor release checklist
-@section Minor release checklist
+@example
+git fetch
+git checkout release/unstable
+git merge origin/master
+make -C $LILYPOND_BUILD_DIR po-replace
+mv $LILYPOND_BUILD_DIR/po/lilypond.pot po/
+gedit Documentation/web/news-front.itexi Documentation/web/news.itexi
+gedit VERSION
+gedit ly/Wel*.ly
+@end example
-A @qq{minor release} means an update of @code{y} in @code{2.x.y}.
+@item
+Commit, push, switch back to master (or wherever else):
+
+@example
+git commit -m "Release: bump VERSION_DEVEL." VERSION
+git commit -m "PO: update template." po/lilypond.pot
+git commit -m "Release: update news." Documentation/web/
+git commit -m "Release: bump Welcome versions." ly/Wel*.ly
+git push origin HEAD:release/unstable
+git checkout master
+@end example
+
+@item
+If you do not have the previous release test-output tarball, download
+it and put it in @code{regtests/}
+
+@item Prepare GUB environment by running:
+
+@example
+### my-gub.sh
+# special terminal, and default PATH environment.
+# import these special environment vars:
+# HOME, HTTP_PROXY, TERM
+env -i \
+ HOME=$HOME \
+ HTTP_PROXY=$HTTP_PROXY \
+ bash --rcfile my-bashrc
+@end example
+
+@example
+### my-bashrc
+export PS1="\[\e[1;33mGUB-ENV \w\]$ \[\e[0m\]"
+export PATH=$PATH
+export TERM=xterm
+@end example
+
+
+@item Build release on GUB by running:
+
+@example
+make LILYPOND_BRANCH=release/unstable lilypond
+@end example
+
+@noindent
+or something like:
+
+@example
+make LILYPOND_BRANCH=stable/2.16 lilypond
+@end example
+
+@item
+Check the regtest comparison in @file{uploads/webtest/} for
+any unintentional breakage. More info in
+@ref{Precompiled regression tests}.
+
+@item
+If any work was done on GUB since the last release, upload
+binaries to a temporary location, ask for feedback, and wait a day
+or two in case there's any major problems.
+
+@warning{Always do this for a stable release.}
-email brief summary to info-lilypond
+@end enumerate
+
+
+@subheading Actual release
+
+@enumerate
+
+@item If you're not the right user on the webserver, remove the
+@code{t} from the rsync command in:
+
+@example
+test-lily/rsync-lily-doc.py
+test-lily/rsync-test.py
+@end example
+
+@item Upload GUB by running:
+
+@example
+make lilypond-upload \
+ LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git \
+ LILYPOND_BRANCH=release/unstable
+@end example
+
+@noindent
+or something like:
+
+@example
+make lilypond-upload \
+ LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git \
+ LILYPOND_BRANCH=stable/2.12
+@end example
+
+@end enumerate
+
+
+@subheading Post release
+
+@enumerate
+
+@item
+Update the current staging branch with the current news:
+
+@example
+git fetch
+git checkout origin/staging
+git merge origin/release/unstable
+@end example
+
+@item
+Update @file{VERSION} in lilypond git and upload changes:
+
+@example
+gedit VERSION
+@end example
+
+@itemize
+@item
+VERSION = what you just did +0.0.1
+@end itemize
+
+@example
+git commit -m "Release: bump VERSION." VERSION
+git push origin HEAD:staging
+@end example
+
+If the push fails with a message like
+
+@example
+ ! [rejected] HEAD -> staging (non-fast-forward)
+@end example
+
+@noindent
+it means that somebody else updated the staging branch while you were
+preparing your change. In that case, you need to restart the Post
+Release process. Otherwise, proceed:
+
+@item Wait a few hours for the website to update.
+
+@item Email release notice to @code{info-lilypond}
+
+@end enumerate
A @qq{major release} means an update of @code{x} in @code{2.x.0}.
-Before release:
+@subheading Main requirements
-* write release notes. note: stringent size requirements for
- various websites, so be brief.
+These are the current official guidelines.
-* write preface section for manual.
+@itemize
+@item
+0 Critical issues for two weeks (14 days) after the latest release
+candidate.
-* submit pots for translation : send url of tarball to
-translation@@iro.umontreal.ca, mentioning lilypond-VERSION.pot
+@end itemize
-* Check reg test
-* Check all 2ly scripts.
+@subheading Potential requirements
-* Run convert-ly on all files, bump parser minimum version.
+These might become official guidelines in the future.
-* Make FTP directories on lilypond.org
+@itemize
+@item
+Check reg test
-* website:
- - Make new table in download.html
+@item
+Check all 2ly scripts
- - add to documentation list
+@item
+Check for emergencies the docs:
- - revise examples tour.html/howto.html
+@example
+grep FIXME --exclude "misc/*" --exclude "*GNUmakefile" \
+ --exclude "snippets/*" ????*/*
+@end example
- - add to front-page quick links
+@item
+Check for altered regtests, and document as necessary:
- - change all links to the stable documentation
+@example
+git diff -u -r release/2.@var{FIRST-CURRENT-STABLE} \
+ -r release/2.@var{LAST-CURRENT-DEVELOPMENT} input/regression/
+@end example
- - doc auto redirects to v2.LATEST-STABLE
+@end itemize
+
+
+@subheading Housekeeping requirements
+
+Before the release:
+
+@itemize
+@item
+write release notes. note: stringent size requirements for
+various websites, so be brief.
+
+@item
+Run convert-ly on all files, bump parser minimum version.
+
+@item
+Update lilypond.pot:
+
+@example
+make -C $LILYPOND_BUILD_DIR po-replace
+mv $LILYPOND_BUILD_DIR/po/lilypond.pot po/
+@end example
+
+@item
+Make directories on lilypond.org:
+
+@example
+~/download/sources/v2.@var{NEW-STABLE}
+~/download/sources/v2.@var{NEW-DEVELOPMENT}
+@end example
+
+@item
+Shortly after the release, move all current contributors to
+previous contributors in
+@file{Documentation/included/authors.itexi}.
+
+@item
+Delete old material in @file{Documentation/changes.tely}, but
+don't forget to check it still compiles! Also update the version
+numbers:
+
+@example
+@@node Top
+@@top New features in 2.@var{NEW-STABLE} since 2.@var{OLD-STABLE}
+@end example
+
+@item
+Website:
+
+@itemize
+@item
+make a link from the old unstable to the next stable in
+lilypond.org's @file{/doc/} dir. Keep all previous unstable->stable
+doc symlinks.
+
+Also, make the old docs self-contained -- if there's a redirect in
+@file{/doc/v2.@var{OLD-STABLE}/Documentation/index.html} , replace it with the
+@file{index.html.old-2.@var{OLD-STABLE}} files.
+
+The post-2.13 docs will need another way of handling the
+self-containment. It won't be hard to whip up a python script
+that changes the link to @file{../../../../manuals.html} to
+@file{../website/manuals.html}, but it's still a 30-minute task that
+needs to be done before 2.16.
+
+@item
+doc auto redirects to @code{v2.@var{NEW-STABLE}}
+
+@item
+add these two lines to @file{Documentation/web/server/robots.txt}:
+
+@example
+Disallow: /doc/v2.@var{OLD-STABLE}/
+Disallow: /doc/v2.@var{NEW-DEVELOPMENT}/
+@end example
+
+@end itemize
+
+@end itemize
+
+@subheading Unsorted
+
+@itemize
+@item
+submit po template for translation: send url of tarball to
+@email{coordinator@@translationproject.org}, mentioning
+lilypond-VERSION.pot
+
+@item
+update links to distros providing lilypond packages? link in:
+@file{Documentation/web/download.itexi}
+
+This has nothing to do with the release, but it's a @qq{periodic
+maintenance} task that might make sense to include with releases.
+
+@item
+Send announcements to...
News:
- comp.music.research
- comp.os.linux.announce
+@example
+comp.music.research
+comp.os.linux.announce
- comp.text.tex
- rec.music.compose
+comp.text.tex
+rec.music.compose
+@end example
Mail:
- info-lilypond@@gnu.org
+@example
+info-lilypond@@gnu.org
linux-audio-announce@@lists.linuxaudio.org
linux-audio-user@@lists.linuxaudio.org
linux-audio-dev@@lists.linuxaudio.org
- tex-music@@icking-music-archive.org
+tex-music@@icking-music-archive.org
- --- non-existant?
- abcusers@@blackmill.net
+--- non-existant?
+abcusers@@blackmill.net
- rosegarden-user@@lists.sourceforge.net
- info-gnu@@gnu.org
- noteedit-user@@berlios.de
+rosegarden-user@@lists.sourceforge.net
+info-gnu@@gnu.org
+noteedit-user@@berlios.de
- gmane.comp.audio.fomus.devel
- gmane.linux.audio.users
- gmane.linux.audio.announce
- gmane.comp.audio.rosegarden.devel
+gmane.comp.audio.fomus.devel
+gmane.linux.audio.users
+gmane.linux.audio.announce
+gmane.comp.audio.rosegarden.devel
+@end example
Web:
- lilypond.org
- freshmeat.net
- linuxfr.com
- http://www.apple.com/downloads
- harmony-central.com (news@@harmony-central.com)
- versiontracker.com [auto]
- hitsquad.com [auto]
- http://www.svgx.org
+@example
+lilypond.org
+freshmeat.net
+linuxfr.com
+http://www.apple.com/downloads
+harmony-central.com (news@@harmony-central.com)
+versiontracker.com [auto]
+hitsquad.com [auto]
+http://www.svgx.org
+https://savannah.gnu.org/news/submit.php?group_id=1673 @c => planet.gnu.org
+@end example
+@end itemize
-@node Making a release
-@section Making a release
+@node Release extra notes
+@section Release extra notes
-@subheading Technical notes
+@subsubheading Regenerating regression tests
-To build GUB:
+Regenerating regtests (if the lilypond-book naming has changed):
@itemize
-@item
-Run the following (from the gub/ dir):
+@item
+git checkout release/lilypond-X.Y.Z-A
-@example
-make -f lilypond.make update-versions
-@end example
+@item
+take lilypond-book and any related makefile updates from the
+latest git.
@item
-Download
-@uref{http://lilypond.org/download/binaries/test-output/lilypond-2.13.0-0.test-output.tar.bz2,lilypond-2.13.0-0.test-output.tar.bz2}
+configure; make; make test
-@item
-Copy / move / link it to uploads, but rename it to
-@file{lilypond-2-13.test-output.tar.bz2}
+@item
+tar -cjf lilypond-X.Y.Z-A.test-output.tar.bz2 input/regression/out-test/
@item
-Run:
+mv lilypond-X.Y.Z-A.test-output.tar.bz2 ../gub/regtests/
-@example
+@item
+cd ../gub/regtests/
+
+@item
make lilypond
-@end example
@end itemize
-To upload:
-@itemize
+@subsubheading stable/2.12
-@item remove the "t" from the rsync command in test-lily/rsync-lily-doc.py
+If releasing stable/2.12, then:
-@item run:
+@itemize
-@example
-python test-lily/upload.py --branch=master --url git://git.sv.gnu.org/lilypond.git --execute
-@end example
+@item
+apply doc patch: patches/rsync-lily.patch (or something like
+that)
+@item
+change infodir in gub/specs/lilypond-doc.py from "lilypond.info"
+to "lilypond-web.info"
@end itemize
+@subsubheading Updating a release (changing a in x.y.z-a)
-@subheading Policy notes
+Really tentative instructions, almost certainly can be done
+better.
-@itemize
+@enumerate
@item
-Build with GUB, and check the regtests.
+change the VERSION back to release you want. push change.
+(hopefully you'll have forgotten to update it when you made your
+last release)
@item
-Upload the tarballs and sh scripts.
+make sure that there aren't any lilypond files floating around in
+target/ (like usr/bin/lilypond).
@item
-(if major)
-Branch MASTER to stable/2.x.
+build the specific package(s) you want, i.e.
+
+@example
+bin/gub mingw::lilypond-installer
+make LILYPOND_BRANCH=stable/2.12 -f lilypond.make doc
+bin/gub --platform=darwin-x86 \
+ 'git://git.sv.gnu.org/lilypond-doc.git?branch=stable/2.12'
+@end example
+
+or
+
+build everything with the normal "make lilypond", then (maybe)
+manually delete stuff you don't want to upload.
@item
-Make announcement.
+manually upload them. good luck figuring out the rsync
+command(s). Hints are in test-lily/
-@end itemize
+or
+
+run the normal lilypond-upload command, and (maybe) manually
+delete stuff you didn't want to upload from the server.
+
+@end enumerate
+
+@node Notes on builds with GUB
+@section Notes on builds with GUB
+@subsubheading Building GUB
+
+GUB - the Grand Unified Builder - is used to build the release
+versions of LilyPond. For background information, see
+@ref{Grand Unified Builder (GUB)}. The simplest way to set up a
+GUB build environment is to use a virtual machine with LilyDev
+(@ref{LilyDev}). Follow the instructions on that page to set this
+up. Make sure that your virtual machine has enough disk space -
+a GUB installation takes over 30 GBytes of disk space, and if you
+allocate too little, it will fail during the setting up stage and
+you will have to start again. 64 GBytes should be sufficient.
+
+While GUB is being built, any interruptions are likely to make it
+almost impossible to restart. If at all possible, leave the build
+to continue uninterrupted.
+
+Download GUB and start the set up:
+
+@example
+git clone git://github.com/gperciva/gub.git
+cd gub
+make bootstrap
+@end example
+
+This will take a very long time, even on a very fast computer.
+You will need to be patient. It's also liable to fail - it
+downloads a number of tools, and some will have moved and others
+won't respond to the network. For example, the perl archive.
+If this happens, download it from
+@uref{http://www.cpan.org/src/5.0/perl-5.10.0.tar.gz}, saving the
+archive to @file{gub/downloads/perl/}. Continue the set up with:
+
+@example
+make bootstrap
+@end example
+
+Once this has completed successfully, you can build the LilyPond
+release package. However, this uses an archived version of the
+regression tests, so it is better to download this first.
+Download the test output from lilypond.org (you will need to
+replace @code{2.15.33-1} with the latest build):
+
+@smallexample
+@uref{http://lilypond.org/downloads/binaries/test-output/lilypond-2.15.33-1.test-output.tar.bz2}
+@end smallexample
+
+Copy the tarball into @file{gub/regtests/}, and tell the build
+system that you have done this:
+
+@example
+touch regtests/ignore
+@end example
+
+Now start the GUB build:
+
+@example
+make lilypond
+@end example
+
+That's it. This will build LilyPond from current master. To build
+the current unstable release, run:
+
+@example
+make LILYPOND_BRANCH=release/unstable lilypond
+@end example
+
+The first time you do this, it will take a very long time.
+
+Assuming the build has gone well, it can be uploaded using:
+
+@example
+make lilypond-upload
+ LILYPOND_BRANCH=release/unstable
+ LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git
+@end example
+
+@subsubheading Output files
+
+GUB builds the files it needs into the directory
+@code{gub/target/}. As a general rule, these don't need to be
+touched unless there is a problem building GUB (see below).
+The files to be uploaded are in @code{gub/uploads/}. Once the
+build has completed successfully, there should be 8
+installation files and 3 archives, totalling about 600MB.
+There are also 4 directories:
+
+@example
+gub/signatures
+gub/localdoc
+gub/webdoc
+gub/webtest
+@end example
+
+@code{signatures} contains files that are used to track whether
+some of the archives have already been built. Don't touch
+these.
+
+@code{localdoc} probably contains local copies of the
+documentation.
+
+@code{webdoc} contains the documentation to be uploaded.
+
+@code{webtest} contains the regtest comparison, which should
+be checked before upload, and is also uploaded for subsequent
+checking.
+
+The total upload is about 700 MB in total, and on an ADSL
+connection will take about 4 hours to upload.
+
+@subsubheading Subsequent builds
+
+In principle, building the next release of LilyPond requires
+no action other then following the instructions in
+@ref{Minor release checklist}. Because much of the
+infrastructure has already been built, it will take much less
+time - about an hour on a fast computer.
+
+Continuing to build LilyPond without any other
+archiving/deletion of previous builds is likely to be successful,
+but will take up a fair amount of disk space (around 2GB per
+build) which may be a problem with a Virtual Machine. It's
+therefore recommended to move (not copy) @code{gub/uploads} to
+another machine/disk after each build, if space is at a premium.
+
+However, if a significant change has been made to the LilyPond
+source (e.g. added source files) the build may fail if tried on
+top of a previous build. If this happens, be sure to
+move/delete @code{gub/uploads} and all mentions of LilyPond
+in @code{gub/target}. The latter can be achieved with this
+command:
+
+@example
+rm -rf target/*/*/*lilypond*
+@end example
+Be @emph{very} careful with this command. Typing it wrongly
+could wipe your disk completely.
+
+@subsubheading Updating the web site
+
+The @code{make lilypond-upload} command updates the documentation
+on the LilyPond web site. However, it does @emph{not} update
+any part of the site that is not part of the documentation - for
+example, the front page (@code{index.html}). The website is
+updated by 2 cron jobs running on the web server. One of these
+pulls git master to the web server, and the other makes the
+website with the standard @code{make website} command. They run
+hourly, 30 minutes apart. So - to update the front page of the
+website, it's necessary to update @code{VERSION} and
+@code{news-front.itexi} in master and then wait for the cron
+jobs to run. (N.B. - this is done by pushing the changes to
+staging and letting patchy do its checks before it pushes to
+master).