@chapter Release work
@menu
-* Development phases::
-* Minor release checklist::
-* Major release checklist::
-* Release extra notes::
+* Development phases::
+* Minor release checklist::
+* Major release checklist::
+* Release extra notes::
+* Notes on builds with GUB::
@end menu
@node Development phases
@section Development phases
-There are 2.5 states of development for LilyPond.
+There are 2 states of development on @code{master}:
+
+@enumerate
+
+@item @strong{Normal development}:
+Any commits are fine.
+
+@item @strong{Build-frozen}:
+Do not require any additional or updated libraries or make
+non-trivial changes to the build process. Any such patch (or
+branch) may not be merged with master during this period.
+
+This should occur approximately 1 month before any alpha version
+of the next stable release, and ends when the next unstable branch
+begins.
+
+@end enumerate
-@itemize
-@item @strong{Stable phase}:
-Starting from the release of a new major version @code{2.x.0}, the
-following patches @strong{MAY NOT} be merged with master:
+After announcing a beta release, branch @code{stable/2.x}. There
+are 2 states of development for this branch:
+
+@enumerate
+@item @strong{Normal maintenance}:
+The following patches @strong{MAY NOT} be merged with this branch:
@itemize
@item Any change to the input syntax. If a file compiled with a
-previous @code{2.x} version, then it must compile in the new
-version.
+previous @code{2.x} (beta) version, then it must compile in the
+new version.
+
+Exception: any bugfix to a Critical issue.
@item New features with new syntax @emph{may be committed},
although once committed that syntax cannot change during the
@end itemize
-@item @strong{Development phase}:
-Any commits are fine. Readers may be familiar with the term
-@qq{merge window} from following Linux kernel news.
-
-
-@item @strong{Release prep phase}:
-FIXME: I don't like that name.
-
-A new git branch @code{stable/2.x} is created, and a major release
-is made in two weeks.
-
-@itemize
-
-@item @code{stable/2.x branch}:
-Only translation updates and important bugfixes are allows.
-
-@item @code{master}:
-Normal @qq{stable phase} development occurs.
-
-@end itemize
-
-If we discover the need to change the syntax or build system, we
-will apply it and re-start the release prep phase.
-
-@end itemize
-
-This marks a radical change from previous practice in LilyPond.
-However, this setup is not intended to slow development -- as a
-rule of thumb, the next development phase will start within a
-month of somebody wanting to commit something which is not
-permitted during the stable phase.
+@item @strong{Release prep}:
+Only translation updates and important bugfixes are allowed.
+@end enumerate
@node Minor release checklist
@enumerate
-@item Add a news item to @file{Documentation/web/news-front.itexi}
+@item
+Using any system with git pull access (not necessarily the GUB
+build machine), use the commands below to switch to the release
+branch, get changes and prepare the release
+announcement. This requires a system which has the release/unstable
+branch. If you get a warning saying you are in @code{detached HEAD}
+state, then you should create a release/unstable branch with
+@code{git checkout release/unstable}.
+
+Check the environment variables are set as in
+@ref{Environment variables}.
-@item Check that lilypond builds from scratch in an out-of-tree
-build.
+You need to ensure you have a clean index and work tree. If the
+checkout displays modified files, you might want to run
+@code{git reset --hard} before continuing.
+
+@example
+git fetch
+git checkout release/unstable
+git merge origin/master
+make -C $LILYPOND_BUILD_DIR po-replace
+mv $LILYPOND_BUILD_DIR/po/lilypond.pot po/
+gedit Documentation/web/news-front.itexi Documentation/web/news.itexi
+@end example
+
+@item
+Commit, push, switch back to master (or wherever else):
+
+@example
+git commit -m "PO: update template." po/lilypond.pot
+git commit -m "Release: update news." Documentation/web/
+git push origin HEAD:release/unstable
+git checkout master
+@end example
@item
If you do not have the previous release test-output tarball, download
it and put it in @code{regtests/}
+@item Prepare GUB environment by running:
+
+@example
+### my-gub.sh
+# special terminal, and default PATH environment.
+# import these special environment vars:
+# HOME, HTTP_PROXY, TERM
+env -i \
+ HOME=$HOME \
+ HTTP_PROXY=$HTTP_PROXY \
+ bash --rcfile my-bashrc
+@end example
+
+@example
+### my-bashrc
+export PS1="\[\e[1;33mGUB-ENV \w\]$ \[\e[0m\]"
+export PATH=$PATH
+export TERM=xterm
+@end example
+
+
@item Build release on GUB by running:
@example
-make lilypond
+make LILYPOND_BRANCH=release/unstable lilypond
@end example
@noindent
make LILYPOND_BRANCH=stable/2.12 lilypond
@end example
-
@item Check the regtest comparison in @file{uploads/webtest/} for
-any unintentional breakage.
-
-@item Check if the mingw build contains lilypad.exe; when you find
-that it doesn't, rebuild @code{mingw::lilypond-installer}. Repeat
-as necessary.
-
-@uref{http://code.google.com/p/lilypond/issues/detail?id=901}
+any unintentional breakage. More info in
+@ref{Precompiled regression tests}.
@item If any work was done on GUB since the last release, upload
binaries to a temporary location, ask for feedback, and wait a day
or two in case there's any major problems.
+@warning{Always do this for a stable release.}
+
@end enumerate
-@subheading Post-release
+@subheading Actual release
@enumerate
-@item If you're not right user on the webserver, remove the "t"
-from the rsync command in @file{test-lily/rsync-lily-doc.py} and
-@file{test-lily/rsync-test.py}
+@item If you're not the right user on the webserver, remove the
+@code{t} from the rsync command in:
+
+@example
+test-lily/rsync-lily-doc.py
+test-lily/rsync-test.py
+@end example
@code{graham} owns v2.13; @code{han-wen} owns v2.12.
@item Upload GUB by running:
@example
-make lilypond-upload LILYPOND_BRANCH=master LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git
+make lilypond-upload \
+ LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git \
+ LILYPOND_BRANCH=release/unstable
@end example
@noindent
or something like:
@example
-make lilypond-upload LILYPOND_BRANCH=stable/2.12 LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git
+make lilypond-upload \
+ LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git \
+ LILYPOND_BRANCH=stable/2.12
+@end example
+
+@end enumerate
+
+
+@subheading Post release
+
+@enumerate
+
+@item Update the current staging branch with the current news:
+
+@example
+git fetch
+git checkout origin/staging
+git merge origin/release/unstable
@end example
+@item Update @file{VERSION} in lilypond git and upload changes:
+
+@example
+vi VERSION
+@end example
+
+@itemize
+@item
+VERSION = what you just did +0.0.1
+
+@item
+DEVEL_VERSION = what you just did (i.e. is now online)
+
+@item
+STABLE_VERSION = what's online (probably no change here)
+
+@end itemize
+
+@example
+git commit -m "Release: bump version." VERSION
+git push origin HEAD:staging
+@end example
+
+If the push fails with a message like
+
+@example
+ ! [rejected] HEAD -> staging (non-fast-forward)
+@end example
+
+@noindent
+it means that somebody else updated the staging branch while you were
+preparing your change. In that case, you need to restart the Post
+Release process. Otherwise, proceed:
+
+@item (for now) do a @code{make doc} and manually upload:
+
+@example
+### upload-lily-web-media.sh
+#!/bin/sh
+BUILD_DIR=$HOME/src/build-lilypond
-@item Update @file{VERSION} in lilypond git.
+PICS=$BUILD_DIR/Documentation/pictures/out-www/
+EXAMPLES=$BUILD_DIR/Documentation/ly-examples/out-www/
+
+cd $BUILD_DIR
+rsync -a $PICS graham@@lilypond.org:media/pictures
+rsync -a $EXAMPLES graham@@lilypond.org:media/ly-examples
+@end example
@item Wait a few hours for the website to update.
@end enumerate
+
@node Major release checklist
@section Major release checklist
A @qq{major release} means an update of @code{x} in @code{2.x.0}.
-Before release:
+@subheading Main requirements
+
+This is the current official guidelines.
+
+@itemize
+@item
+0 Critical issues for two weeks (14 days) after the latest release
+candidate.
+
+@end itemize
-* write release notes. note: stringent size requirements for
- various websites, so be brief.
-* write preface section for manual.
+@subheading Potential requirements
-* submit pots for translation : send url of tarball to
-translation@@iro.umontreal.ca, mentioning lilypond-VERSION.pot
+These might become official guidelines in the future.
-* Check reg test
+@itemize
+@item
+Check reg test
-* Check all 2ly scripts.
+@item
+Check all 2ly scripts
+
+@item
+Check for emergencies the docs:
-* Run convert-ly on all files, bump parser minimum version.
+@example
+grep FIXME --exclude "misc/*" --exclude "*GNUmakefile" \
+ --exclude "snippets/*" ????*/*
+@end example
-* update links to distros providing lilypond packages? link in
-Documentation/web/download.itexi . This has nothing to do with
-the release, but I'm dumping this here so I'll find it when I
-reorganize this list later. -gp
+@item
+Check for altered regtests, and document as necessary. (update
+numbers in the following command as appropriate)
-* Make FTP directories on lilypond.org
+@example
+git diff -u -r release/2.12.0-1 -r release/2.13.13-1 input/regression/
+@end example
-* website:
- - Make new table in download.html
+@end itemize
- - add to documentation list
- - revise examples tour.html/howto.html
+@subheading Housekeeping requirements
- - add to front-page quick links
+Before the release:
- - change all links to the stable documentation
+@itemize
+@item
+write release notes. note: stringent size requirements for
+various websites, so be brief.
- - doc auto redirects to v2.LATEST-STABLE
+@item
+Run convert-ly on all files, bump parser minimum version.
- - add these two lines to http://www.lilypond.org/robots.txt:
+@item
+Update lilypond.pot:
+
+@example
+make -C $LILYPOND_BUILD_DIR po-replace
+mv $LILYPOND_BUILD_DIR/po/lilypond.pot po/
+@end example
+
+@item
+Make directories on lilypond.org:
+
+@example
+~/web/download/sources/v2.14
+~/web/download/sources/v2.15
+@end example
+
+@item
+Shortly after the release, move all current contributors to
+previous contributors in:
+
+@example
+Documentation/included/authors.itexi
+@end example
+
+Also, delete old material in:
+
+@example
+Documentation/changes.tely
+@end example
+
+but don't forget to check it still compiles! also update the
+version numbers.
+
+@item
+Website:
+
+@itemize
+@item
+make a link from the old unstable to the next stable in
+lilypond.org's /doc/ dir. Keep all previous unstable->stable doc
+symlinks.
+
+Also, make the old docs self-contained -- if there's a redirect in
+/doc/v2.12/Documentation/index.html , replace it with the
+index.html.old-2.12 files.
+
+The post-2.13 docs will need another way of handling the
+self-containment. It won't be hard to whip up a python script
+that changes the link to ../../../../manuals.html to
+../website/manuals.html , but it's still a 30-minute task that
+needs to be done before 2.16.
+
+@item
+doc auto redirects to v2.LATEST-STABLE
+
+@item
+add these two lines to http://www.lilypond.org/robots.txt:
@example
Disallow: /doc/v2.PREVIOUS-STABLE/
Disallow: /doc/v2.CURRENT-DEVELOPMENT/
@end example
+@end itemize
+
+@end itemize
+
+@subheading Unsorted
+
+@itemize
+@item
+submit po template for translation: send url of tarball to
+coordinator@@translationproject.org, mentioning lilypond-VERSION.pot
+
+@item
+update links to distros providing lilypond packages? link in:
+@file{Documentation/web/download.itexi}
+
+This has nothing to do with the release, but it's a "periodic
+maintenance" task that might make sense to include with releases.
+
+@item
+Send announcements to...
+
News:
- comp.music.research
- comp.os.linux.announce
+@example
+comp.music.research
+comp.os.linux.announce
- comp.text.tex
- rec.music.compose
+comp.text.tex
+rec.music.compose
+@end example
Mail:
- info-lilypond@@gnu.org
+@example
+info-lilypond@@gnu.org
linux-audio-announce@@lists.linuxaudio.org
linux-audio-user@@lists.linuxaudio.org
linux-audio-dev@@lists.linuxaudio.org
- tex-music@@icking-music-archive.org
+tex-music@@icking-music-archive.org
- --- non-existant?
- abcusers@@blackmill.net
+--- non-existant?
+abcusers@@blackmill.net
- rosegarden-user@@lists.sourceforge.net
- info-gnu@@gnu.org
- noteedit-user@@berlios.de
+rosegarden-user@@lists.sourceforge.net
+info-gnu@@gnu.org
+noteedit-user@@berlios.de
- gmane.comp.audio.fomus.devel
- gmane.linux.audio.users
- gmane.linux.audio.announce
- gmane.comp.audio.rosegarden.devel
+gmane.comp.audio.fomus.devel
+gmane.linux.audio.users
+gmane.linux.audio.announce
+gmane.comp.audio.rosegarden.devel
+@end example
Web:
- lilypond.org
- freshmeat.net
- linuxfr.com
- http://www.apple.com/downloads
- harmony-central.com (news@@harmony-central.com)
- versiontracker.com [auto]
- hitsquad.com [auto]
- http://www.svgx.org
- https://savannah.gnu.org/news/submit.php?group_id=1673 @c => planet.gnu.org
+@example
+lilypond.org
+freshmeat.net
+linuxfr.com
+http://www.apple.com/downloads
+harmony-central.com (news@@harmony-central.com)
+versiontracker.com [auto]
+hitsquad.com [auto]
+http://www.svgx.org
+https://savannah.gnu.org/news/submit.php?group_id=1673 @c => planet.gnu.org
+@end example
+
+@end itemize
@node Release extra notes
@section Release extra notes
+@subsubheading Regenerating regression tests
+
+Regenerating regtests (if the lilypond-book naming has changed):
+
+@itemize
+
+@item
+git checkout release/lilypond-X.Y.Z-A
+
+@item
+take lilypond-book and any related makefile updates from the
+latest git.
+
+@item
+configure; make; make test
+
+@item
+tar -cjf lilypond-X.Y.Z-A.test-output.tar.bz2 input/regression/out-test/
+
+@item
+mv lilypond-X.Y.Z-A.test-output.tar.bz2 ../gub/regtests/
+
+@item
+cd ../gub/regtests/
+
+@item
+make lilypond
+
+@end itemize
+
+
+@subsubheading stable/2.12
+
If releasing stable/2.12, then:
-- apply doc patch: patches/rsync-lily.patch (or something like
- that)
+@itemize
+
+@item
+apply doc patch: patches/rsync-lily.patch (or something like
+that)
-- change infodir in gub/specs/lilypond-doc.py from "lilypond.info"
- to "lilypond-web.info"
+@item
+change infodir in gub/specs/lilypond-doc.py from "lilypond.info"
+to "lilypond-web.info"
+@end itemize
+@subsubheading Updating a release (changing a in x.y.z-a)
-GENERAL STUFF TO BE MOVED ELSEWHERE
+Really tentative instructions, almost certainly can be done
+better.
-Regenerating regtests (if the lilypond-book naming has changed):
+@enumerate
+
+@item
+change the VERSION back to release you want. push change.
+(hopefully you'll have forgotten to update it when you made your
+last release)
-- git checkout release/lilypond-X.Y.Z-A
+@item
+make sure that there aren't any lilypond files floating around in
+target/ (like usr/bin/lilypond).
-- take lilypond-book and any related makefile updates from the
- latest git.
+@item
+build the specific package(s) you want, i.e.
+
+@example
+bin/gub mingw::lilypond-installer
+make LILYPOND_BRANCH=stable/2.12 -f lilypond.make doc
+bin/gub --platform=darwin-x86 \
+ 'git://git.sv.gnu.org/lilypond-doc.git?branch=stable/2.12'
+@end example
-- configure; make; make test
+or
-- tar -cjf lilypond-X.Y.Z-A.test-output.tar.bz2
- input/regression/out-test/
+build everything with the normal "make lilypond", then (maybe)
+manually delete stuff you don't want to upload.
-- mv lilypond-X.Y.Z-A.test-output.tar.bz2 ../gub/regtests/
+@item
+manually upload them. good luck figuring out the rsync
+command(s). Hints are in test-lily/
-- cd ../gub/regtests/
+or
-- make lilypond
+run the normal lilypond-upload command, and (maybe) manually
+delete stuff you didn't want to upload from the server.
+@end enumerate
+@node Notes on builds with GUB
+@section Notes on builds with GUB
-VERSION-SPECIFIC MACROS
+@subsubheading Building GUB
-- made with scripts/build/create-version-itexi.py
+GUB - the Grand Unified Builder - is used to build the release
+versions of LilyPond. For background information, see
+@ref{Grand Unified Builder (GUB)}. The simplest way to set up a
+GUB build environment is to use a virtual machine with LilyDev
+(@ref{LilyDev}). Follow the instructions on that page to set this
+up. Make sure that your virtual machine has enough disk space -
+a GUB installation takes over 30 GBytes of disk space, and if you
+allocate too little, it will fail during the setting up stage and
+you will have to start again. 64 GBytes should be sufficient.
-- used extensively in the WEBSITE_ONLY_BUILD version of the
- website (made with website.make, and used on lilypond.org)
+While GUB is being built, any interruptions are likely to make it
+almost impossible to restart. If at all possible, leave the build
+to continue uninterrupted.
-- not (?) used in the main docs?
+Download GUB and start the set up:
-- FIXME: the numbers in VERSION: MINOR should be 1 more than the
- last release, VERSION_DEVEL should be the last *online* release.
+@example
+git clone git://github.com/gperciva/gub/gub.git
+cd gub
+make bootstrap
+@end example
+This will take a very long time, even on a very fast computer.
+You will need to be patient. It's also liable to fail - it
+downloads a number of tools, and some will have moved and others
+won't respond to the network. For example, the perl archive.
+If this happens, download it from
+@uref{http://www.cpan.org/src/5.0/perl-5.10.0.tar.gz}, saving the
+archive to @file{gub/downloads/perl/}. Continue the set up with:
-LANGUAGE LISTS
+@example
+make bootstrap
+@end example
-- a translator can ask for an official lilypond-xy mailing list
- once they've finished all "priority 1" translation items.
+Once this has completed successfully, you can build the LilyPond
+release package. However, this uses an archived version of the
+regression tests, so it is better to download this first.
+Download the test output from lilypond.org (you will need to
+replace @code{2.15.33-1} with the latest build):
+@smallexample
+@uref{http://lilypond.org/download/binaries/test-output/lilypond-2.15.33-1.test-output.tar.bz2}
+@end smallexample
-UPDATING A RELEASE ( x.y.z-a, where a>1 )
+Copy the tarball into @file{gub/regtests/}, and tell the build
+system that you have done this:
-really tentative instructions, almost certainly can be done better
+@example
+touch regtests/ignore
+@end example
-1. change the VERSION back to release you want. push change.
-(hopefully you'll have forgotten to update it when you made your
-last release)
+Now start the GUB build:
-2. make sure that there aren't any lilypond files floating around
-in target/ (like usr/bin/lilypond).
+@example
+make lilypond
+@end example
-3. build the specific package(s) you want, i.e.
+That's it. This will build LilyPond from current master. To build
+the current unstable release, run:
-bin/gub mingw::lilypond-installer
-make LILYPOND_BRANCH=stable/2.12 -f lilypond.make doc
-bin/gub --platform=darwin-x86 'git://git.sv.gnu.org/lilypond-doc.git?branch=stable/2.12'
+@example
+make LILYPOND_BRANCH=release/unstable lilypond
+@end example
-4. manually upload them. good luck figuring out the rsync
-command(s). Hints are in test-lily/
+The first time you do this, it will take a very long time.
+Assuming the build has gone well, it can be uploaded using:
-MAKING A RELEASE -- NEW INFO
+@example
+make lilypond-upload
+ LILYPOND_BRANCH=release/unstable
+ LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git
+@end example
-(wow, I really need to stop being so messy here. Will clean it up
-after 2.13.11; I need a chance to test these instructions)
+@subsubheading Output files
-1. switch to the release branch:
- git checkout release/unstable
+GUB builds the files it needs into the directory
+@code{gub/target/}. As a general rule, these don't need to be
+touched unless there is a problem building GUB (see below).
+The files to be uploaded are in @code{gub/uploads/}. Once the
+build has completed successfully, there should be 8
+installation files and 3 archives, totalling about 600MB.
+There are also 4 directories:
-1b. get latest changes
- git merge origin
+@example
+gub/signatures
+gub/localdoc
+gub/webdoc
+gub/webtest
+@end example
-2. make a release announcement by editing:
- Documentation/web/news-front.itexi
- Documentation/web/news.itexi
-(moving the oldest news item to news-old.itexi)
+@code{signatures} contains files that are used to track whether
+some of the archives have already been built. Don't touch
+these.
-3. upload:
- git push release/unstable
+@code{localdoc} probably contains local copies of the
+documentation.
-4. make GUB as "normal"
- make LILYPOND_BRANCH=release/unstable lilypond
-(if this works, make it the default for GUB)
+@code{webdoc} contains the documentation to be uploaded.
-5. upload git, etc.
+@code{webtest} contains the regtest comparison, which should
+be checked before upload, and is also uploaded for subsequent
+checking.
-6. switch back to master and get the updated news:
- git checkout master
- git cherry-pick release/unstable
+The total upload is about 700 MB in total, and on an ADSL
+connection will take about 4 hours to upload.
-NOTE: this only works on the last change to release/unstable; if
-you had to hack more things to get the release working, then
-you'll need to put the hash directly in there
+@subsubheading Subsequent builds
-7. update VERSION, commit, push.
- VERSION = what you just did +0.0.1
- DEVEL_VERSION = what you just did (i.e. is now online)
- STABLE_VERSION = what's online
+In principle, building the next release of LilyPond requires
+no action other then following the instructions in
+@ref{Minor release checklist}. Because much of the
+infrastructure has already been built, it will take much less
+time - about an hour on a fast computer.
+Continuing to build LilyPond without any other
+archiving/deletion of previous builds is likely to be successful,
+but will take up a fair amount of disk space (around 2GB per
+build) which may be a problem with a Virtual Machine. It's
+therefore recommended to move (not copy) @code{gub/uploads} to
+another machine/disk after each build, if space is at a premium.
-8. (for now) manually upload
- Documentation/pictures/out-www/
- Documentation/web/ly-examples/out-www/
-to $HOME/media/pictures .../ly-examples on lilypond.org
+However, if a significant change has been made to the LilyPond
+source (e.g. added source files) the build may fail if tried on
+top of a previous build. If this happens, be sure to
+move/delete @code{gub/uploads} and all mentions of LilyPond
+in @code{gub/target}. The latter can be achieved with this
+command:
+@example
+rm -rf target/*/*/*lilypond*
+@end example
+Be @emph{very} careful with this command. Typing it wrongly
+could wipe your disk completely.
+
+@subsubheading Updating the web site
+
+The @code{make lilypond-upload} command updates the documentation
+on the LilyPond web site. However, it does @emph{not} update
+any part of the site that is not part of the documentation - for
+example, the front page (@code{index.html}). The website is
+updated by 2 cron jobs running on the web server. One of these
+pulls git master to the web server, and the other makes the
+website with the standard @code{make website} command. They run
+hourly, 30 minutes apart. So - to update the front page of the
+website, it's necessary to update @code{VERSION} and
+@code{news-front.itexi} in master and then wait for the cron
+jobs to run. (N.B. - this is done by pushing the changes to
+staging and letting patchy do its checks before it pushes to
+master).
projects / lilypond.git / blobdiff