* Minor release checklist::
* Major release checklist::
* Release extra notes::
-* Administrative policies::
+* Notes on builds with GUB::
@end menu
@node Development phases
@section Development phases
-There are 2.5 states of development for LilyPond.
+There are 2 states of development on @code{master}:
-@itemize
+@enumerate
+
+@item @strong{Normal development}:
+Any commits are fine.
-@item @strong{Stable phase}:
-Starting from the release of a new major version @code{2.x.0}, the
-following patches @strong{MAY NOT} be merged with master:
+@item @strong{Build-frozen}:
+Do not require any additional or updated libraries or make
+non-trivial changes to the build process. Any such patch (or
+branch) may not be merged with master during this period.
+
+This should occur approximately 1 month before any alpha version
+of the next stable release, and ends when the next unstable branch
+begins.
+
+@end enumerate
+
+
+After announcing a beta release, branch @code{stable/2.x}. There
+are 2 states of development for this branch:
+
+@enumerate
+@item @strong{Normal maintenance}:
+The following patches @strong{MAY NOT} be merged with this branch:
@itemize
@item Any change to the input syntax. If a file compiled with a
-previous @code{2.x} version, then it must compile in the new
-version.
+previous @code{2.x} (beta) version, then it must compile in the
+new version.
+
+Exception: any bugfix to a Critical issue.
@item New features with new syntax @emph{may be committed},
although once committed that syntax cannot change during the
@end itemize
-@item @strong{Development phase}:
-Any commits are fine. Readers may be familiar with the term
-@qq{merge window} from following Linux kernel news.
+@item @strong{Release prep}:
+Only translation updates and important bugfixes are allowed.
+@end enumerate
-@item @strong{Release prep phase}:
-TODO: I don't like that name.
-A new git branch @code{stable/2.x} is created, and a major release
-is made in two weeks.
+@node Minor release checklist
+@section Minor release checklist
-@itemize
+A @qq{minor release} means an update of @code{y} in @code{2.x.y}.
-@item @code{stable/2.x branch}:
-Only translation updates and important bugfixes are allows.
+@subheading Pre-release
-@item @code{master}:
-Normal @qq{stable phase} development occurs.
+@enumerate
-@end itemize
+@item
+Don't forget to prepare the GUB build machine by deleting and moving unneeded
+files: see @qq{Subsequent builds} in @ref{Notes on builds with GUB}.
-If we discover the need to change the syntax or build system, we
-will apply it and re-start the release prep phase.
+@item
+Using any system with git pull access (not necessarily the GUB
+build machine), use the commands below to do the following:
-@end itemize
+@itemize
+@item
+switch to the release branch
-This marks a radical change from previous practice in LilyPond.
-However, this setup is not intended to slow development -- as a
-rule of thumb, the next development phase will start within a
-month of somebody wanting to commit something which is not
-permitted during the stable phase.
+@item
+update the release branch from origin/master
+@item
+update the translation files
+@item
+create the release announcement
-@node Minor release checklist
-@section Minor release checklist
+@item
+update the build versions.
-A @qq{minor release} means an update of @code{y} in @code{2.x.y}.
+@itemize
+@item
+VERSION_DEVEL = the current development version (previous VERSION_DEVEL + 0.01)
-@subheading Pre-release
+@item
+VERSION_STABLE = the current stable version (probably no change here)
-@enumerate
+@end itemize
@item
-Switch to the release branch, get changes, prep release
-announcement:
+update the @qq{Welcome to Lilypond} version numbers to the version about to be
+released
+
+@end itemize
+
+This requires a system which has the release/unstable
+branch. If you get a warning saying you are in @code{detached HEAD}
+state, then you should create a release/unstable branch with
+@code{git checkout release/unstable}.
+
+Check the environment variables are set as in
+@ref{Environment variables}.
+
+You need to ensure you have a clean index and work tree. If the
+checkout displays modified files, you might want to run
+@code{git reset --hard} before continuing.
@example
+git fetch
git checkout release/unstable
-git merge origin
-vi Documentation/web/news-front.itexi Documentation/web/news.itexi
+git merge origin/master
+make -C $LILYPOND_BUILD_DIR po-replace
+mv $LILYPOND_BUILD_DIR/po/lilypond.pot po/
+gedit Documentation/web/news-front.itexi Documentation/web/news.itexi
+gedit VERSION
+gedit ly/Wel*.ly
@end example
@item
-Commit, push, switch back to master:
+Commit, push, switch back to master (or wherever else):
@example
+git commit -m "Release: bump VERSION_DEVEL." VERSION
+git commit -m "PO: update template." po/lilypond.pot
git commit -m "Release: update news." Documentation/web/
-git push origin
+git commit -m "Release: bump Welcome versions." ly/Wel*.ly
+git push origin HEAD:release/unstable
git checkout master
-git merge release/unstable
@end example
-@item (optional) Check that lilypond builds from scratch in an
-out-of-tree build.
-
@item
If you do not have the previous release test-output tarball, download
it and put it in @code{regtests/}
+@item Prepare GUB environment by running:
+
+@example
+### my-gub.sh
+# special terminal, and default PATH environment.
+# import these special environment vars:
+# HOME, HTTP_PROXY, TERM
+env -i \
+ HOME=$HOME \
+ HTTP_PROXY=$HTTP_PROXY \
+ bash --rcfile my-bashrc
+@end example
+
+@example
+### my-bashrc
+export PS1="\[\e[1;33mGUB-ENV \w\]$ \[\e[0m\]"
+export PATH=$PATH
+export TERM=xterm
+@end example
+
+
@item Build release on GUB by running:
@example
or something like:
@example
-make LILYPOND_BRANCH=stable/2.12 lilypond
+make LILYPOND_BRANCH=stable/2.16 lilypond
@end example
-@item Check the regtest comparison in @file{uploads/webtest/} for
-any unintentional breakage.
-
-Note that this test uses the bounding boxes inside lilypond.
-Errors in ghostscript don't generate differences inside lilypond,
-so they are not registered in the regtest comparison.
+@item
+Check the regtest comparison in @file{uploads/webtest/} for
+any unintentional breakage. More info in
+@ref{Precompiled regression tests}.
-@item If any work was done on GUB since the last release, upload
+@item
+If any work was done on GUB since the last release, upload
binaries to a temporary location, ask for feedback, and wait a day
or two in case there's any major problems.
+@warning{Always do this for a stable release.}
+
@end enumerate
@enumerate
-@item If you're not right user on the webserver, remove the "t"
-from the rsync command in @file{test-lily/rsync-lily-doc.py} and
-@file{test-lily/rsync-test.py}
+@item If you're not the right user on the webserver, remove the
+@code{t} from the rsync command in:
-@code{graham} owns v2.13; @code{han-wen} owns v2.12.
+@example
+test-lily/rsync-lily-doc.py
+test-lily/rsync-test.py
+@end example
@item Upload GUB by running:
@example
-make lilypond-upload LILYPOND_BRANCH=master LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git
+make lilypond-upload \
+ LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git \
+ LILYPOND_BRANCH=release/unstable
@end example
@noindent
or something like:
@example
-make lilypond-upload LILYPOND_BRANCH=stable/2.12 LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git
+make lilypond-upload \
+ LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git \
+ LILYPOND_BRANCH=stable/2.12
@end example
@end enumerate
@enumerate
-@item Switch back to master and get the updated news:
+@item
+Update the current staging branch with the current news:
@example
-git checkout master
-git merge release/unstable
+git fetch
+git checkout origin/staging
+git merge origin/release/unstable
@end example
-@item Update @file{VERSION} in lilypond git and upload changes:
-
-@quotation
-@quotation
-VERSION = what you just did +0.0.1
-DEVEL_VERSION = what you just did (i.e. is now online)
-STABLE_VERSION = what's online
-@end quotation
-@end quotation
+@item
+Update @file{VERSION} in lilypond git and upload changes:
@example
-vi VERSION
-git commit -m "Release: bump version." VERSION
-git push origin
+gedit VERSION
@end example
-
-@item (for now) do a @code{make doc} and manually upload:
+@itemize
+@item
+VERSION = what you just did +0.0.1
+@end itemize
@example
-### upload-lily-web-media.sh
-#!/bin/sh
-BUILD_DIR=$HOME/src/build-lilypond
+git commit -m "Release: bump VERSION." VERSION
+git push origin HEAD:staging
+@end example
-PICS=$BUILD_DIR/Documentation/pictures/out-www/
-EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/out-www/
+If the push fails with a message like
-cd $BUILD_DIR
-rsync -a $PICS graham@@lilypond.org:media/pictures
-rsync -a $EXAMPLES graham@@lilypond.org:media/ly-examples
+@example
+ ! [rejected] HEAD -> staging (non-fast-forward)
@end example
+@noindent
+it means that somebody else updated the staging branch while you were
+preparing your change. In that case, you need to restart the Post
+Release process. Otherwise, proceed:
+
@item Wait a few hours for the website to update.
@item Email release notice to @code{info-lilypond}
A @qq{major release} means an update of @code{x} in @code{2.x.0}.
-- happens when we have 0 Critical issues for two weeks (14 days).
+@subheading Main requirements
+These are the current official guidelines.
-Before release:
+@itemize
+@item
+0 Critical issues for two weeks (14 days) after the latest release
+candidate.
+
+@end itemize
-* write release notes. note: stringent size requirements for
- various websites, so be brief.
-* write preface section for manual.
+@subheading Potential requirements
-* submit pots for translation : send url of tarball to
-translation@@iro.umontreal.ca, mentioning lilypond-VERSION.pot
+These might become official guidelines in the future.
-* Check reg test
+@itemize
+@item
+Check reg test
+
+@item
+Check all 2ly scripts
+
+@item
+Check for emergencies the docs:
+
+@example
+grep FIXME --exclude "misc/*" --exclude "*GNUmakefile" \
+ --exclude "snippets/*" ????*/*
+@end example
-* Check all 2ly scripts.
+@item
+Check for altered regtests, and document as necessary:
-* Run convert-ly on all files, bump parser minimum version.
+@example
+git diff -u -r release/2.@var{FIRST-CURRENT-STABLE} \
+ -r release/2.@var{LAST-CURRENT-DEVELOPMENT} input/regression/
+@end example
-* update links to distros providing lilypond packages? link in
-Documentation/web/download.itexi . This has nothing to do with
-the release, but I'm dumping this here so I'll find it when I
-reorganize this list later. -gp
+@end itemize
-* Make FTP directories on lilypond.org
-* website:
- - Make new table in download.html
+@subheading Housekeeping requirements
- - add to documentation list
+Before the release:
- - revise examples tour.html/howto.html
+@itemize
+@item
+write release notes. note: stringent size requirements for
+various websites, so be brief.
- - add to front-page quick links
+@item
+Run convert-ly on all files, bump parser minimum version.
- - change all links to the stable documentation
+@item
+Update lilypond.pot:
- - doc auto redirects to v2.LATEST-STABLE
+@example
+make -C $LILYPOND_BUILD_DIR po-replace
+mv $LILYPOND_BUILD_DIR/po/lilypond.pot po/
+@end example
- - add these two lines to http://www.lilypond.org/robots.txt:
+@item
+Make directories on lilypond.org:
@example
-Disallow: /doc/v2.PREVIOUS-STABLE/
-Disallow: /doc/v2.CURRENT-DEVELOPMENT/
+~/download/sources/v2.@var{NEW-STABLE}
+~/download/sources/v2.@var{NEW-DEVELOPMENT}
@end example
-- check for emergencies the docs:
+@item
+Shortly after the release, move all current contributors to
+previous contributors in
+@file{Documentation/included/authors.itexi}.
+
+@item
+Delete old material in @file{Documentation/changes.tely}, but
+don't forget to check it still compiles! Also update the version
+numbers:
@example
-grep FIXME --exclude "misc/*" --exclude "*GNUmakefile" \
- --exclude "snippets/*" ????*/*
+@@node Top
+@@top New features in 2.@var{NEW-STABLE} since 2.@var{OLD-STABLE}
@end example
-- check for altered regtests, and document as necessary. (update
- tags as appropriate)
+@item
+Website:
+
+@itemize
+@item
+make a link from the old unstable to the next stable in
+lilypond.org's @file{/doc/} dir. Keep all previous unstable->stable
+doc symlinks.
+
+Also, make the old docs self-contained -- if there's a redirect in
+@file{/doc/v2.@var{OLD-STABLE}/Documentation/index.html} , replace it with the
+@file{index.html.old-2.@var{OLD-STABLE}} files.
+
+The post-2.13 docs will need another way of handling the
+self-containment. It won't be hard to whip up a python script
+that changes the link to @file{../../../../manuals.html} to
+@file{../website/manuals.html}, but it's still a 30-minute task that
+needs to be done before 2.16.
+
+@item
+doc auto redirects to @code{v2.@var{NEW-STABLE}}
+
+@item
+add these two lines to @file{Documentation/web/server/robots.txt}:
@example
-git diff -u -r release/2.12.0-1 -r release/2.13.13-1 input/regression/
+Disallow: /doc/v2.@var{OLD-STABLE}/
+Disallow: /doc/v2.@var{NEW-DEVELOPMENT}/
@end example
+@end itemize
+
+@end itemize
+
+@subheading Unsorted
+
+@itemize
+@item
+submit po template for translation: send url of tarball to
+@email{coordinator@@translationproject.org}, mentioning
+lilypond-VERSION.pot
+
+@item
+update links to distros providing lilypond packages? link in:
+@file{Documentation/web/download.itexi}
+
+This has nothing to do with the release, but it's a @qq{periodic
+maintenance} task that might make sense to include with releases.
+
+@item
+Send announcements to...
News:
- comp.music.research
- comp.os.linux.announce
+@example
+comp.music.research
+comp.os.linux.announce
- comp.text.tex
- rec.music.compose
+comp.text.tex
+rec.music.compose
+@end example
Mail:
- info-lilypond@@gnu.org
+@example
+info-lilypond@@gnu.org
linux-audio-announce@@lists.linuxaudio.org
linux-audio-user@@lists.linuxaudio.org
linux-audio-dev@@lists.linuxaudio.org
- tex-music@@icking-music-archive.org
+tex-music@@icking-music-archive.org
- --- non-existant?
- abcusers@@blackmill.net
+--- non-existant?
+abcusers@@blackmill.net
- rosegarden-user@@lists.sourceforge.net
- info-gnu@@gnu.org
- noteedit-user@@berlios.de
+rosegarden-user@@lists.sourceforge.net
+info-gnu@@gnu.org
+noteedit-user@@berlios.de
- gmane.comp.audio.fomus.devel
- gmane.linux.audio.users
- gmane.linux.audio.announce
- gmane.comp.audio.rosegarden.devel
+gmane.comp.audio.fomus.devel
+gmane.linux.audio.users
+gmane.linux.audio.announce
+gmane.comp.audio.rosegarden.devel
+@end example
Web:
- lilypond.org
- freshmeat.net
- linuxfr.com
- http://www.apple.com/downloads
- harmony-central.com (news@@harmony-central.com)
- versiontracker.com [auto]
- hitsquad.com [auto]
- http://www.svgx.org
- https://savannah.gnu.org/news/submit.php?group_id=1673 @c => planet.gnu.org
+@example
+lilypond.org
+freshmeat.net
+linuxfr.com
+http://www.apple.com/downloads
+harmony-central.com (news@@harmony-central.com)
+versiontracker.com [auto]
+hitsquad.com [auto]
+http://www.svgx.org
+https://savannah.gnu.org/news/submit.php?group_id=1673 @c => planet.gnu.org
+@end example
+
+@end itemize
@node Release extra notes
latest git.
@item
-- configure; make; make test
+configure; make; make test
@item
tar -cjf lilypond-X.Y.Z-A.test-output.tar.bz2 input/regression/out-test/
@example
bin/gub mingw::lilypond-installer
make LILYPOND_BRANCH=stable/2.12 -f lilypond.make doc
-bin/gub --platform=darwin-x86 'git://git.sv.gnu.org/lilypond-doc.git?branch=stable/2.12'
+bin/gub --platform=darwin-x86 \
+ 'git://git.sv.gnu.org/lilypond-doc.git?branch=stable/2.12'
@end example
or
@end enumerate
+@node Notes on builds with GUB
+@section Notes on builds with GUB
+
+@subsubheading Building GUB
+
+GUB - the Grand Unified Builder - is used to build the release
+versions of LilyPond. For background information, see
+@ref{Grand Unified Builder (GUB)}. The simplest way to set up a
+GUB build environment is to use a virtual machine with LilyDev
+(@ref{LilyDev}). Follow the instructions on that page to set this
+up. Make sure that your virtual machine has enough disk space -
+a GUB installation takes over 30 GBytes of disk space, and if you
+allocate too little, it will fail during the setting up stage and
+you will have to start again. 64 GBytes should be sufficient.
+
+While GUB is being built, any interruptions are likely to make it
+almost impossible to restart. If at all possible, leave the build
+to continue uninterrupted.
+
+Download GUB and start the set up:
+
+@example
+git clone git://github.com/gperciva/gub.git
+cd gub
+make bootstrap
+@end example
+
+This will take a very long time, even on a very fast computer.
+You will need to be patient. It's also liable to fail - it
+downloads a number of tools, and some will have moved and others
+won't respond to the network. For example, the perl archive.
+If this happens, download it from
+@uref{http://www.cpan.org/src/5.0/perl-5.10.0.tar.gz}, saving the
+archive to @file{gub/downloads/perl/}. Continue the set up with:
-@node Administrative policies
-@section Administrative policies
+@example
+make bootstrap
+@end example
-Not really release-specific notes, but I don't see enough material
-here to make it a separate chapter, and the release person will
-probably be the one taking care of this anyway.
+Once this has completed successfully, you can build the LilyPond
+release package. However, this uses an archived version of the
+regression tests, so it is better to download this first.
+Download the test output from lilypond.org (you will need to
+replace @code{2.15.33-1} with the latest build):
-@subsubheading Language-specific mailing lists
+@smallexample
+@uref{http://lilypond.org/download/binaries/test-output/lilypond-2.15.33-1.test-output.tar.bz2}
+@end smallexample
-A translator can ask for an official lilypond-xy mailing list once
-they've finished all @qq{priority 1} translation items.
+Copy the tarball into @file{gub/regtests/}, and tell the build
+system that you have done this:
-@subsubheading Performing yearly copyright update (@qq{grand-replace})
+@example
+touch regtests/ignore
+@end example
-At the start of each year, copyright notices for all source files
-should be refreshed by running the following command from the top of
-the source tree:
+Now start the GUB build:
@example
-make grand-replace
+make lilypond
+@end example
+
+That's it. This will build LilyPond from current master. To build
+the current unstable release, run:
+
+@example
+make LILYPOND_BRANCH=release/unstable lilypond
@end example
-Internally, this invokes the script @file{scripts/build/grand-replace.py},
-which performs a regular expression substitution for old-year -> new-year
-wherever it finds a valid copyright notice.
+The first time you do this, it will take a very long time.
+
+Assuming the build has gone well, it can be uploaded using:
+
+@example
+make lilypond-upload
+ LILYPOND_BRANCH=release/unstable
+ LILYPOND_REPO_URL=git://git.sv.gnu.org/lilypond.git
+@end example
+
+@subsubheading Output files
+
+GUB builds the files it needs into the directory
+@code{gub/target/}. As a general rule, these don't need to be
+touched unless there is a problem building GUB (see below).
+The files to be uploaded are in @code{gub/uploads/}. Once the
+build has completed successfully, there should be 8
+installation files and 3 archives, totalling about 600MB.
+There are also 4 directories:
+
+@example
+gub/signatures
+gub/localdoc
+gub/webdoc
+gub/webtest
+@end example
+
+@code{signatures} contains files that are used to track whether
+some of the archives have already been built. Don't touch
+these.
+
+@code{localdoc} probably contains local copies of the
+documentation.
+
+@code{webdoc} contains the documentation to be uploaded.
+
+@code{webtest} contains the regtest comparison, which should
+be checked before upload, and is also uploaded for subsequent
+checking.
+
+The total upload is about 700 MB in total, and on an ADSL
+connection will take about 4 hours to upload.
+
+@subsubheading Subsequent builds
+
+In principle, building the next release of LilyPond requires
+no action other then following the instructions in
+@ref{Minor release checklist}. Because much of the
+infrastructure has already been built, it will take much less
+time - about an hour on a fast computer.
+
+Continuing to build LilyPond without any other
+archiving/deletion of previous builds is likely to be successful,
+but will take up a fair amount of disk space (around 2GB per
+build) which may be a problem with a Virtual Machine. It's
+therefore recommended to move (not copy) @code{gub/uploads} to
+another machine/disk after each build, if space is at a premium.
+
+However, if a significant change has been made to the LilyPond
+source (e.g. added source files) the build may fail if tried on
+top of a previous build. If this happens, be sure to
+move/delete @code{gub/uploads} and all mentions of LilyPond
+in @code{gub/target}. The latter can be achieved with this
+command:
+
+@example
+rm -rf target/*/*/*lilypond*
+@end example
-Note that snapshots of third party files such as @file{texinfo.tex} should
-not be included in the automatic update; @file{grand-replace.py} ignores these
-files if they are listed in the variable @code{copied_files}.
+Be @emph{very} careful with this command. Typing it wrongly
+could wipe your disk completely.
+
+@subsubheading Updating the web site
+
+The @code{make lilypond-upload} command updates the documentation
+on the LilyPond web site. However, it does @emph{not} update
+any part of the site that is not part of the documentation - for
+example, the front page (@code{index.html}). The website is
+updated by 2 cron jobs running on the web server. One of these
+pulls git master to the web server, and the other makes the
+website with the standard @code{make website} command. They run
+hourly, 30 minutes apart. So - to update the front page of the
+website, it's necessary to update @code{VERSION} and
+@code{news-front.itexi} in master and then wait for the cron
+jobs to run. (N.B. - this is done by pushing the changes to
+staging and letting patchy do its checks before it pushes to
+master).
projects / lilypond.git / blobdiff