@menu
* Introduction to website work::
* Uploading and security::
+* Debugging website and docs locally::
* Translating the website::
@end menu
@section Introduction to website work
The website is @emph{not} written directly in HTML;
-instead, the source is Texinfo, which is then generated into HTML,
-PDF, and Info formats. The sources are
+instead it is autogenerated along with the documentation through a
+sophisticated setup, using Texinfo source files. Texinfo is the
+standard for documentation of GNU software and allows generating
+output in HTML, PDF, and Info formats, which drastically reduces
+maintenance effort and ensures that the website content is
+consistent with the rest of the documentation. This makes the
+environment for improving the website rather different from common
+web development.
+
+If you have not contributed to LilyPond before, a good starting
+point might be incremental changes to the CSS file, to be found at
+@uref{http://lilypond.org/css/lilypond-website.css} or in the
+LilyPond source code at @file{./Documentation/css/lilypond-website.css}.
+
+Large scale structural changes tend to require familiarity with
+the project in general, a track record in working on LilyPond
+documentation as well as a prospect of long-term commitment.
+
+The Texinfo source file for generating HTML are to be found in
@example
Documentation/web.texi
@node Uploading and security
@section Uploading and security
-The website is generated hourly by user @code{graham} the host
-@code{lilypond.org}. For security reasons, we do not use the
-makefiles and scripts directly from git; copies of the relevant
-scripts are examined and copied to
-@code{~graham/lilypond/trusted-scripts/}
+@subheading Overall idea
-@subheading Initial setup
+To reduce the CPU burden on the shared host (as well as some
+security concerns), we do not compile all of LilyPond. The
+website build process runs @code{texi2html}, but all media files
+(be they graphical @code{lilypond} output, photos of people, or
+pdfs) are copied from the @code{$LILYPOND_WEB_MEDIA_GIT}
+repository.
-You should symlink your own @file{~/lilypond/} to
-@file{~graham/@/lilypond/}
+All scripts and makefiles used for the website build are run from
+a @qq{trusted} copy. Any modification to those files in git needs
+a human to review the changes (after they have been made in git)
+before they are used on the server.
-If this directory does not exist, make it. Git master should go
-in @file{~/lilypond/@/lilypond@/-git/} but make sure you enable:
-@example
-git config core.filemode false
-@end example
+@subheading Building the website (quick local)
-If you have created any files in @file{~graham/@/lilypond/} then
-please run:
+Initial setup: make sure that you have the environment variables
+@code{$LILYPOND_GIT}, @code{$LILYPOND_BUILD_DIR} and
+@code{$LILYPOND_WEB_MEDIA_GIT} set up correctly. For more
+information, see @ref{Environment variables}.
+
+Once that is done,
@example
-chgrp lilypond ~graham/lilypond/ -R
-chmod 775 ~graham/lilypond/ -R
+cd $LILYPOND_BUILD_DIR
+make website
@end example
-To reduce the CPU burden on the shared host (as well as some
-security concerns), the @file{Documentation/@/pictures/} and
-@file{Documentation/@/web/@/ly-examples/} directories are
-@strong{not} compiled. You need to upload them, and if they ever
-change, a user in the @code{lilypond} group must upload them to
-@file{~graham/@/lilypond@/media} on the host.
+The website is in @file{out-website/website/index.html}.
-Upload latest pictures/ and ly-examples/ (local script):
-@warning{You may need to change a number of items in the below
-script.}
+@subheading Building the website (exactly as on the server)
-@verbatim
-### upload-lily-web-media.sh
-#!/bin/sh
-BUILD_DIR=$HOME/src/build-lilypond
+@subsubheading Setting up (exactly as on the server)
-PICS=$BUILD_DIR/Documentation/pictures/out-www/
-EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/out-www/
-
-cd $BUILD_DIR
-rsync -a $PICS graham@lilypond.org:lilypond/media/pictures
-rsync -a $EXAMPLES graham@lilypond.org:lilypond/media/ly-examples
-@end verbatim
+Initial setup: you still need @code{$LILYPOND_GIT} and
+@code{$LILYPOND_WEB_MEDIA_GIT}.
+Once that's done, create:
+@example
+mkdir -p $HOME/lilypond/
+mkdir -p $HOME/lilypond/bin/
+mkdir -p $HOME/lilypond/cron/
+mkdir -p $HOME/lilypond/trusted-scripts/
+@end example
-@subheading Normal maintenance
+The add these files to @file{$HOME/lilypond/bin/}:
-Get latest source code:
+Update git repositories:
+@smallexample
@verbatim
### update-git.sh
#!/bin/sh
-cd $HOME/lilypond/lilypond-git
+cd $LILYPOND_GIT
+git fetch origin
+git merge origin/master
+cd $LILYPOND_WEB_MEDIA_GIT
git fetch origin
git merge origin/master
@end verbatim
+@end smallexample
Check for any updates to trusted scripts / files:
+@smallexample
@verbatim
### check-git.sh
#!/bin/sh
-GIT=$HOME/lilypond/lilypond-git
DEST=$HOME/lilypond/trusted-scripts
-diff -u $DEST/website.make $GIT/make/website.make
-diff -u $DEST/lilypond-texi2html.init $GIT/Documentation/lilypond-texi2html.init
-diff -u $DEST/extract_texi_filenames.py $GIT/scripts/build/extract_texi_filenames.py
-diff -u $DEST/create-version-itexi.py $GIT/scripts/build/create-version-itexi.py
-diff -u $DEST/create-weblinks-itexi.py $GIT/scripts/build/create-weblinks-itexi.py
-diff -u $DEST/mass-link.py $GIT/scripts/build/mass-link.py
-diff -u $DEST/website_post.py $GIT/scripts/build/website_post.py
-diff -u $DEST/bib2texi.py $GIT/scripts/build/bib2texi.py
-diff -u $DEST/lilypond.org.htaccess $GIT/Documentation/web/server/lilypond.org.htaccess
-diff -u $DEST/website-dir.htaccess $GIT/Documentation/web/server/website-dir.htaccess
+diff -u $DEST/website.make \
+ $LILYPOND_GIT/make/website.make
+diff -u $DEST/lilypond-texi2html.init \
+ $LILYPOND_GIT/Documentation/lilypond-texi2html.init
+diff -u $DEST/extract_texi_filenames.py \
+ $LILYPOND_GIT/scripts/build/extract_texi_filenames.py
+diff -u $DEST/create-version-itexi.py \
+ $LILYPOND_GIT/scripts/build/create-version-itexi.py
+diff -u $DEST/create-weblinks-itexi.py \
+ $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py
+diff -u $DEST/mass-link.py \
+ $LILYPOND_GIT/scripts/build/mass-link.py
+diff -u $DEST/website_post.py \
+ $LILYPOND_GIT/scripts/build/website_post.py
+diff -u $DEST/bib2texi.py \
+ $LILYPOND_GIT/scripts/build/bib2texi.py
+diff -u $DEST/langdefs.py \
+ $LILYPOND_GIT/python/langdefs.py
+diff -u $DEST/lilypond.org.htaccess \
+ $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess
+diff -u $DEST/website-dir.htaccess \
+ $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess
@end verbatim
+@end smallexample
If the changes look ok, make them trusted:
+@smallexample
@verbatim
### copy-from-git.sh
#!/bin/sh
-GIT=$HOME/lilypond/lilypond-git
DEST=$HOME/lilypond/trusted-scripts
-cp $GIT/make/website.make $DEST/website.make
-cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
-cp $GIT/scripts/build/extract_texi_filenames.py $DEST/extract_texi_filenames.py
-cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
-cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
-cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
-cp $GIT/scripts/build/website_post.py $DEST/website_post.py
-cp $GIT/scripts/build/bib2texi.py $DEST/bib2texi.py
-cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
-cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess
+cp $LILYPOND_GIT/make/website.make \
+ $DEST/website.make
+cp $LILYPOND_GIT/Documentation/lilypond-texi2html.init \
+ $DEST/lilypond-texi2html.init
+cp $LILYPOND_GIT/scripts/build/extract_texi_filenames.py \
+ $DEST/extract_texi_filenames.py
+cp $LILYPOND_GIT/scripts/build/create-version-itexi.py \
+ $DEST/create-version-itexi.py
+cp $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py \
+ $DEST/create-weblinks-itexi.py
+cp $LILYPOND_GIT/scripts/build/mass-link.py \
+ $DEST/mass-link.py
+cp $LILYPOND_GIT/scripts/build/website_post.py \
+ $DEST/website_post.py
+cp $LILYPOND_GIT/scripts/build/bib2texi.py \
+ $DEST/bib2texi.py
+cp $LILYPOND_GIT/python/langdefs.py \
+ $DEST/langdefs.py
+cp $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess \
+ $DEST/lilypond.org.htaccess
+cp $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess \
+ $DEST/website-dir.htaccess
@end verbatim
+@end smallexample
Build the website:
+@smallexample
@verbatim
### make-website.sh
#!/bin/sh
cp $BUILD/out-website/pictures $DEST
cp $BUILD/out-website/.htaccess $DEST
@end verbatim
+@end smallexample
+
+Then in the @file{cronjob/} directory, put the cronjob to automate
+the trusted portions:
-Cronjob to automate the trusted portions:
+@warning{@code{cron} will not inherit environment variables from
+your main setup, so you must re-define any variables inside your
+@code{crontab}.}
+@smallexample
@verbatim
# website-rebuild.cron
+LILYPOND_GIT= ... fill this in
+LILYPOND_WEB_MEDIA_GIT= ... fill this in
+
11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1
22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1
@end verbatim
+@end smallexample
+
+As the final stage of the setup, run your @code{copy-from-git.sh}
+script, assuming that you trust the current state of scripts in
+lilypond git.
+
+@subsubheading Normal maintenance
+
+When there is a change to the build scripts and/or website
+makefile, log in to the server (or your own home machine if you're
+testing this there), and do
+
+@example
+update-git.sh
+check-git.sh
+@end example
+
+After reviewing the changes carefully, you can update the trusted
+scripts with @code{copy-from-git.sh}.
+
+
+@subsubheading Building the website (exactly as on the server)
+
+Run @code{make-website.sh}; the final version ends up in
+@file{$HOME/web/}.
+
+On the actual server, the website is generated hourly by user
+@code{graham} the host @code{lilypond.org}. You can set up the
+cronjob by doing:
+
+@example
+crontab $HOME/lilypond/website-rebuild.cron
+@end example
+
+
+@subheading Initial setup for new users on actual serve
+
+You should symlink your own @file{~/lilypond/} to
+@file{~graham/lilypond/}
+
+If this directory does not exist, make it. Git master should go
+in @file{~/lilypond/lilypond-git/} but make sure you enable:
+
+@example
+git config core.filemode false
+@end example
+
+If you have created any files in @file{~graham/lilypond/} then
+please run:
+
+@example
+chgrp lilypond ~graham/lilypond/ -R
+chmod 775 ~graham/lilypond/ -R
+@end example
+
@subsubheading Additional information
Some information about the website is stored in
-@file{~graham/@/lilypond/@/*.txt}; this information should not be
+@file{~graham/lilypond/*.txt}; this information should not be
shared with people without trusted access to the server.
+@node Debugging website and docs locally
+@section Debugging website and docs locally
+
+@itemize
+@item
+Install Apache (you can use version 2, but keep in mind that the server
+hosting lilypond.org runs version 1.3). These instructions
+assume that you also enable @code{mod_userdir}, and use
+@code{$HOME/public_html} as DocumentRoot (i.e. the root directory of
+the web server).
+
+@item
+Build the online docs and website:
+
+@example
+make WEB_TARGETS="offline online" doc
+make website
+@end example
+
+This will make all the language variants of the website. To save
+a little time, just the English version can be made with the
+command @code{make WEB_LANGS='' website} or the English and (for
+example) the French with @code{make WEB_LANGS='fr' website}.
+
+@item
+Choose the web directory where to copy the built stuff.
+If you already have other web projects in your DocumentRoot and don't
+need to test the @file{.htaccess} file, you can copy to
+@code{~/public_html/lilypond.org}. Otherwise you'd better copy
+to @code{~/public_html}.
+It's highly recommended to have your build dir and web dir on the
+same partition.
+
+@item
+Add the directory for the online documentation:
+
+@example
+mkdir -p ~/public_html/doc/v2.19/
+@end example
+
+You may want to add also the stable documentation in
+@code{~/public_html/doc/v2.18/}, extracting the contents of the html
+directory present in the tarball available in @rweb{All}. Just in case
+you want to test the redirects to the stable documentation.
+
+@item
+Copy the files with rsync:
+
+@example
+rsync -av --delete out-website/website ~/public_html/
+cp out-website/.htaccess ~/public_html
+rsync -av --delete out-www/online-root/ ~/public_html/doc/v2.19/
+@end example
+
+@end itemize
+
+
@node Translating the website
@section Translating the website
however, please note that this command is not designed for being
run multiple times. If you see unexpected output (mainly the page
footers getting all messed up), then delete your
-@file{out@/-website} directory and run @code{make website} again.
+@file{out-website} directory and run @code{make website} again.
@item
Some of the translation infrastructure is defined in python files;
@item
Translations are not included by default in @code{make website}.
-To test your translation, edit the @code{WEB_LANGS} line in
-@file{make/@/website@/.make}. Do not submit a patch to add your language
-to this file unless @code{make website} completes with less than 5
-warnings.
+To test your translation, edit the @code{WEB_LANGUAGES =} line in
+@file{python/langdefs.py}. You will need to copy this updated
+script to @code{$LILYPOND_BUILD_DIR/python/out}.
+
+Do not submit a patch to add your language to this file unless
+@code{make website} completes with fewer than 5 warnings.
@item
Links to manuals are done with macros like
@end itemize
-
-
projects / lilypond.git / blobdiff