@chapter Website work
@menu
-* Introduction to website work::
+* Introduction to website work::
* Uploading and security::
+* Debugging website and docs locally::
* Translating the website::
@end menu
link to any pieces of automatically generated documentation, like
the IR or certain NR appendices.
+@item
+The bibliography in Community->Publications is generated automatically
+from @file{.bib} files; formatting is done automatically by
+@file{texi-web.bst}.
+
+@item
@dots{}
@item
@node Uploading and security
@section Uploading and security
-The website is generated hourly by user @code{graham} the host
-@code{lilypond.org}. For security reasons, we do not use the
-makefiles and scripts directly from git; copies of the relevant
-scripts are examined and copied to
-@code{~graham/lilypond/trusted-scripts/}
+@subheading Overall idea
-@subheading Initial setup
+To reduce the CPU burden on the shared host (as well as some
+security concerns), we do not compile all of LilyPond. The
+website build process runs @code{texi2html}, but all media files
+(be they graphical @code{lilypond} output, photos of people, or
+pdfs) are copied from the @code{$LILYPOND_WEB_MEDIA_GIT}
+repository.
-You should symlink your own @file{~/lilypond/} to
-@file{~graham/@/lilypond/}
+All scripts and makefiles used for the website build are run from
+a @qq{trusted} copy. Any modification to those files in git needs
+a human to review the changes (after they have been made in git)
+before they are used on the server.
-If this directory does not exist, make it. Git master should go
-in @file{~/lilypond/@/lilypond@/-git/} but make sure you enable:
-@example
-git config core.filemode false
-@end example
+@subheading Building the website (quick local)
-If you have created any files in @file{~graham/@/lilypond/} then
-please run:
+Initial setup: make sure that you have the environment variables
+@code{$LILYPOND_GIT}, @code{$LILYPOND_BUILD_DIR} and
+@code{$LILYPOND_WEB_MEDIA_GIT} set up correctly. For more
+information, see @ref{Environment variables}.
+
+Once that is done,
@example
-chgrp lilypond ~graham/lilypond/ -R
-chmod 775 ~graham/lilypond/ -R
+cd $LILYPOND_BUILD_DIR
+make website
@end example
-To reduce the CPU burden on the shared host (as well as some
-security concerns), the @file{Documentation/@/pictures/} and
-@file{Documentation/@/web/@/ly-examples/} directories are
-@strong{not} compiled. You need to upload them, and if they ever
-change, a user in the @code{lilypond} group must upload them to
-@file{~graham/@/media} on the host.
-
-Upload latest pictures/ and ly-examples/ (local script):
+The website is in @file{out-website/website/index.html}.
-@warning{You may need to change a number of items in the below
-script.}
-@verbatim
-### upload-lily-web-media.sh
-#!/bin/sh
-BUILD_DIR=$HOME/src/build-lilypond
+@subheading Building the website (exactly as on the server)
-PICS=$BUILD_DIR/Documentation/pictures/out-www/
-EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/out-www/
+@subsubheading Setting up (exactly as on the server)
-cd $BUILD_DIR
-rsync -a $PICS graham@lilypond.org:media/pictures
-rsync -a $EXAMPLES graham@lilypond.org:ly-examples
-@end verbatim
+Initial setup: you still need @code{$LILYPOND_GIT} and
+@code{$LILYPOND_WEB_MEDIA_GIT}.
+Once that's done, create:
+@example
+mkdir -p $HOME/lilypond/
+mkdir -p $HOME/lilypond/bin/
+mkdir -p $HOME/lilypond/cron/
+mkdir -p $HOME/lilypond/trusted-scripts/
+@end example
-@subheading Normal maintenance
+The add these files to @file{$HOME/lilypond/bin/}:
-Get latest source code:
+Update git repositories:
+@smallexample
@verbatim
### update-git.sh
#!/bin/sh
-cd $HOME/lilypond/lilypond-git
+cd $LILYPOND_GIT
+git fetch origin
+git merge origin/master
+cd $LILYPOND_WEB_MEDIA_GIT
git fetch origin
git merge origin/master
@end verbatim
+@end smallexample
Check for any updates to trusted scripts / files:
+@smallexample
@verbatim
### check-git.sh
#!/bin/sh
-GIT=$HOME/lilypond/lilypond-git
DEST=$HOME/lilypond/trusted-scripts
-diff -u $DEST/website.make $GIT/make/website.make
-diff -u $DEST/lilypond-texi2html.init $GIT/Documentation/lilypond-texi2html.init
-diff -u $DEST/extract_texi_filenames.py $GIT/scripts/build/extract_texi_filenames.py
-diff -u $DEST/create-version-itexi.py $GIT/scripts/build/create-version-itexi.py
-diff -u $DEST/create-weblinks-itexi.py $GIT/scripts/build/create-weblinks-itexi.py
-diff -u $DEST/mass-link.py $GIT/scripts/build/mass-link.py
-diff -u $DEST/website_post.py $GIT/scripts/build/website_post.py
-diff -u $DEST/bib2texi.py $GIT/scripts/build/bib2texi.py
-diff -u $DEST/lilypond.org.htaccess $GIT/Documentation/web/server/lilypond.org.htaccess
-diff -u $DEST/website-dir.htaccess $GIT/Documentation/web/server/website-dir.htaccess
+diff -u $DEST/website.make \
+ $LILYPOND_GIT/make/website.make
+diff -u $DEST/lilypond-texi2html.init \
+ $LILYPOND_GIT/Documentation/lilypond-texi2html.init
+diff -u $DEST/extract_texi_filenames.py \
+ $LILYPOND_GIT/scripts/build/extract_texi_filenames.py
+diff -u $DEST/create-version-itexi.py \
+ $LILYPOND_GIT/scripts/build/create-version-itexi.py
+diff -u $DEST/create-weblinks-itexi.py \
+ $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py
+diff -u $DEST/mass-link.py \
+ $LILYPOND_GIT/scripts/build/mass-link.py
+diff -u $DEST/website_post.py \
+ $LILYPOND_GIT/scripts/build/website_post.py
+diff -u $DEST/bib2texi.py \
+ $LILYPOND_GIT/scripts/build/bib2texi.py
+diff -u $DEST/langdefs.py \
+ $LILYPOND_GIT/python/langdefs.py
+diff -u $DEST/lilypond.org.htaccess \
+ $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess
+diff -u $DEST/website-dir.htaccess \
+ $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess
@end verbatim
+@end smallexample
If the changes look ok, make them trusted:
+@smallexample
@verbatim
### copy-from-git.sh
#!/bin/sh
-GIT=$HOME/lilypond/lilypond-git
DEST=$HOME/lilypond/trusted-scripts
-cp $GIT/make/website.make $DEST/website.make
-cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
-cp $GIT/scripts/build/extract_texi_filenames.py $DEST/extract_texi_filenames.py
-cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
-cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
-cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
-cp $GIT/scripts/build/website_post.py $DEST/website_post.py
-cp $GIT/scripts/build/bib2texi.py $DEST/bib2texi.py
-cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
-cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess
+cp $LILYPOND_GIT/make/website.make \
+ $DEST/website.make
+cp $LILYPOND_GIT/Documentation/lilypond-texi2html.init \
+ $DEST/lilypond-texi2html.init
+cp $LILYPOND_GIT/scripts/build/extract_texi_filenames.py \
+ $DEST/extract_texi_filenames.py
+cp $LILYPOND_GIT/scripts/build/create-version-itexi.py \
+ $DEST/create-version-itexi.py
+cp $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py \
+ $DEST/create-weblinks-itexi.py
+cp $LILYPOND_GIT/scripts/build/mass-link.py \
+ $DEST/mass-link.py
+cp $LILYPOND_GIT/scripts/build/website_post.py \
+ $DEST/website_post.py
+cp $LILYPOND_GIT/scripts/build/bib2texi.py \
+ $DEST/bib2texi.py
+cp $LILYPOND_GIT/python/langdefs.py \
+ $DEST/langdefs.py
+cp $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess \
+ $DEST/lilypond.org.htaccess
+cp $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess \
+ $DEST/website-dir.htaccess
@end verbatim
+@end smallexample
Build the website:
+@smallexample
@verbatim
### make-website.sh
#!/bin/sh
cp $BUILD/out-website/pictures $DEST
cp $BUILD/out-website/.htaccess $DEST
@end verbatim
+@end smallexample
-Cronjob to automate the trusted portions:
+Then in the @file{cronjob/} directory, put the cronjob to automate
+the trusted portions:
+@warning{@code{cron} will not inherit environment variables from
+your main setup, so you must re-define any variables inside your
+@code{crontab}.}
+
+@smallexample
@verbatim
# website-rebuild.cron
+LILYPOND_GIT= ... fill this in
+LILYPOND_WEB_MEDIA_GIT= ... fill this in
+
11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1
22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1
@end verbatim
+@end smallexample
+
+As the final stage of the setup, run your @code{copy-from-git.sh}
+script, assuming that you trust the current state of scripts in
+lilypond git.
+
+@subsubheading Normal maintenance
+
+When there is a change to the build scripts and/or website
+makefile, log in to the server (or your own home machine if you're
+testing this there), and do
+
+@example
+update-git.sh
+check-git.sh
+@end example
+
+After reviewing the changes carefully, you can update the trusted
+scripts with @code{copy-from-git.sh}.
+
+
+@subsubheading Building the website (exactly as on the server)
+
+Run @code{make-website.sh}; the final version ends up in
+@file{$HOME/web/}.
+
+On the actual server, the website is generated hourly by user
+@code{graham} the host @code{lilypond.org}. You can set up the
+cronjob by doing:
+
+@example
+crontab $HOME/lilypond/website-rebuild.cron
+@end example
+
+
+@subheading Initial setup for new users on actual serve
+
+You should symlink your own @file{~/lilypond/} to
+@file{~graham/lilypond/}
+
+If this directory does not exist, make it. Git master should go
+in @file{~/lilypond/lilypond-git/} but make sure you enable:
+
+@example
+git config core.filemode false
+@end example
+
+If you have created any files in @file{~graham/lilypond/} then
+please run:
+
+@example
+chgrp lilypond ~graham/lilypond/ -R
+chmod 775 ~graham/lilypond/ -R
+@end example
+
@subsubheading Additional information
Some information about the website is stored in
-@file{~graham/@/lilypond/@/*.txt}; this information should not be
+@file{~graham/lilypond/*.txt}; this information should not be
shared with people without trusted access to the server.
+@node Debugging website and docs locally
+@section Debugging website and docs locally
+
+@itemize
+@item
+Install apache2, or any other http server. These instructions
+assume that you also enable @code{mod_userdir}, and use
+@code{$HOME/public_html} as the location.
+
+@item
+Build the online docs and website:
+
+@example
+make WEB_TARGETS="offline online" doc
+make website
+@end example
+
+This will make all the language variants of the website. To save
+a little time, just the English version can be made with the
+command @code{make WEB_LANGS='' website} or the English and (for
+example) the French with @code{make WEB_LANGS='fr' website}.
+
+@item
+Move the built stuff into those directories. It's highly
+recommended to have your build dir and www dir on the same
+partition. (make @code{$HOME/public_html/} a symlink if
+necessary)
+
+@example
+mv out-website/website/ $HOME/public_html
+mv $HOME/public_html/website/pictures $HOME/public_html/
+mkdir -p $HOME/public_html/doc/v2.13/
+mv out-www/online-root/* $HOME/public_html/doc/v2.13/
+@end example
+
+@end itemize
+
+
@node Translating the website
@section Translating the website
however, please note that this command is not designed for being
run multiple times. If you see unexpected output (mainly the page
footers getting all messed up), then delete your
-@file{out@/-website} directory and run @code{make website} again.
+@file{out-website} directory and run @code{make website} again.
@item
Some of the translation infrastructure is defined in python files;
@item
Translations are not included by default in @code{make website}.
-To test your translation, edit the @code{WEB_LANGS} line in
-@file{make/@/website@/.make}. Do not submit a patch to add your language
-to this file unless @code{make website} completes with less than 5
-warnings.
+To test your translation, edit the @code{WEB_LANGUAGES =} line in
+@file{python/langdefs.py}. You will need to copy this updated
+script to @code{$LILYPOND_BUILD_DIR/python/out}.
+
+Do not submit a patch to add your language to this file unless
+@code{make website} completes with fewer than 5 warnings.
@item
Links to manuals are done with macros like
projects / lilypond.git / blobdiff