@c -*- coding: utf-8; mode: texinfo; -*- @node Website work @chapter Website work @menu * Introduction to website work:: * Uploading and security:: * Debugging website and docs locally:: * Translating the website:: @end menu @node Introduction to website work @section Introduction to website work The website is @emph{not} written directly in HTML; instead, the source is Texinfo, which is then generated into HTML, PDF, and Info formats. The sources are @example Documentation/web.texi Documentation/web/*.texi @end example Unless otherwise specified, follow the instructions and policies given in @ref{Documentation work}. That chapter also contains a quick introduction to Texinfo; consulting an external Texinfo manual should be not necessary. @subheading Exceptions to the documentation policies @itemize @item Sectioning: the website only uses chapters and sections; no subsections or subsubsections. @item @@ref@{@}s to other manuals (@@ruser, @@rlearning, etc): you can't link to any pieces of automatically generated documentation, like the IR or certain NR appendices. @item The bibliography in Community->Publications is generated automatically from @file{.bib} files; formatting is done automatically by @file{texi-web.bst}. @item @dots{} @item For anything not listed here, just follow the same style as the existing website texinfo files. @end itemize @node Uploading and security @section Uploading and security @subheading Overall idea To reduce the CPU burden on the shared host (as well as some security concerns), we do not compile all of LilyPond. The website build process runs @code{texi2html}, but all media files (be they graphical @code{lilypond} output, photos of people, or pdfs) are copied from the @code{$LILYPOND_WEB_MEDIA_GIT} repository. All scripts and makefiles used for the website build are run from a @qq{trusted} copy. Any modification to those files in git needs a human to review the changes (after they have been made in git) before they are used on the server. @subheading Building the website (quick local) Initial setup: make sure that you have the environment variables @code{$LILYPOND_GIT}, @code{$LILYPOND_BUILD_DIR} and @code{$LILYPOND_WEB_MEDIA_GIT} set up correctly. For more information, see @ref{Environment variables}. Once that is done, @example cd $LILYPOND_BUILD_DIR make website @end example The website is in @file{out-website/website/index.html}. @subheading Building the website (exactly as on the server) @subsubheading Setting up (exactly as on the server) Initial setup: you still need @code{$LILYPOND_GIT} and @code{$LILYPOND_WEB_MEDIA_GIT}. Once that's done, create: @example mkdir -p $HOME/lilypond/ mkdir -p $HOME/lilypond/bin/ mkdir -p $HOME/lilypond/cron/ mkdir -p $HOME/lilypond/trusted-scripts/ @end example The add these files to @file{$HOME/lilypond/bin/}: Update git repositories: @smallexample @verbatim ### update-git.sh #!/bin/sh cd $LILYPOND_GIT git fetch origin git merge origin/master cd $LILYPOND_WEB_MEDIA_GIT git fetch origin git merge origin/master @end verbatim @end smallexample Check for any updates to trusted scripts / files: @smallexample @verbatim ### check-git.sh #!/bin/sh DEST=$HOME/lilypond/trusted-scripts diff -u $DEST/website.make \ $LILYPOND_GIT/make/website.make diff -u $DEST/lilypond-texi2html.init \ $LILYPOND_GIT/Documentation/lilypond-texi2html.init diff -u $DEST/extract_texi_filenames.py \ $LILYPOND_GIT/scripts/build/extract_texi_filenames.py diff -u $DEST/create-version-itexi.py \ $LILYPOND_GIT/scripts/build/create-version-itexi.py diff -u $DEST/create-weblinks-itexi.py \ $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py diff -u $DEST/mass-link.py \ $LILYPOND_GIT/scripts/build/mass-link.py diff -u $DEST/website_post.py \ $LILYPOND_GIT/scripts/build/website_post.py diff -u $DEST/bib2texi.py \ $LILYPOND_GIT/scripts/build/bib2texi.py diff -u $DEST/langdefs.py \ $LILYPOND_GIT/python/langdefs.py diff -u $DEST/lilypond.org.htaccess \ $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess diff -u $DEST/website-dir.htaccess \ $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess @end verbatim @end smallexample If the changes look ok, make them trusted: @smallexample @verbatim ### copy-from-git.sh #!/bin/sh DEST=$HOME/lilypond/trusted-scripts cp $LILYPOND_GIT/make/website.make \ $DEST/website.make cp $LILYPOND_GIT/Documentation/lilypond-texi2html.init \ $DEST/lilypond-texi2html.init cp $LILYPOND_GIT/scripts/build/extract_texi_filenames.py \ $DEST/extract_texi_filenames.py cp $LILYPOND_GIT/scripts/build/create-version-itexi.py \ $DEST/create-version-itexi.py cp $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py \ $DEST/create-weblinks-itexi.py cp $LILYPOND_GIT/scripts/build/mass-link.py \ $DEST/mass-link.py cp $LILYPOND_GIT/scripts/build/website_post.py \ $DEST/website_post.py cp $LILYPOND_GIT/scripts/build/bib2texi.py \ $DEST/bib2texi.py cp $LILYPOND_GIT/python/langdefs.py \ $DEST/langdefs.py cp $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess \ $DEST/lilypond.org.htaccess cp $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess \ $DEST/website-dir.htaccess @end verbatim @end smallexample Build the website: @smallexample @verbatim ### make-website.sh #!/bin/sh DEST=$HOME/web/ BUILD=$HOME/lilypond/build-website mkdir -p $BUILD cd $BUILD cp $HOME/lilypond/trusted-scripts/website.make . make -f website.make WEBSITE_ONLY_BUILD=1 website rsync -raO $BUILD/out-website/website/ $DEST/website/ cp $BUILD/out-website/pictures $DEST cp $BUILD/out-website/.htaccess $DEST @end verbatim @end smallexample Then in the @file{cronjob/} directory, put the cronjob to automate the trusted portions: @warning{@code{cron} will not inherit environment variables from your main setup, so you must re-define any variables inside your @code{crontab}.} @smallexample @verbatim # website-rebuild.cron LILYPOND_GIT= ... fill this in LILYPOND_WEB_MEDIA_GIT= ... fill this in 11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1 22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1 @end verbatim @end smallexample As the final stage of the setup, run your @code{copy-from-git.sh} script, assuming that you trust the current state of scripts in lilypond git. @subsubheading Normal maintenance When there is a change to the build scripts and/or website makefile, log in to the server (or your own home machine if you're testing this there), and do @example update-git.sh check-git.sh @end example After reviewing the changes carefully, you can update the trusted scripts with @code{copy-from-git.sh}. @subsubheading Building the website (exactly as on the server) Run @code{make-website.sh}; the final version ends up in @file{$HOME/web/}. On the actual server, the website is generated hourly by user @code{graham} the host @code{lilypond.org}. You can set up the cronjob by doing: @example crontab $HOME/lilypond/website-rebuild.cron @end example @subheading Initial setup for new users on actual serve You should symlink your own @file{~/lilypond/} to @file{~graham/lilypond/} If this directory does not exist, make it. Git master should go in @file{~/lilypond/lilypond-git/} but make sure you enable: @example git config core.filemode false @end example If you have created any files in @file{~graham/lilypond/} then please run: @example chgrp lilypond ~graham/lilypond/ -R chmod 775 ~graham/lilypond/ -R @end example @subsubheading Additional information Some information about the website is stored in @file{~graham/lilypond/*.txt}; this information should not be shared with people without trusted access to the server. @node Debugging website and docs locally @section Debugging website and docs locally @itemize @item Install apache2, or any other http server. These instructions assume that you also enable @code{mod_userdir}, and use @code{$HOME/public_html} as the location. @item Build the online docs and website: @example make WEB_TARGETS="offline online" doc make website @end example This will make all the language variants of the website. To save a little time, just the English version can be made with the command @code{make WEB_LANGS='' website} or the English and (for example) the French with @code{make WEB_LANGS='fr' website}. @item Move the built stuff into those directories. It's highly recommended to have your build dir and www dir on the same partition. (make @code{$HOME/public_html/} a symlink if necessary) @example mv out-website/website/ $HOME/public_html mv $HOME/public_html/website/pictures $HOME/public_html/ mkdir -p $HOME/public_html/doc/v2.13/ mv out-www/online-root/* $HOME/public_html/doc/v2.13/ @end example @end itemize @node Translating the website @section Translating the website As it has much more audience, the website should be translated before the documentation; see @ref{Translating the documentation}. In addition to the normal documentation translation practices, there are a few additional things to note: @itemize @item Build the website with: @example make website @end example @noindent however, please note that this command is not designed for being run multiple times. If you see unexpected output (mainly the page footers getting all messed up), then delete your @file{out-website} directory and run @code{make website} again. @item Some of the translation infrastructure is defined in python files; you must look at the @code{### translation data} sections in: @example scripts/build/create-weblinks-itexi.py scripts/build/website_post.py @end example @item Translations are not included by default in @code{make website}. To test your translation, edit the @code{WEB_LANGUAGES =} line in @file{python/langdefs.py}. You will need to copy this updated script to @code{$LILYPOND_BUILD_DIR/python/out}. Do not submit a patch to add your language to this file unless @code{make website} completes with fewer than 5 warnings. @item Links to manuals are done with macros like @code{@@manualDevelLearningSplit}. To get translated links, you must change that to @code{@@manualDevelLearningSplit-es} (for es/Spanish translations, for example). @end itemize