X-Git-Url: https://git.donarmstrong.com/?a=blobdiff_plain;f=Documentation%2Fcontributor%2Fwebsite-work.itexi;h=d719843b53e3e1e0d58b9c1b326b47cdb24d44b7;hb=ac569ff38c26fb3aaf1bc16d0e213589bdc4fa8f;hp=86892000b41dd9abb7aec1872cee6a4a4aad0645;hpb=ef3f586830c85fc9e6bf0f82917eecef0258b07e;p=lilypond.git diff --git a/Documentation/contributor/website-work.itexi b/Documentation/contributor/website-work.itexi index 86892000b4..d719843b53 100644 --- a/Documentation/contributor/website-work.itexi +++ b/Documentation/contributor/website-work.itexi @@ -1,10 +1,11 @@ -@c -*- coding: us-ascii; mode: texinfo; -*- +@c -*- coding: utf-8; mode: texinfo; -*- @node Website work @chapter Website work @menu -* Introduction to website work:: +* Introduction to website work:: * Uploading and security:: +* Debugging website and docs locally:: * Translating the website:: @end menu @@ -16,10 +17,10 @@ The website is @emph{not} written directly in HTML; instead, the source is Texinfo, which is then generated into HTML, PDF, and Info formats. The sources are -@verbatim +@example Documentation/web.texi Documentation/web/*.texi -@end verbatim +@end example Unless otherwise specified, follow the instructions and policies given in @ref{Documentation work}. That chapter also contains a @@ -39,11 +40,17 @@ subsections or subsubsections. link to any pieces of automatically generated documentation, like the IR or certain NR appendices. +@item +The bibliography in Community->Publications is generated automatically +from @file{.bib} files; formatting is done automatically by +@file{texi-web.bst}. + +@item @dots{} @item For anything not listed here, just follow the same style as the -existing texinfo files. +existing website texinfo files. @end itemize @@ -51,33 +58,318 @@ existing texinfo files. @node Uploading and security @section Uploading and security -FIXME: currently these are plans, not yet completely implemented. -Hey, I'm doing the reponsible "write documentation first, then -code" thing! I rock! -gp +@subheading Overall idea -The website is generated hourly by user @code{graham} the host -@code{lilypond.org}. For security reasons, we do not use the -makefiles and scripts directly from git; copies of the relevant -scripts (below) are examined and copied to -@file{~graham/trusted-scripts} on the host. +To reduce the CPU burden on the shared host (as well as some +security concerns), we do not compile all of LilyPond. The +website build process runs @code{texi2html}, but all media files +(be they graphical @code{lilypond} output, photos of people, or +pdfs) are copied from the @code{$LILYPOND_WEB_MEDIA_GIT} +repository. + +All scripts and makefiles used for the website build are run from +a @qq{trusted} copy. Any modification to those files in git needs +a human to review the changes (after they have been made in git) +before they are used on the server. + + +@subheading Building the website (quick local) + +Initial setup: make sure that you have the environment variables +@code{$LILYPOND_GIT}, @code{$LILYPOND_BUILD_DIR} and +@code{$LILYPOND_WEB_MEDIA_GIT} set up correctly. For more +information, see @ref{Environment variables}. + +Once that is done, @example -website.make -scripts/build/extract_texi_filenames.py +cd $LILYPOND_BUILD_DIR +make website +@end example + +The website is in @file{out-website/website/index.html}. + + +@subheading Building the website (exactly as on the server) + +@subsubheading Setting up (exactly as on the server) + +Initial setup: you still need @code{$LILYPOND_GIT} and +@code{$LILYPOND_WEB_MEDIA_GIT}. + +Once that's done, create: + +@example +mkdir -p $HOME/lilypond/ +mkdir -p $HOME/lilypond/bin/ +mkdir -p $HOME/lilypond/cron/ +mkdir -p $HOME/lilypond/trusted-scripts/ +@end example + +The add these files to @file{$HOME/lilypond/bin/}: + +Update git repositories: + +@smallexample +@verbatim +### update-git.sh +#!/bin/sh +cd $LILYPOND_GIT +git fetch origin +git merge origin/master +cd $LILYPOND_WEB_MEDIA_GIT +git fetch origin +git merge origin/master +@end verbatim +@end smallexample + +Check for any updates to trusted scripts / files: + +@smallexample +@verbatim +### check-git.sh +#!/bin/sh +DEST=$HOME/lilypond/trusted-scripts +diff -u $DEST/website.make \ + $LILYPOND_GIT/make/website.make +diff -u $DEST/lilypond-texi2html.init \ + $LILYPOND_GIT/Documentation/lilypond-texi2html.init +diff -u $DEST/extract_texi_filenames.py \ + $LILYPOND_GIT/scripts/build/extract_texi_filenames.py +diff -u $DEST/create-version-itexi.py \ + $LILYPOND_GIT/scripts/build/create-version-itexi.py +diff -u $DEST/create-weblinks-itexi.py \ + $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py +diff -u $DEST/mass-link.py \ + $LILYPOND_GIT/scripts/build/mass-link.py +diff -u $DEST/website_post.py \ + $LILYPOND_GIT/scripts/build/website_post.py +diff -u $DEST/bib2texi.py \ + $LILYPOND_GIT/scripts/build/bib2texi.py +diff -u $DEST/langdefs.py \ + $LILYPOND_GIT/python/langdefs.py +diff -u $DEST/lilypond.org.htaccess \ + $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess +diff -u $DEST/website-dir.htaccess \ + $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess +@end verbatim +@end smallexample + +If the changes look ok, make them trusted: + +@smallexample +@verbatim +### copy-from-git.sh +#!/bin/sh +DEST=$HOME/lilypond/trusted-scripts +cp $LILYPOND_GIT/make/website.make \ + $DEST/website.make +cp $LILYPOND_GIT/Documentation/lilypond-texi2html.init \ + $DEST/lilypond-texi2html.init +cp $LILYPOND_GIT/scripts/build/extract_texi_filenames.py \ + $DEST/extract_texi_filenames.py +cp $LILYPOND_GIT/scripts/build/create-version-itexi.py \ + $DEST/create-version-itexi.py +cp $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py \ + $DEST/create-weblinks-itexi.py +cp $LILYPOND_GIT/scripts/build/mass-link.py \ + $DEST/mass-link.py +cp $LILYPOND_GIT/scripts/build/website_post.py \ + $DEST/website_post.py +cp $LILYPOND_GIT/scripts/build/bib2texi.py \ + $DEST/bib2texi.py +cp $LILYPOND_GIT/python/langdefs.py \ + $DEST/langdefs.py +cp $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess \ + $DEST/lilypond.org.htaccess +cp $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess \ + $DEST/website-dir.htaccess +@end verbatim +@end smallexample + +Build the website: + +@smallexample +@verbatim +### make-website.sh +#!/bin/sh +DEST=$HOME/web/ +BUILD=$HOME/lilypond/build-website +mkdir -p $BUILD +cd $BUILD +cp $HOME/lilypond/trusted-scripts/website.make . + +make -f website.make WEBSITE_ONLY_BUILD=1 website +rsync -raO $BUILD/out-website/website/ $DEST/website/ +cp $BUILD/out-website/pictures $DEST +cp $BUILD/out-website/.htaccess $DEST +@end verbatim +@end smallexample + +Then in the @file{cronjob/} directory, put the cronjob to automate +the trusted portions: + +@warning{@code{cron} will not inherit environment variables from +your main setup, so you must re-define any variables inside your +@code{crontab}.} + +@smallexample +@verbatim +# website-rebuild.cron +LILYPOND_GIT= ... fill this in +LILYPOND_WEB_MEDIA_GIT= ... fill this in + +11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1 +22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1 +@end verbatim +@end smallexample + +As the final stage of the setup, run your @code{copy-from-git.sh} +script, assuming that you trust the current state of scripts in +lilypond git. + +@subsubheading Normal maintenance + +When there is a change to the build scripts and/or website +makefile, log in to the server (or your own home machine if you're +testing this there), and do + +@example +update-git.sh +check-git.sh +@end example + +After reviewing the changes carefully, you can update the trusted +scripts with @code{copy-from-git.sh}. + + +@subsubheading Building the website (exactly as on the server) + +Run @code{make-website.sh}; the final version ends up in +@file{$HOME/web/}. + +On the actual server, the website is generated hourly by user +@code{graham} the host @code{lilypond.org}. You can set up the +cronjob by doing: + +@example +crontab $HOME/lilypond/website-rebuild.cron +@end example + + +@subheading Initial setup for new users on actual serve + +You should symlink your own @file{~/lilypond/} to +@file{~graham/lilypond/} + +If this directory does not exist, make it. Git master should go +in @file{~/lilypond/lilypond-git/} but make sure you enable: + +@example +git config core.filemode false +@end example + +If you have created any files in @file{~graham/lilypond/} then +please run: + +@example +chgrp lilypond ~graham/lilypond/ -R +chmod 775 ~graham/lilypond/ -R @end example -To reduce the CPU burden on the shared host (as well as some -security concerns), the @file{Documentation/pictures/} and -@file{Documentation/web/ly-examples/} directories are @strong{not} -compiled. If you modify any files in those directories, a user in -the @code{lilypond} group must upload them to @file{~graham/media} -on the host. This is done by running (locally) @code{make doc}, -followed by @code{scripts/build/upload-web-media.sh}. +@subsubheading Additional information + +Some information about the website is stored in +@file{~graham/lilypond/*.txt}; this information should not be +shared with people without trusted access to the server. + + +@node Debugging website and docs locally +@section Debugging website and docs locally + +@itemize +@item +Install apache2, or any other http server. These instructions +assume that you also enable @code{mod_userdir}, and use +@code{$HOME/public_html/lilypond.org} as the location. + +@item +Build the online docs and website: + +@example +make WEB_TARGETS="offline online" doc +make website +@end example + +This will make all the language variants of the website. To save +a little time, just the English version can be made with the +command @code{make WEB_LANGS='' website} or the English and (for +example) the French with @code{make WEB_LANGS='fr' website}. + +@item +Move the built stuff into those directories. It's highly +recommended to have your build dir and www dir on the same +partition. (make @code{$HOME/public_html/lilypond.org} a symlink +if necessary) + +@example +mv out-website/website $HOME/public_html/lilypond.org/ +mkdir -p $HOME/public_html/lilypond.org/doc/v2.19/ +mv out-www/online-root/* $HOME/public_html/lilypond.org/doc/v2.19/ +@end example + +@end itemize + @node Translating the website @section Translating the website As it has much more audience, the website should be translated before the documentation; see @ref{Translating the documentation}. + +In addition to the normal documentation translation practices, +there are a few additional things to note: + +@itemize + +@item +Build the website with: + +@example +make website +@end example + +@noindent +however, please note that this command is not designed for being +run multiple times. If you see unexpected output (mainly the page +footers getting all messed up), then delete your +@file{out-website} directory and run @code{make website} again. + +@item +Some of the translation infrastructure is defined in python files; +you must look at the @code{### translation data} sections in: + +@example +scripts/build/create-weblinks-itexi.py +scripts/build/website_post.py +@end example + +@item +Translations are not included by default in @code{make website}. +To test your translation, edit the @code{WEB_LANGUAGES =} line in +@file{python/langdefs.py}. You will need to copy this updated +script to @code{$LILYPOND_BUILD_DIR/python/out}. + +Do not submit a patch to add your language to this file unless +@code{make website} completes with fewer than 5 warnings. + +@item +Links to manuals are done with macros like +@code{@@manualDevelLearningSplit}. To get translated links, you +must change that to @code{@@manualDevelLearningSplit-es} (for +es/Spanish translations, for example). + +@end itemize +