CG: remove obsolete git info

[lilypond.git] / Documentation / contributor / website-work.itexi

diff --git a/Documentation/contributor/website-work.itexi b/Documentation/contributor/website-work.itexi
index 8617215b074ee4829141c191ffea8c0846016308..3136c1b144ca0864449d8ad3003b65df252dc548 100644 (file)
--- a/Documentation/contributor/website-work.itexi
+++ b/Documentation/contributor/website-work.itexi
@@ -3,8 +3,9 @@
 @chapter Website work
 
 @menu
 @chapter Website work
 
 @menu

-* Introduction to website work::  
+* Introduction to website work::

 * Uploading and security::
 * Uploading and security::

+* Debugging website and docs locally::

 * Translating the website::
 @end menu
 
 * Translating the website::
 @end menu
 


@@ -16,10 +17,10 @@ The website is @emph{not} written directly in HTML;
 instead, the source is Texinfo, which is then generated into HTML,
 PDF, and Info formats.  The sources are
 
 instead, the source is Texinfo, which is then generated into HTML,
 PDF, and Info formats.  The sources are
 

-@verbatim
+@example

 Documentation/web.texi
 Documentation/web/*.texi
 Documentation/web.texi
 Documentation/web/*.texi

-@end verbatim
+@end example

 
 Unless otherwise specified, follow the instructions and policies
 given in @ref{Documentation work}.  That chapter also contains a
 
 Unless otherwise specified, follow the instructions and policies
 given in @ref{Documentation work}.  That chapter also contains a


@@ -39,11 +40,17 @@ subsections or subsubsections.
 link to any pieces of automatically generated documentation, like
 the IR or certain NR appendices.
 
 link to any pieces of automatically generated documentation, like
 the IR or certain NR appendices.
 

+@item
+The bibliography in Community->Publications is generated automatically
+from @file{.bib} files; formatting is done automatically by
+@file{texi-web.bst}.
+
+@item

 @dots{}
 
 @item
 For anything not listed here, just follow the same style as the
 @dots{}
 
 @item
 For anything not listed here, just follow the same style as the

-existing texinfo files.
+existing website texinfo files.

 
 @end itemize
 
 
 @end itemize
 


@@ -51,111 +58,216 @@ existing texinfo files.
 @node Uploading and security
 @section Uploading and security
 
 @node Uploading and security
 @section Uploading and security
 

-The website is generated hourly by user @code{graham} the host
-@code{lilypond.org}.  For security reasons, we do not use the
-makefiles and scripts directly from git; copies of the relevant
-scripts are examined and copied to
-@code{~graham/lilypond/trusted-scripts/}
+@subheading Overall idea
+
+To reduce the CPU burden on the shared host (as well as some
+security concerns), we do not compile all of LilyPond.  The
+website build process runs @code{texi2html}, but all media files
+(be they graphical @code{lilypond} output, photos of people, or
+pdfs) are copied from the @code{$LILYPOND_WEB_MEDIA_GIT}
+repository.
+
+All scripts and makefiles used for the website build are run from
+a @qq{trusted} copy.  Any modification to those files in git needs
+a human to review the changes (after they have been made in git)
+before they are used on the server.
+
+
+@subheading Building the website (quick local)
+
+Initial setup: make sure that you have the environment variables
+@code{$LILYPOND_GIT}, @code{$LILYPOND_BUILD_DIR} and
+@code{$LILYPOND_WEB_MEDIA_GIT} set up correctly.  For more
+information, see @ref{Environment variables}.
+
+Once that is done,
+
+@example
+cd $LILYPOND_BUILD_DIR
+make website
+@end example
+
+The website is in @file{out-website/website/index.html}.
+
+
+@subheading Building the website (exactly as on the server)

 
 

-Get latest source code:
+@subsubheading Setting up (exactly as on the server)

 
 

+Initial setup: you still need @code{$LILYPOND_GIT} and
+@code{$LILYPOND_WEB_MEDIA_GIT}.
+
+Once that's done, create:
+
+@example
+mkdir -p $HOME/lilypond/
+mkdir -p $HOME/lilypond/bin/
+mkdir -p $HOME/lilypond/cron/
+mkdir -p $HOME/lilypond/trusted-scripts/
+@end example
+
+The add these files to @file{$HOME/lilypond/bin/}:
+
+Update git repositories:
+
+@smallexample

 @verbatim
 ### update-git.sh
 #!/bin/sh
 @verbatim
 ### update-git.sh
 #!/bin/sh

-cd $HOME/lilypond/lilypond-git
+cd $LILYPOND_GIT
+git fetch origin
+git merge origin/master
+cd $LILYPOND_WEB_MEDIA_GIT

 git fetch origin
 git merge origin/master
 @end verbatim
 git fetch origin
 git merge origin/master
 @end verbatim

+@end smallexample

 
 Check for any updates to trusted scripts / files:
 
 
 Check for any updates to trusted scripts / files:
 

+@smallexample

 @verbatim
 ### check-git.sh
 #!/bin/sh
 @verbatim
 ### check-git.sh
 #!/bin/sh

-GIT=$HOME/lilypond/lilypond-git

 DEST=$HOME/lilypond/trusted-scripts
 DEST=$HOME/lilypond/trusted-scripts

-diff -u $DEST/website.make $GIT/make/website.make
-diff -u $DEST/lilypond-texi2html.init $GIT/Documentation/lilypond-texi2html.init
-diff -u $DEST/extract_texi_filenames.py $GIT/scripts/build/extract_texi_filenames.py
-diff -u $DEST/create-version-itexi.py $GIT/scripts/build/create-version-itexi.py
-diff -u $DEST/create-weblinks-itexi.py $GIT/scripts/build/create-weblinks-itexi.py
-diff -u $DEST/mass-link.py $GIT/scripts/build/mass-link.py
-diff -u $DEST/website_post.py $GIT/scripts/build/website_post.py
-diff -u $DEST/lilypond.org.htaccess $GIT/Documentation/web/server/lilypond.org.htaccess
-diff -u $DEST/website-dir.htaccess $GIT/Documentation/web/server/website-dir.htaccess
+diff -u $DEST/website.make \
+  $LILYPOND_GIT/make/website.make
+diff -u $DEST/lilypond-texi2html.init \
+  $LILYPOND_GIT/Documentation/lilypond-texi2html.init
+diff -u $DEST/extract_texi_filenames.py \
+  $LILYPOND_GIT/scripts/build/extract_texi_filenames.py
+diff -u $DEST/create-version-itexi.py \
+  $LILYPOND_GIT/scripts/build/create-version-itexi.py
+diff -u $DEST/create-weblinks-itexi.py \
+  $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py
+diff -u $DEST/mass-link.py \
+  $LILYPOND_GIT/scripts/build/mass-link.py
+diff -u $DEST/website_post.py \
+  $LILYPOND_GIT/scripts/build/website_post.py
+diff -u $DEST/bib2texi.py \
+  $LILYPOND_GIT/scripts/build/bib2texi.py
+diff -u $DEST/langdefs.py \
+  $LILYPOND_GIT/python/langdefs.py
+diff -u $DEST/lilypond.org.htaccess \
+  $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess
+diff -u $DEST/website-dir.htaccess \
+  $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess

 @end verbatim
 @end verbatim

+@end smallexample

 
 If the changes look ok, make them trusted:
 
 
 If the changes look ok, make them trusted:
 

+@smallexample

 @verbatim
 ### copy-from-git.sh
 #!/bin/sh
 @verbatim
 ### copy-from-git.sh
 #!/bin/sh

-GIT=$HOME/lilypond/lilypond-git

 DEST=$HOME/lilypond/trusted-scripts
 DEST=$HOME/lilypond/trusted-scripts

-cp $GIT/make/website.make $DEST/website.make
-cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
-cp $GIT/scripts/build/extract_texi_filenames.py $DEST/extract_texi_filenames.py
-cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
-cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
-cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
-cp $GIT/scripts/build/website_post.py $DEST/website_post.py
-cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
-cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess
+cp $LILYPOND_GIT/make/website.make \
+  $DEST/website.make
+cp $LILYPOND_GIT/Documentation/lilypond-texi2html.init \
+  $DEST/lilypond-texi2html.init
+cp $LILYPOND_GIT/scripts/build/extract_texi_filenames.py \
+  $DEST/extract_texi_filenames.py
+cp $LILYPOND_GIT/scripts/build/create-version-itexi.py \
+  $DEST/create-version-itexi.py
+cp $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py \
+  $DEST/create-weblinks-itexi.py
+cp $LILYPOND_GIT/scripts/build/mass-link.py \
+  $DEST/mass-link.py
+cp $LILYPOND_GIT/scripts/build/website_post.py \
+  $DEST/website_post.py
+cp $LILYPOND_GIT/scripts/build/bib2texi.py \
+  $DEST/bib2texi.py
+cp $LILYPOND_GIT/python/langdefs.py \
+  $DEST/langdefs.py
+cp $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess \
+  $DEST/lilypond.org.htaccess
+cp $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess \
+  $DEST/website-dir.htaccess

 @end verbatim
 @end verbatim

+@end smallexample

 
 Build the website:
 
 
 Build the website:
 

+@smallexample

 @verbatim
 ### make-website.sh
 #!/bin/sh
 DEST=$HOME/web/
 @verbatim
 ### make-website.sh
 #!/bin/sh
 DEST=$HOME/web/

-mkdir -p $HOME/lilypond/build-website/
-cd $HOME/lilypond/build-website/
+BUILD=$HOME/lilypond/build-website
+mkdir -p $BUILD
+cd $BUILD

 cp $HOME/lilypond/trusted-scripts/website.make .
 
 make -f website.make WEBSITE_ONLY_BUILD=1 website
 cp $HOME/lilypond/trusted-scripts/website.make .
 
 make -f website.make WEBSITE_ONLY_BUILD=1 website

-rsync -raO $HOME/lilypond/build-website/out-website/website/ $DEST/website/
-cp $HOME/src/build-website/out-website/pictures $DEST
-cp $HOME/src/build-website/out-website/.htaccess $DEST
+rsync -raO $BUILD/out-website/website/ $DEST/website/
+cp $BUILD/out-website/pictures $DEST
+cp $BUILD/out-website/.htaccess $DEST

 @end verbatim
 @end verbatim

+@end smallexample

 
 

-Cronjob to automate the trusted portions:
+Then in the @file{cronjob/} directory, put the cronjob to automate
+the trusted portions:

 
 

+@warning{@code{cron} will not inherit environment variables from
+your main setup, so you must re-define any variables inside your
+@code{crontab}.}
+
+@smallexample

 @verbatim
 # website-rebuild.cron
 @verbatim
 # website-rebuild.cron

+LILYPOND_GIT=   ... fill this in
+LILYPOND_WEB_MEDIA_GIT=   ... fill this in
+

 11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1
 22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1
 @end verbatim
 11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1
 22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1
 @end verbatim

+@end smallexample

 
 

+As the final stage of the setup, run your @code{copy-from-git.sh}
+script, assuming that you trust the current state of scripts in
+lilypond git.

 
 

-To reduce the CPU burden on the shared host (as well as some
-security concerns), the @file{Documentation/pictures/} and
-@file{Documentation/web/ly-examples/} directories are @strong{not}
-compiled.  If you modify any files in those directories, a user in
-the @code{lilypond} group must upload them to @file{~graham/media}
-on the host.
+@subsubheading Normal maintenance

 
 

-Upload latest pictures/ and ly-examples/ (local script):
+When there is a change to the build scripts and/or website
+makefile, log in to the server (or your own home machine if you're
+testing this there), and do

 
 

-@verbatim
-### upload-lily-web-media.sh
-#!/bin/sh
-BUILD_DIR=$HOME/src/build-lilypond
+@example
+update-git.sh
+check-git.sh
+@end example

 
 

-PICS=$BUILD_DIR/Documentation/pictures/out-www/
-EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/out-www/
+After reviewing the changes carefully, you can update the trusted
+scripts with @code{copy-from-git.sh}.

 
 

-cd $BUILD_DIR
-rsync -a $PICS graham@lilypond.org:media/pictures
-rsync -a $EXAMPLES graham@lilypond.org:ly-examples
-@end verbatim

 
 

+@subsubheading Building the website (exactly as on the server)

 
 

-@subsubheading Additional information
+Run @code{make-website.sh}; the final version ends up in
+@file{$HOME/web/}.

 
 

-Some information about the website is stored in
-@file{~graham/lilypond/*.txt}; this information should not be
-shared with people without trusted access to the server.
+On the actual server, the website is generated hourly by user
+@code{graham} the host @code{lilypond.org}.  You can set up the
+cronjob by doing:
+
+@example
+crontab $HOME/lilypond/website-rebuild.cron
+@end example
+
+
+@subheading Initial setup for new users on actual serve
+
+You should symlink your own @file{~/lilypond/} to
+@file{~graham/lilypond/}
+
+If this directory does not exist, make it.  Git master should go
+in @file{~/lilypond/lilypond-git/} but make sure you enable:
+
+@example
+git config core.filemode false
+@end example

 
 If you have created any files in @file{~graham/lilypond/} then
 please run:
 
 If you have created any files in @file{~graham/lilypond/} then
 please run:


@@ -165,8 +277,51 @@ chgrp lilypond ~graham/lilypond/ -R
 chmod 775 ~graham/lilypond/ -R
 @end example
 
 chmod 775 ~graham/lilypond/ -R
 @end example
 

-You should symlink your own @file{~/lilypond/} to
-@file{~graham/lilypond/}
+
+
+@subsubheading Additional information
+
+Some information about the website is stored in
+@file{~graham/lilypond/*.txt}; this information should not be
+shared with people without trusted access to the server.
+
+
+@node Debugging website and docs locally
+@section Debugging website and docs locally
+
+@itemize
+@item
+Install apache2, or any other http server.  These instructions
+assume that you also enable @code{mod_userdir}, and use
+@code{$HOME/public_html} as the location.
+
+@item
+Build the online docs and website:
+
+@example
+make WEB_TARGETS="offline online" doc
+make website
+@end example
+
+This will make all the language variants of the website.  To save
+a little time, just the English version can be made with the
+command @code{make WEB_LANGS='' website} or the English and (for
+example) the French with @code{make WEB_LANGS='fr' website}.
+
+@item
+Move the built stuff into those directories.  It's highly
+recommended to have your build dir and www dir on the same
+partition.  (make @code{$HOME/public_html/} a symlink if
+necessary)
+
+@example
+mv out-website/website/ $HOME/public_html
+mv $HOME/public_html/website/pictures $HOME/public_html/
+mkdir -p $HOME/public_html/doc/v2.13/
+mv out-www/online-root/* $HOME/public_html/doc/v2.13/
+@end example
+
+@end itemize

 
 
 @node Translating the website
 
 
 @node Translating the website


@@ -204,10 +359,12 @@ scripts/build/website_post.py
 
 @item
 Translations are not included by default in @code{make website}.
 
 @item
 Translations are not included by default in @code{make website}.

-To test your translation, edit the @code{WEB_LANGS} line in
-@file{make/website.make}.  Do not submit a patch to add your language
-to this file unless @code{make website} completes with less than 5
-warnings.
+To test your translation, edit the @code{WEB_LANGUAGES =} line in
+@file{python/langdefs.py}.  You will need to copy this updated
+script to @code{$LILYPOND_BUILD_DIR/python/out}.
+
+Do not submit a patch to add your language to this file unless
+@code{make website} completes with fewer than 5 warnings.

 
 @item
 Links to manuals are done with macros like
 
 @item
 Links to manuals are done with macros like