@section Introduction to website work
The website is @emph{not} written directly in HTML;
-instead, the source is Texinfo, which is then generated into HTML,
-PDF, and Info formats. The sources are
+instead it is autogenerated along with the documentation through a
+sophisticated setup, using Texinfo source files. Texinfo is the
+standard for documentation of GNU software and allows generating
+output in HTML, PDF, and Info formats, which drastically reduces
+maintenance effort and ensures that the website content is
+consistent with the rest of the documentation. This makes the
+environment for improving the website rather different from common
+web development.
+
+If you have not contributed to LilyPond before, a good starting
+point might be incremental changes to the CSS file, to be found at
+@uref{http://lilypond.org/css/lilypond-website.css} or in the
+LilyPond source code at @file{./Documentation/css/lilypond-website.css}.
+
+Large scale structural changes tend to require familiarity with
+the project in general, a track record in working on LilyPond
+documentation as well as a prospect of long-term commitment.
+
+The Texinfo source file for generating HTML are to be found in
@example
Documentation/web.texi
@node Uploading and security
@section Uploading and security
-@subheading Building the website from scratch for local checking
-
-Initial setup:
-
-Create directories:
-
-@example
-$HOME/lilypond/
-$HOME/lilypond/media/
-$HOME/lilypond/trusted-scripts/
-@end example
+@subheading Overall idea
To reduce the CPU burden on the shared host (as well as some
-security concerns), the 'Documentation/pictures/' and
-'Documentation/web/ly-examples/' directories are **not** compiled.
-We will do this ourselves right now.
-
-Go to your lilypond build directory. make doc.
-
-Set up some variables (you'll only do this once:)
-
-@example
-BUILD_DIR=$HOME/lilypond-git
-PICS=$BUILD_DIR/Documentation/pictures/
-EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/
-@end example
-
-Copy files over:
-
-@example
-cp -r $PICS $HOME/lilypond/media/pictures
-cp -r $EXAMPLES $HOME/lilypond/media/ly-examples
-@end example
+security concerns), we do not compile all of LilyPond. The
+website build process runs @code{texi2html}, but all media files
+(be they graphical @code{lilypond} output, photos of people, or
+pdfs) are copied from the @code{$LILYPOND_WEB_MEDIA_GIT}
+repository.
-Get the scripts you need. First define these variables:
+All scripts and makefiles used for the website build are run from
+a @qq{trusted} copy. Any modification to those files in git needs
+a human to review the changes (after they have been made in git)
+before they are used on the server.
-@example
-GIT=$HOME/lilypond-git
-DEST=$HOME/lilypond/trusted-scripts
-@end example
-
-Then do the copying:
-
-@smallexample
-cp $GIT/make/website.make $DEST/website.make
-cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
-cp $GIT/scripts/build/extract_texi_filenames.py $DEST/extract_texi_filenames.py
-cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
-cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
-cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
-cp $GIT/scripts/build/website_post.py $DEST/website_post.py
-cp $GIT/scripts/build/bib2texi.py $DEST/bib2texi.py
-cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
-cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess
-@end smallexample
-Delete your build directory (or maybe just rename your build
-directory to build-old).
+@subheading Building the website (quick local)
-@example
-cd $HOME/lilypond
-@end example
+Initial setup: make sure that you have the environment variables
+@code{$LILYPOND_GIT}, @code{$LILYPOND_BUILD_DIR} and
+@code{$LILYPOND_WEB_MEDIA_GIT} set up correctly. For more
+information, see @ref{Environment variables}.
-Run
+Once that is done,
@example
-make -f ../lilypond-git/make/website.make WEBSITE_ONLY_BUILD=1 \
- TOP_SRC_DIR=$HOME/lilypond-git/ \
- TEXI2HTML_PROGRAM=texi2html \
- website
-@end example
-
-The website should be at:
-
-@example
-$HOME/lilypond/out-website/website/index.html
+cd $LILYPOND_BUILD_DIR
+make website
@end example
-@subheading Building the online website
+The website is in @file{out-website/website/index.html}.
-The website is generated hourly by user @code{graham} the host
-@code{lilypond.org}. For security reasons, we do not use the
-makefiles and scripts directly from git; copies of the relevant
-scripts are examined and copied to
-@code{~graham/lilypond/trusted-scripts/}
-@subheading Initial setup
+@subheading Building the website (exactly as on the server)
-You should symlink your own @file{~/lilypond/} to
-@file{~graham/lilypond/}
+@subsubheading Setting up (exactly as on the server)
-If this directory does not exist, make it. Git master should go
-in @file{~/lilypond/lilypond-git/} but make sure you enable:
-
-@example
-git config core.filemode false
-@end example
+Initial setup: you still need @code{$LILYPOND_GIT} and
+@code{$LILYPOND_WEB_MEDIA_GIT}.
-If you have created any files in @file{~graham/lilypond/} then
-please run:
+Once that's done, create:
@example
-chgrp lilypond ~graham/lilypond/ -R
-chmod 775 ~graham/lilypond/ -R
+mkdir -p $HOME/lilypond/
+mkdir -p $HOME/lilypond/bin/
+mkdir -p $HOME/lilypond/cron/
+mkdir -p $HOME/lilypond/trusted-scripts/
@end example
-To reduce the CPU burden on the shared host (as well as some
-security concerns), the @file{Documentation/pictures/} and
-@file{Documentation/web/ly-examples/} directories are
-@strong{not} compiled. You need to upload them, and if they ever
-change, a user in the @code{lilypond} group must upload them to
-@file{~graham/lilypond/media} on the host.
-
-Upload latest pictures/ and ly-examples/ (local script):
+The add these files to @file{$HOME/lilypond/bin/}:
-@warning{You may need to change a number of items in the below
-script.}
-
-@verbatim
-### upload-lily-web-media.sh
-#!/bin/sh
-BUILD_DIR=$HOME/src/build-lilypond
-
-PICS=$BUILD_DIR/Documentation/pictures/out-www/
-EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/out-www/
-
-cd $BUILD_DIR
-rsync -a $PICS graham@lilypond.org:lilypond/media/pictures
-rsync -a $EXAMPLES graham@lilypond.org:lilypond/media/ly-examples
-@end verbatim
-
-
-
-@subheading Normal maintenance
-
-Get latest source code:
+Update git repositories:
+@smallexample
@verbatim
### update-git.sh
#!/bin/sh
-cd $HOME/lilypond/lilypond-git
+cd $LILYPOND_GIT
+git fetch origin
+git merge origin/master
+cd $LILYPOND_WEB_MEDIA_GIT
git fetch origin
git merge origin/master
@end verbatim
+@end smallexample
Check for any updates to trusted scripts / files:
+
@smallexample
@verbatim
### check-git.sh
#!/bin/sh
-GIT=$HOME/lilypond/lilypond-git
DEST=$HOME/lilypond/trusted-scripts
-diff -u $DEST/website.make $GIT/make/website.make
-diff -u $DEST/lilypond-texi2html.init $GIT/Documentation/lilypond-texi2html.init
-diff -u $DEST/extract_texi_filenames.py $GIT/scripts/build/extract_texi_filenames.py
-diff -u $DEST/create-version-itexi.py $GIT/scripts/build/create-version-itexi.py
-diff -u $DEST/create-weblinks-itexi.py $GIT/scripts/build/create-weblinks-itexi.py
-diff -u $DEST/mass-link.py $GIT/scripts/build/mass-link.py
-diff -u $DEST/website_post.py $GIT/scripts/build/website_post.py
-diff -u $DEST/bib2texi.py $GIT/scripts/build/bib2texi.py
-diff -u $DEST/lilypond.org.htaccess $GIT/Documentation/web/server/lilypond.org.htaccess
-diff -u $DEST/website-dir.htaccess $GIT/Documentation/web/server/website-dir.htaccess
+diff -u $DEST/website.make \
+ $LILYPOND_GIT/make/website.make
+diff -u $DEST/lilypond-texi2html.init \
+ $LILYPOND_GIT/Documentation/lilypond-texi2html.init
+diff -u $DEST/extract_texi_filenames.py \
+ $LILYPOND_GIT/scripts/build/extract_texi_filenames.py
+diff -u $DEST/create-version-itexi.py \
+ $LILYPOND_GIT/scripts/build/create-version-itexi.py
+diff -u $DEST/create-weblinks-itexi.py \
+ $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py
+diff -u $DEST/mass-link.py \
+ $LILYPOND_GIT/scripts/build/mass-link.py
+diff -u $DEST/website_post.py \
+ $LILYPOND_GIT/scripts/build/website_post.py
+diff -u $DEST/bib2texi.py \
+ $LILYPOND_GIT/scripts/build/bib2texi.py
+diff -u $DEST/langdefs.py \
+ $LILYPOND_GIT/python/langdefs.py
+diff -u $DEST/lilypond.org.htaccess \
+ $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess
+diff -u $DEST/website-dir.htaccess \
+ $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess
@end verbatim
+@end smallexample
If the changes look ok, make them trusted:
+@smallexample
@verbatim
### copy-from-git.sh
#!/bin/sh
-GIT=$HOME/lilypond/lilypond-git
DEST=$HOME/lilypond/trusted-scripts
-cp $GIT/make/website.make $DEST/website.make
-cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
-cp $GIT/scripts/build/extract_texi_filenames.py $DEST/extract_texi_filenames.py
-cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
-cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
-cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
-cp $GIT/scripts/build/website_post.py $DEST/website_post.py
-cp $GIT/scripts/build/bib2texi.py $DEST/bib2texi.py
-cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
-cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess
+cp $LILYPOND_GIT/make/website.make \
+ $DEST/website.make
+cp $LILYPOND_GIT/Documentation/lilypond-texi2html.init \
+ $DEST/lilypond-texi2html.init
+cp $LILYPOND_GIT/scripts/build/extract_texi_filenames.py \
+ $DEST/extract_texi_filenames.py
+cp $LILYPOND_GIT/scripts/build/create-version-itexi.py \
+ $DEST/create-version-itexi.py
+cp $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py \
+ $DEST/create-weblinks-itexi.py
+cp $LILYPOND_GIT/scripts/build/mass-link.py \
+ $DEST/mass-link.py
+cp $LILYPOND_GIT/scripts/build/website_post.py \
+ $DEST/website_post.py
+cp $LILYPOND_GIT/scripts/build/bib2texi.py \
+ $DEST/bib2texi.py
+cp $LILYPOND_GIT/python/langdefs.py \
+ $DEST/langdefs.py
+cp $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess \
+ $DEST/lilypond.org.htaccess
+cp $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess \
+ $DEST/website-dir.htaccess
@end verbatim
@end smallexample
Build the website:
+@smallexample
@verbatim
### make-website.sh
#!/bin/sh
cp $BUILD/out-website/pictures $DEST
cp $BUILD/out-website/.htaccess $DEST
@end verbatim
+@end smallexample
+
+Then in the @file{cronjob/} directory, put the cronjob to automate
+the trusted portions:
-Cronjob to automate the trusted portions:
+@warning{@code{cron} will not inherit environment variables from
+your main setup, so you must re-define any variables inside your
+@code{crontab}.}
+@smallexample
@verbatim
# website-rebuild.cron
+LILYPOND_GIT= ... fill this in
+LILYPOND_WEB_MEDIA_GIT= ... fill this in
+
11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1
22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1
@end verbatim
+@end smallexample
+
+As the final stage of the setup, run your @code{copy-from-git.sh}
+script, assuming that you trust the current state of scripts in
+lilypond git.
+
+@subsubheading Normal maintenance
+
+When there is a change to the build scripts and/or website
+makefile, log in to the server (or your own home machine if you're
+testing this there), and do
+
+@example
+update-git.sh
+check-git.sh
+@end example
+
+After reviewing the changes carefully, you can update the trusted
+scripts with @code{copy-from-git.sh}.
+
+
+@subsubheading Building the website (exactly as on the server)
+
+Run @code{make-website.sh}; the final version ends up in
+@file{$HOME/web/}.
+
+On the actual server, the website is generated hourly by user
+@code{graham} the host @code{lilypond.org}. You can set up the
+cronjob by doing:
+
+@example
+crontab $HOME/lilypond/website-rebuild.cron
+@end example
+
+
+@subheading Initial setup for new users on actual serve
+
+You should symlink your own @file{~/lilypond/} to
+@file{~graham/lilypond/}
+
+If this directory does not exist, make it. Git master should go
+in @file{~/lilypond/lilypond-git/} but make sure you enable:
+
+@example
+git config core.filemode false
+@end example
+
+If you have created any files in @file{~graham/lilypond/} then
+please run:
+
+@example
+chgrp lilypond ~graham/lilypond/ -R
+chmod 775 ~graham/lilypond/ -R
+@end example
+
@subsubheading Additional information
@itemize
@item
-Install apache2, or any other http server. These instructions
+Install Apache (you can use version 2, but keep in mind that the server
+hosting lilypond.org runs version 1.3). These instructions
assume that you also enable @code{mod_userdir}, and use
-@code{$HOME/public_html} as the location.
+@code{$HOME/public_html} as DocumentRoot (i.e. the root directory of
+the web server).
@item
Build the online docs and website:
make website
@end example
+This will make all the language variants of the website. To save
+a little time, just the English version can be made with the
+command @code{make WEB_LANGS='' website} or the English and (for
+example) the French with @code{make WEB_LANGS='fr' website}.
+
+@item
+Choose the web directory where to copy the built stuff.
+If you already have other web projects in your DocumentRoot and don't
+need to test the @file{.htaccess} file, you can copy to
+@code{~/public_html/lilypond.org}. Otherwise you'd better copy
+to @code{~/public_html}.
+It's highly recommended to have your build dir and web dir on the
+same partition.
@item
-Move the built stuff into those directories. It's highly
-recommended to have your build dir and www dir on the same
-partition. (make @code{$HOME/public_html/} a symlink if
-necessary)
+Add the directory for the online documentation:
@example
-mv out-website/website/ $HOME/public_html
-mv $HOME/public_html/website/pictures $HOME/public_html/
-mkdir -p $HOME/public_html/doc/v2.13/
-mv out-www/online-root/* $HOME/public_html/doc/v2.13/
+mkdir -p ~/public_html/doc/v2.19/
+@end example
+
+You may want to add also the stable documentation in
+@code{~/public_html/doc/v2.18/}, extracting the contents of the html
+directory present in the tarball available in @rweb{All}. Just in case
+you want to test the redirects to the stable documentation.
+
+@item
+Copy the files with rsync:
+
+@example
+rsync -av --delete out-website/website ~/public_html/
+cp out-website/.htaccess ~/public_html
+rsync -av --delete out-www/online-root/ ~/public_html/doc/v2.19/
@end example
@end itemize
@item
Translations are not included by default in @code{make website}.
-To test your translation, edit the @code{WEB_LANGS} line in
-@file{make/website.make}. Do not submit a patch to add your language
-to this file unless @code{make website} completes with less than 5
-warnings.
+To test your translation, edit the @code{WEB_LANGUAGES =} line in
+@file{python/langdefs.py}. You will need to copy this updated
+script to @code{$LILYPOND_BUILD_DIR/python/out}.
+
+Do not submit a patch to add your language to this file unless
+@code{make website} completes with fewer than 5 warnings.
@item
Links to manuals are done with macros like
@end itemize
-
-
projects / lilypond.git / blobdiff