1 @c -*- coding: utf-8; mode: texinfo; -*-
6 * Introduction to website work::
7 * Uploading and security::
8 * Debugging website and docs locally::
9 * Translating the website::
13 @node Introduction to website work
14 @section Introduction to website work
16 The website is @emph{not} written directly in HTML;
17 instead it is autogenerated along with the documentation through a
18 sophisticated setup, using Texinfo source files. Texinfo is the
19 standard for documentation of GNU software and allows generating
20 output in HTML, PDF, and Info formats, which drastically reduces
21 maintenance effort and ensures that the website content is
22 consistent with the rest of the documentation. This makes the
23 environment for improving the website rather different from common
26 If you have not contributed to LilyPond before, a good starting
27 point might be incremental changes to the CSS file, to be found at
28 @uref{http://lilypond.org/css/lilypond-website.css} or in the
29 LilyPond source code at @file{./Documentation/css/lilypond-website.css}.
31 Large scale structural changes tend to require familiarity with
32 the project in general, a track record in working on LilyPond
33 documentation as well as a prospect of long-term commitment.
35 The Texinfo source file for generating HTML are to be found in
38 Documentation/web.texi
39 Documentation/web/*.texi
42 Unless otherwise specified, follow the instructions and policies
43 given in @ref{Documentation work}. That chapter also contains a
44 quick introduction to Texinfo; consulting an external Texinfo
45 manual should be not necessary.
47 @subheading Exceptions to the documentation policies
52 Sectioning: the website only uses chapters and sections; no
53 subsections or subsubsections.
56 @@ref@{@}s to other manuals (@@ruser, @@rlearning, etc): you can't
57 link to any pieces of automatically generated documentation, like
58 the IR or certain NR appendices.
61 The bibliography in Community->Publications is generated automatically
62 from @file{.bib} files; formatting is done automatically by
69 For anything not listed here, just follow the same style as the
70 existing website texinfo files.
75 @node Uploading and security
76 @section Uploading and security
78 @subheading Overall idea
80 To reduce the CPU burden on the shared host (as well as some
81 security concerns), we do not compile all of LilyPond. The
82 website build process runs @code{texi2html}, but all media files
83 (be they graphical @code{lilypond} output, photos of people, or
84 pdfs) are copied from the @code{$LILYPOND_WEB_MEDIA_GIT}
87 All scripts and makefiles used for the website build are run from
88 a @qq{trusted} copy. Any modification to those files in git needs
89 a human to review the changes (after they have been made in git)
90 before they are used on the server.
93 @subheading Building the website (quick local)
95 Initial setup: make sure that you have the environment variables
96 @code{$LILYPOND_GIT}, @code{$LILYPOND_BUILD_DIR} and
97 @code{$LILYPOND_WEB_MEDIA_GIT} set up correctly. For more
98 information, see @ref{Environment variables}.
103 cd $LILYPOND_BUILD_DIR
107 The website is in @file{out-website/website/index.html}.
110 @subheading Building the website (exactly as on the server)
112 @subsubheading Setting up (exactly as on the server)
114 Initial setup: you still need @code{$LILYPOND_GIT} and
115 @code{$LILYPOND_WEB_MEDIA_GIT}.
117 Once that's done, create:
120 mkdir -p $HOME/lilypond/
121 mkdir -p $HOME/lilypond/bin/
122 mkdir -p $HOME/lilypond/cron/
123 mkdir -p $HOME/lilypond/trusted-scripts/
126 The add these files to @file{$HOME/lilypond/bin/}:
128 Update git repositories:
136 git merge origin/master
137 cd $LILYPOND_WEB_MEDIA_GIT
139 git merge origin/master
143 Check for any updates to trusted scripts / files:
149 DEST=$HOME/lilypond/trusted-scripts
150 diff -u $DEST/website.make \
151 $LILYPOND_GIT/make/website.make
152 diff -u $DEST/lilypond-texi2html.init \
153 $LILYPOND_GIT/Documentation/lilypond-texi2html.init
154 diff -u $DEST/extract_texi_filenames.py \
155 $LILYPOND_GIT/scripts/build/extract_texi_filenames.py
156 diff -u $DEST/create-version-itexi.py \
157 $LILYPOND_GIT/scripts/build/create-version-itexi.py
158 diff -u $DEST/create-weblinks-itexi.py \
159 $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py
160 diff -u $DEST/mass-link.py \
161 $LILYPOND_GIT/scripts/build/mass-link.py
162 diff -u $DEST/website_post.py \
163 $LILYPOND_GIT/scripts/build/website_post.py
164 diff -u $DEST/bib2texi.py \
165 $LILYPOND_GIT/scripts/build/bib2texi.py
166 diff -u $DEST/langdefs.py \
167 $LILYPOND_GIT/python/langdefs.py
168 diff -u $DEST/lilypond.org.htaccess \
169 $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess
170 diff -u $DEST/website-dir.htaccess \
171 $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess
175 If the changes look ok, make them trusted:
181 DEST=$HOME/lilypond/trusted-scripts
182 cp $LILYPOND_GIT/make/website.make \
184 cp $LILYPOND_GIT/Documentation/lilypond-texi2html.init \
185 $DEST/lilypond-texi2html.init
186 cp $LILYPOND_GIT/scripts/build/extract_texi_filenames.py \
187 $DEST/extract_texi_filenames.py
188 cp $LILYPOND_GIT/scripts/build/create-version-itexi.py \
189 $DEST/create-version-itexi.py
190 cp $LILYPOND_GIT/scripts/build/create-weblinks-itexi.py \
191 $DEST/create-weblinks-itexi.py
192 cp $LILYPOND_GIT/scripts/build/mass-link.py \
194 cp $LILYPOND_GIT/scripts/build/website_post.py \
195 $DEST/website_post.py
196 cp $LILYPOND_GIT/scripts/build/bib2texi.py \
198 cp $LILYPOND_GIT/python/langdefs.py \
200 cp $LILYPOND_GIT/Documentation/web/server/lilypond.org.htaccess \
201 $DEST/lilypond.org.htaccess
202 cp $LILYPOND_GIT/Documentation/web/server/website-dir.htaccess \
203 $DEST/website-dir.htaccess
214 BUILD=$HOME/lilypond/build-website
217 cp $HOME/lilypond/trusted-scripts/website.make .
219 make -f website.make WEBSITE_ONLY_BUILD=1 website
220 rsync -raO $BUILD/out-website/website/ $DEST/website/
221 cp $BUILD/out-website/pictures $DEST
222 cp $BUILD/out-website/.htaccess $DEST
226 Then in the @file{cronjob/} directory, put the cronjob to automate
227 the trusted portions:
229 @warning{@code{cron} will not inherit environment variables from
230 your main setup, so you must re-define any variables inside your
235 # website-rebuild.cron
236 LILYPOND_GIT= ... fill this in
237 LILYPOND_WEB_MEDIA_GIT= ... fill this in
239 11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1
240 22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1
244 As the final stage of the setup, run your @code{copy-from-git.sh}
245 script, assuming that you trust the current state of scripts in
248 @subsubheading Normal maintenance
250 When there is a change to the build scripts and/or website
251 makefile, log in to the server (or your own home machine if you're
252 testing this there), and do
259 After reviewing the changes carefully, you can update the trusted
260 scripts with @code{copy-from-git.sh}.
263 @subsubheading Building the website (exactly as on the server)
265 Run @code{make-website.sh}; the final version ends up in
268 On the actual server, the website is generated hourly by user
269 @code{graham} the host @code{lilypond.org}. You can set up the
273 crontab $HOME/lilypond/website-rebuild.cron
277 @subheading Initial setup for new users on actual serve
279 You should symlink your own @file{~/lilypond/} to
280 @file{~graham/lilypond/}
282 If this directory does not exist, make it. Git master should go
283 in @file{~/lilypond/lilypond-git/} but make sure you enable:
286 git config core.filemode false
289 If you have created any files in @file{~graham/lilypond/} then
293 chgrp lilypond ~graham/lilypond/ -R
294 chmod 775 ~graham/lilypond/ -R
299 @subsubheading Additional information
301 Some information about the website is stored in
302 @file{~graham/lilypond/*.txt}; this information should not be
303 shared with people without trusted access to the server.
306 @node Debugging website and docs locally
307 @section Debugging website and docs locally
311 Install Apache (you can use version 2, but keep in mind that the server
312 hosting lilypond.org runs version 1.3). These instructions
313 assume that you also enable @code{mod_userdir}, and use
314 @code{$HOME/public_html} as DocumentRoot (i.e. the root directory of
318 Build the online docs and website:
321 make WEB_TARGETS="offline online" doc
325 This will make all the language variants of the website. To save
326 a little time, just the English version can be made with the
327 command @code{make WEB_LANGS='' website} or the English and (for
328 example) the French with @code{make WEB_LANGS='fr' website}.
331 Choose the web directory where to copy the built stuff.
332 If you already have other web projects in your DocumentRoot and don't
333 need to test the @file{.htaccess} file, you can copy to
334 @code{~/public_html/lilypond.org}. Otherwise you'd better copy
335 to @code{~/public_html}.
336 It's highly recommended to have your build dir and web dir on the
340 Add the directory for the online documentation:
343 mkdir -p ~/public_html/doc/v2.19/
346 You may want to add also the stable documentation in
347 @code{~/public_html/doc/v2.18/}, extracting the contents of the html
348 directory present in the tarball available in @rweb{All}. Just in case
349 you want to test the redirects to the stable documentation.
352 Copy the files with rsync:
355 rsync -av --delete out-website/website ~/public_html/
356 cp out-website/.htaccess ~/public_html
357 rsync -av --delete out-www/online-root/ ~/public_html/doc/v2.19/
363 @node Translating the website
364 @section Translating the website
366 As it has much more audience, the website should be translated before
367 the documentation; see @ref{Translating the documentation}.
369 In addition to the normal documentation translation practices,
370 there are a few additional things to note:
375 Build the website with:
382 however, please note that this command is not designed for being
383 run multiple times. If you see unexpected output (mainly the page
384 footers getting all messed up), then delete your
385 @file{out-website} directory and run @code{make website} again.
388 Some of the translation infrastructure is defined in python files;
389 you must look at the @code{### translation data} sections in:
392 scripts/build/create-weblinks-itexi.py
393 scripts/build/website_post.py
397 Translations are not included by default in @code{make website}.
398 To test your translation, edit the @code{WEB_LANGUAGES =} line in
399 @file{python/langdefs.py}. You will need to copy this updated
400 script to @code{$LILYPOND_BUILD_DIR/python/out}.
402 Do not submit a patch to add your language to this file unless
403 @code{make website} completes with fewer than 5 warnings.
406 Links to manuals are done with macros like
407 @code{@@manualDevelLearningSplit}. To get translated links, you
408 must change that to @code{@@manualDevelLearningSplit-es} (for
409 es/Spanish translations, for example).