Documentation/contributor/website-work.itexi

   1 @c -*- coding: utf-8; mode: texinfo; -*-
   2 @node Website work
   3 @chapter Website work
   4
   5 @menu
   6 * Introduction to website work::
   7 * Uploading and security::
   8 * Debugging website and docs locally::
   9 * Translating the website::
  10 @end menu
  11
  12
  13 @node Introduction to website work
  14 @section Introduction to website work
  15
  16 The website is @emph{not} written directly in HTML;
  17 instead, the source is Texinfo, which is then generated into HTML,
  18 PDF, and Info formats.  The sources are
  19
  20 @example
  21 Documentation/web.texi
  22 Documentation/web/*.texi
  23 @end example
  24
  25 Unless otherwise specified, follow the instructions and policies
  26 given in @ref{Documentation work}.  That chapter also contains a
  27 quick introduction to Texinfo; consulting an external Texinfo
  28 manual should be not necessary.
  29
  30 @subheading Exceptions to the documentation policies
  31
  32 @itemize
  33
  34 @item
  35 Sectioning: the website only uses chapters and sections; no
  36 subsections or subsubsections.
  37
  38 @item
  39 @@ref@{@}s to other manuals (@@ruser, @@rlearning, etc): you can't
  40 link to any pieces of automatically generated documentation, like
  41 the IR or certain NR appendices.
  42
  43 @item
  44 The bibliography in Community->Publications is generated automatically
  45 from @file{.bib} files; formatting is done automatically by
  46 @file{texi-web.bst}.
  47
  48 @item
  49 @dots{}
  50
  51 @item
  52 For anything not listed here, just follow the same style as the
  53 existing website texinfo files.
  54
  55 @end itemize
  56
  57
  58 @node Uploading and security
  59 @section Uploading and security
  60
  61 @subheading Building the website from scratch for local checking
  62
  63 Initial setup:
  64
  65 Create directories:
  66
  67 @example
  68 $HOME/lilypond/
  69 $HOME/lilypond/media/
  70 $HOME/lilypond/trusted-scripts/
  71 @end example
  72
  73 To reduce the CPU burden on the shared host (as well as some
  74 security concerns), the 'Documentation/pictures/' and
  75 'Documentation/web/ly-examples/' directories are **not** compiled.
  76 We will do this ourselves right now.
  77
  78 Go to your lilypond build directory.  make doc.
  79
  80 Set up some variables (you'll only do this once:)
  81
  82 @example
  83 BUILD_DIR=$HOME/lilypond-git
  84 PICS=$BUILD_DIR/Documentation/pictures/
  85 EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/
  86 @end example
  87
  88 Copy files over:
  89
  90 @example
  91 cp -r $PICS $HOME/lilypond/media/pictures
  92 cp -r $EXAMPLES $HOME/lilypond/media/ly-examples
  93 @end example
  94
  95 Get the scripts you need.  First define these variables:
  96
  97 @example
  98 GIT=$HOME/lilypond-git
  99 DEST=$HOME/lilypond/trusted-scripts
 100 @end example
 101
 102 Then do the copying:
 103
 104 @smallexample
 105 cp $GIT/make/website.make $DEST/website.make
 106 cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
 107 cp $GIT/scripts/build/extract_texi_filenames.py  $DEST/extract_texi_filenames.py
 108 cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
 109 cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
 110 cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
 111 cp $GIT/scripts/build/website_post.py $DEST/website_post.py
 112 cp $GIT/scripts/build/bib2texi.py $DEST/bib2texi.py
 113 cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
 114 cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess
 115 @end smallexample
 116
 117 Delete your build directory (or maybe just rename your build
 118 directory to build-old).
 119
 120 @example
 121 cd $HOME/lilypond
 122 @end example
 123
 124 Run
 125
 126 @example
 127 make -f ../lilypond-git/make/website.make WEBSITE_ONLY_BUILD=1 \
 128   TOP_SRC_DIR=$HOME/lilypond-git/ \
 129   PYTHONPATH=$HOME/lilypond-git/python \
 130   TEXI2HTML_PROGRAM=texi2html \
 131   website
 132 @end example
 133
 134 The website should be at:
 135
 136 @example
 137 $HOME/lilypond/out-website/website/index.html
 138 @end example
 139
 140 @subheading Building the online website
 141
 142 The website is generated hourly by user @code{graham} the host
 143 @code{lilypond.org}.  For security reasons, we do not use the
 144 makefiles and scripts directly from git; copies of the relevant
 145 scripts are examined and copied to
 146 @code{~graham/lilypond/trusted-scripts/}
 147
 148 @subheading Initial setup
 149
 150 You should symlink your own @file{~/lilypond/} to
 151 @file{~graham/lilypond/}
 152
 153 If this directory does not exist, make it.  Git master should go
 154 in @file{~/lilypond/lilypond-git/} but make sure you enable:
 155
 156 @example
 157 git config core.filemode false
 158 @end example
 159
 160 If you have created any files in @file{~graham/lilypond/} then
 161 please run:
 162
 163 @example
 164 chgrp lilypond ~graham/lilypond/ -R
 165 chmod 775 ~graham/lilypond/ -R
 166 @end example
 167
 168 To reduce the CPU burden on the shared host (as well as some
 169 security concerns), the @file{Documentation/pictures/} and
 170 @file{Documentation/web/ly-examples/} directories are
 171 @strong{not} compiled.  You need to upload them, and if they ever
 172 change, a user in the @code{lilypond} group must upload them to
 173 @file{~graham/lilypond/media} on the host.
 174
 175 Upload latest pictures/ and ly-examples/ (local script):
 176
 177 @warning{You may need to change a number of items in the below
 178 script.}
 179
 180 @verbatim
 181 ### upload-lily-web-media.sh
 182 #!/bin/sh
 183 BUILD_DIR=$HOME/src/build-lilypond
 184
 185 PICS=$BUILD_DIR/Documentation/pictures/out-www/
 186 EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/out-www/
 187
 188 cd $BUILD_DIR
 189 rsync -a $PICS graham@lilypond.org:lilypond/media/pictures
 190 rsync -a $EXAMPLES graham@lilypond.org:lilypond/media/ly-examples
 191 @end verbatim
 192
 193
 194
 195 @subheading Normal maintenance
 196
 197 Get latest source code:
 198
 199 @verbatim
 200 ### update-git.sh
 201 #!/bin/sh
 202 cd $HOME/lilypond/lilypond-git
 203 git fetch origin
 204 git merge origin/master
 205 @end verbatim
 206
 207 Check for any updates to trusted scripts / files:
 208 @smallexample
 209 @verbatim
 210 ### check-git.sh
 211 #!/bin/sh
 212 GIT=$HOME/lilypond/lilypond-git
 213 DEST=$HOME/lilypond/trusted-scripts
 214 diff -u $DEST/website.make $GIT/make/website.make
 215 diff -u $DEST/lilypond-texi2html.init $GIT/Documentation/lilypond-texi2html.init
 216 diff -u $DEST/extract_texi_filenames.py $GIT/scripts/build/extract_texi_filenames.py
 217 diff -u $DEST/create-version-itexi.py $GIT/scripts/build/create-version-itexi.py
 218 diff -u $DEST/create-weblinks-itexi.py $GIT/scripts/build/create-weblinks-itexi.py
 219 diff -u $DEST/mass-link.py $GIT/scripts/build/mass-link.py
 220 diff -u $DEST/website_post.py $GIT/scripts/build/website_post.py
 221 diff -u $DEST/bib2texi.py $GIT/scripts/build/bib2texi.py
 222 diff -u $DEST/langdefs.py $GIT/python/langdefs.py
 223 diff -u $DEST/lilypond.org.htaccess $GIT/Documentation/web/server/lilypond.org.htaccess
 224 diff -u $DEST/website-dir.htaccess $GIT/Documentation/web/server/website-dir.htaccess
 225 @end verbatim
 226
 227 If the changes look ok, make them trusted:
 228
 229 @verbatim
 230 ### copy-from-git.sh
 231 #!/bin/sh
 232 GIT=$HOME/lilypond/lilypond-git
 233 DEST=$HOME/lilypond/trusted-scripts
 234 cp $GIT/make/website.make $DEST/website.make
 235 cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
 236 cp $GIT/scripts/build/extract_texi_filenames.py $DEST/extract_texi_filenames.py
 237 cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
 238 cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
 239 cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
 240 cp $GIT/scripts/build/website_post.py $DEST/website_post.py
 241 cp $GIT/scripts/build/bib2texi.py $DEST/bib2texi.py
 242 cp $GIT/python/langdefs.py $DEST/langdefs.py
 243 cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
 244 cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess
 245 @end verbatim
 246 @end smallexample
 247
 248 Build the website:
 249
 250 @verbatim
 251 ### make-website.sh
 252 #!/bin/sh
 253 DEST=$HOME/web/
 254 BUILD=$HOME/lilypond/build-website
 255 mkdir -p $BUILD
 256 cd $BUILD
 257 cp $HOME/lilypond/trusted-scripts/website.make .
 258
 259 make -f website.make WEBSITE_ONLY_BUILD=1 website
 260 rsync -raO $BUILD/out-website/website/ $DEST/website/
 261 cp $BUILD/out-website/pictures $DEST
 262 cp $BUILD/out-website/.htaccess $DEST
 263 @end verbatim
 264
 265 Cronjob to automate the trusted portions:
 266
 267 @verbatim
 268 # website-rebuild.cron
 269 11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1
 270 22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1
 271 @end verbatim
 272
 273
 274 @subsubheading Additional information
 275
 276 Some information about the website is stored in
 277 @file{~graham/lilypond/*.txt}; this information should not be
 278 shared with people without trusted access to the server.
 279
 280
 281 @node Debugging website and docs locally
 282 @section Debugging website and docs locally
 283
 284 @itemize
 285 @item
 286 Install apache2, or any other http server.  These instructions
 287 assume that you also enable @code{mod_userdir}, and use
 288 @code{$HOME/public_html} as the location.
 289
 290 @item
 291 Build the online docs and website:
 292
 293 @example
 294 make WEB_TARGETS="offline online" doc
 295 make website
 296 @end example
 297
 298 This will make all the language variants of the website.  To save
 299 a little time, just the English version can be made with the
 300 command @code{make WEB_LANGS='' website} or the English and (for
 301 example) the French with @code{make WEB_LANGS='fr' website}.
 302
 303 @item
 304 Move the built stuff into those directories.  It's highly
 305 recommended to have your build dir and www dir on the same
 306 partition.  (make @code{$HOME/public_html/} a symlink if
 307 necessary)
 308
 309 @example
 310 mv out-website/website/ $HOME/public_html
 311 mv $HOME/public_html/website/pictures $HOME/public_html/
 312 mkdir -p $HOME/public_html/doc/v2.13/
 313 mv out-www/online-root/* $HOME/public_html/doc/v2.13/
 314 @end example
 315
 316 @end itemize
 317
 318
 319 @node Translating the website
 320 @section Translating the website
 321
 322 As it has much more audience, the website should be translated before
 323 the documentation; see @ref{Translating the documentation}.
 324
 325 In addition to the normal documentation translation practices,
 326 there are a few additional things to note:
 327
 328 @itemize
 329
 330 @item
 331 Build the website with:
 332
 333 @example
 334 make website
 335 @end example
 336
 337 @noindent
 338 however, please note that this command is not designed for being
 339 run multiple times.  If you see unexpected output (mainly the page
 340 footers getting all messed up), then delete your
 341 @file{out-website} directory and run @code{make website} again.
 342
 343 @item
 344 Some of the translation infrastructure is defined in python files;
 345 you must look at the @code{### translation data} sections in:
 346
 347 @example
 348 scripts/build/create-weblinks-itexi.py
 349 scripts/build/website_post.py
 350 @end example
 351
 352 @item
 353 Translations are not included by default in @code{make website}.
 354 To test your translation, edit the @code{WEB_LANGUAGES =} line in
 355 @file{python/langdefs.py}.  You will need to copy this updated
 356 script to @code{build/python/out}.
 357
 358 Do not submit a patch to add your language to this file unless
 359 @code{make website} completes with fewer than 5 warnings.
 360
 361 @item
 362 Links to manuals are done with macros like
 363 @code{@@manualDevelLearningSplit}.  To get translated links, you
 364 must change that to @code{@@manualDevelLearningSplit-es} (for
 365 es/Spanish translations, for example).
 366
 367 @end itemize
 368
 369
 370