Documentation/contributor/website-work.itexi

   1 @c -*- coding: utf-8; mode: texinfo; -*-
   2 @node Website work
   3 @chapter Website work
   4
   5 @menu
   6 * Introduction to website work::
   7 * Uploading and security::
   8 * Debugging website and docs locally::
   9 * Translating the website::
  10 @end menu
  11
  12
  13 @node Introduction to website work
  14 @section Introduction to website work
  15
  16 The website is @emph{not} written directly in HTML;
  17 instead, the source is Texinfo, which is then generated into HTML,
  18 PDF, and Info formats.  The sources are
  19
  20 @example
  21 Documentation/web.texi
  22 Documentation/web/*.texi
  23 @end example
  24
  25 Unless otherwise specified, follow the instructions and policies
  26 given in @ref{Documentation work}.  That chapter also contains a
  27 quick introduction to Texinfo; consulting an external Texinfo
  28 manual should be not necessary.
  29
  30 @subheading Exceptions to the documentation policies
  31
  32 @itemize
  33
  34 @item
  35 Sectioning: the website only uses chapters and sections; no
  36 subsections or subsubsections.
  37
  38 @item
  39 @@ref@{@}s to other manuals (@@ruser, @@rlearning, etc): you can't
  40 link to any pieces of automatically generated documentation, like
  41 the IR or certain NR appendices.
  42
  43 @item
  44 The bibliography in Community->Publications is generated automatically
  45 from @file{.bib} files; formatting is done automatically by
  46 @file{texi-web.bst}.
  47
  48 @item
  49 @dots{}
  50
  51 @item
  52 For anything not listed here, just follow the same style as the
  53 existing website texinfo files.
  54
  55 @end itemize
  56
  57
  58 @node Uploading and security
  59 @section Uploading and security
  60
  61 @subheading Building the website from scratch for local checking
  62
  63 Initial setup:
  64
  65 Create directories:
  66
  67 @example
  68 $HOME/lilypond/
  69 $HOME/lilypond/media/
  70 $HOME/lilypond/trusted-scripts/
  71 @end example
  72
  73 To reduce the CPU burden on the shared host (as well as some
  74 security concerns), the 'Documentation/pictures/' and
  75 'Documentation/web/ly-examples/' directories are **not** compiled.
  76 We will do this ourselves right now.
  77
  78 Go to your lilypond build directory.  make doc.
  79
  80 Set up some variables (you'll only do this once:)
  81
  82 @example
  83 BUILD_DIR=$HOME/lilypond-git
  84 PICS=$BUILD_DIR/Documentation/pictures/
  85 EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/
  86 @end example
  87
  88 Copy files over:
  89
  90 @example
  91 cp -r $PICS $HOME/lilypond/media/pictures
  92 cp -r $EXAMPLES $HOME/lilypond/media/ly-examples
  93 @end example
  94
  95 Get the scripts you need.  First define these variables:
  96
  97 @example
  98 GIT=$HOME/lilypond-git
  99 DEST=$HOME/lilypond/trusted-scripts
 100 @end example
 101
 102 Then do the copying:
 103
 104 @smallexample
 105 cp $GIT/make/website.make $DEST/website.make
 106 cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
 107 cp $GIT/scripts/build/extract_texi_filenames.py  $DEST/extract_texi_filenames.py
 108 cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
 109 cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
 110 cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
 111 cp $GIT/scripts/build/website_post.py $DEST/website_post.py
 112 cp $GIT/scripts/build/bib2texi.py $DEST/bib2texi.py
 113 cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
 114 cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess
 115 @end smallexample
 116
 117 Delete your build directory (or maybe just rename your build
 118 directory to build-old).
 119
 120 @example
 121 cd $HOME/lilypond
 122 @end example
 123
 124 Run
 125
 126 @example
 127 make -f ../lilypond-git/make/website.make WEBSITE_ONLY_BUILD=1 \
 128   TOP_SRC_DIR=$HOME/lilypond-git/ \
 129   PYTHONPATH=$HOME/lilypond-git/python \
 130   TEXI2HTML_PROGRAM=texi2html \
 131   website
 132 @end example
 133
 134 The website should be at:
 135
 136 @example
 137 $HOME/lilypond/out-website/website/index.html
 138 @end example
 139
 140 @subheading Building the online website
 141
 142 The website is generated hourly by user @code{graham} the host
 143 @code{lilypond.org}.  For security reasons, we do not use the
 144 makefiles and scripts directly from git; copies of the relevant
 145 scripts are examined and copied to
 146 @code{~graham/lilypond/trusted-scripts/}
 147
 148 @subheading Initial setup
 149
 150 You should symlink your own @file{~/lilypond/} to
 151 @file{~graham/lilypond/}
 152
 153 If this directory does not exist, make it.  Git master should go
 154 in @file{~/lilypond/lilypond-git/} but make sure you enable:
 155
 156 @example
 157 git config core.filemode false
 158 @end example
 159
 160 If you have created any files in @file{~graham/lilypond/} then
 161 please run:
 162
 163 @example
 164 chgrp lilypond ~graham/lilypond/ -R
 165 chmod 775 ~graham/lilypond/ -R
 166 @end example
 167
 168 To reduce the CPU burden on the shared host (as well as some
 169 security concerns), the @file{Documentation/pictures/} and
 170 @file{Documentation/web/ly-examples/} directories are
 171 @strong{not} compiled.  You need to upload them, and if they ever
 172 change, a user in the @code{lilypond} group must upload them to
 173 @file{~graham/lilypond/media} on the host.
 174
 175 Upload latest pictures/ and ly-examples/ (local script):
 176
 177 @warning{You may need to change a number of items in the below
 178 script.}
 179
 180 @verbatim
 181 ### upload-lily-web-media.sh
 182 #!/bin/sh
 183 BUILD_DIR=$HOME/src/build-lilypond
 184
 185 PICS=$BUILD_DIR/Documentation/pictures/out-www/
 186 EXAMPLES=$BUILD_DIR/Documentation/web/ly-examples/out-www/
 187
 188 cd $BUILD_DIR
 189 rsync -a $PICS graham@lilypond.org:lilypond/media/pictures
 190 rsync -a $EXAMPLES graham@lilypond.org:lilypond/media/ly-examples
 191 @end verbatim
 192
 193
 194
 195 @subheading Normal maintenance
 196
 197 Get latest source code:
 198
 199 @verbatim
 200 ### update-git.sh
 201 #!/bin/sh
 202 cd $HOME/lilypond/lilypond-git
 203 git fetch origin
 204 git merge origin/master
 205 @end verbatim
 206
 207 Check for any updates to trusted scripts / files:
 208 @smallexample
 209 @verbatim
 210 ### check-git.sh
 211 #!/bin/sh
 212 GIT=$HOME/lilypond/lilypond-git
 213 DEST=$HOME/lilypond/trusted-scripts
 214 diff -u $DEST/website.make $GIT/make/website.make
 215 diff -u $DEST/lilypond-texi2html.init $GIT/Documentation/lilypond-texi2html.init
 216 diff -u $DEST/extract_texi_filenames.py $GIT/scripts/build/extract_texi_filenames.py
 217 diff -u $DEST/create-version-itexi.py $GIT/scripts/build/create-version-itexi.py
 218 diff -u $DEST/create-weblinks-itexi.py $GIT/scripts/build/create-weblinks-itexi.py
 219 diff -u $DEST/mass-link.py $GIT/scripts/build/mass-link.py
 220 diff -u $DEST/website_post.py $GIT/scripts/build/website_post.py
 221 diff -u $DEST/bib2texi.py $GIT/scripts/build/bib2texi.py
 222 diff -u $DEST/lilypond.org.htaccess $GIT/Documentation/web/server/lilypond.org.htaccess
 223 diff -u $DEST/website-dir.htaccess $GIT/Documentation/web/server/website-dir.htaccess
 224 @end verbatim
 225
 226 If the changes look ok, make them trusted:
 227
 228 @verbatim
 229 ### copy-from-git.sh
 230 #!/bin/sh
 231 GIT=$HOME/lilypond/lilypond-git
 232 DEST=$HOME/lilypond/trusted-scripts
 233 cp $GIT/make/website.make $DEST/website.make
 234 cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
 235 cp $GIT/scripts/build/extract_texi_filenames.py $DEST/extract_texi_filenames.py
 236 cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
 237 cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
 238 cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
 239 cp $GIT/scripts/build/website_post.py $DEST/website_post.py
 240 cp $GIT/scripts/build/bib2texi.py $DEST/bib2texi.py
 241 cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
 242 cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess
 243 @end verbatim
 244 @end smallexample
 245
 246 Build the website:
 247
 248 @verbatim
 249 ### make-website.sh
 250 #!/bin/sh
 251 DEST=$HOME/web/
 252 BUILD=$HOME/lilypond/build-website
 253 mkdir -p $BUILD
 254 cd $BUILD
 255 cp $HOME/lilypond/trusted-scripts/website.make .
 256
 257 make -f website.make WEBSITE_ONLY_BUILD=1 website
 258 rsync -raO $BUILD/out-website/website/ $DEST/website/
 259 cp $BUILD/out-website/pictures $DEST
 260 cp $BUILD/out-website/.htaccess $DEST
 261 @end verbatim
 262
 263 Cronjob to automate the trusted portions:
 264
 265 @verbatim
 266 # website-rebuild.cron
 267 11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1
 268 22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1
 269 @end verbatim
 270
 271
 272 @subsubheading Additional information
 273
 274 Some information about the website is stored in
 275 @file{~graham/lilypond/*.txt}; this information should not be
 276 shared with people without trusted access to the server.
 277
 278
 279 @node Debugging website and docs locally
 280 @section Debugging website and docs locally
 281
 282 @itemize
 283 @item
 284 Install apache2, or any other http server.  These instructions
 285 assume that you also enable @code{mod_userdir}, and use
 286 @code{$HOME/public_html} as the location.
 287
 288 @item
 289 Build the online docs and website:
 290
 291 @example
 292 make WEB_TARGETS="offline online" doc
 293 make website
 294 @end example
 295
 296 This will make all the language variants of the website.  To save
 297 a little time, just the English version can be made with the
 298 command @code{make WEB_LANGS='' website} or the English and (for
 299 example) the French with @code{make WEB_LANGS='fr' website}.
 300
 301 @item
 302 Move the built stuff into those directories.  It's highly
 303 recommended to have your build dir and www dir on the same
 304 partition.  (make @code{$HOME/public_html/} a symlink if
 305 necessary)
 306
 307 @example
 308 mv out-website/website/ $HOME/public_html
 309 mv $HOME/public_html/website/pictures $HOME/public_html/
 310 mkdir -p $HOME/public_html/doc/v2.13/
 311 mv out-www/online-root/* $HOME/public_html/doc/v2.13/
 312 @end example
 313
 314 @end itemize
 315
 316
 317 @node Translating the website
 318 @section Translating the website
 319
 320 As it has much more audience, the website should be translated before
 321 the documentation; see @ref{Translating the documentation}.
 322
 323 In addition to the normal documentation translation practices,
 324 there are a few additional things to note:
 325
 326 @itemize
 327
 328 @item
 329 Build the website with:
 330
 331 @example
 332 make website
 333 @end example
 334
 335 @noindent
 336 however, please note that this command is not designed for being
 337 run multiple times.  If you see unexpected output (mainly the page
 338 footers getting all messed up), then delete your
 339 @file{out-website} directory and run @code{make website} again.
 340
 341 @item
 342 Some of the translation infrastructure is defined in python files;
 343 you must look at the @code{### translation data} sections in:
 344
 345 @example
 346 scripts/build/create-weblinks-itexi.py
 347 scripts/build/website_post.py
 348 @end example
 349
 350 @item
 351 Translations are not included by default in @code{make website}.
 352 To test your translation, edit the @code{WEB_LANGUAGES =} line in
 353 @file{python/langdefs.py}.  You will need to copy this updated
 354 script to @code{build/python/out}.
 355
 356 Do not submit a patch to add your language to this file unless
 357 @code{make website} completes with fewer than 5 warnings.
 358
 359 @item
 360 Links to manuals are done with macros like
 361 @code{@@manualDevelLearningSplit}.  To get translated links, you
 362 must change that to @code{@@manualDevelLearningSplit-es} (for
 363 es/Spanish translations, for example).
 364
 365 @end itemize
 366
 367
 368