Documentation/contributor/website-work.itexi

   1 @c -*- coding: utf-8; mode: texinfo; -*-
   2 @node Website work
   3 @chapter Website work
   4
   5 @menu
   6 * Introduction to website work::
   7 * Uploading and security::
   8 * Debugging website and docs locally::
   9 * Translating the website::
  10 @end menu
  11
  12
  13 @node Introduction to website work
  14 @section Introduction to website work
  15
  16 The website is @emph{not} written directly in HTML;
  17 instead, the source is Texinfo, which is then generated into HTML,
  18 PDF, and Info formats.  The sources are
  19
  20 @example
  21 Documentation/web.texi
  22 Documentation/web/*.texi
  23 @end example
  24
  25 Unless otherwise specified, follow the instructions and policies
  26 given in @ref{Documentation work}.  That chapter also contains a
  27 quick introduction to Texinfo; consulting an external Texinfo
  28 manual should be not necessary.
  29
  30 @subheading Exceptions to the documentation policies
  31
  32 @itemize
  33
  34 @item
  35 Sectioning: the website only uses chapters and sections; no
  36 subsections or subsubsections.
  37
  38 @item
  39 @@ref@{@}s to other manuals (@@ruser, @@rlearning, etc): you can't
  40 link to any pieces of automatically generated documentation, like
  41 the IR or certain NR appendices.
  42
  43 @item
  44 The bibliography in Community->Publications is generated automatically
  45 from @file{.bib} files; formatting is done automatically by
  46 @file{texi-web.bst}.
  47
  48 @item
  49 @dots{}
  50
  51 @item
  52 For anything not listed here, just follow the same style as the
  53 existing website texinfo files.
  54
  55 @end itemize
  56
  57
  58 @node Uploading and security
  59 @section Uploading and security
  60
  61 @subheading Building the website from scratch for local checking
  62
  63 Initial setup:
  64
  65 Create directories:
  66
  67 @example
  68 mkdir $HOME/lilypond/
  69 mkdir $HOME/lilypond/media/
  70 mkdir $HOME/lilypond/trusted-scripts/
  71 @end example
  72
  73 To reduce the CPU burden on the shared host (as well as some
  74 security concerns), the 'Documentation/pictures/' and
  75 'Documentation/ly-examples/' directories are **not** compiled.
  76 We will do this ourselves right now.
  77
  78 Go to your lilypond build directory.  make doc.
  79
  80 Set up some variables (you'll only do this once:)
  81
  82 @example
  83 BUILD_DIR=$HOME/lilypond-git
  84 PICS=$BUILD_DIR/Documentation/pictures/
  85 EXAMPLES=$BUILD_DIR/Documentation/ly-examples/
  86 @end example
  87
  88 Copy files over:
  89
  90 @example
  91 cp -r $PICS $HOME/lilypond/media/pictures
  92 cp -r $EXAMPLES $HOME/lilypond/media/ly-examples
  93 @end example
  94
  95 Get the scripts you need.  First define these variables:
  96
  97 @example
  98 GIT=$HOME/lilypond-git
  99 DEST=$HOME/lilypond/trusted-scripts
 100 @end example
 101
 102 Then do the copying:
 103
 104 @smallexample
 105 cp $GIT/make/website.make $DEST/website.make
 106 cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
 107 cp $GIT/scripts/build/extract_texi_filenames.py  $DEST/extract_texi_filenames.py
 108 cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
 109 cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
 110 cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
 111 cp $GIT/scripts/build/website_post.py $DEST/website_post.py
 112 cp $GIT/scripts/build/bib2texi.py $DEST/bib2texi.py
 113 cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
 114 cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess
 115 @end smallexample
 116
 117 For a complete build you will need a copy of @code{lilypond-extra} git repository.
 118 You can checkout a fresh copy easily:
 119
 120 @example
 121 export LILYPOND_WEB_MEDIA_GIT=$HOME/lilypond-extra
 122 git clone git://github.com/gperciva/lilypond-extra.git $LILYPOND_WEB_MEDIA_GIT
 123 @end example
 124
 125 Just note that the example above expects a bash environment. If you are using another shell
 126 you might need to use a different keyword, other than @code{export}.
 127
 128 Delete your build directory (or maybe just rename your build
 129 directory to build-old).
 130
 131 @example
 132 cd $HOME/lilypond
 133 @end example
 134
 135 Run
 136
 137 @example
 138 make -f ../lilypond-git/make/website.make WEBSITE_ONLY_BUILD=1 \
 139   TOP_SRC_DIR=$HOME/lilypond-git/ \
 140   PYTHONPATH=$HOME/lilypond-git/python \
 141   TEXI2HTML_PROGRAM=texi2html \
 142   website
 143 @end example
 144
 145 The website should be at:
 146
 147 @example
 148 $HOME/lilypond/out-website/website/index.html
 149 @end example
 150
 151 @subheading Building the online website
 152
 153 The website is generated hourly by user @code{graham} the host
 154 @code{lilypond.org}.  For security reasons, we do not use the
 155 makefiles and scripts directly from git; copies of the relevant
 156 scripts are examined and copied to
 157 @code{~graham/lilypond/trusted-scripts/}
 158
 159 @subheading Initial setup
 160
 161 You should symlink your own @file{~/lilypond/} to
 162 @file{~graham/lilypond/}
 163
 164 If this directory does not exist, make it.  Git master should go
 165 in @file{~/lilypond/lilypond-git/} but make sure you enable:
 166
 167 @example
 168 git config core.filemode false
 169 @end example
 170
 171 If you have created any files in @file{~graham/lilypond/} then
 172 please run:
 173
 174 @example
 175 chgrp lilypond ~graham/lilypond/ -R
 176 chmod 775 ~graham/lilypond/ -R
 177 @end example
 178
 179 To reduce the CPU burden on the shared host (as well as some
 180 security concerns), the @file{Documentation/pictures/} and
 181 @file{Documentation/ly-examples/} directories are
 182 @strong{not} compiled.  You need to upload them, and if they ever
 183 change, a user in the @code{lilypond} group must upload them to
 184 @file{~graham/lilypond/media} on the host.
 185
 186 Upload latest pictures/ and ly-examples/ (local script):
 187
 188 @warning{You may need to change a number of items in the below
 189 script.}
 190
 191 @verbatim
 192 ### upload-lily-web-media.sh
 193 #!/bin/sh
 194 BUILD_DIR=$HOME/src/build-lilypond
 195
 196 PICS=$BUILD_DIR/Documentation/pictures/out-www/
 197 EXAMPLES=$BUILD_DIR/Documentation/ly-examples/out-www/
 198
 199 cd $BUILD_DIR
 200 rsync -a $PICS graham@lilypond.org:lilypond/media/pictures
 201 rsync -a $EXAMPLES graham@lilypond.org:lilypond/media/ly-examples
 202 @end verbatim
 203
 204
 205
 206 @subheading Normal maintenance
 207
 208 Get latest source code:
 209
 210 @verbatim
 211 ### update-git.sh
 212 #!/bin/sh
 213 cd $HOME/lilypond/lilypond-git
 214 git fetch origin
 215 git merge origin/master
 216 @end verbatim
 217
 218 Check for any updates to trusted scripts / files:
 219 @smallexample
 220 @verbatim
 221 ### check-git.sh
 222 #!/bin/sh
 223 GIT=$HOME/lilypond/lilypond-git
 224 DEST=$HOME/lilypond/trusted-scripts
 225 diff -u $DEST/website.make $GIT/make/website.make
 226 diff -u $DEST/lilypond-texi2html.init $GIT/Documentation/lilypond-texi2html.init
 227 diff -u $DEST/extract_texi_filenames.py $GIT/scripts/build/extract_texi_filenames.py
 228 diff -u $DEST/create-version-itexi.py $GIT/scripts/build/create-version-itexi.py
 229 diff -u $DEST/create-weblinks-itexi.py $GIT/scripts/build/create-weblinks-itexi.py
 230 diff -u $DEST/mass-link.py $GIT/scripts/build/mass-link.py
 231 diff -u $DEST/website_post.py $GIT/scripts/build/website_post.py
 232 diff -u $DEST/bib2texi.py $GIT/scripts/build/bib2texi.py
 233 diff -u $DEST/langdefs.py $GIT/python/langdefs.py
 234 diff -u $DEST/lilypond.org.htaccess $GIT/Documentation/web/server/lilypond.org.htaccess
 235 diff -u $DEST/website-dir.htaccess $GIT/Documentation/web/server/website-dir.htaccess
 236 @end verbatim
 237
 238 If the changes look ok, make them trusted:
 239
 240 @verbatim
 241 ### copy-from-git.sh
 242 #!/bin/sh
 243 GIT=$HOME/lilypond/lilypond-git
 244 DEST=$HOME/lilypond/trusted-scripts
 245 cp $GIT/make/website.make $DEST/website.make
 246 cp $GIT/Documentation/lilypond-texi2html.init $DEST/lilypond-texi2html.init
 247 cp $GIT/scripts/build/extract_texi_filenames.py $DEST/extract_texi_filenames.py
 248 cp $GIT/scripts/build/create-version-itexi.py $DEST/create-version-itexi.py
 249 cp $GIT/scripts/build/create-weblinks-itexi.py $DEST/create-weblinks-itexi.py
 250 cp $GIT/scripts/build/mass-link.py $DEST/mass-link.py
 251 cp $GIT/scripts/build/website_post.py $DEST/website_post.py
 252 cp $GIT/scripts/build/bib2texi.py $DEST/bib2texi.py
 253 cp $GIT/python/langdefs.py $DEST/langdefs.py
 254 cp $GIT/Documentation/web/server/lilypond.org.htaccess $DEST/lilypond.org.htaccess
 255 cp $GIT/Documentation/web/server/website-dir.htaccess $DEST/website-dir.htaccess
 256 @end verbatim
 257 @end smallexample
 258
 259 Build the website:
 260
 261 @verbatim
 262 ### make-website.sh
 263 #!/bin/sh
 264 DEST=$HOME/web/
 265 BUILD=$HOME/lilypond/build-website
 266 mkdir -p $BUILD
 267 cd $BUILD
 268 cp $HOME/lilypond/trusted-scripts/website.make .
 269
 270 make -f website.make WEBSITE_ONLY_BUILD=1 -B website
 271 rsync -raO $BUILD/out-website/website/ $DEST/website/
 272 cp $BUILD/out-website/pictures $DEST
 273 cp $BUILD/out-website/.htaccess $DEST
 274 @end verbatim
 275
 276 Cronjob to automate the trusted portions:
 277
 278 @verbatim
 279 # website-rebuild.cron
 280 11 * * * * $HOME/lilypond/trusted-scripts/update-git.sh >/dev/null 2>&1
 281 22 * * * * $HOME/lilypond/trusted-scripts/make-website.sh >/dev/null 2>&1
 282 @end verbatim
 283
 284
 285 @subsubheading Additional information
 286
 287 Some information about the website is stored in
 288 @file{~graham/lilypond/*.txt}; this information should not be
 289 shared with people without trusted access to the server.
 290
 291
 292 @node Debugging website and docs locally
 293 @section Debugging website and docs locally
 294
 295 @itemize
 296 @item
 297 Install apache2, or any other http server.  These instructions
 298 assume that you also enable @code{mod_userdir}, and use
 299 @code{$HOME/public_html} as the location.
 300
 301 @item
 302 Build the online docs and website:
 303
 304 @example
 305 make WEB_TARGETS="offline online" doc
 306 make website
 307 @end example
 308
 309 This will make all the language variants of the website.  To save
 310 a little time, just the English version can be made with the
 311 command @code{make WEB_LANGS='' website} or the English and (for
 312 example) the French with @code{make WEB_LANGS='fr' website}.
 313
 314 @item
 315 Move the built stuff into those directories.  It's highly
 316 recommended to have your build dir and www dir on the same
 317 partition.  (make @code{$HOME/public_html/} a symlink if
 318 necessary)
 319
 320 @example
 321 mv out-website/website/ $HOME/public_html
 322 mv $HOME/public_html/website/pictures $HOME/public_html/
 323 mkdir -p $HOME/public_html/doc/v2.13/
 324 mv out-www/online-root/* $HOME/public_html/doc/v2.13/
 325 @end example
 326
 327 @end itemize
 328
 329
 330 @node Translating the website
 331 @section Translating the website
 332
 333 As it has much more audience, the website should be translated before
 334 the documentation; see @ref{Translating the documentation}.
 335
 336 In addition to the normal documentation translation practices,
 337 there are a few additional things to note:
 338
 339 @itemize
 340
 341 @item
 342 Build the website with:
 343
 344 @example
 345 make website
 346 @end example
 347
 348 @noindent
 349 however, please note that this command is not designed for being
 350 run multiple times.  If you see unexpected output (mainly the page
 351 footers getting all messed up), then delete your
 352 @file{out-website} directory and run @code{make website} again.
 353
 354 @item
 355 Some of the translation infrastructure is defined in python files;
 356 you must look at the @code{### translation data} sections in:
 357
 358 @example
 359 scripts/build/create-weblinks-itexi.py
 360 scripts/build/website_post.py
 361 @end example
 362
 363 @item
 364 Translations are not included by default in @code{make website}.
 365 To test your translation, edit the @code{WEB_LANGUAGES =} line in
 366 @file{python/langdefs.py}.  You will need to copy this updated
 367 script to @code{build/python/out}.
 368
 369 Do not submit a patch to add your language to this file unless
 370 @code{make website} completes with fewer than 5 warnings.
 371
 372 @item
 373 Links to manuals are done with macros like
 374 @code{@@manualDevelLearningSplit}.  To get translated links, you
 375 must change that to @code{@@manualDevelLearningSplit-es} (for
 376 es/Spanish translations, for example).
 377
 378 @end itemize
 379
 380
 381