1 @c -*- coding: utf-8; mode: texinfo; -*-
6 * Introduction to LSR::
7 * Adding and editing snippets::
10 * Fixing snippets in LilyPond sources::
11 * Renaming a snippet::
12 * Updating the LSR to a new version::
16 @node Introduction to LSR
17 @section Introduction to LSR
20 @uref{http://lsr.di.unimi.it/, LilyPond Snippet Repository (LSR)}
21 is a collection of lilypond examples. A subset of these examples
22 are automatically imported into the documentation, making it easy
23 for users to contribute to the docs without learning Git and
27 @node Adding and editing snippets
28 @section Adding and editing snippets
30 @subheading General guidelines
32 When you create (or find!) a nice snippet, if it is supported by
33 the LilyPond version running on the LSR, please add it to the LSR.
34 Go to @uref{http://lsr.di.unimi.it/, LSR} and log in -- if you haven't
35 already, create an account. Follow the instructions on the website.
36 These instructions also explain how to modify existing snippets.
38 If you think the snippet is particularly informative and you think it
39 should be included in the documentation, tag it with @qq{docs} and one
40 or more other categories, or ask on the development list for
41 somebody who has editing permissions to do it .
43 Please make sure that the lilypond code follows the guidelines in
44 @ref{LilyPond formatting}.
46 If a new snippet created for documentation purposes compiles with
47 LilyPond version currently on LSR, it should be added to LSR, and a
48 reference to the snippet should be added to the documentation.
49 Please ask a documentation editor to add a reference to it in an
50 appropriate place in the docs. (Note -- it should appear in the
51 snippets document automatically, once it has been imported into
52 git and built. See @ref{LSR to Git}.
54 If the new snippet uses new features that are not available in the
55 current LSR version, the snippet should be added to
56 @file{Documentation/snippets/new} and a reference should be added to the
59 Snippets created or updated in @file{Documentation/snippets/new} should
60 be copied to @file{Documentation/snippets} by invoking at top of the
64 scripts/auxiliar/makelsr.py
67 Be sure that @command{make doc} runs successfully before submitting a
68 patch, to prevent breaking compilation.
70 @subheading Formatting snippets in @file{Documentation/snippets/new}
72 When adding a file to this directory, please start the file with
77 % Use existing LSR tags other than 'docs'; see makelsr.py for
78 % the list of tags used to sort snippets. E.g.:
79 lsrtags = "rhythms,expressive-marks"
80 % This texidoc string will be formatted by Texinfo
82 This code demonstrates ...
84 % Please put doctitle last so that the '% begin verbatim'
85 % mark will be added correctly by makelsr.py.
86 doctitle = "Snippet title"
91 and name the file @file{snippet-title.ly}.
94 Please ensure that the version number you use at the top of the
95 example is the minimum version that the file will compile with:
96 for example, if the LSR is currently at 2.14.2 and your example
97 requires 2.15.30, but the current development version of
98 @code{lilypond} is 2.17.5, put @code{\version "2.15.30"} in the
102 Please also pay particular attention to the lines beginning
103 @code{lsrtags = } and @code{doctitle =}. The tags must match tags used
104 in the documentation, and the @code{doctitle} must match the
107 @node Approving snippets
108 @section Approving snippets
110 The main task of LSR editors is approving snippets. To find a list of
111 unapproved snippets, log into @uref{http://lsr.di.unimi.it/, LSR} and
112 select @qq{No} from the dropdown menu to the right of the word
113 @qq{Approved} at the bottom of the interface, then click
121 Does the snippet make sense and does what the author claims that
122 it does? If you think the snippet is particularly helpful, add
123 the @qq{docs} tag and at least one other tag.
126 If the snippet is tagged with @qq{docs}, check to see if it
127 matches our guidelines for @ref{LilyPond formatting}.
129 Also, snippets tagged with @qq{docs} should not be explaining
130 (replicating) existing material in the docs. They should not
131 refer to the docs; the docs should refer to them.
134 If the snippet uses scheme, check that everything looks good and
135 there are no security risks.
137 @warning{Somebody could sneak a @code{#'(system "rm -rf /")}
138 command into our source tree if you do not do this! Take this
139 step @strong{VERY SERIOUSLY}.}
142 If all is well, check the box labelled @qq{approved} and save the
151 @subheading Introduction
152 Snippets used in the documentation are in
153 @file{$LILYPOND_GIT/Documentation/snippets}. This directory
154 contains a complete set of the snippets in the LSR which are
155 tagged with 'docs'. The exact method for getting them there is
156 described below, but in essence they come from downloading a
157 tarball from the LSR and importing into the directory using the
158 @code{makelsr} script.
160 Any snippets which are too bleeding edge to run on the LSR (which
161 uses a stable development version) are put into
162 @file{$LILYPOND_GIT/Documentation/snippets/new}. Once the LSR has
163 been upgraded so that these will run, then they are transferred to
164 the LSR and deleted from @file{/snippets/new}.
166 'Git' is the shorthand name for the Git repository that contains
167 all the development code. For further information on setting this
168 up see, @ref{Working with source code}. An alternative to setting
169 up a Git repository for people wanting to do LSR work is to get
171 @uref{http://lilypond.org/website/development.html}.
173 @subheading Importing the LSR to Git
178 Make sure that @command{convert-ly} script and the @command{lilypond}
179 binary are a bleeding edge version -- the latest release or even
180 better, a fresh snapshot from Git master, with the environment
181 variable @code{LILYPOND_BUILD_DIR} correctly set up, see
182 @ref{Environment variables}.
185 Start by creating a list of updated snippets from your local
186 repository. From the top source directory, run:
189 scripts/auxiliar/makelsr.py
192 Commit the changes and make a patch. Check the patch has nothing
193 other than minor changes. If all is good and you're confident in what
194 you've done, this can be pushed directly to staging.
197 Next, download the updated snippets and run @command{makelsr.py}
198 against them. From the top source directory, run:
201 wget http://lsr.di.unimi.it/download/lsr-snippets-docs-`date +%F`.tar.gz
202 tar -xzf lsr-snippets-docs-`date +%F`.tar.gz
203 make -C $LILYPOND_BUILD_DIR
204 scripts/auxiliar/makelsr.py lsr-snippets-docs-`date +%F`
208 where @command{date +%F} gives the current date in format
209 @var{YYYY-MM-DD} (the snippets archive is usually generated around
210 03:50 CET, you may want to use @command{date -d yesterday +%F}
211 instead, depending on your time zone and the time you run this
212 commands sequence). @command{make} is included in this sequence so
213 that @command{makelsr} can run @command{lilypond} and
214 @command{convert-ly} versions that match current source tree; you can
215 select different binaries if desired or needed, to see options for
219 scripts/auxiliar/makelsr.py --help
223 Follow the instructions printed on the console to manually check for
224 unsafe files. These are:
227 Unsafe files printed in lsr-unsafe.txt: CHECK MANUALLY!
228 git add Documentation/snippets/*.ly
229 xargs git diff HEAD < lsr-unsafe.txt
232 First, it's important to check for any added files and add them to
233 the files git is tracking. Run @code{git status} and look
234 carefully to see if files have been added. If so, add them with
237 As the console says, @command{makelsr} creates a list of possibly
238 unsafe files in @file{lsr-unsafe.txt} by running @code{lilypond}
239 against each snippet using the @code{-dsafe} switch. This list can be
240 quite long. However, by using the command @code{xargs git diff HEAD <
241 lsr-unsafe.txt} git will take that list and check whether any of the
242 snippets are different from the snippet already in master. If any is
243 different it must be checked manually VERY CAREFULLY.
245 @warning{Somebody could sneak a @code{#'(system "rm -rf /")}
246 command into our source tree if you do not do this! Take this
247 step @strong{VERY SERIOUSLY}.}
249 If there is any doubt about any of the files, you are strongly
250 advised to run a review on Rietveld.
253 If a Review is not needed, commit the changes and push to
258 Note that whenever there is a snippet in
259 @file{Documentation/snippets/new} and another from the LSR with
260 the same file name, @command{makelsr.py} will overwrite the LSR
261 version with the one from @file{Documentation/snippets/new}.
264 @node Fixing snippets in LilyPond sources
265 @section Fixing snippets in LilyPond sources
267 If some snippet from @file{Documentation/snippets} causes the
268 documentation compilation to fail, the following steps should be
269 followed to fix it reliably.
274 Look up the snippet filename @file{@var{foo}.ly} in the error output
275 or log, then fix the file @file{Documentation/snippets/@var{foo}.ly} to make the
276 documentation build successfully.
279 Determine where it comes from by looking at its first two lines,
283 head -2 Documentation/snippets/@var{foo}.ly
287 @strong{If the snippet comes from the LSR}, also apply the fix to the
288 snippet in the LSR and send a notification email to an LSR editor with CC to
289 the development list -- see @ref{Adding and editing snippets}. The
290 failure may sometimes not be caused by the snippet in LSR but by the
291 syntax conversion made by @command{convert-ly}; in this case, try to fix
292 @command{convert-ly} or report the problem on the development list, then
293 run @command{makelsr.py} again, see @ref{LSR to Git}. In some cases,
294 when some features has been introduced or vastly changed so it requires
295 (or takes significant advantage of) important changes in the snippet, it
296 is simpler and recommended to write a new version of the snippet in
297 @file{Documentation/snippets/new}, then run @command{makelsr.py}.
300 @strong{If the snippet comes from} @file{Documentation/snippets/new},
301 apply the fix in @file{Documentation/snippets/new/@var{foo}.ly}, then
302 run @command{makelsr.py} without argument from top of the source tree:
305 scripts/auxiliar/makelsr.py
308 Then, inspect @file{Documentation/snippets/@var{foo}.ly} to check that
309 the fix has been well propagated.
311 If the build failure was caused by a translation string, you may have
312 to fix some @file{Documentation/@var{lang}/texidocs/@var{foo}.texidoc}
313 instead; in case the build failure comes only from translation
314 strings, it is not needed to run @command{makelsr.py}.
317 When you've done, commit your changes to Git and ensure they're
318 pushed to the correct branch.
323 @node Renaming a snippet
324 @section Renaming a snippet
326 Due to the potential duality of snippets (i.e. they may exist both
327 in the LSR database, and in @code{Documentation/snippets/new/}),
328 this process is a bit more involved than we might like.
332 Send an email LSR editor, requesting the renaming.
335 The LSR editor does the renaming (or debates the topic with you),
336 then warns the LSR-to-git person (wanted: better title) about the
340 LSR-to-git person does his normal job, but then also renames any
341 copies of the snippets in @code{Documentation/snippets/new/}, and
342 any instances of the snippet name in the documentation.
344 @code{git grep} is highly recommended for this task.
349 @node Updating the LSR to a new version
350 @section Updating the LSR to a new version
352 To update the LSR, perform the following steps:
357 Start by emailing the LSR maintainer, Sebastiano, and liaising
358 with him to ensure that updating the snippets is synchronised with
359 updating the binary running the LSR.
362 Download the latest snippet tarball from
363 @uref{http://lsr.di.unimi.it/download/} and extract it.
364 The relevant files can be found in the @file{all} subdirectory.
365 Make sure your shell is using an English language version, for
366 example @code{LANG=en_US}, then run @command{convert-ly} on all
367 the files. Use the command-line option @code{--to=version} to
368 ensure the snippets are updated to the correct stable version.
371 Make sure that you are using @command{convert-ly} from the latest
372 available release to gain best advantage from the latest
373 @code{converting-rules-updates}.
382 intended LSR-update to 2.14.2
384 latest release 2.15.30
388 Use convert-ly from 2.15.30 and the following terminal command
392 convert-ly -e -t2.14.2 *.ly
396 There might be no conversion rule for some old commands. To make
397 an initial check for possible problems you can run the
398 script at the end of this list on a copy of the @file{all} subdirectory.
401 Copy relevant snippets (i.e. snippets whose version is equal to
402 or less than the new version of LilyPond running on the LSR) from
403 @file{Documentation/snippets/new/} into the set of files to be
404 used to make the tarball. Make sure
405 you only choose snippets which are already present in the LSR,
406 since the LSR software isn't able to create new snippets this way.
407 If you don't have a Git repository for LilyPond, you'll find these
408 snippets in the source-tarball on
409 @uref{http://lilypond.org/website/development.html}.
410 Don't rename any files at this stage.
413 Verify that all files compile with the new version of LilyPond,
414 ideally without any warnings or errors. To ease the process,
415 you may use the shell script that appears after this list.
417 Due to the workload involved, we @emph{do not} require that you
418 verify that all snippets produce the expected output. If you
419 happen to notice any such snippets and can fix them, great; but as
420 long as all snippets compile, don't delay this step due to some
421 weird output. If a snippet is not compiling, update it manually.
422 If it's not possible, delete it for now.
425 Remove all headers and version-statements from the files. Phil
426 Holmes has a @code{python} script that will do this and which
427 needs testing. Please ask him for a copy if you wish to do this.
430 Create a tarball and send it back to Sebastiano. Don't forget to
431 tell him about any deletions.
434 Use the LSR web interface to change any descriptions you want to.
435 Changing the titles of snippets is a bit fraught, since this also
436 changes the filenames. Only do this as a last resort.
439 Use the LSR web interface to add the other snippets from
440 @file{Documentation/snippets/new/} which compile with the new
441 LilyPond version of the LSR. Ensure that they are correctly
442 tagged, including the tag @code{docs} and that they are approved.
445 When LSR has been updated, wait a day for the tarball to update,
446 then download another snippet tarball. Verify that
447 the relevant snippets from @file{Documentation/snippets/new/} are
448 now included, then delete those snippets from
449 @file{Documentation/snippets/new/}.
452 Commit all the changes. @emph{Don't forget to add new files to
453 the git repository with @code{git add}}. Run @code{make},
455 @code{make test} to ensure the changes don't break the build. Any
456 snippets that have had their file name changed or have been
457 deleted could break the build, and these will need correcting
463 Below is a shell script to run LilyPond on all @file{.ly} files in a directory.
464 If the script is run with a -s parameter, it runs silently except for reporting
465 failed files. If run with -c it also runs @code{convert-ly} prior to running
471 while getopts sc opt; do
485 if [ $convert ]; then
488 filter=$@{param:-"*.ly"@}
490 for LILYFILE in $filter
492 STEM=$(basename "$LILYFILE" .ly)
493 if [ $convert ]; then
495 $LILYPOND_BUILD_DIR/out/bin/convert-ly -e "$LILYFILE" >& "$STEM".con.txt
497 $LILYPOND_BUILD_DIR/out/bin/convert-ly -e "$LILYFILE"
500 if [ ! $silent ]; then
501 echo "running $LILYFILE..."
503 $LILYPOND_BUILD_DIR/out/bin/lilypond --format=png "$LILYFILE" >& "$STEM".txt
505 if [ $RetVal -gt 0 ]; then
506 echo "$LILYFILE failed"
511 Output from LilyPond is in @file{filename.txt} and convert-ly in
512 @file{filename.con.txt}.