1 @c -*- coding: utf-8; mode: texinfo; -*-
6 * Introduction to LSR::
7 * Adding and editing snippets::
10 * Fixing snippets in LilyPond sources::
11 * Renaming a snippet::
12 * Updating the LSR to a new version::
16 @node Introduction to LSR
17 @section Introduction to LSR
20 @uref{http://lsr.dsi.unimi.it/, LilyPond Snippet Repository (LSR)}
21 is a collection of lilypond examples. A subset of these examples
22 are automatically imported into the documentation, making it easy
23 for users to contribute to the docs without learning Git and
27 @node Adding and editing snippets
28 @section Adding and editing snippets
30 @subheading General guidelines
32 When you create (or find!) a nice snippet, if it is supported by
33 the LilyPond version running on the LSR, please add it to the LSR.
34 Go to @uref{http://lsr.dsi.unimi.it/, LSR} and log in -- if you haven't
35 already, create an account. Follow the instructions on the website.
36 These instructions also explain how to modify existing snippets.
38 If you think the snippet is particularly informative and you think it
39 should be included in the documentation, tag it with @qq{docs} and one
40 or more other categories, or ask on the development list for
41 somebody who has editing permissions to do it .
43 Please make sure that the lilypond code follows the guidelines in
44 @ref{LilyPond formatting}.
46 If a new snippet created for documentation purposes compiles with
47 LilyPond version currently on LSR, it should be added to LSR, and a
48 reference to the snippet should be added to the documentation.
49 Please ask a documentation editor to add a reference to it in an
50 appropriate place in the docs. (Note -- it should appear in the
51 snippets document automatically, once it has been imported into
52 git and built. See @ref{LSR to Git}.
54 If the new snippet uses new features that are not available in the
55 current LSR version, the snippet should be added to
56 @file{Documentation/snippets/new} and a reference should be added to the
59 Snippets created or updated in @file{Documentation/snippets/new} should
60 be copied to @file{Documentation/snippets} by invoking at top of the
64 scripts/auxiliar/makelsr.py
67 Be sure that @command{make doc} runs successfully before submitting a
68 patch, to prevent breaking compilation.
70 @subheading Formatting snippets in @file{Documentation/snippets/new}
72 When adding a file to this directory, please start the file with
77 % Use existing LSR tags other than 'docs'; see makelsr.py for
78 % the list of tags used to sort snippets. E.g.:
79 lsrtags = "rhythms,expressive-marks"
80 % This texidoc string will be formatted by Texinfo
82 This code demonstrates ...
84 % Please put doctitle last so that the '% begin verbatim'
85 % mark will be added correctly by makelsr.py.
86 doctitle = "Snippet title"
91 and name the file @file{snippet-title.ly}.
94 Please ensure that the version number you use at the top of the
95 example is the minimum version that the file will compile with:
96 for example, if the LSR is currently at 2.14.2 and your example
97 requires 2.15.30, but the current development version of
98 @code{lilypond} is 2.17.5, put @code{\version "2.15.30"} in the
102 Please also pay particular attention to the lines beginning
103 @code{lsrtags = } and @code{doctitle =}. The tags must match tags used
104 in the documentation, and the @code{doctitle} must match the
107 @node Approving snippets
108 @section Approving snippets
110 The main task of LSR editors is approving snippets. To find a list of
111 unapproved snippets, log into @uref{http://lsr.dsi.unimi.it/, LSR} and
112 select @qq{No} from the dropdown menu to the right of the word
113 @qq{Approved} at the bottom of the interface, then click
121 Does the snippet make sense and does what the author claims that
122 it does? If you think the snippet is particularly helpful, add
123 the @qq{docs} tag and at least one other tag.
126 If the snippet is tagged with @qq{docs}, check to see if it
127 matches our guidelines for @ref{LilyPond formatting}.
129 Also, snippets tagged with @qq{docs} should not be explaining
130 (replicating) existing material in the docs. They should not
131 refer to the docs; the docs should refer to them.
134 If the snippet uses scheme, check that everything looks good and
135 there are no security risks.
137 @warning{Somebody could sneak a @code{#'(system "rm -rf /")}
138 command into our source tree if you do not do this! Take this
139 step @strong{VERY SERIOUSLY}.}
142 If all is well, check the box labelled @qq{approved} and save the
151 @subheading Introduction
152 Snippets used in the documentation are in
153 @file{$LILYPOND_GIT/Documentation/snippets}. This directory
154 contains a complete set of the snippets in the LSR which are
155 tagged with 'docs'. The exact method for getting them there is
156 described below, but in essence they come from downloading a
157 tarball from the LSR and importing into the directory using the
158 @code{makelsr} script.
160 Any snippets which are too bleeding edge to run on the LSR (which
161 uses a stable development version) are put into
162 @file{$LILYPOND_GIT/Documentation/snippets/new}. Once the LSR has
163 been upgraded so that these will run, then they are transferred to
164 the LSR and deleted from @file{/snippets/new}.
166 'Git' is the shorthand name for the Git repository that contains
167 all the development code. For further information on setting this
168 up see, @ref{Working with source code}. An alternative to setting
169 up a Git repository for people wanting to do LSR work is to get
171 @uref{http://lilypond.org/website/development.html}.
173 @subheading Importing the LSR to Git
178 Make sure that @command{convert-ly} script and the
179 @command{lilypond} binary are a
180 bleeding edge version -- the latest release or even better, a fresh
181 snapshot from Git master.
184 Start by creating a list of updated snippets from your local
185 repository. From the top source directory, run:
188 scripts/auxiliar/makelsr.py
191 Commit the changes and make a patch. Check the patch has nothing
192 other than minor changes - in particular changes to the commitish
193 for translations. If all is good and you're confident in what
194 you've done, this can be pushed directly to staging.
197 Next, download the updated snippets and run makelsr against them.
198 From the top source directory, run:
201 wget http://lsr.dsi.unimi.it/download/lsr-snippets-docs-`date +%F`.tar.gz
202 tar -xzf lsr-snippets-docs-`date +%F`.tar.gz
203 make -C $LILYPOND_BUILD_DIR
204 scripts/auxiliar/makelsr.py lsr-snippets-docs-`date +%F`
208 where @command{date +%F} gives the current date in format
209 @var{YYYY-MM-DD} (the snippets archive is usually generated around
210 03:50 CET, you may want to use @command{date -d yesterday +%F}
211 instead, depending on your time zone and the time you run this
212 commands sequence). @command{make} is included in this sequence so
213 that @command{makelsr} can run @command{lilypond} and
214 @command{convert-ly} versions that match current source tree; you can
215 select different binaries if desired or needed, to see options for
219 scripts/auxiliar/makelsr.py --help
223 Follow the instructions printed on the console to manually check for
224 unsafe files. These are:
227 Unsafe files printed in lsr-unsafe.txt: CHECK MANUALLY!
228 git add Documentation/snippets/*.ly
229 xargs git diff HEAD < lsr-unsafe.txt
232 First, it's important to check for any added files and add them to
233 the files git is tracking. Run @code{git status} and look
234 carefully to see if files have been added. If so, add them with
237 As the console says, @command{makelsr} creates a list of possibly
238 unsafe files in @file{lsr-unsafe.txt} by running @code{lilypond}
239 against each snippet using the @code{-dsafe} switch. This list can be
240 quite long. However, by using the command @code{xargs git diff HEAD <
241 lsr-unsafe.txt} git will take that list and check whether any of the
242 snippets are different from the snippet already in master. If any is
243 different it must be checked manually VERY CAREFULLY.
245 @warning{Somebody could sneak a @code{#'(system "rm -rf /")}
246 command into our source tree if you do not do this! Take this
247 step @strong{VERY SERIOUSLY}.}
249 If there is any doubt about any of the files, you are strongly
250 advised to run a review on Rietveld.
253 If a Review is not needed, commit the changes and push to staging.
257 Note that whenever there is a snippet in
258 @file{Documentation/snippets/new} and another from the LSR with
259 the same file name, @command{makelsr.py} will overwrite the LSR
260 version with the one from @file{Documentation/snippets/new}.
263 @node Fixing snippets in LilyPond sources
264 @section Fixing snippets in LilyPond sources
266 If some snippet from @file{Documentation/snippets} causes the
267 documentation compilation to fail, the following steps should be
268 followed to fix it reliably.
273 Look up the snippet filename @file{@var{foo}.ly} in the error output
274 or log, then fix the file @file{Documentation/snippets/@var{foo}.ly} to make the
275 documentation build successfully.
278 Determine where it comes from by looking at its first line, e.g. run
281 head -1 Documentation/snippets/@var{foo}.ly
285 @strong{If the snippet comes from the LSR}, also apply the fix to the
286 snippet in the LSR and send a notification email to an LSR editor with CC to
287 the development list -- see @ref{Adding and editing snippets}. The
288 failure may sometimes not be caused by the snippet in LSR but by the
289 syntax conversion made by @command{convert-ly}; in this case, try to fix
290 @command{convert-ly} or report the problem on the development list, then
291 run @command{makelsr.py} again, see @ref{LSR to Git}. In some cases,
292 when some features has been introduced or vastly changed so it requires
293 (or takes significant advantage of) important changes in the snippet, it
294 is simpler and recommended to write a new version of the snippet in
295 @file{Documentation/snippets/new}, then run @command{makelsr.py}.
298 @strong{If the snippet comes from}
299 @file{Documentation/snippets/new}, apply the same fix in
300 @file{Documentation/snippets/new/@var{foo}.ly} that you did in
301 @file{Documentation/snippets/@var{foo}.ly}. If the build failure
302 was caused by a translation string, you may have to fix
303 @file{input/texidocs/@var{foo}.texidoc} instead.
306 When you've done, commit your changes to Git and ensure they're
307 pushed to the correct branch.
312 @node Renaming a snippet
313 @section Renaming a snippet
315 Due to the potential duality of snippets (i.e. they may exist both
316 in the LSR database, and in @code{Documentation/snippets/new/}),
317 this process is a bit more involved than we might like.
321 Send an email LSR editor, requesting the renaming.
324 The LSR editor does the renaming (or debates the topic with you),
325 then warns the LSR-to-git person (wanted: better title) about the
329 LSR-to-git person does his normal job, but then also renames any
330 copies of the snippets in @code{Documentation/snippets/new/}, and
331 any instances of the snippet name in the documentation.
333 @code{git grep} is highly recommended for this task.
338 @node Updating the LSR to a new version
339 @section Updating the LSR to a new version
341 To update the LSR, perform the following steps:
346 Start by emailing the LSR maintainer, Sebastiano, and liaising
347 with him to ensure that updating the snippets is synchronised with
348 updating the binary running the LSR.
351 Download the latest snippet tarball from
352 @uref{http://lsr.dsi.unimi.it/download/} and extract it.
353 The relevant files can be found in the @file{all} subdirectory.
354 Make sure your shell is using an English language version, for
355 example @code{LANG=en_US}, then run @command{convert-ly} on all
356 the files. Use the command-line option @code{--to=version} to
357 ensure the snippets are updated to the correct stable version.
360 Make sure that you are using @command{convert-ly} from the latest
361 available release to gain best advantage from the latest
362 @code{converting-rules-updates}.
371 intended LSR-update to 2.14.2
373 latest release 2.15.30
377 Use convert-ly from 2.15.30 and the following terminal command
381 convert-ly -e -t2.14.2 *.ly
385 There might be no conversion rule for some old commands. To make
386 an initial check for possible problems you can run the following
387 script on a copy of the @file{all} subdirectory:
394 STEM=$(basename "$LILYFILE" .ly)
395 echo "running $LILYFILE..."
396 convert-ly -e -t<version> "$LILYFILE" >& "$STEM".txt
405 Copy relevant snippets (i.e. snippets whose version is equal to
406 or less than the new version of LilyPond running on the LSR) from
407 @file{Documentation/snippets/new/} into the set of files to be
408 used to make the tarball. Make sure
409 you only choose snippets which are already present in the LSR,
410 since the LSR software isn't able to create new snippets this way.
411 If you don't have a Git repository for LilyPond, you'll find these
412 snippets in the source-tarball on
413 @uref{http://lilypond.org/website/development.html}.
414 Don't rename any files at this stage.
417 Verify that all files compile with the new version of LilyPond,
418 ideally without any warnings or errors. To ease the process,
419 you may use the shell script that appears after this list.
421 Due to the workload involved, we @emph{do not} require that you
422 verify that all snippets produce the expected output. If you
423 happen to notice any such snippets and can fix them, great; but as
424 long as all snippets compile, don't delay this step due to some
425 weird output. If a snippet is not compiling, update it manually.
426 If it's not possible, delete it for now.
429 Remove all headers and version-statements from the files. Phil
430 Holmes has a @code{python} script that will do this and which
431 needs testing. Please ask him for a copy if you wish to do this.
434 Create a tarball and send it back to Sebastiano. Don't forget to
435 tell him about any deletions.
438 Use the LSR web interface to change any descriptions you want to.
439 Changing the titles of snippets is a bit fraught, since this also
440 changes the filenames. Only do this as a last resort.
443 Use the LSR web interface to add the other snippets from
444 @file{Documentation/snippets/new/} which compile with the new
445 LilyPond version of the LSR. Ensure that they are correctly
446 tagged, including the tag @code{docs} and that they are approved.
449 When LSR has been updated, wait a day for the tarball to update,
450 then download another snippet tarball. Verify that
451 the relevant snippets from @file{Documentation/snippets/new/} are
452 now included, then delete those snippets from
453 @file{Documentation/snippets/new/}.
456 Commit all the changes. @emph{Don't forget to add new files to
457 the git repository with @code{git add}}. Run @code{make},
459 @code{make test} to ensure the changes don't break the build. Any
460 snippets that have had their file name changed or have been
461 deleted could break the build, and these will need correcting
467 Below is a shell script to run all @file{.ly} files in a directory
468 and redirect terminal output to text files, which are then
469 searched for the word "failed" to see which snippets do not compile.
476 STEM=$(basename "$LILYFILE" .ly)
477 echo "running $LILYFILE..."
478 lilypond --format=png -ddelete-intermediate-files "$LILYFILE" >& "$STEM".txt
485 Sometimes @code{grep failed *.txt} will not discover all
486 problematic files. In addition you may want to use: