1 @c -*- coding: us-ascii; mode: texinfo; -*-
3 @chapter Programming work
6 * Overview of LilyPond architecture::
7 * LilyPond programming languages::
8 * Programming without compiling::
11 * Debugging LilyPond::
14 @node Overview of LilyPond architecture
15 @section Overview of LilyPond architecture
17 LilyPond processes the input file into graphical and musical output in a
18 number of stages. This process, along with the types of routines that
19 accomplish the various stages of the process, is described in this section. A
20 more complete description of the LilyPond architecture and internal program
21 execution is found in Erik Sandberg's
22 @uref{http://lilypond.org/web/images/thesis-erik-sandberg.pdf, master's
26 The first stage of LilyPond processing is @emph{parsing}. In the parsing
27 process, music expressions in LilyPond input format are converted to music
28 expressions in Scheme format. In Scheme format, a music expression is a list
29 in tree form, with nodes that indicate the relationships between various music
30 events. The LilyPond parser is written in Bison.
32 The second stage of LilyPond processing is @emph{iterating}. Iterating
33 assigns each music event to a context, which is the environment in which the
34 music will be finally engraved. The context is responsible for all further
35 processing of the music. It is during the iteration stage that contexts are
36 created as necessary to ensure that every note has a Voice type context (e.g.
37 Voice, TabVoice, DrumVoice, CueVoice, MensuralVoice, VaticanaVoice,
38 GregorianTranscriptionVoice), that the Voice type contexts exist in
39 appropriate Staff type contexts, and that parallel Staff type contexts exist
40 in StaffGroup type contexts. In addition, during the iteration stage each
41 music event is assigned a moment, or a time in the music when the event
44 Each type of music event has an associated iterator. Iterators are defined in
45 *-iterator.cc. During iteration, an
46 event's iterator is called to deliver that music event to the appropriate
49 The final stage of LilyPond processing is @emph{translation}. During
50 translation, music events are prepared for graphical or midi output. The
51 translation step is accomplished by translators or engravers (the distinction
54 Translators are defined in C++ files named *-engraver.cc. In *-engraver.cc, a
55 C++ class of Engraver type is created. The Engraver is also declared as a
56 translator. Much of the work of translating is handled by Scheme functions,
57 which is one of the keys to LilyPond's exceptional flexibility.
60 @node LilyPond programming languages
61 @section LilyPond programming languages
63 Programming in LilyPond is done in a variety of programming languages. Each
64 language is used for a specific purpose or purposes. This section describes
65 the languages used and provides links to reference manuals and tutorials for
66 the relevant language.
70 The core functionality of LilyPond is implemented in C++.
72 C++ is so ubiquitous that it is difficult to identify either a reference
73 manual or a tutorial. Programmers unfamiliar with C++ will need to spend some
74 time to learn the language before attempting to modify the C++ code.
76 The C++ code calls Scheme/GUILE through the GUILE interface, which is
78 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html, GUILE
83 The LilyPond parser is implemented in Bison, a GNU parser generator. The
84 Bison homepage is found at @uref{http://www.gnu.org/software/bison/,
85 gnu.org}. The manual (which includes both a reference and tutorial) is
86 @uref{http://www.gnu.org/software/bison/manual/index.html, available} in a
91 GNU Make is used to control the compiling process and to build the
92 documentation and the website. GNU Make documentation is available at
93 @uref{http://www.gnu.org/software/make/manual/, the GNU website}.
95 @subsection GUILE or Scheme
97 GUILE is the dialect of Scheme that is used as LilyPond's extension language. Many extensions to LilyPond are written entirely in GUILE. The
98 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html,
99 GUILE Reference Manual} is available online.
101 @uref{http://mitpress.mit.edu/sicp/full-text/book/book.html, Structure and
102 Interpretation of Computer Programs}, a popular textbook used to teach
103 programming in Scheme is available in its entirety online.
105 An introduction to Guile/Scheme as used in LilyPond can be found in the
106 Learning Manual, see @rlearning{Scheme tutorial}.
110 MetaFont is used to create the music fonts used by LilyPond. A MetaFont
111 tutorial is available at @uref{http://metafont.tutorial.free.fr/, the
112 METAFONT tutorial page}.
114 @subsection PostScript
116 PostScript is used to generate graphical output. A brief PostScript tutorial
117 is @uref{http://local.wasp.uwa.edu.au/~pbourke/dataformats/postscript/,
118 available online}. The
119 @uref{http://www.adobe.com/devnet/postscript/pdfs/PLRM.pdf, PostScript Lanugage
120 Reference} is available online in PDF format.
124 Python is used for XML2ly and is used for buillding the documentation and the
127 Python documentation is available at @uref{http://www.python.org/doc/,
130 @node Programming without compiling
131 @section Programming without compiling
133 Much of the development work in LilyPond takes place by changing *.ly or
134 *.scm files. These changes can be made without compiling LilyPond. Such
135 changes are described in this section.
138 @subsection Modifying distribution files
140 Much of LilyPond is written in Scheme or LilyPond input files. These
141 files are interpreted when the program is run, rather than being compiled
142 when the program is built, and are present in all LilyPond distributions.
143 You will find .ly files in the ly/ directory and the Scheme files in the
144 scm/ directory. Both Scheme files and .ly files can be modified and
145 saved with any text editor. It's probably wise to make a backup copy of
146 your files before you modify them, although you can reinstall if the
147 files become corrupted.
149 Once you've modified the files, you can test the changes just by running
150 LilyPond on some input file. It's a good idea to create a file that
151 demonstrates the feature you're trying to add. This file will eventually
152 become a regression test and will be part of the LilyPond distribution.
154 @subsection Desired file formatting
156 Files that are part of the LilyPond distribution have Unix-style line
157 endings (LF), rather than DOS (CR+LF) or MacOS 9 and earlier (CR). Make
158 sure you use the necessary tools to ensure that Unix-style line endings are
159 preserved in the patches you create.
161 Tab characters should not be included in files for distribution. All
162 indentation should be done with spaces. Most editors have settings to
163 allow the setting of tab stops and ensuring that no tab characters are
164 included in the file.
166 Scheme files and LilyPond files should be written according to standard
167 style guidelines. Scheme file guidelines can be found at
168 @uref{http://community.schemewiki.org/?scheme-style}. Following these
169 guidelines will make your code easier to read. Both you and others that
170 work on your code will be glad you followed these guidelines.
172 For LilyPond files, you should follow the guidelines for LilyPond snippets
173 in the documentation. You can find these guidelines at
174 @ref{Texinfo introduction and usage policy}.
176 @node Finding functions
177 @section Finding functions
179 When making changes or fixing bugs in LilyPond, one of the initial
180 challenges is finding out where in the code tree the functions to
181 be modified live. With nearly 3000 files in the source tree,
182 trial-and-error searching is generally ineffective. This section
183 describes a process for finding interesting code.
185 @subsection Using the ROADMAP
187 The file ROADMAP is located in the main directory of the lilypond source.
188 ROADMAP lists all of the directories in the LilPond source tree, along
189 with a brief description of the kind of files found in each directory.
190 This can be a very helpful tool for deciding which directories to search
191 when looking for a function.
194 @subsection Using grep to search
196 Having identified a likely subdirectory to search, the grep utility can
197 be used to search for a function name. The format of the grep command is
200 grep -i functionName subdirectory/*
203 This command will search all the contents of the directory subdirectory/
204 and display every line in any of the files that contains
205 functionName. The @code{-i} option makes @command{grep} ignore
206 case -- this can be very useful if you are not yet familiar with
207 our capitalization conventions.
209 The most likely directories to grep for function names are scm/ for
210 scheme files, ly/ for lilypond input (*.ly) files, and lily/ for C++
214 @subsection Using git grep to search
216 If you have used git to obtain the source, you have access to a
217 powerful tool to search for functions. The command:
220 git grep functionName
223 will search through all of the files that are present in the git
224 repository looking for functionName. It also presents the results
225 of the search using @code{less}, so the results are displayed one page
228 @subsection Searching on the git repository at Savannah
230 You can also use the equivalent of git grep on the Savannah server.
235 Go to http://git.sv.gnu.org/gitweb/?p=lilypond.git
238 In the pulldown box that says commit, select grep.
241 Type functionName in the search box, and hit enter/return
245 This will initiate a search of the remote git repository.
250 @c email to wl@gnu.org when I get here.
255 @subsection Handling errors
257 As a general rule, you should always try to continue computations,
258 even if there is some kind of error. When the program stops, it
259 is often very hard for a user to pinpoint what part of the input
260 causes an error. Finding the culprit is much easier if there is
261 some viewable output.
263 So functions and methods do not return errorcodes, they never
264 crash, but report a programming_error and try to carry on.
266 @subsection Languages
268 C++ and Python are preferred. Python code should use PEP 8.
270 @subsection Filenames
272 Definitions of classes that are only accessed via pointers (*) or
273 references (&) shall not be included as include files.
279 ".cc" Implementation files
280 ".icc" Inline definition files
281 ".tcc" non inline Template defs
285 (setq auto-mode-alist
286 (append '(("\\.make$" . makefile-mode)
287 ("\\.cc$" . c++-mode)
288 ("\\.icc$" . c++-mode)
289 ("\\.tcc$" . c++-mode)
290 ("\\.hh$" . c++-mode)
291 ("\\.pod$" . text-mode)
296 The class Class_name is coded in @q{class-name.*}
298 @subsection Indentation
300 Standard GNU coding style is used. In emacs:
303 (add-hook 'c++-mode-hook
304 '(lambda() (c-set-style "gnu")
308 If you like using font-lock, you can also add this to your
312 (setq font-lock-maximum-decoration t)
313 (setq c++-font-lock-keywords-3
315 c++-font-lock-keywords-3
316 '(("\\b\\(a-zA-Z_?+_\\)\\b" 1 font-lock-variable-name-face) ("\\b\\(A-Z?+a-z_?+\\)\\b" 1 font-lock-type-face))
321 @subsection Classes and Types
330 Member variable names end with an underscore:
339 Macro names should be written in uppercase completely.
342 @subsection Broken code
344 Do not write broken code. This includes hardwired dependencies,
345 hardwired constants, slow algorithms and obvious limitations. If
346 you can not avoid it, mark the place clearly, and add a comment
347 explaining shortcomings of the code.
349 We reject broken-in-advance on principle.
356 Messages need to follow Localization.
359 @subsection Localization
361 This document provides some guidelines for programmers write user
362 messages. To help translations, user messages must follow
363 uniform conventions. Follow these rules when coding for LilyPond.
364 Hopefully, this can be replaced by general GNU guidelines in the
365 future. Even better would be to have an English (en_BR, en_AM)
366 guide helping programmers writing consistent messages for all GNU
369 Non-preferred messages are marked with `+'. By convention,
370 ungrammatical examples are marked with `*'. However, such ungrammatical
371 examples may still be preferred.
376 Every message to the user should be localized (and thus be marked
377 for localization). This includes warning and error messages.
380 Don't localize/gettextify:
384 `programming_error ()'s
387 `programming_warning ()'s
393 output strings (PostScript, TeX, etc.)
398 Messages to be localised must be encapsulated in `_ (STRING)' or
399 `_f (FORMAT, ...)'. E.g.:
402 warning (_ ("need music in a score"));
403 error (_f ("cannot open file: `%s'", file_name));
406 In some rare cases you may need to call `gettext ()' by hand. This
407 happens when you pre-define (a list of) string constants for later
408 use. In that case, you'll probably also need to mark these string
409 constants for translation, using `_i (STRING)'. The `_i' macro is
410 a no-op, it only serves as a marker for `xgettext'.
413 char const* messages[] = @{
414 _i ("enable debugging output"),
415 _i ("ignore lilypond version"),
422 puts (gettext (messages i));
426 See also `flower/getopt-long.cc' and `lily/main.cc'.
429 Do not use leading or trailing whitespace in messages. If you need
430 whitespace to be printed, prepend or append it to the translated
434 message ("Calculating line breaks..." + " ");
438 Error or warning messages displayed with a file name and line
439 number never start with a capital, eg,
442 foo.ly: 12: not a duration: 3
445 Messages containing a final verb, or a gerund (`-ing'-form) always
446 start with a capital. Other (simpler) messages start with a
452 Not declaring: `foo'.
456 Avoid abbreviations or short forms, use `cannot' and `do not'
457 rather than `can't' or `don't'
458 To avoid having a number of different messages for the same
459 situation, well will use quoting like this `"message: `%s'"' for all
460 strings. Numbers are not quoted:
463 _f ("cannot open file: `%s'", name_str)
464 _f ("cannot find character number: %d", i)
468 Think about translation issues. In a lot of cases, it is better to
469 translate a whole message. The english grammar must not be imposed
470 on the translator. So, instead of
473 stem at + moment.str () + does not fit in beam
479 _f ("stem at %s does not fit in beam", moment.str ())
483 Split up multi-sentence messages, whenever possible. Instead of
486 warning (_f ("out of tune! Can't find: `%s'", "Key_engraver"));
487 warning (_f ("cannot find font `%s', loading default", font_name));
493 warning (_ ("out of tune:"));
494 warning (_f ("cannot find: `%s', "Key_engraver"));
495 warning (_f ("cannot find font: `%s', font_name));
496 warning (_f ("Loading default font"));
500 If you must have multiple-sentence messages, use full punctuation.
501 Use two spaces after end of sentence punctuation. No punctuation
502 (esp. period) is used at the end of simple messages.
505 _f ("Non-matching braces in text `%s', adding braces", text)
506 _ ("Debug output disabled. Compiled with NPRINT.")
507 _f ("Huh? Not a Request: `%s'. Ignoring.", request)
511 Do not modularise too much; words frequently cannot be translated
512 without context. It is probably safe to treat most occurences of
513 words like stem, beam, crescendo as separately translatable words.
516 When translating, it is preferable to put interesting information
517 at the end of the message, rather than embedded in the middle.
518 This especially applies to frequently used messages, even if this
519 would mean sacrificing a bit of eloquency. This holds for original
520 messages too, of course.
523 en: cannot open: `foo.ly'
524 + nl: kan `foo.ly' niet openen (1)
525 kan niet openen: `foo.ly'* (2)
526 niet te openen: `foo.ly'* (3)
530 The first nl message, although grammatically and stylistically
531 correct, is not friendly for parsing by humans (even if they speak
532 dutch). I guess we would prefer something like (2) or (3).
535 Do not run make po/po-update with GNU gettext < 0.10.35
541 @node Debugging LilyPond
542 @section Debugging LilyPond
544 The most commonly used tool for debugging LilyPond is the GNU debugger
545 gdb. Use of gdb is described in this section.
547 @subsection Debugging overview
549 Using a debugger simplifies troubleshooting in at least two ways.
551 First, breakpoints can be set to pause execution at any desired point.
552 Then, when execution has paused, debugger commands can be issued to
553 explore the values of various variables or to execute functions.
555 Second, the debugger allows the display of a stack trace, which shows
556 the sequence in which functions are called and the arguments to the
557 various function calls.
560 @subsection Compiling with debugging information
562 In order to use a debugger with LilyPond, it is necessary to compile
563 LilyPond with debugging information. This is accomplished by ...
565 TODO -- get good description here, or perhaps add debugging compile
566 to AU1.1 as it comes to CG and just use a reference here.
568 TODO -- Test the following to make sure it is true.
570 If you want to be able to set breakpoints in Scheme functions, it is
571 necessary to compile guile with debugging information. This is done
574 TODO -- get compiling description for guile here.
576 @subsection Typical gdb usage
578 @subsection Typical .gdbinit files
580 The behavior of gdb can be readily customized through the use of
581 @var{.gdbinit} files. A @var{.gdbinit} file is a file named
582 @var{.gdbinit} (notice the @qq{.} at the beginning of the file name)
583 that is placed in a user's home directory.
585 The @var{.gdbinit} file below is from Han-Wen. It sets breakpoints
586 for all errors and defines functions for displaying scheme objects
587 (ps), grobs (pgrob), and parsed music expressions (pmusic).
590 file lily/out/lilypond
593 b Grob::programming_error
596 print ly_display_scm($arg0)
599 print ly_display_scm($arg0->self_scm_)
600 print ly_display_scm($arg0->mutable_property_alist_)
601 print ly_display_scm($arg0->immutable_property_alist_)
602 print ly_display_scm($arg0->object_alist_)
605 print ly_display_scm($arg0->self_scm_)
606 print ly_display_scm($arg0->mutable_property_alist_)
607 print ly_display_scm($arg0->immutable_property_alist_)
611 @subsection Using Guile interactively with LilyPond
613 In order to experiment with Scheme programming in the LilyPond
614 environment, it is convenient to have a Guile interpreter that
615 has all the LilyPond modules loaded. This requires the following
618 First, define a Scheme symbol for the active module
622 #(module-define! (resolve-module '(guile-user))
623 'lilypond-module (current-module))
626 Second, place a Scheme function in the .ly file that gives an interactive Guile
633 When the .ly file is compiled, this causes the compilation to be interrupted
634 and an interactive guile prompt to appear. When the guile prompt appears,
635 the LilyPond active module must be set as the current guile module:
638 guile> (set-current-module lilypond-module)
641 Proper operation of these commands can be demonstrated by typing the name
642 of a LilyPond public scheme function to see if it's properly defined:
645 guile> fret-diagram-verbose-markup
646 #<procedure fret-diagram-verbose-markup (layout props marking-list)>
649 If the LilyPond module has not been correctly loaded, an error
650 message will be generated:
653 guile> fret-diagram-verbose-markup
654 ERROR: Unbound variable: fret-diagram-verbose-markup
655 ABORT: (unbound-variable)
658 Once the module is properly loaded, any valid LilyPond Scheme expression
659 can be entered at the interactive prompt.
661 After the investigation is complete, the interactive guile interpreter
668 The compilation of the .ly file will then continue.