1 @c -*- coding: us-ascii; mode: texinfo; -*-
3 @chapter Programming work
6 * Overview of LilyPond architecture::
7 * LilyPond programming languages::
8 * Programming without compiling::
11 * Debugging LilyPond::
14 @node Overview of LilyPond architecture
15 @section Overview of LilyPond architecture
17 LilyPond processes the input file into graphical and musical output in a
18 number of stages. This process, along with the types of routines that
19 accomplish the various stages of the process, is described in this section. A
20 more complete description of the LilyPond architecture and internal program
21 execution is found in Erik Sandberg's
22 @uref{http://lilypond.org/web/images/thesis-erik-sandberg.pdf, master's
26 The first stage of LilyPond processing is @emph{parsing}. In the parsing
27 process, music expressions in LilyPond input format are converted to music
28 expressions in Scheme format. In Scheme format, a music expression is a list
29 in tree form, with nodes that indicate the relationships between various music
30 events. The LilyPond parser is written in Bison.
32 The second stage of LilyPond processing is @emph{iterating}. Iterating
33 assigns each music event to a context, which is the environment in which the
34 music will be finally engraved. The context is responsible for all further
35 processing of the music. It is during the iteration stage that contexts are
36 created as necessary to ensure that every note has a Voice type context (e.g.
37 Voice, TabVoice, DrumVoice, CueVoice, MensuralVoice, VaticanaVoice,
38 GregorianTranscriptionVoice), that the Voice type contexts exist in
39 appropriate Staff type contexts, and that parallel Staff type contexts exist
40 in StaffGroup type contexts. In addition, during the iteration stage each
41 music event is assigned a moment, or a time in the music when the event
44 Each type of music event has an associated iterator. Iterators are defined in
45 *-iterator.cc. During iteration, an
46 event's iterator is called to deliver that music event to the appropriate
49 The final stage of LilyPond processing is @emph{translation}. During
50 translation, music events are prepared for graphical or midi output. The
51 translation step is accomplished by translators or engravers (the distinction
54 Translators are defined in C++ files named *-engraver.cc. In *-engraver.cc, a
55 C++ class of Engraver type is created. The Engraver is also declared as a
56 translator. Much of the work of translating is handled by Scheme functions,
57 which is one of the keys to LilyPond's exceptional flexibility.
60 @node LilyPond programming languages
61 @section LilyPond programming languages
63 Programming in LilyPond is done in a variety of programming languages. Each
64 language is used for a specific purpose or purposes. This section describes
65 the languages used and provides links to reference manuals and tutorials for
66 the relevant language.
70 The core functionality of LilyPond is implemented in C++.
72 C++ is so ubiquitous that it is difficult to identify either a reference
73 manual or a tutorial. Programmers unfamiliar with C++ will need to spend some
74 time to learn the language before attempting to modify the C++ code.
76 The C++ code calls Scheme/GUILE through the GUILE interface, which is
78 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html, GUILE
83 The LilyPond parser is implemented in Bison, a GNU parser generator. The
84 Bison homepage is found at @uref{http://www.gnu.org/software/bison/,
85 gnu.org}. The manual (which includes both a reference and tutorial) is
86 @uref{http://www.gnu.org/software/bison/manual/index.html, available} in a
91 GNU Make is used to control the compiling process and to build the
92 documentation and the website. GNU Make documentation is available at
93 @uref{http://www.gnu.org/software/make/manual/, the GNU website}.
95 @subsection GUILE or Scheme
97 GUILE is the dialect of Scheme that is used as LilyPond's extension language. Many extensions to LilyPond are written entirely in GUILE. The
98 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html,
99 GUILE Reference Manual} is available online.
101 @uref{http://mitpress.mit.edu/sicp/full-text/book/book.html, Structure and
102 Interpretation of Computer Programs}, a popular textbook used to teach
103 programming in Scheme is available in its entirety online.
107 MetaFont is used to create the music fonts used by LilyPond. A MetaFont
108 tutorial is available at @uref{http://metafont.tutorial.free.fr/, the
109 METAFONT tutorial page}.
111 @subsection PostScript
113 PostScript is used to generate graphical output. A brief PostScript tutorial
114 is @uref{http://local.wasp.uwa.edu.au/~pbourke/dataformats/postscript/,
115 available online}. The
116 @uref{http://www.adobe.com/devnet/postscript/pdfs/PLRM.pdf, PostScript Lanugage
117 Reference} is available online in PDF format.
121 Python is used for XML2ly and is used for buillding the documentation and the
124 Python documentation is available at @uref{http://www.python.org/doc/,
127 @node Programming without compiling
128 @section Programming without compiling
130 Much of the development work in LilyPond takes place by changing *.ly or
131 *.scm files. These changes can be made without compiling LilyPond. Such
132 changes are described in this section.
135 @subsection Modifying distribution files
137 Much of LilyPond is written in Scheme or LilyPond input files. These
138 files are interpreted when the program is run, rather than being compiled
139 when the program is built, and are present in all LilyPond distributions.
140 You will find .ly files in the ly/ directory and the Scheme files in the
141 scm/ directory. Both Scheme files and .ly files can be modified and
142 saved with any text editor. It's probably wise to make a backup copy of
143 your files before you modify them, although you can reinstall if the
144 files become corrupted.
146 Once you've modified the files, you can test the changes just by running
147 LilyPond on some input file. It's a good idea to create a file that
148 demonstrates the feature you're trying to add. This file will eventually
149 become a regression test and will be part of the LilyPond distribution.
151 @subsection Desired file formatting
153 Files that are part of the LilyPond distribution have Unix-style line
154 endings (LF), rather than DOS (CR+LF) or MacOS 9 and earlier (CR). Make
155 sure you use the necessary tools to ensure that Unix-style line endings are
156 preserved in the patches you create.
158 Tab characters should not be included in files for distribution. All
159 indentation should be done with spaces. Most editors have settings to
160 allow the setting of tab stops and ensuring that no tab characters are
161 included in the file.
163 Scheme files and LilyPond files should be written according to standard
164 style guidelines. Scheme file guidelines can be found at
165 @uref{http://community.schemewiki.org/?scheme-style}. Following these
166 guidelines will make your code easier to read. Both you and others that
167 work on your code will be glad you followed these guidelines.
169 For LilyPond files, you should follow the guidelines for LilyPond snippets
170 in the documentation. You can find these guidelines at
171 @ref{Texinfo introduction and usage policy}.
173 @node Finding functions
174 @section Finding functions
176 When making changes or fixing bugs in LilyPond, one of the initial
177 challenges is finding out where in the code tree the functions to
178 be modified live. With nearly 3000 files in the source tree,
179 trial-and-error searching is generally ineffective. This section
180 describes a process for finding interesting code.
182 @subsection Using the ROADMAP
184 The file ROADMAP is located in the main directory of the lilypond source.
185 ROADMAP lists all of the directories in the LilPond source tree, along
186 with a brief description of the kind of files found in each directory.
187 This can be a very helpful tool for deciding which directories to search
188 when looking for a function.
191 @subsection Using grep to search
193 Having identified a likely subdirectory to search, the grep utility can
194 be used to search for a function name. The format of the grep command is
197 grep -i functionName subdirectory/*
200 This command will search all the contents of the directory subdirectory/
201 and display every line in any of the files that contains
202 functionName. The @code{-i} option makes @command{grep} ignore
203 case -- this can be very useful if you are not yet familiar with
204 our capitalization conventions.
206 The most likely directories to grep for function names are scm/ for
207 scheme files, ly/ for lilypond input (*.ly) files, and lily/ for C++
211 @subsection Using git grep to search
213 If you have used git to obtain the source, you have access to a
214 powerful tool to search for functions. The command:
217 git grep functionName
220 will search through all of the files that are present in the git
221 repository looking for functionName. It also presents the results
222 of the search using @code{less}, so the results are displayed one page
225 @subsection Searching on the git repository at Savannah
227 You can also use the equivalent of git grep on the Savannah server.
232 Go to http://git.sv.gnu.org/gitweb/?p=lilypond.git
235 In the pulldown box that says commit, select grep.
238 Type functionName in the search box, and hit enter/return
242 This will initiate a search of the remote git repository.
247 @c email to wl@gnu.org when I get here.
252 @subsection Handling errors
254 As a general rule, you should always try to continue computations,
255 even if there is some kind of error. When the program stops, it
256 is often very hard for a user to pinpoint what part of the input
257 causes an error. Finding the culprit is much easier if there is
258 some viewable output.
260 So functions and methods do not return errorcodes, they never
261 crash, but report a programming_error and try to carry on.
263 @subsection Languages
265 C++ and Python are preferred. Python code should use PEP 8.
267 @subsection Filenames
269 Definitions of classes that are only accessed via pointers (*) or
270 references (&) shall not be included as include files.
276 ".cc" Implementation files
277 ".icc" Inline definition files
278 ".tcc" non inline Template defs
282 (setq auto-mode-alist
283 (append '(("\\.make$" . makefile-mode)
284 ("\\.cc$" . c++-mode)
285 ("\\.icc$" . c++-mode)
286 ("\\.tcc$" . c++-mode)
287 ("\\.hh$" . c++-mode)
288 ("\\.pod$" . text-mode)
293 The class Class_name is coded in @q{class-name.*}
295 @subsection Indentation
297 Standard GNU coding style is used. In emacs:
300 (add-hook 'c++-mode-hook
301 '(lambda() (c-set-style "gnu")
305 If you like using font-lock, you can also add this to your
309 (setq font-lock-maximum-decoration t)
310 (setq c++-font-lock-keywords-3
312 c++-font-lock-keywords-3
313 '(("\\b\\(a-zA-Z_?+_\\)\\b" 1 font-lock-variable-name-face) ("\\b\\(A-Z?+a-z_?+\\)\\b" 1 font-lock-type-face))
318 @subsection Classes and Types
327 Member variable names end with an underscore:
336 Macro names should be written in uppercase completely.
339 @subsection Broken code
341 Do not write broken code. This includes hardwired dependencies,
342 hardwired constants, slow algorithms and obvious limitations. If
343 you can not avoid it, mark the place clearly, and add a comment
344 explaining shortcomings of the code.
346 We reject broken-in-advance on principle.
353 Messages need to follow Localization.
356 @subsection Localization
358 This document provides some guidelines for programmers write user
359 messages. To help translations, user messages must follow
360 uniform conventions. Follow these rules when coding for LilyPond.
361 Hopefully, this can be replaced by general GNU guidelines in the
362 future. Even better would be to have an English (en_BR, en_AM)
363 guide helping programmers writing consistent messages for all GNU
366 Non-preferred messages are marked with `+'. By convention,
367 ungrammatical examples are marked with `*'. However, such ungrammatical
368 examples may still be preferred.
373 Every message to the user should be localized (and thus be marked
374 for localization). This includes warning and error messages.
377 Don't localize/gettextify:
381 `programming_error ()'s
384 `programming_warning ()'s
390 output strings (PostScript, TeX, etc.)
395 Messages to be localised must be encapsulated in `_ (STRING)' or
396 `_f (FORMAT, ...)'. E.g.:
399 warning (_ ("need music in a score"));
400 error (_f ("cannot open file: `%s'", file_name));
403 In some rare cases you may need to call `gettext ()' by hand. This
404 happens when you pre-define (a list of) string constants for later
405 use. In that case, you'll probably also need to mark these string
406 constants for translation, using `_i (STRING)'. The `_i' macro is
407 a no-op, it only serves as a marker for `xgettext'.
410 char const* messages[] = @{
411 _i ("enable debugging output"),
412 _i ("ignore lilypond version"),
419 puts (gettext (messages i));
423 See also `flower/getopt-long.cc' and `lily/main.cc'.
426 Do not use leading or trailing whitespace in messages. If you need
427 whitespace to be printed, prepend or append it to the translated
431 message ("Calculating line breaks..." + " ");
435 Error or warning messages displayed with a file name and line
436 number never start with a capital, eg,
439 foo.ly: 12: not a duration: 3
442 Messages containing a final verb, or a gerund (`-ing'-form) always
443 start with a capital. Other (simpler) messages start with a
449 Not declaring: `foo'.
453 Avoid abbreviations or short forms, use `cannot' and `do not'
454 rather than `can't' or `don't'
455 To avoid having a number of different messages for the same
456 situation, well will use quoting like this `"message: `%s'"' for all
457 strings. Numbers are not quoted:
460 _f ("cannot open file: `%s'", name_str)
461 _f ("cannot find character number: %d", i)
465 Think about translation issues. In a lot of cases, it is better to
466 translate a whole message. The english grammar must not be imposed
467 on the translator. So, instead of
470 stem at + moment.str () + does not fit in beam
476 _f ("stem at %s does not fit in beam", moment.str ())
480 Split up multi-sentence messages, whenever possible. Instead of
483 warning (_f ("out of tune! Can't find: `%s'", "Key_engraver"));
484 warning (_f ("cannot find font `%s', loading default", font_name));
490 warning (_ ("out of tune:"));
491 warning (_f ("cannot find: `%s', "Key_engraver"));
492 warning (_f ("cannot find font: `%s', font_name));
493 warning (_f ("Loading default font"));
497 If you must have multiple-sentence messages, use full punctuation.
498 Use two spaces after end of sentence punctuation. No punctuation
499 (esp. period) is used at the end of simple messages.
502 _f ("Non-matching braces in text `%s', adding braces", text)
503 _ ("Debug output disabled. Compiled with NPRINT.")
504 _f ("Huh? Not a Request: `%s'. Ignoring.", request)
508 Do not modularise too much; words frequently cannot be translated
509 without context. It is probably safe to treat most occurences of
510 words like stem, beam, crescendo as separately translatable words.
513 When translating, it is preferable to put interesting information
514 at the end of the message, rather than embedded in the middle.
515 This especially applies to frequently used messages, even if this
516 would mean sacrificing a bit of eloquency. This holds for original
517 messages too, of course.
520 en: cannot open: `foo.ly'
521 + nl: kan `foo.ly' niet openen (1)
522 kan niet openen: `foo.ly'* (2)
523 niet te openen: `foo.ly'* (3)
527 The first nl message, although grammatically and stylistically
528 correct, is not friendly for parsing by humans (even if they speak
529 dutch). I guess we would prefer something like (2) or (3).
532 Do not run make po/po-update with GNU gettext < 0.10.35
538 @node Debugging LilyPond
539 @section Debugging LilyPond
541 The most commonly used tool for debugging LilyPond is the GNU debugger
542 gdb. Use of gdb is described in this section.
544 @subsection Debugging overview
546 Using a debugger simplifies troubleshooting in at least two ways.
548 First, breakpoints can be set to pause execution at any desired point.
549 Then, when execution has paused, debugger commands can be issued to
550 explore the values of various variables or to execute functions.
552 Second, the debugger allows the display of a stack trace, which shows
553 the sequence in which functions are called and the arguments to the
554 various function calls.
557 @subsection Compiling with debugging information
559 In order to use a debugger with LilyPond, it is necessary to compile
560 LilyPond with debugging information. This is accomplished by ...
562 TODO -- get good description here, or perhaps add debugging compile
563 to AU1.1 as it comes to CG and just use a reference here.
565 TODO -- Test the following to make sure it is true.
567 If you want to be able to set breakpoints in Scheme functions, it is
568 necessary to compile guile with debugging information. This is done
571 TODO -- get compiling description for guile here.
573 @subsection Typical gdb usage
575 @subsection Typical .gdbinit files
577 The behavior of gdb can be readily customized through the use of
578 @var{.gdbinit} files. A @var{.gdbinit} file is a file named
579 @var{.gdbinit} (notice the @qq{.} at the beginning of the file name)
580 that is placed in a user's home directory.
582 The @var{.gdbinit} file below is from Han-Wen. It sets breakpoints
583 for all errors and defines functions for displaying scheme objects
584 (ps), grobs (pgrob), and parsed music expressions (pmusic).
587 file lily/out/lilypond
590 b Grob::programming_error
593 print ly_display_scm($arg0)
596 print ly_display_scm($arg0->self_scm_)
597 print ly_display_scm($arg0->mutable_property_alist_)
598 print ly_display_scm($arg0->immutable_property_alist_)
599 print ly_display_scm($arg0->object_alist_)
602 print ly_display_scm($arg0->self_scm_)
603 print ly_display_scm($arg0->mutable_property_alist_)
604 print ly_display_scm($arg0->immutable_property_alist_)
608 @subsection Using Guile interactively with LilyPond
610 In order to experiment with Scheme programming in the LilyPond
611 environment, it is convenient to have a Guile interpreter that
612 has all the LilyPond modules loaded. This requires the following
615 First, define a Scheme symbol for the active module
619 #(module-define! (resolve-module '(guile-user))
620 'lilypond-module (current-module))
623 Second, place a Scheme function in the .ly file that gives an interactive Guile
630 When the .ly file is compiled, this causes the compilation to be interrupted
631 and an interactive guile prompt to appear. When the guile prompt appears,
632 the LilyPond active module must be set as the current guile module:
635 guile> (set-current-module lilypond-module)
638 Proper operation of these commands can be demonstrated by typing the name
639 of a LilyPond public scheme function to see if it's properly defined:
642 guile> fret-diagram-verbose-markup
643 #<procedure fret-diagram-verbose-markup (layout props marking-list)>
646 If the LilyPond module has not been correctly loaded, an error
647 message will be generated:
650 guile> fret-diagram-verbose-markup
651 ERROR: Unbound variable: fret-diagram-verbose-markup
652 ABORT: (unbound-variable)
655 Once the module is properly loaded, any valid LilyPond Scheme expression
656 can be entered at the interactive prompt.
658 After the investigation is complete, the interactive guile interpreter
665 The compilation of the .ly file will then continue.