1 @c -*- coding: utf-8; mode: texinfo; -*-
3 @chapter Programming work
6 * Overview of LilyPond architecture::
7 * LilyPond programming languages::
8 * Programming without compiling::
11 * Warnings Errors Progress and Debug Output::
12 * Debugging LilyPond::
13 * Tracing object relationships::
14 * Adding or modifying features::
19 * LilyPond miscellany::
22 @node Overview of LilyPond architecture
23 @section Overview of LilyPond architecture
25 LilyPond processes the input file into graphical and musical output in a
26 number of stages. This process, along with the types of routines that
27 accomplish the various stages of the process, is described in this section. A
28 more complete description of the LilyPond architecture and internal program
29 execution is found in Erik Sandberg's
30 @uref{http://lilypond.org/web/images/thesis-erik-sandberg.pdf, master's
33 The first stage of LilyPond processing is @emph{parsing}. In the parsing
34 process, music expressions in LilyPond input format are converted to music
35 expressions in Scheme format. In Scheme format, a music expression is a list
36 in tree form, with nodes that indicate the relationships between various music
37 events. The LilyPond parser is written in Bison.
39 The second stage of LilyPond processing is @emph{iterating}. Iterating
40 assigns each music event to a context, which is the environment in which the
41 music will be finally engraved. The context is responsible for all further
42 processing of the music. It is during the iteration stage that contexts are
43 created as necessary to ensure that every note has a Voice type context (e.g.
44 Voice, TabVoice, DrumVoice, CueVoice, MensuralVoice, VaticanaVoice,
45 GregorianTranscriptionVoice), that the Voice type contexts exist in
46 appropriate Staff type contexts, and that parallel Staff type contexts exist
47 in StaffGroup type contexts. In addition, during the iteration stage each
48 music event is assigned a moment, or a time in the music when the event
51 Each type of music event has an associated iterator. Iterators are defined in
52 @file{*-iterator.cc}. During iteration, an
53 event's iterator is called to deliver that music event to the appropriate
56 The final stage of LilyPond processing is @emph{translation}. During
57 translation, music events are prepared for graphical or midi output. The
58 translation step is accomplished by the polymorphic base class Translator
59 through its two derived classes: Engraver (for graphical output) and
60 Performer (for midi output).
62 Translators are defined in C++ files named @file{*-engraver.cc}
63 and @file{*-performer.cc}.
64 Much of the work of translating is handled by Scheme functions,
65 which is one of the keys to LilyPond's exceptional flexibility.
67 @sourceimage{architecture-diagram,,,png}
70 @node LilyPond programming languages
71 @section LilyPond programming languages
73 Programming in LilyPond is done in a variety of programming languages. Each
74 language is used for a specific purpose or purposes. This section describes
75 the languages used and provides links to reference manuals and tutorials for
76 the relevant language.
80 The core functionality of LilyPond is implemented in C++.
82 C++ is so ubiquitous that it is difficult to identify either a reference
83 manual or a tutorial. Programmers unfamiliar with C++ will need to spend some
84 time to learn the language before attempting to modify the C++ code.
86 The C++ code calls Scheme/GUILE through the GUILE interface, which is
88 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html, GUILE
93 The LilyPond lexer is implemented in Flex, an implementation of the Unix lex
94 lexical analyser generator. Resources for Flex can be found
95 @uref{http://flex.sourceforge.net/, here}.
99 The LilyPond parser is implemented in Bison, a GNU parser generator. The
100 Bison homepage is found at @uref{http://www.gnu.org/software/bison/,
101 gnu.org}. The manual (which includes both a reference and tutorial) is
102 @uref{http://www.gnu.org/software/bison/manual/index.html, available} in a
107 GNU Make is used to control the compiling process and to build the
108 documentation and the website. GNU Make documentation is available at
109 @uref{http://www.gnu.org/software/make/manual/, the GNU website}.
111 @subsection GUILE or Scheme
113 GUILE is the dialect of Scheme that is used as LilyPond's extension language.
114 Many extensions to LilyPond are written entirely in GUILE. The
115 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html,
116 GUILE Reference Manual} is available online.
118 @uref{http://mitpress.mit.edu/sicp/full-text/book/book.html, Structure and
119 Interpretation of Computer Programs}, a popular textbook used to teach
120 programming in Scheme is available in its entirety online.
122 An introduction to Guile/Scheme as used in LilyPond can be found in the
123 @rextend{Scheme tutorial}.
127 MetaFont is used to create the music fonts used by LilyPond. A MetaFont
128 tutorial is available at @uref{http://metafont.tutorial.free.fr/, the
129 METAFONT tutorial page}.
131 @subsection PostScript
133 PostScript is used to generate graphical output. A brief PostScript tutorial
134 is @uref{http://local.wasp.uwa.edu.au/~pbourke/dataformats/postscript/,
135 available online}. The
136 @uref{http://www.adobe.com/devnet/postscript/pdfs/PLRM.pdf, PostScript Language
137 Reference} is available online in PDF format.
141 Python is used for XML2ly and is used for building the documentation and the
144 Python documentation is available at @uref{http://www.python.org/doc/,
147 @node Programming without compiling
148 @section Programming without compiling
150 Much of the development work in LilyPond takes place by changing @file{*.ly} or
151 @file{*.scm} files. These changes can be made without compiling LilyPond. Such
152 changes are described in this section.
155 @subsection Modifying distribution files
157 Much of LilyPond is written in Scheme or LilyPond input files. These
158 files are interpreted when the program is run, rather than being compiled
159 when the program is built, and are present in all LilyPond distributions.
160 You will find @file{.ly} files in the @file{ly/} directory and the Scheme files in the
161 @file{scm/} directory. Both Scheme files and @file{.ly} files can be modified and
162 saved with any text editor. It's probably wise to make a backup copy of
163 your files before you modify them, although you can reinstall if the
164 files become corrupted.
166 Once you've modified the files, you can test the changes just by running
167 LilyPond on some input file. It's a good idea to create a file that
168 demonstrates the feature you're trying to add. This file will eventually
169 become a regression test and will be part of the LilyPond distribution.
171 @subsection Desired file formatting
173 Files that are part of the LilyPond distribution have Unix-style line
174 endings (LF), rather than DOS (CR+LF) or MacOS 9 and earlier (CR). Make
175 sure you use the necessary tools to ensure that Unix-style line endings are
176 preserved in the patches you create.
178 Tab characters should not be included in files for distribution. All
179 indentation should be done with spaces. Most editors have settings to
180 allow the setting of tab stops and ensuring that no tab characters are
181 included in the file.
183 Scheme files and LilyPond files should be written according to standard
184 style guidelines. Scheme file guidelines can be found at
185 @uref{http://community.schemewiki.org/?scheme-style}. Following these
186 guidelines will make your code easier to read. Both you and others that
187 work on your code will be glad you followed these guidelines.
189 For LilyPond files, you should follow the guidelines for LilyPond snippets
190 in the documentation. You can find these guidelines at
191 @ref{Texinfo introduction and usage policy}.
193 @node Finding functions
194 @section Finding functions
196 When making changes or fixing bugs in LilyPond, one of the initial
197 challenges is finding out where in the code tree the functions to
198 be modified live. With nearly 3000 files in the source tree,
199 trial-and-error searching is generally ineffective. This section
200 describes a process for finding interesting code.
202 @subsection Using the ROADMAP
204 The file ROADMAP is located in the main directory of the lilypond source.
205 ROADMAP lists all of the directories in the LilyPond source tree, along
206 with a brief description of the kind of files found in each directory.
207 This can be a very helpful tool for deciding which directories to search
208 when looking for a function.
211 @subsection Using grep to search
213 Having identified a likely subdirectory to search, the grep utility can
214 be used to search for a function name. The format of the grep command is
217 grep -i functionName subdirectory/*
220 This command will search all the contents of the directory subdirectory/
221 and display every line in any of the files that contains
222 functionName. The @option{-i} option makes @command{grep} ignore
223 case -- this can be very useful if you are not yet familiar with
224 our capitalization conventions.
226 The most likely directories to grep for function names are @file{scm/} for
227 scheme files, ly/ for lilypond input (@file{*.ly}) files, and @file{lily/} for C++
231 @subsection Using git grep to search
233 If you have used git to obtain the source, you have access to a
234 powerful tool to search for functions. The command:
237 git grep functionName
240 will search through all of the files that are present in the git
241 repository looking for functionName. It also presents the results
242 of the search using @code{less}, so the results are displayed one page
245 @subsection Searching on the git repository at Savannah
247 You can also use the equivalent of git grep on the Savannah server.
252 Go to http://git.sv.gnu.org/gitweb/?p=lilypond.git
255 In the pulldown box that says commit, select grep.
258 Type functionName in the search box, and hit enter/return
262 This will initiate a search of the remote git repository.
268 This section describes style guidelines for LilyPond
275 * Naming conventions::
284 @subsection Languages
286 C++ and Python are preferred. Python code should use PEP 8.
290 @subsection Filenames
292 Definitions of classes that are only accessed via pointers (*) or
293 references (&) shall not be included as include files.
299 ".cc" Implementation files
300 ".icc" Inline definition files
301 ".tcc" non inline Template defs
305 (setq auto-mode-alist
306 (append '(("\\.make$" . makefile-mode)
307 ("\\.cc$" . c++-mode)
308 ("\\.icc$" . c++-mode)
309 ("\\.tcc$" . c++-mode)
310 ("\\.hh$" . c++-mode)
311 ("\\.pod$" . text-mode)
316 The class Class_name is coded in @q{class-name.*}
320 @subsection Indentation
322 Standard GNU coding style is used.
324 @subsubheading Indenting files with @code{fixcc.py} (recommended)
326 LilyPond provides a python script that will adjust the indentation
327 and spacing on a @code{.cc} or @code{.hh} file to very near the
331 scripts/auxiliar/fixcc.py FILENAME
334 This can be run on all files at once, but this is not recommended
335 for normal contributors or developers.
338 scripts/auxiliar/fixcc.py \
339 $(find flower lily -name '*cc' -o -name '*hh' | grep -v /out)
343 @subsubheading Indenting with emacs
345 The following hooks will produce indentation which is similar to
346 our official indentation as produced with @code{fixcc.py}.
349 (add-hook 'c++-mode-hook
352 (setq indent-tabs-mode nil))
355 If you like using font-lock, you can also add this to your
359 (setq font-lock-maximum-decoration t)
360 (setq c++-font-lock-keywords-3
362 c++-font-lock-keywords-3
363 '(("\\b\\(a-zA-Z_?+_\\)\\b" 1 font-lock-variable-name-face) ("\\b\\(A-Z?+a-z_?+\\)\\b" 1 font-lock-type-face))
368 @subheading Indenting with vim
370 Although emacs indentation is the GNU standard, acceptable
371 indentation can usually be accomplished with vim. Some hints for
383 filetype plugin indent on
385 set ignorecase smartcase
388 set statusline=%F%m%r%h%w\ %@{&ff@}\ %Y\ [ASCII=\%03.3b]\ [HEX=\%02.2B]\ %04l,%04v\ %p%%\ [LEN=%L]
391 " Remove trailing whitespace on write
392 autocmd BufWritePre * :%s/\s\+$//e
395 With this @file{.vimrc}, files can be reindented automatically by
396 highlighting the lines to be indented in visual mode (use V to
397 enter visual mode) and pressing @code{=}.
399 A @file{scheme.vim} file will help improve the indentation. This
400 one was suggested by Patrick McCarty. It should be saved in
401 @file{~/.vim/after/syntax/scheme.vim}.
404 " Additional Guile-specific 'forms'
405 syn keyword schemeSyntax define-public define*-public
406 syn keyword schemeSyntax define* lambda* let-keywords*
407 syn keyword schemeSyntax defmacro defmacro* define-macro
408 syn keyword schemeSyntax defmacro-public defmacro*-public
409 syn keyword schemeSyntax use-modules define-module
410 syn keyword schemeSyntax define-method define-class
412 " Additional LilyPond-specific 'forms'
413 syn keyword schemeSyntax define-markup-command define-markup-list-command
414 syn keyword schemeSyntax define-safe-public define-music-function
415 syn keyword schemeSyntax def-grace-function
417 " All of the above should influence indenting too
418 set lw+=define-public,define*-public
419 set lw+=define*,lambda*,let-keywords*
420 set lw+=defmacro,defmacro*,define-macro
421 set lw+=defmacro-public,defmacro*-public
422 set lw+=use-modules,define-module
423 set lw+=define-method,define-class
424 set lw+=define-markup-command,define-markup-list-command
425 set lw+=define-safe-public,define-music-function
426 set lw+=def-grace-function
428 " These forms should not influence indenting
432 " Try to highlight all ly: procedures
433 syn match schemeFunc "ly:[^) ]\+"
437 @node Naming conventions
438 @subsection Naming Conventions
440 Naming conventions have been established for LilyPond
443 @subheading Classes and Types
445 Classes begin with an uppercase letter, and words
446 in class names are separated with @code{_}:
454 Member variable names end with an underscore:
462 Macro names should be written in uppercase completely,
463 with words separated by @code{_}:
469 @subheading Variables
471 Variable names should be complete words, rather than abbreviations.
472 For example, it is preferred to use @code{thickness} rather than
473 @code{th} or @code{t}.
475 Multi-word variable names in C++ should have the words separated
476 by the underscore character (@q{_}):
479 cxx_multiword_variable
482 Multi-word variable names in Scheme should have the words separated
486 scheme-multiword-variable
490 @subsection Broken code
492 Do not write broken code. This includes hardwired dependencies,
493 hardwired constants, slow algorithms and obvious limitations. If
494 you can not avoid it, mark the place clearly, and add a comment
495 explaining shortcomings of the code.
497 Ideally, the comment marking the shortcoming would include
498 TODO, so that it is marked for future fixing.
500 We reject broken-in-advance on principle.
504 @subsection Code comments
506 Comments may not be needed if descriptive variable names are used
507 in the code and the logic is straightforward. However, if the
508 logic is difficult to follow, and particularly if non-obvious
509 code has been included to resolve a bug, a comment describing
510 the logic and/or the need for the non-obvious code should be included.
512 There are instances where the current code could be commented better.
513 If significant time is required to understand the code as part of
514 preparing a patch, it would be wise to add comments reflecting your
515 understanding to make future work easier.
518 @node Handling errors
519 @subsection Handling errors
521 As a general rule, you should always try to continue computations,
522 even if there is some kind of error. When the program stops, it
523 is often very hard for a user to pinpoint what part of the input
524 causes an error. Finding the culprit is much easier if there is
525 some viewable output.
527 So functions and methods do not return errorcodes, they never
528 crash, but report a programming_error and try to carry on.
530 Error and warning messages need to be localized.
534 @subsection Localization
536 This document provides some guidelines to help programmers write
538 messages. To help translations, user messages must follow
539 uniform conventions. Follow these rules when coding for LilyPond.
540 Hopefully, this can be replaced by general GNU guidelines in the
541 future. Even better would be to have an English (en_BR, en_AM)
542 guide helping programmers writing consistent messages for all GNU
545 Non-preferred messages are marked with `+'. By convention,
546 ungrammatical examples are marked with `*'. However, such ungrammatical
547 examples may still be preferred.
552 Every message to the user should be localized (and thus be marked
553 for localization). This includes warning and error messages.
556 Do not localize/gettextify:
560 `programming_error ()'s
563 `programming_warning ()'s
569 output strings (PostScript, TeX, etc.)
574 Messages to be localized must be encapsulated in `_ (STRING)' or
575 `_f (FORMAT, ...)'. E.g.:
578 warning (_ ("need music in a score"));
579 error (_f ("cannot open file: `%s'", file_name));
582 In some rare cases you may need to call `gettext ()' by hand. This
583 happens when you pre-define (a list of) string constants for later
584 use. In that case, you'll probably also need to mark these string
585 constants for translation, using `_i (STRING)'. The `_i' macro is
586 a no-op, it only serves as a marker for `xgettext'.
589 char const* messages[] = @{
590 _i ("enable debugging output"),
591 _i ("ignore lilypond version"),
598 puts (gettext (messages i));
602 See also @file{flower/getopt-long.cc} and @file{lily/main.cc}.
605 Do not use leading or trailing whitespace in messages. If you need
606 whitespace to be printed, prepend or append it to the translated
610 message ("Calculating line breaks..." + " ");
614 Error or warning messages displayed with a file name and line
615 number never start with a capital, eg,
618 foo.ly: 12: not a duration: 3
621 Messages containing a final verb, or a gerund (`-ing'-form) always
622 start with a capital. Other (simpler) messages start with a
628 Not declaring: `foo'.
632 Avoid abbreviations or short forms, use `cannot' and `do not'
633 rather than `can't' or `don't'
634 To avoid having a number of different messages for the same
635 situation, well will use quoting like this `"message: `%s'"' for all
636 strings. Numbers are not quoted:
639 _f ("cannot open file: `%s'", name_str)
640 _f ("cannot find character number: %d", i)
644 Think about translation issues. In a lot of cases, it is better to
645 translate a whole message. English grammar must not be imposed on the
646 translator. So, instead of
649 stem at + moment.str () + does not fit in beam
655 _f ("stem at %s does not fit in beam", moment.str ())
659 Split up multi-sentence messages, whenever possible. Instead of
662 warning (_f ("out of tune! Can't find: `%s'", "Key_engraver"));
663 warning (_f ("cannot find font `%s', loading default", font_name));
669 warning (_ ("out of tune:"));
670 warning (_f ("cannot find: `%s', "Key_engraver"));
671 warning (_f ("cannot find font: `%s', font_name));
672 warning (_f ("Loading default font"));
676 If you must have multiple-sentence messages, use full punctuation.
677 Use two spaces after end of sentence punctuation. No punctuation
678 (esp. period) is used at the end of simple messages.
681 _f ("Non-matching braces in text `%s', adding braces", text)
682 _ ("Debug output disabled. Compiled with NPRINT.")
683 _f ("Huh? Not a Request: `%s'. Ignoring.", request)
687 Do not modularize too much; words frequently cannot be translated
688 without context. It is probably safe to treat most occurrences of
689 words like stem, beam, crescendo as separately translatable words.
692 When translating, it is preferable to put interesting information
693 at the end of the message, rather than embedded in the middle.
694 This especially applies to frequently used messages, even if this
695 would mean sacrificing a bit of eloquency. This holds for original
696 messages too, of course.
699 en: cannot open: `foo.ly'
700 + nl: kan `foo.ly' niet openen (1)
701 kan niet openen: `foo.ly'* (2)
702 niet te openen: `foo.ly'* (3)
706 The first nl message, although grammatically and stylistically
707 correct, is not friendly for parsing by humans (even if they speak
708 dutch). I guess we would prefer something like (2) or (3).
711 Do not run make po/po-update with GNU gettext < 0.10.35
716 @node Warnings Errors Progress and Debug Output
717 @section Warnings, Errors, Progress and Debug Output
719 @unnumberedsubsec Available log levels
721 LilyPond has several loglevels, which specify how verbose the output on
722 the console should be:
724 @item NONE: No output at all, even on failure
725 @item ERROR: Only error messages
726 @item WARN: Only error messages and warnings
727 @item BASIC_PROGRESS: Warnings, errors and basic progress (success, etc.)
728 @item PROGRESS: Warnings, errors and full progress messages
729 @item INFO: Warnings, errors, progress and more detailed information (default)
730 @item DEBUG: All messages, including vull debug messages (very verbose!)
733 The loglevel can either be set with the environment variable
734 @code{LILYPOND_LOGLEVEL} or on the command line with the @option{--loglevel=...}
737 @unnumberedsubsec Functions for debug and log output
739 LilyPond has two different types of error and log functions:
743 If a warning or error is caused by an identified position in the input file,
744 e.g. by a grob or by a music expression, the functions of the @code{Input}
745 class provide logging functionality that prints the position of the message
746 in addition to the message.
749 If a message can not be associated with a particular position in an input file,
750 e.g. the output file cannot be written, then the functions in the
751 @code{flower/include/warn.hh} file will provide logging functionality that
752 only prints out the message, but no location.
756 There are also Scheme functions to access all of these logging functions from
757 scheme. In addition, the Grob class contains some convenience wrappers for
758 even easier access to these functions.
760 The message and debug functions in @code{warn.hh} also have an optional
761 argument @code{newline}, which specifies whether the message should always
762 start on a new line or continue a previous message.
763 By default, @code{progress_indication} does NOT start on a new line, but rather
764 continue the previous output. They also do not have a particular input
765 position associated, so there are no progress functions in the Input class.
766 All other functions by default start their output on a new line.
768 The error functions come in three different flavors: fatal error messages,
769 programming error messages and normal error messages. Errors written
770 by the @code{error ()} function will cause LilyPond to exit immediately,
771 errors by @code{Input::error ()} will continue the compilation, but
772 return a non-zero return value of the lilypond call (i.e. indicate an
773 unsuccessful program execution). All other errors will be printed on the
774 console, but not exit LilyPond or indicate an unsuccessful return code.
775 Their only differences to a warnings are the displayed text and that
776 they will be shown with loglevel @code{ERROR}.
778 If the Scheme option @code{warning-as-error} is set, any warning will be
779 treated as if @code{Input::error} was called.
782 @unnumberedsubsec All logging functions at a glance
784 @multitable @columnfractions 0.16 0.42 0.42
786 @tab C++, no location
787 @tab C++ from input location
790 @tab @code{error ()}, @code{programming_error (msg)}, @code{non_fatal_error (msg)}
791 @tab @code{Input::error (msg)}, @code{Input::programming_error (msg)}
794 @tab @code{warning (msg)}
795 @tab @code{Input::warning (msg)}
798 @tab @code{successful (msg)}
802 @tab @code{progress_indication (msg)}
806 @tab @code{@code{message (msg)}
807 @tab @code{Input::message (msg)}
810 @tab @code{debug_output (msg)}
811 @tab @code{Input::debug_output (msg)}
817 @tab Scheme, music expression
820 @tab @code{Grob::programming_error (msg)}
824 @tab @code{Grob::warning (msg)}
825 @tab @code{(ly:music-warning music msg)}
837 @tab @code{(ly:music-message music msg)}
846 @tab Scheme, no location
847 @tab Scheme, input location
851 @tab @code{(ly:error msg args)}, @code{(ly:programming-error msg args)}
854 @tab @code{(ly:warning msg args)}
855 @tab @code{(ly:input-warning input msg args)}
858 @tab @code{(ly:success msg args)}
862 @tab @code{(ly:progress msg args)}
866 @tab @code{(ly:message msg args)}
867 @tab @code{(ly:input-message input msg args)}
870 @tab @code{(ly:debug msg args)}
878 @node Debugging LilyPond
879 @section Debugging LilyPond
881 The most commonly used tool for debugging LilyPond is the GNU
882 debugger gdb. The gdb tool is used for investigating and debugging
883 core Lilypond code written in C++. Another tool is available for
884 debugging Scheme code using the Guile debugger. This section
885 describes how to use both gdb and the Guile Debugger.
888 * Debugging overview::
889 * Debugging C++ code::
890 * Debugging Scheme code::
893 @node Debugging overview
894 @subsection Debugging overview
896 Using a debugger simplifies troubleshooting in at least two ways.
898 First, breakpoints can be set to pause execution at any desired point.
899 Then, when execution has paused, debugger commands can be issued to
900 explore the values of various variables or to execute functions.
902 Second, the debugger can display a stack trace, which shows the
903 sequence in which functions have been called and the arguments
904 passed to the called functions.
906 @node Debugging C++ code
907 @subsection Debugging C++ code
909 The GNU debugger, gdb, is the principal tool for debugging C++ code.
911 @subheading Compiling LilyPond for use with gdb
913 In order to use gdb with LilyPond, it is necessary to compile
914 LilyPond with debugging information. This is accomplished by running
915 the following commands in the main LilyPond source directory.
918 ./configure --disable-optimising
922 This will create a version of LilyPond containing debugging
923 information that will allow the debugger to tie the source code
924 to the compiled code.
926 You should not do @var{make install} if you want to use a debugger
927 with LilyPond. The @var{make install} command will strip debugging
928 information from the LilyPond binary.
930 @subheading Typical gdb usage
932 Once you have compiled the Lilypond image with the necessary
933 debugging information it will have been written to a location in a
934 subfolder of your current working directory:
940 This is important as you will need to let gdb know where to find the
941 image containing the symbol tables. You can invoke gdb from the
942 command line using the following:
948 This loads the LilyPond symbol tables into gdb. Then, to run
949 LilyPond on @file{test.ly} under the debugger, enter the following:
958 As an alternative to running gdb at the command line you may try
959 a graphical interface to gdb such as ddd:
965 You can also use sets of standard gdb commands stored in a .gdbinit
966 file (see next section).
968 @subheading Typical .gdbinit files
970 The behavior of gdb can be readily customized through the use of a
971 @var{.gdbinit} file. A @var{.gdbinit} file is a file named
972 @var{.gdbinit} (notice the @qq{.} at the beginning of the file name)
973 that is placed in a user's home directory.
975 The @var{.gdbinit} file below is from Han-Wen. It sets breakpoints
976 for all errors and defines functions for displaying scheme objects
977 (ps), grobs (pgrob), and parsed music expressions (pmusic).
980 file lily/out/lilypond
982 b Grob::programming_error
985 print ly_display_scm($arg0)
988 print ly_display_scm($arg0->self_scm_)
989 print ly_display_scm($arg0->mutable_property_alist_)
990 print ly_display_scm($arg0->immutable_property_alist_)
991 print ly_display_scm($arg0->object_alist_)
994 print ly_display_scm($arg0->self_scm_)
995 print ly_display_scm($arg0->mutable_property_alist_)
996 print ly_display_scm($arg0->immutable_property_alist_)
1000 @node Debugging Scheme code
1001 @subsection Debugging Scheme code
1003 Scheme code can be developed using the Guile command line
1004 interpreter @code{top-repl}. You can either investigate
1005 interactively using just Guile or you can use the debugging
1006 tools available within Guile.
1008 @subheading Using Guile interactively with LilyPond
1010 In order to experiment with Scheme programming in the LilyPond
1011 environment, it is necessary to have a Guile interpreter that
1012 has all the LilyPond modules loaded. This requires the following
1015 First, define a Scheme symbol for the active module in the @file{.ly} file:
1018 #(module-define! (resolve-module '(guile-user))
1019 'lilypond-module (current-module))
1022 Now place a Scheme function in the @file{.ly} file that gives an
1023 interactive Guile prompt:
1029 When the @file{.ly} file is compiled, this causes the compilation to be
1030 interrupted and an interactive guile prompt to appear. Once the
1031 guile prompt appears, the LilyPond active module must be set as the
1032 current guile module:
1035 guile> (set-current-module lilypond-module)
1038 You can demonstrate these commands are operating properly by typing the name
1039 of a LilyPond public scheme function to check it has been defined:
1042 guile> fret-diagram-verbose-markup
1043 #<procedure fret-diagram-verbose-markup (layout props marking-list)>
1046 If the LilyPond module has not been correctly loaded, an error
1047 message will be generated:
1050 guile> fret-diagram-verbose-markup
1051 ERROR: Unbound variable: fret-diagram-verbose-markup
1052 ABORT: (unbound-variable)
1055 Once the module is properly loaded, any valid LilyPond Scheme
1056 expression can be entered at the interactive prompt.
1058 After the investigation is complete, the interactive guile
1059 interpreter can be exited:
1065 The compilation of the @file{.ly} file will then continue.
1067 @subheading Using the Guile debugger
1069 To set breakpoints and/or enable tracing in Scheme functions, put
1072 \include "guile-debugger.ly"
1075 in your input file after any scheme procedures you have defined in
1076 that file. This will invoke the Guile command-line after having set
1077 up the environment for the debug command-line. When your input file
1078 is processed, a guile prompt will be displayed. You may now enter
1079 commands to set up breakpoints and enable tracing by the Guile debugger.
1081 @subheading Using breakpoints
1083 At the guile prompt, you can set breakpoints with
1084 the @code{set-break!} procedure:
1087 guile> (set-break! my-scheme-procedure)
1090 Once you have set the desired breakpoints, you exit the guile repl frame
1097 Then, when one of the scheme routines for which you have set
1098 breakpoints is entered, guile will interrupt execution in a debug
1099 frame. At this point you will have access to Guile debugging
1100 commands. For a listing of these commands, type:
1106 Alternatively you may code the breakpoints in your Lilypond source
1107 file using a command such as:
1110 #(set-break! my-scheme-procedure)
1113 immediately after the @code{\include} statement. In this case the
1114 breakpoint will be set straight after you enter the @code{(quit)}
1115 command at the guile prompt.
1117 Embedding breakpoint commands like this is particularly useful if
1118 you want to look at how the Scheme procedures in the @file{.scm}
1119 files supplied with LilyPond work. To do this, edit the file in
1120 the relevant directory to add this line near the top:
1123 (use-modules (scm guile-debugger))
1126 Now you can set a breakpoint after the procedure you are interested
1127 in has been declared. For example, if you are working on routines
1128 called by @var{print-book-with} in @file{lily-library.scm}:
1131 (define (print-book-with parser book process-procedure)
1132 (let* ((paper (ly:parser-lookup parser '$defaultpaper))
1133 (layout (ly:parser-lookup parser '$defaultlayout))
1134 (outfile-name (get-outfile-name parser)))
1135 (process-procedure book paper layout outfile-name)))
1137 (define-public (print-book-with-defaults parser book)
1138 (print-book-with parser book ly:book-process))
1140 (define-public (print-book-with-defaults-as-systems parser book)
1141 (print-book-with parser book ly:book-process-to-systems))
1145 At this point in the code you could add this to set a breakpoint at
1149 (set-break! print-book-with)
1152 @subheading Tracing procedure calls and evaluator steps
1154 Two forms of trace are available:
1157 (set-trace-call! my-scheme-procedure)
1163 (set-trace-subtree! my-scheme-procedure)
1166 @code{set-trace-call!} causes Scheme to log a line to the standard
1167 output to show when the procedure is called and when it exits.
1169 @code{set-trace-subtree!} traces every step the Scheme evaluator
1170 performs in evaluating the procedure.
1172 @node Tracing object relationships
1173 @section Tracing object relationships
1175 Understanding the LilyPond source often boils down to figuring out what
1176 is happening to the Grobs. Where (and why) are they being created,
1177 modified and destroyed? Tracing Lily through a debugger in order to
1178 identify these relationships can be time-consuming and tedious.
1180 In order to simplify this process, a facility has been added to
1181 display the grobs that are created and the properties that are set
1182 and modified. Although it can be complex to get set up, once set up
1183 it easily provides detailed information about the life of grobs
1184 in the form of a network graph.
1186 Each of the steps necessary to use the graphviz utility
1191 @item Installing graphviz
1193 In order to create the graph of the object relationships, it is
1194 first necessary to install Graphviz. graphviz is available for a
1195 number of different platforms:
1198 @uref{http://www.graphviz.org/Download..php}
1201 @item Modifying config.make
1203 In order for the Graphviz tool to work, config.make must be modified.
1204 It is probably a good idea to first save a copy of config.make under
1205 a different name. Then, edit config.make by removing every occurrence
1206 of @option{-DNDEBUG}.
1208 @item Rebuilding LilyPond
1210 The executable code of LilyPond must be rebuilt from scratch:
1213 make -C lily clean && make -C lily
1216 @item Create a graphviz-compatible @file{.ly} file
1218 In order to use the graphviz utility, the @file{.ly} file must include
1219 @file{ly/graphviz-init.ly}, and should then specify the
1220 grobs and symbols that should be tracked. An example of this
1221 is found in @file{input/regression/graphviz.ly}.
1223 @item Run lilypond with output sent to a log file
1225 The Graphviz data is sent to stderr by lilypond, so it is
1226 necessary to redirect stderr to a logfile:
1229 lilypond graphviz.ly 2> graphviz.log
1232 @item Edit the logfile
1234 The logfile has standard lilypond output, as well as the Graphviz
1235 output data. Delete everything from the beginning of the file
1236 up to but not including the first occurrence of @code{digraph}.
1238 Also, delete the final liypond message about successs from the end
1241 @item Process the logfile with @code{dot}
1243 The directed graph is created from the log file with the program
1247 dot -Tpdf graphviz.log > graphviz.pdf
1252 The pdf file can then be viewed with any pdf viewer.
1254 When compiled without @option{-DNDEBUG}, lilypond may run slower
1255 than normal. The original configuration can be restored by either
1256 renaming the saved copy of @code{config.make} or rerunning
1257 @code{configure}. Then rebuild lilypond with
1260 make -C lily clean && make -C lily
1264 @node Adding or modifying features
1265 @section Adding or modifying features
1267 When a new feature is to be added to LilyPond, it is necessary to
1268 ensure that the feature is properly integrated to maintain
1269 its long-term support. This section describes the steps necessary
1270 for feature addition and modification.
1275 * Write regression tests::
1276 * Write convert-ly rule::
1277 * Automatically update documentation::
1278 * Manually update documentation::
1279 * Edit changes.tely::
1280 * Verify successful build::
1281 * Verify regression tests::
1282 * Post patch for comments::
1284 * Closing the issues::
1287 @node Write the code
1288 @subsection Write the code
1290 You should probably create a new git branch for writing the code, as that
1291 will separate it from the master branch and allow you to continue
1292 to work on small projects related to master.
1294 Please be sure to follow the rules for programming style discussed
1295 earlier in this chapter.
1298 @node Write regression tests
1299 @subsection Write regression tests
1301 In order to demonstrate that the code works properly, you will
1302 need to write one or more regression tests. These tests are
1303 typically @file{.ly} files that are found in @file{input/regression}.
1305 Regression tests should be as brief as possible to demonstrate the
1306 functionality of the code.
1308 Regression tests should generally cover one issue per test. Several
1309 short, single-issue regression tests are preferred to a single, long,
1310 multiple-issue regression test.
1312 Use existing regression tests as templates to demonstrate the type of
1313 header information that should be included in a regression test.
1316 @node Write convert-ly rule
1317 @subsection Write convert-ly rule
1319 If the modification changes the input syntax, a convert-ly rule
1320 should be written to automatically update input files from older
1323 convert-ly rules are found in python/convertrules.py
1325 If possible, the convert-ly rule should allow automatic updating
1326 of the file. In some cases, this will not be possible, so the
1327 rule will simply point out to the user that the feature needs
1330 @subsubheading Updating version numbers
1332 If a development release occurs between you writing your patch and
1333 having it approved+pushed, you will need to update the version
1334 numbers in your tree. This can be done with:
1337 scripts/auxiliar/update-patch-version old.version.number new.version.number
1340 It will change all files in git, so use with caution and examine
1344 @node Automatically update documentation
1345 @subsection Automatically update documentation
1347 @command{convert-ly} should be used to update the documentation,
1348 the snippets, and the regression tests. This not only makes the
1349 necessary syntax changes, it also tests the @command{convert-ly}
1352 The automatic updating is performed by moving to the top-level
1353 source directory, then running:
1356 scripts/auxiliar/update-with-convert-ly.sh
1359 If you did an out-of-tree build, pass in the relative path:
1362 BUILD_DIR=../build-lilypond/ scripts/auxiliar/update-with-convert-ly.sh
1366 @node Manually update documentation
1367 @subsection Manually update documentation
1369 Where the convert-ly rule is not able to automatically update the inline
1370 lilypond code in the documentation (i.e. if a NOT_SMART rule is used), the
1371 documentation must be manually updated. The inline snippets that require
1372 changing must be changed in the English version of the docs and all
1373 translated versions. If the inline code is not changed in the
1374 translated documentation, the old snippets will show up in the
1375 English version of the documentation.
1377 Where the convert-ly rule is not able to automatically update snippets
1378 in Documentation/snippets/, those snippets must be manually updated.
1379 Those snippets should be copied to Documentation/snippets/new. The
1380 comments at the top of the snippet describing its automatic generation
1381 should be removed. All translated texidoc strings should be removed.
1382 The comment @qq{% begin verbatim} should be removed. The syntax of
1383 the snippet should then be manually edited.
1385 Where snippets in Documentation/snippets are made obsolete, the snippet
1386 should be copied to Documentation/snippets/new. The comments and
1387 texidoc strings should be removed as described above. Then the body
1388 of the snippet should be changed to:
1392 This snippet is deprecated as of version X.Y.Z and
1393 will be removed from the documentation.
1398 where X.Y.Z is the version number for which the convert-ly rule was
1401 Update the snippet files by running:
1404 scripts/auxiliar/makelsr.py
1407 Where the convert-ly rule is not able to automatically update regression
1408 tests, the regression tests in input/regression should be manually
1411 Although it is not required, it is helpful if the developer
1412 can write relevant material for inclusion in the Notation
1413 Reference. If the developer does not feel qualified to write
1414 the documentation, a documentation editor will be able to
1415 write it from the regression tests. The text that is added to
1416 or removed from the documentation should be changed only in
1417 the English version.
1420 @node Edit changes.tely
1421 @subsection Edit changes.tely
1423 An entry should be added to Documentation/changes.tely to describe
1424 the feature changes to be implemented. This is especially important
1425 for changes that change input file syntax.
1427 Hints for changes.tely entries are given at the top of the file.
1429 New entries in changes.tely go at the top of the file.
1431 The changes.tely entry should be written to show how the new change
1432 improves LilyPond, if possible.
1435 @node Verify successful build
1436 @subsection Verify successful build
1438 When the changes have been made, successful completion must be
1446 When these commands complete without error, the patch is
1447 considered to function successfully.
1449 Developers on Windows who are unable to build LilyPond should
1450 get help from a Linux or OSX developer to do the make tests.
1453 @node Verify regression tests
1454 @subsection Verify regression tests
1456 In order to avoid breaking LilyPond, it is important to verify that
1457 the regression tests succeed, and that no unwanted changes are
1458 introduced into the output. This process is described in
1459 @ref{Regtest comparison}.
1461 @subheading Typical developer's edit/compile/test cycle
1463 TODO: is @code{[-j@var{X} CPU_COUNT=@var{X}]} useful for
1464 @code{test-baseline}, @code{check}, @code{clean},
1465 @code{test-redo}? Neil Puttock says it is useful for
1466 everything but @code{clean}, which is disk-limited.
1467 Need to check formally.
1476 make [-j@var{X} CPU_COUNT=@var{X}] check
1480 Edit/compile/test cycle:
1483 @emph{## edit source files, then...}
1485 make clean @emph{## only if needed (see below)}
1486 make [-j@var{X}] @emph{## only if needed (see below)}
1487 make test-redo @emph{## redo files differing from baseline}
1488 make [-j@var{X} CPU_COUNT=@var{X}] check @emph{## CPU_COUNT here?}
1499 If you modify any source files that have to be compiled (such as
1500 @file{.cc} or @file{.hh} files in @file{flower/} or @file{lily/}),
1501 then you must run @command{make} before @command{make test-redo},
1502 so @command{make} can compile the modified files and relink all
1503 the object files. If you only modify files which are interpreted,
1504 like those in the @file{scm/} and @file{ly/} directories, then
1505 @command{make} is not needed before @command{make test-redo}.
1507 TODO: Fix the following paragraph. You can do @command{rm mf/out/*}
1508 instead of make clean, and you can probably do
1509 @command{make -C mf/ clean} as well, but I haven't checked it -- cds
1511 Also, if you modify any font definitions in the @file{mf/}
1512 directory then you must run @command{make clean} and
1513 @command{make} before running @command{make test-redo}. This will
1514 recompile everything, whether modified or not, and takes a lot
1517 Running @command{make@tie{}check} will leave an HTML page
1518 @file{out/test-results/index.html}. This page shows all the
1519 important differences that your change introduced, whether in the
1520 layout, MIDI, performance or error reporting.
1525 @node Post patch for comments
1526 @subsection Post patch for comments
1528 See @ref{Uploading a patch for review}.
1532 @subsection Push patch
1534 Once all the comments have been addressed, the patch can be pushed.
1536 If the author has push privileges, the author will push the patch.
1537 Otherwise, a developer with push privileges will push the patch.
1540 @node Closing the issues
1541 @subsection Closing the issues
1543 Once the patch has been pushed, all the relevant issues should be
1546 On Rietveld, the author should log in an close the issue either by
1547 using the @q{Edit Issue} link, or by clicking the circled x icon
1548 to the left of the issue name.
1550 If the changes were in response to a feature request on the Google
1551 issue tracker for LilyPond, the author should change the status to
1552 Fixed and a tag @q{fixed_x_y_z} should be added, where the patch was
1553 fixed in version x.y.z. If
1554 the author does not have privileges to change the status, an email
1555 should be sent to bug-lilypond requesting the BugMeister to change
1559 @node Iterator tutorial
1560 @section Iterator tutorial
1562 TODO -- this is a placeholder for a tutorial on iterators
1564 Iterators are routines written in C++ that process music expressions
1565 and sent the music events to the appropriate engravers and/or
1569 @node Engraver tutorial
1570 @section Engraver tutorial
1572 Engravers are C++ classes that catch music events and
1573 create the appropriate grobs for display on the page. Though the
1574 majority of engravers are responsible for the creation of a single grob,
1575 in some cases (e.g. @code{New_fingering_engraver}), several different grobs
1578 Engravers listen for events and acknowledge grobs. Events are passed to
1579 the engraver in time-step order during the iteration phase. Grobs are
1580 made available to the engraver when they are created by other engravers
1581 during the iteration phase.
1585 * Useful methods for information processing::
1586 * Translation process::
1587 * Preventing garbage collection for SCM member variables::
1588 * Listening to music events::
1589 * Acknowledging grobs::
1590 * Engraver declaration/documentation::
1593 @node Useful methods for information processing
1594 @subsection Useful methods for information processing
1596 An engraver inherits the following public methods from the Translator
1597 base class, which can be used to process listened events and acknowledged
1601 @item @code{virtual void initialize ()}
1602 @item @code{void start_translation_timestep ()}
1603 @item @code{void process_music ()}
1604 @item @code{void process_acknowledged ()}
1605 @item @code{void stop_translation_timestep ()}
1606 @item @code{virtual void finalize ()}
1609 These methods are listed in order of translation time, with
1610 @code{initialize ()} and @code{finalize ()} bookending the whole
1611 process. @code{initialize ()} can be used for one-time initialization
1612 of context properties before translation starts, whereas
1613 @code{finalize ()} is often used to tie up loose ends at the end of
1614 translation: for example, an unterminated spanner might be completed
1615 automatically or reported with a warning message.
1618 @node Translation process
1619 @subsection Translation process
1621 At each timestep in the music, translation proceeds by calling the
1622 following methods in turn:
1624 @code{start_translation_timestep ()} is called before any user
1625 information enters the translators, i.e., no property operations
1626 (\set, \override, etc.) or events have been processed yet.
1628 @code{process_music ()} and @code{process_acknowledged ()} are called
1629 after all events in the current time step have been heard, or all
1630 grobs in the current time step have been acknowledged. The latter
1631 tends to be used exclusively with engravers which only acknowledge
1632 grobs, whereas the former is the default method for main processing
1635 @code{stop_translation_timestep ()} is called after all user
1636 information has been processed prior to beginning the translation for
1640 @node Preventing garbage collection for SCM member variables
1641 @subsection Preventing garbage collection for SCM member variables
1643 In certain cases, an engraver might need to ensure private Scheme
1644 variables (with type SCM) do not get swept away by Guile's garbage
1645 collector: for example, a cache of the previous key signature which
1646 must persist between timesteps. The method
1647 @code{virtual derived_mark () const} can be used in such cases:
1650 Engraver_name::derived_mark ()
1652 scm_gc_mark (private_scm_member_)
1657 @node Listening to music events
1658 @subsection Listening to music events
1660 External interfaces to the engraver are implemented by protected
1661 macros including one or more of the following:
1664 @item @code{DECLARE_TRANSLATOR_LISTENER (event_name)}
1665 @item @code{IMPLEMENT_TRANSLATOR_LISTENER (Engraver_name, event_name)}
1669 where @var{event_name} is the type of event required to provide the
1670 input the engraver needs and @var{Engraver_name} is the name of the
1673 Following declaration of a listener, the method is implemented as follows:
1676 IMPLEMENT_TRANSLATOR_LISTENER (Engraver_name, event_name)
1678 Engraver_name::listen_event_name (Stream event *event)
1680 ...body of listener method...
1685 @node Acknowledging grobs
1686 @subsection Acknowledging grobs
1688 Some engravers also need information from grobs as they are created
1689 and as they terminate. The mechanism and methods to obtain this
1690 information are set up by the macros:
1693 @item @code{DECLARE_ACKNOWLEDGER (grob_interface)}
1694 @item @code{DECLARE_END_ACKNOWLEDGER (grob_interface)}
1697 where @var{grob_interface} is an interface supported by the
1698 grob(s) which should be acknowledged. For example, the following
1699 code would declare acknowledgers for a @code{NoteHead} grob (via the
1700 @code{note-head-interface}) and any grobs which support the
1701 @code{side-position-interface}:
1704 @code{DECLARE_ACKNOWLEDGER (note_head)}
1705 @code{DECLARE_ACKNOWLEDGER (side_position)}
1708 The @code{DECLARE_END_ACKNOWLEDGER ()} macro sets up a spanner-specific
1709 acknowledger which will be called whenever a spanner ends.
1711 Following declaration of an acknowledger, the method is coded as follows:
1715 Engraver_name::acknowledge_interface_name (Grob_info info)
1717 ...body of acknowledger method...
1722 @node Engraver declaration/documentation
1723 @subsection Engraver declaration/documentation
1725 An engraver must have a public macro
1728 @item @code{TRANSLATOR_DECLARATIONS (Engraver_name)}
1732 where @code{Engraver_name} is the name of the engraver. This
1733 defines the common variables and methods used by every engraver.
1735 At the end of the engraver file, one or both of the following
1736 macros are generally called to document the engraver in the
1737 Internals Reference:
1740 @item @code{ADD_ACKNOWLEDGER (Engraver_name, grob_interface)}
1741 @item @code{ADD_TRANSLATOR (Engraver_name, Engraver_doc,
1742 Engraver_creates, Engraver_reads, Engraver_writes)}
1746 where @code{Engraver_name} is the name of the engraver, @code{grob_interface}
1747 is the name of the interface that will be acknowledged,
1748 @code{Engraver_doc} is a docstring for the engraver,
1749 @code{Engraver_creates} is the set of grobs created by the engraver,
1750 @code{Engraver_reads} is the set of properties read by the engraver,
1751 and @code{Engraver_writes} is the set of properties written by
1754 The @code{ADD_ACKNOWLEDGER} and @code{ADD_TRANSLATOR} macros use a
1755 non-standard indentation system. Each interface, grob, read property,
1756 and write property is on its own line, and the closing parenthesis
1757 and semicolon for the macro all occupy a separate line beneath the final
1758 interface or write property. See existing engraver files for more
1762 @node Callback tutorial
1763 @section Callback tutorial
1765 TODO -- This is a placeholder for a tutorial on callback functions.
1767 @node LilyPond scoping
1768 @section LilyPond scoping
1770 The Lilypond language has a concept of scoping, i.e. you can do
1776 (display (+ foo 2)))
1779 @noindent with @code{\paper}, @code{\midi} and @code{\header} being
1780 nested scope inside the @file{.ly} file-level scope. @w{@code{foo = 1}}
1781 is translated in to a scheme variable definition.
1783 This implemented using modules, with each scope being an anonymous
1784 module that imports its enclosing scope's module.
1786 Lilypond's core, loaded from @file{.scm} files, is usually placed in the
1787 @code{lily} module, outside the @file{.ly} level. In the case of
1794 we want to reuse the built-in definitions, without changes effected in
1795 user-level @file{a.ly} leaking into the processing of @file{b.ly}.
1797 The user-accessible definition commands have to take care to avoid
1798 memory leaks that could occur when running multiple files. All
1799 information belonging to user-defined commands and markups is stored in
1800 a manner that allows it to be garbage-collected when the module is
1801 dispersed, either by being stored module-locally, or in weak hash
1804 @node LilyPond miscellany
1805 @section LilyPond miscellany
1807 This is a place to dump information that may be of use to developers
1808 but doesn't yet have a proper home. Ideally, the length of this section
1809 would become zero as items are moved to other homes.
1813 * Spacing algorithms::
1814 * Info from Han-Wen email::
1815 * Music functions and GUILE debugging::
1818 @node Spacing algorithms
1819 @subsection Spacing algorithms
1821 Here is information from an email exchange about spacing algorithms.
1823 On Thu, 2010-02-04 at 15:33 -0500, Boris Shingarov wrote:
1824 I am experimenting with some modifications to the line breaking code,
1825 and I am stuck trying to understand how some of it works. So far my
1826 understanding is that Simple_spacer operates on a vector of Grobs, and
1827 it is a well-known Constrained-QP problem (rods = constraints, springs
1828 = quadratic function to minimize). What I don't understand is, if the
1829 spacer operates at the level of Grobs, which are built at an earlier
1830 stage in the pipeline, how are the changes necessitated by differences
1831 in line breaking, taken into account? in other words, if I take the
1832 last measure of a line and place it on the next line, it is not just a
1833 matter of literally moving that graphic to where the start of the next
1834 line is, but I also need to draw a clef, key signature, and possibly
1835 other fundamental things -- but at that stage in the rendering
1836 pipeline, is it not too late??
1838 Joe Neeman answered:
1840 We create lots of extra grobs (eg. a BarNumber at every bar line) but
1841 most of them are not drawn. See the break-visibility property in
1844 Here is another e-mail exchange. Janek Warchoł asked for a starting point
1845 to fixing 1301 (change clef colliding with notes). Neil Puttock replied:
1847 The clef is on a loose column (it floats before the head), so the
1848 first place I'd look would be lily/spacing-loose-columns.cc (and
1849 possibly lily/spacing-determine-loose-columns.cc).
1850 I'd guess the problem is the way loose columns are spaced between
1851 other columns: in this snippet, the columns for the quaver and tuplet
1852 minim are so close together that the clef's column gets dumped on top
1853 of the quaver (since it's loose, it doesn't influence the spacing).
1855 @node Info from Han-Wen email
1856 @subsection Info from Han-Wen email
1858 In 2004, Douglas Linhardt decided to try starting a document that would
1859 explain LilyPond architecture and design principles. The material below
1860 is extracted from that email, which can be found at
1861 @uref{http://thread.gmane.org/gmane.comp.gnu.lilypond.devel/2992}.
1862 The headings reflect questions from Doug or comments from Han-Wen;
1863 the body text are Han-Wen's answers.
1865 @subheading Figuring out how things work.
1867 I must admit that when I want to know how a program works, I use grep
1868 and emacs and dive into the source code. The comments and the code
1869 itself are usually more revealing than technical documents.
1871 @subheading What's a grob, and how is one used?
1873 Graphical object - they are created from within engravers, either as
1874 Spanners (derived class) -slurs, beams- or Items (also a derived
1875 class) -notes, clefs, etc.
1877 There are two other derived classes System (derived from Spanner,
1878 containing a "line of music") and Paper_column (derived from Item, it
1879 contains all items that happen at the same moment). They are separate
1880 classes because they play a special role in the linebreaking process.
1882 @subheading What's a smob, and how is one used?
1884 A C(++) object that is encapsulated so it can be used as a Scheme
1885 object. See GUILE info, "19.3 Defining New Types (Smobs)"
1887 @@subheading When is each C++ class constructed and used
1894 In the parser.yy see the macro calls MAKE_MUSIC_BY_NAME().
1899 Constructed during "interpreting" phase.
1904 Executive branch of Contexts, plugins that create grobs, usually one
1905 engraver per grob type. Created together with context.
1915 These are not C++ classes per se. The idea of a Grob interface hasn't
1916 crystallized well. ATM, an interface is a symbol, with a bunch of grob
1917 properties. They are not objects that are created or destroyed.
1922 Objects that walk through different music classes, and deliver events
1923 in a synchronized way, so that notes that play together are processed
1924 at the same moment and (as a result) end up on the same horizontal position.
1926 Created during interpreting phase.
1928 BTW, the entry point for interpreting is ly:run-translator
1929 (ly_run_translator on the C++ side)
1933 @subheading Can you get to Context properties from a Music object?
1935 You can create music object with a Scheme function that reads context
1936 properties (the \applycontext syntax). However, that function is
1937 executed during Interpreting, so you can not really get Context
1938 properties from Music objects, since music objects are not directly
1939 connected to Contexts. That connection is made by the Music_iterators
1941 @subheading Can you get to Music properties from a Context object?
1943 Yes, if you are given the music object within a Context
1944 object. Normally, the music objects enter Contexts in synchronized
1945 fashion, and the synchronization is done by Music_iterators.
1947 @subheading What is the relationship between C++ classes and Scheme objects?
1949 Smobs are C++ objects in Scheme. Scheme objects (lists, functions) are
1950 manipulated from C++ as well using the GUILE C function interface
1953 @subheading How do Scheme procedures get called from C++ functions?
1955 scm_call_*, where * is an integer from 0 to 4.
1956 Also scm_c_eval_string (), scm_eval ()
1958 @subheading How do C++ functions get called from Scheme procedures?
1960 Export a C++ function to Scheme with LY_DEFINE.
1962 @subheading What is the flow of control in the program?
1964 Good question. Things used to be clear-cut, but we have Scheme
1965 and SMOBs now, which means that interactions do not follow a very
1966 rigid format anymore. See below for an overview, though.
1968 @subheading Does the parser make Scheme procedure calls or C++ function calls?
1970 Both. And the Scheme calls can call C++ and vice versa. It's nested,
1971 with the SCM datatype as lubrication between the interactions
1973 (I think the word "lubrication" describes the process better than the
1974 traditional word "glue")
1976 @subheading How do the front-end and back-end get started?
1978 Front-end: a file is parsed, the rest follows from that. Specifically,
1980 Parsing leads to a Music + Music_output_def object (see parser.yy,
1981 definition of toplevel_expression )
1983 A Music + Music_output_def object leads to a Global_context object (see
1984 ly_run_translator ())
1986 During interpreting, Global_context + Music leads to a bunch of
1987 Contexts (see Global_translator::run_iterator_on_me ()).
1989 After interpreting, Global_context contains a Score_context (which
1990 contains staves, lyrics etc.) as a child. Score_context::get_output ()
1991 spews a Music_output object (either a Paper_score object for notation
1992 or Performance object for MIDI).
1994 The Music_output object is the entry point for the backend (see
1995 ly_render_output ()).
1997 The main steps of the backend itself are in
2002 @file{paper-score.cc} , Paper_score::process_
2005 @file{system.cc} , System::get_lines()
2008 The step, where things go from grobs to output, is in
2009 System::get_line(): each grob delivers a Stencil (a Device
2010 independent output description), which is interpreted by our
2011 outputting backends (@file{scm/output-tex.scm} and
2012 @file{scm/output-ps.scm}) to produce TeX and PS.
2016 Interactions between grobs and putting things into .tex and .ps files
2017 have gotten a little more complex lately. Jan has implemented
2018 page-breaking, so now the backend also involves Paper_book,
2019 Paper_lines and other things. This area is still heavily in flux, and
2020 perhaps not something you should want to look at.
2022 @subheading How do the front-end and back-end communicate?
2024 There is no communication from backend to front-end. From front-end to
2025 backend is simply the program flow: music + definitions gives
2026 contexts, contexts yield output, after processing, output is written
2029 @subheading Where is the functionality associated with KEYWORDs?
2031 See @file{my-lily-lexer.cc} (keywords, there aren't that many)
2032 and @file{ly/*.ly} (most of the other backslashed @code{/\words} are identifiers)
2034 @subheading What Contexts/Properties/Music/etc. are available when they are processed?
2036 What do you mean exactly with this question?
2038 See @file{ly/engraver-init.ly} for contexts,
2039 see @file{scm/define-*.scm} for other objects.
2041 @subheading How do you decide if something is a Music, Context, or Grob property?
2042 Why is part-combine-status a Music property when it seems (IMO)
2043 to be related to the Staff context?
2045 The Music_iterators and Context communicate through two channels
2047 Music_iterators can set and read context properties, idem for
2048 Engravers and Contexts
2050 Music_iterators can send "synthetic" music events (which aren't in
2051 the input) to a context. These are caught by Engravers. This is
2052 mostly a one way communication channel.
2054 part-combine-status is part of such a synthetic event, used by
2055 Part_combine_iterator to communicate with Part_combine_engraver.
2058 @subheading Deciding between context and music properties
2060 I'm adding a property to affect how \autochange works. It seems to
2061 me that it should be a context property, but the Scheme autochange
2062 procedure has a Music argument. Does this mean I should use
2065 \autochange is one of these extra strange beasts: it requires
2066 look-ahead to decide when to change staves. This is achieved by
2067 running the interpreting step twice (see
2068 @file{scm/part-combiner.scm} , at the bottom), and
2069 storing the result of the first step (where to switch
2070 staves) in a Music property. Since you want to influence that
2071 where-to-switch list, your must affect the code in
2072 make-autochange-music (@file{scm/part-combiner.scm}).
2073 That code is called directly from the parser and there are no
2074 official "parsing properties" yet, so there is no generic way
2075 to tune \autochange. We would have to invent something new
2076 for this, or add a separate argument,
2079 \autochange #around-central-C ..music..
2083 where around-central-C is some function that is called from
2084 make-autochange-music.
2086 @subheading More on context and music properties
2088 From Neil Puttock, in response to a question about transposition:
2090 Context properties (using \set & \unset) are tied to engravers: they
2091 provide information relevant to the generation of graphical objects.
2093 Since transposition occurs at the music interpretation stage, it has
2094 no direct connection with engravers: the pitch of a note is fixed
2095 before a notehead is created. Consider the following minimal snippet:
2101 This generates (simplified) a NoteEvent, with its pitch and duration
2102 as event properties,
2108 (ly:make-duration 2 0 1 1)
2110 (ly:make-pitch 0 0 0)
2113 which the Note_heads_engraver hears. It passes this information on to
2114 the NoteHead grob it creates from the event, so the head's correct
2115 position and duration-log can be determined once it's ready for
2118 If we transpose the snippet,
2121 \transpose c d @{ c' @}
2124 the pitch is changed before it reaches the engraver (in fact, it
2125 happens just after the parsing stage with the creation of a
2126 TransposedMusic music object):
2132 (ly:make-duration 2 0 1 1)
2134 (ly:make-pitch 0 1 0)
2137 You can see an example of a music property relevant to transposition:
2141 \transpose c d @{ c'2 \withMusicProperty #'untransposable ##t c' @}
2144 -> the second c' remains untransposed.
2146 Take a look at @file{lily/music.cc} to see where the transposition takes place.
2149 @subheading How do I tell about the execution environment?
2151 I get lost figuring out what environment the code I'm looking at is in when it
2152 executes. I found both the C++ and Scheme autochange code. Then I was trying
2153 to figure out where the code got called from. I finally figured out that the
2154 Scheme procedure was called before the C++ iterator code, but it took me a
2155 while to figure that out, and I still didn't know who did the calling in the
2156 first place. I only know a little bit about Flex and Bison, so reading those
2157 files helped only a little bit.
2159 @emph{Han-Wen:} GDB can be of help here. Set a breakpoint in C++, and run. When you
2160 hit the breakpoint, do a backtrace. You can inspect Scheme objects
2161 along the way by doing
2164 p ly_display_scm(obj)
2167 this will display OBJ through GUILE.
2169 @node Music functions and GUILE debugging
2170 @subsection Music functions and GUILE debugging
2172 Ian Hulin was trying to do some debugging in music functions, and
2173 came up with the following question
2176 I'm working on the Guile Debugger Stuff, and would like to try
2177 debugging a music function definition such as:
2180 conditionalMark = #(define-music-function (parser location) ()
2181 #@{ \tag #'instrumental-part @{\mark \default@} #@} )
2184 It appears conditionalMark does not get set up as an
2185 equivalent of a Scheme
2188 (define conditionalMark = define-music-function(parser location () ...
2192 although something gets defined because Scheme apparently recognizes
2195 #(set-break! conditionalMark)
2199 later on in the file without signalling any Guile errors.
2201 However the breakpoint trap is never encountered as
2202 define-music-function passed things on to ly:make-music-function,
2203 which is really C++ code ly_make_music_function, so Guile never
2204 finds out about the breakpoint.
2206 Han-Wen answered as follows:
2208 You can see the definition by doing
2211 #(display conditionalMark)
2215 inside the @file{.ly} file.
2217 The breakpoint failing may have to do with the call sequence. See
2218 @file{parser.yy}, run_music_function(). The function is called directly from
2219 C++, without going through the GUILE evaluator, so I think that is why
2220 there is no debugger trap.