1 @c -*- coding: utf-8; mode: texinfo; -*-
3 @chapter Programming work
6 * Overview of LilyPond architecture::
7 * LilyPond programming languages::
8 * Programming without compiling::
11 * Debugging LilyPond::
12 * Tracing object relationships::
13 * Adding or modifying features::
18 * LilyPond miscellany::
21 @node Overview of LilyPond architecture
22 @section Overview of LilyPond architecture
24 LilyPond processes the input file into graphical and musical output in a
25 number of stages. This process, along with the types of routines that
26 accomplish the various stages of the process, is described in this section. A
27 more complete description of the LilyPond architecture and internal program
28 execution is found in Erik Sandberg's
29 @uref{http://lilypond.org/web/images/thesis-erik-sandberg.pdf, master's
32 The first stage of LilyPond processing is @emph{parsing}. In the parsing
33 process, music expressions in LilyPond input format are converted to music
34 expressions in Scheme format. In Scheme format, a music expression is a list
35 in tree form, with nodes that indicate the relationships between various music
36 events. The LilyPond parser is written in Bison.
38 The second stage of LilyPond processing is @emph{iterating}. Iterating
39 assigns each music event to a context, which is the environment in which the
40 music will be finally engraved. The context is responsible for all further
41 processing of the music. It is during the iteration stage that contexts are
42 created as necessary to ensure that every note has a Voice type context (e.g.
43 Voice, TabVoice, DrumVoice, CueVoice, MensuralVoice, VaticanaVoice,
44 GregorianTranscriptionVoice), that the Voice type contexts exist in
45 appropriate Staff type contexts, and that parallel Staff type contexts exist
46 in StaffGroup type contexts. In addition, during the iteration stage each
47 music event is assigned a moment, or a time in the music when the event
50 Each type of music event has an associated iterator. Iterators are defined in
51 @file{*-iterator.cc}. During iteration, an
52 event's iterator is called to deliver that music event to the appropriate
55 The final stage of LilyPond processing is @emph{translation}. During
56 translation, music events are prepared for graphical or midi output. The
57 translation step is accomplished by the polymorphic base class Translator
58 through its two derived classes: Engraver (for graphical output) and
59 Performer (for midi output).
61 Translators are defined in C++ files named @file{*-engraver.cc}
62 and @file{*-performer.cc}.
63 Much of the work of translating is handled by Scheme functions,
64 which is one of the keys to LilyPond's exceptional flexibility.
66 @sourceimage{architecture-diagram,,,png}
69 @node LilyPond programming languages
70 @section LilyPond programming languages
72 Programming in LilyPond is done in a variety of programming languages. Each
73 language is used for a specific purpose or purposes. This section describes
74 the languages used and provides links to reference manuals and tutorials for
75 the relevant language.
79 The core functionality of LilyPond is implemented in C++.
81 C++ is so ubiquitous that it is difficult to identify either a reference
82 manual or a tutorial. Programmers unfamiliar with C++ will need to spend some
83 time to learn the language before attempting to modify the C++ code.
85 The C++ code calls Scheme/GUILE through the GUILE interface, which is
87 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html, GUILE
92 The LilyPond lexer is implemented in Flex, an implementation of the Unix lex
93 lexical analyser generator. Resources for Flex can be found
94 @uref{http://flex.sourceforge.net/, here}.
98 The LilyPond parser is implemented in Bison, a GNU parser generator. The
99 Bison homepage is found at @uref{http://www.gnu.org/software/bison/,
100 gnu.org}. The manual (which includes both a reference and tutorial) is
101 @uref{http://www.gnu.org/software/bison/manual/index.html, available} in a
106 GNU Make is used to control the compiling process and to build the
107 documentation and the website. GNU Make documentation is available at
108 @uref{http://www.gnu.org/software/make/manual/, the GNU website}.
110 @subsection GUILE or Scheme
112 GUILE is the dialect of Scheme that is used as LilyPond's extension language.
113 Many extensions to LilyPond are written entirely in GUILE. The
114 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html,
115 GUILE Reference Manual} is available online.
117 @uref{http://mitpress.mit.edu/sicp/full-text/book/book.html, Structure and
118 Interpretation of Computer Programs}, a popular textbook used to teach
119 programming in Scheme is available in its entirety online.
121 An introduction to Guile/Scheme as used in LilyPond can be found in the
122 @rextend{Scheme tutorial}.
126 MetaFont is used to create the music fonts used by LilyPond. A MetaFont
127 tutorial is available at @uref{http://metafont.tutorial.free.fr/, the
128 METAFONT tutorial page}.
130 @subsection PostScript
132 PostScript is used to generate graphical output. A brief PostScript tutorial
133 is @uref{http://local.wasp.uwa.edu.au/~pbourke/dataformats/postscript/,
134 available online}. The
135 @uref{http://www.adobe.com/devnet/postscript/pdfs/PLRM.pdf, PostScript Language
136 Reference} is available online in PDF format.
140 Python is used for XML2ly and is used for building the documentation and the
143 Python documentation is available at @uref{http://www.python.org/doc/,
146 @node Programming without compiling
147 @section Programming without compiling
149 Much of the development work in LilyPond takes place by changing @file{*.ly} or
150 @file{*.scm} files. These changes can be made without compiling LilyPond. Such
151 changes are described in this section.
154 @subsection Modifying distribution files
156 Much of LilyPond is written in Scheme or LilyPond input files. These
157 files are interpreted when the program is run, rather than being compiled
158 when the program is built, and are present in all LilyPond distributions.
159 You will find @file{.ly} files in the @file{ly/} directory and the Scheme files in the
160 @file{scm/} directory. Both Scheme files and @file{.ly} files can be modified and
161 saved with any text editor. It's probably wise to make a backup copy of
162 your files before you modify them, although you can reinstall if the
163 files become corrupted.
165 Once you've modified the files, you can test the changes just by running
166 LilyPond on some input file. It's a good idea to create a file that
167 demonstrates the feature you're trying to add. This file will eventually
168 become a regression test and will be part of the LilyPond distribution.
170 @subsection Desired file formatting
172 Files that are part of the LilyPond distribution have Unix-style line
173 endings (LF), rather than DOS (CR+LF) or MacOS 9 and earlier (CR). Make
174 sure you use the necessary tools to ensure that Unix-style line endings are
175 preserved in the patches you create.
177 Tab characters should not be included in files for distribution. All
178 indentation should be done with spaces. Most editors have settings to
179 allow the setting of tab stops and ensuring that no tab characters are
180 included in the file.
182 Scheme files and LilyPond files should be written according to standard
183 style guidelines. Scheme file guidelines can be found at
184 @uref{http://community.schemewiki.org/?scheme-style}. Following these
185 guidelines will make your code easier to read. Both you and others that
186 work on your code will be glad you followed these guidelines.
188 For LilyPond files, you should follow the guidelines for LilyPond snippets
189 in the documentation. You can find these guidelines at
190 @ref{Texinfo introduction and usage policy}.
192 @node Finding functions
193 @section Finding functions
195 When making changes or fixing bugs in LilyPond, one of the initial
196 challenges is finding out where in the code tree the functions to
197 be modified live. With nearly 3000 files in the source tree,
198 trial-and-error searching is generally ineffective. This section
199 describes a process for finding interesting code.
201 @subsection Using the ROADMAP
203 The file ROADMAP is located in the main directory of the lilypond source.
204 ROADMAP lists all of the directories in the LilyPond source tree, along
205 with a brief description of the kind of files found in each directory.
206 This can be a very helpful tool for deciding which directories to search
207 when looking for a function.
210 @subsection Using grep to search
212 Having identified a likely subdirectory to search, the grep utility can
213 be used to search for a function name. The format of the grep command is
216 grep -i functionName subdirectory/*
219 This command will search all the contents of the directory subdirectory/
220 and display every line in any of the files that contains
221 functionName. The @code{-i} option makes @command{grep} ignore
222 case -- this can be very useful if you are not yet familiar with
223 our capitalization conventions.
225 The most likely directories to grep for function names are @file{scm/} for
226 scheme files, ly/ for lilypond input (@file{*.ly}) files, and @file{lily/} for C++
230 @subsection Using git grep to search
232 If you have used git to obtain the source, you have access to a
233 powerful tool to search for functions. The command:
236 git grep functionName
239 will search through all of the files that are present in the git
240 repository looking for functionName. It also presents the results
241 of the search using @code{less}, so the results are displayed one page
244 @subsection Searching on the git repository at Savannah
246 You can also use the equivalent of git grep on the Savannah server.
251 Go to http://git.sv.gnu.org/gitweb/?p=lilypond.git
254 In the pulldown box that says commit, select grep.
257 Type functionName in the search box, and hit enter/return
261 This will initiate a search of the remote git repository.
267 This section describes style guidelines for LilyPond
274 * Naming conventions::
283 @subsection Languages
285 C++ and Python are preferred. Python code should use PEP 8.
289 @subsection Filenames
291 Definitions of classes that are only accessed via pointers (*) or
292 references (&) shall not be included as include files.
298 ".cc" Implementation files
299 ".icc" Inline definition files
300 ".tcc" non inline Template defs
304 (setq auto-mode-alist
305 (append '(("\\.make$" . makefile-mode)
306 ("\\.cc$" . c++-mode)
307 ("\\.icc$" . c++-mode)
308 ("\\.tcc$" . c++-mode)
309 ("\\.hh$" . c++-mode)
310 ("\\.pod$" . text-mode)
315 The class Class_name is coded in @q{class-name.*}
319 @subsection Indentation
321 Standard GNU coding style is used.
323 @subsubheading Indenting files with @code{fixcc.py} (recommended)
325 LilyPond provides a python script that will adjust the indentation
326 and spacing on a @code{.cc} or @code{.hh} file to very near the
330 scripts/auxiliar/fixcc.py FILENAME
333 This can be run on all files at once, but this is not recommended
334 for normal contributors or developers.
337 scripts/auxiliar/fixcc.py \
338 $(find flower lily -name '*cc' -o -name '*hh' | grep -v /out)
342 @subsubheading Indenting with emacs
344 The following hooks will produce indentation which is similar to
345 our official indentation as produced with @code{fixcc.py}.
348 (add-hook 'c++-mode-hook
351 (setq indent-tabs-mode nil))
354 If you like using font-lock, you can also add this to your
358 (setq font-lock-maximum-decoration t)
359 (setq c++-font-lock-keywords-3
361 c++-font-lock-keywords-3
362 '(("\\b\\(a-zA-Z_?+_\\)\\b" 1 font-lock-variable-name-face) ("\\b\\(A-Z?+a-z_?+\\)\\b" 1 font-lock-type-face))
367 @subheading Indenting with vim
369 Although emacs indentation is the GNU standard, acceptable
370 indentation can usually be accomplished with vim. Some hints for
382 filetype plugin indent on
384 set ignorecase smartcase
387 set statusline=%F%m%r%h%w\ %@{&ff@}\ %Y\ [ASCII=\%03.3b]\ [HEX=\%02.2B]\ %04l,%04v\ %p%%\ [LEN=%L]
390 " Remove trailing whitespace on write
391 autocmd BufWritePre * :%s/\s\+$//e
394 With this @file{.vimrc}, files can be reindented automatically by
395 highlighting the lines to be indented in visual mode (use V to
396 enter visual mode) and pressing @code{=}.
398 A @file{scheme.vim} file will help improve the indentation. This
399 one was suggested by Patrick McCarty. It should be saved in
400 @file{~/.vim/after/syntax/scheme.vim}.
403 " Additional Guile-specific 'forms'
404 syn keyword schemeSyntax define-public define*-public
405 syn keyword schemeSyntax define* lambda* let-keywords*
406 syn keyword schemeSyntax defmacro defmacro* define-macro
407 syn keyword schemeSyntax defmacro-public defmacro*-public
408 syn keyword schemeSyntax use-modules define-module
409 syn keyword schemeSyntax define-method define-class
411 " Additional LilyPond-specific 'forms'
412 syn keyword schemeSyntax define-markup-command define-markup-list-command
413 syn keyword schemeSyntax define-safe-public define-music-function
414 syn keyword schemeSyntax def-grace-function
416 " All of the above should influence indenting too
417 set lw+=define-public,define*-public
418 set lw+=define*,lambda*,let-keywords*
419 set lw+=defmacro,defmacro*,define-macro
420 set lw+=defmacro-public,defmacro*-public
421 set lw+=use-modules,define-module
422 set lw+=define-method,define-class
423 set lw+=define-markup-command,define-markup-list-command
424 set lw+=define-safe-public,define-music-function
425 set lw+=def-grace-function
427 " These forms should not influence indenting
431 " Try to highlight all ly: procedures
432 syn match schemeFunc "ly:[^) ]\+"
436 @node Naming conventions
437 @subsection Naming Conventions
439 Naming conventions have been established for LilyPond
442 @subheading Classes and Types
444 Classes begin with an uppercase letter, and words
445 in class names are separated with @code{_}:
453 Member variable names end with an underscore:
461 Macro names should be written in uppercase completely,
462 with words separated by @code{_}:
468 @subheading Variables
470 Variable names should be complete words, rather than abbreviations.
471 For example, it is preferred to use @code{thickness} rather than
472 @code{th} or @code{t}.
474 Multi-word variable names in C++ should have the words separated
475 by the underscore character (@q{_}):
478 cxx_multiword_variable
481 Multi-word variable names in Scheme should have the words separated
485 scheme-multiword-variable
489 @subsection Broken code
491 Do not write broken code. This includes hardwired dependencies,
492 hardwired constants, slow algorithms and obvious limitations. If
493 you can not avoid it, mark the place clearly, and add a comment
494 explaining shortcomings of the code.
496 Ideally, the comment marking the shortcoming would include
497 TODO, so that it is marked for future fixing.
499 We reject broken-in-advance on principle.
503 @subsection Code comments
505 Comments may not be needed if descriptive variable names are used
506 in the code and the logic is straightforward. However, if the
507 logic is difficult to follow, and particularly if non-obvious
508 code has been included to resolve a bug, a comment describing
509 the logic and/or the need for the non-obvious code should be included.
511 There are instances where the current code could be commented better.
512 If significant time is required to understand the code as part of
513 preparing a patch, it would be wise to add comments reflecting your
514 understanding to make future work easier.
517 @node Handling errors
518 @subsection Handling errors
520 As a general rule, you should always try to continue computations,
521 even if there is some kind of error. When the program stops, it
522 is often very hard for a user to pinpoint what part of the input
523 causes an error. Finding the culprit is much easier if there is
524 some viewable output.
526 So functions and methods do not return errorcodes, they never
527 crash, but report a programming_error and try to carry on.
529 Error and warning messages need to be localized.
533 @subsection Localization
535 This document provides some guidelines to help programmers write
537 messages. To help translations, user messages must follow
538 uniform conventions. Follow these rules when coding for LilyPond.
539 Hopefully, this can be replaced by general GNU guidelines in the
540 future. Even better would be to have an English (en_BR, en_AM)
541 guide helping programmers writing consistent messages for all GNU
544 Non-preferred messages are marked with `+'. By convention,
545 ungrammatical examples are marked with `*'. However, such ungrammatical
546 examples may still be preferred.
551 Every message to the user should be localized (and thus be marked
552 for localization). This includes warning and error messages.
555 Do not localize/gettextify:
559 `programming_error ()'s
562 `programming_warning ()'s
568 output strings (PostScript, TeX, etc.)
573 Messages to be localized must be encapsulated in `_ (STRING)' or
574 `_f (FORMAT, ...)'. E.g.:
577 warning (_ ("need music in a score"));
578 error (_f ("cannot open file: `%s'", file_name));
581 In some rare cases you may need to call `gettext ()' by hand. This
582 happens when you pre-define (a list of) string constants for later
583 use. In that case, you'll probably also need to mark these string
584 constants for translation, using `_i (STRING)'. The `_i' macro is
585 a no-op, it only serves as a marker for `xgettext'.
588 char const* messages[] = @{
589 _i ("enable debugging output"),
590 _i ("ignore lilypond version"),
597 puts (gettext (messages i));
601 See also @file{flower/getopt-long.cc} and @file{lily/main.cc}.
604 Do not use leading or trailing whitespace in messages. If you need
605 whitespace to be printed, prepend or append it to the translated
609 message ("Calculating line breaks..." + " ");
613 Error or warning messages displayed with a file name and line
614 number never start with a capital, eg,
617 foo.ly: 12: not a duration: 3
620 Messages containing a final verb, or a gerund (`-ing'-form) always
621 start with a capital. Other (simpler) messages start with a
627 Not declaring: `foo'.
631 Avoid abbreviations or short forms, use `cannot' and `do not'
632 rather than `can't' or `don't'
633 To avoid having a number of different messages for the same
634 situation, well will use quoting like this `"message: `%s'"' for all
635 strings. Numbers are not quoted:
638 _f ("cannot open file: `%s'", name_str)
639 _f ("cannot find character number: %d", i)
643 Think about translation issues. In a lot of cases, it is better to
644 translate a whole message. English grammar must not be imposed on the
645 translator. So, instead of
648 stem at + moment.str () + does not fit in beam
654 _f ("stem at %s does not fit in beam", moment.str ())
658 Split up multi-sentence messages, whenever possible. Instead of
661 warning (_f ("out of tune! Can't find: `%s'", "Key_engraver"));
662 warning (_f ("cannot find font `%s', loading default", font_name));
668 warning (_ ("out of tune:"));
669 warning (_f ("cannot find: `%s', "Key_engraver"));
670 warning (_f ("cannot find font: `%s', font_name));
671 warning (_f ("Loading default font"));
675 If you must have multiple-sentence messages, use full punctuation.
676 Use two spaces after end of sentence punctuation. No punctuation
677 (esp. period) is used at the end of simple messages.
680 _f ("Non-matching braces in text `%s', adding braces", text)
681 _ ("Debug output disabled. Compiled with NPRINT.")
682 _f ("Huh? Not a Request: `%s'. Ignoring.", request)
686 Do not modularize too much; words frequently cannot be translated
687 without context. It is probably safe to treat most occurrences of
688 words like stem, beam, crescendo as separately translatable words.
691 When translating, it is preferable to put interesting information
692 at the end of the message, rather than embedded in the middle.
693 This especially applies to frequently used messages, even if this
694 would mean sacrificing a bit of eloquency. This holds for original
695 messages too, of course.
698 en: cannot open: `foo.ly'
699 + nl: kan `foo.ly' niet openen (1)
700 kan niet openen: `foo.ly'* (2)
701 niet te openen: `foo.ly'* (3)
705 The first nl message, although grammatically and stylistically
706 correct, is not friendly for parsing by humans (even if they speak
707 dutch). I guess we would prefer something like (2) or (3).
710 Do not run make po/po-update with GNU gettext < 0.10.35
716 @node Debugging LilyPond
717 @section Debugging LilyPond
719 The most commonly used tool for debugging LilyPond is the GNU
720 debugger gdb. The gdb tool is used for investigating and debugging
721 core Lilypond code written in C++. Another tool is available for
722 debugging Scheme code using the Guile debugger. This section
723 describes how to use both gdb and the Guile Debugger.
726 * Debugging overview::
727 * Debugging C++ code::
728 * Debugging Scheme code::
731 @node Debugging overview
732 @subsection Debugging overview
734 Using a debugger simplifies troubleshooting in at least two ways.
736 First, breakpoints can be set to pause execution at any desired point.
737 Then, when execution has paused, debugger commands can be issued to
738 explore the values of various variables or to execute functions.
740 Second, the debugger can display a stack trace, which shows the
741 sequence in which functions have been called and the arguments
742 passed to the called functions.
744 @node Debugging C++ code
745 @subsection Debugging C++ code
747 The GNU debugger, gdb, is the principal tool for debugging C++ code.
749 @subheading Compiling LilyPond for use with gdb
751 In order to use gdb with LilyPond, it is necessary to compile
752 LilyPond with debugging information. This is accomplished by running
753 the following commands in the main LilyPond source directory.
756 ./configure --disable-optimising
760 This will create a version of LilyPond containing debugging
761 information that will allow the debugger to tie the source code
762 to the compiled code.
764 You should not do @var{make install} if you want to use a debugger
765 with LilyPond. The @var{make install} command will strip debugging
766 information from the LilyPond binary.
768 @subheading Typical gdb usage
770 Once you have compiled the Lilypond image with the necessary
771 debugging information it will have been written to a location in a
772 subfolder of your current working directory:
778 This is important as you will need to let gdb know where to find the
779 image containing the symbol tables. You can invoke gdb from the
780 command line using the following:
786 This loads the LilyPond symbol tables into gdb. Then, to run
787 LilyPond on @file{test.ly} under the debugger, enter the following:
796 As an alternative to running gdb at the command line you may try
797 a graphical interface to gdb such as ddd:
803 You can also use sets of standard gdb commands stored in a .gdbinit
804 file (see next section).
806 @subheading Typical .gdbinit files
808 The behavior of gdb can be readily customized through the use of a
809 @var{.gdbinit} file. A @var{.gdbinit} file is a file named
810 @var{.gdbinit} (notice the @qq{.} at the beginning of the file name)
811 that is placed in a user's home directory.
813 The @var{.gdbinit} file below is from Han-Wen. It sets breakpoints
814 for all errors and defines functions for displaying scheme objects
815 (ps), grobs (pgrob), and parsed music expressions (pmusic).
818 file lily/out/lilypond
820 b Grob::programming_error
823 print ly_display_scm($arg0)
826 print ly_display_scm($arg0->self_scm_)
827 print ly_display_scm($arg0->mutable_property_alist_)
828 print ly_display_scm($arg0->immutable_property_alist_)
829 print ly_display_scm($arg0->object_alist_)
832 print ly_display_scm($arg0->self_scm_)
833 print ly_display_scm($arg0->mutable_property_alist_)
834 print ly_display_scm($arg0->immutable_property_alist_)
838 @node Debugging Scheme code
839 @subsection Debugging Scheme code
841 Scheme code can be developed using the Guile command line
842 interpreter @code{top-repl}. You can either investigate
843 interactively using just Guile or you can use the debugging
844 tools available within Guile.
846 @subheading Using Guile interactively with LilyPond
848 In order to experiment with Scheme programming in the LilyPond
849 environment, it is necessary to have a Guile interpreter that
850 has all the LilyPond modules loaded. This requires the following
853 First, define a Scheme symbol for the active module in the @file{.ly} file:
856 #(module-define! (resolve-module '(guile-user))
857 'lilypond-module (current-module))
860 Now place a Scheme function in the @file{.ly} file that gives an
861 interactive Guile prompt:
867 When the @file{.ly} file is compiled, this causes the compilation to be
868 interrupted and an interactive guile prompt to appear. Once the
869 guile prompt appears, the LilyPond active module must be set as the
870 current guile module:
873 guile> (set-current-module lilypond-module)
876 You can demonstrate these commands are operating properly by typing the name
877 of a LilyPond public scheme function to check it has been defined:
880 guile> fret-diagram-verbose-markup
881 #<procedure fret-diagram-verbose-markup (layout props marking-list)>
884 If the LilyPond module has not been correctly loaded, an error
885 message will be generated:
888 guile> fret-diagram-verbose-markup
889 ERROR: Unbound variable: fret-diagram-verbose-markup
890 ABORT: (unbound-variable)
893 Once the module is properly loaded, any valid LilyPond Scheme
894 expression can be entered at the interactive prompt.
896 After the investigation is complete, the interactive guile
897 interpreter can be exited:
903 The compilation of the @file{.ly} file will then continue.
905 @subheading Using the Guile debugger
907 To set breakpoints and/or enable tracing in Scheme functions, put
910 \include "guile-debugger.ly"
913 in your input file after any scheme procedures you have defined in
914 that file. This will invoke the Guile command-line after having set
915 up the environment for the debug command-line. When your input file
916 is processed, a guile prompt will be displayed. You may now enter
917 commands to set up breakpoints and enable tracing by the Guile debugger.
919 @subheading Using breakpoints
921 At the guile prompt, you can set breakpoints with
922 the @code{set-break!} procedure:
925 guile> (set-break! my-scheme-procedure)
928 Once you have set the desired breakpoints, you exit the guile repl frame
935 Then, when one of the scheme routines for which you have set
936 breakpoints is entered, guile will interrupt execution in a debug
937 frame. At this point you will have access to Guile debugging
938 commands. For a listing of these commands, type:
944 Alternatively you may code the breakpoints in your Lilypond source
945 file using a command such as:
948 #(set-break! my-scheme-procedure)
951 immediately after the @code{\include} statement. In this case the
952 breakpoint will be set straight after you enter the @code{(quit)}
953 command at the guile prompt.
955 Embedding breakpoint commands like this is particularly useful if
956 you want to look at how the Scheme procedures in the @file{.scm}
957 files supplied with LilyPond work. To do this, edit the file in
958 the relevant directory to add this line near the top:
961 (use-modules (scm guile-debugger))
964 Now you can set a breakpoint after the procedure you are interested
965 in has been declared. For example, if you are working on routines
966 called by @var{print-book-with} in @file{lily-library.scm}:
969 (define (print-book-with parser book process-procedure)
970 (let* ((paper (ly:parser-lookup parser '$defaultpaper))
971 (layout (ly:parser-lookup parser '$defaultlayout))
972 (outfile-name (get-outfile-name parser)))
973 (process-procedure book paper layout outfile-name)))
975 (define-public (print-book-with-defaults parser book)
976 (print-book-with parser book ly:book-process))
978 (define-public (print-book-with-defaults-as-systems parser book)
979 (print-book-with parser book ly:book-process-to-systems))
983 At this point in the code you could add this to set a breakpoint at
987 (set-break! print-book-with)
990 @subheading Tracing procedure calls and evaluator steps
992 Two forms of trace are available:
995 (set-trace-call! my-scheme-procedure)
1001 (set-trace-subtree! my-scheme-procedure)
1004 @code{set-trace-call!} causes Scheme to log a line to the standard
1005 output to show when the procedure is called and when it exits.
1007 @code{set-trace-subtree!} traces every step the Scheme evaluator
1008 performs in evaluating the procedure.
1010 @node Tracing object relationships
1011 @section Tracing object relationships
1013 Understanding the LilyPond source often boils down to figuring out what
1014 is happening to the Grobs. Where (and why) are they being created,
1015 modified and destroyed? Tracing Lily through a debugger in order to
1016 identify these relationships can be time-consuming and tedious.
1018 In order to simplify this process, a facility has been added to
1019 display the grobs that are created and the properties that are set
1020 and modified. Although it can be complex to get set up, once set up
1021 it easily provides detailed information about the life of grobs
1022 in the form of a network graph.
1024 Each of the steps necessary to use the graphviz utility
1029 @item Installing graphviz
1031 In order to create the graph of the object relationships, it is
1032 first necessary to install Graphviz. graphviz is available for a
1033 number of different platforms:
1036 @uref{http://www.graphviz.org/Download..php}
1039 @item Modifying config.make
1041 In order for the Graphviz tool to work, config.make must be modified.
1042 It is probably a good idea to first save a copy of config.make under
1043 a different name. Then, edit config.make by removing every occurrence
1046 @item Rebuilding LilyPond
1048 The executable code of LilyPond must be rebuilt from scratch:
1051 make -C lily clean && make -C lily
1054 @item Create a graphviz-compatible @file{.ly} file
1056 In order to use the graphviz utility, the @file{.ly} file must include
1057 @file{ly/graphviz-init.ly}, and should then specify the
1058 grobs and symbols that should be tracked. An example of this
1059 is found in @file{input/regression/graphviz.ly}.
1061 @item Run lilypond with output sent to a log file
1063 The Graphviz data is sent to stderr by lilypond, so it is
1064 necessary to redirect stderr to a logfile:
1067 lilypond graphviz.ly 2> graphviz.log
1070 @item Edit the logfile
1072 The logfile has standard lilypond output, as well as the Graphviz
1073 output data. Delete everything from the beginning of the file
1074 up to but not including the first occurrence of @code{digraph}.
1076 Also, delete the final liypond message about successs from the end
1079 @item Process the logfile with @code{dot}
1081 The directed graph is created from the log file with the program
1085 dot -Tpdf graphviz.log > graphviz.pdf
1090 The pdf file can then be viewed with any pdf viewer.
1092 When compiled without @code{-DNDEBUG}, lilypond may run slower
1093 than normal. The original configuration can be restored by either
1094 renaming the saved copy of @code{config.make} or rerunning
1095 @code{configure}. Then rebuild lilypond with
1098 make -C lily clean && make -C lily
1102 @node Adding or modifying features
1103 @section Adding or modifying features
1105 When a new feature is to be added to LilyPond, it is necessary to
1106 ensure that the feature is properly integrated to maintain
1107 its long-term support. This section describes the steps necessary
1108 for feature addition and modification.
1113 * Write regression tests::
1114 * Write convert-ly rule::
1115 * Automatically update documentation::
1116 * Manually update documentation::
1117 * Edit changes.tely::
1118 * Verify successful build::
1119 * Verify regression tests::
1120 * Post patch for comments::
1122 * Closing the issues::
1125 @node Write the code
1126 @subsection Write the code
1128 You should probably create a new git branch for writing the code, as that
1129 will separate it from the master branch and allow you to continue
1130 to work on small projects related to master.
1132 Please be sure to follow the rules for programming style discussed
1133 earlier in this chapter.
1136 @node Write regression tests
1137 @subsection Write regression tests
1139 In order to demonstrate that the code works properly, you will
1140 need to write one or more regression tests. These tests are
1141 typically @file{.ly} files that are found in @file{input/regression}.
1143 Regression tests should be as brief as possible to demonstrate the
1144 functionality of the code.
1146 Regression tests should generally cover one issue per test. Several
1147 short, single-issue regression tests are preferred to a single, long,
1148 multiple-issue regression test.
1150 Use existing regression tests as templates to demonstrate the type of
1151 header information that should be included in a regression test.
1154 @node Write convert-ly rule
1155 @subsection Write convert-ly rule
1157 If the modification changes the input syntax, a convert-ly rule
1158 should be written to automatically update input files from older
1161 convert-ly rules are found in python/convertrules.py
1163 If possible, the convert-ly rule should allow automatic updating
1164 of the file. In some cases, this will not be possible, so the
1165 rule will simply point out to the user that the feature needs
1168 @subsubheading Updating version numbers
1170 If a development release occurs between you writing your patch and
1171 having it approved+pushed, you will need to update the version
1172 numbers in your tree. This can be done with:
1175 scripts/auxiliar/update-patch-version old.version.number new.version.number
1178 It will change all files in git, so use with caution and examine
1182 @node Automatically update documentation
1183 @subsection Automatically update documentation
1185 @command{convert-ly} should be used to update the documentation,
1186 the snippets, and the regression tests. This not only makes the
1187 necessary syntax changes, it also tests the @command{convert-ly}
1190 The automatic updating is performed by moving to the top-level
1191 source directory, then running:
1194 scripts/auxiliar/update-with-convert-ly.sh
1197 If you did an out-of-tree build, pass in the relative path:
1200 BUILD_DIR=../build-lilypond/ scripts/auxiliar/update-with-convert-ly.sh
1204 @node Manually update documentation
1205 @subsection Manually update documentation
1207 Where the convert-ly rule is not able to automatically update the inline
1208 lilypond code in the documentation (i.e. if a NOT_SMART rule is used), the
1209 documentation must be manually updated. The inline snippets that require
1210 changing must be changed in the English version of the docs and all
1211 translated versions. If the inline code is not changed in the
1212 translated documentation, the old snippets will show up in the
1213 English version of the documentation.
1215 Where the convert-ly rule is not able to automatically update snippets
1216 in Documentation/snippets/, those snippets must be manually updated.
1217 Those snippets should be copied to Documentation/snippets/new. The
1218 comments at the top of the snippet describing its automatic generation
1219 should be removed. All translated texidoc strings should be removed.
1220 The comment @qq{% begin verbatim} should be removed. The syntax of
1221 the snippet should then be manually edited.
1223 Where snippets in Documentation/snippets are made obsolete, the snippet
1224 should be copied to Documentation/snippets/new. The comments and
1225 texidoc strings should be removed as described above. Then the body
1226 of the snippet should be changed to:
1230 This snippet is deprecated as of version X.Y.Z and
1231 will be removed from the documentation.
1236 where X.Y.Z is the version number for which the convert-ly rule was
1239 Update the snippet files by running:
1242 scripts/auxiliar/makelsr.py
1245 Where the convert-ly rule is not able to automatically update regression
1246 tests, the regression tests in input/regression should be manually
1249 Although it is not required, it is helpful if the developer
1250 can write relevant material for inclusion in the Notation
1251 Reference. If the developer does not feel qualified to write
1252 the documentation, a documentation editor will be able to
1253 write it from the regression tests. The text that is added to
1254 or removed from the documentation should be changed only in
1255 the English version.
1258 @node Edit changes.tely
1259 @subsection Edit changes.tely
1261 An entry should be added to Documentation/changes.tely to describe
1262 the feature changes to be implemented. This is especially important
1263 for changes that change input file syntax.
1265 Hints for changes.tely entries are given at the top of the file.
1267 New entries in changes.tely go at the top of the file.
1269 The changes.tely entry should be written to show how the new change
1270 improves LilyPond, if possible.
1273 @node Verify successful build
1274 @subsection Verify successful build
1276 When the changes have been made, successful completion must be
1284 When these commands complete without error, the patch is
1285 considered to function successfully.
1287 Developers on Windows who are unable to build LilyPond should
1288 get help from a Linux or OSX developer to do the make tests.
1291 @node Verify regression tests
1292 @subsection Verify regression tests
1294 In order to avoid breaking LilyPond, it is important to verify that
1295 the regression tests succeed, and that no unwanted changes are
1296 introduced into the output. This process is described in
1297 @ref{Regtest comparison}.
1299 @subheading Typical developer's edit/compile/test cycle
1301 TODO: is @code{[-j@var{X} CPU_COUNT=@var{X}]} useful for
1302 @code{test-baseline}, @code{check}, @code{clean},
1303 @code{test-redo}? Neil Puttock says it is useful for
1304 everything but @code{clean}, which is disk-limited.
1305 Need to check formally.
1314 make [-j@var{X} CPU_COUNT=@var{X}] check
1318 Edit/compile/test cycle:
1321 @emph{## edit source files, then...}
1323 make clean @emph{## only if needed (see below)}
1324 make [-j@var{X}] @emph{## only if needed (see below)}
1325 make test-redo @emph{## redo files differing from baseline}
1326 make [-j@var{X} CPU_COUNT=@var{X}] check @emph{## CPU_COUNT here?}
1337 If you modify any source files that have to be compiled (such as
1338 @file{.cc} or @file{.hh} files in @file{flower/} or @file{lily/}),
1339 then you must run @command{make} before @command{make test-redo},
1340 so @command{make} can compile the modified files and relink all
1341 the object files. If you only modify files which are interpreted,
1342 like those in the @file{scm/} and @file{ly/} directories, then
1343 @command{make} is not needed before @command{make test-redo}.
1345 TODO: Fix the following paragraph. You can do @command{rm mf/out/*}
1346 instead of make clean, and you can probably do
1347 @command{make -C mf/ clean} as well, but I haven't checked it -- cds
1349 Also, if you modify any font definitions in the @file{mf/}
1350 directory then you must run @command{make clean} and
1351 @command{make} before running @command{make test-redo}. This will
1352 recompile everything, whether modified or not, and takes a lot
1355 Running @command{make@tie{}check} will leave an HTML page
1356 @file{out/test-results/index.html}. This page shows all the
1357 important differences that your change introduced, whether in the
1358 layout, MIDI, performance or error reporting.
1363 @node Post patch for comments
1364 @subsection Post patch for comments
1366 See @ref{Uploading a patch for review}.
1370 @subsection Push patch
1372 Once all the comments have been addressed, the patch can be pushed.
1374 If the author has push privileges, the author will push the patch.
1375 Otherwise, a developer with push privileges will push the patch.
1378 @node Closing the issues
1379 @subsection Closing the issues
1381 Once the patch has been pushed, all the relevant issues should be
1384 On Rietveld, the author should log in an close the issue either by
1385 using the @q{Edit Issue} link, or by clicking the circled x icon
1386 to the left of the issue name.
1388 If the changes were in response to a feature request on the Google
1389 issue tracker for LilyPond, the author should change the status to
1390 Fixed and a tag @q{fixed_x_y_z} should be added, where the patch was
1391 fixed in version x.y.z. If
1392 the author does not have privileges to change the status, an email
1393 should be sent to bug-lilypond requesting the BugMeister to change
1397 @node Iterator tutorial
1398 @section Iterator tutorial
1400 TODO -- this is a placeholder for a tutorial on iterators
1402 Iterators are routines written in C++ that process music expressions
1403 and sent the music events to the appropriate engravers and/or
1407 @node Engraver tutorial
1408 @section Engraver tutorial
1410 Engravers are C++ classes that catch music events and
1411 create the appropriate grobs for display on the page. Though the
1412 majority of engravers are responsible for the creation of a single grob,
1413 in some cases (e.g. @code{New_fingering_engraver}), several different grobs
1416 Engravers listen for events and acknowledge grobs. Events are passed to
1417 the engraver in time-step order during the iteration phase. Grobs are
1418 made available to the engraver when they are created by other engravers
1419 during the iteration phase.
1423 * Useful methods for information processing::
1424 * Translation process::
1425 * Preventing garbage collection for SCM member variables::
1426 * Listening to music events::
1427 * Acknowledging grobs::
1428 * Engraver declaration/documentation::
1431 @node Useful methods for information processing
1432 @subsection Useful methods for information processing
1434 An engraver inherits the following public methods from the Translator
1435 base class, which can be used to process listened events and acknowledged
1439 @item @code{virtual void initialize ()}
1440 @item @code{void start_translation_timestep ()}
1441 @item @code{void process_music ()}
1442 @item @code{void process_acknowledged ()}
1443 @item @code{void stop_translation_timestep ()}
1444 @item @code{virtual void finalize ()}
1447 These methods are listed in order of translation time, with
1448 @code{initialize ()} and @code{finalize ()} bookending the whole
1449 process. @code{initialize ()} can be used for one-time initialization
1450 of context properties before translation starts, whereas
1451 @code{finalize ()} is often used to tie up loose ends at the end of
1452 translation: for example, an unterminated spanner might be completed
1453 automatically or reported with a warning message.
1456 @node Translation process
1457 @subsection Translation process
1459 At each timestep in the music, translation proceeds by calling the
1460 following methods in turn:
1462 @code{start_translation_timestep ()} is called before any user
1463 information enters the translators, i.e., no property operations
1464 (\set, \override, etc.) or events have been processed yet.
1466 @code{process_music ()} and @code{process_acknowledged ()} are called
1467 after all events in the current time step have been heard, or all
1468 grobs in the current time step have been acknowledged. The latter
1469 tends to be used exclusively with engravers which only acknowledge
1470 grobs, whereas the former is the default method for main processing
1473 @code{stop_translation_timestep ()} is called after all user
1474 information has been processed prior to beginning the translation for
1478 @node Preventing garbage collection for SCM member variables
1479 @subsection Preventing garbage collection for SCM member variables
1481 In certain cases, an engraver might need to ensure private Scheme
1482 variables (with type SCM) do not get swept away by Guile's garbage
1483 collector: for example, a cache of the previous key signature which
1484 must persist between timesteps. The method
1485 @code{virtual derived_mark () const} can be used in such cases:
1488 Engraver_name::derived_mark ()
1490 scm_gc_mark (private_scm_member_)
1495 @node Listening to music events
1496 @subsection Listening to music events
1498 External interfaces to the engraver are implemented by protected
1499 macros including one or more of the following:
1502 @item @code{DECLARE_TRANSLATOR_LISTENER (event_name)}
1503 @item @code{IMPLEMENT_TRANSLATOR_LISTENER (Engraver_name, event_name)}
1507 where @var{event_name} is the type of event required to provide the
1508 input the engraver needs and @var{Engraver_name} is the name of the
1511 Following declaration of a listener, the method is implemented as follows:
1514 IMPLEMENT_TRANSLATOR_LISTENER (Engraver_name, event_name)
1516 Engraver_name::listen_event_name (Stream event *event)
1518 ...body of listener method...
1523 @node Acknowledging grobs
1524 @subsection Acknowledging grobs
1526 Some engravers also need information from grobs as they are created
1527 and as they terminate. The mechanism and methods to obtain this
1528 information are set up by the macros:
1531 @item @code{DECLARE_ACKNOWLEDGER (grob_interface)}
1532 @item @code{DECLARE_END_ACKNOWLEDGER (grob_interface)}
1535 where @var{grob_interface} is an interface supported by the
1536 grob(s) which should be acknowledged. For example, the following
1537 code would declare acknowledgers for a @code{NoteHead} grob (via the
1538 @code{note-head-interface}) and any grobs which support the
1539 @code{side-position-interface}:
1542 @code{DECLARE_ACKNOWLEDGER (note_head)}
1543 @code{DECLARE_ACKNOWLEDGER (side_position)}
1546 The @code{DECLARE_END_ACKNOWLEDGER ()} macro sets up a spanner-specific
1547 acknowledger which will be called whenever a spanner ends.
1549 Following declaration of an acknowledger, the method is coded as follows:
1553 Engraver_name::acknowledge_interface_name (Grob_info info)
1555 ...body of acknowledger method...
1560 @node Engraver declaration/documentation
1561 @subsection Engraver declaration/documentation
1563 An engraver must have a public macro
1566 @item @code{TRANSLATOR_DECLARATIONS (Engraver_name)}
1570 where @code{Engraver_name} is the name of the engraver. This
1571 defines the common variables and methods used by every engraver.
1573 At the end of the engraver file, one or both of the following
1574 macros are generally called to document the engraver in the
1575 Internals Reference:
1578 @item @code{ADD_ACKNOWLEDGER (Engraver_name, grob_interface)}
1579 @item @code{ADD_TRANSLATOR (Engraver_name, Engraver_doc,
1580 Engraver_creates, Engraver_reads, Engraver_writes)}
1584 where @code{Engraver_name} is the name of the engraver, @code{grob_interface}
1585 is the name of the interface that will be acknowledged,
1586 @code{Engraver_doc} is a docstring for the engraver,
1587 @code{Engraver_creates} is the set of grobs created by the engraver,
1588 @code{Engraver_reads} is the set of properties read by the engraver,
1589 and @code{Engraver_writes} is the set of properties written by
1592 The @code{ADD_ACKNOWLEDGER} and @code{ADD_TRANSLATOR} macros use a
1593 non-standard indentation system. Each interface, grob, read property,
1594 and write property is on its own line, and the closing parenthesis
1595 and semicolon for the macro all occupy a separate line beneath the final
1596 interface or write property. See existing engraver files for more
1600 @node Callback tutorial
1601 @section Callback tutorial
1603 TODO -- This is a placeholder for a tutorial on callback functions.
1605 @node LilyPond scoping
1606 @section LilyPond scoping
1608 The Lilypond language has a concept of scoping, i.e. you can do
1614 (display (+ foo 2)))
1617 @noindent with @code{\paper}, @code{\midi} and @code{\header} being
1618 nested scope inside the @file{.ly} file-level scope. @w{@code{foo = 1}}
1619 is translated in to a scheme variable definition.
1621 This implemented using modules, with each scope being an anonymous
1622 module that imports its enclosing scope's module.
1624 Lilypond's core, loaded from @file{.scm} files, is usually placed in the
1625 @code{lily} module, outside the @file{.ly} level. In the case of
1632 we want to reuse the built-in definitions, without changes effected in
1633 user-level @file{a.ly} leaking into the processing of @file{b.ly}.
1635 The user-accessible definition commands have to take care to avoid
1636 memory leaks that could occur when running multiple files. All
1637 information belonging to user-defined commands and markups is stored in
1638 a manner that allows it to be garbage-collected when the module is
1639 dispersed, either by being stored module-locally, or in weak hash
1642 @node LilyPond miscellany
1643 @section LilyPond miscellany
1645 This is a place to dump information that may be of use to developers
1646 but doesn't yet have a proper home. Ideally, the length of this section
1647 would become zero as items are moved to other homes.
1651 * Spacing algorithms::
1652 * Info from Han-Wen email::
1653 * Music functions and GUILE debugging::
1656 @node Spacing algorithms
1657 @subsection Spacing algorithms
1659 Here is information from an email exchange about spacing algorithms.
1661 On Thu, 2010-02-04 at 15:33 -0500, Boris Shingarov wrote:
1662 I am experimenting with some modifications to the line breaking code,
1663 and I am stuck trying to understand how some of it works. So far my
1664 understanding is that Simple_spacer operates on a vector of Grobs, and
1665 it is a well-known Constrained-QP problem (rods = constraints, springs
1666 = quadratic function to minimize). What I don't understand is, if the
1667 spacer operates at the level of Grobs, which are built at an earlier
1668 stage in the pipeline, how are the changes necessitated by differences
1669 in line breaking, taken into account? in other words, if I take the
1670 last measure of a line and place it on the next line, it is not just a
1671 matter of literally moving that graphic to where the start of the next
1672 line is, but I also need to draw a clef, key signature, and possibly
1673 other fundamental things -- but at that stage in the rendering
1674 pipeline, is it not too late??
1676 Joe Neeman answered:
1678 We create lots of extra grobs (eg. a BarNumber at every bar line) but
1679 most of them are not drawn. See the break-visibility property in
1683 @node Info from Han-Wen email
1684 @subsection Info from Han-Wen email
1686 In 2004, Douglas Linhardt decided to try starting a document that would
1687 explain LilyPond architecture and design principles. The material below
1688 is extracted from that email, which can be found at
1689 @uref{http://thread.gmane.org/gmane.comp.gnu.lilypond.devel/2992}.
1690 The headings reflect questions from Doug or comments from Han-Wen;
1691 the body text are Han-Wen's answers.
1693 @subheading Figuring out how things work.
1695 I must admit that when I want to know how a program works, I use grep
1696 and emacs and dive into the source code. The comments and the code
1697 itself are usually more revealing than technical documents.
1699 @subheading What's a grob, and how is one used?
1701 Graphical object - they are created from within engravers, either as
1702 Spanners (derived class) -slurs, beams- or Items (also a derived
1703 class) -notes, clefs, etc.
1705 There are two other derived classes System (derived from Spanner,
1706 containing a "line of music") and Paper_column (derived from Item, it
1707 contains all items that happen at the same moment). They are separate
1708 classes because they play a special role in the linebreaking process.
1710 @subheading What's a smob, and how is one used?
1712 A C(++) object that is encapsulated so it can be used as a Scheme
1713 object. See GUILE info, "19.3 Defining New Types (Smobs)"
1715 @@subheading When is each C++ class constructed and used
1722 In the parser.yy see the macro calls MAKE_MUSIC_BY_NAME().
1727 Constructed during "interpreting" phase.
1732 Executive branch of Contexts, plugins that create grobs, usually one
1733 engraver per grob type. Created together with context.
1743 These are not C++ classes per se. The idea of a Grob interface hasn't
1744 crystallized well. ATM, an interface is a symbol, with a bunch of grob
1745 properties. They are not objects that are created or destroyed.
1750 Objects that walk through different music classes, and deliver events
1751 in a synchronized way, so that notes that play together are processed
1752 at the same moment and (as a result) end up on the same horizontal position.
1754 Created during interpreting phase.
1756 BTW, the entry point for interpreting is ly:run-translator
1757 (ly_run_translator on the C++ side)
1761 @subheading Can you get to Context properties from a Music object?
1763 You can create music object with a Scheme function that reads context
1764 properties (the \applycontext syntax). However, that function is
1765 executed during Interpreting, so you can not really get Context
1766 properties from Music objects, since music objects are not directly
1767 connected to Contexts. That connection is made by the Music_iterators
1769 @subheading Can you get to Music properties from a Context object?
1771 Yes, if you are given the music object within a Context
1772 object. Normally, the music objects enter Contexts in synchronized
1773 fashion, and the synchronization is done by Music_iterators.
1775 @subheading What is the relationship between C++ classes and Scheme objects?
1777 Smobs are C++ objects in Scheme. Scheme objects (lists, functions) are
1778 manipulated from C++ as well using the GUILE C function interface
1781 @subheading How do Scheme procedures get called from C++ functions?
1783 scm_call_*, where * is an integer from 0 to 4.
1784 Also scm_c_eval_string (), scm_eval ()
1786 @subheading How do C++ functions get called from Scheme procedures?
1788 Export a C++ function to Scheme with LY_DEFINE.
1790 @subheading What is the flow of control in the program?
1792 Good question. Things used to be clear-cut, but we have Scheme
1793 and SMOBs now, which means that interactions do not follow a very
1794 rigid format anymore. See below for an overview, though.
1796 @subheading Does the parser make Scheme procedure calls or C++ function calls?
1798 Both. And the Scheme calls can call C++ and vice versa. It's nested,
1799 with the SCM datatype as lubrication between the interactions
1801 (I think the word "lubrication" describes the process better than the
1802 traditional word "glue")
1804 @subheading How do the front-end and back-end get started?
1806 Front-end: a file is parsed, the rest follows from that. Specifically,
1808 Parsing leads to a Music + Music_output_def object (see parser.yy,
1809 definition of toplevel_expression )
1811 A Music + Music_output_def object leads to a Global_context object (see
1812 ly_run_translator ())
1814 During interpreting, Global_context + Music leads to a bunch of
1815 Contexts (see Global_translator::run_iterator_on_me ()).
1817 After interpreting, Global_context contains a Score_context (which
1818 contains staves, lyrics etc.) as a child. Score_context::get_output ()
1819 spews a Music_output object (either a Paper_score object for notation
1820 or Performance object for MIDI).
1822 The Music_output object is the entry point for the backend (see
1823 ly_render_output ()).
1825 The main steps of the backend itself are in
1830 @file{paper-score.cc} , Paper_score::process_
1833 @file{system.cc} , System::get_lines()
1836 The step, where things go from grobs to output, is in
1837 System::get_line(): each grob delivers a Stencil (a Device
1838 independent output description), which is interpreted by our
1839 outputting backends (@file{scm/output-tex.scm} and
1840 @file{scm/output-ps.scm}) to produce TeX and PS.
1844 Interactions between grobs and putting things into .tex and .ps files
1845 have gotten a little more complex lately. Jan has implemented
1846 page-breaking, so now the backend also involves Paper_book,
1847 Paper_lines and other things. This area is still heavily in flux, and
1848 perhaps not something you should want to look at.
1850 @subheading How do the front-end and back-end communicate?
1852 There is no communication from backend to front-end. From front-end to
1853 backend is simply the program flow: music + definitions gives
1854 contexts, contexts yield output, after processing, output is written
1857 @subheading Where is the functionality associated with KEYWORDs?
1859 See @file{my-lily-lexer.cc} (keywords, there aren't that many)
1860 and @file{ly/*.ly} (most of the other backslashed @code{/\words} are identifiers)
1862 @subheading What Contexts/Properties/Music/etc. are available when they are processed?
1864 What do you mean exactly with this question?
1866 See @file{ly/engraver-init.ly} for contexts,
1867 see @file{scm/define-*.scm} for other objects.
1869 @subheading How do you decide if something is a Music, Context, or Grob property?
1870 Why is part-combine-status a Music property when it seems (IMO)
1871 to be related to the Staff context?
1873 The Music_iterators and Context communicate through two channels
1875 Music_iterators can set and read context properties, idem for
1876 Engravers and Contexts
1878 Music_iterators can send "synthetic" music events (which aren't in
1879 the input) to a context. These are caught by Engravers. This is
1880 mostly a one way communication channel.
1882 part-combine-status is part of such a synthetic event, used by
1883 Part_combine_iterator to communicate with Part_combine_engraver.
1886 @subheading Deciding between context and music properties
1888 I'm adding a property to affect how \autochange works. It seems to
1889 me that it should be a context property, but the Scheme autochange
1890 procedure has a Music argument. Does this mean I should use
1893 \autochange is one of these extra strange beasts: it requires
1894 look-ahead to decide when to change staves. This is achieved by
1895 running the interpreting step twice (see
1896 @file{scm/part-combiner.scm} , at the bottom), and
1897 storing the result of the first step (where to switch
1898 staves) in a Music property. Since you want to influence that
1899 where-to-switch list, your must affect the code in
1900 make-autochange-music (@file{scm/part-combiner.scm}).
1901 That code is called directly from the parser and there are no
1902 official "parsing properties" yet, so there is no generic way
1903 to tune \autochange. We would have to invent something new
1904 for this, or add a separate argument,
1907 \autochange #around-central-C ..music..
1911 where around-central-C is some function that is called from
1912 make-autochange-music.
1914 @subheading More on context and music properties
1916 From Neil Puttock, in response to a question about transposition:
1918 Context properties (using \set & \unset) are tied to engravers: they
1919 provide information relevant to the generation of graphical objects.
1921 Since transposition occurs at the music interpretation stage, it has
1922 no direct connection with engravers: the pitch of a note is fixed
1923 before a notehead is created. Consider the following minimal snippet:
1929 This generates (simplified) a NoteEvent, with its pitch and duration
1930 as event properties,
1936 (ly:make-duration 2 0 1 1)
1938 (ly:make-pitch 0 0 0)
1941 which the Note_heads_engraver hears. It passes this information on to
1942 the NoteHead grob it creates from the event, so the head's correct
1943 position and duration-log can be determined once it's ready for
1946 If we transpose the snippet,
1949 \transpose c d @{ c' @}
1952 the pitch is changed before it reaches the engraver (in fact, it
1953 happens just after the parsing stage with the creation of a
1954 TransposedMusic music object):
1960 (ly:make-duration 2 0 1 1)
1962 (ly:make-pitch 0 1 0)
1965 You can see an example of a music property relevant to transposition:
1969 \transpose c d @{ c'2 \withMusicProperty #'untransposable ##t c' @}
1972 -> the second c' remains untransposed.
1974 Take a look at @file{lily/music.cc} to see where the transposition takes place.
1977 @subheading How do I tell about the execution environment?
1979 I get lost figuring out what environment the code I'm looking at is in when it
1980 executes. I found both the C++ and Scheme autochange code. Then I was trying
1981 to figure out where the code got called from. I finally figured out that the
1982 Scheme procedure was called before the C++ iterator code, but it took me a
1983 while to figure that out, and I still didn't know who did the calling in the
1984 first place. I only know a little bit about Flex and Bison, so reading those
1985 files helped only a little bit.
1987 @emph{Han-Wen:} GDB can be of help here. Set a breakpoint in C++, and run. When you
1988 hit the breakpoint, do a backtrace. You can inspect Scheme objects
1989 along the way by doing
1992 p ly_display_scm(obj)
1995 this will display OBJ through GUILE.
1997 @node Music functions and GUILE debugging
1998 @subsection Music functions and GUILE debugging
2000 Ian Hulin was trying to do some debugging in music functions, and
2001 came up with the following question
2004 I'm working on the Guile Debugger Stuff, and would like to try
2005 debugging a music function definition such as:
2008 conditionalMark = #(define-music-function (parser location) ()
2009 #@{ \tag #'instrumental-part @{\mark \default@} #@} )
2012 It appears conditionalMark does not get set up as an
2013 equivalent of a Scheme
2016 (define conditionalMark = define-music-function(parser location () ...
2020 although something gets defined because Scheme apparently recognizes
2023 #(set-break! conditionalMark)
2027 later on in the file without signalling any Guile errors.
2029 However the breakpoint trap is never encountered as
2030 define-music-function passed things on to ly:make-music-function,
2031 which is really C++ code ly_make_music_function, so Guile never
2032 finds out about the breakpoint.
2034 Han-Wen answered as follows:
2036 You can see the definition by doing
2039 #(display conditionalMark)
2043 inside the @file{.ly} file.
2045 The breakpoint failing may have to do with the call sequence. See
2046 @file{parser.yy}, run_music_function(). The function is called directly from
2047 C++, without going through the GUILE evaluator, so I think that is why
2048 there is no debugger trap.