1 @c -*- coding: utf-8; mode: texinfo; -*-
3 @chapter Programming work
6 * Overview of LilyPond architecture::
7 * LilyPond programming languages::
8 * Programming without compiling::
11 * Debugging LilyPond::
12 * Tracing object relationships::
13 * Adding or modifying features::
18 * LilyPond miscellany::
21 @node Overview of LilyPond architecture
22 @section Overview of LilyPond architecture
24 LilyPond processes the input file into graphical and musical output in a
25 number of stages. This process, along with the types of routines that
26 accomplish the various stages of the process, is described in this section. A
27 more complete description of the LilyPond architecture and internal program
28 execution is found in Erik Sandberg's
29 @uref{http://lilypond.org/web/images/thesis-erik-sandberg.pdf, master's
32 The first stage of LilyPond processing is @emph{parsing}. In the parsing
33 process, music expressions in LilyPond input format are converted to music
34 expressions in Scheme format. In Scheme format, a music expression is a list
35 in tree form, with nodes that indicate the relationships between various music
36 events. The LilyPond parser is written in Bison.
38 The second stage of LilyPond processing is @emph{iterating}. Iterating
39 assigns each music event to a context, which is the environment in which the
40 music will be finally engraved. The context is responsible for all further
41 processing of the music. It is during the iteration stage that contexts are
42 created as necessary to ensure that every note has a Voice type context (e.g.
43 Voice, TabVoice, DrumVoice, CueVoice, MensuralVoice, VaticanaVoice,
44 GregorianTranscriptionVoice), that the Voice type contexts exist in
45 appropriate Staff type contexts, and that parallel Staff type contexts exist
46 in StaffGroup type contexts. In addition, during the iteration stage each
47 music event is assigned a moment, or a time in the music when the event
50 Each type of music event has an associated iterator. Iterators are defined in
51 *-iterator.cc. During iteration, an
52 event's iterator is called to deliver that music event to the appropriate
55 The final stage of LilyPond processing is @emph{translation}. During
56 translation, music events are prepared for graphical or midi output. The
57 translation step is accomplished by the polymorphic base class Translator
58 through its two derived classes: Engraver (for graphical output) and
59 Performer (for midi output).
61 Translators are defined in C++ files named *-engraver.cc and *-performer.cc.
62 Much of the work of translating is handled by Scheme functions,
63 which is one of the keys to LilyPond's exceptional flexibility.
65 @sourceimage{architecture-diagram,,,png}
68 @node LilyPond programming languages
69 @section LilyPond programming languages
71 Programming in LilyPond is done in a variety of programming languages. Each
72 language is used for a specific purpose or purposes. This section describes
73 the languages used and provides links to reference manuals and tutorials for
74 the relevant language.
78 The core functionality of LilyPond is implemented in C++.
80 C++ is so ubiquitous that it is difficult to identify either a reference
81 manual or a tutorial. Programmers unfamiliar with C++ will need to spend some
82 time to learn the language before attempting to modify the C++ code.
84 The C++ code calls Scheme/GUILE through the GUILE interface, which is
86 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html, GUILE
91 The LilyPond lexer is implemented in Flex, an implementation of the Unix lex
92 lexical analyser generator. Resources for Flex can be found
93 @uref{http://flex.sourceforge.net/, here}.
97 The LilyPond parser is implemented in Bison, a GNU parser generator. The
98 Bison homepage is found at @uref{http://www.gnu.org/software/bison/,
99 gnu.org}. The manual (which includes both a reference and tutorial) is
100 @uref{http://www.gnu.org/software/bison/manual/index.html, available} in a
105 GNU Make is used to control the compiling process and to build the
106 documentation and the website. GNU Make documentation is available at
107 @uref{http://www.gnu.org/software/make/manual/, the GNU website}.
109 @subsection GUILE or Scheme
111 GUILE is the dialect of Scheme that is used as LilyPond's extension language.
112 Many extensions to LilyPond are written entirely in GUILE. The
113 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html,
114 GUILE Reference Manual} is available online.
116 @uref{http://mitpress.mit.edu/sicp/full-text/book/book.html, Structure and
117 Interpretation of Computer Programs}, a popular textbook used to teach
118 programming in Scheme is available in its entirety online.
120 An introduction to Guile/Scheme as used in LilyPond can be found in the
121 @rextend{Scheme tutorial}.
125 MetaFont is used to create the music fonts used by LilyPond. A MetaFont
126 tutorial is available at @uref{http://metafont.tutorial.free.fr/, the
127 METAFONT tutorial page}.
129 @subsection PostScript
131 PostScript is used to generate graphical output. A brief PostScript tutorial
132 is @uref{http://local.wasp.uwa.edu.au/~pbourke/dataformats/postscript/,
133 available online}. The
134 @uref{http://www.adobe.com/devnet/postscript/pdfs/PLRM.pdf, PostScript Lanugage
135 Reference} is available online in PDF format.
139 Python is used for XML2ly and is used for buillding the documentation and the
142 Python documentation is available at @uref{http://www.python.org/doc/,
145 @node Programming without compiling
146 @section Programming without compiling
148 Much of the development work in LilyPond takes place by changing *.ly or
149 *.scm files. These changes can be made without compiling LilyPond. Such
150 changes are described in this section.
153 @subsection Modifying distribution files
155 Much of LilyPond is written in Scheme or LilyPond input files. These
156 files are interpreted when the program is run, rather than being compiled
157 when the program is built, and are present in all LilyPond distributions.
158 You will find .ly files in the ly/ directory and the Scheme files in the
159 scm/ directory. Both Scheme files and .ly files can be modified and
160 saved with any text editor. It's probably wise to make a backup copy of
161 your files before you modify them, although you can reinstall if the
162 files become corrupted.
164 Once you've modified the files, you can test the changes just by running
165 LilyPond on some input file. It's a good idea to create a file that
166 demonstrates the feature you're trying to add. This file will eventually
167 become a regression test and will be part of the LilyPond distribution.
169 @subsection Desired file formatting
171 Files that are part of the LilyPond distribution have Unix-style line
172 endings (LF), rather than DOS (CR+LF) or MacOS 9 and earlier (CR). Make
173 sure you use the necessary tools to ensure that Unix-style line endings are
174 preserved in the patches you create.
176 Tab characters should not be included in files for distribution. All
177 indentation should be done with spaces. Most editors have settings to
178 allow the setting of tab stops and ensuring that no tab characters are
179 included in the file.
181 Scheme files and LilyPond files should be written according to standard
182 style guidelines. Scheme file guidelines can be found at
183 @uref{http://community.schemewiki.org/?scheme-style}. Following these
184 guidelines will make your code easier to read. Both you and others that
185 work on your code will be glad you followed these guidelines.
187 For LilyPond files, you should follow the guidelines for LilyPond snippets
188 in the documentation. You can find these guidelines at
189 @ref{Texinfo introduction and usage policy}.
191 @node Finding functions
192 @section Finding functions
194 When making changes or fixing bugs in LilyPond, one of the initial
195 challenges is finding out where in the code tree the functions to
196 be modified live. With nearly 3000 files in the source tree,
197 trial-and-error searching is generally ineffective. This section
198 describes a process for finding interesting code.
200 @subsection Using the ROADMAP
202 The file ROADMAP is located in the main directory of the lilypond source.
203 ROADMAP lists all of the directories in the LilPond source tree, along
204 with a brief description of the kind of files found in each directory.
205 This can be a very helpful tool for deciding which directories to search
206 when looking for a function.
209 @subsection Using grep to search
211 Having identified a likely subdirectory to search, the grep utility can
212 be used to search for a function name. The format of the grep command is
215 grep -i functionName subdirectory/*
218 This command will search all the contents of the directory subdirectory/
219 and display every line in any of the files that contains
220 functionName. The @code{-i} option makes @command{grep} ignore
221 case -- this can be very useful if you are not yet familiar with
222 our capitalization conventions.
224 The most likely directories to grep for function names are scm/ for
225 scheme files, ly/ for lilypond input (*.ly) files, and lily/ for C++
229 @subsection Using git grep to search
231 If you have used git to obtain the source, you have access to a
232 powerful tool to search for functions. The command:
235 git grep functionName
238 will search through all of the files that are present in the git
239 repository looking for functionName. It also presents the results
240 of the search using @code{less}, so the results are displayed one page
243 @subsection Searching on the git repository at Savannah
245 You can also use the equivalent of git grep on the Savannah server.
250 Go to http://git.sv.gnu.org/gitweb/?p=lilypond.git
253 In the pulldown box that says commit, select grep.
256 Type functionName in the search box, and hit enter/return
260 This will initiate a search of the remote git repository.
266 This section describes style guidelines for LilyPond
273 * Naming conventions::
282 @subsection Languages
284 C++ and Python are preferred. Python code should use PEP 8.
288 @subsection Filenames
290 Definitions of classes that are only accessed via pointers (*) or
291 references (&) shall not be included as include files.
297 ".cc" Implementation files
298 ".icc" Inline definition files
299 ".tcc" non inline Template defs
303 (setq auto-mode-alist
304 (append '(("\\.make$" . makefile-mode)
305 ("\\.cc$" . c++-mode)
306 ("\\.icc$" . c++-mode)
307 ("\\.tcc$" . c++-mode)
308 ("\\.hh$" . c++-mode)
309 ("\\.pod$" . text-mode)
314 The class Class_name is coded in @q{class-name.*}
318 @subsection Indentation
320 Standard GNU coding style is used. In emacs:
323 (add-hook 'c++-mode-hook
324 '(lambda() (c-set-style "gnu")
328 If you like using font-lock, you can also add this to your
332 (setq font-lock-maximum-decoration t)
333 (setq c++-font-lock-keywords-3
335 c++-font-lock-keywords-3
336 '(("\\b\\(a-zA-Z_?+_\\)\\b" 1 font-lock-variable-name-face) ("\\b\\(A-Z?+a-z_?+\\)\\b" 1 font-lock-type-face))
340 Some source files may not currently have proper indenting. If this
341 is the case, it is desirable to fix the improper indenting when the
342 file is modified, with the hope of continually improving the code.
345 @subheading Indenting files with fixcc.py
347 LilyPond provides a python script that will correct the indentation
351 scripts/auxiliar/fixcc.py lily/my-test-file.cc
354 Be sure you replace @code{my-test-file.cc} with the name of the file
357 If you are editing a file that contains an ADD_TRANSLATOR or ADD_INTERFACE
358 macro, the fixcc.py script will move the final parenthesis up one line
359 from where it should be. Please check the end of the file before
360 you run fixcc.py, and then put the final parenthesis and semicolon
361 back on a line by themselves.
364 @subheading Indenting files with emacs in script mode
366 @c email to wl@gnu.org when I get here.
368 @warning{this is pending some confirmation on -devel. July 2009 -gp}
370 Command-line script to format stuff with emacs:
374 emacs $1 -batch --eval '(indent-region (point-min) (point-max) nil)' -f save-buffer
377 (that's all on one line)
379 Save it as a shell script, then run on the file(s) you modified.
382 @subheading Indenting with vim
384 Although emacs indentation is the LilyPond standard, acceptable
385 indentation can usually be accomplished with vim. Some hints for
397 filetype plugin indent on
399 set ignorecase smartcase
402 set statusline=%F%m%r%h%w\ %{&ff}\ %Y\ [ASCII=\%03.3b]\ [HEX=\%02.2B]\ %04l,%04v\ %p%%\ [LEN=%L]
405 " Remove trailing whitespace on write
406 autocmd BufWritePre * :%s/\s\+$//e
409 With this .vimrc, files can be reindented automatically by highlihting
410 the lines to be indented in visual mode (use V to enter visual mode)
413 A scheme.vim file will help improve the indentation. This one
414 was suggested by Patrick McCarty. It should be saved in
415 ~/.vim/after/syntax/scheme.vim.
418 " Additional Guile-specific 'forms'
419 syn keyword schemeSyntax define-public define* define-safe-public
420 syn keyword schemeSyntax use-modules define-module
421 syn keyword schemeSyntax defmacro-public define-macro
422 syn keyword schemeSyntax define-markup-command
423 syn keyword schemeSyntax define-markup-list-command
424 syn keyword schemeSyntax let-keywords* lambda* define*-public
425 syn keyword schemeSyntax defmacro* defmacro*-public
427 " All of the above should influence indenting too
428 set lw+=define-public,define*,define-safe-public,use-modules,define-module
429 set lw+=defmacro-public,define-macro
430 set lw+=define-markup-command,define-markup-list-command
431 set lw+=let-keywords*,lambda*,define*-public,defmacro*,defmacro*-public
433 " These forms should not influence indenting
437 " Try to highlight all ly: procedures
438 syn match schemeFunc "ly:[^) ]\+"
442 @node Naming conventions
443 @subsection Naming Conventions
445 Naming conventions have been established for LilyPond
448 @subheading Classes and Types
450 Classes begin with an uppercase letter, and words
451 in class names are separated with @code{_}:
459 Member variable names end with an underscore:
467 Macro names should be written in uppercase completely,
468 with words separated by @code{_}:
474 @subheading Variables
476 Variable names should be complete words, rather than abbreviations.
477 For example, it is preferred to use @code{thickness} rather than
478 @code{th} or @code{t}.
480 Multi-word variable names in C++ should have the words separated
481 by the underscore character (@q{_}):
484 cxx_multiword_variable
487 Multi-word variable names in Scheme should have the words separated
491 scheme-multiword-variable
495 @subsection Broken code
497 Do not write broken code. This includes hardwired dependencies,
498 hardwired constants, slow algorithms and obvious limitations. If
499 you can not avoid it, mark the place clearly, and add a comment
500 explaining shortcomings of the code.
502 Ideally, the comment marking the shortcoming would include
503 TODO, so that it is marked for future fixing.
505 We reject broken-in-advance on principle.
509 @subsection Code comments
511 Comments may not be needed if descriptive variable names are used
512 in the code and the logic is straightforward. However, if the
513 logic is difficult to follow, and particularly if non-obvious
514 code has been included to resolve a bug, a comment describing
515 the logic and/or the need for the non-obvious code should be included.
517 There are instances where the current code could be commented better.
518 If significant time is required to understand the code as part of
519 preparing a patch, it would be wise to add comments reflecting your
520 understanding to make future work easier.
523 @node Handling errors
524 @subsection Handling errors
526 As a general rule, you should always try to continue computations,
527 even if there is some kind of error. When the program stops, it
528 is often very hard for a user to pinpoint what part of the input
529 causes an error. Finding the culprit is much easier if there is
530 some viewable output.
532 So functions and methods do not return errorcodes, they never
533 crash, but report a programming_error and try to carry on.
535 Error and warning messages need to be localized.
539 @subsection Localization
541 This document provides some guidelines to help programmers write
543 messages. To help translations, user messages must follow
544 uniform conventions. Follow these rules when coding for LilyPond.
545 Hopefully, this can be replaced by general GNU guidelines in the
546 future. Even better would be to have an English (en_BR, en_AM)
547 guide helping programmers writing consistent messages for all GNU
550 Non-preferred messages are marked with `+'. By convention,
551 ungrammatical examples are marked with `*'. However, such ungrammatical
552 examples may still be preferred.
557 Every message to the user should be localized (and thus be marked
558 for localization). This includes warning and error messages.
561 Do not localize/gettextify:
565 `programming_error ()'s
568 `programming_warning ()'s
574 output strings (PostScript, TeX, etc.)
579 Messages to be localized must be encapsulated in `_ (STRING)' or
580 `_f (FORMAT, ...)'. E.g.:
583 warning (_ ("need music in a score"));
584 error (_f ("cannot open file: `%s'", file_name));
587 In some rare cases you may need to call `gettext ()' by hand. This
588 happens when you pre-define (a list of) string constants for later
589 use. In that case, you'll probably also need to mark these string
590 constants for translation, using `_i (STRING)'. The `_i' macro is
591 a no-op, it only serves as a marker for `xgettext'.
594 char const* messages[] = @{
595 _i ("enable debugging output"),
596 _i ("ignore lilypond version"),
603 puts (gettext (messages i));
607 See also `flower/getopt-long.cc' and `lily/main.cc'.
610 Do not use leading or trailing whitespace in messages. If you need
611 whitespace to be printed, prepend or append it to the translated
615 message ("Calculating line breaks..." + " ");
619 Error or warning messages displayed with a file name and line
620 number never start with a capital, eg,
623 foo.ly: 12: not a duration: 3
626 Messages containing a final verb, or a gerund (`-ing'-form) always
627 start with a capital. Other (simpler) messages start with a
633 Not declaring: `foo'.
637 Avoid abbreviations or short forms, use `cannot' and `do not'
638 rather than `can't' or `don't'
639 To avoid having a number of different messages for the same
640 situation, well will use quoting like this `"message: `%s'"' for all
641 strings. Numbers are not quoted:
644 _f ("cannot open file: `%s'", name_str)
645 _f ("cannot find character number: %d", i)
649 Think about translation issues. In a lot of cases, it is better to
650 translate a whole message. The english grammar must not be imposed
651 on the translator. So, instead of
654 stem at + moment.str () + does not fit in beam
660 _f ("stem at %s does not fit in beam", moment.str ())
664 Split up multi-sentence messages, whenever possible. Instead of
667 warning (_f ("out of tune! Can't find: `%s'", "Key_engraver"));
668 warning (_f ("cannot find font `%s', loading default", font_name));
674 warning (_ ("out of tune:"));
675 warning (_f ("cannot find: `%s', "Key_engraver"));
676 warning (_f ("cannot find font: `%s', font_name));
677 warning (_f ("Loading default font"));
681 If you must have multiple-sentence messages, use full punctuation.
682 Use two spaces after end of sentence punctuation. No punctuation
683 (esp. period) is used at the end of simple messages.
686 _f ("Non-matching braces in text `%s', adding braces", text)
687 _ ("Debug output disabled. Compiled with NPRINT.")
688 _f ("Huh? Not a Request: `%s'. Ignoring.", request)
692 Do not modularize too much; words frequently cannot be translated
693 without context. It is probably safe to treat most occurences of
694 words like stem, beam, crescendo as separately translatable words.
697 When translating, it is preferable to put interesting information
698 at the end of the message, rather than embedded in the middle.
699 This especially applies to frequently used messages, even if this
700 would mean sacrificing a bit of eloquency. This holds for original
701 messages too, of course.
704 en: cannot open: `foo.ly'
705 + nl: kan `foo.ly' niet openen (1)
706 kan niet openen: `foo.ly'* (2)
707 niet te openen: `foo.ly'* (3)
711 The first nl message, although grammatically and stylistically
712 correct, is not friendly for parsing by humans (even if they speak
713 dutch). I guess we would prefer something like (2) or (3).
716 Do not run make po/po-update with GNU gettext < 0.10.35
722 @node Debugging LilyPond
723 @section Debugging LilyPond
725 The most commonly used tool for debugging LilyPond is the GNU
726 debugger gdb. The gdb tool is used for investigating and debugging
727 core Lilypond code written in C++. Another tool is available for
728 debugging Scheme code using the Guile debugger. This section
729 describes how to use both gdb and the Guile Debugger.
732 * Debugging overview::
733 * Debugging C++ code::
734 * Debugging Scheme code::
737 @node Debugging overview
738 @subsection Debugging overview
740 Using a debugger simplifies troubleshooting in at least two ways.
742 First, breakpoints can be set to pause execution at any desired point.
743 Then, when execution has paused, debugger commands can be issued to
744 explore the values of various variables or to execute functions.
746 Second, the debugger can display a stack trace, which shows the
747 sequence in which functions have been called and the arguments
748 passed to the called functions.
750 @node Debugging C++ code
751 @subsection Debugging C++ code
753 The GNU debugger, gdb, is the principal tool for debugging C++ code.
755 @subheading Compiling LilyPond for use with gdb
757 In order to use gdb with LilyPond, it is necessary to compile
758 LilyPond with debugging information. This is accomplished by running
759 the following commands in the main LilyPond source directory.
762 ./configure --disable-optimising
766 This will create a version of LilyPond containing debugging
767 information that will allow the debugger to tie the source code
768 to the compiled code.
770 You should not do @var{make install} if you want to use a debugger
771 with LilyPond. The @var{make install} command will strip debugging
772 information from the LilyPond binary.
774 @subheading Typical gdb usage
776 Once you have compiled the Lilypond image with the necessary
777 debugging information it will have been written to a location in a
778 subfolder of your current working directory:
784 This is important as you will need to let gdb know where to find the
785 image containing the symbol tables. You can invoke gdb from the
786 command line usinga the following:
792 This loads the LilyPond symbol tables into gdb. Then, to run
793 LilyPond on @code{test.ly} under the debugger, enter the following:
802 As an alternative to running gdb at the command line you may try
803 a graphical interface to gdb such as ddd:
809 You can also use sets of standard gdb commands stored in a .gdbinit
810 file (see next section).
812 @subheading Typical .gdbinit files
814 The behavior of gdb can be readily customized through the use of a
815 @var{.gdbinit} file. A @var{.gdbinit} file is a file named
816 @var{.gdbinit} (notice the @qq{.} at the beginning of the file name)
817 that is placed in a user's home directory.
819 The @var{.gdbinit} file below is from Han-Wen. It sets breakpoints
820 for all errors and defines functions for displaying scheme objects
821 (ps), grobs (pgrob), and parsed music expressions (pmusic).
824 file lily/out/lilypond
826 b Grob::programming_error
829 print ly_display_scm($arg0)
832 print ly_display_scm($arg0->self_scm_)
833 print ly_display_scm($arg0->mutable_property_alist_)
834 print ly_display_scm($arg0->immutable_property_alist_)
835 print ly_display_scm($arg0->object_alist_)
838 print ly_display_scm($arg0->self_scm_)
839 print ly_display_scm($arg0->mutable_property_alist_)
840 print ly_display_scm($arg0->immutable_property_alist_)
844 @node Debugging Scheme code
845 @subsection Debugging Scheme code
847 Scheme code can be developed using the Guile command line
848 interpreter @code{top-repl}. You can either investigate
849 interactively using just Guile or you can use the debugging
850 tools available within Guile.
852 @subheading Using Guile interactively with LilyPond
854 In order to experiment with Scheme programming in the LilyPond
855 environment, it is necessary to have a Guile interpreter that
856 has all the LilyPond modules loaded. This requires the following
859 First, define a Scheme symbol for the active module in the .ly file:
862 #(module-define! (resolve-module '(guile-user))
863 'lilypond-module (current-module))
866 Now place a Scheme function in the .ly file that gives an
867 interactive Guile prompt:
873 When the .ly file is compiled, this causes the compilation to be
874 interrupted and an interactive guile prompt to appear. Once the
875 guile prompt appears, the LilyPond active module must be set as the
876 current guile module:
879 guile> (set-current-module lilypond-module)
882 You can demonstrate these commands are operating properly by typing the name
883 of a LilyPond public scheme function to check it has been defined:
886 guile> fret-diagram-verbose-markup
887 #<procedure fret-diagram-verbose-markup (layout props marking-list)>
890 If the LilyPond module has not been correctly loaded, an error
891 message will be generated:
894 guile> fret-diagram-verbose-markup
895 ERROR: Unbound variable: fret-diagram-verbose-markup
896 ABORT: (unbound-variable)
899 Once the module is properly loaded, any valid LilyPond Scheme
900 expression can be entered at the interactive prompt.
902 After the investigation is complete, the interactive guile
903 interpreter can be exited:
909 The compilation of the .ly file will then continue.
911 @subheading Using the Guile debugger
913 To set breakpoints and/or enable tracing in Scheme functions, put
916 \include "guile-debugger.ly"
919 in your input file after any scheme procedures you have defined in
920 that file. This will invoke the Guile command-line after having set
921 up the environment for the debug command-line. When your input file
922 is processed, a guile prompt will be displayed. You may now enter
923 commands to set up breakpoints and enable tracing by the Guile debugger.
925 @subheading Using breakpoints
927 At the guile prompt, you can set breakpoints with
928 the @code{set-break!} procedure:
931 guile> (set-break! my-scheme-procedure)
934 Once you have set the desired breakpoints, you exit the guile repl frame
941 Then, when one of the scheme routines for which you have set
942 breakpoints is entered, guile will interrupt execution in a debug
943 frame. At this point you will have access to Guile debugging
944 commands. For a listing of these commands, type:
950 Alternatively you may code the breakpoints in your Lilypond source
951 file using a command such as:
954 #(set-break! my-scheme-procedure)
957 immediately after the @code{\include} statement. In this case the
958 breakpoint will be set straight after you enter the @code{(quit)}
959 command at the guile prompt.
961 Embedding breakpoint commands like this is particularly useful if
962 you want to look at how the Scheme procedures in the @var{.scm}
963 files supplied with LilyPond work. To do this, edit the file in
964 the relevant directory to add this line near the top:
967 (use-modules (scm guile-debugger))
970 Now you can set a breakpoint after the procedure you are interested
971 in has been declared. For example, if you are working on routines
972 called by @var{print-book-with} in @var{lily-library.scm}:
975 (define (print-book-with parser book process-procedure)
976 (let* ((paper (ly:parser-lookup parser '$defaultpaper))
977 (layout (ly:parser-lookup parser '$defaultlayout))
978 (outfile-name (get-outfile-name parser)))
979 (process-procedure book paper layout outfile-name)))
981 (define-public (print-book-with-defaults parser book)
982 (print-book-with parser book ly:book-process))
984 (define-public (print-book-with-defaults-as-systems parser book)
985 (print-book-with parser book ly:book-process-to-systems))
989 At this point in the code you could add this to set a breakpoint at
993 (set-break! print-book-with)
996 @subheading Tracing procedure calls and evaluator steps
998 Two forms of trace are available:
1001 (set-trace-call! my-scheme-procedure)
1007 (set-trace-subtree! my-scheme-procedure)
1010 @code{set-trace-call!} causes Scheme to log a line to the standard
1011 output to show when the procedure is called and when it exits.
1013 @code{set-trace-subtree!} traces every step the Scheme evaluator
1014 performs in evaluating the procedure.
1016 @node Tracing object relationships
1017 @section Tracing object relationships
1019 Understanding the LilyPond source often boils down to figuring out what
1020 is happening to the Grobs. Where (and why) are they being created,
1021 modified and destroyed? Tracing Lily through a debugger in order to
1022 identify these relationships can be time-consuming and tedious.
1024 In order to simplify this process, a facility has been added to
1025 display the grobs that are created and the properties that are set
1026 and modified. Although it can be complex to get set up, once set up
1027 it easily provides detailed information about the life of grobs
1028 in the form of a network graph.
1030 Each of the steps necessary to use the graphviz utility
1035 @item Installing graphviz
1037 In order to create the graph of the object relationships, it is
1038 first necessary to install Graphviz. graphviz is available for a
1039 number of different platforms:
1042 @uref{http://www.graphviz.org/Download..php}
1045 @item Modifying config.make
1047 In order for the Graphviz tool to work, config.make must be modified.
1048 It is probably a good idea to first save a copy of config.make under
1049 a different name. Then, edit config.make by removing every occurence
1052 @item Rebuilding LilyPond
1054 The executable code of LilyPond must be rebuilt from scratch:
1057 make -C lily clean && make -C lily
1060 @item Create a graphviz-compatible .ly file
1062 In order to use the graphviz utility, the .ly file must include
1063 @file{ly/graphviz-init.ly}, and should then specify the
1064 grobs and symbols that should be tracked. An example of this
1065 is found in @file{input/regression/graphviz.ly}.
1067 @item Run lilypond with output sent to a log file
1069 The Graphviz data is sent to stderr by lilypond, so it is
1070 necessary to redirect stderr to a logfile:
1073 lilypond graphviz.ly 2> graphviz.log
1076 @item Edit the logfile
1078 The logfile has standard lilypond output, as well as the Graphviz
1079 output data. Delete everything from the beginning of the file
1080 up to but not including the first occurence of @code{digraph}.
1082 @item Process the logfile with @code{dot}
1084 The directed graph is created from the log file with the program
1088 dot -Tpdf graphviz.log > graphviz.pdf
1093 The pdf file can then be viewed with any pdf viewer.
1095 When compiled without @code{-DNDEBUG}, lilypond may run slower
1096 than normal. The original configuration can be restored by either
1097 renaming the saved copy of @code{config.make} or rerunning
1098 @code{configure}. Then rebuild lilypond with
1101 make -C lily clean && make -C lily
1105 @node Adding or modifying features
1106 @section Adding or modifying features
1108 When a new feature is to be added to LilyPond, it is necessary to
1109 ensure that the feature is properly integrated to maintain
1110 its long-term support. This section describes the steps necessary
1111 for feature addition and modification.
1116 * Write regression tests::
1117 * Write convert-ly rule::
1118 * Automaticaly update auxiliary information::
1119 * Manually update auxiliary information::
1120 * Edit changes.tely::
1121 * Verify successful build::
1122 * Verify regression tests::
1123 * Post patch for comments::
1125 * Closing the issues::
1128 @node Write the code
1129 @subsection Write the code
1131 You should probably create a new git branch for writing the code, as that
1132 will separate it from the master branch and allow you to continue
1133 to work on small projects related to master.
1135 Please be sure to follow the rules for programming style discussed
1136 earlier in this chapter.
1139 @node Write regression tests
1140 @subsection Write regression tests
1142 In order to demonstrate that the code works properly, you will
1143 need to write one or more regression tests. These tests are
1144 typically .ly files that are found in input/regression.
1146 Regression tests should be as brief as possible to demonstrate the
1147 functionality of the code.
1149 Regression tests should generally cover one issue per test. Several
1150 short, single-issue regression tests are preferred to a single, long,
1151 multiple-issue regression test.
1153 Use existing regression tests as templates to demonstrate the type of
1154 header information that should be included in a regression test.
1157 @node Write convert-ly rule
1158 @subsection Write convert-ly rule
1160 If the modification changes the input syntax, a convert-ly rule
1161 should be written to automatically update input files from older
1164 convert-ly rules are found in python/convertrules.py
1166 If possible, the convert-ly rule should allow automatic updating
1167 of the file. In some cases, this will not be possible, so the
1168 rule will simply point out to the user that the feature needs
1172 @node Automaticaly update auxiliary information
1173 @subsection Automatically update auxiliary information
1175 convert-ly should be used to update the documentation, the snippets,
1176 and the regression tests. This not only makes the necessary syntax
1177 changes, it also tests the convert-ly rules.
1179 The automatic updating is a three step process. First, be sure you
1180 are in the top-level source directory. Then, for the
1184 find Documentation/ -name '*.itely' | xargs convert-ly -e --from @qq{@var{X.Y.Z}}
1188 where @var{X.Y.Z} is the version number of the last released development
1191 Next, for the snippets, do:
1194 find Documentation/snippets/ -name '*.ly' | xargs convert-ly -e --from @qq{@var{X.Y.Z}}
1197 Finally, for the regression tests, do:
1200 find input/regression/ -name '*.ly' | xargs convert-ly -e --from @qq{@var{X.Y.Z}}
1205 @node Manually update auxiliary information
1206 @subsection Manually update auxiliary information
1208 Where the convert-ly rule is not able to automatically update the inline
1209 lilypond code in the documentation (i.e. if a NOT_SMART rule is used), the
1210 documentation must be manually updated. The inline snippets that require
1211 changing must be changed in the English version of the docs and all
1212 translated versions. If the inline code is not changed in the
1213 translated documentation, the old snippets will show up in the
1214 English version of the documentation.
1216 Where the convert-ly rule is not able to automatically update snippets
1217 in Documentation/snippets/, those snippets must be manually updated.
1218 Those snippets should be copied to Documentation/snippets/new. The
1219 comments at the top of the snippet describing its automatic generation
1220 should be removed. All translated texidoc strings should be removed.
1221 The comment @qq{% begin verbatim} should be removed. The syntax of
1222 the snippet should then be manually edited.
1224 Where snippets in Documentation/snippets are made obsolete, the snippet
1225 should be copied to Documentation/snippets/new. The comments and
1226 texidoc strings should be removed as described above. Then the body
1227 of the snippet should be changed to:
1231 This snippet is deprecated as of version X.Y.Z and
1232 will be removed from the documentation.
1237 where X.Y.Z is the version number for which the convert-ly rule was
1240 Update the snippet files by running:
1243 scripts/auxiliar/makelsr.py
1246 Where the convert-ly rule is not able to automatically update regression
1247 tests, the regression tests in input/regression should be manually
1250 Although it is not required, it is helpful if the developer
1251 can write relevant material for inclusion in the Notation
1252 Reference. If the developer does not feel qualified to write
1253 the documentation, a documentation editor will be able to
1254 write it from the regression tests. The text that is added to
1255 or removed from the documentation should be changed only in
1256 the English version.
1259 @node Edit changes.tely
1260 @subsection Edit changes.tely
1262 An entry should be added to Documentation/changes.tely to describe
1263 the feature changes to be implemented. This is especially important
1264 for changes that change input file syntax.
1266 Hints for changes.tely entries are given at the top of the file.
1268 New entries in changes.tely go at the top of the file.
1270 The changes.tely entry should be written to show how the new change
1271 improves LilyPond, if possible.
1274 @node Verify successful build
1275 @subsection Verify successful build
1277 When the changes have been made, successful completion must be
1285 When these commands complete without error, the patch is
1286 considered to function successfully.
1288 Developers on Windows who are unable to build LilyPond should
1289 get help from a Linux or OSX developer to do the make tests.
1292 @node Verify regression tests
1293 @subsection Verify regression tests
1295 In order to avoid breaking LilyPond, it is important to verify that
1296 the regression tests succeed, and that no unwanted changes are
1297 introduced into the output. This process is described in
1298 @ref{Identifying code regressions}.
1300 @subheading Typical developer's edit/compile/test cycle
1302 TODO: is @code{[-j@var{X} CPU_COUNT=@var{X}]} useful for
1303 @code{test-baseline}, @code{check}, @code{clean},
1304 @code{test-redo}? Neil Puttock says it is useful for
1305 everything but @code{clean}, which is disk-limited.
1306 Need to check formally.
1315 make [-j@var{X} CPU_COUNT=@var{X}] check
1319 Edit/compile/test cycle:
1322 @emph{## edit source files, then...}
1324 make clean @emph{## only if needed (see below)}
1325 make [-j@var{X}] @emph{## only if needed (see below)}
1326 make test-redo @emph{## redo files differing from baseline}
1327 make [-j@var{X} CPU_COUNT=@var{X}] check @emph{## CPU_COUNT here?}
1338 If you modify any source files that have to be compiled (such as
1339 @file{.cc} or @file{.hh} files in @file{flower/} or @file{lily/}),
1340 then you must run @command{make} before @command{make test-redo},
1341 so @command{make} can compile the modified files and relink all
1342 the object files. If you only modify files which are interpreted,
1343 like those in the @file{scm/} and @file{ly/} directories, then
1344 @command{make} is not needed before @command{make test-redo}.
1346 TODO: Fix the following paragraph. You can do @command{rm mf/out/*}
1347 instead of make clean, and you can probably do
1348 @command{make -C mf/ clean} as well, but I haven't checked it -- cds
1350 Also, if you modify any font definitions in the @file{mf/}
1351 directory then you must run @command{make clean} and
1352 @command{make} before running @command{make test-redo}. This will
1353 recompile everything, whether modified or not, and takes a lot
1356 Running @command{make@tie{}check} will leave an HTML page
1357 @file{out/test-results/index.html}. This page shows all the
1358 important differences that your change introduced, whether in the
1359 layout, MIDI, performance or error reporting.
1364 @node Post patch for comments
1365 @subsection Post patch for comments
1367 For any change other than a minor change, a patch set should be
1368 posted on @uref{http://codereview.appspot.com/, Rietveld} for comment.
1369 This requires the use of an external package, git-cl, and an email
1372 git-cl is installed by:
1375 git clone git://neugierig.org/git-cl.git
1378 Then, add the git-cl directory to your PATH, or create a
1379 symbolic link to the git-cl and upload.py in one of your
1380 PATH directories (like usr/bin). git-cl is then
1381 configured by entering the command
1388 in the LilyPond git directory and answering the questions that
1389 are asked. If you do not understand the question answer with just
1392 The patch set is posted to Rietveld as follows. Ensure your changes
1393 are committed in a separate branch, which should differ from the
1394 reference branch to be used by just the changes to be uploaded.
1395 If the reference branch is to be origin/master, ensure this is
1396 up-to-date. If necessary, use git rebase to rebase the branch
1397 containing the changes to the head of origin/master. Finally,
1398 check out branch with the changes and enter the command:
1401 git cl upload <reference SHA1 ID>
1405 where <reference SHA1 ID> is the SHA1 ID of the commit to be used
1406 as a reference source for the patch. Generally, this will be the
1407 SHA1 ID of origin/master, and in that case the command
1410 git cl upload origin/master
1416 After prompting for your Google email address and password, the
1417 patch set will be posted to Rietveld.
1419 You should then announce the patch by sending
1420 an email to lilypond-devel, with a subject line
1421 starting with PATCH:, asking for comments on the patch.
1423 As revisions are made in response to comments, successive patch sets
1424 for the same issue can be uploaded by reissuing the git-cl command
1425 with the modified branch checked out.
1427 Sometimes in response to comments on revisions, the best way to
1428 work may require creation of a new branch in git. In order to
1429 associate the new branch with an existing Reitveld issue,
1430 the following command can be used:
1433 git cl issue issue-number
1437 where @code{issue-number} is the number of the existing Rietveld
1443 @subsection Push patch
1445 Once all the comments have been addressed, the patch can be pushed.
1447 If the author has push privileges, the author will push the patch.
1448 Otherwise, a developer with push privileges will push the patch.
1451 @node Closing the issues
1452 @subsection Closing the issues
1454 Once the patch has been pushed, all the relevant issues should be
1457 On Rietveld, the author should log in an close the issue either by
1458 using the @q{Edit Issue} link, or by clicking the circled x icon
1459 to the left of the issue name.
1461 If the changes were in response to a feature request on the Google
1462 issue tracker for LilyPond, the author should change the status to
1463 Fixed and a tag @q{fixed_x_y_z} should be added, where the patch was
1464 fixed in version x.y.z. If
1465 the author does not have privileges to change the status, an email
1466 should be sent to bug-lilypond requesting the BugMeister to change
1470 @node Iterator tutorial
1471 @section Iterator tutorial
1473 TODO -- this is a placeholder for a tutorial on iterators
1475 Iterators are routines written in C++ that process music expressions
1476 and sent the music events to the appropriate engravers and/or
1480 @node Engraver tutorial
1481 @section Engraver tutorial
1483 Engravers are C++ classes that catch music events and
1484 create the appropriate grobs for display on the page. Though the
1485 majority of engravers are responsible for the creation of a single grob,
1486 in some cases (e.g. @code{New_fingering_engraver}), several different grobs
1489 Engravers listen for events and acknowledge grobs. Events are passed to
1490 the engraver in time-step order during the iteration phase. Grobs are
1491 made available to the engraver when they are created by other engravers
1492 during the iteration phase.
1496 * Useful methods for information processing::
1497 * Translation process::
1498 * Preventing garbage collection for SCM member variables::
1499 * Listening to music events::
1500 * Acknowledging grobs::
1501 * Engraver declaration/documentation::
1504 @node Useful methods for information processing
1505 @subsection Useful methods for information processing
1507 An engraver inherits the following public methods from the Translator
1508 base class, which can be used to process listened events and acknowledged
1512 @item @code{virtual void initialize ()}
1513 @item @code{void start_translation_timestep ()}
1514 @item @code{void process_music ()}
1515 @item @code{void process_acknowledged ()}
1516 @item @code{void stop_translation_timestep ()}
1517 @item @code{virtual void finalize ()}
1520 These methods are listed in order of translation time, with
1521 @code{initialize ()} and @code{finalize ()} bookending the whole
1522 process. @code{initialize ()} can be used for one-time initialization
1523 of context properties before translation starts, whereas
1524 @code{finalize ()} is often used to tie up loose ends at the end of
1525 translation: for example, an unterminated spanner might be completed
1526 automatically or reported with a warning message.
1529 @node Translation process
1530 @subsection Translation process
1532 At each timestep in the music, translation proceeds by calling the
1533 following methods in turn:
1535 @code{start_translation_timestep ()} is called before any user
1536 information enters the translators, i.e., no property operations
1537 (\set, \override, etc.) or events have been processed yet.
1539 @code{process_music ()} and @code{process_acknowledged ()} are called
1540 after all events in the current time step have been heard, or all
1541 grobs in the current time step have been acknowledged. The latter
1542 tends to be used exclusively with engravers which only acknowledge
1543 grobs, whereas the former is the default method for main processing
1546 @code{stop_translation_timestep ()} is called after all user
1547 information has been processed prior to beginning the translation for
1551 @node Preventing garbage collection for SCM member variables
1552 @subsection Preventing garbage collection for SCM member variables
1554 In certain cases, an engraver might need to ensure private Scheme
1555 variables (with type SCM) do not get swept away by Guile's garbage
1556 collector: for example, a cache of the previous key signature which
1557 must persist between timesteps. The method
1558 @code{virtual derived_mark () const} can be used in such cases:
1561 Engraver_name::derived_mark ()
1563 scm_gc_mark (private_scm_member_)
1568 @node Listening to music events
1569 @subsection Listening to music events
1571 External interfaces to the engraver are implemented by protected
1572 macros including one or more of the following:
1575 @item @code{DECLARE_TRANSLATOR_LISTENER (event_name)}
1576 @item @code{IMPLEMENT_TRANSLATOR_LISTENER (Engraver_name, event_name)}
1580 where @var{event_name} is the type of event required to provide the
1581 input the engraver needs and @var{Engraver_name} is the name of the
1584 Following declaration of a listener, the method is implemented as follows:
1587 IMPLEMENT_TRANSLATOR_LISTENER (Engraver_name, event_name)
1589 Engraver_name::listen_event_name (Stream event *event)
1591 ...body of listener method...
1596 @node Acknowledging grobs
1597 @subsection Acknowledging grobs
1599 Some engravers also need information from grobs as they are created
1600 and as they terminate. The mechanism and methods to obtain this
1601 information are set up by the macros:
1604 @item @code{DECLARE_ACKNOWLEDGER (grob_interface)}
1605 @item @code{DECLARE_END_ACKNOWLEDGER (grob_interface)}
1608 where @var{grob_interface} is an interface supported by the
1609 grob(s) which should be acknowledged. For example, the following
1610 code would declare acknowledgers for a @code{NoteHead} grob (via the
1611 @code{note-head-interface}) and any grobs which support the
1612 @code{side-position-interface}:
1615 @code{DECLARE_ACKNOWLEDGER (note_head)}
1616 @code{DECLARE_ACKNOWLEDGER (side_position)}
1619 The @code{DECLARE_END_ACKNOWLEDGER ()} macro sets up a spanner-specific
1620 acknowledger which will be called whenever a spanner ends.
1622 Following declaration of an acknowledger, the method is coded as follows:
1626 Engraver_name::acknowledge_interface_name (Grob_info info)
1628 ...body of acknowledger method...
1633 @node Engraver declaration/documentation
1634 @subsection Engraver declaration/documentation
1636 An engraver must have a public macro
1639 @item @code{TRANSLATOR_DECLARATIONS (Engraver_name)}
1643 where @code{Engraver_name} is the name of the engraver. This
1644 defines the common variables and methods used by every engraver.
1646 At the end of the engraver file, one or both of the following
1647 macros are generally called to document the engraver in the
1648 Internals Reference:
1651 @item @code{ADD_ACKNOWLEDGER (Engraver_name, grob_interface)}
1652 @item @code{ADD_TRANSLATOR (Engraver_name, Engraver_doc,
1653 Engraver_creates, Engraver_reads, Engraver_writes)}
1657 where @code{Engraver_name} is the name of the engraver, @code{grob_interface}
1658 is the name of the interface that will be acknowledged,
1659 @code{Engraver_doc} is a docstring for the engraver,
1660 @code{Engraver_creates} is the set of grobs created by the engraver,
1661 @code{Engraver_reads} is the set of properties read by the engraver,
1662 and @code{Engraver_writes} is the set of properties written by
1665 The @code{ADD_ACKNOWLEDGER} and @code{ADD_TRANSLATOR} macros use a
1666 non-standard indentation system. Each interface, grob, read property,
1667 and write property is on its own line, and the closing parenthesis
1668 and semicolon for the macro all occupy a separate line beneath the final
1669 interface or write property. See existing engraver files for more
1673 @node Callback tutorial
1674 @section Callback tutorial
1676 TODO -- This is a placeholder for a tutorial on callback functions.
1678 @node LilyPond scoping
1679 @section LilyPond scoping
1681 The Lilypond language has a concept of scoping, ie you can do
1687 (display (+ foo 2)))
1690 @noindent with @code{\paper}, @code{\midi} and @code{\header} being
1691 nested scope inside the @file{.ly} file-level scope. @w{@code{foo = 1}}
1692 is translated in to a scheme variable definition.
1694 This implemented using modules, with each scope being an anonymous
1695 module that imports its enclosing scope's module.
1697 Lilypond's core, loaded from @file{.scm} files, is usually placed in the
1698 @code{lily} module, outside the @file{.ly} level. In the case of
1705 we want to reuse the built-in definitions, without changes effected in
1706 user-level @file{a.ly} leaking into the processing of @file{b.ly}.
1708 The user-accessible definition commands have to take care to avoid
1709 memory leaks that could occur when running multiple files. All
1710 information belonging to user-defined commands and markups is stored in
1711 a manner that allows it to be garbage-collected when the module is
1712 dispersed, either by being stored module-locally, or in weak hash
1715 @node LilyPond miscellany
1716 @section LilyPond miscellany
1718 This is a place to dump information that may be of use to developers
1719 but doesn't yet have a proper home. Ideally, the length of this section
1720 would become zero as items are moved to other homes.
1724 * Spacing algorithms::
1725 * Info from Han-Wen email::
1726 * Music functions and GUILE debugging::
1729 @node Spacing algorithms
1730 @subsection Spacing algorithms
1732 Here is information from an email exchange about spacing algorithms.
1734 On Thu, 2010-02-04 at 15:33 -0500, Boris Shingarov wrote:
1735 I am experimenting with some modifications to the line breaking code,
1736 and I am stuck trying to understand how some of it works. So far my
1737 understanding is that Simple_spacer operates on a vector of Grobs, and
1738 it is a well-known Constrained-QP problem (rods = constraints, springs
1739 = quadratic function to minimize). What I don't understand is, if the
1740 spacer operates at the level of Grobs, which are built at an earlier
1741 stage in the pipeline, how are the changes necessitated by differences
1742 in line breaking, taken into account? in other words, if I take the
1743 last measure of a line and place it on the next line, it is not just a
1744 matter of literally moving that graphic to where the start of the next
1745 line is, but I also need to draw a clef, key signature, and possibly
1746 other fundamental things -- but at that stage in the rendering
1747 pipeline, is it not too late??
1749 Joe Neeman answered:
1751 We create lots of extra grobs (eg. a BarNumber at every bar line) but
1752 most of them are not drawn. See the break-visibility property in
1756 @node Info from Han-Wen email
1757 @subsection Info from Han-Wen email
1759 In 2004, Douglas Linhardt decided to try starting a document that would
1760 explain LilyPond architecture and design principles. The material below
1761 is extracted from that email, which can be found at
1762 @uref{http://thread.gmane.org/gmane.comp.gnu.lilypond.devel/2992}.
1763 The headings reflect questions from Doug or comments from Han-Wen;
1764 the body text are Han-Wen's answers.
1766 @subheading Figuring out how things work.
1768 I must admit that when I want to know how a program works, I use grep
1769 and emacs and dive into the source code. The comments and the code
1770 itself are usually more revealing than technical documents.
1772 @subheading What's a grob, and how is one used?
1774 Graphical object - they are created from within engravers, either as
1775 Spanners (derived class) -slurs, beams- or Items (also a derived
1776 class) -notes, clefs, etc.
1778 There are two other derived classes System (derived from Spanner,
1779 contaning a "line of music") and Paper_column (derived from Item, it
1780 contains all items that happen at the same moment). They are separate
1781 classes because they play a special role in the linebreaking process.
1783 @subheading What's a smob, and how is one used?
1785 A C(++) object that is encapsulated so it can be used as a Scheme
1786 object. See GUILE info, "19.3 Defining New Types (Smobs)"
1788 @@subheading When is each C++ class constructed and used
1795 In the parser.yy see the macro calls MAKE_MUSIC_BY_NAME().
1800 Constructed during "interpreting" phase.
1805 Executive branch of Contexts, plugins that create grobs, usually one
1806 engraver per grob type. Created together with context.
1816 These are not C++ classes per se. The idea of a Grob interface hasn't
1817 crystallized well. ATM, an interface is a symbol, with a bunch of grob
1818 properties. They are not objects that are created or destroyed.
1823 Objects that walk through different music classes, and deliver events
1824 in a synchronized way, so that notes that play together are processed
1825 at the same moment and (as a result) end up on the same horizontal position.
1827 Created during interpreting phase.
1829 BTW, the entry point for interpreting is ly:run-translator
1830 (ly_run_translator on the C++ side)
1834 @subheading Can you get to Context properties from a Music object?
1836 You can create music object with a Scheme function that reads context
1837 properties (the \applycontext syntax). However, that function is
1838 executed during Interpreting, so you can not really get Context
1839 properties from Music objects, since music objects are not directly
1840 connected to Contexts. That connection is made by the Music_iterators
1842 @subheading Can you get to Music properties from a Context object?
1844 Yes, if you are given the music object within a Context
1845 object. Normally, the music objects enter Contexts in synchronized
1846 fashion, and the synchronization is done by Music_iterators.
1848 @subheading What is the relationship between C++ classes and Scheme objects?
1850 Smobs are C++ objects in Scheme. Scheme objects (lists, functions) are
1851 manipulated from C++ as well using the GUILE C function interface
1854 @subheading How do Scheme procedures get called from C++ functions?
1856 scm_call_*, where * is an integer from 0 to 4.
1857 Also scm_c_eval_string (), scm_eval ()
1859 @subheading How do C++ functions get called from Scheme procedures?
1861 Export a C++ function to Scheme with LY_DEFINE.
1863 @subheading What is the flow of control in the program?
1865 Good question. Things used to be clear-cut, but we have Scheme
1866 and SMOBs now, which means that interactions do not follow a very
1867 rigid format anymore. See below for an overview, though.
1869 @subheading Does the parser make Scheme procedure calls or C++ function calls?
1871 Both. And the Scheme calls can call C++ and vice versa. It's nested,
1872 with the SCM datatype as lubrication between the interactions
1874 (I think the word "lubrication" describes the process better than the
1875 traditional word "glue")
1877 @subheading How do the front-end and back-end get started?
1879 Front-end: a file is parsed, the rest follows from that. Specifically,
1881 Parsing leads to a Music + Music_output_def object (see parser.yy,
1882 definition of toplevel_expression )
1884 A Music + Music_output_def object leads to a Global_context object (see
1885 ly_run_translator ())
1887 During interpreting, Global_context + Music leads to a bunch of
1888 Contexts. (see Global_translator::run_iterator_on_me () )
1890 After interpreting, Global_context contains a Score_context (which
1891 contains staves, lyrics etc.) as a child. Score_context::get_output ()
1892 spews a Music_output object (either a Paper_score object for notation
1893 or Performance object for MIDI).
1895 The Music_output object is the entry point for the backend. (see
1896 ly_render_output () )
1898 The main steps of the backend itself are in
1903 paper-score.cc , Paper_score::process_
1906 system.cc , System::get_lines()
1909 The step, where things go from grobs to output, is in
1910 System::get_line(): each grob delivers a Stencil (a Device
1911 independent output description), which is interpreted by our
1912 outputting backends (scm/output-tex.scm and scm/output-ps.scm)
1913 to produce TeX and PS.
1917 Interactions between grobs and putting things into .tex and .ps files
1918 have gotten a little more complex lately. Jan has implemented
1919 page-breaking, so now the backend also involves Paper_book,
1920 Paper_lines and other things. This area is still heavily in flux, and
1921 perhaps not something you should want to look at.
1923 @subheading How do the front-end and back-end communicate?
1925 There is no communication from backend to front-end. From front-end to
1926 backend is simply the program flow: music + definitions gives
1927 contexts, contexts yield output, after processing, output is written
1930 @subheading Where is the functionality associated with KEYWORDs?
1932 See my-lily-lexer.cc (keywords, there aren't that many) and ly/*.ly
1933 (most of the other backslashed \words are identifiers)
1935 @subheading What Contexts/Properties/Music/etc. are available when they are processed?
1937 What do you mean exactly with this question?
1939 See ly/engraver-init.ly for contexts, see scm/define-*.scm for other
1942 @subheading How do you decide if something is a Music, Context, or Grob property?
1943 Why is part-combine-status a Music property when it seems (IMO)
1944 to be related to the Staff context?
1946 The Music_iterators and Context communicate through two channels
1948 Music_iterators can set and read context properties, idem for
1949 Engravers and Contexts
1951 Music_iterators can send "synthetic" music events (which aren't in
1952 the input) to a context. These are caught by Engravers. This is
1953 mostly a one way communication channel.
1955 part-combine-status is part of such a synthetic event, used by
1956 Part_combine_iterator to communicate with Part_combine_engraver.
1959 @subheading Deciding between context and music properties
1961 I'm adding a property to affect how \autochange works. It seems to
1962 me that it should be a context property, but the Scheme autochange
1963 procedure has a Music argument. Does this mean I should use
1966 \autochange is one of these extra strange beasts: it requires
1967 look-ahead to decide when to change staves. This is achieved by
1968 running the interpreting step twice (see scm/part-combiner.scm , at
1969 the bottom), and storing the result of the first step (where to switch
1970 staves) in a Music property. Since you want to influence that
1971 where-to-switch list, your must affect the code in
1972 make-autochange-music (scm/part-combiner.scm). That code is called
1973 directly from the parser and there are no official "parsing
1974 properties" yet, so there is no generic way to tune \autochange. We
1975 would have to invent something new for this, or add a separate
1979 \autochange #around-central-C ..music..
1983 where around-central-C is some function that is called from
1984 make-autochange-music.
1986 @subheading More on context and music properties
1988 From Neil Puttock, in response to a question about transposition:
1990 Context properties (using \set & \unset) are tied to engravers: they
1991 provide information relevant to the generation of graphical objects.
1993 Since transposition occurs at the music interpretation stage, it has
1994 no direct connection with engravers: the pitch of a note is fixed
1995 before a notehead is created. Consider the following minimal snippet:
2001 This generates (simplified) a NoteEvent, with its pitch and duration
2002 as event properties,
2008 (ly:make-duration 2 0 1 1)
2010 (ly:make-pitch 0 0 0)
2013 which the Note_heads_engraver hears. It passes this information on to
2014 the NoteHead grob it creates from the event, so the head's correct
2015 position and duration-log can be determined once it's ready for
2018 If we transpose the snippet,
2021 \transpose c d @{ c' @}
2024 the pitch is changed before it reaches the engraver (in fact, it
2025 happens just after the parsing stage with the creation of a
2026 TransposedMusic music object):
2032 (ly:make-duration 2 0 1 1)
2034 (ly:make-pitch 0 1 0)
2037 You can see an example of a music property relevant to transposition:
2041 \transpose c d @{ c'2 \withMusicProperty #'untransposable ##t c' @}
2044 -> the second c' remains untransposed.
2046 Take a look at lily/music.cc to see where the transposition takes place.
2049 @subheading How do I tell about the execution environment?
2051 I get lost figuring out what environment the code I'm looking at is in when it
2052 executes. I found both the C++ and Scheme autochange code. Then I was trying
2053 to figure out where the code got called from. I finally figured out that the
2054 Scheme procedure was called before the C++ iterator code, but it took me a
2055 while to figure that out, and I still didn't know who did the calling in the
2056 first place. I only know a little bit about Flex and Bison, so reading those
2057 files helped only a little bit.
2059 @emph{Han-Wen:} GDB can be of help here. Set a breakpoint in C++, and run. When you
2060 hit the breakpoint, do a backtrace. You can inspect Scheme objects
2061 along the way by doing
2064 p ly_display_scm(obj)
2067 this will display OBJ through GUILE.
2069 @node Music functions and GUILE debugging
2070 @subsection Music functions and GUILE debugging
2072 Ian Hulin was trying to do some debugging in music functions, and
2073 came up with the following question
2076 I'm working on the Guile Debugger Stuff, and would like to try
2077 debugging a music function definition such as:
2080 conditionalMark = #(define-music-function (parser location) ()
2081 #@{ \tag #'instrumental-part @{\mark \default@} #@} )
2084 It appears conditionalMark does not get set up as an
2085 equivalent of a Scheme
2088 (define conditionalMark = define-music-function(parser location () ...
2092 although something gets defined because Scheme apparently recognizes
2095 #(set-break! conditionalMark)
2099 later on in the file without signalling any Guile errors.
2101 However the breakpoint trap is never encountered as
2102 define-music-function passed things on to ly:make-music-function,
2103 which is really C++ code ly_make_music_function, so Guile never
2104 finds out about the breakpoint.
2106 Han-Wen answered as follows:
2108 You can see the defintion by doing
2111 #(display conditionalMark)
2115 inside the .ly file.
2117 The breakpoint failing may have to do with the call sequence. See
2118 parser.yy, run_music_function(). The function is called directly from
2119 C++, without going through the GUILE evaluator, so I think that is why
2120 there is no debugger trap.