1 @c -*- coding: utf-8; mode: texinfo; -*-
3 @chapter Programming work
6 * Overview of LilyPond architecture::
7 * LilyPond programming languages::
8 * Programming without compiling::
11 * Warnings Errors Progress and Debug Output::
12 * Debugging LilyPond::
13 * Tracing object relationships::
14 * Adding or modifying features::
19 * Scheme->C interface::
20 * LilyPond miscellany::
23 @node Overview of LilyPond architecture
24 @section Overview of LilyPond architecture
26 LilyPond processes the input file into graphical and musical output in a
27 number of stages. This process, along with the types of routines that
28 accomplish the various stages of the process, is described in this section. A
29 more complete description of the LilyPond architecture and internal program
30 execution is found in Erik Sandberg's
31 @uref{http://lilypond.org/web/images/thesis-erik-sandberg.pdf, master's
34 The first stage of LilyPond processing is @emph{parsing}. In the parsing
35 process, music expressions in LilyPond input format are converted to music
36 expressions in Scheme format. In Scheme format, a music expression is a list
37 in tree form, with nodes that indicate the relationships between various music
38 events. The LilyPond parser is written in Bison.
40 The second stage of LilyPond processing is @emph{iterating}. Iterating
41 assigns each music event to a context, which is the environment in which the
42 music will be finally engraved. The context is responsible for all further
43 processing of the music. It is during the iteration stage that contexts are
44 created as necessary to ensure that every note has a Voice type context (e.g.
45 Voice, TabVoice, DrumVoice, CueVoice, MensuralVoice, VaticanaVoice,
46 GregorianTranscriptionVoice), that the Voice type contexts exist in
47 appropriate Staff type contexts, and that parallel Staff type contexts exist
48 in StaffGroup type contexts. In addition, during the iteration stage each
49 music event is assigned a moment, or a time in the music when the event
52 Each type of music event has an associated iterator. Iterators are defined in
53 @file{*-iterator.cc}. During iteration, an
54 event's iterator is called to deliver that music event to the appropriate
57 The final stage of LilyPond processing is @emph{translation}. During
58 translation, music events are prepared for graphical or midi output. The
59 translation step is accomplished by the polymorphic base class Translator
60 through its two derived classes: Engraver (for graphical output) and
61 Performer (for midi output).
63 Translators are defined in C++ files named @file{*-engraver.cc}
64 and @file{*-performer.cc}.
65 Much of the work of translating is handled by Scheme functions,
66 which is one of the keys to LilyPond's exceptional flexibility.
68 @sourceimage{architecture-diagram,,,png}
71 @node LilyPond programming languages
72 @section LilyPond programming languages
74 Programming in LilyPond is done in a variety of programming languages. Each
75 language is used for a specific purpose or purposes. This section describes
76 the languages used and provides links to reference manuals and tutorials for
77 the relevant language.
81 The core functionality of LilyPond is implemented in C++.
83 C++ is so ubiquitous that it is difficult to identify either a reference
84 manual or a tutorial. Programmers unfamiliar with C++ will need to spend some
85 time to learn the language before attempting to modify the C++ code.
87 The C++ code calls Scheme/GUILE through the GUILE interface, which is
89 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html, GUILE
94 The LilyPond lexer is implemented in Flex, an implementation of the Unix lex
95 lexical analyser generator. Resources for Flex can be found
96 @uref{http://flex.sourceforge.net/, here}.
100 The LilyPond parser is implemented in Bison, a GNU parser generator. The
101 Bison homepage is found at @uref{http://www.gnu.org/software/bison/,
102 gnu.org}. The manual (which includes both a reference and tutorial) is
103 @uref{http://www.gnu.org/software/bison/manual/index.html, available} in a
108 GNU Make is used to control the compiling process and to build the
109 documentation and the website. GNU Make documentation is available at
110 @uref{http://www.gnu.org/software/make/manual/, the GNU website}.
112 @subsection GUILE or Scheme
114 GUILE is the dialect of Scheme that is used as LilyPond's extension language.
115 Many extensions to LilyPond are written entirely in GUILE. The
116 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html,
117 GUILE Reference Manual} is available online.
119 @uref{http://mitpress.mit.edu/sicp/full-text/book/book.html, Structure and
120 Interpretation of Computer Programs}, a popular textbook used to teach
121 programming in Scheme is available in its entirety online.
123 An introduction to Guile/Scheme as used in LilyPond can be found in the
124 @rextend{Scheme tutorial}.
128 MetaFont is used to create the music fonts used by LilyPond. A MetaFont
129 tutorial is available at @uref{http://metafont.tutorial.free.fr/, the
130 METAFONT tutorial page}.
132 @subsection PostScript
134 PostScript is used to generate graphical output. A brief PostScript tutorial
135 is @uref{http://local.wasp.uwa.edu.au/~pbourke/dataformats/postscript/,
136 available online}. The
137 @uref{http://www.adobe.com/devnet/postscript/pdfs/PLRM.pdf, PostScript Language
138 Reference} is available online in PDF format.
142 Python is used for XML2ly and is used for building the documentation and the
145 Python documentation is available at @uref{http://www.python.org/doc/,
148 @node Programming without compiling
149 @section Programming without compiling
151 Much of the development work in LilyPond takes place by changing @file{*.ly} or
152 @file{*.scm} files. These changes can be made without compiling LilyPond. Such
153 changes are described in this section.
156 @subsection Modifying distribution files
158 Much of LilyPond is written in Scheme or LilyPond input files. These
159 files are interpreted when the program is run, rather than being compiled
160 when the program is built, and are present in all LilyPond distributions.
161 You will find @file{.ly} files in the @file{ly/} directory and the Scheme files in the
162 @file{scm/} directory. Both Scheme files and @file{.ly} files can be modified and
163 saved with any text editor. It's probably wise to make a backup copy of
164 your files before you modify them, although you can reinstall if the
165 files become corrupted.
167 Once you've modified the files, you can test the changes just by running
168 LilyPond on some input file. It's a good idea to create a file that
169 demonstrates the feature you're trying to add. This file will eventually
170 become a regression test and will be part of the LilyPond distribution.
172 @subsection Desired file formatting
174 Files that are part of the LilyPond distribution have Unix-style line
175 endings (LF), rather than DOS (CR+LF) or MacOS 9 and earlier (CR). Make
176 sure you use the necessary tools to ensure that Unix-style line endings are
177 preserved in the patches you create.
179 Tab characters should not be included in files for distribution. All
180 indentation should be done with spaces. Most editors have settings to
181 allow the setting of tab stops and ensuring that no tab characters are
182 included in the file.
184 Scheme files and LilyPond files should be written according to standard
185 style guidelines. Scheme file guidelines can be found at
186 @uref{http://community.schemewiki.org/?scheme-style}. Following these
187 guidelines will make your code easier to read. Both you and others that
188 work on your code will be glad you followed these guidelines.
190 For LilyPond files, you should follow the guidelines for LilyPond snippets
191 in the documentation. You can find these guidelines at
192 @ref{Texinfo introduction and usage policy}.
194 @node Finding functions
195 @section Finding functions
197 When making changes or fixing bugs in LilyPond, one of the initial
198 challenges is finding out where in the code tree the functions to
199 be modified live. With nearly 3000 files in the source tree,
200 trial-and-error searching is generally ineffective. This section
201 describes a process for finding interesting code.
203 @subsection Using the ROADMAP
205 The file ROADMAP is located in the main directory of the lilypond source.
206 ROADMAP lists all of the directories in the LilyPond source tree, along
207 with a brief description of the kind of files found in each directory.
208 This can be a very helpful tool for deciding which directories to search
209 when looking for a function.
212 @subsection Using grep to search
214 Having identified a likely subdirectory to search, the grep utility can
215 be used to search for a function name. The format of the grep command is
218 grep -i functionName subdirectory/*
221 This command will search all the contents of the directory subdirectory/
222 and display every line in any of the files that contains
223 functionName. The @option{-i} option makes @command{grep} ignore
224 case -- this can be very useful if you are not yet familiar with
225 our capitalization conventions.
227 The most likely directories to grep for function names are @file{scm/} for
228 scheme files, ly/ for lilypond input (@file{*.ly}) files, and @file{lily/} for C++
232 @subsection Using git grep to search
234 If you have used git to obtain the source, you have access to a
235 powerful tool to search for functions. The command:
238 git grep functionName
241 will search through all of the files that are present in the git
242 repository looking for functionName. It also presents the results
243 of the search using @code{less}, so the results are displayed one page
246 @subsection Searching on the git repository at Savannah
248 You can also use the equivalent of git grep on the Savannah server.
253 Go to http://git.sv.gnu.org/gitweb/?p=lilypond.git
256 In the pulldown box that says commit, select grep.
259 Type functionName in the search box, and hit enter/return
263 This will initiate a search of the remote git repository.
269 This section describes style guidelines for LilyPond
276 * Naming conventions::
285 @subsection Languages
287 C++ and Python are preferred. Python code should use PEP 8.
291 @subsection Filenames
293 Definitions of classes that are only accessed via pointers (*) or
294 references (&) shall not be included as include files.
300 ".cc" Implementation files
301 ".icc" Inline definition files
302 ".tcc" non inline Template defs
306 (setq auto-mode-alist
307 (append '(("\\.make$" . makefile-mode)
308 ("\\.cc$" . c++-mode)
309 ("\\.icc$" . c++-mode)
310 ("\\.tcc$" . c++-mode)
311 ("\\.hh$" . c++-mode)
312 ("\\.pod$" . text-mode)
317 The class Class_name is coded in @q{class-name.*}
321 @subsection Indentation
323 Standard GNU coding style is used.
325 @subsubheading Indenting files with @code{fixcc.py} (recommended)
327 LilyPond provides a python script that will adjust the indentation
328 and spacing on a @code{.cc} or @code{.hh} file to very near the
332 scripts/auxiliar/fixcc.py FILENAME
335 This can be run on all files at once, but this is not recommended
336 for normal contributors or developers.
339 scripts/auxiliar/fixcc.py \
340 $(find flower lily -name '*cc' -o -name '*hh' | grep -v /out)
344 @subsubheading Indenting with emacs
346 The following hooks will produce indentation which is similar to
347 our official indentation as produced with @code{fixcc.py}.
350 (add-hook 'c++-mode-hook
353 (setq indent-tabs-mode nil))
356 If you like using font-lock, you can also add this to your
360 (setq font-lock-maximum-decoration t)
361 (setq c++-font-lock-keywords-3
363 c++-font-lock-keywords-3
364 '(("\\b\\(a-zA-Z_?+_\\)\\b" 1 font-lock-variable-name-face) ("\\b\\(A-Z?+a-z_?+\\)\\b" 1 font-lock-type-face))
369 @subheading Indenting with vim
371 Although emacs indentation is the GNU standard, acceptable
372 indentation can usually be accomplished with vim. Some hints for
384 filetype plugin indent on
386 set ignorecase smartcase
389 set statusline=%F%m%r%h%w\ %@{&ff@}\ %Y\ [ASCII=\%03.3b]\ [HEX=\%02.2B]\ %04l,%04v\ %p%%\ [LEN=%L]
392 " Remove trailing whitespace on write
393 autocmd BufWritePre * :%s/\s\+$//e
396 With this @file{.vimrc}, files can be reindented automatically by
397 highlighting the lines to be indented in visual mode (use V to
398 enter visual mode) and pressing @code{=}.
400 A @file{scheme.vim} file will help improve the indentation. This
401 one was suggested by Patrick McCarty. It should be saved in
402 @file{~/.vim/after/syntax/scheme.vim}.
405 " Additional Guile-specific 'forms'
406 syn keyword schemeSyntax define-public define*-public
407 syn keyword schemeSyntax define* lambda* let-keywords*
408 syn keyword schemeSyntax defmacro defmacro* define-macro
409 syn keyword schemeSyntax defmacro-public defmacro*-public
410 syn keyword schemeSyntax use-modules define-module
411 syn keyword schemeSyntax define-method define-class
413 " Additional LilyPond-specific 'forms'
414 syn keyword schemeSyntax define-markup-command define-markup-list-command
415 syn keyword schemeSyntax define-safe-public define-music-function
416 syn keyword schemeSyntax def-grace-function
418 " All of the above should influence indenting too
419 set lw+=define-public,define*-public
420 set lw+=define*,lambda*,let-keywords*
421 set lw+=defmacro,defmacro*,define-macro
422 set lw+=defmacro-public,defmacro*-public
423 set lw+=use-modules,define-module
424 set lw+=define-method,define-class
425 set lw+=define-markup-command,define-markup-list-command
426 set lw+=define-safe-public,define-music-function
427 set lw+=def-grace-function
429 " These forms should not influence indenting
433 " Try to highlight all ly: procedures
434 syn match schemeFunc "ly:[^) ]\+"
438 @node Naming conventions
439 @subsection Naming Conventions
441 Naming conventions have been established for LilyPond
444 @subheading Classes and Types
446 Classes begin with an uppercase letter, and words
447 in class names are separated with @code{_}:
455 Member variable names end with an underscore:
463 Macro names should be written in uppercase completely,
464 with words separated by @code{_}:
470 @subheading Variables
472 Variable names should be complete words, rather than abbreviations.
473 For example, it is preferred to use @code{thickness} rather than
474 @code{th} or @code{t}.
476 Multi-word variable names in C++ should have the words separated
477 by the underscore character (@q{_}):
480 cxx_multiword_variable
483 Multi-word variable names in Scheme should have the words separated
487 scheme-multiword-variable
491 @subsection Broken code
493 Do not write broken code. This includes hardwired dependencies,
494 hardwired constants, slow algorithms and obvious limitations. If
495 you can not avoid it, mark the place clearly, and add a comment
496 explaining shortcomings of the code.
498 Ideally, the comment marking the shortcoming would include
499 TODO, so that it is marked for future fixing.
501 We reject broken-in-advance on principle.
505 @subsection Code comments
507 Comments may not be needed if descriptive variable names are used
508 in the code and the logic is straightforward. However, if the
509 logic is difficult to follow, and particularly if non-obvious
510 code has been included to resolve a bug, a comment describing
511 the logic and/or the need for the non-obvious code should be included.
513 There are instances where the current code could be commented better.
514 If significant time is required to understand the code as part of
515 preparing a patch, it would be wise to add comments reflecting your
516 understanding to make future work easier.
519 @node Handling errors
520 @subsection Handling errors
522 As a general rule, you should always try to continue computations,
523 even if there is some kind of error. When the program stops, it
524 is often very hard for a user to pinpoint what part of the input
525 causes an error. Finding the culprit is much easier if there is
526 some viewable output.
528 So functions and methods do not return errorcodes, they never
529 crash, but report a programming_error and try to carry on.
531 Error and warning messages need to be localized.
535 @subsection Localization
537 This document provides some guidelines to help programmers write
539 messages. To help translations, user messages must follow
540 uniform conventions. Follow these rules when coding for LilyPond.
541 Hopefully, this can be replaced by general GNU guidelines in the
542 future. Even better would be to have an English (en_BR, en_AM)
543 guide helping programmers writing consistent messages for all GNU
546 Non-preferred messages are marked with `+'. By convention,
547 ungrammatical examples are marked with `*'. However, such ungrammatical
548 examples may still be preferred.
553 Every message to the user should be localized (and thus be marked
554 for localization). This includes warning and error messages.
557 Do not localize/gettextify:
561 `programming_error ()'s
564 `programming_warning ()'s
570 output strings (PostScript, TeX, etc.)
575 Messages to be localized must be encapsulated in `_ (STRING)' or
576 `_f (FORMAT, ...)'. E.g.:
579 warning (_ ("need music in a score"));
580 error (_f ("cannot open file: `%s'", file_name));
583 In some rare cases you may need to call `gettext ()' by hand. This
584 happens when you pre-define (a list of) string constants for later
585 use. In that case, you'll probably also need to mark these string
586 constants for translation, using `_i (STRING)'. The `_i' macro is
587 a no-op, it only serves as a marker for `xgettext'.
590 char const* messages[] = @{
591 _i ("enable debugging output"),
592 _i ("ignore lilypond version"),
599 puts (gettext (messages i));
603 See also @file{flower/getopt-long.cc} and @file{lily/main.cc}.
606 Do not use leading or trailing whitespace in messages. If you need
607 whitespace to be printed, prepend or append it to the translated
611 message ("Calculating line breaks..." + " ");
615 Error or warning messages displayed with a file name and line
616 number never start with a capital, eg,
619 foo.ly: 12: not a duration: 3
622 Messages containing a final verb, or a gerund (`-ing'-form) always
623 start with a capital. Other (simpler) messages start with a
629 Not declaring: `foo'.
633 Avoid abbreviations or short forms, use `cannot' and `do not'
634 rather than `can't' or `don't'
635 To avoid having a number of different messages for the same
636 situation, well will use quoting like this `"message: `%s'"' for all
637 strings. Numbers are not quoted:
640 _f ("cannot open file: `%s'", name_str)
641 _f ("cannot find character number: %d", i)
645 Think about translation issues. In a lot of cases, it is better to
646 translate a whole message. English grammar must not be imposed on the
647 translator. So, instead of
650 stem at + moment.str () + does not fit in beam
656 _f ("stem at %s does not fit in beam", moment.str ())
660 Split up multi-sentence messages, whenever possible. Instead of
663 warning (_f ("out of tune! Can't find: `%s'", "Key_engraver"));
664 warning (_f ("cannot find font `%s', loading default", font_name));
670 warning (_ ("out of tune:"));
671 warning (_f ("cannot find: `%s', "Key_engraver"));
672 warning (_f ("cannot find font: `%s', font_name));
673 warning (_f ("Loading default font"));
677 If you must have multiple-sentence messages, use full punctuation.
678 Use two spaces after end of sentence punctuation. No punctuation
679 (esp. period) is used at the end of simple messages.
682 _f ("Non-matching braces in text `%s', adding braces", text)
683 _ ("Debug output disabled. Compiled with NPRINT.")
684 _f ("Huh? Not a Request: `%s'. Ignoring.", request)
688 Do not modularize too much; words frequently cannot be translated
689 without context. It is probably safe to treat most occurrences of
690 words like stem, beam, crescendo as separately translatable words.
693 When translating, it is preferable to put interesting information
694 at the end of the message, rather than embedded in the middle.
695 This especially applies to frequently used messages, even if this
696 would mean sacrificing a bit of eloquency. This holds for original
697 messages too, of course.
700 en: cannot open: `foo.ly'
701 + nl: kan `foo.ly' niet openen (1)
702 kan niet openen: `foo.ly'* (2)
703 niet te openen: `foo.ly'* (3)
707 The first nl message, although grammatically and stylistically
708 correct, is not friendly for parsing by humans (even if they speak
709 dutch). I guess we would prefer something like (2) or (3).
712 Do not run make po/po-update with GNU gettext < 0.10.35
717 @node Warnings Errors Progress and Debug Output
718 @section Warnings, Errors, Progress and Debug Output
720 @unnumberedsubsec Available log levels
722 LilyPond has several loglevels, which specify how verbose the output on
723 the console should be:
725 @item NONE: No output at all, even on failure
726 @item ERROR: Only error messages
727 @item WARN: Only error messages and warnings
728 @item BASIC_PROGRESS: Warnings, errors and basic progress (success, etc.)
729 @item PROGRESS: Warnings, errors and full progress messages
730 @item INFO: Warnings, errors, progress and more detailed information (default)
731 @item DEBUG: All messages, including vull debug messages (very verbose!)
734 The loglevel can either be set with the environment variable
735 @code{LILYPOND_LOGLEVEL} or on the command line with the @option{--loglevel=...}
738 @unnumberedsubsec Functions for debug and log output
740 LilyPond has two different types of error and log functions:
744 If a warning or error is caused by an identified position in the input file,
745 e.g. by a grob or by a music expression, the functions of the @code{Input}
746 class provide logging functionality that prints the position of the message
747 in addition to the message.
750 If a message can not be associated with a particular position in an input file,
751 e.g. the output file cannot be written, then the functions in the
752 @code{flower/include/warn.hh} file will provide logging functionality that
753 only prints out the message, but no location.
757 There are also Scheme functions to access all of these logging functions from
758 scheme. In addition, the Grob class contains some convenience wrappers for
759 even easier access to these functions.
761 The message and debug functions in @code{warn.hh} also have an optional
762 argument @code{newline}, which specifies whether the message should always
763 start on a new line or continue a previous message.
764 By default, @code{progress_indication} does NOT start on a new line, but rather
765 continue the previous output. They also do not have a particular input
766 position associated, so there are no progress functions in the Input class.
767 All other functions by default start their output on a new line.
769 The error functions come in three different flavors: fatal error messages,
770 programming error messages and normal error messages. Errors written
771 by the @code{error ()} function will cause LilyPond to exit immediately,
772 errors by @code{Input::error ()} will continue the compilation, but
773 return a non-zero return value of the lilypond call (i.e. indicate an
774 unsuccessful program execution). All other errors will be printed on the
775 console, but not exit LilyPond or indicate an unsuccessful return code.
776 Their only differences to a warnings are the displayed text and that
777 they will be shown with loglevel @code{ERROR}.
779 If the Scheme option @code{warning-as-error} is set, any warning will be
780 treated as if @code{Input::error} was called.
783 @unnumberedsubsec All logging functions at a glance
785 @multitable @columnfractions 0.16 0.42 0.42
787 @tab C++, no location
788 @tab C++ from input location
791 @tab @code{error ()}, @code{programming_error (msg)}, @code{non_fatal_error (msg)}
792 @tab @code{Input::error (msg)}, @code{Input::programming_error (msg)}
795 @tab @code{warning (msg)}
796 @tab @code{Input::warning (msg)}
799 @tab @code{basic_progress (msg)}
803 @tab @code{progress_indication (msg)}
807 @tab @code{message (msg)}
808 @tab @code{Input::message (msg)}
811 @tab @code{debug_output (msg)}
812 @tab @code{Input::debug_output (msg)}
818 @tab Scheme, music expression
821 @tab @code{Grob::programming_error (msg)}
825 @tab @code{Grob::warning (msg)}
826 @tab @code{(ly:music-warning music msg)}
838 @tab @code{(ly:music-message music msg)}
847 @tab Scheme, no location
848 @tab Scheme, input location
852 @tab @code{(ly:error msg args)}, @code{(ly:programming-error msg args)}
855 @tab @code{(ly:warning msg args)}
856 @tab @code{(ly:input-warning input msg args)}
859 @tab @code{(ly:basic-progress msg args)}
863 @tab @code{(ly:progress msg args)}
867 @tab @code{(ly:message msg args)}
868 @tab @code{(ly:input-message input msg args)}
871 @tab @code{(ly:debug msg args)}
879 @node Debugging LilyPond
880 @section Debugging LilyPond
882 The most commonly used tool for debugging LilyPond is the GNU
883 debugger gdb. The gdb tool is used for investigating and debugging
884 core Lilypond code written in C++. Another tool is available for
885 debugging Scheme code using the Guile debugger. This section
886 describes how to use both gdb and the Guile Debugger.
889 * Debugging overview::
890 * Debugging C++ code::
891 * Debugging Scheme code::
894 @node Debugging overview
895 @subsection Debugging overview
897 Using a debugger simplifies troubleshooting in at least two ways.
899 First, breakpoints can be set to pause execution at any desired point.
900 Then, when execution has paused, debugger commands can be issued to
901 explore the values of various variables or to execute functions.
903 Second, the debugger can display a stack trace, which shows the
904 sequence in which functions have been called and the arguments
905 passed to the called functions.
907 @node Debugging C++ code
908 @subsection Debugging C++ code
910 The GNU debugger, gdb, is the principal tool for debugging C++ code.
912 @subheading Compiling LilyPond for use with gdb
914 In order to use gdb with LilyPond, it is necessary to compile
915 LilyPond with debugging information. This is accomplished by running
916 the following commands in the main LilyPond source directory.
919 ./configure --disable-optimising
923 This will create a version of LilyPond containing debugging
924 information that will allow the debugger to tie the source code
925 to the compiled code.
927 You should not do @var{make install} if you want to use a debugger
928 with LilyPond. The @var{make install} command will strip debugging
929 information from the LilyPond binary.
931 @subheading Typical gdb usage
933 Once you have compiled the Lilypond image with the necessary
934 debugging information it will have been written to a location in a
935 subfolder of your current working directory:
941 This is important as you will need to let gdb know where to find the
942 image containing the symbol tables. You can invoke gdb from the
943 command line using the following:
949 This loads the LilyPond symbol tables into gdb. Then, to run
950 LilyPond on @file{test.ly} under the debugger, enter the following:
959 As an alternative to running gdb at the command line you may try
960 a graphical interface to gdb such as ddd:
966 You can also use sets of standard gdb commands stored in a .gdbinit
967 file (see next section).
969 @subheading Typical .gdbinit files
971 The behavior of gdb can be readily customized through the use of a
972 @var{.gdbinit} file. A @var{.gdbinit} file is a file named
973 @var{.gdbinit} (notice the @qq{.} at the beginning of the file name)
974 that is placed in a user's home directory.
976 The @var{.gdbinit} file below is from Han-Wen. It sets breakpoints
977 for all errors and defines functions for displaying scheme objects
978 (ps), grobs (pgrob), and parsed music expressions (pmusic).
981 file lily/out/lilypond
983 b Grob::programming_error
986 print ly_display_scm($arg0)
989 print ly_display_scm($arg0->self_scm_)
990 print ly_display_scm($arg0->mutable_property_alist_)
991 print ly_display_scm($arg0->immutable_property_alist_)
992 print ly_display_scm($arg0->object_alist_)
995 print ly_display_scm($arg0->self_scm_)
996 print ly_display_scm($arg0->mutable_property_alist_)
997 print ly_display_scm($arg0->immutable_property_alist_)
1001 @node Debugging Scheme code
1002 @subsection Debugging Scheme code
1004 Scheme code can be developed using the Guile command line
1005 interpreter @code{top-repl}. You can either investigate
1006 interactively using just Guile or you can use the debugging
1007 tools available within Guile.
1009 @subheading Using Guile interactively with LilyPond
1011 In order to experiment with Scheme programming in the LilyPond
1012 environment, it is necessary to have a Guile interpreter that
1013 has all the LilyPond modules loaded. This requires the following
1016 First, define a Scheme symbol for the active module in the @file{.ly} file:
1019 #(module-define! (resolve-module '(guile-user))
1020 'lilypond-module (current-module))
1023 Now place a Scheme function in the @file{.ly} file that gives an
1024 interactive Guile prompt:
1030 When the @file{.ly} file is compiled, this causes the compilation to be
1031 interrupted and an interactive guile prompt to appear. Once the
1032 guile prompt appears, the LilyPond active module must be set as the
1033 current guile module:
1036 guile> (set-current-module lilypond-module)
1039 You can demonstrate these commands are operating properly by typing the name
1040 of a LilyPond public scheme function to check it has been defined:
1043 guile> fret-diagram-verbose-markup
1044 #<procedure fret-diagram-verbose-markup (layout props marking-list)>
1047 If the LilyPond module has not been correctly loaded, an error
1048 message will be generated:
1051 guile> fret-diagram-verbose-markup
1052 ERROR: Unbound variable: fret-diagram-verbose-markup
1053 ABORT: (unbound-variable)
1056 Once the module is properly loaded, any valid LilyPond Scheme
1057 expression can be entered at the interactive prompt.
1059 After the investigation is complete, the interactive guile
1060 interpreter can be exited:
1066 The compilation of the @file{.ly} file will then continue.
1068 @subheading Using the Guile debugger
1070 To set breakpoints and/or enable tracing in Scheme functions, put
1073 \include "guile-debugger.ly"
1076 in your input file after any scheme procedures you have defined in
1077 that file. This will invoke the Guile command-line after having set
1078 up the environment for the debug command-line. When your input file
1079 is processed, a guile prompt will be displayed. You may now enter
1080 commands to set up breakpoints and enable tracing by the Guile debugger.
1082 @subheading Using breakpoints
1084 At the guile prompt, you can set breakpoints with
1085 the @code{set-break!} procedure:
1088 guile> (set-break! my-scheme-procedure)
1091 Once you have set the desired breakpoints, you exit the guile repl frame
1098 Then, when one of the scheme routines for which you have set
1099 breakpoints is entered, guile will interrupt execution in a debug
1100 frame. At this point you will have access to Guile debugging
1101 commands. For a listing of these commands, type:
1107 Alternatively you may code the breakpoints in your Lilypond source
1108 file using a command such as:
1111 #(set-break! my-scheme-procedure)
1114 immediately after the @code{\include} statement. In this case the
1115 breakpoint will be set straight after you enter the @code{(quit)}
1116 command at the guile prompt.
1118 Embedding breakpoint commands like this is particularly useful if
1119 you want to look at how the Scheme procedures in the @file{.scm}
1120 files supplied with LilyPond work. To do this, edit the file in
1121 the relevant directory to add this line near the top:
1124 (use-modules (scm guile-debugger))
1127 Now you can set a breakpoint after the procedure you are interested
1128 in has been declared. For example, if you are working on routines
1129 called by @var{print-book-with} in @file{lily-library.scm}:
1132 (define (print-book-with parser book process-procedure)
1133 (let* ((paper (ly:parser-lookup parser '$defaultpaper))
1134 (layout (ly:parser-lookup parser '$defaultlayout))
1135 (outfile-name (get-outfile-name parser)))
1136 (process-procedure book paper layout outfile-name)))
1138 (define-public (print-book-with-defaults parser book)
1139 (print-book-with parser book ly:book-process))
1141 (define-public (print-book-with-defaults-as-systems parser book)
1142 (print-book-with parser book ly:book-process-to-systems))
1146 At this point in the code you could add this to set a breakpoint at
1150 (set-break! print-book-with)
1153 @subheading Tracing procedure calls and evaluator steps
1155 Two forms of trace are available:
1158 (set-trace-call! my-scheme-procedure)
1164 (set-trace-subtree! my-scheme-procedure)
1167 @code{set-trace-call!} causes Scheme to log a line to the standard
1168 output to show when the procedure is called and when it exits.
1170 @code{set-trace-subtree!} traces every step the Scheme evaluator
1171 performs in evaluating the procedure.
1173 @node Tracing object relationships
1174 @section Tracing object relationships
1176 Understanding the LilyPond source often boils down to figuring out what
1177 is happening to the Grobs. Where (and why) are they being created,
1178 modified and destroyed? Tracing Lily through a debugger in order to
1179 identify these relationships can be time-consuming and tedious.
1181 In order to simplify this process, a facility has been added to
1182 display the grobs that are created and the properties that are set
1183 and modified. Although it can be complex to get set up, once set up
1184 it easily provides detailed information about the life of grobs
1185 in the form of a network graph.
1187 Each of the steps necessary to use the graphviz utility
1192 @item Installing graphviz
1194 In order to create the graph of the object relationships, it is
1195 first necessary to install Graphviz. graphviz is available for a
1196 number of different platforms:
1199 @uref{http://www.graphviz.org/Download..php}
1202 @item Modifying config.make
1204 In order for the Graphviz tool to work, config.make must be modified.
1205 It is probably a good idea to first save a copy of config.make under
1206 a different name. Then, edit config.make by removing every occurrence
1207 of @option{-DNDEBUG}.
1209 @item Rebuilding LilyPond
1211 The executable code of LilyPond must be rebuilt from scratch:
1214 make -C lily clean && make -C lily
1217 @item Create a graphviz-compatible @file{.ly} file
1219 In order to use the graphviz utility, the @file{.ly} file must include
1220 @file{ly/graphviz-init.ly}, and should then specify the
1221 grobs and symbols that should be tracked. An example of this
1222 is found in @file{input/regression/graphviz.ly}.
1224 @item Run lilypond with output sent to a log file
1226 The Graphviz data is sent to stderr by lilypond, so it is
1227 necessary to redirect stderr to a logfile:
1230 lilypond graphviz.ly 2> graphviz.log
1233 @item Edit the logfile
1235 The logfile has standard lilypond output, as well as the Graphviz
1236 output data. Delete everything from the beginning of the file
1237 up to but not including the first occurrence of @code{digraph}.
1239 Also, delete the final liypond message about successs from the end
1242 @item Process the logfile with @code{dot}
1244 The directed graph is created from the log file with the program
1248 dot -Tpdf graphviz.log > graphviz.pdf
1253 The pdf file can then be viewed with any pdf viewer.
1255 When compiled without @option{-DNDEBUG}, lilypond may run slower
1256 than normal. The original configuration can be restored by either
1257 renaming the saved copy of @code{config.make} or rerunning
1258 @code{configure}. Then rebuild lilypond with
1261 make -C lily clean && make -C lily
1265 @node Adding or modifying features
1266 @section Adding or modifying features
1268 When a new feature is to be added to LilyPond, it is necessary to
1269 ensure that the feature is properly integrated to maintain
1270 its long-term support. This section describes the steps necessary
1271 for feature addition and modification.
1276 * Write regression tests::
1277 * Write convert-ly rule::
1278 * Automatically update documentation::
1279 * Manually update documentation::
1280 * Edit changes.tely::
1281 * Verify successful build::
1282 * Verify regression tests::
1283 * Post patch for comments::
1285 * Closing the issues::
1288 @node Write the code
1289 @subsection Write the code
1291 You should probably create a new git branch for writing the code, as that
1292 will separate it from the master branch and allow you to continue
1293 to work on small projects related to master.
1295 Please be sure to follow the rules for programming style discussed
1296 earlier in this chapter.
1299 @node Write regression tests
1300 @subsection Write regression tests
1302 In order to demonstrate that the code works properly, you will
1303 need to write one or more regression tests. These tests are
1304 typically @file{.ly} files that are found in @file{input/regression}.
1306 Regression tests should be as brief as possible to demonstrate the
1307 functionality of the code.
1309 Regression tests should generally cover one issue per test. Several
1310 short, single-issue regression tests are preferred to a single, long,
1311 multiple-issue regression test.
1313 Use existing regression tests as templates to demonstrate the type of
1314 header information that should be included in a regression test.
1317 @node Write convert-ly rule
1318 @subsection Write convert-ly rule
1320 If the modification changes the input syntax, a convert-ly rule
1321 should be written to automatically update input files from older
1324 convert-ly rules are found in python/convertrules.py
1326 If possible, the convert-ly rule should allow automatic updating
1327 of the file. In some cases, this will not be possible, so the
1328 rule will simply point out to the user that the feature needs
1331 @subsubheading Updating version numbers
1333 If a development release occurs between you writing your patch and
1334 having it approved+pushed, you will need to update the version
1335 numbers in your tree. This can be done with:
1338 scripts/auxiliar/update-patch-version old.version.number new.version.number
1341 It will change all files in git, so use with caution and examine
1345 @node Automatically update documentation
1346 @subsection Automatically update documentation
1348 @command{convert-ly} should be used to update the documentation,
1349 the snippets, and the regression tests. This not only makes the
1350 necessary syntax changes, it also tests the @command{convert-ly}
1353 The automatic updating is performed by moving to the top-level
1354 source directory, then running:
1357 scripts/auxiliar/update-with-convert-ly.sh
1360 If you did an out-of-tree build, pass in the relative path:
1363 BUILD_DIR=../build-lilypond/ scripts/auxiliar/update-with-convert-ly.sh
1367 @node Manually update documentation
1368 @subsection Manually update documentation
1370 Where the convert-ly rule is not able to automatically update the inline
1371 lilypond code in the documentation (i.e. if a NOT_SMART rule is used), the
1372 documentation must be manually updated. The inline snippets that require
1373 changing must be changed in the English version of the docs and all
1374 translated versions. If the inline code is not changed in the
1375 translated documentation, the old snippets will show up in the
1376 English version of the documentation.
1378 Where the convert-ly rule is not able to automatically update snippets
1379 in Documentation/snippets/, those snippets must be manually updated.
1380 Those snippets should be copied to Documentation/snippets/new. The
1381 comments at the top of the snippet describing its automatic generation
1382 should be removed. All translated texidoc strings should be removed.
1383 The comment @qq{% begin verbatim} should be removed. The syntax of
1384 the snippet should then be manually edited.
1386 Where snippets in Documentation/snippets are made obsolete, the snippet
1387 should be copied to Documentation/snippets/new. The comments and
1388 texidoc strings should be removed as described above. Then the body
1389 of the snippet should be changed to:
1393 This snippet is deprecated as of version X.Y.Z and
1394 will be removed from the documentation.
1399 where X.Y.Z is the version number for which the convert-ly rule was
1402 Update the snippet files by running:
1405 scripts/auxiliar/makelsr.py
1408 Where the convert-ly rule is not able to automatically update regression
1409 tests, the regression tests in input/regression should be manually
1412 Although it is not required, it is helpful if the developer
1413 can write relevant material for inclusion in the Notation
1414 Reference. If the developer does not feel qualified to write
1415 the documentation, a documentation editor will be able to
1416 write it from the regression tests. The text that is added to
1417 or removed from the documentation should be changed only in
1418 the English version.
1421 @node Edit changes.tely
1422 @subsection Edit changes.tely
1424 An entry should be added to Documentation/changes.tely to describe
1425 the feature changes to be implemented. This is especially important
1426 for changes that change input file syntax.
1428 Hints for changes.tely entries are given at the top of the file.
1430 New entries in changes.tely go at the top of the file.
1432 The changes.tely entry should be written to show how the new change
1433 improves LilyPond, if possible.
1436 @node Verify successful build
1437 @subsection Verify successful build
1439 When the changes have been made, successful completion must be
1447 When these commands complete without error, the patch is
1448 considered to function successfully.
1450 Developers on Windows who are unable to build LilyPond should
1451 get help from a Linux or OSX developer to do the make tests.
1454 @node Verify regression tests
1455 @subsection Verify regression tests
1457 In order to avoid breaking LilyPond, it is important to verify that
1458 the regression tests succeed, and that no unwanted changes are
1459 introduced into the output. This process is described in
1460 @ref{Regtest comparison}.
1462 @subheading Typical developer's edit/compile/test cycle
1464 TODO: is @code{[-j@var{X} CPU_COUNT=@var{X}]} useful for
1465 @code{test-baseline}, @code{check}, @code{clean},
1466 @code{test-redo}? Neil Puttock says it is useful for
1467 everything but @code{clean}, which is disk-limited.
1468 Need to check formally.
1477 make [-j@var{X} CPU_COUNT=@var{X}] check
1481 Edit/compile/test cycle:
1484 @emph{## edit source files, then...}
1486 make clean @emph{## only if needed (see below)}
1487 make [-j@var{X}] @emph{## only if needed (see below)}
1488 make test-redo @emph{## redo files differing from baseline}
1489 make [-j@var{X} CPU_COUNT=@var{X}] check @emph{## CPU_COUNT here?}
1500 If you modify any source files that have to be compiled (such as
1501 @file{.cc} or @file{.hh} files in @file{flower/} or @file{lily/}),
1502 then you must run @command{make} before @command{make test-redo},
1503 so @command{make} can compile the modified files and relink all
1504 the object files. If you only modify files which are interpreted,
1505 like those in the @file{scm/} and @file{ly/} directories, then
1506 @command{make} is not needed before @command{make test-redo}.
1508 TODO: Fix the following paragraph. You can do @command{rm mf/out/*}
1509 instead of make clean, and you can probably do
1510 @command{make -C mf/ clean} as well, but I haven't checked it -- cds
1512 Also, if you modify any font definitions in the @file{mf/}
1513 directory then you must run @command{make clean} and
1514 @command{make} before running @command{make test-redo}. This will
1515 recompile everything, whether modified or not, and takes a lot
1518 Running @command{make@tie{}check} will leave an HTML page
1519 @file{out/test-results/index.html}. This page shows all the
1520 important differences that your change introduced, whether in the
1521 layout, MIDI, performance or error reporting.
1526 @node Post patch for comments
1527 @subsection Post patch for comments
1529 See @ref{Uploading a patch for review}.
1533 @subsection Push patch
1535 Once all the comments have been addressed, the patch can be pushed.
1537 If the author has push privileges, the author will push the patch.
1538 Otherwise, a developer with push privileges will push the patch.
1541 @node Closing the issues
1542 @subsection Closing the issues
1544 Once the patch has been pushed, all the relevant issues should be
1547 On Rietveld, the author should log in and close the issue either by
1548 using the @q{Edit Issue} link, or by clicking the circled x icon
1549 to the left of the issue name.
1551 If the changes were in response to a feature request on the Google
1552 issue tracker for LilyPond, the author should change the status to
1553 Fixed and a tag @q{fixed_x_y_z} should be added, where the patch was
1554 fixed in version x.y.z. If
1555 the author does not have privileges to change the status, an email
1556 should be sent to bug-lilypond requesting the BugMeister to change
1560 @node Iterator tutorial
1561 @section Iterator tutorial
1563 TODO -- this is a placeholder for a tutorial on iterators
1565 Iterators are routines written in C++ that process music expressions
1566 and sent the music events to the appropriate engravers and/or
1570 @node Engraver tutorial
1571 @section Engraver tutorial
1573 Engravers are C++ classes that catch music events and
1574 create the appropriate grobs for display on the page. Though the
1575 majority of engravers are responsible for the creation of a single grob,
1576 in some cases (e.g. @code{New_fingering_engraver}), several different grobs
1579 Engravers listen for events and acknowledge grobs. Events are passed to
1580 the engraver in time-step order during the iteration phase. Grobs are
1581 made available to the engraver when they are created by other engravers
1582 during the iteration phase.
1586 * Useful methods for information processing::
1587 * Translation process::
1588 * Preventing garbage collection for SCM member variables::
1589 * Listening to music events::
1590 * Acknowledging grobs::
1591 * Engraver declaration/documentation::
1594 @node Useful methods for information processing
1595 @subsection Useful methods for information processing
1597 An engraver inherits the following public methods from the Translator
1598 base class, which can be used to process listened events and acknowledged
1602 @item @code{virtual void initialize ()}
1603 @item @code{void start_translation_timestep ()}
1604 @item @code{void process_music ()}
1605 @item @code{void process_acknowledged ()}
1606 @item @code{void stop_translation_timestep ()}
1607 @item @code{virtual void finalize ()}
1610 These methods are listed in order of translation time, with
1611 @code{initialize ()} and @code{finalize ()} bookending the whole
1612 process. @code{initialize ()} can be used for one-time initialization
1613 of context properties before translation starts, whereas
1614 @code{finalize ()} is often used to tie up loose ends at the end of
1615 translation: for example, an unterminated spanner might be completed
1616 automatically or reported with a warning message.
1619 @node Translation process
1620 @subsection Translation process
1622 At each timestep in the music, translation proceeds by calling the
1623 following methods in turn:
1625 @code{start_translation_timestep ()} is called before any user
1626 information enters the translators, i.e., no property operations
1627 (\set, \override, etc.) or events have been processed yet.
1629 @code{process_music ()} and @code{process_acknowledged ()} are called
1630 after all events in the current time step have been heard, or all
1631 grobs in the current time step have been acknowledged. The latter
1632 tends to be used exclusively with engravers which only acknowledge
1633 grobs, whereas the former is the default method for main processing
1636 @code{stop_translation_timestep ()} is called after all user
1637 information has been processed prior to beginning the translation for
1641 @node Preventing garbage collection for SCM member variables
1642 @subsection Preventing garbage collection for SCM member variables
1644 In certain cases, an engraver might need to ensure private Scheme
1645 variables (with type SCM) do not get swept away by Guile's garbage
1646 collector: for example, a cache of the previous key signature which
1647 must persist between timesteps. The method
1648 @code{virtual derived_mark () const} can be used in such cases:
1651 Engraver_name::derived_mark ()
1653 scm_gc_mark (private_scm_member_)
1658 @node Listening to music events
1659 @subsection Listening to music events
1661 External interfaces to the engraver are implemented by protected
1662 macros including one or more of the following:
1665 @item @code{DECLARE_TRANSLATOR_LISTENER (event_name)}
1666 @item @code{IMPLEMENT_TRANSLATOR_LISTENER (Engraver_name, event_name)}
1670 where @var{event_name} is the type of event required to provide the
1671 input the engraver needs and @var{Engraver_name} is the name of the
1674 Following declaration of a listener, the method is implemented as follows:
1677 IMPLEMENT_TRANSLATOR_LISTENER (Engraver_name, event_name)
1679 Engraver_name::listen_event_name (Stream event *event)
1681 ...body of listener method...
1686 @node Acknowledging grobs
1687 @subsection Acknowledging grobs
1689 Some engravers also need information from grobs as they are created
1690 and as they terminate. The mechanism and methods to obtain this
1691 information are set up by the macros:
1694 @item @code{DECLARE_ACKNOWLEDGER (grob_interface)}
1695 @item @code{DECLARE_END_ACKNOWLEDGER (grob_interface)}
1698 where @var{grob_interface} is an interface supported by the
1699 grob(s) which should be acknowledged. For example, the following
1700 code would declare acknowledgers for a @code{NoteHead} grob (via the
1701 @code{note-head-interface}) and any grobs which support the
1702 @code{side-position-interface}:
1705 @code{DECLARE_ACKNOWLEDGER (note_head)}
1706 @code{DECLARE_ACKNOWLEDGER (side_position)}
1709 The @code{DECLARE_END_ACKNOWLEDGER ()} macro sets up a spanner-specific
1710 acknowledger which will be called whenever a spanner ends.
1712 Following declaration of an acknowledger, the method is coded as follows:
1716 Engraver_name::acknowledge_interface_name (Grob_info info)
1718 ...body of acknowledger method...
1722 Acknowledge functions are called in the order engravers are
1723 @code{\consist}-ed (the only exception is if you set
1724 @code{must-be-last} to @code{#t}).
1726 If useful things are to be done to the acknowledged grobs, this
1727 should be deferred until all the acknowledging has finished, i.e.,
1728 store the acknowledged grobs and process the information in a
1729 @code{process-acknowledged ()} or @code{stop-translation-timestep ()}
1733 @node Engraver declaration/documentation
1734 @subsection Engraver declaration/documentation
1736 An engraver must have a public macro
1739 @item @code{TRANSLATOR_DECLARATIONS (Engraver_name)}
1743 where @code{Engraver_name} is the name of the engraver. This
1744 defines the common variables and methods used by every engraver.
1746 At the end of the engraver file, one or both of the following
1747 macros are generally called to document the engraver in the
1748 Internals Reference:
1751 @item @code{ADD_ACKNOWLEDGER (Engraver_name, grob_interface)}
1752 @item @code{ADD_TRANSLATOR (Engraver_name, Engraver_doc,
1753 Engraver_creates, Engraver_reads, Engraver_writes)}
1757 where @code{Engraver_name} is the name of the engraver, @code{grob_interface}
1758 is the name of the interface that will be acknowledged,
1759 @code{Engraver_doc} is a docstring for the engraver,
1760 @code{Engraver_creates} is the set of grobs created by the engraver,
1761 @code{Engraver_reads} is the set of properties read by the engraver,
1762 and @code{Engraver_writes} is the set of properties written by
1765 The @code{ADD_ACKNOWLEDGER} and @code{ADD_TRANSLATOR} macros use a
1766 non-standard indentation system. Each interface, grob, read property,
1767 and write property is on its own line, and the closing parenthesis
1768 and semicolon for the macro all occupy a separate line beneath the final
1769 interface or write property. See existing engraver files for more
1773 @node Callback tutorial
1774 @section Callback tutorial
1776 TODO -- This is a placeholder for a tutorial on callback functions.
1778 @node LilyPond scoping
1779 @section LilyPond scoping
1781 The Lilypond language has a concept of scoping, i.e. you can do
1787 (display (+ foo 2)))
1790 @noindent with @code{\paper}, @code{\midi} and @code{\header} being
1791 nested scope inside the @file{.ly} file-level scope. @w{@code{foo = 1}}
1792 is translated in to a scheme variable definition.
1794 This implemented using modules, with each scope being an anonymous
1795 module that imports its enclosing scope's module.
1797 Lilypond's core, loaded from @file{.scm} files, is usually placed in the
1798 @code{lily} module, outside the @file{.ly} level. In the case of
1805 we want to reuse the built-in definitions, without changes effected in
1806 user-level @file{a.ly} leaking into the processing of @file{b.ly}.
1808 The user-accessible definition commands have to take care to avoid
1809 memory leaks that could occur when running multiple files. All
1810 information belonging to user-defined commands and markups is stored in
1811 a manner that allows it to be garbage-collected when the module is
1812 dispersed, either by being stored module-locally, or in weak hash
1816 @node Scheme->C interface
1817 @section Scheme->C interface
1819 Most of the C functions interfacing with Guile/Scheme used in LilyPond
1820 are described in the API Reference of the
1821 @uref{http://www.gnu.org/software/guile/manual/html_node/index.html,
1822 GUILE Reference Manual}.
1824 The remaining functions are defined in @file{lily/lily-guile.cc},
1825 @file{lily/include/lily-guile.hh} and
1826 @file{lily/include/lily-guile-macros.hh}.
1827 Although their names are meaningful there's a few things you should know
1836 @subsection Comparison
1838 This is the trickiest part of the interface.
1840 Mixing Scheme values with C comparison operators won't produce any crash
1841 or warning when compiling but must be avoided:
1844 scm_string_p (scm_value) == SCM_BOOL_T
1847 As we can read in the reference, @code{scm_string_p} returns a Scheme
1848 value: either @code{#t} or @code{#f} which are written @code{SCM_BOOL_T}
1849 and @code{SCM_BOOL_F} in C. This will work, but it is not following
1850 to the API guidelines. For further information, read this discussion:
1853 @uref{http://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00646.html}
1856 There are functions in the Guile reference that returns C values
1857 instead of Scheme values. In our example, a function called
1858 @code{scm_is_string} (described after @code{string?} and @code{scm_string_p})
1859 returns the C value 0 or 1.
1861 So the best solution was simply:
1864 scm_is_string (scm_value)
1867 There a simple solution for almost every common comparison. Another example:
1868 we want to know if a Scheme value is a non-empty list. Instead of:
1871 (scm_is_true (scm_list_p (scm_value)) && scm_value != SCM_EOL)
1874 one can usually use:
1877 scm_is_pair (scm_value)
1880 since a list of at least one member is a pair. This test is
1881 cheap; @code{scm_list_p} is actually quite more complex since it makes
1882 sure that its argument is neither a `dotted list' where the last pair
1883 has a non-null @code{cdr}, nor a circular list. There are few
1884 situations where the complexity of those tests make sense.
1886 Unfortunately, there is not a @code{scm_is_[something]} function for
1887 everything. That's one of the reasons why LilyPond has its own Scheme
1888 interface. As a rule of thumb, tests that are cheap enough to be
1889 worth inlining tend to have such a C interface. So there is
1890 @code{scm_is_pair} but not @code{scm_is_list}, and @code{scm_is_eq}
1891 but not @code{scm_is_equal}.
1893 @subheading General definitions
1895 @subsubheading bool to_boolean (SCM b)
1897 Return @code{true} if @var{b} is @code{SCM_BOOL_T}, else return @code{false}.
1899 This should be used instead of @code{scm_is_true} and
1900 @code{scm_is_false} for properties since in Lilypond, unset properties
1901 are read as an empty list, and by convention unset Boolean properties
1902 default to false. Since both @code{scm_is_true} and
1903 @code{scm_is_false} only compare with @code{##f} in line with what
1904 Scheme's conditionals do, they are not really useful for checking the
1905 state of a Boolean property.
1907 @subsubheading bool ly_is_[something] (args)
1909 Behave the same as scm_is_[something] would do if it existed.
1911 @subsubheading bool is_[type] (SCM s)
1913 Test whether the type of @var{s} is [type].
1914 [type] is a LilyPond-only set of values (direction, axis...). More
1915 often than not, the code checks Lilypond specific C++-implemented
1918 @subsubheading [type *] unsmob_[type] (SCM s)
1920 This tries converting a Scheme object to a pointer of the desired
1921 kind. If the Scheme object is of the wrong type, a pointer value
1922 of@w{ }@code{0} is returned, making this suitable for a Boolean test.
1925 @subsection Conversion
1927 @subheading General definitions
1929 @subsubheading bool to_boolean (SCM b)
1931 Return @code{true} if @var{b} is @code{SCM_BOOL_T}, else return @code{false}.
1933 This should be used instead of @code{scm_is_true} and @code{scm_is_false}
1934 for properties since empty lists are sometimes used to unset them.
1936 @subsubheading [C type] ly_scm2[C type] (SCM s)
1938 Behave the same as scm_to_[C type] would do if it existed.
1940 @subsubheading [C type] robust_scm2[C type] (SCM s, [C type] d)
1942 Behave the same as scm_to_[C type] would do if it existed.
1943 Return @var{d} if type verification fails.
1946 @node LilyPond miscellany
1947 @section LilyPond miscellany
1949 This is a place to dump information that may be of use to developers
1950 but doesn't yet have a proper home. Ideally, the length of this section
1951 would become zero as items are moved to other homes.
1955 * Spacing algorithms::
1956 * Info from Han-Wen email::
1957 * Music functions and GUILE debugging::
1960 @node Spacing algorithms
1961 @subsection Spacing algorithms
1963 Here is information from an email exchange about spacing algorithms.
1965 On Thu, 2010-02-04 at 15:33 -0500, Boris Shingarov wrote:
1966 I am experimenting with some modifications to the line breaking code,
1967 and I am stuck trying to understand how some of it works. So far my
1968 understanding is that Simple_spacer operates on a vector of Grobs, and
1969 it is a well-known Constrained-QP problem (rods = constraints, springs
1970 = quadratic function to minimize). What I don't understand is, if the
1971 spacer operates at the level of Grobs, which are built at an earlier
1972 stage in the pipeline, how are the changes necessitated by differences
1973 in line breaking, taken into account? in other words, if I take the
1974 last measure of a line and place it on the next line, it is not just a
1975 matter of literally moving that graphic to where the start of the next
1976 line is, but I also need to draw a clef, key signature, and possibly
1977 other fundamental things -- but at that stage in the rendering
1978 pipeline, is it not too late??
1980 Joe Neeman answered:
1982 We create lots of extra grobs (eg. a BarNumber at every bar line) but
1983 most of them are not drawn. See the break-visibility property in
1986 Here is another e-mail exchange. Janek Warchoł asked for a starting point
1987 to fixing 1301 (change clef colliding with notes). Neil Puttock replied:
1989 The clef is on a loose column (it floats before the head), so the
1990 first place I'd look would be lily/spacing-loose-columns.cc (and
1991 possibly lily/spacing-determine-loose-columns.cc).
1992 I'd guess the problem is the way loose columns are spaced between
1993 other columns: in this snippet, the columns for the quaver and tuplet
1994 minim are so close together that the clef's column gets dumped on top
1995 of the quaver (since it's loose, it doesn't influence the spacing).
1997 @node Info from Han-Wen email
1998 @subsection Info from Han-Wen email
2000 In 2004, Douglas Linhardt decided to try starting a document that would
2001 explain LilyPond architecture and design principles. The material below
2002 is extracted from that email, which can be found at
2003 @uref{http://thread.gmane.org/gmane.comp.gnu.lilypond.devel/2992}.
2004 The headings reflect questions from Doug or comments from Han-Wen;
2005 the body text are Han-Wen's answers.
2007 @subheading Figuring out how things work.
2009 I must admit that when I want to know how a program works, I use grep
2010 and emacs and dive into the source code. The comments and the code
2011 itself are usually more revealing than technical documents.
2013 @subheading What's a grob, and how is one used?
2015 Graphical object - they are created from within engravers, either as
2016 Spanners (derived class) -slurs, beams- or Items (also a derived
2017 class) -notes, clefs, etc.
2019 There are two other derived classes System (derived from Spanner,
2020 containing a "line of music") and Paper_column (derived from Item, it
2021 contains all items that happen at the same moment). They are separate
2022 classes because they play a special role in the linebreaking process.
2024 @subheading What's a smob, and how is one used?
2026 A C(++) object that is encapsulated so it can be used as a Scheme
2027 object. See GUILE info, "19.3 Defining New Types (Smobs)"
2029 @@subheading When is each C++ class constructed and used
2036 In the parser.yy see the macro calls MAKE_MUSIC_BY_NAME().
2041 Constructed during "interpreting" phase.
2046 Executive branch of Contexts, plugins that create grobs, usually one
2047 engraver per grob type. Created together with context.
2057 These are not C++ classes per se. The idea of a Grob interface hasn't
2058 crystallized well. ATM, an interface is a symbol, with a bunch of grob
2059 properties. They are not objects that are created or destroyed.
2064 Objects that walk through different music classes, and deliver events
2065 in a synchronized way, so that notes that play together are processed
2066 at the same moment and (as a result) end up on the same horizontal position.
2068 Created during interpreting phase.
2070 BTW, the entry point for interpreting is ly:run-translator
2071 (ly_run_translator on the C++ side)
2075 @subheading Can you get to Context properties from a Music object?
2077 You can create music object with a Scheme function that reads context
2078 properties (the \applycontext syntax). However, that function is
2079 executed during Interpreting, so you can not really get Context
2080 properties from Music objects, since music objects are not directly
2081 connected to Contexts. That connection is made by the Music_iterators
2083 @subheading Can you get to Music properties from a Context object?
2085 Yes, if you are given the music object within a Context
2086 object. Normally, the music objects enter Contexts in synchronized
2087 fashion, and the synchronization is done by Music_iterators.
2089 @subheading What is the relationship between C++ classes and Scheme objects?
2091 Smobs are C++ objects in Scheme. Scheme objects (lists, functions) are
2092 manipulated from C++ as well using the GUILE C function interface
2095 @subheading How do Scheme procedures get called from C++ functions?
2097 scm_call_*, where * is an integer from 0 to 4.
2098 Also scm_c_eval_string (), scm_eval ()
2100 @subheading How do C++ functions get called from Scheme procedures?
2102 Export a C++ function to Scheme with LY_DEFINE.
2104 @subheading What is the flow of control in the program?
2106 Good question. Things used to be clear-cut, but we have Scheme
2107 and SMOBs now, which means that interactions do not follow a very
2108 rigid format anymore. See below for an overview, though.
2110 @subheading Does the parser make Scheme procedure calls or C++ function calls?
2112 Both. And the Scheme calls can call C++ and vice versa. It's nested,
2113 with the SCM datatype as lubrication between the interactions
2115 (I think the word "lubrication" describes the process better than the
2116 traditional word "glue")
2118 @subheading How do the front-end and back-end get started?
2120 Front-end: a file is parsed, the rest follows from that. Specifically,
2122 Parsing leads to a Music + Music_output_def object (see parser.yy,
2123 definition of toplevel_expression )
2125 A Music + Music_output_def object leads to a Global_context object (see
2126 ly_run_translator ())
2128 During interpreting, Global_context + Music leads to a bunch of
2129 Contexts (see Global_translator::run_iterator_on_me ()).
2131 After interpreting, Global_context contains a Score_context (which
2132 contains staves, lyrics etc.) as a child. Score_context::get_output ()
2133 spews a Music_output object (either a Paper_score object for notation
2134 or Performance object for MIDI).
2136 The Music_output object is the entry point for the backend (see
2137 ly_render_output ()).
2139 The main steps of the backend itself are in
2144 @file{paper-score.cc} , Paper_score::process_
2147 @file{system.cc} , System::get_lines()
2150 The step, where things go from grobs to output, is in
2151 System::get_line(): each grob delivers a Stencil (a Device
2152 independent output description), which is interpreted by our
2153 outputting backends (@file{scm/output-tex.scm} and
2154 @file{scm/output-ps.scm}) to produce TeX and PS.
2158 Interactions between grobs and putting things into .tex and .ps files
2159 have gotten a little more complex lately. Jan has implemented
2160 page-breaking, so now the backend also involves Paper_book,
2161 Paper_lines and other things. This area is still heavily in flux, and
2162 perhaps not something you should want to look at.
2164 @subheading How do the front-end and back-end communicate?
2166 There is no communication from backend to front-end. From front-end to
2167 backend is simply the program flow: music + definitions gives
2168 contexts, contexts yield output, after processing, output is written
2171 @subheading Where is the functionality associated with KEYWORDs?
2173 See @file{my-lily-lexer.cc} (keywords, there aren't that many)
2174 and @file{ly/*.ly} (most of the other backslashed @code{/\words} are identifiers)
2176 @subheading What Contexts/Properties/Music/etc. are available when they are processed?
2178 What do you mean exactly with this question?
2180 See @file{ly/engraver-init.ly} for contexts,
2181 see @file{scm/define-*.scm} for other objects.
2183 @subheading How do you decide if something is a Music, Context, or Grob property?
2184 Why is part-combine-status a Music property when it seems (IMO)
2185 to be related to the Staff context?
2187 The Music_iterators and Context communicate through two channels
2189 Music_iterators can set and read context properties, idem for
2190 Engravers and Contexts
2192 Music_iterators can send "synthetic" music events (which aren't in
2193 the input) to a context. These are caught by Engravers. This is
2194 mostly a one way communication channel.
2196 part-combine-status is part of such a synthetic event, used by
2197 Part_combine_iterator to communicate with Part_combine_engraver.
2200 @subheading Deciding between context and music properties
2202 I'm adding a property to affect how \autochange works. It seems to
2203 me that it should be a context property, but the Scheme autochange
2204 procedure has a Music argument. Does this mean I should use
2207 \autochange is one of these extra strange beasts: it requires
2208 look-ahead to decide when to change staves. This is achieved by
2209 running the interpreting step twice (see
2210 @file{scm/part-combiner.scm} , at the bottom), and
2211 storing the result of the first step (where to switch
2212 staves) in a Music property. Since you want to influence that
2213 where-to-switch list, your must affect the code in
2214 make-autochange-music (@file{scm/part-combiner.scm}).
2215 That code is called directly from the parser and there are no
2216 official "parsing properties" yet, so there is no generic way
2217 to tune \autochange. We would have to invent something new
2218 for this, or add a separate argument,
2221 \autochange #around-central-C ..music..
2225 where around-central-C is some function that is called from
2226 make-autochange-music.
2228 @subheading More on context and music properties
2230 From Neil Puttock, in response to a question about transposition:
2232 Context properties (using \set & \unset) are tied to engravers: they
2233 provide information relevant to the generation of graphical objects.
2235 Since transposition occurs at the music interpretation stage, it has
2236 no direct connection with engravers: the pitch of a note is fixed
2237 before a notehead is created. Consider the following minimal snippet:
2243 This generates (simplified) a NoteEvent, with its pitch and duration
2244 as event properties,
2250 (ly:make-duration 2 0 1 1)
2252 (ly:make-pitch 0 0 0)
2255 which the Note_heads_engraver hears. It passes this information on to
2256 the NoteHead grob it creates from the event, so the head's correct
2257 position and duration-log can be determined once it's ready for
2260 If we transpose the snippet,
2263 \transpose c d @{ c' @}
2266 the pitch is changed before it reaches the engraver (in fact, it
2267 happens just after the parsing stage with the creation of a
2268 TransposedMusic music object):
2274 (ly:make-duration 2 0 1 1)
2276 (ly:make-pitch 0 1 0)
2279 You can see an example of a music property relevant to transposition:
2283 \transpose c d @{ c'2 \withMusicProperty #'untransposable ##t c' @}
2286 -> the second c' remains untransposed.
2288 Take a look at @file{lily/music.cc} to see where the transposition takes place.
2291 @subheading How do I tell about the execution environment?
2293 I get lost figuring out what environment the code I'm looking at is in when it
2294 executes. I found both the C++ and Scheme autochange code. Then I was trying
2295 to figure out where the code got called from. I finally figured out that the
2296 Scheme procedure was called before the C++ iterator code, but it took me a
2297 while to figure that out, and I still didn't know who did the calling in the
2298 first place. I only know a little bit about Flex and Bison, so reading those
2299 files helped only a little bit.
2301 @emph{Han-Wen:} GDB can be of help here. Set a breakpoint in C++, and run. When you
2302 hit the breakpoint, do a backtrace. You can inspect Scheme objects
2303 along the way by doing
2306 p ly_display_scm(obj)
2309 this will display OBJ through GUILE.
2311 @node Music functions and GUILE debugging
2312 @subsection Music functions and GUILE debugging
2314 Ian Hulin was trying to do some debugging in music functions, and
2315 came up with the following question
2318 I'm working on the Guile Debugger Stuff, and would like to try
2319 debugging a music function definition such as:
2322 conditionalMark = #(define-music-function (parser location) ()
2323 #@{ \tag #'instrumental-part @{\mark \default@} #@} )
2326 It appears conditionalMark does not get set up as an
2327 equivalent of a Scheme
2330 (define conditionalMark = define-music-function(parser location () ...
2334 although something gets defined because Scheme apparently recognizes
2337 #(set-break! conditionalMark)
2341 later on in the file without signalling any Guile errors.
2343 However the breakpoint trap is never encountered as
2344 define-music-function passed things on to ly:make-music-function,
2345 which is really C++ code ly_make_music_function, so Guile never
2346 finds out about the breakpoint.
2348 Han-Wen answered as follows:
2350 You can see the definition by doing
2353 #(display conditionalMark)
2357 inside the @file{.ly} file.
2359 The breakpoint failing may have to do with the call sequence. See
2360 @file{parser.yy}, run_music_function(). The function is called directly from
2361 C++, without going through the GUILE evaluator, so I think that is why
2362 there is no debugger trap.