--- /dev/null
+@c -*- coding: utf-8; mode: texinfo; -*-
+@node Programming work
+@chapter Programming work
+
+@menu
+* Overview of LilyPond architecture::
+* LilyPond programming languages::
+* Programming without compiling::
+* Finding functions::
+* Code style::
+* Debugging LilyPond::
+* Tracing object relationships::
+* Adding or modifying features::
+* Iterator tutorial::
+* Engraver tutorial::
+* Callback tutorial::
+* LilyPond scoping::
+* LilyPond miscellany::
+@end menu
+
+@node Overview of LilyPond architecture
+@section Overview of LilyPond architecture
+
+LilyPond processes the input file into graphical and musical output in a
+number of stages. This process, along with the types of routines that
+accomplish the various stages of the process, is described in this section. A
+more complete description of the LilyPond architecture and internal program
+execution is found in Erik Sandberg's
+@uref{http://lilypond.org/web/images/thesis-erik-sandberg.pdf, master's
+thesis}.
+
+The first stage of LilyPond processing is @emph{parsing}. In the parsing
+process, music expressions in LilyPond input format are converted to music
+expressions in Scheme format. In Scheme format, a music expression is a list
+in tree form, with nodes that indicate the relationships between various music
+events. The LilyPond parser is written in Bison.
+
+The second stage of LilyPond processing is @emph{iterating}. Iterating
+assigns each music event to a context, which is the environment in which the
+music will be finally engraved. The context is responsible for all further
+processing of the music. It is during the iteration stage that contexts are
+created as necessary to ensure that every note has a Voice type context (e.g.
+Voice, TabVoice, DrumVoice, CueVoice, MensuralVoice, VaticanaVoice,
+GregorianTranscriptionVoice), that the Voice type contexts exist in
+appropriate Staff type contexts, and that parallel Staff type contexts exist
+in StaffGroup type contexts. In addition, during the iteration stage each
+music event is assigned a moment, or a time in the music when the event
+begins.
+
+Each type of music event has an associated iterator. Iterators are defined in
+@file{*-iterator.cc}. During iteration, an
+event's iterator is called to deliver that music event to the appropriate
+context(s).
+
+The final stage of LilyPond processing is @emph{translation}. During
+translation, music events are prepared for graphical or midi output. The
+translation step is accomplished by the polymorphic base class Translator
+through its two derived classes: Engraver (for graphical output) and
+Performer (for midi output).
+
+Translators are defined in C++ files named @file{*-engraver.cc}
+and @file{*-performer.cc}.
+Much of the work of translating is handled by Scheme functions,
+which is one of the keys to LilyPond's exceptional flexibility.
+
+@sourceimage{architecture-diagram,,,png}
+
+
+@node LilyPond programming languages
+@section LilyPond programming languages
+
+Programming in LilyPond is done in a variety of programming languages. Each
+language is used for a specific purpose or purposes. This section describes
+the languages used and provides links to reference manuals and tutorials for
+the relevant language.
+
+@subsection C++
+
+The core functionality of LilyPond is implemented in C++.
+
+C++ is so ubiquitous that it is difficult to identify either a reference
+manual or a tutorial. Programmers unfamiliar with C++ will need to spend some
+time to learn the language before attempting to modify the C++ code.
+
+The C++ code calls Scheme/GUILE through the GUILE interface, which is
+documented in the
+@uref{http://www.gnu.org/software/guile/manual/html_node/index.html, GUILE
+ Reference Manual}.
+
+@subsection Flex
+
+The LilyPond lexer is implemented in Flex, an implementation of the Unix lex
+lexical analyser generator. Resources for Flex can be found
+@uref{http://flex.sourceforge.net/, here}.
+
+@subsection GNU Bison
+
+The LilyPond parser is implemented in Bison, a GNU parser generator. The
+Bison homepage is found at @uref{http://www.gnu.org/software/bison/,
+gnu.org}. The manual (which includes both a reference and tutorial) is
+@uref{http://www.gnu.org/software/bison/manual/index.html, available} in a
+variety of formats.
+
+@subsection GNU Make
+
+GNU Make is used to control the compiling process and to build the
+documentation and the website. GNU Make documentation is available at
+@uref{http://www.gnu.org/software/make/manual/, the GNU website}.
+
+@subsection GUILE or Scheme
+
+GUILE is the dialect of Scheme that is used as LilyPond's extension language.
+Many extensions to LilyPond are written entirely in GUILE. The
+@uref{http://www.gnu.org/software/guile/manual/html_node/index.html,
+GUILE Reference Manual} is available online.
+
+@uref{http://mitpress.mit.edu/sicp/full-text/book/book.html, Structure and
+Interpretation of Computer Programs}, a popular textbook used to teach
+programming in Scheme is available in its entirety online.
+
+An introduction to Guile/Scheme as used in LilyPond can be found in the
+@rextend{Scheme tutorial}.
+
+@subsection MetaFont
+
+MetaFont is used to create the music fonts used by LilyPond. A MetaFont
+tutorial is available at @uref{http://metafont.tutorial.free.fr/, the
+METAFONT tutorial page}.
+
+@subsection PostScript
+
+PostScript is used to generate graphical output. A brief PostScript tutorial
+is @uref{http://local.wasp.uwa.edu.au/~pbourke/dataformats/postscript/,
+available online}. The
+@uref{http://www.adobe.com/devnet/postscript/pdfs/PLRM.pdf, PostScript Language
+Reference} is available online in PDF format.
+
+@subsection Python
+
+Python is used for XML2ly and is used for building the documentation and the
+website.
+
+Python documentation is available at @uref{http://www.python.org/doc/,
+python.org}.
+
+@node Programming without compiling
+@section Programming without compiling
+
+Much of the development work in LilyPond takes place by changing @file{*.ly} or
+@file{*.scm} files. These changes can be made without compiling LilyPond. Such
+changes are described in this section.
+
+
+@subsection Modifying distribution files
+
+Much of LilyPond is written in Scheme or LilyPond input files. These
+files are interpreted when the program is run, rather than being compiled
+when the program is built, and are present in all LilyPond distributions.
+You will find @file{.ly} files in the @file{ly/} directory and the Scheme files in the
+@file{scm/} directory. Both Scheme files and @file{.ly} files can be modified and
+saved with any text editor. It's probably wise to make a backup copy of
+your files before you modify them, although you can reinstall if the
+files become corrupted.
+
+Once you've modified the files, you can test the changes just by running
+LilyPond on some input file. It's a good idea to create a file that
+demonstrates the feature you're trying to add. This file will eventually
+become a regression test and will be part of the LilyPond distribution.
+
+@subsection Desired file formatting
+
+Files that are part of the LilyPond distribution have Unix-style line
+endings (LF), rather than DOS (CR+LF) or MacOS 9 and earlier (CR). Make
+sure you use the necessary tools to ensure that Unix-style line endings are
+preserved in the patches you create.
+
+Tab characters should not be included in files for distribution. All
+indentation should be done with spaces. Most editors have settings to
+allow the setting of tab stops and ensuring that no tab characters are
+included in the file.
+
+Scheme files and LilyPond files should be written according to standard
+style guidelines. Scheme file guidelines can be found at
+@uref{http://community.schemewiki.org/?scheme-style}. Following these
+guidelines will make your code easier to read. Both you and others that
+work on your code will be glad you followed these guidelines.
+
+For LilyPond files, you should follow the guidelines for LilyPond snippets
+in the documentation. You can find these guidelines at
+@ref{Texinfo introduction and usage policy}.
+
+@node Finding functions
+@section Finding functions
+
+When making changes or fixing bugs in LilyPond, one of the initial
+challenges is finding out where in the code tree the functions to
+be modified live. With nearly 3000 files in the source tree,
+trial-and-error searching is generally ineffective. This section
+describes a process for finding interesting code.
+
+@subsection Using the ROADMAP
+
+The file ROADMAP is located in the main directory of the lilypond source.
+ROADMAP lists all of the directories in the LilyPond source tree, along
+with a brief description of the kind of files found in each directory.
+This can be a very helpful tool for deciding which directories to search
+when looking for a function.
+
+
+@subsection Using grep to search
+
+Having identified a likely subdirectory to search, the grep utility can
+be used to search for a function name. The format of the grep command is
+
+@example
+grep -i functionName subdirectory/*
+@end example
+
+This command will search all the contents of the directory subdirectory/
+and display every line in any of the files that contains
+functionName. The @code{-i} option makes @command{grep} ignore
+case -- this can be very useful if you are not yet familiar with
+our capitalization conventions.
+
+The most likely directories to grep for function names are @file{scm/} for
+scheme files, ly/ for lilypond input (@file{*.ly}) files, and @file{lily/} for C++
+files.
+
+
+@subsection Using git grep to search
+
+If you have used git to obtain the source, you have access to a
+powerful tool to search for functions. The command:
+
+@example
+git grep functionName
+@end example
+
+will search through all of the files that are present in the git
+repository looking for functionName. It also presents the results
+of the search using @code{less}, so the results are displayed one page
+at a time.
+
+@subsection Searching on the git repository at Savannah
+
+You can also use the equivalent of git grep on the Savannah server.
+
+@itemize
+
+@item
+Go to http://git.sv.gnu.org/gitweb/?p=lilypond.git
+
+@item
+In the pulldown box that says commit, select grep.
+
+@item
+Type functionName in the search box, and hit enter/return
+
+@end itemize
+
+This will initiate a search of the remote git repository.
+
+
+@node Code style
+@section Code style
+
+This section describes style guidelines for LilyPond
+source code.
+
+@menu
+* Languages::
+* Filenames::
+* Indentation::
+* Naming conventions::
+* Broken code::
+* Code comments::
+* Handling errors::
+* Localization::
+@end menu
+
+
+@node Languages
+@subsection Languages
+
+C++ and Python are preferred. Python code should use PEP 8.
+
+
+@node Filenames
+@subsection Filenames
+
+Definitions of classes that are only accessed via pointers (*) or
+references (&) shall not be included as include files.
+
+@verbatim
+ filenames
+
+ ".hh" Include files
+ ".cc" Implementation files
+ ".icc" Inline definition files
+ ".tcc" non inline Template defs
+
+ in emacs:
+
+ (setq auto-mode-alist
+ (append '(("\\.make$" . makefile-mode)
+ ("\\.cc$" . c++-mode)
+ ("\\.icc$" . c++-mode)
+ ("\\.tcc$" . c++-mode)
+ ("\\.hh$" . c++-mode)
+ ("\\.pod$" . text-mode)
+ )
+ auto-mode-alist))
+@end verbatim
+
+The class Class_name is coded in @q{class-name.*}
+
+
+@node Indentation
+@subsection Indentation
+
+Standard GNU coding style is used. In emacs:
+
+@verbatim
+ (add-hook 'c++-mode-hook
+ '(lambda() (c-set-style "gnu")
+ ))
+@end verbatim
+
+If you like using font-lock, you can also add this to your
+@q{.emacs}:
+
+@verbatim
+ (setq font-lock-maximum-decoration t)
+ (setq c++-font-lock-keywords-3
+ (append
+ c++-font-lock-keywords-3
+ '(("\\b\\(a-zA-Z_?+_\\)\\b" 1 font-lock-variable-name-face) ("\\b\\(A-Z?+a-z_?+\\)\\b" 1 font-lock-type-face))
+ ))
+@end verbatim
+
+Some source files may not currently have proper indenting. If this
+is the case, it is desirable to fix the improper indenting when the
+file is modified, with the hope of continually improving the code.
+
+
+@subheading Indenting files with fixcc.py
+
+LilyPond provides a python script that will correct the indentation
+on a c++ file:
+
+@example
+scripts/auxiliar/fixcc.py lily/my-test-file.cc
+@end example
+
+Be sure you replace @file{my-test-file.cc} with the name of the file
+that you edited.
+
+If you are editing a file that contains an ADD_TRANSLATOR or ADD_INTERFACE
+macro, the fixcc.py script will move the final parenthesis up one line
+from where it should be. Please check the end of the file before
+you run fixcc.py, and then put the final parenthesis and semicolon
+back on a line by themselves.
+
+
+@subheading Indenting files with emacs in script mode
+
+@c email to wl@gnu.org when I get here.
+
+@warning{this is pending some confirmation on -devel. July 2009 -gp}
+
+Command-line script to format stuff with emacs:
+
+@example
+#!/bin/sh
+emacs $1 -batch --eval '(indent-region (point-min) (point-max) nil)' -f save-buffer
+@end example
+
+(that's all on one line)
+
+Save it as a shell script, then run on the file(s) you modified.
+
+
+@subheading Indenting with vim
+
+Although emacs indentation is the LilyPond standard, acceptable
+indentation can usually be accomplished with vim. Some hints for
+vim are as follows:
+
+A workable .vimrc:
+
+@verbatim
+set cindent
+set smartindent
+set autoindent
+set expandtab
+set softtabstop=2
+set shiftwidth=2
+filetype plugin indent on
+set incsearch
+set ignorecase smartcase
+set hlsearch
+set confirm
+set statusline=%F%m%r%h%w\ %{&ff}\ %Y\ [ASCII=\%03.3b]\ [HEX=\%02.2B]\ %04l,%04v\ %p%%\ [LEN=%L]
+set laststatus=2
+set number
+" Remove trailing whitespace on write
+autocmd BufWritePre * :%s/\s\+$//e
+@end verbatim
+
+With this .vimrc, files can be reindented automatically by highlighting
+the lines to be indented in visual mode (use V to enter visual mode)
+and pressing =.
+
+A scheme.vim file will help improve the indentation. This one
+was suggested by Patrick McCarty. It should be saved in
+~/.vim/after/syntax/scheme.vim.
+
+@verbatim
+" Additional Guile-specific 'forms'
+syn keyword schemeSyntax define-public define*-public
+syn keyword schemeSyntax define* lambda* let-keywords*
+syn keyword schemeSyntax defmacro defmacro* define-macro
+syn keyword schemeSyntax defmacro-public defmacro*-public
+syn keyword schemeSyntax use-modules define-module
+syn keyword schemeSyntax define-method define-class
+
+" Additional LilyPond-specific 'forms'
+syn keyword schemeSyntax define-markup-command define-markup-list-command
+syn keyword schemeSyntax define-safe-public define-music-function
+syn keyword schemeSyntax def-grace-function
+
+" All of the above should influence indenting too
+set lw+=define-public,define*-public
+set lw+=define*,lambda*,let-keywords*
+set lw+=defmacro,defmacro*,define-macro
+set lw+=defmacro-public,defmacro*-public
+set lw+=use-modules,define-module
+set lw+=define-method,define-class
+set lw+=define-markup-command,define-markup-list-command
+set lw+=define-safe-public,define-music-function
+set lw+=def-grace-function
+
+" These forms should not influence indenting
+set lw-=if
+set lw-=set!
+
+" Try to highlight all ly: procedures
+syn match schemeFunc "ly:[^) ]\+"
+@end verbatim
+
+
+@node Naming conventions
+@subsection Naming Conventions
+
+Naming conventions have been established for LilyPond
+source code.
+
+@subheading Classes and Types
+
+Classes begin with an uppercase letter, and words
+in class names are separated with @code{_}:
+
+@verbatim
+This_is_a_class
+@end verbatim
+
+@subheading Members
+
+Member variable names end with an underscore:
+
+@verbatim
+Type Class::member_
+@end verbatim
+
+@subheading Macros
+
+Macro names should be written in uppercase completely,
+with words separated by @code{_}:
+
+@verbatim
+THIS_IS_A_MACRO
+@end verbatim
+
+@subheading Variables
+
+Variable names should be complete words, rather than abbreviations.
+For example, it is preferred to use @code{thickness} rather than
+@code{th} or @code{t}.
+
+Multi-word variable names in C++ should have the words separated
+by the underscore character (@q{_}):
+
+@verbatim
+cxx_multiword_variable
+@end verbatim
+
+Multi-word variable names in Scheme should have the words separated
+by a hyphen (@q{-}):
+
+@verbatim
+scheme-multiword-variable
+@end verbatim
+
+@node Broken code
+@subsection Broken code
+
+Do not write broken code. This includes hardwired dependencies,
+hardwired constants, slow algorithms and obvious limitations. If
+you can not avoid it, mark the place clearly, and add a comment
+explaining shortcomings of the code.
+
+Ideally, the comment marking the shortcoming would include
+TODO, so that it is marked for future fixing.
+
+We reject broken-in-advance on principle.
+
+
+@node Code comments
+@subsection Code comments
+
+Comments may not be needed if descriptive variable names are used
+in the code and the logic is straightforward. However, if the
+logic is difficult to follow, and particularly if non-obvious
+code has been included to resolve a bug, a comment describing
+the logic and/or the need for the non-obvious code should be included.
+
+There are instances where the current code could be commented better.
+If significant time is required to understand the code as part of
+preparing a patch, it would be wise to add comments reflecting your
+understanding to make future work easier.
+
+
+@node Handling errors
+@subsection Handling errors
+
+As a general rule, you should always try to continue computations,
+even if there is some kind of error. When the program stops, it
+is often very hard for a user to pinpoint what part of the input
+causes an error. Finding the culprit is much easier if there is
+some viewable output.
+
+So functions and methods do not return errorcodes, they never
+crash, but report a programming_error and try to carry on.
+
+Error and warning messages need to be localized.
+
+
+@node Localization
+@subsection Localization
+
+This document provides some guidelines to help programmers write
+proper user
+messages. To help translations, user messages must follow
+uniform conventions. Follow these rules when coding for LilyPond.
+Hopefully, this can be replaced by general GNU guidelines in the
+future. Even better would be to have an English (en_BR, en_AM)
+guide helping programmers writing consistent messages for all GNU
+programs.
+
+Non-preferred messages are marked with `+'. By convention,
+ungrammatical examples are marked with `*'. However, such ungrammatical
+examples may still be preferred.
+
+@itemize
+
+@item
+Every message to the user should be localized (and thus be marked
+for localization). This includes warning and error messages.
+
+@item
+Do not localize/gettextify:
+
+@itemize
+@item
+`programming_error ()'s
+
+@item
+`programming_warning ()'s
+
+@item
+debug strings
+
+@item
+output strings (PostScript, TeX, etc.)
+
+@end itemize
+
+@item
+Messages to be localized must be encapsulated in `_ (STRING)' or
+`_f (FORMAT, ...)'. E.g.:
+
+@example
+warning (_ ("need music in a score"));
+error (_f ("cannot open file: `%s'", file_name));
+@end example
+
+In some rare cases you may need to call `gettext ()' by hand. This
+happens when you pre-define (a list of) string constants for later
+use. In that case, you'll probably also need to mark these string
+constants for translation, using `_i (STRING)'. The `_i' macro is
+a no-op, it only serves as a marker for `xgettext'.
+
+@example
+char const* messages[] = @{
+ _i ("enable debugging output"),
+ _i ("ignore lilypond version"),
+ 0
+@};
+
+void
+foo (int i)
+@{
+ puts (gettext (messages i));
+@}
+@end example
+
+See also @file{flower/getopt-long.cc} and @file{lily/main.cc}.
+
+@item
+Do not use leading or trailing whitespace in messages. If you need
+whitespace to be printed, prepend or append it to the translated
+message
+
+@example
+message ("Calculating line breaks..." + " ");
+@end example
+
+@item
+Error or warning messages displayed with a file name and line
+number never start with a capital, eg,
+
+@example
+foo.ly: 12: not a duration: 3
+@end example
+
+Messages containing a final verb, or a gerund (`-ing'-form) always
+start with a capital. Other (simpler) messages start with a
+lowercase letter
+
+@example
+Processing foo.ly...
+`foo': not declared.
+Not declaring: `foo'.
+@end example
+
+@item
+Avoid abbreviations or short forms, use `cannot' and `do not'
+rather than `can't' or `don't'
+To avoid having a number of different messages for the same
+situation, well will use quoting like this `"message: `%s'"' for all
+strings. Numbers are not quoted:
+
+@example
+_f ("cannot open file: `%s'", name_str)
+_f ("cannot find character number: %d", i)
+@end example
+
+@item
+Think about translation issues. In a lot of cases, it is better to
+translate a whole message. English grammar must not be imposed on the
+translator. So, instead of
+
+@example
+stem at + moment.str () + does not fit in beam
+@end example
+
+have
+
+@example
+_f ("stem at %s does not fit in beam", moment.str ())
+@end example
+
+@item
+Split up multi-sentence messages, whenever possible. Instead of
+
+@example
+warning (_f ("out of tune! Can't find: `%s'", "Key_engraver"));
+warning (_f ("cannot find font `%s', loading default", font_name));
+@end example
+
+rather say:
+
+@example
+warning (_ ("out of tune:"));
+warning (_f ("cannot find: `%s', "Key_engraver"));
+warning (_f ("cannot find font: `%s', font_name));
+warning (_f ("Loading default font"));
+@end example
+
+@item
+If you must have multiple-sentence messages, use full punctuation.
+Use two spaces after end of sentence punctuation. No punctuation
+(esp. period) is used at the end of simple messages.
+
+@example
+_f ("Non-matching braces in text `%s', adding braces", text)
+_ ("Debug output disabled. Compiled with NPRINT.")
+_f ("Huh? Not a Request: `%s'. Ignoring.", request)
+@end example
+
+@item
+Do not modularize too much; words frequently cannot be translated
+without context. It is probably safe to treat most occurrences of
+words like stem, beam, crescendo as separately translatable words.
+
+@item
+When translating, it is preferable to put interesting information
+at the end of the message, rather than embedded in the middle.
+This especially applies to frequently used messages, even if this
+would mean sacrificing a bit of eloquency. This holds for original
+messages too, of course.
+
+@example
+en: cannot open: `foo.ly'
++ nl: kan `foo.ly' niet openen (1)
+kan niet openen: `foo.ly'* (2)
+niet te openen: `foo.ly'* (3)
+@end example
+
+
+The first nl message, although grammatically and stylistically
+correct, is not friendly for parsing by humans (even if they speak
+dutch). I guess we would prefer something like (2) or (3).
+
+@item
+Do not run make po/po-update with GNU gettext < 0.10.35
+
+@end itemize
+
+
+
+@node Debugging LilyPond
+@section Debugging LilyPond
+
+The most commonly used tool for debugging LilyPond is the GNU
+debugger gdb. The gdb tool is used for investigating and debugging
+core Lilypond code written in C++. Another tool is available for
+debugging Scheme code using the Guile debugger. This section
+describes how to use both gdb and the Guile Debugger.
+
+@menu
+* Debugging overview::
+* Debugging C++ code::
+* Debugging Scheme code::
+@end menu
+
+@node Debugging overview
+@subsection Debugging overview
+
+Using a debugger simplifies troubleshooting in at least two ways.
+
+First, breakpoints can be set to pause execution at any desired point.
+Then, when execution has paused, debugger commands can be issued to
+explore the values of various variables or to execute functions.
+
+Second, the debugger can display a stack trace, which shows the
+sequence in which functions have been called and the arguments
+passed to the called functions.
+
+@node Debugging C++ code
+@subsection Debugging C++ code
+
+The GNU debugger, gdb, is the principal tool for debugging C++ code.
+
+@subheading Compiling LilyPond for use with gdb
+
+In order to use gdb with LilyPond, it is necessary to compile
+LilyPond with debugging information. This is accomplished by running
+the following commands in the main LilyPond source directory.
+
+@example
+./configure --disable-optimising
+make
+@end example
+
+This will create a version of LilyPond containing debugging
+information that will allow the debugger to tie the source code
+to the compiled code.
+
+You should not do @var{make install} if you want to use a debugger
+with LilyPond. The @var{make install} command will strip debugging
+information from the LilyPond binary.
+
+@subheading Typical gdb usage
+
+Once you have compiled the Lilypond image with the necessary
+debugging information it will have been written to a location in a
+subfolder of your current working directory:
+
+@example
+out/bin/lilypond
+@end example
+
+This is important as you will need to let gdb know where to find the
+image containing the symbol tables. You can invoke gdb from the
+command line using the following:
+
+@example
+gdb out/bin/lilypond
+@end example
+@noindent
+This loads the LilyPond symbol tables into gdb. Then, to run
+LilyPond on @file{test.ly} under the debugger, enter the following:
+
+@example
+run test.ly
+@end example
+
+@noindent
+at the gdb prompt.
+
+As an alternative to running gdb at the command line you may try
+a graphical interface to gdb such as ddd:
+
+@example
+ddd out/bin/lilypond
+@end example
+
+You can also use sets of standard gdb commands stored in a .gdbinit
+file (see next section).
+
+@subheading Typical .gdbinit files
+
+The behavior of gdb can be readily customized through the use of a
+@var{.gdbinit} file. A @var{.gdbinit} file is a file named
+@var{.gdbinit} (notice the @qq{.} at the beginning of the file name)
+that is placed in a user's home directory.
+
+The @var{.gdbinit} file below is from Han-Wen. It sets breakpoints
+for all errors and defines functions for displaying scheme objects
+(ps), grobs (pgrob), and parsed music expressions (pmusic).
+
+@example
+file lily/out/lilypond
+b programming_error
+b Grob::programming_error
+
+define ps
+ print ly_display_scm($arg0)
+end
+define pgrob
+ print ly_display_scm($arg0->self_scm_)
+ print ly_display_scm($arg0->mutable_property_alist_)
+ print ly_display_scm($arg0->immutable_property_alist_)
+ print ly_display_scm($arg0->object_alist_)
+end
+define pmusic
+ print ly_display_scm($arg0->self_scm_)
+ print ly_display_scm($arg0->mutable_property_alist_)
+ print ly_display_scm($arg0->immutable_property_alist_)
+end
+@end example
+
+@node Debugging Scheme code
+@subsection Debugging Scheme code
+
+Scheme code can be developed using the Guile command line
+interpreter @code{top-repl}. You can either investigate
+interactively using just Guile or you can use the debugging
+tools available within Guile.
+
+@subheading Using Guile interactively with LilyPond
+
+In order to experiment with Scheme programming in the LilyPond
+environment, it is necessary to have a Guile interpreter that
+has all the LilyPond modules loaded. This requires the following
+steps.
+
+First, define a Scheme symbol for the active module in the @file{.ly} file:
+
+@example
+#(module-define! (resolve-module '(guile-user))
+ 'lilypond-module (current-module))
+@end example
+
+Now place a Scheme function in the @file{.ly} file that gives an
+interactive Guile prompt:
+
+@example
+#(top-repl)
+@end example
+
+When the @file{.ly} file is compiled, this causes the compilation to be
+interrupted and an interactive guile prompt to appear. Once the
+guile prompt appears, the LilyPond active module must be set as the
+current guile module:
+
+@example
+guile> (set-current-module lilypond-module)
+@end example
+
+You can demonstrate these commands are operating properly by typing the name
+of a LilyPond public scheme function to check it has been defined:
+
+@example
+guile> fret-diagram-verbose-markup
+#<procedure fret-diagram-verbose-markup (layout props marking-list)>
+@end example
+
+If the LilyPond module has not been correctly loaded, an error
+message will be generated:
+
+@example
+guile> fret-diagram-verbose-markup
+ERROR: Unbound variable: fret-diagram-verbose-markup
+ABORT: (unbound-variable)
+@end example
+
+Once the module is properly loaded, any valid LilyPond Scheme
+expression can be entered at the interactive prompt.
+
+After the investigation is complete, the interactive guile
+interpreter can be exited:
+
+@example
+guile> (quit)
+@end example
+
+The compilation of the @file{.ly} file will then continue.
+
+@subheading Using the Guile debugger
+
+To set breakpoints and/or enable tracing in Scheme functions, put
+
+@example
+\include "guile-debugger.ly"
+@end example
+
+in your input file after any scheme procedures you have defined in
+that file. This will invoke the Guile command-line after having set
+up the environment for the debug command-line. When your input file
+is processed, a guile prompt will be displayed. You may now enter
+commands to set up breakpoints and enable tracing by the Guile debugger.
+
+@subheading Using breakpoints
+
+At the guile prompt, you can set breakpoints with
+the @code{set-break!} procedure:
+
+@example
+guile> (set-break! my-scheme-procedure)
+@end example
+
+Once you have set the desired breakpoints, you exit the guile repl frame
+by typing:
+
+@example
+guile> (quit)
+@end example
+
+Then, when one of the scheme routines for which you have set
+breakpoints is entered, guile will interrupt execution in a debug
+frame. At this point you will have access to Guile debugging
+commands. For a listing of these commands, type:
+
+@example
+debug> help
+@end example
+
+Alternatively you may code the breakpoints in your Lilypond source
+file using a command such as:
+
+@example
+#(set-break! my-scheme-procedure)
+@end example
+
+immediately after the @code{\include} statement. In this case the
+breakpoint will be set straight after you enter the @code{(quit)}
+command at the guile prompt.
+
+Embedding breakpoint commands like this is particularly useful if
+you want to look at how the Scheme procedures in the @file{.scm}
+files supplied with LilyPond work. To do this, edit the file in
+the relevant directory to add this line near the top:
+
+@example
+(use-modules (scm guile-debugger))
+@end example
+
+Now you can set a breakpoint after the procedure you are interested
+in has been declared. For example, if you are working on routines
+called by @var{print-book-with} in @file{lily-library.scm}:
+
+@example
+(define (print-book-with parser book process-procedure)
+ (let* ((paper (ly:parser-lookup parser '$defaultpaper))
+ (layout (ly:parser-lookup parser '$defaultlayout))
+ (outfile-name (get-outfile-name parser)))
+ (process-procedure book paper layout outfile-name)))
+
+(define-public (print-book-with-defaults parser book)
+ (print-book-with parser book ly:book-process))
+
+(define-public (print-book-with-defaults-as-systems parser book)
+ (print-book-with parser book ly:book-process-to-systems))
+
+@end example
+
+At this point in the code you could add this to set a breakpoint at
+print-book-with:
+
+@example
+(set-break! print-book-with)
+@end example
+
+@subheading Tracing procedure calls and evaluator steps
+
+Two forms of trace are available:
+
+@example
+(set-trace-call! my-scheme-procedure)
+@end example
+
+and
+
+@example
+(set-trace-subtree! my-scheme-procedure)
+@end example
+
+@code{set-trace-call!} causes Scheme to log a line to the standard
+output to show when the procedure is called and when it exits.
+
+@code{set-trace-subtree!} traces every step the Scheme evaluator
+performs in evaluating the procedure.
+
+@node Tracing object relationships
+@section Tracing object relationships
+
+Understanding the LilyPond source often boils down to figuring out what
+is happening to the Grobs. Where (and why) are they being created,
+modified and destroyed? Tracing Lily through a debugger in order to
+identify these relationships can be time-consuming and tedious.
+
+In order to simplify this process, a facility has been added to
+display the grobs that are created and the properties that are set
+and modified. Although it can be complex to get set up, once set up
+it easily provides detailed information about the life of grobs
+in the form of a network graph.
+
+Each of the steps necessary to use the graphviz utility
+is described below.
+
+@enumerate
+
+@item Installing graphviz
+
+In order to create the graph of the object relationships, it is
+first necessary to install Graphviz. graphviz is available for a
+number of different platforms:
+
+@example
+@uref{http://www.graphviz.org/Download..php}
+@end example
+
+@item Modifying config.make
+
+In order for the Graphviz tool to work, config.make must be modified.
+It is probably a good idea to first save a copy of config.make under
+a different name. Then, edit config.make by removing every occurrence
+of @code{-DNDEBUG}.
+
+@item Rebuilding LilyPond
+
+The executable code of LilyPond must be rebuilt from scratch:
+
+@example
+make -C lily clean && make -C lily
+@end example
+
+@item Create a graphviz-compatible @file{.ly} file
+
+In order to use the graphviz utility, the @file{.ly} file must include
+@file{ly/graphviz-init.ly}, and should then specify the
+grobs and symbols that should be tracked. An example of this
+is found in @file{input/regression/graphviz.ly}.
+
+@item Run lilypond with output sent to a log file
+
+The Graphviz data is sent to stderr by lilypond, so it is
+necessary to redirect stderr to a logfile:
+
+@example
+lilypond graphviz.ly 2> graphviz.log
+@end example
+
+@item Edit the logfile
+
+The logfile has standard lilypond output, as well as the Graphviz
+output data. Delete everything from the beginning of the file
+up to but not including the first occurrence of @code{digraph}.
+
+Also, delete the final liypond message about successs from the end
+of the file.
+
+@item Process the logfile with @code{dot}
+
+The directed graph is created from the log file with the program
+@code{dot}:
+
+@example
+dot -Tpdf graphviz.log > graphviz.pdf
+@end example
+
+@end enumerate
+
+The pdf file can then be viewed with any pdf viewer.
+
+When compiled without @code{-DNDEBUG}, lilypond may run slower
+than normal. The original configuration can be restored by either
+renaming the saved copy of @code{config.make} or rerunning
+@code{configure}. Then rebuild lilypond with
+
+@example
+make -C lily clean && make -C lily
+@end example
+
+
+@node Adding or modifying features
+@section Adding or modifying features
+
+When a new feature is to be added to LilyPond, it is necessary to
+ensure that the feature is properly integrated to maintain
+its long-term support. This section describes the steps necessary
+for feature addition and modification.
+
+
+@menu
+* Write the code::
+* Write regression tests::
+* Write convert-ly rule::
+* Automatically update documentation::
+* Manually update documentation::
+* Edit changes.tely::
+* Verify successful build::
+* Verify regression tests::
+* Post patch for comments::
+* Push patch::
+* Closing the issues::
+@end menu
+
+@node Write the code
+@subsection Write the code
+
+You should probably create a new git branch for writing the code, as that
+will separate it from the master branch and allow you to continue
+to work on small projects related to master.
+
+Please be sure to follow the rules for programming style discussed
+earlier in this chapter.
+
+
+@node Write regression tests
+@subsection Write regression tests
+
+In order to demonstrate that the code works properly, you will
+need to write one or more regression tests. These tests are
+typically @file{.ly} files that are found in @file{input/regression}.
+
+Regression tests should be as brief as possible to demonstrate the
+functionality of the code.
+
+Regression tests should generally cover one issue per test. Several
+short, single-issue regression tests are preferred to a single, long,
+multiple-issue regression test.
+
+Use existing regression tests as templates to demonstrate the type of
+header information that should be included in a regression test.
+
+
+@node Write convert-ly rule
+@subsection Write convert-ly rule
+
+If the modification changes the input syntax, a convert-ly rule
+should be written to automatically update input files from older
+versions.
+
+convert-ly rules are found in python/convertrules.py
+
+If possible, the convert-ly rule should allow automatic updating
+of the file. In some cases, this will not be possible, so the
+rule will simply point out to the user that the feature needs
+manual correction.
+
+@subsubheading Updating version numbers
+
+If a development release occurs between you writing your patch and
+having it approved+pushed, you will need to update the version
+numbers in your tree. This can be done with:
+
+@example
+scripts/auxiliar/update-patch-version old.version.number new.version.number
+@end example
+
+It will change all files in git, so use with caution and examine
+the resulting diff.
+
+
+@node Automatically update documentation
+@subsection Automatically update documentation
+
+@command{convert-ly} should be used to update the documentation,
+the snippets, and the regression tests. This not only makes the
+necessary syntax changes, it also tests the @command{convert-ly}
+rules.
+
+The automatic updating is performed by moving to the top-level
+source directory, then running:
+
+@example
+scripts/auxiliar/update-with-convert-ly.sh
+@end example
+
+If you did an out-of-tree build, pass in the relative path:
+
+@example
+BUILD_DIR=../build-lilypond/ scripts/auxiliar/update-with-convert-ly.sh
+@end example
+
+
+@node Manually update documentation
+@subsection Manually update documentation
+
+Where the convert-ly rule is not able to automatically update the inline
+lilypond code in the documentation (i.e. if a NOT_SMART rule is used), the
+documentation must be manually updated. The inline snippets that require
+changing must be changed in the English version of the docs and all
+translated versions. If the inline code is not changed in the
+translated documentation, the old snippets will show up in the
+English version of the documentation.
+
+Where the convert-ly rule is not able to automatically update snippets
+in Documentation/snippets/, those snippets must be manually updated.
+Those snippets should be copied to Documentation/snippets/new. The
+comments at the top of the snippet describing its automatic generation
+should be removed. All translated texidoc strings should be removed.
+The comment @qq{% begin verbatim} should be removed. The syntax of
+the snippet should then be manually edited.
+
+Where snippets in Documentation/snippets are made obsolete, the snippet
+should be copied to Documentation/snippets/new. The comments and
+texidoc strings should be removed as described above. Then the body
+of the snippet should be changed to:
+
+@example
+\markup @{
+ This snippet is deprecated as of version X.Y.Z and
+ will be removed from the documentation.
+@}
+@end example
+
+@noindent
+where X.Y.Z is the version number for which the convert-ly rule was
+written.
+
+Update the snippet files by running:
+
+@example
+scripts/auxiliar/makelsr.py
+@end example
+
+Where the convert-ly rule is not able to automatically update regression
+tests, the regression tests in input/regression should be manually
+edited.
+
+Although it is not required, it is helpful if the developer
+can write relevant material for inclusion in the Notation
+Reference. If the developer does not feel qualified to write
+the documentation, a documentation editor will be able to
+write it from the regression tests. The text that is added to
+or removed from the documentation should be changed only in
+the English version.
+
+
+@node Edit changes.tely
+@subsection Edit changes.tely
+
+An entry should be added to Documentation/changes.tely to describe
+the feature changes to be implemented. This is especially important
+for changes that change input file syntax.
+
+Hints for changes.tely entries are given at the top of the file.
+
+New entries in changes.tely go at the top of the file.
+
+The changes.tely entry should be written to show how the new change
+improves LilyPond, if possible.
+
+
+@node Verify successful build
+@subsection Verify successful build
+
+When the changes have been made, successful completion must be
+verified by doing
+
+@example
+make all
+make doc
+@end example
+
+When these commands complete without error, the patch is
+considered to function successfully.
+
+Developers on Windows who are unable to build LilyPond should
+get help from a Linux or OSX developer to do the make tests.
+
+
+@node Verify regression tests
+@subsection Verify regression tests
+
+In order to avoid breaking LilyPond, it is important to verify that
+the regression tests succeed, and that no unwanted changes are
+introduced into the output. This process is described in
+@ref{Regtest comparison}.
+
+@subheading Typical developer's edit/compile/test cycle
+
+TODO: is @code{[-j@var{X} CPU_COUNT=@var{X}]} useful for
+@code{test-baseline}, @code{check}, @code{clean},
+@code{test-redo}? Neil Puttock says it is useful for
+everything but @code{clean}, which is disk-limited.
+Need to check formally.
+
+@itemize
+@item
+Initial test:
+
+@example
+make [-j@var{X}]
+make test-baseline
+make [-j@var{X} CPU_COUNT=@var{X}] check
+@end example
+
+@item
+Edit/compile/test cycle:
+
+@example
+@emph{## edit source files, then...}
+
+make clean @emph{## only if needed (see below)}
+make [-j@var{X}] @emph{## only if needed (see below)}
+make test-redo @emph{## redo files differing from baseline}
+make [-j@var{X} CPU_COUNT=@var{X}] check @emph{## CPU_COUNT here?}
+@end example
+
+@item
+Reset:
+
+@example
+make test-clean
+@end example
+@end itemize
+
+If you modify any source files that have to be compiled (such as
+@file{.cc} or @file{.hh} files in @file{flower/} or @file{lily/}),
+then you must run @command{make} before @command{make test-redo},
+so @command{make} can compile the modified files and relink all
+the object files. If you only modify files which are interpreted,
+like those in the @file{scm/} and @file{ly/} directories, then
+@command{make} is not needed before @command{make test-redo}.
+
+TODO: Fix the following paragraph. You can do @command{rm mf/out/*}
+instead of make clean, and you can probably do
+@command{make -C mf/ clean} as well, but I haven't checked it -- cds
+
+Also, if you modify any font definitions in the @file{mf/}
+directory then you must run @command{make clean} and
+@command{make} before running @command{make test-redo}. This will
+recompile everything, whether modified or not, and takes a lot
+longer.
+
+Running @command{make@tie{}check} will leave an HTML page
+@file{out/test-results/index.html}. This page shows all the
+important differences that your change introduced, whether in the
+layout, MIDI, performance or error reporting.
+
+
+
+
+@node Post patch for comments
+@subsection Post patch for comments
+
+See @ref{Uploading a patch for review}.
+
+
+@node Push patch
+@subsection Push patch
+
+Once all the comments have been addressed, the patch can be pushed.
+
+If the author has push privileges, the author will push the patch.
+Otherwise, a developer with push privileges will push the patch.
+
+
+@node Closing the issues
+@subsection Closing the issues
+
+Once the patch has been pushed, all the relevant issues should be
+closed.
+
+On Rietveld, the author should log in an close the issue either by
+using the @q{Edit Issue} link, or by clicking the circled x icon
+to the left of the issue name.
+
+If the changes were in response to a feature request on the Google
+issue tracker for LilyPond, the author should change the status to
+Fixed and a tag @q{fixed_x_y_z} should be added, where the patch was
+fixed in version x.y.z. If
+the author does not have privileges to change the status, an email
+should be sent to bug-lilypond requesting the BugMeister to change
+the status.
+
+
+@node Iterator tutorial
+@section Iterator tutorial
+
+TODO -- this is a placeholder for a tutorial on iterators
+
+Iterators are routines written in C++ that process music expressions
+and sent the music events to the appropriate engravers and/or
+performers.
+
+
+@node Engraver tutorial
+@section Engraver tutorial
+
+Engravers are C++ classes that catch music events and
+create the appropriate grobs for display on the page. Though the
+majority of engravers are responsible for the creation of a single grob,
+in some cases (e.g. @code{New_fingering_engraver}), several different grobs
+may be created.
+
+Engravers listen for events and acknowledge grobs. Events are passed to
+the engraver in time-step order during the iteration phase. Grobs are
+made available to the engraver when they are created by other engravers
+during the iteration phase.
+
+
+@menu
+* Useful methods for information processing::
+* Translation process::
+* Preventing garbage collection for SCM member variables::
+* Listening to music events::
+* Acknowledging grobs::
+* Engraver declaration/documentation::
+@end menu
+
+@node Useful methods for information processing
+@subsection Useful methods for information processing
+
+An engraver inherits the following public methods from the Translator
+base class, which can be used to process listened events and acknowledged
+grobs:
+
+@itemize
+@item @code{virtual void initialize ()}
+@item @code{void start_translation_timestep ()}
+@item @code{void process_music ()}
+@item @code{void process_acknowledged ()}
+@item @code{void stop_translation_timestep ()}
+@item @code{virtual void finalize ()}
+@end itemize
+
+These methods are listed in order of translation time, with
+@code{initialize ()} and @code{finalize ()} bookending the whole
+process. @code{initialize ()} can be used for one-time initialization
+of context properties before translation starts, whereas
+@code{finalize ()} is often used to tie up loose ends at the end of
+translation: for example, an unterminated spanner might be completed
+automatically or reported with a warning message.
+
+
+@node Translation process
+@subsection Translation process
+
+At each timestep in the music, translation proceeds by calling the
+following methods in turn:
+
+@code{start_translation_timestep ()} is called before any user
+information enters the translators, i.e., no property operations
+(\set, \override, etc.) or events have been processed yet.
+
+@code{process_music ()} and @code{process_acknowledged ()} are called
+after all events in the current time step have been heard, or all
+grobs in the current time step have been acknowledged. The latter
+tends to be used exclusively with engravers which only acknowledge
+grobs, whereas the former is the default method for main processing
+within engravers.
+
+@code{stop_translation_timestep ()} is called after all user
+information has been processed prior to beginning the translation for
+the next timestep.
+
+
+@node Preventing garbage collection for SCM member variables
+@subsection Preventing garbage collection for SCM member variables
+
+In certain cases, an engraver might need to ensure private Scheme
+variables (with type SCM) do not get swept away by Guile's garbage
+collector: for example, a cache of the previous key signature which
+must persist between timesteps. The method
+@code{virtual derived_mark () const} can be used in such cases:
+
+@example
+Engraver_name::derived_mark ()
+@{
+ scm_gc_mark (private_scm_member_)
+@}
+@end example
+
+
+@node Listening to music events
+@subsection Listening to music events
+
+External interfaces to the engraver are implemented by protected
+macros including one or more of the following:
+
+@itemize
+@item @code{DECLARE_TRANSLATOR_LISTENER (event_name)}
+@item @code{IMPLEMENT_TRANSLATOR_LISTENER (Engraver_name, event_name)}
+@end itemize
+
+@noindent
+where @var{event_name} is the type of event required to provide the
+input the engraver needs and @var{Engraver_name} is the name of the
+engraver.
+
+Following declaration of a listener, the method is implemented as follows:
+
+@example
+IMPLEMENT_TRANSLATOR_LISTENER (Engraver_name, event_name)
+void
+Engraver_name::listen_event_name (Stream event *event)
+@{
+ ...body of listener method...
+@}
+@end example
+
+
+@node Acknowledging grobs
+@subsection Acknowledging grobs
+
+Some engravers also need information from grobs as they are created
+and as they terminate. The mechanism and methods to obtain this
+information are set up by the macros:
+
+@itemize
+@item @code{DECLARE_ACKNOWLEDGER (grob_interface)}
+@item @code{DECLARE_END_ACKNOWLEDGER (grob_interface)}
+@end itemize
+
+where @var{grob_interface} is an interface supported by the
+grob(s) which should be acknowledged. For example, the following
+code would declare acknowledgers for a @code{NoteHead} grob (via the
+@code{note-head-interface}) and any grobs which support the
+@code{side-position-interface}:
+
+@example
+@code{DECLARE_ACKNOWLEDGER (note_head)}
+@code{DECLARE_ACKNOWLEDGER (side_position)}
+@end example
+
+The @code{DECLARE_END_ACKNOWLEDGER ()} macro sets up a spanner-specific
+acknowledger which will be called whenever a spanner ends.
+
+Following declaration of an acknowledger, the method is coded as follows:
+
+@example
+void
+Engraver_name::acknowledge_interface_name (Grob_info info)
+@{
+ ...body of acknowledger method...
+@}
+@end example
+
+
+@node Engraver declaration/documentation
+@subsection Engraver declaration/documentation
+
+An engraver must have a public macro
+
+@itemize
+@item @code{TRANSLATOR_DECLARATIONS (Engraver_name)}
+@end itemize
+
+@noindent
+where @code{Engraver_name} is the name of the engraver. This
+defines the common variables and methods used by every engraver.
+
+At the end of the engraver file, one or both of the following
+macros are generally called to document the engraver in the
+Internals Reference:
+
+@itemize
+@item @code{ADD_ACKNOWLEDGER (Engraver_name, grob_interface)}
+@item @code{ADD_TRANSLATOR (Engraver_name, Engraver_doc,
+ Engraver_creates, Engraver_reads, Engraver_writes)}
+@end itemize
+
+@noindent
+where @code{Engraver_name} is the name of the engraver, @code{grob_interface}
+is the name of the interface that will be acknowledged,
+@code{Engraver_doc} is a docstring for the engraver,
+@code{Engraver_creates} is the set of grobs created by the engraver,
+@code{Engraver_reads} is the set of properties read by the engraver,
+and @code{Engraver_writes} is the set of properties written by
+the engraver.
+
+The @code{ADD_ACKNOWLEDGER} and @code{ADD_TRANSLATOR} macros use a
+non-standard indentation system. Each interface, grob, read property,
+and write property is on its own line, and the closing parenthesis
+and semicolon for the macro all occupy a separate line beneath the final
+interface or write property. See existing engraver files for more
+information.
+
+
+@node Callback tutorial
+@section Callback tutorial
+
+TODO -- This is a placeholder for a tutorial on callback functions.
+
+@node LilyPond scoping
+@section LilyPond scoping
+
+The Lilypond language has a concept of scoping, i.e. you can do
+
+@example
+foo = 1
+
+#(begin
+ (display (+ foo 2)))
+@end example
+
+@noindent with @code{\paper}, @code{\midi} and @code{\header} being
+nested scope inside the @file{.ly} file-level scope. @w{@code{foo = 1}}
+is translated in to a scheme variable definition.
+
+This implemented using modules, with each scope being an anonymous
+module that imports its enclosing scope's module.
+
+Lilypond's core, loaded from @file{.scm} files, is usually placed in the
+@code{lily} module, outside the @file{.ly} level. In the case of
+
+@example
+lilypond a.ly b.ly
+@end example
+
+@noindent
+we want to reuse the built-in definitions, without changes effected in
+user-level @file{a.ly} leaking into the processing of @file{b.ly}.
+
+The user-accessible definition commands have to take care to avoid
+memory leaks that could occur when running multiple files. All
+information belonging to user-defined commands and markups is stored in
+a manner that allows it to be garbage-collected when the module is
+dispersed, either by being stored module-locally, or in weak hash
+tables.
+
+@node LilyPond miscellany
+@section LilyPond miscellany
+
+This is a place to dump information that may be of use to developers
+but doesn't yet have a proper home. Ideally, the length of this section
+would become zero as items are moved to other homes.
+
+
+@menu
+* Spacing algorithms::
+* Info from Han-Wen email::
+* Music functions and GUILE debugging::
+@end menu
+
+@node Spacing algorithms
+@subsection Spacing algorithms
+
+Here is information from an email exchange about spacing algorithms.
+
+On Thu, 2010-02-04 at 15:33 -0500, Boris Shingarov wrote:
+I am experimenting with some modifications to the line breaking code,
+and I am stuck trying to understand how some of it works. So far my
+understanding is that Simple_spacer operates on a vector of Grobs, and
+it is a well-known Constrained-QP problem (rods = constraints, springs
+= quadratic function to minimize). What I don't understand is, if the
+spacer operates at the level of Grobs, which are built at an earlier
+stage in the pipeline, how are the changes necessitated by differences
+in line breaking, taken into account? in other words, if I take the
+last measure of a line and place it on the next line, it is not just a
+matter of literally moving that graphic to where the start of the next
+line is, but I also need to draw a clef, key signature, and possibly
+other fundamental things -- but at that stage in the rendering
+pipeline, is it not too late??
+
+Joe Neeman answered:
+
+We create lots of extra grobs (eg. a BarNumber at every bar line) but
+most of them are not drawn. See the break-visibility property in
+item-interface.
+
+
+@node Info from Han-Wen email
+@subsection Info from Han-Wen email
+
+In 2004, Douglas Linhardt decided to try starting a document that would
+explain LilyPond architecture and design principles. The material below
+is extracted from that email, which can be found at
+@uref{http://thread.gmane.org/gmane.comp.gnu.lilypond.devel/2992}.
+The headings reflect questions from Doug or comments from Han-Wen;
+the body text are Han-Wen's answers.
+
+@subheading Figuring out how things work.
+
+I must admit that when I want to know how a program works, I use grep
+and emacs and dive into the source code. The comments and the code
+itself are usually more revealing than technical documents.
+
+@subheading What's a grob, and how is one used?
+
+Graphical object - they are created from within engravers, either as
+Spanners (derived class) -slurs, beams- or Items (also a derived
+class) -notes, clefs, etc.
+
+There are two other derived classes System (derived from Spanner,
+containing a "line of music") and Paper_column (derived from Item, it
+contains all items that happen at the same moment). They are separate
+classes because they play a special role in the linebreaking process.
+
+@subheading What's a smob, and how is one used?
+
+A C(++) object that is encapsulated so it can be used as a Scheme
+object. See GUILE info, "19.3 Defining New Types (Smobs)"
+
+@@subheading When is each C++ class constructed and used
+
+@itemize
+
+@item
+Music classes
+
+In the parser.yy see the macro calls MAKE_MUSIC_BY_NAME().
+
+@item
+Contexts
+
+Constructed during "interpreting" phase.
+
+@item
+Engravers
+
+Executive branch of Contexts, plugins that create grobs, usually one
+engraver per grob type. Created together with context.
+
+@item
+Layout Objects
+
+= grobs
+
+@item
+Grob Interfaces
+
+These are not C++ classes per se. The idea of a Grob interface hasn't
+crystallized well. ATM, an interface is a symbol, with a bunch of grob
+properties. They are not objects that are created or destroyed.
+
+@item
+Iterators
+
+Objects that walk through different music classes, and deliver events
+in a synchronized way, so that notes that play together are processed
+at the same moment and (as a result) end up on the same horizontal position.
+
+Created during interpreting phase.
+
+BTW, the entry point for interpreting is ly:run-translator
+(ly_run_translator on the C++ side)
+
+@end itemize
+
+@subheading Can you get to Context properties from a Music object?
+
+You can create music object with a Scheme function that reads context
+properties (the \applycontext syntax). However, that function is
+executed during Interpreting, so you can not really get Context
+properties from Music objects, since music objects are not directly
+connected to Contexts. That connection is made by the Music_iterators
+
+@subheading Can you get to Music properties from a Context object?
+
+Yes, if you are given the music object within a Context
+object. Normally, the music objects enter Contexts in synchronized
+fashion, and the synchronization is done by Music_iterators.
+
+@subheading What is the relationship between C++ classes and Scheme objects?
+
+Smobs are C++ objects in Scheme. Scheme objects (lists, functions) are
+manipulated from C++ as well using the GUILE C function interface
+(prefix: scm_)
+
+@subheading How do Scheme procedures get called from C++ functions?
+
+scm_call_*, where * is an integer from 0 to 4.
+Also scm_c_eval_string (), scm_eval ()
+
+@subheading How do C++ functions get called from Scheme procedures?
+
+Export a C++ function to Scheme with LY_DEFINE.
+
+@subheading What is the flow of control in the program?
+
+Good question. Things used to be clear-cut, but we have Scheme
+and SMOBs now, which means that interactions do not follow a very
+rigid format anymore. See below for an overview, though.
+
+@subheading Does the parser make Scheme procedure calls or C++ function calls?
+
+Both. And the Scheme calls can call C++ and vice versa. It's nested,
+with the SCM datatype as lubrication between the interactions
+
+(I think the word "lubrication" describes the process better than the
+traditional word "glue")
+
+@subheading How do the front-end and back-end get started?
+
+Front-end: a file is parsed, the rest follows from that. Specifically,
+
+Parsing leads to a Music + Music_output_def object (see parser.yy,
+definition of toplevel_expression )
+
+A Music + Music_output_def object leads to a Global_context object (see
+ly_run_translator ())
+
+During interpreting, Global_context + Music leads to a bunch of
+Contexts (see Global_translator::run_iterator_on_me ()).
+
+After interpreting, Global_context contains a Score_context (which
+contains staves, lyrics etc.) as a child. Score_context::get_output ()
+spews a Music_output object (either a Paper_score object for notation
+or Performance object for MIDI).
+
+The Music_output object is the entry point for the backend (see
+ly_render_output ()).
+
+The main steps of the backend itself are in
+
+@itemize
+
+@item
+@file{paper-score.cc} , Paper_score::process_
+
+@item
+@file{system.cc} , System::get_lines()
+
+@item
+The step, where things go from grobs to output, is in
+System::get_line(): each grob delivers a Stencil (a Device
+independent output description), which is interpreted by our
+outputting backends (@file{scm/output-tex.scm} and
+@file{scm/output-ps.scm}) to produce TeX and PS.
+
+@end itemize
+
+Interactions between grobs and putting things into .tex and .ps files
+have gotten a little more complex lately. Jan has implemented
+page-breaking, so now the backend also involves Paper_book,
+Paper_lines and other things. This area is still heavily in flux, and
+perhaps not something you should want to look at.
+
+@subheading How do the front-end and back-end communicate?
+
+There is no communication from backend to front-end. From front-end to
+backend is simply the program flow: music + definitions gives
+contexts, contexts yield output, after processing, output is written
+to disk.
+
+@subheading Where is the functionality associated with KEYWORDs?
+
+See @file{my-lily-lexer.cc} (keywords, there aren't that many)
+and @file{ly/*.ly} (most of the other backslashed @code{/\words} are identifiers)
+
+@subheading What Contexts/Properties/Music/etc. are available when they are processed?
+
+What do you mean exactly with this question?
+
+See @file{ly/engraver-init.ly} for contexts,
+see @file{scm/define-*.scm} for other objects.
+
+@subheading How do you decide if something is a Music, Context, or Grob property?
+Why is part-combine-status a Music property when it seems (IMO)
+to be related to the Staff context?
+
+The Music_iterators and Context communicate through two channels
+
+Music_iterators can set and read context properties, idem for
+Engravers and Contexts
+
+Music_iterators can send "synthetic" music events (which aren't in
+the input) to a context. These are caught by Engravers. This is
+mostly a one way communication channel.
+
+part-combine-status is part of such a synthetic event, used by
+Part_combine_iterator to communicate with Part_combine_engraver.
+
+
+@subheading Deciding between context and music properties
+
+I'm adding a property to affect how \autochange works. It seems to
+me that it should be a context property, but the Scheme autochange
+procedure has a Music argument. Does this mean I should use
+a Music property?
+
+\autochange is one of these extra strange beasts: it requires
+look-ahead to decide when to change staves. This is achieved by
+running the interpreting step twice (see
+@file{scm/part-combiner.scm} , at the bottom), and
+storing the result of the first step (where to switch
+staves) in a Music property. Since you want to influence that
+where-to-switch list, your must affect the code in
+make-autochange-music (@file{scm/part-combiner.scm}).
+That code is called directly from the parser and there are no
+official "parsing properties" yet, so there is no generic way
+to tune \autochange. We would have to invent something new
+for this, or add a separate argument,
+
+@example
+ \autochange #around-central-C ..music..
+@end example
+
+@noindent
+where around-central-C is some function that is called from
+make-autochange-music.
+
+@subheading More on context and music properties
+
+From Neil Puttock, in response to a question about transposition:
+
+Context properties (using \set & \unset) are tied to engravers: they
+provide information relevant to the generation of graphical objects.
+
+Since transposition occurs at the music interpretation stage, it has
+no direct connection with engravers: the pitch of a note is fixed
+before a notehead is created. Consider the following minimal snippet:
+
+@example
+@{ c' @}
+@end example
+
+This generates (simplified) a NoteEvent, with its pitch and duration
+as event properties,
+
+@example
+(make-music
+ 'NoteEvent
+ 'duration
+ (ly:make-duration 2 0 1 1)
+ 'pitch
+ (ly:make-pitch 0 0 0)
+@end example
+
+which the Note_heads_engraver hears. It passes this information on to
+the NoteHead grob it creates from the event, so the head's correct
+position and duration-log can be determined once it's ready for
+printing.
+
+If we transpose the snippet,
+
+@example
+\transpose c d @{ c' @}
+@end example
+
+the pitch is changed before it reaches the engraver (in fact, it
+happens just after the parsing stage with the creation of a
+TransposedMusic music object):
+
+@example
+(make-music
+ 'NoteEvent
+ 'duration
+ (ly:make-duration 2 0 1 1)
+ 'pitch
+ (ly:make-pitch 0 1 0)
+@end example
+
+You can see an example of a music property relevant to transposition:
+untransposable.
+
+@example
+\transpose c d @{ c'2 \withMusicProperty #'untransposable ##t c' @}
+@end example
+
+-> the second c' remains untransposed.
+
+Take a look at @file{lily/music.cc} to see where the transposition takes place.
+
+
+@subheading How do I tell about the execution environment?
+
+I get lost figuring out what environment the code I'm looking at is in when it
+executes. I found both the C++ and Scheme autochange code. Then I was trying
+to figure out where the code got called from. I finally figured out that the
+Scheme procedure was called before the C++ iterator code, but it took me a
+while to figure that out, and I still didn't know who did the calling in the
+first place. I only know a little bit about Flex and Bison, so reading those
+files helped only a little bit.
+
+@emph{Han-Wen:} GDB can be of help here. Set a breakpoint in C++, and run. When you
+hit the breakpoint, do a backtrace. You can inspect Scheme objects
+along the way by doing
+
+@example
+p ly_display_scm(obj)
+@end example
+
+this will display OBJ through GUILE.
+
+@node Music functions and GUILE debugging
+@subsection Music functions and GUILE debugging
+
+Ian Hulin was trying to do some debugging in music functions, and
+came up with the following question
+
+HI all,
+I'm working on the Guile Debugger Stuff, and would like to try
+debugging a music function definition such as:
+
+@example
+conditionalMark = #(define-music-function (parser location) ()
+ #@{ \tag #'instrumental-part @{\mark \default@} #@} )
+@end example
+
+It appears conditionalMark does not get set up as an
+equivalent of a Scheme
+
+@example
+(define conditionalMark = define-music-function(parser location () ...
+@end example
+
+@noindent
+although something gets defined because Scheme apparently recognizes
+
+@example
+#(set-break! conditionalMark)
+@end example
+
+@noindent
+later on in the file without signalling any Guile errors.
+
+However the breakpoint trap is never encountered as
+define-music-function passed things on to ly:make-music-function,
+which is really C++ code ly_make_music_function, so Guile never
+finds out about the breakpoint.
+
+Han-Wen answered as follows:
+
+You can see the definition by doing
+
+@example
+#(display conditionalMark)
+@end example
+
+noindent
+inside the @file{.ly} file.
+
+The breakpoint failing may have to do with the call sequence. See
+@file{parser.yy}, run_music_function(). The function is called directly from
+C++, without going through the GUILE evaluator, so I think that is why
+there is no debugger trap.
projects / lilypond.git / blobdiff