-@c -*- coding: us-ascii; mode: texinfo; -*-
+@c -*- coding: utf-8; mode: texinfo; -*-
@node Programming work
@chapter Programming work
* Finding functions::
* Code style::
* Debugging LilyPond::
+* Tracing object relationships::
* Adding or modifying features::
+* Iterator tutorial::
+* Engraver tutorial::
+* Callback tutorial::
+* LilyPond scoping::
+* LilyPond miscellany::
@end menu
@node Overview of LilyPond architecture
@uref{http://lilypond.org/web/images/thesis-erik-sandberg.pdf, master's
thesis}.
-
The first stage of LilyPond processing is @emph{parsing}. In the parsing
process, music expressions in LilyPond input format are converted to music
expressions in Scheme format. In Scheme format, a music expression is a list
The final stage of LilyPond processing is @emph{translation}. During
translation, music events are prepared for graphical or midi output. The
-translation step is accomplished by translators or engravers (the distinction
-is unclear).
+translation step is accomplished by the polymorphic base class Translator
+through its two derived classes: Engraver (for graphical output) and
+Performer (for midi output).
-Translators are defined in C++ files named *-engraver.cc. In *-engraver.cc, a
-C++ class of Engraver type is created. The Engraver is also declared as a
-translator. Much of the work of translating is handled by Scheme functions,
+Translators are defined in C++ files named *-engraver.cc and *-performer.cc.
+Much of the work of translating is handled by Scheme functions,
which is one of the keys to LilyPond's exceptional flexibility.
+@sourceimage{architecture-diagram,,,png}
+
@node LilyPond programming languages
@section LilyPond programming languages
@uref{http://www.gnu.org/software/guile/manual/html_node/index.html, GUILE
Reference Manual}.
+@subsection Flex
+
+The LilyPond lexer is implemented in Flex, an implementation of the Unix lex
+lexical analyser generator. Resources for Flex can be found
+@uref{http://flex.sourceforge.net/, here}.
+
@subsection GNU Bison
The LilyPond parser is implemented in Bison, a GNU parser generator. The
@subsection GUILE or Scheme
-GUILE is the dialect of Scheme that is used as LilyPond's extension language. Many extensions to LilyPond are written entirely in GUILE. The
+GUILE is the dialect of Scheme that is used as LilyPond's extension language.
+Many extensions to LilyPond are written entirely in GUILE. The
@uref{http://www.gnu.org/software/guile/manual/html_node/index.html,
GUILE Reference Manual} is available online.
programming in Scheme is available in its entirety online.
An introduction to Guile/Scheme as used in LilyPond can be found in the
-Learning Manual, see @rlearning{Scheme tutorial}.
+@rextend{Scheme tutorial}.
@subsection MetaFont
@node Code style
@section Code style
-@c email to wl@gnu.org when I get here.
-
-@warning{this is pending some confirmation on -devel. July 2009 -gp}
-
-Command-line script to format stuff with emacs:
-
-@example
-#!/bin/sh
-emacs $1 -batch --eval '(indent-region (point-min) (point-max) nil)' -f save-buffer
-@end example
-
-(that's all on one line)
-
-Save it as a shell script, then run on the file(s) you modified.
@menu
@end menu
))
@end verbatim
+Some source files may not currently have proper indenting. If this
+is the case, it is desirable to fix the improper indenting when the
+file is modified, with the hope of continually improving the code.
+
+@subsection Indenting files with emacs in script mode
+
+@c email to wl@gnu.org when I get here.
+
+@warning{this is pending some confirmation on -devel. July 2009 -gp}
+
+Command-line script to format stuff with emacs:
+
+@example
+#!/bin/sh
+emacs $1 -batch --eval '(indent-region (point-min) (point-max) nil)' -f save-buffer
+@end example
+
+(that's all on one line)
+
+Save it as a shell script, then run on the file(s) you modified.
+
+@subsection Indenting with vim
+
+Although emacs indentation is the LilyPond standard, acceptable
+indentation can usually be accomplished with vim. Some hints for
+vim are as follows:
+
+A workable .vimrc:
+
+@verbatim
+set cindent
+set smartindent
+set autoindent
+set expandtab
+set softtabstop=2
+set shiftwidth=2
+filetype plugin indent on
+set incsearch
+set ignorecase smartcase
+set hlsearch
+set confirm
+set statusline=%F%m%r%h%w\ %{&ff}\ %Y\ [ASCII=\%03.3b]\ [HEX=\%02.2B]\ %04l,%04v\ %p%%\ [LEN=%L]
+set laststatus=2
+set number
+" Remove trailing whitespace on write
+autocmd BufWritePre * :%s/\s\+$//e
+@end verbatim
+
+With this .vimrc, files can be reindented automatically by highlihting
+the lines to be indented in visual mode (use V to enter visual mode)
+and pressing =.
+
+A scheme.vim file will help improve the indentation. This one
+was suggested by Patrick McCarty. It should be saved in
+~/.vim/after/syntax/scheme.vim.
+
+@verbatim
+" Additional Guile-specific 'forms'
+syn keyword schemeSyntax define-public define* define-safe-public
+syn keyword schemeSyntax use-modules define-module
+syn keyword schemeSyntax defmacro-public define-macro
+syn keyword schemeSyntax define-markup-command
+syn keyword schemeSyntax define-markup-list-command
+syn keyword schemeSyntax let-keywords* lambda* define*-public
+syn keyword schemeSyntax defmacro* defmacro*-public
+
+" All of the above should influence indenting too
+set lw+=define-public,define*,define-safe-public,use-modules,define-module
+set lw+=defmacro-public,define-macro
+set lw+=define-markup-command,define-markup-list-command
+set lw+=let-keywords*,lambda*,define*-public,defmacro*,defmacro*-public
+
+" These forms should not influence indenting
+set lw-=if
+set lw-=set!
+
+" Try to highlight all ly: procedures
+syn match schemeFunc "ly:[^) ]\+"
+@end verbatim
@subsection Classes and Types
@subsection Naming
+Variable names should be complete words, rather than abbreviations.
+For example, it is preferred to use @code{thickness} rather than
+@code{th} or @code{t}.
+
+Multi-word variable names in C++ should have the words separated
+by the underscore character (@q{_}).
+
+Multi-word variable names in Scheme should have the words separated
+by a hyphen (@q{-}).
+
+@subsection Comments
+
+Comments may not be needed if descriptive variable names are used
+in the code and the logic is straightforward. However, if the
+logic is difficult to follow, and particularly if non-obvious
+code has been included to resolve a bug, a comment describing
+the logic and/or the need for the non-obvious code should be included.
+
+There are instances where the current code could be commented better.
+If significant time is required to understand the code as part of
+preparing a patch, it would be wise to add comments reflecting your
+understanding to make future work easier.
@subsection Messages
for localization). This includes warning and error messages.
@item
-Don't localize/gettextify:
+Do not localize/gettextify:
@itemize
@item
@end itemize
@item
-Messages to be localised must be encapsulated in `_ (STRING)' or
+Messages to be localized must be encapsulated in `_ (STRING)' or
`_f (FORMAT, ...)'. E.g.:
@example
@end example
@item
-Do not modularise too much; words frequently cannot be translated
+Do not modularize too much; words frequently cannot be translated
without context. It is probably safe to treat most occurences of
words like stem, beam, crescendo as separately translatable words.
@node Debugging LilyPond
@section Debugging LilyPond
-The most commonly used tool for debugging LilyPond is the GNU debugger
-gdb. Use of gdb is described in this section.
+The most commonly used tool for debugging LilyPond is the GNU
+debugger gdb. The gdb tool is used for investigating and debugging
+core Lilypond code written in C++. Another tool is available for
+debugging Scheme code using the Guile debugger. This section
+describes how to use both gdb and the Guile Debugger.
+
+@menu
+* Debugging overview::
+* Debugging C++ code::
+* Debugging Scheme code::
+@end menu
+@node Debugging overview
@subsection Debugging overview
Using a debugger simplifies troubleshooting in at least two ways.
Then, when execution has paused, debugger commands can be issued to
explore the values of various variables or to execute functions.
-Second, the debugger allows the display of a stack trace, which shows
-the sequence in which functions are called and the arguments to the
-various function calls.
+Second, the debugger can display a stack trace, which shows the
+sequence in which functions have been called and the arguments
+passed to the called functions.
+
+@node Debugging C++ code
+@subsection Debugging C++ code
+The GNU debugger, gdb, is the principal tool for debugging C++ code.
-@subsection Compiling with debugging information
+@unnumberedsubsubsec Compiling LilyPond for use with gdb
-In order to use a debugger with LilyPond, it is necessary to compile
+In order to use gdb with LilyPond, it is necessary to compile
LilyPond with debugging information. This is accomplished by running
the following commands in the main LilyPond source directory.
@example
./configure --disable-optimising
-
make
@end example
-This will create a version of LilyPond that contains the debugging
+This will create a version of LilyPond containing debugging
information that will allow the debugger to tie the source code
to the compiled code.
You should not do @var{make install} if you want to use a debugger
-with LilyPond. @var{make install} will strip the debugging information
-from the LilyPond binary.
+with LilyPond. The @var{make install} command will strip debugging
+information from the LilyPond binary.
+
+@unnumberedsubsubsec Typical gdb usage
-To set breakpoints in Scheme functions, put
+Once you have compiled the Lilypond image with the necessary
+debugging information it will have been written to a location in a
+subfolder of your current working directory:
@example
-\include "guile-debugger.ly"
+out/bin/lilypond
@end example
-in your input file after any scheme procedures you have defined in
-that file. When your input file is processed, a guile prompt
-will be displayed. At the guile prompt, you can set breakpoints with
-the @code{break!} procedure:
+This is important as you will need to let gdb know where to find the
+image containing the symbol tables. You can invoke gdb from the
+command line using
@example
-guile> (break! my-scheme-procedure)
+gdb out/bin/lilypond
@end example
-
-Once you have set the desired breakpoints, you exit the guile repl frame
-by typing:
+@noindent
+This loads the LilyPond symbol tables into gdb. Then, to run
+LilyPond on @code{test.ly} under the debugger, enter
@example
-guile> (quit)
+run test.ly
@end example
+@noindent
+at the gdb prompt.
-When one of the scheme routines for which you have set breakpoints is
-entered, guile will interrupt execution in a debug frame. At this point,
-you will have access to guile debugging commands. For a listing of these
-commands, type:
+As an alternative to running gdb at the command line you may try
+a graphical interface to gdb such as ddd
@example
-debug> help
+ddd out/bin/lilypond
@end example
-@subsection Typical gdb usage
+You can also use sets of standard gdb commands stored in a .gdbinit
+file (see next section).
-@subsection Typical .gdbinit files
+@unnumberedsubsubsec Typical .gdbinit files
-The behavior of gdb can be readily customized through the use of
-@var{.gdbinit} files. A @var{.gdbinit} file is a file named
+The behavior of gdb can be readily customized through the use of a
+@var{.gdbinit} file. A @var{.gdbinit} file is a file named
@var{.gdbinit} (notice the @qq{.} at the beginning of the file name)
that is placed in a user's home directory.
end
@end example
-@subsection Using Guile interactively with LilyPond
+@node Debugging Scheme code
+@subsection Debugging Scheme code
+
+Scheme code can be developed using the Guile command line
+interpreter @code{top-repl}. You can either investigate
+interactively using just Guile or you can use the debugging
+tools available within Guile.
+
+@unnumberedsubsubsec Using Guile interactively with LilyPond
In order to experiment with Scheme programming in the LilyPond
-environment, it is convenient to have a Guile interpreter that
+environment, it is necessary to have a Guile interpreter that
has all the LilyPond modules loaded. This requires the following
steps.
-First, define a Scheme symbol for the active module
-in the .ly file:
+First, define a Scheme symbol for the active module in the .ly file:
@example
#(module-define! (resolve-module '(guile-user))
- 'lilypond-module (current-module))
+ 'lilypond-module (current-module))
@end example
-Second, place a Scheme function in the .ly file that gives an interactive Guile
-prompt:
+Now place a Scheme function in the .ly file that gives an
+interactive Guile prompt:
@example
#(top-repl)
@end example
-When the .ly file is compiled, this causes the compilation to be interrupted
-and an interactive guile prompt to appear. When the guile prompt appears,
-the LilyPond active module must be set as the current guile module:
+When the .ly file is compiled, this causes the compilation to be
+interrupted and an interactive guile prompt to appear. Once the
+guile prompt appears, the LilyPond active module must be set as the
+current guile module:
@example
guile> (set-current-module lilypond-module)
@end example
-Proper operation of these commands can be demonstrated by typing the name
-of a LilyPond public scheme function to see if it's properly defined:
+You can demonstrate these commands are operating properly by typing the name
+of a LilyPond public scheme function to check it has been defined:
@example
guile> fret-diagram-verbose-markup
ABORT: (unbound-variable)
@end example
-Once the module is properly loaded, any valid LilyPond Scheme expression
-can be entered at the interactive prompt.
+Once the module is properly loaded, any valid LilyPond Scheme
+expression can be entered at the interactive prompt.
-After the investigation is complete, the interactive guile interpreter
-can be exited:
+After the investigation is complete, the interactive guile
+interpreter can be exited:
@example
guile> (quit)
The compilation of the .ly file will then continue.
+@unnumberedsubsubsec Using the Guile debugger
+
+To set breakpoints and/or enable tracing in Scheme functions, put
+
+@example
+\include "guile-debugger.ly"
+@end example
+
+in your input file after any scheme procedures you have defined in
+that file. This will invoke the Guile command-line after having set
+up the environment for the debug command-line. When your input file
+is processed, a guile prompt will be displayed. You may now enter
+commands to set up breakpoints and enable tracing by the Guile debugger.
+
+@unnumberedsubsubsec Using breakpoints
+
+At the guile prompt, you can set breakpoints with
+the @code{set-break!} procedure:
+
+@example
+guile> (set-break! my-scheme-procedure)
+@end example
+
+Once you have set the desired breakpoints, you exit the guile repl frame
+by typing:
+
+@example
+guile> (quit)
+@end example
+
+Then, when one of the scheme routines for which you have set
+breakpoints is entered, guile will interrupt execution in a debug
+frame. At this point you will have access to Guile debugging
+commands. For a listing of these commands, type:
+
+@example
+debug> help
+@end example
+
+Alternatively you may code the breakpoints in your Lilypond source
+file using a command such as:
+
+@example
+#(set-break! my-scheme-procedure)
+@end example
+
+immediately after the @code{\include} statement. In this case the
+breakpoint will be set straight after you enter the @code{(quit)}
+command at the guile prompt.
+
+Embedding breakpoint commands like this is particularly useful if
+you want to look at how the Scheme procedures in the @var{.scm}
+files supplied with LilyPond work. To do this, edit the file in
+the relevant directory to add this line near the top:
+
+@example
+(use-modules (scm guile-debugger))
+@end example
+
+Now you can set a breakpoint after the procedure you are interested
+in has been declared. For example, if you are working on routines
+called by @var{print-book-with} in @var{lily-library.scm}:
+
+@example
+(define (print-book-with parser book process-procedure)
+ (let* ((paper (ly:parser-lookup parser '$defaultpaper))
+ (layout (ly:parser-lookup parser '$defaultlayout))
+ (outfile-name (get-outfile-name parser)))
+ (process-procedure book paper layout outfile-name)))
+
+(define-public (print-book-with-defaults parser book)
+ (print-book-with parser book ly:book-process))
+
+(define-public (print-book-with-defaults-as-systems parser book)
+ (print-book-with parser book ly:book-process-to-systems))
+
+@end example
+
+At this point in the code you could add this to set a breakpoint at
+print-book-with:
+
+@example
+(set-break! print-book-with)
+@end example
+
+@unnumberedsubsubsec Tracing procedure calls and evaluator steps
+
+Two forms of trace are available:
+
+@example
+(set-trace-call! my-scheme-procedure)
+@end example
+
+and
+
+@example
+(set-trace-subtree! my-scheme-procedure)
+@end example
+
+@code{set-trace-call!} causes Scheme to log a line to the standard
+output to show when the procedure is called and when it exits.
+
+@code{set-trace-subtree!} traces every step the Scheme evaluator
+performs in evaluating the procedure.
+
+@node Tracing object relationships
+@section Tracing object relationships
+
+Understanding the LilyPond source often boils down to figuring out what
+is happening to the Grobs. Where (and why) are they being created,
+modified and destroyed? Tracing Lily through a debugger in order to
+identify these relationships can be time-consuming and tedious.
+
+In order to simplify this process, a facility has been added to
+display the grobs that are created and the properties that are set
+and modified. Although it can be complex to get set up, once set up
+it easily provides detailed information about the life of grobs
+in the form of a network graph.
+
+Each of the steps necessary to use the graphviz utility
+is described below.
+
+@enumerate
+
+@item Installing graphviz
+
+In order to create the graph of the object relationships, it is
+first necessary to install Graphviz. graphviz is available for a
+number of different platforms:
+
+@example
+@uref{http://www.graphviz.org/Download..php}
+@end example
+
+@item Modifying config.make
+
+In order for the Graphviz tool to work, config.make must be modified.
+It is probably a good idea to first save a copy of config.make under
+a different name. Then, edit config.make by removing every occurence
+of @code{-DNDEBUG}.
+
+@item Rebuilding LilyPond
+
+The executable code of LilyPond must be rebuilt from scratch:
+
+@example
+make -C lily clean && make -C lily
+@end example
+
+@item Create a graphviz-compatible .ly file
+
+In order to use the graphviz utility, the .ly file must include
+@file{ly/graphviz-init.ly}, and should then specify the
+grobs and symbols that should be tracked. An example of this
+is found in @file{input/regression/graphviz.ly}.
+
+@item Run lilypond with output sent to a log file
+
+The Graphviz data is sent to stderr by lilypond, so it is
+necessary to redirect stderr to a logfile:
+
+@example
+lilypond graphviz.ly 2> graphviz.log
+@end example
+
+@item Edit the logfile
+
+The logfile has standard lilypond output, as well as the Graphviz
+output data. Delete everything from the beginning of the file
+up to but not including the first occurence of @code{digraph}.
+
+@item Process the logfile with @code{dot}
+
+The directed graph is created from the log file with the program
+@code{dot}:
+
+@example
+dot -Tpdf graphviz.log > graphviz.pdf
+@end example
+
+@end enumerate
+
+The pdf file can then be viewed with any pdf viewer.
+
+When compiled without @code{-DNDEBUG}, lilypond may run slower
+than normal. The original configuration can be restored by either
+renaming the saved copy of @code{config.make} or rerunning
+@code{configure}. Then rebuild lilypond with
+
+@example
+make -C lily clean && make -C lily
+@end example
+
+
@node Adding or modifying features
@section Adding or modifying features
@example
\markup @{
- "This snippet is deprecated as of version X.Y.Z and
- will be removed from the documentation."
+ This snippet is deprecated as of version X.Y.Z and
+ will be removed from the documentation.
@}
@end example
Update the snippet files by running:
@example
-scripts\auxiliar\makelsr.py
+scripts/auxiliar/makelsr.py
@end example
Where the convert-ly rule is not able to automatically update regression
or removed from the documentation should be changed only in
the English version.
-@subsection Write NEWS entry
+@subsection Edit changes.tely
-An entry should be added to the NEWS file to describe the feature
-changes to be implemented. This is especially important for changes
-that change input file syntax.
+An entry should be added to Documentation/changes.tely to describe
+the feature changes to be implemented. This is especially important
+for changes that change input file syntax.
-Hints for NEWS file entries are given at the top of the NEWS file.
+Hints for changes.tely entries are given at the top of the file.
-New entries in NEWS go at the top of the file.
+New entries in changes.tely go at the top of the file.
-The NEWS entry should be written to show how the new change
+The changes.tely entry should be written to show how the new change
improves LilyPond, if possible.
@subsection Verify successful build
@subsection Post patch for comments
For any change other than a minor change, a patch set should be
-posted on Rietveld for comment. This requires the use of an
-external package, git-cl.
+posted on @uref{http://codereview.appspot.com/, Rietveld} for comment.
+This requires the use of an external package, git-cl, and an email
+account on Google.
git-cl is installed by:
Then, add the git-cl directory to your PATH, or create a
symbolic link to the git-cl and upload.py in one of your
-PATH directories (like usr/bin). git-cl will is then
-configured by
+PATH directories (like usr/bin). git-cl is then
+configured by entering the command
@example
-git-cl config
+git cl config
@end example
@noindent
-and answering the questions that are asked.
-
-The patch set is posted by issuing the following command, after
-first committing all changes:
+in the LilyPond git directory and answering the questions that
+are asked. If you do not understand the question answer with just
+a newline (CR).
+
+The patch set is posted to Rietveld as follows. Ensure your changes
+are committed in a separate branch, which should differ from the
+reference branch to be used by just the changes to be uploaded.
+If the reference branch is to be origin/master, ensure this is
+up-to-date. If necessary, use git rebase to rebase the branch
+containing the changes to the head of origin/master. Finally,
+check out branch with the changes and enter the command:
@example
-git-cl upload <reference SHA1 ID>
+git cl upload <reference SHA1 ID>
@end example
@noindent
where <reference SHA1 ID> is the SHA1 ID of the commit to be used
-as a reference source for the patch (generally, this will be the
-SHA1 ID of origin/master).
+as a reference source for the patch. Generally, this will be the
+SHA1 ID of origin/master, and in that case the command
+
+@example
+git cl upload origin/master
+@end example
+
+@noindent
+can be used.
-After prompting for an email and a password, the patch set will be
-posted to Rietveld.
+After prompting for your Google email address and password, the
+patch set will be posted to Rietveld.
-An email should then be sent to lilypond-devel, with a subject line
+You should then announce the patch by sending
+an email to lilypond-devel, with a subject line
starting with PATCH:, asking for comments on the patch.
As revisions are made in response to comments, successive patch sets
-for the same issue can be uploaded by reissuing the git-cl command.
+for the same issue can be uploaded by reissuing the git-cl command
+with the modified branch checked out.
@subsection Push patch
If the changes were in response to a feature request on the Google
issue tracker for LilyPond, the author should change the status to
-@q{Fixed_x_y_z} where the patch was fixed in version x.y.z. If
+Fixed and a tag @q{fixed_x_y_z} should be added, where the patch was
+fixed in version x.y.z. If
the author does not have privileges to change the status, an email
should be sent to bug-lilypond requesting the BugMeister to change
the status.
+
+@node Iterator tutorial
+@section Iterator tutorial
+
+TODO -- this is a placeholder for a tutorial on iterators
+
+Iterators are routines written in C++ that process music expressions
+and sent the music events to the appropriate engravers and/or
+performers.
+
+@node Engraver tutorial
+@section Engraver tutorial
+
+Engravers are C++ classes that catch music events and
+create the appropriate grobs for display on the page. Though the
+majority of engravers are responsible for the creation of a single grob,
+in some cases (e.g. @code{New_fingering_engraver}), several different grobs
+may be created.
+
+Engravers listen for events and acknowledge grobs. Events are passed to
+the engraver in time-step order during the iteration phase. Grobs are
+made available to the engraver when they are created by other engravers
+during the iteration phase.
+
+@subsection Useful methods for information processing
+
+An engraver inherits the following public methods from the Translator
+base class, which can be used to process listened events and acknowledged
+grobs:
+
+@itemize
+@item @code{virtual void initialize ()}
+@item @code{void start_translation_timestep ()}
+@item @code{void process_music ()}
+@item @code{void process_acknowledged ()}
+@item @code{void stop_translation_timestep ()}
+@item @code{virtual void finalize ()}
+@end itemize
+
+These methods are listed in order of translation time, with
+@code{initialize ()} and @code{finalize ()} bookending the whole
+process. @code{initialize ()} can be used for one-time initialization
+of context properties before translation starts, whereas
+@code{finalize ()} is often used to tie up loose ends at the end of
+translation: for example, an unterminated spanner might be completed
+automatically or reported with a warning message.
+
+@subsection Translation process
+
+At each timestep in the music, translation proceeds by calling the
+following methods in turn:
+
+@code{start_translation_timestep ()} is called before any user
+information enters the translators, i.e., no property operations
+(\set, \override, etc.) or events have been processed yet.
+
+@code{process_music ()} and @code{process_acknowledged ()} are called
+after all events in the current time step have been heard, or all
+grobs in the current time step have been acknowledged. The latter
+tends to be used exclusively with engravers which only acknowledge
+grobs, whereas the former is the default method for main processing
+within engravers.
+
+@code{stop_translation_timestep ()} is called after all user
+information has been processed prior to beginning the translation for
+the next timestep.
+
+@subsection Preventing garbage collection for SCM member variables
+
+In certain cases, an engraver might need to ensure private Scheme
+variables (with type SCM) do not get swept away by Guile's garbage
+collector: for example, a cache of the previous key signature which
+must persist between timesteps. The method
+@code{virtual derived_mark () const} can be used in such cases:
+
+@example
+Engraver_name::derived_mark ()
+@{
+ scm_gc_mark (private_scm_member_)
+@}
+@end example
+
+
+@subsection Listening to music events
+
+External interfaces to the engraver are implemented by protected
+macros including one or more of the following:
+
+@itemize
+@item @code{DECLARE_TRANSLATOR_LISTENER (event_name)}
+@item @code{IMPLEMENT_TRANSLATOR_LISTENER (Engraver_name, event_name)}
+@end itemize
+
+@noindent
+where @var{event_name} is the type of event required to provide the
+input the engraver needs and @var{Engraver_name} is the name of the
+engraver.
+
+Following declaration of a listener, the method is implemented as follows:
+
+@example
+IMPLEMENT_TRANSLATOR_LISTENER (Engraver_name, event_name)
+void
+Engraver_name::listen_event_name (Stream event *event)
+@{
+ ...body of listener method...
+@}
+@end example
+
+@subsection Acknowledging grobs
+
+Some engravers also need information from grobs as they are created
+and as they terminate. The mechanism and methods to obtain this
+information are set up by the macros:
+
+@itemize
+@item @code{DECLARE_ACKNOWLEDGER (grob_interface)}
+@item @code{DECLARE_END_ACKNOWLEDGER (grob_interface)}
+@end itemize
+
+where @var{grob_interface} is an interface supported by the
+grob(s) which should be acknowledged. For example, the following
+code would declare acknowledgers for a @code{NoteHead} grob (via the
+@code{note-head-interface}) and any grobs which support the
+@code{side-position-interface}:
+
+@example
+@code{DECLARE_ACKNOWLEDGER (note_head)}
+@code{DECLARE_ACKNOWLEDGER (side_position)}
+@end example
+
+The @code{DECLARE_END_ACKNOWLEDGER ()} macro sets up a spanner-specific
+acknowledger which will be called whenever a spanner ends.
+
+Following declaration of an acknowledger, the method is coded as follows:
+
+@example
+void
+Engraver_name::acknowledge_interface_name (Grob_info info)
+@{
+ ...body of acknowledger method...
+@}
+@end example
+
+@subsection Engraver declaration/documentation
+
+An engraver must have a public macro
+
+@itemize
+@item @code{TRANSLATOR_DECLARATIONS (Engraver_name)}
+@end itemize
+
+@noindent
+where @code{Engraver_name} is the name of the engraver. This
+defines the common variables and methods used by every engraver.
+
+At the end of the engraver file, one or both of the following
+macros are generally called to document the engraver in the
+Internals Reference:
+
+@itemize
+@item @code{ADD_ACKNOWLEDGER (Engraver_name, grob_interface)}
+@item @code{ADD_TRANSLATOR (Engraver_name, Engraver_doc,
+ Engraver_creates, Engraver_reads, Engraver_writes)}
+@end itemize
+
+@noindent
+where @code{Engraver_name} is the name of the engraver, @code{grob_interface}
+is the name of the interface that will be acknowledged,
+@code{Engraver_doc} is a docstring for the engraver,
+@code{Engraver_creates} is the set of grobs created by the engraver,
+@code{Engraver_reads} is the set of properties read by the engraver,
+and @code{Engraver_writes} is the set of properties written by
+the engraver.
+
+@node Callback tutorial
+@section Callback tutorial
+
+TODO -- This is a placeholder for a tutorial on callback functions.
+
+@node LilyPond scoping
+@section LilyPond scoping
+
+The Lilypond language has a concept of scoping, ie you can do
+
+@example
+foo = 1
+
+#(begin
+ (display (+ foo 2)))
+@end example
+
+@noindent with @code{\paper}, @code{\midi} and @code{\header} being
+nested scope inside the @file{.ly} file-level scope. @w{@code{foo = 1}}
+is translated in to a scheme variable definition.
+
+This implemented using modules, with each scope being an anonymous
+module that imports its enclosing scope's module.
+
+Lilypond's core, loaded from @file{.scm} files, is usually placed in the
+@code{lily} module, outside the @file{.ly} level. In the case of
+
+@example
+lilypond a.ly b.ly
+@end example
+
+@noindent
+we want to reuse the built-in definitions, without changes effected in
+user-level @file{a.ly} leaking into the processing of @file{b.ly}.
+
+The user-accessible definition commands have to take care to avoid
+memory leaks that could occur when running multiple files. All
+information belonging to user-defined commands and markups is stored in
+a manner that allows it to be garbage-collected when the module is
+dispersed, either by being stored module-locally, or in weak hash
+tables.
+
+@node LilyPond miscellany
+@section LilyPond miscellany
+
+This is a place to dump information that may be of use to developers
+but doesn't yet have a proper home. Ideally, the length of this section
+would become zero as items are moved to other homes.
+
+@subsection Spacing algorithms
+
+Here is information from an email exchange about spacing algorithms.
+
+On Thu, 2010-02-04 at 15:33 -0500, Boris Shingarov wrote:
+I am experimenting with some modifications to the line breaking code,
+and I am stuck trying to understand how some of it works. So far my
+understanding is that Simple_spacer operates on a vector of Grobs, and
+it is a well-known Constrained-QP problem (rods = constraints, springs
+= quadratic function to minimize). What I don't understand is, if the
+spacer operates at the level of Grobs, which are built at an earlier
+stage in the pipeline, how are the changes necessitated by differences
+in line breaking, taken into account? in other words, if I take the
+last measure of a line and place it on the next line, it is not just a
+matter of literally moving that graphic to where the start of the next
+line is, but I also need to draw a clef, key signature, and possibly
+other fundamental things -- but at that stage in the rendering
+pipeline, is it not too late??
+
+Joe Neeman answered:
+
+We create lots of extra grobs (eg. a BarNumber at every bar line) but
+most of them are not drawn. See the break-visibility property in
+item-interface.
+
+@subsection Info from Han-Wen Email
+
+In 2004, Douglas Linhardt decided to try starting a document that would
+explain LilyPond architecture and design principles. The material below
+is extracted from that email, which can be found at
+@uref{http://thread.gmane.org/gmane.comp.gnu.lilypond.devel/2992}.
+The headings reflect questions from Doug or comments from Han-Wen;
+the body text are Han-Wen's answers.
+
+@unnumberedsubsubsec Figuring out how things work.
+
+I must admit that when I want to know how a program works, I use grep
+and emacs and dive into the source code. The comments and the code
+itself are usually more revealing than technical documents.
+
+@unnumberedsubsubsec What's a grob, and how is one used?
+
+Graphical object - they are created from within engravers, either as
+Spanners (derived class) -slurs, beams- or Items (also a derived
+class) -notes, clefs, etc.
+
+There are two other derived classes System (derived from Spanner,
+contaning a "line of music") and Paper_column (derived from Item, it
+contains all items that happen at the same moment). They are separate
+classes because they play a special role in the linebreaking process.
+
+@unnumberedsubsubsec What's a smob, and how is one used?
+
+A C(++) object that is encapsulated so it can be used as a Scheme
+object. See GUILE info, "19.3 Defining New Types (Smobs)"
+
+@unnumberedsubsubsec When is each C++ class constructed and used
+
+@itemize
+
+@item
+Music classes
+
+In the parser.yy see the macro calls MAKE_MUSIC_BY_NAME().
+
+@item
+Contexts
+
+Constructed during "interpreting" phase.
+
+@item
+Engravers
+
+Executive branch of Contexts, plugins that create grobs, usually one
+engraver per grob type. Created together with context.
+
+@item
+Layout Objects
+
+= grobs
+
+@item
+Grob Interfaces
+
+These are not C++ classes per se. The idea of a Grob interface hasn't
+crystallized well. ATM, an interface is a symbol, with a bunch of grob
+properties. They are not objects that are created or destroyed.
+
+@item
+Iterators
+
+Objects that walk through different music classes, and deliver events
+in a synchronized way, so that notes that play together are processed
+at the same moment and (as a result) end up on the same horizontal position.
+
+Created during interpreting phase.
+
+BTW, the entry point for interpreting is ly:run-translator
+(ly_run_translator on the C++ side)
+
+@end itemize
+
+@unnumberedsubsubsec Can you get to Context properties from a Music object?
+
+You can create music object with a Scheme function that reads context
+properties (the \applycontext syntax). However, that function is
+executed during Interpreting, so you can not really get Context
+properties from Music objects, since music objects are not directly
+connected to Contexts. That connection is made by the Music_iterators
+
+@unnumberedsubsubsec Can you get to Music properties from a Context object?
+
+Yes, if you are given the music object within a Context
+object. Normally, the music objects enter Contexts in synchronized
+fashion, and the synchronization is done by Music_iterators.
+
+@unnumberedsubsubsec What is the relationship between C++ classes and Scheme objects?
+
+Smobs are C++ objects in Scheme. Scheme objects (lists, functions) are
+manipulated from C++ as well using the GUILE C function interface
+(prefix: scm_)
+
+@unnumberedsubsubsec How do Scheme procedures get called from C++ functions?
+
+scm_call_*, where * is an integer from 0 to 4.
+Also scm_c_eval_string (), scm_eval ()
+
+@unnumberedsubsubsec How do C++ functions get called from Scheme procedures?
+
+Export a C++ function to Scheme with LY_DEFINE.
+
+@unnumberedsubsubsec What is the flow of control in the program?
+
+Good question. Things used to be clear-cut, but we have Scheme
+and SMOBs now, which means that interactions do not follow a very
+rigid format anymore. See below for an overview, though.
+
+@unnumberedsubsubsec Does the parser make Scheme procedure calls or C++ function calls?
+
+Both. And the Scheme calls can call C++ and vice versa. It's nested,
+with the SCM datatype as lubrication between the interactions
+
+(I think the word "lubrication" describes the process better than the
+traditional word "glue")
+
+@unnumberedsubsubsec How do the front-end and back-end get started?
+
+Front-end: a file is parsed, the rest follows from that. Specifically,
+
+Parsing leads to a Music + Music_output_def object (see parser.yy,
+definition of toplevel_expression )
+
+A Music + Music_output_def object leads to a Global_context object (see
+ly_run_translator ())
+
+During interpreting, Global_context + Music leads to a bunch of
+Contexts. (see Global_translator::run_iterator_on_me () )
+
+After interpreting, Global_context contains a Score_context (which
+contains staves, lyrics etc.) as a child. Score_context::get_output ()
+spews a Music_output object (either a Paper_score object for notation
+or Performance object for MIDI).
+
+The Music_output object is the entry point for the backend. (see
+ly_render_output () )
+
+The main steps of the backend itself are in
+
+@itemize
+
+@item
+paper-score.cc , Paper_score::process_
+
+@item
+system.cc , System::get_lines()
+
+@item
+The step, where things go from grobs to output, is in
+System::get_line(): each grob delivers a Stencil (a Device
+independent output description), which is interpreted by our
+outputting backends (scm/output-tex.scm and scm/output-ps.scm)
+to produce TeX and PS.
+
+@end itemize
+
+Interactions between grobs and putting things into .tex and .ps files
+have gotten a little more complex lately. Jan has implemented
+page-breaking, so now the backend also involves Paper_book,
+Paper_lines and other things. This area is still heavily in flux, and
+perhaps not something you should want to look at.
+
+@unnumberedsubsubsec How do the front-end and back-end communicate?
+
+There is no communication from backend to front-end. From front-end to
+backend is simply the program flow: music + definitions gives
+contexts, contexts yield output, after processing, output is written
+to disk.
+
+@unnumberedsubsubsec Where is the functionality associated with KEYWORDs?
+
+See my-lily-lexer.cc (keywords, there aren't that many) and ly/*.ly
+(most of the other backslashed \words are identifiers)
+
+@unnumberedsubsubsec What Contexts/Properties/Music/etc. are available when they are processed?
+
+What do you mean exactly with this question?
+
+See ly/engraver-init.ly for contexts, see scm/define-*.scm for other
+objects.
+
+@unnumberedsubsubsec How do you decide if something is a Music, Context, or Grob property?
+Why is part-combine-status a Music property when it seems (IMO)
+to be related to the Staff context?
+
+The Music_iterators and Context communicate through two channels
+
+Music_iterators can set and read context properties, idem for
+Engravers and Contexts
+
+Music_iterators can send "synthetic" music events (which aren't in
+the input) to a context. These are caught by Engravers. This is
+mostly a one way communication channel.
+
+part-combine-status is part of such a synthetic event, used by
+Part_combine_iterator to communicate with Part_combine_engraver.
+
+
+@unnumberedsubsubsec I'm adding a property to affect how \autochange works. It seems to
+me that it should be a context property, but the Scheme autochange
+procecure has a Music argument. Does this mean I should use
+a Music property?
+
+\autochange is one of these extra strange beasts: it requires
+look-ahead to decide when to change staves. This is achieved by
+running the interpreting step twice (see scm/part-combiner.scm , at
+the bottom), and storing the result of the first step (where to switch
+staves) in a Music property. Since you want to influence that
+where-to-switch list, your must affect the code in
+make-autochange-music (scm/part-combiner.scm). That code is called
+directly from the parser and there are no official "parsing
+properties" yet, so there is no generic way to tune \autochange. We
+would have to invent something new for this, or add a separate
+argument,
+
+@example
+ \autochange #around-central-C ..music..
+@end example
+
+@noindent
+where around-central-C is some function that is called from
+make-autochange-music.
+
+@unnumberedsubsubsec I get lost figuring out what environment the code I'm looking at is in when it executes.
+I found both the C++ and Scheme autochange code. Then I was
+trying to figure out where the code got called from. I finally figured out that
+the Scheme procedure was called before the C++ iterator code, but it took me a
+while to figure that out, and I still didn't know who did the calling in the
+first place. I only know a little bit about Flex and Bison, so reading those
+files helped only a little bit.
+
+@emph{Han-Wen:} GDB can be of help here. Set a breakpoint in C++, and run. When you
+hit the breakpoint, do a backtrace. You can inspect Scheme objects
+along the way by doing
+
+@example
+p ly_display_scm(obj)
+@end example
+
+this will display OBJ through GUILE.
+
+@subsection Music functions and GUILE debugging
+
+Ian Hulin was trying to do some debugging in music functions, and
+came up with the following question
+
+HI all,
+I'm working on the Guile Debugger Stuff, and would like to try
+debugging a music function definition such as:
+
+@example
+conditionalMark = #(define-music-function (parser location) ()
+ #@{ \tag #'instrumental-part @{\mark \default@} #@} )
+@end example
+
+It appears conditionalMark does not get set up as an
+equivalent of a Scheme
+
+@example
+(define conditionalMark = define-music-function(parser location () ...
+@end example
+
+@noindent
+although something gets defined because Scheme apparently recognizes
+
+@example
+#(set-break! conditionalMark)
+@end example
+
+@noindent
+later on in the file without signalling any Guile errors.
+
+However the breakpoint trap is never encountered as
+define-music-function passed things on to ly:make-music-function,
+which is really C++ code ly_make_music_function, so Guile never
+finds out about the breakpoint.
+
+Han-Wen answered as follows:
+
+You can see the defintion by doing
+
+@example
+#(display conditionalMark)
+@end example
+
+noindent
+inside the .ly file.
+
+The breakpoint failing may have to do with the call sequence. See
+parser.yy, run_music_function(). The function is called directly from
+C++, without going through the GUILE evaluator, so I think that is why
+there is no debugger trap.
projects / lilypond.git / blobdiff