* Programming without compiling::
* Finding functions::
* Code style::
+* Warnings Errors Progress and Debug Output::
* Debugging LilyPond::
* Tracing object relationships::
* Adding or modifying features::
* Iterator tutorial::
* Engraver tutorial::
* Callback tutorial::
+* Understanding pure properties::
* LilyPond scoping::
+* Scheme->C interface::
* LilyPond miscellany::
@end menu
accomplish the various stages of the process, is described in this section. A
more complete description of the LilyPond architecture and internal program
execution is found in Erik Sandberg's
-@uref{http://lilypond.org/web/images/thesis-erik-sandberg.pdf, master's
+@uref{http://lilypond.org/website/pdf/thesis-erik-sandberg.pdf, master's
thesis}.
The first stage of LilyPond processing is @emph{parsing}. In the parsing
begins.
Each type of music event has an associated iterator. Iterators are defined in
-*-iterator.cc. During iteration, an
+@file{*-iterator.cc}. During iteration, an
event's iterator is called to deliver that music event to the appropriate
context(s).
through its two derived classes: Engraver (for graphical output) and
Performer (for midi output).
-Translators are defined in C++ files named *-engraver.cc and *-performer.cc.
+Translators are defined in C++ files named @file{*-engraver.cc}
+and @file{*-performer.cc}.
Much of the work of translating is handled by Scheme functions,
which is one of the keys to LilyPond's exceptional flexibility.
PostScript is used to generate graphical output. A brief PostScript tutorial
is @uref{http://local.wasp.uwa.edu.au/~pbourke/dataformats/postscript/,
available online}. The
-@uref{http://www.adobe.com/devnet/postscript/pdfs/PLRM.pdf, PostScript Lanugage
+@uref{http://www.adobe.com/devnet/postscript/pdfs/PLRM.pdf, PostScript Language
Reference} is available online in PDF format.
@subsection Python
-Python is used for XML2ly and is used for buillding the documentation and the
+Python is used for XML2ly and is used for building the documentation and the
website.
Python documentation is available at @uref{http://www.python.org/doc/,
@node Programming without compiling
@section Programming without compiling
-Much of the development work in LilyPond takes place by changing *.ly or
-*.scm files. These changes can be made without compiling LilyPond. Such
+Much of the development work in LilyPond takes place by changing @file{*.ly} or
+@file{*.scm} files. These changes can be made without compiling LilyPond. Such
changes are described in this section.
Much of LilyPond is written in Scheme or LilyPond input files. These
files are interpreted when the program is run, rather than being compiled
when the program is built, and are present in all LilyPond distributions.
-You will find .ly files in the ly/ directory and the Scheme files in the
-scm/ directory. Both Scheme files and .ly files can be modified and
+You will find @file{.ly} files in the @file{ly/} directory and the Scheme files in the
+@file{scm/} directory. Both Scheme files and @file{.ly} files can be modified and
saved with any text editor. It's probably wise to make a backup copy of
your files before you modify them, although you can reinstall if the
files become corrupted.
When making changes or fixing bugs in LilyPond, one of the initial
challenges is finding out where in the code tree the functions to
be modified live. With nearly 3000 files in the source tree,
-trial-and-error searching is generally ineffective. This section
+trial-and-error searching is generally ineffective. This section
describes a process for finding interesting code.
@subsection Using the ROADMAP
The file ROADMAP is located in the main directory of the lilypond source.
-ROADMAP lists all of the directories in the LilPond source tree, along
+ROADMAP lists all of the directories in the LilyPond source tree, along
with a brief description of the kind of files found in each directory.
This can be a very helpful tool for deciding which directories to search
when looking for a function.
This command will search all the contents of the directory subdirectory/
and display every line in any of the files that contains
-functionName. The @code{-i} option makes @command{grep} ignore
+functionName. The @option{-i} option makes @command{grep} ignore
case -- this can be very useful if you are not yet familiar with
our capitalization conventions.
-The most likely directories to grep for function names are scm/ for
-scheme files, ly/ for lilypond input (*.ly) files, and lily/ for C++
+The most likely directories to grep for function names are @file{scm/} for
+scheme files, ly/ for lilypond input (@file{*.ly}) files, and @file{lily/} for C++
files.
@node Code style
@section Code style
+This section describes style guidelines for LilyPond
+source code.
+
@menu
-* Handling errors::
* Languages::
* Filenames::
* Indentation::
-* Indenting files with fixcc.py::
-* Indenting files with emacs in script mode::
-* Indenting with vim::
* Naming conventions::
* Broken code::
* Code comments::
-* Messages::
+* Handling errors::
* Localization::
@end menu
-@node Handling errors
-@subsection Handling errors
-
-As a general rule, you should always try to continue computations,
-even if there is some kind of error. When the program stops, it
-is often very hard for a user to pinpoint what part of the input
-causes an error. Finding the culprit is much easier if there is
-some viewable output.
-
-So functions and methods do not return errorcodes, they never
-crash, but report a programming_error and try to carry on.
-
-
@node Languages
@subsection Languages
@node Indentation
@subsection Indentation
-Standard GNU coding style is used. In emacs:
+Standard GNU coding style is used.
-@verbatim
- (add-hook 'c++-mode-hook
- '(lambda() (c-set-style "gnu")
- ))
-@end verbatim
+@subsubheading Indenting files with @code{fixcc.py} (recommended)
-If you like using font-lock, you can also add this to your
-@q{.emacs}:
-
-@verbatim
- (setq font-lock-maximum-decoration t)
- (setq c++-font-lock-keywords-3
- (append
- c++-font-lock-keywords-3
- '(("\\b\\(a-zA-Z_?+_\\)\\b" 1 font-lock-variable-name-face) ("\\b\\(A-Z?+a-z_?+\\)\\b" 1 font-lock-type-face))
- ))
-@end verbatim
-
-Some source files may not currently have proper indenting. If this
-is the case, it is desirable to fix the improper indenting when the
-file is modified, with the hope of continually improving the code.
-
-
-@node Indenting files with fixcc.py
-@subsection Indenting files with fixcc.py
-
-LilyPond provides a python script that will correct the indentation
-on a c++ file:
+LilyPond provides a python script that will adjust the indentation
+and spacing on a @code{.cc} or @code{.hh} file to very near the
+GNU standard:
@example
-scripts/auxiliar/fixcc.py lily/my-test-file.cc
+scripts/auxiliar/fixcc.py FILENAME
@end example
-Be sure you replace @code{my-test-file.cc} with the name of the file
-that you edited.
-
-If you are editing a file that contains an ADD_TRANSLATOR or ADD_INTERFACE
-macro, the fixcc.py script will move the final parenthesis up one line
-from where it should be. Please check the end of the file before
-you run fixcc.py, and then put the final parenthesis and semicolon
-back on a line by themselves.
-
+This can be run on all files at once, but this is not recommended
+for normal contributors or developers.
-@node Indenting files with emacs in script mode
-@subsection Indenting files with emacs in script mode
+@smallexample
+scripts/auxiliar/fixcc.py \
+ $(find flower lily -name '*cc' -o -name '*hh' | grep -v /out)
+@end smallexample
-@c email to wl@gnu.org when I get here.
-@warning{this is pending some confirmation on -devel. July 2009 -gp}
+@subsubheading Indenting with emacs
-Command-line script to format stuff with emacs:
+The following hooks will produce indentation which is similar to
+our official indentation as produced with @code{fixcc.py}.
@example
-#!/bin/sh
-emacs $1 -batch --eval '(indent-region (point-min) (point-max) nil)' -f save-buffer
+(add-hook 'c++-mode-hook
+ '(lambda ()
+ (c-set-style "gnu")
+ (setq indent-tabs-mode nil))
@end example
-(that's all on one line)
+If you like using font-lock, you can also add this to your
+@file{.emacs}:
-Save it as a shell script, then run on the file(s) you modified.
+@example
+(setq font-lock-maximum-decoration t)
+(setq c++-font-lock-keywords-3
+ (append
+ c++-font-lock-keywords-3
+ '(("\\b\\(a-zA-Z_?+_\\)\\b" 1 font-lock-variable-name-face) ("\\b\\(A-Z?+a-z_?+\\)\\b" 1 font-lock-type-face))
+ ))
+@end example
-@node Indenting with vim
-@subsection Indenting with vim
+@subheading Indenting with vim
-Although emacs indentation is the LilyPond standard, acceptable
+Although emacs indentation is the GNU standard, acceptable
indentation can usually be accomplished with vim. Some hints for
vim are as follows:
A workable .vimrc:
-@verbatim
+@example
set cindent
set smartindent
set autoindent
set ignorecase smartcase
set hlsearch
set confirm
-set statusline=%F%m%r%h%w\ %{&ff}\ %Y\ [ASCII=\%03.3b]\ [HEX=\%02.2B]\ %04l,%04v\ %p%%\ [LEN=%L]
+set statusline=%F%m%r%h%w\ %@{&ff@}\ %Y\ [ASCII=\%03.3b]\ [HEX=\%02.2B]\ %04l,%04v\ %p%%\ [LEN=%L]
set laststatus=2
set number
" Remove trailing whitespace on write
autocmd BufWritePre * :%s/\s\+$//e
-@end verbatim
+@end example
-With this .vimrc, files can be reindented automatically by highlihting
-the lines to be indented in visual mode (use V to enter visual mode)
-and pressing =.
+With this @file{.vimrc}, files can be reindented automatically by
+highlighting the lines to be indented in visual mode (use V to
+enter visual mode) and pressing @code{=}.
-A scheme.vim file will help improve the indentation. This one
-was suggested by Patrick McCarty. It should be saved in
-~/.vim/after/syntax/scheme.vim.
+A @file{scheme.vim} file will help improve the indentation. This
+one was suggested by Patrick McCarty. It should be saved in
+@file{~/.vim/after/syntax/scheme.vim}.
-@verbatim
+@example
" Additional Guile-specific 'forms'
-syn keyword schemeSyntax define-public define* define-safe-public
+syn keyword schemeSyntax define-public define*-public
+syn keyword schemeSyntax define* lambda* let-keywords*
+syn keyword schemeSyntax defmacro defmacro* define-macro
+syn keyword schemeSyntax defmacro-public defmacro*-public
syn keyword schemeSyntax use-modules define-module
-syn keyword schemeSyntax defmacro-public define-macro
-syn keyword schemeSyntax define-markup-command
-syn keyword schemeSyntax define-markup-list-command
-syn keyword schemeSyntax let-keywords* lambda* define*-public
-syn keyword schemeSyntax defmacro* defmacro*-public
+syn keyword schemeSyntax define-method define-class
+
+" Additional LilyPond-specific 'forms'
+syn keyword schemeSyntax define-markup-command define-markup-list-command
+syn keyword schemeSyntax define-safe-public define-music-function
+syn keyword schemeSyntax def-grace-function
" All of the above should influence indenting too
-set lw+=define-public,define*,define-safe-public,use-modules,define-module
-set lw+=defmacro-public,define-macro
+set lw+=define-public,define*-public
+set lw+=define*,lambda*,let-keywords*
+set lw+=defmacro,defmacro*,define-macro
+set lw+=defmacro-public,defmacro*-public
+set lw+=use-modules,define-module
+set lw+=define-method,define-class
set lw+=define-markup-command,define-markup-list-command
-set lw+=let-keywords*,lambda*,define*-public,defmacro*,defmacro*-public
+set lw+=define-safe-public,define-music-function
+set lw+=def-grace-function
" These forms should not influence indenting
set lw-=if
" Try to highlight all ly: procedures
syn match schemeFunc "ly:[^) ]\+"
-@end verbatim
+@end example
@node Naming conventions
@subsection Naming Conventions
+Naming conventions have been established for LilyPond
+source code.
+
@subheading Classes and Types
+Classes begin with an uppercase letter, and words
+in class names are separated with @code{_}:
+
@verbatim
This_is_a_class
@end verbatim
@subheading Macros
-Macro names should be written in uppercase completely.
+Macro names should be written in uppercase completely,
+with words separated by @code{_}:
+
+@verbatim
+THIS_IS_A_MACRO
+@end verbatim
@subheading Variables
@code{th} or @code{t}.
Multi-word variable names in C++ should have the words separated
-by the underscore character (@q{_}).
+by the underscore character (@q{_}):
+
+@verbatim
+cxx_multiword_variable
+@end verbatim
Multi-word variable names in Scheme should have the words separated
-by a hyphen (@q{-}).
+by a hyphen (@q{-}):
+@verbatim
+scheme-multiword-variable
+@end verbatim
@node Broken code
@subsection Broken code
you can not avoid it, mark the place clearly, and add a comment
explaining shortcomings of the code.
+Ideally, the comment marking the shortcoming would include
+TODO, so that it is marked for future fixing.
+
We reject broken-in-advance on principle.
understanding to make future work easier.
-@node Messages
-@subsection Messages
+@node Handling errors
+@subsection Handling errors
-Messages need to follow Localization.
+As a general rule, you should always try to continue computations,
+even if there is some kind of error. When the program stops, it
+is often very hard for a user to pinpoint what part of the input
+causes an error. Finding the culprit is much easier if there is
+some viewable output.
+
+So functions and methods do not return errorcodes, they never
+crash, but report a programming_error and try to carry on.
+
+Error and warning messages need to be localized.
@node Localization
guide helping programmers writing consistent messages for all GNU
programs.
-Non-preferred messages are marked with `+'. By convention,
+Non-preferred messages are marked with `+'. By convention,
ungrammatical examples are marked with `*'. However, such ungrammatical
examples may still be preferred.
@item
Every message to the user should be localized (and thus be marked
-for localization). This includes warning and error messages.
+for localization). This includes warning and error messages.
@item
Do not localize/gettextify:
error (_f ("cannot open file: `%s'", file_name));
@end example
-In some rare cases you may need to call `gettext ()' by hand. This
+In some rare cases you may need to call `gettext ()' by hand. This
happens when you pre-define (a list of) string constants for later
-use. In that case, you'll probably also need to mark these string
-constants for translation, using `_i (STRING)'. The `_i' macro is
+use. In that case, you'll probably also need to mark these string
+constants for translation, using `_i (STRING)'. The `_i' macro is
a no-op, it only serves as a marker for `xgettext'.
@example
@}
@end example
-See also `flower/getopt-long.cc' and `lily/main.cc'.
+See also @file{flower/getopt-long.cc} and @file{lily/main.cc}.
@item
-Do not use leading or trailing whitespace in messages. If you need
+Do not use leading or trailing whitespace in messages. If you need
whitespace to be printed, prepend or append it to the translated
message
@end example
Messages containing a final verb, or a gerund (`-ing'-form) always
-start with a capital. Other (simpler) messages start with a
+start with a capital. Other (simpler) messages start with a
lowercase letter
@example
rather than `can't' or `don't'
To avoid having a number of different messages for the same
situation, well will use quoting like this `"message: `%s'"' for all
-strings. Numbers are not quoted:
+strings. Numbers are not quoted:
@example
_f ("cannot open file: `%s'", name_str)
@end example
@item
-Think about translation issues. In a lot of cases, it is better to
-translate a whole message. The english grammar must not be imposed
-on the translator. So, instead of
+Think about translation issues. In a lot of cases, it is better to
+translate a whole message. English grammar must not be imposed on the
+translator. So, instead of
@example
stem at + moment.str () + does not fit in beam
@end example
@item
-Split up multi-sentence messages, whenever possible. Instead of
+Split up multi-sentence messages, whenever possible. Instead of
@example
warning (_f ("out of tune! Can't find: `%s'", "Key_engraver"));
@item
If you must have multiple-sentence messages, use full punctuation.
-Use two spaces after end of sentence punctuation. No punctuation
+Use two spaces after end of sentence punctuation. No punctuation
(esp. period) is used at the end of simple messages.
@example
@item
Do not modularize too much; words frequently cannot be translated
-without context. It is probably safe to treat most occurences of
+without context. It is probably safe to treat most occurrences of
words like stem, beam, crescendo as separately translatable words.
@item
When translating, it is preferable to put interesting information
at the end of the message, rather than embedded in the middle.
This especially applies to frequently used messages, even if this
-would mean sacrificing a bit of eloquency. This holds for original
+would mean sacrificing a bit of eloquence. This holds for original
messages too, of course.
@example
The first nl message, although grammatically and stylistically
correct, is not friendly for parsing by humans (even if they speak
-dutch). I guess we would prefer something like (2) or (3).
+dutch). I guess we would prefer something like (2) or (3).
@item
Do not run make po/po-update with GNU gettext < 0.10.35
@end itemize
+@node Warnings Errors Progress and Debug Output
+@section Warnings, Errors, Progress and Debug Output
+
+@unnumberedsubsec Available log levels
+
+LilyPond has several loglevels, which specify how verbose the output on
+the console should be:
+@itemize
+@item NONE: No output at all, even on failure
+@item ERROR: Only error messages
+@item WARN: Only error messages and warnings
+@item BASIC_PROGRESS: Warnings, errors and basic progress (success, etc.)
+@item PROGRESS: Warnings, errors and full progress messages
+@item INFO: Warnings, errors, progress and more detailed information (default)
+@item DEBUG: All messages, including full debug messages (very verbose!)
+@end itemize
+
+The loglevel can either be set with the environment variable
+@code{LILYPOND_LOGLEVEL} or on the command line with the @option{--loglevel=...}
+option.
+
+@unnumberedsubsec Functions for debug and log output
+
+LilyPond has two different types of error and log functions:
+@itemize
+
+@item
+If a warning or error is caused by an identified position in the input file,
+e.g. by a grob or by a music expression, the functions of the @code{Input}
+class provide logging functionality that prints the position of the message
+in addition to the message.
+
+@item
+If a message can not be associated with a particular position in an input file,
+e.g. the output file cannot be written, then the functions in the
+@code{flower/include/warn.hh} file will provide logging functionality that
+only prints out the message, but no location.
+
+@end itemize
+
+There are also Scheme functions to access all of these logging functions from
+scheme. In addition, the Grob class contains some convenience wrappers for
+even easier access to these functions.
+
+The message and debug functions in @code{warn.hh} also have an optional
+argument @code{newline}, which specifies whether the message should always
+start on a new line or continue a previous message.
+By default, @code{progress_indication} does NOT start on a new line, but rather
+continue the previous output. They also do not have a particular input
+position associated, so there are no progress functions in the Input class.
+All other functions by default start their output on a new line.
+
+The error functions come in three different flavors: fatal error messages,
+programming error messages and normal error messages. Errors written
+by the @code{error ()} function will cause LilyPond to exit immediately,
+errors by @code{Input::error ()} will continue the compilation, but
+return a non-zero return value of the lilypond call (i.e. indicate an
+unsuccessful program execution). All other errors will be printed on the
+console, but not exit LilyPond or indicate an unsuccessful return code.
+Their only differences to a warnings are the displayed text and that
+they will be shown with loglevel @code{ERROR}.
+
+If the Scheme option @code{warning-as-error} is set, any warning will be
+treated as if @code{Input::error} was called.
+
+
+@unnumberedsubsec All logging functions at a glance
+
+@multitable @columnfractions 0.16 0.42 0.42
+@headitem
+@tab C++, no location
+@tab C++ from input location
+
+@item ERROR
+@tab @code{error ()}, @code{programming_error (msg)}, @code{non_fatal_error (msg)}
+@tab @code{Input::error (msg)}, @code{Input::programming_error (msg)}
+
+@item WARN
+@tab @code{warning (msg)}
+@tab @code{Input::warning (msg)}
+
+@item BASIC
+@tab @code{basic_progress (msg)}
+@tab -
+
+@item PROGRESS
+@tab @code{progress_indication (msg)}
+@tab -
+
+@item INFO
+@tab @code{message (msg)}
+@tab @code{Input::message (msg)}
+
+@item DEBUG
+@tab @code{debug_output (msg)}
+@tab @code{Input::debug_output (msg)}
+
+@item @tab @tab
+
+@headitem
+@tab C++ from a Grob
+@tab Scheme, music expression
+
+@item ERROR
+@tab @code{Grob::programming_error (msg)}
+@tab -
+
+@item WARN
+@tab @code{Grob::warning (msg)}
+@tab @code{(ly:music-warning music msg)}
+
+@item BASIC
+@tab -
+@tab -
+
+@item PROGRESS
+@tab -
+@tab -
+
+@item INFO
+@tab -
+@tab @code{(ly:music-message music msg)}
+
+@item DEBUG
+@tab -
+@tab -
+
+@item @tab @tab
+
+@headitem
+@tab Scheme, no location
+@tab Scheme, input location
+
+@item ERROR
+@tab -
+@tab @code{(ly:error msg args)}, @code{(ly:programming-error msg args)}
+
+@item WARN
+@tab @code{(ly:warning msg args)}
+@tab @code{(ly:input-warning input msg args)}
+
+@item BASIC
+@tab @code{(ly:basic-progress msg args)}
+@tab -
+
+@item PROGRESS
+@tab @code{(ly:progress msg args)}
+@tab -
+
+@item INFO
+@tab @code{(ly:message msg args)}
+@tab @code{(ly:input-message input msg args)}
+
+@item DEBUG
+@tab @code{(ly:debug msg args)}
+@tab -
+
+@end multitable
+
+
+
@node Debugging LilyPond
@section Debugging LilyPond
The GNU debugger, gdb, is the principal tool for debugging C++ code.
-@unnumberedsubsubsec Compiling LilyPond for use with gdb
+@subheading Compiling LilyPond for use with gdb
In order to use gdb with LilyPond, it is necessary to compile
LilyPond with debugging information. This is accomplished by running
with LilyPond. The @var{make install} command will strip debugging
information from the LilyPond binary.
-@unnumberedsubsubsec Typical gdb usage
+@subheading Typical gdb usage
Once you have compiled the Lilypond image with the necessary
debugging information it will have been written to a location in a
This is important as you will need to let gdb know where to find the
image containing the symbol tables. You can invoke gdb from the
-command line using
+command line using the following:
@example
gdb out/bin/lilypond
@end example
@noindent
This loads the LilyPond symbol tables into gdb. Then, to run
-LilyPond on @code{test.ly} under the debugger, enter
+LilyPond on @file{test.ly} under the debugger, enter the following:
@example
run test.ly
@end example
+
@noindent
at the gdb prompt.
As an alternative to running gdb at the command line you may try
-a graphical interface to gdb such as ddd
+a graphical interface to gdb such as ddd:
@example
ddd out/bin/lilypond
You can also use sets of standard gdb commands stored in a .gdbinit
file (see next section).
-@unnumberedsubsubsec Typical .gdbinit files
+@subheading Typical .gdbinit files
The behavior of gdb can be readily customized through the use of a
@var{.gdbinit} file. A @var{.gdbinit} file is a file named
interactively using just Guile or you can use the debugging
tools available within Guile.
-@unnumberedsubsubsec Using Guile interactively with LilyPond
+@subheading Using Guile interactively with LilyPond
In order to experiment with Scheme programming in the LilyPond
environment, it is necessary to have a Guile interpreter that
has all the LilyPond modules loaded. This requires the following
steps.
-First, define a Scheme symbol for the active module in the .ly file:
+First, define a Scheme symbol for the active module in the @file{.ly} file:
@example
#(module-define! (resolve-module '(guile-user))
'lilypond-module (current-module))
@end example
-Now place a Scheme function in the .ly file that gives an
+Now place a Scheme function in the @file{.ly} file that gives an
interactive Guile prompt:
@example
#(top-repl)
@end example
-When the .ly file is compiled, this causes the compilation to be
+When the @file{.ly} file is compiled, this causes the compilation to be
interrupted and an interactive guile prompt to appear. Once the
guile prompt appears, the LilyPond active module must be set as the
current guile module:
guile> (quit)
@end example
-The compilation of the .ly file will then continue.
+The compilation of the @file{.ly} file will then continue.
-@unnumberedsubsubsec Using the Guile debugger
+@subheading Using the Guile debugger
To set breakpoints and/or enable tracing in Scheme functions, put
is processed, a guile prompt will be displayed. You may now enter
commands to set up breakpoints and enable tracing by the Guile debugger.
-@unnumberedsubsubsec Using breakpoints
+@subheading Using breakpoints
At the guile prompt, you can set breakpoints with
the @code{set-break!} procedure:
command at the guile prompt.
Embedding breakpoint commands like this is particularly useful if
-you want to look at how the Scheme procedures in the @var{.scm}
+you want to look at how the Scheme procedures in the @file{.scm}
files supplied with LilyPond work. To do this, edit the file in
the relevant directory to add this line near the top:
Now you can set a breakpoint after the procedure you are interested
in has been declared. For example, if you are working on routines
-called by @var{print-book-with} in @var{lily-library.scm}:
+called by @var{print-book-with} in @file{lily-library.scm}:
@example
(define (print-book-with parser book process-procedure)
(set-break! print-book-with)
@end example
-@unnumberedsubsubsec Tracing procedure calls and evaluator steps
+@subheading Tracing procedure calls and evaluator steps
Two forms of trace are available:
@section Tracing object relationships
Understanding the LilyPond source often boils down to figuring out what
-is happening to the Grobs. Where (and why) are they being created,
+is happening to the Grobs. Where (and why) are they being created,
modified and destroyed? Tracing Lily through a debugger in order to
identify these relationships can be time-consuming and tedious.
@item Installing graphviz
In order to create the graph of the object relationships, it is
-first necessary to install Graphviz. graphviz is available for a
+first necessary to install Graphviz. Graphviz is available for a
number of different platforms:
@example
In order for the Graphviz tool to work, config.make must be modified.
It is probably a good idea to first save a copy of config.make under
-a different name. Then, edit config.make by removing every occurence
-of @code{-DNDEBUG}.
+a different name. Then, edit config.make by removing every occurrence
+of @option{-DNDEBUG}.
@item Rebuilding LilyPond
make -C lily clean && make -C lily
@end example
-@item Create a graphviz-compatible .ly file
+@item Create a graphviz-compatible @file{.ly} file
-In order to use the graphviz utility, the .ly file must include
+In order to use the graphviz utility, the @file{.ly} file must include
@file{ly/graphviz-init.ly}, and should then specify the
grobs and symbols that should be tracked. An example of this
is found in @file{input/regression/graphviz.ly}.
The logfile has standard lilypond output, as well as the Graphviz
output data. Delete everything from the beginning of the file
-up to but not including the first occurence of @code{digraph}.
+up to but not including the first occurrence of @code{digraph}.
+
+Also, delete the final lilypond message about success from the end
+of the file.
@item Process the logfile with @code{dot}
The pdf file can then be viewed with any pdf viewer.
-When compiled without @code{-DNDEBUG}, lilypond may run slower
+When compiled without @option{-DNDEBUG}, lilypond may run slower
than normal. The original configuration can be restored by either
renaming the saved copy of @code{config.make} or rerunning
@code{configure}. Then rebuild lilypond with
its long-term support. This section describes the steps necessary
for feature addition and modification.
+
+@menu
+* Write the code::
+* Write regression tests::
+* Write convert-ly rule::
+* Automatically update documentation::
+* Manually update documentation::
+* Edit changes.tely::
+* Verify successful build::
+* Verify regression tests::
+* Post patch for comments::
+* Push patch::
+* Closing the issues::
+@end menu
+
+@node Write the code
@subsection Write the code
You should probably create a new git branch for writing the code, as that
Please be sure to follow the rules for programming style discussed
earlier in this chapter.
+
+@node Write regression tests
@subsection Write regression tests
In order to demonstrate that the code works properly, you will
need to write one or more regression tests. These tests are
-typically .ly files that are found in input/regression.
+typically @file{.ly} files that are found in @file{input/regression}.
Regression tests should be as brief as possible to demonstrate the
functionality of the code.
Use existing regression tests as templates to demonstrate the type of
header information that should be included in a regression test.
+
+@node Write convert-ly rule
@subsection Write convert-ly rule
If the modification changes the input syntax, a convert-ly rule
rule will simply point out to the user that the feature needs
manual correction.
-@subsection Automatically update documentation, snippets, and regtests
-
-convert-ly should be used to update the documentation, the snippets,
-and the regression tests. This not only makes the necessary syntax
-changes, it also tests the convert-ly rules.
+@subsubheading Updating version numbers
-The automatic updating is a three step process. First, be sure you
-are in the top-level source directory. Then, for the
-documentation, do:
+If a development release occurs between you writing your patch and
+having it approved+pushed, you will need to update the version
+numbers in your tree. This can be done with:
@example
-find Documentation/ -name '*.itely' | xargs convert-ly -e --from @qq{@var{X.Y.Z}}
+scripts/auxiliar/update-patch-version old.version.number new.version.number
@end example
-@noindent
-where @var{X.Y.Z} is the version number of the last released development
-version.
+It will change all files in git, so use with caution and examine
+the resulting diff.
+
+
+@node Automatically update documentation
+@subsection Automatically update documentation
+
+@command{convert-ly} should be used to update the documentation,
+the snippets, and the regression tests. This not only makes the
+necessary syntax changes, it also tests the @command{convert-ly}
+rules.
-Next, for the snippets, do:
+The automatic updating is performed by moving to the top-level
+source directory, then running:
@example
-find Documentation/snippets/ -name '*.ly' | xargs convert-ly -e --from @qq{@var{X.Y.Z}}
+scripts/auxiliar/update-with-convert-ly.sh
@end example
-Finally, for the regression tests, do:
+If you did an out-of-tree build, pass in the relative path:
@example
-find input/regression/ -name '*.ly' | xargs convert-ly -e --from @qq{@var{X.Y.Z}}
-
+BUILD_DIR=../build-lilypond/ scripts/auxiliar/update-with-convert-ly.sh
@end example
-@subsection Manually update documentation, snippets, and regtests
+
+@node Manually update documentation
+@subsection Manually update documentation
Where the convert-ly rule is not able to automatically update the inline
lilypond code in the documentation (i.e. if a NOT_SMART rule is used), the
Where the convert-ly rule is not able to automatically update snippets
in Documentation/snippets/, those snippets must be manually updated.
Those snippets should be copied to Documentation/snippets/new. The
-comments at the top of the snippet describing its automatice generation
+comments at the top of the snippet describing its automatic generation
should be removed. All translated texidoc strings should be removed.
The comment @qq{% begin verbatim} should be removed. The syntax of
the snippet should then be manually edited.
or removed from the documentation should be changed only in
the English version.
+
+@node Edit changes.tely
@subsection Edit changes.tely
An entry should be added to Documentation/changes.tely to describe
The changes.tely entry should be written to show how the new change
improves LilyPond, if possible.
+
+@node Verify successful build
@subsection Verify successful build
When the changes have been made, successful completion must be
Developers on Windows who are unable to build LilyPond should
get help from a Linux or OSX developer to do the make tests.
-@subsection Verify regression test
+
+@node Verify regression tests
+@subsection Verify regression tests
In order to avoid breaking LilyPond, it is important to verify that
-the regression tests all succeed. This process is described in
-@ref{Regression tests}.
+the regression tests succeed, and that no unwanted changes are
+introduced into the output. This process is described in
+@ref{Regtest comparison}.
-@subsection Post patch for comments
+@subheading Typical developer's edit/compile/test cycle
-For any change other than a minor change, a patch set should be
-posted on @uref{http://codereview.appspot.com/, Rietveld} for comment.
-This requires the use of an external package, git-cl, and an email
-account on Google.
+TODO: is @code{[-j@var{X} CPU_COUNT=@var{X}]} useful for
+@code{test-baseline}, @code{check}, @code{clean},
+@code{test-redo}? Neil Puttock says it is useful for
+everything but @code{clean}, which is disk-limited.
+Need to check formally.
-git-cl is installed by:
+@itemize
+@item
+Initial test:
@example
-git clone git://neugierig.org/git-cl.git
+make [-j@var{X}]
+make test-baseline
+make [-j@var{X} CPU_COUNT=@var{X}] check
@end example
-Then, add the git-cl directory to your PATH, or create a
-symbolic link to the git-cl and upload.py in one of your
-PATH directories (like usr/bin). git-cl is then
-configured by entering the command
+@item
+Edit/compile/test cycle:
@example
-git cl config
-@end example
+@emph{## edit source files, then...}
-@noindent
-in the LilyPond git directory and answering the questions that
-are asked. If you do not understand the question answer with just
-a newline (CR).
+make clean @emph{## only if needed (see below)}
+make [-j@var{X}] @emph{## only if needed (see below)}
+make test-redo @emph{## redo files differing from baseline}
+make [-j@var{X} CPU_COUNT=@var{X}] check @emph{## CPU_COUNT here?}
+@end example
-The patch set is posted to Rietveld as follows. Ensure your changes
-are committed in a separate branch, which should differ from the
-reference branch to be used by just the changes to be uploaded.
-If the reference branch is to be origin/master, ensure this is
-up-to-date. If necessary, use git rebase to rebase the branch
-containing the changes to the head of origin/master. Finally,
-check out branch with the changes and enter the command:
+@item
+Reset:
@example
-git cl upload <reference SHA1 ID>
+make test-clean
@end example
+@end itemize
-@noindent
-where <reference SHA1 ID> is the SHA1 ID of the commit to be used
-as a reference source for the patch. Generally, this will be the
-SHA1 ID of origin/master, and in that case the command
+If you modify any source files that have to be compiled (such as
+@file{.cc} or @file{.hh} files in @file{flower/} or @file{lily/}),
+then you must run @command{make} before @command{make test-redo},
+so @command{make} can compile the modified files and relink all
+the object files. If you only modify files which are interpreted,
+like those in the @file{scm/} and @file{ly/} directories, then
+@command{make} is not needed before @command{make test-redo}.
-@example
-git cl upload origin/master
-@end example
+TODO: Fix the following paragraph. You can do @command{rm mf/out/*}
+instead of make clean, and you can probably do
+@command{make -C mf/ clean} as well, but I haven't checked it -- cds
-@noindent
-can be used.
+Also, if you modify any font definitions in the @file{mf/}
+directory then you must run @command{make clean} and
+@command{make} before running @command{make test-redo}. This will
+recompile everything, whether modified or not, and takes a lot
+longer.
-After prompting for your Google email address and password, the
-patch set will be posted to Rietveld.
+Running @command{make@tie{}check} will leave an HTML page
+@file{out/test-results/index.html}. This page shows all the
+important differences that your change introduced, whether in the
+layout, MIDI, performance or error reporting.
-You should then announce the patch by sending
-an email to lilypond-devel, with a subject line
-starting with PATCH:, asking for comments on the patch.
-As revisions are made in response to comments, successive patch sets
-for the same issue can be uploaded by reissuing the git-cl command
-with the modified branch checked out.
+
+@node Post patch for comments
+@subsection Post patch for comments
+
+See @ref{Uploading a patch for review}.
+
+
+@node Push patch
@subsection Push patch
Once all the comments have been addressed, the patch can be pushed.
If the author has push privileges, the author will push the patch.
Otherwise, a developer with push privileges will push the patch.
+
+@node Closing the issues
@subsection Closing the issues
Once the patch has been pushed, all the relevant issues should be
closed.
-On Rietveld, the author should log in an close the issue either by
+On Rietveld, the author should log in and close the issue either by
using the @q{Edit Issue} link, or by clicking the circled x icon
to the left of the issue name.
should be sent to bug-lilypond requesting the BugMeister to change
the status.
+
@node Iterator tutorial
@section Iterator tutorial
and sent the music events to the appropriate engravers and/or
performers.
+See a short example discussing iterators and their duties in
+@ref{Articulations on EventChord}.
+
+
@node Engraver tutorial
@section Engraver tutorial
made available to the engraver when they are created by other engravers
during the iteration phase.
+
+@menu
+* Useful methods for information processing::
+* Translation process::
+* Preventing garbage collection for SCM member variables::
+* Listening to music events::
+* Acknowledging grobs::
+* Engraver declaration/documentation::
+@end menu
+
+@node Useful methods for information processing
@subsection Useful methods for information processing
An engraver inherits the following public methods from the Translator
translation: for example, an unterminated spanner might be completed
automatically or reported with a warning message.
+
+@node Translation process
@subsection Translation process
At each timestep in the music, translation proceeds by calling the
information has been processed prior to beginning the translation for
the next timestep.
+
+@node Preventing garbage collection for SCM member variables
@subsection Preventing garbage collection for SCM member variables
In certain cases, an engraver might need to ensure private Scheme
@end example
+@node Listening to music events
@subsection Listening to music events
External interfaces to the engraver are implemented by protected
@}
@end example
+
+@node Acknowledging grobs
@subsection Acknowledging grobs
Some engravers also need information from grobs as they are created
@}
@end example
+Acknowledge functions are called in the order engravers are
+@code{\consist}-ed (the only exception is if you set
+@code{must-be-last} to @code{#t}).
+
+If useful things are to be done to the acknowledged grobs, this
+should be deferred until all the acknowledging has finished, i.e.,
+store the acknowledged grobs and process the information in a
+@code{process-acknowledged ()} or @code{stop-translation-timestep ()}
+function.
+
+
+@node Engraver declaration/documentation
@subsection Engraver declaration/documentation
An engraver must have a public macro
and @code{Engraver_writes} is the set of properties written by
the engraver.
+The @code{ADD_ACKNOWLEDGER} and @code{ADD_TRANSLATOR} macros use a
+non-standard indentation system. Each interface, grob, read property,
+and write property is on its own line, and the closing parenthesis
+and semicolon for the macro all occupy a separate line beneath the final
+interface or write property. See existing engraver files for more
+information.
+
+
@node Callback tutorial
@section Callback tutorial
TODO -- This is a placeholder for a tutorial on callback functions.
+
+@node Understanding pure properties
+@section Understanding pure properties
+
+@menu
+* Purity in LilyPond::
+* Writing a pure function::
+* How purity is defined and stored::
+* Where purity is used::
+* Case studies::
+* Debugging tips::
+@end menu
+
+Pure properties are some of the most difficult properties to understand
+in LilyPond but, once understood, it is much easier to work with
+horizontal spacing. This document provides an overview of what it means
+for something to be @q{pure} in LilyPond, what this purity guarantees,
+and where pure properties are stored and used. It finishes by
+discussing a few case studies for the pure programmer to save you some
+time and to prevent you some major headaches.
+
+
+@node Purity in LilyPond
+@subsection Purity in LilyPond
+Pure properties in LilyPond that do not have any @q{side effects}.
+That is, looking up a pure property should never result in calls to the
+following functions:
+@itemize
+@item @code{set_property}
+@item @code{set_object}
+@item @code{suicide}
+@end itemize
+This means that, if the property is calculated via a callback, this callback
+must not only avoid the functions above but make sure that any functions
+it calls also avoid the functions above. Also, to date in LilyPond, a pure
+function will always return the same value before line breaking (or, more
+precisely, before any version of @code{break_into_pieces} is called). This
+convention makes it possible to cache pure functions and be more flexible
+about the order in which functions are called. For example; Stem #'length has
+a pure property that will @emph{never} trigger one of the functions listed
+above and will @emph{always} return the same value before line breaking,
+independent of where it is called. Sometimes, this will be the actual length
+of the Stem. But sometimes it will not. For example; stem that links up
+with a beam will need its end set to the Y position of the beam at the stem's
+X position. However, the beam's Y positions can only be known after the score
+is broken up in to several systems (a beam that has a shallow slope on a
+compressed line of music, for example, may have a steeper one on an
+uncompressed line). Thus, we only call the impure version of the properties
+once we are @emph{absolutely certain} that all of the parameters needed to
+calculate their final value have been calculated. The pure version provides a
+useful estimate of what this Stem length (or any property) will be, and
+the art of creating good pure properties is trying to get the estimation
+as close to the actual value as possible.
+
+Of course, like Gregory Peck and Tintin, some Grobs will have properties
+that will always be pure. For example, the height of a note-head in
+not-crazy music will never depend on line breaking or other parameters
+decided late in the typesetting process. Inversely, in rare cases,
+certain properties are difficult to estimate with pure values. For
+example, the height of a Hairpin at a certain cross-section of its
+horizontal span is difficult to know without knowing the horizontal
+distance that the hairpin spans, and LilyPond provides an
+over-estimation by reporting the pure height as the entire height of the
+Hairpin.
+
+Purity, like for those living in a convent, is more like a contract than
+an @emph{a priori}. If you write a pure-function, you are promising
+the user (and the developer who may have to clean up after you) that
+your function will not be dependent on factors that change at different
+stages of the compilation process (compilation of a score, not of
+LilyPond).
+
+One last oddity is that purity, in LilyPond, is currently limited
+exclusively to things that have to do with Y-extent and positioning.
+There is no concept of @q{pure X} as, by design, X is always the
+independent variable (i.e. from column X1 to column X2, what will be the
+Y height of a given grob). Furthermore, there is no purity for
+properties like color, text, and other things for which a meaningful notion
+of estimation is either not necessary or has not yet been found. For example,
+even if a color were susceptible to change at different points of the
+compilation process, it is not clear what a pure estimate of this color
+would be or how this pure color could be used. Thus, in this document and
+in the source, you will see purity discussed almost interchangeably with
+Y-axis positioning issues.
+
+
+@node Writing a pure function
+@subsection Writing a pure function
+Pure functions take, at a minimum, three arguments: the @var{grob}, the
+starting column at which the function is being evaluated (hereafter
+referred to as @var{start}), and the end column at which the grob is
+being evaluated (hereafter referred to as @var{end}). For items,
+@var{start} and @var{end} must be provided (meaning they are not optional)
+but will not have a meaningful impact on the result, as items only occupy
+one column and will thus yield a value or not (if they are not in the range
+from @var{start} to @var{end}). For spanners however, @var{start} and
+@var{end} are important, as we may can get a better pure estimation of a
+slice of the spanner than considering it on the whole. This is useful
+during line breaking, for example, when we want to estimate the Y-extent
+of a spanner broken at given starting and ending columns.
+
+
+@node How purity is defined and stored
+@subsection How purity is defined and stored
+Purity can currently be defined two different ways in LilyPond that
+correspond to two types of scenarios. In one scenario, we know that a
+callback is pure, but we are not necessarily certain what properties
+will use this callback. In another, we want a property to be pure, but
+we don't want to guarantee that its callback function will be pure in
+all circumstances.
+
+In the first scenario, we register the callback in define-grobs.scm in
+one of four places depending on what the function does.
+
+@itemize
+@item @code{pure-print-functions}: If finding a print function's vertical
+extent does not have any @q{side effects} we register it here. We then
+don't have to set the pure Y-extent property, which will be taken from the
+stencil.
+
+@item @code{pure-print-to-height-conversions}: If a stencil can
+eventually be used to glean a grob's Y-extent but is not pure (meaning
+it will have a different height at different stages of the compilation
+process), we add it to this list along with a function for the pure
+Y-extent.
+
+@item @code{pure-conversions-alist}: This list contains pairs of
+functions and their pure equivalents. It is onto but not one-to-one.
+
+@item @code{pure-functions}: Like pure-print-functions in that they work
+for both pure and impure values, but they do not return a stencil.
+@end itemize
+
+At all stages of the compilation process, when LilyPond wants the pure
+version of a property, it will consult these lists and see if it can get
+this property for a given Grob. Note that you do @emph{not} need to
+register the pure property in the grob itself. For example, there is no
+property @q{pure-Y-extent}. Rather, by registering these functions as
+defined above, every time LilyPond needs a pure property, it will check
+to see if a Grob contains one of these functions and, if so, will use
+its value. If LilyPond cannot get a pure function, it will return a
+value of @code{##f} for the property.
+
+LilyPond is smart enough to know if a series of chained functions are
+pure. For example, if a Y-offset property has four chained functions
+and all of them have pure equivalents, LilyPond will read the four pure
+equivalents when calculating the pure property. However, if even one is
+impure, LilyPond will not return a pure property for the offset (instead
+returning something like @code{#f} or @code{'()}) and will likely wreak
+havoc on your score.
+
+In the second scenario, we create an unpure-pure-container (unpure is
+not a word, but hey, neither was Lilypond until the 90s). For example:
+
+@example
+#(define (foo grob)
+ '(-1 . 1))
+
+#(define (bar grob start end)
+ '(-2 . 2))
+
+\override Stem #'length = #(ly:make-unpure-pure-container foo bar)
+@end example
+
+This is useful if we want to:
+
+@itemize
+@item create overrides that have pure alternatives (should not be used
+in development, but useful for users)
+
+@item use return values that are not functions (i.e. pairs or booleans)
+for either pure or unpure values.
+
+@item allow a function to be considered pure in a limited amount of
+circumstances. This is useful if we are sure that, when associated with
+one grob a function will be pure but not necessarily with another grob
+that has different callbacks.
+@end itemize
+
+Items can only ever have two pure heights: their actual pure height if
+they are between @q{start} and @q{end}, or an empty interval if they are
+not. Thus, their pure property is cached to speed LilyPond up. Pure
+heights for spanners are generally not cached as they change depending
+on the start and end values. They are only cached in certain particular
+cases. Before writing a lot of caching code, make sure that it is a
+value that will be reused a lot.
+
+
+@node Where purity is used
+@subsection Where purity is used
+Pure Y values must be used in any functions that are called before
+line breaking. Examples of this can be seen in
+@code{Separation_items::boxes} to construct horizontal skylines and in
+@code{Note_spacing::stem_dir_correction} to correct for optical
+illusions in spacing. Pure properties are also used in the calculation
+of other pure properties. For example, the @code{Axis_group_interface}
+has pure functions that look up other pure functions.
+
+Purity is also implicitly used in any functions that should only ever
+return pure values. For example, extra-spacing-height is only ever used
+before line-breaking and thus should never use values that would only be
+available after line breaking. In this case, there is no need to create
+callbacks with pure equivalents because these functions, by design, need
+to be pure.
+
+To know if a property will be called before and/or after line-breaking
+is sometimes tricky and can, like all things in coding, be found by
+using a debugger and/or adding @var{printf} statements to see where they
+are called in various circumstances.
+
+
+@node Case studies
+@subsection Case studies
+In each of these case studies, we expose a problem in pure properties, a
+solution, and the pros and cons of this solution.
+
+@subheading Time signatures
+A time signature needs to prevent accidentals from passing over or under
+it, but its extent does not necessarily extend to the Y-position of
+accidentals. LilyPond's horizontal spacing sometimes makes a line of
+music compact and, when doing so, allows certain columns to pass over
+each other if they will not collide. This type of passing over is not
+desirable with time signatures in traditional engraving. But how do we
+know if this passing over will happen before line breaking, as we are
+not sure what the X positions will be? We need a pure estimation of how
+much extra spacing height the time signatures would need to prevent this
+form of passing over without making this height so large as to
+overly-distort the Y-extent of an system, which could result in a very
+@q{loose} looking score with lots of horizontal space between columns.
+So, to approximate this extra spacing height, we use the Y-extent of a
+time signature's next-door-neighbor grobs via the pure-from-neighbor
+interface.
+
+@itemize
+@item pros: By extending the extra spacing height of a time signature to
+that of its next-door-neighbors, we make sure that grobs to the right of
+it that could pass above or below it do not.
+
+@item cons: This over-estimation of the vertical height could prevent
+snug vertical spacing of systems, as the system will be registered as
+being taller at the point of the time signature than it actually is.
+This approach can be used for clefs and bar lines as well.
+@end itemize
+
+@subheading Stems
+As described above, Stems need pure height approximations when they are
+beamed, as we do not know the beam positions before line breaking. To
+estimate this pure height, we take all the stems in a beam and find
+their pure heights as if they were not beamed. Then, we find the union
+of all these pure heights and take the intersection between this
+interval (which is large) and an interval going from the note-head of a
+stem to infinity in the direction of the stem so that the interval stops
+at the note head.
+
+@itemize
+@item pros: This is guaranteed to be at least as long as the beamed
+stem, as a beamed stem will never go over the ideal length of the
+extremal beam of a stem.
+
+@item cons: Certain stems will be estimated as being too long, which
+leads to the same problem of too-much-vertical-height as described
+above.
+
+@end itemize
+
+
+@node Debugging tips
+@subsection Debugging tips
+A few questions to ask yourself when working with pure properties:
+
+@itemize
+@item Is the property really pure? Are you sure that its value could
+not be changed later in the compiling process due to other changes?
+
+@item Can the property be made to correspond even more exactly with the
+eventual impure property?
+
+@item For a spanner, is the pure property changing correctly depending
+on the starting and ending points of the spanner?
+
+@item For an Item, will the item's pure height need to act in horizontal
+spacing but not in vertical spacing? If so, use extra-spacing-height
+instead of pure height.
+
+@end itemize
+
+
@node LilyPond scoping
@section LilyPond scoping
-The Lilypond language has a concept of scoping, ie you can do
+The Lilypond language has a concept of scoping, i.e. you can do:
@example
foo = 1
dispersed, either by being stored module-locally, or in weak hash
tables.
+
+@node Scheme->C interface
+@section Scheme->C interface
+
+Most of the C functions interfacing with Guile/Scheme used in LilyPond
+are described in the API Reference of the
+@uref{http://www.gnu.org/software/guile/manual/html_node/index.html,
+GUILE Reference Manual}.
+
+The remaining functions are defined in @file{lily/lily-guile.cc},
+@file{lily/include/lily-guile.hh} and
+@file{lily/include/lily-guile-macros.hh}.
+Although their names are meaningful there's a few things you should know
+about them.
+
+@menu
+* Comparison::
+* Conversion::
+@end menu
+
+@node Comparison
+@subsection Comparison
+
+This is the trickiest part of the interface.
+
+Mixing Scheme values with C comparison operators won't produce any crash
+or warning when compiling but must be avoided:
+
+@example
+scm_string_p (scm_value) == SCM_BOOL_T
+@end example
+
+As we can read in the reference, @code{scm_string_p} returns a Scheme
+value: either @code{#t} or @code{#f} which are written @code{SCM_BOOL_T}
+and @code{SCM_BOOL_F} in C. This will work, but it is not following
+to the API guidelines. For further information, read this discussion:
+
+@smallexample
+@uref{http://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00646.html}
+@end smallexample
+
+There are functions in the Guile reference that returns C values
+instead of Scheme values. In our example, a function called
+@code{scm_is_string} (described after @code{string?} and @code{scm_string_p})
+returns the C value 0 or 1.
+
+So the best solution was simply:
+
+@example
+scm_is_string (scm_value)
+@end example
+
+There a simple solution for almost every common comparison. Another example:
+we want to know if a Scheme value is a non-empty list. Instead of:
+
+@example
+(scm_is_true (scm_list_p (scm_value)) && scm_value != SCM_EOL)
+@end example
+
+one can usually use:
+
+@example
+scm_is_pair (scm_value)
+@end example
+
+since a list of at least one member is a pair. This test is
+cheap; @code{scm_list_p} is actually quite more complex since it makes
+sure that its argument is neither a `dotted list' where the last pair
+has a non-null @code{cdr}, nor a circular list. There are few
+situations where the complexity of those tests make sense.
+
+Unfortunately, there is not a @code{scm_is_[something]} function for
+everything. That's one of the reasons why LilyPond has its own Scheme
+interface. As a rule of thumb, tests that are cheap enough to be
+worth inlining tend to have such a C interface. So there is
+@code{scm_is_pair} but not @code{scm_is_list}, and @code{scm_is_eq}
+but not @code{scm_is_equal}.
+
+@subheading General definitions
+
+@subsubheading bool to_boolean (SCM b)
+
+Return @code{true} if @var{b} is @code{SCM_BOOL_T}, else return @code{false}.
+
+This should be used instead of @code{scm_is_true} and
+@code{scm_is_false} for properties since in Lilypond, unset properties
+are read as an empty list, and by convention unset Boolean properties
+default to false. Since both @code{scm_is_true} and
+@code{scm_is_false} only compare with @code{##f} in line with what
+Scheme's conditionals do, they are not really useful for checking the
+state of a Boolean property.
+
+@subsubheading bool ly_is_[something] (args)
+
+Behave the same as scm_is_[something] would do if it existed.
+
+@subsubheading bool is_[type] (SCM s)
+
+Test whether the type of @var{s} is [type].
+[type] is a LilyPond-only set of values (direction, axis...). More
+often than not, the code checks Lilypond specific C++-implemented
+types using
+
+@subsubheading [type *] unsmob_[type] (SCM s)
+
+This tries converting a Scheme object to a pointer of the desired
+kind. If the Scheme object is of the wrong type, a pointer value
+of@w{ }@code{0} is returned, making this suitable for a Boolean test.
+
+@node Conversion
+@subsection Conversion
+
+@subheading General definitions
+
+@subsubheading bool to_boolean (SCM b)
+
+Return @code{true} if @var{b} is @code{SCM_BOOL_T}, else return @code{false}.
+
+This should be used instead of @code{scm_is_true} and @code{scm_is_false}
+for properties since empty lists are sometimes used to unset them.
+
+@subsubheading [C type] ly_scm2[C type] (SCM s)
+
+Behave the same as scm_to_[C type] would do if it existed.
+
+@subsubheading [C type] robust_scm2[C type] (SCM s, [C type] d)
+
+Behave the same as scm_to_[C type] would do if it existed.
+Return @var{d} if type verification fails.
+
+
@node LilyPond miscellany
@section LilyPond miscellany
but doesn't yet have a proper home. Ideally, the length of this section
would become zero as items are moved to other homes.
+
+@menu
+* Spacing algorithms::
+* Info from Han-Wen email::
+* Music functions and GUILE debugging::
+* Articulations on EventChord::
+@end menu
+
+@node Spacing algorithms
@subsection Spacing algorithms
Here is information from an email exchange about spacing algorithms.
Joe Neeman answered:
We create lots of extra grobs (eg. a BarNumber at every bar line) but
-most of them are not drawn. See the break-visibility property in
+most of them are not drawn. See the break-visibility property in
item-interface.
-@subsection Info from Han-Wen Email
+Here is another e-mail exchange. Janek WarchoĊ asked for a starting point
+to fixing 1301 (change clef colliding with notes). Neil Puttock replied:
+
+The clef is on a loose column (it floats before the head), so the
+first place I'd look would be lily/spacing-loose-columns.cc (and
+possibly lily/spacing-determine-loose-columns.cc).
+I'd guess the problem is the way loose columns are spaced between
+other columns: in this snippet, the columns for the quaver and tuplet
+minim are so close together that the clef's column gets dumped on top
+of the quaver (since it's loose, it doesn't influence the spacing).
+
+@node Info from Han-Wen email
+@subsection Info from Han-Wen email
In 2004, Douglas Linhardt decided to try starting a document that would
explain LilyPond architecture and design principles. The material below
The headings reflect questions from Doug or comments from Han-Wen;
the body text are Han-Wen's answers.
-@unnumberedsubsubsec Figuring out how things work.
+@subheading Figuring out how things work.
I must admit that when I want to know how a program works, I use grep
-and emacs and dive into the source code. The comments and the code
+and emacs and dive into the source code. The comments and the code
itself are usually more revealing than technical documents.
-@unnumberedsubsubsec What's a grob, and how is one used?
+@subheading What's a grob, and how is one used?
Graphical object - they are created from within engravers, either as
Spanners (derived class) -slurs, beams- or Items (also a derived
class) -notes, clefs, etc.
There are two other derived classes System (derived from Spanner,
-contaning a "line of music") and Paper_column (derived from Item, it
-contains all items that happen at the same moment). They are separate
+containing a "line of music") and Paper_column (derived from Item, it
+contains all items that happen at the same moment). They are separate
classes because they play a special role in the linebreaking process.
-@unnumberedsubsubsec What's a smob, and how is one used?
+@subheading What's a smob, and how is one used?
A C(++) object that is encapsulated so it can be used as a Scheme
object. See GUILE info, "19.3 Defining New Types (Smobs)"
-@unnumberedsubsubsec When is each C++ class constructed and used
+@@subheading When is each C++ class constructed and used
@itemize
Engravers
Executive branch of Contexts, plugins that create grobs, usually one
-engraver per grob type. Created together with context.
+engraver per grob type. Created together with context.
@item
Layout Objects
@item
Grob Interfaces
-These are not C++ classes per se. The idea of a Grob interface hasn't
-crystallized well. ATM, an interface is a symbol, with a bunch of grob
-properties. They are not objects that are created or destroyed.
+These are not C++ classes per se. The idea of a Grob interface hasn't
+crystallized well. ATM, an interface is a symbol, with a bunch of grob
+properties. They are not objects that are created or destroyed.
@item
Iterators
@end itemize
-@unnumberedsubsubsec Can you get to Context properties from a Music object?
+@subheading Can you get to Context properties from a Music object?
You can create music object with a Scheme function that reads context
-properties (the \applycontext syntax). However, that function is
+properties (the \applycontext syntax). However, that function is
executed during Interpreting, so you can not really get Context
properties from Music objects, since music objects are not directly
-connected to Contexts. That connection is made by the Music_iterators
+connected to Contexts. That connection is made by the Music_iterators
-@unnumberedsubsubsec Can you get to Music properties from a Context object?
+@subheading Can you get to Music properties from a Context object?
Yes, if you are given the music object within a Context
-object. Normally, the music objects enter Contexts in synchronized
+object. Normally, the music objects enter Contexts in synchronized
fashion, and the synchronization is done by Music_iterators.
-@unnumberedsubsubsec What is the relationship between C++ classes and Scheme objects?
+@subheading What is the relationship between C++ classes and Scheme objects?
-Smobs are C++ objects in Scheme. Scheme objects (lists, functions) are
+Smobs are C++ objects in Scheme. Scheme objects (lists, functions) are
manipulated from C++ as well using the GUILE C function interface
(prefix: scm_)
-@unnumberedsubsubsec How do Scheme procedures get called from C++ functions?
+@subheading How do Scheme procedures get called from C++ functions?
scm_call_*, where * is an integer from 0 to 4.
Also scm_c_eval_string (), scm_eval ()
-@unnumberedsubsubsec How do C++ functions get called from Scheme procedures?
+@subheading How do C++ functions get called from Scheme procedures?
Export a C++ function to Scheme with LY_DEFINE.
-@unnumberedsubsubsec What is the flow of control in the program?
+@subheading What is the flow of control in the program?
Good question. Things used to be clear-cut, but we have Scheme
and SMOBs now, which means that interactions do not follow a very
-rigid format anymore. See below for an overview, though.
+rigid format anymore. See below for an overview, though.
-@unnumberedsubsubsec Does the parser make Scheme procedure calls or C++ function calls?
+@subheading Does the parser make Scheme procedure calls or C++ function calls?
-Both. And the Scheme calls can call C++ and vice versa. It's nested,
+Both. And the Scheme calls can call C++ and vice versa. It's nested,
with the SCM datatype as lubrication between the interactions
(I think the word "lubrication" describes the process better than the
traditional word "glue")
-@unnumberedsubsubsec How do the front-end and back-end get started?
+@subheading How do the front-end and back-end get started?
-Front-end: a file is parsed, the rest follows from that. Specifically,
+Front-end: a file is parsed, the rest follows from that. Specifically,
Parsing leads to a Music + Music_output_def object (see parser.yy,
definition of toplevel_expression )
ly_run_translator ())
During interpreting, Global_context + Music leads to a bunch of
-Contexts. (see Global_translator::run_iterator_on_me () )
+Contexts (see Global_translator::run_iterator_on_me ()).
After interpreting, Global_context contains a Score_context (which
-contains staves, lyrics etc.) as a child. Score_context::get_output ()
+contains staves, lyrics etc.) as a child. Score_context::get_output ()
spews a Music_output object (either a Paper_score object for notation
or Performance object for MIDI).
-The Music_output object is the entry point for the backend. (see
-ly_render_output () )
+The Music_output object is the entry point for the backend (see
+ly_render_output ()).
The main steps of the backend itself are in
@itemize
@item
-paper-score.cc , Paper_score::process_
+@file{paper-score.cc} , Paper_score::process_
@item
-system.cc , System::get_lines()
+@file{system.cc} , System::get_lines()
@item
The step, where things go from grobs to output, is in
System::get_line(): each grob delivers a Stencil (a Device
independent output description), which is interpreted by our
-outputting backends (scm/output-tex.scm and scm/output-ps.scm)
-to produce TeX and PS.
+outputting backends (@file{scm/output-tex.scm} and
+@file{scm/output-ps.scm}) to produce TeX and PS.
@end itemize
Interactions between grobs and putting things into .tex and .ps files
-have gotten a little more complex lately. Jan has implemented
+have gotten a little more complex lately. Jan has implemented
page-breaking, so now the backend also involves Paper_book,
-Paper_lines and other things. This area is still heavily in flux, and
+Paper_lines and other things. This area is still heavily in flux, and
perhaps not something you should want to look at.
-@unnumberedsubsubsec How do the front-end and back-end communicate?
+@subheading How do the front-end and back-end communicate?
-There is no communication from backend to front-end. From front-end to
+There is no communication from backend to front-end. From front-end to
backend is simply the program flow: music + definitions gives
contexts, contexts yield output, after processing, output is written
to disk.
-@unnumberedsubsubsec Where is the functionality associated with KEYWORDs?
+@subheading Where is the functionality associated with KEYWORDs?
-See my-lily-lexer.cc (keywords, there aren't that many) and ly/*.ly
-(most of the other backslashed \words are identifiers)
+See @file{my-lily-lexer.cc} (keywords, there aren't that many)
+and @file{ly/*.ly} (most of the other backslashed @code{/\words} are identifiers)
-@unnumberedsubsubsec What Contexts/Properties/Music/etc. are available when they are processed?
+@subheading What Contexts/Properties/Music/etc. are available when they are processed?
What do you mean exactly with this question?
-See ly/engraver-init.ly for contexts, see scm/define-*.scm for other
-objects.
+See @file{ly/engraver-init.ly} for contexts,
+see @file{scm/define-*.scm} for other objects.
-@unnumberedsubsubsec How do you decide if something is a Music, Context, or Grob property?
+@subheading How do you decide if something is a Music, Context, or Grob property?
Why is part-combine-status a Music property when it seems (IMO)
to be related to the Staff context?
Engravers and Contexts
Music_iterators can send "synthetic" music events (which aren't in
-the input) to a context. These are caught by Engravers. This is
+the input) to a context. These are caught by Engravers. This is
mostly a one way communication channel.
part-combine-status is part of such a synthetic event, used by
Part_combine_iterator to communicate with Part_combine_engraver.
-@unnumberedsubsubsec I'm adding a property to affect how \autochange works. It seems to
+@subheading Deciding between context and music properties
+
+I'm adding a property to affect how \autochange works. It seems to
me that it should be a context property, but the Scheme autochange
-procecure has a Music argument. Does this mean I should use
+procedure has a Music argument. Does this mean I should use
a Music property?
\autochange is one of these extra strange beasts: it requires
-look-ahead to decide when to change staves. This is achieved by
-running the interpreting step twice (see scm/part-combiner.scm , at
-the bottom), and storing the result of the first step (where to switch
+look-ahead to decide when to change staves. This is achieved by
+running the interpreting step twice (see
+@file{scm/part-combiner.scm} , at the bottom), and
+storing the result of the first step (where to switch
staves) in a Music property. Since you want to influence that
where-to-switch list, your must affect the code in
-make-autochange-music (scm/part-combiner.scm). That code is called
-directly from the parser and there are no official "parsing
-properties" yet, so there is no generic way to tune \autochange. We
-would have to invent something new for this, or add a separate
-argument,
+make-autochange-music (@file{scm/part-combiner.scm}).
+That code is called directly from the parser and there are no
+official "parsing properties" yet, so there is no generic way
+to tune \autochange. We would have to invent something new
+for this, or add a separate argument,
@example
\autochange #around-central-C ..music..
where around-central-C is some function that is called from
make-autochange-music.
-@unnumberedsubsubsec I get lost figuring out what environment the code I'm looking at is in when it executes.
-I found both the C++ and Scheme autochange code. Then I was
-trying to figure out where the code got called from. I finally figured out that
-the Scheme procedure was called before the C++ iterator code, but it took me a
+@subheading More on context and music properties
+
+From Neil Puttock, in response to a question about transposition:
+
+Context properties (using \set & \unset) are tied to engravers: they
+provide information relevant to the generation of graphical objects.
+
+Since transposition occurs at the music interpretation stage, it has
+no direct connection with engravers: the pitch of a note is fixed
+before a notehead is created. Consider the following minimal snippet:
+
+@example
+@{ c' @}
+@end example
+
+This generates (simplified) a NoteEvent, with its pitch and duration
+as event properties,
+
+@example
+(make-music
+ 'NoteEvent
+ 'duration
+ (ly:make-duration 2 0 1 1)
+ 'pitch
+ (ly:make-pitch 0 0 0)
+@end example
+
+which the Note_heads_engraver hears. It passes this information on to
+the NoteHead grob it creates from the event, so the head's correct
+position and duration-log can be determined once it's ready for
+printing.
+
+If we transpose the snippet,
+
+@example
+\transpose c d @{ c' @}
+@end example
+
+the pitch is changed before it reaches the engraver (in fact, it
+happens just after the parsing stage with the creation of a
+TransposedMusic music object):
+
+@example
+(make-music
+ 'NoteEvent
+ 'duration
+ (ly:make-duration 2 0 1 1)
+ 'pitch
+ (ly:make-pitch 0 1 0)
+@end example
+
+You can see an example of a music property relevant to transposition:
+untransposable.
+
+@example
+\transpose c d @{ c'2 \withMusicProperty #'untransposable ##t c' @}
+@end example
+
+-> the second c' remains untransposed.
+
+Take a look at @file{lily/music.cc} to see where the transposition takes place.
+
+
+@subheading How do I tell about the execution environment?
+
+I get lost figuring out what environment the code I'm looking at is in when it
+executes. I found both the C++ and Scheme autochange code. Then I was trying
+to figure out where the code got called from. I finally figured out that the
+Scheme procedure was called before the C++ iterator code, but it took me a
while to figure that out, and I still didn't know who did the calling in the
first place. I only know a little bit about Flex and Bison, so reading those
files helped only a little bit.
-@emph{Han-Wen:} GDB can be of help here. Set a breakpoint in C++, and run. When you
-hit the breakpoint, do a backtrace. You can inspect Scheme objects
+@emph{Han-Wen:} GDB can be of help here. Set a breakpoint in C++, and run. When you
+hit the breakpoint, do a backtrace. You can inspect Scheme objects
along the way by doing
@example
this will display OBJ through GUILE.
+@node Music functions and GUILE debugging
@subsection Music functions and GUILE debugging
Ian Hulin was trying to do some debugging in music functions, and
Han-Wen answered as follows:
-You can see the defintion by doing
+You can see the definition by doing
@example
#(display conditionalMark)
@end example
noindent
-inside the .ly file.
+inside the @file{.ly} file.
The breakpoint failing may have to do with the call sequence. See
-parser.yy, run_music_function(). The function is called directly from
+@file{parser.yy}, run_music_function(). The function is called directly from
C++, without going through the GUILE evaluator, so I think that is why
there is no debugger trap.
+
+@node Articulations on EventChord
+@subsection Articulations on EventChord
+
+From David Kastrup's email
+@uref{http://lists.gnu.org/archive/html/lilypond-devel/2012-02/msg00189.html}:
+
+LilyPond's typesetting does not act on music expressions and music
+events. It acts exclusively on stream events. It is the act of
+iterators to convert a music expression into a sequence of stream events
+played in time order.
+
+The EventChord iterator is pretty simple: it just takes its "elements"
+field when its time comes up, turns every member into a StreamEvent and
+plays that through the typesetting process. The parser currently
+appends all postevents belonging to a chord at the end of "elements",
+and thus they get played at the same point of time as the elements of
+the chord. Due to this design, you can add per-chord articulations or
+postevents or even assemble chords with a common stem by using parallel
+music providing additional notes/events: the typesetter does not see a
+chord structure or postevents belonging to a chord, it just sees a
+number of events occuring at the same point of time in a Voice context.
+
+So all one needs to do is let the EventChord iterator play articulations
+after elements, and then adding to articulations in EventChord is
+equivalent to adding them to elements (except in cases where the order
+of events matters).
projects / lilypond.git / blobdiff