-@c -*- coding: us-ascii; mode: texinfo; -*-
+@c -*- coding: utf-8; mode: texinfo; -*-
@node Programming work
@chapter Programming work
* Finding functions::
* Code style::
* Debugging LilyPond::
+* Tracing object relationships::
* Adding or modifying features::
* Iterator tutorial::
* Engraver tutorial::
programming in Scheme is available in its entirety online.
An introduction to Guile/Scheme as used in LilyPond can be found in the
-Learning Manual, see @rlearning{Scheme tutorial}.
+@rextend{Scheme tutorial}.
@subsection MetaFont
@node Debugging LilyPond
@section Debugging LilyPond
-The most commonly used tool for debugging LilyPond is the GNU debugger
-gdb. Use of gdb is described in this section.
+The most commonly used tool for debugging LilyPond is the GNU
+debugger gdb. The gdb tool is used for investigating and debugging
+core Lilypond code written in C++. Another tool is available for
+debugging Scheme code using the Guile debugger. This section
+describes how to use both gdb and the Guile Debugger.
+@menu
+* Debugging overview::
+* Debugging C++ code::
+* Debugging Scheme code::
+@end menu
+
+@node Debugging overview
@subsection Debugging overview
Using a debugger simplifies troubleshooting in at least two ways.
Then, when execution has paused, debugger commands can be issued to
explore the values of various variables or to execute functions.
-Second, the debugger allows the display of a stack trace, which shows
-the sequence in which functions are called and the arguments to the
-various function calls.
+Second, the debugger can display a stack trace, which shows the
+sequence in which functions have been called and the arguments
+passed to the called functions.
+@node Debugging C++ code
+@subsection Debugging C++ code
-@subsection Compiling with debugging information
+The GNU debugger, gdb, is the principal tool for debugging C++ code.
-In order to use a debugger with LilyPond, it is necessary to compile
+@unnumberedsubsubsec Compiling LilyPond for use with gdb
+
+In order to use gdb with LilyPond, it is necessary to compile
LilyPond with debugging information. This is accomplished by running
the following commands in the main LilyPond source directory.
@example
./configure --disable-optimising
-
make
@end example
-This will create a version of LilyPond that contains the debugging
+This will create a version of LilyPond containing debugging
information that will allow the debugger to tie the source code
to the compiled code.
You should not do @var{make install} if you want to use a debugger
-with LilyPond. @var{make install} will strip the debugging information
-from the LilyPond binary.
+with LilyPond. The @var{make install} command will strip debugging
+information from the LilyPond binary.
-To set breakpoints in Scheme functions, put
+@unnumberedsubsubsec Typical gdb usage
+
+Once you have compiled the Lilypond image with the necessary
+debugging information it will have been written to a location in a
+subfolder of your current working directory:
@example
-\include "guile-debugger.ly"
+out/bin/lilypond
@end example
-in your input file after any scheme procedures you have defined in
-that file. When your input file is processed, a guile prompt
-will be displayed. At the guile prompt, you can set breakpoints with
-the @code{break!} procedure:
+This is important as you will need to let gdb know where to find the
+image containing the symbol tables. You can invoke gdb from the
+command line using
@example
-guile> (break! my-scheme-procedure)
+gdb out/bin/lilypond
@end example
-
-Once you have set the desired breakpoints, you exit the guile repl frame
-by typing:
+@noindent
+This loads the LilyPond symbol tables into gdb. Then, to run
+LilyPond on @code{test.ly} under the debugger, enter
@example
-guile> (quit)
+run test.ly
@end example
+@noindent
+at the gdb prompt.
-When one of the scheme routines for which you have set breakpoints is
-entered, guile will interrupt execution in a debug frame. At this point,
-you will have access to guile debugging commands. For a listing of these
-commands, type:
+As an alternative to running gdb at the command line you may try
+a graphical interface to gdb such as ddd
@example
-debug> help
+ddd out/bin/lilypond
@end example
-@subsection Typical gdb usage
+You can also use sets of standard gdb commands stored in a .gdbinit
+file (see next section).
-@subsection Typical .gdbinit files
+@unnumberedsubsubsec Typical .gdbinit files
-The behavior of gdb can be readily customized through the use of
-@var{.gdbinit} files. A @var{.gdbinit} file is a file named
+The behavior of gdb can be readily customized through the use of a
+@var{.gdbinit} file. A @var{.gdbinit} file is a file named
@var{.gdbinit} (notice the @qq{.} at the beginning of the file name)
that is placed in a user's home directory.
end
@end example
-@subsection Using Guile interactively with LilyPond
+@node Debugging Scheme code
+@subsection Debugging Scheme code
+
+Scheme code can be developed using the Guile command line
+interpreter @code{top-repl}. You can either investigate
+interactively using just Guile or you can use the debugging
+tools available within Guile.
+
+@unnumberedsubsubsec Using Guile interactively with LilyPond
In order to experiment with Scheme programming in the LilyPond
-environment, it is convenient to have a Guile interpreter that
+environment, it is necessary to have a Guile interpreter that
has all the LilyPond modules loaded. This requires the following
steps.
-First, define a Scheme symbol for the active module
-in the .ly file:
+First, define a Scheme symbol for the active module in the .ly file:
@example
#(module-define! (resolve-module '(guile-user))
'lilypond-module (current-module))
@end example
-Second, place a Scheme function in the .ly file that gives an interactive Guile
-prompt:
+Now place a Scheme function in the .ly file that gives an
+interactive Guile prompt:
@example
#(top-repl)
@end example
-When the .ly file is compiled, this causes the compilation to be interrupted
-and an interactive guile prompt to appear. When the guile prompt appears,
-the LilyPond active module must be set as the current guile module:
+When the .ly file is compiled, this causes the compilation to be
+interrupted and an interactive guile prompt to appear. Once the
+guile prompt appears, the LilyPond active module must be set as the
+current guile module:
@example
guile> (set-current-module lilypond-module)
@end example
-Proper operation of these commands can be demonstrated by typing the name
-of a LilyPond public scheme function to see if it's properly defined:
+You can demonstrate these commands are operating properly by typing the name
+of a LilyPond public scheme function to check it has been defined:
@example
guile> fret-diagram-verbose-markup
ABORT: (unbound-variable)
@end example
-Once the module is properly loaded, any valid LilyPond Scheme expression
-can be entered at the interactive prompt.
+Once the module is properly loaded, any valid LilyPond Scheme
+expression can be entered at the interactive prompt.
-After the investigation is complete, the interactive guile interpreter
-can be exited:
+After the investigation is complete, the interactive guile
+interpreter can be exited:
@example
guile> (quit)
The compilation of the .ly file will then continue.
+@unnumberedsubsubsec Using the Guile debugger
+
+To set breakpoints and/or enable tracing in Scheme functions, put
+
+@example
+\include "guile-debugger.ly"
+@end example
+
+in your input file after any scheme procedures you have defined in
+that file. This will invoke the Guile command-line after having set
+up the environment for the debug command-line. When your input file
+is processed, a guile prompt will be displayed. You may now enter
+commands to set up breakpoints and enable tracing by the Guile debugger.
+
+@unnumberedsubsubsec Using breakpoints
+
+At the guile prompt, you can set breakpoints with
+the @code{set-break!} procedure:
+
+@example
+guile> (set-break! my-scheme-procedure)
+@end example
+
+Once you have set the desired breakpoints, you exit the guile repl frame
+by typing:
+
+@example
+guile> (quit)
+@end example
+
+Then, when one of the scheme routines for which you have set
+breakpoints is entered, guile will interrupt execution in a debug
+frame. At this point you will have access to Guile debugging
+commands. For a listing of these commands, type:
+
+@example
+debug> help
+@end example
+
+Alternatively you may code the breakpoints in your Lilypond source
+file using a command such as:
+
+@example
+#(set-break! my-scheme-procedure)
+@end example
+
+immediately after the @code{\include} statement. In this case the
+breakpoint will be set straight after you enter the @code{(quit)}
+command at the guile prompt.
+
+Embedding breakpoint commands like this is particularly useful if
+you want to look at how the Scheme procedures in the @var{.scm}
+files supplied with LilyPond work. To do this, edit the file in
+the relevant directory to add this line near the top:
+
+@example
+(use-modules (scm guile-debugger))
+@end example
+
+Now you can set a breakpoint after the procedure you are interested
+in has been declared. For example, if you are working on routines
+called by @var{print-book-with} in @var{lily-library.scm}:
+
+@example
+(define (print-book-with parser book process-procedure)
+ (let* ((paper (ly:parser-lookup parser '$defaultpaper))
+ (layout (ly:parser-lookup parser '$defaultlayout))
+ (outfile-name (get-outfile-name parser)))
+ (process-procedure book paper layout outfile-name)))
+
+(define-public (print-book-with-defaults parser book)
+ (print-book-with parser book ly:book-process))
+
+(define-public (print-book-with-defaults-as-systems parser book)
+ (print-book-with parser book ly:book-process-to-systems))
+
+@end example
+
+At this point in the code you could add this to set a breakpoint at
+print-book-with:
+
+@example
+(set-break! print-book-with)
+@end example
+
+@unnumberedsubsubsec Tracing procedure calls and evaluator steps
+
+Two forms of trace are available:
+
+@example
+(set-trace-call! my-scheme-procedure)
+@end example
+
+and
+
+@example
+(set-trace-subtree! my-scheme-procedure)
+@end example
+
+@code{set-trace-call!} causes Scheme to log a line to the standard
+output to show when the procedure is called and when it exits.
+
+@code{set-trace-subtree!} traces every step the Scheme evaluator
+performs in evaluating the procedure.
+
+@node Tracing object relationships
+@section Tracing object relationships
+
+Understanding the LilyPond source often boils down to figuring out what
+is happening to the Grobs. Where (and why) are they being created,
+modified and destroyed? Tracing Lily through a debugger in order to
+identify these relationships can be time-consuming and tedious.
+
+In order to simplify this process, a facility has been added to
+display the grobs that are created and the properties that are set
+and modified. Although it can be complex to get set up, once set up
+it easily provides detailed information about the life of grobs
+in the form of a network graph.
+
+Each of the steps necessary to use the graphviz utility
+is described below.
+
+@enumerate
+
+@item Installing graphviz
+
+In order to create the graph of the object relationships, it is
+first necessary to install Graphviz. graphviz is available for a
+number of different platforms:
+
+@example
+@uref{http://www.graphviz.org/Download..php}
+@end example
+
+@item Modifying config.make
+
+In order for the Graphviz tool to work, config.make must be modified.
+It is probably a good idea to first save a copy of config.make under
+a different name. Then, edit config.make by removing every occurence
+of @code{-DNDEBUG}.
+
+@item Rebuilding LilyPond
+
+The executable code of LilyPond must be rebuilt from scratch:
+
+@example
+make -C lily clean && make -C lily
+@end example
+
+@item Create a graphviz-compatible .ly file
+
+In order to use the graphviz utility, the .ly file must include
+@file{ly/graphviz-init.ly}, and should then specify the
+grobs and symbols that should be tracked. An example of this
+is found in @file{input/regression/graphviz.ly}.
+
+@item Run lilypond with output sent to a log file
+
+The Graphviz data is sent to stderr by lilypond, so it is
+necessary to redirect stderr to a logfile:
+
+@example
+lilypond graphviz.ly 2> graphviz.log
+@end example
+
+@item Edit the logfile
+
+The logfile has standard lilypond output, as well as the Graphviz
+output data. Delete everything from the beginning of the file
+up to but not including the first occurence of @code{digraph}.
+
+@item Process the logfile with @code{dot}
+
+The directed graph is created from the log file with the program
+@code{dot}:
+
+@example
+dot -Tpdf graphviz.log > graphviz.pdf
+@end example
+
+@end enumerate
+
+The pdf file can then be viewed with any pdf viewer.
+
+When compiled without @code{-DNDEBUG}, lilypond may run slower
+than normal. The original configuration can be restored by either
+renaming the saved copy of @code{config.make} or rerunning
+@code{configure}. Then rebuild lilypond with
+
+@example
+make -C lily clean && make -C lily
+@end example
+
+
@node Adding or modifying features
@section Adding or modifying features
@node Iterator tutorial
@section Iterator tutorial
-FIXME -- this is a placeholder for a tutorial on iterators
+TODO -- this is a placeholder for a tutorial on iterators
Iterators are routines written in C++ that process music expressions
and sent the music events to the appropriate engravers and/or
@node Engraver tutorial
@section Engraver tutorial
-FIXME -- This is a placeholder for a tutorial on how engravers work.
-
Engravers are C++ classes that catch music events and
create the appropriate grobs for display on the page. Though the
majority of engravers are responsible for the creation of a single grob,
in some cases (e.g. @code{New_fingering_engraver}), several different grobs
may be created.
+Engravers listen for events and acknowledge grobs. Events are passed to
+the engraver in time-step order during the iteration phase. Grobs are
+made available to the engraver when they are created by other engravers
+during the iteration phase.
+
@subsection Useful methods for information processing
An engraver inherits the following public methods from the Translator
At each timestep in the music, translation proceeds by calling the
following methods in turn:
-@code{start_translation_timestep ()} is called before any user information enters
-the translators, i.e., no property operations (\set, \override, etc.) or events
-have been processed yet.
+@code{start_translation_timestep ()} is called before any user
+information enters the translators, i.e., no property operations
+(\set, \override, etc.) or events have been processed yet.
-@code{process_music ()} and @code{process_acknowledged ()} are called after events
-have been heard, or grobs have been acknowledged. The latter tends to be used
-exclusively with engravers which only acknowledge grobs, whereas the former is
-the default method for main processing within engravers.
+@code{process_music ()} and @code{process_acknowledged ()} are called
+after all events in the current time step have been heard, or all
+grobs in the current time step have been acknowledged. The latter
+tends to be used exclusively with engravers which only acknowledge
+grobs, whereas the former is the default method for main processing
+within engravers.
-@code{stop_translation_timestep ()} is called after all user information has been
-processed prior to beginning the translation for the next timestep.
+@code{stop_translation_timestep ()} is called after all user
+information has been processed prior to beginning the translation for
+the next timestep.
@subsection Preventing garbage collection for SCM member variables
-In certain cases, an engraver might need to ensure private Scheme variables
-(with type SCM) do not get swept away by Guile's garbage collector: for example,
-a cache of the previous key signature which must persist persist between timesteps.
-The method @code{virtual derived_mark () const} can be used in such cases to mark
-such objects as follows:
+In certain cases, an engraver might need to ensure private Scheme
+variables (with type SCM) do not get swept away by Guile's garbage
+collector: for example, a cache of the previous key signature which
+must persist between timesteps. The method
+@code{virtual derived_mark () const} can be used in such cases:
@example
Engraver_name::derived_mark ()
@subsection Listening to music events
-External interfaces to to the engraver are implemented by protected
+External interfaces to the engraver are implemented by protected
macros including one or more of the following:
@itemize
@node Callback tutorial
@section Callback tutorial
-FIXME -- This is a placeholder for a tutorial on callback functions.
+TODO -- This is a placeholder for a tutorial on callback functions.
@node LilyPond scoping
@section LilyPond scoping
but doesn't yet have a proper home. Ideally, the length of this section
would become zero as items are moved to other homes.
+@subsection Spacing algorithms
+
+Here is information from an email exchange about spacing algorithms.
+
+On Thu, 2010-02-04 at 15:33 -0500, Boris Shingarov wrote:
+I am experimenting with some modifications to the line breaking code,
+and I am stuck trying to understand how some of it works. So far my
+understanding is that Simple_spacer operates on a vector of Grobs, and
+it is a well-known Constrained-QP problem (rods = constraints, springs
+= quadratic function to minimize). What I don't understand is, if the
+spacer operates at the level of Grobs, which are built at an earlier
+stage in the pipeline, how are the changes necessitated by differences
+in line breaking, taken into account? in other words, if I take the
+last measure of a line and place it on the next line, it is not just a
+matter of literally moving that graphic to where the start of the next
+line is, but I also need to draw a clef, key signature, and possibly
+other fundamental things -- but at that stage in the rendering
+pipeline, is it not too late??
+
+Joe Neeman answered:
+
+We create lots of extra grobs (eg. a BarNumber at every bar line) but
+most of them are not drawn. See the break-visibility property in
+item-interface.
+
@subsection Info from Han-Wen Email
In 2004, Douglas Linhardt decided to try starting a document that would
The headings reflect questions from Doug or comments from Han-Wen;
the body text are Han-Wen's answers.
-@subsubsection Figuring out how things work.
+@unnumberedsubsubsec Figuring out how things work.
I must admit that when I want to know how a program works, I use grep
and emacs and dive into the source code. The comments and the code
itself are usually more revealing than technical documents.
-@subsubsection What's a grob, and how is one used?
+@unnumberedsubsubsec What's a grob, and how is one used?
Graphical object - they are created from within engravers, either as
Spanners (derived class) -slurs, beams- or Items (also a derived
contains all items that happen at the same moment). They are separate
classes because they play a special role in the linebreaking process.
-@subsubsection What's a smob, and how is one used?
+@unnumberedsubsubsec What's a smob, and how is one used?
A C(++) object that is encapsulated so it can be used as a Scheme
object. See GUILE info, "19.3 Defining New Types (Smobs)"
-@subsubsection When is each C++ class constructed and used
+@unnumberedsubsubsec When is each C++ class constructed and used
@itemize
@end itemize
-@subsubsection Can you get to Context properties from a Music object?
+@unnumberedsubsubsec Can you get to Context properties from a Music object?
You can create music object with a Scheme function that reads context
properties (the \applycontext syntax). However, that function is
properties from Music objects, since music objects are not directly
connected to Contexts. That connection is made by the Music_iterators
-@subsubsection Can you get to Music properties from a Context object?
+@unnumberedsubsubsec Can you get to Music properties from a Context object?
Yes, if you are given the music object within a Context
object. Normally, the music objects enter Contexts in synchronized
fashion, and the synchronization is done by Music_iterators.
-@subsubsection What is the relationship between C++ classes and Scheme objects?
+@unnumberedsubsubsec What is the relationship between C++ classes and Scheme objects?
Smobs are C++ objects in Scheme. Scheme objects (lists, functions) are
manipulated from C++ as well using the GUILE C function interface
(prefix: scm_)
-@subsubsection How do Scheme procedures get called from C++ functions?
+@unnumberedsubsubsec How do Scheme procedures get called from C++ functions?
scm_call_*, where * is an integer from 0 to 4.
Also scm_c_eval_string (), scm_eval ()
-@subsubsection How do C++ functions get called from Scheme procedures?
+@unnumberedsubsubsec How do C++ functions get called from Scheme procedures?
Export a C++ function to Scheme with LY_DEFINE.
-@subsubsection What is the flow of control in the program?
+@unnumberedsubsubsec What is the flow of control in the program?
Good question. Things used to be clear-cut, but we have Scheme
and SMOBs now, which means that interactions do not follow a very
rigid format anymore. See below for an overview, though.
-@subsubsection Does the parser make Scheme procedure calls or C++ function
-calls?
+@unnumberedsubsubsec Does the parser make Scheme procedure calls or C++ function calls?
Both. And the Scheme calls can call C++ and vice versa. It's nested,
with the SCM datatype as lubrication between the interactions
(I think the word "lubrication" describes the process better than the
traditional word "glue")
-@subsubsection How do the front-end and back-end get started?
+@unnumberedsubsubsec How do the front-end and back-end get started?
Front-end: a file is parsed, the rest follows from that. Specifically,
Paper_lines and other things. This area is still heavily in flux, and
perhaps not something you should want to look at.
-@subsubsection How do the front-end and back-end communicate?
+@unnumberedsubsubsec How do the front-end and back-end communicate?
There is no communication from backend to front-end. From front-end to
backend is simply the program flow: music + definitions gives
contexts, contexts yield output, after processing, output is written
to disk.
-@subsubsection Where is the functionality associated with KEYWORDs?
+@unnumberedsubsubsec Where is the functionality associated with KEYWORDs?
See my-lily-lexer.cc (keywords, there aren't that many) and ly/*.ly
(most of the other backslashed \words are identifiers)
-@subsubsection What Contexts/Properties/Music/etc. are available when they are processed?
+@unnumberedsubsubsec What Contexts/Properties/Music/etc. are available when they are processed?
What do you mean exactly with this question?
See ly/engraver-init.ly for contexts, see scm/define-*.scm for other
objects.
-@subsubsection How do you decide if something is a Music, Context, or Grob property?
+@unnumberedsubsubsec How do you decide if something is a Music, Context, or Grob property?
Why is part-combine-status a Music property when it seems (IMO)
to be related to the Staff context?
Part_combine_iterator to communicate with Part_combine_engraver.
-@subsubsection I'm adding a property to affect how \autochange works. It seems to
+@unnumberedsubsubsec I'm adding a property to affect how \autochange works. It seems to
me that it should be a context property, but the Scheme autochange
procecure has a Music argument. Does this mean I should use
a Music property?
where around-central-C is some function that is called from
make-autochange-music.
-@subsubsection Also, I get lost figuring out what environment the code I'm looking at is in
-when it executes. I found both the C++ and Scheme autochange code. Then I was
+@unnumberedsubsubsec I get lost figuring out what environment the code I'm looking at is in when it executes.
+I found both the C++ and Scheme autochange code. Then I was
trying to figure out where the code got called from. I finally figured out that
the Scheme procedure was called before the C++ iterator code, but it took me a
while to figure that out, and I still didn't know who did the calling in the
first place. I only know a little bit about Flex and Bison, so reading those
files helped only a little bit.
-GDB can be of help here. Set a breakpoint in C++, and run. When you
+@emph{Han-Wen:} GDB can be of help here. Set a breakpoint in C++, and run. When you
hit the breakpoint, do a backtrace. You can inspect Scheme objects
along the way by doing
this will display OBJ through GUILE.
+@subsection Music functions and GUILE debugging
+
+Ian Hulin was trying to do some debugging in music functions, and
+came up with the following question
+
+HI all,
+I'm working on the Guile Debugger Stuff, and would like to try
+debugging a music function definition such as:
+
+@example
+conditionalMark = #(define-music-function (parser location) ()
+ #@{ \tag #'instrumental-part @{\mark \default@} #@} )
+@end example
+
+It appears conditionalMark does not get set up as an
+equivalent of a Scheme
+
+@example
+(define conditionalMark = define-music-function(parser location () ...
+@end example
+
+@noindent
+although something gets defined because Scheme apparently recognizes
+
+@example
+#(set-break! conditionalMark)
+@end example
+
+@noindent
+later on in the file without signalling any Guile errors.
+
+However the breakpoint trap is never encountered as
+define-music-function passed things on to ly:make-music-function,
+which is really C++ code ly_make_music_function, so Guile never
+finds out about the breakpoint.
+
+Han-Wen answered as follows:
+
+You can see the defintion by doing
+
+@example
+#(display conditionalMark)
+@end example
+
+noindent
+inside the .ly file.
+
+The breakpoint failing may have to do with the call sequence. See
+parser.yy, run_music_function(). The function is called directly from
+C++, without going through the GUILE evaluator, so I think that is why
+there is no debugger trap.
projects / lilypond.git / blobdiff