@chapter Programming work
@menu
-* Introduction to programming::
+* Overview of LilyPond architecture::
+* LilyPond programming languages::
+* Programming without compiling::
+* Finding functions::
* Code style::
+* Debugging LilyPond::
@end menu
+@node Overview of LilyPond architecture
+@section Overview of LilyPond architecture
-@node Introduction to programming
-@section Introduction to programming
+LilyPond processes the input file into graphical and musical output in a
+number of stages. This process, along with the types of routines that
+accomplish the various stages of the process, is described in this section. A
+more complete description of the LilyPond architecture and internal program
+execution is found in Erik Sandberg's
+@uref{http://lilypond.org/web/images/thesis-erik-sandberg.pdf, master's
+thesis}.
-blah blah
+
+The first stage of LilyPond processing is @emph{parsing}. In the parsing
+process, music expressions in LilyPond input format are converted to music
+expressions in Scheme format. In Scheme format, a music expression is a list
+in tree form, with nodes that indicate the relationships between various music
+events. The LilyPond parser is written in Bison.
+
+The second stage of LilyPond processing is @emph{iterating}. Iterating
+assigns each music event to a context, which is the environment in which the
+music will be finally engraved. The context is responsible for all further
+processing of the music. It is during the iteration stage that contexts are
+created as necessary to ensure that every note has a Voice type context (e.g.
+Voice, TabVoice, DrumVoice, CueVoice, MensuralVoice, VaticanaVoice,
+GregorianTranscriptionVoice), that the Voice type contexts exist in
+appropriate Staff type contexts, and that parallel Staff type contexts exist
+in StaffGroup type contexts. In addition, during the iteration stage each
+music event is assigned a moment, or a time in the music when the event
+begins.
+
+Each type of music event has an associated iterator. Iterators are defined in
+*-iterator.cc. During iteration, an
+event's iterator is called to deliver that music event to the appropriate
+context(s).
+
+The final stage of LilyPond processing is @emph{translation}. During
+translation, music events are prepared for graphical or midi output. The
+translation step is accomplished by translators or engravers (the distinction
+is unclear).
+
+Translators are defined in C++ files named *-engraver.cc. In *-engraver.cc, a
+C++ class of Engraver type is created. The Engraver is also declared as a
+translator. Much of the work of translating is handled by Scheme functions,
+which is one of the keys to LilyPond's exceptional flexibility.
+
+
+@node LilyPond programming languages
+@section LilyPond programming languages
+
+Programming in LilyPond is done in a variety of programming languages. Each
+language is used for a specific purpose or purposes. This section describes
+the languages used and provides links to reference manuals and tutorials for
+the relevant language.
+
+@subsection C++
+
+The core functionality of LilyPond is implemented in C++.
+
+C++ is so ubiquitous that it is difficult to identify either a reference
+manual or a tutorial. Programmers unfamiliar with C++ will need to spend some
+time to learn the language before attempting to modify the C++ code.
+
+The C++ code calls Scheme/GUILE through the GUILE interface, which is
+documented in the
+@uref{http://www.gnu.org/software/guile/manual/html_node/index.html, GUILE
+ Reference Manual}.
+
+@subsection GNU Bison
+
+The LilyPond parser is implemented in Bison, a GNU parser generator. The
+Bison homepage is found at @uref{http://www.gnu.org/software/bison/,
+gnu.org}. The manual (which includes both a reference and tutorial) is
+@uref{http://www.gnu.org/software/bison/manual/index.html, available} in a
+variety of formats.
+
+@subsection GNU Make
+
+GNU Make is used to control the compiling process and to build the
+documentation and the website. GNU Make documentation is available at
+@uref{http://www.gnu.org/software/make/manual/, the GNU website}.
+
+@subsection GUILE or Scheme
+
+GUILE is the dialect of Scheme that is used as LilyPond's extension language. Many extensions to LilyPond are written entirely in GUILE. The
+@uref{http://www.gnu.org/software/guile/manual/html_node/index.html,
+GUILE Reference Manual} is available online.
+
+@uref{http://mitpress.mit.edu/sicp/full-text/book/book.html, Structure and
+Interpretation of Computer Programs}, a popular textbook used to teach
+programming in Scheme is available in its entirety online.
+
+An introduction to Guile/Scheme as used in LilyPond can be found in the
+Learning Manual, see @rlearning{Scheme tutorial}.
+
+@subsection MetaFont
+
+MetaFont is used to create the music fonts used by LilyPond. A MetaFont
+tutorial is available at @uref{http://metafont.tutorial.free.fr/, the
+METAFONT tutorial page}.
+
+@subsection PostScript
+
+PostScript is used to generate graphical output. A brief PostScript tutorial
+is @uref{http://local.wasp.uwa.edu.au/~pbourke/dataformats/postscript/,
+available online}. The
+@uref{http://www.adobe.com/devnet/postscript/pdfs/PLRM.pdf, PostScript Lanugage
+Reference} is available online in PDF format.
+
+@subsection Python
+
+Python is used for XML2ly and is used for buillding the documentation and the
+website.
+
+Python documentation is available at @uref{http://www.python.org/doc/,
+python.org}.
+
+@node Programming without compiling
+@section Programming without compiling
+
+Much of the development work in LilyPond takes place by changing *.ly or
+*.scm files. These changes can be made without compiling LilyPond. Such
+changes are described in this section.
+
+
+@subsection Modifying distribution files
+
+Much of LilyPond is written in Scheme or LilyPond input files. These
+files are interpreted when the program is run, rather than being compiled
+when the program is built, and are present in all LilyPond distributions.
+You will find .ly files in the ly/ directory and the Scheme files in the
+scm/ directory. Both Scheme files and .ly files can be modified and
+saved with any text editor. It's probably wise to make a backup copy of
+your files before you modify them, although you can reinstall if the
+files become corrupted.
+
+Once you've modified the files, you can test the changes just by running
+LilyPond on some input file. It's a good idea to create a file that
+demonstrates the feature you're trying to add. This file will eventually
+become a regression test and will be part of the LilyPond distribution.
+
+@subsection Desired file formatting
+
+Files that are part of the LilyPond distribution have Unix-style line
+endings (LF), rather than DOS (CR+LF) or MacOS 9 and earlier (CR). Make
+sure you use the necessary tools to ensure that Unix-style line endings are
+preserved in the patches you create.
+
+Tab characters should not be included in files for distribution. All
+indentation should be done with spaces. Most editors have settings to
+allow the setting of tab stops and ensuring that no tab characters are
+included in the file.
+
+Scheme files and LilyPond files should be written according to standard
+style guidelines. Scheme file guidelines can be found at
+@uref{http://community.schemewiki.org/?scheme-style}. Following these
+guidelines will make your code easier to read. Both you and others that
+work on your code will be glad you followed these guidelines.
+
+For LilyPond files, you should follow the guidelines for LilyPond snippets
+in the documentation. You can find these guidelines at
+@ref{Texinfo introduction and usage policy}.
+
+@node Finding functions
+@section Finding functions
+
+When making changes or fixing bugs in LilyPond, one of the initial
+challenges is finding out where in the code tree the functions to
+be modified live. With nearly 3000 files in the source tree,
+trial-and-error searching is generally ineffective. This section
+describes a process for finding interesting code.
+
+@subsection Using the ROADMAP
+
+The file ROADMAP is located in the main directory of the lilypond source.
+ROADMAP lists all of the directories in the LilPond source tree, along
+with a brief description of the kind of files found in each directory.
+This can be a very helpful tool for deciding which directories to search
+when looking for a function.
+
+
+@subsection Using grep to search
+
+Having identified a likely subdirectory to search, the grep utility can
+be used to search for a function name. The format of the grep command is
+
+@example
+grep -i functionName subdirectory/*
+@end example
+
+This command will search all the contents of the directory subdirectory/
+and display every line in any of the files that contains
+functionName. The @code{-i} option makes @command{grep} ignore
+case -- this can be very useful if you are not yet familiar with
+our capitalization conventions.
+
+The most likely directories to grep for function names are scm/ for
+scheme files, ly/ for lilypond input (*.ly) files, and lily/ for C++
+files.
+
+
+@subsection Using git grep to search
+
+If you have used git to obtain the source, you have access to a
+powerful tool to search for functions. The command:
+
+@example
+git grep functionName
+@end example
+
+will search through all of the files that are present in the git
+repository looking for functionName. It also presents the results
+of the search using @code{less}, so the results are displayed one page
+at a time.
+
+@subsection Searching on the git repository at Savannah
+
+You can also use the equivalent of git grep on the Savannah server.
+
+@itemize
+
+@item
+Go to http://git.sv.gnu.org/gitweb/?p=lilypond.git
+
+@item
+In the pulldown box that says commit, select grep.
+
+@item
+Type functionName in the search box, and hit enter/return
+
+@end itemize
+
+This will initiate a search of the remote git repository.
@node Code style
-@section Code style
+@section Code style
@c email to wl@gnu.org when I get here.
@menu
@end menu
-@subsection Outputting errors
+@subsection Handling errors
As a general rule, you should always try to continue computations,
even if there is some kind of error. When the program stops, it
@subsection Classes and Types
@verbatim
- This_is_a_class
+This_is_a_class
@end verbatim
Member variable names end with an underscore:
@verbatim
- Type Class::member_
+Type Class::member_
@end verbatim
@subsection Localization
This document provides some guidelines for programmers write user
-messages. To help translations, user messages must be
-uniformized. Follow these rules when coding for LilyPond.
+messages. To help translations, user messages must follow
+uniform conventions. Follow these rules when coding for LilyPond.
Hopefully, this can be replaced by general GNU guidelines in the
future. Even better would be to have an English (en_BR, en_AM)
-helping programmers writing consistent messages for all GNU
+guide helping programmers writing consistent messages for all GNU
programs.
-Not-preferred messages are marked with `+'. By convention,
-ungrammatical examples are marked with `*'.
+Non-preferred messages are marked with `+'. By convention,
+ungrammatical examples are marked with `*'. However, such ungrammatical
+examples may still be preferred.
@itemize
@item
-Every message to the user should be localised (and thus be marked
-for localisation). This includes warning and error messages.
+Every message to the user should be localized (and thus be marked
+for localization). This includes warning and error messages.
@item
-Don't localise/gettextify:
+Don't localize/gettextify:
@itemize
@item
@item
Messages to be localised must be encapsulated in `_ (STRING)' or
-`_f (FORMAT, ...)'. Eg:
+`_f (FORMAT, ...)'. E.g.:
+
+@example
+warning (_ ("need music in a score"));
+error (_f ("cannot open file: `%s'", file_name));
+@end example
-@verbatim
- warning (_ ("need music in a score"));
- error (_f ("cannot open file: `%s'", file_name));
-@end verbatim
-
In some rare cases you may need to call `gettext ()' by hand. This
happens when you pre-define (a list of) string constants for later
use. In that case, you'll probably also need to mark these string
constants for translation, using `_i (STRING)'. The `_i' macro is
a no-op, it only serves as a marker for `xgettext'.
-@verbatim
- char const* messages[] = {
- _i ("enable debugging output"),
- _i ("ignore lilypond version"),
- 0
- };
-
-
- void
- foo (int i)
- {
- puts (gettext (messages i));
- }
-@end verbatim
-
+@example
+char const* messages[] = @{
+ _i ("enable debugging output"),
+ _i ("ignore lilypond version"),
+ 0
+@};
+
+void
+foo (int i)
+@{
+ puts (gettext (messages i));
+@}
+@end example
+
See also `flower/getopt-long.cc' and `lily/main.cc'.
@item
whitespace to be printed, prepend or append it to the translated
message
-@verbatim
- message (Calculating line breaks... + " ");
-@end verbatim
-
+@example
+message ("Calculating line breaks..." + " ");
+@end example
+
@item
Error or warning messages displayed with a file name and line
number never start with a capital, eg,
-@verbatim
- foo.ly: 12: not a duration: 3
-@end verbatim
-
+@example
+foo.ly: 12: not a duration: 3
+@end example
+
Messages containing a final verb, or a gerund (`-ing'-form) always
start with a capital. Other (simpler) messages start with a
lowercase letter
-@verbatim
- Processing foo.ly...
- `foo': not declared.
- Not declaring: `foo'.
-@end verbatim
-
+@example
+Processing foo.ly...
+`foo': not declared.
+Not declaring: `foo'.
+@end example
+
@item
Avoid abbreviations or short forms, use `cannot' and `do not'
rather than `can't' or `don't'
To avoid having a number of different messages for the same
-situation, we'll use quoting like this `"message: `%s'"' for all
+situation, well will use quoting like this `"message: `%s'"' for all
strings. Numbers are not quoted:
-@verbatim
- _f ("cannot open file: `%s'", name_str)
- _f ("cannot find character number: %d", i)
-@end verbatim
-
+@example
+_f ("cannot open file: `%s'", name_str)
+_f ("cannot find character number: %d", i)
+@end example
+
@item
Think about translation issues. In a lot of cases, it is better to
-translate a whole message. The english grammar mustn't be imposed
+translate a whole message. The english grammar must not be imposed
on the translator. So, instead of
-@verbatim
- stem at + moment.str () + does not fit in beam
-@end verbatim
-
+@example
+stem at + moment.str () + does not fit in beam
+@end example
+
have
-@verbatim
- _f ("stem at %s does not fit in beam", moment.str ())
-@end verbatim
-
+@example
+_f ("stem at %s does not fit in beam", moment.str ())
+@end example
+
@item
Split up multi-sentence messages, whenever possible. Instead of
-@verbatim
- warning (_f ("out of tune! Can't find: `%s'",
-"Key_engraver"));
- warning (_f ("cannot find font `%s', loading default",
- font_name));
-@end verbatim
-
+@example
+warning (_f ("out of tune! Can't find: `%s'", "Key_engraver"));
+warning (_f ("cannot find font `%s', loading default", font_name));
+@end example
+
rather say:
-@verbatim
- warning (out of tune:;
- warning (_f ("cannot find: `%s', "Key_engraver"));
- warning (_f ("cannot find font: `%s', font_name));
- warning (_f ("Loading default font"));
-@end verbatim
-
+@example
+warning (_ ("out of tune:"));
+warning (_f ("cannot find: `%s', "Key_engraver"));
+warning (_f ("cannot find font: `%s', font_name));
+warning (_f ("Loading default font"));
+@end example
+
@item
If you must have multiple-sentence messages, use full punctuation.
Use two spaces after end of sentence punctuation. No punctuation
(esp. period) is used at the end of simple messages.
-@verbatim
- _f ("Non-matching braces in text `%s', adding braces", text)
- Debug output disabled. Compiled with NPRINT.
- _f ("Huh? Not a Request: `%s'. Ignoring.", request)
-@end verbatim
-
+@example
+_f ("Non-matching braces in text `%s', adding braces", text)
+_ ("Debug output disabled. Compiled with NPRINT.")
+_f ("Huh? Not a Request: `%s'. Ignoring.", request)
+@end example
+
@item
-Do not modularise too much; a lot of words cannot be translated
-without context. It's probably safe to treat most occurences of
+Do not modularise too much; words frequently cannot be translated
+without context. It is probably safe to treat most occurences of
words like stem, beam, crescendo as separately translatable words.
@item
would mean sacrificing a bit of eloquency. This holds for original
messages too, of course.
-@verbatim
- en: cannot open: `foo.ly'
- + nl: kan `foo.ly' niet openen (1)
- kan niet openen: `foo.ly'* (2)
- niet te openen: `foo.ly'* (3)
-@end verbatim
+@example
+en: cannot open: `foo.ly'
++ nl: kan `foo.ly' niet openen (1)
+kan niet openen: `foo.ly'* (2)
+niet te openen: `foo.ly'* (3)
+@end example
+
-
The first nl message, although grammatically and stylistically
correct, is not friendly for parsing by humans (even if they speak
-dutch). I guess we'd prefer something like (2) or (3).
+dutch). I guess we would prefer something like (2) or (3).
@item
Do not run make po/po-update with GNU gettext < 0.10.35
+@node Debugging LilyPond
+@section Debugging LilyPond
+
+The most commonly used tool for debugging LilyPond is the GNU debugger
+gdb. Use of gdb is described in this section.
+
+@subsection Debugging overview
+
+Using a debugger simplifies troubleshooting in at least two ways.
+
+First, breakpoints can be set to pause execution at any desired point.
+Then, when execution has paused, debugger commands can be issued to
+explore the values of various variables or to execute functions.
+
+Second, the debugger allows the display of a stack trace, which shows
+the sequence in which functions are called and the arguments to the
+various function calls.
+
+
+@subsection Compiling with debugging information
+
+In order to use a debugger with LilyPond, it is necessary to compile
+LilyPond with debugging information. This is accomplished by ...
+
+TODO -- get good description here, or perhaps add debugging compile
+to AU1.1 as it comes to CG and just use a reference here.
+
+TODO -- Test the following to make sure it is true.
+
+If you want to be able to set breakpoints in Scheme functions, it is
+necessary to compile guile with debugging information. This is done
+by ...
+
+TODO -- get compiling description for guile here.
+
+@subsection Typical gdb usage
+
+@subsection Typical .gdbinit files
+
+The behavior of gdb can be readily customized through the use of
+@var{.gdbinit} files. A @var{.gdbinit} file is a file named
+@var{.gdbinit} (notice the @qq{.} at the beginning of the file name)
+ that is placed in a user's home directory.
+
+The @var{.gdbinit} file below is from Han-Wen. It sets breakpoints
+for all errors and defines functions for displaying scheme objects
+(ps), grobs (pgrob), and parsed music expressions (pmusic).
+
+@example
+file lily/out/lilypond
+b scm_error
+b programming_error
+b Grob::programming_error
+
+define ps
+ print ly_display_scm($arg0)
+ end
+ define pgrob
+ print ly_display_scm($arg0->self_scm_)
+ print ly_display_scm($arg0->mutable_property_alist_)
+ print ly_display_scm($arg0->immutable_property_alist_)
+ print ly_display_scm($arg0->object_alist_)
+ end
+ define pmusic
+ print ly_display_scm($arg0->self_scm_)
+ print ly_display_scm($arg0->mutable_property_alist_)
+ print ly_display_scm($arg0->immutable_property_alist_)
+ end
+@end example
+
+@subsection Using Guile interactively with LilyPond
+
+In order to experiment with Scheme programming in the LilyPond
+environment, it is convenient to have a Guile interpreter that
+has all the LilyPond modules loaded. This requires the following
+steps.
+
+First, define a Scheme symbol for the active module
+in the .ly file:
+
+@example
+#(module-define! (resolve-module '(guile-user))
+ 'lilypond-module (current-module))
+@end example
+
+Second, place a Scheme function in the .ly file that gives an interactive Guile
+prompt:
+
+@example
+#(top-repl)
+@end example
+
+When the .ly file is compiled, this causes the compilation to be interrupted
+and an interactive guile prompt to appear. When the guile prompt appears,
+the LilyPond active module must be set as the current guile module:
+
+@example
+guile> (set-current-module lilypond-module)
+@end example
+
+Proper operation of these commands can be demonstrated by typing the name
+of a LilyPond public scheme function to see if it's properly defined:
+
+@example
+guile> fret-diagram-verbose-markup
+#<procedure fret-diagram-verbose-markup (layout props marking-list)>
+@end example
+
+If the LilyPond module has not been correctly loaded, an error
+message will be generated:
+
+@example
+guile> fret-diagram-verbose-markup
+ERROR: Unbound variable: fret-diagram-verbose-markup
+ABORT: (unbound-variable)
+@end example
+
+Once the module is properly loaded, any valid LilyPond Scheme expression
+can be entered at the interactive prompt.
+
+After the investigation is complete, the interactive guile interpreter
+can be exited:
+
+@example
+guile> (quit)
+@end example
+
+The compilation of the .ly file will then continue.
+