* Programming without compiling::
* Finding functions::
* Code style::
+* Warnings Errors Progress and Debug Output::
* Debugging LilyPond::
* Tracing object relationships::
* Adding or modifying features::
* Iterator tutorial::
* Engraver tutorial::
* Callback tutorial::
+* Understanding pure properties::
* LilyPond scoping::
+* Scheme->C interface::
* LilyPond miscellany::
@end menu
This command will search all the contents of the directory subdirectory/
and display every line in any of the files that contains
-functionName. The @code{-i} option makes @command{grep} ignore
+functionName. The @option{-i} option makes @command{grep} ignore
case -- this can be very useful if you are not yet familiar with
our capitalization conventions.
@end itemize
+@node Warnings Errors Progress and Debug Output
+@section Warnings, Errors, Progress and Debug Output
+
+@unnumberedsubsec Available log levels
+
+LilyPond has several loglevels, which specify how verbose the output on
+the console should be:
+@itemize
+@item NONE: No output at all, even on failure
+@item ERROR: Only error messages
+@item WARN: Only error messages and warnings
+@item BASIC_PROGRESS: Warnings, errors and basic progress (success, etc.)
+@item PROGRESS: Warnings, errors and full progress messages
+@item INFO: Warnings, errors, progress and more detailed information (default)
+@item DEBUG: All messages, including vull debug messages (very verbose!)
+@end itemize
+
+The loglevel can either be set with the environment variable
+@code{LILYPOND_LOGLEVEL} or on the command line with the @option{--loglevel=...}
+option.
+
+@unnumberedsubsec Functions for debug and log output
+
+LilyPond has two different types of error and log functions:
+@itemize
+
+@item
+If a warning or error is caused by an identified position in the input file,
+e.g. by a grob or by a music expression, the functions of the @code{Input}
+class provide logging functionality that prints the position of the message
+in addition to the message.
+
+@item
+If a message can not be associated with a particular position in an input file,
+e.g. the output file cannot be written, then the functions in the
+@code{flower/include/warn.hh} file will provide logging functionality that
+only prints out the message, but no location.
+
+@end itemize
+
+There are also Scheme functions to access all of these logging functions from
+scheme. In addition, the Grob class contains some convenience wrappers for
+even easier access to these functions.
+
+The message and debug functions in @code{warn.hh} also have an optional
+argument @code{newline}, which specifies whether the message should always
+start on a new line or continue a previous message.
+By default, @code{progress_indication} does NOT start on a new line, but rather
+continue the previous output. They also do not have a particular input
+position associated, so there are no progress functions in the Input class.
+All other functions by default start their output on a new line.
+
+The error functions come in three different flavors: fatal error messages,
+programming error messages and normal error messages. Errors written
+by the @code{error ()} function will cause LilyPond to exit immediately,
+errors by @code{Input::error ()} will continue the compilation, but
+return a non-zero return value of the lilypond call (i.e. indicate an
+unsuccessful program execution). All other errors will be printed on the
+console, but not exit LilyPond or indicate an unsuccessful return code.
+Their only differences to a warnings are the displayed text and that
+they will be shown with loglevel @code{ERROR}.
+
+If the Scheme option @code{warning-as-error} is set, any warning will be
+treated as if @code{Input::error} was called.
+
+
+@unnumberedsubsec All logging functions at a glance
+
+@multitable @columnfractions 0.16 0.42 0.42
+@headitem
+@tab C++, no location
+@tab C++ from input location
+
+@item ERROR
+@tab @code{error ()}, @code{programming_error (msg)}, @code{non_fatal_error (msg)}
+@tab @code{Input::error (msg)}, @code{Input::programming_error (msg)}
+
+@item WARN
+@tab @code{warning (msg)}
+@tab @code{Input::warning (msg)}
+
+@item BASIC
+@tab @code{basic_progress (msg)}
+@tab -
+
+@item PROGRESS
+@tab @code{progress_indication (msg)}
+@tab -
+
+@item INFO
+@tab @code{message (msg)}
+@tab @code{Input::message (msg)}
+
+@item DEBUG
+@tab @code{debug_output (msg)}
+@tab @code{Input::debug_output (msg)}
+
+@item @tab @tab
+
+@headitem
+@tab C++ from a Grob
+@tab Scheme, music expression
+
+@item ERROR
+@tab @code{Grob::programming_error (msg)}
+@tab -
+
+@item WARN
+@tab @code{Grob::warning (msg)}
+@tab @code{(ly:music-warning music msg)}
+
+@item BASIC
+@tab -
+@tab -
+
+@item PROGRESS
+@tab -
+@tab -
+
+@item INFO
+@tab -
+@tab @code{(ly:music-message music msg)}
+
+@item DEBUG
+@tab -
+@tab -
+
+@item @tab @tab
+
+@headitem
+@tab Scheme, no location
+@tab Scheme, input location
+
+@item ERROR
+@tab -
+@tab @code{(ly:error msg args)}, @code{(ly:programming-error msg args)}
+
+@item WARN
+@tab @code{(ly:warning msg args)}
+@tab @code{(ly:input-warning input msg args)}
+
+@item BASIC
+@tab @code{(ly:basic-progress msg args)}
+@tab -
+
+@item PROGRESS
+@tab @code{(ly:progress msg args)}
+@tab -
+
+@item INFO
+@tab @code{(ly:message msg args)}
+@tab @code{(ly:input-message input msg args)}
+
+@item DEBUG
+@tab @code{(ly:debug msg args)}
+@tab -
+
+@end multitable
+
+
+
@node Debugging LilyPond
@section Debugging LilyPond
In order for the Graphviz tool to work, config.make must be modified.
It is probably a good idea to first save a copy of config.make under
a different name. Then, edit config.make by removing every occurrence
-of @code{-DNDEBUG}.
+of @option{-DNDEBUG}.
@item Rebuilding LilyPond
The pdf file can then be viewed with any pdf viewer.
-When compiled without @code{-DNDEBUG}, lilypond may run slower
+When compiled without @option{-DNDEBUG}, lilypond may run slower
than normal. The original configuration can be restored by either
renaming the saved copy of @code{config.make} or rerunning
@code{configure}. Then rebuild lilypond with
Once the patch has been pushed, all the relevant issues should be
closed.
-On Rietveld, the author should log in an close the issue either by
+On Rietveld, the author should log in and close the issue either by
using the @q{Edit Issue} link, or by clicking the circled x icon
to the left of the issue name.
@}
@end example
+Acknowledge functions are called in the order engravers are
+@code{\consist}-ed (the only exception is if you set
+@code{must-be-last} to @code{#t}).
+
+If useful things are to be done to the acknowledged grobs, this
+should be deferred until all the acknowledging has finished, i.e.,
+store the acknowledged grobs and process the information in a
+@code{process-acknowledged ()} or @code{stop-translation-timestep ()}
+function.
+
@node Engraver declaration/documentation
@subsection Engraver declaration/documentation
TODO -- This is a placeholder for a tutorial on callback functions.
+
+@node Understanding pure properties
+@section Understanding pure properties
+
+@menu
+* Purity in LilyPond::
+* Writing a pure function::
+* How purity is defined and stored::
+* Where purity is used::
+* Case studies::
+* Debugging tips::
+@end menu
+
+Pure properties are some of the most difficult properties to understand
+in LilyPond but, once understood, it is much easier to work with
+horizontal spacing. This document provides an overview of what it means
+for something to be @q{pure} in LilyPond, what this purity guarantees,
+and where pure properties are stored and used. It finishes by
+discussing a few case studies for the pure programmer to save you some
+time and to prevent you some major headaches.
+
+
+@node Purity in LilyPond
+@subsection Purity in LilyPond
+Pure properties in LilyPond that do not have any @q{side effects}.
+That is, looking up a pure property should never result in calls to the
+following functions:
+@itemize
+@item @code{set_property}
+@item @code{set_object}
+@item @code{suicide}
+@end itemize
+This means that, if the property is calculated via a callback, this callback
+must not only avoid the functions above but make sure that any functions
+it calls also avoid the functions above. Also, to date in LilyPond, a pure
+function will always return the same value before line breaking (or, more
+precisely, before any version of @code{break_into_pieces} is called). This
+convention makes it possible to cache pure functions and be more flexible
+about the order in which functions are called. For example; Stem #'length has
+a pure property that will @emph{never} trigger one of the functions listed
+above and will @emph{always} return the same value before line breaking,
+independent of where it is called. Sometimes, this will be the actual length
+of the Stem. But sometimes it will not. For example; stem that links up
+with a beam will need its end set to the Y position of the beam at the stem's
+X position. However, the beam's Y positions can only be known after the score
+is broken up in to several systems (a beam that has a shallow slope on a
+compressed line of music, for example, may have a steeper one on an
+uncompressed line). Thus, we only call the impure version of the properties
+once we are @emph{absolutely certain} that all of the parameters needed to
+calculate their final value have been calculated. The pure version provides a
+useful estimate of what this Stem length (or any property) will be, and
+the art of creating good pure properties is trying to get the estimation
+as close to the actual value as possible.
+
+Of course, like Gregory Peck and Tintin, some Grobs will have properties
+that will always be pure. For example, the height of a note-head in
+not-crazy music will never depend on line breaking or other parameters
+decided late in the typesetting process. Inversely, in rare cases,
+certain properties are difficult to estimate with pure values. For
+example, the height of a Hairpin at a certain cross-section of its
+horizontal span is difficult to know without knowing the horizontal
+distance that the hairpin spans, and LilyPond provides an
+over-estimation by reporting the pure height as the entire height of the
+Hairpin.
+
+Purity, like for those living in a convent, is more like a contract than
+an @emph{a priori}. If you write a pure-function, you are promising
+the user (and the developer who may have to clean up after you) that
+your function will not be dependent on factors that change at different
+stages of the compilation process (compilation of a score, not of
+LilyPond).
+
+One last oddity is that purity, in LilyPond, is currently limited
+exclusively to things that have to do with Y-extent and positioning.
+There is no concept of @q{pure X} as, by design, X is always the
+independent variable (i.e. from column X1 to column X2, what will be the
+Y height of a given grob). Furthermore, there is no purity for
+properties like color, text, and other things for which a meaningful notion
+of estimation is either not necessary or has not yet been found. For example,
+even if a color were susceptible to change at different points of the
+compilation process, it is not clear what a pure estimate of this color
+would be or how this pure color could be used. Thus, in this document and
+in the source, you will see purity discussed almost interchangeably with
+Y-axis positioning issues.
+
+
+@node Writing a pure function
+@subsection Writing a pure function
+Pure functions take, at a minimum, three arguments: the @var{grob}, the
+starting column at which the function is being evaluated (hereafter
+referred to as @var{start}), and the end column at which the grob is
+being evaluated (hereafter referred to as @var{end}). For items,
+@var{start} and @var{end} must be provided (meaning they are not optional)
+but will not have a meaningful impact on the result, as items only occupy
+one column and will thus yield a value or not (if they are not in the range
+from @var{start} to @var{end}). For spanners however, @var{start} and
+@var{end} are important, as we may can get a better pure estimation of a
+slice of the spanner than considering it on the whole. This is useful
+during line breaking, for example, when we want to estimate the Y-extent
+of a spanner broken at given starting and ending columns.
+
+If the pure function you're writing takes more than three arguments
+(say, for example, a chained offset callback), this is not a problem:
+just make sure that the grob is the first argument and that start and
+end are the last two arguments.
+
+
+@node How purity is defined and stored
+@subsection How purity is defined and stored
+Purity can currently be defined two different ways in LilyPond that
+correspond to two types of scenarios. In one scenario, we know that a
+callback is pure, but we are not necessarily certain what properties
+will use this callback. In another, we want a property to be pure, but
+we don't want to guarantee that its callback function will be pure in
+all circumstances.
+
+In the first scenario, we register the callback in define-grobs.scm in
+one of four places depending on what the function does.
+
+@itemize
+@item @code{pure-print-functions}: If finding a print function's vertical
+extent does not have any @q{side effects} we register it here. We then
+don't have to set the pure Y-extent property, which will be taken from the
+stencil.
+
+@item @code{pure-print-to-height-conversions}: If a stencil can
+eventually be used to glean a grob's Y-extent but is not pure (meaning
+it will have a different height at different stages of the compilation
+process), we add it to this list along with a function for the pure
+Y-extent.
+
+@item @code{pure-conversions-alist}: This list contains pairs of
+functions and their pure equivalents. It is onto but not one-to-one.
+
+@item @code{pure-functions}: Like pure-print-functions in that they work
+for both pure and impure values, but they do not return a stencil.
+@end itemize
+
+At all stages of the compilation process, when LilyPond wants the pure
+version of a property, it will consult these lists and see if it can get
+this property for a given Grob. Note that you do @emph{not} need to
+register the pure property in the grob itself. For example, there is no
+property @q{pure-Y-extent}. Rather, by registering these functions as
+defined above, every time LilyPond needs a pure property, it will check
+to see if a Grob contains one of these functions and, if so, will use
+its value. If LilyPond cannot get a pure function, it will return a
+value of @code{##f} for the property.
+
+LilyPond is smart enough to know if a series of chained functions are
+pure. For example, if a Y-offset property has four chained functions
+and all of them have pure equivalents, LilyPond will read the four pure
+equivalents when calculating the pure property. However, if even one is
+impure, LilyPond will not return a pure property for the offset (instead
+returning something like @code{#f} or @code{'()}) and will likely wreak
+havoc on your score.
+
+In the second scenario, we create an unpure-pure-container (unpure is
+not a word, but hey, neither was Lilypond until the 90s). For example:
+
+@example
+#(define (foo grob)
+ '(-1 . 1))
+
+#(define (bar grob start end)
+ '(-2 . 2))
+
+\override Stem #'length = #(ly:make-unpure-pure-container foo bar)
+@end example
+
+This is useful if we want to:
+
+@itemize
+@item create overrides that have pure alternatives (should not be used
+in development, but useful for users)
+
+@item use return values that are not functions (i.e. pairs or booleans)
+for either pure or unpure values.
+
+@item allow a function to be considered pure in a limited amount of
+circumstances. This is useful if we are sure that, when associated with
+one grob a function will be pure but not necessarily with another grob
+that has different callbacks.
+@end itemize
+
+Items can only ever have two pure heights: their actual pure height if
+they are between @q{start} and @q{end}, or an empty interval if they are
+not. Thus, their pure property is cached to speed LilyPond up. Pure
+heights for spanners are generally not cached as they change depending
+on the start and end values. They are only cached in certain particular
+cases. Before writing a lot of caching code, make sure that it is a
+value that will be reused a lot.
+
+
+@node Where purity is used
+@subsection Where purity is used
+Pure Y values must be used in any functions that are called before
+line breaking. Examples of this can be seen in
+@code{Separation_items::boxes} to construct horizontal skylines and in
+@code{Note_spacing::stem_dir_correction} to correct for optical
+illusions in spacing. Pure properties are also used in the calculation
+of other pure properties. For example, the @code{Axis_group_interface}
+has pure functions that look up other pure functions.
+
+Purity is also implicitly used in any functions that should only ever
+return pure values. For example, extra-spacing-height is only ever used
+before line-breaking and thus should never use values that would only be
+available after line breaking. In this case, there is no need to create
+callbacks with pure equivalents because these functions, by design, need
+to be pure.
+
+To know if a property will be called before and/or after line-breaking
+is sometimes tricky and can, like all things in coding, be found by
+using a debugger and/or adding @var{printf} statements to see where they
+are called in various circumstances.
+
+
+@node Case studies
+@subsection Case studies
+In each of these case studies, we expose a problem in pure properties, a
+solution, and the pros and cons of this solution.
+
+@subheading Time signatures
+A time signature needs to prevent accidentals from passing over or under
+it, but its extent does not necessarily extend to the Y-position of
+accidentals. LilyPond's horizontal spacing sometimes makes a line of
+music compact and, when doing so, allows certain columns to pass over
+each other if they will not collide. This type of passing over is not
+desirable with time signatures in traditional engraving. But how do we
+know if this passing over will happen before line breaking, as we are
+not sure what the X positions will be? We need a pure estimation of how
+much extra spacing height the time signatures would need to prevent this
+form of passing over without making this height so large as to
+overly-distort the Y-extent of an system, which could result in a very
+@q{loose} looking score with lots of horizontal space between columns.
+So, to approximate this extra spacing height, we use the Y-extent of a
+time signature's next-door-neighbor grobs via the pure-from-neighbor
+interface.
+
+@itemize
+@item pros: By extending the extra spacing height of a time signature to
+that of its next-door-neighbors, we make sure that grobs to the right of
+it that could pass above or below it do not.
+
+@item cons: This over-estimation of the vertical height could prevent
+snug vertical spacing of systems, as the system will be registered as
+being taller at the point of the time signature than it actually is.
+This approach can be used for clefs and bar lines as well.
+@end itemize
+
+@subheading Stems
+As described above, Stems need pure height approximations when they are
+beamed, as we do not know the beam positions before line breaking. To
+estimate this pure height, we take all the stems in a beam and find
+their pure heights as if they were not beamed. Then, we find the union
+of all these pure heights and take the intersection between this
+interval (which is large) and an interval going from the note-head of a
+stem to infinity in the direction of the stem so that the interval stops
+at the note head.
+
+@itemize
+@item pros: This is guaranteed to be at least as long as the beamed
+stem, as a beamed stem will never go over the ideal length of the
+extremal beam of a stem.
+
+@item cons: Certain stems will be estimated as being too long, which
+leads to the same problem of too-much-vertical-height as described
+above.
+
+@end itemize
+
+
+@node Debugging tips
+@subsection Debugging tips
+A few questions to ask yourself when working with pure properties:
+
+@itemize
+@item Is the property really pure? Are you sure that its value could
+not be changed later in the compiling process due to other changes?
+
+@item Can the property be made to correspond even more exactly with the
+eventual impure property?
+
+@item For a spanner, is the pure property changing correctly depending
+on the starting and ending points of the spanner?
+
+@item For an Item, will the item's pure height need to act in horizontal
+spacing but not in vertical spacing? If so, use extra-spacing-height
+instead of pure height.
+
+@end itemize
+
+
@node LilyPond scoping
@section LilyPond scoping
-The Lilypond language has a concept of scoping, i.e. you can do
+The Lilypond language has a concept of scoping, i.e. you can do:
@example
foo = 1
dispersed, either by being stored module-locally, or in weak hash
tables.
+
+@node Scheme->C interface
+@section Scheme->C interface
+
+Most of the C functions interfacing with Guile/Scheme used in LilyPond
+are described in the API Reference of the
+@uref{http://www.gnu.org/software/guile/manual/html_node/index.html,
+GUILE Reference Manual}.
+
+The remaining functions are defined in @file{lily/lily-guile.cc},
+@file{lily/include/lily-guile.hh} and
+@file{lily/include/lily-guile-macros.hh}.
+Although their names are meaningful there's a few things you should know
+about them.
+
+@menu
+* Comparison::
+* Conversion::
+@end menu
+
+@node Comparison
+@subsection Comparison
+
+This is the trickiest part of the interface.
+
+Mixing Scheme values with C comparison operators won't produce any crash
+or warning when compiling but must be avoided:
+
+@example
+scm_string_p (scm_value) == SCM_BOOL_T
+@end example
+
+As we can read in the reference, @code{scm_string_p} returns a Scheme
+value: either @code{#t} or @code{#f} which are written @code{SCM_BOOL_T}
+and @code{SCM_BOOL_F} in C. This will work, but it is not following
+to the API guidelines. For further information, read this discussion:
+
+@smallexample
+@uref{http://lists.gnu.org/archive/html/lilypond-devel/2011-08/msg00646.html}
+@end smallexample
+
+There are functions in the Guile reference that returns C values
+instead of Scheme values. In our example, a function called
+@code{scm_is_string} (described after @code{string?} and @code{scm_string_p})
+returns the C value 0 or 1.
+
+So the best solution was simply:
+
+@example
+scm_is_string (scm_value)
+@end example
+
+There a simple solution for almost every common comparison. Another example:
+we want to know if a Scheme value is a non-empty list. Instead of:
+
+@example
+(scm_is_true (scm_list_p (scm_value)) && scm_value != SCM_EOL)
+@end example
+
+one can usually use:
+
+@example
+scm_is_pair (scm_value)
+@end example
+
+since a list of at least one member is a pair. This test is
+cheap; @code{scm_list_p} is actually quite more complex since it makes
+sure that its argument is neither a `dotted list' where the last pair
+has a non-null @code{cdr}, nor a circular list. There are few
+situations where the complexity of those tests make sense.
+
+Unfortunately, there is not a @code{scm_is_[something]} function for
+everything. That's one of the reasons why LilyPond has its own Scheme
+interface. As a rule of thumb, tests that are cheap enough to be
+worth inlining tend to have such a C interface. So there is
+@code{scm_is_pair} but not @code{scm_is_list}, and @code{scm_is_eq}
+but not @code{scm_is_equal}.
+
+@subheading General definitions
+
+@subsubheading bool to_boolean (SCM b)
+
+Return @code{true} if @var{b} is @code{SCM_BOOL_T}, else return @code{false}.
+
+This should be used instead of @code{scm_is_true} and
+@code{scm_is_false} for properties since in Lilypond, unset properties
+are read as an empty list, and by convention unset Boolean properties
+default to false. Since both @code{scm_is_true} and
+@code{scm_is_false} only compare with @code{##f} in line with what
+Scheme's conditionals do, they are not really useful for checking the
+state of a Boolean property.
+
+@subsubheading bool ly_is_[something] (args)
+
+Behave the same as scm_is_[something] would do if it existed.
+
+@subsubheading bool is_[type] (SCM s)
+
+Test whether the type of @var{s} is [type].
+[type] is a LilyPond-only set of values (direction, axis...). More
+often than not, the code checks Lilypond specific C++-implemented
+types using
+
+@subsubheading [type *] unsmob_[type] (SCM s)
+
+This tries converting a Scheme object to a pointer of the desired
+kind. If the Scheme object is of the wrong type, a pointer value
+of@w{ }@code{0} is returned, making this suitable for a Boolean test.
+
+@node Conversion
+@subsection Conversion
+
+@subheading General definitions
+
+@subsubheading bool to_boolean (SCM b)
+
+Return @code{true} if @var{b} is @code{SCM_BOOL_T}, else return @code{false}.
+
+This should be used instead of @code{scm_is_true} and @code{scm_is_false}
+for properties since empty lists are sometimes used to unset them.
+
+@subsubheading [C type] ly_scm2[C type] (SCM s)
+
+Behave the same as scm_to_[C type] would do if it existed.
+
+@subsubheading [C type] robust_scm2[C type] (SCM s, [C type] d)
+
+Behave the same as scm_to_[C type] would do if it existed.
+Return @var{d} if type verification fails.
+
+
@node LilyPond miscellany
@section LilyPond miscellany
most of them are not drawn. See the break-visibility property in
item-interface.
+Here is another e-mail exchange. Janek WarchoĊ asked for a starting point
+to fixing 1301 (change clef colliding with notes). Neil Puttock replied:
+
+The clef is on a loose column (it floats before the head), so the
+first place I'd look would be lily/spacing-loose-columns.cc (and
+possibly lily/spacing-determine-loose-columns.cc).
+I'd guess the problem is the way loose columns are spaced between
+other columns: in this snippet, the columns for the quaver and tuplet
+minim are so close together that the clef's column gets dumped on top
+of the quaver (since it's loose, it doesn't influence the spacing).
@node Info from Han-Wen email
@subsection Info from Han-Wen email
projects / lilypond.git / blobdiff