release: 1.4.2

[lilypond.git] / Documentation / user / internals.itely

@c -*-texinfo-*-
@c Note:
@c
@c A menu is needed before every deeper *section nesting of @nodes
@c Run M-x texinfo-all-menus-update
@c to automagically fill in these menus
@c before saving changes


@node Internals
@chapter Internals


When translating the input to notation, there are number of distinct
phases.  We list them here:

@table @b

@item Parsing:

The LY file is read, and converted to a list of @code{Scores}, which
each contain @code{Music} and paper/midi-definitions. Here @code{Music},
@code{Pitch} and @code{Duration}  objects are created.

@item Interpreting music
@cindex interpreting music

All music events are "read" in the same order as they would be played
(or read from paper). At every step of the interpretation, musical
events are delivered to
interpretation contexts,
@cindex engraver
which use them to build @code{Grob}s (or MIDI objects, for MIDI output).

In this stage @code{Music_iterators} do a traversal of the @code{Music}
structure. The music events thus encountered are reported to
@code{Translator}s, a set of objects that collectively form interpretation
contexts.


@item Prebreaking

@cindex prebreaking

At places where line breaks may occur, clefs and bars are prepared for
a possible line break. 

@item Preprocessing

@cindex preprocessing

In this stage, all information that is needed to determine line breaking
is computed. 

@item Break calculation:

The lines and horizontal positions of the columns are determined.

@item Breaking

Relations between all grobs are modified to reflect line breaks: When a
spanner, e.g. a slur, crosses a line-break, then the spanner is "broken
into pieces", for every line that the spanner is in, a copy of the grob
is made. A substitution process redirects all grob-reference so that
each spanner grob will only reference other grobs in the same line.

@item Outputting:

All vertical dimensions and spanning objects are computed, and all grobs
are output, line by line. The output is encoded in the form of
@code{Molecule}s

@end table

The data types that are mentioned here are all discussed in this
section.

@menu
* Input location::              
* Moment::                      
* Duration::                    
* Pitch data type::             
* Music::                       
* Music_iterator::              
* Translator::                  
* Grobs::                       Graphical object  
* Molecules::                   Molecules are stand-alone descriptions of output
* Font metrics::                Font metrics
* Miscellaneous Scheme functions::  
@end menu

@ignore
  Why not use Scheme syntax for the functions below, such as
  (ly-input-location? obj) and (ly-get-mus-property m sym) ?

  /MB
@end ignore

@node Input location
@section Input location

@c The parser generates

Input location objects point to a location in the input file. This
location is used to generate error messages and to enable the point and
click output.

@defun ly-input-location? obj
Type predicate, return true if @var{obj} is an input location.
@end defun




@node Moment
@section Moment

Moment is a rational number. Since GUILE doesn't support them natively,
so we created our own rational data type.

@defun moment? obj
Type predicate, return true if @var{obj} is a moment.
@end defun

@defun make-moment num den
create the rational number @var{num}/@var{den}. 
@end defun

@node Duration
@section Duration

A duration is a musical duration, i.e. a length of time described by a
power of two (whole, half, quarter, etc.) and a number of augmentation
dots. 

@defun make-duration length dotcount

@var{length} is the negative logarithm (base 2) of the duration:
1 is a half note, 2 is a quarter note, 3 is an eighth
note, etc.  The number of dots after the note is given by
@var{dotcount}.
@end defun


@defun duration? obj
Type predicate, return true if @var{obj} is a duration.
@end defun

@node Pitch data type
@section Pitch data type



@defun make-pitch octave note shift

@var{octave} is specified by an integer, zero for the octave containing
middle C.  @var{note} is a number from 0 to 6, with 0 corresponding to C
and 6 corresponding to B.  The shift is zero for a natural, negative for
flats, or positive for sharps.
@end defun

@defun pitch-octave p 
extract the octave from pitch @var{p}.
@end defun

@defun pitch-notename p
extract the note name from pitch  @var{p}.
@end defun

@defun pitch-alteration p
extract the alteration from pitch  @var{p}.
@end defun

@defun pitch-semitones p
calculate the number of semitones of @var{p} from central C.
@end defun

@defun Pitch::transpose t p
Transpose @var{p} by the amount @var{t}, where @var{t} is the pitch that
central C is transposed to. 
@end defun


@node Music
@section Music

Music is the data type that music expressions are stored in. The data
type does not yet offer many manipulations.

@defun ly-get-mus-property m sym
Get the property @var{sym} of music expression @var{m}.
@end defun

@defun ly-set-mus-property m sym val
Set property @var{sym} in music expression @var{m} to @var{val}.
@end defun

@defun ly-make-music name
Make a music object/expression of type @var{name}. Warning: this
interface will likely change in the near future.
@end defun

@defun music? obj
Type predicate,  return true if @var{obj} is a music object.
@end defun

@defun ly-music-name music
Print the name of @var{music}.
@end defun


@node Music_iterator
@section Music_iterator

Music_iterator is an object type that traverses the Music structure and
reports the events it finds to interpretation contexts. It is not yet
user-serviceable.

@defun c++-function? obj
Type predicate, return true if @var{obj} is a  c++-function.
Music_iterator are created from schemified C++ constructors. Such a
constructor is a @code{c++-function}.
@end defun

@node Translator
@section Translator

Translators are the building blocks of contexts. They are not yet user
accessible.

@defun ly-get-trans-property tr sym
retrieve the value of @var{sym} from context @var{tr}
@end defun

@defun ly-set-trans-property tr sym val
set value of property @var{sym} in context @var{tr} to @var{val}.
@end defun


@node Grobs
@section Grobs

This section is about Grobs (short for Graphical Objects), which are
formatting objects used to create the final output. This material is
normally the domain of LilyPond gurus, but occasionally, a normal user
also has to deal with grobs.

The most simple interaction with Grobs are when you use
@code{\override}:

@example
        \property Voice.Stem \override #'direction = #1
@end example

This piece of lily input causes all stem objects to be stem-up
henceforth.  In effect, you are telling lilypond to extend the definition
of the `Stem' grob with the setting @code{direction := 1}.

@menu
* What is a grob?::             
* Callbacks::                   
* Setting grob properties::     
* Grob interfaces::             
* Items and Spanners::          
* Grob Scheme functions::       
@end menu



@node What is a grob?
@subsection What is a grob?

In music notation, lots of symbols are related in some way.  You can
think of music notation as a graph where nodes are formed by the
symbols, and the arcs by their relations. A grob is a node in that graph.
The directed edges in the graph are formed by references to other grobs
(i.e. pointers).
This big graph of grobs specifies the notation problem. The solution of
this problem is a description of the printout in closed form, i.e. a
list of values.  These values are Molecules. (see @ref{Molecules})

All grobs have an X and Y-position on the page.  These X and Y positions
are stored in a relative format, so they can easily be combined by
stacking them, hanging one grob to the side of another, and coupling
them into a grouping-grob.

Each grob has a reference point (a.k.a.  parent): the position of a grob
is stored relative to that reference point. For example the X-reference
point of a staccato dot usually is the note head that it applies
to. When the note head is moved, the staccato dot moves along
automatically.

If you keep following offset reference points, you will always end up at
the root object. This root object is called @code{Line_of_score}, and it
represents a system (i.e. a line of music).

All grobs carry a set of grob-properties.  In the Stem example above,
the property @code{direction} is set to value @code{1}.  The function
that draws the symbol (@code{Stem::brew_molecule}) uses the value of
@code{direction} to determine how to print the stem and the flag.  The
appearance of a grob is determined solely by the values of its
properties.

A grob is often associated with a symbol, but some grobs do not print
any symbols. They take care of grouping objects. For example, there is a
separate grob that stacks staffs vertically. The @code{NoteCollision}
is also an abstract grob: it only moves around chords, but doesn't print
anything.

A complete list of grob types is found in the generated documentation.


@node Callbacks
@subsection Callbacks

Offsets of grobs are relative to a parent reference point. Most
positions are not known when an object is created, so these are
calculated as needed. This is done by adding a callback for a specific
direction.

Suppose you have the following code in a .ly file.
@example
        #(define (my-callback gr axis)
                (*  2.0 (get-grob-property gr 'direction))
        )

....

        \property Voice.Stem \override #'Y-offset-callbacks = #(list
                        my-callback)
@end example

When the Y-offset of a Stem object is needed, LilyPond will
automatically execute all callbacks for that object. In this case, it
will find @code{my-callback}, and execute that. The result is that the
stem is translated by two staff spaces in its direction.

(note: @code{Y-offset-callbacks} is also a property)



Offset callbacks can be stacked, i.e.

@example
        \property .... \override #'Y-offset-callbacks = #(list
                callback1 callback2 callback3)

@end example

The callbacks will be executed in the order @code{callback3 callback2
callback1}. This is used for quantized positioning: the staccato dot is
above or below a note head, and it must not be on a staff-line.  To
achieve this, the staccato dot has two callbacks: one that positions the
grob above or below the note head, and one that rounds the Y-position of
the grob to the nearest open space.

Similarly, the size of a grob are determined through callbacks, settable
with grob properties @code{X-extent-callback} and
@code{Y-extent-callback}.  There can be only one extent-callback for
each axis. No callback (Scheme value @code{#f}) means: "empty in this
direction". If you fill in a pair of numbers, that pair hard-codes the
extent in that coordinate.


@node Setting grob properties
@subsection Setting grob properties

Grob properties are stored as GUILE association lists, with symbols as
keys.  In GUILE you can access these using functions described in
Section @ref{Grob Scheme functions}.  From C++, grob properties can be
accessed using these functions:

@example
  SCM  get_grob_property (SCM) const;
  void set_grob_property (const char * , SCM val);
  void set_immutable_grob_property (const char * , SCM val);
  void set_immutable_grob_property (SCM key, SCM val);  
  void set_grob_property (SCM , SCM val);  
  void set_grob_pointer (const char*, SCM val);
  SCM  remove_grob_property (const char* nm);
@end example

All lookup functions identify undefined properties with end-of-list
(i.e. @code{'()} in Scheme or @code{SCM_EOL} in C)

Properties are stored in two ways:
@itemize @bullet
@item mutable properties.
Grob properties that change from object to object. The storage of
these are private to a grob. For example pointers to other grobs are
always stored in the mutable properties.

@item immutable properties.
Grob properties that are shared across different grobs of the same
type. The storage is shared, and hence it is read-only. Typically, this
is used to store function callbacks, and default settings. They are
initially read from @file{scm/grob-description.scm}.
@end itemize

You can change immutable grob properties with the \override syntax:

@example
        \property Voice.Stem \override #'direction = #1
@end example

This will push the entry @code{'(direction . 1)} on the immutable
property list for stems, in effect overriding the setting from
@file{scm/grob-description.scm}. This can be undone by 

@example
        \property Voice.stem \revert #'direction
@end example

There is also a shorthand,

@example
        \property Context.GrobType \set #'prop = #VAL
@end example

this does a @code{\revert} followed by a @code{\override}

You can change mutable properties with \outputproperty. This construct
looks like

@example
        \context ContextName \outputproperty @var{pred} #@var{sym} = #@var{val}
@end example

In this case, in every grob that satisfies @var{pred}, the grob property
 @var{sym} is set to @var{val}.  For example

@example
        \outputproperty
                #(lambda (gr) (string? (ly-get-grob-property gr
                        'text)))
                #'extra-offset = #'(-1.0 . 0.0)
@end example

This shifts all grobs that have a @code{text} property one staff
space to the left. This mechanism is rather clumsy to use, but it allows
you tweak any setting of any grob.


@node Grob interfaces
@unnumberedsubsec Grob interfaces

Grob properties form a name space where you can set variables per
object.  Each object however, may have multiple functions. For example,
consider a dynamic symbol, such @code{\ff} (fortissimo). It is printed
above or below the staff, it is a dynamic sign, and it is a kind of
text.

To reflect this different functions of a grob, procedures and variables
are grouped into so-called interfaces.  The dynamic text for example
supports the  following interfaces:
@table @code 
@item font-interface
  The glyph is built from characters from a font, hence the
@code{font-interface}. For objects supporting @code{font-interface}, you
can select alternate fonts by setting @code{font-style},
@code{font-point-size}, etc.

@item dynamic-interface
  Dynamic interface is not associated with any variable or function in
particular, but this makes it possible to distinguish this grob from
other similar grobs (like @code{TextScript}), that have no meaning of
dynamics.

@item text-interface
  This interface is for texts that are to be set using special routines
to stack text into lines, using kerning, etc.

@item general-grob-interface
  This interface is supported by all grob types.
@end table



@node Items and Spanners
@unnumberedsubsec Items and Spanners

Grobs can also be distinguished in their role in the horizontal spacing.
Many grobs define constraints on the spacing by their sizes. For
example, note heads, clefs, stems, and all other symbols with a fixed
shape.  These grobs form a subtype called @code{Item}.

Other grobs have a shape that depends on the horizontal spacing. For
example, slur, beam, tie, etc. These grobs form a subtype called
@code{Spanner}. All spanners have two span-points (these must be
@code{Item}s), one on the left and one on the right. The left bound is
also the X-reference point of the spanner.

Some items need special treatment for line breaking. For example, a
clef is normally only printed at the start of a line (i.e. after a line
break).  To model this, `breakable' items (clef, key signature, bar lines,
etc.) are copied twice. Then we have three versions of each breakable
item: one version if there is no line break, one version that is printed
before the line break (at the end of a system), one version that is
printed after the line break.

Whether these versions are visible and take up space, is determined by
the outcome of the @code{visibility-lambda}. This grob property is a
function taking a direction (-1, 0 or 1) as argument. It returns a cons
of booleans, signifying whether this grob should be transparent and have
no extent.

@node Grob Scheme functions
@unnumberedsubsec Grob Scheme functions

Grob properties can be manipulated from Scheme. In practice, most
manipulations are coded in C++ because of tradition.

@defun ly-get-grob-property g sym
  Get the value of a value in grob @var{g} of property @var{sym}. It
will return @code{'()} (end-of-list) if @var{g} doesn't have @var{sym} set.
@end  defun

@defun ly-set-grob-property g sym val
Set @var{sym} in grob @var{g} to value @var{val}
@end defun

@defun ly-get-spanner-bound spanner dir
Get one of the bounds of @var{spanner}. @var{dir} may be @code{-1} for
left, and @code{1} for right.
@end defun

@defun ly-grob? g
Typecheck: is @var{g} a grob?
@end defun




@node Molecules
@section Molecules

@cindex Molecule
@cindex Atom
@cindex Output description

The objective of any typesetting system is to put ink on paper in the
right places. For LilyPond, this final stage is left to the @TeX{} and
the printer subsystem. For lily, the last stage in processing a score is
outputting a description of what to put where.  This description roughly
looks like

@example
        PUT glyph AT (x,y)
        PUT glyph AT (x,y)
        PUT glyph AT (x,y) 
@end example

you merely have to look at the tex output of lily to see this.
Internally these instructions are encoded in Molecules.@footnote{At some
point LilyPond also contained Atom-objects, but they have been replaced
by Scheme expressions, making the name outdated.}  A molecule is
what-to-print-where information that also contains dimension information
(how large is this glyph?).

Conceptually, Molecules can be constructed from Scheme code, by
translating a Molecule and by combining two molecules. In BNF
notation:

@example
Molecule  :: COMBINE Molecule Molecule
           | TRANSLATE Offset Molecule
           | GLYPH-DESCRIPTION
           ;
@end example

If you are interested in seeing how this information is stored, you
can run with the @code{-f scm} option. The scheme expressions are then
dumped in the output file.

All visible, i.e. non-transparent, grobs have a callback to create a
Molecule. The name of the property is @code{molecule-callback}, and its
value should be a Scheme function taking one argument (the grob) and
returning a Molecule.  Most molecule callbacks are written in C++, but
you can also write them in Scheme. An example is provided in
@code{input/regression/molecule-hacking.ly}.


@defun molecule? m
type predicate.
@end defun

@defun ly-combine-molecule-at-edge  mol1 axis dir mol2 padding
Construct a molecule by putting @var{mol2} next to
@var{mol1}. @var{axis} can be 0 (x-axis) or 1 (y-axis), @var{dir} can be
-1 (left or down) or 1 (right or up).  @var{padding} specifies extra
space to add in between measured in global staff space.
@end defun

@defun ly-get-molecule-extent! mol axis
Return a pair of numbers signifying the extent of @var{mol} in
@var{axis} direction (0 or 1 for x and y axis respectively).
@end defun

@defun ly-set-molecule-extent! mol axis extent
Set the extent (@var{extent} must be a pair of numbers) of @var{mol} in 
@var{axis} direction (0 or 1 for x- and y-axis respectively).

Note that an extent @code{(A . B)} is an interval and hence @code{A} is
smaller than @code{B}, and is often negative.

@end defun

@node Font metrics
@section Font metrics

The font object represents the metric information of a font. Every font
that is loaded into LilyPond can be accessed via Scheme. 

LilyPond only needs to know the dimension of glyph to be able to process
them. This information is stored in font metric files. LilyPond can read
two types of font-metrics: @TeX{} Font Metric files (TFM files) and
Adobe Font Metric files (AFM files).  LilyPond will always try to load
AFM files first since they are more versatile.

@defun ly-get-default-font gr
This returns the default font for grob @var{gr}.
@end defun

@defun ly-find-glyph-by-name font name
This function retrieves a Molecule for the glyph named @var{name} in
@var{font}.  The font must be available as an AFM file.
@cindex afm file

@end defun

@node Miscellaneous Scheme functions
@section Miscellaneous Scheme functions

 
@defun ly-warn msg
Scheme callable function to issue the warning @code{msg}.
@end defun

@defun ly-version
Return the current lilypond version as a list, e.g.
@code{(1 3 127 uu1)}. 
@end defun

@defun ly-gulp-file name
Read the file named @var{name}, and return its contents in a string. The
file is looked up using the lilypond search path.

@end defun

@defun dir?
type predicate. A direction is a -1, 0 or 1, where -1 represents left or
down and 1 represents right or up.
@end defun

@defun ly-number->string num
 converts @var{num} to a string without generating many decimals. It
leaves a space at the end.
@end defun

@defun set-lily-option sym val
 Set a global option for the program. Supported options  include
@table @code
@item midi-debug
If set to true, generate human  readable MIDI
@end table

This function is useful to call from the command line: @code{lilypond -e
"(set-lily-option 'midi-debug #t)"}

@end defun