Minor fixes from discussion.

[lilypond.git] / Documentation / user / programming-interface.itely

@c -*- coding: utf-8; mode: texinfo; -*-
@node Interfaces for programmers
@chapter Interfaces for programmers

Advanced tweaks may be performed by using Scheme.  If you are
not familiar with Scheme, you may wish to read our
@ref{Scheme tutorial}.

@menu
* Programmer interfaces for input::  
* Markup programmer interface::  
* Contexts for programmers::    
@end menu


@node Programmer interfaces for input
@section Programmer interfaces for input

This section discusses how to create functions within LilyPond.

@menu
* Input variables and Scheme::  
* Internal music representation::  
* Extending music syntax::      
* Manipulating music expressions::  
* Displaying music expressions::  
* Using LilyPond syntax inside Scheme::  
@end menu

@node Input variables and Scheme
@subsection Input variables and Scheme


The input format supports the notion of variables: in the following
example, a music expression is assigned to a variable with the name
@code{traLaLa}.
@example
traLaLa = @{ c'4 d'4 @}
@end example

@noindent

There is also a form of scoping: in the following example, the
@code{\layout} block also contains a @code{traLaLa} variable, which is
independent of the outer @code{\traLaLa}.
@example
traLaLa = @{ c'4 d'4 @}
\layout @{ traLaLa = 1.0 @}
@end example
@c
In effect, each input file is a scope, and all @code{\header},
@code{\midi}, and @code{\layout} blocks are scopes nested inside that
toplevel scope.

Both variables and scoping are implemented in the GUILE module system.
An anonymous Scheme module is attached to each scope.  An assignment of
the form
@example
traLaLa = @{ c'4 d'4 @}
@end example

@noindent
is internally converted to a Scheme definition
@example
(define traLaLa @var{Scheme value of ``@code{... }''})
@end example

This means that input variables and Scheme variables may be freely
mixed.  In the following example, a music fragment is stored in the
variable @code{traLaLa}, and duplicated using Scheme.  The result is
imported in a @code{\score} block by means of a second variable
@code{twice}:
@example
traLaLa = @{ c'4 d'4 @}

#(define newLa (map ly:music-deep-copy
  (list traLaLa traLaLa)))
#(define twice
  (make-sequential-music newLa))

@{ \twice @}
@end example

In the above example, music expressions can be `exported' from the
input to the Scheme interpreter.  The opposite is also possible.  By
wrapping a Scheme value in the function @code{ly:export}, a Scheme
value is interpreted as if it were entered in LilyPond syntax.  Instead
of defining @code{\twice}, the example above could also have been
written as
@example
@dots{}
@{ #(ly:export (make-sequential-music (list newLa))) @}
@end example

Scheme code is evaluated as soon as the parser encounters it.  To
define some scheme code in a macro (to be called later), use

@example
#(define (nopc)
  (ly:set-option 'point-and-click #f))

...
#(nopc)
@{ c'4 @}
@end example


@refbugs

Mixing Scheme and LilyPond identifiers is not possible with the
@code{--safe} option.


@node Internal music representation
@subsection Internal music representation

When a music expression is parsed, it is converted into a set of
Scheme music objects.  The defining property of a music object is that
it takes up time.  Time is a rational number that measures the length
of a piece of music in whole notes.

A music object has three kinds of types:
@itemize @bullet
@item
music name: Each music expression has a name.  For example, a note
leads to a @internalsref{NoteEvent}, and @code{\simultaneous} leads to
a @internalsref{SimultaneousMusic}.  A list of all expressions
available is in the Program reference manual, under
@internalsref{Music expressions}.

@item
`type' or interface: Each music name has several `types' or
interfaces, for example, a note is an @code{event}, but it is also a
@code{note-event}, a @code{rhythmic-event}, and a
@code{melodic-event}.  All classes of music are listed in the
Profram reference, under
@internalsref{Music classes}.

@item
C++ object: Each music object is represented by a C++ object.  For
technical reasons, different music objects may be represented by
different C++ object types.  For example, a note is @code{Event}
object, while @code{\grace} creates a @code{Grace_music} object.  We
expect that distinctions between different C++ types will disappear
in the future.
@end itemize

The actual information of a music expression is stored in properties.
For example, a @internalsref{NoteEvent} has @code{pitch} and
@code{duration} properties that store the pitch and duration of that
note.  A list of all properties available is in the internals manual,
under @internalsref{Music properties}.

A compound music expression is a music object that contains other
music objects in its properties.  A list of objects can be stored in
the @code{elements} property of a music object, or a single `child'
music object in the @code{element} object.  For example,
@internalsref{SequentialMusic} has its children in @code{elements},
and @internalsref{GraceMusic} has its single argument in
@code{element}.  The body of a repeat is stored in the @code{element}
property of @internalsref{RepeatedMusic}, and the alternatives in
@code{elements}.




@node Extending music syntax
@subsection Extending music syntax

@c TODO: rewrite example.
@c The use of FUNC as example  argument is rather confusing.

The syntax of composite music expressions, like @code{\repeat},
@code{\transpose}, and @code{\context} follows the general form of

@example
\@code{keyword} @var{non-music-arguments} @var{music-arguments}
@end example

Such syntax can also be defined as user code.  To do this it is
necessary to create a @emph{music function}.  This is a specially marked
Scheme function.  For example, the music function @code{\applyMusic} applies
a user-defined function to a music expression.  Its syntax is

@example
\applyMusic #@var{func} @var{music}
@end example

A music function is created with @code{ly:make-music-function},

@example
(ly:make-music-function
@end example

@code{\applyMusic} takes a Scheme function and a Music expression as
arguments.  This is encoded in its parameter list,

@example
(list procedure? ly:music?)
@end example

The function itself takes another argument: an Input location
object.  That object is used to provide error messages with file names
and line numbers.  The definition is the second argument of
@code{ly:make-music-function}.  The body simply calls the function

@example
(lambda (where func music)
 (func music))
@end example

The above Scheme code only defines the functionality.  The tag
@code{\applyMusic} is selected by defining

@example
applyMusic = #(ly:make-music-function
                (list procedure? ly:music?)
                (lambda (parser location func music)
                  (func music)))
@end example

A @code{define-music-function} macro is introduced on top of
@code{ly:make-music-function} to ease the definition of music
functions:

@example
applyMusic = #(define-music-function (parser location func music)
                (procedure? ly:music?)
                (func music))
@end example

Examples of the use of @code{\applyMusic} are in the next section.

@seealso
@file{ly/@/music@/-functions@/-init@/.ly}.


@node Manipulating music expressions
@subsection Manipulating music expressions

Music objects and their properties can be accessed and manipulated
directly through the @code{\applyMusic} mechanism.
The syntax for @code{\applyMusic} is

@example
\applyMusic #@var{func} @var{music}
@end example

@noindent
This means that the Scheme function @var{func} is called with
@var{music} as its argument.  The return value of @var{func} is the
result of the entire expression.  @var{func} may read and write music
properties using the functions @code{ly:music-property} and
@code{ly:music-set-property!}.

An example is a function that reverses the order of elements in
its argument,
@lilypond[quote,verbatim,ragged-right]
#(define (rev-music-1 m)
  (ly:music-set-property! m 'elements 
    (reverse (ly:music-property m 'elements)))
  m)

\applyMusic #rev-music-1 { c'4 d'4 } 
@end lilypond

The use of such a function is very limited.  The effect of this
function is void when applied to an argument that does not have
multiple children.  The following function application has no effect

@example
\applyMusic #rev-music-1 \grace @{ c4 d4 @}
@end example

@noindent
In this case, @code{\grace} is stored as @internalsref{GraceMusic}, which
has no @code{elements}, only a single @code{element}.  Every generally
applicable function for @code{\applyMusic} must -- like music expressions
themselves -- be recursive.

The following example is such a recursive function: It first extracts
the @code{elements} of an expression, reverses them and puts them
back.  Then it recurses, both on @code{elements} and @code{element}
children.

@example
#(define (reverse-music music)
  (let* ((elements (ly:music-property music 'elements))
         (child (ly:music-property music 'element))
         (reversed (reverse elements)))

    ; set children
    (ly:music-set-property! music 'elements reversed)

    ; recurse
    (if (ly:music? child) (reverse-music child))
    (map reverse-music reversed)

    music))
@end example

A slightly more elaborate example is in
@inputfileref{input/@/test,reverse@/-music@/.ly}.

Some of the input syntax is also implemented as recursive music
functions.  For example, the syntax for polyphony
@example
<<a \\ b>>
@end example

@noindent
is actually implemented as a recursive function that replaces the
above by the internal equivalent of
@example
<< \context Voice = "1" @{ \voiceOne a @}
   \context Voice = "2" @{ \voiceTwo b @} >>
@end example

Other applications of @code{\applyMusic} are writing out repeats
automatically (@inputfileref{input/@/test,unfold@/-all@/-repeats@/.ly}),
saving keystrokes (@inputfileref{input/@/test,music@/-box@/.ly}) and
exporting LilyPond input to other formats
@c no @inputfileref{} here 
(eg. @file{input/@/no@/-notation/@/to@/-xml@/.ly}).

@seealso

@file{scm/@/music@/-functions@/.scm}, @file{scm/@/music@/-types@/.scm},
@inputfileref{input/@/test,add@/-staccato@/.ly},
@inputfileref{input/@/test,unfold@/-all@/-repeats@/.ly}, and
@inputfileref{input/@/test,music@/-box@/.ly}.


@node Displaying music expressions
@subsection Displaying music expressions

@cindex internal storage
@cindex @code{\displayMusic}
@cindex @code{\displayLilyMusic}

When writing a music function it is often instructive to inspect how
a music expression is stored internally.  This can be done with the
music function @code{\displayMusic}

@example
@{
  \displayMusic @{ c'4\f @}
@}
@end example

@noindent
will display

@example
(make-music
  'SequentialMusic
  'elements
  (list (make-music
          'EventChord
          'elements
          (list (make-music
                  'NoteEvent
                  'duration
                  (ly:make-duration 2 0 1 1)
                  'pitch
                  (ly:make-pitch 0 0 0))
                (make-music
                  'AbsoluteDynamicEvent
                  'text
                  "f")))))
@end example

Displaying a music expression in LilyPond notation can be
done using the music function @code{\displayLilyMusic}.  For example,

@example
@{
  \displayLilyMusic \transpose c a, @{ c e g a bes @}
@}
@end example

will display

@example
@{ a, cis e fis g @}
@end example

By default, LilyPond will print these messages to the console along
with all the other messages.  To split up these messages and save
the results of @code{\display@{STUFF@}}, redirect the output to
a file.

@example
lilypond file.ly >display.txt
@end example


@node Using LilyPond syntax inside Scheme
@subsection Using LilyPond syntax inside Scheme

Creating music expressions in Scheme can be tedious, as they are
heavily nested and the resulting Scheme code is large.  For some
simple tasks this can be avoided by using common LilyPond syntax inside
Scheme, with the dedicated @code{#@{ ... #@}} syntax.

The following two expressions give equivalent music expressions:
@example
mynotes = @{ \override Stem #'thickness = #4
            @{ c'8 d' @} @}
  
#(define mynotes #@{ \override Stem #'thickness = #4
                    @{ c'8 d' @} #@})
@end example

The content of @code{#@{ ... #@}} is enclosed in an implicit @code{@{
... @}} block, which is parsed.  The resulting music expression, a
@code{SequentialMusic} music object, is then returned and usable in Scheme.

Arbitrary Scheme forms, including variables, can be used in @code{#@{ ... #@}}
expressions with the @code{$} character (@code{$$} can be used to
produce a single @code{$} character).  This makes the creation of simple
functions straightforward.  In the following example, a function
setting the TextScript's padding is defined:

@lilypond[quote,verbatim,ragged-right]
#(use-modules (ice-9 optargs))
#(define* (textpad padding #:optional once?)
  (ly:export   ; this is necessary for using the expression
               ; directly inside a block
    (if once?
        #{ \once \override TextScript #'padding = #$padding #}
        #{ \override TextScript #'padding = #$padding #})))

 {
   c'^"1"
   #(textpad 3.0 #t) % only once
   c'^"2"
   c'^"3"
   #(textpad 5.0)
   c'^"4"
   c'^"5"
 }
@end lilypond

Here, the variable @code{padding} is a number; music expression
variables may also be used in a similar fashion, as in the following
example:

@lilypond[quote,verbatim,ragged-right]
#(define (with-padding padding)
  (lambda (music)
   #{ \override TextScript #'padding = #$padding
      $music
      \revert TextScript #'padding #}))

{
  c'^"1"
  \applyMusic #(with-padding 3) { c'^"2" c'^"3" }
  c'^"4"
}
@end lilypond

The function created by @code{(with-padding 3)} adds @code{\override} and
@code{\revert} statements around the music given as an argument, and returns
this new expression.  Thus, this example is equivalent to:

@example
@{
  c'^"1"
  @{ \override TextScript #'padding = #3
    @{ c'^"2" c'^"3"@}
    \revert TextScript #'padding
  @}
  c'^"4"
@}
@end example

This function may also be defined as a music function:

@lilypond[quote,verbatim,ragged-right]
withPadding =
  #(define-music-function (parser location padding music) (number? ly:music?)
    #{ \override TextScript #'padding = #$padding
       $music 
       \revert TextScript #'padding #})

{
  c'^"1"
  \withPadding #3 { c'^"2" c'^"3"}
  c'^"4"
}
@end lilypond

Music functions can involve Scheme programming, in
addition to simple substitution,

@lilypond[quote,verbatim,ragged-right]
AltOn = #(define-music-function (parser location mag) (number?) 
  #{ \override Stem #'length = #$(* 7.0 mag)
     \override NoteHead #'font-size = 
       #$(inexact->exact (* (/ 6.0 (log 2.0)) (log mag))) #})

AltOff = {
  \revert Stem #'length
  \revert NoteHead #'font-size 
}

{ c'2 \AltOn #0.5 c'4 c'
  \AltOn #1.5 c' c' \AltOff c'2 }
@end lilypond

@noindent
This example may be rewritten

@lilypond[quote,verbatim,ragged-right]
withAlt = #(define-music-function (parser location mag music) (number? ly:music?) 
  #{ \override Stem #'length = #$(* 7.0 mag)
     \override NoteHead #'font-size = 
       #$(inexact->exact (* (/ 6.0 (log 2.0)) (log mag)))
     $music
     \revert Stem #'length
     \revert NoteHead #'font-size #})

{ c'2 \withAlt #0.5 {c'4 c'}
  \withAlt #1.5 {c' c'} c'2 }
@end lilypond


@node Markup programmer interface
@section Markup programmer interface

Markups are implemented as special Scheme functions.  Markups are
implemented as special Scheme functions which produce a
Stencil object given a number of arguments.

@menu
* Markup construction in Scheme::  
* How markups work internally ::  
* New markup command definition::  
@end menu


@node Markup construction in Scheme
@subsection Markup construction in Scheme

@cindex defining markup commands 

The @code{markup} macro builds markup expressions in Scheme while
providing a LilyPond-like syntax.  For example,
@example
(markup #:column (#:line (#:bold #:italic "hello" #:raise 0.4 "world")
                  #:bigger #:line ("foo" "bar" "baz")))
@end example

@noindent
is equivalent to:
@example
\markup \column @{ \line @{ \bold \italic "hello" \raise #0.4 "world" @}
                  \bigger \line @{ foo bar baz @} @}
@end example

@noindent
This example demonstrates the main translation rules between regular
LilyPond markup syntax and Scheme markup syntax.

@quotation
@multitable @columnfractions .3 .3
@item @b{LilyPond} @tab @b{Scheme}
@item @code{\markup markup1} @tab @code{(markup markup1)}
@item @code{\markup @{ markup1 markup2 ... @}} @tab 
        @code{(markup markup1 markup2 ... )}
@item @code{\command} @tab @code{#:command}
@item @code{\variable} @tab @code{variable}
@item @code{\center-align @{ ... @}} @tab @code{#:center-align ( ... )}
@item @code{string} @tab @code{"string"}
@item @code{#scheme-arg} @tab @code{scheme-arg}
@end multitable
@end quotation

The whole scheme language is accessible inside the
@code{markup} macro.  For example, You may use function calls inside
@code{markup} in order to manipulate character strings.  This is
useful when defining new markup commands (see
@ref{New markup command definition}).

@refbugs

One can not feed the @code{#:line}, @code{#:center}, or
@code{#:column}) commands with a variable or the result of a function
call.  Example:

@lisp
(markup #:line (function-that-returns-markups))
@end lisp

@noindent
is invalid.  One should use the @code{make-line-markup},
@code{make-center-markup}, or @code{make-column-markup} functions
instead,

@lisp
(markup (make-line-markup (function-that-returns-markups)))
@end lisp


@node How markups work internally 
@subsection How markups work internally 

In a markup like

@example
\raise #0.5 "text example"
@end example

@noindent
@code{\raise} is actually represented by the @code{raise-markup}
function.  The markup expression is stored as

@example
(list raise-markup 0.5 (list simple-markup "text example"))
@end example

When the markup is converted to printable objects (Stencils), the
@code{raise-markup} function is called as

@example
(apply raise-markup
       @var{\layout object}
       @var{list of property alists}
       0.5
       @var{the "text example" markup})
@end example

The @code{raise-markup} function first creates the stencil for the
@code{text example} string, and then it raises that Stencil by 0.5
staff space.  This is a rather simple example; more complex examples
are in the rest
of this section, and in @file{scm/@/define@/-markup@/-commands@/.scm}.


@node New markup command definition
@subsection New markup command definition

New markup commands can be defined
with the @code{define-markup-command} scheme macro.

@lisp
(define-markup-command (@var{command-name} @var{layout} @var{props} @var{arg1} @var{arg2} ...)
            (@var{arg1-type?} @var{arg2-type?} ...)
  ..command body..)
@end lisp

The arguments are

@table @var
@item argi
@var{i}th command argument
@item argi-type?
a type predicate for the i@var{th} argument
@item layout
the `layout' definition
@item props
a list of alists, containing all active properties. 
@end table

As a simple example, we show how to add a @code{\smallcaps} command,
which selects a small caps font.  Normally we could select the
small caps font,

@example
\markup @{ \override #'(font-shape . caps) Text-in-caps @}
@end example

@noindent
This selects the caps font by setting the @code{font-shape} property to
@code{#'caps} for interpreting @code{Text-in-caps}.

To make the above available as @code{\smallcaps} command, we must
define a function using @code{define-markup-command}.  The command should
take a single argument of type @code{markup}.  Therefore the start of the
definition should read

@example
(define-markup-command (smallcaps layout props argument) (markup?)
@end example

@noindent

What follows is the content of the command: we should interpret
the @code{argument} as a markup, i.e.,

@example
(interpret-markup layout @dots{} argument)
@end example

@noindent
This interpretation should add @code{'(font-shape . caps)} to the active
properties, so we substitute the following for the @dots{} in the
above example:

@example
(cons (list '(font-shape . caps) ) props)
@end example

@noindent
The variable @code{props} is a list of alists, and we prepend to it by
cons'ing a list with the extra setting.


Suppose that we are typesetting a recitative in an opera and
we would like to define a command that will show character names in a
custom manner.  Names should be printed with small caps and moved a
bit to the left and top.  We will define a @code{\character} command
which takes into account the necessary translation and uses the newly
defined @code{\smallcaps} command:

@example
#(define-markup-command (character layout props name) (string?)
  "Print the character name in small caps, translated to the left and
  top.  Syntax: \\character #\"name\""
  (interpret-markup layout props 
   (markup #:hspace 0 #:translate (cons -3 1) #:smallcaps name)))
@end example

There is one complication that needs explanation: texts above and below
the staff are moved vertically to be at a certain distance (the
@code{padding} property) from the staff and the notes.  To make sure
that this mechanism does not annihilate the vertical effect of our
@code{#:translate}, we add an empty string (@code{#:hspace 0}) before the
translated text.  Now the @code{#:hspace 0} will be put above the notes,
and the
@code{name} is moved in relation to that empty string.  The net effect is
that the text is moved to the upper left.

The final result is as follows:

@example
@{
  c''^\markup \character #"Cleopatra"
  e'^\markup \character #"Giulio Cesare"
@}
@end example

@lilypond[quote,ragged-right]
#(define-markup-command (smallcaps layout props str) (string?)
  "Print the string argument in small caps.  Syntax: \\smallcaps #\"string\""
  (interpret-markup layout props
   (make-line-markup
    (map (lambda (s)
          (if (= (string-length s) 0)
              s
              (markup #:large (string-upcase (substring s 0 1))
                      #:translate (cons -0.6 0)
                      #:tiny (string-upcase (substring s 1)))))
         (string-split str #\Space)))))

#(define-markup-command (character layout props name) (string?)
  "Print the character name in small caps, translated to the left and
  top.  Syntax: \\character #\"name\""
  (interpret-markup layout props 
   (markup #:hspace 0 #:translate (cons -3 1) #:smallcaps name)))

{
  c''^\markup \character #"Cleopatra" c'' c'' c''
  e'^\markup \character #"Giulio Cesare" e' e' e'
}
@end lilypond

We have used the @code{caps} font shape, but suppose that our font
does not have a small-caps variant.  In that case we have to fake
the small caps font by setting a string in upcase with the first
letter a little larger:

@example
#(define-markup-command (smallcaps layout props str) (string?)
  "Print the string argument in small caps."
  (interpret-markup layout props
   (make-line-markup
    (map (lambda (s)
          (if (= (string-length s) 0)
              s
              (markup #:large (string-upcase (substring s 0 1))
                      #:translate (cons -0.6 0)
                      #:tiny (string-upcase (substring s 1)))))
         (string-split str #\Space)))))
@end example

The @code{smallcaps} command first splits its string argument into
tokens separated by spaces (@code{(string-split str #\Space)}); for
each token, a markup is built with the first letter made large and
upcased (@code{#:large (string-upcase (substring s 0 1))}), and a
second markup built with the following letters made tiny and upcased
(@code{#:tiny (string-upcase (substring s 1))}).  As LilyPond
introduces a space between markups on a line, the second markup is
translated to the left (@code{#:translate (cons -0.6 0) ...}).  Then,
the markups built for each token are put in a line by
@code{(make-line-markup ...)}.  Finally, the resulting markup is passed
to the @code{interpret-markup} function, with the @code{layout} and
@code{props} arguments.

Note: there is now an internal command @code{\smallCaps} which can
be used to set text in small caps.  See
@ref{Overview of text markup commands} for details.



@node Contexts for programmers
@section Contexts for programmers


@menu
* Context evaluation::          
* Running a function on all layout objects::  
@end menu

@node Context evaluation
@subsection Context evaluation

@cindex calling code during interpreting
@cindex @code{\applyContext}

Contexts can be modified during interpretation with Scheme code.  The
syntax for this is
@example
\applyContext @var{function}
@end example

@var{function} should be a Scheme function taking a single argument,
being the context to apply it to.  The following code will print the
current bar number on the standard output during the compile:

@example
\applyContext
  #(lambda (x)
    (format #t "\nWe were called in barnumber ~a.\n"
     (ly:context-property x 'currentBarNumber)))
@end example



@node Running a function on all layout objects
@subsection Running a function on all layout objects


@cindex calling code on layout objects
@cindex @code{\applyOutput}


The most versatile way of tuning an object is @code{\applyOutput}.  Its
syntax is
@example
\applyOutput @var{proc}
@end example

@noindent
where @var{proc} is a Scheme function, taking three arguments.

When interpreted, the function @var{proc} is called for every layout
object found in the context, with the following arguments:
@itemize @bullet
@item the layout object itself,
@item the context where the layout object was created, and
@item the context where @code{\applyOutput} is processed.
@end itemize


In addition, the cause of the layout object, i.e., the music
expression or object that was responsible for creating it, is in the
object property @code{cause}.  For example, for a note head, this is a
@internalsref{NoteHead} event, and for a @internalsref{Stem} object,
this is a @internalsref{NoteHead} object.

Here is a function to use for @code{\applyOutput}; it blanks
note-heads on the center-line:

@example
(define (blanker grob grob-origin context)
 (if (and (memq (ly:grob-property grob 'interfaces)
                note-head-interface)
          (eq? (ly:grob-property grob 'staff-position) 0))
     (set! (ly:grob-property grob 'transparent) #t)))
@end example