Provide Scheme sandbox.

[lilypond.git] / Documentation / extending / scheme-tutorial.itely

@c -*- coding: utf-8; mode: texinfo; -*-

@ignore
    Translation of GIT committish: FILL-IN-HEAD-COMMITTISH

    When revising a translation, copy the HEAD committish of the
    version that you are working on.  For details, see the Contributors'
    Guide, node Updating translation committishes..
@end ignore

@c \version "2.15.20"

@node Scheme tutorial
@chapter Scheme tutorial

@cindex Scheme
@cindex GUILE
@cindex Scheme, in-line code
@cindex accessing Scheme
@cindex evaluating Scheme
@cindex LISP

LilyPond uses the Scheme programming language, both as part of the
input syntax, and as internal mechanism to glue modules of the program
together.  This section is a very brief overview of entering data in
Scheme.  If you want to know more about Scheme, see
@uref{http://@/www@/.schemers@/.org}.

LilyPond uses the GNU Guile implementation of Scheme, which is
based on the Scheme @qq{R5RS} standard.  If you are learning Scheme
to use with LilyPond, working with a different implementation (or
referring to a different standard) is not recommended.  Information
on guile can be found at @uref{http://www.gnu.org/software/guile/}.
The @qq{R5RS} Scheme standard is located at
@uref{http://www.schemers.org/Documents/Standards/R5RS/}.

@menu
* Introduction to Scheme::
* Scheme in LilyPond::
* Building complicated functions::
@end menu

@node Introduction to Scheme
@section Introduction to Scheme

We begin with an introduction to Scheme.  For this brief introduction,
we will use the GUILE interpreter to explore how the language works.
Once we are familiar with Scheme, we will show how the language can
be integrated in LilyPond files.


@menu
* Scheme sandbox::
* Scheme variables::
* Scheme simple data types::
* Scheme compound data types::
* Calculations in Scheme::
* Scheme procedures::
* Scheme conditionals::
@end menu

@node Scheme sandbox
@subsection Scheme sandbox

The LilyPond installation includes the Guile implementation of
Scheme.  On most systems you can experiment in a Scheme sandbox by
opening a terminal window and typing @q{guile}.  On some systems,
notably Windows, you may need to set the environment variable
@code{GUILE_LOAD_PATH} to the directory @code{../usr/shr/guile/1.8}
in the LilyPond installation.  For the full path to this directory
see @rlearning{Other sources of information}.  Alternatively, Windows
users may simply choose @q{Run} from the Start menu and enter
@q{guile}.

However, a hands-on Scheme sandbox with all of Lilypond loaded is
available with this command line:
@example
lilypond scheme-sandbox
@end example

@noindent
Once the sandbox is running, you will receive a guile prompt:

@lisp
guile>
@end lisp

You can enter Scheme expressions at this prompt to experiment with Scheme.

@node Scheme variables
@subsection Scheme variables

Scheme variables can have any valid scheme value, including a Scheme
procedure.

Scheme variables are created with @code{define}:

@lisp
guile> (define a 2)
guile>
@end lisp

Scheme variables can be evaluated at the guile prompt simply by
typing the variable name:

@lisp
guile> a
2
guile>
@end lisp

Scheme variables can be printed on the display by using the display function:

@lisp
guile> (display a)
2guile>
@end lisp

@noindent
Note that both the value @code{2} and the guile prompt @code{guile}
showed up on the same line.  This can be avoided by calling the
newline procedure or displaying a newline character.

@lisp
guile> (display a)(newline)
2
guile> (display a)(display "\n")
2
guile>
@end lisp

Once a variable has been created, its value can be changed with @code{set!}:

@lisp
guile> (set! a 12345)
guile> a
12345
guile>
@end lisp

@node Scheme simple data types
@subsection Scheme simple data types

The most basic concept in a language is data typing: numbers, character
strings, lists, etc.  Here is a list of simple Scheme data types that are
often used with LilyPond.

@table @asis
@item Booleans
Boolean values are True or False.  The Scheme for True is @code{#t}
and False is @code{#f}.
@funindex ##t
@funindex ##f

@item Numbers
Numbers are entered in the standard fashion,
@code{1} is the (integer) number one, while @w{@code{-1.5}} is a
floating point number (a non-integer number).

@item Strings
Strings are enclosed in double quotes:

@example
"this is a string"
@end example

Strings may span several lines:

@example
"this
is
a string"
@end example

@noindent
and the newline characters at the end of each line will be included
in the string.

Newline characters can also be added by including @code{\n} in the
string.

@example
"this\nis a\nmultiline string"
@end example


Quotation marks and backslashes are added to strings
by preceding them with a backslash.
The string @code{\a said "b"} is entered as

@example
"\\a said \"b\""
@end example

@end table

There are additional Scheme data types that are not discussed here.
For a complete listing see the Guile reference guide,
@uref{http://www.gnu.org/software/guile/manual/html_node/Simple-Data-Types.html}.

@node Scheme compound data types
@subsection Scheme compound data types

There are also compound data types in Scheme.  The  types commonly used in
LilyPond programming include pairs, lists, alists, and hash tables.

@subheading Pairs

The foundational compound data type of Scheme is the @code{pair}.  As
might be expected from its name, a pair is two values glued together.
The operator used to form a pair is called @code{cons}.

@lisp
guile> (cons 4 5)
(4 . 5)
guile>
@end lisp

Note that the pair is displayed as two items surrounded by
parentheses and separated by whitespace, a period (@code{.}), and
more whitespace.  The period is @emph{not} a decimal point, but
rather an indicator of the pair.

Pairs can also be entered as literal values by preceding them with
a single quote character.

@lisp
guile> '(4 . 5)
(4 . 5)
guile>
@end lisp

The two elements of a pair may be any valid Scheme value:

@lisp
guile> (cons #t #f)
(#t . #f)
guile> '("blah-blah" . 3.1415926535)
("blah-blah" . 3.1415926535)
guile>
@end lisp

The first and second elements of the pair can be accessed by the
Scheme procedures @code{car} and @code{cdr}, respectively.

@lisp
guile> (define mypair (cons 123 "hello there")
... )
guile> (car mypair)
123
guile> (cdr mypair)
"hello there"
guile>
@end lisp

@noindent

Note:  @code{cdr} is pronounced "could-er", according Sussman and
Abelson, see
@uref{http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-14.html#footnote_Temp_133}

@subheading Lists

A very common Scheme data structure is the @emph{list}.  Formally, a
list is defined as either the empty list (represented as @code{'()},
or a pair whose @code{cdr} is a list.

There are many ways of creating lists.  Perhaps the most common is
with the @code{list} procedure:

@lisp
guile> (list 1 2 3 "abc" 17.5)
(1 2 3 "abc" 17.5)
@end lisp

As can be seen, a list is displayed in the form of individual elements
separated by whitespace and enclosed in parentheses.  Unlike a pair,
there is no period between the elements.

A list can also be entered as a literal list by enclosing its
elements in parentheses, and adding a quote:

@lisp
guile> '(17 23 "foo" "bar" "bazzle")
(17 23 "foo" "bar" "bazzle")
@end lisp

Lists are a central part of Scheme.  In, fact, Scheme is considered
a dialect of lisp, where @q{lisp} is an abbreviation for
@q{List Processing}.  Scheme expressions are all lists.

@subheading Association lists (alists)

A special type of list is an @emph{association list} or @emph{alist}.
An alist is used to store data for easy retrieval.

Alists are lists whose elements are pairs.  The @code{car} of each
element is called the @emph{key}, and the @code{cdr} of each element
is called the @emph{value}.  The Scheme procedure @code{assoc} is
used to retrieve an entry from the alist, and @code{cdr} is used to
retrieve the value:

@lisp
guile> (define my-alist '((1  . "A") (2 . "B") (3 . "C")))
guile> my-alist
((1 . "A") (2 . "B") (3 . "C"))
guile> (assoc 2 my-alist)
(2 . "B")
guile> (cdr (assoc 2 my-alist))
"B"
guile>
@end lisp

Alists are widely used in LilyPond to store properties and other data.

@subheading Hash tables

A data structure that is used occasionally in LilyPond.  A hash table
is similar to an array, but the indexes to the array can be any type
of Scheme value, not just integers.

Hash tables are more efficient than alists if there is a lot of data
to store and the data changes very infrequently.

The syntax to create hash tables is a bit complex, but you
can see examples of it in the LilyPond source.

@lisp
guile> (define h (make-hash-table 10))
guile> h
#<hash-table 0/31>
guile> (hashq-set! h 'key1 "val1")
"val1"
guile> (hashq-set! h 'key2 "val2")
"val2"
guile> (hashq-set! h 3 "val3")
"val3"
@end lisp

Values are retrieved from hash tables with @code{hashq-ref}.

@lisp
guile> (hashq-ref h 3)
"val3"
guile> (hashq-ref h 'key2)
"val2"
guile>
@end lisp

Keys and values are retrieved as a pair with @code{hashq-get-handle}.
This is a preferred way, because it will return @code{#f} if a key is
not found.

@lisp
guile> (hashq-get-handle h 'key1)
(key1 . "val1")
guile> (hashq-get-handle h 'frob)
#f
guile>
@end lisp

@node Calculations in Scheme
@subsection Calculations in Scheme

@ignore
We have been using lists all along.  A calculation, like @code{(+ 1 2)}
is also a list (containing the symbol @code{+} and the numbers 1
and@tie{}2).  Normally lists are interpreted as calculations, and the
Scheme interpreter substitutes the outcome of the calculation.  To enter a
list, we stop the evaluation.  This is done by quoting the list with a
quote @code{'} symbol.  So, for calculations do not use a quote.

Inside a quoted list or pair, there is no need to quote anymore.  The
following is a pair of symbols, a list of symbols and a list of lists
respectively,

@example
#'(stem . head)
#'(staff clef key-signature)
#'((1) (2))
@end example
@end ignore

Scheme can be used to do calculations.  It uses @emph{prefix}
syntax.  Adding 1 and@tie{}2 is written as @code{(+ 1 2)} rather than the
traditional @math{1+2}.

@lisp
guile> (+ 1 2)
3
@end lisp

Calculations may be nested; the result of a function may
be used for another calculation.

@lisp
guile> (+ 1 (* 3 4))
13
@end lisp

These calculations are examples of evaluations; an expression like
@code{(* 3 4)} is replaced by its value @code{12}.

Scheme calculations are sensitive to the differences between integers
and non-integers.  Integer calculations are exact, while non-integers
are calculated to the appropriate limits of precision:

@lisp
guile> (/ 7 3)
7/3
guile> (/ 7.0 3.0)
2.33333333333333
@end lisp

When the scheme interpreter encounters an expression that is a list,
the first element of the list is treated as a procedure to be
evaluated with the arguments of the remainder of the list.  Therefore,
all operators in Scheme are prefix operators.

If the first element of a Scheme expression that is a list passed to
the interpreter is @emph{not} an operator or procedure, an error will
occur:

@lisp
guile> (1 2 3)

Backtrace:
In current input:
  52: 0* [1 2 3]

<unnamed port>:52:1: In expression (1 2 3):
<unnamed port>:52:1: Wrong type to apply: 1
ABORT: (misc-error)
guile>
@end lisp

Here you can see that the interpreter was trying to treat 1 as an
operator or procedure, and it couldn't.  Hence the error is "Wrong
type to apply: 1".

Therefore, to create a list we need to use the list operator, or to
quote the list so that the interpreter will not try to evaluate it.

@lisp
guile> (list 1 2 3)
(1 2 3)
guile> '(1 2 3)
(1 2 3)
guile>
@end lisp

This is an error that can appear as you are working with Scheme in LilyPond.

@ignore
The same assignment can be done in completely in Scheme as well,

@example
#(define twentyFour (* 2 twelve))
@end example

@c this next section is confusing -- need to rewrite

The @emph{name} of a variable is also an expression, similar to a
number or a string.  It is entered as

@example
#'twentyFour
@end example

@funindex #'symbol
@cindex quoting in Scheme

The quote mark @code{'} prevents the Scheme interpreter from substituting
@code{24} for the @code{twentyFour}.  Instead, we get the name
@code{twentyFour}.
@end ignore


@node Scheme procedures
@subsection Scheme procedures

Scheme procedures are executable scheme expressions that return a
value resulting from their execution.  They can also manipulate
variables defined outside of the procedure.

@subheading Defining procedures

Procedures are defined in Scheme with define

@example
(define (function-name arg1 arg2 ... argn)
 scheme-expression-that-gives-a-return-value)
@end example

For example, we could define a procedure to calculate the average:

@lisp
guile> (define (average x y) (/ (+ x y) 2))
guile> average
#<procedure average (x y)>
@end lisp

Once a procedure is defined, it is called by putting the procedure
name and the arguments in a list.  For example, we can calculate
the average of 3 and 12:

@lisp
guile> (average 3 12)
15/2
@end lisp

@subheading Predicates

Scheme procedures that return boolean values are often called
@emph{predicates}.  By convention (but not necessity), predicate names
typically end in a question mark:

@lisp
guile> (define (less-than-ten? x) (< x 10))
guile> (less-than-ten? 9)
#t
guile> (less-than-ten? 15)
#f
@end lisp

@subheading Return values

Scheme procedures always return a return value, which is the value
of the last expression executed in the procedure.  The return
value can be any valid Scheme value, including a complex data
structure or a procedure.

Sometimes the user would like to have multiple Scheme expressions in
a procedure.  There are two ways that multiple expressions can be
combined.  The first is the @code{begin} procedure, which allows
multiple expressions to be evaluated, and returns the value of
the last expression.

@lisp
guile> (begin (+ 1 2) (- 5 8) (* 2 2))
4
@end lisp

The second way to combine multiple expressions is in a @code{let} block.
In a let block, a series of bindings are created, and then a sequence
of expressions that can include those bindings is evaluated.  The
return value of the let block is the return value of the last
statement in the let block:

@lisp
guile> (let ((x 2) (y 3) (z 4)) (display (+ x y)) (display (- z 4))
... (+ (* x y) (/ z x)))
508
@end lisp

@node Scheme conditionals
@subsection Scheme conditionals

@subheading if

Scheme has an @code{if} procedure:

@example
(if test-expression true-expression false-expression)
@end example

@var{test-expression} is an expression that returns a boolean
value.  If @var{test-expression} returns @code{#t}, the if
procedure returns the value of @var{true-expression}, otherwise
it returns the value of @var{false-expression}.

@lisp
guile> (define a 3)
guile> (define b 5)
guile> (if (> a b) "a is greater than b" "a is not greater than b")
"a is not greater than b"
@end lisp

@subheading cond

Another conditional procedure in scheme is @code{cond}:

@example
(cond (test-expression-1 result-expression-sequence-1)
      (test-expression-2 result-expression-sequence-2)
      ...
      (test-expression-n result-expression-sequence-n))
@end example

For example:

@lisp
guile> (define a 6)
guile> (define b 8)
guile> (cond ((< a b) "a is less than b")
...          ((= a b) "a equals b")
...          ((> a b) "a is greater than b"))
"a is less than b"
@end lisp

@node Scheme in LilyPond
@section Scheme in LilyPond


@menu
* LilyPond Scheme syntax::
* LilyPond variables::
* Input variables and Scheme::
* Object properties::
* LilyPond compound variables::
* Internal music representation::
@end menu

@node LilyPond Scheme syntax
@subsection LilyPond Scheme syntax
@funindex $
@funindex #

The Guile interpreter is part of LilyPond, which means that
Scheme can be included in LilyPond input files.  There are several
methods for including Scheme in LilyPond.

The simplest way is to use a hash mark@tie{}@code{#} before a Scheme
expression.

Now LilyPond's input is structured into tokens and expressions, much
like human language is structured into words and sentences.  LilyPond
has a lexer that recognizes tokens (literal numbers, strings, Scheme
elements, pitches and so on), and a parser that understands the syntax,
@ruser{LilyPond grammar}.  Once it knows that a particular syntax rule
applies, it executes actions associated with it.

The hash mark@tie{}@code{#} method of embedding Scheme is a natural fit
for this system.  Once the lexer sees a hash mark, it calls the Scheme
reader to read one full Scheme expression (this can be an identifier, an
expression enclosed in parentheses, or several other things).  After the
Scheme expression is read, it is stored away as the value for an
@code{SCM_TOKEN} in the grammar.  Once the parser knows how to make use
of this token, it calls Guile for evaluating the Scheme expression.
Since the parser usually requires a bit of lookahead from the lexer to
make its parsing decisions, this separation of reading and evaluation
between lexer and parser is exactly what is needed to keep the execution
of LilyPond and Scheme expressions in sync.  For this reason, you should
use the hash mark@tie{}@code{#} for calling Scheme whenever this is
feasible.

Another way to call the Scheme interpreter from LilyPond is the use of
dollar@tie{}@code{$} instead of a hash mark for introducing Scheme
expressions.  In this case, Lilypond evaluates the code right after the
lexer has read it.  It checks the resulting type of the Scheme
expression and then picks a token type (one of several
@code{xxx_IDENTIFIER} in the syntax) for it.  It creates a @emph{copy}
of the value and uses that for the value of the token.  If the value of
the expression is void (Guile's value of @code{*unspecified*}), nothing
at all is passed to the parser.

This is, in fact, exactly the same mechanism that Lilypond employs when
you call any variable or music function by name, as @code{\name}, with
the only difference that its end is determined by the Lilypond lexer
without consulting the Scheme reader, and thus only variable names
consistent with the current Lilypond mode are accepted.

The immediate action of @code{$} can lead to surprises, @ref{Input
variables and Scheme}.  Using @code{#} where the parser supports it is
usually preferable.

Now let's take a look at some actual Scheme code.  Scheme procedures can
be defined in LilyPond input files:

@example
#(define (average a b c) (/ (+ a b c) 3))
@end example

Note that LilyPond comments (@code{%} and @code{%@{ %@}}) cannot
be used within Scheme code, even in a LilyPond input file, because
the Guile interpreter, not the LilyPond lexer, is reading
the Scheme expression.  Comments in Guile Scheme are entered
as follows:

@example
; this is a single-line comment

#!
  This a (non-nestable) Guile-style block comment
  But these are rarely used by Schemers and never in
  LilyPond source code
!#
@end example

For the rest of this section, we will assume that the data is entered
in a music file, so we add@tie{}@code{#}s at the beginning of each Scheme
expression.

All of the top-level Scheme expressions in a LilyPond input file can
be combined into a single Scheme expression by the use of the
@code{begin} statement:

@example
#(begin
  (define foo 0)
  (define bar 1))
@end example


@node LilyPond variables
@subsection LilyPond variables

LilyPond variables are stored internally in the form of Scheme
variables.  Thus,

@example
twelve = 12
@end example

@noindent
is equivalent to

@example
#(define twelve 12)
@end example

This means that LilyPond variables are available
for use in Scheme expressions.  For example, we could use

@example
twentyFour = #(* 2 twelve)
@end example

@noindent
which would result in the number 24 being stored in the
LilyPond (and Scheme) variable @code{twentyFour}.

The usual way to refer to Lilypond variables, @ref{LilyPond Scheme
syntax}, is to call them using a backslash, i.e., @code{\twentyFour}.
Since this creates a copy of the value for most of LilyPond's internal
types, in particular music expressions, music functions don't usually
create copies of material they change.  For this reason, music
expressions given with @code{#} should usually not contain material that
is not either created from scratch or explicitly copied rather than
directly referenced.

@node Input variables and Scheme
@subsection Input variables and Scheme

The input format supports the notion of variables: in the following
example, a music expression is assigned to a variable with the name
@code{traLaLa}.

@example
traLaLa = @{ c'4 d'4 @}
@end example

@noindent

There is also a form of scoping: in the following example, the
@code{\layout} block also contains a @code{traLaLa} variable, which is
independent of the outer @code{\traLaLa}.

@example
traLaLa = @{ c'4 d'4 @}
\layout @{ traLaLa = 1.0 @}
@end example

@c
In effect, each input file is a scope, and all @code{\header},
@code{\midi}, and @code{\layout} blocks are scopes nested inside that
toplevel scope.

Both variables and scoping are implemented in the GUILE module system.
An anonymous Scheme module is attached to each scope.  An assignment of
the form:

@example
traLaLa = @{ c'4 d'4 @}
@end example

@noindent
is internally converted to a Scheme definition:

@example
(define traLaLa @var{Scheme value of `@code{... }'})
@end example

This means that LilyPond variables and Scheme variables may be freely
mixed.  In the following example, a music fragment is stored in the
variable @code{traLaLa}, and duplicated using Scheme.  The result is
imported in a @code{\score} block by means of a second variable
@code{twice}:

@lilypond[verbatim]
traLaLa = { c'4 d'4 }

#(define newLa (map ly:music-deep-copy
  (list traLaLa traLaLa)))
#(define twice
  (make-sequential-music newLa))

{ \twice }
@end lilypond

@c Due to parser lookahead

This is actually a rather interesting example.  The assignment will only
take place after the parser has ascertained that nothing akin to
@code{\addlyrics} follows, so it needs to check what comes next.  It
reads @code{#} and the following Scheme expression @emph{without}
evaluating it, so it can go ahead with the assignment, and
@emph{afterwards} execute the Scheme code without problem.

The above example shows how to @q{export} music expressions from the
input to the Scheme interpreter.  The opposite is also possible.  By
placing it after @code{$}, a Scheme
value is interpreted as if it were entered in LilyPond syntax.
Instead of defining @code{\twice}, the example above could also have
been written as

@example
...
@{ $(make-sequential-music (list newLa)) @}
@end example

You can use @code{$} with a Scheme expression anywhere you could use
@code{\@var{name}} after having assigned the Scheme expression to a
variable @var{name}.  This replacement happens in the @q{lexer}, so
Lilypond is not even aware of the difference.

One drawback, however, is that of timing.  If we had been using @code{$}
instead of @code{#} for defining @code{newLa} in the above example, the
following Scheme definition would have failed because @code{traLaLa}
would not yet have been defined.  For an explanation of this timing
problem, @ref{LilyPond Scheme syntax}.

In any case, evaluation of Scheme code happens in the parser at latest.
If you need it to be executed at a later point of time, @ref{Void scheme
functions}, or store it in a macro:

@example
#(define (nopc)
  (ly:set-option 'point-and-click #f))

...
#(nopc)
@{ c'4 @}
@end example

@knownissues

Mixing Scheme and LilyPond variables is not possible with the
@option{--safe} option.


@node Object properties
@subsection Object properties

Object properties are stored in LilyPond in the form of alist-chains,
which are lists of alists.  Properties are set by adding values at
the beginning of the property list.  Properties are read by retrieving
values from the alists.

Setting a new value for a property requires assigning a value to
the alist with both a key and a value.  The LilyPond syntax for doing
this is:

@example
\override Stem #'thickness = #2.6
@end example

This instruction adjusts the appearance of stems.  An alist entry
@code{'(thickness . 2.6)} is added to the property list of the
@code{Stem}
object.  @code{thickness} is measured relative to the thickness of
staff lines, so these stem lines will be @code{2.6} times the
width of staff lines.  This makes stems almost twice as thick as their
normal size.  To distinguish between variables defined in input files (like
@code{twentyFour} in the example above) and variables of internal
objects, we will call the latter @q{properties} and the former
@q{variables.}  So, the stem object has a @code{thickness} property,
while @code{twentyFour} is a variable.

@cindex properties vs. variables
@cindex variables vs. properties

@c  todo -- here we're getting interesting.  We're now introducing
@c  LilyPond variable types.  I think this deserves a section all
@c  its own

@node LilyPond compound variables
@subsection LilyPond compound variables

@subheading Offsets

Two-dimensional offsets (X and Y coordinates) are stored as @code{pairs}.
The @code{car} of the offset is the X coordinate, and the @code{cdr} is
the Y coordinate.

@example
\override TextScript #'extra-offset = #'(1 . 2)
@end example

This assigns the pair @code{(1 . 2)} to the @code{extra-offset}
property of the
TextScript object.  These numbers are measured in staff-spaces, so
this command moves the object 1 staff space to the right, and 2 spaces up.

Procedures for working with offsets are found in @file{scm/lily-library.scm}.

@subheading Extents

Pairs are also used to store intervals, which represent a range of numbers
from the minimum (the @code{car}) to the maximum (the @code{cdr}).
Intervals are used to store the X- and Y- extents of printable objects.
For X extents, the @code{car} is the left hand X coordinate, and the
@code{cdr} is the right hand X coordinate.  For Y extents, the @code{car}
is the bottom coordinate, and the @code{cdr} is the top coordinate.

Procedures for working with intervals are found in
@file{scm/lily-library.scm}.  These procedures should be used when possible
to ensure consistency of code.

@subheading Property alists

A property alist is a LilyPond data structure that is an alist whose
keys are properties and whose values are Scheme expressions that give
the desired value for the property.

LilyPond properties are Scheme symbols, such as @code{'thickness}.

@subheading Alist chains

An alist chain is a list containing property alists.

The set of all properties that will apply to a grob is typically
stored as an alist chain.  In order to find the value for a particular
property that a grob should have, each alist in the chain is searched in
order, looking for an entry containing the property key.  The first alist
entry found is returned, and the value is the property value.

The Scheme procedure @code{chain-assoc-get} is normally used to get
grob property values.

@node Internal music representation
@subsection Internal music representation

Internally, music is represented as a Scheme list.  The list contains
various elements that affect the printed output.  Parsing is the process
of converting music from the LilyPond input representation to the
internal Scheme representation.

When a music expression is parsed, it is converted into a set of
Scheme music objects.  The defining property of a music object is that
it takes up time.  The time it takes up is called its @emph{duration}.
Durations are expressed as a rational number that measures the length
of the music object in whole notes.

A music object has three kinds of types:
@itemize
@item
music name: Each music expression has a name.  For example, a note
leads to a @rinternals{NoteEvent}, and @code{\simultaneous} leads to
a @rinternals{SimultaneousMusic}.  A list of all expressions
available is in the Internals Reference manual, under
@rinternals{Music expressions}.

@item
@q{type} or interface: Each music name has several @q{types} or
interfaces, for example, a note is an @code{event}, but it is also a
@code{note-event}, a @code{rhythmic-event}, and a
@code{melodic-event}.  All classes of music are listed in the
Internals Reference, under
@rinternals{Music classes}.

@item
C++ object: Each music object is represented by an object of the C++
class @code{Music}.
@end itemize

The actual information of a music expression is stored in properties.
For example, a @rinternals{NoteEvent} has @code{pitch} and
@code{duration} properties that store the pitch and duration of that
note.  A list of all properties available can be found in the
Internals Reference, under @rinternals{Music properties}.

A compound music expression is a music object that contains other
music objects in its properties.  A list of objects can be stored in
the @code{elements} property of a music object, or a single @q{child}
music object in the @code{element} property.  For example,
@rinternals{SequentialMusic} has its children in @code{elements},
and @rinternals{GraceMusic} has its single argument in
@code{element}.  The body of a repeat is stored in the @code{element}
property of @rinternals{RepeatedMusic}, and the alternatives in
@code{elements}.

@node Building complicated functions
@section Building complicated functions

This section explains how to gather the information necessary
to create complicated music functions.

@menu
* Displaying music expressions::
* Music properties::
* Doubling a note with slurs (example)::
* Adding articulation to notes (example)::
@end menu

@node Displaying music expressions
@subsection Displaying music expressions

@cindex internal storage
@cindex displaying music expressions
@cindex internal representation, displaying
@cindex displayMusic
@funindex \displayMusic

When writing a music function it is often instructive to inspect how
a music expression is stored internally.  This can be done with the
music function @code{\displayMusic}

@example
@{
  \displayMusic @{ c'4\f @}
@}
@end example

@noindent
will display

@example
(make-music
  'SequentialMusic
  'elements
  (list (make-music
          'EventChord
          'elements
          (list (make-music
                  'NoteEvent
                  'duration
                  (ly:make-duration 2 0 1 1)
                  'pitch
                  (ly:make-pitch 0 0 0))
                (make-music
                  'AbsoluteDynamicEvent
                  'text
                  "f")))))
@end example

By default, LilyPond will print these messages to the console along
with all the other messages.  To split up these messages and save
the results of @code{\display@{STUFF@}}, you can specify an optional
output port to use:

@example
@{
  \displayMusic #(open-output-file "display.txt") @{ c'4\f @}
@}
@end example

This will overwrite a previous output file whenever it is called; if you
need to write more than one expression, you would use a variable for
your port and reuse it:
@example
@{
  port = #(open-output-file "display.txt")
  \displayMusic \port @{ c'4\f @}
  \displayMusic \port @{ d'4 @}
  #(close-output-port port)
@}
@end example

Guile's manual describes ports in detail.  Closing the port is actually
only necessary if you need to read the file before Lilypond finishes; in
the first example, we did not bother to do so.

A bit of reformatting makes the above information easier to read:

@example
(make-music 'SequentialMusic
  'elements (list (make-music 'EventChord
                    'elements (list (make-music 'NoteEvent
                                      'duration (ly:make-duration 2 0 1 1)
                                      'pitch (ly:make-pitch 0 0 0))
                                    (make-music 'AbsoluteDynamicEvent
                                      'text "f")))))
@end example

A @code{@{ ... @}} music sequence has the name @code{SequentialMusic},
and its inner expressions are stored as a list in its @code{'elements}
property.  A note is represented as an @code{EventChord} expression,
containing a @code{NoteEvent} object (storing the duration and
pitch properties) and any extra information (in this case, an
@code{AbsoluteDynamicEvent} with a @code{"f"} text property.

@funindex{\void}
@code{\displayMusic} returns the music it displays, so it will get
interpreted as well as displayed.  To avoid interpretation, write
@code{\void} before @code{\displayMusic}.

@node Music properties
@subsection Music properties

TODO -- make sure we delineate between @emph{music} properties,
@emph{context} properties, and @emph{layout} properties.  These
are potentially confusing.

The @code{NoteEvent} object is the first object of the
@code{'elements} property of @code{someNote}.

@example
someNote = c'
\displayMusic \someNote
===>
(make-music
  'EventChord
  'elements
  (list (make-music
          'NoteEvent
          'duration
          (ly:make-duration 2 0 1 1)
          'pitch
          (ly:make-pitch 0 0 0))))
@end example

The @code{display-scheme-music} function is the function used by
@code{\displayMusic} to display the Scheme representation of a music
expression.

@example
#(display-scheme-music (first (ly:music-property someNote 'elements)))
===>
(make-music
  'NoteEvent
  'duration
  (ly:make-duration 2 0 1 1)
  'pitch
  (ly:make-pitch 0 0 0))
@end example

Then the note pitch is accessed through the @code{'pitch} property
of the @code{NoteEvent} object,

@example
#(display-scheme-music
   (ly:music-property (first (ly:music-property someNote 'elements))
                      'pitch))
===>
(ly:make-pitch 0 0 0)
@end example

The note pitch can be changed by setting this @code{'pitch} property,

@funindex \displayLilyMusic

@example
#(set! (ly:music-property (first (ly:music-property someNote 'elements))
                          'pitch)
       (ly:make-pitch 0 1 0)) ;; set the pitch to d'.
\displayLilyMusic \someNote
===>
d'
@end example


@node Doubling a note with slurs (example)
@subsection Doubling a note with slurs (example)

Suppose we want to create a function that translates input like
@code{a} into @code{a( a)}.  We begin by examining the internal
representation of the desired result.

@example
\displayMusic@{ a'( a') @}
===>
(make-music
  'SequentialMusic
  'elements
  (list (make-music
          'EventChord
          'elements
          (list (make-music
                  'NoteEvent
                  'duration
                  (ly:make-duration 2 0 1 1)
                  'pitch
                  (ly:make-pitch 0 5 0))
                (make-music
                  'SlurEvent
                  'span-direction
                  -1)))
        (make-music
          'EventChord
          'elements
          (list (make-music
                  'NoteEvent
                  'duration
                  (ly:make-duration 2 0 1 1)
                  'pitch
                  (ly:make-pitch 0 5 0))
                (make-music
                  'SlurEvent
                  'span-direction
                  1)))))
@end example

The bad news is that the @code{SlurEvent} expressions
must be added @q{inside} the note (or more precisely,
inside the @code{EventChord} expression).

Now we examine the input,

@example
(make-music
  'SequentialMusic
  'elements
  (list (make-music
          'EventChord
          'elements
          (list (make-music
                  'NoteEvent
                  'duration
                  (ly:make-duration 2 0 1 1)
                  'pitch
                  (ly:make-pitch 0 5 0))))))
@end example

So in our function, we need to clone this expression (so that we
have two notes to build the sequence), add a @code{SlurEvent} to the
@code{'elements} property of each one, and finally make a
@code{SequentialMusic} with the two @code{EventChords}.

@example
doubleSlur = #(define-music-function (parser location note) (ly:music?)
         "Return: @{ note ( note ) @}.
         `note' is supposed to be an EventChord."
         (let ((note2 (ly:music-deep-copy note)))
           (set! (ly:music-property note 'elements)
                 (cons (make-music 'SlurEvent 'span-direction -1)
                       (ly:music-property note 'elements)))
           (set! (ly:music-property note2 'elements)
                 (cons (make-music 'SlurEvent 'span-direction 1)
                       (ly:music-property note2 'elements)))
           (make-music 'SequentialMusic 'elements (list note note2))))
@end example


@node Adding articulation to notes (example)
@subsection Adding articulation to notes (example)

The easy way to add articulation to notes is to merge two music
expressions into one context, as explained in
@ruser{Creating contexts}.  However, suppose that we want to write
a music function that does this.

A @code{$variable} inside the @code{#@{...#@}} notation is like
a regular @code{\variable} in classical LilyPond notation.  We
know that

@example
@{ \music -. -> @}
@end example

@noindent
will not work in LilyPond.  We could avoid this problem by attaching
the articulation to a fake note,

@example
@{ << \music s1*0-.-> @}
@end example

@noindent
but for the sake of this example, we will learn how to do this in
Scheme.  We begin by examining our input and desired output,

@example
%  input
\displayMusic c4
===>
(make-music
  'EventChord
  'elements
  (list (make-music
          'NoteEvent
          'duration
          (ly:make-duration 2 0 1 1)
          'pitch
          (ly:make-pitch -1 0 0))))
=====
%  desired output
\displayMusic c4->
===>
(make-music
  'EventChord
  'elements
  (list (make-music
          'NoteEvent
          'duration
          (ly:make-duration 2 0 1 1)
          'pitch
          (ly:make-pitch -1 0 0))
        (make-music
          'ArticulationEvent
          'articulation-type
          "marcato")))
@end example

We see that a note (@code{c4}) is represented as an @code{EventChord}
expression, with a @code{NoteEvent} expression in its elements list.  To
add a marcato articulation, an @code{ArticulationEvent} expression must
be added to the elements property of the @code{EventChord}
expression.

To build this function, we begin with

@example
(define (add-marcato event-chord)
  "Add a marcato ArticulationEvent to the elements of `event-chord',
  which is supposed to be an EventChord expression."
  (let ((result-event-chord (ly:music-deep-copy event-chord)))
    (set! (ly:music-property result-event-chord 'elements)
          (cons (make-music 'ArticulationEvent
                  'articulation-type "marcato")
                (ly:music-property result-event-chord 'elements)))
    result-event-chord))
@end example

The first line is the way to define a function in Scheme: the function
name is @code{add-marcato}, and has one variable called
@code{event-chord}.  In Scheme, the type of variable is often clear
from its name.  (this is good practice in other programming languages,
too!)

@example
"Add a marcato..."
@end example

@noindent
is a description of what the function does.  This is not strictly
necessary, but just like clear variable names, it is good practice.

@example
(let ((result-event-chord (ly:music-deep-copy event-chord)))
@end example

@code{let} is used to declare local variables.  Here we use one local
variable, named @code{result-event-chord}, to which we give the value
@code{(ly:music-deep-copy event-chord)}.  @code{ly:music-deep-copy} is
a function specific to LilyPond, like all functions prefixed by
@code{ly:}.  It is use to make a copy of a music
expression.  Here we copy @code{event-chord} (the parameter of the
function).  Recall that our purpose is to add a marcato to an
@code{EventChord} expression.  It is better to not modify the
@code{EventChord} which was given as an argument, because it may be
used elsewhere.

Now we have a @code{result-event-chord}, which is a
@code{NoteEventChord} expression and is a copy of
@code{event-chord}.  We add the marcato to its @code{'elements}
list property.

@example
(set! place new-value)
@end example

Here, what we want to set (the @q{place}) is the @code{'elements}
property of @code{result-event-chord} expression.

@example
(ly:music-property result-event-chord 'elements)
@end example

@code{ly:music-property} is the function used to access music properties
(the @code{'elements}, @code{'duration}, @code{'pitch}, etc, that we
see in the @code{\displayMusic} output above).  The new value is the
former @code{'elements} property, with an extra item: the
@code{ArticulationEvent} expression, which we copy from the
@code{\displayMusic} output,

@example
(cons (make-music 'ArticulationEvent
        'articulation-type "marcato")
      (ly:music-property result-event-chord 'elements))
@end example

@code{cons} is used to add an element to a list without modifying
the original list.  This is what we want: the same list as before,
plus the new @code{ArticulationEvent} expression.  The order
inside the @code{'elements} property is not important here.

Finally, once we have added the marcato articulation to its @code{elements}
property, we can return @code{result-event-chord}, hence the last line of
the function.

Now we transform the @code{add-marcato} function into a music
function,

@example
addMarcato = #(define-music-function (parser location event-chord)
                                     (ly:music?)
    "Add a marcato ArticulationEvent to the elements of `event-chord',
    which is supposed to be an EventChord expression."
    (let ((result-event-chord (ly:music-deep-copy event-chord)))
      (set! (ly:music-property result-event-chord 'elements)
            (cons (make-music 'ArticulationEvent
                    'articulation-type "marcato")
                  (ly:music-property result-event-chord 'elements)))
      result-event-chord))
@end example

We may verify that this music function works correctly,

@example
\displayMusic \addMarcato c4
@end example





@ignore
@menu
* Tweaking with Scheme::
@end menu

@c @nod e Tweaking with Scheme
@c @sectio n Tweaking with Scheme

We have seen how LilyPond output can be heavily modified using
commands like
@code{\override TextScript #'extra-offset = ( 1 . -1)}.  But
we have even more power if we use Scheme.  For a full explanation
of this, see the @ref{Scheme tutorial}, and
@ref{Interfaces for programmers}.

We can use Scheme to simply @code{\override} commands,

TODO Find a simple example
@c This isn't a valid example with skylining
@c It works fine without padText  -td
@end ignore

@ignore
@lilypond[quote,verbatim,ragged-right]
padText = #(define-music-function (parser location padding) (number?)
#{
  \once \override TextScript #'padding = #padding
#})

\relative c''' {
  c4^"piu mosso" b a b
  \padText #1.8
  c4^"piu mosso" d e f
  \padText #2.6
  c4^"piu mosso" fis a g
}
@end lilypond
@end ignore

@ignore
We can use it to create new commands:

@c Check this is a valid example with skylining
@c It is - 'padding still works


@lilypond[quote,verbatim,ragged-right]
tempoPadded = #(define-music-function (parser location padding tempotext)
  (number? string?)
#{
  \once \override Score.MetronomeMark #'padding = $padding
  \tempo \markup { \bold #tempotext }
#})

\relative c'' {
  \tempo \markup { "Low tempo" }
  c4 d e f g1
  \tempoPadded #4.0 #"High tempo"
  g4 f e d c1
}
@end lilypond


Even music expressions can be passed in:

@lilypond[quote,verbatim,ragged-right]
pattern = #(define-music-function (parser location x y) (ly:music? ly:music?)
#{
  $x e8 a b $y b a e
#})

\relative c''{
  \pattern c8 c8\f
  \pattern {d16 dis} { ais16-> b\p }
}
@end lilypond
@end ignore