* lily/lyric-phrasing-engraver.cc: remove

[lilypond.git] / Documentation / user / introduction.itely

@c -*-texinfo-*-


@node Introduction
@chapter Introduction

There are a lot of programs that let you print sheet music with a
computer, but most of them do not do a good job.  Most computer
printouts have a bland, mechanical look, and are unpleasant to play
from.  If you agree with us on that, then you will like LilyPond: we
have tried to capture the original look of hand-engraved music.  We
have tuned our algorithms, font-designs, and program settings to make
the program produce prints that match the quality of the old editions
we love to see and love to play from.


@menu
* Notation in LilyPond ::       
* Engraving in LilyPond::       
* Typography and program architecture::  
* Music representation::        
* Example applications::        
* About this manual::           
@end menu


@node Notation in LilyPond
@section Notation in LilyPond


@cindex engraving
@cindex typography


Printing sheet music consists of two non-trivial tasks. First, one has
to master music notation: the science of knowing which symbols to use
for what. Second, one has to master music engraving: the art of
placing symbols such that the result looks pleasing.

Common music notation is a system of recording music that has evolved
over the past 1000 years. The form that is now in common use, dates
from the early renaissance. Although, the basic form (i.e. note heads on a
5-line staff) has not changed, the details still change to express the
innovations of contemporary notation.  Hence, it encompasses some 500
years of music. Its applications range from monophonic melodies to
monstrous counterpoint for large orchestras.

How can we get a grip on such a many-headed beast, and force it into
the confines of a computer program?  Our solution is to make a strict
distinction between notation, @emph{what} symbols to use, and
engraving, @emph{where} to put them.  Anything related to the second
question is considered ``engraving'' (i.e. typography).

For tackling the first problem, notation, we have broken up the
problem into digestible (and programmable) chunks: every type of
symbol is handled by a separate program module, a so-called plug-in.
Each plug-in is completely modular and independent, so each can be
developed and improved separately.  When put together, the plug-ins
can solve the music notation problem in cooperation.  People that translate
musical ideas to graphic symbols are called copyists or engravers, so by
analogy, each plug-in is also called @code{engraver}.

In the following example, we see how we start out with a note head
engraver.

@lilypond[]
\include "engraver-example.lyinc"

\score { \topVoice
\paper {
       \translator { \VoiceContext
       \remove "Stem_engraver"
       \remove "Phrasing_slur_engraver"
       \remove "Slur_engraver"
       \remove "Script_engraver"
       \remove "Beam_engraver"
       \remove "Auto_beam_engraver"
       
       }
       \translator { \StaffContext
       \remove "Accidental_engraver"
       \remove "Key_engraver"
       \remove "Clef_engraver"
       \remove "Bar_engraver"
       \remove "Time_signature_engraver"
       \remove "Staff_symbol_engraver"
       \consists "Pitch_squash_engraver"
        }
       
} 
}
@end lilypond

Then a @code{Staff_symbol_engraver} adds the staff:

@lilypond[]
\include "engraver-example.lyinc"

\score { \topVoice
\paper {
       \translator { \VoiceContext
       \remove "Stem_engraver"
       \remove "Phrasing_slur_engraver"
       \remove "Slur_engraver"
       \remove "Script_engraver"
       \remove "Beam_engraver"
       \remove "Auto_beam_engraver"
       
       }
       \translator { \StaffContext
       \remove "Accidental_engraver"
       \remove "Key_engraver"
       \remove "Clef_engraver"
       \remove "Bar_engraver"
       \consists "Pitch_squash_engraver"
       \remove "Time_signature_engraver"
        }
       
} 
}
@end lilypond

 The @code{Clef_engraver} defines a reference point for the staff:

@lilypond[]
\include "engraver-example.lyinc"

\score { \topVoice
\paper {
       \translator { \VoiceContext
       \remove "Stem_engraver"
       \remove "Phrasing_slur_engraver"
       \remove "Slur_engraver"
       \remove "Script_engraver"
       \remove "Beam_engraver"
       \remove "Auto_beam_engraver"
       }
       \translator { \StaffContext
       \remove "Accidental_engraver"
       \remove "Key_engraver"
       \remove "Bar_engraver"
       \remove "Time_signature_engraver"
        }
       
} 
}
@end lilypond

And the @code{Stem_engraver} adds stems:

@lilypond[]
\include "engraver-example.lyinc"

\score { \topVoice
\paper {
       \translator { \VoiceContext
       \remove "Phrasing_slur_engraver"
       \remove "Slur_engraver"
       \remove "Script_engraver"
       \remove "Beam_engraver"
       \remove "Auto_beam_engraver"
       }
       \translator { \StaffContext
       \remove "Accidental_engraver"
       \remove "Key_engraver"
       \remove "Bar_engraver"
       \remove "Time_signature_engraver"
        }
} 
}
@end lilypond

The @code{Stem_engraver} is notified of any note head coming along.
Every time one (or more, for a chord) note head(s) is seen, a stem
object is created, and attached to the note head.

By adding engravers for beams, slurs, accents, accidentals, bar lines,
time signature, and key signature, we get a complete piece of
notation.

@lilypond[]
\include "engraver-example.lyinc"

\score { \topVoice }
@end lilypond



This system works well for monophonic music, but what about
polyphony? In polyphonic notation, many voices can share a staff.

@lilypond[]
\include "engraver-example.lyinc"
\score { \context Staff << \topVoice \\ \botVoice >> }
@end lilypond

In this situation, the accidentals and staff are shared, but the
stems, slurs, beams, etc. are private to each voice. Hence, engravers
should be grouped. The engravers for note heads, stems, slurs, etc. go
into a group called ``Voice context,'' while the engravers for key,
accidental, bar, etc. go into a group called ``Staff context.'' In the
case of polyphony, a single Staff context contains more than one Voice
context.  In polyphonic notation, many voices can share a staff:
Similarly, more Staff contexts can be put into a single Score context.

@lilypond[]
\include "engraver-example.lyinc"
\score {
<< \new Staff << \topVoice \\ \botVoice >>
\new Staff << \pah \\ \hoom >>
  >>
}
@end lilypond



@node Engraving in LilyPond
@section Engraving in LilyPond


The term music engraving derives from the traditional process of music
printing.  Only a few decades ago, sheet music was made by cutting and
stamping the music into zinc or pewter plates, in mirror image. The plate
would be inked, and the depressions caused by the cutting and stamping
would hold ink.  An image was formed by pressing paper to the
plate. The stamping and cutting was done completely by hand. Making
corrections was cumbersome, so engraving had to be done correctly in
one go. Of course, this was a highly specialized skill, much more so
than the traditional process of printing books.
@cindex craftsmanship
@cindex master
In the traditional German system of craftsmanship six years of full-time
training, more than any other craft, were required before a student
could call himself a master of the art. After that many more years of
practical experience were needed to become an established music
engraver.  Even today, with the use of high-speed computers and
advanced software, music requires lots of manual fine tuning before it
is acceptable for publication.


Sheet music is performance material: everything is done to aid the
musician in letting him perform better.  Music is often far away from
its reader---it might be on a music stand. To make it clearly
readable, traditionally printed sheet music always uses bold symbols,
on heavy staff lines, and is printed on large sheets of paper.  This
``strong'' look is also present in the horizontal spacing.  To
minimize the number of page breaks, (hand-engraved) sheet music is
spaced very tightly. Yet, by a careful distribution of white space,
the feeling of balance is retained, and a clutter of symbols is
avoided.


We have used these observations in designing LilyPond.  The images
below show the flat symbol. On the left, a scan from a Henle edition,
which was made by a computer, and in the center is the flat from a
hand engraved B@"{a}renreiter edition of the same music. The left scan
illustrates typical flaws of computer print: the symbol is much
lighter, the staff lines are thinner, and the glyph has a straight
layout with sharp corners. By contrast, the B@"{a}renreiter has a bold
and almost voluptuous rounded look.  Our flat symbol is designed
after, among others, this one.  It is tuned it to harmonize with the
thickness of our staff lines, which are also much thicker than Henle's
lines.

@multitable @columnfractions  .1 .3 .3 .3
@item @tab
@iftex
@image{henle-flat-bw,4cm}
@end iftex
@html
<img src=henle-flat-bw.png>
@end html

@tab
@iftex
@image{baer-flat-bw,4cm}
@end iftex
@html
<img  src=baer-flat-bw.png>
@end html

@tab
@iftex
@image{lily-flat-bw,4cm}
@end iftex
@html
<img src=lily-flat-bw.png>
@end html

@item @tab
Henle (2000)
@tab
B@"{a}renreiter (1950)
@tab
LilyPond Feta font (2003)

@end multitable


@cindex musical symbols
@cindex font
@cindex blackness
@cindex balance

In spacing, the distribution of space should reflect the durations
between notes.  However, adhering with mathematical precision to the
duration will lead to a poor result. Shown here is an example of a
motive, printed twice. It is printed using exact mathematical spacing,
and with some corrections. Can you spot which fragment is which?

@cindex optical spacing
@lilypond[noindent]
    \score { \notes {
      \override Staff.NoteSpacing   #'stem-spacing-correction
     = #0.6
      c'4 e''4 e'4 b'4 |
       \stemDown b'4 e''4 a'4 e''4| \stemBoth
       \bar "||"
      \override Staff.NoteSpacing   #'stem-spacing-correction
   = #0.0
      \override Staff.StaffSpacing   #'stem-spacing-correction
   = #0.0
      c'4 e''4 e'4 b'4 |
      \stemDown b'4 e''4 a'4 e''4|
    }
    \paper { raggedright = ##t } }
@end lilypond

@cindex regular rhythms
@cindex regular spacing

The fragment only uses quarter notes: notes that are played in a
constant rhythm. The spacing should reflect that. Unfortunately, the
eye deceives us a little: not only does it notice the distance between
note heads, it also takes into account the distance between
consecutive stems. As a result, the notes of an up-stem/down-stem
combination should be put farther apart, and the notes of a down-up
combination should be put closer together, all depending on the
combined vertical positions of the notes. The first two measures are
printed with this correction, the last two measures without. The notes
in the last two measures form down-stem/up-stem clumps of notes.

@node Typography and program architecture
@section Typography and program architecture

Producing good engraving requires skill and knowledge.  As the
previous examples show, there is a lot of subtlety involved in music
engraving, and unfortunately, only a small fraction of these details
are documented.  Master engravers must learn all these details from
experience or from other engravers, which is why it takes so long to
become a master.  As an engraver gets older and wiser, he will be able
to produce better and more complex pieces.  A similar situation is
present when putting typographical knowledge into a computer program.
It is not possible to come up with a definitive solution for a problem
on the first try. Instead, we start out with a simple solution that
might cover 75% of the cases, and gradually refine that solution over
the course of months or years, so that 90 or 95 % of the cases are
handled.

This has an important implication for the design of the program: at
any time, almost every piece of formatting code must be considered as
temporary. When the need arises, it is to be replaced a solution that
will cover even more cases.  A ``plug-in'' architecture is a clean
way to accomplish this. This is an architecture where new pieces of
code can be inserted in the program dynamically.  In such a program, a
new solution can be developed along-side the existing code. For
testing, it is plugged in, but for production use, the old solution is
used. The new module can be perfected separately until it is better
than the existing solution, at which point it replaces the old one.

Until that time, users must have a way to deal with imperfections:
these 25%, 10% or 5% of the cases that are not handled
automatically. In these cases, a user must be able to override
formatting decisions. To accomplish this we store decisions in generic
variables, and let the user manipulate those.  For example, consider
the following fragment of notation:

@lilypond
\score { \notes \relative c'' {
\stemUp
    a4_\f f,8
        }
\paper { raggedright = ##t }
     }
@end lilypond

@noindent
The position of the forte symbol is slightly awkward, because it is
next to the low note, whereas dynamics should be below notes in
general. This may be remedied by inserting extra space between the
high note and the `f', as shown in this example:

@lilypond
\score { \notes \relative c'' {
\stemUp
    \once\override DynamicLineSpanner    #'padding = #4.0 
    a4_\f f,8
        }
\paper { raggedright = ##t }
     }
@end lilypond

This was achieved with the following input statement:
@example
   \once \override DynamicLineSpanner    #'padding = #4.0 
@end example
It increases the amount of space (@code{padding}) between the note and
the dynamic symbol to 4.0 (which is measured in staff spaces, so 4.0
equals the height of a staff). The keyword @code{\once} indicates that
this is a tweak: it is only done one time.

Both design aspects, a plug-in architecture, and formatting variables,
are built on top of GUILE, an interpreter for the programming language
Scheme, which is a member of the LISP family. Variables are stored as
Scheme objects, and attached to graphical objects such as note heads
and stems. The variables are a means to adjust formatting details in
individual cases, but they are used in a more general manner.

Consider the case of a publisher that is not satisfied with the
default layout, and wants heavier stems. Normally, they are @code{1.3}
times the thickness of staff lines, but suppose that their editions
require them to be twice the thickness of the staff lines. The same
mechanism can be used to adjust a setting globally. By issuing the
following command, the entire piece is now formatted with thicker
stems:
@example
    \override Score.Stem   #'thickness = #3.0 
@end example

@lilypond
\score { \notes \relative c'' {
    \override Score.Stem   #'thickness = #3.0 
    \once\override DynamicLineSpanner    #'padding = #4.0 
\stemUp
    a4_\f f,8
        }
\paper { raggedright = ##t }
     }
@end lilypond

@noindent
In effect, by setting these variables, users can define their own
layout styles.

``Plug-ins'' are also implemented using Scheme.  A formatting
``plug-in'' takes the form of a function written in Scheme (or a C++
function made available as a Scheme function), and it is also stored
in a variable.  For example, the placement of the forte symbol in the
example above is calculated by the function
@code{Side_position_interface::aligned_side}.  If we want to replace
this function by a more advanced one, we could issue
@example
    \override DynamicLineSpanner   #'Y-offset-callbacks
    = #(list gee-whiz-gadget)
@end example

@noindent
Now, the formatting process will trigger a call to our new
@code{gee-whiz-gadget} function when the position of the f symbol has
to be determined.

The full scope of this functionality certainly is intimidating, but
there is no need to fear: normally, it is not necessary to define
style-sheets or rewrite formatting functions. In fact, LilyPond gets a
lot of formatting right automatically, so adjusting individual layout
situations is often not needed at all.


@node Music representation
@section Music representation


Our premise is that LilyPond is a system that does music formatting
completely automatically. Under this assumption, the output does not
have to be touched up. Consequently, an interactive display of the
output, where it is possible to reposition notation elements, is
superfluous.  This implies that the program should be a batch program:
the input is entered in a file, which then is @emph{compiled}, i.e.
put through the program.  The final output is produced as a file ready
to view or print. The compiler fills in all the details of the
notation, those details should be left out of the input file. In other
words, the input should mirror the content as closely as possible. In
the case of music notation the content is the music itself, so that is
what the input should consist of.

On paper this theory sounds very good. In practice, it opens a can of
worms. What really @emph{is} music? Many philosophical treatises must
have been written on the subject.  Instead of losing ourselves in
philosophical arguments over the essence of music, we have reversed
the question to yield a more practical approach. Our assumption is
that the printed score contains all of the music of piece. We build a
program that uses some input format to produce such a score. Over the
course of time, the program evolves. While this happens, we can remove
more and more elements of the input format: as the program improves,
it can fill in irrelevant details of the input by itself. At some
(hypothetical) point, the program is finished: there is no possibility
to remove any more elements from the syntax.  What we have left is by
definition exactly the musical meaning of the score.

There are also more practical concerns.  Our users have to key in the
music into the file directly, so the input format should have a
friendly syntax: a quarter note C is entered as @code{c4}, the code
@code{r8.}  signifies a dotted eighth rest.

Notes and rests form the simplest musical expressions in the input
syntax.  More complex constructs are produced by combining them into
compound structures. This is done in much the same way that complex
mathematical formulas are built from simple expressions such as
numbers and operators.

In effect, the input format is a language, and the rules of that
language can be specified succinctly with a so-called context-free
grammar.  The grammar formally specificies what types of input form
valid `sentences'.  Reading such languages, and splitting them into
grammatical structures is a problem with standard solutions.
Moreover, rigid definitions make the format easier to understand: a
concise formal definition permits a simple informal description.

The user-interface of LilyPond is its syntax.  That part is what users
see most.  As a results, some users think that music representation is
a very important or interesting problem. In reality, less than 10% of
the source code of the program handles reading and representing the
input, and they form the easy bits of the program.  In our opinion,
producing music notation, and formatting it prettily are much more
interesting and important than music representation: solving
these problems takes up most of the bulk of the code, and they are the
most difficult things to get right.

@node Example applications
@section Example applications

We have written LilyPond as an experiment of how to condense the art
of music engraving into a computer program. Thanks to all that hard
work, the program can now be used to perform useful tasks.  The
simplest application is printing notes:

@lilypond[relative=1]
  \time 2/4 c4 c g'4 g a4 a g2  
@end lilypond

By adding chord names and lyrics we obtain a lead sheet:

@lilypond[raggedright]
\score { <<
  \context ChordNames \chords  { c2 c f2 c }
  \notes \relative c' { \time 2/4 c4 c g'4 g a4 a g2 }
  \context Lyrics \lyrics  { twin4 kle twin kle lit tle star2 } >> }
@end lilypond

Polyphonic notation and piano music can also be printed. The following
example combines some more exotic constructs:

@lilypondfile{screech-boink.ly}

The fragments shown above have all been written by hand, but that is
not a requirement. Since the formatting engine is mostly automatic, it
can serve as an output means for other programs that manipulate
music. For example, it can also be used to convert databases of
musical fragments to images for use on websites and multimedia
presentations.

This manual also shows an application: the input format is plain text,
and can therefore be easily embedded in other text-based formats, such
as La@TeX{}, HTML or in the case of this manual, Texinfo.  By means of a
special program, the input fragments can be replaced by music images in
the resulting PostScript or HTML output files. This makes it easy to
mix music and text in documents.



@node About this manual
@section About this manual

The manual is divided into the following chapters:
@itemize @bullet
@item
@ifhtml The 
@end ifhtml
@emph{@ref{Tutorial}}
gives a  gentle introduction to typesetting music.
First time users should start here. 
@item
@ifhtml
The
@end ifhtml
@emph{@ref{Notation manual}}
discusses topics grouped by notation construct. Once you master the
basics, this is the place to look up details.
@item
@ifhtml
The
@end ifhtml
@emph{@ref{Literature list}}
 contains a set of useful reference books, for those who wish to know
 more  on notation and engraving. 
@item
@ifhtml
 The
 @end ifhtml
@emph{@ref{Technical manual}}
@c
discusses the general design of the program, and how to extend its
functionality.
@item
@ifhtml
The chapter
@end ifhtml
@emph{@ref{Invoking LilyPond}}  explains how to run LilyPond and its helper
programs.

@item
@ifhtml
The 
@end ifhtml
@emph{@ref{lilypond-book manual}}
explains  the details behind creating documents with in-line music
examples (like this manual).


@item
@ifhtml
The chapter 
@end ifhtml
@emph{@ref{Converting from other formats}}
explains how to run the conversion programs. These programs
are supplied with the LilyPond package, and convert a variety of music
formats to the @code{.ly}  format. In addition, this section explains
how to upgrade input files from previous versions of LilyPond.

@end itemize

Once you are an experienced user, you can use the manual as reference:
there is an extensive index@footnote{If you are looking for something,
and you cannot find it in the manual, that is considered a bug.  In
that case, please file a bug report.}, but the document is also
available in
@ifnothtml
a big HTML page,
@end ifnothtml 
@ifhtml
@uref{../lilypond.html, a big HTML page}
@end ifhtml
which can be searched easily using the search facility of a web
browser.
@cindex search in manual
@cindex using the manual


If you are not familiar with music notation or music terminology
(especially if you are a non-native English speaker), then it is
advisable to consult the glossary as well. The glossary explains
musical terms, and includes translations to various languages. It is a
@ifhtml
@uref{../music-glossary.html,separate document}.
@end ifhtml
@ifnothtml
separate document, available in HTML and PDF.
@end ifnothtml
@cindex idiom
@cindex jargon
@cindex terminology
@cindex foreign languages
@cindex language


This manual is not complete without a number of other documents. They
are not available in print, but should be included with the
documentation package for your platform:

@itemize @bullet
@item
Program reference
@ifhtml
(available @uref{../lilypond-internals/lilypond-internals.html,here})
@end ifhtml

The program reference is a set of heavily crosslinked HTML pages,
which documents the nit-gritty details of each and every LilyPond
class, object and function.  It is produced directly from the
formatting definitions used.

Almost all formatting functionality that is used internally, is
available directly to the user. For example, all variables that
control thicknesses, distances, etc, can be changed in input
files. There are a huge number of formatting options, and all of them
are described in the generated documentation.  Each section of the
notation manual has a @b{See also} subsection, which refers to the
the generated documentation.  In the HTML document, these subsections
have clickable links.

@item
  Templates
@ifhtml
(available @uref{../../../input/template/out-www/collated-files.html,here})
@end ifhtml

After you have gone through the tutorial, you should be able to write
input files. In practice, writing files from scratch turns out to be
intimidating.  To give you a headstart, we have collected a number of
often-used formats in example files.  These files can be used as a
start: simply copy the template, and add notes in the appropriate
places.

@item
  Various input examples
@ifhtml
(available @uref{../../../../input/test/out-www/collated-files.html,here})
@end ifhtml
@cindex snippets

These small files show various tips and tricks, and are available as a
big HTML document, with pictures and explanatory texts included.


@item
  The regression tests
@ifhtml
(available @uref{../../../input/regression/out-www/collated-files.html,here})
@end ifhtml

This collection of files tests each notation and engraving feature of
LilyPond in one file. The collection is primarily there to help us
debug problems, but it can be instructive to see how we excercise the
program. The format is like the tips and tricks document.

@end itemize


In all HTML documents that have music fragments embedded, the LilyPond
input that was used to produce that image can be viewed by clicking
the image.

The location of the documentation files that are mentioned here can
vary from system to system.  On occasion, this manual refers to
initialization and example files.  Throughout this manual, we refer to
input files relative to the top-directory of the source archive. For
example, @file{input/test/bla.ly} may refer to the file
@file{lilypond-1.7.19/input/test/bla.ly}.  On binary packages for the
Unix platform, the documentation and examples can typically be found
somewhere below @file{/usr/share/doc/lilypond/}. Initialization files,
for example @file{scm/lily.scm}, or @file{ly/engraver-init.ly}, are
usually found in the directory @file{/usr/share/lilypond/}.

@cindex adjusting output
@cindex variables
@cindex properties
@cindex lilypond-internals
@cindex internal documentation
@cindex Scheme
@cindex extending lilypond
@cindex bugreport
@cindex index

Finally, this and all other manuals, are available online both as PDF
files and HTML from the web site, which can be found at
@uref{http://www.lilypond.org/}.

@cindex website 
@cindex URL