* Documentation/user/tutorial.itely (Integrating text and music):

[lilypond.git] / Documentation / user / introduction.itely

@c -*-texinfo-*-
@c This file is part of lilypond.tely

@node Introduction
@chapter Introduction

LilyPond is a system for formatting music prettily.  This chapter
discusses the backgrounds of LilyPond.  It explains the problem of
printing music with computers, and our approach to solving those
problems.


@menu
* Engraving::
* Automated engraving::
* What symbols to engrave?::
* Music representation::
* Example applications::
* About this manual::
@end menu


@node Engraving
@section Engraving

The art of music typography is called @emph{(plate) engraving}.  The
term derives from the traditional process of music printing.  Just a
few decades ago, sheet music was made by cutting and stamping the
music into a zinc or pewter plate in mirror image.  The plate would be
inked, the depressions caused by the cutting and stamping would hold
ink.  An image was formed by pressing paper to the plate.  The
stamping and cutting was completely done by hand.  Making a correction
was cumbersome, if possible at all, so the engraving had to be perfect
in one go.  Engraving was a highly specialized skill, a craftsman had
to complete around five years of training before he could
be a master engraver, and another five years of experience were
necessary to become truly skilled.

Nowadays, all newly printed music is produced with computers.  This
has obvious advantages; prints are cheaper to make, editorial work can
be delivered by email.  Unfortunately, the pervasive use of computers
has also decreased the graphical quality of scores.  Computer
printouts have a bland, mechanical look, which makes them unpleasant
to play from.


@c introduce illustrating aspects of engraving, font...
The images below illustrate the difference between traditional
engraving and typical computer output, and the third picture shows how
LilyPond mimics the traditional look.  The left picture shows a scan
of a flat symbol from a Henle edition published in 2000.  The center
depicts a symbol from a hand-engraved B@"{a}renreiter edition of the
same music.  The left scan illustrates typical flaws of computer
print: the staff lines are thin, the weight of the flat symbol matches
the light lines and it has a straight layout with sharp corners.  By
contrast, the B@"{a}renreiter flat has a bold, almost voluptuous
rounded look.  Our flat symbol is designed after, among others, this
one.  It is rounded, and its weight harmonizes with the thickness of
our staff lines, which are also much thicker than Henle's lines.

@multitable @columnfractions  .05 .3 .3 .3 .05
@item @tab
@ifnotinfo
@iftex
@image{henle-flat-bw,4cm}
@end iftex
@ifnottex
@image{henle-flat-bw,,,png}
@end ifnottex

@tab
@iftex
@image{baer-flat-bw,4cm}
@end iftex
@ifnottex
@image{baer-flat-bw,,,png}
@end ifnottex

@tab
@iftex
@image{lily-flat-bw,4cm}
@end iftex
@ifnottex
@image{lily-flat-bw,,,png}
@end ifnottex
@end ifnotinfo
@ifinfo
@c workaround for makeinfo-4.6: line breaks and multi-column cookies
@image{henle-flat-bw,,,png} @image{baer-flat-bw,,,png} @image{lily-flat-bw,,,png}
@end ifinfo

@item @tab
Henle (2000)
@tab
B@"{a}renreiter (1950)
@tab
LilyPond Feta font (2003)

@end multitable


@cindex musical symbols
@cindex font
@cindex blackness
@cindex balance

@c introduce illustrating aspects of engraving, spacing...
In spacing, the distribution of space should reflect the durations
between notes.  However, many modern scores adhere to the durations
with mathematical precision, which leads to poor results.  In the
next example a motive is printed twice.  It is printed once using
exact mathematical spacing, and once with corrections.  Can you
spot which fragment is which?

@cindex optical spacing
@lilypond[quote,noindent,fragment]
{
    \override Staff.NoteSpacing #'stem-spacing-correction = #0.6
    c'4 e''4 e'4 b'4 |
    \stemDown b'4 e''4 a'4 e''4 | \bar "||"
    \override Staff.NoteSpacing #'stem-spacing-correction = #0.0
    \override Staff.StaffSpacing #'stem-spacing-correction = #0.0
    \stemBoth c'4 e''4 e'4 b'4 |
    \stemDown b'4 e''4 a'4 e''4 |
}
@end lilypond

@cindex regular rhythms
@cindex regular spacing

The fragment only uses quarter notes: notes that are played in a
constant rhythm.  The spacing should reflect that.  Unfortunately, the
eye deceives us a little; not only does it notice the distance between
note heads, it also takes into account the distance between
consecutive stems.  As a result, the notes of an up-stem/down-stem
combination should be put farther apart, and the notes of a down-up
combination should be put closer together, all depending on the
combined vertical positions of the notes.  The first two measures are
printed with this correction, the last two measures without.  The notes
in the last two measures form down-stem/up-stem clumps of notes.

@cindex typography

Musicians are usually more absorbed with performing than with studying
the looks of piece of music; nitpicking about typographical details
may seem academical.  But it is not.  In larger pieces with monotonous
rhythms, spacing corrections lead to subtle variations in the layout
of every line, giving each one a distinct visual signature.  Without
this signature all lines would look the same, they become like a
labyrinth.  If the musician looks away once or has a lapse in his
concentration, he will be lost on the page.
@c he/she

Similarly, the strong visual look of bold symbols on heavy staff lines
stands out better when music is far away from reader, for example, if
it is on a music stand.  A careful distribution of white space allows
music to be set very tightly without cluttering symbols together.  The
result minimizes the number of page turns, which is a great advantage.

This is a common characteristic of typography.  Layout should be
pretty, not only for its own sake, but especially because it helps the
reader in his task.  For performance material like sheet music, this is
of double importance: musicians have a limited amount of attention.  The
less attention they need for reading, the more they can focus on
playing itself.  In other words, better typography translates to better
performances.

Hopefully, these examples also demonstrate that music typography is an
art that is subtle and complex, and to produce it requires
considerable expertise, which musicians usually do not have.  LilyPond
is our effort to bring the graphical excellence of hand-engraved music
to the computer age, and make it available to normal musicians.  We
have tuned our algorithms, font-designs, and program settings to
produce prints that match the quality of the old editions we love to
see and love to play from.




@node Automated engraving
@section Automated engraving

How do we go about implementing typography?  If craftsmen need over
ten years to become true masters, how could we simple hackers ever
write a program to take over their jobs?

The answer is: we cannot.  Typography relies on human judgment of
appearance, so people cannot be replaced ultimately.  However, much of
the dull work can be automated.  If LilyPond solves most of the common
situations correctly, this will be a huge improvement over existing
software.  The remaining cases can be tuned by hand.  Over the course
of years, the software can be refined to do more and more
automatically, so manual overrides are less and less necessary.

When we started we wrote the LilyPond program entirely in the C++
programming language; the program's functionality was set in stone by
the developers.  That proved to be unsatisfactory for a number of
reasons:

@itemize @bullet
@item When LilyPond makes mistakes,
users need to override formatting decisions.  Therefore, the user must
have access to the formatting engine.  Hence, rules and settings cannot
be fixed by us at compile time but must be accessible for users at
run-time.

@item Engraving is a matter of visual judgment, and therefore a matter of
taste.  As knowledgeable as we are, users can disagree with our
personal decisions.  Therefore, the definitions of typographical style
must also be accessible to the user.

@item Finally, we continually refine the formatting algorithms, so we
need a flexible approach to rules.  The C++ language forces a certain
method of grouping rules that do not match well with how music
notation works.
@end itemize

These problems have been addressed by integrating the GUILE
interpreter for the Scheme programming language and rewriting parts of
LilyPond in Scheme.  The new, flexible formatting is built around the
notion of graphical objects, described by Scheme variables and
functions.  This architecture encompasses formatting rules,
typographical style and individual formatting decisions.  The user has
direct access to most of these controls.

Scheme variables control layout decisions.  For example, many
graphical objects have a direction variable that encodes the choice
between up and down (or left and right).  Here you see two chords,
with accents and arpeggio.  In the first chord, the graphical objects
have all directions down (or left).  The second chord has all
directions up (right).

@lilypond[quote,raggedright,relative=1]
\new Score \with {
  \override SpacingSpanner #'spacing-increment = #3
  \override TimeSignature #'transparent = ##t
  } {
  \stemDown <e g b>4_>-\arpeggio
  \override Arpeggio #'direction = #RIGHT
  \stemUp <e g b>4^>-\arpeggio
}
@end lilypond

The process of formatting a score consists of reading and writing the
variables of graphical objects.

Some variables have a preset value.  For example, the thickness of many
lines---a characteristic of typographical style---are preset
variables.  Changing them gives a different typographical impression.

@lilypond[quote,raggedright]
fragment =  {
  \clef bass f8 as8
  c'4-~ c'16 as g f e16 g bes c' des'4
}

  <<
    \new Staff \fragment
    \new Staff \with {
      \override Beam #'thickness = #0.3
      \override Stem #'thickness = #0.5
      \override Bar #'thickness = #3.6
      \override Tie #'thickness = #2.2
      \override StaffSymbol #'thickness = #3.0
      \override Tie #'extra-offset = #'(0 .  0.3)
   } \fragment
   >>
@end lilypond

Formatting rules are also preset variables: each object has variables
containing procedures.  These procedures perform the actual formatting,
and by substituting different ones, we can change behavior.  In the
following example, the rule which note head objects use to produce
their symbol is changed during the music fragment.

@lilypond[quote,raggedright]
#(define (mc-squared grob orig current)
  (let ((interfaces (ly:grob-property grob 'interfaces))
        (pos (ly:grob-property grob 'staff-position)))
    (if (and (memq 'note-head-interface interfaces)
             (memq pos '(-2 -3 -5)))
        (begin
          (ly:grob-set-property! grob 'print-function brew-new-markup-stencil)
          (ly:grob-set-property! grob 'font-family 'roman)
          (ly:grob-set-property!
           grob 'text
           (make-raise-markup
            -0.5
            (case pos
              ((-5) (make-simple-markup "m"))
              ((-3) (make-simple-markup "c "))
              ((-2) (make-smaller-markup (make-bold-markup "2")))
              (else (make-simple-markup "bla")))))))))

 \context Voice \relative  c' {
    \stemUp
    \set autoBeaming = ##f
    \time 2/4
    <d f g>4
    \once \override NoteHead #'print-function = #Note_head::brew_ez_stencil
    <d f g>
    \once \override NoteHead #'style = #'cross
    <d f g>
    \applyoutput #mc-squared
    <d f g>
    <<
      { d8[ es-( fis^^ g] fis2-) }
      \repeat unfold 5 { \applyoutput #mc-squared s8 }
    >>
  }
@end lilypond



@node What symbols to engrave?
@section What symbols to engrave?

@cindex engraving
@cindex typography

The formatting process in LilyPond decides where to place
symbols.  However, this can only be done once it is decided @emph{what}
symbols should be printed, in other words what notation to use.

Common music notation is a system of recording music that has evolved
over the past 1000 years.  The form that is now in common use, dates
from the early renaissance.  Although the basic form (i.e., note heads on a
5-line staff) has not changed, the details still change to express the
innovations of contemporary notation.  Hence, it encompasses some 500
years of music.  Its applications range from monophonic melodies to
monstrous counterpoint for large orchestras.

How can we get a grip on such a many-headed beast, and force it into
the confines of a computer program?  We have broken up the problem of
notation (as opposed to engraving, i.e., typography) into digestible
and programmable chunks: every type of symbol is handled by a separate
module, a so-called plug-in.  Each plug-in is completely modular and
independent, so each can be developed and improved separately.  People
who translate musical ideas to graphic symbols are called copyists or
engravers, so by analogy, each plug-in is called @code{engraver}.

In the following example, we see how we start out with a plug-in for
note heads, the @code{Note_heads_engraver}.

@lilypond[quote,raggedright]
\include "engraver-example.lyinc"

\score {
  \topVoice
  \paper {
    \context {
      \Voice
      \remove "Stem_engraver"
      \remove "Phrasing_slur_engraver"
      \remove "Slur_engraver"
      \remove "Script_engraver"
      \remove "Beam_engraver"
      \remove "Auto_beam_engraver"
    }
    \context {
      \Staff
      \remove "Accidental_engraver"
      \remove "Key_engraver"
      \remove "Clef_engraver"
      \remove "Bar_engraver"
      \remove "Time_signature_engraver"
      \remove "Staff_symbol_engraver"
      \consists "Pitch_squash_engraver"
    }
  }
}
@end lilypond

@noindent
Then a @code{Staff_symbol_engraver} adds the staff

@lilypond[quote,raggedright]
\include "engraver-example.lyinc"

\score {
  \topVoice
  \paper {
    \context {
      \Voice
      \remove "Stem_engraver"
      \remove "Phrasing_slur_engraver"
      \remove "Slur_engraver"
      \remove "Script_engraver"
      \remove "Beam_engraver"
      \remove "Auto_beam_engraver"
    }
    \context {
      \Staff
      \remove "Accidental_engraver"
      \remove "Key_engraver"
      \remove "Clef_engraver"
      \remove "Bar_engraver"
      \consists "Pitch_squash_engraver"
      \remove "Time_signature_engraver"
    }
  }
}
@end lilypond

@noindent
the @code{Clef_engraver} defines a reference point for the staff

@lilypond[quote,raggedright]
\include "engraver-example.lyinc"

\score {
  \topVoice
  \paper {
    \context {
      \Voice
      \remove "Stem_engraver"
      \remove "Phrasing_slur_engraver"
      \remove "Slur_engraver"
      \remove "Script_engraver"
      \remove "Beam_engraver"
      \remove "Auto_beam_engraver"
    }
    \context {
      \Staff
      \remove "Accidental_engraver"
      \remove "Key_engraver"
      \remove "Bar_engraver"
      \remove "Time_signature_engraver"
    }
  }
}
@end lilypond

@noindent
and the @code{Stem_engraver} adds stems.

@lilypond[quote,raggedright]
\include "engraver-example.lyinc"

\score {
  \topVoice
  \paper {
    \context {
      \Voice
      \remove "Phrasing_slur_engraver"
      \remove "Slur_engraver"
      \remove "Script_engraver"
      \remove "Beam_engraver"
      \remove "Auto_beam_engraver"
    }
    \context {
      \Staff
      \remove "Accidental_engraver"
      \remove "Key_engraver"
      \remove "Bar_engraver"
      \remove "Time_signature_engraver"
    }
  }
}
@end lilypond

The @code{Stem_engraver} is notified of any note head coming along.
Every time one (or more, for a chord) note head is seen, a stem
object is created and connected to the note head.
By adding engravers for beams, slurs, accents, accidentals, bar lines,
time signature, and key signature, we get a complete piece of
notation.

@lilypond[quote,raggedright]
\include "engraver-example.lyinc"
\score { \topVoice }
@end lilypond

This system works well for monophonic music, but what about
polyphony?  In polyphonic notation, many voices can share a staff.

@lilypond[quote,raggedright]
\include "engraver-example.lyinc"
\context Staff << \topVoice \\ \botVoice >> 
@end lilypond

In this situation, the accidentals and staff are shared, but the
stems, slurs, beams, etc., are private to each voice.  Hence, engravers
should be grouped.  The engravers for note heads, stems, slurs, etc., go
into a group called `Voice context,' while the engravers for key,
accidental, bar, etc., go into a group called `Staff context.'  In the
case of polyphony, a single Staff context contains more than one Voice
context.  In polyphonic notation, many voices can share a staff.
Similarly, more Staff contexts can be put into a single Score context.

@lilypond[quote,raggedright]
\include "engraver-example.lyinc"
\score {
  <<
    \new Staff << \topVoice \\ \botVoice >>
    \new Staff << \pah \\ \hoom >>
  >>
}
@end lilypond

@node Music representation
@section Music representation

Ideally, the input format for any high-level formatting system is an
abstract description of the content.  In this case, that would be the
music itself.  This poses a formidable problem: how can we define what
music really is? Instead of trying to find an answer, we have reversed
the question.  We write a program capable of producing sheet music,
and adjust the format to be as lean as possible.  When the format can
no longer be trimmed down, by definition we are left with content
itself.  Our program serves as a formal definition of a music
document.

The syntax is also the user-interface for LilyPond, hence it is easy
to type

@example
c'4 d'8
@end example

@noindent
a quarter note C1 (middle C) and an eighth note D1 (D above middle C)

@lilypond[quote,fragment]
c'4 d'8
@end lilypond

On a microscopic scale, such syntax is easy to use.  On a larger
scale, syntax also needs structure.  How else can you enter complex
pieces like symphonies and operas?  The structure is formed by the
concept of music expressions: by combining small fragments of music
into larger ones, more complex music can be expressed.  For example

@lilypond[quote,verbatim,fragment,relative=1]
c4
@end lilypond

@noindent
Chords can be constructed with < and > enclosing the notes

@example
<c d e>4
@end example

@lilypond[quote,fragment,relative=1]
\new Voice { <c d e>4 }
@end lilypond

@noindent
This expression is put in sequence by enclosing it in curly braces
@code{@{@tie{}@dots{}@tie{}@}}

@example
@{ <c d e>4 f4 @}
@end example

@lilypond[quote,relative=1]
\new Voice { <c d e>4 f4 }
@end lilypond

@noindent       
The above is an expression also, and thus it may be combined again with
another simultaneous expression (a half note) using <<, @code{\\}, and >>

@example
<< g2 \\ @{ <c d e>4 f4 @} >>
@end example

@lilypond[quote,fragment,relative=2]
\new Voice { << g2 \\ { <c d e>4 f4 } >> }
@end lilypond
        
Such recursive structures can be specified neatly and formally in a
context-free grammar.  The parsing code is also generated from this
grammar.  In other words, the syntax of LilyPond is clearly and
unambiguously defined.

User-interfaces and syntax are what people see and deal with
most.  They are partly a matter of taste, and also subject of much
discussion.  Although discussions on taste do have their merit, they
are not very productive.  In the larger picture of LilyPond, the
importance of input syntax is small: inventing neat syntax is easy,
writing decent formatting code is much harder.  This is also
illustrated by the line-counts for the respective components: parsing
and representation take up less than 10% of the code.


@node Example applications
@section Example applications

We have written LilyPond as an experiment of how to condense the art
of music engraving into a computer program.  Thanks to all that hard
work, the program can now be used to perform useful tasks.  The
simplest application is printing notes.

@lilypond[quote,relative=1,fragment]
\time 2/4 c4 c g'4 g a4 a g2
@end lilypond

@noindent
By adding chord names and lyrics we obtain a lead sheet.

@lilypond[quote,raggedright]
  <<
    \context ChordNames \chords  { c2 c f2 c }
    \new Staff \relative c' { \time 2/4 c4 c g'4 g a4 a g2 }
    \context Lyrics \lyrics  { twin4 kle twin kle lit tle star2 }
  >>
@end lilypond

Polyphonic notation and piano music can also be printed.  The following
example combines some more exotic constructs.

@lilypondfile[quote,raggedright]{screech-boink.ly}

The fragments shown above have all been written by hand, but that is
not a requirement.  Since the formatting engine is mostly automatic, it
can serve as an output means for other programs that manipulate
music.  For example, it can also be used to convert databases of
musical fragments to images for use on websites and multimedia
presentations.

This manual also shows an application: the input format is text, and
can therefore be easily embedded in other text-based formats such as
La@TeX{}, HTML, or in the case of this manual, Texinfo.  By means of a
special program, the input fragments can be replaced by music images
in the resulting PostScript or HTML output files.  This makes it easy
to mix music and text in documents.



@node About this manual
@section About this manual

The manual is divided into the following chapters:
@itemize @bullet

@item
@ifhtml
The
@end ifhtml
@emph{@ref{Tutorial}}
gives a gentle introduction to typesetting music.
First time users should start here.

@item
@ifhtml
The
@end ifhtml
@emph{@ref{Notation manual}}
discusses topics grouped by notation construct.  Once you master the
basics, this is the place to look up details.

@item
@ifhtml
The
@end ifhtml
@emph{@ref{Changing defaults}}
explains how to fine tune layout.

@item
@ifhtml
The chapter
@end ifhtml
@emph{@ref{Invoking LilyPond}} shows how to run LilyPond and its helper
programs.

@item
@ifhtml
The
@end ifhtml
@emph{@ref{lilypond-book manual}}
explains  the details behind creating documents with in-line music
examples (like this manual).

@item
@ifhtml
The chapter
@end ifhtml
@emph{@ref{Converting from other formats}}
explains how to run the conversion programs.  These programs
are supplied with the LilyPond package, and convert a variety of music
formats to the @code{.ly}  format.  In addition, this section explains
how to upgrade input files from previous versions of LilyPond.

@item
@ifhtml
The
@end ifhtml
@emph{@ref{Literature list}}
contains a set of useful reference books, for those who wish to know
more  on notation and engraving.
@end itemize

Once you are an experienced user, you can use the manual as reference:
there is an extensive index@footnote{If you are looking for something,
and you cannot find it in the manual, that is considered a bug.  In
that case, please file a bug report.}, but the document is also
available in
@ifnothtml
a big HTML page,
@end ifnothtml
@ifhtml
@uref{../lilypond.html, a big HTML page}
@end ifhtml
which can be searched easily using the search facility of a web
browser.
@cindex search in manual
@cindex using the manual

@c FIXME:
@c add/integrate glossary, put in list above
If you are not familiar with music notation or music terminology
(especially if you are a non-native English speaker), it is advisable
to consult the glossary as well.  The glossary explains musical terms,
and includes translations to various languages.  It is a
@ifhtml
@uref{../music-glossary.html,separate document}.
@end ifhtml
@ifnothtml
separate document, available in HTML and PDF.
@end ifnothtml
@cindex idiom
@cindex jargon
@cindex terminology
@cindex foreign languages
@cindex language


This manual is not complete without a number of other documents.  They
are not available in print, but should be included with the
documentation package for your platform:

@itemize @bullet
@item
Program reference
@ifhtml
(available @uref{../lilypond-internals/lilypond-internals.html,here})
@end ifhtml

The program reference is a set of heavily cross linked HTML pages,
which document the nit-gritty details of each and every LilyPond
class, object, and function.  It is produced directly from the
formatting definitions used.

Almost all formatting functionality that is used internally, is
available directly to the user.  For example, all variables that
control thickness values, distances, etc., can be changed in input
files.  There are a huge number of formatting options, and all of them
are described in the generated documentation.  Each section of the
notation manual has a @b{See also} subsection, which refers to the
the generated documentation.  In the HTML document, these subsections
have clickable links.

@item
Templates
@ifhtml
(available @uref{../../../../input/template/out-www/collated-files.html,here})
@end ifhtml

After you have gone through the tutorial, you should be able to write
input files.  In practice, writing files from scratch turns out to be
intimidating.  To give you a head start, we have collected a number of
often-used formats in example files; simply copy the template and add
notes in the appropriate places.

@item
Various input examples
@ifhtml
(available @uref{../../../../input/test/out-www/collated-files.html,here})
@end ifhtml
@cindex snippets

This collection of files shows various tips and tricks, and is
available as a big HTML document, with pictures and explanatory texts
included.

@item
The regression tests
@ifhtml
(available @uref{../../../../input/regression/out-www/collated-files.html,here})
@end ifhtml

This collection of files tests each notation and engraving feature of
LilyPond in one file.  The collection is primarily there to help us
debug problems, but it can be instructive to see how we exercise the
program.  The format is similar to the the tips and tricks document.
@end itemize


In all HTML documents that have music fragments embedded, the LilyPond
input that was used to produce that image can be viewed by clicking
the image.

The location of the documentation files that are mentioned here can
vary from system to system.  On occasion, this manual refers to
initialization and example files.  Throughout this manual, we refer to
input files relative to the top-directory of the source archive.  For
example, @file{input/test/bla.ly} may refer to the file
@file{lilypond-1.7.19/input/test/bla.ly}.  On binary packages for the
Unix platform, the documentation and examples can typically be found
somewhere below @file{/usr/share/doc/lilypond/}.  Initialization files,
for example @file{scm/lily.scm}, or @file{ly/engraver-init.ly}, are
usually found in the directory @file{/usr/share/lilypond/}.

@cindex adjusting output
@cindex variables
@cindex properties
@cindex lilypond-internals
@cindex internal documentation
@cindex Scheme
@cindex extending lilypond
@cindex bug report
@cindex index

Finally, this and all other manuals, are available online both as PDF
files and HTML from the web site, which can be found at
@uref{http://www.lilypond.org/}.

@cindex website
@cindex URL